2024-09-16T00:06:33.387 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-16T00:06:33.410 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-16T00:06:33.508 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906384 branch: squid description: fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7906384' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi meta: - desc: 'setup ceph/quincy ' name: teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi no_nested_subset: false os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - OSD_DOWN - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy sha1: 8921c8ef947807b717db041d85f0761b7914d5fc ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 8921c8ef947807b717db041d85f0761b7914d5fc kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: ad35db06e8924912f01e32b35576297c0f1352b4 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - host.a - client.0 - osd.0 - osd.1 - osd.2 - - host.b - client.1 - osd.3 - osd.4 - osd.5 seed: 3393 sha1: 8921c8ef947807b717db041d85f0761b7914d5fc sleep_before_teardown: 0 subset: 294/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: ad35db06e8924912f01e32b35576297c0f1352b4 targets: smithi112.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBO3Pa+ga+wZsky7A/R0nqjiWJbVQ3084nfR3gRU9uRzGn2IAvipvo/w5Ceygv4oXFF8zXquZNmuiVmI9AToUIc4= smithi169.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBg5N7kH6OgEVkcTFgx9+TWJVea6TvkW49kY+Stg04v3n5sOyQsw3Wxg67i20nEbW44ann6gfx8vuLKkrJwpWFo= tasks: - install: branch: quincy exclude_packages: - ceph-volume - print: '**** done install task...' - cephadm: cephadm_branch: quincy cephadm_git_url: https://github.com/ceph/ceph conf: osd: osd_class_default_list: '*' osd_class_load_list: '*' image: quay.ceph.io/ceph-ci/ceph:quincy roleless: true - print: '**** done end installing quincy cephadm ...' - cephadm.shell: host.a: - ceph config set mgr mgr/cephadm/use_repo_digest true --force - print: '**** done cephadm.shell ceph config set mgr...' - cephadm.shell: host.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: host.a: - ceph fs volume create cephfs --placement=4 - ceph fs dump - cephadm.shell: host.a: - ceph fs set cephfs max_mds 2 - cephadm.shell: host.a: - ceph fs set cephfs allow_standby_replay true - cephadm.shell: host.a: - ceph fs set cephfs inline_data true --yes-i-really-really-mean-it - cephadm.shell: host.a: - ceph fs dump - ceph --format=json fs dump | jq -e ".filesystems | length == 1" - while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done - fs.pre_upgrade_save: null - ceph-fuse: null - print: '**** done client' - parallel: - upgrade-tasks - workload-tasks - cephadm.shell: host.a: - ceph fs dump - fs.post_upgrade_checks: null teuthology: fragments_dropped: [] meta: {} postmerge: [] variables: fail_fs: true teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-15_21:24:03 tube: smithi upgrade-tasks: sequential: - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done - ceph versions | jq -e '.mgr | length == 1' - ceph versions | jq -e '.mgr | keys' | grep $sha1 - ceph versions | jq -e '.overall | length == 2' - ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '.up_to_date | length == 2' - ceph orch ps - cephadm.shell: env: - sha1 host.a: - ceph config set mgr mgr/orchestrator/fail_fs true - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - cephadm.shell: env: - sha1 host.a: - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done - ceph orch ps - ceph orch upgrade status - ceph health detail - ceph versions - echo "wait for servicemap items w/ changing names to refresh" - sleep 60 - ceph orch ps - ceph versions - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 workload-tasks: sequential: - workunit: clients: all: - suites/fsstress.sh 2024-09-16T00:06:33.508 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa; will attempt to use it 2024-09-16T00:06:33.509 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa/tasks 2024-09-16T00:06:33.509 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-16T00:06:33.511 INFO:teuthology.task.internal:Checking packages... 2024-09-16T00:06:33.533 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '8921c8ef947807b717db041d85f0761b7914d5fc' 2024-09-16T00:06:33.534 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-16T00:06:33.534 INFO:teuthology.packaging:ref: None 2024-09-16T00:06:33.534 INFO:teuthology.packaging:tag: None 2024-09-16T00:06:33.534 INFO:teuthology.packaging:branch: squid 2024-09-16T00:06:33.534 INFO:teuthology.packaging:sha1: 8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T00:06:33.534 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-16T00:06:33.750 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-328.gad35db06 2024-09-16T00:06:33.751 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-16T00:06:33.782 INFO:teuthology.task.internal:no buildpackages task found 2024-09-16T00:06:33.782 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-16T00:06:33.796 INFO:teuthology.task.internal:Saving configuration 2024-09-16T00:06:33.817 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-16T00:06:33.825 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-16T00:06:33.850 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi112.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906384', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-16 00:01:15.715132', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBO3Pa+ga+wZsky7A/R0nqjiWJbVQ3084nfR3gRU9uRzGn2IAvipvo/w5Ceygv4oXFF8zXquZNmuiVmI9AToUIc4='} 2024-09-16T00:06:33.870 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi169.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906384', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-16 00:01:15.713483', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBg5N7kH6OgEVkcTFgx9+TWJVea6TvkW49kY+Stg04v3n5sOyQsw3Wxg67i20nEbW44ann6gfx8vuLKkrJwpWFo='} 2024-09-16T00:06:33.871 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-16T00:06:33.884 INFO:teuthology.task.internal:roles: ubuntu@smithi112.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2'] 2024-09-16T00:06:33.885 INFO:teuthology.task.internal:roles: ubuntu@smithi169.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5'] 2024-09-16T00:06:33.885 INFO:teuthology.run_tasks:Running task console_log... 2024-09-16T00:06:33.946 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f2472ec7be0>, signals=[15]) 2024-09-16T00:06:33.946 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-16T00:06:33.962 INFO:teuthology.task.internal:Opening connections... 2024-09-16T00:06:33.962 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi112.front.sepia.ceph.com 2024-09-16T00:06:33.964 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T00:06:34.039 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi169.front.sepia.ceph.com 2024-09-16T00:06:34.040 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T00:06:34.119 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-16T00:06:34.159 DEBUG:teuthology.orchestra.run.smithi112:> uname -m 2024-09-16T00:06:34.177 INFO:teuthology.orchestra.run.smithi112.stdout:x86_64 2024-09-16T00:06:34.177 DEBUG:teuthology.orchestra.run.smithi112:> cat /etc/os-release 2024-09-16T00:06:34.237 INFO:teuthology.orchestra.run.smithi112.stdout:NAME="CentOS Stream" 2024-09-16T00:06:34.237 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION="9" 2024-09-16T00:06:34.237 INFO:teuthology.orchestra.run.smithi112.stdout:ID="centos" 2024-09-16T00:06:34.237 INFO:teuthology.orchestra.run.smithi112.stdout:ID_LIKE="rhel fedora" 2024-09-16T00:06:34.238 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION_ID="9" 2024-09-16T00:06:34.238 INFO:teuthology.orchestra.run.smithi112.stdout:PLATFORM_ID="platform:el9" 2024-09-16T00:06:34.238 INFO:teuthology.orchestra.run.smithi112.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-16T00:06:34.238 INFO:teuthology.orchestra.run.smithi112.stdout:ANSI_COLOR="0;31" 2024-09-16T00:06:34.238 INFO:teuthology.orchestra.run.smithi112.stdout:LOGO="fedora-logo-icon" 2024-09-16T00:06:34.238 INFO:teuthology.orchestra.run.smithi112.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-16T00:06:34.238 INFO:teuthology.orchestra.run.smithi112.stdout:HOME_URL="https://centos.org/" 2024-09-16T00:06:34.238 INFO:teuthology.orchestra.run.smithi112.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-16T00:06:34.239 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-16T00:06:34.239 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-16T00:06:34.240 INFO:teuthology.lock.ops:Updating smithi112.front.sepia.ceph.com on lock server 2024-09-16T00:06:34.266 DEBUG:teuthology.orchestra.run.smithi169:> uname -m 2024-09-16T00:06:34.285 INFO:teuthology.orchestra.run.smithi169.stdout:x86_64 2024-09-16T00:06:34.286 DEBUG:teuthology.orchestra.run.smithi169:> cat /etc/os-release 2024-09-16T00:06:34.344 INFO:teuthology.orchestra.run.smithi169.stdout:NAME="CentOS Stream" 2024-09-16T00:06:34.344 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION="9" 2024-09-16T00:06:34.344 INFO:teuthology.orchestra.run.smithi169.stdout:ID="centos" 2024-09-16T00:06:34.344 INFO:teuthology.orchestra.run.smithi169.stdout:ID_LIKE="rhel fedora" 2024-09-16T00:06:34.344 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION_ID="9" 2024-09-16T00:06:34.345 INFO:teuthology.orchestra.run.smithi169.stdout:PLATFORM_ID="platform:el9" 2024-09-16T00:06:34.345 INFO:teuthology.orchestra.run.smithi169.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-16T00:06:34.345 INFO:teuthology.orchestra.run.smithi169.stdout:ANSI_COLOR="0;31" 2024-09-16T00:06:34.345 INFO:teuthology.orchestra.run.smithi169.stdout:LOGO="fedora-logo-icon" 2024-09-16T00:06:34.345 INFO:teuthology.orchestra.run.smithi169.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-16T00:06:34.345 INFO:teuthology.orchestra.run.smithi169.stdout:HOME_URL="https://centos.org/" 2024-09-16T00:06:34.345 INFO:teuthology.orchestra.run.smithi169.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-16T00:06:34.345 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-16T00:06:34.345 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-16T00:06:34.346 INFO:teuthology.lock.ops:Updating smithi169.front.sepia.ceph.com on lock server 2024-09-16T00:06:34.366 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-16T00:06:34.444 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-16T00:06:34.514 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-16T00:06:34.514 DEBUG:teuthology.orchestra.run.smithi112:> test '!' -e /home/ubuntu/cephtest 2024-09-16T00:06:34.517 DEBUG:teuthology.orchestra.run.smithi169:> test '!' -e /home/ubuntu/cephtest 2024-09-16T00:06:34.534 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-16T00:06:34.552 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-16T00:06:34.553 DEBUG:teuthology.orchestra.run.smithi112:> test -z $(ls -A /var/lib/ceph) 2024-09-16T00:06:34.574 DEBUG:teuthology.orchestra.run.smithi169:> test -z $(ls -A /var/lib/ceph) 2024-09-16T00:06:34.653 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-16T00:06:34.728 INFO:teuthology.run_tasks:Running task kernel... 2024-09-16T00:06:34.789 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-16T00:06:34.789 INFO:teuthology.task.kernel:config {'host.a': {'kdb': 1, 'sha1': 'distro'}, 'host.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-16T00:06:34.789 DEBUG:teuthology.orchestra.run.smithi112:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-16T00:06:34.790 DEBUG:teuthology.orchestra.run.smithi169:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-16T00:06:34.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T00:06:34.807 DEBUG:teuthology.orchestra.run.smithi112:> uname -r 2024-09-16T00:06:34.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T00:06:34.808 DEBUG:teuthology.orchestra.run.smithi169:> uname -r 2024-09-16T00:06:34.863 INFO:teuthology.orchestra.run.smithi112.stdout:5.14.0-503.el9.x86_64 2024-09-16T00:06:34.864 INFO:teuthology.task.kernel:Running kernel on smithi112: 5.14.0-503.el9.x86_64 2024-09-16T00:06:34.864 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum install -y kernel 2024-09-16T00:06:34.865 INFO:teuthology.orchestra.run.smithi169.stdout:5.14.0-503.el9.x86_64 2024-09-16T00:06:34.865 INFO:teuthology.task.kernel:Running kernel on smithi169: 5.14.0-503.el9.x86_64 2024-09-16T00:06:34.866 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum install -y kernel 2024-09-16T00:06:37.112 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.2 MB 00:00 2024-09-16T00:06:38.183 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - BaseOS 5.3 MB/s | 8.2 MB 00:01 2024-09-16T00:06:39.929 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - AppStream 12 MB/s | 20 MB 00:01 2024-09-16T00:06:39.976 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - AppStream 30 MB/s | 20 MB 00:00 2024-09-16T00:06:44.697 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - CRB 6.5 MB/s | 6.5 MB 00:00 2024-09-16T00:06:46.368 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - CRB 2.5 MB/s | 6.5 MB 00:02 2024-09-16T00:06:46.489 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 19 kB 00:00 2024-09-16T00:06:47.718 INFO:teuthology.orchestra.run.smithi112.stdout:Extra Packages for Enterprise Linux 21 MB/s | 23 MB 00:01 2024-09-16T00:06:47.969 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - Extras packages 70 kB/s | 19 kB 00:00 2024-09-16T00:06:48.723 INFO:teuthology.orchestra.run.smithi169.stdout:Extra Packages for Enterprise Linux 36 MB/s | 23 MB 00:00 2024-09-16T00:06:53.116 INFO:teuthology.orchestra.run.smithi112.stdout:lab-extras 45 kB/s | 1.7 kB 00:00 2024-09-16T00:06:54.153 INFO:teuthology.orchestra.run.smithi169.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-16T00:06:55.139 INFO:teuthology.orchestra.run.smithi112.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-16T00:06:55.139 INFO:teuthology.orchestra.run.smithi112.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-16T00:06:55.242 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2024-09-16T00:06:55.250 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-09-16T00:06:55.250 INFO:teuthology.orchestra.run.smithi112.stdout: Package Architecture Version Repository Size 2024-09-16T00:06:55.251 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-09-16T00:06:55.251 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2024-09-16T00:06:55.251 INFO:teuthology.orchestra.run.smithi112.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-16T00:06:55.251 INFO:teuthology.orchestra.run.smithi112.stdout:Installing dependencies: 2024-09-16T00:06:55.251 INFO:teuthology.orchestra.run.smithi112.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-16T00:06:55.251 INFO:teuthology.orchestra.run.smithi112.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-16T00:06:55.251 INFO:teuthology.orchestra.run.smithi112.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-16T00:06:55.251 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:06:55.251 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2024-09-16T00:06:55.251 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-09-16T00:06:55.251 INFO:teuthology.orchestra.run.smithi112.stdout:Install 4 Packages 2024-09-16T00:06:55.251 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:06:55.252 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 79 M 2024-09-16T00:06:55.252 INFO:teuthology.orchestra.run.smithi112.stdout:Installed size: 126 M 2024-09-16T00:06:55.252 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2024-09-16T00:06:55.509 INFO:teuthology.orchestra.run.smithi112.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 182 kB/s | 23 kB 00:00 2024-09-16T00:06:56.193 INFO:teuthology.orchestra.run.smithi169.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-16T00:06:56.193 INFO:teuthology.orchestra.run.smithi169.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-16T00:06:56.274 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-09-16T00:06:56.279 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T00:06:56.280 INFO:teuthology.orchestra.run.smithi169.stdout: Package Architecture Version Repository Size 2024-09-16T00:06:56.280 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T00:06:56.280 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2024-09-16T00:06:56.280 INFO:teuthology.orchestra.run.smithi169.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-16T00:06:56.280 INFO:teuthology.orchestra.run.smithi169.stdout:Installing dependencies: 2024-09-16T00:06:56.280 INFO:teuthology.orchestra.run.smithi169.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-16T00:06:56.280 INFO:teuthology.orchestra.run.smithi169.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-16T00:06:56.280 INFO:teuthology.orchestra.run.smithi169.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-16T00:06:56.280 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:06:56.280 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2024-09-16T00:06:56.280 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T00:06:56.280 INFO:teuthology.orchestra.run.smithi169.stdout:Install 4 Packages 2024-09-16T00:06:56.280 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:06:56.281 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 79 M 2024-09-16T00:06:56.281 INFO:teuthology.orchestra.run.smithi169.stdout:Installed size: 126 M 2024-09-16T00:06:56.281 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2024-09-16T00:06:56.301 INFO:teuthology.orchestra.run.smithi112.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 17 MB/s | 16 MB 00:00 2024-09-16T00:06:56.446 INFO:teuthology.orchestra.run.smithi169.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 197 kB/s | 23 kB 00:00 2024-09-16T00:06:56.826 INFO:teuthology.orchestra.run.smithi112.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 22 MB/s | 29 MB 00:01 2024-09-16T00:06:57.143 INFO:teuthology.orchestra.run.smithi112.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 20 MB/s | 35 MB 00:01 2024-09-16T00:06:57.143 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2024-09-16T00:06:57.143 INFO:teuthology.orchestra.run.smithi112.stdout:Total 42 MB/s | 79 MB 00:01 2024-09-16T00:06:57.472 INFO:teuthology.orchestra.run.smithi169.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 14 MB/s | 16 MB 00:01 2024-09-16T00:06:57.824 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2024-09-16T00:06:57.988 INFO:teuthology.orchestra.run.smithi169.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 21 MB/s | 35 MB 00:01 2024-09-16T00:06:58.178 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2024-09-16T00:06:58.178 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2024-09-16T00:06:59.184 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2024-09-16T00:06:59.184 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2024-09-16T00:07:00.463 INFO:teuthology.orchestra.run.smithi169.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 7.1 MB/s | 29 MB 00:04 2024-09-16T00:07:00.464 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2024-09-16T00:07:00.464 INFO:teuthology.orchestra.run.smithi169.stdout:Total 19 MB/s | 79 MB 00:04 2024-09-16T00:07:01.145 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2024-09-16T00:07:01.504 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2024-09-16T00:07:01.504 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2024-09-16T00:07:01.644 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2024-09-16T00:07:02.325 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-16T00:07:02.402 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T00:07:02.470 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2024-09-16T00:07:02.471 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2024-09-16T00:07:04.918 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2024-09-16T00:07:05.212 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T00:07:05.348 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T00:07:05.683 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-16T00:07:05.766 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T00:07:08.563 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T00:07:08.687 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T00:07:10.169 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T00:07:10.256 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-16T00:07:13.535 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T00:07:13.654 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-16T00:07:15.070 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T00:07:18.532 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T00:07:53.219 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T00:07:53.258 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-16T00:07:53.573 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-16T00:07:53.573 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-16T00:07:53.573 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T00:07:53.573 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T00:07:54.444 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T00:07:54.445 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:07:54.445 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2024-09-16T00:07:54.445 INFO:teuthology.orchestra.run.smithi112.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-16T00:07:54.445 INFO:teuthology.orchestra.run.smithi112.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-16T00:07:54.445 INFO:teuthology.orchestra.run.smithi112.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-16T00:07:54.445 INFO:teuthology.orchestra.run.smithi112.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-16T00:07:54.446 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:07:54.446 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2024-09-16T00:07:54.794 DEBUG:teuthology.orchestra.run.smithi112:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T00:07:54.890 INFO:teuthology.orchestra.run.smithi112.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-16T00:07:54.891 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-16T00:07:54.892 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-16T00:07:54.892 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-16T00:07:54.892 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-16T00:07:54.892 INFO:teuthology.task.kernel:Installing distro kernel on host.a... 2024-09-16T00:07:54.892 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi112.front.sepia.ceph.com, path=None, version=distro) 2024-09-16T00:07:54.892 DEBUG:teuthology.orchestra.run.smithi112:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T00:07:54.952 INFO:teuthology.orchestra.run.smithi112.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-16T00:07:54.952 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-16T00:07:54.952 DEBUG:teuthology.orchestra.run.smithi112:> sudo rpm -qi grub2-tools 2024-09-16T00:07:54.983 INFO:teuthology.orchestra.run.smithi112.stdout:Name : grub2-tools 2024-09-16T00:07:54.984 INFO:teuthology.orchestra.run.smithi112.stdout:Epoch : 1 2024-09-16T00:07:54.984 INFO:teuthology.orchestra.run.smithi112.stdout:Version : 2.06 2024-09-16T00:07:54.984 INFO:teuthology.orchestra.run.smithi112.stdout:Release : 82.el9 2024-09-16T00:07:54.984 INFO:teuthology.orchestra.run.smithi112.stdout:Architecture: x86_64 2024-09-16T00:07:54.984 INFO:teuthology.orchestra.run.smithi112.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-16T00:07:54.984 INFO:teuthology.orchestra.run.smithi112.stdout:Group : Unspecified 2024-09-16T00:07:54.984 INFO:teuthology.orchestra.run.smithi112.stdout:Size : 8274670 2024-09-16T00:07:54.984 INFO:teuthology.orchestra.run.smithi112.stdout:License : GPLv3+ 2024-09-16T00:07:54.984 INFO:teuthology.orchestra.run.smithi112.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-16T00:07:54.985 INFO:teuthology.orchestra.run.smithi112.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-16T00:07:54.985 INFO:teuthology.orchestra.run.smithi112.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-16T00:07:54.985 INFO:teuthology.orchestra.run.smithi112.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-16T00:07:54.985 INFO:teuthology.orchestra.run.smithi112.stdout:Packager : builder@centos.org 2024-09-16T00:07:54.985 INFO:teuthology.orchestra.run.smithi112.stdout:Vendor : CentOS 2024-09-16T00:07:54.985 INFO:teuthology.orchestra.run.smithi112.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-16T00:07:54.985 INFO:teuthology.orchestra.run.smithi112.stdout:Summary : Support tools for GRUB. 2024-09-16T00:07:54.985 INFO:teuthology.orchestra.run.smithi112.stdout:Description : 2024-09-16T00:07:54.986 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:07:54.986 INFO:teuthology.orchestra.run.smithi112.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-16T00:07:54.986 INFO:teuthology.orchestra.run.smithi112.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-16T00:07:54.987 INFO:teuthology.orchestra.run.smithi112.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-16T00:07:54.987 INFO:teuthology.orchestra.run.smithi112.stdout:hardware devices. 2024-09-16T00:07:54.987 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:07:54.987 INFO:teuthology.orchestra.run.smithi112.stdout:This subpackage provides tools for support of all platforms. 2024-09-16T00:07:54.988 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-16T00:07:54.988 INFO:teuthology.task.kernel:Updating grub on smithi112 to boot 5.14.0-505.el9.x86_64 2024-09-16T00:07:54.988 DEBUG:teuthology.orchestra.run.smithi112:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-16T00:07:55.632 INFO:teuthology.orchestra.run.smithi112.stderr:Generating grub configuration file ... 2024-09-16T00:07:56.735 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T00:07:56.775 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-16T00:07:57.011 INFO:teuthology.orchestra.run.smithi112.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-16T00:07:57.032 INFO:teuthology.orchestra.run.smithi112.stderr:done 2024-09-16T00:07:57.034 DEBUG:teuthology.orchestra.run.smithi112:> mktemp 2024-09-16T00:07:57.042 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-16T00:07:57.042 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-16T00:07:57.042 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T00:07:57.042 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T00:07:57.071 INFO:teuthology.orchestra.run.smithi112.stdout:/tmp/tmp.wwt9uwuuTb 2024-09-16T00:07:57.071 DEBUG:teuthology.orchestra.run.smithi112:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.wwt9uwuuTb 2024-09-16T00:07:57.144 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 0666 /tmp/tmp.wwt9uwuuTb 2024-09-16T00:07:57.284 DEBUG:teuthology.orchestra.remote:smithi112:/tmp/tmp.wwt9uwuuTb is 6KB 2024-09-16T00:07:57.334 DEBUG:teuthology.orchestra.run.smithi112:> rm -fr /tmp/tmp.wwt9uwuuTb 2024-09-16T00:07:57.349 DEBUG:teuthology.orchestra.run.smithi112:> sudo /bin/ls /boot/loader/entries || true 2024-09-16T00:07:57.412 INFO:teuthology.orchestra.run.smithi112.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-16T00:07:57.412 INFO:teuthology.orchestra.run.smithi112.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-16T00:07:57.413 INFO:teuthology.orchestra.run.smithi112.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-16T00:07:57.413 INFO:teuthology.orchestra.run.smithi112.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-16T00:07:57.414 DEBUG:teuthology.orchestra.run.smithi112:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-16T00:07:57.569 DEBUG:teuthology.orchestra.run.smithi112:> sudo shutdown -r now 2024-09-16T00:07:57.938 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T00:07:57.938 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:07:57.938 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2024-09-16T00:07:57.938 INFO:teuthology.orchestra.run.smithi169.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-16T00:07:57.938 INFO:teuthology.orchestra.run.smithi169.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-16T00:07:57.939 INFO:teuthology.orchestra.run.smithi169.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-16T00:07:57.939 INFO:teuthology.orchestra.run.smithi169.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-16T00:07:57.939 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:07:57.939 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-09-16T00:07:58.259 DEBUG:teuthology.orchestra.run.smithi169:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T00:07:58.346 INFO:teuthology.orchestra.run.smithi169.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-16T00:07:58.346 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-16T00:07:58.346 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-16T00:07:58.346 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-16T00:07:58.346 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-16T00:07:58.346 INFO:teuthology.task.kernel:Installing distro kernel on host.b... 2024-09-16T00:07:58.346 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi169.front.sepia.ceph.com, path=None, version=distro) 2024-09-16T00:07:58.346 DEBUG:teuthology.orchestra.run.smithi169:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T00:07:58.406 INFO:teuthology.orchestra.run.smithi169.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-16T00:07:58.406 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-16T00:07:58.406 DEBUG:teuthology.orchestra.run.smithi169:> sudo rpm -qi grub2-tools 2024-09-16T00:07:58.455 INFO:teuthology.orchestra.run.smithi169.stdout:Name : grub2-tools 2024-09-16T00:07:58.455 INFO:teuthology.orchestra.run.smithi169.stdout:Epoch : 1 2024-09-16T00:07:58.455 INFO:teuthology.orchestra.run.smithi169.stdout:Version : 2.06 2024-09-16T00:07:58.455 INFO:teuthology.orchestra.run.smithi169.stdout:Release : 82.el9 2024-09-16T00:07:58.455 INFO:teuthology.orchestra.run.smithi169.stdout:Architecture: x86_64 2024-09-16T00:07:58.455 INFO:teuthology.orchestra.run.smithi169.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-16T00:07:58.455 INFO:teuthology.orchestra.run.smithi169.stdout:Group : Unspecified 2024-09-16T00:07:58.455 INFO:teuthology.orchestra.run.smithi169.stdout:Size : 8274670 2024-09-16T00:07:58.455 INFO:teuthology.orchestra.run.smithi169.stdout:License : GPLv3+ 2024-09-16T00:07:58.456 INFO:teuthology.orchestra.run.smithi169.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-16T00:07:58.456 INFO:teuthology.orchestra.run.smithi169.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-16T00:07:58.456 INFO:teuthology.orchestra.run.smithi169.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-16T00:07:58.456 INFO:teuthology.orchestra.run.smithi169.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-16T00:07:58.456 INFO:teuthology.orchestra.run.smithi169.stdout:Packager : builder@centos.org 2024-09-16T00:07:58.456 INFO:teuthology.orchestra.run.smithi169.stdout:Vendor : CentOS 2024-09-16T00:07:58.456 INFO:teuthology.orchestra.run.smithi169.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-16T00:07:58.456 INFO:teuthology.orchestra.run.smithi169.stdout:Summary : Support tools for GRUB. 2024-09-16T00:07:58.456 INFO:teuthology.orchestra.run.smithi169.stdout:Description : 2024-09-16T00:07:58.456 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:07:58.456 INFO:teuthology.orchestra.run.smithi169.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-16T00:07:58.456 INFO:teuthology.orchestra.run.smithi169.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-16T00:07:58.456 INFO:teuthology.orchestra.run.smithi169.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-16T00:07:58.457 INFO:teuthology.orchestra.run.smithi169.stdout:hardware devices. 2024-09-16T00:07:58.457 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:07:58.457 INFO:teuthology.orchestra.run.smithi169.stdout:This subpackage provides tools for support of all platforms. 2024-09-16T00:07:58.458 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-16T00:07:58.458 INFO:teuthology.task.kernel:Updating grub on smithi169 to boot 5.14.0-505.el9.x86_64 2024-09-16T00:07:58.458 DEBUG:teuthology.orchestra.run.smithi169:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-16T00:07:59.132 INFO:teuthology.orchestra.run.smithi169.stderr:Generating grub configuration file ... 2024-09-16T00:08:00.473 INFO:teuthology.orchestra.run.smithi169.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-16T00:08:00.499 INFO:teuthology.orchestra.run.smithi169.stderr:done 2024-09-16T00:08:00.501 DEBUG:teuthology.orchestra.run.smithi169:> mktemp 2024-09-16T00:08:00.517 INFO:teuthology.orchestra.run.smithi169.stdout:/tmp/tmp.On6ohuVotZ 2024-09-16T00:08:00.518 DEBUG:teuthology.orchestra.run.smithi169:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.On6ohuVotZ 2024-09-16T00:08:00.585 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 0666 /tmp/tmp.On6ohuVotZ 2024-09-16T00:08:00.731 DEBUG:teuthology.orchestra.remote:smithi169:/tmp/tmp.On6ohuVotZ is 6KB 2024-09-16T00:08:00.779 DEBUG:teuthology.orchestra.run.smithi169:> rm -fr /tmp/tmp.On6ohuVotZ 2024-09-16T00:08:00.794 DEBUG:teuthology.orchestra.run.smithi169:> sudo /bin/ls /boot/loader/entries || true 2024-09-16T00:08:00.858 INFO:teuthology.orchestra.run.smithi169.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-16T00:08:00.858 INFO:teuthology.orchestra.run.smithi169.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-16T00:08:00.858 INFO:teuthology.orchestra.run.smithi169.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-16T00:08:00.858 INFO:teuthology.orchestra.run.smithi169.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-16T00:08:00.859 DEBUG:teuthology.orchestra.run.smithi169:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-16T00:08:01.014 DEBUG:teuthology.orchestra.run.smithi169:> sudo shutdown -r now 2024-09-16T00:08:27.597 INFO:teuthology.task.kernel:Checking client host.a for new kernel version... 2024-09-16T00:08:27.597 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-09-16T00:08:27.598 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T00:08:31.021 INFO:teuthology.task.kernel:Checking client host.b for new kernel version... 2024-09-16T00:08:31.021 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2024-09-16T00:08:31.021 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T00:09:02.269 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.112 2024-09-16T00:09:05.597 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.169 2024-09-16T00:09:11.273 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-09-16T00:09:11.273 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T00:09:14.601 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2024-09-16T00:09:14.602 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T00:09:29.661 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.112 2024-09-16T00:09:32.989 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.169 2024-09-16T00:09:41.669 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-09-16T00:09:41.669 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T00:09:42.039 DEBUG:teuthology.orchestra.run.smithi112:> true 2024-09-16T00:09:42.454 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-09-16T00:09:42.454 INFO:teuthology.task.kernel:Checking kernel version of host.a, want "5.14.0-505.el9.x86_64"... 2024-09-16T00:09:42.455 DEBUG:teuthology.orchestra.run.smithi112:> uname -r 2024-09-16T00:09:42.470 INFO:teuthology.orchestra.run.smithi112.stdout:5.14.0-505.el9.x86_64 2024-09-16T00:09:42.470 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-16T00:09:42.470 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-16T00:09:42.470 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-16T00:09:43.471 INFO:teuthology.task.kernel:Enabling kdb on host.a... 2024-09-16T00:09:43.471 DEBUG:teuthology.orchestra.run.smithi112:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-16T00:09:43.539 INFO:teuthology.orchestra.run.smithi112.stdout:ttyS1 2024-09-16T00:09:43.564 DEBUG:teuthology.parallel:result is None 2024-09-16T00:09:44.990 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2024-09-16T00:09:44.990 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T00:09:47.673 DEBUG:teuthology.orchestra.run.smithi169:> true 2024-09-16T00:09:48.076 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi169.front.sepia.ceph.com' 2024-09-16T00:09:48.077 INFO:teuthology.task.kernel:Checking kernel version of host.b, want "5.14.0-505.el9.x86_64"... 2024-09-16T00:09:48.077 DEBUG:teuthology.orchestra.run.smithi169:> uname -r 2024-09-16T00:09:48.093 INFO:teuthology.orchestra.run.smithi169.stdout:5.14.0-505.el9.x86_64 2024-09-16T00:09:48.094 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-16T00:09:48.094 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-16T00:09:48.094 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-16T00:09:49.094 INFO:teuthology.task.kernel:Enabling kdb on host.b... 2024-09-16T00:09:49.095 DEBUG:teuthology.orchestra.run.smithi169:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-16T00:09:49.213 INFO:teuthology.orchestra.run.smithi169.stdout:ttyS1 2024-09-16T00:09:49.232 DEBUG:teuthology.parallel:result is None 2024-09-16T00:09:49.233 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-16T00:09:49.241 INFO:teuthology.task.internal:Creating test directory... 2024-09-16T00:09:49.241 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-16T00:09:49.244 DEBUG:teuthology.orchestra.run.smithi169:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-16T00:09:49.260 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-16T00:09:49.269 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-16T00:09:49.278 INFO:teuthology.task.internal:Creating archive directory... 2024-09-16T00:09:49.278 DEBUG:teuthology.orchestra.run.smithi112:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-16T00:09:49.303 DEBUG:teuthology.orchestra.run.smithi169:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-16T00:09:49.334 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-16T00:09:49.340 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-16T00:09:49.341 DEBUG:teuthology.orchestra.run.smithi112:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-16T00:09:49.371 DEBUG:teuthology.orchestra.run.smithi169:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-16T00:09:49.396 INFO:teuthology.orchestra.run.smithi112.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T00:09:49.405 INFO:teuthology.orchestra.run.smithi169.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T00:09:49.406 INFO:teuthology.orchestra.run.smithi112.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T00:09:49.414 INFO:teuthology.orchestra.run.smithi169.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T00:09:49.416 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-16T00:09:49.424 INFO:teuthology.task.internal:Configuring sudo... 2024-09-16T00:09:49.424 DEBUG:teuthology.orchestra.run.smithi112:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-16T00:09:49.451 DEBUG:teuthology.orchestra.run.smithi169:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-16T00:09:49.481 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-16T00:09:49.490 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-16T00:09:49.490 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-16T00:09:49.518 DEBUG:teuthology.orchestra.run.smithi169:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-16T00:09:49.536 DEBUG:teuthology.orchestra.run.smithi112:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T00:09:49.611 DEBUG:teuthology.orchestra.run.smithi112:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T00:09:49.665 DEBUG:teuthology.orchestra.run.smithi112:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T00:09:49.742 DEBUG:teuthology.orchestra.run.smithi112:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T00:09:49.785 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-16T00:09:49.786 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-16T00:09:49.850 DEBUG:teuthology.orchestra.run.smithi169:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T00:09:49.891 DEBUG:teuthology.orchestra.run.smithi169:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T00:09:49.936 DEBUG:teuthology.orchestra.run.smithi169:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T00:09:50.019 DEBUG:teuthology.orchestra.run.smithi169:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T00:09:50.060 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-09-16T00:09:50.060 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-16T00:09:50.132 DEBUG:teuthology.orchestra.run.smithi112:> sudo service rsyslog restart 2024-09-16T00:09:50.135 DEBUG:teuthology.orchestra.run.smithi169:> sudo service rsyslog restart 2024-09-16T00:09:50.177 INFO:teuthology.orchestra.run.smithi112.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T00:09:50.214 INFO:teuthology.orchestra.run.smithi169.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T00:09:50.684 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-16T00:09:50.719 INFO:teuthology.task.internal:Starting timer... 2024-09-16T00:09:50.719 INFO:teuthology.run_tasks:Running task pcp... 2024-09-16T00:09:50.729 INFO:teuthology.run_tasks:Running task selinux... 2024-09-16T00:09:50.742 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-16T00:09:50.742 DEBUG:teuthology.orchestra.run.smithi112:> sudo service auditd rotate 2024-09-16T00:09:50.801 INFO:teuthology.orchestra.run.smithi112.stdout:Rotating logs: 2024-09-16T00:09:50.803 DEBUG:teuthology.orchestra.run.smithi169:> sudo service auditd rotate 2024-09-16T00:09:50.853 INFO:teuthology.orchestra.run.smithi169.stdout:Rotating logs: 2024-09-16T00:09:50.856 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-16T00:09:50.856 DEBUG:teuthology.orchestra.run.smithi112:> /usr/sbin/getenforce 2024-09-16T00:09:50.889 INFO:teuthology.orchestra.run.smithi112.stdout:Permissive 2024-09-16T00:09:50.890 DEBUG:teuthology.orchestra.run.smithi169:> /usr/sbin/getenforce 2024-09-16T00:09:50.912 INFO:teuthology.orchestra.run.smithi169.stdout:Permissive 2024-09-16T00:09:50.912 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi112.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi169.front.sepia.ceph.com': 'permissive'} 2024-09-16T00:09:50.913 DEBUG:teuthology.orchestra.run.smithi112:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-16T00:09:50.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T00:09:50.955 DEBUG:teuthology.orchestra.run.smithi169:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-16T00:09:50.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T00:09:50.981 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-16T00:09:50.981 DEBUG:teuthology.orchestra.run.smithi112:> sudo /usr/sbin/setenforce permissive 2024-09-16T00:09:51.025 DEBUG:teuthology.orchestra.run.smithi169:> sudo /usr/sbin/setenforce permissive 2024-09-16T00:09:51.060 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-16T00:09:51.069 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-16T00:09:51.081 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-16T00:09:51.082 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi112.front.sepia.ceph.com,smithi169.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-16T00:13:37.683 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi112.front.sepia.ceph.com'), Remote(name='ubuntu@smithi169.front.sepia.ceph.com')] 2024-09-16T00:13:37.684 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-09-16T00:13:37.685 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T00:13:37.776 DEBUG:teuthology.orchestra.run.smithi112:> true 2024-09-16T00:13:37.860 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-09-16T00:13:37.860 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2024-09-16T00:13:37.861 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T00:13:37.939 DEBUG:teuthology.orchestra.run.smithi169:> true 2024-09-16T00:13:38.016 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi169.front.sepia.ceph.com' 2024-09-16T00:13:38.017 INFO:teuthology.run_tasks:Running task clock... 2024-09-16T00:13:38.026 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-16T00:13:38.026 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T00:13:38.027 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T00:13:38.029 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T00:13:38.029 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T00:13:38.056 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-16T00:13:38.070 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-16T00:13:38.086 INFO:teuthology.orchestra.run.smithi169.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-16T00:13:38.094 INFO:teuthology.orchestra.run.smithi112.stderr:sudo: ntpd: command not found 2024-09-16T00:13:38.100 INFO:teuthology.orchestra.run.smithi169.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-16T00:13:38.105 INFO:teuthology.orchestra.run.smithi112.stdout:506 Cannot talk to daemon 2024-09-16T00:13:38.119 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-16T00:13:38.126 INFO:teuthology.orchestra.run.smithi169.stderr:sudo: ntpd: command not found 2024-09-16T00:13:38.133 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-16T00:13:38.137 INFO:teuthology.orchestra.run.smithi169.stdout:506 Cannot talk to daemon 2024-09-16T00:13:38.151 INFO:teuthology.orchestra.run.smithi169.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-16T00:13:38.166 INFO:teuthology.orchestra.run.smithi169.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-16T00:13:38.189 INFO:teuthology.orchestra.run.smithi112.stderr:bash: line 1: ntpq: command not found 2024-09-16T00:13:38.193 INFO:teuthology.orchestra.run.smithi112.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T00:13:38.193 INFO:teuthology.orchestra.run.smithi112.stdout:=============================================================================== 2024-09-16T00:13:38.193 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:13:38.193 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:13:38.194 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:13:38.194 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:13:38.223 INFO:teuthology.orchestra.run.smithi169.stderr:bash: line 1: ntpq: command not found 2024-09-16T00:13:38.227 INFO:teuthology.orchestra.run.smithi169.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T00:13:38.227 INFO:teuthology.orchestra.run.smithi169.stdout:=============================================================================== 2024-09-16T00:13:38.228 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:13:38.228 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:13:38.228 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:13:38.228 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:13:38.229 INFO:teuthology.run_tasks:Running task install... 2024-09-16T00:13:38.237 DEBUG:teuthology.task.install:project ceph 2024-09-16T00:13:38.237 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc'}} 2024-09-16T00:13:38.237 DEBUG:teuthology.task.install:config {'branch': 'quincy', 'exclude_packages': ['ceph-volume'], 'flavor': 'default', 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc'} 2024-09-16T00:13:38.237 INFO:teuthology.task.install:Using flavor: default 2024-09-16T00:13:38.243 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-16T00:13:38.243 INFO:teuthology.task.install:extra packages: [] 2024-09-16T00:13:38.243 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'quincy', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'tag': None, 'wait_for_package': False} 2024-09-16T00:13:38.243 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-16T00:13:38.243 INFO:teuthology.packaging:ref: None 2024-09-16T00:13:38.244 INFO:teuthology.packaging:tag: None 2024-09-16T00:13:38.244 INFO:teuthology.packaging:branch: quincy 2024-09-16T00:13:38.244 INFO:teuthology.packaging:sha1: 8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T00:13:38.244 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-09-16T00:13:38.246 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'quincy', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'tag': None, 'wait_for_package': False} 2024-09-16T00:13:38.246 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-16T00:13:38.246 INFO:teuthology.packaging:ref: None 2024-09-16T00:13:38.246 INFO:teuthology.packaging:tag: None 2024-09-16T00:13:38.246 INFO:teuthology.packaging:branch: quincy 2024-09-16T00:13:38.246 INFO:teuthology.packaging:sha1: 8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T00:13:38.246 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-09-16T00:13:38.469 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/quincy/6d025bc26f3176c910707f8cb4c8d8e076640615/centos/9/flavors/default/ 2024-09-16T00:13:38.469 INFO:teuthology.task.install.rpm:Package version is 17.2.7-1633.g6d025bc2 2024-09-16T00:13:38.481 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/quincy/6d025bc26f3176c910707f8cb4c8d8e076640615/centos/9/flavors/default/ 2024-09-16T00:13:38.481 INFO:teuthology.task.install.rpm:Package version is 17.2.7-1633.g6d025bc2 2024-09-16T00:13:38.581 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/quincy/6d025bc26f3176c910707f8cb4c8d8e076640615/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/quincy/6d025bc26f3176c910707f8cb4c8d8e076640615/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/quincy/6d025bc26f3176c910707f8cb4c8d8e076640615/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-16T00:13:38.582 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-16T00:13:38.582 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-16T00:13:38.592 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/quincy/6d025bc26f3176c910707f8cb4c8d8e076640615/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/quincy/6d025bc26f3176c910707f8cb4c8d8e076640615/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/quincy/6d025bc26f3176c910707f8cb4c8d8e076640615/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-16T00:13:38.593 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-09-16T00:13:38.593 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-16T00:13:38.608 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-16T00:13:38.608 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-16T00:13:38.608 INFO:teuthology.packaging:ref: None 2024-09-16T00:13:38.608 INFO:teuthology.packaging:tag: None 2024-09-16T00:13:38.609 INFO:teuthology.packaging:branch: quincy 2024-09-16T00:13:38.609 INFO:teuthology.packaging:sha1: 8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T00:13:38.609 DEBUG:teuthology.orchestra.run.smithi112:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/quincy/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-16T00:13:38.620 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-16T00:13:38.620 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-16T00:13:38.620 INFO:teuthology.packaging:ref: None 2024-09-16T00:13:38.620 INFO:teuthology.packaging:tag: None 2024-09-16T00:13:38.620 INFO:teuthology.packaging:branch: quincy 2024-09-16T00:13:38.620 INFO:teuthology.packaging:sha1: 8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T00:13:38.620 DEBUG:teuthology.orchestra.run.smithi169:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/quincy/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-16T00:13:38.674 DEBUG:teuthology.orchestra.run.smithi112:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-16T00:13:38.686 DEBUG:teuthology.orchestra.run.smithi169:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-16T00:13:38.749 DEBUG:teuthology.orchestra.run.smithi112:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-16T00:13:38.761 DEBUG:teuthology.orchestra.run.smithi169:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-16T00:13:38.813 INFO:teuthology.orchestra.run.smithi112.stdout:check_obsoletes = 1 2024-09-16T00:13:38.815 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum clean all 2024-09-16T00:13:38.825 INFO:teuthology.orchestra.run.smithi169.stdout:check_obsoletes = 1 2024-09-16T00:13:38.827 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum clean all 2024-09-16T00:13:39.054 INFO:teuthology.orchestra.run.smithi169.stdout:45 files removed 2024-09-16T00:13:39.055 INFO:teuthology.orchestra.run.smithi112.stdout:45 files removed 2024-09-16T00:13:39.082 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install ceph-radosgw 2024-09-16T00:13:39.083 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install ceph-radosgw 2024-09-16T00:13:39.634 INFO:teuthology.orchestra.run.smithi112.stdout:ceph packages for x86_64 289 kB/s | 77 kB 00:00 2024-09-16T00:13:39.635 INFO:teuthology.orchestra.run.smithi169.stdout:ceph packages for x86_64 292 kB/s | 77 kB 00:00 2024-09-16T00:13:39.918 INFO:teuthology.orchestra.run.smithi112.stdout:ceph noarch packages 45 kB/s | 11 kB 00:00 2024-09-16T00:13:39.918 INFO:teuthology.orchestra.run.smithi169.stdout:ceph noarch packages 45 kB/s | 11 kB 00:00 2024-09-16T00:13:40.185 INFO:teuthology.orchestra.run.smithi112.stdout:ceph source packages 7.8 kB/s | 1.8 kB 00:00 2024-09-16T00:13:40.194 INFO:teuthology.orchestra.run.smithi169.stdout:ceph source packages 7.5 kB/s | 1.8 kB 00:00 2024-09-16T00:13:41.077 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - BaseOS 9.7 MB/s | 8.2 MB 00:00 2024-09-16T00:13:41.719 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - BaseOS 5.5 MB/s | 8.2 MB 00:01 2024-09-16T00:13:43.670 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - AppStream 24 MB/s | 20 MB 00:00 2024-09-16T00:13:47.028 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - AppStream 4.2 MB/s | 20 MB 00:04 2024-09-16T00:13:47.862 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - CRB 18 MB/s | 6.5 MB 00:00 2024-09-16T00:13:49.313 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - Extras packages 83 kB/s | 19 kB 00:00 2024-09-16T00:13:50.353 INFO:teuthology.orchestra.run.smithi112.stdout:Extra Packages for Enterprise Linux 25 MB/s | 23 MB 00:00 2024-09-16T00:13:51.179 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - CRB 17 MB/s | 6.5 MB 00:00 2024-09-16T00:13:52.821 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - Extras packages 60 kB/s | 19 kB 00:00 2024-09-16T00:13:53.592 INFO:teuthology.orchestra.run.smithi169.stdout:Extra Packages for Enterprise Linux 35 MB/s | 23 MB 00:00 2024-09-16T00:13:55.834 INFO:teuthology.orchestra.run.smithi112.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-09-16T00:13:58.184 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2024-09-16T00:13:58.194 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-09-16T00:13:58.195 INFO:teuthology.orchestra.run.smithi112.stdout: Package Arch Version Repository Size 2024-09-16T00:13:58.195 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-09-16T00:13:58.195 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2024-09-16T00:13:58.195 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-radosgw x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 14 M 2024-09-16T00:13:58.195 INFO:teuthology.orchestra.run.smithi112.stdout:Upgrading: 2024-09-16T00:13:58.195 INFO:teuthology.orchestra.run.smithi112.stdout: librados2 x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 3.1 M 2024-09-16T00:13:58.195 INFO:teuthology.orchestra.run.smithi112.stdout: librbd1 x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 2.9 M 2024-09-16T00:13:58.195 INFO:teuthology.orchestra.run.smithi112.stdout:Installing dependencies: 2024-09-16T00:13:58.195 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-base x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 5.0 M 2024-09-16T00:13:58.195 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-common x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 22 M 2024-09-16T00:13:58.196 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-selinux x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 25 k 2024-09-16T00:13:58.196 INFO:teuthology.orchestra.run.smithi112.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-16T00:13:58.196 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs2 x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 637 k 2024-09-16T00:13:58.196 INFO:teuthology.orchestra.run.smithi112.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-16T00:13:58.196 INFO:teuthology.orchestra.run.smithi112.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-16T00:13:58.196 INFO:teuthology.orchestra.run.smithi112.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-16T00:13:58.196 INFO:teuthology.orchestra.run.smithi112.stdout: libradosstriper1 x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 457 k 2024-09-16T00:13:58.196 INFO:teuthology.orchestra.run.smithi112.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-16T00:13:58.196 INFO:teuthology.orchestra.run.smithi112.stdout: librgw2 x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 4.9 M 2024-09-16T00:13:58.196 INFO:teuthology.orchestra.run.smithi112.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-16T00:13:58.196 INFO:teuthology.orchestra.run.smithi112.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-16T00:13:58.196 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-argparse x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 45 k 2024-09-16T00:13:58.196 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-common x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 120 k 2024-09-16T00:13:58.197 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cephfs x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 156 k 2024-09-16T00:13:58.197 INFO:teuthology.orchestra.run.smithi112.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-16T00:13:58.197 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rados x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 320 k 2024-09-16T00:13:58.197 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rbd x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 295 k 2024-09-16T00:13:58.197 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rgw x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 99 k 2024-09-16T00:13:58.197 INFO:teuthology.orchestra.run.smithi112.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-16T00:13:58.197 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:13:58.197 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2024-09-16T00:13:58.197 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-09-16T00:13:58.197 INFO:teuthology.orchestra.run.smithi112.stdout:Install 22 Packages 2024-09-16T00:13:58.197 INFO:teuthology.orchestra.run.smithi112.stdout:Upgrade 2 Packages 2024-09-16T00:13:58.197 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:13:58.198 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 58 M 2024-09-16T00:13:58.198 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2024-09-16T00:13:59.031 INFO:teuthology.orchestra.run.smithi169.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-16T00:13:59.085 INFO:teuthology.orchestra.run.smithi112.stdout:(1/24): ceph-base-17.2.7-1633.g6d025bc2.el9.x86 9.1 MB/s | 5.0 MB 00:00 2024-09-16T00:13:59.143 INFO:teuthology.orchestra.run.smithi112.stdout:(2/24): ceph-selinux-17.2.7-1633.g6d025bc2.el9. 432 kB/s | 25 kB 00:00 2024-09-16T00:13:59.252 INFO:teuthology.orchestra.run.smithi112.stdout:(3/24): libcephfs2-17.2.7-1633.g6d025bc2.el9.x8 5.8 MB/s | 637 kB 00:00 2024-09-16T00:13:59.360 INFO:teuthology.orchestra.run.smithi112.stdout:(4/24): libradosstriper1-17.2.7-1633.g6d025bc2. 4.1 MB/s | 457 kB 00:00 2024-09-16T00:13:59.561 INFO:teuthology.orchestra.run.smithi112.stdout:(5/24): ceph-radosgw-17.2.7-1633.g6d025bc2.el9. 14 MB/s | 14 MB 00:01 2024-09-16T00:13:59.786 INFO:teuthology.orchestra.run.smithi112.stdout:(6/24): ceph-common-17.2.7-1633.g6d025bc2.el9.x 18 MB/s | 22 MB 00:01 2024-09-16T00:13:59.811 INFO:teuthology.orchestra.run.smithi112.stdout:(7/24): python3-ceph-argparse-17.2.7-1633.g6d02 180 kB/s | 45 kB 00:00 2024-09-16T00:13:59.836 INFO:teuthology.orchestra.run.smithi112.stdout:(8/24): python3-ceph-common-17.2.7-1633.g6d025b 2.4 MB/s | 120 kB 00:00 2024-09-16T00:13:59.862 INFO:teuthology.orchestra.run.smithi112.stdout:(9/24): python3-cephfs-17.2.7-1633.g6d025bc2.el 3.0 MB/s | 156 kB 00:00 2024-09-16T00:13:59.887 INFO:teuthology.orchestra.run.smithi112.stdout:(10/24): python3-rados-17.2.7-1633.g6d025bc2.el 6.2 MB/s | 320 kB 00:00 2024-09-16T00:13:59.912 INFO:teuthology.orchestra.run.smithi112.stdout:(11/24): python3-rbd-17.2.7-1633.g6d025bc2.el9. 5.8 MB/s | 295 kB 00:00 2024-09-16T00:13:59.946 INFO:teuthology.orchestra.run.smithi112.stdout:(12/24): python3-rgw-17.2.7-1633.g6d025bc2.el9. 1.7 MB/s | 99 kB 00:00 2024-09-16T00:14:00.021 INFO:teuthology.orchestra.run.smithi112.stdout:(13/24): librgw2-17.2.7-1633.g6d025bc2.el9.x86_ 7.4 MB/s | 4.9 MB 00:00 2024-09-16T00:14:00.113 INFO:teuthology.orchestra.run.smithi112.stdout:(14/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 271 kB/s | 45 kB 00:00 2024-09-16T00:14:00.155 INFO:teuthology.orchestra.run.smithi112.stdout:(15/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 661 kB/s | 160 kB 00:00 2024-09-16T00:14:00.205 INFO:teuthology.orchestra.run.smithi112.stdout:(16/24): python3-prettytable-0.7.2-27.el9.noarc 847 kB/s | 42 kB 00:00 2024-09-16T00:14:00.239 INFO:teuthology.orchestra.run.smithi112.stdout:(17/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.3 MB/s | 292 kB 00:00 2024-09-16T00:14:00.281 INFO:teuthology.orchestra.run.smithi112.stdout:(18/24): librdkafka-1.6.1-102.el9.x86_64.rpm 2.5 MB/s | 662 kB 00:00 2024-09-16T00:14:00.331 INFO:teuthology.orchestra.run.smithi112.stdout:(19/24): liboath-2.6.7-2.el9.x86_64.rpm 571 kB/s | 49 kB 00:00 2024-09-16T00:14:00.356 INFO:teuthology.orchestra.run.smithi112.stdout:(20/24): gperftools-libs-2.9.1-3.el9.x86_64.rpm 2.0 MB/s | 308 kB 00:00 2024-09-16T00:14:00.382 INFO:teuthology.orchestra.run.smithi112.stdout:(21/24): libunwind-1.6.2-1.el9.x86_64.rpm 681 kB/s | 67 kB 00:00 2024-09-16T00:14:00.432 INFO:teuthology.orchestra.run.smithi112.stdout:(22/24): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-16T00:14:00.541 INFO:teuthology.orchestra.run.smithi112.stdout:(23/24): librados2-17.2.7-1633.g6d025bc2.el9.x8 17 MB/s | 3.1 MB 00:00 2024-09-16T00:14:00.616 INFO:teuthology.orchestra.run.smithi112.stdout:(24/24): librbd1-17.2.7-1633.g6d025bc2.el9.x86_ 13 MB/s | 2.9 MB 00:00 2024-09-16T00:14:00.620 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2024-09-16T00:14:00.620 INFO:teuthology.orchestra.run.smithi112.stdout:Total 24 MB/s | 58 MB 00:02 2024-09-16T00:14:00.680 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2024-09-16T00:14:00.712 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2024-09-16T00:14:00.712 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2024-09-16T00:14:01.365 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2024-09-16T00:14:01.366 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2024-09-16T00:14:01.401 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-09-16T00:14:01.404 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T00:14:01.404 INFO:teuthology.orchestra.run.smithi169.stdout: Package Arch Version Repository Size 2024-09-16T00:14:01.404 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T00:14:01.404 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2024-09-16T00:14:01.404 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-radosgw x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 14 M 2024-09-16T00:14:01.404 INFO:teuthology.orchestra.run.smithi169.stdout:Upgrading: 2024-09-16T00:14:01.404 INFO:teuthology.orchestra.run.smithi169.stdout: librados2 x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 3.1 M 2024-09-16T00:14:01.405 INFO:teuthology.orchestra.run.smithi169.stdout: librbd1 x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 2.9 M 2024-09-16T00:14:01.405 INFO:teuthology.orchestra.run.smithi169.stdout:Installing dependencies: 2024-09-16T00:14:01.405 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-base x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 5.0 M 2024-09-16T00:14:01.405 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-common x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 22 M 2024-09-16T00:14:01.405 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-selinux x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 25 k 2024-09-16T00:14:01.405 INFO:teuthology.orchestra.run.smithi169.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-16T00:14:01.405 INFO:teuthology.orchestra.run.smithi169.stdout: libcephfs2 x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 637 k 2024-09-16T00:14:01.405 INFO:teuthology.orchestra.run.smithi169.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-16T00:14:01.405 INFO:teuthology.orchestra.run.smithi169.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-16T00:14:01.405 INFO:teuthology.orchestra.run.smithi169.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-16T00:14:01.405 INFO:teuthology.orchestra.run.smithi169.stdout: libradosstriper1 x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 457 k 2024-09-16T00:14:01.405 INFO:teuthology.orchestra.run.smithi169.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-16T00:14:01.405 INFO:teuthology.orchestra.run.smithi169.stdout: librgw2 x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 4.9 M 2024-09-16T00:14:01.405 INFO:teuthology.orchestra.run.smithi169.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-16T00:14:01.406 INFO:teuthology.orchestra.run.smithi169.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-16T00:14:01.406 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-argparse x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 45 k 2024-09-16T00:14:01.406 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-common x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 120 k 2024-09-16T00:14:01.406 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cephfs x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 156 k 2024-09-16T00:14:01.406 INFO:teuthology.orchestra.run.smithi169.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-16T00:14:01.406 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rados x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 320 k 2024-09-16T00:14:01.406 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rbd x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 295 k 2024-09-16T00:14:01.406 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rgw x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 99 k 2024-09-16T00:14:01.406 INFO:teuthology.orchestra.run.smithi169.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-16T00:14:01.406 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:14:01.406 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2024-09-16T00:14:01.406 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T00:14:01.406 INFO:teuthology.orchestra.run.smithi169.stdout:Install 22 Packages 2024-09-16T00:14:01.406 INFO:teuthology.orchestra.run.smithi169.stdout:Upgrade 2 Packages 2024-09-16T00:14:01.407 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:14:01.407 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 58 M 2024-09-16T00:14:01.407 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2024-09-16T00:14:02.291 INFO:teuthology.orchestra.run.smithi169.stdout:(1/24): ceph-base-17.2.7-1633.g6d025bc2.el9.x86 11 MB/s | 5.0 MB 00:00 2024-09-16T00:14:02.341 INFO:teuthology.orchestra.run.smithi169.stdout:(2/24): ceph-selinux-17.2.7-1633.g6d025bc2.el9. 506 kB/s | 25 kB 00:00 2024-09-16T00:14:02.441 INFO:teuthology.orchestra.run.smithi169.stdout:(3/24): libcephfs2-17.2.7-1633.g6d025bc2.el9.x8 6.2 MB/s | 637 kB 00:00 2024-09-16T00:14:02.541 INFO:teuthology.orchestra.run.smithi169.stdout:(4/24): libradosstriper1-17.2.7-1633.g6d025bc2. 4.5 MB/s | 457 kB 00:00 2024-09-16T00:14:02.612 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2024-09-16T00:14:02.917 INFO:teuthology.orchestra.run.smithi169.stdout:(5/24): ceph-radosgw-17.2.7-1633.g6d025bc2.el9. 14 MB/s | 14 MB 00:01 2024-09-16T00:14:02.986 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-09-16T00:14:02.992 INFO:teuthology.orchestra.run.smithi169.stdout:(6/24): librgw2-17.2.7-1633.g6d025bc2.el9.x86_6 11 MB/s | 4.9 MB 00:00 2024-09-16T00:14:03.017 INFO:teuthology.orchestra.run.smithi169.stdout:(7/24): python3-ceph-argparse-17.2.7-1633.g6d02 451 kB/s | 45 kB 00:00 2024-09-16T00:14:03.042 INFO:teuthology.orchestra.run.smithi169.stdout:(8/24): python3-ceph-common-17.2.7-1633.g6d025b 2.3 MB/s | 120 kB 00:00 2024-09-16T00:14:03.068 INFO:teuthology.orchestra.run.smithi169.stdout:(9/24): python3-cephfs-17.2.7-1633.g6d025bc2.el 3.0 MB/s | 156 kB 00:00 2024-09-16T00:14:03.070 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : librados2-2:17.2.7-1633.g6d025bc2.el9.x86_64 2/26 2024-09-16T00:14:03.093 INFO:teuthology.orchestra.run.smithi169.stdout:(10/24): python3-rados-17.2.7-1633.g6d025bc2.el 6.2 MB/s | 320 kB 00:00 2024-09-16T00:14:03.118 INFO:teuthology.orchestra.run.smithi169.stdout:(11/24): python3-rbd-17.2.7-1633.g6d025bc2.el9. 5.7 MB/s | 295 kB 00:00 2024-09-16T00:14:03.144 INFO:teuthology.orchestra.run.smithi169.stdout:(12/24): python3-rgw-17.2.7-1633.g6d025bc2.el9. 1.9 MB/s | 99 kB 00:00 2024-09-16T00:14:03.168 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librados2-2:17.2.7-1633.g6d025bc2.el9.x86_64 2/26 2024-09-16T00:14:03.255 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rados-2:17.2.7-1633.g6d025bc2.el9.x86_64 3/26 2024-09-16T00:14:03.286 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libcephfs2-2:17.2.7-1633.g6d025bc2.el9.x86_64 4/26 2024-09-16T00:14:03.361 INFO:teuthology.orchestra.run.smithi169.stdout:(13/24): ceph-common-17.2.7-1633.g6d025bc2.el9. 15 MB/s | 22 MB 00:01 2024-09-16T00:14:03.373 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libcephfs2-2:17.2.7-1633.g6d025bc2.el9.x86_64 4/26 2024-09-16T00:14:03.420 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-09-16T00:14:03.449 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-09-16T00:14:03.469 INFO:teuthology.orchestra.run.smithi169.stdout:(14/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 139 kB/s | 45 kB 00:00 2024-09-16T00:14:03.503 INFO:teuthology.orchestra.run.smithi169.stdout:(15/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 417 kB/s | 160 kB 00:00 2024-09-16T00:14:03.553 INFO:teuthology.orchestra.run.smithi169.stdout:(16/24): python3-prettytable-0.7.2-27.el9.noarc 849 kB/s | 42 kB 00:00 2024-09-16T00:14:03.578 INFO:teuthology.orchestra.run.smithi169.stdout:(17/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-09-16T00:14:03.629 INFO:teuthology.orchestra.run.smithi169.stdout:(18/24): librdkafka-1.6.1-102.el9.x86_64.rpm 2.4 MB/s | 662 kB 00:00 2024-09-16T00:14:03.662 INFO:teuthology.orchestra.run.smithi169.stdout:(19/24): liboath-2.6.7-2.el9.x86_64.rpm 646 kB/s | 49 kB 00:00 2024-09-16T00:14:03.688 INFO:teuthology.orchestra.run.smithi169.stdout:(20/24): gperftools-libs-2.9.1-3.el9.x86_64.rpm 2.2 MB/s | 308 kB 00:00 2024-09-16T00:14:03.713 INFO:teuthology.orchestra.run.smithi169.stdout:(21/24): libunwind-1.6.2-1.el9.x86_64.rpm 824 kB/s | 67 kB 00:00 2024-09-16T00:14:03.763 INFO:teuthology.orchestra.run.smithi169.stdout:(22/24): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-16T00:14:03.872 INFO:teuthology.orchestra.run.smithi169.stdout:(23/24): librados2-17.2.7-1633.g6d025bc2.el9.x8 17 MB/s | 3.1 MB 00:00 2024-09-16T00:14:03.886 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-09-16T00:14:03.905 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librgw2-2:17.2.7-1633.g6d025bc2.el9.x86_64 8/26 2024-09-16T00:14:03.938 INFO:teuthology.orchestra.run.smithi169.stdout:(24/24): librbd1-17.2.7-1633.g6d025bc2.el9.x86_ 13 MB/s | 2.9 MB 00:00 2024-09-16T00:14:03.945 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2024-09-16T00:14:03.945 INFO:teuthology.orchestra.run.smithi169.stdout:Total 23 MB/s | 58 MB 00:02 2024-09-16T00:14:03.951 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librgw2-2:17.2.7-1633.g6d025bc2.el9.x86_64 8/26 2024-09-16T00:14:03.996 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-09-16T00:14:04.011 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2024-09-16T00:14:04.033 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-ceph-argparse-2:17.2.7-1633.g6d025bc2.el9. 10/26 2024-09-16T00:14:04.045 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2024-09-16T00:14:04.045 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2024-09-16T00:14:04.067 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cephfs-2:17.2.7-1633.g6d025bc2.el9.x86_64 11/26 2024-09-16T00:14:04.132 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rgw-2:17.2.7-1633.g6d025bc2.el9.x86_64 12/26 2024-09-16T00:14:04.158 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libradosstriper1-2:17.2.7-1633.g6d025bc2.el9.x86_6 13/26 2024-09-16T00:14:04.224 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libradosstriper1-2:17.2.7-1633.g6d025bc2.el9.x86_6 13/26 2024-09-16T00:14:04.272 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-09-16T00:14:04.324 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 15/26 2024-09-16T00:14:04.357 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-09-16T00:14:04.660 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-09-16T00:14:04.703 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2024-09-16T00:14:04.703 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2024-09-16T00:14:04.711 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : librbd1-2:17.2.7-1633.g6d025bc2.el9.x86_64 18/26 2024-09-16T00:14:04.802 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librbd1-2:17.2.7-1633.g6d025bc2.el9.x86_64 18/26 2024-09-16T00:14:04.874 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rbd-2:17.2.7-1633.g6d025bc2.el9.x86_64 19/26 2024-09-16T00:14:04.905 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-ceph-common-2:17.2.7-1633.g6d025bc2.el9.x8 20/26 2024-09-16T00:14:05.915 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2024-09-16T00:14:06.322 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-09-16T00:14:06.374 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : librados2-2:17.2.7-1633.g6d025bc2.el9.x86_64 2/26 2024-09-16T00:14:06.470 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librados2-2:17.2.7-1633.g6d025bc2.el9.x86_64 2/26 2024-09-16T00:14:06.553 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-rados-2:17.2.7-1633.g6d025bc2.el9.x86_64 3/26 2024-09-16T00:14:06.575 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libcephfs2-2:17.2.7-1633.g6d025bc2.el9.x86_64 4/26 2024-09-16T00:14:06.664 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libcephfs2-2:17.2.7-1633.g6d025bc2.el9.x86_64 4/26 2024-09-16T00:14:06.715 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-09-16T00:14:06.747 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-09-16T00:14:07.195 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-09-16T00:14:07.220 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : librgw2-2:17.2.7-1633.g6d025bc2.el9.x86_64 8/26 2024-09-16T00:14:07.284 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librgw2-2:17.2.7-1633.g6d025bc2.el9.x86_64 8/26 2024-09-16T00:14:07.321 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-09-16T00:14:07.328 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-common-2:17.2.7-1633.g6d025bc2.el9.x86_64 21/26 2024-09-16T00:14:07.363 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-common-2:17.2.7-1633.g6d025bc2.el9.x86_64 21/26 2024-09-16T00:14:07.365 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-ceph-argparse-2:17.2.7-1633.g6d025bc2.el9. 10/26 2024-09-16T00:14:07.404 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cephfs-2:17.2.7-1633.g6d025bc2.el9.x86_64 11/26 2024-09-16T00:14:07.463 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-rgw-2:17.2.7-1633.g6d025bc2.el9.x86_64 12/26 2024-09-16T00:14:07.501 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libradosstriper1-2:17.2.7-1633.g6d025bc2.el9.x86_6 13/26 2024-09-16T00:14:07.555 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libradosstriper1-2:17.2.7-1633.g6d025bc2.el9.x86_6 13/26 2024-09-16T00:14:07.606 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-09-16T00:14:07.639 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 15/26 2024-09-16T00:14:07.672 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-09-16T00:14:07.888 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-common-2:17.2.7-1633.g6d025bc2.el9.x86_64 21/26 2024-09-16T00:14:07.930 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-base-2:17.2.7-1633.g6d025bc2.el9.x86_64 22/26 2024-09-16T00:14:07.970 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-09-16T00:14:08.010 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : librbd1-2:17.2.7-1633.g6d025bc2.el9.x86_64 18/26 2024-09-16T00:14:08.031 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-base-2:17.2.7-1633.g6d025bc2.el9.x86_64 22/26 2024-09-16T00:14:08.031 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-16T00:14:08.031 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-16T00:14:08.031 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:14:08.064 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-selinux-2:17.2.7-1633.g6d025bc2.el9.x86_64 23/26 2024-09-16T00:14:08.099 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librbd1-2:17.2.7-1633.g6d025bc2.el9.x86_64 18/26 2024-09-16T00:14:08.174 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-rbd-2:17.2.7-1633.g6d025bc2.el9.x86_64 19/26 2024-09-16T00:14:08.204 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-ceph-common-2:17.2.7-1633.g6d025bc2.el9.x8 20/26 2024-09-16T00:14:10.621 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-common-2:17.2.7-1633.g6d025bc2.el9.x86_64 21/26 2024-09-16T00:14:10.658 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-common-2:17.2.7-1633.g6d025bc2.el9.x86_64 21/26 2024-09-16T00:14:11.183 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-common-2:17.2.7-1633.g6d025bc2.el9.x86_64 21/26 2024-09-16T00:14:11.220 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-base-2:17.2.7-1633.g6d025bc2.el9.x86_64 22/26 2024-09-16T00:14:11.303 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-base-2:17.2.7-1633.g6d025bc2.el9.x86_64 22/26 2024-09-16T00:14:11.303 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-16T00:14:11.303 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-16T00:14:11.303 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:14:11.337 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-selinux-2:17.2.7-1633.g6d025bc2.el9.x86_64 23/26 2024-09-16T00:14:17.960 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-selinux-2:17.2.7-1633.g6d025bc2.el9.x86_64 23/26 2024-09-16T00:14:17.960 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /sys 2024-09-16T00:14:17.961 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /proc 2024-09-16T00:14:17.961 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /mnt 2024-09-16T00:14:17.961 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /var/tmp 2024-09-16T00:14:17.961 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /home 2024-09-16T00:14:17.961 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /root 2024-09-16T00:14:17.961 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /tmp 2024-09-16T00:14:17.961 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:14:19.277 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-radosgw-2:17.2.7-1633.g6d025bc2.el9.x86_64 24/26 2024-09-16T00:14:19.353 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-1633.g6d025bc2.el9.x86_64 24/26 2024-09-16T00:14:19.353 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:14:19.353 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-16T00:14:19.353 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T00:14:19.353 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T00:14:19.353 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:14:19.355 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-16T00:14:19.430 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-16T00:14:19.430 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-16T00:14:20.788 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-16T00:14:20.788 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-base-2:17.2.7-1633.g6d025bc2.el9.x86_64 1/26 2024-09-16T00:14:20.788 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-common-2:17.2.7-1633.g6d025bc2.el9.x86_64 2/26 2024-09-16T00:14:20.788 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-radosgw-2:17.2.7-1633.g6d025bc2.el9.x86_64 3/26 2024-09-16T00:14:20.788 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-selinux-2:17.2.7-1633.g6d025bc2.el9.x86_64 4/26 2024-09-16T00:14:20.788 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libcephfs2-2:17.2.7-1633.g6d025bc2.el9.x86_64 5/26 2024-09-16T00:14:20.788 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libradosstriper1-2:17.2.7-1633.g6d025bc2.el9.x86_6 6/26 2024-09-16T00:14:20.789 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librgw2-2:17.2.7-1633.g6d025bc2.el9.x86_64 7/26 2024-09-16T00:14:20.789 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-ceph-argparse-2:17.2.7-1633.g6d025bc2.el9. 8/26 2024-09-16T00:14:20.789 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-ceph-common-2:17.2.7-1633.g6d025bc2.el9.x8 9/26 2024-09-16T00:14:20.789 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cephfs-2:17.2.7-1633.g6d025bc2.el9.x86_64 10/26 2024-09-16T00:14:20.789 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rados-2:17.2.7-1633.g6d025bc2.el9.x86_64 11/26 2024-09-16T00:14:20.789 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rbd-2:17.2.7-1633.g6d025bc2.el9.x86_64 12/26 2024-09-16T00:14:20.789 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rgw-2:17.2.7-1633.g6d025bc2.el9.x86_64 13/26 2024-09-16T00:14:20.789 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-09-16T00:14:20.789 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-09-16T00:14:20.789 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-09-16T00:14:20.789 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-09-16T00:14:20.789 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-09-16T00:14:20.789 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 19/26 2024-09-16T00:14:20.789 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-09-16T00:14:20.790 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-09-16T00:14:20.790 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-09-16T00:14:20.790 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librados2-2:17.2.7-1633.g6d025bc2.el9.x86_64 23/26 2024-09-16T00:14:20.790 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-09-16T00:14:20.790 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librbd1-2:17.2.7-1633.g6d025bc2.el9.x86_64 25/26 2024-09-16T00:14:21.135 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-selinux-2:17.2.7-1633.g6d025bc2.el9.x86_64 23/26 2024-09-16T00:14:21.135 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /sys 2024-09-16T00:14:21.135 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /proc 2024-09-16T00:14:21.136 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /mnt 2024-09-16T00:14:21.136 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /var/tmp 2024-09-16T00:14:21.136 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /home 2024-09-16T00:14:21.136 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /root 2024-09-16T00:14:21.136 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /tmp 2024-09-16T00:14:21.136 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:14:21.638 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-09-16T00:14:21.638 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:14:21.638 INFO:teuthology.orchestra.run.smithi112.stdout:Upgraded: 2024-09-16T00:14:21.638 INFO:teuthology.orchestra.run.smithi112.stdout: librados2-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-16T00:14:21.639 INFO:teuthology.orchestra.run.smithi112.stdout: librbd1-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-16T00:14:21.639 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2024-09-16T00:14:21.639 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-base-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-16T00:14:21.639 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-common-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-16T00:14:21.639 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-radosgw-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-16T00:14:21.639 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-selinux-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-16T00:14:21.639 INFO:teuthology.orchestra.run.smithi112.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-16T00:14:21.639 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs2-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-16T00:14:21.639 INFO:teuthology.orchestra.run.smithi112.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-16T00:14:21.639 INFO:teuthology.orchestra.run.smithi112.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-16T00:14:21.639 INFO:teuthology.orchestra.run.smithi112.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-16T00:14:21.639 INFO:teuthology.orchestra.run.smithi112.stdout: libradosstriper1-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-16T00:14:21.639 INFO:teuthology.orchestra.run.smithi112.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-16T00:14:21.639 INFO:teuthology.orchestra.run.smithi112.stdout: librgw2-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-16T00:14:21.640 INFO:teuthology.orchestra.run.smithi112.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-16T00:14:21.640 INFO:teuthology.orchestra.run.smithi112.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-16T00:14:21.640 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-argparse-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-16T00:14:21.640 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-common-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-16T00:14:21.640 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cephfs-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-16T00:14:21.640 INFO:teuthology.orchestra.run.smithi112.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-16T00:14:21.640 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rados-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-16T00:14:21.640 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rbd-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-16T00:14:21.640 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rgw-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-16T00:14:21.640 INFO:teuthology.orchestra.run.smithi112.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-16T00:14:21.640 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:14:21.640 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2024-09-16T00:14:21.936 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install ceph-test 2024-09-16T00:14:22.437 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-radosgw-2:17.2.7-1633.g6d025bc2.el9.x86_64 24/26 2024-09-16T00:14:22.509 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:00:27 ago on Mon 16 Sep 2024 12:13:55 AM UTC. 2024-09-16T00:14:22.515 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-1633.g6d025bc2.el9.x86_64 24/26 2024-09-16T00:14:22.515 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:14:22.515 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-16T00:14:22.515 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T00:14:22.515 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T00:14:22.515 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:14:22.517 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-16T00:14:22.606 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-16T00:14:22.606 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-16T00:14:22.651 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2024-09-16T00:14:22.652 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-09-16T00:14:22.652 INFO:teuthology.orchestra.run.smithi112.stdout: Package Arch Version Repository Size 2024-09-16T00:14:22.652 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-09-16T00:14:22.652 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2024-09-16T00:14:22.652 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-test x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 48 M 2024-09-16T00:14:22.652 INFO:teuthology.orchestra.run.smithi112.stdout:Installing dependencies: 2024-09-16T00:14:22.652 INFO:teuthology.orchestra.run.smithi112.stdout: libcephsqlite x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 163 k 2024-09-16T00:14:22.652 INFO:teuthology.orchestra.run.smithi112.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-16T00:14:22.652 INFO:teuthology.orchestra.run.smithi112.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-16T00:14:22.652 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:14:22.652 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2024-09-16T00:14:22.652 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-09-16T00:14:22.652 INFO:teuthology.orchestra.run.smithi112.stdout:Install 4 Packages 2024-09-16T00:14:22.653 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:14:22.653 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 48 M 2024-09-16T00:14:22.653 INFO:teuthology.orchestra.run.smithi112.stdout:Installed size: 193 M 2024-09-16T00:14:22.653 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2024-09-16T00:14:22.837 INFO:teuthology.orchestra.run.smithi112.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 2.3 MB/s | 304 kB 00:00 2024-09-16T00:14:22.878 INFO:teuthology.orchestra.run.smithi112.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-09-16T00:14:22.937 INFO:teuthology.orchestra.run.smithi112.stdout:(3/4): libcephsqlite-17.2.7-1633.g6d025bc2.el9. 704 kB/s | 163 kB 00:00 2024-09-16T00:14:23.856 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-16T00:14:23.856 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-base-2:17.2.7-1633.g6d025bc2.el9.x86_64 1/26 2024-09-16T00:14:23.856 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-common-2:17.2.7-1633.g6d025bc2.el9.x86_64 2/26 2024-09-16T00:14:23.856 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-radosgw-2:17.2.7-1633.g6d025bc2.el9.x86_64 3/26 2024-09-16T00:14:23.856 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-selinux-2:17.2.7-1633.g6d025bc2.el9.x86_64 4/26 2024-09-16T00:14:23.856 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libcephfs2-2:17.2.7-1633.g6d025bc2.el9.x86_64 5/26 2024-09-16T00:14:23.857 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libradosstriper1-2:17.2.7-1633.g6d025bc2.el9.x86_6 6/26 2024-09-16T00:14:23.857 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librgw2-2:17.2.7-1633.g6d025bc2.el9.x86_64 7/26 2024-09-16T00:14:23.857 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-ceph-argparse-2:17.2.7-1633.g6d025bc2.el9. 8/26 2024-09-16T00:14:23.857 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-ceph-common-2:17.2.7-1633.g6d025bc2.el9.x8 9/26 2024-09-16T00:14:23.857 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cephfs-2:17.2.7-1633.g6d025bc2.el9.x86_64 10/26 2024-09-16T00:14:23.857 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-rados-2:17.2.7-1633.g6d025bc2.el9.x86_64 11/26 2024-09-16T00:14:23.857 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-rbd-2:17.2.7-1633.g6d025bc2.el9.x86_64 12/26 2024-09-16T00:14:23.857 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-rgw-2:17.2.7-1633.g6d025bc2.el9.x86_64 13/26 2024-09-16T00:14:23.857 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-09-16T00:14:23.857 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-09-16T00:14:23.857 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-09-16T00:14:23.857 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-09-16T00:14:23.857 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-09-16T00:14:23.857 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 19/26 2024-09-16T00:14:23.858 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-09-16T00:14:23.858 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-09-16T00:14:23.858 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-09-16T00:14:23.858 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librados2-2:17.2.7-1633.g6d025bc2.el9.x86_64 23/26 2024-09-16T00:14:23.858 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-09-16T00:14:23.858 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librbd1-2:17.2.7-1633.g6d025bc2.el9.x86_64 25/26 2024-09-16T00:14:24.713 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-09-16T00:14:24.714 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:14:24.714 INFO:teuthology.orchestra.run.smithi169.stdout:Upgraded: 2024-09-16T00:14:24.714 INFO:teuthology.orchestra.run.smithi169.stdout: librados2-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-16T00:14:24.714 INFO:teuthology.orchestra.run.smithi169.stdout: librbd1-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-16T00:14:24.714 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2024-09-16T00:14:24.714 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-base-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-16T00:14:24.714 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-common-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-16T00:14:24.714 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-radosgw-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-16T00:14:24.714 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-selinux-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-16T00:14:24.714 INFO:teuthology.orchestra.run.smithi169.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-16T00:14:24.714 INFO:teuthology.orchestra.run.smithi169.stdout: libcephfs2-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-16T00:14:24.714 INFO:teuthology.orchestra.run.smithi169.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-16T00:14:24.715 INFO:teuthology.orchestra.run.smithi169.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-16T00:14:24.715 INFO:teuthology.orchestra.run.smithi169.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-16T00:14:24.715 INFO:teuthology.orchestra.run.smithi169.stdout: libradosstriper1-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-16T00:14:24.715 INFO:teuthology.orchestra.run.smithi169.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-16T00:14:24.715 INFO:teuthology.orchestra.run.smithi169.stdout: librgw2-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-16T00:14:24.715 INFO:teuthology.orchestra.run.smithi169.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-16T00:14:24.715 INFO:teuthology.orchestra.run.smithi169.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-16T00:14:24.715 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-argparse-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-16T00:14:24.715 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-common-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-16T00:14:24.715 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cephfs-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-16T00:14:24.715 INFO:teuthology.orchestra.run.smithi169.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-16T00:14:24.715 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rados-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-16T00:14:24.715 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rbd-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-16T00:14:24.715 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rgw-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-16T00:14:24.716 INFO:teuthology.orchestra.run.smithi169.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-16T00:14:24.716 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:14:24.716 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-09-16T00:14:25.099 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install ceph-test 2024-09-16T00:14:25.120 INFO:teuthology.orchestra.run.smithi112.stdout:(4/4): ceph-test-17.2.7-1633.g6d025bc2.el9.x86_ 20 MB/s | 48 MB 00:02 2024-09-16T00:14:25.122 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2024-09-16T00:14:25.123 INFO:teuthology.orchestra.run.smithi112.stdout:Total 20 MB/s | 48 MB 00:02 2024-09-16T00:14:25.151 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2024-09-16T00:14:25.170 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2024-09-16T00:14:25.170 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2024-09-16T00:14:25.538 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2024-09-16T00:14:25.539 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2024-09-16T00:14:25.676 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:00:26 ago on Mon 16 Sep 2024 12:13:59 AM UTC. 2024-09-16T00:14:25.810 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-09-16T00:14:25.810 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T00:14:25.810 INFO:teuthology.orchestra.run.smithi169.stdout: Package Arch Version Repository Size 2024-09-16T00:14:25.810 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T00:14:25.810 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2024-09-16T00:14:25.811 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-test x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 48 M 2024-09-16T00:14:25.811 INFO:teuthology.orchestra.run.smithi169.stdout:Installing dependencies: 2024-09-16T00:14:25.811 INFO:teuthology.orchestra.run.smithi169.stdout: libcephsqlite x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 163 k 2024-09-16T00:14:25.811 INFO:teuthology.orchestra.run.smithi169.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-16T00:14:25.811 INFO:teuthology.orchestra.run.smithi169.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-16T00:14:25.811 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:14:25.811 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2024-09-16T00:14:25.811 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T00:14:25.811 INFO:teuthology.orchestra.run.smithi169.stdout:Install 4 Packages 2024-09-16T00:14:25.811 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:14:25.812 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 48 M 2024-09-16T00:14:25.812 INFO:teuthology.orchestra.run.smithi169.stdout:Installed size: 193 M 2024-09-16T00:14:25.812 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2024-09-16T00:14:26.064 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2024-09-16T00:14:26.138 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-16T00:14:26.172 INFO:teuthology.orchestra.run.smithi169.stdout:(1/4): libcephsqlite-17.2.7-1633.g6d025bc2.el9. 705 kB/s | 163 kB 00:00 2024-09-16T00:14:26.184 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-16T00:14:26.222 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libcephsqlite-2:17.2.7-1633.g6d025bc2.el9.x86_64 3/4 2024-09-16T00:14:26.805 INFO:teuthology.orchestra.run.smithi169.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 102 kB/s | 64 kB 00:00 2024-09-16T00:14:26.830 INFO:teuthology.orchestra.run.smithi169.stdout:(3/4): socat-1.7.4.1-6.el9.x86_64.rpm 342 kB/s | 304 kB 00:00 2024-09-16T00:14:30.628 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libcephsqlite-2:17.2.7-1633.g6d025bc2.el9.x86_64 3/4 2024-09-16T00:14:30.702 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-test-2:17.2.7-1633.g6d025bc2.el9.x86_64 4/4 2024-09-16T00:14:32.488 INFO:teuthology.orchestra.run.smithi169.stdout:(4/4): ceph-test-17.2.7-1633.g6d025bc2.el9.x86_ 7.3 MB/s | 48 MB 00:06 2024-09-16T00:14:32.490 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2024-09-16T00:14:32.490 INFO:teuthology.orchestra.run.smithi169.stdout:Total 7.3 MB/s | 48 MB 00:06 2024-09-16T00:14:32.515 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2024-09-16T00:14:32.533 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2024-09-16T00:14:32.533 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2024-09-16T00:14:32.827 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-test-2:17.2.7-1633.g6d025bc2.el9.x86_64 4/4 2024-09-16T00:14:32.828 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-test-2:17.2.7-1633.g6d025bc2.el9.x86_64 1/4 2024-09-16T00:14:32.828 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libcephsqlite-2:17.2.7-1633.g6d025bc2.el9.x86_64 2/4 2024-09-16T00:14:32.828 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-16T00:14:32.902 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2024-09-16T00:14:32.902 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2024-09-16T00:14:33.135 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-16T00:14:33.135 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:14:33.136 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2024-09-16T00:14:33.136 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-test-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-16T00:14:33.136 INFO:teuthology.orchestra.run.smithi112.stdout: libcephsqlite-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-16T00:14:33.136 INFO:teuthology.orchestra.run.smithi112.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-16T00:14:33.136 INFO:teuthology.orchestra.run.smithi112.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-16T00:14:33.136 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:14:33.136 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2024-09-16T00:14:33.427 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install ceph 2024-09-16T00:14:33.432 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2024-09-16T00:14:33.516 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-16T00:14:33.560 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-16T00:14:33.590 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libcephsqlite-2:17.2.7-1633.g6d025bc2.el9.x86_64 3/4 2024-09-16T00:14:33.993 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:00:38 ago on Mon 16 Sep 2024 12:13:55 AM UTC. 2024-09-16T00:14:34.144 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2024-09-16T00:14:34.146 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-09-16T00:14:34.147 INFO:teuthology.orchestra.run.smithi112.stdout: Package Arch Version Repository Size 2024-09-16T00:14:34.147 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-09-16T00:14:34.147 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2024-09-16T00:14:34.147 INFO:teuthology.orchestra.run.smithi112.stdout: ceph x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 6.5 k 2024-09-16T00:14:34.147 INFO:teuthology.orchestra.run.smithi112.stdout:Installing dependencies: 2024-09-16T00:14:34.147 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mds x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 2.1 M 2024-09-16T00:14:34.147 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 1.4 M 2024-09-16T00:14:34.147 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-modules-core noarch 2:17.2.7-1633.g6d025bc2.el9 ceph-noarch 240 k 2024-09-16T00:14:34.148 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mon x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 3.9 M 2024-09-16T00:14:34.148 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-osd x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 15 M 2024-09-16T00:14:34.148 INFO:teuthology.orchestra.run.smithi112.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-16T00:14:34.148 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-16T00:14:34.148 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-16T00:14:34.148 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-16T00:14:34.148 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-16T00:14:34.148 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-16T00:14:34.148 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-16T00:14:34.148 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-collections 2024-09-16T00:14:34.148 INFO:teuthology.orchestra.run.smithi112.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-16T00:14:34.148 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-16T00:14:34.148 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-16T00:14:34.149 INFO:teuthology.orchestra.run.smithi112.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-16T00:14:34.149 INFO:teuthology.orchestra.run.smithi112.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-16T00:14:34.149 INFO:teuthology.orchestra.run.smithi112.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-16T00:14:34.149 INFO:teuthology.orchestra.run.smithi112.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-16T00:14:34.149 INFO:teuthology.orchestra.run.smithi112.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-16T00:14:34.149 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-16T00:14:34.149 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-16T00:14:34.149 INFO:teuthology.orchestra.run.smithi112.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-16T00:14:34.149 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-16T00:14:34.149 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-16T00:14:34.149 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-16T00:14:34.149 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-16T00:14:34.149 INFO:teuthology.orchestra.run.smithi112.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-16T00:14:34.149 INFO:teuthology.orchestra.run.smithi112.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-16T00:14:34.150 INFO:teuthology.orchestra.run.smithi112.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-16T00:14:34.150 INFO:teuthology.orchestra.run.smithi112.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-16T00:14:34.150 INFO:teuthology.orchestra.run.smithi112.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-16T00:14:34.150 INFO:teuthology.orchestra.run.smithi112.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-16T00:14:34.150 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:14:34.150 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2024-09-16T00:14:34.150 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-09-16T00:14:34.150 INFO:teuthology.orchestra.run.smithi112.stdout:Install 34 Packages 2024-09-16T00:14:34.150 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:14:34.151 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 27 M 2024-09-16T00:14:34.151 INFO:teuthology.orchestra.run.smithi112.stdout:Installed size: 96 M 2024-09-16T00:14:34.151 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2024-09-16T00:14:34.687 INFO:teuthology.orchestra.run.smithi112.stdout:(1/34): ceph-17.2.7-1633.g6d025bc2.el9.x86_64.r 52 kB/s | 6.5 kB 00:00 2024-09-16T00:14:34.862 INFO:teuthology.orchestra.run.smithi112.stdout:(2/34): ceph-mds-17.2.7-1633.g6d025bc2.el9.x86_ 6.9 MB/s | 2.1 MB 00:00 2024-09-16T00:14:34.896 INFO:teuthology.orchestra.run.smithi112.stdout:(3/34): ceph-mgr-17.2.7-1633.g6d025bc2.el9.x86_ 4.2 MB/s | 1.4 MB 00:00 2024-09-16T00:14:34.979 INFO:teuthology.orchestra.run.smithi112.stdout:(4/34): ceph-mgr-modules-core-17.2.7-1633.g6d02 2.8 MB/s | 240 kB 00:00 2024-09-16T00:14:35.038 INFO:teuthology.orchestra.run.smithi112.stdout:(5/34): ceph-mon-17.2.7-1633.g6d025bc2.el9.x86_ 11 MB/s | 3.9 MB 00:00 2024-09-16T00:14:35.205 INFO:teuthology.orchestra.run.smithi112.stdout:(6/34): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.1 MB/s | 253 kB 00:00 2024-09-16T00:14:35.355 INFO:teuthology.orchestra.run.smithi112.stdout:(7/34): ceph-osd-17.2.7-1633.g6d025bc2.el9.x86_ 31 MB/s | 15 MB 00:00 2024-09-16T00:14:35.389 INFO:teuthology.orchestra.run.smithi112.stdout:(8/34): python3-ply-3.11-14.el9.noarch.rpm 580 kB/s | 106 kB 00:00 2024-09-16T00:14:35.439 INFO:teuthology.orchestra.run.smithi112.stdout:(9/34): python3-requests-2.25.1-8.el9.noarch.rp 2.4 MB/s | 125 kB 00:00 2024-09-16T00:14:35.481 INFO:teuthology.orchestra.run.smithi112.stdout:(10/34): python3-cryptography-36.0.1-4.el9.x86_ 2.8 MB/s | 1.2 MB 00:00 2024-09-16T00:14:35.523 INFO:teuthology.orchestra.run.smithi112.stdout:(11/34): python3-pycparser-2.20-6.el9.noarch.rp 808 kB/s | 135 kB 00:00 2024-09-16T00:14:35.548 INFO:teuthology.orchestra.run.smithi112.stdout:(12/34): python3-urllib3-1.26.5-6.el9.noarch.rp 1.9 MB/s | 215 kB 00:00 2024-09-16T00:14:36.073 INFO:teuthology.orchestra.run.smithi112.stdout:(13/34): python3-markupsafe-1.1.1-12.el9.x86_64 63 kB/s | 35 kB 00:00 2024-09-16T00:14:36.173 INFO:teuthology.orchestra.run.smithi112.stdout:(14/34): python3-packaging-20.9-5.el9.noarch.rp 123 kB/s | 77 kB 00:00 2024-09-16T00:14:36.265 INFO:teuthology.orchestra.run.smithi112.stdout:(15/34): python3-mako-1.1.4-6.el9.noarch.rpm 220 kB/s | 172 kB 00:00 2024-09-16T00:14:36.316 INFO:teuthology.orchestra.run.smithi112.stdout:(16/34): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 873 kB/s | 43 kB 00:00 2024-09-16T00:14:36.341 INFO:teuthology.orchestra.run.smithi112.stdout:(17/34): python3-cheroot-8.6.0-4.el9.noarch.rpm 6.8 MB/s | 172 kB 00:00 2024-09-16T00:14:36.374 INFO:teuthology.orchestra.run.smithi112.stdout:(18/34): python3-cherrypy-18.6.1-2.el9.noarch.r 10 MB/s | 358 kB 00:00 2024-09-16T00:14:36.400 INFO:teuthology.orchestra.run.smithi112.stdout:(19/34): python3-jaraco-8.2.1-3.el9.noarch.rpm 441 kB/s | 11 kB 00:00 2024-09-16T00:14:36.425 INFO:teuthology.orchestra.run.smithi112.stdout:(20/34): python3-jaraco-classes-3.2.1-5.el9.noa 707 kB/s | 18 kB 00:00 2024-09-16T00:14:36.458 INFO:teuthology.orchestra.run.smithi112.stdout:(21/34): python3-jaraco-collections-3.0.0-8.el9 877 kB/s | 23 kB 00:00 2024-09-16T00:14:36.483 INFO:teuthology.orchestra.run.smithi112.stdout:(22/34): python3-jaraco-functools-3.5.0-2.el9.n 785 kB/s | 19 kB 00:00 2024-09-16T00:14:36.509 INFO:teuthology.orchestra.run.smithi112.stdout:(23/34): python3-jaraco-text-3.2.0-6.el9.noarch 786 kB/s | 20 kB 00:00 2024-09-16T00:14:36.534 INFO:teuthology.orchestra.run.smithi112.stdout:(24/34): python3-logutils-0.3.5-21.el9.noarch.r 1.8 MB/s | 46 kB 00:00 2024-09-16T00:14:36.568 INFO:teuthology.orchestra.run.smithi112.stdout:(25/34): python3-more-itertools-8.12.0-2.el9.no 2.3 MB/s | 79 kB 00:00 2024-09-16T00:14:36.593 INFO:teuthology.orchestra.run.smithi112.stdout:(26/34): python3-pecan-1.4.2-3.el9.noarch.rpm 11 MB/s | 272 kB 00:00 2024-09-16T00:14:36.618 INFO:teuthology.orchestra.run.smithi112.stdout:(27/34): python3-portend-3.1.0-2.el9.noarch.rpm 664 kB/s | 16 kB 00:00 2024-09-16T00:14:36.644 INFO:teuthology.orchestra.run.smithi112.stdout:(28/34): python3-pyOpenSSL-21.0.0-1.el9.noarch. 3.5 MB/s | 90 kB 00:00 2024-09-16T00:14:36.669 INFO:teuthology.orchestra.run.smithi112.stdout:(29/34): python3-tempora-5.0.0-2.el9.noarch.rpm 1.4 MB/s | 36 kB 00:00 2024-09-16T00:14:36.694 INFO:teuthology.orchestra.run.smithi112.stdout:(30/34): python3-pytz-2021.1-5.el9.noarch.rpm 82 kB/s | 51 kB 00:00 2024-09-16T00:14:36.728 INFO:teuthology.orchestra.run.smithi112.stdout:(31/34): python3-webob-1.8.8-2.el9.noarch.rpm 3.9 MB/s | 230 kB 00:00 2024-09-16T00:14:36.753 INFO:teuthology.orchestra.run.smithi112.stdout:(32/34): python3-zc-lockfile-2.0-10.el9.noarch. 867 kB/s | 20 kB 00:00 2024-09-16T00:14:36.786 INFO:teuthology.orchestra.run.smithi112.stdout:(33/34): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.5 MB/s | 427 kB 00:00 2024-09-16T00:14:36.811 INFO:teuthology.orchestra.run.smithi112.stdout:(34/34): python3-toml-0.10.2-6.el9.noarch.rpm 65 kB/s | 42 kB 00:00 2024-09-16T00:14:36.817 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2024-09-16T00:14:36.817 INFO:teuthology.orchestra.run.smithi112.stdout:Total 10 MB/s | 27 MB 00:02 2024-09-16T00:14:36.950 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2024-09-16T00:14:36.987 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2024-09-16T00:14:36.987 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2024-09-16T00:14:37.398 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2024-09-16T00:14:37.399 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2024-09-16T00:14:38.086 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libcephsqlite-2:17.2.7-1633.g6d025bc2.el9.x86_64 3/4 2024-09-16T00:14:38.180 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-test-2:17.2.7-1633.g6d025bc2.el9.x86_64 4/4 2024-09-16T00:14:38.683 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2024-09-16T00:14:38.740 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/34 2024-09-16T00:14:38.772 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/34 2024-09-16T00:14:38.805 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/34 2024-09-16T00:14:38.898 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/34 2024-09-16T00:14:38.976 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/34 2024-09-16T00:14:39.025 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/34 2024-09-16T00:14:39.068 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/34 2024-09-16T00:14:39.097 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/34 2024-09-16T00:14:39.132 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/34 2024-09-16T00:14:39.165 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/34 2024-09-16T00:14:39.206 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/34 2024-09-16T00:14:39.241 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/34 2024-09-16T00:14:39.270 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/34 2024-09-16T00:14:39.319 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/34 2024-09-16T00:14:39.359 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-16T00:14:39.433 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 16/34 2024-09-16T00:14:39.581 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 17/34 2024-09-16T00:14:39.668 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 18/34 2024-09-16T00:14:39.729 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 19/34 2024-09-16T00:14:39.767 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 20/34 2024-09-16T00:14:39.825 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-ply-3.11-14.el9.noarch 21/34 2024-09-16T00:14:39.888 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 22/34 2024-09-16T00:14:40.085 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 23/34 2024-09-16T00:14:40.137 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 24/34 2024-09-16T00:14:40.216 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 25/34 2024-09-16T00:14:40.305 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 26/34 2024-09-16T00:14:40.346 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/34 2024-09-16T00:14:40.461 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 28/34 2024-09-16T00:14:40.500 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-test-2:17.2.7-1633.g6d025bc2.el9.x86_64 4/4 2024-09-16T00:14:40.500 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-test-2:17.2.7-1633.g6d025bc2.el9.x86_64 1/4 2024-09-16T00:14:40.500 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libcephsqlite-2:17.2.7-1633.g6d025bc2.el9.x86_64 2/4 2024-09-16T00:14:40.500 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-16T00:14:40.615 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-1633.g6d025bc2.el9. 29/34 2024-09-16T00:14:40.638 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-2:17.2.7-1633.g6d025bc2.el9.x86_64 30/34 2024-09-16T00:14:40.664 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-2:17.2.7-1633.g6d025bc2.el9.x86_64 30/34 2024-09-16T00:14:40.664 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:14:40.664 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-16T00:14:40.664 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T00:14:40.664 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T00:14:40.664 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:14:40.800 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-16T00:14:40.800 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:14:40.800 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2024-09-16T00:14:40.800 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-test-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-16T00:14:40.800 INFO:teuthology.orchestra.run.smithi169.stdout: libcephsqlite-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-16T00:14:40.800 INFO:teuthology.orchestra.run.smithi169.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-16T00:14:40.801 INFO:teuthology.orchestra.run.smithi169.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-16T00:14:40.801 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:14:40.801 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-09-16T00:14:41.099 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install ceph 2024-09-16T00:14:41.669 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:00:42 ago on Mon 16 Sep 2024 12:13:59 AM UTC. 2024-09-16T00:14:41.814 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-09-16T00:14:41.816 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T00:14:41.816 INFO:teuthology.orchestra.run.smithi169.stdout: Package Arch Version Repository Size 2024-09-16T00:14:41.816 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T00:14:41.816 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2024-09-16T00:14:41.816 INFO:teuthology.orchestra.run.smithi169.stdout: ceph x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 6.5 k 2024-09-16T00:14:41.816 INFO:teuthology.orchestra.run.smithi169.stdout:Installing dependencies: 2024-09-16T00:14:41.816 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mds x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 2.1 M 2024-09-16T00:14:41.817 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 1.4 M 2024-09-16T00:14:41.817 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-modules-core noarch 2:17.2.7-1633.g6d025bc2.el9 ceph-noarch 240 k 2024-09-16T00:14:41.817 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mon x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 3.9 M 2024-09-16T00:14:41.817 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-osd x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 15 M 2024-09-16T00:14:41.817 INFO:teuthology.orchestra.run.smithi169.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-16T00:14:41.817 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-16T00:14:41.817 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-16T00:14:41.817 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-16T00:14:41.817 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-16T00:14:41.817 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-16T00:14:41.817 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-16T00:14:41.817 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-collections 2024-09-16T00:14:41.818 INFO:teuthology.orchestra.run.smithi169.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-16T00:14:41.818 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-16T00:14:41.818 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-16T00:14:41.818 INFO:teuthology.orchestra.run.smithi169.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-16T00:14:41.818 INFO:teuthology.orchestra.run.smithi169.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-16T00:14:41.818 INFO:teuthology.orchestra.run.smithi169.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-16T00:14:41.818 INFO:teuthology.orchestra.run.smithi169.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-16T00:14:41.818 INFO:teuthology.orchestra.run.smithi169.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-16T00:14:41.818 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-16T00:14:41.818 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-16T00:14:41.818 INFO:teuthology.orchestra.run.smithi169.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-16T00:14:41.818 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-16T00:14:41.818 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-16T00:14:41.818 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-16T00:14:41.818 INFO:teuthology.orchestra.run.smithi169.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-16T00:14:41.819 INFO:teuthology.orchestra.run.smithi169.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-16T00:14:41.819 INFO:teuthology.orchestra.run.smithi169.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-16T00:14:41.819 INFO:teuthology.orchestra.run.smithi169.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-16T00:14:41.819 INFO:teuthology.orchestra.run.smithi169.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-16T00:14:41.819 INFO:teuthology.orchestra.run.smithi169.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-16T00:14:41.819 INFO:teuthology.orchestra.run.smithi169.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-16T00:14:41.819 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:14:41.819 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2024-09-16T00:14:41.819 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T00:14:41.819 INFO:teuthology.orchestra.run.smithi169.stdout:Install 34 Packages 2024-09-16T00:14:41.819 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:14:41.820 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 27 M 2024-09-16T00:14:41.820 INFO:teuthology.orchestra.run.smithi169.stdout:Installed size: 96 M 2024-09-16T00:14:41.820 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2024-09-16T00:14:41.981 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-osd-2:17.2.7-1633.g6d025bc2.el9.x86_64 31/34 2024-09-16T00:14:42.008 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-osd-2:17.2.7-1633.g6d025bc2.el9.x86_64 31/34 2024-09-16T00:14:42.008 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:14:42.009 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-16T00:14:42.009 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T00:14:42.009 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T00:14:42.009 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:14:42.097 INFO:teuthology.orchestra.run.smithi169.stdout:(1/34): ceph-17.2.7-1633.g6d025bc2.el9.x86_64.r 47 kB/s | 6.5 kB 00:00 2024-09-16T00:14:42.247 INFO:teuthology.orchestra.run.smithi169.stdout:(2/34): ceph-mgr-17.2.7-1633.g6d025bc2.el9.x86_ 4.9 MB/s | 1.4 MB 00:00 2024-09-16T00:14:42.306 INFO:teuthology.orchestra.run.smithi169.stdout:(3/34): ceph-mds-17.2.7-1633.g6d025bc2.el9.x86_ 6.0 MB/s | 2.1 MB 00:00 2024-09-16T00:14:42.381 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mon-2:17.2.7-1633.g6d025bc2.el9.x86_64 32/34 2024-09-16T00:14:42.398 INFO:teuthology.orchestra.run.smithi169.stdout:(4/34): ceph-mgr-modules-core-17.2.7-1633.g6d02 2.6 MB/s | 240 kB 00:00 2024-09-16T00:14:42.405 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mon-2:17.2.7-1633.g6d025bc2.el9.x86_64 32/34 2024-09-16T00:14:42.405 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:14:42.406 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-16T00:14:42.406 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T00:14:42.406 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T00:14:42.406 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:14:42.457 INFO:teuthology.orchestra.run.smithi169.stdout:(5/34): ceph-mon-17.2.7-1633.g6d025bc2.el9.x86_ 11 MB/s | 3.9 MB 00:00 2024-09-16T00:14:42.607 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mds-2:17.2.7-1633.g6d025bc2.el9.x86_64 33/34 2024-09-16T00:14:42.631 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mds-2:17.2.7-1633.g6d025bc2.el9.x86_64 33/34 2024-09-16T00:14:42.631 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:14:42.631 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-16T00:14:42.631 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T00:14:42.631 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T00:14:42.631 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:14:42.695 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-2:17.2.7-1633.g6d025bc2.el9.x86_64 34/34 2024-09-16T00:14:42.782 INFO:teuthology.orchestra.run.smithi169.stdout:(6/34): ceph-osd-17.2.7-1633.g6d025bc2.el9.x86_ 29 MB/s | 15 MB 00:00 2024-09-16T00:14:42.807 INFO:teuthology.orchestra.run.smithi169.stdout:(7/34): python3-cffi-1.14.5-5.el9.x86_64.rpm 619 kB/s | 253 kB 00:00 2024-09-16T00:14:42.858 INFO:teuthology.orchestra.run.smithi169.stdout:(8/34): python3-pycparser-2.20-6.el9.noarch.rpm 2.6 MB/s | 135 kB 00:00 2024-09-16T00:14:42.899 INFO:teuthology.orchestra.run.smithi169.stdout:(9/34): python3-cryptography-36.0.1-4.el9.x86_6 2.8 MB/s | 1.2 MB 00:00 2024-09-16T00:14:42.925 INFO:teuthology.orchestra.run.smithi169.stdout:(10/34): python3-requests-2.25.1-8.el9.noarch.r 1.8 MB/s | 125 kB 00:00 2024-09-16T00:14:42.958 INFO:teuthology.orchestra.run.smithi169.stdout:(11/34): python3-urllib3-1.26.5-6.el9.noarch.rp 3.6 MB/s | 215 kB 00:00 2024-09-16T00:14:42.984 INFO:teuthology.orchestra.run.smithi169.stdout:(12/34): python3-ply-3.11-14.el9.noarch.rpm 530 kB/s | 106 kB 00:00 2024-09-16T00:14:43.067 INFO:teuthology.orchestra.run.smithi169.stdout:(13/34): python3-markupsafe-1.1.1-12.el9.x86_64 320 kB/s | 35 kB 00:00 2024-09-16T00:14:43.093 INFO:teuthology.orchestra.run.smithi169.stdout:(14/34): python3-mako-1.1.4-6.el9.noarch.rpm 1.0 MB/s | 172 kB 00:00 2024-09-16T00:14:43.109 INFO:teuthology.orchestra.run.smithi169.stdout:(15/34): python3-packaging-20.9-5.el9.noarch.rp 616 kB/s | 77 kB 00:00 2024-09-16T00:14:43.143 INFO:teuthology.orchestra.run.smithi169.stdout:(16/34): python3-pytz-2021.1-5.el9.noarch.rpm 675 kB/s | 51 kB 00:00 2024-09-16T00:14:43.168 INFO:teuthology.orchestra.run.smithi169.stdout:(17/34): python3-toml-0.10.2-6.el9.noarch.rpm 553 kB/s | 42 kB 00:00 2024-09-16T00:14:43.193 INFO:teuthology.orchestra.run.smithi169.stdout:(18/34): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 518 kB/s | 43 kB 00:00 2024-09-16T00:14:43.227 INFO:teuthology.orchestra.run.smithi169.stdout:(19/34): python3-jaraco-8.2.1-3.el9.noarch.rpm 430 kB/s | 11 kB 00:00 2024-09-16T00:14:43.253 INFO:teuthology.orchestra.run.smithi169.stdout:(20/34): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.6 MB/s | 172 kB 00:00 2024-09-16T00:14:43.269 INFO:teuthology.orchestra.run.smithi169.stdout:(21/34): python3-jaraco-classes-3.2.1-5.el9.noa 440 kB/s | 18 kB 00:00 2024-09-16T00:14:43.286 INFO:teuthology.orchestra.run.smithi169.stdout:(22/34): python3-jaraco-collections-3.0.0-8.el9 695 kB/s | 23 kB 00:00 2024-09-16T00:14:43.303 INFO:teuthology.orchestra.run.smithi169.stdout:(23/34): python3-jaraco-functools-3.5.0-2.el9.n 579 kB/s | 19 kB 00:00 2024-09-16T00:14:43.320 INFO:teuthology.orchestra.run.smithi169.stdout:(24/34): python3-jaraco-text-3.2.0-6.el9.noarch 586 kB/s | 20 kB 00:00 2024-09-16T00:14:43.337 INFO:teuthology.orchestra.run.smithi169.stdout:(25/34): python3-logutils-0.3.5-21.el9.noarch.r 1.3 MB/s | 46 kB 00:00 2024-09-16T00:14:43.362 INFO:teuthology.orchestra.run.smithi169.stdout:(26/34): python3-more-itertools-8.12.0-2.el9.no 1.8 MB/s | 79 kB 00:00 2024-09-16T00:14:43.387 INFO:teuthology.orchestra.run.smithi169.stdout:(27/34): python3-portend-3.1.0-2.el9.noarch.rpm 655 kB/s | 16 kB 00:00 2024-09-16T00:14:43.413 INFO:teuthology.orchestra.run.smithi169.stdout:(28/34): python3-cherrypy-18.6.1-2.el9.noarch.r 1.5 MB/s | 358 kB 00:00 2024-09-16T00:14:43.438 INFO:teuthology.orchestra.run.smithi169.stdout:(29/34): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.7 MB/s | 90 kB 00:00 2024-09-16T00:14:43.463 INFO:teuthology.orchestra.run.smithi169.stdout:(30/34): python3-tempora-5.0.0-2.el9.noarch.rpm 714 kB/s | 36 kB 00:00 2024-09-16T00:14:43.489 INFO:teuthology.orchestra.run.smithi169.stdout:(31/34): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-09-16T00:14:43.514 INFO:teuthology.orchestra.run.smithi169.stdout:(32/34): python3-zc-lockfile-2.0-10.el9.noarch. 802 kB/s | 20 kB 00:00 2024-09-16T00:14:43.539 INFO:teuthology.orchestra.run.smithi169.stdout:(33/34): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-16T00:14:43.564 INFO:teuthology.orchestra.run.smithi169.stdout:(34/34): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.2 MB/s | 427 kB 00:00 2024-09-16T00:14:43.568 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2024-09-16T00:14:43.568 INFO:teuthology.orchestra.run.smithi169.stdout:Total 16 MB/s | 27 MB 00:01 2024-09-16T00:14:43.693 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2024-09-16T00:14:43.731 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2024-09-16T00:14:43.731 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2024-09-16T00:14:44.131 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2024-09-16T00:14:44.132 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2024-09-16T00:14:44.343 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-2:17.2.7-1633.g6d025bc2.el9.x86_64 34/34 2024-09-16T00:14:44.344 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-2:17.2.7-1633.g6d025bc2.el9.x86_64 1/34 2024-09-16T00:14:44.344 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mds-2:17.2.7-1633.g6d025bc2.el9.x86_64 2/34 2024-09-16T00:14:44.344 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-2:17.2.7-1633.g6d025bc2.el9.x86_64 3/34 2024-09-16T00:14:44.344 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mon-2:17.2.7-1633.g6d025bc2.el9.x86_64 4/34 2024-09-16T00:14:44.344 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-osd-2:17.2.7-1633.g6d025bc2.el9.x86_64 5/34 2024-09-16T00:14:44.344 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-1633.g6d025bc2.el9. 6/34 2024-09-16T00:14:44.344 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/34 2024-09-16T00:14:44.344 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/34 2024-09-16T00:14:44.344 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/34 2024-09-16T00:14:44.344 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/34 2024-09-16T00:14:44.344 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/34 2024-09-16T00:14:44.344 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/34 2024-09-16T00:14:44.344 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/34 2024-09-16T00:14:44.345 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/34 2024-09-16T00:14:44.345 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-16T00:14:44.345 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/34 2024-09-16T00:14:44.345 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/34 2024-09-16T00:14:44.345 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 18/34 2024-09-16T00:14:44.345 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 19/34 2024-09-16T00:14:44.345 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 20/34 2024-09-16T00:14:44.345 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 21/34 2024-09-16T00:14:44.345 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 22/34 2024-09-16T00:14:44.345 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 23/34 2024-09-16T00:14:44.345 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 24/34 2024-09-16T00:14:44.345 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 25/34 2024-09-16T00:14:44.346 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 26/34 2024-09-16T00:14:44.346 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 27/34 2024-09-16T00:14:44.346 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 28/34 2024-09-16T00:14:44.346 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 29/34 2024-09-16T00:14:44.346 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 30/34 2024-09-16T00:14:44.346 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 31/34 2024-09-16T00:14:44.346 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 32/34 2024-09-16T00:14:44.346 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 33/34 2024-09-16T00:14:45.334 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 34/34 2024-09-16T00:14:45.334 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:14:45.334 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2024-09-16T00:14:45.334 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-16T00:14:45.334 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mds-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-16T00:14:45.334 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-16T00:14:45.334 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-modules-core-2:17.2.7-1633.g6d025bc2.el9.noarch 2024-09-16T00:14:45.334 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mon-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-16T00:14:45.335 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-osd-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-16T00:14:45.335 INFO:teuthology.orchestra.run.smithi112.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-16T00:14:45.335 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-16T00:14:45.335 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-16T00:14:45.335 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-16T00:14:45.335 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-16T00:14:45.335 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-16T00:14:45.335 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-16T00:14:45.335 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-16T00:14:45.335 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-16T00:14:45.335 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-16T00:14:45.336 INFO:teuthology.orchestra.run.smithi112.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-16T00:14:45.336 INFO:teuthology.orchestra.run.smithi112.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-16T00:14:45.336 INFO:teuthology.orchestra.run.smithi112.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-16T00:14:45.336 INFO:teuthology.orchestra.run.smithi112.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-16T00:14:45.336 INFO:teuthology.orchestra.run.smithi112.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-16T00:14:45.336 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-16T00:14:45.336 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ply-3.11-14.el9.noarch 2024-09-16T00:14:45.336 INFO:teuthology.orchestra.run.smithi112.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-16T00:14:45.336 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-16T00:14:45.336 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-16T00:14:45.336 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-16T00:14:45.336 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-16T00:14:45.336 INFO:teuthology.orchestra.run.smithi112.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-16T00:14:45.336 INFO:teuthology.orchestra.run.smithi112.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-16T00:14:45.337 INFO:teuthology.orchestra.run.smithi112.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-16T00:14:45.337 INFO:teuthology.orchestra.run.smithi112.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-16T00:14:45.337 INFO:teuthology.orchestra.run.smithi112.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-16T00:14:45.337 INFO:teuthology.orchestra.run.smithi112.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-16T00:14:45.337 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:14:45.337 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2024-09-16T00:14:45.391 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2024-09-16T00:14:45.467 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/34 2024-09-16T00:14:45.499 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/34 2024-09-16T00:14:45.533 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/34 2024-09-16T00:14:45.625 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/34 2024-09-16T00:14:45.699 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install ceph-base 2024-09-16T00:14:45.711 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/34 2024-09-16T00:14:45.752 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/34 2024-09-16T00:14:45.787 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/34 2024-09-16T00:14:45.816 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/34 2024-09-16T00:14:45.846 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/34 2024-09-16T00:14:45.879 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/34 2024-09-16T00:14:45.920 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/34 2024-09-16T00:14:45.955 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/34 2024-09-16T00:14:45.989 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/34 2024-09-16T00:14:46.037 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/34 2024-09-16T00:14:46.070 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-16T00:14:46.143 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 16/34 2024-09-16T00:14:46.273 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:00:51 ago on Mon 16 Sep 2024 12:13:55 AM UTC. 2024-09-16T00:14:46.290 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 17/34 2024-09-16T00:14:46.363 INFO:teuthology.orchestra.run.smithi112.stdout:Package ceph-base-2:17.2.7-1633.g6d025bc2.el9.x86_64 is already installed. 2024-09-16T00:14:46.368 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 18/34 2024-09-16T00:14:46.405 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2024-09-16T00:14:46.405 INFO:teuthology.orchestra.run.smithi112.stdout:Nothing to do. 2024-09-16T00:14:46.406 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2024-09-16T00:14:46.422 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 19/34 2024-09-16T00:14:46.465 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 20/34 2024-09-16T00:14:46.482 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install cephadm 2024-09-16T00:14:46.518 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-ply-3.11-14.el9.noarch 21/34 2024-09-16T00:14:46.574 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 22/34 2024-09-16T00:14:46.769 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 23/34 2024-09-16T00:14:46.831 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 24/34 2024-09-16T00:14:46.910 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 25/34 2024-09-16T00:14:46.998 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 26/34 2024-09-16T00:14:47.039 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/34 2024-09-16T00:14:47.040 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:00:52 ago on Mon 16 Sep 2024 12:13:55 AM UTC. 2024-09-16T00:14:47.153 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 28/34 2024-09-16T00:14:47.171 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2024-09-16T00:14:47.171 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-09-16T00:14:47.171 INFO:teuthology.orchestra.run.smithi112.stdout: Package Arch Version Repository Size 2024-09-16T00:14:47.172 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-09-16T00:14:47.172 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2024-09-16T00:14:47.172 INFO:teuthology.orchestra.run.smithi112.stdout: cephadm noarch 2:17.2.7-1633.g6d025bc2.el9 ceph-noarch 83 k 2024-09-16T00:14:47.172 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:14:47.172 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2024-09-16T00:14:47.172 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-09-16T00:14:47.172 INFO:teuthology.orchestra.run.smithi112.stdout:Install 1 Package 2024-09-16T00:14:47.172 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:14:47.172 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 83 k 2024-09-16T00:14:47.172 INFO:teuthology.orchestra.run.smithi112.stdout:Installed size: 375 k 2024-09-16T00:14:47.172 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2024-09-16T00:14:47.307 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-1633.g6d025bc2.el9. 29/34 2024-09-16T00:14:47.333 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-2:17.2.7-1633.g6d025bc2.el9.x86_64 30/34 2024-09-16T00:14:47.358 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-2:17.2.7-1633.g6d025bc2.el9.x86_64 30/34 2024-09-16T00:14:47.359 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:14:47.359 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-16T00:14:47.359 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T00:14:47.359 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T00:14:47.359 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:14:47.372 INFO:teuthology.orchestra.run.smithi112.stdout:cephadm-17.2.7-1633.g6d025bc2.el9.noarch.rpm 414 kB/s | 83 kB 00:00 2024-09-16T00:14:47.372 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2024-09-16T00:14:47.373 INFO:teuthology.orchestra.run.smithi112.stdout:Total 411 kB/s | 83 kB 00:00 2024-09-16T00:14:47.373 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2024-09-16T00:14:47.378 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2024-09-16T00:14:47.378 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2024-09-16T00:14:47.380 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2024-09-16T00:14:47.380 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2024-09-16T00:14:47.469 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2024-09-16T00:14:47.946 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: cephadm-2:17.2.7-1633.g6d025bc2.el9.noarch 1/1 2024-09-16T00:14:48.042 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephadm-2:17.2.7-1633.g6d025bc2.el9.noarch 1/1 2024-09-16T00:14:48.417 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: cephadm-2:17.2.7-1633.g6d025bc2.el9.noarch 1/1 2024-09-16T00:14:48.668 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-osd-2:17.2.7-1633.g6d025bc2.el9.x86_64 31/34 2024-09-16T00:14:49.169 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-osd-2:17.2.7-1633.g6d025bc2.el9.x86_64 31/34 2024-09-16T00:14:49.169 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:14:49.169 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-16T00:14:49.169 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T00:14:49.169 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T00:14:49.169 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:14:49.169 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mon-2:17.2.7-1633.g6d025bc2.el9.x86_64 32/34 2024-09-16T00:14:49.169 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mon-2:17.2.7-1633.g6d025bc2.el9.x86_64 32/34 2024-09-16T00:14:49.169 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:14:49.169 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-16T00:14:49.170 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T00:14:49.170 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T00:14:49.170 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:14:49.170 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephadm-2:17.2.7-1633.g6d025bc2.el9.noarch 1/1 2024-09-16T00:14:49.170 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:14:49.170 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2024-09-16T00:14:49.170 INFO:teuthology.orchestra.run.smithi112.stdout: cephadm-2:17.2.7-1633.g6d025bc2.el9.noarch 2024-09-16T00:14:49.170 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:14:49.170 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2024-09-16T00:14:49.171 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install ceph-immutable-object-cache 2024-09-16T00:14:49.303 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mds-2:17.2.7-1633.g6d025bc2.el9.x86_64 33/34 2024-09-16T00:14:49.327 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mds-2:17.2.7-1633.g6d025bc2.el9.x86_64 33/34 2024-09-16T00:14:49.327 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:14:49.327 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-16T00:14:49.327 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T00:14:49.328 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T00:14:49.328 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:14:49.390 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-2:17.2.7-1633.g6d025bc2.el9.x86_64 34/34 2024-09-16T00:14:49.751 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:00:54 ago on Mon 16 Sep 2024 12:13:55 AM UTC. 2024-09-16T00:14:49.881 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2024-09-16T00:14:49.882 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-09-16T00:14:49.882 INFO:teuthology.orchestra.run.smithi112.stdout: Package Arch Version Repo Size 2024-09-16T00:14:49.882 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-09-16T00:14:49.882 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2024-09-16T00:14:49.882 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 137 k 2024-09-16T00:14:49.882 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:14:49.882 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2024-09-16T00:14:49.882 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-09-16T00:14:49.882 INFO:teuthology.orchestra.run.smithi112.stdout:Install 1 Package 2024-09-16T00:14:49.882 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:14:49.882 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 137 k 2024-09-16T00:14:49.883 INFO:teuthology.orchestra.run.smithi112.stdout:Installed size: 379 k 2024-09-16T00:14:49.883 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2024-09-16T00:14:50.086 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-immutable-object-cache-17.2.7-1633.g6d025b 676 kB/s | 137 kB 00:00 2024-09-16T00:14:50.086 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2024-09-16T00:14:50.087 INFO:teuthology.orchestra.run.smithi112.stdout:Total 671 kB/s | 137 kB 00:00 2024-09-16T00:14:50.087 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2024-09-16T00:14:50.094 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2024-09-16T00:14:50.094 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2024-09-16T00:14:50.151 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2024-09-16T00:14:50.151 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2024-09-16T00:14:50.294 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2024-09-16T00:14:50.362 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-1633.g6d025bc2. 1/1 2024-09-16T00:14:50.387 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-1633.g6d025bc2. 1/1 2024-09-16T00:14:50.388 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:14:50.388 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-16T00:14:50.388 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:14:51.020 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-2:17.2.7-1633.g6d025bc2.el9.x86_64 34/34 2024-09-16T00:14:51.020 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-2:17.2.7-1633.g6d025bc2.el9.x86_64 1/34 2024-09-16T00:14:51.020 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mds-2:17.2.7-1633.g6d025bc2.el9.x86_64 2/34 2024-09-16T00:14:51.020 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-2:17.2.7-1633.g6d025bc2.el9.x86_64 3/34 2024-09-16T00:14:51.021 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mon-2:17.2.7-1633.g6d025bc2.el9.x86_64 4/34 2024-09-16T00:14:51.021 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-osd-2:17.2.7-1633.g6d025bc2.el9.x86_64 5/34 2024-09-16T00:14:51.021 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-1633.g6d025bc2.el9. 6/34 2024-09-16T00:14:51.021 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/34 2024-09-16T00:14:51.021 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/34 2024-09-16T00:14:51.021 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/34 2024-09-16T00:14:51.021 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/34 2024-09-16T00:14:51.021 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/34 2024-09-16T00:14:51.021 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/34 2024-09-16T00:14:51.021 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/34 2024-09-16T00:14:51.021 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/34 2024-09-16T00:14:51.021 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-16T00:14:51.021 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/34 2024-09-16T00:14:51.021 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/34 2024-09-16T00:14:51.022 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 18/34 2024-09-16T00:14:51.022 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 19/34 2024-09-16T00:14:51.022 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 20/34 2024-09-16T00:14:51.022 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 21/34 2024-09-16T00:14:51.022 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 22/34 2024-09-16T00:14:51.022 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 23/34 2024-09-16T00:14:51.022 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 24/34 2024-09-16T00:14:51.022 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 25/34 2024-09-16T00:14:51.022 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 26/34 2024-09-16T00:14:51.022 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 27/34 2024-09-16T00:14:51.022 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 28/34 2024-09-16T00:14:51.022 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 29/34 2024-09-16T00:14:51.022 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 30/34 2024-09-16T00:14:51.023 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 31/34 2024-09-16T00:14:51.023 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 32/34 2024-09-16T00:14:51.023 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 33/34 2024-09-16T00:14:51.385 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-1633.g6d025bc2. 1/1 2024-09-16T00:14:51.385 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:14:51.385 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2024-09-16T00:14:51.385 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-immutable-object-cache-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-16T00:14:51.385 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:14:51.385 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2024-09-16T00:14:51.584 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install ceph-mgr 2024-09-16T00:14:52.011 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 34/34 2024-09-16T00:14:52.011 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:14:52.011 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2024-09-16T00:14:52.011 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-16T00:14:52.011 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mds-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-16T00:14:52.011 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-16T00:14:52.012 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-modules-core-2:17.2.7-1633.g6d025bc2.el9.noarch 2024-09-16T00:14:52.012 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mon-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-16T00:14:52.012 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-osd-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-16T00:14:52.012 INFO:teuthology.orchestra.run.smithi169.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-16T00:14:52.012 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-16T00:14:52.012 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-16T00:14:52.012 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-16T00:14:52.012 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-16T00:14:52.012 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-16T00:14:52.012 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-16T00:14:52.012 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-16T00:14:52.012 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-16T00:14:52.012 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-16T00:14:52.013 INFO:teuthology.orchestra.run.smithi169.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-16T00:14:52.013 INFO:teuthology.orchestra.run.smithi169.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-16T00:14:52.013 INFO:teuthology.orchestra.run.smithi169.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-16T00:14:52.013 INFO:teuthology.orchestra.run.smithi169.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-16T00:14:52.013 INFO:teuthology.orchestra.run.smithi169.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-16T00:14:52.013 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-16T00:14:52.013 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ply-3.11-14.el9.noarch 2024-09-16T00:14:52.013 INFO:teuthology.orchestra.run.smithi169.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-16T00:14:52.013 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-16T00:14:52.013 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-16T00:14:52.013 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-16T00:14:52.013 INFO:teuthology.orchestra.run.smithi169.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-16T00:14:52.013 INFO:teuthology.orchestra.run.smithi169.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-16T00:14:52.013 INFO:teuthology.orchestra.run.smithi169.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-16T00:14:52.014 INFO:teuthology.orchestra.run.smithi169.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-16T00:14:52.014 INFO:teuthology.orchestra.run.smithi169.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-16T00:14:52.014 INFO:teuthology.orchestra.run.smithi169.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-16T00:14:52.014 INFO:teuthology.orchestra.run.smithi169.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-16T00:14:52.014 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:14:52.014 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-09-16T00:14:52.163 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:00:57 ago on Mon 16 Sep 2024 12:13:55 AM UTC. 2024-09-16T00:14:52.257 INFO:teuthology.orchestra.run.smithi112.stdout:Package ceph-mgr-2:17.2.7-1633.g6d025bc2.el9.x86_64 is already installed. 2024-09-16T00:14:52.299 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2024-09-16T00:14:52.300 INFO:teuthology.orchestra.run.smithi112.stdout:Nothing to do. 2024-09-16T00:14:52.300 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2024-09-16T00:14:52.373 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install ceph-mgr-dashboard 2024-09-16T00:14:52.407 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install ceph-base 2024-09-16T00:14:52.933 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:00:57 ago on Mon 16 Sep 2024 12:13:55 AM UTC. 2024-09-16T00:14:52.997 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:00:53 ago on Mon 16 Sep 2024 12:13:59 AM UTC. 2024-09-16T00:14:53.065 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2024-09-16T00:14:53.066 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-09-16T00:14:53.066 INFO:teuthology.orchestra.run.smithi112.stdout: Package Arch Version Repository Size 2024-09-16T00:14:53.066 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-09-16T00:14:53.066 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2024-09-16T00:14:53.066 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-dashboard noarch 2:17.2.7-1633.g6d025bc2.el9 ceph-noarch 3.4 M 2024-09-16T00:14:53.066 INFO:teuthology.orchestra.run.smithi112.stdout:Installing dependencies: 2024-09-16T00:14:53.066 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-grafana-dashboards noarch 2:17.2.7-1633.g6d025bc2.el9 ceph-noarch 24 k 2024-09-16T00:14:53.066 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-prometheus-alerts noarch 2:17.2.7-1633.g6d025bc2.el9 ceph-noarch 15 k 2024-09-16T00:14:53.066 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-09-16T00:14:53.067 INFO:teuthology.orchestra.run.smithi112.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-16T00:14:53.067 INFO:teuthology.orchestra.run.smithi112.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-16T00:14:53.067 INFO:teuthology.orchestra.run.smithi112.stdout:Installing weak dependencies: 2024-09-16T00:14:53.067 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-09-16T00:14:53.067 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:14:53.067 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2024-09-16T00:14:53.067 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-09-16T00:14:53.067 INFO:teuthology.orchestra.run.smithi112.stdout:Install 7 Packages 2024-09-16T00:14:53.067 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:14:53.068 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 3.7 M 2024-09-16T00:14:53.068 INFO:teuthology.orchestra.run.smithi112.stdout:Installed size: 80 M 2024-09-16T00:14:53.068 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2024-09-16T00:14:53.088 INFO:teuthology.orchestra.run.smithi169.stdout:Package ceph-base-2:17.2.7-1633.g6d025bc2.el9.x86_64 is already installed. 2024-09-16T00:14:53.129 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-09-16T00:14:53.130 INFO:teuthology.orchestra.run.smithi169.stdout:Nothing to do. 2024-09-16T00:14:53.130 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-09-16T00:14:53.201 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install cephadm 2024-09-16T00:14:53.597 INFO:teuthology.orchestra.run.smithi112.stdout:(1/7): ceph-prometheus-alerts-17.2.7-1633.g6d02 92 kB/s | 15 kB 00:00 2024-09-16T00:14:53.614 INFO:teuthology.orchestra.run.smithi112.stdout:(2/7): ceph-grafana-dashboards-17.2.7-1633.g6d0 139 kB/s | 24 kB 00:00 2024-09-16T00:14:53.656 INFO:teuthology.orchestra.run.smithi112.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 154 kB/s | 9.0 kB 00:00 2024-09-16T00:14:53.675 INFO:teuthology.orchestra.run.smithi112.stdout:(4/7): python3-jwt-2.4.0-1.el9.noarch.rpm 674 kB/s | 41 kB 00:00 2024-09-16T00:14:53.692 INFO:teuthology.orchestra.run.smithi112.stdout:(5/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 859 kB/s | 31 kB 00:00 2024-09-16T00:14:53.726 INFO:teuthology.orchestra.run.smithi112.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 3.7 MB/s | 188 kB 00:00 2024-09-16T00:14:53.760 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:00:54 ago on Mon 16 Sep 2024 12:13:59 AM UTC. 2024-09-16T00:14:53.834 INFO:teuthology.orchestra.run.smithi112.stdout:(7/7): ceph-mgr-dashboard-17.2.7-1633.g6d025bc2 8.5 MB/s | 3.4 MB 00:00 2024-09-16T00:14:53.838 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2024-09-16T00:14:53.838 INFO:teuthology.orchestra.run.smithi112.stdout:Total 4.7 MB/s | 3.7 MB 00:00 2024-09-16T00:14:53.838 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2024-09-16T00:14:53.846 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2024-09-16T00:14:53.846 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2024-09-16T00:14:53.890 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-09-16T00:14:53.891 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T00:14:53.891 INFO:teuthology.orchestra.run.smithi169.stdout: Package Arch Version Repository Size 2024-09-16T00:14:53.891 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T00:14:53.891 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2024-09-16T00:14:53.891 INFO:teuthology.orchestra.run.smithi169.stdout: cephadm noarch 2:17.2.7-1633.g6d025bc2.el9 ceph-noarch 83 k 2024-09-16T00:14:53.891 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:14:53.891 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2024-09-16T00:14:53.891 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T00:14:53.891 INFO:teuthology.orchestra.run.smithi169.stdout:Install 1 Package 2024-09-16T00:14:53.892 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:14:53.892 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 83 k 2024-09-16T00:14:53.892 INFO:teuthology.orchestra.run.smithi169.stdout:Installed size: 375 k 2024-09-16T00:14:53.892 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2024-09-16T00:14:53.928 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2024-09-16T00:14:53.928 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2024-09-16T00:14:54.086 INFO:teuthology.orchestra.run.smithi169.stdout:cephadm-17.2.7-1633.g6d025bc2.el9.noarch.rpm 427 kB/s | 83 kB 00:00 2024-09-16T00:14:54.086 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2024-09-16T00:14:54.086 INFO:teuthology.orchestra.run.smithi169.stdout:Total 424 kB/s | 83 kB 00:00 2024-09-16T00:14:54.087 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2024-09-16T00:14:54.092 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2024-09-16T00:14:54.092 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2024-09-16T00:14:54.094 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2024-09-16T00:14:54.094 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2024-09-16T00:14:54.163 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2024-09-16T00:14:54.222 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2024-09-16T00:14:54.293 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-09-16T00:14:54.328 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-09-16T00:14:54.374 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-09-16T00:14:54.414 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-09-16T00:14:54.452 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-1633.g6d025bc2.el9.n 5/7 2024-09-16T00:14:54.609 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: cephadm-2:17.2.7-1633.g6d025bc2.el9.noarch 1/1 2024-09-16T00:14:54.723 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : cephadm-2:17.2.7-1633.g6d025bc2.el9.noarch 1/1 2024-09-16T00:14:55.110 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: cephadm-2:17.2.7-1633.g6d025bc2.el9.noarch 1/1 2024-09-16T00:14:55.361 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : cephadm-2:17.2.7-1633.g6d025bc2.el9.noarch 1/1 2024-09-16T00:14:55.362 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:14:55.362 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2024-09-16T00:14:55.362 INFO:teuthology.orchestra.run.smithi169.stdout: cephadm-2:17.2.7-1633.g6d025bc2.el9.noarch 2024-09-16T00:14:55.362 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:14:55.362 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-09-16T00:14:55.440 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-1633.g6d025bc2.el9. 6/7 2024-09-16T00:14:55.480 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-1633.g6d025bc2.el9.noarc 7/7 2024-09-16T00:14:55.567 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install ceph-immutable-object-cache 2024-09-16T00:14:56.149 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:00:57 ago on Mon 16 Sep 2024 12:13:59 AM UTC. 2024-09-16T00:14:56.280 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-09-16T00:14:56.280 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T00:14:56.280 INFO:teuthology.orchestra.run.smithi169.stdout: Package Arch Version Repo Size 2024-09-16T00:14:56.280 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T00:14:56.281 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2024-09-16T00:14:56.281 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 137 k 2024-09-16T00:14:56.281 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:14:56.281 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2024-09-16T00:14:56.281 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T00:14:56.281 INFO:teuthology.orchestra.run.smithi169.stdout:Install 1 Package 2024-09-16T00:14:56.281 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:14:56.281 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 137 k 2024-09-16T00:14:56.281 INFO:teuthology.orchestra.run.smithi169.stdout:Installed size: 379 k 2024-09-16T00:14:56.281 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2024-09-16T00:14:56.491 INFO:teuthology.orchestra.run.smithi169.stdout:ceph-immutable-object-cache-17.2.7-1633.g6d025b 654 kB/s | 137 kB 00:00 2024-09-16T00:14:56.492 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2024-09-16T00:14:56.492 INFO:teuthology.orchestra.run.smithi169.stdout:Total 649 kB/s | 137 kB 00:00 2024-09-16T00:14:56.492 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2024-09-16T00:14:56.500 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2024-09-16T00:14:56.500 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2024-09-16T00:14:56.513 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-1633.g6d025bc2.el9.noarc 7/7 2024-09-16T00:14:56.513 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-1633.g6d025bc2.el9. 1/7 2024-09-16T00:14:56.513 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-1633.g6d025bc2.el9.noarc 2/7 2024-09-16T00:14:56.513 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-1633.g6d025bc2.el9.n 3/7 2024-09-16T00:14:56.514 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-09-16T00:14:56.514 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-09-16T00:14:56.514 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-09-16T00:14:56.556 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2024-09-16T00:14:56.556 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2024-09-16T00:14:56.696 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2024-09-16T00:14:56.751 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-1633.g6d025bc2. 1/1 2024-09-16T00:14:56.776 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-1633.g6d025bc2. 1/1 2024-09-16T00:14:56.776 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:14:56.776 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-16T00:14:56.776 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:14:56.800 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-09-16T00:14:56.800 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:14:56.800 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2024-09-16T00:14:56.801 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-grafana-dashboards-2:17.2.7-1633.g6d025bc2.el9.noarch 2024-09-16T00:14:56.801 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-dashboard-2:17.2.7-1633.g6d025bc2.el9.noarch 2024-09-16T00:14:56.801 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-prometheus-alerts-2:17.2.7-1633.g6d025bc2.el9.noarch 2024-09-16T00:14:56.801 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-09-16T00:14:56.801 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-09-16T00:14:56.801 INFO:teuthology.orchestra.run.smithi112.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-16T00:14:56.801 INFO:teuthology.orchestra.run.smithi112.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-16T00:14:56.801 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:14:56.801 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2024-09-16T00:14:57.009 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-16T00:14:57.567 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:01:02 ago on Mon 16 Sep 2024 12:13:55 AM UTC. 2024-09-16T00:14:57.706 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2024-09-16T00:14:57.707 INFO:teuthology.orchestra.run.smithi112.stdout:======================================================================================= 2024-09-16T00:14:57.707 INFO:teuthology.orchestra.run.smithi112.stdout: Package Arch Version Repository Size 2024-09-16T00:14:57.707 INFO:teuthology.orchestra.run.smithi112.stdout:======================================================================================= 2024-09-16T00:14:57.707 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2024-09-16T00:14:57.707 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-1633.g6d025bc2.el9 ceph-noarch 7.4 M 2024-09-16T00:14:57.707 INFO:teuthology.orchestra.run.smithi112.stdout:Installing dependencies: 2024-09-16T00:14:57.707 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-16T00:14:57.707 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-16T00:14:57.707 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-16T00:14:57.707 INFO:teuthology.orchestra.run.smithi112.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-16T00:14:57.707 INFO:teuthology.orchestra.run.smithi112.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-16T00:14:57.707 INFO:teuthology.orchestra.run.smithi112.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-16T00:14:57.708 INFO:teuthology.orchestra.run.smithi112.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-16T00:14:57.708 INFO:teuthology.orchestra.run.smithi112.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-16T00:14:57.708 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-16T00:14:57.708 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-16T00:14:57.708 INFO:teuthology.orchestra.run.smithi112.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-16T00:14:57.708 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:14:57.708 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2024-09-16T00:14:57.708 INFO:teuthology.orchestra.run.smithi112.stdout:======================================================================================= 2024-09-16T00:14:57.708 INFO:teuthology.orchestra.run.smithi112.stdout:Install 12 Packages 2024-09-16T00:14:57.708 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:14:57.709 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 38 M 2024-09-16T00:14:57.709 INFO:teuthology.orchestra.run.smithi112.stdout:Installed size: 212 M 2024-09-16T00:14:57.709 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2024-09-16T00:14:57.734 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-1633.g6d025bc2. 1/1 2024-09-16T00:14:57.734 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:14:57.734 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2024-09-16T00:14:57.734 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-immutable-object-cache-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-16T00:14:57.734 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:14:57.734 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-09-16T00:14:57.977 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install ceph-mgr 2024-09-16T00:14:58.192 INFO:teuthology.orchestra.run.smithi112.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.7 MB/s | 191 kB 00:00 2024-09-16T00:14:58.250 INFO:teuthology.orchestra.run.smithi112.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 4.6 MB/s | 801 kB 00:00 2024-09-16T00:14:58.292 INFO:teuthology.orchestra.run.smithi112.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 326 kB/s | 33 kB 00:00 2024-09-16T00:14:58.334 INFO:teuthology.orchestra.run.smithi112.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 438 kB/s | 18 kB 00:00 2024-09-16T00:14:58.376 INFO:teuthology.orchestra.run.smithi112.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 924 kB/s | 38 kB 00:00 2024-09-16T00:14:58.510 INFO:teuthology.orchestra.run.smithi112.stdout:(6/12): ceph-mgr-diskprediction-local-17.2.7-16 17 MB/s | 7.4 MB 00:00 2024-09-16T00:14:58.554 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:00:59 ago on Mon 16 Sep 2024 12:13:59 AM UTC. 2024-09-16T00:14:58.650 INFO:teuthology.orchestra.run.smithi169.stdout:Package ceph-mgr-2:17.2.7-1633.g6d025bc2.el9.x86_64 is already installed. 2024-09-16T00:14:58.676 INFO:teuthology.orchestra.run.smithi112.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 1.4 MB/s | 245 kB 00:00 2024-09-16T00:14:58.693 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-09-16T00:14:58.694 INFO:teuthology.orchestra.run.smithi169.stdout:Nothing to do. 2024-09-16T00:14:58.694 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-09-16T00:14:58.778 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install ceph-mgr-dashboard 2024-09-16T00:14:58.868 INFO:teuthology.orchestra.run.smithi112.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 4.8 MB/s | 3.0 MB 00:00 2024-09-16T00:14:58.960 INFO:teuthology.orchestra.run.smithi112.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.7 MB/s | 250 kB 00:00 2024-09-16T00:14:59.345 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:01:00 ago on Mon 16 Sep 2024 12:13:59 AM UTC. 2024-09-16T00:14:59.479 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-09-16T00:14:59.479 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T00:14:59.479 INFO:teuthology.orchestra.run.smithi169.stdout: Package Arch Version Repository Size 2024-09-16T00:14:59.480 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T00:14:59.480 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2024-09-16T00:14:59.480 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-dashboard noarch 2:17.2.7-1633.g6d025bc2.el9 ceph-noarch 3.4 M 2024-09-16T00:14:59.480 INFO:teuthology.orchestra.run.smithi169.stdout:Installing dependencies: 2024-09-16T00:14:59.480 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-grafana-dashboards noarch 2:17.2.7-1633.g6d025bc2.el9 ceph-noarch 24 k 2024-09-16T00:14:59.480 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-prometheus-alerts noarch 2:17.2.7-1633.g6d025bc2.el9 ceph-noarch 15 k 2024-09-16T00:14:59.480 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-09-16T00:14:59.480 INFO:teuthology.orchestra.run.smithi169.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-16T00:14:59.480 INFO:teuthology.orchestra.run.smithi169.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-16T00:14:59.480 INFO:teuthology.orchestra.run.smithi169.stdout:Installing weak dependencies: 2024-09-16T00:14:59.480 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-09-16T00:14:59.480 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:14:59.480 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2024-09-16T00:14:59.480 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T00:14:59.481 INFO:teuthology.orchestra.run.smithi169.stdout:Install 7 Packages 2024-09-16T00:14:59.481 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:14:59.481 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 3.7 M 2024-09-16T00:14:59.481 INFO:teuthology.orchestra.run.smithi169.stdout:Installed size: 80 M 2024-09-16T00:14:59.482 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2024-09-16T00:14:59.552 INFO:teuthology.orchestra.run.smithi112.stdout:(10/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 4.2 MB/s | 4.9 MB 00:01 2024-09-16T00:14:59.669 INFO:teuthology.orchestra.run.smithi112.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 5.1 MB/s | 5.1 MB 00:00 2024-09-16T00:14:59.787 INFO:teuthology.orchestra.run.smithi169.stdout:(1/7): ceph-grafana-dashboards-17.2.7-1633.g6d0 173 kB/s | 24 kB 00:00 2024-09-16T00:14:59.812 INFO:teuthology.orchestra.run.smithi169.stdout:(2/7): ceph-prometheus-alerts-17.2.7-1633.g6d02 88 kB/s | 15 kB 00:00 2024-09-16T00:14:59.854 INFO:teuthology.orchestra.run.smithi169.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 135 kB/s | 9.0 kB 00:00 2024-09-16T00:14:59.880 INFO:teuthology.orchestra.run.smithi169.stdout:(4/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 1.3 MB/s | 31 kB 00:00 2024-09-16T00:14:59.905 INFO:teuthology.orchestra.run.smithi169.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 444 kB/s | 41 kB 00:00 2024-09-16T00:14:59.930 INFO:teuthology.orchestra.run.smithi169.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 3.7 MB/s | 188 kB 00:00 2024-09-16T00:15:00.040 INFO:teuthology.orchestra.run.smithi169.stdout:(7/7): ceph-mgr-dashboard-17.2.7-1633.g6d025bc2 8.5 MB/s | 3.4 MB 00:00 2024-09-16T00:15:00.041 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2024-09-16T00:15:00.042 INFO:teuthology.orchestra.run.smithi169.stdout:Total 6.5 MB/s | 3.7 MB 00:00 2024-09-16T00:15:00.042 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2024-09-16T00:15:00.050 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2024-09-16T00:15:00.050 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2024-09-16T00:15:00.131 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2024-09-16T00:15:00.131 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2024-09-16T00:15:00.441 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2024-09-16T00:15:00.498 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-09-16T00:15:00.532 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-09-16T00:15:00.578 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-09-16T00:15:00.611 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-09-16T00:15:00.639 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-1633.g6d025bc2.el9.n 5/7 2024-09-16T00:15:01.629 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-1633.g6d025bc2.el9. 6/7 2024-09-16T00:15:01.668 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-1633.g6d025bc2.el9.noarc 7/7 2024-09-16T00:15:02.667 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-1633.g6d025bc2.el9.noarc 7/7 2024-09-16T00:15:02.667 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-1633.g6d025bc2.el9. 1/7 2024-09-16T00:15:02.667 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-1633.g6d025bc2.el9.noarc 2/7 2024-09-16T00:15:02.667 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-1633.g6d025bc2.el9.n 3/7 2024-09-16T00:15:02.668 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-09-16T00:15:02.668 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-09-16T00:15:02.668 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-09-16T00:15:02.935 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-09-16T00:15:02.936 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:15:02.936 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2024-09-16T00:15:02.936 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-grafana-dashboards-2:17.2.7-1633.g6d025bc2.el9.noarch 2024-09-16T00:15:02.936 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-dashboard-2:17.2.7-1633.g6d025bc2.el9.noarch 2024-09-16T00:15:02.936 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-prometheus-alerts-2:17.2.7-1633.g6d025bc2.el9.noarch 2024-09-16T00:15:02.936 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-09-16T00:15:02.936 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-09-16T00:15:02.936 INFO:teuthology.orchestra.run.smithi169.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-16T00:15:02.936 INFO:teuthology.orchestra.run.smithi169.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-16T00:15:02.936 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:15:02.936 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-09-16T00:15:03.161 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-16T00:15:03.511 INFO:teuthology.orchestra.run.smithi112.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 3.6 MB/s | 16 MB 00:04 2024-09-16T00:15:03.512 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2024-09-16T00:15:03.512 INFO:teuthology.orchestra.run.smithi112.stdout:Total 6.6 MB/s | 38 MB 00:05 2024-09-16T00:15:03.730 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:01:04 ago on Mon 16 Sep 2024 12:13:59 AM UTC. 2024-09-16T00:15:03.862 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2024-09-16T00:15:03.866 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-09-16T00:15:03.867 INFO:teuthology.orchestra.run.smithi169.stdout:======================================================================================= 2024-09-16T00:15:03.867 INFO:teuthology.orchestra.run.smithi169.stdout: Package Arch Version Repository Size 2024-09-16T00:15:03.867 INFO:teuthology.orchestra.run.smithi169.stdout:======================================================================================= 2024-09-16T00:15:03.867 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2024-09-16T00:15:03.867 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-1633.g6d025bc2.el9 ceph-noarch 7.4 M 2024-09-16T00:15:03.867 INFO:teuthology.orchestra.run.smithi169.stdout:Installing dependencies: 2024-09-16T00:15:03.867 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-16T00:15:03.868 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-16T00:15:03.868 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-16T00:15:03.868 INFO:teuthology.orchestra.run.smithi169.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-16T00:15:03.868 INFO:teuthology.orchestra.run.smithi169.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-16T00:15:03.868 INFO:teuthology.orchestra.run.smithi169.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-16T00:15:03.868 INFO:teuthology.orchestra.run.smithi169.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-16T00:15:03.868 INFO:teuthology.orchestra.run.smithi169.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-16T00:15:03.868 INFO:teuthology.orchestra.run.smithi169.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-16T00:15:03.868 INFO:teuthology.orchestra.run.smithi169.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-16T00:15:03.868 INFO:teuthology.orchestra.run.smithi169.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-16T00:15:03.868 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:15:03.868 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2024-09-16T00:15:03.869 INFO:teuthology.orchestra.run.smithi169.stdout:======================================================================================= 2024-09-16T00:15:03.869 INFO:teuthology.orchestra.run.smithi169.stdout:Install 12 Packages 2024-09-16T00:15:03.869 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:15:03.869 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 38 M 2024-09-16T00:15:03.869 INFO:teuthology.orchestra.run.smithi169.stdout:Installed size: 212 M 2024-09-16T00:15:03.869 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2024-09-16T00:15:03.889 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2024-09-16T00:15:03.889 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2024-09-16T00:15:04.029 INFO:teuthology.orchestra.run.smithi169.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.6 MB/s | 191 kB 00:00 2024-09-16T00:15:04.096 INFO:teuthology.orchestra.run.smithi169.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 4.3 MB/s | 801 kB 00:00 2024-09-16T00:15:04.255 INFO:teuthology.orchestra.run.smithi169.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 145 kB/s | 33 kB 00:00 2024-09-16T00:15:04.322 INFO:teuthology.orchestra.run.smithi169.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 274 kB/s | 18 kB 00:00 2024-09-16T00:15:04.387 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2024-09-16T00:15:04.388 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2024-09-16T00:15:04.423 INFO:teuthology.orchestra.run.smithi169.stdout:(5/12): ceph-mgr-diskprediction-local-17.2.7-16 15 MB/s | 7.4 MB 00:00 2024-09-16T00:15:04.448 INFO:teuthology.orchestra.run.smithi169.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 307 kB/s | 38 kB 00:00 2024-09-16T00:15:04.706 INFO:teuthology.orchestra.run.smithi169.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 947 kB/s | 245 kB 00:00 2024-09-16T00:15:05.142 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2024-09-16T00:15:05.213 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-16T00:15:05.250 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-16T00:15:05.350 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-16T00:15:05.365 INFO:teuthology.orchestra.run.smithi169.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 2.4 MB/s | 3.0 MB 00:01 2024-09-16T00:15:05.393 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-16T00:15:05.490 INFO:teuthology.orchestra.run.smithi169.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.0 MB/s | 250 kB 00:00 2024-09-16T00:15:05.576 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-16T00:15:05.600 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-16T00:15:05.678 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-16T00:15:06.174 INFO:teuthology.orchestra.run.smithi169.stdout:(10/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 2.8 MB/s | 4.9 MB 00:01 2024-09-16T00:15:06.299 INFO:teuthology.orchestra.run.smithi169.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.2 MB/s | 5.1 MB 00:01 2024-09-16T00:15:06.334 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-16T00:15:06.446 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-16T00:15:07.696 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-16T00:15:08.726 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-16T00:15:08.757 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-1633.g6d025 12/12 2024-09-16T00:15:09.049 INFO:teuthology.orchestra.run.smithi169.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 4.6 MB/s | 16 MB 00:03 2024-09-16T00:15:09.050 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2024-09-16T00:15:09.050 INFO:teuthology.orchestra.run.smithi169.stdout:Total 7.4 MB/s | 38 MB 00:05 2024-09-16T00:15:09.414 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2024-09-16T00:15:09.440 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2024-09-16T00:15:09.440 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2024-09-16T00:15:09.939 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2024-09-16T00:15:09.939 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2024-09-16T00:15:10.701 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2024-09-16T00:15:10.770 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-16T00:15:10.813 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-16T00:15:10.917 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-16T00:15:10.965 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-16T00:15:11.150 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-16T00:15:11.180 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-16T00:15:11.257 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-16T00:15:11.413 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-1633.g6d025 12/12 2024-09-16T00:15:11.413 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-1633.g6d025 1/12 2024-09-16T00:15:11.413 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-16T00:15:11.413 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-16T00:15:11.413 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-16T00:15:11.413 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-16T00:15:11.413 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-16T00:15:11.413 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-16T00:15:11.413 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-16T00:15:11.413 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-16T00:15:11.413 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-16T00:15:11.413 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-16T00:15:11.834 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-16T00:15:11.834 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:15:11.834 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2024-09-16T00:15:11.834 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-diskprediction-local-2:17.2.7-1633.g6d025bc2.el9.noarch 2024-09-16T00:15:11.834 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-16T00:15:11.834 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-16T00:15:11.834 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-16T00:15:11.835 INFO:teuthology.orchestra.run.smithi112.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-16T00:15:11.835 INFO:teuthology.orchestra.run.smithi112.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-16T00:15:11.835 INFO:teuthology.orchestra.run.smithi112.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-16T00:15:11.835 INFO:teuthology.orchestra.run.smithi112.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-16T00:15:11.835 INFO:teuthology.orchestra.run.smithi112.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-16T00:15:11.835 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-16T00:15:11.835 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-16T00:15:11.835 INFO:teuthology.orchestra.run.smithi112.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-16T00:15:11.835 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:15:11.835 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2024-09-16T00:15:11.924 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-16T00:15:12.044 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-16T00:15:12.184 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install ceph-mgr-rook 2024-09-16T00:15:12.752 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:01:17 ago on Mon 16 Sep 2024 12:13:55 AM UTC. 2024-09-16T00:15:12.884 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2024-09-16T00:15:12.886 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-09-16T00:15:12.886 INFO:teuthology.orchestra.run.smithi112.stdout: Package Arch Version Repository Size 2024-09-16T00:15:12.886 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-09-16T00:15:12.886 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2024-09-16T00:15:12.886 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-rook noarch 2:17.2.7-1633.g6d025bc2.el9 ceph-noarch 47 k 2024-09-16T00:15:12.886 INFO:teuthology.orchestra.run.smithi112.stdout:Installing dependencies: 2024-09-16T00:15:12.886 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-16T00:15:12.886 INFO:teuthology.orchestra.run.smithi112.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-16T00:15:12.886 INFO:teuthology.orchestra.run.smithi112.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-16T00:15:12.886 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-16T00:15:12.886 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-16T00:15:12.886 INFO:teuthology.orchestra.run.smithi112.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-16T00:15:12.886 INFO:teuthology.orchestra.run.smithi112.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-16T00:15:12.887 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-16T00:15:12.887 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-16T00:15:12.887 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-16T00:15:12.887 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-16T00:15:12.887 INFO:teuthology.orchestra.run.smithi112.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-16T00:15:12.887 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:15:12.887 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2024-09-16T00:15:12.887 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-09-16T00:15:12.887 INFO:teuthology.orchestra.run.smithi112.stdout:Install 13 Packages 2024-09-16T00:15:12.887 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:15:12.888 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 2.2 M 2024-09-16T00:15:12.888 INFO:teuthology.orchestra.run.smithi112.stdout:Installed size: 26 M 2024-09-16T00:15:12.888 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2024-09-16T00:15:13.158 INFO:teuthology.orchestra.run.smithi112.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 245 kB/s | 19 kB 00:00 2024-09-16T00:15:13.183 INFO:teuthology.orchestra.run.smithi112.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 247 kB/s | 26 kB 00:00 2024-09-16T00:15:13.242 INFO:teuthology.orchestra.run.smithi112.stdout:(3/13): ceph-mgr-rook-17.2.7-1633.g6d025bc2.el9 292 kB/s | 47 kB 00:00 2024-09-16T00:15:13.292 INFO:teuthology.orchestra.run.smithi112.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.6 MB/s | 222 kB 00:00 2024-09-16T00:15:13.306 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-16T00:15:13.317 INFO:teuthology.orchestra.run.smithi112.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-09-16T00:15:13.351 INFO:teuthology.orchestra.run.smithi112.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 918 kB/s | 54 kB 00:00 2024-09-16T00:15:13.384 INFO:teuthology.orchestra.run.smithi112.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 481 kB/s | 32 kB 00:00 2024-09-16T00:15:13.410 INFO:teuthology.orchestra.run.smithi112.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 248 kB/s | 14 kB 00:00 2024-09-16T00:15:13.435 INFO:teuthology.orchestra.run.smithi112.stdout:(9/13): python3-google-auth-2.34.0-1.el9.noarch 4.4 MB/s | 220 kB 00:00 2024-09-16T00:15:13.460 INFO:teuthology.orchestra.run.smithi112.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-16T00:15:13.485 INFO:teuthology.orchestra.run.smithi112.stdout:(11/13): python3-pyasn1-modules-0.4.8-6.el9.noa 1.1 MB/s | 279 kB 00:00 2024-09-16T00:15:13.510 INFO:teuthology.orchestra.run.smithi112.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.7 MB/s | 90 kB 00:00 2024-09-16T00:15:13.544 INFO:teuthology.orchestra.run.smithi112.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 7.6 MB/s | 1.0 MB 00:00 2024-09-16T00:15:13.546 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2024-09-16T00:15:13.546 INFO:teuthology.orchestra.run.smithi112.stdout:Total 3.4 MB/s | 2.2 MB 00:00 2024-09-16T00:15:13.609 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2024-09-16T00:15:13.623 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2024-09-16T00:15:13.624 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2024-09-16T00:15:13.709 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2024-09-16T00:15:13.711 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2024-09-16T00:15:14.216 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2024-09-16T00:15:14.344 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-16T00:15:14.373 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-1633.g6d025 12/12 2024-09-16T00:15:14.383 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-16T00:15:14.437 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-16T00:15:14.500 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-16T00:15:14.537 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-16T00:15:14.579 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-16T00:15:14.682 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-16T00:15:14.783 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-16T00:15:14.839 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-16T00:15:15.559 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-16T00:15:15.603 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-16T00:15:15.637 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-16T00:15:15.681 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-16T00:15:15.706 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-rook-2:17.2.7-1633.g6d025bc2.el9.noarch 13/13 2024-09-16T00:15:16.539 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-1633.g6d025bc2.el9.noarch 13/13 2024-09-16T00:15:16.539 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-rook-2:17.2.7-1633.g6d025bc2.el9.noarch 1/13 2024-09-16T00:15:16.539 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-16T00:15:16.539 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-16T00:15:16.540 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-16T00:15:16.540 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-16T00:15:16.540 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-16T00:15:16.540 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-16T00:15:16.540 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-16T00:15:16.540 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-16T00:15:16.541 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-16T00:15:16.541 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-16T00:15:16.541 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-16T00:15:16.865 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-16T00:15:16.865 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:15:16.865 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2024-09-16T00:15:16.865 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-rook-2:17.2.7-1633.g6d025bc2.el9.noarch 2024-09-16T00:15:16.865 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-16T00:15:16.865 INFO:teuthology.orchestra.run.smithi112.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-16T00:15:16.865 INFO:teuthology.orchestra.run.smithi112.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-16T00:15:16.865 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-16T00:15:16.865 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-16T00:15:16.865 INFO:teuthology.orchestra.run.smithi112.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-16T00:15:16.865 INFO:teuthology.orchestra.run.smithi112.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-16T00:15:16.866 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-16T00:15:16.866 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-16T00:15:16.866 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-16T00:15:16.866 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-16T00:15:16.866 INFO:teuthology.orchestra.run.smithi112.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-16T00:15:16.866 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:15:16.866 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2024-09-16T00:15:16.892 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-1633.g6d025 12/12 2024-09-16T00:15:16.893 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-1633.g6d025 1/12 2024-09-16T00:15:16.893 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-16T00:15:16.893 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-16T00:15:16.893 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-16T00:15:16.893 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-16T00:15:16.893 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-16T00:15:16.893 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-16T00:15:16.893 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-16T00:15:16.893 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-16T00:15:16.893 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-16T00:15:16.893 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-16T00:15:17.106 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install ceph-mgr-cephadm 2024-09-16T00:15:17.312 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-16T00:15:17.312 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:15:17.313 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2024-09-16T00:15:17.313 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-diskprediction-local-2:17.2.7-1633.g6d025bc2.el9.noarch 2024-09-16T00:15:17.313 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-16T00:15:17.313 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-16T00:15:17.313 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-16T00:15:17.313 INFO:teuthology.orchestra.run.smithi169.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-16T00:15:17.313 INFO:teuthology.orchestra.run.smithi169.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-16T00:15:17.313 INFO:teuthology.orchestra.run.smithi169.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-16T00:15:17.313 INFO:teuthology.orchestra.run.smithi169.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-16T00:15:17.313 INFO:teuthology.orchestra.run.smithi169.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-16T00:15:17.313 INFO:teuthology.orchestra.run.smithi169.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-16T00:15:17.313 INFO:teuthology.orchestra.run.smithi169.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-16T00:15:17.314 INFO:teuthology.orchestra.run.smithi169.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-16T00:15:17.314 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:15:17.314 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-09-16T00:15:17.672 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:01:22 ago on Mon 16 Sep 2024 12:13:55 AM UTC. 2024-09-16T00:15:17.692 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install ceph-mgr-rook 2024-09-16T00:15:17.804 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2024-09-16T00:15:17.805 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-09-16T00:15:17.805 INFO:teuthology.orchestra.run.smithi112.stdout: Package Arch Version Repository Size 2024-09-16T00:15:17.805 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-09-16T00:15:17.805 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2024-09-16T00:15:17.805 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-cephadm noarch 2:17.2.7-1633.g6d025bc2.el9 ceph-noarch 124 k 2024-09-16T00:15:17.805 INFO:teuthology.orchestra.run.smithi112.stdout:Installing dependencies: 2024-09-16T00:15:17.805 INFO:teuthology.orchestra.run.smithi112.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-16T00:15:17.805 INFO:teuthology.orchestra.run.smithi112.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-16T00:15:17.805 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-16T00:15:17.805 INFO:teuthology.orchestra.run.smithi112.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-16T00:15:17.806 INFO:teuthology.orchestra.run.smithi112.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-16T00:15:17.806 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:15:17.806 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2024-09-16T00:15:17.806 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-09-16T00:15:17.806 INFO:teuthology.orchestra.run.smithi112.stdout:Install 6 Packages 2024-09-16T00:15:17.806 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:15:17.806 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 7.0 M 2024-09-16T00:15:17.807 INFO:teuthology.orchestra.run.smithi112.stdout:Installed size: 33 M 2024-09-16T00:15:17.807 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2024-09-16T00:15:18.079 INFO:teuthology.orchestra.run.smithi112.stdout:(1/6): ceph-mgr-cephadm-17.2.7-1633.g6d025bc2.e 643 kB/s | 124 kB 00:00 2024-09-16T00:15:18.104 INFO:teuthology.orchestra.run.smithi112.stdout:(2/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.1 MB/s | 248 kB 00:00 2024-09-16T00:15:18.154 INFO:teuthology.orchestra.run.smithi112.stdout:(3/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 7.1 MB/s | 548 kB 00:00 2024-09-16T00:15:18.179 INFO:teuthology.orchestra.run.smithi112.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 767 kB/s | 58 kB 00:00 2024-09-16T00:15:18.196 INFO:teuthology.orchestra.run.smithi112.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.7 MB/s | 74 kB 00:00 2024-09-16T00:15:18.267 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:01:19 ago on Mon 16 Sep 2024 12:13:59 AM UTC. 2024-09-16T00:15:18.380 INFO:teuthology.orchestra.run.smithi112.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 12 MB/s | 6.0 MB 00:00 2024-09-16T00:15:18.382 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2024-09-16T00:15:18.382 INFO:teuthology.orchestra.run.smithi112.stdout:Total 12 MB/s | 7.0 MB 00:00 2024-09-16T00:15:18.401 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-09-16T00:15:18.402 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T00:15:18.402 INFO:teuthology.orchestra.run.smithi169.stdout: Package Arch Version Repository Size 2024-09-16T00:15:18.402 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T00:15:18.402 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2024-09-16T00:15:18.402 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-rook noarch 2:17.2.7-1633.g6d025bc2.el9 ceph-noarch 47 k 2024-09-16T00:15:18.402 INFO:teuthology.orchestra.run.smithi169.stdout:Installing dependencies: 2024-09-16T00:15:18.402 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-16T00:15:18.402 INFO:teuthology.orchestra.run.smithi169.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-16T00:15:18.402 INFO:teuthology.orchestra.run.smithi169.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-16T00:15:18.402 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-16T00:15:18.403 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-16T00:15:18.403 INFO:teuthology.orchestra.run.smithi169.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-16T00:15:18.403 INFO:teuthology.orchestra.run.smithi169.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-16T00:15:18.403 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-16T00:15:18.403 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-16T00:15:18.403 INFO:teuthology.orchestra.run.smithi169.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-16T00:15:18.403 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-16T00:15:18.403 INFO:teuthology.orchestra.run.smithi169.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-16T00:15:18.403 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:15:18.403 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2024-09-16T00:15:18.403 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T00:15:18.403 INFO:teuthology.orchestra.run.smithi169.stdout:Install 13 Packages 2024-09-16T00:15:18.403 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:15:18.404 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 2.2 M 2024-09-16T00:15:18.404 INFO:teuthology.orchestra.run.smithi169.stdout:Installed size: 26 M 2024-09-16T00:15:18.404 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2024-09-16T00:15:18.451 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2024-09-16T00:15:18.462 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2024-09-16T00:15:18.462 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2024-09-16T00:15:18.563 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2024-09-16T00:15:18.564 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2024-09-16T00:15:18.730 INFO:teuthology.orchestra.run.smithi169.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 376 kB/s | 26 kB 00:00 2024-09-16T00:15:18.756 INFO:teuthology.orchestra.run.smithi169.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 205 kB/s | 19 kB 00:00 2024-09-16T00:15:18.831 INFO:teuthology.orchestra.run.smithi169.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.9 MB/s | 222 kB 00:00 2024-09-16T00:15:18.856 INFO:teuthology.orchestra.run.smithi169.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.6 MB/s | 159 kB 00:00 2024-09-16T00:15:18.872 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2024-09-16T00:15:18.881 INFO:teuthology.orchestra.run.smithi169.stdout:(5/13): ceph-mgr-rook-17.2.7-1633.g6d025bc2.el9 216 kB/s | 47 kB 00:00 2024-09-16T00:15:18.907 INFO:teuthology.orchestra.run.smithi169.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 1.0 MB/s | 54 kB 00:00 2024-09-16T00:15:18.932 INFO:teuthology.orchestra.run.smithi169.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 2.7 MB/s | 279 kB 00:00 2024-09-16T00:15:18.982 INFO:teuthology.orchestra.run.smithi169.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 320 kB/s | 32 kB 00:00 2024-09-16T00:15:19.007 INFO:teuthology.orchestra.run.smithi169.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 140 kB/s | 14 kB 00:00 2024-09-16T00:15:19.029 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-16T00:15:19.041 INFO:teuthology.orchestra.run.smithi169.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 1.7 MB/s | 59 kB 00:00 2024-09-16T00:15:19.066 INFO:teuthology.orchestra.run.smithi169.stdout:(11/13): python3-google-auth-2.34.0-1.el9.noarc 1.8 MB/s | 220 kB 00:00 2024-09-16T00:15:19.081 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-16T00:15:19.092 INFO:teuthology.orchestra.run.smithi169.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.7 MB/s | 90 kB 00:00 2024-09-16T00:15:19.134 INFO:teuthology.orchestra.run.smithi169.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 6.8 MB/s | 1.0 MB 00:00 2024-09-16T00:15:19.137 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2024-09-16T00:15:19.137 INFO:teuthology.orchestra.run.smithi169.stdout:Total 3.0 MB/s | 2.2 MB 00:00 2024-09-16T00:15:19.199 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2024-09-16T00:15:19.214 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2024-09-16T00:15:19.214 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2024-09-16T00:15:19.303 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2024-09-16T00:15:19.304 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2024-09-16T00:15:19.605 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-16T00:15:19.717 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-16T00:15:19.782 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-16T00:15:19.810 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-1633.g6d025bc2.el9.noarch 6/6 2024-09-16T00:15:19.814 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2024-09-16T00:15:19.989 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-16T00:15:20.051 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-16T00:15:20.107 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-16T00:15:20.144 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-16T00:15:20.185 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-16T00:15:20.290 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-16T00:15:20.398 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-16T00:15:20.446 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-16T00:15:20.560 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-1633.g6d025bc2.el9.noarch 6/6 2024-09-16T00:15:20.560 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-1633.g6d025bc2.el9.noarch 1/6 2024-09-16T00:15:20.561 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-16T00:15:20.561 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-16T00:15:20.561 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-16T00:15:20.561 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-16T00:15:20.846 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-16T00:15:20.847 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:15:20.847 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2024-09-16T00:15:20.847 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-cephadm-2:17.2.7-1633.g6d025bc2.el9.noarch 2024-09-16T00:15:20.847 INFO:teuthology.orchestra.run.smithi112.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-16T00:15:20.847 INFO:teuthology.orchestra.run.smithi112.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-16T00:15:20.847 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-16T00:15:20.847 INFO:teuthology.orchestra.run.smithi112.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-16T00:15:20.847 INFO:teuthology.orchestra.run.smithi112.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-16T00:15:20.847 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:15:20.847 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2024-09-16T00:15:21.071 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install ceph-fuse 2024-09-16T00:15:21.161 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-16T00:15:21.210 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-16T00:15:21.243 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-16T00:15:21.289 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-16T00:15:21.313 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-rook-2:17.2.7-1633.g6d025bc2.el9.noarch 13/13 2024-09-16T00:15:21.640 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:01:26 ago on Mon 16 Sep 2024 12:13:55 AM UTC. 2024-09-16T00:15:21.773 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2024-09-16T00:15:21.774 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-09-16T00:15:21.774 INFO:teuthology.orchestra.run.smithi112.stdout: Package Arch Version Repository Size 2024-09-16T00:15:21.774 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-09-16T00:15:21.774 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2024-09-16T00:15:21.774 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-fuse x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 814 k 2024-09-16T00:15:21.774 INFO:teuthology.orchestra.run.smithi112.stdout:Installing dependencies: 2024-09-16T00:15:21.774 INFO:teuthology.orchestra.run.smithi112.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-16T00:15:21.774 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:15:21.774 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2024-09-16T00:15:21.774 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-09-16T00:15:21.775 INFO:teuthology.orchestra.run.smithi112.stdout:Install 2 Packages 2024-09-16T00:15:21.775 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:15:21.775 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 893 k 2024-09-16T00:15:21.775 INFO:teuthology.orchestra.run.smithi112.stdout:Installed size: 2.6 M 2024-09-16T00:15:21.775 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2024-09-16T00:15:21.981 INFO:teuthology.orchestra.run.smithi112.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 503 kB/s | 80 kB 00:00 2024-09-16T00:15:22.093 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-1633.g6d025bc2.el9.noarch 13/13 2024-09-16T00:15:22.093 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-rook-2:17.2.7-1633.g6d025bc2.el9.noarch 1/13 2024-09-16T00:15:22.093 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-16T00:15:22.094 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-16T00:15:22.094 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-16T00:15:22.094 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-16T00:15:22.094 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-16T00:15:22.094 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-16T00:15:22.094 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-16T00:15:22.094 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-16T00:15:22.094 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-16T00:15:22.094 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-16T00:15:22.094 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-16T00:15:22.115 INFO:teuthology.orchestra.run.smithi112.stdout:(2/2): ceph-fuse-17.2.7-1633.g6d025bc2.el9.x86_ 2.7 MB/s | 814 kB 00:00 2024-09-16T00:15:22.115 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2024-09-16T00:15:22.115 INFO:teuthology.orchestra.run.smithi112.stdout:Total 2.6 MB/s | 893 kB 00:00 2024-09-16T00:15:22.127 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2024-09-16T00:15:22.149 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2024-09-16T00:15:22.149 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2024-09-16T00:15:22.227 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2024-09-16T00:15:22.228 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2024-09-16T00:15:22.411 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2024-09-16T00:15:22.425 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-16T00:15:22.425 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:15:22.426 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2024-09-16T00:15:22.426 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-rook-2:17.2.7-1633.g6d025bc2.el9.noarch 2024-09-16T00:15:22.426 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-16T00:15:22.426 INFO:teuthology.orchestra.run.smithi169.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-16T00:15:22.426 INFO:teuthology.orchestra.run.smithi169.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-16T00:15:22.426 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-16T00:15:22.426 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-16T00:15:22.426 INFO:teuthology.orchestra.run.smithi169.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-16T00:15:22.426 INFO:teuthology.orchestra.run.smithi169.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-16T00:15:22.426 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-16T00:15:22.426 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-16T00:15:22.427 INFO:teuthology.orchestra.run.smithi169.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-16T00:15:22.427 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-16T00:15:22.427 INFO:teuthology.orchestra.run.smithi169.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-16T00:15:22.427 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:15:22.427 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-09-16T00:15:22.552 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-16T00:15:22.622 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-fuse-2:17.2.7-1633.g6d025bc2.el9.x86_64 2/2 2024-09-16T00:15:22.675 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install ceph-mgr-cephadm 2024-09-16T00:15:23.252 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:01:24 ago on Mon 16 Sep 2024 12:13:59 AM UTC. 2024-09-16T00:15:23.275 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-fuse-2:17.2.7-1633.g6d025bc2.el9.x86_64 2/2 2024-09-16T00:15:23.275 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-fuse-2:17.2.7-1633.g6d025bc2.el9.x86_64 1/2 2024-09-16T00:15:23.387 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-09-16T00:15:23.387 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T00:15:23.387 INFO:teuthology.orchestra.run.smithi169.stdout: Package Arch Version Repository Size 2024-09-16T00:15:23.387 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T00:15:23.388 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2024-09-16T00:15:23.388 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-cephadm noarch 2:17.2.7-1633.g6d025bc2.el9 ceph-noarch 124 k 2024-09-16T00:15:23.388 INFO:teuthology.orchestra.run.smithi169.stdout:Installing dependencies: 2024-09-16T00:15:23.388 INFO:teuthology.orchestra.run.smithi169.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-16T00:15:23.388 INFO:teuthology.orchestra.run.smithi169.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-16T00:15:23.388 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-16T00:15:23.388 INFO:teuthology.orchestra.run.smithi169.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-16T00:15:23.388 INFO:teuthology.orchestra.run.smithi169.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-16T00:15:23.388 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:15:23.388 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2024-09-16T00:15:23.388 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T00:15:23.389 INFO:teuthology.orchestra.run.smithi169.stdout:Install 6 Packages 2024-09-16T00:15:23.389 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:15:23.389 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 7.0 M 2024-09-16T00:15:23.389 INFO:teuthology.orchestra.run.smithi169.stdout:Installed size: 33 M 2024-09-16T00:15:23.389 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2024-09-16T00:15:23.532 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-16T00:15:23.532 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:15:23.532 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2024-09-16T00:15:23.532 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-fuse-2:17.2.7-1633.g6d025bc2.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-16T00:15:23.532 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:15:23.532 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2024-09-16T00:15:23.819 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install librados-devel 2024-09-16T00:15:23.935 INFO:teuthology.orchestra.run.smithi169.stdout:(1/6): ceph-mgr-cephadm-17.2.7-1633.g6d025bc2.e 608 kB/s | 124 kB 00:00 2024-09-16T00:15:23.986 INFO:teuthology.orchestra.run.smithi169.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 11 MB/s | 548 kB 00:00 2024-09-16T00:15:24.011 INFO:teuthology.orchestra.run.smithi169.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 2.2 MB/s | 58 kB 00:00 2024-09-16T00:15:24.036 INFO:teuthology.orchestra.run.smithi169.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 2.9 MB/s | 74 kB 00:00 2024-09-16T00:15:24.396 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:01:29 ago on Mon 16 Sep 2024 12:13:55 AM UTC. 2024-09-16T00:15:24.527 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2024-09-16T00:15:24.528 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-09-16T00:15:24.528 INFO:teuthology.orchestra.run.smithi112.stdout: Package Arch Version Repo Size 2024-09-16T00:15:24.528 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-09-16T00:15:24.528 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2024-09-16T00:15:24.528 INFO:teuthology.orchestra.run.smithi112.stdout: librados-devel x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 126 k 2024-09-16T00:15:24.528 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:15:24.528 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2024-09-16T00:15:24.528 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-09-16T00:15:24.528 INFO:teuthology.orchestra.run.smithi112.stdout:Install 1 Package 2024-09-16T00:15:24.528 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:15:24.529 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 126 k 2024-09-16T00:15:24.529 INFO:teuthology.orchestra.run.smithi112.stdout:Installed size: 448 k 2024-09-16T00:15:24.529 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2024-09-16T00:15:24.645 INFO:teuthology.orchestra.run.smithi169.stdout:(5/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 272 kB/s | 248 kB 00:00 2024-09-16T00:15:24.743 INFO:teuthology.orchestra.run.smithi112.stdout:librados-devel-17.2.7-1633.g6d025bc2.el9.x86_64 589 kB/s | 126 kB 00:00 2024-09-16T00:15:24.744 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2024-09-16T00:15:24.744 INFO:teuthology.orchestra.run.smithi112.stdout:Total 585 kB/s | 126 kB 00:00 2024-09-16T00:15:24.744 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2024-09-16T00:15:24.750 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2024-09-16T00:15:24.750 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2024-09-16T00:15:24.814 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2024-09-16T00:15:24.814 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2024-09-16T00:15:24.985 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2024-09-16T00:15:25.053 INFO:teuthology.orchestra.run.smithi169.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 4.5 MB/s | 6.0 MB 00:01 2024-09-16T00:15:25.055 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2024-09-16T00:15:25.056 INFO:teuthology.orchestra.run.smithi169.stdout:Total 4.2 MB/s | 7.0 MB 00:01 2024-09-16T00:15:25.081 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librados-devel-2:17.2.7-1633.g6d025bc2.el9.x86_64 1/1 2024-09-16T00:15:25.129 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2024-09-16T00:15:25.141 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2024-09-16T00:15:25.141 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2024-09-16T00:15:25.241 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2024-09-16T00:15:25.241 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2024-09-16T00:15:25.531 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librados-devel-2:17.2.7-1633.g6d025bc2.el9.x86_64 1/1 2024-09-16T00:15:25.549 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2024-09-16T00:15:25.735 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-16T00:15:25.789 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-16T00:15:25.795 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librados-devel-2:17.2.7-1633.g6d025bc2.el9.x86_64 1/1 2024-09-16T00:15:25.795 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:15:25.795 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2024-09-16T00:15:25.795 INFO:teuthology.orchestra.run.smithi112.stdout: librados-devel-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-16T00:15:25.795 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:15:25.795 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2024-09-16T00:15:26.027 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install libcephfs2 2024-09-16T00:15:26.325 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-16T00:15:26.407 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-16T00:15:26.464 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-16T00:15:26.492 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-1633.g6d025bc2.el9.noarch 6/6 2024-09-16T00:15:26.606 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:01:31 ago on Mon 16 Sep 2024 12:13:55 AM UTC. 2024-09-16T00:15:26.700 INFO:teuthology.orchestra.run.smithi112.stdout:Package libcephfs2-2:17.2.7-1633.g6d025bc2.el9.x86_64 is already installed. 2024-09-16T00:15:26.744 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2024-09-16T00:15:26.745 INFO:teuthology.orchestra.run.smithi112.stdout:Nothing to do. 2024-09-16T00:15:26.745 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2024-09-16T00:15:26.809 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install libcephfs-devel 2024-09-16T00:15:27.250 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-1633.g6d025bc2.el9.noarch 6/6 2024-09-16T00:15:27.251 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-1633.g6d025bc2.el9.noarch 1/6 2024-09-16T00:15:27.251 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-16T00:15:27.251 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-16T00:15:27.251 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-16T00:15:27.251 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-16T00:15:27.371 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:01:32 ago on Mon 16 Sep 2024 12:13:55 AM UTC. 2024-09-16T00:15:27.502 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2024-09-16T00:15:27.502 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-09-16T00:15:27.502 INFO:teuthology.orchestra.run.smithi112.stdout: Package Arch Version Repo Size 2024-09-16T00:15:27.502 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-09-16T00:15:27.503 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2024-09-16T00:15:27.503 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs-devel x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 25 k 2024-09-16T00:15:27.503 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:15:27.503 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2024-09-16T00:15:27.503 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-09-16T00:15:27.503 INFO:teuthology.orchestra.run.smithi112.stdout:Install 1 Package 2024-09-16T00:15:27.503 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:15:27.503 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 25 k 2024-09-16T00:15:27.504 INFO:teuthology.orchestra.run.smithi112.stdout:Installed size: 110 k 2024-09-16T00:15:27.504 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2024-09-16T00:15:27.506 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-16T00:15:27.506 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:15:27.506 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2024-09-16T00:15:27.506 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-cephadm-2:17.2.7-1633.g6d025bc2.el9.noarch 2024-09-16T00:15:27.507 INFO:teuthology.orchestra.run.smithi169.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-16T00:15:27.507 INFO:teuthology.orchestra.run.smithi169.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-16T00:15:27.507 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-16T00:15:27.507 INFO:teuthology.orchestra.run.smithi169.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-16T00:15:27.507 INFO:teuthology.orchestra.run.smithi169.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-16T00:15:27.507 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:15:27.507 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-09-16T00:15:27.654 INFO:teuthology.orchestra.run.smithi112.stdout:libcephfs-devel-17.2.7-1633.g6d025bc2.el9.x86_6 166 kB/s | 25 kB 00:00 2024-09-16T00:15:27.655 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2024-09-16T00:15:27.655 INFO:teuthology.orchestra.run.smithi112.stdout:Total 164 kB/s | 25 kB 00:00 2024-09-16T00:15:27.655 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2024-09-16T00:15:27.658 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2024-09-16T00:15:27.659 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2024-09-16T00:15:27.660 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2024-09-16T00:15:27.660 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2024-09-16T00:15:27.741 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2024-09-16T00:15:27.784 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install ceph-fuse 2024-09-16T00:15:27.835 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libcephfs-devel-2:17.2.7-1633.g6d025bc2.el9.x86_64 1/1 2024-09-16T00:15:28.166 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-1633.g6d025bc2.el9.x86_64 1/1 2024-09-16T00:15:28.352 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:01:29 ago on Mon 16 Sep 2024 12:13:59 AM UTC. 2024-09-16T00:15:28.412 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libcephfs-devel-2:17.2.7-1633.g6d025bc2.el9.x86_64 1/1 2024-09-16T00:15:28.412 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:15:28.412 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2024-09-16T00:15:28.412 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs-devel-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-16T00:15:28.413 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:15:28.413 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2024-09-16T00:15:28.483 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-09-16T00:15:28.484 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T00:15:28.484 INFO:teuthology.orchestra.run.smithi169.stdout: Package Arch Version Repository Size 2024-09-16T00:15:28.484 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T00:15:28.484 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2024-09-16T00:15:28.484 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-fuse x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 814 k 2024-09-16T00:15:28.484 INFO:teuthology.orchestra.run.smithi169.stdout:Installing dependencies: 2024-09-16T00:15:28.484 INFO:teuthology.orchestra.run.smithi169.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-16T00:15:28.484 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:15:28.484 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2024-09-16T00:15:28.485 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T00:15:28.485 INFO:teuthology.orchestra.run.smithi169.stdout:Install 2 Packages 2024-09-16T00:15:28.485 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:15:28.485 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 893 k 2024-09-16T00:15:28.485 INFO:teuthology.orchestra.run.smithi169.stdout:Installed size: 2.6 M 2024-09-16T00:15:28.485 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2024-09-16T00:15:28.620 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install librados2 2024-09-16T00:15:28.880 INFO:teuthology.orchestra.run.smithi169.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 565 kB/s | 80 kB 00:00 2024-09-16T00:15:29.013 INFO:teuthology.orchestra.run.smithi169.stdout:(2/2): ceph-fuse-17.2.7-1633.g6d025bc2.el9.x86_ 2.9 MB/s | 814 kB 00:00 2024-09-16T00:15:29.014 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2024-09-16T00:15:29.014 INFO:teuthology.orchestra.run.smithi169.stdout:Total 1.6 MB/s | 893 kB 00:00 2024-09-16T00:15:29.025 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2024-09-16T00:15:29.047 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2024-09-16T00:15:29.048 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2024-09-16T00:15:29.128 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2024-09-16T00:15:29.128 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2024-09-16T00:15:29.186 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:01:34 ago on Mon 16 Sep 2024 12:13:55 AM UTC. 2024-09-16T00:15:29.278 INFO:teuthology.orchestra.run.smithi112.stdout:Package librados2-2:17.2.7-1633.g6d025bc2.el9.x86_64 is already installed. 2024-09-16T00:15:29.321 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2024-09-16T00:15:29.322 INFO:teuthology.orchestra.run.smithi112.stdout:Nothing to do. 2024-09-16T00:15:29.322 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2024-09-16T00:15:29.325 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2024-09-16T00:15:29.403 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install librbd1 2024-09-16T00:15:29.456 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-16T00:15:29.520 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-fuse-2:17.2.7-1633.g6d025bc2.el9.x86_64 2/2 2024-09-16T00:15:29.961 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:01:34 ago on Mon 16 Sep 2024 12:13:55 AM UTC. 2024-09-16T00:15:30.052 INFO:teuthology.orchestra.run.smithi112.stdout:Package librbd1-2:17.2.7-1633.g6d025bc2.el9.x86_64 is already installed. 2024-09-16T00:15:30.096 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2024-09-16T00:15:30.097 INFO:teuthology.orchestra.run.smithi112.stdout:Nothing to do. 2024-09-16T00:15:30.097 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2024-09-16T00:15:30.143 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-fuse-2:17.2.7-1633.g6d025bc2.el9.x86_64 2/2 2024-09-16T00:15:30.143 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-fuse-2:17.2.7-1633.g6d025bc2.el9.x86_64 1/2 2024-09-16T00:15:30.170 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install python3-rados 2024-09-16T00:15:30.408 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-16T00:15:30.409 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:15:30.409 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2024-09-16T00:15:30.409 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-fuse-2:17.2.7-1633.g6d025bc2.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-16T00:15:30.409 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:15:30.409 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-09-16T00:15:30.715 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install librados-devel 2024-09-16T00:15:30.733 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:01:35 ago on Mon 16 Sep 2024 12:13:55 AM UTC. 2024-09-16T00:15:30.821 INFO:teuthology.orchestra.run.smithi112.stdout:Package python3-rados-2:17.2.7-1633.g6d025bc2.el9.x86_64 is already installed. 2024-09-16T00:15:30.864 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2024-09-16T00:15:30.865 INFO:teuthology.orchestra.run.smithi112.stdout:Nothing to do. 2024-09-16T00:15:30.865 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2024-09-16T00:15:30.944 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install python3-rgw 2024-09-16T00:15:31.286 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:01:32 ago on Mon 16 Sep 2024 12:13:59 AM UTC. 2024-09-16T00:15:31.423 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-09-16T00:15:31.424 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T00:15:31.424 INFO:teuthology.orchestra.run.smithi169.stdout: Package Arch Version Repo Size 2024-09-16T00:15:31.424 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T00:15:31.424 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2024-09-16T00:15:31.424 INFO:teuthology.orchestra.run.smithi169.stdout: librados-devel x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 126 k 2024-09-16T00:15:31.424 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:15:31.424 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2024-09-16T00:15:31.425 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T00:15:31.425 INFO:teuthology.orchestra.run.smithi169.stdout:Install 1 Package 2024-09-16T00:15:31.425 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:15:31.425 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 126 k 2024-09-16T00:15:31.425 INFO:teuthology.orchestra.run.smithi169.stdout:Installed size: 448 k 2024-09-16T00:15:31.425 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2024-09-16T00:15:31.508 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:01:36 ago on Mon 16 Sep 2024 12:13:55 AM UTC. 2024-09-16T00:15:31.597 INFO:teuthology.orchestra.run.smithi112.stdout:Package python3-rgw-2:17.2.7-1633.g6d025bc2.el9.x86_64 is already installed. 2024-09-16T00:15:31.609 INFO:teuthology.orchestra.run.smithi169.stdout:librados-devel-17.2.7-1633.g6d025bc2.el9.x86_64 685 kB/s | 126 kB 00:00 2024-09-16T00:15:31.610 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2024-09-16T00:15:31.610 INFO:teuthology.orchestra.run.smithi169.stdout:Total 680 kB/s | 126 kB 00:00 2024-09-16T00:15:31.610 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2024-09-16T00:15:31.616 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2024-09-16T00:15:31.616 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2024-09-16T00:15:31.640 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2024-09-16T00:15:31.641 INFO:teuthology.orchestra.run.smithi112.stdout:Nothing to do. 2024-09-16T00:15:31.641 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2024-09-16T00:15:31.678 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2024-09-16T00:15:31.679 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2024-09-16T00:15:31.716 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install python3-cephfs 2024-09-16T00:15:31.835 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2024-09-16T00:15:31.923 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : librados-devel-2:17.2.7-1633.g6d025bc2.el9.x86_64 1/1 2024-09-16T00:15:32.279 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:01:37 ago on Mon 16 Sep 2024 12:13:55 AM UTC. 2024-09-16T00:15:32.342 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librados-devel-2:17.2.7-1633.g6d025bc2.el9.x86_64 1/1 2024-09-16T00:15:32.368 INFO:teuthology.orchestra.run.smithi112.stdout:Package python3-cephfs-2:17.2.7-1633.g6d025bc2.el9.x86_64 is already installed. 2024-09-16T00:15:32.411 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2024-09-16T00:15:32.412 INFO:teuthology.orchestra.run.smithi112.stdout:Nothing to do. 2024-09-16T00:15:32.412 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2024-09-16T00:15:32.486 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install python3-rbd 2024-09-16T00:15:32.578 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librados-devel-2:17.2.7-1633.g6d025bc2.el9.x86_64 1/1 2024-09-16T00:15:32.578 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:15:32.578 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2024-09-16T00:15:32.578 INFO:teuthology.orchestra.run.smithi169.stdout: librados-devel-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-16T00:15:32.579 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:15:32.579 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-09-16T00:15:32.844 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install libcephfs2 2024-09-16T00:15:33.049 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:01:38 ago on Mon 16 Sep 2024 12:13:55 AM UTC. 2024-09-16T00:15:33.140 INFO:teuthology.orchestra.run.smithi112.stdout:Package python3-rbd-2:17.2.7-1633.g6d025bc2.el9.x86_64 is already installed. 2024-09-16T00:15:33.183 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2024-09-16T00:15:33.184 INFO:teuthology.orchestra.run.smithi112.stdout:Nothing to do. 2024-09-16T00:15:33.184 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2024-09-16T00:15:33.254 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install rbd-fuse 2024-09-16T00:15:33.422 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:01:34 ago on Mon 16 Sep 2024 12:13:59 AM UTC. 2024-09-16T00:15:33.512 INFO:teuthology.orchestra.run.smithi169.stdout:Package libcephfs2-2:17.2.7-1633.g6d025bc2.el9.x86_64 is already installed. 2024-09-16T00:15:33.557 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-09-16T00:15:33.558 INFO:teuthology.orchestra.run.smithi169.stdout:Nothing to do. 2024-09-16T00:15:33.558 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-09-16T00:15:33.638 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install libcephfs-devel 2024-09-16T00:15:33.827 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:01:38 ago on Mon 16 Sep 2024 12:13:55 AM UTC. 2024-09-16T00:15:33.960 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2024-09-16T00:15:33.961 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-09-16T00:15:33.961 INFO:teuthology.orchestra.run.smithi112.stdout: Package Architecture Version Repository Size 2024-09-16T00:15:33.961 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-09-16T00:15:33.961 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2024-09-16T00:15:33.962 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-fuse x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 82 k 2024-09-16T00:15:33.962 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:15:33.962 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2024-09-16T00:15:33.962 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-09-16T00:15:33.962 INFO:teuthology.orchestra.run.smithi112.stdout:Install 1 Package 2024-09-16T00:15:33.962 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:15:33.962 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 82 k 2024-09-16T00:15:33.962 INFO:teuthology.orchestra.run.smithi112.stdout:Installed size: 222 k 2024-09-16T00:15:33.962 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2024-09-16T00:15:34.157 INFO:teuthology.orchestra.run.smithi112.stdout:rbd-fuse-17.2.7-1633.g6d025bc2.el9.x86_64.rpm 418 kB/s | 82 kB 00:00 2024-09-16T00:15:34.158 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2024-09-16T00:15:34.158 INFO:teuthology.orchestra.run.smithi112.stdout:Total 416 kB/s | 82 kB 00:00 2024-09-16T00:15:34.158 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2024-09-16T00:15:34.165 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2024-09-16T00:15:34.165 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2024-09-16T00:15:34.204 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:01:35 ago on Mon 16 Sep 2024 12:13:59 AM UTC. 2024-09-16T00:15:34.228 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2024-09-16T00:15:34.229 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2024-09-16T00:15:34.336 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-09-16T00:15:34.337 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T00:15:34.337 INFO:teuthology.orchestra.run.smithi169.stdout: Package Arch Version Repo Size 2024-09-16T00:15:34.337 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T00:15:34.337 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2024-09-16T00:15:34.337 INFO:teuthology.orchestra.run.smithi169.stdout: libcephfs-devel x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 25 k 2024-09-16T00:15:34.337 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:15:34.338 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2024-09-16T00:15:34.338 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T00:15:34.338 INFO:teuthology.orchestra.run.smithi169.stdout:Install 1 Package 2024-09-16T00:15:34.338 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:15:34.338 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 25 k 2024-09-16T00:15:34.338 INFO:teuthology.orchestra.run.smithi169.stdout:Installed size: 110 k 2024-09-16T00:15:34.338 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2024-09-16T00:15:34.372 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2024-09-16T00:15:34.480 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : rbd-fuse-2:17.2.7-1633.g6d025bc2.el9.x86_64 1/1 2024-09-16T00:15:34.487 INFO:teuthology.orchestra.run.smithi169.stdout:libcephfs-devel-17.2.7-1633.g6d025bc2.el9.x86_6 167 kB/s | 25 kB 00:00 2024-09-16T00:15:34.488 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2024-09-16T00:15:34.488 INFO:teuthology.orchestra.run.smithi169.stdout:Total 166 kB/s | 25 kB 00:00 2024-09-16T00:15:34.488 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2024-09-16T00:15:34.491 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2024-09-16T00:15:34.491 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2024-09-16T00:15:34.493 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2024-09-16T00:15:34.493 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2024-09-16T00:15:34.575 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2024-09-16T00:15:34.692 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libcephfs-devel-2:17.2.7-1633.g6d025bc2.el9.x86_64 1/1 2024-09-16T00:15:34.845 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: rbd-fuse-2:17.2.7-1633.g6d025bc2.el9.x86_64 1/1 2024-09-16T00:15:35.038 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-1633.g6d025bc2.el9.x86_64 1/1 2024-09-16T00:15:35.118 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : rbd-fuse-2:17.2.7-1633.g6d025bc2.el9.x86_64 1/1 2024-09-16T00:15:35.118 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:15:35.118 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2024-09-16T00:15:35.118 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-fuse-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-16T00:15:35.118 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:15:35.119 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2024-09-16T00:15:35.263 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libcephfs-devel-2:17.2.7-1633.g6d025bc2.el9.x86_64 1/1 2024-09-16T00:15:35.263 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:15:35.263 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2024-09-16T00:15:35.263 INFO:teuthology.orchestra.run.smithi169.stdout: libcephfs-devel-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-16T00:15:35.263 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:15:35.263 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-09-16T00:15:35.340 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install rbd-mirror 2024-09-16T00:15:35.426 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install librados2 2024-09-16T00:15:35.909 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:01:40 ago on Mon 16 Sep 2024 12:13:55 AM UTC. 2024-09-16T00:15:35.990 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:01:36 ago on Mon 16 Sep 2024 12:13:59 AM UTC. 2024-09-16T00:15:36.045 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2024-09-16T00:15:36.046 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-09-16T00:15:36.046 INFO:teuthology.orchestra.run.smithi112.stdout: Package Arch Version Repo Size 2024-09-16T00:15:36.046 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-09-16T00:15:36.046 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2024-09-16T00:15:36.046 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-mirror x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 2.9 M 2024-09-16T00:15:36.047 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:15:36.047 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2024-09-16T00:15:36.047 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-09-16T00:15:36.047 INFO:teuthology.orchestra.run.smithi112.stdout:Install 1 Package 2024-09-16T00:15:36.047 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:15:36.047 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 2.9 M 2024-09-16T00:15:36.047 INFO:teuthology.orchestra.run.smithi112.stdout:Installed size: 12 M 2024-09-16T00:15:36.047 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2024-09-16T00:15:36.080 INFO:teuthology.orchestra.run.smithi169.stdout:Package librados2-2:17.2.7-1633.g6d025bc2.el9.x86_64 is already installed. 2024-09-16T00:15:36.123 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-09-16T00:15:36.124 INFO:teuthology.orchestra.run.smithi169.stdout:Nothing to do. 2024-09-16T00:15:36.124 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-09-16T00:15:36.203 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install librbd1 2024-09-16T00:15:36.377 INFO:teuthology.orchestra.run.smithi112.stdout:rbd-mirror-17.2.7-1633.g6d025bc2.el9.x86_64.rpm 8.9 MB/s | 2.9 MB 00:00 2024-09-16T00:15:36.377 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2024-09-16T00:15:36.378 INFO:teuthology.orchestra.run.smithi112.stdout:Total 8.9 MB/s | 2.9 MB 00:00 2024-09-16T00:15:36.378 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2024-09-16T00:15:36.386 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2024-09-16T00:15:36.387 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2024-09-16T00:15:36.466 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2024-09-16T00:15:36.466 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2024-09-16T00:15:36.768 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:01:37 ago on Mon 16 Sep 2024 12:13:59 AM UTC. 2024-09-16T00:15:36.857 INFO:teuthology.orchestra.run.smithi169.stdout:Package librbd1-2:17.2.7-1633.g6d025bc2.el9.x86_64 is already installed. 2024-09-16T00:15:36.873 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2024-09-16T00:15:36.900 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-09-16T00:15:36.901 INFO:teuthology.orchestra.run.smithi169.stdout:Nothing to do. 2024-09-16T00:15:36.901 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-09-16T00:15:36.944 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : rbd-mirror-2:17.2.7-1633.g6d025bc2.el9.x86_64 1/1 2024-09-16T00:15:36.968 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: rbd-mirror-2:17.2.7-1633.g6d025bc2.el9.x86_64 1/1 2024-09-16T00:15:36.968 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:15:36.968 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-16T00:15:36.969 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T00:15:36.970 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T00:15:36.970 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:15:36.978 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install python3-rados 2024-09-16T00:15:37.541 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:01:38 ago on Mon 16 Sep 2024 12:13:59 AM UTC. 2024-09-16T00:15:37.630 INFO:teuthology.orchestra.run.smithi169.stdout:Package python3-rados-2:17.2.7-1633.g6d025bc2.el9.x86_64 is already installed. 2024-09-16T00:15:37.672 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-09-16T00:15:37.673 INFO:teuthology.orchestra.run.smithi169.stdout:Nothing to do. 2024-09-16T00:15:37.673 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-09-16T00:15:37.753 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install python3-rgw 2024-09-16T00:15:37.970 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : rbd-mirror-2:17.2.7-1633.g6d025bc2.el9.x86_64 1/1 2024-09-16T00:15:37.970 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:15:37.970 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2024-09-16T00:15:37.970 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-mirror-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-16T00:15:37.970 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:15:37.970 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2024-09-16T00:15:38.215 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install rbd-nbd 2024-09-16T00:15:38.316 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:01:39 ago on Mon 16 Sep 2024 12:13:59 AM UTC. 2024-09-16T00:15:38.405 INFO:teuthology.orchestra.run.smithi169.stdout:Package python3-rgw-2:17.2.7-1633.g6d025bc2.el9.x86_64 is already installed. 2024-09-16T00:15:38.448 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-09-16T00:15:38.449 INFO:teuthology.orchestra.run.smithi169.stdout:Nothing to do. 2024-09-16T00:15:38.449 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-09-16T00:15:38.533 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install python3-cephfs 2024-09-16T00:15:38.799 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:01:43 ago on Mon 16 Sep 2024 12:13:55 AM UTC. 2024-09-16T00:15:38.935 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2024-09-16T00:15:38.936 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-09-16T00:15:38.936 INFO:teuthology.orchestra.run.smithi112.stdout: Package Architecture Version Repository Size 2024-09-16T00:15:38.936 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-09-16T00:15:38.936 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2024-09-16T00:15:38.936 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-nbd x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 165 k 2024-09-16T00:15:38.936 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:15:38.936 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2024-09-16T00:15:38.936 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-09-16T00:15:38.936 INFO:teuthology.orchestra.run.smithi112.stdout:Install 1 Package 2024-09-16T00:15:38.936 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:15:38.937 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 165 k 2024-09-16T00:15:38.937 INFO:teuthology.orchestra.run.smithi112.stdout:Installed size: 477 k 2024-09-16T00:15:38.937 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2024-09-16T00:15:39.102 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:01:40 ago on Mon 16 Sep 2024 12:13:59 AM UTC. 2024-09-16T00:15:39.138 INFO:teuthology.orchestra.run.smithi112.stdout:rbd-nbd-17.2.7-1633.g6d025bc2.el9.x86_64.rpm 821 kB/s | 165 kB 00:00 2024-09-16T00:15:39.138 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2024-09-16T00:15:39.139 INFO:teuthology.orchestra.run.smithi112.stdout:Total 814 kB/s | 165 kB 00:00 2024-09-16T00:15:39.139 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2024-09-16T00:15:39.146 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2024-09-16T00:15:39.147 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2024-09-16T00:15:39.193 INFO:teuthology.orchestra.run.smithi169.stdout:Package python3-cephfs-2:17.2.7-1633.g6d025bc2.el9.x86_64 is already installed. 2024-09-16T00:15:39.209 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2024-09-16T00:15:39.209 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2024-09-16T00:15:39.235 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-09-16T00:15:39.237 INFO:teuthology.orchestra.run.smithi169.stdout:Nothing to do. 2024-09-16T00:15:39.237 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-09-16T00:15:39.314 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install python3-rbd 2024-09-16T00:15:39.362 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2024-09-16T00:15:39.460 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : rbd-nbd-2:17.2.7-1633.g6d025bc2.el9.x86_64 1/1 2024-09-16T00:15:39.877 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:01:40 ago on Mon 16 Sep 2024 12:13:59 AM UTC. 2024-09-16T00:15:39.966 INFO:teuthology.orchestra.run.smithi169.stdout:Package python3-rbd-2:17.2.7-1633.g6d025bc2.el9.x86_64 is already installed. 2024-09-16T00:15:40.003 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: rbd-nbd-2:17.2.7-1633.g6d025bc2.el9.x86_64 1/1 2024-09-16T00:15:40.008 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-09-16T00:15:40.009 INFO:teuthology.orchestra.run.smithi169.stdout:Nothing to do. 2024-09-16T00:15:40.009 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-09-16T00:15:40.091 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install rbd-fuse 2024-09-16T00:15:40.253 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : rbd-nbd-2:17.2.7-1633.g6d025bc2.el9.x86_64 1/1 2024-09-16T00:15:40.254 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:15:40.254 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2024-09-16T00:15:40.254 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-nbd-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-16T00:15:40.254 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:15:40.254 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2024-09-16T00:15:40.469 DEBUG:teuthology.parallel:result is None 2024-09-16T00:15:40.662 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:01:41 ago on Mon 16 Sep 2024 12:13:59 AM UTC. 2024-09-16T00:15:40.798 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-09-16T00:15:40.799 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T00:15:40.799 INFO:teuthology.orchestra.run.smithi169.stdout: Package Architecture Version Repository Size 2024-09-16T00:15:40.799 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T00:15:40.799 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2024-09-16T00:15:40.799 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-fuse x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 82 k 2024-09-16T00:15:40.799 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:15:40.799 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2024-09-16T00:15:40.800 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T00:15:40.800 INFO:teuthology.orchestra.run.smithi169.stdout:Install 1 Package 2024-09-16T00:15:40.800 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:15:40.800 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 82 k 2024-09-16T00:15:40.800 INFO:teuthology.orchestra.run.smithi169.stdout:Installed size: 222 k 2024-09-16T00:15:40.800 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2024-09-16T00:15:40.999 INFO:teuthology.orchestra.run.smithi169.stdout:rbd-fuse-17.2.7-1633.g6d025bc2.el9.x86_64.rpm 411 kB/s | 82 kB 00:00 2024-09-16T00:15:40.999 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2024-09-16T00:15:40.999 INFO:teuthology.orchestra.run.smithi169.stdout:Total 408 kB/s | 82 kB 00:00 2024-09-16T00:15:41.000 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2024-09-16T00:15:41.006 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2024-09-16T00:15:41.006 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2024-09-16T00:15:41.070 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2024-09-16T00:15:41.071 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2024-09-16T00:15:41.217 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2024-09-16T00:15:41.312 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : rbd-fuse-2:17.2.7-1633.g6d025bc2.el9.x86_64 1/1 2024-09-16T00:15:41.669 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: rbd-fuse-2:17.2.7-1633.g6d025bc2.el9.x86_64 1/1 2024-09-16T00:15:41.922 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : rbd-fuse-2:17.2.7-1633.g6d025bc2.el9.x86_64 1/1 2024-09-16T00:15:41.923 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:15:41.923 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2024-09-16T00:15:41.923 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-fuse-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-16T00:15:41.923 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:15:41.923 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-09-16T00:15:42.141 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install rbd-mirror 2024-09-16T00:15:42.718 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:01:43 ago on Mon 16 Sep 2024 12:13:59 AM UTC. 2024-09-16T00:15:42.855 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-09-16T00:15:42.855 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T00:15:42.856 INFO:teuthology.orchestra.run.smithi169.stdout: Package Arch Version Repo Size 2024-09-16T00:15:42.856 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T00:15:42.856 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2024-09-16T00:15:42.856 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-mirror x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 2.9 M 2024-09-16T00:15:42.856 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:15:42.856 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2024-09-16T00:15:42.856 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T00:15:42.856 INFO:teuthology.orchestra.run.smithi169.stdout:Install 1 Package 2024-09-16T00:15:42.856 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:15:42.857 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 2.9 M 2024-09-16T00:15:42.857 INFO:teuthology.orchestra.run.smithi169.stdout:Installed size: 12 M 2024-09-16T00:15:42.857 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2024-09-16T00:15:43.193 INFO:teuthology.orchestra.run.smithi169.stdout:rbd-mirror-17.2.7-1633.g6d025bc2.el9.x86_64.rpm 8.7 MB/s | 2.9 MB 00:00 2024-09-16T00:15:43.194 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2024-09-16T00:15:43.194 INFO:teuthology.orchestra.run.smithi169.stdout:Total 8.7 MB/s | 2.9 MB 00:00 2024-09-16T00:15:43.194 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2024-09-16T00:15:43.203 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2024-09-16T00:15:43.203 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2024-09-16T00:15:43.284 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2024-09-16T00:15:43.284 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2024-09-16T00:15:43.702 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2024-09-16T00:15:43.769 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : rbd-mirror-2:17.2.7-1633.g6d025bc2.el9.x86_64 1/1 2024-09-16T00:15:43.793 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: rbd-mirror-2:17.2.7-1633.g6d025bc2.el9.x86_64 1/1 2024-09-16T00:15:43.793 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:15:43.793 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-16T00:15:43.793 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T00:15:43.793 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T00:15:43.793 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:15:44.880 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : rbd-mirror-2:17.2.7-1633.g6d025bc2.el9.x86_64 1/1 2024-09-16T00:15:44.880 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:15:44.880 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2024-09-16T00:15:44.880 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-mirror-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-16T00:15:44.880 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:15:44.880 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-09-16T00:15:45.165 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install rbd-nbd 2024-09-16T00:15:45.749 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:01:46 ago on Mon 16 Sep 2024 12:13:59 AM UTC. 2024-09-16T00:15:45.884 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-09-16T00:15:45.885 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T00:15:45.885 INFO:teuthology.orchestra.run.smithi169.stdout: Package Architecture Version Repository Size 2024-09-16T00:15:45.885 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T00:15:45.885 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2024-09-16T00:15:45.885 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-nbd x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 165 k 2024-09-16T00:15:45.885 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:15:45.885 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2024-09-16T00:15:45.885 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-09-16T00:15:45.885 INFO:teuthology.orchestra.run.smithi169.stdout:Install 1 Package 2024-09-16T00:15:45.885 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:15:45.885 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 165 k 2024-09-16T00:15:45.885 INFO:teuthology.orchestra.run.smithi169.stdout:Installed size: 477 k 2024-09-16T00:15:45.886 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2024-09-16T00:15:46.089 INFO:teuthology.orchestra.run.smithi169.stdout:rbd-nbd-17.2.7-1633.g6d025bc2.el9.x86_64.rpm 812 kB/s | 165 kB 00:00 2024-09-16T00:15:46.089 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2024-09-16T00:15:46.090 INFO:teuthology.orchestra.run.smithi169.stdout:Total 806 kB/s | 165 kB 00:00 2024-09-16T00:15:46.090 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2024-09-16T00:15:46.097 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2024-09-16T00:15:46.097 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2024-09-16T00:15:46.162 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2024-09-16T00:15:46.162 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2024-09-16T00:15:46.314 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2024-09-16T00:15:46.419 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : rbd-nbd-2:17.2.7-1633.g6d025bc2.el9.x86_64 1/1 2024-09-16T00:15:46.906 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: rbd-nbd-2:17.2.7-1633.g6d025bc2.el9.x86_64 1/1 2024-09-16T00:15:47.206 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : rbd-nbd-2:17.2.7-1633.g6d025bc2.el9.x86_64 1/1 2024-09-16T00:15:47.207 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:15:47.207 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2024-09-16T00:15:47.207 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-nbd-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-16T00:15:47.207 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:15:47.207 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-09-16T00:15:47.477 DEBUG:teuthology.parallel:result is None 2024-09-16T00:15:47.478 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-16T00:15:47.478 INFO:teuthology.packaging:ref: None 2024-09-16T00:15:47.478 INFO:teuthology.packaging:tag: None 2024-09-16T00:15:47.478 INFO:teuthology.packaging:branch: quincy 2024-09-16T00:15:47.478 INFO:teuthology.packaging:sha1: 8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T00:15:47.478 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-09-16T00:15:47.712 DEBUG:teuthology.orchestra.run.smithi112:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-16T00:15:47.738 INFO:teuthology.orchestra.run.smithi112.stdout:17.2.7-1633.g6d025bc2.el9 2024-09-16T00:15:47.739 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1633.g6d025bc2.el9 2024-09-16T00:15:47.739 INFO:teuthology.task.install:The correct ceph version 17.2.7-1633.g6d025bc2 is installed. 2024-09-16T00:15:47.741 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-16T00:15:47.741 INFO:teuthology.packaging:ref: None 2024-09-16T00:15:47.741 INFO:teuthology.packaging:tag: None 2024-09-16T00:15:47.742 INFO:teuthology.packaging:branch: quincy 2024-09-16T00:15:47.742 INFO:teuthology.packaging:sha1: 8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T00:15:47.742 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-09-16T00:15:47.957 DEBUG:teuthology.orchestra.run.smithi169:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-16T00:15:47.980 INFO:teuthology.orchestra.run.smithi169.stdout:17.2.7-1633.g6d025bc2.el9 2024-09-16T00:15:47.980 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1633.g6d025bc2.el9 2024-09-16T00:15:47.980 INFO:teuthology.task.install:The correct ceph version 17.2.7-1633.g6d025bc2 is installed. 2024-09-16T00:15:47.982 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-16T00:15:47.982 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-16T00:15:47.982 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-16T00:15:48.014 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-09-16T00:15:48.014 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-16T00:15:48.046 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-16T00:15:48.046 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-16T00:15:48.047 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/usr/bin/daemon-helper 2024-09-16T00:15:48.083 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-16T00:15:48.152 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-09-16T00:15:48.153 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/usr/bin/daemon-helper 2024-09-16T00:15:48.177 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-16T00:15:48.242 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-16T00:15:48.242 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-16T00:15:48.242 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-16T00:15:48.271 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-16T00:15:48.339 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-09-16T00:15:48.339 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-16T00:15:48.364 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-16T00:15:48.428 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-16T00:15:48.429 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-16T00:15:48.429 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/usr/bin/stdin-killer 2024-09-16T00:15:48.457 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-16T00:15:48.525 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-09-16T00:15:48.525 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/usr/bin/stdin-killer 2024-09-16T00:15:48.550 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-16T00:15:48.615 INFO:teuthology.run_tasks:Running task print... 2024-09-16T00:15:48.624 INFO:teuthology.task.print:**** done install task... 2024-09-16T00:15:48.624 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-16T00:15:48.731 INFO:tasks.cephadm:Config: {'cephadm_branch': 'quincy', 'cephadm_git_url': 'https://github.com/ceph/ceph', 'conf': {'osd': {'osd_class_default_list': '*', 'osd_class_load_list': '*', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}}, 'image': 'quay.ceph.io/ceph-ci/ceph:quincy', 'roleless': True, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'OSD_DOWN', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc'} 2024-09-16T00:15:48.732 INFO:tasks.cephadm:Cluster image is quay.ceph.io/ceph-ci/ceph:quincy 2024-09-16T00:15:48.732 INFO:tasks.cephadm:Cluster fsid is d346dd56-73c0-11ef-bceb-c7b262605968 2024-09-16T00:15:48.732 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-16T00:15:48.733 INFO:tasks.cephadm:No mon roles; fabricating mons 2024-09-16T00:15:48.733 INFO:tasks.cephadm:Monitor IPs: {'mon.smithi112': '172.21.15.112', 'mon.smithi169': '172.21.15.169'} 2024-09-16T00:15:48.733 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-16T00:15:48.733 DEBUG:teuthology.orchestra.run.smithi112:> sudo hostname $(hostname -s) 2024-09-16T00:15:48.762 DEBUG:teuthology.orchestra.run.smithi169:> sudo hostname $(hostname -s) 2024-09-16T00:15:48.787 INFO:tasks.cephadm:Downloading cephadm (repo https://github.com/ceph/ceph ref quincy)... 2024-09-16T00:15:48.787 DEBUG:teuthology.orchestra.run.smithi112:> curl --silent https://raw.githubusercontent.com/ceph/ceph/quincy/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-16T00:15:49.035 INFO:teuthology.orchestra.run.smithi112.stdout:-rw-r--r--. 1 ubuntu ubuntu 378585 Sep 16 00:15 /home/ubuntu/cephtest/cephadm 2024-09-16T00:15:49.035 DEBUG:teuthology.orchestra.run.smithi169:> curl --silent https://raw.githubusercontent.com/ceph/ceph/quincy/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-16T00:15:49.169 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r--. 1 ubuntu ubuntu 378585 Sep 16 00:15 /home/ubuntu/cephtest/cephadm 2024-09-16T00:15:49.169 DEBUG:teuthology.orchestra.run.smithi112:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-16T00:15:49.189 DEBUG:teuthology.orchestra.run.smithi169:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-16T00:15:49.214 INFO:tasks.cephadm:Pulling image quay.ceph.io/ceph-ci/ceph:quincy on all hosts... 2024-09-16T00:15:49.214 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy pull 2024-09-16T00:15:49.232 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy pull 2024-09-16T00:15:50.875 INFO:teuthology.orchestra.run.smithi112.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-09-16T00:15:50.936 INFO:teuthology.orchestra.run.smithi169.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-09-16T00:16:20.109 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:16:20.110 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_version": "ceph version 17.2.7-1633-g6d025bc2 (6d025bc26f3176c910707f8cb4c8d8e076640615) quincy (stable)", 2024-09-16T00:16:20.110 INFO:teuthology.orchestra.run.smithi112.stdout: "image_id": "33ed8dba399669a1d738ac1a85bc5f36daf3048afed44b7a93f620fc77496a99", 2024-09-16T00:16:20.110 INFO:teuthology.orchestra.run.smithi112.stdout: "repo_digests": [ 2024-09-16T00:16:20.110 INFO:teuthology.orchestra.run.smithi112.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:83596473fc70117d507bb3d925c5bd11b70f46bbd4dffcc8452f145d93eaedc4" 2024-09-16T00:16:20.110 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-09-16T00:16:20.110 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:16:23.680 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2024-09-16T00:16:23.680 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_version": "ceph version 17.2.7-1633-g6d025bc2 (6d025bc26f3176c910707f8cb4c8d8e076640615) quincy (stable)", 2024-09-16T00:16:23.680 INFO:teuthology.orchestra.run.smithi169.stdout: "image_id": "33ed8dba399669a1d738ac1a85bc5f36daf3048afed44b7a93f620fc77496a99", 2024-09-16T00:16:23.680 INFO:teuthology.orchestra.run.smithi169.stdout: "repo_digests": [ 2024-09-16T00:16:23.680 INFO:teuthology.orchestra.run.smithi169.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:83596473fc70117d507bb3d925c5bd11b70f46bbd4dffcc8452f145d93eaedc4" 2024-09-16T00:16:23.681 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2024-09-16T00:16:23.681 INFO:teuthology.orchestra.run.smithi169.stdout:} 2024-09-16T00:16:23.698 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /etc/ceph 2024-09-16T00:16:23.733 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /etc/ceph 2024-09-16T00:16:23.760 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 777 /etc/ceph 2024-09-16T00:16:23.802 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 777 /etc/ceph 2024-09-16T00:16:23.827 INFO:tasks.cephadm:Writing seed config... 2024-09-16T00:16:23.828 INFO:tasks.cephadm: override: [osd] osd_class_default_list = * 2024-09-16T00:16:23.828 INFO:tasks.cephadm: override: [osd] osd_class_load_list = * 2024-09-16T00:16:23.828 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-09-16T00:16:23.828 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-09-16T00:16:23.828 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-09-16T00:16:23.828 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-16T00:16:23.828 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-16T00:16:23.828 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-09-16T00:16:23.828 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-09-16T00:16:23.829 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-16T00:16:23.829 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-16T00:16:23.829 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-09-16T00:16:23.829 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-16T00:16:23.829 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-16T00:16:23.829 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-16T00:16:23.829 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-16T00:16:23.829 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-16T00:16:23.829 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-16T00:16:23.829 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-16T00:16:23.829 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-16T00:16:23.829 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-16T00:16:23.829 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-16T00:16:23.829 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-16T00:16:23.830 INFO:tasks.cephadm: override: [global] mon pg warn min per osd = 0 2024-09-16T00:16:23.830 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-16T00:16:23.830 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-16T00:16:23.830 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-16T00:16:23.830 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-16T00:16:23.830 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-16T00:16:23.830 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-16T00:16:23.830 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-16T00:16:23.830 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-16T00:16:23.830 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-16T00:16:23.830 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-16T00:16:23.830 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-16T00:16:23.830 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-16T00:16:23.830 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-16T00:16:23.830 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-16T00:16:23.831 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-16T00:16:23.831 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-16T00:16:23.831 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-16T00:16:23.831 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-16T00:16:23.831 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-16T00:16:23.832 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-16T00:16:23.832 DEBUG:teuthology.orchestra.run.smithi112:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-16T00:16:23.860 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = d346dd56-73c0-11ef-bceb-c7b262605968 mon pg warn min per osd = 0 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = true bdev debug aio = true osd sloppy crc = true osd_class_default_list = * osd_class_load_list = * bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-16T00:16:23.860 DEBUG:teuthology.orchestra.run.smithi112:mon.smithi112> sudo journalctl -f -n 0 -u ceph-d346dd56-73c0-11ef-bceb-c7b262605968@mon.smithi112.service 2024-09-16T00:16:23.903 INFO:tasks.cephadm:Bootstrapping... 2024-09-16T00:16:23.903 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy -v bootstrap --fsid d346dd56-73c0-11ef-bceb-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-ip 172.21.15.112 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-16T00:16:24.105 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2024-09-16T00:16:24.105 INFO:teuthology.orchestra.run.smithi112.stdout:cephadm ['--image', 'quay.ceph.io/ceph-ci/ceph:quincy', '-v', 'bootstrap', '--fsid', 'd346dd56-73c0-11ef-bceb-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-ip', '172.21.15.112', '--skip-admin-label'] 2024-09-16T00:16:24.129 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stdout 5.2.2 2024-09-16T00:16:24.129 INFO:teuthology.orchestra.run.smithi112.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-16T00:16:24.129 INFO:teuthology.orchestra.run.smithi112.stdout:Verifying podman|docker is present... 2024-09-16T00:16:24.151 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stdout 5.2.2 2024-09-16T00:16:24.151 INFO:teuthology.orchestra.run.smithi112.stdout:Verifying lvm2 is present... 2024-09-16T00:16:24.151 INFO:teuthology.orchestra.run.smithi112.stdout:Verifying time synchronization is in place... 2024-09-16T00:16:24.159 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-16T00:16:24.159 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-16T00:16:24.166 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-16T00:16:24.166 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout inactive 2024-09-16T00:16:24.173 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout enabled 2024-09-16T00:16:24.179 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout active 2024-09-16T00:16:24.180 INFO:teuthology.orchestra.run.smithi112.stdout:Unit chronyd.service is enabled and running 2024-09-16T00:16:24.180 INFO:teuthology.orchestra.run.smithi112.stdout:Repeating the final host check... 2024-09-16T00:16:24.201 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stdout 5.2.2 2024-09-16T00:16:24.201 INFO:teuthology.orchestra.run.smithi112.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-16T00:16:24.201 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl is present 2024-09-16T00:16:24.202 INFO:teuthology.orchestra.run.smithi112.stdout:lvcreate is present 2024-09-16T00:16:24.208 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-16T00:16:24.208 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-16T00:16:24.215 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-16T00:16:24.215 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout inactive 2024-09-16T00:16:24.222 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout enabled 2024-09-16T00:16:24.228 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout active 2024-09-16T00:16:24.229 INFO:teuthology.orchestra.run.smithi112.stdout:Unit chronyd.service is enabled and running 2024-09-16T00:16:24.229 INFO:teuthology.orchestra.run.smithi112.stdout:Host looks OK 2024-09-16T00:16:24.229 INFO:teuthology.orchestra.run.smithi112.stdout:Cluster fsid: d346dd56-73c0-11ef-bceb-c7b262605968 2024-09-16T00:16:24.229 INFO:teuthology.orchestra.run.smithi112.stdout:Acquiring lock 140043941642592 on /run/cephadm/d346dd56-73c0-11ef-bceb-c7b262605968.lock 2024-09-16T00:16:24.229 INFO:teuthology.orchestra.run.smithi112.stdout:Lock 140043941642592 acquired on /run/cephadm/d346dd56-73c0-11ef-bceb-c7b262605968.lock 2024-09-16T00:16:24.230 INFO:teuthology.orchestra.run.smithi112.stdout:Verifying IP 172.21.15.112 port 3300 ... 2024-09-16T00:16:24.230 INFO:teuthology.orchestra.run.smithi112.stdout:Verifying IP 172.21.15.112 port 6789 ... 2024-09-16T00:16:24.230 INFO:teuthology.orchestra.run.smithi112.stdout:Base mon IP(s) is [172.21.15.112:3300, 172.21.15.112:6789], mon addrv is [v2:172.21.15.112:3300,v1:172.21.15.112:6789] 2024-09-16T00:16:24.233 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.112 metric 100 2024-09-16T00:16:24.233 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.112 metric 100 2024-09-16T00:16:24.236 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-16T00:16:24.236 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-16T00:16:24.236 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-16T00:16:24.238 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-16T00:16:24.238 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-16T00:16:24.238 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-16T00:16:24.238 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-16T00:16:24.239 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:cb69/64 scope link noprefixroute 2024-09-16T00:16:24.239 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-16T00:16:24.239 INFO:teuthology.orchestra.run.smithi112.stdout:Mon IP `172.21.15.112` is in CIDR network `172.21.0.0/20` 2024-09-16T00:16:24.239 INFO:teuthology.orchestra.run.smithi112.stdout:Mon IP `172.21.15.112` is in CIDR network `172.21.0.0/20` 2024-09-16T00:16:24.239 INFO:teuthology.orchestra.run.smithi112.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-16T00:16:24.240 INFO:teuthology.orchestra.run.smithi112.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-16T00:16:24.240 INFO:teuthology.orchestra.run.smithi112.stdout:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-09-16T00:16:24.726 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stdout 33ed8dba399669a1d738ac1a85bc5f36daf3048afed44b7a93f620fc77496a99 2024-09-16T00:16:24.726 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stderr Trying to pull quay.ceph.io/ceph-ci/ceph:quincy... 2024-09-16T00:16:24.726 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stderr Getting image source signatures 2024-09-16T00:16:24.726 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stderr Copying blob sha256:16f62ad1b9e7009e3336687c29bef330814f97ad91514d727da356692f597064 2024-09-16T00:16:24.726 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-16T00:16:24.726 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stderr Copying config sha256:33ed8dba399669a1d738ac1a85bc5f36daf3048afed44b7a93f620fc77496a99 2024-09-16T00:16:24.727 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-16T00:16:25.681 INFO:teuthology.orchestra.run.smithi112.stdout:ceph: stdout ceph version 17.2.7-1633-g6d025bc2 (6d025bc26f3176c910707f8cb4c8d8e076640615) quincy (stable) 2024-09-16T00:16:25.681 INFO:teuthology.orchestra.run.smithi112.stdout:Ceph version: ceph version 17.2.7-1633-g6d025bc2 (6d025bc26f3176c910707f8cb4c8d8e076640615) quincy (stable) 2024-09-16T00:16:25.682 INFO:teuthology.orchestra.run.smithi112.stdout:Extracting ceph user uid/gid from container image... 2024-09-16T00:16:26.681 INFO:teuthology.orchestra.run.smithi112.stdout:stat: stdout 167 167 2024-09-16T00:16:26.681 INFO:teuthology.orchestra.run.smithi112.stdout:Creating initial keys... 2024-09-16T00:16:34.367 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph-authtool: stdout AQDbeOdmlrtwORAA3Lm/9twtTfF8t5re5Wz6fg== 2024-09-16T00:16:39.106 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph-authtool: stdout AQDleOdmaJK+CBAAmvgnbwTcMIMTSKBr/200Ug== 2024-09-16T00:16:40.229 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph-authtool: stdout AQDneOdmpKeJGhAA8bjGRmrDLYt/iBj2Puw8Kw== 2024-09-16T00:16:40.230 INFO:teuthology.orchestra.run.smithi112.stdout:Creating initial monmap... 2024-09-16T00:16:41.195 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-16T00:16:41.195 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = octopus 2024-09-16T00:16:41.195 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to d346dd56-73c0-11ef-bceb-c7b262605968 2024-09-16T00:16:41.195 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-16T00:16:41.195 INFO:teuthology.orchestra.run.smithi112.stdout:monmaptool for smithi112 [v2:172.21.15.112:3300,v1:172.21.15.112:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-16T00:16:41.195 INFO:teuthology.orchestra.run.smithi112.stdout:setting min_mon_release = octopus 2024-09-16T00:16:41.195 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/monmaptool: set fsid to d346dd56-73c0-11ef-bceb-c7b262605968 2024-09-16T00:16:41.195 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-16T00:16:41.195 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:16:41.196 INFO:teuthology.orchestra.run.smithi112.stdout:Creating mon... 2024-09-16T00:16:42.268 INFO:teuthology.orchestra.run.smithi112.stdout:create mon.smithi112 on 2024-09-16T00:16:42.480 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-16T00:16:42.651 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-16T00:16:42.811 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-d346dd56-73c0-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-d346dd56-73c0-11ef-bceb-c7b262605968.target. 2024-09-16T00:16:42.811 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-d346dd56-73c0-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-d346dd56-73c0-11ef-bceb-c7b262605968.target. 2024-09-16T00:16:43.006 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-d346dd56-73c0-11ef-bceb-c7b262605968@mon.smithi112 2024-09-16T00:16:43.006 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Failed to reset failed state of unit ceph-d346dd56-73c0-11ef-bceb-c7b262605968@mon.smithi112.service: Unit ceph-d346dd56-73c0-11ef-bceb-c7b262605968@mon.smithi112.service not loaded. 2024-09-16T00:16:43.176 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-d346dd56-73c0-11ef-bceb-c7b262605968.target.wants/ceph-d346dd56-73c0-11ef-bceb-c7b262605968@mon.smithi112.service → /etc/systemd/system/ceph-d346dd56-73c0-11ef-bceb-c7b262605968@.service. 2024-09-16T00:16:43.923 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-16T00:16:43.923 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout disabled 2024-09-16T00:16:43.930 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-16T00:16:43.930 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout inactive 2024-09-16T00:16:43.930 INFO:teuthology.orchestra.run.smithi112.stdout:firewalld.service is not enabled 2024-09-16T00:16:43.930 INFO:teuthology.orchestra.run.smithi112.stdout:Not possible to enable service . firewalld.service is not available 2024-09-16T00:16:43.931 INFO:teuthology.orchestra.run.smithi112.stdout:Waiting for mon to start... 2024-09-16T00:16:43.931 INFO:teuthology.orchestra.run.smithi112.stdout:Waiting for mon... 2024-09-16T00:16:44.055 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:43 smithi112 systemd[1]: Started Ceph mon.smithi112 for d346dd56-73c0-11ef-bceb-c7b262605968. 2024-09-16T00:16:44.954 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:44 smithi112 ceph-mon[27501]: mon.smithi112 is new leader, mons smithi112 in quorum (ranks 0) 2024-09-16T00:16:44.954 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:44 smithi112 ceph-mon[27501]: monmap e1: 1 mons at {smithi112=[v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-16T00:16:44.954 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:44 smithi112 ceph-mon[27501]: fsmap 2024-09-16T00:16:44.955 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:44 smithi112 ceph-mon[27501]: osdmap e1: 0 total, 0 up, 0 in 2024-09-16T00:16:44.955 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:44 smithi112 ceph-mon[27501]: mgrmap e1: no daemons active 2024-09-16T00:16:45.613 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout cluster: 2024-09-16T00:16:45.613 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout id: d346dd56-73c0-11ef-bceb-c7b262605968 2024-09-16T00:16:45.613 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-16T00:16:45.613 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-09-16T00:16:45.613 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout services: 2024-09-16T00:16:45.613 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum smithi112 (age 1.27363s) 2024-09-16T00:16:45.613 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-16T00:16:45.613 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-16T00:16:45.613 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-09-16T00:16:45.613 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout data: 2024-09-16T00:16:45.613 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-16T00:16:45.614 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-16T00:16:45.614 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-16T00:16:45.614 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout pgs: 2024-09-16T00:16:45.614 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-09-16T00:16:45.614 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:44.953+0000 7f5fef349640 1 Processor -- start 2024-09-16T00:16:45.614 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:44.953+0000 7f5fef349640 1 -- start start 2024-09-16T00:16:45.614 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:44.953+0000 7f5fef349640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fe81035b0 0x7f5fe81039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:45.614 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:44.953+0000 7f5fef349640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe8070e50 con 0x7f5fe81035b0 2024-09-16T00:16:45.614 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:44.954+0000 7f5fed0e5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fe81035b0 0x7f5fe81039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:45.614 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:44.954+0000 7f5fed0e5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fe81035b0 0x7f5fe81039c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54856/0 (socket says 172.21.15.112:54856) 2024-09-16T00:16:45.614 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:44.954+0000 7f5fed0e5640 1 -- 172.21.15.112:0/881694503 learned_addr learned my addr 172.21.15.112:0/881694503 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:16:45.614 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:44.954+0000 7f5fed0e5640 1 -- 172.21.15.112:0/881694503 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fe8103f00 con 0x7f5fe81035b0 2024-09-16T00:16:45.614 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:44.956+0000 7f5fed0e5640 1 --2- 172.21.15.112:0/881694503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fe81035b0 0x7f5fe81039c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4009840 tx=0x7f5fe40091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bae94358e4e6cfa4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:45.614 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:44.956+0000 7f5fdffff640 1 -- 172.21.15.112:0/881694503 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5fe4032440 con 0x7f5fe81035b0 2024-09-16T00:16:45.615 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:44.956+0000 7f5fdffff640 1 -- 172.21.15.112:0/881694503 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fe40325a0 con 0x7f5fe81035b0 2024-09-16T00:16:45.615 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:44.956+0000 7f5fdffff640 1 -- 172.21.15.112:0/881694503 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5fe40328d0 con 0x7f5fe81035b0 2024-09-16T00:16:45.615 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:44.957+0000 7f5fef349640 1 -- 172.21.15.112:0/881694503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fe81035b0 msgr2=0x7f5fe81039c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:45.615 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:44.957+0000 7f5fef349640 1 --2- 172.21.15.112:0/881694503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fe81035b0 0x7f5fe81039c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4009840 tx=0x7f5fe40091d0 comp rx=0 tx=0).stop 2024-09-16T00:16:45.615 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:44.957+0000 7f5fef349640 1 -- 172.21.15.112:0/881694503 shutdown_connections 2024-09-16T00:16:45.615 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:44.957+0000 7f5fef349640 1 --2- 172.21.15.112:0/881694503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fe81035b0 0x7f5fe81039c0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:45.615 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:44.957+0000 7f5fef349640 1 -- 172.21.15.112:0/881694503 >> 172.21.15.112:0/881694503 conn(0x7f5fe80fbdd0 msgr2=0x7f5fe80fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:45.615 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:44.957+0000 7f5fef349640 1 -- 172.21.15.112:0/881694503 shutdown_connections 2024-09-16T00:16:45.615 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:44.957+0000 7f5fef349640 1 -- 172.21.15.112:0/881694503 wait complete. 2024-09-16T00:16:45.615 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:44.957+0000 7f5fef349640 1 Processor -- start 2024-09-16T00:16:45.615 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:44.957+0000 7f5fef349640 1 -- start start 2024-09-16T00:16:45.615 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:44.958+0000 7f5fef349640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fe81035b0 0x7f5fe8110670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:45.615 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:44.958+0000 7f5fef349640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe8110bb0 con 0x7f5fe81035b0 2024-09-16T00:16:45.615 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:44.958+0000 7f5fed0e5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fe81035b0 0x7f5fe8110670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:45.616 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:44.958+0000 7f5fed0e5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fe81035b0 0x7f5fe8110670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54858/0 (socket says 172.21.15.112:54858) 2024-09-16T00:16:45.616 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:44.958+0000 7f5fed0e5640 1 -- 172.21.15.112:0/3185303997 learned_addr learned my addr 172.21.15.112:0/3185303997 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:16:45.616 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:44.958+0000 7f5fed0e5640 1 -- 172.21.15.112:0/3185303997 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fe4009450 con 0x7f5fe81035b0 2024-09-16T00:16:45.616 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:44.958+0000 7f5fed0e5640 1 --2- 172.21.15.112:0/3185303997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fe81035b0 0x7f5fe8110670 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5fe402a700 tx=0x7f5fe4032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:45.616 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:44.958+0000 7f5fddffb640 1 -- 172.21.15.112:0/3185303997 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5fe4031be0 con 0x7f5fe81035b0 2024-09-16T00:16:45.616 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:44.958+0000 7f5fddffb640 1 -- 172.21.15.112:0/3185303997 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fe4031d40 con 0x7f5fe81035b0 2024-09-16T00:16:45.616 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:44.958+0000 7f5fddffb640 1 -- 172.21.15.112:0/3185303997 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5fe403b5a0 con 0x7f5fe81035b0 2024-09-16T00:16:45.616 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:44.958+0000 7f5fef349640 1 -- 172.21.15.112:0/3185303997 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fe8110db0 con 0x7f5fe81035b0 2024-09-16T00:16:45.616 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:44.959+0000 7f5fef349640 1 -- 172.21.15.112:0/3185303997 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fe81111c0 con 0x7f5fe81035b0 2024-09-16T00:16:45.616 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:44.959+0000 7f5fddffb640 1 -- 172.21.15.112:0/3185303997 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f5fe4039040 con 0x7f5fe81035b0 2024-09-16T00:16:45.616 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:44.959+0000 7f5fddffb640 1 -- 172.21.15.112:0/3185303997 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f5fe4041030 con 0x7f5fe81035b0 2024-09-16T00:16:45.616 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:44.960+0000 7f5fef349640 1 -- 172.21.15.112:0/3185303997 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fe804b720 con 0x7f5fe81035b0 2024-09-16T00:16:45.616 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:44.962+0000 7f5fddffb640 1 -- 172.21.15.112:0/3185303997 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f5fe4037070 con 0x7f5fe81035b0 2024-09-16T00:16:45.617 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:45.019+0000 7f5fef349640 1 -- 172.21.15.112:0/3185303997 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f5fe8109bb0 con 0x7f5fe81035b0 2024-09-16T00:16:45.617 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:45.019+0000 7f5fddffb640 1 -- 172.21.15.112:0/3185303997 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+324 (secure 0 0 0) 0x7f5fe403bbd0 con 0x7f5fe81035b0 2024-09-16T00:16:45.617 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:45.020+0000 7f5fef349640 1 -- 172.21.15.112:0/3185303997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fe81035b0 msgr2=0x7f5fe8110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:45.617 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:45.020+0000 7f5fef349640 1 --2- 172.21.15.112:0/3185303997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fe81035b0 0x7f5fe8110670 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5fe402a700 tx=0x7f5fe4032000 comp rx=0 tx=0).stop 2024-09-16T00:16:45.617 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:45.020+0000 7f5fef349640 1 -- 172.21.15.112:0/3185303997 shutdown_connections 2024-09-16T00:16:45.617 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:45.020+0000 7f5fef349640 1 --2- 172.21.15.112:0/3185303997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fe81035b0 0x7f5fe8110670 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:45.617 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:45.020+0000 7f5fef349640 1 -- 172.21.15.112:0/3185303997 >> 172.21.15.112:0/3185303997 conn(0x7f5fe80fbdd0 msgr2=0x7f5fe8104b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:45.617 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:45.020+0000 7f5fef349640 1 -- 172.21.15.112:0/3185303997 shutdown_connections 2024-09-16T00:16:45.617 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:45.020+0000 7f5fef349640 1 -- 172.21.15.112:0/3185303997 wait complete. 2024-09-16T00:16:45.617 INFO:teuthology.orchestra.run.smithi112.stdout:mon is available 2024-09-16T00:16:45.617 INFO:teuthology.orchestra.run.smithi112.stdout:Assimilating anything we can from ceph.conf... 2024-09-16T00:16:45.932 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:45 smithi112 ceph-mon[27501]: from='client.? 172.21.15.112:0/3185303997' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-16T00:16:46.639 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-09-16T00:16:46.646 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout [global] 2024-09-16T00:16:46.646 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout fsid = d346dd56-73c0-11ef-bceb-c7b262605968 2024-09-16T00:16:46.646 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-16T00:16:46.646 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.112:3300,v1:172.21.15.112:6789] 2024-09-16T00:16:46.646 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-16T00:16:46.646 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-16T00:16:46.646 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-16T00:16:46.646 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-16T00:16:46.646 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-09-16T00:16:46.647 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-16T00:16:46.647 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-16T00:16:46.647 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-09-16T00:16:46.647 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout [osd] 2024-09-16T00:16:46.647 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-16T00:16:46.647 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-16T00:16:46.647 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-16T00:16:46.647 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-16T00:16:46.647 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-16T00:16:46.647 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:46.005+0000 7f7580a43640 1 Processor -- start 2024-09-16T00:16:46.647 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:46.005+0000 7f7580a43640 1 -- start start 2024-09-16T00:16:46.647 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:46.006+0000 7f7580a43640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f757c078b70 0x7f757c077230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:46.647 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:46.006+0000 7f7580a43640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757c078f80 con 0x7f757c078b70 2024-09-16T00:16:46.647 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:46.006+0000 7f757a59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f757c078b70 0x7f757c077230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:46.647 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:46.006+0000 7f757a59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f757c078b70 0x7f757c077230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54872/0 (socket says 172.21.15.112:54872) 2024-09-16T00:16:46.648 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:46.006+0000 7f757a59c640 1 -- 172.21.15.112:0/3851738618 learned_addr learned my addr 172.21.15.112:0/3851738618 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:16:46.648 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:46.006+0000 7f757a59c640 1 -- 172.21.15.112:0/3851738618 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f757c0790c0 con 0x7f757c078b70 2024-09-16T00:16:46.648 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:46.007+0000 7f757a59c640 1 --2- 172.21.15.112:0/3851738618 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f757c078b70 0x7f757c077230 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f756c009840 tx=0x7f756c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a432ab0b644afd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:46.648 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:46.007+0000 7f757959a640 1 -- 172.21.15.112:0/3851738618 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f756c032440 con 0x7f757c078b70 2024-09-16T00:16:46.648 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:46.007+0000 7f757959a640 1 -- 172.21.15.112:0/3851738618 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f756c0325a0 con 0x7f757c078b70 2024-09-16T00:16:46.648 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:46.007+0000 7f757959a640 1 -- 172.21.15.112:0/3851738618 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f756c032890 con 0x7f757c078b70 2024-09-16T00:16:46.648 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:46.008+0000 7f7580a43640 1 -- 172.21.15.112:0/3851738618 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f757c078b70 msgr2=0x7f757c077230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:46.648 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:46.008+0000 7f7580a43640 1 --2- 172.21.15.112:0/3851738618 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f757c078b70 0x7f757c077230 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f756c009840 tx=0x7f756c0091d0 comp rx=0 tx=0).stop 2024-09-16T00:16:46.648 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:46.008+0000 7f7580a43640 1 -- 172.21.15.112:0/3851738618 shutdown_connections 2024-09-16T00:16:46.648 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:46.008+0000 7f7580a43640 1 --2- 172.21.15.112:0/3851738618 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f757c078b70 0x7f757c077230 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:46.648 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:46.008+0000 7f7580a43640 1 -- 172.21.15.112:0/3851738618 >> 172.21.15.112:0/3851738618 conn(0x7f757c0fd850 msgr2=0x7f757c0ffc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:46.648 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:46.008+0000 7f7580a43640 1 -- 172.21.15.112:0/3851738618 shutdown_connections 2024-09-16T00:16:46.648 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:46.008+0000 7f7580a43640 1 -- 172.21.15.112:0/3851738618 wait complete. 2024-09-16T00:16:46.648 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:46.008+0000 7f7580a43640 1 Processor -- start 2024-09-16T00:16:46.649 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:46.008+0000 7f7580a43640 1 -- start start 2024-09-16T00:16:46.649 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:46.008+0000 7f7580a43640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f757c078b70 0x7f757c112720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:46.649 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:46.008+0000 7f7580a43640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757c112c60 con 0x7f757c078b70 2024-09-16T00:16:46.649 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:46.008+0000 7f757a59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f757c078b70 0x7f757c112720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:46.649 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:46.008+0000 7f757a59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f757c078b70 0x7f757c112720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54880/0 (socket says 172.21.15.112:54880) 2024-09-16T00:16:46.649 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:46.008+0000 7f757a59c640 1 -- 172.21.15.112:0/2476657383 learned_addr learned my addr 172.21.15.112:0/2476657383 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:16:46.649 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:46.009+0000 7f757a59c640 1 -- 172.21.15.112:0/2476657383 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f756c009450 con 0x7f757c078b70 2024-09-16T00:16:46.649 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:46.009+0000 7f757a59c640 1 --2- 172.21.15.112:0/2476657383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f757c078b70 0x7f757c112720 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f756c02a790 tx=0x7f756c032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:46.649 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:46.009+0000 7f7572ffd640 1 -- 172.21.15.112:0/2476657383 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f756c031be0 con 0x7f757c078b70 2024-09-16T00:16:46.649 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:46.009+0000 7f7572ffd640 1 -- 172.21.15.112:0/2476657383 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f756c031d40 con 0x7f757c078b70 2024-09-16T00:16:46.649 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:46.009+0000 7f7572ffd640 1 -- 172.21.15.112:0/2476657383 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f756c03b5a0 con 0x7f757c078b70 2024-09-16T00:16:46.649 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:46.009+0000 7f7580a43640 1 -- 172.21.15.112:0/2476657383 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f757c112e60 con 0x7f757c078b70 2024-09-16T00:16:46.649 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:46.009+0000 7f7580a43640 1 -- 172.21.15.112:0/2476657383 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f757c1139f0 con 0x7f757c078b70 2024-09-16T00:16:46.649 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:46.009+0000 7f7572ffd640 1 -- 172.21.15.112:0/2476657383 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f756c039040 con 0x7f757c078b70 2024-09-16T00:16:46.650 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:46.009+0000 7f7572ffd640 1 -- 172.21.15.112:0/2476657383 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f756c041070 con 0x7f757c078b70 2024-09-16T00:16:46.650 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:46.010+0000 7f7580a43640 1 -- 172.21.15.112:0/2476657383 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f757c1014f0 con 0x7f757c078b70 2024-09-16T00:16:46.650 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:46.012+0000 7f7572ffd640 1 -- 172.21.15.112:0/2476657383 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f756c02ae60 con 0x7f757c078b70 2024-09-16T00:16:46.650 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:46.068+0000 7f7580a43640 1 -- 172.21.15.112:0/2476657383 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f757c077a80 con 0x7f757c078b70 2024-09-16T00:16:46.650 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:46.071+0000 7f7572ffd640 1 -- 172.21.15.112:0/2476657383 <== mon.0 v2:172.21.15.112:3300/0 7 ==== config(26 keys) v1 ==== 1005+0+0 (secure 0 0 0) 0x7f756c04e890 con 0x7f757c078b70 2024-09-16T00:16:46.650 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:46.071+0000 7f7572ffd640 1 -- 172.21.15.112:0/2476657383 <== mon.0 v2:172.21.15.112:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+467 (secure 0 0 0) 0x7f757c077a80 con 0x7f757c078b70 2024-09-16T00:16:46.650 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:46.072+0000 7f7580a43640 1 -- 172.21.15.112:0/2476657383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f757c078b70 msgr2=0x7f757c112720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:46.650 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:46.072+0000 7f7580a43640 1 --2- 172.21.15.112:0/2476657383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f757c078b70 0x7f757c112720 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f756c02a790 tx=0x7f756c032000 comp rx=0 tx=0).stop 2024-09-16T00:16:46.650 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:46.072+0000 7f7580a43640 1 -- 172.21.15.112:0/2476657383 shutdown_connections 2024-09-16T00:16:46.650 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:46.072+0000 7f7580a43640 1 --2- 172.21.15.112:0/2476657383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f757c078b70 0x7f757c112720 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:46.650 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:46.072+0000 7f7580a43640 1 -- 172.21.15.112:0/2476657383 >> 172.21.15.112:0/2476657383 conn(0x7f757c0fd850 msgr2=0x7f757c1095e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:46.650 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:46.072+0000 7f7580a43640 1 -- 172.21.15.112:0/2476657383 shutdown_connections 2024-09-16T00:16:46.650 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:46.072+0000 7f7580a43640 1 -- 172.21.15.112:0/2476657383 wait complete. 2024-09-16T00:16:46.650 INFO:teuthology.orchestra.run.smithi112.stdout:Generating new minimal ceph.conf... 2024-09-16T00:16:46.890 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:46 smithi112 ceph-mon[27501]: from='client.? 172.21.15.112:0/2476657383' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-16T00:16:46.890 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:46 smithi112 ceph-mon[27501]: from='client.? 172.21.15.112:0/2476657383' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-16T00:16:47.741 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:47.112+0000 7fa953d03640 1 Processor -- start 2024-09-16T00:16:47.741 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:47.113+0000 7fa953d03640 1 -- start start 2024-09-16T00:16:47.741 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:47.113+0000 7fa953d03640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa94c100f30 0x7fa94c101340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:47.741 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:47.113+0000 7fa953d03640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa94c101880 con 0x7fa94c100f30 2024-09-16T00:16:47.741 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:47.113+0000 7fa951a9f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa94c100f30 0x7fa94c101340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:47.741 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:47.113+0000 7fa951a9f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa94c100f30 0x7fa94c101340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54886/0 (socket says 172.21.15.112:54886) 2024-09-16T00:16:47.741 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:47.113+0000 7fa951a9f640 1 -- 172.21.15.112:0/4171773298 learned_addr learned my addr 172.21.15.112:0/4171773298 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:16:47.741 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:47.113+0000 7fa951a9f640 1 -- 172.21.15.112:0/4171773298 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa94c1019c0 con 0x7fa94c100f30 2024-09-16T00:16:47.742 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:47.114+0000 7fa951a9f640 1 --2- 172.21.15.112:0/4171773298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa94c100f30 0x7fa94c101340 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa948009840 tx=0x7fa9480091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa0460a9e2bdaaba server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:47.742 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:47.114+0000 7fa950a9d640 1 -- 172.21.15.112:0/4171773298 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa948032440 con 0x7fa94c100f30 2024-09-16T00:16:47.742 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:47.114+0000 7fa950a9d640 1 -- 172.21.15.112:0/4171773298 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1005+0+0 (secure 0 0 0) 0x7fa948032a00 con 0x7fa94c100f30 2024-09-16T00:16:47.742 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:47.114+0000 7fa950a9d640 1 -- 172.21.15.112:0/4171773298 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa9480319e0 con 0x7fa94c100f30 2024-09-16T00:16:47.742 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:47.115+0000 7fa953d03640 1 -- 172.21.15.112:0/4171773298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa94c100f30 msgr2=0x7fa94c101340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:47.742 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:47.115+0000 7fa953d03640 1 --2- 172.21.15.112:0/4171773298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa94c100f30 0x7fa94c101340 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa948009840 tx=0x7fa9480091d0 comp rx=0 tx=0).stop 2024-09-16T00:16:47.742 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:47.115+0000 7fa953d03640 1 -- 172.21.15.112:0/4171773298 shutdown_connections 2024-09-16T00:16:47.742 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:47.115+0000 7fa953d03640 1 --2- 172.21.15.112:0/4171773298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa94c100f30 0x7fa94c101340 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:47.742 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:47.115+0000 7fa953d03640 1 -- 172.21.15.112:0/4171773298 >> 172.21.15.112:0/4171773298 conn(0x7fa94c0fc730 msgr2=0x7fa94c0feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:47.742 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:47.115+0000 7fa953d03640 1 -- 172.21.15.112:0/4171773298 shutdown_connections 2024-09-16T00:16:47.742 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:47.115+0000 7fa953d03640 1 -- 172.21.15.112:0/4171773298 wait complete. 2024-09-16T00:16:47.742 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:47.115+0000 7fa953d03640 1 Processor -- start 2024-09-16T00:16:47.742 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:47.115+0000 7fa953d03640 1 -- start start 2024-09-16T00:16:47.742 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:47.115+0000 7fa953d03640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa94c100f30 0x7fa94c110e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:47.743 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:47.115+0000 7fa953d03640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa94c111340 con 0x7fa94c100f30 2024-09-16T00:16:47.743 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:47.116+0000 7fa951a9f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa94c100f30 0x7fa94c110e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:47.743 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:47.116+0000 7fa951a9f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa94c100f30 0x7fa94c110e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54894/0 (socket says 172.21.15.112:54894) 2024-09-16T00:16:47.743 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:47.116+0000 7fa951a9f640 1 -- 172.21.15.112:0/3539628204 learned_addr learned my addr 172.21.15.112:0/3539628204 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:16:47.743 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:47.116+0000 7fa951a9f640 1 -- 172.21.15.112:0/3539628204 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa948009450 con 0x7fa94c100f30 2024-09-16T00:16:47.743 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:47.116+0000 7fa951a9f640 1 --2- 172.21.15.112:0/3539628204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa94c100f30 0x7fa94c110e00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa9480098a0 tx=0x7fa948008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:47.743 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:47.116+0000 7fa9427fc640 1 -- 172.21.15.112:0/3539628204 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa948032880 con 0x7fa94c100f30 2024-09-16T00:16:47.743 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:47.116+0000 7fa9427fc640 1 -- 172.21.15.112:0/3539628204 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1005+0+0 (secure 0 0 0) 0x7fa94803d8b0 con 0x7fa94c100f30 2024-09-16T00:16:47.743 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:47.116+0000 7fa9427fc640 1 -- 172.21.15.112:0/3539628204 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa94803b910 con 0x7fa94c100f30 2024-09-16T00:16:47.743 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:47.116+0000 7fa953d03640 1 -- 172.21.15.112:0/3539628204 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa94c111540 con 0x7fa94c100f30 2024-09-16T00:16:47.743 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:47.116+0000 7fa953d03640 1 -- 172.21.15.112:0/3539628204 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa94c111f90 con 0x7fa94c100f30 2024-09-16T00:16:47.743 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:47.116+0000 7fa9427fc640 1 -- 172.21.15.112:0/3539628204 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7fa948039070 con 0x7fa94c100f30 2024-09-16T00:16:47.743 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:47.116+0000 7fa9427fc640 1 -- 172.21.15.112:0/3539628204 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fa948046710 con 0x7fa94c100f30 2024-09-16T00:16:47.743 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:47.117+0000 7fa953d03640 1 -- 172.21.15.112:0/3539628204 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa94c100390 con 0x7fa94c100f30 2024-09-16T00:16:47.744 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:47.119+0000 7fa9427fc640 1 -- 172.21.15.112:0/3539628204 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7fa9480657a0 con 0x7fa94c100f30 2024-09-16T00:16:47.744 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:47.174+0000 7fa953d03640 1 -- 172.21.15.112:0/3539628204 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fa94c1122d0 con 0x7fa94c100f30 2024-09-16T00:16:47.744 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:47.174+0000 7fa9427fc640 1 -- 172.21.15.112:0/3539628204 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+177 (secure 0 0 0) 0x7fa948054030 con 0x7fa94c100f30 2024-09-16T00:16:47.744 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:47.175+0000 7fa953d03640 1 -- 172.21.15.112:0/3539628204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa94c100f30 msgr2=0x7fa94c110e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:47.744 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:47.175+0000 7fa953d03640 1 --2- 172.21.15.112:0/3539628204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa94c100f30 0x7fa94c110e00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa9480098a0 tx=0x7fa948008040 comp rx=0 tx=0).stop 2024-09-16T00:16:47.744 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:47.175+0000 7fa953d03640 1 -- 172.21.15.112:0/3539628204 shutdown_connections 2024-09-16T00:16:47.744 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:47.175+0000 7fa953d03640 1 --2- 172.21.15.112:0/3539628204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa94c100f30 0x7fa94c110e00 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:47.744 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:47.175+0000 7fa953d03640 1 -- 172.21.15.112:0/3539628204 >> 172.21.15.112:0/3539628204 conn(0x7fa94c0fc730 msgr2=0x7fa94c0fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:47.744 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:47.175+0000 7fa953d03640 1 -- 172.21.15.112:0/3539628204 shutdown_connections 2024-09-16T00:16:47.744 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:47.175+0000 7fa953d03640 1 -- 172.21.15.112:0/3539628204 wait complete. 2024-09-16T00:16:47.744 INFO:teuthology.orchestra.run.smithi112.stdout:Restarting the monitor... 2024-09-16T00:16:48.022 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:47 smithi112 ceph-mon[27501]: from='client.? 172.21.15.112:0/3539628204' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:16:48.022 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:47 smithi112 systemd[1]: Stopping Ceph mon.smithi112 for d346dd56-73c0-11ef-bceb-c7b262605968... 2024-09-16T00:16:48.313 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:48 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-mon-smithi112[27497]: 2024-09-16T00:16:48.021+0000 7f24c2208640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi112 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:16:48.313 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:48 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-mon-smithi112[27497]: 2024-09-16T00:16:48.021+0000 7f24c2208640 -1 mon.smithi112@0(leader) e1 *** Got Signal Terminated *** 2024-09-16T00:16:48.313 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:48 smithi112 podman[27745]: 2024-09-16 00:16:48.042197485 +0000 UTC m=+0.132091070 container died 4059713a0f6e7a171539431fca4e72e0ed14a1ecd4d68fc61fb6bba439faa09d (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-mon-smithi112, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=quincy-6d025bc, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-16T00:16:48.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:48 smithi112 podman[27745]: 2024-09-16 00:16:48.412642596 +0000 UTC m=+0.502536178 container cleanup 4059713a0f6e7a171539431fca4e72e0ed14a1ecd4d68fc61fb6bba439faa09d (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-mon-smithi112, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=quincy-6d025bc, CEPH_POINT_RELEASE=, ceph=True) 2024-09-16T00:16:48.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:48 smithi112 bash[27745]: ceph-d346dd56-73c0-11ef-bceb-c7b262605968-mon-smithi112 2024-09-16T00:16:49.032 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:48 smithi112 podman[27757]: 2024-09-16 00:16:48.672089419 +0000 UTC m=+0.633516718 container remove 4059713a0f6e7a171539431fca4e72e0ed14a1ecd4d68fc61fb6bba439faa09d (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-mon-smithi112, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=quincy-6d025bc, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:16:49.032 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:48 smithi112 systemd[1]: ceph-d346dd56-73c0-11ef-bceb-c7b262605968@mon.smithi112.service: Deactivated successfully. 2024-09-16T00:16:49.032 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:48 smithi112 systemd[1]: Stopped Ceph mon.smithi112 for d346dd56-73c0-11ef-bceb-c7b262605968. 2024-09-16T00:16:49.032 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:48 smithi112 systemd[1]: Starting Ceph mon.smithi112 for d346dd56-73c0-11ef-bceb-c7b262605968... 2024-09-16T00:16:49.334 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 podman[27851]: 2024-09-16 00:16:48.934619173 +0000 UTC m=+0.015194466 image pull 33ed8dba399669a1d738ac1a85bc5f36daf3048afed44b7a93f620fc77496a99 quay.ceph.io/ceph-ci/ceph:quincy 2024-09-16T00:16:49.334 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 podman[27851]: 2024-09-16 00:16:49.064699685 +0000 UTC m=+0.145274954 container create ddaaced54bf8f3f21ac04a1d448f7340d5b51a8a869c703e359c81b1e715d401 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-mon-smithi112, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=quincy-6d025bc, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-16T00:16:49.334 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 podman[27851]: 2024-09-16 00:16:49.250596595 +0000 UTC m=+0.331171869 container init ddaaced54bf8f3f21ac04a1d448f7340d5b51a8a869c703e359c81b1e715d401 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-mon-smithi112, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=quincy-6d025bc, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-16T00:16:49.334 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 podman[27851]: 2024-09-16 00:16:49.253600219 +0000 UTC m=+0.334175489 container start ddaaced54bf8f3f21ac04a1d448f7340d5b51a8a869c703e359c81b1e715d401 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-mon-smithi112, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=quincy-6d025bc, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-16T00:16:49.334 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: set uid:gid to 167:167 (ceph:ceph) 2024-09-16T00:16:49.334 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: ceph version 17.2.7-1633-g6d025bc2 (6d025bc26f3176c910707f8cb4c8d8e076640615) quincy (stable), process ceph-mon, pid 2 2024-09-16T00:16:49.335 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: pidfile_write: ignore empty --pid-file 2024-09-16T00:16:49.335 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: load: jerasure load: lrc 2024-09-16T00:16:49.335 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: RocksDB version: 6.15.5 2024-09-16T00:16:49.335 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-09-16T00:16:49.335 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Compile date Sep 15 2024 2024-09-16T00:16:49.335 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: DB SUMMARY 2024-09-16T00:16:49.335 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: DB Session ID: E4CRJ4NCPU9HSM7ZF2AU 2024-09-16T00:16:49.335 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: CURRENT file: CURRENT 2024-09-16T00:16:49.335 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: IDENTITY file: IDENTITY 2024-09-16T00:16:49.336 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: MANIFEST file: MANIFEST-000009 size: 131 Bytes 2024-09-16T00:16:49.336 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi112/store.db dir, Total Num: 1, files: 000008.sst 2024-09-16T00:16:49.336 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi112/store.db: 000010.log size: 98450 ; 2024-09-16T00:16:49.336 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.error_if_exists: 0 2024-09-16T00:16:49.336 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.create_if_missing: 0 2024-09-16T00:16:49.336 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.paranoid_checks: 1 2024-09-16T00:16:49.337 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-16T00:16:49.337 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.env: 0x557ecce8a440 2024-09-16T00:16:49.337 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.fs: Posix File System 2024-09-16T00:16:49.337 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.info_log: 0x557ecf6910c0 2024-09-16T00:16:49.338 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.max_file_opening_threads: 16 2024-09-16T00:16:49.338 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.statistics: (nil) 2024-09-16T00:16:49.338 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.use_fsync: 0 2024-09-16T00:16:49.338 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.max_log_file_size: 0 2024-09-16T00:16:49.338 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-16T00:16:49.338 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-16T00:16:49.338 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.keep_log_file_num: 1000 2024-09-16T00:16:49.338 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.recycle_log_file_num: 0 2024-09-16T00:16:49.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.allow_fallocate: 1 2024-09-16T00:16:49.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.allow_mmap_reads: 0 2024-09-16T00:16:49.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.allow_mmap_writes: 0 2024-09-16T00:16:49.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.use_direct_reads: 0 2024-09-16T00:16:49.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-16T00:16:49.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.create_missing_column_families: 0 2024-09-16T00:16:49.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.db_log_dir: 2024-09-16T00:16:49.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-smithi112/store.db 2024-09-16T00:16:49.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-16T00:16:49.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-16T00:16:49.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-16T00:16:49.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-16T00:16:49.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-16T00:16:49.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-16T00:16:49.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.advise_random_on_open: 1 2024-09-16T00:16:49.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.db_write_buffer_size: 0 2024-09-16T00:16:49.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.write_buffer_manager: 0x557ecf74e810 2024-09-16T00:16:49.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-16T00:16:49.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-09-16T00:16:49.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-16T00:16:49.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-16T00:16:49.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.rate_limiter: (nil) 2024-09-16T00:16:49.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-16T00:16:49.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.wal_recovery_mode: 2 2024-09-16T00:16:49.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.enable_thread_tracking: 0 2024-09-16T00:16:49.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.enable_pipelined_write: 0 2024-09-16T00:16:49.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.unordered_write: 0 2024-09-16T00:16:49.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-16T00:16:49.342 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-16T00:16:49.342 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-16T00:16:49.342 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-16T00:16:49.342 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.row_cache: None 2024-09-16T00:16:49.342 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.wal_filter: None 2024-09-16T00:16:49.342 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-16T00:16:49.342 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.allow_ingest_behind: 0 2024-09-16T00:16:49.342 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.preserve_deletes: 0 2024-09-16T00:16:49.342 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.two_write_queues: 0 2024-09-16T00:16:49.342 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.manual_wal_flush: 0 2024-09-16T00:16:49.342 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.atomic_flush: 0 2024-09-16T00:16:49.342 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-16T00:16:49.343 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-16T00:16:49.343 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-16T00:16:49.343 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.log_readahead_size: 0 2024-09-16T00:16:49.343 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-16T00:16:49.343 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.best_efforts_recovery: 0 2024-09-16T00:16:49.343 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-16T00:16:49.343 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-16T00:16:49.343 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.allow_data_in_errors: 0 2024-09-16T00:16:49.343 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.db_host_id: __hostname__ 2024-09-16T00:16:49.343 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.max_background_jobs: 2 2024-09-16T00:16:49.343 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.max_background_compactions: -1 2024-09-16T00:16:49.344 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.max_subcompactions: 1 2024-09-16T00:16:49.344 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-16T00:16:49.344 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-16T00:16:49.344 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-16T00:16:49.344 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.max_total_wal_size: 0 2024-09-16T00:16:49.344 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-16T00:16:49.344 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-16T00:16:49.344 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-16T00:16:49.344 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-16T00:16:49.344 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.max_open_files: -1 2024-09-16T00:16:49.344 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.bytes_per_sync: 0 2024-09-16T00:16:49.344 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-16T00:16:49.344 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-16T00:16:49.345 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.compaction_readahead_size: 0 2024-09-16T00:16:49.345 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.max_background_flushes: -1 2024-09-16T00:16:49.345 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Compression algorithms supported: 2024-09-16T00:16:49.345 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: kZSTD supported: 0 2024-09-16T00:16:49.345 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: kXpressCompression supported: 0 2024-09-16T00:16:49.345 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: kBZip2Compression supported: 0 2024-09-16T00:16:49.345 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-16T00:16:49.345 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: kLZ4Compression supported: 1 2024-09-16T00:16:49.345 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: kZlibCompression supported: 1 2024-09-16T00:16:49.345 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: kLZ4HCCompression supported: 1 2024-09-16T00:16:49.345 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: kSnappyCompression supported: 1 2024-09-16T00:16:49.345 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-16T00:16:49.346 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi112/store.db/MANIFEST-000009 2024-09-16T00:16:49.346 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-09-16T00:16:49.346 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-16T00:16:49.346 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.merge_operator: 2024-09-16T00:16:49.346 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.compaction_filter: None 2024-09-16T00:16:49.346 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.compaction_filter_factory: None 2024-09-16T00:16:49.346 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.sst_partitioner_factory: None 2024-09-16T00:16:49.346 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-16T00:16:49.346 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-16T00:16:49.346 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x557ecf637a38) 2024-09-16T00:16:49.346 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: cache_index_and_filter_blocks: 1 2024-09-16T00:16:49.346 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-16T00:16:49.346 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-16T00:16:49.346 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: pin_top_level_index_and_filter: 1 2024-09-16T00:16:49.346 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: index_type: 0 2024-09-16T00:16:49.347 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: data_block_index_type: 0 2024-09-16T00:16:49.347 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: index_shortening: 1 2024-09-16T00:16:49.347 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-16T00:16:49.347 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: hash_index_allow_collision: 1 2024-09-16T00:16:49.347 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: checksum: 1 2024-09-16T00:16:49.347 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: no_block_cache: 0 2024-09-16T00:16:49.347 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: block_cache: 0x557ecf66d1f0 2024-09-16T00:16:49.347 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: block_cache_name: BinnedLRUCache 2024-09-16T00:16:49.347 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: block_cache_options: 2024-09-16T00:16:49.347 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: capacity : 536870912 2024-09-16T00:16:49.347 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: num_shard_bits : 4 2024-09-16T00:16:49.347 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: strict_capacity_limit : 0 2024-09-16T00:16:49.347 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: high_pri_pool_ratio: 0.000 2024-09-16T00:16:49.348 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: block_cache_compressed: (nil) 2024-09-16T00:16:49.348 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: persistent_cache: (nil) 2024-09-16T00:16:49.348 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: block_size: 4096 2024-09-16T00:16:49.348 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: block_size_deviation: 10 2024-09-16T00:16:49.348 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: block_restart_interval: 16 2024-09-16T00:16:49.348 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: index_block_restart_interval: 1 2024-09-16T00:16:49.348 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: metadata_block_size: 4096 2024-09-16T00:16:49.348 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: partition_filters: 0 2024-09-16T00:16:49.348 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: use_delta_encoding: 1 2024-09-16T00:16:49.348 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-09-16T00:16:49.348 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: whole_key_filtering: 1 2024-09-16T00:16:49.348 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: verify_compression: 0 2024-09-16T00:16:49.348 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: read_amp_bytes_per_bit: 0 2024-09-16T00:16:49.348 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: format_version: 4 2024-09-16T00:16:49.349 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: enable_index_compression: 1 2024-09-16T00:16:49.349 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: block_align: 0 2024-09-16T00:16:49.349 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.write_buffer_size: 33554432 2024-09-16T00:16:49.349 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.max_write_buffer_number: 2 2024-09-16T00:16:49.349 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.compression: NoCompression 2024-09-16T00:16:49.349 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.bottommost_compression: Disabled 2024-09-16T00:16:49.349 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.prefix_extractor: nullptr 2024-09-16T00:16:49.349 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-16T00:16:49.349 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.num_levels: 7 2024-09-16T00:16:49.349 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-16T00:16:49.349 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-16T00:16:49.349 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-16T00:16:49.349 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-16T00:16:49.350 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-16T00:16:49.350 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-16T00:16:49.350 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-16T00:16:49.350 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-16T00:16:49.350 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-16T00:16:49.350 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-16T00:16:49.350 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-16T00:16:49.350 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.compression_opts.level: 32767 2024-09-16T00:16:49.350 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.compression_opts.strategy: 0 2024-09-16T00:16:49.350 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-16T00:16:49.350 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-16T00:16:49.351 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-16T00:16:49.351 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.compression_opts.enabled: false 2024-09-16T00:16:49.351 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-16T00:16:49.351 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-16T00:16:49.351 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-16T00:16:49.351 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.target_file_size_base: 67108864 2024-09-16T00:16:49.351 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-16T00:16:49.351 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-16T00:16:49.352 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-16T00:16:49.352 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-16T00:16:49.352 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-16T00:16:49.352 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-16T00:16:49.352 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-16T00:16:49.352 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-16T00:16:49.352 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-16T00:16:49.352 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-16T00:16:49.353 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-16T00:16:49.353 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-16T00:16:49.353 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-16T00:16:49.353 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.arena_block_size: 4194304 2024-09-16T00:16:49.353 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-16T00:16:49.353 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-16T00:16:49.354 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-09-16T00:16:49.354 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.disable_auto_compactions: 0 2024-09-16T00:16:49.354 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-16T00:16:49.354 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-16T00:16:49.354 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-16T00:16:49.354 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-16T00:16:49.355 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-16T00:16:49.355 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-16T00:16:49.355 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-16T00:16:49.355 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-16T00:16:49.355 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-16T00:16:49.355 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-16T00:16:49.355 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-16T00:16:49.356 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.inplace_update_support: 0 2024-09-16T00:16:49.356 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-16T00:16:49.356 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-16T00:16:49.356 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-16T00:16:49.356 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-16T00:16:49.356 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.bloom_locality: 0 2024-09-16T00:16:49.357 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.max_successive_merges: 0 2024-09-16T00:16:49.357 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-16T00:16:49.357 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.paranoid_file_checks: 0 2024-09-16T00:16:49.357 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.force_consistency_checks: 1 2024-09-16T00:16:49.357 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.report_bg_io_stats: 0 2024-09-16T00:16:49.357 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.ttl: 2592000 2024-09-16T00:16:49.357 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-16T00:16:49.358 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.enable_blob_files: false 2024-09-16T00:16:49.358 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.min_blob_size: 0 2024-09-16T00:16:49.359 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.blob_file_size: 268435456 2024-09-16T00:16:49.359 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-16T00:16:49.359 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-16T00:16:49.360 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-16T00:16:49.360 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi112/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 11, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-16T00:16:49.360 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 5 2024-09-16T00:16:49.360 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: [db/version_set.cc:4082] Creating manifest 13 2024-09-16T00:16:49.360 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726445809277134, "job": 1, "event": "recovery_started", "wal_files": [10]} 2024-09-16T00:16:49.361 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #10 mode 2 2024-09-16T00:16:49.361 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-09-16T00:16:49.361 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726445809278013, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 14, "file_size": 93527, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 91607, "index_size": 254, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13594, "raw_average_key_size": 49, "raw_value_size": 84163, "raw_average_value_size": 306, "num_data_blocks": 11, "num_entries": 275, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1726445809, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "398da636-9c8c-42fa-8d49-90bdbe1a55a9", "db_session_id": "E4CRJ4NCPU9HSM7ZF2AU"}} 2024-09-16T00:16:49.361 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: [db/version_set.cc:4082] Creating manifest 15 2024-09-16T00:16:49.361 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726445809278337, "job": 1, "event": "recovery_finished"} 2024-09-16T00:16:49.361 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-smithi112/store.db/000010.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-16T00:16:49.362 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x557ecf6a0700 2024-09-16T00:16:49.362 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: DB pointer 0x557ecf670000 2024-09-16T00:16:49.362 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-09-16T00:16:49.362 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-09-16T00:16:49.362 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: ** DB Stats ** 2024-09-16T00:16:49.362 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T00:16:49.363 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-16T00:16:49.363 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T00:16:49.363 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T00:16:49.363 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-16T00:16:49.363 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-09-16T00:16:49.363 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T00:16:49.364 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: 2024-09-16T00:16:49.364 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: ** Compaction Stats [default] ** 2024-09-16T00:16:49.364 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-16T00:16:49.364 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-16T00:16:49.364 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: L0 2/0 93.16 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 146.2 0.00 0.00 1 0.001 0 0 2024-09-16T00:16:49.364 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: Sum 2/0 93.16 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 146.2 0.00 0.00 1 0.001 0 0 2024-09-16T00:16:49.365 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 146.2 0.00 0.00 1 0.001 0 0 2024-09-16T00:16:49.365 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: 2024-09-16T00:16:49.365 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: ** Compaction Stats [default] ** 2024-09-16T00:16:49.365 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-16T00:16:49.365 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-16T00:16:49.365 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 146.2 0.00 0.00 1 0.001 0 0 2024-09-16T00:16:49.366 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T00:16:49.366 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-16T00:16:49.366 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-16T00:16:49.366 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-16T00:16:49.366 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-16T00:16:49.366 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-16T00:16:49.366 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: Cumulative compaction: 0.00 GB write, 28.61 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T00:16:49.367 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: Interval compaction: 0.00 GB write, 28.61 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T00:16:49.367 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-16T00:16:49.367 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: 2024-09-16T00:16:49.367 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-16T00:16:49.367 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: 2024-09-16T00:16:49.367 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: ** Compaction Stats [default] ** 2024-09-16T00:16:49.367 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-16T00:16:49.368 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-16T00:16:49.368 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: L0 2/0 93.16 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 146.2 0.00 0.00 1 0.001 0 0 2024-09-16T00:16:49.368 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: Sum 2/0 93.16 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 146.2 0.00 0.00 1 0.001 0 0 2024-09-16T00:16:49.368 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-09-16T00:16:49.368 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: 2024-09-16T00:16:49.368 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: ** Compaction Stats [default] ** 2024-09-16T00:16:49.369 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-16T00:16:49.369 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-16T00:16:49.369 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 146.2 0.00 0.00 1 0.001 0 0 2024-09-16T00:16:49.369 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T00:16:49.369 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-16T00:16:49.369 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-16T00:16:49.370 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-16T00:16:49.370 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-16T00:16:49.370 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-16T00:16:49.370 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: Cumulative compaction: 0.00 GB write, 28.32 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T00:16:49.370 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T00:16:49.370 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-16T00:16:49.371 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: 2024-09-16T00:16:49.371 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-16T00:16:49.371 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: starting mon.smithi112 rank 0 at public addrs [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] at bind addrs [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi112 fsid d346dd56-73c0-11ef-bceb-c7b262605968 2024-09-16T00:16:49.371 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: mon.smithi112@-1(???) e1 preinit fsid d346dd56-73c0-11ef-bceb-c7b262605968 2024-09-16T00:16:49.371 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: mon.smithi112@-1(???).mds e1 new map 2024-09-16T00:16:49.371 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: mon.smithi112@-1(???).mds e1 print_map 2024-09-16T00:16:49.372 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: e1 2024-09-16T00:16:49.372 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T00:16:49.372 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:16:49.372 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: legacy client fscid: -1 2024-09-16T00:16:49.372 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: 2024-09-16T00:16:49.372 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: No filesystems configured 2024-09-16T00:16:49.372 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: mon.smithi112@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-16T00:16:49.373 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: mon.smithi112@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T00:16:49.373 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: mon.smithi112@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T00:16:49.373 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: mon.smithi112@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T00:16:49.373 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: mon.smithi112@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-16T00:16:49.373 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 bash[27851]: ddaaced54bf8f3f21ac04a1d448f7340d5b51a8a869c703e359c81b1e715d401 2024-09-16T00:16:49.373 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 systemd[1]: Started Ceph mon.smithi112 for d346dd56-73c0-11ef-bceb-c7b262605968. 2024-09-16T00:16:49.374 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: mon.smithi112 is new leader, mons smithi112 in quorum (ranks 0) 2024-09-16T00:16:49.374 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: monmap e1: 1 mons at {smithi112=[v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-16T00:16:49.374 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: fsmap 2024-09-16T00:16:49.374 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: osdmap e1: 0 total, 0 up, 0 in 2024-09-16T00:16:49.374 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:49 smithi112 ceph-mon[27869]: mgrmap e1: no daemons active 2024-09-16T00:16:49.375 INFO:teuthology.orchestra.run.smithi112.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-16T00:16:50.960 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:50.196+0000 7f20f3983640 1 Processor -- start 2024-09-16T00:16:50.964 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:50.196+0000 7f20f3983640 1 -- start start 2024-09-16T00:16:50.964 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:50.196+0000 7f20f3983640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20ec102390 0x7f20ec1027a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:50.964 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:50.196+0000 7f20f3983640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20ec102d70 con 0x7f20ec102390 2024-09-16T00:16:50.964 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:50.197+0000 7f20f171f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20ec102390 0x7f20ec1027a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:50.964 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:50.197+0000 7f20f171f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20ec102390 0x7f20ec1027a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54910/0 (socket says 172.21.15.112:54910) 2024-09-16T00:16:50.965 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:50.197+0000 7f20f171f640 1 -- 172.21.15.112:0/356480171 learned_addr learned my addr 172.21.15.112:0/356480171 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:16:50.965 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:50.197+0000 7f20f171f640 1 -- 172.21.15.112:0/356480171 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20ec102eb0 con 0x7f20ec102390 2024-09-16T00:16:50.965 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:50.199+0000 7f20f171f640 1 --2- 172.21.15.112:0/356480171 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20ec102390 0x7f20ec1027a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f20e8009840 tx=0x7f20e80091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f242af95a7240c59 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:50.965 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:50.199+0000 7f20e3fff640 1 -- 172.21.15.112:0/356480171 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f20e8032440 con 0x7f20ec102390 2024-09-16T00:16:50.965 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:50.199+0000 7f20e3fff640 1 -- 172.21.15.112:0/356480171 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1005+0+0 (secure 0 0 0) 0x7f20e8032a00 con 0x7f20ec102390 2024-09-16T00:16:50.965 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:50.199+0000 7f20e3fff640 1 -- 172.21.15.112:0/356480171 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f20e80319e0 con 0x7f20ec102390 2024-09-16T00:16:50.965 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:50.200+0000 7f20f3983640 1 -- 172.21.15.112:0/356480171 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20ec102390 msgr2=0x7f20ec1027a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:50.965 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:50.200+0000 7f20f3983640 1 --2- 172.21.15.112:0/356480171 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20ec102390 0x7f20ec1027a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f20e8009840 tx=0x7f20e80091d0 comp rx=0 tx=0).stop 2024-09-16T00:16:50.965 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:50.200+0000 7f20f3983640 1 -- 172.21.15.112:0/356480171 shutdown_connections 2024-09-16T00:16:50.965 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:50.200+0000 7f20f3983640 1 --2- 172.21.15.112:0/356480171 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20ec102390 0x7f20ec1027a0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:50.965 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:50.200+0000 7f20f3983640 1 -- 172.21.15.112:0/356480171 >> 172.21.15.112:0/356480171 conn(0x7f20ec0fbdd0 msgr2=0x7f20ec0fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:50.965 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:50.200+0000 7f20f3983640 1 -- 172.21.15.112:0/356480171 shutdown_connections 2024-09-16T00:16:50.965 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:50.200+0000 7f20f3983640 1 -- 172.21.15.112:0/356480171 wait complete. 2024-09-16T00:16:50.966 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:50.200+0000 7f20f3983640 1 Processor -- start 2024-09-16T00:16:50.966 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:50.200+0000 7f20f3983640 1 -- start start 2024-09-16T00:16:50.966 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:50.201+0000 7f20f3983640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20ec102390 0x7f20ec112f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:50.966 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:50.201+0000 7f20f3983640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20ec113470 con 0x7f20ec102390 2024-09-16T00:16:50.966 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:50.201+0000 7f20f171f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20ec102390 0x7f20ec112f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:50.966 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:50.201+0000 7f20f171f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20ec102390 0x7f20ec112f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54920/0 (socket says 172.21.15.112:54920) 2024-09-16T00:16:50.966 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:50.201+0000 7f20f171f640 1 -- 172.21.15.112:0/216858373 learned_addr learned my addr 172.21.15.112:0/216858373 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:16:50.966 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:50.201+0000 7f20f171f640 1 -- 172.21.15.112:0/216858373 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e8009450 con 0x7f20ec102390 2024-09-16T00:16:50.966 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:50.201+0000 7f20f171f640 1 --2- 172.21.15.112:0/216858373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20ec102390 0x7f20ec112f30 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f20e80098a0 tx=0x7f20e8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:50.966 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:50.201+0000 7f20e1ffb640 1 -- 172.21.15.112:0/216858373 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f20e8032880 con 0x7f20ec102390 2024-09-16T00:16:50.966 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:50.201+0000 7f20e1ffb640 1 -- 172.21.15.112:0/216858373 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1005+0+0 (secure 0 0 0) 0x7f20e803d8b0 con 0x7f20ec102390 2024-09-16T00:16:50.966 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:50.201+0000 7f20f3983640 1 -- 172.21.15.112:0/216858373 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20ec113670 con 0x7f20ec102390 2024-09-16T00:16:50.966 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:50.201+0000 7f20e1ffb640 1 -- 172.21.15.112:0/216858373 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f20e803b910 con 0x7f20ec102390 2024-09-16T00:16:50.966 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:50.202+0000 7f20f3983640 1 -- 172.21.15.112:0/216858373 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20ec113ae0 con 0x7f20ec102390 2024-09-16T00:16:50.967 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:50.202+0000 7f20e1ffb640 1 -- 172.21.15.112:0/216858373 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f20e8039070 con 0x7f20ec102390 2024-09-16T00:16:50.967 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:50.202+0000 7f20e1ffb640 1 -- 172.21.15.112:0/216858373 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f20e8046760 con 0x7f20ec102390 2024-09-16T00:16:50.967 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:50.203+0000 7f20f3983640 1 -- 172.21.15.112:0/216858373 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20ec04b720 con 0x7f20ec102390 2024-09-16T00:16:50.967 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:50.205+0000 7f20e1ffb640 1 -- 172.21.15.112:0/216858373 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f20e80657a0 con 0x7f20ec102390 2024-09-16T00:16:50.967 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:50.259+0000 7f20f3983640 1 -- 172.21.15.112:0/216858373 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f20ec108110 con 0x7f20ec102390 2024-09-16T00:16:50.967 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:50.263+0000 7f20e1ffb640 1 -- 172.21.15.112:0/216858373 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f20e8054020 con 0x7f20ec102390 2024-09-16T00:16:50.967 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:50.264+0000 7f20f3983640 1 -- 172.21.15.112:0/216858373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20ec102390 msgr2=0x7f20ec112f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:50.967 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:50.264+0000 7f20f3983640 1 --2- 172.21.15.112:0/216858373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20ec102390 0x7f20ec112f30 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f20e80098a0 tx=0x7f20e8008040 comp rx=0 tx=0).stop 2024-09-16T00:16:50.967 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:50.264+0000 7f20f3983640 1 -- 172.21.15.112:0/216858373 shutdown_connections 2024-09-16T00:16:50.967 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:50.264+0000 7f20f3983640 1 --2- 172.21.15.112:0/216858373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20ec102390 0x7f20ec112f30 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:50.967 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:50.264+0000 7f20f3983640 1 -- 172.21.15.112:0/216858373 >> 172.21.15.112:0/216858373 conn(0x7f20ec0fbdd0 msgr2=0x7f20ec0fc950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:50.967 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:50.264+0000 7f20f3983640 1 -- 172.21.15.112:0/216858373 shutdown_connections 2024-09-16T00:16:50.967 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:50.264+0000 7f20f3983640 1 -- 172.21.15.112:0/216858373 wait complete. 2024-09-16T00:16:50.967 INFO:teuthology.orchestra.run.smithi112.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-16T00:16:50.968 INFO:teuthology.orchestra.run.smithi112.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-16T00:16:50.968 INFO:teuthology.orchestra.run.smithi112.stdout:Creating mgr... 2024-09-16T00:16:50.968 INFO:teuthology.orchestra.run.smithi112.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-16T00:16:50.968 INFO:teuthology.orchestra.run.smithi112.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-16T00:16:50.968 INFO:teuthology.orchestra.run.smithi112.stdout:Verifying port 0.0.0.0:8443 ... 2024-09-16T00:16:51.142 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-d346dd56-73c0-11ef-bceb-c7b262605968@mgr.smithi112.xrkkbc 2024-09-16T00:16:51.142 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Failed to reset failed state of unit ceph-d346dd56-73c0-11ef-bceb-c7b262605968@mgr.smithi112.xrkkbc.service: Unit ceph-d346dd56-73c0-11ef-bceb-c7b262605968@mgr.smithi112.xrkkbc.service not loaded. 2024-09-16T00:16:51.288 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-d346dd56-73c0-11ef-bceb-c7b262605968.target.wants/ceph-d346dd56-73c0-11ef-bceb-c7b262605968@mgr.smithi112.xrkkbc.service → /etc/systemd/system/ceph-d346dd56-73c0-11ef-bceb-c7b262605968@.service. 2024-09-16T00:16:51.513 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:51 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/216858373' entity='client.admin' 2024-09-16T00:16:51.858 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-16T00:16:51.859 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout disabled 2024-09-16T00:16:51.866 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-16T00:16:51.867 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout inactive 2024-09-16T00:16:51.867 INFO:teuthology.orchestra.run.smithi112.stdout:firewalld.service is not enabled 2024-09-16T00:16:51.867 INFO:teuthology.orchestra.run.smithi112.stdout:Not possible to enable service . firewalld.service is not available 2024-09-16T00:16:51.875 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-16T00:16:51.876 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout disabled 2024-09-16T00:16:51.883 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-16T00:16:51.884 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout inactive 2024-09-16T00:16:51.884 INFO:teuthology.orchestra.run.smithi112.stdout:firewalld.service is not enabled 2024-09-16T00:16:51.884 INFO:teuthology.orchestra.run.smithi112.stdout:Not possible to open ports <[9283, 8765, 8443]>. firewalld.service is not available 2024-09-16T00:16:51.884 INFO:teuthology.orchestra.run.smithi112.stdout:Waiting for mgr to start... 2024-09-16T00:16:51.884 INFO:teuthology.orchestra.run.smithi112.stdout:Waiting for mgr... 2024-09-16T00:16:52.659 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:52 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/2623812681' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-16T00:16:52.964 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-09-16T00:16:52.964 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout { 2024-09-16T00:16:52.965 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "fsid": "d346dd56-73c0-11ef-bceb-c7b262605968", 2024-09-16T00:16:52.965 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "health": { 2024-09-16T00:16:52.965 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-16T00:16:52.965 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-16T00:16:52.965 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-16T00:16:52.965 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:16:52.965 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-16T00:16:52.965 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-16T00:16:52.965 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 0 2024-09-16T00:16:52.965 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-09-16T00:16:52.965 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-16T00:16:52.965 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "smithi112" 2024-09-16T00:16:52.965 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-09-16T00:16:52.965 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-09-16T00:16:52.965 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-16T00:16:52.966 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T00:16:52.966 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-09-16T00:16:52.966 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-16T00:16:52.966 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:16:52.966 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-16T00:16:52.966 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T00:16:52.966 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-16T00:16:52.966 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-16T00:16:52.966 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-16T00:16:52.973 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-16T00:16:52.973 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-16T00:16:52.973 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-16T00:16:52.973 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:16:52.974 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-16T00:16:52.974 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-16T00:16:52.974 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-16T00:16:52.974 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-16T00:16:52.974 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-16T00:16:52.974 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-16T00:16:52.974 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-16T00:16:52.974 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-16T00:16:52.974 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-16T00:16:52.974 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:16:52.974 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-16T00:16:52.974 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T00:16:52.974 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-16T00:16:52.974 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-16T00:16:52.975 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:16:52.975 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-16T00:16:52.975 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-16T00:16:52.975 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-16T00:16:52.975 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-16T00:16:52.975 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-16T00:16:52.975 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-16T00:16:52.975 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "restful" 2024-09-16T00:16:52.975 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-09-16T00:16:52.975 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T00:16:52.975 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:16:52.975 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-16T00:16:52.975 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T00:16:52.975 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "modified": "2024-09-16T00:16:43.747582+0000", 2024-09-16T00:16:52.975 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T00:16:52.976 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:16:52.976 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-16T00:16:52.976 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout } 2024-09-16T00:16:52.976 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:52.314+0000 7f661c785640 1 Processor -- start 2024-09-16T00:16:52.976 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:52.315+0000 7f661c785640 1 -- start start 2024-09-16T00:16:52.976 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:52.315+0000 7f661c785640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6614078c90 0x7f6614077210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:52.976 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:52.315+0000 7f661c785640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66140790a0 con 0x7f6614078c90 2024-09-16T00:16:52.976 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:52.315+0000 7f661a521640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6614078c90 0x7f6614077210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:52.976 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:52.315+0000 7f661a521640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6614078c90 0x7f6614077210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35424/0 (socket says 172.21.15.112:35424) 2024-09-16T00:16:52.976 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:52.315+0000 7f661a521640 1 -- 172.21.15.112:0/3884415606 learned_addr learned my addr 172.21.15.112:0/3884415606 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:16:52.976 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:52.316+0000 7f661a521640 1 -- 172.21.15.112:0/3884415606 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6614077750 con 0x7f6614078c90 2024-09-16T00:16:52.976 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:52.317+0000 7f661a521640 1 --2- 172.21.15.112:0/3884415606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6614078c90 0x7f6614077210 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6610009840 tx=0x7f66100091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfe39ee1721c23b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:52.976 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:52.317+0000 7f661951f640 1 -- 172.21.15.112:0/3884415606 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6610032440 con 0x7f6614078c90 2024-09-16T00:16:52.976 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:52.317+0000 7f661951f640 1 -- 172.21.15.112:0/3884415606 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1005+0+0 (secure 0 0 0) 0x7f6610032a00 con 0x7f6614078c90 2024-09-16T00:16:52.977 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:52.317+0000 7f661951f640 1 -- 172.21.15.112:0/3884415606 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f66100319e0 con 0x7f6614078c90 2024-09-16T00:16:52.977 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:52.318+0000 7f661c785640 1 -- 172.21.15.112:0/3884415606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6614078c90 msgr2=0x7f6614077210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:52.977 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:52.318+0000 7f661c785640 1 --2- 172.21.15.112:0/3884415606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6614078c90 0x7f6614077210 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6610009840 tx=0x7f66100091d0 comp rx=0 tx=0).stop 2024-09-16T00:16:52.977 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:52.318+0000 7f661c785640 1 -- 172.21.15.112:0/3884415606 shutdown_connections 2024-09-16T00:16:52.977 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:52.318+0000 7f661c785640 1 --2- 172.21.15.112:0/3884415606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6614078c90 0x7f6614077210 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:52.977 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:52.318+0000 7f661c785640 1 -- 172.21.15.112:0/3884415606 >> 172.21.15.112:0/3884415606 conn(0x7f66140fd740 msgr2=0x7f66140ffb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:52.977 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:52.318+0000 7f661c785640 1 -- 172.21.15.112:0/3884415606 shutdown_connections 2024-09-16T00:16:52.977 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:52.318+0000 7f661c785640 1 -- 172.21.15.112:0/3884415606 wait complete. 2024-09-16T00:16:52.977 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:52.319+0000 7f661c785640 1 Processor -- start 2024-09-16T00:16:52.977 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:52.319+0000 7f661c785640 1 -- start start 2024-09-16T00:16:52.977 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:52.319+0000 7f661c785640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6614078c90 0x7f6614112f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:52.977 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:52.319+0000 7f661c785640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66141134a0 con 0x7f6614078c90 2024-09-16T00:16:52.977 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:52.319+0000 7f661a521640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6614078c90 0x7f6614112f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:52.977 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:52.319+0000 7f661a521640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6614078c90 0x7f6614112f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35436/0 (socket says 172.21.15.112:35436) 2024-09-16T00:16:52.978 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:52.319+0000 7f661a521640 1 -- 172.21.15.112:0/2623812681 learned_addr learned my addr 172.21.15.112:0/2623812681 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:16:52.978 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:52.319+0000 7f661a521640 1 -- 172.21.15.112:0/2623812681 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6610009450 con 0x7f6614078c90 2024-09-16T00:16:52.978 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:52.319+0000 7f661a521640 1 --2- 172.21.15.112:0/2623812681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6614078c90 0x7f6614112f60 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6610006f00 tx=0x7f6610008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:52.978 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:52.320+0000 7f660affd640 1 -- 172.21.15.112:0/2623812681 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6610031e30 con 0x7f6614078c90 2024-09-16T00:16:52.978 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:52.320+0000 7f660affd640 1 -- 172.21.15.112:0/2623812681 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1005+0+0 (secure 0 0 0) 0x7f661003d8b0 con 0x7f6614078c90 2024-09-16T00:16:52.978 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:52.320+0000 7f661c785640 1 -- 172.21.15.112:0/2623812681 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66141136a0 con 0x7f6614078c90 2024-09-16T00:16:52.978 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:52.320+0000 7f660affd640 1 -- 172.21.15.112:0/2623812681 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f661003b8f0 con 0x7f6614078c90 2024-09-16T00:16:52.978 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:52.320+0000 7f661c785640 1 -- 172.21.15.112:0/2623812681 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6614113b10 con 0x7f6614078c90 2024-09-16T00:16:52.978 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:52.320+0000 7f660affd640 1 -- 172.21.15.112:0/2623812681 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f6610039070 con 0x7f6614078c90 2024-09-16T00:16:52.978 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:52.320+0000 7f660affd640 1 -- 172.21.15.112:0/2623812681 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f6610046710 con 0x7f6614078c90 2024-09-16T00:16:52.978 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:52.321+0000 7f661c785640 1 -- 172.21.15.112:0/2623812681 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f661404b720 con 0x7f6614078c90 2024-09-16T00:16:52.978 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:52.323+0000 7f660affd640 1 -- 172.21.15.112:0/2623812681 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f66100657a0 con 0x7f6614078c90 2024-09-16T00:16:52.978 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:52.381+0000 7f661c785640 1 -- 172.21.15.112:0/2623812681 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f6614114230 con 0x7f6614078c90 2024-09-16T00:16:52.978 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:52.382+0000 7f660affd640 1 -- 172.21.15.112:0/2623812681 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7f6610054020 con 0x7f6614078c90 2024-09-16T00:16:52.979 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:52.382+0000 7f661c785640 1 -- 172.21.15.112:0/2623812681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6614078c90 msgr2=0x7f6614112f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:52.979 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:52.382+0000 7f661c785640 1 --2- 172.21.15.112:0/2623812681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6614078c90 0x7f6614112f60 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6610006f00 tx=0x7f6610008040 comp rx=0 tx=0).stop 2024-09-16T00:16:52.979 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:52.383+0000 7f661c785640 1 -- 172.21.15.112:0/2623812681 shutdown_connections 2024-09-16T00:16:52.979 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:52.383+0000 7f661c785640 1 --2- 172.21.15.112:0/2623812681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6614078c90 0x7f6614112f60 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:52.979 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:52.383+0000 7f661c785640 1 -- 172.21.15.112:0/2623812681 >> 172.21.15.112:0/2623812681 conn(0x7f66140fd740 msgr2=0x7f66140fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:52.979 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:52.383+0000 7f661c785640 1 -- 172.21.15.112:0/2623812681 shutdown_connections 2024-09-16T00:16:52.979 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:52.383+0000 7f661c785640 1 -- 172.21.15.112:0/2623812681 wait complete. 2024-09-16T00:16:52.979 INFO:teuthology.orchestra.run.smithi112.stdout:mgr not available, waiting (1/15)... 2024-09-16T00:16:55.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:55 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/1413375172' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-16T00:16:56.338 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-09-16T00:16:56.339 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout { 2024-09-16T00:16:56.339 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "fsid": "d346dd56-73c0-11ef-bceb-c7b262605968", 2024-09-16T00:16:56.339 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "health": { 2024-09-16T00:16:56.339 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-16T00:16:56.339 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-16T00:16:56.339 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-16T00:16:56.339 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:16:56.339 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-16T00:16:56.339 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-16T00:16:56.339 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 0 2024-09-16T00:16:56.339 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-09-16T00:16:56.339 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-16T00:16:56.340 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "smithi112" 2024-09-16T00:16:56.340 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-09-16T00:16:56.340 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-09-16T00:16:56.340 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-16T00:16:56.340 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T00:16:56.340 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-09-16T00:16:56.340 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-16T00:16:56.340 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:16:56.340 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-16T00:16:56.340 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T00:16:56.340 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-16T00:16:56.340 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-16T00:16:56.341 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-16T00:16:56.348 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-16T00:16:56.348 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-16T00:16:56.348 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-16T00:16:56.348 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:16:56.349 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-16T00:16:56.349 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-16T00:16:56.349 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-16T00:16:56.349 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-16T00:16:56.349 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-16T00:16:56.349 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-16T00:16:56.349 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-16T00:16:56.350 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-16T00:16:56.350 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-16T00:16:56.350 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:16:56.350 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-16T00:16:56.350 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T00:16:56.350 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-16T00:16:56.351 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-16T00:16:56.351 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:16:56.351 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-16T00:16:56.351 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-16T00:16:56.351 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-16T00:16:56.351 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-16T00:16:56.352 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-16T00:16:56.352 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-16T00:16:56.352 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "restful" 2024-09-16T00:16:56.352 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-09-16T00:16:56.352 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T00:16:56.352 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:16:56.352 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-16T00:16:56.353 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T00:16:56.353 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "modified": "2024-09-16T00:16:43.747582+0000", 2024-09-16T00:16:56.353 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T00:16:56.353 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:16:56.353 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-16T00:16:56.353 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout } 2024-09-16T00:16:56.353 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:55.427+0000 7fbc0f850640 1 Processor -- start 2024-09-16T00:16:56.353 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:55.428+0000 7fbc0f850640 1 -- start start 2024-09-16T00:16:56.354 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:55.428+0000 7fbc0f850640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc081035f0 0x7fbc08103a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:56.354 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:55.428+0000 7fbc0f850640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc08103f40 con 0x7fbc081035f0 2024-09-16T00:16:56.354 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:55.428+0000 7fbc0d5ec640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc081035f0 0x7fbc08103a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:56.354 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:55.428+0000 7fbc0d5ec640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc081035f0 0x7fbc08103a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35448/0 (socket says 172.21.15.112:35448) 2024-09-16T00:16:56.354 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:55.428+0000 7fbc0d5ec640 1 -- 172.21.15.112:0/3282628564 learned_addr learned my addr 172.21.15.112:0/3282628564 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:16:56.354 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:55.428+0000 7fbc0d5ec640 1 -- 172.21.15.112:0/3282628564 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc08104080 con 0x7fbc081035f0 2024-09-16T00:16:56.354 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:55.429+0000 7fbc0d5ec640 1 --2- 172.21.15.112:0/3282628564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc081035f0 0x7fbc08103a00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbc04009840 tx=0x7fbc040091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38b49947cea4e447 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:56.355 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:55.429+0000 7fbbfffff640 1 -- 172.21.15.112:0/3282628564 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbc04032440 con 0x7fbc081035f0 2024-09-16T00:16:56.355 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:55.429+0000 7fbbfffff640 1 -- 172.21.15.112:0/3282628564 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1005+0+0 (secure 0 0 0) 0x7fbc04032a00 con 0x7fbc081035f0 2024-09-16T00:16:56.355 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:55.429+0000 7fbbfffff640 1 -- 172.21.15.112:0/3282628564 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbc040319e0 con 0x7fbc081035f0 2024-09-16T00:16:56.355 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:55.430+0000 7fbc0f850640 1 -- 172.21.15.112:0/3282628564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc081035f0 msgr2=0x7fbc08103a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:56.355 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:55.430+0000 7fbc0f850640 1 --2- 172.21.15.112:0/3282628564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc081035f0 0x7fbc08103a00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbc04009840 tx=0x7fbc040091d0 comp rx=0 tx=0).stop 2024-09-16T00:16:56.355 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:55.430+0000 7fbc0f850640 1 -- 172.21.15.112:0/3282628564 shutdown_connections 2024-09-16T00:16:56.356 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:55.430+0000 7fbc0f850640 1 --2- 172.21.15.112:0/3282628564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc081035f0 0x7fbc08103a00 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:56.356 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:55.430+0000 7fbc0f850640 1 -- 172.21.15.112:0/3282628564 >> 172.21.15.112:0/3282628564 conn(0x7fbc080fbd90 msgr2=0x7fbc080fe1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:56.356 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:55.430+0000 7fbc0f850640 1 -- 172.21.15.112:0/3282628564 shutdown_connections 2024-09-16T00:16:56.356 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:55.430+0000 7fbc0f850640 1 -- 172.21.15.112:0/3282628564 wait complete. 2024-09-16T00:16:56.356 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:55.431+0000 7fbc0f850640 1 Processor -- start 2024-09-16T00:16:56.356 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:55.431+0000 7fbc0f850640 1 -- start start 2024-09-16T00:16:56.357 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:55.431+0000 7fbc0f850640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc081035f0 0x7fbc08110e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:56.357 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:55.431+0000 7fbc0f850640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc08111370 con 0x7fbc081035f0 2024-09-16T00:16:56.357 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:55.431+0000 7fbc0d5ec640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc081035f0 0x7fbc08110e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:56.357 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:55.431+0000 7fbc0d5ec640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc081035f0 0x7fbc08110e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35464/0 (socket says 172.21.15.112:35464) 2024-09-16T00:16:56.357 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:55.431+0000 7fbc0d5ec640 1 -- 172.21.15.112:0/1413375172 learned_addr learned my addr 172.21.15.112:0/1413375172 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:16:56.357 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:55.431+0000 7fbc0d5ec640 1 -- 172.21.15.112:0/1413375172 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc04009450 con 0x7fbc081035f0 2024-09-16T00:16:56.357 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:55.432+0000 7fbc0d5ec640 1 --2- 172.21.15.112:0/1413375172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc081035f0 0x7fbc08110e30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbc04000c00 tx=0x7fbc04008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:56.358 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:55.432+0000 7fbbfdffb640 1 -- 172.21.15.112:0/1413375172 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbc0403d450 con 0x7fbc081035f0 2024-09-16T00:16:56.358 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:55.432+0000 7fbbfdffb640 1 -- 172.21.15.112:0/1413375172 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1005+0+0 (secure 0 0 0) 0x7fbc0403da10 con 0x7fbc081035f0 2024-09-16T00:16:56.358 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:55.432+0000 7fbc0f850640 1 -- 172.21.15.112:0/1413375172 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc08111570 con 0x7fbc081035f0 2024-09-16T00:16:56.358 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:55.432+0000 7fbbfdffb640 1 -- 172.21.15.112:0/1413375172 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbc0403b8e0 con 0x7fbc081035f0 2024-09-16T00:16:56.358 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:55.432+0000 7fbc0f850640 1 -- 172.21.15.112:0/1413375172 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc08111fc0 con 0x7fbc081035f0 2024-09-16T00:16:56.358 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:55.432+0000 7fbbfdffb640 1 -- 172.21.15.112:0/1413375172 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7fbc04039070 con 0x7fbc081035f0 2024-09-16T00:16:56.358 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:55.432+0000 7fbbfdffb640 1 -- 172.21.15.112:0/1413375172 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fbc04046760 con 0x7fbc081035f0 2024-09-16T00:16:56.358 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:55.433+0000 7fbc0f850640 1 -- 172.21.15.112:0/1413375172 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc080ffa10 con 0x7fbc081035f0 2024-09-16T00:16:56.358 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:55.435+0000 7fbbfdffb640 1 -- 172.21.15.112:0/1413375172 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7fbc040657a0 con 0x7fbc081035f0 2024-09-16T00:16:56.358 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:55.494+0000 7fbc0f850640 1 -- 172.21.15.112:0/1413375172 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fbc08100a60 con 0x7fbc081035f0 2024-09-16T00:16:56.358 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:55.494+0000 7fbbfdffb640 1 -- 172.21.15.112:0/1413375172 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7fbc04054020 con 0x7fbc081035f0 2024-09-16T00:16:56.358 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:55.495+0000 7fbc0f850640 1 -- 172.21.15.112:0/1413375172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc081035f0 msgr2=0x7fbc08110e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:56.358 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:55.495+0000 7fbc0f850640 1 --2- 172.21.15.112:0/1413375172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc081035f0 0x7fbc08110e30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbc04000c00 tx=0x7fbc04008040 comp rx=0 tx=0).stop 2024-09-16T00:16:56.359 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:55.495+0000 7fbc0f850640 1 -- 172.21.15.112:0/1413375172 shutdown_connections 2024-09-16T00:16:56.359 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:55.495+0000 7fbc0f850640 1 --2- 172.21.15.112:0/1413375172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc081035f0 0x7fbc08110e30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:56.359 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:55.495+0000 7fbc0f850640 1 -- 172.21.15.112:0/1413375172 >> 172.21.15.112:0/1413375172 conn(0x7fbc080fbd90 msgr2=0x7fbc080fc910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:56.359 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:55.495+0000 7fbc0f850640 1 -- 172.21.15.112:0/1413375172 shutdown_connections 2024-09-16T00:16:56.359 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:55.495+0000 7fbc0f850640 1 -- 172.21.15.112:0/1413375172 wait complete. 2024-09-16T00:16:56.359 INFO:teuthology.orchestra.run.smithi112.stdout:mgr not available, waiting (2/15)... 2024-09-16T00:16:56.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:56 smithi112 ceph-mon[27869]: Activating manager daemon smithi112.xrkkbc 2024-09-16T00:16:56.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:56 smithi112 ceph-mon[27869]: mgrmap e2: smithi112.xrkkbc(active, starting, since 0.00273181s) 2024-09-16T00:16:56.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:56 smithi112 ceph-mon[27869]: from='mgr.14100 172.21.15.112:0/1340528867' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T00:16:56.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:56 smithi112 ceph-mon[27869]: from='mgr.14100 172.21.15.112:0/1340528867' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T00:16:56.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:56 smithi112 ceph-mon[27869]: from='mgr.14100 172.21.15.112:0/1340528867' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T00:16:56.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:56 smithi112 ceph-mon[27869]: from='mgr.14100 172.21.15.112:0/1340528867' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mon metadata", "id": "smithi112"}]: dispatch 2024-09-16T00:16:56.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:56 smithi112 ceph-mon[27869]: from='mgr.14100 172.21.15.112:0/1340528867' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mgr metadata", "who": "smithi112.xrkkbc", "id": "smithi112.xrkkbc"}]: dispatch 2024-09-16T00:16:56.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:56 smithi112 ceph-mon[27869]: Manager daemon smithi112.xrkkbc is now available 2024-09-16T00:16:56.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:56 smithi112 ceph-mon[27869]: from='mgr.14100 172.21.15.112:0/1340528867' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi112.xrkkbc/mirror_snapshot_schedule"}]: dispatch 2024-09-16T00:16:56.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:56 smithi112 ceph-mon[27869]: from='mgr.14100 172.21.15.112:0/1340528867' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi112.xrkkbc/trash_purge_schedule"}]: dispatch 2024-09-16T00:16:56.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:56 smithi112 ceph-mon[27869]: from='mgr.14100 172.21.15.112:0/1340528867' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:16:56.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:56 smithi112 ceph-mon[27869]: from='mgr.14100 172.21.15.112:0/1340528867' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:16:56.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:56 smithi112 ceph-mon[27869]: from='mgr.14100 172.21.15.112:0/1340528867' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:16:58.468 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:58 smithi112 ceph-mon[27869]: mgrmap e3: smithi112.xrkkbc(active, since 1.00628s) 2024-09-16T00:16:59.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:59 smithi112 ceph-mon[27869]: mgrmap e4: smithi112.xrkkbc(active, since 2s) 2024-09-16T00:16:59.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:16:59 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/2495442322' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-16T00:16:59.553 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-09-16T00:16:59.553 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout { 2024-09-16T00:16:59.553 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "fsid": "d346dd56-73c0-11ef-bceb-c7b262605968", 2024-09-16T00:16:59.553 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "health": { 2024-09-16T00:16:59.553 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-16T00:16:59.553 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-16T00:16:59.553 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-16T00:16:59.553 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:16:59.553 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-16T00:16:59.553 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-16T00:16:59.553 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 0 2024-09-16T00:16:59.553 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-09-16T00:16:59.554 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-16T00:16:59.554 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "smithi112" 2024-09-16T00:16:59.554 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-09-16T00:16:59.554 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-09-16T00:16:59.554 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-16T00:16:59.554 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T00:16:59.554 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-09-16T00:16:59.554 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-16T00:16:59.554 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:16:59.554 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-16T00:16:59.554 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T00:16:59.554 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-16T00:16:59.554 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-16T00:16:59.555 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-16T00:16:59.563 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-16T00:16:59.563 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-16T00:16:59.563 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-16T00:16:59.563 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:16:59.563 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-16T00:16:59.563 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-16T00:16:59.563 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-16T00:16:59.563 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-16T00:16:59.563 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-16T00:16:59.563 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-16T00:16:59.563 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-16T00:16:59.564 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-16T00:16:59.564 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-16T00:16:59.564 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:16:59.564 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-16T00:16:59.564 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T00:16:59.564 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-16T00:16:59.564 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-16T00:16:59.564 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:16:59.564 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-16T00:16:59.564 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-16T00:16:59.564 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-16T00:16:59.565 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-16T00:16:59.565 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-16T00:16:59.565 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-16T00:16:59.565 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "restful" 2024-09-16T00:16:59.565 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-09-16T00:16:59.565 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T00:16:59.565 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:16:59.565 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-16T00:16:59.565 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T00:16:59.565 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "modified": "2024-09-16T00:16:43.747582+0000", 2024-09-16T00:16:59.565 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T00:16:59.565 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:16:59.565 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-16T00:16:59.565 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout } 2024-09-16T00:16:59.566 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:58.765+0000 7f3949256640 1 Processor -- start 2024-09-16T00:16:59.566 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:58.765+0000 7f3949256640 1 -- start start 2024-09-16T00:16:59.566 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:58.765+0000 7f3949256640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3944102430 0x7f3944102840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:59.566 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:58.765+0000 7f3949256640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3944102d80 con 0x7f3944102430 2024-09-16T00:16:59.566 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:58.765+0000 7f3942d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3944102430 0x7f3944102840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:59.566 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:58.765+0000 7f3942d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3944102430 0x7f3944102840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35540/0 (socket says 172.21.15.112:35540) 2024-09-16T00:16:59.566 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:58.765+0000 7f3942d9d640 1 -- 172.21.15.112:0/2749883759 learned_addr learned my addr 172.21.15.112:0/2749883759 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:16:59.566 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:58.766+0000 7f3942d9d640 1 -- 172.21.15.112:0/2749883759 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3944102ec0 con 0x7f3944102430 2024-09-16T00:16:59.566 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:58.766+0000 7f3942d9d640 1 --2- 172.21.15.112:0/2749883759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3944102430 0x7f3944102840 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f393c009e50 tx=0x7f393c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=623a7dbe53423fec server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:59.566 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:58.766+0000 7f3941d9b640 1 -- 172.21.15.112:0/2749883759 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f393c031560 con 0x7f3944102430 2024-09-16T00:16:59.566 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:58.766+0000 7f3941d9b640 1 -- 172.21.15.112:0/2749883759 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1005+0+0 (secure 0 0 0) 0x7f393c02d050 con 0x7f3944102430 2024-09-16T00:16:59.566 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:58.766+0000 7f3941d9b640 1 -- 172.21.15.112:0/2749883759 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f393c0079b0 con 0x7f3944102430 2024-09-16T00:16:59.566 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:58.767+0000 7f3949256640 1 -- 172.21.15.112:0/2749883759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3944102430 msgr2=0x7f3944102840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:59.566 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:58.767+0000 7f3949256640 1 --2- 172.21.15.112:0/2749883759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3944102430 0x7f3944102840 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f393c009e50 tx=0x7f393c009840 comp rx=0 tx=0).stop 2024-09-16T00:16:59.567 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:58.767+0000 7f3949256640 1 -- 172.21.15.112:0/2749883759 shutdown_connections 2024-09-16T00:16:59.567 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:58.767+0000 7f3949256640 1 --2- 172.21.15.112:0/2749883759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3944102430 0x7f3944102840 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:59.567 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:58.767+0000 7f3949256640 1 -- 172.21.15.112:0/2749883759 >> 172.21.15.112:0/2749883759 conn(0x7f39440fbdd0 msgr2=0x7f39440fe1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:59.567 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:58.767+0000 7f3949256640 1 -- 172.21.15.112:0/2749883759 shutdown_connections 2024-09-16T00:16:59.567 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:58.767+0000 7f3949256640 1 -- 172.21.15.112:0/2749883759 wait complete. 2024-09-16T00:16:59.567 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:58.768+0000 7f3949256640 1 Processor -- start 2024-09-16T00:16:59.567 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:58.768+0000 7f3949256640 1 -- start start 2024-09-16T00:16:59.567 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:58.768+0000 7f3949256640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3944102430 0x7f3944110dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:59.567 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:58.768+0000 7f3949256640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3944111310 con 0x7f3944102430 2024-09-16T00:16:59.567 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:58.768+0000 7f3942d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3944102430 0x7f3944110dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:59.567 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:58.768+0000 7f3942d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3944102430 0x7f3944110dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35544/0 (socket says 172.21.15.112:35544) 2024-09-16T00:16:59.567 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:58.768+0000 7f3942d9d640 1 -- 172.21.15.112:0/2495442322 learned_addr learned my addr 172.21.15.112:0/2495442322 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:16:59.567 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:58.768+0000 7f3942d9d640 1 -- 172.21.15.112:0/2495442322 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f393c009ae0 con 0x7f3944102430 2024-09-16T00:16:59.568 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:58.769+0000 7f3942d9d640 1 --2- 172.21.15.112:0/2495442322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3944102430 0x7f3944110dd0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f393c007030 tx=0x7f393c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:59.568 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:58.769+0000 7f393b7fe640 1 -- 172.21.15.112:0/2495442322 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f393c045430 con 0x7f3944102430 2024-09-16T00:16:59.568 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:58.769+0000 7f393b7fe640 1 -- 172.21.15.112:0/2495442322 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1005+0+0 (secure 0 0 0) 0x7f393c0459f0 con 0x7f3944102430 2024-09-16T00:16:59.568 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:58.769+0000 7f3949256640 1 -- 172.21.15.112:0/2495442322 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3944111510 con 0x7f3944102430 2024-09-16T00:16:59.568 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:58.769+0000 7f393b7fe640 1 -- 172.21.15.112:0/2495442322 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f393c03a7f0 con 0x7f3944102430 2024-09-16T00:16:59.568 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:58.769+0000 7f3949256640 1 -- 172.21.15.112:0/2495442322 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3944111f60 con 0x7f3944102430 2024-09-16T00:16:59.568 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:58.770+0000 7f393b7fe640 1 -- 172.21.15.112:0/2495442322 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 4) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f393c038040 con 0x7f3944102430 2024-09-16T00:16:59.568 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:58.770+0000 7f393b7fe640 1 --2- 172.21.15.112:0/2495442322 >> [v2:172.21.15.112:6800/663349889,v1:172.21.15.112:6801/663349889] conn(0x7f3930039450 0x7f393003b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:59.568 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:58.770+0000 7f3949256640 1 -- 172.21.15.112:0/2495442322 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39440ffa30 con 0x7f3944102430 2024-09-16T00:16:59.568 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:58.770+0000 7f393b7fe640 1 -- 172.21.15.112:0/2495442322 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f393c040030 con 0x7f3944102430 2024-09-16T00:16:59.568 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:58.770+0000 7f394259c640 1 --2- 172.21.15.112:0/2495442322 >> [v2:172.21.15.112:6800/663349889,v1:172.21.15.112:6801/663349889] conn(0x7f3930039450 0x7f393003b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:59.568 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:58.770+0000 7f394259c640 1 --2- 172.21.15.112:0/2495442322 >> [v2:172.21.15.112:6800/663349889,v1:172.21.15.112:6801/663349889] conn(0x7f3930039450 0x7f393003b900 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f392c006fd0 tx=0x7f392c006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:59.568 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:58.775+0000 7f393b7fe640 1 -- 172.21.15.112:0/2495442322 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f393c038500 con 0x7f3944102430 2024-09-16T00:16:59.568 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:58.992+0000 7f3949256640 1 -- 172.21.15.112:0/2495442322 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f394404b720 con 0x7f3944102430 2024-09-16T00:16:59.568 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:58.992+0000 7f393b7fe640 1 -- 172.21.15.112:0/2495442322 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1247 (secure 0 0 0) 0x7f393c036030 con 0x7f3944102430 2024-09-16T00:16:59.569 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:58.994+0000 7f3949256640 1 -- 172.21.15.112:0/2495442322 >> [v2:172.21.15.112:6800/663349889,v1:172.21.15.112:6801/663349889] conn(0x7f3930039450 msgr2=0x7f393003b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:59.569 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:58.994+0000 7f3949256640 1 --2- 172.21.15.112:0/2495442322 >> [v2:172.21.15.112:6800/663349889,v1:172.21.15.112:6801/663349889] conn(0x7f3930039450 0x7f393003b900 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f392c006fd0 tx=0x7f392c006db0 comp rx=0 tx=0).stop 2024-09-16T00:16:59.569 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:58.994+0000 7f3949256640 1 -- 172.21.15.112:0/2495442322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3944102430 msgr2=0x7f3944110dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:59.569 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:58.994+0000 7f3949256640 1 --2- 172.21.15.112:0/2495442322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3944102430 0x7f3944110dd0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f393c007030 tx=0x7f393c009840 comp rx=0 tx=0).stop 2024-09-16T00:16:59.569 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:58.994+0000 7f3949256640 1 -- 172.21.15.112:0/2495442322 shutdown_connections 2024-09-16T00:16:59.569 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:58.994+0000 7f3949256640 1 --2- 172.21.15.112:0/2495442322 >> [v2:172.21.15.112:6800/663349889,v1:172.21.15.112:6801/663349889] conn(0x7f3930039450 0x7f393003b900 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:59.569 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:58.994+0000 7f3949256640 1 --2- 172.21.15.112:0/2495442322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3944102430 0x7f3944110dd0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:59.569 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:58.994+0000 7f3949256640 1 -- 172.21.15.112:0/2495442322 >> 172.21.15.112:0/2495442322 conn(0x7f39440fbdd0 msgr2=0x7f39440fc930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:59.569 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:58.994+0000 7f3949256640 1 -- 172.21.15.112:0/2495442322 shutdown_connections 2024-09-16T00:16:59.569 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:58.994+0000 7f3949256640 1 -- 172.21.15.112:0/2495442322 wait complete. 2024-09-16T00:16:59.569 INFO:teuthology.orchestra.run.smithi112.stdout:mgr is available 2024-09-16T00:17:00.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:00 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/1597821874' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-16T00:17:00.889 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-09-16T00:17:00.889 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout [global] 2024-09-16T00:17:00.889 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout fsid = d346dd56-73c0-11ef-bceb-c7b262605968 2024-09-16T00:17:00.889 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-16T00:17:00.889 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-16T00:17:00.890 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-16T00:17:00.890 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-16T00:17:00.890 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-16T00:17:00.890 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-09-16T00:17:00.890 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-16T00:17:00.890 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-16T00:17:00.890 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-09-16T00:17:00.890 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout [osd] 2024-09-16T00:17:00.890 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-16T00:17:00.890 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-16T00:17:00.890 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-16T00:17:00.890 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-16T00:17:00.890 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-16T00:17:00.890 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:59.957+0000 7f07a8ea7640 1 Processor -- start 2024-09-16T00:17:00.891 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:59.957+0000 7f07a8ea7640 1 -- start start 2024-09-16T00:17:00.891 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:59.957+0000 7f07a8ea7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07a4101f80 0x7f07a4102390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:00.891 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:59.957+0000 7f07a8ea7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a41028d0 con 0x7f07a4101f80 2024-09-16T00:17:00.891 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:59.958+0000 7f07a259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07a4101f80 0x7f07a4102390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:00.891 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:59.958+0000 7f07a259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07a4101f80 0x7f07a4102390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35552/0 (socket says 172.21.15.112:35552) 2024-09-16T00:17:00.891 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:59.958+0000 7f07a259c640 1 -- 172.21.15.112:0/2203851710 learned_addr learned my addr 172.21.15.112:0/2203851710 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:17:00.891 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:59.958+0000 7f07a259c640 1 -- 172.21.15.112:0/2203851710 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07a4102a10 con 0x7f07a4101f80 2024-09-16T00:17:00.891 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:59.959+0000 7f07a259c640 1 --2- 172.21.15.112:0/2203851710 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07a4101f80 0x7f07a4102390 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0794009e50 tx=0x7f0794009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db30565118281962 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:00.891 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:59.959+0000 7f07a159a640 1 -- 172.21.15.112:0/2203851710 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0794031560 con 0x7f07a4101f80 2024-09-16T00:17:00.891 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:59.959+0000 7f07a159a640 1 -- 172.21.15.112:0/2203851710 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1005+0+0 (secure 0 0 0) 0x7f079402d050 con 0x7f07a4101f80 2024-09-16T00:17:00.891 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:59.959+0000 7f07a159a640 1 -- 172.21.15.112:0/2203851710 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f07940079b0 con 0x7f07a4101f80 2024-09-16T00:17:00.891 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:59.959+0000 7f07a8ea7640 1 -- 172.21.15.112:0/2203851710 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07a4101f80 msgr2=0x7f07a4102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:00.891 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:59.959+0000 7f07a8ea7640 1 --2- 172.21.15.112:0/2203851710 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07a4101f80 0x7f07a4102390 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0794009e50 tx=0x7f0794009840 comp rx=0 tx=0).stop 2024-09-16T00:17:00.892 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:59.959+0000 7f07a8ea7640 1 -- 172.21.15.112:0/2203851710 shutdown_connections 2024-09-16T00:17:00.892 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:59.959+0000 7f07a8ea7640 1 --2- 172.21.15.112:0/2203851710 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07a4101f80 0x7f07a4102390 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:00.892 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:59.959+0000 7f07a8ea7640 1 -- 172.21.15.112:0/2203851710 >> 172.21.15.112:0/2203851710 conn(0x7f07a40fd7a0 msgr2=0x7f07a40ffbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:00.892 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:59.959+0000 7f07a8ea7640 1 -- 172.21.15.112:0/2203851710 shutdown_connections 2024-09-16T00:17:00.892 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:59.959+0000 7f07a8ea7640 1 -- 172.21.15.112:0/2203851710 wait complete. 2024-09-16T00:17:00.892 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:59.960+0000 7f07a8ea7640 1 Processor -- start 2024-09-16T00:17:00.892 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:59.960+0000 7f07a8ea7640 1 -- start start 2024-09-16T00:17:00.892 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:59.960+0000 7f07a8ea7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07a4101f80 0x7f07a4078c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:00.892 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:59.960+0000 7f07a8ea7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a4079140 con 0x7f07a4101f80 2024-09-16T00:17:00.892 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:59.960+0000 7f07a259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07a4101f80 0x7f07a4078c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:00.892 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:59.960+0000 7f07a259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07a4101f80 0x7f07a4078c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35564/0 (socket says 172.21.15.112:35564) 2024-09-16T00:17:00.892 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:59.960+0000 7f07a259c640 1 -- 172.21.15.112:0/1597821874 learned_addr learned my addr 172.21.15.112:0/1597821874 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:17:00.892 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:59.960+0000 7f07a259c640 1 -- 172.21.15.112:0/1597821874 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0794009ae0 con 0x7f07a4101f80 2024-09-16T00:17:00.892 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:59.961+0000 7f07a259c640 1 --2- 172.21.15.112:0/1597821874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07a4101f80 0x7f07a4078c00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0794007690 tx=0x7f0794009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:00.893 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:59.961+0000 7f0792ffd640 1 -- 172.21.15.112:0/1597821874 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0794031b70 con 0x7f07a4101f80 2024-09-16T00:17:00.893 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:59.961+0000 7f07a8ea7640 1 -- 172.21.15.112:0/1597821874 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07a40771d0 con 0x7f07a4101f80 2024-09-16T00:17:00.893 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:59.961+0000 7f0792ffd640 1 -- 172.21.15.112:0/1597821874 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1005+0+0 (secure 0 0 0) 0x7f0794031cd0 con 0x7f07a4101f80 2024-09-16T00:17:00.893 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:59.961+0000 7f0792ffd640 1 -- 172.21.15.112:0/1597821874 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f07940443e0 con 0x7f07a4101f80 2024-09-16T00:17:00.893 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:59.961+0000 7f07a8ea7640 1 -- 172.21.15.112:0/1597821874 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07a4077640 con 0x7f07a4101f80 2024-09-16T00:17:00.893 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:59.962+0000 7f0792ffd640 1 -- 172.21.15.112:0/1597821874 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 4) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f0794038040 con 0x7f07a4101f80 2024-09-16T00:17:00.893 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:59.962+0000 7f07a8ea7640 1 -- 172.21.15.112:0/1597821874 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07a404b720 con 0x7f07a4101f80 2024-09-16T00:17:00.893 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:59.962+0000 7f0792ffd640 1 --2- 172.21.15.112:0/1597821874 >> [v2:172.21.15.112:6800/663349889,v1:172.21.15.112:6801/663349889] conn(0x7f078c039450 0x7f078c03b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:00.893 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:59.962+0000 7f0792ffd640 1 -- 172.21.15.112:0/1597821874 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f0794040030 con 0x7f07a4101f80 2024-09-16T00:17:00.893 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:59.962+0000 7f07a1d9b640 1 --2- 172.21.15.112:0/1597821874 >> [v2:172.21.15.112:6800/663349889,v1:172.21.15.112:6801/663349889] conn(0x7f078c039450 0x7f078c03b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:00.893 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:59.963+0000 7f07a1d9b640 1 --2- 172.21.15.112:0/1597821874 >> [v2:172.21.15.112:6800/663349889,v1:172.21.15.112:6801/663349889] conn(0x7f078c039450 0x7f078c03b900 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0798006fd0 tx=0x7f0798006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:00.893 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:16:59.966+0000 7f0792ffd640 1 -- 172.21.15.112:0/1597821874 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f079404c4d0 con 0x7f07a4101f80 2024-09-16T00:17:00.893 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:00.132+0000 7f07a8ea7640 1 -- 172.21.15.112:0/1597821874 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f07a4103ce0 con 0x7f07a4101f80 2024-09-16T00:17:00.893 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:00.133+0000 7f0792ffd640 1 -- 172.21.15.112:0/1597821874 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+409 (secure 0 0 0) 0x7f0794036030 con 0x7f07a4101f80 2024-09-16T00:17:00.894 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:00.135+0000 7f07a8ea7640 1 -- 172.21.15.112:0/1597821874 >> [v2:172.21.15.112:6800/663349889,v1:172.21.15.112:6801/663349889] conn(0x7f078c039450 msgr2=0x7f078c03b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:00.894 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:00.135+0000 7f07a8ea7640 1 --2- 172.21.15.112:0/1597821874 >> [v2:172.21.15.112:6800/663349889,v1:172.21.15.112:6801/663349889] conn(0x7f078c039450 0x7f078c03b900 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0798006fd0 tx=0x7f0798006db0 comp rx=0 tx=0).stop 2024-09-16T00:17:00.894 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:00.135+0000 7f07a8ea7640 1 -- 172.21.15.112:0/1597821874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07a4101f80 msgr2=0x7f07a4078c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:00.894 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:00.135+0000 7f07a8ea7640 1 --2- 172.21.15.112:0/1597821874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07a4101f80 0x7f07a4078c00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0794007690 tx=0x7f0794009840 comp rx=0 tx=0).stop 2024-09-16T00:17:00.894 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:00.135+0000 7f07a8ea7640 1 -- 172.21.15.112:0/1597821874 shutdown_connections 2024-09-16T00:17:00.894 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:00.135+0000 7f07a8ea7640 1 --2- 172.21.15.112:0/1597821874 >> [v2:172.21.15.112:6800/663349889,v1:172.21.15.112:6801/663349889] conn(0x7f078c039450 0x7f078c03b900 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:00.894 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:00.135+0000 7f07a8ea7640 1 --2- 172.21.15.112:0/1597821874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07a4101f80 0x7f07a4078c00 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:00.894 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:00.135+0000 7f07a8ea7640 1 -- 172.21.15.112:0/1597821874 >> 172.21.15.112:0/1597821874 conn(0x7f07a40fd7a0 msgr2=0x7f07a40fe320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:00.894 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:00.135+0000 7f07a8ea7640 1 -- 172.21.15.112:0/1597821874 shutdown_connections 2024-09-16T00:17:00.894 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:00.135+0000 7f07a8ea7640 1 -- 172.21.15.112:0/1597821874 wait complete. 2024-09-16T00:17:00.894 INFO:teuthology.orchestra.run.smithi112.stdout:Enabling cephadm module... 2024-09-16T00:17:01.823 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:01 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/2481744595' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-16T00:17:02.219 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:01.319+0000 7f6352664640 1 Processor -- start 2024-09-16T00:17:02.219 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:01.319+0000 7f6352664640 1 -- start start 2024-09-16T00:17:02.219 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:01.319+0000 7f6352664640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f634c078b20 0x7f634c077230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:02.220 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:01.319+0000 7f6352664640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f634c078fc0 con 0x7f634c078b20 2024-09-16T00:17:02.220 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:01.319+0000 7f634bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f634c078b20 0x7f634c077230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:02.220 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:01.319+0000 7f634bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f634c078b20 0x7f634c077230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35574/0 (socket says 172.21.15.112:35574) 2024-09-16T00:17:02.220 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:01.319+0000 7f634bfff640 1 -- 172.21.15.112:0/2325065356 learned_addr learned my addr 172.21.15.112:0/2325065356 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:17:02.220 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:01.319+0000 7f634bfff640 1 -- 172.21.15.112:0/2325065356 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f634c079100 con 0x7f634c078b20 2024-09-16T00:17:02.220 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:01.320+0000 7f634bfff640 1 --2- 172.21.15.112:0/2325065356 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f634c078b20 0x7f634c077230 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f633c009840 tx=0x7f633c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fb7da23bfc7e9c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:02.220 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:01.320+0000 7f634affd640 1 -- 172.21.15.112:0/2325065356 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f633c032440 con 0x7f634c078b20 2024-09-16T00:17:02.220 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:01.320+0000 7f634affd640 1 -- 172.21.15.112:0/2325065356 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1005+0+0 (secure 0 0 0) 0x7f633c032a00 con 0x7f634c078b20 2024-09-16T00:17:02.220 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:01.320+0000 7f634affd640 1 -- 172.21.15.112:0/2325065356 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f633c0319e0 con 0x7f634c078b20 2024-09-16T00:17:02.220 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:01.321+0000 7f6352664640 1 -- 172.21.15.112:0/2325065356 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f634c078b20 msgr2=0x7f634c077230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:02.220 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:01.321+0000 7f6352664640 1 --2- 172.21.15.112:0/2325065356 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f634c078b20 0x7f634c077230 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f633c009840 tx=0x7f633c0091d0 comp rx=0 tx=0).stop 2024-09-16T00:17:02.220 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:01.321+0000 7f6352664640 1 -- 172.21.15.112:0/2325065356 shutdown_connections 2024-09-16T00:17:02.220 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:01.321+0000 7f6352664640 1 --2- 172.21.15.112:0/2325065356 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f634c078b20 0x7f634c077230 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:02.221 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:01.321+0000 7f6352664640 1 -- 172.21.15.112:0/2325065356 >> 172.21.15.112:0/2325065356 conn(0x7f634c0fd850 msgr2=0x7f634c0ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:02.221 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:01.321+0000 7f6352664640 1 -- 172.21.15.112:0/2325065356 shutdown_connections 2024-09-16T00:17:02.221 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:01.321+0000 7f6352664640 1 -- 172.21.15.112:0/2325065356 wait complete. 2024-09-16T00:17:02.221 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:01.321+0000 7f6352664640 1 Processor -- start 2024-09-16T00:17:02.221 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:01.322+0000 7f6352664640 1 -- start start 2024-09-16T00:17:02.221 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:01.322+0000 7f6352664640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f634c078b20 0x7f634c112f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:02.221 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:01.322+0000 7f6352664640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f634c1134a0 con 0x7f634c078b20 2024-09-16T00:17:02.221 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:01.322+0000 7f634bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f634c078b20 0x7f634c112f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:02.221 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:01.322+0000 7f634bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f634c078b20 0x7f634c112f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35588/0 (socket says 172.21.15.112:35588) 2024-09-16T00:17:02.221 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:01.322+0000 7f634bfff640 1 -- 172.21.15.112:0/2481744595 learned_addr learned my addr 172.21.15.112:0/2481744595 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:17:02.221 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:01.322+0000 7f634bfff640 1 -- 172.21.15.112:0/2481744595 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f633c009450 con 0x7f634c078b20 2024-09-16T00:17:02.221 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:01.322+0000 7f634bfff640 1 --2- 172.21.15.112:0/2481744595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f634c078b20 0x7f634c112f60 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f633c006f00 tx=0x7f633c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:02.221 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:01.322+0000 7f6348ff9640 1 -- 172.21.15.112:0/2481744595 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f633c032880 con 0x7f634c078b20 2024-09-16T00:17:02.221 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:01.322+0000 7f6352664640 1 -- 172.21.15.112:0/2481744595 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f634c1136a0 con 0x7f634c078b20 2024-09-16T00:17:02.222 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:01.322+0000 7f6348ff9640 1 -- 172.21.15.112:0/2481744595 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1005+0+0 (secure 0 0 0) 0x7f633c03b8d0 con 0x7f634c078b20 2024-09-16T00:17:02.222 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:01.322+0000 7f6348ff9640 1 -- 172.21.15.112:0/2481744595 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f633c03a8f0 con 0x7f634c078b20 2024-09-16T00:17:02.222 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:01.322+0000 7f6352664640 1 -- 172.21.15.112:0/2481744595 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f634c113b10 con 0x7f634c078b20 2024-09-16T00:17:02.222 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:01.323+0000 7f6348ff9640 1 -- 172.21.15.112:0/2481744595 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 4) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f633c03abb0 con 0x7f634c078b20 2024-09-16T00:17:02.222 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:01.323+0000 7f6348ff9640 1 --2- 172.21.15.112:0/2481744595 >> [v2:172.21.15.112:6800/663349889,v1:172.21.15.112:6801/663349889] conn(0x7f6334039450 0x7f633403b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:02.222 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:01.323+0000 7f6348ff9640 1 -- 172.21.15.112:0/2481744595 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f633c06cd50 con 0x7f634c078b20 2024-09-16T00:17:02.222 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:01.324+0000 7f634b7fe640 1 --2- 172.21.15.112:0/2481744595 >> [v2:172.21.15.112:6800/663349889,v1:172.21.15.112:6801/663349889] conn(0x7f6334039450 0x7f633403b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:02.222 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:01.324+0000 7f6352664640 1 -- 172.21.15.112:0/2481744595 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f634c101510 con 0x7f634c078b20 2024-09-16T00:17:02.222 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:01.324+0000 7f634b7fe640 1 --2- 172.21.15.112:0/2481744595 >> [v2:172.21.15.112:6800/663349889,v1:172.21.15.112:6801/663349889] conn(0x7f6334039450 0x7f633403b900 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6340006fd0 tx=0x7f6340006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:02.222 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:01.328+0000 7f6348ff9640 1 -- 172.21.15.112:0/2481744595 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f633c037070 con 0x7f634c078b20 2024-09-16T00:17:02.222 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:01.524+0000 7f6352664640 1 -- 172.21.15.112:0/2481744595 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f634c04b720 con 0x7f634c078b20 2024-09-16T00:17:02.222 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:01.577+0000 7f6348ff9640 1 -- 172.21.15.112:0/2481744595 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mgrmap(e 5) v1 ==== 45507+0+0 (secure 0 0 0) 0x7f633c040d00 con 0x7f634c078b20 2024-09-16T00:17:02.222 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:01.578+0000 7f6348ff9640 1 -- 172.21.15.112:0/2481744595 <== mon.0 v2:172.21.15.112:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f633c030af0 con 0x7f634c078b20 2024-09-16T00:17:02.222 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:01.581+0000 7f6352664640 1 -- 172.21.15.112:0/2481744595 >> [v2:172.21.15.112:6800/663349889,v1:172.21.15.112:6801/663349889] conn(0x7f6334039450 msgr2=0x7f633403b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:02.223 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:01.581+0000 7f6352664640 1 --2- 172.21.15.112:0/2481744595 >> [v2:172.21.15.112:6800/663349889,v1:172.21.15.112:6801/663349889] conn(0x7f6334039450 0x7f633403b900 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6340006fd0 tx=0x7f6340006db0 comp rx=0 tx=0).stop 2024-09-16T00:17:02.223 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:01.581+0000 7f6352664640 1 -- 172.21.15.112:0/2481744595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f634c078b20 msgr2=0x7f634c112f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:02.223 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:01.581+0000 7f6352664640 1 --2- 172.21.15.112:0/2481744595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f634c078b20 0x7f634c112f60 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f633c006f00 tx=0x7f633c008040 comp rx=0 tx=0).stop 2024-09-16T00:17:02.223 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:01.581+0000 7f6352664640 1 -- 172.21.15.112:0/2481744595 shutdown_connections 2024-09-16T00:17:02.223 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:01.581+0000 7f6352664640 1 --2- 172.21.15.112:0/2481744595 >> [v2:172.21.15.112:6800/663349889,v1:172.21.15.112:6801/663349889] conn(0x7f6334039450 0x7f633403b900 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:02.223 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:01.581+0000 7f6352664640 1 --2- 172.21.15.112:0/2481744595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f634c078b20 0x7f634c112f60 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:02.223 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:01.581+0000 7f6352664640 1 -- 172.21.15.112:0/2481744595 >> 172.21.15.112:0/2481744595 conn(0x7f634c0fd850 msgr2=0x7f634c0fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:02.223 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:01.581+0000 7f6352664640 1 -- 172.21.15.112:0/2481744595 shutdown_connections 2024-09-16T00:17:02.223 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:01.581+0000 7f6352664640 1 -- 172.21.15.112:0/2481744595 wait complete. 2024-09-16T00:17:02.639 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:02 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/2481744595' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-16T00:17:02.639 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:02 smithi112 ceph-mon[27869]: mgrmap e5: smithi112.xrkkbc(active, since 5s) 2024-09-16T00:17:03.519 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout { 2024-09-16T00:17:03.519 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-16T00:17:03.519 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-16T00:17:03.519 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "active_name": "smithi112.xrkkbc", 2024-09-16T00:17:03.519 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-16T00:17:03.519 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout } 2024-09-16T00:17:03.519 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:02.638+0000 7f64aa21a640 1 Processor -- start 2024-09-16T00:17:03.520 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:02.638+0000 7f64aa21a640 1 -- start start 2024-09-16T00:17:03.520 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:02.638+0000 7f64aa21a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64a4076b50 0x7f64a4104c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:03.520 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:02.638+0000 7f64aa21a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a4076f60 con 0x7f64a4076b50 2024-09-16T00:17:03.520 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:02.638+0000 7f64a37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64a4076b50 0x7f64a4104c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:03.520 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:02.639+0000 7f64a37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64a4076b50 0x7f64a4104c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51106/0 (socket says 172.21.15.112:51106) 2024-09-16T00:17:03.520 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:02.639+0000 7f64a37fe640 1 -- 172.21.15.112:0/1264446066 learned_addr learned my addr 172.21.15.112:0/1264446066 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:17:03.520 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:02.639+0000 7f64a37fe640 1 -- 172.21.15.112:0/1264446066 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64a4105190 con 0x7f64a4076b50 2024-09-16T00:17:03.520 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:02.640+0000 7f64a37fe640 1 --2- 172.21.15.112:0/1264446066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64a4076b50 0x7f64a4104c50 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6494009840 tx=0x7f64940091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20cb5587878bb49b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:03.520 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:02.640+0000 7f64a2ffd640 1 -- 172.21.15.112:0/1264446066 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6494032440 con 0x7f64a4076b50 2024-09-16T00:17:03.520 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:02.640+0000 7f64a2ffd640 1 -- 172.21.15.112:0/1264446066 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1005+0+0 (secure 0 0 0) 0x7f6494032a00 con 0x7f64a4076b50 2024-09-16T00:17:03.520 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:02.640+0000 7f64a2ffd640 1 -- 172.21.15.112:0/1264446066 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f64940319e0 con 0x7f64a4076b50 2024-09-16T00:17:03.520 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:02.640+0000 7f64aa21a640 1 -- 172.21.15.112:0/1264446066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64a4076b50 msgr2=0x7f64a4104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:03.520 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:02.640+0000 7f64aa21a640 1 --2- 172.21.15.112:0/1264446066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64a4076b50 0x7f64a4104c50 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6494009840 tx=0x7f64940091d0 comp rx=0 tx=0).stop 2024-09-16T00:17:03.521 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:02.640+0000 7f64aa21a640 1 -- 172.21.15.112:0/1264446066 shutdown_connections 2024-09-16T00:17:03.521 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:02.640+0000 7f64aa21a640 1 --2- 172.21.15.112:0/1264446066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64a4076b50 0x7f64a4104c50 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:03.521 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:02.640+0000 7f64aa21a640 1 -- 172.21.15.112:0/1264446066 >> 172.21.15.112:0/1264446066 conn(0x7f64a40fc6e0 msgr2=0x7f64a40feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:03.521 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:02.641+0000 7f64aa21a640 1 -- 172.21.15.112:0/1264446066 shutdown_connections 2024-09-16T00:17:03.521 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:02.641+0000 7f64aa21a640 1 -- 172.21.15.112:0/1264446066 wait complete. 2024-09-16T00:17:03.521 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:02.641+0000 7f64aa21a640 1 Processor -- start 2024-09-16T00:17:03.521 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:02.641+0000 7f64aa21a640 1 -- start start 2024-09-16T00:17:03.521 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:02.641+0000 7f64aa21a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64a4076b50 0x7f64a4102580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:03.521 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:02.641+0000 7f64aa21a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a4102ac0 con 0x7f64a4076b50 2024-09-16T00:17:03.521 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:02.641+0000 7f64a37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64a4076b50 0x7f64a4102580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:03.522 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:02.641+0000 7f64a37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64a4076b50 0x7f64a4102580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51108/0 (socket says 172.21.15.112:51108) 2024-09-16T00:17:03.522 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:02.641+0000 7f64a37fe640 1 -- 172.21.15.112:0/1785324028 learned_addr learned my addr 172.21.15.112:0/1785324028 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:17:03.522 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:02.642+0000 7f64a37fe640 1 -- 172.21.15.112:0/1785324028 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6494009450 con 0x7f64a4076b50 2024-09-16T00:17:03.522 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:02.642+0000 7f64a37fe640 1 --2- 172.21.15.112:0/1785324028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64a4076b50 0x7f64a4102580 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6494006f00 tx=0x7f6494008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:03.522 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:02.642+0000 7f64a0ff9640 1 -- 172.21.15.112:0/1785324028 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6494031e30 con 0x7f64a4076b50 2024-09-16T00:17:03.522 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:02.642+0000 7f64a0ff9640 1 -- 172.21.15.112:0/1785324028 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1005+0+0 (secure 0 0 0) 0x7f649403d8b0 con 0x7f64a4076b50 2024-09-16T00:17:03.522 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:02.642+0000 7f64aa21a640 1 -- 172.21.15.112:0/1785324028 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64a4100b50 con 0x7f64a4076b50 2024-09-16T00:17:03.522 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:02.642+0000 7f64a0ff9640 1 -- 172.21.15.112:0/1785324028 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f649403b8f0 con 0x7f64a4076b50 2024-09-16T00:17:03.522 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:02.642+0000 7f64aa21a640 1 -- 172.21.15.112:0/1785324028 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64a4100fc0 con 0x7f64a4076b50 2024-09-16T00:17:03.522 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:02.643+0000 7f64aa21a640 1 -- 172.21.15.112:0/1785324028 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64a404b720 con 0x7f64a4076b50 2024-09-16T00:17:03.522 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:02.643+0000 7f64a0ff9640 1 -- 172.21.15.112:0/1785324028 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 45507+0+0 (secure 0 0 0) 0x7f6494039070 con 0x7f64a4076b50 2024-09-16T00:17:03.522 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:02.644+0000 7f64a0ff9640 1 --2- 172.21.15.112:0/1785324028 >> [v2:172.21.15.112:6800/663349889,v1:172.21.15.112:6801/663349889] conn(0x7f64900394e0 0x7f649003b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:03.522 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:02.644+0000 7f64a0ff9640 1 -- 172.21.15.112:0/1785324028 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f649406cf40 con 0x7f64a4076b50 2024-09-16T00:17:03.522 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:02.644+0000 7f649bfff640 1 -- 172.21.15.112:0/1785324028 >> [v2:172.21.15.112:6800/663349889,v1:172.21.15.112:6801/663349889] conn(0x7f64900394e0 msgr2=0x7f649003b990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/663349889 2024-09-16T00:17:03.523 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:02.644+0000 7f649bfff640 1 --2- 172.21.15.112:0/1785324028 >> [v2:172.21.15.112:6800/663349889,v1:172.21.15.112:6801/663349889] conn(0x7f64900394e0 0x7f649003b990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T00:17:03.523 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:02.648+0000 7f64a0ff9640 1 -- 172.21.15.112:0/1785324028 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f649403da20 con 0x7f64a4076b50 2024-09-16T00:17:03.523 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:02.845+0000 7f649bfff640 1 -- 172.21.15.112:0/1785324028 >> [v2:172.21.15.112:6800/663349889,v1:172.21.15.112:6801/663349889] conn(0x7f64900394e0 msgr2=0x7f649003b990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/663349889 2024-09-16T00:17:03.523 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:02.845+0000 7f649bfff640 1 --2- 172.21.15.112:0/1785324028 >> [v2:172.21.15.112:6800/663349889,v1:172.21.15.112:6801/663349889] conn(0x7f64900394e0 0x7f649003b990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T00:17:03.523 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:02.850+0000 7f64aa21a640 1 -- 172.21.15.112:0/1785324028 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f64a4101700 con 0x7f64a4076b50 2024-09-16T00:17:03.523 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:02.850+0000 7f64a0ff9640 1 -- 172.21.15.112:0/1785324028 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+103 (secure 0 0 0) 0x7f6494041070 con 0x7f64a4076b50 2024-09-16T00:17:03.523 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:02.853+0000 7f64aa21a640 1 -- 172.21.15.112:0/1785324028 >> [v2:172.21.15.112:6800/663349889,v1:172.21.15.112:6801/663349889] conn(0x7f64900394e0 msgr2=0x7f649003b990 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T00:17:03.523 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:02.853+0000 7f64aa21a640 1 --2- 172.21.15.112:0/1785324028 >> [v2:172.21.15.112:6800/663349889,v1:172.21.15.112:6801/663349889] conn(0x7f64900394e0 0x7f649003b990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:03.523 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:02.853+0000 7f64aa21a640 1 -- 172.21.15.112:0/1785324028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64a4076b50 msgr2=0x7f64a4102580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:03.523 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:02.853+0000 7f64aa21a640 1 --2- 172.21.15.112:0/1785324028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64a4076b50 0x7f64a4102580 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6494006f00 tx=0x7f6494008040 comp rx=0 tx=0).stop 2024-09-16T00:17:03.523 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:02.853+0000 7f64aa21a640 1 -- 172.21.15.112:0/1785324028 shutdown_connections 2024-09-16T00:17:03.523 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:02.853+0000 7f64aa21a640 1 --2- 172.21.15.112:0/1785324028 >> [v2:172.21.15.112:6800/663349889,v1:172.21.15.112:6801/663349889] conn(0x7f64900394e0 0x7f649003b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:03.523 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:02.853+0000 7f64aa21a640 1 --2- 172.21.15.112:0/1785324028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64a4076b50 0x7f64a4102580 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:03.524 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:02.853+0000 7f64aa21a640 1 -- 172.21.15.112:0/1785324028 >> 172.21.15.112:0/1785324028 conn(0x7f64a40fc6e0 msgr2=0x7f64a40fd280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:03.524 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:02.853+0000 7f64aa21a640 1 -- 172.21.15.112:0/1785324028 shutdown_connections 2024-09-16T00:17:03.524 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:02.853+0000 7f64aa21a640 1 -- 172.21.15.112:0/1785324028 wait complete. 2024-09-16T00:17:03.524 INFO:teuthology.orchestra.run.smithi112.stdout:Waiting for the mgr to restart... 2024-09-16T00:17:03.524 INFO:teuthology.orchestra.run.smithi112.stdout:Waiting for mgr epoch 5... 2024-09-16T00:17:03.790 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:03 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/1785324028' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-16T00:17:06.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:06 smithi112 ceph-mon[27869]: Active manager daemon smithi112.xrkkbc restarted 2024-09-16T00:17:06.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:06 smithi112 ceph-mon[27869]: Activating manager daemon smithi112.xrkkbc 2024-09-16T00:17:06.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:06 smithi112 ceph-mon[27869]: osdmap e2: 0 total, 0 up, 0 in 2024-09-16T00:17:06.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:06 smithi112 ceph-mon[27869]: mgrmap e6: smithi112.xrkkbc(active, starting, since 0.00375289s) 2024-09-16T00:17:06.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:06 smithi112 ceph-mon[27869]: from='mgr.14118 172.21.15.112:0/2162065181' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mon metadata", "id": "smithi112"}]: dispatch 2024-09-16T00:17:06.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:06 smithi112 ceph-mon[27869]: from='mgr.14118 172.21.15.112:0/2162065181' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mgr metadata", "who": "smithi112.xrkkbc", "id": "smithi112.xrkkbc"}]: dispatch 2024-09-16T00:17:06.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:06 smithi112 ceph-mon[27869]: from='mgr.14118 172.21.15.112:0/2162065181' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T00:17:06.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:06 smithi112 ceph-mon[27869]: from='mgr.14118 172.21.15.112:0/2162065181' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T00:17:06.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:06 smithi112 ceph-mon[27869]: from='mgr.14118 172.21.15.112:0/2162065181' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T00:17:06.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:06 smithi112 ceph-mon[27869]: Manager daemon smithi112.xrkkbc is now available 2024-09-16T00:17:06.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:06 smithi112 ceph-mon[27869]: from='mgr.14118 172.21.15.112:0/2162065181' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:17:06.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:06 smithi112 ceph-mon[27869]: from='mgr.14118 172.21.15.112:0/2162065181' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:17:06.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:06 smithi112 ceph-mon[27869]: from='mgr.14118 172.21.15.112:0/2162065181' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:17:06.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:06 smithi112 ceph-mon[27869]: from='mgr.14118 172.21.15.112:0/2162065181' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:17:06.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:06 smithi112 ceph-mon[27869]: from='mgr.14118 172.21.15.112:0/2162065181' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi112.xrkkbc/mirror_snapshot_schedule"}]: dispatch 2024-09-16T00:17:06.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:06 smithi112 ceph-mon[27869]: from='mgr.14118 172.21.15.112:0/2162065181' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi112.xrkkbc/trash_purge_schedule"}]: dispatch 2024-09-16T00:17:07.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:07 smithi112 ceph-mon[27869]: Found migration_current of "None". Setting to last migration. 2024-09-16T00:17:07.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:07 smithi112 ceph-mon[27869]: mgrmap e7: smithi112.xrkkbc(active, since 1.00517s) 2024-09-16T00:17:07.643 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout { 2024-09-16T00:17:07.643 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-16T00:17:07.643 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-16T00:17:07.643 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout } 2024-09-16T00:17:07.643 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:03.996+0000 7f4dc252b640 1 Processor -- start 2024-09-16T00:17:07.643 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:03.996+0000 7f4dc252b640 1 -- start start 2024-09-16T00:17:07.643 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:03.996+0000 7f4dc252b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4dbc078b20 0x7f4dbc077230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:07.643 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:03.996+0000 7f4dc252b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dbc078fc0 con 0x7f4dbc078b20 2024-09-16T00:17:07.643 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:03.997+0000 7f4dbbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4dbc078b20 0x7f4dbc077230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:07.643 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:03.997+0000 7f4dbbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4dbc078b20 0x7f4dbc077230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51118/0 (socket says 172.21.15.112:51118) 2024-09-16T00:17:07.643 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:03.997+0000 7f4dbbfff640 1 -- 172.21.15.112:0/3555757010 learned_addr learned my addr 172.21.15.112:0/3555757010 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:17:07.643 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:03.997+0000 7f4dbbfff640 1 -- 172.21.15.112:0/3555757010 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dbc079100 con 0x7f4dbc078b20 2024-09-16T00:17:07.644 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:03.997+0000 7f4dbbfff640 1 --2- 172.21.15.112:0/3555757010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4dbc078b20 0x7f4dbc077230 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4dac009840 tx=0x7f4dac0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c5789cdfcd4ab75 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:07.644 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:03.998+0000 7f4dbaffd640 1 -- 172.21.15.112:0/3555757010 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4dac032440 con 0x7f4dbc078b20 2024-09-16T00:17:07.644 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:03.998+0000 7f4dbaffd640 1 -- 172.21.15.112:0/3555757010 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1005+0+0 (secure 0 0 0) 0x7f4dac032a00 con 0x7f4dbc078b20 2024-09-16T00:17:07.644 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:03.998+0000 7f4dbaffd640 1 -- 172.21.15.112:0/3555757010 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4dac0319e0 con 0x7f4dbc078b20 2024-09-16T00:17:07.644 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:03.998+0000 7f4dc252b640 1 -- 172.21.15.112:0/3555757010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4dbc078b20 msgr2=0x7f4dbc077230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:07.644 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:03.998+0000 7f4dc252b640 1 --2- 172.21.15.112:0/3555757010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4dbc078b20 0x7f4dbc077230 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4dac009840 tx=0x7f4dac0091d0 comp rx=0 tx=0).stop 2024-09-16T00:17:07.644 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:03.998+0000 7f4dc252b640 1 -- 172.21.15.112:0/3555757010 shutdown_connections 2024-09-16T00:17:07.644 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:03.998+0000 7f4dc252b640 1 --2- 172.21.15.112:0/3555757010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4dbc078b20 0x7f4dbc077230 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:07.644 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:03.998+0000 7f4dc252b640 1 -- 172.21.15.112:0/3555757010 >> 172.21.15.112:0/3555757010 conn(0x7f4dbc0fd880 msgr2=0x7f4dbc0ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:07.644 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:03.998+0000 7f4dc252b640 1 -- 172.21.15.112:0/3555757010 shutdown_connections 2024-09-16T00:17:07.644 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:03.998+0000 7f4dc252b640 1 -- 172.21.15.112:0/3555757010 wait complete. 2024-09-16T00:17:07.644 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:03.999+0000 7f4dc252b640 1 Processor -- start 2024-09-16T00:17:07.644 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:03.999+0000 7f4dc252b640 1 -- start start 2024-09-16T00:17:07.645 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:03.999+0000 7f4dc252b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4dbc078b20 0x7f4dbc112fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:07.645 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:03.999+0000 7f4dc252b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dbc1134f0 con 0x7f4dbc078b20 2024-09-16T00:17:07.645 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:03.999+0000 7f4dbbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4dbc078b20 0x7f4dbc112fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:07.645 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:03.999+0000 7f4dbbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4dbc078b20 0x7f4dbc112fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51126/0 (socket says 172.21.15.112:51126) 2024-09-16T00:17:07.645 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:03.999+0000 7f4dbbfff640 1 -- 172.21.15.112:0/1978867579 learned_addr learned my addr 172.21.15.112:0/1978867579 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:17:07.645 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:04.000+0000 7f4dbbfff640 1 -- 172.21.15.112:0/1978867579 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dac009450 con 0x7f4dbc078b20 2024-09-16T00:17:07.645 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:04.000+0000 7f4dbbfff640 1 --2- 172.21.15.112:0/1978867579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4dbc078b20 0x7f4dbc112fb0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4dac006f00 tx=0x7f4dac008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:07.645 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:04.000+0000 7f4db8ff9640 1 -- 172.21.15.112:0/1978867579 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4dac03b470 con 0x7f4dbc078b20 2024-09-16T00:17:07.645 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:04.000+0000 7f4db8ff9640 1 -- 172.21.15.112:0/1978867579 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1005+0+0 (secure 0 0 0) 0x7f4dac03ba30 con 0x7f4dbc078b20 2024-09-16T00:17:07.645 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:04.000+0000 7f4dc252b640 1 -- 172.21.15.112:0/1978867579 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dbc1136f0 con 0x7f4dbc078b20 2024-09-16T00:17:07.645 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:04.000+0000 7f4db8ff9640 1 -- 172.21.15.112:0/1978867579 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4dac03a820 con 0x7f4dbc078b20 2024-09-16T00:17:07.645 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:04.000+0000 7f4dc252b640 1 -- 172.21.15.112:0/1978867579 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dbc113b60 con 0x7f4dbc078b20 2024-09-16T00:17:07.645 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:04.001+0000 7f4db8ff9640 1 -- 172.21.15.112:0/1978867579 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 45507+0+0 (secure 0 0 0) 0x7f4dac03aaa0 con 0x7f4dbc078b20 2024-09-16T00:17:07.645 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:04.001+0000 7f4db8ff9640 1 --2- 172.21.15.112:0/1978867579 >> [v2:172.21.15.112:6800/663349889,v1:172.21.15.112:6801/663349889] conn(0x7f4da8039490 0x7f4da803b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:07.646 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:04.001+0000 7f4db8ff9640 1 -- 172.21.15.112:0/1978867579 --> [v2:172.21.15.112:6800/663349889,v1:172.21.15.112:6801/663349889] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4da803c050 con 0x7f4da8039490 2024-09-16T00:17:07.646 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:04.001+0000 7f4dbb7fe640 1 -- 172.21.15.112:0/1978867579 >> [v2:172.21.15.112:6800/663349889,v1:172.21.15.112:6801/663349889] conn(0x7f4da8039490 msgr2=0x7f4da803b940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/663349889 2024-09-16T00:17:07.646 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:04.001+0000 7f4db8ff9640 1 -- 172.21.15.112:0/1978867579 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f4dac06c9c0 con 0x7f4dbc078b20 2024-09-16T00:17:07.646 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:04.001+0000 7f4dbb7fe640 1 --2- 172.21.15.112:0/1978867579 >> [v2:172.21.15.112:6800/663349889,v1:172.21.15.112:6801/663349889] conn(0x7f4da8039490 0x7f4da803b940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T00:17:07.646 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:04.202+0000 7f4dbb7fe640 1 -- 172.21.15.112:0/1978867579 >> [v2:172.21.15.112:6800/663349889,v1:172.21.15.112:6801/663349889] conn(0x7f4da8039490 msgr2=0x7f4da803b940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/663349889 2024-09-16T00:17:07.646 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:04.202+0000 7f4dbb7fe640 1 --2- 172.21.15.112:0/1978867579 >> [v2:172.21.15.112:6800/663349889,v1:172.21.15.112:6801/663349889] conn(0x7f4da8039490 0x7f4da803b940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T00:17:07.646 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:04.603+0000 7f4dbb7fe640 1 -- 172.21.15.112:0/1978867579 >> [v2:172.21.15.112:6800/663349889,v1:172.21.15.112:6801/663349889] conn(0x7f4da8039490 msgr2=0x7f4da803b940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/663349889 2024-09-16T00:17:07.646 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:04.603+0000 7f4dbb7fe640 1 --2- 172.21.15.112:0/1978867579 >> [v2:172.21.15.112:6800/663349889,v1:172.21.15.112:6801/663349889] conn(0x7f4da8039490 0x7f4da803b940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T00:17:07.646 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:05.404+0000 7f4dbb7fe640 1 -- 172.21.15.112:0/1978867579 >> [v2:172.21.15.112:6800/663349889,v1:172.21.15.112:6801/663349889] conn(0x7f4da8039490 msgr2=0x7f4da803b940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/663349889 2024-09-16T00:17:07.646 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:05.404+0000 7f4dbb7fe640 1 --2- 172.21.15.112:0/1978867579 >> [v2:172.21.15.112:6800/663349889,v1:172.21.15.112:6801/663349889] conn(0x7f4da8039490 0x7f4da803b940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T00:17:07.646 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:05.976+0000 7f4db8ff9640 1 -- 172.21.15.112:0/1978867579 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mgrmap(e 6) v1 ==== 45274+0+0 (secure 0 0 0) 0x7f4dac06bb70 con 0x7f4dbc078b20 2024-09-16T00:17:07.646 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:05.976+0000 7f4db8ff9640 1 -- 172.21.15.112:0/1978867579 >> [v2:172.21.15.112:6800/663349889,v1:172.21.15.112:6801/663349889] conn(0x7f4da8039490 msgr2=0x7f4da803b940 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T00:17:07.646 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:05.976+0000 7f4db8ff9640 1 --2- 172.21.15.112:0/1978867579 >> [v2:172.21.15.112:6800/663349889,v1:172.21.15.112:6801/663349889] conn(0x7f4da8039490 0x7f4da803b940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:07.647 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:06.976+0000 7f4db8ff9640 1 -- 172.21.15.112:0/1978867579 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mgrmap(e 7) v1 ==== 45401+0+0 (secure 0 0 0) 0x7f4dac03ad50 con 0x7f4dbc078b20 2024-09-16T00:17:07.647 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:06.977+0000 7f4db8ff9640 1 --2- 172.21.15.112:0/1978867579 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7f4da803cdf0 0x7f4da803f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:07.647 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:06.977+0000 7f4db8ff9640 1 -- 172.21.15.112:0/1978867579 --> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4da803c050 con 0x7f4da803cdf0 2024-09-16T00:17:07.647 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:06.977+0000 7f4dbb7fe640 1 --2- 172.21.15.112:0/1978867579 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7f4da803cdf0 0x7f4da803f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:07.647 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:06.978+0000 7f4dbb7fe640 1 --2- 172.21.15.112:0/1978867579 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7f4da803cdf0 0x7f4da803f1d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f4da4003940 tx=0x7f4da4009350 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:07.647 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:06.979+0000 7f4db8ff9640 1 -- 172.21.15.112:0/1978867579 <== mgr.14118 v2:172.21.15.112:6800/122493397 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (secure 0 0 0) 0x7f4da803c050 con 0x7f4da803cdf0 2024-09-16T00:17:07.647 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:06.984+0000 7f4dc252b640 1 -- 172.21.15.112:0/1978867579 --> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f4dbc113b60 con 0x7f4da803cdf0 2024-09-16T00:17:07.647 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:06.984+0000 7f4db8ff9640 1 -- 172.21.15.112:0/1978867579 <== mgr.14118 v2:172.21.15.112:6800/122493397 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f4dbc113b60 con 0x7f4da803cdf0 2024-09-16T00:17:07.647 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:06.984+0000 7f4dc252b640 1 -- 172.21.15.112:0/1978867579 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7f4da803cdf0 msgr2=0x7f4da803f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:07.647 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:06.984+0000 7f4dc252b640 1 --2- 172.21.15.112:0/1978867579 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7f4da803cdf0 0x7f4da803f1d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f4da4003940 tx=0x7f4da4009350 comp rx=0 tx=0).stop 2024-09-16T00:17:07.647 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:06.984+0000 7f4dc252b640 1 -- 172.21.15.112:0/1978867579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4dbc078b20 msgr2=0x7f4dbc112fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:07.647 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:06.984+0000 7f4dc252b640 1 --2- 172.21.15.112:0/1978867579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4dbc078b20 0x7f4dbc112fb0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4dac006f00 tx=0x7f4dac008040 comp rx=0 tx=0).stop 2024-09-16T00:17:07.647 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:06.984+0000 7f4dc252b640 1 -- 172.21.15.112:0/1978867579 shutdown_connections 2024-09-16T00:17:07.647 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:06.984+0000 7f4dc252b640 1 --2- 172.21.15.112:0/1978867579 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7f4da803cdf0 0x7f4da803f1d0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:07.648 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:06.984+0000 7f4dc252b640 1 --2- 172.21.15.112:0/1978867579 >> [v2:172.21.15.112:6800/663349889,v1:172.21.15.112:6801/663349889] conn(0x7f4da8039490 0x7f4da803b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:07.648 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:06.984+0000 7f4dc252b640 1 --2- 172.21.15.112:0/1978867579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4dbc078b20 0x7f4dbc112fb0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:07.648 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:06.984+0000 7f4dc252b640 1 -- 172.21.15.112:0/1978867579 >> 172.21.15.112:0/1978867579 conn(0x7f4dbc0fd880 msgr2=0x7f4dbc0fe470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:07.648 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:06.984+0000 7f4dc252b640 1 -- 172.21.15.112:0/1978867579 shutdown_connections 2024-09-16T00:17:07.648 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:06.984+0000 7f4dc252b640 1 -- 172.21.15.112:0/1978867579 wait complete. 2024-09-16T00:17:07.648 INFO:teuthology.orchestra.run.smithi112.stdout:mgr epoch 5 is available 2024-09-16T00:17:07.648 INFO:teuthology.orchestra.run.smithi112.stdout:Setting orchestrator backend to cephadm... 2024-09-16T00:17:08.219 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:08 smithi112 ceph-mon[27869]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-16T00:17:08.219 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:08 smithi112 ceph-mon[27869]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-16T00:17:08.219 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:08 smithi112 ceph-mon[27869]: [16/Sep/2024:00:17:07] ENGINE Bus STARTING 2024-09-16T00:17:08.219 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:08 smithi112 ceph-mon[27869]: from='mgr.14118 172.21.15.112:0/2162065181' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:17:08.219 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:08 smithi112 ceph-mon[27869]: from='mgr.14118 172.21.15.112:0/2162065181' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:17:08.795 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:08.034+0000 7f29f4154640 1 Processor -- start 2024-09-16T00:17:08.795 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:08.034+0000 7f29f4154640 1 -- start start 2024-09-16T00:17:08.795 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:08.034+0000 7f29f4154640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29ec076af0 0x7f29ec104c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:08.796 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:08.035+0000 7f29f4154640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29ec076f00 con 0x7f29ec076af0 2024-09-16T00:17:08.796 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:08.035+0000 7f29f1ef0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29ec076af0 0x7f29ec104c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:08.796 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:08.035+0000 7f29f1ef0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29ec076af0 0x7f29ec104c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51188/0 (socket says 172.21.15.112:51188) 2024-09-16T00:17:08.796 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:08.035+0000 7f29f1ef0640 1 -- 172.21.15.112:0/1851509106 learned_addr learned my addr 172.21.15.112:0/1851509106 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:17:08.796 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:08.035+0000 7f29f1ef0640 1 -- 172.21.15.112:0/1851509106 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29ec105150 con 0x7f29ec076af0 2024-09-16T00:17:08.796 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:08.036+0000 7f29f1ef0640 1 --2- 172.21.15.112:0/1851509106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29ec076af0 0x7f29ec104c10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f29e0009840 tx=0x7f29e00091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=207a6d550956be2f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:08.797 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:08.036+0000 7f29f0eee640 1 -- 172.21.15.112:0/1851509106 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f29e0032440 con 0x7f29ec076af0 2024-09-16T00:17:08.797 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:08.036+0000 7f29f0eee640 1 -- 172.21.15.112:0/1851509106 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1005+0+0 (secure 0 0 0) 0x7f29e0032a00 con 0x7f29ec076af0 2024-09-16T00:17:08.797 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:08.036+0000 7f29f0eee640 1 -- 172.21.15.112:0/1851509106 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f29e0031a10 con 0x7f29ec076af0 2024-09-16T00:17:08.797 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:08.036+0000 7f29f4154640 1 -- 172.21.15.112:0/1851509106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29ec076af0 msgr2=0x7f29ec104c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:08.797 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:08.036+0000 7f29f4154640 1 --2- 172.21.15.112:0/1851509106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29ec076af0 0x7f29ec104c10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f29e0009840 tx=0x7f29e00091d0 comp rx=0 tx=0).stop 2024-09-16T00:17:08.797 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:08.036+0000 7f29f4154640 1 -- 172.21.15.112:0/1851509106 shutdown_connections 2024-09-16T00:17:08.798 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:08.036+0000 7f29f4154640 1 --2- 172.21.15.112:0/1851509106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29ec076af0 0x7f29ec104c10 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:08.798 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:08.036+0000 7f29f4154640 1 -- 172.21.15.112:0/1851509106 >> 172.21.15.112:0/1851509106 conn(0x7f29ec0fc700 msgr2=0x7f29ec0feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:08.798 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:08.037+0000 7f29f4154640 1 -- 172.21.15.112:0/1851509106 shutdown_connections 2024-09-16T00:17:08.798 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:08.037+0000 7f29f4154640 1 -- 172.21.15.112:0/1851509106 wait complete. 2024-09-16T00:17:08.798 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:08.037+0000 7f29f4154640 1 Processor -- start 2024-09-16T00:17:08.799 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:08.037+0000 7f29f4154640 1 -- start start 2024-09-16T00:17:08.799 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:08.037+0000 7f29f4154640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29ec076af0 0x7f29ec110e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:08.799 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:08.037+0000 7f29f4154640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29ec111390 con 0x7f29ec076af0 2024-09-16T00:17:08.799 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:08.037+0000 7f29f1ef0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29ec076af0 0x7f29ec110e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:08.799 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:08.037+0000 7f29f1ef0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29ec076af0 0x7f29ec110e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51192/0 (socket says 172.21.15.112:51192) 2024-09-16T00:17:08.799 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:08.037+0000 7f29f1ef0640 1 -- 172.21.15.112:0/2164005150 learned_addr learned my addr 172.21.15.112:0/2164005150 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:17:08.800 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:08.037+0000 7f29f1ef0640 1 -- 172.21.15.112:0/2164005150 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29e0009450 con 0x7f29ec076af0 2024-09-16T00:17:08.800 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:08.038+0000 7f29f1ef0640 1 --2- 172.21.15.112:0/2164005150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29ec076af0 0x7f29ec110e50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f29e0006a70 tx=0x7f29e002c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:08.800 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:08.038+0000 7f29de7fc640 1 -- 172.21.15.112:0/2164005150 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f29e0031c90 con 0x7f29ec076af0 2024-09-16T00:17:08.800 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:08.038+0000 7f29de7fc640 1 -- 172.21.15.112:0/2164005150 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1005+0+0 (secure 0 0 0) 0x7f29e0031df0 con 0x7f29ec076af0 2024-09-16T00:17:08.800 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:08.038+0000 7f29de7fc640 1 -- 172.21.15.112:0/2164005150 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f29e003b5e0 con 0x7f29ec076af0 2024-09-16T00:17:08.800 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:08.038+0000 7f29f4154640 1 -- 172.21.15.112:0/2164005150 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29ec1115f0 con 0x7f29ec076af0 2024-09-16T00:17:08.801 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:08.038+0000 7f29f4154640 1 -- 172.21.15.112:0/2164005150 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29ec111f90 con 0x7f29ec076af0 2024-09-16T00:17:08.801 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:08.039+0000 7f29de7fc640 1 -- 172.21.15.112:0/2164005150 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 7) v1 ==== 45401+0+0 (secure 0 0 0) 0x7f29e0039070 con 0x7f29ec076af0 2024-09-16T00:17:08.801 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:08.039+0000 7f29de7fc640 1 --2- 172.21.15.112:0/2164005150 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7f29d8039380 0x7f29d803b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:08.801 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:08.039+0000 7f29f4154640 1 -- 172.21.15.112:0/2164005150 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29ec04b720 con 0x7f29ec076af0 2024-09-16T00:17:08.801 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:08.039+0000 7f29de7fc640 1 -- 172.21.15.112:0/2164005150 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f29e006ce50 con 0x7f29ec076af0 2024-09-16T00:17:08.801 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:08.039+0000 7f29f16ef640 1 --2- 172.21.15.112:0/2164005150 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7f29d8039380 0x7f29d803b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:08.802 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:08.039+0000 7f29f16ef640 1 --2- 172.21.15.112:0/2164005150 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7f29d8039380 0x7f29d803b830 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f29e8006fd0 tx=0x7f29e8006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:08.802 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:08.043+0000 7f29de7fc640 1 -- 172.21.15.112:0/2164005150 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f29e0041020 con 0x7f29ec076af0 2024-09-16T00:17:08.802 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:08.209+0000 7f29f4154640 1 -- 172.21.15.112:0/2164005150 --> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f29ec112120 con 0x7f29d8039380 2024-09-16T00:17:08.802 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:08.217+0000 7f29de7fc640 1 -- 172.21.15.112:0/2164005150 <== mgr.14118 v2:172.21.15.112:6800/122493397 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f29ec112120 con 0x7f29d8039380 2024-09-16T00:17:08.802 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:08.219+0000 7f29f4154640 1 -- 172.21.15.112:0/2164005150 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7f29d8039380 msgr2=0x7f29d803b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:08.802 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:08.219+0000 7f29f4154640 1 --2- 172.21.15.112:0/2164005150 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7f29d8039380 0x7f29d803b830 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f29e8006fd0 tx=0x7f29e8006db0 comp rx=0 tx=0).stop 2024-09-16T00:17:08.803 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:08.219+0000 7f29f4154640 1 -- 172.21.15.112:0/2164005150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29ec076af0 msgr2=0x7f29ec110e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:08.803 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:08.219+0000 7f29f4154640 1 --2- 172.21.15.112:0/2164005150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29ec076af0 0x7f29ec110e50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f29e0006a70 tx=0x7f29e002c040 comp rx=0 tx=0).stop 2024-09-16T00:17:08.803 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:08.219+0000 7f29f4154640 1 -- 172.21.15.112:0/2164005150 shutdown_connections 2024-09-16T00:17:08.804 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:08.219+0000 7f29f4154640 1 --2- 172.21.15.112:0/2164005150 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7f29d8039380 0x7f29d803b830 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:08.804 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:08.219+0000 7f29f4154640 1 --2- 172.21.15.112:0/2164005150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29ec076af0 0x7f29ec110e50 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:08.804 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:08.219+0000 7f29f4154640 1 -- 172.21.15.112:0/2164005150 >> 172.21.15.112:0/2164005150 conn(0x7f29ec0fc700 msgr2=0x7f29ec0fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:08.804 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:08.219+0000 7f29f4154640 1 -- 172.21.15.112:0/2164005150 shutdown_connections 2024-09-16T00:17:08.804 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:08.220+0000 7f29f4154640 1 -- 172.21.15.112:0/2164005150 wait complete. 2024-09-16T00:17:09.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:09 smithi112 ceph-mon[27869]: [16/Sep/2024:00:17:07] ENGINE Serving on https://172.21.15.112:7150 2024-09-16T00:17:09.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:09 smithi112 ceph-mon[27869]: [16/Sep/2024:00:17:07] ENGINE Bus STARTED 2024-09-16T00:17:09.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:09 smithi112 ceph-mon[27869]: [16/Sep/2024:00:17:07] ENGINE Error in HTTPServer.serve 2024-09-16T00:17:09.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: Traceback (most recent call last): 2024-09-16T00:17:09.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T00:17:09.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: self._connections.run(self.expiration_interval) 2024-09-16T00:17:09.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T00:17:09.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: self._run(expiration_interval) 2024-09-16T00:17:09.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T00:17:09.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T00:17:09.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T00:17:09.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T00:17:09.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T00:17:09.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: s = self.context.wrap_socket( 2024-09-16T00:17:09.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T00:17:09.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: return self.sslsocket_class._create( 2024-09-16T00:17:09.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T00:17:09.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: self.do_handshake() 2024-09-16T00:17:09.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T00:17:09.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: self._sslobj.do_handshake() 2024-09-16T00:17:09.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T00:17:09.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:09 smithi112 ceph-mon[27869]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:17:09.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:09 smithi112 ceph-mon[27869]: from='mgr.14118 172.21.15.112:0/2162065181' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:17:09.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:09 smithi112 ceph-mon[27869]: from='mgr.14118 172.21.15.112:0/2162065181' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:17:09.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:09 smithi112 ceph-mon[27869]: mgrmap e8: smithi112.xrkkbc(active, since 2s) 2024-09-16T00:17:10.021 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-16T00:17:10.022 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:09.215+0000 7f633a442640 1 Processor -- start 2024-09-16T00:17:10.022 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:09.215+0000 7f633a442640 1 -- start start 2024-09-16T00:17:10.022 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:09.215+0000 7f633a442640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6334076ab0 0x7f6334104cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:10.022 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:09.215+0000 7f633a442640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6334076ec0 con 0x7f6334076ab0 2024-09-16T00:17:10.022 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:09.216+0000 7f6333fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6334076ab0 0x7f6334104cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:10.022 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:09.216+0000 7f6333fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6334076ab0 0x7f6334104cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51208/0 (socket says 172.21.15.112:51208) 2024-09-16T00:17:10.022 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:09.216+0000 7f6333fff640 1 -- 172.21.15.112:0/709059573 learned_addr learned my addr 172.21.15.112:0/709059573 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:17:10.022 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:09.216+0000 7f6333fff640 1 -- 172.21.15.112:0/709059573 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6334105200 con 0x7f6334076ab0 2024-09-16T00:17:10.022 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:09.217+0000 7f6333fff640 1 --2- 172.21.15.112:0/709059573 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6334076ab0 0x7f6334104cc0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6324009840 tx=0x7f63240091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=169c7d249d4632e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:10.022 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:09.217+0000 7f63337fe640 1 -- 172.21.15.112:0/709059573 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6324032440 con 0x7f6334076ab0 2024-09-16T00:17:10.022 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:09.217+0000 7f63337fe640 1 -- 172.21.15.112:0/709059573 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1005+0+0 (secure 0 0 0) 0x7f6324032a00 con 0x7f6334076ab0 2024-09-16T00:17:10.022 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:09.217+0000 7f63337fe640 1 -- 172.21.15.112:0/709059573 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f63240319e0 con 0x7f6334076ab0 2024-09-16T00:17:10.023 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:09.218+0000 7f633a442640 1 -- 172.21.15.112:0/709059573 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6334076ab0 msgr2=0x7f6334104cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:10.023 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:09.218+0000 7f633a442640 1 --2- 172.21.15.112:0/709059573 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6334076ab0 0x7f6334104cc0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6324009840 tx=0x7f63240091d0 comp rx=0 tx=0).stop 2024-09-16T00:17:10.023 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:09.218+0000 7f633a442640 1 -- 172.21.15.112:0/709059573 shutdown_connections 2024-09-16T00:17:10.023 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:09.218+0000 7f633a442640 1 --2- 172.21.15.112:0/709059573 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6334076ab0 0x7f6334104cc0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:10.023 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:09.218+0000 7f633a442640 1 -- 172.21.15.112:0/709059573 >> 172.21.15.112:0/709059573 conn(0x7f63340fc710 msgr2=0x7f63340feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:10.023 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:09.218+0000 7f633a442640 1 -- 172.21.15.112:0/709059573 shutdown_connections 2024-09-16T00:17:10.023 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:09.218+0000 7f633a442640 1 -- 172.21.15.112:0/709059573 wait complete. 2024-09-16T00:17:10.023 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:09.219+0000 7f633a442640 1 Processor -- start 2024-09-16T00:17:10.023 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:09.219+0000 7f633a442640 1 -- start start 2024-09-16T00:17:10.023 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:09.219+0000 7f633a442640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6334076ab0 0x7f63341025f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:10.023 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:09.219+0000 7f633a442640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6334102b30 con 0x7f6334076ab0 2024-09-16T00:17:10.023 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:09.219+0000 7f6333fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6334076ab0 0x7f63341025f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:10.023 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:09.219+0000 7f6333fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6334076ab0 0x7f63341025f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51210/0 (socket says 172.21.15.112:51210) 2024-09-16T00:17:10.024 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:09.219+0000 7f6333fff640 1 -- 172.21.15.112:0/3437964642 learned_addr learned my addr 172.21.15.112:0/3437964642 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:17:10.024 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:09.219+0000 7f6333fff640 1 -- 172.21.15.112:0/3437964642 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6324009450 con 0x7f6334076ab0 2024-09-16T00:17:10.024 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:09.220+0000 7f6333fff640 1 --2- 172.21.15.112:0/3437964642 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6334076ab0 0x7f63341025f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f63240098a0 tx=0x7f6324008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:10.024 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:09.220+0000 7f63317fa640 1 -- 172.21.15.112:0/3437964642 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6324032880 con 0x7f6334076ab0 2024-09-16T00:17:10.024 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:09.220+0000 7f63317fa640 1 -- 172.21.15.112:0/3437964642 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1005+0+0 (secure 0 0 0) 0x7f632403d8b0 con 0x7f6334076ab0 2024-09-16T00:17:10.024 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:09.220+0000 7f633a442640 1 -- 172.21.15.112:0/3437964642 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6334100bc0 con 0x7f6334076ab0 2024-09-16T00:17:10.024 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:09.220+0000 7f63317fa640 1 -- 172.21.15.112:0/3437964642 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f632403b8e0 con 0x7f6334076ab0 2024-09-16T00:17:10.024 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:09.220+0000 7f633a442640 1 -- 172.21.15.112:0/3437964642 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6334101030 con 0x7f6334076ab0 2024-09-16T00:17:10.024 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:09.221+0000 7f63317fa640 1 -- 172.21.15.112:0/3437964642 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) v1 ==== 45507+0+0 (secure 0 0 0) 0x7f6324039070 con 0x7f6334076ab0 2024-09-16T00:17:10.024 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:09.221+0000 7f63317fa640 1 --2- 172.21.15.112:0/3437964642 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7f6320039490 0x7f632003b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:10.024 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:09.221+0000 7f63317fa640 1 -- 172.21.15.112:0/3437964642 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f632406e1d0 con 0x7f6334076ab0 2024-09-16T00:17:10.024 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:09.221+0000 7f632bfff640 1 --2- 172.21.15.112:0/3437964642 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7f6320039490 0x7f632003b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:10.025 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:09.221+0000 7f633a442640 1 -- 172.21.15.112:0/3437964642 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f633404b720 con 0x7f6334076ab0 2024-09-16T00:17:10.025 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:09.221+0000 7f632bfff640 1 --2- 172.21.15.112:0/3437964642 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7f6320039490 0x7f632003b940 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f631c006fd0 tx=0x7f631c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:10.025 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:09.226+0000 7f63317fa640 1 -- 172.21.15.112:0/3437964642 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f6324041080 con 0x7f6334076ab0 2024-09-16T00:17:10.025 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:09.387+0000 7f633a442640 1 -- 172.21.15.112:0/3437964642 --> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f63341015b0 con 0x7f6320039490 2024-09-16T00:17:10.025 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:09.387+0000 7f63317fa640 1 -- 172.21.15.112:0/3437964642 <== mgr.14118 v2:172.21.15.112:6800/122493397 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f63341015b0 con 0x7f6320039490 2024-09-16T00:17:10.025 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:09.389+0000 7f633a442640 1 -- 172.21.15.112:0/3437964642 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7f6320039490 msgr2=0x7f632003b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:10.025 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:09.389+0000 7f633a442640 1 --2- 172.21.15.112:0/3437964642 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7f6320039490 0x7f632003b940 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f631c006fd0 tx=0x7f631c006db0 comp rx=0 tx=0).stop 2024-09-16T00:17:10.025 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:09.389+0000 7f633a442640 1 -- 172.21.15.112:0/3437964642 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6334076ab0 msgr2=0x7f63341025f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:10.025 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:09.389+0000 7f633a442640 1 --2- 172.21.15.112:0/3437964642 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6334076ab0 0x7f63341025f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f63240098a0 tx=0x7f6324008040 comp rx=0 tx=0).stop 2024-09-16T00:17:10.025 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:09.389+0000 7f633a442640 1 -- 172.21.15.112:0/3437964642 shutdown_connections 2024-09-16T00:17:10.025 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:09.389+0000 7f633a442640 1 --2- 172.21.15.112:0/3437964642 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7f6320039490 0x7f632003b940 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:10.025 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:09.389+0000 7f633a442640 1 --2- 172.21.15.112:0/3437964642 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6334076ab0 0x7f63341025f0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:10.026 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:09.389+0000 7f633a442640 1 -- 172.21.15.112:0/3437964642 >> 172.21.15.112:0/3437964642 conn(0x7f63340fc710 msgr2=0x7f63340fd2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:10.026 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:09.389+0000 7f633a442640 1 -- 172.21.15.112:0/3437964642 shutdown_connections 2024-09-16T00:17:10.026 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:09.389+0000 7f633a442640 1 -- 172.21.15.112:0/3437964642 wait complete. 2024-09-16T00:17:10.026 INFO:teuthology.orchestra.run.smithi112.stdout:Generating ssh key... 2024-09-16T00:17:10.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:10 smithi112 ceph-mon[27869]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:17:11.303 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:11 smithi112 ceph-mon[27869]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:17:11.303 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:11 smithi112 ceph-mon[27869]: Generating ssh key... 2024-09-16T00:17:11.303 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:11 smithi112 ceph-mon[27869]: from='mgr.14118 172.21.15.112:0/2162065181' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:17:11.304 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:11 smithi112 ceph-mon[27869]: from='mgr.14118 172.21.15.112:0/2162065181' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:17:11.306 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:10.427+0000 7fc6f9836640 1 Processor -- start 2024-09-16T00:17:11.313 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:10.427+0000 7fc6f9836640 1 -- start start 2024-09-16T00:17:11.313 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:10.427+0000 7fc6f9836640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6f4100e90 0x7fc6f41012a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:11.313 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:10.427+0000 7fc6f9836640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f41017e0 con 0x7fc6f4100e90 2024-09-16T00:17:11.313 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:10.427+0000 7fc6f2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6f4100e90 0x7fc6f41012a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:11.313 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:10.427+0000 7fc6f2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6f4100e90 0x7fc6f41012a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51214/0 (socket says 172.21.15.112:51214) 2024-09-16T00:17:11.313 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:10.428+0000 7fc6f2ffd640 1 -- 172.21.15.112:0/615276743 learned_addr learned my addr 172.21.15.112:0/615276743 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:17:11.313 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:10.428+0000 7fc6f2ffd640 1 -- 172.21.15.112:0/615276743 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6f4101920 con 0x7fc6f4100e90 2024-09-16T00:17:11.313 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:10.428+0000 7fc6f2ffd640 1 --2- 172.21.15.112:0/615276743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6f4100e90 0x7fc6f41012a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4009e50 tx=0x7fc6e4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c813249da15cce0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:11.313 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:10.429+0000 7fc6f1ffb640 1 -- 172.21.15.112:0/615276743 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc6e4031560 con 0x7fc6f4100e90 2024-09-16T00:17:11.313 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:10.429+0000 7fc6f1ffb640 1 -- 172.21.15.112:0/615276743 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1005+0+0 (secure 0 0 0) 0x7fc6e402d050 con 0x7fc6f4100e90 2024-09-16T00:17:11.313 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:10.429+0000 7fc6f1ffb640 1 -- 172.21.15.112:0/615276743 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc6e40079b0 con 0x7fc6f4100e90 2024-09-16T00:17:11.314 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:10.429+0000 7fc6f9836640 1 -- 172.21.15.112:0/615276743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6f4100e90 msgr2=0x7fc6f41012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:11.314 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:10.429+0000 7fc6f9836640 1 --2- 172.21.15.112:0/615276743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6f4100e90 0x7fc6f41012a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4009e50 tx=0x7fc6e4009840 comp rx=0 tx=0).stop 2024-09-16T00:17:11.314 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:10.429+0000 7fc6f9836640 1 -- 172.21.15.112:0/615276743 shutdown_connections 2024-09-16T00:17:11.314 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:10.429+0000 7fc6f9836640 1 --2- 172.21.15.112:0/615276743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6f4100e90 0x7fc6f41012a0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:11.314 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:10.429+0000 7fc6f9836640 1 -- 172.21.15.112:0/615276743 >> 172.21.15.112:0/615276743 conn(0x7fc6f40fc6d0 msgr2=0x7fc6f40feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:11.314 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:10.429+0000 7fc6f9836640 1 -- 172.21.15.112:0/615276743 shutdown_connections 2024-09-16T00:17:11.314 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:10.429+0000 7fc6f9836640 1 -- 172.21.15.112:0/615276743 wait complete. 2024-09-16T00:17:11.314 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:10.430+0000 7fc6f9836640 1 Processor -- start 2024-09-16T00:17:11.314 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:10.430+0000 7fc6f9836640 1 -- start start 2024-09-16T00:17:11.314 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:10.430+0000 7fc6f9836640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6f4100e90 0x7fc6f410fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:11.314 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:10.430+0000 7fc6f9836640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f4110180 con 0x7fc6f4100e90 2024-09-16T00:17:11.314 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:10.430+0000 7fc6f2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6f4100e90 0x7fc6f410fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:11.315 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:10.430+0000 7fc6f2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6f4100e90 0x7fc6f410fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51226/0 (socket says 172.21.15.112:51226) 2024-09-16T00:17:11.315 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:10.430+0000 7fc6f2ffd640 1 -- 172.21.15.112:0/3936719258 learned_addr learned my addr 172.21.15.112:0/3936719258 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:17:11.315 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:10.431+0000 7fc6f2ffd640 1 -- 172.21.15.112:0/3936719258 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6e4009ae0 con 0x7fc6f4100e90 2024-09-16T00:17:11.315 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:10.431+0000 7fc6f2ffd640 1 --2- 172.21.15.112:0/3936719258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6f4100e90 0x7fc6f410fc40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4007030 tx=0x7fc6e4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:11.315 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:10.431+0000 7fc6eb7fe640 1 -- 172.21.15.112:0/3936719258 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc6e4045430 con 0x7fc6f4100e90 2024-09-16T00:17:11.315 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:10.431+0000 7fc6eb7fe640 1 -- 172.21.15.112:0/3936719258 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1005+0+0 (secure 0 0 0) 0x7fc6e40459f0 con 0x7fc6f4100e90 2024-09-16T00:17:11.315 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:10.431+0000 7fc6f9836640 1 -- 172.21.15.112:0/3936719258 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6f4110380 con 0x7fc6f4100e90 2024-09-16T00:17:11.315 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:10.431+0000 7fc6eb7fe640 1 -- 172.21.15.112:0/3936719258 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc6e403a8f0 con 0x7fc6f4100e90 2024-09-16T00:17:11.315 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:10.431+0000 7fc6f9836640 1 -- 172.21.15.112:0/3936719258 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6f41107f0 con 0x7fc6f4100e90 2024-09-16T00:17:11.315 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:10.432+0000 7fc6eb7fe640 1 -- 172.21.15.112:0/3936719258 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) v1 ==== 45507+0+0 (secure 0 0 0) 0x7fc6e4038040 con 0x7fc6f4100e90 2024-09-16T00:17:11.315 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:10.432+0000 7fc6f9836640 1 -- 172.21.15.112:0/3936719258 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6f404b720 con 0x7fc6f4100e90 2024-09-16T00:17:11.315 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:10.432+0000 7fc6eb7fe640 1 --2- 172.21.15.112:0/3936719258 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7fc6e0039490 0x7fc6e003b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:11.316 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:10.432+0000 7fc6eb7fe640 1 -- 172.21.15.112:0/3936719258 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fc6e4040030 con 0x7fc6f4100e90 2024-09-16T00:17:11.316 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:10.432+0000 7fc6f27fc640 1 --2- 172.21.15.112:0/3936719258 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7fc6e0039490 0x7fc6e003b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:11.316 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:10.433+0000 7fc6f27fc640 1 --2- 172.21.15.112:0/3936719258 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7fc6e0039490 0x7fc6e003b940 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc006fd0 tx=0x7fc6dc006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:11.316 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:10.437+0000 7fc6eb7fe640 1 -- 172.21.15.112:0/3936719258 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fc6e40382f0 con 0x7fc6f4100e90 2024-09-16T00:17:11.316 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:10.591+0000 7fc6f9836640 1 -- 172.21.15.112:0/3936719258 --> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fc6f41107f0 con 0x7fc6e0039490 2024-09-16T00:17:11.316 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:10.697+0000 7fc6eb7fe640 1 -- 172.21.15.112:0/3936719258 <== mgr.14118 v2:172.21.15.112:6800/122493397 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fc6f41107f0 con 0x7fc6e0039490 2024-09-16T00:17:11.316 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:10.699+0000 7fc6f9836640 1 -- 172.21.15.112:0/3936719258 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7fc6e0039490 msgr2=0x7fc6e003b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:11.316 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:10.699+0000 7fc6f9836640 1 --2- 172.21.15.112:0/3936719258 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7fc6e0039490 0x7fc6e003b940 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc006fd0 tx=0x7fc6dc006db0 comp rx=0 tx=0).stop 2024-09-16T00:17:11.316 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:10.699+0000 7fc6f9836640 1 -- 172.21.15.112:0/3936719258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6f4100e90 msgr2=0x7fc6f410fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:11.316 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:10.699+0000 7fc6f9836640 1 --2- 172.21.15.112:0/3936719258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6f4100e90 0x7fc6f410fc40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4007030 tx=0x7fc6e4009840 comp rx=0 tx=0).stop 2024-09-16T00:17:11.317 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:10.699+0000 7fc6f9836640 1 -- 172.21.15.112:0/3936719258 shutdown_connections 2024-09-16T00:17:11.317 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:10.699+0000 7fc6f9836640 1 --2- 172.21.15.112:0/3936719258 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7fc6e0039490 0x7fc6e003b940 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:11.317 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:10.699+0000 7fc6f9836640 1 --2- 172.21.15.112:0/3936719258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6f4100e90 0x7fc6f410fc40 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:11.317 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:10.699+0000 7fc6f9836640 1 -- 172.21.15.112:0/3936719258 >> 172.21.15.112:0/3936719258 conn(0x7fc6f40fc6d0 msgr2=0x7fc6f40fd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:11.317 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:10.699+0000 7fc6f9836640 1 -- 172.21.15.112:0/3936719258 shutdown_connections 2024-09-16T00:17:11.317 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:10.699+0000 7fc6f9836640 1 -- 172.21.15.112:0/3936719258 wait complete. 2024-09-16T00:17:12.451 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7l4AiGYDDQARb/gsM1EGdF3xeIpdQ7vfY8ZU2NDvIGeQndFGtL0R0lJB3An7epXqaOeVCgSo2m172+4hnYyi0YAGsqZaoCF1AwJOTOiZWrIynZ+7buBqctQq2KCY6bv+sNXOk0Lxox7C7479vyc4JAJtWeOK3u+2EoiIF1KbnHP9CENMf4TAzxFAowDU87z/rHoUTKiXvYT8dL0GiMWsKwGdgbALaWlRIWVglPZYYOcvgwpobpHjB1m3ahPe1WVI703RTyWvtvPtrMs1eQthnm0r3qpWfsV6LPj60A2KcSh7S288ZYiGGpJx1c5rOSEVQspRadHVxXxSWzVApTz2IKP0y8OBc0OJYUcKr80F7Wm4FKRMYyNTJ+4LgAotrvuc7hlNp449tURHqG5pIN7n8q2V8whwvnlsRoNDMWUBoTuxHHkWN6Wk6nZhJRHCcLAqx7kCEyrt1IcYYIvrxZ6GzR6x/kkDqxZ1GVCJe19KLgGCTonkacBT4SLIip7w47AE= ceph-d346dd56-73c0-11ef-bceb-c7b262605968 2024-09-16T00:17:12.451 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:11.718+0000 7fd9c625f640 1 Processor -- start 2024-09-16T00:17:12.452 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:11.718+0000 7fd9c625f640 1 -- start start 2024-09-16T00:17:12.452 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:11.718+0000 7fd9c625f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9c0102050 0x7fd9c0102460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:12.452 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:11.719+0000 7fd9c625f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c01029a0 con 0x7fd9c0102050 2024-09-16T00:17:12.452 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:11.719+0000 7fd9bf7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9c0102050 0x7fd9c0102460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:12.452 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:11.719+0000 7fd9bf7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9c0102050 0x7fd9c0102460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51230/0 (socket says 172.21.15.112:51230) 2024-09-16T00:17:12.452 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:11.719+0000 7fd9bf7fe640 1 -- 172.21.15.112:0/2758467537 learned_addr learned my addr 172.21.15.112:0/2758467537 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:17:12.452 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:11.719+0000 7fd9bf7fe640 1 -- 172.21.15.112:0/2758467537 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9c0102ae0 con 0x7fd9c0102050 2024-09-16T00:17:12.452 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:11.720+0000 7fd9bf7fe640 1 --2- 172.21.15.112:0/2758467537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9c0102050 0x7fd9c0102460 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd9b0009840 tx=0x7fd9b00091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e1073e45c11371b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:12.452 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:11.720+0000 7fd9be7fc640 1 -- 172.21.15.112:0/2758467537 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd9b0032440 con 0x7fd9c0102050 2024-09-16T00:17:12.452 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:11.720+0000 7fd9be7fc640 1 -- 172.21.15.112:0/2758467537 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1005+0+0 (secure 0 0 0) 0x7fd9b0032a00 con 0x7fd9c0102050 2024-09-16T00:17:12.452 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:11.720+0000 7fd9be7fc640 1 -- 172.21.15.112:0/2758467537 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd9b00319e0 con 0x7fd9c0102050 2024-09-16T00:17:12.452 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:11.720+0000 7fd9c625f640 1 -- 172.21.15.112:0/2758467537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9c0102050 msgr2=0x7fd9c0102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:12.453 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:11.720+0000 7fd9c625f640 1 --2- 172.21.15.112:0/2758467537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9c0102050 0x7fd9c0102460 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd9b0009840 tx=0x7fd9b00091d0 comp rx=0 tx=0).stop 2024-09-16T00:17:12.453 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:11.721+0000 7fd9c625f640 1 -- 172.21.15.112:0/2758467537 shutdown_connections 2024-09-16T00:17:12.453 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:11.721+0000 7fd9c625f640 1 --2- 172.21.15.112:0/2758467537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9c0102050 0x7fd9c0102460 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:12.453 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:11.721+0000 7fd9c625f640 1 -- 172.21.15.112:0/2758467537 >> 172.21.15.112:0/2758467537 conn(0x7fd9c00fd7f0 msgr2=0x7fd9c00ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:12.453 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:11.721+0000 7fd9c625f640 1 -- 172.21.15.112:0/2758467537 shutdown_connections 2024-09-16T00:17:12.453 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:11.721+0000 7fd9c625f640 1 -- 172.21.15.112:0/2758467537 wait complete. 2024-09-16T00:17:12.453 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:11.721+0000 7fd9c625f640 1 Processor -- start 2024-09-16T00:17:12.453 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:11.721+0000 7fd9c625f640 1 -- start start 2024-09-16T00:17:12.453 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:11.721+0000 7fd9c625f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9c0102050 0x7fd9c0110dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:12.453 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:11.721+0000 7fd9c625f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c0111310 con 0x7fd9c0102050 2024-09-16T00:17:12.453 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:11.722+0000 7fd9bf7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9c0102050 0x7fd9c0110dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:12.453 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:11.722+0000 7fd9bf7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9c0102050 0x7fd9c0110dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51232/0 (socket says 172.21.15.112:51232) 2024-09-16T00:17:12.453 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:11.722+0000 7fd9bf7fe640 1 -- 172.21.15.112:0/1032832762 learned_addr learned my addr 172.21.15.112:0/1032832762 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:17:12.454 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:11.722+0000 7fd9bf7fe640 1 -- 172.21.15.112:0/1032832762 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b0009450 con 0x7fd9c0102050 2024-09-16T00:17:12.454 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:11.722+0000 7fd9bf7fe640 1 --2- 172.21.15.112:0/1032832762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9c0102050 0x7fd9c0110dd0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd9b00098a0 tx=0x7fd9b0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:12.454 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:11.722+0000 7fd9a7fff640 1 -- 172.21.15.112:0/1032832762 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd9b0032880 con 0x7fd9c0102050 2024-09-16T00:17:12.454 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:11.722+0000 7fd9a7fff640 1 -- 172.21.15.112:0/1032832762 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1005+0+0 (secure 0 0 0) 0x7fd9b003d8b0 con 0x7fd9c0102050 2024-09-16T00:17:12.454 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:11.722+0000 7fd9c625f640 1 -- 172.21.15.112:0/1032832762 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9c0111510 con 0x7fd9c0102050 2024-09-16T00:17:12.454 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:11.722+0000 7fd9a7fff640 1 -- 172.21.15.112:0/1032832762 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd9b003b910 con 0x7fd9c0102050 2024-09-16T00:17:12.454 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:11.722+0000 7fd9c625f640 1 -- 172.21.15.112:0/1032832762 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9c0111f60 con 0x7fd9c0102050 2024-09-16T00:17:12.454 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:11.723+0000 7fd9a7fff640 1 -- 172.21.15.112:0/1032832762 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) v1 ==== 45507+0+0 (secure 0 0 0) 0x7fd9b0039070 con 0x7fd9c0102050 2024-09-16T00:17:12.454 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:11.723+0000 7fd9c625f640 1 -- 172.21.15.112:0/1032832762 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9c01014b0 con 0x7fd9c0102050 2024-09-16T00:17:12.454 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:11.723+0000 7fd9a7fff640 1 --2- 172.21.15.112:0/1032832762 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7fd9a80394e0 0x7fd9a803b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:12.454 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:11.723+0000 7fd9a7fff640 1 -- 172.21.15.112:0/1032832762 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fd9b006d2e0 con 0x7fd9c0102050 2024-09-16T00:17:12.454 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:11.724+0000 7fd9beffd640 1 --2- 172.21.15.112:0/1032832762 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7fd9a80394e0 0x7fd9a803b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:12.454 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:11.724+0000 7fd9beffd640 1 --2- 172.21.15.112:0/1032832762 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7fd9a80394e0 0x7fd9a803b990 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd9b4006fd0 tx=0x7fd9b4006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:12.455 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:11.728+0000 7fd9a7fff640 1 -- 172.21.15.112:0/1032832762 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fd9b0039320 con 0x7fd9c0102050 2024-09-16T00:17:12.455 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:11.884+0000 7fd9c625f640 1 -- 172.21.15.112:0/1032832762 --> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fd9c00feae0 con 0x7fd9a80394e0 2024-09-16T00:17:12.455 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:11.885+0000 7fd9a7fff640 1 -- 172.21.15.112:0/1032832762 <== mgr.14118 v2:172.21.15.112:6800/122493397 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7fd9c00feae0 con 0x7fd9a80394e0 2024-09-16T00:17:12.455 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:11.887+0000 7fd9c625f640 1 -- 172.21.15.112:0/1032832762 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7fd9a80394e0 msgr2=0x7fd9a803b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:12.455 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:11.887+0000 7fd9c625f640 1 --2- 172.21.15.112:0/1032832762 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7fd9a80394e0 0x7fd9a803b990 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd9b4006fd0 tx=0x7fd9b4006db0 comp rx=0 tx=0).stop 2024-09-16T00:17:12.455 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:11.887+0000 7fd9c625f640 1 -- 172.21.15.112:0/1032832762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9c0102050 msgr2=0x7fd9c0110dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:12.455 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:11.887+0000 7fd9c625f640 1 --2- 172.21.15.112:0/1032832762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9c0102050 0x7fd9c0110dd0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd9b00098a0 tx=0x7fd9b0008040 comp rx=0 tx=0).stop 2024-09-16T00:17:12.455 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:11.887+0000 7fd9c625f640 1 -- 172.21.15.112:0/1032832762 shutdown_connections 2024-09-16T00:17:12.455 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:11.887+0000 7fd9c625f640 1 --2- 172.21.15.112:0/1032832762 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7fd9a80394e0 0x7fd9a803b990 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:12.455 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:11.887+0000 7fd9c625f640 1 --2- 172.21.15.112:0/1032832762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9c0102050 0x7fd9c0110dd0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:12.455 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:11.887+0000 7fd9c625f640 1 -- 172.21.15.112:0/1032832762 >> 172.21.15.112:0/1032832762 conn(0x7fd9c00fd7f0 msgr2=0x7fd9c00fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:12.455 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:11.887+0000 7fd9c625f640 1 -- 172.21.15.112:0/1032832762 shutdown_connections 2024-09-16T00:17:12.455 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:11.887+0000 7fd9c625f640 1 -- 172.21.15.112:0/1032832762 wait complete. 2024-09-16T00:17:12.456 INFO:teuthology.orchestra.run.smithi112.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-16T00:17:12.456 INFO:teuthology.orchestra.run.smithi112.stdout:Adding key to root@localhost authorized_keys... 2024-09-16T00:17:12.457 INFO:teuthology.orchestra.run.smithi112.stdout:Adding host smithi112... 2024-09-16T00:17:12.710 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:12 smithi112 ceph-mon[27869]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:17:13.868 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:13 smithi112 ceph-mon[27869]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi112", "addr": "172.21.15.112", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:17:14.812 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:14 smithi112 ceph-mon[27869]: Deploying cephadm binary to smithi112 2024-09-16T00:17:16.359 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout Added host 'smithi112' with addr '172.21.15.112' 2024-09-16T00:17:16.359 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:12.898+0000 7f96ef3b9640 1 Processor -- start 2024-09-16T00:17:16.359 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:12.899+0000 7f96ef3b9640 1 -- start start 2024-09-16T00:17:16.359 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:12.899+0000 7f96ef3b9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96e8078b00 0x7f96e8077210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:16.359 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:12.899+0000 7f96ef3b9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e8078fa0 con 0x7f96e8078b00 2024-09-16T00:17:16.359 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:12.899+0000 7f96ed155640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96e8078b00 0x7f96e8077210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:16.359 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:12.899+0000 7f96ed155640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96e8078b00 0x7f96e8077210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48300/0 (socket says 172.21.15.112:48300) 2024-09-16T00:17:16.359 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:12.899+0000 7f96ed155640 1 -- 172.21.15.112:0/4099853620 learned_addr learned my addr 172.21.15.112:0/4099853620 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:17:16.360 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:12.899+0000 7f96ed155640 1 -- 172.21.15.112:0/4099853620 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96e80790e0 con 0x7f96e8078b00 2024-09-16T00:17:16.360 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:12.900+0000 7f96ed155640 1 --2- 172.21.15.112:0/4099853620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96e8078b00 0x7f96e8077210 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f96e4009840 tx=0x7f96e40091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e25da4165a0c6ab0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:16.360 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:12.900+0000 7f96dbfff640 1 -- 172.21.15.112:0/4099853620 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f96e4032440 con 0x7f96e8078b00 2024-09-16T00:17:16.360 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:12.900+0000 7f96dbfff640 1 -- 172.21.15.112:0/4099853620 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1005+0+0 (secure 0 0 0) 0x7f96e4032a00 con 0x7f96e8078b00 2024-09-16T00:17:16.360 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:12.900+0000 7f96dbfff640 1 -- 172.21.15.112:0/4099853620 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f96e40319e0 con 0x7f96e8078b00 2024-09-16T00:17:16.360 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:12.901+0000 7f96ef3b9640 1 -- 172.21.15.112:0/4099853620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96e8078b00 msgr2=0x7f96e8077210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:16.360 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:12.901+0000 7f96ef3b9640 1 --2- 172.21.15.112:0/4099853620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96e8078b00 0x7f96e8077210 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f96e4009840 tx=0x7f96e40091d0 comp rx=0 tx=0).stop 2024-09-16T00:17:16.360 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:12.901+0000 7f96ef3b9640 1 -- 172.21.15.112:0/4099853620 shutdown_connections 2024-09-16T00:17:16.360 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:12.901+0000 7f96ef3b9640 1 --2- 172.21.15.112:0/4099853620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96e8078b00 0x7f96e8077210 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:16.360 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:12.901+0000 7f96ef3b9640 1 -- 172.21.15.112:0/4099853620 >> 172.21.15.112:0/4099853620 conn(0x7f96e80fd850 msgr2=0x7f96e80ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:16.360 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:12.901+0000 7f96ef3b9640 1 -- 172.21.15.112:0/4099853620 shutdown_connections 2024-09-16T00:17:16.360 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:12.901+0000 7f96ef3b9640 1 -- 172.21.15.112:0/4099853620 wait complete. 2024-09-16T00:17:16.360 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:12.901+0000 7f96ef3b9640 1 Processor -- start 2024-09-16T00:17:16.360 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:12.901+0000 7f96ef3b9640 1 -- start start 2024-09-16T00:17:16.361 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:12.902+0000 7f96ef3b9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96e8078b00 0x7f96e8112fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:16.361 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:12.902+0000 7f96ef3b9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e81134e0 con 0x7f96e8078b00 2024-09-16T00:17:16.361 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:12.902+0000 7f96ed155640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96e8078b00 0x7f96e8112fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:16.361 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:12.902+0000 7f96ed155640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96e8078b00 0x7f96e8112fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48312/0 (socket says 172.21.15.112:48312) 2024-09-16T00:17:16.361 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:12.902+0000 7f96ed155640 1 -- 172.21.15.112:0/3157127466 learned_addr learned my addr 172.21.15.112:0/3157127466 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:17:16.361 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:12.902+0000 7f96ed155640 1 -- 172.21.15.112:0/3157127466 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96e4009450 con 0x7f96e8078b00 2024-09-16T00:17:16.361 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:12.902+0000 7f96ed155640 1 --2- 172.21.15.112:0/3157127466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96e8078b00 0x7f96e8112fa0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f96e4006f00 tx=0x7f96e4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:16.361 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:12.902+0000 7f96d9ffb640 1 -- 172.21.15.112:0/3157127466 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f96e403b470 con 0x7f96e8078b00 2024-09-16T00:17:16.361 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:12.902+0000 7f96d9ffb640 1 -- 172.21.15.112:0/3157127466 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1005+0+0 (secure 0 0 0) 0x7f96e403ba30 con 0x7f96e8078b00 2024-09-16T00:17:16.361 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:12.902+0000 7f96ef3b9640 1 -- 172.21.15.112:0/3157127466 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96e81136e0 con 0x7f96e8078b00 2024-09-16T00:17:16.361 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:12.902+0000 7f96d9ffb640 1 -- 172.21.15.112:0/3157127466 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f96e403a820 con 0x7f96e8078b00 2024-09-16T00:17:16.361 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:12.903+0000 7f96ef3b9640 1 -- 172.21.15.112:0/3157127466 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96e8113b50 con 0x7f96e8078b00 2024-09-16T00:17:16.361 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:12.903+0000 7f96d9ffb640 1 -- 172.21.15.112:0/3157127466 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) v1 ==== 45507+0+0 (secure 0 0 0) 0x7f96e403aaa0 con 0x7f96e8078b00 2024-09-16T00:17:16.361 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:12.903+0000 7f96ef3b9640 1 -- 172.21.15.112:0/3157127466 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96e804b720 con 0x7f96e8078b00 2024-09-16T00:17:16.362 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:12.904+0000 7f96d9ffb640 1 --2- 172.21.15.112:0/3157127466 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7f96d4039490 0x7f96d403b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:16.362 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:12.904+0000 7f96d9ffb640 1 -- 172.21.15.112:0/3157127466 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f96e406c9e0 con 0x7f96e8078b00 2024-09-16T00:17:16.362 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:12.904+0000 7f96ec954640 1 --2- 172.21.15.112:0/3157127466 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7f96d4039490 0x7f96d403b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:16.362 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:12.904+0000 7f96ec954640 1 --2- 172.21.15.112:0/3157127466 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7f96d4039490 0x7f96d403b940 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f96dc006fd0 tx=0x7f96dc006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:16.362 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:12.908+0000 7f96d9ffb640 1 -- 172.21.15.112:0/3157127466 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f96e4045460 con 0x7f96e8078b00 2024-09-16T00:17:16.362 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:13.071+0000 7f96ef3b9640 1 -- 172.21.15.112:0/3157127466 --> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi112", "addr": "172.21.15.112", "target": ["mon-mgr", ""]}) v1 -- 0x7f96e8113b50 con 0x7f96d4039490 2024-09-16T00:17:16.362 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:15.769+0000 7f96d9ffb640 1 -- 172.21.15.112:0/3157127466 <== mgr.14118 v2:172.21.15.112:6800/122493397 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f96e8113b50 con 0x7f96d4039490 2024-09-16T00:17:16.362 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:15.772+0000 7f96ef3b9640 1 -- 172.21.15.112:0/3157127466 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7f96d4039490 msgr2=0x7f96d403b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:16.362 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:15.772+0000 7f96ef3b9640 1 --2- 172.21.15.112:0/3157127466 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7f96d4039490 0x7f96d403b940 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f96dc006fd0 tx=0x7f96dc006db0 comp rx=0 tx=0).stop 2024-09-16T00:17:16.362 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:15.772+0000 7f96ef3b9640 1 -- 172.21.15.112:0/3157127466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96e8078b00 msgr2=0x7f96e8112fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:16.362 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:15.772+0000 7f96ef3b9640 1 --2- 172.21.15.112:0/3157127466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96e8078b00 0x7f96e8112fa0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f96e4006f00 tx=0x7f96e4008040 comp rx=0 tx=0).stop 2024-09-16T00:17:16.362 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:15.772+0000 7f96ef3b9640 1 -- 172.21.15.112:0/3157127466 shutdown_connections 2024-09-16T00:17:16.362 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:15.772+0000 7f96ef3b9640 1 --2- 172.21.15.112:0/3157127466 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7f96d4039490 0x7f96d403b940 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:16.363 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:15.772+0000 7f96ef3b9640 1 --2- 172.21.15.112:0/3157127466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96e8078b00 0x7f96e8112fa0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:16.363 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:15.772+0000 7f96ef3b9640 1 -- 172.21.15.112:0/3157127466 >> 172.21.15.112:0/3157127466 conn(0x7f96e80fd850 msgr2=0x7f96e80fe460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:16.363 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:15.772+0000 7f96ef3b9640 1 -- 172.21.15.112:0/3157127466 shutdown_connections 2024-09-16T00:17:16.363 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:15.772+0000 7f96ef3b9640 1 -- 172.21.15.112:0/3157127466 wait complete. 2024-09-16T00:17:16.363 INFO:teuthology.orchestra.run.smithi112.stdout:Deploying mon service with default placement... 2024-09-16T00:17:16.977 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:16 smithi112 ceph-mon[27869]: from='mgr.14118 172.21.15.112:0/2162065181' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:17:16.977 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:16 smithi112 ceph-mon[27869]: Added host smithi112 2024-09-16T00:17:16.977 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:16 smithi112 ceph-mon[27869]: from='mgr.14118 172.21.15.112:0/2162065181' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:17:17.803 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-16T00:17:17.804 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:16.788+0000 7faeaa930640 1 Processor -- start 2024-09-16T00:17:17.804 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:16.788+0000 7faeaa930640 1 -- start start 2024-09-16T00:17:17.804 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:16.788+0000 7faeaa930640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faea4101f90 0x7faea41023a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:17.804 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:16.788+0000 7faeaa930640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea41028e0 con 0x7faea4101f90 2024-09-16T00:17:17.804 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:16.788+0000 7faea3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faea4101f90 0x7faea41023a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:17.804 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:16.788+0000 7faea3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faea4101f90 0x7faea41023a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48314/0 (socket says 172.21.15.112:48314) 2024-09-16T00:17:17.804 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:16.788+0000 7faea3fff640 1 -- 172.21.15.112:0/2050318869 learned_addr learned my addr 172.21.15.112:0/2050318869 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:17:17.804 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:16.789+0000 7faea3fff640 1 -- 172.21.15.112:0/2050318869 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faea4102a20 con 0x7faea4101f90 2024-09-16T00:17:17.804 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:16.789+0000 7faea3fff640 1 --2- 172.21.15.112:0/2050318869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faea4101f90 0x7faea41023a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fae94009840 tx=0x7fae940091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=107c12ecc18d1212 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:17.804 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:16.789+0000 7faea2ffd640 1 -- 172.21.15.112:0/2050318869 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fae94032440 con 0x7faea4101f90 2024-09-16T00:17:17.804 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:16.790+0000 7faea2ffd640 1 -- 172.21.15.112:0/2050318869 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1005+0+0 (secure 0 0 0) 0x7fae94032a00 con 0x7faea4101f90 2024-09-16T00:17:17.804 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:16.790+0000 7faea2ffd640 1 -- 172.21.15.112:0/2050318869 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fae940319e0 con 0x7faea4101f90 2024-09-16T00:17:17.805 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:16.790+0000 7faeaa930640 1 -- 172.21.15.112:0/2050318869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faea4101f90 msgr2=0x7faea41023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:17.805 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:16.790+0000 7faeaa930640 1 --2- 172.21.15.112:0/2050318869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faea4101f90 0x7faea41023a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fae94009840 tx=0x7fae940091d0 comp rx=0 tx=0).stop 2024-09-16T00:17:17.805 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:16.790+0000 7faeaa930640 1 -- 172.21.15.112:0/2050318869 shutdown_connections 2024-09-16T00:17:17.805 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:16.790+0000 7faeaa930640 1 --2- 172.21.15.112:0/2050318869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faea4101f90 0x7faea41023a0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:17.805 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:16.790+0000 7faeaa930640 1 -- 172.21.15.112:0/2050318869 >> 172.21.15.112:0/2050318869 conn(0x7faea40fd710 msgr2=0x7faea40ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:17.805 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:16.790+0000 7faeaa930640 1 -- 172.21.15.112:0/2050318869 shutdown_connections 2024-09-16T00:17:17.805 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:16.790+0000 7faeaa930640 1 -- 172.21.15.112:0/2050318869 wait complete. 2024-09-16T00:17:17.805 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:16.791+0000 7faeaa930640 1 Processor -- start 2024-09-16T00:17:17.805 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:16.791+0000 7faeaa930640 1 -- start start 2024-09-16T00:17:17.805 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:16.791+0000 7faeaa930640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faea4101f90 0x7faea4110ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:17.805 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:16.791+0000 7faeaa930640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea4111220 con 0x7faea4101f90 2024-09-16T00:17:17.805 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:16.791+0000 7faea3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faea4101f90 0x7faea4110ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:17.805 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:16.791+0000 7faea3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faea4101f90 0x7faea4110ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48326/0 (socket says 172.21.15.112:48326) 2024-09-16T00:17:17.805 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:16.791+0000 7faea3fff640 1 -- 172.21.15.112:0/2843405507 learned_addr learned my addr 172.21.15.112:0/2843405507 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:17:17.806 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:16.791+0000 7faea3fff640 1 -- 172.21.15.112:0/2843405507 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae94009450 con 0x7faea4101f90 2024-09-16T00:17:17.806 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:16.791+0000 7faea3fff640 1 --2- 172.21.15.112:0/2843405507 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faea4101f90 0x7faea4110ce0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fae940098a0 tx=0x7fae94008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:17.806 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:16.792+0000 7faea0ff9640 1 -- 172.21.15.112:0/2843405507 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fae94032880 con 0x7faea4101f90 2024-09-16T00:17:17.806 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:16.792+0000 7faeaa930640 1 -- 172.21.15.112:0/2843405507 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faea4111420 con 0x7faea4101f90 2024-09-16T00:17:17.806 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:16.792+0000 7faea0ff9640 1 -- 172.21.15.112:0/2843405507 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1005+0+0 (secure 0 0 0) 0x7fae9403d8b0 con 0x7faea4101f90 2024-09-16T00:17:17.806 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:16.792+0000 7faea0ff9640 1 -- 172.21.15.112:0/2843405507 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fae9403b910 con 0x7faea4101f90 2024-09-16T00:17:17.806 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:16.792+0000 7faeaa930640 1 -- 172.21.15.112:0/2843405507 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faea4111890 con 0x7faea4101f90 2024-09-16T00:17:17.806 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:16.793+0000 7faea0ff9640 1 -- 172.21.15.112:0/2843405507 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) v1 ==== 45507+0+0 (secure 0 0 0) 0x7fae94039070 con 0x7faea4101f90 2024-09-16T00:17:17.806 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:16.793+0000 7faeaa930640 1 -- 172.21.15.112:0/2843405507 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faea404b720 con 0x7faea4101f90 2024-09-16T00:17:17.806 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:16.793+0000 7faea0ff9640 1 --2- 172.21.15.112:0/2843405507 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7fae8c039490 0x7fae8c03b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:17.806 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:16.793+0000 7faea0ff9640 1 -- 172.21.15.112:0/2843405507 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fae9406d2a0 con 0x7faea4101f90 2024-09-16T00:17:17.806 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:16.793+0000 7faea37fe640 1 --2- 172.21.15.112:0/2843405507 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7fae8c039490 0x7fae8c03b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:17.806 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:16.793+0000 7faea37fe640 1 --2- 172.21.15.112:0/2843405507 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7fae8c039490 0x7fae8c03b940 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fae98006fd0 tx=0x7fae98006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:17.807 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:16.797+0000 7faea0ff9640 1 -- 172.21.15.112:0/2843405507 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fae9404ed70 con 0x7faea4101f90 2024-09-16T00:17:17.807 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:16.976+0000 7faeaa930640 1 -- 172.21.15.112:0/2843405507 --> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}) v1 -- 0x7faea4111890 con 0x7fae8c039490 2024-09-16T00:17:17.807 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:16.984+0000 7faea0ff9640 1 -- 172.21.15.112:0/2843405507 <== mgr.14118 v2:172.21.15.112:6800/122493397 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7faea4111890 con 0x7fae8c039490 2024-09-16T00:17:17.807 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:16.985+0000 7faeaa930640 1 -- 172.21.15.112:0/2843405507 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7fae8c039490 msgr2=0x7fae8c03b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:17.807 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:16.985+0000 7faeaa930640 1 --2- 172.21.15.112:0/2843405507 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7fae8c039490 0x7fae8c03b940 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fae98006fd0 tx=0x7fae98006db0 comp rx=0 tx=0).stop 2024-09-16T00:17:17.807 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:16.985+0000 7faeaa930640 1 -- 172.21.15.112:0/2843405507 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faea4101f90 msgr2=0x7faea4110ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:17.807 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:16.985+0000 7faeaa930640 1 --2- 172.21.15.112:0/2843405507 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faea4101f90 0x7faea4110ce0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fae940098a0 tx=0x7fae94008040 comp rx=0 tx=0).stop 2024-09-16T00:17:17.807 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:16.985+0000 7faeaa930640 1 -- 172.21.15.112:0/2843405507 shutdown_connections 2024-09-16T00:17:17.807 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:16.985+0000 7faeaa930640 1 --2- 172.21.15.112:0/2843405507 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7fae8c039490 0x7fae8c03b940 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:17.807 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:16.985+0000 7faeaa930640 1 --2- 172.21.15.112:0/2843405507 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faea4101f90 0x7faea4110ce0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:17.807 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:16.985+0000 7faeaa930640 1 -- 172.21.15.112:0/2843405507 >> 172.21.15.112:0/2843405507 conn(0x7faea40fd710 msgr2=0x7faea40fe260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:17.807 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:16.986+0000 7faeaa930640 1 -- 172.21.15.112:0/2843405507 shutdown_connections 2024-09-16T00:17:17.807 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:16.986+0000 7faeaa930640 1 -- 172.21.15.112:0/2843405507 wait complete. 2024-09-16T00:17:17.807 INFO:teuthology.orchestra.run.smithi112.stdout:Deploying mgr service with default placement... 2024-09-16T00:17:18.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:17 smithi112 ceph-mon[27869]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:17:18.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:17 smithi112 ceph-mon[27869]: Saving service mon spec with placement count:5 2024-09-16T00:17:18.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:17 smithi112 ceph-mon[27869]: from='mgr.14118 172.21.15.112:0/2162065181' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:17:19.079 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-16T00:17:19.079 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:18.217+0000 7fe2c43fd640 1 Processor -- start 2024-09-16T00:17:19.079 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:18.217+0000 7fe2c43fd640 1 -- start start 2024-09-16T00:17:19.079 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:18.217+0000 7fe2c43fd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2bc101f90 0x7fe2bc1023a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:19.079 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:18.217+0000 7fe2c43fd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2bc1028e0 con 0x7fe2bc101f90 2024-09-16T00:17:19.079 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:18.217+0000 7fe2c2199640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2bc101f90 0x7fe2bc1023a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:19.079 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:18.217+0000 7fe2c2199640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2bc101f90 0x7fe2bc1023a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48342/0 (socket says 172.21.15.112:48342) 2024-09-16T00:17:19.079 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:18.217+0000 7fe2c2199640 1 -- 172.21.15.112:0/264658314 learned_addr learned my addr 172.21.15.112:0/264658314 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:17:19.079 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:18.218+0000 7fe2c2199640 1 -- 172.21.15.112:0/264658314 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2bc102a20 con 0x7fe2bc101f90 2024-09-16T00:17:19.079 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:18.218+0000 7fe2c2199640 1 --2- 172.21.15.112:0/264658314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2bc101f90 0x7fe2bc1023a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8009840 tx=0x7fe2b80091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34a0b530732ceacd server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:19.080 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:18.218+0000 7fe2c1197640 1 -- 172.21.15.112:0/264658314 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe2b8032440 con 0x7fe2bc101f90 2024-09-16T00:17:19.080 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:18.218+0000 7fe2c1197640 1 -- 172.21.15.112:0/264658314 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1005+0+0 (secure 0 0 0) 0x7fe2b8032a00 con 0x7fe2bc101f90 2024-09-16T00:17:19.080 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:18.218+0000 7fe2c1197640 1 -- 172.21.15.112:0/264658314 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe2b80319e0 con 0x7fe2bc101f90 2024-09-16T00:17:19.080 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:18.219+0000 7fe2c43fd640 1 -- 172.21.15.112:0/264658314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2bc101f90 msgr2=0x7fe2bc1023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:19.080 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:18.219+0000 7fe2c43fd640 1 --2- 172.21.15.112:0/264658314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2bc101f90 0x7fe2bc1023a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8009840 tx=0x7fe2b80091d0 comp rx=0 tx=0).stop 2024-09-16T00:17:19.080 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:18.219+0000 7fe2c43fd640 1 -- 172.21.15.112:0/264658314 shutdown_connections 2024-09-16T00:17:19.080 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:18.219+0000 7fe2c43fd640 1 --2- 172.21.15.112:0/264658314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2bc101f90 0x7fe2bc1023a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:19.080 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:18.219+0000 7fe2c43fd640 1 -- 172.21.15.112:0/264658314 >> 172.21.15.112:0/264658314 conn(0x7fe2bc0fd710 msgr2=0x7fe2bc0ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:19.080 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:18.219+0000 7fe2c43fd640 1 -- 172.21.15.112:0/264658314 shutdown_connections 2024-09-16T00:17:19.080 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:18.219+0000 7fe2c43fd640 1 -- 172.21.15.112:0/264658314 wait complete. 2024-09-16T00:17:19.080 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:18.219+0000 7fe2c43fd640 1 Processor -- start 2024-09-16T00:17:19.080 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:18.220+0000 7fe2c43fd640 1 -- start start 2024-09-16T00:17:19.080 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:18.220+0000 7fe2c43fd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2bc101f90 0x7fe2bc110ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:19.081 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:18.220+0000 7fe2c43fd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2bc111220 con 0x7fe2bc101f90 2024-09-16T00:17:19.081 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:18.220+0000 7fe2c2199640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2bc101f90 0x7fe2bc110ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:19.081 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:18.220+0000 7fe2c2199640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2bc101f90 0x7fe2bc110ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48352/0 (socket says 172.21.15.112:48352) 2024-09-16T00:17:19.081 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:18.220+0000 7fe2c2199640 1 -- 172.21.15.112:0/2456949460 learned_addr learned my addr 172.21.15.112:0/2456949460 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:17:19.081 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:18.220+0000 7fe2c2199640 1 -- 172.21.15.112:0/2456949460 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2b8009450 con 0x7fe2bc101f90 2024-09-16T00:17:19.081 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:18.220+0000 7fe2c2199640 1 --2- 172.21.15.112:0/2456949460 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2bc101f90 0x7fe2bc110ce0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe2b80098a0 tx=0x7fe2b8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:19.081 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:18.220+0000 7fe2b2ffd640 1 -- 172.21.15.112:0/2456949460 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe2b8032880 con 0x7fe2bc101f90 2024-09-16T00:17:19.081 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:18.220+0000 7fe2b2ffd640 1 -- 172.21.15.112:0/2456949460 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1005+0+0 (secure 0 0 0) 0x7fe2b803d8b0 con 0x7fe2bc101f90 2024-09-16T00:17:19.081 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:18.220+0000 7fe2c43fd640 1 -- 172.21.15.112:0/2456949460 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2bc111420 con 0x7fe2bc101f90 2024-09-16T00:17:19.081 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:18.220+0000 7fe2b2ffd640 1 -- 172.21.15.112:0/2456949460 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe2b803b910 con 0x7fe2bc101f90 2024-09-16T00:17:19.081 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:18.220+0000 7fe2c43fd640 1 -- 172.21.15.112:0/2456949460 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2bc111890 con 0x7fe2bc101f90 2024-09-16T00:17:19.081 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:18.221+0000 7fe2b2ffd640 1 -- 172.21.15.112:0/2456949460 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) v1 ==== 45507+0+0 (secure 0 0 0) 0x7fe2b8039070 con 0x7fe2bc101f90 2024-09-16T00:17:19.081 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:18.221+0000 7fe2c43fd640 1 -- 172.21.15.112:0/2456949460 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2bc04b720 con 0x7fe2bc101f90 2024-09-16T00:17:19.081 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:18.221+0000 7fe2b2ffd640 1 --2- 172.21.15.112:0/2456949460 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7fe2a80394e0 0x7fe2a803b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:19.082 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:18.221+0000 7fe2b2ffd640 1 -- 172.21.15.112:0/2456949460 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fe2b806d0d0 con 0x7fe2bc101f90 2024-09-16T00:17:19.082 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:18.222+0000 7fe2c1998640 1 --2- 172.21.15.112:0/2456949460 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7fe2a80394e0 0x7fe2a803b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:19.082 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:18.222+0000 7fe2c1998640 1 --2- 172.21.15.112:0/2456949460 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7fe2a80394e0 0x7fe2a803b990 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac006fd0 tx=0x7fe2ac006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:19.082 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:18.226+0000 7fe2b2ffd640 1 -- 172.21.15.112:0/2456949460 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fe2b803ca00 con 0x7fe2bc101f90 2024-09-16T00:17:19.082 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:18.396+0000 7fe2c43fd640 1 -- 172.21.15.112:0/2456949460 --> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7fe2bc111890 con 0x7fe2a80394e0 2024-09-16T00:17:19.082 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:18.403+0000 7fe2b2ffd640 1 -- 172.21.15.112:0/2456949460 <== mgr.14118 v2:172.21.15.112:6800/122493397 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fe2bc111890 con 0x7fe2a80394e0 2024-09-16T00:17:19.082 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:18.405+0000 7fe2c43fd640 1 -- 172.21.15.112:0/2456949460 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7fe2a80394e0 msgr2=0x7fe2a803b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:19.082 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:18.405+0000 7fe2c43fd640 1 --2- 172.21.15.112:0/2456949460 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7fe2a80394e0 0x7fe2a803b990 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac006fd0 tx=0x7fe2ac006db0 comp rx=0 tx=0).stop 2024-09-16T00:17:19.082 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:18.405+0000 7fe2c43fd640 1 -- 172.21.15.112:0/2456949460 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2bc101f90 msgr2=0x7fe2bc110ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:19.082 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:18.405+0000 7fe2c43fd640 1 --2- 172.21.15.112:0/2456949460 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2bc101f90 0x7fe2bc110ce0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe2b80098a0 tx=0x7fe2b8008040 comp rx=0 tx=0).stop 2024-09-16T00:17:19.082 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:18.405+0000 7fe2c43fd640 1 -- 172.21.15.112:0/2456949460 shutdown_connections 2024-09-16T00:17:19.082 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:18.405+0000 7fe2c43fd640 1 --2- 172.21.15.112:0/2456949460 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7fe2a80394e0 0x7fe2a803b990 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:19.083 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:18.405+0000 7fe2c43fd640 1 --2- 172.21.15.112:0/2456949460 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2bc101f90 0x7fe2bc110ce0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:19.083 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:18.405+0000 7fe2c43fd640 1 -- 172.21.15.112:0/2456949460 >> 172.21.15.112:0/2456949460 conn(0x7fe2bc0fd710 msgr2=0x7fe2bc0fe260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:19.083 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:18.405+0000 7fe2c43fd640 1 -- 172.21.15.112:0/2456949460 shutdown_connections 2024-09-16T00:17:19.083 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:18.405+0000 7fe2c43fd640 1 -- 172.21.15.112:0/2456949460 wait complete. 2024-09-16T00:17:19.083 INFO:teuthology.orchestra.run.smithi112.stdout:Deploying crash service with default placement... 2024-09-16T00:17:19.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:19 smithi112 ceph-mon[27869]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:17:19.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:19 smithi112 ceph-mon[27869]: Saving service mgr spec with placement count:2 2024-09-16T00:17:19.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:19 smithi112 ceph-mon[27869]: from='mgr.14118 172.21.15.112:0/2162065181' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:17:21.033 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:20 smithi112 ceph-mon[27869]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:17:21.034 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:20 smithi112 ceph-mon[27869]: Saving service crash spec with placement * 2024-09-16T00:17:21.034 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:20 smithi112 ceph-mon[27869]: from='mgr.14118 172.21.15.112:0/2162065181' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:17:21.034 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:20 smithi112 ceph-mon[27869]: from='mgr.14118 172.21.15.112:0/2162065181' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:17:21.040 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout Scheduled crash update... 2024-09-16T00:17:21.040 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:19.506+0000 7f630dc43640 1 Processor -- start 2024-09-16T00:17:21.041 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:19.506+0000 7f630dc43640 1 -- start start 2024-09-16T00:17:21.041 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:19.506+0000 7f630dc43640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6308076a00 0x7f6308106fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:21.041 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:19.506+0000 7f630dc43640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6308076e10 con 0x7f6308076a00 2024-09-16T00:17:21.041 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:19.507+0000 7f63077fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6308076a00 0x7f6308106fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:21.041 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:19.507+0000 7f63077fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6308076a00 0x7f6308106fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48356/0 (socket says 172.21.15.112:48356) 2024-09-16T00:17:21.041 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:19.507+0000 7f63077fe640 1 -- 172.21.15.112:0/3727838905 learned_addr learned my addr 172.21.15.112:0/3727838905 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:17:21.041 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:19.507+0000 7f63077fe640 1 -- 172.21.15.112:0/3727838905 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6308107520 con 0x7f6308076a00 2024-09-16T00:17:21.041 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:19.508+0000 7f63077fe640 1 --2- 172.21.15.112:0/3727838905 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6308076a00 0x7f6308106fe0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f62f8009840 tx=0x7f62f80091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94350fb57e62ce88 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:21.041 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:19.508+0000 7f6306ffd640 1 -- 172.21.15.112:0/3727838905 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f62f8032440 con 0x7f6308076a00 2024-09-16T00:17:21.041 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:19.508+0000 7f6306ffd640 1 -- 172.21.15.112:0/3727838905 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1005+0+0 (secure 0 0 0) 0x7f62f8032a00 con 0x7f6308076a00 2024-09-16T00:17:21.041 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:19.508+0000 7f6306ffd640 1 -- 172.21.15.112:0/3727838905 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f62f8031a10 con 0x7f6308076a00 2024-09-16T00:17:21.041 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:19.508+0000 7f630dc43640 1 -- 172.21.15.112:0/3727838905 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6308076a00 msgr2=0x7f6308106fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:21.041 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:19.508+0000 7f630dc43640 1 --2- 172.21.15.112:0/3727838905 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6308076a00 0x7f6308106fe0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f62f8009840 tx=0x7f62f80091d0 comp rx=0 tx=0).stop 2024-09-16T00:17:21.041 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:19.508+0000 7f630dc43640 1 -- 172.21.15.112:0/3727838905 shutdown_connections 2024-09-16T00:17:21.042 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:19.508+0000 7f630dc43640 1 --2- 172.21.15.112:0/3727838905 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6308076a00 0x7f6308106fe0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:21.042 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:19.508+0000 7f630dc43640 1 -- 172.21.15.112:0/3727838905 >> 172.21.15.112:0/3727838905 conn(0x7f63080fc6d0 msgr2=0x7f63080feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:21.042 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:19.508+0000 7f630dc43640 1 -- 172.21.15.112:0/3727838905 shutdown_connections 2024-09-16T00:17:21.042 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:19.508+0000 7f630dc43640 1 -- 172.21.15.112:0/3727838905 wait complete. 2024-09-16T00:17:21.042 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:19.509+0000 7f630dc43640 1 Processor -- start 2024-09-16T00:17:21.042 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:19.509+0000 7f630dc43640 1 -- start start 2024-09-16T00:17:21.042 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:19.509+0000 7f630dc43640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6308076a00 0x7f6308112f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:21.042 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:19.509+0000 7f630dc43640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6308113490 con 0x7f6308076a00 2024-09-16T00:17:21.042 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:19.509+0000 7f63077fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6308076a00 0x7f6308112f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:21.042 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:19.509+0000 7f63077fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6308076a00 0x7f6308112f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48364/0 (socket says 172.21.15.112:48364) 2024-09-16T00:17:21.042 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:19.509+0000 7f63077fe640 1 -- 172.21.15.112:0/1367708498 learned_addr learned my addr 172.21.15.112:0/1367708498 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:17:21.042 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:19.510+0000 7f63077fe640 1 -- 172.21.15.112:0/1367708498 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62f8009450 con 0x7f6308076a00 2024-09-16T00:17:21.042 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:19.510+0000 7f63077fe640 1 --2- 172.21.15.112:0/1367708498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6308076a00 0x7f6308112f50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f62f8006a70 tx=0x7f62f802c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:21.042 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:19.510+0000 7f6304ff9640 1 -- 172.21.15.112:0/1367708498 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f62f8031d10 con 0x7f6308076a00 2024-09-16T00:17:21.043 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:19.510+0000 7f6304ff9640 1 -- 172.21.15.112:0/1367708498 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1005+0+0 (secure 0 0 0) 0x7f62f8031e70 con 0x7f6308076a00 2024-09-16T00:17:21.043 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:19.510+0000 7f630dc43640 1 -- 172.21.15.112:0/1367708498 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6308113690 con 0x7f6308076a00 2024-09-16T00:17:21.043 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:19.510+0000 7f6304ff9640 1 -- 172.21.15.112:0/1367708498 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f62f803b6f0 con 0x7f6308076a00 2024-09-16T00:17:21.043 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:19.510+0000 7f630dc43640 1 -- 172.21.15.112:0/1367708498 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6308113b00 con 0x7f6308076a00 2024-09-16T00:17:21.043 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:19.511+0000 7f6304ff9640 1 -- 172.21.15.112:0/1367708498 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) v1 ==== 45507+0+0 (secure 0 0 0) 0x7f62f8039070 con 0x7f6308076a00 2024-09-16T00:17:21.043 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:19.511+0000 7f6304ff9640 1 --2- 172.21.15.112:0/1367708498 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7f62f4039490 0x7f62f403b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:21.043 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:19.511+0000 7f6304ff9640 1 -- 172.21.15.112:0/1367708498 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f62f806dc30 con 0x7f6308076a00 2024-09-16T00:17:21.043 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:19.511+0000 7f62feffd640 1 --2- 172.21.15.112:0/1367708498 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7f62f4039490 0x7f62f403b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:21.043 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:19.511+0000 7f630dc43640 1 -- 172.21.15.112:0/1367708498 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63081002f0 con 0x7f6308076a00 2024-09-16T00:17:21.043 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:19.512+0000 7f62feffd640 1 --2- 172.21.15.112:0/1367708498 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7f62f4039490 0x7f62f403b940 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f62f0006fd0 tx=0x7f62f0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:21.043 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:19.516+0000 7f6304ff9640 1 -- 172.21.15.112:0/1367708498 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f62f8037030 con 0x7f6308076a00 2024-09-16T00:17:21.043 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:19.680+0000 7f630dc43640 1 -- 172.21.15.112:0/1367708498 --> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}) v1 -- 0x7f63080fdb70 con 0x7f62f4039490 2024-09-16T00:17:21.043 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:19.684+0000 7f6304ff9640 1 -- 172.21.15.112:0/1367708498 <== mgr.14118 v2:172.21.15.112:6800/122493397 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f63080fdb70 con 0x7f62f4039490 2024-09-16T00:17:21.044 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:19.687+0000 7f630dc43640 1 -- 172.21.15.112:0/1367708498 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7f62f4039490 msgr2=0x7f62f403b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:21.044 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:19.687+0000 7f630dc43640 1 --2- 172.21.15.112:0/1367708498 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7f62f4039490 0x7f62f403b940 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f62f0006fd0 tx=0x7f62f0006db0 comp rx=0 tx=0).stop 2024-09-16T00:17:21.044 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:19.687+0000 7f630dc43640 1 -- 172.21.15.112:0/1367708498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6308076a00 msgr2=0x7f6308112f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:21.044 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:19.687+0000 7f630dc43640 1 --2- 172.21.15.112:0/1367708498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6308076a00 0x7f6308112f50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f62f8006a70 tx=0x7f62f802c040 comp rx=0 tx=0).stop 2024-09-16T00:17:21.044 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:19.687+0000 7f630dc43640 1 -- 172.21.15.112:0/1367708498 shutdown_connections 2024-09-16T00:17:21.044 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:19.687+0000 7f630dc43640 1 --2- 172.21.15.112:0/1367708498 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7f62f4039490 0x7f62f403b940 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:21.044 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:19.687+0000 7f630dc43640 1 --2- 172.21.15.112:0/1367708498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6308076a00 0x7f6308112f50 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:21.044 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:19.687+0000 7f630dc43640 1 -- 172.21.15.112:0/1367708498 >> 172.21.15.112:0/1367708498 conn(0x7f63080fc6d0 msgr2=0x7f63080fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:21.044 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:19.687+0000 7f630dc43640 1 -- 172.21.15.112:0/1367708498 shutdown_connections 2024-09-16T00:17:21.044 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:19.687+0000 7f630dc43640 1 -- 172.21.15.112:0/1367708498 wait complete. 2024-09-16T00:17:21.044 INFO:teuthology.orchestra.run.smithi112.stdout:Deploying prometheus service with default placement... 2024-09-16T00:17:21.898 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:21 smithi112 ceph-mon[27869]: from='mgr.14118 172.21.15.112:0/2162065181' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:17:21.898 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:21 smithi112 ceph-mon[27869]: from='mgr.14118 172.21.15.112:0/2162065181' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:17:22.559 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout Scheduled prometheus update... 2024-09-16T00:17:22.559 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:21.445+0000 7fb74ea14640 1 Processor -- start 2024-09-16T00:17:22.559 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:21.445+0000 7fb74ea14640 1 -- start start 2024-09-16T00:17:22.560 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:21.445+0000 7fb74ea14640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb748101f90 0x7fb7481023a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:22.560 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:21.445+0000 7fb74ea14640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7481028e0 con 0x7fb748101f90 2024-09-16T00:17:22.560 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:21.445+0000 7fb747fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb748101f90 0x7fb7481023a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:22.560 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:21.445+0000 7fb747fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb748101f90 0x7fb7481023a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48376/0 (socket says 172.21.15.112:48376) 2024-09-16T00:17:22.560 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:21.445+0000 7fb747fff640 1 -- 172.21.15.112:0/1813424017 learned_addr learned my addr 172.21.15.112:0/1813424017 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:17:22.560 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:21.446+0000 7fb747fff640 1 -- 172.21.15.112:0/1813424017 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb748102a20 con 0x7fb748101f90 2024-09-16T00:17:22.560 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:21.446+0000 7fb747fff640 1 --2- 172.21.15.112:0/1813424017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb748101f90 0x7fb7481023a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb738009840 tx=0x7fb7380091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b6f24eaee06a360 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:22.560 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:21.446+0000 7fb746ffd640 1 -- 172.21.15.112:0/1813424017 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb738032440 con 0x7fb748101f90 2024-09-16T00:17:22.560 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:21.447+0000 7fb746ffd640 1 -- 172.21.15.112:0/1813424017 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fb738032a40 con 0x7fb748101f90 2024-09-16T00:17:22.560 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:21.447+0000 7fb746ffd640 1 -- 172.21.15.112:0/1813424017 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb738031a70 con 0x7fb748101f90 2024-09-16T00:17:22.560 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:21.447+0000 7fb74ea14640 1 -- 172.21.15.112:0/1813424017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb748101f90 msgr2=0x7fb7481023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:22.560 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:21.447+0000 7fb74ea14640 1 --2- 172.21.15.112:0/1813424017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb748101f90 0x7fb7481023a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb738009840 tx=0x7fb7380091d0 comp rx=0 tx=0).stop 2024-09-16T00:17:22.560 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:21.447+0000 7fb74ea14640 1 -- 172.21.15.112:0/1813424017 shutdown_connections 2024-09-16T00:17:22.561 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:21.447+0000 7fb74ea14640 1 --2- 172.21.15.112:0/1813424017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb748101f90 0x7fb7481023a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:22.561 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:21.447+0000 7fb74ea14640 1 -- 172.21.15.112:0/1813424017 >> 172.21.15.112:0/1813424017 conn(0x7fb7480fd730 msgr2=0x7fb7480ffb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:22.561 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:21.447+0000 7fb74ea14640 1 -- 172.21.15.112:0/1813424017 shutdown_connections 2024-09-16T00:17:22.561 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:21.447+0000 7fb74ea14640 1 -- 172.21.15.112:0/1813424017 wait complete. 2024-09-16T00:17:22.561 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:21.448+0000 7fb74ea14640 1 Processor -- start 2024-09-16T00:17:22.561 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:21.448+0000 7fb74ea14640 1 -- start start 2024-09-16T00:17:22.561 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:21.448+0000 7fb74ea14640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb748101f90 0x7fb748110e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:22.561 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:21.448+0000 7fb74ea14640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb748111350 con 0x7fb748101f90 2024-09-16T00:17:22.561 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:21.448+0000 7fb747fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb748101f90 0x7fb748110e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:22.561 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:21.448+0000 7fb747fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb748101f90 0x7fb748110e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48388/0 (socket says 172.21.15.112:48388) 2024-09-16T00:17:22.561 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:21.448+0000 7fb747fff640 1 -- 172.21.15.112:0/1806137938 learned_addr learned my addr 172.21.15.112:0/1806137938 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:17:22.561 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:21.448+0000 7fb747fff640 1 -- 172.21.15.112:0/1806137938 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb738009450 con 0x7fb748101f90 2024-09-16T00:17:22.561 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:21.449+0000 7fb747fff640 1 --2- 172.21.15.112:0/1806137938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb748101f90 0x7fb748110e10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb738009810 tx=0x7fb7380091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:22.561 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:21.449+0000 7fb744ff9640 1 -- 172.21.15.112:0/1806137938 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb73803c400 con 0x7fb748101f90 2024-09-16T00:17:22.562 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:21.449+0000 7fb744ff9640 1 -- 172.21.15.112:0/1806137938 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fb73803ca00 con 0x7fb748101f90 2024-09-16T00:17:22.562 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:21.449+0000 7fb74ea14640 1 -- 172.21.15.112:0/1806137938 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb748111550 con 0x7fb748101f90 2024-09-16T00:17:22.562 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:21.449+0000 7fb744ff9640 1 -- 172.21.15.112:0/1806137938 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb73803a870 con 0x7fb748101f90 2024-09-16T00:17:22.562 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:21.449+0000 7fb74ea14640 1 -- 172.21.15.112:0/1806137938 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb748111fa0 con 0x7fb748101f90 2024-09-16T00:17:22.562 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:21.450+0000 7fb744ff9640 1 -- 172.21.15.112:0/1806137938 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) v1 ==== 45507+0+0 (secure 0 0 0) 0x7fb73803ab30 con 0x7fb748101f90 2024-09-16T00:17:22.562 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:21.450+0000 7fb744ff9640 1 --2- 172.21.15.112:0/1806137938 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7fb730039490 0x7fb73003b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:22.562 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:21.450+0000 7fb74ea14640 1 -- 172.21.15.112:0/1806137938 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7481013f0 con 0x7fb748101f90 2024-09-16T00:17:22.562 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:21.450+0000 7fb744ff9640 1 -- 172.21.15.112:0/1806137938 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fb73806ced0 con 0x7fb748101f90 2024-09-16T00:17:22.562 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:21.450+0000 7fb7477fe640 1 --2- 172.21.15.112:0/1806137938 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7fb730039490 0x7fb73003b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:22.562 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:21.451+0000 7fb7477fe640 1 --2- 172.21.15.112:0/1806137938 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7fb730039490 0x7fb73003b940 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb73c006fd0 tx=0x7fb73c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:22.562 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:21.454+0000 7fb744ff9640 1 -- 172.21.15.112:0/1806137938 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fb738030570 con 0x7fb748101f90 2024-09-16T00:17:22.562 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:21.640+0000 7fb74ea14640 1 -- 172.21.15.112:0/1806137938 --> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}) v1 -- 0x7fb7480fe9e0 con 0x7fb730039490 2024-09-16T00:17:22.562 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:21.645+0000 7fb744ff9640 1 -- 172.21.15.112:0/1806137938 <== mgr.14118 v2:172.21.15.112:6800/122493397 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7fb7480fe9e0 con 0x7fb730039490 2024-09-16T00:17:22.562 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:21.649+0000 7fb74ea14640 1 -- 172.21.15.112:0/1806137938 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7fb730039490 msgr2=0x7fb73003b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:22.563 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:21.649+0000 7fb74ea14640 1 --2- 172.21.15.112:0/1806137938 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7fb730039490 0x7fb73003b940 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb73c006fd0 tx=0x7fb73c006db0 comp rx=0 tx=0).stop 2024-09-16T00:17:22.563 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:21.649+0000 7fb74ea14640 1 -- 172.21.15.112:0/1806137938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb748101f90 msgr2=0x7fb748110e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:22.563 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:21.649+0000 7fb74ea14640 1 --2- 172.21.15.112:0/1806137938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb748101f90 0x7fb748110e10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb738009810 tx=0x7fb7380091d0 comp rx=0 tx=0).stop 2024-09-16T00:17:22.563 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:21.649+0000 7fb74ea14640 1 -- 172.21.15.112:0/1806137938 shutdown_connections 2024-09-16T00:17:22.563 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:21.649+0000 7fb74ea14640 1 --2- 172.21.15.112:0/1806137938 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7fb730039490 0x7fb73003b940 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:22.563 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:21.649+0000 7fb74ea14640 1 --2- 172.21.15.112:0/1806137938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb748101f90 0x7fb748110e10 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:22.563 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:21.649+0000 7fb74ea14640 1 -- 172.21.15.112:0/1806137938 >> 172.21.15.112:0/1806137938 conn(0x7fb7480fd730 msgr2=0x7fb7480fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:22.563 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:21.649+0000 7fb74ea14640 1 -- 172.21.15.112:0/1806137938 shutdown_connections 2024-09-16T00:17:22.563 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:21.649+0000 7fb74ea14640 1 -- 172.21.15.112:0/1806137938 wait complete. 2024-09-16T00:17:22.563 INFO:teuthology.orchestra.run.smithi112.stdout:Deploying grafana service with default placement... 2024-09-16T00:17:22.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:22 smithi112 ceph-mon[27869]: from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:17:22.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:22 smithi112 ceph-mon[27869]: Saving service prometheus spec with placement count:1 2024-09-16T00:17:24.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:23 smithi112 ceph-mon[27869]: from='mgr.14118 172.21.15.112:0/2162065181' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:17:24.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:23 smithi112 ceph-mon[27869]: from='client.14148 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:17:24.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:23 smithi112 ceph-mon[27869]: Saving service grafana spec with placement count:1 2024-09-16T00:17:24.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:23 smithi112 ceph-mon[27869]: from='mgr.14118 172.21.15.112:0/2162065181' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:17:24.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:23 smithi112 ceph-mon[27869]: from='mgr.14118 172.21.15.112:0/2162065181' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:17:24.149 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout Scheduled grafana update... 2024-09-16T00:17:24.149 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:23.315+0000 7f28a89a7640 1 Processor -- start 2024-09-16T00:17:24.149 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:23.315+0000 7f28a89a7640 1 -- start start 2024-09-16T00:17:24.149 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:23.316+0000 7f28a89a7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28a4101f80 0x7f28a4102390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:24.149 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:23.316+0000 7f28a89a7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a41028d0 con 0x7f28a4101f80 2024-09-16T00:17:24.149 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:23.316+0000 7f28a259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28a4101f80 0x7f28a4102390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:24.150 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:23.316+0000 7f28a259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28a4101f80 0x7f28a4102390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57536/0 (socket says 172.21.15.112:57536) 2024-09-16T00:17:24.150 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:23.316+0000 7f28a259c640 1 -- 172.21.15.112:0/3196841086 learned_addr learned my addr 172.21.15.112:0/3196841086 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:17:24.150 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:23.316+0000 7f28a259c640 1 -- 172.21.15.112:0/3196841086 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28a4102a10 con 0x7f28a4101f80 2024-09-16T00:17:24.150 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:23.317+0000 7f28a259c640 1 --2- 172.21.15.112:0/3196841086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28a4101f80 0x7f28a4102390 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2894009840 tx=0x7f28940091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80d57015f60fb668 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:24.150 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:23.317+0000 7f28a159a640 1 -- 172.21.15.112:0/3196841086 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2894032440 con 0x7f28a4101f80 2024-09-16T00:17:24.150 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:23.317+0000 7f28a159a640 1 -- 172.21.15.112:0/3196841086 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f2894032a40 con 0x7f28a4101f80 2024-09-16T00:17:24.150 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:23.317+0000 7f28a159a640 1 -- 172.21.15.112:0/3196841086 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2894031a70 con 0x7f28a4101f80 2024-09-16T00:17:24.150 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:23.318+0000 7f28a89a7640 1 -- 172.21.15.112:0/3196841086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28a4101f80 msgr2=0x7f28a4102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:24.150 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:23.318+0000 7f28a89a7640 1 --2- 172.21.15.112:0/3196841086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28a4101f80 0x7f28a4102390 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2894009840 tx=0x7f28940091d0 comp rx=0 tx=0).stop 2024-09-16T00:17:24.150 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:23.318+0000 7f28a89a7640 1 -- 172.21.15.112:0/3196841086 shutdown_connections 2024-09-16T00:17:24.150 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:23.318+0000 7f28a89a7640 1 --2- 172.21.15.112:0/3196841086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28a4101f80 0x7f28a4102390 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:24.150 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:23.318+0000 7f28a89a7640 1 -- 172.21.15.112:0/3196841086 >> 172.21.15.112:0/3196841086 conn(0x7f28a40fd7a0 msgr2=0x7f28a40ffbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:24.150 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:23.318+0000 7f28a89a7640 1 -- 172.21.15.112:0/3196841086 shutdown_connections 2024-09-16T00:17:24.151 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:23.318+0000 7f28a89a7640 1 -- 172.21.15.112:0/3196841086 wait complete. 2024-09-16T00:17:24.151 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:23.319+0000 7f28a89a7640 1 Processor -- start 2024-09-16T00:17:24.151 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:23.319+0000 7f28a89a7640 1 -- start start 2024-09-16T00:17:24.151 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:23.319+0000 7f28a89a7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28a4101f80 0x7f28a4110e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:24.151 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:23.319+0000 7f28a89a7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a4111380 con 0x7f28a4101f80 2024-09-16T00:17:24.151 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:23.319+0000 7f28a259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28a4101f80 0x7f28a4110e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:24.151 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:23.319+0000 7f28a259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28a4101f80 0x7f28a4110e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57552/0 (socket says 172.21.15.112:57552) 2024-09-16T00:17:24.151 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:23.319+0000 7f28a259c640 1 -- 172.21.15.112:0/1130723076 learned_addr learned my addr 172.21.15.112:0/1130723076 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:17:24.151 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:23.319+0000 7f28a259c640 1 -- 172.21.15.112:0/1130723076 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2894009450 con 0x7f28a4101f80 2024-09-16T00:17:24.151 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:23.319+0000 7f28a259c640 1 --2- 172.21.15.112:0/1130723076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28a4101f80 0x7f28a4110e40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2894031670 tx=0x7f28940091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:24.151 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:23.319+0000 7f2892ffd640 1 -- 172.21.15.112:0/1130723076 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f289403c400 con 0x7f28a4101f80 2024-09-16T00:17:24.151 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:23.319+0000 7f2892ffd640 1 -- 172.21.15.112:0/1130723076 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f289403ca00 con 0x7f28a4101f80 2024-09-16T00:17:24.151 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:23.319+0000 7f28a89a7640 1 -- 172.21.15.112:0/1130723076 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28a4111580 con 0x7f28a4101f80 2024-09-16T00:17:24.152 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:23.319+0000 7f2892ffd640 1 -- 172.21.15.112:0/1130723076 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f289403a910 con 0x7f28a4101f80 2024-09-16T00:17:24.152 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:23.320+0000 7f28a89a7640 1 -- 172.21.15.112:0/1130723076 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28a4112060 con 0x7f28a4101f80 2024-09-16T00:17:24.152 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:23.320+0000 7f28a89a7640 1 -- 172.21.15.112:0/1130723076 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28a41013e0 con 0x7f28a4101f80 2024-09-16T00:17:24.152 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:23.321+0000 7f2892ffd640 1 -- 172.21.15.112:0/1130723076 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) v1 ==== 45507+0+0 (secure 0 0 0) 0x7f289403aa70 con 0x7f28a4101f80 2024-09-16T00:17:24.152 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:23.321+0000 7f2892ffd640 1 --2- 172.21.15.112:0/1130723076 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7f288c0394e0 0x7f288c03b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:24.152 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:23.321+0000 7f2892ffd640 1 -- 172.21.15.112:0/1130723076 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f289406cd90 con 0x7f28a4101f80 2024-09-16T00:17:24.152 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:23.321+0000 7f28a1d9b640 1 --2- 172.21.15.112:0/1130723076 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7f288c0394e0 0x7f288c03b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:24.152 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:23.321+0000 7f28a1d9b640 1 --2- 172.21.15.112:0/1130723076 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7f288c0394e0 0x7f288c03b990 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2898006fd0 tx=0x7f2898006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:24.152 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:23.326+0000 7f2892ffd640 1 -- 172.21.15.112:0/1130723076 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f2894039ac0 con 0x7f28a4101f80 2024-09-16T00:17:24.152 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:23.498+0000 7f28a89a7640 1 -- 172.21.15.112:0/1130723076 --> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}) v1 -- 0x7f28a40febe0 con 0x7f288c0394e0 2024-09-16T00:17:24.152 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:23.502+0000 7f2892ffd640 1 -- 172.21.15.112:0/1130723076 <== mgr.14118 v2:172.21.15.112:6800/122493397 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f28a40febe0 con 0x7f288c0394e0 2024-09-16T00:17:24.152 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:23.504+0000 7f28a89a7640 1 -- 172.21.15.112:0/1130723076 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7f288c0394e0 msgr2=0x7f288c03b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:24.152 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:23.504+0000 7f28a89a7640 1 --2- 172.21.15.112:0/1130723076 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7f288c0394e0 0x7f288c03b990 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2898006fd0 tx=0x7f2898006db0 comp rx=0 tx=0).stop 2024-09-16T00:17:24.153 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:23.504+0000 7f28a89a7640 1 -- 172.21.15.112:0/1130723076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28a4101f80 msgr2=0x7f28a4110e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:24.153 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:23.504+0000 7f28a89a7640 1 --2- 172.21.15.112:0/1130723076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28a4101f80 0x7f28a4110e40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2894031670 tx=0x7f28940091d0 comp rx=0 tx=0).stop 2024-09-16T00:17:24.153 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:23.504+0000 7f28a89a7640 1 -- 172.21.15.112:0/1130723076 shutdown_connections 2024-09-16T00:17:24.153 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:23.504+0000 7f28a89a7640 1 --2- 172.21.15.112:0/1130723076 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7f288c0394e0 0x7f288c03b990 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:24.153 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:23.504+0000 7f28a89a7640 1 --2- 172.21.15.112:0/1130723076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28a4101f80 0x7f28a4110e40 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:24.153 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:23.504+0000 7f28a89a7640 1 -- 172.21.15.112:0/1130723076 >> 172.21.15.112:0/1130723076 conn(0x7f28a40fd7a0 msgr2=0x7f28a40fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:24.153 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:23.504+0000 7f28a89a7640 1 -- 172.21.15.112:0/1130723076 shutdown_connections 2024-09-16T00:17:24.153 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:23.504+0000 7f28a89a7640 1 -- 172.21.15.112:0/1130723076 wait complete. 2024-09-16T00:17:24.153 INFO:teuthology.orchestra.run.smithi112.stdout:Deploying node-exporter service with default placement... 2024-09-16T00:17:25.391 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout Scheduled node-exporter update... 2024-09-16T00:17:25.391 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:24.545+0000 7f5c3b32d640 1 Processor -- start 2024-09-16T00:17:25.391 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:24.545+0000 7f5c3b32d640 1 -- start start 2024-09-16T00:17:25.392 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:24.545+0000 7f5c3b32d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c34101f90 0x7f5c341023a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:25.392 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:24.545+0000 7f5c3b32d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c341028e0 con 0x7f5c34101f90 2024-09-16T00:17:25.392 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:24.546+0000 7f5c390c9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c34101f90 0x7f5c341023a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:25.392 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:24.546+0000 7f5c390c9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c34101f90 0x7f5c341023a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57568/0 (socket says 172.21.15.112:57568) 2024-09-16T00:17:25.392 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:24.546+0000 7f5c390c9640 1 -- 172.21.15.112:0/1973659190 learned_addr learned my addr 172.21.15.112:0/1973659190 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:17:25.392 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:24.546+0000 7f5c390c9640 1 -- 172.21.15.112:0/1973659190 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c34102a20 con 0x7f5c34101f90 2024-09-16T00:17:25.392 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:24.546+0000 7f5c390c9640 1 --2- 172.21.15.112:0/1973659190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c34101f90 0x7f5c341023a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5c24009e50 tx=0x7f5c24009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1b4bbf5ba6fa886 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:25.392 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:24.547+0000 7f5c33fff640 1 -- 172.21.15.112:0/1973659190 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5c24031560 con 0x7f5c34101f90 2024-09-16T00:17:25.392 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:24.547+0000 7f5c33fff640 1 -- 172.21.15.112:0/1973659190 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f5c2402d050 con 0x7f5c34101f90 2024-09-16T00:17:25.392 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:24.547+0000 7f5c33fff640 1 -- 172.21.15.112:0/1973659190 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5c24007a60 con 0x7f5c34101f90 2024-09-16T00:17:25.392 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:24.547+0000 7f5c3b32d640 1 -- 172.21.15.112:0/1973659190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c34101f90 msgr2=0x7f5c341023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:25.392 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:24.547+0000 7f5c3b32d640 1 --2- 172.21.15.112:0/1973659190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c34101f90 0x7f5c341023a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5c24009e50 tx=0x7f5c24009840 comp rx=0 tx=0).stop 2024-09-16T00:17:25.393 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:24.547+0000 7f5c3b32d640 1 -- 172.21.15.112:0/1973659190 shutdown_connections 2024-09-16T00:17:25.393 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:24.547+0000 7f5c3b32d640 1 --2- 172.21.15.112:0/1973659190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c34101f90 0x7f5c341023a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:25.393 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:24.547+0000 7f5c3b32d640 1 -- 172.21.15.112:0/1973659190 >> 172.21.15.112:0/1973659190 conn(0x7f5c340fd710 msgr2=0x7f5c340ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:25.393 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:24.547+0000 7f5c3b32d640 1 -- 172.21.15.112:0/1973659190 shutdown_connections 2024-09-16T00:17:25.393 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:24.547+0000 7f5c3b32d640 1 -- 172.21.15.112:0/1973659190 wait complete. 2024-09-16T00:17:25.393 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:24.548+0000 7f5c3b32d640 1 Processor -- start 2024-09-16T00:17:25.393 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:24.548+0000 7f5c3b32d640 1 -- start start 2024-09-16T00:17:25.393 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:24.548+0000 7f5c3b32d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c34101f90 0x7f5c34110d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:25.393 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:24.548+0000 7f5c3b32d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c34111260 con 0x7f5c34101f90 2024-09-16T00:17:25.393 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:24.548+0000 7f5c390c9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c34101f90 0x7f5c34110d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:25.393 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:24.548+0000 7f5c390c9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c34101f90 0x7f5c34110d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57582/0 (socket says 172.21.15.112:57582) 2024-09-16T00:17:25.393 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:24.548+0000 7f5c390c9640 1 -- 172.21.15.112:0/80862049 learned_addr learned my addr 172.21.15.112:0/80862049 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:17:25.394 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:24.548+0000 7f5c390c9640 1 -- 172.21.15.112:0/80862049 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c24009ae0 con 0x7f5c34101f90 2024-09-16T00:17:25.394 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:24.548+0000 7f5c390c9640 1 --2- 172.21.15.112:0/80862049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c34101f90 0x7f5c34110d20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5c24009560 tx=0x7f5c24009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:25.394 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:24.549+0000 7f5c31ffb640 1 -- 172.21.15.112:0/80862049 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5c2403b410 con 0x7f5c34101f90 2024-09-16T00:17:25.394 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:24.549+0000 7f5c31ffb640 1 -- 172.21.15.112:0/80862049 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f5c2403ba10 con 0x7f5c34101f90 2024-09-16T00:17:25.394 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:24.549+0000 7f5c3b32d640 1 -- 172.21.15.112:0/80862049 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c34111460 con 0x7f5c34101f90 2024-09-16T00:17:25.394 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:24.549+0000 7f5c31ffb640 1 -- 172.21.15.112:0/80862049 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5c24045710 con 0x7f5c34101f90 2024-09-16T00:17:25.394 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:24.549+0000 7f5c3b32d640 1 -- 172.21.15.112:0/80862049 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c341118d0 con 0x7f5c34101f90 2024-09-16T00:17:25.394 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:24.550+0000 7f5c31ffb640 1 -- 172.21.15.112:0/80862049 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) v1 ==== 45507+0+0 (secure 0 0 0) 0x7f5c24038050 con 0x7f5c34101f90 2024-09-16T00:17:25.394 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:24.550+0000 7f5c3b32d640 1 -- 172.21.15.112:0/80862049 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c3404b720 con 0x7f5c34101f90 2024-09-16T00:17:25.394 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:24.550+0000 7f5c31ffb640 1 --2- 172.21.15.112:0/80862049 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7f5c1c0394e0 0x7f5c1c03b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:25.394 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:24.550+0000 7f5c31ffb640 1 -- 172.21.15.112:0/80862049 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f5c2406cf30 con 0x7f5c34101f90 2024-09-16T00:17:25.394 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:24.550+0000 7f5c388c8640 1 --2- 172.21.15.112:0/80862049 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7f5c1c0394e0 0x7f5c1c03b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:25.394 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:24.550+0000 7f5c388c8640 1 --2- 172.21.15.112:0/80862049 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7f5c1c0394e0 0x7f5c1c03b990 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5c28006fd0 tx=0x7f5c28006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:25.395 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:24.554+0000 7f5c31ffb640 1 -- 172.21.15.112:0/80862049 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f5c2403a370 con 0x7f5c34101f90 2024-09-16T00:17:25.395 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:24.729+0000 7f5c3b32d640 1 -- 172.21.15.112:0/80862049 --> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7f5c341118d0 con 0x7f5c1c0394e0 2024-09-16T00:17:25.395 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:24.733+0000 7f5c31ffb640 1 -- 172.21.15.112:0/80862049 <== mgr.14118 v2:172.21.15.112:6800/122493397 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7f5c341118d0 con 0x7f5c1c0394e0 2024-09-16T00:17:25.395 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:24.735+0000 7f5c3b32d640 1 -- 172.21.15.112:0/80862049 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7f5c1c0394e0 msgr2=0x7f5c1c03b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:25.395 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:24.735+0000 7f5c3b32d640 1 --2- 172.21.15.112:0/80862049 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7f5c1c0394e0 0x7f5c1c03b990 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5c28006fd0 tx=0x7f5c28006db0 comp rx=0 tx=0).stop 2024-09-16T00:17:25.395 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:24.735+0000 7f5c3b32d640 1 -- 172.21.15.112:0/80862049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c34101f90 msgr2=0x7f5c34110d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:25.395 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:24.735+0000 7f5c3b32d640 1 --2- 172.21.15.112:0/80862049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c34101f90 0x7f5c34110d20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5c24009560 tx=0x7f5c24009840 comp rx=0 tx=0).stop 2024-09-16T00:17:25.395 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:24.735+0000 7f5c3b32d640 1 -- 172.21.15.112:0/80862049 shutdown_connections 2024-09-16T00:17:25.395 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:24.735+0000 7f5c3b32d640 1 --2- 172.21.15.112:0/80862049 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7f5c1c0394e0 0x7f5c1c03b990 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:25.395 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:24.735+0000 7f5c3b32d640 1 --2- 172.21.15.112:0/80862049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c34101f90 0x7f5c34110d20 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:25.395 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:24.735+0000 7f5c3b32d640 1 -- 172.21.15.112:0/80862049 >> 172.21.15.112:0/80862049 conn(0x7f5c340fd710 msgr2=0x7f5c340fe260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:25.395 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:24.735+0000 7f5c3b32d640 1 -- 172.21.15.112:0/80862049 shutdown_connections 2024-09-16T00:17:25.395 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:24.735+0000 7f5c3b32d640 1 -- 172.21.15.112:0/80862049 wait complete. 2024-09-16T00:17:25.396 INFO:teuthology.orchestra.run.smithi112.stdout:Deploying alertmanager service with default placement... 2024-09-16T00:17:25.837 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:25 smithi112 ceph-mon[27869]: from='client.14150 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:17:25.837 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:25 smithi112 ceph-mon[27869]: Saving service node-exporter spec with placement * 2024-09-16T00:17:25.838 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:25 smithi112 ceph-mon[27869]: from='mgr.14118 172.21.15.112:0/2162065181' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:17:27.262 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout Scheduled alertmanager update... 2024-09-16T00:17:27.262 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:26.132+0000 7fc690cc6640 1 Processor -- start 2024-09-16T00:17:27.262 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:26.132+0000 7fc690cc6640 1 -- start start 2024-09-16T00:17:27.262 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:26.132+0000 7fc690cc6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc68c100370 0x7fc68c100780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:27.262 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:26.132+0000 7fc690cc6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc68c100cc0 con 0x7fc68c100370 2024-09-16T00:17:27.262 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:26.132+0000 7fc68a59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc68c100370 0x7fc68c100780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:27.262 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:26.132+0000 7fc68a59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc68c100370 0x7fc68c100780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57586/0 (socket says 172.21.15.112:57586) 2024-09-16T00:17:27.263 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:26.132+0000 7fc68a59c640 1 -- 172.21.15.112:0/3854631167 learned_addr learned my addr 172.21.15.112:0/3854631167 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:17:27.263 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:26.132+0000 7fc68a59c640 1 -- 172.21.15.112:0/3854631167 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc68c100e00 con 0x7fc68c100370 2024-09-16T00:17:27.263 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:26.133+0000 7fc68a59c640 1 --2- 172.21.15.112:0/3854631167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc68c100370 0x7fc68c100780 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc67c009840 tx=0x7fc67c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f75ac71ef4263bd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:27.263 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:26.133+0000 7fc68959a640 1 -- 172.21.15.112:0/3854631167 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc67c032440 con 0x7fc68c100370 2024-09-16T00:17:27.263 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:26.133+0000 7fc68959a640 1 -- 172.21.15.112:0/3854631167 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fc67c032a40 con 0x7fc68c100370 2024-09-16T00:17:27.263 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:26.133+0000 7fc68959a640 1 -- 172.21.15.112:0/3854631167 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc67c031a70 con 0x7fc68c100370 2024-09-16T00:17:27.263 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:26.134+0000 7fc690cc6640 1 -- 172.21.15.112:0/3854631167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc68c100370 msgr2=0x7fc68c100780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:27.264 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:26.134+0000 7fc690cc6640 1 --2- 172.21.15.112:0/3854631167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc68c100370 0x7fc68c100780 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc67c009840 tx=0x7fc67c0091d0 comp rx=0 tx=0).stop 2024-09-16T00:17:27.264 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:26.134+0000 7fc690cc6640 1 -- 172.21.15.112:0/3854631167 shutdown_connections 2024-09-16T00:17:27.264 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:26.134+0000 7fc690cc6640 1 --2- 172.21.15.112:0/3854631167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc68c100370 0x7fc68c100780 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:27.264 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:26.134+0000 7fc690cc6640 1 -- 172.21.15.112:0/3854631167 >> 172.21.15.112:0/3854631167 conn(0x7fc68c0fbdd0 msgr2=0x7fc68c0fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:27.264 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:26.134+0000 7fc690cc6640 1 -- 172.21.15.112:0/3854631167 shutdown_connections 2024-09-16T00:17:27.264 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:26.134+0000 7fc690cc6640 1 -- 172.21.15.112:0/3854631167 wait complete. 2024-09-16T00:17:27.264 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:26.134+0000 7fc690cc6640 1 Processor -- start 2024-09-16T00:17:27.264 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:26.135+0000 7fc690cc6640 1 -- start start 2024-09-16T00:17:27.264 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:26.135+0000 7fc690cc6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc68c100370 0x7fc68c110e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:27.264 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:26.135+0000 7fc690cc6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc68c111350 con 0x7fc68c100370 2024-09-16T00:17:27.264 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:26.135+0000 7fc68a59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc68c100370 0x7fc68c110e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:27.264 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:26.135+0000 7fc68a59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc68c100370 0x7fc68c110e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57598/0 (socket says 172.21.15.112:57598) 2024-09-16T00:17:27.265 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:26.135+0000 7fc68a59c640 1 -- 172.21.15.112:0/3376689087 learned_addr learned my addr 172.21.15.112:0/3376689087 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:17:27.265 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:26.135+0000 7fc68a59c640 1 -- 172.21.15.112:0/3376689087 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc67c009450 con 0x7fc68c100370 2024-09-16T00:17:27.265 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:26.135+0000 7fc68a59c640 1 --2- 172.21.15.112:0/3376689087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc68c100370 0x7fc68c110e10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc67c009810 tx=0x7fc67c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:27.265 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:26.135+0000 7fc67affd640 1 -- 172.21.15.112:0/3376689087 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc67c03c400 con 0x7fc68c100370 2024-09-16T00:17:27.265 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:26.135+0000 7fc690cc6640 1 -- 172.21.15.112:0/3376689087 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc68c111550 con 0x7fc68c100370 2024-09-16T00:17:27.265 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:26.135+0000 7fc67affd640 1 -- 172.21.15.112:0/3376689087 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fc67c03ca00 con 0x7fc68c100370 2024-09-16T00:17:27.265 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:26.135+0000 7fc67affd640 1 -- 172.21.15.112:0/3376689087 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc67c03a870 con 0x7fc68c100370 2024-09-16T00:17:27.265 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:26.135+0000 7fc690cc6640 1 -- 172.21.15.112:0/3376689087 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc68c111fa0 con 0x7fc68c100370 2024-09-16T00:17:27.265 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:26.136+0000 7fc67affd640 1 -- 172.21.15.112:0/3376689087 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) v1 ==== 45507+0+0 (secure 0 0 0) 0x7fc67c03cb70 con 0x7fc68c100370 2024-09-16T00:17:27.265 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:26.136+0000 7fc690cc6640 1 -- 172.21.15.112:0/3376689087 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc68c0ffa70 con 0x7fc68c100370 2024-09-16T00:17:27.265 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:26.136+0000 7fc67affd640 1 --2- 172.21.15.112:0/3376689087 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7fc674039490 0x7fc67403b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:27.265 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:26.137+0000 7fc67affd640 1 -- 172.21.15.112:0/3376689087 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fc67c06ce90 con 0x7fc68c100370 2024-09-16T00:17:27.265 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:26.137+0000 7fc689d9b640 1 --2- 172.21.15.112:0/3376689087 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7fc674039490 0x7fc67403b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:27.266 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:26.137+0000 7fc689d9b640 1 --2- 172.21.15.112:0/3376689087 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7fc674039490 0x7fc67403b940 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc680006fd0 tx=0x7fc680006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:27.266 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:26.141+0000 7fc67affd640 1 -- 172.21.15.112:0/3376689087 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fc67c03b960 con 0x7fc68c100370 2024-09-16T00:17:27.266 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:26.307+0000 7fc690cc6640 1 -- 172.21.15.112:0/3376689087 --> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}) v1 -- 0x7fc68c0fd150 con 0x7fc674039490 2024-09-16T00:17:27.266 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:26.312+0000 7fc67affd640 1 -- 172.21.15.112:0/3376689087 <== mgr.14118 v2:172.21.15.112:6800/122493397 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7fc68c0fd150 con 0x7fc674039490 2024-09-16T00:17:27.266 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:26.314+0000 7fc690cc6640 1 -- 172.21.15.112:0/3376689087 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7fc674039490 msgr2=0x7fc67403b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:27.266 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:26.314+0000 7fc690cc6640 1 --2- 172.21.15.112:0/3376689087 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7fc674039490 0x7fc67403b940 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc680006fd0 tx=0x7fc680006db0 comp rx=0 tx=0).stop 2024-09-16T00:17:27.266 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:26.314+0000 7fc690cc6640 1 -- 172.21.15.112:0/3376689087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc68c100370 msgr2=0x7fc68c110e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:27.266 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:26.314+0000 7fc690cc6640 1 --2- 172.21.15.112:0/3376689087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc68c100370 0x7fc68c110e10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc67c009810 tx=0x7fc67c0091d0 comp rx=0 tx=0).stop 2024-09-16T00:17:27.266 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:26.314+0000 7fc690cc6640 1 -- 172.21.15.112:0/3376689087 shutdown_connections 2024-09-16T00:17:27.266 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:26.314+0000 7fc690cc6640 1 --2- 172.21.15.112:0/3376689087 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7fc674039490 0x7fc67403b940 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:27.266 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:26.314+0000 7fc690cc6640 1 --2- 172.21.15.112:0/3376689087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc68c100370 0x7fc68c110e10 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:27.267 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:26.314+0000 7fc690cc6640 1 -- 172.21.15.112:0/3376689087 >> 172.21.15.112:0/3376689087 conn(0x7fc68c0fbdd0 msgr2=0x7fc68c0fc9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:27.267 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:26.314+0000 7fc690cc6640 1 -- 172.21.15.112:0/3376689087 shutdown_connections 2024-09-16T00:17:27.267 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:26.314+0000 7fc690cc6640 1 -- 172.21.15.112:0/3376689087 wait complete. 2024-09-16T00:17:27.531 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:27 smithi112 ceph-mon[27869]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:17:27.532 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:27 smithi112 ceph-mon[27869]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:17:27.532 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:27 smithi112 ceph-mon[27869]: Saving service alertmanager spec with placement count:1 2024-09-16T00:17:27.532 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:27 smithi112 ceph-mon[27869]: from='mgr.14118 172.21.15.112:0/2162065181' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:17:28.460 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:27.725+0000 7f3c530f0640 1 Processor -- start 2024-09-16T00:17:28.461 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:27.725+0000 7f3c530f0640 1 -- start start 2024-09-16T00:17:28.461 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:27.725+0000 7f3c530f0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c4c078b70 0x7f3c4c077230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:28.461 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:27.725+0000 7f3c530f0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4c078f80 con 0x7f3c4c078b70 2024-09-16T00:17:28.461 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:27.725+0000 7f3c50e8c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c4c078b70 0x7f3c4c077230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:28.461 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:27.725+0000 7f3c50e8c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c4c078b70 0x7f3c4c077230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57604/0 (socket says 172.21.15.112:57604) 2024-09-16T00:17:28.461 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:27.725+0000 7f3c50e8c640 1 -- 172.21.15.112:0/1789885685 learned_addr learned my addr 172.21.15.112:0/1789885685 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:17:28.461 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:27.725+0000 7f3c50e8c640 1 -- 172.21.15.112:0/1789885685 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c4c0790c0 con 0x7f3c4c078b70 2024-09-16T00:17:28.461 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:27.726+0000 7f3c50e8c640 1 --2- 172.21.15.112:0/1789885685 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c4c078b70 0x7f3c4c077230 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3c40009840 tx=0x7f3c400091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d5ad65617f1af46 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:28.461 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:27.726+0000 7f3c4b7fe640 1 -- 172.21.15.112:0/1789885685 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3c40032440 con 0x7f3c4c078b70 2024-09-16T00:17:28.461 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:27.726+0000 7f3c4b7fe640 1 -- 172.21.15.112:0/1789885685 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f3c40032a40 con 0x7f3c4c078b70 2024-09-16T00:17:28.461 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:27.726+0000 7f3c4b7fe640 1 -- 172.21.15.112:0/1789885685 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3c40031aa0 con 0x7f3c4c078b70 2024-09-16T00:17:28.461 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:27.727+0000 7f3c530f0640 1 -- 172.21.15.112:0/1789885685 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c4c078b70 msgr2=0x7f3c4c077230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:28.462 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:27.727+0000 7f3c530f0640 1 --2- 172.21.15.112:0/1789885685 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c4c078b70 0x7f3c4c077230 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3c40009840 tx=0x7f3c400091d0 comp rx=0 tx=0).stop 2024-09-16T00:17:28.462 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:27.727+0000 7f3c530f0640 1 -- 172.21.15.112:0/1789885685 shutdown_connections 2024-09-16T00:17:28.462 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:27.727+0000 7f3c530f0640 1 --2- 172.21.15.112:0/1789885685 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c4c078b70 0x7f3c4c077230 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:28.462 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:27.727+0000 7f3c530f0640 1 -- 172.21.15.112:0/1789885685 >> 172.21.15.112:0/1789885685 conn(0x7f3c4c0fd850 msgr2=0x7f3c4c0ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:28.462 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:27.727+0000 7f3c530f0640 1 -- 172.21.15.112:0/1789885685 shutdown_connections 2024-09-16T00:17:28.462 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:27.727+0000 7f3c530f0640 1 -- 172.21.15.112:0/1789885685 wait complete. 2024-09-16T00:17:28.462 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:27.727+0000 7f3c530f0640 1 Processor -- start 2024-09-16T00:17:28.462 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:27.727+0000 7f3c530f0640 1 -- start start 2024-09-16T00:17:28.462 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:27.728+0000 7f3c530f0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c4c078b70 0x7f3c4c112f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:28.462 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:27.728+0000 7f3c530f0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4c113460 con 0x7f3c4c078b70 2024-09-16T00:17:28.462 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:27.728+0000 7f3c50e8c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c4c078b70 0x7f3c4c112f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:28.462 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:27.728+0000 7f3c50e8c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c4c078b70 0x7f3c4c112f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57618/0 (socket says 172.21.15.112:57618) 2024-09-16T00:17:28.462 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:27.728+0000 7f3c50e8c640 1 -- 172.21.15.112:0/419816114 learned_addr learned my addr 172.21.15.112:0/419816114 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:17:28.462 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:27.728+0000 7f3c50e8c640 1 -- 172.21.15.112:0/419816114 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c40009450 con 0x7f3c4c078b70 2024-09-16T00:17:28.462 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:27.728+0000 7f3c50e8c640 1 --2- 172.21.15.112:0/419816114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c4c078b70 0x7f3c4c112f20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3c40031780 tx=0x7f3c400091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:28.463 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:27.728+0000 7f3c497fa640 1 -- 172.21.15.112:0/419816114 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3c400327b0 con 0x7f3c4c078b70 2024-09-16T00:17:28.463 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:27.728+0000 7f3c530f0640 1 -- 172.21.15.112:0/419816114 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c4c1136c0 con 0x7f3c4c078b70 2024-09-16T00:17:28.463 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:27.728+0000 7f3c497fa640 1 -- 172.21.15.112:0/419816114 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f3c4003b8d0 con 0x7f3c4c078b70 2024-09-16T00:17:28.463 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:27.728+0000 7f3c497fa640 1 -- 172.21.15.112:0/419816114 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3c4003a790 con 0x7f3c4c078b70 2024-09-16T00:17:28.463 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:27.728+0000 7f3c530f0640 1 -- 172.21.15.112:0/419816114 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c4c1141f0 con 0x7f3c4c078b70 2024-09-16T00:17:28.463 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:27.729+0000 7f3c497fa640 1 -- 172.21.15.112:0/419816114 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) v1 ==== 45507+0+0 (secure 0 0 0) 0x7f3c40039040 con 0x7f3c4c078b70 2024-09-16T00:17:28.463 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:27.729+0000 7f3c497fa640 1 --2- 172.21.15.112:0/419816114 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7f3c3c0394e0 0x7f3c3c03b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:28.463 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:27.729+0000 7f3c497fa640 1 -- 172.21.15.112:0/419816114 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f3c4006caa0 con 0x7f3c4c078b70 2024-09-16T00:17:28.463 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:27.729+0000 7f3c530f0640 1 -- 172.21.15.112:0/419816114 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c4c1014d0 con 0x7f3c4c078b70 2024-09-16T00:17:28.463 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:27.730+0000 7f3c4bfff640 1 --2- 172.21.15.112:0/419816114 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7f3c3c0394e0 0x7f3c3c03b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:28.463 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:27.730+0000 7f3c4bfff640 1 --2- 172.21.15.112:0/419816114 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7f3c3c0394e0 0x7f3c3c03b990 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3c38006fd0 tx=0x7f3c38006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:28.463 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:27.735+0000 7f3c497fa640 1 -- 172.21.15.112:0/419816114 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f3c4003f020 con 0x7f3c4c078b70 2024-09-16T00:17:28.463 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:27.891+0000 7f3c530f0640 1 -- 172.21.15.112:0/419816114 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f3c4c1144f0 con 0x7f3c4c078b70 2024-09-16T00:17:28.463 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:27.894+0000 7f3c497fa640 1 -- 172.21.15.112:0/419816114 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f3c40037070 con 0x7f3c4c078b70 2024-09-16T00:17:28.464 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:27.897+0000 7f3c530f0640 1 -- 172.21.15.112:0/419816114 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7f3c3c0394e0 msgr2=0x7f3c3c03b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:28.464 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:27.897+0000 7f3c530f0640 1 --2- 172.21.15.112:0/419816114 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7f3c3c0394e0 0x7f3c3c03b990 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3c38006fd0 tx=0x7f3c38006db0 comp rx=0 tx=0).stop 2024-09-16T00:17:28.464 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:27.897+0000 7f3c530f0640 1 -- 172.21.15.112:0/419816114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c4c078b70 msgr2=0x7f3c4c112f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:28.464 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:27.897+0000 7f3c530f0640 1 --2- 172.21.15.112:0/419816114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c4c078b70 0x7f3c4c112f20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3c40031780 tx=0x7f3c400091d0 comp rx=0 tx=0).stop 2024-09-16T00:17:28.464 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:27.897+0000 7f3c530f0640 1 -- 172.21.15.112:0/419816114 shutdown_connections 2024-09-16T00:17:28.464 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:27.897+0000 7f3c530f0640 1 --2- 172.21.15.112:0/419816114 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7f3c3c0394e0 0x7f3c3c03b990 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:28.464 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:27.897+0000 7f3c530f0640 1 --2- 172.21.15.112:0/419816114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c4c078b70 0x7f3c4c112f20 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:28.464 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:27.897+0000 7f3c530f0640 1 -- 172.21.15.112:0/419816114 >> 172.21.15.112:0/419816114 conn(0x7f3c4c0fd850 msgr2=0x7f3c4c0fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:28.464 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:27.897+0000 7f3c530f0640 1 -- 172.21.15.112:0/419816114 shutdown_connections 2024-09-16T00:17:28.464 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:27.897+0000 7f3c530f0640 1 -- 172.21.15.112:0/419816114 wait complete. 2024-09-16T00:17:29.049 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:28 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/419816114' entity='client.admin' 2024-09-16T00:17:29.050 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:28 smithi112 ceph-mon[27869]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:17:29.649 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:28.881+0000 7fd4fed9d640 1 Processor -- start 2024-09-16T00:17:29.649 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:28.881+0000 7fd4fed9d640 1 -- start start 2024-09-16T00:17:29.649 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:28.881+0000 7fd4fed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4f8101f80 0x7fd4f8102390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:29.649 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:28.881+0000 7fd4fed9d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f81028d0 con 0x7fd4f8101f80 2024-09-16T00:17:29.649 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:28.882+0000 7fd4fcb39640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4f8101f80 0x7fd4f8102390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:29.649 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:28.882+0000 7fd4fcb39640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4f8101f80 0x7fd4f8102390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57622/0 (socket says 172.21.15.112:57622) 2024-09-16T00:17:29.649 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:28.882+0000 7fd4fcb39640 1 -- 172.21.15.112:0/3051145220 learned_addr learned my addr 172.21.15.112:0/3051145220 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:17:29.649 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:28.882+0000 7fd4fcb39640 1 -- 172.21.15.112:0/3051145220 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f8102a10 con 0x7fd4f8101f80 2024-09-16T00:17:29.650 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:28.883+0000 7fd4fcb39640 1 --2- 172.21.15.112:0/3051145220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4f8101f80 0x7fd4f8102390 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd4e8009840 tx=0x7fd4e80091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=840d2a67688349c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:29.650 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:28.883+0000 7fd4f77fe640 1 -- 172.21.15.112:0/3051145220 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd4e8032440 con 0x7fd4f8101f80 2024-09-16T00:17:29.650 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:28.883+0000 7fd4f77fe640 1 -- 172.21.15.112:0/3051145220 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fd4e8032a40 con 0x7fd4f8101f80 2024-09-16T00:17:29.650 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:28.883+0000 7fd4f77fe640 1 -- 172.21.15.112:0/3051145220 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd4e8031a70 con 0x7fd4f8101f80 2024-09-16T00:17:29.650 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:28.883+0000 7fd4fed9d640 1 -- 172.21.15.112:0/3051145220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4f8101f80 msgr2=0x7fd4f8102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:29.650 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:28.883+0000 7fd4fed9d640 1 --2- 172.21.15.112:0/3051145220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4f8101f80 0x7fd4f8102390 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd4e8009840 tx=0x7fd4e80091d0 comp rx=0 tx=0).stop 2024-09-16T00:17:29.650 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:28.883+0000 7fd4fed9d640 1 -- 172.21.15.112:0/3051145220 shutdown_connections 2024-09-16T00:17:29.650 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:28.883+0000 7fd4fed9d640 1 --2- 172.21.15.112:0/3051145220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4f8101f80 0x7fd4f8102390 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:29.650 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:28.883+0000 7fd4fed9d640 1 -- 172.21.15.112:0/3051145220 >> 172.21.15.112:0/3051145220 conn(0x7fd4f80fd7a0 msgr2=0x7fd4f80ffbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:29.650 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:28.883+0000 7fd4fed9d640 1 -- 172.21.15.112:0/3051145220 shutdown_connections 2024-09-16T00:17:29.650 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:28.883+0000 7fd4fed9d640 1 -- 172.21.15.112:0/3051145220 wait complete. 2024-09-16T00:17:29.650 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:28.884+0000 7fd4fed9d640 1 Processor -- start 2024-09-16T00:17:29.650 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:28.884+0000 7fd4fed9d640 1 -- start start 2024-09-16T00:17:29.651 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:28.884+0000 7fd4fed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4f8101f80 0x7fd4f8110e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:29.651 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:28.884+0000 7fd4fed9d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f8111360 con 0x7fd4f8101f80 2024-09-16T00:17:29.651 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:28.884+0000 7fd4fcb39640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4f8101f80 0x7fd4f8110e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:29.651 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:28.884+0000 7fd4fcb39640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4f8101f80 0x7fd4f8110e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57638/0 (socket says 172.21.15.112:57638) 2024-09-16T00:17:29.651 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:28.885+0000 7fd4fcb39640 1 -- 172.21.15.112:0/3196107659 learned_addr learned my addr 172.21.15.112:0/3196107659 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:17:29.651 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:28.885+0000 7fd4fcb39640 1 -- 172.21.15.112:0/3196107659 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4e8009450 con 0x7fd4f8101f80 2024-09-16T00:17:29.651 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:28.885+0000 7fd4fcb39640 1 --2- 172.21.15.112:0/3196107659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4f8101f80 0x7fd4f8110e20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd4e8031670 tx=0x7fd4e80091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:29.651 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:28.885+0000 7fd4f57fa640 1 -- 172.21.15.112:0/3196107659 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd4e803c400 con 0x7fd4f8101f80 2024-09-16T00:17:29.651 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:28.885+0000 7fd4f57fa640 1 -- 172.21.15.112:0/3196107659 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fd4e803ca00 con 0x7fd4f8101f80 2024-09-16T00:17:29.651 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:28.885+0000 7fd4fed9d640 1 -- 172.21.15.112:0/3196107659 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4f8111560 con 0x7fd4f8101f80 2024-09-16T00:17:29.651 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:28.885+0000 7fd4f57fa640 1 -- 172.21.15.112:0/3196107659 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd4e803a910 con 0x7fd4f8101f80 2024-09-16T00:17:29.651 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:28.885+0000 7fd4fed9d640 1 -- 172.21.15.112:0/3196107659 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4f8112040 con 0x7fd4f8101f80 2024-09-16T00:17:29.651 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:28.886+0000 7fd4f57fa640 1 -- 172.21.15.112:0/3196107659 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) v1 ==== 45507+0+0 (secure 0 0 0) 0x7fd4e803c560 con 0x7fd4f8101f80 2024-09-16T00:17:29.651 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:28.886+0000 7fd4fed9d640 1 -- 172.21.15.112:0/3196107659 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4f81013e0 con 0x7fd4f8101f80 2024-09-16T00:17:29.652 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:28.886+0000 7fd4f57fa640 1 --2- 172.21.15.112:0/3196107659 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7fd4e0039490 0x7fd4e003b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:29.652 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:28.886+0000 7fd4f57fa640 1 -- 172.21.15.112:0/3196107659 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fd4e806cf10 con 0x7fd4f8101f80 2024-09-16T00:17:29.652 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:28.886+0000 7fd4f7fff640 1 --2- 172.21.15.112:0/3196107659 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7fd4e0039490 0x7fd4e003b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:29.652 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:28.887+0000 7fd4f7fff640 1 --2- 172.21.15.112:0/3196107659 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7fd4e0039490 0x7fd4e003b940 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec00ad90 tx=0x7fd4ec009410 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:29.652 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:28.891+0000 7fd4f57fa640 1 -- 172.21.15.112:0/3196107659 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fd4e8046460 con 0x7fd4f8101f80 2024-09-16T00:17:29.652 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:29.049+0000 7fd4fed9d640 1 -- 172.21.15.112:0/3196107659 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7fd4f804b720 con 0x7fd4f8101f80 2024-09-16T00:17:29.652 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:29.053+0000 7fd4f57fa640 1 -- 172.21.15.112:0/3196107659 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7fd4e8037020 con 0x7fd4f8101f80 2024-09-16T00:17:29.652 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:29.055+0000 7fd4fed9d640 1 -- 172.21.15.112:0/3196107659 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7fd4e0039490 msgr2=0x7fd4e003b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:29.652 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:29.055+0000 7fd4fed9d640 1 --2- 172.21.15.112:0/3196107659 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7fd4e0039490 0x7fd4e003b940 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec00ad90 tx=0x7fd4ec009410 comp rx=0 tx=0).stop 2024-09-16T00:17:29.652 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:29.055+0000 7fd4fed9d640 1 -- 172.21.15.112:0/3196107659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4f8101f80 msgr2=0x7fd4f8110e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:29.652 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:29.055+0000 7fd4fed9d640 1 --2- 172.21.15.112:0/3196107659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4f8101f80 0x7fd4f8110e20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd4e8031670 tx=0x7fd4e80091d0 comp rx=0 tx=0).stop 2024-09-16T00:17:29.652 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:29.055+0000 7fd4fed9d640 1 -- 172.21.15.112:0/3196107659 shutdown_connections 2024-09-16T00:17:29.652 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:29.055+0000 7fd4fed9d640 1 --2- 172.21.15.112:0/3196107659 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7fd4e0039490 0x7fd4e003b940 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:29.653 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:29.055+0000 7fd4fed9d640 1 --2- 172.21.15.112:0/3196107659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4f8101f80 0x7fd4f8110e20 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:29.653 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:29.055+0000 7fd4fed9d640 1 -- 172.21.15.112:0/3196107659 >> 172.21.15.112:0/3196107659 conn(0x7fd4f80fd7a0 msgr2=0x7fd4f80fe320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:29.653 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:29.055+0000 7fd4fed9d640 1 -- 172.21.15.112:0/3196107659 shutdown_connections 2024-09-16T00:17:29.653 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:29.055+0000 7fd4fed9d640 1 -- 172.21.15.112:0/3196107659 wait complete. 2024-09-16T00:17:29.653 INFO:teuthology.orchestra.run.smithi112.stdout:Enabling the dashboard module... 2024-09-16T00:17:30.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:30 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/3196107659' entity='client.admin' 2024-09-16T00:17:31.333 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:31 smithi112 ceph-mon[27869]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:17:31.333 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:31 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/3289339779' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-16T00:17:31.719 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:30.052+0000 7effca2ec640 1 Processor -- start 2024-09-16T00:17:31.719 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:30.052+0000 7effca2ec640 1 -- start start 2024-09-16T00:17:31.719 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:30.052+0000 7effca2ec640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7effc4076ab0 0x7effc4104c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:31.719 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:30.052+0000 7effca2ec640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc4076ec0 con 0x7effc4076ab0 2024-09-16T00:17:31.720 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:30.052+0000 7effc3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7effc4076ab0 0x7effc4104c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:31.720 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:30.053+0000 7effc3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7effc4076ab0 0x7effc4104c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57648/0 (socket says 172.21.15.112:57648) 2024-09-16T00:17:31.720 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:30.053+0000 7effc3fff640 1 -- 172.21.15.112:0/4071928527 learned_addr learned my addr 172.21.15.112:0/4071928527 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:17:31.720 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:30.053+0000 7effc3fff640 1 -- 172.21.15.112:0/4071928527 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effc4105190 con 0x7effc4076ab0 2024-09-16T00:17:31.720 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:30.054+0000 7effc3fff640 1 --2- 172.21.15.112:0/4071928527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7effc4076ab0 0x7effc4104c50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7effb4009840 tx=0x7effb40091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=826228ddeb4af69 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:31.720 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:30.055+0000 7effc2ffd640 1 -- 172.21.15.112:0/4071928527 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7effb4032440 con 0x7effc4076ab0 2024-09-16T00:17:31.720 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:30.055+0000 7effc2ffd640 1 -- 172.21.15.112:0/4071928527 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7effb4032a40 con 0x7effc4076ab0 2024-09-16T00:17:31.721 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:30.055+0000 7effc2ffd640 1 -- 172.21.15.112:0/4071928527 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7effb4031aa0 con 0x7effc4076ab0 2024-09-16T00:17:31.721 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:30.056+0000 7effca2ec640 1 -- 172.21.15.112:0/4071928527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7effc4076ab0 msgr2=0x7effc4104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:31.721 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:30.056+0000 7effca2ec640 1 --2- 172.21.15.112:0/4071928527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7effc4076ab0 0x7effc4104c50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7effb4009840 tx=0x7effb40091d0 comp rx=0 tx=0).stop 2024-09-16T00:17:31.721 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:30.056+0000 7effca2ec640 1 -- 172.21.15.112:0/4071928527 shutdown_connections 2024-09-16T00:17:31.721 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:30.056+0000 7effca2ec640 1 --2- 172.21.15.112:0/4071928527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7effc4076ab0 0x7effc4104c50 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:31.721 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:30.056+0000 7effca2ec640 1 -- 172.21.15.112:0/4071928527 >> 172.21.15.112:0/4071928527 conn(0x7effc40fc700 msgr2=0x7effc40feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:31.721 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:30.056+0000 7effca2ec640 1 -- 172.21.15.112:0/4071928527 shutdown_connections 2024-09-16T00:17:31.721 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:30.056+0000 7effca2ec640 1 -- 172.21.15.112:0/4071928527 wait complete. 2024-09-16T00:17:31.721 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:30.056+0000 7effca2ec640 1 Processor -- start 2024-09-16T00:17:31.721 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:30.057+0000 7effca2ec640 1 -- start start 2024-09-16T00:17:31.721 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:30.057+0000 7effca2ec640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7effc4076ab0 0x7effc4110ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:31.721 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:30.057+0000 7effca2ec640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc41113e0 con 0x7effc4076ab0 2024-09-16T00:17:31.721 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:30.057+0000 7effc3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7effc4076ab0 0x7effc4110ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:31.722 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:30.057+0000 7effc3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7effc4076ab0 0x7effc4110ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57660/0 (socket says 172.21.15.112:57660) 2024-09-16T00:17:31.722 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:30.057+0000 7effc3fff640 1 -- 172.21.15.112:0/3289339779 learned_addr learned my addr 172.21.15.112:0/3289339779 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:17:31.722 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:30.057+0000 7effc3fff640 1 -- 172.21.15.112:0/3289339779 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effb4009450 con 0x7effc4076ab0 2024-09-16T00:17:31.722 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:30.057+0000 7effc3fff640 1 --2- 172.21.15.112:0/3289339779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7effc4076ab0 0x7effc4110ea0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7effb40316a0 tx=0x7effb40091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:31.722 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:30.057+0000 7effc0ff9640 1 -- 172.21.15.112:0/3289339779 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7effb40327b0 con 0x7effc4076ab0 2024-09-16T00:17:31.722 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:30.057+0000 7effc0ff9640 1 -- 172.21.15.112:0/3289339779 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7effb403b8d0 con 0x7effc4076ab0 2024-09-16T00:17:31.722 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:30.057+0000 7effca2ec640 1 -- 172.21.15.112:0/3289339779 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effc41115e0 con 0x7effc4076ab0 2024-09-16T00:17:31.722 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:30.057+0000 7effc0ff9640 1 -- 172.21.15.112:0/3289339779 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7effb403a780 con 0x7effc4076ab0 2024-09-16T00:17:31.722 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:30.058+0000 7effca2ec640 1 -- 172.21.15.112:0/3289339779 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effc4112070 con 0x7effc4076ab0 2024-09-16T00:17:31.722 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:30.058+0000 7effc0ff9640 1 -- 172.21.15.112:0/3289339779 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) v1 ==== 45507+0+0 (secure 0 0 0) 0x7effb4039040 con 0x7effc4076ab0 2024-09-16T00:17:31.722 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:30.058+0000 7effca2ec640 1 -- 172.21.15.112:0/3289339779 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effc4100300 con 0x7effc4076ab0 2024-09-16T00:17:31.722 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:30.058+0000 7effc0ff9640 1 --2- 172.21.15.112:0/3289339779 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7effac039490 0x7effac03b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:31.722 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:30.059+0000 7effc0ff9640 1 -- 172.21.15.112:0/3289339779 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7effb406cab0 con 0x7effc4076ab0 2024-09-16T00:17:31.723 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:30.059+0000 7effc37fe640 1 --2- 172.21.15.112:0/3289339779 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7effac039490 0x7effac03b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:31.723 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:30.059+0000 7effc37fe640 1 --2- 172.21.15.112:0/3289339779 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7effac039490 0x7effac03b940 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7effb8006fd0 tx=0x7effb8006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:31.723 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:30.064+0000 7effc0ff9640 1 -- 172.21.15.112:0/3289339779 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7effb404baa0 con 0x7effc4076ab0 2024-09-16T00:17:31.723 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:30.259+0000 7effca2ec640 1 -- 172.21.15.112:0/3289339779 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7effc404b720 con 0x7effc4076ab0 2024-09-16T00:17:31.723 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:31.059+0000 7effc0ff9640 1 -- 172.21.15.112:0/3289339779 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mgrmap(e 9) v1 ==== 45520+0+0 (secure 0 0 0) 0x7effb404bcd0 con 0x7effc4076ab0 2024-09-16T00:17:31.723 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:31.061+0000 7effc0ff9640 1 -- 172.21.15.112:0/3289339779 <== mon.0 v2:172.21.15.112:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7effb4039d70 con 0x7effc4076ab0 2024-09-16T00:17:31.723 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:31.063+0000 7effca2ec640 1 -- 172.21.15.112:0/3289339779 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7effac039490 msgr2=0x7effac03b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:31.723 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:31.063+0000 7effca2ec640 1 --2- 172.21.15.112:0/3289339779 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7effac039490 0x7effac03b940 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7effb8006fd0 tx=0x7effb8006db0 comp rx=0 tx=0).stop 2024-09-16T00:17:31.723 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:31.064+0000 7effca2ec640 1 -- 172.21.15.112:0/3289339779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7effc4076ab0 msgr2=0x7effc4110ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:31.723 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:31.064+0000 7effca2ec640 1 --2- 172.21.15.112:0/3289339779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7effc4076ab0 0x7effc4110ea0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7effb40316a0 tx=0x7effb40091d0 comp rx=0 tx=0).stop 2024-09-16T00:17:31.723 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:31.064+0000 7effca2ec640 1 -- 172.21.15.112:0/3289339779 shutdown_connections 2024-09-16T00:17:31.723 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:31.064+0000 7effca2ec640 1 --2- 172.21.15.112:0/3289339779 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7effac039490 0x7effac03b940 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:31.724 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:31.064+0000 7effca2ec640 1 --2- 172.21.15.112:0/3289339779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7effc4076ab0 0x7effc4110ea0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:31.724 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:31.064+0000 7effca2ec640 1 -- 172.21.15.112:0/3289339779 >> 172.21.15.112:0/3289339779 conn(0x7effc40fc700 msgr2=0x7effc40fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:31.724 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:31.064+0000 7effca2ec640 1 -- 172.21.15.112:0/3289339779 shutdown_connections 2024-09-16T00:17:31.724 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:31.064+0000 7effca2ec640 1 -- 172.21.15.112:0/3289339779 wait complete. 2024-09-16T00:17:32.169 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:32 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/3289339779' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-16T00:17:32.169 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:32 smithi112 ceph-mon[27869]: mgrmap e9: smithi112.xrkkbc(active, since 25s) 2024-09-16T00:17:33.056 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout { 2024-09-16T00:17:33.057 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-16T00:17:33.057 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-16T00:17:33.057 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "active_name": "smithi112.xrkkbc", 2024-09-16T00:17:33.057 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-16T00:17:33.057 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout } 2024-09-16T00:17:33.057 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:32.168+0000 7f938efe4640 1 Processor -- start 2024-09-16T00:17:33.057 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:32.168+0000 7f938efe4640 1 -- start start 2024-09-16T00:17:33.057 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:32.168+0000 7f938efe4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9388076c80 0x7f9388104ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:33.057 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:32.168+0000 7f938efe4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9388105010 con 0x7f9388076c80 2024-09-16T00:17:33.057 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:32.169+0000 7f938cd80640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9388076c80 0x7f9388104ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:33.057 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:32.169+0000 7f938cd80640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9388076c80 0x7f9388104ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53014/0 (socket says 172.21.15.112:53014) 2024-09-16T00:17:33.061 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:32.169+0000 7f938cd80640 1 -- 172.21.15.112:0/3160312072 learned_addr learned my addr 172.21.15.112:0/3160312072 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:17:33.062 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:32.169+0000 7f938cd80640 1 -- 172.21.15.112:0/3160312072 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9388105150 con 0x7f9388076c80 2024-09-16T00:17:33.062 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:32.170+0000 7f938cd80640 1 --2- 172.21.15.112:0/3160312072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9388076c80 0x7f9388104ad0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f937c009840 tx=0x7f937c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d3e4d9795b7f80f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:33.062 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:32.170+0000 7f93877fe640 1 -- 172.21.15.112:0/3160312072 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f937c032440 con 0x7f9388076c80 2024-09-16T00:17:33.062 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:32.170+0000 7f93877fe640 1 -- 172.21.15.112:0/3160312072 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f937c032a40 con 0x7f9388076c80 2024-09-16T00:17:33.062 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:32.170+0000 7f93877fe640 1 -- 172.21.15.112:0/3160312072 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f937c031a70 con 0x7f9388076c80 2024-09-16T00:17:33.062 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:32.171+0000 7f938efe4640 1 -- 172.21.15.112:0/3160312072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9388076c80 msgr2=0x7f9388104ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:33.062 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:32.171+0000 7f938efe4640 1 --2- 172.21.15.112:0/3160312072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9388076c80 0x7f9388104ad0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f937c009840 tx=0x7f937c0091d0 comp rx=0 tx=0).stop 2024-09-16T00:17:33.062 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:32.171+0000 7f938efe4640 1 -- 172.21.15.112:0/3160312072 shutdown_connections 2024-09-16T00:17:33.062 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:32.171+0000 7f938efe4640 1 --2- 172.21.15.112:0/3160312072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9388076c80 0x7f9388104ad0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:33.062 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:32.171+0000 7f938efe4640 1 -- 172.21.15.112:0/3160312072 >> 172.21.15.112:0/3160312072 conn(0x7f93880fc5c0 msgr2=0x7f93880fe9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:33.062 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:32.171+0000 7f938efe4640 1 -- 172.21.15.112:0/3160312072 shutdown_connections 2024-09-16T00:17:33.062 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:32.171+0000 7f938efe4640 1 -- 172.21.15.112:0/3160312072 wait complete. 2024-09-16T00:17:33.063 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:32.172+0000 7f938efe4640 1 Processor -- start 2024-09-16T00:17:33.063 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:32.172+0000 7f938efe4640 1 -- start start 2024-09-16T00:17:33.063 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:32.172+0000 7f938efe4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9388076c80 0x7f9388110e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:33.063 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:32.172+0000 7f938efe4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9388111360 con 0x7f9388076c80 2024-09-16T00:17:33.063 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:32.172+0000 7f938cd80640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9388076c80 0x7f9388110e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:33.063 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:32.172+0000 7f938cd80640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9388076c80 0x7f9388110e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53018/0 (socket says 172.21.15.112:53018) 2024-09-16T00:17:33.063 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:32.172+0000 7f938cd80640 1 -- 172.21.15.112:0/238748202 learned_addr learned my addr 172.21.15.112:0/238748202 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:17:33.063 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:32.173+0000 7f938cd80640 1 -- 172.21.15.112:0/238748202 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f937c009450 con 0x7f9388076c80 2024-09-16T00:17:33.063 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:32.173+0000 7f938cd80640 1 --2- 172.21.15.112:0/238748202 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9388076c80 0x7f9388110e20 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f937c031750 tx=0x7f937c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:33.063 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:32.173+0000 7f93857fa640 1 -- 172.21.15.112:0/238748202 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f937c03c400 con 0x7f9388076c80 2024-09-16T00:17:33.063 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:32.173+0000 7f93857fa640 1 -- 172.21.15.112:0/238748202 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f937c03ca00 con 0x7f9388076c80 2024-09-16T00:17:33.063 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:32.173+0000 7f938efe4640 1 -- 172.21.15.112:0/238748202 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9388111560 con 0x7f9388076c80 2024-09-16T00:17:33.063 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:32.173+0000 7f93857fa640 1 -- 172.21.15.112:0/238748202 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f937c03a880 con 0x7f9388076c80 2024-09-16T00:17:33.063 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:32.173+0000 7f938efe4640 1 -- 172.21.15.112:0/238748202 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9388111fb0 con 0x7f9388076c80 2024-09-16T00:17:33.064 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:32.174+0000 7f93857fa640 1 -- 172.21.15.112:0/238748202 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) v1 ==== 45520+0+0 (secure 0 0 0) 0x7f937c03a9e0 con 0x7f9388076c80 2024-09-16T00:17:33.064 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:32.174+0000 7f938efe4640 1 -- 172.21.15.112:0/238748202 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f938805ed70 con 0x7f9388076c80 2024-09-16T00:17:33.064 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:32.174+0000 7f93857fa640 1 --2- 172.21.15.112:0/238748202 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7f9378039530 0x7f937803b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:33.064 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:32.174+0000 7f9387fff640 1 -- 172.21.15.112:0/238748202 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7f9378039530 msgr2=0x7f937803b9e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/122493397 2024-09-16T00:17:33.064 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:32.174+0000 7f93857fa640 1 -- 172.21.15.112:0/238748202 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f937c06ce70 con 0x7f9388076c80 2024-09-16T00:17:33.064 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:32.174+0000 7f9387fff640 1 --2- 172.21.15.112:0/238748202 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7f9378039530 0x7f937803b9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T00:17:33.064 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:32.178+0000 7f93857fa640 1 -- 172.21.15.112:0/238748202 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f937c030570 con 0x7f9388076c80 2024-09-16T00:17:33.064 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:32.375+0000 7f9387fff640 1 -- 172.21.15.112:0/238748202 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7f9378039530 msgr2=0x7f937803b9e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/122493397 2024-09-16T00:17:33.064 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:32.375+0000 7f9387fff640 1 --2- 172.21.15.112:0/238748202 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7f9378039530 0x7f937803b9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T00:17:33.064 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:32.382+0000 7f938efe4640 1 -- 172.21.15.112:0/238748202 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f9388112420 con 0x7f9388076c80 2024-09-16T00:17:33.064 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:32.382+0000 7f93857fa640 1 -- 172.21.15.112:0/238748202 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+103 (secure 0 0 0) 0x7f937c037030 con 0x7f9388076c80 2024-09-16T00:17:33.064 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:32.385+0000 7f938efe4640 1 -- 172.21.15.112:0/238748202 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7f9378039530 msgr2=0x7f937803b9e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T00:17:33.064 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:32.385+0000 7f938efe4640 1 --2- 172.21.15.112:0/238748202 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7f9378039530 0x7f937803b9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:33.065 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:32.385+0000 7f938efe4640 1 -- 172.21.15.112:0/238748202 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9388076c80 msgr2=0x7f9388110e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:33.065 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:32.385+0000 7f938efe4640 1 --2- 172.21.15.112:0/238748202 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9388076c80 0x7f9388110e20 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f937c031750 tx=0x7f937c0091d0 comp rx=0 tx=0).stop 2024-09-16T00:17:33.065 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:32.385+0000 7f938efe4640 1 -- 172.21.15.112:0/238748202 shutdown_connections 2024-09-16T00:17:33.065 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:32.385+0000 7f938efe4640 1 --2- 172.21.15.112:0/238748202 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7f9378039530 0x7f937803b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:33.065 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:32.385+0000 7f938efe4640 1 --2- 172.21.15.112:0/238748202 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9388076c80 0x7f9388110e20 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:33.065 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:32.385+0000 7f938efe4640 1 -- 172.21.15.112:0/238748202 >> 172.21.15.112:0/238748202 conn(0x7f93880fc5c0 msgr2=0x7f93880fd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:33.065 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:32.386+0000 7f938efe4640 1 -- 172.21.15.112:0/238748202 shutdown_connections 2024-09-16T00:17:33.065 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:32.386+0000 7f938efe4640 1 -- 172.21.15.112:0/238748202 wait complete. 2024-09-16T00:17:33.065 INFO:teuthology.orchestra.run.smithi112.stdout:Waiting for the mgr to restart... 2024-09-16T00:17:33.065 INFO:teuthology.orchestra.run.smithi112.stdout:Waiting for mgr epoch 9... 2024-09-16T00:17:33.282 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:33 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/238748202' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-16T00:17:35.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:35 smithi112 ceph-mon[27869]: Active manager daemon smithi112.xrkkbc restarted 2024-09-16T00:17:35.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:35 smithi112 ceph-mon[27869]: Activating manager daemon smithi112.xrkkbc 2024-09-16T00:17:35.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:35 smithi112 ceph-mon[27869]: osdmap e3: 0 total, 0 up, 0 in 2024-09-16T00:17:35.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:35 smithi112 ceph-mon[27869]: mgrmap e10: smithi112.xrkkbc(active, starting, since 0.00332434s) 2024-09-16T00:17:35.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:35 smithi112 ceph-mon[27869]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mon metadata", "id": "smithi112"}]: dispatch 2024-09-16T00:17:35.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:35 smithi112 ceph-mon[27869]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mgr metadata", "who": "smithi112.xrkkbc", "id": "smithi112.xrkkbc"}]: dispatch 2024-09-16T00:17:35.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:35 smithi112 ceph-mon[27869]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T00:17:35.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:35 smithi112 ceph-mon[27869]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T00:17:35.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:35 smithi112 ceph-mon[27869]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T00:17:35.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:35 smithi112 ceph-mon[27869]: Manager daemon smithi112.xrkkbc is now available 2024-09-16T00:17:35.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:35 smithi112 ceph-mon[27869]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:17:35.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:35 smithi112 ceph-mon[27869]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi112.xrkkbc/mirror_snapshot_schedule"}]: dispatch 2024-09-16T00:17:35.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:35 smithi112 ceph-mon[27869]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi112.xrkkbc/trash_purge_schedule"}]: dispatch 2024-09-16T00:17:37.491 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout { 2024-09-16T00:17:37.491 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-16T00:17:37.491 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-16T00:17:37.491 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout } 2024-09-16T00:17:37.492 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:33.466+0000 7f13dfcf9640 1 Processor -- start 2024-09-16T00:17:37.492 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:33.466+0000 7f13dfcf9640 1 -- start start 2024-09-16T00:17:37.492 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:33.466+0000 7f13dfcf9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13d8078c80 0x7f13d8077250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:37.492 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:33.466+0000 7f13dfcf9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d8079090 con 0x7f13d8078c80 2024-09-16T00:17:37.492 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:33.466+0000 7f13dda95640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13d8078c80 0x7f13d8077250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:37.492 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:33.466+0000 7f13dda95640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13d8078c80 0x7f13d8077250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53034/0 (socket says 172.21.15.112:53034) 2024-09-16T00:17:37.492 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:33.466+0000 7f13dda95640 1 -- 172.21.15.112:0/1662132888 learned_addr learned my addr 172.21.15.112:0/1662132888 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:17:37.492 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:33.466+0000 7f13dda95640 1 -- 172.21.15.112:0/1662132888 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d8077790 con 0x7f13d8078c80 2024-09-16T00:17:37.492 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:33.467+0000 7f13dda95640 1 --2- 172.21.15.112:0/1662132888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13d8078c80 0x7f13d8077250 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f13d4009840 tx=0x7f13d40091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=885d2d2d990faf12 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:37.492 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:33.467+0000 7f13dca93640 1 -- 172.21.15.112:0/1662132888 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f13d4032440 con 0x7f13d8078c80 2024-09-16T00:17:37.492 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:33.467+0000 7f13dca93640 1 -- 172.21.15.112:0/1662132888 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f13d4032a40 con 0x7f13d8078c80 2024-09-16T00:17:37.492 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:33.467+0000 7f13dca93640 1 -- 172.21.15.112:0/1662132888 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f13d4031a70 con 0x7f13d8078c80 2024-09-16T00:17:37.492 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:33.468+0000 7f13dfcf9640 1 -- 172.21.15.112:0/1662132888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13d8078c80 msgr2=0x7f13d8077250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:37.493 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:33.468+0000 7f13dfcf9640 1 --2- 172.21.15.112:0/1662132888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13d8078c80 0x7f13d8077250 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f13d4009840 tx=0x7f13d40091d0 comp rx=0 tx=0).stop 2024-09-16T00:17:37.493 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:33.468+0000 7f13dfcf9640 1 -- 172.21.15.112:0/1662132888 shutdown_connections 2024-09-16T00:17:37.493 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:33.468+0000 7f13dfcf9640 1 --2- 172.21.15.112:0/1662132888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13d8078c80 0x7f13d8077250 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:37.493 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:33.468+0000 7f13dfcf9640 1 -- 172.21.15.112:0/1662132888 >> 172.21.15.112:0/1662132888 conn(0x7f13d80fd7a0 msgr2=0x7f13d80ffbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:37.493 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:33.468+0000 7f13dfcf9640 1 -- 172.21.15.112:0/1662132888 shutdown_connections 2024-09-16T00:17:37.493 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:33.468+0000 7f13dfcf9640 1 -- 172.21.15.112:0/1662132888 wait complete. 2024-09-16T00:17:37.493 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:33.469+0000 7f13dfcf9640 1 Processor -- start 2024-09-16T00:17:37.493 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:33.469+0000 7f13dfcf9640 1 -- start start 2024-09-16T00:17:37.493 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:33.469+0000 7f13dfcf9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13d8078c80 0x7f13d8113040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:37.493 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:33.469+0000 7f13dfcf9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d8113580 con 0x7f13d8078c80 2024-09-16T00:17:37.493 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:33.469+0000 7f13dda95640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13d8078c80 0x7f13d8113040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:37.493 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:33.469+0000 7f13dda95640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13d8078c80 0x7f13d8113040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53046/0 (socket says 172.21.15.112:53046) 2024-09-16T00:17:37.493 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:33.469+0000 7f13dda95640 1 -- 172.21.15.112:0/2906337289 learned_addr learned my addr 172.21.15.112:0/2906337289 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:17:37.493 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:33.470+0000 7f13dda95640 1 -- 172.21.15.112:0/2906337289 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d4009450 con 0x7f13d8078c80 2024-09-16T00:17:37.493 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:33.470+0000 7f13dda95640 1 --2- 172.21.15.112:0/2906337289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13d8078c80 0x7f13d8113040 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f13d4031670 tx=0x7f13d40091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:37.494 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:33.470+0000 7f13ce7fc640 1 -- 172.21.15.112:0/2906337289 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f13d4031e90 con 0x7f13d8078c80 2024-09-16T00:17:37.494 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:33.470+0000 7f13ce7fc640 1 -- 172.21.15.112:0/2906337289 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f13d403c8a0 con 0x7f13d8078c80 2024-09-16T00:17:37.494 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:33.470+0000 7f13ce7fc640 1 -- 172.21.15.112:0/2906337289 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f13d403a820 con 0x7f13d8078c80 2024-09-16T00:17:37.494 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:33.470+0000 7f13dfcf9640 1 -- 172.21.15.112:0/2906337289 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13d8113780 con 0x7f13d8078c80 2024-09-16T00:17:37.494 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:33.470+0000 7f13dfcf9640 1 -- 172.21.15.112:0/2906337289 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13d8113bf0 con 0x7f13d8078c80 2024-09-16T00:17:37.494 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:33.471+0000 7f13ce7fc640 1 -- 172.21.15.112:0/2906337289 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) v1 ==== 45520+0+0 (secure 0 0 0) 0x7f13d403ca10 con 0x7f13d8078c80 2024-09-16T00:17:37.494 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:33.471+0000 7f13ce7fc640 1 --2- 172.21.15.112:0/2906337289 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7f13c4039530 0x7f13c403b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:37.494 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:33.471+0000 7f13ce7fc640 1 -- 172.21.15.112:0/2906337289 --> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f13c403c0f0 con 0x7f13c4039530 2024-09-16T00:17:37.494 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:33.471+0000 7f13ce7fc640 1 -- 172.21.15.112:0/2906337289 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f13d406cc00 con 0x7f13d8078c80 2024-09-16T00:17:37.494 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:33.471+0000 7f13dd294640 1 -- 172.21.15.112:0/2906337289 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7f13c4039530 msgr2=0x7f13c403b9e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/122493397 2024-09-16T00:17:37.494 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:33.471+0000 7f13dd294640 1 --2- 172.21.15.112:0/2906337289 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7f13c4039530 0x7f13c403b9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T00:17:37.494 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:33.672+0000 7f13dd294640 1 -- 172.21.15.112:0/2906337289 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7f13c4039530 msgr2=0x7f13c403b9e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/122493397 2024-09-16T00:17:37.494 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:33.672+0000 7f13dd294640 1 --2- 172.21.15.112:0/2906337289 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7f13c4039530 0x7f13c403b9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T00:17:37.495 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:34.073+0000 7f13dd294640 1 -- 172.21.15.112:0/2906337289 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7f13c4039530 msgr2=0x7f13c403b9e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/122493397 2024-09-16T00:17:37.495 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:34.073+0000 7f13dd294640 1 --2- 172.21.15.112:0/2906337289 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7f13c4039530 0x7f13c403b9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T00:17:37.495 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:34.874+0000 7f13dd294640 1 -- 172.21.15.112:0/2906337289 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7f13c4039530 msgr2=0x7f13c403b9e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/122493397 2024-09-16T00:17:37.495 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:34.874+0000 7f13dd294640 1 --2- 172.21.15.112:0/2906337289 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7f13c4039530 0x7f13c403b9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T00:17:37.495 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:35.478+0000 7f13ce7fc640 1 -- 172.21.15.112:0/2906337289 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mgrmap(e 10) v1 ==== 45287+0+0 (secure 0 0 0) 0x7f13d406bda0 con 0x7f13d8078c80 2024-09-16T00:17:37.495 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:35.479+0000 7f13ce7fc640 1 -- 172.21.15.112:0/2906337289 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7f13c4039530 msgr2=0x7f13c403b9e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T00:17:37.495 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:35.479+0000 7f13ce7fc640 1 --2- 172.21.15.112:0/2906337289 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7f13c4039530 0x7f13c403b9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:37.495 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:36.480+0000 7f13ce7fc640 1 -- 172.21.15.112:0/2906337289 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mgrmap(e 11) v1 ==== 45414+0+0 (secure 0 0 0) 0x7f13d4030d10 con 0x7f13d8078c80 2024-09-16T00:17:37.495 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:36.480+0000 7f13ce7fc640 1 --2- 172.21.15.112:0/2906337289 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f13c403cfd0 0x7f13c403f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:37.495 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:36.480+0000 7f13ce7fc640 1 -- 172.21.15.112:0/2906337289 --> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f13c403c0f0 con 0x7f13c403cfd0 2024-09-16T00:17:37.495 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:36.480+0000 7f13dd294640 1 --2- 172.21.15.112:0/2906337289 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f13c403cfd0 0x7f13c403f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:37.495 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:36.481+0000 7f13dd294640 1 --2- 172.21.15.112:0/2906337289 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f13c403cfd0 0x7f13c403f3b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f13c8003940 tx=0x7f13c8009350 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:37.495 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:36.482+0000 7f13ce7fc640 1 -- 172.21.15.112:0/2906337289 <== mgr.14160 v2:172.21.15.112:6800/2021251488 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (secure 0 0 0) 0x7f13c403c0f0 con 0x7f13c403cfd0 2024-09-16T00:17:37.495 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:36.487+0000 7f13dfcf9640 1 -- 172.21.15.112:0/2906337289 --> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f13d8113bf0 con 0x7f13c403cfd0 2024-09-16T00:17:37.495 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:36.487+0000 7f13ce7fc640 1 -- 172.21.15.112:0/2906337289 <== mgr.14160 v2:172.21.15.112:6800/2021251488 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f13d8113bf0 con 0x7f13c403cfd0 2024-09-16T00:17:37.496 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:36.487+0000 7f13dfcf9640 1 -- 172.21.15.112:0/2906337289 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f13c403cfd0 msgr2=0x7f13c403f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:37.496 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:36.487+0000 7f13dfcf9640 1 --2- 172.21.15.112:0/2906337289 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f13c403cfd0 0x7f13c403f3b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f13c8003940 tx=0x7f13c8009350 comp rx=0 tx=0).stop 2024-09-16T00:17:37.496 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:36.487+0000 7f13dfcf9640 1 -- 172.21.15.112:0/2906337289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13d8078c80 msgr2=0x7f13d8113040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:37.496 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:36.487+0000 7f13dfcf9640 1 --2- 172.21.15.112:0/2906337289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13d8078c80 0x7f13d8113040 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f13d4031670 tx=0x7f13d40091d0 comp rx=0 tx=0).stop 2024-09-16T00:17:37.496 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:36.487+0000 7f13dfcf9640 1 -- 172.21.15.112:0/2906337289 shutdown_connections 2024-09-16T00:17:37.496 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:36.487+0000 7f13dfcf9640 1 --2- 172.21.15.112:0/2906337289 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f13c403cfd0 0x7f13c403f3b0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:37.496 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:36.487+0000 7f13dfcf9640 1 --2- 172.21.15.112:0/2906337289 >> [v2:172.21.15.112:6800/122493397,v1:172.21.15.112:6801/122493397] conn(0x7f13c4039530 0x7f13c403b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:37.496 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:36.487+0000 7f13dfcf9640 1 --2- 172.21.15.112:0/2906337289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13d8078c80 0x7f13d8113040 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:37.496 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:36.487+0000 7f13dfcf9640 1 -- 172.21.15.112:0/2906337289 >> 172.21.15.112:0/2906337289 conn(0x7f13d80fd7a0 msgr2=0x7f13d80fe3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:37.496 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:36.488+0000 7f13dfcf9640 1 -- 172.21.15.112:0/2906337289 shutdown_connections 2024-09-16T00:17:37.496 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:36.488+0000 7f13dfcf9640 1 -- 172.21.15.112:0/2906337289 wait complete. 2024-09-16T00:17:37.496 INFO:teuthology.orchestra.run.smithi112.stdout:mgr epoch 9 is available 2024-09-16T00:17:37.496 INFO:teuthology.orchestra.run.smithi112.stdout:Generating a dashboard self-signed certificate... 2024-09-16T00:17:37.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:37 smithi112 ceph-mon[27869]: mgrmap e11: smithi112.xrkkbc(active, since 1.00576s) 2024-09-16T00:17:37.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:37 smithi112 ceph-mon[27869]: [16/Sep/2024:00:17:36] ENGINE Bus STARTING 2024-09-16T00:17:37.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:37 smithi112 ceph-mon[27869]: [16/Sep/2024:00:17:36] ENGINE Serving on https://172.21.15.112:7150 2024-09-16T00:17:37.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:37 smithi112 ceph-mon[27869]: [16/Sep/2024:00:17:36] ENGINE Bus STARTED 2024-09-16T00:17:37.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:37 smithi112 ceph-mon[27869]: [16/Sep/2024:00:17:36] ENGINE Error in HTTPServer.serve 2024-09-16T00:17:37.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: Traceback (most recent call last): 2024-09-16T00:17:37.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T00:17:37.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: self._connections.run(self.expiration_interval) 2024-09-16T00:17:37.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T00:17:37.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: self._run(expiration_interval) 2024-09-16T00:17:37.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T00:17:37.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T00:17:37.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T00:17:37.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T00:17:37.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T00:17:37.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: s = self.context.wrap_socket( 2024-09-16T00:17:37.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T00:17:37.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: return self.sslsocket_class._create( 2024-09-16T00:17:37.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T00:17:37.591 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: self.do_handshake() 2024-09-16T00:17:37.591 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T00:17:37.591 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: self._sslobj.do_handshake() 2024-09-16T00:17:37.591 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T00:17:37.591 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:37 smithi112 ceph-mon[27869]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:17:38.813 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-16T00:17:38.814 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:37.934+0000 7f3e19ee8640 1 Processor -- start 2024-09-16T00:17:38.814 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:37.934+0000 7f3e19ee8640 1 -- start start 2024-09-16T00:17:38.814 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:37.934+0000 7f3e19ee8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e14078b90 0x7f3e14077250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:38.814 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:37.934+0000 7f3e19ee8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e14078fa0 con 0x7f3e14078b90 2024-09-16T00:17:38.814 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:37.934+0000 7f3e137fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e14078b90 0x7f3e14077250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:38.814 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:37.934+0000 7f3e137fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e14078b90 0x7f3e14077250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53134/0 (socket says 172.21.15.112:53134) 2024-09-16T00:17:38.814 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:37.934+0000 7f3e137fe640 1 -- 172.21.15.112:0/2162705379 learned_addr learned my addr 172.21.15.112:0/2162705379 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:17:38.814 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:37.934+0000 7f3e137fe640 1 -- 172.21.15.112:0/2162705379 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e140790e0 con 0x7f3e14078b90 2024-09-16T00:17:38.814 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:37.935+0000 7f3e137fe640 1 --2- 172.21.15.112:0/2162705379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e14078b90 0x7f3e14077250 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3e04009840 tx=0x7f3e040091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a73e1e0ca157cdda server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:38.814 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:37.935+0000 7f3e127fc640 1 -- 172.21.15.112:0/2162705379 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3e04032440 con 0x7f3e14078b90 2024-09-16T00:17:38.814 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:37.935+0000 7f3e127fc640 1 -- 172.21.15.112:0/2162705379 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f3e04032a40 con 0x7f3e14078b90 2024-09-16T00:17:38.815 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:37.935+0000 7f3e127fc640 1 -- 172.21.15.112:0/2162705379 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3e04031aa0 con 0x7f3e14078b90 2024-09-16T00:17:38.815 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:37.936+0000 7f3e19ee8640 1 -- 172.21.15.112:0/2162705379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e14078b90 msgr2=0x7f3e14077250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:38.815 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:37.936+0000 7f3e19ee8640 1 --2- 172.21.15.112:0/2162705379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e14078b90 0x7f3e14077250 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3e04009840 tx=0x7f3e040091d0 comp rx=0 tx=0).stop 2024-09-16T00:17:38.815 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:37.936+0000 7f3e19ee8640 1 -- 172.21.15.112:0/2162705379 shutdown_connections 2024-09-16T00:17:38.815 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:37.936+0000 7f3e19ee8640 1 --2- 172.21.15.112:0/2162705379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e14078b90 0x7f3e14077250 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:38.815 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:37.936+0000 7f3e19ee8640 1 -- 172.21.15.112:0/2162705379 >> 172.21.15.112:0/2162705379 conn(0x7f3e140fd850 msgr2=0x7f3e140ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:38.815 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:37.936+0000 7f3e19ee8640 1 -- 172.21.15.112:0/2162705379 shutdown_connections 2024-09-16T00:17:38.815 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:37.936+0000 7f3e19ee8640 1 -- 172.21.15.112:0/2162705379 wait complete. 2024-09-16T00:17:38.815 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:37.937+0000 7f3e19ee8640 1 Processor -- start 2024-09-16T00:17:38.815 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:37.937+0000 7f3e19ee8640 1 -- start start 2024-09-16T00:17:38.815 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:37.937+0000 7f3e19ee8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e14078b90 0x7f3e14112f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:38.815 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:37.937+0000 7f3e19ee8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e141134c0 con 0x7f3e14078b90 2024-09-16T00:17:38.815 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:37.937+0000 7f3e137fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e14078b90 0x7f3e14112f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:38.815 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:37.937+0000 7f3e137fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e14078b90 0x7f3e14112f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53142/0 (socket says 172.21.15.112:53142) 2024-09-16T00:17:38.816 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:37.937+0000 7f3e137fe640 1 -- 172.21.15.112:0/3103995374 learned_addr learned my addr 172.21.15.112:0/3103995374 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:17:38.816 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:37.937+0000 7f3e137fe640 1 -- 172.21.15.112:0/3103995374 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e04009450 con 0x7f3e14078b90 2024-09-16T00:17:38.816 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:37.937+0000 7f3e137fe640 1 --2- 172.21.15.112:0/3103995374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e14078b90 0x7f3e14112f80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3e04031780 tx=0x7f3e040091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:38.816 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:37.938+0000 7f3e0bfff640 1 -- 172.21.15.112:0/3103995374 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3e040327b0 con 0x7f3e14078b90 2024-09-16T00:17:38.816 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:37.938+0000 7f3e19ee8640 1 -- 172.21.15.112:0/3103995374 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e141136c0 con 0x7f3e14078b90 2024-09-16T00:17:38.816 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:37.938+0000 7f3e0bfff640 1 -- 172.21.15.112:0/3103995374 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f3e0403b8d0 con 0x7f3e14078b90 2024-09-16T00:17:38.816 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:37.938+0000 7f3e0bfff640 1 -- 172.21.15.112:0/3103995374 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3e0403a790 con 0x7f3e14078b90 2024-09-16T00:17:38.816 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:37.938+0000 7f3e19ee8640 1 -- 172.21.15.112:0/3103995374 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e14113b30 con 0x7f3e14078b90 2024-09-16T00:17:38.816 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:37.938+0000 7f3e0bfff640 1 -- 172.21.15.112:0/3103995374 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 45520+0+0 (secure 0 0 0) 0x7f3e04039040 con 0x7f3e14078b90 2024-09-16T00:17:38.816 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:37.939+0000 7f3e0bfff640 1 --2- 172.21.15.112:0/3103995374 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f3e000394e0 0x7f3e0003b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:38.816 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:37.939+0000 7f3e0bfff640 1 -- 172.21.15.112:0/3103995374 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f3e0406ca70 con 0x7f3e14078b90 2024-09-16T00:17:38.816 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:37.939+0000 7f3e12ffd640 1 --2- 172.21.15.112:0/3103995374 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f3e000394e0 0x7f3e0003b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:38.817 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:37.939+0000 7f3e19ee8640 1 -- 172.21.15.112:0/3103995374 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e141014d0 con 0x7f3e14078b90 2024-09-16T00:17:38.817 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:37.939+0000 7f3e12ffd640 1 --2- 172.21.15.112:0/3103995374 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f3e000394e0 0x7f3e0003b990 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc006fd0 tx=0x7f3dfc006db0 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:38.817 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:37.943+0000 7f3e0bfff640 1 -- 172.21.15.112:0/3103995374 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f3e0403f020 con 0x7f3e14078b90 2024-09-16T00:17:38.817 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:38.110+0000 7f3e19ee8640 1 -- 172.21.15.112:0/3103995374 --> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f3e140febc0 con 0x7f3e000394e0 2024-09-16T00:17:38.817 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:38.196+0000 7f3e0bfff640 1 -- 172.21.15.112:0/3103995374 <== mgr.14160 v2:172.21.15.112:6800/2021251488 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f3e140febc0 con 0x7f3e000394e0 2024-09-16T00:17:38.817 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:38.198+0000 7f3e19ee8640 1 -- 172.21.15.112:0/3103995374 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f3e000394e0 msgr2=0x7f3e0003b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:38.817 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:38.198+0000 7f3e19ee8640 1 --2- 172.21.15.112:0/3103995374 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f3e000394e0 0x7f3e0003b990 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc006fd0 tx=0x7f3dfc006db0 comp rx=0 tx=0).stop 2024-09-16T00:17:38.817 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:38.198+0000 7f3e19ee8640 1 -- 172.21.15.112:0/3103995374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e14078b90 msgr2=0x7f3e14112f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:38.817 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:38.198+0000 7f3e19ee8640 1 --2- 172.21.15.112:0/3103995374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e14078b90 0x7f3e14112f80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3e04031780 tx=0x7f3e040091d0 comp rx=0 tx=0).stop 2024-09-16T00:17:38.817 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:38.198+0000 7f3e19ee8640 1 -- 172.21.15.112:0/3103995374 shutdown_connections 2024-09-16T00:17:38.817 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:38.198+0000 7f3e19ee8640 1 --2- 172.21.15.112:0/3103995374 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f3e000394e0 0x7f3e0003b990 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:38.817 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:38.198+0000 7f3e19ee8640 1 --2- 172.21.15.112:0/3103995374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e14078b90 0x7f3e14112f80 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:38.817 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:38.198+0000 7f3e19ee8640 1 -- 172.21.15.112:0/3103995374 >> 172.21.15.112:0/3103995374 conn(0x7f3e140fd850 msgr2=0x7f3e140fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:38.817 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:38.199+0000 7f3e19ee8640 1 -- 172.21.15.112:0/3103995374 shutdown_connections 2024-09-16T00:17:38.818 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:38.199+0000 7f3e19ee8640 1 -- 172.21.15.112:0/3103995374 wait complete. 2024-09-16T00:17:38.818 INFO:teuthology.orchestra.run.smithi112.stdout:Creating initial admin user... 2024-09-16T00:17:38.932 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:38 smithi112 ceph-mon[27869]: mgrmap e12: smithi112.xrkkbc(active, since 2s) 2024-09-16T00:17:38.932 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:38 smithi112 ceph-mon[27869]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:17:38.932 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:38 smithi112 ceph-mon[27869]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:17:38.932 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:38 smithi112 ceph-mon[27869]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:17:38.932 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:38 smithi112 ceph-mon[27869]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:17:40.215 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:39 smithi112 ceph-mon[27869]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:17:40.215 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:39 smithi112 ceph-mon[27869]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:17:40.224 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$3wp1R.CpX/TiHL/Xg4u6HuBpmnDPRxZSuqf1DquVoxkHWkTLyaO2S", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726445859, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-16T00:17:40.224 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:39.219+0000 7fa6af4a2640 1 Processor -- start 2024-09-16T00:17:40.225 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:39.219+0000 7fa6af4a2640 1 -- start start 2024-09-16T00:17:40.225 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:39.219+0000 7fa6af4a2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6a8076af0 0x7fa6a8104c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:40.225 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:39.219+0000 7fa6af4a2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a8076f00 con 0x7fa6a8076af0 2024-09-16T00:17:40.225 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:39.219+0000 7fa6ad23e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6a8076af0 0x7fa6a8104c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:40.225 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:39.219+0000 7fa6ad23e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6a8076af0 0x7fa6a8104c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53144/0 (socket says 172.21.15.112:53144) 2024-09-16T00:17:40.225 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:39.219+0000 7fa6ad23e640 1 -- 172.21.15.112:0/174822948 learned_addr learned my addr 172.21.15.112:0/174822948 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:17:40.225 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:39.220+0000 7fa6ad23e640 1 -- 172.21.15.112:0/174822948 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6a81051c0 con 0x7fa6a8076af0 2024-09-16T00:17:40.225 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:39.220+0000 7fa6ad23e640 1 --2- 172.21.15.112:0/174822948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6a8076af0 0x7fa6a8104c80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4009840 tx=0x7fa6a40091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e25a291e911e38a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:40.225 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:39.220+0000 7fa69ffff640 1 -- 172.21.15.112:0/174822948 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa6a4032440 con 0x7fa6a8076af0 2024-09-16T00:17:40.226 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:39.220+0000 7fa69ffff640 1 -- 172.21.15.112:0/174822948 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fa6a4032a40 con 0x7fa6a8076af0 2024-09-16T00:17:40.226 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:39.220+0000 7fa69ffff640 1 -- 172.21.15.112:0/174822948 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa6a4031aa0 con 0x7fa6a8076af0 2024-09-16T00:17:40.226 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:39.221+0000 7fa6af4a2640 1 -- 172.21.15.112:0/174822948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6a8076af0 msgr2=0x7fa6a8104c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:40.226 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:39.221+0000 7fa6af4a2640 1 --2- 172.21.15.112:0/174822948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6a8076af0 0x7fa6a8104c80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4009840 tx=0x7fa6a40091d0 comp rx=0 tx=0).stop 2024-09-16T00:17:40.226 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:39.221+0000 7fa6af4a2640 1 -- 172.21.15.112:0/174822948 shutdown_connections 2024-09-16T00:17:40.226 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:39.221+0000 7fa6af4a2640 1 --2- 172.21.15.112:0/174822948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6a8076af0 0x7fa6a8104c80 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:40.226 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:39.221+0000 7fa6af4a2640 1 -- 172.21.15.112:0/174822948 >> 172.21.15.112:0/174822948 conn(0x7fa6a80fc790 msgr2=0x7fa6a80feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:40.227 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:39.221+0000 7fa6af4a2640 1 -- 172.21.15.112:0/174822948 shutdown_connections 2024-09-16T00:17:40.227 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:39.221+0000 7fa6af4a2640 1 -- 172.21.15.112:0/174822948 wait complete. 2024-09-16T00:17:40.227 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:39.222+0000 7fa6af4a2640 1 Processor -- start 2024-09-16T00:17:40.227 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:39.222+0000 7fa6af4a2640 1 -- start start 2024-09-16T00:17:40.227 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:39.222+0000 7fa6af4a2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6a8076af0 0x7fa6a81025b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:40.227 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:39.222+0000 7fa6af4a2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a8102af0 con 0x7fa6a8076af0 2024-09-16T00:17:40.227 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:39.222+0000 7fa6ad23e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6a8076af0 0x7fa6a81025b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:40.228 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:39.222+0000 7fa6ad23e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6a8076af0 0x7fa6a81025b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53150/0 (socket says 172.21.15.112:53150) 2024-09-16T00:17:40.228 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:39.222+0000 7fa6ad23e640 1 -- 172.21.15.112:0/1815929122 learned_addr learned my addr 172.21.15.112:0/1815929122 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:17:40.228 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:39.222+0000 7fa6ad23e640 1 -- 172.21.15.112:0/1815929122 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6a4009450 con 0x7fa6a8076af0 2024-09-16T00:17:40.228 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:39.222+0000 7fa6ad23e640 1 --2- 172.21.15.112:0/1815929122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6a8076af0 0x7fa6a81025b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4009420 tx=0x7fa6a40091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:40.228 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:39.222+0000 7fa69dffb640 1 -- 172.21.15.112:0/1815929122 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa6a4031d20 con 0x7fa6a8076af0 2024-09-16T00:17:40.228 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:39.222+0000 7fa69dffb640 1 -- 172.21.15.112:0/1815929122 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fa6a4031e80 con 0x7fa6a8076af0 2024-09-16T00:17:40.229 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:39.222+0000 7fa6af4a2640 1 -- 172.21.15.112:0/1815929122 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6a8100be0 con 0x7fa6a8076af0 2024-09-16T00:17:40.229 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:39.222+0000 7fa69dffb640 1 -- 172.21.15.112:0/1815929122 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa6a403a750 con 0x7fa6a8076af0 2024-09-16T00:17:40.229 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:39.223+0000 7fa6af4a2640 1 -- 172.21.15.112:0/1815929122 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6a81010b0 con 0x7fa6a8076af0 2024-09-16T00:17:40.229 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:39.223+0000 7fa69dffb640 1 -- 172.21.15.112:0/1815929122 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 45520+0+0 (secure 0 0 0) 0x7fa6a4039040 con 0x7fa6a8076af0 2024-09-16T00:17:40.229 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:39.224+0000 7fa6af4a2640 1 -- 172.21.15.112:0/1815929122 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6a804b720 con 0x7fa6a8076af0 2024-09-16T00:17:40.229 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:39.224+0000 7fa69dffb640 1 --2- 172.21.15.112:0/1815929122 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7fa694039530 0x7fa69403b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:40.230 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:39.224+0000 7fa69dffb640 1 -- 172.21.15.112:0/1815929122 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fa6a406cda0 con 0x7fa6a8076af0 2024-09-16T00:17:40.230 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:39.224+0000 7fa6aca3d640 1 --2- 172.21.15.112:0/1815929122 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7fa694039530 0x7fa69403b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:40.230 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:39.224+0000 7fa6aca3d640 1 --2- 172.21.15.112:0/1815929122 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7fa694039530 0x7fa69403b9e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa698006fd0 tx=0x7fa698006db0 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:40.230 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:39.228+0000 7fa69dffb640 1 -- 172.21.15.112:0/1815929122 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fa6a404bae0 con 0x7fa6a8076af0 2024-09-16T00:17:40.230 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:39.407+0000 7fa6af4a2640 1 -- 172.21.15.112:0/1815929122 --> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fa6a8105ef0 con 0x7fa694039530 2024-09-16T00:17:40.230 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:39.632+0000 7fa69dffb640 1 -- 172.21.15.112:0/1815929122 <== mgr.14160 v2:172.21.15.112:6800/2021251488 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7fa6a8105ef0 con 0x7fa694039530 2024-09-16T00:17:40.231 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:39.634+0000 7fa6af4a2640 1 -- 172.21.15.112:0/1815929122 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7fa694039530 msgr2=0x7fa69403b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:40.231 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:39.634+0000 7fa6af4a2640 1 --2- 172.21.15.112:0/1815929122 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7fa694039530 0x7fa69403b9e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa698006fd0 tx=0x7fa698006db0 comp rx=0 tx=0).stop 2024-09-16T00:17:40.231 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:39.634+0000 7fa6af4a2640 1 -- 172.21.15.112:0/1815929122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6a8076af0 msgr2=0x7fa6a81025b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:40.231 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:39.634+0000 7fa6af4a2640 1 --2- 172.21.15.112:0/1815929122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6a8076af0 0x7fa6a81025b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4009420 tx=0x7fa6a40091d0 comp rx=0 tx=0).stop 2024-09-16T00:17:40.231 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:39.634+0000 7fa6af4a2640 1 -- 172.21.15.112:0/1815929122 shutdown_connections 2024-09-16T00:17:40.231 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:39.634+0000 7fa6af4a2640 1 --2- 172.21.15.112:0/1815929122 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7fa694039530 0x7fa69403b9e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:40.232 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:39.634+0000 7fa6af4a2640 1 --2- 172.21.15.112:0/1815929122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6a8076af0 0x7fa6a81025b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:40.232 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:39.634+0000 7fa6af4a2640 1 -- 172.21.15.112:0/1815929122 >> 172.21.15.112:0/1815929122 conn(0x7fa6a80fc790 msgr2=0x7fa6a80fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:40.232 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:39.634+0000 7fa6af4a2640 1 -- 172.21.15.112:0/1815929122 shutdown_connections 2024-09-16T00:17:40.232 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:39.634+0000 7fa6af4a2640 1 -- 172.21.15.112:0/1815929122 wait complete. 2024-09-16T00:17:40.232 INFO:teuthology.orchestra.run.smithi112.stdout:Fetching dashboard port number... 2024-09-16T00:17:41.088 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:40 smithi112 ceph-mon[27869]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:17:41.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:40 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/1463711504' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-16T00:17:41.984 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 8443 2024-09-16T00:17:41.984 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:40.577+0000 7f56a2ed7640 1 Processor -- start 2024-09-16T00:17:41.984 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:40.577+0000 7f56a2ed7640 1 -- start start 2024-09-16T00:17:41.984 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:40.577+0000 7f56a2ed7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f569c076a90 0x7f569c104c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:41.984 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:40.577+0000 7f56a2ed7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f569c076ea0 con 0x7f569c076a90 2024-09-16T00:17:41.984 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:40.577+0000 7f56a0c73640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f569c076a90 0x7f569c104c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:41.984 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:40.577+0000 7f56a0c73640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f569c076a90 0x7f569c104c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53152/0 (socket says 172.21.15.112:53152) 2024-09-16T00:17:41.985 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:40.577+0000 7f56a0c73640 1 -- 172.21.15.112:0/168663053 learned_addr learned my addr 172.21.15.112:0/168663053 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:17:41.985 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:40.577+0000 7f56a0c73640 1 -- 172.21.15.112:0/168663053 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f569c1051b0 con 0x7f569c076a90 2024-09-16T00:17:41.985 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:40.578+0000 7f56a0c73640 1 --2- 172.21.15.112:0/168663053 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f569c076a90 0x7f569c104c70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5690009840 tx=0x7f56900091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=605eb34d5121e65a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:41.985 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:40.578+0000 7f569b7fe640 1 -- 172.21.15.112:0/168663053 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5690032440 con 0x7f569c076a90 2024-09-16T00:17:41.985 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:40.578+0000 7f569b7fe640 1 -- 172.21.15.112:0/168663053 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f5690032a40 con 0x7f569c076a90 2024-09-16T00:17:41.985 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:40.578+0000 7f569b7fe640 1 -- 172.21.15.112:0/168663053 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5690031aa0 con 0x7f569c076a90 2024-09-16T00:17:41.985 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:40.579+0000 7f56a2ed7640 1 -- 172.21.15.112:0/168663053 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f569c076a90 msgr2=0x7f569c104c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:41.985 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:40.579+0000 7f56a2ed7640 1 --2- 172.21.15.112:0/168663053 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f569c076a90 0x7f569c104c70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5690009840 tx=0x7f56900091d0 comp rx=0 tx=0).stop 2024-09-16T00:17:41.985 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:40.579+0000 7f56a2ed7640 1 -- 172.21.15.112:0/168663053 shutdown_connections 2024-09-16T00:17:41.985 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:40.579+0000 7f56a2ed7640 1 --2- 172.21.15.112:0/168663053 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f569c076a90 0x7f569c104c70 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:41.985 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:40.579+0000 7f56a2ed7640 1 -- 172.21.15.112:0/168663053 >> 172.21.15.112:0/168663053 conn(0x7f569c0fc700 msgr2=0x7f569c0feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:41.985 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:40.579+0000 7f56a2ed7640 1 -- 172.21.15.112:0/168663053 shutdown_connections 2024-09-16T00:17:41.985 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:40.579+0000 7f56a2ed7640 1 -- 172.21.15.112:0/168663053 wait complete. 2024-09-16T00:17:41.986 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:40.579+0000 7f56a2ed7640 1 Processor -- start 2024-09-16T00:17:41.986 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:40.580+0000 7f56a2ed7640 1 -- start start 2024-09-16T00:17:41.986 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:40.580+0000 7f56a2ed7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f569c076a90 0x7f569c110e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:41.986 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:40.580+0000 7f56a2ed7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f569c1113d0 con 0x7f569c076a90 2024-09-16T00:17:41.986 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:40.580+0000 7f56a0c73640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f569c076a90 0x7f569c110e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:41.986 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:40.580+0000 7f56a0c73640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f569c076a90 0x7f569c110e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53164/0 (socket says 172.21.15.112:53164) 2024-09-16T00:17:41.986 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:40.580+0000 7f56a0c73640 1 -- 172.21.15.112:0/1463711504 learned_addr learned my addr 172.21.15.112:0/1463711504 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:17:41.986 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:40.580+0000 7f56a0c73640 1 -- 172.21.15.112:0/1463711504 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5690009450 con 0x7f569c076a90 2024-09-16T00:17:41.986 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:40.580+0000 7f56a0c73640 1 --2- 172.21.15.112:0/1463711504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f569c076a90 0x7f569c110e90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f56900316a0 tx=0x7f56900091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:41.986 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:40.581+0000 7f56997fa640 1 -- 172.21.15.112:0/1463711504 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f56900327b0 con 0x7f569c076a90 2024-09-16T00:17:41.986 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:40.581+0000 7f56997fa640 1 -- 172.21.15.112:0/1463711504 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f569003b8d0 con 0x7f569c076a90 2024-09-16T00:17:41.986 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:40.581+0000 7f56a2ed7640 1 -- 172.21.15.112:0/1463711504 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f569c111630 con 0x7f569c076a90 2024-09-16T00:17:41.986 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:40.581+0000 7f56997fa640 1 -- 172.21.15.112:0/1463711504 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f569003a780 con 0x7f569c076a90 2024-09-16T00:17:41.987 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:40.581+0000 7f56a2ed7640 1 -- 172.21.15.112:0/1463711504 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f569c111fd0 con 0x7f569c076a90 2024-09-16T00:17:41.987 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:40.581+0000 7f56997fa640 1 -- 172.21.15.112:0/1463711504 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) v1 ==== 45520+0+0 (secure 0 0 0) 0x7f5690039040 con 0x7f569c076a90 2024-09-16T00:17:41.987 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:40.582+0000 7f56a2ed7640 1 -- 172.21.15.112:0/1463711504 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f569c04b720 con 0x7f569c076a90 2024-09-16T00:17:41.987 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:40.582+0000 7f56997fa640 1 --2- 172.21.15.112:0/1463711504 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f568c039530 0x7f568c03b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:41.987 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:40.582+0000 7f56997fa640 1 -- 172.21.15.112:0/1463711504 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f569006cba0 con 0x7f569c076a90 2024-09-16T00:17:41.987 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:40.582+0000 7f569bfff640 1 --2- 172.21.15.112:0/1463711504 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f568c039530 0x7f568c03b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:41.987 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:40.582+0000 7f569bfff640 1 --2- 172.21.15.112:0/1463711504 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f568c039530 0x7f568c03b9e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5688006fd0 tx=0x7f5688006db0 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:41.987 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:40.586+0000 7f56997fa640 1 -- 172.21.15.112:0/1463711504 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f569004bac0 con 0x7f569c076a90 2024-09-16T00:17:41.987 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:40.752+0000 7f56a2ed7640 1 -- 172.21.15.112:0/1463711504 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f569c112e40 con 0x7f569c076a90 2024-09-16T00:17:41.987 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:40.752+0000 7f56997fa640 1 -- 172.21.15.112:0/1463711504 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f569c112e40 con 0x7f569c076a90 2024-09-16T00:17:41.987 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:40.754+0000 7f56a2ed7640 1 -- 172.21.15.112:0/1463711504 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f568c039530 msgr2=0x7f568c03b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:41.987 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:40.754+0000 7f56a2ed7640 1 --2- 172.21.15.112:0/1463711504 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f568c039530 0x7f568c03b9e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5688006fd0 tx=0x7f5688006db0 comp rx=0 tx=0).stop 2024-09-16T00:17:41.987 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:40.754+0000 7f56a2ed7640 1 -- 172.21.15.112:0/1463711504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f569c076a90 msgr2=0x7f569c110e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:41.988 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:40.754+0000 7f56a2ed7640 1 --2- 172.21.15.112:0/1463711504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f569c076a90 0x7f569c110e90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f56900316a0 tx=0x7f56900091d0 comp rx=0 tx=0).stop 2024-09-16T00:17:41.988 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:40.755+0000 7f56a2ed7640 1 -- 172.21.15.112:0/1463711504 shutdown_connections 2024-09-16T00:17:41.988 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:40.755+0000 7f56a2ed7640 1 --2- 172.21.15.112:0/1463711504 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f568c039530 0x7f568c03b9e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:41.988 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:40.755+0000 7f56a2ed7640 1 --2- 172.21.15.112:0/1463711504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f569c076a90 0x7f569c110e90 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:41.988 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:40.755+0000 7f56a2ed7640 1 -- 172.21.15.112:0/1463711504 >> 172.21.15.112:0/1463711504 conn(0x7f569c0fc700 msgr2=0x7f569c0fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:41.988 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:40.755+0000 7f56a2ed7640 1 -- 172.21.15.112:0/1463711504 shutdown_connections 2024-09-16T00:17:41.988 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:40.755+0000 7f56a2ed7640 1 -- 172.21.15.112:0/1463711504 wait complete. 2024-09-16T00:17:41.992 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-16T00:17:41.992 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout disabled 2024-09-16T00:17:42.001 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-16T00:17:42.001 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout inactive 2024-09-16T00:17:42.001 INFO:teuthology.orchestra.run.smithi112.stdout:firewalld.service is not enabled 2024-09-16T00:17:42.001 INFO:teuthology.orchestra.run.smithi112.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-16T00:17:42.003 INFO:teuthology.orchestra.run.smithi112.stdout:Ceph Dashboard is now available at: 2024-09-16T00:17:42.003 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:17:42.004 INFO:teuthology.orchestra.run.smithi112.stdout: URL: https://smithi112.front.sepia.ceph.com:8443/ 2024-09-16T00:17:42.004 INFO:teuthology.orchestra.run.smithi112.stdout: User: admin 2024-09-16T00:17:42.004 INFO:teuthology.orchestra.run.smithi112.stdout: Password: h95tb2wuw0 2024-09-16T00:17:42.004 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:17:42.004 INFO:teuthology.orchestra.run.smithi112.stdout:Saving cluster configuration to /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/config directory 2024-09-16T00:17:42.005 INFO:teuthology.orchestra.run.smithi112.stdout:Enabling autotune for osd_memory_target 2024-09-16T00:17:43.157 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:42 smithi112 ceph-mon[27869]: mgrmap e13: smithi112.xrkkbc(active, since 6s) 2024-09-16T00:17:43.165 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:42.426+0000 7f8f8c0cb640 1 Processor -- start 2024-09-16T00:17:43.165 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:42.427+0000 7f8f8c0cb640 1 -- start start 2024-09-16T00:17:43.165 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:42.427+0000 7f8f8c0cb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f84076a90 0x7f8f841070e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:43.165 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:42.427+0000 7f8f8c0cb640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f84102ad0 con 0x7f8f84076a90 2024-09-16T00:17:43.165 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:42.427+0000 7f8f89e67640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f84076a90 0x7f8f841070e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:43.165 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:42.427+0000 7f8f89e67640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f84076a90 0x7f8f841070e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48486/0 (socket says 172.21.15.112:48486) 2024-09-16T00:17:43.166 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:42.427+0000 7f8f89e67640 1 -- 172.21.15.112:0/3161900070 learned_addr learned my addr 172.21.15.112:0/3161900070 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:17:43.166 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:42.427+0000 7f8f89e67640 1 -- 172.21.15.112:0/3161900070 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f84076ea0 con 0x7f8f84076a90 2024-09-16T00:17:43.166 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:42.428+0000 7f8f89e67640 1 --2- 172.21.15.112:0/3161900070 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f84076a90 0x7f8f841070e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8f80009840 tx=0x7f8f800091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=478a8fa9159251d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:43.166 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:42.428+0000 7f8f88e65640 1 -- 172.21.15.112:0/3161900070 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8f80032440 con 0x7f8f84076a90 2024-09-16T00:17:43.166 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:42.428+0000 7f8f88e65640 1 -- 172.21.15.112:0/3161900070 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f8f80032a40 con 0x7f8f84076a90 2024-09-16T00:17:43.166 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:42.428+0000 7f8f88e65640 1 -- 172.21.15.112:0/3161900070 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8f80031aa0 con 0x7f8f84076a90 2024-09-16T00:17:43.166 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:42.429+0000 7f8f8c0cb640 1 -- 172.21.15.112:0/3161900070 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f84076a90 msgr2=0x7f8f841070e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:43.166 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:42.429+0000 7f8f8c0cb640 1 --2- 172.21.15.112:0/3161900070 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f84076a90 0x7f8f841070e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8f80009840 tx=0x7f8f800091d0 comp rx=0 tx=0).stop 2024-09-16T00:17:43.166 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:42.429+0000 7f8f8c0cb640 1 -- 172.21.15.112:0/3161900070 shutdown_connections 2024-09-16T00:17:43.166 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:42.429+0000 7f8f8c0cb640 1 --2- 172.21.15.112:0/3161900070 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f84076a90 0x7f8f841070e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:43.167 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:42.429+0000 7f8f8c0cb640 1 -- 172.21.15.112:0/3161900070 >> 172.21.15.112:0/3161900070 conn(0x7f8f840fc750 msgr2=0x7f8f840feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:43.167 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:42.429+0000 7f8f8c0cb640 1 -- 172.21.15.112:0/3161900070 shutdown_connections 2024-09-16T00:17:43.167 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:42.429+0000 7f8f8c0cb640 1 -- 172.21.15.112:0/3161900070 wait complete. 2024-09-16T00:17:43.167 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:42.429+0000 7f8f8c0cb640 1 Processor -- start 2024-09-16T00:17:43.167 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:42.429+0000 7f8f8c0cb640 1 -- start start 2024-09-16T00:17:43.167 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:42.430+0000 7f8f8c0cb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f84076a90 0x7f8f84112fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:43.167 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:42.430+0000 7f8f8c0cb640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f84113510 con 0x7f8f84076a90 2024-09-16T00:17:43.167 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:42.430+0000 7f8f89e67640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f84076a90 0x7f8f84112fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:43.167 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:42.430+0000 7f8f89e67640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f84076a90 0x7f8f84112fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48490/0 (socket says 172.21.15.112:48490) 2024-09-16T00:17:43.167 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:42.430+0000 7f8f89e67640 1 -- 172.21.15.112:0/1436543660 learned_addr learned my addr 172.21.15.112:0/1436543660 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:17:43.167 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:42.430+0000 7f8f89e67640 1 -- 172.21.15.112:0/1436543660 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f80009450 con 0x7f8f84076a90 2024-09-16T00:17:43.167 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:42.430+0000 7f8f89e67640 1 --2- 172.21.15.112:0/1436543660 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f84076a90 0x7f8f84112fd0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8f80031780 tx=0x7f8f800091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:43.167 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:42.430+0000 7f8f7a7fc640 1 -- 172.21.15.112:0/1436543660 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8f800327b0 con 0x7f8f84076a90 2024-09-16T00:17:43.167 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:42.430+0000 7f8f7a7fc640 1 -- 172.21.15.112:0/1436543660 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f8f8003b8d0 con 0x7f8f84076a90 2024-09-16T00:17:43.167 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:42.430+0000 7f8f7a7fc640 1 -- 172.21.15.112:0/1436543660 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8f8003a790 con 0x7f8f84076a90 2024-09-16T00:17:43.168 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:42.430+0000 7f8f8c0cb640 1 -- 172.21.15.112:0/1436543660 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f84113710 con 0x7f8f84076a90 2024-09-16T00:17:43.168 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:42.430+0000 7f8f8c0cb640 1 -- 172.21.15.112:0/1436543660 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f84113b80 con 0x7f8f84076a90 2024-09-16T00:17:43.168 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:42.431+0000 7f8f7a7fc640 1 -- 172.21.15.112:0/1436543660 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) v1 ==== 45564+0+0 (secure 0 0 0) 0x7f8f80039040 con 0x7f8f84076a90 2024-09-16T00:17:43.168 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:42.431+0000 7f8f8c0cb640 1 -- 172.21.15.112:0/1436543660 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f8404b720 con 0x7f8f84076a90 2024-09-16T00:17:43.168 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:42.431+0000 7f8f7a7fc640 1 --2- 172.21.15.112:0/1436543660 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f8f700395d0 0x7f8f7003ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:43.168 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:42.431+0000 7f8f7a7fc640 1 -- 172.21.15.112:0/1436543660 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f8f8006caf0 con 0x7f8f84076a90 2024-09-16T00:17:43.168 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:42.431+0000 7f8f89666640 1 --2- 172.21.15.112:0/1436543660 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f8f700395d0 0x7f8f7003ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:43.168 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:42.432+0000 7f8f89666640 1 --2- 172.21.15.112:0/1436543660 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f8f700395d0 0x7f8f7003ba80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8f74006fd0 tx=0x7f8f74006db0 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:43.168 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:42.436+0000 7f8f7a7fc640 1 -- 172.21.15.112:0/1436543660 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f8f8003ba40 con 0x7f8f84076a90 2024-09-16T00:17:43.168 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:42.596+0000 7f8f8c0cb640 1 -- 172.21.15.112:0/1436543660 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f8f841144e0 con 0x7f8f84076a90 2024-09-16T00:17:43.168 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:42.596+0000 7f8f7a7fc640 1 -- 172.21.15.112:0/1436543660 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f8f8003f070 con 0x7f8f84076a90 2024-09-16T00:17:43.168 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:42.598+0000 7f8f8c0cb640 1 -- 172.21.15.112:0/1436543660 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f8f700395d0 msgr2=0x7f8f7003ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:43.168 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:42.598+0000 7f8f8c0cb640 1 --2- 172.21.15.112:0/1436543660 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f8f700395d0 0x7f8f7003ba80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8f74006fd0 tx=0x7f8f74006db0 comp rx=0 tx=0).stop 2024-09-16T00:17:43.169 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:42.598+0000 7f8f8c0cb640 1 -- 172.21.15.112:0/1436543660 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f84076a90 msgr2=0x7f8f84112fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:43.169 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:42.598+0000 7f8f8c0cb640 1 --2- 172.21.15.112:0/1436543660 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f84076a90 0x7f8f84112fd0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8f80031780 tx=0x7f8f800091d0 comp rx=0 tx=0).stop 2024-09-16T00:17:43.169 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:42.598+0000 7f8f8c0cb640 1 -- 172.21.15.112:0/1436543660 shutdown_connections 2024-09-16T00:17:43.169 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:42.598+0000 7f8f8c0cb640 1 --2- 172.21.15.112:0/1436543660 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f8f700395d0 0x7f8f7003ba80 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:43.169 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:42.598+0000 7f8f8c0cb640 1 --2- 172.21.15.112:0/1436543660 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f84076a90 0x7f8f84112fd0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:43.169 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:42.598+0000 7f8f8c0cb640 1 -- 172.21.15.112:0/1436543660 >> 172.21.15.112:0/1436543660 conn(0x7f8f840fc750 msgr2=0x7f8f840fd390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:43.169 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:42.598+0000 7f8f8c0cb640 1 -- 172.21.15.112:0/1436543660 shutdown_connections 2024-09-16T00:17:43.169 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:42.598+0000 7f8f8c0cb640 1 -- 172.21.15.112:0/1436543660 wait complete. 2024-09-16T00:17:44.413 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:43.589+0000 7fe9ed7d0640 1 Processor -- start 2024-09-16T00:17:44.414 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:43.589+0000 7fe9ed7d0640 1 -- start start 2024-09-16T00:17:44.414 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:43.589+0000 7fe9ed7d0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9e8100e20 0x7fe9e8101230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:44.414 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:43.589+0000 7fe9ed7d0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e8101770 con 0x7fe9e8100e20 2024-09-16T00:17:44.414 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:43.589+0000 7fe9e6ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9e8100e20 0x7fe9e8101230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:44.414 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:43.589+0000 7fe9e6ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9e8100e20 0x7fe9e8101230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48494/0 (socket says 172.21.15.112:48494) 2024-09-16T00:17:44.414 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:43.589+0000 7fe9e6ffd640 1 -- 172.21.15.112:0/2555785140 learned_addr learned my addr 172.21.15.112:0/2555785140 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:17:44.414 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:43.590+0000 7fe9e6ffd640 1 -- 172.21.15.112:0/2555785140 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9e81018b0 con 0x7fe9e8100e20 2024-09-16T00:17:44.414 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:43.590+0000 7fe9e6ffd640 1 --2- 172.21.15.112:0/2555785140 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9e8100e20 0x7fe9e8101230 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe9e000baf0 tx=0x7fe9e000b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bccf6bf338359dc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:44.414 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:43.590+0000 7fe9e5ffb640 1 -- 172.21.15.112:0/2555785140 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe9e00327a0 con 0x7fe9e8100e20 2024-09-16T00:17:44.414 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:43.591+0000 7fe9e5ffb640 1 -- 172.21.15.112:0/2555785140 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fe9e0032da0 con 0x7fe9e8100e20 2024-09-16T00:17:44.414 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:43.591+0000 7fe9e5ffb640 1 -- 172.21.15.112:0/2555785140 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe9e0031e60 con 0x7fe9e8100e20 2024-09-16T00:17:44.414 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:43.591+0000 7fe9ed7d0640 1 -- 172.21.15.112:0/2555785140 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9e8100e20 msgr2=0x7fe9e8101230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:44.414 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:43.591+0000 7fe9ed7d0640 1 --2- 172.21.15.112:0/2555785140 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9e8100e20 0x7fe9e8101230 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe9e000baf0 tx=0x7fe9e000b460 comp rx=0 tx=0).stop 2024-09-16T00:17:44.414 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:43.591+0000 7fe9ed7d0640 1 -- 172.21.15.112:0/2555785140 shutdown_connections 2024-09-16T00:17:44.415 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:43.591+0000 7fe9ed7d0640 1 --2- 172.21.15.112:0/2555785140 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9e8100e20 0x7fe9e8101230 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:44.415 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:43.591+0000 7fe9ed7d0640 1 -- 172.21.15.112:0/2555785140 >> 172.21.15.112:0/2555785140 conn(0x7fe9e80fc5f0 msgr2=0x7fe9e80fea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:44.415 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:43.591+0000 7fe9ed7d0640 1 -- 172.21.15.112:0/2555785140 shutdown_connections 2024-09-16T00:17:44.415 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:43.591+0000 7fe9ed7d0640 1 -- 172.21.15.112:0/2555785140 wait complete. 2024-09-16T00:17:44.415 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:43.592+0000 7fe9ed7d0640 1 Processor -- start 2024-09-16T00:17:44.415 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:43.592+0000 7fe9ed7d0640 1 -- start start 2024-09-16T00:17:44.415 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:43.592+0000 7fe9ed7d0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9e8100e20 0x7fe9e810eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:44.415 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:43.592+0000 7fe9ed7d0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e810f090 con 0x7fe9e8100e20 2024-09-16T00:17:44.415 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:43.592+0000 7fe9e6ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9e8100e20 0x7fe9e810eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:44.415 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:43.592+0000 7fe9e6ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9e8100e20 0x7fe9e810eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48504/0 (socket says 172.21.15.112:48504) 2024-09-16T00:17:44.415 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:43.592+0000 7fe9e6ffd640 1 -- 172.21.15.112:0/3419875435 learned_addr learned my addr 172.21.15.112:0/3419875435 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:17:44.415 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:43.592+0000 7fe9e6ffd640 1 -- 172.21.15.112:0/3419875435 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9e000b700 con 0x7fe9e8100e20 2024-09-16T00:17:44.415 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:43.592+0000 7fe9e6ffd640 1 --2- 172.21.15.112:0/3419875435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9e8100e20 0x7fe9e810eb50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe9e000b2c0 tx=0x7fe9e003d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:44.415 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:43.593+0000 7fe9df7fe640 1 -- 172.21.15.112:0/3419875435 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe9e00329b0 con 0x7fe9e8100e20 2024-09-16T00:17:44.416 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:43.593+0000 7fe9df7fe640 1 -- 172.21.15.112:0/3419875435 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fe9e0032b10 con 0x7fe9e8100e20 2024-09-16T00:17:44.416 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:43.593+0000 7fe9df7fe640 1 -- 172.21.15.112:0/3419875435 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe9e0045640 con 0x7fe9e8100e20 2024-09-16T00:17:44.416 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:43.593+0000 7fe9ed7d0640 1 -- 172.21.15.112:0/3419875435 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9e810f290 con 0x7fe9e8100e20 2024-09-16T00:17:44.416 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:43.593+0000 7fe9ed7d0640 1 -- 172.21.15.112:0/3419875435 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9e810f700 con 0x7fe9e8100e20 2024-09-16T00:17:44.416 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:43.594+0000 7fe9df7fe640 1 -- 172.21.15.112:0/3419875435 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) v1 ==== 45564+0+0 (secure 0 0 0) 0x7fe9e003b420 con 0x7fe9e8100e20 2024-09-16T00:17:44.416 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:43.594+0000 7fe9df7fe640 1 --2- 172.21.15.112:0/3419875435 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7fe9d40395d0 0x7fe9d403ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:44.416 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:43.594+0000 7fe9ed7d0640 1 -- 172.21.15.112:0/3419875435 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9e804b720 con 0x7fe9e8100e20 2024-09-16T00:17:44.416 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:43.594+0000 7fe9df7fe640 1 -- 172.21.15.112:0/3419875435 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fe9e006fe80 con 0x7fe9e8100e20 2024-09-16T00:17:44.416 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:43.594+0000 7fe9e67fc640 1 --2- 172.21.15.112:0/3419875435 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7fe9d40395d0 0x7fe9d403ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:44.416 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:43.594+0000 7fe9e67fc640 1 --2- 172.21.15.112:0/3419875435 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7fe9d40395d0 0x7fe9d403ba80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0006fd0 tx=0x7fe9d0006db0 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:44.416 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:43.598+0000 7fe9df7fe640 1 -- 172.21.15.112:0/3419875435 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fe9e0036020 con 0x7fe9e8100e20 2024-09-16T00:17:44.416 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:43.835+0000 7fe9ed7d0640 1 -- 172.21.15.112:0/3419875435 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7fe9e8102bf0 con 0x7fe9e8100e20 2024-09-16T00:17:44.416 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:43.838+0000 7fe9df7fe640 1 -- 172.21.15.112:0/3419875435 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v33)=0 set mgr/dashboard/cluster/status v33) v1 ==== 153+0+0 (secure 0 0 0) 0x7fe9e004d7d0 con 0x7fe9e8100e20 2024-09-16T00:17:44.416 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-16T00:17:44.417 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:43.840+0000 7fe9ed7d0640 1 -- 172.21.15.112:0/3419875435 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7fe9d40395d0 msgr2=0x7fe9d403ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:44.417 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:43.840+0000 7fe9ed7d0640 1 --2- 172.21.15.112:0/3419875435 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7fe9d40395d0 0x7fe9d403ba80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0006fd0 tx=0x7fe9d0006db0 comp rx=0 tx=0).stop 2024-09-16T00:17:44.417 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:43.840+0000 7fe9ed7d0640 1 -- 172.21.15.112:0/3419875435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9e8100e20 msgr2=0x7fe9e810eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:44.417 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:43.840+0000 7fe9ed7d0640 1 --2- 172.21.15.112:0/3419875435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9e8100e20 0x7fe9e810eb50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe9e000b2c0 tx=0x7fe9e003d040 comp rx=0 tx=0).stop 2024-09-16T00:17:44.417 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:43.840+0000 7fe9ed7d0640 1 -- 172.21.15.112:0/3419875435 shutdown_connections 2024-09-16T00:17:44.417 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:43.840+0000 7fe9ed7d0640 1 --2- 172.21.15.112:0/3419875435 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7fe9d40395d0 0x7fe9d403ba80 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:44.417 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:43.840+0000 7fe9ed7d0640 1 --2- 172.21.15.112:0/3419875435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9e8100e20 0x7fe9e810eb50 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:44.417 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:43.840+0000 7fe9ed7d0640 1 -- 172.21.15.112:0/3419875435 >> 172.21.15.112:0/3419875435 conn(0x7fe9e80fc5f0 msgr2=0x7fe9e8105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:44.417 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:43.840+0000 7fe9ed7d0640 1 -- 172.21.15.112:0/3419875435 shutdown_connections 2024-09-16T00:17:44.418 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-09-16T00:17:43.840+0000 7fe9ed7d0640 1 -- 172.21.15.112:0/3419875435 wait complete. 2024-09-16T00:17:44.418 INFO:teuthology.orchestra.run.smithi112.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-16T00:17:44.418 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:17:44.418 INFO:teuthology.orchestra.run.smithi112.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-16T00:17:44.418 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:17:44.418 INFO:teuthology.orchestra.run.smithi112.stdout:Or, if you are only running a single cluster on this host: 2024-09-16T00:17:44.418 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:17:44.418 INFO:teuthology.orchestra.run.smithi112.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-16T00:17:44.418 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:17:44.418 INFO:teuthology.orchestra.run.smithi112.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-16T00:17:44.418 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:17:44.418 INFO:teuthology.orchestra.run.smithi112.stdout: ceph telemetry on 2024-09-16T00:17:44.418 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:17:44.418 INFO:teuthology.orchestra.run.smithi112.stdout:For more information see: 2024-09-16T00:17:44.419 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:17:44.419 INFO:teuthology.orchestra.run.smithi112.stdout: https://docs.ceph.com/docs/master/mgr/telemetry/ 2024-09-16T00:17:44.419 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:17:44.419 INFO:teuthology.orchestra.run.smithi112.stdout:Bootstrap complete. 2024-09-16T00:17:44.450 INFO:tasks.cephadm:Fetching config... 2024-09-16T00:17:44.450 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-16T00:17:44.451 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-16T00:17:44.467 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-16T00:17:44.467 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-16T00:17:44.467 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-16T00:17:44.523 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-16T00:17:44.523 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-16T00:17:44.523 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/keyring of=/dev/stdout 2024-09-16T00:17:44.587 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-16T00:17:44.588 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-16T00:17:44.588 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-16T00:17:44.644 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-16T00:17:44.644 DEBUG:teuthology.orchestra.run.smithi112:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7l4AiGYDDQARb/gsM1EGdF3xeIpdQ7vfY8ZU2NDvIGeQndFGtL0R0lJB3An7epXqaOeVCgSo2m172+4hnYyi0YAGsqZaoCF1AwJOTOiZWrIynZ+7buBqctQq2KCY6bv+sNXOk0Lxox7C7479vyc4JAJtWeOK3u+2EoiIF1KbnHP9CENMf4TAzxFAowDU87z/rHoUTKiXvYT8dL0GiMWsKwGdgbALaWlRIWVglPZYYOcvgwpobpHjB1m3ahPe1WVI703RTyWvtvPtrMs1eQthnm0r3qpWfsV6LPj60A2KcSh7S288ZYiGGpJx1c5rOSEVQspRadHVxXxSWzVApTz2IKP0y8OBc0OJYUcKr80F7Wm4FKRMYyNTJ+4LgAotrvuc7hlNp449tURHqG5pIN7n8q2V8whwvnlsRoNDMWUBoTuxHHkWN6Wk6nZhJRHCcLAqx7kCEyrt1IcYYIvrxZ6GzR6x/kkDqxZ1GVCJe19KLgGCTonkacBT4SLIip7w47AE= ceph-d346dd56-73c0-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-16T00:17:44.733 INFO:teuthology.orchestra.run.smithi112.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7l4AiGYDDQARb/gsM1EGdF3xeIpdQ7vfY8ZU2NDvIGeQndFGtL0R0lJB3An7epXqaOeVCgSo2m172+4hnYyi0YAGsqZaoCF1AwJOTOiZWrIynZ+7buBqctQq2KCY6bv+sNXOk0Lxox7C7479vyc4JAJtWeOK3u+2EoiIF1KbnHP9CENMf4TAzxFAowDU87z/rHoUTKiXvYT8dL0GiMWsKwGdgbALaWlRIWVglPZYYOcvgwpobpHjB1m3ahPe1WVI703RTyWvtvPtrMs1eQthnm0r3qpWfsV6LPj60A2KcSh7S288ZYiGGpJx1c5rOSEVQspRadHVxXxSWzVApTz2IKP0y8OBc0OJYUcKr80F7Wm4FKRMYyNTJ+4LgAotrvuc7hlNp449tURHqG5pIN7n8q2V8whwvnlsRoNDMWUBoTuxHHkWN6Wk6nZhJRHCcLAqx7kCEyrt1IcYYIvrxZ6GzR6x/kkDqxZ1GVCJe19KLgGCTonkacBT4SLIip7w47AE= ceph-d346dd56-73c0-11ef-bceb-c7b262605968 2024-09-16T00:17:44.746 DEBUG:teuthology.orchestra.run.smithi169:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7l4AiGYDDQARb/gsM1EGdF3xeIpdQ7vfY8ZU2NDvIGeQndFGtL0R0lJB3An7epXqaOeVCgSo2m172+4hnYyi0YAGsqZaoCF1AwJOTOiZWrIynZ+7buBqctQq2KCY6bv+sNXOk0Lxox7C7479vyc4JAJtWeOK3u+2EoiIF1KbnHP9CENMf4TAzxFAowDU87z/rHoUTKiXvYT8dL0GiMWsKwGdgbALaWlRIWVglPZYYOcvgwpobpHjB1m3ahPe1WVI703RTyWvtvPtrMs1eQthnm0r3qpWfsV6LPj60A2KcSh7S288ZYiGGpJx1c5rOSEVQspRadHVxXxSWzVApTz2IKP0y8OBc0OJYUcKr80F7Wm4FKRMYyNTJ+4LgAotrvuc7hlNp449tURHqG5pIN7n8q2V8whwvnlsRoNDMWUBoTuxHHkWN6Wk6nZhJRHCcLAqx7kCEyrt1IcYYIvrxZ6GzR6x/kkDqxZ1GVCJe19KLgGCTonkacBT4SLIip7w47AE= ceph-d346dd56-73c0-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-16T00:17:44.880 INFO:teuthology.orchestra.run.smithi169.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7l4AiGYDDQARb/gsM1EGdF3xeIpdQ7vfY8ZU2NDvIGeQndFGtL0R0lJB3An7epXqaOeVCgSo2m172+4hnYyi0YAGsqZaoCF1AwJOTOiZWrIynZ+7buBqctQq2KCY6bv+sNXOk0Lxox7C7479vyc4JAJtWeOK3u+2EoiIF1KbnHP9CENMf4TAzxFAowDU87z/rHoUTKiXvYT8dL0GiMWsKwGdgbALaWlRIWVglPZYYOcvgwpobpHjB1m3ahPe1WVI703RTyWvtvPtrMs1eQthnm0r3qpWfsV6LPj60A2KcSh7S288ZYiGGpJx1c5rOSEVQspRadHVxXxSWzVApTz2IKP0y8OBc0OJYUcKr80F7Wm4FKRMYyNTJ+4LgAotrvuc7hlNp449tURHqG5pIN7n8q2V8whwvnlsRoNDMWUBoTuxHHkWN6Wk6nZhJRHCcLAqx7kCEyrt1IcYYIvrxZ6GzR6x/kkDqxZ1GVCJe19KLgGCTonkacBT4SLIip7w47AE= ceph-d346dd56-73c0-11ef-bceb-c7b262605968 2024-09-16T00:17:44.895 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-16T00:17:45.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:44 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/3419875435' entity='client.admin' 2024-09-16T00:17:45.140 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:17:47.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:47.411+0000 7f5e386f1640 1 -- 172.21.15.112:0/2196537069 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e300fd8b0 msgr2=0x7f5e300fdcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:47.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:47.411+0000 7f5e386f1640 1 --2- 172.21.15.112:0/2196537069 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e300fd8b0 0x7f5e300fdcc0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c004850 tx=0x7f5e2c009da0 comp rx=0 tx=0).stop 2024-09-16T00:17:47.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:47.411+0000 7f5e386f1640 1 -- 172.21.15.112:0/2196537069 shutdown_connections 2024-09-16T00:17:47.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:47.411+0000 7f5e386f1640 1 --2- 172.21.15.112:0/2196537069 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e300fd8b0 0x7f5e300fdcc0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:47.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:47.411+0000 7f5e386f1640 1 -- 172.21.15.112:0/2196537069 >> 172.21.15.112:0/2196537069 conn(0x7f5e300f6010 msgr2=0x7f5e300f8440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:47.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:47.411+0000 7f5e386f1640 1 -- 172.21.15.112:0/2196537069 shutdown_connections 2024-09-16T00:17:47.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:47.411+0000 7f5e386f1640 1 -- 172.21.15.112:0/2196537069 wait complete. 2024-09-16T00:17:47.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:47.412+0000 7f5e386f1640 1 Processor -- start 2024-09-16T00:17:47.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:47.412+0000 7f5e386f1640 1 -- start start 2024-09-16T00:17:47.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:47.412+0000 7f5e386f1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e300fd8b0 0x7f5e3010c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:47.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:47.412+0000 7f5e386f1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3010ccc0 con 0x7f5e300fd8b0 2024-09-16T00:17:47.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:47.412+0000 7f5e3648d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e300fd8b0 0x7f5e3010c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:47.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:47.412+0000 7f5e3648d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e300fd8b0 0x7f5e3010c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48510/0 (socket says 172.21.15.112:48510) 2024-09-16T00:17:47.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:47.412+0000 7f5e3648d640 1 -- 172.21.15.112:0/2580087793 learned_addr learned my addr 172.21.15.112:0/2580087793 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:17:47.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:47.413+0000 7f5e3648d640 1 -- 172.21.15.112:0/2580087793 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e2c005040 con 0x7f5e300fd8b0 2024-09-16T00:17:47.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:47.413+0000 7f5e3648d640 1 --2- 172.21.15.112:0/2580087793 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e300fd8b0 0x7f5e3010c780 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c02a6d0 tx=0x7f5e2c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:47.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:47.413+0000 7f5e22ffd640 1 -- 172.21.15.112:0/2580087793 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5e2c033630 con 0x7f5e300fd8b0 2024-09-16T00:17:47.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:47.413+0000 7f5e22ffd640 1 -- 172.21.15.112:0/2580087793 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f5e2c033c30 con 0x7f5e300fd8b0 2024-09-16T00:17:47.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:47.413+0000 7f5e386f1640 1 -- 172.21.15.112:0/2580087793 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e3010cec0 con 0x7f5e300fd8b0 2024-09-16T00:17:47.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:47.413+0000 7f5e22ffd640 1 -- 172.21.15.112:0/2580087793 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5e2c03dc30 con 0x7f5e300fd8b0 2024-09-16T00:17:47.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:47.413+0000 7f5e386f1640 1 -- 172.21.15.112:0/2580087793 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e3010d330 con 0x7f5e300fd8b0 2024-09-16T00:17:47.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:47.414+0000 7f5e22ffd640 1 -- 172.21.15.112:0/2580087793 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) v1 ==== 45564+0+0 (secure 0 0 0) 0x7f5e2c030020 con 0x7f5e300fd8b0 2024-09-16T00:17:47.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:47.414+0000 7f5e386f1640 1 -- 172.21.15.112:0/2580087793 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e3005ed70 con 0x7f5e300fd8b0 2024-09-16T00:17:47.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:47.414+0000 7f5e22ffd640 1 --2- 172.21.15.112:0/2580087793 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f5e1c0395d0 0x7f5e1c03ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:47.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:47.414+0000 7f5e22ffd640 1 -- 172.21.15.112:0/2580087793 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f5e2c06c860 con 0x7f5e300fd8b0 2024-09-16T00:17:47.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:47.414+0000 7f5e35c8c640 1 --2- 172.21.15.112:0/2580087793 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f5e1c0395d0 0x7f5e1c03ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:47.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:47.415+0000 7f5e35c8c640 1 --2- 172.21.15.112:0/2580087793 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f5e1c0395d0 0x7f5e1c03ba80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5e24006fd0 tx=0x7f5e24006db0 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:47.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:47.419+0000 7f5e22ffd640 1 -- 172.21.15.112:0/2580087793 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f5e2c03b580 con 0x7f5e300fd8b0 2024-09-16T00:17:47.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:47.580+0000 7f5e386f1640 1 -- 172.21.15.112:0/2580087793 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f5e3010d330 con 0x7f5e300fd8b0 2024-09-16T00:17:47.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:47.583+0000 7f5e22ffd640 1 -- 172.21.15.112:0/2580087793 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f5e2c045b80 con 0x7f5e300fd8b0 2024-09-16T00:17:47.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:47.585+0000 7f5e386f1640 1 -- 172.21.15.112:0/2580087793 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f5e1c0395d0 msgr2=0x7f5e1c03ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:47.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:47.585+0000 7f5e386f1640 1 --2- 172.21.15.112:0/2580087793 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f5e1c0395d0 0x7f5e1c03ba80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5e24006fd0 tx=0x7f5e24006db0 comp rx=0 tx=0).stop 2024-09-16T00:17:47.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:47.585+0000 7f5e386f1640 1 -- 172.21.15.112:0/2580087793 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e300fd8b0 msgr2=0x7f5e3010c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:47.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:47.585+0000 7f5e386f1640 1 --2- 172.21.15.112:0/2580087793 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e300fd8b0 0x7f5e3010c780 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c02a6d0 tx=0x7f5e2c009da0 comp rx=0 tx=0).stop 2024-09-16T00:17:47.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:47.585+0000 7f5e386f1640 1 -- 172.21.15.112:0/2580087793 shutdown_connections 2024-09-16T00:17:47.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:47.585+0000 7f5e386f1640 1 --2- 172.21.15.112:0/2580087793 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f5e1c0395d0 0x7f5e1c03ba80 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:47.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:47.585+0000 7f5e386f1640 1 --2- 172.21.15.112:0/2580087793 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e300fd8b0 0x7f5e3010c780 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:47.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:47.585+0000 7f5e386f1640 1 -- 172.21.15.112:0/2580087793 >> 172.21.15.112:0/2580087793 conn(0x7f5e300f6010 msgr2=0x7f5e300f68f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:47.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:47.586+0000 7f5e386f1640 1 -- 172.21.15.112:0/2580087793 shutdown_connections 2024-09-16T00:17:47.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:47.586+0000 7f5e386f1640 1 -- 172.21.15.112:0/2580087793 wait complete. 2024-09-16T00:17:48.263 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-16T00:17:48.263 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-16T00:17:48.835 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:17:48.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:48 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/2580087793' entity='client.admin' 2024-09-16T00:17:50.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:49 smithi112 ceph-mon[27869]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:17:50.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:49 smithi112 ceph-mon[27869]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:17:50.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:49 smithi112 ceph-mon[27869]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "who": "osd/host:smithi112", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:17:50.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:49 smithi112 ceph-mon[27869]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:17:50.091 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:49 smithi112 ceph-mon[27869]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi112", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-16T00:17:50.091 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:49 smithi112 ceph-mon[27869]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi112", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-09-16T00:17:50.091 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:49 smithi112 ceph-mon[27869]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:17:50.091 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:49 smithi112 ceph-mon[27869]: Deploying daemon crash.smithi112 on smithi112 2024-09-16T00:17:50.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:50.440+0000 7f85f1caf640 1 -- 172.21.15.112:0/4088256113 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85ec0661a0 msgr2=0x7f85ec0fede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:50.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:50.440+0000 7f85f1caf640 1 --2- 172.21.15.112:0/4088256113 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85ec0661a0 0x7f85ec0fede0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f85dc004850 tx=0x7f85dc009da0 comp rx=0 tx=0).stop 2024-09-16T00:17:50.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:50.440+0000 7f85f1caf640 1 -- 172.21.15.112:0/4088256113 shutdown_connections 2024-09-16T00:17:50.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:50.440+0000 7f85f1caf640 1 --2- 172.21.15.112:0/4088256113 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85ec0661a0 0x7f85ec0fede0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:50.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:50.440+0000 7f85f1caf640 1 -- 172.21.15.112:0/4088256113 >> 172.21.15.112:0/4088256113 conn(0x7f85ec0f6890 msgr2=0x7f85ec0f8cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:50.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:50.440+0000 7f85f1caf640 1 -- 172.21.15.112:0/4088256113 shutdown_connections 2024-09-16T00:17:50.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:50.440+0000 7f85f1caf640 1 -- 172.21.15.112:0/4088256113 wait complete. 2024-09-16T00:17:50.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:50.441+0000 7f85f1caf640 1 Processor -- start 2024-09-16T00:17:50.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:50.441+0000 7f85f1caf640 1 -- start start 2024-09-16T00:17:50.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:50.441+0000 7f85f1caf640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85ec0661a0 0x7f85ec10c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:50.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:50.441+0000 7f85f1caf640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85ec10cc50 con 0x7f85ec0661a0 2024-09-16T00:17:50.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:50.441+0000 7f85eb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85ec0661a0 0x7f85ec10c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:50.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:50.441+0000 7f85eb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85ec0661a0 0x7f85ec10c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48522/0 (socket says 172.21.15.112:48522) 2024-09-16T00:17:50.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:50.441+0000 7f85eb7fe640 1 -- 172.21.15.112:0/884581383 learned_addr learned my addr 172.21.15.112:0/884581383 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:17:50.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:50.441+0000 7f85eb7fe640 1 -- 172.21.15.112:0/884581383 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85dc005040 con 0x7f85ec0661a0 2024-09-16T00:17:50.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:50.442+0000 7f85eb7fe640 1 --2- 172.21.15.112:0/884581383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85ec0661a0 0x7f85ec10c710 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f85dc02a6d0 tx=0x7f85dc009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:50.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:50.442+0000 7f85e8ff9640 1 -- 172.21.15.112:0/884581383 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f85dc033630 con 0x7f85ec0661a0 2024-09-16T00:17:50.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:50.442+0000 7f85e8ff9640 1 -- 172.21.15.112:0/884581383 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f85dc033c30 con 0x7f85ec0661a0 2024-09-16T00:17:50.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:50.442+0000 7f85f1caf640 1 -- 172.21.15.112:0/884581383 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85ec10ce50 con 0x7f85ec0661a0 2024-09-16T00:17:50.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:50.442+0000 7f85e8ff9640 1 -- 172.21.15.112:0/884581383 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f85dc03dc30 con 0x7f85ec0661a0 2024-09-16T00:17:50.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:50.442+0000 7f85f1caf640 1 -- 172.21.15.112:0/884581383 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85ec10d2c0 con 0x7f85ec0661a0 2024-09-16T00:17:50.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:50.443+0000 7f85e8ff9640 1 -- 172.21.15.112:0/884581383 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) v1 ==== 45564+0+0 (secure 0 0 0) 0x7f85dc030020 con 0x7f85ec0661a0 2024-09-16T00:17:50.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:50.443+0000 7f85f1caf640 1 -- 172.21.15.112:0/884581383 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85ec05ed70 con 0x7f85ec0661a0 2024-09-16T00:17:50.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:50.443+0000 7f85e8ff9640 1 --2- 172.21.15.112:0/884581383 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f85d80395d0 0x7f85d803ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:50.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:50.443+0000 7f85e8ff9640 1 -- 172.21.15.112:0/884581383 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f85dc06c7b0 con 0x7f85ec0661a0 2024-09-16T00:17:50.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:50.443+0000 7f85e3fff640 1 --2- 172.21.15.112:0/884581383 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f85d80395d0 0x7f85d803ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:50.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:50.443+0000 7f85e3fff640 1 --2- 172.21.15.112:0/884581383 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f85d80395d0 0x7f85d803ba80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f85d4006fd0 tx=0x7f85d4006db0 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:50.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:50.448+0000 7f85e8ff9640 1 -- 172.21.15.112:0/884581383 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f85dc045db0 con 0x7f85ec0661a0 2024-09-16T00:17:50.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:50.613+0000 7f85f1caf640 1 -- 172.21.15.112:0/884581383 --> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f85ec10d2c0 con 0x7f85d80395d0 2024-09-16T00:17:50.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:50.616+0000 7f85e8ff9640 1 -- 172.21.15.112:0/884581383 <== mgr.14160 v2:172.21.15.112:6800/2021251488 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f85ec10d2c0 con 0x7f85d80395d0 2024-09-16T00:17:50.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:50.618+0000 7f85f1caf640 1 -- 172.21.15.112:0/884581383 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f85d80395d0 msgr2=0x7f85d803ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:50.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:50.618+0000 7f85f1caf640 1 --2- 172.21.15.112:0/884581383 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f85d80395d0 0x7f85d803ba80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f85d4006fd0 tx=0x7f85d4006db0 comp rx=0 tx=0).stop 2024-09-16T00:17:50.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:50.618+0000 7f85f1caf640 1 -- 172.21.15.112:0/884581383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85ec0661a0 msgr2=0x7f85ec10c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:50.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:50.618+0000 7f85f1caf640 1 --2- 172.21.15.112:0/884581383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85ec0661a0 0x7f85ec10c710 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f85dc02a6d0 tx=0x7f85dc009da0 comp rx=0 tx=0).stop 2024-09-16T00:17:50.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:50.618+0000 7f85f1caf640 1 -- 172.21.15.112:0/884581383 shutdown_connections 2024-09-16T00:17:50.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:50.618+0000 7f85f1caf640 1 --2- 172.21.15.112:0/884581383 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f85d80395d0 0x7f85d803ba80 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:50.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:50.618+0000 7f85f1caf640 1 --2- 172.21.15.112:0/884581383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85ec0661a0 0x7f85ec10c710 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:50.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:50.618+0000 7f85f1caf640 1 -- 172.21.15.112:0/884581383 >> 172.21.15.112:0/884581383 conn(0x7f85ec0f6890 msgr2=0x7f85ec0f7170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:50.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:50.618+0000 7f85f1caf640 1 -- 172.21.15.112:0/884581383 shutdown_connections 2024-09-16T00:17:50.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:50.618+0000 7f85f1caf640 1 -- 172.21.15.112:0/884581383 wait complete. 2024-09-16T00:17:51.709 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:51 smithi112 ceph-mon[27869]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:17:51.709 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:51 smithi112 ceph-mon[27869]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:17:51.734 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi169 2024-09-16T00:17:51.735 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-09-16T00:17:51.735 DEBUG:teuthology.orchestra.run.smithi169:> dd of=/etc/ceph/ceph.conf 2024-09-16T00:17:51.754 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-09-16T00:17:51.754 DEBUG:teuthology.orchestra.run.smithi169:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-16T00:17:51.812 INFO:tasks.cephadm:Adding host smithi169 to orchestrator... 2024-09-16T00:17:51.812 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph orch host add smithi169 2024-09-16T00:17:52.197 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:17:53.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:53.681+0000 7f9996c79640 1 -- 172.21.15.112:0/2884488183 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99900fc1b0 msgr2=0x7f99900fc5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:53.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:53.681+0000 7f9996c79640 1 --2- 172.21.15.112:0/2884488183 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99900fc1b0 0x7f99900fc5c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f998c004850 tx=0x7f998c009da0 comp rx=0 tx=0).stop 2024-09-16T00:17:53.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:53.681+0000 7f9996c79640 1 -- 172.21.15.112:0/2884488183 shutdown_connections 2024-09-16T00:17:53.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:53.681+0000 7f9996c79640 1 --2- 172.21.15.112:0/2884488183 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99900fc1b0 0x7f99900fc5c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:53.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:53.681+0000 7f9996c79640 1 -- 172.21.15.112:0/2884488183 >> 172.21.15.112:0/2884488183 conn(0x7f99900f7990 msgr2=0x7f99900f9da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:53.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:53.681+0000 7f9996c79640 1 -- 172.21.15.112:0/2884488183 shutdown_connections 2024-09-16T00:17:53.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:53.681+0000 7f9996c79640 1 -- 172.21.15.112:0/2884488183 wait complete. 2024-09-16T00:17:53.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:53.682+0000 7f9996c79640 1 Processor -- start 2024-09-16T00:17:53.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:53.682+0000 7f9996c79640 1 -- start start 2024-09-16T00:17:53.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:53.682+0000 7f9996c79640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99900fc1b0 0x7f999010e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:53.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:53.682+0000 7f9996c79640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f999010ee30 con 0x7f99900fc1b0 2024-09-16T00:17:53.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:53.682+0000 7f9994a15640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99900fc1b0 0x7f999010e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:53.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:53.682+0000 7f9994a15640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99900fc1b0 0x7f999010e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45876/0 (socket says 172.21.15.112:45876) 2024-09-16T00:17:53.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:53.682+0000 7f9994a15640 1 -- 172.21.15.112:0/1752812883 learned_addr learned my addr 172.21.15.112:0/1752812883 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:17:53.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:53.683+0000 7f9994a15640 1 -- 172.21.15.112:0/1752812883 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f998c005040 con 0x7f99900fc1b0 2024-09-16T00:17:53.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:53.683+0000 7f9994a15640 1 --2- 172.21.15.112:0/1752812883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99900fc1b0 0x7f999010e8f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f998c02a6d0 tx=0x7f998c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:53.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:53.683+0000 7f99857fa640 1 -- 172.21.15.112:0/1752812883 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f998c033630 con 0x7f99900fc1b0 2024-09-16T00:17:53.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:53.683+0000 7f99857fa640 1 -- 172.21.15.112:0/1752812883 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f998c033c30 con 0x7f99900fc1b0 2024-09-16T00:17:53.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:53.683+0000 7f9996c79640 1 -- 172.21.15.112:0/1752812883 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f999010f030 con 0x7f99900fc1b0 2024-09-16T00:17:53.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:53.683+0000 7f99857fa640 1 -- 172.21.15.112:0/1752812883 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f998c03dc00 con 0x7f99900fc1b0 2024-09-16T00:17:53.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:53.683+0000 7f9996c79640 1 -- 172.21.15.112:0/1752812883 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f999010f4a0 con 0x7f99900fc1b0 2024-09-16T00:17:53.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:53.684+0000 7f99857fa640 1 -- 172.21.15.112:0/1752812883 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) v1 ==== 45564+0+0 (secure 0 0 0) 0x7f998c030020 con 0x7f99900fc1b0 2024-09-16T00:17:53.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:53.684+0000 7f99857fa640 1 --2- 172.21.15.112:0/1752812883 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f9978039580 0x7f997803ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:53.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:53.684+0000 7f9996c79640 1 -- 172.21.15.112:0/1752812883 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f999005ed70 con 0x7f99900fc1b0 2024-09-16T00:17:53.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:53.684+0000 7f9987fff640 1 --2- 172.21.15.112:0/1752812883 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f9978039580 0x7f997803ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:53.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:53.684+0000 7f99857fa640 1 -- 172.21.15.112:0/1752812883 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f998c06ca30 con 0x7f99900fc1b0 2024-09-16T00:17:53.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:53.684+0000 7f9987fff640 1 --2- 172.21.15.112:0/1752812883 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f9978039580 0x7f997803ba30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9980006fd0 tx=0x7f9980006db0 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:53.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:53.688+0000 7f99857fa640 1 -- 172.21.15.112:0/1752812883 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f998c03bb50 con 0x7f99900fc1b0 2024-09-16T00:17:53.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:53.853+0000 7f9996c79640 1 -- 172.21.15.112:0/1752812883 --> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi169", "target": ["mon-mgr", ""]}) v1 -- 0x7f999006fec0 con 0x7f9978039580 2024-09-16T00:17:54.447 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:54 smithi112 ceph-mon[27869]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi169", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:17:54.447 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:54 smithi112 ceph-mon[27869]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:17:54.447 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:54 smithi112 ceph-mon[27869]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:17:54.447 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:54 smithi112 ceph-mon[27869]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:17:54.447 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:54 smithi112 ceph-mon[27869]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:17:55.453 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:55 smithi112 ceph-mon[27869]: Deploying daemon node-exporter.smithi112 on smithi112 2024-09-16T00:17:55.453 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:55 smithi112 ceph-mon[27869]: Deploying cephadm binary to smithi169 2024-09-16T00:17:56.701 INFO:teuthology.orchestra.run.smithi112.stdout:Added host 'smithi169' with addr '172.21.15.169' 2024-09-16T00:17:56.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:56.699+0000 7f99857fa640 1 -- 172.21.15.112:0/1752812883 <== mgr.14160 v2:172.21.15.112:6800/2021251488 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f999006fec0 con 0x7f9978039580 2024-09-16T00:17:56.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:56.702+0000 7f9996c79640 1 -- 172.21.15.112:0/1752812883 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f9978039580 msgr2=0x7f997803ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:56.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:56.702+0000 7f9996c79640 1 --2- 172.21.15.112:0/1752812883 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f9978039580 0x7f997803ba30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9980006fd0 tx=0x7f9980006db0 comp rx=0 tx=0).stop 2024-09-16T00:17:56.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:56.702+0000 7f9996c79640 1 -- 172.21.15.112:0/1752812883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99900fc1b0 msgr2=0x7f999010e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:56.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:56.702+0000 7f9996c79640 1 --2- 172.21.15.112:0/1752812883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99900fc1b0 0x7f999010e8f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f998c02a6d0 tx=0x7f998c009da0 comp rx=0 tx=0).stop 2024-09-16T00:17:56.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:56.702+0000 7f9996c79640 1 -- 172.21.15.112:0/1752812883 shutdown_connections 2024-09-16T00:17:56.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:56.702+0000 7f9996c79640 1 --2- 172.21.15.112:0/1752812883 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f9978039580 0x7f997803ba30 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:56.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:56.702+0000 7f9996c79640 1 --2- 172.21.15.112:0/1752812883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99900fc1b0 0x7f999010e8f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:56.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:56.702+0000 7f9996c79640 1 -- 172.21.15.112:0/1752812883 >> 172.21.15.112:0/1752812883 conn(0x7f99900f7990 msgr2=0x7f99900f9d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:56.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:56.702+0000 7f9996c79640 1 -- 172.21.15.112:0/1752812883 shutdown_connections 2024-09-16T00:17:56.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:56.702+0000 7f9996c79640 1 -- 172.21.15.112:0/1752812883 wait complete. 2024-09-16T00:17:56.766 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:56 smithi112 ceph-mon[27869]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:17:56.766 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:56 smithi112 ceph-mon[27869]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:17:57.796 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-16T00:17:58.009 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:57 smithi112 ceph-mon[27869]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:17:58.010 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:57 smithi112 ceph-mon[27869]: Added host smithi169 2024-09-16T00:17:58.244 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:17:59.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:58 smithi112 ceph-mon[27869]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:17:59.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:58 smithi112 ceph-mon[27869]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:17:59.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:58 smithi112 ceph-mon[27869]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:17:59.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:58 smithi112 ceph-mon[27869]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:17:59.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:58 smithi112 ceph-mon[27869]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:17:59.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:59.912+0000 7f4bba3ad640 1 -- 172.21.15.112:0/4186417453 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bb40fd810 msgr2=0x7f4bb40fdc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:59.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:59.912+0000 7f4bba3ad640 1 --2- 172.21.15.112:0/4186417453 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bb40fd810 0x7f4bb40fdc20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4004850 tx=0x7f4ba4009da0 comp rx=0 tx=0).stop 2024-09-16T00:17:59.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:59.913+0000 7f4bba3ad640 1 -- 172.21.15.112:0/4186417453 shutdown_connections 2024-09-16T00:17:59.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:59.913+0000 7f4bba3ad640 1 --2- 172.21.15.112:0/4186417453 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bb40fd810 0x7f4bb40fdc20 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:59.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:59.913+0000 7f4bba3ad640 1 -- 172.21.15.112:0/4186417453 >> 172.21.15.112:0/4186417453 conn(0x7f4bb40f6010 msgr2=0x7f4bb40f8440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:59.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:59.913+0000 7f4bba3ad640 1 -- 172.21.15.112:0/4186417453 shutdown_connections 2024-09-16T00:17:59.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:59.913+0000 7f4bba3ad640 1 -- 172.21.15.112:0/4186417453 wait complete. 2024-09-16T00:17:59.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:59.913+0000 7f4bba3ad640 1 Processor -- start 2024-09-16T00:17:59.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:59.913+0000 7f4bba3ad640 1 -- start start 2024-09-16T00:17:59.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:59.913+0000 7f4bba3ad640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bb40fd810 0x7f4bb410e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:59.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:59.913+0000 7f4bba3ad640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb410ee80 con 0x7f4bb40fd810 2024-09-16T00:17:59.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:59.913+0000 7f4bb3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bb40fd810 0x7f4bb410e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:59.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:59.913+0000 7f4bb3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bb40fd810 0x7f4bb410e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45924/0 (socket says 172.21.15.112:45924) 2024-09-16T00:17:59.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:59.913+0000 7f4bb3fff640 1 -- 172.21.15.112:0/4092598595 learned_addr learned my addr 172.21.15.112:0/4092598595 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:17:59.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:59.914+0000 7f4bb3fff640 1 -- 172.21.15.112:0/4092598595 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ba4005040 con 0x7f4bb40fd810 2024-09-16T00:17:59.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:59.914+0000 7f4bb3fff640 1 --2- 172.21.15.112:0/4092598595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bb40fd810 0x7f4bb410e940 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4009920 tx=0x7f4ba4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:59.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:59.914+0000 7f4bb17fa640 1 -- 172.21.15.112:0/4092598595 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4ba4033630 con 0x7f4bb40fd810 2024-09-16T00:17:59.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:59.914+0000 7f4bb17fa640 1 -- 172.21.15.112:0/4092598595 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f4ba4033c30 con 0x7f4bb40fd810 2024-09-16T00:17:59.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:59.914+0000 7f4bba3ad640 1 -- 172.21.15.112:0/4092598595 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bb410f080 con 0x7f4bb40fd810 2024-09-16T00:17:59.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:59.914+0000 7f4bb17fa640 1 -- 172.21.15.112:0/4092598595 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4ba403dbb0 con 0x7f4bb40fd810 2024-09-16T00:17:59.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:59.914+0000 7f4bba3ad640 1 -- 172.21.15.112:0/4092598595 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bb410f4f0 con 0x7f4bb40fd810 2024-09-16T00:17:59.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:59.915+0000 7f4bb17fa640 1 -- 172.21.15.112:0/4092598595 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) v1 ==== 45564+0+0 (secure 0 0 0) 0x7f4ba4030020 con 0x7f4bb40fd810 2024-09-16T00:17:59.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:59.915+0000 7f4bba3ad640 1 -- 172.21.15.112:0/4092598595 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bb405ed70 con 0x7f4bb40fd810 2024-09-16T00:17:59.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:59.915+0000 7f4bb17fa640 1 --2- 172.21.15.112:0/4092598595 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f4ba0039580 0x7f4ba003ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:59.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:59.915+0000 7f4bb17fa640 1 -- 172.21.15.112:0/4092598595 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f4ba406c850 con 0x7f4bb40fd810 2024-09-16T00:17:59.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:59.915+0000 7f4babfff640 1 --2- 172.21.15.112:0/4092598595 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f4ba0039580 0x7f4ba003ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:59.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:59.916+0000 7f4babfff640 1 --2- 172.21.15.112:0/4092598595 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f4ba0039580 0x7f4ba003ba30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4b9c006fd0 tx=0x7f4b9c006db0 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:59.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:17:59.920+0000 7f4bb17fa640 1 -- 172.21.15.112:0/4092598595 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f4ba4033790 con 0x7f4bb40fd810 2024-09-16T00:18:00.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:17:59 smithi112 ceph-mon[27869]: Deploying daemon alertmanager.smithi112 on smithi112 2024-09-16T00:18:00.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:18:00.093+0000 7f4bba3ad640 1 -- 172.21.15.112:0/4092598595 --> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4bb410f4f0 con 0x7f4ba0039580 2024-09-16T00:18:00.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:18:00.094+0000 7f4bb17fa640 1 -- 172.21.15.112:0/4092598595 <== mgr.14160 v2:172.21.15.112:6800/2021251488 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+161 (secure 0 0 0) 0x7f4bb410f4f0 con 0x7f4ba0039580 2024-09-16T00:18:00.095 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:18:00.095 INFO:teuthology.orchestra.run.smithi112.stdout:[{"addr": "172.21.15.112", "hostname": "smithi112", "labels": [], "status": ""}, {"addr": "172.21.15.169", "hostname": "smithi169", "labels": [], "status": ""}] 2024-09-16T00:18:00.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:18:00.096+0000 7f4bba3ad640 1 -- 172.21.15.112:0/4092598595 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f4ba0039580 msgr2=0x7f4ba003ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:00.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:18:00.096+0000 7f4bba3ad640 1 --2- 172.21.15.112:0/4092598595 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f4ba0039580 0x7f4ba003ba30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4b9c006fd0 tx=0x7f4b9c006db0 comp rx=0 tx=0).stop 2024-09-16T00:18:00.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:18:00.096+0000 7f4bba3ad640 1 -- 172.21.15.112:0/4092598595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bb40fd810 msgr2=0x7f4bb410e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:00.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:18:00.096+0000 7f4bba3ad640 1 --2- 172.21.15.112:0/4092598595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bb40fd810 0x7f4bb410e940 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4009920 tx=0x7f4ba4009da0 comp rx=0 tx=0).stop 2024-09-16T00:18:00.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:18:00.097+0000 7f4bba3ad640 1 -- 172.21.15.112:0/4092598595 shutdown_connections 2024-09-16T00:18:00.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:18:00.097+0000 7f4bba3ad640 1 --2- 172.21.15.112:0/4092598595 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f4ba0039580 0x7f4ba003ba30 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:00.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:18:00.097+0000 7f4bba3ad640 1 --2- 172.21.15.112:0/4092598595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bb40fd810 0x7f4bb410e940 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:00.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:18:00.097+0000 7f4bba3ad640 1 -- 172.21.15.112:0/4092598595 >> 172.21.15.112:0/4092598595 conn(0x7f4bb40f6010 msgr2=0x7f4bb40f8440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:00.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:18:00.097+0000 7f4bba3ad640 1 -- 172.21.15.112:0/4092598595 shutdown_connections 2024-09-16T00:18:00.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:18:00.097+0000 7f4bba3ad640 1 -- 172.21.15.112:0/4092598595 wait complete. 2024-09-16T00:18:00.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:00 smithi112 ceph-mon[27869]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:18:00.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:00 smithi112 ceph-mon[27869]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:18:00.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:00 smithi112 ceph-mon[27869]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:18:01.186 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-16T00:18:01.187 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph osd crush tunables default 2024-09-16T00:18:01.427 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:18:03.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:02 smithi112 ceph-mon[27869]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:18:04.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:18:04.033+0000 7f43fa6e8640 1 -- 172.21.15.112:0/558480780 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43f40fb0c0 msgr2=0x7f43f40fb4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:04.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:18:04.033+0000 7f43fa6e8640 1 --2- 172.21.15.112:0/558480780 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43f40fb0c0 0x7f43f40fb4d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f43e4004850 tx=0x7f43e4009da0 comp rx=0 tx=0).stop 2024-09-16T00:18:04.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:18:04.033+0000 7f43fa6e8640 1 -- 172.21.15.112:0/558480780 shutdown_connections 2024-09-16T00:18:04.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:18:04.033+0000 7f43fa6e8640 1 --2- 172.21.15.112:0/558480780 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43f40fb0c0 0x7f43f40fb4d0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:04.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:18:04.033+0000 7f43fa6e8640 1 -- 172.21.15.112:0/558480780 >> 172.21.15.112:0/558480780 conn(0x7f43f40f6880 msgr2=0x7f43f40f8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:04.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:18:04.033+0000 7f43fa6e8640 1 -- 172.21.15.112:0/558480780 shutdown_connections 2024-09-16T00:18:04.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:18:04.033+0000 7f43fa6e8640 1 -- 172.21.15.112:0/558480780 wait complete. 2024-09-16T00:18:04.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:18:04.034+0000 7f43fa6e8640 1 Processor -- start 2024-09-16T00:18:04.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:18:04.034+0000 7f43fa6e8640 1 -- start start 2024-09-16T00:18:04.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:18:04.034+0000 7f43fa6e8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43f40fb0c0 0x7f43f410b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:04.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:18:04.034+0000 7f43fa6e8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f410bba0 con 0x7f43f40fb0c0 2024-09-16T00:18:04.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:18:04.034+0000 7f43f3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43f40fb0c0 0x7f43f410b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:04.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:18:04.034+0000 7f43f3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43f40fb0c0 0x7f43f410b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57092/0 (socket says 172.21.15.112:57092) 2024-09-16T00:18:04.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:18:04.034+0000 7f43f3fff640 1 -- 172.21.15.112:0/2577888393 learned_addr learned my addr 172.21.15.112:0/2577888393 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:18:04.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:18:04.035+0000 7f43f3fff640 1 -- 172.21.15.112:0/2577888393 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43e4005040 con 0x7f43f40fb0c0 2024-09-16T00:18:04.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:18:04.035+0000 7f43f3fff640 1 --2- 172.21.15.112:0/2577888393 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43f40fb0c0 0x7f43f410b660 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f43e402a6d0 tx=0x7f43e4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:04.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:18:04.035+0000 7f43f0ff9640 1 -- 172.21.15.112:0/2577888393 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f43e4033630 con 0x7f43f40fb0c0 2024-09-16T00:18:04.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:18:04.035+0000 7f43fa6e8640 1 -- 172.21.15.112:0/2577888393 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43f410bda0 con 0x7f43f40fb0c0 2024-09-16T00:18:04.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:18:04.035+0000 7f43f0ff9640 1 -- 172.21.15.112:0/2577888393 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f43e4033c30 con 0x7f43f40fb0c0 2024-09-16T00:18:04.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:18:04.035+0000 7f43f0ff9640 1 -- 172.21.15.112:0/2577888393 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f43e403dc30 con 0x7f43f40fb0c0 2024-09-16T00:18:04.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:18:04.035+0000 7f43fa6e8640 1 -- 172.21.15.112:0/2577888393 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43f410c210 con 0x7f43f40fb0c0 2024-09-16T00:18:04.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:18:04.036+0000 7f43f0ff9640 1 -- 172.21.15.112:0/2577888393 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) v1 ==== 45564+0+0 (secure 0 0 0) 0x7f43e4030020 con 0x7f43f40fb0c0 2024-09-16T00:18:04.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:18:04.036+0000 7f43fa6e8640 1 -- 172.21.15.112:0/2577888393 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43f405ed70 con 0x7f43f40fb0c0 2024-09-16T00:18:04.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:18:04.036+0000 7f43f0ff9640 1 --2- 172.21.15.112:0/2577888393 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f43dc039580 0x7f43dc03ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:04.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:18:04.036+0000 7f43f0ff9640 1 -- 172.21.15.112:0/2577888393 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f43e406c980 con 0x7f43f40fb0c0 2024-09-16T00:18:04.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:18:04.036+0000 7f43f37fe640 1 --2- 172.21.15.112:0/2577888393 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f43dc039580 0x7f43dc03ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:04.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:18:04.037+0000 7f43f37fe640 1 --2- 172.21.15.112:0/2577888393 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f43dc039580 0x7f43dc03ba30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f43e8006fd0 tx=0x7f43e8006db0 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:04.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:18:04.041+0000 7f43f0ff9640 1 -- 172.21.15.112:0/2577888393 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f43e40463e0 con 0x7f43f40fb0c0 2024-09-16T00:18:04.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:18:04.201+0000 7f43fa6e8640 1 -- 172.21.15.112:0/2577888393 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f43f404b720 con 0x7f43f40fb0c0 2024-09-16T00:18:04.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:18:04.715+0000 7f43f0ff9640 1 -- 172.21.15.112:0/2577888393 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f43e403b370 con 0x7f43f40fb0c0 2024-09-16T00:18:04.715 INFO:teuthology.orchestra.run.smithi112.stderr:adjusted tunables profile to default 2024-09-16T00:18:04.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:18:04.717+0000 7f43fa6e8640 1 -- 172.21.15.112:0/2577888393 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f43dc039580 msgr2=0x7f43dc03ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:04.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:18:04.717+0000 7f43fa6e8640 1 --2- 172.21.15.112:0/2577888393 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f43dc039580 0x7f43dc03ba30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f43e8006fd0 tx=0x7f43e8006db0 comp rx=0 tx=0).stop 2024-09-16T00:18:04.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:18:04.717+0000 7f43fa6e8640 1 -- 172.21.15.112:0/2577888393 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43f40fb0c0 msgr2=0x7f43f410b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:04.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:18:04.717+0000 7f43fa6e8640 1 --2- 172.21.15.112:0/2577888393 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43f40fb0c0 0x7f43f410b660 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f43e402a6d0 tx=0x7f43e4009da0 comp rx=0 tx=0).stop 2024-09-16T00:18:04.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:18:04.717+0000 7f43fa6e8640 1 -- 172.21.15.112:0/2577888393 shutdown_connections 2024-09-16T00:18:04.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:18:04.717+0000 7f43fa6e8640 1 --2- 172.21.15.112:0/2577888393 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f43dc039580 0x7f43dc03ba30 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:04.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:18:04.717+0000 7f43fa6e8640 1 --2- 172.21.15.112:0/2577888393 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43f40fb0c0 0x7f43f410b660 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:04.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:18:04.717+0000 7f43fa6e8640 1 -- 172.21.15.112:0/2577888393 >> 172.21.15.112:0/2577888393 conn(0x7f43f40f6880 msgr2=0x7f43f40f71c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:04.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:18:04.717+0000 7f43fa6e8640 1 -- 172.21.15.112:0/2577888393 shutdown_connections 2024-09-16T00:18:04.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:18:04.717+0000 7f43fa6e8640 1 -- 172.21.15.112:0/2577888393 wait complete. 2024-09-16T00:18:05.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:04 smithi112 ceph-mon[27869]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:18:05.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:04 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/2577888393' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-16T00:18:05.887 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:05 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/2577888393' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-16T00:18:05.887 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:05 smithi112 ceph-mon[27869]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T00:18:07.002 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:06 smithi112 ceph-mon[27869]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:18:07.032 INFO:tasks.cephadm:Adding mon.smithi112 on smithi112 2024-09-16T00:18:07.032 INFO:tasks.cephadm:Adding mon.smithi169 on smithi169 2024-09-16T00:18:07.032 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph orch apply mon '2;smithi112:172.21.15.112=smithi112;smithi169:172.21.15.169=smithi169' 2024-09-16T00:18:07.272 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T00:18:07.317 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T00:18:08.875 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:08.873+0000 7f802d040640 1 -- 172.21.15.169:0/998969700 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80280fb0c0 msgr2=0x7f80280fb4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:08.875 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:08.873+0000 7f802d040640 1 --2- 172.21.15.169:0/998969700 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80280fb0c0 0x7f80280fb4d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8018004850 tx=0x7f8018009da0 comp rx=0 tx=0).stop 2024-09-16T00:18:08.876 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:08.873+0000 7f802d040640 1 -- 172.21.15.169:0/998969700 shutdown_connections 2024-09-16T00:18:08.876 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:08.873+0000 7f802d040640 1 --2- 172.21.15.169:0/998969700 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80280fb0c0 0x7f80280fb4d0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:08.876 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:08.873+0000 7f802d040640 1 -- 172.21.15.169:0/998969700 >> 172.21.15.169:0/998969700 conn(0x7f80280f6880 msgr2=0x7f80280f8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:08.876 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:08.873+0000 7f802d040640 1 -- 172.21.15.169:0/998969700 shutdown_connections 2024-09-16T00:18:08.876 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:08.873+0000 7f802d040640 1 -- 172.21.15.169:0/998969700 wait complete. 2024-09-16T00:18:08.876 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:08.874+0000 7f802d040640 1 Processor -- start 2024-09-16T00:18:08.876 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:08.874+0000 7f802d040640 1 -- start start 2024-09-16T00:18:08.876 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:08.874+0000 7f802d040640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80280fb0c0 0x7f8028066040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:08.876 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:08.874+0000 7f802d040640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f802810f410 con 0x7f80280fb0c0 2024-09-16T00:18:08.876 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:08.874+0000 7f8026d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80280fb0c0 0x7f8028066040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:08.876 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:08.874+0000 7f8026d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80280fb0c0 0x7f8028066040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.169:43894/0 (socket says 172.21.15.169:43894) 2024-09-16T00:18:08.876 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:08.874+0000 7f8026d9d640 1 -- 172.21.15.169:0/3713346585 learned_addr learned my addr 172.21.15.169:0/3713346585 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-09-16T00:18:08.877 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:08.875+0000 7f8026d9d640 1 -- 172.21.15.169:0/3713346585 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8018005040 con 0x7f80280fb0c0 2024-09-16T00:18:08.877 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:08.875+0000 7f8026d9d640 1 --2- 172.21.15.169:0/3713346585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80280fb0c0 0x7f8028066040 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8018009920 tx=0x7f8018009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:08.877 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:08.875+0000 7f800f7fe640 1 -- 172.21.15.169:0/3713346585 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8018033630 con 0x7f80280fb0c0 2024-09-16T00:18:08.877 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:08.875+0000 7f800f7fe640 1 -- 172.21.15.169:0/3713346585 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f8018033c30 con 0x7f80280fb0c0 2024-09-16T00:18:08.877 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:08.875+0000 7f800f7fe640 1 -- 172.21.15.169:0/3713346585 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f801803dc40 con 0x7f80280fb0c0 2024-09-16T00:18:08.877 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:08.875+0000 7f802d040640 1 -- 172.21.15.169:0/3713346585 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f802810f550 con 0x7f80280fb0c0 2024-09-16T00:18:08.877 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:08.875+0000 7f802d040640 1 -- 172.21.15.169:0/3713346585 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f802810f960 con 0x7f80280fb0c0 2024-09-16T00:18:08.878 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:08.876+0000 7f800f7fe640 1 -- 172.21.15.169:0/3713346585 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) v1 ==== 45564+0+0 (secure 0 0 0) 0x7f8018030020 con 0x7f80280fb0c0 2024-09-16T00:18:08.878 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:08.876+0000 7f802d040640 1 -- 172.21.15.169:0/3713346585 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f802805ed70 con 0x7f80280fb0c0 2024-09-16T00:18:08.878 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:08.876+0000 7f800f7fe640 1 --2- 172.21.15.169:0/3713346585 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f8010039580 0x7f801003ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:08.878 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:08.876+0000 7f800f7fe640 1 -- 172.21.15.169:0/3713346585 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f801806cb00 con 0x7f80280fb0c0 2024-09-16T00:18:08.878 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:08.876+0000 7f802659c640 1 --2- 172.21.15.169:0/3713346585 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f8010039580 0x7f801003ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:08.878 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:08.877+0000 7f802659c640 1 --2- 172.21.15.169:0/3713346585 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f8010039580 0x7f801003ba30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f801c006fd0 tx=0x7f801c006db0 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:08.881 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:08.881+0000 7f800f7fe640 1 -- 172.21.15.169:0/3713346585 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f801806ee60 con 0x7f80280fb0c0 2024-09-16T00:18:09.045 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:09.045+0000 7f802d040640 1 -- 172.21.15.169:0/3713346585 --> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi112:172.21.15.112=smithi112;smithi169:172.21.15.169=smithi169", "target": ["mon-mgr", ""]}) v1 -- 0x7f802802ac20 con 0x7f8010039580 2024-09-16T00:18:09.051 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:09.051+0000 7f800f7fe640 1 -- 172.21.15.169:0/3713346585 <== mgr.14160 v2:172.21.15.112:6800/2021251488 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f802802ac20 con 0x7f8010039580 2024-09-16T00:18:09.051 INFO:teuthology.orchestra.run.smithi169.stdout:Scheduled mon update... 2024-09-16T00:18:09.053 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:09.053+0000 7f802d040640 1 -- 172.21.15.169:0/3713346585 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f8010039580 msgr2=0x7f801003ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:09.053 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:09.053+0000 7f802d040640 1 --2- 172.21.15.169:0/3713346585 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f8010039580 0x7f801003ba30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f801c006fd0 tx=0x7f801c006db0 comp rx=0 tx=0).stop 2024-09-16T00:18:09.053 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:09.053+0000 7f802d040640 1 -- 172.21.15.169:0/3713346585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80280fb0c0 msgr2=0x7f8028066040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:09.054 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:09.053+0000 7f802d040640 1 --2- 172.21.15.169:0/3713346585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80280fb0c0 0x7f8028066040 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8018009920 tx=0x7f8018009da0 comp rx=0 tx=0).stop 2024-09-16T00:18:09.054 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:09.053+0000 7f802d040640 1 -- 172.21.15.169:0/3713346585 shutdown_connections 2024-09-16T00:18:09.054 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:09.053+0000 7f802d040640 1 --2- 172.21.15.169:0/3713346585 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f8010039580 0x7f801003ba30 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:09.054 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:09.053+0000 7f802d040640 1 --2- 172.21.15.169:0/3713346585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80280fb0c0 0x7f8028066040 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:09.054 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:09.053+0000 7f802d040640 1 -- 172.21.15.169:0/3713346585 >> 172.21.15.169:0/3713346585 conn(0x7f80280f6880 msgr2=0x7f80280f70e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:09.054 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:09.053+0000 7f802d040640 1 -- 172.21.15.169:0/3713346585 shutdown_connections 2024-09-16T00:18:09.054 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:09.053+0000 7f802d040640 1 -- 172.21.15.169:0/3713346585 wait complete. 2024-09-16T00:18:09.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:08 smithi112 ceph-mon[27869]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:18:09.681 DEBUG:teuthology.orchestra.run.smithi169:mon.smithi169> sudo journalctl -f -n 0 -u ceph-d346dd56-73c0-11ef-bceb-c7b262605968@mon.smithi169.service 2024-09-16T00:18:09.683 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-16T00:18:09.683 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T00:18:09.952 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T00:18:09.997 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T00:18:10.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:09 smithi112 ceph-mon[27869]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:18:10.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:09 smithi112 ceph-mon[27869]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:18:10.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:09 smithi112 ceph-mon[27869]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:18:10.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:09 smithi112 ceph-mon[27869]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:18:10.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:09 smithi112 ceph-mon[27869]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:18:10.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:09 smithi112 ceph-mon[27869]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:18:10.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:09 smithi112 ceph-mon[27869]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-16T00:18:10.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:09 smithi112 ceph-mon[27869]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-16T00:18:10.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:09 smithi112 ceph-mon[27869]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:18:10.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:09 smithi112 ceph-mon[27869]: Deploying daemon grafana.smithi112 on smithi112 2024-09-16T00:18:10.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:09 smithi112 ceph-mon[27869]: from='client.14192 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi112:172.21.15.112=smithi112;smithi169:172.21.15.169=smithi169", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:18:10.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:09 smithi112 ceph-mon[27869]: Saving service mon spec with placement smithi112:172.21.15.112=smithi112;smithi169:172.21.15.169=smithi169;count:2 2024-09-16T00:18:10.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:09 smithi112 ceph-mon[27869]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:18:11.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:11 smithi112 ceph-mon[27869]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:18:11.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:11 smithi112 ceph-mon[27869]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:18:11.460 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:11.458+0000 7f41cfa37640 1 -- 172.21.15.169:0/3339222640 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41c8065610 msgr2=0x7f41c80ff030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:11.460 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:11.458+0000 7f41cfa37640 1 --2- 172.21.15.169:0/3339222640 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41c8065610 0x7f41c80ff030 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f41c4004850 tx=0x7f41c4009da0 comp rx=0 tx=0).stop 2024-09-16T00:18:11.460 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:11.458+0000 7f41cfa37640 1 -- 172.21.15.169:0/3339222640 shutdown_connections 2024-09-16T00:18:11.460 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:11.458+0000 7f41cfa37640 1 --2- 172.21.15.169:0/3339222640 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41c8065610 0x7f41c80ff030 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:11.460 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:11.458+0000 7f41cfa37640 1 -- 172.21.15.169:0/3339222640 >> 172.21.15.169:0/3339222640 conn(0x7f41c80f6ac0 msgr2=0x7f41c80f8ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:11.461 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:11.458+0000 7f41cfa37640 1 -- 172.21.15.169:0/3339222640 shutdown_connections 2024-09-16T00:18:11.461 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:11.458+0000 7f41cfa37640 1 -- 172.21.15.169:0/3339222640 wait complete. 2024-09-16T00:18:11.461 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:11.458+0000 7f41cfa37640 1 Processor -- start 2024-09-16T00:18:11.461 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:11.459+0000 7f41cfa37640 1 -- start start 2024-09-16T00:18:11.461 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:11.459+0000 7f41cfa37640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41c8065610 0x7f41c810c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:11.461 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:11.459+0000 7f41cfa37640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41c810cee0 con 0x7f41c8065610 2024-09-16T00:18:11.461 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:11.459+0000 7f41cd7d3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41c8065610 0x7f41c810c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:11.461 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:11.459+0000 7f41cd7d3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41c8065610 0x7f41c810c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.169:43908/0 (socket says 172.21.15.169:43908) 2024-09-16T00:18:11.461 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:11.459+0000 7f41cd7d3640 1 -- 172.21.15.169:0/1622913869 learned_addr learned my addr 172.21.15.169:0/1622913869 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-09-16T00:18:11.462 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:11.459+0000 7f41cd7d3640 1 -- 172.21.15.169:0/1622913869 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41c4005040 con 0x7f41c8065610 2024-09-16T00:18:11.463 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:11.459+0000 7f41cd7d3640 1 --2- 172.21.15.169:0/1622913869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41c8065610 0x7f41c810c9a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f41c402a6d0 tx=0x7f41c4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:11.463 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:11.460+0000 7f41bdffb640 1 -- 172.21.15.169:0/1622913869 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f41c4033630 con 0x7f41c8065610 2024-09-16T00:18:11.463 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:11.460+0000 7f41bdffb640 1 -- 172.21.15.169:0/1622913869 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f41c4033c30 con 0x7f41c8065610 2024-09-16T00:18:11.463 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:11.460+0000 7f41cfa37640 1 -- 172.21.15.169:0/1622913869 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41c810d0e0 con 0x7f41c8065610 2024-09-16T00:18:11.463 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:11.460+0000 7f41bdffb640 1 -- 172.21.15.169:0/1622913869 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f41c403dc30 con 0x7f41c8065610 2024-09-16T00:18:11.463 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:11.460+0000 7f41cfa37640 1 -- 172.21.15.169:0/1622913869 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41c810d550 con 0x7f41c8065610 2024-09-16T00:18:11.463 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:11.461+0000 7f41bdffb640 1 -- 172.21.15.169:0/1622913869 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) v1 ==== 45564+0+0 (secure 0 0 0) 0x7f41c4030020 con 0x7f41c8065610 2024-09-16T00:18:11.463 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:11.461+0000 7f41cfa37640 1 -- 172.21.15.169:0/1622913869 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41c805ed70 con 0x7f41c8065610 2024-09-16T00:18:11.463 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:11.461+0000 7f41bdffb640 1 --2- 172.21.15.169:0/1622913869 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f41b4039580 0x7f41b403ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:11.463 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:11.461+0000 7f41bdffb640 1 -- 172.21.15.169:0/1622913869 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f41c406ca20 con 0x7f41c8065610 2024-09-16T00:18:11.463 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:11.461+0000 7f41ccfd2640 1 --2- 172.21.15.169:0/1622913869 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f41b4039580 0x7f41b403ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:11.463 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:11.461+0000 7f41ccfd2640 1 --2- 172.21.15.169:0/1622913869 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f41b4039580 0x7f41b403ba30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f41b8006fd0 tx=0x7f41b8006db0 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:11.470 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:11.469+0000 7f41bdffb640 1 -- 172.21.15.169:0/1622913869 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f41c4045b80 con 0x7f41c8065610 2024-09-16T00:18:11.682 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:11.681+0000 7f41cfa37640 1 -- 172.21.15.169:0/1622913869 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f41c804b720 con 0x7f41c8065610 2024-09-16T00:18:11.683 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:11.682+0000 7f41bdffb640 1 -- 172.21.15.169:0/1622913869 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f41c4045b80 con 0x7f41c8065610 2024-09-16T00:18:11.683 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:18:11.683 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":1,"fsid":"d346dd56-73c0-11ef-bceb-c7b262605968","modified":"2024-09-16T00:16:40.582056Z","created":"2024-09-16T00:16:40.582056Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi112","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-16T00:18:11.684 INFO:teuthology.orchestra.run.smithi169.stderr:dumped monmap epoch 1 2024-09-16T00:18:11.685 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:11.684+0000 7f41cfa37640 1 -- 172.21.15.169:0/1622913869 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f41b4039580 msgr2=0x7f41b403ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:11.685 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:11.684+0000 7f41cfa37640 1 --2- 172.21.15.169:0/1622913869 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f41b4039580 0x7f41b403ba30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f41b8006fd0 tx=0x7f41b8006db0 comp rx=0 tx=0).stop 2024-09-16T00:18:11.685 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:11.684+0000 7f41cfa37640 1 -- 172.21.15.169:0/1622913869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41c8065610 msgr2=0x7f41c810c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:11.686 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:11.684+0000 7f41cfa37640 1 --2- 172.21.15.169:0/1622913869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41c8065610 0x7f41c810c9a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f41c402a6d0 tx=0x7f41c4009da0 comp rx=0 tx=0).stop 2024-09-16T00:18:11.686 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:11.684+0000 7f41cfa37640 1 -- 172.21.15.169:0/1622913869 shutdown_connections 2024-09-16T00:18:11.686 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:11.684+0000 7f41cfa37640 1 --2- 172.21.15.169:0/1622913869 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f41b4039580 0x7f41b403ba30 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:11.686 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:11.684+0000 7f41cfa37640 1 --2- 172.21.15.169:0/1622913869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41c8065610 0x7f41c810c9a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:11.686 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:11.684+0000 7f41cfa37640 1 -- 172.21.15.169:0/1622913869 >> 172.21.15.169:0/1622913869 conn(0x7f41c80f6ac0 msgr2=0x7f41c80f7350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:11.686 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:11.684+0000 7f41cfa37640 1 -- 172.21.15.169:0/1622913869 shutdown_connections 2024-09-16T00:18:11.686 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:11.684+0000 7f41cfa37640 1 -- 172.21.15.169:0/1622913869 wait complete. 2024-09-16T00:18:12.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:12 smithi112 ceph-mon[27869]: from='client.? 172.21.15.169:0/1622913869' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T00:18:13.178 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-16T00:18:13.178 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T00:18:13.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:13 smithi112 ceph-mon[27869]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:18:13.420 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T00:18:13.465 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T00:18:14.861 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:14.854+0000 7fd7b04a8640 1 -- 172.21.15.169:0/3636893725 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7a80fb2e0 msgr2=0x7fd7a80fb6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:14.861 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:14.854+0000 7fd7b04a8640 1 --2- 172.21.15.169:0/3636893725 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7a80fb2e0 0x7fd7a80fb6f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd7a4004850 tx=0x7fd7a4009da0 comp rx=0 tx=0).stop 2024-09-16T00:18:14.862 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:14.854+0000 7fd7b04a8640 1 -- 172.21.15.169:0/3636893725 shutdown_connections 2024-09-16T00:18:14.862 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:14.854+0000 7fd7b04a8640 1 --2- 172.21.15.169:0/3636893725 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7a80fb2e0 0x7fd7a80fb6f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:14.862 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:14.854+0000 7fd7b04a8640 1 -- 172.21.15.169:0/3636893725 >> 172.21.15.169:0/3636893725 conn(0x7fd7a80f6ac0 msgr2=0x7fd7a80f8ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:14.862 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:14.854+0000 7fd7b04a8640 1 -- 172.21.15.169:0/3636893725 shutdown_connections 2024-09-16T00:18:14.862 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:14.854+0000 7fd7b04a8640 1 -- 172.21.15.169:0/3636893725 wait complete. 2024-09-16T00:18:14.862 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:14.855+0000 7fd7b04a8640 1 Processor -- start 2024-09-16T00:18:14.862 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:14.855+0000 7fd7b04a8640 1 -- start start 2024-09-16T00:18:14.862 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:14.855+0000 7fd7b04a8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7a80fb2e0 0x7fd7a80654d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:14.862 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:14.855+0000 7fd7b04a8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a810d450 con 0x7fd7a80fb2e0 2024-09-16T00:18:14.862 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:14.855+0000 7fd7ae244640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7a80fb2e0 0x7fd7a80654d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:14.862 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:14.855+0000 7fd7ae244640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7a80fb2e0 0x7fd7a80654d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.169:40366/0 (socket says 172.21.15.169:40366) 2024-09-16T00:18:14.862 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:14.855+0000 7fd7ae244640 1 -- 172.21.15.169:0/765597099 learned_addr learned my addr 172.21.15.169:0/765597099 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-09-16T00:18:14.863 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:14.855+0000 7fd7ae244640 1 -- 172.21.15.169:0/765597099 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7a4005040 con 0x7fd7a80fb2e0 2024-09-16T00:18:14.863 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:14.855+0000 7fd7ae244640 1 --2- 172.21.15.169:0/765597099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7a80fb2e0 0x7fd7a80654d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd7a402a6d0 tx=0x7fd7a4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:14.863 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:14.856+0000 7fd79effd640 1 -- 172.21.15.169:0/765597099 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd7a4033630 con 0x7fd7a80fb2e0 2024-09-16T00:18:14.863 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:14.856+0000 7fd7b04a8640 1 -- 172.21.15.169:0/765597099 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7a8111a60 con 0x7fd7a80fb2e0 2024-09-16T00:18:14.863 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:14.856+0000 7fd79effd640 1 -- 172.21.15.169:0/765597099 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fd7a4033c30 con 0x7fd7a80fb2e0 2024-09-16T00:18:14.863 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:14.856+0000 7fd79effd640 1 -- 172.21.15.169:0/765597099 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd7a403dc00 con 0x7fd7a80fb2e0 2024-09-16T00:18:14.863 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:14.856+0000 7fd7b04a8640 1 -- 172.21.15.169:0/765597099 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7a8111d90 con 0x7fd7a80fb2e0 2024-09-16T00:18:14.863 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:14.857+0000 7fd7b04a8640 1 -- 172.21.15.169:0/765597099 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7a805ed70 con 0x7fd7a80fb2e0 2024-09-16T00:18:14.863 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:14.857+0000 7fd79effd640 1 -- 172.21.15.169:0/765597099 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) v1 ==== 45564+0+0 (secure 0 0 0) 0x7fd7a4030020 con 0x7fd7a80fb2e0 2024-09-16T00:18:14.863 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:14.857+0000 7fd79effd640 1 --2- 172.21.15.169:0/765597099 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7fd794039580 0x7fd79403ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:14.863 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:14.857+0000 7fd79effd640 1 -- 172.21.15.169:0/765597099 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fd7a406ca70 con 0x7fd7a80fb2e0 2024-09-16T00:18:14.863 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:14.857+0000 7fd7ada43640 1 --2- 172.21.15.169:0/765597099 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7fd794039580 0x7fd79403ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:14.863 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:14.857+0000 7fd7ada43640 1 --2- 172.21.15.169:0/765597099 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7fd794039580 0x7fd79403ba30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd798006fd0 tx=0x7fd798006db0 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:14.864 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:14.862+0000 7fd79effd640 1 -- 172.21.15.169:0/765597099 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fd7a4045800 con 0x7fd7a80fb2e0 2024-09-16T00:18:15.077 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:15.075+0000 7fd7b04a8640 1 -- 172.21.15.169:0/765597099 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fd7a804b720 con 0x7fd7a80fb2e0 2024-09-16T00:18:15.077 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:15.076+0000 7fd79effd640 1 -- 172.21.15.169:0/765597099 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7fd7a4042210 con 0x7fd7a80fb2e0 2024-09-16T00:18:15.078 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:18:15.078 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":1,"fsid":"d346dd56-73c0-11ef-bceb-c7b262605968","modified":"2024-09-16T00:16:40.582056Z","created":"2024-09-16T00:16:40.582056Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi112","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-16T00:18:15.078 INFO:teuthology.orchestra.run.smithi169.stderr:dumped monmap epoch 1 2024-09-16T00:18:15.079 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:15.078+0000 7fd7b04a8640 1 -- 172.21.15.169:0/765597099 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7fd794039580 msgr2=0x7fd79403ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:15.079 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:15.078+0000 7fd7b04a8640 1 --2- 172.21.15.169:0/765597099 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7fd794039580 0x7fd79403ba30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd798006fd0 tx=0x7fd798006db0 comp rx=0 tx=0).stop 2024-09-16T00:18:15.079 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:15.078+0000 7fd7b04a8640 1 -- 172.21.15.169:0/765597099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7a80fb2e0 msgr2=0x7fd7a80654d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:15.079 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:15.078+0000 7fd7b04a8640 1 --2- 172.21.15.169:0/765597099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7a80fb2e0 0x7fd7a80654d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd7a402a6d0 tx=0x7fd7a4009da0 comp rx=0 tx=0).stop 2024-09-16T00:18:15.080 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:15.078+0000 7fd7b04a8640 1 -- 172.21.15.169:0/765597099 shutdown_connections 2024-09-16T00:18:15.080 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:15.078+0000 7fd7b04a8640 1 --2- 172.21.15.169:0/765597099 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7fd794039580 0x7fd79403ba30 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:15.080 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:15.078+0000 7fd7b04a8640 1 --2- 172.21.15.169:0/765597099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7a80fb2e0 0x7fd7a80654d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:15.080 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:15.078+0000 7fd7b04a8640 1 -- 172.21.15.169:0/765597099 >> 172.21.15.169:0/765597099 conn(0x7fd7a80f6ac0 msgr2=0x7fd7a80f7350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:15.080 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:15.078+0000 7fd7b04a8640 1 -- 172.21.15.169:0/765597099 shutdown_connections 2024-09-16T00:18:15.080 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:15.078+0000 7fd7b04a8640 1 -- 172.21.15.169:0/765597099 wait complete. 2024-09-16T00:18:15.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:15 smithi112 ceph-mon[27869]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:18:16.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:16 smithi112 ceph-mon[27869]: from='client.? 172.21.15.169:0/765597099' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T00:18:16.691 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-16T00:18:16.691 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T00:18:16.924 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T00:18:16.976 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T00:18:17.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:17 smithi112 ceph-mon[27869]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:18:18.382 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:18.377+0000 7f88cbe02640 1 -- 172.21.15.169:0/698865794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88c40fd7f0 msgr2=0x7f88c40fdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:18.382 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:18.377+0000 7f88cbe02640 1 --2- 172.21.15.169:0/698865794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88c40fd7f0 0x7f88c40fdc00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f88b8004850 tx=0x7f88b8009da0 comp rx=0 tx=0).stop 2024-09-16T00:18:18.382 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:18.377+0000 7f88cbe02640 1 -- 172.21.15.169:0/698865794 shutdown_connections 2024-09-16T00:18:18.383 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:18.377+0000 7f88cbe02640 1 --2- 172.21.15.169:0/698865794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88c40fd7f0 0x7f88c40fdc00 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:18.383 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:18.377+0000 7f88cbe02640 1 -- 172.21.15.169:0/698865794 >> 172.21.15.169:0/698865794 conn(0x7f88c40f6010 msgr2=0x7f88c40f8420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:18.383 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:18.377+0000 7f88cbe02640 1 -- 172.21.15.169:0/698865794 shutdown_connections 2024-09-16T00:18:18.383 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:18.377+0000 7f88cbe02640 1 -- 172.21.15.169:0/698865794 wait complete. 2024-09-16T00:18:18.383 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:18.378+0000 7f88cbe02640 1 Processor -- start 2024-09-16T00:18:18.383 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:18.378+0000 7f88cbe02640 1 -- start start 2024-09-16T00:18:18.383 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:18.378+0000 7f88cbe02640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88c40fd7f0 0x7f88c410c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:18.383 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:18.378+0000 7f88cbe02640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c410cc70 con 0x7f88c40fd7f0 2024-09-16T00:18:18.383 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:18.378+0000 7f88c9b9e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88c40fd7f0 0x7f88c410c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:18.383 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:18.379+0000 7f88c9b9e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88c40fd7f0 0x7f88c410c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.169:40386/0 (socket says 172.21.15.169:40386) 2024-09-16T00:18:18.383 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:18.379+0000 7f88c9b9e640 1 -- 172.21.15.169:0/3486629834 learned_addr learned my addr 172.21.15.169:0/3486629834 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-09-16T00:18:18.383 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:18.379+0000 7f88c9b9e640 1 -- 172.21.15.169:0/3486629834 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88b8005040 con 0x7f88c40fd7f0 2024-09-16T00:18:18.383 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:18.379+0000 7f88c9b9e640 1 --2- 172.21.15.169:0/3486629834 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88c40fd7f0 0x7f88c410c730 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f88b802a6d0 tx=0x7f88b8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:18.384 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:18.379+0000 7f88b67fc640 1 -- 172.21.15.169:0/3486629834 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f88b8033630 con 0x7f88c40fd7f0 2024-09-16T00:18:18.384 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:18.379+0000 7f88b67fc640 1 -- 172.21.15.169:0/3486629834 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f88b8033c30 con 0x7f88c40fd7f0 2024-09-16T00:18:18.384 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:18.379+0000 7f88cbe02640 1 -- 172.21.15.169:0/3486629834 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88c410ce70 con 0x7f88c40fd7f0 2024-09-16T00:18:18.384 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:18.379+0000 7f88b67fc640 1 -- 172.21.15.169:0/3486629834 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f88b803dc00 con 0x7f88c40fd7f0 2024-09-16T00:18:18.384 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:18.379+0000 7f88cbe02640 1 -- 172.21.15.169:0/3486629834 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88c410d2e0 con 0x7f88c40fd7f0 2024-09-16T00:18:18.384 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:18.380+0000 7f88b67fc640 1 -- 172.21.15.169:0/3486629834 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) v1 ==== 45564+0+0 (secure 0 0 0) 0x7f88b8030020 con 0x7f88c40fd7f0 2024-09-16T00:18:18.384 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:18.380+0000 7f88cbe02640 1 -- 172.21.15.169:0/3486629834 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88c405ed70 con 0x7f88c40fd7f0 2024-09-16T00:18:18.384 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:18.380+0000 7f88b67fc640 1 --2- 172.21.15.169:0/3486629834 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f88b0039580 0x7f88b003ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:18.384 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:18.380+0000 7f88b67fc640 1 -- 172.21.15.169:0/3486629834 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f88b806c980 con 0x7f88c40fd7f0 2024-09-16T00:18:18.384 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:18.381+0000 7f88c939d640 1 --2- 172.21.15.169:0/3486629834 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f88b0039580 0x7f88b003ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:18.384 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:18.381+0000 7f88c939d640 1 --2- 172.21.15.169:0/3486629834 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f88b0039580 0x7f88b003ba30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f88c0006fd0 tx=0x7f88c0006db0 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:18.387 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:18.385+0000 7f88b67fc640 1 -- 172.21.15.169:0/3486629834 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f88b8033790 con 0x7f88c40fd7f0 2024-09-16T00:18:18.601 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:18.599+0000 7f88cbe02640 1 -- 172.21.15.169:0/3486629834 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f88c404b720 con 0x7f88c40fd7f0 2024-09-16T00:18:18.601 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:18.600+0000 7f88b67fc640 1 -- 172.21.15.169:0/3486629834 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f88b8042210 con 0x7f88c40fd7f0 2024-09-16T00:18:18.602 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:18:18.602 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":1,"fsid":"d346dd56-73c0-11ef-bceb-c7b262605968","modified":"2024-09-16T00:16:40.582056Z","created":"2024-09-16T00:16:40.582056Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi112","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-16T00:18:18.602 INFO:teuthology.orchestra.run.smithi169.stderr:dumped monmap epoch 1 2024-09-16T00:18:18.604 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:18.602+0000 7f88cbe02640 1 -- 172.21.15.169:0/3486629834 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f88b0039580 msgr2=0x7f88b003ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:18.604 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:18.602+0000 7f88cbe02640 1 --2- 172.21.15.169:0/3486629834 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f88b0039580 0x7f88b003ba30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f88c0006fd0 tx=0x7f88c0006db0 comp rx=0 tx=0).stop 2024-09-16T00:18:18.604 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:18.602+0000 7f88cbe02640 1 -- 172.21.15.169:0/3486629834 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88c40fd7f0 msgr2=0x7f88c410c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:18.604 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:18.602+0000 7f88cbe02640 1 --2- 172.21.15.169:0/3486629834 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88c40fd7f0 0x7f88c410c730 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f88b802a6d0 tx=0x7f88b8009da0 comp rx=0 tx=0).stop 2024-09-16T00:18:18.604 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:18.602+0000 7f88cbe02640 1 -- 172.21.15.169:0/3486629834 shutdown_connections 2024-09-16T00:18:18.604 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:18.602+0000 7f88cbe02640 1 --2- 172.21.15.169:0/3486629834 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f88b0039580 0x7f88b003ba30 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:18.604 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:18.602+0000 7f88cbe02640 1 --2- 172.21.15.169:0/3486629834 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88c40fd7f0 0x7f88c410c730 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:18.604 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:18.602+0000 7f88cbe02640 1 -- 172.21.15.169:0/3486629834 >> 172.21.15.169:0/3486629834 conn(0x7f88c40f6010 msgr2=0x7f88c40f6880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:18.604 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:18.602+0000 7f88cbe02640 1 -- 172.21.15.169:0/3486629834 shutdown_connections 2024-09-16T00:18:18.604 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:18.602+0000 7f88cbe02640 1 -- 172.21.15.169:0/3486629834 wait complete. 2024-09-16T00:18:19.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:19 smithi112 ceph-mon[27869]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:18:19.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:19 smithi112 ceph-mon[27869]: from='client.? 172.21.15.169:0/3486629834' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T00:18:20.255 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-16T00:18:20.256 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T00:18:20.508 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T00:18:20.553 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T00:18:21.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:21 smithi112 ceph-mon[27869]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:18:21.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:21 smithi112 ceph-mon[27869]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:18:21.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:21 smithi112 ceph-mon[27869]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:18:21.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:21 smithi112 ceph-mon[27869]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:18:21.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:21 smithi112 ceph-mon[27869]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:18:21.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:21 smithi112 ceph-mon[27869]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:18:21.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:21 smithi112 ceph-mon[27869]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:18:21.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:21 smithi112 ceph-mon[27869]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:18:21.907 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:21.906+0000 7f56865e2640 1 -- 172.21.15.169:0/2349691158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56800fc1b0 msgr2=0x7f56800fc5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:21.907 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:21.906+0000 7f56865e2640 1 --2- 172.21.15.169:0/2349691158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56800fc1b0 0x7f56800fc5c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5670004850 tx=0x7f5670009da0 comp rx=0 tx=0).stop 2024-09-16T00:18:21.907 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:21.906+0000 7f56865e2640 1 -- 172.21.15.169:0/2349691158 shutdown_connections 2024-09-16T00:18:21.907 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:21.906+0000 7f56865e2640 1 --2- 172.21.15.169:0/2349691158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56800fc1b0 0x7f56800fc5c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:21.907 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:21.906+0000 7f56865e2640 1 -- 172.21.15.169:0/2349691158 >> 172.21.15.169:0/2349691158 conn(0x7f56800f7990 msgr2=0x7f56800f9da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:21.907 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:21.906+0000 7f56865e2640 1 -- 172.21.15.169:0/2349691158 shutdown_connections 2024-09-16T00:18:21.907 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:21.906+0000 7f56865e2640 1 -- 172.21.15.169:0/2349691158 wait complete. 2024-09-16T00:18:21.907 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:21.906+0000 7f56865e2640 1 Processor -- start 2024-09-16T00:18:21.907 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:21.906+0000 7f56865e2640 1 -- start start 2024-09-16T00:18:21.907 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:21.907+0000 7f56865e2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56800fc1b0 0x7f568010c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:21.908 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:21.907+0000 7f56865e2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f568010cc30 con 0x7f56800fc1b0 2024-09-16T00:18:21.909 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:21.907+0000 7f567ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56800fc1b0 0x7f568010c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:21.909 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:21.907+0000 7f567ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56800fc1b0 0x7f568010c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.169:40412/0 (socket says 172.21.15.169:40412) 2024-09-16T00:18:21.909 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:21.907+0000 7f567ffff640 1 -- 172.21.15.169:0/1977695463 learned_addr learned my addr 172.21.15.169:0/1977695463 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-09-16T00:18:21.910 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:21.907+0000 7f567ffff640 1 -- 172.21.15.169:0/1977695463 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5670005040 con 0x7f56800fc1b0 2024-09-16T00:18:21.910 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:21.907+0000 7f567ffff640 1 --2- 172.21.15.169:0/1977695463 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56800fc1b0 0x7f568010c6f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f567002a6d0 tx=0x7f5670009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:21.910 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:21.907+0000 7f567cff9640 1 -- 172.21.15.169:0/1977695463 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5670033630 con 0x7f56800fc1b0 2024-09-16T00:18:21.910 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:21.907+0000 7f56865e2640 1 -- 172.21.15.169:0/1977695463 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f568010ce30 con 0x7f56800fc1b0 2024-09-16T00:18:21.910 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:21.907+0000 7f567cff9640 1 -- 172.21.15.169:0/1977695463 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f5670033c30 con 0x7f56800fc1b0 2024-09-16T00:18:21.910 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:21.907+0000 7f567cff9640 1 -- 172.21.15.169:0/1977695463 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f567003dc00 con 0x7f56800fc1b0 2024-09-16T00:18:21.910 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:21.908+0000 7f56865e2640 1 -- 172.21.15.169:0/1977695463 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f568010d2a0 con 0x7f56800fc1b0 2024-09-16T00:18:21.910 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:21.908+0000 7f567cff9640 1 -- 172.21.15.169:0/1977695463 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) v1 ==== 45564+0+0 (secure 0 0 0) 0x7f5670030020 con 0x7f56800fc1b0 2024-09-16T00:18:21.910 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:21.908+0000 7f56865e2640 1 -- 172.21.15.169:0/1977695463 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f568005ed70 con 0x7f56800fc1b0 2024-09-16T00:18:21.910 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:21.909+0000 7f567cff9640 1 --2- 172.21.15.169:0/1977695463 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f5668039580 0x7f566803ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:21.910 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:21.909+0000 7f567cff9640 1 -- 172.21.15.169:0/1977695463 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f567006c980 con 0x7f56800fc1b0 2024-09-16T00:18:21.910 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:21.909+0000 7f567f7fe640 1 --2- 172.21.15.169:0/1977695463 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f5668039580 0x7f566803ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:21.910 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:21.909+0000 7f567f7fe640 1 --2- 172.21.15.169:0/1977695463 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f5668039580 0x7f566803ba30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5674006fd0 tx=0x7f5674006db0 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:21.916 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:21.916+0000 7f567cff9640 1 -- 172.21.15.169:0/1977695463 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f5670033790 con 0x7f56800fc1b0 2024-09-16T00:18:22.129 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:22.129+0000 7f56865e2640 1 -- 172.21.15.169:0/1977695463 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f568004b720 con 0x7f56800fc1b0 2024-09-16T00:18:22.130 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:22.130+0000 7f567cff9640 1 -- 172.21.15.169:0/1977695463 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f5670042210 con 0x7f56800fc1b0 2024-09-16T00:18:22.130 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:18:22.130 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":1,"fsid":"d346dd56-73c0-11ef-bceb-c7b262605968","modified":"2024-09-16T00:16:40.582056Z","created":"2024-09-16T00:16:40.582056Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi112","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-16T00:18:22.131 INFO:teuthology.orchestra.run.smithi169.stderr:dumped monmap epoch 1 2024-09-16T00:18:22.132 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:22.132+0000 7f56865e2640 1 -- 172.21.15.169:0/1977695463 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f5668039580 msgr2=0x7f566803ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:22.132 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:22.132+0000 7f56865e2640 1 --2- 172.21.15.169:0/1977695463 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f5668039580 0x7f566803ba30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5674006fd0 tx=0x7f5674006db0 comp rx=0 tx=0).stop 2024-09-16T00:18:22.132 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:22.132+0000 7f56865e2640 1 -- 172.21.15.169:0/1977695463 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56800fc1b0 msgr2=0x7f568010c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:22.133 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:22.132+0000 7f56865e2640 1 --2- 172.21.15.169:0/1977695463 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56800fc1b0 0x7f568010c6f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f567002a6d0 tx=0x7f5670009da0 comp rx=0 tx=0).stop 2024-09-16T00:18:22.133 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:22.132+0000 7f56865e2640 1 -- 172.21.15.169:0/1977695463 shutdown_connections 2024-09-16T00:18:22.133 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:22.132+0000 7f56865e2640 1 --2- 172.21.15.169:0/1977695463 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f5668039580 0x7f566803ba30 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:22.133 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:22.132+0000 7f56865e2640 1 --2- 172.21.15.169:0/1977695463 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56800fc1b0 0x7f568010c6f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:22.133 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:22.132+0000 7f56865e2640 1 -- 172.21.15.169:0/1977695463 >> 172.21.15.169:0/1977695463 conn(0x7f56800f7990 msgr2=0x7f56800f8200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:22.133 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:22.132+0000 7f56865e2640 1 -- 172.21.15.169:0/1977695463 shutdown_connections 2024-09-16T00:18:22.133 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:22.132+0000 7f56865e2640 1 -- 172.21.15.169:0/1977695463 wait complete. 2024-09-16T00:18:22.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:22 smithi112 ceph-mon[27869]: Deploying daemon prometheus.smithi112 on smithi112 2024-09-16T00:18:23.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:23 smithi112 ceph-mon[27869]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:18:23.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:23 smithi112 ceph-mon[27869]: from='client.? 172.21.15.169:0/1977695463' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T00:18:23.658 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-16T00:18:23.659 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T00:18:23.903 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T00:18:23.947 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T00:18:25.318 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:25.309+0000 7fe0a4649640 1 -- 172.21.15.169:0/3483064278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe09c0655b0 msgr2=0x7fe09c0ff090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:25.318 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:25.309+0000 7fe0a4649640 1 --2- 172.21.15.169:0/3483064278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe09c0655b0 0x7fe09c0ff090 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe094004850 tx=0x7fe094009da0 comp rx=0 tx=0).stop 2024-09-16T00:18:25.318 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:25.309+0000 7fe0a4649640 1 -- 172.21.15.169:0/3483064278 shutdown_connections 2024-09-16T00:18:25.318 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:25.309+0000 7fe0a4649640 1 --2- 172.21.15.169:0/3483064278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe09c0655b0 0x7fe09c0ff090 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:25.318 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:25.309+0000 7fe0a4649640 1 -- 172.21.15.169:0/3483064278 >> 172.21.15.169:0/3483064278 conn(0x7fe09c0f6ac0 msgr2=0x7fe09c0f8ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:25.318 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:25.309+0000 7fe0a4649640 1 -- 172.21.15.169:0/3483064278 shutdown_connections 2024-09-16T00:18:25.319 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:25.309+0000 7fe0a4649640 1 -- 172.21.15.169:0/3483064278 wait complete. 2024-09-16T00:18:25.319 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:25.310+0000 7fe0a4649640 1 Processor -- start 2024-09-16T00:18:25.319 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:25.310+0000 7fe0a4649640 1 -- start start 2024-09-16T00:18:25.319 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:25.310+0000 7fe0a4649640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe09c0655b0 0x7fe09c10c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:25.319 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:25.310+0000 7fe0a4649640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe09c10cee0 con 0x7fe09c0655b0 2024-09-16T00:18:25.319 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:25.310+0000 7fe0a23e5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe09c0655b0 0x7fe09c10c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:25.319 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:25.310+0000 7fe0a23e5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe09c0655b0 0x7fe09c10c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.169:34962/0 (socket says 172.21.15.169:34962) 2024-09-16T00:18:25.319 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:25.310+0000 7fe0a23e5640 1 -- 172.21.15.169:0/3634706786 learned_addr learned my addr 172.21.15.169:0/3634706786 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-09-16T00:18:25.319 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:25.310+0000 7fe0a23e5640 1 -- 172.21.15.169:0/3634706786 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe094005040 con 0x7fe09c0655b0 2024-09-16T00:18:25.319 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:25.310+0000 7fe0a23e5640 1 --2- 172.21.15.169:0/3634706786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe09c0655b0 0x7fe09c10c9a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe09402a6d0 tx=0x7fe094009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:25.319 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:25.311+0000 7fe09affd640 1 -- 172.21.15.169:0/3634706786 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe094033630 con 0x7fe09c0655b0 2024-09-16T00:18:25.319 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:25.311+0000 7fe0a4649640 1 -- 172.21.15.169:0/3634706786 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe09c10d0e0 con 0x7fe09c0655b0 2024-09-16T00:18:25.319 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:25.311+0000 7fe09affd640 1 -- 172.21.15.169:0/3634706786 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fe094033c30 con 0x7fe09c0655b0 2024-09-16T00:18:25.320 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:25.311+0000 7fe0a4649640 1 -- 172.21.15.169:0/3634706786 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe09c10d550 con 0x7fe09c0655b0 2024-09-16T00:18:25.320 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:25.311+0000 7fe09affd640 1 -- 172.21.15.169:0/3634706786 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe09403dc40 con 0x7fe09c0655b0 2024-09-16T00:18:25.320 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:25.312+0000 7fe0a4649640 1 -- 172.21.15.169:0/3634706786 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe09c05ed70 con 0x7fe09c0655b0 2024-09-16T00:18:25.320 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:25.312+0000 7fe09affd640 1 -- 172.21.15.169:0/3634706786 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) v1 ==== 45564+0+0 (secure 0 0 0) 0x7fe094030020 con 0x7fe09c0655b0 2024-09-16T00:18:25.320 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:25.312+0000 7fe09affd640 1 --2- 172.21.15.169:0/3634706786 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7fe084039580 0x7fe08403ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:25.320 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:25.312+0000 7fe09affd640 1 -- 172.21.15.169:0/3634706786 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fe09406c880 con 0x7fe09c0655b0 2024-09-16T00:18:25.320 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:25.312+0000 7fe0a1be4640 1 --2- 172.21.15.169:0/3634706786 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7fe084039580 0x7fe08403ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:25.320 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:25.313+0000 7fe0a1be4640 1 --2- 172.21.15.169:0/3634706786 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7fe084039580 0x7fe08403ba30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe09000ad90 tx=0x7fe090009410 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:25.320 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:25.318+0000 7fe09affd640 1 -- 172.21.15.169:0/3634706786 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fe094033790 con 0x7fe09c0655b0 2024-09-16T00:18:25.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:25 smithi112 ceph-mon[27869]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:18:25.530 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:25.530+0000 7fe0a4649640 1 -- 172.21.15.169:0/3634706786 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fe09c04b720 con 0x7fe09c0655b0 2024-09-16T00:18:25.531 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:25.530+0000 7fe09affd640 1 -- 172.21.15.169:0/3634706786 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7fe094045800 con 0x7fe09c0655b0 2024-09-16T00:18:25.531 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:18:25.531 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":1,"fsid":"d346dd56-73c0-11ef-bceb-c7b262605968","modified":"2024-09-16T00:16:40.582056Z","created":"2024-09-16T00:16:40.582056Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi112","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-16T00:18:25.531 INFO:teuthology.orchestra.run.smithi169.stderr:dumped monmap epoch 1 2024-09-16T00:18:25.533 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:25.532+0000 7fe0a4649640 1 -- 172.21.15.169:0/3634706786 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7fe084039580 msgr2=0x7fe08403ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:25.533 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:25.532+0000 7fe0a4649640 1 --2- 172.21.15.169:0/3634706786 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7fe084039580 0x7fe08403ba30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe09000ad90 tx=0x7fe090009410 comp rx=0 tx=0).stop 2024-09-16T00:18:25.533 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:25.532+0000 7fe0a4649640 1 -- 172.21.15.169:0/3634706786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe09c0655b0 msgr2=0x7fe09c10c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:25.533 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:25.532+0000 7fe0a4649640 1 --2- 172.21.15.169:0/3634706786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe09c0655b0 0x7fe09c10c9a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe09402a6d0 tx=0x7fe094009da0 comp rx=0 tx=0).stop 2024-09-16T00:18:25.533 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:25.532+0000 7fe0a4649640 1 -- 172.21.15.169:0/3634706786 shutdown_connections 2024-09-16T00:18:25.533 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:25.532+0000 7fe0a4649640 1 --2- 172.21.15.169:0/3634706786 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7fe084039580 0x7fe08403ba30 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:25.533 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:25.532+0000 7fe0a4649640 1 --2- 172.21.15.169:0/3634706786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe09c0655b0 0x7fe09c10c9a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:25.534 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:25.532+0000 7fe0a4649640 1 -- 172.21.15.169:0/3634706786 >> 172.21.15.169:0/3634706786 conn(0x7fe09c0f6ac0 msgr2=0x7fe09c0f73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:25.534 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:25.532+0000 7fe0a4649640 1 -- 172.21.15.169:0/3634706786 shutdown_connections 2024-09-16T00:18:25.534 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:25.532+0000 7fe0a4649640 1 -- 172.21.15.169:0/3634706786 wait complete. 2024-09-16T00:18:26.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:26 smithi112 ceph-mon[27869]: from='client.? 172.21.15.169:0/3634706786' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T00:18:27.104 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-16T00:18:27.104 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T00:18:27.335 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:27 smithi112 ceph-mon[27869]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:18:27.345 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T00:18:27.390 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T00:18:28.698 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:28.689+0000 7f967d1ef640 1 -- 172.21.15.169:0/2061154196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96780fb6e0 msgr2=0x7f96780fbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:28.698 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:28.689+0000 7f967d1ef640 1 --2- 172.21.15.169:0/2061154196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96780fb6e0 0x7f96780fbaf0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9670009bd0 tx=0x7f9670004d10 comp rx=0 tx=0).stop 2024-09-16T00:18:28.698 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:28.689+0000 7f967d1ef640 1 -- 172.21.15.169:0/2061154196 shutdown_connections 2024-09-16T00:18:28.698 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:28.689+0000 7f967d1ef640 1 --2- 172.21.15.169:0/2061154196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96780fb6e0 0x7f96780fbaf0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:28.698 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:28.689+0000 7f967d1ef640 1 -- 172.21.15.169:0/2061154196 >> 172.21.15.169:0/2061154196 conn(0x7f96780f6010 msgr2=0x7f96780f8440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:28.698 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:28.689+0000 7f967d1ef640 1 -- 172.21.15.169:0/2061154196 shutdown_connections 2024-09-16T00:18:28.698 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:28.689+0000 7f967d1ef640 1 -- 172.21.15.169:0/2061154196 wait complete. 2024-09-16T00:18:28.698 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:28.690+0000 7f967d1ef640 1 Processor -- start 2024-09-16T00:18:28.698 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:28.690+0000 7f967d1ef640 1 -- start start 2024-09-16T00:18:28.698 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:28.690+0000 7f967d1ef640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96780fb6e0 0x7f96780faee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:28.698 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:28.690+0000 7f967d1ef640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f967810f460 con 0x7f96780fb6e0 2024-09-16T00:18:28.699 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:28.690+0000 7f9676d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96780fb6e0 0x7f96780faee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:28.699 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:28.690+0000 7f9676d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96780fb6e0 0x7f96780faee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.169:34980/0 (socket says 172.21.15.169:34980) 2024-09-16T00:18:28.699 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:28.690+0000 7f9676d9d640 1 -- 172.21.15.169:0/2207924228 learned_addr learned my addr 172.21.15.169:0/2207924228 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-09-16T00:18:28.699 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:28.691+0000 7f9676d9d640 1 -- 172.21.15.169:0/2207924228 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96700098b0 con 0x7f96780fb6e0 2024-09-16T00:18:28.699 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:28.691+0000 7f9676d9d640 1 --2- 172.21.15.169:0/2207924228 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96780fb6e0 0x7f96780faee0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9670009ba0 tx=0x7f9670004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:28.699 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:28.691+0000 7f966f7fe640 1 -- 172.21.15.169:0/2207924228 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9670032910 con 0x7f96780fb6e0 2024-09-16T00:18:28.699 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:28.691+0000 7f966f7fe640 1 -- 172.21.15.169:0/2207924228 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f967003d420 con 0x7f96780fb6e0 2024-09-16T00:18:28.699 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:28.691+0000 7f967d1ef640 1 -- 172.21.15.169:0/2207924228 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f967810f5a0 con 0x7f96780fb6e0 2024-09-16T00:18:28.699 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:28.691+0000 7f966f7fe640 1 -- 172.21.15.169:0/2207924228 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9670031590 con 0x7f96780fb6e0 2024-09-16T00:18:28.700 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:28.691+0000 7f967d1ef640 1 -- 172.21.15.169:0/2207924228 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f967810fa10 con 0x7f96780fb6e0 2024-09-16T00:18:28.700 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:28.692+0000 7f967d1ef640 1 -- 172.21.15.169:0/2207924228 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f967805ed70 con 0x7f96780fb6e0 2024-09-16T00:18:28.700 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:28.692+0000 7f966f7fe640 1 -- 172.21.15.169:0/2207924228 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) v1 ==== 45564+0+0 (secure 0 0 0) 0x7f967002f030 con 0x7f96780fb6e0 2024-09-16T00:18:28.700 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:28.692+0000 7f966f7fe640 1 --2- 172.21.15.169:0/2207924228 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f9664039580 0x7f966403ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:28.700 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:28.692+0000 7f966f7fe640 1 -- 172.21.15.169:0/2207924228 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f967006d770 con 0x7f96780fb6e0 2024-09-16T00:18:28.700 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:28.692+0000 7f967659c640 1 --2- 172.21.15.169:0/2207924228 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f9664039580 0x7f966403ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:28.700 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:28.693+0000 7f967659c640 1 --2- 172.21.15.169:0/2207924228 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f9664039580 0x7f966403ba30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9660006fd0 tx=0x7f9660006db0 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:28.700 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:28.697+0000 7f966f7fe640 1 -- 172.21.15.169:0/2207924228 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f967004eb10 con 0x7f96780fb6e0 2024-09-16T00:18:28.917 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:28.917+0000 7f967d1ef640 1 -- 172.21.15.169:0/2207924228 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f9678105790 con 0x7f96780fb6e0 2024-09-16T00:18:28.918 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:28.918+0000 7f966f7fe640 1 -- 172.21.15.169:0/2207924228 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f9670041020 con 0x7f96780fb6e0 2024-09-16T00:18:28.918 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:18:28.918 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":1,"fsid":"d346dd56-73c0-11ef-bceb-c7b262605968","modified":"2024-09-16T00:16:40.582056Z","created":"2024-09-16T00:16:40.582056Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi112","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-16T00:18:28.919 INFO:teuthology.orchestra.run.smithi169.stderr:dumped monmap epoch 1 2024-09-16T00:18:28.920 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:28.920+0000 7f967d1ef640 1 -- 172.21.15.169:0/2207924228 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f9664039580 msgr2=0x7f966403ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:28.920 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:28.920+0000 7f967d1ef640 1 --2- 172.21.15.169:0/2207924228 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f9664039580 0x7f966403ba30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9660006fd0 tx=0x7f9660006db0 comp rx=0 tx=0).stop 2024-09-16T00:18:28.920 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:28.920+0000 7f967d1ef640 1 -- 172.21.15.169:0/2207924228 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96780fb6e0 msgr2=0x7f96780faee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:28.920 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:28.920+0000 7f967d1ef640 1 --2- 172.21.15.169:0/2207924228 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96780fb6e0 0x7f96780faee0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9670009ba0 tx=0x7f9670004d10 comp rx=0 tx=0).stop 2024-09-16T00:18:28.921 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:28.920+0000 7f967d1ef640 1 -- 172.21.15.169:0/2207924228 shutdown_connections 2024-09-16T00:18:28.921 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:28.920+0000 7f967d1ef640 1 --2- 172.21.15.169:0/2207924228 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f9664039580 0x7f966403ba30 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:28.921 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:28.920+0000 7f967d1ef640 1 --2- 172.21.15.169:0/2207924228 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96780fb6e0 0x7f96780faee0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:28.921 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:28.920+0000 7f967d1ef640 1 -- 172.21.15.169:0/2207924228 >> 172.21.15.169:0/2207924228 conn(0x7f96780f6010 msgr2=0x7f96780f68f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:28.921 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:28.920+0000 7f967d1ef640 1 -- 172.21.15.169:0/2207924228 shutdown_connections 2024-09-16T00:18:28.921 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:28.920+0000 7f967d1ef640 1 -- 172.21.15.169:0/2207924228 wait complete. 2024-09-16T00:18:29.169 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:29 smithi112 ceph-mon[27869]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:18:29.170 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:29 smithi112 ceph-mon[27869]: from='client.? 172.21.15.169:0/2207924228' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T00:18:30.552 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-16T00:18:30.553 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T00:18:30.797 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T00:18:30.842 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T00:18:31.171 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:31 smithi112 ceph-mon[27869]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:18:32.252 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:32.236+0000 7fcdcc34e640 1 -- 172.21.15.169:0/3880547583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcdc4071570 msgr2=0x7fcdc406faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:32.252 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:32.236+0000 7fcdcc34e640 1 --2- 172.21.15.169:0/3880547583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcdc4071570 0x7fcdc406faa0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fcdb4004850 tx=0x7fcdb4009da0 comp rx=0 tx=0).stop 2024-09-16T00:18:32.252 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:32.237+0000 7fcdcc34e640 1 -- 172.21.15.169:0/3880547583 shutdown_connections 2024-09-16T00:18:32.252 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:32.237+0000 7fcdcc34e640 1 --2- 172.21.15.169:0/3880547583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcdc4071570 0x7fcdc406faa0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:32.252 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:32.237+0000 7fcdcc34e640 1 -- 172.21.15.169:0/3880547583 >> 172.21.15.169:0/3880547583 conn(0x7fcdc40f79b0 msgr2=0x7fcdc40f9dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:32.252 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:32.237+0000 7fcdcc34e640 1 -- 172.21.15.169:0/3880547583 shutdown_connections 2024-09-16T00:18:32.252 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:32.237+0000 7fcdcc34e640 1 -- 172.21.15.169:0/3880547583 wait complete. 2024-09-16T00:18:32.252 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:32.237+0000 7fcdcc34e640 1 Processor -- start 2024-09-16T00:18:32.252 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:32.237+0000 7fcdcc34e640 1 -- start start 2024-09-16T00:18:32.253 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:32.237+0000 7fcdcc34e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcdc4071570 0x7fcdc410e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:32.253 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:32.237+0000 7fcdcc34e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc410ee50 con 0x7fcdc4071570 2024-09-16T00:18:32.253 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:32.237+0000 7fcdca0ea640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcdc4071570 0x7fcdc410e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:32.253 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:32.237+0000 7fcdca0ea640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcdc4071570 0x7fcdc410e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.169:34996/0 (socket says 172.21.15.169:34996) 2024-09-16T00:18:32.253 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:32.237+0000 7fcdca0ea640 1 -- 172.21.15.169:0/3556635127 learned_addr learned my addr 172.21.15.169:0/3556635127 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-09-16T00:18:32.253 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:32.238+0000 7fcdca0ea640 1 -- 172.21.15.169:0/3556635127 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdb4005040 con 0x7fcdc4071570 2024-09-16T00:18:32.253 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:32.238+0000 7fcdca0ea640 1 --2- 172.21.15.169:0/3556635127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcdc4071570 0x7fcdc410e910 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fcdb4009920 tx=0x7fcdb4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:32.253 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:32.238+0000 7fcdbaffd640 1 -- 172.21.15.169:0/3556635127 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcdb4033630 con 0x7fcdc4071570 2024-09-16T00:18:32.253 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:32.238+0000 7fcdbaffd640 1 -- 172.21.15.169:0/3556635127 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fcdb4033c30 con 0x7fcdc4071570 2024-09-16T00:18:32.253 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:32.238+0000 7fcdcc34e640 1 -- 172.21.15.169:0/3556635127 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdc410f050 con 0x7fcdc4071570 2024-09-16T00:18:32.253 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:32.238+0000 7fcdbaffd640 1 -- 172.21.15.169:0/3556635127 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcdb403dc00 con 0x7fcdc4071570 2024-09-16T00:18:32.253 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:32.238+0000 7fcdcc34e640 1 -- 172.21.15.169:0/3556635127 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdc410f4c0 con 0x7fcdc4071570 2024-09-16T00:18:32.253 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:32.239+0000 7fcdbaffd640 1 -- 172.21.15.169:0/3556635127 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) v1 ==== 45564+0+0 (secure 0 0 0) 0x7fcdb4030020 con 0x7fcdc4071570 2024-09-16T00:18:32.254 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:32.239+0000 7fcdbaffd640 1 --2- 172.21.15.169:0/3556635127 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7fcdb0039580 0x7fcdb003ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:32.254 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:32.239+0000 7fcdcc34e640 1 -- 172.21.15.169:0/3556635127 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdc405ed70 con 0x7fcdc4071570 2024-09-16T00:18:32.254 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:32.239+0000 7fcdbaffd640 1 -- 172.21.15.169:0/3556635127 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fcdb406ca00 con 0x7fcdc4071570 2024-09-16T00:18:32.254 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:32.239+0000 7fcdc98e9640 1 --2- 172.21.15.169:0/3556635127 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7fcdb0039580 0x7fcdb003ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:32.254 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:32.240+0000 7fcdc98e9640 1 --2- 172.21.15.169:0/3556635127 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7fcdb0039580 0x7fcdb003ba30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0006fd0 tx=0x7fcdc0006db0 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:32.254 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:32.244+0000 7fcdbaffd640 1 -- 172.21.15.169:0/3556635127 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fcdb4065070 con 0x7fcdc4071570 2024-09-16T00:18:32.324 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:32.323+0000 7fcdbaffd640 1 -- 172.21.15.169:0/3556635127 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mgrmap(e 14) v1 ==== 45578+0+0 (secure 0 0 0) 0x7fcdb403bd40 con 0x7fcdc4071570 2024-09-16T00:18:32.349 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:32.347+0000 7fcdc98e9640 1 -- 172.21.15.169:0/3556635127 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7fcdb0039580 msgr2=0x7fcdb003ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-16T00:18:32.349 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:32.347+0000 7fcdc98e9640 1 -- 172.21.15.169:0/3556635127 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7fcdb0039580 msgr2=0x7fcdb003ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T00:18:32.349 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:32.347+0000 7fcdc98e9640 1 --2- 172.21.15.169:0/3556635127 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7fcdb0039580 0x7fcdb003ba30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0006fd0 tx=0x7fcdc0006db0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-16T00:18:32.349 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:32.347+0000 7fcdc98e9640 1 --2- 172.21.15.169:0/3556635127 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7fcdb0039580 0x7fcdb003ba30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0006fd0 tx=0x7fcdc0006db0 comp rx=0 tx=0).stop 2024-09-16T00:18:32.350 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:32.347+0000 7fcdbaffd640 1 -- 172.21.15.169:0/3556635127 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7fcdb0039580 msgr2=0x7fcdb003ba30 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T00:18:32.350 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:32.347+0000 7fcdbaffd640 1 --2- 172.21.15.169:0/3556635127 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7fcdb0039580 0x7fcdb003ba30 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:32.471 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:32.470+0000 7fcdcc34e640 1 -- 172.21.15.169:0/3556635127 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fcdac0050c0 con 0x7fcdc4071570 2024-09-16T00:18:32.471 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:32.471+0000 7fcdbaffd640 1 -- 172.21.15.169:0/3556635127 <== mon.0 v2:172.21.15.112:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7fcdb403b6f0 con 0x7fcdc4071570 2024-09-16T00:18:32.472 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:18:32.472 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":1,"fsid":"d346dd56-73c0-11ef-bceb-c7b262605968","modified":"2024-09-16T00:16:40.582056Z","created":"2024-09-16T00:16:40.582056Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi112","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-16T00:18:32.473 INFO:teuthology.orchestra.run.smithi169.stderr:dumped monmap epoch 1 2024-09-16T00:18:32.475 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:32.474+0000 7fcdcc34e640 1 -- 172.21.15.169:0/3556635127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcdc4071570 msgr2=0x7fcdc410e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:32.475 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:32.474+0000 7fcdcc34e640 1 --2- 172.21.15.169:0/3556635127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcdc4071570 0x7fcdc410e910 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fcdb4009920 tx=0x7fcdb4009da0 comp rx=0 tx=0).stop 2024-09-16T00:18:32.475 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:32.474+0000 7fcdcc34e640 1 -- 172.21.15.169:0/3556635127 shutdown_connections 2024-09-16T00:18:32.475 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:32.474+0000 7fcdcc34e640 1 --2- 172.21.15.169:0/3556635127 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7fcdb0039580 0x7fcdb003ba30 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:32.475 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:32.474+0000 7fcdcc34e640 1 --2- 172.21.15.169:0/3556635127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcdc4071570 0x7fcdc410e910 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:32.475 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:32.474+0000 7fcdcc34e640 1 -- 172.21.15.169:0/3556635127 >> 172.21.15.169:0/3556635127 conn(0x7fcdc40f79b0 msgr2=0x7fcdc40f8200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:32.475 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:32.474+0000 7fcdcc34e640 1 -- 172.21.15.169:0/3556635127 shutdown_connections 2024-09-16T00:18:32.475 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:32.474+0000 7fcdcc34e640 1 -- 172.21.15.169:0/3556635127 wait complete. 2024-09-16T00:18:32.575 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:32 smithi112 ceph-mon[27869]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:18:32.575 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:32 smithi112 ceph-mon[27869]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:18:32.575 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:32 smithi112 ceph-mon[27869]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:18:32.576 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:32 smithi112 ceph-mon[27869]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-16T00:18:32.576 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:32 smithi112 ceph-mon[27869]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:18:33.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:33 smithi112 ceph-mon[27869]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-16T00:18:33.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:33 smithi112 ceph-mon[27869]: mgrmap e14: smithi112.xrkkbc(active, since 56s) 2024-09-16T00:18:33.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:33 smithi112 ceph-mon[27869]: from='client.? 172.21.15.169:0/3556635127' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T00:18:34.024 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-16T00:18:34.024 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T00:18:34.267 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T00:18:34.323 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T00:18:35.781 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:35.775+0000 7f7ddad95640 1 -- 172.21.15.169:0/668549249 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7dd40fb0c0 msgr2=0x7f7dd40fb4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:35.781 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:35.775+0000 7f7ddad95640 1 --2- 172.21.15.169:0/668549249 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7dd40fb0c0 0x7f7dd40fb4d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7dc8004850 tx=0x7f7dc8009da0 comp rx=0 tx=0).stop 2024-09-16T00:18:35.781 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:35.775+0000 7f7ddad95640 1 -- 172.21.15.169:0/668549249 shutdown_connections 2024-09-16T00:18:35.782 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:35.775+0000 7f7ddad95640 1 --2- 172.21.15.169:0/668549249 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7dd40fb0c0 0x7f7dd40fb4d0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:35.782 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:35.775+0000 7f7ddad95640 1 -- 172.21.15.169:0/668549249 >> 172.21.15.169:0/668549249 conn(0x7f7dd40f6880 msgr2=0x7f7dd40f8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:35.782 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:35.775+0000 7f7ddad95640 1 -- 172.21.15.169:0/668549249 shutdown_connections 2024-09-16T00:18:35.782 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:35.775+0000 7f7ddad95640 1 -- 172.21.15.169:0/668549249 wait complete. 2024-09-16T00:18:35.782 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:35.776+0000 7f7ddad95640 1 Processor -- start 2024-09-16T00:18:35.782 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:35.776+0000 7f7ddad95640 1 -- start start 2024-09-16T00:18:35.782 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:35.776+0000 7f7ddad95640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7dd40fb0c0 0x7f7dd410c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:35.782 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:35.776+0000 7f7ddad95640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd410cca0 con 0x7f7dd40fb0c0 2024-09-16T00:18:35.782 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:35.776+0000 7f7dd8b31640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7dd40fb0c0 0x7f7dd410c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:35.782 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:35.776+0000 7f7dd8b31640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7dd40fb0c0 0x7f7dd410c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.169:39754/0 (socket says 172.21.15.169:39754) 2024-09-16T00:18:35.782 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:35.776+0000 7f7dd8b31640 1 -- 172.21.15.169:0/4066141314 learned_addr learned my addr 172.21.15.169:0/4066141314 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-09-16T00:18:35.782 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:35.776+0000 7f7dd8b31640 1 -- 172.21.15.169:0/4066141314 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dc8005040 con 0x7f7dd40fb0c0 2024-09-16T00:18:35.782 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:35.777+0000 7f7dd8b31640 1 --2- 172.21.15.169:0/4066141314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7dd40fb0c0 0x7f7dd410c760 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7dc802a6d0 tx=0x7f7dc8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:35.783 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:35.777+0000 7f7dd17fa640 1 -- 172.21.15.169:0/4066141314 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7dc8033630 con 0x7f7dd40fb0c0 2024-09-16T00:18:35.783 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:35.777+0000 7f7dd17fa640 1 -- 172.21.15.169:0/4066141314 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f7dc8033c30 con 0x7f7dd40fb0c0 2024-09-16T00:18:35.783 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:35.777+0000 7f7ddad95640 1 -- 172.21.15.169:0/4066141314 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dd410cea0 con 0x7f7dd40fb0c0 2024-09-16T00:18:35.783 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:35.777+0000 7f7dd17fa640 1 -- 172.21.15.169:0/4066141314 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7dc803dc30 con 0x7f7dd40fb0c0 2024-09-16T00:18:35.783 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:35.777+0000 7f7ddad95640 1 -- 172.21.15.169:0/4066141314 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dd410d310 con 0x7f7dd40fb0c0 2024-09-16T00:18:35.783 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:35.778+0000 7f7dd17fa640 1 -- 172.21.15.169:0/4066141314 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 14) v1 ==== 45578+0+0 (secure 0 0 0) 0x7f7dc8030020 con 0x7f7dd40fb0c0 2024-09-16T00:18:35.783 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:35.778+0000 7f7ddad95640 1 -- 172.21.15.169:0/4066141314 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dd405ed70 con 0x7f7dd40fb0c0 2024-09-16T00:18:35.783 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:35.778+0000 7f7dd17fa640 1 --2- 172.21.15.169:0/4066141314 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f7dc40395d0 0x7f7dc403ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:35.783 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:35.778+0000 7f7dd17fa640 1 -- 172.21.15.169:0/4066141314 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f7dc806c920 con 0x7f7dd40fb0c0 2024-09-16T00:18:35.783 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:35.778+0000 7f7dd3fff640 1 -- 172.21.15.169:0/4066141314 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f7dc40395d0 msgr2=0x7f7dc403ba80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/2021251488 2024-09-16T00:18:35.783 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:35.778+0000 7f7dd3fff640 1 --2- 172.21.15.169:0/4066141314 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f7dc40395d0 0x7f7dc403ba80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T00:18:35.784 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:35.783+0000 7f7dd17fa640 1 -- 172.21.15.169:0/4066141314 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f7dc8033790 con 0x7f7dd40fb0c0 2024-09-16T00:18:35.979 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:35.979+0000 7f7dd3fff640 1 -- 172.21.15.169:0/4066141314 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f7dc40395d0 msgr2=0x7f7dc403ba80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/2021251488 2024-09-16T00:18:35.979 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:35.979+0000 7f7dd3fff640 1 --2- 172.21.15.169:0/4066141314 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f7dc40395d0 0x7f7dc403ba80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T00:18:35.998 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:35.997+0000 7f7ddad95640 1 -- 172.21.15.169:0/4066141314 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f7dd404b720 con 0x7f7dd40fb0c0 2024-09-16T00:18:35.998 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:35.998+0000 7f7dd17fa640 1 -- 172.21.15.169:0/4066141314 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f7dc8042210 con 0x7f7dd40fb0c0 2024-09-16T00:18:35.999 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:18:35.999 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":1,"fsid":"d346dd56-73c0-11ef-bceb-c7b262605968","modified":"2024-09-16T00:16:40.582056Z","created":"2024-09-16T00:16:40.582056Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi112","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-16T00:18:35.999 INFO:teuthology.orchestra.run.smithi169.stderr:dumped monmap epoch 1 2024-09-16T00:18:36.000 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:36.000+0000 7f7ddad95640 1 -- 172.21.15.169:0/4066141314 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f7dc40395d0 msgr2=0x7f7dc403ba80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T00:18:36.000 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:36.000+0000 7f7ddad95640 1 --2- 172.21.15.169:0/4066141314 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f7dc40395d0 0x7f7dc403ba80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:36.000 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:36.000+0000 7f7ddad95640 1 -- 172.21.15.169:0/4066141314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7dd40fb0c0 msgr2=0x7f7dd410c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:36.001 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:36.000+0000 7f7ddad95640 1 --2- 172.21.15.169:0/4066141314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7dd40fb0c0 0x7f7dd410c760 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7dc802a6d0 tx=0x7f7dc8009da0 comp rx=0 tx=0).stop 2024-09-16T00:18:36.001 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:36.000+0000 7f7ddad95640 1 -- 172.21.15.169:0/4066141314 shutdown_connections 2024-09-16T00:18:36.001 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:36.000+0000 7f7ddad95640 1 --2- 172.21.15.169:0/4066141314 >> [v2:172.21.15.112:6800/2021251488,v1:172.21.15.112:6801/2021251488] conn(0x7f7dc40395d0 0x7f7dc403ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:36.001 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:36.000+0000 7f7ddad95640 1 --2- 172.21.15.169:0/4066141314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7dd40fb0c0 0x7f7dd410c760 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:36.001 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:36.000+0000 7f7ddad95640 1 -- 172.21.15.169:0/4066141314 >> 172.21.15.169:0/4066141314 conn(0x7f7dd40f6880 msgr2=0x7f7dd4065540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:36.001 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:36.000+0000 7f7ddad95640 1 -- 172.21.15.169:0/4066141314 shutdown_connections 2024-09-16T00:18:36.001 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:36.000+0000 7f7ddad95640 1 -- 172.21.15.169:0/4066141314 wait complete. 2024-09-16T00:18:36.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:36 smithi112 ceph-mon[27869]: from='client.? 172.21.15.169:0/4066141314' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T00:18:37.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:37 smithi112 ceph-mon[27869]: Active manager daemon smithi112.xrkkbc restarted 2024-09-16T00:18:37.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:37 smithi112 ceph-mon[27869]: Activating manager daemon smithi112.xrkkbc 2024-09-16T00:18:37.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:37 smithi112 ceph-mon[27869]: osdmap e5: 0 total, 0 up, 0 in 2024-09-16T00:18:37.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:37 smithi112 ceph-mon[27869]: mgrmap e15: smithi112.xrkkbc(active, starting, since 0.00318727s) 2024-09-16T00:18:37.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:37 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mon metadata", "id": "smithi112"}]: dispatch 2024-09-16T00:18:37.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:37 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mgr metadata", "who": "smithi112.xrkkbc", "id": "smithi112.xrkkbc"}]: dispatch 2024-09-16T00:18:37.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:37 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T00:18:37.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:37 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T00:18:37.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:37 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T00:18:37.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:37 smithi112 ceph-mon[27869]: Manager daemon smithi112.xrkkbc is now available 2024-09-16T00:18:37.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:37 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:18:37.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:37 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:18:37.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:37 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:18:37.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:37 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi112.xrkkbc/mirror_snapshot_schedule"}]: dispatch 2024-09-16T00:18:37.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:37 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi112.xrkkbc/trash_purge_schedule"}]: dispatch 2024-09-16T00:18:37.646 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-16T00:18:37.646 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T00:18:37.881 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T00:18:37.928 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T00:18:38.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:38 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:18:38.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:38 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:18:38.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:38 smithi112 ceph-mon[27869]: mgrmap e16: smithi112.xrkkbc(active, since 1.00523s) 2024-09-16T00:18:38.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:38 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:18:38.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:38 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:18:39.290 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:39.289+0000 7fe2fd965640 1 -- 172.21.15.169:0/3997888760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2f80fd810 msgr2=0x7fe2f80fdc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:39.290 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:39.289+0000 7fe2fd965640 1 --2- 172.21.15.169:0/3997888760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2f80fd810 0x7fe2f80fdc20 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe2e8004850 tx=0x7fe2e8009da0 comp rx=0 tx=0).stop 2024-09-16T00:18:39.290 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:39.289+0000 7fe2fd965640 1 -- 172.21.15.169:0/3997888760 shutdown_connections 2024-09-16T00:18:39.290 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:39.289+0000 7fe2fd965640 1 --2- 172.21.15.169:0/3997888760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2f80fd810 0x7fe2f80fdc20 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:39.290 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:39.289+0000 7fe2fd965640 1 -- 172.21.15.169:0/3997888760 >> 172.21.15.169:0/3997888760 conn(0x7fe2f80f6010 msgr2=0x7fe2f80f8440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:39.290 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:39.289+0000 7fe2fd965640 1 -- 172.21.15.169:0/3997888760 shutdown_connections 2024-09-16T00:18:39.290 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:39.289+0000 7fe2fd965640 1 -- 172.21.15.169:0/3997888760 wait complete. 2024-09-16T00:18:39.292 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:39.290+0000 7fe2fd965640 1 Processor -- start 2024-09-16T00:18:39.292 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:39.290+0000 7fe2fd965640 1 -- start start 2024-09-16T00:18:39.292 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:39.290+0000 7fe2fd965640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2f80fd810 0x7fe2f810c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:39.292 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:39.290+0000 7fe2fd965640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f810ccc0 con 0x7fe2f80fd810 2024-09-16T00:18:39.292 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:39.290+0000 7fe2f6ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2f80fd810 0x7fe2f810c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:39.292 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:39.290+0000 7fe2f6ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2f80fd810 0x7fe2f810c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.169:39770/0 (socket says 172.21.15.169:39770) 2024-09-16T00:18:39.292 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:39.290+0000 7fe2f6ffd640 1 -- 172.21.15.169:0/1095252553 learned_addr learned my addr 172.21.15.169:0/1095252553 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-09-16T00:18:39.292 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:39.291+0000 7fe2f6ffd640 1 -- 172.21.15.169:0/1095252553 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e8005040 con 0x7fe2f80fd810 2024-09-16T00:18:39.292 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:39.291+0000 7fe2f6ffd640 1 --2- 172.21.15.169:0/1095252553 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2f80fd810 0x7fe2f810c780 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe2e8009920 tx=0x7fe2e8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:39.292 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:39.291+0000 7fe2df7fe640 1 -- 172.21.15.169:0/1095252553 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe2e8033630 con 0x7fe2f80fd810 2024-09-16T00:18:39.292 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:39.291+0000 7fe2df7fe640 1 -- 172.21.15.169:0/1095252553 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fe2e8033c30 con 0x7fe2f80fd810 2024-09-16T00:18:39.293 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:39.291+0000 7fe2df7fe640 1 -- 172.21.15.169:0/1095252553 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe2e803dbb0 con 0x7fe2f80fd810 2024-09-16T00:18:39.293 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:39.291+0000 7fe2fd965640 1 -- 172.21.15.169:0/1095252553 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2f810cec0 con 0x7fe2f80fd810 2024-09-16T00:18:39.293 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:39.291+0000 7fe2fd965640 1 -- 172.21.15.169:0/1095252553 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2f810d330 con 0x7fe2f80fd810 2024-09-16T00:18:39.294 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:39.292+0000 7fe2fd965640 1 -- 172.21.15.169:0/1095252553 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2f805ed70 con 0x7fe2f80fd810 2024-09-16T00:18:39.294 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:39.292+0000 7fe2df7fe640 1 -- 172.21.15.169:0/1095252553 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) v1 ==== 45428+0+0 (secure 0 0 0) 0x7fe2e8030020 con 0x7fe2f80fd810 2024-09-16T00:18:39.294 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:39.293+0000 7fe2df7fe640 1 --2- 172.21.15.169:0/1095252553 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fe2e0039420 0x7fe2e003b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:39.294 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:39.293+0000 7fe2df7fe640 1 -- 172.21.15.169:0/1095252553 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fe2e806c700 con 0x7fe2f80fd810 2024-09-16T00:18:39.294 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:39.293+0000 7fe2f67fc640 1 --2- 172.21.15.169:0/1095252553 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fe2e0039420 0x7fe2e003b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:39.294 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:39.293+0000 7fe2f67fc640 1 --2- 172.21.15.169:0/1095252553 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fe2e0039420 0x7fe2e003b8d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec006fd0 tx=0x7fe2ec006db0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:39.298 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:39.297+0000 7fe2df7fe640 1 -- 172.21.15.169:0/1095252553 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fe2e8045de0 con 0x7fe2f80fd810 2024-09-16T00:18:39.310 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:39.310+0000 7fe2df7fe640 1 -- 172.21.15.169:0/1095252553 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mgrmap(e 17) v1 ==== 45622+0+0 (secure 0 0 0) 0x7fe2e8043d70 con 0x7fe2f80fd810 2024-09-16T00:18:39.523 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:39.522+0000 7fe2fd965640 1 -- 172.21.15.169:0/1095252553 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fe2f804b720 con 0x7fe2f80fd810 2024-09-16T00:18:39.523 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:39.523+0000 7fe2df7fe640 1 -- 172.21.15.169:0/1095252553 <== mon.0 v2:172.21.15.112:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7fe2e803bc20 con 0x7fe2f80fd810 2024-09-16T00:18:39.524 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:18:39.524 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":1,"fsid":"d346dd56-73c0-11ef-bceb-c7b262605968","modified":"2024-09-16T00:16:40.582056Z","created":"2024-09-16T00:16:40.582056Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi112","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-16T00:18:39.524 INFO:teuthology.orchestra.run.smithi169.stderr:dumped monmap epoch 1 2024-09-16T00:18:39.526 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:39.525+0000 7fe2fd965640 1 -- 172.21.15.169:0/1095252553 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fe2e0039420 msgr2=0x7fe2e003b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:39.526 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:39.525+0000 7fe2fd965640 1 --2- 172.21.15.169:0/1095252553 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fe2e0039420 0x7fe2e003b8d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec006fd0 tx=0x7fe2ec006db0 comp rx=0 tx=0).stop 2024-09-16T00:18:39.526 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:39.525+0000 7fe2fd965640 1 -- 172.21.15.169:0/1095252553 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2f80fd810 msgr2=0x7fe2f810c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:39.526 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:39.525+0000 7fe2fd965640 1 --2- 172.21.15.169:0/1095252553 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2f80fd810 0x7fe2f810c780 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe2e8009920 tx=0x7fe2e8009da0 comp rx=0 tx=0).stop 2024-09-16T00:18:39.526 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:39.525+0000 7fe2fd965640 1 -- 172.21.15.169:0/1095252553 shutdown_connections 2024-09-16T00:18:39.526 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:39.525+0000 7fe2fd965640 1 --2- 172.21.15.169:0/1095252553 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fe2e0039420 0x7fe2e003b8d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:39.526 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:39.525+0000 7fe2fd965640 1 --2- 172.21.15.169:0/1095252553 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2f80fd810 0x7fe2f810c780 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:39.526 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:39.525+0000 7fe2fd965640 1 -- 172.21.15.169:0/1095252553 >> 172.21.15.169:0/1095252553 conn(0x7fe2f80f6010 msgr2=0x7fe2f80f68d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:39.526 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:39.525+0000 7fe2fd965640 1 -- 172.21.15.169:0/1095252553 shutdown_connections 2024-09-16T00:18:39.527 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:39.525+0000 7fe2fd965640 1 -- 172.21.15.169:0/1095252553 wait complete. 2024-09-16T00:18:39.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:39 smithi112 ceph-mon[27869]: [16/Sep/2024:00:18:38] ENGINE Bus STARTING 2024-09-16T00:18:39.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:39 smithi112 ceph-mon[27869]: [16/Sep/2024:00:18:38] ENGINE Serving on https://172.21.15.112:7150 2024-09-16T00:18:39.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:39 smithi112 ceph-mon[27869]: [16/Sep/2024:00:18:38] ENGINE Bus STARTED 2024-09-16T00:18:39.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:39 smithi112 ceph-mon[27869]: [16/Sep/2024:00:18:38] ENGINE Error in HTTPServer.serve 2024-09-16T00:18:39.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: Traceback (most recent call last): 2024-09-16T00:18:39.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T00:18:39.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: self._connections.run(self.expiration_interval) 2024-09-16T00:18:39.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T00:18:39.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: self._run(expiration_interval) 2024-09-16T00:18:39.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T00:18:39.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T00:18:39.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T00:18:39.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T00:18:39.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T00:18:39.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: s = self.context.wrap_socket( 2024-09-16T00:18:39.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T00:18:39.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: return self.sslsocket_class._create( 2024-09-16T00:18:39.591 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T00:18:39.591 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: self.do_handshake() 2024-09-16T00:18:39.591 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T00:18:39.591 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: self._sslobj.do_handshake() 2024-09-16T00:18:39.591 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T00:18:40.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:40 smithi112 ceph-mon[27869]: mgrmap e17: smithi112.xrkkbc(active, since 2s) 2024-09-16T00:18:40.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:40 smithi112 ceph-mon[27869]: from='client.? 172.21.15.169:0/1095252553' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T00:18:41.362 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-16T00:18:41.363 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T00:18:41.620 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T00:18:41.675 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T00:18:43.687 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:43.681+0000 7fc9146c6640 1 -- 172.21.15.169:0/2248933258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc90c0fb0c0 msgr2=0x7fc90c0fb4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:43.687 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:43.681+0000 7fc9146c6640 1 --2- 172.21.15.169:0/2248933258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc90c0fb0c0 0x7fc90c0fb4d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc908004850 tx=0x7fc908009da0 comp rx=0 tx=0).stop 2024-09-16T00:18:43.687 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:43.681+0000 7fc9146c6640 1 -- 172.21.15.169:0/2248933258 shutdown_connections 2024-09-16T00:18:43.688 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:43.681+0000 7fc9146c6640 1 --2- 172.21.15.169:0/2248933258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc90c0fb0c0 0x7fc90c0fb4d0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:43.688 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:43.681+0000 7fc9146c6640 1 -- 172.21.15.169:0/2248933258 >> 172.21.15.169:0/2248933258 conn(0x7fc90c0f6880 msgr2=0x7fc90c0f8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:43.688 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:43.681+0000 7fc9146c6640 1 -- 172.21.15.169:0/2248933258 shutdown_connections 2024-09-16T00:18:43.688 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:43.681+0000 7fc9146c6640 1 -- 172.21.15.169:0/2248933258 wait complete. 2024-09-16T00:18:43.688 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:43.682+0000 7fc9146c6640 1 Processor -- start 2024-09-16T00:18:43.688 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:43.682+0000 7fc9146c6640 1 -- start start 2024-09-16T00:18:43.688 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:43.682+0000 7fc9146c6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc90c0fb0c0 0x7fc90c10b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:43.688 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:43.682+0000 7fc9146c6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc90c10bb10 con 0x7fc90c0fb0c0 2024-09-16T00:18:43.688 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:43.682+0000 7fc912462640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc90c0fb0c0 0x7fc90c10b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:43.688 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:43.682+0000 7fc912462640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc90c0fb0c0 0x7fc90c10b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.169:46542/0 (socket says 172.21.15.169:46542) 2024-09-16T00:18:43.688 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:43.682+0000 7fc912462640 1 -- 172.21.15.169:0/3735332677 learned_addr learned my addr 172.21.15.169:0/3735332677 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-09-16T00:18:43.688 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:43.682+0000 7fc912462640 1 -- 172.21.15.169:0/3735332677 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc908005040 con 0x7fc90c0fb0c0 2024-09-16T00:18:43.688 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:43.683+0000 7fc912462640 1 --2- 172.21.15.169:0/3735332677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc90c0fb0c0 0x7fc90c10b5d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc90802a6d0 tx=0x7fc908009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:43.689 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:43.683+0000 7fc902ffd640 1 -- 172.21.15.169:0/3735332677 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc908033630 con 0x7fc90c0fb0c0 2024-09-16T00:18:43.689 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:43.683+0000 7fc9146c6640 1 -- 172.21.15.169:0/3735332677 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc90c10bd10 con 0x7fc90c0fb0c0 2024-09-16T00:18:43.689 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:43.683+0000 7fc902ffd640 1 -- 172.21.15.169:0/3735332677 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fc908033c30 con 0x7fc90c0fb0c0 2024-09-16T00:18:43.689 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:43.683+0000 7fc902ffd640 1 -- 172.21.15.169:0/3735332677 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc90803dc00 con 0x7fc90c0fb0c0 2024-09-16T00:18:43.689 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:43.683+0000 7fc9146c6640 1 -- 172.21.15.169:0/3735332677 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc90c10c180 con 0x7fc90c0fb0c0 2024-09-16T00:18:43.689 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:43.684+0000 7fc902ffd640 1 -- 172.21.15.169:0/3735332677 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) v1 ==== 45622+0+0 (secure 0 0 0) 0x7fc908030020 con 0x7fc90c0fb0c0 2024-09-16T00:18:43.689 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:43.684+0000 7fc9146c6640 1 -- 172.21.15.169:0/3735332677 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc90c05ed70 con 0x7fc90c0fb0c0 2024-09-16T00:18:43.689 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:43.684+0000 7fc902ffd640 1 --2- 172.21.15.169:0/3735332677 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fc8f8039670 0x7fc8f803bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:43.689 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:43.684+0000 7fc902ffd640 1 -- 172.21.15.169:0/3735332677 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fc90806cb90 con 0x7fc90c0fb0c0 2024-09-16T00:18:43.689 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:43.684+0000 7fc911c61640 1 --2- 172.21.15.169:0/3735332677 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fc8f8039670 0x7fc8f803bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:43.689 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:43.684+0000 7fc911c61640 1 --2- 172.21.15.169:0/3735332677 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fc8f8039670 0x7fc8f803bb20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc006fd0 tx=0x7fc8fc006db0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:43.690 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:43.688+0000 7fc902ffd640 1 -- 172.21.15.169:0/3735332677 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fc908033790 con 0x7fc90c0fb0c0 2024-09-16T00:18:43.906 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:43.905+0000 7fc9146c6640 1 -- 172.21.15.169:0/3735332677 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fc90c04b720 con 0x7fc90c0fb0c0 2024-09-16T00:18:43.906 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:43.906+0000 7fc902ffd640 1 -- 172.21.15.169:0/3735332677 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7fc908045800 con 0x7fc90c0fb0c0 2024-09-16T00:18:43.907 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:18:43.907 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":1,"fsid":"d346dd56-73c0-11ef-bceb-c7b262605968","modified":"2024-09-16T00:16:40.582056Z","created":"2024-09-16T00:16:40.582056Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi112","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-16T00:18:43.907 INFO:teuthology.orchestra.run.smithi169.stderr:dumped monmap epoch 1 2024-09-16T00:18:43.908 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:43.908+0000 7fc9146c6640 1 -- 172.21.15.169:0/3735332677 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fc8f8039670 msgr2=0x7fc8f803bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:43.908 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:43.908+0000 7fc9146c6640 1 --2- 172.21.15.169:0/3735332677 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fc8f8039670 0x7fc8f803bb20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc006fd0 tx=0x7fc8fc006db0 comp rx=0 tx=0).stop 2024-09-16T00:18:43.908 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:43.908+0000 7fc9146c6640 1 -- 172.21.15.169:0/3735332677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc90c0fb0c0 msgr2=0x7fc90c10b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:43.908 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:43.908+0000 7fc9146c6640 1 --2- 172.21.15.169:0/3735332677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc90c0fb0c0 0x7fc90c10b5d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc90802a6d0 tx=0x7fc908009da0 comp rx=0 tx=0).stop 2024-09-16T00:18:43.909 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:43.908+0000 7fc9146c6640 1 -- 172.21.15.169:0/3735332677 shutdown_connections 2024-09-16T00:18:43.909 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:43.908+0000 7fc9146c6640 1 --2- 172.21.15.169:0/3735332677 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fc8f8039670 0x7fc8f803bb20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:43.909 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:43.908+0000 7fc9146c6640 1 --2- 172.21.15.169:0/3735332677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc90c0fb0c0 0x7fc90c10b5d0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:43.909 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:43.908+0000 7fc9146c6640 1 -- 172.21.15.169:0/3735332677 >> 172.21.15.169:0/3735332677 conn(0x7fc90c0f6880 msgr2=0x7fc90c0f70e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:43.909 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:43.908+0000 7fc9146c6640 1 -- 172.21.15.169:0/3735332677 shutdown_connections 2024-09-16T00:18:43.909 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:43.908+0000 7fc9146c6640 1 -- 172.21.15.169:0/3735332677 wait complete. 2024-09-16T00:18:44.178 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:43 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:18:44.178 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:43 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:18:44.178 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:43 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:18:44.179 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:43 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:18:44.179 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:43 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "who": "osd/host:smithi169", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:18:45.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:44 smithi112 ceph-mon[27869]: from='client.? 172.21.15.169:0/3735332677' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T00:18:45.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:44 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:18:45.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:44 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:18:45.492 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-16T00:18:45.492 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T00:18:45.753 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:18:46.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:46 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:18:46.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:46 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:18:46.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:46 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "who": "osd/host:smithi112", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:18:46.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:46 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:18:46.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:46 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:18:46.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:46 smithi112 ceph-mon[27869]: Updating smithi112:/etc/ceph/ceph.conf 2024-09-16T00:18:46.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:46 smithi112 ceph-mon[27869]: Updating smithi169:/etc/ceph/ceph.conf 2024-09-16T00:18:46.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:46 smithi112 ceph-mon[27869]: Updating smithi112:/var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:18:46.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:46 smithi112 ceph-mon[27869]: Updating smithi169:/var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:18:46.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:46 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:18:46.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:46 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:18:46.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:46 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:18:46.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:46 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:18:46.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:46 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:18:46.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:46 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi169", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-16T00:18:46.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:46 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi169", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-09-16T00:18:46.591 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:46 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:18:47.197 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:47.197+0000 7f3dcd5a7640 1 -- 172.21.15.169:0/2710009567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3dc8102200 msgr2=0x7f3dc81025d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:47.198 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:47.197+0000 7f3dcd5a7640 1 --2- 172.21.15.169:0/2710009567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3dc8102200 0x7f3dc81025d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3db8004850 tx=0x7f3db8009da0 comp rx=0 tx=0).stop 2024-09-16T00:18:47.198 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:47.197+0000 7f3dcd5a7640 1 -- 172.21.15.169:0/2710009567 shutdown_connections 2024-09-16T00:18:47.198 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:47.197+0000 7f3dcd5a7640 1 --2- 172.21.15.169:0/2710009567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3dc8102200 0x7f3dc81025d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:47.198 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:47.197+0000 7f3dcd5a7640 1 -- 172.21.15.169:0/2710009567 >> 172.21.15.169:0/2710009567 conn(0x7f3dc80f7dd0 msgr2=0x7f3dc80fa1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:47.198 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:47.197+0000 7f3dcd5a7640 1 -- 172.21.15.169:0/2710009567 shutdown_connections 2024-09-16T00:18:47.198 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:47.197+0000 7f3dcd5a7640 1 -- 172.21.15.169:0/2710009567 wait complete. 2024-09-16T00:18:47.199 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:47.198+0000 7f3dcd5a7640 1 Processor -- start 2024-09-16T00:18:47.200 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:47.198+0000 7f3dcd5a7640 1 -- start start 2024-09-16T00:18:47.200 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:47.198+0000 7f3dcd5a7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3dc8102200 0x7f3dc8112780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:47.200 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:47.198+0000 7f3dcd5a7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc8112cc0 con 0x7f3dc8102200 2024-09-16T00:18:47.200 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:47.198+0000 7f3dc6ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3dc8102200 0x7f3dc8112780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:47.200 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:47.198+0000 7f3dc6ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3dc8102200 0x7f3dc8112780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.169:46556/0 (socket says 172.21.15.169:46556) 2024-09-16T00:18:47.200 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:47.198+0000 7f3dc6ffd640 1 -- 172.21.15.169:0/1579302928 learned_addr learned my addr 172.21.15.169:0/1579302928 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-09-16T00:18:47.200 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:47.199+0000 7f3dc6ffd640 1 -- 172.21.15.169:0/1579302928 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3db8005040 con 0x7f3dc8102200 2024-09-16T00:18:47.200 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:47.199+0000 7f3dc6ffd640 1 --2- 172.21.15.169:0/1579302928 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3dc8102200 0x7f3dc8112780 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3db802a6d0 tx=0x7f3db8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:47.200 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:47.199+0000 7f3dbf7fe640 1 -- 172.21.15.169:0/1579302928 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3db802cf10 con 0x7f3dc8102200 2024-09-16T00:18:47.200 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:47.199+0000 7f3dcd5a7640 1 -- 172.21.15.169:0/1579302928 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dc8112ec0 con 0x7f3dc8102200 2024-09-16T00:18:47.200 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:47.199+0000 7f3dbf7fe640 1 -- 172.21.15.169:0/1579302928 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f3db8033ad0 con 0x7f3dc8102200 2024-09-16T00:18:47.200 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:47.199+0000 7f3dbf7fe640 1 -- 172.21.15.169:0/1579302928 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3db803ebb0 con 0x7f3dc8102200 2024-09-16T00:18:47.200 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:47.199+0000 7f3dcd5a7640 1 -- 172.21.15.169:0/1579302928 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dc8113390 con 0x7f3dc8102200 2024-09-16T00:18:47.201 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:47.200+0000 7f3dbf7fe640 1 -- 172.21.15.169:0/1579302928 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) v1 ==== 45622+0+0 (secure 0 0 0) 0x7f3db8030020 con 0x7f3dc8102200 2024-09-16T00:18:47.201 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:47.200+0000 7f3dcd5a7640 1 -- 172.21.15.169:0/1579302928 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dc80fd970 con 0x7f3dc8102200 2024-09-16T00:18:47.201 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:47.200+0000 7f3dbf7fe640 1 --2- 172.21.15.169:0/1579302928 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f3db4039670 0x7f3db403bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:47.202 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:47.200+0000 7f3dbf7fe640 1 -- 172.21.15.169:0/1579302928 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f3db806c610 con 0x7f3dc8102200 2024-09-16T00:18:47.202 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:47.201+0000 7f3dc67fc640 1 --2- 172.21.15.169:0/1579302928 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f3db4039670 0x7f3db403bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:47.202 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:47.201+0000 7f3dc67fc640 1 --2- 172.21.15.169:0/1579302928 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f3db4039670 0x7f3db403bb20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3db0006fd0 tx=0x7f3db0006db0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:47.205 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:47.205+0000 7f3dbf7fe640 1 -- 172.21.15.169:0/1579302928 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f3db803c510 con 0x7f3dc8102200 2024-09-16T00:18:47.429 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:47.428+0000 7f3dcd5a7640 1 -- 172.21.15.169:0/1579302928 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f3dc8113390 con 0x7f3dc8102200 2024-09-16T00:18:47.429 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:47.429+0000 7f3dbf7fe640 1 -- 172.21.15.169:0/1579302928 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f3dc8113390 con 0x7f3dc8102200 2024-09-16T00:18:47.429 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:18:47.430 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":1,"fsid":"d346dd56-73c0-11ef-bceb-c7b262605968","modified":"2024-09-16T00:16:40.582056Z","created":"2024-09-16T00:16:40.582056Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi112","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-16T00:18:47.430 INFO:teuthology.orchestra.run.smithi169.stderr:dumped monmap epoch 1 2024-09-16T00:18:47.432 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:47.431+0000 7f3dcd5a7640 1 -- 172.21.15.169:0/1579302928 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f3db4039670 msgr2=0x7f3db403bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:47.432 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:47.431+0000 7f3dcd5a7640 1 --2- 172.21.15.169:0/1579302928 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f3db4039670 0x7f3db403bb20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3db0006fd0 tx=0x7f3db0006db0 comp rx=0 tx=0).stop 2024-09-16T00:18:47.432 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:47.431+0000 7f3dcd5a7640 1 -- 172.21.15.169:0/1579302928 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3dc8102200 msgr2=0x7f3dc8112780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:47.433 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:47.431+0000 7f3dcd5a7640 1 --2- 172.21.15.169:0/1579302928 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3dc8102200 0x7f3dc8112780 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3db802a6d0 tx=0x7f3db8009da0 comp rx=0 tx=0).stop 2024-09-16T00:18:47.433 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:47.431+0000 7f3dcd5a7640 1 -- 172.21.15.169:0/1579302928 shutdown_connections 2024-09-16T00:18:47.433 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:47.431+0000 7f3dcd5a7640 1 --2- 172.21.15.169:0/1579302928 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f3db4039670 0x7f3db403bb20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:47.433 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:47.431+0000 7f3dcd5a7640 1 --2- 172.21.15.169:0/1579302928 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3dc8102200 0x7f3dc8112780 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:47.433 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:47.431+0000 7f3dcd5a7640 1 -- 172.21.15.169:0/1579302928 >> 172.21.15.169:0/1579302928 conn(0x7f3dc80f7dd0 msgr2=0x7f3dc80f95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:47.433 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:47.431+0000 7f3dcd5a7640 1 -- 172.21.15.169:0/1579302928 shutdown_connections 2024-09-16T00:18:47.433 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:47.431+0000 7f3dcd5a7640 1 -- 172.21.15.169:0/1579302928 wait complete. 2024-09-16T00:18:47.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:47 smithi112 ceph-mon[27869]: Updating smithi112:/etc/ceph/ceph.client.admin.keyring 2024-09-16T00:18:47.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:47 smithi112 ceph-mon[27869]: Updating smithi169:/etc/ceph/ceph.client.admin.keyring 2024-09-16T00:18:47.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:47 smithi112 ceph-mon[27869]: Updating smithi112:/var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T00:18:47.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:47 smithi112 ceph-mon[27869]: Updating smithi169:/var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T00:18:47.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:47 smithi112 ceph-mon[27869]: Deploying daemon crash.smithi169 on smithi169 2024-09-16T00:18:48.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:48 smithi112 ceph-mon[27869]: from='client.? 172.21.15.169:0/1579302928' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T00:18:49.702 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-16T00:18:49.702 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T00:18:49.959 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:18:51.293 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:50 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:18:51.293 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:50 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:18:51.293 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:50 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:18:51.293 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:50 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:18:51.293 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:50 smithi112 ceph-mon[27869]: Deploying daemon node-exporter.smithi169 on smithi169 2024-09-16T00:18:51.661 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:51.615+0000 7fc6086c4640 1 -- 172.21.15.169:0/4058502182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6001010e0 msgr2=0x7fc6001014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:51.661 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:51.615+0000 7fc6086c4640 1 --2- 172.21.15.169:0/4058502182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6001010e0 0x7fc6001014b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc004850 tx=0x7fc5fc009da0 comp rx=0 tx=0).stop 2024-09-16T00:18:51.662 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:51.615+0000 7fc6086c4640 1 -- 172.21.15.169:0/4058502182 shutdown_connections 2024-09-16T00:18:51.662 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:51.615+0000 7fc6086c4640 1 --2- 172.21.15.169:0/4058502182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6001010e0 0x7fc6001014b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:51.662 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:51.615+0000 7fc6086c4640 1 -- 172.21.15.169:0/4058502182 >> 172.21.15.169:0/4058502182 conn(0x7fc6000710b0 msgr2=0x7fc6000714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:51.662 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:51.615+0000 7fc6086c4640 1 -- 172.21.15.169:0/4058502182 shutdown_connections 2024-09-16T00:18:51.662 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:51.615+0000 7fc6086c4640 1 -- 172.21.15.169:0/4058502182 wait complete. 2024-09-16T00:18:51.662 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:51.616+0000 7fc6086c4640 1 Processor -- start 2024-09-16T00:18:51.662 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:51.616+0000 7fc6086c4640 1 -- start start 2024-09-16T00:18:51.662 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:51.616+0000 7fc6086c4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6001010e0 0x7fc60010d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:51.662 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:51.616+0000 7fc6086c4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc60010dcf0 con 0x7fc6001010e0 2024-09-16T00:18:51.662 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:51.616+0000 7fc606460640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6001010e0 0x7fc60010d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:51.662 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:51.616+0000 7fc606460640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6001010e0 0x7fc60010d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.169:46576/0 (socket says 172.21.15.169:46576) 2024-09-16T00:18:51.662 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:51.616+0000 7fc606460640 1 -- 172.21.15.169:0/1272468706 learned_addr learned my addr 172.21.15.169:0/1272468706 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-09-16T00:18:51.662 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:51.616+0000 7fc606460640 1 -- 172.21.15.169:0/1272468706 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5fc005040 con 0x7fc6001010e0 2024-09-16T00:18:51.663 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:51.617+0000 7fc606460640 1 --2- 172.21.15.169:0/1272468706 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6001010e0 0x7fc60010d7b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc02a6d0 tx=0x7fc5fc009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:51.663 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:51.617+0000 7fc5f6ffd640 1 -- 172.21.15.169:0/1272468706 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc5fc02cf10 con 0x7fc6001010e0 2024-09-16T00:18:51.663 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:51.617+0000 7fc6086c4640 1 -- 172.21.15.169:0/1272468706 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc60010def0 con 0x7fc6001010e0 2024-09-16T00:18:51.663 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:51.617+0000 7fc5f6ffd640 1 -- 172.21.15.169:0/1272468706 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fc5fc033ad0 con 0x7fc6001010e0 2024-09-16T00:18:51.663 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:51.617+0000 7fc5f6ffd640 1 -- 172.21.15.169:0/1272468706 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc5fc03eb50 con 0x7fc6001010e0 2024-09-16T00:18:51.663 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:51.617+0000 7fc6086c4640 1 -- 172.21.15.169:0/1272468706 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc60010e340 con 0x7fc6001010e0 2024-09-16T00:18:51.663 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:51.618+0000 7fc5f6ffd640 1 -- 172.21.15.169:0/1272468706 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) v1 ==== 45622+0+0 (secure 0 0 0) 0x7fc5fc030020 con 0x7fc6001010e0 2024-09-16T00:18:51.663 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:51.618+0000 7fc5f6ffd640 1 --2- 172.21.15.169:0/1272468706 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fc5ec039670 0x7fc5ec03bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:51.663 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:51.618+0000 7fc5f6ffd640 1 -- 172.21.15.169:0/1272468706 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fc5fc06c770 con 0x7fc6001010e0 2024-09-16T00:18:51.663 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:51.618+0000 7fc6086c4640 1 -- 172.21.15.169:0/1272468706 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6000fc850 con 0x7fc6001010e0 2024-09-16T00:18:51.663 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:51.618+0000 7fc605c5f640 1 --2- 172.21.15.169:0/1272468706 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fc5ec039670 0x7fc5ec03bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:51.663 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:51.618+0000 7fc605c5f640 1 --2- 172.21.15.169:0/1272468706 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fc5ec039670 0x7fc5ec03bb20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc5f0006fd0 tx=0x7fc5f0006db0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:51.663 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:51.623+0000 7fc5f6ffd640 1 -- 172.21.15.169:0/1272468706 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fc5fc0473f0 con 0x7fc6001010e0 2024-09-16T00:18:51.841 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:51.841+0000 7fc6086c4640 1 -- 172.21.15.169:0/1272468706 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fc60005ed70 con 0x7fc6001010e0 2024-09-16T00:18:51.842 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:51.842+0000 7fc5f6ffd640 1 -- 172.21.15.169:0/1272468706 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7fc5fc043210 con 0x7fc6001010e0 2024-09-16T00:18:51.842 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:18:51.842 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":1,"fsid":"d346dd56-73c0-11ef-bceb-c7b262605968","modified":"2024-09-16T00:16:40.582056Z","created":"2024-09-16T00:16:40.582056Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi112","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-16T00:18:51.843 INFO:teuthology.orchestra.run.smithi169.stderr:dumped monmap epoch 1 2024-09-16T00:18:51.844 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:51.843+0000 7fc6086c4640 1 -- 172.21.15.169:0/1272468706 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fc5ec039670 msgr2=0x7fc5ec03bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:51.844 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:51.843+0000 7fc6086c4640 1 --2- 172.21.15.169:0/1272468706 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fc5ec039670 0x7fc5ec03bb20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc5f0006fd0 tx=0x7fc5f0006db0 comp rx=0 tx=0).stop 2024-09-16T00:18:51.844 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:51.843+0000 7fc6086c4640 1 -- 172.21.15.169:0/1272468706 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6001010e0 msgr2=0x7fc60010d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:51.844 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:51.843+0000 7fc6086c4640 1 --2- 172.21.15.169:0/1272468706 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6001010e0 0x7fc60010d7b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc02a6d0 tx=0x7fc5fc009da0 comp rx=0 tx=0).stop 2024-09-16T00:18:51.844 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:51.843+0000 7fc6086c4640 1 -- 172.21.15.169:0/1272468706 shutdown_connections 2024-09-16T00:18:51.844 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:51.843+0000 7fc6086c4640 1 --2- 172.21.15.169:0/1272468706 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fc5ec039670 0x7fc5ec03bb20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:51.845 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:51.843+0000 7fc6086c4640 1 --2- 172.21.15.169:0/1272468706 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6001010e0 0x7fc60010d7b0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:51.845 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:51.843+0000 7fc6086c4640 1 -- 172.21.15.169:0/1272468706 >> 172.21.15.169:0/1272468706 conn(0x7fc6000710b0 msgr2=0x7fc600102630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:51.845 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:51.844+0000 7fc6086c4640 1 -- 172.21.15.169:0/1272468706 shutdown_connections 2024-09-16T00:18:51.845 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:51.844+0000 7fc6086c4640 1 -- 172.21.15.169:0/1272468706 wait complete. 2024-09-16T00:18:52.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:52 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:18:52.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:52 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:18:52.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:52 smithi112 ceph-mon[27869]: from='client.? 172.21.15.169:0/1272468706' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T00:18:53.721 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-16T00:18:53.722 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T00:18:54.403 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:18:55.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:55 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:18:55.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:55 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:18:55.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:55 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:18:55.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:55 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:18:55.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:55 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi169.mtrobk", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T00:18:55.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:55 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi169.mtrobk", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-16T00:18:55.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:55 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T00:18:55.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:55 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:18:55.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:55 smithi112 ceph-mon[27869]: Deploying daemon mgr.smithi169.mtrobk on smithi169 2024-09-16T00:18:56.747 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:56.745+0000 7f84c88e9640 1 -- 172.21.15.169:0/1933667498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84c4102200 msgr2=0x7f84c41025d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:56.747 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:56.745+0000 7f84c88e9640 1 --2- 172.21.15.169:0/1933667498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84c4102200 0x7f84c41025d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f84b4004850 tx=0x7f84b4009da0 comp rx=0 tx=0).stop 2024-09-16T00:18:56.747 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:56.745+0000 7f84c88e9640 1 -- 172.21.15.169:0/1933667498 shutdown_connections 2024-09-16T00:18:56.747 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:56.745+0000 7f84c88e9640 1 --2- 172.21.15.169:0/1933667498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84c4102200 0x7f84c41025d0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:56.747 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:56.745+0000 7f84c88e9640 1 -- 172.21.15.169:0/1933667498 >> 172.21.15.169:0/1933667498 conn(0x7f84c40f7dd0 msgr2=0x7f84c40fa1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:56.748 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:56.745+0000 7f84c88e9640 1 -- 172.21.15.169:0/1933667498 shutdown_connections 2024-09-16T00:18:56.748 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:56.745+0000 7f84c88e9640 1 -- 172.21.15.169:0/1933667498 wait complete. 2024-09-16T00:18:56.748 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:56.746+0000 7f84c88e9640 1 Processor -- start 2024-09-16T00:18:56.748 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:56.746+0000 7f84c88e9640 1 -- start start 2024-09-16T00:18:56.748 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:56.746+0000 7f84c88e9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84c4102200 0x7f84c4112780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:56.748 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:56.746+0000 7f84c88e9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c4112cc0 con 0x7f84c4102200 2024-09-16T00:18:56.748 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:56.747+0000 7f84c259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84c4102200 0x7f84c4112780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:56.748 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:56.747+0000 7f84c259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84c4102200 0x7f84c4112780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.169:51722/0 (socket says 172.21.15.169:51722) 2024-09-16T00:18:56.748 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:56.747+0000 7f84c259c640 1 -- 172.21.15.169:0/263542728 learned_addr learned my addr 172.21.15.169:0/263542728 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-09-16T00:18:56.750 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:56.747+0000 7f84c259c640 1 -- 172.21.15.169:0/263542728 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b4005040 con 0x7f84c4102200 2024-09-16T00:18:56.750 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:56.747+0000 7f84c259c640 1 --2- 172.21.15.169:0/263542728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84c4102200 0x7f84c4112780 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f84b402a6d0 tx=0x7f84b4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:56.750 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:56.747+0000 7f84bb7fe640 1 -- 172.21.15.169:0/263542728 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f84b402cf10 con 0x7f84c4102200 2024-09-16T00:18:56.750 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:56.747+0000 7f84c88e9640 1 -- 172.21.15.169:0/263542728 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84c4112ec0 con 0x7f84c4102200 2024-09-16T00:18:56.750 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:56.747+0000 7f84bb7fe640 1 -- 172.21.15.169:0/263542728 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f84b4033ad0 con 0x7f84c4102200 2024-09-16T00:18:56.750 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:56.747+0000 7f84bb7fe640 1 -- 172.21.15.169:0/263542728 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f84b403ebb0 con 0x7f84c4102200 2024-09-16T00:18:56.750 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:56.747+0000 7f84c88e9640 1 -- 172.21.15.169:0/263542728 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84c4113390 con 0x7f84c4102200 2024-09-16T00:18:56.750 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:56.749+0000 7f84bb7fe640 1 -- 172.21.15.169:0/263542728 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) v1 ==== 45622+0+0 (secure 0 0 0) 0x7f84b4030020 con 0x7f84c4102200 2024-09-16T00:18:56.751 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:56.749+0000 7f84c88e9640 1 -- 172.21.15.169:0/263542728 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84c40fd970 con 0x7f84c4102200 2024-09-16T00:18:56.751 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:56.749+0000 7f84bb7fe640 1 --2- 172.21.15.169:0/263542728 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f84b0039670 0x7f84b003bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:56.751 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:56.749+0000 7f84bb7fe640 1 -- 172.21.15.169:0/263542728 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f84b406c5a0 con 0x7f84c4102200 2024-09-16T00:18:56.751 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:56.749+0000 7f84b9d9b640 1 --2- 172.21.15.169:0/263542728 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f84b0039670 0x7f84b003bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:56.751 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:56.750+0000 7f84b9d9b640 1 --2- 172.21.15.169:0/263542728 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f84b0039670 0x7f84b003bb20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f84ac006fd0 tx=0x7f84ac006db0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:56.754 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:56.754+0000 7f84bb7fe640 1 -- 172.21.15.169:0/263542728 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f84b403c930 con 0x7f84c4102200 2024-09-16T00:18:56.984 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:56.984+0000 7f84c88e9640 1 -- 172.21.15.169:0/263542728 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f84c4113390 con 0x7f84c4102200 2024-09-16T00:18:56.984 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:56.984+0000 7f84bb7fe640 1 -- 172.21.15.169:0/263542728 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f84c4113390 con 0x7f84c4102200 2024-09-16T00:18:56.985 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:18:56.985 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":1,"fsid":"d346dd56-73c0-11ef-bceb-c7b262605968","modified":"2024-09-16T00:16:40.582056Z","created":"2024-09-16T00:16:40.582056Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi112","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-16T00:18:56.985 INFO:teuthology.orchestra.run.smithi169.stderr:dumped monmap epoch 1 2024-09-16T00:18:56.988 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:56.987+0000 7f84c88e9640 1 -- 172.21.15.169:0/263542728 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f84b0039670 msgr2=0x7f84b003bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:56.988 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:56.987+0000 7f84c88e9640 1 --2- 172.21.15.169:0/263542728 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f84b0039670 0x7f84b003bb20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f84ac006fd0 tx=0x7f84ac006db0 comp rx=0 tx=0).stop 2024-09-16T00:18:56.988 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:56.987+0000 7f84c88e9640 1 -- 172.21.15.169:0/263542728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84c4102200 msgr2=0x7f84c4112780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:56.988 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:56.987+0000 7f84c88e9640 1 --2- 172.21.15.169:0/263542728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84c4102200 0x7f84c4112780 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f84b402a6d0 tx=0x7f84b4009da0 comp rx=0 tx=0).stop 2024-09-16T00:18:56.988 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:56.987+0000 7f84c88e9640 1 -- 172.21.15.169:0/263542728 shutdown_connections 2024-09-16T00:18:56.988 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:56.987+0000 7f84c88e9640 1 --2- 172.21.15.169:0/263542728 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f84b0039670 0x7f84b003bb20 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:56.988 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:56.987+0000 7f84c88e9640 1 --2- 172.21.15.169:0/263542728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84c4102200 0x7f84c4112780 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:56.988 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:56.987+0000 7f84c88e9640 1 -- 172.21.15.169:0/263542728 >> 172.21.15.169:0/263542728 conn(0x7f84c40f7dd0 msgr2=0x7f84c40f95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:56.988 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:56.987+0000 7f84c88e9640 1 -- 172.21.15.169:0/263542728 shutdown_connections 2024-09-16T00:18:56.988 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:18:56.987+0000 7f84c88e9640 1 -- 172.21.15.169:0/263542728 wait complete. 2024-09-16T00:18:58.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:57 smithi112 ceph-mon[27869]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:18:58.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:57 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:18:58.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:57 smithi112 ceph-mon[27869]: from='client.? 172.21.15.169:0/263542728' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T00:18:58.495 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-16T00:18:58.496 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T00:18:59.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:59 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:18:59.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:59 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:18:59.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:59 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:18:59.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:59 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:18:59.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:59 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T00:18:59.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:59 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:18:59.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:18:59 smithi112 ceph-mon[27869]: Deploying daemon mon.smithi169 on smithi169 2024-09-16T00:19:00.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:00 smithi112 ceph-mon[27869]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:00.525 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi169/config 2024-09-16T00:19:02.064 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:02.060+0000 7f732cdce640 1 -- 172.21.15.169:0/794610768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73281022f0 msgr2=0x7f73281026c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:02.064 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:02.060+0000 7f732cdce640 1 --2- 172.21.15.169:0/794610768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73281022f0 0x7f73281026c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7318004850 tx=0x7f7318009da0 comp rx=0 tx=0).stop 2024-09-16T00:19:02.065 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:02.060+0000 7f732cdce640 1 -- 172.21.15.169:0/794610768 shutdown_connections 2024-09-16T00:19:02.065 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:02.060+0000 7f732cdce640 1 --2- 172.21.15.169:0/794610768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73281022f0 0x7f73281026c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:02.065 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:02.060+0000 7f732cdce640 1 -- 172.21.15.169:0/794610768 >> 172.21.15.169:0/794610768 conn(0x7f73280711c0 msgr2=0x7f73280715c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:02.065 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:02.060+0000 7f732cdce640 1 -- 172.21.15.169:0/794610768 shutdown_connections 2024-09-16T00:19:02.065 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:02.060+0000 7f732cdce640 1 -- 172.21.15.169:0/794610768 wait complete. 2024-09-16T00:19:02.065 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:02.061+0000 7f732cdce640 1 Processor -- start 2024-09-16T00:19:02.065 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:02.061+0000 7f732cdce640 1 -- start start 2024-09-16T00:19:02.065 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:02.061+0000 7f732cdce640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73281022f0 0x7f73280fb270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:02.065 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:02.061+0000 7f732cdce640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73280fb7b0 con 0x7f73281022f0 2024-09-16T00:19:02.065 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:02.061+0000 7f732659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73281022f0 0x7f73280fb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:02.065 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:02.061+0000 7f732659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73281022f0 0x7f73280fb270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.169:51748/0 (socket says 172.21.15.169:51748) 2024-09-16T00:19:02.065 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:02.061+0000 7f732659c640 1 -- 172.21.15.169:0/2330117574 learned_addr learned my addr 172.21.15.169:0/2330117574 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-09-16T00:19:02.065 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:02.062+0000 7f732659c640 1 -- 172.21.15.169:0/2330117574 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7318005040 con 0x7f73281022f0 2024-09-16T00:19:02.066 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:02.062+0000 7f732659c640 1 --2- 172.21.15.169:0/2330117574 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73281022f0 0x7f73280fb270 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f731802a6d0 tx=0x7f7318009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:02.066 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:02.064+0000 7f731f7fe640 1 -- 172.21.15.169:0/2330117574 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f731802cf10 con 0x7f73281022f0 2024-09-16T00:19:02.067 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:02.064+0000 7f732cdce640 1 -- 172.21.15.169:0/2330117574 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73280fb9b0 con 0x7f73281022f0 2024-09-16T00:19:02.067 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:02.064+0000 7f731f7fe640 1 -- 172.21.15.169:0/2330117574 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f7318033ad0 con 0x7f73281022f0 2024-09-16T00:19:02.067 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:02.064+0000 7f731f7fe640 1 -- 172.21.15.169:0/2330117574 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f731803ebb0 con 0x7f73281022f0 2024-09-16T00:19:02.067 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:02.064+0000 7f732cdce640 1 -- 172.21.15.169:0/2330117574 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73280fbe80 con 0x7f73281022f0 2024-09-16T00:19:02.067 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:02.065+0000 7f732cdce640 1 -- 172.21.15.169:0/2330117574 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73280f9710 con 0x7f73281022f0 2024-09-16T00:19:02.067 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:02.065+0000 7f731f7fe640 1 -- 172.21.15.169:0/2330117574 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) v1 ==== 45622+0+0 (secure 0 0 0) 0x7f7318030020 con 0x7f73281022f0 2024-09-16T00:19:02.067 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:02.065+0000 7f731f7fe640 1 --2- 172.21.15.169:0/2330117574 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f73140396c0 0x7f731403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:02.067 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:02.065+0000 7f731f7fe640 1 -- 172.21.15.169:0/2330117574 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f731806c7b0 con 0x7f73281022f0 2024-09-16T00:19:02.067 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:02.065+0000 7f731dd9b640 1 --2- 172.21.15.169:0/2330117574 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f73140396c0 0x7f731403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:02.068 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:02.066+0000 7f731dd9b640 1 --2- 172.21.15.169:0/2330117574 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f73140396c0 0x7f731403bb70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7310006fd0 tx=0x7f7310006db0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:02.071 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:02.070+0000 7f731f7fe640 1 -- 172.21.15.169:0/2330117574 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f731803ed10 con 0x7f73281022f0 2024-09-16T00:19:02.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 podman[34573]: 2024-09-16 00:19:02.101489705 +0000 UTC m=+0.847030976 image pull 33ed8dba399669a1d738ac1a85bc5f36daf3048afed44b7a93f620fc77496a99 quay.ceph.io/ceph-ci/ceph@sha256:83596473fc70117d507bb3d925c5bd11b70f46bbd4dffcc8452f145d93eaedc4 2024-09-16T00:19:02.301 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:02.299+0000 7f732cdce640 1 -- 172.21.15.169:0/2330117574 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f73280fda60 con 0x7f73281022f0 2024-09-16T00:19:02.301 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:02.299+0000 7f731f7fe640 1 -- 172.21.15.169:0/2330117574 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f731802c110 con 0x7f73281022f0 2024-09-16T00:19:02.301 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:19:02.301 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":1,"fsid":"d346dd56-73c0-11ef-bceb-c7b262605968","modified":"2024-09-16T00:16:40.582056Z","created":"2024-09-16T00:16:40.582056Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi112","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-16T00:19:02.302 INFO:teuthology.orchestra.run.smithi169.stderr:dumped monmap epoch 1 2024-09-16T00:19:02.304 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:02.302+0000 7f732cdce640 1 -- 172.21.15.169:0/2330117574 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f73140396c0 msgr2=0x7f731403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:02.304 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:02.302+0000 7f732cdce640 1 --2- 172.21.15.169:0/2330117574 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f73140396c0 0x7f731403bb70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7310006fd0 tx=0x7f7310006db0 comp rx=0 tx=0).stop 2024-09-16T00:19:02.304 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:02.302+0000 7f732cdce640 1 -- 172.21.15.169:0/2330117574 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73281022f0 msgr2=0x7f73280fb270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:02.304 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:02.302+0000 7f732cdce640 1 --2- 172.21.15.169:0/2330117574 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73281022f0 0x7f73280fb270 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f731802a6d0 tx=0x7f7318009da0 comp rx=0 tx=0).stop 2024-09-16T00:19:02.304 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:02.302+0000 7f732cdce640 1 -- 172.21.15.169:0/2330117574 shutdown_connections 2024-09-16T00:19:02.304 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:02.302+0000 7f732cdce640 1 --2- 172.21.15.169:0/2330117574 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f73140396c0 0x7f731403bb70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:02.304 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:02.302+0000 7f732cdce640 1 --2- 172.21.15.169:0/2330117574 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73281022f0 0x7f73280fb270 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:02.304 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:02.302+0000 7f732cdce640 1 -- 172.21.15.169:0/2330117574 >> 172.21.15.169:0/2330117574 conn(0x7f73280711c0 msgr2=0x7f73280fa150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:02.304 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:02.302+0000 7f732cdce640 1 -- 172.21.15.169:0/2330117574 shutdown_connections 2024-09-16T00:19:02.304 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:02.302+0000 7f732cdce640 1 -- 172.21.15.169:0/2330117574 wait complete. 2024-09-16T00:19:02.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:02 smithi112 ceph-mon[27869]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:02.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:02 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:02.623 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 podman[34573]: 2024-09-16 00:19:02.429619938 +0000 UTC m=+1.175161202 container init 6c2289bc3e11d143a2e6df08d0ff86f4766325acc0c76bb93177ee99d0dab9a0 (image=quay.ceph.io/ceph-ci/ceph@sha256:83596473fc70117d507bb3d925c5bd11b70f46bbd4dffcc8452f145d93eaedc4, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-mon-smithi169, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=quincy-6d025bc) 2024-09-16T00:19:02.623 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 podman[34573]: 2024-09-16 00:19:02.433282664 +0000 UTC m=+1.178823934 container start 6c2289bc3e11d143a2e6df08d0ff86f4766325acc0c76bb93177ee99d0dab9a0 (image=quay.ceph.io/ceph-ci/ceph@sha256:83596473fc70117d507bb3d925c5bd11b70f46bbd4dffcc8452f145d93eaedc4, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-mon-smithi169, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=quincy-6d025bc) 2024-09-16T00:19:02.623 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: set uid:gid to 167:167 (ceph:ceph) 2024-09-16T00:19:02.623 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: ceph version 17.2.7-1633-g6d025bc2 (6d025bc26f3176c910707f8cb4c8d8e076640615) quincy (stable), process ceph-mon, pid 2 2024-09-16T00:19:02.623 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: pidfile_write: ignore empty --pid-file 2024-09-16T00:19:02.623 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: load: jerasure load: lrc 2024-09-16T00:19:02.623 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: RocksDB version: 6.15.5 2024-09-16T00:19:02.623 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-09-16T00:19:02.623 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Compile date Sep 15 2024 2024-09-16T00:19:02.624 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: DB SUMMARY 2024-09-16T00:19:02.624 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: DB Session ID: BF6KLD89BROK7JUSQ8UU 2024-09-16T00:19:02.624 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: CURRENT file: CURRENT 2024-09-16T00:19:02.624 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: IDENTITY file: IDENTITY 2024-09-16T00:19:02.624 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: MANIFEST file: MANIFEST-000003 size: 57 Bytes 2024-09-16T00:19:02.624 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi169/store.db dir, Total Num: 0, files: 2024-09-16T00:19:02.624 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi169/store.db: 000004.log size: 511 ; 2024-09-16T00:19:02.624 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.error_if_exists: 0 2024-09-16T00:19:02.624 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.create_if_missing: 0 2024-09-16T00:19:02.624 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.paranoid_checks: 1 2024-09-16T00:19:02.624 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-16T00:19:02.624 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.env: 0x55d3ac0ea440 2024-09-16T00:19:02.624 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.fs: Posix File System 2024-09-16T00:19:02.625 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.info_log: 0x55d3ae62d0c0 2024-09-16T00:19:02.625 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.max_file_opening_threads: 16 2024-09-16T00:19:02.625 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.statistics: (nil) 2024-09-16T00:19:02.625 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.use_fsync: 0 2024-09-16T00:19:02.625 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.max_log_file_size: 0 2024-09-16T00:19:02.625 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-16T00:19:02.625 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-16T00:19:02.625 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.keep_log_file_num: 1000 2024-09-16T00:19:02.625 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.recycle_log_file_num: 0 2024-09-16T00:19:02.625 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.allow_fallocate: 1 2024-09-16T00:19:02.625 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.allow_mmap_reads: 0 2024-09-16T00:19:02.625 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.allow_mmap_writes: 0 2024-09-16T00:19:02.625 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.use_direct_reads: 0 2024-09-16T00:19:02.625 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-16T00:19:02.626 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.create_missing_column_families: 0 2024-09-16T00:19:02.626 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.db_log_dir: 2024-09-16T00:19:02.626 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-smithi169/store.db 2024-09-16T00:19:02.626 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-16T00:19:02.626 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-16T00:19:02.626 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-16T00:19:02.626 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-16T00:19:02.626 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-16T00:19:02.626 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-16T00:19:02.626 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.advise_random_on_open: 1 2024-09-16T00:19:02.626 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.db_write_buffer_size: 0 2024-09-16T00:19:02.626 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.write_buffer_manager: 0x55d3ae6ea810 2024-09-16T00:19:02.626 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-16T00:19:02.626 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-09-16T00:19:02.626 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-16T00:19:02.627 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-16T00:19:02.627 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.rate_limiter: (nil) 2024-09-16T00:19:02.627 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-16T00:19:02.627 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.wal_recovery_mode: 2 2024-09-16T00:19:02.627 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.enable_thread_tracking: 0 2024-09-16T00:19:02.627 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.enable_pipelined_write: 0 2024-09-16T00:19:02.627 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.unordered_write: 0 2024-09-16T00:19:02.627 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-16T00:19:02.627 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-16T00:19:02.627 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-16T00:19:02.627 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-16T00:19:02.627 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.row_cache: None 2024-09-16T00:19:02.627 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.wal_filter: None 2024-09-16T00:19:02.627 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-16T00:19:02.629 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.allow_ingest_behind: 0 2024-09-16T00:19:02.629 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.preserve_deletes: 0 2024-09-16T00:19:02.630 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.two_write_queues: 0 2024-09-16T00:19:02.630 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.manual_wal_flush: 0 2024-09-16T00:19:02.630 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.atomic_flush: 0 2024-09-16T00:19:02.630 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-16T00:19:02.630 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-16T00:19:02.630 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-16T00:19:02.630 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.log_readahead_size: 0 2024-09-16T00:19:02.630 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-16T00:19:02.630 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.best_efforts_recovery: 0 2024-09-16T00:19:02.630 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-16T00:19:02.630 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-16T00:19:02.630 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.allow_data_in_errors: 0 2024-09-16T00:19:02.630 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.db_host_id: __hostname__ 2024-09-16T00:19:02.630 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.max_background_jobs: 2 2024-09-16T00:19:02.631 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.max_background_compactions: -1 2024-09-16T00:19:02.631 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.max_subcompactions: 1 2024-09-16T00:19:02.631 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-16T00:19:02.631 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-16T00:19:02.631 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-16T00:19:02.631 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.max_total_wal_size: 0 2024-09-16T00:19:02.631 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-16T00:19:02.631 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-16T00:19:02.631 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-16T00:19:02.631 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-16T00:19:02.631 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.max_open_files: -1 2024-09-16T00:19:02.631 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.bytes_per_sync: 0 2024-09-16T00:19:02.631 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-16T00:19:02.631 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-16T00:19:02.632 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.compaction_readahead_size: 0 2024-09-16T00:19:02.632 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.max_background_flushes: -1 2024-09-16T00:19:02.632 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Compression algorithms supported: 2024-09-16T00:19:02.632 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: kZSTD supported: 0 2024-09-16T00:19:02.632 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: kXpressCompression supported: 0 2024-09-16T00:19:02.632 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: kBZip2Compression supported: 0 2024-09-16T00:19:02.632 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-16T00:19:02.632 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: kLZ4Compression supported: 1 2024-09-16T00:19:02.632 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: kZlibCompression supported: 1 2024-09-16T00:19:02.632 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: kLZ4HCCompression supported: 1 2024-09-16T00:19:02.632 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: kSnappyCompression supported: 1 2024-09-16T00:19:02.632 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-16T00:19:02.632 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi169/store.db/MANIFEST-000003 2024-09-16T00:19:02.632 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-09-16T00:19:02.632 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-16T00:19:02.633 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.merge_operator: 2024-09-16T00:19:02.633 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.compaction_filter: None 2024-09-16T00:19:02.633 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.compaction_filter_factory: None 2024-09-16T00:19:02.633 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.sst_partitioner_factory: None 2024-09-16T00:19:02.633 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-16T00:19:02.633 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-16T00:19:02.633 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55d3ae5d3a38) 2024-09-16T00:19:02.633 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: cache_index_and_filter_blocks: 1 2024-09-16T00:19:02.633 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-16T00:19:02.634 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-16T00:19:02.634 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: pin_top_level_index_and_filter: 1 2024-09-16T00:19:02.635 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: index_type: 0 2024-09-16T00:19:02.635 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: data_block_index_type: 0 2024-09-16T00:19:02.635 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: index_shortening: 1 2024-09-16T00:19:02.635 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-16T00:19:02.635 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: hash_index_allow_collision: 1 2024-09-16T00:19:02.635 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: checksum: 1 2024-09-16T00:19:02.635 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: no_block_cache: 0 2024-09-16T00:19:02.635 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: block_cache: 0x55d3ae6091f0 2024-09-16T00:19:02.635 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: block_cache_name: BinnedLRUCache 2024-09-16T00:19:02.635 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: block_cache_options: 2024-09-16T00:19:02.635 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: capacity : 536870912 2024-09-16T00:19:02.635 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: num_shard_bits : 4 2024-09-16T00:19:02.635 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: strict_capacity_limit : 0 2024-09-16T00:19:02.635 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: high_pri_pool_ratio: 0.000 2024-09-16T00:19:02.636 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: block_cache_compressed: (nil) 2024-09-16T00:19:02.636 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: persistent_cache: (nil) 2024-09-16T00:19:02.636 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: block_size: 4096 2024-09-16T00:19:02.636 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: block_size_deviation: 10 2024-09-16T00:19:02.636 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: block_restart_interval: 16 2024-09-16T00:19:02.636 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: index_block_restart_interval: 1 2024-09-16T00:19:02.636 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: metadata_block_size: 4096 2024-09-16T00:19:02.636 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: partition_filters: 0 2024-09-16T00:19:02.636 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: use_delta_encoding: 1 2024-09-16T00:19:02.636 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-09-16T00:19:02.636 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: whole_key_filtering: 1 2024-09-16T00:19:02.636 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: verify_compression: 0 2024-09-16T00:19:02.636 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: read_amp_bytes_per_bit: 0 2024-09-16T00:19:02.636 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: format_version: 4 2024-09-16T00:19:02.637 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: enable_index_compression: 1 2024-09-16T00:19:02.637 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: block_align: 0 2024-09-16T00:19:02.637 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.write_buffer_size: 33554432 2024-09-16T00:19:02.637 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.max_write_buffer_number: 2 2024-09-16T00:19:02.637 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.compression: NoCompression 2024-09-16T00:19:02.637 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.bottommost_compression: Disabled 2024-09-16T00:19:02.637 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.prefix_extractor: nullptr 2024-09-16T00:19:02.637 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-16T00:19:02.637 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.num_levels: 7 2024-09-16T00:19:02.637 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-16T00:19:02.637 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-16T00:19:02.637 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-16T00:19:02.637 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-16T00:19:02.638 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-16T00:19:02.638 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-16T00:19:02.638 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-16T00:19:02.638 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-16T00:19:02.638 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-16T00:19:02.638 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-16T00:19:02.638 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-16T00:19:02.638 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.compression_opts.level: 32767 2024-09-16T00:19:02.638 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.compression_opts.strategy: 0 2024-09-16T00:19:02.638 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-16T00:19:02.639 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-16T00:19:02.639 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-16T00:19:02.639 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.compression_opts.enabled: false 2024-09-16T00:19:02.639 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-16T00:19:02.639 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-16T00:19:02.639 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-16T00:19:02.639 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.target_file_size_base: 67108864 2024-09-16T00:19:02.639 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-16T00:19:02.639 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-16T00:19:02.639 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-16T00:19:02.639 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-16T00:19:02.640 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-16T00:19:02.640 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-16T00:19:02.640 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-16T00:19:02.640 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-16T00:19:02.640 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-16T00:19:02.640 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-16T00:19:02.640 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-16T00:19:02.640 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-16T00:19:02.640 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-16T00:19:02.640 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.arena_block_size: 4194304 2024-09-16T00:19:02.640 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-16T00:19:02.640 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-16T00:19:02.640 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-09-16T00:19:02.640 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.disable_auto_compactions: 0 2024-09-16T00:19:02.641 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-16T00:19:02.641 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-16T00:19:02.641 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-16T00:19:02.641 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-16T00:19:02.641 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-16T00:19:02.641 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-16T00:19:02.641 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-16T00:19:02.641 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-16T00:19:02.641 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-16T00:19:02.641 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-16T00:19:02.641 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-16T00:19:02.641 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.inplace_update_support: 0 2024-09-16T00:19:02.641 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-16T00:19:02.641 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-16T00:19:02.642 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-16T00:19:02.642 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-16T00:19:02.642 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.bloom_locality: 0 2024-09-16T00:19:02.642 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.max_successive_merges: 0 2024-09-16T00:19:02.642 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-16T00:19:02.642 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.paranoid_file_checks: 0 2024-09-16T00:19:02.642 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.force_consistency_checks: 1 2024-09-16T00:19:02.642 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.report_bg_io_stats: 0 2024-09-16T00:19:02.642 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.ttl: 2592000 2024-09-16T00:19:02.642 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-16T00:19:02.642 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.enable_blob_files: false 2024-09-16T00:19:02.642 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.min_blob_size: 0 2024-09-16T00:19:02.642 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.blob_file_size: 268435456 2024-09-16T00:19:02.642 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-16T00:19:02.642 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-16T00:19:02.643 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-16T00:19:02.643 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi169/store.db/MANIFEST-000003 succeeded,manifest_file_number is 3, next_file_number is 5, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-16T00:19:02.643 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 0 2024-09-16T00:19:02.643 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: [db/version_set.cc:4082] Creating manifest 7 2024-09-16T00:19:02.643 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726445942456823, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-16T00:19:02.643 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #4 mode 2 2024-09-16T00:19:02.643 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-09-16T00:19:02.643 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726445942459445, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1571, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1726445942, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "df03730b-16ce-43ff-a098-3a54a9c13c09", "db_session_id": "BF6KLD89BROK7JUSQ8UU"}} 2024-09-16T00:19:02.643 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: [db/version_set.cc:4082] Creating manifest 9 2024-09-16T00:19:02.643 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726445942459765, "job": 1, "event": "recovery_finished"} 2024-09-16T00:19:02.643 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-smithi169/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-16T00:19:02.643 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x55d3ae63c700 2024-09-16T00:19:02.643 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: DB pointer 0x55d3ae60c000 2024-09-16T00:19:02.644 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-09-16T00:19:02.644 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-09-16T00:19:02.644 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: ** DB Stats ** 2024-09-16T00:19:02.644 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T00:19:02.644 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-16T00:19:02.644 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T00:19:02.644 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T00:19:02.644 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-16T00:19:02.644 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-09-16T00:19:02.644 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T00:19:02.645 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: 2024-09-16T00:19:02.645 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: ** Compaction Stats [default] ** 2024-09-16T00:19:02.645 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-16T00:19:02.645 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-16T00:19:02.645 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: L0 1/0 1.53 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 0.6 0.00 0.00 1 0.003 0 0 2024-09-16T00:19:02.645 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Sum 1/0 1.53 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 0.6 0.00 0.00 1 0.003 0 0 2024-09-16T00:19:02.645 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 0.6 0.00 0.00 1 0.003 0 0 2024-09-16T00:19:02.645 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: 2024-09-16T00:19:02.645 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: ** Compaction Stats [default] ** 2024-09-16T00:19:02.645 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-16T00:19:02.645 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-16T00:19:02.645 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.6 0.00 0.00 1 0.003 0 0 2024-09-16T00:19:02.645 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T00:19:02.645 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-16T00:19:02.646 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-16T00:19:02.646 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-16T00:19:02.646 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-16T00:19:02.646 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-16T00:19:02.646 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Cumulative compaction: 0.00 GB write, 0.29 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T00:19:02.646 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Interval compaction: 0.00 GB write, 0.29 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T00:19:02.646 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-16T00:19:02.646 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: 2024-09-16T00:19:02.646 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-16T00:19:02.646 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: 2024-09-16T00:19:02.646 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: ** Compaction Stats [default] ** 2024-09-16T00:19:02.646 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-16T00:19:02.646 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-16T00:19:02.646 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: L0 1/0 1.53 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 0.6 0.00 0.00 1 0.003 0 0 2024-09-16T00:19:02.646 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Sum 1/0 1.53 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 0.6 0.00 0.00 1 0.003 0 0 2024-09-16T00:19:02.647 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-09-16T00:19:02.647 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: 2024-09-16T00:19:02.647 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: ** Compaction Stats [default] ** 2024-09-16T00:19:02.648 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-16T00:19:02.648 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-16T00:19:02.648 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.6 0.00 0.00 1 0.003 0 0 2024-09-16T00:19:02.648 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T00:19:02.648 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-16T00:19:02.648 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-16T00:19:02.648 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-16T00:19:02.648 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-16T00:19:02.648 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-16T00:19:02.648 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Cumulative compaction: 0.00 GB write, 0.29 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T00:19:02.648 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T00:19:02.648 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-16T00:19:02.648 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: 2024-09-16T00:19:02.649 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-16T00:19:02.649 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: mon.smithi169 does not exist in monmap, will attempt to join an existing cluster 2024-09-16T00:19:02.649 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: using public_addr v2:172.21.15.169:0/0 -> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] 2024-09-16T00:19:02.649 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: starting mon.smithi169 rank -1 at public addrs [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] at bind addrs [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi169 fsid d346dd56-73c0-11ef-bceb-c7b262605968 2024-09-16T00:19:02.649 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: mon.smithi169@-1(???) e0 preinit fsid d346dd56-73c0-11ef-bceb-c7b262605968 2024-09-16T00:19:02.649 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: mon.smithi169@-1(synchronizing).mds e1 new map 2024-09-16T00:19:02.649 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: mon.smithi169@-1(synchronizing).mds e1 print_map 2024-09-16T00:19:02.649 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: e1 2024-09-16T00:19:02.649 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T00:19:02.649 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:19:02.649 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: legacy client fscid: -1 2024-09-16T00:19:02.649 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: 2024-09-16T00:19:02.649 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: No filesystems configured 2024-09-16T00:19:02.649 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: mon.smithi169@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-16T00:19:02.650 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: mon.smithi169@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-16T00:19:02.650 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: mon.smithi169@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-16T00:19:02.650 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: mon.smithi169@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-16T00:19:02.650 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: mon.smithi169@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-16T00:19:02.650 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: mon.smithi169@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-16T00:19:02.650 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: mon.smithi169@-1(synchronizing).osd e5 e5: 0 total, 0 up, 0 in 2024-09-16T00:19:02.650 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: mon.smithi169@-1(synchronizing).osd e5 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-16T00:19:02.650 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: mon.smithi169@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T00:19:02.650 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: mon.smithi169@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T00:19:02.650 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: mon.smithi169@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T00:19:02.650 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: Deploying daemon node-exporter.smithi112 on smithi112 2024-09-16T00:19:02.650 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: Deploying cephadm binary to smithi169 2024-09-16T00:19:02.650 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:02.650 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:02.650 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:02.651 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: Added host smithi169 2024-09-16T00:19:02.651 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:02.651 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:02.651 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:02.651 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:02.651 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:02.651 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: Deploying daemon alertmanager.smithi112 on smithi112 2024-09-16T00:19:02.651 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:02.651 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:19:02.651 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:02.651 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:02.651 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:02.651 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='client.? 172.21.15.112:0/2577888393' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-16T00:19:02.651 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='client.? 172.21.15.112:0/2577888393' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-16T00:19:02.652 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T00:19:02.652 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:02.652 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:02.652 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:02.652 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:02.652 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:02.652 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:02.652 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:02.652 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:02.652 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-16T00:19:02.652 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-16T00:19:02.652 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:02.652 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: Deploying daemon grafana.smithi112 on smithi112 2024-09-16T00:19:02.652 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='client.14192 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi112:172.21.15.112=smithi112;smithi169:172.21.15.169=smithi169", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:19:02.653 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: Saving service mon spec with placement smithi112:172.21.15.112=smithi112;smithi169:172.21.15.169=smithi169;count:2 2024-09-16T00:19:02.653 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:02.653 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:02.653 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:02.653 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='client.? 172.21.15.169:0/1622913869' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T00:19:02.653 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:02.653 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:02.653 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='client.? 172.21.15.169:0/765597099' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T00:19:02.653 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:02.653 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:02.653 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='client.? 172.21.15.169:0/3486629834' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T00:19:02.653 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:02.653 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:02.653 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:02.653 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:02.654 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:02.654 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:02.654 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:02.654 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:02.654 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: Deploying daemon prometheus.smithi112 on smithi112 2024-09-16T00:19:02.654 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:02.654 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='client.? 172.21.15.169:0/1977695463' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T00:19:02.654 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:02.654 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='client.? 172.21.15.169:0/3634706786' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T00:19:02.654 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:02.654 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:02.654 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='client.? 172.21.15.169:0/2207924228' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T00:19:02.654 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:02.654 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:02.654 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:02.655 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:02.655 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-16T00:19:02.655 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:02.655 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14160 172.21.15.112:0/1160773296' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-16T00:19:02.655 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: mgrmap e14: smithi112.xrkkbc(active, since 56s) 2024-09-16T00:19:02.655 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='client.? 172.21.15.169:0/3556635127' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T00:19:02.655 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='client.? 172.21.15.169:0/4066141314' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T00:19:02.655 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: Active manager daemon smithi112.xrkkbc restarted 2024-09-16T00:19:02.655 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: Activating manager daemon smithi112.xrkkbc 2024-09-16T00:19:02.655 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: osdmap e5: 0 total, 0 up, 0 in 2024-09-16T00:19:02.655 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: mgrmap e15: smithi112.xrkkbc(active, starting, since 0.00318727s) 2024-09-16T00:19:02.655 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mon metadata", "id": "smithi112"}]: dispatch 2024-09-16T00:19:02.655 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mgr metadata", "who": "smithi112.xrkkbc", "id": "smithi112.xrkkbc"}]: dispatch 2024-09-16T00:19:02.655 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T00:19:02.655 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T00:19:02.656 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T00:19:02.656 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: Manager daemon smithi112.xrkkbc is now available 2024-09-16T00:19:02.656 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:19:02.656 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:02.656 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:19:02.656 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi112.xrkkbc/mirror_snapshot_schedule"}]: dispatch 2024-09-16T00:19:02.656 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi112.xrkkbc/trash_purge_schedule"}]: dispatch 2024-09-16T00:19:02.656 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:02.656 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:02.656 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: mgrmap e16: smithi112.xrkkbc(active, since 1.00523s) 2024-09-16T00:19:02.656 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:02.656 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:02.656 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: [16/Sep/2024:00:18:38] ENGINE Bus STARTING 2024-09-16T00:19:02.656 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: [16/Sep/2024:00:18:38] ENGINE Serving on https://172.21.15.112:7150 2024-09-16T00:19:02.657 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: [16/Sep/2024:00:18:38] ENGINE Bus STARTED 2024-09-16T00:19:02.657 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: [16/Sep/2024:00:18:38] ENGINE Error in HTTPServer.serve 2024-09-16T00:19:02.657 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Traceback (most recent call last): 2024-09-16T00:19:02.657 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T00:19:02.657 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: self._connections.run(self.expiration_interval) 2024-09-16T00:19:02.657 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T00:19:02.657 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: self._run(expiration_interval) 2024-09-16T00:19:02.657 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T00:19:02.657 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T00:19:02.657 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T00:19:02.657 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T00:19:02.657 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T00:19:02.657 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: s = self.context.wrap_socket( 2024-09-16T00:19:02.657 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T00:19:02.657 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: return self.sslsocket_class._create( 2024-09-16T00:19:02.658 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T00:19:02.658 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: self.do_handshake() 2024-09-16T00:19:02.658 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T00:19:02.658 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: self._sslobj.do_handshake() 2024-09-16T00:19:02.658 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T00:19:02.658 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: mgrmap e17: smithi112.xrkkbc(active, since 2s) 2024-09-16T00:19:02.658 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='client.? 172.21.15.169:0/1095252553' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T00:19:02.658 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:02.658 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:02.658 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:02.658 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:02.658 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "who": "osd/host:smithi169", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:19:02.658 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='client.? 172.21.15.169:0/3735332677' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T00:19:02.658 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:02.658 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:02.659 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:02.659 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:02.659 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "who": "osd/host:smithi112", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:19:02.659 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:19:02.659 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:19:02.659 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: Updating smithi112:/etc/ceph/ceph.conf 2024-09-16T00:19:02.659 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: Updating smithi169:/etc/ceph/ceph.conf 2024-09-16T00:19:02.659 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: Updating smithi112:/var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:19:02.659 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: Updating smithi169:/var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:19:02.659 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:02.659 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:02.659 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:02.659 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:02.659 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:02.659 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi169", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-16T00:19:02.660 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi169", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-09-16T00:19:02.660 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:19:02.660 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: Updating smithi112:/etc/ceph/ceph.client.admin.keyring 2024-09-16T00:19:02.660 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: Updating smithi169:/etc/ceph/ceph.client.admin.keyring 2024-09-16T00:19:02.660 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: Updating smithi112:/var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T00:19:02.660 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: Updating smithi169:/var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T00:19:02.660 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: Deploying daemon crash.smithi169 on smithi169 2024-09-16T00:19:02.660 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='client.? 172.21.15.169:0/1579302928' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T00:19:02.660 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:02.660 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:02.660 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:02.660 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:02.660 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: Deploying daemon node-exporter.smithi169 on smithi169 2024-09-16T00:19:02.660 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:02.661 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:19:02.661 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='client.? 172.21.15.169:0/1272468706' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T00:19:02.661 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:02.661 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:02.661 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:02.661 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:02.661 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi169.mtrobk", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T00:19:02.661 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi169.mtrobk", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-16T00:19:02.661 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T00:19:02.661 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:19:02.661 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: Deploying daemon mgr.smithi169.mtrobk on smithi169 2024-09-16T00:19:02.662 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:02.662 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:02.662 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='client.? 172.21.15.169:0/263542728' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T00:19:02.662 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:02.662 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:02.662 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:02.662 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:02.662 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T00:19:02.662 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:19:02.662 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: Deploying daemon mon.smithi169 on smithi169 2024-09-16T00:19:02.662 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:02.662 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:02.662 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:02.662 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 ceph-mon[34653]: mon.smithi169@-1(synchronizing).paxosservice(auth 1..6) refresh upgraded, format 0 -> 3 2024-09-16T00:19:02.891 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 bash[34573]: 6c2289bc3e11d143a2e6df08d0ff86f4766325acc0c76bb93177ee99d0dab9a0 2024-09-16T00:19:02.891 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:02 smithi169 systemd[1]: Started Ceph mon.smithi169 for d346dd56-73c0-11ef-bceb-c7b262605968. 2024-09-16T00:19:04.173 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-16T00:19:04.173 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T00:19:04.430 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi169/config 2024-09-16T00:19:07.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:07 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mon metadata", "id": "smithi112"}]: dispatch 2024-09-16T00:19:07.841 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:07 smithi112 ceph-mon[27869]: mon.smithi112 calling monitor election 2024-09-16T00:19:07.841 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:07 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mon metadata", "id": "smithi169"}]: dispatch 2024-09-16T00:19:07.841 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:07 smithi112 ceph-mon[27869]: from='mgr.? 172.21.15.169:0/2396696635' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi169.mtrobk/crt"}]: dispatch 2024-09-16T00:19:07.841 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:07 smithi112 ceph-mon[27869]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:07.841 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:07 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mon metadata", "id": "smithi169"}]: dispatch 2024-09-16T00:19:07.841 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:07 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mon metadata", "id": "smithi169"}]: dispatch 2024-09-16T00:19:07.841 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:07 smithi112 ceph-mon[27869]: mon.smithi169 calling monitor election 2024-09-16T00:19:07.841 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:07 smithi112 ceph-mon[27869]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:07.841 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:07 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mon metadata", "id": "smithi169"}]: dispatch 2024-09-16T00:19:07.841 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:07 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mon metadata", "id": "smithi169"}]: dispatch 2024-09-16T00:19:07.841 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:07 smithi112 ceph-mon[27869]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:07.842 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:07 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:19:07.842 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:07 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mon metadata", "id": "smithi169"}]: dispatch 2024-09-16T00:19:07.842 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:07 smithi112 ceph-mon[27869]: mon.smithi112 is new leader, mons smithi112,smithi169 in quorum (ranks 0,1) 2024-09-16T00:19:07.842 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:07 smithi112 ceph-mon[27869]: monmap e2: 2 mons at {smithi112=[v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0],smithi169=[v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-16T00:19:07.842 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:07 smithi112 ceph-mon[27869]: fsmap 2024-09-16T00:19:07.842 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:07 smithi112 ceph-mon[27869]: osdmap e5: 0 total, 0 up, 0 in 2024-09-16T00:19:07.842 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:07 smithi112 ceph-mon[27869]: mgrmap e17: smithi112.xrkkbc(active, since 30s) 2024-09-16T00:19:07.842 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:07 smithi112 ceph-mon[27869]: overall HEALTH_OK 2024-09-16T00:19:07.843 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:07 smithi112 ceph-mon[27869]: Standby manager daemon smithi169.mtrobk started 2024-09-16T00:19:07.843 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:07 smithi112 ceph-mon[27869]: from='mgr.? 172.21.15.169:0/2396696635' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T00:19:07.843 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:07 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:07.843 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:07 smithi112 ceph-mon[27869]: from='mgr.? 172.21.15.169:0/2396696635' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi169.mtrobk/key"}]: dispatch 2024-09-16T00:19:07.843 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:07 smithi112 ceph-mon[27869]: from='mgr.? 172.21.15.169:0/2396696635' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T00:19:07.843 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:07 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:07.843 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:07 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:07.843 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:07 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:07.843 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:07 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:08.028 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:07 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mon metadata", "id": "smithi112"}]: dispatch 2024-09-16T00:19:08.028 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:07 smithi169 ceph-mon[34653]: mon.smithi112 calling monitor election 2024-09-16T00:19:08.028 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:07 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mon metadata", "id": "smithi169"}]: dispatch 2024-09-16T00:19:08.028 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:07 smithi169 ceph-mon[34653]: from='mgr.? 172.21.15.169:0/2396696635' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi169.mtrobk/crt"}]: dispatch 2024-09-16T00:19:08.028 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:07 smithi169 ceph-mon[34653]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:08.028 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:07 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mon metadata", "id": "smithi169"}]: dispatch 2024-09-16T00:19:08.028 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:07 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mon metadata", "id": "smithi169"}]: dispatch 2024-09-16T00:19:08.028 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:07 smithi169 ceph-mon[34653]: mon.smithi169 calling monitor election 2024-09-16T00:19:08.028 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:07 smithi169 ceph-mon[34653]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:08.029 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:07 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mon metadata", "id": "smithi169"}]: dispatch 2024-09-16T00:19:08.029 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:07 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mon metadata", "id": "smithi169"}]: dispatch 2024-09-16T00:19:08.029 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:07 smithi169 ceph-mon[34653]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:08.029 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:07 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:19:08.029 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:07 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mon metadata", "id": "smithi169"}]: dispatch 2024-09-16T00:19:08.029 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:07 smithi169 ceph-mon[34653]: mon.smithi112 is new leader, mons smithi112,smithi169 in quorum (ranks 0,1) 2024-09-16T00:19:08.029 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:07 smithi169 ceph-mon[34653]: monmap e2: 2 mons at {smithi112=[v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0],smithi169=[v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-16T00:19:08.029 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:07 smithi169 ceph-mon[34653]: fsmap 2024-09-16T00:19:08.029 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:07 smithi169 ceph-mon[34653]: osdmap e5: 0 total, 0 up, 0 in 2024-09-16T00:19:08.029 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:07 smithi169 ceph-mon[34653]: mgrmap e17: smithi112.xrkkbc(active, since 30s) 2024-09-16T00:19:08.029 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:07 smithi169 ceph-mon[34653]: overall HEALTH_OK 2024-09-16T00:19:08.029 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:07 smithi169 ceph-mon[34653]: Standby manager daemon smithi169.mtrobk started 2024-09-16T00:19:08.029 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:07 smithi169 ceph-mon[34653]: from='mgr.? 172.21.15.169:0/2396696635' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T00:19:08.029 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:07 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:08.029 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:07 smithi169 ceph-mon[34653]: from='mgr.? 172.21.15.169:0/2396696635' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi169.mtrobk/key"}]: dispatch 2024-09-16T00:19:08.030 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:07 smithi169 ceph-mon[34653]: from='mgr.? 172.21.15.169:0/2396696635' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T00:19:08.030 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:07 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:08.030 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:07 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:08.030 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:07 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:08.030 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:07 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:08.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:08 smithi112 ceph-mon[27869]: mgrmap e18: smithi112.xrkkbc(active, since 30s), standbys: smithi169.mtrobk 2024-09-16T00:19:08.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:08 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mgr metadata", "who": "smithi169.mtrobk", "id": "smithi169.mtrobk"}]: dispatch 2024-09-16T00:19:08.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:08 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:19:08.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:08 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mon metadata", "id": "smithi169"}]: dispatch 2024-09-16T00:19:08.847 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:08 smithi169 ceph-mon[34653]: mgrmap e18: smithi112.xrkkbc(active, since 30s), standbys: smithi169.mtrobk 2024-09-16T00:19:08.847 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:08 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mgr metadata", "who": "smithi169.mtrobk", "id": "smithi169.mtrobk"}]: dispatch 2024-09-16T00:19:08.847 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:08 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:19:08.847 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:08 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mon metadata", "id": "smithi169"}]: dispatch 2024-09-16T00:19:08.870 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:08.869+0000 7ff559507640 1 -- 172.21.15.169:0/2904335506 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff554102360 msgr2=0x7ff540005540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:08.870 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:08.869+0000 7ff559507640 1 --2- 172.21.15.169:0/2904335506 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff554102360 0x7ff540005540 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff544007d80 tx=0x7ff54400b040 comp rx=0 tx=0).stop 2024-09-16T00:19:08.870 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:08.869+0000 7ff559507640 1 -- 172.21.15.169:0/2904335506 shutdown_connections 2024-09-16T00:19:08.870 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:08.869+0000 7ff559507640 1 --2- 172.21.15.169:0/2904335506 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff554102360 0x7ff540005540 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:08.870 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:08.869+0000 7ff559507640 1 -- 172.21.15.169:0/2904335506 >> 172.21.15.169:0/2904335506 conn(0x7ff5540707b0 msgr2=0x7ff554070bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:08.870 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:08.869+0000 7ff559507640 1 -- 172.21.15.169:0/2904335506 shutdown_connections 2024-09-16T00:19:08.870 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:08.869+0000 7ff559507640 1 -- 172.21.15.169:0/2904335506 wait complete. 2024-09-16T00:19:08.872 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:08.870+0000 7ff559507640 1 Processor -- start 2024-09-16T00:19:08.872 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:08.870+0000 7ff559507640 1 -- start start 2024-09-16T00:19:08.872 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:08.870+0000 7ff559507640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff554102360 0x7ff554112990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:08.873 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:08.870+0000 7ff559507640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff554112ed0 0x7ff55410c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:08.873 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:08.870+0000 7ff559507640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff554113490 con 0x7ff554112ed0 2024-09-16T00:19:08.873 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:08.870+0000 7ff559507640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5541135d0 con 0x7ff554102360 2024-09-16T00:19:08.873 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:08.871+0000 7ff552ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff554102360 0x7ff554112990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:08.873 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:08.871+0000 7ff552ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff554102360 0x7ff554112990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:49102/0 (socket says 172.21.15.169:49102) 2024-09-16T00:19:08.873 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:08.871+0000 7ff552ffd640 1 -- 172.21.15.169:0/4260951484 learned_addr learned my addr 172.21.15.169:0/4260951484 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-09-16T00:19:08.873 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:08.871+0000 7ff552ffd640 1 -- 172.21.15.169:0/4260951484 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff554102360 msgr2=0x7ff554112990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-16T00:19:08.873 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:08.871+0000 7ff552ffd640 1 -- 172.21.15.169:0/4260951484 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff554102360 msgr2=0x7ff554112990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T00:19:08.873 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:08.871+0000 7ff552ffd640 1 --2- 172.21.15.169:0/4260951484 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff554102360 0x7ff554112990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-16T00:19:08.874 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:08.871+0000 7ff552ffd640 1 --2- 172.21.15.169:0/4260951484 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff554102360 0x7ff554112990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T00:19:08.874 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:08.871+0000 7ff5527fc640 1 --2- 172.21.15.169:0/4260951484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff554112ed0 0x7ff55410c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:08.874 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:08.871+0000 7ff5527fc640 1 -- 172.21.15.169:0/4260951484 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff554102360 msgr2=0x7ff554112990 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T00:19:08.874 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:08.871+0000 7ff5527fc640 1 --2- 172.21.15.169:0/4260951484 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff554102360 0x7ff554112990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:08.874 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:08.871+0000 7ff5527fc640 1 -- 172.21.15.169:0/4260951484 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff544005040 con 0x7ff554112ed0 2024-09-16T00:19:08.874 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:08.871+0000 7ff5527fc640 1 --2- 172.21.15.169:0/4260951484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff554112ed0 0x7ff55410c930 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff53c00b580 tx=0x7ff53c0090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:08.874 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:08.871+0000 7ff54b7fe640 1 -- 172.21.15.169:0/4260951484 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff53c011690 con 0x7ff554112ed0 2024-09-16T00:19:08.874 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:08.872+0000 7ff559507640 1 -- 172.21.15.169:0/4260951484 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff55410cf30 con 0x7ff554112ed0 2024-09-16T00:19:08.874 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:08.872+0000 7ff54b7fe640 1 -- 172.21.15.169:0/4260951484 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7ff53c011c90 con 0x7ff554112ed0 2024-09-16T00:19:08.874 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:08.872+0000 7ff54b7fe640 1 -- 172.21.15.169:0/4260951484 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff53c010e10 con 0x7ff554112ed0 2024-09-16T00:19:08.874 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:08.872+0000 7ff559507640 1 -- 172.21.15.169:0/4260951484 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff55410d450 con 0x7ff554112ed0 2024-09-16T00:19:08.875 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:08.873+0000 7ff559507640 1 -- 172.21.15.169:0/4260951484 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff55410d730 con 0x7ff554112ed0 2024-09-16T00:19:08.876 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:08.873+0000 7ff54b7fe640 1 -- 172.21.15.169:0/4260951484 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) v1 ==== 90813+0+0 (secure 0 0 0) 0x7ff53c010600 con 0x7ff554112ed0 2024-09-16T00:19:08.876 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:08.874+0000 7ff54b7fe640 1 --2- 172.21.15.169:0/4260951484 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7ff53406edd0 0x7ff534071280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:08.876 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:08.874+0000 7ff54b7fe640 1 -- 172.21.15.169:0/4260951484 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7ff53c08d470 con 0x7ff554112ed0 2024-09-16T00:19:08.876 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:08.874+0000 7ff552ffd640 1 --2- 172.21.15.169:0/4260951484 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7ff53406edd0 0x7ff534071280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:08.876 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:08.874+0000 7ff552ffd640 1 --2- 172.21.15.169:0/4260951484 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7ff53406edd0 0x7ff534071280 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff544007c10 tx=0x7ff54400b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:08.878 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:08.878+0000 7ff54b7fe640 1 -- 172.21.15.169:0/4260951484 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7ff53c05b5d0 con 0x7ff554112ed0 2024-09-16T00:19:09.095 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:09.095+0000 7ff559507640 1 -- 172.21.15.169:0/4260951484 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ff5540fdcc0 con 0x7ff554112ed0 2024-09-16T00:19:09.096 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:09.096+0000 7ff54b7fe640 1 -- 172.21.15.169:0/4260951484 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1021 (secure 0 0 0) 0x7ff53c015090 con 0x7ff554112ed0 2024-09-16T00:19:09.096 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:19:09.096 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":2,"fsid":"d346dd56-73c0-11ef-bceb-c7b262605968","modified":"2024-09-16T00:19:02.487171Z","created":"2024-09-16T00:16:40.582056Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi112","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"smithi169","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:3300","nonce":0},{"type":"v1","addr":"172.21.15.169:6789","nonce":0}]},"addr":"172.21.15.169:6789/0","public_addr":"172.21.15.169:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-16T00:19:09.096 INFO:teuthology.orchestra.run.smithi169.stderr:dumped monmap epoch 2 2024-09-16T00:19:09.098 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:09.098+0000 7ff559507640 1 -- 172.21.15.169:0/4260951484 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7ff53406edd0 msgr2=0x7ff534071280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:09.098 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:09.098+0000 7ff559507640 1 --2- 172.21.15.169:0/4260951484 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7ff53406edd0 0x7ff534071280 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff544007c10 tx=0x7ff54400b040 comp rx=0 tx=0).stop 2024-09-16T00:19:09.098 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:09.098+0000 7ff559507640 1 -- 172.21.15.169:0/4260951484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff554112ed0 msgr2=0x7ff55410c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:09.099 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:09.098+0000 7ff559507640 1 --2- 172.21.15.169:0/4260951484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff554112ed0 0x7ff55410c930 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff53c00b580 tx=0x7ff53c0090f0 comp rx=0 tx=0).stop 2024-09-16T00:19:09.099 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:09.098+0000 7ff559507640 1 -- 172.21.15.169:0/4260951484 shutdown_connections 2024-09-16T00:19:09.099 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:09.098+0000 7ff559507640 1 --2- 172.21.15.169:0/4260951484 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7ff53406edd0 0x7ff534071280 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:09.099 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:09.098+0000 7ff559507640 1 --2- 172.21.15.169:0/4260951484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff554112ed0 0x7ff55410c930 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:09.099 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:09.098+0000 7ff559507640 1 --2- 172.21.15.169:0/4260951484 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff554102360 0x7ff554112990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:09.099 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:09.098+0000 7ff559507640 1 -- 172.21.15.169:0/4260951484 >> 172.21.15.169:0/4260951484 conn(0x7ff5540707b0 msgr2=0x7ff5541057d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:09.099 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:09.098+0000 7ff559507640 1 -- 172.21.15.169:0/4260951484 shutdown_connections 2024-09-16T00:19:09.099 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:19:09.098+0000 7ff559507640 1 -- 172.21.15.169:0/4260951484 wait complete. 2024-09-16T00:19:09.827 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:09 smithi169 ceph-mon[34653]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:09.828 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:09 smithi169 ceph-mon[34653]: from='client.? 172.21.15.169:0/4260951484' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T00:19:09.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:09 smithi112 ceph-mon[27869]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:09.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:09 smithi112 ceph-mon[27869]: from='client.? 172.21.15.169:0/4260951484' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T00:19:09.854 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-16T00:19:09.855 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph config generate-minimal-conf 2024-09-16T00:19:10.129 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:19:11.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:10 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:11.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:10 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:11.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:10 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:19:11.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:10 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:19:11.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:10 smithi169 ceph-mon[34653]: Updating smithi112:/etc/ceph/ceph.conf 2024-09-16T00:19:11.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:10 smithi169 ceph-mon[34653]: Updating smithi169:/etc/ceph/ceph.conf 2024-09-16T00:19:11.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:10 smithi169 ceph-mon[34653]: Updating smithi169:/var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:19:11.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:10 smithi169 ceph-mon[34653]: Updating smithi112:/var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:19:11.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:10 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:11.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:10 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:11.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:10 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:11.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:10 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:11.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:10 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:11.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:10 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T00:19:11.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:10 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T00:19:11.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:10 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:19:11.295 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:10 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:11.295 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:10 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:11.295 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:10 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:19:11.295 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:10 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:19:11.295 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:10 smithi112 ceph-mon[27869]: Updating smithi112:/etc/ceph/ceph.conf 2024-09-16T00:19:11.295 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:10 smithi112 ceph-mon[27869]: Updating smithi169:/etc/ceph/ceph.conf 2024-09-16T00:19:11.296 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:10 smithi112 ceph-mon[27869]: Updating smithi169:/var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:19:11.296 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:10 smithi112 ceph-mon[27869]: Updating smithi112:/var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:19:11.296 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:10 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:11.296 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:10 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:11.296 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:10 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:11.296 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:10 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:11.296 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:10 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:11.296 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:10 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T00:19:11.296 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:10 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T00:19:11.296 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:10 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:19:12.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:11 smithi169 ceph-mon[34653]: Reconfiguring mon.smithi112 (unknown last config time)... 2024-09-16T00:19:12.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:11 smithi169 ceph-mon[34653]: Reconfiguring daemon mon.smithi112 on smithi112 2024-09-16T00:19:12.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:11 smithi169 ceph-mon[34653]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:12.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:11 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:12.322 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:11 smithi112 ceph-mon[27869]: Reconfiguring mon.smithi112 (unknown last config time)... 2024-09-16T00:19:12.322 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:11 smithi112 ceph-mon[27869]: Reconfiguring daemon mon.smithi112 on smithi112 2024-09-16T00:19:12.323 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:11 smithi112 ceph-mon[27869]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:12.323 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:11 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:12.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:12.819+0000 7fb1c5b5c640 1 -- 172.21.15.112:0/2033300840 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1c00fcf80 msgr2=0x7fb1c00fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:12.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:12.819+0000 7fb1c5b5c640 1 --2- 172.21.15.112:0/2033300840 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1c00fcf80 0x7fb1c00fd3d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb1a80048d0 tx=0x7fb1a8009da0 comp rx=0 tx=0).stop 2024-09-16T00:19:12.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:12.819+0000 7fb1c5b5c640 1 -- 172.21.15.112:0/2033300840 shutdown_connections 2024-09-16T00:19:12.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:12.819+0000 7fb1c5b5c640 1 --2- 172.21.15.112:0/2033300840 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1c00fcf80 0x7fb1c00fd3d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:12.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:12.819+0000 7fb1c5b5c640 1 --2- 172.21.15.112:0/2033300840 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1c00fb810 0x7fb1c00fbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:12.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:12.819+0000 7fb1c5b5c640 1 -- 172.21.15.112:0/2033300840 >> 172.21.15.112:0/2033300840 conn(0x7fb1c00723b0 msgr2=0x7fb1c00727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:12.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:12.819+0000 7fb1c5b5c640 1 -- 172.21.15.112:0/2033300840 shutdown_connections 2024-09-16T00:19:12.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:12.819+0000 7fb1c5b5c640 1 -- 172.21.15.112:0/2033300840 wait complete. 2024-09-16T00:19:12.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:12.819+0000 7fb1c5b5c640 1 Processor -- start 2024-09-16T00:19:12.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:12.820+0000 7fb1c5b5c640 1 -- start start 2024-09-16T00:19:12.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:12.820+0000 7fb1c5b5c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1c00fb810 0x7fb1c010c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:12.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:12.820+0000 7fb1c5b5c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1c00fcf80 0x7fb1c010c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:12.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:12.820+0000 7fb1c5b5c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c010cab0 con 0x7fb1c00fb810 2024-09-16T00:19:12.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:12.820+0000 7fb1c5b5c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c010cc20 con 0x7fb1c00fcf80 2024-09-16T00:19:12.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:12.820+0000 7fb1bf7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1c00fb810 0x7fb1c010c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:12.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:12.820+0000 7fb1bf7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1c00fb810 0x7fb1c010c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55252/0 (socket says 172.21.15.112:55252) 2024-09-16T00:19:12.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:12.820+0000 7fb1bf7fe640 1 -- 172.21.15.112:0/2181785759 learned_addr learned my addr 172.21.15.112:0/2181785759 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:19:12.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:12.820+0000 7fb1bf7fe640 1 -- 172.21.15.112:0/2181785759 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1c00fcf80 msgr2=0x7fb1c010c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:12.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:12.820+0000 7fb1bf7fe640 1 --2- 172.21.15.112:0/2181785759 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1c00fcf80 0x7fb1c010c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:12.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:12.820+0000 7fb1bf7fe640 1 -- 172.21.15.112:0/2181785759 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1a8005040 con 0x7fb1c00fb810 2024-09-16T00:19:12.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:12.820+0000 7fb1bf7fe640 1 --2- 172.21.15.112:0/2181785759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1c00fb810 0x7fb1c010c030 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb1b800bbb0 tx=0x7fb1b8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:12.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:12.821+0000 7fb1b7fff640 1 -- 172.21.15.112:0/2181785759 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb1b800c9c0 con 0x7fb1c00fb810 2024-09-16T00:19:12.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:12.821+0000 7fb1b7fff640 1 -- 172.21.15.112:0/2181785759 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fb1b80092e0 con 0x7fb1c00fb810 2024-09-16T00:19:12.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:12.821+0000 7fb1c5b5c640 1 -- 172.21.15.112:0/2181785759 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1c010ceb0 con 0x7fb1c00fb810 2024-09-16T00:19:12.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:12.821+0000 7fb1b7fff640 1 -- 172.21.15.112:0/2181785759 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb1b8007640 con 0x7fb1c00fb810 2024-09-16T00:19:12.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:12.821+0000 7fb1c5b5c640 1 -- 172.21.15.112:0/2181785759 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1c01199d0 con 0x7fb1c00fb810 2024-09-16T00:19:12.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:12.822+0000 7fb1c5b5c640 1 -- 172.21.15.112:0/2181785759 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1c00fcbc0 con 0x7fb1c00fb810 2024-09-16T00:19:12.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:12.822+0000 7fb1b7fff640 1 -- 172.21.15.112:0/2181785759 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) v1 ==== 90813+0+0 (secure 0 0 0) 0x7fb1b8012070 con 0x7fb1c00fb810 2024-09-16T00:19:12.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:12.823+0000 7fb1b7fff640 1 --2- 172.21.15.112:0/2181785759 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fb1ac06ecc0 0x7fb1ac071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:12.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:12.823+0000 7fb1beffd640 1 --2- 172.21.15.112:0/2181785759 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fb1ac06ecc0 0x7fb1ac071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:12.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:12.823+0000 7fb1b7fff640 1 -- 172.21.15.112:0/2181785759 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fb1b808cf40 con 0x7fb1c00fb810 2024-09-16T00:19:12.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:12.823+0000 7fb1beffd640 1 --2- 172.21.15.112:0/2181785759 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fb1ac06ecc0 0x7fb1ac071170 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb1a802a530 tx=0x7fb1a8009da0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:12.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:12.827+0000 7fb1b7fff640 1 -- 172.21.15.112:0/2181785759 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fb1b805b1c0 con 0x7fb1c00fb810 2024-09-16T00:19:12.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:12.992+0000 7fb1c5b5c640 1 -- 172.21.15.112:0/2181785759 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fb1c0072ec0 con 0x7fb1c00fb810 2024-09-16T00:19:12.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:12.992+0000 7fb1b7fff640 1 -- 172.21.15.112:0/2181785759 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v10) v1 ==== 76+0+227 (secure 0 0 0) 0x7fb1b808c6f0 con 0x7fb1c00fb810 2024-09-16T00:19:12.994 INFO:teuthology.orchestra.run.smithi112.stdout:# minimal ceph.conf for d346dd56-73c0-11ef-bceb-c7b262605968 2024-09-16T00:19:12.994 INFO:teuthology.orchestra.run.smithi112.stdout:[global] 2024-09-16T00:19:12.994 INFO:teuthology.orchestra.run.smithi112.stdout: fsid = d346dd56-73c0-11ef-bceb-c7b262605968 2024-09-16T00:19:12.994 INFO:teuthology.orchestra.run.smithi112.stdout: mon_host = [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] 2024-09-16T00:19:12.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:12.994+0000 7fb1c5b5c640 1 -- 172.21.15.112:0/2181785759 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fb1ac06ecc0 msgr2=0x7fb1ac071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:12.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:12.994+0000 7fb1c5b5c640 1 --2- 172.21.15.112:0/2181785759 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fb1ac06ecc0 0x7fb1ac071170 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb1a802a530 tx=0x7fb1a8009da0 comp rx=0 tx=0).stop 2024-09-16T00:19:12.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:12.994+0000 7fb1c5b5c640 1 -- 172.21.15.112:0/2181785759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1c00fb810 msgr2=0x7fb1c010c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:12.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:12.994+0000 7fb1c5b5c640 1 --2- 172.21.15.112:0/2181785759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1c00fb810 0x7fb1c010c030 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb1b800bbb0 tx=0x7fb1b8009840 comp rx=0 tx=0).stop 2024-09-16T00:19:12.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:12.994+0000 7fb1c5b5c640 1 -- 172.21.15.112:0/2181785759 shutdown_connections 2024-09-16T00:19:12.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:12.994+0000 7fb1c5b5c640 1 --2- 172.21.15.112:0/2181785759 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fb1ac06ecc0 0x7fb1ac071170 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:12.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:12.994+0000 7fb1c5b5c640 1 --2- 172.21.15.112:0/2181785759 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1c00fcf80 0x7fb1c010c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:12.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:12.994+0000 7fb1c5b5c640 1 --2- 172.21.15.112:0/2181785759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1c00fb810 0x7fb1c010c030 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:12.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:12.994+0000 7fb1c5b5c640 1 -- 172.21.15.112:0/2181785759 >> 172.21.15.112:0/2181785759 conn(0x7fb1c00723b0 msgr2=0x7fb1c00fa480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:12.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:12.994+0000 7fb1c5b5c640 1 -- 172.21.15.112:0/2181785759 shutdown_connections 2024-09-16T00:19:12.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:12.994+0000 7fb1c5b5c640 1 -- 172.21.15.112:0/2181785759 wait complete. 2024-09-16T00:19:13.736 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:13 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:13.736 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:13 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:13.736 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:13 smithi112 ceph-mon[27869]: Reconfiguring mgr.smithi112.xrkkbc (unknown last config time)... 2024-09-16T00:19:13.736 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:13 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi112.xrkkbc", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T00:19:13.736 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:13 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T00:19:13.736 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:13 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:19:13.736 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:13 smithi112 ceph-mon[27869]: Reconfiguring daemon mgr.smithi112.xrkkbc on smithi112 2024-09-16T00:19:13.737 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:13 smithi112 ceph-mon[27869]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:13.737 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:13 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/2181785759' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:19:13.760 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-16T00:19:13.760 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-16T00:19:13.760 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/ceph/ceph.conf 2024-09-16T00:19:13.788 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-16T00:19:13.788 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-16T00:19:13.859 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-09-16T00:19:13.859 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/ceph/ceph.conf 2024-09-16T00:19:13.887 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:13 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:13.887 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:13 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:13.887 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:13 smithi169 ceph-mon[34653]: Reconfiguring mgr.smithi112.xrkkbc (unknown last config time)... 2024-09-16T00:19:13.887 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:13 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi112.xrkkbc", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T00:19:13.887 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:13 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T00:19:13.887 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:13 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:19:13.888 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:13 smithi169 ceph-mon[34653]: Reconfiguring daemon mgr.smithi112.xrkkbc on smithi112 2024-09-16T00:19:13.888 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:13 smithi169 ceph-mon[34653]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:13.888 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:13 smithi169 ceph-mon[34653]: from='client.? 172.21.15.112:0/2181785759' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:19:13.892 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-09-16T00:19:13.892 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-16T00:19:13.962 INFO:tasks.cephadm:Deploying OSDs... 2024-09-16T00:19:13.962 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-16T00:19:13.962 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T00:19:13.979 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T00:19:13.979 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_1 2024-09-16T00:19:14.035 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T00:19:14.035 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T00:19:14.035 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-16T00:19:14.035 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T00:19:14.035 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T00:19:14.035 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-09-16 00:18:30.640914250 +0000 2024-09-16T00:19:14.035 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-09-16 00:13:16.052723718 +0000 2024-09-16T00:19:14.035 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-09-16 00:13:16.052723718 +0000 2024-09-16T00:19:14.035 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-09-16 00:13:16.052723718 +0000 2024-09-16T00:19:14.036 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T00:19:14.098 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-09-16T00:19:14.099 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-09-16T00:19:14.099 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000228606 s, 2.2 MB/s 2024-09-16T00:19:14.101 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T00:19:14.158 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_2 2024-09-16T00:19:14.213 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T00:19:14.213 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T00:19:14.213 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-16T00:19:14.213 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T00:19:14.213 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T00:19:14.214 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-09-16 00:18:30.641914225 +0000 2024-09-16T00:19:14.214 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-09-16 00:13:16.380715258 +0000 2024-09-16T00:19:14.214 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-09-16 00:13:16.380715258 +0000 2024-09-16T00:19:14.214 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-09-16 00:13:16.380715258 +0000 2024-09-16T00:19:14.214 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T00:19:14.276 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-09-16T00:19:14.276 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-09-16T00:19:14.276 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000235435 s, 2.2 MB/s 2024-09-16T00:19:14.278 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T00:19:14.337 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_3 2024-09-16T00:19:14.393 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T00:19:14.393 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T00:19:14.393 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-16T00:19:14.393 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T00:19:14.393 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T00:19:14.393 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-09-16 00:18:30.641914225 +0000 2024-09-16T00:19:14.393 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-09-16 00:13:16.709706772 +0000 2024-09-16T00:19:14.393 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-09-16 00:13:16.709706772 +0000 2024-09-16T00:19:14.393 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-09-16 00:13:16.709706772 +0000 2024-09-16T00:19:14.394 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T00:19:14.456 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-09-16T00:19:14.456 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-09-16T00:19:14.456 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000202475 s, 2.5 MB/s 2024-09-16T00:19:14.457 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T00:19:14.515 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_4 2024-09-16T00:19:14.570 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T00:19:14.570 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T00:19:14.570 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-16T00:19:14.570 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T00:19:14.571 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T00:19:14.571 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-09-16 00:18:30.641914225 +0000 2024-09-16T00:19:14.571 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-09-16 00:13:17.045698106 +0000 2024-09-16T00:19:14.571 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-09-16 00:13:17.045698106 +0000 2024-09-16T00:19:14.571 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-09-16 00:13:17.045698106 +0000 2024-09-16T00:19:14.571 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T00:19:14.634 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-09-16T00:19:14.634 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-09-16T00:19:14.634 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000213486 s, 2.4 MB/s 2024-09-16T00:19:14.635 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T00:19:14.691 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-09-16T00:19:14.692 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T00:19:14.712 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T00:19:14.712 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_1 2024-09-16T00:19:14.770 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T00:19:14.771 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T00:19:14.771 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-16T00:19:14.771 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T00:19:14.771 INFO:teuthology.orchestra.run.smithi169.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T00:19:14.771 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2024-09-16 00:19:00.749950914 +0000 2024-09-16T00:19:14.771 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2024-09-16 00:13:16.028999113 +0000 2024-09-16T00:19:14.771 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2024-09-16 00:13:16.028999113 +0000 2024-09-16T00:19:14.771 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: 2024-09-16 00:13:16.028999113 +0000 2024-09-16T00:19:14.771 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T00:19:14.836 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2024-09-16T00:19:14.836 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2024-09-16T00:19:14.836 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.000206835 s, 2.5 MB/s 2024-09-16T00:19:14.837 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T00:19:14.893 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_2 2024-09-16T00:19:14.949 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T00:19:14.949 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T00:19:14.949 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-16T00:19:14.949 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T00:19:14.949 INFO:teuthology.orchestra.run.smithi169.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T00:19:14.950 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2024-09-16 00:19:00.750950893 +0000 2024-09-16T00:19:14.950 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2024-09-16 00:13:16.341992457 +0000 2024-09-16T00:19:14.950 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2024-09-16 00:13:16.341992457 +0000 2024-09-16T00:19:14.950 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: 2024-09-16 00:13:16.341992457 +0000 2024-09-16T00:19:14.950 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T00:19:15.013 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2024-09-16T00:19:15.013 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2024-09-16T00:19:15.013 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.000204952 s, 2.5 MB/s 2024-09-16T00:19:15.014 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T00:19:15.070 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_3 2024-09-16T00:19:15.128 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T00:19:15.129 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T00:19:15.129 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-16T00:19:15.129 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T00:19:15.129 INFO:teuthology.orchestra.run.smithi169.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T00:19:15.129 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2024-09-16 00:19:00.750950893 +0000 2024-09-16T00:19:15.129 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2024-09-16 00:13:16.656985759 +0000 2024-09-16T00:19:15.129 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2024-09-16 00:13:16.656985759 +0000 2024-09-16T00:19:15.129 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: 2024-09-16 00:13:16.656985759 +0000 2024-09-16T00:19:15.129 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T00:19:15.194 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2024-09-16T00:19:15.194 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2024-09-16T00:19:15.194 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.000200124 s, 2.6 MB/s 2024-09-16T00:19:15.195 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T00:19:15.251 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_4 2024-09-16T00:19:15.307 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T00:19:15.307 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T00:19:15.307 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-16T00:19:15.307 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T00:19:15.307 INFO:teuthology.orchestra.run.smithi169.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T00:19:15.307 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2024-09-16 00:19:00.750950893 +0000 2024-09-16T00:19:15.307 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2024-09-16 00:13:16.990978657 +0000 2024-09-16T00:19:15.308 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2024-09-16 00:13:16.990978657 +0000 2024-09-16T00:19:15.308 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: 2024-09-16 00:13:16.990978657 +0000 2024-09-16T00:19:15.308 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T00:19:15.371 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2024-09-16T00:19:15.371 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2024-09-16T00:19:15.372 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.000232626 s, 2.2 MB/s 2024-09-16T00:19:15.373 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T00:19:15.429 INFO:tasks.cephadm:Deploying osd.0 on smithi112 with /dev/vg_nvme/lv_4... 2024-09-16T00:19:15.430 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-16T00:19:15.704 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:19:16.081 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:15 smithi112 ceph-mon[27869]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:16.082 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:15 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:16.082 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:15 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:16.082 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:15 smithi112 ceph-mon[27869]: Reconfiguring crash.smithi112 (monmap changed)... 2024-09-16T00:19:16.082 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:15 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi112", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-16T00:19:16.082 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:15 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:19:16.082 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:15 smithi112 ceph-mon[27869]: Reconfiguring daemon crash.smithi112 on smithi112 2024-09-16T00:19:16.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:15 smithi169 ceph-mon[34653]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:16.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:15 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:16.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:15 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:16.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:15 smithi169 ceph-mon[34653]: Reconfiguring crash.smithi112 (monmap changed)... 2024-09-16T00:19:16.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:15 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi112", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-16T00:19:16.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:15 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:19:16.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:15 smithi169 ceph-mon[34653]: Reconfiguring daemon crash.smithi112 on smithi112 2024-09-16T00:19:17.322 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:17 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:17.322 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:17 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:17.323 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:17 smithi112 ceph-mon[27869]: Reconfiguring alertmanager.smithi112 (dependencies changed)... 2024-09-16T00:19:17.323 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:17 smithi112 ceph-mon[27869]: Reconfiguring daemon alertmanager.smithi112 on smithi112 2024-09-16T00:19:17.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:17 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:17.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:17 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:17.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:17 smithi169 ceph-mon[34653]: Reconfiguring alertmanager.smithi112 (dependencies changed)... 2024-09-16T00:19:17.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:17 smithi169 ceph-mon[34653]: Reconfiguring daemon alertmanager.smithi112 on smithi112 2024-09-16T00:19:18.477 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:18 smithi112 ceph-mon[27869]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:18.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:18 smithi169 ceph-mon[34653]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:20.299 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:20 smithi112 ceph-mon[27869]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:20.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:20 smithi169 ceph-mon[34653]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:22.475 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:22 smithi112 ceph-mon[27869]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:22.475 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:22 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:19:22.475 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:22 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:22.475 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:22 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:22.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:22 smithi169 ceph-mon[34653]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:22.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:22 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:19:22.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:22 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:22.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:22 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:23.428 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:23 smithi112 ceph-mon[27869]: Reconfiguring grafana.smithi112 (dependencies changed)... 2024-09-16T00:19:23.429 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:23 smithi112 ceph-mon[27869]: Reconfiguring daemon grafana.smithi112 on smithi112 2024-09-16T00:19:23.435 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:19:23.453 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi112:vg_nvme/lv_4 2024-09-16T00:19:23.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:23 smithi169 ceph-mon[34653]: Reconfiguring grafana.smithi112 (dependencies changed)... 2024-09-16T00:19:23.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:23 smithi169 ceph-mon[34653]: Reconfiguring daemon grafana.smithi112 on smithi112 2024-09-16T00:19:24.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:24 smithi169 ceph-mon[34653]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:24.560 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:24 smithi112 ceph-mon[27869]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:24.564 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:19:26.261 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:26 smithi112 ceph-mon[27869]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:26.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:26 smithi169 ceph-mon[34653]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:27.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:27.197+0000 7f131c37c640 1 -- 172.21.15.112:0/3720381760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13140fd830 msgr2=0x7f13140fdca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:27.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:27.197+0000 7f131c37c640 1 --2- 172.21.15.112:0/3720381760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13140fd830 0x7f13140fdca0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f13080048d0 tx=0x7f1308009da0 comp rx=0 tx=0).stop 2024-09-16T00:19:27.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:27.197+0000 7f131c37c640 1 -- 172.21.15.112:0/3720381760 shutdown_connections 2024-09-16T00:19:27.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:27.197+0000 7f131c37c640 1 --2- 172.21.15.112:0/3720381760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13140fd830 0x7f13140fdca0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:27.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:27.197+0000 7f131c37c640 1 --2- 172.21.15.112:0/3720381760 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13140fc560 0x7f13140fc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:27.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:27.197+0000 7f131c37c640 1 -- 172.21.15.112:0/3720381760 >> 172.21.15.112:0/3720381760 conn(0x7f13140f7d20 msgr2=0x7f13140fa150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:27.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:27.197+0000 7f131c37c640 1 -- 172.21.15.112:0/3720381760 shutdown_connections 2024-09-16T00:19:27.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:27.197+0000 7f131c37c640 1 -- 172.21.15.112:0/3720381760 wait complete. 2024-09-16T00:19:27.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:27.197+0000 7f131c37c640 1 Processor -- start 2024-09-16T00:19:27.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:27.197+0000 7f131c37c640 1 -- start start 2024-09-16T00:19:27.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:27.197+0000 7f131c37c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13140fc560 0x7f131410d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:27.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:27.197+0000 7f131c37c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13140fd830 0x7f131410d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:27.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:27.197+0000 7f131c37c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f131410e470 con 0x7f13140fd830 2024-09-16T00:19:27.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:27.197+0000 7f131c37c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f131410e5e0 con 0x7f13140fc560 2024-09-16T00:19:27.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:27.197+0000 7f1319917640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13140fd830 0x7f131410d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:27.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:27.197+0000 7f1319917640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13140fd830 0x7f131410d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60654/0 (socket says 172.21.15.112:60654) 2024-09-16T00:19:27.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:27.197+0000 7f1319917640 1 -- 172.21.15.112:0/2548008287 learned_addr learned my addr 172.21.15.112:0/2548008287 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:19:27.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:27.198+0000 7f131a118640 1 --2- 172.21.15.112:0/2548008287 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13140fc560 0x7f131410d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:27.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:27.198+0000 7f1319917640 1 -- 172.21.15.112:0/2548008287 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13140fc560 msgr2=0x7f131410d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:27.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:27.198+0000 7f1319917640 1 --2- 172.21.15.112:0/2548008287 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13140fc560 0x7f131410d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:27.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:27.198+0000 7f1319917640 1 -- 172.21.15.112:0/2548008287 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1308005040 con 0x7f13140fd830 2024-09-16T00:19:27.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:27.198+0000 7f1319917640 1 --2- 172.21.15.112:0/2548008287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13140fd830 0x7f131410d600 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1308003080 tx=0x7f1308009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:27.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:27.198+0000 7f1312ffd640 1 -- 172.21.15.112:0/2548008287 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1308038070 con 0x7f13140fd830 2024-09-16T00:19:27.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:27.198+0000 7f131c37c640 1 -- 172.21.15.112:0/2548008287 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f131410e860 con 0x7f13140fd830 2024-09-16T00:19:27.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:27.198+0000 7f1312ffd640 1 -- 172.21.15.112:0/2548008287 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f1308009800 con 0x7f13140fd830 2024-09-16T00:19:27.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:27.198+0000 7f1312ffd640 1 -- 172.21.15.112:0/2548008287 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1308033660 con 0x7f13140fd830 2024-09-16T00:19:27.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:27.198+0000 7f131c37c640 1 -- 172.21.15.112:0/2548008287 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f131410ed20 con 0x7f13140fd830 2024-09-16T00:19:27.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:27.199+0000 7f131c37c640 1 -- 172.21.15.112:0/2548008287 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13140fb9c0 con 0x7f13140fd830 2024-09-16T00:19:27.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:27.199+0000 7f1312ffd640 1 -- 172.21.15.112:0/2548008287 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) v1 ==== 90813+0+0 (secure 0 0 0) 0x7f1308030020 con 0x7f13140fd830 2024-09-16T00:19:27.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:27.200+0000 7f1312ffd640 1 --2- 172.21.15.112:0/2548008287 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f12fc06ecc0 0x7f12fc071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:27.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:27.200+0000 7f1312ffd640 1 -- 172.21.15.112:0/2548008287 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f13080ad210 con 0x7f13140fd830 2024-09-16T00:19:27.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:27.200+0000 7f131a118640 1 --2- 172.21.15.112:0/2548008287 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f12fc06ecc0 0x7f12fc071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:27.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:27.200+0000 7f131a118640 1 --2- 172.21.15.112:0/2548008287 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f12fc06ecc0 0x7f12fc071170 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1304001250 tx=0x7f1304009620 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:27.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:27.204+0000 7f1312ffd640 1 -- 172.21.15.112:0/2548008287 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f130807b410 con 0x7f13140fd830 2024-09-16T00:19:27.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:27.373+0000 7f131c37c640 1 -- 172.21.15.112:0/2548008287 --> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f131410ef40 con 0x7f12fc06ecc0 2024-09-16T00:19:28.337 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:28 smithi112 ceph-mon[27869]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:28.338 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:28 smithi112 ceph-mon[27869]: from='client.14242 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:19:28.338 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:28 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:19:28.338 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:28 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:19:28.338 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:28 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:19:28.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:28 smithi169 ceph-mon[34653]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:28.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:28 smithi169 ceph-mon[34653]: from='client.14242 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:19:28.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:28 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:19:28.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:28 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:19:28.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:28 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:19:30.376 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:30 smithi112 ceph-mon[27869]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:30.376 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:30 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:30.376 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:30 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:30.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:30 smithi169 ceph-mon[34653]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:30.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:30 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:30.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:30 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:31.405 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:31 smithi112 ceph-mon[27869]: Reconfiguring prometheus.smithi112 (dependencies changed)... 2024-09-16T00:19:31.405 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:31 smithi112 ceph-mon[27869]: Reconfiguring daemon prometheus.smithi112 on smithi112 2024-09-16T00:19:31.405 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:31 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/3950732504' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f6c0fb9d-a0e6-44b1-a66d-ed3a1955c461"}]: dispatch 2024-09-16T00:19:31.405 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:31 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/3950732504' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f6c0fb9d-a0e6-44b1-a66d-ed3a1955c461"}]': finished 2024-09-16T00:19:31.405 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:31 smithi112 ceph-mon[27869]: osdmap e6: 1 total, 0 up, 1 in 2024-09-16T00:19:31.405 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:31 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:19:31.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:31 smithi169 ceph-mon[34653]: Reconfiguring prometheus.smithi112 (dependencies changed)... 2024-09-16T00:19:31.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:31 smithi169 ceph-mon[34653]: Reconfiguring daemon prometheus.smithi112 on smithi112 2024-09-16T00:19:31.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:31 smithi169 ceph-mon[34653]: from='client.? 172.21.15.112:0/3950732504' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f6c0fb9d-a0e6-44b1-a66d-ed3a1955c461"}]: dispatch 2024-09-16T00:19:31.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:31 smithi169 ceph-mon[34653]: from='client.? 172.21.15.112:0/3950732504' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f6c0fb9d-a0e6-44b1-a66d-ed3a1955c461"}]': finished 2024-09-16T00:19:31.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:31 smithi169 ceph-mon[34653]: osdmap e6: 1 total, 0 up, 1 in 2024-09-16T00:19:31.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:31 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:19:32.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:32 smithi169 ceph-mon[34653]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:32.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:32 smithi169 ceph-mon[34653]: from='client.? 172.21.15.112:0/257381013' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:19:32.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:32 smithi112 ceph-mon[27869]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:32.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:32 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/257381013' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:19:34.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:34 smithi169 ceph-mon[34653]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:34.558 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:34 smithi112 ceph-mon[27869]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:36.489 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:36 smithi112 ceph-mon[27869]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:36.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:36 smithi169 ceph-mon[34653]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:37.505 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:37 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:19:37.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:37 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:19:38.403 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:38 smithi112 ceph-mon[27869]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:38.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:38 smithi169 ceph-mon[34653]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:40.417 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:40 smithi112 ceph-mon[27869]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:40.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:40 smithi169 ceph-mon[34653]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:41.758 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:41 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:41.759 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:41 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:41.759 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:41 smithi112 ceph-mon[27869]: Reconfiguring crash.smithi169 (monmap changed)... 2024-09-16T00:19:41.759 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:41 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi169", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-16T00:19:41.759 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:41 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:19:41.759 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:41 smithi112 ceph-mon[27869]: Reconfiguring daemon crash.smithi169 on smithi169 2024-09-16T00:19:41.759 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:41 smithi112 ceph-mon[27869]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:41.866 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:41 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:41.866 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:41 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:41.866 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:41 smithi169 ceph-mon[34653]: Reconfiguring crash.smithi169 (monmap changed)... 2024-09-16T00:19:41.866 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:41 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi169", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-16T00:19:41.866 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:41 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:19:41.866 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:41 smithi169 ceph-mon[34653]: Reconfiguring daemon crash.smithi169 on smithi169 2024-09-16T00:19:41.866 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:41 smithi169 ceph-mon[34653]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:42.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:42 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-16T00:19:42.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:42 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:19:42.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:42 smithi112 ceph-mon[27869]: Deploying daemon osd.0 on smithi112 2024-09-16T00:19:42.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:42 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:42.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:42 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:42.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:42 smithi112 ceph-mon[27869]: Reconfiguring mgr.smithi169.mtrobk (monmap changed)... 2024-09-16T00:19:42.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:42 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi169.mtrobk", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T00:19:42.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:42 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T00:19:42.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:42 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:19:42.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:42 smithi112 ceph-mon[27869]: Reconfiguring daemon mgr.smithi169.mtrobk on smithi169 2024-09-16T00:19:43.005 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:42 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-16T00:19:43.005 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:42 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:19:43.005 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:42 smithi169 ceph-mon[34653]: Deploying daemon osd.0 on smithi112 2024-09-16T00:19:43.005 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:42 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:43.005 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:42 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:43.005 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:42 smithi169 ceph-mon[34653]: Reconfiguring mgr.smithi169.mtrobk (monmap changed)... 2024-09-16T00:19:43.005 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:42 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi169.mtrobk", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T00:19:43.005 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:42 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T00:19:43.005 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:42 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:19:43.006 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:42 smithi169 ceph-mon[34653]: Reconfiguring daemon mgr.smithi169.mtrobk on smithi169 2024-09-16T00:19:44.445 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:44 smithi169 ceph-mon[34653]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:44.446 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:44 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:44.446 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:44 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:44.446 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:44 smithi169 ceph-mon[34653]: Reconfiguring mon.smithi169 (monmap changed)... 2024-09-16T00:19:44.446 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:44 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T00:19:44.446 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:44 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T00:19:44.446 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:44 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:19:44.447 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:44 smithi169 ceph-mon[34653]: Reconfiguring daemon mon.smithi169 on smithi169 2024-09-16T00:19:44.519 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:44 smithi112 ceph-mon[27869]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:44.519 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:44 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:44.519 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:44 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:44.519 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:44 smithi112 ceph-mon[27869]: Reconfiguring mon.smithi169 (monmap changed)... 2024-09-16T00:19:44.519 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:44 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T00:19:44.519 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:44 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T00:19:44.520 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:44 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:19:44.520 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:44 smithi112 ceph-mon[27869]: Reconfiguring daemon mon.smithi169 on smithi169 2024-09-16T00:19:45.587 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:45 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:45.587 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:45 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:45.588 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:45 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-16T00:19:45.588 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:45 smithi112 ceph-mon[27869]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-16T00:19:45.588 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:45 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi112.front.sepia.ceph.com:9093"}]: dispatch 2024-09-16T00:19:45.588 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:45 smithi112 ceph-mon[27869]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi112.front.sepia.ceph.com:9093"}]: dispatch 2024-09-16T00:19:45.588 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:45 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:45.588 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:45 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-16T00:19:45.588 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:45 smithi112 ceph-mon[27869]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-16T00:19:45.588 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:45 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi112.front.sepia.ceph.com:3000"}]: dispatch 2024-09-16T00:19:45.588 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:45 smithi112 ceph-mon[27869]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi112.front.sepia.ceph.com:3000"}]: dispatch 2024-09-16T00:19:45.588 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:45 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:45.588 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:45 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-16T00:19:45.588 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:45 smithi112 ceph-mon[27869]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-16T00:19:45.588 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:45 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi112.front.sepia.ceph.com:9095"}]: dispatch 2024-09-16T00:19:45.588 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:45 smithi112 ceph-mon[27869]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi112.front.sepia.ceph.com:9095"}]: dispatch 2024-09-16T00:19:45.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:45 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:45.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:45 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:19:45.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:45 smithi112 ceph-mon[27869]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:45.751 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:45 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:45.751 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:45 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:45.752 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:45 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-16T00:19:45.752 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:45 smithi169 ceph-mon[34653]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-16T00:19:45.752 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:45 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi112.front.sepia.ceph.com:9093"}]: dispatch 2024-09-16T00:19:45.752 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:45 smithi169 ceph-mon[34653]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi112.front.sepia.ceph.com:9093"}]: dispatch 2024-09-16T00:19:45.752 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:45 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:45.752 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:45 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-16T00:19:45.752 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:45 smithi169 ceph-mon[34653]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-16T00:19:45.752 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:45 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi112.front.sepia.ceph.com:3000"}]: dispatch 2024-09-16T00:19:45.752 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:45 smithi169 ceph-mon[34653]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi112.front.sepia.ceph.com:3000"}]: dispatch 2024-09-16T00:19:45.752 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:45 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:45.752 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:45 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-16T00:19:45.752 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:45 smithi169 ceph-mon[34653]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-16T00:19:45.752 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:45 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi112.front.sepia.ceph.com:9095"}]: dispatch 2024-09-16T00:19:45.753 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:45 smithi169 ceph-mon[34653]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi112.front.sepia.ceph.com:9095"}]: dispatch 2024-09-16T00:19:45.753 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:45 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:45.753 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:45 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:19:45.753 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:45 smithi169 ceph-mon[34653]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:47.571 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:47 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:47.571 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:47 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:47.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:47 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:47.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:47 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:48.635 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:48 smithi112 ceph-mon[27869]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:48.635 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:48 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:48.635 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:48 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:48.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:48 smithi169 ceph-mon[34653]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:48.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:48 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:48.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:48 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:50.529 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:50 smithi112 ceph-mon[27869]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:50.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:50 smithi169 ceph-mon[34653]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:51.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:51 smithi112 ceph-mon[27869]: from='osd.0 [v2:172.21.15.112:6802/2840222594,v1:172.21.15.112:6803/2840222594]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-16T00:19:51.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:51 smithi169 ceph-mon[34653]: from='osd.0 [v2:172.21.15.112:6802/2840222594,v1:172.21.15.112:6803/2840222594]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-16T00:19:51.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:51.987+0000 7f1312ffd640 1 -- 172.21.15.112:0/2548008287 <== mgr.14208 v2:172.21.15.112:6800/2480414866 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f131410ef40 con 0x7f12fc06ecc0 2024-09-16T00:19:51.988 INFO:teuthology.orchestra.run.smithi112.stdout:Created osd(s) 0 on host 'smithi112' 2024-09-16T00:19:51.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:51.989+0000 7f131c37c640 1 -- 172.21.15.112:0/2548008287 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f12fc06ecc0 msgr2=0x7f12fc071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:51.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:51.989+0000 7f131c37c640 1 --2- 172.21.15.112:0/2548008287 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f12fc06ecc0 0x7f12fc071170 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1304001250 tx=0x7f1304009620 comp rx=0 tx=0).stop 2024-09-16T00:19:51.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:51.989+0000 7f131c37c640 1 -- 172.21.15.112:0/2548008287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13140fd830 msgr2=0x7f131410d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:51.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:51.989+0000 7f131c37c640 1 --2- 172.21.15.112:0/2548008287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13140fd830 0x7f131410d600 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1308003080 tx=0x7f1308009da0 comp rx=0 tx=0).stop 2024-09-16T00:19:51.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:51.989+0000 7f131c37c640 1 -- 172.21.15.112:0/2548008287 shutdown_connections 2024-09-16T00:19:51.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:51.989+0000 7f131c37c640 1 --2- 172.21.15.112:0/2548008287 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f12fc06ecc0 0x7f12fc071170 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:51.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:51.989+0000 7f131c37c640 1 --2- 172.21.15.112:0/2548008287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13140fd830 0x7f131410d600 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:51.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:51.989+0000 7f131c37c640 1 --2- 172.21.15.112:0/2548008287 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13140fc560 0x7f131410d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:51.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:51.989+0000 7f131c37c640 1 -- 172.21.15.112:0/2548008287 >> 172.21.15.112:0/2548008287 conn(0x7f13140f7d20 msgr2=0x7f13140f9480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:51.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:51.989+0000 7f131c37c640 1 -- 172.21.15.112:0/2548008287 shutdown_connections 2024-09-16T00:19:51.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:51.989+0000 7f131c37c640 1 -- 172.21.15.112:0/2548008287 wait complete. 2024-09-16T00:19:52.558 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:52 smithi112 ceph-mon[27869]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:52.559 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:52 smithi112 ceph-mon[27869]: from='osd.0 [v2:172.21.15.112:6802/2840222594,v1:172.21.15.112:6803/2840222594]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-16T00:19:52.559 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:52 smithi112 ceph-mon[27869]: osdmap e7: 1 total, 0 up, 1 in 2024-09-16T00:19:52.559 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:52 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:19:52.559 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:52 smithi112 ceph-mon[27869]: from='osd.0 [v2:172.21.15.112:6802/2840222594,v1:172.21.15.112:6803/2840222594]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-09-16T00:19:52.559 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:52 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:19:52.559 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:52 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:52.559 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:52 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:52.583 DEBUG:teuthology.orchestra.run.smithi112:osd.0> sudo journalctl -f -n 0 -u ceph-d346dd56-73c0-11ef-bceb-c7b262605968@osd.0.service 2024-09-16T00:19:52.593 INFO:tasks.cephadm:Deploying osd.1 on smithi112 with /dev/vg_nvme/lv_3... 2024-09-16T00:19:52.593 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-16T00:19:52.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:52 smithi169 ceph-mon[34653]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:52.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:52 smithi169 ceph-mon[34653]: from='osd.0 [v2:172.21.15.112:6802/2840222594,v1:172.21.15.112:6803/2840222594]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-16T00:19:52.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:52 smithi169 ceph-mon[34653]: osdmap e7: 1 total, 0 up, 1 in 2024-09-16T00:19:52.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:52 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:19:52.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:52 smithi169 ceph-mon[34653]: from='osd.0 [v2:172.21.15.112:6802/2840222594,v1:172.21.15.112:6803/2840222594]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-09-16T00:19:52.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:52 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:19:52.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:52 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:52.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:52 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:53.012 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:19:53.421 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:53 smithi112 ceph-mon[27869]: from='osd.0 [v2:172.21.15.112:6802/2840222594,v1:172.21.15.112:6803/2840222594]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-09-16T00:19:53.421 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:53 smithi112 ceph-mon[27869]: osdmap e8: 1 total, 0 up, 1 in 2024-09-16T00:19:53.421 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:53 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:19:53.421 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:53 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:19:53.421 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:53 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:53.421 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:53 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:53.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:53 smithi169 ceph-mon[34653]: from='osd.0 [v2:172.21.15.112:6802/2840222594,v1:172.21.15.112:6803/2840222594]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-09-16T00:19:53.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:53 smithi169 ceph-mon[34653]: osdmap e8: 1 total, 0 up, 1 in 2024-09-16T00:19:53.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:53 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:19:53.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:53 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:19:53.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:53 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:53.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:53 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:54.419 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:54 smithi112 ceph-mon[27869]: purged_snaps scrub starts 2024-09-16T00:19:54.420 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:54 smithi112 ceph-mon[27869]: purged_snaps scrub ok 2024-09-16T00:19:54.420 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:54 smithi112 ceph-mon[27869]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:54.420 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:54 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:19:54.420 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:54 smithi112 ceph-mon[27869]: osd.0 [v2:172.21.15.112:6802/2840222594,v1:172.21.15.112:6803/2840222594] boot 2024-09-16T00:19:54.420 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:54 smithi112 ceph-mon[27869]: osdmap e9: 1 total, 1 up, 1 in 2024-09-16T00:19:54.420 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:54 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:19:54.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:54 smithi169 ceph-mon[34653]: purged_snaps scrub starts 2024-09-16T00:19:54.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:54 smithi169 ceph-mon[34653]: purged_snaps scrub ok 2024-09-16T00:19:54.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:54 smithi169 ceph-mon[34653]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:19:54.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:54 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:19:54.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:54 smithi169 ceph-mon[34653]: osd.0 [v2:172.21.15.112:6802/2840222594,v1:172.21.15.112:6803/2840222594] boot 2024-09-16T00:19:54.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:54 smithi169 ceph-mon[34653]: osdmap e9: 1 total, 1 up, 1 in 2024-09-16T00:19:54.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:54 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:19:55.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:55 smithi169 ceph-mon[34653]: osdmap e10: 1 total, 1 up, 1 in 2024-09-16T00:19:55.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:55 smithi169 ceph-mon[34653]: pgmap v37: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:19:55.786 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:55 smithi112 ceph-mon[27869]: osdmap e10: 1 total, 1 up, 1 in 2024-09-16T00:19:55.786 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:55 smithi112 ceph-mon[27869]: pgmap v37: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:19:56.186 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:19:56.211 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi112:vg_nvme/lv_3 2024-09-16T00:19:56.768 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:19:57.833 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:57 smithi112 ceph-mon[27869]: pgmap v38: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:19:58.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:57 smithi169 ceph-mon[34653]: pgmap v38: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:19:59.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:59.051+0000 7fdf4c828640 1 -- 172.21.15.112:0/180958658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf480fb450 msgr2=0x7fdf480fb860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:59.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:59.051+0000 7fdf4c828640 1 --2- 172.21.15.112:0/180958658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf480fb450 0x7fdf480fb860 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdf380048d0 tx=0x7fdf38009da0 comp rx=0 tx=0).stop 2024-09-16T00:19:59.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:59.051+0000 7fdf4c828640 1 -- 172.21.15.112:0/180958658 shutdown_connections 2024-09-16T00:19:59.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:59.051+0000 7fdf4c828640 1 --2- 172.21.15.112:0/180958658 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf480fc720 0x7fdf480fcb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:59.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:59.051+0000 7fdf4c828640 1 --2- 172.21.15.112:0/180958658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf480fb450 0x7fdf480fb860 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:59.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:59.051+0000 7fdf4c828640 1 -- 172.21.15.112:0/180958658 >> 172.21.15.112:0/180958658 conn(0x7fdf480f6c10 msgr2=0x7fdf480f9040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:59.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:59.051+0000 7fdf4c828640 1 -- 172.21.15.112:0/180958658 shutdown_connections 2024-09-16T00:19:59.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:59.051+0000 7fdf4c828640 1 -- 172.21.15.112:0/180958658 wait complete. 2024-09-16T00:19:59.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:59.052+0000 7fdf4c828640 1 Processor -- start 2024-09-16T00:19:59.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:59.052+0000 7fdf4c828640 1 -- start start 2024-09-16T00:19:59.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:59.052+0000 7fdf4c828640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf480fb450 0x7fdf4810d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:59.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:59.052+0000 7fdf4c828640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf480fc720 0x7fdf4810d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:59.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:59.052+0000 7fdf4c828640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4810dc60 con 0x7fdf480fc720 2024-09-16T00:19:59.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:59.052+0000 7fdf4c828640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4810e480 con 0x7fdf480fb450 2024-09-16T00:19:59.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:59.052+0000 7fdf45d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf480fc720 0x7fdf4810d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:59.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:59.052+0000 7fdf45d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf480fc720 0x7fdf4810d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34102/0 (socket says 172.21.15.112:34102) 2024-09-16T00:19:59.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:59.052+0000 7fdf45d9b640 1 -- 172.21.15.112:0/1081464867 learned_addr learned my addr 172.21.15.112:0/1081464867 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:19:59.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:59.052+0000 7fdf4659c640 1 --2- 172.21.15.112:0/1081464867 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf480fb450 0x7fdf4810d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:59.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:59.052+0000 7fdf45d9b640 1 -- 172.21.15.112:0/1081464867 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf480fb450 msgr2=0x7fdf4810d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:59.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:59.052+0000 7fdf45d9b640 1 --2- 172.21.15.112:0/1081464867 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf480fb450 0x7fdf4810d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:59.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:59.052+0000 7fdf45d9b640 1 -- 172.21.15.112:0/1081464867 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf38005040 con 0x7fdf480fc720 2024-09-16T00:19:59.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:59.053+0000 7fdf45d9b640 1 --2- 172.21.15.112:0/1081464867 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf480fc720 0x7fdf4810d610 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fdf3000eb70 tx=0x7fdf3000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:59.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:59.053+0000 7fdf3effd640 1 -- 172.21.15.112:0/1081464867 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf30007c30 con 0x7fdf480fc720 2024-09-16T00:19:59.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:59.053+0000 7fdf3effd640 1 -- 172.21.15.112:0/1081464867 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fdf3000f040 con 0x7fdf480fc720 2024-09-16T00:19:59.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:59.053+0000 7fdf3effd640 1 -- 172.21.15.112:0/1081464867 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf3000ca60 con 0x7fdf480fc720 2024-09-16T00:19:59.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:59.053+0000 7fdf4c828640 1 -- 172.21.15.112:0/1081464867 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf4810e6b0 con 0x7fdf480fc720 2024-09-16T00:19:59.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:59.053+0000 7fdf4c828640 1 -- 172.21.15.112:0/1081464867 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf4810ec50 con 0x7fdf480fc720 2024-09-16T00:19:59.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:59.054+0000 7fdf4c828640 1 -- 172.21.15.112:0/1081464867 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf480fa8b0 con 0x7fdf480fc720 2024-09-16T00:19:59.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:59.054+0000 7fdf3effd640 1 -- 172.21.15.112:0/1081464867 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) v1 ==== 90813+0+0 (secure 0 0 0) 0x7fdf3000cbc0 con 0x7fdf480fc720 2024-09-16T00:19:59.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:59.055+0000 7fdf3effd640 1 --2- 172.21.15.112:0/1081464867 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fdf3406ecc0 0x7fdf34071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:59.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:59.055+0000 7fdf3effd640 1 -- 172.21.15.112:0/1081464867 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1978+0+0 (secure 0 0 0) 0x7fdf3008cec0 con 0x7fdf480fc720 2024-09-16T00:19:59.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:59.055+0000 7fdf4659c640 1 --2- 172.21.15.112:0/1081464867 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fdf3406ecc0 0x7fdf34071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:59.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:59.055+0000 7fdf4659c640 1 --2- 172.21.15.112:0/1081464867 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fdf3406ecc0 0x7fdf34071170 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fdf38004540 tx=0x7fdf38035040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:59.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:59.059+0000 7fdf3effd640 1 -- 172.21.15.112:0/1081464867 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fdf3005b050 con 0x7fdf480fc720 2024-09-16T00:19:59.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:19:59.234+0000 7fdf4c828640 1 -- 172.21.15.112:0/1081464867 --> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fdf48103fe0 con 0x7fdf3406ecc0 2024-09-16T00:19:59.413 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:59 smithi112 ceph-mon[27869]: Detected new or changed devices on smithi112 2024-09-16T00:19:59.413 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:59 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:59.413 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:59 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:59.413 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:59 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:19:59.413 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:59 smithi112 ceph-mon[27869]: Adjusting osd_memory_target on smithi112 to 12864M 2024-09-16T00:19:59.413 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:59 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:59.413 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:59 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:19:59.413 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:59 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:19:59.413 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:59 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:59.414 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:19:59 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:19:59.447 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:59 smithi169 ceph-mon[34653]: Detected new or changed devices on smithi112 2024-09-16T00:19:59.447 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:59 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:59.447 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:59 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:59.448 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:59 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:19:59.448 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:59 smithi169 ceph-mon[34653]: Adjusting osd_memory_target on smithi112 to 12864M 2024-09-16T00:19:59.448 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:59 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:59.448 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:59 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:19:59.448 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:59 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:19:59.448 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:59 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:19:59.448 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:19:59 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:20:00.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:00 smithi169 ceph-mon[34653]: pgmap v39: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:20:00.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:00 smithi169 ceph-mon[34653]: from='client.14260 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:20:00.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:00 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:20:00.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:00 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:20:00.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:00 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:20:00.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:00 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:20:00.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:00 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:20:00.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:00 smithi169 ceph-mon[34653]: overall HEALTH_OK 2024-09-16T00:20:00.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:00 smithi112 ceph-mon[27869]: pgmap v39: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:20:00.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:00 smithi112 ceph-mon[27869]: from='client.14260 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:20:00.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:00 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:20:00.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:00 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:20:00.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:00 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:20:00.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:00 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:20:00.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:00 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:20:00.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:00 smithi112 ceph-mon[27869]: overall HEALTH_OK 2024-09-16T00:20:02.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:02 smithi112 ceph-mon[27869]: pgmap v40: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:20:02.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:02 smithi169 ceph-mon[34653]: pgmap v40: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:20:03.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:03 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/1856717479' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6fee7bb7-5935-4218-b0b5-ecd103121196"}]: dispatch 2024-09-16T00:20:03.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:03 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/1856717479' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6fee7bb7-5935-4218-b0b5-ecd103121196"}]': finished 2024-09-16T00:20:03.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:03 smithi112 ceph-mon[27869]: osdmap e11: 2 total, 1 up, 2 in 2024-09-16T00:20:03.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:03 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:20:03.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:03 smithi169 ceph-mon[34653]: from='client.? 172.21.15.112:0/1856717479' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6fee7bb7-5935-4218-b0b5-ecd103121196"}]: dispatch 2024-09-16T00:20:03.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:03 smithi169 ceph-mon[34653]: from='client.? 172.21.15.112:0/1856717479' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6fee7bb7-5935-4218-b0b5-ecd103121196"}]': finished 2024-09-16T00:20:03.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:03 smithi169 ceph-mon[34653]: osdmap e11: 2 total, 1 up, 2 in 2024-09-16T00:20:03.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:03 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:20:04.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:04 smithi169 ceph-mon[34653]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:20:04.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:04 smithi169 ceph-mon[34653]: from='client.? 172.21.15.112:0/1789455124' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:20:04.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:04 smithi112 ceph-mon[27869]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:20:04.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:04 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/1789455124' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:20:05.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:05 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:20:05.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:05 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:20:05.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:05 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:20:05.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:05 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:20:05.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:05 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:20:05.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:05 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:20:05.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:05 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:20:05.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:05 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:20:05.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:05 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:20:05.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:05 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:20:06.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:06 smithi112 ceph-mon[27869]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:20:06.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:06 smithi169 ceph-mon[34653]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:20:07.570 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:07 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:20:07.573 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:07 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:20:08.593 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:08 smithi112 ceph-mon[27869]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:20:08.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:08 smithi169 ceph-mon[34653]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:20:10.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:10 smithi112 ceph-mon[27869]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:20:10.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:10 smithi169 ceph-mon[34653]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:20:11.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:11 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-16T00:20:11.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:11 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:20:11.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:11 smithi112 ceph-mon[27869]: Deploying daemon osd.1 on smithi112 2024-09-16T00:20:11.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:11 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-16T00:20:11.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:11 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:20:11.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:11 smithi169 ceph-mon[34653]: Deploying daemon osd.1 on smithi112 2024-09-16T00:20:12.358 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:12 smithi112 ceph-mon[27869]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:20:12.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:12 smithi169 ceph-mon[34653]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:20:14.522 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:14 smithi112 ceph-mon[27869]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:20:14.579 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:14 smithi169 ceph-mon[34653]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:20:16.357 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:16 smithi112 ceph-mon[27869]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:20:16.357 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:16 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:20:16.357 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:16 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:20:16.357 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:16 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:20:16.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:16 smithi169 ceph-mon[34653]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:20:16.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:16 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:20:16.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:16 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:20:16.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:16 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:20:18.318 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:18 smithi112 ceph-mon[27869]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:20:18.318 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:18 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:20:18.318 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:18 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:20:18.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:18 smithi169 ceph-mon[34653]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:20:18.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:18 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:20:18.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:18 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:20:18.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:20:18.951+0000 7fdf3effd640 1 -- 172.21.15.112:0/1081464867 <== mgr.14208 v2:172.21.15.112:6800/2480414866 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fdf48103fe0 con 0x7fdf3406ecc0 2024-09-16T00:20:18.953 INFO:teuthology.orchestra.run.smithi112.stdout:Created osd(s) 1 on host 'smithi112' 2024-09-16T00:20:18.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:20:18.954+0000 7fdf4c828640 1 -- 172.21.15.112:0/1081464867 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fdf3406ecc0 msgr2=0x7fdf34071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:18.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:20:18.954+0000 7fdf4c828640 1 --2- 172.21.15.112:0/1081464867 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fdf3406ecc0 0x7fdf34071170 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fdf38004540 tx=0x7fdf38035040 comp rx=0 tx=0).stop 2024-09-16T00:20:18.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:20:18.954+0000 7fdf4c828640 1 -- 172.21.15.112:0/1081464867 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf480fc720 msgr2=0x7fdf4810d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:18.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:20:18.954+0000 7fdf4c828640 1 --2- 172.21.15.112:0/1081464867 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf480fc720 0x7fdf4810d610 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fdf3000eb70 tx=0x7fdf3000b040 comp rx=0 tx=0).stop 2024-09-16T00:20:18.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:20:18.954+0000 7fdf4c828640 1 -- 172.21.15.112:0/1081464867 shutdown_connections 2024-09-16T00:20:18.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:20:18.954+0000 7fdf4c828640 1 --2- 172.21.15.112:0/1081464867 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fdf3406ecc0 0x7fdf34071170 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:18.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:20:18.954+0000 7fdf4c828640 1 --2- 172.21.15.112:0/1081464867 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf480fc720 0x7fdf4810d610 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:18.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:20:18.954+0000 7fdf4c828640 1 --2- 172.21.15.112:0/1081464867 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf480fb450 0x7fdf4810d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:18.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:20:18.954+0000 7fdf4c828640 1 -- 172.21.15.112:0/1081464867 >> 172.21.15.112:0/1081464867 conn(0x7fdf480f6c10 msgr2=0x7fdf48066140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:18.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:20:18.954+0000 7fdf4c828640 1 -- 172.21.15.112:0/1081464867 shutdown_connections 2024-09-16T00:20:18.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:20:18.954+0000 7fdf4c828640 1 -- 172.21.15.112:0/1081464867 wait complete. 2024-09-16T00:20:19.897 DEBUG:teuthology.orchestra.run.smithi112:osd.1> sudo journalctl -f -n 0 -u ceph-d346dd56-73c0-11ef-bceb-c7b262605968@osd.1.service 2024-09-16T00:20:19.900 INFO:tasks.cephadm:Deploying osd.2 on smithi112 with /dev/vg_nvme/lv_2... 2024-09-16T00:20:19.900 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-16T00:20:20.176 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:19 smithi112 ceph-mon[27869]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:20:20.176 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:19 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:20:20.176 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:19 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:20:20.176 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:19 smithi112 ceph-mon[27869]: from='osd.1 [v2:172.21.15.112:6810/946524214,v1:172.21.15.112:6811/946524214]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-16T00:20:20.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:19 smithi169 ceph-mon[34653]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:20:20.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:19 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:20:20.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:19 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:20:20.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:19 smithi169 ceph-mon[34653]: from='osd.1 [v2:172.21.15.112:6810/946524214,v1:172.21.15.112:6811/946524214]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-16T00:20:20.664 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:20:21.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:21 smithi112 ceph-mon[27869]: from='osd.1 [v2:172.21.15.112:6810/946524214,v1:172.21.15.112:6811/946524214]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-16T00:20:21.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:21 smithi112 ceph-mon[27869]: osdmap e12: 2 total, 1 up, 2 in 2024-09-16T00:20:21.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:21 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:20:21.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:21 smithi112 ceph-mon[27869]: from='osd.1 [v2:172.21.15.112:6810/946524214,v1:172.21.15.112:6811/946524214]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-09-16T00:20:21.340 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:20:21 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-1[53484]: 2024-09-16T00:20:21.088+0000 7fb1d0869640 -1 osd.1 0 waiting for initial osdmap 2024-09-16T00:20:21.340 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:20:21 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-1[53484]: 2024-09-16T00:20:21.092+0000 7fb1ccea3640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T00:20:21.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:21 smithi169 ceph-mon[34653]: from='osd.1 [v2:172.21.15.112:6810/946524214,v1:172.21.15.112:6811/946524214]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-16T00:20:21.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:21 smithi169 ceph-mon[34653]: osdmap e12: 2 total, 1 up, 2 in 2024-09-16T00:20:21.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:21 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:20:21.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:21 smithi169 ceph-mon[34653]: from='osd.1 [v2:172.21.15.112:6810/946524214,v1:172.21.15.112:6811/946524214]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-09-16T00:20:22.312 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:22 smithi112 ceph-mon[27869]: pgmap v52: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:20:22.313 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:22 smithi112 ceph-mon[27869]: from='osd.1 [v2:172.21.15.112:6810/946524214,v1:172.21.15.112:6811/946524214]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-09-16T00:20:22.313 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:22 smithi112 ceph-mon[27869]: osdmap e13: 2 total, 1 up, 2 in 2024-09-16T00:20:22.313 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:22 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:20:22.313 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:22 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:20:22.313 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:22 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:20:22.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:22 smithi169 ceph-mon[34653]: pgmap v52: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:20:22.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:22 smithi169 ceph-mon[34653]: from='osd.1 [v2:172.21.15.112:6810/946524214,v1:172.21.15.112:6811/946524214]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-09-16T00:20:22.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:22 smithi169 ceph-mon[34653]: osdmap e13: 2 total, 1 up, 2 in 2024-09-16T00:20:22.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:22 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:20:22.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:22 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:20:22.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:22 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:20:23.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:23 smithi169 ceph-mon[34653]: purged_snaps scrub starts 2024-09-16T00:20:23.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:23 smithi169 ceph-mon[34653]: purged_snaps scrub ok 2024-09-16T00:20:23.528 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:23 smithi169 ceph-mon[34653]: osd.1 [v2:172.21.15.112:6810/946524214,v1:172.21.15.112:6811/946524214] boot 2024-09-16T00:20:23.528 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:23 smithi169 ceph-mon[34653]: osdmap e14: 2 total, 2 up, 2 in 2024-09-16T00:20:23.528 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:23 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:20:23.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:23 smithi112 ceph-mon[27869]: purged_snaps scrub starts 2024-09-16T00:20:23.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:23 smithi112 ceph-mon[27869]: purged_snaps scrub ok 2024-09-16T00:20:23.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:23 smithi112 ceph-mon[27869]: osd.1 [v2:172.21.15.112:6810/946524214,v1:172.21.15.112:6811/946524214] boot 2024-09-16T00:20:23.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:23 smithi112 ceph-mon[27869]: osdmap e14: 2 total, 2 up, 2 in 2024-09-16T00:20:23.591 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:23 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:20:24.412 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:24 smithi112 ceph-mon[27869]: pgmap v55: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-16T00:20:24.412 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:24 smithi112 ceph-mon[27869]: osdmap e15: 2 total, 2 up, 2 in 2024-09-16T00:20:24.418 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:20:24.438 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi112:vg_nvme/lv_2 2024-09-16T00:20:24.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:24 smithi169 ceph-mon[34653]: pgmap v55: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-16T00:20:24.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:24 smithi169 ceph-mon[34653]: osdmap e15: 2 total, 2 up, 2 in 2024-09-16T00:20:24.724 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:20:25.810 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:25 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:20:25.810 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:25 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:20:25.810 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:25 smithi112 ceph-mon[27869]: pgmap v57: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-16T00:20:25.891 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:25 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:20:25.891 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:25 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:20:25.891 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:25 smithi169 ceph-mon[34653]: pgmap v57: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-16T00:20:27.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:20:27.009+0000 7fc62c2af640 1 -- 172.21.15.112:0/1911149743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6240fc720 msgr2=0x7fc6240fcb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:27.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:20:27.009+0000 7fc62c2af640 1 --2- 172.21.15.112:0/1911149743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6240fc720 0x7fc6240fcb90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc6180048d0 tx=0x7fc618009da0 comp rx=0 tx=0).stop 2024-09-16T00:20:27.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:20:27.009+0000 7fc62c2af640 1 -- 172.21.15.112:0/1911149743 shutdown_connections 2024-09-16T00:20:27.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:20:27.009+0000 7fc62c2af640 1 --2- 172.21.15.112:0/1911149743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6240fc720 0x7fc6240fcb90 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:27.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:20:27.009+0000 7fc62c2af640 1 --2- 172.21.15.112:0/1911149743 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc6240fb450 0x7fc6240fb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:27.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:20:27.009+0000 7fc62c2af640 1 -- 172.21.15.112:0/1911149743 >> 172.21.15.112:0/1911149743 conn(0x7fc6240f6c10 msgr2=0x7fc6240f9040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:27.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:20:27.009+0000 7fc62c2af640 1 -- 172.21.15.112:0/1911149743 shutdown_connections 2024-09-16T00:20:27.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:20:27.009+0000 7fc62c2af640 1 -- 172.21.15.112:0/1911149743 wait complete. 2024-09-16T00:20:27.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:20:27.010+0000 7fc62c2af640 1 Processor -- start 2024-09-16T00:20:27.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:20:27.010+0000 7fc62c2af640 1 -- start start 2024-09-16T00:20:27.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:20:27.010+0000 7fc62c2af640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6240fb450 0x7fc6240ff570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:27.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:20:27.010+0000 7fc62c2af640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc6240fc720 0x7fc6240fda60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:27.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:20:27.010+0000 7fc62c2af640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6240fdff0 con 0x7fc6240fb450 2024-09-16T00:20:27.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:20:27.010+0000 7fc62c2af640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6240fe130 con 0x7fc6240fc720 2024-09-16T00:20:27.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:20:27.010+0000 7fc62a04b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6240fb450 0x7fc6240ff570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:27.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:20:27.010+0000 7fc62a04b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6240fb450 0x7fc6240ff570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47962/0 (socket says 172.21.15.112:47962) 2024-09-16T00:20:27.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:20:27.010+0000 7fc62a04b640 1 -- 172.21.15.112:0/2122551611 learned_addr learned my addr 172.21.15.112:0/2122551611 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:20:27.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:20:27.010+0000 7fc62984a640 1 --2- 172.21.15.112:0/2122551611 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc6240fc720 0x7fc6240fda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:27.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:20:27.011+0000 7fc62a04b640 1 -- 172.21.15.112:0/2122551611 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc6240fc720 msgr2=0x7fc6240fda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:27.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:20:27.011+0000 7fc62a04b640 1 --2- 172.21.15.112:0/2122551611 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc6240fc720 0x7fc6240fda60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:27.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:20:27.011+0000 7fc62a04b640 1 -- 172.21.15.112:0/2122551611 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc618005040 con 0x7fc6240fb450 2024-09-16T00:20:27.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:20:27.011+0000 7fc62984a640 1 --2- 172.21.15.112:0/2122551611 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc6240fc720 0x7fc6240fda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:20:27.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:20:27.011+0000 7fc62a04b640 1 --2- 172.21.15.112:0/2122551611 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6240fb450 0x7fc6240ff570 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc61c002a50 tx=0x7fc61c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:27.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:20:27.011+0000 7fc622ffd640 1 -- 172.21.15.112:0/2122551611 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc61c007c30 con 0x7fc6240fb450 2024-09-16T00:20:27.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:20:27.011+0000 7fc622ffd640 1 -- 172.21.15.112:0/2122551611 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fc61c00f040 con 0x7fc6240fb450 2024-09-16T00:20:27.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:20:27.011+0000 7fc62c2af640 1 -- 172.21.15.112:0/2122551611 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6240fe410 con 0x7fc6240fb450 2024-09-16T00:20:27.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:20:27.011+0000 7fc622ffd640 1 -- 172.21.15.112:0/2122551611 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc61c00da90 con 0x7fc6240fb450 2024-09-16T00:20:27.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:20:27.011+0000 7fc62c2af640 1 -- 172.21.15.112:0/2122551611 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc624115150 con 0x7fc6240fb450 2024-09-16T00:20:27.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:20:27.012+0000 7fc62c2af640 1 -- 172.21.15.112:0/2122551611 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6240fa8b0 con 0x7fc6240fb450 2024-09-16T00:20:27.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:20:27.012+0000 7fc622ffd640 1 -- 172.21.15.112:0/2122551611 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) v1 ==== 90813+0+0 (secure 0 0 0) 0x7fc61c00dc30 con 0x7fc6240fb450 2024-09-16T00:20:27.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:20:27.013+0000 7fc622ffd640 1 --2- 172.21.15.112:0/2122551611 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fc60c06ecc0 0x7fc60c071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:27.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:20:27.013+0000 7fc622ffd640 1 -- 172.21.15.112:0/2122551611 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2410+0+0 (secure 0 0 0) 0x7fc61c08ccc0 con 0x7fc6240fb450 2024-09-16T00:20:27.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:20:27.013+0000 7fc62984a640 1 --2- 172.21.15.112:0/2122551611 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fc60c06ecc0 0x7fc60c071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:27.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:20:27.013+0000 7fc62984a640 1 --2- 172.21.15.112:0/2122551611 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fc60c06ecc0 0x7fc60c071170 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc618007ac0 tx=0x7fc618035040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:27.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:20:27.017+0000 7fc622ffd640 1 -- 172.21.15.112:0/2122551611 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fc61c04d2a0 con 0x7fc6240fb450 2024-09-16T00:20:27.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:20:27.185+0000 7fc62c2af640 1 -- 172.21.15.112:0/2122551611 --> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fc624103ff0 con 0x7fc60c06ecc0 2024-09-16T00:20:28.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:27 smithi169 ceph-mon[34653]: pgmap v58: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-16T00:20:28.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:27 smithi169 ceph-mon[34653]: from='client.14278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:20:28.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:27 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:20:28.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:27 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:20:28.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:27 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:20:28.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:27 smithi112 ceph-mon[27869]: pgmap v58: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-16T00:20:28.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:27 smithi112 ceph-mon[27869]: from='client.14278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:20:28.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:27 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:20:28.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:27 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:20:28.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:27 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:20:29.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:29 smithi112 ceph-mon[27869]: pgmap v59: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-16T00:20:30.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:29 smithi169 ceph-mon[34653]: pgmap v59: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-16T00:20:31.268 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:30 smithi112 ceph-mon[27869]: Detected new or changed devices on smithi112 2024-09-16T00:20:31.268 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:30 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:20:31.268 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:30 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:20:31.268 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:30 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:20:31.268 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:30 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:20:31.269 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:30 smithi112 ceph-mon[27869]: Adjusting osd_memory_target on smithi112 to 6432M 2024-09-16T00:20:31.269 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:30 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:20:31.269 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:30 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:20:31.269 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:30 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:20:31.269 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:30 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:20:31.269 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:30 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:20:31.269 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:30 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/3625253356' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "acf3f3a5-2941-49ed-a653-728564ba08df"}]: dispatch 2024-09-16T00:20:31.269 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:30 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/3625253356' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "acf3f3a5-2941-49ed-a653-728564ba08df"}]': finished 2024-09-16T00:20:31.269 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:30 smithi112 ceph-mon[27869]: osdmap e16: 3 total, 2 up, 3 in 2024-09-16T00:20:31.269 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:30 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:20:31.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:30 smithi169 ceph-mon[34653]: Detected new or changed devices on smithi112 2024-09-16T00:20:31.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:30 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:20:31.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:30 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:20:31.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:30 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:20:31.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:30 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:20:31.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:30 smithi169 ceph-mon[34653]: Adjusting osd_memory_target on smithi112 to 6432M 2024-09-16T00:20:31.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:30 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:20:31.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:30 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:20:31.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:30 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:20:31.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:30 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:20:31.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:30 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:20:31.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:30 smithi169 ceph-mon[34653]: from='client.? 172.21.15.112:0/3625253356' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "acf3f3a5-2941-49ed-a653-728564ba08df"}]: dispatch 2024-09-16T00:20:31.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:30 smithi169 ceph-mon[34653]: from='client.? 172.21.15.112:0/3625253356' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "acf3f3a5-2941-49ed-a653-728564ba08df"}]': finished 2024-09-16T00:20:31.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:30 smithi169 ceph-mon[34653]: osdmap e16: 3 total, 2 up, 3 in 2024-09-16T00:20:31.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:30 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:20:32.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:32 smithi169 ceph-mon[34653]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-16T00:20:32.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:32 smithi169 ceph-mon[34653]: from='client.? 172.21.15.112:0/619314499' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:20:32.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:32 smithi112 ceph-mon[27869]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-16T00:20:32.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:32 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/619314499' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:20:34.244 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:34 smithi112 ceph-mon[27869]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-16T00:20:34.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:34 smithi169 ceph-mon[34653]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-16T00:20:35.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:35 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:20:35.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:35 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:20:35.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:35 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:20:35.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:35 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:20:35.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:35 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:20:35.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:35 smithi112 ceph-mon[27869]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-16T00:20:36.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:35 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:20:36.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:35 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:20:36.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:35 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:20:36.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:35 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:20:36.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:35 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:20:36.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:35 smithi169 ceph-mon[34653]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-16T00:20:36.863 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:36 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:20:37.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:36 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:20:38.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:37 smithi169 ceph-mon[34653]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-16T00:20:38.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:37 smithi112 ceph-mon[27869]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-16T00:20:38.811 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:38 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-16T00:20:38.811 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:38 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:20:38.811 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:38 smithi112 ceph-mon[27869]: Deploying daemon osd.2 on smithi112 2024-09-16T00:20:39.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:38 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-16T00:20:39.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:38 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:20:39.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:38 smithi169 ceph-mon[34653]: Deploying daemon osd.2 on smithi112 2024-09-16T00:20:40.017 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:39 smithi112 ceph-mon[27869]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-16T00:20:40.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:39 smithi169 ceph-mon[34653]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-16T00:20:41.886 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:41 smithi112 ceph-mon[27869]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-16T00:20:42.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:41 smithi169 ceph-mon[34653]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-16T00:20:43.827 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:43 smithi112 ceph-mon[27869]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-16T00:20:43.827 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:43 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:20:43.827 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:43 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:20:43.827 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:43 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:20:44.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:43 smithi169 ceph-mon[34653]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-16T00:20:44.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:43 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:20:44.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:43 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:20:44.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:43 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:20:46.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:45 smithi169 ceph-mon[34653]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-16T00:20:46.065 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:45 smithi112 ceph-mon[27869]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-16T00:20:46.876 INFO:teuthology.orchestra.run.smithi112.stdout:Created osd(s) 2 on host 'smithi112' 2024-09-16T00:20:46.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:20:46.876+0000 7fc622ffd640 1 -- 172.21.15.112:0/2122551611 <== mgr.14208 v2:172.21.15.112:6800/2480414866 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fc624103ff0 con 0x7fc60c06ecc0 2024-09-16T00:20:46.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:20:46.878+0000 7fc62c2af640 1 -- 172.21.15.112:0/2122551611 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fc60c06ecc0 msgr2=0x7fc60c071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:46.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:20:46.878+0000 7fc62c2af640 1 --2- 172.21.15.112:0/2122551611 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fc60c06ecc0 0x7fc60c071170 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc618007ac0 tx=0x7fc618035040 comp rx=0 tx=0).stop 2024-09-16T00:20:46.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:20:46.878+0000 7fc62c2af640 1 -- 172.21.15.112:0/2122551611 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6240fb450 msgr2=0x7fc6240ff570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:46.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:20:46.878+0000 7fc62c2af640 1 --2- 172.21.15.112:0/2122551611 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6240fb450 0x7fc6240ff570 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc61c002a50 tx=0x7fc61c004600 comp rx=0 tx=0).stop 2024-09-16T00:20:46.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:20:46.878+0000 7fc62c2af640 1 -- 172.21.15.112:0/2122551611 shutdown_connections 2024-09-16T00:20:46.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:20:46.878+0000 7fc62c2af640 1 --2- 172.21.15.112:0/2122551611 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fc60c06ecc0 0x7fc60c071170 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:46.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:20:46.878+0000 7fc62c2af640 1 --2- 172.21.15.112:0/2122551611 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc6240fc720 0x7fc6240fda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:46.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:20:46.878+0000 7fc62c2af640 1 --2- 172.21.15.112:0/2122551611 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6240fb450 0x7fc6240ff570 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:46.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:20:46.878+0000 7fc62c2af640 1 -- 172.21.15.112:0/2122551611 >> 172.21.15.112:0/2122551611 conn(0x7fc6240f6c10 msgr2=0x7fc624066140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:46.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:20:46.878+0000 7fc62c2af640 1 -- 172.21.15.112:0/2122551611 shutdown_connections 2024-09-16T00:20:46.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:20:46.878+0000 7fc62c2af640 1 -- 172.21.15.112:0/2122551611 wait complete. 2024-09-16T00:20:47.572 DEBUG:teuthology.orchestra.run.smithi112:osd.2> sudo journalctl -f -n 0 -u ceph-d346dd56-73c0-11ef-bceb-c7b262605968@osd.2.service 2024-09-16T00:20:47.575 INFO:tasks.cephadm:Deploying osd.3 on smithi169 with /dev/vg_nvme/lv_4... 2024-09-16T00:20:47.575 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-16T00:20:47.817 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi169/config 2024-09-16T00:20:48.138 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:47 smithi169 ceph-mon[34653]: pgmap v69: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-16T00:20:48.138 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:47 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:20:48.138 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:47 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:20:48.138 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:47 smithi169 ceph-mon[34653]: from='osd.2 [v2:172.21.15.112:6818/3837096766,v1:172.21.15.112:6819/3837096766]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T00:20:48.152 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:47 smithi112 ceph-mon[27869]: pgmap v69: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-16T00:20:48.152 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:47 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:20:48.152 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:47 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:20:48.152 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:47 smithi112 ceph-mon[27869]: from='osd.2 [v2:172.21.15.112:6818/3837096766,v1:172.21.15.112:6819/3837096766]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T00:20:49.077 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:48 smithi169 ceph-mon[34653]: from='osd.2 [v2:172.21.15.112:6818/3837096766,v1:172.21.15.112:6819/3837096766]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-16T00:20:49.077 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:48 smithi169 ceph-mon[34653]: osdmap e17: 3 total, 2 up, 3 in 2024-09-16T00:20:49.077 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:48 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:20:49.078 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:48 smithi169 ceph-mon[34653]: from='osd.2 [v2:172.21.15.112:6818/3837096766,v1:172.21.15.112:6819/3837096766]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-09-16T00:20:49.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:48 smithi112 ceph-mon[27869]: from='osd.2 [v2:172.21.15.112:6818/3837096766,v1:172.21.15.112:6819/3837096766]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-16T00:20:49.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:48 smithi112 ceph-mon[27869]: osdmap e17: 3 total, 2 up, 3 in 2024-09-16T00:20:49.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:48 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:20:49.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:48 smithi112 ceph-mon[27869]: from='osd.2 [v2:172.21.15.112:6818/3837096766,v1:172.21.15.112:6819/3837096766]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-09-16T00:20:49.340 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:20:48 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-2[60918]: 2024-09-16T00:20:48.882+0000 7fd50008d640 -1 osd.2 0 waiting for initial osdmap 2024-09-16T00:20:49.340 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:20:48 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-2[60918]: 2024-09-16T00:20:48.890+0000 7fd4fb6a3640 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T00:20:50.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:49 smithi112 ceph-mon[27869]: pgmap v71: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-16T00:20:50.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:49 smithi112 ceph-mon[27869]: from='osd.2 [v2:172.21.15.112:6818/3837096766,v1:172.21.15.112:6819/3837096766]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-09-16T00:20:50.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:49 smithi112 ceph-mon[27869]: osdmap e18: 3 total, 2 up, 3 in 2024-09-16T00:20:50.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:49 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:20:50.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:49 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:20:50.137 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:20:50.154 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:49 smithi169 ceph-mon[34653]: pgmap v71: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-16T00:20:50.155 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:49 smithi169 ceph-mon[34653]: from='osd.2 [v2:172.21.15.112:6818/3837096766,v1:172.21.15.112:6819/3837096766]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-09-16T00:20:50.155 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:49 smithi169 ceph-mon[34653]: osdmap e18: 3 total, 2 up, 3 in 2024-09-16T00:20:50.155 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:49 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:20:50.155 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:49 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:20:50.156 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi169:vg_nvme/lv_4 2024-09-16T00:20:50.398 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi169/config 2024-09-16T00:20:51.079 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:50 smithi169 ceph-mon[34653]: purged_snaps scrub starts 2024-09-16T00:20:51.079 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:50 smithi169 ceph-mon[34653]: purged_snaps scrub ok 2024-09-16T00:20:51.079 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:50 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:20:51.079 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:50 smithi169 ceph-mon[34653]: osd.2 [v2:172.21.15.112:6818/3837096766,v1:172.21.15.112:6819/3837096766] boot 2024-09-16T00:20:51.079 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:50 smithi169 ceph-mon[34653]: osdmap e19: 3 total, 3 up, 3 in 2024-09-16T00:20:51.080 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:50 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:20:51.080 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:50 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:20:51.080 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:50 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:20:51.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:50 smithi112 ceph-mon[27869]: purged_snaps scrub starts 2024-09-16T00:20:51.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:50 smithi112 ceph-mon[27869]: purged_snaps scrub ok 2024-09-16T00:20:51.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:50 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:20:51.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:50 smithi112 ceph-mon[27869]: osd.2 [v2:172.21.15.112:6818/3837096766,v1:172.21.15.112:6819/3837096766] boot 2024-09-16T00:20:51.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:50 smithi112 ceph-mon[27869]: osdmap e19: 3 total, 3 up, 3 in 2024-09-16T00:20:51.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:50 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:20:51.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:50 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:20:51.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:50 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:20:51.864 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:20:51.863+0000 7fce3ed62640 1 -- 172.21.15.169:0/317853282 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce380fb2c0 msgr2=0x7fce38101af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:51.865 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:20:51.863+0000 7fce3ed62640 1 --2- 172.21.15.169:0/317853282 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce380fb2c0 0x7fce38101af0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fce2c0048d0 tx=0x7fce2c009da0 comp rx=0 tx=0).stop 2024-09-16T00:20:51.865 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:20:51.863+0000 7fce3ed62640 1 -- 172.21.15.169:0/317853282 shutdown_connections 2024-09-16T00:20:51.865 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:20:51.863+0000 7fce3ed62640 1 --2- 172.21.15.169:0/317853282 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce380fb2c0 0x7fce38101af0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:51.865 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:20:51.863+0000 7fce3ed62640 1 --2- 172.21.15.169:0/317853282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce380fa950 0x7fce380fad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:51.865 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:20:51.863+0000 7fce3ed62640 1 -- 172.21.15.169:0/317853282 >> 172.21.15.169:0/317853282 conn(0x7fce380f6390 msgr2=0x7fce380f87a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:51.865 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:20:51.863+0000 7fce3ed62640 1 -- 172.21.15.169:0/317853282 shutdown_connections 2024-09-16T00:20:51.865 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:20:51.863+0000 7fce3ed62640 1 -- 172.21.15.169:0/317853282 wait complete. 2024-09-16T00:20:51.865 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:20:51.863+0000 7fce3ed62640 1 Processor -- start 2024-09-16T00:20:51.865 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:20:51.863+0000 7fce3ed62640 1 -- start start 2024-09-16T00:20:51.865 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:20:51.864+0000 7fce3ed62640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce380fa950 0x7fce3810d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:51.865 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:20:51.864+0000 7fce3ed62640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce380fb2c0 0x7fce3810d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:51.865 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:20:51.864+0000 7fce3ed62640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce3810dc10 con 0x7fce380fb2c0 2024-09-16T00:20:51.866 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:20:51.864+0000 7fce3ed62640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce3810e430 con 0x7fce380fa950 2024-09-16T00:20:51.866 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:20:51.864+0000 7fce3cafe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce380fa950 0x7fce3810d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:51.866 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:20:51.864+0000 7fce3cafe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce380fa950 0x7fce3810d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:38962/0 (socket says 172.21.15.169:38962) 2024-09-16T00:20:51.866 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:20:51.864+0000 7fce3cafe640 1 -- 172.21.15.169:0/1623306204 learned_addr learned my addr 172.21.15.169:0/1623306204 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-09-16T00:20:51.866 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:20:51.864+0000 7fce37fff640 1 --2- 172.21.15.169:0/1623306204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce380fb2c0 0x7fce3810d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:51.866 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:20:51.864+0000 7fce3cafe640 1 -- 172.21.15.169:0/1623306204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce380fb2c0 msgr2=0x7fce3810d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:51.866 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:20:51.864+0000 7fce3cafe640 1 --2- 172.21.15.169:0/1623306204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce380fb2c0 0x7fce3810d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:51.866 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:20:51.864+0000 7fce3cafe640 1 -- 172.21.15.169:0/1623306204 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce2c005040 con 0x7fce380fa950 2024-09-16T00:20:51.866 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:20:51.864+0000 7fce3cafe640 1 --2- 172.21.15.169:0/1623306204 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce380fa950 0x7fce3810d080 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fce28002a80 tx=0x7fce28004570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:51.866 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:20:51.864+0000 7fce357fa640 1 -- 172.21.15.169:0/1623306204 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce28007c30 con 0x7fce380fa950 2024-09-16T00:20:51.866 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:20:51.864+0000 7fce357fa640 1 -- 172.21.15.169:0/1623306204 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fce2800f040 con 0x7fce380fa950 2024-09-16T00:20:51.866 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:20:51.864+0000 7fce3ed62640 1 -- 172.21.15.169:0/1623306204 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce3810e6c0 con 0x7fce380fa950 2024-09-16T00:20:51.866 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:20:51.864+0000 7fce357fa640 1 -- 172.21.15.169:0/1623306204 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce2800dad0 con 0x7fce380fa950 2024-09-16T00:20:51.867 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:20:51.864+0000 7fce3ed62640 1 -- 172.21.15.169:0/1623306204 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce3810ec30 con 0x7fce380fa950 2024-09-16T00:20:51.867 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:20:51.865+0000 7fce3ed62640 1 -- 172.21.15.169:0/1623306204 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce38105760 con 0x7fce380fa950 2024-09-16T00:20:51.867 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:20:51.866+0000 7fce357fa640 1 -- 172.21.15.169:0/1623306204 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 18) v1 ==== 90813+0+0 (secure 0 0 0) 0x7fce2800dc30 con 0x7fce380fa950 2024-09-16T00:20:51.868 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:20:51.866+0000 7fce357fa640 1 --2- 172.21.15.169:0/1623306204 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fce2006ecc0 0x7fce20071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:51.868 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:20:51.866+0000 7fce357fa640 1 -- 172.21.15.169:0/1623306204 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3217+0+0 (secure 0 0 0) 0x7fce2808d5b0 con 0x7fce380fa950 2024-09-16T00:20:51.868 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:20:51.867+0000 7fce37fff640 1 --2- 172.21.15.169:0/1623306204 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fce2006ecc0 0x7fce20071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:51.868 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:20:51.867+0000 7fce37fff640 1 --2- 172.21.15.169:0/1623306204 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fce2006ecc0 0x7fce20071170 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fce2c000c00 tx=0x7fce2c035040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:51.871 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:20:51.871+0000 7fce357fa640 1 -- 172.21.15.169:0/1623306204 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fce2805b070 con 0x7fce380fa950 2024-09-16T00:20:52.034 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:20:52.034+0000 7fce3ed62640 1 -- 172.21.15.169:0/1623306204 --> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fce3810ef90 con 0x7fce2006ecc0 2024-09-16T00:20:52.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:51 smithi169 ceph-mon[34653]: pgmap v74: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-16T00:20:52.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:51 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-09-16T00:20:52.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:51 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-09-16T00:20:52.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:51 smithi169 ceph-mon[34653]: osdmap e20: 3 total, 3 up, 3 in 2024-09-16T00:20:52.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:51 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-16T00:20:52.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:51 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:20:52.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:51 smithi112 ceph-mon[27869]: pgmap v74: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-16T00:20:52.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:51 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-09-16T00:20:52.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:51 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-09-16T00:20:52.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:51 smithi112 ceph-mon[27869]: osdmap e20: 3 total, 3 up, 3 in 2024-09-16T00:20:52.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:51 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-16T00:20:52.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:51 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:20:52.839 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:20:52 smithi112 sudo[64986]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T00:20:52.839 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:20:52 smithi112 sudo[64986]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:20:52.839 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:20:52 smithi112 sudo[64986]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:20:52.839 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:20:52 smithi112 sudo[64986]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:20:53.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:52 smithi169 ceph-mon[34653]: from='client.24125 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:20:53.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:52 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:20:53.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:52 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:20:53.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:52 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:20:53.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:52 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-16T00:20:53.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:52 smithi169 ceph-mon[34653]: osdmap e21: 3 total, 3 up, 3 in 2024-09-16T00:20:53.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:52 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:20:53.127 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:20:52 smithi112 sudo[64990]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T00:20:53.127 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:20:52 smithi112 sudo[64990]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:20:53.127 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:20:52 smithi112 sudo[64990]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:20:53.128 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:20:52 smithi112 sudo[64990]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:20:53.128 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:52 smithi112 ceph-mon[27869]: from='client.24125 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:20:53.128 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:52 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:20:53.128 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:52 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:20:53.128 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:52 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:20:53.128 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:52 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-16T00:20:53.128 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:52 smithi112 ceph-mon[27869]: osdmap e21: 3 total, 3 up, 3 in 2024-09-16T00:20:53.128 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:52 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:20:53.182 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:20:53.181+0000 7fce357fa640 1 -- 172.21.15.169:0/1623306204 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mgrmap(e 19) v1 ==== 90869+0+0 (secure 0 0 0) 0x7fce280575d0 con 0x7fce380fa950 2024-09-16T00:20:53.393 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:20:53 smithi112 sudo[64996]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T00:20:53.393 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:20:53 smithi112 sudo[64996]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:20:53.393 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:20:53 smithi112 sudo[64996]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:20:53.393 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:20:53 smithi112 sudo[64996]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:20:53.647 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:20:53 smithi112 sudo[65047]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T00:20:53.647 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:20:53 smithi112 sudo[65047]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:20:53.647 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:20:53 smithi112 sudo[65047]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:20:53.647 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:20:53 smithi112 sudo[65047]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:20:53.917 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:20:53 smithi112 sudo[65136]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T00:20:53.917 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:20:53 smithi112 sudo[65136]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:20:53.917 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:20:53 smithi112 sudo[65136]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:20:53.917 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:20:53 smithi112 sudo[65136]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:20:54.168 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:20:53 smithi112 sudo[65186]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T00:20:54.169 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:20:53 smithi112 sudo[65186]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:20:54.169 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:20:53 smithi112 sudo[65186]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:20:54.169 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:20:53 smithi112 sudo[65186]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:20:54.424 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:54 smithi112 sudo[65205]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T00:20:54.424 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:54 smithi112 sudo[65205]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:20:54.424 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:54 smithi112 sudo[65205]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:20:54.424 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:54 smithi112 sudo[65205]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:20:54.684 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:54 smithi169 ceph-mon[34653]: pgmap v77: 1 pgs: 1 creating+peering; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-16T00:20:54.685 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:54 smithi169 ceph-mon[34653]: mgrmap e19: smithi112.xrkkbc(active, since 2m), standbys: smithi169.mtrobk 2024-09-16T00:20:54.685 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:54 smithi169 ceph-mon[34653]: osdmap e22: 3 total, 3 up, 3 in 2024-09-16T00:20:54.685 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:54 smithi169 ceph-mon[34653]: Detected new or changed devices on smithi112 2024-09-16T00:20:54.685 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:54 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:20:54.685 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:54 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:20:54.685 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:54 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:20:54.685 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:54 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:20:54.685 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:54 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:20:54.685 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:54 smithi169 ceph-mon[34653]: Adjusting osd_memory_target on smithi112 to 4288M 2024-09-16T00:20:54.685 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:54 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:20:54.685 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:54 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:20:54.685 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:54 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:20:54.685 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:54 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:20:54.686 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:54 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:20:54.686 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:54 smithi169 ceph-mon[34653]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T00:20:54.686 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:54 smithi169 sudo[38086]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T00:20:54.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:54 smithi112 sudo[65210]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T00:20:54.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:54 smithi112 sudo[65210]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:20:54.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:54 smithi112 sudo[65210]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:20:54.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:54 smithi112 sudo[65210]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:20:54.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:54 smithi112 ceph-mon[27869]: pgmap v77: 1 pgs: 1 creating+peering; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-16T00:20:54.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:54 smithi112 ceph-mon[27869]: mgrmap e19: smithi112.xrkkbc(active, since 2m), standbys: smithi169.mtrobk 2024-09-16T00:20:54.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:54 smithi112 ceph-mon[27869]: osdmap e22: 3 total, 3 up, 3 in 2024-09-16T00:20:54.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:54 smithi112 ceph-mon[27869]: Detected new or changed devices on smithi112 2024-09-16T00:20:54.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:54 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:20:54.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:54 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:20:54.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:54 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:20:54.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:54 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:20:54.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:54 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:20:54.841 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:54 smithi112 ceph-mon[27869]: Adjusting osd_memory_target on smithi112 to 4288M 2024-09-16T00:20:54.841 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:54 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:20:54.841 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:54 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:20:54.841 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:54 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:20:54.841 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:54 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:20:54.841 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:54 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:20:54.841 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:54 smithi112 ceph-mon[27869]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T00:20:54.947 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:54 smithi169 sudo[38086]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:20:54.947 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:54 smithi169 sudo[38086]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:20:54.947 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:54 smithi169 sudo[38086]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:20:55.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:54 smithi169 sudo[38115]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T00:20:55.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:54 smithi169 sudo[38115]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:20:55.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:54 smithi169 sudo[38115]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:20:55.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:54 smithi169 sudo[38115]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:20:55.772 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:55 smithi112 ceph-mon[27869]: from='client.? 172.21.15.169:0/1597381369' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f0e8ddbe-ed38-4cd9-af68-a68957311e52"}]: dispatch 2024-09-16T00:20:55.773 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:55 smithi112 ceph-mon[27869]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T00:20:55.773 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:55 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mon metadata", "id": "smithi112"}]: dispatch 2024-09-16T00:20:55.773 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:55 smithi112 ceph-mon[27869]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f0e8ddbe-ed38-4cd9-af68-a68957311e52"}]: dispatch 2024-09-16T00:20:55.773 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:55 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mon metadata", "id": "smithi169"}]: dispatch 2024-09-16T00:20:55.773 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:55 smithi112 ceph-mon[27869]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f0e8ddbe-ed38-4cd9-af68-a68957311e52"}]': finished 2024-09-16T00:20:55.773 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:55 smithi112 ceph-mon[27869]: osdmap e23: 4 total, 3 up, 4 in 2024-09-16T00:20:55.773 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:55 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:20:55.774 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:55 smithi112 ceph-mon[27869]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T00:20:55.774 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:55 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mon metadata", "id": "smithi112"}]: dispatch 2024-09-16T00:20:55.774 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:55 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mon metadata", "id": "smithi169"}]: dispatch 2024-09-16T00:20:55.774 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:55 smithi112 ceph-mon[27869]: pgmap v80: 1 pgs: 1 creating+peering; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-16T00:20:55.774 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:55 smithi112 ceph-mon[27869]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T00:20:55.774 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:55 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:20:55.774 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:55 smithi112 ceph-mon[27869]: from='client.? 172.21.15.169:0/2575956316' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:20:55.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:55 smithi169 ceph-mon[34653]: from='client.? 172.21.15.169:0/1597381369' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f0e8ddbe-ed38-4cd9-af68-a68957311e52"}]: dispatch 2024-09-16T00:20:55.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:55 smithi169 ceph-mon[34653]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T00:20:55.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:55 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mon metadata", "id": "smithi112"}]: dispatch 2024-09-16T00:20:55.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:55 smithi169 ceph-mon[34653]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f0e8ddbe-ed38-4cd9-af68-a68957311e52"}]: dispatch 2024-09-16T00:20:55.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:55 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mon metadata", "id": "smithi169"}]: dispatch 2024-09-16T00:20:55.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:55 smithi169 ceph-mon[34653]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f0e8ddbe-ed38-4cd9-af68-a68957311e52"}]': finished 2024-09-16T00:20:55.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:55 smithi169 ceph-mon[34653]: osdmap e23: 4 total, 3 up, 4 in 2024-09-16T00:20:55.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:55 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:20:55.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:55 smithi169 ceph-mon[34653]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T00:20:55.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:55 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mon metadata", "id": "smithi112"}]: dispatch 2024-09-16T00:20:55.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:55 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mon metadata", "id": "smithi169"}]: dispatch 2024-09-16T00:20:55.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:55 smithi169 ceph-mon[34653]: pgmap v80: 1 pgs: 1 creating+peering; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-16T00:20:55.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:55 smithi169 ceph-mon[34653]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T00:20:55.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:55 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:20:55.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:55 smithi169 ceph-mon[34653]: from='client.? 172.21.15.169:0/2575956316' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:20:58.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:57 smithi169 ceph-mon[34653]: pgmap v81: 1 pgs: 1 creating+peering; 0 B data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-16T00:20:58.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:57 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:20:58.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:57 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:20:58.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:57 smithi112 ceph-mon[27869]: pgmap v81: 1 pgs: 1 creating+peering; 0 B data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-16T00:20:58.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:57 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:20:58.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:57 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:20:58.857 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:58 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:20:58.857 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:58 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:20:58.857 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:58 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:20:59.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:58 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:20:59.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:58 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:20:59.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:58 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:21:00.206 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:20:59 smithi169 ceph-mon[34653]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-16T00:21:00.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:20:59 smithi112 ceph-mon[27869]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-16T00:21:02.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:01 smithi169 ceph-mon[34653]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-16T00:21:02.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:01 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-16T00:21:02.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:01 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:21:02.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:01 smithi169 ceph-mon[34653]: Deploying daemon osd.3 on smithi169 2024-09-16T00:21:02.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:01 smithi112 ceph-mon[27869]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-16T00:21:02.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:01 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-16T00:21:02.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:01 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:21:02.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:01 smithi112 ceph-mon[27869]: Deploying daemon osd.3 on smithi169 2024-09-16T00:21:04.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:03 smithi169 ceph-mon[34653]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-16T00:21:04.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:03 smithi112 ceph-mon[27869]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-16T00:21:06.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:06 smithi169 ceph-mon[34653]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-16T00:21:06.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:05 smithi112 ceph-mon[27869]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-16T00:21:07.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:07 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:21:07.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:07 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:21:07.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:07 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:21:07.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:07 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:21:07.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:07 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:21:07.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:07 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:21:07.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:07 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:21:07.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:07 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:21:08.193 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:08 smithi169 ceph-mon[34653]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-16T00:21:08.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:08 smithi112 ceph-mon[27869]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-16T00:21:09.075 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:09.075+0000 7fce357fa640 1 -- 172.21.15.169:0/1623306204 <== mgr.14208 v2:172.21.15.112:6800/2480414866 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fce3810ef90 con 0x7fce2006ecc0 2024-09-16T00:21:09.076 INFO:teuthology.orchestra.run.smithi169.stdout:Created osd(s) 3 on host 'smithi169' 2024-09-16T00:21:09.078 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:09.078+0000 7fce3ed62640 1 -- 172.21.15.169:0/1623306204 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fce2006ecc0 msgr2=0x7fce20071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:09.078 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:09.078+0000 7fce3ed62640 1 --2- 172.21.15.169:0/1623306204 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fce2006ecc0 0x7fce20071170 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fce2c000c00 tx=0x7fce2c035040 comp rx=0 tx=0).stop 2024-09-16T00:21:09.079 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:09.078+0000 7fce3ed62640 1 -- 172.21.15.169:0/1623306204 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce380fa950 msgr2=0x7fce3810d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:09.079 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:09.078+0000 7fce3ed62640 1 --2- 172.21.15.169:0/1623306204 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce380fa950 0x7fce3810d080 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fce28002a80 tx=0x7fce28004570 comp rx=0 tx=0).stop 2024-09-16T00:21:09.079 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:09.078+0000 7fce3ed62640 1 -- 172.21.15.169:0/1623306204 shutdown_connections 2024-09-16T00:21:09.079 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:09.078+0000 7fce3ed62640 1 --2- 172.21.15.169:0/1623306204 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fce2006ecc0 0x7fce20071170 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:09.079 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:09.078+0000 7fce3ed62640 1 --2- 172.21.15.169:0/1623306204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce380fb2c0 0x7fce3810d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:09.079 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:09.078+0000 7fce3ed62640 1 --2- 172.21.15.169:0/1623306204 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce380fa950 0x7fce3810d080 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:09.079 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:09.078+0000 7fce3ed62640 1 -- 172.21.15.169:0/1623306204 >> 172.21.15.169:0/1623306204 conn(0x7fce380f6390 msgr2=0x7fce380ffc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:09.079 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:09.078+0000 7fce3ed62640 1 -- 172.21.15.169:0/1623306204 shutdown_connections 2024-09-16T00:21:09.079 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:09.078+0000 7fce3ed62640 1 -- 172.21.15.169:0/1623306204 wait complete. 2024-09-16T00:21:10.002 DEBUG:teuthology.orchestra.run.smithi169:osd.3> sudo journalctl -f -n 0 -u ceph-d346dd56-73c0-11ef-bceb-c7b262605968@osd.3.service 2024-09-16T00:21:10.004 INFO:tasks.cephadm:Deploying osd.4 on smithi169 with /dev/vg_nvme/lv_3... 2024-09-16T00:21:10.004 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-16T00:21:10.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:10 smithi169 ceph-mon[34653]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-16T00:21:10.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:10 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:21:10.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:10 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:21:10.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:10 smithi169 ceph-mon[34653]: from='osd.3 [v2:172.21.15.169:6800/1327586825,v1:172.21.15.169:6801/1327586825]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-16T00:21:10.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:10 smithi169 ceph-mon[34653]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-16T00:21:10.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:10 smithi112 ceph-mon[27869]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-16T00:21:10.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:10 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:21:10.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:10 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:21:10.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:10 smithi112 ceph-mon[27869]: from='osd.3 [v2:172.21.15.169:6800/1327586825,v1:172.21.15.169:6801/1327586825]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-16T00:21:10.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:10 smithi112 ceph-mon[27869]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-16T00:21:10.393 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi169/config 2024-09-16T00:21:11.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:11 smithi112 ceph-mon[27869]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-16T00:21:11.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:11 smithi112 ceph-mon[27869]: osdmap e24: 4 total, 3 up, 4 in 2024-09-16T00:21:11.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:11 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:21:11.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:11 smithi112 ceph-mon[27869]: from='osd.3 [v2:172.21.15.169:6800/1327586825,v1:172.21.15.169:6801/1327586825]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-09-16T00:21:11.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:11 smithi112 ceph-mon[27869]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-09-16T00:21:11.386 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:11 smithi169 ceph-mon[34653]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-16T00:21:11.386 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:11 smithi169 ceph-mon[34653]: osdmap e24: 4 total, 3 up, 4 in 2024-09-16T00:21:11.386 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:11 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:21:11.386 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:11 smithi169 ceph-mon[34653]: from='osd.3 [v2:172.21.15.169:6800/1327586825,v1:172.21.15.169:6801/1327586825]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-09-16T00:21:11.386 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:11 smithi169 ceph-mon[34653]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-09-16T00:21:11.386 INFO:journalctl@ceph.osd.3.smithi169.stdout:Sep 16 00:21:11 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-3[40159]: 2024-09-16T00:21:11.087+0000 7fb24db3e640 -1 osd.3 0 waiting for initial osdmap 2024-09-16T00:21:11.386 INFO:journalctl@ceph.osd.3.smithi169.stdout:Sep 16 00:21:11 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-3[40159]: 2024-09-16T00:21:11.095+0000 7fb248975640 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T00:21:12.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:12 smithi112 ceph-mon[27869]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-16T00:21:12.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:12 smithi112 ceph-mon[27869]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-09-16T00:21:12.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:12 smithi112 ceph-mon[27869]: osdmap e25: 4 total, 3 up, 4 in 2024-09-16T00:21:12.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:12 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:21:12.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:12 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:21:12.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:12 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:21:12.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:12 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:21:12.468 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:12 smithi169 ceph-mon[34653]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-16T00:21:12.468 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:12 smithi169 ceph-mon[34653]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-09-16T00:21:12.468 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:12 smithi169 ceph-mon[34653]: osdmap e25: 4 total, 3 up, 4 in 2024-09-16T00:21:12.469 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:12 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:21:12.469 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:12 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:21:12.469 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:12 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:21:12.469 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:12 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:21:13.164 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:13 smithi169 ceph-mon[34653]: purged_snaps scrub starts 2024-09-16T00:21:13.164 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:13 smithi169 ceph-mon[34653]: purged_snaps scrub ok 2024-09-16T00:21:13.164 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:13 smithi169 ceph-mon[34653]: osd.3 [v2:172.21.15.169:6800/1327586825,v1:172.21.15.169:6801/1327586825] boot 2024-09-16T00:21:13.164 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:13 smithi169 ceph-mon[34653]: osdmap e26: 4 total, 4 up, 4 in 2024-09-16T00:21:13.164 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:13 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:21:13.169 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:21:13.190 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi169:vg_nvme/lv_3 2024-09-16T00:21:13.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:13 smithi112 ceph-mon[27869]: purged_snaps scrub starts 2024-09-16T00:21:13.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:13 smithi112 ceph-mon[27869]: purged_snaps scrub ok 2024-09-16T00:21:13.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:13 smithi112 ceph-mon[27869]: osd.3 [v2:172.21.15.169:6800/1327586825,v1:172.21.15.169:6801/1327586825] boot 2024-09-16T00:21:13.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:13 smithi112 ceph-mon[27869]: osdmap e26: 4 total, 4 up, 4 in 2024-09-16T00:21:13.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:13 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:21:14.123 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi169/config 2024-09-16T00:21:14.244 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:14 smithi169 ceph-mon[34653]: pgmap v92: 1 pgs: 1 remapped+peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-16T00:21:14.244 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:14 smithi169 ceph-mon[34653]: osdmap e27: 4 total, 4 up, 4 in 2024-09-16T00:21:14.244 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:14 smithi169 ceph-mon[34653]: osdmap e28: 4 total, 4 up, 4 in 2024-09-16T00:21:14.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:14 smithi112 ceph-mon[27869]: pgmap v92: 1 pgs: 1 remapped+peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-16T00:21:14.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:14 smithi112 ceph-mon[27869]: osdmap e27: 4 total, 4 up, 4 in 2024-09-16T00:21:14.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:14 smithi112 ceph-mon[27869]: osdmap e28: 4 total, 4 up, 4 in 2024-09-16T00:21:16.267 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:16.265+0000 7f70bf73f640 1 -- 172.21.15.169:0/1636970865 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70b80fc700 msgr2=0x7f70b80fcb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:16.268 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:16.265+0000 7f70bf73f640 1 --2- 172.21.15.169:0/1636970865 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70b80fc700 0x7f70b80fcb70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f70a8004810 tx=0x7f70a8009da0 comp rx=0 tx=0).stop 2024-09-16T00:21:16.268 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:16.265+0000 7f70bf73f640 1 -- 172.21.15.169:0/1636970865 shutdown_connections 2024-09-16T00:21:16.268 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:16.265+0000 7f70bf73f640 1 --2- 172.21.15.169:0/1636970865 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70b80fc700 0x7f70b80fcb70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:16.268 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:16.265+0000 7f70bf73f640 1 --2- 172.21.15.169:0/1636970865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70b80fb430 0x7f70b80fb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:16.268 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:16.265+0000 7f70bf73f640 1 -- 172.21.15.169:0/1636970865 >> 172.21.15.169:0/1636970865 conn(0x7f70b80f6c10 msgr2=0x7f70b80f9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:16.268 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:16.265+0000 7f70bf73f640 1 -- 172.21.15.169:0/1636970865 shutdown_connections 2024-09-16T00:21:16.268 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:16.265+0000 7f70bf73f640 1 -- 172.21.15.169:0/1636970865 wait complete. 2024-09-16T00:21:16.268 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:16.266+0000 7f70bf73f640 1 Processor -- start 2024-09-16T00:21:16.268 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:16.266+0000 7f70bf73f640 1 -- start start 2024-09-16T00:21:16.268 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:16.266+0000 7f70bf73f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70b80fb430 0x7f70b810d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:16.268 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:16.266+0000 7f70bf73f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70b80fc700 0x7f70b810d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:16.269 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:16.266+0000 7f70bf73f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b810dbf0 con 0x7f70b80fc700 2024-09-16T00:21:16.269 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:16.266+0000 7f70bf73f640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b810e410 con 0x7f70b80fb430 2024-09-16T00:21:16.269 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:16.266+0000 7f70bd4db640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70b80fb430 0x7f70b810d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:16.269 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:16.266+0000 7f70bd4db640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70b80fb430 0x7f70b810d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:59466/0 (socket says 172.21.15.169:59466) 2024-09-16T00:21:16.269 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:16.266+0000 7f70bd4db640 1 -- 172.21.15.169:0/2253436314 learned_addr learned my addr 172.21.15.169:0/2253436314 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-09-16T00:21:16.269 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:16.266+0000 7f70bccda640 1 --2- 172.21.15.169:0/2253436314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70b80fc700 0x7f70b810d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:16.270 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:16.266+0000 7f70bd4db640 1 -- 172.21.15.169:0/2253436314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70b80fc700 msgr2=0x7f70b810d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:16.270 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:16.266+0000 7f70bd4db640 1 --2- 172.21.15.169:0/2253436314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70b80fc700 0x7f70b810d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:16.270 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:16.266+0000 7f70bd4db640 1 -- 172.21.15.169:0/2253436314 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70a8005040 con 0x7f70b80fb430 2024-09-16T00:21:16.270 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:16.266+0000 7f70bd4db640 1 --2- 172.21.15.169:0/2253436314 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70b80fb430 0x7f70b810d060 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f70b400eb40 tx=0x7f70b400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:16.270 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:16.267+0000 7f70adffb640 1 -- 172.21.15.169:0/2253436314 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70b4007c30 con 0x7f70b80fb430 2024-09-16T00:21:16.270 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:16.267+0000 7f70adffb640 1 -- 172.21.15.169:0/2253436314 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f70b400f040 con 0x7f70b80fb430 2024-09-16T00:21:16.270 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:16.267+0000 7f70bf73f640 1 -- 172.21.15.169:0/2253436314 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70b810e640 con 0x7f70b80fb430 2024-09-16T00:21:16.270 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:16.267+0000 7f70adffb640 1 -- 172.21.15.169:0/2253436314 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70b400ca00 con 0x7f70b80fb430 2024-09-16T00:21:16.270 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:16.267+0000 7f70bf73f640 1 -- 172.21.15.169:0/2253436314 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70b810ebe0 con 0x7f70b80fb430 2024-09-16T00:21:16.271 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:16.268+0000 7f70bf73f640 1 -- 172.21.15.169:0/2253436314 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70b80fa890 con 0x7f70b80fb430 2024-09-16T00:21:16.271 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:16.268+0000 7f70adffb640 1 -- 172.21.15.169:0/2253436314 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 19) v1 ==== 90869+0+0 (secure 0 0 0) 0x7f70b400cc70 con 0x7f70b80fb430 2024-09-16T00:21:16.271 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:16.269+0000 7f70adffb640 1 --2- 172.21.15.169:0/2253436314 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f70a406ed50 0x7f70a4071200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:16.271 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:16.269+0000 7f70adffb640 1 -- 172.21.15.169:0/2253436314 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 3770+0+0 (secure 0 0 0) 0x7f70b408d720 con 0x7f70b80fb430 2024-09-16T00:21:16.271 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:16.269+0000 7f70bccda640 1 --2- 172.21.15.169:0/2253436314 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f70a406ed50 0x7f70a4071200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:16.271 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:16.270+0000 7f70bccda640 1 --2- 172.21.15.169:0/2253436314 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f70a406ed50 0x7f70a4071200 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f70b810ce10 tx=0x7f70a8035040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:16.274 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:16.273+0000 7f70adffb640 1 -- 172.21.15.169:0/2253436314 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f70b404e2a0 con 0x7f70b80fb430 2024-09-16T00:21:16.330 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:16 smithi169 ceph-mon[34653]: pgmap v95: 1 pgs: 1 remapped+peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-16T00:21:16.332 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:16 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:21:16.332 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:16 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:21:16.332 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:16 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:21:16.332 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:16 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:21:16.332 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:16 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:21:16.332 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:16 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:21:16.332 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:16 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:21:16.332 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:16 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:21:16.332 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:16 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:21:16.332 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:16 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:21:16.332 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:16 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:21:16.443 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:16.443+0000 7f70bf73f640 1 -- 172.21.15.169:0/2253436314 --> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f70b810ee00 con 0x7f70a406ed50 2024-09-16T00:21:16.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:16 smithi112 ceph-mon[27869]: pgmap v95: 1 pgs: 1 remapped+peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-16T00:21:16.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:16 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:21:16.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:16 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:21:16.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:16 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:21:16.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:16 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:21:16.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:16 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:21:16.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:16 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:21:16.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:16 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:21:16.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:16 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:21:16.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:16 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:21:16.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:16 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:21:16.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:16 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:21:17.212 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:17 smithi169 ceph-mon[34653]: Detected new or changed devices on smithi169 2024-09-16T00:21:17.212 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:17 smithi169 ceph-mon[34653]: Adjusting osd_memory_target on smithi169 to 15936M 2024-09-16T00:21:17.212 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:17 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:21:17.212 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:17 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:21:17.212 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:17 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:21:17.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:17 smithi112 ceph-mon[27869]: Detected new or changed devices on smithi169 2024-09-16T00:21:17.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:17 smithi112 ceph-mon[27869]: Adjusting osd_memory_target on smithi169 to 15936M 2024-09-16T00:21:17.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:17 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:21:17.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:17 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:21:17.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:17 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:21:18.466 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:18 smithi169 ceph-mon[34653]: from='client.24147 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:21:18.466 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:18 smithi169 ceph-mon[34653]: pgmap v96: 1 pgs: 1 remapped+peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-16T00:21:18.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:18 smithi112 ceph-mon[27869]: from='client.24147 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:21:18.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:18 smithi112 ceph-mon[27869]: pgmap v96: 1 pgs: 1 remapped+peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-16T00:21:19.347 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:19 smithi169 ceph-mon[34653]: from='client.? 172.21.15.169:0/1524829069' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06b021d6-0611-4a51-b49e-108783df2837"}]: dispatch 2024-09-16T00:21:19.348 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:19 smithi169 ceph-mon[34653]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06b021d6-0611-4a51-b49e-108783df2837"}]: dispatch 2024-09-16T00:21:19.348 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:19 smithi169 ceph-mon[34653]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "06b021d6-0611-4a51-b49e-108783df2837"}]': finished 2024-09-16T00:21:19.348 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:19 smithi169 ceph-mon[34653]: osdmap e29: 5 total, 4 up, 5 in 2024-09-16T00:21:19.348 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:19 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:21:19.497 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:19 smithi112 ceph-mon[27869]: from='client.? 172.21.15.169:0/1524829069' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06b021d6-0611-4a51-b49e-108783df2837"}]: dispatch 2024-09-16T00:21:19.497 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:19 smithi112 ceph-mon[27869]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06b021d6-0611-4a51-b49e-108783df2837"}]: dispatch 2024-09-16T00:21:19.498 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:19 smithi112 ceph-mon[27869]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "06b021d6-0611-4a51-b49e-108783df2837"}]': finished 2024-09-16T00:21:19.498 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:19 smithi112 ceph-mon[27869]: osdmap e29: 5 total, 4 up, 5 in 2024-09-16T00:21:19.498 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:19 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:21:20.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:20 smithi169 ceph-mon[34653]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-16T00:21:20.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:20 smithi169 ceph-mon[34653]: from='client.? 172.21.15.169:0/4268945609' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:21:20.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:20 smithi112 ceph-mon[27869]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-16T00:21:20.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:20 smithi112 ceph-mon[27869]: from='client.? 172.21.15.169:0/4268945609' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:21:22.392 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:22 smithi112 ceph-mon[27869]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-09-16T00:21:22.392 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:22 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:21:22.505 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:22 smithi169 ceph-mon[34653]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-09-16T00:21:22.505 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:22 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:21:24.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:24 smithi169 ceph-mon[34653]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-09-16T00:21:24.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:24 smithi112 ceph-mon[27869]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-09-16T00:21:26.497 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:26 smithi169 ceph-mon[34653]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T00:21:26.498 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:26 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-16T00:21:26.498 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:26 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:21:26.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:26 smithi112 ceph-mon[27869]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T00:21:26.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:26 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-16T00:21:26.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:26 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:21:27.500 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:27 smithi169 ceph-mon[34653]: Deploying daemon osd.4 on smithi169 2024-09-16T00:21:27.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:27 smithi112 ceph-mon[27869]: Deploying daemon osd.4 on smithi169 2024-09-16T00:21:28.410 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:28 smithi169 ceph-mon[34653]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T00:21:28.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:28 smithi112 ceph-mon[27869]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T00:21:30.176 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:30 smithi169 ceph-mon[34653]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-16T00:21:30.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:30 smithi112 ceph-mon[27869]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-16T00:21:31.424 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:31 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:21:31.424 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:31 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:21:31.424 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:31 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:21:31.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:31 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:21:31.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:31 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:21:31.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:31 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:21:32.428 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:32 smithi169 ceph-mon[34653]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-16T00:21:32.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:32 smithi112 ceph-mon[27869]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-16T00:21:34.425 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:34 smithi169 ceph-mon[34653]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-16T00:21:34.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:34 smithi112 ceph-mon[27869]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-16T00:21:34.716 INFO:teuthology.orchestra.run.smithi169.stdout:Created osd(s) 4 on host 'smithi169' 2024-09-16T00:21:34.716 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:34.715+0000 7f70adffb640 1 -- 172.21.15.169:0/2253436314 <== mgr.14208 v2:172.21.15.112:6800/2480414866 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f70b810ee00 con 0x7f70a406ed50 2024-09-16T00:21:34.718 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:34.717+0000 7f70bf73f640 1 -- 172.21.15.169:0/2253436314 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f70a406ed50 msgr2=0x7f70a4071200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:34.718 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:34.717+0000 7f70bf73f640 1 --2- 172.21.15.169:0/2253436314 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f70a406ed50 0x7f70a4071200 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f70b810ce10 tx=0x7f70a8035040 comp rx=0 tx=0).stop 2024-09-16T00:21:34.718 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:34.717+0000 7f70bf73f640 1 -- 172.21.15.169:0/2253436314 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70b80fb430 msgr2=0x7f70b810d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:34.718 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:34.717+0000 7f70bf73f640 1 --2- 172.21.15.169:0/2253436314 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70b80fb430 0x7f70b810d060 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f70b400eb40 tx=0x7f70b400b040 comp rx=0 tx=0).stop 2024-09-16T00:21:34.718 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:34.718+0000 7f70bf73f640 1 -- 172.21.15.169:0/2253436314 shutdown_connections 2024-09-16T00:21:34.718 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:34.718+0000 7f70bf73f640 1 --2- 172.21.15.169:0/2253436314 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f70a406ed50 0x7f70a4071200 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:34.718 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:34.718+0000 7f70bf73f640 1 --2- 172.21.15.169:0/2253436314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70b80fc700 0x7f70b810d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:34.718 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:34.718+0000 7f70bf73f640 1 --2- 172.21.15.169:0/2253436314 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70b80fb430 0x7f70b810d060 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:34.718 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:34.718+0000 7f70bf73f640 1 -- 172.21.15.169:0/2253436314 >> 172.21.15.169:0/2253436314 conn(0x7f70b80f6c10 msgr2=0x7f70b80f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:34.719 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:34.718+0000 7f70bf73f640 1 -- 172.21.15.169:0/2253436314 shutdown_connections 2024-09-16T00:21:34.719 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:34.718+0000 7f70bf73f640 1 -- 172.21.15.169:0/2253436314 wait complete. 2024-09-16T00:21:35.280 DEBUG:teuthology.orchestra.run.smithi169:osd.4> sudo journalctl -f -n 0 -u ceph-d346dd56-73c0-11ef-bceb-c7b262605968@osd.4.service 2024-09-16T00:21:35.283 INFO:tasks.cephadm:Deploying osd.5 on smithi169 with /dev/vg_nvme/lv_2... 2024-09-16T00:21:35.283 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-16T00:21:35.305 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:35 smithi169 ceph-mon[34653]: from='osd.4 [v2:172.21.15.169:6808/3980576686,v1:172.21.15.169:6809/3980576686]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T00:21:35.305 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:35 smithi169 ceph-mon[34653]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T00:21:35.305 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:35 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:21:35.305 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:35 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:21:35.560 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi169/config 2024-09-16T00:21:35.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:35 smithi112 ceph-mon[27869]: from='osd.4 [v2:172.21.15.169:6808/3980576686,v1:172.21.15.169:6809/3980576686]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T00:21:35.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:35 smithi112 ceph-mon[27869]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T00:21:35.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:35 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:21:35.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:35 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:21:36.275 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:36 smithi169 ceph-mon[34653]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-16T00:21:36.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:36 smithi169 ceph-mon[34653]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-16T00:21:36.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:36 smithi169 ceph-mon[34653]: osdmap e30: 5 total, 4 up, 5 in 2024-09-16T00:21:36.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:36 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:21:36.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:36 smithi169 ceph-mon[34653]: from='osd.4 [v2:172.21.15.169:6808/3980576686,v1:172.21.15.169:6809/3980576686]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-09-16T00:21:36.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:36 smithi169 ceph-mon[34653]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-09-16T00:21:36.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:36 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:21:36.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:36 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:21:36.276 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:21:36 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-4[46505]: 2024-09-16T00:21:36.157+0000 7f0c61d22640 -1 osd.4 0 waiting for initial osdmap 2024-09-16T00:21:36.276 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:21:36 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-4[46505]: 2024-09-16T00:21:36.167+0000 7f0c5c358640 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T00:21:36.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:36 smithi112 ceph-mon[27869]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-16T00:21:36.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:36 smithi112 ceph-mon[27869]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-16T00:21:36.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:36 smithi112 ceph-mon[27869]: osdmap e30: 5 total, 4 up, 5 in 2024-09-16T00:21:36.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:36 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:21:36.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:36 smithi112 ceph-mon[27869]: from='osd.4 [v2:172.21.15.169:6808/3980576686,v1:172.21.15.169:6809/3980576686]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-09-16T00:21:36.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:36 smithi112 ceph-mon[27869]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-09-16T00:21:36.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:36 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:21:36.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:36 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:21:37.435 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:37 smithi169 ceph-mon[34653]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-09-16T00:21:37.435 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:37 smithi169 ceph-mon[34653]: osdmap e31: 5 total, 4 up, 5 in 2024-09-16T00:21:37.435 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:37 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:21:37.435 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:37 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:21:37.435 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:37 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:21:37.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:37 smithi112 ceph-mon[27869]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-09-16T00:21:37.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:37 smithi112 ceph-mon[27869]: osdmap e31: 5 total, 4 up, 5 in 2024-09-16T00:21:37.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:37 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:21:37.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:37 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:21:37.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:37 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:21:38.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:38 smithi169 ceph-mon[34653]: purged_snaps scrub starts 2024-09-16T00:21:38.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:38 smithi169 ceph-mon[34653]: purged_snaps scrub ok 2024-09-16T00:21:38.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:38 smithi169 ceph-mon[34653]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-16T00:21:38.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:38 smithi169 ceph-mon[34653]: osd.4 [v2:172.21.15.169:6808/3980576686,v1:172.21.15.169:6809/3980576686] boot 2024-09-16T00:21:38.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:38 smithi169 ceph-mon[34653]: osdmap e32: 5 total, 5 up, 5 in 2024-09-16T00:21:38.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:38 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:21:38.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:38 smithi112 ceph-mon[27869]: purged_snaps scrub starts 2024-09-16T00:21:38.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:38 smithi112 ceph-mon[27869]: purged_snaps scrub ok 2024-09-16T00:21:38.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:38 smithi112 ceph-mon[27869]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-16T00:21:38.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:38 smithi112 ceph-mon[27869]: osd.4 [v2:172.21.15.169:6808/3980576686,v1:172.21.15.169:6809/3980576686] boot 2024-09-16T00:21:38.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:38 smithi112 ceph-mon[27869]: osdmap e32: 5 total, 5 up, 5 in 2024-09-16T00:21:38.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:38 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:21:39.430 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:39 smithi169 ceph-mon[34653]: osdmap e33: 5 total, 5 up, 5 in 2024-09-16T00:21:39.430 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:39 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:21:39.430 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:39 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:21:39.430 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:39 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:21:39.430 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:39 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:21:39.430 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:39 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:21:39.430 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:39 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:21:39.430 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:39 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:21:39.430 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:39 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:21:39.496 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:39 smithi112 ceph-mon[27869]: osdmap e33: 5 total, 5 up, 5 in 2024-09-16T00:21:39.496 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:39 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:21:39.497 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:39 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:21:39.497 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:39 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:21:39.497 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:39 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:21:39.497 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:39 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:21:39.497 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:39 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:21:39.497 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:39 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:21:39.497 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:39 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:21:40.177 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:40 smithi169 ceph-mon[34653]: pgmap v112: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-16T00:21:40.482 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:40 smithi169 ceph-mon[34653]: Detected new or changed devices on smithi169 2024-09-16T00:21:40.482 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:40 smithi169 ceph-mon[34653]: Adjusting osd_memory_target on smithi169 to 7968M 2024-09-16T00:21:40.482 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:40 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:21:40.482 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:40 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:21:40.482 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:40 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:21:40.482 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:40 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:21:40.488 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-09-16T00:21:40.509 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi169:vg_nvme/lv_2 2024-09-16T00:21:40.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:40 smithi112 ceph-mon[27869]: pgmap v112: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-16T00:21:40.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:40 smithi112 ceph-mon[27869]: Detected new or changed devices on smithi169 2024-09-16T00:21:40.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:40 smithi112 ceph-mon[27869]: Adjusting osd_memory_target on smithi169 to 7968M 2024-09-16T00:21:40.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:40 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:21:40.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:40 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:21:40.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:40 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:21:40.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:40 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:21:40.755 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi169/config 2024-09-16T00:21:42.106 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:42.105+0000 7f3549312640 1 -- 172.21.15.169:0/1496108131 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35440fc540 msgr2=0x7f35440fc950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:42.106 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:42.105+0000 7f3549312640 1 --2- 172.21.15.169:0/1496108131 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35440fc540 0x7f35440fc950 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f35340048d0 tx=0x7f3534009da0 comp rx=0 tx=0).stop 2024-09-16T00:21:42.106 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:42.105+0000 7f3549312640 1 -- 172.21.15.169:0/1496108131 shutdown_connections 2024-09-16T00:21:42.106 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:42.105+0000 7f3549312640 1 --2- 172.21.15.169:0/1496108131 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35440fd810 0x7f35440fdc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:42.106 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:42.105+0000 7f3549312640 1 --2- 172.21.15.169:0/1496108131 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35440fc540 0x7f35440fc950 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:42.106 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:42.105+0000 7f3549312640 1 -- 172.21.15.169:0/1496108131 >> 172.21.15.169:0/1496108131 conn(0x7f35440f7d00 msgr2=0x7f35440fa130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:42.106 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:42.105+0000 7f3549312640 1 -- 172.21.15.169:0/1496108131 shutdown_connections 2024-09-16T00:21:42.106 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:42.105+0000 7f3549312640 1 -- 172.21.15.169:0/1496108131 wait complete. 2024-09-16T00:21:42.106 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:42.106+0000 7f3549312640 1 Processor -- start 2024-09-16T00:21:42.107 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:42.106+0000 7f3549312640 1 -- start start 2024-09-16T00:21:42.108 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:42.106+0000 7f3549312640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35440fc540 0x7f354410d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:42.108 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:42.106+0000 7f3549312640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35440fd810 0x7f354410d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:42.108 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:42.106+0000 7f3549312640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f354410e480 con 0x7f35440fd810 2024-09-16T00:21:42.108 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:42.106+0000 7f3549312640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f354410e5f0 con 0x7f35440fc540 2024-09-16T00:21:42.108 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:42.106+0000 7f3542ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35440fc540 0x7f354410d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:42.108 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:42.106+0000 7f3542ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35440fc540 0x7f354410d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:59642/0 (socket says 172.21.15.169:59642) 2024-09-16T00:21:42.109 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:42.106+0000 7f3542ffd640 1 -- 172.21.15.169:0/3880096562 learned_addr learned my addr 172.21.15.169:0/3880096562 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-09-16T00:21:42.109 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:42.106+0000 7f35427fc640 1 --2- 172.21.15.169:0/3880096562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35440fd810 0x7f354410d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:42.109 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:42.106+0000 7f3542ffd640 1 -- 172.21.15.169:0/3880096562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35440fd810 msgr2=0x7f354410d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:42.109 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:42.106+0000 7f3542ffd640 1 --2- 172.21.15.169:0/3880096562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35440fd810 0x7f354410d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:42.109 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:42.106+0000 7f3542ffd640 1 -- 172.21.15.169:0/3880096562 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3534005040 con 0x7f35440fc540 2024-09-16T00:21:42.109 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:42.106+0000 7f3542ffd640 1 --2- 172.21.15.169:0/3880096562 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35440fc540 0x7f354410d0d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3534003080 tx=0x7f3534009da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:42.109 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:42.107+0000 7f352b7fe640 1 -- 172.21.15.169:0/3880096562 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3534038070 con 0x7f35440fc540 2024-09-16T00:21:42.109 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:42.107+0000 7f352b7fe640 1 -- 172.21.15.169:0/3880096562 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f353402abd0 con 0x7f35440fc540 2024-09-16T00:21:42.109 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:42.107+0000 7f352b7fe640 1 -- 172.21.15.169:0/3880096562 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f353403e670 con 0x7f35440fc540 2024-09-16T00:21:42.109 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:42.107+0000 7f3549312640 1 -- 172.21.15.169:0/3880096562 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f354410e870 con 0x7f35440fc540 2024-09-16T00:21:42.109 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:42.107+0000 7f3549312640 1 -- 172.21.15.169:0/3880096562 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f354410ed90 con 0x7f35440fc540 2024-09-16T00:21:42.110 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:42.108+0000 7f3549312640 1 -- 172.21.15.169:0/3880096562 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35441057d0 con 0x7f35440fc540 2024-09-16T00:21:42.110 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:42.108+0000 7f352b7fe640 1 -- 172.21.15.169:0/3880096562 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 19) v1 ==== 90869+0+0 (secure 0 0 0) 0x7f3534030020 con 0x7f35440fc540 2024-09-16T00:21:42.110 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:42.109+0000 7f352b7fe640 1 --2- 172.21.15.169:0/3880096562 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f352c06ed60 0x7f352c071210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:42.110 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:42.109+0000 7f352b7fe640 1 -- 172.21.15.169:0/3880096562 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4202+0+0 (secure 0 0 0) 0x7f35340adfd0 con 0x7f35440fc540 2024-09-16T00:21:42.111 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:42.109+0000 7f35427fc640 1 --2- 172.21.15.169:0/3880096562 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f352c06ed60 0x7f352c071210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:42.111 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:42.109+0000 7f35427fc640 1 --2- 172.21.15.169:0/3880096562 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f352c06ed60 0x7f352c071210 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3538006fd0 tx=0x7f3538008040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:42.113 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:42.113+0000 7f352b7fe640 1 -- 172.21.15.169:0/3880096562 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f353407b660 con 0x7f35440fc540 2024-09-16T00:21:42.275 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:42.274+0000 7f3549312640 1 -- 172.21.15.169:0/3880096562 --> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f354410f000 con 0x7f352c06ed60 2024-09-16T00:21:42.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:42 smithi169 ceph-mon[34653]: pgmap v113: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-16T00:21:42.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:42 smithi112 ceph-mon[27869]: pgmap v113: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-16T00:21:43.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:43 smithi169 ceph-mon[34653]: from='client.24165 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:21:43.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:43 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:21:43.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:43 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:21:43.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:43 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:21:43.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:43 smithi112 ceph-mon[27869]: from='client.24165 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:21:43.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:43 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:21:43.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:43 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:21:43.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:43 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:21:44.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:44 smithi169 ceph-mon[34653]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-16T00:21:44.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:44 smithi112 ceph-mon[27869]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-16T00:21:45.286 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:45 smithi169 ceph-mon[34653]: from='client.? 172.21.15.169:0/2798669732' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "666b01f8-29fa-427d-a2c6-2ef236d0ddd7"}]: dispatch 2024-09-16T00:21:45.287 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:45 smithi169 ceph-mon[34653]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "666b01f8-29fa-427d-a2c6-2ef236d0ddd7"}]: dispatch 2024-09-16T00:21:45.287 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:45 smithi169 ceph-mon[34653]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "666b01f8-29fa-427d-a2c6-2ef236d0ddd7"}]': finished 2024-09-16T00:21:45.287 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:45 smithi169 ceph-mon[34653]: osdmap e34: 6 total, 5 up, 6 in 2024-09-16T00:21:45.287 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:45 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:21:45.287 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:45 smithi169 ceph-mon[34653]: from='client.? 172.21.15.169:0/1877146583' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:21:45.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:45 smithi112 ceph-mon[27869]: from='client.? 172.21.15.169:0/2798669732' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "666b01f8-29fa-427d-a2c6-2ef236d0ddd7"}]: dispatch 2024-09-16T00:21:45.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:45 smithi112 ceph-mon[27869]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "666b01f8-29fa-427d-a2c6-2ef236d0ddd7"}]: dispatch 2024-09-16T00:21:45.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:45 smithi112 ceph-mon[27869]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "666b01f8-29fa-427d-a2c6-2ef236d0ddd7"}]': finished 2024-09-16T00:21:45.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:45 smithi112 ceph-mon[27869]: osdmap e34: 6 total, 5 up, 6 in 2024-09-16T00:21:45.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:45 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:21:45.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:45 smithi112 ceph-mon[27869]: from='client.? 172.21.15.169:0/1877146583' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:21:46.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:46 smithi169 ceph-mon[34653]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-16T00:21:46.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:46 smithi112 ceph-mon[27869]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-16T00:21:48.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:48 smithi169 ceph-mon[34653]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-16T00:21:48.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:48 smithi112 ceph-mon[27869]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-16T00:21:50.296 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:50 smithi169 ceph-mon[34653]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-16T00:21:50.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:50 smithi112 ceph-mon[27869]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-16T00:21:52.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:52 smithi169 ceph-mon[34653]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-16T00:21:52.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:52 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-16T00:21:52.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:52 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:21:52.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:52 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:21:52.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:52 smithi112 ceph-mon[27869]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-16T00:21:52.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:52 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-16T00:21:52.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:52 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:21:52.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:52 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:21:53.419 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:53 smithi169 ceph-mon[34653]: Deploying daemon osd.5 on smithi169 2024-09-16T00:21:53.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:53 smithi112 ceph-mon[27869]: Deploying daemon osd.5 on smithi169 2024-09-16T00:21:54.506 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:54 smithi169 ceph-mon[34653]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-16T00:21:54.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:54 smithi112 ceph-mon[27869]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-16T00:21:56.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:56 smithi169 ceph-mon[34653]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-16T00:21:56.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:56 smithi112 ceph-mon[27869]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-16T00:21:57.437 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:57 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:21:57.438 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:57 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:21:57.438 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:57 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:21:57.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:57 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:21:57.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:57 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:21:57.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:57 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:21:58.641 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:21:58 smithi169 ceph-mon[34653]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-16T00:21:58.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:21:58 smithi112 ceph-mon[27869]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-16T00:21:59.986 INFO:teuthology.orchestra.run.smithi169.stdout:Created osd(s) 5 on host 'smithi169' 2024-09-16T00:21:59.986 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:59.985+0000 7f352b7fe640 1 -- 172.21.15.169:0/3880096562 <== mgr.14208 v2:172.21.15.112:6800/2480414866 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f354410f000 con 0x7f352c06ed60 2024-09-16T00:21:59.988 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:59.987+0000 7f3549312640 1 -- 172.21.15.169:0/3880096562 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f352c06ed60 msgr2=0x7f352c071210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:59.988 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:59.987+0000 7f3549312640 1 --2- 172.21.15.169:0/3880096562 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f352c06ed60 0x7f352c071210 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3538006fd0 tx=0x7f3538008040 comp rx=0 tx=0).stop 2024-09-16T00:21:59.988 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:59.987+0000 7f3549312640 1 -- 172.21.15.169:0/3880096562 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35440fc540 msgr2=0x7f354410d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:59.989 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:59.987+0000 7f3549312640 1 --2- 172.21.15.169:0/3880096562 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35440fc540 0x7f354410d0d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3534003080 tx=0x7f3534009da0 comp rx=0 tx=0).stop 2024-09-16T00:21:59.989 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:59.987+0000 7f3549312640 1 -- 172.21.15.169:0/3880096562 shutdown_connections 2024-09-16T00:21:59.989 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:59.987+0000 7f3549312640 1 --2- 172.21.15.169:0/3880096562 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f352c06ed60 0x7f352c071210 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:59.989 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:59.987+0000 7f3549312640 1 --2- 172.21.15.169:0/3880096562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35440fd810 0x7f354410d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:59.989 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:59.987+0000 7f3549312640 1 --2- 172.21.15.169:0/3880096562 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35440fc540 0x7f354410d0d0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:59.989 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:59.987+0000 7f3549312640 1 -- 172.21.15.169:0/3880096562 >> 172.21.15.169:0/3880096562 conn(0x7f35440f7d00 msgr2=0x7f35440f9470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:59.989 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:59.987+0000 7f3549312640 1 -- 172.21.15.169:0/3880096562 shutdown_connections 2024-09-16T00:21:59.989 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:21:59.987+0000 7f3549312640 1 -- 172.21.15.169:0/3880096562 wait complete. 2024-09-16T00:22:00.501 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:00 smithi169 ceph-mon[34653]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-16T00:22:00.501 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:00 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:22:00.501 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:00 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:22:00.501 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:00 smithi169 ceph-mon[34653]: from='osd.5 [v2:172.21.15.169:6816/2190808120,v1:172.21.15.169:6817/2190808120]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T00:22:00.501 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:00 smithi169 ceph-mon[34653]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T00:22:00.560 DEBUG:teuthology.orchestra.run.smithi169:osd.5> sudo journalctl -f -n 0 -u ceph-d346dd56-73c0-11ef-bceb-c7b262605968@osd.5.service 2024-09-16T00:22:00.563 INFO:tasks.cephadm:Waiting for 6 OSDs to come up... 2024-09-16T00:22:00.563 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-16T00:22:00.822 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:22:00.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:00 smithi112 ceph-mon[27869]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-16T00:22:00.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:00 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:22:00.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:00 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:22:00.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:00 smithi112 ceph-mon[27869]: from='osd.5 [v2:172.21.15.169:6816/2190808120,v1:172.21.15.169:6817/2190808120]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T00:22:00.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:00 smithi112 ceph-mon[27869]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T00:22:02.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:01 smithi112 ceph-mon[27869]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-16T00:22:02.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:01 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:22:02.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:01 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:22:02.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:01 smithi112 ceph-mon[27869]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-16T00:22:02.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:01 smithi112 ceph-mon[27869]: osdmap e35: 6 total, 5 up, 6 in 2024-09-16T00:22:02.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:01 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:22:02.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:01 smithi112 ceph-mon[27869]: from='osd.5 [v2:172.21.15.169:6816/2190808120,v1:172.21.15.169:6817/2190808120]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-09-16T00:22:02.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:01 smithi112 ceph-mon[27869]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-09-16T00:22:02.245 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:01 smithi169 ceph-mon[34653]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-16T00:22:02.245 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:01 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:22:02.245 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:01 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:22:02.245 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:01 smithi169 ceph-mon[34653]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-16T00:22:02.245 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:01 smithi169 ceph-mon[34653]: osdmap e35: 6 total, 5 up, 6 in 2024-09-16T00:22:02.245 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:01 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:22:02.245 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:01 smithi169 ceph-mon[34653]: from='osd.5 [v2:172.21.15.169:6816/2190808120,v1:172.21.15.169:6817/2190808120]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-09-16T00:22:02.245 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:01 smithi169 ceph-mon[34653]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-09-16T00:22:02.246 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:22:01 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5[53200]: 2024-09-16T00:22:01.995+0000 7f72c33c2640 -1 osd.5 0 waiting for initial osdmap 2024-09-16T00:22:02.246 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:22:02 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5[53200]: 2024-09-16T00:22:02.003+0000 7f72be9fa640 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T00:22:02.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:02.588+0000 7f6c4deac640 1 -- 172.21.15.112:0/2043928314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c480fd820 msgr2=0x7f6c480fdc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:02.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:02.588+0000 7f6c4deac640 1 --2- 172.21.15.112:0/2043928314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c480fd820 0x7f6c480fdc90 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c0048d0 tx=0x7f6c3c009da0 comp rx=0 tx=0).stop 2024-09-16T00:22:02.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:02.588+0000 7f6c4deac640 1 -- 172.21.15.112:0/2043928314 shutdown_connections 2024-09-16T00:22:02.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:02.588+0000 7f6c4deac640 1 --2- 172.21.15.112:0/2043928314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c480fd820 0x7f6c480fdc90 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:02.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:02.588+0000 7f6c4deac640 1 --2- 172.21.15.112:0/2043928314 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c480fc550 0x7f6c480fc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:02.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:02.588+0000 7f6c4deac640 1 -- 172.21.15.112:0/2043928314 >> 172.21.15.112:0/2043928314 conn(0x7f6c480f7d30 msgr2=0x7f6c480fa140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:02.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:02.588+0000 7f6c4deac640 1 -- 172.21.15.112:0/2043928314 shutdown_connections 2024-09-16T00:22:02.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:02.589+0000 7f6c4deac640 1 -- 172.21.15.112:0/2043928314 wait complete. 2024-09-16T00:22:02.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:02.589+0000 7f6c4deac640 1 Processor -- start 2024-09-16T00:22:02.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:02.589+0000 7f6c4deac640 1 -- start start 2024-09-16T00:22:02.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:02.589+0000 7f6c4deac640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c480fc550 0x7f6c4810f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:02.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:02.589+0000 7f6c4deac640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c480fd820 0x7f6c4810f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:02.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:02.589+0000 7f6c4deac640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4810fe70 con 0x7f6c480fc550 2024-09-16T00:22:02.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:02.589+0000 7f6c4deac640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c48110690 con 0x7f6c480fd820 2024-09-16T00:22:02.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:02.589+0000 7f6c477fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c480fc550 0x7f6c4810f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:02.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:02.589+0000 7f6c477fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c480fc550 0x7f6c4810f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59844/0 (socket says 172.21.15.112:59844) 2024-09-16T00:22:02.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:02.589+0000 7f6c477fe640 1 -- 172.21.15.112:0/1428176559 learned_addr learned my addr 172.21.15.112:0/1428176559 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:22:02.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:02.590+0000 7f6c46ffd640 1 --2- 172.21.15.112:0/1428176559 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c480fd820 0x7f6c4810f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:02.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:02.590+0000 7f6c477fe640 1 -- 172.21.15.112:0/1428176559 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c480fd820 msgr2=0x7f6c4810f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:02.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:02.590+0000 7f6c477fe640 1 --2- 172.21.15.112:0/1428176559 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c480fd820 0x7f6c4810f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:02.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:02.590+0000 7f6c477fe640 1 -- 172.21.15.112:0/1428176559 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c3c005040 con 0x7f6c480fc550 2024-09-16T00:22:02.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:02.590+0000 7f6c477fe640 1 --2- 172.21.15.112:0/1428176559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c480fc550 0x7f6c4810f2e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6c38002a50 tx=0x7f6c38004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:02.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:02.590+0000 7f6c2ffff640 1 -- 172.21.15.112:0/1428176559 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c38007c30 con 0x7f6c480fc550 2024-09-16T00:22:02.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:02.590+0000 7f6c2ffff640 1 -- 172.21.15.112:0/1428176559 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f6c3800f040 con 0x7f6c480fc550 2024-09-16T00:22:02.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:02.590+0000 7f6c4deac640 1 -- 172.21.15.112:0/1428176559 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c48110920 con 0x7f6c480fc550 2024-09-16T00:22:02.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:02.590+0000 7f6c2ffff640 1 -- 172.21.15.112:0/1428176559 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c3800da90 con 0x7f6c480fc550 2024-09-16T00:22:02.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:02.590+0000 7f6c4deac640 1 -- 172.21.15.112:0/1428176559 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c48110ec0 con 0x7f6c480fc550 2024-09-16T00:22:02.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:02.591+0000 7f6c4deac640 1 -- 172.21.15.112:0/1428176559 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c480fb9b0 con 0x7f6c480fc550 2024-09-16T00:22:02.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:02.592+0000 7f6c2ffff640 1 -- 172.21.15.112:0/1428176559 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) v1 ==== 90869+0+0 (secure 0 0 0) 0x7f6c3800dc30 con 0x7f6c480fc550 2024-09-16T00:22:02.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:02.592+0000 7f6c2ffff640 1 --2- 172.21.15.112:0/1428176559 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f6c3006ed60 0x7f6c30071210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:02.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:02.592+0000 7f6c2ffff640 1 -- 172.21.15.112:0/1428176559 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4354+0+0 (secure 0 0 0) 0x7f6c3808d580 con 0x7f6c480fc550 2024-09-16T00:22:02.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:02.592+0000 7f6c46ffd640 1 --2- 172.21.15.112:0/1428176559 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f6c3006ed60 0x7f6c30071210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:02.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:02.593+0000 7f6c46ffd640 1 --2- 172.21.15.112:0/1428176559 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f6c3006ed60 0x7f6c30071210 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c007ac0 tx=0x7f6c3c035040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:02.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:02.596+0000 7f6c2ffff640 1 -- 172.21.15.112:0/1428176559 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f6c3804d2a0 con 0x7f6c480fc550 2024-09-16T00:22:02.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:02.753+0000 7f6c4deac640 1 -- 172.21.15.112:0/1428176559 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f6c4804b720 con 0x7f6c480fc550 2024-09-16T00:22:02.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:02.753+0000 7f6c2ffff640 1 -- 172.21.15.112:0/1428176559 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v36) v1 ==== 74+0+130 (secure 0 0 0) 0x7f6c4804b720 con 0x7f6c480fc550 2024-09-16T00:22:02.754 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:22:02.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:02.756+0000 7f6c4deac640 1 -- 172.21.15.112:0/1428176559 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f6c3006ed60 msgr2=0x7f6c30071210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:02.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:02.756+0000 7f6c4deac640 1 --2- 172.21.15.112:0/1428176559 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f6c3006ed60 0x7f6c30071210 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c007ac0 tx=0x7f6c3c035040 comp rx=0 tx=0).stop 2024-09-16T00:22:02.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:02.756+0000 7f6c4deac640 1 -- 172.21.15.112:0/1428176559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c480fc550 msgr2=0x7f6c4810f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:02.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:02.756+0000 7f6c4deac640 1 --2- 172.21.15.112:0/1428176559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c480fc550 0x7f6c4810f2e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6c38002a50 tx=0x7f6c38004600 comp rx=0 tx=0).stop 2024-09-16T00:22:02.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:02.756+0000 7f6c4deac640 1 -- 172.21.15.112:0/1428176559 shutdown_connections 2024-09-16T00:22:02.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:02.756+0000 7f6c4deac640 1 --2- 172.21.15.112:0/1428176559 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f6c3006ed60 0x7f6c30071210 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:02.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:02.756+0000 7f6c4deac640 1 --2- 172.21.15.112:0/1428176559 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c480fd820 0x7f6c4810f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:02.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:02.756+0000 7f6c4deac640 1 --2- 172.21.15.112:0/1428176559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c480fc550 0x7f6c4810f2e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:02.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:02.756+0000 7f6c4deac640 1 -- 172.21.15.112:0/1428176559 >> 172.21.15.112:0/1428176559 conn(0x7f6c480f7d30 msgr2=0x7f6c480f93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:02.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:02.756+0000 7f6c4deac640 1 -- 172.21.15.112:0/1428176559 shutdown_connections 2024-09-16T00:22:02.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:02.756+0000 7f6c4deac640 1 -- 172.21.15.112:0/1428176559 wait complete. 2024-09-16T00:22:03.078 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:02 smithi112 ceph-mon[27869]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-09-16T00:22:03.079 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:02 smithi112 ceph-mon[27869]: osdmap e36: 6 total, 5 up, 6 in 2024-09-16T00:22:03.079 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:02 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:22:03.079 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:02 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:22:03.079 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:02 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/1428176559' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T00:22:03.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:02 smithi169 ceph-mon[34653]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-09-16T00:22:03.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:02 smithi169 ceph-mon[34653]: osdmap e36: 6 total, 5 up, 6 in 2024-09-16T00:22:03.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:02 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:22:03.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:02 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:22:03.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:02 smithi169 ceph-mon[34653]: from='client.? 172.21.15.112:0/1428176559' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T00:22:03.529 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":36,"num_osds":6,"num_up_osds":5,"osd_up_since":1726446097,"num_in_osds":6,"osd_in_since":1726446104,"num_remapped_pgs":0} 2024-09-16T00:22:04.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:04 smithi112 ceph-mon[27869]: purged_snaps scrub starts 2024-09-16T00:22:04.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:04 smithi112 ceph-mon[27869]: purged_snaps scrub ok 2024-09-16T00:22:04.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:04 smithi112 ceph-mon[27869]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-16T00:22:04.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:04 smithi112 ceph-mon[27869]: osd.5 [v2:172.21.15.169:6816/2190808120,v1:172.21.15.169:6817/2190808120] boot 2024-09-16T00:22:04.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:04 smithi112 ceph-mon[27869]: osdmap e37: 6 total, 6 up, 6 in 2024-09-16T00:22:04.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:04 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:22:04.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:04 smithi169 ceph-mon[34653]: purged_snaps scrub starts 2024-09-16T00:22:04.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:04 smithi169 ceph-mon[34653]: purged_snaps scrub ok 2024-09-16T00:22:04.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:04 smithi169 ceph-mon[34653]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-16T00:22:04.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:04 smithi169 ceph-mon[34653]: osd.5 [v2:172.21.15.169:6816/2190808120,v1:172.21.15.169:6817/2190808120] boot 2024-09-16T00:22:04.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:04 smithi169 ceph-mon[34653]: osdmap e37: 6 total, 6 up, 6 in 2024-09-16T00:22:04.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:04 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:22:04.530 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-16T00:22:04.787 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:22:05.217 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:05 smithi112 ceph-mon[27869]: osdmap e38: 6 total, 6 up, 6 in 2024-09-16T00:22:05.217 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:05 smithi112 ceph-mon[27869]: Detected new or changed devices on smithi169 2024-09-16T00:22:05.217 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:05 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:22:05.217 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:05 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:22:05.217 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:05 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:22:05.217 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:05 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:22:05.217 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:05 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:22:05.217 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:05 smithi112 ceph-mon[27869]: Adjusting osd_memory_target on smithi169 to 5312M 2024-09-16T00:22:05.217 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:05 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:22:05.217 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:05 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:22:05.217 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:05 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:22:05.218 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:05 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:22:05.218 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:05 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:22:05.218 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:05 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:22:05.218 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:05 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:22:05.218 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:05 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:22:05.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:05 smithi169 ceph-mon[34653]: osdmap e38: 6 total, 6 up, 6 in 2024-09-16T00:22:05.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:05 smithi169 ceph-mon[34653]: Detected new or changed devices on smithi169 2024-09-16T00:22:05.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:05 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:22:05.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:05 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:22:05.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:05 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:22:05.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:05 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:22:05.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:05 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:22:05.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:05 smithi169 ceph-mon[34653]: Adjusting osd_memory_target on smithi169 to 5312M 2024-09-16T00:22:05.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:05 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:22:05.278 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:05 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:22:05.278 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:05 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:22:05.278 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:05 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:22:05.278 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:05 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:22:05.278 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:05 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:22:05.278 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:05 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:22:05.278 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:05 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:22:06.233 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:06 smithi112 ceph-mon[27869]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:22:06.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:06 smithi169 ceph-mon[34653]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:22:06.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:06.428+0000 7f0cb4f19640 1 -- 172.21.15.112:0/2518483940 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0cb00fd830 msgr2=0x7f0cb00fdca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:06.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:06.428+0000 7f0cb4f19640 1 --2- 172.21.15.112:0/2518483940 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0cb00fd830 0x7f0cb00fdca0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0c980048d0 tx=0x7f0c98009da0 comp rx=0 tx=0).stop 2024-09-16T00:22:06.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:06.428+0000 7f0cb4f19640 1 -- 172.21.15.112:0/2518483940 shutdown_connections 2024-09-16T00:22:06.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:06.428+0000 7f0cb4f19640 1 --2- 172.21.15.112:0/2518483940 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0cb00fd830 0x7f0cb00fdca0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:06.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:06.428+0000 7f0cb4f19640 1 --2- 172.21.15.112:0/2518483940 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0cb00fc560 0x7f0cb00fc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:06.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:06.428+0000 7f0cb4f19640 1 -- 172.21.15.112:0/2518483940 >> 172.21.15.112:0/2518483940 conn(0x7f0cb00f7d40 msgr2=0x7f0cb00fa150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:06.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:06.428+0000 7f0cb4f19640 1 -- 172.21.15.112:0/2518483940 shutdown_connections 2024-09-16T00:22:06.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:06.428+0000 7f0cb4f19640 1 -- 172.21.15.112:0/2518483940 wait complete. 2024-09-16T00:22:06.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:06.429+0000 7f0cb4f19640 1 Processor -- start 2024-09-16T00:22:06.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:06.429+0000 7f0cb4f19640 1 -- start start 2024-09-16T00:22:06.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:06.429+0000 7f0cb4f19640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0cb00fc560 0x7f0cb0071520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:06.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:06.429+0000 7f0cb4f19640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0cb00fd830 0x7f0cb006fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:06.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:06.429+0000 7f0cb4f19640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb006ffa0 con 0x7f0cb00fd830 2024-09-16T00:22:06.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:06.429+0000 7f0cb4f19640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb00700e0 con 0x7f0cb00fc560 2024-09-16T00:22:06.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:06.429+0000 7f0ca7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0cb00fd830 0x7f0cb006fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:06.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:06.429+0000 7f0ca7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0cb00fd830 0x7f0cb006fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59858/0 (socket says 172.21.15.112:59858) 2024-09-16T00:22:06.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:06.429+0000 7f0ca7fff640 1 -- 172.21.15.112:0/406318474 learned_addr learned my addr 172.21.15.112:0/406318474 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:22:06.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:06.429+0000 7f0cae59c640 1 --2- 172.21.15.112:0/406318474 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0cb00fc560 0x7f0cb0071520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:06.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:06.429+0000 7f0ca7fff640 1 -- 172.21.15.112:0/406318474 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0cb00fc560 msgr2=0x7f0cb0071520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:06.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:06.429+0000 7f0ca7fff640 1 --2- 172.21.15.112:0/406318474 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0cb00fc560 0x7f0cb0071520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:06.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:06.429+0000 7f0ca7fff640 1 -- 172.21.15.112:0/406318474 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c98005040 con 0x7f0cb00fd830 2024-09-16T00:22:06.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:06.430+0000 7f0ca7fff640 1 --2- 172.21.15.112:0/406318474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0cb00fd830 0x7f0cb006fa60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0c98003080 tx=0x7f0c98009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:06.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:06.430+0000 7f0ca6ffd640 1 -- 172.21.15.112:0/406318474 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c98038070 con 0x7f0cb00fd830 2024-09-16T00:22:06.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:06.430+0000 7f0ca6ffd640 1 -- 172.21.15.112:0/406318474 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f0c9802abd0 con 0x7f0cb00fd830 2024-09-16T00:22:06.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:06.430+0000 7f0cb4f19640 1 -- 172.21.15.112:0/406318474 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cb0070360 con 0x7f0cb00fd830 2024-09-16T00:22:06.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:06.430+0000 7f0ca6ffd640 1 -- 172.21.15.112:0/406318474 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c9803e670 con 0x7f0cb00fd830 2024-09-16T00:22:06.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:06.430+0000 7f0cb4f19640 1 -- 172.21.15.112:0/406318474 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cb0115100 con 0x7f0cb00fd830 2024-09-16T00:22:06.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:06.431+0000 7f0cb4f19640 1 -- 172.21.15.112:0/406318474 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cb00fb9c0 con 0x7f0cb00fd830 2024-09-16T00:22:06.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:06.431+0000 7f0ca6ffd640 1 -- 172.21.15.112:0/406318474 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) v1 ==== 90869+0+0 (secure 0 0 0) 0x7f0c98030020 con 0x7f0cb00fd830 2024-09-16T00:22:06.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:06.432+0000 7f0ca6ffd640 1 --2- 172.21.15.112:0/406318474 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f0c9c06ed60 0x7f0c9c071210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:06.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:06.432+0000 7f0ca6ffd640 1 -- 172.21.15.112:0/406318474 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f0c980ae190 con 0x7f0cb00fd830 2024-09-16T00:22:06.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:06.432+0000 7f0cae59c640 1 --2- 172.21.15.112:0/406318474 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f0c9c06ed60 0x7f0c9c071210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:06.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:06.432+0000 7f0cae59c640 1 --2- 172.21.15.112:0/406318474 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f0c9c06ed60 0x7f0c9c071210 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0cb00712d0 tx=0x7f0ca0009040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:06.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:06.436+0000 7f0ca6ffd640 1 -- 172.21.15.112:0/406318474 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f0c9807b670 con 0x7f0cb00fd830 2024-09-16T00:22:06.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:06.591+0000 7f0cb4f19640 1 -- 172.21.15.112:0/406318474 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f0cb0115290 con 0x7f0cb00fd830 2024-09-16T00:22:06.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:06.592+0000 7f0ca6ffd640 1 -- 172.21.15.112:0/406318474 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v38) v1 ==== 74+0+130 (secure 0 0 0) 0x7f0c9807b010 con 0x7f0cb00fd830 2024-09-16T00:22:06.593 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:22:06.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:06.594+0000 7f0cb4f19640 1 -- 172.21.15.112:0/406318474 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f0c9c06ed60 msgr2=0x7f0c9c071210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:06.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:06.594+0000 7f0cb4f19640 1 --2- 172.21.15.112:0/406318474 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f0c9c06ed60 0x7f0c9c071210 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0cb00712d0 tx=0x7f0ca0009040 comp rx=0 tx=0).stop 2024-09-16T00:22:06.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:06.594+0000 7f0cb4f19640 1 -- 172.21.15.112:0/406318474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0cb00fd830 msgr2=0x7f0cb006fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:06.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:06.594+0000 7f0cb4f19640 1 --2- 172.21.15.112:0/406318474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0cb00fd830 0x7f0cb006fa60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0c98003080 tx=0x7f0c98009da0 comp rx=0 tx=0).stop 2024-09-16T00:22:06.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:06.594+0000 7f0cb4f19640 1 -- 172.21.15.112:0/406318474 shutdown_connections 2024-09-16T00:22:06.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:06.594+0000 7f0cb4f19640 1 --2- 172.21.15.112:0/406318474 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f0c9c06ed60 0x7f0c9c071210 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:06.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:06.594+0000 7f0cb4f19640 1 --2- 172.21.15.112:0/406318474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0cb00fd830 0x7f0cb006fa60 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:06.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:06.594+0000 7f0cb4f19640 1 --2- 172.21.15.112:0/406318474 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0cb00fc560 0x7f0cb0071520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:06.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:06.594+0000 7f0cb4f19640 1 -- 172.21.15.112:0/406318474 >> 172.21.15.112:0/406318474 conn(0x7f0cb00f7d40 msgr2=0x7f0cb00f9470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:06.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:06.594+0000 7f0cb4f19640 1 -- 172.21.15.112:0/406318474 shutdown_connections 2024-09-16T00:22:06.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:06.594+0000 7f0cb4f19640 1 -- 172.21.15.112:0/406318474 wait complete. 2024-09-16T00:22:07.274 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:07 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/406318474' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T00:22:07.274 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:07 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:22:07.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:07 smithi169 ceph-mon[34653]: from='client.? 172.21.15.112:0/406318474' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T00:22:07.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:07 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:22:07.299 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":38,"num_osds":6,"num_up_osds":6,"osd_up_since":1726446122,"num_in_osds":6,"osd_in_since":1726446104,"num_remapped_pgs":0} 2024-09-16T00:22:07.300 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T00:22:07.543 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:22:08.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:08 smithi169 ceph-mon[34653]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:22:08.319 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:08 smithi112 ceph-mon[27869]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:22:09.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:09.189+0000 7f3055770640 1 -- 172.21.15.112:0/2880849700 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30500fe460 msgr2=0x7f30500fe830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:09.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:09.189+0000 7f3055770640 1 --2- 172.21.15.112:0/2880849700 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30500fe460 0x7f30500fe830 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3040009c50 tx=0x7f3040004d10 comp rx=0 tx=0).stop 2024-09-16T00:22:09.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:09.189+0000 7f3055770640 1 -- 172.21.15.112:0/2880849700 shutdown_connections 2024-09-16T00:22:09.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:09.189+0000 7f3055770640 1 --2- 172.21.15.112:0/2880849700 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30500fabe0 0x7f30500fb030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:09.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:09.189+0000 7f3055770640 1 --2- 172.21.15.112:0/2880849700 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30500fe460 0x7f30500fe830 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:09.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:09.189+0000 7f3055770640 1 -- 172.21.15.112:0/2880849700 >> 172.21.15.112:0/2880849700 conn(0x7f3050071b00 msgr2=0x7f3050071f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:09.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:09.189+0000 7f3055770640 1 -- 172.21.15.112:0/2880849700 shutdown_connections 2024-09-16T00:22:09.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:09.189+0000 7f3055770640 1 -- 172.21.15.112:0/2880849700 wait complete. 2024-09-16T00:22:09.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:09.190+0000 7f3055770640 1 Processor -- start 2024-09-16T00:22:09.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:09.190+0000 7f3055770640 1 -- start start 2024-09-16T00:22:09.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:09.190+0000 7f3055770640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30500fabe0 0x7f3050113190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:09.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:09.190+0000 7f3055770640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30500fe460 0x7f30501136d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:09.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:09.190+0000 7f3055770640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3050113cd0 con 0x7f30500fabe0 2024-09-16T00:22:09.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:09.190+0000 7f3055770640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f305010d110 con 0x7f30500fe460 2024-09-16T00:22:09.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:09.190+0000 7f304effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30500fabe0 0x7f3050113190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:09.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:09.190+0000 7f304effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30500fabe0 0x7f3050113190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59872/0 (socket says 172.21.15.112:59872) 2024-09-16T00:22:09.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:09.190+0000 7f304effd640 1 -- 172.21.15.112:0/446486753 learned_addr learned my addr 172.21.15.112:0/446486753 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:22:09.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:09.190+0000 7f304e7fc640 1 --2- 172.21.15.112:0/446486753 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30500fe460 0x7f30501136d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:09.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:09.190+0000 7f304effd640 1 -- 172.21.15.112:0/446486753 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30500fe460 msgr2=0x7f30501136d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:09.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:09.190+0000 7f304effd640 1 --2- 172.21.15.112:0/446486753 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30500fe460 0x7f30501136d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:09.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:09.190+0000 7f304effd640 1 -- 172.21.15.112:0/446486753 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30400098b0 con 0x7f30500fabe0 2024-09-16T00:22:09.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:09.191+0000 7f304effd640 1 --2- 172.21.15.112:0/446486753 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30500fabe0 0x7f3050113190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3040004750 tx=0x7f3040004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:09.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:09.191+0000 7f30377fe640 1 -- 172.21.15.112:0/446486753 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3040009610 con 0x7f30500fabe0 2024-09-16T00:22:09.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:09.191+0000 7f3055770640 1 -- 172.21.15.112:0/446486753 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f305010d390 con 0x7f30500fabe0 2024-09-16T00:22:09.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:09.191+0000 7f30377fe640 1 -- 172.21.15.112:0/446486753 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f3040032b10 con 0x7f30500fabe0 2024-09-16T00:22:09.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:09.191+0000 7f30377fe640 1 -- 172.21.15.112:0/446486753 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f304003d990 con 0x7f30500fabe0 2024-09-16T00:22:09.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:09.191+0000 7f3055770640 1 -- 172.21.15.112:0/446486753 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f305010d930 con 0x7f30500fabe0 2024-09-16T00:22:09.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:09.192+0000 7f3055770640 1 -- 172.21.15.112:0/446486753 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3050072610 con 0x7f30500fabe0 2024-09-16T00:22:09.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:09.192+0000 7f30377fe640 1 -- 172.21.15.112:0/446486753 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) v1 ==== 90869+0+0 (secure 0 0 0) 0x7f304002f030 con 0x7f30500fabe0 2024-09-16T00:22:09.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:09.193+0000 7f30377fe640 1 --2- 172.21.15.112:0/446486753 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f303806ed60 0x7f3038071210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:09.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:09.193+0000 7f30377fe640 1 -- 172.21.15.112:0/446486753 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f30400aef00 con 0x7f30500fabe0 2024-09-16T00:22:09.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:09.193+0000 7f304e7fc640 1 --2- 172.21.15.112:0/446486753 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f303806ed60 0x7f3038071210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:09.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:09.193+0000 7f304e7fc640 1 --2- 172.21.15.112:0/446486753 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f303806ed60 0x7f3038071210 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f305010ea50 tx=0x7f304400b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:09.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:09.197+0000 7f30377fe640 1 -- 172.21.15.112:0/446486753 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f304007c360 con 0x7f30500fabe0 2024-09-16T00:22:09.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:09.351+0000 7f3055770640 1 -- 172.21.15.112:0/446486753 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3050064d50 con 0x7f30500fabe0 2024-09-16T00:22:09.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:09.352+0000 7f30377fe640 1 -- 172.21.15.112:0/446486753 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+11130 (secure 0 0 0) 0x7f30400480c0 con 0x7f30500fabe0 2024-09-16T00:22:09.353 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:22:09.353 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":38,"fsid":"d346dd56-73c0-11ef-bceb-c7b262605968","created":"2024-09-16T00:16:43.747157+0000","modified":"2024-09-16T00:22:03.998980+0000","last_up_change":"2024-09-16T00:22:02.988149+0000","last_in_change":"2024-09-16T00:21:44.510744+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:20:50.964734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"f6c0fb9d-a0e6-44b1-a66d-ed3a1955c461","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":2840222594},{"type":"v1","addr":"172.21.15.112:6803","nonce":2840222594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":2840222594},{"type":"v1","addr":"172.21.15.112:6805","nonce":2840222594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2840222594},{"type":"v1","addr":"172.21.15.112:6809","nonce":2840222594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":2840222594},{"type":"v1","addr":"172.21.15.112:6807","nonce":2840222594}]},"public_addr":"172.21.15.112:6803/2840222594","cluster_addr":"172.21.15.112:6805/2840222594","heartbeat_back_addr":"172.21.15.112:6809/2840222594","heartbeat_front_addr":"172.21.15.112:6807/2840222594","state":["exists","up"]},{"osd":1,"uuid":"6fee7bb7-5935-4218-b0b5-ecd103121196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":946524214},{"type":"v1","addr":"172.21.15.112:6811","nonce":946524214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":946524214},{"type":"v1","addr":"172.21.15.112:6813","nonce":946524214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":946524214},{"type":"v1","addr":"172.21.15.112:6817","nonce":946524214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":946524214},{"type":"v1","addr":"172.21.15.112:6815","nonce":946524214}]},"public_addr":"172.21.15.112:6811/946524214","cluster_addr":"172.21.15.112:6813/946524214","heartbeat_back_addr":"172.21.15.112:6817/946524214","heartbeat_front_addr":"172.21.15.112:6815/946524214","state":["exists","up"]},{"osd":2,"uuid":"acf3f3a5-2941-49ed-a653-728564ba08df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3837096766},{"type":"v1","addr":"172.21.15.112:6819","nonce":3837096766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3837096766},{"type":"v1","addr":"172.21.15.112:6821","nonce":3837096766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3837096766},{"type":"v1","addr":"172.21.15.112:6825","nonce":3837096766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3837096766},{"type":"v1","addr":"172.21.15.112:6823","nonce":3837096766}]},"public_addr":"172.21.15.112:6819/3837096766","cluster_addr":"172.21.15.112:6821/3837096766","heartbeat_back_addr":"172.21.15.112:6825/3837096766","heartbeat_front_addr":"172.21.15.112:6823/3837096766","state":["exists","up"]},{"osd":3,"uuid":"f0e8ddbe-ed38-4cd9-af68-a68957311e52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":1327586825},{"type":"v1","addr":"172.21.15.169:6801","nonce":1327586825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":1327586825},{"type":"v1","addr":"172.21.15.169:6803","nonce":1327586825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1327586825},{"type":"v1","addr":"172.21.15.169:6807","nonce":1327586825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":1327586825},{"type":"v1","addr":"172.21.15.169:6805","nonce":1327586825}]},"public_addr":"172.21.15.169:6801/1327586825","cluster_addr":"172.21.15.169:6803/1327586825","heartbeat_back_addr":"172.21.15.169:6807/1327586825","heartbeat_front_addr":"172.21.15.169:6805/1327586825","state":["exists","up"]},{"osd":4,"uuid":"06b021d6-0611-4a51-b49e-108783df2837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3980576686},{"type":"v1","addr":"172.21.15.169:6809","nonce":3980576686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":3980576686},{"type":"v1","addr":"172.21.15.169:6811","nonce":3980576686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":3980576686},{"type":"v1","addr":"172.21.15.169:6815","nonce":3980576686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3980576686},{"type":"v1","addr":"172.21.15.169:6813","nonce":3980576686}]},"public_addr":"172.21.15.169:6809/3980576686","cluster_addr":"172.21.15.169:6811/3980576686","heartbeat_back_addr":"172.21.15.169:6815/3980576686","heartbeat_front_addr":"172.21.15.169:6813/3980576686","state":["exists","up"]},{"osd":5,"uuid":"666b01f8-29fa-427d-a2c6-2ef236d0ddd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":2190808120},{"type":"v1","addr":"172.21.15.169:6817","nonce":2190808120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":2190808120},{"type":"v1","addr":"172.21.15.169:6819","nonce":2190808120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":2190808120},{"type":"v1","addr":"172.21.15.169:6823","nonce":2190808120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":2190808120},{"type":"v1","addr":"172.21.15.169:6821","nonce":2190808120}]},"public_addr":"172.21.15.169:6817/2190808120","cluster_addr":"172.21.15.169:6819/2190808120","heartbeat_back_addr":"172.21.15.169:6823/2190808120","heartbeat_front_addr":"172.21.15.169:6821/2190808120","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:19:52.243994+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:20:20.249290+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:20:48.168672+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:21:10.906749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:21:35.636536+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:22:01.186977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:6801/2021251488":"2024-09-17T00:18:36.660338+0000","172.21.15.112:6801/663349889":"2024-09-17T00:17:05.974495+0000","172.21.15.112:0/3430050847":"2024-09-17T00:18:36.660338+0000","172.21.15.112:6800/2021251488":"2024-09-17T00:18:36.660338+0000","172.21.15.112:6800/663349889":"2024-09-17T00:17:05.974495+0000","172.21.15.112:0/2155316648":"2024-09-17T00:17:05.974495+0000","172.21.15.112:0/1686004085":"2024-09-17T00:17:35.477448+0000","172.21.15.112:0/214318070":"2024-09-17T00:17:05.974495+0000","172.21.15.112:6800/122493397":"2024-09-17T00:17:35.477448+0000","172.21.15.112:0/2081985644":"2024-09-17T00:17:05.974495+0000","172.21.15.112:0/4168804950":"2024-09-17T00:17:35.477448+0000","172.21.15.112:0/3817770007":"2024-09-17T00:18:36.660338+0000","172.21.15.112:0/648744088":"2024-09-17T00:17:35.477448+0000","172.21.15.112:6801/122493397":"2024-09-17T00:17:35.477448+0000","172.21.15.112:0/2126830902":"2024-09-17T00:18:36.660338+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:22:09.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:09.354+0000 7f3055770640 1 -- 172.21.15.112:0/446486753 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f303806ed60 msgr2=0x7f3038071210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:09.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:09.354+0000 7f3055770640 1 --2- 172.21.15.112:0/446486753 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f303806ed60 0x7f3038071210 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f305010ea50 tx=0x7f304400b040 comp rx=0 tx=0).stop 2024-09-16T00:22:09.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:09.354+0000 7f3055770640 1 -- 172.21.15.112:0/446486753 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30500fabe0 msgr2=0x7f3050113190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:09.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:09.354+0000 7f3055770640 1 --2- 172.21.15.112:0/446486753 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30500fabe0 0x7f3050113190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3040004750 tx=0x7f3040004d10 comp rx=0 tx=0).stop 2024-09-16T00:22:09.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:09.354+0000 7f3055770640 1 -- 172.21.15.112:0/446486753 shutdown_connections 2024-09-16T00:22:09.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:09.354+0000 7f3055770640 1 --2- 172.21.15.112:0/446486753 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f303806ed60 0x7f3038071210 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:09.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:09.354+0000 7f3055770640 1 --2- 172.21.15.112:0/446486753 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30500fe460 0x7f30501136d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:09.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:09.354+0000 7f3055770640 1 --2- 172.21.15.112:0/446486753 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30500fabe0 0x7f3050113190 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:09.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:09.354+0000 7f3055770640 1 -- 172.21.15.112:0/446486753 >> 172.21.15.112:0/446486753 conn(0x7f3050071b00 msgr2=0x7f30500f99b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:09.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:09.354+0000 7f3055770640 1 -- 172.21.15.112:0/446486753 shutdown_connections 2024-09-16T00:22:09.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:09.354+0000 7f3055770640 1 -- 172.21.15.112:0/446486753 wait complete. 2024-09-16T00:22:10.102 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T00:20:50.964734+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-09-16T00:22:10.102 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-16T00:22:10.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:10 smithi169 ceph-mon[34653]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:22:10.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:10 smithi169 ceph-mon[34653]: from='client.? 172.21.15.112:0/446486753' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:22:10.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:10 smithi112 ceph-mon[27869]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:22:10.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:10 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/446486753' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:22:10.345 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:22:12.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:12.083+0000 7fc5d7d33640 1 -- 172.21.15.112:0/1126250592 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5d0102ef0 msgr2=0x7fc5d00fc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:12.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:12.083+0000 7fc5d7d33640 1 --2- 172.21.15.112:0/1126250592 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5d0102ef0 0x7fc5d00fc5b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc5c00048d0 tx=0x7fc5c0009da0 comp rx=0 tx=0).stop 2024-09-16T00:22:12.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:12.083+0000 7fc5d7d33640 1 -- 172.21.15.112:0/1126250592 shutdown_connections 2024-09-16T00:22:12.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:12.083+0000 7fc5d7d33640 1 --2- 172.21.15.112:0/1126250592 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5d0102ef0 0x7fc5d00fc5b0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:12.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:12.083+0000 7fc5d7d33640 1 --2- 172.21.15.112:0/1126250592 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5d0102550 0x7fc5d0102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:12.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:12.083+0000 7fc5d7d33640 1 -- 172.21.15.112:0/1126250592 >> 172.21.15.112:0/1126250592 conn(0x7fc5d00f80f0 msgr2=0x7fc5d00fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:12.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:12.083+0000 7fc5d7d33640 1 -- 172.21.15.112:0/1126250592 shutdown_connections 2024-09-16T00:22:12.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:12.083+0000 7fc5d7d33640 1 -- 172.21.15.112:0/1126250592 wait complete. 2024-09-16T00:22:12.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:12.084+0000 7fc5d7d33640 1 Processor -- start 2024-09-16T00:22:12.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:12.084+0000 7fc5d7d33640 1 -- start start 2024-09-16T00:22:12.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:12.084+0000 7fc5d7d33640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5d0102550 0x7fc5d01135b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:12.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:12.084+0000 7fc5d7d33640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5d0102ef0 0x7fc5d0113af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:12.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:12.084+0000 7fc5d7d33640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d01141d0 con 0x7fc5d0102ef0 2024-09-16T00:22:12.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:12.084+0000 7fc5d7d33640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d010d550 con 0x7fc5d0102550 2024-09-16T00:22:12.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:12.084+0000 7fc5d52ce640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5d0102ef0 0x7fc5d0113af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:12.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:12.084+0000 7fc5d52ce640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5d0102ef0 0x7fc5d0113af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59880/0 (socket says 172.21.15.112:59880) 2024-09-16T00:22:12.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:12.084+0000 7fc5d52ce640 1 -- 172.21.15.112:0/4143132187 learned_addr learned my addr 172.21.15.112:0/4143132187 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:22:12.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:12.084+0000 7fc5d5acf640 1 --2- 172.21.15.112:0/4143132187 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5d0102550 0x7fc5d01135b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:12.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:12.084+0000 7fc5d52ce640 1 -- 172.21.15.112:0/4143132187 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5d0102550 msgr2=0x7fc5d01135b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:12.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:12.084+0000 7fc5d52ce640 1 --2- 172.21.15.112:0/4143132187 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5d0102550 0x7fc5d01135b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:12.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:12.084+0000 7fc5d52ce640 1 -- 172.21.15.112:0/4143132187 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5c0005040 con 0x7fc5d0102ef0 2024-09-16T00:22:12.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:12.084+0000 7fc5d5acf640 1 --2- 172.21.15.112:0/4143132187 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5d0102550 0x7fc5d01135b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:22:12.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:12.084+0000 7fc5d52ce640 1 --2- 172.21.15.112:0/4143132187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5d0102ef0 0x7fc5d0113af0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc5c0003080 tx=0x7fc5c0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:12.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:12.085+0000 7fc5c67fc640 1 -- 172.21.15.112:0/4143132187 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc5c0038070 con 0x7fc5d0102ef0 2024-09-16T00:22:12.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:12.085+0000 7fc5c67fc640 1 -- 172.21.15.112:0/4143132187 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fc5c0009760 con 0x7fc5d0102ef0 2024-09-16T00:22:12.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:12.085+0000 7fc5c67fc640 1 -- 172.21.15.112:0/4143132187 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc5c0033420 con 0x7fc5d0102ef0 2024-09-16T00:22:12.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:12.085+0000 7fc5d7d33640 1 -- 172.21.15.112:0/4143132187 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5d010d780 con 0x7fc5d0102ef0 2024-09-16T00:22:12.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:12.085+0000 7fc5d7d33640 1 -- 172.21.15.112:0/4143132187 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5d010dbe0 con 0x7fc5d0102ef0 2024-09-16T00:22:12.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:12.086+0000 7fc5d7d33640 1 -- 172.21.15.112:0/4143132187 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5d00fdcd0 con 0x7fc5d0102ef0 2024-09-16T00:22:12.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:12.086+0000 7fc5c67fc640 1 -- 172.21.15.112:0/4143132187 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) v1 ==== 90869+0+0 (secure 0 0 0) 0x7fc5c0030020 con 0x7fc5d0102ef0 2024-09-16T00:22:12.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:12.087+0000 7fc5c67fc640 1 --2- 172.21.15.112:0/4143132187 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fc5bc06ed60 0x7fc5bc071210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:12.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:12.087+0000 7fc5c67fc640 1 -- 172.21.15.112:0/4143132187 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fc5c00adb70 con 0x7fc5d0102ef0 2024-09-16T00:22:12.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:12.087+0000 7fc5d5acf640 1 --2- 172.21.15.112:0/4143132187 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fc5bc06ed60 0x7fc5bc071210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:12.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:12.087+0000 7fc5d5acf640 1 --2- 172.21.15.112:0/4143132187 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fc5bc06ed60 0x7fc5bc071210 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc009a80 tx=0x7fc5cc008040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:12.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:12.091+0000 7fc5c67fc640 1 -- 172.21.15.112:0/4143132187 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fc5c007afd0 con 0x7fc5d0102ef0 2024-09-16T00:22:12.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:12.243+0000 7fc5d7d33640 1 -- 172.21.15.112:0/4143132187 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc5d004b720 con 0x7fc5d0102ef0 2024-09-16T00:22:12.244 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:12 smithi112 ceph-mon[27869]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:22:12.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:12.244+0000 7fc5c67fc640 1 -- 172.21.15.112:0/4143132187 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v38) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc5c007a970 con 0x7fc5d0102ef0 2024-09-16T00:22:12.244 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-09-16T00:22:12.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:12.246+0000 7fc5d7d33640 1 -- 172.21.15.112:0/4143132187 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fc5bc06ed60 msgr2=0x7fc5bc071210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:12.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:12.246+0000 7fc5d7d33640 1 --2- 172.21.15.112:0/4143132187 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fc5bc06ed60 0x7fc5bc071210 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc009a80 tx=0x7fc5cc008040 comp rx=0 tx=0).stop 2024-09-16T00:22:12.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:12.246+0000 7fc5d7d33640 1 -- 172.21.15.112:0/4143132187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5d0102ef0 msgr2=0x7fc5d0113af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:12.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:12.246+0000 7fc5d7d33640 1 --2- 172.21.15.112:0/4143132187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5d0102ef0 0x7fc5d0113af0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc5c0003080 tx=0x7fc5c0009da0 comp rx=0 tx=0).stop 2024-09-16T00:22:12.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:12.246+0000 7fc5d7d33640 1 -- 172.21.15.112:0/4143132187 shutdown_connections 2024-09-16T00:22:12.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:12.246+0000 7fc5d7d33640 1 --2- 172.21.15.112:0/4143132187 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fc5bc06ed60 0x7fc5bc071210 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:12.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:12.246+0000 7fc5d7d33640 1 --2- 172.21.15.112:0/4143132187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5d0102ef0 0x7fc5d0113af0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:12.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:12.246+0000 7fc5d7d33640 1 --2- 172.21.15.112:0/4143132187 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5d0102550 0x7fc5d01135b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:12.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:12.246+0000 7fc5d7d33640 1 -- 172.21.15.112:0/4143132187 >> 172.21.15.112:0/4143132187 conn(0x7fc5d00f80f0 msgr2=0x7fc5d00f91a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:12.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:12.246+0000 7fc5d7d33640 1 -- 172.21.15.112:0/4143132187 shutdown_connections 2024-09-16T00:22:12.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:12.247+0000 7fc5d7d33640 1 -- 172.21.15.112:0/4143132187 wait complete. 2024-09-16T00:22:12.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:12 smithi169 ceph-mon[34653]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:22:13.009 INFO:tasks.cephadm:Setting up client nodes... 2024-09-16T00:22:13.009 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-16T00:22:13.256 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:22:13.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:13 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/4143132187' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T00:22:13.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:13 smithi169 ceph-mon[34653]: from='client.? 172.21.15.112:0/4143132187' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T00:22:14.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:14 smithi112 ceph-mon[27869]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:22:14.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:14 smithi169 ceph-mon[34653]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:22:14.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:14.729+0000 7f050408d640 1 -- 172.21.15.112:0/4036135457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04fc065bc0 msgr2=0x7f04fc0ff1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:14.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:14.729+0000 7f050408d640 1 --2- 172.21.15.112:0/4036135457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04fc065bc0 0x7f04fc0ff1c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f04f80048d0 tx=0x7f04f8009da0 comp rx=0 tx=0).stop 2024-09-16T00:22:14.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:14.729+0000 7f050408d640 1 -- 172.21.15.112:0/4036135457 shutdown_connections 2024-09-16T00:22:14.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:14.729+0000 7f050408d640 1 --2- 172.21.15.112:0/4036135457 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04fc0ff700 0x7f04fc101ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:14.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:14.729+0000 7f050408d640 1 --2- 172.21.15.112:0/4036135457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04fc065bc0 0x7f04fc0ff1c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:14.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:14.729+0000 7f050408d640 1 -- 172.21.15.112:0/4036135457 >> 172.21.15.112:0/4036135457 conn(0x7f04fc0f6bf0 msgr2=0x7f04fc0f9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:14.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:14.729+0000 7f050408d640 1 -- 172.21.15.112:0/4036135457 shutdown_connections 2024-09-16T00:22:14.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:14.729+0000 7f050408d640 1 -- 172.21.15.112:0/4036135457 wait complete. 2024-09-16T00:22:14.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:14.729+0000 7f050408d640 1 Processor -- start 2024-09-16T00:22:14.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:14.729+0000 7f050408d640 1 -- start start 2024-09-16T00:22:14.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:14.729+0000 7f050408d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04fc065bc0 0x7f04fc10d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:14.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:14.730+0000 7f050408d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04fc0ff700 0x7f04fc10d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:14.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:14.730+0000 7f050408d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04fc10dc30 con 0x7f04fc065bc0 2024-09-16T00:22:14.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:14.730+0000 7f050408d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04fc10e450 con 0x7f04fc0ff700 2024-09-16T00:22:14.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:14.730+0000 7f0501e29640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04fc065bc0 0x7f04fc10d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:14.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:14.730+0000 7f0501e29640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04fc065bc0 0x7f04fc10d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55830/0 (socket says 172.21.15.112:55830) 2024-09-16T00:22:14.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:14.730+0000 7f0501e29640 1 -- 172.21.15.112:0/3981405191 learned_addr learned my addr 172.21.15.112:0/3981405191 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:22:14.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:14.730+0000 7f0501628640 1 --2- 172.21.15.112:0/3981405191 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04fc0ff700 0x7f04fc10d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:14.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:14.730+0000 7f0501e29640 1 -- 172.21.15.112:0/3981405191 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04fc0ff700 msgr2=0x7f04fc10d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:14.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:14.730+0000 7f0501e29640 1 --2- 172.21.15.112:0/3981405191 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04fc0ff700 0x7f04fc10d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:14.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:14.730+0000 7f0501e29640 1 -- 172.21.15.112:0/3981405191 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04f8005040 con 0x7f04fc065bc0 2024-09-16T00:22:14.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:14.730+0000 7f0501e29640 1 --2- 172.21.15.112:0/3981405191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04fc065bc0 0x7f04fc10d0a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f04f80043d0 tx=0x7f04f8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:14.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:14.730+0000 7f04f27fc640 1 -- 172.21.15.112:0/3981405191 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f04f8038070 con 0x7f04fc065bc0 2024-09-16T00:22:14.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:14.730+0000 7f050408d640 1 -- 172.21.15.112:0/3981405191 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04fc10e680 con 0x7f04fc065bc0 2024-09-16T00:22:14.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:14.730+0000 7f04f27fc640 1 -- 172.21.15.112:0/3981405191 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f04f802ae90 con 0x7f04fc065bc0 2024-09-16T00:22:14.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:14.730+0000 7f04f27fc640 1 -- 172.21.15.112:0/3981405191 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f04f803e670 con 0x7f04fc065bc0 2024-09-16T00:22:14.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:14.730+0000 7f050408d640 1 -- 172.21.15.112:0/3981405191 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04fc10ec20 con 0x7f04fc065bc0 2024-09-16T00:22:14.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:14.731+0000 7f050408d640 1 -- 172.21.15.112:0/3981405191 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04fc0fa890 con 0x7f04fc065bc0 2024-09-16T00:22:14.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:14.732+0000 7f04f27fc640 1 -- 172.21.15.112:0/3981405191 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) v1 ==== 90869+0+0 (secure 0 0 0) 0x7f04f8030020 con 0x7f04fc065bc0 2024-09-16T00:22:14.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:14.732+0000 7f04f27fc640 1 --2- 172.21.15.112:0/3981405191 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f04e806ed60 0x7f04e8071210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:14.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:14.732+0000 7f04f27fc640 1 -- 172.21.15.112:0/3981405191 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f04f80adf80 con 0x7f04fc065bc0 2024-09-16T00:22:14.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:14.732+0000 7f0501628640 1 --2- 172.21.15.112:0/3981405191 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f04e806ed60 0x7f04e8071210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:14.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:14.733+0000 7f0501628640 1 --2- 172.21.15.112:0/3981405191 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f04e806ed60 0x7f04e8071210 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f04ec007fd0 tx=0x7f04ec009e90 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:14.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:14.737+0000 7f04f27fc640 1 -- 172.21.15.112:0/3981405191 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f04f807b460 con 0x7f04fc065bc0 2024-09-16T00:22:14.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:14.970+0000 7f050408d640 1 -- 172.21.15.112:0/3981405191 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f04fc04b720 con 0x7f04fc065bc0 2024-09-16T00:22:14.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:14.974+0000 7f04f27fc640 1 -- 172.21.15.112:0/3981405191 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v14) v1 ==== 170+0+59 (secure 0 0 0) 0x7f04f807ae00 con 0x7f04fc065bc0 2024-09-16T00:22:14.975 INFO:teuthology.orchestra.run.smithi112.stdout:[client.0] 2024-09-16T00:22:14.975 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQA2eudm2WfwORAAz7FabKZtU0Ge6knpDaonKQ== 2024-09-16T00:22:14.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:14.976+0000 7f050408d640 1 -- 172.21.15.112:0/3981405191 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f04e806ed60 msgr2=0x7f04e8071210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:14.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:14.976+0000 7f050408d640 1 --2- 172.21.15.112:0/3981405191 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f04e806ed60 0x7f04e8071210 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f04ec007fd0 tx=0x7f04ec009e90 comp rx=0 tx=0).stop 2024-09-16T00:22:14.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:14.976+0000 7f050408d640 1 -- 172.21.15.112:0/3981405191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04fc065bc0 msgr2=0x7f04fc10d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:14.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:14.976+0000 7f050408d640 1 --2- 172.21.15.112:0/3981405191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04fc065bc0 0x7f04fc10d0a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f04f80043d0 tx=0x7f04f8009da0 comp rx=0 tx=0).stop 2024-09-16T00:22:14.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:14.977+0000 7f050408d640 1 -- 172.21.15.112:0/3981405191 shutdown_connections 2024-09-16T00:22:14.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:14.977+0000 7f050408d640 1 --2- 172.21.15.112:0/3981405191 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f04e806ed60 0x7f04e8071210 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:14.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:14.977+0000 7f050408d640 1 --2- 172.21.15.112:0/3981405191 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04fc0ff700 0x7f04fc10d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:14.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:14.977+0000 7f050408d640 1 --2- 172.21.15.112:0/3981405191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04fc065bc0 0x7f04fc10d0a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:14.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:14.977+0000 7f050408d640 1 -- 172.21.15.112:0/3981405191 >> 172.21.15.112:0/3981405191 conn(0x7f04fc0f6bf0 msgr2=0x7f04fc0f7010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:14.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:14.977+0000 7f050408d640 1 -- 172.21.15.112:0/3981405191 shutdown_connections 2024-09-16T00:22:14.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:14.977+0000 7f050408d640 1 -- 172.21.15.112:0/3981405191 wait complete. 2024-09-16T00:22:15.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:15 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/3981405191' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T00:22:15.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:15 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/3981405191' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-16T00:22:15.428 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:15 smithi169 ceph-mon[34653]: from='client.? 172.21.15.112:0/3981405191' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T00:22:15.428 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:15 smithi169 ceph-mon[34653]: from='client.? 172.21.15.112:0/3981405191' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-16T00:22:15.731 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-16T00:22:15.731 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-16T00:22:15.731 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-16T00:22:15.769 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-16T00:22:16.032 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi169/config 2024-09-16T00:22:16.338 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:16 smithi169 ceph-mon[34653]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:22:16.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:16 smithi112 ceph-mon[27869]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:22:17.498 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:22:17.496+0000 7fd9e2975640 1 -- 172.21.15.169:0/2923432728 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9dc0fc3b0 msgr2=0x7fd9dc0fc820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:17.498 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:22:17.496+0000 7fd9e2975640 1 --2- 172.21.15.169:0/2923432728 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9dc0fc3b0 0x7fd9dc0fc820 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd9d00048d0 tx=0x7fd9d0009da0 comp rx=0 tx=0).stop 2024-09-16T00:22:17.498 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:22:17.496+0000 7fd9e2975640 1 -- 172.21.15.169:0/2923432728 shutdown_connections 2024-09-16T00:22:17.498 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:22:17.496+0000 7fd9e2975640 1 --2- 172.21.15.169:0/2923432728 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9dc0fc3b0 0x7fd9dc0fc820 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:17.498 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:22:17.496+0000 7fd9e2975640 1 --2- 172.21.15.169:0/2923432728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9dc0fba60 0x7fd9dc0fbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:17.498 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:22:17.496+0000 7fd9e2975640 1 -- 172.21.15.169:0/2923432728 >> 172.21.15.169:0/2923432728 conn(0x7fd9dc0f6370 msgr2=0x7fd9dc0f87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:17.498 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:22:17.496+0000 7fd9e2975640 1 -- 172.21.15.169:0/2923432728 shutdown_connections 2024-09-16T00:22:17.498 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:22:17.496+0000 7fd9e2975640 1 -- 172.21.15.169:0/2923432728 wait complete. 2024-09-16T00:22:17.498 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:22:17.497+0000 7fd9e2975640 1 Processor -- start 2024-09-16T00:22:17.499 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:22:17.497+0000 7fd9e2975640 1 -- start start 2024-09-16T00:22:17.499 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:22:17.497+0000 7fd9e2975640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9dc0fba60 0x7fd9dc10c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:17.499 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:22:17.497+0000 7fd9e2975640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9dc0fc3b0 0x7fd9dc10c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:17.499 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:22:17.497+0000 7fd9e2975640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9dc10d3c0 con 0x7fd9dc0fba60 2024-09-16T00:22:17.499 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:22:17.497+0000 7fd9e2975640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9dc10d530 con 0x7fd9dc0fc3b0 2024-09-16T00:22:17.499 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:22:17.498+0000 7fd9db7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9dc0fc3b0 0x7fd9dc10c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:17.499 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:22:17.498+0000 7fd9db7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9dc0fc3b0 0x7fd9dc10c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:35992/0 (socket says 172.21.15.169:35992) 2024-09-16T00:22:17.499 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:22:17.498+0000 7fd9db7fe640 1 -- 172.21.15.169:0/2338886657 learned_addr learned my addr 172.21.15.169:0/2338886657 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-09-16T00:22:17.500 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:22:17.498+0000 7fd9db7fe640 1 -- 172.21.15.169:0/2338886657 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9dc0fba60 msgr2=0x7fd9dc10c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:17.501 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:22:17.498+0000 7fd9db7fe640 1 --2- 172.21.15.169:0/2338886657 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9dc0fba60 0x7fd9dc10c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:17.501 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:22:17.498+0000 7fd9db7fe640 1 -- 172.21.15.169:0/2338886657 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d0005040 con 0x7fd9dc0fc3b0 2024-09-16T00:22:17.501 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:22:17.498+0000 7fd9db7fe640 1 --2- 172.21.15.169:0/2338886657 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9dc0fc3b0 0x7fd9dc10c5a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd9d0003080 tx=0x7fd9d0009da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:17.501 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:22:17.498+0000 7fd9d8ff9640 1 -- 172.21.15.169:0/2338886657 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd9d0038070 con 0x7fd9dc0fc3b0 2024-09-16T00:22:17.501 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:22:17.498+0000 7fd9d8ff9640 1 -- 172.21.15.169:0/2338886657 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fd9d002abd0 con 0x7fd9dc0fc3b0 2024-09-16T00:22:17.501 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:22:17.498+0000 7fd9e2975640 1 -- 172.21.15.169:0/2338886657 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9dc10d7b0 con 0x7fd9dc0fc3b0 2024-09-16T00:22:17.501 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:22:17.498+0000 7fd9d8ff9640 1 -- 172.21.15.169:0/2338886657 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd9d003e670 con 0x7fd9dc0fc3b0 2024-09-16T00:22:17.501 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:22:17.498+0000 7fd9e2975640 1 -- 172.21.15.169:0/2338886657 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9dc10dcd0 con 0x7fd9dc0fc3b0 2024-09-16T00:22:17.501 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:22:17.499+0000 7fd9e2975640 1 -- 172.21.15.169:0/2338886657 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9dc0fa050 con 0x7fd9dc0fc3b0 2024-09-16T00:22:17.501 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:22:17.500+0000 7fd9d8ff9640 1 -- 172.21.15.169:0/2338886657 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 19) v1 ==== 90869+0+0 (secure 0 0 0) 0x7fd9d0030020 con 0x7fd9dc0fc3b0 2024-09-16T00:22:17.501 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:22:17.500+0000 7fd9d8ff9640 1 --2- 172.21.15.169:0/2338886657 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fd9c406ed60 0x7fd9c4071210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:17.501 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:22:17.500+0000 7fd9d8ff9640 1 -- 172.21.15.169:0/2338886657 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fd9d00ae250 con 0x7fd9dc0fc3b0 2024-09-16T00:22:17.502 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:22:17.500+0000 7fd9dbfff640 1 --2- 172.21.15.169:0/2338886657 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fd9c406ed60 0x7fd9c4071210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:17.502 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:22:17.501+0000 7fd9dbfff640 1 --2- 172.21.15.169:0/2338886657 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fd9c406ed60 0x7fd9c4071210 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc065810 tx=0x7fd9cc006d20 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:17.504 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:22:17.504+0000 7fd9d8ff9640 1 -- 172.21.15.169:0/2338886657 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fd9d007b6b0 con 0x7fd9dc0fc3b0 2024-09-16T00:22:17.720 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:22:17.720+0000 7fd9e2975640 1 -- 172.21.15.169:0/2338886657 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fd9dc04b720 con 0x7fd9dc0fc3b0 2024-09-16T00:22:17.725 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:22:17.725+0000 7fd9d8ff9640 1 -- 172.21.15.169:0/2338886657 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v15) v1 ==== 170+0+59 (secure 0 0 0) 0x7fd9d007b050 con 0x7fd9dc0fc3b0 2024-09-16T00:22:17.725 INFO:teuthology.orchestra.run.smithi169.stdout:[client.1] 2024-09-16T00:22:17.725 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQA5eudmT1kFKxAAw1LxxrRmsO93jk5T3IhtUQ== 2024-09-16T00:22:17.727 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:22:17.727+0000 7fd9e2975640 1 -- 172.21.15.169:0/2338886657 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fd9c406ed60 msgr2=0x7fd9c4071210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:17.727 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:22:17.727+0000 7fd9e2975640 1 --2- 172.21.15.169:0/2338886657 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fd9c406ed60 0x7fd9c4071210 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc065810 tx=0x7fd9cc006d20 comp rx=0 tx=0).stop 2024-09-16T00:22:17.727 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:22:17.727+0000 7fd9e2975640 1 -- 172.21.15.169:0/2338886657 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9dc0fc3b0 msgr2=0x7fd9dc10c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:17.727 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:22:17.727+0000 7fd9e2975640 1 --2- 172.21.15.169:0/2338886657 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9dc0fc3b0 0x7fd9dc10c5a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd9d0003080 tx=0x7fd9d0009da0 comp rx=0 tx=0).stop 2024-09-16T00:22:17.727 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:22:17.727+0000 7fd9e2975640 1 -- 172.21.15.169:0/2338886657 shutdown_connections 2024-09-16T00:22:17.728 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:22:17.727+0000 7fd9e2975640 1 --2- 172.21.15.169:0/2338886657 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fd9c406ed60 0x7fd9c4071210 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:17.728 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:22:17.727+0000 7fd9e2975640 1 --2- 172.21.15.169:0/2338886657 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9dc0fc3b0 0x7fd9dc10c5a0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:17.728 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:22:17.727+0000 7fd9e2975640 1 --2- 172.21.15.169:0/2338886657 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9dc0fba60 0x7fd9dc10c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:17.728 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:22:17.727+0000 7fd9e2975640 1 -- 172.21.15.169:0/2338886657 >> 172.21.15.169:0/2338886657 conn(0x7fd9dc0f6370 msgr2=0x7fd9dc0f7b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:17.728 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:22:17.727+0000 7fd9e2975640 1 -- 172.21.15.169:0/2338886657 shutdown_connections 2024-09-16T00:22:17.728 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:22:17.727+0000 7fd9e2975640 1 -- 172.21.15.169:0/2338886657 wait complete. 2024-09-16T00:22:18.314 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:18 smithi169 ceph-mon[34653]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:22:18.314 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:18 smithi169 ceph-mon[34653]: from='client.? 172.21.15.169:0/2338886657' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T00:22:18.314 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:18 smithi169 ceph-mon[34653]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T00:22:18.314 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:18 smithi169 ceph-mon[34653]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-16T00:22:18.315 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-09-16T00:22:18.315 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-09-16T00:22:18.315 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-16T00:22:18.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:18 smithi112 ceph-mon[27869]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:22:18.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:18 smithi112 ceph-mon[27869]: from='client.? 172.21.15.169:0/2338886657' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T00:22:18.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:18 smithi112 ceph-mon[27869]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T00:22:18.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:18 smithi112 ceph-mon[27869]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-16T00:22:18.353 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-16T00:22:18.353 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-16T00:22:18.354 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph mgr dump --format=json 2024-09-16T00:22:18.611 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:22:20.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:20 smithi112 ceph-mon[27869]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:22:20.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:20.394+0000 7fdd0c502640 1 -- 172.21.15.112:0/4193215323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd04102ed0 msgr2=0x7fdd040fc590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:20.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:20.394+0000 7fdd0c502640 1 --2- 172.21.15.112:0/4193215323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd04102ed0 0x7fdd040fc590 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdcf80048d0 tx=0x7fdcf8009da0 comp rx=0 tx=0).stop 2024-09-16T00:22:20.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:20.394+0000 7fdd0c502640 1 -- 172.21.15.112:0/4193215323 shutdown_connections 2024-09-16T00:22:20.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:20.394+0000 7fdd0c502640 1 --2- 172.21.15.112:0/4193215323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd04102ed0 0x7fdd040fc590 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:20.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:20.394+0000 7fdd0c502640 1 --2- 172.21.15.112:0/4193215323 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd04102530 0x7fdd04102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:20.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:20.394+0000 7fdd0c502640 1 -- 172.21.15.112:0/4193215323 >> 172.21.15.112:0/4193215323 conn(0x7fdd040f80f0 msgr2=0x7fdd040fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:20.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:20.394+0000 7fdd0c502640 1 -- 172.21.15.112:0/4193215323 shutdown_connections 2024-09-16T00:22:20.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:20.394+0000 7fdd0c502640 1 -- 172.21.15.112:0/4193215323 wait complete. 2024-09-16T00:22:20.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:20.394+0000 7fdd0c502640 1 Processor -- start 2024-09-16T00:22:20.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:20.395+0000 7fdd0c502640 1 -- start start 2024-09-16T00:22:20.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:20.395+0000 7fdd0c502640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd04102530 0x7fdd041131f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:20.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:20.395+0000 7fdd0c502640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd04102ed0 0x7fdd04113730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:20.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:20.395+0000 7fdd0c502640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd04113d30 con 0x7fdd04102ed0 2024-09-16T00:22:20.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:20.395+0000 7fdd0c502640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0410d170 con 0x7fdd04102530 2024-09-16T00:22:20.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:20.395+0000 7fdd09a9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd04102ed0 0x7fdd04113730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:20.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:20.395+0000 7fdd09a9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd04102ed0 0x7fdd04113730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55852/0 (socket says 172.21.15.112:55852) 2024-09-16T00:22:20.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:20.395+0000 7fdd09a9d640 1 -- 172.21.15.112:0/3987676957 learned_addr learned my addr 172.21.15.112:0/3987676957 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:22:20.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:20.395+0000 7fdd0a29e640 1 --2- 172.21.15.112:0/3987676957 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd04102530 0x7fdd041131f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:20.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:20.395+0000 7fdd09a9d640 1 -- 172.21.15.112:0/3987676957 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd04102530 msgr2=0x7fdd041131f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:20.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:20.395+0000 7fdd09a9d640 1 --2- 172.21.15.112:0/3987676957 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd04102530 0x7fdd041131f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:20.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:20.395+0000 7fdd09a9d640 1 -- 172.21.15.112:0/3987676957 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcf8005040 con 0x7fdd04102ed0 2024-09-16T00:22:20.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:20.395+0000 7fdd09a9d640 1 --2- 172.21.15.112:0/3987676957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd04102ed0 0x7fdd04113730 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fdcf8003080 tx=0x7fdcf8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:20.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:20.395+0000 7fdd02ffd640 1 -- 172.21.15.112:0/3987676957 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdcf8038070 con 0x7fdd04102ed0 2024-09-16T00:22:20.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:20.395+0000 7fdd0c502640 1 -- 172.21.15.112:0/3987676957 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd0410d3f0 con 0x7fdd04102ed0 2024-09-16T00:22:20.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:20.396+0000 7fdd02ffd640 1 -- 172.21.15.112:0/3987676957 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fdcf8009760 con 0x7fdd04102ed0 2024-09-16T00:22:20.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:20.396+0000 7fdd02ffd640 1 -- 172.21.15.112:0/3987676957 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdcf8033420 con 0x7fdd04102ed0 2024-09-16T00:22:20.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:20.396+0000 7fdd0c502640 1 -- 172.21.15.112:0/3987676957 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd0410d8d0 con 0x7fdd04102ed0 2024-09-16T00:22:20.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:20.397+0000 7fdd0c502640 1 -- 172.21.15.112:0/3987676957 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd040fdcb0 con 0x7fdd04102ed0 2024-09-16T00:22:20.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:20.397+0000 7fdd02ffd640 1 -- 172.21.15.112:0/3987676957 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) v1 ==== 90869+0+0 (secure 0 0 0) 0x7fdcf8030020 con 0x7fdd04102ed0 2024-09-16T00:22:20.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:20.397+0000 7fdd02ffd640 1 --2- 172.21.15.112:0/3987676957 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fdcec06ed60 0x7fdcec071210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:20.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:20.398+0000 7fdd02ffd640 1 -- 172.21.15.112:0/3987676957 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fdcf80adb00 con 0x7fdd04102ed0 2024-09-16T00:22:20.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:20.398+0000 7fdd0a29e640 1 --2- 172.21.15.112:0/3987676957 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fdcec06ed60 0x7fdcec071210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:20.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:20.398+0000 7fdd0a29e640 1 --2- 172.21.15.112:0/3987676957 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fdcec06ed60 0x7fdcec071210 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fdcf4004550 tx=0x7fdcf4009040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:20.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:20.402+0000 7fdd02ffd640 1 -- 172.21.15.112:0/3987676957 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fdcf807af60 con 0x7fdd04102ed0 2024-09-16T00:22:20.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:20 smithi169 ceph-mon[34653]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:22:20.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:20.599+0000 7fdd0c502640 1 -- 172.21.15.112:0/3987676957 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fdd041095f0 con 0x7fdd04102ed0 2024-09-16T00:22:20.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:20.602+0000 7fdd02ffd640 1 -- 172.21.15.112:0/3987676957 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v19) v1 ==== 74+0+176722 (secure 0 0 0) 0x7fdcf807a900 con 0x7fdd04102ed0 2024-09-16T00:22:20.604 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:22:20.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:20.605+0000 7fdd0c502640 1 -- 172.21.15.112:0/3987676957 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fdcec06ed60 msgr2=0x7fdcec071210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:20.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:20.605+0000 7fdd0c502640 1 --2- 172.21.15.112:0/3987676957 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fdcec06ed60 0x7fdcec071210 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fdcf4004550 tx=0x7fdcf4009040 comp rx=0 tx=0).stop 2024-09-16T00:22:20.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:20.605+0000 7fdd0c502640 1 -- 172.21.15.112:0/3987676957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd04102ed0 msgr2=0x7fdd04113730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:20.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:20.605+0000 7fdd0c502640 1 --2- 172.21.15.112:0/3987676957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd04102ed0 0x7fdd04113730 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fdcf8003080 tx=0x7fdcf8009da0 comp rx=0 tx=0).stop 2024-09-16T00:22:20.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:20.605+0000 7fdd0c502640 1 -- 172.21.15.112:0/3987676957 shutdown_connections 2024-09-16T00:22:20.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:20.605+0000 7fdd0c502640 1 --2- 172.21.15.112:0/3987676957 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fdcec06ed60 0x7fdcec071210 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:20.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:20.605+0000 7fdd0c502640 1 --2- 172.21.15.112:0/3987676957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd04102ed0 0x7fdd04113730 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:20.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:20.605+0000 7fdd0c502640 1 --2- 172.21.15.112:0/3987676957 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd04102530 0x7fdd041131f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:20.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:20.605+0000 7fdd0c502640 1 -- 172.21.15.112:0/3987676957 >> 172.21.15.112:0/3987676957 conn(0x7fdd040f80f0 msgr2=0x7fdd040f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:20.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:20.605+0000 7fdd0c502640 1 -- 172.21.15.112:0/3987676957 shutdown_connections 2024-09-16T00:22:20.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:20.605+0000 7fdd0c502640 1 -- 172.21.15.112:0/3987676957 wait complete. 2024-09-16T00:22:21.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:21 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/3987676957' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-16T00:22:21.385 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":19,"flags":0,"active_gid":14208,"active_name":"smithi112.xrkkbc","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":2480414866},{"type":"v1","addr":"172.21.15.112:6801","nonce":2480414866}]},"active_addr":"172.21.15.112:6801/2480414866","active_change":"2024-09-16T00:18:36.660446+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":14228,"name":"smithi169.mtrobk","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.112:8443/","prometheus":"http://172.21.15.112:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.112:0","nonce":2455516344}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.112:0","nonce":3758162203}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.112:0","nonce":3031370105}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.112:0","nonce":4017814212}]}]} 2024-09-16T00:22:21.391 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-16T00:22:21.391 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-16T00:22:21.391 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T00:22:21.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:21 smithi169 ceph-mon[34653]: from='client.? 172.21.15.112:0/3987676957' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-16T00:22:21.639 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:22:22.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:22 smithi112 ceph-mon[27869]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:22:22.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:22 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:22:22.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:22 smithi169 ceph-mon[34653]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:22:22.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:22 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:22:23.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:23.270+0000 7fc9baeac640 1 -- 172.21.15.112:0/3651658323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9b40fe450 msgr2=0x7fc9b40fe820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:23.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:23.270+0000 7fc9baeac640 1 --2- 172.21.15.112:0/3651658323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9b40fe450 0x7fc9b40fe820 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc9a80048d0 tx=0x7fc9a8009da0 comp rx=0 tx=0).stop 2024-09-16T00:22:23.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:23.270+0000 7fc9baeac640 1 -- 172.21.15.112:0/3651658323 shutdown_connections 2024-09-16T00:22:23.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:23.270+0000 7fc9baeac640 1 --2- 172.21.15.112:0/3651658323 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9b4064fe0 0x7fc9b4065430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:23.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:23.270+0000 7fc9baeac640 1 --2- 172.21.15.112:0/3651658323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9b40fe450 0x7fc9b40fe820 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:23.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:23.270+0000 7fc9baeac640 1 -- 172.21.15.112:0/3651658323 >> 172.21.15.112:0/3651658323 conn(0x7fc9b40723b0 msgr2=0x7fc9b40727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:23.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:23.270+0000 7fc9baeac640 1 -- 172.21.15.112:0/3651658323 shutdown_connections 2024-09-16T00:22:23.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:23.270+0000 7fc9baeac640 1 -- 172.21.15.112:0/3651658323 wait complete. 2024-09-16T00:22:23.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:23.271+0000 7fc9baeac640 1 Processor -- start 2024-09-16T00:22:23.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:23.271+0000 7fc9baeac640 1 -- start start 2024-09-16T00:22:23.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:23.271+0000 7fc9baeac640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9b4064fe0 0x7fc9b4113250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:23.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:23.271+0000 7fc9baeac640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9b40fe450 0x7fc9b4113790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:23.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:23.271+0000 7fc9baeac640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b4113e20 con 0x7fc9b4064fe0 2024-09-16T00:22:23.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:23.271+0000 7fc9baeac640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b410d1d0 con 0x7fc9b40fe450 2024-09-16T00:22:23.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:23.271+0000 7fc9b8c48640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9b4064fe0 0x7fc9b4113250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:23.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:23.271+0000 7fc9b8c48640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9b4064fe0 0x7fc9b4113250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39164/0 (socket says 172.21.15.112:39164) 2024-09-16T00:22:23.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:23.271+0000 7fc9b8c48640 1 -- 172.21.15.112:0/3547099970 learned_addr learned my addr 172.21.15.112:0/3547099970 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:22:23.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:23.272+0000 7fc9b3fff640 1 --2- 172.21.15.112:0/3547099970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9b40fe450 0x7fc9b4113790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:23.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:23.272+0000 7fc9b8c48640 1 -- 172.21.15.112:0/3547099970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9b40fe450 msgr2=0x7fc9b4113790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:23.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:23.272+0000 7fc9b8c48640 1 --2- 172.21.15.112:0/3547099970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9b40fe450 0x7fc9b4113790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:23.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:23.272+0000 7fc9b8c48640 1 -- 172.21.15.112:0/3547099970 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9a8005040 con 0x7fc9b4064fe0 2024-09-16T00:22:23.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:23.272+0000 7fc9b8c48640 1 --2- 172.21.15.112:0/3547099970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9b4064fe0 0x7fc9b4113250 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc9a8003080 tx=0x7fc9a8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:23.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:23.272+0000 7fc9b17fa640 1 -- 172.21.15.112:0/3547099970 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc9a8038070 con 0x7fc9b4064fe0 2024-09-16T00:22:23.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:23.272+0000 7fc9baeac640 1 -- 172.21.15.112:0/3547099970 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9b410d450 con 0x7fc9b4064fe0 2024-09-16T00:22:23.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:23.272+0000 7fc9b17fa640 1 -- 172.21.15.112:0/3547099970 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fc9a80097e0 con 0x7fc9b4064fe0 2024-09-16T00:22:23.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:23.272+0000 7fc9b17fa640 1 -- 172.21.15.112:0/3547099970 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc9a803d420 con 0x7fc9b4064fe0 2024-09-16T00:22:23.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:23.272+0000 7fc9baeac640 1 -- 172.21.15.112:0/3547099970 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9b410d910 con 0x7fc9b4064fe0 2024-09-16T00:22:23.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:23.273+0000 7fc9baeac640 1 -- 172.21.15.112:0/3547099970 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9b40fdce0 con 0x7fc9b4064fe0 2024-09-16T00:22:23.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:23.274+0000 7fc9b17fa640 1 -- 172.21.15.112:0/3547099970 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) v1 ==== 90869+0+0 (secure 0 0 0) 0x7fc9a8030020 con 0x7fc9b4064fe0 2024-09-16T00:22:23.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:23.274+0000 7fc9b17fa640 1 --2- 172.21.15.112:0/3547099970 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fc9a406ed60 0x7fc9a4071210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:23.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:23.274+0000 7fc9b17fa640 1 -- 172.21.15.112:0/3547099970 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fc9a80ae370 con 0x7fc9b4064fe0 2024-09-16T00:22:23.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:23.274+0000 7fc9b3fff640 1 --2- 172.21.15.112:0/3547099970 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fc9a406ed60 0x7fc9a4071210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:23.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:23.275+0000 7fc9b3fff640 1 --2- 172.21.15.112:0/3547099970 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fc9a406ed60 0x7fc9a4071210 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc9b410eb10 tx=0x7fc9a0009120 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:23.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:23.278+0000 7fc9b17fa640 1 -- 172.21.15.112:0/3547099970 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fc9a807b7d0 con 0x7fc9b4064fe0 2024-09-16T00:22:23.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:23.439+0000 7fc9baeac640 1 -- 172.21.15.112:0/3547099970 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc9b404b750 con 0x7fc9b4064fe0 2024-09-16T00:22:23.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:23.440+0000 7fc9b17fa640 1 -- 172.21.15.112:0/3547099970 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+11130 (secure 0 0 0) 0x7fc9a807b170 con 0x7fc9b4064fe0 2024-09-16T00:22:23.444 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:22:23.444 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":38,"fsid":"d346dd56-73c0-11ef-bceb-c7b262605968","created":"2024-09-16T00:16:43.747157+0000","modified":"2024-09-16T00:22:03.998980+0000","last_up_change":"2024-09-16T00:22:02.988149+0000","last_in_change":"2024-09-16T00:21:44.510744+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:20:50.964734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"f6c0fb9d-a0e6-44b1-a66d-ed3a1955c461","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":2840222594},{"type":"v1","addr":"172.21.15.112:6803","nonce":2840222594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":2840222594},{"type":"v1","addr":"172.21.15.112:6805","nonce":2840222594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2840222594},{"type":"v1","addr":"172.21.15.112:6809","nonce":2840222594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":2840222594},{"type":"v1","addr":"172.21.15.112:6807","nonce":2840222594}]},"public_addr":"172.21.15.112:6803/2840222594","cluster_addr":"172.21.15.112:6805/2840222594","heartbeat_back_addr":"172.21.15.112:6809/2840222594","heartbeat_front_addr":"172.21.15.112:6807/2840222594","state":["exists","up"]},{"osd":1,"uuid":"6fee7bb7-5935-4218-b0b5-ecd103121196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":946524214},{"type":"v1","addr":"172.21.15.112:6811","nonce":946524214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":946524214},{"type":"v1","addr":"172.21.15.112:6813","nonce":946524214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":946524214},{"type":"v1","addr":"172.21.15.112:6817","nonce":946524214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":946524214},{"type":"v1","addr":"172.21.15.112:6815","nonce":946524214}]},"public_addr":"172.21.15.112:6811/946524214","cluster_addr":"172.21.15.112:6813/946524214","heartbeat_back_addr":"172.21.15.112:6817/946524214","heartbeat_front_addr":"172.21.15.112:6815/946524214","state":["exists","up"]},{"osd":2,"uuid":"acf3f3a5-2941-49ed-a653-728564ba08df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3837096766},{"type":"v1","addr":"172.21.15.112:6819","nonce":3837096766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3837096766},{"type":"v1","addr":"172.21.15.112:6821","nonce":3837096766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3837096766},{"type":"v1","addr":"172.21.15.112:6825","nonce":3837096766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3837096766},{"type":"v1","addr":"172.21.15.112:6823","nonce":3837096766}]},"public_addr":"172.21.15.112:6819/3837096766","cluster_addr":"172.21.15.112:6821/3837096766","heartbeat_back_addr":"172.21.15.112:6825/3837096766","heartbeat_front_addr":"172.21.15.112:6823/3837096766","state":["exists","up"]},{"osd":3,"uuid":"f0e8ddbe-ed38-4cd9-af68-a68957311e52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":1327586825},{"type":"v1","addr":"172.21.15.169:6801","nonce":1327586825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":1327586825},{"type":"v1","addr":"172.21.15.169:6803","nonce":1327586825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1327586825},{"type":"v1","addr":"172.21.15.169:6807","nonce":1327586825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":1327586825},{"type":"v1","addr":"172.21.15.169:6805","nonce":1327586825}]},"public_addr":"172.21.15.169:6801/1327586825","cluster_addr":"172.21.15.169:6803/1327586825","heartbeat_back_addr":"172.21.15.169:6807/1327586825","heartbeat_front_addr":"172.21.15.169:6805/1327586825","state":["exists","up"]},{"osd":4,"uuid":"06b021d6-0611-4a51-b49e-108783df2837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3980576686},{"type":"v1","addr":"172.21.15.169:6809","nonce":3980576686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":3980576686},{"type":"v1","addr":"172.21.15.169:6811","nonce":3980576686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":3980576686},{"type":"v1","addr":"172.21.15.169:6815","nonce":3980576686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3980576686},{"type":"v1","addr":"172.21.15.169:6813","nonce":3980576686}]},"public_addr":"172.21.15.169:6809/3980576686","cluster_addr":"172.21.15.169:6811/3980576686","heartbeat_back_addr":"172.21.15.169:6815/3980576686","heartbeat_front_addr":"172.21.15.169:6813/3980576686","state":["exists","up"]},{"osd":5,"uuid":"666b01f8-29fa-427d-a2c6-2ef236d0ddd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":2190808120},{"type":"v1","addr":"172.21.15.169:6817","nonce":2190808120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":2190808120},{"type":"v1","addr":"172.21.15.169:6819","nonce":2190808120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":2190808120},{"type":"v1","addr":"172.21.15.169:6823","nonce":2190808120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":2190808120},{"type":"v1","addr":"172.21.15.169:6821","nonce":2190808120}]},"public_addr":"172.21.15.169:6817/2190808120","cluster_addr":"172.21.15.169:6819/2190808120","heartbeat_back_addr":"172.21.15.169:6823/2190808120","heartbeat_front_addr":"172.21.15.169:6821/2190808120","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:19:52.243994+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:20:20.249290+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:20:48.168672+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:21:10.906749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:21:35.636536+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:22:01.186977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:6801/2021251488":"2024-09-17T00:18:36.660338+0000","172.21.15.112:6801/663349889":"2024-09-17T00:17:05.974495+0000","172.21.15.112:0/3430050847":"2024-09-17T00:18:36.660338+0000","172.21.15.112:6800/2021251488":"2024-09-17T00:18:36.660338+0000","172.21.15.112:6800/663349889":"2024-09-17T00:17:05.974495+0000","172.21.15.112:0/2155316648":"2024-09-17T00:17:05.974495+0000","172.21.15.112:0/1686004085":"2024-09-17T00:17:35.477448+0000","172.21.15.112:0/214318070":"2024-09-17T00:17:05.974495+0000","172.21.15.112:6800/122493397":"2024-09-17T00:17:35.477448+0000","172.21.15.112:0/2081985644":"2024-09-17T00:17:05.974495+0000","172.21.15.112:0/4168804950":"2024-09-17T00:17:35.477448+0000","172.21.15.112:0/3817770007":"2024-09-17T00:18:36.660338+0000","172.21.15.112:0/648744088":"2024-09-17T00:17:35.477448+0000","172.21.15.112:6801/122493397":"2024-09-17T00:17:35.477448+0000","172.21.15.112:0/2126830902":"2024-09-17T00:18:36.660338+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:22:23.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:23.442+0000 7fc9baeac640 1 -- 172.21.15.112:0/3547099970 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fc9a406ed60 msgr2=0x7fc9a4071210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:23.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:23.442+0000 7fc9baeac640 1 --2- 172.21.15.112:0/3547099970 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fc9a406ed60 0x7fc9a4071210 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc9b410eb10 tx=0x7fc9a0009120 comp rx=0 tx=0).stop 2024-09-16T00:22:23.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:23.442+0000 7fc9baeac640 1 -- 172.21.15.112:0/3547099970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9b4064fe0 msgr2=0x7fc9b4113250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:23.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:23.442+0000 7fc9baeac640 1 --2- 172.21.15.112:0/3547099970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9b4064fe0 0x7fc9b4113250 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc9a8003080 tx=0x7fc9a8009da0 comp rx=0 tx=0).stop 2024-09-16T00:22:23.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:23.442+0000 7fc9baeac640 1 -- 172.21.15.112:0/3547099970 shutdown_connections 2024-09-16T00:22:23.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:23.442+0000 7fc9baeac640 1 --2- 172.21.15.112:0/3547099970 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fc9a406ed60 0x7fc9a4071210 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:23.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:23.442+0000 7fc9baeac640 1 --2- 172.21.15.112:0/3547099970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9b40fe450 0x7fc9b4113790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:23.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:23.442+0000 7fc9baeac640 1 --2- 172.21.15.112:0/3547099970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9b4064fe0 0x7fc9b4113250 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:23.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:23.442+0000 7fc9baeac640 1 -- 172.21.15.112:0/3547099970 >> 172.21.15.112:0/3547099970 conn(0x7fc9b40723b0 msgr2=0x7fc9b40fa290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:23.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:23.442+0000 7fc9baeac640 1 -- 172.21.15.112:0/3547099970 shutdown_connections 2024-09-16T00:22:23.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:23.442+0000 7fc9baeac640 1 -- 172.21.15.112:0/3547099970 wait complete. 2024-09-16T00:22:24.204 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-16T00:22:24.204 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T00:22:24.311 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:24 smithi112 ceph-mon[27869]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:22:24.311 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:24 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/3547099970' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:22:24.453 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:22:24.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:24 smithi169 ceph-mon[34653]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:22:24.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:24 smithi169 ceph-mon[34653]: from='client.? 172.21.15.112:0/3547099970' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:22:26.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:26.167+0000 7facdfab9640 1 -- 172.21.15.112:0/2314652973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facd80fe460 msgr2=0x7facd80fe830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:26.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:26.167+0000 7facdfab9640 1 --2- 172.21.15.112:0/2314652973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facd80fe460 0x7facd80fe830 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7facc8009c50 tx=0x7facc8004d10 comp rx=0 tx=0).stop 2024-09-16T00:22:26.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:26.167+0000 7facdfab9640 1 -- 172.21.15.112:0/2314652973 shutdown_connections 2024-09-16T00:22:26.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:26.167+0000 7facdfab9640 1 --2- 172.21.15.112:0/2314652973 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7facd8064fc0 0x7facd8065430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:26.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:26.167+0000 7facdfab9640 1 --2- 172.21.15.112:0/2314652973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facd80fe460 0x7facd80fe830 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:26.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:26.167+0000 7facdfab9640 1 -- 172.21.15.112:0/2314652973 >> 172.21.15.112:0/2314652973 conn(0x7facd80723b0 msgr2=0x7facd80727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:26.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:26.167+0000 7facdfab9640 1 -- 172.21.15.112:0/2314652973 shutdown_connections 2024-09-16T00:22:26.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:26.167+0000 7facdfab9640 1 -- 172.21.15.112:0/2314652973 wait complete. 2024-09-16T00:22:26.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:26.167+0000 7facdfab9640 1 Processor -- start 2024-09-16T00:22:26.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:26.167+0000 7facdfab9640 1 -- start start 2024-09-16T00:22:26.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:26.168+0000 7facdfab9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facd8064fc0 0x7facd8113200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:26.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:26.168+0000 7facdfab9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7facd80fe460 0x7facd8113740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:26.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:26.168+0000 7facdfab9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facd8113dd0 con 0x7facd8064fc0 2024-09-16T00:22:26.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:26.168+0000 7facdfab9640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facd810d180 con 0x7facd80fe460 2024-09-16T00:22:26.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:26.168+0000 7facdd855640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facd8064fc0 0x7facd8113200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:26.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:26.168+0000 7facdd855640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facd8064fc0 0x7facd8113200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39182/0 (socket says 172.21.15.112:39182) 2024-09-16T00:22:26.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:26.168+0000 7facdd855640 1 -- 172.21.15.112:0/4079627545 learned_addr learned my addr 172.21.15.112:0/4079627545 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:22:26.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:26.168+0000 7facdd054640 1 --2- 172.21.15.112:0/4079627545 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7facd80fe460 0x7facd8113740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:26.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:26.168+0000 7facdd855640 1 -- 172.21.15.112:0/4079627545 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7facd80fe460 msgr2=0x7facd8113740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:26.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:26.168+0000 7facdd855640 1 --2- 172.21.15.112:0/4079627545 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7facd80fe460 0x7facd8113740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:26.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:26.168+0000 7facdd855640 1 -- 172.21.15.112:0/4079627545 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc80098b0 con 0x7facd8064fc0 2024-09-16T00:22:26.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:26.168+0000 7facdd054640 1 --2- 172.21.15.112:0/4079627545 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7facd80fe460 0x7facd8113740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:22:26.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:26.168+0000 7facdd855640 1 --2- 172.21.15.112:0/4079627545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facd8064fc0 0x7facd8113200 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7facc8004a30 tx=0x7facc8004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:26.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:26.168+0000 7facd67fc640 1 -- 172.21.15.112:0/4079627545 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7facc8009690 con 0x7facd8064fc0 2024-09-16T00:22:26.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:26.168+0000 7facd67fc640 1 -- 172.21.15.112:0/4079627545 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7facc8032b10 con 0x7facd8064fc0 2024-09-16T00:22:26.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:26.168+0000 7facdfab9640 1 -- 172.21.15.112:0/4079627545 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facd810d400 con 0x7facd8064fc0 2024-09-16T00:22:26.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:26.168+0000 7facd67fc640 1 -- 172.21.15.112:0/4079627545 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7facc803daa0 con 0x7facd8064fc0 2024-09-16T00:22:26.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:26.168+0000 7facdfab9640 1 -- 172.21.15.112:0/4079627545 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facd810d920 con 0x7facd8064fc0 2024-09-16T00:22:26.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:26.169+0000 7facdfab9640 1 -- 172.21.15.112:0/4079627545 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facd80fdcf0 con 0x7facd8064fc0 2024-09-16T00:22:26.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:26.170+0000 7facd67fc640 1 -- 172.21.15.112:0/4079627545 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) v1 ==== 90869+0+0 (secure 0 0 0) 0x7facc802f030 con 0x7facd8064fc0 2024-09-16T00:22:26.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:26.170+0000 7facd67fc640 1 --2- 172.21.15.112:0/4079627545 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7facc006ed60 0x7facc0071210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:26.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:26.170+0000 7facd67fc640 1 -- 172.21.15.112:0/4079627545 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4634+0+0 (secure 0 0 0) 0x7facc80aeec0 con 0x7facd8064fc0 2024-09-16T00:22:26.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:26.170+0000 7facdd054640 1 --2- 172.21.15.112:0/4079627545 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7facc006ed60 0x7facc0071210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:26.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:26.171+0000 7facdd054640 1 --2- 172.21.15.112:0/4079627545 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7facc006ed60 0x7facc0071210 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7faccc006fd0 tx=0x7faccc008040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:26.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:26.175+0000 7facd67fc640 1 -- 172.21.15.112:0/4079627545 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7facc807c3a0 con 0x7facd8064fc0 2024-09-16T00:22:26.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:26.329+0000 7facdfab9640 1 -- 172.21.15.112:0/4079627545 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7facd804b7b0 con 0x7facd8064fc0 2024-09-16T00:22:26.330 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:26 smithi112 ceph-mon[27869]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:22:26.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:26.329+0000 7facd67fc640 1 -- 172.21.15.112:0/4079627545 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+11130 (secure 0 0 0) 0x7facc80480c0 con 0x7facd8064fc0 2024-09-16T00:22:26.330 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:22:26.330 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":38,"fsid":"d346dd56-73c0-11ef-bceb-c7b262605968","created":"2024-09-16T00:16:43.747157+0000","modified":"2024-09-16T00:22:03.998980+0000","last_up_change":"2024-09-16T00:22:02.988149+0000","last_in_change":"2024-09-16T00:21:44.510744+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:20:50.964734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"f6c0fb9d-a0e6-44b1-a66d-ed3a1955c461","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":2840222594},{"type":"v1","addr":"172.21.15.112:6803","nonce":2840222594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":2840222594},{"type":"v1","addr":"172.21.15.112:6805","nonce":2840222594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2840222594},{"type":"v1","addr":"172.21.15.112:6809","nonce":2840222594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":2840222594},{"type":"v1","addr":"172.21.15.112:6807","nonce":2840222594}]},"public_addr":"172.21.15.112:6803/2840222594","cluster_addr":"172.21.15.112:6805/2840222594","heartbeat_back_addr":"172.21.15.112:6809/2840222594","heartbeat_front_addr":"172.21.15.112:6807/2840222594","state":["exists","up"]},{"osd":1,"uuid":"6fee7bb7-5935-4218-b0b5-ecd103121196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":946524214},{"type":"v1","addr":"172.21.15.112:6811","nonce":946524214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":946524214},{"type":"v1","addr":"172.21.15.112:6813","nonce":946524214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":946524214},{"type":"v1","addr":"172.21.15.112:6817","nonce":946524214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":946524214},{"type":"v1","addr":"172.21.15.112:6815","nonce":946524214}]},"public_addr":"172.21.15.112:6811/946524214","cluster_addr":"172.21.15.112:6813/946524214","heartbeat_back_addr":"172.21.15.112:6817/946524214","heartbeat_front_addr":"172.21.15.112:6815/946524214","state":["exists","up"]},{"osd":2,"uuid":"acf3f3a5-2941-49ed-a653-728564ba08df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3837096766},{"type":"v1","addr":"172.21.15.112:6819","nonce":3837096766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3837096766},{"type":"v1","addr":"172.21.15.112:6821","nonce":3837096766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3837096766},{"type":"v1","addr":"172.21.15.112:6825","nonce":3837096766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3837096766},{"type":"v1","addr":"172.21.15.112:6823","nonce":3837096766}]},"public_addr":"172.21.15.112:6819/3837096766","cluster_addr":"172.21.15.112:6821/3837096766","heartbeat_back_addr":"172.21.15.112:6825/3837096766","heartbeat_front_addr":"172.21.15.112:6823/3837096766","state":["exists","up"]},{"osd":3,"uuid":"f0e8ddbe-ed38-4cd9-af68-a68957311e52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":1327586825},{"type":"v1","addr":"172.21.15.169:6801","nonce":1327586825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":1327586825},{"type":"v1","addr":"172.21.15.169:6803","nonce":1327586825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1327586825},{"type":"v1","addr":"172.21.15.169:6807","nonce":1327586825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":1327586825},{"type":"v1","addr":"172.21.15.169:6805","nonce":1327586825}]},"public_addr":"172.21.15.169:6801/1327586825","cluster_addr":"172.21.15.169:6803/1327586825","heartbeat_back_addr":"172.21.15.169:6807/1327586825","heartbeat_front_addr":"172.21.15.169:6805/1327586825","state":["exists","up"]},{"osd":4,"uuid":"06b021d6-0611-4a51-b49e-108783df2837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":3980576686},{"type":"v1","addr":"172.21.15.169:6809","nonce":3980576686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":3980576686},{"type":"v1","addr":"172.21.15.169:6811","nonce":3980576686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":3980576686},{"type":"v1","addr":"172.21.15.169:6815","nonce":3980576686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":3980576686},{"type":"v1","addr":"172.21.15.169:6813","nonce":3980576686}]},"public_addr":"172.21.15.169:6809/3980576686","cluster_addr":"172.21.15.169:6811/3980576686","heartbeat_back_addr":"172.21.15.169:6815/3980576686","heartbeat_front_addr":"172.21.15.169:6813/3980576686","state":["exists","up"]},{"osd":5,"uuid":"666b01f8-29fa-427d-a2c6-2ef236d0ddd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":2190808120},{"type":"v1","addr":"172.21.15.169:6817","nonce":2190808120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":2190808120},{"type":"v1","addr":"172.21.15.169:6819","nonce":2190808120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":2190808120},{"type":"v1","addr":"172.21.15.169:6823","nonce":2190808120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":2190808120},{"type":"v1","addr":"172.21.15.169:6821","nonce":2190808120}]},"public_addr":"172.21.15.169:6817/2190808120","cluster_addr":"172.21.15.169:6819/2190808120","heartbeat_back_addr":"172.21.15.169:6823/2190808120","heartbeat_front_addr":"172.21.15.169:6821/2190808120","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:19:52.243994+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:20:20.249290+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:20:48.168672+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:21:10.906749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:21:35.636536+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:22:01.186977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:6801/2021251488":"2024-09-17T00:18:36.660338+0000","172.21.15.112:6801/663349889":"2024-09-17T00:17:05.974495+0000","172.21.15.112:0/3430050847":"2024-09-17T00:18:36.660338+0000","172.21.15.112:6800/2021251488":"2024-09-17T00:18:36.660338+0000","172.21.15.112:6800/663349889":"2024-09-17T00:17:05.974495+0000","172.21.15.112:0/2155316648":"2024-09-17T00:17:05.974495+0000","172.21.15.112:0/1686004085":"2024-09-17T00:17:35.477448+0000","172.21.15.112:0/214318070":"2024-09-17T00:17:05.974495+0000","172.21.15.112:6800/122493397":"2024-09-17T00:17:35.477448+0000","172.21.15.112:0/2081985644":"2024-09-17T00:17:05.974495+0000","172.21.15.112:0/4168804950":"2024-09-17T00:17:35.477448+0000","172.21.15.112:0/3817770007":"2024-09-17T00:18:36.660338+0000","172.21.15.112:0/648744088":"2024-09-17T00:17:35.477448+0000","172.21.15.112:6801/122493397":"2024-09-17T00:17:35.477448+0000","172.21.15.112:0/2126830902":"2024-09-17T00:18:36.660338+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:22:26.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:26.332+0000 7facdfab9640 1 -- 172.21.15.112:0/4079627545 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7facc006ed60 msgr2=0x7facc0071210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:26.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:26.332+0000 7facdfab9640 1 --2- 172.21.15.112:0/4079627545 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7facc006ed60 0x7facc0071210 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7faccc006fd0 tx=0x7faccc008040 comp rx=0 tx=0).stop 2024-09-16T00:22:26.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:26.332+0000 7facdfab9640 1 -- 172.21.15.112:0/4079627545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facd8064fc0 msgr2=0x7facd8113200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:26.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:26.332+0000 7facdfab9640 1 --2- 172.21.15.112:0/4079627545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facd8064fc0 0x7facd8113200 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7facc8004a30 tx=0x7facc8004d10 comp rx=0 tx=0).stop 2024-09-16T00:22:26.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:26.332+0000 7facdfab9640 1 -- 172.21.15.112:0/4079627545 shutdown_connections 2024-09-16T00:22:26.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:26.332+0000 7facdfab9640 1 --2- 172.21.15.112:0/4079627545 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7facc006ed60 0x7facc0071210 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:26.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:26.332+0000 7facdfab9640 1 --2- 172.21.15.112:0/4079627545 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7facd80fe460 0x7facd8113740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:26.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:26.332+0000 7facdfab9640 1 --2- 172.21.15.112:0/4079627545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facd8064fc0 0x7facd8113200 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:26.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:26.332+0000 7facdfab9640 1 -- 172.21.15.112:0/4079627545 >> 172.21.15.112:0/4079627545 conn(0x7facd80723b0 msgr2=0x7facd8105410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:26.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:26.332+0000 7facdfab9640 1 -- 172.21.15.112:0/4079627545 shutdown_connections 2024-09-16T00:22:26.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:26.332+0000 7facdfab9640 1 -- 172.21.15.112:0/4079627545 wait complete. 2024-09-16T00:22:26.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:26 smithi169 ceph-mon[34653]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:22:26.981 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-16T00:22:26.981 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-16T00:22:26.982 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-16T00:22:26.982 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-16T00:22:26.982 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-16T00:22:26.983 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-16T00:22:27.295 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:22:27.306 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:22:27.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:27 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/4079627545' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:22:27.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:27 smithi169 ceph-mon[34653]: from='client.? 172.21.15.112:0/4079627545' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:22:27.695 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:22:27.696 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:22:28.336 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:28 smithi112 ceph-mon[27869]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:22:28.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:28 smithi169 ceph-mon[34653]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:22:30.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:30 smithi112 ceph-mon[27869]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:22:30.405 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:22:30.405 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:22:30.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:30 smithi169 ceph-mon[34653]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:22:32.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:32 smithi112 ceph-mon[27869]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:22:32.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:32 smithi169 ceph-mon[34653]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:22:34.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:34 smithi112 ceph-mon[27869]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:22:34.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:34 smithi169 ceph-mon[34653]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:22:34.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.197+0000 7fa88bc25640 1 -- 172.21.15.112:0/818321581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa884102550 msgr2=0x7fa884102920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:34.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.197+0000 7fa88bc25640 1 --2- 172.21.15.112:0/818321581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa884102550 0x7fa884102920 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa87c0048d0 tx=0x7fa87c009da0 comp rx=0 tx=0).stop 2024-09-16T00:22:34.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.197+0000 7fa88bc25640 1 -- 172.21.15.112:0/818321581 shutdown_connections 2024-09-16T00:22:34.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.197+0000 7fa88bc25640 1 --2- 172.21.15.112:0/818321581 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa884102ef0 0x7fa8840fc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:34.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.197+0000 7fa88bc25640 1 --2- 172.21.15.112:0/818321581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa884102550 0x7fa884102920 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:34.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.197+0000 7fa88bc25640 1 -- 172.21.15.112:0/818321581 >> 172.21.15.112:0/818321581 conn(0x7fa8840f80f0 msgr2=0x7fa8840fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:34.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.197+0000 7fa88bc25640 1 -- 172.21.15.112:0/818321581 shutdown_connections 2024-09-16T00:22:34.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.197+0000 7fa88bc25640 1 -- 172.21.15.112:0/818321581 wait complete. 2024-09-16T00:22:34.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.198+0000 7fa88bc25640 1 Processor -- start 2024-09-16T00:22:34.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.198+0000 7fa88bc25640 1 -- start start 2024-09-16T00:22:34.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.198+0000 7fa88bc25640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa884102550 0x7fa8841135b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:34.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.198+0000 7fa88bc25640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa884102ef0 0x7fa884113af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:34.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.198+0000 7fa88bc25640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8841141d0 con 0x7fa884102550 2024-09-16T00:22:34.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.198+0000 7fa88bc25640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa88410d550 con 0x7fa884102ef0 2024-09-16T00:22:34.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.198+0000 7fa8899c1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa884102550 0x7fa8841135b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:34.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.198+0000 7fa8899c1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa884102550 0x7fa8841135b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34028/0 (socket says 172.21.15.112:34028) 2024-09-16T00:22:34.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.198+0000 7fa8899c1640 1 -- 172.21.15.112:0/2250100721 learned_addr learned my addr 172.21.15.112:0/2250100721 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:22:34.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.198+0000 7fa8899c1640 1 -- 172.21.15.112:0/2250100721 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa884102ef0 msgr2=0x7fa884113af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:34.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.198+0000 7fa8891c0640 1 --2- 172.21.15.112:0/2250100721 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa884102ef0 0x7fa884113af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:34.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.198+0000 7fa8899c1640 1 --2- 172.21.15.112:0/2250100721 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa884102ef0 0x7fa884113af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:34.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.198+0000 7fa8899c1640 1 -- 172.21.15.112:0/2250100721 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa87c005040 con 0x7fa884102550 2024-09-16T00:22:34.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.198+0000 7fa8891c0640 1 --2- 172.21.15.112:0/2250100721 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa884102ef0 0x7fa884113af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:22:34.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.199+0000 7fa8899c1640 1 --2- 172.21.15.112:0/2250100721 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa884102550 0x7fa8841135b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa87c000c00 tx=0x7fa87c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:34.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.199+0000 7fa87a7fc640 1 -- 172.21.15.112:0/2250100721 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa87c038070 con 0x7fa884102550 2024-09-16T00:22:34.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.199+0000 7fa87a7fc640 1 -- 172.21.15.112:0/2250100721 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fa87c03daf0 con 0x7fa884102550 2024-09-16T00:22:34.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.199+0000 7fa88bc25640 1 -- 172.21.15.112:0/2250100721 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa88410d780 con 0x7fa884102550 2024-09-16T00:22:34.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.199+0000 7fa87a7fc640 1 -- 172.21.15.112:0/2250100721 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa87c033b20 con 0x7fa884102550 2024-09-16T00:22:34.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.199+0000 7fa88bc25640 1 -- 172.21.15.112:0/2250100721 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa88410dca0 con 0x7fa884102550 2024-09-16T00:22:34.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.200+0000 7fa88bc25640 1 -- 172.21.15.112:0/2250100721 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa8840fdcd0 con 0x7fa884102550 2024-09-16T00:22:34.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.200+0000 7fa87a7fc640 1 -- 172.21.15.112:0/2250100721 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) v1 ==== 90869+0+0 (secure 0 0 0) 0x7fa87c030020 con 0x7fa884102550 2024-09-16T00:22:34.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.201+0000 7fa87a7fc640 1 --2- 172.21.15.112:0/2250100721 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fa87006ed60 0x7fa870071210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:34.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.201+0000 7fa87a7fc640 1 -- 172.21.15.112:0/2250100721 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fa87c0ae0f0 con 0x7fa884102550 2024-09-16T00:22:34.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.201+0000 7fa8891c0640 1 --2- 172.21.15.112:0/2250100721 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fa87006ed60 0x7fa870071210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:34.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.201+0000 7fa87a7fc640 1 --2- 172.21.15.112:0/2250100721 >> [v2:172.21.15.112:6818/3837096766,v1:172.21.15.112:6819/3837096766] conn(0x7fa870074ae0 0x7fa870076ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:34.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.201+0000 7fa87a7fc640 1 -- 172.21.15.112:0/2250100721 --> [v2:172.21.15.112:6818/3837096766,v1:172.21.15.112:6819/3837096766] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa8700775a0 con 0x7fa870074ae0 2024-09-16T00:22:34.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.201+0000 7fa87a7fc640 1 -- 172.21.15.112:0/2250100721 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa87c0b06e0 con 0x7fa884102550 2024-09-16T00:22:34.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.201+0000 7fa88a1c2640 1 --2- 172.21.15.112:0/2250100721 >> [v2:172.21.15.112:6818/3837096766,v1:172.21.15.112:6819/3837096766] conn(0x7fa870074ae0 0x7fa870076ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:34.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.201+0000 7fa8891c0640 1 --2- 172.21.15.112:0/2250100721 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fa87006ed60 0x7fa870071210 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa874006fd0 tx=0x7fa874008040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:34.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.201+0000 7fa88a1c2640 1 --2- 172.21.15.112:0/2250100721 >> [v2:172.21.15.112:6818/3837096766,v1:172.21.15.112:6819/3837096766] conn(0x7fa870074ae0 0x7fa870076ef0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:34.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.202+0000 7fa87a7fc640 1 -- 172.21.15.112:0/2250100721 <== osd.2 v2:172.21.15.112:6818/3837096766 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fa8700775a0 con 0x7fa870074ae0 2024-09-16T00:22:34.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.214+0000 7fa88bc25640 1 -- 172.21.15.112:0/2250100721 --> [v2:172.21.15.112:6818/3837096766,v1:172.21.15.112:6819/3837096766] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa8840fbd50 con 0x7fa870074ae0 2024-09-16T00:22:34.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.214+0000 7fa87a7fc640 1 -- 172.21.15.112:0/2250100721 <== osd.2 v2:172.21.15.112:6818/3837096766 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa8840fbd50 con 0x7fa870074ae0 2024-09-16T00:22:34.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.215+0000 7fa88bc25640 1 -- 172.21.15.112:0/2250100721 >> [v2:172.21.15.112:6818/3837096766,v1:172.21.15.112:6819/3837096766] conn(0x7fa870074ae0 msgr2=0x7fa870076ef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:34.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.215+0000 7fa88bc25640 1 --2- 172.21.15.112:0/2250100721 >> [v2:172.21.15.112:6818/3837096766,v1:172.21.15.112:6819/3837096766] conn(0x7fa870074ae0 0x7fa870076ef0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:34.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.215+0000 7fa88bc25640 1 -- 172.21.15.112:0/2250100721 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fa87006ed60 msgr2=0x7fa870071210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:34.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.215+0000 7fa88bc25640 1 --2- 172.21.15.112:0/2250100721 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fa87006ed60 0x7fa870071210 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa874006fd0 tx=0x7fa874008040 comp rx=0 tx=0).stop 2024-09-16T00:22:34.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.215+0000 7fa88bc25640 1 -- 172.21.15.112:0/2250100721 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa884102550 msgr2=0x7fa8841135b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:34.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.215+0000 7fa88bc25640 1 --2- 172.21.15.112:0/2250100721 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa884102550 0x7fa8841135b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa87c000c00 tx=0x7fa87c009da0 comp rx=0 tx=0).stop 2024-09-16T00:22:34.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.215+0000 7fa88bc25640 1 -- 172.21.15.112:0/2250100721 shutdown_connections 2024-09-16T00:22:34.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.215+0000 7fa88bc25640 1 --2- 172.21.15.112:0/2250100721 >> [v2:172.21.15.112:6818/3837096766,v1:172.21.15.112:6819/3837096766] conn(0x7fa870074ae0 0x7fa870076ef0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:34.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.215+0000 7fa88bc25640 1 --2- 172.21.15.112:0/2250100721 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fa87006ed60 0x7fa870071210 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:34.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.215+0000 7fa88bc25640 1 --2- 172.21.15.112:0/2250100721 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa884102ef0 0x7fa884113af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:34.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.215+0000 7fa88bc25640 1 --2- 172.21.15.112:0/2250100721 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa884102550 0x7fa8841135b0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:34.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.215+0000 7fa88bc25640 1 -- 172.21.15.112:0/2250100721 >> 172.21.15.112:0/2250100721 conn(0x7fa8840f80f0 msgr2=0x7fa8840f9180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:34.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.215+0000 7fa88bc25640 1 -- 172.21.15.112:0/2250100721 shutdown_connections 2024-09-16T00:22:34.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.215+0000 7fa88bc25640 1 -- 172.21.15.112:0/2250100721 wait complete. 2024-09-16T00:22:35.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.782+0000 7fe0da90b640 1 -- 172.21.15.112:0/746580609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0d41014b0 msgr2=0x7fe0d4101880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:35.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.782+0000 7fe0da90b640 1 --2- 172.21.15.112:0/746580609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0d41014b0 0x7fe0d4101880 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe0c40048d0 tx=0x7fe0c4009da0 comp rx=0 tx=0).stop 2024-09-16T00:22:35.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.782+0000 7fe0da90b640 1 -- 172.21.15.112:0/746580609 shutdown_connections 2024-09-16T00:22:35.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.782+0000 7fe0da90b640 1 --2- 172.21.15.112:0/746580609 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0d40fb470 0x7fe0d40fb8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:35.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.782+0000 7fe0da90b640 1 --2- 172.21.15.112:0/746580609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0d41014b0 0x7fe0d4101880 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:35.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.782+0000 7fe0da90b640 1 -- 172.21.15.112:0/746580609 >> 172.21.15.112:0/746580609 conn(0x7fe0d40f7030 msgr2=0x7fe0d40f9440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:35.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.782+0000 7fe0da90b640 1 -- 172.21.15.112:0/746580609 shutdown_connections 2024-09-16T00:22:35.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.782+0000 7fe0da90b640 1 -- 172.21.15.112:0/746580609 wait complete. 2024-09-16T00:22:35.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.783+0000 7fe0da90b640 1 Processor -- start 2024-09-16T00:22:35.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.783+0000 7fe0da90b640 1 -- start start 2024-09-16T00:22:35.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.784+0000 7fe0da90b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0d40fb470 0x7fe0d410bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:35.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.784+0000 7fe0da90b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0d41014b0 0x7fe0d410c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:35.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.784+0000 7fe0da90b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d410c9d0 con 0x7fe0d40fb470 2024-09-16T00:22:35.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.784+0000 7fe0da90b640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d410cb10 con 0x7fe0d41014b0 2024-09-16T00:22:35.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.784+0000 7fe0d3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0d40fb470 0x7fe0d410bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:35.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.784+0000 7fe0d3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0d40fb470 0x7fe0d410bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34076/0 (socket says 172.21.15.112:34076) 2024-09-16T00:22:35.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.784+0000 7fe0d3fff640 1 -- 172.21.15.112:0/684557385 learned_addr learned my addr 172.21.15.112:0/684557385 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:22:35.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.784+0000 7fe0d37fe640 1 --2- 172.21.15.112:0/684557385 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0d41014b0 0x7fe0d410c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:35.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.784+0000 7fe0d3fff640 1 -- 172.21.15.112:0/684557385 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0d41014b0 msgr2=0x7fe0d410c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:35.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.784+0000 7fe0d3fff640 1 --2- 172.21.15.112:0/684557385 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0d41014b0 0x7fe0d410c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:35.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.784+0000 7fe0d3fff640 1 -- 172.21.15.112:0/684557385 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0c4005040 con 0x7fe0d40fb470 2024-09-16T00:22:35.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.784+0000 7fe0d37fe640 1 --2- 172.21.15.112:0/684557385 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0d41014b0 0x7fe0d410c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:22:35.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.784+0000 7fe0d3fff640 1 --2- 172.21.15.112:0/684557385 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0d40fb470 0x7fe0d410bf50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe0c4000c00 tx=0x7fe0c4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:35.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.785+0000 7fe0d0ff9640 1 -- 172.21.15.112:0/684557385 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe0c4038070 con 0x7fe0d40fb470 2024-09-16T00:22:35.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.785+0000 7fe0d0ff9640 1 -- 172.21.15.112:0/684557385 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fe0c402ade0 con 0x7fe0d40fb470 2024-09-16T00:22:35.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.785+0000 7fe0da90b640 1 -- 172.21.15.112:0/684557385 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0d410cd90 con 0x7fe0d40fb470 2024-09-16T00:22:35.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.785+0000 7fe0d0ff9640 1 -- 172.21.15.112:0/684557385 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe0c402c7e0 con 0x7fe0d40fb470 2024-09-16T00:22:35.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.785+0000 7fe0da90b640 1 -- 172.21.15.112:0/684557385 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0d4119890 con 0x7fe0d40fb470 2024-09-16T00:22:35.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.786+0000 7fe0da90b640 1 -- 172.21.15.112:0/684557385 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe0d40f92f0 con 0x7fe0d40fb470 2024-09-16T00:22:35.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.787+0000 7fe0d0ff9640 1 -- 172.21.15.112:0/684557385 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) v1 ==== 90869+0+0 (secure 0 0 0) 0x7fe0c4030020 con 0x7fe0d40fb470 2024-09-16T00:22:35.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.787+0000 7fe0d0ff9640 1 --2- 172.21.15.112:0/684557385 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fe0bc06ed60 0x7fe0bc071210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:35.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.787+0000 7fe0d0ff9640 1 -- 172.21.15.112:0/684557385 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fe0c40adad0 con 0x7fe0d40fb470 2024-09-16T00:22:35.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.787+0000 7fe0d37fe640 1 --2- 172.21.15.112:0/684557385 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fe0bc06ed60 0x7fe0bc071210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:35.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.787+0000 7fe0d0ff9640 1 --2- 172.21.15.112:0/684557385 >> [v2:172.21.15.112:6810/946524214,v1:172.21.15.112:6811/946524214] conn(0x7fe0bc074ae0 0x7fe0bc076ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:35.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.787+0000 7fe0d0ff9640 1 -- 172.21.15.112:0/684557385 --> [v2:172.21.15.112:6810/946524214,v1:172.21.15.112:6811/946524214] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe0bc0775a0 con 0x7fe0bc074ae0 2024-09-16T00:22:35.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.787+0000 7fe0d0ff9640 1 -- 172.21.15.112:0/684557385 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0c40b2220 con 0x7fe0d40fb470 2024-09-16T00:22:35.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.787+0000 7fe0d8ea8640 1 --2- 172.21.15.112:0/684557385 >> [v2:172.21.15.112:6810/946524214,v1:172.21.15.112:6811/946524214] conn(0x7fe0bc074ae0 0x7fe0bc076ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:35.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.788+0000 7fe0d37fe640 1 --2- 172.21.15.112:0/684557385 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fe0bc06ed60 0x7fe0bc071210 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe0c8004fd0 tx=0x7fe0c800b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:35.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.788+0000 7fe0d8ea8640 1 --2- 172.21.15.112:0/684557385 >> [v2:172.21.15.112:6810/946524214,v1:172.21.15.112:6811/946524214] conn(0x7fe0bc074ae0 0x7fe0bc076ef0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:35.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.788+0000 7fe0d0ff9640 1 -- 172.21.15.112:0/684557385 <== osd.1 v2:172.21.15.112:6810/946524214 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fe0bc0775a0 con 0x7fe0bc074ae0 2024-09-16T00:22:35.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.801+0000 7fe0da90b640 1 -- 172.21.15.112:0/684557385 --> [v2:172.21.15.112:6810/946524214,v1:172.21.15.112:6811/946524214] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe0d40fcbb0 con 0x7fe0bc074ae0 2024-09-16T00:22:35.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.801+0000 7fe0d0ff9640 1 -- 172.21.15.112:0/684557385 <== osd.1 v2:172.21.15.112:6810/946524214 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe0d40fcbb0 con 0x7fe0bc074ae0 2024-09-16T00:22:35.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.801+0000 7fe0da90b640 1 -- 172.21.15.112:0/684557385 >> [v2:172.21.15.112:6810/946524214,v1:172.21.15.112:6811/946524214] conn(0x7fe0bc074ae0 msgr2=0x7fe0bc076ef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:35.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.801+0000 7fe0da90b640 1 --2- 172.21.15.112:0/684557385 >> [v2:172.21.15.112:6810/946524214,v1:172.21.15.112:6811/946524214] conn(0x7fe0bc074ae0 0x7fe0bc076ef0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:35.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.801+0000 7fe0da90b640 1 -- 172.21.15.112:0/684557385 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fe0bc06ed60 msgr2=0x7fe0bc071210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:35.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.801+0000 7fe0da90b640 1 --2- 172.21.15.112:0/684557385 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fe0bc06ed60 0x7fe0bc071210 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe0c8004fd0 tx=0x7fe0c800b040 comp rx=0 tx=0).stop 2024-09-16T00:22:35.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.801+0000 7fe0da90b640 1 -- 172.21.15.112:0/684557385 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0d40fb470 msgr2=0x7fe0d410bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:35.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.801+0000 7fe0da90b640 1 --2- 172.21.15.112:0/684557385 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0d40fb470 0x7fe0d410bf50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe0c4000c00 tx=0x7fe0c4009da0 comp rx=0 tx=0).stop 2024-09-16T00:22:35.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.801+0000 7fe0da90b640 1 -- 172.21.15.112:0/684557385 shutdown_connections 2024-09-16T00:22:35.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.801+0000 7fe0da90b640 1 --2- 172.21.15.112:0/684557385 >> [v2:172.21.15.112:6810/946524214,v1:172.21.15.112:6811/946524214] conn(0x7fe0bc074ae0 0x7fe0bc076ef0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:35.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.801+0000 7fe0da90b640 1 --2- 172.21.15.112:0/684557385 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fe0bc06ed60 0x7fe0bc071210 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:35.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.801+0000 7fe0da90b640 1 --2- 172.21.15.112:0/684557385 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0d41014b0 0x7fe0d410c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:35.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.801+0000 7fe0da90b640 1 --2- 172.21.15.112:0/684557385 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0d40fb470 0x7fe0d410bf50 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:35.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.801+0000 7fe0da90b640 1 -- 172.21.15.112:0/684557385 >> 172.21.15.112:0/684557385 conn(0x7fe0d40f7030 msgr2=0x7fe0d40f88e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:35.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.802+0000 7fe0da90b640 1 -- 172.21.15.112:0/684557385 shutdown_connections 2024-09-16T00:22:35.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.802+0000 7fe0da90b640 1 -- 172.21.15.112:0/684557385 wait complete. 2024-09-16T00:22:35.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.970+0000 7f6a0db70640 1 -- 172.21.15.112:0/2239019688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a08064fe0 msgr2=0x7f6a080653b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:35.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.970+0000 7f6a0db70640 1 --2- 172.21.15.112:0/2239019688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a08064fe0 0x7f6a080653b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f69f80048d0 tx=0x7f69f8009da0 comp rx=0 tx=0).stop 2024-09-16T00:22:35.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.970+0000 7f6a0db70640 1 -- 172.21.15.112:0/2239019688 shutdown_connections 2024-09-16T00:22:35.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.970+0000 7f6a0db70640 1 --2- 172.21.15.112:0/2239019688 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6a080658f0 0x7f6a08108b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:35.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.970+0000 7f6a0db70640 1 --2- 172.21.15.112:0/2239019688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a08064fe0 0x7f6a080653b0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:35.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.971+0000 7f6a0db70640 1 -- 172.21.15.112:0/2239019688 >> 172.21.15.112:0/2239019688 conn(0x7f6a080723b0 msgr2=0x7f6a080727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:35.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.971+0000 7f6a0db70640 1 -- 172.21.15.112:0/2239019688 shutdown_connections 2024-09-16T00:22:35.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.971+0000 7f6a0db70640 1 -- 172.21.15.112:0/2239019688 wait complete. 2024-09-16T00:22:35.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.971+0000 7f6a0db70640 1 Processor -- start 2024-09-16T00:22:35.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.971+0000 7f6a0db70640 1 -- start start 2024-09-16T00:22:35.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.972+0000 7f6a0db70640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a08064fe0 0x7f6a08115410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:35.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.972+0000 7f6a0db70640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6a080658f0 0x7f6a08115950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:35.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.972+0000 7f6a0db70640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a08115f50 con 0x7f6a08064fe0 2024-09-16T00:22:35.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.972+0000 7f6a0db70640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a081160c0 con 0x7f6a080658f0 2024-09-16T00:22:35.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.972+0000 7f6a077fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a08064fe0 0x7f6a08115410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:35.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.972+0000 7f6a077fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a08064fe0 0x7f6a08115410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34092/0 (socket says 172.21.15.112:34092) 2024-09-16T00:22:35.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.972+0000 7f6a077fe640 1 -- 172.21.15.112:0/469506155 learned_addr learned my addr 172.21.15.112:0/469506155 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:22:35.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.972+0000 7f6a06ffd640 1 --2- 172.21.15.112:0/469506155 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6a080658f0 0x7f6a08115950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:35.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.972+0000 7f6a077fe640 1 -- 172.21.15.112:0/469506155 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6a080658f0 msgr2=0x7f6a08115950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:35.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.972+0000 7f6a077fe640 1 --2- 172.21.15.112:0/469506155 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6a080658f0 0x7f6a08115950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:35.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.972+0000 7f6a077fe640 1 -- 172.21.15.112:0/469506155 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f8005040 con 0x7f6a08064fe0 2024-09-16T00:22:35.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.972+0000 7f6a06ffd640 1 --2- 172.21.15.112:0/469506155 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6a080658f0 0x7f6a08115950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:22:35.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.972+0000 7f6a077fe640 1 --2- 172.21.15.112:0/469506155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a08064fe0 0x7f6a08115410 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f69f8003080 tx=0x7f69f8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:35.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.973+0000 7f69effff640 1 -- 172.21.15.112:0/469506155 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69f8038070 con 0x7f6a08064fe0 2024-09-16T00:22:35.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.973+0000 7f6a0db70640 1 -- 172.21.15.112:0/469506155 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a0810f530 con 0x7f6a08064fe0 2024-09-16T00:22:35.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.973+0000 7f69effff640 1 -- 172.21.15.112:0/469506155 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f69f80097e0 con 0x7f6a08064fe0 2024-09-16T00:22:35.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.973+0000 7f69effff640 1 -- 172.21.15.112:0/469506155 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69f803d420 con 0x7f6a08064fe0 2024-09-16T00:22:35.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.973+0000 7f6a0db70640 1 -- 172.21.15.112:0/469506155 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a0810f9f0 con 0x7f6a08064fe0 2024-09-16T00:22:35.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.974+0000 7f6a0db70640 1 -- 172.21.15.112:0/469506155 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6a0810ffd0 con 0x7f6a08064fe0 2024-09-16T00:22:35.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.975+0000 7f69effff640 1 -- 172.21.15.112:0/469506155 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) v1 ==== 90869+0+0 (secure 0 0 0) 0x7f69f8030020 con 0x7f6a08064fe0 2024-09-16T00:22:35.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.975+0000 7f69effff640 1 --2- 172.21.15.112:0/469506155 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f69f006ed60 0x7f69f0071210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:35.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.975+0000 7f69effff640 1 -- 172.21.15.112:0/469506155 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f69f80ae470 con 0x7f6a08064fe0 2024-09-16T00:22:35.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.975+0000 7f6a06ffd640 1 --2- 172.21.15.112:0/469506155 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f69f006ed60 0x7f69f0071210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:35.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.976+0000 7f69effff640 1 --2- 172.21.15.112:0/469506155 >> [v2:172.21.15.169:6808/3980576686,v1:172.21.15.169:6809/3980576686] conn(0x7f69f0074ae0 0x7f69f0076ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:35.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.976+0000 7f69effff640 1 -- 172.21.15.112:0/469506155 --> [v2:172.21.15.169:6808/3980576686,v1:172.21.15.169:6809/3980576686] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f69f00775a0 con 0x7f69f0074ae0 2024-09-16T00:22:35.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.976+0000 7f69effff640 1 -- 172.21.15.112:0/469506155 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f69f80b0ac0 con 0x7f6a08064fe0 2024-09-16T00:22:35.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.976+0000 7f6a07fff640 1 --2- 172.21.15.112:0/469506155 >> [v2:172.21.15.169:6808/3980576686,v1:172.21.15.169:6809/3980576686] conn(0x7f69f0074ae0 0x7f69f0076ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:35.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.976+0000 7f6a06ffd640 1 --2- 172.21.15.112:0/469506155 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f69f006ed60 0x7f69f0071210 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6a08110d10 tx=0x7f69fc009040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:35.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.976+0000 7f6a07fff640 1 --2- 172.21.15.112:0/469506155 >> [v2:172.21.15.169:6808/3980576686,v1:172.21.15.169:6809/3980576686] conn(0x7f69f0074ae0 0x7f69f0076ef0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:35.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.977+0000 7f69effff640 1 -- 172.21.15.112:0/469506155 <== osd.4 v2:172.21.15.169:6808/3980576686 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f69f00775a0 con 0x7f69f0074ae0 2024-09-16T00:22:35.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.990+0000 7f6a0db70640 1 -- 172.21.15.112:0/469506155 --> [v2:172.21.15.169:6808/3980576686,v1:172.21.15.169:6809/3980576686] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6a080fbad0 con 0x7f69f0074ae0 2024-09-16T00:22:35.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.991+0000 7f69effff640 1 -- 172.21.15.112:0/469506155 <== osd.4 v2:172.21.15.169:6808/3980576686 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f6a080fbad0 con 0x7f69f0074ae0 2024-09-16T00:22:35.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.991+0000 7f6a0db70640 1 -- 172.21.15.112:0/469506155 >> [v2:172.21.15.169:6808/3980576686,v1:172.21.15.169:6809/3980576686] conn(0x7f69f0074ae0 msgr2=0x7f69f0076ef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:35.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.991+0000 7f6a0db70640 1 --2- 172.21.15.112:0/469506155 >> [v2:172.21.15.169:6808/3980576686,v1:172.21.15.169:6809/3980576686] conn(0x7f69f0074ae0 0x7f69f0076ef0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:35.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.991+0000 7f6a0db70640 1 -- 172.21.15.112:0/469506155 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f69f006ed60 msgr2=0x7f69f0071210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:35.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.991+0000 7f6a0db70640 1 --2- 172.21.15.112:0/469506155 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f69f006ed60 0x7f69f0071210 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6a08110d10 tx=0x7f69fc009040 comp rx=0 tx=0).stop 2024-09-16T00:22:35.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.991+0000 7f6a0db70640 1 -- 172.21.15.112:0/469506155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a08064fe0 msgr2=0x7f6a08115410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:35.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.991+0000 7f6a0db70640 1 --2- 172.21.15.112:0/469506155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a08064fe0 0x7f6a08115410 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f69f8003080 tx=0x7f69f8009da0 comp rx=0 tx=0).stop 2024-09-16T00:22:35.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.991+0000 7f6a0db70640 1 -- 172.21.15.112:0/469506155 shutdown_connections 2024-09-16T00:22:35.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.991+0000 7f6a0db70640 1 --2- 172.21.15.112:0/469506155 >> [v2:172.21.15.169:6808/3980576686,v1:172.21.15.169:6809/3980576686] conn(0x7f69f0074ae0 0x7f69f0076ef0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:35.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.991+0000 7f6a0db70640 1 --2- 172.21.15.112:0/469506155 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f69f006ed60 0x7f69f0071210 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:35.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.991+0000 7f6a0db70640 1 --2- 172.21.15.112:0/469506155 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6a080658f0 0x7f6a08115950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:35.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.991+0000 7f6a0db70640 1 --2- 172.21.15.112:0/469506155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a08064fe0 0x7f6a08115410 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:35.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.991+0000 7f6a0db70640 1 -- 172.21.15.112:0/469506155 >> 172.21.15.112:0/469506155 conn(0x7f6a080723b0 msgr2=0x7f6a08103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:35.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.991+0000 7f6a0db70640 1 -- 172.21.15.112:0/469506155 shutdown_connections 2024-09-16T00:22:35.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:35.991+0000 7f6a0db70640 1 -- 172.21.15.112:0/469506155 wait complete. 2024-09-16T00:22:36.271 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:36 smithi112 ceph-mon[27869]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:22:36.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:36 smithi169 ceph-mon[34653]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:22:37.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.280+0000 7f2209df6640 1 -- 172.21.15.112:0/3503493300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22040facf0 msgr2=0x7f22040fb160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:37.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.280+0000 7f2209df6640 1 --2- 172.21.15.112:0/3503493300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22040facf0 0x7f22040fb160 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f21f80048d0 tx=0x7f21f8009da0 comp rx=0 tx=0).stop 2024-09-16T00:22:37.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.280+0000 7f2209df6640 1 -- 172.21.15.112:0/3503493300 shutdown_connections 2024-09-16T00:22:37.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.280+0000 7f2209df6640 1 --2- 172.21.15.112:0/3503493300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22040facf0 0x7f22040fb160 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:37.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.280+0000 7f2209df6640 1 --2- 172.21.15.112:0/3503493300 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2204102680 0x7f2204102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:37.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.280+0000 7f2209df6640 1 -- 172.21.15.112:0/3503493300 >> 172.21.15.112:0/3503493300 conn(0x7f2204070d50 msgr2=0x7f2204071150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:37.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.280+0000 7f2209df6640 1 -- 172.21.15.112:0/3503493300 shutdown_connections 2024-09-16T00:22:37.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.280+0000 7f2209df6640 1 -- 172.21.15.112:0/3503493300 wait complete. 2024-09-16T00:22:37.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.281+0000 7f2209df6640 1 Processor -- start 2024-09-16T00:22:37.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.281+0000 7f2209df6640 1 -- start start 2024-09-16T00:22:37.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.281+0000 7f2209df6640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22040facf0 0x7f2204115540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:37.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.281+0000 7f2209df6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2204102680 0x7f2204115a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:37.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.281+0000 7f2209df6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2204116160 con 0x7f2204102680 2024-09-16T00:22:37.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.281+0000 7f2209df6640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f220410f4e0 con 0x7f22040facf0 2024-09-16T00:22:37.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.281+0000 7f2202ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2204102680 0x7f2204115a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:37.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.281+0000 7f2202ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2204102680 0x7f2204115a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34108/0 (socket says 172.21.15.112:34108) 2024-09-16T00:22:37.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.281+0000 7f2202ffd640 1 -- 172.21.15.112:0/1151997806 learned_addr learned my addr 172.21.15.112:0/1151997806 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:22:37.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.282+0000 7f22037fe640 1 --2- 172.21.15.112:0/1151997806 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22040facf0 0x7f2204115540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:37.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.282+0000 7f2202ffd640 1 -- 172.21.15.112:0/1151997806 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22040facf0 msgr2=0x7f2204115540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:37.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.282+0000 7f2202ffd640 1 --2- 172.21.15.112:0/1151997806 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22040facf0 0x7f2204115540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:37.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.282+0000 7f2202ffd640 1 -- 172.21.15.112:0/1151997806 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21f8005040 con 0x7f2204102680 2024-09-16T00:22:37.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.282+0000 7f2202ffd640 1 --2- 172.21.15.112:0/1151997806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2204102680 0x7f2204115a80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f21f8003080 tx=0x7f21f8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:37.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.282+0000 7f21ebfff640 1 -- 172.21.15.112:0/1151997806 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f21f8038070 con 0x7f2204102680 2024-09-16T00:22:37.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.282+0000 7f2209df6640 1 -- 172.21.15.112:0/1151997806 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f220410f710 con 0x7f2204102680 2024-09-16T00:22:37.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.282+0000 7f21ebfff640 1 -- 172.21.15.112:0/1151997806 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f21f80097e0 con 0x7f2204102680 2024-09-16T00:22:37.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.282+0000 7f21ebfff640 1 -- 172.21.15.112:0/1151997806 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f21f803d420 con 0x7f2204102680 2024-09-16T00:22:37.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.282+0000 7f2209df6640 1 -- 172.21.15.112:0/1151997806 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f220410fbd0 con 0x7f2204102680 2024-09-16T00:22:37.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.283+0000 7f2209df6640 1 -- 172.21.15.112:0/1151997806 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2204110070 con 0x7f2204102680 2024-09-16T00:22:37.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.283+0000 7f21ebfff640 1 -- 172.21.15.112:0/1151997806 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) v1 ==== 90869+0+0 (secure 0 0 0) 0x7f21f8030020 con 0x7f2204102680 2024-09-16T00:22:37.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.284+0000 7f21ebfff640 1 --2- 172.21.15.112:0/1151997806 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f21ec06ed60 0x7f21ec071210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:37.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.284+0000 7f21ebfff640 1 -- 172.21.15.112:0/1151997806 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f21f80ae3a0 con 0x7f2204102680 2024-09-16T00:22:37.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.284+0000 7f22037fe640 1 --2- 172.21.15.112:0/1151997806 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f21ec06ed60 0x7f21ec071210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:37.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.284+0000 7f21ebfff640 1 --2- 172.21.15.112:0/1151997806 >> [v2:172.21.15.169:6816/2190808120,v1:172.21.15.169:6817/2190808120] conn(0x7f21ec074ae0 0x7f21ec076ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:37.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.284+0000 7f21ebfff640 1 -- 172.21.15.112:0/1151997806 --> [v2:172.21.15.169:6816/2190808120,v1:172.21.15.169:6817/2190808120] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f21ec0775a0 con 0x7f21ec074ae0 2024-09-16T00:22:37.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.284+0000 7f21ebfff640 1 -- 172.21.15.112:0/1151997806 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f21f80b0990 con 0x7f2204102680 2024-09-16T00:22:37.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.284+0000 7f2203fff640 1 --2- 172.21.15.112:0/1151997806 >> [v2:172.21.15.169:6816/2190808120,v1:172.21.15.169:6817/2190808120] conn(0x7f21ec074ae0 0x7f21ec076ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:37.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.284+0000 7f22037fe640 1 --2- 172.21.15.112:0/1151997806 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f21ec06ed60 0x7f21ec071210 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f21f4006fd0 tx=0x7f21f4008040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:37.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.285+0000 7f2203fff640 1 --2- 172.21.15.112:0/1151997806 >> [v2:172.21.15.169:6816/2190808120,v1:172.21.15.169:6817/2190808120] conn(0x7f21ec074ae0 0x7f21ec076ef0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:37.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.286+0000 7f21ebfff640 1 -- 172.21.15.112:0/1151997806 <== osd.5 v2:172.21.15.169:6816/2190808120 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f21ec0775a0 con 0x7f21ec074ae0 2024-09-16T00:22:37.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.298+0000 7f2209df6640 1 -- 172.21.15.112:0/1151997806 --> [v2:172.21.15.169:6816/2190808120,v1:172.21.15.169:6817/2190808120] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f22040657f0 con 0x7f21ec074ae0 2024-09-16T00:22:37.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.298+0000 7f21ebfff640 1 -- 172.21.15.112:0/1151997806 <== osd.5 v2:172.21.15.169:6816/2190808120 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f22040657f0 con 0x7f21ec074ae0 2024-09-16T00:22:37.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.299+0000 7f2209df6640 1 -- 172.21.15.112:0/1151997806 >> [v2:172.21.15.169:6816/2190808120,v1:172.21.15.169:6817/2190808120] conn(0x7f21ec074ae0 msgr2=0x7f21ec076ef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:37.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.299+0000 7f2209df6640 1 --2- 172.21.15.112:0/1151997806 >> [v2:172.21.15.169:6816/2190808120,v1:172.21.15.169:6817/2190808120] conn(0x7f21ec074ae0 0x7f21ec076ef0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:37.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.299+0000 7f2209df6640 1 -- 172.21.15.112:0/1151997806 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f21ec06ed60 msgr2=0x7f21ec071210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:37.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.299+0000 7f2209df6640 1 --2- 172.21.15.112:0/1151997806 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f21ec06ed60 0x7f21ec071210 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f21f4006fd0 tx=0x7f21f4008040 comp rx=0 tx=0).stop 2024-09-16T00:22:37.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.299+0000 7f2209df6640 1 -- 172.21.15.112:0/1151997806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2204102680 msgr2=0x7f2204115a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:37.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.299+0000 7f2209df6640 1 --2- 172.21.15.112:0/1151997806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2204102680 0x7f2204115a80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f21f8003080 tx=0x7f21f8009da0 comp rx=0 tx=0).stop 2024-09-16T00:22:37.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.299+0000 7f2209df6640 1 -- 172.21.15.112:0/1151997806 shutdown_connections 2024-09-16T00:22:37.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.299+0000 7f2209df6640 1 --2- 172.21.15.112:0/1151997806 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f21ec06ed60 0x7f21ec071210 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:37.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.299+0000 7f2209df6640 1 --2- 172.21.15.112:0/1151997806 >> [v2:172.21.15.169:6816/2190808120,v1:172.21.15.169:6817/2190808120] conn(0x7f21ec074ae0 0x7f21ec076ef0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:37.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.299+0000 7f2209df6640 1 --2- 172.21.15.112:0/1151997806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2204102680 0x7f2204115a80 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:37.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.299+0000 7f2209df6640 1 --2- 172.21.15.112:0/1151997806 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22040facf0 0x7f2204115540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:37.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.299+0000 7f2209df6640 1 -- 172.21.15.112:0/1151997806 >> 172.21.15.112:0/1151997806 conn(0x7f2204070d50 msgr2=0x7f22040f9dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:37.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.299+0000 7f2209df6640 1 -- 172.21.15.112:0/1151997806 shutdown_connections 2024-09-16T00:22:37.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.299+0000 7f2209df6640 1 -- 172.21.15.112:0/1151997806 wait complete. 2024-09-16T00:22:37.395 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:37 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:22:37.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.471+0000 7f2a8b78f640 1 -- 172.21.15.112:0/2828905483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a84064fc0 msgr2=0x7f2a84065430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:37.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.471+0000 7f2a8b78f640 1 --2- 172.21.15.112:0/2828905483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a84064fc0 0x7f2a84065430 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2a780048d0 tx=0x7f2a78009da0 comp rx=0 tx=0).stop 2024-09-16T00:22:37.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.471+0000 7f2a8b78f640 1 -- 172.21.15.112:0/2828905483 shutdown_connections 2024-09-16T00:22:37.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.471+0000 7f2a8b78f640 1 --2- 172.21.15.112:0/2828905483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a84064fc0 0x7f2a84065430 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:37.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.471+0000 7f2a8b78f640 1 --2- 172.21.15.112:0/2828905483 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a840fe460 0x7f2a840fe830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:37.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.471+0000 7f2a8b78f640 1 -- 172.21.15.112:0/2828905483 >> 172.21.15.112:0/2828905483 conn(0x7f2a840723b0 msgr2=0x7f2a840727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:37.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.471+0000 7f2a8b78f640 1 -- 172.21.15.112:0/2828905483 shutdown_connections 2024-09-16T00:22:37.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.471+0000 7f2a8b78f640 1 -- 172.21.15.112:0/2828905483 wait complete. 2024-09-16T00:22:37.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.471+0000 7f2a8b78f640 1 Processor -- start 2024-09-16T00:22:37.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.471+0000 7f2a8b78f640 1 -- start start 2024-09-16T00:22:37.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.471+0000 7f2a8b78f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a84064fc0 0x7f2a841131d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:37.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.471+0000 7f2a8b78f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a840fe460 0x7f2a84113710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:37.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.471+0000 7f2a8b78f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a84113d10 con 0x7f2a840fe460 2024-09-16T00:22:37.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.471+0000 7f2a8b78f640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8410d1a0 con 0x7f2a84064fc0 2024-09-16T00:22:37.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.472+0000 7f2a88d2a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a840fe460 0x7f2a84113710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:37.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.472+0000 7f2a88d2a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a840fe460 0x7f2a84113710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34124/0 (socket says 172.21.15.112:34124) 2024-09-16T00:22:37.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.472+0000 7f2a88d2a640 1 -- 172.21.15.112:0/2927098775 learned_addr learned my addr 172.21.15.112:0/2927098775 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:22:37.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.472+0000 7f2a8952b640 1 --2- 172.21.15.112:0/2927098775 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a84064fc0 0x7f2a841131d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:37.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.472+0000 7f2a88d2a640 1 -- 172.21.15.112:0/2927098775 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a84064fc0 msgr2=0x7f2a841131d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:37.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.472+0000 7f2a88d2a640 1 --2- 172.21.15.112:0/2927098775 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a84064fc0 0x7f2a841131d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:37.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.472+0000 7f2a88d2a640 1 -- 172.21.15.112:0/2927098775 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a78005040 con 0x7f2a840fe460 2024-09-16T00:22:37.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.472+0000 7f2a88d2a640 1 --2- 172.21.15.112:0/2927098775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a840fe460 0x7f2a84113710 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2a78000c00 tx=0x7f2a78009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:37.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.472+0000 7f2a81ffb640 1 -- 172.21.15.112:0/2927098775 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a78038070 con 0x7f2a840fe460 2024-09-16T00:22:37.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.472+0000 7f2a81ffb640 1 -- 172.21.15.112:0/2927098775 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f2a7802ade0 con 0x7f2a840fe460 2024-09-16T00:22:37.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.472+0000 7f2a8b78f640 1 -- 172.21.15.112:0/2927098775 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a8410d420 con 0x7f2a840fe460 2024-09-16T00:22:37.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.472+0000 7f2a81ffb640 1 -- 172.21.15.112:0/2927098775 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a7802c7e0 con 0x7f2a840fe460 2024-09-16T00:22:37.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.472+0000 7f2a8b78f640 1 -- 172.21.15.112:0/2927098775 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a8410d960 con 0x7f2a840fe460 2024-09-16T00:22:37.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.473+0000 7f2a8b78f640 1 -- 172.21.15.112:0/2927098775 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2a8410de90 con 0x7f2a840fe460 2024-09-16T00:22:37.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.474+0000 7f2a81ffb640 1 -- 172.21.15.112:0/2927098775 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) v1 ==== 90869+0+0 (secure 0 0 0) 0x7f2a78030020 con 0x7f2a840fe460 2024-09-16T00:22:37.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.474+0000 7f2a81ffb640 1 --2- 172.21.15.112:0/2927098775 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f2a6c06ed60 0x7f2a6c071210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:37.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.474+0000 7f2a81ffb640 1 -- 172.21.15.112:0/2927098775 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f2a780ada00 con 0x7f2a840fe460 2024-09-16T00:22:37.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.474+0000 7f2a8952b640 1 --2- 172.21.15.112:0/2927098775 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f2a6c06ed60 0x7f2a6c071210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:37.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.475+0000 7f2a81ffb640 1 --2- 172.21.15.112:0/2927098775 >> [v2:172.21.15.112:6802/2840222594,v1:172.21.15.112:6803/2840222594] conn(0x7f2a6c074ae0 0x7f2a6c076ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:37.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.475+0000 7f2a81ffb640 1 -- 172.21.15.112:0/2927098775 --> [v2:172.21.15.112:6802/2840222594,v1:172.21.15.112:6803/2840222594] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2a6c0775a0 con 0x7f2a6c074ae0 2024-09-16T00:22:37.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.475+0000 7f2a81ffb640 1 -- 172.21.15.112:0/2927098775 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a780b2220 con 0x7f2a840fe460 2024-09-16T00:22:37.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.475+0000 7f2a89d2c640 1 --2- 172.21.15.112:0/2927098775 >> [v2:172.21.15.112:6802/2840222594,v1:172.21.15.112:6803/2840222594] conn(0x7f2a6c074ae0 0x7f2a6c076ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:37.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.475+0000 7f2a8952b640 1 --2- 172.21.15.112:0/2927098775 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f2a6c06ed60 0x7f2a6c071210 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2a74004fd0 tx=0x7f2a74009040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:37.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.475+0000 7f2a89d2c640 1 --2- 172.21.15.112:0/2927098775 >> [v2:172.21.15.112:6802/2840222594,v1:172.21.15.112:6803/2840222594] conn(0x7f2a6c074ae0 0x7f2a6c076ef0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:37.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.476+0000 7f2a81ffb640 1 -- 172.21.15.112:0/2927098775 <== osd.0 v2:172.21.15.112:6802/2840222594 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f2a6c0775a0 con 0x7f2a6c074ae0 2024-09-16T00:22:37.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.488+0000 7f2a8b78f640 1 -- 172.21.15.112:0/2927098775 --> [v2:172.21.15.112:6802/2840222594,v1:172.21.15.112:6803/2840222594] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2a84106d90 con 0x7f2a6c074ae0 2024-09-16T00:22:37.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.488+0000 7f2a81ffb640 1 -- 172.21.15.112:0/2927098775 <== osd.0 v2:172.21.15.112:6802/2840222594 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2a84106d90 con 0x7f2a6c074ae0 2024-09-16T00:22:37.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.488+0000 7f2a8b78f640 1 -- 172.21.15.112:0/2927098775 >> [v2:172.21.15.112:6802/2840222594,v1:172.21.15.112:6803/2840222594] conn(0x7f2a6c074ae0 msgr2=0x7f2a6c076ef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:37.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.488+0000 7f2a8b78f640 1 --2- 172.21.15.112:0/2927098775 >> [v2:172.21.15.112:6802/2840222594,v1:172.21.15.112:6803/2840222594] conn(0x7f2a6c074ae0 0x7f2a6c076ef0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:37.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.488+0000 7f2a8b78f640 1 -- 172.21.15.112:0/2927098775 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f2a6c06ed60 msgr2=0x7f2a6c071210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:37.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.488+0000 7f2a8b78f640 1 --2- 172.21.15.112:0/2927098775 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f2a6c06ed60 0x7f2a6c071210 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2a74004fd0 tx=0x7f2a74009040 comp rx=0 tx=0).stop 2024-09-16T00:22:37.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.488+0000 7f2a8b78f640 1 -- 172.21.15.112:0/2927098775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a840fe460 msgr2=0x7f2a84113710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:37.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.488+0000 7f2a8b78f640 1 --2- 172.21.15.112:0/2927098775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a840fe460 0x7f2a84113710 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2a78000c00 tx=0x7f2a78009da0 comp rx=0 tx=0).stop 2024-09-16T00:22:37.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.488+0000 7f2a8b78f640 1 -- 172.21.15.112:0/2927098775 shutdown_connections 2024-09-16T00:22:37.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.488+0000 7f2a8b78f640 1 --2- 172.21.15.112:0/2927098775 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f2a6c06ed60 0x7f2a6c071210 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:37.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.488+0000 7f2a8b78f640 1 --2- 172.21.15.112:0/2927098775 >> [v2:172.21.15.112:6802/2840222594,v1:172.21.15.112:6803/2840222594] conn(0x7f2a6c074ae0 0x7f2a6c076ef0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:37.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.488+0000 7f2a8b78f640 1 --2- 172.21.15.112:0/2927098775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a840fe460 0x7f2a84113710 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:37.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.488+0000 7f2a8b78f640 1 --2- 172.21.15.112:0/2927098775 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a84064fc0 0x7f2a841131d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:37.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.488+0000 7f2a8b78f640 1 -- 172.21.15.112:0/2927098775 >> 172.21.15.112:0/2927098775 conn(0x7f2a840723b0 msgr2=0x7f2a841053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:37.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.488+0000 7f2a8b78f640 1 -- 172.21.15.112:0/2927098775 shutdown_connections 2024-09-16T00:22:37.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:37.488+0000 7f2a8b78f640 1 -- 172.21.15.112:0/2927098775 wait complete. 2024-09-16T00:22:37.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:37 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:22:38.082 INFO:teuthology.orchestra.run.smithi112.stdout:38654705700 2024-09-16T00:22:38.082 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-16T00:22:38.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.903+0000 7f0aabee2640 1 -- 172.21.15.112:0/1641295071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0aa4102ef0 msgr2=0x7f0aa40fc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:38.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.903+0000 7f0aabee2640 1 --2- 172.21.15.112:0/1641295071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0aa4102ef0 0x7f0aa40fc5b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0a940048d0 tx=0x7f0a94009da0 comp rx=0 tx=0).stop 2024-09-16T00:22:38.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.903+0000 7f0aabee2640 1 -- 172.21.15.112:0/1641295071 shutdown_connections 2024-09-16T00:22:38.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.903+0000 7f0aabee2640 1 --2- 172.21.15.112:0/1641295071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0aa4102ef0 0x7f0aa40fc5b0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:38.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.903+0000 7f0aabee2640 1 --2- 172.21.15.112:0/1641295071 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0aa4102550 0x7f0aa4102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:38.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.903+0000 7f0aabee2640 1 -- 172.21.15.112:0/1641295071 >> 172.21.15.112:0/1641295071 conn(0x7f0aa40f80f0 msgr2=0x7f0aa40fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:38.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.903+0000 7f0aabee2640 1 -- 172.21.15.112:0/1641295071 shutdown_connections 2024-09-16T00:22:38.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.903+0000 7f0aabee2640 1 -- 172.21.15.112:0/1641295071 wait complete. 2024-09-16T00:22:38.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.904+0000 7f0aabee2640 1 Processor -- start 2024-09-16T00:22:38.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.904+0000 7f0aabee2640 1 -- start start 2024-09-16T00:22:38.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.904+0000 7f0aabee2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0aa4102550 0x7f0aa41135b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:38.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.904+0000 7f0aabee2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0aa4102ef0 0x7f0aa4113af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:38.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.904+0000 7f0aabee2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa41141d0 con 0x7f0aa4102ef0 2024-09-16T00:22:38.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.904+0000 7f0aabee2640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa410d550 con 0x7f0aa4102550 2024-09-16T00:22:38.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.904+0000 7f0aa947d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0aa4102ef0 0x7f0aa4113af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:38.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.904+0000 7f0aa947d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0aa4102ef0 0x7f0aa4113af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34048/0 (socket says 172.21.15.112:34048) 2024-09-16T00:22:38.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.904+0000 7f0aa947d640 1 -- 172.21.15.112:0/2924014659 learned_addr learned my addr 172.21.15.112:0/2924014659 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:22:38.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.904+0000 7f0aa9c7e640 1 --2- 172.21.15.112:0/2924014659 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0aa4102550 0x7f0aa41135b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:38.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.904+0000 7f0aa947d640 1 -- 172.21.15.112:0/2924014659 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0aa4102550 msgr2=0x7f0aa41135b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:38.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.904+0000 7f0aa947d640 1 --2- 172.21.15.112:0/2924014659 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0aa4102550 0x7f0aa41135b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:38.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.904+0000 7f0aa947d640 1 -- 172.21.15.112:0/2924014659 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a94005040 con 0x7f0aa4102ef0 2024-09-16T00:22:38.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.904+0000 7f0aa947d640 1 --2- 172.21.15.112:0/2924014659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0aa4102ef0 0x7f0aa4113af0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0a94003080 tx=0x7f0a94009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:38.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.905+0000 7f0a9a7fc640 1 -- 172.21.15.112:0/2924014659 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a94038070 con 0x7f0aa4102ef0 2024-09-16T00:22:38.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.905+0000 7f0a9a7fc640 1 -- 172.21.15.112:0/2924014659 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f0a94009760 con 0x7f0aa4102ef0 2024-09-16T00:22:38.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.905+0000 7f0a9a7fc640 1 -- 172.21.15.112:0/2924014659 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a94033420 con 0x7f0aa4102ef0 2024-09-16T00:22:38.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.905+0000 7f0aabee2640 1 -- 172.21.15.112:0/2924014659 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0aa410d780 con 0x7f0aa4102ef0 2024-09-16T00:22:38.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.905+0000 7f0aabee2640 1 -- 172.21.15.112:0/2924014659 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0aa410dbe0 con 0x7f0aa4102ef0 2024-09-16T00:22:38.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.905+0000 7f0aabee2640 1 -- 172.21.15.112:0/2924014659 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0aa40fdcd0 con 0x7f0aa4102ef0 2024-09-16T00:22:38.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.906+0000 7f0a9a7fc640 1 -- 172.21.15.112:0/2924014659 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) v1 ==== 90869+0+0 (secure 0 0 0) 0x7f0a94030020 con 0x7f0aa4102ef0 2024-09-16T00:22:38.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.907+0000 7f0a9a7fc640 1 --2- 172.21.15.112:0/2924014659 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f0a9006ed60 0x7f0a90071210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:38.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.907+0000 7f0a9a7fc640 1 -- 172.21.15.112:0/2924014659 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f0a940ae080 con 0x7f0aa4102ef0 2024-09-16T00:22:38.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.907+0000 7f0aa9c7e640 1 --2- 172.21.15.112:0/2924014659 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f0a9006ed60 0x7f0a90071210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:38.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.907+0000 7f0a9a7fc640 1 --2- 172.21.15.112:0/2924014659 >> [v2:172.21.15.169:6800/1327586825,v1:172.21.15.169:6801/1327586825] conn(0x7f0a90074ae0 0x7f0a90076ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:38.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.907+0000 7f0a9a7fc640 1 -- 172.21.15.112:0/2924014659 --> [v2:172.21.15.169:6800/1327586825,v1:172.21.15.169:6801/1327586825] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0a900775a0 con 0x7f0a90074ae0 2024-09-16T00:22:38.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.907+0000 7f0a9a7fc640 1 -- 172.21.15.112:0/2924014659 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a940ae460 con 0x7f0aa4102ef0 2024-09-16T00:22:38.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.907+0000 7f0aa9c7e640 1 --2- 172.21.15.112:0/2924014659 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f0a9006ed60 0x7f0a90071210 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0aa0001250 tx=0x7f0aa0009620 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:38.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.907+0000 7f0aaa47f640 1 --2- 172.21.15.112:0/2924014659 >> [v2:172.21.15.169:6800/1327586825,v1:172.21.15.169:6801/1327586825] conn(0x7f0a90074ae0 0x7f0a90076ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:38.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.908+0000 7f0aaa47f640 1 --2- 172.21.15.112:0/2924014659 >> [v2:172.21.15.169:6800/1327586825,v1:172.21.15.169:6801/1327586825] conn(0x7f0a90074ae0 0x7f0a90076ef0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:38.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.908+0000 7f0a9a7fc640 1 -- 172.21.15.112:0/2924014659 <== osd.3 v2:172.21.15.169:6800/1327586825 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f0a900775a0 con 0x7f0a90074ae0 2024-09-16T00:22:38.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.921+0000 7f0aabee2640 1 -- 172.21.15.112:0/2924014659 --> [v2:172.21.15.169:6800/1327586825,v1:172.21.15.169:6801/1327586825] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0aa410e0f0 con 0x7f0a90074ae0 2024-09-16T00:22:38.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.922+0000 7f0a9a7fc640 1 -- 172.21.15.112:0/2924014659 <== osd.3 v2:172.21.15.169:6800/1327586825 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f0aa410e0f0 con 0x7f0a90074ae0 2024-09-16T00:22:38.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.922+0000 7f0aabee2640 1 -- 172.21.15.112:0/2924014659 >> [v2:172.21.15.169:6800/1327586825,v1:172.21.15.169:6801/1327586825] conn(0x7f0a90074ae0 msgr2=0x7f0a90076ef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:38.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.922+0000 7f0aabee2640 1 --2- 172.21.15.112:0/2924014659 >> [v2:172.21.15.169:6800/1327586825,v1:172.21.15.169:6801/1327586825] conn(0x7f0a90074ae0 0x7f0a90076ef0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:38.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.922+0000 7f0aabee2640 1 -- 172.21.15.112:0/2924014659 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f0a9006ed60 msgr2=0x7f0a90071210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:38.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.922+0000 7f0aabee2640 1 --2- 172.21.15.112:0/2924014659 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f0a9006ed60 0x7f0a90071210 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0aa0001250 tx=0x7f0aa0009620 comp rx=0 tx=0).stop 2024-09-16T00:22:38.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.922+0000 7f0aabee2640 1 -- 172.21.15.112:0/2924014659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0aa4102ef0 msgr2=0x7f0aa4113af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:38.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.922+0000 7f0aabee2640 1 --2- 172.21.15.112:0/2924014659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0aa4102ef0 0x7f0aa4113af0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0a94003080 tx=0x7f0a94009da0 comp rx=0 tx=0).stop 2024-09-16T00:22:38.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.922+0000 7f0aabee2640 1 -- 172.21.15.112:0/2924014659 shutdown_connections 2024-09-16T00:22:38.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.922+0000 7f0aabee2640 1 --2- 172.21.15.112:0/2924014659 >> [v2:172.21.15.169:6800/1327586825,v1:172.21.15.169:6801/1327586825] conn(0x7f0a90074ae0 0x7f0a90076ef0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:38.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.922+0000 7f0aabee2640 1 --2- 172.21.15.112:0/2924014659 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f0a9006ed60 0x7f0a90071210 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:38.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.922+0000 7f0aabee2640 1 --2- 172.21.15.112:0/2924014659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0aa4102ef0 0x7f0aa4113af0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:38.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.922+0000 7f0aabee2640 1 --2- 172.21.15.112:0/2924014659 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0aa4102550 0x7f0aa41135b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:38.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.922+0000 7f0aabee2640 1 -- 172.21.15.112:0/2924014659 >> 172.21.15.112:0/2924014659 conn(0x7f0aa40f80f0 msgr2=0x7f0aa40f91a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:38.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.923+0000 7f0aabee2640 1 -- 172.21.15.112:0/2924014659 shutdown_connections 2024-09-16T00:22:38.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:34.923+0000 7f0aabee2640 1 -- 172.21.15.112:0/2924014659 wait complete. 2024-09-16T00:22:38.338 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:38 smithi112 ceph-mon[27869]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:22:38.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:38 smithi169 ceph-mon[34653]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:22:38.862 INFO:teuthology.orchestra.run.smithi112.stdout:81604378648 2024-09-16T00:22:38.862 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-16T00:22:39.371 INFO:teuthology.orchestra.run.smithi112.stdout:137438953486 2024-09-16T00:22:39.371 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-16T00:22:39.754 INFO:teuthology.orchestra.run.smithi112.stdout:111669149715 2024-09-16T00:22:39.755 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-16T00:22:40.144 INFO:teuthology.orchestra.run.smithi112.stdout:60129542173 2024-09-16T00:22:40.144 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-16T00:22:40.407 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:40 smithi112 ceph-mon[27869]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:22:40.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:40 smithi169 ceph-mon[34653]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:22:40.800 INFO:teuthology.orchestra.run.smithi112.stdout:158913789961 2024-09-16T00:22:40.801 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-16T00:22:40.830 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:22:40.830 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:22:40.831 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:22:40.832 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:22:40.832 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:22:42.199 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:42 smithi112 ceph-mon[27869]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:22:42.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:42 smithi169 ceph-mon[34653]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:22:44.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:44 smithi112 ceph-mon[27869]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:22:44.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:44 smithi169 ceph-mon[34653]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:22:45.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:45.798+0000 7f976ecd2640 1 -- 172.21.15.112:0/2374608282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9768102ef0 msgr2=0x7f97680fc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:45.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:45.798+0000 7f976ecd2640 1 --2- 172.21.15.112:0/2374608282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9768102ef0 0x7f97680fc5b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f97580048d0 tx=0x7f9758009da0 comp rx=0 tx=0).stop 2024-09-16T00:22:45.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:45.798+0000 7f976ecd2640 1 -- 172.21.15.112:0/2374608282 shutdown_connections 2024-09-16T00:22:45.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:45.798+0000 7f976ecd2640 1 --2- 172.21.15.112:0/2374608282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9768102ef0 0x7f97680fc5b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:45.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:45.798+0000 7f976ecd2640 1 --2- 172.21.15.112:0/2374608282 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9768102550 0x7f9768102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:45.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:45.798+0000 7f976ecd2640 1 -- 172.21.15.112:0/2374608282 >> 172.21.15.112:0/2374608282 conn(0x7f97680f80f0 msgr2=0x7f97680fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:45.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:45.798+0000 7f976ecd2640 1 -- 172.21.15.112:0/2374608282 shutdown_connections 2024-09-16T00:22:45.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:45.798+0000 7f976ecd2640 1 -- 172.21.15.112:0/2374608282 wait complete. 2024-09-16T00:22:45.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:45.799+0000 7f976ecd2640 1 Processor -- start 2024-09-16T00:22:45.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:45.799+0000 7f976ecd2640 1 -- start start 2024-09-16T00:22:45.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:45.799+0000 7f976ecd2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9768102550 0x7f97681135b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:45.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:45.800+0000 7f976ecd2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9768102ef0 0x7f9768113af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:45.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:45.800+0000 7f976ecd2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97681141d0 con 0x7f9768102ef0 2024-09-16T00:22:45.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:45.800+0000 7f976ecd2640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f976810d550 con 0x7f9768102550 2024-09-16T00:22:45.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:45.800+0000 7f975ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9768102ef0 0x7f9768113af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:45.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:45.800+0000 7f975ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9768102ef0 0x7f9768113af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54062/0 (socket says 172.21.15.112:54062) 2024-09-16T00:22:45.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:45.800+0000 7f975ffff640 1 -- 172.21.15.112:0/2067515114 learned_addr learned my addr 172.21.15.112:0/2067515114 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:22:45.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:45.800+0000 7f976ca6e640 1 --2- 172.21.15.112:0/2067515114 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9768102550 0x7f97681135b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:45.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:45.800+0000 7f975ffff640 1 -- 172.21.15.112:0/2067515114 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9768102550 msgr2=0x7f97681135b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:45.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:45.800+0000 7f975ffff640 1 --2- 172.21.15.112:0/2067515114 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9768102550 0x7f97681135b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:45.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:45.800+0000 7f975ffff640 1 -- 172.21.15.112:0/2067515114 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9758005040 con 0x7f9768102ef0 2024-09-16T00:22:45.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:45.800+0000 7f975ffff640 1 --2- 172.21.15.112:0/2067515114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9768102ef0 0x7f9768113af0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9758003080 tx=0x7f9758009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:45.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:45.800+0000 7f975d7fa640 1 -- 172.21.15.112:0/2067515114 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9758038070 con 0x7f9768102ef0 2024-09-16T00:22:45.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:45.800+0000 7f976ecd2640 1 -- 172.21.15.112:0/2067515114 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f976810d780 con 0x7f9768102ef0 2024-09-16T00:22:45.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:45.800+0000 7f975d7fa640 1 -- 172.21.15.112:0/2067515114 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f9758009760 con 0x7f9768102ef0 2024-09-16T00:22:45.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:45.800+0000 7f975d7fa640 1 -- 172.21.15.112:0/2067515114 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f975803d420 con 0x7f9768102ef0 2024-09-16T00:22:45.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:45.801+0000 7f976ecd2640 1 -- 172.21.15.112:0/2067515114 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f976810dc40 con 0x7f9768102ef0 2024-09-16T00:22:45.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:45.801+0000 7f976ecd2640 1 -- 172.21.15.112:0/2067515114 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97680fdcd0 con 0x7f9768102ef0 2024-09-16T00:22:45.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:45.802+0000 7f975d7fa640 1 -- 172.21.15.112:0/2067515114 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) v1 ==== 90869+0+0 (secure 0 0 0) 0x7f9758030020 con 0x7f9768102ef0 2024-09-16T00:22:45.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:45.802+0000 7f975d7fa640 1 --2- 172.21.15.112:0/2067515114 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f975006ed50 0x7f9750071200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:45.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:45.803+0000 7f975d7fa640 1 -- 172.21.15.112:0/2067515114 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f97580ae260 con 0x7f9768102ef0 2024-09-16T00:22:45.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:45.803+0000 7f976ca6e640 1 --2- 172.21.15.112:0/2067515114 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f975006ed50 0x7f9750071200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:45.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:45.803+0000 7f976ca6e640 1 --2- 172.21.15.112:0/2067515114 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f975006ed50 0x7f9750071200 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9764007580 tx=0x7f9764009780 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:45.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:45.807+0000 7f975d7fa640 1 -- 172.21.15.112:0/2067515114 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f975807b6c0 con 0x7f9768102ef0 2024-09-16T00:22:45.838 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:22:45.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:45.970+0000 7f976ecd2640 1 -- 172.21.15.112:0/2067515114 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f97680fbd50 con 0x7f9768102ef0 2024-09-16T00:22:45.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:45.971+0000 7f975d7fa640 1 -- 172.21.15.112:0/2067515114 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f975807b060 con 0x7f9768102ef0 2024-09-16T00:22:45.971 INFO:teuthology.orchestra.run.smithi112.stdout:38654705702 2024-09-16T00:22:45.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:45.972+0000 7f976ecd2640 1 -- 172.21.15.112:0/2067515114 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f975006ed50 msgr2=0x7f9750071200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:45.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:45.972+0000 7f976ecd2640 1 --2- 172.21.15.112:0/2067515114 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f975006ed50 0x7f9750071200 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9764007580 tx=0x7f9764009780 comp rx=0 tx=0).stop 2024-09-16T00:22:45.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:45.972+0000 7f976ecd2640 1 -- 172.21.15.112:0/2067515114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9768102ef0 msgr2=0x7f9768113af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:45.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:45.972+0000 7f976ecd2640 1 --2- 172.21.15.112:0/2067515114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9768102ef0 0x7f9768113af0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9758003080 tx=0x7f9758009da0 comp rx=0 tx=0).stop 2024-09-16T00:22:45.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:45.973+0000 7f976ecd2640 1 -- 172.21.15.112:0/2067515114 shutdown_connections 2024-09-16T00:22:45.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:45.973+0000 7f976ecd2640 1 --2- 172.21.15.112:0/2067515114 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f975006ed50 0x7f9750071200 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:45.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:45.973+0000 7f976ecd2640 1 --2- 172.21.15.112:0/2067515114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9768102ef0 0x7f9768113af0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:45.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:45.973+0000 7f976ecd2640 1 --2- 172.21.15.112:0/2067515114 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9768102550 0x7f97681135b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:45.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:45.973+0000 7f976ecd2640 1 -- 172.21.15.112:0/2067515114 >> 172.21.15.112:0/2067515114 conn(0x7f97680f80f0 msgr2=0x7f97680f9100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:45.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:45.973+0000 7f976ecd2640 1 -- 172.21.15.112:0/2067515114 shutdown_connections 2024-09-16T00:22:45.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:45.973+0000 7f976ecd2640 1 -- 172.21.15.112:0/2067515114 wait complete. 2024-09-16T00:22:46.334 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:46 smithi112 ceph-mon[27869]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:22:46.334 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:46 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/2067515114' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-16T00:22:46.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:46 smithi169 ceph-mon[34653]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:22:46.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:46 smithi169 ceph-mon[34653]: from='client.? 172.21.15.112:0/2067515114' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-16T00:22:47.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.770+0000 7f6792f2c640 1 -- 172.21.15.112:0/1700826630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f678c0fb810 msgr2=0x7f678c0fbbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:47.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.770+0000 7f6792f2c640 1 --2- 172.21.15.112:0/1700826630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f678c0fb810 0x7f678c0fbbe0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f67880048d0 tx=0x7f6788009da0 comp rx=0 tx=0).stop 2024-09-16T00:22:47.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.770+0000 7f6792f2c640 1 -- 172.21.15.112:0/1700826630 shutdown_connections 2024-09-16T00:22:47.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.770+0000 7f6792f2c640 1 --2- 172.21.15.112:0/1700826630 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f678c0fcf80 0x7f678c0fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:47.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.770+0000 7f6792f2c640 1 --2- 172.21.15.112:0/1700826630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f678c0fb810 0x7f678c0fbbe0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:47.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.770+0000 7f6792f2c640 1 -- 172.21.15.112:0/1700826630 >> 172.21.15.112:0/1700826630 conn(0x7f678c0723b0 msgr2=0x7f678c0727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:47.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.770+0000 7f6792f2c640 1 -- 172.21.15.112:0/1700826630 shutdown_connections 2024-09-16T00:22:47.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.770+0000 7f6792f2c640 1 -- 172.21.15.112:0/1700826630 wait complete. 2024-09-16T00:22:47.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.771+0000 7f6792f2c640 1 Processor -- start 2024-09-16T00:22:47.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.771+0000 7f6792f2c640 1 -- start start 2024-09-16T00:22:47.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.772+0000 7f6792f2c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f678c0fb810 0x7f678c1120f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:47.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.772+0000 7f6792f2c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f678c0fcf80 0x7f678c112630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:47.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.772+0000 7f6792f2c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678c112c30 con 0x7f678c0fb810 2024-09-16T00:22:47.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.772+0000 7f6792f2c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678c10c070 con 0x7f678c0fcf80 2024-09-16T00:22:47.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.772+0000 7f6790cc8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f678c0fb810 0x7f678c1120f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:47.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.772+0000 7f6790cc8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f678c0fb810 0x7f678c1120f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54090/0 (socket says 172.21.15.112:54090) 2024-09-16T00:22:47.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.772+0000 7f6790cc8640 1 -- 172.21.15.112:0/1968416209 learned_addr learned my addr 172.21.15.112:0/1968416209 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:22:47.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.772+0000 7f6783fff640 1 --2- 172.21.15.112:0/1968416209 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f678c0fcf80 0x7f678c112630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:47.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.772+0000 7f6790cc8640 1 -- 172.21.15.112:0/1968416209 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f678c0fcf80 msgr2=0x7f678c112630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:47.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.772+0000 7f6790cc8640 1 --2- 172.21.15.112:0/1968416209 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f678c0fcf80 0x7f678c112630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:47.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.772+0000 7f6790cc8640 1 -- 172.21.15.112:0/1968416209 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6788005040 con 0x7f678c0fb810 2024-09-16T00:22:47.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.772+0000 7f6790cc8640 1 --2- 172.21.15.112:0/1968416209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f678c0fb810 0x7f678c1120f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6788000c00 tx=0x7f6788009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:47.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.772+0000 7f67817fa640 1 -- 172.21.15.112:0/1968416209 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6788038070 con 0x7f678c0fb810 2024-09-16T00:22:47.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.772+0000 7f67817fa640 1 -- 172.21.15.112:0/1968416209 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f678802ade0 con 0x7f678c0fb810 2024-09-16T00:22:47.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.772+0000 7f6792f2c640 1 -- 172.21.15.112:0/1968416209 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f678c10c2f0 con 0x7f678c0fb810 2024-09-16T00:22:47.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.772+0000 7f67817fa640 1 -- 172.21.15.112:0/1968416209 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f678802c7e0 con 0x7f678c0fb810 2024-09-16T00:22:47.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.773+0000 7f6792f2c640 1 -- 172.21.15.112:0/1968416209 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f678c10c830 con 0x7f678c0fb810 2024-09-16T00:22:47.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.774+0000 7f6792f2c640 1 -- 172.21.15.112:0/1968416209 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f678c0fcbc0 con 0x7f678c0fb810 2024-09-16T00:22:47.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.775+0000 7f67817fa640 1 -- 172.21.15.112:0/1968416209 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) v1 ==== 90869+0+0 (secure 0 0 0) 0x7f6788030020 con 0x7f678c0fb810 2024-09-16T00:22:47.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.775+0000 7f67817fa640 1 --2- 172.21.15.112:0/1968416209 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f677406ed60 0x7f6774071210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:47.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.775+0000 7f67817fa640 1 -- 172.21.15.112:0/1968416209 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f67880adad0 con 0x7f678c0fb810 2024-09-16T00:22:47.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.775+0000 7f6783fff640 1 --2- 172.21.15.112:0/1968416209 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f677406ed60 0x7f6774071210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:47.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.776+0000 7f6783fff640 1 --2- 172.21.15.112:0/1968416209 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f677406ed60 0x7f6774071210 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f678c10d9b0 tx=0x7f677c00b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:47.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.779+0000 7f67817fa640 1 -- 172.21.15.112:0/1968416209 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f678807afe0 con 0x7f678c0fb810 2024-09-16T00:22:47.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.939+0000 7f6792f2c640 1 -- 172.21.15.112:0/1968416209 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f678c04b750 con 0x7f678c0fb810 2024-09-16T00:22:47.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.939+0000 7f67817fa640 1 -- 172.21.15.112:0/1968416209 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f678807a980 con 0x7f678c0fb810 2024-09-16T00:22:47.940 INFO:teuthology.orchestra.run.smithi112.stdout:81604378651 2024-09-16T00:22:47.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.941+0000 7f6792f2c640 1 -- 172.21.15.112:0/1968416209 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f677406ed60 msgr2=0x7f6774071210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:47.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.941+0000 7f6792f2c640 1 --2- 172.21.15.112:0/1968416209 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f677406ed60 0x7f6774071210 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f678c10d9b0 tx=0x7f677c00b040 comp rx=0 tx=0).stop 2024-09-16T00:22:47.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.941+0000 7f6792f2c640 1 -- 172.21.15.112:0/1968416209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f678c0fb810 msgr2=0x7f678c1120f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:47.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.941+0000 7f6792f2c640 1 --2- 172.21.15.112:0/1968416209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f678c0fb810 0x7f678c1120f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6788000c00 tx=0x7f6788009da0 comp rx=0 tx=0).stop 2024-09-16T00:22:47.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.941+0000 7f6792f2c640 1 -- 172.21.15.112:0/1968416209 shutdown_connections 2024-09-16T00:22:47.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.941+0000 7f6792f2c640 1 --2- 172.21.15.112:0/1968416209 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f677406ed60 0x7f6774071210 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:47.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.941+0000 7f6792f2c640 1 --2- 172.21.15.112:0/1968416209 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f678c0fcf80 0x7f678c112630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:47.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.941+0000 7f6792f2c640 1 --2- 172.21.15.112:0/1968416209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f678c0fb810 0x7f678c1120f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:47.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.941+0000 7f6792f2c640 1 -- 172.21.15.112:0/1968416209 >> 172.21.15.112:0/1968416209 conn(0x7f678c0723b0 msgr2=0x7f678c0fa480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:47.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.941+0000 7f6792f2c640 1 -- 172.21.15.112:0/1968416209 shutdown_connections 2024-09-16T00:22:47.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.941+0000 7f6792f2c640 1 -- 172.21.15.112:0/1968416209 wait complete. 2024-09-16T00:22:47.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.964+0000 7f35608dd640 1 -- 172.21.15.112:0/2909961173 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f355c0fb810 msgr2=0x7f355c0fbbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:47.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.964+0000 7f35608dd640 1 --2- 172.21.15.112:0/2909961173 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f355c0fb810 0x7f355c0fbbe0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f354c0048d0 tx=0x7f354c009da0 comp rx=0 tx=0).stop 2024-09-16T00:22:47.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.965+0000 7f35608dd640 1 -- 172.21.15.112:0/2909961173 shutdown_connections 2024-09-16T00:22:47.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.965+0000 7f35608dd640 1 --2- 172.21.15.112:0/2909961173 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f355c0fcf80 0x7f355c0fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:47.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.965+0000 7f35608dd640 1 --2- 172.21.15.112:0/2909961173 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f355c0fb810 0x7f355c0fbbe0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:47.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.965+0000 7f35608dd640 1 -- 172.21.15.112:0/2909961173 >> 172.21.15.112:0/2909961173 conn(0x7f355c0723b0 msgr2=0x7f355c0727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:47.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.965+0000 7f35608dd640 1 -- 172.21.15.112:0/2909961173 shutdown_connections 2024-09-16T00:22:47.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.965+0000 7f35608dd640 1 -- 172.21.15.112:0/2909961173 wait complete. 2024-09-16T00:22:47.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.965+0000 7f35608dd640 1 Processor -- start 2024-09-16T00:22:47.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.966+0000 7f35608dd640 1 -- start start 2024-09-16T00:22:47.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.966+0000 7f35608dd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f355c0fb810 0x7f355c1120b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:47.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.966+0000 7f35608dd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f355c0fcf80 0x7f355c1125f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:47.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.966+0000 7f35608dd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f355c112bf0 con 0x7f355c0fb810 2024-09-16T00:22:47.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.966+0000 7f35608dd640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f355c10c030 con 0x7f355c0fcf80 2024-09-16T00:22:47.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.966+0000 7f355a59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f355c0fb810 0x7f355c1120b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:47.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.966+0000 7f355a59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f355c0fb810 0x7f355c1120b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54112/0 (socket says 172.21.15.112:54112) 2024-09-16T00:22:47.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.966+0000 7f355a59c640 1 -- 172.21.15.112:0/2980800257 learned_addr learned my addr 172.21.15.112:0/2980800257 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:22:47.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.966+0000 7f3559d9b640 1 --2- 172.21.15.112:0/2980800257 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f355c0fcf80 0x7f355c1125f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:47.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.966+0000 7f355a59c640 1 -- 172.21.15.112:0/2980800257 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f355c0fcf80 msgr2=0x7f355c1125f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:47.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.966+0000 7f355a59c640 1 --2- 172.21.15.112:0/2980800257 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f355c0fcf80 0x7f355c1125f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:47.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.966+0000 7f355a59c640 1 -- 172.21.15.112:0/2980800257 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f354c005040 con 0x7f355c0fb810 2024-09-16T00:22:47.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.967+0000 7f355a59c640 1 --2- 172.21.15.112:0/2980800257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f355c0fb810 0x7f355c1120b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f354c000c00 tx=0x7f354c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:47.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.967+0000 7f354affd640 1 -- 172.21.15.112:0/2980800257 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f354c038070 con 0x7f355c0fb810 2024-09-16T00:22:47.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.967+0000 7f354affd640 1 -- 172.21.15.112:0/2980800257 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f354c02ade0 con 0x7f355c0fb810 2024-09-16T00:22:47.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.967+0000 7f354affd640 1 -- 172.21.15.112:0/2980800257 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f354c02c7e0 con 0x7f355c0fb810 2024-09-16T00:22:47.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.967+0000 7f35608dd640 1 -- 172.21.15.112:0/2980800257 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f355c10c250 con 0x7f355c0fb810 2024-09-16T00:22:47.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.967+0000 7f35608dd640 1 -- 172.21.15.112:0/2980800257 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f355c10c770 con 0x7f355c0fb810 2024-09-16T00:22:47.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.968+0000 7f35608dd640 1 -- 172.21.15.112:0/2980800257 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f355c0fcbc0 con 0x7f355c0fb810 2024-09-16T00:22:47.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.969+0000 7f354affd640 1 -- 172.21.15.112:0/2980800257 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) v1 ==== 90869+0+0 (secure 0 0 0) 0x7f354c030020 con 0x7f355c0fb810 2024-09-16T00:22:47.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.969+0000 7f354affd640 1 --2- 172.21.15.112:0/2980800257 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f354406ed60 0x7f3544071210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:47.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.970+0000 7f354affd640 1 -- 172.21.15.112:0/2980800257 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f354c0ada00 con 0x7f355c0fb810 2024-09-16T00:22:47.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.970+0000 7f3559d9b640 1 --2- 172.21.15.112:0/2980800257 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f354406ed60 0x7f3544071210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:47.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.970+0000 7f3559d9b640 1 --2- 172.21.15.112:0/2980800257 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f354406ed60 0x7f3544071210 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f355c10d970 tx=0x7f355000b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:47.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:47.974+0000 7f354affd640 1 -- 172.21.15.112:0/2980800257 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f354c07af90 con 0x7f355c0fb810 2024-09-16T00:22:48.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:48.134+0000 7f35608dd640 1 -- 172.21.15.112:0/2980800257 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f355c072ec0 con 0x7f355c0fb810 2024-09-16T00:22:48.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:48.134+0000 7f354affd640 1 -- 172.21.15.112:0/2980800257 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f354c07a930 con 0x7f355c0fb810 2024-09-16T00:22:48.135 INFO:teuthology.orchestra.run.smithi112.stdout:60129542175 2024-09-16T00:22:48.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:48.136+0000 7f35608dd640 1 -- 172.21.15.112:0/2980800257 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f354406ed60 msgr2=0x7f3544071210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:48.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:48.136+0000 7f35608dd640 1 --2- 172.21.15.112:0/2980800257 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f354406ed60 0x7f3544071210 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f355c10d970 tx=0x7f355000b040 comp rx=0 tx=0).stop 2024-09-16T00:22:48.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:48.136+0000 7f35608dd640 1 -- 172.21.15.112:0/2980800257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f355c0fb810 msgr2=0x7f355c1120b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:48.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:48.136+0000 7f35608dd640 1 --2- 172.21.15.112:0/2980800257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f355c0fb810 0x7f355c1120b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f354c000c00 tx=0x7f354c009da0 comp rx=0 tx=0).stop 2024-09-16T00:22:48.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:48.136+0000 7f35608dd640 1 -- 172.21.15.112:0/2980800257 shutdown_connections 2024-09-16T00:22:48.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:48.136+0000 7f35608dd640 1 --2- 172.21.15.112:0/2980800257 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f354406ed60 0x7f3544071210 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:48.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:48.136+0000 7f35608dd640 1 --2- 172.21.15.112:0/2980800257 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f355c0fcf80 0x7f355c1125f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:48.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:48.136+0000 7f35608dd640 1 --2- 172.21.15.112:0/2980800257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f355c0fb810 0x7f355c1120b0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:48.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:48.136+0000 7f35608dd640 1 -- 172.21.15.112:0/2980800257 >> 172.21.15.112:0/2980800257 conn(0x7f355c0723b0 msgr2=0x7f355c0fa480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:48.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:48.136+0000 7f35608dd640 1 -- 172.21.15.112:0/2980800257 shutdown_connections 2024-09-16T00:22:48.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:48.136+0000 7f35608dd640 1 -- 172.21.15.112:0/2980800257 wait complete. 2024-09-16T00:22:48.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:48 smithi112 ceph-mon[27869]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:22:48.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:48 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/1968416209' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-16T00:22:48.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:48 smithi169 ceph-mon[34653]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:22:48.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:48 smithi169 ceph-mon[34653]: from='client.? 172.21.15.112:0/1968416209' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-16T00:22:49.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:49 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/2980800257' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-16T00:22:49.429 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378648 got 81604378651 for osd.2 2024-09-16T00:22:49.429 DEBUG:teuthology.parallel:result is None 2024-09-16T00:22:49.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:49 smithi169 ceph-mon[34653]: from='client.? 172.21.15.112:0/2980800257' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-16T00:22:49.850 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705700 got 38654705702 for osd.0 2024-09-16T00:22:49.850 DEBUG:teuthology.parallel:result is None 2024-09-16T00:22:50.488 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:50 smithi112 ceph-mon[27869]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:22:50.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:50 smithi169 ceph-mon[34653]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:22:50.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:50.568+0000 7f9295bf6640 1 -- 172.21.15.112:0/1240399294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92901014d0 msgr2=0x7f92901018a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:50.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:50.568+0000 7f9295bf6640 1 --2- 172.21.15.112:0/1240399294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92901014d0 0x7f92901018a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f92800048d0 tx=0x7f9280009da0 comp rx=0 tx=0).stop 2024-09-16T00:22:50.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:50.568+0000 7f9295bf6640 1 -- 172.21.15.112:0/1240399294 shutdown_connections 2024-09-16T00:22:50.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:50.568+0000 7f9295bf6640 1 --2- 172.21.15.112:0/1240399294 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92900fb490 0x7f92900fb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:50.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:50.568+0000 7f9295bf6640 1 --2- 172.21.15.112:0/1240399294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92901014d0 0x7f92901018a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:50.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:50.568+0000 7f9295bf6640 1 -- 172.21.15.112:0/1240399294 >> 172.21.15.112:0/1240399294 conn(0x7f92900f7050 msgr2=0x7f92900f9460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:50.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:50.568+0000 7f9295bf6640 1 -- 172.21.15.112:0/1240399294 shutdown_connections 2024-09-16T00:22:50.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:50.568+0000 7f9295bf6640 1 -- 172.21.15.112:0/1240399294 wait complete. 2024-09-16T00:22:50.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:50.569+0000 7f9295bf6640 1 Processor -- start 2024-09-16T00:22:50.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:50.569+0000 7f9295bf6640 1 -- start start 2024-09-16T00:22:50.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:50.569+0000 7f9295bf6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92900fb490 0x7f9290112130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:50.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:50.569+0000 7f9295bf6640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92901014d0 0x7f9290112670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:50.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:50.569+0000 7f9295bf6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9290112d00 con 0x7f92900fb490 2024-09-16T00:22:50.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:50.569+0000 7f9295bf6640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f929010c0b0 con 0x7f92901014d0 2024-09-16T00:22:50.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:50.569+0000 7f928f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92900fb490 0x7f9290112130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:50.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:50.569+0000 7f928f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92900fb490 0x7f9290112130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54130/0 (socket says 172.21.15.112:54130) 2024-09-16T00:22:50.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:50.569+0000 7f928f7fe640 1 -- 172.21.15.112:0/1771057077 learned_addr learned my addr 172.21.15.112:0/1771057077 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:22:50.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:50.569+0000 7f928f7fe640 1 -- 172.21.15.112:0/1771057077 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92901014d0 msgr2=0x7f9290112670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:50.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:50.569+0000 7f928f7fe640 1 --2- 172.21.15.112:0/1771057077 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92901014d0 0x7f9290112670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:50.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:50.569+0000 7f928f7fe640 1 -- 172.21.15.112:0/1771057077 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9280005040 con 0x7f92900fb490 2024-09-16T00:22:50.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:50.569+0000 7f928f7fe640 1 --2- 172.21.15.112:0/1771057077 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92900fb490 0x7f9290112130 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9280003080 tx=0x7f9280009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:50.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:50.569+0000 7f928cff9640 1 -- 172.21.15.112:0/1771057077 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9280038070 con 0x7f92900fb490 2024-09-16T00:22:50.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:50.570+0000 7f9295bf6640 1 -- 172.21.15.112:0/1771057077 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f929010c330 con 0x7f92900fb490 2024-09-16T00:22:50.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:50.570+0000 7f928cff9640 1 -- 172.21.15.112:0/1771057077 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f92800097e0 con 0x7f92900fb490 2024-09-16T00:22:50.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:50.570+0000 7f928cff9640 1 -- 172.21.15.112:0/1771057077 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f928003d420 con 0x7f92900fb490 2024-09-16T00:22:50.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:50.570+0000 7f9295bf6640 1 -- 172.21.15.112:0/1771057077 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f929010c7f0 con 0x7f92900fb490 2024-09-16T00:22:50.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:50.571+0000 7f9295bf6640 1 -- 172.21.15.112:0/1771057077 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92900fcbd0 con 0x7f92900fb490 2024-09-16T00:22:50.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:50.571+0000 7f928cff9640 1 -- 172.21.15.112:0/1771057077 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) v1 ==== 90869+0+0 (secure 0 0 0) 0x7f9280030020 con 0x7f92900fb490 2024-09-16T00:22:50.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:50.572+0000 7f928cff9640 1 --2- 172.21.15.112:0/1771057077 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f927c06ed60 0x7f927c071210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:50.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:50.572+0000 7f928cff9640 1 -- 172.21.15.112:0/1771057077 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f92800ae470 con 0x7f92900fb490 2024-09-16T00:22:50.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:50.572+0000 7f9287fff640 1 --2- 172.21.15.112:0/1771057077 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f927c06ed60 0x7f927c071210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:50.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:50.572+0000 7f9287fff640 1 --2- 172.21.15.112:0/1771057077 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f927c06ed60 0x7f927c071210 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f929010d9f0 tx=0x7f9278006d20 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:50.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:50.576+0000 7f928cff9640 1 -- 172.21.15.112:0/1771057077 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f928007b8d0 con 0x7f92900fb490 2024-09-16T00:22:50.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:50.734+0000 7f9295bf6640 1 -- 172.21.15.112:0/1771057077 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f9290108560 con 0x7f92900fb490 2024-09-16T00:22:50.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:50.735+0000 7f928cff9640 1 -- 172.21.15.112:0/1771057077 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f928007b270 con 0x7f92900fb490 2024-09-16T00:22:50.735 INFO:teuthology.orchestra.run.smithi112.stdout:137438953489 2024-09-16T00:22:50.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:50.737+0000 7f9295bf6640 1 -- 172.21.15.112:0/1771057077 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f927c06ed60 msgr2=0x7f927c071210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:50.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:50.737+0000 7f9295bf6640 1 --2- 172.21.15.112:0/1771057077 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f927c06ed60 0x7f927c071210 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f929010d9f0 tx=0x7f9278006d20 comp rx=0 tx=0).stop 2024-09-16T00:22:50.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:50.737+0000 7f9295bf6640 1 -- 172.21.15.112:0/1771057077 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92900fb490 msgr2=0x7f9290112130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:50.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:50.737+0000 7f9295bf6640 1 --2- 172.21.15.112:0/1771057077 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92900fb490 0x7f9290112130 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9280003080 tx=0x7f9280009da0 comp rx=0 tx=0).stop 2024-09-16T00:22:50.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:50.737+0000 7f9295bf6640 1 -- 172.21.15.112:0/1771057077 shutdown_connections 2024-09-16T00:22:50.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:50.737+0000 7f9295bf6640 1 --2- 172.21.15.112:0/1771057077 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f927c06ed60 0x7f927c071210 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:50.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:50.737+0000 7f9295bf6640 1 --2- 172.21.15.112:0/1771057077 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92901014d0 0x7f9290112670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:50.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:50.737+0000 7f9295bf6640 1 --2- 172.21.15.112:0/1771057077 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92900fb490 0x7f9290112130 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:50.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:50.737+0000 7f9295bf6640 1 -- 172.21.15.112:0/1771057077 >> 172.21.15.112:0/1771057077 conn(0x7f92900f7050 msgr2=0x7f92900f8900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:50.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:50.737+0000 7f9295bf6640 1 -- 172.21.15.112:0/1771057077 shutdown_connections 2024-09-16T00:22:50.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:50.737+0000 7f9295bf6640 1 -- 172.21.15.112:0/1771057077 wait complete. 2024-09-16T00:22:51.009 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542173 got 60129542175 for osd.1 2024-09-16T00:22:51.009 DEBUG:teuthology.parallel:result is None 2024-09-16T00:22:51.328 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:51 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/1771057077' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-16T00:22:51.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:51.404+0000 7fc1a0e9e640 1 -- 172.21.15.112:0/1360194041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc19c0fe460 msgr2=0x7fc19c0fe830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:51.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:51.404+0000 7fc1a0e9e640 1 --2- 172.21.15.112:0/1360194041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc19c0fe460 0x7fc19c0fe830 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc18c009c50 tx=0x7fc18c004d10 comp rx=0 tx=0).stop 2024-09-16T00:22:51.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:51.405+0000 7fc1a0e9e640 1 -- 172.21.15.112:0/1360194041 shutdown_connections 2024-09-16T00:22:51.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:51.405+0000 7fc1a0e9e640 1 --2- 172.21.15.112:0/1360194041 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc19c064fc0 0x7fc19c065430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:51.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:51.405+0000 7fc1a0e9e640 1 --2- 172.21.15.112:0/1360194041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc19c0fe460 0x7fc19c0fe830 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:51.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:51.405+0000 7fc1a0e9e640 1 -- 172.21.15.112:0/1360194041 >> 172.21.15.112:0/1360194041 conn(0x7fc19c0723b0 msgr2=0x7fc19c0727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:51.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:51.405+0000 7fc1a0e9e640 1 -- 172.21.15.112:0/1360194041 shutdown_connections 2024-09-16T00:22:51.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:51.405+0000 7fc1a0e9e640 1 -- 172.21.15.112:0/1360194041 wait complete. 2024-09-16T00:22:51.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:51.405+0000 7fc1a0e9e640 1 Processor -- start 2024-09-16T00:22:51.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:51.405+0000 7fc1a0e9e640 1 -- start start 2024-09-16T00:22:51.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:51.405+0000 7fc1a0e9e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc19c064fc0 0x7fc19c1131b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:51.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:51.405+0000 7fc1a0e9e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc19c0fe460 0x7fc19c1136f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:51.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:51.405+0000 7fc1a0e9e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc19c113cf0 con 0x7fc19c0fe460 2024-09-16T00:22:51.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:51.405+0000 7fc1a0e9e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc19c10d130 con 0x7fc19c064fc0 2024-09-16T00:22:51.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:51.406+0000 7fc199d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc19c0fe460 0x7fc19c1136f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:51.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:51.406+0000 7fc199d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc19c0fe460 0x7fc19c1136f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54142/0 (socket says 172.21.15.112:54142) 2024-09-16T00:22:51.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:51.406+0000 7fc199d9b640 1 -- 172.21.15.112:0/565839193 learned_addr learned my addr 172.21.15.112:0/565839193 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:22:51.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:51.406+0000 7fc19a59c640 1 --2- 172.21.15.112:0/565839193 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc19c064fc0 0x7fc19c1131b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:51.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:51.406+0000 7fc199d9b640 1 -- 172.21.15.112:0/565839193 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc19c064fc0 msgr2=0x7fc19c1131b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:51.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:51.406+0000 7fc199d9b640 1 --2- 172.21.15.112:0/565839193 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc19c064fc0 0x7fc19c1131b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:51.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:51.406+0000 7fc199d9b640 1 -- 172.21.15.112:0/565839193 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc18c0098b0 con 0x7fc19c0fe460 2024-09-16T00:22:51.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:51.406+0000 7fc199d9b640 1 --2- 172.21.15.112:0/565839193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc19c0fe460 0x7fc19c1136f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc184002a50 tx=0x7fc184004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:51.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:51.406+0000 7fc192ffd640 1 -- 172.21.15.112:0/565839193 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc184007c30 con 0x7fc19c0fe460 2024-09-16T00:22:51.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:51.406+0000 7fc192ffd640 1 -- 172.21.15.112:0/565839193 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fc18400f040 con 0x7fc19c0fe460 2024-09-16T00:22:51.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:51.406+0000 7fc192ffd640 1 -- 172.21.15.112:0/565839193 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc18400da90 con 0x7fc19c0fe460 2024-09-16T00:22:51.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:51.406+0000 7fc1a0e9e640 1 -- 172.21.15.112:0/565839193 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc19c10d410 con 0x7fc19c0fe460 2024-09-16T00:22:51.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:51.406+0000 7fc1a0e9e640 1 -- 172.21.15.112:0/565839193 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc19c10d9b0 con 0x7fc19c0fe460 2024-09-16T00:22:51.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:51.407+0000 7fc1a0e9e640 1 -- 172.21.15.112:0/565839193 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc19c0fdcf0 con 0x7fc19c0fe460 2024-09-16T00:22:51.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:51.408+0000 7fc192ffd640 1 -- 172.21.15.112:0/565839193 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) v1 ==== 90869+0+0 (secure 0 0 0) 0x7fc18400dc30 con 0x7fc19c0fe460 2024-09-16T00:22:51.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:51.408+0000 7fc192ffd640 1 --2- 172.21.15.112:0/565839193 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fc18806ed60 0x7fc188071210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:51.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:51.408+0000 7fc19a59c640 1 --2- 172.21.15.112:0/565839193 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fc18806ed60 0x7fc188071210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:51.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:51.408+0000 7fc192ffd640 1 -- 172.21.15.112:0/565839193 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fc18408d5e0 con 0x7fc19c0fe460 2024-09-16T00:22:51.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:51.409+0000 7fc19a59c640 1 --2- 172.21.15.112:0/565839193 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fc18806ed60 0x7fc188071210 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc18c009c20 tx=0x7fc18c004d10 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:51.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:51.412+0000 7fc192ffd640 1 -- 172.21.15.112:0/565839193 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fc18404d2a0 con 0x7fc19c0fe460 2024-09-16T00:22:51.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:51 smithi169 ceph-mon[34653]: from='client.? 172.21.15.112:0/1771057077' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-16T00:22:51.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:51.569+0000 7fc1a0e9e640 1 -- 172.21.15.112:0/565839193 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fc19c10e850 con 0x7fc19c0fe460 2024-09-16T00:22:51.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:51.570+0000 7fc192ffd640 1 -- 172.21.15.112:0/565839193 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc18405ac30 con 0x7fc19c0fe460 2024-09-16T00:22:51.570 INFO:teuthology.orchestra.run.smithi112.stdout:158913789964 2024-09-16T00:22:51.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:51.571+0000 7fc1a0e9e640 1 -- 172.21.15.112:0/565839193 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fc18806ed60 msgr2=0x7fc188071210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:51.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:51.571+0000 7fc1a0e9e640 1 --2- 172.21.15.112:0/565839193 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fc18806ed60 0x7fc188071210 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc18c009c20 tx=0x7fc18c004d10 comp rx=0 tx=0).stop 2024-09-16T00:22:51.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:51.571+0000 7fc1a0e9e640 1 -- 172.21.15.112:0/565839193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc19c0fe460 msgr2=0x7fc19c1136f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:51.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:51.571+0000 7fc1a0e9e640 1 --2- 172.21.15.112:0/565839193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc19c0fe460 0x7fc19c1136f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc184002a50 tx=0x7fc184004600 comp rx=0 tx=0).stop 2024-09-16T00:22:51.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:51.571+0000 7fc1a0e9e640 1 -- 172.21.15.112:0/565839193 shutdown_connections 2024-09-16T00:22:51.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:51.571+0000 7fc1a0e9e640 1 --2- 172.21.15.112:0/565839193 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fc18806ed60 0x7fc188071210 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:51.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:51.571+0000 7fc1a0e9e640 1 --2- 172.21.15.112:0/565839193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc19c0fe460 0x7fc19c1136f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:51.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:51.571+0000 7fc1a0e9e640 1 --2- 172.21.15.112:0/565839193 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc19c064fc0 0x7fc19c1131b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:51.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:51.571+0000 7fc1a0e9e640 1 -- 172.21.15.112:0/565839193 >> 172.21.15.112:0/565839193 conn(0x7fc19c0723b0 msgr2=0x7fc19c105410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:51.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:51.572+0000 7fc1a0e9e640 1 -- 172.21.15.112:0/565839193 shutdown_connections 2024-09-16T00:22:51.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:51.572+0000 7fc1a0e9e640 1 -- 172.21.15.112:0/565839193 wait complete. 2024-09-16T00:22:52.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:52.109+0000 7f3e49ceb640 1 -- 172.21.15.112:0/4080078187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e441014b0 msgr2=0x7f3e44101880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:52.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:52.109+0000 7f3e49ceb640 1 --2- 172.21.15.112:0/4080078187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e441014b0 0x7f3e44101880 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3e340048d0 tx=0x7f3e34009da0 comp rx=0 tx=0).stop 2024-09-16T00:22:52.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:52.109+0000 7f3e49ceb640 1 -- 172.21.15.112:0/4080078187 shutdown_connections 2024-09-16T00:22:52.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:52.109+0000 7f3e49ceb640 1 --2- 172.21.15.112:0/4080078187 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e440fb470 0x7f3e440fb8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:52.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:52.109+0000 7f3e49ceb640 1 --2- 172.21.15.112:0/4080078187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e441014b0 0x7f3e44101880 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:52.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:52.109+0000 7f3e49ceb640 1 -- 172.21.15.112:0/4080078187 >> 172.21.15.112:0/4080078187 conn(0x7f3e440f7030 msgr2=0x7f3e440f9440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:52.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:52.109+0000 7f3e49ceb640 1 -- 172.21.15.112:0/4080078187 shutdown_connections 2024-09-16T00:22:52.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:52.109+0000 7f3e49ceb640 1 -- 172.21.15.112:0/4080078187 wait complete. 2024-09-16T00:22:52.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:52.110+0000 7f3e49ceb640 1 Processor -- start 2024-09-16T00:22:52.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:52.110+0000 7f3e49ceb640 1 -- start start 2024-09-16T00:22:52.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:52.110+0000 7f3e49ceb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e440fb470 0x7f3e4410e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:52.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:52.110+0000 7f3e49ceb640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e441014b0 0x7f3e4410e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:52.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:52.110+0000 7f3e49ceb640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4410ed10 con 0x7f3e440fb470 2024-09-16T00:22:52.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:52.110+0000 7f3e49ceb640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4410ee80 con 0x7f3e441014b0 2024-09-16T00:22:52.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:52.110+0000 7f3e437fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e440fb470 0x7f3e4410e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:52.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:52.110+0000 7f3e437fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e440fb470 0x7f3e4410e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54150/0 (socket says 172.21.15.112:54150) 2024-09-16T00:22:52.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:52.110+0000 7f3e437fe640 1 -- 172.21.15.112:0/3773995254 learned_addr learned my addr 172.21.15.112:0/3773995254 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:22:52.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:52.111+0000 7f3e42ffd640 1 --2- 172.21.15.112:0/3773995254 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e441014b0 0x7f3e4410e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:52.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:52.111+0000 7f3e437fe640 1 -- 172.21.15.112:0/3773995254 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e441014b0 msgr2=0x7f3e4410e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:52.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:52.111+0000 7f3e437fe640 1 --2- 172.21.15.112:0/3773995254 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e441014b0 0x7f3e4410e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:52.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:52.111+0000 7f3e437fe640 1 -- 172.21.15.112:0/3773995254 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e34005040 con 0x7f3e440fb470 2024-09-16T00:22:52.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:52.111+0000 7f3e437fe640 1 --2- 172.21.15.112:0/3773995254 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e440fb470 0x7f3e4410e180 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3e340043d0 tx=0x7f3e34009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:52.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:52.111+0000 7f3e2bfff640 1 -- 172.21.15.112:0/3773995254 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e34038070 con 0x7f3e440fb470 2024-09-16T00:22:52.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:52.111+0000 7f3e2bfff640 1 -- 172.21.15.112:0/3773995254 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f3e340096e0 con 0x7f3e440fb470 2024-09-16T00:22:52.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:52.111+0000 7f3e49ceb640 1 -- 172.21.15.112:0/3773995254 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e441171f0 con 0x7f3e440fb470 2024-09-16T00:22:52.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:52.111+0000 7f3e2bfff640 1 -- 172.21.15.112:0/3773995254 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e3403d420 con 0x7f3e440fb470 2024-09-16T00:22:52.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:52.111+0000 7f3e49ceb640 1 -- 172.21.15.112:0/3773995254 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e441176b0 con 0x7f3e440fb470 2024-09-16T00:22:52.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:52.112+0000 7f3e49ceb640 1 -- 172.21.15.112:0/3773995254 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e440fcbb0 con 0x7f3e440fb470 2024-09-16T00:22:52.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:52.112+0000 7f3e2bfff640 1 -- 172.21.15.112:0/3773995254 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) v1 ==== 90869+0+0 (secure 0 0 0) 0x7f3e34030020 con 0x7f3e440fb470 2024-09-16T00:22:52.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:52.113+0000 7f3e2bfff640 1 --2- 172.21.15.112:0/3773995254 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f3e2c06ed50 0x7f3e2c071200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:52.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:52.113+0000 7f3e2bfff640 1 -- 172.21.15.112:0/3773995254 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f3e340ae330 con 0x7f3e440fb470 2024-09-16T00:22:52.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:52.113+0000 7f3e42ffd640 1 --2- 172.21.15.112:0/3773995254 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f3e2c06ed50 0x7f3e2c071200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:52.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:52.113+0000 7f3e42ffd640 1 --2- 172.21.15.112:0/3773995254 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f3e2c06ed50 0x7f3e2c071200 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3e38007fd0 tx=0x7f3e38009e90 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:52.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:52.117+0000 7f3e2bfff640 1 -- 172.21.15.112:0/3773995254 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f3e3407b790 con 0x7f3e440fb470 2024-09-16T00:22:52.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:52.277+0000 7f3e49ceb640 1 -- 172.21.15.112:0/3773995254 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f3e440fac30 con 0x7f3e440fb470 2024-09-16T00:22:52.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:52.277+0000 7f3e2bfff640 1 -- 172.21.15.112:0/3773995254 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3e3407b130 con 0x7f3e440fb470 2024-09-16T00:22:52.278 INFO:teuthology.orchestra.run.smithi112.stdout:111669149718 2024-09-16T00:22:52.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:52.279+0000 7f3e49ceb640 1 -- 172.21.15.112:0/3773995254 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f3e2c06ed50 msgr2=0x7f3e2c071200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:52.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:52.279+0000 7f3e49ceb640 1 --2- 172.21.15.112:0/3773995254 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f3e2c06ed50 0x7f3e2c071200 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3e38007fd0 tx=0x7f3e38009e90 comp rx=0 tx=0).stop 2024-09-16T00:22:52.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:52.279+0000 7f3e49ceb640 1 -- 172.21.15.112:0/3773995254 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e440fb470 msgr2=0x7f3e4410e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:52.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:52.279+0000 7f3e49ceb640 1 --2- 172.21.15.112:0/3773995254 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e440fb470 0x7f3e4410e180 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3e340043d0 tx=0x7f3e34009da0 comp rx=0 tx=0).stop 2024-09-16T00:22:52.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:52.279+0000 7f3e49ceb640 1 -- 172.21.15.112:0/3773995254 shutdown_connections 2024-09-16T00:22:52.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:52.279+0000 7f3e49ceb640 1 --2- 172.21.15.112:0/3773995254 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f3e2c06ed50 0x7f3e2c071200 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:52.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:52.279+0000 7f3e49ceb640 1 --2- 172.21.15.112:0/3773995254 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e441014b0 0x7f3e4410e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:52.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:52.279+0000 7f3e49ceb640 1 --2- 172.21.15.112:0/3773995254 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e440fb470 0x7f3e4410e180 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:52.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:52.279+0000 7f3e49ceb640 1 -- 172.21.15.112:0/3773995254 >> 172.21.15.112:0/3773995254 conn(0x7f3e440f7030 msgr2=0x7f3e440f88e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:52.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:52.279+0000 7f3e49ceb640 1 -- 172.21.15.112:0/3773995254 shutdown_connections 2024-09-16T00:22:52.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:52.279+0000 7f3e49ceb640 1 -- 172.21.15.112:0/3773995254 wait complete. 2024-09-16T00:22:52.289 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:52 smithi112 ceph-mon[27869]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:22:52.289 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:52 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/565839193' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-16T00:22:52.290 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:52 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:22:52.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:52 smithi169 ceph-mon[34653]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:22:52.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:52 smithi169 ceph-mon[34653]: from='client.? 172.21.15.112:0/565839193' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-16T00:22:52.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:52 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:22:52.551 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913789961 got 158913789964 for osd.5 2024-09-16T00:22:52.552 DEBUG:teuthology.parallel:result is None 2024-09-16T00:22:53.098 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953486 got 137438953489 for osd.4 2024-09-16T00:22:53.098 DEBUG:teuthology.parallel:result is None 2024-09-16T00:22:53.349 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:53 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/3773995254' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-16T00:22:53.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:53 smithi169 ceph-mon[34653]: from='client.? 172.21.15.112:0/3773995254' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-16T00:22:53.774 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149715 got 111669149718 for osd.3 2024-09-16T00:22:53.775 DEBUG:teuthology.parallel:result is None 2024-09-16T00:22:53.775 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-16T00:22:53.775 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T00:22:54.023 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:22:54.447 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:54 smithi112 ceph-mon[27869]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:22:54.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:54 smithi169 ceph-mon[34653]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:22:55.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:55.608+0000 7f03fe9b2640 1 -- 172.21.15.112:0/1088603154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03f8102680 msgr2=0x7f03f8102a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:55.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:55.608+0000 7f03fe9b2640 1 --2- 172.21.15.112:0/1088603154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03f8102680 0x7f03f8102a50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f03e80048d0 tx=0x7f03e8009da0 comp rx=0 tx=0).stop 2024-09-16T00:22:55.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:55.608+0000 7f03fe9b2640 1 -- 172.21.15.112:0/1088603154 shutdown_connections 2024-09-16T00:22:55.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:55.608+0000 7f03fe9b2640 1 --2- 172.21.15.112:0/1088603154 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03f80facf0 0x7f03f80fb160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:55.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:55.608+0000 7f03fe9b2640 1 --2- 172.21.15.112:0/1088603154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03f8102680 0x7f03f8102a50 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:55.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:55.608+0000 7f03fe9b2640 1 -- 172.21.15.112:0/1088603154 >> 172.21.15.112:0/1088603154 conn(0x7f03f8070d50 msgr2=0x7f03f8071150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:55.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:55.608+0000 7f03fe9b2640 1 -- 172.21.15.112:0/1088603154 shutdown_connections 2024-09-16T00:22:55.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:55.609+0000 7f03fe9b2640 1 -- 172.21.15.112:0/1088603154 wait complete. 2024-09-16T00:22:55.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:55.609+0000 7f03fe9b2640 1 Processor -- start 2024-09-16T00:22:55.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:55.609+0000 7f03fe9b2640 1 -- start start 2024-09-16T00:22:55.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:55.609+0000 7f03fe9b2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03f80facf0 0x7f03f81132d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:55.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:55.609+0000 7f03fe9b2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03f8102680 0x7f03f8113810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:55.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:55.609+0000 7f03fe9b2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03f8113ea0 con 0x7f03f80facf0 2024-09-16T00:22:55.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:55.609+0000 7f03fe9b2640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03f810d250 con 0x7f03f8102680 2024-09-16T00:22:55.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:55.609+0000 7f03f7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03f80facf0 0x7f03f81132d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:55.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:55.609+0000 7f03f7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03f80facf0 0x7f03f81132d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45154/0 (socket says 172.21.15.112:45154) 2024-09-16T00:22:55.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:55.609+0000 7f03f7fff640 1 -- 172.21.15.112:0/210088726 learned_addr learned my addr 172.21.15.112:0/210088726 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:22:55.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:55.610+0000 7f03f7fff640 1 -- 172.21.15.112:0/210088726 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03f8102680 msgr2=0x7f03f8113810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:55.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:55.610+0000 7f03f77fe640 1 --2- 172.21.15.112:0/210088726 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03f8102680 0x7f03f8113810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:55.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:55.610+0000 7f03f7fff640 1 --2- 172.21.15.112:0/210088726 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03f8102680 0x7f03f8113810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:55.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:55.610+0000 7f03f7fff640 1 -- 172.21.15.112:0/210088726 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03e8005040 con 0x7f03f80facf0 2024-09-16T00:22:55.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:55.610+0000 7f03f7fff640 1 --2- 172.21.15.112:0/210088726 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03f80facf0 0x7f03f81132d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f03e8003080 tx=0x7f03e8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:55.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:55.610+0000 7f03f4ff9640 1 -- 172.21.15.112:0/210088726 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03e8038070 con 0x7f03f80facf0 2024-09-16T00:22:55.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:55.610+0000 7f03f4ff9640 1 -- 172.21.15.112:0/210088726 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f03e80097e0 con 0x7f03f80facf0 2024-09-16T00:22:55.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:55.610+0000 7f03fe9b2640 1 -- 172.21.15.112:0/210088726 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03f810d4d0 con 0x7f03f80facf0 2024-09-16T00:22:55.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:55.610+0000 7f03f4ff9640 1 -- 172.21.15.112:0/210088726 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03e803d420 con 0x7f03f80facf0 2024-09-16T00:22:55.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:55.610+0000 7f03fe9b2640 1 -- 172.21.15.112:0/210088726 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03f810d990 con 0x7f03f80facf0 2024-09-16T00:22:55.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:55.611+0000 7f03fe9b2640 1 -- 172.21.15.112:0/210088726 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03f80657f0 con 0x7f03f80facf0 2024-09-16T00:22:55.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:55.612+0000 7f03f4ff9640 1 -- 172.21.15.112:0/210088726 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) v1 ==== 90869+0+0 (secure 0 0 0) 0x7f03e8030020 con 0x7f03f80facf0 2024-09-16T00:22:55.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:55.612+0000 7f03f4ff9640 1 --2- 172.21.15.112:0/210088726 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f03e006ed60 0x7f03e0071210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:55.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:55.612+0000 7f03f77fe640 1 --2- 172.21.15.112:0/210088726 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f03e006ed60 0x7f03e0071210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:55.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:55.612+0000 7f03f4ff9640 1 -- 172.21.15.112:0/210088726 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f03e80ae3a0 con 0x7f03f80facf0 2024-09-16T00:22:55.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:55.613+0000 7f03f77fe640 1 --2- 172.21.15.112:0/210088726 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f03e006ed60 0x7f03e0071210 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f03f810eb90 tx=0x7f03ec008040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:55.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:55.616+0000 7f03f4ff9640 1 -- 172.21.15.112:0/210088726 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f03e807b800 con 0x7f03f80facf0 2024-09-16T00:22:55.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:55.768+0000 7f03fe9b2640 1 -- 172.21.15.112:0/210088726 --> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f03f804b720 con 0x7f03e006ed60 2024-09-16T00:22:55.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:55.769+0000 7f03f4ff9640 1 -- 172.21.15.112:0/210088726 <== mgr.14208 v2:172.21.15.112:6800/2480414866 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+27953 (secure 0 0 0) 0x7f03f804b720 con 0x7f03e006ed60 2024-09-16T00:22:55.770 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:22:55.770 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-09-16T00:22:55.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:55.771+0000 7f03fe9b2640 1 -- 172.21.15.112:0/210088726 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f03e006ed60 msgr2=0x7f03e0071210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:55.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:55.771+0000 7f03fe9b2640 1 --2- 172.21.15.112:0/210088726 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f03e006ed60 0x7f03e0071210 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f03f810eb90 tx=0x7f03ec008040 comp rx=0 tx=0).stop 2024-09-16T00:22:55.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:55.771+0000 7f03fe9b2640 1 -- 172.21.15.112:0/210088726 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03f80facf0 msgr2=0x7f03f81132d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:55.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:55.771+0000 7f03fe9b2640 1 --2- 172.21.15.112:0/210088726 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03f80facf0 0x7f03f81132d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f03e8003080 tx=0x7f03e8009da0 comp rx=0 tx=0).stop 2024-09-16T00:22:55.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:55.771+0000 7f03fe9b2640 1 -- 172.21.15.112:0/210088726 shutdown_connections 2024-09-16T00:22:55.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:55.771+0000 7f03fe9b2640 1 --2- 172.21.15.112:0/210088726 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f03e006ed60 0x7f03e0071210 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:55.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:55.771+0000 7f03fe9b2640 1 --2- 172.21.15.112:0/210088726 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03f8102680 0x7f03f8113810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:55.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:55.771+0000 7f03fe9b2640 1 --2- 172.21.15.112:0/210088726 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03f80facf0 0x7f03f81132d0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:55.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:55.771+0000 7f03fe9b2640 1 -- 172.21.15.112:0/210088726 >> 172.21.15.112:0/210088726 conn(0x7f03f8070d50 msgr2=0x7f03f80fa000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:55.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:55.771+0000 7f03fe9b2640 1 -- 172.21.15.112:0/210088726 shutdown_connections 2024-09-16T00:22:55.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:55.771+0000 7f03fe9b2640 1 -- 172.21.15.112:0/210088726 wait complete. 2024-09-16T00:22:56.413 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:56 smithi112 ceph-mon[27869]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:22:56.496 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":155,"stamp":"2024-09-16T00:22:54.707243+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":1787928,"kb_used_data":3780,"kb_used_omap":9,"kb_used_meta":1784054,"kb_avail":560682984,"statfs":{"total":575970213888,"available":574139375616,"internally_reserved":0,"allocated":3870720,"data_stored":2558100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10224,"internal_metadata":1826871312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002175"},"pg_stats":[{"pgid":"1.0","version":"23'72","reported_seq":130,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T00:22:03.176828+0000","last_change":"2024-09-16T00:21:37.157575+0000","last_active":"2024-09-16T00:22:03.176828+0000","last_peered":"2024-09-16T00:22:03.176828+0000","last_clean":"2024-09-16T00:22:03.176828+0000","last_became_active":"2024-09-16T00:21:37.157284+0000","last_became_peered":"2024-09-16T00:21:37.157284+0000","last_unstale":"2024-09-16T00:22:03.176828+0000","last_undegraded":"2024-09-16T00:22:03.176828+0000","last_fullsized":"2024-09-16T00:22:03.176828+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:20:51.170553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:20:51.170553+0000","last_clean_scrub_stamp":"2024-09-16T00:20:51.170553+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:14:23.834388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":37,"seq":158913789965,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297696,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447456,"statfs":{"total":95995035648,"available":95690194944,"internally_reserved":0,"allocated":348160,"data_stored":131166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40400000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28100000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26700000000000002}]}]},{"osd":4,"up_from":32,"seq":137438953490,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297700,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447452,"statfs":{"total":95995035648,"available":95690190848,"internally_reserved":0,"allocated":348160,"data_stored":131166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 00:22:40 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.35199999999999998}]},{"osd":1,"last update":"Mon Sep 16 00:22:40 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.40300000000000002}]},{"osd":2,"last update":"Mon Sep 16 00:22:40 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.316}]},{"osd":3,"last update":"Mon Sep 16 00:22:40 2024","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.371,"15min":0.371},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"last":0.29999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]}]},{"osd":3,"up_from":26,"seq":111669149719,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298276,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446876,"statfs":{"total":95995035648,"available":95689601024,"internally_reserved":0,"allocated":942080,"data_stored":721534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 00:22:14 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.47399999999999998}]},{"osd":1,"last update":"Mon Sep 16 00:22:14 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.437}]},{"osd":2,"last update":"Mon Sep 16 00:22:14 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.58999999999999997}]},{"osd":4,"last update":"Mon Sep 16 00:22:39 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.61199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]}]},{"osd":2,"up_from":19,"seq":81604378652,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297696,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447456,"statfs":{"total":95995035648,"available":95690194944,"internally_reserved":0,"allocated":348160,"data_stored":131166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 00:21:51 2024","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"last":0.23499999999999999},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"last":0.27700000000000002}]},{"osd":1,"last update":"Mon Sep 16 00:21:51 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.44,"5min":0.44,"15min":0.44},"last":0.251},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"last":0.25900000000000001}]},{"osd":3,"last update":"Mon Sep 16 00:22:13 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.51100000000000001}]},{"osd":4,"last update":"Mon Sep 16 00:22:41 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.48799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]}]},{"osd":1,"up_from":14,"seq":60129542177,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298284,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446868,"statfs":{"total":95995035648,"available":95689592832,"internally_reserved":0,"allocated":942080,"data_stored":721534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 00:22:27 2024","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.47699999999999998,"5min":0.50800000000000001,"15min":0.50800000000000001},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.34599999999999997,"5min":0.35199999999999998,"15min":0.35299999999999998},"min":{"1min":0.23799999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"last":0.36399999999999999}]},{"osd":2,"last update":"Mon Sep 16 00:21:52 2024","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.372,"5min":0.372,"15min":0.372},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"last":0.40300000000000002}]},{"osd":3,"last update":"Mon Sep 16 00:22:15 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.59599999999999997}]},{"osd":4,"last update":"Mon Sep 16 00:22:37 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.55800000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]}]},{"osd":0,"up_from":9,"seq":38654705704,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298276,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446876,"statfs":{"total":95995035648,"available":95689601024,"internally_reserved":0,"allocated":942080,"data_stored":721534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 00:22:25 2024","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.312,"15min":0.311},"min":{"1min":0.249,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.42099999999999999,"5min":0.495,"15min":0.495},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.32800000000000001,"15min":0.32600000000000001},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.45500000000000002,"5min":0.46700000000000003,"15min":0.46700000000000003},"last":0.29199999999999998}]},{"osd":2,"last update":"Mon Sep 16 00:21:52 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.376,"5min":0.376,"15min":0.376},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.496,"5min":0.496,"15min":0.496},"last":0.30599999999999999}]},{"osd":3,"last update":"Mon Sep 16 00:22:14 2024","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":11.175000000000001,"5min":11.175000000000001,"15min":11.175000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.48399999999999999}]},{"osd":4,"last update":"Mon Sep 16 00:22:40 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.51600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T00:22:56.497 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T00:22:56.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:56 smithi169 ceph-mon[34653]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:22:56.744 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:22:57.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:57 smithi169 ceph-mon[34653]: from='client.14420 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:22:57.540 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:57 smithi112 ceph-mon[27869]: from='client.14420 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:22:58.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:58.234+0000 7ffacf216640 1 -- 172.21.15.112:0/4279472231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffac81014d0 msgr2=0x7ffac81018a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:58.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:58.234+0000 7ffacf216640 1 --2- 172.21.15.112:0/4279472231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffac81014d0 0x7ffac81018a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ffabc0048d0 tx=0x7ffabc009da0 comp rx=0 tx=0).stop 2024-09-16T00:22:58.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:58.234+0000 7ffacf216640 1 -- 172.21.15.112:0/4279472231 shutdown_connections 2024-09-16T00:22:58.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:58.234+0000 7ffacf216640 1 --2- 172.21.15.112:0/4279472231 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffac80fb490 0x7ffac80fb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:58.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:58.234+0000 7ffacf216640 1 --2- 172.21.15.112:0/4279472231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffac81014d0 0x7ffac81018a0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:58.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:58.234+0000 7ffacf216640 1 -- 172.21.15.112:0/4279472231 >> 172.21.15.112:0/4279472231 conn(0x7ffac80f7010 msgr2=0x7ffac80f9420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:58.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:58.234+0000 7ffacf216640 1 -- 172.21.15.112:0/4279472231 shutdown_connections 2024-09-16T00:22:58.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:58.234+0000 7ffacf216640 1 -- 172.21.15.112:0/4279472231 wait complete. 2024-09-16T00:22:58.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:58.235+0000 7ffacf216640 1 Processor -- start 2024-09-16T00:22:58.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:58.235+0000 7ffacf216640 1 -- start start 2024-09-16T00:22:58.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:58.235+0000 7ffacf216640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffac80fb490 0x7ffac80f74b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:58.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:58.235+0000 7ffacf216640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffac81014d0 0x7ffac80f79f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:58.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:58.235+0000 7ffacf216640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac80f7fc0 con 0x7ffac81014d0 2024-09-16T00:22:58.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:58.235+0000 7ffacf216640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac80f8130 con 0x7ffac80fb490 2024-09-16T00:22:58.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:58.235+0000 7ffac7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffac81014d0 0x7ffac80f79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:58.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:58.235+0000 7ffac7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffac81014d0 0x7ffac80f79f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45172/0 (socket says 172.21.15.112:45172) 2024-09-16T00:22:58.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:58.235+0000 7ffac7fff640 1 -- 172.21.15.112:0/3999020002 learned_addr learned my addr 172.21.15.112:0/3999020002 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:22:58.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:58.235+0000 7ffaccfb2640 1 --2- 172.21.15.112:0/3999020002 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffac80fb490 0x7ffac80f74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:58.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:58.235+0000 7ffac7fff640 1 -- 172.21.15.112:0/3999020002 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffac80fb490 msgr2=0x7ffac80f74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:58.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:58.235+0000 7ffac7fff640 1 --2- 172.21.15.112:0/3999020002 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffac80fb490 0x7ffac80f74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:58.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:58.235+0000 7ffac7fff640 1 -- 172.21.15.112:0/3999020002 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffabc005040 con 0x7ffac81014d0 2024-09-16T00:22:58.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:58.236+0000 7ffac7fff640 1 --2- 172.21.15.112:0/3999020002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffac81014d0 0x7ffac80f79f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ffab400b790 tx=0x7ffab40091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:58.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:58.236+0000 7ffac57fa640 1 -- 172.21.15.112:0/3999020002 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffab400c700 con 0x7ffac81014d0 2024-09-16T00:22:58.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:58.236+0000 7ffacf216640 1 -- 172.21.15.112:0/3999020002 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffac81195c0 con 0x7ffac81014d0 2024-09-16T00:22:58.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:58.236+0000 7ffac57fa640 1 -- 172.21.15.112:0/3999020002 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7ffab400cd00 con 0x7ffac81014d0 2024-09-16T00:22:58.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:58.236+0000 7ffac57fa640 1 -- 172.21.15.112:0/3999020002 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffab4015e80 con 0x7ffac81014d0 2024-09-16T00:22:58.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:58.236+0000 7ffacf216640 1 -- 172.21.15.112:0/3999020002 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffac8119a00 con 0x7ffac81014d0 2024-09-16T00:22:58.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:58.237+0000 7ffacf216640 1 -- 172.21.15.112:0/3999020002 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffac80fcbd0 con 0x7ffac81014d0 2024-09-16T00:22:58.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:58.237+0000 7ffac57fa640 1 -- 172.21.15.112:0/3999020002 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) v1 ==== 90869+0+0 (secure 0 0 0) 0x7ffab4012070 con 0x7ffac81014d0 2024-09-16T00:22:58.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:58.238+0000 7ffac57fa640 1 --2- 172.21.15.112:0/3999020002 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7ffab806ed60 0x7ffab8071210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:58.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:58.238+0000 7ffaccfb2640 1 --2- 172.21.15.112:0/3999020002 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7ffab806ed60 0x7ffab8071210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:58.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:58.238+0000 7ffac57fa640 1 -- 172.21.15.112:0/3999020002 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4634+0+0 (secure 0 0 0) 0x7ffab408d720 con 0x7ffac81014d0 2024-09-16T00:22:58.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:58.238+0000 7ffaccfb2640 1 --2- 172.21.15.112:0/3999020002 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7ffab806ed60 0x7ffab8071210 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ffabc009ac0 tx=0x7ffabc035040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:58.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:58.242+0000 7ffac57fa640 1 -- 172.21.15.112:0/3999020002 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7ffab405ab80 con 0x7ffac81014d0 2024-09-16T00:22:58.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:58 smithi112 ceph-mon[27869]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:22:58.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:58.395+0000 7ffacf216640 1 -- 172.21.15.112:0/3999020002 --> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ffac8108540 con 0x7ffab806ed60 2024-09-16T00:22:58.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:58.397+0000 7ffac57fa640 1 -- 172.21.15.112:0/3999020002 <== mgr.14208 v2:172.21.15.112:6800/2480414866 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+27941 (secure 0 0 0) 0x7ffac8108540 con 0x7ffab806ed60 2024-09-16T00:22:58.397 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:22:58.398 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-09-16T00:22:58.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:58.399+0000 7ffacf216640 1 -- 172.21.15.112:0/3999020002 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7ffab806ed60 msgr2=0x7ffab8071210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:58.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:58.399+0000 7ffacf216640 1 --2- 172.21.15.112:0/3999020002 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7ffab806ed60 0x7ffab8071210 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ffabc009ac0 tx=0x7ffabc035040 comp rx=0 tx=0).stop 2024-09-16T00:22:58.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:58.399+0000 7ffacf216640 1 -- 172.21.15.112:0/3999020002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffac81014d0 msgr2=0x7ffac80f79f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:58.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:58.399+0000 7ffacf216640 1 --2- 172.21.15.112:0/3999020002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffac81014d0 0x7ffac80f79f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ffab400b790 tx=0x7ffab40091d0 comp rx=0 tx=0).stop 2024-09-16T00:22:58.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:58.399+0000 7ffacf216640 1 -- 172.21.15.112:0/3999020002 shutdown_connections 2024-09-16T00:22:58.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:58.399+0000 7ffacf216640 1 --2- 172.21.15.112:0/3999020002 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7ffab806ed60 0x7ffab8071210 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:58.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:58.399+0000 7ffacf216640 1 --2- 172.21.15.112:0/3999020002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffac81014d0 0x7ffac80f79f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:58.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:58.399+0000 7ffacf216640 1 --2- 172.21.15.112:0/3999020002 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffac80fb490 0x7ffac80f74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:58.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:58.399+0000 7ffacf216640 1 -- 172.21.15.112:0/3999020002 >> 172.21.15.112:0/3999020002 conn(0x7ffac80f7010 msgr2=0x7ffac81052c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:58.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:58.399+0000 7ffacf216640 1 -- 172.21.15.112:0/3999020002 shutdown_connections 2024-09-16T00:22:58.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:22:58.399+0000 7ffacf216640 1 -- 172.21.15.112:0/3999020002 wait complete. 2024-09-16T00:22:58.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:58 smithi169 ceph-mon[34653]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:22:59.080 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":156,"stamp":"2024-09-16T00:22:56.707571+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":1787928,"kb_used_data":3780,"kb_used_omap":9,"kb_used_meta":1784054,"kb_avail":560682984,"statfs":{"total":575970213888,"available":574139375616,"internally_reserved":0,"allocated":3870720,"data_stored":2558100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10224,"internal_metadata":1826871312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002179"},"pg_stats":[{"pgid":"1.0","version":"23'72","reported_seq":130,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T00:22:03.176828+0000","last_change":"2024-09-16T00:21:37.157575+0000","last_active":"2024-09-16T00:22:03.176828+0000","last_peered":"2024-09-16T00:22:03.176828+0000","last_clean":"2024-09-16T00:22:03.176828+0000","last_became_active":"2024-09-16T00:21:37.157284+0000","last_became_peered":"2024-09-16T00:21:37.157284+0000","last_unstale":"2024-09-16T00:22:03.176828+0000","last_undegraded":"2024-09-16T00:22:03.176828+0000","last_fullsized":"2024-09-16T00:22:03.176828+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:20:51.170553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:20:51.170553+0000","last_clean_scrub_stamp":"2024-09-16T00:20:51.170553+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:14:23.834388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":37,"seq":158913789965,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297696,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447456,"statfs":{"total":95995035648,"available":95690194944,"internally_reserved":0,"allocated":348160,"data_stored":131166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40400000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28100000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26700000000000002}]}]},{"osd":4,"up_from":32,"seq":137438953490,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297700,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447452,"statfs":{"total":95995035648,"available":95690190848,"internally_reserved":0,"allocated":348160,"data_stored":131166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 00:22:40 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.35199999999999998}]},{"osd":1,"last update":"Mon Sep 16 00:22:40 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.40300000000000002}]},{"osd":2,"last update":"Mon Sep 16 00:22:40 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.316}]},{"osd":3,"last update":"Mon Sep 16 00:22:40 2024","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.371,"15min":0.371},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"last":0.29999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]}]},{"osd":3,"up_from":26,"seq":111669149719,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298276,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446876,"statfs":{"total":95995035648,"available":95689601024,"internally_reserved":0,"allocated":942080,"data_stored":721534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 00:22:14 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.47399999999999998}]},{"osd":1,"last update":"Mon Sep 16 00:22:14 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.437}]},{"osd":2,"last update":"Mon Sep 16 00:22:14 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.58999999999999997}]},{"osd":4,"last update":"Mon Sep 16 00:22:39 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.61199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]}]},{"osd":2,"up_from":19,"seq":81604378653,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297696,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447456,"statfs":{"total":95995035648,"available":95690194944,"internally_reserved":0,"allocated":348160,"data_stored":131166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 00:22:52 2024","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.33400000000000002,"15min":0.33300000000000002},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.46000000000000002,"5min":0.50900000000000001,"15min":0.50900000000000001},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.48299999999999998,"5min":0.57099999999999995,"15min":0.57099999999999995},"last":0.46500000000000002}]},{"osd":1,"last update":"Mon Sep 16 00:22:52 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.33900000000000002,"15min":0.33800000000000002},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.502,"5min":0.502,"15min":0.502},"last":0.44}]},{"osd":3,"last update":"Mon Sep 16 00:22:13 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.56799999999999995}]},{"osd":4,"last update":"Mon Sep 16 00:22:41 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.622}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]}]},{"osd":1,"up_from":14,"seq":60129542177,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298284,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446868,"statfs":{"total":95995035648,"available":95689592832,"internally_reserved":0,"allocated":942080,"data_stored":721534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 00:22:27 2024","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.47699999999999998,"5min":0.50800000000000001,"15min":0.50800000000000001},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.34599999999999997,"5min":0.35199999999999998,"15min":0.35299999999999998},"min":{"1min":0.23799999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"last":0.36399999999999999}]},{"osd":2,"last update":"Mon Sep 16 00:21:52 2024","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.372,"5min":0.372,"15min":0.372},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"last":0.40300000000000002}]},{"osd":3,"last update":"Mon Sep 16 00:22:15 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.59599999999999997}]},{"osd":4,"last update":"Mon Sep 16 00:22:37 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.55800000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]}]},{"osd":0,"up_from":9,"seq":38654705704,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298276,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446876,"statfs":{"total":95995035648,"available":95689601024,"internally_reserved":0,"allocated":942080,"data_stored":721534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 00:22:25 2024","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.312,"15min":0.311},"min":{"1min":0.249,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.42099999999999999,"5min":0.495,"15min":0.495},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.32800000000000001,"15min":0.32600000000000001},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.45500000000000002,"5min":0.46700000000000003,"15min":0.46700000000000003},"last":0.29199999999999998}]},{"osd":2,"last update":"Mon Sep 16 00:21:52 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.376,"5min":0.376,"15min":0.376},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.496,"5min":0.496,"15min":0.496},"last":0.30599999999999999}]},{"osd":3,"last update":"Mon Sep 16 00:22:14 2024","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":11.175000000000001,"5min":11.175000000000001,"15min":11.175000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.48399999999999999}]},{"osd":4,"last update":"Mon Sep 16 00:22:40 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.51600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T00:22:59.081 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-16T00:22:59.082 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-16T00:22:59.082 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-16T00:22:59.082 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph health --format=json 2024-09-16T00:22:59.326 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:22:59.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:22:59 smithi112 ceph-mon[27869]: from='client.14424 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:22:59.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:22:59 smithi169 ceph-mon[34653]: from='client.14424 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:23:00.477 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:00 smithi112 ceph-mon[27869]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:23:00.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:00 smithi169 ceph-mon[34653]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:23:01.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:01.061+0000 7f6866b89640 1 -- 172.21.15.112:0/819944654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68600fb4a0 msgr2=0x7f68600fb910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:01.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:01.061+0000 7f6866b89640 1 --2- 172.21.15.112:0/819944654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68600fb4a0 0x7f68600fb910 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f684c0048d0 tx=0x7f684c009da0 comp rx=0 tx=0).stop 2024-09-16T00:23:01.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:01.061+0000 7f6866b89640 1 -- 172.21.15.112:0/819944654 shutdown_connections 2024-09-16T00:23:01.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:01.061+0000 7f6866b89640 1 --2- 172.21.15.112:0/819944654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68600fb4a0 0x7f68600fb910 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:01.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:01.061+0000 7f6866b89640 1 --2- 172.21.15.112:0/819944654 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68601014e0 0x7f68601018b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:01.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:01.061+0000 7f6866b89640 1 -- 172.21.15.112:0/819944654 >> 172.21.15.112:0/819944654 conn(0x7f68600f7060 msgr2=0x7f68600f9470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:01.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:01.061+0000 7f6866b89640 1 -- 172.21.15.112:0/819944654 shutdown_connections 2024-09-16T00:23:01.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:01.061+0000 7f6866b89640 1 -- 172.21.15.112:0/819944654 wait complete. 2024-09-16T00:23:01.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:01.062+0000 7f6866b89640 1 Processor -- start 2024-09-16T00:23:01.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:01.062+0000 7f6866b89640 1 -- start start 2024-09-16T00:23:01.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:01.063+0000 7f6866b89640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68600fb4a0 0x7f6860112130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:01.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:01.063+0000 7f6866b89640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68601014e0 0x7f6860112670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:01.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:01.063+0000 7f6866b89640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6860112d00 con 0x7f68601014e0 2024-09-16T00:23:01.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:01.063+0000 7f6866b89640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f686010c0b0 con 0x7f68600fb4a0 2024-09-16T00:23:01.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:01.063+0000 7f685ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68601014e0 0x7f6860112670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:01.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:01.063+0000 7f685ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68601014e0 0x7f6860112670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45186/0 (socket says 172.21.15.112:45186) 2024-09-16T00:23:01.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:01.063+0000 7f685ffff640 1 -- 172.21.15.112:0/3628685651 learned_addr learned my addr 172.21.15.112:0/3628685651 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:23:01.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:01.063+0000 7f6864925640 1 --2- 172.21.15.112:0/3628685651 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68600fb4a0 0x7f6860112130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:01.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:01.063+0000 7f685ffff640 1 -- 172.21.15.112:0/3628685651 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68600fb4a0 msgr2=0x7f6860112130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:01.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:01.063+0000 7f685ffff640 1 --2- 172.21.15.112:0/3628685651 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68600fb4a0 0x7f6860112130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:01.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:01.063+0000 7f685ffff640 1 -- 172.21.15.112:0/3628685651 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f684c005040 con 0x7f68601014e0 2024-09-16T00:23:01.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:01.063+0000 7f685ffff640 1 --2- 172.21.15.112:0/3628685651 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68601014e0 0x7f6860112670 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f684c003080 tx=0x7f684c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:01.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:01.063+0000 7f685d7fa640 1 -- 172.21.15.112:0/3628685651 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f684c038070 con 0x7f68601014e0 2024-09-16T00:23:01.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:01.063+0000 7f6866b89640 1 -- 172.21.15.112:0/3628685651 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f686010c330 con 0x7f68601014e0 2024-09-16T00:23:01.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:01.063+0000 7f685d7fa640 1 -- 172.21.15.112:0/3628685651 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f684c0097e0 con 0x7f68601014e0 2024-09-16T00:23:01.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:01.063+0000 7f685d7fa640 1 -- 172.21.15.112:0/3628685651 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f684c03d420 con 0x7f68601014e0 2024-09-16T00:23:01.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:01.063+0000 7f6866b89640 1 -- 172.21.15.112:0/3628685651 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f686010c7f0 con 0x7f68601014e0 2024-09-16T00:23:01.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:01.064+0000 7f6866b89640 1 -- 172.21.15.112:0/3628685651 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68600fcbe0 con 0x7f68601014e0 2024-09-16T00:23:01.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:01.065+0000 7f685d7fa640 1 -- 172.21.15.112:0/3628685651 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) v1 ==== 90869+0+0 (secure 0 0 0) 0x7f684c030020 con 0x7f68601014e0 2024-09-16T00:23:01.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:01.065+0000 7f685d7fa640 1 --2- 172.21.15.112:0/3628685651 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f6850070f70 0x7f6850073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:01.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:01.065+0000 7f6864925640 1 --2- 172.21.15.112:0/3628685651 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f6850070f70 0x7f6850073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:01.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:01.065+0000 7f685d7fa640 1 -- 172.21.15.112:0/3628685651 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f684c0ae470 con 0x7f68601014e0 2024-09-16T00:23:01.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:01.066+0000 7f6864925640 1 --2- 172.21.15.112:0/3628685651 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f6850070f70 0x7f6850073420 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6854006fd0 tx=0x7f6854008040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:01.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:01.069+0000 7f685d7fa640 1 -- 172.21.15.112:0/3628685651 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f684c07b8d0 con 0x7f68601014e0 2024-09-16T00:23:01.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:01.256+0000 7f6866b89640 1 -- 172.21.15.112:0/3628685651 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f6860108510 con 0x7f68601014e0 2024-09-16T00:23:01.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:01.257+0000 7f685d7fa640 1 -- 172.21.15.112:0/3628685651 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f684c07b270 con 0x7f68601014e0 2024-09-16T00:23:01.257 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:23:01.258 INFO:teuthology.orchestra.run.smithi112.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-16T00:23:01.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:01.259+0000 7f6866b89640 1 -- 172.21.15.112:0/3628685651 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f6850070f70 msgr2=0x7f6850073420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:01.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:01.259+0000 7f6866b89640 1 --2- 172.21.15.112:0/3628685651 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f6850070f70 0x7f6850073420 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6854006fd0 tx=0x7f6854008040 comp rx=0 tx=0).stop 2024-09-16T00:23:01.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:01.259+0000 7f6866b89640 1 -- 172.21.15.112:0/3628685651 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68601014e0 msgr2=0x7f6860112670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:01.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:01.259+0000 7f6866b89640 1 --2- 172.21.15.112:0/3628685651 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68601014e0 0x7f6860112670 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f684c003080 tx=0x7f684c009da0 comp rx=0 tx=0).stop 2024-09-16T00:23:01.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:01.259+0000 7f6866b89640 1 -- 172.21.15.112:0/3628685651 shutdown_connections 2024-09-16T00:23:01.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:01.259+0000 7f6866b89640 1 --2- 172.21.15.112:0/3628685651 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f6850070f70 0x7f6850073420 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:01.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:01.259+0000 7f6866b89640 1 --2- 172.21.15.112:0/3628685651 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68601014e0 0x7f6860112670 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:01.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:01.259+0000 7f6866b89640 1 --2- 172.21.15.112:0/3628685651 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68600fb4a0 0x7f6860112130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:01.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:01.259+0000 7f6866b89640 1 -- 172.21.15.112:0/3628685651 >> 172.21.15.112:0/3628685651 conn(0x7f68600f7060 msgr2=0x7f68600f8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:01.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:01.259+0000 7f6866b89640 1 -- 172.21.15.112:0/3628685651 shutdown_connections 2024-09-16T00:23:01.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:01.259+0000 7f6866b89640 1 -- 172.21.15.112:0/3628685651 wait complete. 2024-09-16T00:23:01.977 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-16T00:23:01.977 INFO:tasks.cephadm:Setup complete, yielding 2024-09-16T00:23:01.977 INFO:teuthology.run_tasks:Running task print... 2024-09-16T00:23:01.988 INFO:teuthology.task.print:**** done end installing quincy cephadm ... 2024-09-16T00:23:01.988 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-16T00:23:02.061 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi112.front.sepia.ceph.com 2024-09-16T00:23:02.062 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- bash -c 'ceph config set mgr mgr/cephadm/use_repo_digest true --force' 2024-09-16T00:23:02.315 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:23:02.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:02 smithi112 ceph-mon[27869]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:23:02.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:02 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/3628685651' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-16T00:23:02.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:02 smithi169 ceph-mon[34653]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:23:02.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:02 smithi169 ceph-mon[34653]: from='client.? 172.21.15.112:0/3628685651' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-16T00:23:04.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:04.102+0000 7fb58e6f4640 1 -- 172.21.15.112:0/2672324600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5880fd820 msgr2=0x7fb5880fdc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:04.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:04.102+0000 7fb58e6f4640 1 --2- 172.21.15.112:0/2672324600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5880fd820 0x7fb5880fdc90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb5700048d0 tx=0x7fb570009da0 comp rx=0 tx=0).stop 2024-09-16T00:23:04.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:04.102+0000 7fb58e6f4640 1 -- 172.21.15.112:0/2672324600 shutdown_connections 2024-09-16T00:23:04.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:04.102+0000 7fb58e6f4640 1 --2- 172.21.15.112:0/2672324600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5880fd820 0x7fb5880fdc90 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:04.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:04.102+0000 7fb58e6f4640 1 --2- 172.21.15.112:0/2672324600 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5880fc550 0x7fb5880fc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:04.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:04.102+0000 7fb58e6f4640 1 -- 172.21.15.112:0/2672324600 >> 172.21.15.112:0/2672324600 conn(0x7fb5880f7d30 msgr2=0x7fb5880fa140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:04.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:04.102+0000 7fb58e6f4640 1 -- 172.21.15.112:0/2672324600 shutdown_connections 2024-09-16T00:23:04.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:04.102+0000 7fb58e6f4640 1 -- 172.21.15.112:0/2672324600 wait complete. 2024-09-16T00:23:04.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:04.103+0000 7fb58e6f4640 1 Processor -- start 2024-09-16T00:23:04.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:04.103+0000 7fb58e6f4640 1 -- start start 2024-09-16T00:23:04.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:04.103+0000 7fb58e6f4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5880fc550 0x7fb58810d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:04.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:04.103+0000 7fb58e6f4640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5880fd820 0x7fb58810d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:04.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:04.103+0000 7fb58e6f4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb58810dc00 con 0x7fb5880fc550 2024-09-16T00:23:04.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:04.103+0000 7fb58e6f4640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb58810e420 con 0x7fb5880fd820 2024-09-16T00:23:04.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:04.103+0000 7fb587fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5880fc550 0x7fb58810d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:04.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:04.103+0000 7fb587fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5880fc550 0x7fb58810d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51242/0 (socket says 172.21.15.112:51242) 2024-09-16T00:23:04.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:04.103+0000 7fb587fff640 1 -- 172.21.15.112:0/3992346825 learned_addr learned my addr 172.21.15.112:0/3992346825 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:23:04.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:04.103+0000 7fb57ffff640 1 --2- 172.21.15.112:0/3992346825 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5880fd820 0x7fb58810d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:04.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:04.103+0000 7fb587fff640 1 -- 172.21.15.112:0/3992346825 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5880fd820 msgr2=0x7fb58810d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:04.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:04.103+0000 7fb587fff640 1 --2- 172.21.15.112:0/3992346825 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5880fd820 0x7fb58810d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:04.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:04.103+0000 7fb587fff640 1 -- 172.21.15.112:0/3992346825 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb570005040 con 0x7fb5880fc550 2024-09-16T00:23:04.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:04.103+0000 7fb57ffff640 1 --2- 172.21.15.112:0/3992346825 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5880fd820 0x7fb58810d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:23:04.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:04.103+0000 7fb587fff640 1 --2- 172.21.15.112:0/3992346825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5880fc550 0x7fb58810d070 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb57800c810 tx=0x7fb5780043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:04.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:04.104+0000 7fb5857fa640 1 -- 172.21.15.112:0/3992346825 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb57800a820 con 0x7fb5880fc550 2024-09-16T00:23:04.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:04.104+0000 7fb5857fa640 1 -- 172.21.15.112:0/3992346825 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fb57800ae20 con 0x7fb5880fc550 2024-09-16T00:23:04.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:04.104+0000 7fb58e6f4640 1 -- 172.21.15.112:0/3992346825 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb58810e6b0 con 0x7fb5880fc550 2024-09-16T00:23:04.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:04.104+0000 7fb5857fa640 1 -- 172.21.15.112:0/3992346825 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5780115c0 con 0x7fb5880fc550 2024-09-16T00:23:04.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:04.104+0000 7fb58e6f4640 1 -- 172.21.15.112:0/3992346825 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb58810ec50 con 0x7fb5880fc550 2024-09-16T00:23:04.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:04.105+0000 7fb58e6f4640 1 -- 172.21.15.112:0/3992346825 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5880fb9b0 con 0x7fb5880fc550 2024-09-16T00:23:04.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:04.105+0000 7fb5857fa640 1 -- 172.21.15.112:0/3992346825 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) v1 ==== 90869+0+0 (secure 0 0 0) 0x7fb578012840 con 0x7fb5880fc550 2024-09-16T00:23:04.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:04.106+0000 7fb5857fa640 1 --2- 172.21.15.112:0/3992346825 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fb57406ed60 0x7fb574071210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:04.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:04.106+0000 7fb5857fa640 1 -- 172.21.15.112:0/3992346825 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fb57808e490 con 0x7fb5880fc550 2024-09-16T00:23:04.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:04.106+0000 7fb57ffff640 1 --2- 172.21.15.112:0/3992346825 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fb57406ed60 0x7fb574071210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:04.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:04.107+0000 7fb57ffff640 1 --2- 172.21.15.112:0/3992346825 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fb57406ed60 0x7fb574071210 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb570007ac0 tx=0x7fb570035040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:04.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:04.110+0000 7fb5857fa640 1 -- 172.21.15.112:0/3992346825 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fb57805b3d0 con 0x7fb5880fc550 2024-09-16T00:23:04.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:04.267+0000 7fb58e6f4640 1 -- 172.21.15.112:0/3992346825 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/use_repo_digest}] v 0) v1 -- 0x7fb5881050c0 con 0x7fb5880fc550 2024-09-16T00:23:04.268 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:04 smithi112 ceph-mon[27869]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:23:04.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:04.272+0000 7fb5857fa640 1 -- 172.21.15.112:0/3992346825 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/use_repo_digest}]=0 v20)=0 v20) v1 ==== 143+0+0 (secure 0 0 0) 0x7fb578012e50 con 0x7fb5880fc550 2024-09-16T00:23:04.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:04.274+0000 7fb58e6f4640 1 -- 172.21.15.112:0/3992346825 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fb57406ed60 msgr2=0x7fb574071210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:04.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:04.274+0000 7fb58e6f4640 1 --2- 172.21.15.112:0/3992346825 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fb57406ed60 0x7fb574071210 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb570007ac0 tx=0x7fb570035040 comp rx=0 tx=0).stop 2024-09-16T00:23:04.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:04.274+0000 7fb58e6f4640 1 -- 172.21.15.112:0/3992346825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5880fc550 msgr2=0x7fb58810d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:04.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:04.274+0000 7fb58e6f4640 1 --2- 172.21.15.112:0/3992346825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5880fc550 0x7fb58810d070 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb57800c810 tx=0x7fb5780043c0 comp rx=0 tx=0).stop 2024-09-16T00:23:04.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:04.274+0000 7fb58e6f4640 1 -- 172.21.15.112:0/3992346825 shutdown_connections 2024-09-16T00:23:04.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:04.274+0000 7fb58e6f4640 1 --2- 172.21.15.112:0/3992346825 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fb57406ed60 0x7fb574071210 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:04.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:04.274+0000 7fb58e6f4640 1 --2- 172.21.15.112:0/3992346825 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5880fd820 0x7fb58810d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:04.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:04.274+0000 7fb58e6f4640 1 --2- 172.21.15.112:0/3992346825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5880fc550 0x7fb58810d070 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:04.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:04.274+0000 7fb58e6f4640 1 -- 172.21.15.112:0/3992346825 >> 172.21.15.112:0/3992346825 conn(0x7fb5880f7d30 msgr2=0x7fb5880f94c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:04.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:04.274+0000 7fb58e6f4640 1 -- 172.21.15.112:0/3992346825 shutdown_connections 2024-09-16T00:23:04.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:04.274+0000 7fb58e6f4640 1 -- 172.21.15.112:0/3992346825 wait complete. 2024-09-16T00:23:04.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:04 smithi169 ceph-mon[34653]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:23:05.008 INFO:teuthology.run_tasks:Running task print... 2024-09-16T00:23:05.021 INFO:teuthology.task.print:**** done cephadm.shell ceph config set mgr... 2024-09-16T00:23:05.021 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-16T00:23:05.115 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi112.front.sepia.ceph.com 2024-09-16T00:23:05.115 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- bash -c 'ceph orch status' 2024-09-16T00:23:05.360 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:23:05.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:05 smithi169 ceph-mon[34653]: from='client.? 172.21.15.112:0/3992346825' entity='client.admin' 2024-09-16T00:23:05.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:05 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:23:05.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:05 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:23:05.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:05 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:23:05.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:05 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:23:05.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:05 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/3992346825' entity='client.admin' 2024-09-16T00:23:05.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:05 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:23:05.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:05 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:23:05.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:05 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:23:05.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:05 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:23:06.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:06 smithi169 ceph-mon[34653]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:23:06.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:06 smithi112 ceph-mon[27869]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:23:07.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:07.050+0000 7fb657f67640 1 -- 172.21.15.112:0/3387701617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6500fc720 msgr2=0x7fb6500fcb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:07.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:07.050+0000 7fb657f67640 1 --2- 172.21.15.112:0/3387701617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6500fc720 0x7fb6500fcb90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb64c0048d0 tx=0x7fb64c009da0 comp rx=0 tx=0).stop 2024-09-16T00:23:07.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:07.050+0000 7fb657f67640 1 -- 172.21.15.112:0/3387701617 shutdown_connections 2024-09-16T00:23:07.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:07.050+0000 7fb657f67640 1 --2- 172.21.15.112:0/3387701617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6500fc720 0x7fb6500fcb90 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:07.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:07.050+0000 7fb657f67640 1 --2- 172.21.15.112:0/3387701617 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6500fb450 0x7fb6500fb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:07.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:07.050+0000 7fb657f67640 1 -- 172.21.15.112:0/3387701617 >> 172.21.15.112:0/3387701617 conn(0x7fb6500f6c10 msgr2=0x7fb6500f9040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:07.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:07.050+0000 7fb657f67640 1 -- 172.21.15.112:0/3387701617 shutdown_connections 2024-09-16T00:23:07.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:07.050+0000 7fb657f67640 1 -- 172.21.15.112:0/3387701617 wait complete. 2024-09-16T00:23:07.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:07.050+0000 7fb657f67640 1 Processor -- start 2024-09-16T00:23:07.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:07.051+0000 7fb657f67640 1 -- start start 2024-09-16T00:23:07.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:07.051+0000 7fb657f67640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6500fb450 0x7fb65010bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:07.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:07.051+0000 7fb657f67640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6500fc720 0x7fb65010c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:07.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:07.051+0000 7fb657f67640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb65010cb00 con 0x7fb6500fc720 2024-09-16T00:23:07.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:07.051+0000 7fb657f67640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb65010d320 con 0x7fb6500fb450 2024-09-16T00:23:07.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:07.051+0000 7fb655502640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6500fc720 0x7fb65010c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:07.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:07.051+0000 7fb655502640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6500fc720 0x7fb65010c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51252/0 (socket says 172.21.15.112:51252) 2024-09-16T00:23:07.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:07.051+0000 7fb655502640 1 -- 172.21.15.112:0/3177932679 learned_addr learned my addr 172.21.15.112:0/3177932679 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:23:07.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:07.051+0000 7fb655502640 1 -- 172.21.15.112:0/3177932679 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6500fb450 msgr2=0x7fb65010bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:07.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:07.051+0000 7fb655502640 1 --2- 172.21.15.112:0/3177932679 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6500fb450 0x7fb65010bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:07.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:07.051+0000 7fb655502640 1 -- 172.21.15.112:0/3177932679 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb64c005040 con 0x7fb6500fc720 2024-09-16T00:23:07.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:07.051+0000 7fb655502640 1 --2- 172.21.15.112:0/3177932679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6500fc720 0x7fb65010c4b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb64c0043d0 tx=0x7fb64c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:07.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:07.051+0000 7fb6427fc640 1 -- 172.21.15.112:0/3177932679 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb64c038070 con 0x7fb6500fc720 2024-09-16T00:23:07.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:07.051+0000 7fb6427fc640 1 -- 172.21.15.112:0/3177932679 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fb64c02ae90 con 0x7fb6500fc720 2024-09-16T00:23:07.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:07.051+0000 7fb657f67640 1 -- 172.21.15.112:0/3177932679 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb65010d550 con 0x7fb6500fc720 2024-09-16T00:23:07.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:07.051+0000 7fb6427fc640 1 -- 172.21.15.112:0/3177932679 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb64c03e670 con 0x7fb6500fc720 2024-09-16T00:23:07.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:07.051+0000 7fb657f67640 1 -- 172.21.15.112:0/3177932679 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb65010daf0 con 0x7fb6500fc720 2024-09-16T00:23:07.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:07.052+0000 7fb657f67640 1 -- 172.21.15.112:0/3177932679 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6500fa8b0 con 0x7fb6500fc720 2024-09-16T00:23:07.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:07.054+0000 7fb6427fc640 1 -- 172.21.15.112:0/3177932679 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) v1 ==== 90869+0+0 (secure 0 0 0) 0x7fb64c030020 con 0x7fb6500fc720 2024-09-16T00:23:07.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:07.054+0000 7fb6427fc640 1 --2- 172.21.15.112:0/3177932679 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fb63c06ed60 0x7fb63c071210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:07.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:07.054+0000 7fb6427fc640 1 -- 172.21.15.112:0/3177932679 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fb64c0ae020 con 0x7fb6500fc720 2024-09-16T00:23:07.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:07.054+0000 7fb655d03640 1 --2- 172.21.15.112:0/3177932679 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fb63c06ed60 0x7fb63c071210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:07.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:07.055+0000 7fb655d03640 1 --2- 172.21.15.112:0/3177932679 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fb63c06ed60 0x7fb63c071210 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb644004c80 tx=0x7fb644009c50 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:07.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:07.058+0000 7fb6427fc640 1 -- 172.21.15.112:0/3177932679 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fb64c07b500 con 0x7fb6500fc720 2024-09-16T00:23:07.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:07.226+0000 7fb657f67640 1 -- 172.21.15.112:0/3177932679 --> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb65010ded0 con 0x7fb63c06ed60 2024-09-16T00:23:07.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:07.227+0000 7fb6427fc640 1 -- 172.21.15.112:0/3177932679 <== mgr.14208 v2:172.21.15.112:6800/2480414866 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fb65010ded0 con 0x7fb63c06ed60 2024-09-16T00:23:07.228 INFO:teuthology.orchestra.run.smithi112.stdout:Backend: cephadm 2024-09-16T00:23:07.228 INFO:teuthology.orchestra.run.smithi112.stdout:Available: Yes 2024-09-16T00:23:07.228 INFO:teuthology.orchestra.run.smithi112.stdout:Paused: No 2024-09-16T00:23:07.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:07.229+0000 7fb657f67640 1 -- 172.21.15.112:0/3177932679 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fb63c06ed60 msgr2=0x7fb63c071210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:07.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:07.229+0000 7fb657f67640 1 --2- 172.21.15.112:0/3177932679 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fb63c06ed60 0x7fb63c071210 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb644004c80 tx=0x7fb644009c50 comp rx=0 tx=0).stop 2024-09-16T00:23:07.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:07.229+0000 7fb657f67640 1 -- 172.21.15.112:0/3177932679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6500fc720 msgr2=0x7fb65010c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:07.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:07.229+0000 7fb657f67640 1 --2- 172.21.15.112:0/3177932679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6500fc720 0x7fb65010c4b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb64c0043d0 tx=0x7fb64c009da0 comp rx=0 tx=0).stop 2024-09-16T00:23:07.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:07.229+0000 7fb657f67640 1 -- 172.21.15.112:0/3177932679 shutdown_connections 2024-09-16T00:23:07.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:07.229+0000 7fb657f67640 1 --2- 172.21.15.112:0/3177932679 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fb63c06ed60 0x7fb63c071210 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:07.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:07.229+0000 7fb657f67640 1 --2- 172.21.15.112:0/3177932679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6500fc720 0x7fb65010c4b0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:07.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:07.229+0000 7fb657f67640 1 --2- 172.21.15.112:0/3177932679 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6500fb450 0x7fb65010bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:07.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:07.229+0000 7fb657f67640 1 -- 172.21.15.112:0/3177932679 >> 172.21.15.112:0/3177932679 conn(0x7fb6500f6c10 msgr2=0x7fb6500f83b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:07.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:07.229+0000 7fb657f67640 1 -- 172.21.15.112:0/3177932679 shutdown_connections 2024-09-16T00:23:07.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:07.229+0000 7fb657f67640 1 -- 172.21.15.112:0/3177932679 wait complete. 2024-09-16T00:23:07.569 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:07 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:23:07.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:07 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:23:07.899 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-16T00:23:08.144 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:23:08.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:08 smithi112 ceph-mon[27869]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:23:08.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:08 smithi112 ceph-mon[27869]: from='client.14436 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:23:08.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:08 smithi169 ceph-mon[34653]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:23:08.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:08 smithi169 ceph-mon[34653]: from='client.14436 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:23:09.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:09.685+0000 7f6065936640 1 -- 172.21.15.112:0/1108800160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60600ff6a0 msgr2=0x7f6060101a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:09.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:09.685+0000 7f6065936640 1 --2- 172.21.15.112:0/1108800160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60600ff6a0 0x7f6060101a80 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f60540048d0 tx=0x7f6054009da0 comp rx=0 tx=0).stop 2024-09-16T00:23:09.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:09.685+0000 7f6065936640 1 -- 172.21.15.112:0/1108800160 shutdown_connections 2024-09-16T00:23:09.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:09.685+0000 7f6065936640 1 --2- 172.21.15.112:0/1108800160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60600ff6a0 0x7f6060101a80 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:09.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:09.685+0000 7f6065936640 1 --2- 172.21.15.112:0/1108800160 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6060066790 0x7f60600ff160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:09.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:09.685+0000 7f6065936640 1 -- 172.21.15.112:0/1108800160 >> 172.21.15.112:0/1108800160 conn(0x7f60600f6c10 msgr2=0x7f60600f9040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:09.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:09.685+0000 7f6065936640 1 -- 172.21.15.112:0/1108800160 shutdown_connections 2024-09-16T00:23:09.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:09.685+0000 7f6065936640 1 -- 172.21.15.112:0/1108800160 wait complete. 2024-09-16T00:23:09.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:09.686+0000 7f6065936640 1 Processor -- start 2024-09-16T00:23:09.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:09.686+0000 7f6065936640 1 -- start start 2024-09-16T00:23:09.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:09.686+0000 7f6065936640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6060066790 0x7f606010d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:09.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:09.686+0000 7f6065936640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60600ff6a0 0x7f606010d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:09.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:09.687+0000 7f6065936640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f606010dc40 con 0x7f60600ff6a0 2024-09-16T00:23:09.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:09.687+0000 7f6065936640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f606010e460 con 0x7f6060066790 2024-09-16T00:23:09.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:09.687+0000 7f605e7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60600ff6a0 0x7f606010d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:09.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:09.687+0000 7f605e7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60600ff6a0 0x7f606010d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51274/0 (socket says 172.21.15.112:51274) 2024-09-16T00:23:09.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:09.687+0000 7f605e7fc640 1 -- 172.21.15.112:0/1908614303 learned_addr learned my addr 172.21.15.112:0/1908614303 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:23:09.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:09.687+0000 7f605effd640 1 --2- 172.21.15.112:0/1908614303 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6060066790 0x7f606010d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:09.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:09.687+0000 7f605e7fc640 1 -- 172.21.15.112:0/1908614303 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6060066790 msgr2=0x7f606010d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:09.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:09.687+0000 7f605e7fc640 1 --2- 172.21.15.112:0/1908614303 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6060066790 0x7f606010d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:09.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:09.687+0000 7f605e7fc640 1 -- 172.21.15.112:0/1908614303 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6054005040 con 0x7f60600ff6a0 2024-09-16T00:23:09.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:09.687+0000 7f605effd640 1 --2- 172.21.15.112:0/1908614303 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6060066790 0x7f606010d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:23:09.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:09.687+0000 7f605e7fc640 1 --2- 172.21.15.112:0/1908614303 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60600ff6a0 0x7f606010d5f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f60540043d0 tx=0x7f6054009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:09.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:09.687+0000 7f60477fe640 1 -- 172.21.15.112:0/1908614303 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6054038070 con 0x7f60600ff6a0 2024-09-16T00:23:09.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:09.687+0000 7f6065936640 1 -- 172.21.15.112:0/1908614303 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f606010e690 con 0x7f60600ff6a0 2024-09-16T00:23:09.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:09.687+0000 7f60477fe640 1 -- 172.21.15.112:0/1908614303 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f605402ae90 con 0x7f60600ff6a0 2024-09-16T00:23:09.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:09.687+0000 7f60477fe640 1 -- 172.21.15.112:0/1908614303 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f605403e670 con 0x7f60600ff6a0 2024-09-16T00:23:09.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:09.687+0000 7f6065936640 1 -- 172.21.15.112:0/1908614303 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f606010ec30 con 0x7f60600ff6a0 2024-09-16T00:23:09.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:09.688+0000 7f6065936640 1 -- 172.21.15.112:0/1908614303 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60600fa8b0 con 0x7f60600ff6a0 2024-09-16T00:23:09.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:09.689+0000 7f60477fe640 1 -- 172.21.15.112:0/1908614303 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) v1 ==== 90869+0+0 (secure 0 0 0) 0x7f6054030020 con 0x7f60600ff6a0 2024-09-16T00:23:09.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:09.689+0000 7f60477fe640 1 --2- 172.21.15.112:0/1908614303 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f604806ed60 0x7f6048071210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:09.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:09.689+0000 7f60477fe640 1 -- 172.21.15.112:0/1908614303 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f60540ae030 con 0x7f60600ff6a0 2024-09-16T00:23:09.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:09.689+0000 7f605effd640 1 --2- 172.21.15.112:0/1908614303 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f604806ed60 0x7f6048071210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:09.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:09.690+0000 7f605effd640 1 --2- 172.21.15.112:0/1908614303 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f604806ed60 0x7f6048071210 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6050004fd0 tx=0x7f605000b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:09.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:09.693+0000 7f60477fe640 1 -- 172.21.15.112:0/1908614303 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f605407b540 con 0x7f60600ff6a0 2024-09-16T00:23:09.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:09.858+0000 7f6065936640 1 -- 172.21.15.112:0/1908614303 --> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f606010ef60 con 0x7f604806ed60 2024-09-16T00:23:09.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:09.865+0000 7f60477fe640 1 -- 172.21.15.112:0/1908614303 <== mgr.14208 v2:172.21.15.112:6800/2480414866 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+2790 (secure 0 0 0) 0x7f606010ef60 con 0x7f604806ed60 2024-09-16T00:23:09.867 INFO:teuthology.orchestra.run.smithi112.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T00:23:09.867 INFO:teuthology.orchestra.run.smithi112.stdout:alertmanager.smithi112 smithi112 *:9093,9094 running (3m) 2m ago 5m 25.7M - 0.25.0 c8568f914cd2 89acc17eab5c 2024-09-16T00:23:09.867 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi112 smithi112 running (5m) 2m ago 5m 6891k - 17.2.7-1633-g6d025bc2 33ed8dba3996 c94220ed2776 2024-09-16T00:23:09.867 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi169 smithi169 running (4m) 69s ago 4m 6899k - 17.2.7-1633-g6d025bc2 33ed8dba3996 d0d3726280a2 2024-09-16T00:23:09.868 INFO:teuthology.orchestra.run.smithi112.stdout:grafana.smithi112 smithi112 *:3000 running (3m) 2m ago 4m 79.8M - 9.4.7 954c08fa6188 ffeb9c3bdf66 2024-09-16T00:23:09.868 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi112.xrkkbc smithi112 *:9283,8765,8443 running (6m) 2m ago 6m 441M - 17.2.7-1633-g6d025bc2 33ed8dba3996 b3d3adb94a85 2024-09-16T00:23:09.868 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi169.mtrobk smithi169 *:8443,9283 running (4m) 69s ago 4m 404M - 17.2.7-1633-g6d025bc2 33ed8dba3996 1e67dbe21770 2024-09-16T00:23:09.868 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi112 smithi112 running (6m) 2m ago 6m 43.7M 2048M 17.2.7-1633-g6d025bc2 33ed8dba3996 ddaaced54bf8 2024-09-16T00:23:09.868 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi169 smithi169 running (4m) 69s ago 4m 41.3M 2048M 17.2.7-1633-g6d025bc2 33ed8dba3996 6c2289bc3e11 2024-09-16T00:23:09.868 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi112 smithi112 *:9100 running (5m) 2m ago 5m 15.6M - 1.5.0 0da6a335fe13 89c6f32288bc 2024-09-16T00:23:09.868 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi169 smithi169 *:9100 running (4m) 69s ago 4m 15.7M - 1.5.0 0da6a335fe13 5d6fe1dc6fa9 2024-09-16T00:23:09.868 INFO:teuthology.orchestra.run.smithi112.stdout:osd.0 smithi112 running (3m) 2m ago 3m 39.3M 4288M 17.2.7-1633-g6d025bc2 33ed8dba3996 f6b2e146ffdf 2024-09-16T00:23:09.868 INFO:teuthology.orchestra.run.smithi112.stdout:osd.1 smithi112 running (2m) 2m ago 2m 40.3M 4288M 17.2.7-1633-g6d025bc2 33ed8dba3996 e9bee87bba0f 2024-09-16T00:23:09.868 INFO:teuthology.orchestra.run.smithi112.stdout:osd.2 smithi112 running (2m) 2m ago 2m 38.9M 4288M 17.2.7-1633-g6d025bc2 33ed8dba3996 100ec4e4852a 2024-09-16T00:23:09.868 INFO:teuthology.orchestra.run.smithi112.stdout:osd.3 smithi169 running (2m) 69s ago 2m 41.4M 5312M 17.2.7-1633-g6d025bc2 33ed8dba3996 f88463e19f23 2024-09-16T00:23:09.868 INFO:teuthology.orchestra.run.smithi112.stdout:osd.4 smithi169 running (99s) 69s ago 99s 37.0M 5312M 17.2.7-1633-g6d025bc2 33ed8dba3996 d3e4cdb016b9 2024-09-16T00:23:09.868 INFO:teuthology.orchestra.run.smithi112.stdout:osd.5 smithi169 running (73s) 69s ago 73s 13.5M 5312M 17.2.7-1633-g6d025bc2 33ed8dba3996 8e7d708d74eb 2024-09-16T00:23:09.869 INFO:teuthology.orchestra.run.smithi112.stdout:prometheus.smithi112 smithi112 *:9095 running (3m) 2m ago 4m 40.3M - 2.43.0 a07b618ecd1d 7f5a86d91011 2024-09-16T00:23:09.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:09.867+0000 7f6065936640 1 -- 172.21.15.112:0/1908614303 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f604806ed60 msgr2=0x7f6048071210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:09.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:09.867+0000 7f6065936640 1 --2- 172.21.15.112:0/1908614303 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f604806ed60 0x7f6048071210 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6050004fd0 tx=0x7f605000b040 comp rx=0 tx=0).stop 2024-09-16T00:23:09.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:09.867+0000 7f6065936640 1 -- 172.21.15.112:0/1908614303 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60600ff6a0 msgr2=0x7f606010d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:09.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:09.867+0000 7f6065936640 1 --2- 172.21.15.112:0/1908614303 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60600ff6a0 0x7f606010d5f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f60540043d0 tx=0x7f6054009da0 comp rx=0 tx=0).stop 2024-09-16T00:23:09.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:09.868+0000 7f6065936640 1 -- 172.21.15.112:0/1908614303 shutdown_connections 2024-09-16T00:23:09.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:09.868+0000 7f6065936640 1 --2- 172.21.15.112:0/1908614303 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f604806ed60 0x7f6048071210 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:09.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:09.868+0000 7f6065936640 1 --2- 172.21.15.112:0/1908614303 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60600ff6a0 0x7f606010d5f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:09.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:09.868+0000 7f6065936640 1 --2- 172.21.15.112:0/1908614303 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6060066790 0x7f606010d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:09.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:09.868+0000 7f6065936640 1 -- 172.21.15.112:0/1908614303 >> 172.21.15.112:0/1908614303 conn(0x7f60600f6c10 msgr2=0x7f60600f7030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:09.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:09.868+0000 7f6065936640 1 -- 172.21.15.112:0/1908614303 shutdown_connections 2024-09-16T00:23:09.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:09.868+0000 7f6065936640 1 -- 172.21.15.112:0/1908614303 wait complete. 2024-09-16T00:23:10.564 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:10 smithi112 ceph-mon[27869]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:23:10.590 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-16T00:23:10.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:10 smithi169 ceph-mon[34653]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:23:10.833 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:23:11.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:11 smithi112 ceph-mon[27869]: from='client.14440 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:23:11.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:11 smithi169 ceph-mon[34653]: from='client.14440 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:23:12.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:12.433+0000 7f095a3fb640 1 -- 172.21.15.112:0/1637307117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09540fc570 msgr2=0x7f09540fc980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:12.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:12.433+0000 7f095a3fb640 1 --2- 172.21.15.112:0/1637307117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09540fc570 0x7f09540fc980 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f09440048d0 tx=0x7f0944009da0 comp rx=0 tx=0).stop 2024-09-16T00:23:12.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:12.433+0000 7f095a3fb640 1 -- 172.21.15.112:0/1637307117 shutdown_connections 2024-09-16T00:23:12.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:12.433+0000 7f095a3fb640 1 --2- 172.21.15.112:0/1637307117 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09540fd840 0x7f09540fdcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:12.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:12.433+0000 7f095a3fb640 1 --2- 172.21.15.112:0/1637307117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09540fc570 0x7f09540fc980 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:12.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:12.433+0000 7f095a3fb640 1 -- 172.21.15.112:0/1637307117 >> 172.21.15.112:0/1637307117 conn(0x7f09540f7d30 msgr2=0x7f09540fa160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:12.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:12.433+0000 7f095a3fb640 1 -- 172.21.15.112:0/1637307117 shutdown_connections 2024-09-16T00:23:12.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:12.433+0000 7f095a3fb640 1 -- 172.21.15.112:0/1637307117 wait complete. 2024-09-16T00:23:12.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:12.433+0000 7f095a3fb640 1 Processor -- start 2024-09-16T00:23:12.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:12.433+0000 7f095a3fb640 1 -- start start 2024-09-16T00:23:12.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:12.434+0000 7f095a3fb640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09540fc570 0x7f0954071520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:12.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:12.434+0000 7f095a3fb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09540fd840 0x7f095406fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:12.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:12.434+0000 7f095a3fb640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0954110d10 con 0x7f09540fd840 2024-09-16T00:23:12.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:12.434+0000 7f095a3fb640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0954110e50 con 0x7f09540fc570 2024-09-16T00:23:12.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:12.434+0000 7f094bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09540fd840 0x7f095406fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:12.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:12.434+0000 7f094bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09540fd840 0x7f095406fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34246/0 (socket says 172.21.15.112:34246) 2024-09-16T00:23:12.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:12.434+0000 7f094bfff640 1 -- 172.21.15.112:0/2446787115 learned_addr learned my addr 172.21.15.112:0/2446787115 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:23:12.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:12.434+0000 7f0953fff640 1 --2- 172.21.15.112:0/2446787115 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09540fc570 0x7f0954071520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:12.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:12.434+0000 7f094bfff640 1 -- 172.21.15.112:0/2446787115 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09540fc570 msgr2=0x7f0954071520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:12.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:12.434+0000 7f094bfff640 1 --2- 172.21.15.112:0/2446787115 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09540fc570 0x7f0954071520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:12.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:12.434+0000 7f094bfff640 1 -- 172.21.15.112:0/2446787115 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0944005040 con 0x7f09540fd840 2024-09-16T00:23:12.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:12.434+0000 7f0953fff640 1 --2- 172.21.15.112:0/2446787115 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09540fc570 0x7f0954071520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:23:12.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:12.434+0000 7f094bfff640 1 --2- 172.21.15.112:0/2446787115 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09540fd840 0x7f095406fa60 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f093c002a50 tx=0x7f093c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:12.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:12.434+0000 7f09517fa640 1 -- 172.21.15.112:0/2446787115 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f093c007c30 con 0x7f09540fd840 2024-09-16T00:23:12.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:12.434+0000 7f09517fa640 1 -- 172.21.15.112:0/2446787115 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f093c00f040 con 0x7f09540fd840 2024-09-16T00:23:12.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:12.434+0000 7f09517fa640 1 -- 172.21.15.112:0/2446787115 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f093c00da90 con 0x7f09540fd840 2024-09-16T00:23:12.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:12.434+0000 7f095a3fb640 1 -- 172.21.15.112:0/2446787115 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0954070000 con 0x7f09540fd840 2024-09-16T00:23:12.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:12.435+0000 7f095a3fb640 1 -- 172.21.15.112:0/2446787115 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0954070570 con 0x7f09540fd840 2024-09-16T00:23:12.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:12.435+0000 7f095a3fb640 1 -- 172.21.15.112:0/2446787115 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09540fb9d0 con 0x7f09540fd840 2024-09-16T00:23:12.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:12.436+0000 7f09517fa640 1 -- 172.21.15.112:0/2446787115 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) v1 ==== 90869+0+0 (secure 0 0 0) 0x7f093c00dc30 con 0x7f09540fd840 2024-09-16T00:23:12.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:12.436+0000 7f09517fa640 1 --2- 172.21.15.112:0/2446787115 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f094006ed60 0x7f0940071210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:12.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:12.437+0000 7f0953fff640 1 --2- 172.21.15.112:0/2446787115 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f094006ed60 0x7f0940071210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:12.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:12.437+0000 7f09517fa640 1 -- 172.21.15.112:0/2446787115 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f093c08d690 con 0x7f09540fd840 2024-09-16T00:23:12.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:12.437+0000 7f0953fff640 1 --2- 172.21.15.112:0/2446787115 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f094006ed60 0x7f0940071210 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f094400b020 tx=0x7f0944009da0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:12.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:12.440+0000 7f09517fa640 1 -- 172.21.15.112:0/2446787115 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f093c04d2a0 con 0x7f09540fd840 2024-09-16T00:23:12.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:12 smithi112 ceph-mon[27869]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:23:12.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:12.606+0000 7f095a3fb640 1 -- 172.21.15.112:0/2446787115 --> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f0954101c20 con 0x7f094006ed60 2024-09-16T00:23:12.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:12.610+0000 7f09517fa640 1 -- 172.21.15.112:0/2446787115 <== mgr.14208 v2:172.21.15.112:6800/2480414866 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+1179 (secure 0 0 0) 0x7f0954101c20 con 0x7f094006ed60 2024-09-16T00:23:12.612 INFO:teuthology.orchestra.run.smithi112.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-16T00:23:12.612 INFO:teuthology.orchestra.run.smithi112.stdout:alertmanager ?:9093,9094 1/1 2m ago 5m count:1 2024-09-16T00:23:12.612 INFO:teuthology.orchestra.run.smithi112.stdout:crash 2/2 2m ago 5m * 2024-09-16T00:23:12.612 INFO:teuthology.orchestra.run.smithi112.stdout:grafana ?:3000 1/1 2m ago 5m count:1 2024-09-16T00:23:12.612 INFO:teuthology.orchestra.run.smithi112.stdout:mgr 2/2 2m ago 5m count:2 2024-09-16T00:23:12.612 INFO:teuthology.orchestra.run.smithi112.stdout:mon 2/2 2m ago 5m smithi112:172.21.15.112=smithi112;smithi169:172.21.15.169=smithi169;count:2 2024-09-16T00:23:12.612 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter ?:9100 2/2 2m ago 5m * 2024-09-16T00:23:12.612 INFO:teuthology.orchestra.run.smithi112.stdout:osd 6 2m ago - 2024-09-16T00:23:12.612 INFO:teuthology.orchestra.run.smithi112.stdout:prometheus ?:9095 1/1 2m ago 5m count:1 2024-09-16T00:23:12.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:12.612+0000 7f095a3fb640 1 -- 172.21.15.112:0/2446787115 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f094006ed60 msgr2=0x7f0940071210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:12.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:12.612+0000 7f095a3fb640 1 --2- 172.21.15.112:0/2446787115 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f094006ed60 0x7f0940071210 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f094400b020 tx=0x7f0944009da0 comp rx=0 tx=0).stop 2024-09-16T00:23:12.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:12.612+0000 7f095a3fb640 1 -- 172.21.15.112:0/2446787115 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09540fd840 msgr2=0x7f095406fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:12.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:12.612+0000 7f095a3fb640 1 --2- 172.21.15.112:0/2446787115 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09540fd840 0x7f095406fa60 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f093c002a50 tx=0x7f093c004600 comp rx=0 tx=0).stop 2024-09-16T00:23:12.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:12.613+0000 7f095a3fb640 1 -- 172.21.15.112:0/2446787115 shutdown_connections 2024-09-16T00:23:12.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:12.613+0000 7f095a3fb640 1 --2- 172.21.15.112:0/2446787115 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f094006ed60 0x7f0940071210 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:12.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:12.613+0000 7f095a3fb640 1 --2- 172.21.15.112:0/2446787115 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09540fd840 0x7f095406fa60 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:12.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:12.613+0000 7f095a3fb640 1 --2- 172.21.15.112:0/2446787115 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09540fc570 0x7f0954071520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:12.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:12.613+0000 7f095a3fb640 1 -- 172.21.15.112:0/2446787115 >> 172.21.15.112:0/2446787115 conn(0x7f09540f7d30 msgr2=0x7f09540f9500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:12.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:12.613+0000 7f095a3fb640 1 -- 172.21.15.112:0/2446787115 shutdown_connections 2024-09-16T00:23:12.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:12.613+0000 7f095a3fb640 1 -- 172.21.15.112:0/2446787115 wait complete. 2024-09-16T00:23:12.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:12 smithi169 ceph-mon[34653]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:23:13.312 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-16T00:23:13.555 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:23:14.577 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:14 smithi112 ceph-mon[27869]: from='client.14444 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:23:14.578 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:14 smithi112 ceph-mon[27869]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:23:14.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:14 smithi169 ceph-mon[34653]: from='client.14444 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:23:14.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:14 smithi169 ceph-mon[34653]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:23:15.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:15.049+0000 7f3ff6317640 1 -- 172.21.15.112:0/1076889499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ff00fd820 msgr2=0x7f3ff00fdc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:15.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:15.049+0000 7f3ff6317640 1 --2- 172.21.15.112:0/1076889499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ff00fd820 0x7f3ff00fdc90 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3fe40048d0 tx=0x7f3fe4009da0 comp rx=0 tx=0).stop 2024-09-16T00:23:15.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:15.049+0000 7f3ff6317640 1 -- 172.21.15.112:0/1076889499 shutdown_connections 2024-09-16T00:23:15.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:15.049+0000 7f3ff6317640 1 --2- 172.21.15.112:0/1076889499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ff00fd820 0x7f3ff00fdc90 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:15.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:15.049+0000 7f3ff6317640 1 --2- 172.21.15.112:0/1076889499 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ff00fc550 0x7f3ff00fc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:15.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:15.049+0000 7f3ff6317640 1 -- 172.21.15.112:0/1076889499 >> 172.21.15.112:0/1076889499 conn(0x7f3ff00f7d30 msgr2=0x7f3ff00fa140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:15.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:15.049+0000 7f3ff6317640 1 -- 172.21.15.112:0/1076889499 shutdown_connections 2024-09-16T00:23:15.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:15.049+0000 7f3ff6317640 1 -- 172.21.15.112:0/1076889499 wait complete. 2024-09-16T00:23:15.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:15.050+0000 7f3ff6317640 1 Processor -- start 2024-09-16T00:23:15.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:15.050+0000 7f3ff6317640 1 -- start start 2024-09-16T00:23:15.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:15.050+0000 7f3ff6317640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ff00fc550 0x7f3ff010d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:15.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:15.050+0000 7f3ff6317640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ff00fd820 0x7f3ff010d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:15.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:15.050+0000 7f3ff6317640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff010e490 con 0x7f3ff00fc550 2024-09-16T00:23:15.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:15.050+0000 7f3ff6317640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff010e600 con 0x7f3ff00fd820 2024-09-16T00:23:15.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:15.050+0000 7f3feffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ff00fc550 0x7f3ff010d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:15.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:15.050+0000 7f3feffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ff00fc550 0x7f3ff010d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34276/0 (socket says 172.21.15.112:34276) 2024-09-16T00:23:15.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:15.050+0000 7f3feffff640 1 -- 172.21.15.112:0/2372405920 learned_addr learned my addr 172.21.15.112:0/2372405920 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:23:15.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:15.050+0000 7f3fef7fe640 1 --2- 172.21.15.112:0/2372405920 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ff00fd820 0x7f3ff010d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:15.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:15.050+0000 7f3feffff640 1 -- 172.21.15.112:0/2372405920 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ff00fd820 msgr2=0x7f3ff010d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:15.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:15.050+0000 7f3feffff640 1 --2- 172.21.15.112:0/2372405920 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ff00fd820 0x7f3ff010d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:15.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:15.050+0000 7f3feffff640 1 -- 172.21.15.112:0/2372405920 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe4005040 con 0x7f3ff00fc550 2024-09-16T00:23:15.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:15.050+0000 7f3feffff640 1 --2- 172.21.15.112:0/2372405920 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ff00fc550 0x7f3ff010d0e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3fe000b790 tx=0x7f3fe00091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:15.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:15.051+0000 7f3fecff9640 1 -- 172.21.15.112:0/2372405920 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3fe000c700 con 0x7f3ff00fc550 2024-09-16T00:23:15.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:15.051+0000 7f3fecff9640 1 -- 172.21.15.112:0/2372405920 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f3fe000cd00 con 0x7f3ff00fc550 2024-09-16T00:23:15.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:15.051+0000 7f3ff6317640 1 -- 172.21.15.112:0/2372405920 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ff010e8e0 con 0x7f3ff00fc550 2024-09-16T00:23:15.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:15.051+0000 7f3fecff9640 1 -- 172.21.15.112:0/2372405920 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3fe0015e70 con 0x7f3ff00fc550 2024-09-16T00:23:15.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:15.051+0000 7f3ff6317640 1 -- 172.21.15.112:0/2372405920 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ff010ee00 con 0x7f3ff00fc550 2024-09-16T00:23:15.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:15.052+0000 7f3ff6317640 1 -- 172.21.15.112:0/2372405920 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ff01057e0 con 0x7f3ff00fc550 2024-09-16T00:23:15.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:15.052+0000 7f3fecff9640 1 -- 172.21.15.112:0/2372405920 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) v1 ==== 90869+0+0 (secure 0 0 0) 0x7f3fe0012070 con 0x7f3ff00fc550 2024-09-16T00:23:15.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:15.053+0000 7f3fecff9640 1 --2- 172.21.15.112:0/2372405920 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f3fd806ed60 0x7f3fd8071210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:15.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:15.053+0000 7f3fef7fe640 1 --2- 172.21.15.112:0/2372405920 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f3fd806ed60 0x7f3fd8071210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:15.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:15.053+0000 7f3fecff9640 1 -- 172.21.15.112:0/2372405920 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f3fe008d930 con 0x7f3ff00fc550 2024-09-16T00:23:15.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:15.053+0000 7f3fef7fe640 1 --2- 172.21.15.112:0/2372405920 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f3fd806ed60 0x7f3fd8071210 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3fe4007ac0 tx=0x7f3fe4035040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:15.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:15.057+0000 7f3fecff9640 1 -- 172.21.15.112:0/2372405920 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f3fe005ae10 con 0x7f3ff00fc550 2024-09-16T00:23:15.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:15.223+0000 7f3ff6317640 1 -- 172.21.15.112:0/2372405920 --> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f3ff010f080 con 0x7f3fd806ed60 2024-09-16T00:23:15.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:15.224+0000 7f3fecff9640 1 -- 172.21.15.112:0/2372405920 <== mgr.14208 v2:172.21.15.112:6800/2480414866 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+148 (secure 0 0 0) 0x7f3ff010f080 con 0x7f3fd806ed60 2024-09-16T00:23:15.226 INFO:teuthology.orchestra.run.smithi112.stdout:HOST ADDR LABELS STATUS 2024-09-16T00:23:15.226 INFO:teuthology.orchestra.run.smithi112.stdout:smithi112 172.21.15.112 2024-09-16T00:23:15.226 INFO:teuthology.orchestra.run.smithi112.stdout:smithi169 172.21.15.169 2024-09-16T00:23:15.226 INFO:teuthology.orchestra.run.smithi112.stdout:2 hosts in cluster 2024-09-16T00:23:15.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:15.226+0000 7f3ff6317640 1 -- 172.21.15.112:0/2372405920 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f3fd806ed60 msgr2=0x7f3fd8071210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:15.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:15.226+0000 7f3ff6317640 1 --2- 172.21.15.112:0/2372405920 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f3fd806ed60 0x7f3fd8071210 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3fe4007ac0 tx=0x7f3fe4035040 comp rx=0 tx=0).stop 2024-09-16T00:23:15.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:15.226+0000 7f3ff6317640 1 -- 172.21.15.112:0/2372405920 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ff00fc550 msgr2=0x7f3ff010d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:15.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:15.226+0000 7f3ff6317640 1 --2- 172.21.15.112:0/2372405920 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ff00fc550 0x7f3ff010d0e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3fe000b790 tx=0x7f3fe00091d0 comp rx=0 tx=0).stop 2024-09-16T00:23:15.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:15.226+0000 7f3ff6317640 1 -- 172.21.15.112:0/2372405920 shutdown_connections 2024-09-16T00:23:15.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:15.226+0000 7f3ff6317640 1 --2- 172.21.15.112:0/2372405920 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f3fd806ed60 0x7f3fd8071210 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:15.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:15.226+0000 7f3ff6317640 1 --2- 172.21.15.112:0/2372405920 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ff00fd820 0x7f3ff010d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:15.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:15.226+0000 7f3ff6317640 1 --2- 172.21.15.112:0/2372405920 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ff00fc550 0x7f3ff010d0e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:15.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:15.226+0000 7f3ff6317640 1 -- 172.21.15.112:0/2372405920 >> 172.21.15.112:0/2372405920 conn(0x7f3ff00f7d30 msgr2=0x7f3ff00f94c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:15.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:15.227+0000 7f3ff6317640 1 -- 172.21.15.112:0/2372405920 shutdown_connections 2024-09-16T00:23:15.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:15.227+0000 7f3ff6317640 1 -- 172.21.15.112:0/2372405920 wait complete. 2024-09-16T00:23:15.869 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-16T00:23:16.111 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:23:16.514 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:16 smithi112 ceph-mon[27869]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:23:16.515 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:16 smithi112 ceph-mon[27869]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:23:16.578 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:16 smithi169 ceph-mon[34653]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:23:16.578 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:16 smithi169 ceph-mon[34653]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:23:17.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:17.783+0000 7f63170e1640 1 -- 172.21.15.112:0/1446250472 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63100ff720 msgr2=0x7f6310101b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:17.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:17.783+0000 7f63170e1640 1 --2- 172.21.15.112:0/1446250472 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63100ff720 0x7f6310101b00 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f63040048d0 tx=0x7f6304009da0 comp rx=0 tx=0).stop 2024-09-16T00:23:17.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:17.783+0000 7f63170e1640 1 -- 172.21.15.112:0/1446250472 shutdown_connections 2024-09-16T00:23:17.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:17.783+0000 7f63170e1640 1 --2- 172.21.15.112:0/1446250472 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63100ff720 0x7f6310101b00 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:17.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:17.783+0000 7f63170e1640 1 --2- 172.21.15.112:0/1446250472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63100666b0 0x7f63100ff1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:17.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:17.783+0000 7f63170e1640 1 -- 172.21.15.112:0/1446250472 >> 172.21.15.112:0/1446250472 conn(0x7f63100f6c10 msgr2=0x7f63100f9040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:17.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:17.783+0000 7f63170e1640 1 -- 172.21.15.112:0/1446250472 shutdown_connections 2024-09-16T00:23:17.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:17.783+0000 7f63170e1640 1 -- 172.21.15.112:0/1446250472 wait complete. 2024-09-16T00:23:17.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:17.783+0000 7f63170e1640 1 Processor -- start 2024-09-16T00:23:17.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:17.783+0000 7f63170e1640 1 -- start start 2024-09-16T00:23:17.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:17.784+0000 7f63170e1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63100666b0 0x7f631010d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:17.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:17.784+0000 7f63170e1640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63100ff720 0x7f631010d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:17.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:17.784+0000 7f63170e1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f631010dc50 con 0x7f63100666b0 2024-09-16T00:23:17.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:17.784+0000 7f63170e1640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f631010e470 con 0x7f63100ff720 2024-09-16T00:23:17.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:17.784+0000 7f6314e7d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63100666b0 0x7f631010d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:17.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:17.784+0000 7f6314e7d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63100666b0 0x7f631010d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34300/0 (socket says 172.21.15.112:34300) 2024-09-16T00:23:17.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:17.784+0000 7f6314e7d640 1 -- 172.21.15.112:0/3347111700 learned_addr learned my addr 172.21.15.112:0/3347111700 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:23:17.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:17.784+0000 7f630ffff640 1 --2- 172.21.15.112:0/3347111700 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63100ff720 0x7f631010d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:17.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:17.784+0000 7f6314e7d640 1 -- 172.21.15.112:0/3347111700 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63100ff720 msgr2=0x7f631010d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:17.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:17.784+0000 7f6314e7d640 1 --2- 172.21.15.112:0/3347111700 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63100ff720 0x7f631010d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:17.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:17.784+0000 7f6314e7d640 1 -- 172.21.15.112:0/3347111700 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6300005040 con 0x7f63100666b0 2024-09-16T00:23:17.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:17.784+0000 7f6314e7d640 1 --2- 172.21.15.112:0/3347111700 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63100666b0 0x7f631010d0c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f63000096d0 tx=0x7f6300009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:17.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:17.784+0000 7f630d7fa640 1 -- 172.21.15.112:0/3347111700 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f630000e8b0 con 0x7f63100666b0 2024-09-16T00:23:17.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:17.784+0000 7f63170e1640 1 -- 172.21.15.112:0/3347111700 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6304005040 con 0x7f63100666b0 2024-09-16T00:23:17.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:17.784+0000 7f630d7fa640 1 -- 172.21.15.112:0/3347111700 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f630000f040 con 0x7f63100666b0 2024-09-16T00:23:17.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:17.784+0000 7f630d7fa640 1 -- 172.21.15.112:0/3347111700 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6300013e00 con 0x7f63100666b0 2024-09-16T00:23:17.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:17.785+0000 7f63170e1640 1 -- 172.21.15.112:0/3347111700 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f631010eab0 con 0x7f63100666b0 2024-09-16T00:23:17.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:17.786+0000 7f63170e1640 1 -- 172.21.15.112:0/3347111700 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63100fa8b0 con 0x7f63100666b0 2024-09-16T00:23:17.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:17.786+0000 7f630d7fa640 1 -- 172.21.15.112:0/3347111700 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) v1 ==== 90869+0+0 (secure 0 0 0) 0x7f630000ea10 con 0x7f63100666b0 2024-09-16T00:23:17.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:17.786+0000 7f630d7fa640 1 --2- 172.21.15.112:0/3347111700 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f62f806ed60 0x7f62f8071210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:17.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:17.786+0000 7f630d7fa640 1 -- 172.21.15.112:0/3347111700 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f630008dc50 con 0x7f63100666b0 2024-09-16T00:23:17.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:17.786+0000 7f630ffff640 1 --2- 172.21.15.112:0/3347111700 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f62f806ed60 0x7f62f8071210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:17.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:17.787+0000 7f630ffff640 1 --2- 172.21.15.112:0/3347111700 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f62f806ed60 0x7f62f8071210 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f631010ce70 tx=0x7f6304035040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:17.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:17.791+0000 7f630d7fa640 1 -- 172.21.15.112:0/3347111700 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f630005b0b0 con 0x7f63100666b0 2024-09-16T00:23:17.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:17.951+0000 7f63170e1640 1 -- 172.21.15.112:0/3347111700 --> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f631010ee80 con 0x7f62f806ed60 2024-09-16T00:23:17.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:17.953+0000 7f630d7fa640 1 -- 172.21.15.112:0/3347111700 <== mgr.14208 v2:172.21.15.112:6800/2480414866 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+399 (secure 0 0 0) 0x7f631010ee80 con 0x7f62f806ed60 2024-09-16T00:23:17.953 INFO:teuthology.orchestra.run.smithi112.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-16T00:23:17.953 INFO:teuthology.orchestra.run.smithi112.stdout:smithi112 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400NV400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-16T00:23:17.953 INFO:teuthology.orchestra.run.smithi112.stdout:smithi169 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300LN400BGN 372G No 73s ago Has a FileSystem, LVM detected 2024-09-16T00:23:17.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:17.954+0000 7f63170e1640 1 -- 172.21.15.112:0/3347111700 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f62f806ed60 msgr2=0x7f62f8071210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:17.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:17.954+0000 7f63170e1640 1 --2- 172.21.15.112:0/3347111700 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f62f806ed60 0x7f62f8071210 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f631010ce70 tx=0x7f6304035040 comp rx=0 tx=0).stop 2024-09-16T00:23:17.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:17.954+0000 7f63170e1640 1 -- 172.21.15.112:0/3347111700 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63100666b0 msgr2=0x7f631010d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:17.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:17.954+0000 7f63170e1640 1 --2- 172.21.15.112:0/3347111700 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63100666b0 0x7f631010d0c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f63000096d0 tx=0x7f6300009da0 comp rx=0 tx=0).stop 2024-09-16T00:23:17.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:17.955+0000 7f63170e1640 1 -- 172.21.15.112:0/3347111700 shutdown_connections 2024-09-16T00:23:17.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:17.955+0000 7f63170e1640 1 --2- 172.21.15.112:0/3347111700 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f62f806ed60 0x7f62f8071210 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:17.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:17.955+0000 7f63170e1640 1 --2- 172.21.15.112:0/3347111700 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63100ff720 0x7f631010d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:17.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:17.955+0000 7f63170e1640 1 --2- 172.21.15.112:0/3347111700 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63100666b0 0x7f631010d0c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:17.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:17.955+0000 7f63170e1640 1 -- 172.21.15.112:0/3347111700 >> 172.21.15.112:0/3347111700 conn(0x7f63100f6c10 msgr2=0x7f63100f7030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:17.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:17.955+0000 7f63170e1640 1 -- 172.21.15.112:0/3347111700 shutdown_connections 2024-09-16T00:23:17.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:17.955+0000 7f63170e1640 1 -- 172.21.15.112:0/3347111700 wait complete. 2024-09-16T00:23:18.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:18 smithi112 ceph-mon[27869]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:23:18.719 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-16T00:23:18.729 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi112.front.sepia.ceph.com 2024-09-16T00:23:18.729 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- bash -c 'ceph fs volume create cephfs --placement=4' 2024-09-16T00:23:18.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:18 smithi169 ceph-mon[34653]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:23:18.972 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:23:19.411 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:19 smithi112 ceph-mon[27869]: from='client.14452 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:23:19.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:19 smithi169 ceph-mon[34653]: from='client.14452 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:23:20.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:20.547+0000 7f4e61580640 1 -- 172.21.15.112:0/3211083470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e5c0fb430 msgr2=0x7f4e5c0fb840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:20.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:20.547+0000 7f4e61580640 1 --2- 172.21.15.112:0/3211083470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e5c0fb430 0x7f4e5c0fb840 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c009c50 tx=0x7f4e4c004d10 comp rx=0 tx=0).stop 2024-09-16T00:23:20.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:20.547+0000 7f4e61580640 1 -- 172.21.15.112:0/3211083470 shutdown_connections 2024-09-16T00:23:20.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:20.547+0000 7f4e61580640 1 --2- 172.21.15.112:0/3211083470 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e5c0fc700 0x7f4e5c0fcb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:20.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:20.547+0000 7f4e61580640 1 --2- 172.21.15.112:0/3211083470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e5c0fb430 0x7f4e5c0fb840 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:20.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:20.547+0000 7f4e61580640 1 -- 172.21.15.112:0/3211083470 >> 172.21.15.112:0/3211083470 conn(0x7f4e5c0f6c10 msgr2=0x7f4e5c0f9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:20.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:20.547+0000 7f4e61580640 1 -- 172.21.15.112:0/3211083470 shutdown_connections 2024-09-16T00:23:20.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:20.547+0000 7f4e61580640 1 -- 172.21.15.112:0/3211083470 wait complete. 2024-09-16T00:23:20.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:20.547+0000 7f4e61580640 1 Processor -- start 2024-09-16T00:23:20.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:20.547+0000 7f4e61580640 1 -- start start 2024-09-16T00:23:20.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:20.548+0000 7f4e61580640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e5c0fb430 0x7f4e5c10bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:20.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:20.548+0000 7f4e61580640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e5c0fc700 0x7f4e5c10c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:20.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:20.548+0000 7f4e61580640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5c10d390 con 0x7f4e5c0fc700 2024-09-16T00:23:20.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:20.548+0000 7f4e61580640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5c10d500 con 0x7f4e5c0fb430 2024-09-16T00:23:20.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:20.548+0000 7f4e5a7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e5c0fc700 0x7f4e5c10c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:20.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:20.548+0000 7f4e5a7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e5c0fc700 0x7f4e5c10c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34326/0 (socket says 172.21.15.112:34326) 2024-09-16T00:23:20.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:20.548+0000 7f4e5a7fc640 1 -- 172.21.15.112:0/3050014338 learned_addr learned my addr 172.21.15.112:0/3050014338 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:23:20.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:20.548+0000 7f4e5affd640 1 --2- 172.21.15.112:0/3050014338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e5c0fb430 0x7f4e5c10bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:20.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:20.548+0000 7f4e5a7fc640 1 -- 172.21.15.112:0/3050014338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e5c0fb430 msgr2=0x7f4e5c10bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:20.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:20.548+0000 7f4e5a7fc640 1 --2- 172.21.15.112:0/3050014338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e5c0fb430 0x7f4e5c10bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:20.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:20.548+0000 7f4e5a7fc640 1 -- 172.21.15.112:0/3050014338 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e4c0098b0 con 0x7f4e5c0fc700 2024-09-16T00:23:20.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:20.548+0000 7f4e5affd640 1 --2- 172.21.15.112:0/3050014338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e5c0fb430 0x7f4e5c10bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:23:20.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:20.548+0000 7f4e5a7fc640 1 --2- 172.21.15.112:0/3050014338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e5c0fc700 0x7f4e5c10c520 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4e4400daf0 tx=0x7f4e4400b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:20.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:20.548+0000 7f4e537fe640 1 -- 172.21.15.112:0/3050014338 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e4400edb0 con 0x7f4e5c0fc700 2024-09-16T00:23:20.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:20.548+0000 7f4e537fe640 1 -- 172.21.15.112:0/3050014338 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f4e44017910 con 0x7f4e5c0fc700 2024-09-16T00:23:20.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:20.548+0000 7f4e61580640 1 -- 172.21.15.112:0/3050014338 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e5c10d780 con 0x7f4e5c0fc700 2024-09-16T00:23:20.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:20.548+0000 7f4e537fe640 1 -- 172.21.15.112:0/3050014338 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e440169f0 con 0x7f4e5c0fc700 2024-09-16T00:23:20.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:20.548+0000 7f4e61580640 1 -- 172.21.15.112:0/3050014338 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e5c10dca0 con 0x7f4e5c0fc700 2024-09-16T00:23:20.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:20.549+0000 7f4e61580640 1 -- 172.21.15.112:0/3050014338 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e5c0fa890 con 0x7f4e5c0fc700 2024-09-16T00:23:20.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:20.550+0000 7f4e537fe640 1 -- 172.21.15.112:0/3050014338 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) v1 ==== 90869+0+0 (secure 0 0 0) 0x7f4e44014070 con 0x7f4e5c0fc700 2024-09-16T00:23:20.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:20.550+0000 7f4e537fe640 1 --2- 172.21.15.112:0/3050014338 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f4e4806ed60 0x7f4e48071210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:20.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:20.550+0000 7f4e5affd640 1 --2- 172.21.15.112:0/3050014338 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f4e4806ed60 0x7f4e48071210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:20.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:20.550+0000 7f4e537fe640 1 -- 172.21.15.112:0/3050014338 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f4e440901d0 con 0x7f4e5c0fc700 2024-09-16T00:23:20.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:20.551+0000 7f4e5affd640 1 --2- 172.21.15.112:0/3050014338 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f4e4806ed60 0x7f4e48071210 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c000f80 tx=0x7f4e4c004d10 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:20.554 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:20 smithi112 ceph-mon[27869]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:23:20.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:20.554+0000 7f4e537fe640 1 -- 172.21.15.112:0/3050014338 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f4e4405d6b0 con 0x7f4e5c0fc700 2024-09-16T00:23:20.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:20.730+0000 7f4e61580640 1 -- 172.21.15.112:0/3050014338 --> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f4e5c10df20 con 0x7f4e4806ed60 2024-09-16T00:23:20.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:20 smithi169 ceph-mon[34653]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:23:21.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:21 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-09-16T00:23:21.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:21 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-09-16T00:23:22.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:22.343+0000 7f4e537fe640 1 -- 172.21.15.112:0/3050014338 <== mgr.14208 v2:172.21.15.112:6800/2480414866 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f4e5c10df20 con 0x7f4e4806ed60 2024-09-16T00:23:22.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:22.347+0000 7f4e61580640 1 -- 172.21.15.112:0/3050014338 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f4e4806ed60 msgr2=0x7f4e48071210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:22.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:22.347+0000 7f4e61580640 1 --2- 172.21.15.112:0/3050014338 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f4e4806ed60 0x7f4e48071210 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c000f80 tx=0x7f4e4c004d10 comp rx=0 tx=0).stop 2024-09-16T00:23:22.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:22.347+0000 7f4e61580640 1 -- 172.21.15.112:0/3050014338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e5c0fc700 msgr2=0x7f4e5c10c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:22.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:22.347+0000 7f4e61580640 1 --2- 172.21.15.112:0/3050014338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e5c0fc700 0x7f4e5c10c520 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4e4400daf0 tx=0x7f4e4400b460 comp rx=0 tx=0).stop 2024-09-16T00:23:22.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:22.347+0000 7f4e61580640 1 -- 172.21.15.112:0/3050014338 shutdown_connections 2024-09-16T00:23:22.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:22.347+0000 7f4e61580640 1 --2- 172.21.15.112:0/3050014338 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f4e4806ed60 0x7f4e48071210 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:22.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:22.347+0000 7f4e61580640 1 --2- 172.21.15.112:0/3050014338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e5c0fc700 0x7f4e5c10c520 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:22.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:22.347+0000 7f4e61580640 1 --2- 172.21.15.112:0/3050014338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e5c0fb430 0x7f4e5c10bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:22.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:22.347+0000 7f4e61580640 1 -- 172.21.15.112:0/3050014338 >> 172.21.15.112:0/3050014338 conn(0x7f4e5c0f6c10 msgr2=0x7f4e5c0f83c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:22.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:22.347+0000 7f4e61580640 1 -- 172.21.15.112:0/3050014338 shutdown_connections 2024-09-16T00:23:22.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:22.347+0000 7f4e61580640 1 -- 172.21.15.112:0/3050014338 wait complete. 2024-09-16T00:23:22.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:22 smithi112 ceph-mon[27869]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:23:22.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:22 smithi112 ceph-mon[27869]: from='client.14456 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:23:22.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:22 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-09-16T00:23:22.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:22 smithi112 ceph-mon[27869]: osdmap e39: 6 total, 6 up, 6 in 2024-09-16T00:23:22.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:22 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-09-16T00:23:22.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:22 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:23:22.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:22 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-mon-smithi112[27865]: 2024-09-16T00:23:22.327+0000 7f0cd0ba5640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T00:23:22.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:22 smithi169 ceph-mon[34653]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:23:22.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:22 smithi169 ceph-mon[34653]: from='client.14456 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:23:22.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:22 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-09-16T00:23:22.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:22 smithi169 ceph-mon[34653]: osdmap e39: 6 total, 6 up, 6 in 2024-09-16T00:23:22.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:22 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-09-16T00:23:22.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:22 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:23:23.054 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-16T00:23:23.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:23 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-09-16T00:23:23.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:23 smithi112 ceph-mon[27869]: osdmap e40: 6 total, 6 up, 6 in 2024-09-16T00:23:23.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:23 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-09-16T00:23:23.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:23 smithi112 ceph-mon[27869]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T00:23:23.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:23 smithi112 ceph-mon[27869]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-16T00:23:23.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:23 smithi112 ceph-mon[27869]: osdmap e41: 6 total, 6 up, 6 in 2024-09-16T00:23:23.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:23 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-09-16T00:23:23.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:23 smithi112 ceph-mon[27869]: fsmap cephfs:0 2024-09-16T00:23:23.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:23 smithi112 ceph-mon[27869]: Saving service mds.cephfs spec with placement count:4 2024-09-16T00:23:23.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:23 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:23:23.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:23 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:23:23.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:23 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:23:23.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:23 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:23:23.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:23 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:23:23.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:23 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi112.ithwzj", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:23:23.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:23 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi112.ithwzj", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:23:23.591 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:23 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:23:23.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:23 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-09-16T00:23:23.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:23 smithi169 ceph-mon[34653]: osdmap e40: 6 total, 6 up, 6 in 2024-09-16T00:23:23.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:23 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-09-16T00:23:23.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:23 smithi169 ceph-mon[34653]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T00:23:23.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:23 smithi169 ceph-mon[34653]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-16T00:23:23.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:23 smithi169 ceph-mon[34653]: osdmap e41: 6 total, 6 up, 6 in 2024-09-16T00:23:23.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:23 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-09-16T00:23:23.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:23 smithi169 ceph-mon[34653]: fsmap cephfs:0 2024-09-16T00:23:23.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:23 smithi169 ceph-mon[34653]: Saving service mds.cephfs spec with placement count:4 2024-09-16T00:23:23.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:23 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:23:23.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:23 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:23:23.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:23 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:23:23.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:23 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:23:23.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:23 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:23:23.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:23 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi112.ithwzj", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:23:23.778 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:23 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi112.ithwzj", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:23:23.778 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:23 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:23:24.300 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:23:24.473 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:24 smithi112 ceph-mon[27869]: pgmap v172: 65 pgs: 19 creating+peering, 1 active+clean, 45 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:23:24.473 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:24 smithi112 ceph-mon[27869]: Deploying daemon mds.cephfs.smithi112.ithwzj on smithi112 2024-09-16T00:23:24.473 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:24 smithi112 ceph-mon[27869]: osdmap e42: 6 total, 6 up, 6 in 2024-09-16T00:23:24.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:24 smithi169 ceph-mon[34653]: pgmap v172: 65 pgs: 19 creating+peering, 1 active+clean, 45 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:23:24.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:24 smithi169 ceph-mon[34653]: Deploying daemon mds.cephfs.smithi112.ithwzj on smithi112 2024-09-16T00:23:24.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:24 smithi169 ceph-mon[34653]: osdmap e42: 6 total, 6 up, 6 in 2024-09-16T00:23:25.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:25 smithi112 ceph-mon[27869]: osdmap e43: 6 total, 6 up, 6 in 2024-09-16T00:23:25.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:25 smithi169 ceph-mon[34653]: osdmap e43: 6 total, 6 up, 6 in 2024-09-16T00:23:26.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:26.161+0000 7f3061f44640 1 -- 172.21.15.112:0/2384132181 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f305c0ff700 msgr2=0x7f305c101ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:26.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:26.161+0000 7f3061f44640 1 --2- 172.21.15.112:0/2384132181 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f305c0ff700 0x7f305c101ae0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f30440048d0 tx=0x7f3044009da0 comp rx=0 tx=0).stop 2024-09-16T00:23:26.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:26.161+0000 7f3061f44640 1 -- 172.21.15.112:0/2384132181 shutdown_connections 2024-09-16T00:23:26.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:26.161+0000 7f3061f44640 1 --2- 172.21.15.112:0/2384132181 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f305c0ff700 0x7f305c101ae0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:26.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:26.161+0000 7f3061f44640 1 --2- 172.21.15.112:0/2384132181 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f305c065bc0 0x7f305c0ff1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:26.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:26.161+0000 7f3061f44640 1 -- 172.21.15.112:0/2384132181 >> 172.21.15.112:0/2384132181 conn(0x7f305c0f6c10 msgr2=0x7f305c0f9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:26.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:26.161+0000 7f3061f44640 1 -- 172.21.15.112:0/2384132181 shutdown_connections 2024-09-16T00:23:26.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:26.161+0000 7f3061f44640 1 -- 172.21.15.112:0/2384132181 wait complete. 2024-09-16T00:23:26.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:26.161+0000 7f3061f44640 1 Processor -- start 2024-09-16T00:23:26.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:26.162+0000 7f3061f44640 1 -- start start 2024-09-16T00:23:26.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:26.162+0000 7f3061f44640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f305c065bc0 0x7f305c10d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:26.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:26.162+0000 7f3061f44640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f305c0ff700 0x7f305c10d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:26.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:26.162+0000 7f3061f44640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f305c10e440 con 0x7f305c065bc0 2024-09-16T00:23:26.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:26.162+0000 7f3061f44640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f305c10e5b0 con 0x7f305c0ff700 2024-09-16T00:23:26.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:26.162+0000 7f305b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f305c065bc0 0x7f305c10d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:26.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:26.162+0000 7f305b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f305c065bc0 0x7f305c10d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56468/0 (socket says 172.21.15.112:56468) 2024-09-16T00:23:26.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:26.162+0000 7f305b7fe640 1 -- 172.21.15.112:0/2580368614 learned_addr learned my addr 172.21.15.112:0/2580368614 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:23:26.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:26.162+0000 7f3052ffd640 1 --2- 172.21.15.112:0/2580368614 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f305c0ff700 0x7f305c10d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:26.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:26.162+0000 7f305b7fe640 1 -- 172.21.15.112:0/2580368614 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f305c0ff700 msgr2=0x7f305c10d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:26.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:26.162+0000 7f305b7fe640 1 --2- 172.21.15.112:0/2580368614 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f305c0ff700 0x7f305c10d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:26.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:26.162+0000 7f305b7fe640 1 -- 172.21.15.112:0/2580368614 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3044005040 con 0x7f305c065bc0 2024-09-16T00:23:26.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:26.162+0000 7f305b7fe640 1 --2- 172.21.15.112:0/2580368614 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f305c065bc0 0x7f305c10d1a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f304c00b580 tx=0x7f304c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:26.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:26.162+0000 7f3058ff9640 1 -- 172.21.15.112:0/2580368614 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f304c00f570 con 0x7f305c065bc0 2024-09-16T00:23:26.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:26.162+0000 7f3061f44640 1 -- 172.21.15.112:0/2580368614 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f305c10e890 con 0x7f305c065bc0 2024-09-16T00:23:26.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:26.163+0000 7f3058ff9640 1 -- 172.21.15.112:0/2580368614 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f304c00fb70 con 0x7f305c065bc0 2024-09-16T00:23:26.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:26.163+0000 7f3058ff9640 1 -- 172.21.15.112:0/2580368614 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f304c010d40 con 0x7f305c065bc0 2024-09-16T00:23:26.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:26.163+0000 7f3061f44640 1 -- 172.21.15.112:0/2580368614 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f305c10edb0 con 0x7f305c065bc0 2024-09-16T00:23:26.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:26.163+0000 7f3061f44640 1 -- 172.21.15.112:0/2580368614 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f305c1057f0 con 0x7f305c065bc0 2024-09-16T00:23:26.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:26.164+0000 7f3058ff9640 1 -- 172.21.15.112:0/2580368614 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) v1 ==== 90869+0+0 (secure 0 0 0) 0x7f304c00fce0 con 0x7f305c065bc0 2024-09-16T00:23:26.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:26.165+0000 7f3058ff9640 1 --2- 172.21.15.112:0/2580368614 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f304806ed60 0x7f3048071210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:26.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:26.165+0000 7f3052ffd640 1 --2- 172.21.15.112:0/2580368614 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f304806ed60 0x7f3048071210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:26.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:26.165+0000 7f3058ff9640 1 -- 172.21.15.112:0/2580368614 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f304c08e060 con 0x7f305c065bc0 2024-09-16T00:23:26.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:26.165+0000 7f3052ffd640 1 --2- 172.21.15.112:0/2580368614 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f304806ed60 0x7f3048071210 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3044000c00 tx=0x7f3044009da0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:26.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:26.169+0000 7f3058ff9640 1 -- 172.21.15.112:0/2580368614 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f304c05b200 con 0x7f305c065bc0 2024-09-16T00:23:26.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:26.360+0000 7f3061f44640 1 -- 172.21.15.112:0/2580368614 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f305c04b720 con 0x7f305c065bc0 2024-09-16T00:23:26.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:26.361+0000 7f3058ff9640 1 -- 172.21.15.112:0/2580368614 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 2 v2) v1 ==== 75+0+1076 (secure 0 0 0) 0x7f304c05aba0 con 0x7f305c065bc0 2024-09-16T00:23:26.361 INFO:teuthology.orchestra.run.smithi112.stdout:e2 2024-09-16T00:23:26.361 INFO:teuthology.orchestra.run.smithi112.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T00:23:26.361 INFO:teuthology.orchestra.run.smithi112.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:23:26.362 INFO:teuthology.orchestra.run.smithi112.stdout:legacy client fscid: 1 2024-09-16T00:23:26.362 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:23:26.362 INFO:teuthology.orchestra.run.smithi112.stdout:Filesystem 'cephfs' (1) 2024-09-16T00:23:26.362 INFO:teuthology.orchestra.run.smithi112.stdout:fs_name cephfs 2024-09-16T00:23:26.362 INFO:teuthology.orchestra.run.smithi112.stdout:epoch 2 2024-09-16T00:23:26.362 INFO:teuthology.orchestra.run.smithi112.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-16T00:23:26.362 INFO:teuthology.orchestra.run.smithi112.stdout:created 2024-09-16T00:23:22.328199+0000 2024-09-16T00:23:26.362 INFO:teuthology.orchestra.run.smithi112.stdout:modified 2024-09-16T00:23:22.328259+0000 2024-09-16T00:23:26.362 INFO:teuthology.orchestra.run.smithi112.stdout:tableserver 0 2024-09-16T00:23:26.362 INFO:teuthology.orchestra.run.smithi112.stdout:root 0 2024-09-16T00:23:26.362 INFO:teuthology.orchestra.run.smithi112.stdout:session_timeout 60 2024-09-16T00:23:26.362 INFO:teuthology.orchestra.run.smithi112.stdout:session_autoclose 300 2024-09-16T00:23:26.362 INFO:teuthology.orchestra.run.smithi112.stdout:max_file_size 1099511627776 2024-09-16T00:23:26.362 INFO:teuthology.orchestra.run.smithi112.stdout:required_client_features {} 2024-09-16T00:23:26.363 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure 0 2024-09-16T00:23:26.363 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure_osd_epoch 0 2024-09-16T00:23:26.363 INFO:teuthology.orchestra.run.smithi112.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:23:26.363 INFO:teuthology.orchestra.run.smithi112.stdout:max_mds 1 2024-09-16T00:23:26.363 INFO:teuthology.orchestra.run.smithi112.stdout:in 2024-09-16T00:23:26.363 INFO:teuthology.orchestra.run.smithi112.stdout:up {} 2024-09-16T00:23:26.363 INFO:teuthology.orchestra.run.smithi112.stdout:failed 2024-09-16T00:23:26.363 INFO:teuthology.orchestra.run.smithi112.stdout:damaged 2024-09-16T00:23:26.363 INFO:teuthology.orchestra.run.smithi112.stdout:stopped 2024-09-16T00:23:26.363 INFO:teuthology.orchestra.run.smithi112.stdout:data_pools [3] 2024-09-16T00:23:26.363 INFO:teuthology.orchestra.run.smithi112.stdout:metadata_pool 2 2024-09-16T00:23:26.363 INFO:teuthology.orchestra.run.smithi112.stdout:inline_data disabled 2024-09-16T00:23:26.363 INFO:teuthology.orchestra.run.smithi112.stdout:balancer 2024-09-16T00:23:26.364 INFO:teuthology.orchestra.run.smithi112.stdout:standby_count_wanted 0 2024-09-16T00:23:26.364 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:23:26.364 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:23:26.364 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 2 2024-09-16T00:23:26.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:26.362+0000 7f3061f44640 1 -- 172.21.15.112:0/2580368614 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f304806ed60 msgr2=0x7f3048071210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:26.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:26.362+0000 7f3061f44640 1 --2- 172.21.15.112:0/2580368614 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f304806ed60 0x7f3048071210 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3044000c00 tx=0x7f3044009da0 comp rx=0 tx=0).stop 2024-09-16T00:23:26.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:26.362+0000 7f3061f44640 1 -- 172.21.15.112:0/2580368614 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f305c065bc0 msgr2=0x7f305c10d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:26.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:26.362+0000 7f3061f44640 1 --2- 172.21.15.112:0/2580368614 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f305c065bc0 0x7f305c10d1a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f304c00b580 tx=0x7f304c0091a0 comp rx=0 tx=0).stop 2024-09-16T00:23:26.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:26.362+0000 7f3061f44640 1 -- 172.21.15.112:0/2580368614 shutdown_connections 2024-09-16T00:23:26.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:26.362+0000 7f3061f44640 1 --2- 172.21.15.112:0/2580368614 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f304806ed60 0x7f3048071210 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:26.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:26.362+0000 7f3061f44640 1 --2- 172.21.15.112:0/2580368614 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f305c0ff700 0x7f305c10d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:26.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:26.362+0000 7f3061f44640 1 --2- 172.21.15.112:0/2580368614 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f305c065bc0 0x7f305c10d1a0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:26.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:26.363+0000 7f3061f44640 1 -- 172.21.15.112:0/2580368614 >> 172.21.15.112:0/2580368614 conn(0x7f305c0f6c10 msgr2=0x7f305c0f6fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:26.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:26.363+0000 7f3061f44640 1 -- 172.21.15.112:0/2580368614 shutdown_connections 2024-09-16T00:23:26.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:26.363+0000 7f3061f44640 1 -- 172.21.15.112:0/2580368614 wait complete. 2024-09-16T00:23:26.468 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:26 smithi112 ceph-mon[27869]: pgmap v175: 65 pgs: 27 creating+peering, 15 active+clean, 23 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:23:26.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:26 smithi169 ceph-mon[34653]: pgmap v175: 65 pgs: 27 creating+peering, 15 active+clean, 23 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:23:27.079 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-16T00:23:27.088 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi112.front.sepia.ceph.com 2024-09-16T00:23:27.088 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- bash -c 'ceph fs set cephfs max_mds 2' 2024-09-16T00:23:27.336 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:23:27.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:27 smithi169 ceph-mon[34653]: from='client.? 172.21.15.112:0/2580368614' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:23:27.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:27 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:23:27.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:27 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:23:27.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:27 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:23:27.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:27 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi169.gawwgw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:23:27.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:27 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi169.gawwgw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:23:27.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:27 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:23:27.624 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:27 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/2580368614' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:23:27.625 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:27 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:23:27.625 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:27 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:23:27.625 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:27 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:23:27.625 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:27 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi169.gawwgw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:23:27.625 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:27 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi169.gawwgw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:23:27.625 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:27 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:23:28.463 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:28 smithi112 ceph-mon[27869]: Deploying daemon mds.cephfs.smithi169.gawwgw on smithi169 2024-09-16T00:23:28.463 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:28 smithi112 ceph-mon[27869]: pgmap v176: 65 pgs: 27 creating+peering, 24 active+clean, 14 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:23:28.463 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:28 smithi112 ceph-mon[27869]: mds.? [v2:172.21.15.112:6826/777980670,v1:172.21.15.112:6827/777980670] up:boot 2024-09-16T00:23:28.463 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:28 smithi112 ceph-mon[27869]: daemon mds.cephfs.smithi112.ithwzj assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-09-16T00:23:28.463 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:28 smithi112 ceph-mon[27869]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-16T00:23:28.463 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:28 smithi112 ceph-mon[27869]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-16T00:23:28.463 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:28 smithi112 ceph-mon[27869]: Cluster is now healthy 2024-09-16T00:23:28.463 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:28 smithi112 ceph-mon[27869]: fsmap cephfs:0 1 up:standby 2024-09-16T00:23:28.463 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:28 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi112.ithwzj"}]: dispatch 2024-09-16T00:23:28.463 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:28 smithi112 ceph-mon[27869]: fsmap cephfs:1 {0=cephfs.smithi112.ithwzj=up:creating} 2024-09-16T00:23:28.464 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:28 smithi112 ceph-mon[27869]: daemon mds.cephfs.smithi112.ithwzj is now active in filesystem cephfs as rank 0 2024-09-16T00:23:28.595 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:28 smithi169 ceph-mon[34653]: Deploying daemon mds.cephfs.smithi169.gawwgw on smithi169 2024-09-16T00:23:28.595 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:28 smithi169 ceph-mon[34653]: pgmap v176: 65 pgs: 27 creating+peering, 24 active+clean, 14 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:23:28.595 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:28 smithi169 ceph-mon[34653]: mds.? [v2:172.21.15.112:6826/777980670,v1:172.21.15.112:6827/777980670] up:boot 2024-09-16T00:23:28.595 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:28 smithi169 ceph-mon[34653]: daemon mds.cephfs.smithi112.ithwzj assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-09-16T00:23:28.595 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:28 smithi169 ceph-mon[34653]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-16T00:23:28.595 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:28 smithi169 ceph-mon[34653]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-16T00:23:28.595 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:28 smithi169 ceph-mon[34653]: Cluster is now healthy 2024-09-16T00:23:28.595 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:28 smithi169 ceph-mon[34653]: fsmap cephfs:0 1 up:standby 2024-09-16T00:23:28.595 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:28 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi112.ithwzj"}]: dispatch 2024-09-16T00:23:28.595 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:28 smithi169 ceph-mon[34653]: fsmap cephfs:1 {0=cephfs.smithi112.ithwzj=up:creating} 2024-09-16T00:23:28.595 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:28 smithi169 ceph-mon[34653]: daemon mds.cephfs.smithi112.ithwzj is now active in filesystem cephfs as rank 0 2024-09-16T00:23:28.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:28.929+0000 7f4960c0e640 1 -- 172.21.15.112:0/1527252020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f495c0ff6a0 msgr2=0x7f495c101a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:28.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:28.929+0000 7f4960c0e640 1 --2- 172.21.15.112:0/1527252020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f495c0ff6a0 0x7f495c101a80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f49500048d0 tx=0x7f4950009da0 comp rx=0 tx=0).stop 2024-09-16T00:23:28.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:28.930+0000 7f4960c0e640 1 -- 172.21.15.112:0/1527252020 shutdown_connections 2024-09-16T00:23:28.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:28.930+0000 7f4960c0e640 1 --2- 172.21.15.112:0/1527252020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f495c0ff6a0 0x7f495c101a80 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:28.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:28.930+0000 7f4960c0e640 1 --2- 172.21.15.112:0/1527252020 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f495c066790 0x7f495c0ff160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:28.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:28.930+0000 7f4960c0e640 1 -- 172.21.15.112:0/1527252020 >> 172.21.15.112:0/1527252020 conn(0x7f495c0f6c10 msgr2=0x7f495c0f9040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:28.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:28.930+0000 7f4960c0e640 1 -- 172.21.15.112:0/1527252020 shutdown_connections 2024-09-16T00:23:28.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:28.930+0000 7f4960c0e640 1 -- 172.21.15.112:0/1527252020 wait complete. 2024-09-16T00:23:28.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:28.930+0000 7f4960c0e640 1 Processor -- start 2024-09-16T00:23:28.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:28.930+0000 7f4960c0e640 1 -- start start 2024-09-16T00:23:28.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:28.930+0000 7f4960c0e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f495c066790 0x7f495c10d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:28.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:28.930+0000 7f4960c0e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f495c0ff6a0 0x7f495c10d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:28.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:28.931+0000 7f4960c0e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f495c10dc40 con 0x7f495c066790 2024-09-16T00:23:28.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:28.931+0000 7f4960c0e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f495c10e460 con 0x7f495c0ff6a0 2024-09-16T00:23:28.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:28.931+0000 7f495a59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f495c066790 0x7f495c10d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:28.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:28.931+0000 7f495a59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f495c066790 0x7f495c10d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56506/0 (socket says 172.21.15.112:56506) 2024-09-16T00:23:28.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:28.931+0000 7f495a59c640 1 -- 172.21.15.112:0/3913243802 learned_addr learned my addr 172.21.15.112:0/3913243802 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:23:28.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:28.931+0000 7f4959d9b640 1 --2- 172.21.15.112:0/3913243802 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f495c0ff6a0 0x7f495c10d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:28.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:28.931+0000 7f495a59c640 1 -- 172.21.15.112:0/3913243802 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f495c0ff6a0 msgr2=0x7f495c10d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:28.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:28.931+0000 7f495a59c640 1 --2- 172.21.15.112:0/3913243802 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f495c0ff6a0 0x7f495c10d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:28.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:28.931+0000 7f495a59c640 1 -- 172.21.15.112:0/3913243802 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4950005040 con 0x7f495c066790 2024-09-16T00:23:28.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:28.931+0000 7f4959d9b640 1 --2- 172.21.15.112:0/3913243802 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f495c0ff6a0 0x7f495c10d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:23:28.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:28.931+0000 7f495a59c640 1 --2- 172.21.15.112:0/3913243802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f495c066790 0x7f495c10d0b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f494c002840 tx=0x7f494c004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:28.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:28.931+0000 7f494affd640 1 -- 172.21.15.112:0/3913243802 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f494c007c30 con 0x7f495c066790 2024-09-16T00:23:28.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:28.931+0000 7f4960c0e640 1 -- 172.21.15.112:0/3913243802 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f495c10e6f0 con 0x7f495c066790 2024-09-16T00:23:28.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:28.931+0000 7f494affd640 1 -- 172.21.15.112:0/3913243802 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f494c00f040 con 0x7f495c066790 2024-09-16T00:23:28.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:28.931+0000 7f494affd640 1 -- 172.21.15.112:0/3913243802 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f494c009510 con 0x7f495c066790 2024-09-16T00:23:28.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:28.931+0000 7f4960c0e640 1 -- 172.21.15.112:0/3913243802 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f495c10ec90 con 0x7f495c066790 2024-09-16T00:23:28.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:28.932+0000 7f4960c0e640 1 -- 172.21.15.112:0/3913243802 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f495c0fa8b0 con 0x7f495c066790 2024-09-16T00:23:28.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:28.933+0000 7f494affd640 1 -- 172.21.15.112:0/3913243802 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) v1 ==== 90869+0+0 (secure 0 0 0) 0x7f494c009780 con 0x7f495c066790 2024-09-16T00:23:28.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:28.933+0000 7f494affd640 1 --2- 172.21.15.112:0/3913243802 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f494406ed60 0x7f4944071210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:28.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:28.934+0000 7f4959d9b640 1 --2- 172.21.15.112:0/3913243802 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f494406ed60 0x7f4944071210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:28.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:28.934+0000 7f494affd640 1 -- 172.21.15.112:0/3913243802 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f494c08e110 con 0x7f495c066790 2024-09-16T00:23:28.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:28.934+0000 7f4959d9b640 1 --2- 172.21.15.112:0/3913243802 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f494406ed60 0x7f4944071210 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f495000b020 tx=0x7f4950009da0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:28.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:28.938+0000 7f494affd640 1 -- 172.21.15.112:0/3913243802 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f494c05b2f0 con 0x7f495c066790 2024-09-16T00:23:29.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:29.140+0000 7f4960c0e640 1 -- 172.21.15.112:0/3913243802 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"} v 0) v1 -- 0x7f495c10efb0 con 0x7f495c066790 2024-09-16T00:23:29.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:29.369+0000 7f494affd640 1 -- 172.21.15.112:0/3913243802 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]=0 v6) v1 ==== 105+0+0 (secure 0 0 0) 0x7f494c05ac90 con 0x7f495c066790 2024-09-16T00:23:29.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:29.371+0000 7f4960c0e640 1 -- 172.21.15.112:0/3913243802 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f494406ed60 msgr2=0x7f4944071210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:29.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:29.371+0000 7f4960c0e640 1 --2- 172.21.15.112:0/3913243802 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f494406ed60 0x7f4944071210 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f495000b020 tx=0x7f4950009da0 comp rx=0 tx=0).stop 2024-09-16T00:23:29.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:29.371+0000 7f4960c0e640 1 -- 172.21.15.112:0/3913243802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f495c066790 msgr2=0x7f495c10d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:29.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:29.371+0000 7f4960c0e640 1 --2- 172.21.15.112:0/3913243802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f495c066790 0x7f495c10d0b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f494c002840 tx=0x7f494c004570 comp rx=0 tx=0).stop 2024-09-16T00:23:29.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:29.371+0000 7f4960c0e640 1 -- 172.21.15.112:0/3913243802 shutdown_connections 2024-09-16T00:23:29.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:29.371+0000 7f4960c0e640 1 --2- 172.21.15.112:0/3913243802 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f494406ed60 0x7f4944071210 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:29.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:29.371+0000 7f4960c0e640 1 --2- 172.21.15.112:0/3913243802 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f495c0ff6a0 0x7f495c10d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:29.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:29.371+0000 7f4960c0e640 1 --2- 172.21.15.112:0/3913243802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f495c066790 0x7f495c10d0b0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:29.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:29.371+0000 7f4960c0e640 1 -- 172.21.15.112:0/3913243802 >> 172.21.15.112:0/3913243802 conn(0x7f495c0f6c10 msgr2=0x7f495c0f7030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:29.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:29.371+0000 7f4960c0e640 1 -- 172.21.15.112:0/3913243802 shutdown_connections 2024-09-16T00:23:29.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:29.371+0000 7f4960c0e640 1 -- 172.21.15.112:0/3913243802 wait complete. 2024-09-16T00:23:29.426 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:29 smithi112 ceph-mon[27869]: mds.? [v2:172.21.15.112:6826/777980670,v1:172.21.15.112:6827/777980670] up:active 2024-09-16T00:23:29.427 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:29 smithi112 ceph-mon[27869]: fsmap cephfs:1 {0=cephfs.smithi112.ithwzj=up:active} 2024-09-16T00:23:29.427 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:29 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:23:29.427 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:29 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:23:29.427 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:29 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:23:29.427 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:29 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi112.hxfwkh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:23:29.427 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:29 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi112.hxfwkh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:23:29.427 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:29 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:23:29.427 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:29 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/3913243802' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-09-16T00:23:29.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:29 smithi169 ceph-mon[34653]: mds.? [v2:172.21.15.112:6826/777980670,v1:172.21.15.112:6827/777980670] up:active 2024-09-16T00:23:29.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:29 smithi169 ceph-mon[34653]: fsmap cephfs:1 {0=cephfs.smithi112.ithwzj=up:active} 2024-09-16T00:23:29.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:29 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:23:29.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:29 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:23:29.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:29 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:23:29.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:29 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi112.hxfwkh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:23:29.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:29 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi112.hxfwkh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:23:29.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:29 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:23:29.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:29 smithi169 ceph-mon[34653]: from='client.? 172.21.15.112:0/3913243802' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-09-16T00:23:30.201 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-16T00:23:30.211 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi112.front.sepia.ceph.com 2024-09-16T00:23:30.211 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- bash -c 'ceph fs set cephfs allow_standby_replay true' 2024-09-16T00:23:30.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:30 smithi112 ceph-mon[27869]: pgmap v177: 65 pgs: 65 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:23:30.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:30 smithi112 ceph-mon[27869]: Deploying daemon mds.cephfs.smithi112.hxfwkh on smithi112 2024-09-16T00:23:30.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:30 smithi112 ceph-mon[27869]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-16T00:23:30.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:30 smithi112 ceph-mon[27869]: mds.? [v2:172.21.15.169:6824/2892476953,v1:172.21.15.169:6825/2892476953] up:boot 2024-09-16T00:23:30.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:30 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/3913243802' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-09-16T00:23:30.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:30 smithi112 ceph-mon[27869]: daemon mds.cephfs.smithi169.gawwgw assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-16T00:23:30.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:30 smithi112 ceph-mon[27869]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-09-16T00:23:30.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:30 smithi112 ceph-mon[27869]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-16T00:23:30.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:30 smithi112 ceph-mon[27869]: fsmap cephfs:1 {0=cephfs.smithi112.ithwzj=up:active} 1 up:standby 2024-09-16T00:23:30.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:30 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi169.gawwgw"}]: dispatch 2024-09-16T00:23:30.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:30 smithi112 ceph-mon[27869]: fsmap cephfs:2 {0=cephfs.smithi112.ithwzj=up:active,1=cephfs.smithi169.gawwgw=up:creating} 2024-09-16T00:23:30.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:30 smithi112 ceph-mon[27869]: daemon mds.cephfs.smithi169.gawwgw is now active in filesystem cephfs as rank 1 2024-09-16T00:23:30.653 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:23:30.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:30 smithi169 ceph-mon[34653]: pgmap v177: 65 pgs: 65 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:23:30.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:30 smithi169 ceph-mon[34653]: Deploying daemon mds.cephfs.smithi112.hxfwkh on smithi112 2024-09-16T00:23:30.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:30 smithi169 ceph-mon[34653]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-16T00:23:30.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:30 smithi169 ceph-mon[34653]: mds.? [v2:172.21.15.169:6824/2892476953,v1:172.21.15.169:6825/2892476953] up:boot 2024-09-16T00:23:30.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:30 smithi169 ceph-mon[34653]: from='client.? 172.21.15.112:0/3913243802' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-09-16T00:23:30.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:30 smithi169 ceph-mon[34653]: daemon mds.cephfs.smithi169.gawwgw assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-16T00:23:30.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:30 smithi169 ceph-mon[34653]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-09-16T00:23:30.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:30 smithi169 ceph-mon[34653]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-16T00:23:30.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:30 smithi169 ceph-mon[34653]: fsmap cephfs:1 {0=cephfs.smithi112.ithwzj=up:active} 1 up:standby 2024-09-16T00:23:30.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:30 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi169.gawwgw"}]: dispatch 2024-09-16T00:23:30.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:30 smithi169 ceph-mon[34653]: fsmap cephfs:2 {0=cephfs.smithi112.ithwzj=up:active,1=cephfs.smithi169.gawwgw=up:creating} 2024-09-16T00:23:30.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:30 smithi169 ceph-mon[34653]: daemon mds.cephfs.smithi169.gawwgw is now active in filesystem cephfs as rank 1 2024-09-16T00:23:31.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:31 smithi169 ceph-mon[34653]: mds.? [v2:172.21.15.169:6824/2892476953,v1:172.21.15.169:6825/2892476953] up:active 2024-09-16T00:23:31.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:31 smithi169 ceph-mon[34653]: fsmap cephfs:2 {0=cephfs.smithi112.ithwzj=up:active,1=cephfs.smithi169.gawwgw=up:active} 2024-09-16T00:23:31.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:31 smithi112 ceph-mon[27869]: mds.? [v2:172.21.15.169:6824/2892476953,v1:172.21.15.169:6825/2892476953] up:active 2024-09-16T00:23:31.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:31 smithi112 ceph-mon[27869]: fsmap cephfs:2 {0=cephfs.smithi112.ithwzj=up:active,1=cephfs.smithi169.gawwgw=up:active} 2024-09-16T00:23:32.562 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:32 smithi112 ceph-mon[27869]: pgmap v178: 65 pgs: 65 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 895 B/s wr, 1 op/s 2024-09-16T00:23:32.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:32.734+0000 7faafd1a5640 1 -- 172.21.15.112:0/165544474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faaf8066710 msgr2=0x7faaf80ff160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:32.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:32.734+0000 7faafd1a5640 1 --2- 172.21.15.112:0/165544474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faaf8066710 0x7faaf80ff160 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7faae80048d0 tx=0x7faae8009da0 comp rx=0 tx=0).stop 2024-09-16T00:23:32.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:32.734+0000 7faafd1a5640 1 -- 172.21.15.112:0/165544474 shutdown_connections 2024-09-16T00:23:32.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:32.734+0000 7faafd1a5640 1 --2- 172.21.15.112:0/165544474 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faaf80ff6a0 0x7faaf8101a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:32.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:32.734+0000 7faafd1a5640 1 --2- 172.21.15.112:0/165544474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faaf8066710 0x7faaf80ff160 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:32.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:32.734+0000 7faafd1a5640 1 -- 172.21.15.112:0/165544474 >> 172.21.15.112:0/165544474 conn(0x7faaf80f6c10 msgr2=0x7faaf80f9040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:32.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:32.734+0000 7faafd1a5640 1 -- 172.21.15.112:0/165544474 shutdown_connections 2024-09-16T00:23:32.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:32.734+0000 7faafd1a5640 1 -- 172.21.15.112:0/165544474 wait complete. 2024-09-16T00:23:32.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:32.735+0000 7faafd1a5640 1 Processor -- start 2024-09-16T00:23:32.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:32.735+0000 7faafd1a5640 1 -- start start 2024-09-16T00:23:32.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:32.735+0000 7faafd1a5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faaf8066710 0x7faaf80fcb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:32.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:32.735+0000 7faafd1a5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faaf80ff6a0 0x7faaf80fb060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:32.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:32.735+0000 7faafd1a5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaf80fb5a0 con 0x7faaf8066710 2024-09-16T00:23:32.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:32.735+0000 7faafd1a5640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaf80fb6e0 con 0x7faaf80ff6a0 2024-09-16T00:23:32.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:32.735+0000 7faaf6d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faaf8066710 0x7faaf80fcb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:32.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:32.735+0000 7faaf6d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faaf8066710 0x7faaf80fcb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37180/0 (socket says 172.21.15.112:37180) 2024-09-16T00:23:32.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:32.735+0000 7faaf6d9d640 1 -- 172.21.15.112:0/173399371 learned_addr learned my addr 172.21.15.112:0/173399371 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:23:32.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:32.736+0000 7faaf659c640 1 --2- 172.21.15.112:0/173399371 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faaf80ff6a0 0x7faaf80fb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:32.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:32.736+0000 7faaf6d9d640 1 -- 172.21.15.112:0/173399371 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faaf80ff6a0 msgr2=0x7faaf80fb060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:32.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:32.736+0000 7faaf6d9d640 1 --2- 172.21.15.112:0/173399371 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faaf80ff6a0 0x7faaf80fb060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:32.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:32.736+0000 7faaf6d9d640 1 -- 172.21.15.112:0/173399371 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faae8005040 con 0x7faaf8066710 2024-09-16T00:23:32.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:32.736+0000 7faaf6d9d640 1 --2- 172.21.15.112:0/173399371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faaf8066710 0x7faaf80fcb20 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7faae80094f0 tx=0x7faae8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:32.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:32.736+0000 7faadf7fe640 1 -- 172.21.15.112:0/173399371 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faae8038070 con 0x7faaf8066710 2024-09-16T00:23:32.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:32.736+0000 7faafd1a5640 1 -- 172.21.15.112:0/173399371 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaf80fb960 con 0x7faaf8066710 2024-09-16T00:23:32.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:32.736+0000 7faadf7fe640 1 -- 172.21.15.112:0/173399371 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7faae802abd0 con 0x7faaf8066710 2024-09-16T00:23:32.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:32.736+0000 7faadf7fe640 1 -- 172.21.15.112:0/173399371 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faae803e670 con 0x7faaf8066710 2024-09-16T00:23:32.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:32.736+0000 7faafd1a5640 1 -- 172.21.15.112:0/173399371 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaf81151e0 con 0x7faaf8066710 2024-09-16T00:23:32.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:32.737+0000 7faafd1a5640 1 -- 172.21.15.112:0/173399371 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faaf80fa8b0 con 0x7faaf8066710 2024-09-16T00:23:32.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:32.738+0000 7faadf7fe640 1 -- 172.21.15.112:0/173399371 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) v1 ==== 90869+0+0 (secure 0 0 0) 0x7faae8030020 con 0x7faaf8066710 2024-09-16T00:23:32.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:32.738+0000 7faadf7fe640 1 --2- 172.21.15.112:0/173399371 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7faae006ed60 0x7faae0071210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:32.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:32.738+0000 7faaf659c640 1 --2- 172.21.15.112:0/173399371 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7faae006ed60 0x7faae0071210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:32.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:32.738+0000 7faadf7fe640 1 -- 172.21.15.112:0/173399371 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5464+0+0 (secure 0 0 0) 0x7faae80ae480 con 0x7faaf8066710 2024-09-16T00:23:32.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:32.738+0000 7faaf659c640 1 --2- 172.21.15.112:0/173399371 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7faae006ed60 0x7faae0071210 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7faaec0041d0 tx=0x7faaec009040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:32.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:32.743+0000 7faadf7fe640 1 -- 172.21.15.112:0/173399371 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7faae807b5a0 con 0x7faaf8066710 2024-09-16T00:23:32.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:32 smithi169 ceph-mon[34653]: pgmap v178: 65 pgs: 65 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 895 B/s wr, 1 op/s 2024-09-16T00:23:32.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:32.952+0000 7faafd1a5640 1 -- 172.21.15.112:0/173399371 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7faaf804b720 con 0x7faaf8066710 2024-09-16T00:23:33.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:33.393+0000 7faadf7fe640 1 -- 172.21.15.112:0/173399371 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v9) v1 ==== 121+0+0 (secure 0 0 0) 0x7faae807af40 con 0x7faaf8066710 2024-09-16T00:23:33.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:33.396+0000 7faafd1a5640 1 -- 172.21.15.112:0/173399371 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7faae006ed60 msgr2=0x7faae0071210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:33.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:33.396+0000 7faafd1a5640 1 --2- 172.21.15.112:0/173399371 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7faae006ed60 0x7faae0071210 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7faaec0041d0 tx=0x7faaec009040 comp rx=0 tx=0).stop 2024-09-16T00:23:33.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:33.396+0000 7faafd1a5640 1 -- 172.21.15.112:0/173399371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faaf8066710 msgr2=0x7faaf80fcb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:33.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:33.396+0000 7faafd1a5640 1 --2- 172.21.15.112:0/173399371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faaf8066710 0x7faaf80fcb20 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7faae80094f0 tx=0x7faae8009da0 comp rx=0 tx=0).stop 2024-09-16T00:23:33.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:33.396+0000 7faafd1a5640 1 -- 172.21.15.112:0/173399371 shutdown_connections 2024-09-16T00:23:33.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:33.396+0000 7faafd1a5640 1 --2- 172.21.15.112:0/173399371 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7faae006ed60 0x7faae0071210 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:33.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:33.396+0000 7faafd1a5640 1 --2- 172.21.15.112:0/173399371 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faaf80ff6a0 0x7faaf80fb060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:33.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:33.396+0000 7faafd1a5640 1 --2- 172.21.15.112:0/173399371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faaf8066710 0x7faaf80fcb20 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:33.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:33.396+0000 7faafd1a5640 1 -- 172.21.15.112:0/173399371 >> 172.21.15.112:0/173399371 conn(0x7faaf80f6c10 msgr2=0x7faaf80f7030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:33.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:33.396+0000 7faafd1a5640 1 -- 172.21.15.112:0/173399371 shutdown_connections 2024-09-16T00:23:33.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:33.396+0000 7faafd1a5640 1 -- 172.21.15.112:0/173399371 wait complete. 2024-09-16T00:23:33.432 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:33 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/173399371' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-16T00:23:33.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:33 smithi169 ceph-mon[34653]: from='client.? 172.21.15.112:0/173399371' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-16T00:23:34.020 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-16T00:23:34.031 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi112.front.sepia.ceph.com 2024-09-16T00:23:34.031 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- bash -c 'ceph fs set cephfs inline_data true --yes-i-really-really-mean-it' 2024-09-16T00:23:34.279 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:23:34.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:34 smithi169 ceph-mon[34653]: pgmap v179: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s wr, 7 op/s 2024-09-16T00:23:34.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:34 smithi169 ceph-mon[34653]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-09-16T00:23:34.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:34 smithi169 ceph-mon[34653]: Cluster is now healthy 2024-09-16T00:23:34.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:34 smithi169 ceph-mon[34653]: from='client.? 172.21.15.112:0/173399371' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-16T00:23:34.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:34 smithi169 ceph-mon[34653]: mds.? [v2:172.21.15.112:6828/486654885,v1:172.21.15.112:6829/486654885] up:boot 2024-09-16T00:23:34.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:34 smithi169 ceph-mon[34653]: fsmap cephfs:2 {0=cephfs.smithi112.ithwzj=up:active,1=cephfs.smithi169.gawwgw=up:active} 1 up:standby 2024-09-16T00:23:34.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:34 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi112.hxfwkh"}]: dispatch 2024-09-16T00:23:34.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:34 smithi169 ceph-mon[34653]: fsmap cephfs:2 {0=cephfs.smithi112.ithwzj=up:active,1=cephfs.smithi169.gawwgw=up:active} 1 up:standby-replay 2024-09-16T00:23:34.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:34 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:23:34.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:34 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:23:34.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:34 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:23:34.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:34 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi169.lgeqrj", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:23:34.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:34 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi169.lgeqrj", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:23:34.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:34 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:23:34.637 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:34 smithi112 ceph-mon[27869]: pgmap v179: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s wr, 7 op/s 2024-09-16T00:23:34.637 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:34 smithi112 ceph-mon[27869]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-09-16T00:23:34.637 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:34 smithi112 ceph-mon[27869]: Cluster is now healthy 2024-09-16T00:23:34.637 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:34 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/173399371' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-16T00:23:34.637 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:34 smithi112 ceph-mon[27869]: mds.? [v2:172.21.15.112:6828/486654885,v1:172.21.15.112:6829/486654885] up:boot 2024-09-16T00:23:34.638 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:34 smithi112 ceph-mon[27869]: fsmap cephfs:2 {0=cephfs.smithi112.ithwzj=up:active,1=cephfs.smithi169.gawwgw=up:active} 1 up:standby 2024-09-16T00:23:34.638 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:34 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi112.hxfwkh"}]: dispatch 2024-09-16T00:23:34.638 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:34 smithi112 ceph-mon[27869]: fsmap cephfs:2 {0=cephfs.smithi112.ithwzj=up:active,1=cephfs.smithi169.gawwgw=up:active} 1 up:standby-replay 2024-09-16T00:23:34.638 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:34 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:23:34.638 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:34 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:23:34.638 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:34 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:23:34.638 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:34 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi169.lgeqrj", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:23:34.638 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:34 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi169.lgeqrj", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:23:34.638 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:34 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:23:35.648 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:35 smithi169 ceph-mon[34653]: Deploying daemon mds.cephfs.smithi169.lgeqrj on smithi169 2024-09-16T00:23:35.649 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:35 smithi112 ceph-mon[27869]: Deploying daemon mds.cephfs.smithi169.lgeqrj on smithi169 2024-09-16T00:23:35.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:35.835+0000 7f34cff45640 1 -- 172.21.15.112:0/2997140895 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34c80fc550 msgr2=0x7f34c80fc960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:35.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:35.835+0000 7f34cff45640 1 --2- 172.21.15.112:0/2997140895 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34c80fc550 0x7f34c80fc960 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f34b8009c50 tx=0x7f34b8004d10 comp rx=0 tx=0).stop 2024-09-16T00:23:35.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:35.836+0000 7f34cff45640 1 -- 172.21.15.112:0/2997140895 shutdown_connections 2024-09-16T00:23:35.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:35.836+0000 7f34cff45640 1 --2- 172.21.15.112:0/2997140895 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34c80fd820 0x7f34c80fdc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:35.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:35.836+0000 7f34cff45640 1 --2- 172.21.15.112:0/2997140895 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34c80fc550 0x7f34c80fc960 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:35.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:35.836+0000 7f34cff45640 1 -- 172.21.15.112:0/2997140895 >> 172.21.15.112:0/2997140895 conn(0x7f34c80f7d30 msgr2=0x7f34c80fa140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:35.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:35.836+0000 7f34cff45640 1 -- 172.21.15.112:0/2997140895 shutdown_connections 2024-09-16T00:23:35.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:35.836+0000 7f34cff45640 1 -- 172.21.15.112:0/2997140895 wait complete. 2024-09-16T00:23:35.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:35.836+0000 7f34cff45640 1 Processor -- start 2024-09-16T00:23:35.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:35.836+0000 7f34cff45640 1 -- start start 2024-09-16T00:23:35.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:35.836+0000 7f34cff45640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34c80fc550 0x7f34c810d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:35.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:35.836+0000 7f34cff45640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34c80fd820 0x7f34c810d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:35.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:35.836+0000 7f34cff45640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34c810dc30 con 0x7f34c80fc550 2024-09-16T00:23:35.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:35.836+0000 7f34cff45640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34c810e450 con 0x7f34c80fd820 2024-09-16T00:23:35.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:35.837+0000 7f34cdce1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34c80fc550 0x7f34c810d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:35.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:35.837+0000 7f34cdce1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34c80fc550 0x7f34c810d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37232/0 (socket says 172.21.15.112:37232) 2024-09-16T00:23:35.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:35.837+0000 7f34cdce1640 1 -- 172.21.15.112:0/2928242756 learned_addr learned my addr 172.21.15.112:0/2928242756 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:23:35.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:35.837+0000 7f34cd4e0640 1 --2- 172.21.15.112:0/2928242756 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34c80fd820 0x7f34c810d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:35.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:35.837+0000 7f34cdce1640 1 -- 172.21.15.112:0/2928242756 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34c80fd820 msgr2=0x7f34c810d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:35.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:35.837+0000 7f34cdce1640 1 --2- 172.21.15.112:0/2928242756 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34c80fd820 0x7f34c810d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:35.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:35.837+0000 7f34cdce1640 1 -- 172.21.15.112:0/2928242756 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34b80098b0 con 0x7f34c80fc550 2024-09-16T00:23:35.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:35.837+0000 7f34cdce1640 1 --2- 172.21.15.112:0/2928242756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34c80fc550 0x7f34c810d0a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f34b8004a30 tx=0x7f34b8004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:35.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:35.837+0000 7f34c67fc640 1 -- 172.21.15.112:0/2928242756 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34b8032850 con 0x7f34c80fc550 2024-09-16T00:23:35.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:35.837+0000 7f34cff45640 1 -- 172.21.15.112:0/2928242756 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34c810e680 con 0x7f34c80fc550 2024-09-16T00:23:35.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:35.837+0000 7f34c67fc640 1 -- 172.21.15.112:0/2928242756 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f34b8009710 con 0x7f34c80fc550 2024-09-16T00:23:35.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:35.837+0000 7f34c67fc640 1 -- 172.21.15.112:0/2928242756 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34b803dc20 con 0x7f34c80fc550 2024-09-16T00:23:35.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:35.837+0000 7f34cff45640 1 -- 172.21.15.112:0/2928242756 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34c810ec20 con 0x7f34c80fc550 2024-09-16T00:23:35.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:35.838+0000 7f34cff45640 1 -- 172.21.15.112:0/2928242756 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34c80fb9b0 con 0x7f34c80fc550 2024-09-16T00:23:35.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:35.839+0000 7f34c67fc640 1 -- 172.21.15.112:0/2928242756 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) v1 ==== 90869+0+0 (secure 0 0 0) 0x7f34b802f030 con 0x7f34c80fc550 2024-09-16T00:23:35.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:35.839+0000 7f34c67fc640 1 --2- 172.21.15.112:0/2928242756 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f34b006ed60 0x7f34b0071210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:35.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:35.839+0000 7f34c67fc640 1 -- 172.21.15.112:0/2928242756 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f34b80af6f0 con 0x7f34c80fc550 2024-09-16T00:23:35.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:35.839+0000 7f34cd4e0640 1 --2- 172.21.15.112:0/2928242756 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f34b006ed60 0x7f34b0071210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:35.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:35.840+0000 7f34cd4e0640 1 --2- 172.21.15.112:0/2928242756 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f34b006ed60 0x7f34b0071210 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f34c810ce50 tx=0x7f34bc004330 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:35.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:35.843+0000 7f34c67fc640 1 -- 172.21.15.112:0/2928242756 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f34b807c890 con 0x7f34c80fc550 2024-09-16T00:23:36.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:36.036+0000 7f34cff45640 1 -- 172.21.15.112:0/2928242756 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f34c804b720 con 0x7f34c80fc550 2024-09-16T00:23:36.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:36.393+0000 7f34c67fc640 1 -- 172.21.15.112:0/2928242756 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]=0 inline data enabled v11) v1 ==== 168+0+0 (secure 0 0 0) 0x7f34b807c230 con 0x7f34c80fc550 2024-09-16T00:23:36.394 INFO:teuthology.orchestra.run.smithi112.stderr:inline data enabled 2024-09-16T00:23:36.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:36.396+0000 7f34cff45640 1 -- 172.21.15.112:0/2928242756 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f34b006ed60 msgr2=0x7f34b0071210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:36.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:36.396+0000 7f34cff45640 1 --2- 172.21.15.112:0/2928242756 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f34b006ed60 0x7f34b0071210 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f34c810ce50 tx=0x7f34bc004330 comp rx=0 tx=0).stop 2024-09-16T00:23:36.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:36.396+0000 7f34cff45640 1 -- 172.21.15.112:0/2928242756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34c80fc550 msgr2=0x7f34c810d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:36.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:36.396+0000 7f34cff45640 1 --2- 172.21.15.112:0/2928242756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34c80fc550 0x7f34c810d0a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f34b8004a30 tx=0x7f34b8004d10 comp rx=0 tx=0).stop 2024-09-16T00:23:36.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:36.396+0000 7f34cff45640 1 -- 172.21.15.112:0/2928242756 shutdown_connections 2024-09-16T00:23:36.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:36.396+0000 7f34cff45640 1 --2- 172.21.15.112:0/2928242756 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f34b006ed60 0x7f34b0071210 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:36.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:36.396+0000 7f34cff45640 1 --2- 172.21.15.112:0/2928242756 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34c80fd820 0x7f34c810d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:36.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:36.396+0000 7f34cff45640 1 --2- 172.21.15.112:0/2928242756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34c80fc550 0x7f34c810d0a0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:36.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:36.396+0000 7f34cff45640 1 -- 172.21.15.112:0/2928242756 >> 172.21.15.112:0/2928242756 conn(0x7f34c80f7d30 msgr2=0x7f34c80f9510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:36.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:36.397+0000 7f34cff45640 1 -- 172.21.15.112:0/2928242756 shutdown_connections 2024-09-16T00:23:36.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:36.397+0000 7f34cff45640 1 -- 172.21.15.112:0/2928242756 wait complete. 2024-09-16T00:23:36.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:36 smithi112 ceph-mon[27869]: pgmap v180: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 197 B/s rd, 2.3 KiB/s wr, 8 op/s 2024-09-16T00:23:36.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:36 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/2928242756' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T00:23:36.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:36 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:23:36.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:36 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:23:36.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:36 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:23:36.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:36 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:23:36.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:36 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:23:36.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:36 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:23:36.714 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:36 smithi169 ceph-mon[34653]: pgmap v180: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 197 B/s rd, 2.3 KiB/s wr, 8 op/s 2024-09-16T00:23:36.714 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:36 smithi169 ceph-mon[34653]: from='client.? 172.21.15.112:0/2928242756' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T00:23:36.714 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:36 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:23:36.714 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:36 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:23:36.714 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:36 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:23:36.714 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:36 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:23:36.714 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:36 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:23:36.714 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:36 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:23:37.034 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-16T00:23:37.044 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi112.front.sepia.ceph.com 2024-09-16T00:23:37.044 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-16T00:23:37.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:37 smithi112 ceph-mon[27869]: Health check failed: 1 filesystem with deprecated feature inline_data (FS_INLINE_DATA_DEPRECATED) 2024-09-16T00:23:37.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:37 smithi112 ceph-mon[27869]: mds.? [v2:172.21.15.169:6826/861725728,v1:172.21.15.169:6827/861725728] up:boot 2024-09-16T00:23:37.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:37 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/2928242756' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T00:23:37.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:37 smithi112 ceph-mon[27869]: fsmap cephfs:2 {0=cephfs.smithi112.ithwzj=up:active,1=cephfs.smithi169.gawwgw=up:active} 1 up:standby-replay 1 up:standby 2024-09-16T00:23:37.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:37 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi169.lgeqrj"}]: dispatch 2024-09-16T00:23:37.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:37 smithi112 ceph-mon[27869]: fsmap cephfs:2 {0=cephfs.smithi112.ithwzj=up:active,1=cephfs.smithi169.gawwgw=up:active} 2 up:standby-replay 2024-09-16T00:23:37.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:37 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:23:37.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:37 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:23:37.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:37 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:23:37.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:37 smithi169 ceph-mon[34653]: Health check failed: 1 filesystem with deprecated feature inline_data (FS_INLINE_DATA_DEPRECATED) 2024-09-16T00:23:37.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:37 smithi169 ceph-mon[34653]: mds.? [v2:172.21.15.169:6826/861725728,v1:172.21.15.169:6827/861725728] up:boot 2024-09-16T00:23:37.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:37 smithi169 ceph-mon[34653]: from='client.? 172.21.15.112:0/2928242756' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T00:23:37.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:37 smithi169 ceph-mon[34653]: fsmap cephfs:2 {0=cephfs.smithi112.ithwzj=up:active,1=cephfs.smithi169.gawwgw=up:active} 1 up:standby-replay 1 up:standby 2024-09-16T00:23:37.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:37 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi169.lgeqrj"}]: dispatch 2024-09-16T00:23:37.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:37 smithi169 ceph-mon[34653]: fsmap cephfs:2 {0=cephfs.smithi112.ithwzj=up:active,1=cephfs.smithi169.gawwgw=up:active} 2 up:standby-replay 2024-09-16T00:23:37.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:37 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:23:37.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:37 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:23:37.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:37 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:23:37.935 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:23:38.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:38 smithi112 ceph-mon[27869]: pgmap v181: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 170 B/s rd, 2.0 KiB/s wr, 7 op/s 2024-09-16T00:23:38.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:38 smithi112 ceph-mon[27869]: mds.? [v2:172.21.15.112:6828/486654885,v1:172.21.15.112:6829/486654885] up:standby-replay 2024-09-16T00:23:38.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:38 smithi112 ceph-mon[27869]: mds.? [v2:172.21.15.169:6824/2892476953,v1:172.21.15.169:6825/2892476953] up:active 2024-09-16T00:23:38.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:38 smithi112 ceph-mon[27869]: Dropping low affinity active daemon mds.cephfs.smithi112.ithwzj in favor of higher affinity standby. 2024-09-16T00:23:38.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:38 smithi112 ceph-mon[27869]: Replacing daemon mds.cephfs.smithi112.ithwzj as rank 0 with standby daemon mds.cephfs.smithi112.hxfwkh 2024-09-16T00:23:38.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:38 smithi112 ceph-mon[27869]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-16T00:23:38.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:38 smithi112 ceph-mon[27869]: fsmap cephfs:2 {0=cephfs.smithi112.ithwzj=up:active,1=cephfs.smithi169.gawwgw=up:active} 2 up:standby-replay 2024-09-16T00:23:38.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:38 smithi112 ceph-mon[27869]: osdmap e44: 6 total, 6 up, 6 in 2024-09-16T00:23:38.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:38 smithi112 ceph-mon[27869]: fsmap cephfs:2/2 {0=cephfs.smithi112.hxfwkh=up:replay,1=cephfs.smithi169.gawwgw=up:active} 1 up:standby-replay 2024-09-16T00:23:38.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:38 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:23:38.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:38 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:23:38.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:38 smithi169 ceph-mon[34653]: pgmap v181: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 170 B/s rd, 2.0 KiB/s wr, 7 op/s 2024-09-16T00:23:38.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:38 smithi169 ceph-mon[34653]: mds.? [v2:172.21.15.112:6828/486654885,v1:172.21.15.112:6829/486654885] up:standby-replay 2024-09-16T00:23:38.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:38 smithi169 ceph-mon[34653]: mds.? [v2:172.21.15.169:6824/2892476953,v1:172.21.15.169:6825/2892476953] up:active 2024-09-16T00:23:38.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:38 smithi169 ceph-mon[34653]: Dropping low affinity active daemon mds.cephfs.smithi112.ithwzj in favor of higher affinity standby. 2024-09-16T00:23:38.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:38 smithi169 ceph-mon[34653]: Replacing daemon mds.cephfs.smithi112.ithwzj as rank 0 with standby daemon mds.cephfs.smithi112.hxfwkh 2024-09-16T00:23:38.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:38 smithi169 ceph-mon[34653]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-16T00:23:38.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:38 smithi169 ceph-mon[34653]: fsmap cephfs:2 {0=cephfs.smithi112.ithwzj=up:active,1=cephfs.smithi169.gawwgw=up:active} 2 up:standby-replay 2024-09-16T00:23:38.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:38 smithi169 ceph-mon[34653]: osdmap e44: 6 total, 6 up, 6 in 2024-09-16T00:23:38.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:38 smithi169 ceph-mon[34653]: fsmap cephfs:2/2 {0=cephfs.smithi112.hxfwkh=up:replay,1=cephfs.smithi169.gawwgw=up:active} 1 up:standby-replay 2024-09-16T00:23:38.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:38 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:23:38.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:38 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:23:39.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:39.575+0000 7fbf19181640 1 -- 172.21.15.112:0/3749386798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf140ff710 msgr2=0x7fbf14101af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:39.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:39.575+0000 7fbf19181640 1 --2- 172.21.15.112:0/3749386798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf140ff710 0x7fbf14101af0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fbefc0048d0 tx=0x7fbefc009da0 comp rx=0 tx=0).stop 2024-09-16T00:23:39.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:39.576+0000 7fbf19181640 1 -- 172.21.15.112:0/3749386798 shutdown_connections 2024-09-16T00:23:39.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:39.576+0000 7fbf19181640 1 --2- 172.21.15.112:0/3749386798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf140ff710 0x7fbf14101af0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:39.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:39.576+0000 7fbf19181640 1 --2- 172.21.15.112:0/3749386798 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf14066430 0x7fbf140ff1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:39.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:39.576+0000 7fbf19181640 1 -- 172.21.15.112:0/3749386798 >> 172.21.15.112:0/3749386798 conn(0x7fbf140f6c00 msgr2=0x7fbf140f9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:39.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:39.576+0000 7fbf19181640 1 -- 172.21.15.112:0/3749386798 shutdown_connections 2024-09-16T00:23:39.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:39.576+0000 7fbf19181640 1 -- 172.21.15.112:0/3749386798 wait complete. 2024-09-16T00:23:39.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:39.576+0000 7fbf19181640 1 Processor -- start 2024-09-16T00:23:39.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:39.576+0000 7fbf19181640 1 -- start start 2024-09-16T00:23:39.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:39.577+0000 7fbf19181640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf14066430 0x7fbf1410d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:39.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:39.577+0000 7fbf19181640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf140ff710 0x7fbf1410d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:39.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:39.577+0000 7fbf19181640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1410dca0 con 0x7fbf14066430 2024-09-16T00:23:39.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:39.577+0000 7fbf19181640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1410e470 con 0x7fbf140ff710 2024-09-16T00:23:39.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:39.577+0000 7fbf12d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf14066430 0x7fbf1410d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:39.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:39.577+0000 7fbf12d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf14066430 0x7fbf1410d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37270/0 (socket says 172.21.15.112:37270) 2024-09-16T00:23:39.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:39.577+0000 7fbf12d9d640 1 -- 172.21.15.112:0/2726827487 learned_addr learned my addr 172.21.15.112:0/2726827487 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:23:39.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:39.577+0000 7fbf1259c640 1 --2- 172.21.15.112:0/2726827487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf140ff710 0x7fbf1410d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:39.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:39.577+0000 7fbf12d9d640 1 -- 172.21.15.112:0/2726827487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf140ff710 msgr2=0x7fbf1410d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:39.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:39.577+0000 7fbf12d9d640 1 --2- 172.21.15.112:0/2726827487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf140ff710 0x7fbf1410d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:39.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:39.577+0000 7fbf12d9d640 1 -- 172.21.15.112:0/2726827487 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbefc005040 con 0x7fbf14066430 2024-09-16T00:23:39.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:39.577+0000 7fbf1259c640 1 --2- 172.21.15.112:0/2726827487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf140ff710 0x7fbf1410d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:23:39.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:39.577+0000 7fbf12d9d640 1 --2- 172.21.15.112:0/2726827487 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf14066430 0x7fbf1410d110 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fbf0400db00 tx=0x7fbf04004e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:39.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:39.577+0000 7fbf0b7fe640 1 -- 172.21.15.112:0/2726827487 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf0400b910 con 0x7fbf14066430 2024-09-16T00:23:39.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:39.577+0000 7fbf0b7fe640 1 -- 172.21.15.112:0/2726827487 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fbf0400f040 con 0x7fbf14066430 2024-09-16T00:23:39.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:39.577+0000 7fbf19181640 1 -- 172.21.15.112:0/2726827487 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf1410e700 con 0x7fbf14066430 2024-09-16T00:23:39.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:39.577+0000 7fbf0b7fe640 1 -- 172.21.15.112:0/2726827487 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf04012390 con 0x7fbf14066430 2024-09-16T00:23:39.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:39.577+0000 7fbf19181640 1 -- 172.21.15.112:0/2726827487 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf1410eca0 con 0x7fbf14066430 2024-09-16T00:23:39.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:39.578+0000 7fbf19181640 1 -- 172.21.15.112:0/2726827487 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf140fa880 con 0x7fbf14066430 2024-09-16T00:23:39.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:39.579+0000 7fbf0b7fe640 1 -- 172.21.15.112:0/2726827487 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) v1 ==== 90869+0+0 (secure 0 0 0) 0x7fbf04013430 con 0x7fbf14066430 2024-09-16T00:23:39.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:39.579+0000 7fbf0b7fe640 1 --2- 172.21.15.112:0/2726827487 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fbf0006ed60 0x7fbf00071210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:39.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:39.580+0000 7fbf1259c640 1 --2- 172.21.15.112:0/2726827487 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fbf0006ed60 0x7fbf00071210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:39.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:39.580+0000 7fbf0b7fe640 1 -- 172.21.15.112:0/2726827487 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fbf0408e790 con 0x7fbf14066430 2024-09-16T00:23:39.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:39.580+0000 7fbf1259c640 1 --2- 172.21.15.112:0/2726827487 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fbf0006ed60 0x7fbf00071210 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbf1410cec0 tx=0x7fbefc035040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:39.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:39.584+0000 7fbf0b7fe640 1 -- 172.21.15.112:0/2726827487 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fbf0405b860 con 0x7fbf14066430 2024-09-16T00:23:39.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:39.776+0000 7fbf19181640 1 -- 172.21.15.112:0/2726827487 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fbf1410f0a0 con 0x7fbf14066430 2024-09-16T00:23:39.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:39.777+0000 7fbf0b7fe640 1 -- 172.21.15.112:0/2726827487 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 16 v16) v1 ==== 76+0+1834 (secure 0 0 0) 0x7fbf0408e070 con 0x7fbf14066430 2024-09-16T00:23:39.778 INFO:teuthology.orchestra.run.smithi112.stdout:e16 2024-09-16T00:23:39.778 INFO:teuthology.orchestra.run.smithi112.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T00:23:39.778 INFO:teuthology.orchestra.run.smithi112.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:23:39.778 INFO:teuthology.orchestra.run.smithi112.stdout:legacy client fscid: 1 2024-09-16T00:23:39.778 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:23:39.778 INFO:teuthology.orchestra.run.smithi112.stdout:Filesystem 'cephfs' (1) 2024-09-16T00:23:39.778 INFO:teuthology.orchestra.run.smithi112.stdout:fs_name cephfs 2024-09-16T00:23:39.778 INFO:teuthology.orchestra.run.smithi112.stdout:epoch 16 2024-09-16T00:23:39.778 INFO:teuthology.orchestra.run.smithi112.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-16T00:23:39.779 INFO:teuthology.orchestra.run.smithi112.stdout:created 2024-09-16T00:23:22.328199+0000 2024-09-16T00:23:39.779 INFO:teuthology.orchestra.run.smithi112.stdout:modified 2024-09-16T00:23:39.745179+0000 2024-09-16T00:23:39.779 INFO:teuthology.orchestra.run.smithi112.stdout:tableserver 0 2024-09-16T00:23:39.779 INFO:teuthology.orchestra.run.smithi112.stdout:root 0 2024-09-16T00:23:39.779 INFO:teuthology.orchestra.run.smithi112.stdout:session_timeout 60 2024-09-16T00:23:39.779 INFO:teuthology.orchestra.run.smithi112.stdout:session_autoclose 300 2024-09-16T00:23:39.779 INFO:teuthology.orchestra.run.smithi112.stdout:max_file_size 1099511627776 2024-09-16T00:23:39.779 INFO:teuthology.orchestra.run.smithi112.stdout:required_client_features {} 2024-09-16T00:23:39.779 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure 0 2024-09-16T00:23:39.779 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure_osd_epoch 44 2024-09-16T00:23:39.779 INFO:teuthology.orchestra.run.smithi112.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:23:39.779 INFO:teuthology.orchestra.run.smithi112.stdout:max_mds 2 2024-09-16T00:23:39.779 INFO:teuthology.orchestra.run.smithi112.stdout:in 0,1 2024-09-16T00:23:39.779 INFO:teuthology.orchestra.run.smithi112.stdout:up {0=14480,1=24259} 2024-09-16T00:23:39.779 INFO:teuthology.orchestra.run.smithi112.stdout:failed 2024-09-16T00:23:39.780 INFO:teuthology.orchestra.run.smithi112.stdout:damaged 2024-09-16T00:23:39.780 INFO:teuthology.orchestra.run.smithi112.stdout:stopped 2024-09-16T00:23:39.780 INFO:teuthology.orchestra.run.smithi112.stdout:data_pools [3] 2024-09-16T00:23:39.780 INFO:teuthology.orchestra.run.smithi112.stdout:metadata_pool 2 2024-09-16T00:23:39.780 INFO:teuthology.orchestra.run.smithi112.stdout:inline_data enabled 2024-09-16T00:23:39.780 INFO:teuthology.orchestra.run.smithi112.stdout:balancer 2024-09-16T00:23:39.780 INFO:teuthology.orchestra.run.smithi112.stdout:standby_count_wanted 1 2024-09-16T00:23:39.780 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.hxfwkh{0:14480} state up:reconnect seq 4 join_fscid=1 addr [v2:172.21.15.112:6828/486654885,v1:172.21.15.112:6829/486654885] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T00:23:39.780 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.gawwgw{1:24259} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.169:6824/2892476953,v1:172.21.15.169:6825/2892476953] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T00:23:39.780 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.lgeqrj{1:24269} state up:standby-replay seq 1 addr [v2:172.21.15.169:6826/861725728,v1:172.21.15.169:6827/861725728] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T00:23:39.780 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:23:39.780 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:23:39.780 INFO:teuthology.orchestra.run.smithi112.stdout:Standby daemons: 2024-09-16T00:23:39.780 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:23:39.781 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.ithwzj{-1:14492} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.112:6826/381830565,v1:172.21.15.112:6827/381830565] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T00:23:39.781 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 16 2024-09-16T00:23:39.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:39.779+0000 7fbf19181640 1 -- 172.21.15.112:0/2726827487 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fbf0006ed60 msgr2=0x7fbf00071210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:39.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:39.779+0000 7fbf19181640 1 --2- 172.21.15.112:0/2726827487 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fbf0006ed60 0x7fbf00071210 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbf1410cec0 tx=0x7fbefc035040 comp rx=0 tx=0).stop 2024-09-16T00:23:39.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:39.779+0000 7fbf19181640 1 -- 172.21.15.112:0/2726827487 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf14066430 msgr2=0x7fbf1410d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:39.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:39.779+0000 7fbf19181640 1 --2- 172.21.15.112:0/2726827487 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf14066430 0x7fbf1410d110 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fbf0400db00 tx=0x7fbf04004e30 comp rx=0 tx=0).stop 2024-09-16T00:23:39.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:39.779+0000 7fbf19181640 1 -- 172.21.15.112:0/2726827487 shutdown_connections 2024-09-16T00:23:39.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:39.779+0000 7fbf19181640 1 --2- 172.21.15.112:0/2726827487 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fbf0006ed60 0x7fbf00071210 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:39.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:39.779+0000 7fbf19181640 1 --2- 172.21.15.112:0/2726827487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf140ff710 0x7fbf1410d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:39.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:39.779+0000 7fbf19181640 1 --2- 172.21.15.112:0/2726827487 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf14066430 0x7fbf1410d110 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:39.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:39.779+0000 7fbf19181640 1 -- 172.21.15.112:0/2726827487 >> 172.21.15.112:0/2726827487 conn(0x7fbf140f6c00 msgr2=0x7fbf140f6fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:39.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:39.779+0000 7fbf19181640 1 -- 172.21.15.112:0/2726827487 shutdown_connections 2024-09-16T00:23:39.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:39.779+0000 7fbf19181640 1 -- 172.21.15.112:0/2726827487 wait complete. 2024-09-16T00:23:40.015 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:39 smithi112 ceph-mon[27869]: pgmap v183: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 2.4 KiB/s wr, 10 op/s 2024-09-16T00:23:40.015 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:39 smithi112 ceph-mon[27869]: mds.? [v2:172.21.15.112:6826/381830565,v1:172.21.15.112:6827/381830565] up:boot 2024-09-16T00:23:40.015 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:39 smithi112 ceph-mon[27869]: mds.? [v2:172.21.15.112:6828/486654885,v1:172.21.15.112:6829/486654885] up:resolve 2024-09-16T00:23:40.015 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:39 smithi112 ceph-mon[27869]: fsmap cephfs:2/2 {0=cephfs.smithi112.hxfwkh=up:resolve,1=cephfs.smithi169.gawwgw=up:active} 1 up:standby-replay 1 up:standby 2024-09-16T00:23:40.015 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:39 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi112.ithwzj"}]: dispatch 2024-09-16T00:23:40.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:39 smithi169 ceph-mon[34653]: pgmap v183: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 2.4 KiB/s wr, 10 op/s 2024-09-16T00:23:40.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:39 smithi169 ceph-mon[34653]: mds.? [v2:172.21.15.112:6826/381830565,v1:172.21.15.112:6827/381830565] up:boot 2024-09-16T00:23:40.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:39 smithi169 ceph-mon[34653]: mds.? [v2:172.21.15.112:6828/486654885,v1:172.21.15.112:6829/486654885] up:resolve 2024-09-16T00:23:40.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:39 smithi169 ceph-mon[34653]: fsmap cephfs:2/2 {0=cephfs.smithi112.hxfwkh=up:resolve,1=cephfs.smithi169.gawwgw=up:active} 1 up:standby-replay 1 up:standby 2024-09-16T00:23:40.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:39 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi112.ithwzj"}]: dispatch 2024-09-16T00:23:40.915 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:40 smithi112 ceph-mon[27869]: mds.? [v2:172.21.15.112:6828/486654885,v1:172.21.15.112:6829/486654885] up:reconnect 2024-09-16T00:23:40.915 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:40 smithi112 ceph-mon[27869]: fsmap cephfs:2/2 {0=cephfs.smithi112.hxfwkh=up:reconnect,1=cephfs.smithi169.gawwgw=up:active} 1 up:standby-replay 1 up:standby 2024-09-16T00:23:40.915 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:40 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/2726827487' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:23:40.939 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- bash -c 'ceph --format=json fs dump | jq -e ".filesystems | length == 1"' 2024-09-16T00:23:41.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:40 smithi169 ceph-mon[34653]: mds.? [v2:172.21.15.112:6828/486654885,v1:172.21.15.112:6829/486654885] up:reconnect 2024-09-16T00:23:41.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:40 smithi169 ceph-mon[34653]: fsmap cephfs:2/2 {0=cephfs.smithi112.hxfwkh=up:reconnect,1=cephfs.smithi169.gawwgw=up:active} 1 up:standby-replay 1 up:standby 2024-09-16T00:23:41.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:40 smithi169 ceph-mon[34653]: from='client.? 172.21.15.112:0/2726827487' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:23:42.012 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:41 smithi112 ceph-mon[27869]: pgmap v184: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-09-16T00:23:42.013 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:41 smithi112 ceph-mon[27869]: mds.? [v2:172.21.15.112:6828/486654885,v1:172.21.15.112:6829/486654885] up:rejoin 2024-09-16T00:23:42.013 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:41 smithi112 ceph-mon[27869]: mds.? [v2:172.21.15.169:6826/861725728,v1:172.21.15.169:6827/861725728] up:standby-replay 2024-09-16T00:23:42.013 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:41 smithi112 ceph-mon[27869]: fsmap cephfs:2/2 {0=cephfs.smithi112.hxfwkh=up:rejoin,1=cephfs.smithi169.gawwgw=up:active} 1 up:standby-replay 1 up:standby 2024-09-16T00:23:42.013 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:41 smithi112 ceph-mon[27869]: daemon mds.cephfs.smithi112.hxfwkh is now active in filesystem cephfs as rank 0 2024-09-16T00:23:42.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:41 smithi169 ceph-mon[34653]: pgmap v184: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-09-16T00:23:42.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:41 smithi169 ceph-mon[34653]: mds.? [v2:172.21.15.112:6828/486654885,v1:172.21.15.112:6829/486654885] up:rejoin 2024-09-16T00:23:42.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:41 smithi169 ceph-mon[34653]: mds.? [v2:172.21.15.169:6826/861725728,v1:172.21.15.169:6827/861725728] up:standby-replay 2024-09-16T00:23:42.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:41 smithi169 ceph-mon[34653]: fsmap cephfs:2/2 {0=cephfs.smithi112.hxfwkh=up:rejoin,1=cephfs.smithi169.gawwgw=up:active} 1 up:standby-replay 1 up:standby 2024-09-16T00:23:42.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:41 smithi169 ceph-mon[34653]: daemon mds.cephfs.smithi112.hxfwkh is now active in filesystem cephfs as rank 0 2024-09-16T00:23:42.332 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:23:43.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:42 smithi169 ceph-mon[34653]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-16T00:23:43.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:42 smithi169 ceph-mon[34653]: mds.? [v2:172.21.15.112:6828/486654885,v1:172.21.15.112:6829/486654885] up:active 2024-09-16T00:23:43.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:42 smithi169 ceph-mon[34653]: fsmap cephfs:2 {0=cephfs.smithi112.hxfwkh=up:active,1=cephfs.smithi169.gawwgw=up:active} 1 up:standby-replay 1 up:standby 2024-09-16T00:23:43.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:42 smithi169 ceph-mon[34653]: fsmap cephfs:2 {0=cephfs.smithi112.hxfwkh=up:active,1=cephfs.smithi169.gawwgw=up:active} 2 up:standby-replay 2024-09-16T00:23:43.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:42 smithi112 ceph-mon[27869]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-16T00:23:43.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:42 smithi112 ceph-mon[27869]: mds.? [v2:172.21.15.112:6828/486654885,v1:172.21.15.112:6829/486654885] up:active 2024-09-16T00:23:43.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:42 smithi112 ceph-mon[27869]: fsmap cephfs:2 {0=cephfs.smithi112.hxfwkh=up:active,1=cephfs.smithi169.gawwgw=up:active} 1 up:standby-replay 1 up:standby 2024-09-16T00:23:43.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:42 smithi112 ceph-mon[27869]: fsmap cephfs:2 {0=cephfs.smithi112.hxfwkh=up:active,1=cephfs.smithi169.gawwgw=up:active} 2 up:standby-replay 2024-09-16T00:23:44.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:43 smithi169 ceph-mon[34653]: pgmap v185: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 4.8 KiB/s rd, 614 B/s wr, 7 op/s 2024-09-16T00:23:44.040 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:43 smithi112 ceph-mon[27869]: pgmap v185: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 4.8 KiB/s rd, 614 B/s wr, 7 op/s 2024-09-16T00:23:44.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:44.553+0000 7ffb4e4fc640 1 -- 172.21.15.112:0/3212445781 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb480fdbc0 msgr2=0x7ffb480fdfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:44.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:44.553+0000 7ffb4e4fc640 1 --2- 172.21.15.112:0/3212445781 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb480fdbc0 0x7ffb480fdfd0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ffb380048d0 tx=0x7ffb38009da0 comp rx=0 tx=0).stop 2024-09-16T00:23:44.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:44.553+0000 7ffb4e4fc640 1 -- 172.21.15.112:0/3212445781 shutdown_connections 2024-09-16T00:23:44.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:44.553+0000 7ffb4e4fc640 1 --2- 172.21.15.112:0/3212445781 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb480fe510 0x7ffb480fa670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:44.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:44.553+0000 7ffb4e4fc640 1 --2- 172.21.15.112:0/3212445781 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb480fdbc0 0x7ffb480fdfd0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:44.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:44.553+0000 7ffb4e4fc640 1 -- 172.21.15.112:0/3212445781 >> 172.21.15.112:0/3212445781 conn(0x7ffb480f63c0 msgr2=0x7ffb480f87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:44.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:44.553+0000 7ffb4e4fc640 1 -- 172.21.15.112:0/3212445781 shutdown_connections 2024-09-16T00:23:44.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:44.553+0000 7ffb4e4fc640 1 -- 172.21.15.112:0/3212445781 wait complete. 2024-09-16T00:23:44.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:44.554+0000 7ffb4e4fc640 1 Processor -- start 2024-09-16T00:23:44.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:44.554+0000 7ffb4e4fc640 1 -- start start 2024-09-16T00:23:44.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:44.554+0000 7ffb4e4fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb480fdbc0 0x7ffb4810d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:44.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:44.554+0000 7ffb4e4fc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb480fe510 0x7ffb4810d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:44.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:44.554+0000 7ffb4e4fc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4810dce0 con 0x7ffb480fdbc0 2024-09-16T00:23:44.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:44.554+0000 7ffb4e4fc640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4810e500 con 0x7ffb480fe510 2024-09-16T00:23:44.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:44.554+0000 7ffb47fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb480fdbc0 0x7ffb4810d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:44.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:44.554+0000 7ffb47fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb480fdbc0 0x7ffb4810d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51936/0 (socket says 172.21.15.112:51936) 2024-09-16T00:23:44.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:44.554+0000 7ffb47fff640 1 -- 172.21.15.112:0/2356525924 learned_addr learned my addr 172.21.15.112:0/2356525924 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:23:44.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:44.554+0000 7ffb477fe640 1 --2- 172.21.15.112:0/2356525924 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb480fe510 0x7ffb4810d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:44.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:44.554+0000 7ffb47fff640 1 -- 172.21.15.112:0/2356525924 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb480fe510 msgr2=0x7ffb4810d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:44.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:44.555+0000 7ffb47fff640 1 --2- 172.21.15.112:0/2356525924 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb480fe510 0x7ffb4810d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:44.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:44.555+0000 7ffb47fff640 1 -- 172.21.15.112:0/2356525924 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb38005040 con 0x7ffb480fdbc0 2024-09-16T00:23:44.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:44.555+0000 7ffb47fff640 1 --2- 172.21.15.112:0/2356525924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb480fdbc0 0x7ffb4810d150 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ffb380043d0 tx=0x7ffb38009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:44.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:44.555+0000 7ffb44ff9640 1 -- 172.21.15.112:0/2356525924 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb38038070 con 0x7ffb480fdbc0 2024-09-16T00:23:44.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:44.555+0000 7ffb44ff9640 1 -- 172.21.15.112:0/2356525924 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7ffb3802ae90 con 0x7ffb480fdbc0 2024-09-16T00:23:44.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:44.555+0000 7ffb4e4fc640 1 -- 172.21.15.112:0/2356525924 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb4810e730 con 0x7ffb480fdbc0 2024-09-16T00:23:44.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:44.555+0000 7ffb44ff9640 1 -- 172.21.15.112:0/2356525924 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb3803e670 con 0x7ffb480fdbc0 2024-09-16T00:23:44.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:44.555+0000 7ffb4e4fc640 1 -- 172.21.15.112:0/2356525924 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb4810ecd0 con 0x7ffb480fdbc0 2024-09-16T00:23:44.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:44.556+0000 7ffb4e4fc640 1 -- 172.21.15.112:0/2356525924 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb480fa060 con 0x7ffb480fdbc0 2024-09-16T00:23:44.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:44.556+0000 7ffb44ff9640 1 -- 172.21.15.112:0/2356525924 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) v1 ==== 90869+0+0 (secure 0 0 0) 0x7ffb38030020 con 0x7ffb480fdbc0 2024-09-16T00:23:44.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:44.557+0000 7ffb44ff9640 1 --2- 172.21.15.112:0/2356525924 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7ffb3006ed60 0x7ffb30071210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:44.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:44.557+0000 7ffb44ff9640 1 -- 172.21.15.112:0/2356525924 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5550+0+0 (secure 0 0 0) 0x7ffb380ae410 con 0x7ffb480fdbc0 2024-09-16T00:23:44.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:44.557+0000 7ffb477fe640 1 --2- 172.21.15.112:0/2356525924 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7ffb3006ed60 0x7ffb30071210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:44.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:44.557+0000 7ffb477fe640 1 --2- 172.21.15.112:0/2356525924 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7ffb3006ed60 0x7ffb30071210 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ffb4810cf00 tx=0x7ffb3c00b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:44.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:44.561+0000 7ffb44ff9640 1 -- 172.21.15.112:0/2356525924 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7ffb3807b4e0 con 0x7ffb480fdbc0 2024-09-16T00:23:44.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:44.764+0000 7ffb4e4fc640 1 -- 172.21.15.112:0/2356525924 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffb4804b720 con 0x7ffb480fdbc0 2024-09-16T00:23:44.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:44.765+0000 7ffb44ff9640 1 -- 172.21.15.112:0/2356525924 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+4722 (secure 0 0 0) 0x7ffb3807ae80 con 0x7ffb480fdbc0 2024-09-16T00:23:44.765 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 19 2024-09-16T00:23:44.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:44.769+0000 7ffb4e4fc640 1 -- 172.21.15.112:0/2356525924 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7ffb3006ed60 msgr2=0x7ffb30071210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:44.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:44.769+0000 7ffb4e4fc640 1 --2- 172.21.15.112:0/2356525924 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7ffb3006ed60 0x7ffb30071210 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ffb4810cf00 tx=0x7ffb3c00b040 comp rx=0 tx=0).stop 2024-09-16T00:23:44.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:44.769+0000 7ffb4e4fc640 1 -- 172.21.15.112:0/2356525924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb480fdbc0 msgr2=0x7ffb4810d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:44.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:44.769+0000 7ffb4e4fc640 1 --2- 172.21.15.112:0/2356525924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb480fdbc0 0x7ffb4810d150 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ffb380043d0 tx=0x7ffb38009da0 comp rx=0 tx=0).stop 2024-09-16T00:23:44.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:44.769+0000 7ffb4e4fc640 1 -- 172.21.15.112:0/2356525924 shutdown_connections 2024-09-16T00:23:44.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:44.769+0000 7ffb4e4fc640 1 --2- 172.21.15.112:0/2356525924 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7ffb3006ed60 0x7ffb30071210 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:44.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:44.769+0000 7ffb4e4fc640 1 --2- 172.21.15.112:0/2356525924 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb480fe510 0x7ffb4810d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:44.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:44.769+0000 7ffb4e4fc640 1 --2- 172.21.15.112:0/2356525924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb480fdbc0 0x7ffb4810d150 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:44.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:44.769+0000 7ffb4e4fc640 1 -- 172.21.15.112:0/2356525924 >> 172.21.15.112:0/2356525924 conn(0x7ffb480f63c0 msgr2=0x7ffb480f7bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:44.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:44.769+0000 7ffb4e4fc640 1 -- 172.21.15.112:0/2356525924 shutdown_connections 2024-09-16T00:23:44.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:44.769+0000 7ffb4e4fc640 1 -- 172.21.15.112:0/2356525924 wait complete. 2024-09-16T00:23:44.779 INFO:teuthology.orchestra.run.smithi112.stdout:true 2024-09-16T00:23:45.370 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- bash -c 'while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done' 2024-09-16T00:23:45.898 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:45 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:23:45.898 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:45 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:23:45.898 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:45 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:23:45.898 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:45 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:23:45.898 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:45 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:23:45.898 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:45 smithi112 ceph-mon[27869]: pgmap v186: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 6.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-16T00:23:45.898 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:45 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:23:45.898 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:45 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/2356525924' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:23:45.954 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:45 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:23:45.954 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:45 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:23:45.954 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:45 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:23:45.954 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:45 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:23:45.954 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:45 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:23:45.954 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:45 smithi169 ceph-mon[34653]: pgmap v186: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 6.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-16T00:23:45.955 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:45 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:23:45.955 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:45 smithi169 ceph-mon[34653]: from='client.? 172.21.15.112:0/2356525924' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:23:46.267 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:23:47.910 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:47 smithi112 ceph-mon[27869]: pgmap v187: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 7.7 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-16T00:23:47.910 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:47 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:23:47.910 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:47 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:23:48.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:47 smithi169 ceph-mon[34653]: pgmap v187: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 7.7 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-16T00:23:48.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:47 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:23:48.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:47 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:23:48.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:48.297+0000 7fccd7b59640 1 -- 172.21.15.112:0/1984276953 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccd00fd800 msgr2=0x7fccd00fdc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:48.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:48.297+0000 7fccd7b59640 1 --2- 172.21.15.112:0/1984276953 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccd00fd800 0x7fccd00fdc70 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fccc00048d0 tx=0x7fccc0009da0 comp rx=0 tx=0).stop 2024-09-16T00:23:48.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:48.297+0000 7fccd7b59640 1 -- 172.21.15.112:0/1984276953 shutdown_connections 2024-09-16T00:23:48.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:48.297+0000 7fccd7b59640 1 --2- 172.21.15.112:0/1984276953 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccd00fd800 0x7fccd00fdc70 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:48.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:48.297+0000 7fccd7b59640 1 --2- 172.21.15.112:0/1984276953 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccd00fc530 0x7fccd00fc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:48.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:48.297+0000 7fccd7b59640 1 -- 172.21.15.112:0/1984276953 >> 172.21.15.112:0/1984276953 conn(0x7fccd00f7d10 msgr2=0x7fccd00fa120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:48.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:48.297+0000 7fccd7b59640 1 -- 172.21.15.112:0/1984276953 shutdown_connections 2024-09-16T00:23:48.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:48.297+0000 7fccd7b59640 1 -- 172.21.15.112:0/1984276953 wait complete. 2024-09-16T00:23:48.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:48.297+0000 7fccd7b59640 1 Processor -- start 2024-09-16T00:23:48.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:48.297+0000 7fccd7b59640 1 -- start start 2024-09-16T00:23:48.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:48.298+0000 7fccd7b59640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccd00fc530 0x7fccd0071570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:48.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:48.298+0000 7fccd7b59640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccd00fd800 0x7fccd006fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:48.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:48.298+0000 7fccd7b59640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd006fff0 con 0x7fccd00fc530 2024-09-16T00:23:48.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:48.298+0000 7fccd7b59640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd0070130 con 0x7fccd00fd800 2024-09-16T00:23:48.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:48.298+0000 7fccd58f5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccd00fc530 0x7fccd0071570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:48.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:48.298+0000 7fccd58f5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccd00fc530 0x7fccd0071570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51946/0 (socket says 172.21.15.112:51946) 2024-09-16T00:23:48.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:48.298+0000 7fccd58f5640 1 -- 172.21.15.112:0/3902556096 learned_addr learned my addr 172.21.15.112:0/3902556096 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:23:48.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:48.298+0000 7fccd50f4640 1 --2- 172.21.15.112:0/3902556096 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccd00fd800 0x7fccd006fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:48.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:48.298+0000 7fccd58f5640 1 -- 172.21.15.112:0/3902556096 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccd00fd800 msgr2=0x7fccd006fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:48.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:48.298+0000 7fccd58f5640 1 --2- 172.21.15.112:0/3902556096 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccd00fd800 0x7fccd006fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:48.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:48.298+0000 7fccd58f5640 1 -- 172.21.15.112:0/3902556096 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccc0005040 con 0x7fccd00fc530 2024-09-16T00:23:48.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:48.298+0000 7fccd58f5640 1 --2- 172.21.15.112:0/3902556096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccd00fc530 0x7fccd0071570 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fcccc002a50 tx=0x7fcccc004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:48.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:48.298+0000 7fccc67fc640 1 -- 172.21.15.112:0/3902556096 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcccc007c30 con 0x7fccd00fc530 2024-09-16T00:23:48.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:48.298+0000 7fccd7b59640 1 -- 172.21.15.112:0/3902556096 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccd0070410 con 0x7fccd00fc530 2024-09-16T00:23:48.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:48.298+0000 7fccc67fc640 1 -- 172.21.15.112:0/3902556096 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fcccc00f040 con 0x7fccd00fc530 2024-09-16T00:23:48.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:48.298+0000 7fccc67fc640 1 -- 172.21.15.112:0/3902556096 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcccc00da90 con 0x7fccd00fc530 2024-09-16T00:23:48.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:48.298+0000 7fccd7b59640 1 -- 172.21.15.112:0/3902556096 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccd01150f0 con 0x7fccd00fc530 2024-09-16T00:23:48.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:48.299+0000 7fccd7b59640 1 -- 172.21.15.112:0/3902556096 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccd00fb990 con 0x7fccd00fc530 2024-09-16T00:23:48.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:48.300+0000 7fccc67fc640 1 -- 172.21.15.112:0/3902556096 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) v1 ==== 90869+0+0 (secure 0 0 0) 0x7fcccc00dd00 con 0x7fccd00fc530 2024-09-16T00:23:48.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:48.300+0000 7fccc67fc640 1 --2- 172.21.15.112:0/3902556096 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fccbc06ed60 0x7fccbc071210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:48.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:48.300+0000 7fccc67fc640 1 -- 172.21.15.112:0/3902556096 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fcccc08dab0 con 0x7fccd00fc530 2024-09-16T00:23:48.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:48.300+0000 7fccd50f4640 1 --2- 172.21.15.112:0/3902556096 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fccbc06ed60 0x7fccbc071210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:48.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:48.301+0000 7fccd50f4640 1 --2- 172.21.15.112:0/3902556096 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fccbc06ed60 0x7fccbc071210 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fccc0007ac0 tx=0x7fccc0035040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:48.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:48.304+0000 7fccc67fc640 1 -- 172.21.15.112:0/3902556096 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fcccc04d2a0 con 0x7fccd00fc530 2024-09-16T00:23:48.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:48.524+0000 7fccd7b59640 1 -- 172.21.15.112:0/3902556096 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mds versions", "format": "json"} v 0) v1 -- 0x7fccd0115320 con 0x7fccd00fc530 2024-09-16T00:23:48.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:48.524+0000 7fccc67fc640 1 -- 172.21.15.112:0/3902556096 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mds versions", "format": "json"}]=0 v19) v1 ==== 78+0+100 (secure 0 0 0) 0x7fcccc05ac40 con 0x7fccd00fc530 2024-09-16T00:23:48.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:48.526+0000 7fccd7b59640 1 -- 172.21.15.112:0/3902556096 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fccbc06ed60 msgr2=0x7fccbc071210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:48.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:48.526+0000 7fccd7b59640 1 --2- 172.21.15.112:0/3902556096 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fccbc06ed60 0x7fccbc071210 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fccc0007ac0 tx=0x7fccc0035040 comp rx=0 tx=0).stop 2024-09-16T00:23:48.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:48.526+0000 7fccd7b59640 1 -- 172.21.15.112:0/3902556096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccd00fc530 msgr2=0x7fccd0071570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:48.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:48.526+0000 7fccd7b59640 1 --2- 172.21.15.112:0/3902556096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccd00fc530 0x7fccd0071570 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fcccc002a50 tx=0x7fcccc004600 comp rx=0 tx=0).stop 2024-09-16T00:23:48.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:48.527+0000 7fccd7b59640 1 -- 172.21.15.112:0/3902556096 shutdown_connections 2024-09-16T00:23:48.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:48.527+0000 7fccd7b59640 1 --2- 172.21.15.112:0/3902556096 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fccbc06ed60 0x7fccbc071210 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:48.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:48.527+0000 7fccd7b59640 1 --2- 172.21.15.112:0/3902556096 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccd00fd800 0x7fccd006fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:48.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:48.527+0000 7fccd7b59640 1 --2- 172.21.15.112:0/3902556096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccd00fc530 0x7fccd0071570 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:48.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:48.527+0000 7fccd7b59640 1 -- 172.21.15.112:0/3902556096 >> 172.21.15.112:0/3902556096 conn(0x7fccd00f7d10 msgr2=0x7fccd00f94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:48.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:48.527+0000 7fccd7b59640 1 -- 172.21.15.112:0/3902556096 shutdown_connections 2024-09-16T00:23:48.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:48.527+0000 7fccd7b59640 1 -- 172.21.15.112:0/3902556096 wait complete. 2024-09-16T00:23:48.538 INFO:teuthology.orchestra.run.smithi112.stdout:true 2024-09-16T00:23:48.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:48 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/3902556096' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-09-16T00:23:49.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:48 smithi169 ceph-mon[34653]: from='client.? 172.21.15.112:0/3902556096' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-09-16T00:23:49.267 INFO:teuthology.run_tasks:Running task fs.pre_upgrade_save... 2024-09-16T00:23:49.285 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:23:49.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:49 smithi112 ceph-mon[27869]: pgmap v188: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 6.7 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-16T00:23:50.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:49 smithi169 ceph-mon[34653]: pgmap v188: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 6.7 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-16T00:23:50.363 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:23:52.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:51 smithi169 ceph-mon[34653]: pgmap v189: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 5.7 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-16T00:23:52.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:51 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:23:52.044 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:51 smithi112 ceph-mon[27869]: pgmap v189: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 5.7 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-16T00:23:52.044 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:51 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:23:52.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:52.772+0000 7fb65d108640 1 -- 172.21.15.112:0/1694272474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb658102ed0 msgr2=0x7fb6580fc590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:52.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:52.772+0000 7fb65d108640 1 --2- 172.21.15.112:0/1694272474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb658102ed0 0x7fb6580fc590 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb64c0048d0 tx=0x7fb64c009da0 comp rx=0 tx=0).stop 2024-09-16T00:23:52.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:52.772+0000 7fb65d108640 1 -- 172.21.15.112:0/1694272474 shutdown_connections 2024-09-16T00:23:52.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:52.772+0000 7fb65d108640 1 --2- 172.21.15.112:0/1694272474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb658102ed0 0x7fb6580fc590 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:52.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:52.772+0000 7fb65d108640 1 --2- 172.21.15.112:0/1694272474 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb658102530 0x7fb658102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:52.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:52.772+0000 7fb65d108640 1 -- 172.21.15.112:0/1694272474 >> 172.21.15.112:0/1694272474 conn(0x7fb6580f80f0 msgr2=0x7fb6580fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:52.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:52.772+0000 7fb65d108640 1 -- 172.21.15.112:0/1694272474 shutdown_connections 2024-09-16T00:23:52.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:52.772+0000 7fb65d108640 1 -- 172.21.15.112:0/1694272474 wait complete. 2024-09-16T00:23:52.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:52.773+0000 7fb65d108640 1 Processor -- start 2024-09-16T00:23:52.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:52.773+0000 7fb65d108640 1 -- start start 2024-09-16T00:23:52.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:52.773+0000 7fb65d108640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb658102530 0x7fb6581132b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:52.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:52.773+0000 7fb65d108640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb658102ed0 0x7fb6581137f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:52.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:52.773+0000 7fb65d108640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb658113e80 con 0x7fb658102530 2024-09-16T00:23:52.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:52.773+0000 7fb65d108640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb65810d230 con 0x7fb658102ed0 2024-09-16T00:23:52.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:52.773+0000 7fb656d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb658102530 0x7fb6581132b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:52.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:52.773+0000 7fb656d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb658102530 0x7fb6581132b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35618/0 (socket says 172.21.15.112:35618) 2024-09-16T00:23:52.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:52.773+0000 7fb656d9d640 1 -- 172.21.15.112:0/1723351681 learned_addr learned my addr 172.21.15.112:0/1723351681 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:23:52.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:52.773+0000 7fb65659c640 1 --2- 172.21.15.112:0/1723351681 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb658102ed0 0x7fb6581137f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:52.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:52.773+0000 7fb656d9d640 1 -- 172.21.15.112:0/1723351681 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb658102ed0 msgr2=0x7fb6581137f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:52.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:52.773+0000 7fb656d9d640 1 --2- 172.21.15.112:0/1723351681 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb658102ed0 0x7fb6581137f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:52.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:52.773+0000 7fb656d9d640 1 -- 172.21.15.112:0/1723351681 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb64c005040 con 0x7fb658102530 2024-09-16T00:23:52.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:52.773+0000 7fb656d9d640 1 --2- 172.21.15.112:0/1723351681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb658102530 0x7fb6581132b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb64800b790 tx=0x7fb6480091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:52.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:52.774+0000 7fb63f7fe640 1 -- 172.21.15.112:0/1723351681 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb64800c700 con 0x7fb658102530 2024-09-16T00:23:52.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:52.774+0000 7fb63f7fe640 1 -- 172.21.15.112:0/1723351681 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fb64800cd00 con 0x7fb658102530 2024-09-16T00:23:52.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:52.774+0000 7fb65d108640 1 -- 172.21.15.112:0/1723351681 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb65810d510 con 0x7fb658102530 2024-09-16T00:23:52.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:52.774+0000 7fb63f7fe640 1 -- 172.21.15.112:0/1723351681 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb648015e70 con 0x7fb658102530 2024-09-16T00:23:52.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:52.774+0000 7fb65d108640 1 -- 172.21.15.112:0/1723351681 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb65810da30 con 0x7fb658102530 2024-09-16T00:23:52.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:52.775+0000 7fb65d108640 1 -- 172.21.15.112:0/1723351681 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6580fdcb0 con 0x7fb658102530 2024-09-16T00:23:52.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:52.775+0000 7fb63f7fe640 1 -- 172.21.15.112:0/1723351681 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) v1 ==== 90869+0+0 (secure 0 0 0) 0x7fb648012070 con 0x7fb658102530 2024-09-16T00:23:52.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:52.776+0000 7fb63f7fe640 1 --2- 172.21.15.112:0/1723351681 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fb64006ed60 0x7fb640071210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:52.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:52.776+0000 7fb63f7fe640 1 -- 172.21.15.112:0/1723351681 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fb64808dd50 con 0x7fb658102530 2024-09-16T00:23:52.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:52.776+0000 7fb65659c640 1 --2- 172.21.15.112:0/1723351681 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fb64006ed60 0x7fb640071210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:52.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:52.776+0000 7fb65659c640 1 --2- 172.21.15.112:0/1723351681 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fb64006ed60 0x7fb640071210 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb65810eb70 tx=0x7fb64c009da0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:52.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:52.781+0000 7fb63f7fe640 1 -- 172.21.15.112:0/1723351681 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fb64805ae20 con 0x7fb658102530 2024-09-16T00:23:52.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:52.992+0000 7fb65d108640 1 -- 172.21.15.112:0/1723351681 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb65804b7b0 con 0x7fb658102530 2024-09-16T00:23:52.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:52.993+0000 7fb63f7fe640 1 -- 172.21.15.112:0/1723351681 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+4722 (secure 0 0 0) 0x7fb64805a7c0 con 0x7fb658102530 2024-09-16T00:23:52.994 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:23:52.995 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:23:22.328199+0000","modified":"2024-09-16T00:23:41.761380+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14480,"mds_1":24259},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14480":{"gid":14480,"name":"cephfs.smithi112.hxfwkh","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.112:6829/486654885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":486654885},{"type":"v1","addr":"172.21.15.112:6829","nonce":486654885}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14492":{"gid":14492,"name":"cephfs.smithi112.ithwzj","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6827/381830565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":381830565},{"type":"v1","addr":"172.21.15.112:6827","nonce":381830565}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24259":{"gid":24259,"name":"cephfs.smithi169.gawwgw","rank":1,"incarnation":7,"state":"up:active","state_seq":4,"addr":"172.21.15.169:6825/2892476953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2892476953},{"type":"v1","addr":"172.21.15.169:6825","nonce":2892476953}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24269":{"gid":24269,"name":"cephfs.smithi169.lgeqrj","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.169:6827/861725728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":861725728},{"type":"v1","addr":"172.21.15.169:6827","nonce":861725728}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-09-16T00:23:52.995 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 19 2024-09-16T00:23:52.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:52.995+0000 7fb65d108640 1 -- 172.21.15.112:0/1723351681 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fb64006ed60 msgr2=0x7fb640071210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:52.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:52.995+0000 7fb65d108640 1 --2- 172.21.15.112:0/1723351681 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fb64006ed60 0x7fb640071210 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb65810eb70 tx=0x7fb64c009da0 comp rx=0 tx=0).stop 2024-09-16T00:23:52.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:52.995+0000 7fb65d108640 1 -- 172.21.15.112:0/1723351681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb658102530 msgr2=0x7fb6581132b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:52.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:52.995+0000 7fb65d108640 1 --2- 172.21.15.112:0/1723351681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb658102530 0x7fb6581132b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb64800b790 tx=0x7fb6480091d0 comp rx=0 tx=0).stop 2024-09-16T00:23:52.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:52.995+0000 7fb65d108640 1 -- 172.21.15.112:0/1723351681 shutdown_connections 2024-09-16T00:23:52.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:52.995+0000 7fb65d108640 1 --2- 172.21.15.112:0/1723351681 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fb64006ed60 0x7fb640071210 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:52.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:52.995+0000 7fb65d108640 1 --2- 172.21.15.112:0/1723351681 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb658102ed0 0x7fb6581137f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:52.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:52.995+0000 7fb65d108640 1 --2- 172.21.15.112:0/1723351681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb658102530 0x7fb6581132b0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:52.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:52.995+0000 7fb65d108640 1 -- 172.21.15.112:0/1723351681 >> 172.21.15.112:0/1723351681 conn(0x7fb6580f80f0 msgr2=0x7fb6580f9130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:52.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:52.995+0000 7fb65d108640 1 -- 172.21.15.112:0/1723351681 shutdown_connections 2024-09-16T00:23:52.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:52.995+0000 7fb65d108640 1 -- 172.21.15.112:0/1723351681 wait complete. 2024-09-16T00:23:53.677 DEBUG:tasks.fs:fs fscid=1,name=cephfs state = {'epoch': 19, 'max_mds': 2, 'flags': 50} 2024-09-16T00:23:53.677 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-09-16T00:23:53.705 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-09-16T00:23:53.706 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.1': {}} 2024-09-16T00:23:53.706 INFO:tasks.ceph_fuse:client.0 config is {} 2024-09-16T00:23:53.706 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-16T00:23:53.706 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-16T00:23:53.706 INFO:tasks.ceph_fuse:client.1 config is {} 2024-09-16T00:23:53.706 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-16T00:23:53.706 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-16T00:23:53.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:23:53.707 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list 2024-09-16T00:23:53.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:23:53.727 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link delete ceph-brx 2024-09-16T00:23:53.802 INFO:teuthology.orchestra.run.smithi169.stderr:Cannot find device "ceph-brx" 2024-09-16T00:23:53.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T00:23:53.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:23:53.804 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2024-09-16T00:23:53.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:23:53.820 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link delete ceph-brx 2024-09-16T00:23:53.888 INFO:teuthology.orchestra.run.smithi112.stderr:Cannot find device "ceph-brx" 2024-09-16T00:23:53.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T00:23:53.890 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-09-16T00:23:53.890 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-16T00:23:53.890 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph fs ls 2024-09-16T00:23:54.184 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:23:54.252 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:53 smithi112 ceph-mon[27869]: pgmap v190: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 6.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-16T00:23:54.253 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:53 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:23:54.253 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:53 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:23:54.253 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:53 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:23:54.253 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:53 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:23:54.253 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:53 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:23:54.253 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:53 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/1723351681' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:23:54.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:53 smithi169 ceph-mon[34653]: pgmap v190: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 6.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-16T00:23:54.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:53 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:23:54.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:53 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:23:54.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:53 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:23:54.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:53 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:23:54.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:53 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:23:54.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:53 smithi169 ceph-mon[34653]: from='client.? 172.21.15.112:0/1723351681' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:23:55.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:55.728+0000 7fc84a1b2640 1 -- 172.21.15.112:0/3245088008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8441014b0 msgr2=0x7fc844101880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:55.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:55.728+0000 7fc84a1b2640 1 --2- 172.21.15.112:0/3245088008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8441014b0 0x7fc844101880 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc8340048d0 tx=0x7fc834009da0 comp rx=0 tx=0).stop 2024-09-16T00:23:55.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:55.728+0000 7fc84a1b2640 1 -- 172.21.15.112:0/3245088008 shutdown_connections 2024-09-16T00:23:55.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:55.728+0000 7fc84a1b2640 1 --2- 172.21.15.112:0/3245088008 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8440fb470 0x7fc8440fb8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:55.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:55.728+0000 7fc84a1b2640 1 --2- 172.21.15.112:0/3245088008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8441014b0 0x7fc844101880 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:55.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:55.728+0000 7fc84a1b2640 1 -- 172.21.15.112:0/3245088008 >> 172.21.15.112:0/3245088008 conn(0x7fc8440f6ff0 msgr2=0x7fc8440f9400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:55.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:55.728+0000 7fc84a1b2640 1 -- 172.21.15.112:0/3245088008 shutdown_connections 2024-09-16T00:23:55.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:55.728+0000 7fc84a1b2640 1 -- 172.21.15.112:0/3245088008 wait complete. 2024-09-16T00:23:55.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:55.729+0000 7fc84a1b2640 1 Processor -- start 2024-09-16T00:23:55.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:55.729+0000 7fc84a1b2640 1 -- start start 2024-09-16T00:23:55.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:55.729+0000 7fc84a1b2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8440fb470 0x7fc84410e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:55.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:55.729+0000 7fc84a1b2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8441014b0 0x7fc84410e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:55.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:55.729+0000 7fc84a1b2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc84410ee20 con 0x7fc8440fb470 2024-09-16T00:23:55.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:55.729+0000 7fc84a1b2640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc84410ef90 con 0x7fc8441014b0 2024-09-16T00:23:55.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:55.729+0000 7fc8437fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8440fb470 0x7fc84410e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:55.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:55.729+0000 7fc8437fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8440fb470 0x7fc84410e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35642/0 (socket says 172.21.15.112:35642) 2024-09-16T00:23:55.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:55.729+0000 7fc8437fe640 1 -- 172.21.15.112:0/3273667072 learned_addr learned my addr 172.21.15.112:0/3273667072 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:23:55.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:55.729+0000 7fc842ffd640 1 --2- 172.21.15.112:0/3273667072 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8441014b0 0x7fc84410e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:55.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:55.729+0000 7fc8437fe640 1 -- 172.21.15.112:0/3273667072 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8441014b0 msgr2=0x7fc84410e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:55.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:55.729+0000 7fc8437fe640 1 --2- 172.21.15.112:0/3273667072 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8441014b0 0x7fc84410e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:55.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:55.729+0000 7fc8437fe640 1 -- 172.21.15.112:0/3273667072 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc834005040 con 0x7fc8440fb470 2024-09-16T00:23:55.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:55.729+0000 7fc842ffd640 1 --2- 172.21.15.112:0/3273667072 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8441014b0 0x7fc84410e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:23:55.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:55.730+0000 7fc8437fe640 1 --2- 172.21.15.112:0/3273667072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8440fb470 0x7fc84410e200 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc834003080 tx=0x7fc834009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:55.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:55.730+0000 7fc82bfff640 1 -- 172.21.15.112:0/3273667072 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc834038070 con 0x7fc8440fb470 2024-09-16T00:23:55.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:55.730+0000 7fc82bfff640 1 -- 172.21.15.112:0/3273667072 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fc8340097e0 con 0x7fc8440fb470 2024-09-16T00:23:55.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:55.730+0000 7fc84a1b2640 1 -- 172.21.15.112:0/3273667072 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc844117270 con 0x7fc8440fb470 2024-09-16T00:23:55.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:55.730+0000 7fc82bfff640 1 -- 172.21.15.112:0/3273667072 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc83403d420 con 0x7fc8440fb470 2024-09-16T00:23:55.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:55.730+0000 7fc84a1b2640 1 -- 172.21.15.112:0/3273667072 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8441176e0 con 0x7fc8440fb470 2024-09-16T00:23:55.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:55.731+0000 7fc84a1b2640 1 -- 172.21.15.112:0/3273667072 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8440fcbb0 con 0x7fc8440fb470 2024-09-16T00:23:55.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:55.731+0000 7fc82bfff640 1 -- 172.21.15.112:0/3273667072 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) v1 ==== 90869+0+0 (secure 0 0 0) 0x7fc834030020 con 0x7fc8440fb470 2024-09-16T00:23:55.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:55.732+0000 7fc82bfff640 1 --2- 172.21.15.112:0/3273667072 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fc82c06ed60 0x7fc82c071210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:55.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:55.732+0000 7fc842ffd640 1 --2- 172.21.15.112:0/3273667072 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fc82c06ed60 0x7fc82c071210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:55.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:55.732+0000 7fc82bfff640 1 -- 172.21.15.112:0/3273667072 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fc8340ae800 con 0x7fc8440fb470 2024-09-16T00:23:55.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:55.732+0000 7fc842ffd640 1 --2- 172.21.15.112:0/3273667072 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fc82c06ed60 0x7fc82c071210 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc838006fd0 tx=0x7fc838008040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:55.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:55.736+0000 7fc82bfff640 1 -- 172.21.15.112:0/3273667072 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fc83407b8d0 con 0x7fc8440fb470 2024-09-16T00:23:55.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:55.926+0000 7fc84a1b2640 1 -- 172.21.15.112:0/3273667072 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc84404b7b0 con 0x7fc8440fb470 2024-09-16T00:23:55.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:55.927+0000 7fc82bfff640 1 -- 172.21.15.112:0/3273667072 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v19) v1 ==== 53+0+83 (secure 0 0 0) 0x7fc83407b270 con 0x7fc8440fb470 2024-09-16T00:23:55.927 INFO:teuthology.orchestra.run.smithi112.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-09-16T00:23:55.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:55.929+0000 7fc84a1b2640 1 -- 172.21.15.112:0/3273667072 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fc82c06ed60 msgr2=0x7fc82c071210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:55.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:55.929+0000 7fc84a1b2640 1 --2- 172.21.15.112:0/3273667072 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fc82c06ed60 0x7fc82c071210 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc838006fd0 tx=0x7fc838008040 comp rx=0 tx=0).stop 2024-09-16T00:23:55.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:55.929+0000 7fc84a1b2640 1 -- 172.21.15.112:0/3273667072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8440fb470 msgr2=0x7fc84410e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:55.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:55.929+0000 7fc84a1b2640 1 --2- 172.21.15.112:0/3273667072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8440fb470 0x7fc84410e200 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc834003080 tx=0x7fc834009da0 comp rx=0 tx=0).stop 2024-09-16T00:23:55.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:55.929+0000 7fc84a1b2640 1 -- 172.21.15.112:0/3273667072 shutdown_connections 2024-09-16T00:23:55.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:55.929+0000 7fc84a1b2640 1 --2- 172.21.15.112:0/3273667072 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fc82c06ed60 0x7fc82c071210 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:55.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:55.929+0000 7fc84a1b2640 1 --2- 172.21.15.112:0/3273667072 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8441014b0 0x7fc84410e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:55.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:55.929+0000 7fc84a1b2640 1 --2- 172.21.15.112:0/3273667072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8440fb470 0x7fc84410e200 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:55.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:55.929+0000 7fc84a1b2640 1 -- 172.21.15.112:0/3273667072 >> 172.21.15.112:0/3273667072 conn(0x7fc8440f6ff0 msgr2=0x7fc8440f73c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:55.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:55.929+0000 7fc84a1b2640 1 -- 172.21.15.112:0/3273667072 shutdown_connections 2024-09-16T00:23:55.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:55.929+0000 7fc84a1b2640 1 -- 172.21.15.112:0/3273667072 wait complete. 2024-09-16T00:23:56.215 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:55 smithi112 ceph-mon[27869]: pgmap v191: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 4.2 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-16T00:23:56.215 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:55 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/3273667072' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-16T00:23:56.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:55 smithi169 ceph-mon[34653]: pgmap v191: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 4.2 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-16T00:23:56.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:55 smithi169 ceph-mon[34653]: from='client.? 172.21.15.112:0/3273667072' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-16T00:23:56.553 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-16T00:23:56.554 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T00:23:56.554 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi112.front.sepia.ceph.com 2024-09-16T00:23:56.554 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-16T00:23:56.554 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-16T00:23:56.554 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-16T00:23:56.554 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-16T00:23:56.555 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T00:23:56.555 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T00:23:56.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:23:56.555 DEBUG:teuthology.orchestra.run.smithi112:> ip addr 2024-09-16T00:23:56.573 INFO:teuthology.orchestra.run.smithi112.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T00:23:56.573 INFO:teuthology.orchestra.run.smithi112.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T00:23:56.573 INFO:teuthology.orchestra.run.smithi112.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T00:23:56.573 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-16T00:23:56.573 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 ::1/128 scope host 2024-09-16T00:23:56.573 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-16T00:23:56.573 INFO:teuthology.orchestra.run.smithi112.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T00:23:56.573 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2c brd ff:ff:ff:ff:ff:ff 2024-09-16T00:23:56.574 INFO:teuthology.orchestra.run.smithi112.stdout: altname enp7s0f0 2024-09-16T00:23:56.574 INFO:teuthology.orchestra.run.smithi112.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T00:23:56.574 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2d brd ff:ff:ff:ff:ff:ff 2024-09-16T00:23:56.574 INFO:teuthology.orchestra.run.smithi112.stdout: altname enp7s0f1 2024-09-16T00:23:56.574 INFO:teuthology.orchestra.run.smithi112.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T00:23:56.574 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:68 brd ff:ff:ff:ff:ff:ff 2024-09-16T00:23:56.574 INFO:teuthology.orchestra.run.smithi112.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T00:23:56.574 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:69 brd ff:ff:ff:ff:ff:ff 2024-09-16T00:23:56.574 INFO:teuthology.orchestra.run.smithi112.stdout: inet 172.21.15.112/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-16T00:23:56.574 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft 42342sec preferred_lft 42342sec 2024-09-16T00:23:56.574 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::ec4:7aff:fe8f:cb69/64 scope link noprefixroute 2024-09-16T00:23:56.574 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-16T00:23:56.575 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-16T00:23:56.575 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T00:23:56.575 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-09-16T00:23:56.575 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link add name ceph-brx type bridge 2024-09-16T00:23:56.575 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip addr flush dev ceph-brx 2024-09-16T00:23:56.575 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set ceph-brx up 2024-09-16T00:23:56.575 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-16T00:23:56.575 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-09-16T00:23:56.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T00:23:56.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:23:56.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:23:56.786 DEBUG:teuthology.orchestra.run.smithi112:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T00:23:56.811 INFO:teuthology.orchestra.run.smithi112.stdout:1 2024-09-16T00:23:56.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:23:56.814 DEBUG:teuthology.orchestra.run.smithi112:> route 2024-09-16T00:23:56.882 INFO:teuthology.orchestra.run.smithi112.stdout:Kernel IP routing table 2024-09-16T00:23:56.882 INFO:teuthology.orchestra.run.smithi112.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T00:23:56.882 INFO:teuthology.orchestra.run.smithi112.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-16T00:23:56.882 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-16T00:23:56.882 INFO:teuthology.orchestra.run.smithi112.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T00:23:56.882 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T00:23:56.882 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-09-16T00:23:56.883 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-16T00:23:56.883 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-16T00:23:56.883 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-16T00:23:56.883 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-09-16T00:23:56.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T00:23:57.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:23:57.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:23:57.203 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2024-09-16T00:23:57.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:23:57.219 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list-id 2024-09-16T00:23:57.275 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T00:23:57.275 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-09-16T00:23:57.275 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T00:23:57.275 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-16T00:23:57.275 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-09-16T00:23:57.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T00:23:57.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:23:57.392 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-16T00:23:57.392 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T00:23:57.392 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-09-16T00:23:57.393 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-16T00:23:57.393 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-16T00:23:57.393 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-16T00:23:57.393 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-16T00:23:57.393 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-16T00:23:57.393 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-09-16T00:23:57.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T00:23:57.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:23:57.523 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T00:23:57.523 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-09-16T00:23:57.523 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set brx.0 up 2024-09-16T00:23:57.523 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set dev brx.0 master ceph-brx 2024-09-16T00:23:57.523 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-09-16T00:23:57.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T00:23:57.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:23:57.599 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-16T00:23:57.599 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T00:23:57.600 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-16T00:23:57.614 INFO:teuthology.orchestra.run.smithi112.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-16T00:23:57.615 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T00:23:57.615 DEBUG:teuthology.orchestra.run.smithi112:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-16T00:23:57.671 DEBUG:teuthology.orchestra.run.smithi112:> sudo modprobe fuse 2024-09-16T00:23:57.737 DEBUG:teuthology.orchestra.run.smithi112:> cat /proc/self/mounts | awk '{print $2}' 2024-09-16T00:23:57.794 INFO:teuthology.orchestra.run.smithi112.stdout:/proc 2024-09-16T00:23:57.795 INFO:teuthology.orchestra.run.smithi112.stdout:/sys 2024-09-16T00:23:57.795 INFO:teuthology.orchestra.run.smithi112.stdout:/dev 2024-09-16T00:23:57.795 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/security 2024-09-16T00:23:57.795 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/shm 2024-09-16T00:23:57.795 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/pts 2024-09-16T00:23:57.795 INFO:teuthology.orchestra.run.smithi112.stdout:/run 2024-09-16T00:23:57.795 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/cgroup 2024-09-16T00:23:57.795 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/pstore 2024-09-16T00:23:57.795 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/bpf 2024-09-16T00:23:57.795 INFO:teuthology.orchestra.run.smithi112.stdout:/ 2024-09-16T00:23:57.795 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/selinux 2024-09-16T00:23:57.795 INFO:teuthology.orchestra.run.smithi112.stdout:/proc/sys/fs/binfmt_misc 2024-09-16T00:23:57.796 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/mqueue 2024-09-16T00:23:57.796 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/hugepages 2024-09-16T00:23:57.796 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/debug 2024-09-16T00:23:57.796 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/tracing 2024-09-16T00:23:57.796 INFO:teuthology.orchestra.run.smithi112.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-16T00:23:57.796 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/fuse/connections 2024-09-16T00:23:57.796 INFO:teuthology.orchestra.run.smithi112.stdout:/run/credentials/systemd-sysctl.service 2024-09-16T00:23:57.796 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/config 2024-09-16T00:23:57.796 INFO:teuthology.orchestra.run.smithi112.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-16T00:23:57.796 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/nfs/rpc_pipefs 2024-09-16T00:23:57.796 INFO:teuthology.orchestra.run.smithi112.stdout:/run/user/1000 2024-09-16T00:23:57.796 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph 2024-09-16T00:23:57.796 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay 2024-09-16T00:23:57.797 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/13182da69f059afd7c1fdf22edff40790c977c73cf31f749a37ea151efb5b40e/merged 2024-09-16T00:23:57.797 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/48aa935ad1cfb0aa3b4dd762b2799dbd0db1f7758492eb224a11935e0eed7d84/merged 2024-09-16T00:23:57.797 INFO:teuthology.orchestra.run.smithi112.stdout:/run/user/0 2024-09-16T00:23:57.797 INFO:teuthology.orchestra.run.smithi112.stdout:/proc/sys/fs/binfmt_misc 2024-09-16T00:23:57.797 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/0f8452add2a54376b13d338840c7126b87c4afe3f93189a3b5fc2a975dbae794/merged 2024-09-16T00:23:57.797 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/3a5016ca802dd1e70af935475980a2db093e1451147393133956d3db459bbce8/merged 2024-09-16T00:23:57.797 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/bdc13818ff1350c7d1d2bb6d9d9a3e0683a2e4f08769c1beaf3a5573eeae6a2a/merged 2024-09-16T00:23:57.797 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/f59bfb4372926f7218862248695b83eb47c541da4b7e9f0f078da62d7e4b9207/merged 2024-09-16T00:23:57.797 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/7688436f0a03072b231bfe3ffa065d471eb56d06014d7f51dccf5ec598d0d1e9/merged 2024-09-16T00:23:57.797 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/67efaedfa864c78efe96c74cb34afd2274f8c9445dfb7f9e1a9131e0e55d7706/merged 2024-09-16T00:23:57.797 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/f863c478fb85e3234eefd9846ad8c1b2fa3f752dc6598371b10b05f159f7feb8/merged 2024-09-16T00:23:57.797 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/4b31954a1c8d22dae09f24ceb09c485ff0e173d6dd2ff2292692120939f71693/merged 2024-09-16T00:23:57.797 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/e0d0148aecfad458eba7c1a749e6454da20649edcaa2425994709ae520b4bcc8/merged 2024-09-16T00:23:57.797 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/1788e9328ae874df7f5457ee96b9bd4858c6d4344f645024f27883808c2ff6ff/merged 2024-09-16T00:23:57.797 INFO:teuthology.orchestra.run.smithi112.stdout:/run/netns 2024-09-16T00:23:57.798 INFO:teuthology.orchestra.run.smithi112.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T00:23:57.798 INFO:teuthology.orchestra.run.smithi112.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T00:23:57.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:23:57.798 DEBUG:teuthology.orchestra.run.smithi112:> ls /sys/fs/fuse/connections 2024-09-16T00:23:57.850 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-16T00:23:57.851 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-16T00:23:57.893 DEBUG:teuthology.orchestra.run.smithi112:> sudo modprobe fuse 2024-09-16T00:23:57.918 DEBUG:teuthology.orchestra.run.smithi112:> cat /proc/self/mounts | awk '{print $2}' 2024-09-16T00:23:57.975 INFO:teuthology.orchestra.run.smithi112.stdout:/proc 2024-09-16T00:23:57.975 INFO:teuthology.orchestra.run.smithi112.stdout:/sys 2024-09-16T00:23:57.975 INFO:teuthology.orchestra.run.smithi112.stdout:/dev 2024-09-16T00:23:57.975 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/security 2024-09-16T00:23:57.975 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/shm 2024-09-16T00:23:57.976 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/pts 2024-09-16T00:23:57.976 INFO:teuthology.orchestra.run.smithi112.stdout:/run 2024-09-16T00:23:57.976 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/cgroup 2024-09-16T00:23:57.976 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/pstore 2024-09-16T00:23:57.976 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/bpf 2024-09-16T00:23:57.976 INFO:teuthology.orchestra.run.smithi112.stdout:/ 2024-09-16T00:23:57.977 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/selinux 2024-09-16T00:23:57.977 INFO:teuthology.orchestra.run.smithi112.stdout:/proc/sys/fs/binfmt_misc 2024-09-16T00:23:57.977 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/mqueue 2024-09-16T00:23:57.978 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/hugepages 2024-09-16T00:23:57.978 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/debug 2024-09-16T00:23:57.978 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/tracing 2024-09-16T00:23:57.978 INFO:teuthology.orchestra.run.smithi112.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-16T00:23:57.979 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/fuse/connections 2024-09-16T00:23:57.979 INFO:teuthology.orchestra.run.smithi112.stdout:/run/credentials/systemd-sysctl.service 2024-09-16T00:23:57.979 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/config 2024-09-16T00:23:57.980 INFO:teuthology.orchestra.run.smithi112.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-16T00:23:57.980 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/nfs/rpc_pipefs 2024-09-16T00:23:57.980 INFO:teuthology.orchestra.run.smithi112.stdout:/run/user/1000 2024-09-16T00:23:57.980 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph 2024-09-16T00:23:57.980 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay 2024-09-16T00:23:57.982 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/13182da69f059afd7c1fdf22edff40790c977c73cf31f749a37ea151efb5b40e/merged 2024-09-16T00:23:57.982 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/48aa935ad1cfb0aa3b4dd762b2799dbd0db1f7758492eb224a11935e0eed7d84/merged 2024-09-16T00:23:57.983 INFO:teuthology.orchestra.run.smithi112.stdout:/run/user/0 2024-09-16T00:23:57.983 INFO:teuthology.orchestra.run.smithi112.stdout:/proc/sys/fs/binfmt_misc 2024-09-16T00:23:57.983 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/0f8452add2a54376b13d338840c7126b87c4afe3f93189a3b5fc2a975dbae794/merged 2024-09-16T00:23:57.983 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/3a5016ca802dd1e70af935475980a2db093e1451147393133956d3db459bbce8/merged 2024-09-16T00:23:57.983 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/bdc13818ff1350c7d1d2bb6d9d9a3e0683a2e4f08769c1beaf3a5573eeae6a2a/merged 2024-09-16T00:23:57.983 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/f59bfb4372926f7218862248695b83eb47c541da4b7e9f0f078da62d7e4b9207/merged 2024-09-16T00:23:57.984 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/7688436f0a03072b231bfe3ffa065d471eb56d06014d7f51dccf5ec598d0d1e9/merged 2024-09-16T00:23:57.984 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/67efaedfa864c78efe96c74cb34afd2274f8c9445dfb7f9e1a9131e0e55d7706/merged 2024-09-16T00:23:57.984 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/f863c478fb85e3234eefd9846ad8c1b2fa3f752dc6598371b10b05f159f7feb8/merged 2024-09-16T00:23:57.985 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/4b31954a1c8d22dae09f24ceb09c485ff0e173d6dd2ff2292692120939f71693/merged 2024-09-16T00:23:57.985 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/e0d0148aecfad458eba7c1a749e6454da20649edcaa2425994709ae520b4bcc8/merged 2024-09-16T00:23:57.985 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/1788e9328ae874df7f5457ee96b9bd4858c6d4344f645024f27883808c2ff6ff/merged 2024-09-16T00:23:57.985 INFO:teuthology.orchestra.run.smithi112.stdout:/run/netns 2024-09-16T00:23:57.985 INFO:teuthology.orchestra.run.smithi112.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T00:23:57.985 INFO:teuthology.orchestra.run.smithi112.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T00:23:57.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:23:57.986 DEBUG:teuthology.orchestra.run.smithi112:> ls /sys/fs/fuse/connections 2024-09-16T00:23:58.021 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi112.stderr:2024-09-16T00:23:58.021+0000 7fc4734c1300 -1 init, newargv = 0x562d8ec5c740 newargc=15 2024-09-16T00:23:58.021 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi112.stderr:ceph-fuse[74690]: starting ceph client 2024-09-16T00:23:58.029 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi112.stderr:ceph-fuse[74690]: starting fuse 2024-09-16T00:23:58.031 INFO:teuthology.orchestra.run.smithi112.stdout:122 2024-09-16T00:23:58.032 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [122] 2024-09-16T00:23:58.032 DEBUG:teuthology.orchestra.run.smithi112:> sudo stdin-killer -- python3 -c ' 2024-09-16T00:23:58.032 DEBUG:teuthology.orchestra.run.smithi112:> import glob 2024-09-16T00:23:58.032 DEBUG:teuthology.orchestra.run.smithi112:> import re 2024-09-16T00:23:58.032 DEBUG:teuthology.orchestra.run.smithi112:> import os 2024-09-16T00:23:58.032 DEBUG:teuthology.orchestra.run.smithi112:> import subprocess 2024-09-16T00:23:58.032 DEBUG:teuthology.orchestra.run.smithi112:> 2024-09-16T00:23:58.032 DEBUG:teuthology.orchestra.run.smithi112:> def _find_admin_socket(client_name): 2024-09-16T00:23:58.032 DEBUG:teuthology.orchestra.run.smithi112:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-16T00:23:58.032 DEBUG:teuthology.orchestra.run.smithi112:> files = glob.glob(asok_path) 2024-09-16T00:23:58.032 DEBUG:teuthology.orchestra.run.smithi112:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-16T00:23:58.032 DEBUG:teuthology.orchestra.run.smithi112:> 2024-09-16T00:23:58.032 DEBUG:teuthology.orchestra.run.smithi112:> # Given a non-glob path, it better be there 2024-09-16T00:23:58.033 DEBUG:teuthology.orchestra.run.smithi112:> if "*" not in asok_path: 2024-09-16T00:23:58.033 DEBUG:teuthology.orchestra.run.smithi112:> assert(len(files) == 1) 2024-09-16T00:23:58.033 DEBUG:teuthology.orchestra.run.smithi112:> return files[0] 2024-09-16T00:23:58.033 DEBUG:teuthology.orchestra.run.smithi112:> 2024-09-16T00:23:58.033 DEBUG:teuthology.orchestra.run.smithi112:> for f in files: 2024-09-16T00:23:58.033 DEBUG:teuthology.orchestra.run.smithi112:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-16T00:23:58.033 DEBUG:teuthology.orchestra.run.smithi112:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-16T00:23:58.033 DEBUG:teuthology.orchestra.run.smithi112:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-16T00:23:58.033 DEBUG:teuthology.orchestra.run.smithi112:> contents = proc_f.read() 2024-09-16T00:23:58.033 DEBUG:teuthology.orchestra.run.smithi112:> if mountpoint in contents: 2024-09-16T00:23:58.033 DEBUG:teuthology.orchestra.run.smithi112:> return f 2024-09-16T00:23:58.033 DEBUG:teuthology.orchestra.run.smithi112:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-16T00:23:58.033 DEBUG:teuthology.orchestra.run.smithi112:> 2024-09-16T00:23:58.033 DEBUG:teuthology.orchestra.run.smithi112:> print(_find_admin_socket("client.0")) 2024-09-16T00:23:58.033 DEBUG:teuthology.orchestra.run.smithi112:> ' 2024-09-16T00:23:58.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:57 smithi112 ceph-mon[27869]: pgmap v192: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.9 KiB/s rd, 0 B/s wr, 4 op/s 2024-09-16T00:23:58.148 INFO:teuthology.orchestra.run.smithi112.stdout:/var/run/ceph/ceph-client.0.74690.asok 2024-09-16T00:23:58.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:23:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:23:58.158 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.74690.asok 2024-09-16T00:23:58.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:23:58.158 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.74690.asok status 2024-09-16T00:23:58.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:57 smithi169 ceph-mon[34653]: pgmap v192: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.9 KiB/s rd, 0 B/s wr, 4 op/s 2024-09-16T00:23:58.286 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:23:58.286 INFO:teuthology.orchestra.run.smithi112.stdout: "metadata": { 2024-09-16T00:23:58.286 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_sha1": "6d025bc26f3176c910707f8cb4c8d8e076640615", 2024-09-16T00:23:58.286 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_version": "ceph version 17.2.7-1633-g6d025bc2 (6d025bc26f3176c910707f8cb4c8d8e076640615) quincy (stable)", 2024-09-16T00:23:58.286 INFO:teuthology.orchestra.run.smithi112.stdout: "entity_id": "0", 2024-09-16T00:23:58.286 INFO:teuthology.orchestra.run.smithi112.stdout: "hostname": "smithi112", 2024-09-16T00:23:58.286 INFO:teuthology.orchestra.run.smithi112.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-16T00:23:58.286 INFO:teuthology.orchestra.run.smithi112.stdout: "pid": "74690", 2024-09-16T00:23:58.286 INFO:teuthology.orchestra.run.smithi112.stdout: "root": "/" 2024-09-16T00:23:58.286 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:23:58.287 INFO:teuthology.orchestra.run.smithi112.stdout: "dentry_count": 0, 2024-09-16T00:23:58.287 INFO:teuthology.orchestra.run.smithi112.stdout: "dentry_pinned_count": 0, 2024-09-16T00:23:58.287 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 14516, 2024-09-16T00:23:58.287 INFO:teuthology.orchestra.run.smithi112.stdout: "inst": { 2024-09-16T00:23:58.287 INFO:teuthology.orchestra.run.smithi112.stdout: "name": { 2024-09-16T00:23:58.287 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "client", 2024-09-16T00:23:58.287 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 14516 2024-09-16T00:23:58.287 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:23:58.287 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": { 2024-09-16T00:23:58.287 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-09-16T00:23:58.287 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "192.168.0.1:0", 2024-09-16T00:23:58.287 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 4053007363 2024-09-16T00:23:58.287 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-16T00:23:58.287 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:23:58.288 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": { 2024-09-16T00:23:58.288 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-09-16T00:23:58.288 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "192.168.0.1:0", 2024-09-16T00:23:58.288 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 4053007363 2024-09-16T00:23:58.288 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:23:58.288 INFO:teuthology.orchestra.run.smithi112.stdout: "inst_str": "client.14516 192.168.0.1:0/4053007363", 2024-09-16T00:23:58.288 INFO:teuthology.orchestra.run.smithi112.stdout: "addr_str": "192.168.0.1:0/4053007363", 2024-09-16T00:23:58.288 INFO:teuthology.orchestra.run.smithi112.stdout: "inode_count": 1, 2024-09-16T00:23:58.288 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_epoch": 19, 2024-09-16T00:23:58.289 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_epoch": 44, 2024-09-16T00:23:58.289 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_epoch_barrier": 0, 2024-09-16T00:23:58.289 INFO:teuthology.orchestra.run.smithi112.stdout: "blocklisted": false, 2024-09-16T00:23:58.289 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs" 2024-09-16T00:23:58.289 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:23:58.294 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-16T00:23:58.294 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph fs ls 2024-09-16T00:23:58.539 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:24:00.213 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:23:59 smithi112 ceph-mon[27869]: pgmap v193: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-16T00:24:00.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:00.213+0000 7f9553c15640 1 -- 172.21.15.112:0/2786325107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f954c0fcf80 msgr2=0x7f954c0fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:00.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:00.213+0000 7f9553c15640 1 --2- 172.21.15.112:0/2786325107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f954c0fcf80 0x7f954c0fd3d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f95400048d0 tx=0x7f9540009da0 comp rx=0 tx=0).stop 2024-09-16T00:24:00.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:00.213+0000 7f9553c15640 1 -- 172.21.15.112:0/2786325107 shutdown_connections 2024-09-16T00:24:00.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:00.213+0000 7f9553c15640 1 --2- 172.21.15.112:0/2786325107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f954c0fcf80 0x7f954c0fd3d0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:00.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:00.213+0000 7f9553c15640 1 --2- 172.21.15.112:0/2786325107 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f954c0fb810 0x7f954c0fbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:00.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:00.213+0000 7f9553c15640 1 -- 172.21.15.112:0/2786325107 >> 172.21.15.112:0/2786325107 conn(0x7f954c0723b0 msgr2=0x7f954c0727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:00.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:00.213+0000 7f9553c15640 1 -- 172.21.15.112:0/2786325107 shutdown_connections 2024-09-16T00:24:00.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:00.213+0000 7f9553c15640 1 -- 172.21.15.112:0/2786325107 wait complete. 2024-09-16T00:24:00.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:00.214+0000 7f9553c15640 1 Processor -- start 2024-09-16T00:24:00.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:00.214+0000 7f9553c15640 1 -- start start 2024-09-16T00:24:00.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:00.214+0000 7f9553c15640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f954c0fb810 0x7f954c1120b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:00.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:00.214+0000 7f9553c15640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f954c0fcf80 0x7f954c1125f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:00.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:00.214+0000 7f9553c15640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f954c112bf0 con 0x7f954c0fb810 2024-09-16T00:24:00.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:00.214+0000 7f9553c15640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f954c10c030 con 0x7f954c0fcf80 2024-09-16T00:24:00.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:00.214+0000 7f95519b1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f954c0fb810 0x7f954c1120b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:00.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:00.214+0000 7f95519b1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f954c0fb810 0x7f954c1120b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35662/0 (socket says 172.21.15.112:35662) 2024-09-16T00:24:00.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:00.214+0000 7f95519b1640 1 -- 172.21.15.112:0/4038913382 learned_addr learned my addr 172.21.15.112:0/4038913382 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:24:00.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:00.214+0000 7f95511b0640 1 --2- 172.21.15.112:0/4038913382 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f954c0fcf80 0x7f954c1125f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:00.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:00.214+0000 7f95519b1640 1 -- 172.21.15.112:0/4038913382 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f954c0fcf80 msgr2=0x7f954c1125f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:00.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:00.214+0000 7f95519b1640 1 --2- 172.21.15.112:0/4038913382 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f954c0fcf80 0x7f954c1125f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:00.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:00.214+0000 7f95519b1640 1 -- 172.21.15.112:0/4038913382 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9540005040 con 0x7f954c0fb810 2024-09-16T00:24:00.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:00.214+0000 7f95519b1640 1 --2- 172.21.15.112:0/4038913382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f954c0fb810 0x7f954c1120b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9544002a50 tx=0x7f9544004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:00.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:00.215+0000 7f954a7fc640 1 -- 172.21.15.112:0/4038913382 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9544007c30 con 0x7f954c0fb810 2024-09-16T00:24:00.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:00.215+0000 7f954a7fc640 1 -- 172.21.15.112:0/4038913382 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f954400f040 con 0x7f954c0fb810 2024-09-16T00:24:00.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:00.215+0000 7f9553c15640 1 -- 172.21.15.112:0/4038913382 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f954c10c310 con 0x7f954c0fb810 2024-09-16T00:24:00.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:00.215+0000 7f954a7fc640 1 -- 172.21.15.112:0/4038913382 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f954400dad0 con 0x7f954c0fb810 2024-09-16T00:24:00.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:00.215+0000 7f9553c15640 1 -- 172.21.15.112:0/4038913382 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f954c10c880 con 0x7f954c0fb810 2024-09-16T00:24:00.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:00.216+0000 7f9553c15640 1 -- 172.21.15.112:0/4038913382 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f954c0fcbc0 con 0x7f954c0fb810 2024-09-16T00:24:00.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:00.216+0000 7f954a7fc640 1 -- 172.21.15.112:0/4038913382 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) v1 ==== 90869+0+0 (secure 0 0 0) 0x7f954400dc30 con 0x7f954c0fb810 2024-09-16T00:24:00.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:00.217+0000 7f954a7fc640 1 --2- 172.21.15.112:0/4038913382 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f953406ed60 0x7f9534071210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:00.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:00.217+0000 7f954a7fc640 1 -- 172.21.15.112:0/4038913382 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f954408e020 con 0x7f954c0fb810 2024-09-16T00:24:00.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:00.217+0000 7f95511b0640 1 --2- 172.21.15.112:0/4038913382 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f953406ed60 0x7f9534071210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:00.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:00.217+0000 7f95511b0640 1 --2- 172.21.15.112:0/4038913382 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f953406ed60 0x7f9534071210 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9540009ac0 tx=0x7f9540035040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:00.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:00.221+0000 7f954a7fc640 1 -- 172.21.15.112:0/4038913382 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f954405b0f0 con 0x7f954c0fb810 2024-09-16T00:24:00.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:23:59 smithi169 ceph-mon[34653]: pgmap v193: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-16T00:24:00.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:00.412+0000 7f9553c15640 1 -- 172.21.15.112:0/4038913382 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f954c04b750 con 0x7f954c0fb810 2024-09-16T00:24:00.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:00.412+0000 7f954a7fc640 1 -- 172.21.15.112:0/4038913382 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v19) v1 ==== 53+0+83 (secure 0 0 0) 0x7f954405aa90 con 0x7f954c0fb810 2024-09-16T00:24:00.413 INFO:teuthology.orchestra.run.smithi112.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-09-16T00:24:00.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:00.414+0000 7f9553c15640 1 -- 172.21.15.112:0/4038913382 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f953406ed60 msgr2=0x7f9534071210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:00.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:00.414+0000 7f9553c15640 1 --2- 172.21.15.112:0/4038913382 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f953406ed60 0x7f9534071210 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9540009ac0 tx=0x7f9540035040 comp rx=0 tx=0).stop 2024-09-16T00:24:00.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:00.414+0000 7f9553c15640 1 -- 172.21.15.112:0/4038913382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f954c0fb810 msgr2=0x7f954c1120b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:00.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:00.414+0000 7f9553c15640 1 --2- 172.21.15.112:0/4038913382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f954c0fb810 0x7f954c1120b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9544002a50 tx=0x7f9544004600 comp rx=0 tx=0).stop 2024-09-16T00:24:00.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:00.414+0000 7f9553c15640 1 -- 172.21.15.112:0/4038913382 shutdown_connections 2024-09-16T00:24:00.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:00.414+0000 7f9553c15640 1 --2- 172.21.15.112:0/4038913382 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f953406ed60 0x7f9534071210 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:00.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:00.414+0000 7f9553c15640 1 --2- 172.21.15.112:0/4038913382 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f954c0fcf80 0x7f954c1125f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:00.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:00.414+0000 7f9553c15640 1 --2- 172.21.15.112:0/4038913382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f954c0fb810 0x7f954c1120b0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:00.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:00.414+0000 7f9553c15640 1 -- 172.21.15.112:0/4038913382 >> 172.21.15.112:0/4038913382 conn(0x7f954c0723b0 msgr2=0x7f954c0fa480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:00.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:00.414+0000 7f9553c15640 1 -- 172.21.15.112:0/4038913382 shutdown_connections 2024-09-16T00:24:00.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:00.414+0000 7f9553c15640 1 -- 172.21.15.112:0/4038913382 wait complete. 2024-09-16T00:24:01.050 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:24:00 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/4038913382' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-16T00:24:01.076 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-16T00:24:01.076 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T00:24:01.076 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi169.front.sepia.ceph.com 2024-09-16T00:24:01.076 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-16T00:24:01.076 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-16T00:24:01.076 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-16T00:24:01.076 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-16T00:24:01.076 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T00:24:01.076 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-16T00:24:01.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:24:01.077 DEBUG:teuthology.orchestra.run.smithi169:> ip addr 2024-09-16T00:24:01.097 INFO:teuthology.orchestra.run.smithi169.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T00:24:01.097 INFO:teuthology.orchestra.run.smithi169.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T00:24:01.097 INFO:teuthology.orchestra.run.smithi169.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T00:24:01.097 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-09-16T00:24:01.097 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 ::1/128 scope host 2024-09-16T00:24:01.097 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-09-16T00:24:01.097 INFO:teuthology.orchestra.run.smithi169.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T00:24:01.097 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:d9:3b:b4 brd ff:ff:ff:ff:ff:ff 2024-09-16T00:24:01.097 INFO:teuthology.orchestra.run.smithi169.stdout: altname enp7s0f0 2024-09-16T00:24:01.098 INFO:teuthology.orchestra.run.smithi169.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T00:24:01.098 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:d9:3b:b5 brd ff:ff:ff:ff:ff:ff 2024-09-16T00:24:01.098 INFO:teuthology.orchestra.run.smithi169.stdout: altname enp7s0f1 2024-09-16T00:24:01.098 INFO:teuthology.orchestra.run.smithi169.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T00:24:01.098 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:88:7f:4c brd ff:ff:ff:ff:ff:ff 2024-09-16T00:24:01.098 INFO:teuthology.orchestra.run.smithi169.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T00:24:01.098 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:88:7f:4d brd ff:ff:ff:ff:ff:ff 2024-09-16T00:24:01.098 INFO:teuthology.orchestra.run.smithi169.stdout: inet 172.21.15.169/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-16T00:24:01.098 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft 42346sec preferred_lft 42346sec 2024-09-16T00:24:01.098 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::ec4:7aff:fe88:7f4d/64 scope link noprefixroute 2024-09-16T00:24:01.098 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-09-16T00:24:01.099 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-16T00:24:01.099 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T00:24:01.099 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2024-09-16T00:24:01.099 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link add name ceph-brx type bridge 2024-09-16T00:24:01.099 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip addr flush dev ceph-brx 2024-09-16T00:24:01.099 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link set ceph-brx up 2024-09-16T00:24:01.099 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-16T00:24:01.099 DEBUG:teuthology.orchestra.run.smithi169:> ') 2024-09-16T00:24:01.194 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:24:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T00:24:01.259 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:24:00 smithi169 ceph-mon[34653]: from='client.? 172.21.15.112:0/4038913382' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-16T00:24:01.307 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:24:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:24:01.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:24:01.313 DEBUG:teuthology.orchestra.run.smithi169:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T00:24:01.338 INFO:teuthology.orchestra.run.smithi169.stdout:1 2024-09-16T00:24:01.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:24:01.340 DEBUG:teuthology.orchestra.run.smithi169:> route 2024-09-16T00:24:01.410 INFO:teuthology.orchestra.run.smithi169.stdout:Kernel IP routing table 2024-09-16T00:24:01.410 INFO:teuthology.orchestra.run.smithi169.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T00:24:01.410 INFO:teuthology.orchestra.run.smithi169.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-16T00:24:01.410 INFO:teuthology.orchestra.run.smithi169.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-16T00:24:01.410 INFO:teuthology.orchestra.run.smithi169.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T00:24:01.411 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T00:24:01.411 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2024-09-16T00:24:01.411 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-16T00:24:01.411 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-16T00:24:01.411 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-16T00:24:01.411 DEBUG:teuthology.orchestra.run.smithi169:> ') 2024-09-16T00:24:01.498 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:24:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T00:24:01.718 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:24:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:24:01.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:24:01.724 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list 2024-09-16T00:24:01.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:24:01.741 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list-id 2024-09-16T00:24:01.796 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T00:24:01.797 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2024-09-16T00:24:01.797 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-16T00:24:01.797 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-09-16T00:24:01.797 DEBUG:teuthology.orchestra.run.smithi169:> ') 2024-09-16T00:24:01.885 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:24:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T00:24:01.909 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:24:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:24:01.915 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-09-16T00:24:01.916 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T00:24:01.916 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2024-09-16T00:24:01.916 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-09-16T00:24:01.916 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-16T00:24:01.916 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-09-16T00:24:01.916 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-09-16T00:24:01.916 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-09-16T00:24:01.916 DEBUG:teuthology.orchestra.run.smithi169:> ') 2024-09-16T00:24:01.964 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:24:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T00:24:02.035 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:24:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:24:02.041 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T00:24:02.042 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2024-09-16T00:24:02.042 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link set brx.0 up 2024-09-16T00:24:02.042 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link set dev brx.0 master ceph-brx 2024-09-16T00:24:02.042 DEBUG:teuthology.orchestra.run.smithi169:> ') 2024-09-16T00:24:02.090 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:24:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T00:24:02.112 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:24:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:24:02.118 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-16T00:24:02.119 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T00:24:02.119 DEBUG:teuthology.orchestra.run.smithi169:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-16T00:24:02.134 INFO:teuthology.orchestra.run.smithi169.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-16T00:24:02.134 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T00:24:02.134 DEBUG:teuthology.orchestra.run.smithi169:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-16T00:24:02.190 DEBUG:teuthology.orchestra.run.smithi169:> sudo modprobe fuse 2024-09-16T00:24:02.251 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:24:01 smithi169 ceph-mon[34653]: pgmap v194: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:24:02.256 DEBUG:teuthology.orchestra.run.smithi169:> cat /proc/self/mounts | awk '{print $2}' 2024-09-16T00:24:02.313 INFO:teuthology.orchestra.run.smithi169.stdout:/proc 2024-09-16T00:24:02.313 INFO:teuthology.orchestra.run.smithi169.stdout:/sys 2024-09-16T00:24:02.313 INFO:teuthology.orchestra.run.smithi169.stdout:/dev 2024-09-16T00:24:02.313 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/kernel/security 2024-09-16T00:24:02.313 INFO:teuthology.orchestra.run.smithi169.stdout:/dev/shm 2024-09-16T00:24:02.313 INFO:teuthology.orchestra.run.smithi169.stdout:/dev/pts 2024-09-16T00:24:02.313 INFO:teuthology.orchestra.run.smithi169.stdout:/run 2024-09-16T00:24:02.314 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/fs/cgroup 2024-09-16T00:24:02.314 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/fs/pstore 2024-09-16T00:24:02.314 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/fs/bpf 2024-09-16T00:24:02.314 INFO:teuthology.orchestra.run.smithi169.stdout:/ 2024-09-16T00:24:02.314 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/fs/selinux 2024-09-16T00:24:02.314 INFO:teuthology.orchestra.run.smithi169.stdout:/proc/sys/fs/binfmt_misc 2024-09-16T00:24:02.314 INFO:teuthology.orchestra.run.smithi169.stdout:/dev/hugepages 2024-09-16T00:24:02.314 INFO:teuthology.orchestra.run.smithi169.stdout:/dev/mqueue 2024-09-16T00:24:02.314 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/kernel/debug 2024-09-16T00:24:02.314 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/kernel/tracing 2024-09-16T00:24:02.314 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/fs/fuse/connections 2024-09-16T00:24:02.314 INFO:teuthology.orchestra.run.smithi169.stdout:/run/credentials/systemd-sysctl.service 2024-09-16T00:24:02.314 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/kernel/config 2024-09-16T00:24:02.314 INFO:teuthology.orchestra.run.smithi169.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-16T00:24:02.315 INFO:teuthology.orchestra.run.smithi169.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-16T00:24:02.315 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/nfs/rpc_pipefs 2024-09-16T00:24:02.315 INFO:teuthology.orchestra.run.smithi169.stdout:/run/user/1000 2024-09-16T00:24:02.315 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/ceph 2024-09-16T00:24:02.315 INFO:teuthology.orchestra.run.smithi169.stdout:/run/user/0 2024-09-16T00:24:02.315 INFO:teuthology.orchestra.run.smithi169.stdout:/proc/sys/fs/binfmt_misc 2024-09-16T00:24:02.315 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/containers/storage/overlay 2024-09-16T00:24:02.315 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/containers/storage/overlay/d18213b74e6dc6e26a9c1c3aaf2239dd8e50a5b90cd70b65d57f2cc7e56f75eb/merged 2024-09-16T00:24:02.315 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/containers/storage/overlay/d4bfba7a1eacbc77be2a58c5ecabf7f5898d6b9a3d3f71e3df7db3a1c5e78753/merged 2024-09-16T00:24:02.315 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/containers/storage/overlay/c327a4e47aafa232ae6e0bc9ba0cdb0a6a23dff1062ef8343457d57d50949730/merged 2024-09-16T00:24:02.315 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/containers/storage/overlay/e121f4f5925c47bc27c517c917fb21f08d6b38307ca57eed18c3c83ba7ca654b/merged 2024-09-16T00:24:02.315 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/containers/storage/overlay/c6a0099adb92cd6cd6c97ec1b36f98ce2eae2799147a8ec81990158eacf29996/merged 2024-09-16T00:24:02.315 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/containers/storage/overlay/1a00bb720e30ccf22fdcbdf1722d1e811262a13697f7f6c204ca6e5a3da2cfe3/merged 2024-09-16T00:24:02.316 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/containers/storage/overlay/98b715d3e681902c6eaf021e661df75ecc93a83349fe5b3fe8ef9175910284fb/merged 2024-09-16T00:24:02.316 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/containers/storage/overlay/b61128aff91d11dab56c7efb0459342f0dfbb56f8bc970e1c07c583554519dbb/merged 2024-09-16T00:24:02.316 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/containers/storage/overlay/dae4e7abbd9b97c4474de974d31f4b0b16af71b3a71002c98606fa047051a49b/merged 2024-09-16T00:24:02.316 INFO:teuthology.orchestra.run.smithi169.stdout:/run/netns 2024-09-16T00:24:02.316 INFO:teuthology.orchestra.run.smithi169.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-16T00:24:02.316 INFO:teuthology.orchestra.run.smithi169.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-16T00:24:02.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:24:02.316 DEBUG:teuthology.orchestra.run.smithi169:> ls /sys/fs/fuse/connections 2024-09-16T00:24:02.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:24:01 smithi112 ceph-mon[27869]: pgmap v194: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:24:02.368 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-16T00:24:02.368 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-16T00:24:02.411 DEBUG:teuthology.orchestra.run.smithi169:> sudo modprobe fuse 2024-09-16T00:24:02.435 DEBUG:teuthology.orchestra.run.smithi169:> cat /proc/self/mounts | awk '{print $2}' 2024-09-16T00:24:02.491 INFO:teuthology.orchestra.run.smithi169.stdout:/proc 2024-09-16T00:24:02.491 INFO:teuthology.orchestra.run.smithi169.stdout:/sys 2024-09-16T00:24:02.492 INFO:teuthology.orchestra.run.smithi169.stdout:/dev 2024-09-16T00:24:02.492 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/kernel/security 2024-09-16T00:24:02.492 INFO:teuthology.orchestra.run.smithi169.stdout:/dev/shm 2024-09-16T00:24:02.492 INFO:teuthology.orchestra.run.smithi169.stdout:/dev/pts 2024-09-16T00:24:02.492 INFO:teuthology.orchestra.run.smithi169.stdout:/run 2024-09-16T00:24:02.492 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/fs/cgroup 2024-09-16T00:24:02.492 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/fs/pstore 2024-09-16T00:24:02.492 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/fs/bpf 2024-09-16T00:24:02.492 INFO:teuthology.orchestra.run.smithi169.stdout:/ 2024-09-16T00:24:02.492 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/fs/selinux 2024-09-16T00:24:02.492 INFO:teuthology.orchestra.run.smithi169.stdout:/proc/sys/fs/binfmt_misc 2024-09-16T00:24:02.492 INFO:teuthology.orchestra.run.smithi169.stdout:/dev/hugepages 2024-09-16T00:24:02.492 INFO:teuthology.orchestra.run.smithi169.stdout:/dev/mqueue 2024-09-16T00:24:02.492 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/kernel/debug 2024-09-16T00:24:02.493 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/kernel/tracing 2024-09-16T00:24:02.493 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/fs/fuse/connections 2024-09-16T00:24:02.493 INFO:teuthology.orchestra.run.smithi169.stdout:/run/credentials/systemd-sysctl.service 2024-09-16T00:24:02.493 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/kernel/config 2024-09-16T00:24:02.493 INFO:teuthology.orchestra.run.smithi169.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-16T00:24:02.493 INFO:teuthology.orchestra.run.smithi169.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-16T00:24:02.493 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/nfs/rpc_pipefs 2024-09-16T00:24:02.493 INFO:teuthology.orchestra.run.smithi169.stdout:/run/user/1000 2024-09-16T00:24:02.493 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/ceph 2024-09-16T00:24:02.493 INFO:teuthology.orchestra.run.smithi169.stdout:/run/user/0 2024-09-16T00:24:02.493 INFO:teuthology.orchestra.run.smithi169.stdout:/proc/sys/fs/binfmt_misc 2024-09-16T00:24:02.493 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/containers/storage/overlay 2024-09-16T00:24:02.493 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/containers/storage/overlay/d18213b74e6dc6e26a9c1c3aaf2239dd8e50a5b90cd70b65d57f2cc7e56f75eb/merged 2024-09-16T00:24:02.494 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/containers/storage/overlay/d4bfba7a1eacbc77be2a58c5ecabf7f5898d6b9a3d3f71e3df7db3a1c5e78753/merged 2024-09-16T00:24:02.494 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/containers/storage/overlay/c327a4e47aafa232ae6e0bc9ba0cdb0a6a23dff1062ef8343457d57d50949730/merged 2024-09-16T00:24:02.494 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/containers/storage/overlay/e121f4f5925c47bc27c517c917fb21f08d6b38307ca57eed18c3c83ba7ca654b/merged 2024-09-16T00:24:02.494 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/containers/storage/overlay/c6a0099adb92cd6cd6c97ec1b36f98ce2eae2799147a8ec81990158eacf29996/merged 2024-09-16T00:24:02.494 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/containers/storage/overlay/1a00bb720e30ccf22fdcbdf1722d1e811262a13697f7f6c204ca6e5a3da2cfe3/merged 2024-09-16T00:24:02.494 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/containers/storage/overlay/98b715d3e681902c6eaf021e661df75ecc93a83349fe5b3fe8ef9175910284fb/merged 2024-09-16T00:24:02.494 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/containers/storage/overlay/b61128aff91d11dab56c7efb0459342f0dfbb56f8bc970e1c07c583554519dbb/merged 2024-09-16T00:24:02.494 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/containers/storage/overlay/dae4e7abbd9b97c4474de974d31f4b0b16af71b3a71002c98606fa047051a49b/merged 2024-09-16T00:24:02.494 INFO:teuthology.orchestra.run.smithi169.stdout:/run/netns 2024-09-16T00:24:02.494 INFO:teuthology.orchestra.run.smithi169.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-16T00:24:02.494 INFO:teuthology.orchestra.run.smithi169.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-16T00:24:02.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:24:02.494 DEBUG:teuthology.orchestra.run.smithi169:> ls /sys/fs/fuse/connections 2024-09-16T00:24:02.506 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi169.stderr:2024-09-16T00:24:02.505+0000 7f956ea59300 -1 init, newargv = 0x564906e5e420 newargc=15 2024-09-16T00:24:02.506 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi169.stderr:ceph-fuse[59771]: starting ceph client 2024-09-16T00:24:02.513 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi169.stderr:ceph-fuse[59771]: starting fuse 2024-09-16T00:24:02.522 INFO:teuthology.orchestra.run.smithi169.stdout:117 2024-09-16T00:24:02.522 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [117] 2024-09-16T00:24:02.522 DEBUG:teuthology.orchestra.run.smithi169:> sudo stdin-killer -- python3 -c ' 2024-09-16T00:24:02.522 DEBUG:teuthology.orchestra.run.smithi169:> import glob 2024-09-16T00:24:02.522 DEBUG:teuthology.orchestra.run.smithi169:> import re 2024-09-16T00:24:02.523 DEBUG:teuthology.orchestra.run.smithi169:> import os 2024-09-16T00:24:02.523 DEBUG:teuthology.orchestra.run.smithi169:> import subprocess 2024-09-16T00:24:02.523 DEBUG:teuthology.orchestra.run.smithi169:> 2024-09-16T00:24:02.523 DEBUG:teuthology.orchestra.run.smithi169:> def _find_admin_socket(client_name): 2024-09-16T00:24:02.523 DEBUG:teuthology.orchestra.run.smithi169:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-16T00:24:02.523 DEBUG:teuthology.orchestra.run.smithi169:> files = glob.glob(asok_path) 2024-09-16T00:24:02.523 DEBUG:teuthology.orchestra.run.smithi169:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-16T00:24:02.523 DEBUG:teuthology.orchestra.run.smithi169:> 2024-09-16T00:24:02.523 DEBUG:teuthology.orchestra.run.smithi169:> # Given a non-glob path, it better be there 2024-09-16T00:24:02.523 DEBUG:teuthology.orchestra.run.smithi169:> if "*" not in asok_path: 2024-09-16T00:24:02.523 DEBUG:teuthology.orchestra.run.smithi169:> assert(len(files) == 1) 2024-09-16T00:24:02.524 DEBUG:teuthology.orchestra.run.smithi169:> return files[0] 2024-09-16T00:24:02.524 DEBUG:teuthology.orchestra.run.smithi169:> 2024-09-16T00:24:02.524 DEBUG:teuthology.orchestra.run.smithi169:> for f in files: 2024-09-16T00:24:02.524 DEBUG:teuthology.orchestra.run.smithi169:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-16T00:24:02.524 DEBUG:teuthology.orchestra.run.smithi169:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-16T00:24:02.524 DEBUG:teuthology.orchestra.run.smithi169:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-16T00:24:02.524 DEBUG:teuthology.orchestra.run.smithi169:> contents = proc_f.read() 2024-09-16T00:24:02.524 DEBUG:teuthology.orchestra.run.smithi169:> if mountpoint in contents: 2024-09-16T00:24:02.524 DEBUG:teuthology.orchestra.run.smithi169:> return f 2024-09-16T00:24:02.524 DEBUG:teuthology.orchestra.run.smithi169:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-16T00:24:02.524 DEBUG:teuthology.orchestra.run.smithi169:> 2024-09-16T00:24:02.524 DEBUG:teuthology.orchestra.run.smithi169:> print(_find_admin_socket("client.1")) 2024-09-16T00:24:02.524 DEBUG:teuthology.orchestra.run.smithi169:> ' 2024-09-16T00:24:02.640 INFO:teuthology.orchestra.run.smithi169.stdout:/var/run/ceph/ceph-client.1.59771.asok 2024-09-16T00:24:02.643 INFO:teuthology.orchestra.run.smithi169.stderr:2024-09-16T00:24:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:24:02.650 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.59771.asok 2024-09-16T00:24:02.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:24:02.650 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.59771.asok status 2024-09-16T00:24:02.780 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2024-09-16T00:24:02.780 INFO:teuthology.orchestra.run.smithi169.stdout: "metadata": { 2024-09-16T00:24:02.781 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_sha1": "6d025bc26f3176c910707f8cb4c8d8e076640615", 2024-09-16T00:24:02.781 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_version": "ceph version 17.2.7-1633-g6d025bc2 (6d025bc26f3176c910707f8cb4c8d8e076640615) quincy (stable)", 2024-09-16T00:24:02.781 INFO:teuthology.orchestra.run.smithi169.stdout: "entity_id": "1", 2024-09-16T00:24:02.781 INFO:teuthology.orchestra.run.smithi169.stdout: "hostname": "smithi169", 2024-09-16T00:24:02.781 INFO:teuthology.orchestra.run.smithi169.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-16T00:24:02.781 INFO:teuthology.orchestra.run.smithi169.stdout: "pid": "59771", 2024-09-16T00:24:02.781 INFO:teuthology.orchestra.run.smithi169.stdout: "root": "/" 2024-09-16T00:24:02.781 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-09-16T00:24:02.781 INFO:teuthology.orchestra.run.smithi169.stdout: "dentry_count": 0, 2024-09-16T00:24:02.781 INFO:teuthology.orchestra.run.smithi169.stdout: "dentry_pinned_count": 0, 2024-09-16T00:24:02.781 INFO:teuthology.orchestra.run.smithi169.stdout: "id": 24291, 2024-09-16T00:24:02.781 INFO:teuthology.orchestra.run.smithi169.stdout: "inst": { 2024-09-16T00:24:02.782 INFO:teuthology.orchestra.run.smithi169.stdout: "name": { 2024-09-16T00:24:02.782 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "client", 2024-09-16T00:24:02.782 INFO:teuthology.orchestra.run.smithi169.stdout: "num": 24291 2024-09-16T00:24:02.782 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-09-16T00:24:02.782 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": { 2024-09-16T00:24:02.782 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2024-09-16T00:24:02.782 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "192.168.0.1:0", 2024-09-16T00:24:02.782 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 26155201 2024-09-16T00:24:02.782 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-09-16T00:24:02.782 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-09-16T00:24:02.782 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": { 2024-09-16T00:24:02.782 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2024-09-16T00:24:02.782 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "192.168.0.1:0", 2024-09-16T00:24:02.782 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 26155201 2024-09-16T00:24:02.783 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-09-16T00:24:02.783 INFO:teuthology.orchestra.run.smithi169.stdout: "inst_str": "client.24291 192.168.0.1:0/26155201", 2024-09-16T00:24:02.783 INFO:teuthology.orchestra.run.smithi169.stdout: "addr_str": "192.168.0.1:0/26155201", 2024-09-16T00:24:02.783 INFO:teuthology.orchestra.run.smithi169.stdout: "inode_count": 1, 2024-09-16T00:24:02.783 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_epoch": 19, 2024-09-16T00:24:02.783 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_epoch": 44, 2024-09-16T00:24:02.783 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_epoch_barrier": 0, 2024-09-16T00:24:02.783 INFO:teuthology.orchestra.run.smithi169.stdout: "blocklisted": false, 2024-09-16T00:24:02.783 INFO:teuthology.orchestra.run.smithi169.stdout: "fs_name": "cephfs" 2024-09-16T00:24:02.783 INFO:teuthology.orchestra.run.smithi169.stdout:} 2024-09-16T00:24:02.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:24:02.788 DEBUG:teuthology.orchestra.run.smithi112:> stat --file-system '--printf=%T 2024-09-16T00:24:02.788 DEBUG:teuthology.orchestra.run.smithi112:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-16T00:24:02.809 INFO:teuthology.orchestra.run.smithi112.stdout:fuseblk 2024-09-16T00:24:02.810 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-16T00:24:02.810 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:24:02.810 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-16T00:24:02.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:24:02.884 DEBUG:teuthology.orchestra.run.smithi169:> stat --file-system '--printf=%T 2024-09-16T00:24:02.884 DEBUG:teuthology.orchestra.run.smithi169:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-16T00:24:02.900 INFO:teuthology.orchestra.run.smithi169.stdout:fuseblk 2024-09-16T00:24:02.900 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-16T00:24:02.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:24:02.900 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-16T00:24:03.331 INFO:teuthology.run_tasks:Running task print... 2024-09-16T00:24:03.341 INFO:teuthology.task.print:**** done client 2024-09-16T00:24:03.341 INFO:teuthology.run_tasks:Running task parallel... 2024-09-16T00:24:03.385 INFO:teuthology.task.parallel:starting parallel... 2024-09-16T00:24:03.385 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-09-16T00:24:03.386 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-09-16T00:24:03.386 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi112.front.sepia.ceph.com 2024-09-16T00:24:03.386 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -e sha1=8921c8ef947807b717db041d85f0761b7914d5fc -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-09-16T00:24:03.387 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-09-16T00:24:03.387 INFO:teuthology.task.sequential:In sequential, running task workunit... 2024-09-16T00:24:03.391 INFO:tasks.workunit:Pulling workunits from ref ad35db06e8924912f01e32b35576297c0f1352b4 2024-09-16T00:24:03.392 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-16T00:24:03.392 INFO:tasks.workunit:timeout=3h 2024-09-16T00:24:03.392 INFO:tasks.workunit:cleanup=True 2024-09-16T00:24:03.392 DEBUG:teuthology.orchestra.run.smithi112:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-16T00:24:03.413 INFO:teuthology.orchestra.run.smithi112.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-16T00:24:03.414 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-09-16T00:24:03.414 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 7ah/122d Inode: 1 Links: 2 2024-09-16T00:24:03.414 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T00:24:03.414 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:fusefs_t:s0 2024-09-16T00:24:03.414 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-09-16T00:24:03.414 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-09-16 00:23:27.359995083 +0000 2024-09-16T00:24:03.414 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-09-16 00:24:02.881418302 +0000 2024-09-16T00:24:03.414 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: - 2024-09-16T00:24:03.414 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-16T00:24:03.415 DEBUG:teuthology.orchestra.run.smithi112:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-16T00:24:03.486 DEBUG:teuthology.orchestra.run.smithi169:> stat -- /home/ubuntu/cephtest/mnt.1 2024-09-16T00:24:03.504 INFO:teuthology.orchestra.run.smithi169.stdout: File: /home/ubuntu/cephtest/mnt.1 2024-09-16T00:24:03.504 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-09-16T00:24:03.504 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 75h/117d Inode: 1 Links: 3 2024-09-16T00:24:03.504 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T00:24:03.504 INFO:teuthology.orchestra.run.smithi169.stdout:Context: system_u:object_r:fusefs_t:s0 2024-09-16T00:24:03.504 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-09-16T00:24:03.504 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2024-09-16 00:24:03.481011441 +0000 2024-09-16T00:24:03.504 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2024-09-16 00:24:03.481011441 +0000 2024-09-16T00:24:03.505 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: - 2024-09-16T00:24:03.505 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.1 2024-09-16T00:24:03.505 DEBUG:teuthology.orchestra.run.smithi169:> cd -- /home/ubuntu/cephtest/mnt.1 && sudo install -d -m 0755 --owner=ubuntu -- client.1 2024-09-16T00:24:03.575 DEBUG:teuthology.orchestra.run.smithi112:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout ad35db06e8924912f01e32b35576297c0f1352b4 2024-09-16T00:24:03.575 DEBUG:teuthology.orchestra.run.smithi169:> rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout ad35db06e8924912f01e32b35576297c0f1352b4 2024-09-16T00:24:03.592 INFO:tasks.workunit.client.0.smithi112.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-16T00:24:03.631 INFO:tasks.workunit.client.1.smithi169.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.1'... 2024-09-16T00:24:03.653 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:24:04.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:24:03 smithi169 ceph-mon[34653]: pgmap v195: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 255 B/s wr, 4 op/s 2024-09-16T00:24:04.316 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:24:03 smithi112 ceph-mon[27869]: pgmap v195: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 255 B/s wr, 4 op/s 2024-09-16T00:24:05.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:05.188+0000 7f3472d70640 1 -- 172.21.15.112:0/451813768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f346c0fd820 msgr2=0x7f346c0fdc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:05.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:05.188+0000 7f3472d70640 1 --2- 172.21.15.112:0/451813768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f346c0fd820 0x7f346c0fdc90 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f34580048d0 tx=0x7f3458009da0 comp rx=0 tx=0).stop 2024-09-16T00:24:05.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:05.188+0000 7f3472d70640 1 -- 172.21.15.112:0/451813768 shutdown_connections 2024-09-16T00:24:05.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:05.188+0000 7f3472d70640 1 --2- 172.21.15.112:0/451813768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f346c0fd820 0x7f346c0fdc90 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:05.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:05.188+0000 7f3472d70640 1 --2- 172.21.15.112:0/451813768 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f346c0fc550 0x7f346c0fc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:05.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:05.188+0000 7f3472d70640 1 -- 172.21.15.112:0/451813768 >> 172.21.15.112:0/451813768 conn(0x7f346c0f7d30 msgr2=0x7f346c0fa140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:05.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:05.188+0000 7f3472d70640 1 -- 172.21.15.112:0/451813768 shutdown_connections 2024-09-16T00:24:05.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:05.188+0000 7f3472d70640 1 -- 172.21.15.112:0/451813768 wait complete. 2024-09-16T00:24:05.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:05.188+0000 7f3472d70640 1 Processor -- start 2024-09-16T00:24:05.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:05.188+0000 7f3472d70640 1 -- start start 2024-09-16T00:24:05.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:05.189+0000 7f3472d70640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f346c0fc550 0x7f346c10d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:05.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:05.189+0000 7f3472d70640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f346c0fd820 0x7f346c10d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:05.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:05.189+0000 7f3472d70640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f346c10dc00 con 0x7f346c0fc550 2024-09-16T00:24:05.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:05.189+0000 7f3472d70640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f346c10e420 con 0x7f346c0fd820 2024-09-16T00:24:05.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:05.189+0000 7f3470b0c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f346c0fc550 0x7f346c10d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:05.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:05.189+0000 7f3470b0c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f346c0fc550 0x7f346c10d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33360/0 (socket says 172.21.15.112:33360) 2024-09-16T00:24:05.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:05.189+0000 7f3470b0c640 1 -- 172.21.15.112:0/447020326 learned_addr learned my addr 172.21.15.112:0/447020326 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:24:05.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:05.189+0000 7f346bfff640 1 --2- 172.21.15.112:0/447020326 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f346c0fd820 0x7f346c10d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:05.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:05.189+0000 7f3470b0c640 1 -- 172.21.15.112:0/447020326 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f346c0fd820 msgr2=0x7f346c10d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:05.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:05.189+0000 7f3470b0c640 1 --2- 172.21.15.112:0/447020326 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f346c0fd820 0x7f346c10d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:05.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:05.189+0000 7f3470b0c640 1 -- 172.21.15.112:0/447020326 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3458005040 con 0x7f346c0fc550 2024-09-16T00:24:05.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:05.189+0000 7f3470b0c640 1 --2- 172.21.15.112:0/447020326 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f346c0fc550 0x7f346c10d070 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f346000c830 tx=0x7f3460004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:05.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:05.189+0000 7f34697fa640 1 -- 172.21.15.112:0/447020326 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f346000a7d0 con 0x7f346c0fc550 2024-09-16T00:24:05.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:05.189+0000 7f34697fa640 1 -- 172.21.15.112:0/447020326 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f346000add0 con 0x7f346c0fc550 2024-09-16T00:24:05.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:05.189+0000 7f34697fa640 1 -- 172.21.15.112:0/447020326 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34600113a0 con 0x7f346c0fc550 2024-09-16T00:24:05.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:05.189+0000 7f3472d70640 1 -- 172.21.15.112:0/447020326 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f346c10e6b0 con 0x7f346c0fc550 2024-09-16T00:24:05.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:05.190+0000 7f3472d70640 1 -- 172.21.15.112:0/447020326 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f346c10eba0 con 0x7f346c0fc550 2024-09-16T00:24:05.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:05.191+0000 7f3472d70640 1 -- 172.21.15.112:0/447020326 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f346c0fb9b0 con 0x7f346c0fc550 2024-09-16T00:24:05.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:05.191+0000 7f34697fa640 1 -- 172.21.15.112:0/447020326 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) v1 ==== 90869+0+0 (secure 0 0 0) 0x7f3460012400 con 0x7f346c0fc550 2024-09-16T00:24:05.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:05.192+0000 7f34697fa640 1 --2- 172.21.15.112:0/447020326 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f345c06ed50 0x7f345c071200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:05.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:05.192+0000 7f34697fa640 1 -- 172.21.15.112:0/447020326 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f346008e350 con 0x7f346c0fc550 2024-09-16T00:24:05.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:05.192+0000 7f346bfff640 1 --2- 172.21.15.112:0/447020326 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f345c06ed50 0x7f345c071200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:05.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:05.192+0000 7f346bfff640 1 --2- 172.21.15.112:0/447020326 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f345c06ed50 0x7f345c071200 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3458007ac0 tx=0x7f3458035040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:05.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:05.196+0000 7f34697fa640 1 -- 172.21.15.112:0/447020326 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f346005b560 con 0x7f346c0fc550 2024-09-16T00:24:05.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:05.351+0000 7f3472d70640 1 -- 172.21.15.112:0/447020326 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7f346c04b720 con 0x7f346c0fc550 2024-09-16T00:24:05.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:05.351+0000 7f34697fa640 1 -- 172.21.15.112:0/447020326 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v21)=0 v21) v1 ==== 155+0+0 (secure 0 0 0) 0x7f346005af00 con 0x7f346c0fc550 2024-09-16T00:24:05.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:05.353+0000 7f3472d70640 1 -- 172.21.15.112:0/447020326 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f345c06ed50 msgr2=0x7f345c071200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:05.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:05.353+0000 7f3472d70640 1 --2- 172.21.15.112:0/447020326 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f345c06ed50 0x7f345c071200 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3458007ac0 tx=0x7f3458035040 comp rx=0 tx=0).stop 2024-09-16T00:24:05.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:05.353+0000 7f3472d70640 1 -- 172.21.15.112:0/447020326 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f346c0fc550 msgr2=0x7f346c10d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:05.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:05.353+0000 7f3472d70640 1 --2- 172.21.15.112:0/447020326 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f346c0fc550 0x7f346c10d070 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f346000c830 tx=0x7f3460004430 comp rx=0 tx=0).stop 2024-09-16T00:24:05.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:05.353+0000 7f3472d70640 1 -- 172.21.15.112:0/447020326 shutdown_connections 2024-09-16T00:24:05.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:05.353+0000 7f3472d70640 1 --2- 172.21.15.112:0/447020326 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f345c06ed50 0x7f345c071200 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:05.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:05.353+0000 7f3472d70640 1 --2- 172.21.15.112:0/447020326 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f346c0fd820 0x7f346c10d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:05.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:05.353+0000 7f3472d70640 1 --2- 172.21.15.112:0/447020326 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f346c0fc550 0x7f346c10d070 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:05.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:05.353+0000 7f3472d70640 1 -- 172.21.15.112:0/447020326 >> 172.21.15.112:0/447020326 conn(0x7f346c0f7d30 msgr2=0x7f346c0f94c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:05.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:05.353+0000 7f3472d70640 1 -- 172.21.15.112:0/447020326 shutdown_connections 2024-09-16T00:24:05.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:05.353+0000 7f3472d70640 1 -- 172.21.15.112:0/447020326 wait complete. 2024-09-16T00:24:05.935 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -e sha1=8921c8ef947807b717db041d85f0761b7914d5fc -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-09-16T00:24:06.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:24:05 smithi112 ceph-mon[27869]: pgmap v196: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 341 B/s wr, 4 op/s 2024-09-16T00:24:06.220 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:24:06.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:24:05 smithi169 ceph-mon[34653]: pgmap v196: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 341 B/s wr, 4 op/s 2024-09-16T00:24:07.236 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:24:07 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:24:07.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:24:07 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:24:07.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:07.736+0000 7f9faf878640 1 -- 172.21.15.112:0/2172324206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fa80715d0 msgr2=0x7f9fa806fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:07.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:07.736+0000 7f9faf878640 1 --2- 172.21.15.112:0/2172324206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fa80715d0 0x7f9fa806fb00 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9fa40048d0 tx=0x7f9fa4009da0 comp rx=0 tx=0).stop 2024-09-16T00:24:07.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:07.736+0000 7f9faf878640 1 -- 172.21.15.112:0/2172324206 shutdown_connections 2024-09-16T00:24:07.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:07.736+0000 7f9faf878640 1 --2- 172.21.15.112:0/2172324206 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9fa8070040 0x7f9fa80704b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:07.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:07.736+0000 7f9faf878640 1 --2- 172.21.15.112:0/2172324206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fa80715d0 0x7f9fa806fb00 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:07.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:07.736+0000 7f9faf878640 1 -- 172.21.15.112:0/2172324206 >> 172.21.15.112:0/2172324206 conn(0x7f9fa80f7d40 msgr2=0x7f9fa80fa150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:07.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:07.736+0000 7f9faf878640 1 -- 172.21.15.112:0/2172324206 shutdown_connections 2024-09-16T00:24:07.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:07.736+0000 7f9faf878640 1 -- 172.21.15.112:0/2172324206 wait complete. 2024-09-16T00:24:07.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:07.737+0000 7f9faf878640 1 Processor -- start 2024-09-16T00:24:07.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:07.737+0000 7f9faf878640 1 -- start start 2024-09-16T00:24:07.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:07.737+0000 7f9faf878640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9fa8070040 0x7f9fa8113700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:07.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:07.737+0000 7f9faf878640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fa80715d0 0x7f9fa8113c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:07.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:07.737+0000 7f9faf878640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa8114ab0 con 0x7f9fa80715d0 2024-09-16T00:24:07.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:07.737+0000 7f9faf878640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa8114c20 con 0x7f9fa8070040 2024-09-16T00:24:07.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:07.737+0000 7f9face13640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fa80715d0 0x7f9fa8113c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:07.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:07.737+0000 7f9face13640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fa80715d0 0x7f9fa8113c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33372/0 (socket says 172.21.15.112:33372) 2024-09-16T00:24:07.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:07.737+0000 7f9face13640 1 -- 172.21.15.112:0/2997677344 learned_addr learned my addr 172.21.15.112:0/2997677344 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:24:07.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:07.737+0000 7f9fad614640 1 --2- 172.21.15.112:0/2997677344 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9fa8070040 0x7f9fa8113700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:07.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:07.737+0000 7f9face13640 1 -- 172.21.15.112:0/2997677344 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9fa8070040 msgr2=0x7f9fa8113700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:07.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:07.737+0000 7f9face13640 1 --2- 172.21.15.112:0/2997677344 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9fa8070040 0x7f9fa8113700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:07.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:07.737+0000 7f9face13640 1 -- 172.21.15.112:0/2997677344 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fa4005040 con 0x7f9fa80715d0 2024-09-16T00:24:07.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:07.737+0000 7f9face13640 1 --2- 172.21.15.112:0/2997677344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fa80715d0 0x7f9fa8113c40 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9f9800b790 tx=0x7f9f980091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:07.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:07.737+0000 7f9f9dffb640 1 -- 172.21.15.112:0/2997677344 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f9800c700 con 0x7f9fa80715d0 2024-09-16T00:24:07.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:07.737+0000 7f9f9dffb640 1 -- 172.21.15.112:0/2997677344 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f9f9800cd00 con 0x7f9fa80715d0 2024-09-16T00:24:07.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:07.737+0000 7f9faf878640 1 -- 172.21.15.112:0/2997677344 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fa8114f00 con 0x7f9fa80715d0 2024-09-16T00:24:07.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:07.737+0000 7f9f9dffb640 1 -- 172.21.15.112:0/2997677344 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f98015e70 con 0x7f9fa80715d0 2024-09-16T00:24:07.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:07.738+0000 7f9faf878640 1 -- 172.21.15.112:0/2997677344 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fa8115420 con 0x7f9fa80715d0 2024-09-16T00:24:07.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:07.739+0000 7f9faf878640 1 -- 172.21.15.112:0/2997677344 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fa80fb9c0 con 0x7f9fa80715d0 2024-09-16T00:24:07.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:07.739+0000 7f9f9dffb640 1 -- 172.21.15.112:0/2997677344 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) v1 ==== 90869+0+0 (secure 0 0 0) 0x7f9f98012070 con 0x7f9fa80715d0 2024-09-16T00:24:07.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:07.739+0000 7f9f9dffb640 1 --2- 172.21.15.112:0/2997677344 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f9f9406ed60 0x7f9f94071210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:07.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:07.740+0000 7f9f9dffb640 1 -- 172.21.15.112:0/2997677344 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f9f9808dd50 con 0x7f9fa80715d0 2024-09-16T00:24:07.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:07.740+0000 7f9fad614640 1 --2- 172.21.15.112:0/2997677344 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f9f9406ed60 0x7f9f94071210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:07.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:07.740+0000 7f9fad614640 1 --2- 172.21.15.112:0/2997677344 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f9f9406ed60 0x7f9f94071210 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9fa4007ac0 tx=0x7f9fa4035040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:07.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:07.743+0000 7f9f9dffb640 1 -- 172.21.15.112:0/2997677344 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f9f9805ae20 con 0x7f9fa80715d0 2024-09-16T00:24:07.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:07.902+0000 7f9faf878640 1 -- 172.21.15.112:0/2997677344 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7f9fa804b720 con 0x7f9fa80715d0 2024-09-16T00:24:07.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:07.902+0000 7f9f9dffb640 1 -- 172.21.15.112:0/2997677344 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v21)=0 v21) v1 ==== 163+0+0 (secure 0 0 0) 0x7f9fa804b720 con 0x7f9fa80715d0 2024-09-16T00:24:07.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:07.904+0000 7f9faf878640 1 -- 172.21.15.112:0/2997677344 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f9f9406ed60 msgr2=0x7f9f94071210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:07.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:07.904+0000 7f9faf878640 1 --2- 172.21.15.112:0/2997677344 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f9f9406ed60 0x7f9f94071210 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9fa4007ac0 tx=0x7f9fa4035040 comp rx=0 tx=0).stop 2024-09-16T00:24:07.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:07.904+0000 7f9faf878640 1 -- 172.21.15.112:0/2997677344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fa80715d0 msgr2=0x7f9fa8113c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:07.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:07.904+0000 7f9faf878640 1 --2- 172.21.15.112:0/2997677344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fa80715d0 0x7f9fa8113c40 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9f9800b790 tx=0x7f9f980091d0 comp rx=0 tx=0).stop 2024-09-16T00:24:07.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:07.904+0000 7f9faf878640 1 -- 172.21.15.112:0/2997677344 shutdown_connections 2024-09-16T00:24:07.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:07.904+0000 7f9faf878640 1 --2- 172.21.15.112:0/2997677344 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f9f9406ed60 0x7f9f94071210 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:07.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:07.904+0000 7f9faf878640 1 --2- 172.21.15.112:0/2997677344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fa80715d0 0x7f9fa8113c40 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:07.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:07.904+0000 7f9faf878640 1 --2- 172.21.15.112:0/2997677344 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9fa8070040 0x7f9fa8113700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:07.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:07.904+0000 7f9faf878640 1 -- 172.21.15.112:0/2997677344 >> 172.21.15.112:0/2997677344 conn(0x7f9fa80f7d40 msgr2=0x7f9fa80f9500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:07.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:07.904+0000 7f9faf878640 1 -- 172.21.15.112:0/2997677344 shutdown_connections 2024-09-16T00:24:07.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:07.904+0000 7f9faf878640 1 -- 172.21.15.112:0/2997677344 wait complete. 2024-09-16T00:24:08.251 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:24:08 smithi112 ceph-mon[27869]: pgmap v197: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 341 B/s wr, 3 op/s 2024-09-16T00:24:08.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:24:08 smithi169 ceph-mon[34653]: pgmap v197: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 341 B/s wr, 3 op/s 2024-09-16T00:24:08.566 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -e sha1=8921c8ef947807b717db041d85f0761b7914d5fc -- bash -c 'ceph config set global log_to_journald false --force' 2024-09-16T00:24:08.823 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:24:10.067 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:24:10 smithi112 ceph-mon[27869]: pgmap v198: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.2 KiB/s rd, 1023 B/s wr, 6 op/s 2024-09-16T00:24:10.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:24:10 smithi169 ceph-mon[34653]: pgmap v198: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.2 KiB/s rd, 1023 B/s wr, 6 op/s 2024-09-16T00:24:10.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:10.378+0000 7f87dc244640 1 -- 172.21.15.112:0/1590811011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87d40fb450 msgr2=0x7f87d40fb860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:10.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:10.378+0000 7f87dc244640 1 --2- 172.21.15.112:0/1590811011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87d40fb450 0x7f87d40fb860 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f87d00048d0 tx=0x7f87d0009da0 comp rx=0 tx=0).stop 2024-09-16T00:24:10.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:10.378+0000 7f87dc244640 1 -- 172.21.15.112:0/1590811011 shutdown_connections 2024-09-16T00:24:10.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:10.378+0000 7f87dc244640 1 --2- 172.21.15.112:0/1590811011 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f87d40fc720 0x7f87d40fcb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:10.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:10.378+0000 7f87dc244640 1 --2- 172.21.15.112:0/1590811011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87d40fb450 0x7f87d40fb860 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:10.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:10.378+0000 7f87dc244640 1 -- 172.21.15.112:0/1590811011 >> 172.21.15.112:0/1590811011 conn(0x7f87d40f6c10 msgr2=0x7f87d40f9040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:10.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:10.379+0000 7f87dc244640 1 -- 172.21.15.112:0/1590811011 shutdown_connections 2024-09-16T00:24:10.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:10.379+0000 7f87dc244640 1 -- 172.21.15.112:0/1590811011 wait complete. 2024-09-16T00:24:10.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:10.379+0000 7f87dc244640 1 Processor -- start 2024-09-16T00:24:10.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:10.379+0000 7f87dc244640 1 -- start start 2024-09-16T00:24:10.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:10.379+0000 7f87dc244640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f87d40fb450 0x7f87d410fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:10.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:10.379+0000 7f87dc244640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87d40fc720 0x7f87d4112390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:10.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:10.379+0000 7f87dc244640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d4066840 con 0x7f87d40fc720 2024-09-16T00:24:10.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:10.379+0000 7f87dc244640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d40669b0 con 0x7f87d40fb450 2024-09-16T00:24:10.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:10.379+0000 7f87d97df640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87d40fc720 0x7f87d4112390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:10.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:10.379+0000 7f87d97df640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87d40fc720 0x7f87d4112390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33394/0 (socket says 172.21.15.112:33394) 2024-09-16T00:24:10.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:10.379+0000 7f87d97df640 1 -- 172.21.15.112:0/4213380810 learned_addr learned my addr 172.21.15.112:0/4213380810 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:24:10.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:10.380+0000 7f87d9fe0640 1 --2- 172.21.15.112:0/4213380810 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f87d40fb450 0x7f87d410fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:10.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:10.380+0000 7f87d97df640 1 -- 172.21.15.112:0/4213380810 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f87d40fb450 msgr2=0x7f87d410fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:10.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:10.380+0000 7f87d97df640 1 --2- 172.21.15.112:0/4213380810 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f87d40fb450 0x7f87d410fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:10.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:10.380+0000 7f87d97df640 1 -- 172.21.15.112:0/4213380810 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d0005040 con 0x7f87d40fc720 2024-09-16T00:24:10.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:10.380+0000 7f87d97df640 1 --2- 172.21.15.112:0/4213380810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87d40fc720 0x7f87d4112390 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f87c400b790 tx=0x7f87c40091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:10.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:10.380+0000 7f87ca7fc640 1 -- 172.21.15.112:0/4213380810 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87c400c700 con 0x7f87d40fc720 2024-09-16T00:24:10.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:10.380+0000 7f87dc244640 1 -- 172.21.15.112:0/4213380810 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87d4112990 con 0x7f87d40fc720 2024-09-16T00:24:10.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:10.380+0000 7f87ca7fc640 1 -- 172.21.15.112:0/4213380810 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f87c400cd00 con 0x7f87d40fc720 2024-09-16T00:24:10.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:10.380+0000 7f87ca7fc640 1 -- 172.21.15.112:0/4213380810 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87c4015e80 con 0x7f87d40fc720 2024-09-16T00:24:10.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:10.380+0000 7f87dc244640 1 -- 172.21.15.112:0/4213380810 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87d4112eb0 con 0x7f87d40fc720 2024-09-16T00:24:10.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:10.381+0000 7f87dc244640 1 -- 172.21.15.112:0/4213380810 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87d40fa8b0 con 0x7f87d40fc720 2024-09-16T00:24:10.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:10.382+0000 7f87ca7fc640 1 -- 172.21.15.112:0/4213380810 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) v1 ==== 90869+0+0 (secure 0 0 0) 0x7f87c4012070 con 0x7f87d40fc720 2024-09-16T00:24:10.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:10.382+0000 7f87ca7fc640 1 --2- 172.21.15.112:0/4213380810 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f87c006ed60 0x7f87c0071210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:10.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:10.382+0000 7f87ca7fc640 1 -- 172.21.15.112:0/4213380810 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f87c408db50 con 0x7f87d40fc720 2024-09-16T00:24:10.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:10.382+0000 7f87d9fe0640 1 --2- 172.21.15.112:0/4213380810 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f87c006ed60 0x7f87c0071210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:10.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:10.383+0000 7f87d9fe0640 1 --2- 172.21.15.112:0/4213380810 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f87c006ed60 0x7f87c0071210 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f87d0000c00 tx=0x7f87d0035040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:10.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:10.386+0000 7f87ca7fc640 1 -- 172.21.15.112:0/4213380810 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f87c405abc0 con 0x7f87d40fc720 2024-09-16T00:24:10.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:10.541+0000 7f87dc244640 1 -- 172.21.15.112:0/4213380810 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7f87d404b720 con 0x7f87d40fc720 2024-09-16T00:24:10.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:10.542+0000 7f87ca7fc640 1 -- 172.21.15.112:0/4213380810 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v21)=0 v21) v1 ==== 135+0+0 (secure 0 0 0) 0x7f87c405a560 con 0x7f87d40fc720 2024-09-16T00:24:10.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:10.544+0000 7f87dc244640 1 -- 172.21.15.112:0/4213380810 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f87c006ed60 msgr2=0x7f87c0071210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:10.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:10.544+0000 7f87dc244640 1 --2- 172.21.15.112:0/4213380810 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f87c006ed60 0x7f87c0071210 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f87d0000c00 tx=0x7f87d0035040 comp rx=0 tx=0).stop 2024-09-16T00:24:10.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:10.544+0000 7f87dc244640 1 -- 172.21.15.112:0/4213380810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87d40fc720 msgr2=0x7f87d4112390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:10.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:10.544+0000 7f87dc244640 1 --2- 172.21.15.112:0/4213380810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87d40fc720 0x7f87d4112390 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f87c400b790 tx=0x7f87c40091d0 comp rx=0 tx=0).stop 2024-09-16T00:24:10.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:10.544+0000 7f87dc244640 1 -- 172.21.15.112:0/4213380810 shutdown_connections 2024-09-16T00:24:10.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:10.544+0000 7f87dc244640 1 --2- 172.21.15.112:0/4213380810 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f87c006ed60 0x7f87c0071210 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:10.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:10.544+0000 7f87dc244640 1 --2- 172.21.15.112:0/4213380810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87d40fc720 0x7f87d4112390 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:10.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:10.544+0000 7f87dc244640 1 --2- 172.21.15.112:0/4213380810 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f87d40fb450 0x7f87d410fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:10.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:10.544+0000 7f87dc244640 1 -- 172.21.15.112:0/4213380810 >> 172.21.15.112:0/4213380810 conn(0x7f87d40f6c10 msgr2=0x7f87d40f83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:10.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:10.544+0000 7f87dc244640 1 -- 172.21.15.112:0/4213380810 shutdown_connections 2024-09-16T00:24:10.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:10.544+0000 7f87dc244640 1 -- 172.21.15.112:0/4213380810 wait complete. 2024-09-16T00:24:11.273 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -e sha1=8921c8ef947807b717db041d85f0761b7914d5fc -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr' 2024-09-16T00:24:11.520 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:24:12.248 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:24:12 smithi112 ceph-mon[27869]: pgmap v199: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 1023 B/s wr, 4 op/s 2024-09-16T00:24:12.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:24:12 smithi169 ceph-mon[34653]: pgmap v199: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 1023 B/s wr, 4 op/s 2024-09-16T00:24:13.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:13.091+0000 7f58e34b5640 1 -- 172.21.15.112:0/2119311283 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58dc0fc720 msgr2=0x7f58dc0fcb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:13.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:13.091+0000 7f58e34b5640 1 --2- 172.21.15.112:0/2119311283 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58dc0fc720 0x7f58dc0fcb90 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f58d00048d0 tx=0x7f58d0009da0 comp rx=0 tx=0).stop 2024-09-16T00:24:13.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:13.091+0000 7f58e34b5640 1 -- 172.21.15.112:0/2119311283 shutdown_connections 2024-09-16T00:24:13.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:13.091+0000 7f58e34b5640 1 --2- 172.21.15.112:0/2119311283 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58dc0fc720 0x7f58dc0fcb90 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:13.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:13.091+0000 7f58e34b5640 1 --2- 172.21.15.112:0/2119311283 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58dc0fb450 0x7f58dc0fb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:13.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:13.091+0000 7f58e34b5640 1 -- 172.21.15.112:0/2119311283 >> 172.21.15.112:0/2119311283 conn(0x7f58dc0f6c10 msgr2=0x7f58dc0f9040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:13.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:13.091+0000 7f58e34b5640 1 -- 172.21.15.112:0/2119311283 shutdown_connections 2024-09-16T00:24:13.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:13.091+0000 7f58e34b5640 1 -- 172.21.15.112:0/2119311283 wait complete. 2024-09-16T00:24:13.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:13.092+0000 7f58e34b5640 1 Processor -- start 2024-09-16T00:24:13.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:13.092+0000 7f58e34b5640 1 -- start start 2024-09-16T00:24:13.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:13.092+0000 7f58e34b5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58dc0fb450 0x7f58dc10c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:13.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:13.092+0000 7f58e34b5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58dc0fc720 0x7f58dc10c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:13.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:13.092+0000 7f58e34b5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58dc10d400 con 0x7f58dc0fc720 2024-09-16T00:24:13.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:13.092+0000 7f58e34b5640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58dc10d570 con 0x7f58dc0fb450 2024-09-16T00:24:13.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:13.092+0000 7f58e0a50640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58dc0fc720 0x7f58dc10c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:13.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:13.092+0000 7f58e0a50640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58dc0fc720 0x7f58dc10c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51044/0 (socket says 172.21.15.112:51044) 2024-09-16T00:24:13.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:13.092+0000 7f58e0a50640 1 -- 172.21.15.112:0/572800512 learned_addr learned my addr 172.21.15.112:0/572800512 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:24:13.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:13.092+0000 7f58e0a50640 1 -- 172.21.15.112:0/572800512 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58dc0fb450 msgr2=0x7f58dc10c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:13.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:13.092+0000 7f58e0a50640 1 --2- 172.21.15.112:0/572800512 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58dc0fb450 0x7f58dc10c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:13.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:13.092+0000 7f58e0a50640 1 -- 172.21.15.112:0/572800512 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58d0005040 con 0x7f58dc0fc720 2024-09-16T00:24:13.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:13.092+0000 7f58e0a50640 1 --2- 172.21.15.112:0/572800512 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58dc0fc720 0x7f58dc10c590 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f58d0000c00 tx=0x7f58d0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:13.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:13.092+0000 7f58cdffb640 1 -- 172.21.15.112:0/572800512 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58d0038070 con 0x7f58dc0fc720 2024-09-16T00:24:13.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:13.093+0000 7f58cdffb640 1 -- 172.21.15.112:0/572800512 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f58d002ae90 con 0x7f58dc0fc720 2024-09-16T00:24:13.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:13.093+0000 7f58e34b5640 1 -- 172.21.15.112:0/572800512 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58dc10d7f0 con 0x7f58dc0fc720 2024-09-16T00:24:13.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:13.093+0000 7f58cdffb640 1 -- 172.21.15.112:0/572800512 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58d003d420 con 0x7f58dc0fc720 2024-09-16T00:24:13.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:13.093+0000 7f58e34b5640 1 -- 172.21.15.112:0/572800512 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58dc10dcb0 con 0x7f58dc0fc720 2024-09-16T00:24:13.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:13.094+0000 7f58e34b5640 1 -- 172.21.15.112:0/572800512 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58dc0fa8b0 con 0x7f58dc0fc720 2024-09-16T00:24:13.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:13.094+0000 7f58cdffb640 1 -- 172.21.15.112:0/572800512 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) v1 ==== 90869+0+0 (secure 0 0 0) 0x7f58d0030020 con 0x7f58dc0fc720 2024-09-16T00:24:13.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:13.095+0000 7f58cdffb640 1 --2- 172.21.15.112:0/572800512 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f58c806ed60 0x7f58c8071210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:13.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:13.095+0000 7f58cdffb640 1 -- 172.21.15.112:0/572800512 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f58d00ae9b0 con 0x7f58dc0fc720 2024-09-16T00:24:13.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:13.095+0000 7f58e1251640 1 --2- 172.21.15.112:0/572800512 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f58c806ed60 0x7f58c8071210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:13.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:13.095+0000 7f58e1251640 1 --2- 172.21.15.112:0/572800512 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f58c806ed60 0x7f58c8071210 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f58d80097e0 tx=0x7f58d8006d20 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:13.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:13.099+0000 7f58cdffb640 1 -- 172.21.15.112:0/572800512 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f58d007ba80 con 0x7f58dc0fc720 2024-09-16T00:24:13.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:13.269+0000 7f58e34b5640 1 -- 172.21.15.112:0/572800512 --> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc", "daemon_types": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7f58dc10df80 con 0x7f58c806ed60 2024-09-16T00:24:14.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:24:14 smithi169 ceph-mon[34653]: pgmap v200: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.8 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-16T00:24:14.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:24:14 smithi169 ceph-mon[34653]: from='client.14538 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:24:14.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:24:14 smithi112 ceph-mon[27869]: pgmap v200: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.8 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-16T00:24:14.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:24:14 smithi112 ceph-mon[27869]: from='client.14538 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:24:16.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:24:16 smithi169 ceph-mon[34653]: pgmap v201: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 853 B/s wr, 4 op/s 2024-09-16T00:24:16.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:24:16 smithi112 ceph-mon[27869]: pgmap v201: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 853 B/s wr, 4 op/s 2024-09-16T00:24:18.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:24:18 smithi169 ceph-mon[34653]: pgmap v202: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 767 B/s wr, 4 op/s 2024-09-16T00:24:18.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:24:18 smithi112 ceph-mon[27869]: pgmap v202: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 767 B/s wr, 4 op/s 2024-09-16T00:24:20.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:24:20 smithi169 ceph-mon[34653]: pgmap v203: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-16T00:24:20.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:24:20 smithi112 ceph-mon[27869]: pgmap v203: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-16T00:24:22.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:24:22 smithi169 ceph-mon[34653]: pgmap v204: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 597 B/s wr, 3 op/s 2024-09-16T00:24:22.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:24:22 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:24:22.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:24:22 smithi112 ceph-mon[27869]: pgmap v204: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 597 B/s wr, 3 op/s 2024-09-16T00:24:22.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:24:22 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:24:24.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:24:24 smithi169 ceph-mon[34653]: pgmap v205: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-16T00:24:24.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:24:24 smithi112 ceph-mon[27869]: pgmap v205: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-16T00:24:26.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:24:26 smithi112 ceph-mon[27869]: pgmap v206: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 511 B/s wr, 4 op/s 2024-09-16T00:24:26.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:24:26 smithi169 ceph-mon[34653]: pgmap v206: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 511 B/s wr, 4 op/s 2024-09-16T00:24:28.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:24:28 smithi169 ceph-mon[34653]: pgmap v207: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 3 op/s 2024-09-16T00:24:28.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:24:28 smithi112 ceph-mon[27869]: pgmap v207: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 3 op/s 2024-09-16T00:24:30.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:24:30 smithi112 ceph-mon[27869]: pgmap v208: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.8 KiB/s rd, 511 B/s wr, 5 op/s 2024-09-16T00:24:30.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:24:30 smithi169 ceph-mon[34653]: pgmap v208: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.8 KiB/s rd, 511 B/s wr, 5 op/s 2024-09-16T00:24:32.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:24:32 smithi112 ceph-mon[27869]: pgmap v209: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:24:32.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:24:32 smithi169 ceph-mon[34653]: pgmap v209: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:24:34.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:24:34 smithi112 ceph-mon[27869]: pgmap v210: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:24:34.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:24:34 smithi169 ceph-mon[34653]: pgmap v210: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:24:36.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:24:36 smithi112 ceph-mon[27869]: pgmap v211: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-16T00:24:36.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:24:36 smithi169 ceph-mon[34653]: pgmap v211: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-16T00:24:37.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:24:37 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:24:37.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:24:37 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:24:38.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:24:38 smithi112 ceph-mon[27869]: pgmap v212: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:24:38.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:24:38 smithi169 ceph-mon[34653]: pgmap v212: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:24:40.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:24:40 smithi112 ceph-mon[27869]: pgmap v213: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:24:40.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:24:40 smithi169 ceph-mon[34653]: pgmap v213: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:24:42.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:24:42 smithi112 ceph-mon[27869]: pgmap v214: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:24:42.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:24:42 smithi169 ceph-mon[34653]: pgmap v214: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:24:44.053 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:24:44 smithi112 ceph-mon[27869]: pgmap v215: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:24:44.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:24:44 smithi169 ceph-mon[34653]: pgmap v215: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:24:44.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:44.717+0000 7f58cdffb640 1 -- 172.21.15.112:0/572800512 <== mgr.14208 v2:172.21.15.112:6800/2480414866 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7f58dc10df80 con 0x7f58c806ed60 2024-09-16T00:24:44.718 INFO:teuthology.orchestra.run.smithi112.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T00:24:44.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:44.720+0000 7f58e34b5640 1 -- 172.21.15.112:0/572800512 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f58c806ed60 msgr2=0x7f58c8071210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:44.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:44.720+0000 7f58e34b5640 1 --2- 172.21.15.112:0/572800512 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f58c806ed60 0x7f58c8071210 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f58d80097e0 tx=0x7f58d8006d20 comp rx=0 tx=0).stop 2024-09-16T00:24:44.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:44.720+0000 7f58e34b5640 1 -- 172.21.15.112:0/572800512 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58dc0fc720 msgr2=0x7f58dc10c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:44.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:44.720+0000 7f58e34b5640 1 --2- 172.21.15.112:0/572800512 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58dc0fc720 0x7f58dc10c590 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f58d0000c00 tx=0x7f58d0009da0 comp rx=0 tx=0).stop 2024-09-16T00:24:44.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:44.720+0000 7f58e34b5640 1 -- 172.21.15.112:0/572800512 shutdown_connections 2024-09-16T00:24:44.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:44.720+0000 7f58e34b5640 1 --2- 172.21.15.112:0/572800512 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f58c806ed60 0x7f58c8071210 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:44.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:44.720+0000 7f58e34b5640 1 --2- 172.21.15.112:0/572800512 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58dc0fc720 0x7f58dc10c590 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:44.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:44.720+0000 7f58e34b5640 1 --2- 172.21.15.112:0/572800512 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58dc0fb450 0x7f58dc10c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:44.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:44.720+0000 7f58e34b5640 1 -- 172.21.15.112:0/572800512 >> 172.21.15.112:0/572800512 conn(0x7f58dc0f6c10 msgr2=0x7f58dc0f83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:44.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:44.721+0000 7f58e34b5640 1 -- 172.21.15.112:0/572800512 shutdown_connections 2024-09-16T00:24:44.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:44.721+0000 7f58e34b5640 1 -- 172.21.15.112:0/572800512 wait complete. 2024-09-16T00:24:45.388 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -e sha1=8921c8ef947807b717db041d85f0761b7914d5fc -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done' 2024-09-16T00:24:45.722 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:24:45 smithi112 ceph-mon[27869]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T00:24:45.722 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:24:45 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:24:45.722 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:24:45 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:24:45.722 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:24:45 smithi112 ceph-mon[27869]: pgmap v216: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-16T00:24:45.722 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:24:45 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:45.722 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:24:45 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:24:45.722 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:24:45 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:24:45.770 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:24:46.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:24:45 smithi169 ceph-mon[34653]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T00:24:46.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:24:45 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:24:46.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:24:45 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:24:46.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:24:45 smithi169 ceph-mon[34653]: pgmap v216: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-16T00:24:46.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:24:45 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:46.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:24:45 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:24:46.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:24:45 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:24:47.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:24:46 smithi169 ceph-mon[34653]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T00:24:47.073 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:24:46 smithi112 ceph-mon[27869]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T00:24:47.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.591+0000 7fa0c0ff9640 1 -- 172.21.15.112:0/3028294345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0bc0fc520 msgr2=0x7fa0bc0fc930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:47.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.591+0000 7fa0c0ff9640 1 --2- 172.21.15.112:0/3028294345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0bc0fc520 0x7fa0bc0fc930 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac0048d0 tx=0x7fa0ac009da0 comp rx=0 tx=0).stop 2024-09-16T00:24:47.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.592+0000 7fa0c0ff9640 1 -- 172.21.15.112:0/3028294345 shutdown_connections 2024-09-16T00:24:47.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.592+0000 7fa0c0ff9640 1 --2- 172.21.15.112:0/3028294345 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0bc0fd7f0 0x7fa0bc0fdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:47.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.592+0000 7fa0c0ff9640 1 --2- 172.21.15.112:0/3028294345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0bc0fc520 0x7fa0bc0fc930 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:47.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.592+0000 7fa0c0ff9640 1 -- 172.21.15.112:0/3028294345 >> 172.21.15.112:0/3028294345 conn(0x7fa0bc0f7ce0 msgr2=0x7fa0bc0fa110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:47.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.592+0000 7fa0c0ff9640 1 -- 172.21.15.112:0/3028294345 shutdown_connections 2024-09-16T00:24:47.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.592+0000 7fa0c0ff9640 1 -- 172.21.15.112:0/3028294345 wait complete. 2024-09-16T00:24:47.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.593+0000 7fa0c0ff9640 1 Processor -- start 2024-09-16T00:24:47.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.593+0000 7fa0c0ff9640 1 -- start start 2024-09-16T00:24:47.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.593+0000 7fa0c0ff9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0bc0fc520 0x7fa0bc071570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:47.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.593+0000 7fa0c0ff9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0bc0fd7f0 0x7fa0bc06fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:47.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.593+0000 7fa0c0ff9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0bc06fff0 con 0x7fa0bc0fc520 2024-09-16T00:24:47.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.593+0000 7fa0c0ff9640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0bc070130 con 0x7fa0bc0fd7f0 2024-09-16T00:24:47.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.593+0000 7fa0ba59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0bc0fc520 0x7fa0bc071570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:47.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.593+0000 7fa0ba59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0bc0fc520 0x7fa0bc071570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47598/0 (socket says 172.21.15.112:47598) 2024-09-16T00:24:47.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.593+0000 7fa0ba59c640 1 -- 172.21.15.112:0/4221705578 learned_addr learned my addr 172.21.15.112:0/4221705578 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:24:47.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.593+0000 7fa0b9d9b640 1 --2- 172.21.15.112:0/4221705578 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0bc0fd7f0 0x7fa0bc06fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:47.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.593+0000 7fa0ba59c640 1 -- 172.21.15.112:0/4221705578 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0bc0fd7f0 msgr2=0x7fa0bc06fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:47.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.594+0000 7fa0ba59c640 1 --2- 172.21.15.112:0/4221705578 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0bc0fd7f0 0x7fa0bc06fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:47.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.594+0000 7fa0ba59c640 1 -- 172.21.15.112:0/4221705578 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0ac005040 con 0x7fa0bc0fc520 2024-09-16T00:24:47.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.594+0000 7fa0ba59c640 1 --2- 172.21.15.112:0/4221705578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0bc0fc520 0x7fa0bc071570 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac0048a0 tx=0x7fa0ac009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:47.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.594+0000 7fa0aaffd640 1 -- 172.21.15.112:0/4221705578 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa0ac038070 con 0x7fa0bc0fc520 2024-09-16T00:24:47.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.594+0000 7fa0aaffd640 1 -- 172.21.15.112:0/4221705578 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fa0ac009800 con 0x7fa0bc0fc520 2024-09-16T00:24:47.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.594+0000 7fa0aaffd640 1 -- 172.21.15.112:0/4221705578 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa0ac03e670 con 0x7fa0bc0fc520 2024-09-16T00:24:47.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.595+0000 7fa0c0ff9640 1 -- 172.21.15.112:0/4221705578 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0bc0703b0 con 0x7fa0bc0fc520 2024-09-16T00:24:47.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.595+0000 7fa0c0ff9640 1 -- 172.21.15.112:0/4221705578 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0bc115100 con 0x7fa0bc0fc520 2024-09-16T00:24:47.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.596+0000 7fa0c0ff9640 1 -- 172.21.15.112:0/4221705578 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0bc0fb980 con 0x7fa0bc0fc520 2024-09-16T00:24:47.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.596+0000 7fa0aaffd640 1 -- 172.21.15.112:0/4221705578 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) v1 ==== 90869+0+0 (secure 0 0 0) 0x7fa0ac030020 con 0x7fa0bc0fc520 2024-09-16T00:24:47.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.597+0000 7fa0aaffd640 1 --2- 172.21.15.112:0/4221705578 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fa0a406ed50 0x7fa0a4071200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:47.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.597+0000 7fa0aaffd640 1 -- 172.21.15.112:0/4221705578 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fa0ac0ae280 con 0x7fa0bc0fc520 2024-09-16T00:24:47.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.597+0000 7fa0b9d9b640 1 --2- 172.21.15.112:0/4221705578 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fa0a406ed50 0x7fa0a4071200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:47.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.598+0000 7fa0b9d9b640 1 --2- 172.21.15.112:0/4221705578 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fa0a406ed50 0x7fa0a4071200 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa0b0004ac0 tx=0x7fa0b000b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:47.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.601+0000 7fa0aaffd640 1 -- 172.21.15.112:0/4221705578 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fa0ac07b3d0 con 0x7fa0bc0fc520 2024-09-16T00:24:47.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.788+0000 7fa0c0ff9640 1 -- 172.21.15.112:0/4221705578 --> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa0bc101c50 con 0x7fa0a406ed50 2024-09-16T00:24:47.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.789+0000 7fa0aaffd640 1 -- 172.21.15.112:0/4221705578 <== mgr.14208 v2:172.21.15.112:6800/2480414866 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+351 (secure 0 0 0) 0x7fa0bc101c50 con 0x7fa0a406ed50 2024-09-16T00:24:47.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.791+0000 7fa0c0ff9640 1 -- 172.21.15.112:0/4221705578 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fa0a406ed50 msgr2=0x7fa0a4071200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:47.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.791+0000 7fa0c0ff9640 1 --2- 172.21.15.112:0/4221705578 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fa0a406ed50 0x7fa0a4071200 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa0b0004ac0 tx=0x7fa0b000b040 comp rx=0 tx=0).stop 2024-09-16T00:24:47.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.791+0000 7fa0c0ff9640 1 -- 172.21.15.112:0/4221705578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0bc0fc520 msgr2=0x7fa0bc071570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:47.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.791+0000 7fa0c0ff9640 1 --2- 172.21.15.112:0/4221705578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0bc0fc520 0x7fa0bc071570 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac0048a0 tx=0x7fa0ac009da0 comp rx=0 tx=0).stop 2024-09-16T00:24:47.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.792+0000 7fa0c0ff9640 1 -- 172.21.15.112:0/4221705578 shutdown_connections 2024-09-16T00:24:47.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.792+0000 7fa0c0ff9640 1 --2- 172.21.15.112:0/4221705578 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fa0a406ed50 0x7fa0a4071200 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:47.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.792+0000 7fa0c0ff9640 1 --2- 172.21.15.112:0/4221705578 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0bc0fd7f0 0x7fa0bc06fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:47.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.792+0000 7fa0c0ff9640 1 --2- 172.21.15.112:0/4221705578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0bc0fc520 0x7fa0bc071570 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:47.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.792+0000 7fa0c0ff9640 1 -- 172.21.15.112:0/4221705578 >> 172.21.15.112:0/4221705578 conn(0x7fa0bc0f7ce0 msgr2=0x7fa0bc0f9500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:47.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.792+0000 7fa0c0ff9640 1 -- 172.21.15.112:0/4221705578 shutdown_connections 2024-09-16T00:24:47.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.792+0000 7fa0c0ff9640 1 -- 172.21.15.112:0/4221705578 wait complete. 2024-09-16T00:24:47.802 INFO:teuthology.orchestra.run.smithi112.stdout:true 2024-09-16T00:24:47.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.876+0000 7f32d7e53640 1 -- 172.21.15.112:0/1733961638 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32d00fca00 msgr2=0x7f32d00fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:47.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.876+0000 7f32d7e53640 1 --2- 172.21.15.112:0/1733961638 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32d00fca00 0x7f32d00fce10 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f32cc0048d0 tx=0x7f32cc009da0 comp rx=0 tx=0).stop 2024-09-16T00:24:47.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.876+0000 7f32d7e53640 1 -- 172.21.15.112:0/1733961638 shutdown_connections 2024-09-16T00:24:47.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.876+0000 7f32d7e53640 1 --2- 172.21.15.112:0/1733961638 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32d00fd350 0x7f32d00fb750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:47.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.876+0000 7f32d7e53640 1 --2- 172.21.15.112:0/1733961638 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32d00fca00 0x7f32d00fce10 secure :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f32cc0048d0 tx=0x7f32cc009da0 comp rx=0 tx=0).stop 2024-09-16T00:24:47.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.876+0000 7f32d7e53640 1 -- 172.21.15.112:0/1733961638 >> 172.21.15.112:0/1733961638 conn(0x7f32d00f6390 msgr2=0x7f32d00f87a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:47.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.877+0000 7f32d7e53640 1 -- 172.21.15.112:0/1733961638 shutdown_connections 2024-09-16T00:24:47.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.877+0000 7f32d7e53640 1 -- 172.21.15.112:0/1733961638 wait complete. 2024-09-16T00:24:47.878 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:24:47 smithi112 ceph-mon[27869]: pgmap v217: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:24:47.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.877+0000 7f32d7e53640 1 Processor -- start 2024-09-16T00:24:47.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.877+0000 7f32d7e53640 1 -- start start 2024-09-16T00:24:47.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.877+0000 7f32d7e53640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32d00fd350 0x7f32d010f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:47.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.877+0000 7f32d7e53640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32d010f9e0 0x7f32d0110790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:47.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.877+0000 7f32d7e53640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d010fea0 con 0x7f32d00fd350 2024-09-16T00:24:47.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.878+0000 7f32d7e53640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d010ffe0 con 0x7f32d010f9e0 2024-09-16T00:24:47.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.878+0000 7f32d5bef640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32d00fd350 0x7f32d010f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:47.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.878+0000 7f32d5bef640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32d00fd350 0x7f32d010f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47608/0 (socket says 172.21.15.112:47608) 2024-09-16T00:24:47.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.878+0000 7f32d5bef640 1 -- 172.21.15.112:0/1237687309 learned_addr learned my addr 172.21.15.112:0/1237687309 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:24:47.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.878+0000 7f32d53ee640 1 --2- 172.21.15.112:0/1237687309 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32d010f9e0 0x7f32d0110790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:47.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.878+0000 7f32d5bef640 1 -- 172.21.15.112:0/1237687309 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32d010f9e0 msgr2=0x7f32d0110790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:47.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.878+0000 7f32d5bef640 1 --2- 172.21.15.112:0/1237687309 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32d010f9e0 0x7f32d0110790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:47.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.878+0000 7f32d5bef640 1 -- 172.21.15.112:0/1237687309 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32cc005040 con 0x7f32d00fd350 2024-09-16T00:24:47.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.878+0000 7f32d53ee640 1 --2- 172.21.15.112:0/1237687309 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32d010f9e0 0x7f32d0110790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:24:47.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.878+0000 7f32d5bef640 1 --2- 172.21.15.112:0/1237687309 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32d00fd350 0x7f32d010f4a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f32cc000c00 tx=0x7f32cc009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:47.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.879+0000 7f32c67fc640 1 -- 172.21.15.112:0/1237687309 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32cc038070 con 0x7f32d00fd350 2024-09-16T00:24:47.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.879+0000 7f32c67fc640 1 -- 172.21.15.112:0/1237687309 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f32cc004290 con 0x7f32d00fd350 2024-09-16T00:24:47.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.879+0000 7f32d7e53640 1 -- 172.21.15.112:0/1237687309 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32d0110d30 con 0x7f32d00fd350 2024-09-16T00:24:47.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.879+0000 7f32c67fc640 1 -- 172.21.15.112:0/1237687309 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32cc032610 con 0x7f32d00fd350 2024-09-16T00:24:47.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.880+0000 7f32d7e53640 1 -- 172.21.15.112:0/1237687309 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32d0111270 con 0x7f32d00fd350 2024-09-16T00:24:47.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.882+0000 7f32d7e53640 1 -- 172.21.15.112:0/1237687309 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32d0103560 con 0x7f32d00fd350 2024-09-16T00:24:47.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.882+0000 7f32c67fc640 1 -- 172.21.15.112:0/1237687309 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) v1 ==== 90869+0+0 (secure 0 0 0) 0x7f32cc030020 con 0x7f32d00fd350 2024-09-16T00:24:47.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.883+0000 7f32c67fc640 1 --2- 172.21.15.112:0/1237687309 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f32bc06ee20 0x7f32bc0712d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:47.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.883+0000 7f32d53ee640 1 --2- 172.21.15.112:0/1237687309 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f32bc06ee20 0x7f32bc0712d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:47.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.883+0000 7f32c67fc640 1 -- 172.21.15.112:0/1237687309 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f32cc0ae7e0 con 0x7f32d00fd350 2024-09-16T00:24:47.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.885+0000 7f32d53ee640 1 --2- 172.21.15.112:0/1237687309 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f32bc06ee20 0x7f32bc0712d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f32d00fd720 tx=0x7f32c0017040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:47.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:47.890+0000 7f32c67fc640 1 -- 172.21.15.112:0/1237687309 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f32cc07b900 con 0x7f32d00fd350 2024-09-16T00:24:48.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:24:47 smithi169 ceph-mon[34653]: pgmap v217: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:24:48.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.178+0000 7f32d7e53640 1 -- 172.21.15.112:0/1237687309 --> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f32d0102ec0 con 0x7f32bc06ee20 2024-09-16T00:24:48.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.180+0000 7f32c67fc640 1 -- 172.21.15.112:0/1237687309 <== mgr.14208 v2:172.21.15.112:6800/2480414866 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+351 (secure 0 0 0) 0x7f32d0102ec0 con 0x7f32bc06ee20 2024-09-16T00:24:48.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.182+0000 7f32d7e53640 1 -- 172.21.15.112:0/1237687309 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f32bc06ee20 msgr2=0x7f32bc0712d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:48.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.182+0000 7f32d7e53640 1 --2- 172.21.15.112:0/1237687309 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f32bc06ee20 0x7f32bc0712d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f32d00fd720 tx=0x7f32c0017040 comp rx=0 tx=0).stop 2024-09-16T00:24:48.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.182+0000 7f32d7e53640 1 -- 172.21.15.112:0/1237687309 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32d00fd350 msgr2=0x7f32d010f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:48.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.182+0000 7f32d7e53640 1 --2- 172.21.15.112:0/1237687309 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32d00fd350 0x7f32d010f4a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f32cc000c00 tx=0x7f32cc009da0 comp rx=0 tx=0).stop 2024-09-16T00:24:48.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.182+0000 7f32d7e53640 1 -- 172.21.15.112:0/1237687309 shutdown_connections 2024-09-16T00:24:48.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.182+0000 7f32d7e53640 1 --2- 172.21.15.112:0/1237687309 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f32bc06ee20 0x7f32bc0712d0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:48.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.182+0000 7f32d7e53640 1 --2- 172.21.15.112:0/1237687309 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32d010f9e0 0x7f32d0110790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:48.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.182+0000 7f32d7e53640 1 --2- 172.21.15.112:0/1237687309 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32d00fd350 0x7f32d010f4a0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:48.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.182+0000 7f32d7e53640 1 -- 172.21.15.112:0/1237687309 >> 172.21.15.112:0/1237687309 conn(0x7f32d00f6390 msgr2=0x7f32d00ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:48.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.182+0000 7f32d7e53640 1 -- 172.21.15.112:0/1237687309 shutdown_connections 2024-09-16T00:24:48.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.182+0000 7f32d7e53640 1 -- 172.21.15.112:0/1237687309 wait complete. 2024-09-16T00:24:48.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.265+0000 7f11b81dc640 1 -- 172.21.15.112:0/378870085 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11b00ff680 msgr2=0x7f11b0101a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:48.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.265+0000 7f11b81dc640 1 --2- 172.21.15.112:0/378870085 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11b00ff680 0x7f11b0101a60 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f11a40048d0 tx=0x7f11a4009da0 comp rx=0 tx=0).stop 2024-09-16T00:24:48.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.265+0000 7f11b81dc640 1 -- 172.21.15.112:0/378870085 shutdown_connections 2024-09-16T00:24:48.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.265+0000 7f11b81dc640 1 --2- 172.21.15.112:0/378870085 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11b00ff680 0x7f11b0101a60 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:48.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.265+0000 7f11b81dc640 1 --2- 172.21.15.112:0/378870085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11b0065c40 0x7f11b00ff140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:48.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.265+0000 7f11b81dc640 1 -- 172.21.15.112:0/378870085 >> 172.21.15.112:0/378870085 conn(0x7f11b00f6c10 msgr2=0x7f11b00f9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:48.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.265+0000 7f11b81dc640 1 -- 172.21.15.112:0/378870085 shutdown_connections 2024-09-16T00:24:48.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.265+0000 7f11b81dc640 1 -- 172.21.15.112:0/378870085 wait complete. 2024-09-16T00:24:48.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.266+0000 7f11b81dc640 1 Processor -- start 2024-09-16T00:24:48.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.266+0000 7f11b81dc640 1 -- start start 2024-09-16T00:24:48.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.266+0000 7f11b81dc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11b0065c40 0x7f11b00fcb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:48.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.266+0000 7f11b81dc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11b00ff680 0x7f11b00fb040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:48.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.266+0000 7f11b81dc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b0110d00 con 0x7f11b0065c40 2024-09-16T00:24:48.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.266+0000 7f11b81dc640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b0110e40 con 0x7f11b00ff680 2024-09-16T00:24:48.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.266+0000 7f11b5777640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11b00ff680 0x7f11b00fb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:48.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.266+0000 7f11b5777640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11b00ff680 0x7f11b00fb040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.112:55066/0 (socket says 172.21.15.112:55066) 2024-09-16T00:24:48.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.266+0000 7f11b5777640 1 -- 172.21.15.112:0/4275202926 learned_addr learned my addr 172.21.15.112:0/4275202926 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:24:48.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.267+0000 7f11b5777640 1 -- 172.21.15.112:0/4275202926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11b0065c40 msgr2=0x7f11b00fcb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:48.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.267+0000 7f11b5777640 1 --2- 172.21.15.112:0/4275202926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11b0065c40 0x7f11b00fcb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:48.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.267+0000 7f11b5777640 1 -- 172.21.15.112:0/4275202926 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11a4005040 con 0x7f11b00ff680 2024-09-16T00:24:48.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.267+0000 7f11b5777640 1 --2- 172.21.15.112:0/4275202926 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11b00ff680 0x7f11b00fb040 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f11a40043d0 tx=0x7f11a4009da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:48.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.267+0000 7f11a27fc640 1 -- 172.21.15.112:0/4275202926 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11a4038070 con 0x7f11b00ff680 2024-09-16T00:24:48.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.267+0000 7f11b81dc640 1 -- 172.21.15.112:0/4275202926 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11b00fb5e0 con 0x7f11b00ff680 2024-09-16T00:24:48.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.267+0000 7f11a27fc640 1 -- 172.21.15.112:0/4275202926 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f11a402ae90 con 0x7f11b00ff680 2024-09-16T00:24:48.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.267+0000 7f11b81dc640 1 -- 172.21.15.112:0/4275202926 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11b00fbb80 con 0x7f11b00ff680 2024-09-16T00:24:48.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.267+0000 7f11a27fc640 1 -- 172.21.15.112:0/4275202926 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11a403e670 con 0x7f11b00ff680 2024-09-16T00:24:48.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.269+0000 7f11b81dc640 1 -- 172.21.15.112:0/4275202926 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11b0105760 con 0x7f11b00ff680 2024-09-16T00:24:48.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.269+0000 7f11a27fc640 1 -- 172.21.15.112:0/4275202926 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 19) v1 ==== 90869+0+0 (secure 0 0 0) 0x7f11a4030020 con 0x7f11b00ff680 2024-09-16T00:24:48.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.270+0000 7f11a27fc640 1 --2- 172.21.15.112:0/4275202926 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f119c06ed60 0x7f119c071210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:48.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.270+0000 7f11a27fc640 1 -- 172.21.15.112:0/4275202926 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f11a40ae460 con 0x7f11b00ff680 2024-09-16T00:24:48.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.270+0000 7f11b5f78640 1 --2- 172.21.15.112:0/4275202926 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f119c06ed60 0x7f119c071210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:48.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.270+0000 7f11b5f78640 1 --2- 172.21.15.112:0/4275202926 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f119c06ed60 0x7f119c071210 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f11ac009fd0 tx=0x7f11ac009c50 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:48.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.278+0000 7f11a27fc640 1 -- 172.21.15.112:0/4275202926 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f11a407b530 con 0x7f11b00ff680 2024-09-16T00:24:48.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.491+0000 7f11b81dc640 1 -- 172.21.15.112:0/4275202926 --> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f11b0105110 con 0x7f119c06ed60 2024-09-16T00:24:48.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.500+0000 7f11a27fc640 1 -- 172.21.15.112:0/4275202926 <== mgr.14208 v2:172.21.15.112:6800/2480414866 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f11b0105110 con 0x7f119c06ed60 2024-09-16T00:24:48.501 INFO:teuthology.orchestra.run.smithi112.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T00:24:48.501 INFO:teuthology.orchestra.run.smithi112.stdout:alertmanager.smithi112 smithi112 *:9093,9094 running (5m) 55s ago 6m 24.2M - 0.25.0 c8568f914cd2 89acc17eab5c 2024-09-16T00:24:48.501 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi112 smithi112 running (6m) 55s ago 6m 6891k - 17.2.7-1633-g6d025bc2 33ed8dba3996 c94220ed2776 2024-09-16T00:24:48.501 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi169 smithi169 running (5m) 61s ago 5m 6899k - 17.2.7-1633-g6d025bc2 33ed8dba3996 d0d3726280a2 2024-09-16T00:24:48.501 INFO:teuthology.orchestra.run.smithi112.stdout:grafana.smithi112 smithi112 *:3000 running (5m) 55s ago 6m 84.6M - 9.4.7 954c08fa6188 ffeb9c3bdf66 2024-09-16T00:24:48.501 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.hxfwkh smithi112 running (75s) 55s ago 75s 15.6M - 17.2.7-1633-g6d025bc2 33ed8dba3996 cf92d9461fc6 2024-09-16T00:24:48.502 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.ithwzj smithi112 running (82s) 55s ago 82s 11.8M - 17.2.7-1633-g6d025bc2 33ed8dba3996 02786c8a83c7 2024-09-16T00:24:48.502 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.gawwgw smithi169 running (80s) 61s ago 79s 14.4M - 17.2.7-1633-g6d025bc2 33ed8dba3996 dcf3a4dfa879 2024-09-16T00:24:48.502 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.lgeqrj smithi169 running (72s) 61s ago 72s 13.8M - 17.2.7-1633-g6d025bc2 33ed8dba3996 445c14e55831 2024-09-16T00:24:48.502 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi112.xrkkbc smithi112 *:9283,8765,8443 running (7m) 55s ago 7m 449M - 17.2.7-1633-g6d025bc2 33ed8dba3996 b3d3adb94a85 2024-09-16T00:24:48.502 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi169.mtrobk smithi169 *:8443,9283 running (5m) 61s ago 5m 405M - 17.2.7-1633-g6d025bc2 33ed8dba3996 1e67dbe21770 2024-09-16T00:24:48.502 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi112 smithi112 running (7m) 55s ago 8m 43.3M 2048M 17.2.7-1633-g6d025bc2 33ed8dba3996 ddaaced54bf8 2024-09-16T00:24:48.502 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi169 smithi169 running (5m) 61s ago 5m 35.4M 2048M 17.2.7-1633-g6d025bc2 33ed8dba3996 6c2289bc3e11 2024-09-16T00:24:48.502 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi112 smithi112 *:9100 running (6m) 55s ago 6m 15.9M - 1.5.0 0da6a335fe13 89c6f32288bc 2024-09-16T00:24:48.502 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi169 smithi169 *:9100 running (5m) 61s ago 5m 15.6M - 1.5.0 0da6a335fe13 5d6fe1dc6fa9 2024-09-16T00:24:48.502 INFO:teuthology.orchestra.run.smithi112.stdout:osd.0 smithi112 running (5m) 55s ago 5m 46.0M 4288M 17.2.7-1633-g6d025bc2 33ed8dba3996 f6b2e146ffdf 2024-09-16T00:24:48.502 INFO:teuthology.orchestra.run.smithi112.stdout:osd.1 smithi112 running (4m) 55s ago 4m 47.9M 4288M 17.2.7-1633-g6d025bc2 33ed8dba3996 e9bee87bba0f 2024-09-16T00:24:48.502 INFO:teuthology.orchestra.run.smithi112.stdout:osd.2 smithi112 running (4m) 55s ago 4m 43.3M 4288M 17.2.7-1633-g6d025bc2 33ed8dba3996 100ec4e4852a 2024-09-16T00:24:48.502 INFO:teuthology.orchestra.run.smithi112.stdout:osd.3 smithi169 running (3m) 61s ago 3m 46.3M 5312M 17.2.7-1633-g6d025bc2 33ed8dba3996 f88463e19f23 2024-09-16T00:24:48.502 INFO:teuthology.orchestra.run.smithi112.stdout:osd.4 smithi169 running (3m) 61s ago 3m 42.7M 5312M 17.2.7-1633-g6d025bc2 33ed8dba3996 d3e4cdb016b9 2024-09-16T00:24:48.502 INFO:teuthology.orchestra.run.smithi112.stdout:osd.5 smithi169 running (2m) 61s ago 2m 42.2M 5312M 17.2.7-1633-g6d025bc2 33ed8dba3996 8e7d708d74eb 2024-09-16T00:24:48.503 INFO:teuthology.orchestra.run.smithi112.stdout:prometheus.smithi112 smithi112 *:9095 running (5m) 55s ago 6m 55.1M - 2.43.0 a07b618ecd1d 7f5a86d91011 2024-09-16T00:24:48.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.503+0000 7f11b81dc640 1 -- 172.21.15.112:0/4275202926 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f119c06ed60 msgr2=0x7f119c071210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:48.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.503+0000 7f11b81dc640 1 --2- 172.21.15.112:0/4275202926 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f119c06ed60 0x7f119c071210 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f11ac009fd0 tx=0x7f11ac009c50 comp rx=0 tx=0).stop 2024-09-16T00:24:48.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.503+0000 7f11b81dc640 1 -- 172.21.15.112:0/4275202926 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11b00ff680 msgr2=0x7f11b00fb040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:48.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.503+0000 7f11b81dc640 1 --2- 172.21.15.112:0/4275202926 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11b00ff680 0x7f11b00fb040 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f11a40043d0 tx=0x7f11a4009da0 comp rx=0 tx=0).stop 2024-09-16T00:24:48.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.503+0000 7f11b81dc640 1 -- 172.21.15.112:0/4275202926 shutdown_connections 2024-09-16T00:24:48.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.503+0000 7f11b81dc640 1 --2- 172.21.15.112:0/4275202926 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f119c06ed60 0x7f119c071210 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:48.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.503+0000 7f11b81dc640 1 --2- 172.21.15.112:0/4275202926 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11b00ff680 0x7f11b00fb040 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:48.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.503+0000 7f11b81dc640 1 --2- 172.21.15.112:0/4275202926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11b0065c40 0x7f11b00fcb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:48.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.503+0000 7f11b81dc640 1 -- 172.21.15.112:0/4275202926 >> 172.21.15.112:0/4275202926 conn(0x7f11b00f6c10 msgr2=0x7f11b00f6fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:48.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.503+0000 7f11b81dc640 1 -- 172.21.15.112:0/4275202926 shutdown_connections 2024-09-16T00:24:48.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.503+0000 7f11b81dc640 1 -- 172.21.15.112:0/4275202926 wait complete. 2024-09-16T00:24:48.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.590+0000 7f08d0051640 1 -- 172.21.15.112:0/2772938072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08c80ff720 msgr2=0x7f08c8101b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:48.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.590+0000 7f08d0051640 1 --2- 172.21.15.112:0/2772938072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08c80ff720 0x7f08c8101b00 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f08b80048d0 tx=0x7f08b8009da0 comp rx=0 tx=0).stop 2024-09-16T00:24:48.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.590+0000 7f08d0051640 1 -- 172.21.15.112:0/2772938072 shutdown_connections 2024-09-16T00:24:48.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.590+0000 7f08d0051640 1 --2- 172.21.15.112:0/2772938072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08c80ff720 0x7f08c8101b00 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:48.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.590+0000 7f08d0051640 1 --2- 172.21.15.112:0/2772938072 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08c8065ba0 0x7f08c80ff1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:48.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.590+0000 7f08d0051640 1 -- 172.21.15.112:0/2772938072 >> 172.21.15.112:0/2772938072 conn(0x7f08c80f6c10 msgr2=0x7f08c80f9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:48.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.590+0000 7f08d0051640 1 -- 172.21.15.112:0/2772938072 shutdown_connections 2024-09-16T00:24:48.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.590+0000 7f08d0051640 1 -- 172.21.15.112:0/2772938072 wait complete. 2024-09-16T00:24:48.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.591+0000 7f08d0051640 1 Processor -- start 2024-09-16T00:24:48.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.591+0000 7f08d0051640 1 -- start start 2024-09-16T00:24:48.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.591+0000 7f08d0051640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08c8065ba0 0x7f08c80fcbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:48.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.591+0000 7f08d0051640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08c80ff720 0x7f08c80fb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:48.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.591+0000 7f08cdded640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08c8065ba0 0x7f08c80fcbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:48.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.591+0000 7f08cd5ec640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08c80ff720 0x7f08c80fb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:48.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.591+0000 7f08cdded640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08c8065ba0 0x7f08c80fcbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.112:55080/0 (socket says 172.21.15.112:55080) 2024-09-16T00:24:48.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.591+0000 7f08cdded640 1 -- 172.21.15.112:0/2262628026 learned_addr learned my addr 172.21.15.112:0/2262628026 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:24:48.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.591+0000 7f08d0051640 1 -- 172.21.15.112:0/2262628026 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08c80fb670 con 0x7f08c80ff720 2024-09-16T00:24:48.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.591+0000 7f08d0051640 1 -- 172.21.15.112:0/2262628026 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08c80fb7b0 con 0x7f08c8065ba0 2024-09-16T00:24:48.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.592+0000 7f08cdded640 1 -- 172.21.15.112:0/2262628026 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08c80ff720 msgr2=0x7f08c80fb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:48.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.592+0000 7f08cdded640 1 --2- 172.21.15.112:0/2262628026 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08c80ff720 0x7f08c80fb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:48.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.592+0000 7f08cdded640 1 -- 172.21.15.112:0/2262628026 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08b8005040 con 0x7f08c8065ba0 2024-09-16T00:24:48.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.592+0000 7f08cd5ec640 1 --2- 172.21.15.112:0/2262628026 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08c80ff720 0x7f08c80fb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:24:48.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.592+0000 7f08cdded640 1 --2- 172.21.15.112:0/2262628026 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08c8065ba0 0x7f08c80fcbf0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f08c400eb70 tx=0x7f08c400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:48.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.592+0000 7f08be7fc640 1 -- 172.21.15.112:0/2262628026 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08c4007c30 con 0x7f08c8065ba0 2024-09-16T00:24:48.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.592+0000 7f08be7fc640 1 -- 172.21.15.112:0/2262628026 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f08c400f040 con 0x7f08c8065ba0 2024-09-16T00:24:48.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.592+0000 7f08d0051640 1 -- 172.21.15.112:0/2262628026 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08c80fba90 con 0x7f08c8065ba0 2024-09-16T00:24:48.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.592+0000 7f08be7fc640 1 -- 172.21.15.112:0/2262628026 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08c4007c30 con 0x7f08c8065ba0 2024-09-16T00:24:48.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.592+0000 7f08d0051640 1 -- 172.21.15.112:0/2262628026 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08c81150f0 con 0x7f08c8065ba0 2024-09-16T00:24:48.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.593+0000 7f08d0051640 1 -- 172.21.15.112:0/2262628026 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08c80fa890 con 0x7f08c8065ba0 2024-09-16T00:24:48.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.595+0000 7f08be7fc640 1 -- 172.21.15.112:0/2262628026 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 19) v1 ==== 90869+0+0 (secure 0 0 0) 0x7f08c4009110 con 0x7f08c8065ba0 2024-09-16T00:24:48.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.596+0000 7f08be7fc640 1 --2- 172.21.15.112:0/2262628026 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f08b406edc0 0x7f08b4071270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:48.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.596+0000 7f08cd5ec640 1 --2- 172.21.15.112:0/2262628026 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f08b406edc0 0x7f08b4071270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:48.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.596+0000 7f08be7fc640 1 -- 172.21.15.112:0/2262628026 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f08c408e600 con 0x7f08c8065ba0 2024-09-16T00:24:48.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.597+0000 7f08cd5ec640 1 --2- 172.21.15.112:0/2262628026 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f08b406edc0 0x7f08b4071270 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f08b8000c00 tx=0x7f08b8035040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:48.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.602+0000 7f08be7fc640 1 -- 172.21.15.112:0/2262628026 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f08c405b750 con 0x7f08c8065ba0 2024-09-16T00:24:48.832 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:24:48 smithi112 ceph-mon[27869]: from='client.14542 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:24:48.832 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:24:48 smithi112 ceph-mon[27869]: from='client.14546 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:24:48.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.833+0000 7f08d0051640 1 -- 172.21.15.112:0/2262628026 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f08c8104f60 con 0x7f08c8065ba0 2024-09-16T00:24:48.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.834+0000 7f08be7fc640 1 -- 172.21.15.112:0/2262628026 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+643 (secure 0 0 0) 0x7f08c4007d90 con 0x7f08c8065ba0 2024-09-16T00:24:48.836 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:24:48.836 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": { 2024-09-16T00:24:48.836 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 17.2.7-1633-g6d025bc2 (6d025bc26f3176c910707f8cb4c8d8e076640615) quincy (stable)": 2 2024-09-16T00:24:48.836 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:24:48.836 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": { 2024-09-16T00:24:48.836 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 17.2.7-1633-g6d025bc2 (6d025bc26f3176c910707f8cb4c8d8e076640615) quincy (stable)": 2 2024-09-16T00:24:48.837 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:24:48.837 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": { 2024-09-16T00:24:48.837 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 17.2.7-1633-g6d025bc2 (6d025bc26f3176c910707f8cb4c8d8e076640615) quincy (stable)": 6 2024-09-16T00:24:48.837 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:24:48.837 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": { 2024-09-16T00:24:48.837 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 17.2.7-1633-g6d025bc2 (6d025bc26f3176c910707f8cb4c8d8e076640615) quincy (stable)": 4 2024-09-16T00:24:48.837 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:24:48.837 INFO:teuthology.orchestra.run.smithi112.stdout: "overall": { 2024-09-16T00:24:48.837 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 17.2.7-1633-g6d025bc2 (6d025bc26f3176c910707f8cb4c8d8e076640615) quincy (stable)": 14 2024-09-16T00:24:48.837 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-16T00:24:48.837 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:24:48.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.837+0000 7f08d0051640 1 -- 172.21.15.112:0/2262628026 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f08b406edc0 msgr2=0x7f08b4071270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:48.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.837+0000 7f08d0051640 1 --2- 172.21.15.112:0/2262628026 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f08b406edc0 0x7f08b4071270 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f08b8000c00 tx=0x7f08b8035040 comp rx=0 tx=0).stop 2024-09-16T00:24:48.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.837+0000 7f08d0051640 1 -- 172.21.15.112:0/2262628026 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08c8065ba0 msgr2=0x7f08c80fcbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:48.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.837+0000 7f08d0051640 1 --2- 172.21.15.112:0/2262628026 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08c8065ba0 0x7f08c80fcbf0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f08c400eb70 tx=0x7f08c400b040 comp rx=0 tx=0).stop 2024-09-16T00:24:48.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.837+0000 7f08d0051640 1 -- 172.21.15.112:0/2262628026 shutdown_connections 2024-09-16T00:24:48.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.837+0000 7f08d0051640 1 --2- 172.21.15.112:0/2262628026 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f08b406edc0 0x7f08b4071270 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:48.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.837+0000 7f08d0051640 1 --2- 172.21.15.112:0/2262628026 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08c80ff720 0x7f08c80fb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:48.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.837+0000 7f08d0051640 1 --2- 172.21.15.112:0/2262628026 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08c8065ba0 0x7f08c80fcbf0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:48.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.837+0000 7f08d0051640 1 -- 172.21.15.112:0/2262628026 >> 172.21.15.112:0/2262628026 conn(0x7f08c80f6c10 msgr2=0x7f08c80f6fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:48.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.837+0000 7f08d0051640 1 -- 172.21.15.112:0/2262628026 shutdown_connections 2024-09-16T00:24:48.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.838+0000 7f08d0051640 1 -- 172.21.15.112:0/2262628026 wait complete. 2024-09-16T00:24:48.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.941+0000 7ff5f1e82640 1 -- 172.21.15.112:0/3077305395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5ec0ff680 msgr2=0x7ff5ec101a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:48.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.941+0000 7ff5f1e82640 1 --2- 172.21.15.112:0/3077305395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5ec0ff680 0x7ff5ec101a60 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff5e00048d0 tx=0x7ff5e0009da0 comp rx=0 tx=0).stop 2024-09-16T00:24:48.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.941+0000 7ff5f1e82640 1 -- 172.21.15.112:0/3077305395 shutdown_connections 2024-09-16T00:24:48.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.941+0000 7ff5f1e82640 1 --2- 172.21.15.112:0/3077305395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5ec0ff680 0x7ff5ec101a60 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:48.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.941+0000 7ff5f1e82640 1 --2- 172.21.15.112:0/3077305395 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5ec065c40 0x7ff5ec0ff140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:48.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.941+0000 7ff5f1e82640 1 -- 172.21.15.112:0/3077305395 >> 172.21.15.112:0/3077305395 conn(0x7ff5ec0f6c10 msgr2=0x7ff5ec0f9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:48.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.942+0000 7ff5f1e82640 1 -- 172.21.15.112:0/3077305395 shutdown_connections 2024-09-16T00:24:48.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.942+0000 7ff5f1e82640 1 -- 172.21.15.112:0/3077305395 wait complete. 2024-09-16T00:24:48.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.942+0000 7ff5f1e82640 1 Processor -- start 2024-09-16T00:24:48.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.942+0000 7ff5f1e82640 1 -- start start 2024-09-16T00:24:48.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.943+0000 7ff5f1e82640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5ec065c40 0x7ff5ec0fcab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:48.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.943+0000 7ff5f1e82640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5ec0ff680 0x7ff5ec0fb040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:48.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.943+0000 7ff5f1e82640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ec0fb580 con 0x7ff5ec065c40 2024-09-16T00:24:48.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.943+0000 7ff5f1e82640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ec0fb6f0 con 0x7ff5ec0ff680 2024-09-16T00:24:48.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.943+0000 7ff5eb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5ec065c40 0x7ff5ec0fcab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:48.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.943+0000 7ff5eb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5ec065c40 0x7ff5ec0fcab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47658/0 (socket says 172.21.15.112:47658) 2024-09-16T00:24:48.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.943+0000 7ff5eb7fe640 1 -- 172.21.15.112:0/316558760 learned_addr learned my addr 172.21.15.112:0/316558760 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:24:48.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.943+0000 7ff5eaffd640 1 --2- 172.21.15.112:0/316558760 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5ec0ff680 0x7ff5ec0fb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:48.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.943+0000 7ff5eaffd640 1 -- 172.21.15.112:0/316558760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5ec065c40 msgr2=0x7ff5ec0fcab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:48.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.943+0000 7ff5eaffd640 1 --2- 172.21.15.112:0/316558760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5ec065c40 0x7ff5ec0fcab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:48.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.943+0000 7ff5eaffd640 1 -- 172.21.15.112:0/316558760 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e0005040 con 0x7ff5ec0ff680 2024-09-16T00:24:48.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.943+0000 7ff5eaffd640 1 --2- 172.21.15.112:0/316558760 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5ec0ff680 0x7ff5ec0fb040 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff5e00043d0 tx=0x7ff5e0009da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:48.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.944+0000 7ff5d3fff640 1 -- 172.21.15.112:0/316558760 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff5e0038070 con 0x7ff5ec0ff680 2024-09-16T00:24:48.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.944+0000 7ff5f1e82640 1 -- 172.21.15.112:0/316558760 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5ec0fb970 con 0x7ff5ec0ff680 2024-09-16T00:24:48.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.944+0000 7ff5d3fff640 1 -- 172.21.15.112:0/316558760 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7ff5e0009800 con 0x7ff5ec0ff680 2024-09-16T00:24:48.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.944+0000 7ff5d3fff640 1 -- 172.21.15.112:0/316558760 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff5e0033660 con 0x7ff5ec0ff680 2024-09-16T00:24:48.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.944+0000 7ff5f1e82640 1 -- 172.21.15.112:0/316558760 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5ec0fbe90 con 0x7ff5ec0ff680 2024-09-16T00:24:48.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.945+0000 7ff5f1e82640 1 -- 172.21.15.112:0/316558760 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5ec0fa890 con 0x7ff5ec0ff680 2024-09-16T00:24:48.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.946+0000 7ff5d3fff640 1 -- 172.21.15.112:0/316558760 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 19) v1 ==== 90869+0+0 (secure 0 0 0) 0x7ff5e0030020 con 0x7ff5ec0ff680 2024-09-16T00:24:48.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.946+0000 7ff5d3fff640 1 --2- 172.21.15.112:0/316558760 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7ff5d406ed50 0x7ff5d4071200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:48.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.946+0000 7ff5d3fff640 1 -- 172.21.15.112:0/316558760 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5550+0+0 (secure 0 0 0) 0x7ff5e00ae260 con 0x7ff5ec0ff680 2024-09-16T00:24:48.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.947+0000 7ff5eb7fe640 1 --2- 172.21.15.112:0/316558760 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7ff5d406ed50 0x7ff5d4071200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:48.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.947+0000 7ff5eb7fe640 1 --2- 172.21.15.112:0/316558760 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7ff5d406ed50 0x7ff5d4071200 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec0fc860 tx=0x7ff5dc009f10 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:48.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:48.955+0000 7ff5d3fff640 1 -- 172.21.15.112:0/316558760 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7ff5e007b3b0 con 0x7ff5ec0ff680 2024-09-16T00:24:49.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:24:48 smithi169 ceph-mon[34653]: from='client.14542 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:24:49.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:24:48 smithi169 ceph-mon[34653]: from='client.14546 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:24:49.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:49.135+0000 7ff5f1e82640 1 -- 172.21.15.112:0/316558760 --> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff5ec0f8700 con 0x7ff5d406ed50 2024-09-16T00:24:49.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:49.136+0000 7ff5d3fff640 1 -- 172.21.15.112:0/316558760 <== mgr.14208 v2:172.21.15.112:6800/2480414866 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7ff5ec0f8700 con 0x7ff5d406ed50 2024-09-16T00:24:49.136 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:24:49.136 INFO:teuthology.orchestra.run.smithi112.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80", 2024-09-16T00:24:49.137 INFO:teuthology.orchestra.run.smithi112.stdout: "in_progress": true, 2024-09-16T00:24:49.137 INFO:teuthology.orchestra.run.smithi112.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-09-16T00:24:49.137 INFO:teuthology.orchestra.run.smithi112.stdout: "services_complete": [], 2024-09-16T00:24:49.137 INFO:teuthology.orchestra.run.smithi112.stdout: "progress": "0/2 daemons upgraded", 2024-09-16T00:24:49.137 INFO:teuthology.orchestra.run.smithi112.stdout: "message": "Currently upgrading mgr daemons", 2024-09-16T00:24:49.137 INFO:teuthology.orchestra.run.smithi112.stdout: "is_paused": false 2024-09-16T00:24:49.137 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:24:49.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:49.138+0000 7ff5f1e82640 1 -- 172.21.15.112:0/316558760 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7ff5d406ed50 msgr2=0x7ff5d4071200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:49.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:49.139+0000 7ff5f1e82640 1 --2- 172.21.15.112:0/316558760 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7ff5d406ed50 0x7ff5d4071200 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec0fc860 tx=0x7ff5dc009f10 comp rx=0 tx=0).stop 2024-09-16T00:24:49.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:49.139+0000 7ff5f1e82640 1 -- 172.21.15.112:0/316558760 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5ec0ff680 msgr2=0x7ff5ec0fb040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:49.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:49.139+0000 7ff5f1e82640 1 --2- 172.21.15.112:0/316558760 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5ec0ff680 0x7ff5ec0fb040 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff5e00043d0 tx=0x7ff5e0009da0 comp rx=0 tx=0).stop 2024-09-16T00:24:49.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:49.139+0000 7ff5f1e82640 1 -- 172.21.15.112:0/316558760 shutdown_connections 2024-09-16T00:24:49.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:49.139+0000 7ff5f1e82640 1 --2- 172.21.15.112:0/316558760 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7ff5d406ed50 0x7ff5d4071200 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:49.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:49.139+0000 7ff5f1e82640 1 --2- 172.21.15.112:0/316558760 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5ec0ff680 0x7ff5ec0fb040 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:49.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:49.139+0000 7ff5f1e82640 1 --2- 172.21.15.112:0/316558760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5ec065c40 0x7ff5ec0fcab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:49.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:49.139+0000 7ff5f1e82640 1 -- 172.21.15.112:0/316558760 >> 172.21.15.112:0/316558760 conn(0x7ff5ec0f6c10 msgr2=0x7ff5ec0f6fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:49.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:49.139+0000 7ff5f1e82640 1 -- 172.21.15.112:0/316558760 shutdown_connections 2024-09-16T00:24:49.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:24:49.139+0000 7ff5f1e82640 1 -- 172.21.15.112:0/316558760 wait complete. 2024-09-16T00:24:49.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:24:49 smithi112 ceph-mon[27869]: from='client.24309 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:24:49.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:24:49 smithi112 ceph-mon[27869]: pgmap v218: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:24:49.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:24:49 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/2262628026' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:24:49.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:24:49 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:24:49.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:24:49 smithi112 ceph-mon[27869]: Upgrade: Target is version 19.1.1-326-g8921c8ef (unknown) 2024-09-16T00:24:49.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:24:49 smithi112 ceph-mon[27869]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, digests ['quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80'] 2024-09-16T00:24:49.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:24:49 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:24:49.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:24:49 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:24:49.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:24:49 smithi112 ceph-mon[27869]: Upgrade: Need to upgrade myself (mgr.smithi112.xrkkbc) 2024-09-16T00:24:49.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:24:49 smithi112 ceph-mon[27869]: from='client.24317 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:24:49.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:24:49 smithi112 ceph-mon[27869]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 on smithi169 2024-09-16T00:24:50.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:24:49 smithi169 ceph-mon[34653]: from='client.24309 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:24:50.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:24:49 smithi169 ceph-mon[34653]: pgmap v218: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:24:50.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:24:49 smithi169 ceph-mon[34653]: from='client.? 172.21.15.112:0/2262628026' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:24:50.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:24:49 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:24:50.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:24:49 smithi169 ceph-mon[34653]: Upgrade: Target is version 19.1.1-326-g8921c8ef (unknown) 2024-09-16T00:24:50.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:24:49 smithi169 ceph-mon[34653]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, digests ['quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80'] 2024-09-16T00:24:50.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:24:49 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:24:50.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:24:49 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:24:50.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:24:49 smithi169 ceph-mon[34653]: Upgrade: Need to upgrade myself (mgr.smithi112.xrkkbc) 2024-09-16T00:24:50.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:24:49 smithi169 ceph-mon[34653]: from='client.24317 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:24:50.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:24:49 smithi169 ceph-mon[34653]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 on smithi169 2024-09-16T00:24:52.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:24:51 smithi169 ceph-mon[34653]: pgmap v219: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:24:52.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:24:51 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:24:52.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:24:51 smithi112 ceph-mon[27869]: pgmap v219: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:24:52.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:24:51 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:24:54.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:24:53 smithi169 ceph-mon[34653]: pgmap v220: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:24:54.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:24:53 smithi112 ceph-mon[27869]: pgmap v220: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:24:56.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:24:55 smithi169 ceph-mon[34653]: pgmap v221: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-16T00:24:56.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:24:55 smithi112 ceph-mon[27869]: pgmap v221: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-16T00:24:58.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:24:57 smithi169 ceph-mon[34653]: pgmap v222: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:24:58.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:24:57 smithi112 ceph-mon[27869]: pgmap v222: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:24:59.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:24:59 smithi112 ceph-mon[27869]: pgmap v223: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:25:00.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:24:59 smithi169 ceph-mon[34653]: pgmap v223: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:25:02.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:25:01 smithi169 ceph-mon[34653]: pgmap v224: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:25:02.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:25:01 smithi112 ceph-mon[27869]: pgmap v224: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:25:03.381 INFO:tasks.workunit.client.1.smithi169.stderr:Updating files: 89% (11504/12889) Updating files: 90% (11601/12889) Updating files: 91% (11729/12889) Updating files: 92% (11858/12889) Updating files: 93% (11987/12889) Updating files: 94% (12116/12889) Updating files: 95% (12245/12889) Updating files: 96% (12374/12889) Updating files: 97% (12503/12889) Updating files: 98% (12632/12889) Updating files: 99% (12761/12889) Updating files: 100% (12889/12889) Updating files: 100% (12889/12889), done. 2024-09-16T00:25:04.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:25:03 smithi169 ceph-mon[34653]: pgmap v225: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:25:04.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:25:03 smithi112 ceph-mon[27869]: pgmap v225: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:25:04.096 INFO:tasks.workunit.client.1.smithi169.stderr:Note: switching to 'ad35db06e8924912f01e32b35576297c0f1352b4'. 2024-09-16T00:25:04.096 INFO:tasks.workunit.client.1.smithi169.stderr: 2024-09-16T00:25:04.096 INFO:tasks.workunit.client.1.smithi169.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-16T00:25:04.096 INFO:tasks.workunit.client.1.smithi169.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-16T00:25:04.096 INFO:tasks.workunit.client.1.smithi169.stderr:state without impacting any branches by switching back to a branch. 2024-09-16T00:25:04.096 INFO:tasks.workunit.client.1.smithi169.stderr: 2024-09-16T00:25:04.096 INFO:tasks.workunit.client.1.smithi169.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-16T00:25:04.096 INFO:tasks.workunit.client.1.smithi169.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-16T00:25:04.096 INFO:tasks.workunit.client.1.smithi169.stderr: 2024-09-16T00:25:04.096 INFO:tasks.workunit.client.1.smithi169.stderr: git switch -c 2024-09-16T00:25:04.096 INFO:tasks.workunit.client.1.smithi169.stderr: 2024-09-16T00:25:04.097 INFO:tasks.workunit.client.1.smithi169.stderr:Or undo this operation with: 2024-09-16T00:25:04.097 INFO:tasks.workunit.client.1.smithi169.stderr: 2024-09-16T00:25:04.097 INFO:tasks.workunit.client.1.smithi169.stderr: git switch - 2024-09-16T00:25:04.097 INFO:tasks.workunit.client.1.smithi169.stderr: 2024-09-16T00:25:04.097 INFO:tasks.workunit.client.1.smithi169.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-16T00:25:04.097 INFO:tasks.workunit.client.1.smithi169.stderr: 2024-09-16T00:25:04.097 INFO:tasks.workunit.client.1.smithi169.stderr:HEAD is now at ad35db06e89 Merge pull request #59794 from zdover23/wip-doc-2024-09-15-backport-59613-to-squid 2024-09-16T00:25:04.101 DEBUG:teuthology.orchestra.run.smithi169:> cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.1 2024-09-16T00:25:04.284 INFO:tasks.workunit.client.1.smithi169.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-16T00:25:04.286 INFO:tasks.workunit.client.1.smithi169.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-09-16T00:25:04.287 INFO:tasks.workunit.client.1.smithi169.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-16T00:25:04.923 INFO:tasks.workunit.client.1.smithi169.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-16T00:25:05.003 INFO:tasks.workunit.client.1.smithi169.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-16T00:25:05.047 INFO:tasks.workunit.client.1.smithi169.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-09-16T00:25:05.049 INFO:tasks.workunit.client.1.smithi169.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-09-16T00:25:05.049 INFO:tasks.workunit.client.1.smithi169.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-16T00:25:05.090 INFO:tasks.workunit.client.1.smithi169.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-09-16T00:25:05.093 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-09-16T00:25:05.094 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/home/ubuntu/cephtest/workunits.list.client.1 of=/dev/stdout 2024-09-16T00:25:05.110 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.1... 2024-09-16T00:25:05.111 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-09-16T00:25:05.111 DEBUG:teuthology.orchestra.run.smithi169:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ad35db06e8924912f01e32b35576297c0f1352b4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh 2024-09-16T00:25:05.178 INFO:tasks.workunit.client.1.smithi169.stderr:+ mkdir -p fsstress 2024-09-16T00:25:05.180 INFO:tasks.workunit.client.1.smithi169.stderr:+ pushd fsstress 2024-09-16T00:25:05.182 INFO:tasks.workunit.client.1.smithi169.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-09-16T00:25:05.182 INFO:tasks.workunit.client.1.smithi169.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-09-16T00:25:05.730 INFO:tasks.workunit.client.1.smithi169.stderr:+ tar xzf ltp-full.tgz 2024-09-16T00:25:06.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:25:05 smithi169 ceph-mon[34653]: pgmap v226: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-16T00:25:06.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:25:05 smithi112 ceph-mon[27869]: pgmap v226: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-16T00:25:06.351 INFO:tasks.workunit.client.0.smithi112.stderr:Note: switching to 'ad35db06e8924912f01e32b35576297c0f1352b4'. 2024-09-16T00:25:06.351 INFO:tasks.workunit.client.0.smithi112.stderr: 2024-09-16T00:25:06.351 INFO:tasks.workunit.client.0.smithi112.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-16T00:25:06.351 INFO:tasks.workunit.client.0.smithi112.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-16T00:25:06.351 INFO:tasks.workunit.client.0.smithi112.stderr:state without impacting any branches by switching back to a branch. 2024-09-16T00:25:06.352 INFO:tasks.workunit.client.0.smithi112.stderr: 2024-09-16T00:25:06.352 INFO:tasks.workunit.client.0.smithi112.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-16T00:25:06.352 INFO:tasks.workunit.client.0.smithi112.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-16T00:25:06.352 INFO:tasks.workunit.client.0.smithi112.stderr: 2024-09-16T00:25:06.352 INFO:tasks.workunit.client.0.smithi112.stderr: git switch -c 2024-09-16T00:25:06.352 INFO:tasks.workunit.client.0.smithi112.stderr: 2024-09-16T00:25:06.352 INFO:tasks.workunit.client.0.smithi112.stderr:Or undo this operation with: 2024-09-16T00:25:06.352 INFO:tasks.workunit.client.0.smithi112.stderr: 2024-09-16T00:25:06.352 INFO:tasks.workunit.client.0.smithi112.stderr: git switch - 2024-09-16T00:25:06.352 INFO:tasks.workunit.client.0.smithi112.stderr: 2024-09-16T00:25:06.353 INFO:tasks.workunit.client.0.smithi112.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-16T00:25:06.353 INFO:tasks.workunit.client.0.smithi112.stderr: 2024-09-16T00:25:06.353 INFO:tasks.workunit.client.0.smithi112.stderr:HEAD is now at ad35db06e89 Merge pull request #59794 from zdover23/wip-doc-2024-09-15-backport-59613-to-squid 2024-09-16T00:25:06.358 DEBUG:teuthology.orchestra.run.smithi112:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-16T00:25:06.399 INFO:tasks.workunit.client.0.smithi112.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-16T00:25:06.401 INFO:tasks.workunit.client.0.smithi112.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-16T00:25:06.402 INFO:tasks.workunit.client.0.smithi112.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-16T00:25:07.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:25:06 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:25:07.074 INFO:tasks.workunit.client.0.smithi112.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-16T00:25:07.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:25:06 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:25:07.161 INFO:tasks.workunit.client.0.smithi112.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-16T00:25:07.200 INFO:tasks.workunit.client.0.smithi112.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-16T00:25:07.202 INFO:tasks.workunit.client.0.smithi112.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-16T00:25:07.202 INFO:tasks.workunit.client.0.smithi112.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-16T00:25:07.244 INFO:tasks.workunit.client.0.smithi112.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-16T00:25:07.247 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-16T00:25:07.247 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-16T00:25:07.303 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.0... 2024-09-16T00:25:07.306 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-09-16T00:25:07.306 DEBUG:teuthology.orchestra.run.smithi112:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ad35db06e8924912f01e32b35576297c0f1352b4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh 2024-09-16T00:25:07.367 INFO:tasks.workunit.client.0.smithi112.stderr:+ mkdir -p fsstress 2024-09-16T00:25:07.370 INFO:tasks.workunit.client.0.smithi112.stderr:+ pushd fsstress 2024-09-16T00:25:07.371 INFO:tasks.workunit.client.0.smithi112.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-09-16T00:25:07.371 INFO:tasks.workunit.client.0.smithi112.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-09-16T00:25:07.970 INFO:tasks.workunit.client.0.smithi112.stderr:+ tar xzf ltp-full.tgz 2024-09-16T00:25:08.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:25:07 smithi169 ceph-mon[34653]: pgmap v227: 65 pgs: 65 active+clean; 598 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 3 op/s 2024-09-16T00:25:08.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:25:07 smithi112 ceph-mon[27869]: pgmap v227: 65 pgs: 65 active+clean; 598 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 3 op/s 2024-09-16T00:25:10.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:25:09 smithi169 ceph-mon[34653]: pgmap v228: 65 pgs: 65 active+clean; 17 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 84 KiB/s rd, 1.4 MiB/s wr, 23 op/s 2024-09-16T00:25:10.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:25:09 smithi112 ceph-mon[27869]: pgmap v228: 65 pgs: 65 active+clean; 17 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 84 KiB/s rd, 1.4 MiB/s wr, 23 op/s 2024-09-16T00:25:12.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:25:11 smithi169 ceph-mon[34653]: pgmap v229: 65 pgs: 65 active+clean; 22 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 83 KiB/s rd, 1.8 MiB/s wr, 32 op/s 2024-09-16T00:25:12.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:25:11 smithi112 ceph-mon[27869]: pgmap v229: 65 pgs: 65 active+clean; 22 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 83 KiB/s rd, 1.8 MiB/s wr, 32 op/s 2024-09-16T00:25:14.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:25:13 smithi169 ceph-mon[34653]: pgmap v230: 65 pgs: 65 active+clean; 34 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 725 KiB/s rd, 2.8 MiB/s wr, 88 op/s 2024-09-16T00:25:14.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:25:13 smithi112 ceph-mon[27869]: pgmap v230: 65 pgs: 65 active+clean; 34 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 725 KiB/s rd, 2.8 MiB/s wr, 88 op/s 2024-09-16T00:25:15.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:25:14 smithi169 ceph-mon[34653]: fsmap cephfs:2 {0=cephfs.smithi112.hxfwkh=up:active,1=cephfs.smithi169.gawwgw=up:active} 2 up:standby-replay 2024-09-16T00:25:15.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:25:14 smithi112 ceph-mon[27869]: fsmap cephfs:2 {0=cephfs.smithi112.hxfwkh=up:active,1=cephfs.smithi169.gawwgw=up:active} 2 up:standby-replay 2024-09-16T00:25:16.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:25:15 smithi112 ceph-mon[27869]: pgmap v231: 65 pgs: 65 active+clean; 40 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 924 KiB/s rd, 3.3 MiB/s wr, 130 op/s 2024-09-16T00:25:16.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:25:15 smithi169 ceph-mon[34653]: pgmap v231: 65 pgs: 65 active+clean; 40 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 924 KiB/s rd, 3.3 MiB/s wr, 130 op/s 2024-09-16T00:25:18.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:25:17 smithi169 ceph-mon[34653]: pgmap v232: 65 pgs: 65 active+clean; 40 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 924 KiB/s rd, 3.3 MiB/s wr, 136 op/s 2024-09-16T00:25:18.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:25:17 smithi112 ceph-mon[27869]: pgmap v232: 65 pgs: 65 active+clean; 40 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 924 KiB/s rd, 3.3 MiB/s wr, 136 op/s 2024-09-16T00:25:19.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.244+0000 7f21a4767640 1 -- 172.21.15.112:0/955605197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f219c0fc700 msgr2=0x7f219c0fcb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:19.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.244+0000 7f21a4767640 1 --2- 172.21.15.112:0/955605197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f219c0fc700 0x7f219c0fcb70 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f218c0048d0 tx=0x7f218c009da0 comp rx=0 tx=0).stop 2024-09-16T00:25:19.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.244+0000 7f21a4767640 1 -- 172.21.15.112:0/955605197 shutdown_connections 2024-09-16T00:25:19.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.244+0000 7f21a4767640 1 --2- 172.21.15.112:0/955605197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f219c0fc700 0x7f219c0fcb70 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:19.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.244+0000 7f21a4767640 1 --2- 172.21.15.112:0/955605197 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f219c0fb430 0x7f219c0fb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:19.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.244+0000 7f21a4767640 1 -- 172.21.15.112:0/955605197 >> 172.21.15.112:0/955605197 conn(0x7f219c0f6c10 msgr2=0x7f219c0f9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:19.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.244+0000 7f21a4767640 1 -- 172.21.15.112:0/955605197 shutdown_connections 2024-09-16T00:25:19.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.244+0000 7f21a4767640 1 -- 172.21.15.112:0/955605197 wait complete. 2024-09-16T00:25:19.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.244+0000 7f21a4767640 1 Processor -- start 2024-09-16T00:25:19.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.244+0000 7f21a4767640 1 -- start start 2024-09-16T00:25:19.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.245+0000 7f21a4767640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f219c0fb430 0x7f219c10fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:19.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.245+0000 7f21a4767640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f219c0fc700 0x7f219c112270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:19.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.245+0000 7f21a4767640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f219c065cc0 con 0x7f219c0fc700 2024-09-16T00:25:19.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.245+0000 7f21a4767640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f219c065e30 con 0x7f219c0fb430 2024-09-16T00:25:19.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.245+0000 7f21a1d02640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f219c0fc700 0x7f219c112270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:19.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.245+0000 7f21a1d02640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f219c0fc700 0x7f219c112270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52312/0 (socket says 172.21.15.112:52312) 2024-09-16T00:25:19.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.245+0000 7f21a1d02640 1 -- 172.21.15.112:0/412228159 learned_addr learned my addr 172.21.15.112:0/412228159 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:25:19.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.245+0000 7f21a2503640 1 --2- 172.21.15.112:0/412228159 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f219c0fb430 0x7f219c10fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:19.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.245+0000 7f21a1d02640 1 -- 172.21.15.112:0/412228159 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f219c0fb430 msgr2=0x7f219c10fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:19.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.245+0000 7f21a1d02640 1 --2- 172.21.15.112:0/412228159 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f219c0fb430 0x7f219c10fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:19.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.245+0000 7f21a1d02640 1 -- 172.21.15.112:0/412228159 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f218c005040 con 0x7f219c0fc700 2024-09-16T00:25:19.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.245+0000 7f21a1d02640 1 --2- 172.21.15.112:0/412228159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f219c0fc700 0x7f219c112270 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f218c0043d0 tx=0x7f218c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:19.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.246+0000 7f2192ffd640 1 -- 172.21.15.112:0/412228159 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f218c038070 con 0x7f219c0fc700 2024-09-16T00:25:19.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.246+0000 7f21a4767640 1 -- 172.21.15.112:0/412228159 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f219c112810 con 0x7f219c0fc700 2024-09-16T00:25:19.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.246+0000 7f2192ffd640 1 -- 172.21.15.112:0/412228159 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f218c02ae90 con 0x7f219c0fc700 2024-09-16T00:25:19.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.246+0000 7f2192ffd640 1 -- 172.21.15.112:0/412228159 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f218c03e670 con 0x7f219c0fc700 2024-09-16T00:25:19.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.246+0000 7f21a4767640 1 -- 172.21.15.112:0/412228159 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f219c112d80 con 0x7f219c0fc700 2024-09-16T00:25:19.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.247+0000 7f21a4767640 1 -- 172.21.15.112:0/412228159 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f219c0fa890 con 0x7f219c0fc700 2024-09-16T00:25:19.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.247+0000 7f2192ffd640 1 -- 172.21.15.112:0/412228159 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) v1 ==== 90869+0+0 (secure 0 0 0) 0x7f218c030020 con 0x7f219c0fc700 2024-09-16T00:25:19.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.248+0000 7f2192ffd640 1 --2- 172.21.15.112:0/412228159 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f218806ed60 0x7f2188071210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:19.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.248+0000 7f21a2503640 1 --2- 172.21.15.112:0/412228159 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f218806ed60 0x7f2188071210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:19.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.248+0000 7f2192ffd640 1 -- 172.21.15.112:0/412228159 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f218c0ae460 con 0x7f219c0fc700 2024-09-16T00:25:19.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.248+0000 7f21a2503640 1 --2- 172.21.15.112:0/412228159 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f218806ed60 0x7f2188071210 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f219c0659b0 tx=0x7f219800b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:19.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.252+0000 7f2192ffd640 1 -- 172.21.15.112:0/412228159 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f218c07b540 con 0x7f219c0fc700 2024-09-16T00:25:19.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.500+0000 7f21a4767640 1 -- 172.21.15.112:0/412228159 --> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f219c100bb0 con 0x7f218806ed60 2024-09-16T00:25:19.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.501+0000 7f2192ffd640 1 -- 172.21.15.112:0/412228159 <== mgr.14208 v2:172.21.15.112:6800/2480414866 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f219c100bb0 con 0x7f218806ed60 2024-09-16T00:25:19.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.503+0000 7f21a4767640 1 -- 172.21.15.112:0/412228159 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f218806ed60 msgr2=0x7f2188071210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:19.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.503+0000 7f21a4767640 1 --2- 172.21.15.112:0/412228159 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f218806ed60 0x7f2188071210 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f219c0659b0 tx=0x7f219800b040 comp rx=0 tx=0).stop 2024-09-16T00:25:19.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.503+0000 7f21a4767640 1 -- 172.21.15.112:0/412228159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f219c0fc700 msgr2=0x7f219c112270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:19.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.503+0000 7f21a4767640 1 --2- 172.21.15.112:0/412228159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f219c0fc700 0x7f219c112270 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f218c0043d0 tx=0x7f218c009da0 comp rx=0 tx=0).stop 2024-09-16T00:25:19.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.503+0000 7f21a4767640 1 -- 172.21.15.112:0/412228159 shutdown_connections 2024-09-16T00:25:19.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.503+0000 7f21a4767640 1 --2- 172.21.15.112:0/412228159 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f218806ed60 0x7f2188071210 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:19.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.503+0000 7f21a4767640 1 --2- 172.21.15.112:0/412228159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f219c0fc700 0x7f219c112270 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:19.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.503+0000 7f21a4767640 1 --2- 172.21.15.112:0/412228159 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f219c0fb430 0x7f219c10fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:19.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.503+0000 7f21a4767640 1 -- 172.21.15.112:0/412228159 >> 172.21.15.112:0/412228159 conn(0x7f219c0f6c10 msgr2=0x7f219c0f83b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:19.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.504+0000 7f21a4767640 1 -- 172.21.15.112:0/412228159 shutdown_connections 2024-09-16T00:25:19.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.504+0000 7f21a4767640 1 -- 172.21.15.112:0/412228159 wait complete. 2024-09-16T00:25:19.513 INFO:teuthology.orchestra.run.smithi112.stdout:true 2024-09-16T00:25:19.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.593+0000 7f78736a9640 1 -- 172.21.15.112:0/137019589 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f786c101ae0 msgr2=0x7f786c103ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:19.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.593+0000 7f78736a9640 1 --2- 172.21.15.112:0/137019589 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f786c101ae0 0x7f786c103ec0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f785c0048d0 tx=0x7f785c009da0 comp rx=0 tx=0).stop 2024-09-16T00:25:19.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.593+0000 7f78736a9640 1 -- 172.21.15.112:0/137019589 shutdown_connections 2024-09-16T00:25:19.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.593+0000 7f78736a9640 1 --2- 172.21.15.112:0/137019589 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f786c101ae0 0x7f786c103ec0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:19.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.593+0000 7f78736a9640 1 --2- 172.21.15.112:0/137019589 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f786c065be0 0x7f786c1015a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:19.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.593+0000 7f78736a9640 1 -- 172.21.15.112:0/137019589 >> 172.21.15.112:0/137019589 conn(0x7f786c0f6c10 msgr2=0x7f786c0f9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:19.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.593+0000 7f78736a9640 1 -- 172.21.15.112:0/137019589 shutdown_connections 2024-09-16T00:25:19.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.593+0000 7f78736a9640 1 -- 172.21.15.112:0/137019589 wait complete. 2024-09-16T00:25:19.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.594+0000 7f78736a9640 1 Processor -- start 2024-09-16T00:25:19.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.594+0000 7f78736a9640 1 -- start start 2024-09-16T00:25:19.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.594+0000 7f78736a9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f786c065be0 0x7f786c10f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:19.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.594+0000 7f78736a9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f786c101ae0 0x7f786c10f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:19.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.594+0000 7f78736a9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786c10fdd0 con 0x7f786c065be0 2024-09-16T00:25:19.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.594+0000 7f78736a9640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786c1105f0 con 0x7f786c101ae0 2024-09-16T00:25:19.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.594+0000 7f7871445640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f786c065be0 0x7f786c10f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:19.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.594+0000 7f7871445640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f786c065be0 0x7f786c10f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52324/0 (socket says 172.21.15.112:52324) 2024-09-16T00:25:19.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.594+0000 7f7871445640 1 -- 172.21.15.112:0/2394232789 learned_addr learned my addr 172.21.15.112:0/2394232789 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:25:19.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.594+0000 7f7870c44640 1 --2- 172.21.15.112:0/2394232789 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f786c101ae0 0x7f786c10f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:19.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.594+0000 7f7871445640 1 -- 172.21.15.112:0/2394232789 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f786c101ae0 msgr2=0x7f786c10f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:19.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.594+0000 7f7871445640 1 --2- 172.21.15.112:0/2394232789 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f786c101ae0 0x7f786c10f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:19.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.594+0000 7f7871445640 1 -- 172.21.15.112:0/2394232789 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f785c005040 con 0x7f786c065be0 2024-09-16T00:25:19.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.595+0000 7f7870c44640 1 --2- 172.21.15.112:0/2394232789 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f786c101ae0 0x7f786c10f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:25:19.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.595+0000 7f7871445640 1 --2- 172.21.15.112:0/2394232789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f786c065be0 0x7f786c10f240 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7868002a80 tx=0x7f7868004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:19.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.595+0000 7f7861ffb640 1 -- 172.21.15.112:0/2394232789 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7868007c30 con 0x7f786c065be0 2024-09-16T00:25:19.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.595+0000 7f7861ffb640 1 -- 172.21.15.112:0/2394232789 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f786800f040 con 0x7f786c065be0 2024-09-16T00:25:19.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.595+0000 7f78736a9640 1 -- 172.21.15.112:0/2394232789 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f786c110880 con 0x7f786c065be0 2024-09-16T00:25:19.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.595+0000 7f7861ffb640 1 -- 172.21.15.112:0/2394232789 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7868009540 con 0x7f786c065be0 2024-09-16T00:25:19.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.595+0000 7f78736a9640 1 -- 172.21.15.112:0/2394232789 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f786c110e20 con 0x7f786c065be0 2024-09-16T00:25:19.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.596+0000 7f78736a9640 1 -- 172.21.15.112:0/2394232789 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f786c0fa890 con 0x7f786c065be0 2024-09-16T00:25:19.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.597+0000 7f7861ffb640 1 -- 172.21.15.112:0/2394232789 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) v1 ==== 90869+0+0 (secure 0 0 0) 0x7f786800d6b0 con 0x7f786c065be0 2024-09-16T00:25:19.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.597+0000 7f7861ffb640 1 --2- 172.21.15.112:0/2394232789 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f785806ed60 0x7f7858071210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:19.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.598+0000 7f7861ffb640 1 -- 172.21.15.112:0/2394232789 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f786808e1a0 con 0x7f786c065be0 2024-09-16T00:25:19.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.598+0000 7f7870c44640 1 --2- 172.21.15.112:0/2394232789 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f785806ed60 0x7f7858071210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:19.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.598+0000 7f7870c44640 1 --2- 172.21.15.112:0/2394232789 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f785806ed60 0x7f7858071210 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f786c10eff0 tx=0x7f785c009da0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:19.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.602+0000 7f7861ffb640 1 -- 172.21.15.112:0/2394232789 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f786805adf0 con 0x7f786c065be0 2024-09-16T00:25:19.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.798+0000 7f78736a9640 1 -- 172.21.15.112:0/2394232789 --> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f786c1111f0 con 0x7f785806ed60 2024-09-16T00:25:19.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.800+0000 7f7861ffb640 1 -- 172.21.15.112:0/2394232789 <== mgr.14208 v2:172.21.15.112:6800/2480414866 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f786c1111f0 con 0x7f785806ed60 2024-09-16T00:25:19.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.802+0000 7f78736a9640 1 -- 172.21.15.112:0/2394232789 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f785806ed60 msgr2=0x7f7858071210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:19.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.802+0000 7f78736a9640 1 --2- 172.21.15.112:0/2394232789 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f785806ed60 0x7f7858071210 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f786c10eff0 tx=0x7f785c009da0 comp rx=0 tx=0).stop 2024-09-16T00:25:19.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.802+0000 7f78736a9640 1 -- 172.21.15.112:0/2394232789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f786c065be0 msgr2=0x7f786c10f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:19.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.802+0000 7f78736a9640 1 --2- 172.21.15.112:0/2394232789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f786c065be0 0x7f786c10f240 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7868002a80 tx=0x7f7868004570 comp rx=0 tx=0).stop 2024-09-16T00:25:19.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.802+0000 7f78736a9640 1 -- 172.21.15.112:0/2394232789 shutdown_connections 2024-09-16T00:25:19.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.802+0000 7f78736a9640 1 --2- 172.21.15.112:0/2394232789 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f785806ed60 0x7f7858071210 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:19.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.802+0000 7f78736a9640 1 --2- 172.21.15.112:0/2394232789 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f786c101ae0 0x7f786c10f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:19.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.802+0000 7f78736a9640 1 --2- 172.21.15.112:0/2394232789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f786c065be0 0x7f786c10f240 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:19.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.802+0000 7f78736a9640 1 -- 172.21.15.112:0/2394232789 >> 172.21.15.112:0/2394232789 conn(0x7f786c0f6c10 msgr2=0x7f786c0f6fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:19.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.802+0000 7f78736a9640 1 -- 172.21.15.112:0/2394232789 shutdown_connections 2024-09-16T00:25:19.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.802+0000 7f78736a9640 1 -- 172.21.15.112:0/2394232789 wait complete. 2024-09-16T00:25:19.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.896+0000 7f01118e7640 1 -- 172.21.15.112:0/1363769307 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f010c0fb430 msgr2=0x7f010c0fb840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:19.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.896+0000 7f01118e7640 1 --2- 172.21.15.112:0/1363769307 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f010c0fb430 0x7f010c0fb840 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f00fc0048d0 tx=0x7f00fc009da0 comp rx=0 tx=0).stop 2024-09-16T00:25:19.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.896+0000 7f01118e7640 1 -- 172.21.15.112:0/1363769307 shutdown_connections 2024-09-16T00:25:19.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.896+0000 7f01118e7640 1 --2- 172.21.15.112:0/1363769307 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f010c0fc700 0x7f010c0fcb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:19.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.896+0000 7f01118e7640 1 --2- 172.21.15.112:0/1363769307 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f010c0fb430 0x7f010c0fb840 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:19.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.896+0000 7f01118e7640 1 -- 172.21.15.112:0/1363769307 >> 172.21.15.112:0/1363769307 conn(0x7f010c0f6c10 msgr2=0x7f010c0f9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:19.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.896+0000 7f01118e7640 1 -- 172.21.15.112:0/1363769307 shutdown_connections 2024-09-16T00:25:19.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.896+0000 7f01118e7640 1 -- 172.21.15.112:0/1363769307 wait complete. 2024-09-16T00:25:19.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.897+0000 7f01118e7640 1 Processor -- start 2024-09-16T00:25:19.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.897+0000 7f01118e7640 1 -- start start 2024-09-16T00:25:19.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.897+0000 7f01118e7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f010c0fb430 0x7f010c10fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:19.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.897+0000 7f01118e7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f010c0fc700 0x7f010c1122f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:19.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.897+0000 7f01118e7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010c065c80 con 0x7f010c0fc700 2024-09-16T00:25:19.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.897+0000 7f01118e7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010c065df0 con 0x7f010c0fb430 2024-09-16T00:25:19.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.897+0000 7f010a7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f010c0fc700 0x7f010c1122f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:19.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.898+0000 7f010a7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f010c0fc700 0x7f010c1122f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52342/0 (socket says 172.21.15.112:52342) 2024-09-16T00:25:19.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.898+0000 7f010affd640 1 --2- 172.21.15.112:0/2064365439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f010c0fb430 0x7f010c10fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:19.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.898+0000 7f010a7fc640 1 -- 172.21.15.112:0/2064365439 learned_addr learned my addr 172.21.15.112:0/2064365439 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:25:19.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.898+0000 7f010a7fc640 1 -- 172.21.15.112:0/2064365439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f010c0fb430 msgr2=0x7f010c10fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:19.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.898+0000 7f010a7fc640 1 --2- 172.21.15.112:0/2064365439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f010c0fb430 0x7f010c10fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:19.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.898+0000 7f010a7fc640 1 -- 172.21.15.112:0/2064365439 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00fc005040 con 0x7f010c0fc700 2024-09-16T00:25:19.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.898+0000 7f010a7fc640 1 --2- 172.21.15.112:0/2064365439 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f010c0fc700 0x7f010c1122f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f010000eb70 tx=0x7f010000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:19.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.898+0000 7f00f37fe640 1 -- 172.21.15.112:0/2064365439 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0100007c30 con 0x7f010c0fc700 2024-09-16T00:25:19.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.898+0000 7f01118e7640 1 -- 172.21.15.112:0/2064365439 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f010c112830 con 0x7f010c0fc700 2024-09-16T00:25:19.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.898+0000 7f00f37fe640 1 -- 172.21.15.112:0/2064365439 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f010000f040 con 0x7f010c0fc700 2024-09-16T00:25:19.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.898+0000 7f01118e7640 1 -- 172.21.15.112:0/2064365439 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f010c112dd0 con 0x7f010c0fc700 2024-09-16T00:25:19.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.899+0000 7f00f37fe640 1 -- 172.21.15.112:0/2064365439 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f010000ca60 con 0x7f010c0fc700 2024-09-16T00:25:19.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.900+0000 7f01118e7640 1 -- 172.21.15.112:0/2064365439 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00f8005240 con 0x7f010c0fc700 2024-09-16T00:25:19.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.900+0000 7f00f37fe640 1 -- 172.21.15.112:0/2064365439 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) v1 ==== 90869+0+0 (secure 0 0 0) 0x7f010000cbc0 con 0x7f010c0fc700 2024-09-16T00:25:19.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.901+0000 7f00f37fe640 1 --2- 172.21.15.112:0/2064365439 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f00f406ed50 0x7f00f4071200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:19.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.901+0000 7f00f37fe640 1 -- 172.21.15.112:0/2064365439 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f010008dc30 con 0x7f010c0fc700 2024-09-16T00:25:19.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.901+0000 7f010affd640 1 --2- 172.21.15.112:0/2064365439 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f00f406ed50 0x7f00f4071200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:19.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.901+0000 7f010affd640 1 --2- 172.21.15.112:0/2064365439 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f00f406ed50 0x7f00f4071200 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f00fc000c00 tx=0x7f00fc035040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:19.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:19.906+0000 7f00f37fe640 1 -- 172.21.15.112:0/2064365439 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f010005ae30 con 0x7f010c0fc700 2024-09-16T00:25:20.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:25:19 smithi112 ceph-mon[27869]: pgmap v233: 65 pgs: 65 active+clean; 55 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 4.6 MiB/s wr, 187 op/s 2024-09-16T00:25:20.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.140+0000 7f01118e7640 1 -- 172.21.15.112:0/2064365439 --> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f00f8000bf0 con 0x7f00f406ed50 2024-09-16T00:25:20.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.150+0000 7f00f37fe640 1 -- 172.21.15.112:0/2064365439 <== mgr.14208 v2:172.21.15.112:6800/2480414866 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3542 (secure 0 0 0) 0x7f00f8000bf0 con 0x7f00f406ed50 2024-09-16T00:25:20.151 INFO:teuthology.orchestra.run.smithi112.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T00:25:20.151 INFO:teuthology.orchestra.run.smithi112.stdout:alertmanager.smithi112 smithi112 *:9093,9094 running (5m) 87s ago 7m 24.2M - 0.25.0 c8568f914cd2 89acc17eab5c 2024-09-16T00:25:20.151 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi112 smithi112 running (7m) 87s ago 7m 6891k - 17.2.7-1633-g6d025bc2 33ed8dba3996 c94220ed2776 2024-09-16T00:25:20.151 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi169 smithi169 running (6m) 93s ago 6m 6899k - 17.2.7-1633-g6d025bc2 33ed8dba3996 d0d3726280a2 2024-09-16T00:25:20.151 INFO:teuthology.orchestra.run.smithi112.stdout:grafana.smithi112 smithi112 *:3000 running (5m) 87s ago 6m 84.6M - 9.4.7 954c08fa6188 ffeb9c3bdf66 2024-09-16T00:25:20.151 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.hxfwkh smithi112 running (107s) 87s ago 106s 15.6M - 17.2.7-1633-g6d025bc2 33ed8dba3996 cf92d9461fc6 2024-09-16T00:25:20.151 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.ithwzj smithi112 running (114s) 87s ago 113s 11.8M - 17.2.7-1633-g6d025bc2 33ed8dba3996 02786c8a83c7 2024-09-16T00:25:20.152 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.gawwgw smithi169 running (111s) 93s ago 111s 14.4M - 17.2.7-1633-g6d025bc2 33ed8dba3996 dcf3a4dfa879 2024-09-16T00:25:20.152 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.lgeqrj smithi169 running (104s) 93s ago 104s 13.8M - 17.2.7-1633-g6d025bc2 33ed8dba3996 445c14e55831 2024-09-16T00:25:20.152 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi112.xrkkbc smithi112 *:9283,8765,8443 running (8m) 87s ago 8m 449M - 17.2.7-1633-g6d025bc2 33ed8dba3996 b3d3adb94a85 2024-09-16T00:25:20.152 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi169.mtrobk smithi169 *:8443,9283 running (6m) 93s ago 6m 405M - 17.2.7-1633-g6d025bc2 33ed8dba3996 1e67dbe21770 2024-09-16T00:25:20.152 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi112 smithi112 running (8m) 87s ago 8m 43.3M 2048M 17.2.7-1633-g6d025bc2 33ed8dba3996 ddaaced54bf8 2024-09-16T00:25:20.152 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi169 smithi169 running (6m) 93s ago 6m 35.4M 2048M 17.2.7-1633-g6d025bc2 33ed8dba3996 6c2289bc3e11 2024-09-16T00:25:20.152 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi112 smithi112 *:9100 running (7m) 87s ago 7m 15.9M - 1.5.0 0da6a335fe13 89c6f32288bc 2024-09-16T00:25:20.152 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi169 smithi169 *:9100 running (6m) 93s ago 6m 15.6M - 1.5.0 0da6a335fe13 5d6fe1dc6fa9 2024-09-16T00:25:20.152 INFO:teuthology.orchestra.run.smithi112.stdout:osd.0 smithi112 running (5m) 87s ago 5m 46.0M 4288M 17.2.7-1633-g6d025bc2 33ed8dba3996 f6b2e146ffdf 2024-09-16T00:25:20.152 INFO:teuthology.orchestra.run.smithi112.stdout:osd.1 smithi112 running (5m) 87s ago 5m 47.9M 4288M 17.2.7-1633-g6d025bc2 33ed8dba3996 e9bee87bba0f 2024-09-16T00:25:20.152 INFO:teuthology.orchestra.run.smithi112.stdout:osd.2 smithi112 running (4m) 87s ago 4m 43.3M 4288M 17.2.7-1633-g6d025bc2 33ed8dba3996 100ec4e4852a 2024-09-16T00:25:20.152 INFO:teuthology.orchestra.run.smithi112.stdout:osd.3 smithi169 running (4m) 93s ago 4m 46.3M 5312M 17.2.7-1633-g6d025bc2 33ed8dba3996 f88463e19f23 2024-09-16T00:25:20.152 INFO:teuthology.orchestra.run.smithi112.stdout:osd.4 smithi169 running (3m) 93s ago 3m 42.7M 5312M 17.2.7-1633-g6d025bc2 33ed8dba3996 d3e4cdb016b9 2024-09-16T00:25:20.152 INFO:teuthology.orchestra.run.smithi112.stdout:osd.5 smithi169 running (3m) 93s ago 3m 42.2M 5312M 17.2.7-1633-g6d025bc2 33ed8dba3996 8e7d708d74eb 2024-09-16T00:25:20.153 INFO:teuthology.orchestra.run.smithi112.stdout:prometheus.smithi112 smithi112 *:9095 running (5m) 87s ago 6m 55.1M - 2.43.0 a07b618ecd1d 7f5a86d91011 2024-09-16T00:25:20.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.154+0000 7f01118e7640 1 -- 172.21.15.112:0/2064365439 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f00f406ed50 msgr2=0x7f00f4071200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:20.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.154+0000 7f01118e7640 1 --2- 172.21.15.112:0/2064365439 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f00f406ed50 0x7f00f4071200 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f00fc000c00 tx=0x7f00fc035040 comp rx=0 tx=0).stop 2024-09-16T00:25:20.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.154+0000 7f01118e7640 1 -- 172.21.15.112:0/2064365439 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f010c0fc700 msgr2=0x7f010c1122f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:20.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.154+0000 7f01118e7640 1 --2- 172.21.15.112:0/2064365439 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f010c0fc700 0x7f010c1122f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f010000eb70 tx=0x7f010000b040 comp rx=0 tx=0).stop 2024-09-16T00:25:20.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.154+0000 7f01118e7640 1 -- 172.21.15.112:0/2064365439 shutdown_connections 2024-09-16T00:25:20.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.154+0000 7f01118e7640 1 --2- 172.21.15.112:0/2064365439 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f00f406ed50 0x7f00f4071200 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:20.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.154+0000 7f01118e7640 1 --2- 172.21.15.112:0/2064365439 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f010c0fc700 0x7f010c1122f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:20.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.154+0000 7f01118e7640 1 --2- 172.21.15.112:0/2064365439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f010c0fb430 0x7f010c10fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:20.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.154+0000 7f01118e7640 1 -- 172.21.15.112:0/2064365439 >> 172.21.15.112:0/2064365439 conn(0x7f010c0f6c10 msgr2=0x7f010c0f83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:20.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.154+0000 7f01118e7640 1 -- 172.21.15.112:0/2064365439 shutdown_connections 2024-09-16T00:25:20.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.154+0000 7f01118e7640 1 -- 172.21.15.112:0/2064365439 wait complete. 2024-09-16T00:25:20.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.254+0000 7fb0f3684640 1 -- 172.21.15.112:0/110351360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0ec0ff680 msgr2=0x7fb0ec101a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:20.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.254+0000 7fb0f3684640 1 --2- 172.21.15.112:0/110351360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0ec0ff680 0x7fb0ec101a60 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc0048d0 tx=0x7fb0dc009da0 comp rx=0 tx=0).stop 2024-09-16T00:25:20.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.254+0000 7fb0f3684640 1 -- 172.21.15.112:0/110351360 shutdown_connections 2024-09-16T00:25:20.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.254+0000 7fb0f3684640 1 --2- 172.21.15.112:0/110351360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0ec0ff680 0x7fb0ec101a60 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:20.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.254+0000 7fb0f3684640 1 --2- 172.21.15.112:0/110351360 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0ec065c40 0x7fb0ec0ff140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:20.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.254+0000 7fb0f3684640 1 -- 172.21.15.112:0/110351360 >> 172.21.15.112:0/110351360 conn(0x7fb0ec0f6c10 msgr2=0x7fb0ec0f9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:20.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.254+0000 7fb0f3684640 1 -- 172.21.15.112:0/110351360 shutdown_connections 2024-09-16T00:25:20.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.254+0000 7fb0f3684640 1 -- 172.21.15.112:0/110351360 wait complete. 2024-09-16T00:25:20.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.255+0000 7fb0f3684640 1 Processor -- start 2024-09-16T00:25:20.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.255+0000 7fb0f3684640 1 -- start start 2024-09-16T00:25:20.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.255+0000 7fb0f3684640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0ec065c40 0x7fb0ec0fcb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:20.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.255+0000 7fb0f3684640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0ec0ff680 0x7fb0ec0fb040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:20.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.255+0000 7fb0f3684640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ec0fb5d0 con 0x7fb0ec065c40 2024-09-16T00:25:20.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.255+0000 7fb0f3684640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ec0fb710 con 0x7fb0ec0ff680 2024-09-16T00:25:20.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.255+0000 7fb0f1420640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0ec065c40 0x7fb0ec0fcb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:20.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.255+0000 7fb0f1420640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0ec065c40 0x7fb0ec0fcb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52356/0 (socket says 172.21.15.112:52356) 2024-09-16T00:25:20.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.255+0000 7fb0f1420640 1 -- 172.21.15.112:0/845739553 learned_addr learned my addr 172.21.15.112:0/845739553 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:25:20.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.255+0000 7fb0f0c1f640 1 --2- 172.21.15.112:0/845739553 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0ec0ff680 0x7fb0ec0fb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:20.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.255+0000 7fb0f1420640 1 -- 172.21.15.112:0/845739553 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0ec0ff680 msgr2=0x7fb0ec0fb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:20.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.255+0000 7fb0f1420640 1 --2- 172.21.15.112:0/845739553 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0ec0ff680 0x7fb0ec0fb040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:20.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.255+0000 7fb0f1420640 1 -- 172.21.15.112:0/845739553 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0dc005040 con 0x7fb0ec065c40 2024-09-16T00:25:20.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.255+0000 7fb0f1420640 1 --2- 172.21.15.112:0/845739553 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0ec065c40 0x7fb0ec0fcb50 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb0e800eb70 tx=0x7fb0e800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:20.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.256+0000 7fb0e1ffb640 1 -- 172.21.15.112:0/845739553 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb0e8007c30 con 0x7fb0ec065c40 2024-09-16T00:25:20.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.256+0000 7fb0e1ffb640 1 -- 172.21.15.112:0/845739553 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fb0e800f040 con 0x7fb0ec065c40 2024-09-16T00:25:20.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.256+0000 7fb0f3684640 1 -- 172.21.15.112:0/845739553 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0ec0fb9f0 con 0x7fb0ec065c40 2024-09-16T00:25:20.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.256+0000 7fb0e1ffb640 1 -- 172.21.15.112:0/845739553 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb0e800caa0 con 0x7fb0ec065c40 2024-09-16T00:25:20.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.256+0000 7fb0f3684640 1 -- 172.21.15.112:0/845739553 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0ec1150c0 con 0x7fb0ec065c40 2024-09-16T00:25:20.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.258+0000 7fb0f3684640 1 -- 172.21.15.112:0/845739553 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0ec0fa890 con 0x7fb0ec065c40 2024-09-16T00:25:20.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.258+0000 7fb0e1ffb640 1 -- 172.21.15.112:0/845739553 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) v1 ==== 90869+0+0 (secure 0 0 0) 0x7fb0e800cd30 con 0x7fb0ec065c40 2024-09-16T00:25:20.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.258+0000 7fb0e1ffb640 1 --2- 172.21.15.112:0/845739553 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fb0d806ed60 0x7fb0d8071210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:20.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.259+0000 7fb0e1ffb640 1 -- 172.21.15.112:0/845739553 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fb0e808e340 con 0x7fb0ec065c40 2024-09-16T00:25:20.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.259+0000 7fb0f0c1f640 1 --2- 172.21.15.112:0/845739553 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fb0d806ed60 0x7fb0d8071210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:20.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.259+0000 7fb0f0c1f640 1 --2- 172.21.15.112:0/845739553 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fb0d806ed60 0x7fb0d8071210 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc004540 tx=0x7fb0dc035040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:20.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.264+0000 7fb0e1ffb640 1 -- 172.21.15.112:0/845739553 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fb0e805b490 con 0x7fb0ec065c40 2024-09-16T00:25:20.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:25:19 smithi169 ceph-mon[34653]: pgmap v233: 65 pgs: 65 active+clean; 55 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 4.6 MiB/s wr, 187 op/s 2024-09-16T00:25:20.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.552+0000 7fb0f3684640 1 -- 172.21.15.112:0/845739553 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb0ec115440 con 0x7fb0ec065c40 2024-09-16T00:25:20.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.552+0000 7fb0e1ffb640 1 -- 172.21.15.112:0/845739553 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+643 (secure 0 0 0) 0x7fb0e805ae30 con 0x7fb0ec065c40 2024-09-16T00:25:20.553 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:25:20.553 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": { 2024-09-16T00:25:20.553 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 17.2.7-1633-g6d025bc2 (6d025bc26f3176c910707f8cb4c8d8e076640615) quincy (stable)": 2 2024-09-16T00:25:20.553 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:25:20.553 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": { 2024-09-16T00:25:20.553 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 17.2.7-1633-g6d025bc2 (6d025bc26f3176c910707f8cb4c8d8e076640615) quincy (stable)": 2 2024-09-16T00:25:20.553 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:25:20.554 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": { 2024-09-16T00:25:20.554 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 17.2.7-1633-g6d025bc2 (6d025bc26f3176c910707f8cb4c8d8e076640615) quincy (stable)": 6 2024-09-16T00:25:20.554 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:25:20.554 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": { 2024-09-16T00:25:20.554 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 17.2.7-1633-g6d025bc2 (6d025bc26f3176c910707f8cb4c8d8e076640615) quincy (stable)": 4 2024-09-16T00:25:20.554 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:25:20.554 INFO:teuthology.orchestra.run.smithi112.stdout: "overall": { 2024-09-16T00:25:20.554 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 17.2.7-1633-g6d025bc2 (6d025bc26f3176c910707f8cb4c8d8e076640615) quincy (stable)": 14 2024-09-16T00:25:20.554 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-16T00:25:20.554 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:25:20.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.554+0000 7fb0f3684640 1 -- 172.21.15.112:0/845739553 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fb0d806ed60 msgr2=0x7fb0d8071210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:20.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.554+0000 7fb0f3684640 1 --2- 172.21.15.112:0/845739553 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fb0d806ed60 0x7fb0d8071210 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc004540 tx=0x7fb0dc035040 comp rx=0 tx=0).stop 2024-09-16T00:25:20.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.555+0000 7fb0f3684640 1 -- 172.21.15.112:0/845739553 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0ec065c40 msgr2=0x7fb0ec0fcb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:20.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.555+0000 7fb0f3684640 1 --2- 172.21.15.112:0/845739553 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0ec065c40 0x7fb0ec0fcb50 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb0e800eb70 tx=0x7fb0e800b040 comp rx=0 tx=0).stop 2024-09-16T00:25:20.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.555+0000 7fb0f3684640 1 -- 172.21.15.112:0/845739553 shutdown_connections 2024-09-16T00:25:20.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.555+0000 7fb0f3684640 1 --2- 172.21.15.112:0/845739553 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fb0d806ed60 0x7fb0d8071210 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:20.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.555+0000 7fb0f3684640 1 --2- 172.21.15.112:0/845739553 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0ec0ff680 0x7fb0ec0fb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:20.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.555+0000 7fb0f3684640 1 --2- 172.21.15.112:0/845739553 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0ec065c40 0x7fb0ec0fcb50 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:20.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.555+0000 7fb0f3684640 1 -- 172.21.15.112:0/845739553 >> 172.21.15.112:0/845739553 conn(0x7fb0ec0f6c10 msgr2=0x7fb0ec0f6fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:20.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.555+0000 7fb0f3684640 1 -- 172.21.15.112:0/845739553 shutdown_connections 2024-09-16T00:25:20.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.555+0000 7fb0f3684640 1 -- 172.21.15.112:0/845739553 wait complete. 2024-09-16T00:25:20.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.650+0000 7fc0f900d640 1 -- 172.21.15.112:0/3967707842 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0f40fb440 msgr2=0x7fc0f40fb850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:20.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.650+0000 7fc0f900d640 1 --2- 172.21.15.112:0/3967707842 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0f40fb440 0x7fc0f40fb850 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc0e40048d0 tx=0x7fc0e4009da0 comp rx=0 tx=0).stop 2024-09-16T00:25:20.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.650+0000 7fc0f900d640 1 -- 172.21.15.112:0/3967707842 shutdown_connections 2024-09-16T00:25:20.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.650+0000 7fc0f900d640 1 --2- 172.21.15.112:0/3967707842 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0f40fc710 0x7fc0f40fcb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:20.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.650+0000 7fc0f900d640 1 --2- 172.21.15.112:0/3967707842 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0f40fb440 0x7fc0f40fb850 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:20.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.650+0000 7fc0f900d640 1 -- 172.21.15.112:0/3967707842 >> 172.21.15.112:0/3967707842 conn(0x7fc0f40f6c20 msgr2=0x7fc0f40f9030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:20.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.650+0000 7fc0f900d640 1 -- 172.21.15.112:0/3967707842 shutdown_connections 2024-09-16T00:25:20.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.650+0000 7fc0f900d640 1 -- 172.21.15.112:0/3967707842 wait complete. 2024-09-16T00:25:20.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.651+0000 7fc0f900d640 1 Processor -- start 2024-09-16T00:25:20.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.651+0000 7fc0f900d640 1 -- start start 2024-09-16T00:25:20.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.651+0000 7fc0f900d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0f40fb440 0x7fc0f410d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:20.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.651+0000 7fc0f900d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0f40fc710 0x7fc0f410d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:20.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.651+0000 7fc0f900d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f410dc80 con 0x7fc0f40fc710 2024-09-16T00:25:20.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.651+0000 7fc0f900d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f410e4a0 con 0x7fc0f40fb440 2024-09-16T00:25:20.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.651+0000 7fc0f259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0f40fc710 0x7fc0f410d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:20.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.651+0000 7fc0f259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0f40fc710 0x7fc0f410d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52362/0 (socket says 172.21.15.112:52362) 2024-09-16T00:25:20.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.651+0000 7fc0f259c640 1 -- 172.21.15.112:0/136051733 learned_addr learned my addr 172.21.15.112:0/136051733 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:25:20.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.651+0000 7fc0f2d9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0f40fb440 0x7fc0f410d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:20.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.652+0000 7fc0f259c640 1 -- 172.21.15.112:0/136051733 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0f40fb440 msgr2=0x7fc0f410d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:20.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.652+0000 7fc0f259c640 1 --2- 172.21.15.112:0/136051733 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0f40fb440 0x7fc0f410d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:20.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.652+0000 7fc0f259c640 1 -- 172.21.15.112:0/136051733 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0e4005040 con 0x7fc0f40fc710 2024-09-16T00:25:20.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.652+0000 7fc0f259c640 1 --2- 172.21.15.112:0/136051733 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0f40fc710 0x7fc0f410d630 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc0e800eb70 tx=0x7fc0e800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:20.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.652+0000 7fc0db7fe640 1 -- 172.21.15.112:0/136051733 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0e8007c30 con 0x7fc0f40fc710 2024-09-16T00:25:20.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.652+0000 7fc0f900d640 1 -- 172.21.15.112:0/136051733 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0f410e6d0 con 0x7fc0f40fc710 2024-09-16T00:25:20.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.652+0000 7fc0db7fe640 1 -- 172.21.15.112:0/136051733 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fc0e800f040 con 0x7fc0f40fc710 2024-09-16T00:25:20.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.652+0000 7fc0f900d640 1 -- 172.21.15.112:0/136051733 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0f410ec70 con 0x7fc0f40fc710 2024-09-16T00:25:20.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.653+0000 7fc0db7fe640 1 -- 172.21.15.112:0/136051733 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0e800ca60 con 0x7fc0f40fc710 2024-09-16T00:25:20.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.653+0000 7fc0f900d640 1 -- 172.21.15.112:0/136051733 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0f40fa8a0 con 0x7fc0f40fc710 2024-09-16T00:25:20.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.654+0000 7fc0db7fe640 1 -- 172.21.15.112:0/136051733 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) v1 ==== 90869+0+0 (secure 0 0 0) 0x7fc0e800cbc0 con 0x7fc0f40fc710 2024-09-16T00:25:20.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.655+0000 7fc0db7fe640 1 --2- 172.21.15.112:0/136051733 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fc0dc06ed60 0x7fc0dc071210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:20.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.655+0000 7fc0db7fe640 1 -- 172.21.15.112:0/136051733 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fc0e808ddd0 con 0x7fc0f40fc710 2024-09-16T00:25:20.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.655+0000 7fc0f2d9d640 1 --2- 172.21.15.112:0/136051733 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fc0dc06ed60 0x7fc0dc071210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:20.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.655+0000 7fc0f2d9d640 1 --2- 172.21.15.112:0/136051733 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fc0dc06ed60 0x7fc0dc071210 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4000c00 tx=0x7fc0e4035040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:20.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.660+0000 7fc0db7fe640 1 -- 172.21.15.112:0/136051733 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fc0e805b090 con 0x7fc0f40fc710 2024-09-16T00:25:20.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:25:20 smithi112 ceph-mon[27869]: from='client.14560 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:25:20.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:25:20 smithi112 ceph-mon[27869]: from='client.14564 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:25:20.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:25:20 smithi112 ceph-mon[27869]: from='client.14568 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:25:20.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:25:20 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/845739553' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:25:20.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.884+0000 7fc0f900d640 1 -- 172.21.15.112:0/136051733 --> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc0f4103fd0 con 0x7fc0dc06ed60 2024-09-16T00:25:20.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.885+0000 7fc0db7fe640 1 -- 172.21.15.112:0/136051733 <== mgr.14208 v2:172.21.15.112:6800/2480414866 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7fc0f4103fd0 con 0x7fc0dc06ed60 2024-09-16T00:25:20.885 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:25:20.885 INFO:teuthology.orchestra.run.smithi112.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80", 2024-09-16T00:25:20.885 INFO:teuthology.orchestra.run.smithi112.stdout: "in_progress": true, 2024-09-16T00:25:20.886 INFO:teuthology.orchestra.run.smithi112.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-09-16T00:25:20.886 INFO:teuthology.orchestra.run.smithi112.stdout: "services_complete": [], 2024-09-16T00:25:20.886 INFO:teuthology.orchestra.run.smithi112.stdout: "progress": "0/2 daemons upgraded", 2024-09-16T00:25:20.886 INFO:teuthology.orchestra.run.smithi112.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 image on host smithi169", 2024-09-16T00:25:20.886 INFO:teuthology.orchestra.run.smithi112.stdout: "is_paused": false 2024-09-16T00:25:20.886 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:25:20.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.887+0000 7fc0f900d640 1 -- 172.21.15.112:0/136051733 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fc0dc06ed60 msgr2=0x7fc0dc071210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:20.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.887+0000 7fc0f900d640 1 --2- 172.21.15.112:0/136051733 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fc0dc06ed60 0x7fc0dc071210 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4000c00 tx=0x7fc0e4035040 comp rx=0 tx=0).stop 2024-09-16T00:25:20.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.887+0000 7fc0f900d640 1 -- 172.21.15.112:0/136051733 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0f40fc710 msgr2=0x7fc0f410d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:20.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.887+0000 7fc0f900d640 1 --2- 172.21.15.112:0/136051733 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0f40fc710 0x7fc0f410d630 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc0e800eb70 tx=0x7fc0e800b040 comp rx=0 tx=0).stop 2024-09-16T00:25:20.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.887+0000 7fc0f900d640 1 -- 172.21.15.112:0/136051733 shutdown_connections 2024-09-16T00:25:20.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.887+0000 7fc0f900d640 1 --2- 172.21.15.112:0/136051733 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fc0dc06ed60 0x7fc0dc071210 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:20.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.887+0000 7fc0f900d640 1 --2- 172.21.15.112:0/136051733 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0f40fc710 0x7fc0f410d630 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:20.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.887+0000 7fc0f900d640 1 --2- 172.21.15.112:0/136051733 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0f40fb440 0x7fc0f410d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:20.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.887+0000 7fc0f900d640 1 -- 172.21.15.112:0/136051733 >> 172.21.15.112:0/136051733 conn(0x7fc0f40f6c20 msgr2=0x7fc0f4066120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:20.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.887+0000 7fc0f900d640 1 -- 172.21.15.112:0/136051733 shutdown_connections 2024-09-16T00:25:20.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:20.887+0000 7fc0f900d640 1 -- 172.21.15.112:0/136051733 wait complete. 2024-09-16T00:25:21.123 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:25:20 smithi169 ceph-mon[34653]: from='client.14560 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:25:21.123 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:25:20 smithi169 ceph-mon[34653]: from='client.14564 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:25:21.123 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:25:20 smithi169 ceph-mon[34653]: from='client.14568 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:25:21.124 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:25:20 smithi169 ceph-mon[34653]: from='client.? 172.21.15.112:0/845739553' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:25:22.066 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:25:21 smithi112 ceph-mon[27869]: pgmap v234: 65 pgs: 65 active+clean; 60 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 3.6 MiB/s wr, 181 op/s 2024-09-16T00:25:22.067 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:25:21 smithi112 ceph-mon[27869]: from='client.14576 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:25:22.067 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:25:21 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:25:22.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:25:21 smithi169 ceph-mon[34653]: pgmap v234: 65 pgs: 65 active+clean; 60 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 3.6 MiB/s wr, 181 op/s 2024-09-16T00:25:22.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:25:21 smithi169 ceph-mon[34653]: from='client.14576 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:25:22.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:25:21 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:25:23.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:25:23 smithi169 ceph-mon[34653]: Upgrade: Updating mgr.smithi169.mtrobk 2024-09-16T00:25:23.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:25:23 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:25:23.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:25:23 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi169.mtrobk", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T00:25:23.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:25:23 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T00:25:23.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:25:23 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:23.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:25:23 smithi169 ceph-mon[34653]: Deploying daemon mgr.smithi169.mtrobk on smithi169 2024-09-16T00:25:23.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:25:23 smithi112 ceph-mon[27869]: Upgrade: Updating mgr.smithi169.mtrobk 2024-09-16T00:25:23.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:25:23 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:25:23.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:25:23 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi169.mtrobk", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T00:25:23.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:25:23 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T00:25:23.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:25:23 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:23.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:25:23 smithi112 ceph-mon[27869]: Deploying daemon mgr.smithi169.mtrobk on smithi169 2024-09-16T00:25:24.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:25:24 smithi169 ceph-mon[34653]: pgmap v235: 65 pgs: 65 active+clean; 91 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 3.4 MiB/s rd, 5.8 MiB/s wr, 234 op/s 2024-09-16T00:25:24.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:25:24 smithi112 ceph-mon[27869]: pgmap v235: 65 pgs: 65 active+clean; 91 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 3.4 MiB/s rd, 5.8 MiB/s wr, 234 op/s 2024-09-16T00:25:26.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:25:26 smithi169 ceph-mon[34653]: pgmap v236: 65 pgs: 65 active+clean; 97 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.9 MiB/s rd, 5.3 MiB/s wr, 227 op/s 2024-09-16T00:25:26.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:25:26 smithi112 ceph-mon[27869]: pgmap v236: 65 pgs: 65 active+clean; 97 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.9 MiB/s rd, 5.3 MiB/s wr, 227 op/s 2024-09-16T00:25:28.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:25:28 smithi169 ceph-mon[34653]: pgmap v237: 65 pgs: 65 active+clean; 100 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.9 MiB/s rd, 5.1 MiB/s wr, 199 op/s 2024-09-16T00:25:28.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:25:28 smithi112 ceph-mon[27869]: pgmap v237: 65 pgs: 65 active+clean; 100 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.9 MiB/s rd, 5.1 MiB/s wr, 199 op/s 2024-09-16T00:25:30.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:25:30 smithi169 ceph-mon[34653]: pgmap v238: 65 pgs: 65 active+clean; 112 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 3.7 MiB/s rd, 6.1 MiB/s wr, 254 op/s 2024-09-16T00:25:30.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:25:30 smithi112 ceph-mon[27869]: pgmap v238: 65 pgs: 65 active+clean; 112 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 3.7 MiB/s rd, 6.1 MiB/s wr, 254 op/s 2024-09-16T00:25:32.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:25:32 smithi169 ceph-mon[34653]: pgmap v239: 65 pgs: 65 active+clean; 115 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 3.3 MiB/s rd, 5.1 MiB/s wr, 219 op/s 2024-09-16T00:25:32.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:25:32 smithi112 ceph-mon[27869]: pgmap v239: 65 pgs: 65 active+clean; 115 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 3.3 MiB/s rd, 5.1 MiB/s wr, 219 op/s 2024-09-16T00:25:34.743 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:25:34 smithi169 ceph-mon[34653]: pgmap v240: 65 pgs: 65 active+clean; 131 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 4.0 MiB/s rd, 6.1 MiB/s wr, 283 op/s 2024-09-16T00:25:34.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:25:34 smithi112 ceph-mon[27869]: pgmap v240: 65 pgs: 65 active+clean; 131 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 4.0 MiB/s rd, 6.1 MiB/s wr, 283 op/s 2024-09-16T00:25:36.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:25:36 smithi169 ceph-mon[34653]: pgmap v241: 65 pgs: 65 active+clean; 144 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.8 MiB/s rd, 4.5 MiB/s wr, 288 op/s 2024-09-16T00:25:36.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:25:36 smithi112 ceph-mon[27869]: pgmap v241: 65 pgs: 65 active+clean; 144 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.8 MiB/s rd, 4.5 MiB/s wr, 288 op/s 2024-09-16T00:25:37.667 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:25:37 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:25:37.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:25:37 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:25:38.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:25:38 smithi169 ceph-mon[34653]: pgmap v242: 65 pgs: 65 active+clean; 147 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.7 MiB/s rd, 4.2 MiB/s wr, 259 op/s 2024-09-16T00:25:38.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:25:38 smithi112 ceph-mon[27869]: pgmap v242: 65 pgs: 65 active+clean; 147 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.7 MiB/s rd, 4.2 MiB/s wr, 259 op/s 2024-09-16T00:25:40.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:25:40 smithi169 ceph-mon[34653]: pgmap v243: 65 pgs: 65 active+clean; 174 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 3.9 MiB/s rd, 6.3 MiB/s wr, 385 op/s 2024-09-16T00:25:40.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:25:40 smithi112 ceph-mon[27869]: pgmap v243: 65 pgs: 65 active+clean; 174 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 3.9 MiB/s rd, 6.3 MiB/s wr, 385 op/s 2024-09-16T00:25:42.473 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:25:42 smithi169 ceph-mon[34653]: pgmap v244: 65 pgs: 65 active+clean; 175 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 3.0 MiB/s rd, 5.4 MiB/s wr, 355 op/s 2024-09-16T00:25:42.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:25:42 smithi112 ceph-mon[27869]: pgmap v244: 65 pgs: 65 active+clean; 175 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 3.0 MiB/s rd, 5.4 MiB/s wr, 355 op/s 2024-09-16T00:25:44.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:25:44 smithi112 ceph-mon[27869]: pgmap v245: 65 pgs: 65 active+clean; 199 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 4.1 MiB/s rd, 7.2 MiB/s wr, 503 op/s 2024-09-16T00:25:44.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:25:44 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:25:44.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:25:44 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:25:44.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:25:44 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:25:44.478 INFO:tasks.workunit.client.0.smithi112.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-09-16T00:25:44.482 INFO:tasks.workunit.client.0.smithi112.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-09-16T00:25:44.482 INFO:tasks.workunit.client.0.smithi112.stderr:+ make 2024-09-16T00:25:44.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:25:44 smithi169 ceph-mon[34653]: pgmap v245: 65 pgs: 65 active+clean; 199 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 4.1 MiB/s rd, 7.2 MiB/s wr, 503 op/s 2024-09-16T00:25:44.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:25:44 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:25:44.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:25:44 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:25:44.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:25:44 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:25:44.533 INFO:tasks.workunit.client.0.smithi112.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-09-16T00:25:44.954 INFO:tasks.workunit.client.1.smithi169.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-09-16T00:25:44.963 INFO:tasks.workunit.client.1.smithi169.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-09-16T00:25:44.963 INFO:tasks.workunit.client.1.smithi169.stderr:+ make 2024-09-16T00:25:45.054 INFO:tasks.workunit.client.1.smithi169.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-09-16T00:25:45.429 INFO:tasks.workunit.client.0.smithi112.stderr:++ readlink -f fsstress 2024-09-16T00:25:45.430 INFO:tasks.workunit.client.0.smithi112.stderr:+ BIN=/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-09-16T00:25:45.431 INFO:tasks.workunit.client.0.smithi112.stderr:+ popd 2024-09-16T00:25:45.431 INFO:tasks.workunit.client.0.smithi112.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-09-16T00:25:45.432 INFO:tasks.workunit.client.0.smithi112.stderr:+ popd 2024-09-16T00:25:45.432 INFO:tasks.workunit.client.0.smithi112.stdout:~/cephtest/mnt.0/client.0/tmp 2024-09-16T00:25:45.432 INFO:tasks.workunit.client.0.smithi112.stderr:++ mktemp -d -p . 2024-09-16T00:25:45.434 INFO:tasks.workunit.client.0.smithi112.stderr:+ T=./tmp.ZUfysPtAcL 2024-09-16T00:25:45.434 INFO:tasks.workunit.client.0.smithi112.stderr:+ /home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.ZUfysPtAcL -l 1 -n 1000 -p 10 -v 2024-09-16T00:25:45.437 INFO:tasks.workunit.client.0.smithi112.stdout:seed = 1726076130 2024-09-16T00:25:45.440 INFO:tasks.workunit.client.0.smithi112.stdout:0/0: chown . 283 1 2024-09-16T00:25:45.440 INFO:tasks.workunit.client.0.smithi112.stdout:0/1: dread - no filename 2024-09-16T00:25:45.441 INFO:tasks.workunit.client.0.smithi112.stdout:0/2: creat f0 x:0 0 0 2024-09-16T00:25:45.443 INFO:tasks.workunit.client.0.smithi112.stdout:1/0: mkdir d0 0 2024-09-16T00:25:45.443 INFO:tasks.workunit.client.0.smithi112.stdout:1/1: truncate - no filename 2024-09-16T00:25:45.445 INFO:tasks.workunit.client.0.smithi112.stdout:1/2: getdents d0 0 2024-09-16T00:25:45.445 INFO:tasks.workunit.client.0.smithi112.stdout:1/3: unlink - no file 2024-09-16T00:25:45.446 INFO:tasks.workunit.client.0.smithi112.stdout:4/0: fdatasync - no filename 2024-09-16T00:25:45.446 INFO:tasks.workunit.client.0.smithi112.stdout:4/1: readlink - no filename 2024-09-16T00:25:45.447 INFO:tasks.workunit.client.0.smithi112.stdout:4/2: readlink - no filename 2024-09-16T00:25:45.447 INFO:tasks.workunit.client.0.smithi112.stdout:4/3: dread - no filename 2024-09-16T00:25:45.447 INFO:tasks.workunit.client.0.smithi112.stdout:4/4: dread - no filename 2024-09-16T00:25:45.447 INFO:tasks.workunit.client.0.smithi112.stdout:4/5: dwrite - no filename 2024-09-16T00:25:45.447 INFO:tasks.workunit.client.0.smithi112.stdout:4/6: read - no filename 2024-09-16T00:25:45.447 INFO:tasks.workunit.client.0.smithi112.stdout:4/7: dwrite - no filename 2024-09-16T00:25:45.447 INFO:tasks.workunit.client.0.smithi112.stdout:4/8: dwrite - no filename 2024-09-16T00:25:45.447 INFO:tasks.workunit.client.0.smithi112.stdout:4/9: dwrite - no filename 2024-09-16T00:25:45.447 INFO:tasks.workunit.client.0.smithi112.stdout:4/10: chown . 3322 1 2024-09-16T00:25:45.447 INFO:tasks.workunit.client.0.smithi112.stdout:3/0: symlink l0 0 2024-09-16T00:25:45.447 INFO:tasks.workunit.client.0.smithi112.stdout:3/1: read - no filename 2024-09-16T00:25:45.447 INFO:tasks.workunit.client.0.smithi112.stdout:3/2: write - no filename 2024-09-16T00:25:45.448 INFO:tasks.workunit.client.0.smithi112.stdout:1/4: creat d0/f1 x:0 0 0 2024-09-16T00:25:45.448 INFO:tasks.workunit.client.0.smithi112.stdout:5/0: truncate - no filename 2024-09-16T00:25:45.448 INFO:tasks.workunit.client.0.smithi112.stdout:5/1: fsync - no filename 2024-09-16T00:25:45.448 INFO:tasks.workunit.client.0.smithi112.stdout:5/2: read - no filename 2024-09-16T00:25:45.449 INFO:tasks.workunit.client.0.smithi112.stdout:5/3: fdatasync - no filename 2024-09-16T00:25:45.449 INFO:tasks.workunit.client.0.smithi112.stdout:5/4: chown . 7693 1 2024-09-16T00:25:45.449 INFO:tasks.workunit.client.0.smithi112.stdout:3/3: symlink l1 0 2024-09-16T00:25:45.449 INFO:tasks.workunit.client.0.smithi112.stdout:3/4: write - no filename 2024-09-16T00:25:45.449 INFO:tasks.workunit.client.0.smithi112.stdout:3/5: readlink l1 0 2024-09-16T00:25:45.449 INFO:tasks.workunit.client.0.smithi112.stdout:3/6: read - no filename 2024-09-16T00:25:45.449 INFO:tasks.workunit.client.0.smithi112.stdout:3/7: chown l1 87786804 1 2024-09-16T00:25:45.449 INFO:tasks.workunit.client.0.smithi112.stdout:3/8: dread - no filename 2024-09-16T00:25:45.450 INFO:tasks.workunit.client.0.smithi112.stdout:3/9: dread - no filename 2024-09-16T00:25:45.451 INFO:tasks.workunit.client.0.smithi112.stdout:4/11: symlink l0 0 2024-09-16T00:25:45.451 INFO:tasks.workunit.client.0.smithi112.stdout:4/12: dread - no filename 2024-09-16T00:25:45.451 INFO:tasks.workunit.client.0.smithi112.stdout:3/10: chown l1 130 1 2024-09-16T00:25:45.451 INFO:tasks.workunit.client.0.smithi112.stdout:6/0: readlink - no filename 2024-09-16T00:25:45.451 INFO:tasks.workunit.client.0.smithi112.stdout:6/1: truncate - no filename 2024-09-16T00:25:45.452 INFO:tasks.workunit.client.0.smithi112.stdout:1/5: rename d0/f1 to d0/f2 0 2024-09-16T00:25:45.453 INFO:tasks.workunit.client.0.smithi112.stdout:3/11: mkdir d2 0 2024-09-16T00:25:45.453 INFO:tasks.workunit.client.0.smithi112.stdout:3/12: write - no filename 2024-09-16T00:25:45.453 INFO:tasks.workunit.client.0.smithi112.stdout:3/13: read - no filename 2024-09-16T00:25:45.453 INFO:tasks.workunit.client.0.smithi112.stdout:3/14: read - no filename 2024-09-16T00:25:45.454 INFO:tasks.workunit.client.0.smithi112.stdout:3/15: chown l0 0 1 2024-09-16T00:25:45.454 INFO:tasks.workunit.client.0.smithi112.stdout:3/16: stat l0 0 2024-09-16T00:25:45.454 INFO:tasks.workunit.client.0.smithi112.stdout:3/17: write - no filename 2024-09-16T00:25:45.456 INFO:tasks.workunit.client.0.smithi112.stdout:5/5: symlink l0 0 2024-09-16T00:25:45.456 INFO:tasks.workunit.client.0.smithi112.stdout:5/6: write - no filename 2024-09-16T00:25:45.457 INFO:tasks.workunit.client.0.smithi112.stdout:1/6: write d0/f2 [741740,25523] 0 2024-09-16T00:25:45.459 INFO:tasks.workunit.client.0.smithi112.stdout:6/2: mknod c0 0 2024-09-16T00:25:45.459 INFO:tasks.workunit.client.0.smithi112.stdout:6/3: dread - no filename 2024-09-16T00:25:45.461 INFO:tasks.workunit.client.0.smithi112.stdout:5/7: mknod c1 0 2024-09-16T00:25:45.462 INFO:tasks.workunit.client.0.smithi112.stdout:1/7: mkdir d0/d3 0 2024-09-16T00:25:45.463 INFO:tasks.workunit.client.0.smithi112.stdout:4/13: link l0 l1 0 2024-09-16T00:25:45.463 INFO:tasks.workunit.client.0.smithi112.stdout:4/14: write - no filename 2024-09-16T00:25:45.464 INFO:tasks.workunit.client.0.smithi112.stdout:3/18: creat d2/f3 x:0 0 0 2024-09-16T00:25:45.465 INFO:tasks.workunit.client.0.smithi112.stdout:6/4: symlink l1 0 2024-09-16T00:25:45.465 INFO:tasks.workunit.client.0.smithi112.stdout:3/19: dread - d2/f3 zero size 2024-09-16T00:25:45.465 INFO:tasks.workunit.client.0.smithi112.stdout:6/5: chown l1 187235853 1 2024-09-16T00:25:45.465 INFO:tasks.workunit.client.0.smithi112.stdout:6/6: dread - no filename 2024-09-16T00:25:45.465 INFO:tasks.workunit.client.0.smithi112.stdout:3/20: readlink l0 0 2024-09-16T00:25:45.465 INFO:tasks.workunit.client.0.smithi112.stdout:6/7: write - no filename 2024-09-16T00:25:45.465 INFO:tasks.workunit.client.0.smithi112.stdout:6/8: dread - no filename 2024-09-16T00:25:45.465 INFO:tasks.workunit.client.0.smithi112.stdout:6/9: dwrite - no filename 2024-09-16T00:25:45.466 INFO:tasks.workunit.client.0.smithi112.stdout:7/0: symlink l0 0 2024-09-16T00:25:45.466 INFO:tasks.workunit.client.0.smithi112.stdout:6/10: readlink l1 0 2024-09-16T00:25:45.467 INFO:tasks.workunit.client.0.smithi112.stdout:1/8: dread d0/f2 [0,4194304] 0 2024-09-16T00:25:45.467 INFO:tasks.workunit.client.0.smithi112.stdout:9/0: truncate - no filename 2024-09-16T00:25:45.467 INFO:tasks.workunit.client.0.smithi112.stdout:9/1: unlink - no file 2024-09-16T00:25:45.467 INFO:tasks.workunit.client.0.smithi112.stdout:3/21: write d2/f3 [476509,11003] 0 2024-09-16T00:25:45.469 INFO:tasks.workunit.client.0.smithi112.stdout:5/8: creat f2 x:0 0 0 2024-09-16T00:25:45.469 INFO:tasks.workunit.client.0.smithi112.stdout:1/9: write d0/f2 [994816,130702] 0 2024-09-16T00:25:45.472 INFO:tasks.workunit.client.0.smithi112.stdout:7/1: creat f1 x:0 0 0 2024-09-16T00:25:45.472 INFO:tasks.workunit.client.0.smithi112.stdout:6/11: symlink l2 0 2024-09-16T00:25:45.473 INFO:tasks.workunit.client.0.smithi112.stdout:1/10: dread d0/f2 [0,4194304] 0 2024-09-16T00:25:45.473 INFO:tasks.workunit.client.0.smithi112.stdout:6/12: chown l1 231 1 2024-09-16T00:25:45.473 INFO:tasks.workunit.client.0.smithi112.stdout:6/13: fsync - no filename 2024-09-16T00:25:45.473 INFO:tasks.workunit.client.0.smithi112.stdout:8/0: mknod c0 0 2024-09-16T00:25:45.475 INFO:tasks.workunit.client.0.smithi112.stdout:3/22: symlink d2/l4 0 2024-09-16T00:25:45.475 INFO:tasks.workunit.client.0.smithi112.stdout:1/11: dread d0/f2 [0,4194304] 0 2024-09-16T00:25:45.476 INFO:tasks.workunit.client.0.smithi112.stdout:5/9: symlink l3 0 2024-09-16T00:25:45.476 INFO:tasks.workunit.client.0.smithi112.stdout:5/10: fdatasync f2 0 2024-09-16T00:25:45.478 INFO:tasks.workunit.client.0.smithi112.stdout:7/2: mknod c2 0 2024-09-16T00:25:45.479 INFO:tasks.workunit.client.0.smithi112.stdout:7/3: write f1 [994858,119597] 0 2024-09-16T00:25:45.480 INFO:tasks.workunit.client.0.smithi112.stdout:6/14: creat f3 x:0 0 0 2024-09-16T00:25:45.480 INFO:tasks.workunit.client.0.smithi112.stdout:9/2: symlink l0 0 2024-09-16T00:25:45.481 INFO:tasks.workunit.client.0.smithi112.stdout:6/15: write f3 [756600,42788] 0 2024-09-16T00:25:45.482 INFO:tasks.workunit.client.0.smithi112.stdout:8/1: rename c0 to c1 0 2024-09-16T00:25:45.482 INFO:tasks.workunit.client.0.smithi112.stdout:8/2: readlink - no filename 2024-09-16T00:25:45.482 INFO:tasks.workunit.client.0.smithi112.stdout:8/3: readlink - no filename 2024-09-16T00:25:45.482 INFO:tasks.workunit.client.0.smithi112.stdout:8/4: write - no filename 2024-09-16T00:25:45.483 INFO:tasks.workunit.client.0.smithi112.stdout:3/23: creat d2/f5 x:0 0 0 2024-09-16T00:25:45.483 INFO:tasks.workunit.client.0.smithi112.stdout:3/24: dread - d2/f5 zero size 2024-09-16T00:25:45.484 INFO:tasks.workunit.client.0.smithi112.stdout:4/15: link l0 l2 0 2024-09-16T00:25:45.487 INFO:tasks.workunit.client.0.smithi112.stdout:9/3: mkdir d1 0 2024-09-16T00:25:45.487 INFO:tasks.workunit.client.0.smithi112.stdout:9/4: dread - no filename 2024-09-16T00:25:45.487 INFO:tasks.workunit.client.0.smithi112.stdout:9/5: write - no filename 2024-09-16T00:25:45.487 INFO:tasks.workunit.client.0.smithi112.stdout:9/6: fsync - no filename 2024-09-16T00:25:45.488 INFO:tasks.workunit.client.0.smithi112.stdout:6/16: mknod c4 0 2024-09-16T00:25:45.488 INFO:tasks.workunit.client.0.smithi112.stdout:8/5: mknod c2 0 2024-09-16T00:25:45.491 INFO:tasks.workunit.client.0.smithi112.stdout:3/25: dwrite d2/f5 [0,4194304] 0 2024-09-16T00:25:45.492 INFO:tasks.workunit.client.0.smithi112.stdout:8/6: mknod c3 0 2024-09-16T00:25:45.492 INFO:tasks.workunit.client.0.smithi112.stdout:8/7: readlink - no filename 2024-09-16T00:25:45.492 INFO:tasks.workunit.client.0.smithi112.stdout:8/8: write - no filename 2024-09-16T00:25:45.493 INFO:tasks.workunit.client.0.smithi112.stdout:3/26: write d2/f5 [3411964,102889] 0 2024-09-16T00:25:45.494 INFO:tasks.workunit.client.0.smithi112.stdout:9/7: mknod d1/c2 0 2024-09-16T00:25:45.495 INFO:tasks.workunit.client.0.smithi112.stdout:4/16: rename l0 to l3 0 2024-09-16T00:25:45.496 INFO:tasks.workunit.client.0.smithi112.stdout:3/27: dread d2/f3 [0,4194304] 0 2024-09-16T00:25:45.499 INFO:tasks.workunit.client.0.smithi112.stdout:8/9: creat f4 x:0 0 0 2024-09-16T00:25:45.502 INFO:tasks.workunit.client.0.smithi112.stdout:9/8: creat d1/f3 x:0 0 0 2024-09-16T00:25:45.503 INFO:tasks.workunit.client.0.smithi112.stdout:9/9: rename d1 to d1/d4 22 2024-09-16T00:25:45.504 INFO:tasks.workunit.client.0.smithi112.stdout:4/17: creat f4 x:0 0 0 2024-09-16T00:25:45.504 INFO:tasks.workunit.client.0.smithi112.stdout:6/17: link l1 l5 0 2024-09-16T00:25:45.505 INFO:tasks.workunit.client.0.smithi112.stdout:3/28: creat d2/f6 x:0 0 0 2024-09-16T00:25:45.506 INFO:tasks.workunit.client.0.smithi112.stdout:9/10: unlink d1/f3 0 2024-09-16T00:25:45.506 INFO:tasks.workunit.client.0.smithi112.stdout:9/11: read - no filename 2024-09-16T00:25:45.508 INFO:tasks.workunit.client.0.smithi112.stdout:3/29: symlink d2/l7 0 2024-09-16T00:25:45.511 INFO:tasks.workunit.client.0.smithi112.stdout:9/12: mkdir d1/d5 0 2024-09-16T00:25:45.514 INFO:tasks.workunit.client.0.smithi112.stdout:9/13: mkdir d1/d5/d6 0 2024-09-16T00:25:45.514 INFO:tasks.workunit.client.0.smithi112.stdout:9/14: read - no filename 2024-09-16T00:25:45.514 INFO:tasks.workunit.client.0.smithi112.stdout:9/15: dwrite - no filename 2024-09-16T00:25:45.514 INFO:tasks.workunit.client.0.smithi112.stdout:9/16: write - no filename 2024-09-16T00:25:45.515 INFO:tasks.workunit.client.0.smithi112.stdout:3/30: dwrite d2/f5 [0,4194304] 0 2024-09-16T00:25:45.515 INFO:tasks.workunit.client.0.smithi112.stdout:9/17: chown d1/d5 4744496 1 2024-09-16T00:25:45.515 INFO:tasks.workunit.client.0.smithi112.stdout:3/31: read - d2/f6 zero size 2024-09-16T00:25:45.521 INFO:tasks.workunit.client.0.smithi112.stdout:9/18: link d1/c2 d1/d5/d6/c7 0 2024-09-16T00:25:45.524 INFO:tasks.workunit.client.0.smithi112.stdout:9/19: unlink d1/d5/d6/c7 0 2024-09-16T00:25:45.524 INFO:tasks.workunit.client.0.smithi112.stdout:9/20: write - no filename 2024-09-16T00:25:45.526 INFO:tasks.workunit.client.0.smithi112.stdout:9/21: creat d1/d5/d6/f8 x:0 0 0 2024-09-16T00:25:45.529 INFO:tasks.workunit.client.0.smithi112.stdout:9/22: link l0 d1/d5/l9 0 2024-09-16T00:25:45.529 INFO:tasks.workunit.client.0.smithi112.stdout:9/23: dread - d1/d5/d6/f8 zero size 2024-09-16T00:25:45.531 INFO:tasks.workunit.client.0.smithi112.stdout:9/24: mkdir d1/d5/da 0 2024-09-16T00:25:45.533 INFO:tasks.workunit.client.0.smithi112.stdout:9/25: symlink d1/d5/da/lb 0 2024-09-16T00:25:45.540 INFO:tasks.workunit.client.0.smithi112.stdout:9/26: dwrite d1/d5/d6/f8 [0,4194304] 0 2024-09-16T00:25:45.543 INFO:tasks.workunit.client.0.smithi112.stdout:9/27: chown d1/d5/l9 1 1 2024-09-16T00:25:45.548 INFO:tasks.workunit.client.0.smithi112.stdout:9/28: dread d1/d5/d6/f8 [0,4194304] 0 2024-09-16T00:25:45.556 INFO:tasks.workunit.client.0.smithi112.stdout:9/29: fdatasync d1/d5/d6/f8 0 2024-09-16T00:25:45.556 INFO:tasks.workunit.client.0.smithi112.stdout:9/30: rename d1/d5 to d1/d5/d6/dc 22 2024-09-16T00:25:45.558 INFO:tasks.workunit.client.0.smithi112.stdout:9/31: mkdir d1/d5/d6/dd 0 2024-09-16T00:25:45.559 INFO:tasks.workunit.client.0.smithi112.stdout:9/32: write d1/d5/d6/f8 [2972177,81796] 0 2024-09-16T00:25:45.559 INFO:tasks.workunit.client.0.smithi112.stdout:9/33: chown d1/d5/d6 8110 1 2024-09-16T00:25:45.561 INFO:tasks.workunit.client.0.smithi112.stdout:9/34: mknod d1/ce 0 2024-09-16T00:25:45.578 INFO:tasks.workunit.client.0.smithi112.stdout:0/3: fsync f0 0 2024-09-16T00:25:45.578 INFO:tasks.workunit.client.0.smithi112.stdout:0/4: stat f0 0 2024-09-16T00:25:45.578 INFO:tasks.workunit.client.0.smithi112.stdout:0/5: read - f0 zero size 2024-09-16T00:25:45.579 INFO:tasks.workunit.client.0.smithi112.stdout:0/6: write f0 [263883,96736] 0 2024-09-16T00:25:45.579 INFO:tasks.workunit.client.0.smithi112.stdout:0/7: stat f0 0 2024-09-16T00:25:45.582 INFO:tasks.workunit.client.0.smithi112.stdout:1/12: fsync d0/f2 0 2024-09-16T00:25:45.582 INFO:tasks.workunit.client.0.smithi112.stdout:1/13: chown d0 42363 1 2024-09-16T00:25:45.583 INFO:tasks.workunit.client.0.smithi112.stdout:0/8: dread f0 [0,4194304] 0 2024-09-16T00:25:45.585 INFO:tasks.workunit.client.0.smithi112.stdout:0/9: symlink l1 0 2024-09-16T00:25:45.586 INFO:tasks.workunit.client.0.smithi112.stdout:1/14: mkdir d0/d3/d4 0 2024-09-16T00:25:45.586 INFO:tasks.workunit.client.0.smithi112.stdout:1/15: chown d0/f2 217 1 2024-09-16T00:25:45.587 INFO:tasks.workunit.client.0.smithi112.stdout:0/10: rename l1 to l2 0 2024-09-16T00:25:45.590 INFO:tasks.workunit.client.0.smithi112.stdout:7/4: fsync f1 0 2024-09-16T00:25:45.591 INFO:tasks.workunit.client.0.smithi112.stdout:7/5: stat l0 0 2024-09-16T00:25:45.592 INFO:tasks.workunit.client.0.smithi112.stdout:5/11: getdents . 0 2024-09-16T00:25:45.593 INFO:tasks.workunit.client.0.smithi112.stdout:7/6: symlink l3 0 2024-09-16T00:25:45.596 INFO:tasks.workunit.client.0.smithi112.stdout:5/12: mkdir d4 0 2024-09-16T00:25:45.599 INFO:tasks.workunit.client.0.smithi112.stdout:8/10: chown c1 18 1 2024-09-16T00:25:45.599 INFO:tasks.workunit.client.0.smithi112.stdout:1/16: dwrite d0/f2 [0,4194304] 0 2024-09-16T00:25:45.600 INFO:tasks.workunit.client.0.smithi112.stdout:5/13: mknod d4/c5 0 2024-09-16T00:25:45.601 INFO:tasks.workunit.client.0.smithi112.stdout:8/11: chown c1 15394580 1 2024-09-16T00:25:45.601 INFO:tasks.workunit.client.0.smithi112.stdout:8/12: rmdir - no directory 2024-09-16T00:25:45.601 INFO:tasks.workunit.client.0.smithi112.stdout:7/7: dread f1 [0,4194304] 0 2024-09-16T00:25:45.602 INFO:tasks.workunit.client.0.smithi112.stdout:1/17: rename d0/d3 to d0/d3/d5 22 2024-09-16T00:25:45.602 INFO:tasks.workunit.client.0.smithi112.stdout:0/11: dwrite f0 [0,4194304] 0 2024-09-16T00:25:45.605 INFO:tasks.workunit.client.0.smithi112.stdout:5/14: creat d4/f6 x:0 0 0 2024-09-16T00:25:45.607 INFO:tasks.workunit.client.0.smithi112.stdout:6/18: getdents . 0 2024-09-16T00:25:45.607 INFO:tasks.workunit.client.0.smithi112.stdout:7/8: creat f4 x:0 0 0 2024-09-16T00:25:45.609 INFO:tasks.workunit.client.0.smithi112.stdout:4/18: chown l3 1959 1 2024-09-16T00:25:45.611 INFO:tasks.workunit.client.0.smithi112.stdout:1/18: creat d0/d3/f6 x:0 0 0 2024-09-16T00:25:45.613 INFO:tasks.workunit.client.0.smithi112.stdout:5/15: mkdir d4/d7 0 2024-09-16T00:25:45.614 INFO:tasks.workunit.client.0.smithi112.stdout:6/19: symlink l6 0 2024-09-16T00:25:45.618 INFO:tasks.workunit.client.0.smithi112.stdout:3/32: getdents d2 0 2024-09-16T00:25:45.618 INFO:tasks.workunit.client.0.smithi112.stdout:8/13: dwrite f4 [0,4194304] 0 2024-09-16T00:25:45.620 INFO:tasks.workunit.client.0.smithi112.stdout:3/33: fdatasync d2/f6 0 2024-09-16T00:25:45.624 INFO:tasks.workunit.client.0.smithi112.stdout:7/9: dwrite f4 [0,4194304] 0 2024-09-16T00:25:45.624 INFO:tasks.workunit.client.0.smithi112.stdout:5/16: creat d4/d7/f8 x:0 0 0 2024-09-16T00:25:45.626 INFO:tasks.workunit.client.0.smithi112.stdout:8/14: rename c3 to c5 0 2024-09-16T00:25:45.627 INFO:tasks.workunit.client.0.smithi112.stdout:4/19: getdents . 0 2024-09-16T00:25:45.631 INFO:tasks.workunit.client.0.smithi112.stdout:3/34: unlink d2/f3 0 2024-09-16T00:25:45.632 INFO:tasks.workunit.client.0.smithi112.stdout:3/35: fsync d2/f6 0 2024-09-16T00:25:45.633 INFO:tasks.workunit.client.0.smithi112.stdout:8/15: dread f4 [0,4194304] 0 2024-09-16T00:25:45.635 INFO:tasks.workunit.client.0.smithi112.stdout:4/20: symlink l5 0 2024-09-16T00:25:45.636 INFO:tasks.workunit.client.0.smithi112.stdout:3/36: creat d2/f8 x:0 0 0 2024-09-16T00:25:45.638 INFO:tasks.workunit.client.0.smithi112.stdout:7/10: link f4 f5 0 2024-09-16T00:25:45.639 INFO:tasks.workunit.client.0.smithi112.stdout:4/21: mkdir d6 0 2024-09-16T00:25:45.640 INFO:tasks.workunit.client.0.smithi112.stdout:5/17: fsync d4/f6 0 2024-09-16T00:25:45.643 INFO:tasks.workunit.client.0.smithi112.stdout:0/12: write f0 [4364438,75619] 0 2024-09-16T00:25:45.645 INFO:tasks.workunit.client.0.smithi112.stdout:4/22: rename l2 to d6/l7 0 2024-09-16T00:25:45.645 INFO:tasks.workunit.client.0.smithi112.stdout:1/19: rmdir d0/d3 39 2024-09-16T00:25:45.647 INFO:tasks.workunit.client.0.smithi112.stdout:4/23: dread - f4 zero size 2024-09-16T00:25:45.647 INFO:tasks.workunit.client.0.smithi112.stdout:0/13: mknod c3 0 2024-09-16T00:25:45.651 INFO:tasks.workunit.client.0.smithi112.stdout:0/14: creat f4 x:0 0 0 2024-09-16T00:25:45.652 INFO:tasks.workunit.client.0.smithi112.stdout:6/20: truncate f3 74595 0 2024-09-16T00:25:45.652 INFO:tasks.workunit.client.0.smithi112.stdout:4/24: symlink d6/l8 0 2024-09-16T00:25:45.653 INFO:tasks.workunit.client.0.smithi112.stdout:5/18: dwrite d4/d7/f8 [0,4194304] 0 2024-09-16T00:25:45.654 INFO:tasks.workunit.client.0.smithi112.stdout:7/11: dwrite f5 [0,4194304] 0 2024-09-16T00:25:45.654 INFO:tasks.workunit.client.0.smithi112.stdout:0/15: chown f4 217042363 1 2024-09-16T00:25:45.655 INFO:tasks.workunit.client.0.smithi112.stdout:5/19: write d4/f6 [860356,68487] 0 2024-09-16T00:25:45.656 INFO:tasks.workunit.client.0.smithi112.stdout:8/16: rename c5 to c6 0 2024-09-16T00:25:45.661 INFO:tasks.workunit.client.0.smithi112.stdout:1/20: symlink d0/d3/d4/l7 0 2024-09-16T00:25:45.663 INFO:tasks.workunit.client.0.smithi112.stdout:7/12: dread f5 [0,4194304] 0 2024-09-16T00:25:45.663 INFO:tasks.workunit.client.0.smithi112.stdout:4/25: creat d6/f9 x:0 0 0 2024-09-16T00:25:45.664 INFO:tasks.workunit.client.0.smithi112.stdout:4/26: dread - d6/f9 zero size 2024-09-16T00:25:45.664 INFO:tasks.workunit.client.0.smithi112.stdout:4/27: read - d6/f9 zero size 2024-09-16T00:25:45.665 INFO:tasks.workunit.client.0.smithi112.stdout:0/16: mkdir d5 0 2024-09-16T00:25:45.666 INFO:tasks.workunit.client.0.smithi112.stdout:3/37: truncate d2/f5 1644844 0 2024-09-16T00:25:45.666 INFO:tasks.workunit.client.0.smithi112.stdout:0/17: dread - f4 zero size 2024-09-16T00:25:45.667 INFO:tasks.workunit.client.0.smithi112.stdout:0/18: rename d5 to d5/d6 22 2024-09-16T00:25:45.667 INFO:tasks.workunit.client.0.smithi112.stdout:8/17: write f4 [1713589,128245] 0 2024-09-16T00:25:45.668 INFO:tasks.workunit.client.0.smithi112.stdout:8/18: read f4 [1786032,21818] 0 2024-09-16T00:25:45.668 INFO:tasks.workunit.client.0.smithi112.stdout:5/20: creat d4/d7/f9 x:0 0 0 2024-09-16T00:25:45.669 INFO:tasks.workunit.client.0.smithi112.stdout:5/21: dread - f2 zero size 2024-09-16T00:25:45.671 INFO:tasks.workunit.client.0.smithi112.stdout:1/21: unlink d0/d3/d4/l7 0 2024-09-16T00:25:45.671 INFO:tasks.workunit.client.0.smithi112.stdout:7/13: mkdir d6 0 2024-09-16T00:25:45.672 INFO:tasks.workunit.client.0.smithi112.stdout:7/14: fdatasync f1 0 2024-09-16T00:25:45.672 INFO:tasks.workunit.client.0.smithi112.stdout:1/22: truncate d0/d3/f6 559260 0 2024-09-16T00:25:45.672 INFO:tasks.workunit.client.0.smithi112.stdout:4/28: mkdir d6/da 0 2024-09-16T00:25:45.673 INFO:tasks.workunit.client.0.smithi112.stdout:3/38: mkdir d2/d9 0 2024-09-16T00:25:45.674 INFO:tasks.workunit.client.0.smithi112.stdout:1/23: read d0/d3/f6 [178442,108833] 0 2024-09-16T00:25:45.675 INFO:tasks.workunit.client.0.smithi112.stdout:5/22: mknod d4/ca 0 2024-09-16T00:25:45.676 INFO:tasks.workunit.client.0.smithi112.stdout:0/19: fsync f4 0 2024-09-16T00:25:45.678 INFO:tasks.workunit.client.0.smithi112.stdout:6/21: dread f3 [0,4194304] 0 2024-09-16T00:25:45.682 INFO:tasks.workunit.client.0.smithi112.stdout:7/15: creat d6/f7 x:0 0 0 2024-09-16T00:25:45.688 INFO:tasks.workunit.client.0.smithi112.stdout:4/29: dwrite d6/f9 [0,4194304] 0 2024-09-16T00:25:45.692 INFO:tasks.workunit.client.0.smithi112.stdout:7/16: dread f1 [0,4194304] 0 2024-09-16T00:25:45.692 INFO:tasks.workunit.client.0.smithi112.stdout:5/23: dwrite d4/d7/f8 [0,4194304] 0 2024-09-16T00:25:45.692 INFO:tasks.workunit.client.0.smithi112.stdout:0/20: rename l2 to d5/l7 0 2024-09-16T00:25:45.693 INFO:tasks.workunit.client.0.smithi112.stdout:1/24: dwrite d0/f2 [0,4194304] 0 2024-09-16T00:25:45.694 INFO:tasks.workunit.client.0.smithi112.stdout:0/21: read - f4 zero size 2024-09-16T00:25:45.694 INFO:tasks.workunit.client.0.smithi112.stdout:4/30: readlink l3 0 2024-09-16T00:25:45.695 INFO:tasks.workunit.client.0.smithi112.stdout:7/17: write d6/f7 [886062,41603] 0 2024-09-16T00:25:45.695 INFO:tasks.workunit.client.0.smithi112.stdout:0/22: fdatasync f4 0 2024-09-16T00:25:45.696 INFO:tasks.workunit.client.0.smithi112.stdout:6/22: rename l1 to l7 0 2024-09-16T00:25:45.698 INFO:tasks.workunit.client.0.smithi112.stdout:5/24: creat d4/fb x:0 0 0 2024-09-16T00:25:45.699 INFO:tasks.workunit.client.0.smithi112.stdout:6/23: creat f8 x:0 0 0 2024-09-16T00:25:45.700 INFO:tasks.workunit.client.0.smithi112.stdout:4/31: mkdir d6/db 0 2024-09-16T00:25:45.700 INFO:tasks.workunit.client.0.smithi112.stdout:6/24: write f8 [155889,2883] 0 2024-09-16T00:25:45.701 INFO:tasks.workunit.client.0.smithi112.stdout:8/19: truncate f4 855095 0 2024-09-16T00:25:45.701 INFO:tasks.workunit.client.0.smithi112.stdout:8/20: readlink - no filename 2024-09-16T00:25:45.701 INFO:tasks.workunit.client.0.smithi112.stdout:4/32: read - f4 zero size 2024-09-16T00:25:45.703 INFO:tasks.workunit.client.0.smithi112.stdout:7/18: dread f5 [0,4194304] 0 2024-09-16T00:25:45.703 INFO:tasks.workunit.client.0.smithi112.stdout:6/25: read f3 [21144,121812] 0 2024-09-16T00:25:45.708 INFO:tasks.workunit.client.0.smithi112.stdout:7/19: mknod d6/c8 0 2024-09-16T00:25:45.710 INFO:tasks.workunit.client.0.smithi112.stdout:0/23: readlink d5/l7 0 2024-09-16T00:25:45.710 INFO:tasks.workunit.client.0.smithi112.stdout:7/20: write d6/f7 [415531,129473] 0 2024-09-16T00:25:45.710 INFO:tasks.workunit.client.0.smithi112.stdout:7/21: stat d6/f7 0 2024-09-16T00:25:45.714 INFO:tasks.workunit.client.0.smithi112.stdout:6/26: unlink f3 0 2024-09-16T00:25:45.714 INFO:tasks.workunit.client.0.smithi112.stdout:5/25: link d4/ca d4/cc 0 2024-09-16T00:25:45.715 INFO:tasks.workunit.client.0.smithi112.stdout:4/33: mkdir d6/da/dc 0 2024-09-16T00:25:45.717 INFO:tasks.workunit.client.0.smithi112.stdout:8/21: dread f4 [0,4194304] 0 2024-09-16T00:25:45.718 INFO:tasks.workunit.client.0.smithi112.stdout:7/22: fsync f4 0 2024-09-16T00:25:45.718 INFO:tasks.workunit.client.0.smithi112.stdout:0/24: chown d5/l7 361966 1 2024-09-16T00:25:45.719 INFO:tasks.workunit.client.0.smithi112.stdout:6/27: unlink c0 0 2024-09-16T00:25:45.719 INFO:tasks.workunit.client.0.smithi112.stdout:0/25: rename d5 to d5/d8 22 2024-09-16T00:25:45.721 INFO:tasks.workunit.client.0.smithi112.stdout:8/22: symlink l7 0 2024-09-16T00:25:45.722 INFO:tasks.workunit.client.0.smithi112.stdout:7/23: creat d6/f9 x:0 0 0 2024-09-16T00:25:45.723 INFO:tasks.workunit.client.0.smithi112.stdout:6/28: rename l6 to l9 0 2024-09-16T00:25:45.725 INFO:tasks.workunit.client.0.smithi112.stdout:5/26: unlink d4/cc 0 2024-09-16T00:25:45.743 INFO:tasks.workunit.client.0.smithi112.stdout:6/29: dread f8 [0,4194304] 0 2024-09-16T00:25:45.743 INFO:tasks.workunit.client.0.smithi112.stdout:7/24: dread f5 [0,4194304] 0 2024-09-16T00:25:45.743 INFO:tasks.workunit.client.0.smithi112.stdout:3/39: truncate d2/f5 1458434 0 2024-09-16T00:25:45.744 INFO:tasks.workunit.client.0.smithi112.stdout:8/23: fsync f4 0 2024-09-16T00:25:45.749 INFO:tasks.workunit.client.0.smithi112.stdout:7/25: dwrite d6/f9 [0,4194304] 0 2024-09-16T00:25:45.873 INFO:tasks.workunit.client.0.smithi112.stdout:5/27: mkdir d4/dd 0 2024-09-16T00:25:45.873 INFO:tasks.workunit.client.0.smithi112.stdout:6/30: creat fa x:0 0 0 2024-09-16T00:25:45.874 INFO:tasks.workunit.client.0.smithi112.stdout:5/28: truncate d4/d7/f9 680407 0 2024-09-16T00:25:45.874 INFO:tasks.workunit.client.0.smithi112.stdout:6/31: dread - fa zero size 2024-09-16T00:25:45.875 INFO:tasks.workunit.client.0.smithi112.stdout:8/24: mknod c8 0 2024-09-16T00:25:45.875 INFO:tasks.workunit.client.0.smithi112.stdout:8/25: chown c8 15763596 1 2024-09-16T00:25:45.875 INFO:tasks.workunit.client.0.smithi112.stdout:8/26: rmdir - no directory 2024-09-16T00:25:45.878 INFO:tasks.workunit.client.0.smithi112.stdout:7/26: rmdir d6 39 2024-09-16T00:25:45.880 INFO:tasks.workunit.client.0.smithi112.stdout:0/26: link d5/l7 d5/l9 0 2024-09-16T00:25:45.882 INFO:tasks.workunit.client.0.smithi112.stdout:6/32: mknod cb 0 2024-09-16T00:25:45.882 INFO:tasks.workunit.client.0.smithi112.stdout:5/29: symlink d4/le 0 2024-09-16T00:25:45.884 INFO:tasks.workunit.client.0.smithi112.stdout:0/27: mkdir d5/da 0 2024-09-16T00:25:45.885 INFO:tasks.workunit.client.0.smithi112.stdout:6/33: fdatasync f8 0 2024-09-16T00:25:45.886 INFO:tasks.workunit.client.0.smithi112.stdout:5/30: mkdir d4/df 0 2024-09-16T00:25:45.887 INFO:tasks.workunit.client.0.smithi112.stdout:0/28: rmdir d5 39 2024-09-16T00:25:45.887 INFO:tasks.workunit.client.0.smithi112.stdout:5/31: chown d4/d7 2 1 2024-09-16T00:25:45.888 INFO:tasks.workunit.client.0.smithi112.stdout:0/29: fsync f0 0 2024-09-16T00:25:45.888 INFO:tasks.workunit.client.0.smithi112.stdout:5/32: dread - f2 zero size 2024-09-16T00:25:45.890 INFO:tasks.workunit.client.0.smithi112.stdout:0/30: write f4 [866863,2678] 0 2024-09-16T00:25:45.891 INFO:tasks.workunit.client.0.smithi112.stdout:5/33: mknod d4/d7/c10 0 2024-09-16T00:25:45.894 INFO:tasks.workunit.client.0.smithi112.stdout:6/34: dwrite fa [0,4194304] 0 2024-09-16T00:25:45.896 INFO:tasks.workunit.client.0.smithi112.stdout:5/34: creat d4/df/f11 x:0 0 0 2024-09-16T00:25:45.896 INFO:tasks.workunit.client.0.smithi112.stdout:0/31: mknod d5/cb 0 2024-09-16T00:25:45.896 INFO:tasks.workunit.client.0.smithi112.stdout:0/32: chown d5/da 65927 1 2024-09-16T00:25:45.897 INFO:tasks.workunit.client.0.smithi112.stdout:5/35: fsync f2 0 2024-09-16T00:25:45.897 INFO:tasks.workunit.client.0.smithi112.stdout:6/35: mknod cc 0 2024-09-16T00:25:45.957 INFO:tasks.workunit.client.0.smithi112.stdout:8/27: truncate f4 1368373 0 2024-09-16T00:25:45.961 INFO:tasks.workunit.client.0.smithi112.stdout:5/36: rename d4/df to d4/dd/d12 0 2024-09-16T00:25:45.962 INFO:tasks.workunit.client.0.smithi112.stdout:5/37: truncate d4/fb 194209 0 2024-09-16T00:25:45.963 INFO:tasks.workunit.client.0.smithi112.stdout:6/36: getdents . 0 2024-09-16T00:25:45.965 INFO:tasks.workunit.client.0.smithi112.stdout:5/38: rename d4/le to d4/dd/d12/l13 0 2024-09-16T00:25:45.966 INFO:tasks.workunit.client.0.smithi112.stdout:6/37: readlink l5 0 2024-09-16T00:25:45.967 INFO:tasks.workunit.client.0.smithi112.stdout:8/28: write f4 [700785,127682] 0 2024-09-16T00:25:45.967 INFO:tasks.workunit.client.0.smithi112.stdout:5/39: mkdir d4/d7/d14 0 2024-09-16T00:25:45.967 INFO:tasks.workunit.client.0.smithi112.stdout:8/29: read f4 [495512,78988] 0 2024-09-16T00:25:45.969 INFO:tasks.workunit.client.0.smithi112.stdout:5/40: write d4/f6 [592995,57821] 0 2024-09-16T00:25:45.969 INFO:tasks.workunit.client.0.smithi112.stdout:6/38: getdents . 0 2024-09-16T00:25:45.970 INFO:tasks.workunit.client.0.smithi112.stdout:8/30: creat f9 x:0 0 0 2024-09-16T00:25:45.970 INFO:tasks.workunit.client.0.smithi112.stdout:8/31: chown c1 416960702 1 2024-09-16T00:25:45.970 INFO:tasks.workunit.client.0.smithi112.stdout:6/39: mkdir dd 0 2024-09-16T00:25:45.971 INFO:tasks.workunit.client.0.smithi112.stdout:5/41: symlink d4/dd/d12/l15 0 2024-09-16T00:25:45.971 INFO:tasks.workunit.client.0.smithi112.stdout:6/40: write f8 [987049,77134] 0 2024-09-16T00:25:45.972 INFO:tasks.workunit.client.0.smithi112.stdout:5/42: stat d4/d7 0 2024-09-16T00:25:45.972 INFO:tasks.workunit.client.0.smithi112.stdout:6/41: rename dd to dd/de 22 2024-09-16T00:25:45.973 INFO:tasks.workunit.client.0.smithi112.stdout:5/43: dread d4/fb [0,4194304] 0 2024-09-16T00:25:45.974 INFO:tasks.workunit.client.0.smithi112.stdout:8/32: link l7 la 0 2024-09-16T00:25:45.975 INFO:tasks.workunit.client.0.smithi112.stdout:6/42: mkdir dd/df 0 2024-09-16T00:25:45.975 INFO:tasks.workunit.client.0.smithi112.stdout:5/44: creat d4/f16 x:0 0 0 2024-09-16T00:25:45.976 INFO:tasks.workunit.client.0.smithi112.stdout:8/33: symlink lb 0 2024-09-16T00:25:45.977 INFO:tasks.workunit.client.0.smithi112.stdout:6/43: mkdir dd/df/d10 0 2024-09-16T00:25:45.977 INFO:tasks.workunit.client.0.smithi112.stdout:6/44: fsync f8 0 2024-09-16T00:25:45.979 INFO:tasks.workunit.client.0.smithi112.stdout:5/45: rename d4/c5 to d4/dd/c17 0 2024-09-16T00:25:45.980 INFO:tasks.workunit.client.0.smithi112.stdout:5/46: write d4/d7/f9 [1609966,844] 0 2024-09-16T00:25:45.980 INFO:tasks.workunit.client.0.smithi112.stdout:6/45: dread fa [0,4194304] 0 2024-09-16T00:25:45.981 INFO:tasks.workunit.client.0.smithi112.stdout:8/34: chown l7 15 1 2024-09-16T00:25:45.981 INFO:tasks.workunit.client.0.smithi112.stdout:5/47: fsync d4/d7/f8 0 2024-09-16T00:25:45.982 INFO:tasks.workunit.client.0.smithi112.stdout:8/35: dread - f9 zero size 2024-09-16T00:25:45.982 INFO:tasks.workunit.client.0.smithi112.stdout:6/46: read f8 [521340,52826] 0 2024-09-16T00:25:45.983 INFO:tasks.workunit.client.0.smithi112.stdout:5/48: truncate d4/d7/f9 2632803 0 2024-09-16T00:25:45.984 INFO:tasks.workunit.client.0.smithi112.stdout:8/36: symlink lc 0 2024-09-16T00:25:45.984 INFO:tasks.workunit.client.0.smithi112.stdout:6/47: mkdir dd/df/d11 0 2024-09-16T00:25:45.985 INFO:tasks.workunit.client.0.smithi112.stdout:5/49: mkdir d4/d18 0 2024-09-16T00:25:45.987 INFO:tasks.workunit.client.0.smithi112.stdout:8/37: rename c1 to cd 0 2024-09-16T00:25:45.987 INFO:tasks.workunit.client.0.smithi112.stdout:6/48: symlink dd/l12 0 2024-09-16T00:25:45.989 INFO:tasks.workunit.client.0.smithi112.stdout:8/38: creat fe x:0 0 0 2024-09-16T00:25:45.990 INFO:tasks.workunit.client.0.smithi112.stdout:8/39: chown f9 183356258 1 2024-09-16T00:25:45.994 INFO:tasks.workunit.client.0.smithi112.stdout:8/40: symlink lf 0 2024-09-16T00:25:45.994 INFO:tasks.workunit.client.0.smithi112.stdout:5/50: dwrite d4/f16 [0,4194304] 0 2024-09-16T00:25:45.995 INFO:tasks.workunit.client.0.smithi112.stdout:6/49: dread fa [0,4194304] 0 2024-09-16T00:25:45.997 INFO:tasks.workunit.client.0.smithi112.stdout:6/50: chown dd/l12 7 1 2024-09-16T00:25:45.998 INFO:tasks.workunit.client.0.smithi112.stdout:5/51: rename d4/f6 to d4/f19 0 2024-09-16T00:25:45.998 INFO:tasks.workunit.client.0.smithi112.stdout:5/52: readlink d4/dd/d12/l15 0 2024-09-16T00:25:46.000 INFO:tasks.workunit.client.0.smithi112.stdout:6/51: write fa [3044756,98573] 0 2024-09-16T00:25:46.004 INFO:tasks.workunit.client.0.smithi112.stdout:5/53: creat d4/d7/d14/f1a x:0 0 0 2024-09-16T00:25:46.004 INFO:tasks.workunit.client.0.smithi112.stdout:6/52: mkdir dd/df/d13 0 2024-09-16T00:25:46.004 INFO:tasks.workunit.client.0.smithi112.stdout:6/53: write fa [1863407,128883] 0 2024-09-16T00:25:46.005 INFO:tasks.workunit.client.0.smithi112.stdout:5/54: rmdir d4/d18 0 2024-09-16T00:25:46.005 INFO:tasks.workunit.client.0.smithi112.stdout:5/55: stat l0 0 2024-09-16T00:25:46.005 INFO:tasks.workunit.client.0.smithi112.stdout:5/56: stat d4/d7/d14/f1a 0 2024-09-16T00:25:46.007 INFO:tasks.workunit.client.0.smithi112.stdout:5/57: symlink d4/l1b 0 2024-09-16T00:25:46.013 INFO:tasks.workunit.client.0.smithi112.stdout:5/58: fsync d4/f16 0 2024-09-16T00:25:46.013 INFO:tasks.workunit.client.0.smithi112.stdout:6/54: truncate fa 622501 0 2024-09-16T00:25:46.015 INFO:tasks.workunit.client.0.smithi112.stdout:6/55: symlink dd/df/d10/l14 0 2024-09-16T00:25:46.016 INFO:tasks.workunit.client.0.smithi112.stdout:6/56: mkdir dd/d15 0 2024-09-16T00:25:46.016 INFO:tasks.workunit.client.0.smithi112.stdout:6/57: readlink l7 0 2024-09-16T00:25:46.017 INFO:tasks.workunit.client.0.smithi112.stdout:6/58: chown cc 89133 1 2024-09-16T00:25:46.019 INFO:tasks.workunit.client.0.smithi112.stdout:6/59: dread f8 [0,4194304] 0 2024-09-16T00:25:46.022 INFO:tasks.workunit.client.0.smithi112.stdout:6/60: dread fa [0,4194304] 0 2024-09-16T00:25:46.024 INFO:tasks.workunit.client.0.smithi112.stdout:6/61: symlink dd/df/d10/l16 0 2024-09-16T00:25:46.024 INFO:tasks.workunit.client.0.smithi112.stdout:6/62: chown fa 195081 1 2024-09-16T00:25:46.024 INFO:tasks.workunit.client.0.smithi112.stdout:6/63: stat l2 0 2024-09-16T00:25:46.026 INFO:tasks.workunit.client.0.smithi112.stdout:6/64: mkdir dd/df/d17 0 2024-09-16T00:25:46.027 INFO:tasks.workunit.client.0.smithi112.stdout:6/65: mkdir dd/df/d13/d18 0 2024-09-16T00:25:46.028 INFO:tasks.workunit.client.0.smithi112.stdout:6/66: chown c4 0 1 2024-09-16T00:25:46.029 INFO:tasks.workunit.client.0.smithi112.stdout:6/67: mkdir dd/df/d19 0 2024-09-16T00:25:46.033 INFO:tasks.workunit.client.0.smithi112.stdout:6/68: truncate fa 238035 0 2024-09-16T00:25:46.042 INFO:tasks.workunit.client.0.smithi112.stdout:6/69: dwrite fa [0,4194304] 0 2024-09-16T00:25:46.045 INFO:tasks.workunit.client.0.smithi112.stdout:6/70: creat dd/df/d10/f1a x:0 0 0 2024-09-16T00:25:46.053 INFO:tasks.workunit.client.0.smithi112.stdout:6/71: dwrite f8 [0,4194304] 0 2024-09-16T00:25:46.055 INFO:tasks.workunit.client.0.smithi112.stdout:6/72: creat dd/df/d10/f1b x:0 0 0 2024-09-16T00:25:46.056 INFO:tasks.workunit.client.0.smithi112.stdout:6/73: write f8 [1749408,75651] 0 2024-09-16T00:25:46.060 INFO:tasks.workunit.client.0.smithi112.stdout:6/74: dread f8 [0,4194304] 0 2024-09-16T00:25:46.061 INFO:tasks.workunit.client.0.smithi112.stdout:6/75: read - dd/df/d10/f1b zero size 2024-09-16T00:25:46.061 INFO:tasks.workunit.client.0.smithi112.stdout:6/76: write fa [2159238,43576] 0 2024-09-16T00:25:46.070 INFO:tasks.workunit.client.0.smithi112.stdout:6/77: dwrite dd/df/d10/f1b [0,4194304] 0 2024-09-16T00:25:46.072 INFO:tasks.workunit.client.0.smithi112.stdout:6/78: creat dd/df/d13/f1c x:0 0 0 2024-09-16T00:25:46.074 INFO:tasks.workunit.client.0.smithi112.stdout:6/79: mkdir dd/df/d10/d1d 0 2024-09-16T00:25:46.077 INFO:tasks.workunit.client.0.smithi112.stdout:6/80: rmdir dd/df/d19 0 2024-09-16T00:25:46.079 INFO:tasks.workunit.client.0.smithi112.stdout:6/81: creat dd/df/d11/f1e x:0 0 0 2024-09-16T00:25:46.234 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:25:46 smithi169 ceph-mon[34653]: pgmap v246: 65 pgs: 65 active+clean; 211 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 3.6 MiB/s rd, 6.8 MiB/s wr, 522 op/s 2024-09-16T00:25:46.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:25:46 smithi112 ceph-mon[27869]: pgmap v246: 65 pgs: 65 active+clean; 211 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 3.6 MiB/s rd, 6.8 MiB/s wr, 522 op/s 2024-09-16T00:25:47.007 INFO:tasks.workunit.client.1.smithi169.stderr:++ readlink -f fsstress 2024-09-16T00:25:47.009 INFO:tasks.workunit.client.1.smithi169.stderr:+ BIN=/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-09-16T00:25:47.009 INFO:tasks.workunit.client.1.smithi169.stderr:+ popd 2024-09-16T00:25:47.010 INFO:tasks.workunit.client.1.smithi169.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-09-16T00:25:47.010 INFO:tasks.workunit.client.1.smithi169.stderr:+ popd 2024-09-16T00:25:47.011 INFO:tasks.workunit.client.1.smithi169.stdout:~/cephtest/mnt.1/client.1/tmp 2024-09-16T00:25:47.011 INFO:tasks.workunit.client.1.smithi169.stderr:++ mktemp -d -p . 2024-09-16T00:25:47.014 INFO:tasks.workunit.client.1.smithi169.stderr:+ T=./tmp.RVqddVnjCD 2024-09-16T00:25:47.015 INFO:tasks.workunit.client.1.smithi169.stderr:+ /home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.RVqddVnjCD -l 1 -n 1000 -p 10 -v 2024-09-16T00:25:47.018 INFO:tasks.workunit.client.1.smithi169.stdout:seed = 1726429166 2024-09-16T00:25:47.024 INFO:tasks.workunit.client.1.smithi169.stdout:1/0: dread - no filename 2024-09-16T00:25:47.024 INFO:tasks.workunit.client.1.smithi169.stdout:1/1: dread - no filename 2024-09-16T00:25:47.024 INFO:tasks.workunit.client.1.smithi169.stdout:1/2: stat - no entries 2024-09-16T00:25:47.024 INFO:tasks.workunit.client.1.smithi169.stdout:1/3: dread - no filename 2024-09-16T00:25:47.025 INFO:tasks.workunit.client.1.smithi169.stdout:1/4: truncate - no filename 2024-09-16T00:25:47.025 INFO:tasks.workunit.client.1.smithi169.stdout:1/5: write - no filename 2024-09-16T00:25:47.025 INFO:tasks.workunit.client.1.smithi169.stdout:1/6: dwrite - no filename 2024-09-16T00:25:47.025 INFO:tasks.workunit.client.1.smithi169.stdout:1/7: truncate - no filename 2024-09-16T00:25:47.028 INFO:tasks.workunit.client.1.smithi169.stdout:2/0: truncate - no filename 2024-09-16T00:25:47.028 INFO:tasks.workunit.client.1.smithi169.stdout:2/1: link - no file 2024-09-16T00:25:47.028 INFO:tasks.workunit.client.1.smithi169.stdout:2/2: readlink - no filename 2024-09-16T00:25:47.028 INFO:tasks.workunit.client.1.smithi169.stdout:2/3: dwrite - no filename 2024-09-16T00:25:47.028 INFO:tasks.workunit.client.1.smithi169.stdout:2/4: link - no file 2024-09-16T00:25:47.028 INFO:tasks.workunit.client.1.smithi169.stdout:2/5: fdatasync - no filename 2024-09-16T00:25:47.028 INFO:tasks.workunit.client.1.smithi169.stdout:2/6: rmdir - no directory 2024-09-16T00:25:47.028 INFO:tasks.workunit.client.1.smithi169.stdout:2/7: unlink - no file 2024-09-16T00:25:47.029 INFO:tasks.workunit.client.1.smithi169.stdout:1/8: creat f0 x:0 0 0 2024-09-16T00:25:47.030 INFO:tasks.workunit.client.1.smithi169.stdout:1/9: dread - f0 zero size 2024-09-16T00:25:47.030 INFO:tasks.workunit.client.1.smithi169.stdout:1/10: read - f0 zero size 2024-09-16T00:25:47.030 INFO:tasks.workunit.client.1.smithi169.stdout:3/0: rename - no filename 2024-09-16T00:25:47.030 INFO:tasks.workunit.client.1.smithi169.stdout:3/1: fsync - no filename 2024-09-16T00:25:47.030 INFO:tasks.workunit.client.1.smithi169.stdout:2/8: creat f0 x:0 0 0 2024-09-16T00:25:47.030 INFO:tasks.workunit.client.1.smithi169.stdout:2/9: fdatasync f0 0 2024-09-16T00:25:47.030 INFO:tasks.workunit.client.1.smithi169.stdout:1/11: creat f1 x:0 0 0 2024-09-16T00:25:47.031 INFO:tasks.workunit.client.1.smithi169.stdout:2/10: creat f1 x:0 0 0 2024-09-16T00:25:47.032 INFO:tasks.workunit.client.1.smithi169.stdout:1/12: mknod c2 0 2024-09-16T00:25:47.033 INFO:tasks.workunit.client.1.smithi169.stdout:3/2: creat f0 x:0 0 0 2024-09-16T00:25:47.036 INFO:tasks.workunit.client.1.smithi169.stdout:2/11: rename f1 to f2 0 2024-09-16T00:25:47.037 INFO:tasks.workunit.client.1.smithi169.stdout:3/3: creat f1 x:0 0 0 2024-09-16T00:25:47.037 INFO:tasks.workunit.client.1.smithi169.stdout:4/0: symlink l0 0 2024-09-16T00:25:47.038 INFO:tasks.workunit.client.1.smithi169.stdout:3/4: chown f0 1 1 2024-09-16T00:25:47.040 INFO:tasks.workunit.client.1.smithi169.stdout:6/0: chown . 377650012 1 2024-09-16T00:25:47.041 INFO:tasks.workunit.client.1.smithi169.stdout:4/1: mknod c1 0 2024-09-16T00:25:47.041 INFO:tasks.workunit.client.1.smithi169.stdout:4/2: dwrite - no filename 2024-09-16T00:25:47.042 INFO:tasks.workunit.client.1.smithi169.stdout:4/3: rmdir - no directory 2024-09-16T00:25:47.042 INFO:tasks.workunit.client.1.smithi169.stdout:4/4: write - no filename 2024-09-16T00:25:47.042 INFO:tasks.workunit.client.1.smithi169.stdout:4/5: rmdir - no directory 2024-09-16T00:25:47.043 INFO:tasks.workunit.client.1.smithi169.stdout:5/0: creat f0 x:0 0 0 2024-09-16T00:25:47.043 INFO:tasks.workunit.client.1.smithi169.stdout:3/5: symlink l2 0 2024-09-16T00:25:47.043 INFO:tasks.workunit.client.1.smithi169.stdout:1/13: dwrite f0 [0,4194304] 0 2024-09-16T00:25:47.045 INFO:tasks.workunit.client.1.smithi169.stdout:1/14: truncate f1 444991 0 2024-09-16T00:25:47.045 INFO:tasks.workunit.client.1.smithi169.stdout:7/0: chown . 106698 1 2024-09-16T00:25:47.045 INFO:tasks.workunit.client.1.smithi169.stdout:4/6: mkdir d2 0 2024-09-16T00:25:47.045 INFO:tasks.workunit.client.1.smithi169.stdout:4/7: stat c1 0 2024-09-16T00:25:47.046 INFO:tasks.workunit.client.1.smithi169.stdout:5/1: mknod c1 0 2024-09-16T00:25:47.046 INFO:tasks.workunit.client.1.smithi169.stdout:4/8: chown c1 932175 1 2024-09-16T00:25:47.047 INFO:tasks.workunit.client.1.smithi169.stdout:4/9: readlink l0 0 2024-09-16T00:25:47.048 INFO:tasks.workunit.client.1.smithi169.stdout:4/10: chown c1 708 1 2024-09-16T00:25:47.048 INFO:tasks.workunit.client.1.smithi169.stdout:3/6: creat f3 x:0 0 0 2024-09-16T00:25:47.048 INFO:tasks.workunit.client.1.smithi169.stdout:4/11: chown l0 10 1 2024-09-16T00:25:47.049 INFO:tasks.workunit.client.1.smithi169.stdout:6/1: mknod c0 0 2024-09-16T00:25:47.049 INFO:tasks.workunit.client.1.smithi169.stdout:6/2: dread - no filename 2024-09-16T00:25:47.051 INFO:tasks.workunit.client.1.smithi169.stdout:1/15: creat f3 x:0 0 0 2024-09-16T00:25:47.052 INFO:tasks.workunit.client.1.smithi169.stdout:1/16: chown f3 1 1 2024-09-16T00:25:47.053 INFO:tasks.workunit.client.1.smithi169.stdout:5/2: mkdir d2 0 2024-09-16T00:25:47.055 INFO:tasks.workunit.client.1.smithi169.stdout:3/7: symlink l4 0 2024-09-16T00:25:47.056 INFO:tasks.workunit.client.1.smithi169.stdout:1/17: dread f0 [0,4194304] 0 2024-09-16T00:25:47.057 INFO:tasks.workunit.client.1.smithi169.stdout:6/3: rename c0 to c1 0 2024-09-16T00:25:47.057 INFO:tasks.workunit.client.1.smithi169.stdout:6/4: write - no filename 2024-09-16T00:25:47.057 INFO:tasks.workunit.client.1.smithi169.stdout:6/5: write - no filename 2024-09-16T00:25:47.057 INFO:tasks.workunit.client.1.smithi169.stdout:6/6: fsync - no filename 2024-09-16T00:25:47.057 INFO:tasks.workunit.client.1.smithi169.stdout:6/7: fsync - no filename 2024-09-16T00:25:47.057 INFO:tasks.workunit.client.1.smithi169.stdout:6/8: dwrite - no filename 2024-09-16T00:25:47.057 INFO:tasks.workunit.client.1.smithi169.stdout:6/9: dwrite - no filename 2024-09-16T00:25:47.057 INFO:tasks.workunit.client.1.smithi169.stdout:6/10: dread - no filename 2024-09-16T00:25:47.058 INFO:tasks.workunit.client.1.smithi169.stdout:1/18: write f3 [493473,17747] 0 2024-09-16T00:25:47.059 INFO:tasks.workunit.client.1.smithi169.stdout:9/0: dwrite - no filename 2024-09-16T00:25:47.059 INFO:tasks.workunit.client.1.smithi169.stdout:7/1: getdents . 0 2024-09-16T00:25:47.063 INFO:tasks.workunit.client.1.smithi169.stdout:3/8: rename f0 to f5 0 2024-09-16T00:25:47.063 INFO:tasks.workunit.client.1.smithi169.stdout:1/19: dread f0 [0,4194304] 0 2024-09-16T00:25:47.064 INFO:tasks.workunit.client.1.smithi169.stdout:3/9: chown f5 264 1 2024-09-16T00:25:47.064 INFO:tasks.workunit.client.1.smithi169.stdout:4/12: creat d2/f3 x:0 0 0 2024-09-16T00:25:47.065 INFO:tasks.workunit.client.1.smithi169.stdout:6/11: mkdir d2 0 2024-09-16T00:25:47.065 INFO:tasks.workunit.client.1.smithi169.stdout:6/12: fsync - no filename 2024-09-16T00:25:47.067 INFO:tasks.workunit.client.1.smithi169.stdout:8/0: creat f0 x:0 0 0 2024-09-16T00:25:47.068 INFO:tasks.workunit.client.1.smithi169.stdout:8/1: write f0 [17072,112690] 0 2024-09-16T00:25:47.069 INFO:tasks.workunit.client.1.smithi169.stdout:5/3: mknod d2/c3 0 2024-09-16T00:25:47.071 INFO:tasks.workunit.client.1.smithi169.stdout:8/2: dread f0 [0,4194304] 0 2024-09-16T00:25:47.071 INFO:tasks.workunit.client.1.smithi169.stdout:1/20: rename f3 to f4 0 2024-09-16T00:25:47.072 INFO:tasks.workunit.client.1.smithi169.stdout:4/13: creat d2/f4 x:0 0 0 2024-09-16T00:25:47.073 INFO:tasks.workunit.client.1.smithi169.stdout:1/21: dread f1 [0,4194304] 0 2024-09-16T00:25:47.074 INFO:tasks.workunit.client.1.smithi169.stdout:4/14: stat l0 0 2024-09-16T00:25:47.074 INFO:tasks.workunit.client.1.smithi169.stdout:9/1: creat f0 x:0 0 0 2024-09-16T00:25:47.075 INFO:tasks.workunit.client.1.smithi169.stdout:7/2: getdents . 0 2024-09-16T00:25:47.075 INFO:tasks.workunit.client.1.smithi169.stdout:7/3: write - no filename 2024-09-16T00:25:47.077 INFO:tasks.workunit.client.1.smithi169.stdout:5/4: creat d2/f4 x:0 0 0 2024-09-16T00:25:47.080 INFO:tasks.workunit.client.1.smithi169.stdout:4/15: mknod d2/c5 0 2024-09-16T00:25:47.083 INFO:tasks.workunit.client.1.smithi169.stdout:5/5: symlink d2/l5 0 2024-09-16T00:25:47.084 INFO:tasks.workunit.client.1.smithi169.stdout:8/3: dwrite f0 [0,4194304] 0 2024-09-16T00:25:47.084 INFO:tasks.workunit.client.1.smithi169.stdout:7/4: symlink l0 0 2024-09-16T00:25:47.084 INFO:tasks.workunit.client.1.smithi169.stdout:7/5: truncate - no filename 2024-09-16T00:25:47.084 INFO:tasks.workunit.client.1.smithi169.stdout:7/6: dwrite - no filename 2024-09-16T00:25:47.085 INFO:tasks.workunit.client.1.smithi169.stdout:4/16: creat d2/f6 x:0 0 0 2024-09-16T00:25:47.088 INFO:tasks.workunit.client.1.smithi169.stdout:7/7: creat f1 x:0 0 0 2024-09-16T00:25:47.090 INFO:tasks.workunit.client.1.smithi169.stdout:9/2: dwrite f0 [0,4194304] 0 2024-09-16T00:25:47.091 INFO:tasks.workunit.client.1.smithi169.stdout:5/6: link d2/f4 d2/f6 0 2024-09-16T00:25:47.091 INFO:tasks.workunit.client.1.smithi169.stdout:5/7: chown d2 43317 1 2024-09-16T00:25:47.092 INFO:tasks.workunit.client.1.smithi169.stdout:7/8: creat f2 x:0 0 0 2024-09-16T00:25:47.092 INFO:tasks.workunit.client.1.smithi169.stdout:5/8: dread - d2/f4 zero size 2024-09-16T00:25:47.092 INFO:tasks.workunit.client.1.smithi169.stdout:7/9: stat l0 0 2024-09-16T00:25:47.093 INFO:tasks.workunit.client.1.smithi169.stdout:7/10: stat l0 0 2024-09-16T00:25:47.093 INFO:tasks.workunit.client.1.smithi169.stdout:7/11: fdatasync f2 0 2024-09-16T00:25:47.094 INFO:tasks.workunit.client.1.smithi169.stdout:5/9: write f0 [247305,34363] 0 2024-09-16T00:25:47.094 INFO:tasks.workunit.client.1.smithi169.stdout:9/3: creat f1 x:0 0 0 2024-09-16T00:25:47.097 INFO:tasks.workunit.client.1.smithi169.stdout:7/12: mknod c3 0 2024-09-16T00:25:47.098 INFO:tasks.workunit.client.1.smithi169.stdout:5/10: symlink d2/l7 0 2024-09-16T00:25:47.099 INFO:tasks.workunit.client.1.smithi169.stdout:9/4: mknod c2 0 2024-09-16T00:25:47.101 INFO:tasks.workunit.client.1.smithi169.stdout:7/13: rename c3 to c4 0 2024-09-16T00:25:47.103 INFO:tasks.workunit.client.1.smithi169.stdout:7/14: dread - f1 zero size 2024-09-16T00:25:47.103 INFO:tasks.workunit.client.1.smithi169.stdout:7/15: rmdir - no directory 2024-09-16T00:25:47.106 INFO:tasks.workunit.client.1.smithi169.stdout:5/11: write d2/f6 [768332,29131] 0 2024-09-16T00:25:47.108 INFO:tasks.workunit.client.1.smithi169.stdout:7/16: write f2 [275380,112859] 0 2024-09-16T00:25:47.112 INFO:tasks.workunit.client.1.smithi169.stdout:5/12: rename d2/f6 to d2/f8 0 2024-09-16T00:25:47.115 INFO:tasks.workunit.client.1.smithi169.stdout:9/5: dwrite f1 [0,4194304] 0 2024-09-16T00:25:47.119 INFO:tasks.workunit.client.1.smithi169.stdout:5/13: link d2/l7 d2/l9 0 2024-09-16T00:25:47.124 INFO:tasks.workunit.client.1.smithi169.stdout:5/14: write d2/f8 [132788,1791] 0 2024-09-16T00:25:47.126 INFO:tasks.workunit.client.1.smithi169.stdout:5/15: write d2/f4 [530383,116819] 0 2024-09-16T00:25:47.126 INFO:tasks.workunit.client.1.smithi169.stdout:9/6: dwrite f0 [0,4194304] 0 2024-09-16T00:25:47.128 INFO:tasks.workunit.client.1.smithi169.stdout:5/16: write d2/f8 [1414877,68965] 0 2024-09-16T00:25:47.129 INFO:tasks.workunit.client.1.smithi169.stdout:9/7: mknod c3 0 2024-09-16T00:25:47.130 INFO:tasks.workunit.client.1.smithi169.stdout:5/17: mkdir d2/da 0 2024-09-16T00:25:47.132 INFO:tasks.workunit.client.1.smithi169.stdout:9/8: mknod c4 0 2024-09-16T00:25:47.133 INFO:tasks.workunit.client.1.smithi169.stdout:5/18: creat d2/fb x:0 0 0 2024-09-16T00:25:47.135 INFO:tasks.workunit.client.1.smithi169.stdout:9/9: unlink f1 0 2024-09-16T00:25:47.137 INFO:tasks.workunit.client.1.smithi169.stdout:9/10: chown f0 1002551 1 2024-09-16T00:25:47.140 INFO:tasks.workunit.client.1.smithi169.stdout:9/11: mkdir d5 0 2024-09-16T00:25:47.144 INFO:tasks.workunit.client.1.smithi169.stdout:5/19: dwrite d2/fb [0,4194304] 0 2024-09-16T00:25:47.145 INFO:tasks.workunit.client.1.smithi169.stdout:9/12: creat d5/f6 x:0 0 0 2024-09-16T00:25:47.145 INFO:tasks.workunit.client.1.smithi169.stdout:9/13: readlink - no filename 2024-09-16T00:25:47.146 INFO:tasks.workunit.client.1.smithi169.stdout:9/14: dread - d5/f6 zero size 2024-09-16T00:25:47.147 INFO:tasks.workunit.client.1.smithi169.stdout:5/20: chown d2/l9 55 1 2024-09-16T00:25:47.150 INFO:tasks.workunit.client.1.smithi169.stdout:9/15: mknod d5/c7 0 2024-09-16T00:25:47.151 INFO:tasks.workunit.client.1.smithi169.stdout:9/16: write f0 [3709309,32777] 0 2024-09-16T00:25:47.154 INFO:tasks.workunit.client.1.smithi169.stdout:9/17: write f0 [1566471,114370] 0 2024-09-16T00:25:47.161 INFO:tasks.workunit.client.1.smithi169.stdout:5/21: dwrite f0 [0,4194304] 0 2024-09-16T00:25:47.163 INFO:tasks.workunit.client.1.smithi169.stdout:5/22: write f0 [2882780,9813] 0 2024-09-16T00:25:47.166 INFO:tasks.workunit.client.1.smithi169.stdout:9/18: dwrite f0 [0,4194304] 0 2024-09-16T00:25:47.170 INFO:tasks.workunit.client.1.smithi169.stdout:9/19: creat d5/f8 x:0 0 0 2024-09-16T00:25:47.172 INFO:tasks.workunit.client.1.smithi169.stdout:9/20: mknod d5/c9 0 2024-09-16T00:25:47.175 INFO:tasks.workunit.client.1.smithi169.stdout:2/12: getdents . 0 2024-09-16T00:25:47.175 INFO:tasks.workunit.client.1.smithi169.stdout:9/21: write f0 [1069745,14491] 0 2024-09-16T00:25:47.175 INFO:tasks.workunit.client.1.smithi169.stdout:9/22: readlink - no filename 2024-09-16T00:25:47.178 INFO:tasks.workunit.client.1.smithi169.stdout:5/23: dwrite f0 [0,4194304] 0 2024-09-16T00:25:47.178 INFO:tasks.workunit.client.1.smithi169.stdout:2/13: creat f3 x:0 0 0 2024-09-16T00:25:47.184 INFO:tasks.workunit.client.1.smithi169.stdout:2/14: mknod c4 0 2024-09-16T00:25:47.184 INFO:tasks.workunit.client.1.smithi169.stdout:2/15: chown f3 1242 1 2024-09-16T00:25:47.187 INFO:tasks.workunit.client.1.smithi169.stdout:2/16: chown f3 0 1 2024-09-16T00:25:47.187 INFO:tasks.workunit.client.1.smithi169.stdout:2/17: chown c4 2694028 1 2024-09-16T00:25:47.189 INFO:tasks.workunit.client.1.smithi169.stdout:6/13: rename c1 to d2/c3 0 2024-09-16T00:25:47.190 INFO:tasks.workunit.client.1.smithi169.stdout:5/24: rename d2 to d2/dc 22 2024-09-16T00:25:47.195 INFO:tasks.workunit.client.1.smithi169.stdout:5/25: dread d2/f4 [0,4194304] 0 2024-09-16T00:25:47.197 INFO:tasks.workunit.client.1.smithi169.stdout:5/26: creat d2/fd x:0 0 0 2024-09-16T00:25:47.198 INFO:tasks.workunit.client.1.smithi169.stdout:4/17: getdents d2 0 2024-09-16T00:25:47.200 INFO:tasks.workunit.client.1.smithi169.stdout:9/23: fdatasync f0 0 2024-09-16T00:25:47.201 INFO:tasks.workunit.client.1.smithi169.stdout:4/18: symlink d2/l7 0 2024-09-16T00:25:47.201 INFO:tasks.workunit.client.1.smithi169.stdout:4/19: dread - d2/f3 zero size 2024-09-16T00:25:47.202 INFO:tasks.workunit.client.1.smithi169.stdout:4/20: read - d2/f4 zero size 2024-09-16T00:25:47.203 INFO:tasks.workunit.client.1.smithi169.stdout:5/27: unlink d2/l7 0 2024-09-16T00:25:47.203 INFO:tasks.workunit.client.1.smithi169.stdout:4/21: chown l0 3257241 1 2024-09-16T00:25:47.204 INFO:tasks.workunit.client.1.smithi169.stdout:4/22: chown d2/f3 8407964 1 2024-09-16T00:25:47.207 INFO:tasks.workunit.client.1.smithi169.stdout:5/28: stat d2/l9 0 2024-09-16T00:25:47.210 INFO:tasks.workunit.client.1.smithi169.stdout:9/24: dwrite d5/f8 [0,4194304] 0 2024-09-16T00:25:47.211 INFO:tasks.workunit.client.1.smithi169.stdout:5/29: write d2/f8 [859726,116080] 0 2024-09-16T00:25:47.215 INFO:tasks.workunit.client.1.smithi169.stdout:7/17: chown c4 900 1 2024-09-16T00:25:47.215 INFO:tasks.workunit.client.1.smithi169.stdout:7/18: rmdir - no directory 2024-09-16T00:25:47.216 INFO:tasks.workunit.client.1.smithi169.stdout:5/30: creat d2/fe x:0 0 0 2024-09-16T00:25:47.218 INFO:tasks.workunit.client.1.smithi169.stdout:4/23: dwrite d2/f4 [0,4194304] 0 2024-09-16T00:25:47.222 INFO:tasks.workunit.client.1.smithi169.stdout:5/31: dread d2/fb [0,4194304] 0 2024-09-16T00:25:47.224 INFO:tasks.workunit.client.1.smithi169.stdout:7/19: getdents . 0 2024-09-16T00:25:47.224 INFO:tasks.workunit.client.1.smithi169.stdout:7/20: rmdir - no directory 2024-09-16T00:25:47.226 INFO:tasks.workunit.client.1.smithi169.stdout:5/32: write d2/fb [2109655,129473] 0 2024-09-16T00:25:47.227 INFO:tasks.workunit.client.1.smithi169.stdout:7/21: creat f5 x:0 0 0 2024-09-16T00:25:47.227 INFO:tasks.workunit.client.1.smithi169.stdout:7/22: dread - f5 zero size 2024-09-16T00:25:47.228 INFO:tasks.workunit.client.1.smithi169.stdout:7/23: write f1 [399662,129655] 0 2024-09-16T00:25:47.228 INFO:tasks.workunit.client.1.smithi169.stdout:7/24: chown c4 37 1 2024-09-16T00:25:47.358 INFO:tasks.workunit.client.1.smithi169.stdout:5/33: creat d2/ff x:0 0 0 2024-09-16T00:25:47.359 INFO:tasks.workunit.client.1.smithi169.stdout:9/25: fsync d5/f8 0 2024-09-16T00:25:47.359 INFO:tasks.workunit.client.1.smithi169.stdout:2/18: fsync f3 0 2024-09-16T00:25:47.360 INFO:tasks.workunit.client.1.smithi169.stdout:5/34: write d2/fe [131420,76266] 0 2024-09-16T00:25:47.361 INFO:tasks.workunit.client.1.smithi169.stdout:4/24: getdents d2 0 2024-09-16T00:25:47.458 INFO:tasks.workunit.client.1.smithi169.stdout:9/26: unlink f0 0 2024-09-16T00:25:47.458 INFO:tasks.workunit.client.1.smithi169.stdout:2/19: creat f5 x:0 0 0 2024-09-16T00:25:47.459 INFO:tasks.workunit.client.1.smithi169.stdout:6/14: rename d2/c3 to d2/c4 0 2024-09-16T00:25:47.459 INFO:tasks.workunit.client.1.smithi169.stdout:6/15: write - no filename 2024-09-16T00:25:47.463 INFO:tasks.workunit.client.1.smithi169.stdout:4/25: unlink d2/f6 0 2024-09-16T00:25:47.464 INFO:tasks.workunit.client.1.smithi169.stdout:9/27: mkdir d5/da 0 2024-09-16T00:25:47.465 INFO:tasks.workunit.client.1.smithi169.stdout:6/16: rmdir d2 39 2024-09-16T00:25:47.465 INFO:tasks.workunit.client.1.smithi169.stdout:6/17: dread - no filename 2024-09-16T00:25:47.465 INFO:tasks.workunit.client.1.smithi169.stdout:6/18: dread - no filename 2024-09-16T00:25:47.466 INFO:tasks.workunit.client.1.smithi169.stdout:6/19: write - no filename 2024-09-16T00:25:47.466 INFO:tasks.workunit.client.1.smithi169.stdout:5/35: mkdir d2/da/d10 0 2024-09-16T00:25:47.467 INFO:tasks.workunit.client.1.smithi169.stdout:4/26: symlink d2/l8 0 2024-09-16T00:25:47.468 INFO:tasks.workunit.client.1.smithi169.stdout:9/28: creat d5/fb x:0 0 0 2024-09-16T00:25:47.469 INFO:tasks.workunit.client.1.smithi169.stdout:4/27: write d2/f4 [2623707,81315] 0 2024-09-16T00:25:47.470 INFO:tasks.workunit.client.1.smithi169.stdout:2/20: link f0 f6 0 2024-09-16T00:25:47.472 INFO:tasks.workunit.client.1.smithi169.stdout:4/28: fsync d2/f4 0 2024-09-16T00:25:47.472 INFO:tasks.workunit.client.1.smithi169.stdout:4/29: readlink l0 0 2024-09-16T00:25:47.473 INFO:tasks.workunit.client.1.smithi169.stdout:4/30: read d2/f4 [994176,20685] 0 2024-09-16T00:25:47.476 INFO:tasks.workunit.client.1.smithi169.stdout:9/29: creat d5/da/fc x:0 0 0 2024-09-16T00:25:47.477 INFO:tasks.workunit.client.1.smithi169.stdout:4/31: dread d2/f4 [0,4194304] 0 2024-09-16T00:25:47.478 INFO:tasks.workunit.client.1.smithi169.stdout:9/30: truncate d5/fb 712106 0 2024-09-16T00:25:47.478 INFO:tasks.workunit.client.1.smithi169.stdout:9/31: write d5/f6 [538843,670] 0 2024-09-16T00:25:47.479 INFO:tasks.workunit.client.1.smithi169.stdout:9/32: fsync d5/f8 0 2024-09-16T00:25:47.480 INFO:tasks.workunit.client.1.smithi169.stdout:4/32: creat d2/f9 x:0 0 0 2024-09-16T00:25:47.482 INFO:tasks.workunit.client.1.smithi169.stdout:4/33: mknod d2/ca 0 2024-09-16T00:25:47.483 INFO:tasks.workunit.client.1.smithi169.stdout:4/34: creat d2/fb x:0 0 0 2024-09-16T00:25:47.484 INFO:tasks.workunit.client.1.smithi169.stdout:9/33: dread d5/f8 [0,4194304] 0 2024-09-16T00:25:47.485 INFO:tasks.workunit.client.1.smithi169.stdout:4/35: symlink d2/lc 0 2024-09-16T00:25:47.486 INFO:tasks.workunit.client.1.smithi169.stdout:9/34: creat d5/da/fd x:0 0 0 2024-09-16T00:25:47.487 INFO:tasks.workunit.client.1.smithi169.stdout:9/35: write d5/f8 [1450755,94819] 0 2024-09-16T00:25:47.487 INFO:tasks.workunit.client.1.smithi169.stdout:4/36: mkdir d2/dd 0 2024-09-16T00:25:47.489 INFO:tasks.workunit.client.1.smithi169.stdout:4/37: unlink d2/lc 0 2024-09-16T00:25:47.490 INFO:tasks.workunit.client.1.smithi169.stdout:9/36: mknod d5/ce 0 2024-09-16T00:25:47.496 INFO:tasks.workunit.client.1.smithi169.stdout:9/37: link d5/da/fd d5/da/ff 0 2024-09-16T00:25:47.496 INFO:tasks.workunit.client.1.smithi169.stdout:9/38: truncate d5/fb 1079817 0 2024-09-16T00:25:47.496 INFO:tasks.workunit.client.1.smithi169.stdout:9/39: read - d5/da/ff zero size 2024-09-16T00:25:47.496 INFO:tasks.workunit.client.1.smithi169.stdout:9/40: creat d5/da/f10 x:0 0 0 2024-09-16T00:25:47.497 INFO:tasks.workunit.client.1.smithi169.stdout:9/41: symlink d5/l11 0 2024-09-16T00:25:47.499 INFO:tasks.workunit.client.1.smithi169.stdout:9/42: mkdir d5/d12 0 2024-09-16T00:25:47.500 INFO:tasks.workunit.client.1.smithi169.stdout:9/43: chown d5/da/fc 384 1 2024-09-16T00:25:47.502 INFO:tasks.workunit.client.1.smithi169.stdout:9/44: link d5/fb d5/d12/f13 0 2024-09-16T00:25:47.503 INFO:tasks.workunit.client.1.smithi169.stdout:9/45: dread - d5/da/fc zero size 2024-09-16T00:25:47.546 INFO:tasks.workunit.client.1.smithi169.stdout:6/20: link d2/c4 d2/c5 0 2024-09-16T00:25:47.547 INFO:tasks.workunit.client.1.smithi169.stdout:6/21: readlink - no filename 2024-09-16T00:25:47.547 INFO:tasks.workunit.client.1.smithi169.stdout:6/22: write - no filename 2024-09-16T00:25:47.547 INFO:tasks.workunit.client.1.smithi169.stdout:6/23: dread - no filename 2024-09-16T00:25:47.548 INFO:tasks.workunit.client.1.smithi169.stdout:5/36: truncate d2/fd 501450 0 2024-09-16T00:25:47.549 INFO:tasks.workunit.client.1.smithi169.stdout:5/37: truncate d2/fd 506896 0 2024-09-16T00:25:47.550 INFO:tasks.workunit.client.1.smithi169.stdout:5/38: stat d2/fb 0 2024-09-16T00:25:47.552 INFO:tasks.workunit.client.1.smithi169.stdout:5/39: unlink d2/c3 0 2024-09-16T00:25:47.554 INFO:tasks.workunit.client.1.smithi169.stdout:5/40: chown d2/l5 82 1 2024-09-16T00:25:47.561 INFO:tasks.workunit.client.1.smithi169.stdout:5/41: dwrite d2/fb [4194304,4194304] 0 2024-09-16T00:25:47.564 INFO:tasks.workunit.client.1.smithi169.stdout:5/42: dread d2/f4 [0,4194304] 0 2024-09-16T00:25:47.564 INFO:tasks.workunit.client.1.smithi169.stdout:5/43: chown d2/l5 197468609 1 2024-09-16T00:25:48.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:25:48 smithi112 ceph-mon[27869]: pgmap v247: 65 pgs: 65 active+clean; 220 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 2.9 MiB/s rd, 7.3 MiB/s wr, 471 op/s 2024-09-16T00:25:48.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:25:48 smithi169 ceph-mon[34653]: pgmap v247: 65 pgs: 65 active+clean; 220 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 2.9 MiB/s rd, 7.3 MiB/s wr, 471 op/s 2024-09-16T00:25:49.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:25:49 smithi112 ceph-mon[27869]: Standby manager daemon smithi169.mtrobk restarted 2024-09-16T00:25:49.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:25:49 smithi112 ceph-mon[27869]: Standby manager daemon smithi169.mtrobk started 2024-09-16T00:25:49.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:25:49 smithi112 ceph-mon[27869]: from='mgr.? 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi169.mtrobk/crt"}]: dispatch 2024-09-16T00:25:49.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:25:49 smithi112 ceph-mon[27869]: from='mgr.? 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T00:25:49.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:25:49 smithi112 ceph-mon[27869]: from='mgr.? 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi169.mtrobk/key"}]: dispatch 2024-09-16T00:25:49.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:25:49 smithi112 ceph-mon[27869]: from='mgr.? 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T00:25:49.402 INFO:tasks.workunit.client.0.smithi112.stdout:2/0: sync 2024-09-16T00:25:49.402 INFO:tasks.workunit.client.0.smithi112.stdout:2/1: fdatasync - no filename 2024-09-16T00:25:49.402 INFO:tasks.workunit.client.0.smithi112.stdout:2/2: chown . 1673 1 2024-09-16T00:25:49.402 INFO:tasks.workunit.client.0.smithi112.stdout:2/3: dread - no filename 2024-09-16T00:25:49.402 INFO:tasks.workunit.client.0.smithi112.stdout:2/4: rmdir - no directory 2024-09-16T00:25:49.403 INFO:tasks.workunit.client.0.smithi112.stdout:2/5: mknod c0 0 2024-09-16T00:25:49.404 INFO:tasks.workunit.client.0.smithi112.stdout:2/6: fsync - no filename 2024-09-16T00:25:49.404 INFO:tasks.workunit.client.0.smithi112.stdout:2/7: write - no filename 2024-09-16T00:25:49.404 INFO:tasks.workunit.client.0.smithi112.stdout:2/8: write - no filename 2024-09-16T00:25:49.405 INFO:tasks.workunit.client.0.smithi112.stdout:2/9: creat f1 x:0 0 0 2024-09-16T00:25:49.405 INFO:tasks.workunit.client.0.smithi112.stdout:2/10: fdatasync f1 0 2024-09-16T00:25:49.406 INFO:tasks.workunit.client.0.smithi112.stdout:2/11: chown f1 135350 1 2024-09-16T00:25:49.407 INFO:tasks.workunit.client.0.smithi112.stdout:2/12: mkdir d2 0 2024-09-16T00:25:49.409 INFO:tasks.workunit.client.0.smithi112.stdout:2/13: creat d2/f3 x:0 0 0 2024-09-16T00:25:49.409 INFO:tasks.workunit.client.0.smithi112.stdout:2/14: chown d2/f3 1166379587 1 2024-09-16T00:25:49.418 INFO:tasks.workunit.client.0.smithi112.stdout:2/15: dwrite f1 [0,4194304] 0 2024-09-16T00:25:49.420 INFO:tasks.workunit.client.0.smithi112.stdout:2/16: creat d2/f4 x:0 0 0 2024-09-16T00:25:49.422 INFO:tasks.workunit.client.0.smithi112.stdout:2/17: unlink d2/f3 0 2024-09-16T00:25:49.423 INFO:tasks.workunit.client.0.smithi112.stdout:2/18: symlink d2/l5 0 2024-09-16T00:25:49.424 INFO:tasks.workunit.client.0.smithi112.stdout:2/19: write f1 [2864927,11598] 0 2024-09-16T00:25:49.427 INFO:tasks.workunit.client.0.smithi112.stdout:2/20: symlink d2/l6 0 2024-09-16T00:25:49.441 INFO:tasks.workunit.client.0.smithi112.stdout:2/21: fdatasync f1 0 2024-09-16T00:25:49.444 INFO:tasks.workunit.client.0.smithi112.stdout:2/22: dread f1 [0,4194304] 0 2024-09-16T00:25:49.446 INFO:tasks.workunit.client.0.smithi112.stdout:2/23: symlink d2/l7 0 2024-09-16T00:25:49.447 INFO:tasks.workunit.client.0.smithi112.stdout:2/24: truncate d2/f4 1023386 0 2024-09-16T00:25:49.449 INFO:tasks.workunit.client.0.smithi112.stdout:2/25: rename d2/f4 to d2/f8 0 2024-09-16T00:25:49.449 INFO:tasks.workunit.client.0.smithi112.stdout:2/26: chown d2/l7 613472535 1 2024-09-16T00:25:49.450 INFO:tasks.workunit.client.0.smithi112.stdout:2/27: rename d2 to d2/d9 22 2024-09-16T00:25:49.451 INFO:tasks.workunit.client.0.smithi112.stdout:2/28: creat d2/fa x:0 0 0 2024-09-16T00:25:49.458 INFO:tasks.workunit.client.0.smithi112.stdout:2/29: dwrite f1 [0,4194304] 0 2024-09-16T00:25:49.459 INFO:tasks.workunit.client.0.smithi112.stdout:2/30: write d2/fa [1029988,82669] 0 2024-09-16T00:25:49.462 INFO:tasks.workunit.client.0.smithi112.stdout:2/31: symlink d2/lb 0 2024-09-16T00:25:49.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:25:49 smithi169 ceph-mon[34653]: Standby manager daemon smithi169.mtrobk restarted 2024-09-16T00:25:49.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:25:49 smithi169 ceph-mon[34653]: Standby manager daemon smithi169.mtrobk started 2024-09-16T00:25:49.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:25:49 smithi169 ceph-mon[34653]: from='mgr.? 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi169.mtrobk/crt"}]: dispatch 2024-09-16T00:25:49.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:25:49 smithi169 ceph-mon[34653]: from='mgr.? 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T00:25:49.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:25:49 smithi169 ceph-mon[34653]: from='mgr.? 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi169.mtrobk/key"}]: dispatch 2024-09-16T00:25:49.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:25:49 smithi169 ceph-mon[34653]: from='mgr.? 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T00:25:49.564 INFO:tasks.workunit.client.0.smithi112.stdout:9/35: sync 2024-09-16T00:25:49.565 INFO:tasks.workunit.client.0.smithi112.stdout:9/36: write d1/d5/d6/f8 [4134745,78611] 0 2024-09-16T00:25:49.571 INFO:tasks.workunit.client.0.smithi112.stdout:9/37: dwrite d1/d5/d6/f8 [4194304,4194304] 0 2024-09-16T00:25:49.572 INFO:tasks.workunit.client.0.smithi112.stdout:9/38: rename d1/d5/d6/dd to d1/d5/d6/dd/df 22 2024-09-16T00:25:49.579 INFO:tasks.workunit.client.0.smithi112.stdout:9/39: dwrite d1/d5/d6/f8 [4194304,4194304] 0 2024-09-16T00:25:49.581 INFO:tasks.workunit.client.0.smithi112.stdout:9/40: chown l0 5740370 1 2024-09-16T00:25:49.582 INFO:tasks.workunit.client.0.smithi112.stdout:9/41: write d1/d5/d6/f8 [1096718,100264] 0 2024-09-16T00:25:49.587 INFO:tasks.workunit.client.0.smithi112.stdout:9/42: dread d1/d5/d6/f8 [0,4194304] 0 2024-09-16T00:25:49.589 INFO:tasks.workunit.client.0.smithi112.stdout:9/43: mkdir d1/d10 0 2024-09-16T00:25:49.595 INFO:tasks.workunit.client.0.smithi112.stdout:9/44: dwrite d1/d5/d6/f8 [8388608,4194304] 0 2024-09-16T00:25:49.597 INFO:tasks.workunit.client.0.smithi112.stdout:9/45: getdents d1/d5 0 2024-09-16T00:25:49.606 INFO:tasks.workunit.client.0.smithi112.stdout:9/46: dwrite d1/d5/d6/f8 [0,4194304] 0 2024-09-16T00:25:49.609 INFO:tasks.workunit.client.0.smithi112.stdout:9/47: rename d1/d5/da to d1/d5/d6/d11 0 2024-09-16T00:25:49.617 INFO:tasks.workunit.client.0.smithi112.stdout:9/48: getdents d1/d5 0 2024-09-16T00:25:49.617 INFO:tasks.workunit.client.0.smithi112.stdout:9/49: read d1/d5/d6/f8 [293090,40165] 0 2024-09-16T00:25:49.618 INFO:tasks.workunit.client.0.smithi112.stdout:9/50: chown d1/d5/d6/d11/lb 643092 1 2024-09-16T00:25:49.619 INFO:tasks.workunit.client.0.smithi112.stdout:9/51: write d1/d5/d6/f8 [4245985,24245] 0 2024-09-16T00:25:49.621 INFO:tasks.workunit.client.0.smithi112.stdout:9/52: rename d1/d5/l9 to d1/d5/l12 0 2024-09-16T00:25:49.622 INFO:tasks.workunit.client.0.smithi112.stdout:9/53: readlink d1/d5/d6/d11/lb 0 2024-09-16T00:25:49.625 INFO:tasks.workunit.client.0.smithi112.stdout:9/54: link d1/ce d1/c13 0 2024-09-16T00:25:49.627 INFO:tasks.workunit.client.0.smithi112.stdout:9/55: link d1/ce d1/c14 0 2024-09-16T00:25:49.629 INFO:tasks.workunit.client.0.smithi112.stdout:9/56: creat d1/d5/d6/dd/f15 x:0 0 0 2024-09-16T00:25:49.630 INFO:tasks.workunit.client.0.smithi112.stdout:9/57: readlink l0 0 2024-09-16T00:25:49.632 INFO:tasks.workunit.client.0.smithi112.stdout:9/58: rmdir d1/d10 0 2024-09-16T00:25:49.633 INFO:tasks.workunit.client.0.smithi112.stdout:9/59: write d1/d5/d6/f8 [8389229,17649] 0 2024-09-16T00:25:49.633 INFO:tasks.workunit.client.0.smithi112.stdout:9/60: stat d1 0 2024-09-16T00:25:49.636 INFO:tasks.workunit.client.0.smithi112.stdout:9/61: rename d1/d5/d6/dd/f15 to d1/d5/f16 0 2024-09-16T00:25:49.637 INFO:tasks.workunit.client.0.smithi112.stdout:9/62: symlink d1/d5/d6/l17 0 2024-09-16T00:25:49.639 INFO:tasks.workunit.client.0.smithi112.stdout:9/63: mkdir d1/d5/d6/d18 0 2024-09-16T00:25:49.640 INFO:tasks.workunit.client.0.smithi112.stdout:9/64: readlink l0 0 2024-09-16T00:25:49.640 INFO:tasks.workunit.client.0.smithi112.stdout:9/65: dread - d1/d5/f16 zero size 2024-09-16T00:25:49.645 INFO:tasks.workunit.client.0.smithi112.stdout:9/66: fsync d1/d5/f16 0 2024-09-16T00:25:49.650 INFO:tasks.workunit.client.0.smithi112.stdout:9/67: dwrite d1/d5/d6/f8 [4194304,4194304] 0 2024-09-16T00:25:49.665 INFO:tasks.workunit.client.0.smithi112.stdout:9/68: fsync d1/d5/d6/f8 0 2024-09-16T00:25:49.667 INFO:tasks.workunit.client.0.smithi112.stdout:9/69: creat d1/d5/d6/d18/f19 x:0 0 0 2024-09-16T00:25:49.667 INFO:tasks.workunit.client.0.smithi112.stdout:9/70: dread - d1/d5/f16 zero size 2024-09-16T00:25:49.669 INFO:tasks.workunit.client.0.smithi112.stdout:9/71: mkdir d1/d5/d6/d11/d1a 0 2024-09-16T00:25:49.670 INFO:tasks.workunit.client.0.smithi112.stdout:9/72: creat d1/f1b x:0 0 0 2024-09-16T00:25:49.675 INFO:tasks.workunit.client.0.smithi112.stdout:9/73: dwrite d1/d5/d6/f8 [8388608,4194304] 0 2024-09-16T00:25:49.675 INFO:tasks.workunit.client.0.smithi112.stdout:4/34: sync 2024-09-16T00:25:49.675 INFO:tasks.workunit.client.0.smithi112.stdout:1/25: sync 2024-09-16T00:25:49.676 INFO:tasks.workunit.client.0.smithi112.stdout:9/74: write d1/d5/f16 [198957,112877] 0 2024-09-16T00:25:49.677 INFO:tasks.workunit.client.0.smithi112.stdout:4/35: write d6/f9 [3529922,66491] 0 2024-09-16T00:25:49.678 INFO:tasks.workunit.client.0.smithi112.stdout:9/75: write d1/d5/f16 [1166665,57565] 0 2024-09-16T00:25:49.681 INFO:tasks.workunit.client.0.smithi112.stdout:9/76: creat d1/d5/d6/dd/f1c x:0 0 0 2024-09-16T00:25:49.684 INFO:tasks.workunit.client.0.smithi112.stdout:9/77: creat d1/d5/d6/d11/f1d x:0 0 0 2024-09-16T00:25:49.690 INFO:tasks.workunit.client.0.smithi112.stdout:9/78: dwrite d1/d5/d6/d18/f19 [0,4194304] 0 2024-09-16T00:25:49.698 INFO:tasks.workunit.client.0.smithi112.stdout:4/36: truncate d6/f9 1963732 0 2024-09-16T00:25:49.699 INFO:tasks.workunit.client.0.smithi112.stdout:9/79: dwrite d1/d5/d6/dd/f1c [0,4194304] 0 2024-09-16T00:25:49.701 INFO:tasks.workunit.client.0.smithi112.stdout:9/80: mknod d1/d5/d6/d11/c1e 0 2024-09-16T00:25:49.705 INFO:tasks.workunit.client.0.smithi112.stdout:9/81: creat d1/d5/d6/d18/f1f x:0 0 0 2024-09-16T00:25:49.706 INFO:tasks.workunit.client.0.smithi112.stdout:4/37: dwrite f4 [0,4194304] 0 2024-09-16T00:25:49.707 INFO:tasks.workunit.client.0.smithi112.stdout:9/82: mkdir d1/d5/d20 0 2024-09-16T00:25:49.710 INFO:tasks.workunit.client.0.smithi112.stdout:4/38: unlink d6/l7 0 2024-09-16T00:25:49.718 INFO:tasks.workunit.client.0.smithi112.stdout:9/83: dwrite d1/d5/d6/d11/f1d [0,4194304] 0 2024-09-16T00:25:49.737 INFO:tasks.workunit.client.0.smithi112.stdout:4/39: fsync f4 0 2024-09-16T00:25:49.738 INFO:tasks.workunit.client.0.smithi112.stdout:4/40: symlink d6/ld 0 2024-09-16T00:25:49.740 INFO:tasks.workunit.client.0.smithi112.stdout:9/84: truncate d1/d5/f16 663898 0 2024-09-16T00:25:49.746 INFO:tasks.workunit.client.0.smithi112.stdout:4/41: dwrite f4 [4194304,4194304] 0 2024-09-16T00:25:49.748 INFO:tasks.workunit.client.0.smithi112.stdout:9/85: dread d1/d5/d6/d18/f19 [0,4194304] 0 2024-09-16T00:25:49.749 INFO:tasks.workunit.client.0.smithi112.stdout:4/42: mknod d6/ce 0 2024-09-16T00:25:49.760 INFO:tasks.workunit.client.0.smithi112.stdout:9/86: truncate d1/d5/d6/d18/f19 3328092 0 2024-09-16T00:25:49.761 INFO:tasks.workunit.client.0.smithi112.stdout:4/43: truncate f4 6367368 0 2024-09-16T00:25:49.763 INFO:tasks.workunit.client.0.smithi112.stdout:9/87: rename d1/d5/d6/f8 to d1/d5/d6/d11/f21 0 2024-09-16T00:25:49.764 INFO:tasks.workunit.client.0.smithi112.stdout:4/44: mkdir d6/db/df 0 2024-09-16T00:25:49.764 INFO:tasks.workunit.client.0.smithi112.stdout:9/88: write d1/d5/d6/d11/f1d [2478857,42976] 0 2024-09-16T00:25:49.766 INFO:tasks.workunit.client.0.smithi112.stdout:4/45: fdatasync d6/f9 0 2024-09-16T00:25:49.769 INFO:tasks.workunit.client.0.smithi112.stdout:4/46: mknod d6/da/c10 0 2024-09-16T00:25:49.769 INFO:tasks.workunit.client.0.smithi112.stdout:9/89: dread d1/d5/d6/dd/f1c [0,4194304] 0 2024-09-16T00:25:49.770 INFO:tasks.workunit.client.0.smithi112.stdout:4/47: read d6/f9 [1405130,42335] 0 2024-09-16T00:25:49.770 INFO:tasks.workunit.client.0.smithi112.stdout:9/90: chown d1/d5/d6/d18 381 1 2024-09-16T00:25:49.772 INFO:tasks.workunit.client.0.smithi112.stdout:9/91: read d1/d5/d6/dd/f1c [3403635,67262] 0 2024-09-16T00:25:49.776 INFO:tasks.workunit.client.0.smithi112.stdout:9/92: rename d1/c2 to d1/d5/d6/dd/c22 0 2024-09-16T00:25:49.776 INFO:tasks.workunit.client.0.smithi112.stdout:4/48: dwrite f4 [0,4194304] 0 2024-09-16T00:25:49.777 INFO:tasks.workunit.client.0.smithi112.stdout:4/49: chown f4 214 1 2024-09-16T00:25:49.778 INFO:tasks.workunit.client.0.smithi112.stdout:9/93: write d1/d5/d6/dd/f1c [4673681,43885] 0 2024-09-16T00:25:49.780 INFO:tasks.workunit.client.0.smithi112.stdout:4/50: mknod d6/db/df/c11 0 2024-09-16T00:25:49.786 INFO:tasks.workunit.client.0.smithi112.stdout:4/51: dwrite f4 [0,4194304] 0 2024-09-16T00:25:49.787 INFO:tasks.workunit.client.0.smithi112.stdout:9/94: truncate d1/d5/d6/dd/f1c 728097 0 2024-09-16T00:25:49.788 INFO:tasks.workunit.client.0.smithi112.stdout:4/52: symlink d6/db/df/l12 0 2024-09-16T00:25:49.790 INFO:tasks.workunit.client.0.smithi112.stdout:9/95: mkdir d1/d5/d6/d18/d23 0 2024-09-16T00:25:49.791 INFO:tasks.workunit.client.0.smithi112.stdout:9/96: stat d1 0 2024-09-16T00:25:49.791 INFO:tasks.workunit.client.0.smithi112.stdout:4/53: dread d6/f9 [0,4194304] 0 2024-09-16T00:25:49.792 INFO:tasks.workunit.client.0.smithi112.stdout:4/54: write f4 [2821599,98547] 0 2024-09-16T00:25:49.792 INFO:tasks.workunit.client.0.smithi112.stdout:9/97: write d1/d5/d6/d18/f1f [19215,68183] 0 2024-09-16T00:25:49.795 INFO:tasks.workunit.client.0.smithi112.stdout:4/55: unlink d6/l8 0 2024-09-16T00:25:49.795 INFO:tasks.workunit.client.0.smithi112.stdout:9/98: mknod d1/d5/d6/d11/c24 0 2024-09-16T00:25:49.796 INFO:tasks.workunit.client.0.smithi112.stdout:9/99: readlink d1/d5/d6/l17 0 2024-09-16T00:25:49.796 INFO:tasks.workunit.client.0.smithi112.stdout:4/56: mkdir d6/db/d13 0 2024-09-16T00:25:49.797 INFO:tasks.workunit.client.0.smithi112.stdout:9/100: rmdir d1/d5 39 2024-09-16T00:25:49.798 INFO:tasks.workunit.client.0.smithi112.stdout:4/57: symlink d6/db/d13/l14 0 2024-09-16T00:25:49.799 INFO:tasks.workunit.client.0.smithi112.stdout:9/101: unlink l0 0 2024-09-16T00:25:49.800 INFO:tasks.workunit.client.0.smithi112.stdout:4/58: mkdir d6/db/d15 0 2024-09-16T00:25:49.818 INFO:tasks.workunit.client.0.smithi112.stdout:4/59: creat d6/da/f16 x:0 0 0 2024-09-16T00:25:49.819 INFO:tasks.workunit.client.0.smithi112.stdout:4/60: dread - d6/da/f16 zero size 2024-09-16T00:25:49.821 INFO:tasks.workunit.client.0.smithi112.stdout:8/41: sync 2024-09-16T00:25:49.821 INFO:tasks.workunit.client.0.smithi112.stdout:7/27: sync 2024-09-16T00:25:49.821 INFO:tasks.workunit.client.0.smithi112.stdout:2/32: sync 2024-09-16T00:25:49.821 INFO:tasks.workunit.client.0.smithi112.stdout:6/82: sync 2024-09-16T00:25:49.821 INFO:tasks.workunit.client.0.smithi112.stdout:0/33: sync 2024-09-16T00:25:49.821 INFO:tasks.workunit.client.0.smithi112.stdout:5/59: sync 2024-09-16T00:25:49.821 INFO:tasks.workunit.client.0.smithi112.stdout:3/40: sync 2024-09-16T00:25:49.822 INFO:tasks.workunit.client.0.smithi112.stdout:7/28: fsync f5 0 2024-09-16T00:25:49.822 INFO:tasks.workunit.client.0.smithi112.stdout:2/33: fdatasync f1 0 2024-09-16T00:25:49.826 INFO:tasks.workunit.client.0.smithi112.stdout:7/29: dread f4 [0,4194304] 0 2024-09-16T00:25:49.827 INFO:tasks.workunit.client.0.smithi112.stdout:2/34: dwrite f1 [0,4194304] 0 2024-09-16T00:25:49.852 INFO:tasks.workunit.client.0.smithi112.stdout:1/26: sync 2024-09-16T00:25:49.853 INFO:tasks.workunit.client.0.smithi112.stdout:1/27: write d0/f2 [4020020,88635] 0 2024-09-16T00:25:49.854 INFO:tasks.workunit.client.0.smithi112.stdout:1/28: chown d0 169197147 1 2024-09-16T00:25:49.859 INFO:tasks.workunit.client.0.smithi112.stdout:1/29: dwrite d0/f2 [0,4194304] 0 2024-09-16T00:25:49.859 INFO:tasks.workunit.client.0.smithi112.stdout:1/30: write d0/f2 [4456155,84229] 0 2024-09-16T00:25:49.909 INFO:tasks.workunit.client.0.smithi112.stdout:8/42: mkdir d10 0 2024-09-16T00:25:49.909 INFO:tasks.workunit.client.0.smithi112.stdout:8/43: dread - fe zero size 2024-09-16T00:25:49.910 INFO:tasks.workunit.client.0.smithi112.stdout:5/60: dread d4/fb [0,4194304] 0 2024-09-16T00:25:49.914 INFO:tasks.workunit.client.0.smithi112.stdout:2/35: mkdir d2/dc 0 2024-09-16T00:25:49.917 INFO:tasks.workunit.client.0.smithi112.stdout:5/61: dwrite d4/d7/d14/f1a [0,4194304] 0 2024-09-16T00:25:49.918 INFO:tasks.workunit.client.0.smithi112.stdout:5/62: fdatasync d4/f16 0 2024-09-16T00:25:49.918 INFO:tasks.workunit.client.0.smithi112.stdout:1/31: rename d0/d3/f6 to d0/d3/f8 0 2024-09-16T00:25:49.919 INFO:tasks.workunit.client.0.smithi112.stdout:4/61: unlink l3 0 2024-09-16T00:25:49.922 INFO:tasks.workunit.client.0.smithi112.stdout:8/44: unlink f9 0 2024-09-16T00:25:49.924 INFO:tasks.workunit.client.0.smithi112.stdout:3/41: truncate d2/f5 2147288 0 2024-09-16T00:25:49.925 INFO:tasks.workunit.client.0.smithi112.stdout:3/42: truncate d2/f6 355334 0 2024-09-16T00:25:49.925 INFO:tasks.workunit.client.0.smithi112.stdout:0/34: mkdir d5/da/dc 0 2024-09-16T00:25:49.926 INFO:tasks.workunit.client.0.smithi112.stdout:3/43: read - d2/f8 zero size 2024-09-16T00:25:49.931 INFO:tasks.workunit.client.0.smithi112.stdout:7/30: creat d6/fa x:0 0 0 2024-09-16T00:25:49.932 INFO:tasks.workunit.client.0.smithi112.stdout:7/31: stat d6 0 2024-09-16T00:25:49.935 INFO:tasks.workunit.client.0.smithi112.stdout:5/63: mkdir d4/d1c 0 2024-09-16T00:25:49.935 INFO:tasks.workunit.client.0.smithi112.stdout:0/35: dwrite f4 [0,4194304] 0 2024-09-16T00:25:49.939 INFO:tasks.workunit.client.0.smithi112.stdout:8/45: fdatasync f4 0 2024-09-16T00:25:49.941 INFO:tasks.workunit.client.0.smithi112.stdout:2/36: dwrite d2/f8 [0,4194304] 0 2024-09-16T00:25:49.942 INFO:tasks.workunit.client.0.smithi112.stdout:3/44: symlink d2/la 0 2024-09-16T00:25:49.943 INFO:tasks.workunit.client.0.smithi112.stdout:3/45: readlink d2/la 0 2024-09-16T00:25:49.943 INFO:tasks.workunit.client.0.smithi112.stdout:6/83: link l7 dd/df/d10/d1d/l1f 0 2024-09-16T00:25:49.944 INFO:tasks.workunit.client.0.smithi112.stdout:3/46: read d2/f6 [253893,14599] 0 2024-09-16T00:25:49.945 INFO:tasks.workunit.client.0.smithi112.stdout:7/32: creat d6/fb x:0 0 0 2024-09-16T00:25:49.946 INFO:tasks.workunit.client.0.smithi112.stdout:5/64: stat d4/dd/d12/l13 0 2024-09-16T00:25:49.947 INFO:tasks.workunit.client.0.smithi112.stdout:5/65: readlink d4/dd/d12/l13 0 2024-09-16T00:25:49.948 INFO:tasks.workunit.client.0.smithi112.stdout:4/62: creat d6/db/d15/f17 x:0 0 0 2024-09-16T00:25:49.950 INFO:tasks.workunit.client.0.smithi112.stdout:2/37: mkdir d2/dd 0 2024-09-16T00:25:49.954 INFO:tasks.workunit.client.0.smithi112.stdout:1/32: rmdir d0/d3/d4 0 2024-09-16T00:25:49.955 INFO:tasks.workunit.client.0.smithi112.stdout:7/33: symlink d6/lc 0 2024-09-16T00:25:49.956 INFO:tasks.workunit.client.0.smithi112.stdout:0/36: getdents d5/da/dc 0 2024-09-16T00:25:49.961 INFO:tasks.workunit.client.0.smithi112.stdout:7/34: mknod d6/cd 0 2024-09-16T00:25:49.962 INFO:tasks.workunit.client.0.smithi112.stdout:5/66: truncate d4/d7/f8 2725953 0 2024-09-16T00:25:49.963 INFO:tasks.workunit.client.0.smithi112.stdout:5/67: chown d4/fb 156 1 2024-09-16T00:25:49.964 INFO:tasks.workunit.client.0.smithi112.stdout:5/68: rename d4/dd/d12 to d4/dd/d12/d1d 22 2024-09-16T00:25:49.964 INFO:tasks.workunit.client.0.smithi112.stdout:2/38: mknod d2/dc/ce 0 2024-09-16T00:25:49.964 INFO:tasks.workunit.client.0.smithi112.stdout:0/37: getdents d5/da/dc 0 2024-09-16T00:25:49.965 INFO:tasks.workunit.client.0.smithi112.stdout:2/39: read d2/fa [436817,45610] 0 2024-09-16T00:25:49.966 INFO:tasks.workunit.client.0.smithi112.stdout:2/40: read d2/f8 [1829142,97633] 0 2024-09-16T00:25:49.966 INFO:tasks.workunit.client.0.smithi112.stdout:5/69: creat d4/d7/f1e x:0 0 0 2024-09-16T00:25:49.967 INFO:tasks.workunit.client.0.smithi112.stdout:5/70: readlink d4/dd/d12/l13 0 2024-09-16T00:25:49.968 INFO:tasks.workunit.client.0.smithi112.stdout:5/71: readlink d4/l1b 0 2024-09-16T00:25:49.968 INFO:tasks.workunit.client.0.smithi112.stdout:5/72: rename d4 to d4/d1f 22 2024-09-16T00:25:49.969 INFO:tasks.workunit.client.0.smithi112.stdout:4/63: getdents d6/da 0 2024-09-16T00:25:49.971 INFO:tasks.workunit.client.0.smithi112.stdout:2/41: unlink d2/dc/ce 0 2024-09-16T00:25:49.973 INFO:tasks.workunit.client.0.smithi112.stdout:0/38: truncate f0 131800 0 2024-09-16T00:25:49.974 INFO:tasks.workunit.client.0.smithi112.stdout:5/73: rename d4/dd/d12/l15 to d4/d1c/l20 0 2024-09-16T00:25:49.976 INFO:tasks.workunit.client.0.smithi112.stdout:4/64: creat d6/db/f18 x:0 0 0 2024-09-16T00:25:49.978 INFO:tasks.workunit.client.0.smithi112.stdout:2/42: write d2/fa [640637,31718] 0 2024-09-16T00:25:49.980 INFO:tasks.workunit.client.0.smithi112.stdout:5/74: symlink d4/d7/l21 0 2024-09-16T00:25:49.980 INFO:tasks.workunit.client.0.smithi112.stdout:2/43: mknod d2/dc/cf 0 2024-09-16T00:25:49.983 INFO:tasks.workunit.client.0.smithi112.stdout:5/75: dread d4/f19 [0,4194304] 0 2024-09-16T00:25:49.985 INFO:tasks.workunit.client.0.smithi112.stdout:5/76: rmdir d4/d1c 39 2024-09-16T00:25:49.988 INFO:tasks.workunit.client.0.smithi112.stdout:5/77: rename c1 to d4/dd/c22 0 2024-09-16T00:25:49.989 INFO:tasks.workunit.client.0.smithi112.stdout:5/78: write d4/d7/f1e [1031016,76362] 0 2024-09-16T00:25:49.995 INFO:tasks.workunit.client.0.smithi112.stdout:5/79: dread d4/d7/f9 [0,4194304] 0 2024-09-16T00:25:49.998 INFO:tasks.workunit.client.0.smithi112.stdout:5/80: link d4/d7/f1e d4/dd/f23 0 2024-09-16T00:25:49.999 INFO:tasks.workunit.client.0.smithi112.stdout:5/81: creat d4/dd/f24 x:0 0 0 2024-09-16T00:25:50.000 INFO:tasks.workunit.client.0.smithi112.stdout:5/82: chown d4/dd/d12/l13 13 1 2024-09-16T00:25:50.029 INFO:tasks.workunit.client.0.smithi112.stdout:9/102: dwrite d1/d5/f16 [0,4194304] 0 2024-09-16T00:25:50.032 INFO:tasks.workunit.client.0.smithi112.stdout:3/47: rename d2/f5 to d2/fb 0 2024-09-16T00:25:50.039 INFO:tasks.workunit.client.0.smithi112.stdout:3/48: write d2/f6 [955801,81129] 0 2024-09-16T00:25:50.041 INFO:tasks.workunit.client.0.smithi112.stdout:5/83: rmdir d4/d1c 39 2024-09-16T00:25:50.049 INFO:tasks.workunit.client.0.smithi112.stdout:3/49: dread d2/f6 [0,4194304] 0 2024-09-16T00:25:50.049 INFO:tasks.workunit.client.0.smithi112.stdout:6/84: truncate f8 370484 0 2024-09-16T00:25:50.052 INFO:tasks.workunit.client.0.smithi112.stdout:6/85: mknod dd/df/d13/d18/c20 0 2024-09-16T00:25:50.056 INFO:tasks.workunit.client.0.smithi112.stdout:6/86: dread fa [0,4194304] 0 2024-09-16T00:25:50.058 INFO:tasks.workunit.client.0.smithi112.stdout:2/44: rmdir d2/dc 39 2024-09-16T00:25:50.060 INFO:tasks.workunit.client.0.smithi112.stdout:2/45: write d2/fa [149658,116333] 0 2024-09-16T00:25:50.062 INFO:tasks.workunit.client.0.smithi112.stdout:5/84: rmdir d4/dd 39 2024-09-16T00:25:50.063 INFO:tasks.workunit.client.0.smithi112.stdout:6/87: dwrite fa [0,4194304] 0 2024-09-16T00:25:50.063 INFO:tasks.workunit.client.0.smithi112.stdout:5/85: chown d4/l1b 29636 1 2024-09-16T00:25:50.065 INFO:tasks.workunit.client.0.smithi112.stdout:2/46: creat d2/dd/f10 x:0 0 0 2024-09-16T00:25:50.065 INFO:tasks.workunit.client.0.smithi112.stdout:2/47: dread - d2/dd/f10 zero size 2024-09-16T00:25:50.068 INFO:tasks.workunit.client.0.smithi112.stdout:9/103: fsync d1/d5/d6/dd/f1c 0 2024-09-16T00:25:50.069 INFO:tasks.workunit.client.0.smithi112.stdout:1/33: write d0/d3/f8 [1123195,32420] 0 2024-09-16T00:25:50.070 INFO:tasks.workunit.client.0.smithi112.stdout:6/88: fdatasync f8 0 2024-09-16T00:25:50.074 INFO:tasks.workunit.client.0.smithi112.stdout:5/86: fsync d4/d7/f8 0 2024-09-16T00:25:50.074 INFO:tasks.workunit.client.0.smithi112.stdout:2/48: dwrite d2/fa [0,4194304] 0 2024-09-16T00:25:50.080 INFO:tasks.workunit.client.0.smithi112.stdout:6/89: readlink dd/df/d10/d1d/l1f 0 2024-09-16T00:25:50.082 INFO:tasks.workunit.client.0.smithi112.stdout:1/34: dread d0/f2 [0,4194304] 0 2024-09-16T00:25:50.082 INFO:tasks.workunit.client.0.smithi112.stdout:1/35: readlink - no filename 2024-09-16T00:25:50.083 INFO:tasks.workunit.client.0.smithi112.stdout:5/87: symlink d4/d7/l25 0 2024-09-16T00:25:50.083 INFO:tasks.workunit.client.0.smithi112.stdout:6/90: read f8 [63066,62034] 0 2024-09-16T00:25:50.085 INFO:tasks.workunit.client.0.smithi112.stdout:3/50: dwrite d2/f6 [0,4194304] 0 2024-09-16T00:25:50.086 INFO:tasks.workunit.client.0.smithi112.stdout:6/91: chown c4 0 1 2024-09-16T00:25:50.089 INFO:tasks.workunit.client.0.smithi112.stdout:2/49: creat d2/dc/f11 x:0 0 0 2024-09-16T00:25:50.090 INFO:tasks.workunit.client.0.smithi112.stdout:5/88: mkdir d4/d7/d26 0 2024-09-16T00:25:50.090 INFO:tasks.workunit.client.0.smithi112.stdout:3/51: write d2/f6 [3577979,30799] 0 2024-09-16T00:25:50.092 INFO:tasks.workunit.client.0.smithi112.stdout:0/39: dwrite f0 [0,4194304] 0 2024-09-16T00:25:50.094 INFO:tasks.workunit.client.0.smithi112.stdout:9/104: rename d1/d5/d6/dd to d1/d5/d20/d25 0 2024-09-16T00:25:50.097 INFO:tasks.workunit.client.0.smithi112.stdout:6/92: rmdir dd/df/d11 39 2024-09-16T00:25:50.100 INFO:tasks.workunit.client.0.smithi112.stdout:2/50: creat d2/dd/f12 x:0 0 0 2024-09-16T00:25:50.105 INFO:tasks.workunit.client.0.smithi112.stdout:2/51: dread d2/fa [0,4194304] 0 2024-09-16T00:25:50.107 INFO:tasks.workunit.client.0.smithi112.stdout:0/40: creat d5/da/dc/fd x:0 0 0 2024-09-16T00:25:50.108 INFO:tasks.workunit.client.0.smithi112.stdout:5/89: dread d4/dd/f23 [0,4194304] 0 2024-09-16T00:25:50.109 INFO:tasks.workunit.client.0.smithi112.stdout:9/105: dread d1/d5/f16 [0,4194304] 0 2024-09-16T00:25:50.112 INFO:tasks.workunit.client.0.smithi112.stdout:6/93: creat dd/df/f21 x:0 0 0 2024-09-16T00:25:50.112 INFO:tasks.workunit.client.0.smithi112.stdout:5/90: rename d4/d7/f8 to d4/dd/f27 0 2024-09-16T00:25:50.113 INFO:tasks.workunit.client.0.smithi112.stdout:6/94: write dd/df/d10/f1b [34229,24191] 0 2024-09-16T00:25:50.115 INFO:tasks.workunit.client.0.smithi112.stdout:1/36: truncate d0/f2 207849 0 2024-09-16T00:25:50.116 INFO:tasks.workunit.client.0.smithi112.stdout:1/37: truncate d0/d3/f8 1473464 0 2024-09-16T00:25:50.118 INFO:tasks.workunit.client.0.smithi112.stdout:1/38: read d0/d3/f8 [324154,54579] 0 2024-09-16T00:25:50.118 INFO:tasks.workunit.client.0.smithi112.stdout:5/91: unlink d4/d7/f9 0 2024-09-16T00:25:50.118 INFO:tasks.workunit.client.0.smithi112.stdout:2/52: rmdir d2/dd 39 2024-09-16T00:25:50.118 INFO:tasks.workunit.client.0.smithi112.stdout:5/92: dread - d4/dd/d12/f11 zero size 2024-09-16T00:25:50.119 INFO:tasks.workunit.client.0.smithi112.stdout:3/52: write d2/fb [1920040,16391] 0 2024-09-16T00:25:50.123 INFO:tasks.workunit.client.0.smithi112.stdout:2/53: mkdir d2/dc/d13 0 2024-09-16T00:25:50.126 INFO:tasks.workunit.client.0.smithi112.stdout:0/41: getdents d5/da 0 2024-09-16T00:25:50.129 INFO:tasks.workunit.client.0.smithi112.stdout:9/106: write d1/d5/d6/d18/f19 [3144842,124010] 0 2024-09-16T00:25:50.131 INFO:tasks.workunit.client.0.smithi112.stdout:2/54: dread - d2/dd/f10 zero size 2024-09-16T00:25:50.133 INFO:tasks.workunit.client.0.smithi112.stdout:2/55: stat d2/dd/f12 0 2024-09-16T00:25:50.134 INFO:tasks.workunit.client.0.smithi112.stdout:0/42: dread f4 [0,4194304] 0 2024-09-16T00:25:50.136 INFO:tasks.workunit.client.0.smithi112.stdout:1/39: dread d0/f2 [0,4194304] 0 2024-09-16T00:25:50.136 INFO:tasks.workunit.client.0.smithi112.stdout:6/95: dwrite f8 [0,4194304] 0 2024-09-16T00:25:50.137 INFO:tasks.workunit.client.0.smithi112.stdout:5/93: getdents d4/dd 0 2024-09-16T00:25:50.137 INFO:tasks.workunit.client.0.smithi112.stdout:9/107: mknod d1/d5/d6/d18/c26 0 2024-09-16T00:25:50.138 INFO:tasks.workunit.client.0.smithi112.stdout:5/94: dread - f2 zero size 2024-09-16T00:25:50.140 INFO:tasks.workunit.client.0.smithi112.stdout:0/43: rename d5/da/dc/fd to d5/da/dc/fe 0 2024-09-16T00:25:50.142 INFO:tasks.workunit.client.0.smithi112.stdout:2/56: rmdir d2/dc/d13 0 2024-09-16T00:25:50.143 INFO:tasks.workunit.client.0.smithi112.stdout:2/57: read - d2/dd/f12 zero size 2024-09-16T00:25:50.144 INFO:tasks.workunit.client.0.smithi112.stdout:6/96: rename dd/df/d13/d18/c20 to dd/df/d13/d18/c22 0 2024-09-16T00:25:50.145 INFO:tasks.workunit.client.0.smithi112.stdout:1/40: creat d0/f9 x:0 0 0 2024-09-16T00:25:50.145 INFO:tasks.workunit.client.0.smithi112.stdout:9/108: symlink d1/d5/d6/d11/d1a/l27 0 2024-09-16T00:25:50.147 INFO:tasks.workunit.client.0.smithi112.stdout:5/95: truncate d4/f16 1323121 0 2024-09-16T00:25:50.148 INFO:tasks.workunit.client.0.smithi112.stdout:1/41: creat d0/d3/fa x:0 0 0 2024-09-16T00:25:50.151 INFO:tasks.workunit.client.0.smithi112.stdout:9/109: creat d1/d5/d6/d18/f28 x:0 0 0 2024-09-16T00:25:50.152 INFO:tasks.workunit.client.0.smithi112.stdout:2/58: dwrite f1 [0,4194304] 0 2024-09-16T00:25:50.152 INFO:tasks.workunit.client.0.smithi112.stdout:5/96: rmdir d4/dd/d12 39 2024-09-16T00:25:50.154 INFO:tasks.workunit.client.0.smithi112.stdout:1/42: symlink d0/d3/lb 0 2024-09-16T00:25:50.156 INFO:tasks.workunit.client.0.smithi112.stdout:1/43: dread d0/f2 [0,4194304] 0 2024-09-16T00:25:50.156 INFO:tasks.workunit.client.0.smithi112.stdout:9/110: mkdir d1/d5/d6/d11/d29 0 2024-09-16T00:25:50.157 INFO:tasks.workunit.client.0.smithi112.stdout:2/59: dread f1 [0,4194304] 0 2024-09-16T00:25:50.158 INFO:tasks.workunit.client.0.smithi112.stdout:6/97: getdents dd/df/d13/d18 0 2024-09-16T00:25:50.158 INFO:tasks.workunit.client.0.smithi112.stdout:2/60: fdatasync d2/dc/f11 0 2024-09-16T00:25:50.158 INFO:tasks.workunit.client.0.smithi112.stdout:9/111: chown d1/d5/d20/d25/f1c 612 1 2024-09-16T00:25:50.158 INFO:tasks.workunit.client.0.smithi112.stdout:5/97: mknod d4/d7/d14/c28 0 2024-09-16T00:25:50.160 INFO:tasks.workunit.client.0.smithi112.stdout:2/61: truncate d2/dd/f10 337363 0 2024-09-16T00:25:50.160 INFO:tasks.workunit.client.0.smithi112.stdout:5/98: chown d4/d7/l21 64388 1 2024-09-16T00:25:50.160 INFO:tasks.workunit.client.0.smithi112.stdout:6/98: mkdir dd/d23 0 2024-09-16T00:25:50.161 INFO:tasks.workunit.client.0.smithi112.stdout:1/44: symlink d0/lc 0 2024-09-16T00:25:50.161 INFO:tasks.workunit.client.0.smithi112.stdout:6/99: chown dd/df/d17 20 1 2024-09-16T00:25:50.162 INFO:tasks.workunit.client.0.smithi112.stdout:2/62: write d2/dd/f10 [1299309,115200] 0 2024-09-16T00:25:50.162 INFO:tasks.workunit.client.0.smithi112.stdout:9/112: truncate d1/d5/d6/d11/f1d 4566892 0 2024-09-16T00:25:50.164 INFO:tasks.workunit.client.0.smithi112.stdout:5/99: symlink d4/d1c/l29 0 2024-09-16T00:25:50.164 INFO:tasks.workunit.client.0.smithi112.stdout:1/45: truncate d0/d3/fa 986119 0 2024-09-16T00:25:50.165 INFO:tasks.workunit.client.0.smithi112.stdout:9/113: read d1/d5/d6/d11/f1d [3478986,122026] 0 2024-09-16T00:25:50.166 INFO:tasks.workunit.client.0.smithi112.stdout:1/46: truncate d0/f9 695957 0 2024-09-16T00:25:50.166 INFO:tasks.workunit.client.0.smithi112.stdout:6/100: getdents dd/df/d17 0 2024-09-16T00:25:50.169 INFO:tasks.workunit.client.0.smithi112.stdout:1/47: creat d0/d3/fd x:0 0 0 2024-09-16T00:25:50.169 INFO:tasks.workunit.client.0.smithi112.stdout:9/114: mknod d1/d5/d6/d18/d23/c2a 0 2024-09-16T00:25:50.170 INFO:tasks.workunit.client.0.smithi112.stdout:6/101: mknod dd/df/d17/c24 0 2024-09-16T00:25:50.171 INFO:tasks.workunit.client.0.smithi112.stdout:2/63: dwrite d2/dd/f10 [0,4194304] 0 2024-09-16T00:25:50.173 INFO:tasks.workunit.client.0.smithi112.stdout:1/48: mknod d0/ce 0 2024-09-16T00:25:50.173 INFO:tasks.workunit.client.0.smithi112.stdout:9/115: dread - d1/d5/d6/d18/f28 zero size 2024-09-16T00:25:50.175 INFO:tasks.workunit.client.0.smithi112.stdout:6/102: creat dd/df/d13/d18/f25 x:0 0 0 2024-09-16T00:25:50.178 INFO:tasks.workunit.client.0.smithi112.stdout:1/49: mkdir d0/d3/df 0 2024-09-16T00:25:50.178 INFO:tasks.workunit.client.0.smithi112.stdout:5/100: rmdir d4/d7 39 2024-09-16T00:25:50.180 INFO:tasks.workunit.client.0.smithi112.stdout:9/116: creat d1/d5/d6/d18/d23/f2b x:0 0 0 2024-09-16T00:25:50.182 INFO:tasks.workunit.client.0.smithi112.stdout:2/64: creat d2/f14 x:0 0 0 2024-09-16T00:25:50.183 INFO:tasks.workunit.client.0.smithi112.stdout:6/103: dwrite dd/df/d10/f1b [0,4194304] 0 2024-09-16T00:25:50.183 INFO:tasks.workunit.client.0.smithi112.stdout:2/65: chown c0 2236 1 2024-09-16T00:25:50.184 INFO:tasks.workunit.client.0.smithi112.stdout:6/104: readlink dd/l12 0 2024-09-16T00:25:50.185 INFO:tasks.workunit.client.0.smithi112.stdout:5/101: dread d4/f16 [0,4194304] 0 2024-09-16T00:25:50.189 INFO:tasks.workunit.client.0.smithi112.stdout:2/66: creat d2/dd/f15 x:0 0 0 2024-09-16T00:25:50.191 INFO:tasks.workunit.client.0.smithi112.stdout:1/50: link d0/lc d0/d3/df/l10 0 2024-09-16T00:25:50.192 INFO:tasks.workunit.client.0.smithi112.stdout:9/117: rmdir d1/d5/d6/d11/d29 0 2024-09-16T00:25:50.193 INFO:tasks.workunit.client.0.smithi112.stdout:6/105: creat dd/df/d10/d1d/f26 x:0 0 0 2024-09-16T00:25:50.194 INFO:tasks.workunit.client.0.smithi112.stdout:2/67: read d2/f8 [1120636,52131] 0 2024-09-16T00:25:50.195 INFO:tasks.workunit.client.0.smithi112.stdout:6/106: stat dd/df/d13/d18/f25 0 2024-09-16T00:25:50.195 INFO:tasks.workunit.client.0.smithi112.stdout:2/68: readlink d2/lb 0 2024-09-16T00:25:50.196 INFO:tasks.workunit.client.0.smithi112.stdout:9/118: mkdir d1/d5/d2c 0 2024-09-16T00:25:50.197 INFO:tasks.workunit.client.0.smithi112.stdout:5/102: rename d4/d7/f1e to d4/dd/f2a 0 2024-09-16T00:25:50.198 INFO:tasks.workunit.client.0.smithi112.stdout:6/107: mkdir dd/df/d17/d27 0 2024-09-16T00:25:50.198 INFO:tasks.workunit.client.0.smithi112.stdout:5/103: read - d4/dd/f24 zero size 2024-09-16T00:25:50.198 INFO:tasks.workunit.client.0.smithi112.stdout:9/119: mkdir d1/d5/d6/d2d 0 2024-09-16T00:25:50.200 INFO:tasks.workunit.client.0.smithi112.stdout:2/69: dread d2/fa [0,4194304] 0 2024-09-16T00:25:50.200 INFO:tasks.workunit.client.0.smithi112.stdout:6/108: write dd/df/d13/f1c [767993,69477] 0 2024-09-16T00:25:50.201 INFO:tasks.workunit.client.0.smithi112.stdout:5/104: creat d4/dd/f2b x:0 0 0 2024-09-16T00:25:50.202 INFO:tasks.workunit.client.0.smithi112.stdout:9/120: creat d1/d5/d20/d25/f2e x:0 0 0 2024-09-16T00:25:50.203 INFO:tasks.workunit.client.0.smithi112.stdout:6/109: creat dd/df/d13/d18/f28 x:0 0 0 2024-09-16T00:25:50.204 INFO:tasks.workunit.client.0.smithi112.stdout:2/70: dread d2/f8 [0,4194304] 0 2024-09-16T00:25:50.206 INFO:tasks.workunit.client.0.smithi112.stdout:2/71: write d2/fa [3012400,124186] 0 2024-09-16T00:25:50.207 INFO:tasks.workunit.client.0.smithi112.stdout:9/121: creat d1/d5/f2f x:0 0 0 2024-09-16T00:25:50.208 INFO:tasks.workunit.client.0.smithi112.stdout:5/105: rename d4/dd/d12/l13 to d4/l2c 0 2024-09-16T00:25:50.211 INFO:tasks.workunit.client.0.smithi112.stdout:9/122: truncate d1/d5/d6/d11/f1d 5093133 0 2024-09-16T00:25:50.215 INFO:tasks.workunit.client.0.smithi112.stdout:5/106: dwrite d4/dd/f24 [0,4194304] 0 2024-09-16T00:25:50.216 INFO:tasks.workunit.client.0.smithi112.stdout:9/123: creat d1/d5/f30 x:0 0 0 2024-09-16T00:25:50.217 INFO:tasks.workunit.client.0.smithi112.stdout:9/124: stat d1/d5/d6/l17 0 2024-09-16T00:25:50.218 INFO:tasks.workunit.client.0.smithi112.stdout:5/107: creat d4/d1c/f2d x:0 0 0 2024-09-16T00:25:50.219 INFO:tasks.workunit.client.0.smithi112.stdout:9/125: creat d1/d5/d20/f31 x:0 0 0 2024-09-16T00:25:50.221 INFO:tasks.workunit.client.0.smithi112.stdout:2/72: getdents d2/dd 0 2024-09-16T00:25:50.221 INFO:tasks.workunit.client.0.smithi112.stdout:2/73: dread - d2/f14 zero size 2024-09-16T00:25:50.223 INFO:tasks.workunit.client.0.smithi112.stdout:2/74: dread - d2/dd/f12 zero size 2024-09-16T00:25:50.223 INFO:tasks.workunit.client.0.smithi112.stdout:9/126: creat d1/d5/d20/f32 x:0 0 0 2024-09-16T00:25:50.225 INFO:tasks.workunit.client.0.smithi112.stdout:2/75: unlink c0 0 2024-09-16T00:25:50.227 INFO:tasks.workunit.client.0.smithi112.stdout:5/108: fsync d4/dd/f2b 0 2024-09-16T00:25:50.227 INFO:tasks.workunit.client.0.smithi112.stdout:9/127: rmdir d1/d5/d20/d25 39 2024-09-16T00:25:50.228 INFO:tasks.workunit.client.0.smithi112.stdout:2/76: dread d2/f8 [0,4194304] 0 2024-09-16T00:25:50.229 INFO:tasks.workunit.client.0.smithi112.stdout:5/109: truncate d4/fb 798506 0 2024-09-16T00:25:50.230 INFO:tasks.workunit.client.0.smithi112.stdout:5/110: dread - d4/dd/f2b zero size 2024-09-16T00:25:50.231 INFO:tasks.workunit.client.0.smithi112.stdout:2/77: rmdir d2/dd 39 2024-09-16T00:25:50.232 INFO:tasks.workunit.client.0.smithi112.stdout:5/111: write d4/dd/f24 [4570218,24090] 0 2024-09-16T00:25:50.233 INFO:tasks.workunit.client.0.smithi112.stdout:5/112: fsync d4/dd/d12/f11 0 2024-09-16T00:25:50.237 INFO:tasks.workunit.client.0.smithi112.stdout:5/113: mkdir d4/dd/d12/d2e 0 2024-09-16T00:25:50.238 INFO:tasks.workunit.client.0.smithi112.stdout:2/78: mknod d2/c16 0 2024-09-16T00:25:50.238 INFO:tasks.workunit.client.0.smithi112.stdout:9/128: link d1/d5/d6/d18/c26 d1/c33 0 2024-09-16T00:25:50.239 INFO:tasks.workunit.client.0.smithi112.stdout:5/114: truncate d4/d7/d14/f1a 4824634 0 2024-09-16T00:25:50.240 INFO:tasks.workunit.client.0.smithi112.stdout:9/129: fdatasync d1/d5/f16 0 2024-09-16T00:25:50.242 INFO:tasks.workunit.client.0.smithi112.stdout:9/130: write d1/d5/d6/d18/f19 [762578,54164] 0 2024-09-16T00:25:50.242 INFO:tasks.workunit.client.0.smithi112.stdout:2/79: rename d2/l7 to d2/l17 0 2024-09-16T00:25:50.243 INFO:tasks.workunit.client.0.smithi112.stdout:9/131: readlink d1/d5/d6/l17 0 2024-09-16T00:25:50.246 INFO:tasks.workunit.client.0.smithi112.stdout:9/132: rmdir d1/d5/d6/d18 39 2024-09-16T00:25:50.251 INFO:tasks.workunit.client.0.smithi112.stdout:5/115: dwrite d4/f16 [0,4194304] 0 2024-09-16T00:25:50.251 INFO:tasks.workunit.client.0.smithi112.stdout:2/80: dread d2/dd/f10 [0,4194304] 0 2024-09-16T00:25:50.252 INFO:tasks.workunit.client.0.smithi112.stdout:5/116: fdatasync d4/dd/f2b 0 2024-09-16T00:25:50.252 INFO:tasks.workunit.client.0.smithi112.stdout:2/81: chown d2/dc/cf 122076538 1 2024-09-16T00:25:50.252 INFO:tasks.workunit.client.0.smithi112.stdout:5/117: stat d4/dd/d12/d2e 0 2024-09-16T00:25:50.253 INFO:tasks.workunit.client.0.smithi112.stdout:9/133: creat d1/d5/d6/f34 x:0 0 0 2024-09-16T00:25:50.254 INFO:tasks.workunit.client.0.smithi112.stdout:5/118: chown d4/d7/l25 1504762 1 2024-09-16T00:25:50.255 INFO:tasks.workunit.client.0.smithi112.stdout:9/134: chown d1/d5/d6/f34 730477 1 2024-09-16T00:25:50.256 INFO:tasks.workunit.client.0.smithi112.stdout:2/82: getdents d2 0 2024-09-16T00:25:50.259 INFO:tasks.workunit.client.0.smithi112.stdout:5/119: dread d4/f16 [0,4194304] 0 2024-09-16T00:25:50.259 INFO:tasks.workunit.client.0.smithi112.stdout:9/135: mknod d1/d5/d6/d18/d23/c35 0 2024-09-16T00:25:50.260 INFO:tasks.workunit.client.0.smithi112.stdout:5/120: dread - d4/dd/d12/f11 zero size 2024-09-16T00:25:50.262 INFO:tasks.workunit.client.0.smithi112.stdout:5/121: stat d4/l2c 0 2024-09-16T00:25:50.263 INFO:tasks.workunit.client.0.smithi112.stdout:9/136: creat d1/d5/d20/f36 x:0 0 0 2024-09-16T00:25:50.265 INFO:tasks.workunit.client.0.smithi112.stdout:9/137: symlink d1/d5/d6/d18/l37 0 2024-09-16T00:25:50.265 INFO:tasks.workunit.client.0.smithi112.stdout:5/122: getdents d4/dd/d12 0 2024-09-16T00:25:50.267 INFO:tasks.workunit.client.0.smithi112.stdout:9/138: truncate d1/d5/d20/d25/f1c 393168 0 2024-09-16T00:25:50.272 INFO:tasks.workunit.client.0.smithi112.stdout:5/123: rmdir d4/dd/d12/d2e 0 2024-09-16T00:25:50.274 INFO:tasks.workunit.client.0.smithi112.stdout:9/139: mkdir d1/d5/d6/d38 0 2024-09-16T00:25:50.274 INFO:tasks.workunit.client.0.smithi112.stdout:9/140: write d1/d5/d6/d18/f28 [89255,49254] 0 2024-09-16T00:25:50.275 INFO:tasks.workunit.client.0.smithi112.stdout:9/141: stat d1/d5/d6/d11 0 2024-09-16T00:25:50.278 INFO:tasks.workunit.client.0.smithi112.stdout:9/142: mkdir d1/d5/d6/d2d/d39 0 2024-09-16T00:25:50.278 INFO:tasks.workunit.client.0.smithi112.stdout:5/124: write d4/dd/f27 [1979971,110386] 0 2024-09-16T00:25:50.279 INFO:tasks.workunit.client.0.smithi112.stdout:9/143: readlink d1/d5/d6/l17 0 2024-09-16T00:25:50.281 INFO:tasks.workunit.client.0.smithi112.stdout:5/125: rename d4/dd/f2b to d4/dd/d12/f2f 0 2024-09-16T00:25:50.282 INFO:tasks.workunit.client.0.smithi112.stdout:5/126: chown d4/d1c/f2d 1 1 2024-09-16T00:25:50.284 INFO:tasks.workunit.client.0.smithi112.stdout:9/144: creat d1/d5/d6/f3a x:0 0 0 2024-09-16T00:25:50.285 INFO:tasks.workunit.client.0.smithi112.stdout:5/127: truncate d4/dd/f2a 1111528 0 2024-09-16T00:25:50.287 INFO:tasks.workunit.client.0.smithi112.stdout:9/145: getdents d1/d5/d20/d25 0 2024-09-16T00:25:50.293 INFO:tasks.workunit.client.0.smithi112.stdout:9/146: dwrite d1/d5/d20/f31 [0,4194304] 0 2024-09-16T00:25:50.295 INFO:tasks.workunit.client.0.smithi112.stdout:9/147: creat d1/d5/d6/d2d/f3b x:0 0 0 2024-09-16T00:25:50.296 INFO:tasks.workunit.client.0.smithi112.stdout:9/148: stat d1/c14 0 2024-09-16T00:25:50.296 INFO:tasks.workunit.client.0.smithi112.stdout:9/149: write d1/d5/d6/f3a [637571,61656] 0 2024-09-16T00:25:50.297 INFO:tasks.workunit.client.0.smithi112.stdout:9/150: read d1/d5/d20/f31 [362092,9429] 0 2024-09-16T00:25:50.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:25:50 smithi112 ceph-mon[27869]: pgmap v248: 65 pgs: 65 active+clean; 283 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 4.0 MiB/s rd, 13 MiB/s wr, 532 op/s 2024-09-16T00:25:50.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:25:50 smithi112 ceph-mon[27869]: mgrmap e20: smithi112.xrkkbc(active, since 7m), standbys: smithi169.mtrobk 2024-09-16T00:25:50.436 INFO:tasks.workunit.client.0.smithi112.stdout:9/151: write d1/d5/d20/d25/f1c [232347,121548] 0 2024-09-16T00:25:50.436 INFO:tasks.workunit.client.0.smithi112.stdout:5/128: write d4/dd/f2a [2019588,33840] 0 2024-09-16T00:25:50.437 INFO:tasks.workunit.client.0.smithi112.stdout:5/129: dread - d4/dd/d12/f2f zero size 2024-09-16T00:25:50.439 INFO:tasks.workunit.client.0.smithi112.stdout:5/130: write d4/dd/f23 [2929284,83116] 0 2024-09-16T00:25:50.440 INFO:tasks.workunit.client.0.smithi112.stdout:5/131: write d4/dd/f2a [1375862,32301] 0 2024-09-16T00:25:50.443 INFO:tasks.workunit.client.0.smithi112.stdout:5/132: fdatasync d4/f19 0 2024-09-16T00:25:50.445 INFO:tasks.workunit.client.0.smithi112.stdout:9/152: dwrite d1/d5/f30 [0,4194304] 0 2024-09-16T00:25:50.448 INFO:tasks.workunit.client.0.smithi112.stdout:9/153: creat d1/d5/d6/d2d/d39/f3c x:0 0 0 2024-09-16T00:25:50.452 INFO:tasks.workunit.client.0.smithi112.stdout:5/133: dwrite d4/dd/f27 [0,4194304] 0 2024-09-16T00:25:50.453 INFO:tasks.workunit.client.0.smithi112.stdout:9/154: unlink d1/d5/d6/d11/d1a/l27 0 2024-09-16T00:25:50.455 INFO:tasks.workunit.client.0.smithi112.stdout:9/155: write d1/d5/d20/d25/f2e [384355,109287] 0 2024-09-16T00:25:50.457 INFO:tasks.workunit.client.0.smithi112.stdout:9/156: dread - d1/d5/d6/d18/d23/f2b zero size 2024-09-16T00:25:50.457 INFO:tasks.workunit.client.0.smithi112.stdout:5/134: dread d4/dd/f24 [0,4194304] 0 2024-09-16T00:25:50.459 INFO:tasks.workunit.client.0.smithi112.stdout:5/135: dread d4/f19 [0,4194304] 0 2024-09-16T00:25:50.459 INFO:tasks.workunit.client.0.smithi112.stdout:9/157: symlink d1/d5/d6/d18/d23/l3d 0 2024-09-16T00:25:50.497 INFO:tasks.workunit.client.0.smithi112.stdout:8/46: sync 2024-09-16T00:25:50.498 INFO:tasks.workunit.client.0.smithi112.stdout:9/158: dread d1/d5/d6/f3a [0,4194304] 0 2024-09-16T00:25:50.500 INFO:tasks.workunit.client.0.smithi112.stdout:9/159: unlink d1/d5/d6/d11/c1e 0 2024-09-16T00:25:50.501 INFO:tasks.workunit.client.0.smithi112.stdout:8/47: link l7 d10/l11 0 2024-09-16T00:25:50.501 INFO:tasks.workunit.client.0.smithi112.stdout:8/48: truncate fe 1026831 0 2024-09-16T00:25:50.502 INFO:tasks.workunit.client.0.smithi112.stdout:9/160: creat d1/d5/d6/d2d/f3e x:0 0 0 2024-09-16T00:25:50.504 INFO:tasks.workunit.client.0.smithi112.stdout:9/161: dread d1/d5/d6/d18/f1f [0,4194304] 0 2024-09-16T00:25:50.505 INFO:tasks.workunit.client.0.smithi112.stdout:9/162: write d1/d5/d6/d18/f28 [220392,106025] 0 2024-09-16T00:25:50.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:25:50 smithi169 ceph-mon[34653]: pgmap v248: 65 pgs: 65 active+clean; 283 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 4.0 MiB/s rd, 13 MiB/s wr, 532 op/s 2024-09-16T00:25:50.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:25:50 smithi169 ceph-mon[34653]: mgrmap e20: smithi112.xrkkbc(active, since 7m), standbys: smithi169.mtrobk 2024-09-16T00:25:50.569 INFO:tasks.workunit.client.0.smithi112.stdout:8/49: getdents d10 0 2024-09-16T00:25:50.569 INFO:tasks.workunit.client.0.smithi112.stdout:9/163: fdatasync d1/d5/d6/d18/f28 0 2024-09-16T00:25:50.569 INFO:tasks.workunit.client.0.smithi112.stdout:8/50: write fe [1985153,110572] 0 2024-09-16T00:25:50.576 INFO:tasks.workunit.client.0.smithi112.stdout:9/164: dwrite d1/d5/d6/d2d/f3b [0,4194304] 0 2024-09-16T00:25:50.584 INFO:tasks.workunit.client.0.smithi112.stdout:9/165: dwrite d1/d5/d6/d18/f28 [0,4194304] 0 2024-09-16T00:25:50.587 INFO:tasks.workunit.client.0.smithi112.stdout:9/166: creat d1/d5/f3f x:0 0 0 2024-09-16T00:25:50.588 INFO:tasks.workunit.client.0.smithi112.stdout:9/167: read d1/d5/d20/f31 [2290651,93380] 0 2024-09-16T00:25:50.590 INFO:tasks.workunit.client.0.smithi112.stdout:9/168: creat d1/d5/d6/d18/f40 x:0 0 0 2024-09-16T00:25:50.593 INFO:tasks.workunit.client.0.smithi112.stdout:9/169: getdents d1/d5/d6 0 2024-09-16T00:25:50.594 INFO:tasks.workunit.client.0.smithi112.stdout:9/170: readlink d1/d5/l12 0 2024-09-16T00:25:50.596 INFO:tasks.workunit.client.0.smithi112.stdout:9/171: getdents d1/d5/d20 0 2024-09-16T00:25:50.597 INFO:tasks.workunit.client.0.smithi112.stdout:4/65: sync 2024-09-16T00:25:50.597 INFO:tasks.workunit.client.0.smithi112.stdout:7/35: sync 2024-09-16T00:25:50.597 INFO:tasks.workunit.client.0.smithi112.stdout:9/172: fdatasync d1/d5/d6/d11/f21 0 2024-09-16T00:25:50.600 INFO:tasks.workunit.client.0.smithi112.stdout:4/66: mkdir d6/da/dc/d19 0 2024-09-16T00:25:50.601 INFO:tasks.workunit.client.0.smithi112.stdout:7/36: getdents d6 0 2024-09-16T00:25:50.603 INFO:tasks.workunit.client.0.smithi112.stdout:7/37: unlink l0 0 2024-09-16T00:25:50.605 INFO:tasks.workunit.client.0.smithi112.stdout:9/173: dwrite d1/d5/d6/d2d/f3b [4194304,4194304] 0 2024-09-16T00:25:50.606 INFO:tasks.workunit.client.0.smithi112.stdout:4/67: link d6/db/f18 d6/f1a 0 2024-09-16T00:25:50.606 INFO:tasks.workunit.client.0.smithi112.stdout:7/38: symlink d6/le 0 2024-09-16T00:25:50.606 INFO:tasks.workunit.client.0.smithi112.stdout:4/68: dread - d6/db/f18 zero size 2024-09-16T00:25:50.607 INFO:tasks.workunit.client.0.smithi112.stdout:9/174: fdatasync d1/d5/d6/f34 0 2024-09-16T00:25:50.609 INFO:tasks.workunit.client.0.smithi112.stdout:9/175: write d1/d5/d6/f34 [839728,70303] 0 2024-09-16T00:25:50.610 INFO:tasks.workunit.client.0.smithi112.stdout:4/69: rename d6/da/f16 to d6/f1b 0 2024-09-16T00:25:50.613 INFO:tasks.workunit.client.0.smithi112.stdout:4/70: unlink d6/f1a 0 2024-09-16T00:25:50.619 INFO:tasks.workunit.client.0.smithi112.stdout:9/176: dwrite d1/d5/d20/f31 [4194304,4194304] 0 2024-09-16T00:25:50.621 INFO:tasks.workunit.client.0.smithi112.stdout:9/177: write d1/d5/d20/d25/f1c [135343,1076] 0 2024-09-16T00:25:50.623 INFO:tasks.workunit.client.0.smithi112.stdout:4/71: dwrite f4 [0,4194304] 0 2024-09-16T00:25:50.624 INFO:tasks.workunit.client.0.smithi112.stdout:9/178: write d1/d5/d20/f32 [379172,90898] 0 2024-09-16T00:25:50.625 INFO:tasks.workunit.client.0.smithi112.stdout:7/39: getdents d6 0 2024-09-16T00:25:50.628 INFO:tasks.workunit.client.0.smithi112.stdout:9/179: write d1/d5/d6/d18/d23/f2b [609065,59281] 0 2024-09-16T00:25:50.629 INFO:tasks.workunit.client.0.smithi112.stdout:7/40: symlink d6/lf 0 2024-09-16T00:25:50.630 INFO:tasks.workunit.client.0.smithi112.stdout:4/72: mkdir d6/da/dc/d19/d1c 0 2024-09-16T00:25:50.631 INFO:tasks.workunit.client.0.smithi112.stdout:7/41: mkdir d6/d10 0 2024-09-16T00:25:50.631 INFO:tasks.workunit.client.0.smithi112.stdout:9/180: write d1/d5/d20/f36 [394347,43858] 0 2024-09-16T00:25:50.633 INFO:tasks.workunit.client.0.smithi112.stdout:7/42: creat d6/f11 x:0 0 0 2024-09-16T00:25:50.635 INFO:tasks.workunit.client.0.smithi112.stdout:4/73: getdents d6/db/d15 0 2024-09-16T00:25:50.637 INFO:tasks.workunit.client.0.smithi112.stdout:7/43: rename f1 to d6/d10/f12 0 2024-09-16T00:25:50.637 INFO:tasks.workunit.client.0.smithi112.stdout:7/44: chown d6/d10 1056 1 2024-09-16T00:25:50.637 INFO:tasks.workunit.client.0.smithi112.stdout:9/181: rename d1/f1b to d1/f41 0 2024-09-16T00:25:50.638 INFO:tasks.workunit.client.0.smithi112.stdout:4/74: symlink d6/db/l1d 0 2024-09-16T00:25:50.639 INFO:tasks.workunit.client.0.smithi112.stdout:4/75: write f4 [5654157,123055] 0 2024-09-16T00:25:50.640 INFO:tasks.workunit.client.0.smithi112.stdout:7/45: unlink d6/c8 0 2024-09-16T00:25:50.641 INFO:tasks.workunit.client.0.smithi112.stdout:9/182: creat d1/d5/d6/d2d/d39/f42 x:0 0 0 2024-09-16T00:25:50.641 INFO:tasks.workunit.client.0.smithi112.stdout:4/76: truncate d6/f9 2879721 0 2024-09-16T00:25:50.641 INFO:tasks.workunit.client.0.smithi112.stdout:9/183: chown d1/f41 1053696524 1 2024-09-16T00:25:50.642 INFO:tasks.workunit.client.0.smithi112.stdout:7/46: creat d6/d10/f13 x:0 0 0 2024-09-16T00:25:50.642 INFO:tasks.workunit.client.0.smithi112.stdout:7/47: chown l3 252 1 2024-09-16T00:25:50.644 INFO:tasks.workunit.client.0.smithi112.stdout:9/184: creat d1/d5/d20/d25/f43 x:0 0 0 2024-09-16T00:25:50.645 INFO:tasks.workunit.client.0.smithi112.stdout:7/48: mknod d6/c14 0 2024-09-16T00:25:50.646 INFO:tasks.workunit.client.0.smithi112.stdout:9/185: mknod d1/d5/d20/c44 0 2024-09-16T00:25:50.647 INFO:tasks.workunit.client.0.smithi112.stdout:7/49: creat d6/d10/f15 x:0 0 0 2024-09-16T00:25:50.648 INFO:tasks.workunit.client.0.smithi112.stdout:7/50: write d6/f9 [117605,110855] 0 2024-09-16T00:25:50.651 INFO:tasks.workunit.client.0.smithi112.stdout:9/186: rename d1/d5/d6/d11 to d1/d5/d6/d2d/d45 0 2024-09-16T00:25:50.652 INFO:tasks.workunit.client.0.smithi112.stdout:7/51: mkdir d6/d10/d16 0 2024-09-16T00:25:50.655 INFO:tasks.workunit.client.0.smithi112.stdout:7/52: symlink d6/d10/d16/l17 0 2024-09-16T00:25:50.655 INFO:tasks.workunit.client.0.smithi112.stdout:7/53: rename d6 to d6/d10/d18 22 2024-09-16T00:25:50.656 INFO:tasks.workunit.client.0.smithi112.stdout:4/77: truncate f4 3215423 0 2024-09-16T00:25:50.656 INFO:tasks.workunit.client.0.smithi112.stdout:7/54: fdatasync d6/d10/f15 0 2024-09-16T00:25:50.657 INFO:tasks.workunit.client.0.smithi112.stdout:7/55: chown d6/lf 37137273 1 2024-09-16T00:25:50.662 INFO:tasks.workunit.client.0.smithi112.stdout:4/78: dwrite d6/f9 [0,4194304] 0 2024-09-16T00:25:50.663 INFO:tasks.workunit.client.0.smithi112.stdout:7/56: getdents d6/d10/d16 0 2024-09-16T00:25:50.666 INFO:tasks.workunit.client.0.smithi112.stdout:7/57: write f5 [1351402,11345] 0 2024-09-16T00:25:50.666 INFO:tasks.workunit.client.0.smithi112.stdout:4/79: dread - d6/db/f18 zero size 2024-09-16T00:25:50.668 INFO:tasks.workunit.client.0.smithi112.stdout:7/58: write d6/d10/f13 [33680,76346] 0 2024-09-16T00:25:50.671 INFO:tasks.workunit.client.0.smithi112.stdout:7/59: symlink d6/l19 0 2024-09-16T00:25:50.673 INFO:tasks.workunit.client.0.smithi112.stdout:7/60: readlink d6/lc 0 2024-09-16T00:25:50.673 INFO:tasks.workunit.client.0.smithi112.stdout:9/187: dwrite d1/d5/d6/f3a [0,4194304] 0 2024-09-16T00:25:50.673 INFO:tasks.workunit.client.0.smithi112.stdout:7/61: stat d6/f11 0 2024-09-16T00:25:50.675 INFO:tasks.workunit.client.0.smithi112.stdout:7/62: creat d6/f1a x:0 0 0 2024-09-16T00:25:50.676 INFO:tasks.workunit.client.0.smithi112.stdout:3/53: sync 2024-09-16T00:25:50.677 INFO:tasks.workunit.client.0.smithi112.stdout:1/51: sync 2024-09-16T00:25:50.677 INFO:tasks.workunit.client.0.smithi112.stdout:6/110: sync 2024-09-16T00:25:50.677 INFO:tasks.workunit.client.0.smithi112.stdout:0/44: sync 2024-09-16T00:25:50.677 INFO:tasks.workunit.client.0.smithi112.stdout:7/63: fsync d6/f7 0 2024-09-16T00:25:50.678 INFO:tasks.workunit.client.0.smithi112.stdout:4/80: dwrite f4 [0,4194304] 0 2024-09-16T00:25:50.678 INFO:tasks.workunit.client.0.smithi112.stdout:9/188: creat d1/f46 x:0 0 0 2024-09-16T00:25:50.679 INFO:tasks.workunit.client.0.smithi112.stdout:6/111: chown c4 22673690 1 2024-09-16T00:25:50.681 INFO:tasks.workunit.client.0.smithi112.stdout:1/52: write d0/d3/fa [865626,55268] 0 2024-09-16T00:25:50.681 INFO:tasks.workunit.client.0.smithi112.stdout:0/45: creat d5/ff x:0 0 0 2024-09-16T00:25:50.684 INFO:tasks.workunit.client.0.smithi112.stdout:4/81: symlink d6/l1e 0 2024-09-16T00:25:50.684 INFO:tasks.workunit.client.0.smithi112.stdout:7/64: mkdir d6/d10/d1b 0 2024-09-16T00:25:50.685 INFO:tasks.workunit.client.0.smithi112.stdout:9/189: write d1/f41 [382171,107747] 0 2024-09-16T00:25:50.685 INFO:tasks.workunit.client.0.smithi112.stdout:6/112: unlink dd/l12 0 2024-09-16T00:25:50.687 INFO:tasks.workunit.client.0.smithi112.stdout:1/53: creat d0/d3/f11 x:0 0 0 2024-09-16T00:25:50.688 INFO:tasks.workunit.client.0.smithi112.stdout:7/65: write d6/fa [668873,4038] 0 2024-09-16T00:25:50.689 INFO:tasks.workunit.client.0.smithi112.stdout:4/82: mkdir d6/db/d1f 0 2024-09-16T00:25:50.692 INFO:tasks.workunit.client.0.smithi112.stdout:0/46: getdents d5/da/dc 0 2024-09-16T00:25:50.693 INFO:tasks.workunit.client.0.smithi112.stdout:6/113: truncate dd/df/d10/d1d/f26 151061 0 2024-09-16T00:25:50.694 INFO:tasks.workunit.client.0.smithi112.stdout:1/54: creat d0/d3/df/f12 x:0 0 0 2024-09-16T00:25:50.694 INFO:tasks.workunit.client.0.smithi112.stdout:3/54: dwrite d2/f6 [0,4194304] 0 2024-09-16T00:25:50.695 INFO:tasks.workunit.client.0.smithi112.stdout:7/66: dread d6/d10/f12 [0,4194304] 0 2024-09-16T00:25:50.695 INFO:tasks.workunit.client.0.smithi112.stdout:4/83: mknod d6/db/df/c20 0 2024-09-16T00:25:50.697 INFO:tasks.workunit.client.0.smithi112.stdout:4/84: stat f4 0 2024-09-16T00:25:50.697 INFO:tasks.workunit.client.0.smithi112.stdout:7/67: readlink d6/le 0 2024-09-16T00:25:50.698 INFO:tasks.workunit.client.0.smithi112.stdout:0/47: readlink d5/l7 0 2024-09-16T00:25:50.698 INFO:tasks.workunit.client.0.smithi112.stdout:6/114: write dd/df/d10/d1d/f26 [421181,104915] 0 2024-09-16T00:25:50.700 INFO:tasks.workunit.client.0.smithi112.stdout:3/55: rename d2/l4 to d2/lc 0 2024-09-16T00:25:50.702 INFO:tasks.workunit.client.0.smithi112.stdout:4/85: mknod d6/db/d15/c21 0 2024-09-16T00:25:50.703 INFO:tasks.workunit.client.0.smithi112.stdout:7/68: creat d6/d10/f1c x:0 0 0 2024-09-16T00:25:50.704 INFO:tasks.workunit.client.0.smithi112.stdout:1/55: mknod d0/c13 0 2024-09-16T00:25:50.705 INFO:tasks.workunit.client.0.smithi112.stdout:0/48: dread f4 [0,4194304] 0 2024-09-16T00:25:50.705 INFO:tasks.workunit.client.0.smithi112.stdout:9/190: getdents d1/d5/d6/d2d/d45 0 2024-09-16T00:25:50.707 INFO:tasks.workunit.client.0.smithi112.stdout:3/56: rename d2/la to d2/ld 0 2024-09-16T00:25:50.709 INFO:tasks.workunit.client.0.smithi112.stdout:0/49: dread f0 [0,4194304] 0 2024-09-16T00:25:50.710 INFO:tasks.workunit.client.0.smithi112.stdout:7/69: rename d6/f1a to d6/d10/f1d 0 2024-09-16T00:25:50.713 INFO:tasks.workunit.client.0.smithi112.stdout:6/115: creat dd/d15/f29 x:0 0 0 2024-09-16T00:25:50.717 INFO:tasks.workunit.client.0.smithi112.stdout:0/50: dwrite d5/ff [0,4194304] 0 2024-09-16T00:25:50.718 INFO:tasks.workunit.client.0.smithi112.stdout:7/70: creat d6/d10/d16/f1e x:0 0 0 2024-09-16T00:25:50.718 INFO:tasks.workunit.client.0.smithi112.stdout:0/51: chown d5/cb 115848129 1 2024-09-16T00:25:50.718 INFO:tasks.workunit.client.0.smithi112.stdout:7/71: chown d6/c14 11 1 2024-09-16T00:25:50.719 INFO:tasks.workunit.client.0.smithi112.stdout:7/72: chown d6/l19 354541 1 2024-09-16T00:25:50.719 INFO:tasks.workunit.client.0.smithi112.stdout:7/73: chown l3 377146088 1 2024-09-16T00:25:50.721 INFO:tasks.workunit.client.0.smithi112.stdout:1/56: write d0/d3/f8 [1764715,115660] 0 2024-09-16T00:25:50.722 INFO:tasks.workunit.client.0.smithi112.stdout:1/57: dread - d0/d3/fd zero size 2024-09-16T00:25:50.723 INFO:tasks.workunit.client.0.smithi112.stdout:1/58: read - d0/d3/f11 zero size 2024-09-16T00:25:50.723 INFO:tasks.workunit.client.0.smithi112.stdout:6/116: rename f8 to dd/f2a 0 2024-09-16T00:25:50.724 INFO:tasks.workunit.client.0.smithi112.stdout:9/191: truncate d1/d5/d6/d2d/d45/f21 433479 0 2024-09-16T00:25:50.725 INFO:tasks.workunit.client.0.smithi112.stdout:2/83: sync 2024-09-16T00:25:50.726 INFO:tasks.workunit.client.0.smithi112.stdout:8/51: sync 2024-09-16T00:25:50.726 INFO:tasks.workunit.client.0.smithi112.stdout:5/136: sync 2024-09-16T00:25:50.726 INFO:tasks.workunit.client.0.smithi112.stdout:3/57: creat d2/d9/fe x:0 0 0 2024-09-16T00:25:50.731 INFO:tasks.workunit.client.0.smithi112.stdout:6/117: unlink cc 0 2024-09-16T00:25:50.732 INFO:tasks.workunit.client.0.smithi112.stdout:4/86: getdents d6/da/dc/d19 0 2024-09-16T00:25:50.733 INFO:tasks.workunit.client.0.smithi112.stdout:6/118: readlink dd/df/d10/l16 0 2024-09-16T00:25:50.734 INFO:tasks.workunit.client.0.smithi112.stdout:9/192: rename d1/d5/d6/f3a to d1/d5/d6/d2d/d45/f47 0 2024-09-16T00:25:50.734 INFO:tasks.workunit.client.0.smithi112.stdout:2/84: dread f1 [0,4194304] 0 2024-09-16T00:25:50.737 INFO:tasks.workunit.client.0.smithi112.stdout:2/85: write d2/fa [1494790,108882] 0 2024-09-16T00:25:50.737 INFO:tasks.workunit.client.0.smithi112.stdout:5/137: dwrite d4/d1c/f2d [0,4194304] 0 2024-09-16T00:25:50.740 INFO:tasks.workunit.client.0.smithi112.stdout:0/52: write f0 [729820,117242] 0 2024-09-16T00:25:50.741 INFO:tasks.workunit.client.0.smithi112.stdout:6/119: fdatasync dd/f2a 0 2024-09-16T00:25:50.746 INFO:tasks.workunit.client.0.smithi112.stdout:8/52: truncate f4 907494 0 2024-09-16T00:25:50.747 INFO:tasks.workunit.client.0.smithi112.stdout:2/86: mknod d2/dc/c18 0 2024-09-16T00:25:50.748 INFO:tasks.workunit.client.0.smithi112.stdout:2/87: fsync d2/dd/f15 0 2024-09-16T00:25:50.749 INFO:tasks.workunit.client.0.smithi112.stdout:1/59: rename d0/d3 to d0/d14 0 2024-09-16T00:25:50.750 INFO:tasks.workunit.client.0.smithi112.stdout:3/58: rename d2/lc to d2/d9/lf 0 2024-09-16T00:25:50.752 INFO:tasks.workunit.client.0.smithi112.stdout:7/74: getdents d6 0 2024-09-16T00:25:50.752 INFO:tasks.workunit.client.0.smithi112.stdout:9/193: mknod d1/c48 0 2024-09-16T00:25:50.753 INFO:tasks.workunit.client.0.smithi112.stdout:0/53: creat d5/da/dc/f10 x:0 0 0 2024-09-16T00:25:50.755 INFO:tasks.workunit.client.0.smithi112.stdout:5/138: unlink d4/d1c/f2d 0 2024-09-16T00:25:50.756 INFO:tasks.workunit.client.0.smithi112.stdout:2/88: creat d2/f19 x:0 0 0 2024-09-16T00:25:50.758 INFO:tasks.workunit.client.0.smithi112.stdout:5/139: dread d4/dd/f27 [0,4194304] 0 2024-09-16T00:25:50.759 INFO:tasks.workunit.client.0.smithi112.stdout:7/75: mknod d6/d10/c1f 0 2024-09-16T00:25:50.759 INFO:tasks.workunit.client.0.smithi112.stdout:0/54: fsync f4 0 2024-09-16T00:25:50.761 INFO:tasks.workunit.client.0.smithi112.stdout:0/55: dread - d5/da/dc/f10 zero size 2024-09-16T00:25:50.762 INFO:tasks.workunit.client.0.smithi112.stdout:0/56: write f4 [4499625,423] 0 2024-09-16T00:25:50.763 INFO:tasks.workunit.client.0.smithi112.stdout:8/53: dread f4 [0,4194304] 0 2024-09-16T00:25:50.764 INFO:tasks.workunit.client.0.smithi112.stdout:1/60: rmdir d0/d14 39 2024-09-16T00:25:50.764 INFO:tasks.workunit.client.0.smithi112.stdout:0/57: dread - d5/da/dc/fe zero size 2024-09-16T00:25:50.765 INFO:tasks.workunit.client.0.smithi112.stdout:7/76: dread d6/f9 [0,4194304] 0 2024-09-16T00:25:50.766 INFO:tasks.workunit.client.0.smithi112.stdout:0/58: truncate d5/da/dc/f10 963653 0 2024-09-16T00:25:50.769 INFO:tasks.workunit.client.0.smithi112.stdout:7/77: dread d6/f7 [0,4194304] 0 2024-09-16T00:25:50.770 INFO:tasks.workunit.client.0.smithi112.stdout:9/194: mknod d1/d5/d6/c49 0 2024-09-16T00:25:50.772 INFO:tasks.workunit.client.0.smithi112.stdout:8/54: mkdir d10/d12 0 2024-09-16T00:25:50.776 INFO:tasks.workunit.client.0.smithi112.stdout:0/59: dwrite f4 [4194304,4194304] 0 2024-09-16T00:25:50.776 INFO:tasks.workunit.client.0.smithi112.stdout:3/59: getdents d2/d9 0 2024-09-16T00:25:50.778 INFO:tasks.workunit.client.0.smithi112.stdout:2/89: dread - d2/f14 zero size 2024-09-16T00:25:50.780 INFO:tasks.workunit.client.0.smithi112.stdout:7/78: dwrite d6/f11 [0,4194304] 0 2024-09-16T00:25:50.783 INFO:tasks.workunit.client.0.smithi112.stdout:3/60: creat d2/d9/f10 x:0 0 0 2024-09-16T00:25:50.784 INFO:tasks.workunit.client.0.smithi112.stdout:9/195: write d1/d5/d6/f34 [158204,38015] 0 2024-09-16T00:25:50.787 INFO:tasks.workunit.client.0.smithi112.stdout:2/90: chown d2/l6 0 1 2024-09-16T00:25:50.788 INFO:tasks.workunit.client.0.smithi112.stdout:5/140: truncate d4/dd/f2a 1619090 0 2024-09-16T00:25:50.795 INFO:tasks.workunit.client.0.smithi112.stdout:0/60: link c3 d5/da/c11 0 2024-09-16T00:25:50.795 INFO:tasks.workunit.client.0.smithi112.stdout:5/141: mknod d4/dd/d12/c30 0 2024-09-16T00:25:50.796 INFO:tasks.workunit.client.0.smithi112.stdout:7/79: link d6/f9 d6/d10/d16/f20 0 2024-09-16T00:25:50.797 INFO:tasks.workunit.client.0.smithi112.stdout:9/196: write d1/d5/d6/d18/f28 [1008839,24834] 0 2024-09-16T00:25:50.799 INFO:tasks.workunit.client.0.smithi112.stdout:3/61: dwrite d2/f8 [0,4194304] 0 2024-09-16T00:25:50.801 INFO:tasks.workunit.client.0.smithi112.stdout:0/61: read - d5/da/dc/fe zero size 2024-09-16T00:25:50.801 INFO:tasks.workunit.client.0.smithi112.stdout:9/197: stat d1/d5/d6/c49 0 2024-09-16T00:25:50.801 INFO:tasks.workunit.client.0.smithi112.stdout:2/91: link d2/f19 d2/dd/f1a 0 2024-09-16T00:25:50.803 INFO:tasks.workunit.client.0.smithi112.stdout:8/55: dwrite f4 [0,4194304] 0 2024-09-16T00:25:50.803 INFO:tasks.workunit.client.0.smithi112.stdout:7/80: read d6/d10/f12 [153988,30551] 0 2024-09-16T00:25:50.806 INFO:tasks.workunit.client.0.smithi112.stdout:8/56: chown l7 21369 1 2024-09-16T00:25:50.808 INFO:tasks.workunit.client.0.smithi112.stdout:0/62: dread f0 [0,4194304] 0 2024-09-16T00:25:50.808 INFO:tasks.workunit.client.0.smithi112.stdout:8/57: write fe [3134759,91887] 0 2024-09-16T00:25:50.813 INFO:tasks.workunit.client.0.smithi112.stdout:7/81: symlink d6/d10/d16/l21 0 2024-09-16T00:25:50.813 INFO:tasks.workunit.client.0.smithi112.stdout:8/58: dread f4 [0,4194304] 0 2024-09-16T00:25:50.814 INFO:tasks.workunit.client.0.smithi112.stdout:2/92: symlink d2/l1b 0 2024-09-16T00:25:50.816 INFO:tasks.workunit.client.0.smithi112.stdout:7/82: write d6/d10/f13 [1049114,98677] 0 2024-09-16T00:25:50.816 INFO:tasks.workunit.client.0.smithi112.stdout:5/142: link l3 d4/dd/d12/l31 0 2024-09-16T00:25:50.816 INFO:tasks.workunit.client.0.smithi112.stdout:0/63: mkdir d5/da/dc/d12 0 2024-09-16T00:25:50.817 INFO:tasks.workunit.client.0.smithi112.stdout:5/143: dread d4/f19 [0,4194304] 0 2024-09-16T00:25:50.818 INFO:tasks.workunit.client.0.smithi112.stdout:3/62: link d2/d9/f10 d2/d9/f11 0 2024-09-16T00:25:50.823 INFO:tasks.workunit.client.0.smithi112.stdout:7/83: creat d6/f22 x:0 0 0 2024-09-16T00:25:50.824 INFO:tasks.workunit.client.0.smithi112.stdout:5/144: creat d4/d1c/f32 x:0 0 0 2024-09-16T00:25:50.825 INFO:tasks.workunit.client.0.smithi112.stdout:2/93: symlink d2/dc/l1c 0 2024-09-16T00:25:50.826 INFO:tasks.workunit.client.0.smithi112.stdout:0/64: dread d5/ff [0,4194304] 0 2024-09-16T00:25:50.826 INFO:tasks.workunit.client.0.smithi112.stdout:3/63: unlink d2/l7 0 2024-09-16T00:25:50.827 INFO:tasks.workunit.client.0.smithi112.stdout:3/64: write d2/d9/fe [531207,120539] 0 2024-09-16T00:25:50.827 INFO:tasks.workunit.client.0.smithi112.stdout:7/84: creat d6/f23 x:0 0 0 2024-09-16T00:25:50.829 INFO:tasks.workunit.client.0.smithi112.stdout:5/145: creat d4/d7/d14/f33 x:0 0 0 2024-09-16T00:25:50.830 INFO:tasks.workunit.client.0.smithi112.stdout:2/94: stat d2/l5 0 2024-09-16T00:25:50.831 INFO:tasks.workunit.client.0.smithi112.stdout:0/65: creat d5/da/dc/f13 x:0 0 0 2024-09-16T00:25:50.832 INFO:tasks.workunit.client.0.smithi112.stdout:3/65: dread d2/f8 [0,4194304] 0 2024-09-16T00:25:50.833 INFO:tasks.workunit.client.0.smithi112.stdout:2/95: creat d2/dc/f1d x:0 0 0 2024-09-16T00:25:50.833 INFO:tasks.workunit.client.0.smithi112.stdout:5/146: read d4/dd/f23 [1288215,6943] 0 2024-09-16T00:25:50.836 INFO:tasks.workunit.client.0.smithi112.stdout:5/147: fsync d4/d1c/f32 0 2024-09-16T00:25:50.837 INFO:tasks.workunit.client.0.smithi112.stdout:0/66: creat d5/da/dc/d12/f14 x:0 0 0 2024-09-16T00:25:50.838 INFO:tasks.workunit.client.0.smithi112.stdout:5/148: write f2 [1025961,71454] 0 2024-09-16T00:25:50.838 INFO:tasks.workunit.client.0.smithi112.stdout:2/96: unlink d2/l1b 0 2024-09-16T00:25:50.839 INFO:tasks.workunit.client.0.smithi112.stdout:0/67: dread - d5/da/dc/fe zero size 2024-09-16T00:25:50.840 INFO:tasks.workunit.client.0.smithi112.stdout:3/66: rename d2/ld to d2/l12 0 2024-09-16T00:25:50.840 INFO:tasks.workunit.client.0.smithi112.stdout:7/85: dwrite d6/d10/f12 [0,4194304] 0 2024-09-16T00:25:50.844 INFO:tasks.workunit.client.0.smithi112.stdout:0/68: creat d5/da/f15 x:0 0 0 2024-09-16T00:25:50.845 INFO:tasks.workunit.client.0.smithi112.stdout:3/67: symlink d2/d9/l13 0 2024-09-16T00:25:50.847 INFO:tasks.workunit.client.0.smithi112.stdout:2/97: unlink d2/dd/f10 0 2024-09-16T00:25:50.848 INFO:tasks.workunit.client.0.smithi112.stdout:0/69: mkdir d5/da/d16 0 2024-09-16T00:25:50.849 INFO:tasks.workunit.client.0.smithi112.stdout:0/70: truncate d5/ff 5191564 0 2024-09-16T00:25:50.849 INFO:tasks.workunit.client.0.smithi112.stdout:7/86: creat d6/d10/d1b/f24 x:0 0 0 2024-09-16T00:25:50.850 INFO:tasks.workunit.client.0.smithi112.stdout:3/68: creat d2/d9/f14 x:0 0 0 2024-09-16T00:25:50.851 INFO:tasks.workunit.client.0.smithi112.stdout:7/87: write d6/d10/f12 [755197,128078] 0 2024-09-16T00:25:50.852 INFO:tasks.workunit.client.0.smithi112.stdout:0/71: creat d5/da/dc/d12/f17 x:0 0 0 2024-09-16T00:25:50.855 INFO:tasks.workunit.client.0.smithi112.stdout:2/98: dread d2/f8 [0,4194304] 0 2024-09-16T00:25:50.855 INFO:tasks.workunit.client.0.smithi112.stdout:7/88: rename d6/d10/f13 to d6/d10/d16/f25 0 2024-09-16T00:25:50.856 INFO:tasks.workunit.client.0.smithi112.stdout:2/99: chown d2/dd/f1a 24803 1 2024-09-16T00:25:50.857 INFO:tasks.workunit.client.0.smithi112.stdout:3/69: getdents d2 0 2024-09-16T00:25:50.859 INFO:tasks.workunit.client.0.smithi112.stdout:7/89: unlink d6/le 0 2024-09-16T00:25:50.860 INFO:tasks.workunit.client.0.smithi112.stdout:7/90: fdatasync d6/fb 0 2024-09-16T00:25:50.860 INFO:tasks.workunit.client.0.smithi112.stdout:2/100: symlink d2/dc/l1e 0 2024-09-16T00:25:50.861 INFO:tasks.workunit.client.0.smithi112.stdout:2/101: dread - d2/f19 zero size 2024-09-16T00:25:50.864 INFO:tasks.workunit.client.0.smithi112.stdout:3/70: link l1 d2/d9/l15 0 2024-09-16T00:25:50.865 INFO:tasks.workunit.client.0.smithi112.stdout:2/102: symlink d2/l1f 0 2024-09-16T00:25:50.866 INFO:tasks.workunit.client.0.smithi112.stdout:3/71: rename d2/f8 to d2/d9/f16 0 2024-09-16T00:25:50.867 INFO:tasks.workunit.client.0.smithi112.stdout:2/103: mknod d2/c20 0 2024-09-16T00:25:50.868 INFO:tasks.workunit.client.0.smithi112.stdout:3/72: creat d2/d9/f17 x:0 0 0 2024-09-16T00:25:50.869 INFO:tasks.workunit.client.0.smithi112.stdout:3/73: write d2/d9/f16 [4739579,12874] 0 2024-09-16T00:25:50.870 INFO:tasks.workunit.client.0.smithi112.stdout:3/74: chown d2/f6 264483 1 2024-09-16T00:25:50.870 INFO:tasks.workunit.client.0.smithi112.stdout:2/104: dread f1 [0,4194304] 0 2024-09-16T00:25:50.871 INFO:tasks.workunit.client.0.smithi112.stdout:2/105: read - d2/dd/f1a zero size 2024-09-16T00:25:50.872 INFO:tasks.workunit.client.0.smithi112.stdout:3/75: mknod d2/d9/c18 0 2024-09-16T00:25:50.873 INFO:tasks.workunit.client.0.smithi112.stdout:3/76: creat d2/f19 x:0 0 0 2024-09-16T00:25:50.874 INFO:tasks.workunit.client.0.smithi112.stdout:3/77: write d2/d9/f17 [44325,129146] 0 2024-09-16T00:25:50.875 INFO:tasks.workunit.client.0.smithi112.stdout:2/106: rename d2/c20 to d2/c21 0 2024-09-16T00:25:50.876 INFO:tasks.workunit.client.0.smithi112.stdout:3/78: stat d2/d9/c18 0 2024-09-16T00:25:50.876 INFO:tasks.workunit.client.0.smithi112.stdout:2/107: mkdir d2/dc/d22 0 2024-09-16T00:25:50.877 INFO:tasks.workunit.client.0.smithi112.stdout:2/108: creat d2/dc/f23 x:0 0 0 2024-09-16T00:25:50.881 INFO:tasks.workunit.client.0.smithi112.stdout:3/79: dwrite d2/d9/f16 [4194304,4194304] 0 2024-09-16T00:25:50.883 INFO:tasks.workunit.client.0.smithi112.stdout:3/80: rename d2/f6 to d2/f1a 0 2024-09-16T00:25:50.889 INFO:tasks.workunit.client.0.smithi112.stdout:3/81: dwrite d2/d9/f14 [0,4194304] 0 2024-09-16T00:25:50.889 INFO:tasks.workunit.client.0.smithi112.stdout:3/82: chown d2/f19 13104642 1 2024-09-16T00:25:50.891 INFO:tasks.workunit.client.0.smithi112.stdout:3/83: symlink d2/l1b 0 2024-09-16T00:25:50.892 INFO:tasks.workunit.client.0.smithi112.stdout:3/84: creat d2/d9/f1c x:0 0 0 2024-09-16T00:25:50.894 INFO:tasks.workunit.client.0.smithi112.stdout:3/85: creat d2/f1d x:0 0 0 2024-09-16T00:25:50.895 INFO:tasks.workunit.client.0.smithi112.stdout:3/86: rename d2/d9/f10 to d2/d9/f1e 0 2024-09-16T00:25:50.897 INFO:tasks.workunit.client.0.smithi112.stdout:3/87: creat d2/f1f x:0 0 0 2024-09-16T00:25:50.898 INFO:tasks.workunit.client.0.smithi112.stdout:3/88: mkdir d2/d9/d20 0 2024-09-16T00:25:50.899 INFO:tasks.workunit.client.0.smithi112.stdout:3/89: write d2/f1a [2834718,45333] 0 2024-09-16T00:25:50.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:50.970+0000 7f0c2d84b640 1 -- 172.21.15.112:0/262988361 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c280ff700 msgr2=0x7f0c28101ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:50.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:50.970+0000 7f0c2d84b640 1 --2- 172.21.15.112:0/262988361 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c280ff700 0x7f0c28101ae0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0c100048d0 tx=0x7f0c10009da0 comp rx=0 tx=0).stop 2024-09-16T00:25:50.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:50.970+0000 7f0c2d84b640 1 -- 172.21.15.112:0/262988361 shutdown_connections 2024-09-16T00:25:50.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:50.970+0000 7f0c2d84b640 1 --2- 172.21.15.112:0/262988361 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c280ff700 0x7f0c28101ae0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:50.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:50.970+0000 7f0c2d84b640 1 --2- 172.21.15.112:0/262988361 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c28065bc0 0x7f0c280ff1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:50.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:50.970+0000 7f0c2d84b640 1 -- 172.21.15.112:0/262988361 >> 172.21.15.112:0/262988361 conn(0x7f0c280f6c10 msgr2=0x7f0c280f9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:50.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:50.970+0000 7f0c2d84b640 1 -- 172.21.15.112:0/262988361 shutdown_connections 2024-09-16T00:25:50.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:50.971+0000 7f0c2d84b640 1 -- 172.21.15.112:0/262988361 wait complete. 2024-09-16T00:25:50.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:50.971+0000 7f0c2d84b640 1 Processor -- start 2024-09-16T00:25:50.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:50.971+0000 7f0c2d84b640 1 -- start start 2024-09-16T00:25:50.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:50.971+0000 7f0c2d84b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c28065bc0 0x7f0c2810d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:50.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:50.971+0000 7f0c2d84b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c280ff700 0x7f0c2810d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:50.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:50.971+0000 7f0c2d84b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2810dcb0 con 0x7f0c280ff700 2024-09-16T00:25:50.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:50.971+0000 7f0c2d84b640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2810e480 con 0x7f0c28065bc0 2024-09-16T00:25:50.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:50.971+0000 7f0c267fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c280ff700 0x7f0c2810d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:50.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:50.972+0000 7f0c267fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c280ff700 0x7f0c2810d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55236/0 (socket says 172.21.15.112:55236) 2024-09-16T00:25:50.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:50.972+0000 7f0c267fc640 1 -- 172.21.15.112:0/4081382271 learned_addr learned my addr 172.21.15.112:0/4081382271 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:25:50.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:50.972+0000 7f0c267fc640 1 -- 172.21.15.112:0/4081382271 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c28065bc0 msgr2=0x7f0c2810d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:50.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:50.972+0000 7f0c267fc640 1 --2- 172.21.15.112:0/4081382271 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c28065bc0 0x7f0c2810d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:50.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:50.972+0000 7f0c267fc640 1 -- 172.21.15.112:0/4081382271 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c10005040 con 0x7f0c280ff700 2024-09-16T00:25:50.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:50.972+0000 7f0c267fc640 1 --2- 172.21.15.112:0/4081382271 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c280ff700 0x7f0c2810d660 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f0c100043d0 tx=0x7f0c10009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:50.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:50.972+0000 7f0c1f7fe640 1 -- 172.21.15.112:0/4081382271 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c10038070 con 0x7f0c280ff700 2024-09-16T00:25:50.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:50.972+0000 7f0c1f7fe640 1 -- 172.21.15.112:0/4081382271 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f0c1002ae90 con 0x7f0c280ff700 2024-09-16T00:25:50.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:50.972+0000 7f0c2d84b640 1 -- 172.21.15.112:0/4081382271 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c2810e6b0 con 0x7f0c280ff700 2024-09-16T00:25:50.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:50.972+0000 7f0c1f7fe640 1 -- 172.21.15.112:0/4081382271 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c1003e670 con 0x7f0c280ff700 2024-09-16T00:25:50.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:50.972+0000 7f0c2d84b640 1 -- 172.21.15.112:0/4081382271 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c2810ec00 con 0x7f0c280ff700 2024-09-16T00:25:50.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:50.973+0000 7f0c2d84b640 1 -- 172.21.15.112:0/4081382271 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c280fa890 con 0x7f0c280ff700 2024-09-16T00:25:50.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:50.974+0000 7f0c1f7fe640 1 -- 172.21.15.112:0/4081382271 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 20) v1 ==== 91584+0+0 (secure 0 0 0) 0x7f0c10030020 con 0x7f0c280ff700 2024-09-16T00:25:50.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:50.974+0000 7f0c1f7fe640 1 --2- 172.21.15.112:0/4081382271 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f0c1406e5c0 0x7f0c14070a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:50.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:50.974+0000 7f0c1f7fe640 1 -- 172.21.15.112:0/4081382271 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f0c100addb0 con 0x7f0c280ff700 2024-09-16T00:25:50.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:50.974+0000 7f0c26ffd640 1 --2- 172.21.15.112:0/4081382271 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f0c1406e5c0 0x7f0c14070a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:50.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:50.975+0000 7f0c26ffd640 1 --2- 172.21.15.112:0/4081382271 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f0c1406e5c0 0x7f0c14070a70 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0c28065920 tx=0x7f0c18004b00 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:50.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:50.979+0000 7f0c1f7fe640 1 -- 172.21.15.112:0/4081382271 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f0c1007ae80 con 0x7f0c280ff700 2024-09-16T00:25:50.995 INFO:tasks.workunit.client.0.smithi112.stdout:0/72: truncate d5/ff 1969093 0 2024-09-16T00:25:51.012 INFO:tasks.workunit.client.0.smithi112.stdout:2/109: truncate f1 3231235 0 2024-09-16T00:25:51.016 INFO:tasks.workunit.client.0.smithi112.stdout:3/90: getdents d2/d9 0 2024-09-16T00:25:51.019 INFO:tasks.workunit.client.0.smithi112.stdout:0/73: dread d5/ff [0,4194304] 0 2024-09-16T00:25:51.019 INFO:tasks.workunit.client.0.smithi112.stdout:3/91: creat d2/d9/f21 x:0 0 0 2024-09-16T00:25:51.021 INFO:tasks.workunit.client.0.smithi112.stdout:0/74: write d5/da/dc/f13 [153351,73111] 0 2024-09-16T00:25:51.027 INFO:tasks.workunit.client.0.smithi112.stdout:3/92: rmdir d2/d9/d20 0 2024-09-16T00:25:51.028 INFO:tasks.workunit.client.0.smithi112.stdout:3/93: mknod d2/c22 0 2024-09-16T00:25:51.029 INFO:tasks.workunit.client.0.smithi112.stdout:3/94: write d2/f1f [618468,36505] 0 2024-09-16T00:25:51.031 INFO:tasks.workunit.client.0.smithi112.stdout:3/95: symlink d2/d9/l23 0 2024-09-16T00:25:51.035 INFO:tasks.workunit.client.0.smithi112.stdout:3/96: link d2/d9/f14 d2/f24 0 2024-09-16T00:25:51.037 INFO:tasks.workunit.client.0.smithi112.stdout:3/97: creat d2/f25 x:0 0 0 2024-09-16T00:25:51.039 INFO:tasks.workunit.client.0.smithi112.stdout:3/98: mknod d2/c26 0 2024-09-16T00:25:51.039 INFO:tasks.workunit.client.0.smithi112.stdout:3/99: write d2/f1f [1073158,28566] 0 2024-09-16T00:25:51.044 INFO:tasks.workunit.client.0.smithi112.stdout:3/100: fsync d2/f24 0 2024-09-16T00:25:51.049 INFO:tasks.workunit.client.0.smithi112.stdout:3/101: getdents d2 0 2024-09-16T00:25:51.050 INFO:tasks.workunit.client.0.smithi112.stdout:3/102: fsync d2/d9/fe 0 2024-09-16T00:25:51.056 INFO:tasks.workunit.client.0.smithi112.stdout:3/103: dwrite d2/d9/f17 [0,4194304] 0 2024-09-16T00:25:51.058 INFO:tasks.workunit.client.0.smithi112.stdout:3/104: symlink d2/d9/l27 0 2024-09-16T00:25:51.060 INFO:tasks.workunit.client.0.smithi112.stdout:3/105: mkdir d2/d9/d28 0 2024-09-16T00:25:51.062 INFO:tasks.workunit.client.0.smithi112.stdout:3/106: creat d2/d9/d28/f29 x:0 0 0 2024-09-16T00:25:51.062 INFO:tasks.workunit.client.0.smithi112.stdout:3/107: readlink d2/d9/l13 0 2024-09-16T00:25:51.064 INFO:tasks.workunit.client.0.smithi112.stdout:3/108: symlink d2/d9/l2a 0 2024-09-16T00:25:51.065 INFO:tasks.workunit.client.0.smithi112.stdout:3/109: write d2/fb [1858649,98368] 0 2024-09-16T00:25:51.066 INFO:tasks.workunit.client.0.smithi112.stdout:3/110: chown d2/d9/d28/f29 727052245 1 2024-09-16T00:25:51.067 INFO:tasks.workunit.client.0.smithi112.stdout:3/111: dread - d2/d9/d28/f29 zero size 2024-09-16T00:25:51.076 INFO:tasks.workunit.client.0.smithi112.stdout:3/112: rmdir d2/d9/d28 39 2024-09-16T00:25:51.078 INFO:tasks.workunit.client.0.smithi112.stdout:3/113: rename d2/d9/d28 to d2/d9/d2b 0 2024-09-16T00:25:51.084 INFO:tasks.workunit.client.0.smithi112.stdout:4/87: sync 2024-09-16T00:25:51.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.174+0000 7f0c2d84b640 1 -- 172.21.15.112:0/4081382271 --> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0c2810efe0 con 0x7f0c1406e5c0 2024-09-16T00:25:51.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.175+0000 7f0c1f7fe640 1 -- 172.21.15.112:0/4081382271 <== mgr.14208 v2:172.21.15.112:6800/2480414866 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7f0c2810efe0 con 0x7f0c1406e5c0 2024-09-16T00:25:51.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.177+0000 7f0c2d84b640 1 -- 172.21.15.112:0/4081382271 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f0c1406e5c0 msgr2=0x7f0c14070a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:51.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.177+0000 7f0c2d84b640 1 --2- 172.21.15.112:0/4081382271 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f0c1406e5c0 0x7f0c14070a70 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0c28065920 tx=0x7f0c18004b00 comp rx=0 tx=0).stop 2024-09-16T00:25:51.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.178+0000 7f0c2d84b640 1 -- 172.21.15.112:0/4081382271 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c280ff700 msgr2=0x7f0c2810d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:51.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.178+0000 7f0c2d84b640 1 --2- 172.21.15.112:0/4081382271 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c280ff700 0x7f0c2810d660 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f0c100043d0 tx=0x7f0c10009da0 comp rx=0 tx=0).stop 2024-09-16T00:25:51.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.178+0000 7f0c2d84b640 1 -- 172.21.15.112:0/4081382271 shutdown_connections 2024-09-16T00:25:51.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.178+0000 7f0c2d84b640 1 --2- 172.21.15.112:0/4081382271 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f0c1406e5c0 0x7f0c14070a70 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:51.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.178+0000 7f0c2d84b640 1 --2- 172.21.15.112:0/4081382271 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c280ff700 0x7f0c2810d660 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:51.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.178+0000 7f0c2d84b640 1 --2- 172.21.15.112:0/4081382271 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c28065bc0 0x7f0c2810d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:51.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.178+0000 7f0c2d84b640 1 -- 172.21.15.112:0/4081382271 >> 172.21.15.112:0/4081382271 conn(0x7f0c280f6c10 msgr2=0x7f0c280f6fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:51.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.178+0000 7f0c2d84b640 1 -- 172.21.15.112:0/4081382271 shutdown_connections 2024-09-16T00:25:51.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.178+0000 7f0c2d84b640 1 -- 172.21.15.112:0/4081382271 wait complete. 2024-09-16T00:25:51.187 INFO:teuthology.orchestra.run.smithi112.stdout:true 2024-09-16T00:25:51.219 INFO:tasks.workunit.client.0.smithi112.stdout:9/198: sync 2024-09-16T00:25:51.219 INFO:tasks.workunit.client.0.smithi112.stdout:5/149: sync 2024-09-16T00:25:51.219 INFO:tasks.workunit.client.0.smithi112.stdout:8/59: sync 2024-09-16T00:25:51.219 INFO:tasks.workunit.client.0.smithi112.stdout:1/61: sync 2024-09-16T00:25:51.219 INFO:tasks.workunit.client.0.smithi112.stdout:6/120: sync 2024-09-16T00:25:51.219 INFO:tasks.workunit.client.0.smithi112.stdout:7/91: sync 2024-09-16T00:25:51.219 INFO:tasks.workunit.client.0.smithi112.stdout:2/110: sync 2024-09-16T00:25:51.220 INFO:tasks.workunit.client.0.smithi112.stdout:7/92: fdatasync d6/f11 0 2024-09-16T00:25:51.220 INFO:tasks.workunit.client.0.smithi112.stdout:6/121: read - dd/d15/f29 zero size 2024-09-16T00:25:51.222 INFO:tasks.workunit.client.0.smithi112.stdout:2/111: write d2/dc/f11 [548542,13300] 0 2024-09-16T00:25:51.222 INFO:tasks.workunit.client.0.smithi112.stdout:8/60: write fe [836596,61690] 0 2024-09-16T00:25:51.222 INFO:tasks.workunit.client.0.smithi112.stdout:7/93: chown l3 0 1 2024-09-16T00:25:51.224 INFO:tasks.workunit.client.0.smithi112.stdout:5/150: read d4/fb [654175,80844] 0 2024-09-16T00:25:51.225 INFO:tasks.workunit.client.0.smithi112.stdout:8/61: fdatasync fe 0 2024-09-16T00:25:51.226 INFO:tasks.workunit.client.0.smithi112.stdout:2/112: write d2/fa [1459851,15429] 0 2024-09-16T00:25:51.227 INFO:tasks.workunit.client.0.smithi112.stdout:2/113: chown d2/f8 1646941998 1 2024-09-16T00:25:51.227 INFO:tasks.workunit.client.0.smithi112.stdout:2/114: dread - d2/dd/f12 zero size 2024-09-16T00:25:51.228 INFO:tasks.workunit.client.0.smithi112.stdout:6/122: symlink dd/df/d17/l2b 0 2024-09-16T00:25:51.228 INFO:tasks.workunit.client.0.smithi112.stdout:2/115: write d2/dc/f11 [1164679,37375] 0 2024-09-16T00:25:51.229 INFO:tasks.workunit.client.0.smithi112.stdout:7/94: mkdir d6/d26 0 2024-09-16T00:25:51.231 INFO:tasks.workunit.client.0.smithi112.stdout:9/199: creat d1/d5/d6/f4a x:0 0 0 2024-09-16T00:25:51.232 INFO:tasks.workunit.client.0.smithi112.stdout:1/62: link d0/d14/df/f12 d0/d14/df/f15 0 2024-09-16T00:25:51.236 INFO:tasks.workunit.client.0.smithi112.stdout:1/63: readlink d0/d14/df/l10 0 2024-09-16T00:25:51.236 INFO:tasks.workunit.client.0.smithi112.stdout:7/95: creat d6/d26/f27 x:0 0 0 2024-09-16T00:25:51.238 INFO:tasks.workunit.client.0.smithi112.stdout:8/62: dwrite f4 [0,4194304] 0 2024-09-16T00:25:51.238 INFO:tasks.workunit.client.0.smithi112.stdout:9/200: creat d1/d5/d6/d18/f4b x:0 0 0 2024-09-16T00:25:51.239 INFO:tasks.workunit.client.0.smithi112.stdout:1/64: dread - d0/d14/f11 zero size 2024-09-16T00:25:51.241 INFO:tasks.workunit.client.0.smithi112.stdout:9/201: dread - d1/d5/d6/d18/f40 zero size 2024-09-16T00:25:51.243 INFO:tasks.workunit.client.0.smithi112.stdout:1/65: symlink d0/d14/l16 0 2024-09-16T00:25:51.244 INFO:tasks.workunit.client.0.smithi112.stdout:8/63: mkdir d10/d12/d13 0 2024-09-16T00:25:51.244 INFO:tasks.workunit.client.0.smithi112.stdout:8/64: fdatasync fe 0 2024-09-16T00:25:51.247 INFO:tasks.workunit.client.0.smithi112.stdout:1/66: mknod d0/c17 0 2024-09-16T00:25:51.248 INFO:tasks.workunit.client.0.smithi112.stdout:7/96: dread d6/f9 [0,4194304] 0 2024-09-16T00:25:51.252 INFO:tasks.workunit.client.0.smithi112.stdout:1/67: unlink d0/lc 0 2024-09-16T00:25:51.258 INFO:tasks.workunit.client.0.smithi112.stdout:7/97: getdents d6/d26 0 2024-09-16T00:25:51.259 INFO:tasks.workunit.client.0.smithi112.stdout:1/68: chown d0/d14/df/f15 60699779 1 2024-09-16T00:25:51.261 INFO:tasks.workunit.client.0.smithi112.stdout:2/116: dwrite d2/dd/f1a [0,4194304] 0 2024-09-16T00:25:51.263 INFO:tasks.workunit.client.0.smithi112.stdout:9/202: dwrite d1/d5/d6/f34 [0,4194304] 0 2024-09-16T00:25:51.264 INFO:tasks.workunit.client.0.smithi112.stdout:8/65: dread fe [0,4194304] 0 2024-09-16T00:25:51.265 INFO:tasks.workunit.client.0.smithi112.stdout:7/98: truncate d6/d26/f27 805358 0 2024-09-16T00:25:51.265 INFO:tasks.workunit.client.0.smithi112.stdout:9/203: read d1/d5/d6/d18/f28 [368750,89308] 0 2024-09-16T00:25:51.267 INFO:tasks.workunit.client.0.smithi112.stdout:7/99: write d6/d10/f1c [414969,122716] 0 2024-09-16T00:25:51.268 INFO:tasks.workunit.client.0.smithi112.stdout:7/100: read d6/f9 [1734031,106278] 0 2024-09-16T00:25:51.271 INFO:tasks.workunit.client.0.smithi112.stdout:7/101: truncate d6/fa 1077730 0 2024-09-16T00:25:51.277 INFO:tasks.workunit.client.0.smithi112.stdout:2/117: link d2/f8 d2/dc/f24 0 2024-09-16T00:25:51.278 INFO:tasks.workunit.client.0.smithi112.stdout:2/118: write d2/f19 [1275076,9124] 0 2024-09-16T00:25:51.278 INFO:tasks.workunit.client.0.smithi112.stdout:8/66: rmdir d10/d12/d13 0 2024-09-16T00:25:51.279 INFO:tasks.workunit.client.0.smithi112.stdout:8/67: chown l7 0 1 2024-09-16T00:25:51.280 INFO:tasks.workunit.client.0.smithi112.stdout:1/69: rename d0/d14/df/f12 to d0/f18 0 2024-09-16T00:25:51.283 INFO:tasks.workunit.client.0.smithi112.stdout:9/204: creat d1/d5/d20/f4c x:0 0 0 2024-09-16T00:25:51.283 INFO:tasks.workunit.client.0.smithi112.stdout:8/68: dread fe [0,4194304] 0 2024-09-16T00:25:51.284 INFO:tasks.workunit.client.0.smithi112.stdout:8/69: write fe [3221692,48617] 0 2024-09-16T00:25:51.286 INFO:tasks.workunit.client.0.smithi112.stdout:8/70: unlink l7 0 2024-09-16T00:25:51.287 INFO:tasks.workunit.client.0.smithi112.stdout:8/71: read f4 [2836453,99342] 0 2024-09-16T00:25:51.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.287+0000 7f75c85e0640 1 -- 172.21.15.112:0/1421260893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75c00fc370 msgr2=0x7f75c00fc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:51.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.287+0000 7f75c85e0640 1 --2- 172.21.15.112:0/1421260893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75c00fc370 0x7f75c00fc7e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f75b00048d0 tx=0x7f75b0009da0 comp rx=0 tx=0).stop 2024-09-16T00:25:51.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.288+0000 7f75c85e0640 1 -- 172.21.15.112:0/1421260893 shutdown_connections 2024-09-16T00:25:51.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.288+0000 7f75c85e0640 1 --2- 172.21.15.112:0/1421260893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75c00fc370 0x7f75c00fc7e0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:51.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.288+0000 7f75c85e0640 1 --2- 172.21.15.112:0/1421260893 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75c00fba20 0x7f75c00fbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:51.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.288+0000 7f75c85e0640 1 -- 172.21.15.112:0/1421260893 >> 172.21.15.112:0/1421260893 conn(0x7f75c00f6390 msgr2=0x7f75c00f87a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:51.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.288+0000 7f75c85e0640 1 -- 172.21.15.112:0/1421260893 shutdown_connections 2024-09-16T00:25:51.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.288+0000 7f75c85e0640 1 -- 172.21.15.112:0/1421260893 wait complete. 2024-09-16T00:25:51.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.289+0000 7f75c85e0640 1 Processor -- start 2024-09-16T00:25:51.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.289+0000 7f75c85e0640 1 -- start start 2024-09-16T00:25:51.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.289+0000 7f75c85e0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75c00fba20 0x7f75c006dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:51.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.289+0000 7f75c85e0640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75c00fc370 0x7f75c006e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:51.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.289+0000 7f75c85e0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75c006f7a0 con 0x7f75c00fba20 2024-09-16T00:25:51.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.289+0000 7f75c85e0640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75c006e650 con 0x7f75c00fc370 2024-09-16T00:25:51.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.290+0000 7f75c5b7b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75c00fc370 0x7f75c006e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:51.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.290+0000 7f75c637c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75c00fba20 0x7f75c006dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:51.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.290+0000 7f75c637c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75c00fba20 0x7f75c006dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55262/0 (socket says 172.21.15.112:55262) 2024-09-16T00:25:51.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.290+0000 7f75c637c640 1 -- 172.21.15.112:0/1480106705 learned_addr learned my addr 172.21.15.112:0/1480106705 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:25:51.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.290+0000 7f75c637c640 1 -- 172.21.15.112:0/1480106705 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75c00fc370 msgr2=0x7f75c006e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:51.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.290+0000 7f75c637c640 1 --2- 172.21.15.112:0/1480106705 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75c00fc370 0x7f75c006e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:51.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.290+0000 7f75c637c640 1 -- 172.21.15.112:0/1480106705 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75b0005040 con 0x7f75c00fba20 2024-09-16T00:25:51.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.290+0000 7f75c637c640 1 --2- 172.21.15.112:0/1480106705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75c00fba20 0x7f75c006dbd0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f75bc00eb70 tx=0x7f75bc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:51.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.291+0000 7f75b6ffd640 1 -- 172.21.15.112:0/1480106705 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f75bc007c30 con 0x7f75c00fba20 2024-09-16T00:25:51.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.291+0000 7f75b6ffd640 1 -- 172.21.15.112:0/1480106705 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f75bc00f040 con 0x7f75c00fba20 2024-09-16T00:25:51.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.291+0000 7f75b6ffd640 1 -- 172.21.15.112:0/1480106705 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f75bc00cb80 con 0x7f75c00fba20 2024-09-16T00:25:51.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.291+0000 7f75c85e0640 1 -- 172.21.15.112:0/1480106705 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75c006e900 con 0x7f75c00fba20 2024-09-16T00:25:51.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.291+0000 7f75c85e0640 1 -- 172.21.15.112:0/1480106705 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75c01193f0 con 0x7f75c00fba20 2024-09-16T00:25:51.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.292+0000 7f75c85e0640 1 -- 172.21.15.112:0/1480106705 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75c00fa010 con 0x7f75c00fba20 2024-09-16T00:25:51.294 INFO:tasks.workunit.client.0.smithi112.stdout:9/205: dwrite d1/d5/d6/d18/f40 [0,4194304] 0 2024-09-16T00:25:51.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.293+0000 7f75b6ffd640 1 -- 172.21.15.112:0/1480106705 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 20) v1 ==== 91584+0+0 (secure 0 0 0) 0x7f75bc00cd00 con 0x7f75c00fba20 2024-09-16T00:25:51.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.294+0000 7f75b6ffd640 1 --2- 172.21.15.112:0/1480106705 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f75ac0729d0 0x7f75ac074e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:51.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.294+0000 7f75b6ffd640 1 -- 172.21.15.112:0/1480106705 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f75bc08dc40 con 0x7f75c00fba20 2024-09-16T00:25:51.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.295+0000 7f75c5b7b640 1 --2- 172.21.15.112:0/1480106705 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f75ac0729d0 0x7f75ac074e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:51.298 INFO:tasks.workunit.client.0.smithi112.stdout:8/72: link lc d10/l14 0 2024-09-16T00:25:51.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.295+0000 7f75c5b7b640 1 --2- 172.21.15.112:0/1480106705 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f75ac0729d0 0x7f75ac074e80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f75b000b020 tx=0x7f75b0035040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:51.298 INFO:tasks.workunit.client.0.smithi112.stdout:1/70: dread - d0/d14/df/f15 zero size 2024-09-16T00:25:51.300 INFO:tasks.workunit.client.0.smithi112.stdout:2/119: truncate d2/f19 2395104 0 2024-09-16T00:25:51.300 INFO:tasks.workunit.client.0.smithi112.stdout:8/73: rmdir d10 39 2024-09-16T00:25:51.304 INFO:tasks.workunit.client.0.smithi112.stdout:1/71: dread - d0/d14/f11 zero size 2024-09-16T00:25:51.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.302+0000 7f75b6ffd640 1 -- 172.21.15.112:0/1480106705 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f75bc05ad90 con 0x7f75c00fba20 2024-09-16T00:25:51.305 INFO:tasks.workunit.client.0.smithi112.stdout:1/72: rmdir d0 39 2024-09-16T00:25:51.306 INFO:tasks.workunit.client.0.smithi112.stdout:9/206: dwrite d1/d5/d6/d18/d23/f2b [0,4194304] 0 2024-09-16T00:25:51.309 INFO:tasks.workunit.client.0.smithi112.stdout:2/120: rename d2/c16 to d2/c25 0 2024-09-16T00:25:51.312 INFO:tasks.workunit.client.0.smithi112.stdout:8/74: creat d10/f15 x:0 0 0 2024-09-16T00:25:51.316 INFO:tasks.workunit.client.0.smithi112.stdout:2/121: fdatasync d2/f14 0 2024-09-16T00:25:51.317 INFO:tasks.workunit.client.0.smithi112.stdout:9/207: creat d1/d5/d6/d2d/d45/f4d x:0 0 0 2024-09-16T00:25:51.318 INFO:tasks.workunit.client.0.smithi112.stdout:1/73: dwrite d0/d14/f8 [0,4194304] 0 2024-09-16T00:25:51.323 INFO:tasks.workunit.client.0.smithi112.stdout:0/75: sync 2024-09-16T00:25:51.323 INFO:tasks.workunit.client.0.smithi112.stdout:3/114: sync 2024-09-16T00:25:51.323 INFO:tasks.workunit.client.0.smithi112.stdout:4/88: sync 2024-09-16T00:25:51.329 INFO:tasks.workunit.client.0.smithi112.stdout:9/208: chown d1/d5/d6/d18/f19 1853 1 2024-09-16T00:25:51.329 INFO:tasks.workunit.client.0.smithi112.stdout:2/122: dwrite d2/dd/f12 [0,4194304] 0 2024-09-16T00:25:51.331 INFO:tasks.workunit.client.0.smithi112.stdout:1/74: link d0/d14/f8 d0/d14/df/f19 0 2024-09-16T00:25:51.331 INFO:tasks.workunit.client.0.smithi112.stdout:8/75: truncate fe 874385 0 2024-09-16T00:25:51.334 INFO:tasks.workunit.client.0.smithi112.stdout:3/115: mkdir d2/d9/d2b/d2c 0 2024-09-16T00:25:51.336 INFO:tasks.workunit.client.0.smithi112.stdout:9/209: symlink d1/d5/d6/d2d/d45/l4e 0 2024-09-16T00:25:51.336 INFO:tasks.workunit.client.0.smithi112.stdout:9/210: chown d1/d5/d6/d2d 1015080677 1 2024-09-16T00:25:51.338 INFO:tasks.workunit.client.0.smithi112.stdout:9/211: dread - d1/d5/d6/d2d/d39/f3c zero size 2024-09-16T00:25:51.339 INFO:tasks.workunit.client.0.smithi112.stdout:3/116: creat d2/f2d x:0 0 0 2024-09-16T00:25:51.340 INFO:tasks.workunit.client.0.smithi112.stdout:2/123: fsync f1 0 2024-09-16T00:25:51.341 INFO:tasks.workunit.client.0.smithi112.stdout:1/75: symlink d0/l1a 0 2024-09-16T00:25:51.343 INFO:tasks.workunit.client.0.smithi112.stdout:9/212: dread d1/d5/f30 [0,4194304] 0 2024-09-16T00:25:51.344 INFO:tasks.workunit.client.0.smithi112.stdout:1/76: dread - d0/d14/f11 zero size 2024-09-16T00:25:51.344 INFO:tasks.workunit.client.0.smithi112.stdout:0/76: getdents d5/da/dc 0 2024-09-16T00:25:51.346 INFO:tasks.workunit.client.0.smithi112.stdout:2/124: mknod d2/dd/c26 0 2024-09-16T00:25:51.346 INFO:tasks.workunit.client.0.smithi112.stdout:9/213: truncate d1/d5/d6/d2d/d39/f42 158247 0 2024-09-16T00:25:51.348 INFO:tasks.workunit.client.0.smithi112.stdout:2/125: write d2/f14 [858542,59237] 0 2024-09-16T00:25:51.350 INFO:tasks.workunit.client.0.smithi112.stdout:3/117: symlink d2/d9/d2b/d2c/l2e 0 2024-09-16T00:25:51.353 INFO:tasks.workunit.client.0.smithi112.stdout:1/77: dwrite d0/d14/f11 [0,4194304] 0 2024-09-16T00:25:51.354 INFO:tasks.workunit.client.0.smithi112.stdout:0/77: unlink c3 0 2024-09-16T00:25:51.354 INFO:tasks.workunit.client.0.smithi112.stdout:1/78: truncate d0/d14/fd 388702 0 2024-09-16T00:25:51.356 INFO:tasks.workunit.client.0.smithi112.stdout:2/126: truncate f1 2889495 0 2024-09-16T00:25:51.359 INFO:tasks.workunit.client.0.smithi112.stdout:8/76: link f4 d10/d12/f16 0 2024-09-16T00:25:51.360 INFO:tasks.workunit.client.0.smithi112.stdout:9/214: rename d1/f46 to d1/d5/d6/d38/f4f 0 2024-09-16T00:25:51.361 INFO:tasks.workunit.client.0.smithi112.stdout:3/118: rmdir d2 39 2024-09-16T00:25:51.363 INFO:tasks.workunit.client.0.smithi112.stdout:8/77: rmdir d10 39 2024-09-16T00:25:51.366 INFO:tasks.workunit.client.0.smithi112.stdout:9/215: readlink d1/d5/d6/d18/d23/l3d 0 2024-09-16T00:25:51.369 INFO:tasks.workunit.client.0.smithi112.stdout:0/78: dwrite d5/da/dc/f13 [0,4194304] 0 2024-09-16T00:25:51.373 INFO:tasks.workunit.client.0.smithi112.stdout:2/127: dwrite d2/dd/f15 [0,4194304] 0 2024-09-16T00:25:51.373 INFO:tasks.workunit.client.0.smithi112.stdout:1/79: dread d0/d14/df/f19 [0,4194304] 0 2024-09-16T00:25:51.374 INFO:tasks.workunit.client.0.smithi112.stdout:9/216: creat d1/f50 x:0 0 0 2024-09-16T00:25:51.376 INFO:tasks.workunit.client.0.smithi112.stdout:2/128: creat d2/dc/f27 x:0 0 0 2024-09-16T00:25:51.377 INFO:tasks.workunit.client.0.smithi112.stdout:1/80: fsync d0/f18 0 2024-09-16T00:25:51.378 INFO:tasks.workunit.client.0.smithi112.stdout:3/119: dread d2/f1f [0,4194304] 0 2024-09-16T00:25:51.383 INFO:tasks.workunit.client.0.smithi112.stdout:9/217: symlink d1/d5/d6/l51 0 2024-09-16T00:25:51.385 INFO:tasks.workunit.client.0.smithi112.stdout:9/218: chown d1/d5/d6/d18/f19 801773 1 2024-09-16T00:25:51.386 INFO:tasks.workunit.client.0.smithi112.stdout:0/79: chown d5/da/dc/f13 153131 1 2024-09-16T00:25:51.386 INFO:tasks.workunit.client.0.smithi112.stdout:3/120: rmdir d2/d9/d2b/d2c 39 2024-09-16T00:25:51.387 INFO:tasks.workunit.client.0.smithi112.stdout:1/81: dwrite d0/d14/fd [0,4194304] 0 2024-09-16T00:25:51.388 INFO:tasks.workunit.client.0.smithi112.stdout:3/121: chown d2/d9/l13 54709 1 2024-09-16T00:25:51.394 INFO:tasks.workunit.client.0.smithi112.stdout:8/78: write fe [1709269,127910] 0 2024-09-16T00:25:51.394 INFO:tasks.workunit.client.0.smithi112.stdout:9/219: mknod d1/d5/d6/c52 0 2024-09-16T00:25:51.394 INFO:tasks.workunit.client.0.smithi112.stdout:2/129: getdents d2/dc 0 2024-09-16T00:25:51.397 INFO:tasks.workunit.client.0.smithi112.stdout:1/82: creat d0/f1b x:0 0 0 2024-09-16T00:25:51.400 INFO:tasks.workunit.client.0.smithi112.stdout:1/83: mkdir d0/d14/d1c 0 2024-09-16T00:25:51.404 INFO:tasks.workunit.client.0.smithi112.stdout:1/84: read d0/d14/f11 [1865312,103076] 0 2024-09-16T00:25:51.407 INFO:tasks.workunit.client.0.smithi112.stdout:0/80: dwrite d5/da/f15 [0,4194304] 0 2024-09-16T00:25:51.412 INFO:tasks.workunit.client.0.smithi112.stdout:2/130: dwrite d2/fa [0,4194304] 0 2024-09-16T00:25:51.412 INFO:tasks.workunit.client.0.smithi112.stdout:0/81: chown d5/da/d16 125 1 2024-09-16T00:25:51.414 INFO:tasks.workunit.client.0.smithi112.stdout:9/220: dwrite d1/d5/d6/f4a [0,4194304] 0 2024-09-16T00:25:51.415 INFO:tasks.workunit.client.0.smithi112.stdout:8/79: dread d10/d12/f16 [0,4194304] 0 2024-09-16T00:25:51.416 INFO:tasks.workunit.client.0.smithi112.stdout:1/85: dread d0/d14/fa [0,4194304] 0 2024-09-16T00:25:51.416 INFO:tasks.workunit.client.0.smithi112.stdout:0/82: read d5/da/dc/f10 [736466,89539] 0 2024-09-16T00:25:51.417 INFO:tasks.workunit.client.0.smithi112.stdout:1/86: dread d0/f2 [0,4194304] 0 2024-09-16T00:25:51.419 INFO:tasks.workunit.client.0.smithi112.stdout:8/80: fsync f4 0 2024-09-16T00:25:51.421 INFO:tasks.workunit.client.0.smithi112.stdout:1/87: fsync d0/f18 0 2024-09-16T00:25:51.423 INFO:tasks.workunit.client.0.smithi112.stdout:8/81: read fe [1692081,47205] 0 2024-09-16T00:25:51.425 INFO:tasks.workunit.client.0.smithi112.stdout:9/221: symlink d1/d5/d20/d25/l53 0 2024-09-16T00:25:51.430 INFO:tasks.workunit.client.0.smithi112.stdout:1/88: link d0/f9 d0/d14/f1d 0 2024-09-16T00:25:51.434 INFO:tasks.workunit.client.0.smithi112.stdout:0/83: dwrite d5/da/dc/fe [0,4194304] 0 2024-09-16T00:25:51.439 INFO:tasks.workunit.client.0.smithi112.stdout:8/82: rename cd to d10/c17 0 2024-09-16T00:25:51.440 INFO:tasks.workunit.client.0.smithi112.stdout:9/222: dwrite d1/d5/d6/d18/f19 [0,4194304] 0 2024-09-16T00:25:51.440 INFO:tasks.workunit.client.0.smithi112.stdout:0/84: write d5/ff [1906065,12046] 0 2024-09-16T00:25:51.442 INFO:tasks.workunit.client.0.smithi112.stdout:9/223: fsync d1/d5/d6/d2d/f3b 0 2024-09-16T00:25:51.442 INFO:tasks.workunit.client.0.smithi112.stdout:8/83: mknod d10/d12/c18 0 2024-09-16T00:25:51.446 INFO:tasks.workunit.client.0.smithi112.stdout:9/224: fdatasync d1/d5/d6/d18/f28 0 2024-09-16T00:25:51.449 INFO:tasks.workunit.client.0.smithi112.stdout:9/225: mkdir d1/d5/d54 0 2024-09-16T00:25:51.449 INFO:tasks.workunit.client.0.smithi112.stdout:9/226: fdatasync d1/d5/d20/d25/f43 0 2024-09-16T00:25:51.451 INFO:tasks.workunit.client.0.smithi112.stdout:9/227: symlink d1/d5/d54/l55 0 2024-09-16T00:25:51.453 INFO:tasks.workunit.client.0.smithi112.stdout:9/228: symlink d1/d5/d6/d2d/d45/d1a/l56 0 2024-09-16T00:25:51.454 INFO:tasks.workunit.client.0.smithi112.stdout:9/229: fsync d1/d5/f2f 0 2024-09-16T00:25:51.456 INFO:tasks.workunit.client.0.smithi112.stdout:9/230: creat d1/d5/d2c/f57 x:0 0 0 2024-09-16T00:25:51.501 INFO:tasks.workunit.client.0.smithi112.stdout:0/85: fsync d5/da/dc/f13 0 2024-09-16T00:25:51.502 INFO:tasks.workunit.client.0.smithi112.stdout:0/86: truncate d5/da/dc/d12/f17 516194 0 2024-09-16T00:25:51.508 INFO:tasks.workunit.client.0.smithi112.stdout:0/87: dread f4 [4194304,4194304] 0 2024-09-16T00:25:51.510 INFO:tasks.workunit.client.0.smithi112.stdout:0/88: mkdir d5/da/d18 0 2024-09-16T00:25:51.520 INFO:tasks.workunit.client.0.smithi112.stdout:1/89: read - d0/d14/df/f15 zero size 2024-09-16T00:25:51.521 INFO:tasks.workunit.client.0.smithi112.stdout:2/131: fdatasync d2/fa 0 2024-09-16T00:25:51.523 INFO:tasks.workunit.client.0.smithi112.stdout:2/132: mknod d2/dd/c28 0 2024-09-16T00:25:51.524 INFO:tasks.workunit.client.0.smithi112.stdout:8/84: getdents d10 0 2024-09-16T00:25:51.526 INFO:tasks.workunit.client.0.smithi112.stdout:8/85: symlink d10/l19 0 2024-09-16T00:25:51.528 INFO:tasks.workunit.client.0.smithi112.stdout:2/133: creat d2/dc/d22/f29 x:0 0 0 2024-09-16T00:25:51.530 INFO:tasks.workunit.client.0.smithi112.stdout:1/90: getdents d0/d14 0 2024-09-16T00:25:51.533 INFO:tasks.workunit.client.0.smithi112.stdout:8/86: creat d10/d12/f1a x:0 0 0 2024-09-16T00:25:51.535 INFO:tasks.workunit.client.0.smithi112.stdout:9/231: rmdir d1/d5 39 2024-09-16T00:25:51.535 INFO:tasks.workunit.client.0.smithi112.stdout:1/91: fsync d0/f18 0 2024-09-16T00:25:51.536 INFO:tasks.workunit.client.0.smithi112.stdout:5/151: sync 2024-09-16T00:25:51.538 INFO:tasks.workunit.client.0.smithi112.stdout:0/89: rmdir d5 39 2024-09-16T00:25:51.538 INFO:tasks.workunit.client.0.smithi112.stdout:2/134: dwrite d2/dd/f15 [0,4194304] 0 2024-09-16T00:25:51.545 INFO:tasks.workunit.client.0.smithi112.stdout:5/152: symlink d4/d7/d26/l34 0 2024-09-16T00:25:51.546 INFO:tasks.workunit.client.0.smithi112.stdout:8/87: write f4 [300703,117629] 0 2024-09-16T00:25:51.548 INFO:tasks.workunit.client.0.smithi112.stdout:0/90: rename d5/da/d16 to d5/da/dc/d19 0 2024-09-16T00:25:51.552 INFO:tasks.workunit.client.0.smithi112.stdout:5/153: fsync d4/dd/d12/f11 0 2024-09-16T00:25:51.552 INFO:tasks.workunit.client.0.smithi112.stdout:1/92: dwrite d0/f1b [0,4194304] 0 2024-09-16T00:25:51.555 INFO:tasks.workunit.client.0.smithi112.stdout:2/135: dwrite d2/dd/f12 [0,4194304] 0 2024-09-16T00:25:51.558 INFO:tasks.workunit.client.0.smithi112.stdout:2/136: fdatasync d2/f14 0 2024-09-16T00:25:51.560 INFO:tasks.workunit.client.0.smithi112.stdout:9/232: dwrite d1/d5/f30 [0,4194304] 0 2024-09-16T00:25:51.564 INFO:tasks.workunit.client.0.smithi112.stdout:8/88: truncate f4 3800498 0 2024-09-16T00:25:51.564 INFO:tasks.workunit.client.0.smithi112.stdout:5/154: unlink d4/dd/f23 0 2024-09-16T00:25:51.567 INFO:tasks.workunit.client.0.smithi112.stdout:0/91: dwrite d5/da/dc/fe [0,4194304] 0 2024-09-16T00:25:51.568 INFO:tasks.workunit.client.0.smithi112.stdout:5/155: write d4/d7/d14/f33 [466523,25857] 0 2024-09-16T00:25:51.570 INFO:tasks.workunit.client.0.smithi112.stdout:1/93: creat d0/f1e x:0 0 0 2024-09-16T00:25:51.572 INFO:tasks.workunit.client.0.smithi112.stdout:9/233: symlink d1/d5/d20/l58 0 2024-09-16T00:25:51.576 INFO:tasks.workunit.client.0.smithi112.stdout:5/156: fdatasync d4/fb 0 2024-09-16T00:25:51.577 INFO:tasks.workunit.client.0.smithi112.stdout:2/137: dread d2/f19 [0,4194304] 0 2024-09-16T00:25:51.579 INFO:tasks.workunit.client.0.smithi112.stdout:9/234: unlink d1/d5/d6/d18/d23/f2b 0 2024-09-16T00:25:51.580 INFO:tasks.workunit.client.0.smithi112.stdout:2/138: fsync d2/dc/f27 0 2024-09-16T00:25:51.581 INFO:tasks.workunit.client.0.smithi112.stdout:5/157: truncate d4/d1c/f32 270545 0 2024-09-16T00:25:51.583 INFO:tasks.workunit.client.0.smithi112.stdout:0/92: dwrite d5/da/f15 [4194304,4194304] 0 2024-09-16T00:25:51.584 INFO:tasks.workunit.client.0.smithi112.stdout:9/235: dread - d1/d5/d6/d2d/d39/f3c zero size 2024-09-16T00:25:51.585 INFO:tasks.workunit.client.0.smithi112.stdout:2/139: write d2/dc/f27 [576850,10527] 0 2024-09-16T00:25:51.586 INFO:tasks.workunit.client.0.smithi112.stdout:0/93: chown d5/ff 40 1 2024-09-16T00:25:51.588 INFO:tasks.workunit.client.0.smithi112.stdout:8/89: write d10/d12/f16 [217308,93680] 0 2024-09-16T00:25:51.588 INFO:tasks.workunit.client.0.smithi112.stdout:1/94: dwrite d0/f2 [0,4194304] 0 2024-09-16T00:25:51.591 INFO:tasks.workunit.client.0.smithi112.stdout:0/94: chown d5/da/f15 3221296 1 2024-09-16T00:25:51.594 INFO:tasks.workunit.client.0.smithi112.stdout:9/236: write d1/d5/d6/d2d/d39/f3c [43237,88063] 0 2024-09-16T00:25:51.594 INFO:tasks.workunit.client.0.smithi112.stdout:8/90: write fe [2721336,128210] 0 2024-09-16T00:25:51.595 INFO:tasks.workunit.client.0.smithi112.stdout:2/140: creat d2/dc/f2a x:0 0 0 2024-09-16T00:25:51.598 INFO:tasks.workunit.client.0.smithi112.stdout:2/141: write d2/dc/d22/f29 [135379,39759] 0 2024-09-16T00:25:51.598 INFO:tasks.workunit.client.0.smithi112.stdout:0/95: mknod d5/da/dc/d12/c1a 0 2024-09-16T00:25:51.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.598+0000 7f75c85e0640 1 -- 172.21.15.112:0/1480106705 --> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f75c0100c10 con 0x7f75ac0729d0 2024-09-16T00:25:51.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.600+0000 7f75b6ffd640 1 -- 172.21.15.112:0/1480106705 <== mgr.14208 v2:172.21.15.112:6800/2480414866 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7f75c0100c10 con 0x7f75ac0729d0 2024-09-16T00:25:51.602 INFO:tasks.workunit.client.0.smithi112.stdout:5/158: truncate d4/f16 2689833 0 2024-09-16T00:25:51.603 INFO:tasks.workunit.client.0.smithi112.stdout:9/237: mkdir d1/d5/d6/d59 0 2024-09-16T00:25:51.604 INFO:tasks.workunit.client.0.smithi112.stdout:2/142: read d2/f8 [2574855,57311] 0 2024-09-16T00:25:51.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.603+0000 7f75c85e0640 1 -- 172.21.15.112:0/1480106705 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f75ac0729d0 msgr2=0x7f75ac074e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:51.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.603+0000 7f75c85e0640 1 --2- 172.21.15.112:0/1480106705 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f75ac0729d0 0x7f75ac074e80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f75b000b020 tx=0x7f75b0035040 comp rx=0 tx=0).stop 2024-09-16T00:25:51.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.603+0000 7f75c85e0640 1 -- 172.21.15.112:0/1480106705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75c00fba20 msgr2=0x7f75c006dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:51.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.603+0000 7f75c85e0640 1 --2- 172.21.15.112:0/1480106705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75c00fba20 0x7f75c006dbd0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f75bc00eb70 tx=0x7f75bc00b040 comp rx=0 tx=0).stop 2024-09-16T00:25:51.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.603+0000 7f75c85e0640 1 -- 172.21.15.112:0/1480106705 shutdown_connections 2024-09-16T00:25:51.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.603+0000 7f75c85e0640 1 --2- 172.21.15.112:0/1480106705 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f75ac0729d0 0x7f75ac074e80 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:51.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.603+0000 7f75c85e0640 1 --2- 172.21.15.112:0/1480106705 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75c00fc370 0x7f75c006e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:51.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.603+0000 7f75c85e0640 1 --2- 172.21.15.112:0/1480106705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75c00fba20 0x7f75c006dbd0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:51.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.603+0000 7f75c85e0640 1 -- 172.21.15.112:0/1480106705 >> 172.21.15.112:0/1480106705 conn(0x7f75c00f6390 msgr2=0x7f75c00f7a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:51.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.603+0000 7f75c85e0640 1 -- 172.21.15.112:0/1480106705 shutdown_connections 2024-09-16T00:25:51.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.603+0000 7f75c85e0640 1 -- 172.21.15.112:0/1480106705 wait complete. 2024-09-16T00:25:51.608 INFO:tasks.workunit.client.0.smithi112.stdout:1/95: dwrite d0/d14/df/f19 [4194304,4194304] 0 2024-09-16T00:25:51.608 INFO:tasks.workunit.client.0.smithi112.stdout:8/91: creat d10/f1b x:0 0 0 2024-09-16T00:25:51.609 INFO:tasks.workunit.client.0.smithi112.stdout:9/238: creat d1/d5/d6/d38/f5a x:0 0 0 2024-09-16T00:25:51.610 INFO:tasks.workunit.client.0.smithi112.stdout:0/96: link d5/l9 d5/l1b 0 2024-09-16T00:25:51.614 INFO:tasks.workunit.client.0.smithi112.stdout:1/96: rename d0/c17 to d0/d14/d1c/c1f 0 2024-09-16T00:25:51.616 INFO:tasks.workunit.client.0.smithi112.stdout:0/97: dread d5/ff [0,4194304] 0 2024-09-16T00:25:51.616 INFO:tasks.workunit.client.0.smithi112.stdout:9/239: creat d1/d5/d2c/f5b x:0 0 0 2024-09-16T00:25:51.617 INFO:tasks.workunit.client.0.smithi112.stdout:0/98: read d5/da/f15 [2877549,26678] 0 2024-09-16T00:25:51.621 INFO:tasks.workunit.client.0.smithi112.stdout:9/240: mknod d1/d5/d6/d18/c5c 0 2024-09-16T00:25:51.622 INFO:tasks.workunit.client.0.smithi112.stdout:9/241: chown d1/d5/d6/d18/l37 460 1 2024-09-16T00:25:51.623 INFO:tasks.workunit.client.0.smithi112.stdout:0/99: mknod d5/da/d18/c1c 0 2024-09-16T00:25:51.624 INFO:tasks.workunit.client.0.smithi112.stdout:9/242: fdatasync d1/d5/d6/d18/f1f 0 2024-09-16T00:25:51.624 INFO:tasks.workunit.client.0.smithi112.stdout:9/243: readlink d1/d5/l12 0 2024-09-16T00:25:51.624 INFO:tasks.workunit.client.0.smithi112.stdout:9/244: read - d1/f50 zero size 2024-09-16T00:25:51.631 INFO:tasks.workunit.client.0.smithi112.stdout:0/100: dwrite d5/da/f15 [8388608,4194304] 0 2024-09-16T00:25:51.632 INFO:tasks.workunit.client.0.smithi112.stdout:0/101: write f0 [1863261,25277] 0 2024-09-16T00:25:51.634 INFO:tasks.workunit.client.0.smithi112.stdout:0/102: chown d5/l7 100844 1 2024-09-16T00:25:51.635 INFO:tasks.workunit.client.0.smithi112.stdout:0/103: write d5/da/dc/f13 [3769340,1688] 0 2024-09-16T00:25:51.637 INFO:tasks.workunit.client.0.smithi112.stdout:0/104: mkdir d5/da/d1d 0 2024-09-16T00:25:51.638 INFO:tasks.workunit.client.0.smithi112.stdout:0/105: chown f0 24 1 2024-09-16T00:25:51.639 INFO:tasks.workunit.client.0.smithi112.stdout:0/106: mkdir d5/da/d18/d1e 0 2024-09-16T00:25:51.640 INFO:tasks.workunit.client.0.smithi112.stdout:0/107: write d5/da/f15 [10984744,64804] 0 2024-09-16T00:25:51.643 INFO:tasks.workunit.client.0.smithi112.stdout:0/108: link d5/da/dc/fe d5/da/d18/f1f 0 2024-09-16T00:25:51.645 INFO:tasks.workunit.client.0.smithi112.stdout:0/109: mknod d5/da/d18/d1e/c20 0 2024-09-16T00:25:51.645 INFO:tasks.workunit.client.0.smithi112.stdout:0/110: readlink d5/l7 0 2024-09-16T00:25:51.647 INFO:tasks.workunit.client.0.smithi112.stdout:0/111: dread d5/ff [0,4194304] 0 2024-09-16T00:25:51.656 INFO:tasks.workunit.client.0.smithi112.stdout:0/112: dwrite d5/da/dc/fe [4194304,4194304] 0 2024-09-16T00:25:51.660 INFO:tasks.workunit.client.0.smithi112.stdout:0/113: mkdir d5/da/d21 0 2024-09-16T00:25:51.661 INFO:tasks.workunit.client.0.smithi112.stdout:0/114: chown d5 409 1 2024-09-16T00:25:51.661 INFO:tasks.workunit.client.0.smithi112.stdout:5/159: write d4/f16 [2848949,48380] 0 2024-09-16T00:25:51.662 INFO:tasks.workunit.client.0.smithi112.stdout:2/143: truncate d2/f19 2169778 0 2024-09-16T00:25:51.663 INFO:tasks.workunit.client.0.smithi112.stdout:0/115: mknod d5/da/d18/d1e/c22 0 2024-09-16T00:25:51.664 INFO:tasks.workunit.client.0.smithi112.stdout:2/144: write d2/f14 [902102,2971] 0 2024-09-16T00:25:51.665 INFO:tasks.workunit.client.0.smithi112.stdout:5/160: creat d4/d7/f35 x:0 0 0 2024-09-16T00:25:51.665 INFO:tasks.workunit.client.0.smithi112.stdout:2/145: chown d2/dc/l1e 36 1 2024-09-16T00:25:51.666 INFO:tasks.workunit.client.0.smithi112.stdout:0/116: chown d5/da/dc/d12/c1a 233 1 2024-09-16T00:25:51.666 INFO:tasks.workunit.client.0.smithi112.stdout:2/146: fsync d2/dd/f12 0 2024-09-16T00:25:51.666 INFO:tasks.workunit.client.0.smithi112.stdout:0/117: stat d5/l9 0 2024-09-16T00:25:51.669 INFO:tasks.workunit.client.0.smithi112.stdout:5/161: dread d4/f19 [0,4194304] 0 2024-09-16T00:25:51.669 INFO:tasks.workunit.client.0.smithi112.stdout:0/118: chown d5/da/d18/f1f 0 1 2024-09-16T00:25:51.671 INFO:tasks.workunit.client.0.smithi112.stdout:0/119: readlink d5/l1b 0 2024-09-16T00:25:51.672 INFO:tasks.workunit.client.0.smithi112.stdout:5/162: mkdir d4/d7/d26/d36 0 2024-09-16T00:25:51.674 INFO:tasks.workunit.client.0.smithi112.stdout:0/120: creat d5/f23 x:0 0 0 2024-09-16T00:25:51.675 INFO:tasks.workunit.client.0.smithi112.stdout:9/245: dwrite d1/d5/d6/d2d/d45/f21 [0,4194304] 0 2024-09-16T00:25:51.678 INFO:tasks.workunit.client.0.smithi112.stdout:5/163: chown d4/dd/f2a 46618 1 2024-09-16T00:25:51.678 INFO:tasks.workunit.client.0.smithi112.stdout:0/121: chown d5/da/dc/fe 0 1 2024-09-16T00:25:51.678 INFO:tasks.workunit.client.0.smithi112.stdout:2/147: dread d2/dd/f12 [0,4194304] 0 2024-09-16T00:25:51.680 INFO:tasks.workunit.client.0.smithi112.stdout:0/122: read d5/da/f15 [4517063,70885] 0 2024-09-16T00:25:51.680 INFO:tasks.workunit.client.0.smithi112.stdout:9/246: chown d1/d5/d6/d2d/f3b 120225585 1 2024-09-16T00:25:51.684 INFO:tasks.workunit.client.0.smithi112.stdout:2/148: mkdir d2/dc/d22/d2b 0 2024-09-16T00:25:51.685 INFO:tasks.workunit.client.0.smithi112.stdout:2/149: fsync d2/dd/f15 0 2024-09-16T00:25:51.685 INFO:tasks.workunit.client.0.smithi112.stdout:9/247: rename d1/c14 to d1/d5/d6/d2d/d39/c5d 0 2024-09-16T00:25:51.688 INFO:tasks.workunit.client.0.smithi112.stdout:5/164: dwrite d4/dd/d12/f11 [0,4194304] 0 2024-09-16T00:25:51.691 INFO:tasks.workunit.client.0.smithi112.stdout:2/150: mknod d2/c2c 0 2024-09-16T00:25:51.693 INFO:tasks.workunit.client.0.smithi112.stdout:9/248: symlink d1/d5/l5e 0 2024-09-16T00:25:51.697 INFO:tasks.workunit.client.0.smithi112.stdout:9/249: write d1/d5/d6/d2d/d39/f42 [1054607,17992] 0 2024-09-16T00:25:51.698 INFO:tasks.workunit.client.0.smithi112.stdout:0/123: truncate d5/da/d18/f1f 8336465 0 2024-09-16T00:25:51.700 INFO:tasks.workunit.client.0.smithi112.stdout:5/165: dwrite d4/d1c/f32 [0,4194304] 0 2024-09-16T00:25:51.703 INFO:tasks.workunit.client.0.smithi112.stdout:0/124: dread d5/da/dc/d12/f17 [0,4194304] 0 2024-09-16T00:25:51.703 INFO:tasks.workunit.client.0.smithi112.stdout:2/151: dwrite d2/dd/f15 [0,4194304] 0 2024-09-16T00:25:51.705 INFO:tasks.workunit.client.0.smithi112.stdout:5/166: creat d4/d7/d14/f37 x:0 0 0 2024-09-16T00:25:51.707 INFO:tasks.workunit.client.0.smithi112.stdout:2/152: creat d2/dc/f2d x:0 0 0 2024-09-16T00:25:51.709 INFO:tasks.workunit.client.0.smithi112.stdout:5/167: creat d4/d1c/f38 x:0 0 0 2024-09-16T00:25:51.710 INFO:tasks.workunit.client.0.smithi112.stdout:0/125: rename d5/da/d18/d1e to d5/da/d18/d24 0 2024-09-16T00:25:51.714 INFO:tasks.workunit.client.0.smithi112.stdout:2/153: truncate d2/f19 1753372 0 2024-09-16T00:25:51.715 INFO:tasks.workunit.client.0.smithi112.stdout:5/168: symlink d4/l39 0 2024-09-16T00:25:51.717 INFO:tasks.workunit.client.0.smithi112.stdout:2/154: mkdir d2/dc/d2e 0 2024-09-16T00:25:51.718 INFO:tasks.workunit.client.0.smithi112.stdout:5/169: write d4/d7/f35 [119881,65774] 0 2024-09-16T00:25:51.721 INFO:tasks.workunit.client.0.smithi112.stdout:2/155: creat d2/dc/f2f x:0 0 0 2024-09-16T00:25:51.722 INFO:tasks.workunit.client.0.smithi112.stdout:5/170: fsync d4/dd/d12/f2f 0 2024-09-16T00:25:51.722 INFO:tasks.workunit.client.0.smithi112.stdout:0/126: dwrite d5/da/dc/d12/f17 [0,4194304] 0 2024-09-16T00:25:51.724 INFO:tasks.workunit.client.0.smithi112.stdout:0/127: dread - d5/f23 zero size 2024-09-16T00:25:51.727 INFO:tasks.workunit.client.0.smithi112.stdout:2/156: dread d2/f8 [0,4194304] 0 2024-09-16T00:25:51.729 INFO:tasks.workunit.client.0.smithi112.stdout:5/171: read d4/d7/f35 [82563,66833] 0 2024-09-16T00:25:51.729 INFO:tasks.workunit.client.0.smithi112.stdout:2/157: write d2/fa [829519,92677] 0 2024-09-16T00:25:51.733 INFO:tasks.workunit.client.0.smithi112.stdout:5/172: creat d4/dd/d12/f3a x:0 0 0 2024-09-16T00:25:51.737 INFO:tasks.workunit.client.0.smithi112.stdout:5/173: creat d4/d7/d26/d36/f3b x:0 0 0 2024-09-16T00:25:51.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.738+0000 7f1b334b2640 1 -- 172.21.15.112:0/1805981614 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b2c0fb8a0 msgr2=0x7f1b2c0fbcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:51.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.738+0000 7f1b334b2640 1 --2- 172.21.15.112:0/1805981614 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b2c0fb8a0 0x7f1b2c0fbcb0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1b200048d0 tx=0x7f1b20009da0 comp rx=0 tx=0).stop 2024-09-16T00:25:51.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.738+0000 7f1b334b2640 1 -- 172.21.15.112:0/1805981614 shutdown_connections 2024-09-16T00:25:51.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.738+0000 7f1b334b2640 1 --2- 172.21.15.112:0/1805981614 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b2c0fc1f0 0x7f1b2c0fa620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:51.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.738+0000 7f1b334b2640 1 --2- 172.21.15.112:0/1805981614 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b2c0fb8a0 0x7f1b2c0fbcb0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:51.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.738+0000 7f1b334b2640 1 -- 172.21.15.112:0/1805981614 >> 172.21.15.112:0/1805981614 conn(0x7f1b2c0f6390 msgr2=0x7f1b2c0f87a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:51.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.738+0000 7f1b334b2640 1 -- 172.21.15.112:0/1805981614 shutdown_connections 2024-09-16T00:25:51.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.738+0000 7f1b334b2640 1 -- 172.21.15.112:0/1805981614 wait complete. 2024-09-16T00:25:51.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.739+0000 7f1b334b2640 1 Processor -- start 2024-09-16T00:25:51.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.739+0000 7f1b334b2640 1 -- start start 2024-09-16T00:25:51.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.739+0000 7f1b334b2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b2c0fb8a0 0x7f1b2c06dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:51.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.739+0000 7f1b334b2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b2c0fc1f0 0x7f1b2c06e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:51.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.739+0000 7f1b334b2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b2c06f7a0 con 0x7f1b2c0fb8a0 2024-09-16T00:25:51.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.739+0000 7f1b334b2640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b2c06e650 con 0x7f1b2c0fc1f0 2024-09-16T00:25:51.742 INFO:tasks.workunit.client.0.smithi112.stdout:0/128: link d5/da/d18/f1f d5/da/d21/f25 0 2024-09-16T00:25:51.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.740+0000 7f1b30a4d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b2c0fc1f0 0x7f1b2c06e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:51.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.740+0000 7f1b30a4d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b2c0fc1f0 0x7f1b2c06e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.112:58882/0 (socket says 172.21.15.112:58882) 2024-09-16T00:25:51.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.740+0000 7f1b3124e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b2c0fb8a0 0x7f1b2c06dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:51.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.740+0000 7f1b30a4d640 1 -- 172.21.15.112:0/1602670535 learned_addr learned my addr 172.21.15.112:0/1602670535 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:25:51.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.740+0000 7f1b30a4d640 1 -- 172.21.15.112:0/1602670535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b2c0fb8a0 msgr2=0x7f1b2c06dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:51.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.740+0000 7f1b30a4d640 1 --2- 172.21.15.112:0/1602670535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b2c0fb8a0 0x7f1b2c06dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:51.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.740+0000 7f1b30a4d640 1 -- 172.21.15.112:0/1602670535 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b20005040 con 0x7f1b2c0fc1f0 2024-09-16T00:25:51.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.740+0000 7f1b30a4d640 1 --2- 172.21.15.112:0/1602670535 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b2c0fc1f0 0x7f1b2c06e110 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1b28002a50 tx=0x7f1b28004600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:51.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.741+0000 7f1b1dffb640 1 -- 172.21.15.112:0/1602670535 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b28007c30 con 0x7f1b2c0fc1f0 2024-09-16T00:25:51.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.741+0000 7f1b1dffb640 1 -- 172.21.15.112:0/1602670535 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f1b2800f040 con 0x7f1b2c0fc1f0 2024-09-16T00:25:51.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.741+0000 7f1b1dffb640 1 -- 172.21.15.112:0/1602670535 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b2800dad0 con 0x7f1b2c0fc1f0 2024-09-16T00:25:51.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.741+0000 7f1b334b2640 1 -- 172.21.15.112:0/1602670535 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b2c06e900 con 0x7f1b2c0fc1f0 2024-09-16T00:25:51.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.741+0000 7f1b334b2640 1 -- 172.21.15.112:0/1602670535 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b2c121be0 con 0x7f1b2c0fc1f0 2024-09-16T00:25:51.744 INFO:tasks.workunit.client.0.smithi112.stdout:0/129: dread d5/ff [0,4194304] 0 2024-09-16T00:25:51.744 INFO:tasks.workunit.client.0.smithi112.stdout:0/130: chown d5/da/f15 5723 1 2024-09-16T00:25:51.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.742+0000 7f1b334b2640 1 -- 172.21.15.112:0/1602670535 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b2c0fa010 con 0x7f1b2c0fc1f0 2024-09-16T00:25:51.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.744+0000 7f1b1dffb640 1 -- 172.21.15.112:0/1602670535 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 20) v1 ==== 91584+0+0 (secure 0 0 0) 0x7f1b2800dc30 con 0x7f1b2c0fc1f0 2024-09-16T00:25:51.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.744+0000 7f1b1dffb640 1 --2- 172.21.15.112:0/1602670535 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f1b1806e5c0 0x7f1b18070a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:51.746 INFO:tasks.workunit.client.0.smithi112.stdout:5/174: link d4/d7/d26/l34 d4/d1c/l3c 0 2024-09-16T00:25:51.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.744+0000 7f1b3124e640 1 --2- 172.21.15.112:0/1602670535 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f1b1806e5c0 0x7f1b18070a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:51.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.745+0000 7f1b1dffb640 1 -- 172.21.15.112:0/1602670535 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f1b2808d960 con 0x7f1b2c0fc1f0 2024-09-16T00:25:51.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.745+0000 7f1b3124e640 1 --2- 172.21.15.112:0/1602670535 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f1b1806e5c0 0x7f1b18070a70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1b20000c00 tx=0x7f1b20035040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:51.747 INFO:tasks.workunit.client.0.smithi112.stdout:0/131: mknod d5/da/d18/c26 0 2024-09-16T00:25:51.748 INFO:tasks.workunit.client.0.smithi112.stdout:5/175: symlink d4/d1c/l3d 0 2024-09-16T00:25:51.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.748+0000 7f1b1dffb640 1 -- 172.21.15.112:0/1602670535 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f1b2805aab0 con 0x7f1b2c0fc1f0 2024-09-16T00:25:51.750 INFO:tasks.workunit.client.0.smithi112.stdout:5/176: mkdir d4/d7/d26/d3e 0 2024-09-16T00:25:51.750 INFO:tasks.workunit.client.0.smithi112.stdout:5/177: read d4/f19 [61640,20299] 0 2024-09-16T00:25:51.751 INFO:tasks.workunit.client.0.smithi112.stdout:5/178: fsync d4/d7/d14/f33 0 2024-09-16T00:25:51.753 INFO:tasks.workunit.client.0.smithi112.stdout:0/132: write d5/ff [338816,43495] 0 2024-09-16T00:25:51.753 INFO:tasks.workunit.client.0.smithi112.stdout:5/179: mknod d4/d7/d14/c3f 0 2024-09-16T00:25:51.755 INFO:tasks.workunit.client.0.smithi112.stdout:5/180: creat d4/d7/f40 x:0 0 0 2024-09-16T00:25:51.756 INFO:tasks.workunit.client.0.smithi112.stdout:5/181: chown d4/d7/d26 0 1 2024-09-16T00:25:51.758 INFO:tasks.workunit.client.0.smithi112.stdout:0/133: link d5/ff d5/da/f27 0 2024-09-16T00:25:51.759 INFO:tasks.workunit.client.0.smithi112.stdout:5/182: creat d4/d7/d26/d36/f41 x:0 0 0 2024-09-16T00:25:51.762 INFO:tasks.workunit.client.0.smithi112.stdout:0/134: dread d5/da/dc/d12/f17 [0,4194304] 0 2024-09-16T00:25:51.763 INFO:tasks.workunit.client.0.smithi112.stdout:0/135: read - d5/da/dc/d12/f14 zero size 2024-09-16T00:25:51.767 INFO:tasks.workunit.client.0.smithi112.stdout:0/136: creat d5/da/d21/f28 x:0 0 0 2024-09-16T00:25:51.769 INFO:tasks.workunit.client.0.smithi112.stdout:5/183: write d4/dd/f2a [85342,89680] 0 2024-09-16T00:25:51.772 INFO:tasks.workunit.client.0.smithi112.stdout:5/184: symlink d4/l42 0 2024-09-16T00:25:51.772 INFO:tasks.workunit.client.0.smithi112.stdout:0/137: dwrite f0 [0,4194304] 0 2024-09-16T00:25:51.774 INFO:tasks.workunit.client.0.smithi112.stdout:5/185: write d4/dd/d12/f11 [2206445,72777] 0 2024-09-16T00:25:51.776 INFO:tasks.workunit.client.0.smithi112.stdout:5/186: mkdir d4/d1c/d43 0 2024-09-16T00:25:51.777 INFO:tasks.workunit.client.0.smithi112.stdout:5/187: readlink d4/d1c/l3c 0 2024-09-16T00:25:51.779 INFO:tasks.workunit.client.0.smithi112.stdout:5/188: write d4/d7/d14/f37 [651570,50978] 0 2024-09-16T00:25:51.780 INFO:tasks.workunit.client.0.smithi112.stdout:0/138: rmdir d5 39 2024-09-16T00:25:51.782 INFO:tasks.workunit.client.0.smithi112.stdout:5/189: creat d4/d7/d26/f44 x:0 0 0 2024-09-16T00:25:51.783 INFO:tasks.workunit.client.0.smithi112.stdout:5/190: write f2 [1383847,64761] 0 2024-09-16T00:25:51.784 INFO:tasks.workunit.client.0.smithi112.stdout:5/191: dread - d4/dd/d12/f2f zero size 2024-09-16T00:25:51.792 INFO:tasks.workunit.client.0.smithi112.stdout:5/192: dwrite d4/d7/f40 [0,4194304] 0 2024-09-16T00:25:51.795 INFO:tasks.workunit.client.0.smithi112.stdout:0/139: dwrite d5/da/dc/f10 [0,4194304] 0 2024-09-16T00:25:51.797 INFO:tasks.workunit.client.0.smithi112.stdout:5/193: link d4/l42 d4/d7/d26/d3e/l45 0 2024-09-16T00:25:51.800 INFO:tasks.workunit.client.0.smithi112.stdout:5/194: creat d4/d7/d26/f46 x:0 0 0 2024-09-16T00:25:51.803 INFO:tasks.workunit.client.0.smithi112.stdout:5/195: unlink d4/dd/d12/c30 0 2024-09-16T00:25:51.803 INFO:tasks.workunit.client.0.smithi112.stdout:5/196: chown d4/dd/d12/f11 225719311 1 2024-09-16T00:25:51.806 INFO:tasks.workunit.client.0.smithi112.stdout:5/197: dread d4/d7/d14/f1a [4194304,4194304] 0 2024-09-16T00:25:51.806 INFO:tasks.workunit.client.0.smithi112.stdout:5/198: truncate f2 2150890 0 2024-09-16T00:25:51.810 INFO:tasks.workunit.client.0.smithi112.stdout:0/140: write d5/da/d21/f28 [2349,73811] 0 2024-09-16T00:25:51.814 INFO:tasks.workunit.client.0.smithi112.stdout:5/199: dwrite d4/d7/d26/d36/f3b [0,4194304] 0 2024-09-16T00:25:51.817 INFO:tasks.workunit.client.0.smithi112.stdout:5/200: rename d4/l1b to d4/d7/d26/d3e/l47 0 2024-09-16T00:25:51.820 INFO:tasks.workunit.client.0.smithi112.stdout:5/201: creat d4/dd/d12/f48 x:0 0 0 2024-09-16T00:25:51.821 INFO:tasks.workunit.client.0.smithi112.stdout:0/141: dwrite d5/da/f15 [0,4194304] 0 2024-09-16T00:25:51.846 INFO:tasks.workunit.client.0.smithi112.stdout:5/202: mkdir d4/dd/d49 0 2024-09-16T00:25:51.847 INFO:tasks.workunit.client.0.smithi112.stdout:0/142: rmdir d5/da/dc/d19 0 2024-09-16T00:25:51.848 INFO:tasks.workunit.client.0.smithi112.stdout:5/203: mknod d4/d1c/d43/c4a 0 2024-09-16T00:25:51.851 INFO:tasks.workunit.client.0.smithi112.stdout:5/204: getdents d4/d1c 0 2024-09-16T00:25:51.852 INFO:tasks.workunit.client.0.smithi112.stdout:5/205: read d4/d7/d14/f33 [195632,63333] 0 2024-09-16T00:25:51.853 INFO:tasks.workunit.client.0.smithi112.stdout:0/143: getdents d5/da/dc 0 2024-09-16T00:25:51.854 INFO:tasks.workunit.client.0.smithi112.stdout:0/144: fsync d5/da/d21/f28 0 2024-09-16T00:25:51.855 INFO:tasks.workunit.client.0.smithi112.stdout:5/206: mknod d4/d7/d26/d36/c4b 0 2024-09-16T00:25:51.856 INFO:tasks.workunit.client.0.smithi112.stdout:0/145: mknod d5/da/dc/c29 0 2024-09-16T00:25:51.857 INFO:tasks.workunit.client.0.smithi112.stdout:5/207: mkdir d4/d7/d26/d4c 0 2024-09-16T00:25:51.857 INFO:tasks.workunit.client.0.smithi112.stdout:0/146: creat d5/f2a x:0 0 0 2024-09-16T00:25:51.858 INFO:tasks.workunit.client.0.smithi112.stdout:5/208: creat d4/d7/f4d x:0 0 0 2024-09-16T00:25:51.858 INFO:tasks.workunit.client.0.smithi112.stdout:0/147: rmdir d5/da 39 2024-09-16T00:25:51.860 INFO:tasks.workunit.client.0.smithi112.stdout:5/209: creat d4/dd/f4e x:0 0 0 2024-09-16T00:25:51.861 INFO:tasks.workunit.client.0.smithi112.stdout:0/148: creat d5/da/f2b x:0 0 0 2024-09-16T00:25:51.863 INFO:tasks.workunit.client.0.smithi112.stdout:0/149: creat d5/f2c x:0 0 0 2024-09-16T00:25:51.866 INFO:tasks.workunit.client.0.smithi112.stdout:0/150: dread d5/da/dc/f10 [0,4194304] 0 2024-09-16T00:25:51.867 INFO:tasks.workunit.client.0.smithi112.stdout:0/151: chown d5/da/d1d 25717 1 2024-09-16T00:25:51.868 INFO:tasks.workunit.client.0.smithi112.stdout:0/152: write d5/da/f2b [639895,102081] 0 2024-09-16T00:25:51.916 INFO:tasks.workunit.client.0.smithi112.stdout:7/102: sync 2024-09-16T00:25:51.917 INFO:tasks.workunit.client.0.smithi112.stdout:6/123: sync 2024-09-16T00:25:51.917 INFO:tasks.workunit.client.0.smithi112.stdout:3/122: sync 2024-09-16T00:25:51.917 INFO:tasks.workunit.client.0.smithi112.stdout:4/89: sync 2024-09-16T00:25:51.917 INFO:tasks.workunit.client.0.smithi112.stdout:4/90: dread - d6/db/d15/f17 zero size 2024-09-16T00:25:51.932 INFO:tasks.workunit.client.0.smithi112.stdout:0/153: mknod d5/da/d18/c2d 0 2024-09-16T00:25:51.933 INFO:tasks.workunit.client.0.smithi112.stdout:7/103: symlink d6/d26/l28 0 2024-09-16T00:25:51.935 INFO:tasks.workunit.client.0.smithi112.stdout:4/91: creat d6/db/d15/f22 x:0 0 0 2024-09-16T00:25:51.936 INFO:tasks.workunit.client.0.smithi112.stdout:6/124: mknod dd/df/c2c 0 2024-09-16T00:25:51.937 INFO:tasks.workunit.client.0.smithi112.stdout:3/123: mkdir d2/d9/d2b/d2f 0 2024-09-16T00:25:51.939 INFO:tasks.workunit.client.0.smithi112.stdout:0/154: dread d5/da/f15 [8388608,4194304] 0 2024-09-16T00:25:51.939 INFO:tasks.workunit.client.0.smithi112.stdout:6/125: creat dd/d23/f2d x:0 0 0 2024-09-16T00:25:51.941 INFO:tasks.workunit.client.0.smithi112.stdout:4/92: rename d6/db/df/c11 to d6/c23 0 2024-09-16T00:25:51.941 INFO:tasks.workunit.client.0.smithi112.stdout:0/155: chown d5/da/d18 1823 1 2024-09-16T00:25:51.942 INFO:tasks.workunit.client.0.smithi112.stdout:6/126: write dd/d15/f29 [920100,70963] 0 2024-09-16T00:25:51.943 INFO:tasks.workunit.client.0.smithi112.stdout:7/104: dwrite d6/d26/f27 [0,4194304] 0 2024-09-16T00:25:51.944 INFO:tasks.workunit.client.0.smithi112.stdout:4/93: write d6/db/d15/f17 [594801,49883] 0 2024-09-16T00:25:51.945 INFO:tasks.workunit.client.0.smithi112.stdout:3/124: rename d2/d9/fe to d2/d9/d2b/d2f/f30 0 2024-09-16T00:25:51.945 INFO:tasks.workunit.client.0.smithi112.stdout:7/105: chown l3 1 1 2024-09-16T00:25:51.947 INFO:tasks.workunit.client.0.smithi112.stdout:4/94: chown l5 1 1 2024-09-16T00:25:51.949 INFO:tasks.workunit.client.0.smithi112.stdout:6/127: rename l9 to dd/d23/l2e 0 2024-09-16T00:25:51.949 INFO:tasks.workunit.client.0.smithi112.stdout:0/156: dread d5/da/dc/f10 [0,4194304] 0 2024-09-16T00:25:51.950 INFO:tasks.workunit.client.0.smithi112.stdout:7/106: mknod d6/d26/c29 0 2024-09-16T00:25:51.951 INFO:tasks.workunit.client.0.smithi112.stdout:6/128: rename dd/df to dd/df/d17/d27/d2f 22 2024-09-16T00:25:51.952 INFO:tasks.workunit.client.0.smithi112.stdout:4/95: mknod d6/da/dc/c24 0 2024-09-16T00:25:51.952 INFO:tasks.workunit.client.0.smithi112.stdout:4/96: stat d6/db 0 2024-09-16T00:25:51.954 INFO:tasks.workunit.client.0.smithi112.stdout:7/107: creat d6/f2a x:0 0 0 2024-09-16T00:25:51.956 INFO:tasks.workunit.client.0.smithi112.stdout:6/129: rmdir dd/df/d10 39 2024-09-16T00:25:51.958 INFO:tasks.workunit.client.0.smithi112.stdout:4/97: fsync d6/f1b 0 2024-09-16T00:25:51.959 INFO:tasks.workunit.client.0.smithi112.stdout:7/108: symlink d6/d10/d16/l2b 0 2024-09-16T00:25:51.959 INFO:tasks.workunit.client.0.smithi112.stdout:3/125: link d2/d9/c18 d2/c31 0 2024-09-16T00:25:51.960 INFO:tasks.workunit.client.0.smithi112.stdout:0/157: creat d5/da/f2e x:0 0 0 2024-09-16T00:25:51.961 INFO:tasks.workunit.client.0.smithi112.stdout:4/98: creat d6/db/f25 x:0 0 0 2024-09-16T00:25:51.962 INFO:tasks.workunit.client.0.smithi112.stdout:4/99: rename d6/db/d15 to d6/db/d15/d26 22 2024-09-16T00:25:51.962 INFO:tasks.workunit.client.0.smithi112.stdout:3/126: write d2/d9/d2b/d2f/f30 [1026261,18887] 0 2024-09-16T00:25:51.964 INFO:tasks.workunit.client.0.smithi112.stdout:0/158: dread f0 [0,4194304] 0 2024-09-16T00:25:51.965 INFO:tasks.workunit.client.0.smithi112.stdout:0/159: read - d5/f2a zero size 2024-09-16T00:25:51.966 INFO:tasks.workunit.client.0.smithi112.stdout:0/160: chown d5/da/d1d 22759664 1 2024-09-16T00:25:51.966 INFO:tasks.workunit.client.0.smithi112.stdout:4/100: getdents d6/da 0 2024-09-16T00:25:51.968 INFO:tasks.workunit.client.0.smithi112.stdout:4/101: dread - d6/db/f25 zero size 2024-09-16T00:25:51.968 INFO:tasks.workunit.client.0.smithi112.stdout:0/161: mknod d5/c2f 0 2024-09-16T00:25:51.968 INFO:tasks.workunit.client.0.smithi112.stdout:0/162: readlink d5/l9 0 2024-09-16T00:25:51.969 INFO:tasks.workunit.client.0.smithi112.stdout:0/163: chown d5/da/dc/f13 10122082 1 2024-09-16T00:25:51.969 INFO:tasks.workunit.client.0.smithi112.stdout:4/102: creat d6/da/dc/d19/d1c/f27 x:0 0 0 2024-09-16T00:25:51.971 INFO:tasks.workunit.client.0.smithi112.stdout:4/103: write d6/da/dc/d19/d1c/f27 [185494,130743] 0 2024-09-16T00:25:51.972 INFO:tasks.workunit.client.0.smithi112.stdout:0/164: truncate d5/da/f27 456088 0 2024-09-16T00:25:51.972 INFO:tasks.workunit.client.0.smithi112.stdout:4/104: write d6/db/d15/f22 [509127,113872] 0 2024-09-16T00:25:51.974 INFO:tasks.workunit.client.0.smithi112.stdout:4/105: rename d6/da/dc to d6/da/dc/d19/d1c/d28 22 2024-09-16T00:25:51.974 INFO:tasks.workunit.client.0.smithi112.stdout:0/165: chown d5/da/d18/f1f 2 1 2024-09-16T00:25:51.975 INFO:tasks.workunit.client.0.smithi112.stdout:4/106: write d6/db/f25 [449497,84143] 0 2024-09-16T00:25:51.975 INFO:tasks.workunit.client.0.smithi112.stdout:0/166: mknod d5/da/d18/d24/c30 0 2024-09-16T00:25:51.977 INFO:tasks.workunit.client.0.smithi112.stdout:4/107: fsync d6/db/f18 0 2024-09-16T00:25:51.980 INFO:tasks.workunit.client.0.smithi112.stdout:0/167: dread d5/da/dc/f13 [0,4194304] 0 2024-09-16T00:25:51.980 INFO:tasks.workunit.client.0.smithi112.stdout:0/168: readlink d5/l1b 0 2024-09-16T00:25:51.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.980+0000 7f1b334b2640 1 -- 172.21.15.112:0/1602670535 --> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1b2c0ffa60 con 0x7f1b1806e5c0 2024-09-16T00:25:51.982 INFO:tasks.workunit.client.0.smithi112.stdout:0/169: write f0 [1581382,74643] 0 2024-09-16T00:25:51.983 INFO:tasks.workunit.client.0.smithi112.stdout:4/108: dwrite d6/f9 [0,4194304] 0 2024-09-16T00:25:51.987 INFO:tasks.workunit.client.0.smithi112.stdout:4/109: creat d6/db/d13/f29 x:0 0 0 2024-09-16T00:25:51.993 INFO:tasks.workunit.client.0.smithi112.stdout:0/170: dwrite d5/f23 [0,4194304] 0 2024-09-16T00:25:51.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.994+0000 7f1b1dffb640 1 -- 172.21.15.112:0/1602670535 <== mgr.14208 v2:172.21.15.112:6800/2480414866 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7f1b2c0ffa60 con 0x7f1b1806e5c0 2024-09-16T00:25:51.995 INFO:teuthology.orchestra.run.smithi112.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T00:25:51.995 INFO:teuthology.orchestra.run.smithi112.stdout:alertmanager.smithi112 smithi112 *:9093,9094 running (6m) 119s ago 7m 24.2M - 0.25.0 c8568f914cd2 89acc17eab5c 2024-09-16T00:25:51.996 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi112 smithi112 running (7m) 119s ago 7m 6891k - 17.2.7-1633-g6d025bc2 33ed8dba3996 c94220ed2776 2024-09-16T00:25:51.996 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi169 smithi169 running (7m) 2m ago 7m 6899k - 17.2.7-1633-g6d025bc2 33ed8dba3996 d0d3726280a2 2024-09-16T00:25:51.996 INFO:teuthology.orchestra.run.smithi112.stdout:grafana.smithi112 smithi112 *:3000 running (6m) 119s ago 7m 84.6M - 9.4.7 954c08fa6188 ffeb9c3bdf66 2024-09-16T00:25:51.996 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.hxfwkh smithi112 running (2m) 119s ago 2m 15.6M - 17.2.7-1633-g6d025bc2 33ed8dba3996 cf92d9461fc6 2024-09-16T00:25:51.996 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.ithwzj smithi112 running (2m) 119s ago 2m 11.8M - 17.2.7-1633-g6d025bc2 33ed8dba3996 02786c8a83c7 2024-09-16T00:25:51.996 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.gawwgw smithi169 running (2m) 2m ago 2m 14.4M - 17.2.7-1633-g6d025bc2 33ed8dba3996 dcf3a4dfa879 2024-09-16T00:25:51.996 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.lgeqrj smithi169 running (2m) 2m ago 2m 13.8M - 17.2.7-1633-g6d025bc2 33ed8dba3996 445c14e55831 2024-09-16T00:25:51.996 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi112.xrkkbc smithi112 *:9283,8765,8443 running (9m) 119s ago 9m 449M - 17.2.7-1633-g6d025bc2 33ed8dba3996 b3d3adb94a85 2024-09-16T00:25:51.996 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi169.mtrobk smithi169 *:8443,9283 starting - - - - 2024-09-16T00:25:51.996 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi112 smithi112 running (9m) 119s ago 9m 43.3M 2048M 17.2.7-1633-g6d025bc2 33ed8dba3996 ddaaced54bf8 2024-09-16T00:25:51.996 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi169 smithi169 running (6m) 2m ago 6m 35.4M 2048M 17.2.7-1633-g6d025bc2 33ed8dba3996 6c2289bc3e11 2024-09-16T00:25:51.996 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi112 smithi112 *:9100 running (7m) 119s ago 7m 15.9M - 1.5.0 0da6a335fe13 89c6f32288bc 2024-09-16T00:25:51.996 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi169 smithi169 *:9100 running (6m) 2m ago 6m 15.6M - 1.5.0 0da6a335fe13 5d6fe1dc6fa9 2024-09-16T00:25:51.996 INFO:teuthology.orchestra.run.smithi112.stdout:osd.0 smithi112 running (6m) 119s ago 6m 46.0M 4288M 17.2.7-1633-g6d025bc2 33ed8dba3996 f6b2e146ffdf 2024-09-16T00:25:51.997 INFO:teuthology.orchestra.run.smithi112.stdout:osd.1 smithi112 running (5m) 119s ago 5m 47.9M 4288M 17.2.7-1633-g6d025bc2 33ed8dba3996 e9bee87bba0f 2024-09-16T00:25:51.997 INFO:teuthology.orchestra.run.smithi112.stdout:osd.2 smithi112 running (5m) 119s ago 5m 43.3M 4288M 17.2.7-1633-g6d025bc2 33ed8dba3996 100ec4e4852a 2024-09-16T00:25:51.997 INFO:teuthology.orchestra.run.smithi112.stdout:osd.3 smithi169 running (4m) 2m ago 4m 46.3M 5312M 17.2.7-1633-g6d025bc2 33ed8dba3996 f88463e19f23 2024-09-16T00:25:51.997 INFO:teuthology.orchestra.run.smithi112.stdout:osd.4 smithi169 running (4m) 2m ago 4m 42.7M 5312M 17.2.7-1633-g6d025bc2 33ed8dba3996 d3e4cdb016b9 2024-09-16T00:25:51.997 INFO:teuthology.orchestra.run.smithi112.stdout:osd.5 smithi169 running (3m) 2m ago 3m 42.2M 5312M 17.2.7-1633-g6d025bc2 33ed8dba3996 8e7d708d74eb 2024-09-16T00:25:51.997 INFO:teuthology.orchestra.run.smithi112.stdout:prometheus.smithi112 smithi112 *:9095 running (6m) 119s ago 7m 55.1M - 2.43.0 a07b618ecd1d 7f5a86d91011 2024-09-16T00:25:51.997 INFO:tasks.workunit.client.0.smithi112.stdout:4/110: dwrite d6/f1b [0,4194304] 0 2024-09-16T00:25:51.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.997+0000 7f1b334b2640 1 -- 172.21.15.112:0/1602670535 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f1b1806e5c0 msgr2=0x7f1b18070a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:51.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.997+0000 7f1b334b2640 1 --2- 172.21.15.112:0/1602670535 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f1b1806e5c0 0x7f1b18070a70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1b20000c00 tx=0x7f1b20035040 comp rx=0 tx=0).stop 2024-09-16T00:25:51.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.997+0000 7f1b334b2640 1 -- 172.21.15.112:0/1602670535 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b2c0fc1f0 msgr2=0x7f1b2c06e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:51.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.997+0000 7f1b334b2640 1 --2- 172.21.15.112:0/1602670535 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b2c0fc1f0 0x7f1b2c06e110 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1b28002a50 tx=0x7f1b28004600 comp rx=0 tx=0).stop 2024-09-16T00:25:51.999 INFO:tasks.workunit.client.0.smithi112.stdout:4/111: write f4 [1479174,47995] 0 2024-09-16T00:25:51.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.997+0000 7f1b334b2640 1 -- 172.21.15.112:0/1602670535 shutdown_connections 2024-09-16T00:25:51.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.997+0000 7f1b334b2640 1 --2- 172.21.15.112:0/1602670535 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7f1b1806e5c0 0x7f1b18070a70 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:51.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.997+0000 7f1b334b2640 1 --2- 172.21.15.112:0/1602670535 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b2c0fc1f0 0x7f1b2c06e110 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:51.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.997+0000 7f1b334b2640 1 --2- 172.21.15.112:0/1602670535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b2c0fb8a0 0x7f1b2c06dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:51.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.997+0000 7f1b334b2640 1 -- 172.21.15.112:0/1602670535 >> 172.21.15.112:0/1602670535 conn(0x7f1b2c0f6390 msgr2=0x7f1b2c0f7ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:52.000 INFO:tasks.workunit.client.0.smithi112.stdout:0/171: rename d5/da/d18/d24/c20 to d5/da/d18/c31 0 2024-09-16T00:25:52.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.998+0000 7f1b334b2640 1 -- 172.21.15.112:0/1602670535 shutdown_connections 2024-09-16T00:25:52.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:51.998+0000 7f1b334b2640 1 -- 172.21.15.112:0/1602670535 wait complete. 2024-09-16T00:25:52.001 INFO:tasks.workunit.client.0.smithi112.stdout:4/112: symlink d6/db/l2a 0 2024-09-16T00:25:52.003 INFO:tasks.workunit.client.0.smithi112.stdout:0/172: chown d5/ff 799539106 1 2024-09-16T00:25:52.004 INFO:tasks.workunit.client.0.smithi112.stdout:0/173: chown d5/da/dc/c29 0 1 2024-09-16T00:25:52.025 INFO:tasks.workunit.client.0.smithi112.stdout:3/127: getdents d2/d9 0 2024-09-16T00:25:52.028 INFO:tasks.workunit.client.0.smithi112.stdout:6/130: rename dd/d23/l2e to dd/d23/l30 0 2024-09-16T00:25:52.029 INFO:tasks.workunit.client.0.smithi112.stdout:3/128: mkdir d2/d9/d2b/d2c/d32 0 2024-09-16T00:25:52.031 INFO:tasks.workunit.client.0.smithi112.stdout:3/129: unlink d2/d9/f14 0 2024-09-16T00:25:52.031 INFO:tasks.workunit.client.0.smithi112.stdout:6/131: symlink dd/df/d10/l31 0 2024-09-16T00:25:52.036 INFO:tasks.workunit.client.0.smithi112.stdout:6/132: rename dd/df/c2c to dd/df/d11/c32 0 2024-09-16T00:25:52.037 INFO:tasks.workunit.client.0.smithi112.stdout:6/133: fsync dd/df/d10/d1d/f26 0 2024-09-16T00:25:52.037 INFO:tasks.workunit.client.0.smithi112.stdout:7/109: write d6/f9 [915786,55944] 0 2024-09-16T00:25:52.037 INFO:tasks.workunit.client.0.smithi112.stdout:7/110: readlink l3 0 2024-09-16T00:25:52.038 INFO:tasks.workunit.client.0.smithi112.stdout:6/134: stat dd/df/d10/f1b 0 2024-09-16T00:25:52.038 INFO:tasks.workunit.client.0.smithi112.stdout:7/111: rename d6 to d6/d10/d2c 22 2024-09-16T00:25:52.039 INFO:tasks.workunit.client.0.smithi112.stdout:7/112: dread - d6/f2a zero size 2024-09-16T00:25:52.041 INFO:tasks.workunit.client.0.smithi112.stdout:4/113: fsync d6/db/d13/f29 0 2024-09-16T00:25:52.042 INFO:tasks.workunit.client.0.smithi112.stdout:7/113: write d6/d10/d16/f20 [688324,17663] 0 2024-09-16T00:25:52.045 INFO:tasks.workunit.client.0.smithi112.stdout:4/114: chown d6/db/d15/c21 1637267 1 2024-09-16T00:25:52.047 INFO:tasks.workunit.client.0.smithi112.stdout:8/92: sync 2024-09-16T00:25:52.048 INFO:tasks.workunit.client.0.smithi112.stdout:6/135: dwrite dd/df/f21 [0,4194304] 0 2024-09-16T00:25:52.049 INFO:tasks.workunit.client.0.smithi112.stdout:7/114: link d6/d10/d16/f25 d6/d26/f2d 0 2024-09-16T00:25:52.050 INFO:tasks.workunit.client.0.smithi112.stdout:8/93: readlink d10/l19 0 2024-09-16T00:25:52.051 INFO:tasks.workunit.client.0.smithi112.stdout:8/94: stat c2 0 2024-09-16T00:25:52.051 INFO:tasks.workunit.client.0.smithi112.stdout:7/115: dread - d6/d10/d16/f1e zero size 2024-09-16T00:25:52.052 INFO:tasks.workunit.client.0.smithi112.stdout:6/136: rmdir dd/df/d13/d18 39 2024-09-16T00:25:52.057 INFO:tasks.workunit.client.0.smithi112.stdout:7/116: write d6/f11 [709452,68809] 0 2024-09-16T00:25:52.057 INFO:tasks.workunit.client.0.smithi112.stdout:0/174: write d5/da/f27 [627737,84660] 0 2024-09-16T00:25:52.061 INFO:tasks.workunit.client.0.smithi112.stdout:4/115: dwrite d6/db/d13/f29 [0,4194304] 0 2024-09-16T00:25:52.064 INFO:tasks.workunit.client.0.smithi112.stdout:7/117: dread - d6/d10/d1b/f24 zero size 2024-09-16T00:25:52.066 INFO:tasks.workunit.client.0.smithi112.stdout:8/95: dread d10/d12/f16 [0,4194304] 0 2024-09-16T00:25:52.067 INFO:tasks.workunit.client.0.smithi112.stdout:0/175: mkdir d5/da/d18/d24/d32 0 2024-09-16T00:25:52.068 INFO:tasks.workunit.client.0.smithi112.stdout:3/130: dwrite d2/d9/f11 [0,4194304] 0 2024-09-16T00:25:52.069 INFO:tasks.workunit.client.0.smithi112.stdout:8/96: dread - d10/f1b zero size 2024-09-16T00:25:52.069 INFO:tasks.workunit.client.0.smithi112.stdout:4/116: creat d6/da/dc/d19/d1c/f2b x:0 0 0 2024-09-16T00:25:52.070 INFO:tasks.workunit.client.0.smithi112.stdout:7/118: symlink d6/d10/d16/l2e 0 2024-09-16T00:25:52.073 INFO:tasks.workunit.client.0.smithi112.stdout:8/97: mkdir d10/d12/d1c 0 2024-09-16T00:25:52.074 INFO:tasks.workunit.client.0.smithi112.stdout:7/119: symlink d6/d10/d16/l2f 0 2024-09-16T00:25:52.075 INFO:tasks.workunit.client.0.smithi112.stdout:8/98: write d10/f15 [470780,91334] 0 2024-09-16T00:25:52.076 INFO:tasks.workunit.client.0.smithi112.stdout:7/120: write f4 [5045941,3369] 0 2024-09-16T00:25:52.077 INFO:tasks.workunit.client.0.smithi112.stdout:4/117: rename d6/db/d13 to d6/da/dc/d19/d1c/d2c 0 2024-09-16T00:25:52.080 INFO:tasks.workunit.client.0.smithi112.stdout:7/121: mkdir d6/d10/d1b/d30 0 2024-09-16T00:25:52.080 INFO:tasks.workunit.client.0.smithi112.stdout:0/176: rmdir d5/da 39 2024-09-16T00:25:52.082 INFO:tasks.workunit.client.0.smithi112.stdout:4/118: mknod d6/db/df/c2d 0 2024-09-16T00:25:52.082 INFO:tasks.workunit.client.0.smithi112.stdout:7/122: rename d6/d10 to d6/d10/d16/d31 22 2024-09-16T00:25:52.084 INFO:tasks.workunit.client.0.smithi112.stdout:7/123: chown d6/d10/f1d 277 1 2024-09-16T00:25:52.087 INFO:tasks.workunit.client.0.smithi112.stdout:4/119: unlink d6/db/df/l12 0 2024-09-16T00:25:52.087 INFO:tasks.workunit.client.0.smithi112.stdout:3/131: dwrite d2/f1a [4194304,4194304] 0 2024-09-16T00:25:52.088 INFO:tasks.workunit.client.0.smithi112.stdout:7/124: creat d6/d10/f32 x:0 0 0 2024-09-16T00:25:52.091 INFO:tasks.workunit.client.0.smithi112.stdout:3/132: write d2/f2d [322631,122614] 0 2024-09-16T00:25:52.097 INFO:tasks.workunit.client.0.smithi112.stdout:8/99: truncate d10/d12/f16 2245036 0 2024-09-16T00:25:52.097 INFO:tasks.workunit.client.0.smithi112.stdout:0/177: dwrite d5/da/f2e [0,4194304] 0 2024-09-16T00:25:52.098 INFO:tasks.workunit.client.0.smithi112.stdout:7/125: creat d6/d10/d1b/d30/f33 x:0 0 0 2024-09-16T00:25:52.101 INFO:tasks.workunit.client.0.smithi112.stdout:7/126: dread - d6/d10/d1b/f24 zero size 2024-09-16T00:25:52.101 INFO:tasks.workunit.client.0.smithi112.stdout:3/133: rmdir d2/d9/d2b 39 2024-09-16T00:25:52.102 INFO:tasks.workunit.client.0.smithi112.stdout:0/178: getdents d5/da/d1d 0 2024-09-16T00:25:52.104 INFO:tasks.workunit.client.0.smithi112.stdout:3/134: write d2/d9/f21 [279757,5090] 0 2024-09-16T00:25:52.106 INFO:tasks.workunit.client.0.smithi112.stdout:8/100: getdents d10/d12 0 2024-09-16T00:25:52.106 INFO:tasks.workunit.client.0.smithi112.stdout:4/120: rmdir d6 39 2024-09-16T00:25:52.108 INFO:tasks.workunit.client.0.smithi112.stdout:0/179: chown d5/da/d18/c31 1 1 2024-09-16T00:25:52.108 INFO:tasks.workunit.client.0.smithi112.stdout:0/180: fsync d5/f2a 0 2024-09-16T00:25:52.110 INFO:tasks.workunit.client.0.smithi112.stdout:7/127: link d6/d26/f27 d6/d10/d16/f34 0 2024-09-16T00:25:52.118 INFO:tasks.workunit.client.0.smithi112.stdout:7/128: rename d6/f7 to d6/d26/f35 0 2024-09-16T00:25:52.119 INFO:tasks.workunit.client.0.smithi112.stdout:4/121: dwrite d6/f9 [4194304,4194304] 0 2024-09-16T00:25:52.120 INFO:tasks.workunit.client.0.smithi112.stdout:8/101: creat d10/d12/d1c/f1d x:0 0 0 2024-09-16T00:25:52.122 INFO:tasks.workunit.client.0.smithi112.stdout:3/135: mkdir d2/d9/d2b/d33 0 2024-09-16T00:25:52.124 INFO:tasks.workunit.client.0.smithi112.stdout:3/136: chown d2/d9/f21 259826899 1 2024-09-16T00:25:52.128 INFO:tasks.workunit.client.0.smithi112.stdout:3/137: mknod d2/d9/d2b/d2f/c34 0 2024-09-16T00:25:52.129 INFO:tasks.workunit.client.0.smithi112.stdout:0/181: link d5/da/dc/fe d5/f33 0 2024-09-16T00:25:52.131 INFO:tasks.workunit.client.0.smithi112.stdout:8/102: rmdir d10/d12/d1c 39 2024-09-16T00:25:52.132 INFO:tasks.workunit.client.0.smithi112.stdout:0/182: write d5/da/d21/f28 [778649,113050] 0 2024-09-16T00:25:52.132 INFO:tasks.workunit.client.0.smithi112.stdout:7/129: write d6/d26/f2d [1524907,101388] 0 2024-09-16T00:25:52.132 INFO:tasks.workunit.client.0.smithi112.stdout:4/122: dwrite d6/db/f25 [0,4194304] 0 2024-09-16T00:25:52.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.132+0000 7ff71600e640 1 -- 172.21.15.112:0/924533515 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7100715d0 msgr2=0x7ff71006fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:52.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.132+0000 7ff71600e640 1 --2- 172.21.15.112:0/924533515 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7100715d0 0x7ff71006fb00 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff7080048d0 tx=0x7ff708009da0 comp rx=0 tx=0).stop 2024-09-16T00:25:52.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.132+0000 7ff71600e640 1 -- 172.21.15.112:0/924533515 shutdown_connections 2024-09-16T00:25:52.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.132+0000 7ff71600e640 1 --2- 172.21.15.112:0/924533515 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff710070040 0x7ff7100704b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:52.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.132+0000 7ff71600e640 1 --2- 172.21.15.112:0/924533515 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7100715d0 0x7ff71006fb00 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:52.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.132+0000 7ff71600e640 1 -- 172.21.15.112:0/924533515 >> 172.21.15.112:0/924533515 conn(0x7ff7100f7d10 msgr2=0x7ff7100fa120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:52.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.132+0000 7ff71600e640 1 -- 172.21.15.112:0/924533515 shutdown_connections 2024-09-16T00:25:52.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.133+0000 7ff71600e640 1 -- 172.21.15.112:0/924533515 wait complete. 2024-09-16T00:25:52.135 INFO:tasks.workunit.client.0.smithi112.stdout:4/123: read d6/da/dc/d19/d1c/d2c/f29 [328721,96675] 0 2024-09-16T00:25:52.136 INFO:tasks.workunit.client.0.smithi112.stdout:3/138: chown d2/c22 621597177 1 2024-09-16T00:25:52.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.134+0000 7ff71600e640 1 Processor -- start 2024-09-16T00:25:52.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.135+0000 7ff71600e640 1 -- start start 2024-09-16T00:25:52.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.135+0000 7ff71600e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff710070040 0x7ff7101136a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:52.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.135+0000 7ff71600e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7100715d0 0x7ff710113be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:52.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.135+0000 7ff71600e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff710114230 con 0x7ff710070040 2024-09-16T00:25:52.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.135+0000 7ff71600e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff710114a50 con 0x7ff7100715d0 2024-09-16T00:25:52.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.135+0000 7ff70f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff710070040 0x7ff7101136a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:52.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.135+0000 7ff70f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff710070040 0x7ff7101136a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54576/0 (socket says 172.21.15.112:54576) 2024-09-16T00:25:52.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.135+0000 7ff70f7fe640 1 -- 172.21.15.112:0/2608904056 learned_addr learned my addr 172.21.15.112:0/2608904056 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:25:52.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.136+0000 7ff70effd640 1 --2- 172.21.15.112:0/2608904056 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7100715d0 0x7ff710113be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:52.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.136+0000 7ff70f7fe640 1 -- 172.21.15.112:0/2608904056 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7100715d0 msgr2=0x7ff710113be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:52.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.136+0000 7ff70f7fe640 1 --2- 172.21.15.112:0/2608904056 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7100715d0 0x7ff710113be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:52.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.136+0000 7ff70f7fe640 1 -- 172.21.15.112:0/2608904056 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff708005040 con 0x7ff710070040 2024-09-16T00:25:52.138 INFO:tasks.workunit.client.0.smithi112.stdout:8/103: symlink d10/d12/d1c/l1e 0 2024-09-16T00:25:52.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.136+0000 7ff70f7fe640 1 --2- 172.21.15.112:0/2608904056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff710070040 0x7ff7101136a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff7080048a0 tx=0x7ff708009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:52.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.137+0000 7ff6f7fff640 1 -- 172.21.15.112:0/2608904056 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff708038070 con 0x7ff710070040 2024-09-16T00:25:52.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.137+0000 7ff71600e640 1 -- 172.21.15.112:0/2608904056 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff710114c80 con 0x7ff710070040 2024-09-16T00:25:52.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.137+0000 7ff71600e640 1 -- 172.21.15.112:0/2608904056 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff710115160 con 0x7ff710070040 2024-09-16T00:25:52.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.138+0000 7ff6f7fff640 1 -- 172.21.15.112:0/2608904056 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7ff708004290 con 0x7ff710070040 2024-09-16T00:25:52.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.138+0000 7ff6f7fff640 1 -- 172.21.15.112:0/2608904056 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff70803e670 con 0x7ff710070040 2024-09-16T00:25:52.140 INFO:tasks.workunit.client.0.smithi112.stdout:7/130: creat d6/d10/d1b/d30/f36 x:0 0 0 2024-09-16T00:25:52.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.139+0000 7ff71600e640 1 -- 172.21.15.112:0/2608904056 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7100fb990 con 0x7ff710070040 2024-09-16T00:25:52.141 INFO:tasks.workunit.client.0.smithi112.stdout:4/124: symlink d6/db/l2e 0 2024-09-16T00:25:52.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.140+0000 7ff6f7fff640 1 -- 172.21.15.112:0/2608904056 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 20) v1 ==== 91584+0+0 (secure 0 0 0) 0x7ff708030020 con 0x7ff710070040 2024-09-16T00:25:52.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.141+0000 7ff6f7fff640 1 --2- 172.21.15.112:0/2608904056 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7ff6f806e690 0x7ff6f8070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:52.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.141+0000 7ff6f7fff640 1 -- 172.21.15.112:0/2608904056 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5550+0+0 (secure 0 0 0) 0x7ff7080adc70 con 0x7ff710070040 2024-09-16T00:25:52.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.141+0000 7ff70effd640 1 --2- 172.21.15.112:0/2608904056 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7ff6f806e690 0x7ff6f8070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:52.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.142+0000 7ff70effd640 1 --2- 172.21.15.112:0/2608904056 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7ff6f806e690 0x7ff6f8070b40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff710113450 tx=0x7ff70000b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:52.144 INFO:tasks.workunit.client.0.smithi112.stdout:7/131: rename d6/d10/d16/l2b to d6/l37 0 2024-09-16T00:25:52.145 INFO:tasks.workunit.client.0.smithi112.stdout:7/132: write d6/f22 [608598,24877] 0 2024-09-16T00:25:52.147 INFO:tasks.workunit.client.0.smithi112.stdout:4/125: creat d6/da/dc/d19/d1c/f2f x:0 0 0 2024-09-16T00:25:52.148 INFO:tasks.workunit.client.0.smithi112.stdout:3/139: creat d2/d9/f35 x:0 0 0 2024-09-16T00:25:52.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.148+0000 7ff6f7fff640 1 -- 172.21.15.112:0/2608904056 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7ff70807ad20 con 0x7ff710070040 2024-09-16T00:25:52.150 INFO:tasks.workunit.client.0.smithi112.stdout:3/140: dread - d2/d9/f35 zero size 2024-09-16T00:25:52.150 INFO:tasks.workunit.client.0.smithi112.stdout:8/104: creat d10/f1f x:0 0 0 2024-09-16T00:25:52.151 INFO:tasks.workunit.client.0.smithi112.stdout:7/133: creat d6/d10/d1b/d30/f38 x:0 0 0 2024-09-16T00:25:52.153 INFO:tasks.workunit.client.0.smithi112.stdout:4/126: dread d6/f1b [0,4194304] 0 2024-09-16T00:25:52.153 INFO:tasks.workunit.client.0.smithi112.stdout:7/134: dread - d6/d10/d1b/d30/f33 zero size 2024-09-16T00:25:52.154 INFO:tasks.workunit.client.0.smithi112.stdout:3/141: write d2/d9/f16 [5771396,125755] 0 2024-09-16T00:25:52.154 INFO:tasks.workunit.client.0.smithi112.stdout:4/127: chown d6/db/d1f 1 1 2024-09-16T00:25:52.154 INFO:tasks.workunit.client.0.smithi112.stdout:7/135: fsync d6/f23 0 2024-09-16T00:25:52.157 INFO:tasks.workunit.client.0.smithi112.stdout:3/142: stat d2/f1d 0 2024-09-16T00:25:52.157 INFO:tasks.workunit.client.0.smithi112.stdout:8/105: dread fe [0,4194304] 0 2024-09-16T00:25:52.158 INFO:tasks.workunit.client.0.smithi112.stdout:4/128: mknod d6/da/c30 0 2024-09-16T00:25:52.159 INFO:tasks.workunit.client.0.smithi112.stdout:7/136: mkdir d6/d10/d1b/d39 0 2024-09-16T00:25:52.163 INFO:tasks.workunit.client.0.smithi112.stdout:7/137: creat d6/d10/d16/f3a x:0 0 0 2024-09-16T00:25:52.164 INFO:tasks.workunit.client.0.smithi112.stdout:7/138: fsync d6/d10/d1b/d30/f33 0 2024-09-16T00:25:52.164 INFO:tasks.workunit.client.0.smithi112.stdout:4/129: dwrite d6/f9 [8388608,4194304] 0 2024-09-16T00:25:52.165 INFO:tasks.workunit.client.0.smithi112.stdout:3/143: rename d2/d9/d2b/d2c/l2e to d2/d9/d2b/l36 0 2024-09-16T00:25:52.165 INFO:tasks.workunit.client.0.smithi112.stdout:8/106: rename d10/d12/d1c to d10/d12/d1c/d20 22 2024-09-16T00:25:52.166 INFO:tasks.workunit.client.0.smithi112.stdout:8/107: stat d10/d12/d1c/l1e 0 2024-09-16T00:25:52.167 INFO:tasks.workunit.client.0.smithi112.stdout:4/130: dread - d6/da/dc/d19/d1c/f2b zero size 2024-09-16T00:25:52.168 INFO:tasks.workunit.client.0.smithi112.stdout:4/131: dread - d6/da/dc/d19/d1c/f2f zero size 2024-09-16T00:25:52.173 INFO:tasks.workunit.client.0.smithi112.stdout:3/144: mknod d2/d9/d2b/d33/c37 0 2024-09-16T00:25:52.173 INFO:tasks.workunit.client.0.smithi112.stdout:4/132: mkdir d6/db/d15/d31 0 2024-09-16T00:25:52.173 INFO:tasks.workunit.client.0.smithi112.stdout:4/133: stat d6/ld 0 2024-09-16T00:25:52.173 INFO:tasks.workunit.client.0.smithi112.stdout:7/139: rmdir d6/d10 39 2024-09-16T00:25:52.173 INFO:tasks.workunit.client.0.smithi112.stdout:8/108: creat d10/f21 x:0 0 0 2024-09-16T00:25:52.174 INFO:tasks.workunit.client.0.smithi112.stdout:3/145: dread d2/f1f [0,4194304] 0 2024-09-16T00:25:52.175 INFO:tasks.workunit.client.0.smithi112.stdout:4/134: rename d6/db/l2a to d6/da/dc/l32 0 2024-09-16T00:25:52.178 INFO:tasks.workunit.client.0.smithi112.stdout:8/109: mknod d10/c22 0 2024-09-16T00:25:52.179 INFO:tasks.workunit.client.0.smithi112.stdout:8/110: stat d10/c22 0 2024-09-16T00:25:52.179 INFO:tasks.workunit.client.0.smithi112.stdout:3/146: creat d2/d9/d2b/d2c/d32/f38 x:0 0 0 2024-09-16T00:25:52.181 INFO:tasks.workunit.client.0.smithi112.stdout:8/111: fsync d10/f1b 0 2024-09-16T00:25:52.181 INFO:tasks.workunit.client.0.smithi112.stdout:7/140: dwrite f5 [0,4194304] 0 2024-09-16T00:25:52.182 INFO:tasks.workunit.client.0.smithi112.stdout:8/112: dread - d10/f21 zero size 2024-09-16T00:25:52.186 INFO:tasks.workunit.client.0.smithi112.stdout:3/147: mkdir d2/d9/d2b/d39 0 2024-09-16T00:25:52.187 INFO:tasks.workunit.client.0.smithi112.stdout:7/141: mkdir d6/d10/d16/d3b 0 2024-09-16T00:25:52.192 INFO:tasks.workunit.client.0.smithi112.stdout:8/113: dwrite d10/d12/d1c/f1d [0,4194304] 0 2024-09-16T00:25:52.276 INFO:tasks.workunit.client.0.smithi112.stdout:4/135: truncate f4 1527606 0 2024-09-16T00:25:52.276 INFO:tasks.workunit.client.0.smithi112.stdout:7/142: dread d6/d10/d16/f20 [0,4194304] 0 2024-09-16T00:25:52.276 INFO:tasks.workunit.client.0.smithi112.stdout:3/148: rename d2/d9/f11 to d2/f3a 0 2024-09-16T00:25:52.276 INFO:tasks.workunit.client.0.smithi112.stdout:4/136: truncate d6/da/dc/d19/d1c/f27 1070965 0 2024-09-16T00:25:52.276 INFO:tasks.workunit.client.0.smithi112.stdout:4/137: chown d6/da/dc/d19/d1c/f2b 4266587 1 2024-09-16T00:25:52.276 INFO:tasks.workunit.client.0.smithi112.stdout:7/143: mknod d6/d26/c3c 0 2024-09-16T00:25:52.277 INFO:tasks.workunit.client.0.smithi112.stdout:3/149: dread d2/d9/f21 [0,4194304] 0 2024-09-16T00:25:52.277 INFO:tasks.workunit.client.0.smithi112.stdout:8/114: link d10/f1f d10/f23 0 2024-09-16T00:25:52.277 INFO:tasks.workunit.client.0.smithi112.stdout:3/150: rmdir d2/d9/d2b/d2c/d32 39 2024-09-16T00:25:52.277 INFO:tasks.workunit.client.0.smithi112.stdout:8/115: chown d10/c17 1260 1 2024-09-16T00:25:52.277 INFO:tasks.workunit.client.0.smithi112.stdout:4/138: link d6/da/c10 d6/db/d1f/c33 0 2024-09-16T00:25:52.277 INFO:tasks.workunit.client.0.smithi112.stdout:3/151: creat d2/d9/d2b/d33/f3b x:0 0 0 2024-09-16T00:25:52.277 INFO:tasks.workunit.client.0.smithi112.stdout:3/152: stat d2/d9/d2b/l36 0 2024-09-16T00:25:52.278 INFO:tasks.workunit.client.0.smithi112.stdout:4/139: mkdir d6/db/d15/d31/d34 0 2024-09-16T00:25:52.278 INFO:tasks.workunit.client.0.smithi112.stdout:3/153: write d2/d9/d2b/d33/f3b [328423,84156] 0 2024-09-16T00:25:52.278 INFO:tasks.workunit.client.0.smithi112.stdout:7/144: truncate d6/d10/d16/f20 2136039 0 2024-09-16T00:25:52.278 INFO:tasks.workunit.client.0.smithi112.stdout:7/145: write d6/f11 [5147987,5798] 0 2024-09-16T00:25:52.278 INFO:tasks.workunit.client.0.smithi112.stdout:3/154: creat d2/d9/d2b/d39/f3c x:0 0 0 2024-09-16T00:25:52.278 INFO:tasks.workunit.client.0.smithi112.stdout:3/155: write d2/f3a [4376633,103858] 0 2024-09-16T00:25:52.278 INFO:tasks.workunit.client.0.smithi112.stdout:4/140: write d6/da/dc/d19/d1c/d2c/f29 [2538634,28448] 0 2024-09-16T00:25:52.278 INFO:tasks.workunit.client.0.smithi112.stdout:8/116: dwrite d10/f1f [0,4194304] 0 2024-09-16T00:25:52.279 INFO:tasks.workunit.client.0.smithi112.stdout:8/117: write d10/d12/f1a [241754,49086] 0 2024-09-16T00:25:52.279 INFO:tasks.workunit.client.0.smithi112.stdout:3/156: write d2/f19 [609742,28644] 0 2024-09-16T00:25:52.279 INFO:tasks.workunit.client.0.smithi112.stdout:4/141: chown d6/da/dc/d19/d1c/d2c/l14 0 1 2024-09-16T00:25:52.279 INFO:tasks.workunit.client.0.smithi112.stdout:8/118: chown d10/d12/f16 233239856 1 2024-09-16T00:25:52.279 INFO:tasks.workunit.client.0.smithi112.stdout:8/119: stat d10/d12 0 2024-09-16T00:25:52.279 INFO:tasks.workunit.client.0.smithi112.stdout:3/157: link d2/c22 d2/d9/d2b/d2c/c3d 0 2024-09-16T00:25:52.279 INFO:tasks.workunit.client.0.smithi112.stdout:3/158: symlink d2/d9/d2b/d2c/d32/l3e 0 2024-09-16T00:25:52.279 INFO:tasks.workunit.client.0.smithi112.stdout:4/142: write d6/db/f18 [521934,49382] 0 2024-09-16T00:25:52.279 INFO:tasks.workunit.client.0.smithi112.stdout:1/97: sync 2024-09-16T00:25:52.279 INFO:tasks.workunit.client.0.smithi112.stdout:2/158: sync 2024-09-16T00:25:52.279 INFO:tasks.workunit.client.0.smithi112.stdout:9/250: sync 2024-09-16T00:25:52.279 INFO:tasks.workunit.client.0.smithi112.stdout:5/210: sync 2024-09-16T00:25:52.280 INFO:tasks.workunit.client.0.smithi112.stdout:5/211: stat d4/dd/f2a 0 2024-09-16T00:25:52.280 INFO:tasks.workunit.client.0.smithi112.stdout:4/143: creat d6/da/f35 x:0 0 0 2024-09-16T00:25:52.280 INFO:tasks.workunit.client.0.smithi112.stdout:7/146: dread d6/f11 [0,4194304] 0 2024-09-16T00:25:52.280 INFO:tasks.workunit.client.0.smithi112.stdout:1/98: mkdir d0/d14/d1c/d20 0 2024-09-16T00:25:52.280 INFO:tasks.workunit.client.0.smithi112.stdout:4/144: mkdir d6/da/dc/d19/d36 0 2024-09-16T00:25:52.280 INFO:tasks.workunit.client.0.smithi112.stdout:2/159: rename d2/dc/d2e to d2/dc/d22/d2b/d30 0 2024-09-16T00:25:52.280 INFO:tasks.workunit.client.0.smithi112.stdout:8/120: dwrite d10/d12/f16 [0,4194304] 0 2024-09-16T00:25:52.280 INFO:tasks.workunit.client.0.smithi112.stdout:5/212: mkdir d4/d4f 0 2024-09-16T00:25:52.280 INFO:tasks.workunit.client.0.smithi112.stdout:3/159: truncate d2/d9/f16 5584325 0 2024-09-16T00:25:52.280 INFO:tasks.workunit.client.0.smithi112.stdout:7/147: rename f5 to d6/d10/f3d 0 2024-09-16T00:25:52.280 INFO:tasks.workunit.client.0.smithi112.stdout:9/251: dwrite d1/d5/d6/d18/f4b [0,4194304] 0 2024-09-16T00:25:52.280 INFO:tasks.workunit.client.0.smithi112.stdout:2/160: creat d2/dc/d22/d2b/f31 x:0 0 0 2024-09-16T00:25:52.280 INFO:tasks.workunit.client.0.smithi112.stdout:1/99: rename d0/d14/df/f19 to d0/d14/d1c/d20/f21 0 2024-09-16T00:25:52.280 INFO:tasks.workunit.client.0.smithi112.stdout:2/161: symlink d2/dc/d22/d2b/d30/l32 0 2024-09-16T00:25:52.281 INFO:tasks.workunit.client.0.smithi112.stdout:5/213: dwrite d4/dd/f4e [0,4194304] 0 2024-09-16T00:25:52.281 INFO:tasks.workunit.client.0.smithi112.stdout:3/160: creat d2/f3f x:0 0 0 2024-09-16T00:25:52.281 INFO:tasks.workunit.client.0.smithi112.stdout:2/162: write d2/dd/f15 [3001951,53798] 0 2024-09-16T00:25:52.281 INFO:tasks.workunit.client.0.smithi112.stdout:2/163: dread - d2/dc/f23 zero size 2024-09-16T00:25:52.281 INFO:tasks.workunit.client.0.smithi112.stdout:7/148: rename d6/d26/f35 to d6/d10/d16/f3e 0 2024-09-16T00:25:52.281 INFO:tasks.workunit.client.0.smithi112.stdout:1/100: truncate d0/f18 409139 0 2024-09-16T00:25:52.281 INFO:tasks.workunit.client.0.smithi112.stdout:5/214: symlink d4/d1c/d43/l50 0 2024-09-16T00:25:52.282 INFO:tasks.workunit.client.0.smithi112.stdout:3/161: creat d2/d9/d2b/d2c/d32/f40 x:0 0 0 2024-09-16T00:25:52.282 INFO:tasks.workunit.client.0.smithi112.stdout:7/149: read d6/d10/d16/f3e [551202,28798] 0 2024-09-16T00:25:52.282 INFO:tasks.workunit.client.0.smithi112.stdout:1/101: creat d0/d14/f22 x:0 0 0 2024-09-16T00:25:52.282 INFO:tasks.workunit.client.0.smithi112.stdout:2/164: mknod d2/c33 0 2024-09-16T00:25:52.282 INFO:tasks.workunit.client.0.smithi112.stdout:1/102: creat d0/d14/f23 x:0 0 0 2024-09-16T00:25:52.282 INFO:tasks.workunit.client.0.smithi112.stdout:3/162: symlink d2/d9/d2b/d33/l41 0 2024-09-16T00:25:52.282 INFO:tasks.workunit.client.0.smithi112.stdout:7/150: mknod d6/d10/d16/c3f 0 2024-09-16T00:25:52.283 INFO:tasks.workunit.client.0.smithi112.stdout:3/163: fsync d2/f1a 0 2024-09-16T00:25:52.286 INFO:tasks.workunit.client.0.smithi112.stdout:5/215: dwrite d4/d7/d26/f46 [0,4194304] 0 2024-09-16T00:25:52.286 INFO:tasks.workunit.client.0.smithi112.stdout:1/103: mkdir d0/d14/df/d24 0 2024-09-16T00:25:52.290 INFO:tasks.workunit.client.0.smithi112.stdout:5/216: unlink d4/d7/d14/f37 0 2024-09-16T00:25:52.293 INFO:tasks.workunit.client.0.smithi112.stdout:3/164: rename d2/d9/f17 to d2/d9/d2b/d2c/f42 0 2024-09-16T00:25:52.293 INFO:tasks.workunit.client.0.smithi112.stdout:3/165: stat l0 0 2024-09-16T00:25:52.294 INFO:tasks.workunit.client.0.smithi112.stdout:7/151: dwrite d6/d10/f1d [0,4194304] 0 2024-09-16T00:25:52.296 INFO:tasks.workunit.client.0.smithi112.stdout:5/217: mknod d4/dd/d12/c51 0 2024-09-16T00:25:52.298 INFO:tasks.workunit.client.0.smithi112.stdout:7/152: symlink d6/d10/l40 0 2024-09-16T00:25:52.301 INFO:tasks.workunit.client.0.smithi112.stdout:7/153: symlink d6/d10/d1b/l41 0 2024-09-16T00:25:52.303 INFO:tasks.workunit.client.0.smithi112.stdout:7/154: unlink d6/c14 0 2024-09-16T00:25:52.304 INFO:tasks.workunit.client.0.smithi112.stdout:3/166: dwrite d2/d9/d2b/d39/f3c [0,4194304] 0 2024-09-16T00:25:52.305 INFO:tasks.workunit.client.0.smithi112.stdout:7/155: rmdir d6/d10/d1b 39 2024-09-16T00:25:52.306 INFO:tasks.workunit.client.0.smithi112.stdout:3/167: mkdir d2/d9/d2b/d39/d43 0 2024-09-16T00:25:52.307 INFO:tasks.workunit.client.0.smithi112.stdout:7/156: symlink d6/d10/d16/l42 0 2024-09-16T00:25:52.308 INFO:tasks.workunit.client.0.smithi112.stdout:3/168: write d2/f1f [889293,103129] 0 2024-09-16T00:25:52.311 INFO:tasks.workunit.client.0.smithi112.stdout:7/157: creat d6/d10/d1b/d39/f43 x:0 0 0 2024-09-16T00:25:52.312 INFO:tasks.workunit.client.0.smithi112.stdout:3/169: rename d2/d9/l15 to d2/d9/d2b/d39/l44 0 2024-09-16T00:25:52.312 INFO:tasks.workunit.client.0.smithi112.stdout:7/158: rmdir d6/d10 39 2024-09-16T00:25:52.313 INFO:tasks.workunit.client.0.smithi112.stdout:3/170: write d2/d9/d2b/d2f/f30 [93238,46445] 0 2024-09-16T00:25:52.316 INFO:tasks.workunit.client.0.smithi112.stdout:7/159: mknod d6/d10/d16/d3b/c44 0 2024-09-16T00:25:52.317 INFO:tasks.workunit.client.0.smithi112.stdout:7/160: fsync d6/fa 0 2024-09-16T00:25:52.318 INFO:tasks.workunit.client.0.smithi112.stdout:7/161: readlink d6/d10/d16/l42 0 2024-09-16T00:25:52.320 INFO:tasks.workunit.client.0.smithi112.stdout:3/171: dwrite d2/f1a [0,4194304] 0 2024-09-16T00:25:52.321 INFO:tasks.workunit.client.0.smithi112.stdout:7/162: truncate d6/d26/f27 3354084 0 2024-09-16T00:25:52.322 INFO:tasks.workunit.client.0.smithi112.stdout:3/172: rmdir d2/d9/d2b 39 2024-09-16T00:25:52.324 INFO:tasks.workunit.client.0.smithi112.stdout:7/163: write d6/d10/f15 [403331,82455] 0 2024-09-16T00:25:52.328 INFO:tasks.workunit.client.0.smithi112.stdout:3/173: rmdir d2/d9/d2b/d39/d43 0 2024-09-16T00:25:52.330 INFO:tasks.workunit.client.0.smithi112.stdout:7/164: dwrite d6/d10/d1b/d39/f43 [0,4194304] 0 2024-09-16T00:25:52.333 INFO:tasks.workunit.client.0.smithi112.stdout:3/174: rename d2/c22 to d2/d9/d2b/d2f/c45 0 2024-09-16T00:25:52.334 INFO:tasks.workunit.client.0.smithi112.stdout:7/165: symlink d6/d10/l45 0 2024-09-16T00:25:52.335 INFO:tasks.workunit.client.0.smithi112.stdout:3/175: mknod d2/d9/d2b/d33/c46 0 2024-09-16T00:25:52.339 INFO:tasks.workunit.client.0.smithi112.stdout:7/166: rename d6/d10/d1b/d30 to d6/d10/d46 0 2024-09-16T00:25:52.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:25:52 smithi112 ceph-mon[27869]: pgmap v249: 65 pgs: 65 active+clean; 297 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.8 MiB/s rd, 12 MiB/s wr, 398 op/s 2024-09-16T00:25:52.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:25:52 smithi112 ceph-mon[27869]: from='client.14584 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:25:52.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:25:52 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:25:52.341 INFO:tasks.workunit.client.0.smithi112.stdout:7/167: creat d6/d10/d1b/f47 x:0 0 0 2024-09-16T00:25:52.342 INFO:tasks.workunit.client.0.smithi112.stdout:3/176: dwrite d2/f3a [0,4194304] 0 2024-09-16T00:25:52.345 INFO:tasks.workunit.client.0.smithi112.stdout:7/168: dread d6/d10/d16/f20 [0,4194304] 0 2024-09-16T00:25:52.346 INFO:tasks.workunit.client.0.smithi112.stdout:7/169: mknod d6/d26/c48 0 2024-09-16T00:25:52.349 INFO:tasks.workunit.client.0.smithi112.stdout:7/170: dread d6/d10/d16/f20 [0,4194304] 0 2024-09-16T00:25:52.456 INFO:tasks.workunit.client.0.smithi112.stdout:6/137: sync 2024-09-16T00:25:52.463 INFO:tasks.workunit.client.0.smithi112.stdout:8/121: truncate d10/f1f 341048 0 2024-09-16T00:25:52.464 INFO:tasks.workunit.client.0.smithi112.stdout:9/252: write d1/d5/d6/d2d/d45/f1d [3571367,106865] 0 2024-09-16T00:25:52.466 INFO:tasks.workunit.client.0.smithi112.stdout:8/122: mkdir d10/d12/d24 0 2024-09-16T00:25:52.468 INFO:tasks.workunit.client.0.smithi112.stdout:9/253: write d1/d5/d20/f36 [1262025,68959] 0 2024-09-16T00:25:52.469 INFO:tasks.workunit.client.0.smithi112.stdout:8/123: stat d10/l14 0 2024-09-16T00:25:52.469 INFO:tasks.workunit.client.0.smithi112.stdout:9/254: stat d1/d5/d6/d18/f40 0 2024-09-16T00:25:52.472 INFO:tasks.workunit.client.0.smithi112.stdout:8/124: mknod d10/d12/d1c/c25 0 2024-09-16T00:25:52.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.472+0000 7ff71600e640 1 -- 172.21.15.112:0/2608904056 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff710115430 con 0x7ff710070040 2024-09-16T00:25:52.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.473+0000 7ff6f7fff640 1 -- 172.21.15.112:0/2608904056 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+855 (secure 0 0 0) 0x7ff70807a6c0 con 0x7ff710070040 2024-09-16T00:25:52.475 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:25:52.475 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": { 2024-09-16T00:25:52.475 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 17.2.7-1633-g6d025bc2 (6d025bc26f3176c910707f8cb4c8d8e076640615) quincy (stable)": 2 2024-09-16T00:25:52.475 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:25:52.475 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": { 2024-09-16T00:25:52.475 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 17.2.7-1633-g6d025bc2 (6d025bc26f3176c910707f8cb4c8d8e076640615) quincy (stable)": 1, 2024-09-16T00:25:52.475 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 1 2024-09-16T00:25:52.475 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:25:52.475 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": { 2024-09-16T00:25:52.475 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 17.2.7-1633-g6d025bc2 (6d025bc26f3176c910707f8cb4c8d8e076640615) quincy (stable)": 6 2024-09-16T00:25:52.475 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:25:52.475 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": { 2024-09-16T00:25:52.476 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 17.2.7-1633-g6d025bc2 (6d025bc26f3176c910707f8cb4c8d8e076640615) quincy (stable)": 4 2024-09-16T00:25:52.476 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:25:52.476 INFO:teuthology.orchestra.run.smithi112.stdout: "overall": { 2024-09-16T00:25:52.476 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 17.2.7-1633-g6d025bc2 (6d025bc26f3176c910707f8cb4c8d8e076640615) quincy (stable)": 13, 2024-09-16T00:25:52.476 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 1 2024-09-16T00:25:52.476 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-16T00:25:52.476 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:25:52.476 INFO:tasks.workunit.client.0.smithi112.stdout:2/165: write d2/dc/f24 [786941,5415] 0 2024-09-16T00:25:52.476 INFO:tasks.workunit.client.0.smithi112.stdout:2/166: chown d2 203 1 2024-09-16T00:25:52.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.476+0000 7ff71600e640 1 -- 172.21.15.112:0/2608904056 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7ff6f806e690 msgr2=0x7ff6f8070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:52.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.476+0000 7ff71600e640 1 --2- 172.21.15.112:0/2608904056 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7ff6f806e690 0x7ff6f8070b40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff710113450 tx=0x7ff70000b040 comp rx=0 tx=0).stop 2024-09-16T00:25:52.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.476+0000 7ff71600e640 1 -- 172.21.15.112:0/2608904056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff710070040 msgr2=0x7ff7101136a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:52.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.476+0000 7ff71600e640 1 --2- 172.21.15.112:0/2608904056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff710070040 0x7ff7101136a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff7080048a0 tx=0x7ff708009da0 comp rx=0 tx=0).stop 2024-09-16T00:25:52.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.477+0000 7ff71600e640 1 -- 172.21.15.112:0/2608904056 shutdown_connections 2024-09-16T00:25:52.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.477+0000 7ff71600e640 1 --2- 172.21.15.112:0/2608904056 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7ff6f806e690 0x7ff6f8070b40 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:52.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.477+0000 7ff71600e640 1 --2- 172.21.15.112:0/2608904056 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7100715d0 0x7ff710113be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:52.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.477+0000 7ff71600e640 1 --2- 172.21.15.112:0/2608904056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff710070040 0x7ff7101136a0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:52.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.477+0000 7ff71600e640 1 -- 172.21.15.112:0/2608904056 >> 172.21.15.112:0/2608904056 conn(0x7ff7100f7d10 msgr2=0x7ff7100f9500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:52.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.477+0000 7ff71600e640 1 -- 172.21.15.112:0/2608904056 shutdown_connections 2024-09-16T00:25:52.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.477+0000 7ff71600e640 1 -- 172.21.15.112:0/2608904056 wait complete. 2024-09-16T00:25:52.479 INFO:tasks.workunit.client.0.smithi112.stdout:6/138: dread fa [0,4194304] 0 2024-09-16T00:25:52.480 INFO:tasks.workunit.client.0.smithi112.stdout:8/125: link la d10/d12/d24/l26 0 2024-09-16T00:25:52.482 INFO:tasks.workunit.client.0.smithi112.stdout:9/255: dwrite d1/d5/d6/d2d/d45/f47 [4194304,4194304] 0 2024-09-16T00:25:52.482 INFO:tasks.workunit.client.0.smithi112.stdout:2/167: creat d2/dc/d22/d2b/f34 x:0 0 0 2024-09-16T00:25:52.482 INFO:tasks.workunit.client.0.smithi112.stdout:8/126: write d10/f15 [1410548,105415] 0 2024-09-16T00:25:52.485 INFO:tasks.workunit.client.0.smithi112.stdout:5/218: write d4/d7/d14/f1a [1987790,126474] 0 2024-09-16T00:25:52.490 INFO:tasks.workunit.client.0.smithi112.stdout:8/127: symlink d10/d12/d24/l27 0 2024-09-16T00:25:52.492 INFO:tasks.workunit.client.0.smithi112.stdout:9/256: creat d1/d5/d20/f5f x:0 0 0 2024-09-16T00:25:52.494 INFO:tasks.workunit.client.0.smithi112.stdout:8/128: creat d10/d12/d24/f28 x:0 0 0 2024-09-16T00:25:52.496 INFO:tasks.workunit.client.0.smithi112.stdout:8/129: creat d10/d12/d24/f29 x:0 0 0 2024-09-16T00:25:52.498 INFO:tasks.workunit.client.0.smithi112.stdout:9/257: mkdir d1/d5/d20/d60 0 2024-09-16T00:25:52.502 INFO:tasks.workunit.client.0.smithi112.stdout:9/258: symlink d1/d5/d6/d2d/d39/l61 0 2024-09-16T00:25:52.504 INFO:tasks.workunit.client.0.smithi112.stdout:3/177: rename d2/d9/d2b/d2f to d2/d9/d2b/d47 0 2024-09-16T00:25:52.504 INFO:tasks.workunit.client.0.smithi112.stdout:8/130: dwrite d10/d12/d1c/f1d [0,4194304] 0 2024-09-16T00:25:52.511 INFO:tasks.workunit.client.0.smithi112.stdout:8/131: write d10/d12/d1c/f1d [2883208,36059] 0 2024-09-16T00:25:52.513 INFO:tasks.workunit.client.0.smithi112.stdout:7/171: write d6/f9 [2410560,83820] 0 2024-09-16T00:25:52.515 INFO:tasks.workunit.client.0.smithi112.stdout:6/139: dread dd/f2a [0,4194304] 0 2024-09-16T00:25:52.516 INFO:tasks.workunit.client.0.smithi112.stdout:1/104: write d0/f18 [986919,119040] 0 2024-09-16T00:25:52.518 INFO:tasks.workunit.client.0.smithi112.stdout:9/259: dwrite d1/d5/f30 [0,4194304] 0 2024-09-16T00:25:52.519 INFO:tasks.workunit.client.0.smithi112.stdout:8/132: dread - d10/d12/d24/f29 zero size 2024-09-16T00:25:52.522 INFO:tasks.workunit.client.0.smithi112.stdout:3/178: dwrite d2/f3a [4194304,4194304] 0 2024-09-16T00:25:52.523 INFO:tasks.workunit.client.0.smithi112.stdout:1/105: dread - d0/d14/f23 zero size 2024-09-16T00:25:52.525 INFO:tasks.workunit.client.0.smithi112.stdout:9/260: write d1/d5/d6/d18/f19 [645307,42475] 0 2024-09-16T00:25:52.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:25:52 smithi169 ceph-mon[34653]: pgmap v249: 65 pgs: 65 active+clean; 297 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.8 MiB/s rd, 12 MiB/s wr, 398 op/s 2024-09-16T00:25:52.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:25:52 smithi169 ceph-mon[34653]: from='client.14584 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:25:52.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:25:52 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:25:52.526 INFO:tasks.workunit.client.0.smithi112.stdout:8/133: write d10/d12/f16 [3674258,93556] 0 2024-09-16T00:25:52.527 INFO:tasks.workunit.client.0.smithi112.stdout:7/172: mkdir d6/d10/d16/d49 0 2024-09-16T00:25:52.527 INFO:tasks.workunit.client.0.smithi112.stdout:2/168: dread f1 [0,4194304] 0 2024-09-16T00:25:52.529 INFO:tasks.workunit.client.0.smithi112.stdout:2/169: fdatasync d2/dc/d22/d2b/f34 0 2024-09-16T00:25:52.531 INFO:tasks.workunit.client.0.smithi112.stdout:9/261: fsync d1/d5/d20/d25/f1c 0 2024-09-16T00:25:52.531 INFO:tasks.workunit.client.0.smithi112.stdout:8/134: readlink d10/l11 0 2024-09-16T00:25:52.532 INFO:tasks.workunit.client.0.smithi112.stdout:7/173: chown d6/d26/l28 32923026 1 2024-09-16T00:25:52.533 INFO:tasks.workunit.client.0.smithi112.stdout:2/170: truncate d2/dc/d22/f29 388217 0 2024-09-16T00:25:52.534 INFO:tasks.workunit.client.0.smithi112.stdout:7/174: truncate d6/d10/d1b/f47 55973 0 2024-09-16T00:25:52.534 INFO:tasks.workunit.client.0.smithi112.stdout:3/179: creat d2/d9/d2b/f48 x:0 0 0 2024-09-16T00:25:52.536 INFO:tasks.workunit.client.0.smithi112.stdout:8/135: symlink d10/d12/d24/l2a 0 2024-09-16T00:25:52.536 INFO:tasks.workunit.client.0.smithi112.stdout:2/171: truncate d2/dc/f23 581138 0 2024-09-16T00:25:52.537 INFO:tasks.workunit.client.0.smithi112.stdout:7/175: rmdir d6/d10/d1b/d39 39 2024-09-16T00:25:52.540 INFO:tasks.workunit.client.0.smithi112.stdout:9/262: creat d1/f62 x:0 0 0 2024-09-16T00:25:52.540 INFO:tasks.workunit.client.0.smithi112.stdout:3/180: mkdir d2/d9/d2b/d33/d49 0 2024-09-16T00:25:52.549 INFO:tasks.workunit.client.0.smithi112.stdout:8/136: dwrite d10/f15 [0,4194304] 0 2024-09-16T00:25:52.550 INFO:tasks.workunit.client.0.smithi112.stdout:2/172: dwrite d2/dc/f23 [0,4194304] 0 2024-09-16T00:25:52.553 INFO:tasks.workunit.client.0.smithi112.stdout:7/176: dwrite d6/d10/d16/f1e [0,4194304] 0 2024-09-16T00:25:52.555 INFO:tasks.workunit.client.0.smithi112.stdout:3/181: dwrite d2/f1a [0,4194304] 0 2024-09-16T00:25:52.556 INFO:tasks.workunit.client.0.smithi112.stdout:2/173: write d2/dc/d22/d2b/f34 [439598,121180] 0 2024-09-16T00:25:52.556 INFO:tasks.workunit.client.0.smithi112.stdout:7/177: dread - d6/f23 zero size 2024-09-16T00:25:52.559 INFO:tasks.workunit.client.0.smithi112.stdout:9/263: dwrite d1/d5/d6/d2d/d39/f3c [0,4194304] 0 2024-09-16T00:25:52.568 INFO:tasks.workunit.client.0.smithi112.stdout:3/182: getdents d2/d9/d2b/d47 0 2024-09-16T00:25:52.571 INFO:tasks.workunit.client.0.smithi112.stdout:2/174: dwrite d2/f14 [0,4194304] 0 2024-09-16T00:25:52.572 INFO:tasks.workunit.client.0.smithi112.stdout:3/183: truncate d2/f25 971624 0 2024-09-16T00:25:52.574 INFO:tasks.workunit.client.0.smithi112.stdout:7/178: dwrite d6/d10/d16/f3a [0,4194304] 0 2024-09-16T00:25:52.574 INFO:tasks.workunit.client.0.smithi112.stdout:2/175: mknod d2/dd/c35 0 2024-09-16T00:25:52.578 INFO:tasks.workunit.client.0.smithi112.stdout:2/176: creat d2/dc/d22/d2b/d30/f36 x:0 0 0 2024-09-16T00:25:52.578 INFO:tasks.workunit.client.0.smithi112.stdout:3/184: mknod d2/c4a 0 2024-09-16T00:25:52.585 INFO:tasks.workunit.client.0.smithi112.stdout:7/179: dwrite d6/fa [0,4194304] 0 2024-09-16T00:25:52.586 INFO:tasks.workunit.client.0.smithi112.stdout:2/177: dread f1 [0,4194304] 0 2024-09-16T00:25:52.586 INFO:tasks.workunit.client.0.smithi112.stdout:7/180: write d6/d10/f1c [1194647,62264] 0 2024-09-16T00:25:52.589 INFO:tasks.workunit.client.0.smithi112.stdout:2/178: truncate d2/dc/f2d 97677 0 2024-09-16T00:25:52.590 INFO:tasks.workunit.client.0.smithi112.stdout:7/181: creat d6/d10/d46/f4a x:0 0 0 2024-09-16T00:25:52.591 INFO:tasks.workunit.client.0.smithi112.stdout:3/185: dwrite d2/d9/d2b/d39/f3c [0,4194304] 0 2024-09-16T00:25:52.591 INFO:tasks.workunit.client.0.smithi112.stdout:9/264: dread d1/d5/d6/d2d/d45/f1d [0,4194304] 0 2024-09-16T00:25:52.592 INFO:tasks.workunit.client.0.smithi112.stdout:3/186: dread - d2/f1d zero size 2024-09-16T00:25:52.592 INFO:tasks.workunit.client.0.smithi112.stdout:2/179: mknod d2/c37 0 2024-09-16T00:25:52.593 INFO:tasks.workunit.client.0.smithi112.stdout:7/182: creat d6/d10/d16/d3b/f4b x:0 0 0 2024-09-16T00:25:52.594 INFO:tasks.workunit.client.0.smithi112.stdout:7/183: write d6/d10/f12 [1954825,87595] 0 2024-09-16T00:25:52.596 INFO:tasks.workunit.client.0.smithi112.stdout:9/265: creat d1/d5/d6/d2d/d39/f63 x:0 0 0 2024-09-16T00:25:52.596 INFO:tasks.workunit.client.0.smithi112.stdout:9/266: chown d1/d5/d2c/f57 215170 1 2024-09-16T00:25:52.598 INFO:tasks.workunit.client.0.smithi112.stdout:3/187: rename d2/d9/d2b/d33/f3b to d2/d9/d2b/f4b 0 2024-09-16T00:25:52.599 INFO:tasks.workunit.client.0.smithi112.stdout:3/188: chown d2/d9/d2b/d33/l41 3 1 2024-09-16T00:25:52.599 INFO:tasks.workunit.client.0.smithi112.stdout:9/267: mkdir d1/d5/d6/d59/d64 0 2024-09-16T00:25:52.600 INFO:tasks.workunit.client.0.smithi112.stdout:3/189: write d2/d9/d2b/d47/f30 [1027184,70691] 0 2024-09-16T00:25:52.601 INFO:tasks.workunit.client.0.smithi112.stdout:9/268: truncate d1/d5/d2c/f57 560316 0 2024-09-16T00:25:52.602 INFO:tasks.workunit.client.0.smithi112.stdout:3/190: creat d2/d9/d2b/d33/d49/f4c x:0 0 0 2024-09-16T00:25:52.604 INFO:tasks.workunit.client.0.smithi112.stdout:9/269: rename d1/d5/f3f to d1/d5/d6/d2d/d39/f65 0 2024-09-16T00:25:52.607 INFO:tasks.workunit.client.0.smithi112.stdout:9/270: getdents d1/d5/d6/d38 0 2024-09-16T00:25:52.609 INFO:tasks.workunit.client.0.smithi112.stdout:3/191: dwrite d2/d9/d2b/d47/f30 [0,4194304] 0 2024-09-16T00:25:52.611 INFO:tasks.workunit.client.0.smithi112.stdout:9/271: rename d1/d5/d20/d60 to d1/d5/d20/d25/d66 0 2024-09-16T00:25:52.614 INFO:tasks.workunit.client.0.smithi112.stdout:4/145: sync 2024-09-16T00:25:52.614 INFO:tasks.workunit.client.0.smithi112.stdout:0/183: sync 2024-09-16T00:25:52.614 INFO:tasks.workunit.client.0.smithi112.stdout:9/272: creat d1/d5/d6/d2d/d45/f67 x:0 0 0 2024-09-16T00:25:52.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.616+0000 7fb8f07a1640 1 -- 172.21.15.112:0/77214067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8e8103e50 msgr2=0x7fb8e81042c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:52.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.616+0000 7fb8f07a1640 1 --2- 172.21.15.112:0/77214067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8e8103e50 0x7fb8e81042c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb8d80048d0 tx=0x7fb8d8009da0 comp rx=0 tx=0).stop 2024-09-16T00:25:52.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.616+0000 7fb8f07a1640 1 -- 172.21.15.112:0/77214067 shutdown_connections 2024-09-16T00:25:52.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.616+0000 7fb8f07a1640 1 --2- 172.21.15.112:0/77214067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8e8103e50 0x7fb8e81042c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:52.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.616+0000 7fb8f07a1640 1 --2- 172.21.15.112:0/77214067 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8e8102b80 0x7fb8e8102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:52.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.617+0000 7fb8f07a1640 1 -- 172.21.15.112:0/77214067 >> 172.21.15.112:0/77214067 conn(0x7fb8e8071d90 msgr2=0x7fb8e80741a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:52.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.617+0000 7fb8f07a1640 1 -- 172.21.15.112:0/77214067 shutdown_connections 2024-09-16T00:25:52.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.617+0000 7fb8f07a1640 1 -- 172.21.15.112:0/77214067 wait complete. 2024-09-16T00:25:52.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.617+0000 7fb8f07a1640 1 Processor -- start 2024-09-16T00:25:52.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.617+0000 7fb8f07a1640 1 -- start start 2024-09-16T00:25:52.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.618+0000 7fb8f07a1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8e8102b80 0x7fb8e806db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:52.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.618+0000 7fb8f07a1640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8e8103e50 0x7fb8e806e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:52.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.618+0000 7fb8f07a1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e806f7d0 con 0x7fb8e8102b80 2024-09-16T00:25:52.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.618+0000 7fb8f07a1640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e806e5f0 con 0x7fb8e8103e50 2024-09-16T00:25:52.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.618+0000 7fb8ee53d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8e8102b80 0x7fb8e806db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:52.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.618+0000 7fb8ee53d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8e8102b80 0x7fb8e806db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54596/0 (socket says 172.21.15.112:54596) 2024-09-16T00:25:52.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.618+0000 7fb8ee53d640 1 -- 172.21.15.112:0/992810121 learned_addr learned my addr 172.21.15.112:0/992810121 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:25:52.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.618+0000 7fb8edd3c640 1 --2- 172.21.15.112:0/992810121 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8e8103e50 0x7fb8e806e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:52.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.618+0000 7fb8ee53d640 1 -- 172.21.15.112:0/992810121 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8e8103e50 msgr2=0x7fb8e806e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:52.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.618+0000 7fb8ee53d640 1 --2- 172.21.15.112:0/992810121 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8e8103e50 0x7fb8e806e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:52.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.618+0000 7fb8ee53d640 1 -- 172.21.15.112:0/992810121 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d8005040 con 0x7fb8e8102b80 2024-09-16T00:25:52.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.618+0000 7fb8ee53d640 1 --2- 172.21.15.112:0/992810121 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8e8102b80 0x7fb8e806db70 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb8e400b790 tx=0x7fb8e40091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:52.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.619+0000 7fb8deffd640 1 -- 172.21.15.112:0/992810121 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8e400c700 con 0x7fb8e8102b80 2024-09-16T00:25:52.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.619+0000 7fb8deffd640 1 -- 172.21.15.112:0/992810121 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fb8e400cd00 con 0x7fb8e8102b80 2024-09-16T00:25:52.621 INFO:tasks.workunit.client.0.smithi112.stdout:9/273: symlink d1/d5/d6/d2d/d39/l68 0 2024-09-16T00:25:52.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.619+0000 7fb8deffd640 1 -- 172.21.15.112:0/992810121 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8e4015e50 con 0x7fb8e8102b80 2024-09-16T00:25:52.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.619+0000 7fb8f07a1640 1 -- 172.21.15.112:0/992810121 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8e806e8d0 con 0x7fb8e8102b80 2024-09-16T00:25:52.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.619+0000 7fb8f07a1640 1 -- 172.21.15.112:0/992810121 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8e8121bf0 con 0x7fb8e8102b80 2024-09-16T00:25:52.622 INFO:tasks.workunit.client.0.smithi112.stdout:3/192: dwrite d2/d9/d2b/d2c/f42 [4194304,4194304] 0 2024-09-16T00:25:52.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.621+0000 7fb8f07a1640 1 -- 172.21.15.112:0/992810121 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8e8075a10 con 0x7fb8e8102b80 2024-09-16T00:25:52.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.622+0000 7fb8deffd640 1 -- 172.21.15.112:0/992810121 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 20) v1 ==== 91584+0+0 (secure 0 0 0) 0x7fb8e4012070 con 0x7fb8e8102b80 2024-09-16T00:25:52.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.622+0000 7fb8deffd640 1 --2- 172.21.15.112:0/992810121 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fb8d406e5c0 0x7fb8d4070a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:52.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.622+0000 7fb8deffd640 1 -- 172.21.15.112:0/992810121 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fb8e408d6b0 con 0x7fb8e8102b80 2024-09-16T00:25:52.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.622+0000 7fb8edd3c640 1 --2- 172.21.15.112:0/992810121 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fb8d406e5c0 0x7fb8d4070a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:52.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.623+0000 7fb8edd3c640 1 --2- 172.21.15.112:0/992810121 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fb8d406e5c0 0x7fb8d4070a70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb8d8000c00 tx=0x7fb8d8035040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:52.626 INFO:tasks.workunit.client.0.smithi112.stdout:3/193: dread d2/f3a [0,4194304] 0 2024-09-16T00:25:52.627 INFO:tasks.workunit.client.0.smithi112.stdout:9/274: link d1/d5/d6/d2d/d39/f63 d1/d5/d20/f69 0 2024-09-16T00:25:52.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.628+0000 7fb8deffd640 1 -- 172.21.15.112:0/992810121 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fb8e405a780 con 0x7fb8e8102b80 2024-09-16T00:25:52.630 INFO:tasks.workunit.client.0.smithi112.stdout:9/275: mkdir d1/d5/d6/d2d/d6a 0 2024-09-16T00:25:52.633 INFO:tasks.workunit.client.0.smithi112.stdout:3/194: dwrite d2/d9/d2b/d47/f30 [0,4194304] 0 2024-09-16T00:25:52.637 INFO:tasks.workunit.client.0.smithi112.stdout:3/195: unlink d2/d9/l23 0 2024-09-16T00:25:52.639 INFO:tasks.workunit.client.0.smithi112.stdout:9/276: dwrite d1/d5/d6/f34 [0,4194304] 0 2024-09-16T00:25:52.641 INFO:tasks.workunit.client.0.smithi112.stdout:3/196: unlink d2/d9/c18 0 2024-09-16T00:25:52.642 INFO:tasks.workunit.client.0.smithi112.stdout:9/277: symlink d1/d5/d20/d25/l6b 0 2024-09-16T00:25:52.644 INFO:tasks.workunit.client.0.smithi112.stdout:9/278: fdatasync d1/d5/d20/f5f 0 2024-09-16T00:25:52.647 INFO:tasks.workunit.client.0.smithi112.stdout:9/279: unlink d1/d5/d6/d2d/d45/f4d 0 2024-09-16T00:25:52.651 INFO:tasks.workunit.client.0.smithi112.stdout:3/197: dwrite d2/d9/f35 [0,4194304] 0 2024-09-16T00:25:52.651 INFO:tasks.workunit.client.0.smithi112.stdout:9/280: mknod d1/d5/d20/d25/c6c 0 2024-09-16T00:25:52.652 INFO:tasks.workunit.client.0.smithi112.stdout:3/198: write d2/d9/d2b/d47/f30 [2759830,108081] 0 2024-09-16T00:25:52.653 INFO:tasks.workunit.client.0.smithi112.stdout:9/281: symlink d1/d5/d6/d59/l6d 0 2024-09-16T00:25:52.655 INFO:tasks.workunit.client.0.smithi112.stdout:9/282: write d1/d5/d6/d38/f4f [521750,29541] 0 2024-09-16T00:25:52.655 INFO:tasks.workunit.client.0.smithi112.stdout:9/283: read d1/d5/d6/d2d/d45/f21 [4165500,2157] 0 2024-09-16T00:25:52.675 INFO:tasks.workunit.client.0.smithi112.stdout:8/137: dread d10/d12/f1a [0,4194304] 0 2024-09-16T00:25:52.678 INFO:tasks.workunit.client.0.smithi112.stdout:8/138: creat d10/d12/d24/f2b x:0 0 0 2024-09-16T00:25:52.682 INFO:tasks.workunit.client.0.smithi112.stdout:8/139: write d10/d12/f1a [291264,112724] 0 2024-09-16T00:25:52.687 INFO:tasks.workunit.client.0.smithi112.stdout:8/140: rename d10/l19 to d10/d12/d24/l2c 0 2024-09-16T00:25:52.689 INFO:tasks.workunit.client.0.smithi112.stdout:8/141: truncate d10/d12/d24/f28 527166 0 2024-09-16T00:25:52.692 INFO:tasks.workunit.client.0.smithi112.stdout:8/142: mknod d10/d12/c2d 0 2024-09-16T00:25:52.696 INFO:tasks.workunit.client.0.smithi112.stdout:8/143: creat d10/d12/d24/f2e x:0 0 0 2024-09-16T00:25:52.711 INFO:tasks.workunit.client.0.smithi112.stdout:1/106: write d0/d14/f1d [442681,10516] 0 2024-09-16T00:25:52.714 INFO:tasks.workunit.client.0.smithi112.stdout:1/107: chown d0/d14/fa 8253133 1 2024-09-16T00:25:52.718 INFO:tasks.workunit.client.0.smithi112.stdout:6/140: dwrite dd/df/d11/f1e [0,4194304] 0 2024-09-16T00:25:52.719 INFO:tasks.workunit.client.0.smithi112.stdout:6/141: fdatasync dd/df/d10/f1b 0 2024-09-16T00:25:52.722 INFO:tasks.workunit.client.0.smithi112.stdout:1/108: truncate d0/d14/f8 8771467 0 2024-09-16T00:25:52.725 INFO:tasks.workunit.client.0.smithi112.stdout:8/144: dwrite d10/f23 [0,4194304] 0 2024-09-16T00:25:52.729 INFO:tasks.workunit.client.0.smithi112.stdout:8/145: read - d10/d12/d24/f29 zero size 2024-09-16T00:25:52.737 INFO:tasks.workunit.client.0.smithi112.stdout:1/109: dwrite d0/d14/f1d [0,4194304] 0 2024-09-16T00:25:52.738 INFO:tasks.workunit.client.0.smithi112.stdout:8/146: symlink d10/d12/d24/l2f 0 2024-09-16T00:25:52.739 INFO:tasks.workunit.client.0.smithi112.stdout:8/147: chown d10/d12/d24/l2f 166669 1 2024-09-16T00:25:52.741 INFO:tasks.workunit.client.0.smithi112.stdout:6/142: link dd/d23/l30 dd/l33 0 2024-09-16T00:25:52.744 INFO:tasks.workunit.client.0.smithi112.stdout:8/148: rename c8 to d10/c30 0 2024-09-16T00:25:52.745 INFO:tasks.workunit.client.0.smithi112.stdout:6/143: dread - dd/df/d13/d18/f28 zero size 2024-09-16T00:25:52.745 INFO:tasks.workunit.client.0.smithi112.stdout:8/149: read d10/d12/d24/f28 [59509,111790] 0 2024-09-16T00:25:52.748 INFO:tasks.workunit.client.0.smithi112.stdout:6/144: mkdir dd/d15/d34 0 2024-09-16T00:25:52.751 INFO:tasks.workunit.client.0.smithi112.stdout:4/146: dread d6/db/d15/f17 [0,4194304] 0 2024-09-16T00:25:52.753 INFO:tasks.workunit.client.0.smithi112.stdout:5/219: sync 2024-09-16T00:25:52.754 INFO:tasks.workunit.client.0.smithi112.stdout:2/180: sync 2024-09-16T00:25:52.755 INFO:tasks.workunit.client.0.smithi112.stdout:5/220: fdatasync d4/dd/d12/f11 0 2024-09-16T00:25:52.756 INFO:tasks.workunit.client.0.smithi112.stdout:5/221: chown d4/d7/d26/d36/f3b 356810080 1 2024-09-16T00:25:52.757 INFO:tasks.workunit.client.0.smithi112.stdout:5/222: chown d4/d1c/d43/c4a 125 1 2024-09-16T00:25:52.757 INFO:tasks.workunit.client.0.smithi112.stdout:2/181: mknod d2/dc/d22/c38 0 2024-09-16T00:25:52.758 INFO:tasks.workunit.client.0.smithi112.stdout:6/145: dread dd/df/d13/f1c [0,4194304] 0 2024-09-16T00:25:52.759 INFO:tasks.workunit.client.0.smithi112.stdout:5/223: getdents d4/dd/d49 0 2024-09-16T00:25:52.760 INFO:tasks.workunit.client.0.smithi112.stdout:6/146: readlink dd/df/d10/l14 0 2024-09-16T00:25:52.760 INFO:tasks.workunit.client.0.smithi112.stdout:7/184: dread d6/d10/f12 [0,4194304] 0 2024-09-16T00:25:52.762 INFO:tasks.workunit.client.0.smithi112.stdout:2/182: creat d2/f39 x:0 0 0 2024-09-16T00:25:52.762 INFO:tasks.workunit.client.0.smithi112.stdout:6/147: chown dd/df/d10/l31 1978 1 2024-09-16T00:25:52.762 INFO:tasks.workunit.client.0.smithi112.stdout:5/224: write d4/d7/f40 [1520416,3108] 0 2024-09-16T00:25:52.763 INFO:tasks.workunit.client.0.smithi112.stdout:6/148: fsync dd/df/f21 0 2024-09-16T00:25:52.764 INFO:tasks.workunit.client.0.smithi112.stdout:7/185: mknod d6/d10/c4c 0 2024-09-16T00:25:52.764 INFO:tasks.workunit.client.0.smithi112.stdout:5/225: fsync d4/d7/f4d 0 2024-09-16T00:25:52.765 INFO:tasks.workunit.client.0.smithi112.stdout:6/149: chown dd/df/d17/l2b 118891328 1 2024-09-16T00:25:52.766 INFO:tasks.workunit.client.0.smithi112.stdout:6/150: chown dd/df/d11/f1e 5 1 2024-09-16T00:25:52.768 INFO:tasks.workunit.client.0.smithi112.stdout:5/226: unlink d4/d1c/d43/c4a 0 2024-09-16T00:25:52.770 INFO:tasks.workunit.client.0.smithi112.stdout:7/186: fdatasync d6/d10/d16/f1e 0 2024-09-16T00:25:52.771 INFO:tasks.workunit.client.0.smithi112.stdout:8/150: fsync d10/d12/f1a 0 2024-09-16T00:25:52.771 INFO:tasks.workunit.client.0.smithi112.stdout:6/151: creat dd/d23/f35 x:0 0 0 2024-09-16T00:25:52.774 INFO:tasks.workunit.client.0.smithi112.stdout:7/187: fdatasync d6/d10/d16/f1e 0 2024-09-16T00:25:52.777 INFO:tasks.workunit.client.0.smithi112.stdout:6/152: getdents dd/df/d17/d27 0 2024-09-16T00:25:52.778 INFO:tasks.workunit.client.0.smithi112.stdout:8/151: dread f4 [0,4194304] 0 2024-09-16T00:25:52.782 INFO:tasks.workunit.client.0.smithi112.stdout:3/199: dread d2/d9/f16 [0,4194304] 0 2024-09-16T00:25:52.785 INFO:tasks.workunit.client.0.smithi112.stdout:6/153: rename dd/d23/f2d to dd/df/d10/d1d/f36 0 2024-09-16T00:25:52.785 INFO:tasks.workunit.client.0.smithi112.stdout:3/200: readlink d2/d9/d2b/d33/l41 0 2024-09-16T00:25:52.785 INFO:tasks.workunit.client.0.smithi112.stdout:7/188: dwrite d6/fb [0,4194304] 0 2024-09-16T00:25:52.789 INFO:tasks.workunit.client.0.smithi112.stdout:8/152: write d10/d12/f16 [2506489,74007] 0 2024-09-16T00:25:52.792 INFO:tasks.workunit.client.0.smithi112.stdout:7/189: read d6/d10/d16/f25 [1439330,123606] 0 2024-09-16T00:25:52.793 INFO:tasks.workunit.client.0.smithi112.stdout:3/201: mkdir d2/d4d 0 2024-09-16T00:25:52.793 INFO:tasks.workunit.client.0.smithi112.stdout:3/202: fsync d2/f2d 0 2024-09-16T00:25:52.798 INFO:tasks.workunit.client.0.smithi112.stdout:0/184: dwrite d5/da/dc/fe [4194304,4194304] 0 2024-09-16T00:25:52.801 INFO:tasks.workunit.client.0.smithi112.stdout:8/153: link d10/d12/f1a d10/d12/d1c/f31 0 2024-09-16T00:25:52.803 INFO:tasks.workunit.client.0.smithi112.stdout:0/185: fsync f4 0 2024-09-16T00:25:52.803 INFO:tasks.workunit.client.0.smithi112.stdout:3/203: dwrite d2/f1d [0,4194304] 0 2024-09-16T00:25:52.804 INFO:tasks.workunit.client.0.smithi112.stdout:3/204: chown d2/d9/d2b/d2c/d32/f40 0 1 2024-09-16T00:25:52.805 INFO:tasks.workunit.client.0.smithi112.stdout:3/205: fsync d2/f3f 0 2024-09-16T00:25:52.807 INFO:tasks.workunit.client.0.smithi112.stdout:7/190: creat d6/d10/d16/d49/f4d x:0 0 0 2024-09-16T00:25:52.809 INFO:tasks.workunit.client.0.smithi112.stdout:8/154: unlink f4 0 2024-09-16T00:25:52.814 INFO:tasks.workunit.client.0.smithi112.stdout:3/206: dwrite d2/d9/d2b/d39/f3c [0,4194304] 0 2024-09-16T00:25:52.818 INFO:tasks.workunit.client.0.smithi112.stdout:0/186: mkdir d5/da/d34 0 2024-09-16T00:25:52.818 INFO:tasks.workunit.client.0.smithi112.stdout:9/284: truncate d1/d5/f30 1606815 0 2024-09-16T00:25:52.820 INFO:tasks.workunit.client.0.smithi112.stdout:9/285: dread - d1/d5/d6/d2d/d39/f63 zero size 2024-09-16T00:25:52.822 INFO:tasks.workunit.client.0.smithi112.stdout:7/191: link d6/d10/f12 d6/d10/d16/f4e 0 2024-09-16T00:25:52.823 INFO:tasks.workunit.client.0.smithi112.stdout:7/192: fdatasync d6/f22 0 2024-09-16T00:25:52.823 INFO:tasks.workunit.client.0.smithi112.stdout:8/155: mkdir d10/d32 0 2024-09-16T00:25:52.826 INFO:tasks.workunit.client.0.smithi112.stdout:9/286: rmdir d1/d5/d2c 39 2024-09-16T00:25:52.830 INFO:tasks.workunit.client.0.smithi112.stdout:1/110: write d0/f9 [4172270,66392] 0 2024-09-16T00:25:52.831 INFO:tasks.workunit.client.0.smithi112.stdout:9/287: rename d1/d5/d20/f4c to d1/d5/d6/d2d/d6a/f6e 0 2024-09-16T00:25:52.833 INFO:tasks.workunit.client.0.smithi112.stdout:9/288: truncate d1/d5/d6/d2d/f3b 8676224 0 2024-09-16T00:25:52.837 INFO:tasks.workunit.client.0.smithi112.stdout:7/193: link l3 d6/d10/d16/d49/l4f 0 2024-09-16T00:25:52.838 INFO:tasks.workunit.client.0.smithi112.stdout:4/147: truncate d6/da/dc/d19/d1c/d2c/f29 2743782 0 2024-09-16T00:25:52.839 INFO:tasks.workunit.client.0.smithi112.stdout:4/148: fdatasync d6/f9 0 2024-09-16T00:25:52.839 INFO:tasks.workunit.client.0.smithi112.stdout:9/289: stat d1/c33 0 2024-09-16T00:25:52.840 INFO:tasks.workunit.client.0.smithi112.stdout:0/187: link d5/l9 d5/l35 0 2024-09-16T00:25:52.840 INFO:tasks.workunit.client.0.smithi112.stdout:4/149: dread - d6/da/dc/d19/d1c/f2f zero size 2024-09-16T00:25:52.842 INFO:tasks.workunit.client.0.smithi112.stdout:8/156: creat d10/d12/f33 x:0 0 0 2024-09-16T00:25:52.843 INFO:tasks.workunit.client.0.smithi112.stdout:8/157: dread - d10/d12/d24/f29 zero size 2024-09-16T00:25:52.843 INFO:tasks.workunit.client.0.smithi112.stdout:1/111: mknod d0/d14/df/d24/c25 0 2024-09-16T00:25:52.844 INFO:tasks.workunit.client.0.smithi112.stdout:2/183: getdents d2 0 2024-09-16T00:25:52.848 INFO:tasks.workunit.client.0.smithi112.stdout:9/290: write d1/d5/d20/f32 [216884,89690] 0 2024-09-16T00:25:52.849 INFO:tasks.workunit.client.0.smithi112.stdout:9/291: fdatasync d1/d5/d6/d18/f19 0 2024-09-16T00:25:52.850 INFO:tasks.workunit.client.0.smithi112.stdout:4/150: creat d6/db/f37 x:0 0 0 2024-09-16T00:25:52.851 INFO:tasks.workunit.client.0.smithi112.stdout:9/292: write d1/d5/d20/f5f [518893,93821] 0 2024-09-16T00:25:52.852 INFO:tasks.workunit.client.0.smithi112.stdout:4/151: read d6/db/d15/f22 [455436,47544] 0 2024-09-16T00:25:52.852 INFO:tasks.workunit.client.0.smithi112.stdout:8/158: unlink d10/d12/d1c/l1e 0 2024-09-16T00:25:52.853 INFO:tasks.workunit.client.0.smithi112.stdout:9/293: readlink d1/d5/d6/d2d/d45/d1a/l56 0 2024-09-16T00:25:52.853 INFO:tasks.workunit.client.0.smithi112.stdout:1/112: mkdir d0/d14/d1c/d26 0 2024-09-16T00:25:52.855 INFO:tasks.workunit.client.0.smithi112.stdout:2/184: symlink d2/dc/d22/d2b/l3a 0 2024-09-16T00:25:52.855 INFO:tasks.workunit.client.0.smithi112.stdout:8/159: dread d10/d12/f1a [0,4194304] 0 2024-09-16T00:25:52.856 INFO:tasks.workunit.client.0.smithi112.stdout:0/188: symlink d5/da/d34/l36 0 2024-09-16T00:25:52.861 INFO:tasks.workunit.client.0.smithi112.stdout:2/185: truncate f1 1889823 0 2024-09-16T00:25:52.862 INFO:tasks.workunit.client.0.smithi112.stdout:0/189: mknod d5/da/dc/d12/c37 0 2024-09-16T00:25:52.863 INFO:tasks.workunit.client.0.smithi112.stdout:0/190: fsync d5/da/dc/fe 0 2024-09-16T00:25:52.865 INFO:tasks.workunit.client.0.smithi112.stdout:2/186: dread d2/dc/f27 [0,4194304] 0 2024-09-16T00:25:52.865 INFO:tasks.workunit.client.0.smithi112.stdout:9/294: mkdir d1/d5/d2c/d6f 0 2024-09-16T00:25:52.865 INFO:tasks.workunit.client.0.smithi112.stdout:2/187: stat d2/dc/f1d 0 2024-09-16T00:25:52.866 INFO:tasks.workunit.client.0.smithi112.stdout:7/194: getdents d6/d26 0 2024-09-16T00:25:52.866 INFO:tasks.workunit.client.0.smithi112.stdout:9/295: read - d1/d5/d6/d2d/f3e zero size 2024-09-16T00:25:52.867 INFO:tasks.workunit.client.0.smithi112.stdout:7/195: write d6/d10/d16/f1e [2384504,39840] 0 2024-09-16T00:25:52.869 INFO:tasks.workunit.client.0.smithi112.stdout:6/154: truncate dd/df/d11/f1e 2628788 0 2024-09-16T00:25:52.874 INFO:tasks.workunit.client.0.smithi112.stdout:9/296: dwrite d1/d5/d6/f34 [4194304,4194304] 0 2024-09-16T00:25:52.875 INFO:tasks.workunit.client.0.smithi112.stdout:0/191: mknod d5/da/dc/d12/c38 0 2024-09-16T00:25:52.876 INFO:tasks.workunit.client.0.smithi112.stdout:2/188: mkdir d2/dc/d22/d2b/d3b 0 2024-09-16T00:25:52.876 INFO:tasks.workunit.client.0.smithi112.stdout:0/192: readlink d5/da/d34/l36 0 2024-09-16T00:25:52.881 INFO:tasks.workunit.client.0.smithi112.stdout:6/155: mkdir dd/df/d10/d1d/d37 0 2024-09-16T00:25:52.882 INFO:tasks.workunit.client.0.smithi112.stdout:9/297: dwrite d1/d5/d6/d18/f19 [0,4194304] 0 2024-09-16T00:25:52.883 INFO:tasks.workunit.client.0.smithi112.stdout:8/160: creat d10/f34 x:0 0 0 2024-09-16T00:25:52.888 INFO:tasks.workunit.client.0.smithi112.stdout:2/189: mknod d2/dc/c3c 0 2024-09-16T00:25:52.888 INFO:tasks.workunit.client.0.smithi112.stdout:7/196: dwrite d6/d26/f2d [0,4194304] 0 2024-09-16T00:25:52.889 INFO:tasks.workunit.client.0.smithi112.stdout:7/197: fsync d6/f22 0 2024-09-16T00:25:52.890 INFO:tasks.workunit.client.0.smithi112.stdout:4/152: getdents d6/da/dc/d19/d1c/d2c 0 2024-09-16T00:25:52.893 INFO:tasks.workunit.client.0.smithi112.stdout:9/298: mknod d1/d5/d2c/c70 0 2024-09-16T00:25:52.899 INFO:tasks.workunit.client.0.smithi112.stdout:4/153: write d6/db/f25 [1846722,65328] 0 2024-09-16T00:25:52.901 INFO:tasks.workunit.client.0.smithi112.stdout:2/190: rename d2/c2c to d2/dc/d22/d2b/d3b/c3d 0 2024-09-16T00:25:52.902 INFO:tasks.workunit.client.0.smithi112.stdout:6/156: symlink dd/df/d10/d1d/d37/l38 0 2024-09-16T00:25:52.903 INFO:tasks.workunit.client.0.smithi112.stdout:9/299: symlink d1/d5/d6/d38/l71 0 2024-09-16T00:25:52.905 INFO:tasks.workunit.client.0.smithi112.stdout:4/154: write d6/f1b [2377225,127420] 0 2024-09-16T00:25:52.915 INFO:tasks.workunit.client.0.smithi112.stdout:9/300: dwrite d1/d5/f30 [0,4194304] 0 2024-09-16T00:25:52.917 INFO:tasks.workunit.client.0.smithi112.stdout:1/113: truncate d0/f2 3649915 0 2024-09-16T00:25:52.917 INFO:tasks.workunit.client.0.smithi112.stdout:9/301: dread - d1/f50 zero size 2024-09-16T00:25:52.921 INFO:tasks.workunit.client.0.smithi112.stdout:3/207: unlink d2/d9/f35 0 2024-09-16T00:25:52.922 INFO:tasks.workunit.client.0.smithi112.stdout:0/193: rmdir d5 39 2024-09-16T00:25:52.925 INFO:tasks.workunit.client.0.smithi112.stdout:6/157: dread dd/df/d10/f1b [0,4194304] 0 2024-09-16T00:25:52.926 INFO:tasks.workunit.client.0.smithi112.stdout:6/158: truncate dd/df/d10/d1d/f36 664937 0 2024-09-16T00:25:52.928 INFO:tasks.workunit.client.0.smithi112.stdout:0/194: chown d5/da/dc/fe 146971 1 2024-09-16T00:25:52.929 INFO:tasks.workunit.client.0.smithi112.stdout:9/302: mknod d1/c72 0 2024-09-16T00:25:52.929 INFO:tasks.workunit.client.0.smithi112.stdout:3/208: stat d2/c26 0 2024-09-16T00:25:52.931 INFO:tasks.workunit.client.0.smithi112.stdout:3/209: write d2/d9/d2b/d47/f30 [1330856,60875] 0 2024-09-16T00:25:52.931 INFO:tasks.workunit.client.0.smithi112.stdout:1/114: creat d0/f27 x:0 0 0 2024-09-16T00:25:52.933 INFO:tasks.workunit.client.0.smithi112.stdout:1/115: fsync d0/f9 0 2024-09-16T00:25:52.933 INFO:tasks.workunit.client.0.smithi112.stdout:0/195: mknod d5/da/d18/c39 0 2024-09-16T00:25:52.934 INFO:tasks.workunit.client.0.smithi112.stdout:9/303: symlink d1/d5/d6/d59/l73 0 2024-09-16T00:25:52.937 INFO:tasks.workunit.client.0.smithi112.stdout:6/159: symlink dd/df/l39 0 2024-09-16T00:25:52.938 INFO:tasks.workunit.client.0.smithi112.stdout:0/196: dread d5/f33 [4194304,4194304] 0 2024-09-16T00:25:52.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.939+0000 7fb8f07a1640 1 -- 172.21.15.112:0/992810121 --> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb8e8108200 con 0x7fb8d406e5c0 2024-09-16T00:25:52.942 INFO:tasks.workunit.client.0.smithi112.stdout:7/198: truncate d6/fa 3305148 0 2024-09-16T00:25:52.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.940+0000 7fb8deffd640 1 -- 172.21.15.112:0/992810121 <== mgr.14208 v2:172.21.15.112:6800/2480414866 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7fb8e8108200 con 0x7fb8d406e5c0 2024-09-16T00:25:52.942 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:25:52.942 INFO:teuthology.orchestra.run.smithi112.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80", 2024-09-16T00:25:52.942 INFO:teuthology.orchestra.run.smithi112.stdout: "in_progress": true, 2024-09-16T00:25:52.942 INFO:teuthology.orchestra.run.smithi112.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-09-16T00:25:52.942 INFO:teuthology.orchestra.run.smithi112.stdout: "services_complete": [], 2024-09-16T00:25:52.943 INFO:teuthology.orchestra.run.smithi112.stdout: "progress": "", 2024-09-16T00:25:52.943 INFO:teuthology.orchestra.run.smithi112.stdout: "message": "Currently upgrading mgr daemons", 2024-09-16T00:25:52.943 INFO:teuthology.orchestra.run.smithi112.stdout: "is_paused": false 2024-09-16T00:25:52.943 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:25:52.943 INFO:tasks.workunit.client.0.smithi112.stdout:5/227: sync 2024-09-16T00:25:52.943 INFO:tasks.workunit.client.0.smithi112.stdout:4/155: dread d6/da/dc/d19/d1c/d2c/f29 [0,4194304] 0 2024-09-16T00:25:52.944 INFO:tasks.workunit.client.0.smithi112.stdout:5/228: stat d4/d1c/l3d 0 2024-09-16T00:25:52.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.943+0000 7fb8f07a1640 1 -- 172.21.15.112:0/992810121 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fb8d406e5c0 msgr2=0x7fb8d4070a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:52.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.943+0000 7fb8f07a1640 1 --2- 172.21.15.112:0/992810121 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fb8d406e5c0 0x7fb8d4070a70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb8d8000c00 tx=0x7fb8d8035040 comp rx=0 tx=0).stop 2024-09-16T00:25:52.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.943+0000 7fb8f07a1640 1 -- 172.21.15.112:0/992810121 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8e8102b80 msgr2=0x7fb8e806db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:52.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.943+0000 7fb8f07a1640 1 --2- 172.21.15.112:0/992810121 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8e8102b80 0x7fb8e806db70 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb8e400b790 tx=0x7fb8e40091d0 comp rx=0 tx=0).stop 2024-09-16T00:25:52.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.943+0000 7fb8f07a1640 1 -- 172.21.15.112:0/992810121 shutdown_connections 2024-09-16T00:25:52.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.943+0000 7fb8f07a1640 1 --2- 172.21.15.112:0/992810121 >> [v2:172.21.15.112:6800/2480414866,v1:172.21.15.112:6801/2480414866] conn(0x7fb8d406e5c0 0x7fb8d4070a70 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:52.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.943+0000 7fb8f07a1640 1 --2- 172.21.15.112:0/992810121 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8e8103e50 0x7fb8e806e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:52.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.943+0000 7fb8f07a1640 1 --2- 172.21.15.112:0/992810121 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8e8102b80 0x7fb8e806db70 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:52.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.943+0000 7fb8f07a1640 1 -- 172.21.15.112:0/992810121 >> 172.21.15.112:0/992810121 conn(0x7fb8e8071d90 msgr2=0x7fb8e8073470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:52.945 INFO:tasks.workunit.client.0.smithi112.stdout:5/229: fdatasync d4/f19 0 2024-09-16T00:25:52.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.943+0000 7fb8f07a1640 1 -- 172.21.15.112:0/992810121 shutdown_connections 2024-09-16T00:25:52.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:25:52.943+0000 7fb8f07a1640 1 -- 172.21.15.112:0/992810121 wait complete. 2024-09-16T00:25:52.946 INFO:tasks.workunit.client.0.smithi112.stdout:4/156: stat d6/da/dc/d19/d36 0 2024-09-16T00:25:52.948 INFO:tasks.workunit.client.0.smithi112.stdout:5/230: write d4/d7/d26/f46 [582080,101352] 0 2024-09-16T00:25:52.949 INFO:tasks.workunit.client.0.smithi112.stdout:8/161: chown d10/d12/f33 505671250 1 2024-09-16T00:25:52.951 INFO:tasks.workunit.client.0.smithi112.stdout:8/162: readlink lc 0 2024-09-16T00:25:52.954 INFO:tasks.workunit.client.0.smithi112.stdout:4/157: dwrite d6/db/f25 [0,4194304] 0 2024-09-16T00:25:52.954 INFO:tasks.workunit.client.0.smithi112.stdout:6/160: dread dd/df/f21 [0,4194304] 0 2024-09-16T00:25:52.956 INFO:tasks.workunit.client.0.smithi112.stdout:4/158: write d6/db/f37 [715001,2035] 0 2024-09-16T00:25:52.958 INFO:tasks.workunit.client.0.smithi112.stdout:5/231: mkdir d4/dd/d12/d52 0 2024-09-16T00:25:52.961 INFO:tasks.workunit.client.0.smithi112.stdout:9/304: write d1/d5/d6/d2d/d39/f63 [346224,37571] 0 2024-09-16T00:25:52.962 INFO:tasks.workunit.client.0.smithi112.stdout:7/199: fdatasync d6/fa 0 2024-09-16T00:25:52.963 INFO:tasks.workunit.client.0.smithi112.stdout:9/305: write d1/d5/f16 [2298989,37448] 0 2024-09-16T00:25:52.964 INFO:tasks.workunit.client.0.smithi112.stdout:3/210: creat d2/d9/f4e x:0 0 0 2024-09-16T00:25:52.965 INFO:tasks.workunit.client.0.smithi112.stdout:6/161: creat dd/df/d10/f3a x:0 0 0 2024-09-16T00:25:52.966 INFO:tasks.workunit.client.0.smithi112.stdout:3/211: dread - d2/d9/d2b/f29 zero size 2024-09-16T00:25:52.966 INFO:tasks.workunit.client.0.smithi112.stdout:1/116: chown d0/f2 12265 1 2024-09-16T00:25:52.966 INFO:tasks.workunit.client.0.smithi112.stdout:9/306: write d1/d5/d6/d2d/d39/f63 [341877,106091] 0 2024-09-16T00:25:52.967 INFO:tasks.workunit.client.0.smithi112.stdout:1/117: readlink d0/l1a 0 2024-09-16T00:25:52.968 INFO:tasks.workunit.client.0.smithi112.stdout:6/162: fdatasync dd/df/d13/d18/f28 0 2024-09-16T00:25:52.968 INFO:tasks.workunit.client.0.smithi112.stdout:4/159: mknod d6/da/dc/c38 0 2024-09-16T00:25:52.968 INFO:tasks.workunit.client.0.smithi112.stdout:6/163: readlink l7 0 2024-09-16T00:25:52.968 INFO:tasks.workunit.client.0.smithi112.stdout:1/118: chown d0/d14/fd 811 1 2024-09-16T00:25:52.969 INFO:tasks.workunit.client.0.smithi112.stdout:4/160: chown d6/da/dc/c38 0 1 2024-09-16T00:25:52.969 INFO:tasks.workunit.client.0.smithi112.stdout:6/164: write dd/f2a [3365316,12147] 0 2024-09-16T00:25:52.969 INFO:tasks.workunit.client.0.smithi112.stdout:5/232: rename d4/l2c to d4/dd/d12/l53 0 2024-09-16T00:25:52.970 INFO:tasks.workunit.client.0.smithi112.stdout:8/163: symlink d10/l35 0 2024-09-16T00:25:52.971 INFO:tasks.workunit.client.0.smithi112.stdout:1/119: write d0/d14/f1d [2836,127055] 0 2024-09-16T00:25:52.972 INFO:tasks.workunit.client.0.smithi112.stdout:5/233: stat d4/d1c/l20 0 2024-09-16T00:25:52.972 INFO:tasks.workunit.client.0.smithi112.stdout:0/197: creat d5/f3a x:0 0 0 2024-09-16T00:25:52.973 INFO:tasks.workunit.client.0.smithi112.stdout:2/191: write d2/dc/f27 [1177473,42601] 0 2024-09-16T00:25:52.974 INFO:tasks.workunit.client.0.smithi112.stdout:5/234: write d4/dd/d12/f48 [471460,79575] 0 2024-09-16T00:25:52.975 INFO:tasks.workunit.client.0.smithi112.stdout:5/235: dread - d4/dd/d12/f3a zero size 2024-09-16T00:25:52.977 INFO:tasks.workunit.client.0.smithi112.stdout:3/212: chown d2/f24 18450134 1 2024-09-16T00:25:52.977 INFO:tasks.workunit.client.0.smithi112.stdout:9/307: fdatasync d1/d5/d20/f31 0 2024-09-16T00:25:52.979 INFO:tasks.workunit.client.0.smithi112.stdout:4/161: rename d6/da/dc/d19 to d6/d39 0 2024-09-16T00:25:52.980 INFO:tasks.workunit.client.0.smithi112.stdout:6/165: mknod dd/df/d13/d18/c3b 0 2024-09-16T00:25:52.981 INFO:tasks.workunit.client.0.smithi112.stdout:6/166: chown dd/f2a 0 1 2024-09-16T00:25:52.982 INFO:tasks.workunit.client.0.smithi112.stdout:0/198: creat d5/da/d34/f3b x:0 0 0 2024-09-16T00:25:52.983 INFO:tasks.workunit.client.0.smithi112.stdout:7/200: mknod d6/d10/c50 0 2024-09-16T00:25:52.985 INFO:tasks.workunit.client.0.smithi112.stdout:7/201: read - d6/d10/d46/f33 zero size 2024-09-16T00:25:52.987 INFO:tasks.workunit.client.0.smithi112.stdout:4/162: dwrite d6/db/f25 [0,4194304] 0 2024-09-16T00:25:52.987 INFO:tasks.workunit.client.0.smithi112.stdout:5/236: dread d4/d7/f40 [0,4194304] 0 2024-09-16T00:25:52.988 INFO:tasks.workunit.client.0.smithi112.stdout:5/237: chown f2 110 1 2024-09-16T00:25:52.989 INFO:tasks.workunit.client.0.smithi112.stdout:9/308: mknod d1/d5/d6/d2d/d39/c74 0 2024-09-16T00:25:52.992 INFO:tasks.workunit.client.0.smithi112.stdout:6/167: symlink dd/df/d10/d1d/l3c 0 2024-09-16T00:25:52.992 INFO:tasks.workunit.client.0.smithi112.stdout:0/199: creat d5/da/dc/d12/f3c x:0 0 0 2024-09-16T00:25:52.993 INFO:tasks.workunit.client.0.smithi112.stdout:3/213: write d2/f3a [1753653,90029] 0 2024-09-16T00:25:52.993 INFO:tasks.workunit.client.0.smithi112.stdout:6/168: chown dd/df/d10/d1d/d37/l38 477 1 2024-09-16T00:25:52.993 INFO:tasks.workunit.client.0.smithi112.stdout:7/202: mknod d6/d10/d16/c51 0 2024-09-16T00:25:52.994 INFO:tasks.workunit.client.0.smithi112.stdout:4/163: symlink d6/d39/l3a 0 2024-09-16T00:25:52.995 INFO:tasks.workunit.client.0.smithi112.stdout:3/214: dread - d2/d9/d2b/d2c/d32/f38 zero size 2024-09-16T00:25:52.995 INFO:tasks.workunit.client.0.smithi112.stdout:8/164: symlink d10/d12/d1c/l36 0 2024-09-16T00:25:52.997 INFO:tasks.workunit.client.0.smithi112.stdout:0/200: unlink f4 0 2024-09-16T00:25:53.001 INFO:tasks.workunit.client.0.smithi112.stdout:4/164: dwrite d6/db/f25 [0,4194304] 0 2024-09-16T00:25:53.003 INFO:tasks.workunit.client.0.smithi112.stdout:9/309: link d1/d5/d6/c52 d1/d5/d6/d2d/d45/c75 0 2024-09-16T00:25:53.004 INFO:tasks.workunit.client.0.smithi112.stdout:7/203: creat d6/d10/d1b/f52 x:0 0 0 2024-09-16T00:25:53.006 INFO:tasks.workunit.client.0.smithi112.stdout:6/169: rename dd/df/d13/d18/f25 to dd/df/f3d 0 2024-09-16T00:25:53.011 INFO:tasks.workunit.client.0.smithi112.stdout:0/201: rename d5/f3a to d5/da/d18/f3d 0 2024-09-16T00:25:53.013 INFO:tasks.workunit.client.0.smithi112.stdout:7/204: dwrite d6/d10/d46/f38 [0,4194304] 0 2024-09-16T00:25:53.020 INFO:tasks.workunit.client.0.smithi112.stdout:7/205: creat d6/d10/f53 x:0 0 0 2024-09-16T00:25:53.021 INFO:tasks.workunit.client.0.smithi112.stdout:5/238: truncate d4/d7/d14/f1a 1968234 0 2024-09-16T00:25:53.022 INFO:tasks.workunit.client.0.smithi112.stdout:7/206: mknod d6/d10/d46/c54 0 2024-09-16T00:25:53.023 INFO:tasks.workunit.client.0.smithi112.stdout:5/239: truncate d4/dd/d12/f2f 779962 0 2024-09-16T00:25:53.024 INFO:tasks.workunit.client.0.smithi112.stdout:3/215: write d2/d9/f16 [6127051,46859] 0 2024-09-16T00:25:53.030 INFO:tasks.workunit.client.0.smithi112.stdout:4/165: truncate d6/db/f25 2157362 0 2024-09-16T00:25:53.032 INFO:tasks.workunit.client.0.smithi112.stdout:3/216: dread d2/f19 [0,4194304] 0 2024-09-16T00:25:53.032 INFO:tasks.workunit.client.0.smithi112.stdout:6/170: truncate dd/df/d13/f1c 749475 0 2024-09-16T00:25:53.035 INFO:tasks.workunit.client.0.smithi112.stdout:4/166: symlink d6/d39/d1c/l3b 0 2024-09-16T00:25:53.035 INFO:tasks.workunit.client.0.smithi112.stdout:7/207: dwrite d6/d10/d16/f3e [0,4194304] 0 2024-09-16T00:25:53.037 INFO:tasks.workunit.client.0.smithi112.stdout:3/217: write d2/d9/d2b/d47/f30 [4890552,2482] 0 2024-09-16T00:25:53.038 INFO:tasks.workunit.client.0.smithi112.stdout:9/310: dread d1/d5/d6/d2d/d45/f47 [0,4194304] 0 2024-09-16T00:25:53.042 INFO:tasks.workunit.client.0.smithi112.stdout:5/240: dread d4/dd/f2a [0,4194304] 0 2024-09-16T00:25:53.043 INFO:tasks.workunit.client.0.smithi112.stdout:6/171: symlink dd/df/d13/d18/l3e 0 2024-09-16T00:25:53.044 INFO:tasks.workunit.client.0.smithi112.stdout:4/167: mkdir d6/db/df/d3c 0 2024-09-16T00:25:53.046 INFO:tasks.workunit.client.0.smithi112.stdout:3/218: mknod d2/d9/d2b/d33/c4f 0 2024-09-16T00:25:53.046 INFO:tasks.workunit.client.0.smithi112.stdout:7/208: dread d6/f11 [0,4194304] 0 2024-09-16T00:25:53.047 INFO:tasks.workunit.client.0.smithi112.stdout:6/172: mknod dd/df/d17/c3f 0 2024-09-16T00:25:53.047 INFO:tasks.workunit.client.0.smithi112.stdout:7/209: fsync d6/f2a 0 2024-09-16T00:25:53.048 INFO:tasks.workunit.client.0.smithi112.stdout:9/311: mkdir d1/d76 0 2024-09-16T00:25:53.049 INFO:tasks.workunit.client.0.smithi112.stdout:7/210: dread - d6/d10/d46/f33 zero size 2024-09-16T00:25:53.049 INFO:tasks.workunit.client.0.smithi112.stdout:9/312: dread - d1/d5/d2c/f5b zero size 2024-09-16T00:25:53.051 INFO:tasks.workunit.client.0.smithi112.stdout:5/241: link d4/d7/d26/d36/f3b d4/d1c/d43/f54 0 2024-09-16T00:25:53.053 INFO:tasks.workunit.client.0.smithi112.stdout:6/173: unlink l7 0 2024-09-16T00:25:53.054 INFO:tasks.workunit.client.0.smithi112.stdout:4/168: getdents d6/d39/d36 0 2024-09-16T00:25:53.056 INFO:tasks.workunit.client.0.smithi112.stdout:9/313: mknod d1/d5/d6/d2d/d45/d1a/c77 0 2024-09-16T00:25:53.058 INFO:tasks.workunit.client.0.smithi112.stdout:5/242: symlink d4/d7/l55 0 2024-09-16T00:25:53.060 INFO:tasks.workunit.client.0.smithi112.stdout:7/211: dwrite d6/d10/f15 [0,4194304] 0 2024-09-16T00:25:53.061 INFO:tasks.workunit.client.0.smithi112.stdout:5/243: chown d4/dd/d12/f48 14 1 2024-09-16T00:25:53.062 INFO:tasks.workunit.client.0.smithi112.stdout:6/174: mknod dd/d15/d34/c40 0 2024-09-16T00:25:53.062 INFO:tasks.workunit.client.0.smithi112.stdout:3/219: dwrite d2/d9/f21 [0,4194304] 0 2024-09-16T00:25:53.063 INFO:tasks.workunit.client.0.smithi112.stdout:4/169: mknod d6/db/df/d3c/c3d 0 2024-09-16T00:25:53.064 INFO:tasks.workunit.client.0.smithi112.stdout:4/170: chown d6/d39/d36 20897 1 2024-09-16T00:25:53.064 INFO:tasks.workunit.client.0.smithi112.stdout:5/244: chown d4/d7/d14/c3f 606189 1 2024-09-16T00:25:53.067 INFO:tasks.workunit.client.0.smithi112.stdout:3/220: creat d2/d9/d2b/d33/f50 x:0 0 0 2024-09-16T00:25:53.068 INFO:tasks.workunit.client.0.smithi112.stdout:7/212: write d6/d10/d1b/d39/f43 [2991241,23651] 0 2024-09-16T00:25:53.071 INFO:tasks.workunit.client.0.smithi112.stdout:5/245: mkdir d4/dd/d12/d56 0 2024-09-16T00:25:53.075 INFO:tasks.workunit.client.0.smithi112.stdout:7/213: dread - d6/d10/d1b/f24 zero size 2024-09-16T00:25:53.075 INFO:tasks.workunit.client.0.smithi112.stdout:3/221: truncate d2/d9/f4e 124469 0 2024-09-16T00:25:53.077 INFO:tasks.workunit.client.0.smithi112.stdout:7/214: fdatasync d6/fa 0 2024-09-16T00:25:53.078 INFO:tasks.workunit.client.0.smithi112.stdout:6/175: dwrite dd/df/d10/f1b [4194304,4194304] 0 2024-09-16T00:25:53.078 INFO:tasks.workunit.client.0.smithi112.stdout:4/171: creat d6/db/d15/d31/d34/f3e x:0 0 0 2024-09-16T00:25:53.078 INFO:tasks.workunit.client.0.smithi112.stdout:9/314: dwrite d1/d5/d6/d2d/d6a/f6e [0,4194304] 0 2024-09-16T00:25:53.083 INFO:tasks.workunit.client.0.smithi112.stdout:4/172: write d6/db/d15/d31/d34/f3e [919172,24520] 0 2024-09-16T00:25:53.083 INFO:tasks.workunit.client.0.smithi112.stdout:6/176: write dd/f2a [945267,59006] 0 2024-09-16T00:25:53.083 INFO:tasks.workunit.client.0.smithi112.stdout:7/215: truncate d6/d10/f32 983143 0 2024-09-16T00:25:53.088 INFO:tasks.workunit.client.0.smithi112.stdout:5/246: rename d4/dd/f4e to d4/f57 0 2024-09-16T00:25:53.089 INFO:tasks.workunit.client.0.smithi112.stdout:9/315: creat d1/d5/d2c/d6f/f78 x:0 0 0 2024-09-16T00:25:53.092 INFO:tasks.workunit.client.0.smithi112.stdout:9/316: write d1/d5/d6/f34 [2701839,39299] 0 2024-09-16T00:25:53.095 INFO:tasks.workunit.client.0.smithi112.stdout:6/177: creat dd/df/d17/f41 x:0 0 0 2024-09-16T00:25:53.095 INFO:tasks.workunit.client.0.smithi112.stdout:7/216: dwrite d6/d10/d16/f1e [0,4194304] 0 2024-09-16T00:25:53.097 INFO:tasks.workunit.client.0.smithi112.stdout:9/317: chown d1/d5/d6/d59/l6d 2555 1 2024-09-16T00:25:53.097 INFO:tasks.workunit.client.0.smithi112.stdout:5/247: symlink d4/d7/d14/l58 0 2024-09-16T00:25:53.099 INFO:tasks.workunit.client.0.smithi112.stdout:5/248: read - d4/d7/d26/f44 zero size 2024-09-16T00:25:53.101 INFO:tasks.workunit.client.0.smithi112.stdout:9/318: dread d1/d5/d20/f69 [0,4194304] 0 2024-09-16T00:25:53.103 INFO:tasks.workunit.client.0.smithi112.stdout:6/178: chown dd/df/d11/f1e 60089204 1 2024-09-16T00:25:53.105 INFO:tasks.workunit.client.0.smithi112.stdout:7/217: rename c2 to d6/d10/d16/d3b/c55 0 2024-09-16T00:25:53.107 INFO:tasks.workunit.client.0.smithi112.stdout:5/249: mknod d4/d7/d26/c59 0 2024-09-16T00:25:53.109 INFO:tasks.workunit.client.0.smithi112.stdout:5/250: write f2 [2126435,24210] 0 2024-09-16T00:25:53.111 INFO:tasks.workunit.client.0.smithi112.stdout:6/179: dwrite dd/df/d10/f3a [0,4194304] 0 2024-09-16T00:25:53.112 INFO:tasks.workunit.client.0.smithi112.stdout:4/173: getdents d6/db/df/d3c 0 2024-09-16T00:25:53.113 INFO:tasks.workunit.client.0.smithi112.stdout:9/319: creat d1/d76/f79 x:0 0 0 2024-09-16T00:25:53.116 INFO:tasks.workunit.client.0.smithi112.stdout:6/180: mkdir dd/df/d10/d1d/d37/d42 0 2024-09-16T00:25:53.117 INFO:tasks.workunit.client.0.smithi112.stdout:9/320: fdatasync d1/d5/d6/d2d/d45/f21 0 2024-09-16T00:25:53.118 INFO:tasks.workunit.client.0.smithi112.stdout:6/181: chown dd/df/d10/f1a 479331 1 2024-09-16T00:25:53.119 INFO:tasks.workunit.client.0.smithi112.stdout:6/182: chown dd/df/d13 21439 1 2024-09-16T00:25:53.119 INFO:tasks.workunit.client.0.smithi112.stdout:7/218: getdents d6/d10/d16/d49 0 2024-09-16T00:25:53.120 INFO:tasks.workunit.client.0.smithi112.stdout:9/321: chown d1/d5/d2c/d6f/f78 7323266 1 2024-09-16T00:25:53.123 INFO:tasks.workunit.client.0.smithi112.stdout:7/219: unlink d6/d10/d1b/l41 0 2024-09-16T00:25:53.124 INFO:tasks.workunit.client.0.smithi112.stdout:9/322: dread d1/d5/d6/d38/f4f [0,4194304] 0 2024-09-16T00:25:53.125 INFO:tasks.workunit.client.0.smithi112.stdout:6/183: dread dd/df/d10/d1d/f26 [0,4194304] 0 2024-09-16T00:25:53.125 INFO:tasks.workunit.client.0.smithi112.stdout:9/323: readlink d1/d5/d6/d59/l73 0 2024-09-16T00:25:53.126 INFO:tasks.workunit.client.0.smithi112.stdout:5/251: write d4/d7/d26/d36/f3b [451240,27947] 0 2024-09-16T00:25:53.128 INFO:tasks.workunit.client.0.smithi112.stdout:6/184: truncate dd/df/d17/f41 802766 0 2024-09-16T00:25:53.129 INFO:tasks.workunit.client.0.smithi112.stdout:5/252: unlink d4/dd/f2a 0 2024-09-16T00:25:53.130 INFO:tasks.workunit.client.0.smithi112.stdout:5/253: write d4/dd/d12/f48 [304368,20224] 0 2024-09-16T00:25:53.134 INFO:tasks.workunit.client.0.smithi112.stdout:5/254: creat d4/d7/d26/f5a x:0 0 0 2024-09-16T00:25:53.137 INFO:tasks.workunit.client.0.smithi112.stdout:7/220: truncate d6/d10/f1d 3527362 0 2024-09-16T00:25:53.140 INFO:tasks.workunit.client.0.smithi112.stdout:7/221: chown d6/d10/c1f 214 1 2024-09-16T00:25:53.146 INFO:tasks.workunit.client.0.smithi112.stdout:5/255: dwrite d4/f19 [0,4194304] 0 2024-09-16T00:25:53.146 INFO:tasks.workunit.client.0.smithi112.stdout:7/222: mknod d6/d10/d1b/c56 0 2024-09-16T00:25:53.150 INFO:tasks.workunit.client.0.smithi112.stdout:5/256: symlink d4/d7/d26/d3e/l5b 0 2024-09-16T00:25:53.150 INFO:tasks.workunit.client.0.smithi112.stdout:9/324: dread d1/d5/d6/d2d/f3b [0,4194304] 0 2024-09-16T00:25:53.152 INFO:tasks.workunit.client.0.smithi112.stdout:6/185: dwrite dd/df/d13/f1c [0,4194304] 0 2024-09-16T00:25:53.156 INFO:tasks.workunit.client.0.smithi112.stdout:9/325: chown d1/d5/d6/d2d/d45/c75 206303753 1 2024-09-16T00:25:53.159 INFO:tasks.workunit.client.0.smithi112.stdout:5/257: creat d4/f5c x:0 0 0 2024-09-16T00:25:53.160 INFO:tasks.workunit.client.0.smithi112.stdout:6/186: symlink dd/df/d17/l43 0 2024-09-16T00:25:53.162 INFO:tasks.workunit.client.0.smithi112.stdout:6/187: mkdir dd/df/d11/d44 0 2024-09-16T00:25:53.163 INFO:tasks.workunit.client.0.smithi112.stdout:9/326: rmdir d1/d5/d6/d2d/d6a 39 2024-09-16T00:25:53.166 INFO:tasks.workunit.client.0.smithi112.stdout:9/327: truncate d1/d5/d6/d2d/d39/f3c 4904789 0 2024-09-16T00:25:53.167 INFO:tasks.workunit.client.0.smithi112.stdout:9/328: truncate d1/f41 932032 0 2024-09-16T00:25:53.168 INFO:tasks.workunit.client.0.smithi112.stdout:9/329: symlink d1/d5/d6/d18/d23/l7a 0 2024-09-16T00:25:53.171 INFO:tasks.workunit.client.0.smithi112.stdout:9/330: creat d1/d5/d6/d38/f7b x:0 0 0 2024-09-16T00:25:53.175 INFO:tasks.workunit.client.0.smithi112.stdout:6/188: dread dd/d15/f29 [0,4194304] 0 2024-09-16T00:25:53.175 INFO:tasks.workunit.client.0.smithi112.stdout:9/331: truncate d1/d5/d6/d2d/d39/f63 228991 0 2024-09-16T00:25:53.177 INFO:tasks.workunit.client.0.smithi112.stdout:5/258: dread d4/d7/d14/f1a [0,4194304] 0 2024-09-16T00:25:53.180 INFO:tasks.workunit.client.0.smithi112.stdout:5/259: write d4/d1c/f32 [4064921,18874] 0 2024-09-16T00:25:53.181 INFO:tasks.workunit.client.0.smithi112.stdout:7/223: dread d6/d10/f3d [0,4194304] 0 2024-09-16T00:25:53.183 INFO:tasks.workunit.client.0.smithi112.stdout:7/224: dread - d6/d10/d46/f33 zero size 2024-09-16T00:25:53.184 INFO:tasks.workunit.client.0.smithi112.stdout:5/260: symlink d4/d1c/l5d 0 2024-09-16T00:25:53.184 INFO:tasks.workunit.client.0.smithi112.stdout:6/189: dwrite dd/d23/f35 [0,4194304] 0 2024-09-16T00:25:53.189 INFO:tasks.workunit.client.0.smithi112.stdout:7/225: dread d6/d26/f2d [0,4194304] 0 2024-09-16T00:25:53.189 INFO:tasks.workunit.client.0.smithi112.stdout:6/190: rmdir dd 39 2024-09-16T00:25:53.190 INFO:tasks.workunit.client.0.smithi112.stdout:5/261: dread d4/d7/f40 [0,4194304] 0 2024-09-16T00:25:53.191 INFO:tasks.workunit.client.0.smithi112.stdout:7/226: write d6/f22 [651938,30492] 0 2024-09-16T00:25:53.194 INFO:tasks.workunit.client.0.smithi112.stdout:5/262: symlink d4/d7/d26/l5e 0 2024-09-16T00:25:53.194 INFO:tasks.workunit.client.0.smithi112.stdout:5/263: chown f2 0 1 2024-09-16T00:25:53.196 INFO:tasks.workunit.client.0.smithi112.stdout:6/191: symlink dd/df/d10/d1d/d37/l45 0 2024-09-16T00:25:53.197 INFO:tasks.workunit.client.0.smithi112.stdout:7/227: chown d6/l37 22 1 2024-09-16T00:25:53.198 INFO:tasks.workunit.client.0.smithi112.stdout:5/264: getdents d4/dd/d12/d52 0 2024-09-16T00:25:53.200 INFO:tasks.workunit.client.0.smithi112.stdout:6/192: mknod dd/d23/c46 0 2024-09-16T00:25:53.202 INFO:tasks.workunit.client.0.smithi112.stdout:6/193: write dd/d15/f29 [1283566,32029] 0 2024-09-16T00:25:53.204 INFO:tasks.workunit.client.0.smithi112.stdout:9/332: dwrite d1/d5/d6/d2d/d39/f63 [0,4194304] 0 2024-09-16T00:25:53.206 INFO:tasks.workunit.client.0.smithi112.stdout:7/228: truncate d6/d26/f27 327114 0 2024-09-16T00:25:53.211 INFO:tasks.workunit.client.0.smithi112.stdout:7/229: dread d6/d10/f32 [0,4194304] 0 2024-09-16T00:25:53.211 INFO:tasks.workunit.client.0.smithi112.stdout:5/265: dread d4/d7/f40 [0,4194304] 0 2024-09-16T00:25:53.211 INFO:tasks.workunit.client.0.smithi112.stdout:7/230: stat d6/d26/l28 0 2024-09-16T00:25:53.214 INFO:tasks.workunit.client.0.smithi112.stdout:7/231: symlink d6/d10/d1b/d39/l57 0 2024-09-16T00:25:53.217 INFO:tasks.workunit.client.0.smithi112.stdout:6/194: dread dd/df/d13/f1c [0,4194304] 0 2024-09-16T00:25:53.222 INFO:tasks.workunit.client.0.smithi112.stdout:9/333: dwrite d1/d5/d6/d2d/f3e [0,4194304] 0 2024-09-16T00:25:53.224 INFO:tasks.workunit.client.0.smithi112.stdout:7/232: link d6/d10/d16/l42 d6/d10/d16/d3b/l58 0 2024-09-16T00:25:53.227 INFO:tasks.workunit.client.0.smithi112.stdout:9/334: symlink d1/d5/d6/d59/l7c 0 2024-09-16T00:25:53.228 INFO:tasks.workunit.client.0.smithi112.stdout:9/335: chown d1/d5/d6/d59 181 1 2024-09-16T00:25:53.231 INFO:tasks.workunit.client.0.smithi112.stdout:5/266: dwrite d4/fb [0,4194304] 0 2024-09-16T00:25:53.233 INFO:tasks.workunit.client.0.smithi112.stdout:7/233: getdents d6 0 2024-09-16T00:25:53.233 INFO:tasks.workunit.client.0.smithi112.stdout:5/267: write d4/dd/d12/f2f [1025467,77712] 0 2024-09-16T00:25:53.234 INFO:tasks.workunit.client.0.smithi112.stdout:5/268: chown d4 553884 1 2024-09-16T00:25:53.235 INFO:tasks.workunit.client.0.smithi112.stdout:9/336: creat d1/d5/d6/d2d/d45/d1a/f7d x:0 0 0 2024-09-16T00:25:53.236 INFO:tasks.workunit.client.0.smithi112.stdout:7/234: creat d6/d10/d16/f59 x:0 0 0 2024-09-16T00:25:53.239 INFO:tasks.workunit.client.0.smithi112.stdout:6/195: truncate dd/df/d10/f3a 194248 0 2024-09-16T00:25:53.240 INFO:tasks.workunit.client.0.smithi112.stdout:7/235: dread d6/f11 [0,4194304] 0 2024-09-16T00:25:53.241 INFO:tasks.workunit.client.0.smithi112.stdout:6/196: mkdir dd/df/d10/d47 0 2024-09-16T00:25:53.242 INFO:tasks.workunit.client.0.smithi112.stdout:7/236: write d6/d10/d16/f20 [1441985,25899] 0 2024-09-16T00:25:53.244 INFO:tasks.workunit.client.0.smithi112.stdout:9/337: truncate d1/d5/d6/d2d/d45/f1d 2734577 0 2024-09-16T00:25:53.250 INFO:tasks.workunit.client.0.smithi112.stdout:7/237: dwrite d6/d10/d1b/f24 [0,4194304] 0 2024-09-16T00:25:53.251 INFO:tasks.workunit.client.0.smithi112.stdout:7/238: chown d6/d10/d16/d49/f4d 4055202 1 2024-09-16T00:25:53.255 INFO:tasks.workunit.client.0.smithi112.stdout:7/239: getdents d6/d26 0 2024-09-16T00:25:53.256 INFO:tasks.workunit.client.0.smithi112.stdout:7/240: symlink d6/l5a 0 2024-09-16T00:25:53.257 INFO:tasks.workunit.client.0.smithi112.stdout:7/241: write d6/f2a [758534,23894] 0 2024-09-16T00:25:53.258 INFO:tasks.workunit.client.0.smithi112.stdout:7/242: fsync d6/d10/d46/f38 0 2024-09-16T00:25:53.262 INFO:tasks.workunit.client.0.smithi112.stdout:7/243: creat d6/d10/d46/f5b x:0 0 0 2024-09-16T00:25:53.264 INFO:tasks.workunit.client.0.smithi112.stdout:9/338: dread d1/d5/d6/d18/f4b [0,4194304] 0 2024-09-16T00:25:53.265 INFO:tasks.workunit.client.0.smithi112.stdout:9/339: readlink d1/d5/d6/d2d/d45/l4e 0 2024-09-16T00:25:53.267 INFO:tasks.workunit.client.0.smithi112.stdout:9/340: rename d1/d5/d2c/d6f/f78 to d1/d5/d54/f7e 0 2024-09-16T00:25:53.268 INFO:tasks.workunit.client.0.smithi112.stdout:7/244: truncate d6/f22 281127 0 2024-09-16T00:25:53.269 INFO:tasks.workunit.client.0.smithi112.stdout:9/341: creat d1/d5/d6/d38/f7f x:0 0 0 2024-09-16T00:25:53.273 INFO:tasks.workunit.client.0.smithi112.stdout:9/342: getdents d1/d5/d6 0 2024-09-16T00:25:53.275 INFO:tasks.workunit.client.0.smithi112.stdout:9/343: mknod d1/d5/d20/d25/c80 0 2024-09-16T00:25:53.275 INFO:tasks.workunit.client.0.smithi112.stdout:7/245: dwrite d6/d10/d46/f36 [0,4194304] 0 2024-09-16T00:25:53.276 INFO:tasks.workunit.client.0.smithi112.stdout:9/344: fdatasync d1/d5/d2c/f5b 0 2024-09-16T00:25:53.283 INFO:tasks.workunit.client.0.smithi112.stdout:9/345: rename d1/d5/d6/d2d/d45 to d1/d5/d20/d81 0 2024-09-16T00:25:53.283 INFO:tasks.workunit.client.0.smithi112.stdout:9/346: stat d1/d76/f79 0 2024-09-16T00:25:53.289 INFO:tasks.workunit.client.0.smithi112.stdout:7/246: dwrite d6/d10/f12 [4194304,4194304] 0 2024-09-16T00:25:53.291 INFO:tasks.workunit.client.0.smithi112.stdout:7/247: mkdir d6/d26/d5c 0 2024-09-16T00:25:53.294 INFO:tasks.workunit.client.0.smithi112.stdout:7/248: link d6/d10/d16/d3b/f4b d6/d10/d46/f5d 0 2024-09-16T00:25:53.298 INFO:tasks.workunit.client.0.smithi112.stdout:7/249: dread d6/d10/f12 [4194304,4194304] 0 2024-09-16T00:25:53.299 INFO:tasks.workunit.client.0.smithi112.stdout:7/250: rmdir d6 39 2024-09-16T00:25:53.302 INFO:tasks.workunit.client.0.smithi112.stdout:9/347: dread d1/d5/d6/d18/f40 [0,4194304] 0 2024-09-16T00:25:53.305 INFO:tasks.workunit.client.0.smithi112.stdout:9/348: creat d1/f82 x:0 0 0 2024-09-16T00:25:53.307 INFO:tasks.workunit.client.0.smithi112.stdout:9/349: mkdir d1/d5/d83 0 2024-09-16T00:25:53.314 INFO:tasks.workunit.client.0.smithi112.stdout:7/251: dwrite d6/d10/f32 [0,4194304] 0 2024-09-16T00:25:53.315 INFO:tasks.workunit.client.0.smithi112.stdout:7/252: readlink d6/d10/d16/l21 0 2024-09-16T00:25:53.317 INFO:tasks.workunit.client.0.smithi112.stdout:9/350: dwrite d1/d5/d2c/f57 [0,4194304] 0 2024-09-16T00:25:53.318 INFO:tasks.workunit.client.0.smithi112.stdout:7/253: chown d6/d10/d46/f5b 183 1 2024-09-16T00:25:53.324 INFO:tasks.workunit.client.0.smithi112.stdout:7/254: link d6/d10/f53 d6/d10/d16/d3b/f5e 0 2024-09-16T00:25:53.328 INFO:tasks.workunit.client.0.smithi112.stdout:9/351: dwrite d1/d5/d6/d18/f1f [0,4194304] 0 2024-09-16T00:25:53.328 INFO:tasks.workunit.client.0.smithi112.stdout:7/255: mknod d6/d10/d1b/c5f 0 2024-09-16T00:25:53.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:25:53 smithi112 ceph-mon[27869]: from='client.14588 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:25:53.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:25:53 smithi112 ceph-mon[27869]: from='client.24351 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:25:53.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:25:53 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/2608904056' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:25:53.348 INFO:tasks.workunit.client.0.smithi112.stdout:7/256: creat d6/d10/d1b/f60 x:0 0 0 2024-09-16T00:25:53.349 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:25:53 smithi169 ceph-mon[34653]: from='client.14588 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:25:53.349 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:25:53 smithi169 ceph-mon[34653]: from='client.24351 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:25:53.349 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:25:53 smithi169 ceph-mon[34653]: from='client.? 172.21.15.112:0/2608904056' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:25:53.372 INFO:tasks.workunit.client.0.smithi112.stdout:9/352: rename d1/d5/d6/d59/l73 to d1/l84 0 2024-09-16T00:25:53.373 INFO:tasks.workunit.client.0.smithi112.stdout:9/353: write d1/d5/d6/d2d/d39/f63 [469173,18944] 0 2024-09-16T00:25:53.375 INFO:tasks.workunit.client.0.smithi112.stdout:7/257: rename d6/d10/d16/d49/f4d to d6/d10/d1b/f61 0 2024-09-16T00:25:53.376 INFO:tasks.workunit.client.0.smithi112.stdout:9/354: truncate d1/d5/d6/d2d/d39/f63 4591096 0 2024-09-16T00:25:53.378 INFO:tasks.workunit.client.0.smithi112.stdout:7/258: creat d6/d10/d46/f62 x:0 0 0 2024-09-16T00:25:53.378 INFO:tasks.workunit.client.0.smithi112.stdout:9/355: dread d1/d5/d6/d2d/d39/f63 [4194304,4194304] 0 2024-09-16T00:25:53.379 INFO:tasks.workunit.client.0.smithi112.stdout:7/259: stat d6/d10/d16/f1e 0 2024-09-16T00:25:53.380 INFO:tasks.workunit.client.0.smithi112.stdout:9/356: write d1/d5/d20/d81/d1a/f7d [394628,7977] 0 2024-09-16T00:25:53.384 INFO:tasks.workunit.client.0.smithi112.stdout:7/260: write d6/d10/d46/f5d [233902,109488] 0 2024-09-16T00:25:53.387 INFO:tasks.workunit.client.0.smithi112.stdout:9/357: dwrite d1/d5/d6/d2d/d39/f42 [0,4194304] 0 2024-09-16T00:25:53.389 INFO:tasks.workunit.client.0.smithi112.stdout:9/358: mknod d1/d5/d83/c85 0 2024-09-16T00:25:53.391 INFO:tasks.workunit.client.0.smithi112.stdout:9/359: mknod d1/d76/c86 0 2024-09-16T00:25:53.396 INFO:tasks.workunit.client.0.smithi112.stdout:9/360: link d1/d5/d6/d2d/d39/l68 d1/d5/d54/l87 0 2024-09-16T00:25:53.396 INFO:tasks.workunit.client.0.smithi112.stdout:9/361: fsync d1/d5/d20/f32 0 2024-09-16T00:25:53.397 INFO:tasks.workunit.client.0.smithi112.stdout:9/362: chown d1/d5/d20/f5f 117071 1 2024-09-16T00:25:53.406 INFO:tasks.workunit.client.0.smithi112.stdout:9/363: rename d1/d5/d54 to d1/d5/d20/d81/d1a/d88 0 2024-09-16T00:25:53.411 INFO:tasks.workunit.client.0.smithi112.stdout:9/364: rename d1/d5/d6/d18/d23 to d1/d5/d6/d2d/d6a/d89 0 2024-09-16T00:25:53.414 INFO:tasks.workunit.client.0.smithi112.stdout:9/365: rename d1/d5/d6/d2d/d39/f42 to d1/d5/d20/d25/f8a 0 2024-09-16T00:25:53.421 INFO:tasks.workunit.client.0.smithi112.stdout:9/366: dwrite d1/d5/d6/d18/f40 [0,4194304] 0 2024-09-16T00:25:53.423 INFO:tasks.workunit.client.0.smithi112.stdout:9/367: stat d1/d76 0 2024-09-16T00:25:53.423 INFO:tasks.workunit.client.0.smithi112.stdout:9/368: chown d1/d5/d6/d38 332 1 2024-09-16T00:25:53.423 INFO:tasks.workunit.client.0.smithi112.stdout:9/369: truncate d1/d5/d2c/f5b 680063 0 2024-09-16T00:25:53.426 INFO:tasks.workunit.client.0.smithi112.stdout:9/370: creat d1/d5/f8b x:0 0 0 2024-09-16T00:25:53.429 INFO:tasks.workunit.client.0.smithi112.stdout:9/371: creat d1/d5/f8c x:0 0 0 2024-09-16T00:25:53.430 INFO:tasks.workunit.client.0.smithi112.stdout:9/372: unlink d1/d5/d6/d59/l7c 0 2024-09-16T00:25:53.433 INFO:tasks.workunit.client.0.smithi112.stdout:9/373: link d1/d5/d6/f34 d1/d5/d6/d2d/f8d 0 2024-09-16T00:25:53.435 INFO:tasks.workunit.client.0.smithi112.stdout:9/374: creat d1/d5/d2c/d6f/f8e x:0 0 0 2024-09-16T00:25:53.437 INFO:tasks.workunit.client.0.smithi112.stdout:9/375: truncate d1/d5/d6/d18/f1f 4594155 0 2024-09-16T00:25:53.445 INFO:tasks.workunit.client.0.smithi112.stdout:9/376: truncate d1/d5/d6/d2d/f3b 3686553 0 2024-09-16T00:25:53.445 INFO:tasks.workunit.client.0.smithi112.stdout:9/377: chown d1/d5/d6/d2d/d6a 223768 1 2024-09-16T00:25:53.451 INFO:tasks.workunit.client.0.smithi112.stdout:9/378: dwrite d1/f62 [0,4194304] 0 2024-09-16T00:25:53.457 INFO:tasks.workunit.client.0.smithi112.stdout:9/379: write d1/d5/d6/d2d/d39/f63 [1035428,9276] 0 2024-09-16T00:25:53.465 INFO:tasks.workunit.client.0.smithi112.stdout:9/380: dwrite d1/f82 [0,4194304] 0 2024-09-16T00:25:53.468 INFO:tasks.workunit.client.0.smithi112.stdout:9/381: creat d1/d5/d6/f8f x:0 0 0 2024-09-16T00:25:53.475 INFO:tasks.workunit.client.0.smithi112.stdout:9/382: dwrite d1/d5/f8c [0,4194304] 0 2024-09-16T00:25:53.477 INFO:tasks.workunit.client.0.smithi112.stdout:1/120: sync 2024-09-16T00:25:53.478 INFO:tasks.workunit.client.0.smithi112.stdout:1/121: read d0/f9 [2550468,84804] 0 2024-09-16T00:25:53.482 INFO:tasks.workunit.client.0.smithi112.stdout:1/122: mkdir d0/d14/df/d28 0 2024-09-16T00:25:53.483 INFO:tasks.workunit.client.0.smithi112.stdout:1/123: fdatasync d0/d14/f11 0 2024-09-16T00:25:53.487 INFO:tasks.workunit.client.0.smithi112.stdout:1/124: creat d0/d14/d1c/d26/f29 x:0 0 0 2024-09-16T00:25:53.488 INFO:tasks.workunit.client.0.smithi112.stdout:9/383: creat d1/d5/d6/d2d/d6a/d89/f90 x:0 0 0 2024-09-16T00:25:53.490 INFO:tasks.workunit.client.0.smithi112.stdout:1/125: write d0/d14/f1d [4169677,25908] 0 2024-09-16T00:25:53.490 INFO:tasks.workunit.client.0.smithi112.stdout:9/384: chown d1/d5/d6/d2d/d39/c5d 7 1 2024-09-16T00:25:53.491 INFO:tasks.workunit.client.0.smithi112.stdout:1/126: chown d0/d14/d1c/d20/f21 3 1 2024-09-16T00:25:53.492 INFO:tasks.workunit.client.0.smithi112.stdout:1/127: write d0/d14/f11 [787086,8597] 0 2024-09-16T00:25:53.493 INFO:tasks.workunit.client.0.smithi112.stdout:9/385: dread d1/d5/d2c/f5b [0,4194304] 0 2024-09-16T00:25:53.495 INFO:tasks.workunit.client.0.smithi112.stdout:9/386: creat d1/d5/d2c/d6f/f91 x:0 0 0 2024-09-16T00:25:53.496 INFO:tasks.workunit.client.0.smithi112.stdout:1/128: mknod d0/d14/d1c/d20/c2a 0 2024-09-16T00:25:53.497 INFO:tasks.workunit.client.0.smithi112.stdout:1/129: write d0/d14/df/f15 [1109256,86068] 0 2024-09-16T00:25:53.500 INFO:tasks.workunit.client.0.smithi112.stdout:1/130: write d0/d14/d1c/d20/f21 [8880626,128055] 0 2024-09-16T00:25:53.504 INFO:tasks.workunit.client.0.smithi112.stdout:1/131: dread d0/d14/fa [0,4194304] 0 2024-09-16T00:25:53.505 INFO:tasks.workunit.client.0.smithi112.stdout:9/387: truncate d1/d5/d6/f4a 3766561 0 2024-09-16T00:25:53.506 INFO:tasks.workunit.client.0.smithi112.stdout:1/132: write d0/d14/f1d [1025115,116014] 0 2024-09-16T00:25:53.510 INFO:tasks.workunit.client.0.smithi112.stdout:1/133: rename d0/d14/df/d24/c25 to d0/c2b 0 2024-09-16T00:25:53.510 INFO:tasks.workunit.client.1.smithi169.stdout:0/0: sync 2024-09-16T00:25:53.512 INFO:tasks.workunit.client.1.smithi169.stdout:0/1: mkdir d0 0 2024-09-16T00:25:53.512 INFO:tasks.workunit.client.1.smithi169.stdout:0/2: fdatasync - no filename 2024-09-16T00:25:53.513 INFO:tasks.workunit.client.1.smithi169.stdout:0/3: getdents d0 0 2024-09-16T00:25:53.513 INFO:tasks.workunit.client.1.smithi169.stdout:0/4: write - no filename 2024-09-16T00:25:53.513 INFO:tasks.workunit.client.1.smithi169.stdout:0/5: dwrite - no filename 2024-09-16T00:25:53.513 INFO:tasks.workunit.client.1.smithi169.stdout:0/6: chown d0 1046621 1 2024-09-16T00:25:53.513 INFO:tasks.workunit.client.1.smithi169.stdout:0/7: dwrite - no filename 2024-09-16T00:25:53.513 INFO:tasks.workunit.client.1.smithi169.stdout:0/8: dread - no filename 2024-09-16T00:25:53.513 INFO:tasks.workunit.client.1.smithi169.stdout:0/9: truncate - no filename 2024-09-16T00:25:53.513 INFO:tasks.workunit.client.1.smithi169.stdout:0/10: write - no filename 2024-09-16T00:25:53.514 INFO:tasks.workunit.client.0.smithi112.stdout:9/388: rename d1/d5/d20/d81/f47 to d1/d5/d6/d2d/f92 0 2024-09-16T00:25:53.515 INFO:tasks.workunit.client.1.smithi169.stdout:1/22: sync 2024-09-16T00:25:53.515 INFO:tasks.workunit.client.1.smithi169.stdout:3/10: sync 2024-09-16T00:25:53.517 INFO:tasks.workunit.client.1.smithi169.stdout:3/11: write f5 [89906,7751] 0 2024-09-16T00:25:53.517 INFO:tasks.workunit.client.0.smithi112.stdout:1/134: dwrite d0/d14/f22 [0,4194304] 0 2024-09-16T00:25:53.518 INFO:tasks.workunit.client.0.smithi112.stdout:9/389: creat d1/d5/d20/f93 x:0 0 0 2024-09-16T00:25:53.519 INFO:tasks.workunit.client.0.smithi112.stdout:1/135: write d0/f9 [5286955,99689] 0 2024-09-16T00:25:53.519 INFO:tasks.workunit.client.1.smithi169.stdout:1/23: dread f4 [0,4194304] 0 2024-09-16T00:25:53.523 INFO:tasks.workunit.client.0.smithi112.stdout:9/390: rename d1/d5/d6/d2d/d6a/d89/c2a to d1/d5/d6/d2d/d39/c94 0 2024-09-16T00:25:53.526 INFO:tasks.workunit.client.0.smithi112.stdout:1/136: link d0/d14/df/l10 d0/d14/d1c/l2c 0 2024-09-16T00:25:53.526 INFO:tasks.workunit.client.0.smithi112.stdout:9/391: mknod d1/d5/d6/d59/c95 0 2024-09-16T00:25:53.528 INFO:tasks.workunit.client.0.smithi112.stdout:1/137: creat d0/d14/df/d28/f2d x:0 0 0 2024-09-16T00:25:53.531 INFO:tasks.workunit.client.0.smithi112.stdout:1/138: creat d0/d14/df/d28/f2e x:0 0 0 2024-09-16T00:25:53.532 INFO:tasks.workunit.client.0.smithi112.stdout:1/139: rmdir d0/d14/d1c 39 2024-09-16T00:25:53.535 INFO:tasks.workunit.client.0.smithi112.stdout:1/140: mkdir d0/d14/df/d2f 0 2024-09-16T00:25:53.536 INFO:tasks.workunit.client.0.smithi112.stdout:1/141: fsync d0/d14/d1c/d20/f21 0 2024-09-16T00:25:53.537 INFO:tasks.workunit.client.0.smithi112.stdout:9/392: write d1/d5/d6/d38/f4f [662637,77150] 0 2024-09-16T00:25:53.537 INFO:tasks.workunit.client.0.smithi112.stdout:1/142: fsync d0/d14/f23 0 2024-09-16T00:25:53.538 INFO:tasks.workunit.client.0.smithi112.stdout:9/393: chown d1/d5/d20/d25/f8a 43695475 1 2024-09-16T00:25:53.539 INFO:tasks.workunit.client.0.smithi112.stdout:1/143: chown d0/c2b 15538 1 2024-09-16T00:25:53.541 INFO:tasks.workunit.client.0.smithi112.stdout:9/394: unlink d1/c48 0 2024-09-16T00:25:53.541 INFO:tasks.workunit.client.0.smithi112.stdout:1/144: write d0/d14/fa [934612,92393] 0 2024-09-16T00:25:53.542 INFO:tasks.workunit.client.0.smithi112.stdout:1/145: chown d0/d14/df/d2f 2 1 2024-09-16T00:25:53.543 INFO:tasks.workunit.client.0.smithi112.stdout:1/146: truncate d0/d14/fa 1497913 0 2024-09-16T00:25:53.545 INFO:tasks.workunit.client.0.smithi112.stdout:9/395: link d1/d5/d6/d18/f28 d1/d5/d6/d2d/d39/f96 0 2024-09-16T00:25:53.546 INFO:tasks.workunit.client.0.smithi112.stdout:1/147: mkdir d0/d14/df/d2f/d30 0 2024-09-16T00:25:53.548 INFO:tasks.workunit.client.0.smithi112.stdout:1/148: creat d0/d14/d1c/d20/f31 x:0 0 0 2024-09-16T00:25:53.549 INFO:tasks.workunit.client.0.smithi112.stdout:1/149: stat d0/d14/d1c/c1f 0 2024-09-16T00:25:53.550 INFO:tasks.workunit.client.0.smithi112.stdout:1/150: write d0/f1b [2525531,22085] 0 2024-09-16T00:25:53.551 INFO:tasks.workunit.client.0.smithi112.stdout:1/151: read d0/d14/f1d [2088281,629] 0 2024-09-16T00:25:53.559 INFO:tasks.workunit.client.0.smithi112.stdout:1/152: dwrite d0/d14/d1c/d26/f29 [0,4194304] 0 2024-09-16T00:25:53.561 INFO:tasks.workunit.client.0.smithi112.stdout:9/396: dwrite d1/d5/d20/d81/f21 [0,4194304] 0 2024-09-16T00:25:53.565 INFO:tasks.workunit.client.0.smithi112.stdout:1/153: truncate d0/f2 795048 0 2024-09-16T00:25:53.569 INFO:tasks.workunit.client.0.smithi112.stdout:9/397: creat d1/d5/d6/d2d/f97 x:0 0 0 2024-09-16T00:25:53.569 INFO:tasks.workunit.client.0.smithi112.stdout:1/154: creat d0/d14/df/d28/f32 x:0 0 0 2024-09-16T00:25:53.569 INFO:tasks.workunit.client.0.smithi112.stdout:9/398: chown d1/d5/d6 26420653 1 2024-09-16T00:25:53.570 INFO:tasks.workunit.client.0.smithi112.stdout:1/155: stat d0/d14/df/d28/f2e 0 2024-09-16T00:25:53.576 INFO:tasks.workunit.client.0.smithi112.stdout:9/399: unlink d1/d5/d20/f5f 0 2024-09-16T00:25:53.577 INFO:tasks.workunit.client.0.smithi112.stdout:9/400: fsync d1/d5/f16 0 2024-09-16T00:25:53.577 INFO:tasks.workunit.client.0.smithi112.stdout:9/401: readlink d1/l84 0 2024-09-16T00:25:53.578 INFO:tasks.workunit.client.0.smithi112.stdout:1/156: dread d0/d14/df/f15 [0,4194304] 0 2024-09-16T00:25:53.578 INFO:tasks.workunit.client.0.smithi112.stdout:9/402: creat d1/d5/d20/d81/f98 x:0 0 0 2024-09-16T00:25:53.580 INFO:tasks.workunit.client.0.smithi112.stdout:1/157: dread d0/f9 [0,4194304] 0 2024-09-16T00:25:53.581 INFO:tasks.workunit.client.0.smithi112.stdout:8/165: sync 2024-09-16T00:25:53.582 INFO:tasks.workunit.client.0.smithi112.stdout:2/192: sync 2024-09-16T00:25:53.582 INFO:tasks.workunit.client.0.smithi112.stdout:2/193: readlink d2/dc/l1c 0 2024-09-16T00:25:53.582 INFO:tasks.workunit.client.0.smithi112.stdout:8/166: chown d10/f34 41548 1 2024-09-16T00:25:53.583 INFO:tasks.workunit.client.0.smithi112.stdout:1/158: rmdir d0/d14/df/d2f 39 2024-09-16T00:25:53.583 INFO:tasks.workunit.client.0.smithi112.stdout:1/159: dread - d0/f27 zero size 2024-09-16T00:25:53.585 INFO:tasks.workunit.client.0.smithi112.stdout:1/160: read d0/d14/fd [551387,28913] 0 2024-09-16T00:25:53.593 INFO:tasks.workunit.client.0.smithi112.stdout:2/194: write d2/dd/f1a [225147,114434] 0 2024-09-16T00:25:53.594 INFO:tasks.workunit.client.0.smithi112.stdout:9/403: dread d1/d5/d6/d2d/d39/f3c [0,4194304] 0 2024-09-16T00:25:53.597 INFO:tasks.workunit.client.0.smithi112.stdout:2/195: symlink d2/dc/d22/d2b/l3e 0 2024-09-16T00:25:53.597 INFO:tasks.workunit.client.0.smithi112.stdout:1/161: truncate d0/f2 1146436 0 2024-09-16T00:25:53.598 INFO:tasks.workunit.client.0.smithi112.stdout:2/196: chown d2/f39 5 1 2024-09-16T00:25:53.600 INFO:tasks.workunit.client.0.smithi112.stdout:9/404: link d1/d5/l5e d1/d5/d6/d59/d64/l99 0 2024-09-16T00:25:53.602 INFO:tasks.workunit.client.0.smithi112.stdout:9/405: creat d1/d5/d6/d18/f9a x:0 0 0 2024-09-16T00:25:53.604 INFO:tasks.workunit.client.0.smithi112.stdout:2/197: dwrite d2/dd/f1a [0,4194304] 0 2024-09-16T00:25:53.605 INFO:tasks.workunit.client.0.smithi112.stdout:9/406: mknod d1/d5/d6/d2d/c9b 0 2024-09-16T00:25:53.606 INFO:tasks.workunit.client.0.smithi112.stdout:2/198: fdatasync d2/f8 0 2024-09-16T00:25:53.608 INFO:tasks.workunit.client.0.smithi112.stdout:2/199: read f1 [1742331,102136] 0 2024-09-16T00:25:53.608 INFO:tasks.workunit.client.0.smithi112.stdout:2/200: fdatasync d2/dd/f15 0 2024-09-16T00:25:53.612 INFO:tasks.workunit.client.0.smithi112.stdout:2/201: dread d2/f19 [0,4194304] 0 2024-09-16T00:25:53.617 INFO:tasks.workunit.client.0.smithi112.stdout:2/202: write d2/dc/f2a [818482,81901] 0 2024-09-16T00:25:53.617 INFO:tasks.workunit.client.0.smithi112.stdout:2/203: write d2/dc/f2a [173119,26089] 0 2024-09-16T00:25:53.618 INFO:tasks.workunit.client.0.smithi112.stdout:9/407: truncate d1/f82 3391679 0 2024-09-16T00:25:53.619 INFO:tasks.workunit.client.0.smithi112.stdout:2/204: mknod d2/dc/d22/d2b/d30/c3f 0 2024-09-16T00:25:53.621 INFO:tasks.workunit.client.0.smithi112.stdout:9/408: creat d1/d5/d20/d81/d1a/f9c x:0 0 0 2024-09-16T00:25:53.621 INFO:tasks.workunit.client.0.smithi112.stdout:2/205: rename d2/dc to d2/dc/d22/d2b/d3b/d40 22 2024-09-16T00:25:53.621 INFO:tasks.workunit.client.0.smithi112.stdout:9/409: chown d1/d5/d20/l58 70571 1 2024-09-16T00:25:53.623 INFO:tasks.workunit.client.0.smithi112.stdout:9/410: chown d1/d5/d20/d25/l53 441354 1 2024-09-16T00:25:53.626 INFO:tasks.workunit.client.0.smithi112.stdout:9/411: link d1/d5/d6/f8f d1/d5/d20/d25/d66/f9d 0 2024-09-16T00:25:53.628 INFO:tasks.workunit.client.0.smithi112.stdout:9/412: mknod d1/d5/d6/d2d/d6a/c9e 0 2024-09-16T00:25:53.631 INFO:tasks.workunit.client.0.smithi112.stdout:9/413: write d1/d5/d6/d18/f4b [931460,4401] 0 2024-09-16T00:25:53.632 INFO:tasks.workunit.client.0.smithi112.stdout:9/414: fsync d1/d5/d20/f32 0 2024-09-16T00:25:53.635 INFO:tasks.workunit.client.0.smithi112.stdout:9/415: unlink d1/d5/d20/d81/l4e 0 2024-09-16T00:25:53.638 INFO:tasks.workunit.client.0.smithi112.stdout:2/206: dread d2/f14 [0,4194304] 0 2024-09-16T00:25:53.638 INFO:tasks.workunit.client.0.smithi112.stdout:9/416: dread d1/d5/d20/f69 [0,4194304] 0 2024-09-16T00:25:53.639 INFO:tasks.workunit.client.0.smithi112.stdout:9/417: chown d1/d5/d20/d81/c24 3771 1 2024-09-16T00:25:53.641 INFO:tasks.workunit.client.0.smithi112.stdout:2/207: rename d2/dc/d22/d2b/l3e to d2/dd/l41 0 2024-09-16T00:25:53.642 INFO:tasks.workunit.client.0.smithi112.stdout:9/418: mknod d1/d76/c9f 0 2024-09-16T00:25:53.642 INFO:tasks.workunit.client.0.smithi112.stdout:2/208: fsync d2/dc/f24 0 2024-09-16T00:25:53.642 INFO:tasks.workunit.client.0.smithi112.stdout:2/209: readlink d2/lb 0 2024-09-16T00:25:53.646 INFO:tasks.workunit.client.0.smithi112.stdout:2/210: dread d2/dc/f11 [0,4194304] 0 2024-09-16T00:25:53.647 INFO:tasks.workunit.client.0.smithi112.stdout:2/211: chown d2/dd/f1a 1124118 1 2024-09-16T00:25:53.647 INFO:tasks.workunit.client.0.smithi112.stdout:2/212: dread d2/dc/f2d [0,4194304] 0 2024-09-16T00:25:53.649 INFO:tasks.workunit.client.1.smithi169.stdout:3/12: truncate f3 923782 0 2024-09-16T00:25:53.650 INFO:tasks.workunit.client.0.smithi112.stdout:2/213: symlink d2/dc/d22/d2b/d30/l42 0 2024-09-16T00:25:53.651 INFO:tasks.workunit.client.1.smithi169.stdout:3/13: symlink l6 0 2024-09-16T00:25:53.655 INFO:tasks.workunit.client.0.smithi112.stdout:2/214: dwrite d2/fa [0,4194304] 0 2024-09-16T00:25:53.658 INFO:tasks.workunit.client.0.smithi112.stdout:2/215: creat d2/dc/d22/d2b/f43 x:0 0 0 2024-09-16T00:25:53.658 INFO:tasks.workunit.client.1.smithi169.stdout:3/14: write f3 [732288,115906] 0 2024-09-16T00:25:53.659 INFO:tasks.workunit.client.0.smithi112.stdout:2/216: creat d2/dc/d22/d2b/d30/f44 x:0 0 0 2024-09-16T00:25:53.660 INFO:tasks.workunit.client.1.smithi169.stdout:3/15: mkdir d7 0 2024-09-16T00:25:53.661 INFO:tasks.workunit.client.0.smithi112.stdout:2/217: mknod d2/dd/c45 0 2024-09-16T00:25:53.662 INFO:tasks.workunit.client.1.smithi169.stdout:1/24: dwrite f1 [0,4194304] 0 2024-09-16T00:25:53.662 INFO:tasks.workunit.client.1.smithi169.stdout:3/16: dread - f1 zero size 2024-09-16T00:25:53.663 INFO:tasks.workunit.client.1.smithi169.stdout:3/17: dread - f1 zero size 2024-09-16T00:25:53.664 INFO:tasks.workunit.client.1.smithi169.stdout:3/18: chown f1 16469898 1 2024-09-16T00:25:53.664 INFO:tasks.workunit.client.1.smithi169.stdout:3/19: chown l4 28174 1 2024-09-16T00:25:53.666 INFO:tasks.workunit.client.1.smithi169.stdout:3/20: rename f5 to d7/f8 0 2024-09-16T00:25:53.667 INFO:tasks.workunit.client.0.smithi112.stdout:2/218: dwrite d2/fa [0,4194304] 0 2024-09-16T00:25:53.668 INFO:tasks.workunit.client.1.smithi169.stdout:3/21: chown f1 0 1 2024-09-16T00:25:53.670 INFO:tasks.workunit.client.0.smithi112.stdout:2/219: readlink d2/l6 0 2024-09-16T00:25:53.670 INFO:tasks.workunit.client.1.smithi169.stdout:1/25: dwrite f0 [0,4194304] 0 2024-09-16T00:25:53.671 INFO:tasks.workunit.client.0.smithi112.stdout:2/220: fdatasync d2/dd/f15 0 2024-09-16T00:25:53.671 INFO:tasks.workunit.client.0.smithi112.stdout:2/221: dread - d2/dc/f1d zero size 2024-09-16T00:25:53.672 INFO:tasks.workunit.client.1.smithi169.stdout:1/26: write f1 [1394748,46115] 0 2024-09-16T00:25:53.672 INFO:tasks.workunit.client.0.smithi112.stdout:2/222: dread - d2/dc/f1d zero size 2024-09-16T00:25:53.676 INFO:tasks.workunit.client.0.smithi112.stdout:2/223: creat d2/f46 x:0 0 0 2024-09-16T00:25:53.680 INFO:tasks.workunit.client.0.smithi112.stdout:2/224: write d2/dc/f11 [1676197,38401] 0 2024-09-16T00:25:53.683 INFO:tasks.workunit.client.0.smithi112.stdout:2/225: mknod d2/dc/d22/d2b/d30/c47 0 2024-09-16T00:25:53.683 INFO:tasks.workunit.client.1.smithi169.stdout:3/22: rename f1 to d7/f9 0 2024-09-16T00:25:53.686 INFO:tasks.workunit.client.1.smithi169.stdout:3/23: mknod d7/ca 0 2024-09-16T00:25:53.690 INFO:tasks.workunit.client.1.smithi169.stdout:1/27: getdents . 0 2024-09-16T00:25:53.691 INFO:tasks.workunit.client.1.smithi169.stdout:1/28: write f0 [433365,122173] 0 2024-09-16T00:25:53.705 INFO:tasks.workunit.client.1.smithi169.stdout:1/29: fdatasync f0 0 2024-09-16T00:25:53.706 INFO:tasks.workunit.client.1.smithi169.stdout:3/24: rename d7/f8 to d7/fb 0 2024-09-16T00:25:53.707 INFO:tasks.workunit.client.1.smithi169.stdout:3/25: mkdir d7/dc 0 2024-09-16T00:25:53.710 INFO:tasks.workunit.client.1.smithi169.stdout:1/30: dwrite f0 [0,4194304] 0 2024-09-16T00:25:53.712 INFO:tasks.workunit.client.0.smithi112.stdout:2/226: rename d2/dc to d2/dd/d48 0 2024-09-16T00:25:53.714 INFO:tasks.workunit.client.0.smithi112.stdout:2/227: mkdir d2/dd/d48/d22/d2b/d3b/d49 0 2024-09-16T00:25:53.714 INFO:tasks.workunit.client.0.smithi112.stdout:2/228: chown d2/dd/d48/d22/d2b 1062626002 1 2024-09-16T00:25:53.716 INFO:tasks.workunit.client.0.smithi112.stdout:2/229: mknod d2/dd/d48/c4a 0 2024-09-16T00:25:53.717 INFO:tasks.workunit.client.1.smithi169.stdout:1/31: dwrite f4 [0,4194304] 0 2024-09-16T00:25:53.717 INFO:tasks.workunit.client.0.smithi112.stdout:2/230: stat d2/dd/d48/c18 0 2024-09-16T00:25:53.718 INFO:tasks.workunit.client.1.smithi169.stdout:1/32: write f0 [2324333,85772] 0 2024-09-16T00:25:53.718 INFO:tasks.workunit.client.0.smithi112.stdout:2/231: chown d2/dd/c45 1835156730 1 2024-09-16T00:25:53.720 INFO:tasks.workunit.client.0.smithi112.stdout:2/232: stat d2/lb 0 2024-09-16T00:25:53.721 INFO:tasks.workunit.client.1.smithi169.stdout:1/33: write f4 [4568647,130347] 0 2024-09-16T00:25:53.722 INFO:tasks.workunit.client.0.smithi112.stdout:2/233: read d2/dd/d48/f2a [284170,106763] 0 2024-09-16T00:25:53.723 INFO:tasks.workunit.client.0.smithi112.stdout:2/234: readlink d2/dd/d48/d22/d2b/d30/l42 0 2024-09-16T00:25:53.724 INFO:tasks.workunit.client.1.smithi169.stdout:1/34: rename f1 to f5 0 2024-09-16T00:25:53.730 INFO:tasks.workunit.client.0.smithi112.stdout:2/235: dwrite d2/f8 [0,4194304] 0 2024-09-16T00:25:53.731 INFO:tasks.workunit.client.1.smithi169.stdout:1/35: dwrite f4 [0,4194304] 0 2024-09-16T00:25:53.732 INFO:tasks.workunit.client.0.smithi112.stdout:2/236: symlink d2/dd/d48/l4b 0 2024-09-16T00:25:53.733 INFO:tasks.workunit.client.1.smithi169.stdout:1/36: mknod c6 0 2024-09-16T00:25:53.734 INFO:tasks.workunit.client.0.smithi112.stdout:2/237: dread - d2/dd/d48/d22/d2b/d30/f44 zero size 2024-09-16T00:25:53.735 INFO:tasks.workunit.client.1.smithi169.stdout:1/37: creat f7 x:0 0 0 2024-09-16T00:25:53.737 INFO:tasks.workunit.client.1.smithi169.stdout:1/38: link f0 f8 0 2024-09-16T00:25:53.738 INFO:tasks.workunit.client.0.smithi112.stdout:2/238: truncate d2/dd/f15 557792 0 2024-09-16T00:25:53.740 INFO:tasks.workunit.client.1.smithi169.stdout:1/39: write f0 [4209753,98005] 0 2024-09-16T00:25:53.741 INFO:tasks.workunit.client.1.smithi169.stdout:1/40: write f4 [2298269,130218] 0 2024-09-16T00:25:53.741 INFO:tasks.workunit.client.1.smithi169.stdout:1/41: readlink - no filename 2024-09-16T00:25:53.743 INFO:tasks.workunit.client.0.smithi112.stdout:2/239: symlink d2/dd/d48/d22/d2b/d3b/d49/l4c 0 2024-09-16T00:25:53.745 INFO:tasks.workunit.client.0.smithi112.stdout:2/240: write d2/f46 [853944,12759] 0 2024-09-16T00:25:53.745 INFO:tasks.workunit.client.1.smithi169.stdout:1/42: dread f0 [0,4194304] 0 2024-09-16T00:25:53.746 INFO:tasks.workunit.client.0.smithi112.stdout:2/241: read d2/dd/d48/f24 [1470376,47096] 0 2024-09-16T00:25:53.748 INFO:tasks.workunit.client.1.smithi169.stdout:1/43: dread f0 [0,4194304] 0 2024-09-16T00:25:53.749 INFO:tasks.workunit.client.1.smithi169.stdout:1/44: chown f4 1059009540 1 2024-09-16T00:25:53.750 INFO:tasks.workunit.client.1.smithi169.stdout:1/45: write f5 [3534092,44199] 0 2024-09-16T00:25:53.753 INFO:tasks.workunit.client.0.smithi112.stdout:2/242: dwrite d2/dd/d48/f23 [0,4194304] 0 2024-09-16T00:25:53.754 INFO:tasks.workunit.client.0.smithi112.stdout:2/243: readlink d2/dd/d48/l1e 0 2024-09-16T00:25:53.754 INFO:tasks.workunit.client.1.smithi169.stdout:1/46: link f7 f9 0 2024-09-16T00:25:53.756 INFO:tasks.workunit.client.1.smithi169.stdout:1/47: write f7 [54639,56585] 0 2024-09-16T00:25:53.756 INFO:tasks.workunit.client.0.smithi112.stdout:0/202: sync 2024-09-16T00:25:53.757 INFO:tasks.workunit.client.0.smithi112.stdout:0/203: read - d5/f2a zero size 2024-09-16T00:25:53.758 INFO:tasks.workunit.client.1.smithi169.stdout:1/48: symlink la 0 2024-09-16T00:25:53.760 INFO:tasks.workunit.client.1.smithi169.stdout:1/49: creat fb x:0 0 0 2024-09-16T00:25:53.760 INFO:tasks.workunit.client.0.smithi112.stdout:0/204: mknod d5/da/dc/d12/c3e 0 2024-09-16T00:25:53.761 INFO:tasks.workunit.client.0.smithi112.stdout:0/205: chown d5 307943 1 2024-09-16T00:25:53.762 INFO:tasks.workunit.client.0.smithi112.stdout:0/206: write d5/da/f2e [3538872,125941] 0 2024-09-16T00:25:53.764 INFO:tasks.workunit.client.0.smithi112.stdout:0/207: creat d5/da/dc/d12/f3f x:0 0 0 2024-09-16T00:25:53.768 INFO:tasks.workunit.client.1.smithi169.stdout:1/50: dwrite f0 [0,4194304] 0 2024-09-16T00:25:53.768 INFO:tasks.workunit.client.0.smithi112.stdout:0/208: unlink d5/l1b 0 2024-09-16T00:25:53.768 INFO:tasks.workunit.client.1.smithi169.stdout:1/51: dread - fb zero size 2024-09-16T00:25:53.770 INFO:tasks.workunit.client.0.smithi112.stdout:0/209: mknod d5/da/dc/c40 0 2024-09-16T00:25:53.770 INFO:tasks.workunit.client.1.smithi169.stdout:1/52: mkdir dc 0 2024-09-16T00:25:53.772 INFO:tasks.workunit.client.1.smithi169.stdout:1/53: rename f5 to dc/fd 0 2024-09-16T00:25:53.772 INFO:tasks.workunit.client.0.smithi112.stdout:0/210: creat d5/da/d18/d24/f41 x:0 0 0 2024-09-16T00:25:53.773 INFO:tasks.workunit.client.0.smithi112.stdout:0/211: chown f0 1258186 1 2024-09-16T00:25:53.773 INFO:tasks.workunit.client.1.smithi169.stdout:1/54: symlink dc/le 0 2024-09-16T00:25:53.773 INFO:tasks.workunit.client.1.smithi169.stdout:1/55: chown dc 2 1 2024-09-16T00:25:53.774 INFO:tasks.workunit.client.0.smithi112.stdout:2/244: dread d2/dd/f15 [0,4194304] 0 2024-09-16T00:25:53.774 INFO:tasks.workunit.client.0.smithi112.stdout:0/212: write d5/da/dc/d12/f3c [843272,106659] 0 2024-09-16T00:25:53.774 INFO:tasks.workunit.client.1.smithi169.stdout:1/56: mkdir dc/df 0 2024-09-16T00:25:53.777 INFO:tasks.workunit.client.1.smithi169.stdout:1/57: dread f7 [0,4194304] 0 2024-09-16T00:25:53.777 INFO:tasks.workunit.client.1.smithi169.stdout:1/58: chown f7 357096572 1 2024-09-16T00:25:53.779 INFO:tasks.workunit.client.1.smithi169.stdout:1/59: write dc/fd [4416122,98514] 0 2024-09-16T00:25:53.781 INFO:tasks.workunit.client.0.smithi112.stdout:2/245: dread d2/dd/f1a [0,4194304] 0 2024-09-16T00:25:53.782 INFO:tasks.workunit.client.0.smithi112.stdout:0/213: dwrite d5/da/d34/f3b [0,4194304] 0 2024-09-16T00:25:53.783 INFO:tasks.workunit.client.0.smithi112.stdout:2/246: fsync d2/dd/d48/d22/d2b/d30/f36 0 2024-09-16T00:25:53.790 INFO:tasks.workunit.client.0.smithi112.stdout:0/214: dwrite d5/da/d18/f1f [4194304,4194304] 0 2024-09-16T00:25:53.791 INFO:tasks.workunit.client.1.smithi169.stdout:1/60: truncate f0 3461935 0 2024-09-16T00:25:53.792 INFO:tasks.workunit.client.0.smithi112.stdout:2/247: dwrite d2/dd/d48/d22/d2b/f34 [0,4194304] 0 2024-09-16T00:25:53.794 INFO:tasks.workunit.client.0.smithi112.stdout:0/215: truncate d5/da/d18/f3d 561450 0 2024-09-16T00:25:53.798 INFO:tasks.workunit.client.0.smithi112.stdout:0/216: rmdir d5/da/d18/d24/d32 0 2024-09-16T00:25:53.798 INFO:tasks.workunit.client.1.smithi169.stdout:8/4: sync 2024-09-16T00:25:53.798 INFO:tasks.workunit.client.1.smithi169.stdout:7/25: sync 2024-09-16T00:25:53.799 INFO:tasks.workunit.client.0.smithi112.stdout:0/217: read - d5/da/d18/d24/f41 zero size 2024-09-16T00:25:53.800 INFO:tasks.workunit.client.0.smithi112.stdout:0/218: write d5/da/d21/f25 [8359135,119890] 0 2024-09-16T00:25:53.800 INFO:tasks.workunit.client.1.smithi169.stdout:1/61: rename f8 to dc/f10 0 2024-09-16T00:25:53.805 INFO:tasks.workunit.client.1.smithi169.stdout:8/5: creat f1 x:0 0 0 2024-09-16T00:25:53.806 INFO:tasks.workunit.client.1.smithi169.stdout:1/62: mknod dc/df/c11 0 2024-09-16T00:25:53.807 INFO:tasks.workunit.client.1.smithi169.stdout:8/6: creat f2 x:0 0 0 2024-09-16T00:25:53.808 INFO:tasks.workunit.client.1.smithi169.stdout:1/63: creat dc/df/f12 x:0 0 0 2024-09-16T00:25:53.809 INFO:tasks.workunit.client.0.smithi112.stdout:0/219: dwrite d5/da/dc/d12/f17 [0,4194304] 0 2024-09-16T00:25:53.810 INFO:tasks.workunit.client.1.smithi169.stdout:1/64: symlink dc/df/l13 0 2024-09-16T00:25:53.811 INFO:tasks.workunit.client.1.smithi169.stdout:1/65: chown dc/fd 15436 1 2024-09-16T00:25:53.811 INFO:tasks.workunit.client.1.smithi169.stdout:8/7: dread f0 [0,4194304] 0 2024-09-16T00:25:53.812 INFO:tasks.workunit.client.0.smithi112.stdout:0/220: write d5/da/d21/f28 [178910,119992] 0 2024-09-16T00:25:53.813 INFO:tasks.workunit.client.1.smithi169.stdout:8/8: symlink l3 0 2024-09-16T00:25:53.814 INFO:tasks.workunit.client.1.smithi169.stdout:8/9: read - f2 zero size 2024-09-16T00:25:53.814 INFO:tasks.workunit.client.0.smithi112.stdout:2/248: dwrite d2/dd/d48/f2a [0,4194304] 0 2024-09-16T00:25:53.814 INFO:tasks.workunit.client.0.smithi112.stdout:0/221: rename d5/da/d18 to d5/da/d18/d24/d42 22 2024-09-16T00:25:53.815 INFO:tasks.workunit.client.1.smithi169.stdout:8/10: symlink l4 0 2024-09-16T00:25:53.815 INFO:tasks.workunit.client.1.smithi169.stdout:8/11: write f1 [653201,75244] 0 2024-09-16T00:25:53.816 INFO:tasks.workunit.client.0.smithi112.stdout:2/249: fdatasync d2/dd/d48/d22/d2b/d30/f36 0 2024-09-16T00:25:53.817 INFO:tasks.workunit.client.1.smithi169.stdout:8/12: creat f5 x:0 0 0 2024-09-16T00:25:53.820 INFO:tasks.workunit.client.1.smithi169.stdout:8/13: dread - f2 zero size 2024-09-16T00:25:53.820 INFO:tasks.workunit.client.1.smithi169.stdout:8/14: fdatasync f0 0 2024-09-16T00:25:53.821 INFO:tasks.workunit.client.0.smithi112.stdout:0/222: symlink d5/da/d18/l43 0 2024-09-16T00:25:53.821 INFO:tasks.workunit.client.0.smithi112.stdout:2/250: mknod d2/dd/d48/d22/c4d 0 2024-09-16T00:25:53.821 INFO:tasks.workunit.client.0.smithi112.stdout:0/223: read - d5/da/dc/d12/f14 zero size 2024-09-16T00:25:53.821 INFO:tasks.workunit.client.0.smithi112.stdout:2/251: dread d2/dd/d48/f2d [0,4194304] 0 2024-09-16T00:25:53.822 INFO:tasks.workunit.client.0.smithi112.stdout:0/224: symlink d5/da/d18/d24/l44 0 2024-09-16T00:25:53.823 INFO:tasks.workunit.client.1.smithi169.stdout:8/15: creat f6 x:0 0 0 2024-09-16T00:25:53.825 INFO:tasks.workunit.client.1.smithi169.stdout:1/66: fsync f0 0 2024-09-16T00:25:53.827 INFO:tasks.workunit.client.0.smithi112.stdout:2/252: dread d2/dd/d48/f24 [0,4194304] 0 2024-09-16T00:25:53.827 INFO:tasks.workunit.client.1.smithi169.stdout:8/16: mkdir d7 0 2024-09-16T00:25:53.828 INFO:tasks.workunit.client.1.smithi169.stdout:7/26: dwrite f2 [0,4194304] 0 2024-09-16T00:25:53.830 INFO:tasks.workunit.client.1.smithi169.stdout:7/27: dread - f5 zero size 2024-09-16T00:25:53.831 INFO:tasks.workunit.client.1.smithi169.stdout:7/28: fdatasync f5 0 2024-09-16T00:25:53.832 INFO:tasks.workunit.client.1.smithi169.stdout:1/67: link dc/le dc/l14 0 2024-09-16T00:25:53.832 INFO:tasks.workunit.client.0.smithi112.stdout:2/253: rename d2/l17 to d2/dd/l4e 0 2024-09-16T00:25:53.833 INFO:tasks.workunit.client.1.smithi169.stdout:1/68: read f4 [1811230,58630] 0 2024-09-16T00:25:53.833 INFO:tasks.workunit.client.1.smithi169.stdout:1/69: stat c6 0 2024-09-16T00:25:53.834 INFO:tasks.workunit.client.0.smithi112.stdout:2/254: symlink d2/dd/l4f 0 2024-09-16T00:25:53.834 INFO:tasks.workunit.client.1.smithi169.stdout:1/70: creat dc/f15 x:0 0 0 2024-09-16T00:25:53.835 INFO:tasks.workunit.client.0.smithi112.stdout:2/255: fdatasync d2/dd/d48/f1d 0 2024-09-16T00:25:53.835 INFO:tasks.workunit.client.1.smithi169.stdout:1/71: write f9 [82230,81642] 0 2024-09-16T00:25:53.836 INFO:tasks.workunit.client.0.smithi112.stdout:2/256: fsync d2/dd/d48/f1d 0 2024-09-16T00:25:53.837 INFO:tasks.workunit.client.1.smithi169.stdout:1/72: rmdir dc/df 39 2024-09-16T00:25:53.840 INFO:tasks.workunit.client.1.smithi169.stdout:1/73: write f0 [486983,13962] 0 2024-09-16T00:25:53.841 INFO:tasks.workunit.client.0.smithi112.stdout:2/257: dread d2/f19 [0,4194304] 0 2024-09-16T00:25:53.842 INFO:tasks.workunit.client.0.smithi112.stdout:2/258: mkdir d2/dd/d50 0 2024-09-16T00:25:53.843 INFO:tasks.workunit.client.0.smithi112.stdout:2/259: stat d2/dd/d48/l1c 0 2024-09-16T00:25:53.843 INFO:tasks.workunit.client.0.smithi112.stdout:2/260: fsync d2/dd/d48/f11 0 2024-09-16T00:25:53.845 INFO:tasks.workunit.client.0.smithi112.stdout:2/261: write d2/fa [4529732,115234] 0 2024-09-16T00:25:53.847 INFO:tasks.workunit.client.0.smithi112.stdout:2/262: mknod d2/dd/d48/d22/d2b/c51 0 2024-09-16T00:25:53.869 INFO:tasks.workunit.client.0.smithi112.stdout:0/225: dread f0 [0,4194304] 0 2024-09-16T00:25:53.869 INFO:tasks.workunit.client.0.smithi112.stdout:2/263: rmdir d2/dd/d50 0 2024-09-16T00:25:53.870 INFO:tasks.workunit.client.0.smithi112.stdout:2/264: fsync d2/dd/d48/d22/d2b/f31 0 2024-09-16T00:25:53.872 INFO:tasks.workunit.client.0.smithi112.stdout:0/226: rename d5/da/f15 to d5/da/f45 0 2024-09-16T00:25:53.873 INFO:tasks.workunit.client.0.smithi112.stdout:0/227: write d5/da/f27 [935460,9580] 0 2024-09-16T00:25:53.875 INFO:tasks.workunit.client.1.smithi169.stdout:2/21: sync 2024-09-16T00:25:53.876 INFO:tasks.workunit.client.0.smithi112.stdout:2/265: getdents d2/dd/d48 0 2024-09-16T00:25:53.878 INFO:tasks.workunit.client.0.smithi112.stdout:2/266: mknod d2/dd/d48/d22/d2b/c52 0 2024-09-16T00:25:53.880 INFO:tasks.workunit.client.0.smithi112.stdout:0/228: getdents d5/da/d18 0 2024-09-16T00:25:53.883 INFO:tasks.workunit.client.0.smithi112.stdout:2/267: dread d2/dd/d48/f23 [0,4194304] 0 2024-09-16T00:25:53.887 INFO:tasks.workunit.client.1.smithi169.stdout:1/74: dwrite fb [0,4194304] 0 2024-09-16T00:25:53.887 INFO:tasks.workunit.client.1.smithi169.stdout:2/22: getdents . 0 2024-09-16T00:25:53.888 INFO:tasks.workunit.client.1.smithi169.stdout:7/29: dwrite f1 [0,4194304] 0 2024-09-16T00:25:53.888 INFO:tasks.workunit.client.1.smithi169.stdout:2/23: chown f5 915891143 1 2024-09-16T00:25:53.888 INFO:tasks.workunit.client.1.smithi169.stdout:2/24: dread - f6 zero size 2024-09-16T00:25:53.889 INFO:tasks.workunit.client.1.smithi169.stdout:2/25: rmdir - no directory 2024-09-16T00:25:53.890 INFO:tasks.workunit.client.1.smithi169.stdout:7/30: creat f6 x:0 0 0 2024-09-16T00:25:53.892 INFO:tasks.workunit.client.1.smithi169.stdout:2/26: mkdir d7 0 2024-09-16T00:25:53.893 INFO:tasks.workunit.client.1.smithi169.stdout:2/27: dread - f0 zero size 2024-09-16T00:25:53.893 INFO:tasks.workunit.client.1.smithi169.stdout:7/31: mkdir d7 0 2024-09-16T00:25:53.894 INFO:tasks.workunit.client.1.smithi169.stdout:7/32: dread - f6 zero size 2024-09-16T00:25:53.896 INFO:tasks.workunit.client.1.smithi169.stdout:7/33: creat d7/f8 x:0 0 0 2024-09-16T00:25:53.896 INFO:tasks.workunit.client.1.smithi169.stdout:7/34: fdatasync f6 0 2024-09-16T00:25:53.896 INFO:tasks.workunit.client.1.smithi169.stdout:7/35: chown c4 98219 1 2024-09-16T00:25:53.898 INFO:tasks.workunit.client.1.smithi169.stdout:7/36: creat d7/f9 x:0 0 0 2024-09-16T00:25:53.898 INFO:tasks.workunit.client.1.smithi169.stdout:7/37: fdatasync f2 0 2024-09-16T00:25:53.900 INFO:tasks.workunit.client.1.smithi169.stdout:7/38: mkdir d7/da 0 2024-09-16T00:25:53.901 INFO:tasks.workunit.client.1.smithi169.stdout:7/39: symlink d7/da/lb 0 2024-09-16T00:25:53.902 INFO:tasks.workunit.client.1.smithi169.stdout:7/40: creat d7/da/fc x:0 0 0 2024-09-16T00:25:53.994 INFO:tasks.workunit.client.1.smithi169.stdout:1/75: write f4 [5370354,48220] 0 2024-09-16T00:25:53.997 INFO:tasks.workunit.client.1.smithi169.stdout:1/76: creat dc/df/f16 x:0 0 0 2024-09-16T00:25:53.997 INFO:tasks.workunit.client.1.smithi169.stdout:1/77: readlink la 0 2024-09-16T00:25:53.997 INFO:tasks.workunit.client.1.smithi169.stdout:1/78: dread - dc/df/f12 zero size 2024-09-16T00:25:53.998 INFO:tasks.workunit.client.1.smithi169.stdout:7/41: fsync d7/da/fc 0 2024-09-16T00:25:53.999 INFO:tasks.workunit.client.1.smithi169.stdout:7/42: chown d7/da/fc 7 1 2024-09-16T00:25:53.999 INFO:tasks.workunit.client.1.smithi169.stdout:1/79: unlink dc/f10 0 2024-09-16T00:25:54.000 INFO:tasks.workunit.client.1.smithi169.stdout:7/43: write d7/f8 [835729,47153] 0 2024-09-16T00:25:54.000 INFO:tasks.workunit.client.1.smithi169.stdout:7/44: rename d7 to d7/dd 22 2024-09-16T00:25:54.001 INFO:tasks.workunit.client.1.smithi169.stdout:7/45: chown f6 627025 1 2024-09-16T00:25:54.003 INFO:tasks.workunit.client.1.smithi169.stdout:7/46: rename d7/da/lb to d7/le 0 2024-09-16T00:25:54.004 INFO:tasks.workunit.client.1.smithi169.stdout:7/47: stat f2 0 2024-09-16T00:25:54.004 INFO:tasks.workunit.client.1.smithi169.stdout:7/48: fsync d7/f9 0 2024-09-16T00:25:54.006 INFO:tasks.workunit.client.1.smithi169.stdout:7/49: write d7/da/fc [477967,8877] 0 2024-09-16T00:25:54.006 INFO:tasks.workunit.client.1.smithi169.stdout:1/80: dwrite fb [0,4194304] 0 2024-09-16T00:25:54.007 INFO:tasks.workunit.client.1.smithi169.stdout:7/50: chown d7 479 1 2024-09-16T00:25:54.009 INFO:tasks.workunit.client.1.smithi169.stdout:1/81: rename f4 to dc/df/f17 0 2024-09-16T00:25:54.014 INFO:tasks.workunit.client.1.smithi169.stdout:7/51: getdents d7 0 2024-09-16T00:25:54.015 INFO:tasks.workunit.client.1.smithi169.stdout:7/52: dread - d7/f9 zero size 2024-09-16T00:25:54.017 INFO:tasks.workunit.client.1.smithi169.stdout:7/53: fdatasync d7/f8 0 2024-09-16T00:25:54.018 INFO:tasks.workunit.client.1.smithi169.stdout:1/82: dwrite f7 [0,4194304] 0 2024-09-16T00:25:54.018 INFO:tasks.workunit.client.1.smithi169.stdout:7/54: dread - d7/f9 zero size 2024-09-16T00:25:54.019 INFO:tasks.workunit.client.1.smithi169.stdout:1/83: chown f7 115916163 1 2024-09-16T00:25:54.020 INFO:tasks.workunit.client.1.smithi169.stdout:7/55: mkdir d7/df 0 2024-09-16T00:25:54.021 INFO:tasks.workunit.client.1.smithi169.stdout:7/56: dread - d7/f9 zero size 2024-09-16T00:25:54.022 INFO:tasks.workunit.client.1.smithi169.stdout:1/84: mkdir dc/d18 0 2024-09-16T00:25:54.025 INFO:tasks.workunit.client.1.smithi169.stdout:1/85: fsync fb 0 2024-09-16T00:25:54.028 INFO:tasks.workunit.client.1.smithi169.stdout:1/86: symlink dc/d18/l19 0 2024-09-16T00:25:54.029 INFO:tasks.workunit.client.1.smithi169.stdout:1/87: chown dc/df 2045 1 2024-09-16T00:25:54.029 INFO:tasks.workunit.client.1.smithi169.stdout:7/57: dwrite d7/da/fc [0,4194304] 0 2024-09-16T00:25:54.030 INFO:tasks.workunit.client.1.smithi169.stdout:7/58: fsync f1 0 2024-09-16T00:25:54.033 INFO:tasks.workunit.client.0.smithi112.stdout:3/222: sync 2024-09-16T00:25:54.035 INFO:tasks.workunit.client.0.smithi112.stdout:3/223: fdatasync d2/f24 0 2024-09-16T00:25:54.037 INFO:tasks.workunit.client.1.smithi169.stdout:1/88: dread f0 [0,4194304] 0 2024-09-16T00:25:54.038 INFO:tasks.workunit.client.0.smithi112.stdout:3/224: creat d2/d9/d2b/d2c/f51 x:0 0 0 2024-09-16T00:25:54.039 INFO:tasks.workunit.client.1.smithi169.stdout:7/59: dwrite d7/da/fc [0,4194304] 0 2024-09-16T00:25:54.039 INFO:tasks.workunit.client.1.smithi169.stdout:7/60: stat f5 0 2024-09-16T00:25:54.042 INFO:tasks.workunit.client.1.smithi169.stdout:7/61: symlink d7/df/l10 0 2024-09-16T00:25:54.042 INFO:tasks.workunit.client.0.smithi112.stdout:3/225: rename d2/d9/d2b/d2c/d32 to d2/d9/d2b/d47/d52 0 2024-09-16T00:25:54.042 INFO:tasks.workunit.client.1.smithi169.stdout:1/89: dread fb [0,4194304] 0 2024-09-16T00:25:54.045 INFO:tasks.workunit.client.1.smithi169.stdout:1/90: symlink dc/df/l1a 0 2024-09-16T00:25:54.046 INFO:tasks.workunit.client.1.smithi169.stdout:1/91: dread - dc/f15 zero size 2024-09-16T00:25:54.047 INFO:tasks.workunit.client.1.smithi169.stdout:1/92: symlink dc/d18/l1b 0 2024-09-16T00:25:54.048 INFO:tasks.workunit.client.1.smithi169.stdout:1/93: rmdir dc/d18 39 2024-09-16T00:25:54.049 INFO:tasks.workunit.client.1.smithi169.stdout:1/94: write f9 [1101063,55019] 0 2024-09-16T00:25:54.049 INFO:tasks.workunit.client.1.smithi169.stdout:1/95: stat dc/df/l13 0 2024-09-16T00:25:54.050 INFO:tasks.workunit.client.1.smithi169.stdout:1/96: write dc/df/f12 [4674,23676] 0 2024-09-16T00:25:54.050 INFO:tasks.workunit.client.0.smithi112.stdout:3/226: dwrite d2/f1d [0,4194304] 0 2024-09-16T00:25:54.052 INFO:tasks.workunit.client.1.smithi169.stdout:1/97: creat dc/df/f1c x:0 0 0 2024-09-16T00:25:54.055 INFO:tasks.workunit.client.1.smithi169.stdout:1/98: rename f9 to dc/f1d 0 2024-09-16T00:25:54.056 INFO:tasks.workunit.client.0.smithi112.stdout:3/227: getdents d2/d9/d2b/d33/d49 0 2024-09-16T00:25:54.056 INFO:tasks.workunit.client.1.smithi169.stdout:7/62: write f5 [238730,23924] 0 2024-09-16T00:25:54.057 INFO:tasks.workunit.client.1.smithi169.stdout:1/99: mknod dc/df/c1e 0 2024-09-16T00:25:54.059 INFO:tasks.workunit.client.0.smithi112.stdout:3/228: rename d2/f1d to d2/f53 0 2024-09-16T00:25:54.059 INFO:tasks.workunit.client.0.smithi112.stdout:3/229: chown d2/d9/d2b/d33 10269 1 2024-09-16T00:25:54.060 INFO:tasks.workunit.client.1.smithi169.stdout:1/100: truncate f0 3907942 0 2024-09-16T00:25:54.061 INFO:tasks.workunit.client.0.smithi112.stdout:3/230: stat d2/d9/d2b/d33/d49 0 2024-09-16T00:25:54.065 INFO:tasks.workunit.client.1.smithi169.stdout:7/63: dwrite d7/da/fc [4194304,4194304] 0 2024-09-16T00:25:54.065 INFO:tasks.workunit.client.1.smithi169.stdout:7/64: stat f6 0 2024-09-16T00:25:54.066 INFO:tasks.workunit.client.1.smithi169.stdout:1/101: dread fb [0,4194304] 0 2024-09-16T00:25:54.068 INFO:tasks.workunit.client.0.smithi112.stdout:3/231: dwrite d2/f24 [0,4194304] 0 2024-09-16T00:25:54.069 INFO:tasks.workunit.client.1.smithi169.stdout:7/65: link f2 d7/df/f11 0 2024-09-16T00:25:54.070 INFO:tasks.workunit.client.1.smithi169.stdout:6/24: sync 2024-09-16T00:25:54.070 INFO:tasks.workunit.client.1.smithi169.stdout:5/44: sync 2024-09-16T00:25:54.070 INFO:tasks.workunit.client.1.smithi169.stdout:0/11: sync 2024-09-16T00:25:54.070 INFO:tasks.workunit.client.1.smithi169.stdout:0/12: dwrite - no filename 2024-09-16T00:25:54.070 INFO:tasks.workunit.client.1.smithi169.stdout:0/13: truncate - no filename 2024-09-16T00:25:54.070 INFO:tasks.workunit.client.1.smithi169.stdout:9/46: sync 2024-09-16T00:25:54.070 INFO:tasks.workunit.client.1.smithi169.stdout:4/38: sync 2024-09-16T00:25:54.070 INFO:tasks.workunit.client.1.smithi169.stdout:3/26: sync 2024-09-16T00:25:54.070 INFO:tasks.workunit.client.1.smithi169.stdout:7/66: write d7/da/fc [5364391,110639] 0 2024-09-16T00:25:54.071 INFO:tasks.workunit.client.1.smithi169.stdout:3/27: stat d7 0 2024-09-16T00:25:54.071 INFO:tasks.workunit.client.0.smithi112.stdout:3/232: creat d2/d9/d2b/d47/f54 x:0 0 0 2024-09-16T00:25:54.073 INFO:tasks.workunit.client.1.smithi169.stdout:6/25: symlink d2/l6 0 2024-09-16T00:25:54.074 INFO:tasks.workunit.client.0.smithi112.stdout:3/233: mknod d2/d9/d2b/c55 0 2024-09-16T00:25:54.074 INFO:tasks.workunit.client.1.smithi169.stdout:6/26: rename d2 to d2/d7 22 2024-09-16T00:25:54.075 INFO:tasks.workunit.client.1.smithi169.stdout:9/47: creat d5/f14 x:0 0 0 2024-09-16T00:25:54.075 INFO:tasks.workunit.client.0.smithi112.stdout:3/234: mknod d2/d9/d2b/d33/d49/c56 0 2024-09-16T00:25:54.076 INFO:tasks.workunit.client.1.smithi169.stdout:7/67: fdatasync f2 0 2024-09-16T00:25:54.076 INFO:tasks.workunit.client.1.smithi169.stdout:3/28: dread f3 [0,4194304] 0 2024-09-16T00:25:54.077 INFO:tasks.workunit.client.0.smithi112.stdout:3/235: write d2/d9/d2b/f48 [504393,109568] 0 2024-09-16T00:25:54.078 INFO:tasks.workunit.client.1.smithi169.stdout:7/68: fdatasync f5 0 2024-09-16T00:25:54.078 INFO:tasks.workunit.client.1.smithi169.stdout:9/48: dread d5/fb [0,4194304] 0 2024-09-16T00:25:54.079 INFO:tasks.workunit.client.0.smithi112.stdout:3/236: truncate d2/f1f 1487218 0 2024-09-16T00:25:54.080 INFO:tasks.workunit.client.1.smithi169.stdout:7/69: read d7/f8 [650892,7673] 0 2024-09-16T00:25:54.081 INFO:tasks.workunit.client.1.smithi169.stdout:0/14: mknod d0/c1 0 2024-09-16T00:25:54.081 INFO:tasks.workunit.client.1.smithi169.stdout:1/102: getdents dc/df 0 2024-09-16T00:25:54.083 INFO:tasks.workunit.client.1.smithi169.stdout:6/27: mknod d2/c8 0 2024-09-16T00:25:54.084 INFO:tasks.workunit.client.1.smithi169.stdout:4/39: mknod d2/dd/ce 0 2024-09-16T00:25:54.085 INFO:tasks.workunit.client.1.smithi169.stdout:7/70: dread d7/df/f11 [0,4194304] 0 2024-09-16T00:25:54.086 INFO:tasks.workunit.client.0.smithi112.stdout:3/237: dwrite d2/d9/d2b/f48 [0,4194304] 0 2024-09-16T00:25:54.087 INFO:tasks.workunit.client.1.smithi169.stdout:3/29: unlink d7/ca 0 2024-09-16T00:25:54.089 INFO:tasks.workunit.client.1.smithi169.stdout:0/15: mkdir d0/d2 0 2024-09-16T00:25:54.089 INFO:tasks.workunit.client.1.smithi169.stdout:0/16: dwrite - no filename 2024-09-16T00:25:54.089 INFO:tasks.workunit.client.1.smithi169.stdout:0/17: dwrite - no filename 2024-09-16T00:25:54.089 INFO:tasks.workunit.client.1.smithi169.stdout:0/18: stat d0/d2 0 2024-09-16T00:25:54.089 INFO:tasks.workunit.client.1.smithi169.stdout:0/19: stat d0/c1 0 2024-09-16T00:25:54.091 INFO:tasks.workunit.client.1.smithi169.stdout:0/20: chown d0/c1 200436 1 2024-09-16T00:25:54.091 INFO:tasks.workunit.client.1.smithi169.stdout:0/21: truncate - no filename 2024-09-16T00:25:54.091 INFO:tasks.workunit.client.1.smithi169.stdout:0/22: dread - no filename 2024-09-16T00:25:54.092 INFO:tasks.workunit.client.0.smithi112.stdout:3/238: rmdir d2/d9/d2b/d33 39 2024-09-16T00:25:54.092 INFO:tasks.workunit.client.1.smithi169.stdout:4/40: symlink d2/dd/lf 0 2024-09-16T00:25:54.093 INFO:tasks.workunit.client.0.smithi112.stdout:3/239: fdatasync d2/f53 0 2024-09-16T00:25:54.094 INFO:tasks.workunit.client.0.smithi112.stdout:3/240: write d2/d9/f4e [545976,28022] 0 2024-09-16T00:25:54.095 INFO:tasks.workunit.client.1.smithi169.stdout:3/30: unlink f3 0 2024-09-16T00:25:54.095 INFO:tasks.workunit.client.1.smithi169.stdout:1/103: symlink dc/d18/l1f 0 2024-09-16T00:25:54.096 INFO:tasks.workunit.client.1.smithi169.stdout:4/41: dread - d2/f9 zero size 2024-09-16T00:25:54.097 INFO:tasks.workunit.client.1.smithi169.stdout:0/23: mkdir d0/d2/d3 0 2024-09-16T00:25:54.099 INFO:tasks.workunit.client.1.smithi169.stdout:3/31: creat d7/fd x:0 0 0 2024-09-16T00:25:54.101 INFO:tasks.workunit.client.1.smithi169.stdout:5/45: dwrite d2/fe [0,4194304] 0 2024-09-16T00:25:54.102 INFO:tasks.workunit.client.1.smithi169.stdout:0/24: unlink d0/c1 0 2024-09-16T00:25:54.102 INFO:tasks.workunit.client.1.smithi169.stdout:0/25: write - no filename 2024-09-16T00:25:54.102 INFO:tasks.workunit.client.1.smithi169.stdout:3/32: mknod d7/ce 0 2024-09-16T00:25:54.103 INFO:tasks.workunit.client.1.smithi169.stdout:0/26: chown d0/d2/d3 103192044 1 2024-09-16T00:25:54.104 INFO:tasks.workunit.client.1.smithi169.stdout:0/27: write - no filename 2024-09-16T00:25:54.104 INFO:tasks.workunit.client.1.smithi169.stdout:0/28: dread - no filename 2024-09-16T00:25:54.106 INFO:tasks.workunit.client.1.smithi169.stdout:4/42: link d2/f3 d2/f10 0 2024-09-16T00:25:54.108 INFO:tasks.workunit.client.1.smithi169.stdout:4/43: rename d2/dd to d2/dd/d11 22 2024-09-16T00:25:54.109 INFO:tasks.workunit.client.1.smithi169.stdout:0/29: mknod d0/c4 0 2024-09-16T00:25:54.111 INFO:tasks.workunit.client.1.smithi169.stdout:0/30: chown d0/d2/d3 12788778 1 2024-09-16T00:25:54.111 INFO:tasks.workunit.client.1.smithi169.stdout:0/31: write - no filename 2024-09-16T00:25:54.112 INFO:tasks.workunit.client.1.smithi169.stdout:4/44: mkdir d2/d12 0 2024-09-16T00:25:54.114 INFO:tasks.workunit.client.1.smithi169.stdout:5/46: creat d2/da/d10/f11 x:0 0 0 2024-09-16T00:25:54.115 INFO:tasks.workunit.client.1.smithi169.stdout:4/45: readlink d2/l7 0 2024-09-16T00:25:54.115 INFO:tasks.workunit.client.1.smithi169.stdout:3/33: dwrite d7/fb [0,4194304] 0 2024-09-16T00:25:54.116 INFO:tasks.workunit.client.1.smithi169.stdout:7/71: truncate d7/df/f11 2609052 0 2024-09-16T00:25:54.118 INFO:tasks.workunit.client.0.smithi112.stdout:3/241: dread d2/d9/d2b/d39/f3c [0,4194304] 0 2024-09-16T00:25:54.119 INFO:tasks.workunit.client.1.smithi169.stdout:3/34: read - d7/f9 zero size 2024-09-16T00:25:54.119 INFO:tasks.workunit.client.0.smithi112.stdout:3/242: write d2/d9/d2b/d47/f54 [943809,113456] 0 2024-09-16T00:25:54.120 INFO:tasks.workunit.client.1.smithi169.stdout:3/35: dread - d7/f9 zero size 2024-09-16T00:25:54.120 INFO:tasks.workunit.client.1.smithi169.stdout:0/32: mkdir d0/d2/d3/d5 0 2024-09-16T00:25:54.120 INFO:tasks.workunit.client.1.smithi169.stdout:0/33: dread - no filename 2024-09-16T00:25:54.121 INFO:tasks.workunit.client.1.smithi169.stdout:1/104: write f0 [3219964,59860] 0 2024-09-16T00:25:54.121 INFO:tasks.workunit.client.1.smithi169.stdout:4/46: truncate d2/fb 691837 0 2024-09-16T00:25:54.122 INFO:tasks.workunit.client.1.smithi169.stdout:3/36: dread - d7/f9 zero size 2024-09-16T00:25:54.123 INFO:tasks.workunit.client.0.smithi112.stdout:3/243: symlink d2/d9/d2b/d33/d49/l57 0 2024-09-16T00:25:54.124 INFO:tasks.workunit.client.1.smithi169.stdout:0/34: creat d0/d2/f6 x:0 0 0 2024-09-16T00:25:54.127 INFO:tasks.workunit.client.1.smithi169.stdout:4/47: truncate d2/f3 700105 0 2024-09-16T00:25:54.129 INFO:tasks.workunit.client.0.smithi112.stdout:3/244: dwrite d2/f3a [4194304,4194304] 0 2024-09-16T00:25:54.129 INFO:tasks.workunit.client.1.smithi169.stdout:5/47: getdents d2 0 2024-09-16T00:25:54.130 INFO:tasks.workunit.client.1.smithi169.stdout:0/35: symlink d0/d2/d3/d5/l7 0 2024-09-16T00:25:54.131 INFO:tasks.workunit.client.1.smithi169.stdout:7/72: getdents d7/da 0 2024-09-16T00:25:54.132 INFO:tasks.workunit.client.0.smithi112.stdout:3/245: mknod d2/d9/d2b/d33/d49/c58 0 2024-09-16T00:25:54.132 INFO:tasks.workunit.client.1.smithi169.stdout:0/36: write d0/d2/f6 [593168,92549] 0 2024-09-16T00:25:54.132 INFO:tasks.workunit.client.1.smithi169.stdout:3/37: link l4 d7/lf 0 2024-09-16T00:25:54.135 INFO:tasks.workunit.client.0.smithi112.stdout:3/246: dread d2/f2d [0,4194304] 0 2024-09-16T00:25:54.137 INFO:tasks.workunit.client.0.smithi112.stdout:3/247: write d2/d9/f21 [753793,82733] 0 2024-09-16T00:25:54.138 INFO:tasks.workunit.client.0.smithi112.stdout:3/248: write d2/f1f [981890,41606] 0 2024-09-16T00:25:54.139 INFO:tasks.workunit.client.1.smithi169.stdout:4/48: mkdir d2/d12/d13 0 2024-09-16T00:25:54.139 INFO:tasks.workunit.client.1.smithi169.stdout:1/105: getdents dc/d18 0 2024-09-16T00:25:54.141 INFO:tasks.workunit.client.0.smithi112.stdout:3/249: rename d2/c26 to d2/d9/d2b/d47/d52/c59 0 2024-09-16T00:25:54.143 INFO:tasks.workunit.client.0.smithi112.stdout:3/250: symlink d2/d9/d2b/d47/d52/l5a 0 2024-09-16T00:25:54.144 INFO:tasks.workunit.client.1.smithi169.stdout:4/49: chown d2/dd/ce 8835066 1 2024-09-16T00:25:54.144 INFO:tasks.workunit.client.1.smithi169.stdout:5/48: dread d2/fb [4194304,4194304] 0 2024-09-16T00:25:54.144 INFO:tasks.workunit.client.0.smithi112.stdout:3/251: dread d2/f2d [0,4194304] 0 2024-09-16T00:25:54.144 INFO:tasks.workunit.client.0.smithi112.stdout:3/252: chown d2/d9/d2b/d33 153517867 1 2024-09-16T00:25:54.146 INFO:tasks.workunit.client.0.smithi112.stdout:3/253: symlink d2/d9/d2b/d47/l5b 0 2024-09-16T00:25:54.149 INFO:tasks.workunit.client.1.smithi169.stdout:5/49: symlink d2/l12 0 2024-09-16T00:25:54.149 INFO:tasks.workunit.client.1.smithi169.stdout:7/73: dwrite d7/f8 [0,4194304] 0 2024-09-16T00:25:54.150 INFO:tasks.workunit.client.1.smithi169.stdout:3/38: dwrite d7/fd [0,4194304] 0 2024-09-16T00:25:54.151 INFO:tasks.workunit.client.1.smithi169.stdout:4/50: dread d2/f10 [0,4194304] 0 2024-09-16T00:25:54.152 INFO:tasks.workunit.client.1.smithi169.stdout:3/39: write d7/fd [4545069,56699] 0 2024-09-16T00:25:54.152 INFO:tasks.workunit.client.1.smithi169.stdout:0/37: truncate d0/d2/f6 270242 0 2024-09-16T00:25:54.154 INFO:tasks.workunit.client.0.smithi112.stdout:3/254: truncate d2/d9/d2b/d47/f30 3193398 0 2024-09-16T00:25:54.158 INFO:tasks.workunit.client.1.smithi169.stdout:7/74: dread d7/f8 [0,4194304] 0 2024-09-16T00:25:54.159 INFO:tasks.workunit.client.1.smithi169.stdout:4/51: truncate d2/f10 1411066 0 2024-09-16T00:25:54.161 INFO:tasks.workunit.client.1.smithi169.stdout:0/38: fsync d0/d2/f6 0 2024-09-16T00:25:54.162 INFO:tasks.workunit.client.0.smithi112.stdout:3/255: dwrite d2/d9/d2b/d33/f50 [0,4194304] 0 2024-09-16T00:25:54.163 INFO:tasks.workunit.client.1.smithi169.stdout:7/75: mknod d7/c12 0 2024-09-16T00:25:54.164 INFO:tasks.workunit.client.1.smithi169.stdout:7/76: fsync d7/da/fc 0 2024-09-16T00:25:54.165 INFO:tasks.workunit.client.0.smithi112.stdout:3/256: symlink d2/d9/d2b/d47/l5c 0 2024-09-16T00:25:54.166 INFO:tasks.workunit.client.1.smithi169.stdout:3/40: rmdir d7/dc 0 2024-09-16T00:25:54.167 INFO:tasks.workunit.client.1.smithi169.stdout:0/39: rename d0/d2/d3 to d0/d2/d8 0 2024-09-16T00:25:54.169 INFO:tasks.workunit.client.0.smithi112.stdout:3/257: mknod d2/d9/d2b/c5d 0 2024-09-16T00:25:54.172 INFO:tasks.workunit.client.1.smithi169.stdout:3/41: creat d7/f10 x:0 0 0 2024-09-16T00:25:54.173 INFO:tasks.workunit.client.1.smithi169.stdout:0/40: write d0/d2/f6 [138306,60762] 0 2024-09-16T00:25:54.175 INFO:tasks.workunit.client.1.smithi169.stdout:0/41: write d0/d2/f6 [588702,35336] 0 2024-09-16T00:25:54.175 INFO:tasks.workunit.client.1.smithi169.stdout:0/42: stat d0 0 2024-09-16T00:25:54.178 INFO:tasks.workunit.client.0.smithi112.stdout:4/174: sync 2024-09-16T00:25:54.181 INFO:tasks.workunit.client.0.smithi112.stdout:4/175: getdents d6/db/df 0 2024-09-16T00:25:54.183 INFO:tasks.workunit.client.1.smithi169.stdout:3/42: fsync d7/fd 0 2024-09-16T00:25:54.184 INFO:tasks.workunit.client.1.smithi169.stdout:0/43: dread d0/d2/f6 [0,4194304] 0 2024-09-16T00:25:54.185 INFO:tasks.workunit.client.0.smithi112.stdout:4/176: creat d6/db/d15/d31/d34/f3f x:0 0 0 2024-09-16T00:25:54.185 INFO:tasks.workunit.client.0.smithi112.stdout:4/177: stat d6/d39/d1c/f2f 0 2024-09-16T00:25:54.186 INFO:tasks.workunit.client.1.smithi169.stdout:4/52: write d2/f10 [2259629,47574] 0 2024-09-16T00:25:54.186 INFO:tasks.workunit.client.1.smithi169.stdout:4/53: readlink l0 0 2024-09-16T00:25:54.187 INFO:tasks.workunit.client.0.smithi112.stdout:4/178: truncate d6/da/f35 291391 0 2024-09-16T00:25:54.187 INFO:tasks.workunit.client.0.smithi112.stdout:3/258: truncate d2/f24 2433559 0 2024-09-16T00:25:54.187 INFO:tasks.workunit.client.0.smithi112.stdout:4/179: stat d6/db/df/c20 0 2024-09-16T00:25:54.189 INFO:tasks.workunit.client.0.smithi112.stdout:4/180: unlink d6/l1e 0 2024-09-16T00:25:54.189 INFO:tasks.workunit.client.1.smithi169.stdout:0/44: write d0/d2/f6 [309492,6017] 0 2024-09-16T00:25:54.189 INFO:tasks.workunit.client.1.smithi169.stdout:3/43: link d7/fd d7/f11 0 2024-09-16T00:25:54.191 INFO:tasks.workunit.client.1.smithi169.stdout:5/50: dwrite d2/f8 [0,4194304] 0 2024-09-16T00:25:54.193 INFO:tasks.workunit.client.0.smithi112.stdout:3/259: dwrite d2/d9/d2b/d47/d52/f38 [0,4194304] 0 2024-09-16T00:25:54.193 INFO:tasks.workunit.client.1.smithi169.stdout:4/54: dread d2/fb [0,4194304] 0 2024-09-16T00:25:54.194 INFO:tasks.workunit.client.0.smithi112.stdout:3/260: stat d2/f19 0 2024-09-16T00:25:54.194 INFO:tasks.workunit.client.1.smithi169.stdout:5/51: read - d2/ff zero size 2024-09-16T00:25:54.196 INFO:tasks.workunit.client.0.smithi112.stdout:4/181: dwrite d6/db/f37 [0,4194304] 0 2024-09-16T00:25:54.197 INFO:tasks.workunit.client.1.smithi169.stdout:3/44: rename d7/ce to d7/c12 0 2024-09-16T00:25:54.198 INFO:tasks.workunit.client.0.smithi112.stdout:3/261: creat d2/f5e x:0 0 0 2024-09-16T00:25:54.203 INFO:tasks.workunit.client.0.smithi112.stdout:3/262: symlink d2/d9/l5f 0 2024-09-16T00:25:54.203 INFO:tasks.workunit.client.0.smithi112.stdout:4/182: dread d6/db/d15/d31/d34/f3e [0,4194304] 0 2024-09-16T00:25:54.203 INFO:tasks.workunit.client.1.smithi169.stdout:5/52: mkdir d2/da/d10/d13 0 2024-09-16T00:25:54.203 INFO:tasks.workunit.client.1.smithi169.stdout:4/55: dread d2/f10 [0,4194304] 0 2024-09-16T00:25:54.204 INFO:tasks.workunit.client.0.smithi112.stdout:4/183: readlink d6/d39/l3a 0 2024-09-16T00:25:54.206 INFO:tasks.workunit.client.1.smithi169.stdout:4/56: symlink d2/d12/l14 0 2024-09-16T00:25:54.207 INFO:tasks.workunit.client.1.smithi169.stdout:5/53: creat d2/da/d10/d13/f14 x:0 0 0 2024-09-16T00:25:54.209 INFO:tasks.workunit.client.1.smithi169.stdout:5/54: truncate d2/f4 4691359 0 2024-09-16T00:25:54.211 INFO:tasks.workunit.client.1.smithi169.stdout:0/45: truncate d0/d2/f6 431031 0 2024-09-16T00:25:54.211 INFO:tasks.workunit.client.0.smithi112.stdout:4/184: dwrite d6/f1b [4194304,4194304] 0 2024-09-16T00:25:54.212 INFO:tasks.workunit.client.0.smithi112.stdout:5/269: sync 2024-09-16T00:25:54.213 INFO:tasks.workunit.client.0.smithi112.stdout:1/162: sync 2024-09-16T00:25:54.213 INFO:tasks.workunit.client.0.smithi112.stdout:6/197: sync 2024-09-16T00:25:54.213 INFO:tasks.workunit.client.0.smithi112.stdout:7/261: sync 2024-09-16T00:25:54.213 INFO:tasks.workunit.client.0.smithi112.stdout:9/419: sync 2024-09-16T00:25:54.213 INFO:tasks.workunit.client.0.smithi112.stdout:8/167: sync 2024-09-16T00:25:54.213 INFO:tasks.workunit.client.0.smithi112.stdout:0/229: sync 2024-09-16T00:25:54.213 INFO:tasks.workunit.client.0.smithi112.stdout:2/268: sync 2024-09-16T00:25:54.214 INFO:tasks.workunit.client.1.smithi169.stdout:5/55: stat d2/fb 0 2024-09-16T00:25:54.215 INFO:tasks.workunit.client.1.smithi169.stdout:5/56: chown d2/fd 6 1 2024-09-16T00:25:54.216 INFO:tasks.workunit.client.0.smithi112.stdout:7/262: chown d6/d10/d16/c51 55615 1 2024-09-16T00:25:54.217 INFO:tasks.workunit.client.0.smithi112.stdout:4/185: creat d6/da/f40 x:0 0 0 2024-09-16T00:25:54.217 INFO:tasks.workunit.client.0.smithi112.stdout:8/168: readlink d10/l35 0 2024-09-16T00:25:54.217 INFO:tasks.workunit.client.0.smithi112.stdout:9/420: chown d1/d5/d2c/d6f 1 1 2024-09-16T00:25:54.218 INFO:tasks.workunit.client.0.smithi112.stdout:5/270: stat d4/dd/d12/l53 0 2024-09-16T00:25:54.220 INFO:tasks.workunit.client.1.smithi169.stdout:3/45: dwrite d7/fd [4194304,4194304] 0 2024-09-16T00:25:54.222 INFO:tasks.workunit.client.1.smithi169.stdout:3/46: fsync d7/f10 0 2024-09-16T00:25:54.222 INFO:tasks.workunit.client.1.smithi169.stdout:8/17: sync 2024-09-16T00:25:54.223 INFO:tasks.workunit.client.1.smithi169.stdout:3/47: stat d7/f9 0 2024-09-16T00:25:54.223 INFO:tasks.workunit.client.0.smithi112.stdout:8/169: unlink d10/f23 0 2024-09-16T00:25:54.224 INFO:tasks.workunit.client.1.smithi169.stdout:8/18: chown f6 466603 1 2024-09-16T00:25:54.225 INFO:tasks.workunit.client.0.smithi112.stdout:3/263: dread d2/fb [0,4194304] 0 2024-09-16T00:25:54.231 INFO:tasks.workunit.client.1.smithi169.stdout:5/57: dwrite d2/da/d10/d13/f14 [0,4194304] 0 2024-09-16T00:25:54.231 INFO:tasks.workunit.client.0.smithi112.stdout:1/163: link d0/f1e d0/d14/d1c/d20/f33 0 2024-09-16T00:25:54.233 INFO:tasks.workunit.client.1.smithi169.stdout:8/19: creat d7/f8 x:0 0 0 2024-09-16T00:25:54.234 INFO:tasks.workunit.client.0.smithi112.stdout:0/230: dwrite d5/da/d18/d24/f41 [0,4194304] 0 2024-09-16T00:25:54.234 INFO:tasks.workunit.client.1.smithi169.stdout:5/58: rename d2 to d2/da/d10/d13/d15 22 2024-09-16T00:25:54.235 INFO:tasks.workunit.client.0.smithi112.stdout:4/186: mkdir d6/db/d1f/d41 0 2024-09-16T00:25:54.235 INFO:tasks.workunit.client.0.smithi112.stdout:7/263: rename d6/d10/l40 to d6/d10/d1b/d39/l63 0 2024-09-16T00:25:54.236 INFO:tasks.workunit.client.1.smithi169.stdout:8/20: mknod d7/c9 0 2024-09-16T00:25:54.236 INFO:tasks.workunit.client.0.smithi112.stdout:9/421: readlink d1/d5/d6/d2d/d39/l68 0 2024-09-16T00:25:54.237 INFO:tasks.workunit.client.1.smithi169.stdout:5/59: mknod d2/da/d10/c16 0 2024-09-16T00:25:54.238 INFO:tasks.workunit.client.1.smithi169.stdout:8/21: creat d7/fa x:0 0 0 2024-09-16T00:25:54.240 INFO:tasks.workunit.client.1.smithi169.stdout:8/22: write d7/fa [375167,15627] 0 2024-09-16T00:25:54.241 INFO:tasks.workunit.client.1.smithi169.stdout:3/48: dwrite d7/fb [0,4194304] 0 2024-09-16T00:25:54.242 INFO:tasks.workunit.client.0.smithi112.stdout:6/198: dwrite fa [0,4194304] 0 2024-09-16T00:25:54.242 INFO:tasks.workunit.client.1.smithi169.stdout:8/23: symlink d7/lb 0 2024-09-16T00:25:54.244 INFO:tasks.workunit.client.1.smithi169.stdout:3/49: rename l2 to d7/l13 0 2024-09-16T00:25:54.247 INFO:tasks.workunit.client.0.smithi112.stdout:5/271: rename d4/d7/d14/c3f to d4/d7/d26/d4c/c5f 0 2024-09-16T00:25:54.247 INFO:tasks.workunit.client.0.smithi112.stdout:8/170: mknod d10/d32/c37 0 2024-09-16T00:25:54.253 INFO:tasks.workunit.client.1.smithi169.stdout:3/50: dread d7/f11 [4194304,4194304] 0 2024-09-16T00:25:54.254 INFO:tasks.workunit.client.1.smithi169.stdout:8/24: dwrite f2 [0,4194304] 0 2024-09-16T00:25:54.254 INFO:tasks.workunit.client.0.smithi112.stdout:7/264: mkdir d6/d10/d16/d49/d64 0 2024-09-16T00:25:54.255 INFO:tasks.workunit.client.0.smithi112.stdout:0/231: dwrite d5/da/dc/fe [4194304,4194304] 0 2024-09-16T00:25:54.255 INFO:tasks.workunit.client.0.smithi112.stdout:2/269: dread d2/dd/d48/f27 [0,4194304] 0 2024-09-16T00:25:54.255 INFO:tasks.workunit.client.0.smithi112.stdout:9/422: mknod d1/d5/d6/d18/ca0 0 2024-09-16T00:25:54.258 INFO:tasks.workunit.client.1.smithi169.stdout:0/46: truncate d0/d2/f6 1139938 0 2024-09-16T00:25:54.259 INFO:tasks.workunit.client.0.smithi112.stdout:2/270: rename d2/dd to d2/dd/d48/d22/d2b/d3b/d53 22 2024-09-16T00:25:54.259 INFO:tasks.workunit.client.1.smithi169.stdout:3/51: fdatasync d7/fd 0 2024-09-16T00:25:54.261 INFO:tasks.workunit.client.1.smithi169.stdout:8/25: mkdir d7/dc 0 2024-09-16T00:25:54.262 INFO:tasks.workunit.client.0.smithi112.stdout:2/271: write d2/dd/d48/d22/d2b/f31 [644776,54990] 0 2024-09-16T00:25:54.263 INFO:tasks.workunit.client.0.smithi112.stdout:8/171: creat d10/d12/d1c/f38 x:0 0 0 2024-09-16T00:25:54.263 INFO:tasks.workunit.client.1.smithi169.stdout:8/26: readlink l3 0 2024-09-16T00:25:54.265 INFO:tasks.workunit.client.0.smithi112.stdout:1/164: dwrite d0/d14/fd [4194304,4194304] 0 2024-09-16T00:25:54.268 INFO:tasks.workunit.client.0.smithi112.stdout:4/187: link d6/d39/d1c/f27 d6/db/f42 0 2024-09-16T00:25:54.268 INFO:tasks.workunit.client.1.smithi169.stdout:8/27: fdatasync d7/f8 0 2024-09-16T00:25:54.269 INFO:tasks.workunit.client.0.smithi112.stdout:7/265: mknod d6/d10/d46/c65 0 2024-09-16T00:25:54.270 INFO:tasks.workunit.client.1.smithi169.stdout:8/28: dread - d7/f8 zero size 2024-09-16T00:25:54.271 INFO:tasks.workunit.client.0.smithi112.stdout:3/264: link d2/fb d2/d4d/f60 0 2024-09-16T00:25:54.272 INFO:tasks.workunit.client.0.smithi112.stdout:0/232: mkdir d5/da/d18/d24/d46 0 2024-09-16T00:25:54.272 INFO:tasks.workunit.client.1.smithi169.stdout:8/29: fsync f1 0 2024-09-16T00:25:54.272 INFO:tasks.workunit.client.1.smithi169.stdout:5/60: dwrite f0 [0,4194304] 0 2024-09-16T00:25:54.273 INFO:tasks.workunit.client.0.smithi112.stdout:8/172: write d10/d12/d1c/f38 [763100,1876] 0 2024-09-16T00:25:54.277 INFO:tasks.workunit.client.0.smithi112.stdout:7/266: creat d6/d10/d16/f66 x:0 0 0 2024-09-16T00:25:54.277 INFO:tasks.workunit.client.0.smithi112.stdout:4/188: write d6/d39/d1c/f2f [261122,63831] 0 2024-09-16T00:25:54.277 INFO:tasks.workunit.client.0.smithi112.stdout:6/199: creat dd/df/d13/f48 x:0 0 0 2024-09-16T00:25:54.278 INFO:tasks.workunit.client.1.smithi169.stdout:3/52: dread d7/fb [0,4194304] 0 2024-09-16T00:25:54.279 INFO:tasks.workunit.client.0.smithi112.stdout:1/165: truncate d0/d14/df/d28/f2d 587602 0 2024-09-16T00:25:54.279 INFO:tasks.workunit.client.0.smithi112.stdout:9/423: creat d1/d5/d6/d59/d64/fa1 x:0 0 0 2024-09-16T00:25:54.279 INFO:tasks.workunit.client.0.smithi112.stdout:5/272: link d4/d1c/l3d d4/d7/l60 0 2024-09-16T00:25:54.280 INFO:tasks.workunit.client.0.smithi112.stdout:3/265: write d2/f3f [267301,48457] 0 2024-09-16T00:25:54.280 INFO:tasks.workunit.client.0.smithi112.stdout:7/267: dread - d6/d10/d16/f59 zero size 2024-09-16T00:25:54.281 INFO:tasks.workunit.client.1.smithi169.stdout:5/61: chown d2/da/d10 3190240 1 2024-09-16T00:25:54.281 INFO:tasks.workunit.client.0.smithi112.stdout:8/173: mknod d10/d12/d1c/c39 0 2024-09-16T00:25:54.281 INFO:tasks.workunit.client.0.smithi112.stdout:7/268: stat d6/l19 0 2024-09-16T00:25:54.282 INFO:tasks.workunit.client.0.smithi112.stdout:4/189: read - d6/d39/d1c/f2b zero size 2024-09-16T00:25:54.282 INFO:tasks.workunit.client.0.smithi112.stdout:5/273: readlink d4/d7/d14/l58 0 2024-09-16T00:25:54.285 INFO:tasks.workunit.client.1.smithi169.stdout:5/62: rmdir d2/da/d10/d13 39 2024-09-16T00:25:54.286 INFO:tasks.workunit.client.0.smithi112.stdout:0/233: read d5/f23 [1302119,118254] 0 2024-09-16T00:25:54.286 INFO:tasks.workunit.client.0.smithi112.stdout:5/274: dread - d4/d7/d26/d36/f41 zero size 2024-09-16T00:25:54.286 INFO:tasks.workunit.client.0.smithi112.stdout:2/272: dread d2/dd/d48/d22/d2b/f31 [0,4194304] 0 2024-09-16T00:25:54.287 INFO:tasks.workunit.client.1.smithi169.stdout:8/30: dwrite f1 [0,4194304] 0 2024-09-16T00:25:54.287 INFO:tasks.workunit.client.0.smithi112.stdout:7/269: write d6/d10/d1b/f52 [1011813,3790] 0 2024-09-16T00:25:54.288 INFO:tasks.workunit.client.0.smithi112.stdout:3/266: symlink d2/d9/l61 0 2024-09-16T00:25:54.290 INFO:tasks.workunit.client.0.smithi112.stdout:4/190: unlink d6/d39/d1c/d2c/l14 0 2024-09-16T00:25:54.291 INFO:tasks.workunit.client.0.smithi112.stdout:5/275: write d4/f19 [355306,92321] 0 2024-09-16T00:25:54.292 INFO:tasks.workunit.client.0.smithi112.stdout:2/273: chown d2/dd/f12 44 1 2024-09-16T00:25:54.293 INFO:tasks.workunit.client.0.smithi112.stdout:6/200: getdents dd/d15 0 2024-09-16T00:25:54.293 INFO:tasks.workunit.client.1.smithi169.stdout:8/31: creat d7/fd x:0 0 0 2024-09-16T00:25:54.295 INFO:tasks.workunit.client.1.smithi169.stdout:3/53: dwrite d7/f10 [0,4194304] 0 2024-09-16T00:25:54.295 INFO:tasks.workunit.client.0.smithi112.stdout:0/234: mkdir d5/da/dc/d12/d47 0 2024-09-16T00:25:54.297 INFO:tasks.workunit.client.0.smithi112.stdout:2/274: write d2/dd/d48/f2f [847919,41073] 0 2024-09-16T00:25:54.298 INFO:tasks.workunit.client.0.smithi112.stdout:5/276: chown d4/dd/d12/d56 67859222 1 2024-09-16T00:25:54.298 INFO:tasks.workunit.client.0.smithi112.stdout:7/270: symlink d6/d10/d1b/l67 0 2024-09-16T00:25:54.299 INFO:tasks.workunit.client.1.smithi169.stdout:3/54: creat d7/f14 x:0 0 0 2024-09-16T00:25:54.299 INFO:tasks.workunit.client.1.smithi169.stdout:5/63: dread d2/fe [0,4194304] 0 2024-09-16T00:25:54.299 INFO:tasks.workunit.client.0.smithi112.stdout:3/267: creat d2/d9/d2b/d33/d49/f62 x:0 0 0 2024-09-16T00:25:54.299 INFO:tasks.workunit.client.0.smithi112.stdout:9/424: dwrite d1/d5/d6/d2d/d39/f63 [4194304,4194304] 0 2024-09-16T00:25:54.300 INFO:tasks.workunit.client.0.smithi112.stdout:4/191: mkdir d6/db/d43 0 2024-09-16T00:25:54.301 INFO:tasks.workunit.client.0.smithi112.stdout:0/235: chown d5/da/dc 77 1 2024-09-16T00:25:54.303 INFO:tasks.workunit.client.0.smithi112.stdout:8/174: rename d10/d12/d1c/c25 to d10/d12/d24/c3a 0 2024-09-16T00:25:54.303 INFO:tasks.workunit.client.0.smithi112.stdout:6/201: creat dd/df/d11/f49 x:0 0 0 2024-09-16T00:25:54.304 INFO:tasks.workunit.client.1.smithi169.stdout:5/64: mknod d2/da/c17 0 2024-09-16T00:25:54.304 INFO:tasks.workunit.client.1.smithi169.stdout:8/32: dwrite f2 [0,4194304] 0 2024-09-16T00:25:54.305 INFO:tasks.workunit.client.0.smithi112.stdout:2/275: dread d2/dd/d48/f11 [0,4194304] 0 2024-09-16T00:25:54.306 INFO:tasks.workunit.client.0.smithi112.stdout:4/192: symlink d6/d39/d1c/l44 0 2024-09-16T00:25:54.307 INFO:tasks.workunit.client.0.smithi112.stdout:9/425: symlink d1/d5/d6/d38/la2 0 2024-09-16T00:25:54.308 INFO:tasks.workunit.client.0.smithi112.stdout:0/236: mkdir d5/da/d34/d48 0 2024-09-16T00:25:54.309 INFO:tasks.workunit.client.0.smithi112.stdout:7/271: mknod d6/d10/c68 0 2024-09-16T00:25:54.310 INFO:tasks.workunit.client.0.smithi112.stdout:4/193: write d6/db/f37 [381567,98402] 0 2024-09-16T00:25:54.310 INFO:tasks.workunit.client.1.smithi169.stdout:8/33: getdents d7/dc 0 2024-09-16T00:25:54.311 INFO:tasks.workunit.client.0.smithi112.stdout:5/277: symlink d4/dd/d12/d56/l61 0 2024-09-16T00:25:54.311 INFO:tasks.workunit.client.0.smithi112.stdout:8/175: mkdir d10/d12/d1c/d3b 0 2024-09-16T00:25:54.313 INFO:tasks.workunit.client.0.smithi112.stdout:9/426: rmdir d1/d5/d6/d2d/d6a 39 2024-09-16T00:25:54.313 INFO:tasks.workunit.client.0.smithi112.stdout:2/276: fdatasync d2/f19 0 2024-09-16T00:25:54.315 INFO:tasks.workunit.client.1.smithi169.stdout:5/65: dwrite d2/f4 [0,4194304] 0 2024-09-16T00:25:54.315 INFO:tasks.workunit.client.0.smithi112.stdout:6/202: creat dd/df/d10/d1d/d37/d42/f4a x:0 0 0 2024-09-16T00:25:54.316 INFO:tasks.workunit.client.0.smithi112.stdout:5/278: rename d4/dd/d49 to d4/dd/d49/d62 22 2024-09-16T00:25:54.316 INFO:tasks.workunit.client.0.smithi112.stdout:9/427: write d1/d5/f30 [2599478,40869] 0 2024-09-16T00:25:54.317 INFO:tasks.workunit.client.1.smithi169.stdout:8/34: link f1 d7/dc/fe 0 2024-09-16T00:25:54.319 INFO:tasks.workunit.client.0.smithi112.stdout:8/176: symlink d10/l3c 0 2024-09-16T00:25:54.321 INFO:tasks.workunit.client.0.smithi112.stdout:9/428: readlink d1/d5/d20/d25/l6b 0 2024-09-16T00:25:54.322 INFO:tasks.workunit.client.0.smithi112.stdout:0/237: link d5/da/d18/d24/c22 d5/da/d1d/c49 0 2024-09-16T00:25:54.323 INFO:tasks.workunit.client.1.smithi169.stdout:8/35: rename f1 to d7/dc/ff 0 2024-09-16T00:25:54.324 INFO:tasks.workunit.client.0.smithi112.stdout:5/279: creat d4/dd/d12/d52/f63 x:0 0 0 2024-09-16T00:25:54.324 INFO:tasks.workunit.client.1.smithi169.stdout:5/66: dread d2/da/d10/d13/f14 [0,4194304] 0 2024-09-16T00:25:54.325 INFO:tasks.workunit.client.1.smithi169.stdout:8/36: creat d7/f10 x:0 0 0 2024-09-16T00:25:54.326 INFO:tasks.workunit.client.1.smithi169.stdout:0/47: truncate d0/d2/f6 1905936 0 2024-09-16T00:25:54.326 INFO:tasks.workunit.client.1.smithi169.stdout:5/67: read - d2/da/d10/f11 zero size 2024-09-16T00:25:54.328 INFO:tasks.workunit.client.0.smithi112.stdout:4/194: dwrite d6/d39/d1c/f2f [0,4194304] 0 2024-09-16T00:25:54.330 INFO:tasks.workunit.client.0.smithi112.stdout:8/177: mknod d10/c3d 0 2024-09-16T00:25:54.332 INFO:tasks.workunit.client.1.smithi169.stdout:5/68: dread d2/f8 [0,4194304] 0 2024-09-16T00:25:54.333 INFO:tasks.workunit.client.0.smithi112.stdout:0/238: rename d5/da/d21/f25 to d5/da/dc/d12/d47/f4a 0 2024-09-16T00:25:54.335 INFO:tasks.workunit.client.0.smithi112.stdout:9/429: dwrite d1/d5/d20/f93 [0,4194304] 0 2024-09-16T00:25:54.335 INFO:tasks.workunit.client.1.smithi169.stdout:8/37: dwrite d7/dc/fe [0,4194304] 0 2024-09-16T00:25:54.336 INFO:tasks.workunit.client.0.smithi112.stdout:5/280: mkdir d4/dd/d12/d52/d64 0 2024-09-16T00:25:54.337 INFO:tasks.workunit.client.1.smithi169.stdout:8/38: rename d7 to d7/dc/d11 22 2024-09-16T00:25:54.339 INFO:tasks.workunit.client.0.smithi112.stdout:2/277: getdents d2/dd/d48/d22/d2b/d30 0 2024-09-16T00:25:54.348 INFO:tasks.workunit.client.0.smithi112.stdout:1/166: write d0/f1e [350060,31277] 0 2024-09-16T00:25:54.348 INFO:tasks.workunit.client.1.smithi169.stdout:8/39: dwrite f2 [0,4194304] 0 2024-09-16T00:25:54.348 INFO:tasks.workunit.client.1.smithi169.stdout:8/40: chown d7/dc 892938 1 2024-09-16T00:25:54.350 INFO:tasks.workunit.client.0.smithi112.stdout:1/167: dread d0/d14/df/f15 [0,4194304] 0 2024-09-16T00:25:54.352 INFO:tasks.workunit.client.1.smithi169.stdout:3/55: write d7/f9 [226216,47398] 0 2024-09-16T00:25:54.352 INFO:tasks.workunit.client.0.smithi112.stdout:4/195: symlink d6/da/dc/l45 0 2024-09-16T00:25:54.352 INFO:tasks.workunit.client.0.smithi112.stdout:8/178: mknod d10/c3e 0 2024-09-16T00:25:54.359 INFO:tasks.workunit.client.1.smithi169.stdout:3/56: symlink d7/l15 0 2024-09-16T00:25:54.359 INFO:tasks.workunit.client.0.smithi112.stdout:8/179: dread - d10/f34 zero size 2024-09-16T00:25:54.359 INFO:tasks.workunit.client.0.smithi112.stdout:3/268: rmdir d2 39 2024-09-16T00:25:54.359 INFO:tasks.workunit.client.0.smithi112.stdout:8/180: dread - d10/d12/d24/f29 zero size 2024-09-16T00:25:54.359 INFO:tasks.workunit.client.0.smithi112.stdout:7/272: dread d6/d10/d16/f3a [0,4194304] 0 2024-09-16T00:25:54.360 INFO:tasks.workunit.client.0.smithi112.stdout:2/278: mkdir d2/dd/d48/d22/d2b/d54 0 2024-09-16T00:25:54.364 INFO:tasks.workunit.client.0.smithi112.stdout:0/239: rename d5/da/d1d/c49 to d5/da/d18/d24/c4b 0 2024-09-16T00:25:54.366 INFO:tasks.workunit.client.0.smithi112.stdout:2/279: dread d2/dd/d48/f11 [0,4194304] 0 2024-09-16T00:25:54.368 INFO:tasks.workunit.client.0.smithi112.stdout:7/273: dread d6/d10/d16/d3b/f4b [0,4194304] 0 2024-09-16T00:25:54.368 INFO:tasks.workunit.client.0.smithi112.stdout:1/168: mknod d0/d14/df/d2f/c34 0 2024-09-16T00:25:54.368 INFO:tasks.workunit.client.0.smithi112.stdout:8/181: dwrite d10/f1b [0,4194304] 0 2024-09-16T00:25:54.371 INFO:tasks.workunit.client.0.smithi112.stdout:3/269: chown d2/d9/d2b/d33/d49/f4c 3491309 1 2024-09-16T00:25:54.373 INFO:tasks.workunit.client.0.smithi112.stdout:9/430: getdents d1/d5/d6/d38 0 2024-09-16T00:25:54.378 INFO:tasks.workunit.client.0.smithi112.stdout:9/431: dread - d1/d5/d2c/d6f/f91 zero size 2024-09-16T00:25:54.381 INFO:tasks.workunit.client.0.smithi112.stdout:1/169: mkdir d0/d14/d1c/d26/d35 0 2024-09-16T00:25:54.383 INFO:tasks.workunit.client.0.smithi112.stdout:9/432: read - d1/f50 zero size 2024-09-16T00:25:54.386 INFO:tasks.workunit.client.0.smithi112.stdout:0/240: dwrite d5/da/dc/d12/d47/f4a [4194304,4194304] 0 2024-09-16T00:25:54.388 INFO:tasks.workunit.client.0.smithi112.stdout:3/270: rename d2/f53 to d2/d9/d2b/d2c/f63 0 2024-09-16T00:25:54.389 INFO:tasks.workunit.client.0.smithi112.stdout:2/280: dwrite d2/dd/f12 [4194304,4194304] 0 2024-09-16T00:25:54.390 INFO:tasks.workunit.client.1.smithi169.stdout:3/57: creat d7/f16 x:0 0 0 2024-09-16T00:25:54.390 INFO:tasks.workunit.client.0.smithi112.stdout:9/433: write d1/d5/f8c [2941456,72056] 0 2024-09-16T00:25:54.392 INFO:tasks.workunit.client.1.smithi169.stdout:2/28: sync 2024-09-16T00:25:54.396 INFO:tasks.workunit.client.0.smithi112.stdout:4/196: dread d6/f9 [4194304,4194304] 0 2024-09-16T00:25:54.396 INFO:tasks.workunit.client.0.smithi112.stdout:3/271: rmdir d2/d9/d2b/d2c 39 2024-09-16T00:25:54.398 INFO:tasks.workunit.client.0.smithi112.stdout:0/241: mkdir d5/da/d34/d4c 0 2024-09-16T00:25:54.399 INFO:tasks.workunit.client.1.smithi169.stdout:8/41: dwrite d7/dc/fe [4194304,4194304] 0 2024-09-16T00:25:54.399 INFO:tasks.workunit.client.0.smithi112.stdout:7/274: dread d6/d10/d16/f20 [0,4194304] 0 2024-09-16T00:25:54.406 INFO:tasks.workunit.client.0.smithi112.stdout:1/170: dwrite d0/d14/d1c/d26/f29 [4194304,4194304] 0 2024-09-16T00:25:54.406 INFO:tasks.workunit.client.1.smithi169.stdout:2/29: link f5 d7/f8 0 2024-09-16T00:25:54.407 INFO:tasks.workunit.client.0.smithi112.stdout:7/275: dread - d6/d10/d46/f5b zero size 2024-09-16T00:25:54.407 INFO:tasks.workunit.client.0.smithi112.stdout:5/281: dread d4/dd/f27 [0,4194304] 0 2024-09-16T00:25:54.411 INFO:tasks.workunit.client.0.smithi112.stdout:8/182: dwrite d10/d12/f1a [0,4194304] 0 2024-09-16T00:25:54.412 INFO:tasks.workunit.client.1.smithi169.stdout:0/48: dwrite d0/d2/f6 [0,4194304] 0 2024-09-16T00:25:54.412 INFO:tasks.workunit.client.0.smithi112.stdout:5/282: readlink d4/dd/d12/d56/l61 0 2024-09-16T00:25:54.415 INFO:tasks.workunit.client.0.smithi112.stdout:7/276: write d6/d10/d46/f33 [223480,93879] 0 2024-09-16T00:25:54.415 INFO:tasks.workunit.client.0.smithi112.stdout:8/183: write d10/f15 [3109035,115886] 0 2024-09-16T00:25:54.419 INFO:tasks.workunit.client.0.smithi112.stdout:5/283: truncate d4/dd/d12/f2f 1730221 0 2024-09-16T00:25:54.419 INFO:tasks.workunit.client.0.smithi112.stdout:3/272: mknod d2/d9/c64 0 2024-09-16T00:25:54.419 INFO:tasks.workunit.client.0.smithi112.stdout:4/197: rename d6/db/df/d3c/c3d to d6/d39/d1c/d2c/c46 0 2024-09-16T00:25:54.419 INFO:tasks.workunit.client.0.smithi112.stdout:8/184: read - d10/d12/d24/f29 zero size 2024-09-16T00:25:54.420 INFO:tasks.workunit.client.1.smithi169.stdout:5/69: dwrite d2/ff [0,4194304] 0 2024-09-16T00:25:54.421 INFO:tasks.workunit.client.0.smithi112.stdout:0/242: dread d5/da/f2b [0,4194304] 0 2024-09-16T00:25:54.421 INFO:tasks.workunit.client.0.smithi112.stdout:9/434: creat d1/fa3 x:0 0 0 2024-09-16T00:25:54.421 INFO:tasks.workunit.client.0.smithi112.stdout:5/284: stat d4/d7/f35 0 2024-09-16T00:25:54.421 INFO:tasks.workunit.client.0.smithi112.stdout:2/281: link d2/dd/f15 d2/dd/d48/d22/d2b/d54/f55 0 2024-09-16T00:25:54.422 INFO:tasks.workunit.client.1.smithi169.stdout:3/58: dwrite d7/fb [0,4194304] 0 2024-09-16T00:25:54.422 INFO:tasks.workunit.client.0.smithi112.stdout:3/273: write d2/f2d [1074608,77212] 0 2024-09-16T00:25:54.423 INFO:tasks.workunit.client.0.smithi112.stdout:1/171: creat d0/d14/df/f36 x:0 0 0 2024-09-16T00:25:54.424 INFO:tasks.workunit.client.0.smithi112.stdout:0/243: stat d5/da/d18/d24/l44 0 2024-09-16T00:25:54.424 INFO:tasks.workunit.client.0.smithi112.stdout:9/435: fsync d1/d5/d6/d38/f4f 0 2024-09-16T00:25:54.426 INFO:tasks.workunit.client.0.smithi112.stdout:1/172: fdatasync d0/d14/f8 0 2024-09-16T00:25:54.427 INFO:tasks.workunit.client.1.smithi169.stdout:8/42: dwrite d7/dc/fe [4194304,4194304] 0 2024-09-16T00:25:54.427 INFO:tasks.workunit.client.0.smithi112.stdout:3/274: stat d2/d9/d2b/d33/d49/c56 0 2024-09-16T00:25:54.429 INFO:tasks.workunit.client.0.smithi112.stdout:1/173: write d0/d14/f8 [7035951,1291] 0 2024-09-16T00:25:54.429 INFO:tasks.workunit.client.0.smithi112.stdout:5/285: mknod d4/dd/d12/d52/c65 0 2024-09-16T00:25:54.437 INFO:tasks.workunit.client.0.smithi112.stdout:0/244: creat d5/da/d34/f4d x:0 0 0 2024-09-16T00:25:54.437 INFO:tasks.workunit.client.0.smithi112.stdout:2/282: unlink d2/l5 0 2024-09-16T00:25:54.438 INFO:tasks.workunit.client.0.smithi112.stdout:9/436: chown d1/d5/d6/d2d/d6a/f6e 909357 1 2024-09-16T00:25:54.438 INFO:tasks.workunit.client.0.smithi112.stdout:0/245: read d5/da/f2b [740671,32846] 0 2024-09-16T00:25:54.438 INFO:tasks.workunit.client.0.smithi112.stdout:8/185: getdents d10/d32 0 2024-09-16T00:25:54.438 INFO:tasks.workunit.client.0.smithi112.stdout:0/246: stat d5/da/f2b 0 2024-09-16T00:25:54.440 INFO:tasks.workunit.client.1.smithi169.stdout:5/70: symlink d2/da/d10/l18 0 2024-09-16T00:25:54.440 INFO:tasks.workunit.client.0.smithi112.stdout:1/174: truncate d0/d14/df/f15 1716260 0 2024-09-16T00:25:54.441 INFO:tasks.workunit.client.1.smithi169.stdout:8/43: rename f0 to d7/f12 0 2024-09-16T00:25:54.441 INFO:tasks.workunit.client.1.smithi169.stdout:0/49: link d0/d2/d8/d5/l7 d0/d2/d8/l9 0 2024-09-16T00:25:54.442 INFO:tasks.workunit.client.1.smithi169.stdout:8/44: fsync d7/f8 0 2024-09-16T00:25:54.443 INFO:tasks.workunit.client.1.smithi169.stdout:5/71: creat d2/da/f19 x:0 0 0 2024-09-16T00:25:54.444 INFO:tasks.workunit.client.1.smithi169.stdout:3/59: dwrite d7/f9 [0,4194304] 0 2024-09-16T00:25:54.445 INFO:tasks.workunit.client.1.smithi169.stdout:5/72: chown d2/ff 1 1 2024-09-16T00:25:54.446 INFO:tasks.workunit.client.0.smithi112.stdout:3/275: dread d2/d9/d2b/d39/f3c [0,4194304] 0 2024-09-16T00:25:54.448 INFO:tasks.workunit.client.0.smithi112.stdout:8/186: creat d10/d32/f3f x:0 0 0 2024-09-16T00:25:54.449 INFO:tasks.workunit.client.0.smithi112.stdout:7/277: truncate f4 4875560 0 2024-09-16T00:25:54.450 INFO:tasks.workunit.client.0.smithi112.stdout:0/247: unlink d5/da/f45 0 2024-09-16T00:25:54.451 INFO:tasks.workunit.client.0.smithi112.stdout:1/175: creat d0/d14/df/d2f/f37 x:0 0 0 2024-09-16T00:25:54.451 INFO:tasks.workunit.client.0.smithi112.stdout:8/187: unlink lf 0 2024-09-16T00:25:54.453 INFO:tasks.workunit.client.1.smithi169.stdout:0/50: chown d0/d2/d8/l9 3825 1 2024-09-16T00:25:54.454 INFO:tasks.workunit.client.0.smithi112.stdout:9/437: dread d1/d5/d2c/f57 [0,4194304] 0 2024-09-16T00:25:54.455 INFO:tasks.workunit.client.1.smithi169.stdout:3/60: unlink d7/lf 0 2024-09-16T00:25:54.456 INFO:tasks.workunit.client.1.smithi169.stdout:5/73: dread d2/fb [0,4194304] 0 2024-09-16T00:25:54.456 INFO:tasks.workunit.client.1.smithi169.stdout:2/30: write d7/f8 [192366,112042] 0 2024-09-16T00:25:54.457 INFO:tasks.workunit.client.1.smithi169.stdout:0/51: creat d0/d2/d8/d5/fa x:0 0 0 2024-09-16T00:25:54.457 INFO:tasks.workunit.client.1.smithi169.stdout:8/45: dwrite f5 [0,4194304] 0 2024-09-16T00:25:54.458 INFO:tasks.workunit.client.0.smithi112.stdout:4/198: dread d6/db/d15/f17 [0,4194304] 0 2024-09-16T00:25:54.458 INFO:tasks.workunit.client.1.smithi169.stdout:3/61: write d7/fb [841525,76083] 0 2024-09-16T00:25:54.459 INFO:tasks.workunit.client.0.smithi112.stdout:5/286: dread f2 [0,4194304] 0 2024-09-16T00:25:54.460 INFO:tasks.workunit.client.1.smithi169.stdout:8/46: fdatasync d7/fa 0 2024-09-16T00:25:54.460 INFO:tasks.workunit.client.0.smithi112.stdout:8/188: symlink d10/d12/d24/l40 0 2024-09-16T00:25:54.461 INFO:tasks.workunit.client.0.smithi112.stdout:0/248: unlink d5/da/dc/d12/c38 0 2024-09-16T00:25:54.461 INFO:tasks.workunit.client.0.smithi112.stdout:2/283: dwrite d2/dd/d48/d22/d2b/d30/f36 [0,4194304] 0 2024-09-16T00:25:54.462 INFO:tasks.workunit.client.1.smithi169.stdout:5/74: write d2/fe [3652638,6119] 0 2024-09-16T00:25:54.463 INFO:tasks.workunit.client.0.smithi112.stdout:1/176: mknod d0/d14/c38 0 2024-09-16T00:25:54.463 INFO:tasks.workunit.client.1.smithi169.stdout:3/62: mknod d7/c17 0 2024-09-16T00:25:54.465 INFO:tasks.workunit.client.1.smithi169.stdout:8/47: mknod d7/c13 0 2024-09-16T00:25:54.466 INFO:tasks.workunit.client.0.smithi112.stdout:9/438: rmdir d1/d76 39 2024-09-16T00:25:54.466 INFO:tasks.workunit.client.0.smithi112.stdout:5/287: chown d4/dd/d12/f48 51533759 1 2024-09-16T00:25:54.466 INFO:tasks.workunit.client.1.smithi169.stdout:0/52: symlink d0/d2/lb 0 2024-09-16T00:25:54.467 INFO:tasks.workunit.client.0.smithi112.stdout:4/199: mkdir d6/da/d47 0 2024-09-16T00:25:54.469 INFO:tasks.workunit.client.0.smithi112.stdout:8/189: write d10/d12/f1a [3865409,25755] 0 2024-09-16T00:25:54.469 INFO:tasks.workunit.client.0.smithi112.stdout:3/276: dwrite d2/d9/f1c [0,4194304] 0 2024-09-16T00:25:54.471 INFO:tasks.workunit.client.0.smithi112.stdout:0/249: unlink d5/da/d18/l43 0 2024-09-16T00:25:54.472 INFO:tasks.workunit.client.1.smithi169.stdout:0/53: readlink d0/d2/d8/l9 0 2024-09-16T00:25:54.473 INFO:tasks.workunit.client.0.smithi112.stdout:4/200: write d6/db/f37 [4600525,15467] 0 2024-09-16T00:25:54.473 INFO:tasks.workunit.client.0.smithi112.stdout:5/288: fdatasync d4/dd/d12/d52/f63 0 2024-09-16T00:25:54.475 INFO:tasks.workunit.client.1.smithi169.stdout:5/75: dwrite d2/fe [0,4194304] 0 2024-09-16T00:25:54.476 INFO:tasks.workunit.client.0.smithi112.stdout:0/250: mkdir d5/da/dc/d4e 0 2024-09-16T00:25:54.477 INFO:tasks.workunit.client.0.smithi112.stdout:0/251: write d5/f33 [4116006,84421] 0 2024-09-16T00:25:54.479 INFO:tasks.workunit.client.0.smithi112.stdout:9/439: creat d1/d5/d6/fa4 x:0 0 0 2024-09-16T00:25:54.480 INFO:tasks.workunit.client.0.smithi112.stdout:1/177: getdents d0/d14/df/d24 0 2024-09-16T00:25:54.480 INFO:tasks.workunit.client.1.smithi169.stdout:0/54: rename d0/d2/d8 to d0/d2/d8/d5/dc 22 2024-09-16T00:25:54.482 INFO:tasks.workunit.client.0.smithi112.stdout:3/277: getdents d2/d9/d2b/d33/d49 0 2024-09-16T00:25:54.482 INFO:tasks.workunit.client.0.smithi112.stdout:9/440: rename d1/d5 to d1/d5/d20/d81/d1a/d88/da5 22 2024-09-16T00:25:54.482 INFO:tasks.workunit.client.1.smithi169.stdout:3/63: dwrite d7/f10 [0,4194304] 0 2024-09-16T00:25:54.482 INFO:tasks.workunit.client.0.smithi112.stdout:0/252: creat d5/da/dc/d4e/f4f x:0 0 0 2024-09-16T00:25:54.483 INFO:tasks.workunit.client.1.smithi169.stdout:8/48: dwrite f6 [0,4194304] 0 2024-09-16T00:25:54.484 INFO:tasks.workunit.client.1.smithi169.stdout:8/49: stat d7/f8 0 2024-09-16T00:25:54.485 INFO:tasks.workunit.client.1.smithi169.stdout:0/55: chown d0/d2/d8/d5/fa 214152 1 2024-09-16T00:25:54.485 INFO:tasks.workunit.client.0.smithi112.stdout:1/178: symlink d0/d14/d1c/d20/l39 0 2024-09-16T00:25:54.487 INFO:tasks.workunit.client.1.smithi169.stdout:3/64: mknod d7/c18 0 2024-09-16T00:25:54.489 INFO:tasks.workunit.client.0.smithi112.stdout:0/253: creat d5/da/d34/d4c/f50 x:0 0 0 2024-09-16T00:25:54.489 INFO:tasks.workunit.client.1.smithi169.stdout:8/50: creat d7/dc/f14 x:0 0 0 2024-09-16T00:25:54.491 INFO:tasks.workunit.client.0.smithi112.stdout:3/278: unlink d2/d9/l2a 0 2024-09-16T00:25:54.491 INFO:tasks.workunit.client.0.smithi112.stdout:3/279: fdatasync d2/f5e 0 2024-09-16T00:25:54.492 INFO:tasks.workunit.client.0.smithi112.stdout:0/254: creat d5/da/d21/f51 x:0 0 0 2024-09-16T00:25:54.492 INFO:tasks.workunit.client.1.smithi169.stdout:0/56: link d0/d2/d8/d5/fa d0/d2/fd 0 2024-09-16T00:25:54.492 INFO:tasks.workunit.client.0.smithi112.stdout:1/179: creat d0/d14/df/d2f/d30/f3a x:0 0 0 2024-09-16T00:25:54.494 INFO:tasks.workunit.client.1.smithi169.stdout:8/51: unlink l4 0 2024-09-16T00:25:54.494 INFO:tasks.workunit.client.1.smithi169.stdout:3/65: dread d7/fb [0,4194304] 0 2024-09-16T00:25:54.494 INFO:tasks.workunit.client.1.smithi169.stdout:8/52: dread - d7/dc/f14 zero size 2024-09-16T00:25:54.496 INFO:tasks.workunit.client.0.smithi112.stdout:0/255: chown d5/da/dc/d12/c3e 1 1 2024-09-16T00:25:54.497 INFO:tasks.workunit.client.1.smithi169.stdout:8/53: symlink d7/dc/l15 0 2024-09-16T00:25:54.498 INFO:tasks.workunit.client.1.smithi169.stdout:8/54: truncate d7/fa 432084 0 2024-09-16T00:25:54.501 INFO:tasks.workunit.client.1.smithi169.stdout:3/66: dread d7/f11 [4194304,4194304] 0 2024-09-16T00:25:54.501 INFO:tasks.workunit.client.1.smithi169.stdout:8/55: dread d7/fa [0,4194304] 0 2024-09-16T00:25:54.502 INFO:tasks.workunit.client.0.smithi112.stdout:3/280: dread d2/d9/d2b/d39/f3c [0,4194304] 0 2024-09-16T00:25:54.502 INFO:tasks.workunit.client.0.smithi112.stdout:7/278: write d6/d10/f1d [2982753,37812] 0 2024-09-16T00:25:54.503 INFO:tasks.workunit.client.1.smithi169.stdout:8/56: write d7/dc/f14 [578019,102101] 0 2024-09-16T00:25:54.506 INFO:tasks.workunit.client.0.smithi112.stdout:8/190: write d10/f1f [3650948,44565] 0 2024-09-16T00:25:54.506 INFO:tasks.workunit.client.0.smithi112.stdout:7/279: symlink d6/d10/d16/l69 0 2024-09-16T00:25:54.507 INFO:tasks.workunit.client.1.smithi169.stdout:8/57: symlink d7/l16 0 2024-09-16T00:25:54.508 INFO:tasks.workunit.client.0.smithi112.stdout:0/256: link d5/da/d18/d24/c30 d5/da/d18/c52 0 2024-09-16T00:25:54.508 INFO:tasks.workunit.client.0.smithi112.stdout:3/281: mknod d2/d9/d2b/c65 0 2024-09-16T00:25:54.508 INFO:tasks.workunit.client.0.smithi112.stdout:1/180: dwrite d0/d14/d1c/d20/f33 [0,4194304] 0 2024-09-16T00:25:54.510 INFO:tasks.workunit.client.1.smithi169.stdout:3/67: dwrite d7/f16 [0,4194304] 0 2024-09-16T00:25:54.512 INFO:tasks.workunit.client.1.smithi169.stdout:8/58: mkdir d7/dc/d17 0 2024-09-16T00:25:54.514 INFO:tasks.workunit.client.0.smithi112.stdout:8/191: readlink d10/d12/d24/l2c 0 2024-09-16T00:25:54.515 INFO:tasks.workunit.client.1.smithi169.stdout:3/68: symlink d7/l19 0 2024-09-16T00:25:54.515 INFO:tasks.workunit.client.1.smithi169.stdout:8/59: creat d7/f18 x:0 0 0 2024-09-16T00:25:54.515 INFO:tasks.workunit.client.1.smithi169.stdout:3/69: readlink l6 0 2024-09-16T00:25:54.516 INFO:tasks.workunit.client.1.smithi169.stdout:8/60: write f6 [807786,107117] 0 2024-09-16T00:25:54.518 INFO:tasks.workunit.client.0.smithi112.stdout:1/181: dread d0/d14/f22 [0,4194304] 0 2024-09-16T00:25:54.519 INFO:tasks.workunit.client.0.smithi112.stdout:4/201: write d6/d39/d1c/f27 [745189,103072] 0 2024-09-16T00:25:54.519 INFO:tasks.workunit.client.1.smithi169.stdout:8/61: symlink d7/dc/d17/l19 0 2024-09-16T00:25:54.520 INFO:tasks.workunit.client.0.smithi112.stdout:7/280: getdents d6/d10 0 2024-09-16T00:25:54.520 INFO:tasks.workunit.client.0.smithi112.stdout:8/192: mkdir d10/d12/d24/d41 0 2024-09-16T00:25:54.521 INFO:tasks.workunit.client.0.smithi112.stdout:1/182: chown d0/d14/df/f36 22697426 1 2024-09-16T00:25:54.522 INFO:tasks.workunit.client.0.smithi112.stdout:3/282: dread d2/d9/d2b/d39/f3c [0,4194304] 0 2024-09-16T00:25:54.522 INFO:tasks.workunit.client.1.smithi169.stdout:8/62: truncate d7/fd 1036771 0 2024-09-16T00:25:54.523 INFO:tasks.workunit.client.0.smithi112.stdout:8/193: dread - d10/d12/f33 zero size 2024-09-16T00:25:54.525 INFO:tasks.workunit.client.0.smithi112.stdout:7/281: unlink d6/l19 0 2024-09-16T00:25:54.525 INFO:tasks.workunit.client.0.smithi112.stdout:1/183: chown d0/d14/f8 7790691 1 2024-09-16T00:25:54.525 INFO:tasks.workunit.client.0.smithi112.stdout:0/257: dread d5/da/dc/d12/f3c [0,4194304] 0 2024-09-16T00:25:54.526 INFO:tasks.workunit.client.0.smithi112.stdout:8/194: read - d10/d12/d24/f29 zero size 2024-09-16T00:25:54.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:25:54 smithi169 ceph-mon[34653]: pgmap v250: 65 pgs: 65 active+clean; 455 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail; 3.9 MiB/s rd, 34 MiB/s wr, 420 op/s 2024-09-16T00:25:54.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:25:54 smithi169 ceph-mon[34653]: from='client.14600 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:25:54.527 INFO:tasks.workunit.client.0.smithi112.stdout:1/184: chown d0/d14/d1c/d20 1907089 1 2024-09-16T00:25:54.530 INFO:tasks.workunit.client.0.smithi112.stdout:8/195: fdatasync fe 0 2024-09-16T00:25:54.533 INFO:tasks.workunit.client.1.smithi169.stdout:3/70: dwrite d7/f16 [0,4194304] 0 2024-09-16T00:25:54.537 INFO:tasks.workunit.client.1.smithi169.stdout:5/76: dwrite d2/fb [8388608,4194304] 0 2024-09-16T00:25:54.537 INFO:tasks.workunit.client.1.smithi169.stdout:5/77: chown d2/da 1017906421 1 2024-09-16T00:25:54.537 INFO:tasks.workunit.client.1.smithi169.stdout:3/71: chown d7/c12 63 1 2024-09-16T00:25:54.538 INFO:tasks.workunit.client.0.smithi112.stdout:8/196: mknod d10/d12/d24/c42 0 2024-09-16T00:25:54.539 INFO:tasks.workunit.client.0.smithi112.stdout:8/197: dread - d10/f21 zero size 2024-09-16T00:25:54.540 INFO:tasks.workunit.client.0.smithi112.stdout:8/198: write d10/f21 [955645,8360] 0 2024-09-16T00:25:54.540 INFO:tasks.workunit.client.0.smithi112.stdout:7/282: rmdir d6/d10/d16/d49/d64 0 2024-09-16T00:25:54.541 INFO:tasks.workunit.client.1.smithi169.stdout:8/63: dwrite f6 [4194304,4194304] 0 2024-09-16T00:25:54.543 INFO:tasks.workunit.client.0.smithi112.stdout:8/199: read d10/f1b [1979154,88008] 0 2024-09-16T00:25:54.543 INFO:tasks.workunit.client.0.smithi112.stdout:1/185: mknod d0/d14/d1c/d26/d35/c3b 0 2024-09-16T00:25:54.544 INFO:tasks.workunit.client.0.smithi112.stdout:4/202: link d6/da/dc/l45 d6/db/d15/d31/d34/l48 0 2024-09-16T00:25:54.545 INFO:tasks.workunit.client.0.smithi112.stdout:1/186: fsync d0/d14/f11 0 2024-09-16T00:25:54.547 INFO:tasks.workunit.client.1.smithi169.stdout:8/64: write d7/dc/ff [9079782,746] 0 2024-09-16T00:25:54.547 INFO:tasks.workunit.client.0.smithi112.stdout:0/258: link d5/da/dc/d12/d47/f4a d5/da/f53 0 2024-09-16T00:25:54.547 INFO:tasks.workunit.client.0.smithi112.stdout:4/203: write d6/db/d15/d31/d34/f3f [357620,75969] 0 2024-09-16T00:25:54.549 INFO:tasks.workunit.client.1.smithi169.stdout:5/78: dwrite d2/ff [0,4194304] 0 2024-09-16T00:25:54.550 INFO:tasks.workunit.client.0.smithi112.stdout:7/283: rename d6/d10/d16/d3b/c44 to d6/d10/c6a 0 2024-09-16T00:25:54.552 INFO:tasks.workunit.client.0.smithi112.stdout:1/187: write d0/d14/d1c/d20/f33 [3717323,32519] 0 2024-09-16T00:25:54.552 INFO:tasks.workunit.client.0.smithi112.stdout:3/283: getdents d2/d9/d2b/d2c 0 2024-09-16T00:25:54.553 INFO:tasks.workunit.client.1.smithi169.stdout:8/65: mknod d7/dc/c1a 0 2024-09-16T00:25:54.558 INFO:tasks.workunit.client.1.smithi169.stdout:5/79: symlink d2/da/d10/l1a 0 2024-09-16T00:25:54.558 INFO:tasks.workunit.client.1.smithi169.stdout:5/80: chown d2 90148654 1 2024-09-16T00:25:54.559 INFO:tasks.workunit.client.1.smithi169.stdout:8/66: dread d7/dc/fe [0,4194304] 0 2024-09-16T00:25:54.559 INFO:tasks.workunit.client.0.smithi112.stdout:3/284: mkdir d2/d9/d2b/d39/d66 0 2024-09-16T00:25:54.559 INFO:tasks.workunit.client.0.smithi112.stdout:4/204: dread d6/db/f37 [0,4194304] 0 2024-09-16T00:25:54.560 INFO:tasks.workunit.client.0.smithi112.stdout:1/188: dread - d0/d14/df/d2f/d30/f3a zero size 2024-09-16T00:25:54.560 INFO:tasks.workunit.client.1.smithi169.stdout:5/81: symlink d2/da/l1b 0 2024-09-16T00:25:54.561 INFO:tasks.workunit.client.0.smithi112.stdout:8/200: rmdir d10/d12/d24/d41 0 2024-09-16T00:25:54.562 INFO:tasks.workunit.client.0.smithi112.stdout:3/285: chown d2/f3f 11 1 2024-09-16T00:25:54.563 INFO:tasks.workunit.client.1.smithi169.stdout:3/72: truncate d7/f16 1524652 0 2024-09-16T00:25:54.563 INFO:tasks.workunit.client.1.smithi169.stdout:5/82: rmdir d2/da/d10/d13 39 2024-09-16T00:25:54.564 INFO:tasks.workunit.client.0.smithi112.stdout:0/259: dwrite d5/da/dc/d12/f3f [0,4194304] 0 2024-09-16T00:25:54.566 INFO:tasks.workunit.client.1.smithi169.stdout:9/49: sync 2024-09-16T00:25:54.566 INFO:tasks.workunit.client.1.smithi169.stdout:1/106: sync 2024-09-16T00:25:54.566 INFO:tasks.workunit.client.1.smithi169.stdout:6/28: sync 2024-09-16T00:25:54.566 INFO:tasks.workunit.client.1.smithi169.stdout:6/29: write - no filename 2024-09-16T00:25:54.568 INFO:tasks.workunit.client.1.smithi169.stdout:8/67: dwrite f2 [0,4194304] 0 2024-09-16T00:25:54.568 INFO:tasks.workunit.client.0.smithi112.stdout:0/260: read d5/da/dc/d12/f3c [255909,8777] 0 2024-09-16T00:25:54.568 INFO:tasks.workunit.client.0.smithi112.stdout:8/201: rename c2 to d10/d12/d24/c43 0 2024-09-16T00:25:54.569 INFO:tasks.workunit.client.0.smithi112.stdout:4/205: unlink d6/db/df/c20 0 2024-09-16T00:25:54.569 INFO:tasks.workunit.client.0.smithi112.stdout:3/286: dread d2/fb [0,4194304] 0 2024-09-16T00:25:54.569 INFO:tasks.workunit.client.0.smithi112.stdout:0/261: stat d5/da/d18/c31 0 2024-09-16T00:25:54.569 INFO:tasks.workunit.client.1.smithi169.stdout:9/50: chown d5/c9 3022141 1 2024-09-16T00:25:54.569 INFO:tasks.workunit.client.1.smithi169.stdout:9/51: chown d5 0 1 2024-09-16T00:25:54.569 INFO:tasks.workunit.client.1.smithi169.stdout:1/107: fdatasync f7 0 2024-09-16T00:25:54.570 INFO:tasks.workunit.client.0.smithi112.stdout:0/262: chown d5/da/d18/d24/l44 4668 1 2024-09-16T00:25:54.571 INFO:tasks.workunit.client.1.smithi169.stdout:9/52: chown c3 32671784 1 2024-09-16T00:25:54.571 INFO:tasks.workunit.client.1.smithi169.stdout:1/108: fsync dc/f15 0 2024-09-16T00:25:54.573 INFO:tasks.workunit.client.0.smithi112.stdout:0/263: read d5/da/dc/d12/f3f [1789198,99089] 0 2024-09-16T00:25:54.575 INFO:tasks.workunit.client.0.smithi112.stdout:4/206: dread d6/db/d15/d31/d34/f3f [0,4194304] 0 2024-09-16T00:25:54.576 INFO:tasks.workunit.client.1.smithi169.stdout:6/30: symlink d2/l9 0 2024-09-16T00:25:54.576 INFO:tasks.workunit.client.0.smithi112.stdout:4/207: stat d6/d39/d1c/l3b 0 2024-09-16T00:25:54.577 INFO:tasks.workunit.client.1.smithi169.stdout:6/31: chown d2/c8 53858 1 2024-09-16T00:25:54.577 INFO:tasks.workunit.client.1.smithi169.stdout:8/68: creat d7/dc/d17/f1b x:0 0 0 2024-09-16T00:25:54.578 INFO:tasks.workunit.client.1.smithi169.stdout:5/83: mkdir d2/da/d10/d13/d1c 0 2024-09-16T00:25:54.578 INFO:tasks.workunit.client.0.smithi112.stdout:3/287: creat d2/d9/d2b/d39/f67 x:0 0 0 2024-09-16T00:25:54.579 INFO:tasks.workunit.client.1.smithi169.stdout:1/109: dread dc/f1d [0,4194304] 0 2024-09-16T00:25:54.579 INFO:tasks.workunit.client.1.smithi169.stdout:5/84: readlink d2/da/l1b 0 2024-09-16T00:25:54.580 INFO:tasks.workunit.client.1.smithi169.stdout:6/32: mknod d2/ca 0 2024-09-16T00:25:54.580 INFO:tasks.workunit.client.1.smithi169.stdout:6/33: truncate - no filename 2024-09-16T00:25:54.581 INFO:tasks.workunit.client.1.smithi169.stdout:8/69: rename d7/dc/d17 to d7/dc/d17/d1c 22 2024-09-16T00:25:54.584 INFO:tasks.workunit.client.1.smithi169.stdout:6/34: creat d2/fb x:0 0 0 2024-09-16T00:25:54.585 INFO:tasks.workunit.client.1.smithi169.stdout:8/70: mkdir d7/dc/d1d 0 2024-09-16T00:25:54.586 INFO:tasks.workunit.client.0.smithi112.stdout:4/208: rename d6/db/d15 to d6/db/d1f/d41/d49 0 2024-09-16T00:25:54.587 INFO:tasks.workunit.client.0.smithi112.stdout:4/209: fsync d6/da/f40 0 2024-09-16T00:25:54.589 INFO:tasks.workunit.client.1.smithi169.stdout:5/85: link d2/ff d2/da/d10/d13/f1d 0 2024-09-16T00:25:54.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:25:54 smithi112 ceph-mon[27869]: pgmap v250: 65 pgs: 65 active+clean; 455 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail; 3.9 MiB/s rd, 34 MiB/s wr, 420 op/s 2024-09-16T00:25:54.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:25:54 smithi112 ceph-mon[27869]: from='client.14600 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:25:54.589 INFO:tasks.workunit.client.1.smithi169.stdout:8/71: write d7/f8 [647315,81069] 0 2024-09-16T00:25:54.590 INFO:tasks.workunit.client.0.smithi112.stdout:7/284: chown d6/d10/d46/f5b 341 1 2024-09-16T00:25:54.590 INFO:tasks.workunit.client.1.smithi169.stdout:9/53: write d5/d12/f13 [261872,105750] 0 2024-09-16T00:25:54.591 INFO:tasks.workunit.client.1.smithi169.stdout:3/73: dwrite d7/f16 [0,4194304] 0 2024-09-16T00:25:54.591 INFO:tasks.workunit.client.0.smithi112.stdout:3/288: truncate d2/d4d/f60 1427423 0 2024-09-16T00:25:54.593 INFO:tasks.workunit.client.1.smithi169.stdout:6/35: creat d2/fc x:0 0 0 2024-09-16T00:25:54.594 INFO:tasks.workunit.client.1.smithi169.stdout:6/36: chown d2 1233 1 2024-09-16T00:25:54.595 INFO:tasks.workunit.client.0.smithi112.stdout:4/210: truncate d6/db/d1f/d41/d49/d31/d34/f3f 304474 0 2024-09-16T00:25:54.596 INFO:tasks.workunit.client.1.smithi169.stdout:1/110: dread f7 [0,4194304] 0 2024-09-16T00:25:54.596 INFO:tasks.workunit.client.1.smithi169.stdout:9/54: unlink d5/fb 0 2024-09-16T00:25:54.596 INFO:tasks.workunit.client.0.smithi112.stdout:1/189: dwrite d0/f9 [4194304,4194304] 0 2024-09-16T00:25:54.597 INFO:tasks.workunit.client.1.smithi169.stdout:5/86: mknod d2/da/c1e 0 2024-09-16T00:25:54.597 INFO:tasks.workunit.client.0.smithi112.stdout:8/202: getdents d10/d32 0 2024-09-16T00:25:54.597 INFO:tasks.workunit.client.1.smithi169.stdout:1/111: fdatasync dc/fd 0 2024-09-16T00:25:54.598 INFO:tasks.workunit.client.1.smithi169.stdout:8/72: symlink d7/l1e 0 2024-09-16T00:25:54.600 INFO:tasks.workunit.client.0.smithi112.stdout:3/289: mkdir d2/d9/d2b/d33/d68 0 2024-09-16T00:25:54.600 INFO:tasks.workunit.client.1.smithi169.stdout:9/55: creat d5/f15 x:0 0 0 2024-09-16T00:25:54.602 INFO:tasks.workunit.client.0.smithi112.stdout:4/211: dread d6/db/d1f/d41/d49/d31/d34/f3e [0,4194304] 0 2024-09-16T00:25:54.604 INFO:tasks.workunit.client.0.smithi112.stdout:0/264: getdents d5/da/dc 0 2024-09-16T00:25:54.606 INFO:tasks.workunit.client.0.smithi112.stdout:4/212: fdatasync d6/da/f35 0 2024-09-16T00:25:54.607 INFO:tasks.workunit.client.1.smithi169.stdout:3/74: dwrite d7/f10 [4194304,4194304] 0 2024-09-16T00:25:54.609 INFO:tasks.workunit.client.0.smithi112.stdout:4/213: truncate d6/da/f40 428741 0 2024-09-16T00:25:54.610 INFO:tasks.workunit.client.0.smithi112.stdout:1/190: dwrite d0/d14/fd [4194304,4194304] 0 2024-09-16T00:25:54.610 INFO:tasks.workunit.client.0.smithi112.stdout:3/290: fdatasync d2/d9/d2b/d47/f30 0 2024-09-16T00:25:54.611 INFO:tasks.workunit.client.1.smithi169.stdout:5/87: creat d2/f1f x:0 0 0 2024-09-16T00:25:54.612 INFO:tasks.workunit.client.0.smithi112.stdout:0/265: creat d5/da/d18/f54 x:0 0 0 2024-09-16T00:25:54.613 INFO:tasks.workunit.client.0.smithi112.stdout:1/191: stat d0/d14/d1c/d20/f33 0 2024-09-16T00:25:54.613 INFO:tasks.workunit.client.1.smithi169.stdout:6/37: dwrite d2/fc [0,4194304] 0 2024-09-16T00:25:54.614 INFO:tasks.workunit.client.0.smithi112.stdout:4/214: write d6/f9 [2178824,117260] 0 2024-09-16T00:25:54.617 INFO:tasks.workunit.client.0.smithi112.stdout:3/291: dread d2/f2d [0,4194304] 0 2024-09-16T00:25:54.617 INFO:tasks.workunit.client.0.smithi112.stdout:4/215: fdatasync d6/da/f40 0 2024-09-16T00:25:54.617 INFO:tasks.workunit.client.1.smithi169.stdout:3/75: mkdir d7/d1a 0 2024-09-16T00:25:54.617 INFO:tasks.workunit.client.1.smithi169.stdout:8/73: dread f6 [4194304,4194304] 0 2024-09-16T00:25:54.618 INFO:tasks.workunit.client.0.smithi112.stdout:4/216: chown d6/d39/d1c/d2c 29164 1 2024-09-16T00:25:54.619 INFO:tasks.workunit.client.0.smithi112.stdout:1/192: write d0/d14/d1c/d20/f21 [7264065,16629] 0 2024-09-16T00:25:54.619 INFO:tasks.workunit.client.0.smithi112.stdout:3/292: stat d2/d9/d2b/d33/d49/f62 0 2024-09-16T00:25:54.621 INFO:tasks.workunit.client.1.smithi169.stdout:8/74: chown d7/dc/l15 1245992 1 2024-09-16T00:25:54.623 INFO:tasks.workunit.client.0.smithi112.stdout:0/266: write d5/da/dc/d12/f3f [1356335,37429] 0 2024-09-16T00:25:54.623 INFO:tasks.workunit.client.0.smithi112.stdout:8/203: write d10/d12/f16 [2195527,77803] 0 2024-09-16T00:25:54.625 INFO:tasks.workunit.client.0.smithi112.stdout:0/267: truncate d5/da/dc/f10 5065354 0 2024-09-16T00:25:54.626 INFO:tasks.workunit.client.0.smithi112.stdout:1/193: mknod d0/d14/d1c/c3c 0 2024-09-16T00:25:54.628 INFO:tasks.workunit.client.1.smithi169.stdout:8/75: read d7/fa [166849,5861] 0 2024-09-16T00:25:54.629 INFO:tasks.workunit.client.0.smithi112.stdout:0/268: write d5/da/f27 [1333799,8169] 0 2024-09-16T00:25:54.630 INFO:tasks.workunit.client.0.smithi112.stdout:8/204: unlink d10/d12/d1c/f1d 0 2024-09-16T00:25:54.631 INFO:tasks.workunit.client.1.smithi169.stdout:8/76: readlink d7/dc/d17/l19 0 2024-09-16T00:25:54.632 INFO:tasks.workunit.client.1.smithi169.stdout:6/38: dwrite d2/fb [0,4194304] 0 2024-09-16T00:25:54.633 INFO:tasks.workunit.client.0.smithi112.stdout:7/285: dread d6/d10/d16/f4e [0,4194304] 0 2024-09-16T00:25:54.633 INFO:tasks.workunit.client.0.smithi112.stdout:0/269: read - d5/da/d18/f54 zero size 2024-09-16T00:25:54.634 INFO:tasks.workunit.client.0.smithi112.stdout:8/205: fsync d10/f15 0 2024-09-16T00:25:54.634 INFO:tasks.workunit.client.1.smithi169.stdout:3/76: creat d7/d1a/f1b x:0 0 0 2024-09-16T00:25:54.634 INFO:tasks.workunit.client.1.smithi169.stdout:3/77: chown d7/d1a 2953922 1 2024-09-16T00:25:54.637 INFO:tasks.workunit.client.1.smithi169.stdout:5/88: creat d2/f20 x:0 0 0 2024-09-16T00:25:54.637 INFO:tasks.workunit.client.1.smithi169.stdout:8/77: write d7/dc/fe [7490111,55223] 0 2024-09-16T00:25:54.637 INFO:tasks.workunit.client.1.smithi169.stdout:3/78: chown d7/l19 310352 1 2024-09-16T00:25:54.641 INFO:tasks.workunit.client.1.smithi169.stdout:7/77: sync 2024-09-16T00:25:54.642 INFO:tasks.workunit.client.0.smithi112.stdout:3/293: getdents d2 0 2024-09-16T00:25:54.642 INFO:tasks.workunit.client.0.smithi112.stdout:7/286: write d6/d10/d46/f36 [2455589,31402] 0 2024-09-16T00:25:54.642 INFO:tasks.workunit.client.0.smithi112.stdout:1/194: dwrite d0/f1e [4194304,4194304] 0 2024-09-16T00:25:54.642 INFO:tasks.workunit.client.0.smithi112.stdout:7/287: stat d6/f2a 0 2024-09-16T00:25:54.643 INFO:tasks.workunit.client.1.smithi169.stdout:9/56: write d5/da/fd [507657,5667] 0 2024-09-16T00:25:54.643 INFO:tasks.workunit.client.1.smithi169.stdout:5/89: write d2/fe [5198916,16854] 0 2024-09-16T00:25:54.645 INFO:tasks.workunit.client.1.smithi169.stdout:7/78: truncate d7/f8 5202418 0 2024-09-16T00:25:54.647 INFO:tasks.workunit.client.0.smithi112.stdout:8/206: rename d10/d12/d24/l26 to d10/d32/l44 0 2024-09-16T00:25:54.649 INFO:tasks.workunit.client.1.smithi169.stdout:9/57: creat d5/da/f16 x:0 0 0 2024-09-16T00:25:54.649 INFO:tasks.workunit.client.0.smithi112.stdout:8/207: readlink d10/d12/d24/l40 0 2024-09-16T00:25:54.650 INFO:tasks.workunit.client.0.smithi112.stdout:3/294: dwrite d2/d9/d2b/d47/f54 [0,4194304] 0 2024-09-16T00:25:54.650 INFO:tasks.workunit.client.1.smithi169.stdout:6/39: dwrite d2/fc [4194304,4194304] 0 2024-09-16T00:25:54.651 INFO:tasks.workunit.client.1.smithi169.stdout:7/79: mknod d7/c13 0 2024-09-16T00:25:54.651 INFO:tasks.workunit.client.0.smithi112.stdout:8/208: stat d10/d32/c37 0 2024-09-16T00:25:54.651 INFO:tasks.workunit.client.0.smithi112.stdout:7/288: creat d6/d10/f6b x:0 0 0 2024-09-16T00:25:54.652 INFO:tasks.workunit.client.0.smithi112.stdout:8/209: readlink d10/d12/d24/l2a 0 2024-09-16T00:25:54.653 INFO:tasks.workunit.client.0.smithi112.stdout:1/195: dread d0/f1e [0,4194304] 0 2024-09-16T00:25:54.653 INFO:tasks.workunit.client.1.smithi169.stdout:5/90: fsync d2/ff 0 2024-09-16T00:25:54.656 INFO:tasks.workunit.client.1.smithi169.stdout:9/58: creat d5/f17 x:0 0 0 2024-09-16T00:25:54.657 INFO:tasks.workunit.client.0.smithi112.stdout:3/295: unlink d2/f1f 0 2024-09-16T00:25:54.661 INFO:tasks.workunit.client.0.smithi112.stdout:1/196: unlink d0/d14/c38 0 2024-09-16T00:25:54.661 INFO:tasks.workunit.client.1.smithi169.stdout:3/79: dwrite d7/f10 [0,4194304] 0 2024-09-16T00:25:54.662 INFO:tasks.workunit.client.1.smithi169.stdout:1/112: truncate dc/df/f12 17362 0 2024-09-16T00:25:54.664 INFO:tasks.workunit.client.1.smithi169.stdout:5/91: readlink d2/da/d10/l18 0 2024-09-16T00:25:54.665 INFO:tasks.workunit.client.1.smithi169.stdout:8/78: dwrite d7/fd [0,4194304] 0 2024-09-16T00:25:54.665 INFO:tasks.workunit.client.0.smithi112.stdout:7/289: rename d6/d10/d46 to d6/d10/d16/d49/d6c 0 2024-09-16T00:25:54.666 INFO:tasks.workunit.client.0.smithi112.stdout:7/290: readlink d6/l37 0 2024-09-16T00:25:54.667 INFO:tasks.workunit.client.0.smithi112.stdout:4/217: dread d6/db/f25 [0,4194304] 0 2024-09-16T00:25:54.667 INFO:tasks.workunit.client.0.smithi112.stdout:1/197: creat d0/d14/df/f3d x:0 0 0 2024-09-16T00:25:54.669 INFO:tasks.workunit.client.1.smithi169.stdout:6/40: write d2/fc [8486483,41360] 0 2024-09-16T00:25:54.669 INFO:tasks.workunit.client.0.smithi112.stdout:4/218: chown d6/da/dc/c24 1950 1 2024-09-16T00:25:54.669 INFO:tasks.workunit.client.0.smithi112.stdout:7/291: creat d6/d10/d16/d3b/f6d x:0 0 0 2024-09-16T00:25:54.671 INFO:tasks.workunit.client.0.smithi112.stdout:4/219: dread d6/da/f35 [0,4194304] 0 2024-09-16T00:25:54.672 INFO:tasks.workunit.client.1.smithi169.stdout:3/80: mkdir d7/d1a/d1c 0 2024-09-16T00:25:54.673 INFO:tasks.workunit.client.0.smithi112.stdout:1/198: rename d0/d14/df/f15 to d0/d14/d1c/d26/f3e 0 2024-09-16T00:25:54.674 INFO:tasks.workunit.client.0.smithi112.stdout:4/220: dread d6/da/f40 [0,4194304] 0 2024-09-16T00:25:54.674 INFO:tasks.workunit.client.0.smithi112.stdout:6/203: sync 2024-09-16T00:25:54.675 INFO:tasks.workunit.client.1.smithi169.stdout:4/57: sync 2024-09-16T00:25:54.675 INFO:tasks.workunit.client.0.smithi112.stdout:0/270: write d5/f33 [5536334,116099] 0 2024-09-16T00:25:54.676 INFO:tasks.workunit.client.1.smithi169.stdout:8/79: mknod d7/dc/d1d/c1f 0 2024-09-16T00:25:54.676 INFO:tasks.workunit.client.0.smithi112.stdout:6/204: chown dd/df/d10/d1d/d37 8638 1 2024-09-16T00:25:54.677 INFO:tasks.workunit.client.0.smithi112.stdout:0/271: readlink d5/da/d34/l36 0 2024-09-16T00:25:54.677 INFO:tasks.workunit.client.0.smithi112.stdout:4/221: write d6/db/f42 [83940,19165] 0 2024-09-16T00:25:54.678 INFO:tasks.workunit.client.1.smithi169.stdout:2/31: sync 2024-09-16T00:25:54.678 INFO:tasks.workunit.client.1.smithi169.stdout:0/57: sync 2024-09-16T00:25:54.678 INFO:tasks.workunit.client.1.smithi169.stdout:4/58: fsync d2/f10 0 2024-09-16T00:25:54.679 INFO:tasks.workunit.client.0.smithi112.stdout:1/199: symlink d0/d14/df/d24/l3f 0 2024-09-16T00:25:54.679 INFO:tasks.workunit.client.0.smithi112.stdout:8/210: rename d10/d32 to d10/d45 0 2024-09-16T00:25:54.680 INFO:tasks.workunit.client.1.smithi169.stdout:5/92: write d2/ff [1371100,33823] 0 2024-09-16T00:25:54.681 INFO:tasks.workunit.client.0.smithi112.stdout:0/272: symlink d5/da/dc/l55 0 2024-09-16T00:25:54.682 INFO:tasks.workunit.client.1.smithi169.stdout:8/80: mkdir d7/d20 0 2024-09-16T00:25:54.683 INFO:tasks.workunit.client.1.smithi169.stdout:2/32: write f2 [334705,101427] 0 2024-09-16T00:25:54.684 INFO:tasks.workunit.client.1.smithi169.stdout:3/81: write d7/fd [9110891,70621] 0 2024-09-16T00:25:54.684 INFO:tasks.workunit.client.1.smithi169.stdout:1/113: dread dc/df/f17 [0,4194304] 0 2024-09-16T00:25:54.685 INFO:tasks.workunit.client.0.smithi112.stdout:4/222: dread d6/db/d1f/d41/d49/d31/d34/f3f [0,4194304] 0 2024-09-16T00:25:54.687 INFO:tasks.workunit.client.0.smithi112.stdout:6/205: mkdir dd/df/d4b 0 2024-09-16T00:25:54.687 INFO:tasks.workunit.client.0.smithi112.stdout:1/200: write d0/d14/d1c/d20/f33 [2763868,74094] 0 2024-09-16T00:25:54.688 INFO:tasks.workunit.client.0.smithi112.stdout:6/206: dread - dd/df/d11/f49 zero size 2024-09-16T00:25:54.689 INFO:tasks.workunit.client.0.smithi112.stdout:0/273: rename d5/da/d18/f3d to d5/da/d1d/f56 0 2024-09-16T00:25:54.690 INFO:tasks.workunit.client.1.smithi169.stdout:7/80: dread d7/da/fc [4194304,4194304] 0 2024-09-16T00:25:54.690 INFO:tasks.workunit.client.0.smithi112.stdout:0/274: truncate f0 4683448 0 2024-09-16T00:25:54.690 INFO:tasks.workunit.client.0.smithi112.stdout:3/296: mkdir d2/d9/d2b/d2c/d69 0 2024-09-16T00:25:54.691 INFO:tasks.workunit.client.0.smithi112.stdout:0/275: truncate d5/f2a 999448 0 2024-09-16T00:25:54.691 INFO:tasks.workunit.client.0.smithi112.stdout:4/223: creat d6/d39/f4a x:0 0 0 2024-09-16T00:25:54.691 INFO:tasks.workunit.client.1.smithi169.stdout:9/59: dwrite d5/da/fd [0,4194304] 0 2024-09-16T00:25:54.692 INFO:tasks.workunit.client.1.smithi169.stdout:0/58: unlink d0/d2/d8/d5/l7 0 2024-09-16T00:25:54.692 INFO:tasks.workunit.client.0.smithi112.stdout:8/211: creat d10/f46 x:0 0 0 2024-09-16T00:25:54.694 INFO:tasks.workunit.client.0.smithi112.stdout:1/201: symlink d0/d14/df/d28/l40 0 2024-09-16T00:25:54.694 INFO:tasks.workunit.client.1.smithi169.stdout:3/82: mknod d7/d1a/c1d 0 2024-09-16T00:25:54.695 INFO:tasks.workunit.client.1.smithi169.stdout:1/114: stat dc/f1d 0 2024-09-16T00:25:54.695 INFO:tasks.workunit.client.0.smithi112.stdout:0/276: readlink d5/l35 0 2024-09-16T00:25:54.697 INFO:tasks.workunit.client.1.smithi169.stdout:6/41: dwrite d2/fc [4194304,4194304] 0 2024-09-16T00:25:54.698 INFO:tasks.workunit.client.1.smithi169.stdout:4/59: symlink d2/d12/d13/l15 0 2024-09-16T00:25:54.698 INFO:tasks.workunit.client.0.smithi112.stdout:8/212: symlink d10/d45/l47 0 2024-09-16T00:25:54.699 INFO:tasks.workunit.client.1.smithi169.stdout:8/81: write d7/fa [816137,85395] 0 2024-09-16T00:25:54.700 INFO:tasks.workunit.client.1.smithi169.stdout:9/60: mknod d5/da/c18 0 2024-09-16T00:25:54.700 INFO:tasks.workunit.client.0.smithi112.stdout:4/224: mknod d6/da/d47/c4b 0 2024-09-16T00:25:54.700 INFO:tasks.workunit.client.0.smithi112.stdout:6/207: creat dd/df/f4c x:0 0 0 2024-09-16T00:25:54.702 INFO:tasks.workunit.client.0.smithi112.stdout:1/202: mkdir d0/d14/d1c/d26/d41 0 2024-09-16T00:25:54.703 INFO:tasks.workunit.client.1.smithi169.stdout:9/61: chown c4 1898202 1 2024-09-16T00:25:54.703 INFO:tasks.workunit.client.0.smithi112.stdout:8/213: creat d10/d12/d24/f48 x:0 0 0 2024-09-16T00:25:54.704 INFO:tasks.workunit.client.1.smithi169.stdout:2/33: mknod d7/c9 0 2024-09-16T00:25:54.704 INFO:tasks.workunit.client.0.smithi112.stdout:3/297: dwrite d2/d9/d2b/d33/d49/f4c [0,4194304] 0 2024-09-16T00:25:54.705 INFO:tasks.workunit.client.0.smithi112.stdout:0/277: dread d5/f2a [0,4194304] 0 2024-09-16T00:25:54.706 INFO:tasks.workunit.client.1.smithi169.stdout:3/83: creat d7/f1e x:0 0 0 2024-09-16T00:25:54.706 INFO:tasks.workunit.client.0.smithi112.stdout:7/292: dread d6/d10/d16/d49/d6c/f38 [0,4194304] 0 2024-09-16T00:25:54.707 INFO:tasks.workunit.client.1.smithi169.stdout:1/115: truncate dc/df/f1c 813267 0 2024-09-16T00:25:54.707 INFO:tasks.workunit.client.1.smithi169.stdout:7/81: rename d7/f9 to d7/da/f14 0 2024-09-16T00:25:54.710 INFO:tasks.workunit.client.0.smithi112.stdout:1/203: write d0/d14/d1c/d20/f33 [1679419,90913] 0 2024-09-16T00:25:54.710 INFO:tasks.workunit.client.1.smithi169.stdout:6/42: mknod d2/cd 0 2024-09-16T00:25:54.710 INFO:tasks.workunit.client.1.smithi169.stdout:5/93: dwrite d2/da/d10/d13/f1d [0,4194304] 0 2024-09-16T00:25:54.710 INFO:tasks.workunit.client.0.smithi112.stdout:4/225: creat d6/db/d1f/d41/d49/f4c x:0 0 0 2024-09-16T00:25:54.712 INFO:tasks.workunit.client.1.smithi169.stdout:7/82: truncate d7/da/f14 550966 0 2024-09-16T00:25:54.712 INFO:tasks.workunit.client.0.smithi112.stdout:8/214: fdatasync d10/d12/d24/f2e 0 2024-09-16T00:25:54.712 INFO:tasks.workunit.client.0.smithi112.stdout:1/204: readlink d0/d14/df/d28/l40 0 2024-09-16T00:25:54.713 INFO:tasks.workunit.client.0.smithi112.stdout:1/205: read - d0/d14/df/d28/f32 zero size 2024-09-16T00:25:54.714 INFO:tasks.workunit.client.0.smithi112.stdout:4/226: stat d6/da/c30 0 2024-09-16T00:25:54.714 INFO:tasks.workunit.client.1.smithi169.stdout:0/59: creat d0/d2/d8/fe x:0 0 0 2024-09-16T00:25:54.716 INFO:tasks.workunit.client.1.smithi169.stdout:1/116: creat dc/d18/f20 x:0 0 0 2024-09-16T00:25:54.716 INFO:tasks.workunit.client.1.smithi169.stdout:3/84: dread d7/f9 [0,4194304] 0 2024-09-16T00:25:54.716 INFO:tasks.workunit.client.0.smithi112.stdout:1/206: write d0/d14/fd [5306142,18933] 0 2024-09-16T00:25:54.717 INFO:tasks.workunit.client.1.smithi169.stdout:7/83: unlink l0 0 2024-09-16T00:25:54.717 INFO:tasks.workunit.client.0.smithi112.stdout:3/298: unlink d2/d9/d2b/d33/d49/f62 0 2024-09-16T00:25:54.717 INFO:tasks.workunit.client.0.smithi112.stdout:4/227: write d6/d39/d1c/f2f [1731117,87550] 0 2024-09-16T00:25:54.720 INFO:tasks.workunit.client.0.smithi112.stdout:3/299: chown d2/d9/d2b/d33/f50 9223 1 2024-09-16T00:25:54.720 INFO:tasks.workunit.client.1.smithi169.stdout:6/43: chown d2/c5 28056238 1 2024-09-16T00:25:54.721 INFO:tasks.workunit.client.0.smithi112.stdout:0/278: mkdir d5/da/d57 0 2024-09-16T00:25:54.721 INFO:tasks.workunit.client.1.smithi169.stdout:5/94: creat d2/da/d10/d13/f21 x:0 0 0 2024-09-16T00:25:54.722 INFO:tasks.workunit.client.0.smithi112.stdout:7/293: truncate f4 5377521 0 2024-09-16T00:25:54.723 INFO:tasks.workunit.client.1.smithi169.stdout:7/84: readlink d7/le 0 2024-09-16T00:25:54.724 INFO:tasks.workunit.client.1.smithi169.stdout:5/95: write d2/da/d10/d13/f21 [390663,27724] 0 2024-09-16T00:25:54.725 INFO:tasks.workunit.client.0.smithi112.stdout:8/215: dwrite d10/f21 [0,4194304] 0 2024-09-16T00:25:54.725 INFO:tasks.workunit.client.1.smithi169.stdout:6/44: mknod d2/ce 0 2024-09-16T00:25:54.727 INFO:tasks.workunit.client.0.smithi112.stdout:8/216: write d10/d12/d1c/f31 [115165,120729] 0 2024-09-16T00:25:54.727 INFO:tasks.workunit.client.1.smithi169.stdout:5/96: chown f0 0 1 2024-09-16T00:25:54.728 INFO:tasks.workunit.client.1.smithi169.stdout:8/82: getdents d7/dc/d1d 0 2024-09-16T00:25:54.728 INFO:tasks.workunit.client.1.smithi169.stdout:1/117: dwrite dc/fd [0,4194304] 0 2024-09-16T00:25:54.729 INFO:tasks.workunit.client.0.smithi112.stdout:0/279: symlink d5/da/d21/l58 0 2024-09-16T00:25:54.730 INFO:tasks.workunit.client.1.smithi169.stdout:3/85: rename l4 to d7/l1f 0 2024-09-16T00:25:54.732 INFO:tasks.workunit.client.0.smithi112.stdout:4/228: read d6/db/d1f/d41/d49/f22 [461323,5290] 0 2024-09-16T00:25:54.732 INFO:tasks.workunit.client.1.smithi169.stdout:2/34: getdents d7 0 2024-09-16T00:25:54.732 INFO:tasks.workunit.client.1.smithi169.stdout:2/35: readlink - no filename 2024-09-16T00:25:54.733 INFO:tasks.workunit.client.0.smithi112.stdout:6/208: getdents dd/df/d10/d1d/d37 0 2024-09-16T00:25:54.734 INFO:tasks.workunit.client.0.smithi112.stdout:8/217: mkdir d10/d12/d1c/d49 0 2024-09-16T00:25:54.734 INFO:tasks.workunit.client.0.smithi112.stdout:1/207: fsync d0/d14/fd 0 2024-09-16T00:25:54.735 INFO:tasks.workunit.client.1.smithi169.stdout:6/45: rename d2/c5 to d2/cf 0 2024-09-16T00:25:54.735 INFO:tasks.workunit.client.1.smithi169.stdout:2/36: stat f0 0 2024-09-16T00:25:54.736 INFO:tasks.workunit.client.0.smithi112.stdout:3/300: dwrite d2/d9/d2b/d39/f67 [0,4194304] 0 2024-09-16T00:25:54.737 INFO:tasks.workunit.client.1.smithi169.stdout:2/37: dread - f0 zero size 2024-09-16T00:25:54.738 INFO:tasks.workunit.client.1.smithi169.stdout:9/62: write d5/f6 [265873,50112] 0 2024-09-16T00:25:54.739 INFO:tasks.workunit.client.1.smithi169.stdout:1/118: readlink dc/le 0 2024-09-16T00:25:54.741 INFO:tasks.workunit.client.1.smithi169.stdout:6/46: chown d2/fb 107 1 2024-09-16T00:25:54.741 INFO:tasks.workunit.client.0.smithi112.stdout:6/209: chown dd/df/d17/c3f 2217301 1 2024-09-16T00:25:54.742 INFO:tasks.workunit.client.0.smithi112.stdout:8/218: fsync d10/f1b 0 2024-09-16T00:25:54.742 INFO:tasks.workunit.client.0.smithi112.stdout:0/280: write d5/da/dc/d4e/f4f [276301,89663] 0 2024-09-16T00:25:54.742 INFO:tasks.workunit.client.0.smithi112.stdout:6/210: readlink dd/df/l39 0 2024-09-16T00:25:54.743 INFO:tasks.workunit.client.0.smithi112.stdout:1/208: dread d0/f18 [0,4194304] 0 2024-09-16T00:25:54.743 INFO:tasks.workunit.client.0.smithi112.stdout:0/281: stat d5/c2f 0 2024-09-16T00:25:54.744 INFO:tasks.workunit.client.0.smithi112.stdout:8/219: fsync d10/d12/d24/f2b 0 2024-09-16T00:25:54.746 INFO:tasks.workunit.client.0.smithi112.stdout:1/209: read - d0/d14/df/d28/f2e zero size 2024-09-16T00:25:54.749 INFO:tasks.workunit.client.0.smithi112.stdout:3/301: dread d2/d9/d2b/d39/f67 [0,4194304] 0 2024-09-16T00:25:54.750 INFO:tasks.workunit.client.1.smithi169.stdout:6/47: chown d2/fb 179247 1 2024-09-16T00:25:54.752 INFO:tasks.workunit.client.1.smithi169.stdout:4/60: dwrite d2/f4 [0,4194304] 0 2024-09-16T00:25:54.754 INFO:tasks.workunit.client.1.smithi169.stdout:7/85: dwrite f1 [4194304,4194304] 0 2024-09-16T00:25:54.754 INFO:tasks.workunit.client.0.smithi112.stdout:0/282: dread d5/da/dc/d12/f17 [0,4194304] 0 2024-09-16T00:25:54.755 INFO:tasks.workunit.client.0.smithi112.stdout:7/294: write d6/d10/d16/d3b/f5e [529748,117869] 0 2024-09-16T00:25:54.756 INFO:tasks.workunit.client.1.smithi169.stdout:1/119: unlink dc/d18/l1f 0 2024-09-16T00:25:54.758 INFO:tasks.workunit.client.0.smithi112.stdout:1/210: mkdir d0/d14/df/d28/d42 0 2024-09-16T00:25:54.762 INFO:tasks.workunit.client.1.smithi169.stdout:5/97: dread d2/da/d10/d13/f1d [0,4194304] 0 2024-09-16T00:25:54.763 INFO:tasks.workunit.client.1.smithi169.stdout:7/86: chown d7/da/fc 9 1 2024-09-16T00:25:54.764 INFO:tasks.workunit.client.1.smithi169.stdout:0/60: creat d0/d2/d8/ff x:0 0 0 2024-09-16T00:25:54.766 INFO:tasks.workunit.client.0.smithi112.stdout:8/220: mknod d10/d12/d1c/d3b/c4a 0 2024-09-16T00:25:54.766 INFO:tasks.workunit.client.1.smithi169.stdout:4/61: rename d2/l8 to d2/d12/d13/l16 0 2024-09-16T00:25:54.767 INFO:tasks.workunit.client.0.smithi112.stdout:4/229: link d6/db/d1f/c33 d6/d39/d36/c4d 0 2024-09-16T00:25:54.767 INFO:tasks.workunit.client.1.smithi169.stdout:8/83: dwrite d7/fd [0,4194304] 0 2024-09-16T00:25:54.769 INFO:tasks.workunit.client.0.smithi112.stdout:1/211: write d0/d14/d1c/d26/f3e [2613902,65310] 0 2024-09-16T00:25:54.770 INFO:tasks.workunit.client.1.smithi169.stdout:1/120: chown dc/df/f12 429431 1 2024-09-16T00:25:54.770 INFO:tasks.workunit.client.0.smithi112.stdout:8/221: write d10/f34 [711944,101005] 0 2024-09-16T00:25:54.772 INFO:tasks.workunit.client.0.smithi112.stdout:0/283: dwrite d5/da/d34/f3b [4194304,4194304] 0 2024-09-16T00:25:54.772 INFO:tasks.workunit.client.1.smithi169.stdout:9/63: dwrite d5/f6 [0,4194304] 0 2024-09-16T00:25:54.773 INFO:tasks.workunit.client.1.smithi169.stdout:0/61: creat d0/f10 x:0 0 0 2024-09-16T00:25:54.775 INFO:tasks.workunit.client.1.smithi169.stdout:9/64: stat d5/da/f16 0 2024-09-16T00:25:54.776 INFO:tasks.workunit.client.0.smithi112.stdout:7/295: rename d6/d10/d16/d3b/f6d to d6/d10/f6e 0 2024-09-16T00:25:54.777 INFO:tasks.workunit.client.1.smithi169.stdout:7/87: creat d7/df/f15 x:0 0 0 2024-09-16T00:25:54.777 INFO:tasks.workunit.client.1.smithi169.stdout:3/86: dwrite d7/f1e [0,4194304] 0 2024-09-16T00:25:54.777 INFO:tasks.workunit.client.0.smithi112.stdout:6/211: dwrite dd/df/d10/f1a [0,4194304] 0 2024-09-16T00:25:54.780 INFO:tasks.workunit.client.0.smithi112.stdout:3/302: link d2/d9/d2b/d47/d52/l3e d2/d9/d2b/d2c/d69/l6a 0 2024-09-16T00:25:54.780 INFO:tasks.workunit.client.0.smithi112.stdout:8/222: chown d10/l11 112236049 1 2024-09-16T00:25:54.780 INFO:tasks.workunit.client.0.smithi112.stdout:4/230: fsync d6/db/d1f/d41/d49/d31/d34/f3e 0 2024-09-16T00:25:54.781 INFO:tasks.workunit.client.0.smithi112.stdout:7/296: fsync d6/f9 0 2024-09-16T00:25:54.783 INFO:tasks.workunit.client.0.smithi112.stdout:8/223: chown d10/d12/f16 202219503 1 2024-09-16T00:25:54.784 INFO:tasks.workunit.client.1.smithi169.stdout:1/121: chown dc/df/l1a 126247367 1 2024-09-16T00:25:54.785 INFO:tasks.workunit.client.0.smithi112.stdout:6/212: write dd/df/d13/d18/f28 [29187,108429] 0 2024-09-16T00:25:54.785 INFO:tasks.workunit.client.0.smithi112.stdout:1/212: read d0/d14/f1d [410869,119443] 0 2024-09-16T00:25:54.785 INFO:tasks.workunit.client.0.smithi112.stdout:0/284: symlink d5/da/d34/d48/l59 0 2024-09-16T00:25:54.786 INFO:tasks.workunit.client.1.smithi169.stdout:7/88: mkdir d7/d16 0 2024-09-16T00:25:54.786 INFO:tasks.workunit.client.0.smithi112.stdout:4/231: mknod d6/db/d1f/d41/d49/d31/d34/c4e 0 2024-09-16T00:25:54.786 INFO:tasks.workunit.client.0.smithi112.stdout:1/213: chown d0/ce 9427362 1 2024-09-16T00:25:54.787 INFO:tasks.workunit.client.0.smithi112.stdout:8/224: dread - d10/d12/d24/f2e zero size 2024-09-16T00:25:54.787 INFO:tasks.workunit.client.1.smithi169.stdout:9/65: creat d5/f19 x:0 0 0 2024-09-16T00:25:54.788 INFO:tasks.workunit.client.1.smithi169.stdout:7/89: readlink d7/le 0 2024-09-16T00:25:54.789 INFO:tasks.workunit.client.0.smithi112.stdout:8/225: stat d10/f34 0 2024-09-16T00:25:54.789 INFO:tasks.workunit.client.0.smithi112.stdout:3/303: symlink d2/d4d/l6b 0 2024-09-16T00:25:54.789 INFO:tasks.workunit.client.0.smithi112.stdout:0/285: rmdir d5/da/d34/d48 39 2024-09-16T00:25:54.789 INFO:tasks.workunit.client.0.smithi112.stdout:6/213: chown dd/df/d13/d18/c22 85039 1 2024-09-16T00:25:54.790 INFO:tasks.workunit.client.1.smithi169.stdout:3/87: creat d7/f20 x:0 0 0 2024-09-16T00:25:54.791 INFO:tasks.workunit.client.0.smithi112.stdout:1/214: mknod d0/d14/d1c/d26/c43 0 2024-09-16T00:25:54.791 INFO:tasks.workunit.client.1.smithi169.stdout:8/84: creat d7/d20/f21 x:0 0 0 2024-09-16T00:25:54.792 INFO:tasks.workunit.client.0.smithi112.stdout:6/214: chown dd/df 416 1 2024-09-16T00:25:54.793 INFO:tasks.workunit.client.0.smithi112.stdout:0/286: fdatasync d5/da/d18/d24/f41 0 2024-09-16T00:25:54.794 INFO:tasks.workunit.client.1.smithi169.stdout:9/66: mknod d5/c1a 0 2024-09-16T00:25:54.794 INFO:tasks.workunit.client.0.smithi112.stdout:4/232: readlink d6/db/d1f/d41/d49/d31/d34/l48 0 2024-09-16T00:25:54.794 INFO:tasks.workunit.client.0.smithi112.stdout:8/226: truncate d10/d12/d24/f48 773481 0 2024-09-16T00:25:54.795 INFO:tasks.workunit.client.0.smithi112.stdout:6/215: truncate dd/df/f4c 929591 0 2024-09-16T00:25:54.796 INFO:tasks.workunit.client.0.smithi112.stdout:1/215: mkdir d0/d14/d44 0 2024-09-16T00:25:54.796 INFO:tasks.workunit.client.1.smithi169.stdout:1/122: creat dc/f21 x:0 0 0 2024-09-16T00:25:54.797 INFO:tasks.workunit.client.1.smithi169.stdout:9/67: stat d5/f6 0 2024-09-16T00:25:54.797 INFO:tasks.workunit.client.0.smithi112.stdout:1/216: chown d0/d14/df/d28/d42 4731214 1 2024-09-16T00:25:54.798 INFO:tasks.workunit.client.1.smithi169.stdout:8/85: mkdir d7/d20/d22 0 2024-09-16T00:25:54.799 INFO:tasks.workunit.client.0.smithi112.stdout:1/217: fsync d0/d14/df/f3d 0 2024-09-16T00:25:54.799 INFO:tasks.workunit.client.1.smithi169.stdout:3/88: write d7/f16 [3012665,21941] 0 2024-09-16T00:25:54.800 INFO:tasks.workunit.client.0.smithi112.stdout:0/287: mkdir d5/da/d18/d24/d5a 0 2024-09-16T00:25:54.800 INFO:tasks.workunit.client.1.smithi169.stdout:1/123: read f0 [1357017,16734] 0 2024-09-16T00:25:54.800 INFO:tasks.workunit.client.0.smithi112.stdout:8/227: symlink d10/l4b 0 2024-09-16T00:25:54.802 INFO:tasks.workunit.client.1.smithi169.stdout:7/90: dread f1 [4194304,4194304] 0 2024-09-16T00:25:54.803 INFO:tasks.workunit.client.1.smithi169.stdout:8/86: fsync d7/dc/ff 0 2024-09-16T00:25:54.803 INFO:tasks.workunit.client.0.smithi112.stdout:3/304: rename d2/d9/d2b/d47/c45 to d2/d9/d2b/d2c/c6c 0 2024-09-16T00:25:54.804 INFO:tasks.workunit.client.1.smithi169.stdout:5/98: dread f0 [0,4194304] 0 2024-09-16T00:25:54.804 INFO:tasks.workunit.client.1.smithi169.stdout:4/62: dread d2/f4 [0,4194304] 0 2024-09-16T00:25:54.804 INFO:tasks.workunit.client.1.smithi169.stdout:9/68: mknod d5/d12/c1b 0 2024-09-16T00:25:54.805 INFO:tasks.workunit.client.0.smithi112.stdout:4/233: rename d6/ce to d6/db/c4f 0 2024-09-16T00:25:54.806 INFO:tasks.workunit.client.1.smithi169.stdout:8/87: dread - d7/f10 zero size 2024-09-16T00:25:54.807 INFO:tasks.workunit.client.1.smithi169.stdout:5/99: read d2/fd [45063,42263] 0 2024-09-16T00:25:54.807 INFO:tasks.workunit.client.1.smithi169.stdout:9/69: chown c4 1966794723 1 2024-09-16T00:25:54.807 INFO:tasks.workunit.client.1.smithi169.stdout:3/89: rename d7/l19 to d7/d1a/l21 0 2024-09-16T00:25:54.808 INFO:tasks.workunit.client.1.smithi169.stdout:0/62: dwrite d0/f10 [0,4194304] 0 2024-09-16T00:25:54.808 INFO:tasks.workunit.client.0.smithi112.stdout:3/305: symlink d2/d9/d2b/d47/l6d 0 2024-09-16T00:25:54.808 INFO:tasks.workunit.client.0.smithi112.stdout:6/216: dwrite dd/df/f21 [0,4194304] 0 2024-09-16T00:25:54.809 INFO:tasks.workunit.client.0.smithi112.stdout:1/218: rename d0/d14/df/d2f to d0/d14/df/d28/d45 0 2024-09-16T00:25:54.811 INFO:tasks.workunit.client.1.smithi169.stdout:8/88: fsync d7/f8 0 2024-09-16T00:25:54.814 INFO:tasks.workunit.client.1.smithi169.stdout:0/63: symlink d0/l11 0 2024-09-16T00:25:54.814 INFO:tasks.workunit.client.1.smithi169.stdout:7/91: rename f2 to d7/f17 0 2024-09-16T00:25:54.817 INFO:tasks.workunit.client.1.smithi169.stdout:8/89: readlink d7/dc/l15 0 2024-09-16T00:25:54.820 INFO:tasks.workunit.client.1.smithi169.stdout:5/100: mknod d2/da/d10/c22 0 2024-09-16T00:25:54.821 INFO:tasks.workunit.client.1.smithi169.stdout:5/101: fdatasync d2/fe 0 2024-09-16T00:25:54.822 INFO:tasks.workunit.client.1.smithi169.stdout:3/90: dwrite d7/f16 [4194304,4194304] 0 2024-09-16T00:25:54.822 INFO:tasks.workunit.client.1.smithi169.stdout:0/64: mknod d0/c12 0 2024-09-16T00:25:54.824 INFO:tasks.workunit.client.1.smithi169.stdout:2/38: write f3 [993286,55004] 0 2024-09-16T00:25:54.825 INFO:tasks.workunit.client.1.smithi169.stdout:2/39: dread - f0 zero size 2024-09-16T00:25:54.825 INFO:tasks.workunit.client.1.smithi169.stdout:2/40: readlink - no filename 2024-09-16T00:25:54.826 INFO:tasks.workunit.client.1.smithi169.stdout:3/91: write d7/d1a/f1b [653354,38128] 0 2024-09-16T00:25:54.826 INFO:tasks.workunit.client.1.smithi169.stdout:2/41: chown f2 38 1 2024-09-16T00:25:54.827 INFO:tasks.workunit.client.1.smithi169.stdout:8/90: rename d7/dc/c1a to d7/dc/c23 0 2024-09-16T00:25:54.828 INFO:tasks.workunit.client.1.smithi169.stdout:2/42: write f5 [282847,94969] 0 2024-09-16T00:25:54.830 INFO:tasks.workunit.client.1.smithi169.stdout:7/92: dread f1 [0,4194304] 0 2024-09-16T00:25:54.830 INFO:tasks.workunit.client.1.smithi169.stdout:8/91: dread - d7/f10 zero size 2024-09-16T00:25:54.835 INFO:tasks.workunit.client.1.smithi169.stdout:7/93: write d7/f8 [1472955,94849] 0 2024-09-16T00:25:54.840 INFO:tasks.workunit.client.1.smithi169.stdout:7/94: truncate d7/df/f15 605404 0 2024-09-16T00:25:54.843 INFO:tasks.workunit.client.1.smithi169.stdout:3/92: dwrite d7/f10 [0,4194304] 0 2024-09-16T00:25:54.845 INFO:tasks.workunit.client.1.smithi169.stdout:0/65: dread - d0/d2/d8/d5/fa zero size 2024-09-16T00:25:54.846 INFO:tasks.workunit.client.1.smithi169.stdout:2/43: dread f3 [0,4194304] 0 2024-09-16T00:25:54.846 INFO:tasks.workunit.client.1.smithi169.stdout:8/92: dread f6 [0,4194304] 0 2024-09-16T00:25:54.850 INFO:tasks.workunit.client.1.smithi169.stdout:0/66: write d0/d2/f6 [407437,9767] 0 2024-09-16T00:25:54.850 INFO:tasks.workunit.client.1.smithi169.stdout:3/93: rename d7/d1a/d1c to d7/d1a/d1c/d22 22 2024-09-16T00:25:54.853 INFO:tasks.workunit.client.1.smithi169.stdout:2/44: mknod d7/ca 0 2024-09-16T00:25:54.854 INFO:tasks.workunit.client.1.smithi169.stdout:7/95: dread f1 [4194304,4194304] 0 2024-09-16T00:25:54.856 INFO:tasks.workunit.client.1.smithi169.stdout:5/102: getdents d2/da/d10 0 2024-09-16T00:25:54.857 INFO:tasks.workunit.client.1.smithi169.stdout:0/67: creat d0/f13 x:0 0 0 2024-09-16T00:25:54.859 INFO:tasks.workunit.client.1.smithi169.stdout:9/70: creat d5/d12/f1c x:0 0 0 2024-09-16T00:25:54.861 INFO:tasks.workunit.client.1.smithi169.stdout:2/45: dread f5 [0,4194304] 0 2024-09-16T00:25:54.864 INFO:tasks.workunit.client.1.smithi169.stdout:7/96: chown d7/df/f11 0 1 2024-09-16T00:25:54.864 INFO:tasks.workunit.client.1.smithi169.stdout:2/46: dread - f6 zero size 2024-09-16T00:25:54.865 INFO:tasks.workunit.client.1.smithi169.stdout:0/68: mkdir d0/d2/d8/d5/d14 0 2024-09-16T00:25:54.865 INFO:tasks.workunit.client.1.smithi169.stdout:9/71: write d5/da/f16 [503784,43267] 0 2024-09-16T00:25:54.867 INFO:tasks.workunit.client.1.smithi169.stdout:2/47: dread - f0 zero size 2024-09-16T00:25:54.868 INFO:tasks.workunit.client.1.smithi169.stdout:9/72: stat c4 0 2024-09-16T00:25:54.869 INFO:tasks.workunit.client.1.smithi169.stdout:3/94: truncate d7/f11 8543394 0 2024-09-16T00:25:54.869 INFO:tasks.workunit.client.1.smithi169.stdout:4/63: dwrite d2/fb [0,4194304] 0 2024-09-16T00:25:54.872 INFO:tasks.workunit.client.1.smithi169.stdout:0/69: mkdir d0/d2/d15 0 2024-09-16T00:25:54.873 INFO:tasks.workunit.client.1.smithi169.stdout:8/93: dwrite d7/fd [4194304,4194304] 0 2024-09-16T00:25:54.874 INFO:tasks.workunit.client.1.smithi169.stdout:2/48: chown d7/f8 2632465 1 2024-09-16T00:25:54.879 INFO:tasks.workunit.client.1.smithi169.stdout:2/49: fdatasync f6 0 2024-09-16T00:25:54.883 INFO:tasks.workunit.client.1.smithi169.stdout:7/97: dread d7/f17 [0,4194304] 0 2024-09-16T00:25:54.883 INFO:tasks.workunit.client.1.smithi169.stdout:8/94: mkdir d7/dc/d17/d24 0 2024-09-16T00:25:54.884 INFO:tasks.workunit.client.1.smithi169.stdout:2/50: dread - f0 zero size 2024-09-16T00:25:54.885 INFO:tasks.workunit.client.0.smithi112.stdout:3/306: truncate d2/d9/f1c 4721299 0 2024-09-16T00:25:54.886 INFO:tasks.workunit.client.1.smithi169.stdout:0/70: mkdir d0/d2/d8/d5/d14/d16 0 2024-09-16T00:25:54.889 INFO:tasks.workunit.client.0.smithi112.stdout:4/234: symlink d6/db/df/d3c/l50 0 2024-09-16T00:25:54.889 INFO:tasks.workunit.client.1.smithi169.stdout:5/103: dread d2/f4 [0,4194304] 0 2024-09-16T00:25:54.891 INFO:tasks.workunit.client.0.smithi112.stdout:6/217: truncate dd/df/d17/f41 887221 0 2024-09-16T00:25:54.891 INFO:tasks.workunit.client.1.smithi169.stdout:8/95: mknod d7/dc/c25 0 2024-09-16T00:25:54.892 INFO:tasks.workunit.client.0.smithi112.stdout:7/297: truncate d6/d10/d16/d49/d6c/f36 3847741 0 2024-09-16T00:25:54.892 INFO:tasks.workunit.client.1.smithi169.stdout:4/64: dwrite d2/fb [0,4194304] 0 2024-09-16T00:25:54.893 INFO:tasks.workunit.client.0.smithi112.stdout:1/219: write d0/d14/d1c/d20/f33 [4264236,111687] 0 2024-09-16T00:25:54.895 INFO:tasks.workunit.client.1.smithi169.stdout:2/51: dread d7/f8 [0,4194304] 0 2024-09-16T00:25:54.895 INFO:tasks.workunit.client.1.smithi169.stdout:0/71: fdatasync d0/d2/d8/fe 0 2024-09-16T00:25:54.897 INFO:tasks.workunit.client.0.smithi112.stdout:6/218: write dd/df/d13/f1c [1857902,40207] 0 2024-09-16T00:25:54.897 INFO:tasks.workunit.client.1.smithi169.stdout:2/52: dread - f6 zero size 2024-09-16T00:25:54.898 INFO:tasks.workunit.client.0.smithi112.stdout:7/298: mknod d6/d10/d1b/d39/c6f 0 2024-09-16T00:25:54.901 INFO:tasks.workunit.client.1.smithi169.stdout:4/65: mknod d2/dd/c17 0 2024-09-16T00:25:54.902 INFO:tasks.workunit.client.0.smithi112.stdout:7/299: readlink d6/d10/d16/l2e 0 2024-09-16T00:25:54.903 INFO:tasks.workunit.client.0.smithi112.stdout:1/220: mkdir d0/d14/df/d28/d45/d46 0 2024-09-16T00:25:54.903 INFO:tasks.workunit.client.1.smithi169.stdout:3/95: dwrite d7/f1e [0,4194304] 0 2024-09-16T00:25:54.904 INFO:tasks.workunit.client.1.smithi169.stdout:2/53: unlink c4 0 2024-09-16T00:25:54.905 INFO:tasks.workunit.client.1.smithi169.stdout:8/96: read d7/fa [644059,53723] 0 2024-09-16T00:25:54.906 INFO:tasks.workunit.client.1.smithi169.stdout:9/73: dwrite d5/f6 [0,4194304] 0 2024-09-16T00:25:54.907 INFO:tasks.workunit.client.0.smithi112.stdout:6/219: unlink dd/d23/c46 0 2024-09-16T00:25:54.907 INFO:tasks.workunit.client.0.smithi112.stdout:0/288: dread d5/da/dc/fe [0,4194304] 0 2024-09-16T00:25:54.908 INFO:tasks.workunit.client.0.smithi112.stdout:4/235: dwrite d6/d39/d1c/f2b [0,4194304] 0 2024-09-16T00:25:54.909 INFO:tasks.workunit.client.0.smithi112.stdout:7/300: creat d6/d26/f70 x:0 0 0 2024-09-16T00:25:54.909 INFO:tasks.workunit.client.0.smithi112.stdout:3/307: dwrite d2/d9/d2b/d33/f50 [0,4194304] 0 2024-09-16T00:25:54.912 INFO:tasks.workunit.client.1.smithi169.stdout:2/54: mkdir d7/db 0 2024-09-16T00:25:54.913 INFO:tasks.workunit.client.0.smithi112.stdout:6/220: fsync dd/df/d10/f1a 0 2024-09-16T00:25:54.914 INFO:tasks.workunit.client.0.smithi112.stdout:1/221: rename d0/d14/df/d24 to d0/d14/d1c/d26/d47 0 2024-09-16T00:25:54.916 INFO:tasks.workunit.client.1.smithi169.stdout:5/104: rename d2/da/d10/d13/f21 to d2/da/d10/d13/f23 0 2024-09-16T00:25:54.916 INFO:tasks.workunit.client.1.smithi169.stdout:7/98: dwrite d7/df/f15 [0,4194304] 0 2024-09-16T00:25:54.916 INFO:tasks.workunit.client.1.smithi169.stdout:3/96: mkdir d7/d1a/d1c/d23 0 2024-09-16T00:25:54.917 INFO:tasks.workunit.client.0.smithi112.stdout:7/301: write d6/d10/d16/f3a [5164578,67851] 0 2024-09-16T00:25:54.917 INFO:tasks.workunit.client.1.smithi169.stdout:9/74: truncate d5/da/ff 5228624 0 2024-09-16T00:25:54.918 INFO:tasks.workunit.client.0.smithi112.stdout:6/221: readlink dd/l33 0 2024-09-16T00:25:54.920 INFO:tasks.workunit.client.0.smithi112.stdout:4/236: dwrite d6/db/f18 [0,4194304] 0 2024-09-16T00:25:54.920 INFO:tasks.workunit.client.0.smithi112.stdout:1/222: write d0/d14/df/d28/f2d [367126,68620] 0 2024-09-16T00:25:54.921 INFO:tasks.workunit.client.0.smithi112.stdout:6/222: rmdir dd/df/d10/d1d 39 2024-09-16T00:25:54.922 INFO:tasks.workunit.client.1.smithi169.stdout:7/99: symlink d7/l18 0 2024-09-16T00:25:54.922 INFO:tasks.workunit.client.0.smithi112.stdout:1/223: fsync d0/f1b 0 2024-09-16T00:25:54.924 INFO:tasks.workunit.client.0.smithi112.stdout:1/224: write d0/d14/fa [343653,99978] 0 2024-09-16T00:25:54.924 INFO:tasks.workunit.client.0.smithi112.stdout:7/302: creat d6/d10/f71 x:0 0 0 2024-09-16T00:25:54.925 INFO:tasks.workunit.client.1.smithi169.stdout:8/97: write d7/dc/ff [4358065,56360] 0 2024-09-16T00:25:54.927 INFO:tasks.workunit.client.0.smithi112.stdout:1/225: chown d0/d14/df/d28/d45/c34 1030103 1 2024-09-16T00:25:54.927 INFO:tasks.workunit.client.1.smithi169.stdout:4/66: dread d2/f3 [0,4194304] 0 2024-09-16T00:25:54.928 INFO:tasks.workunit.client.0.smithi112.stdout:1/226: chown d0/d14/df/d28 101031102 1 2024-09-16T00:25:54.930 INFO:tasks.workunit.client.0.smithi112.stdout:4/237: rename d6/db/f37 to d6/db/d1f/d41/d49/d31/f51 0 2024-09-16T00:25:54.930 INFO:tasks.workunit.client.0.smithi112.stdout:6/223: readlink dd/df/d10/d1d/l1f 0 2024-09-16T00:25:54.931 INFO:tasks.workunit.client.0.smithi112.stdout:7/303: mknod d6/c72 0 2024-09-16T00:25:54.931 INFO:tasks.workunit.client.0.smithi112.stdout:1/227: fsync d0/f9 0 2024-09-16T00:25:54.932 INFO:tasks.workunit.client.1.smithi169.stdout:7/100: rmdir d7/da 39 2024-09-16T00:25:54.932 INFO:tasks.workunit.client.1.smithi169.stdout:0/72: dwrite d0/f13 [0,4194304] 0 2024-09-16T00:25:54.933 INFO:tasks.workunit.client.1.smithi169.stdout:5/105: mkdir d2/da/d10/d13/d24 0 2024-09-16T00:25:54.933 INFO:tasks.workunit.client.0.smithi112.stdout:1/228: readlink d0/d14/d1c/d26/d47/l3f 0 2024-09-16T00:25:54.933 INFO:tasks.workunit.client.0.smithi112.stdout:6/224: chown dd/df/d10/f3a 322823 1 2024-09-16T00:25:54.934 INFO:tasks.workunit.client.1.smithi169.stdout:8/98: mkdir d7/dc/d26 0 2024-09-16T00:25:54.934 INFO:tasks.workunit.client.1.smithi169.stdout:4/67: write d2/f10 [566453,44056] 0 2024-09-16T00:25:54.934 INFO:tasks.workunit.client.0.smithi112.stdout:7/304: symlink d6/d10/d16/l73 0 2024-09-16T00:25:54.940 INFO:tasks.workunit.client.1.smithi169.stdout:8/99: truncate d7/f10 160499 0 2024-09-16T00:25:54.940 INFO:tasks.workunit.client.1.smithi169.stdout:5/106: symlink d2/l25 0 2024-09-16T00:25:54.942 INFO:tasks.workunit.client.0.smithi112.stdout:1/229: unlink d0/d14/df/d28/d45/f37 0 2024-09-16T00:25:54.942 INFO:tasks.workunit.client.1.smithi169.stdout:4/68: chown d2/d12/d13/l15 189140696 1 2024-09-16T00:25:54.945 INFO:tasks.workunit.client.1.smithi169.stdout:0/73: mknod d0/d2/d8/c17 0 2024-09-16T00:25:54.945 INFO:tasks.workunit.client.0.smithi112.stdout:6/225: truncate dd/df/d10/f3a 280943 0 2024-09-16T00:25:54.945 INFO:tasks.workunit.client.0.smithi112.stdout:6/226: readlink dd/df/d10/l14 0 2024-09-16T00:25:54.945 INFO:tasks.workunit.client.0.smithi112.stdout:3/308: write d2/f24 [2038732,977] 0 2024-09-16T00:25:54.946 INFO:tasks.workunit.client.1.smithi169.stdout:2/55: dwrite f2 [0,4194304] 0 2024-09-16T00:25:54.946 INFO:tasks.workunit.client.1.smithi169.stdout:4/69: write d2/fb [1519543,26813] 0 2024-09-16T00:25:54.947 INFO:tasks.workunit.client.1.smithi169.stdout:5/107: symlink d2/da/l26 0 2024-09-16T00:25:54.948 INFO:tasks.workunit.client.0.smithi112.stdout:5/289: sync 2024-09-16T00:25:54.948 INFO:tasks.workunit.client.0.smithi112.stdout:1/230: creat d0/f48 x:0 0 0 2024-09-16T00:25:54.949 INFO:tasks.workunit.client.0.smithi112.stdout:2/284: sync 2024-09-16T00:25:54.950 INFO:tasks.workunit.client.0.smithi112.stdout:1/231: dread - d0/d14/d1c/d20/f31 zero size 2024-09-16T00:25:54.951 INFO:tasks.workunit.client.0.smithi112.stdout:1/232: chown d0/d14/df/d28/d45/d46 11083 1 2024-09-16T00:25:54.952 INFO:tasks.workunit.client.1.smithi169.stdout:9/75: dread d5/f8 [0,4194304] 0 2024-09-16T00:25:54.952 INFO:tasks.workunit.client.1.smithi169.stdout:7/101: link d7/c12 d7/da/c19 0 2024-09-16T00:25:54.953 INFO:tasks.workunit.client.0.smithi112.stdout:5/290: creat d4/dd/f66 x:0 0 0 2024-09-16T00:25:54.953 INFO:tasks.workunit.client.0.smithi112.stdout:2/285: dread d2/dd/d48/d22/d2b/f31 [0,4194304] 0 2024-09-16T00:25:54.954 INFO:tasks.workunit.client.0.smithi112.stdout:5/291: chown d4/f19 60 1 2024-09-16T00:25:54.954 INFO:tasks.workunit.client.1.smithi169.stdout:3/97: dwrite d7/f14 [0,4194304] 0 2024-09-16T00:25:54.955 INFO:tasks.workunit.client.0.smithi112.stdout:5/292: fdatasync d4/dd/d12/f11 0 2024-09-16T00:25:54.955 INFO:tasks.workunit.client.0.smithi112.stdout:1/233: creat d0/d14/df/d28/d45/d46/f49 x:0 0 0 2024-09-16T00:25:54.956 INFO:tasks.workunit.client.1.smithi169.stdout:4/70: creat d2/dd/f18 x:0 0 0 2024-09-16T00:25:54.956 INFO:tasks.workunit.client.1.smithi169.stdout:0/74: symlink d0/d2/d8/d5/l18 0 2024-09-16T00:25:54.956 INFO:tasks.workunit.client.1.smithi169.stdout:8/100: getdents d7/d20 0 2024-09-16T00:25:54.957 INFO:tasks.workunit.client.1.smithi169.stdout:7/102: chown d7/df/f15 9257 1 2024-09-16T00:25:54.957 INFO:tasks.workunit.client.0.smithi112.stdout:5/293: mknod d4/dd/d12/c67 0 2024-09-16T00:25:54.957 INFO:tasks.workunit.client.0.smithi112.stdout:2/286: read d2/dd/f1a [1314020,26225] 0 2024-09-16T00:25:54.957 INFO:tasks.workunit.client.0.smithi112.stdout:6/227: dread dd/d15/f29 [0,4194304] 0 2024-09-16T00:25:54.958 INFO:tasks.workunit.client.1.smithi169.stdout:7/103: stat d7/l18 0 2024-09-16T00:25:54.958 INFO:tasks.workunit.client.1.smithi169.stdout:3/98: fdatasync d7/f9 0 2024-09-16T00:25:54.959 INFO:tasks.workunit.client.1.smithi169.stdout:5/108: truncate d2/f8 3149828 0 2024-09-16T00:25:54.959 INFO:tasks.workunit.client.0.smithi112.stdout:1/234: dread d0/d14/f22 [0,4194304] 0 2024-09-16T00:25:54.960 INFO:tasks.workunit.client.0.smithi112.stdout:2/287: symlink d2/dd/d48/d22/d2b/d3b/d49/l56 0 2024-09-16T00:25:54.961 INFO:tasks.workunit.client.1.smithi169.stdout:4/71: rmdir d2/d12/d13 39 2024-09-16T00:25:54.963 INFO:tasks.workunit.client.0.smithi112.stdout:1/235: read d0/d14/f8 [5751280,37054] 0 2024-09-16T00:25:54.963 INFO:tasks.workunit.client.0.smithi112.stdout:2/288: creat d2/dd/d48/d22/d2b/f57 x:0 0 0 2024-09-16T00:25:54.963 INFO:tasks.workunit.client.0.smithi112.stdout:6/228: link cb dd/df/d10/d1d/c4d 0 2024-09-16T00:25:54.964 INFO:tasks.workunit.client.0.smithi112.stdout:6/229: fdatasync dd/df/d10/d1d/f26 0 2024-09-16T00:25:54.965 INFO:tasks.workunit.client.1.smithi169.stdout:9/76: rename d5/l11 to d5/d12/l1d 0 2024-09-16T00:25:54.966 INFO:tasks.workunit.client.0.smithi112.stdout:1/236: creat d0/d14/df/d28/d45/d30/f4a x:0 0 0 2024-09-16T00:25:54.967 INFO:tasks.workunit.client.0.smithi112.stdout:6/230: mknod dd/df/d10/c4e 0 2024-09-16T00:25:54.967 INFO:tasks.workunit.client.1.smithi169.stdout:9/77: stat d5/f8 0 2024-09-16T00:25:54.969 INFO:tasks.workunit.client.0.smithi112.stdout:6/231: read dd/df/f4c [128037,14720] 0 2024-09-16T00:25:54.969 INFO:tasks.workunit.client.1.smithi169.stdout:8/101: dread f6 [4194304,4194304] 0 2024-09-16T00:25:54.970 INFO:tasks.workunit.client.0.smithi112.stdout:2/289: dwrite d2/dd/d48/f23 [0,4194304] 0 2024-09-16T00:25:54.971 INFO:tasks.workunit.client.0.smithi112.stdout:1/237: creat d0/d14/df/f4b x:0 0 0 2024-09-16T00:25:54.971 INFO:tasks.workunit.client.1.smithi169.stdout:5/109: link d2/f20 d2/da/d10/f27 0 2024-09-16T00:25:54.971 INFO:tasks.workunit.client.0.smithi112.stdout:6/232: truncate dd/df/d17/f41 946257 0 2024-09-16T00:25:54.972 INFO:tasks.workunit.client.0.smithi112.stdout:2/290: stat d2/dd/d48/d22/d2b/d30/c3f 0 2024-09-16T00:25:54.973 INFO:tasks.workunit.client.1.smithi169.stdout:7/104: dread d7/df/f11 [0,4194304] 0 2024-09-16T00:25:54.973 INFO:tasks.workunit.client.1.smithi169.stdout:0/75: dwrite d0/d2/d8/ff [0,4194304] 0 2024-09-16T00:25:54.975 INFO:tasks.workunit.client.1.smithi169.stdout:8/102: unlink d7/f18 0 2024-09-16T00:25:54.975 INFO:tasks.workunit.client.0.smithi112.stdout:6/233: getdents dd/d15/d34 0 2024-09-16T00:25:54.977 INFO:tasks.workunit.client.0.smithi112.stdout:2/291: rename d2/dd/d48/f27 to d2/dd/d48/d22/d2b/f58 0 2024-09-16T00:25:54.978 INFO:tasks.workunit.client.0.smithi112.stdout:6/234: symlink dd/df/d10/d1d/l4f 0 2024-09-16T00:25:54.978 INFO:tasks.workunit.client.0.smithi112.stdout:1/238: dwrite d0/f1b [0,4194304] 0 2024-09-16T00:25:54.979 INFO:tasks.workunit.client.1.smithi169.stdout:5/110: creat d2/da/f28 x:0 0 0 2024-09-16T00:25:54.982 INFO:tasks.workunit.client.1.smithi169.stdout:9/78: creat d5/d12/f1e x:0 0 0 2024-09-16T00:25:54.984 INFO:tasks.workunit.client.1.smithi169.stdout:7/105: creat d7/d16/f1a x:0 0 0 2024-09-16T00:25:54.984 INFO:tasks.workunit.client.1.smithi169.stdout:5/111: chown c1 21569 1 2024-09-16T00:25:54.985 INFO:tasks.workunit.client.1.smithi169.stdout:7/106: chown d7 26220493 1 2024-09-16T00:25:54.986 INFO:tasks.workunit.client.1.smithi169.stdout:0/76: symlink d0/d2/d15/l19 0 2024-09-16T00:25:54.986 INFO:tasks.workunit.client.0.smithi112.stdout:2/292: truncate d2/dd/d48/f2d 164498 0 2024-09-16T00:25:54.986 INFO:tasks.workunit.client.0.smithi112.stdout:6/235: mkdir dd/df/d50 0 2024-09-16T00:25:54.987 INFO:tasks.workunit.client.1.smithi169.stdout:3/99: dwrite d7/fb [4194304,4194304] 0 2024-09-16T00:25:54.988 INFO:tasks.workunit.client.0.smithi112.stdout:6/236: chown dd/df/d11 16617 1 2024-09-16T00:25:54.989 INFO:tasks.workunit.client.1.smithi169.stdout:8/103: creat d7/dc/d17/d24/f27 x:0 0 0 2024-09-16T00:25:54.989 INFO:tasks.workunit.client.0.smithi112.stdout:1/239: mknod d0/d14/df/c4c 0 2024-09-16T00:25:54.989 INFO:tasks.workunit.client.0.smithi112.stdout:7/305: rmdir d6 39 2024-09-16T00:25:54.990 INFO:tasks.workunit.client.1.smithi169.stdout:0/77: chown d0/d2/d8/d5/fa 57985806 1 2024-09-16T00:25:54.990 INFO:tasks.workunit.client.0.smithi112.stdout:1/240: chown d0/d14/df/c4c 488 1 2024-09-16T00:25:54.991 INFO:tasks.workunit.client.1.smithi169.stdout:7/107: symlink d7/da/l1b 0 2024-09-16T00:25:54.993 INFO:tasks.workunit.client.1.smithi169.stdout:5/112: dread d2/ff [0,4194304] 0 2024-09-16T00:25:54.995 INFO:tasks.workunit.client.1.smithi169.stdout:3/100: rename d7/fb to d7/f24 0 2024-09-16T00:25:54.995 INFO:tasks.workunit.client.1.smithi169.stdout:3/101: fdatasync d7/f9 0 2024-09-16T00:25:54.995 INFO:tasks.workunit.client.0.smithi112.stdout:3/309: write d2/fb [938330,111845] 0 2024-09-16T00:25:54.996 INFO:tasks.workunit.client.1.smithi169.stdout:8/104: unlink d7/dc/ff 0 2024-09-16T00:25:54.996 INFO:tasks.workunit.client.1.smithi169.stdout:3/102: chown d7/c17 473995 1 2024-09-16T00:25:54.996 INFO:tasks.workunit.client.1.smithi169.stdout:8/105: chown d7/dc/d26 32785520 1 2024-09-16T00:25:54.997 INFO:tasks.workunit.client.0.smithi112.stdout:2/293: dread d2/dd/d48/d22/d2b/d30/f36 [0,4194304] 0 2024-09-16T00:25:54.998 INFO:tasks.workunit.client.1.smithi169.stdout:8/106: write d7/d20/f21 [649802,53661] 0 2024-09-16T00:25:54.998 INFO:tasks.workunit.client.0.smithi112.stdout:3/310: write d2/d9/d2b/f29 [867522,50875] 0 2024-09-16T00:25:54.998 INFO:tasks.workunit.client.0.smithi112.stdout:6/237: rename dd/df/d10/d1d/c4d to dd/df/d10/d1d/d37/c51 0 2024-09-16T00:25:54.999 INFO:tasks.workunit.client.0.smithi112.stdout:1/241: symlink d0/d14/d44/l4d 0 2024-09-16T00:25:55.000 INFO:tasks.workunit.client.0.smithi112.stdout:1/242: read - d0/d14/d1c/d20/f31 zero size 2024-09-16T00:25:55.001 INFO:tasks.workunit.client.0.smithi112.stdout:7/306: creat d6/d10/d1b/f74 x:0 0 0 2024-09-16T00:25:55.001 INFO:tasks.workunit.client.0.smithi112.stdout:1/243: read - d0/f48 zero size 2024-09-16T00:25:55.002 INFO:tasks.workunit.client.0.smithi112.stdout:7/307: readlink d6/d10/d16/l2f 0 2024-09-16T00:25:55.002 INFO:tasks.workunit.client.0.smithi112.stdout:2/294: link d2/dd/f12 d2/dd/d48/d22/d2b/f59 0 2024-09-16T00:25:55.003 INFO:tasks.workunit.client.0.smithi112.stdout:3/311: mknod d2/c6e 0 2024-09-16T00:25:55.003 INFO:tasks.workunit.client.1.smithi169.stdout:0/78: mkdir d0/d2/d8/d5/d14/d16/d1a 0 2024-09-16T00:25:55.004 INFO:tasks.workunit.client.0.smithi112.stdout:6/238: mknod dd/df/d10/d47/c52 0 2024-09-16T00:25:55.006 INFO:tasks.workunit.client.1.smithi169.stdout:5/113: creat d2/f29 x:0 0 0 2024-09-16T00:25:55.009 INFO:tasks.workunit.client.1.smithi169.stdout:5/114: write d2/f1f [461951,57343] 0 2024-09-16T00:25:55.009 INFO:tasks.workunit.client.1.smithi169.stdout:0/79: getdents d0/d2/d8/d5/d14/d16/d1a 0 2024-09-16T00:25:55.010 INFO:tasks.workunit.client.0.smithi112.stdout:6/239: mknod dd/df/d13/c53 0 2024-09-16T00:25:55.010 INFO:tasks.workunit.client.1.smithi169.stdout:0/80: symlink d0/d2/d8/d5/d14/l1b 0 2024-09-16T00:25:55.011 INFO:tasks.workunit.client.0.smithi112.stdout:1/244: creat d0/f4e x:0 0 0 2024-09-16T00:25:55.011 INFO:tasks.workunit.client.1.smithi169.stdout:5/115: rmdir d2/da/d10/d13 39 2024-09-16T00:25:55.012 INFO:tasks.workunit.client.0.smithi112.stdout:3/312: creat d2/d9/d2b/d2c/d69/f6f x:0 0 0 2024-09-16T00:25:55.013 INFO:tasks.workunit.client.1.smithi169.stdout:0/81: creat d0/d2/d8/d5/d14/f1c x:0 0 0 2024-09-16T00:25:55.014 INFO:tasks.workunit.client.0.smithi112.stdout:6/240: readlink dd/df/d10/d1d/l1f 0 2024-09-16T00:25:55.014 INFO:tasks.workunit.client.1.smithi169.stdout:4/72: truncate d2/fb 3272997 0 2024-09-16T00:25:55.015 INFO:tasks.workunit.client.0.smithi112.stdout:2/295: dwrite d2/dd/d48/d22/d2b/f59 [4194304,4194304] 0 2024-09-16T00:25:55.015 INFO:tasks.workunit.client.1.smithi169.stdout:0/82: rename d0/d2/d8/d5/d14/d16 to d0/d2/d8/d5/d14/d16/d1d 22 2024-09-16T00:25:55.016 INFO:tasks.workunit.client.0.smithi112.stdout:1/245: mkdir d0/d14/d1c/d20/d4f 0 2024-09-16T00:25:55.016 INFO:tasks.workunit.client.0.smithi112.stdout:6/241: read dd/df/d10/f1a [653707,93607] 0 2024-09-16T00:25:55.016 INFO:tasks.workunit.client.1.smithi169.stdout:5/116: creat d2/f2a x:0 0 0 2024-09-16T00:25:55.017 INFO:tasks.workunit.client.0.smithi112.stdout:3/313: truncate d2/d4d/f60 2171043 0 2024-09-16T00:25:55.017 INFO:tasks.workunit.client.0.smithi112.stdout:1/246: read d0/f1e [859837,72308] 0 2024-09-16T00:25:55.017 INFO:tasks.workunit.client.1.smithi169.stdout:9/79: rmdir d5/d12 39 2024-09-16T00:25:55.022 INFO:tasks.workunit.client.0.smithi112.stdout:1/247: rmdir d0/d14/d1c/d26/d35 39 2024-09-16T00:25:55.023 INFO:tasks.workunit.client.1.smithi169.stdout:0/83: symlink d0/d2/d8/l1e 0 2024-09-16T00:25:55.024 INFO:tasks.workunit.client.0.smithi112.stdout:6/242: mkdir dd/df/d17/d27/d54 0 2024-09-16T00:25:55.025 INFO:tasks.workunit.client.1.smithi169.stdout:9/80: mknod d5/da/c1f 0 2024-09-16T00:25:55.026 INFO:tasks.workunit.client.0.smithi112.stdout:2/296: creat d2/dd/d48/d22/d2b/d54/f5a x:0 0 0 2024-09-16T00:25:55.027 INFO:tasks.workunit.client.1.smithi169.stdout:0/84: mknod d0/d2/c1f 0 2024-09-16T00:25:55.028 INFO:tasks.workunit.client.0.smithi112.stdout:3/314: dwrite d2/d9/d2b/d2c/f42 [4194304,4194304] 0 2024-09-16T00:25:55.029 INFO:tasks.workunit.client.0.smithi112.stdout:1/248: symlink d0/d14/d1c/l50 0 2024-09-16T00:25:55.029 INFO:tasks.workunit.client.1.smithi169.stdout:9/81: unlink d5/da/fd 0 2024-09-16T00:25:55.030 INFO:tasks.workunit.client.1.smithi169.stdout:7/108: write d7/f17 [1618177,90040] 0 2024-09-16T00:25:55.031 INFO:tasks.workunit.client.0.smithi112.stdout:1/249: chown d0/l1a 1676111906 1 2024-09-16T00:25:55.032 INFO:tasks.workunit.client.0.smithi112.stdout:7/308: dread d6/d10/f15 [0,4194304] 0 2024-09-16T00:25:55.036 INFO:tasks.workunit.client.0.smithi112.stdout:1/250: creat d0/d14/d1c/f51 x:0 0 0 2024-09-16T00:25:55.039 INFO:tasks.workunit.client.0.smithi112.stdout:3/315: symlink d2/d9/d2b/d33/d68/l70 0 2024-09-16T00:25:55.042 INFO:tasks.workunit.client.1.smithi169.stdout:4/73: truncate d2/f3 999132 0 2024-09-16T00:25:55.042 INFO:tasks.workunit.client.0.smithi112.stdout:3/316: fsync d2/f3a 0 2024-09-16T00:25:55.043 INFO:tasks.workunit.client.1.smithi169.stdout:9/82: chown d5/d12/l1d 13 1 2024-09-16T00:25:55.044 INFO:tasks.workunit.client.1.smithi169.stdout:9/83: fdatasync d5/da/f16 0 2024-09-16T00:25:55.048 INFO:tasks.workunit.client.0.smithi112.stdout:1/251: rmdir d0/d14/d1c/d20/d4f 0 2024-09-16T00:25:55.049 INFO:tasks.workunit.client.0.smithi112.stdout:1/252: chown d0/f4e 1 1 2024-09-16T00:25:55.049 INFO:tasks.workunit.client.0.smithi112.stdout:3/317: chown d2/d9/d2b/d39/l44 1 1 2024-09-16T00:25:55.049 INFO:tasks.workunit.client.0.smithi112.stdout:3/318: readlink l1 0 2024-09-16T00:25:55.049 INFO:tasks.workunit.client.0.smithi112.stdout:7/309: getdents d6/d10/d1b/d39 0 2024-09-16T00:25:55.050 INFO:tasks.workunit.client.1.smithi169.stdout:5/117: dwrite d2/f20 [0,4194304] 0 2024-09-16T00:25:55.050 INFO:tasks.workunit.client.0.smithi112.stdout:1/253: write d0/d14/df/d28/d45/d46/f49 [419205,3414] 0 2024-09-16T00:25:55.052 INFO:tasks.workunit.client.0.smithi112.stdout:7/310: write d6/d10/f71 [325875,70630] 0 2024-09-16T00:25:55.052 INFO:tasks.workunit.client.0.smithi112.stdout:2/297: write f1 [306778,12731] 0 2024-09-16T00:25:55.053 INFO:tasks.workunit.client.1.smithi169.stdout:0/85: truncate d0/d2/d8/ff 8669 0 2024-09-16T00:25:55.054 INFO:tasks.workunit.client.1.smithi169.stdout:4/74: symlink d2/d12/l19 0 2024-09-16T00:25:55.057 INFO:tasks.workunit.client.0.smithi112.stdout:2/298: readlink d2/dd/d48/l4b 0 2024-09-16T00:25:55.057 INFO:tasks.workunit.client.0.smithi112.stdout:3/319: dread d2/fb [0,4194304] 0 2024-09-16T00:25:55.058 INFO:tasks.workunit.client.0.smithi112.stdout:7/311: mkdir d6/d10/d75 0 2024-09-16T00:25:55.059 INFO:tasks.workunit.client.1.smithi169.stdout:9/84: symlink d5/da/l20 0 2024-09-16T00:25:55.059 INFO:tasks.workunit.client.1.smithi169.stdout:8/107: dwrite d7/dc/fe [0,4194304] 0 2024-09-16T00:25:55.060 INFO:tasks.workunit.client.1.smithi169.stdout:9/85: read d5/f8 [4051182,70930] 0 2024-09-16T00:25:55.062 INFO:tasks.workunit.client.0.smithi112.stdout:6/243: dwrite dd/df/f4c [0,4194304] 0 2024-09-16T00:25:55.065 INFO:tasks.workunit.client.0.smithi112.stdout:3/320: dread d2/f25 [0,4194304] 0 2024-09-16T00:25:55.066 INFO:tasks.workunit.client.1.smithi169.stdout:4/75: mkdir d2/dd/d1a 0 2024-09-16T00:25:55.067 INFO:tasks.workunit.client.0.smithi112.stdout:1/254: dwrite d0/f9 [0,4194304] 0 2024-09-16T00:25:55.069 INFO:tasks.workunit.client.0.smithi112.stdout:6/244: mkdir dd/df/d10/d47/d55 0 2024-09-16T00:25:55.072 INFO:tasks.workunit.client.1.smithi169.stdout:5/118: mknod d2/da/d10/d13/c2b 0 2024-09-16T00:25:55.074 INFO:tasks.workunit.client.0.smithi112.stdout:1/255: fsync d0/d14/d1c/d26/f3e 0 2024-09-16T00:25:55.077 INFO:tasks.workunit.client.1.smithi169.stdout:0/86: fsync d0/d2/d8/ff 0 2024-09-16T00:25:55.077 INFO:tasks.workunit.client.0.smithi112.stdout:2/299: dwrite d2/dd/d48/f24 [0,4194304] 0 2024-09-16T00:25:55.079 INFO:tasks.workunit.client.0.smithi112.stdout:6/245: creat dd/df/d17/d27/d54/f56 x:0 0 0 2024-09-16T00:25:55.079 INFO:tasks.workunit.client.1.smithi169.stdout:4/76: creat d2/dd/d1a/f1b x:0 0 0 2024-09-16T00:25:55.079 INFO:tasks.workunit.client.1.smithi169.stdout:8/108: chown d7/dc/l15 0 1 2024-09-16T00:25:55.081 INFO:tasks.workunit.client.1.smithi169.stdout:4/77: fsync d2/dd/f18 0 2024-09-16T00:25:55.084 INFO:tasks.workunit.client.1.smithi169.stdout:5/119: dwrite d2/fe [0,4194304] 0 2024-09-16T00:25:55.084 INFO:tasks.workunit.client.0.smithi112.stdout:7/312: dwrite d6/d26/f2d [0,4194304] 0 2024-09-16T00:25:55.085 INFO:tasks.workunit.client.0.smithi112.stdout:1/256: creat d0/d14/d1c/d26/f52 x:0 0 0 2024-09-16T00:25:55.090 INFO:tasks.workunit.client.0.smithi112.stdout:3/321: dwrite d2/d4d/f60 [0,4194304] 0 2024-09-16T00:25:55.091 INFO:tasks.workunit.client.0.smithi112.stdout:9/441: sync 2024-09-16T00:25:55.091 INFO:tasks.workunit.client.1.smithi169.stdout:0/87: write d0/d2/d8/d5/fa [35343,127531] 0 2024-09-16T00:25:55.092 INFO:tasks.workunit.client.1.smithi169.stdout:9/86: write d5/da/fc [252684,128204] 0 2024-09-16T00:25:55.094 INFO:tasks.workunit.client.1.smithi169.stdout:8/109: dwrite d7/fd [0,4194304] 0 2024-09-16T00:25:55.094 INFO:tasks.workunit.client.0.smithi112.stdout:7/313: creat d6/d10/f76 x:0 0 0 2024-09-16T00:25:55.095 INFO:tasks.workunit.client.0.smithi112.stdout:2/300: dread d2/dd/d48/f24 [0,4194304] 0 2024-09-16T00:25:55.100 INFO:tasks.workunit.client.0.smithi112.stdout:9/442: stat d1/d5/d6/d18/f9a 0 2024-09-16T00:25:55.103 INFO:tasks.workunit.client.0.smithi112.stdout:6/246: dwrite dd/df/d10/f1b [0,4194304] 0 2024-09-16T00:25:55.103 INFO:tasks.workunit.client.0.smithi112.stdout:2/301: creat d2/dd/f5b x:0 0 0 2024-09-16T00:25:55.105 INFO:tasks.workunit.client.0.smithi112.stdout:7/314: symlink d6/d10/d75/l77 0 2024-09-16T00:25:55.106 INFO:tasks.workunit.client.1.smithi169.stdout:0/88: symlink d0/d2/d15/l20 0 2024-09-16T00:25:55.106 INFO:tasks.workunit.client.1.smithi169.stdout:5/120: dwrite d2/da/d10/f27 [0,4194304] 0 2024-09-16T00:25:55.108 INFO:tasks.workunit.client.1.smithi169.stdout:8/110: readlink d7/dc/d17/l19 0 2024-09-16T00:25:55.111 INFO:tasks.workunit.client.0.smithi112.stdout:3/322: unlink d2/f5e 0 2024-09-16T00:25:55.111 INFO:tasks.workunit.client.1.smithi169.stdout:0/89: chown d0/f10 6840 1 2024-09-16T00:25:55.111 INFO:tasks.workunit.client.1.smithi169.stdout:5/121: write d2/f29 [970321,121443] 0 2024-09-16T00:25:55.112 INFO:tasks.workunit.client.1.smithi169.stdout:4/78: dwrite d2/f9 [0,4194304] 0 2024-09-16T00:25:55.113 INFO:tasks.workunit.client.0.smithi112.stdout:2/302: creat d2/dd/d48/d22/d2b/d3b/d49/f5c x:0 0 0 2024-09-16T00:25:55.115 INFO:tasks.workunit.client.0.smithi112.stdout:6/247: dwrite dd/df/d10/f1a [0,4194304] 0 2024-09-16T00:25:55.115 INFO:tasks.workunit.client.0.smithi112.stdout:3/323: unlink d2/f25 0 2024-09-16T00:25:55.117 INFO:tasks.workunit.client.1.smithi169.stdout:0/90: rmdir d0/d2/d15 39 2024-09-16T00:25:55.118 INFO:tasks.workunit.client.0.smithi112.stdout:2/303: rmdir d2/dd/d48/d22/d2b/d3b 39 2024-09-16T00:25:55.118 INFO:tasks.workunit.client.1.smithi169.stdout:9/87: dwrite d5/f14 [0,4194304] 0 2024-09-16T00:25:55.119 INFO:tasks.workunit.client.1.smithi169.stdout:0/91: chown d0/d2/c1f 1 1 2024-09-16T00:25:55.120 INFO:tasks.workunit.client.0.smithi112.stdout:2/304: write d2/dd/d48/d22/d2b/d54/f5a [801904,114978] 0 2024-09-16T00:25:55.121 INFO:tasks.workunit.client.0.smithi112.stdout:6/248: creat dd/df/d10/d47/f57 x:0 0 0 2024-09-16T00:25:55.122 INFO:tasks.workunit.client.1.smithi169.stdout:8/111: rmdir d7/d20/d22 0 2024-09-16T00:25:55.123 INFO:tasks.workunit.client.0.smithi112.stdout:3/324: chown d2/d9/d2b/d47/d52/l3e 3 1 2024-09-16T00:25:55.125 INFO:tasks.workunit.client.1.smithi169.stdout:5/122: creat d2/da/d10/d13/d24/f2c x:0 0 0 2024-09-16T00:25:55.126 INFO:tasks.workunit.client.1.smithi169.stdout:9/88: mkdir d5/da/d21 0 2024-09-16T00:25:55.128 INFO:tasks.workunit.client.0.smithi112.stdout:6/249: creat dd/df/d10/d1d/d37/d42/f58 x:0 0 0 2024-09-16T00:25:55.129 INFO:tasks.workunit.client.0.smithi112.stdout:6/250: fdatasync dd/df/d13/f1c 0 2024-09-16T00:25:55.131 INFO:tasks.workunit.client.1.smithi169.stdout:9/89: mknod d5/c22 0 2024-09-16T00:25:55.133 INFO:tasks.workunit.client.0.smithi112.stdout:3/325: link d2/d9/f16 d2/d9/d2b/d2c/d69/f71 0 2024-09-16T00:25:55.134 INFO:tasks.workunit.client.0.smithi112.stdout:1/257: dread d0/d14/f8 [0,4194304] 0 2024-09-16T00:25:55.135 INFO:tasks.workunit.client.1.smithi169.stdout:0/92: dwrite d0/d2/d8/d5/d14/f1c [0,4194304] 0 2024-09-16T00:25:55.138 INFO:tasks.workunit.client.0.smithi112.stdout:1/258: readlink d0/d14/df/l10 0 2024-09-16T00:25:55.138 INFO:tasks.workunit.client.1.smithi169.stdout:5/123: dread f0 [0,4194304] 0 2024-09-16T00:25:55.140 INFO:tasks.workunit.client.1.smithi169.stdout:8/112: dwrite d7/fa [0,4194304] 0 2024-09-16T00:25:55.140 INFO:tasks.workunit.client.1.smithi169.stdout:5/124: readlink d2/da/l26 0 2024-09-16T00:25:55.143 INFO:tasks.workunit.client.1.smithi169.stdout:9/90: dread d5/da/ff [0,4194304] 0 2024-09-16T00:25:55.144 INFO:tasks.workunit.client.0.smithi112.stdout:1/259: creat d0/d14/d1c/d20/f53 x:0 0 0 2024-09-16T00:25:55.145 INFO:tasks.workunit.client.1.smithi169.stdout:0/93: getdents d0/d2/d8/d5/d14/d16 0 2024-09-16T00:25:55.145 INFO:tasks.workunit.client.0.smithi112.stdout:1/260: dread - d0/d14/df/f36 zero size 2024-09-16T00:25:55.146 INFO:tasks.workunit.client.0.smithi112.stdout:3/326: dwrite d2/f3f [0,4194304] 0 2024-09-16T00:25:55.146 INFO:tasks.workunit.client.1.smithi169.stdout:0/94: fdatasync d0/d2/d8/d5/fa 0 2024-09-16T00:25:55.147 INFO:tasks.workunit.client.1.smithi169.stdout:8/113: stat d7/dc/fe 0 2024-09-16T00:25:55.147 INFO:tasks.workunit.client.1.smithi169.stdout:5/125: mkdir d2/da/d10/d13/d1c/d2d 0 2024-09-16T00:25:55.148 INFO:tasks.workunit.client.0.smithi112.stdout:3/327: truncate d2/d9/d2b/f29 1636249 0 2024-09-16T00:25:55.149 INFO:tasks.workunit.client.1.smithi169.stdout:9/91: mkdir d5/da/d21/d23 0 2024-09-16T00:25:55.149 INFO:tasks.workunit.client.1.smithi169.stdout:5/126: truncate d2/f2a 468873 0 2024-09-16T00:25:55.151 INFO:tasks.workunit.client.0.smithi112.stdout:1/261: rename d0/l1a to d0/d14/d1c/d20/l54 0 2024-09-16T00:25:55.151 INFO:tasks.workunit.client.1.smithi169.stdout:0/95: mkdir d0/d2/d8/d5/d14/d21 0 2024-09-16T00:25:55.153 INFO:tasks.workunit.client.0.smithi112.stdout:7/315: dread d6/d10/f32 [0,4194304] 0 2024-09-16T00:25:55.154 INFO:tasks.workunit.client.1.smithi169.stdout:5/127: truncate d2/da/d10/f11 678389 0 2024-09-16T00:25:55.155 INFO:tasks.workunit.client.0.smithi112.stdout:2/305: fdatasync d2/dd/d48/f24 0 2024-09-16T00:25:55.156 INFO:tasks.workunit.client.0.smithi112.stdout:7/316: truncate d6/d10/f3d 5473119 0 2024-09-16T00:25:55.157 INFO:tasks.workunit.client.1.smithi169.stdout:0/96: creat d0/d2/d15/f22 x:0 0 0 2024-09-16T00:25:55.159 INFO:tasks.workunit.client.1.smithi169.stdout:0/97: dread - d0/d2/d8/fe zero size 2024-09-16T00:25:55.159 INFO:tasks.workunit.client.1.smithi169.stdout:5/128: unlink d2/da/d10/d13/f23 0 2024-09-16T00:25:55.162 INFO:tasks.workunit.client.1.smithi169.stdout:9/92: dwrite d5/d12/f1e [0,4194304] 0 2024-09-16T00:25:55.162 INFO:tasks.workunit.client.1.smithi169.stdout:0/98: creat d0/f23 x:0 0 0 2024-09-16T00:25:55.165 INFO:tasks.workunit.client.1.smithi169.stdout:0/99: readlink d0/d2/d8/l1e 0 2024-09-16T00:25:55.166 INFO:tasks.workunit.client.1.smithi169.stdout:5/129: fdatasync d2/ff 0 2024-09-16T00:25:55.170 INFO:tasks.workunit.client.1.smithi169.stdout:9/93: mknod d5/d12/c24 0 2024-09-16T00:25:55.172 INFO:tasks.workunit.client.1.smithi169.stdout:9/94: stat d5/c9 0 2024-09-16T00:25:55.175 INFO:tasks.workunit.client.1.smithi169.stdout:9/95: chown d5/d12/l1d 31671 1 2024-09-16T00:25:55.175 INFO:tasks.workunit.client.1.smithi169.stdout:5/130: symlink d2/da/d10/d13/d1c/d2d/l2e 0 2024-09-16T00:25:55.179 INFO:tasks.workunit.client.1.smithi169.stdout:0/100: dwrite d0/d2/d8/d5/d14/f1c [0,4194304] 0 2024-09-16T00:25:55.179 INFO:tasks.workunit.client.1.smithi169.stdout:5/131: fdatasync d2/fd 0 2024-09-16T00:25:55.181 INFO:tasks.workunit.client.1.smithi169.stdout:5/132: chown d2/f2a 1855477941 1 2024-09-16T00:25:55.181 INFO:tasks.workunit.client.1.smithi169.stdout:0/101: fsync d0/d2/d8/d5/fa 0 2024-09-16T00:25:55.182 INFO:tasks.workunit.client.1.smithi169.stdout:4/79: dwrite d2/f3 [0,4194304] 0 2024-09-16T00:25:55.184 INFO:tasks.workunit.client.1.smithi169.stdout:4/80: mknod d2/c1c 0 2024-09-16T00:25:55.184 INFO:tasks.workunit.client.0.smithi112.stdout:6/251: truncate dd/df/f4c 849054 0 2024-09-16T00:25:55.184 INFO:tasks.workunit.client.0.smithi112.stdout:9/443: dwrite d1/d5/d6/d2d/f3b [0,4194304] 0 2024-09-16T00:25:55.185 INFO:tasks.workunit.client.1.smithi169.stdout:0/102: creat d0/d2/d8/d5/f24 x:0 0 0 2024-09-16T00:25:55.186 INFO:tasks.workunit.client.0.smithi112.stdout:9/444: dread - d1/d5/d20/d25/d66/f9d zero size 2024-09-16T00:25:55.187 INFO:tasks.workunit.client.1.smithi169.stdout:0/103: dread - d0/d2/d15/f22 zero size 2024-09-16T00:25:55.188 INFO:tasks.workunit.client.0.smithi112.stdout:9/445: readlink d1/d5/d6/d2d/d39/l68 0 2024-09-16T00:25:55.192 INFO:tasks.workunit.client.0.smithi112.stdout:9/446: rename d1/fa3 to d1/d5/fa6 0 2024-09-16T00:25:55.193 INFO:tasks.workunit.client.1.smithi169.stdout:0/104: unlink d0/d2/d8/ff 0 2024-09-16T00:25:55.193 INFO:tasks.workunit.client.0.smithi112.stdout:9/447: write d1/d5/f16 [156515,98915] 0 2024-09-16T00:25:55.195 INFO:tasks.workunit.client.1.smithi169.stdout:5/133: dwrite d2/da/f28 [0,4194304] 0 2024-09-16T00:25:55.197 INFO:tasks.workunit.client.0.smithi112.stdout:9/448: mknod d1/d5/d6/d59/ca7 0 2024-09-16T00:25:55.201 INFO:tasks.workunit.client.0.smithi112.stdout:9/449: fsync d1/d5/d6/f34 0 2024-09-16T00:25:55.201 INFO:tasks.workunit.client.0.smithi112.stdout:3/328: dwrite d2/d9/d2b/d47/f30 [0,4194304] 0 2024-09-16T00:25:55.202 INFO:tasks.workunit.client.1.smithi169.stdout:4/81: dread d2/f3 [0,4194304] 0 2024-09-16T00:25:55.203 INFO:tasks.workunit.client.0.smithi112.stdout:9/450: read - d1/d5/d20/d25/f43 zero size 2024-09-16T00:25:55.205 INFO:tasks.workunit.client.1.smithi169.stdout:0/105: symlink d0/d2/d8/d5/d14/d16/d1a/l25 0 2024-09-16T00:25:55.206 INFO:tasks.workunit.client.0.smithi112.stdout:3/329: unlink d2/d9/d2b/d2c/c6c 0 2024-09-16T00:25:55.207 INFO:tasks.workunit.client.1.smithi169.stdout:4/82: rename d2/c5 to d2/d12/d13/c1d 0 2024-09-16T00:25:55.209 INFO:tasks.workunit.client.0.smithi112.stdout:9/451: chown d1/d5/d6/d38/la2 601160 1 2024-09-16T00:25:55.209 INFO:tasks.workunit.client.0.smithi112.stdout:3/330: chown d2/d9/d2b/d47/l5c 89 1 2024-09-16T00:25:55.211 INFO:tasks.workunit.client.1.smithi169.stdout:9/96: fdatasync d5/d12/f1e 0 2024-09-16T00:25:55.213 INFO:tasks.workunit.client.0.smithi112.stdout:3/331: creat d2/d9/d2b/d2c/d69/f72 x:0 0 0 2024-09-16T00:25:55.214 INFO:tasks.workunit.client.1.smithi169.stdout:0/106: dwrite d0/d2/fd [0,4194304] 0 2024-09-16T00:25:55.214 INFO:tasks.workunit.client.0.smithi112.stdout:9/452: mkdir d1/d5/d20/d81/d1a/d88/da8 0 2024-09-16T00:25:55.216 INFO:tasks.workunit.client.0.smithi112.stdout:2/306: dwrite d2/dd/f15 [0,4194304] 0 2024-09-16T00:25:55.216 INFO:tasks.workunit.client.0.smithi112.stdout:3/332: fdatasync d2/fb 0 2024-09-16T00:25:55.216 INFO:tasks.workunit.client.1.smithi169.stdout:4/83: creat d2/d12/f1e x:0 0 0 2024-09-16T00:25:55.217 INFO:tasks.workunit.client.0.smithi112.stdout:9/453: dread - d1/d5/d6/d2d/f97 zero size 2024-09-16T00:25:55.218 INFO:tasks.workunit.client.0.smithi112.stdout:2/307: dread - d2/dd/d48/d22/d2b/d30/f44 zero size 2024-09-16T00:25:55.220 INFO:tasks.workunit.client.1.smithi169.stdout:0/107: rename d0/d2/d15/l20 to d0/d2/d15/l26 0 2024-09-16T00:25:55.221 INFO:tasks.workunit.client.1.smithi169.stdout:4/84: dread d2/fb [0,4194304] 0 2024-09-16T00:25:55.222 INFO:tasks.workunit.client.1.smithi169.stdout:9/97: rename d5/c7 to d5/da/c25 0 2024-09-16T00:25:55.225 INFO:tasks.workunit.client.1.smithi169.stdout:0/108: dread d0/d2/d8/d5/d14/f1c [0,4194304] 0 2024-09-16T00:25:55.226 INFO:tasks.workunit.client.1.smithi169.stdout:9/98: truncate d5/da/fc 520783 0 2024-09-16T00:25:55.226 INFO:tasks.workunit.client.0.smithi112.stdout:2/308: stat d2/dd/d48/d22/d2b/d3b 0 2024-09-16T00:25:55.226 INFO:tasks.workunit.client.0.smithi112.stdout:3/333: dwrite d2/f3f [0,4194304] 0 2024-09-16T00:25:55.227 INFO:tasks.workunit.client.1.smithi169.stdout:0/109: dread - d0/f23 zero size 2024-09-16T00:25:55.227 INFO:tasks.workunit.client.1.smithi169.stdout:4/85: dread d2/f10 [0,4194304] 0 2024-09-16T00:25:55.229 INFO:tasks.workunit.client.1.smithi169.stdout:4/86: dread - d2/dd/d1a/f1b zero size 2024-09-16T00:25:55.230 INFO:tasks.workunit.client.0.smithi112.stdout:6/252: dread dd/f2a [0,4194304] 0 2024-09-16T00:25:55.231 INFO:tasks.workunit.client.0.smithi112.stdout:3/334: rmdir d2/d9/d2b/d33/d68 39 2024-09-16T00:25:55.231 INFO:tasks.workunit.client.1.smithi169.stdout:0/110: write d0/d2/d8/d5/d14/f1c [120963,23818] 0 2024-09-16T00:25:55.231 INFO:tasks.workunit.client.1.smithi169.stdout:4/87: fsync d2/f3 0 2024-09-16T00:25:55.232 INFO:tasks.workunit.client.0.smithi112.stdout:2/309: symlink d2/dd/d48/d22/d2b/d3b/l5d 0 2024-09-16T00:25:55.233 INFO:tasks.workunit.client.1.smithi169.stdout:4/88: mkdir d2/d12/d13/d1f 0 2024-09-16T00:25:55.233 INFO:tasks.workunit.client.1.smithi169.stdout:4/89: chown d2/d12/d13 251381 1 2024-09-16T00:25:55.234 INFO:tasks.workunit.client.0.smithi112.stdout:3/335: write d2/d9/d2b/f4b [499316,47433] 0 2024-09-16T00:25:55.239 INFO:tasks.workunit.client.0.smithi112.stdout:3/336: symlink d2/d9/d2b/d2c/l73 0 2024-09-16T00:25:55.239 INFO:tasks.workunit.client.0.smithi112.stdout:6/253: dwrite dd/df/d17/d27/d54/f56 [0,4194304] 0 2024-09-16T00:25:55.242 INFO:tasks.workunit.client.1.smithi169.stdout:4/90: dwrite d2/d12/f1e [0,4194304] 0 2024-09-16T00:25:55.245 INFO:tasks.workunit.client.0.smithi112.stdout:2/310: dwrite d2/f46 [0,4194304] 0 2024-09-16T00:25:55.245 INFO:tasks.workunit.client.0.smithi112.stdout:6/254: truncate dd/df/d10/d1d/d37/d42/f4a 197218 0 2024-09-16T00:25:55.246 INFO:tasks.workunit.client.0.smithi112.stdout:3/337: mknod d2/d9/d2b/d2c/c74 0 2024-09-16T00:25:55.248 INFO:tasks.workunit.client.0.smithi112.stdout:6/255: creat dd/df/d17/f59 x:0 0 0 2024-09-16T00:25:55.258 INFO:tasks.workunit.client.0.smithi112.stdout:3/338: dwrite d2/d9/d2b/d47/d52/f40 [0,4194304] 0 2024-09-16T00:25:55.259 INFO:tasks.workunit.client.1.smithi169.stdout:4/91: truncate d2/f4 4130507 0 2024-09-16T00:25:55.259 INFO:tasks.workunit.client.0.smithi112.stdout:9/454: truncate d1/d5/d6/d2d/f3b 3115148 0 2024-09-16T00:25:55.261 INFO:tasks.workunit.client.0.smithi112.stdout:3/339: write d2/d9/d2b/d33/f50 [2122840,48074] 0 2024-09-16T00:25:55.262 INFO:tasks.workunit.client.0.smithi112.stdout:9/455: write d1/d5/f2f [204771,45486] 0 2024-09-16T00:25:55.262 INFO:tasks.workunit.client.1.smithi169.stdout:5/134: dwrite d2/f4 [0,4194304] 0 2024-09-16T00:25:55.266 INFO:tasks.workunit.client.1.smithi169.stdout:4/92: creat d2/d12/d13/d1f/f20 x:0 0 0 2024-09-16T00:25:55.268 INFO:tasks.workunit.client.1.smithi169.stdout:9/99: dwrite d5/f8 [4194304,4194304] 0 2024-09-16T00:25:55.270 INFO:tasks.workunit.client.1.smithi169.stdout:4/93: fsync d2/f3 0 2024-09-16T00:25:55.270 INFO:tasks.workunit.client.0.smithi112.stdout:3/340: dwrite d2/d9/d2b/d33/d49/f4c [4194304,4194304] 0 2024-09-16T00:25:55.271 INFO:tasks.workunit.client.0.smithi112.stdout:9/456: dwrite d1/d5/d6/d2d/d6a/f6e [0,4194304] 0 2024-09-16T00:25:55.274 INFO:tasks.workunit.client.1.smithi169.stdout:4/94: write d2/f9 [4919070,79929] 0 2024-09-16T00:25:55.277 INFO:tasks.workunit.client.0.smithi112.stdout:9/457: mkdir d1/d5/d20/d25/da9 0 2024-09-16T00:25:55.281 INFO:tasks.workunit.client.1.smithi169.stdout:5/135: dwrite d2/da/d10/d13/d24/f2c [0,4194304] 0 2024-09-16T00:25:55.282 INFO:tasks.workunit.client.0.smithi112.stdout:9/458: creat d1/d5/d20/d81/faa x:0 0 0 2024-09-16T00:25:55.282 INFO:tasks.workunit.client.0.smithi112.stdout:3/341: dwrite d2/d9/d2b/d33/f50 [4194304,4194304] 0 2024-09-16T00:25:55.283 INFO:tasks.workunit.client.1.smithi169.stdout:9/100: rename d5/f17 to d5/d12/f26 0 2024-09-16T00:25:55.284 INFO:tasks.workunit.client.0.smithi112.stdout:9/459: stat d1/d5/d6/d2d/d39 0 2024-09-16T00:25:55.286 INFO:tasks.workunit.client.1.smithi169.stdout:4/95: mkdir d2/d12/d21 0 2024-09-16T00:25:55.287 INFO:tasks.workunit.client.0.smithi112.stdout:3/342: symlink d2/d9/d2b/d47/d52/l75 0 2024-09-16T00:25:55.287 INFO:tasks.workunit.client.0.smithi112.stdout:9/460: chown d1/d5/d6/d18/f28 175592999 1 2024-09-16T00:25:55.287 INFO:tasks.workunit.client.0.smithi112.stdout:3/343: stat d2/d9/l5f 0 2024-09-16T00:25:55.288 INFO:tasks.workunit.client.1.smithi169.stdout:9/101: creat d5/da/d21/d23/f27 x:0 0 0 2024-09-16T00:25:55.289 INFO:tasks.workunit.client.0.smithi112.stdout:8/228: sync 2024-09-16T00:25:55.292 INFO:tasks.workunit.client.0.smithi112.stdout:9/461: symlink d1/d5/d2c/d6f/lab 0 2024-09-16T00:25:55.293 INFO:tasks.workunit.client.1.smithi169.stdout:9/102: rename d5/f19 to d5/da/f28 0 2024-09-16T00:25:55.295 INFO:tasks.workunit.client.0.smithi112.stdout:2/311: dread d2/fa [0,4194304] 0 2024-09-16T00:25:55.296 INFO:tasks.workunit.client.1.smithi169.stdout:9/103: link c3 d5/da/d21/c29 0 2024-09-16T00:25:55.296 INFO:tasks.workunit.client.0.smithi112.stdout:8/229: rename d10/d12/d24/l40 to d10/d45/l4c 0 2024-09-16T00:25:55.297 INFO:tasks.workunit.client.0.smithi112.stdout:3/344: dread d2/d9/d2b/d33/f50 [0,4194304] 0 2024-09-16T00:25:55.297 INFO:tasks.workunit.client.0.smithi112.stdout:2/312: fsync d2/f19 0 2024-09-16T00:25:55.300 INFO:tasks.workunit.client.0.smithi112.stdout:8/230: write d10/d12/d24/f2e [434049,49690] 0 2024-09-16T00:25:55.300 INFO:tasks.workunit.client.1.smithi169.stdout:9/104: dread d5/f6 [0,4194304] 0 2024-09-16T00:25:55.301 INFO:tasks.workunit.client.0.smithi112.stdout:2/313: creat d2/dd/f5e x:0 0 0 2024-09-16T00:25:55.302 INFO:tasks.workunit.client.1.smithi169.stdout:9/105: unlink d5/c9 0 2024-09-16T00:25:55.304 INFO:tasks.workunit.client.1.smithi169.stdout:9/106: symlink d5/l2a 0 2024-09-16T00:25:55.305 INFO:tasks.workunit.client.0.smithi112.stdout:2/314: mknod d2/c5f 0 2024-09-16T00:25:55.305 INFO:tasks.workunit.client.1.smithi169.stdout:9/107: fdatasync d5/d12/f1e 0 2024-09-16T00:25:55.305 INFO:tasks.workunit.client.0.smithi112.stdout:9/462: dread d1/d5/d20/d81/f1d [0,4194304] 0 2024-09-16T00:25:55.306 INFO:tasks.workunit.client.1.smithi169.stdout:9/108: write d5/da/fc [367943,80091] 0 2024-09-16T00:25:55.306 INFO:tasks.workunit.client.1.smithi169.stdout:9/109: readlink d5/l2a 0 2024-09-16T00:25:55.307 INFO:tasks.workunit.client.1.smithi169.stdout:9/110: dread - d5/d12/f26 zero size 2024-09-16T00:25:55.307 INFO:tasks.workunit.client.1.smithi169.stdout:9/111: dread - d5/da/d21/d23/f27 zero size 2024-09-16T00:25:55.309 INFO:tasks.workunit.client.0.smithi112.stdout:3/345: dwrite d2/f1a [4194304,4194304] 0 2024-09-16T00:25:55.309 INFO:tasks.workunit.client.0.smithi112.stdout:9/463: mknod d1/d5/cac 0 2024-09-16T00:25:55.310 INFO:tasks.workunit.client.1.smithi169.stdout:9/112: creat d5/da/f2b x:0 0 0 2024-09-16T00:25:55.311 INFO:tasks.workunit.client.0.smithi112.stdout:9/464: write d1/d5/d6/d2d/d6a/d89/f90 [667046,124577] 0 2024-09-16T00:25:55.312 INFO:tasks.workunit.client.0.smithi112.stdout:2/315: dread d2/dd/d48/d22/f29 [0,4194304] 0 2024-09-16T00:25:55.312 INFO:tasks.workunit.client.0.smithi112.stdout:3/346: mkdir d2/d9/d2b/d33/d76 0 2024-09-16T00:25:55.313 INFO:tasks.workunit.client.0.smithi112.stdout:9/465: truncate d1/d5/fa6 376398 0 2024-09-16T00:25:55.315 INFO:tasks.workunit.client.0.smithi112.stdout:3/347: chown d2/d9/l5f 84 1 2024-09-16T00:25:55.315 INFO:tasks.workunit.client.0.smithi112.stdout:6/256: dread dd/df/d10/f1b [4194304,4194304] 0 2024-09-16T00:25:55.316 INFO:tasks.workunit.client.0.smithi112.stdout:2/316: mkdir d2/dd/d48/d22/d2b/d54/d60 0 2024-09-16T00:25:55.317 INFO:tasks.workunit.client.0.smithi112.stdout:6/257: fsync dd/df/d10/d1d/d37/d42/f4a 0 2024-09-16T00:25:55.319 INFO:tasks.workunit.client.0.smithi112.stdout:3/348: creat d2/d9/d2b/d33/d49/f77 x:0 0 0 2024-09-16T00:25:55.322 INFO:tasks.workunit.client.0.smithi112.stdout:6/258: rename dd/df/d10/d1d to dd/d15/d5a 0 2024-09-16T00:25:55.322 INFO:tasks.workunit.client.1.smithi169.stdout:9/113: getdents d5/d12 0 2024-09-16T00:25:55.323 INFO:tasks.workunit.client.1.smithi169.stdout:9/114: read d5/d12/f1e [4110874,34253] 0 2024-09-16T00:25:55.324 INFO:tasks.workunit.client.0.smithi112.stdout:3/349: mkdir d2/d9/d2b/d47/d78 0 2024-09-16T00:25:55.324 INFO:tasks.workunit.client.1.smithi169.stdout:4/96: truncate d2/f9 3713795 0 2024-09-16T00:25:55.325 INFO:tasks.workunit.client.0.smithi112.stdout:6/259: creat dd/df/d10/d47/f5b x:0 0 0 2024-09-16T00:25:55.325 INFO:tasks.workunit.client.1.smithi169.stdout:4/97: chown d2/dd/d1a/f1b 318 1 2024-09-16T00:25:55.326 INFO:tasks.workunit.client.0.smithi112.stdout:2/317: getdents d2/dd/d48 0 2024-09-16T00:25:55.328 INFO:tasks.workunit.client.1.smithi169.stdout:9/115: rename d5/d12/f1e to d5/da/d21/f2c 0 2024-09-16T00:25:55.328 INFO:tasks.workunit.client.0.smithi112.stdout:6/260: write dd/df/d17/d27/d54/f56 [1419618,126238] 0 2024-09-16T00:25:55.328 INFO:tasks.workunit.client.0.smithi112.stdout:3/350: mknod d2/d9/d2b/c79 0 2024-09-16T00:25:55.328 INFO:tasks.workunit.client.0.smithi112.stdout:2/318: stat d2/dd/d48/d22/d2b/d54/f55 0 2024-09-16T00:25:55.329 INFO:tasks.workunit.client.0.smithi112.stdout:2/319: stat d2/dd/d48/d22/c38 0 2024-09-16T00:25:55.331 INFO:tasks.workunit.client.0.smithi112.stdout:3/351: write d2/d9/d2b/d39/f3c [4659716,71845] 0 2024-09-16T00:25:55.332 INFO:tasks.workunit.client.0.smithi112.stdout:6/261: mknod dd/df/d17/d27/d54/c5c 0 2024-09-16T00:25:55.334 INFO:tasks.workunit.client.0.smithi112.stdout:8/231: dread fe [0,4194304] 0 2024-09-16T00:25:55.337 INFO:tasks.workunit.client.0.smithi112.stdout:2/320: dread d2/dd/d48/d22/d2b/d30/f36 [0,4194304] 0 2024-09-16T00:25:55.337 INFO:tasks.workunit.client.0.smithi112.stdout:6/262: fdatasync dd/d15/f29 0 2024-09-16T00:25:55.337 INFO:tasks.workunit.client.1.smithi169.stdout:4/98: dread d2/f9 [0,4194304] 0 2024-09-16T00:25:55.337 INFO:tasks.workunit.client.0.smithi112.stdout:8/232: truncate d10/d12/d24/f48 1302230 0 2024-09-16T00:25:55.339 INFO:tasks.workunit.client.1.smithi169.stdout:4/99: write d2/f9 [2873619,78307] 0 2024-09-16T00:25:55.342 INFO:tasks.workunit.client.0.smithi112.stdout:8/233: creat d10/d12/d1c/d3b/f4d x:0 0 0 2024-09-16T00:25:55.342 INFO:tasks.workunit.client.0.smithi112.stdout:3/352: dread d2/d9/d2b/f29 [0,4194304] 0 2024-09-16T00:25:55.344 INFO:tasks.workunit.client.0.smithi112.stdout:2/321: creat d2/dd/d48/f61 x:0 0 0 2024-09-16T00:25:55.344 INFO:tasks.workunit.client.1.smithi169.stdout:9/116: dwrite d5/d12/f26 [0,4194304] 0 2024-09-16T00:25:55.345 INFO:tasks.workunit.client.0.smithi112.stdout:6/263: rename dd/df/d13/d18/l3e to dd/df/d13/d18/l5d 0 2024-09-16T00:25:55.345 INFO:tasks.workunit.client.1.smithi169.stdout:4/100: rename d2/d12/d13/l16 to d2/dd/d1a/l22 0 2024-09-16T00:25:55.346 INFO:tasks.workunit.client.1.smithi169.stdout:4/101: dread - d2/dd/f18 zero size 2024-09-16T00:25:55.347 INFO:tasks.workunit.client.1.smithi169.stdout:9/117: chown d5/da/c18 33636383 1 2024-09-16T00:25:55.348 INFO:tasks.workunit.client.1.smithi169.stdout:9/118: fdatasync d5/da/fc 0 2024-09-16T00:25:55.349 INFO:tasks.workunit.client.0.smithi112.stdout:9/466: truncate d1/d5/d20/f93 57586 0 2024-09-16T00:25:55.350 INFO:tasks.workunit.client.0.smithi112.stdout:3/353: mkdir d2/d9/d2b/d47/d52/d7a 0 2024-09-16T00:25:55.350 INFO:tasks.workunit.client.0.smithi112.stdout:9/467: fsync d1/d5/d20/f69 0 2024-09-16T00:25:55.351 INFO:tasks.workunit.client.1.smithi169.stdout:4/102: rename l0 to d2/d12/l23 0 2024-09-16T00:25:55.352 INFO:tasks.workunit.client.1.smithi169.stdout:9/119: creat d5/da/d21/d23/f2d x:0 0 0 2024-09-16T00:25:55.352 INFO:tasks.workunit.client.1.smithi169.stdout:4/103: chown d2/f3 161 1 2024-09-16T00:25:55.353 INFO:tasks.workunit.client.0.smithi112.stdout:2/322: dwrite d2/dd/f5e [0,4194304] 0 2024-09-16T00:25:55.354 INFO:tasks.workunit.client.0.smithi112.stdout:6/264: unlink dd/d15/d5a/d37/l38 0 2024-09-16T00:25:55.354 INFO:tasks.workunit.client.1.smithi169.stdout:4/104: creat d2/dd/f24 x:0 0 0 2024-09-16T00:25:55.354 INFO:tasks.workunit.client.1.smithi169.stdout:6/48: sync 2024-09-16T00:25:55.358 INFO:tasks.workunit.client.1.smithi169.stdout:6/49: mknod d2/c10 0 2024-09-16T00:25:55.359 INFO:tasks.workunit.client.0.smithi112.stdout:8/234: rmdir d10/d12/d1c/d49 0 2024-09-16T00:25:55.360 INFO:tasks.workunit.client.1.smithi169.stdout:9/120: fdatasync d5/d12/f26 0 2024-09-16T00:25:55.362 INFO:tasks.workunit.client.0.smithi112.stdout:9/468: creat d1/d5/d6/d2d/d6a/fad x:0 0 0 2024-09-16T00:25:55.363 INFO:tasks.workunit.client.1.smithi169.stdout:4/105: dwrite d2/dd/f18 [0,4194304] 0 2024-09-16T00:25:55.363 INFO:tasks.workunit.client.1.smithi169.stdout:9/121: chown d5/da/fc 2619808 1 2024-09-16T00:25:55.363 INFO:tasks.workunit.client.0.smithi112.stdout:9/469: truncate d1/f41 1224761 0 2024-09-16T00:25:55.364 INFO:tasks.workunit.client.0.smithi112.stdout:3/354: mkdir d2/d9/d2b/d33/d7b 0 2024-09-16T00:25:55.364 INFO:tasks.workunit.client.0.smithi112.stdout:6/265: dwrite dd/d15/f29 [0,4194304] 0 2024-09-16T00:25:55.365 INFO:tasks.workunit.client.1.smithi169.stdout:4/106: fsync d2/fb 0 2024-09-16T00:25:55.366 INFO:tasks.workunit.client.1.smithi169.stdout:6/50: dread d2/fc [4194304,4194304] 0 2024-09-16T00:25:55.367 INFO:tasks.workunit.client.1.smithi169.stdout:9/122: mkdir d5/da/d2e 0 2024-09-16T00:25:55.368 INFO:tasks.workunit.client.0.smithi112.stdout:9/470: read d1/d5/d6/d2d/f92 [464114,77439] 0 2024-09-16T00:25:55.368 INFO:tasks.workunit.client.1.smithi169.stdout:4/107: creat d2/d12/d13/f25 x:0 0 0 2024-09-16T00:25:55.369 INFO:tasks.workunit.client.1.smithi169.stdout:4/108: read d2/dd/f18 [1554941,114313] 0 2024-09-16T00:25:55.371 INFO:tasks.workunit.client.0.smithi112.stdout:8/235: mkdir d10/d4e 0 2024-09-16T00:25:55.372 INFO:tasks.workunit.client.0.smithi112.stdout:8/236: dread - d10/d12/f33 zero size 2024-09-16T00:25:55.374 INFO:tasks.workunit.client.0.smithi112.stdout:2/323: rename d2/dd/d48/d22/d2b/d3b/d49/l56 to d2/l62 0 2024-09-16T00:25:55.374 INFO:tasks.workunit.client.0.smithi112.stdout:8/237: read d10/f21 [2588671,71360] 0 2024-09-16T00:25:55.380 INFO:tasks.workunit.client.0.smithi112.stdout:3/355: fdatasync d2/d9/d2b/d2c/f42 0 2024-09-16T00:25:55.380 INFO:tasks.workunit.client.0.smithi112.stdout:2/324: symlink d2/dd/d48/d22/d2b/d3b/l63 0 2024-09-16T00:25:55.380 INFO:tasks.workunit.client.0.smithi112.stdout:3/356: stat d2/d9/d2b/d33/d7b 0 2024-09-16T00:25:55.381 INFO:tasks.workunit.client.0.smithi112.stdout:3/357: chown d2/d9/f21 422548 1 2024-09-16T00:25:55.383 INFO:tasks.workunit.client.1.smithi169.stdout:6/51: mknod d2/c11 0 2024-09-16T00:25:55.383 INFO:tasks.workunit.client.0.smithi112.stdout:8/238: unlink d10/d12/d24/f48 0 2024-09-16T00:25:55.383 INFO:tasks.workunit.client.1.smithi169.stdout:9/123: write d5/da/d21/f2c [5149190,68650] 0 2024-09-16T00:25:55.384 INFO:tasks.workunit.client.0.smithi112.stdout:8/239: fsync d10/d12/f16 0 2024-09-16T00:25:55.384 INFO:tasks.workunit.client.1.smithi169.stdout:6/52: write d2/fb [1692597,125715] 0 2024-09-16T00:25:55.384 INFO:tasks.workunit.client.1.smithi169.stdout:4/109: creat d2/dd/f26 x:0 0 0 2024-09-16T00:25:55.385 INFO:tasks.workunit.client.1.smithi169.stdout:9/124: write d5/f14 [2649037,28256] 0 2024-09-16T00:25:55.387 INFO:tasks.workunit.client.1.smithi169.stdout:6/53: mknod d2/c12 0 2024-09-16T00:25:55.389 INFO:tasks.workunit.client.0.smithi112.stdout:2/325: rmdir d2/dd/d48/d22/d2b/d3b/d49 39 2024-09-16T00:25:55.390 INFO:tasks.workunit.client.1.smithi169.stdout:4/110: dread d2/f3 [0,4194304] 0 2024-09-16T00:25:55.391 INFO:tasks.workunit.client.0.smithi112.stdout:9/471: rename d1/d5/d6/d2d/f8d to d1/d5/d20/fae 0 2024-09-16T00:25:55.392 INFO:tasks.workunit.client.1.smithi169.stdout:9/125: link d5/d12/c1b d5/da/d21/d23/c2f 0 2024-09-16T00:25:55.393 INFO:tasks.workunit.client.1.smithi169.stdout:4/111: truncate d2/dd/f24 205356 0 2024-09-16T00:25:55.393 INFO:tasks.workunit.client.1.smithi169.stdout:9/126: dread - d5/da/d21/d23/f2d zero size 2024-09-16T00:25:55.394 INFO:tasks.workunit.client.0.smithi112.stdout:3/358: rename l0 to d2/d9/d2b/d2c/d69/l7c 0 2024-09-16T00:25:55.395 INFO:tasks.workunit.client.0.smithi112.stdout:6/266: write dd/df/d10/f3a [404139,37906] 0 2024-09-16T00:25:55.396 INFO:tasks.workunit.client.1.smithi169.stdout:6/54: dwrite d2/fc [4194304,4194304] 0 2024-09-16T00:25:55.397 INFO:tasks.workunit.client.0.smithi112.stdout:9/472: dread d1/d5/d2c/f5b [0,4194304] 0 2024-09-16T00:25:55.397 INFO:tasks.workunit.client.0.smithi112.stdout:6/267: fsync dd/d15/f29 0 2024-09-16T00:25:55.397 INFO:tasks.workunit.client.0.smithi112.stdout:3/359: symlink d2/d4d/l7d 0 2024-09-16T00:25:55.397 INFO:tasks.workunit.client.1.smithi169.stdout:9/127: rename d5/f6 to d5/da/d21/f30 0 2024-09-16T00:25:55.398 INFO:tasks.workunit.client.0.smithi112.stdout:2/326: creat d2/f64 x:0 0 0 2024-09-16T00:25:55.398 INFO:tasks.workunit.client.1.smithi169.stdout:4/112: rmdir d2/d12/d21 0 2024-09-16T00:25:55.399 INFO:tasks.workunit.client.0.smithi112.stdout:6/268: write dd/df/d10/d47/f57 [683275,64224] 0 2024-09-16T00:25:55.400 INFO:tasks.workunit.client.0.smithi112.stdout:6/269: chown dd/df/d11/d44 45105 1 2024-09-16T00:25:55.400 INFO:tasks.workunit.client.1.smithi169.stdout:4/113: dread - d2/d12/d13/d1f/f20 zero size 2024-09-16T00:25:55.401 INFO:tasks.workunit.client.0.smithi112.stdout:3/360: unlink d2/d9/d2b/d33/d49/l57 0 2024-09-16T00:25:55.401 INFO:tasks.workunit.client.0.smithi112.stdout:6/270: dread - dd/df/d13/f48 zero size 2024-09-16T00:25:55.401 INFO:tasks.workunit.client.1.smithi169.stdout:4/114: read d2/f9 [3454306,84292] 0 2024-09-16T00:25:55.404 INFO:tasks.workunit.client.0.smithi112.stdout:6/271: dread dd/df/f4c [0,4194304] 0 2024-09-16T00:25:55.405 INFO:tasks.workunit.client.1.smithi169.stdout:9/128: creat d5/d12/f31 x:0 0 0 2024-09-16T00:25:55.406 INFO:tasks.workunit.client.0.smithi112.stdout:6/272: creat dd/df/d10/d47/f5e x:0 0 0 2024-09-16T00:25:55.406 INFO:tasks.workunit.client.1.smithi169.stdout:9/129: mknod d5/da/d21/c32 0 2024-09-16T00:25:55.409 INFO:tasks.workunit.client.0.smithi112.stdout:6/273: dread dd/df/f21 [0,4194304] 0 2024-09-16T00:25:55.410 INFO:tasks.workunit.client.0.smithi112.stdout:3/361: write d2/f2d [1313251,81679] 0 2024-09-16T00:25:55.411 INFO:tasks.workunit.client.0.smithi112.stdout:0/289: sync 2024-09-16T00:25:55.411 INFO:tasks.workunit.client.0.smithi112.stdout:6/274: read dd/d15/d5a/d37/d42/f4a [61475,35224] 0 2024-09-16T00:25:55.412 INFO:tasks.workunit.client.0.smithi112.stdout:5/294: sync 2024-09-16T00:25:55.412 INFO:tasks.workunit.client.0.smithi112.stdout:4/238: sync 2024-09-16T00:25:55.413 INFO:tasks.workunit.client.0.smithi112.stdout:3/362: write d2/f2d [1126593,107336] 0 2024-09-16T00:25:55.415 INFO:tasks.workunit.client.0.smithi112.stdout:4/239: symlink d6/d39/d1c/l52 0 2024-09-16T00:25:55.416 INFO:tasks.workunit.client.0.smithi112.stdout:5/295: unlink d4/dd/d12/c51 0 2024-09-16T00:25:55.418 INFO:tasks.workunit.client.0.smithi112.stdout:9/473: dwrite d1/f82 [0,4194304] 0 2024-09-16T00:25:55.419 INFO:tasks.workunit.client.0.smithi112.stdout:5/296: dread - d4/dd/d12/f3a zero size 2024-09-16T00:25:55.419 INFO:tasks.workunit.client.0.smithi112.stdout:3/363: creat d2/d9/d2b/d2c/f7e x:0 0 0 2024-09-16T00:25:55.421 INFO:tasks.workunit.client.0.smithi112.stdout:4/240: symlink d6/db/d1f/d41/d49/d31/l53 0 2024-09-16T00:25:55.423 INFO:tasks.workunit.client.0.smithi112.stdout:9/474: fsync d1/d5/d6/d18/f28 0 2024-09-16T00:25:55.426 INFO:tasks.workunit.client.0.smithi112.stdout:6/275: dwrite dd/df/d11/f49 [0,4194304] 0 2024-09-16T00:25:55.426 INFO:tasks.workunit.client.0.smithi112.stdout:4/241: chown d6/db/d1f/d41/d49/d31/d34/f3e 972421731 1 2024-09-16T00:25:55.426 INFO:tasks.workunit.client.0.smithi112.stdout:9/475: read - d1/d5/d6/d38/f5a zero size 2024-09-16T00:25:55.428 INFO:tasks.workunit.client.0.smithi112.stdout:5/297: creat d4/dd/d12/d56/f68 x:0 0 0 2024-09-16T00:25:55.430 INFO:tasks.workunit.client.0.smithi112.stdout:4/242: write d6/db/f42 [1051736,78582] 0 2024-09-16T00:25:55.430 INFO:tasks.workunit.client.0.smithi112.stdout:6/276: dread dd/df/d13/d18/f28 [0,4194304] 0 2024-09-16T00:25:55.432 INFO:tasks.workunit.client.0.smithi112.stdout:9/476: creat d1/d5/d20/d25/d66/faf x:0 0 0 2024-09-16T00:25:55.433 INFO:tasks.workunit.client.0.smithi112.stdout:5/298: rmdir d4/d1c/d43 39 2024-09-16T00:25:55.435 INFO:tasks.workunit.client.0.smithi112.stdout:6/277: fsync dd/f2a 0 2024-09-16T00:25:55.437 INFO:tasks.workunit.client.0.smithi112.stdout:9/477: creat d1/d5/d6/d18/fb0 x:0 0 0 2024-09-16T00:25:55.437 INFO:tasks.workunit.client.0.smithi112.stdout:4/243: truncate d6/db/d1f/d41/d49/d31/d34/f3f 1023080 0 2024-09-16T00:25:55.437 INFO:tasks.workunit.client.0.smithi112.stdout:3/364: link d2/d9/d2b/d33/d49/c58 d2/d9/d2b/c7f 0 2024-09-16T00:25:55.438 INFO:tasks.workunit.client.0.smithi112.stdout:5/299: mknod d4/c69 0 2024-09-16T00:25:55.439 INFO:tasks.workunit.client.0.smithi112.stdout:9/478: truncate d1/d5/d6/d59/d64/fa1 580236 0 2024-09-16T00:25:55.439 INFO:tasks.workunit.client.0.smithi112.stdout:5/300: dread - d4/dd/d12/d52/f63 zero size 2024-09-16T00:25:55.440 INFO:tasks.workunit.client.0.smithi112.stdout:5/301: chown d4/dd/d12 1345 1 2024-09-16T00:25:55.441 INFO:tasks.workunit.client.0.smithi112.stdout:9/479: dread - d1/d5/d20/d25/d66/f9d zero size 2024-09-16T00:25:55.443 INFO:tasks.workunit.client.0.smithi112.stdout:3/365: unlink d2/d9/d2b/d2c/l73 0 2024-09-16T00:25:55.444 INFO:tasks.workunit.client.0.smithi112.stdout:3/366: dread - d2/d9/d2b/d2c/f7e zero size 2024-09-16T00:25:55.445 INFO:tasks.workunit.client.0.smithi112.stdout:4/244: rename d6/c23 to d6/db/d1f/d41/c54 0 2024-09-16T00:25:55.449 INFO:tasks.workunit.client.0.smithi112.stdout:6/278: truncate dd/df/d11/f49 1736274 0 2024-09-16T00:25:55.453 INFO:tasks.workunit.client.0.smithi112.stdout:9/480: write d1/d5/d20/fae [834815,120055] 0 2024-09-16T00:25:55.456 INFO:tasks.workunit.client.0.smithi112.stdout:5/302: getdents d4/d7/d26 0 2024-09-16T00:25:55.456 INFO:tasks.workunit.client.0.smithi112.stdout:6/279: mknod dd/df/d4b/c5f 0 2024-09-16T00:25:55.458 INFO:tasks.workunit.client.0.smithi112.stdout:4/245: getdents d6 0 2024-09-16T00:25:55.459 INFO:tasks.workunit.client.0.smithi112.stdout:6/280: rmdir dd/d15/d5a 39 2024-09-16T00:25:55.459 INFO:tasks.workunit.client.0.smithi112.stdout:3/367: dwrite d2/d9/d2b/f29 [0,4194304] 0 2024-09-16T00:25:55.461 INFO:tasks.workunit.client.0.smithi112.stdout:5/303: write d4/dd/d12/f11 [2722579,46648] 0 2024-09-16T00:25:55.464 INFO:tasks.workunit.client.0.smithi112.stdout:5/304: unlink d4/ca 0 2024-09-16T00:25:55.464 INFO:tasks.workunit.client.0.smithi112.stdout:9/481: dwrite d1/d5/d6/d18/f4b [0,4194304] 0 2024-09-16T00:25:55.465 INFO:tasks.workunit.client.0.smithi112.stdout:6/281: creat dd/df/d10/f60 x:0 0 0 2024-09-16T00:25:55.467 INFO:tasks.workunit.client.0.smithi112.stdout:3/368: mkdir d2/d9/d2b/d47/d52/d7a/d80 0 2024-09-16T00:25:55.469 INFO:tasks.workunit.client.0.smithi112.stdout:5/305: creat d4/d7/d26/d4c/f6a x:0 0 0 2024-09-16T00:25:55.476 INFO:tasks.workunit.client.0.smithi112.stdout:9/482: dwrite d1/f41 [0,4194304] 0 2024-09-16T00:25:55.476 INFO:tasks.workunit.client.0.smithi112.stdout:5/306: creat d4/dd/f6b x:0 0 0 2024-09-16T00:25:55.477 INFO:tasks.workunit.client.0.smithi112.stdout:3/369: dread d2/d9/d2b/d47/f30 [0,4194304] 0 2024-09-16T00:25:55.479 INFO:tasks.workunit.client.0.smithi112.stdout:6/282: link dd/d15/d5a/d37/c51 dd/df/d11/c61 0 2024-09-16T00:25:55.480 INFO:tasks.workunit.client.0.smithi112.stdout:3/370: fsync d2/d9/d2b/d33/d49/f77 0 2024-09-16T00:25:55.486 INFO:tasks.workunit.client.0.smithi112.stdout:3/371: symlink d2/d9/d2b/d2c/d69/l81 0 2024-09-16T00:25:55.488 INFO:tasks.workunit.client.0.smithi112.stdout:6/283: rename dd/d15/d5a/d37/d42 to dd/d15/d5a/d37/d62 0 2024-09-16T00:25:55.489 INFO:tasks.workunit.client.0.smithi112.stdout:5/307: dwrite d4/dd/f6b [0,4194304] 0 2024-09-16T00:25:55.491 INFO:tasks.workunit.client.0.smithi112.stdout:6/284: fsync dd/d15/d5a/f26 0 2024-09-16T00:25:55.492 INFO:tasks.workunit.client.0.smithi112.stdout:9/483: dwrite d1/d5/d6/d18/f4b [0,4194304] 0 2024-09-16T00:25:55.494 INFO:tasks.workunit.client.0.smithi112.stdout:3/372: getdents d2/d9/d2b/d39/d66 0 2024-09-16T00:25:55.495 INFO:tasks.workunit.client.0.smithi112.stdout:3/373: fsync d2/f3f 0 2024-09-16T00:25:55.495 INFO:tasks.workunit.client.0.smithi112.stdout:5/308: creat d4/dd/d12/d56/f6c x:0 0 0 2024-09-16T00:25:55.500 INFO:tasks.workunit.client.0.smithi112.stdout:4/246: dread d6/d39/d1c/f2f [0,4194304] 0 2024-09-16T00:25:55.501 INFO:tasks.workunit.client.0.smithi112.stdout:6/285: truncate dd/df/d13/f1c 3962427 0 2024-09-16T00:25:55.502 INFO:tasks.workunit.client.0.smithi112.stdout:6/286: fsync dd/f2a 0 2024-09-16T00:25:55.503 INFO:tasks.workunit.client.0.smithi112.stdout:5/309: creat d4/d1c/d43/f6d x:0 0 0 2024-09-16T00:25:55.504 INFO:tasks.workunit.client.0.smithi112.stdout:3/374: creat d2/d9/d2b/f82 x:0 0 0 2024-09-16T00:25:55.506 INFO:tasks.workunit.client.0.smithi112.stdout:6/287: write dd/df/d13/d18/f28 [749860,25782] 0 2024-09-16T00:25:55.508 INFO:tasks.workunit.client.0.smithi112.stdout:6/288: chown dd/df/d17/c24 260076204 1 2024-09-16T00:25:55.509 INFO:tasks.workunit.client.0.smithi112.stdout:4/247: rename d6/d39/d36/c4d to d6/db/df/c55 0 2024-09-16T00:25:55.510 INFO:tasks.workunit.client.0.smithi112.stdout:6/289: read dd/d15/f29 [269639,81774] 0 2024-09-16T00:25:55.510 INFO:tasks.workunit.client.0.smithi112.stdout:5/310: dwrite d4/dd/d12/f48 [0,4194304] 0 2024-09-16T00:25:55.510 INFO:tasks.workunit.client.0.smithi112.stdout:3/375: unlink d2/d9/d2b/c79 0 2024-09-16T00:25:55.511 INFO:tasks.workunit.client.0.smithi112.stdout:4/248: write d6/da/f35 [495770,88631] 0 2024-09-16T00:25:55.513 INFO:tasks.workunit.client.0.smithi112.stdout:9/484: fdatasync d1/d5/d6/d18/f4b 0 2024-09-16T00:25:55.513 INFO:tasks.workunit.client.1.smithi169.stdout:6/55: getdents d2 0 2024-09-16T00:25:55.514 INFO:tasks.workunit.client.1.smithi169.stdout:4/115: dwrite d2/d12/f1e [0,4194304] 0 2024-09-16T00:25:55.514 INFO:tasks.workunit.client.1.smithi169.stdout:9/130: dread - d5/da/d21/d23/f27 zero size 2024-09-16T00:25:55.515 INFO:tasks.workunit.client.1.smithi169.stdout:6/56: write d2/fc [3088459,88093] 0 2024-09-16T00:25:55.515 INFO:tasks.workunit.client.1.smithi169.stdout:4/116: read - d2/dd/d1a/f1b zero size 2024-09-16T00:25:55.515 INFO:tasks.workunit.client.0.smithi112.stdout:5/311: mknod d4/d7/d26/d4c/c6e 0 2024-09-16T00:25:55.516 INFO:tasks.workunit.client.0.smithi112.stdout:3/376: fsync d2/d9/d2b/d2c/f63 0 2024-09-16T00:25:55.517 INFO:tasks.workunit.client.1.smithi169.stdout:4/117: dread - d2/d12/d13/d1f/f20 zero size 2024-09-16T00:25:55.517 INFO:tasks.workunit.client.0.smithi112.stdout:3/377: write d2/d9/d2b/d2c/d69/f6f [575958,124110] 0 2024-09-16T00:25:55.517 INFO:tasks.workunit.client.0.smithi112.stdout:5/312: readlink l3 0 2024-09-16T00:25:55.518 INFO:tasks.workunit.client.1.smithi169.stdout:4/118: write d2/fb [2315238,25870] 0 2024-09-16T00:25:55.518 INFO:tasks.workunit.client.0.smithi112.stdout:6/290: getdents dd/df/d10/d47 0 2024-09-16T00:25:55.518 INFO:tasks.workunit.client.1.smithi169.stdout:6/57: unlink d2/fc 0 2024-09-16T00:25:55.519 INFO:tasks.workunit.client.0.smithi112.stdout:9/485: symlink d1/d5/d20/d25/da9/lb1 0 2024-09-16T00:25:55.520 INFO:tasks.workunit.client.1.smithi169.stdout:9/131: unlink d5/da/c25 0 2024-09-16T00:25:55.521 INFO:tasks.workunit.client.0.smithi112.stdout:9/486: readlink d1/d5/d20/d25/da9/lb1 0 2024-09-16T00:25:55.522 INFO:tasks.workunit.client.1.smithi169.stdout:9/132: chown d5/da/d21/d23/f27 3995782 1 2024-09-16T00:25:55.522 INFO:tasks.workunit.client.1.smithi169.stdout:6/58: symlink d2/l13 0 2024-09-16T00:25:55.524 INFO:tasks.workunit.client.1.smithi169.stdout:6/59: symlink d2/l14 0 2024-09-16T00:25:55.525 INFO:tasks.workunit.client.1.smithi169.stdout:6/60: chown d2/ce 3241559 1 2024-09-16T00:25:55.525 INFO:tasks.workunit.client.1.smithi169.stdout:9/133: creat d5/da/d21/f33 x:0 0 0 2024-09-16T00:25:55.526 INFO:tasks.workunit.client.1.smithi169.stdout:6/61: read d2/fb [2459899,51796] 0 2024-09-16T00:25:55.528 INFO:tasks.workunit.client.1.smithi169.stdout:6/62: mkdir d2/d15 0 2024-09-16T00:25:55.529 INFO:tasks.workunit.client.0.smithi112.stdout:6/291: dwrite dd/d23/f35 [0,4194304] 0 2024-09-16T00:25:55.530 INFO:tasks.workunit.client.0.smithi112.stdout:4/249: getdents d6/db/d1f/d41/d49/d31 0 2024-09-16T00:25:55.530 INFO:tasks.workunit.client.1.smithi169.stdout:9/134: rmdir d5/da/d21/d23 39 2024-09-16T00:25:55.530 INFO:tasks.workunit.client.1.smithi169.stdout:6/63: fdatasync d2/fb 0 2024-09-16T00:25:55.531 INFO:tasks.workunit.client.0.smithi112.stdout:5/313: mkdir d4/dd/d6f 0 2024-09-16T00:25:55.531 INFO:tasks.workunit.client.0.smithi112.stdout:6/292: dread - dd/df/d17/f59 zero size 2024-09-16T00:25:55.533 INFO:tasks.workunit.client.1.smithi169.stdout:9/135: creat d5/da/f34 x:0 0 0 2024-09-16T00:25:55.533 INFO:tasks.workunit.client.0.smithi112.stdout:5/314: readlink d4/d1c/l5d 0 2024-09-16T00:25:55.536 INFO:tasks.workunit.client.1.smithi169.stdout:9/136: creat d5/f35 x:0 0 0 2024-09-16T00:25:55.536 INFO:tasks.workunit.client.0.smithi112.stdout:5/315: fdatasync d4/d1c/f32 0 2024-09-16T00:25:55.536 INFO:tasks.workunit.client.0.smithi112.stdout:9/487: creat d1/d5/d20/d81/d1a/d88/da8/fb2 x:0 0 0 2024-09-16T00:25:55.537 INFO:tasks.workunit.client.0.smithi112.stdout:3/378: dwrite d2/f19 [0,4194304] 0 2024-09-16T00:25:55.539 INFO:tasks.workunit.client.0.smithi112.stdout:4/250: rename d6/da/dc/l45 to d6/db/df/l56 0 2024-09-16T00:25:55.541 INFO:tasks.workunit.client.0.smithi112.stdout:5/316: mkdir d4/dd/d49/d70 0 2024-09-16T00:25:55.541 INFO:tasks.workunit.client.0.smithi112.stdout:4/251: write d6/f9 [9506330,98571] 0 2024-09-16T00:25:55.544 INFO:tasks.workunit.client.0.smithi112.stdout:5/317: write d4/d1c/d43/f6d [18516,53314] 0 2024-09-16T00:25:55.544 INFO:tasks.workunit.client.0.smithi112.stdout:6/293: getdents dd/d15 0 2024-09-16T00:25:55.546 INFO:tasks.workunit.client.0.smithi112.stdout:3/379: rename d2/d9/d2b/d33 to d2/d9/d2b/d47/d52/d7a/d80/d83 0 2024-09-16T00:25:55.547 INFO:tasks.workunit.client.0.smithi112.stdout:4/252: stat d6/da/dc/l32 0 2024-09-16T00:25:55.549 INFO:tasks.workunit.client.0.smithi112.stdout:5/318: write d4/d7/d26/f5a [984211,27019] 0 2024-09-16T00:25:55.549 INFO:tasks.workunit.client.1.smithi169.stdout:9/137: rmdir d5/da/d21 39 2024-09-16T00:25:55.552 INFO:tasks.workunit.client.0.smithi112.stdout:9/488: dwrite d1/d5/d20/d81/d1a/d88/f7e [0,4194304] 0 2024-09-16T00:25:55.552 INFO:tasks.workunit.client.1.smithi169.stdout:9/138: fsync d5/da/d21/f30 0 2024-09-16T00:25:55.553 INFO:tasks.workunit.client.0.smithi112.stdout:6/294: link dd/df/d10/f1a dd/d23/f63 0 2024-09-16T00:25:55.553 INFO:tasks.workunit.client.1.smithi169.stdout:9/139: stat d5/da/d21/d23/f27 0 2024-09-16T00:25:55.555 INFO:tasks.workunit.client.1.smithi169.stdout:9/140: creat d5/da/d21/d23/f36 x:0 0 0 2024-09-16T00:25:55.557 INFO:tasks.workunit.client.0.smithi112.stdout:9/489: chown d1/d5/d6/l51 14723353 1 2024-09-16T00:25:55.557 INFO:tasks.workunit.client.0.smithi112.stdout:6/295: creat dd/d23/f64 x:0 0 0 2024-09-16T00:25:55.560 INFO:tasks.workunit.client.0.smithi112.stdout:4/253: dwrite d6/d39/d1c/f27 [0,4194304] 0 2024-09-16T00:25:55.563 INFO:tasks.workunit.client.1.smithi169.stdout:9/141: dwrite d5/da/f34 [0,4194304] 0 2024-09-16T00:25:55.564 INFO:tasks.workunit.client.0.smithi112.stdout:9/490: creat d1/d5/d20/d81/d1a/d88/fb3 x:0 0 0 2024-09-16T00:25:55.564 INFO:tasks.workunit.client.1.smithi169.stdout:9/142: readlink d5/da/l20 0 2024-09-16T00:25:55.565 INFO:tasks.workunit.client.0.smithi112.stdout:4/254: creat d6/d39/d1c/f57 x:0 0 0 2024-09-16T00:25:55.569 INFO:tasks.workunit.client.0.smithi112.stdout:9/491: read d1/d5/d6/d38/f4f [621812,13196] 0 2024-09-16T00:25:55.569 INFO:tasks.workunit.client.0.smithi112.stdout:5/319: truncate d4/f16 2336709 0 2024-09-16T00:25:55.570 INFO:tasks.workunit.client.1.smithi169.stdout:9/143: link d5/da/d21/d23/c2f d5/da/d2e/c37 0 2024-09-16T00:25:55.571 INFO:tasks.workunit.client.0.smithi112.stdout:6/296: dwrite dd/df/d10/d47/f5e [0,4194304] 0 2024-09-16T00:25:55.573 INFO:tasks.workunit.client.0.smithi112.stdout:4/255: rename d6/db/d1f/d41/d49/d31/l53 to d6/l58 0 2024-09-16T00:25:55.573 INFO:tasks.workunit.client.1.smithi169.stdout:9/144: symlink d5/da/d2e/l38 0 2024-09-16T00:25:55.574 INFO:tasks.workunit.client.1.smithi169.stdout:9/145: dread - d5/d12/f1c zero size 2024-09-16T00:25:55.575 INFO:tasks.workunit.client.0.smithi112.stdout:6/297: truncate dd/df/d11/f1e 585791 0 2024-09-16T00:25:55.575 INFO:tasks.workunit.client.1.smithi169.stdout:9/146: read d5/f14 [2641300,112151] 0 2024-09-16T00:25:55.576 INFO:tasks.workunit.client.0.smithi112.stdout:4/256: creat d6/db/df/f59 x:0 0 0 2024-09-16T00:25:55.577 INFO:tasks.workunit.client.0.smithi112.stdout:6/298: mknod dd/df/d17/c65 0 2024-09-16T00:25:55.578 INFO:tasks.workunit.client.0.smithi112.stdout:4/257: truncate d6/db/df/f59 539456 0 2024-09-16T00:25:55.579 INFO:tasks.workunit.client.0.smithi112.stdout:9/492: rename d1/d5/d6/d2d/d39/f3c to d1/d5/d6/fb4 0 2024-09-16T00:25:55.580 INFO:tasks.workunit.client.0.smithi112.stdout:4/258: symlink d6/db/d43/l5a 0 2024-09-16T00:25:55.582 INFO:tasks.workunit.client.0.smithi112.stdout:9/493: truncate d1/d5/d20/d81/d1a/f7d 1248294 0 2024-09-16T00:25:55.584 INFO:tasks.workunit.client.1.smithi169.stdout:9/147: dwrite d5/da/d21/f33 [0,4194304] 0 2024-09-16T00:25:55.585 INFO:tasks.workunit.client.0.smithi112.stdout:6/299: rename dd/df/d10/d47/f5e to dd/df/f66 0 2024-09-16T00:25:55.586 INFO:tasks.workunit.client.0.smithi112.stdout:9/494: truncate d1/d5/d6/d18/f28 597513 0 2024-09-16T00:25:55.587 INFO:tasks.workunit.client.0.smithi112.stdout:6/300: creat dd/df/d11/f67 x:0 0 0 2024-09-16T00:25:55.588 INFO:tasks.workunit.client.0.smithi112.stdout:3/380: dread d2/d9/f1e [4194304,4194304] 0 2024-09-16T00:25:55.589 INFO:tasks.workunit.client.1.smithi169.stdout:9/148: chown d5/da/d21/d23/c2f 122252 1 2024-09-16T00:25:55.589 INFO:tasks.workunit.client.0.smithi112.stdout:3/381: read d2/d9/d2b/d47/d52/f40 [3367694,112065] 0 2024-09-16T00:25:55.590 INFO:tasks.workunit.client.0.smithi112.stdout:6/301: chown dd/df/d11/c61 59813 1 2024-09-16T00:25:55.591 INFO:tasks.workunit.client.1.smithi169.stdout:9/149: rmdir d5/da/d21 39 2024-09-16T00:25:55.591 INFO:tasks.workunit.client.1.smithi169.stdout:9/150: chown d5/f14 9112 1 2024-09-16T00:25:55.594 INFO:tasks.workunit.client.0.smithi112.stdout:6/302: symlink dd/df/d11/d44/l68 0 2024-09-16T00:25:55.594 INFO:tasks.workunit.client.1.smithi169.stdout:9/151: truncate d5/d12/f26 1289695 0 2024-09-16T00:25:55.595 INFO:tasks.workunit.client.0.smithi112.stdout:9/495: dwrite d1/d5/d6/d2d/f92 [0,4194304] 0 2024-09-16T00:25:55.595 INFO:tasks.workunit.client.1.smithi169.stdout:9/152: readlink d5/d12/l1d 0 2024-09-16T00:25:55.597 INFO:tasks.workunit.client.0.smithi112.stdout:3/382: dread d2/f1a [4194304,4194304] 0 2024-09-16T00:25:55.598 INFO:tasks.workunit.client.1.smithi169.stdout:9/153: symlink d5/da/d21/d23/l39 0 2024-09-16T00:25:55.598 INFO:tasks.workunit.client.0.smithi112.stdout:3/383: write d2/d9/d2b/d39/f3c [3482824,79045] 0 2024-09-16T00:25:55.600 INFO:tasks.workunit.client.0.smithi112.stdout:6/303: rename dd/df/d11/c61 to dd/df/d11/d44/c69 0 2024-09-16T00:25:55.600 INFO:tasks.workunit.client.0.smithi112.stdout:9/496: mknod d1/d5/d6/cb5 0 2024-09-16T00:25:55.601 INFO:tasks.workunit.client.1.smithi169.stdout:9/154: symlink d5/l3a 0 2024-09-16T00:25:55.601 INFO:tasks.workunit.client.1.smithi169.stdout:9/155: truncate d5/f15 565949 0 2024-09-16T00:25:55.602 INFO:tasks.workunit.client.0.smithi112.stdout:3/384: rename d2/d9 to d2/d9/d84 22 2024-09-16T00:25:55.602 INFO:tasks.workunit.client.0.smithi112.stdout:3/385: chown d2/d4d/l7d 533 1 2024-09-16T00:25:55.603 INFO:tasks.workunit.client.0.smithi112.stdout:6/304: creat dd/df/d17/d27/d54/f6a x:0 0 0 2024-09-16T00:25:55.604 INFO:tasks.workunit.client.1.smithi169.stdout:9/156: creat d5/da/d21/d23/f3b x:0 0 0 2024-09-16T00:25:55.605 INFO:tasks.workunit.client.0.smithi112.stdout:9/497: creat d1/d5/d20/d81/fb6 x:0 0 0 2024-09-16T00:25:55.606 INFO:tasks.workunit.client.1.smithi169.stdout:9/157: chown d5/d12/f13 28005365 1 2024-09-16T00:25:55.622 INFO:tasks.workunit.client.1.smithi169.stdout:9/158: rmdir d5 39 2024-09-16T00:25:55.625 INFO:tasks.workunit.client.1.smithi169.stdout:9/159: symlink d5/da/d21/d23/l3c 0 2024-09-16T00:25:55.630 INFO:tasks.workunit.client.1.smithi169.stdout:9/160: truncate d5/da/d21/f2c 505991 0 2024-09-16T00:25:55.635 INFO:tasks.workunit.client.1.smithi169.stdout:9/161: dwrite d5/da/d21/d23/f27 [0,4194304] 0 2024-09-16T00:25:55.636 INFO:tasks.workunit.client.1.smithi169.stdout:9/162: chown d5/da/d2e/l38 1393 1 2024-09-16T00:25:55.639 INFO:tasks.workunit.client.1.smithi169.stdout:9/163: rename d5/c22 to d5/da/d2e/c3d 0 2024-09-16T00:25:55.641 INFO:tasks.workunit.client.1.smithi169.stdout:9/164: dread - d5/da/f10 zero size 2024-09-16T00:25:55.651 INFO:tasks.workunit.client.1.smithi169.stdout:9/165: truncate d5/da/f34 3972506 0 2024-09-16T00:25:55.660 INFO:tasks.workunit.client.1.smithi169.stdout:9/166: dwrite d5/f14 [0,4194304] 0 2024-09-16T00:25:55.665 INFO:tasks.workunit.client.1.smithi169.stdout:9/167: dwrite d5/da/f2b [0,4194304] 0 2024-09-16T00:25:55.689 INFO:tasks.workunit.client.1.smithi169.stdout:3/103: sync 2024-09-16T00:25:55.689 INFO:tasks.workunit.client.1.smithi169.stdout:2/56: sync 2024-09-16T00:25:55.690 INFO:tasks.workunit.client.1.smithi169.stdout:1/124: sync 2024-09-16T00:25:55.690 INFO:tasks.workunit.client.1.smithi169.stdout:3/104: write d7/f10 [8355339,1269] 0 2024-09-16T00:25:55.691 INFO:tasks.workunit.client.1.smithi169.stdout:3/105: write d7/f1e [4364060,35414] 0 2024-09-16T00:25:55.691 INFO:tasks.workunit.client.1.smithi169.stdout:2/57: symlink d7/lc 0 2024-09-16T00:25:55.691 INFO:tasks.workunit.client.1.smithi169.stdout:1/125: mknod dc/df/c22 0 2024-09-16T00:25:55.692 INFO:tasks.workunit.client.1.smithi169.stdout:1/126: chown dc/l14 1 1 2024-09-16T00:25:55.693 INFO:tasks.workunit.client.1.smithi169.stdout:2/58: rename d7/ca to d7/db/cd 0 2024-09-16T00:25:55.699 INFO:tasks.workunit.client.1.smithi169.stdout:2/59: fsync f6 0 2024-09-16T00:25:55.699 INFO:tasks.workunit.client.1.smithi169.stdout:1/127: dwrite dc/d18/f20 [0,4194304] 0 2024-09-16T00:25:55.700 INFO:tasks.workunit.client.1.smithi169.stdout:1/128: dread - dc/df/f16 zero size 2024-09-16T00:25:55.701 INFO:tasks.workunit.client.1.smithi169.stdout:1/129: chown dc/d18/f20 18427 1 2024-09-16T00:25:55.704 INFO:tasks.workunit.client.1.smithi169.stdout:1/130: link f0 dc/df/f23 0 2024-09-16T00:25:55.706 INFO:tasks.workunit.client.1.smithi169.stdout:2/60: dwrite d7/f8 [0,4194304] 0 2024-09-16T00:25:55.707 INFO:tasks.workunit.client.1.smithi169.stdout:1/131: getdents dc 0 2024-09-16T00:25:55.710 INFO:tasks.workunit.client.1.smithi169.stdout:2/61: rename f6 to d7/fe 0 2024-09-16T00:25:55.710 INFO:tasks.workunit.client.1.smithi169.stdout:2/62: chown d7/db 3709508 1 2024-09-16T00:25:55.711 INFO:tasks.workunit.client.1.smithi169.stdout:1/132: write dc/df/f23 [631959,8274] 0 2024-09-16T00:25:55.712 INFO:tasks.workunit.client.1.smithi169.stdout:2/63: mknod d7/cf 0 2024-09-16T00:25:55.721 INFO:tasks.workunit.client.0.smithi112.stdout:9/498: write d1/d5/d20/d25/f1c [322436,105415] 0 2024-09-16T00:25:55.724 INFO:tasks.workunit.client.0.smithi112.stdout:3/386: dwrite d2/d9/d2b/d2c/f7e [0,4194304] 0 2024-09-16T00:25:55.724 INFO:tasks.workunit.client.0.smithi112.stdout:6/305: dread dd/d15/f29 [0,4194304] 0 2024-09-16T00:25:55.725 INFO:tasks.workunit.client.0.smithi112.stdout:4/259: dread d6/d39/d1c/f2b [0,4194304] 0 2024-09-16T00:25:55.726 INFO:tasks.workunit.client.0.smithi112.stdout:7/317: sync 2024-09-16T00:25:55.730 INFO:tasks.workunit.client.0.smithi112.stdout:6/306: mknod dd/d23/c6b 0 2024-09-16T00:25:55.732 INFO:tasks.workunit.client.0.smithi112.stdout:3/387: creat d2/d9/d2b/d39/f85 x:0 0 0 2024-09-16T00:25:55.735 INFO:tasks.workunit.client.0.smithi112.stdout:1/262: sync 2024-09-16T00:25:55.745 INFO:tasks.workunit.client.0.smithi112.stdout:4/260: symlink d6/d39/l5b 0 2024-09-16T00:25:55.747 INFO:tasks.workunit.client.0.smithi112.stdout:4/261: chown d6/db/d1f/d41/d49/d31/d34/c4e 231599 1 2024-09-16T00:25:55.748 INFO:tasks.workunit.client.0.smithi112.stdout:6/307: link dd/df/d10/l31 dd/df/d11/d44/l6c 0 2024-09-16T00:25:55.748 INFO:tasks.workunit.client.1.smithi169.stdout:2/64: getdents d7/db 0 2024-09-16T00:25:55.749 INFO:tasks.workunit.client.0.smithi112.stdout:4/262: stat d6/d39/d1c/l3b 0 2024-09-16T00:25:55.749 INFO:tasks.workunit.client.1.smithi169.stdout:3/106: truncate d7/f24 4069155 0 2024-09-16T00:25:55.750 INFO:tasks.workunit.client.1.smithi169.stdout:2/65: symlink d7/l10 0 2024-09-16T00:25:55.751 INFO:tasks.workunit.client.0.smithi112.stdout:3/388: dread d2/d9/d2b/d2c/f42 [4194304,4194304] 0 2024-09-16T00:25:55.751 INFO:tasks.workunit.client.0.smithi112.stdout:6/308: mknod dd/df/d13/d18/c6d 0 2024-09-16T00:25:55.752 INFO:tasks.workunit.client.1.smithi169.stdout:2/66: creat d7/f11 x:0 0 0 2024-09-16T00:25:55.752 INFO:tasks.workunit.client.0.smithi112.stdout:7/318: dread d6/f9 [0,4194304] 0 2024-09-16T00:25:55.754 INFO:tasks.workunit.client.1.smithi169.stdout:2/67: write d7/f8 [1442579,32035] 0 2024-09-16T00:25:55.754 INFO:tasks.workunit.client.1.smithi169.stdout:3/107: dread d7/f10 [0,4194304] 0 2024-09-16T00:25:55.755 INFO:tasks.workunit.client.1.smithi169.stdout:2/68: chown d7/f8 1 1 2024-09-16T00:25:55.756 INFO:tasks.workunit.client.0.smithi112.stdout:4/263: dread d6/db/f42 [0,4194304] 0 2024-09-16T00:25:55.757 INFO:tasks.workunit.client.0.smithi112.stdout:7/319: symlink d6/d10/d16/d49/l78 0 2024-09-16T00:25:55.758 INFO:tasks.workunit.client.0.smithi112.stdout:6/309: creat dd/df/f6e x:0 0 0 2024-09-16T00:25:55.758 INFO:tasks.workunit.client.1.smithi169.stdout:2/69: creat d7/f12 x:0 0 0 2024-09-16T00:25:55.761 INFO:tasks.workunit.client.0.smithi112.stdout:7/320: write d6/d10/d16/f3a [4625589,79458] 0 2024-09-16T00:25:55.763 INFO:tasks.workunit.client.1.smithi169.stdout:2/70: rename d7/f8 to d7/db/f13 0 2024-09-16T00:25:55.763 INFO:tasks.workunit.client.1.smithi169.stdout:2/71: stat f0 0 2024-09-16T00:25:55.763 INFO:tasks.workunit.client.0.smithi112.stdout:4/264: mkdir d6/d39/d36/d5c 0 2024-09-16T00:25:55.763 INFO:tasks.workunit.client.0.smithi112.stdout:3/389: dwrite d2/d9/d2b/f82 [0,4194304] 0 2024-09-16T00:25:55.764 INFO:tasks.workunit.client.1.smithi169.stdout:1/133: write dc/df/f12 [680946,74998] 0 2024-09-16T00:25:55.765 INFO:tasks.workunit.client.1.smithi169.stdout:1/134: write dc/df/f12 [343729,7632] 0 2024-09-16T00:25:55.765 INFO:tasks.workunit.client.0.smithi112.stdout:4/265: read d6/d39/d1c/f27 [4061677,51481] 0 2024-09-16T00:25:55.765 INFO:tasks.workunit.client.0.smithi112.stdout:1/263: truncate d0/d14/d1c/d26/f29 7280961 0 2024-09-16T00:25:55.767 INFO:tasks.workunit.client.0.smithi112.stdout:7/321: creat d6/d10/d16/d49/d6c/f79 x:0 0 0 2024-09-16T00:25:55.768 INFO:tasks.workunit.client.0.smithi112.stdout:1/264: chown d0/d14/d1c/d20/f31 1 1 2024-09-16T00:25:55.769 INFO:tasks.workunit.client.0.smithi112.stdout:3/390: creat d2/d9/d2b/d2c/d69/f86 x:0 0 0 2024-09-16T00:25:55.770 INFO:tasks.workunit.client.0.smithi112.stdout:1/265: write d0/d14/df/d28/d45/d30/f3a [752458,64941] 0 2024-09-16T00:25:55.770 INFO:tasks.workunit.client.1.smithi169.stdout:1/135: creat dc/f24 x:0 0 0 2024-09-16T00:25:55.771 INFO:tasks.workunit.client.0.smithi112.stdout:1/266: chown d0/d14/fd 49489155 1 2024-09-16T00:25:55.774 INFO:tasks.workunit.client.0.smithi112.stdout:7/322: unlink d6/d10/d16/d3b/c55 0 2024-09-16T00:25:55.774 INFO:tasks.workunit.client.1.smithi169.stdout:3/108: write d7/fd [3506089,129254] 0 2024-09-16T00:25:55.775 INFO:tasks.workunit.client.0.smithi112.stdout:6/310: getdents dd/df/d10 0 2024-09-16T00:25:55.777 INFO:tasks.workunit.client.0.smithi112.stdout:3/391: symlink d2/d9/d2b/l87 0 2024-09-16T00:25:55.778 INFO:tasks.workunit.client.0.smithi112.stdout:1/267: symlink d0/d14/d1c/d26/d41/l55 0 2024-09-16T00:25:55.778 INFO:tasks.workunit.client.1.smithi169.stdout:3/109: dread d7/f16 [4194304,4194304] 0 2024-09-16T00:25:55.781 INFO:tasks.workunit.client.1.smithi169.stdout:3/110: rename d7/f20 to d7/d1a/d1c/f25 0 2024-09-16T00:25:55.782 INFO:tasks.workunit.client.1.smithi169.stdout:2/72: fsync f5 0 2024-09-16T00:25:55.782 INFO:tasks.workunit.client.1.smithi169.stdout:2/73: readlink d7/lc 0 2024-09-16T00:25:55.783 INFO:tasks.workunit.client.1.smithi169.stdout:3/111: mknod d7/c26 0 2024-09-16T00:25:55.784 INFO:tasks.workunit.client.1.smithi169.stdout:2/74: mknod d7/c14 0 2024-09-16T00:25:55.785 INFO:tasks.workunit.client.1.smithi169.stdout:3/112: fdatasync d7/f16 0 2024-09-16T00:25:55.787 INFO:tasks.workunit.client.0.smithi112.stdout:4/266: truncate d6/da/f35 191888 0 2024-09-16T00:25:55.788 INFO:tasks.workunit.client.0.smithi112.stdout:7/323: dwrite d6/d10/d16/d3b/f4b [0,4194304] 0 2024-09-16T00:25:55.788 INFO:tasks.workunit.client.1.smithi169.stdout:2/75: rename d7/c9 to d7/db/c15 0 2024-09-16T00:25:55.789 INFO:tasks.workunit.client.1.smithi169.stdout:3/113: stat d7/l13 0 2024-09-16T00:25:55.790 INFO:tasks.workunit.client.0.smithi112.stdout:4/267: rename d6 to d6/d39/d36/d5d 22 2024-09-16T00:25:55.791 INFO:tasks.workunit.client.1.smithi169.stdout:1/136: mknod dc/d18/c25 0 2024-09-16T00:25:55.792 INFO:tasks.workunit.client.1.smithi169.stdout:3/114: mknod d7/d1a/d1c/c27 0 2024-09-16T00:25:55.792 INFO:tasks.workunit.client.0.smithi112.stdout:1/268: dwrite d0/d14/df/d28/d45/d30/f3a [0,4194304] 0 2024-09-16T00:25:55.793 INFO:tasks.workunit.client.0.smithi112.stdout:3/392: dwrite d2/d9/d2b/d47/d52/d7a/d80/d83/f50 [0,4194304] 0 2024-09-16T00:25:55.794 INFO:tasks.workunit.client.0.smithi112.stdout:7/324: mknod d6/d10/d16/d49/c7a 0 2024-09-16T00:25:55.794 INFO:tasks.workunit.client.1.smithi169.stdout:1/137: mkdir dc/d26 0 2024-09-16T00:25:55.794 INFO:tasks.workunit.client.0.smithi112.stdout:4/268: mkdir d6/db/df/d5e 0 2024-09-16T00:25:55.795 INFO:tasks.workunit.client.1.smithi169.stdout:1/138: chown fb 106366 1 2024-09-16T00:25:55.798 INFO:tasks.workunit.client.0.smithi112.stdout:4/269: creat d6/db/df/f5f x:0 0 0 2024-09-16T00:25:55.798 INFO:tasks.workunit.client.1.smithi169.stdout:2/76: dwrite f3 [0,4194304] 0 2024-09-16T00:25:55.800 INFO:tasks.workunit.client.1.smithi169.stdout:1/139: dread dc/df/f1c [0,4194304] 0 2024-09-16T00:25:55.801 INFO:tasks.workunit.client.0.smithi112.stdout:4/270: creat d6/db/f60 x:0 0 0 2024-09-16T00:25:55.804 INFO:tasks.workunit.client.1.smithi169.stdout:2/77: symlink d7/l16 0 2024-09-16T00:25:55.804 INFO:tasks.workunit.client.0.smithi112.stdout:3/393: dwrite d2/fb [4194304,4194304] 0 2024-09-16T00:25:55.805 INFO:tasks.workunit.client.1.smithi169.stdout:1/140: dread - dc/f15 zero size 2024-09-16T00:25:55.807 INFO:tasks.workunit.client.0.smithi112.stdout:3/394: chown d2/f3a 57681 1 2024-09-16T00:25:55.807 INFO:tasks.workunit.client.1.smithi169.stdout:2/78: creat d7/db/f17 x:0 0 0 2024-09-16T00:25:55.807 INFO:tasks.workunit.client.1.smithi169.stdout:1/141: write f0 [2725528,10492] 0 2024-09-16T00:25:55.808 INFO:tasks.workunit.client.1.smithi169.stdout:2/79: write d7/f11 [138560,28537] 0 2024-09-16T00:25:55.812 INFO:tasks.workunit.client.1.smithi169.stdout:2/80: rename d7/c14 to d7/db/c18 0 2024-09-16T00:25:55.813 INFO:tasks.workunit.client.1.smithi169.stdout:1/142: mknod dc/d18/c27 0 2024-09-16T00:25:55.813 INFO:tasks.workunit.client.0.smithi112.stdout:3/395: creat d2/d9/d2b/d47/d52/d7a/d80/d83/d68/f88 x:0 0 0 2024-09-16T00:25:55.814 INFO:tasks.workunit.client.1.smithi169.stdout:2/81: truncate d7/fe 357903 0 2024-09-16T00:25:55.814 INFO:tasks.workunit.client.1.smithi169.stdout:1/143: readlink dc/df/l13 0 2024-09-16T00:25:55.816 INFO:tasks.workunit.client.1.smithi169.stdout:1/144: symlink dc/d26/l28 0 2024-09-16T00:25:55.818 INFO:tasks.workunit.client.0.smithi112.stdout:3/396: rename d2/d9/lf to d2/d9/d2b/d47/d52/d7a/d80/d83/d68/l89 0 2024-09-16T00:25:55.819 INFO:tasks.workunit.client.1.smithi169.stdout:1/145: chown dc/f21 3 1 2024-09-16T00:25:55.819 INFO:tasks.workunit.client.0.smithi112.stdout:3/397: read - d2/d9/d2b/d2c/d69/f86 zero size 2024-09-16T00:25:55.821 INFO:tasks.workunit.client.0.smithi112.stdout:7/325: dread d6/d10/f12 [4194304,4194304] 0 2024-09-16T00:25:55.821 INFO:tasks.workunit.client.1.smithi169.stdout:2/82: dwrite f5 [0,4194304] 0 2024-09-16T00:25:55.821 INFO:tasks.workunit.client.0.smithi112.stdout:3/398: creat d2/d9/d2b/d47/f8a x:0 0 0 2024-09-16T00:25:55.823 INFO:tasks.workunit.client.0.smithi112.stdout:7/326: creat d6/f7b x:0 0 0 2024-09-16T00:25:55.823 INFO:tasks.workunit.client.0.smithi112.stdout:3/399: creat d2/d9/d2b/d47/f8b x:0 0 0 2024-09-16T00:25:55.824 INFO:tasks.workunit.client.0.smithi112.stdout:3/400: chown d2/f3a 33 1 2024-09-16T00:25:55.828 INFO:tasks.workunit.client.0.smithi112.stdout:3/401: rename d2/d9/d2b/d39/d66 to d2/d9/d8c 0 2024-09-16T00:25:55.830 INFO:tasks.workunit.client.0.smithi112.stdout:7/327: dwrite d6/d10/f32 [0,4194304] 0 2024-09-16T00:25:55.830 INFO:tasks.workunit.client.0.smithi112.stdout:3/402: read - d2/d9/d2b/d47/d52/d7a/d80/d83/d68/f88 zero size 2024-09-16T00:25:55.838 INFO:tasks.workunit.client.0.smithi112.stdout:3/403: dwrite d2/f24 [0,4194304] 0 2024-09-16T00:25:55.848 INFO:tasks.workunit.client.0.smithi112.stdout:1/269: dread d0/d14/f22 [0,4194304] 0 2024-09-16T00:25:55.851 INFO:tasks.workunit.client.1.smithi169.stdout:3/115: getdents d7 0 2024-09-16T00:25:55.853 INFO:tasks.workunit.client.1.smithi169.stdout:3/116: creat d7/d1a/d1c/d23/f28 x:0 0 0 2024-09-16T00:25:55.854 INFO:tasks.workunit.client.0.smithi112.stdout:4/271: dwrite d6/db/d1f/d41/d49/d31/d34/f3f [0,4194304] 0 2024-09-16T00:25:55.856 INFO:tasks.workunit.client.0.smithi112.stdout:1/270: getdents d0/d14/d1c/d20 0 2024-09-16T00:25:55.857 INFO:tasks.workunit.client.1.smithi169.stdout:1/146: getdents dc/d26 0 2024-09-16T00:25:55.857 INFO:tasks.workunit.client.0.smithi112.stdout:1/271: chown d0/d14/f23 116 1 2024-09-16T00:25:55.859 INFO:tasks.workunit.client.1.smithi169.stdout:1/147: mkdir dc/df/d29 0 2024-09-16T00:25:55.859 INFO:tasks.workunit.client.0.smithi112.stdout:1/272: fdatasync d0/d14/df/d28/d45/d30/f3a 0 2024-09-16T00:25:55.860 INFO:tasks.workunit.client.0.smithi112.stdout:4/272: rmdir d6/db/d1f/d41/d49 39 2024-09-16T00:25:55.860 INFO:tasks.workunit.client.0.smithi112.stdout:4/273: fdatasync d6/f9 0 2024-09-16T00:25:55.862 INFO:tasks.workunit.client.0.smithi112.stdout:1/273: symlink d0/l56 0 2024-09-16T00:25:55.865 INFO:tasks.workunit.client.0.smithi112.stdout:4/274: truncate d6/db/d1f/d41/d49/d31/f51 5161977 0 2024-09-16T00:25:55.867 INFO:tasks.workunit.client.0.smithi112.stdout:1/274: rename d0/d14/f1d to d0/d14/df/d28/d45/d30/f57 0 2024-09-16T00:25:55.869 INFO:tasks.workunit.client.0.smithi112.stdout:7/328: dread d6/fb [0,4194304] 0 2024-09-16T00:25:55.870 INFO:tasks.workunit.client.0.smithi112.stdout:4/275: rename d6/db/d1f/d41/d49/d31/d34 to d6/db/df/d5e/d61 0 2024-09-16T00:25:55.870 INFO:tasks.workunit.client.0.smithi112.stdout:1/275: fsync d0/d14/d1c/d20/f33 0 2024-09-16T00:25:55.871 INFO:tasks.workunit.client.0.smithi112.stdout:4/276: truncate d6/db/df/f59 684663 0 2024-09-16T00:25:55.873 INFO:tasks.workunit.client.0.smithi112.stdout:7/329: mknod d6/d10/c7c 0 2024-09-16T00:25:55.874 INFO:tasks.workunit.client.0.smithi112.stdout:4/277: mknod d6/d39/d1c/d2c/c62 0 2024-09-16T00:25:55.875 INFO:tasks.workunit.client.0.smithi112.stdout:7/330: rmdir d6/d10/d16 39 2024-09-16T00:25:55.878 INFO:tasks.workunit.client.0.smithi112.stdout:1/276: dwrite d0/d14/f11 [4194304,4194304] 0 2024-09-16T00:25:55.879 INFO:tasks.workunit.client.0.smithi112.stdout:7/331: creat d6/d10/d1b/f7d x:0 0 0 2024-09-16T00:25:55.881 INFO:tasks.workunit.client.0.smithi112.stdout:7/332: fsync d6/d10/d1b/f74 0 2024-09-16T00:25:55.884 INFO:tasks.workunit.client.0.smithi112.stdout:4/278: dwrite d6/d39/d1c/f2b [0,4194304] 0 2024-09-16T00:25:55.886 INFO:tasks.workunit.client.0.smithi112.stdout:7/333: truncate d6/d26/f27 1259539 0 2024-09-16T00:25:55.887 INFO:tasks.workunit.client.0.smithi112.stdout:4/279: rmdir d6/da 39 2024-09-16T00:25:55.888 INFO:tasks.workunit.client.0.smithi112.stdout:7/334: chown d6/d10/c50 4482739 1 2024-09-16T00:25:55.889 INFO:tasks.workunit.client.0.smithi112.stdout:4/280: dread d6/db/df/f59 [0,4194304] 0 2024-09-16T00:25:55.895 INFO:tasks.workunit.client.0.smithi112.stdout:4/281: creat d6/d39/d36/d5c/f63 x:0 0 0 2024-09-16T00:25:55.896 INFO:tasks.workunit.client.0.smithi112.stdout:4/282: fsync d6/d39/f4a 0 2024-09-16T00:25:55.897 INFO:tasks.workunit.client.0.smithi112.stdout:4/283: fdatasync d6/db/d1f/d41/d49/f4c 0 2024-09-16T00:25:55.899 INFO:tasks.workunit.client.0.smithi112.stdout:7/335: dread d6/d10/d16/f4e [0,4194304] 0 2024-09-16T00:25:55.900 INFO:tasks.workunit.client.0.smithi112.stdout:4/284: mkdir d6/db/df/d5e/d61/d64 0 2024-09-16T00:25:55.901 INFO:tasks.workunit.client.0.smithi112.stdout:7/336: fdatasync d6/d10/f12 0 2024-09-16T00:25:55.901 INFO:tasks.workunit.client.0.smithi112.stdout:4/285: write d6/d39/d1c/f27 [4756660,86019] 0 2024-09-16T00:25:55.903 INFO:tasks.workunit.client.1.smithi169.stdout:3/117: dwrite d7/f14 [0,4194304] 0 2024-09-16T00:25:55.903 INFO:tasks.workunit.client.1.smithi169.stdout:1/148: dread - dc/f21 zero size 2024-09-16T00:25:55.903 INFO:tasks.workunit.client.0.smithi112.stdout:7/337: mknod d6/d10/d1b/c7e 0 2024-09-16T00:25:55.904 INFO:tasks.workunit.client.0.smithi112.stdout:7/338: write d6/f7b [926864,105772] 0 2024-09-16T00:25:55.905 INFO:tasks.workunit.client.0.smithi112.stdout:7/339: mkdir d6/d7f 0 2024-09-16T00:25:55.906 INFO:tasks.workunit.client.1.smithi169.stdout:3/118: unlink l6 0 2024-09-16T00:25:55.911 INFO:tasks.workunit.client.1.smithi169.stdout:3/119: symlink d7/d1a/d1c/d23/l29 0 2024-09-16T00:25:55.912 INFO:tasks.workunit.client.1.smithi169.stdout:1/149: dwrite f0 [0,4194304] 0 2024-09-16T00:25:55.913 INFO:tasks.workunit.client.1.smithi169.stdout:1/150: stat dc/d18 0 2024-09-16T00:25:55.913 INFO:tasks.workunit.client.1.smithi169.stdout:3/120: rmdir d7/d1a/d1c 39 2024-09-16T00:25:55.914 INFO:tasks.workunit.client.0.smithi112.stdout:4/286: write d6/da/f35 [570338,45668] 0 2024-09-16T00:25:55.914 INFO:tasks.workunit.client.0.smithi112.stdout:7/340: write d6/d26/f27 [1267785,39393] 0 2024-09-16T00:25:55.914 INFO:tasks.workunit.client.1.smithi169.stdout:3/121: write d7/f14 [795317,28773] 0 2024-09-16T00:25:55.915 INFO:tasks.workunit.client.0.smithi112.stdout:7/341: symlink d6/l80 0 2024-09-16T00:25:55.915 INFO:tasks.workunit.client.1.smithi169.stdout:1/151: mkdir dc/df/d29/d2a 0 2024-09-16T00:25:55.916 INFO:tasks.workunit.client.0.smithi112.stdout:4/287: symlink d6/db/df/d5e/d61/l65 0 2024-09-16T00:25:55.916 INFO:tasks.workunit.client.0.smithi112.stdout:7/342: dread - d6/d10/d16/f59 zero size 2024-09-16T00:25:55.917 INFO:tasks.workunit.client.1.smithi169.stdout:1/152: chown dc/df/f12 67282536 1 2024-09-16T00:25:55.918 INFO:tasks.workunit.client.1.smithi169.stdout:1/153: write dc/f1d [1108838,57609] 0 2024-09-16T00:25:55.920 INFO:tasks.workunit.client.1.smithi169.stdout:1/154: creat dc/d18/f2b x:0 0 0 2024-09-16T00:25:55.921 INFO:tasks.workunit.client.0.smithi112.stdout:7/343: rmdir d6/d26/d5c 0 2024-09-16T00:25:55.924 INFO:tasks.workunit.client.0.smithi112.stdout:4/288: dwrite d6/db/f42 [0,4194304] 0 2024-09-16T00:25:55.925 INFO:tasks.workunit.client.1.smithi169.stdout:3/122: getdents d7/d1a/d1c/d23 0 2024-09-16T00:25:55.927 INFO:tasks.workunit.client.0.smithi112.stdout:4/289: mkdir d6/d39/d1c/d66 0 2024-09-16T00:25:55.931 INFO:tasks.workunit.client.0.smithi112.stdout:4/290: link d6/db/df/f59 d6/db/d1f/d41/d49/f67 0 2024-09-16T00:25:55.937 INFO:tasks.workunit.client.0.smithi112.stdout:4/291: dwrite d6/d39/d1c/f27 [0,4194304] 0 2024-09-16T00:25:55.939 INFO:tasks.workunit.client.0.smithi112.stdout:4/292: mknod d6/da/dc/c68 0 2024-09-16T00:25:55.954 INFO:tasks.workunit.client.0.smithi112.stdout:7/344: dread d6/d10/d16/f3e [0,4194304] 0 2024-09-16T00:25:55.955 INFO:tasks.workunit.client.0.smithi112.stdout:7/345: write d6/d10/d1b/f24 [355847,16132] 0 2024-09-16T00:25:55.958 INFO:tasks.workunit.client.0.smithi112.stdout:7/346: creat d6/d10/d1b/f81 x:0 0 0 2024-09-16T00:25:55.959 INFO:tasks.workunit.client.0.smithi112.stdout:7/347: fsync d6/d10/d16/f66 0 2024-09-16T00:25:55.960 INFO:tasks.workunit.client.1.smithi169.stdout:3/123: dread d7/f16 [0,4194304] 0 2024-09-16T00:25:55.960 INFO:tasks.workunit.client.1.smithi169.stdout:3/124: chown d7/d1a/d1c/d23 246069 1 2024-09-16T00:25:55.961 INFO:tasks.workunit.client.0.smithi112.stdout:7/348: unlink d6/fb 0 2024-09-16T00:25:55.962 INFO:tasks.workunit.client.1.smithi169.stdout:3/125: mkdir d7/d1a/d1c/d23/d2a 0 2024-09-16T00:25:55.962 INFO:tasks.workunit.client.0.smithi112.stdout:7/349: unlink d6/d10/d16/d49/d6c/f79 0 2024-09-16T00:25:55.963 INFO:tasks.workunit.client.1.smithi169.stdout:3/126: mkdir d7/d1a/d2b 0 2024-09-16T00:25:55.964 INFO:tasks.workunit.client.0.smithi112.stdout:7/350: getdents d6/d7f 0 2024-09-16T00:25:55.964 INFO:tasks.workunit.client.0.smithi112.stdout:7/351: dread - d6/d10/d16/f59 zero size 2024-09-16T00:25:55.966 INFO:tasks.workunit.client.0.smithi112.stdout:7/352: symlink d6/d10/d16/d49/d6c/l82 0 2024-09-16T00:25:55.966 INFO:tasks.workunit.client.1.smithi169.stdout:3/127: rename d7/d1a/d1c/d23/l29 to d7/d1a/l2c 0 2024-09-16T00:25:55.967 INFO:tasks.workunit.client.0.smithi112.stdout:7/353: symlink d6/d10/d1b/l83 0 2024-09-16T00:25:55.971 INFO:tasks.workunit.client.0.smithi112.stdout:7/354: link d6/d10/l45 d6/d10/l84 0 2024-09-16T00:25:55.971 INFO:tasks.workunit.client.1.smithi169.stdout:3/128: truncate d7/f14 311774 0 2024-09-16T00:25:55.971 INFO:tasks.workunit.client.0.smithi112.stdout:7/355: fsync d6/d10/f76 0 2024-09-16T00:25:55.972 INFO:tasks.workunit.client.1.smithi169.stdout:3/129: chown d7/l13 2112536245 1 2024-09-16T00:25:55.972 INFO:tasks.workunit.client.1.smithi169.stdout:3/130: fsync d7/d1a/d1c/d23/f28 0 2024-09-16T00:25:55.973 INFO:tasks.workunit.client.0.smithi112.stdout:7/356: mkdir d6/d7f/d85 0 2024-09-16T00:25:55.974 INFO:tasks.workunit.client.0.smithi112.stdout:7/357: symlink d6/d26/l86 0 2024-09-16T00:25:55.977 INFO:tasks.workunit.client.1.smithi169.stdout:3/131: truncate d7/f10 2696264 0 2024-09-16T00:25:55.979 INFO:tasks.workunit.client.1.smithi169.stdout:3/132: symlink d7/l2d 0 2024-09-16T00:25:55.979 INFO:tasks.workunit.client.0.smithi112.stdout:7/358: dwrite d6/d10/d16/f59 [0,4194304] 0 2024-09-16T00:25:55.980 INFO:tasks.workunit.client.1.smithi169.stdout:3/133: write d7/f16 [5956577,21533] 0 2024-09-16T00:25:55.983 INFO:tasks.workunit.client.1.smithi169.stdout:3/134: getdents d7/d1a/d1c 0 2024-09-16T00:25:55.983 INFO:tasks.workunit.client.0.smithi112.stdout:7/359: creat d6/d10/f87 x:0 0 0 2024-09-16T00:25:55.984 INFO:tasks.workunit.client.1.smithi169.stdout:3/135: symlink d7/d1a/d1c/l2e 0 2024-09-16T00:25:55.984 INFO:tasks.workunit.client.0.smithi112.stdout:7/360: write d6/d10/d16/f1e [1347043,75872] 0 2024-09-16T00:25:55.987 INFO:tasks.workunit.client.0.smithi112.stdout:7/361: write d6/d10/d16/f3e [4419278,38457] 0 2024-09-16T00:25:55.987 INFO:tasks.workunit.client.0.smithi112.stdout:7/362: dread - d6/f23 zero size 2024-09-16T00:25:55.989 INFO:tasks.workunit.client.0.smithi112.stdout:7/363: fsync d6/d10/d16/d49/d6c/f5d 0 2024-09-16T00:25:55.989 INFO:tasks.workunit.client.0.smithi112.stdout:7/364: readlink d6/d10/d16/l2f 0 2024-09-16T00:25:55.992 INFO:tasks.workunit.client.0.smithi112.stdout:7/365: rename d6/d26 to d6/d10/d75/d88 0 2024-09-16T00:25:55.994 INFO:tasks.workunit.client.0.smithi112.stdout:7/366: unlink d6/fa 0 2024-09-16T00:25:56.000 INFO:tasks.workunit.client.1.smithi169.stdout:3/136: dwrite d7/f14 [0,4194304] 0 2024-09-16T00:25:56.002 INFO:tasks.workunit.client.0.smithi112.stdout:7/367: getdents d6 0 2024-09-16T00:25:56.003 INFO:tasks.workunit.client.0.smithi112.stdout:7/368: creat d6/d7f/f89 x:0 0 0 2024-09-16T00:25:56.004 INFO:tasks.workunit.client.0.smithi112.stdout:7/369: chown d6/d10/d16/c51 4766168 1 2024-09-16T00:25:56.004 INFO:tasks.workunit.client.0.smithi112.stdout:7/370: readlink d6/d10/d16/l21 0 2024-09-16T00:25:56.005 INFO:tasks.workunit.client.0.smithi112.stdout:7/371: stat d6/d10/d16/f3e 0 2024-09-16T00:25:56.009 INFO:tasks.workunit.client.0.smithi112.stdout:7/372: getdents d6/d7f 0 2024-09-16T00:25:56.009 INFO:tasks.workunit.client.0.smithi112.stdout:7/373: fdatasync d6/d10/d1b/f60 0 2024-09-16T00:25:56.010 INFO:tasks.workunit.client.0.smithi112.stdout:7/374: stat d6/d10/d16/d49/d6c/f62 0 2024-09-16T00:25:56.012 INFO:tasks.workunit.client.0.smithi112.stdout:7/375: link d6/d10/d1b/c5f d6/d10/d16/c8a 0 2024-09-16T00:25:56.012 INFO:tasks.workunit.client.0.smithi112.stdout:7/376: readlink d6/l80 0 2024-09-16T00:25:56.013 INFO:tasks.workunit.client.1.smithi169.stdout:8/114: sync 2024-09-16T00:25:56.013 INFO:tasks.workunit.client.1.smithi169.stdout:0/111: sync 2024-09-16T00:25:56.013 INFO:tasks.workunit.client.1.smithi169.stdout:7/109: sync 2024-09-16T00:25:56.013 INFO:tasks.workunit.client.1.smithi169.stdout:4/119: sync 2024-09-16T00:25:56.013 INFO:tasks.workunit.client.1.smithi169.stdout:5/136: sync 2024-09-16T00:25:56.013 INFO:tasks.workunit.client.1.smithi169.stdout:6/64: sync 2024-09-16T00:25:56.013 INFO:tasks.workunit.client.1.smithi169.stdout:9/168: sync 2024-09-16T00:25:56.013 INFO:tasks.workunit.client.0.smithi112.stdout:7/377: write d6/d10/f1d [3957756,28220] 0 2024-09-16T00:25:56.014 INFO:tasks.workunit.client.1.smithi169.stdout:0/112: read - d0/d2/d8/d5/f24 zero size 2024-09-16T00:25:56.015 INFO:tasks.workunit.client.0.smithi112.stdout:7/378: symlink d6/l8b 0 2024-09-16T00:25:56.015 INFO:tasks.workunit.client.1.smithi169.stdout:5/137: truncate d2/f2a 827227 0 2024-09-16T00:25:56.015 INFO:tasks.workunit.client.1.smithi169.stdout:9/169: chown d5/da/d21/d23/f2d 30829 1 2024-09-16T00:25:56.015 INFO:tasks.workunit.client.1.smithi169.stdout:8/115: creat d7/dc/d17/f28 x:0 0 0 2024-09-16T00:25:56.016 INFO:tasks.workunit.client.1.smithi169.stdout:0/113: readlink d0/l11 0 2024-09-16T00:25:56.018 INFO:tasks.workunit.client.1.smithi169.stdout:3/137: fdatasync d7/f14 0 2024-09-16T00:25:56.019 INFO:tasks.workunit.client.1.smithi169.stdout:7/110: truncate d7/da/fc 7052185 0 2024-09-16T00:25:56.020 INFO:tasks.workunit.client.1.smithi169.stdout:7/111: chown d7 120782787 1 2024-09-16T00:25:56.020 INFO:tasks.workunit.client.1.smithi169.stdout:8/116: write d7/dc/d17/f1b [852196,80964] 0 2024-09-16T00:25:56.021 INFO:tasks.workunit.client.1.smithi169.stdout:3/138: write d7/f14 [607017,64193] 0 2024-09-16T00:25:56.021 INFO:tasks.workunit.client.1.smithi169.stdout:5/138: creat d2/da/d10/d13/f2f x:0 0 0 2024-09-16T00:25:56.022 INFO:tasks.workunit.client.1.smithi169.stdout:9/170: write d5/da/f16 [1000543,104395] 0 2024-09-16T00:25:56.024 INFO:tasks.workunit.client.1.smithi169.stdout:6/65: getdents d2 0 2024-09-16T00:25:56.024 INFO:tasks.workunit.client.1.smithi169.stdout:4/120: dread d2/f4 [0,4194304] 0 2024-09-16T00:25:56.026 INFO:tasks.workunit.client.1.smithi169.stdout:0/114: creat d0/d2/d15/f27 x:0 0 0 2024-09-16T00:25:56.026 INFO:tasks.workunit.client.1.smithi169.stdout:7/112: creat d7/df/f1c x:0 0 0 2024-09-16T00:25:56.026 INFO:tasks.workunit.client.1.smithi169.stdout:3/139: read - d7/d1a/d1c/f25 zero size 2024-09-16T00:25:56.027 INFO:tasks.workunit.client.1.smithi169.stdout:6/66: write d2/fb [4329720,50529] 0 2024-09-16T00:25:56.027 INFO:tasks.workunit.client.1.smithi169.stdout:5/139: read d2/ff [4073769,111078] 0 2024-09-16T00:25:56.029 INFO:tasks.workunit.client.1.smithi169.stdout:8/117: dread d7/d20/f21 [0,4194304] 0 2024-09-16T00:25:56.030 INFO:tasks.workunit.client.1.smithi169.stdout:4/121: write d2/dd/f24 [886887,115206] 0 2024-09-16T00:25:56.030 INFO:tasks.workunit.client.1.smithi169.stdout:7/113: truncate d7/f17 3055515 0 2024-09-16T00:25:56.030 INFO:tasks.workunit.client.1.smithi169.stdout:6/67: fdatasync d2/fb 0 2024-09-16T00:25:56.032 INFO:tasks.workunit.client.1.smithi169.stdout:6/68: readlink d2/l6 0 2024-09-16T00:25:56.033 INFO:tasks.workunit.client.1.smithi169.stdout:5/140: fdatasync d2/da/d10/f11 0 2024-09-16T00:25:56.034 INFO:tasks.workunit.client.1.smithi169.stdout:8/118: dread d7/d20/f21 [0,4194304] 0 2024-09-16T00:25:56.034 INFO:tasks.workunit.client.1.smithi169.stdout:0/115: creat d0/d2/d8/d5/d14/f28 x:0 0 0 2024-09-16T00:25:56.035 INFO:tasks.workunit.client.1.smithi169.stdout:9/171: write d5/da/d21/f2c [1132736,88456] 0 2024-09-16T00:25:56.036 INFO:tasks.workunit.client.1.smithi169.stdout:8/119: readlink d7/l1e 0 2024-09-16T00:25:56.036 INFO:tasks.workunit.client.1.smithi169.stdout:6/69: write d2/fb [2946932,72130] 0 2024-09-16T00:25:56.037 INFO:tasks.workunit.client.1.smithi169.stdout:4/122: rmdir d2/dd/d1a 39 2024-09-16T00:25:56.038 INFO:tasks.workunit.client.0.smithi112.stdout:7/379: dread d6/d10/d1b/f24 [0,4194304] 0 2024-09-16T00:25:56.040 INFO:tasks.workunit.client.1.smithi169.stdout:3/140: link d7/f16 d7/d1a/d1c/d23/f2f 0 2024-09-16T00:25:56.040 INFO:tasks.workunit.client.1.smithi169.stdout:0/116: read d0/d2/d8/d5/fa [2992085,10784] 0 2024-09-16T00:25:56.040 INFO:tasks.workunit.client.0.smithi112.stdout:7/380: symlink d6/d10/d16/d49/d6c/l8c 0 2024-09-16T00:25:56.041 INFO:tasks.workunit.client.1.smithi169.stdout:6/70: fdatasync d2/fb 0 2024-09-16T00:25:56.042 INFO:tasks.workunit.client.1.smithi169.stdout:5/141: mkdir d2/d30 0 2024-09-16T00:25:56.042 INFO:tasks.workunit.client.0.smithi112.stdout:7/381: creat d6/d7f/f8d x:0 0 0 2024-09-16T00:25:56.042 INFO:tasks.workunit.client.0.smithi112.stdout:7/382: chown d6/f23 977186 1 2024-09-16T00:25:56.043 INFO:tasks.workunit.client.1.smithi169.stdout:1/155: sync 2024-09-16T00:25:56.043 INFO:tasks.workunit.client.1.smithi169.stdout:2/83: sync 2024-09-16T00:25:56.043 INFO:tasks.workunit.client.0.smithi112.stdout:7/383: symlink d6/d10/d75/l8e 0 2024-09-16T00:25:56.045 INFO:tasks.workunit.client.1.smithi169.stdout:4/123: mkdir d2/d12/d13/d27 0 2024-09-16T00:25:56.046 INFO:tasks.workunit.client.1.smithi169.stdout:1/156: readlink dc/df/l1a 0 2024-09-16T00:25:56.047 INFO:tasks.workunit.client.1.smithi169.stdout:7/114: dwrite d7/df/f11 [0,4194304] 0 2024-09-16T00:25:56.048 INFO:tasks.workunit.client.1.smithi169.stdout:6/71: symlink d2/l16 0 2024-09-16T00:25:56.050 INFO:tasks.workunit.client.1.smithi169.stdout:9/172: dread d5/da/ff [0,4194304] 0 2024-09-16T00:25:56.050 INFO:tasks.workunit.client.1.smithi169.stdout:2/84: creat d7/db/f19 x:0 0 0 2024-09-16T00:25:56.050 INFO:tasks.workunit.client.1.smithi169.stdout:8/120: dread d7/fd [4194304,4194304] 0 2024-09-16T00:25:56.052 INFO:tasks.workunit.client.1.smithi169.stdout:1/157: unlink dc/d18/l19 0 2024-09-16T00:25:56.052 INFO:tasks.workunit.client.1.smithi169.stdout:1/158: fdatasync dc/f1d 0 2024-09-16T00:25:56.054 INFO:tasks.workunit.client.1.smithi169.stdout:4/124: symlink d2/d12/l28 0 2024-09-16T00:25:56.055 INFO:tasks.workunit.client.1.smithi169.stdout:7/115: read d7/f8 [413227,84018] 0 2024-09-16T00:25:56.055 INFO:tasks.workunit.client.1.smithi169.stdout:3/141: link d7/f14 d7/d1a/f30 0 2024-09-16T00:25:56.057 INFO:tasks.workunit.client.1.smithi169.stdout:0/117: mknod d0/d2/d8/d5/d14/d21/c29 0 2024-09-16T00:25:56.057 INFO:tasks.workunit.client.1.smithi169.stdout:7/116: chown d7/d16 1001564 1 2024-09-16T00:25:56.057 INFO:tasks.workunit.client.1.smithi169.stdout:2/85: creat d7/f1a x:0 0 0 2024-09-16T00:25:56.061 INFO:tasks.workunit.client.1.smithi169.stdout:4/125: rmdir d2/dd 39 2024-09-16T00:25:56.062 INFO:tasks.workunit.client.1.smithi169.stdout:9/173: dread d5/d12/f26 [0,4194304] 0 2024-09-16T00:25:56.062 INFO:tasks.workunit.client.1.smithi169.stdout:3/142: fsync d7/f16 0 2024-09-16T00:25:56.063 INFO:tasks.workunit.client.1.smithi169.stdout:1/159: mkdir dc/d2c 0 2024-09-16T00:25:56.063 INFO:tasks.workunit.client.1.smithi169.stdout:2/86: fdatasync f0 0 2024-09-16T00:25:56.064 INFO:tasks.workunit.client.1.smithi169.stdout:7/117: creat d7/d16/f1d x:0 0 0 2024-09-16T00:25:56.065 INFO:tasks.workunit.client.1.smithi169.stdout:5/142: dwrite d2/fd [0,4194304] 0 2024-09-16T00:25:56.067 INFO:tasks.workunit.client.1.smithi169.stdout:8/121: symlink d7/dc/d26/l29 0 2024-09-16T00:25:56.072 INFO:tasks.workunit.client.1.smithi169.stdout:9/174: write d5/d12/f13 [257602,12297] 0 2024-09-16T00:25:56.072 INFO:tasks.workunit.client.1.smithi169.stdout:4/126: mknod d2/c29 0 2024-09-16T00:25:56.073 INFO:tasks.workunit.client.1.smithi169.stdout:3/143: dread d7/f10 [0,4194304] 0 2024-09-16T00:25:56.073 INFO:tasks.workunit.client.1.smithi169.stdout:1/160: unlink dc/d18/c25 0 2024-09-16T00:25:56.074 INFO:tasks.workunit.client.1.smithi169.stdout:1/161: truncate fb 4694272 0 2024-09-16T00:25:56.075 INFO:tasks.workunit.client.1.smithi169.stdout:1/162: chown dc/d18/c27 8 1 2024-09-16T00:25:56.076 INFO:tasks.workunit.client.1.smithi169.stdout:5/143: mkdir d2/da/d10/d13/d31 0 2024-09-16T00:25:56.076 INFO:tasks.workunit.client.1.smithi169.stdout:1/163: fsync dc/f21 0 2024-09-16T00:25:56.077 INFO:tasks.workunit.client.1.smithi169.stdout:9/175: dread d5/da/f16 [0,4194304] 0 2024-09-16T00:25:56.077 INFO:tasks.workunit.client.1.smithi169.stdout:7/118: mknod d7/df/c1e 0 2024-09-16T00:25:56.080 INFO:tasks.workunit.client.1.smithi169.stdout:4/127: symlink d2/d12/d13/l2a 0 2024-09-16T00:25:56.084 INFO:tasks.workunit.client.1.smithi169.stdout:8/122: mkdir d7/dc/d2a 0 2024-09-16T00:25:56.085 INFO:tasks.workunit.client.0.smithi112.stdout:8/240: sync 2024-09-16T00:25:56.086 INFO:tasks.workunit.client.1.smithi169.stdout:1/164: dread dc/df/f23 [0,4194304] 0 2024-09-16T00:25:56.086 INFO:tasks.workunit.client.1.smithi169.stdout:5/144: dread d2/da/d10/f27 [0,4194304] 0 2024-09-16T00:25:56.086 INFO:tasks.workunit.client.1.smithi169.stdout:2/87: link d7/cf d7/c1b 0 2024-09-16T00:25:56.090 INFO:tasks.workunit.client.1.smithi169.stdout:7/119: dwrite f6 [0,4194304] 0 2024-09-16T00:25:56.090 INFO:tasks.workunit.client.1.smithi169.stdout:1/165: write dc/f15 [382443,111886] 0 2024-09-16T00:25:56.093 INFO:tasks.workunit.client.1.smithi169.stdout:1/166: dread - dc/d18/f2b zero size 2024-09-16T00:25:56.094 INFO:tasks.workunit.client.1.smithi169.stdout:0/118: dwrite d0/d2/d8/d5/fa [0,4194304] 0 2024-09-16T00:25:56.095 INFO:tasks.workunit.client.1.smithi169.stdout:3/144: link d7/c18 d7/d1a/d1c/c31 0 2024-09-16T00:25:56.096 INFO:tasks.workunit.client.1.smithi169.stdout:2/88: mkdir d7/d1c 0 2024-09-16T00:25:56.097 INFO:tasks.workunit.client.1.smithi169.stdout:9/176: creat d5/da/d21/f3e x:0 0 0 2024-09-16T00:25:56.098 INFO:tasks.workunit.client.0.smithi112.stdout:8/241: dread d10/d12/f16 [0,4194304] 0 2024-09-16T00:25:56.100 INFO:tasks.workunit.client.0.smithi112.stdout:8/242: symlink d10/l4f 0 2024-09-16T00:25:56.107 INFO:tasks.workunit.client.1.smithi169.stdout:2/89: mknod d7/db/c1d 0 2024-09-16T00:25:56.111 INFO:tasks.workunit.client.1.smithi169.stdout:8/123: dwrite d7/dc/f14 [0,4194304] 0 2024-09-16T00:25:56.111 INFO:tasks.workunit.client.1.smithi169.stdout:5/145: truncate d2/f29 48865 0 2024-09-16T00:25:56.112 INFO:tasks.workunit.client.1.smithi169.stdout:7/120: dread d7/f17 [0,4194304] 0 2024-09-16T00:25:56.112 INFO:tasks.workunit.client.1.smithi169.stdout:1/167: dread f0 [0,4194304] 0 2024-09-16T00:25:56.114 INFO:tasks.workunit.client.1.smithi169.stdout:0/119: creat d0/d2/d8/d5/d14/d21/f2a x:0 0 0 2024-09-16T00:25:56.114 INFO:tasks.workunit.client.0.smithi112.stdout:8/243: dwrite d10/f1b [0,4194304] 0 2024-09-16T00:25:56.115 INFO:tasks.workunit.client.1.smithi169.stdout:4/128: link d2/dd/lf d2/dd/d1a/l2b 0 2024-09-16T00:25:56.116 INFO:tasks.workunit.client.1.smithi169.stdout:2/90: unlink d7/f12 0 2024-09-16T00:25:56.116 INFO:tasks.workunit.client.0.smithi112.stdout:8/244: mknod d10/d4e/c50 0 2024-09-16T00:25:56.117 INFO:tasks.workunit.client.1.smithi169.stdout:3/145: dwrite d7/d1a/d1c/d23/f28 [0,4194304] 0 2024-09-16T00:25:56.119 INFO:tasks.workunit.client.0.smithi112.stdout:8/245: symlink d10/d45/l51 0 2024-09-16T00:25:56.120 INFO:tasks.workunit.client.0.smithi112.stdout:8/246: write d10/f1b [5135395,17794] 0 2024-09-16T00:25:56.121 INFO:tasks.workunit.client.1.smithi169.stdout:2/91: dread - d7/f1a zero size 2024-09-16T00:25:56.123 INFO:tasks.workunit.client.0.smithi112.stdout:8/247: creat d10/d45/f52 x:0 0 0 2024-09-16T00:25:56.126 INFO:tasks.workunit.client.1.smithi169.stdout:7/121: symlink d7/da/l1f 0 2024-09-16T00:25:56.126 INFO:tasks.workunit.client.1.smithi169.stdout:8/124: dread d7/dc/d17/f1b [0,4194304] 0 2024-09-16T00:25:56.128 INFO:tasks.workunit.client.0.smithi112.stdout:2/327: sync 2024-09-16T00:25:56.128 INFO:tasks.workunit.client.1.smithi169.stdout:1/168: symlink dc/df/l2d 0 2024-09-16T00:25:56.128 INFO:tasks.workunit.client.1.smithi169.stdout:9/177: rename d5/d12/c1b to d5/d12/c3f 0 2024-09-16T00:25:56.128 INFO:tasks.workunit.client.1.smithi169.stdout:5/146: mkdir d2/da/d10/d13/d1c/d32 0 2024-09-16T00:25:56.130 INFO:tasks.workunit.client.1.smithi169.stdout:9/178: readlink d5/da/d21/d23/l39 0 2024-09-16T00:25:56.130 INFO:tasks.workunit.client.1.smithi169.stdout:0/120: creat d0/d2/d8/d5/d14/f2b x:0 0 0 2024-09-16T00:25:56.133 INFO:tasks.workunit.client.1.smithi169.stdout:7/122: creat d7/da/f20 x:0 0 0 2024-09-16T00:25:56.133 INFO:tasks.workunit.client.1.smithi169.stdout:2/92: symlink d7/db/l1e 0 2024-09-16T00:25:56.134 INFO:tasks.workunit.client.1.smithi169.stdout:5/147: symlink d2/da/l33 0 2024-09-16T00:25:56.134 INFO:tasks.workunit.client.0.smithi112.stdout:0/290: sync 2024-09-16T00:25:56.135 INFO:tasks.workunit.client.1.smithi169.stdout:4/129: dwrite d2/f10 [4194304,4194304] 0 2024-09-16T00:25:56.135 INFO:tasks.workunit.client.1.smithi169.stdout:8/125: truncate d7/dc/d17/f28 510637 0 2024-09-16T00:25:56.135 INFO:tasks.workunit.client.1.smithi169.stdout:2/93: read f0 [222824,103748] 0 2024-09-16T00:25:56.137 INFO:tasks.workunit.client.1.smithi169.stdout:0/121: write d0/d2/d8/d5/d14/f28 [189095,1462] 0 2024-09-16T00:25:56.138 INFO:tasks.workunit.client.1.smithi169.stdout:3/146: rename d7/f11 to d7/d1a/f32 0 2024-09-16T00:25:56.138 INFO:tasks.workunit.client.1.smithi169.stdout:4/130: chown c1 8151635 1 2024-09-16T00:25:56.139 INFO:tasks.workunit.client.1.smithi169.stdout:7/123: creat d7/df/f21 x:0 0 0 2024-09-16T00:25:56.141 INFO:tasks.workunit.client.1.smithi169.stdout:9/179: rename d5/da/d21/d23/f36 to d5/da/d21/d23/f40 0 2024-09-16T00:25:56.141 INFO:tasks.workunit.client.1.smithi169.stdout:8/126: write d7/f8 [267831,72833] 0 2024-09-16T00:25:56.141 INFO:tasks.workunit.client.0.smithi112.stdout:0/291: dwrite d5/da/f2b [0,4194304] 0 2024-09-16T00:25:56.143 INFO:tasks.workunit.client.1.smithi169.stdout:5/148: read d2/da/d10/d13/d24/f2c [3401995,31792] 0 2024-09-16T00:25:56.144 INFO:tasks.workunit.client.1.smithi169.stdout:0/122: mkdir d0/d2c 0 2024-09-16T00:25:56.145 INFO:tasks.workunit.client.1.smithi169.stdout:7/124: dread - d7/d16/f1d zero size 2024-09-16T00:25:56.145 INFO:tasks.workunit.client.1.smithi169.stdout:5/149: fdatasync d2/ff 0 2024-09-16T00:25:56.145 INFO:tasks.workunit.client.1.smithi169.stdout:4/131: dread - d2/d12/d13/d1f/f20 zero size 2024-09-16T00:25:56.146 INFO:tasks.workunit.client.1.smithi169.stdout:9/180: creat d5/da/d21/d23/f41 x:0 0 0 2024-09-16T00:25:56.148 INFO:tasks.workunit.client.0.smithi112.stdout:8/248: getdents d10/d4e 0 2024-09-16T00:25:56.148 INFO:tasks.workunit.client.1.smithi169.stdout:2/94: mknod d7/d1c/c1f 0 2024-09-16T00:25:56.149 INFO:tasks.workunit.client.0.smithi112.stdout:8/249: stat d10/c30 0 2024-09-16T00:25:56.150 INFO:tasks.workunit.client.1.smithi169.stdout:8/127: write f6 [7037511,55875] 0 2024-09-16T00:25:56.151 INFO:tasks.workunit.client.1.smithi169.stdout:0/123: write d0/d2/f6 [2705232,82944] 0 2024-09-16T00:25:56.152 INFO:tasks.workunit.client.0.smithi112.stdout:0/292: dwrite f0 [0,4194304] 0 2024-09-16T00:25:56.152 INFO:tasks.workunit.client.1.smithi169.stdout:2/95: mkdir d7/db/d20 0 2024-09-16T00:25:56.153 INFO:tasks.workunit.client.1.smithi169.stdout:0/124: readlink d0/d2/lb 0 2024-09-16T00:25:56.153 INFO:tasks.workunit.client.1.smithi169.stdout:1/169: dwrite dc/df/f23 [0,4194304] 0 2024-09-16T00:25:56.154 INFO:tasks.workunit.client.0.smithi112.stdout:0/293: write d5/f23 [1967276,40876] 0 2024-09-16T00:25:56.155 INFO:tasks.workunit.client.1.smithi169.stdout:8/128: chown d7/dc/d2a 3 1 2024-09-16T00:25:56.157 INFO:tasks.workunit.client.0.smithi112.stdout:2/328: dwrite d2/dd/d48/d22/f29 [0,4194304] 0 2024-09-16T00:25:56.158 INFO:tasks.workunit.client.1.smithi169.stdout:5/150: mkdir d2/da/d10/d13/d1c/d32/d34 0 2024-09-16T00:25:56.160 INFO:tasks.workunit.client.0.smithi112.stdout:0/294: read d5/da/dc/d12/f3c [606495,78917] 0 2024-09-16T00:25:56.160 INFO:tasks.workunit.client.1.smithi169.stdout:0/125: dread d0/d2/d8/d5/fa [0,4194304] 0 2024-09-16T00:25:56.161 INFO:tasks.workunit.client.0.smithi112.stdout:2/329: mknod d2/dd/d48/d22/d2b/d54/c65 0 2024-09-16T00:25:56.163 INFO:tasks.workunit.client.0.smithi112.stdout:0/295: creat d5/da/dc/f5b x:0 0 0 2024-09-16T00:25:56.164 INFO:tasks.workunit.client.1.smithi169.stdout:2/96: mkdir d7/db/d20/d21 0 2024-09-16T00:25:56.166 INFO:tasks.workunit.client.1.smithi169.stdout:1/170: mkdir dc/df/d29/d2a/d2e 0 2024-09-16T00:25:56.167 INFO:tasks.workunit.client.1.smithi169.stdout:1/171: chown dc/df/f23 18410429 1 2024-09-16T00:25:56.168 INFO:tasks.workunit.client.0.smithi112.stdout:2/330: creat d2/dd/d48/d22/d2b/d3b/f66 x:0 0 0 2024-09-16T00:25:56.169 INFO:tasks.workunit.client.1.smithi169.stdout:7/125: getdents d7/d16 0 2024-09-16T00:25:56.169 INFO:tasks.workunit.client.0.smithi112.stdout:0/296: getdents d5/da/d34/d48 0 2024-09-16T00:25:56.170 INFO:tasks.workunit.client.0.smithi112.stdout:0/297: write d5/ff [965734,102570] 0 2024-09-16T00:25:56.171 INFO:tasks.workunit.client.1.smithi169.stdout:0/126: symlink d0/d2/l2d 0 2024-09-16T00:25:56.171 INFO:tasks.workunit.client.0.smithi112.stdout:2/331: mkdir d2/dd/d48/d22/d67 0 2024-09-16T00:25:56.171 INFO:tasks.workunit.client.1.smithi169.stdout:1/172: mknod dc/df/c2f 0 2024-09-16T00:25:56.172 INFO:tasks.workunit.client.1.smithi169.stdout:7/126: truncate d7/da/f20 736649 0 2024-09-16T00:25:56.174 INFO:tasks.workunit.client.1.smithi169.stdout:2/97: creat d7/db/d20/d21/f22 x:0 0 0 2024-09-16T00:25:56.175 INFO:tasks.workunit.client.1.smithi169.stdout:3/147: dwrite d7/d1a/f30 [0,4194304] 0 2024-09-16T00:25:56.175 INFO:tasks.workunit.client.0.smithi112.stdout:2/332: getdents d2/dd/d48 0 2024-09-16T00:25:56.176 INFO:tasks.workunit.client.1.smithi169.stdout:2/98: stat d7/db/f17 0 2024-09-16T00:25:56.176 INFO:tasks.workunit.client.0.smithi112.stdout:2/333: write d2/dd/d48/f2f [1039495,100601] 0 2024-09-16T00:25:56.176 INFO:tasks.workunit.client.1.smithi169.stdout:1/173: unlink f0 0 2024-09-16T00:25:56.178 INFO:tasks.workunit.client.1.smithi169.stdout:2/99: fdatasync d7/f1a 0 2024-09-16T00:25:56.179 INFO:tasks.workunit.client.1.smithi169.stdout:0/127: creat d0/d2/d8/d5/f2e x:0 0 0 2024-09-16T00:25:56.179 INFO:tasks.workunit.client.1.smithi169.stdout:1/174: readlink dc/d26/l28 0 2024-09-16T00:25:56.179 INFO:tasks.workunit.client.1.smithi169.stdout:7/127: symlink d7/d16/l22 0 2024-09-16T00:25:56.181 INFO:tasks.workunit.client.0.smithi112.stdout:2/334: dwrite d2/f19 [0,4194304] 0 2024-09-16T00:25:56.183 INFO:tasks.workunit.client.1.smithi169.stdout:2/100: read d7/db/f13 [36717,112479] 0 2024-09-16T00:25:56.183 INFO:tasks.workunit.client.1.smithi169.stdout:2/101: write f3 [156249,105041] 0 2024-09-16T00:25:56.184 INFO:tasks.workunit.client.1.smithi169.stdout:0/128: chown d0/l11 72977214 1 2024-09-16T00:25:56.184 INFO:tasks.workunit.client.0.smithi112.stdout:2/335: write d2/dd/d48/d22/d2b/d3b/d49/f5c [166704,67065] 0 2024-09-16T00:25:56.185 INFO:tasks.workunit.client.1.smithi169.stdout:0/129: dread - d0/f23 zero size 2024-09-16T00:25:56.187 INFO:tasks.workunit.client.1.smithi169.stdout:0/130: fsync d0/d2/d15/f27 0 2024-09-16T00:25:56.187 INFO:tasks.workunit.client.1.smithi169.stdout:3/148: dread d7/f16 [0,4194304] 0 2024-09-16T00:25:56.187 INFO:tasks.workunit.client.1.smithi169.stdout:1/175: mknod dc/df/c30 0 2024-09-16T00:25:56.189 INFO:tasks.workunit.client.0.smithi112.stdout:2/336: dread d2/dd/d48/d22/d2b/d54/f5a [0,4194304] 0 2024-09-16T00:25:56.189 INFO:tasks.workunit.client.1.smithi169.stdout:7/128: creat d7/da/f23 x:0 0 0 2024-09-16T00:25:56.190 INFO:tasks.workunit.client.1.smithi169.stdout:3/149: write d7/d1a/d1c/d23/f28 [118353,130361] 0 2024-09-16T00:25:56.192 INFO:tasks.workunit.client.1.smithi169.stdout:2/102: link f5 d7/db/f23 0 2024-09-16T00:25:56.196 INFO:tasks.workunit.client.1.smithi169.stdout:0/131: unlink d0/d2/d8/l1e 0 2024-09-16T00:25:56.196 INFO:tasks.workunit.client.1.smithi169.stdout:7/129: dread d7/f17 [0,4194304] 0 2024-09-16T00:25:56.197 INFO:tasks.workunit.client.1.smithi169.stdout:1/176: rename dc/d26/l28 to dc/d18/l31 0 2024-09-16T00:25:56.198 INFO:tasks.workunit.client.0.smithi112.stdout:0/298: dwrite d5/f2a [0,4194304] 0 2024-09-16T00:25:56.199 INFO:tasks.workunit.client.1.smithi169.stdout:4/132: write d2/f9 [4295471,61029] 0 2024-09-16T00:25:56.199 INFO:tasks.workunit.client.0.smithi112.stdout:0/299: write d5/da/f27 [176871,42098] 0 2024-09-16T00:25:56.199 INFO:tasks.workunit.client.1.smithi169.stdout:2/103: creat d7/f24 x:0 0 0 2024-09-16T00:25:56.200 INFO:tasks.workunit.client.1.smithi169.stdout:4/133: read - d2/d12/d13/d1f/f20 zero size 2024-09-16T00:25:56.200 INFO:tasks.workunit.client.0.smithi112.stdout:0/300: chown d5/da/dc/f13 3880 1 2024-09-16T00:25:56.201 INFO:tasks.workunit.client.1.smithi169.stdout:3/150: chown d7/d1a/l21 15 1 2024-09-16T00:25:56.202 INFO:tasks.workunit.client.1.smithi169.stdout:3/151: chown d7 944 1 2024-09-16T00:25:56.202 INFO:tasks.workunit.client.0.smithi112.stdout:0/301: write d5/f2c [1029059,123023] 0 2024-09-16T00:25:56.202 INFO:tasks.workunit.client.0.smithi112.stdout:0/302: readlink d5/l9 0 2024-09-16T00:25:56.204 INFO:tasks.workunit.client.1.smithi169.stdout:0/132: creat d0/d2/d8/d5/d14/d21/f2f x:0 0 0 2024-09-16T00:25:56.204 INFO:tasks.workunit.client.1.smithi169.stdout:9/181: truncate d5/f14 4039621 0 2024-09-16T00:25:56.206 INFO:tasks.workunit.client.0.smithi112.stdout:2/337: truncate d2/dd/d48/d22/d2b/d30/f36 710744 0 2024-09-16T00:25:56.206 INFO:tasks.workunit.client.0.smithi112.stdout:2/338: readlink d2/dd/d48/l4b 0 2024-09-16T00:25:56.208 INFO:tasks.workunit.client.0.smithi112.stdout:2/339: mkdir d2/d68 0 2024-09-16T00:25:56.209 INFO:tasks.workunit.client.1.smithi169.stdout:2/104: truncate d7/fe 124012 0 2024-09-16T00:25:56.209 INFO:tasks.workunit.client.1.smithi169.stdout:6/72: sync 2024-09-16T00:25:56.210 INFO:tasks.workunit.client.0.smithi112.stdout:0/303: truncate d5/da/f53 1108160 0 2024-09-16T00:25:56.211 INFO:tasks.workunit.client.1.smithi169.stdout:3/152: dread d7/f10 [0,4194304] 0 2024-09-16T00:25:56.212 INFO:tasks.workunit.client.1.smithi169.stdout:1/177: creat dc/d2c/f32 x:0 0 0 2024-09-16T00:25:56.213 INFO:tasks.workunit.client.1.smithi169.stdout:7/130: dwrite d7/f8 [0,4194304] 0 2024-09-16T00:25:56.213 INFO:tasks.workunit.client.1.smithi169.stdout:5/151: write d2/f29 [498067,69149] 0 2024-09-16T00:25:56.214 INFO:tasks.workunit.client.0.smithi112.stdout:0/304: creat d5/da/d34/f5c x:0 0 0 2024-09-16T00:25:56.214 INFO:tasks.workunit.client.0.smithi112.stdout:2/340: dwrite d2/f8 [0,4194304] 0 2024-09-16T00:25:56.215 INFO:tasks.workunit.client.1.smithi169.stdout:8/129: dread f6 [0,4194304] 0 2024-09-16T00:25:56.215 INFO:tasks.workunit.client.1.smithi169.stdout:7/131: fdatasync f6 0 2024-09-16T00:25:56.216 INFO:tasks.workunit.client.0.smithi112.stdout:0/305: write d5/da/d18/f54 [15508,9069] 0 2024-09-16T00:25:56.217 INFO:tasks.workunit.client.0.smithi112.stdout:2/341: mknod d2/dd/d48/d22/c69 0 2024-09-16T00:25:56.218 INFO:tasks.workunit.client.1.smithi169.stdout:0/133: mknod d0/d2/d8/d5/d14/d21/c30 0 2024-09-16T00:25:56.218 INFO:tasks.workunit.client.0.smithi112.stdout:0/306: write d5/da/dc/f10 [1491151,68714] 0 2024-09-16T00:25:56.218 INFO:tasks.workunit.client.0.smithi112.stdout:2/342: chown d2/dd/d48/f2a 0 1 2024-09-16T00:25:56.219 INFO:tasks.workunit.client.1.smithi169.stdout:2/105: symlink d7/l25 0 2024-09-16T00:25:56.219 INFO:tasks.workunit.client.1.smithi169.stdout:9/182: mknod d5/da/d21/c42 0 2024-09-16T00:25:56.220 INFO:tasks.workunit.client.0.smithi112.stdout:0/307: write d5/f23 [2063872,74264] 0 2024-09-16T00:25:56.227 INFO:tasks.workunit.client.1.smithi169.stdout:3/153: symlink d7/d1a/d1c/l33 0 2024-09-16T00:25:56.228 INFO:tasks.workunit.client.1.smithi169.stdout:3/154: chown d7/f10 231928 1 2024-09-16T00:25:56.228 INFO:tasks.workunit.client.0.smithi112.stdout:2/343: dwrite d2/dd/d48/f23 [0,4194304] 0 2024-09-16T00:25:56.231 INFO:tasks.workunit.client.1.smithi169.stdout:4/134: creat d2/d12/f2c x:0 0 0 2024-09-16T00:25:56.231 INFO:tasks.workunit.client.1.smithi169.stdout:8/130: symlink d7/dc/d17/l2b 0 2024-09-16T00:25:56.232 INFO:tasks.workunit.client.0.smithi112.stdout:2/344: unlink d2/dd/d48/d22/d2b/c52 0 2024-09-16T00:25:56.233 INFO:tasks.workunit.client.1.smithi169.stdout:0/134: write d0/d2/d8/d5/d14/f28 [713806,99995] 0 2024-09-16T00:25:56.233 INFO:tasks.workunit.client.0.smithi112.stdout:2/345: creat d2/dd/d48/d22/f6a x:0 0 0 2024-09-16T00:25:56.235 INFO:tasks.workunit.client.0.smithi112.stdout:2/346: write d2/dd/d48/d22/d2b/f34 [503419,57673] 0 2024-09-16T00:25:56.236 INFO:tasks.workunit.client.1.smithi169.stdout:1/178: dwrite dc/f15 [0,4194304] 0 2024-09-16T00:25:56.236 INFO:tasks.workunit.client.1.smithi169.stdout:6/73: dwrite d2/fb [0,4194304] 0 2024-09-16T00:25:56.236 INFO:tasks.workunit.client.1.smithi169.stdout:2/106: mkdir d7/d1c/d26 0 2024-09-16T00:25:56.237 INFO:tasks.workunit.client.0.smithi112.stdout:2/347: fdatasync d2/f14 0 2024-09-16T00:25:56.240 INFO:tasks.workunit.client.0.smithi112.stdout:2/348: rename d2/dd/d48/d22/d2b/d3b/l5d to d2/dd/l6b 0 2024-09-16T00:25:56.242 INFO:tasks.workunit.client.1.smithi169.stdout:1/179: fsync dc/f21 0 2024-09-16T00:25:56.242 INFO:tasks.workunit.client.1.smithi169.stdout:2/107: fdatasync d7/f24 0 2024-09-16T00:25:56.243 INFO:tasks.workunit.client.1.smithi169.stdout:7/132: dread d7/df/f11 [0,4194304] 0 2024-09-16T00:25:56.244 INFO:tasks.workunit.client.1.smithi169.stdout:8/131: stat d7/f12 0 2024-09-16T00:25:56.244 INFO:tasks.workunit.client.0.smithi112.stdout:2/349: mkdir d2/dd/d48/d22/d2b/d54/d60/d6c 0 2024-09-16T00:25:56.246 INFO:tasks.workunit.client.1.smithi169.stdout:3/155: creat d7/d1a/d1c/d23/f34 x:0 0 0 2024-09-16T00:25:56.246 INFO:tasks.workunit.client.1.smithi169.stdout:2/108: chown d7/db/f17 247 1 2024-09-16T00:25:56.249 INFO:tasks.workunit.client.1.smithi169.stdout:2/109: chown d7 163 1 2024-09-16T00:25:56.249 INFO:tasks.workunit.client.1.smithi169.stdout:6/74: dread d2/fb [0,4194304] 0 2024-09-16T00:25:56.250 INFO:tasks.workunit.client.1.smithi169.stdout:8/132: read d7/dc/d17/f28 [441441,20117] 0 2024-09-16T00:25:56.250 INFO:tasks.workunit.client.1.smithi169.stdout:3/156: write d7/d1a/d1c/d23/f34 [498193,4165] 0 2024-09-16T00:25:56.250 INFO:tasks.workunit.client.1.smithi169.stdout:2/110: dread - d7/f24 zero size 2024-09-16T00:25:56.251 INFO:tasks.workunit.client.0.smithi112.stdout:2/350: mknod d2/dd/d48/d22/d2b/d54/d60/c6d 0 2024-09-16T00:25:56.252 INFO:tasks.workunit.client.1.smithi169.stdout:2/111: chown d7/lc 9 1 2024-09-16T00:25:56.253 INFO:tasks.workunit.client.1.smithi169.stdout:7/133: creat d7/d16/f24 x:0 0 0 2024-09-16T00:25:56.254 INFO:tasks.workunit.client.1.smithi169.stdout:4/135: dread d2/d12/f1e [0,4194304] 0 2024-09-16T00:25:56.254 INFO:tasks.workunit.client.0.smithi112.stdout:2/351: symlink d2/dd/d48/d22/l6e 0 2024-09-16T00:25:56.255 INFO:tasks.workunit.client.1.smithi169.stdout:2/112: read f3 [3637217,7027] 0 2024-09-16T00:25:56.255 INFO:tasks.workunit.client.1.smithi169.stdout:8/133: readlink d7/dc/d17/l2b 0 2024-09-16T00:25:56.256 INFO:tasks.workunit.client.1.smithi169.stdout:7/134: write d7/df/f15 [3238723,34660] 0 2024-09-16T00:25:56.257 INFO:tasks.workunit.client.1.smithi169.stdout:6/75: creat d2/f17 x:0 0 0 2024-09-16T00:25:56.260 INFO:tasks.workunit.client.1.smithi169.stdout:4/136: chown d2/f4 13718660 1 2024-09-16T00:25:56.260 INFO:tasks.workunit.client.1.smithi169.stdout:5/152: rmdir d2/da/d10/d13/d1c/d32/d34 0 2024-09-16T00:25:56.260 INFO:tasks.workunit.client.1.smithi169.stdout:6/76: readlink d2/l13 0 2024-09-16T00:25:56.261 INFO:tasks.workunit.client.1.smithi169.stdout:9/183: getdents d5/da/d2e 0 2024-09-16T00:25:56.262 INFO:tasks.workunit.client.1.smithi169.stdout:9/184: write d5/d12/f1c [921374,55333] 0 2024-09-16T00:25:56.264 INFO:tasks.workunit.client.1.smithi169.stdout:5/153: dread d2/da/f28 [0,4194304] 0 2024-09-16T00:25:56.265 INFO:tasks.workunit.client.1.smithi169.stdout:1/180: dread dc/df/f1c [0,4194304] 0 2024-09-16T00:25:56.265 INFO:tasks.workunit.client.1.smithi169.stdout:9/185: chown d5/da/d21/d23/l39 99 1 2024-09-16T00:25:56.268 INFO:tasks.workunit.client.1.smithi169.stdout:2/113: mkdir d7/db/d27 0 2024-09-16T00:25:56.268 INFO:tasks.workunit.client.1.smithi169.stdout:7/135: rmdir d7/da 39 2024-09-16T00:25:56.269 INFO:tasks.workunit.client.0.smithi112.stdout:0/308: write d5/da/f53 [1497484,90366] 0 2024-09-16T00:25:56.271 INFO:tasks.workunit.client.1.smithi169.stdout:2/114: chown d7/db/f17 7020097 1 2024-09-16T00:25:56.271 INFO:tasks.workunit.client.0.smithi112.stdout:0/309: unlink d5/da/dc/d12/f3f 0 2024-09-16T00:25:56.276 INFO:tasks.workunit.client.1.smithi169.stdout:9/186: dwrite d5/f15 [0,4194304] 0 2024-09-16T00:25:56.277 INFO:tasks.workunit.client.1.smithi169.stdout:1/181: dwrite f7 [0,4194304] 0 2024-09-16T00:25:56.277 INFO:tasks.workunit.client.0.smithi112.stdout:0/310: dwrite d5/f2c [0,4194304] 0 2024-09-16T00:25:56.278 INFO:tasks.workunit.client.1.smithi169.stdout:5/154: unlink d2/f20 0 2024-09-16T00:25:56.278 INFO:tasks.workunit.client.0.smithi112.stdout:0/311: fdatasync d5/ff 0 2024-09-16T00:25:56.279 INFO:tasks.workunit.client.0.smithi112.stdout:0/312: stat d5/da/dc/f5b 0 2024-09-16T00:25:56.279 INFO:tasks.workunit.client.1.smithi169.stdout:7/136: chown d7/da/f23 705 1 2024-09-16T00:25:56.282 INFO:tasks.workunit.client.1.smithi169.stdout:4/137: unlink d2/dd/d1a/l2b 0 2024-09-16T00:25:56.282 INFO:tasks.workunit.client.0.smithi112.stdout:0/313: creat d5/da/d34/d48/f5d x:0 0 0 2024-09-16T00:25:56.285 INFO:tasks.workunit.client.1.smithi169.stdout:0/135: getdents d0 0 2024-09-16T00:25:56.286 INFO:tasks.workunit.client.0.smithi112.stdout:0/314: read d5/da/dc/fe [244752,93992] 0 2024-09-16T00:25:56.287 INFO:tasks.workunit.client.1.smithi169.stdout:0/136: write d0/d2/d15/f27 [151964,100611] 0 2024-09-16T00:25:56.287 INFO:tasks.workunit.client.0.smithi112.stdout:5/320: sync 2024-09-16T00:25:56.288 INFO:tasks.workunit.client.0.smithi112.stdout:5/321: write d4/d7/d26/f44 [999110,9871] 0 2024-09-16T00:25:56.288 INFO:tasks.workunit.client.1.smithi169.stdout:0/137: write d0/f10 [2169385,115388] 0 2024-09-16T00:25:56.289 INFO:tasks.workunit.client.0.smithi112.stdout:5/322: dread - d4/d7/d26/d4c/f6a zero size 2024-09-16T00:25:56.290 INFO:tasks.workunit.client.0.smithi112.stdout:5/323: chown d4/d1c/d43/f6d 4492404 1 2024-09-16T00:25:56.291 INFO:tasks.workunit.client.1.smithi169.stdout:6/77: creat d2/d15/f18 x:0 0 0 2024-09-16T00:25:56.293 INFO:tasks.workunit.client.1.smithi169.stdout:1/182: unlink fb 0 2024-09-16T00:25:56.293 INFO:tasks.workunit.client.1.smithi169.stdout:3/157: dread d7/d1a/f32 [4194304,4194304] 0 2024-09-16T00:25:56.294 INFO:tasks.workunit.client.0.smithi112.stdout:5/324: rename d4/d7/d14/l58 to d4/d7/d14/l71 0 2024-09-16T00:25:56.294 INFO:tasks.workunit.client.1.smithi169.stdout:9/187: creat d5/da/d2e/f43 x:0 0 0 2024-09-16T00:25:56.294 INFO:tasks.workunit.client.1.smithi169.stdout:5/155: creat d2/da/d10/d13/f35 x:0 0 0 2024-09-16T00:25:56.294 INFO:tasks.workunit.client.1.smithi169.stdout:2/115: truncate f3 3076363 0 2024-09-16T00:25:56.296 INFO:tasks.workunit.client.1.smithi169.stdout:7/137: unlink d7/df/l10 0 2024-09-16T00:25:56.296 INFO:tasks.workunit.client.0.smithi112.stdout:5/325: rename d4/d7/d26/f5a to d4/dd/d6f/f72 0 2024-09-16T00:25:56.297 INFO:tasks.workunit.client.0.smithi112.stdout:5/326: dread - d4/dd/d12/d52/f63 zero size 2024-09-16T00:25:56.298 INFO:tasks.workunit.client.0.smithi112.stdout:5/327: fdatasync d4/f57 0 2024-09-16T00:25:56.299 INFO:tasks.workunit.client.1.smithi169.stdout:8/134: truncate f5 3285604 0 2024-09-16T00:25:56.300 INFO:tasks.workunit.client.1.smithi169.stdout:4/138: mknod d2/d12/d13/c2d 0 2024-09-16T00:25:56.300 INFO:tasks.workunit.client.1.smithi169.stdout:0/138: mkdir d0/d2/d8/d5/d14/d21/d31 0 2024-09-16T00:25:56.301 INFO:tasks.workunit.client.0.smithi112.stdout:5/328: getdents d4/d7/d26/d4c 0 2024-09-16T00:25:56.302 INFO:tasks.workunit.client.0.smithi112.stdout:5/329: truncate d4/dd/f66 291294 0 2024-09-16T00:25:56.303 INFO:tasks.workunit.client.0.smithi112.stdout:5/330: dread - d4/dd/d12/d52/f63 zero size 2024-09-16T00:25:56.305 INFO:tasks.workunit.client.1.smithi169.stdout:1/183: write dc/fd [3584987,127264] 0 2024-09-16T00:25:56.306 INFO:tasks.workunit.client.1.smithi169.stdout:6/78: dwrite d2/f17 [0,4194304] 0 2024-09-16T00:25:56.306 INFO:tasks.workunit.client.1.smithi169.stdout:2/116: symlink d7/d1c/d26/l28 0 2024-09-16T00:25:56.306 INFO:tasks.workunit.client.0.smithi112.stdout:5/331: creat d4/d1c/f73 x:0 0 0 2024-09-16T00:25:56.306 INFO:tasks.workunit.client.1.smithi169.stdout:4/139: creat d2/d12/d13/f2e x:0 0 0 2024-09-16T00:25:56.307 INFO:tasks.workunit.client.1.smithi169.stdout:8/135: mknod d7/dc/d26/c2c 0 2024-09-16T00:25:56.311 INFO:tasks.workunit.client.1.smithi169.stdout:8/136: chown d7/dc/d17 14136156 1 2024-09-16T00:25:56.316 INFO:tasks.workunit.client.1.smithi169.stdout:3/158: dwrite d7/d1a/f30 [0,4194304] 0 2024-09-16T00:25:56.316 INFO:tasks.workunit.client.0.smithi112.stdout:0/315: dread d5/da/d34/f3b [4194304,4194304] 0 2024-09-16T00:25:56.317 INFO:tasks.workunit.client.1.smithi169.stdout:9/188: creat d5/da/d21/f44 x:0 0 0 2024-09-16T00:25:56.317 INFO:tasks.workunit.client.0.smithi112.stdout:0/316: mknod d5/da/d18/c5e 0 2024-09-16T00:25:56.318 INFO:tasks.workunit.client.0.smithi112.stdout:0/317: readlink d5/da/d34/d48/l59 0 2024-09-16T00:25:56.318 INFO:tasks.workunit.client.0.smithi112.stdout:0/318: stat d5/da/dc/d12/f17 0 2024-09-16T00:25:56.319 INFO:tasks.workunit.client.1.smithi169.stdout:7/138: dwrite d7/df/f11 [0,4194304] 0 2024-09-16T00:25:56.320 INFO:tasks.workunit.client.0.smithi112.stdout:0/319: mkdir d5/da/d18/d24/d46/d5f 0 2024-09-16T00:25:56.320 INFO:tasks.workunit.client.0.smithi112.stdout:0/320: chown d5/da/d34/f4d 1240163396 1 2024-09-16T00:25:56.322 INFO:tasks.workunit.client.1.smithi169.stdout:1/184: stat dc/df/d29 0 2024-09-16T00:25:56.322 INFO:tasks.workunit.client.1.smithi169.stdout:1/185: fsync f7 0 2024-09-16T00:25:56.322 INFO:tasks.workunit.client.0.smithi112.stdout:0/321: rmdir d5/da/d34 39 2024-09-16T00:25:56.325 INFO:tasks.workunit.client.1.smithi169.stdout:7/139: dread - d7/df/f1c zero size 2024-09-16T00:25:56.325 INFO:tasks.workunit.client.1.smithi169.stdout:8/137: symlink d7/dc/d26/l2d 0 2024-09-16T00:25:56.325 INFO:tasks.workunit.client.1.smithi169.stdout:2/117: dwrite d7/f11 [0,4194304] 0 2024-09-16T00:25:56.328 INFO:tasks.workunit.client.1.smithi169.stdout:7/140: write d7/df/f1c [766352,10614] 0 2024-09-16T00:25:56.328 INFO:tasks.workunit.client.1.smithi169.stdout:6/79: rmdir d2 39 2024-09-16T00:25:56.329 INFO:tasks.workunit.client.1.smithi169.stdout:2/118: readlink d7/db/l1e 0 2024-09-16T00:25:56.331 INFO:tasks.workunit.client.1.smithi169.stdout:7/141: write d7/da/f23 [148173,45279] 0 2024-09-16T00:25:56.332 INFO:tasks.workunit.client.1.smithi169.stdout:2/119: dread - d7/f24 zero size 2024-09-16T00:25:56.336 INFO:tasks.workunit.client.1.smithi169.stdout:3/159: dread d7/d1a/f30 [0,4194304] 0 2024-09-16T00:25:56.337 INFO:tasks.workunit.client.1.smithi169.stdout:3/160: stat d7/fd 0 2024-09-16T00:25:56.340 INFO:tasks.workunit.client.1.smithi169.stdout:3/161: chown d7/c26 4430 1 2024-09-16T00:25:56.341 INFO:tasks.workunit.client.1.smithi169.stdout:6/80: truncate d2/d15/f18 721469 0 2024-09-16T00:25:56.342 INFO:tasks.workunit.client.1.smithi169.stdout:7/142: rename d7/da/f14 to d7/d16/f25 0 2024-09-16T00:25:56.342 INFO:tasks.workunit.client.1.smithi169.stdout:8/138: dwrite f6 [8388608,4194304] 0 2024-09-16T00:25:56.343 INFO:tasks.workunit.client.1.smithi169.stdout:7/143: write d7/da/f23 [1010461,88396] 0 2024-09-16T00:25:56.343 INFO:tasks.workunit.client.1.smithi169.stdout:9/189: creat d5/da/d21/f45 x:0 0 0 2024-09-16T00:25:56.346 INFO:tasks.workunit.client.1.smithi169.stdout:2/120: mkdir d7/db/d27/d29 0 2024-09-16T00:25:56.347 INFO:tasks.workunit.client.1.smithi169.stdout:2/121: write f2 [2888999,117325] 0 2024-09-16T00:25:56.349 INFO:tasks.workunit.client.1.smithi169.stdout:3/162: mkdir d7/d1a/d1c/d35 0 2024-09-16T00:25:56.351 INFO:tasks.workunit.client.1.smithi169.stdout:4/140: rmdir d2 39 2024-09-16T00:25:56.352 INFO:tasks.workunit.client.1.smithi169.stdout:6/81: unlink d2/l9 0 2024-09-16T00:25:56.352 INFO:tasks.workunit.client.1.smithi169.stdout:5/156: truncate f0 1536752 0 2024-09-16T00:25:56.354 INFO:tasks.workunit.client.1.smithi169.stdout:8/139: dwrite f6 [0,4194304] 0 2024-09-16T00:25:56.354 INFO:tasks.workunit.client.1.smithi169.stdout:3/163: read d7/d1a/d1c/d23/f2f [6961003,38545] 0 2024-09-16T00:25:56.354 INFO:tasks.workunit.client.1.smithi169.stdout:3/164: stat d7 0 2024-09-16T00:25:56.355 INFO:tasks.workunit.client.1.smithi169.stdout:8/140: stat d7/dc/d17/d24 0 2024-09-16T00:25:56.356 INFO:tasks.workunit.client.1.smithi169.stdout:1/186: rename dc/df/f12 to dc/df/f33 0 2024-09-16T00:25:56.358 INFO:tasks.workunit.client.1.smithi169.stdout:1/187: chown f7 399455 1 2024-09-16T00:25:56.358 INFO:tasks.workunit.client.1.smithi169.stdout:2/122: dread d7/fe [0,4194304] 0 2024-09-16T00:25:56.364 INFO:tasks.workunit.client.1.smithi169.stdout:6/82: creat d2/f19 x:0 0 0 2024-09-16T00:25:56.366 INFO:tasks.workunit.client.1.smithi169.stdout:1/188: dwrite f7 [4194304,4194304] 0 2024-09-16T00:25:56.369 INFO:tasks.workunit.client.1.smithi169.stdout:8/141: creat d7/f2e x:0 0 0 2024-09-16T00:25:56.370 INFO:tasks.workunit.client.1.smithi169.stdout:4/141: dread d2/d12/f1e [0,4194304] 0 2024-09-16T00:25:56.371 INFO:tasks.workunit.client.1.smithi169.stdout:4/142: dread - d2/d12/d13/d1f/f20 zero size 2024-09-16T00:25:56.373 INFO:tasks.workunit.client.1.smithi169.stdout:2/123: chown d7/c1b 153 1 2024-09-16T00:25:56.374 INFO:tasks.workunit.client.1.smithi169.stdout:2/124: read - d7/f24 zero size 2024-09-16T00:25:56.378 INFO:tasks.workunit.client.1.smithi169.stdout:6/83: mknod d2/d15/c1a 0 2024-09-16T00:25:56.379 INFO:tasks.workunit.client.1.smithi169.stdout:3/165: dread d7/f9 [0,4194304] 0 2024-09-16T00:25:56.380 INFO:tasks.workunit.client.1.smithi169.stdout:8/142: symlink d7/l2f 0 2024-09-16T00:25:56.382 INFO:tasks.workunit.client.1.smithi169.stdout:7/144: rename f6 to d7/f26 0 2024-09-16T00:25:56.383 INFO:tasks.workunit.client.1.smithi169.stdout:9/190: creat d5/da/d21/f46 x:0 0 0 2024-09-16T00:25:56.384 INFO:tasks.workunit.client.1.smithi169.stdout:4/143: mkdir d2/d12/d13/d1f/d2f 0 2024-09-16T00:25:56.385 INFO:tasks.workunit.client.1.smithi169.stdout:5/157: mkdir d2/da/d10/d13/d1c/d32/d36 0 2024-09-16T00:25:56.388 INFO:tasks.workunit.client.1.smithi169.stdout:6/84: creat d2/d15/f1b x:0 0 0 2024-09-16T00:25:56.389 INFO:tasks.workunit.client.1.smithi169.stdout:3/166: mknod d7/d1a/d1c/d23/c36 0 2024-09-16T00:25:56.390 INFO:tasks.workunit.client.1.smithi169.stdout:8/143: rmdir d7/dc/d26 39 2024-09-16T00:25:56.391 INFO:tasks.workunit.client.1.smithi169.stdout:7/145: mknod d7/df/c27 0 2024-09-16T00:25:56.395 INFO:tasks.workunit.client.1.smithi169.stdout:2/125: link d7/f1a d7/db/d27/f2a 0 2024-09-16T00:25:56.396 INFO:tasks.workunit.client.1.smithi169.stdout:1/189: truncate f7 4914231 0 2024-09-16T00:25:56.399 INFO:tasks.workunit.client.1.smithi169.stdout:4/144: mknod d2/dd/c30 0 2024-09-16T00:25:56.399 INFO:tasks.workunit.client.1.smithi169.stdout:1/190: dread - dc/f21 zero size 2024-09-16T00:25:56.399 INFO:tasks.workunit.client.1.smithi169.stdout:8/144: write d7/fd [5142126,7178] 0 2024-09-16T00:25:56.400 INFO:tasks.workunit.client.1.smithi169.stdout:3/167: mkdir d7/d1a/d1c/d23/d2a/d37 0 2024-09-16T00:25:56.400 INFO:tasks.workunit.client.1.smithi169.stdout:9/191: dwrite d5/da/f28 [0,4194304] 0 2024-09-16T00:25:56.402 INFO:tasks.workunit.client.1.smithi169.stdout:7/146: mknod d7/c28 0 2024-09-16T00:25:56.408 INFO:tasks.workunit.client.0.smithi112.stdout:0/322: unlink d5/da/d18/c5e 0 2024-09-16T00:25:56.409 INFO:tasks.workunit.client.1.smithi169.stdout:2/126: dwrite d7/db/f19 [0,4194304] 0 2024-09-16T00:25:56.410 INFO:tasks.workunit.client.0.smithi112.stdout:0/323: truncate d5/da/d18/f1f 2272811 0 2024-09-16T00:25:56.414 INFO:tasks.workunit.client.0.smithi112.stdout:0/324: rename d5/l7 to d5/da/dc/d12/l60 0 2024-09-16T00:25:56.414 INFO:tasks.workunit.client.1.smithi169.stdout:5/158: unlink f0 0 2024-09-16T00:25:56.415 INFO:tasks.workunit.client.0.smithi112.stdout:0/325: chown d5/da/d18/d24/d5a 323938 1 2024-09-16T00:25:56.415 INFO:tasks.workunit.client.1.smithi169.stdout:7/147: dread d7/da/f23 [0,4194304] 0 2024-09-16T00:25:56.415 INFO:tasks.workunit.client.1.smithi169.stdout:1/191: dwrite dc/f15 [0,4194304] 0 2024-09-16T00:25:56.416 INFO:tasks.workunit.client.0.smithi112.stdout:0/326: write d5/f2c [2936749,995] 0 2024-09-16T00:25:56.417 INFO:tasks.workunit.client.1.smithi169.stdout:3/168: mknod d7/d1a/c38 0 2024-09-16T00:25:56.418 INFO:tasks.workunit.client.1.smithi169.stdout:8/145: write f5 [3331975,123011] 0 2024-09-16T00:25:56.419 INFO:tasks.workunit.client.1.smithi169.stdout:4/145: creat d2/d12/f31 x:0 0 0 2024-09-16T00:25:56.419 INFO:tasks.workunit.client.1.smithi169.stdout:9/192: write d5/da/f16 [875681,23171] 0 2024-09-16T00:25:56.420 INFO:tasks.workunit.client.0.smithi112.stdout:0/327: getdents d5/da/dc/d4e 0 2024-09-16T00:25:56.421 INFO:tasks.workunit.client.1.smithi169.stdout:4/146: stat d2/d12/d13/d1f/f20 0 2024-09-16T00:25:56.422 INFO:tasks.workunit.client.0.smithi112.stdout:0/328: mknod d5/da/c61 0 2024-09-16T00:25:56.424 INFO:tasks.workunit.client.1.smithi169.stdout:5/159: dread d2/f29 [0,4194304] 0 2024-09-16T00:25:56.425 INFO:tasks.workunit.client.1.smithi169.stdout:1/192: write dc/df/f16 [713879,1294] 0 2024-09-16T00:25:56.426 INFO:tasks.workunit.client.1.smithi169.stdout:2/127: dread f3 [0,4194304] 0 2024-09-16T00:25:56.430 INFO:tasks.workunit.client.1.smithi169.stdout:8/146: creat d7/dc/d2a/f30 x:0 0 0 2024-09-16T00:25:56.430 INFO:tasks.workunit.client.1.smithi169.stdout:4/147: creat d2/d12/d13/d27/f32 x:0 0 0 2024-09-16T00:25:56.432 INFO:tasks.workunit.client.1.smithi169.stdout:2/128: write d7/f24 [809597,31640] 0 2024-09-16T00:25:56.435 INFO:tasks.workunit.client.1.smithi169.stdout:4/148: write d2/f10 [3057273,34330] 0 2024-09-16T00:25:56.438 INFO:tasks.workunit.client.1.smithi169.stdout:5/160: dwrite d2/da/f19 [0,4194304] 0 2024-09-16T00:25:56.438 INFO:tasks.workunit.client.1.smithi169.stdout:3/169: link d7/l1f d7/d1a/d2b/l39 0 2024-09-16T00:25:56.441 INFO:tasks.workunit.client.1.smithi169.stdout:2/129: mknod d7/db/c2b 0 2024-09-16T00:25:56.442 INFO:tasks.workunit.client.1.smithi169.stdout:7/148: write d7/da/fc [5341307,123144] 0 2024-09-16T00:25:56.443 INFO:tasks.workunit.client.1.smithi169.stdout:3/170: creat d7/d1a/d1c/d23/f3a x:0 0 0 2024-09-16T00:25:56.445 INFO:tasks.workunit.client.1.smithi169.stdout:1/193: dwrite dc/df/f23 [0,4194304] 0 2024-09-16T00:25:56.445 INFO:tasks.workunit.client.1.smithi169.stdout:4/149: chown d2/d12/d13/c1d 4875 1 2024-09-16T00:25:56.446 INFO:tasks.workunit.client.1.smithi169.stdout:4/150: chown d2/ca 3115 1 2024-09-16T00:25:56.446 INFO:tasks.workunit.client.1.smithi169.stdout:3/171: fdatasync d7/d1a/d1c/d23/f3a 0 2024-09-16T00:25:56.447 INFO:tasks.workunit.client.1.smithi169.stdout:2/130: creat d7/d1c/d26/f2c x:0 0 0 2024-09-16T00:25:56.448 INFO:tasks.workunit.client.1.smithi169.stdout:5/161: rmdir d2/da/d10/d13/d1c/d32/d36 0 2024-09-16T00:25:56.449 INFO:tasks.workunit.client.1.smithi169.stdout:7/149: mkdir d7/da/d29 0 2024-09-16T00:25:56.450 INFO:tasks.workunit.client.1.smithi169.stdout:3/172: truncate d7/d1a/d1c/d23/f3a 68298 0 2024-09-16T00:25:56.451 INFO:tasks.workunit.client.1.smithi169.stdout:4/151: mkdir d2/d12/d13/d27/d33 0 2024-09-16T00:25:56.451 INFO:tasks.workunit.client.1.smithi169.stdout:5/162: rmdir d2/da/d10/d13/d1c/d2d 39 2024-09-16T00:25:56.452 INFO:tasks.workunit.client.1.smithi169.stdout:4/152: stat d2/d12/d13/d1f 0 2024-09-16T00:25:56.453 INFO:tasks.workunit.client.1.smithi169.stdout:2/131: symlink d7/d1c/d26/l2d 0 2024-09-16T00:25:56.455 INFO:tasks.workunit.client.1.smithi169.stdout:7/150: creat d7/da/f2a x:0 0 0 2024-09-16T00:25:56.457 INFO:tasks.workunit.client.1.smithi169.stdout:2/132: mkdir d7/d2e 0 2024-09-16T00:25:56.458 INFO:tasks.workunit.client.1.smithi169.stdout:2/133: dread - d7/db/f17 zero size 2024-09-16T00:25:56.458 INFO:tasks.workunit.client.1.smithi169.stdout:3/173: symlink d7/d1a/d2b/l3b 0 2024-09-16T00:25:56.459 INFO:tasks.workunit.client.1.smithi169.stdout:7/151: rename d7/da/f20 to d7/df/f2b 0 2024-09-16T00:25:56.460 INFO:tasks.workunit.client.1.smithi169.stdout:4/153: symlink d2/l34 0 2024-09-16T00:25:56.461 INFO:tasks.workunit.client.1.smithi169.stdout:4/154: dread - d2/d12/f2c zero size 2024-09-16T00:25:56.462 INFO:tasks.workunit.client.1.smithi169.stdout:3/174: dread d7/f10 [0,4194304] 0 2024-09-16T00:25:56.463 INFO:tasks.workunit.client.1.smithi169.stdout:2/134: rename d7/l10 to d7/d2e/l2f 0 2024-09-16T00:25:56.465 INFO:tasks.workunit.client.1.smithi169.stdout:4/155: fdatasync d2/dd/f18 0 2024-09-16T00:25:56.465 INFO:tasks.workunit.client.1.smithi169.stdout:7/152: dread d7/df/f1c [0,4194304] 0 2024-09-16T00:25:56.466 INFO:tasks.workunit.client.1.smithi169.stdout:3/175: creat d7/d1a/d1c/d23/f3c x:0 0 0 2024-09-16T00:25:56.467 INFO:tasks.workunit.client.1.smithi169.stdout:2/135: symlink d7/db/d27/l30 0 2024-09-16T00:25:56.467 INFO:tasks.workunit.client.1.smithi169.stdout:4/156: write d2/d12/d13/f2e [238093,69931] 0 2024-09-16T00:25:56.470 INFO:tasks.workunit.client.1.smithi169.stdout:4/157: write d2/dd/f26 [570483,112457] 0 2024-09-16T00:25:56.470 INFO:tasks.workunit.client.0.smithi112.stdout:9/499: sync 2024-09-16T00:25:56.470 INFO:tasks.workunit.client.0.smithi112.stdout:6/311: sync 2024-09-16T00:25:56.473 INFO:tasks.workunit.client.1.smithi169.stdout:3/176: truncate d7/f24 1073218 0 2024-09-16T00:25:56.473 INFO:tasks.workunit.client.1.smithi169.stdout:8/147: read f5 [2631020,43304] 0 2024-09-16T00:25:56.474 INFO:tasks.workunit.client.0.smithi112.stdout:0/329: fsync d5/da/dc/fe 0 2024-09-16T00:25:56.475 INFO:tasks.workunit.client.0.smithi112.stdout:0/330: chown d5/f2c 29985239 1 2024-09-16T00:25:56.475 INFO:tasks.workunit.client.0.smithi112.stdout:0/331: fsync d5/da/dc/f5b 0 2024-09-16T00:25:56.476 INFO:tasks.workunit.client.1.smithi169.stdout:1/194: dread dc/df/f16 [0,4194304] 0 2024-09-16T00:25:56.476 INFO:tasks.workunit.client.1.smithi169.stdout:5/163: truncate d2/fd 1745228 0 2024-09-16T00:25:56.477 INFO:tasks.workunit.client.1.smithi169.stdout:7/153: dwrite d7/d16/f1d [0,4194304] 0 2024-09-16T00:25:56.478 INFO:tasks.workunit.client.1.smithi169.stdout:4/158: creat d2/d12/d13/d1f/f35 x:0 0 0 2024-09-16T00:25:56.478 INFO:tasks.workunit.client.1.smithi169.stdout:2/136: dread d7/f11 [0,4194304] 0 2024-09-16T00:25:56.480 INFO:tasks.workunit.client.1.smithi169.stdout:3/177: fsync d7/f14 0 2024-09-16T00:25:56.481 INFO:tasks.workunit.client.1.smithi169.stdout:1/195: write f7 [5553641,103820] 0 2024-09-16T00:25:56.484 INFO:tasks.workunit.client.1.smithi169.stdout:5/164: mkdir d2/da/d37 0 2024-09-16T00:25:56.484 INFO:tasks.workunit.client.1.smithi169.stdout:1/196: dread - dc/f21 zero size 2024-09-16T00:25:56.486 INFO:tasks.workunit.client.1.smithi169.stdout:3/178: creat d7/d1a/d1c/d23/f3d x:0 0 0 2024-09-16T00:25:56.488 INFO:tasks.workunit.client.1.smithi169.stdout:5/165: write d2/f29 [348933,113777] 0 2024-09-16T00:25:56.488 INFO:tasks.workunit.client.1.smithi169.stdout:4/159: read d2/fb [223791,94395] 0 2024-09-16T00:25:56.489 INFO:tasks.workunit.client.1.smithi169.stdout:8/148: getdents d7/dc/d1d 0 2024-09-16T00:25:56.489 INFO:tasks.workunit.client.1.smithi169.stdout:7/154: dwrite d7/f8 [4194304,4194304] 0 2024-09-16T00:25:56.490 INFO:tasks.workunit.client.1.smithi169.stdout:5/166: readlink d2/da/l33 0 2024-09-16T00:25:56.492 INFO:tasks.workunit.client.1.smithi169.stdout:8/149: stat d7/dc/d17/l19 0 2024-09-16T00:25:56.492 INFO:tasks.workunit.client.1.smithi169.stdout:1/197: creat dc/f34 x:0 0 0 2024-09-16T00:25:56.495 INFO:tasks.workunit.client.1.smithi169.stdout:8/150: fdatasync d7/dc/f14 0 2024-09-16T00:25:56.495 INFO:tasks.workunit.client.1.smithi169.stdout:7/155: unlink d7/df/f1c 0 2024-09-16T00:25:56.496 INFO:tasks.workunit.client.1.smithi169.stdout:3/179: dread d7/d1a/d1c/d23/f28 [0,4194304] 0 2024-09-16T00:25:56.502 INFO:tasks.workunit.client.1.smithi169.stdout:4/160: rename d2/f10 to d2/d12/f36 0 2024-09-16T00:25:56.502 INFO:tasks.workunit.client.1.smithi169.stdout:2/137: truncate f2 1210277 0 2024-09-16T00:25:56.504 INFO:tasks.workunit.client.1.smithi169.stdout:8/151: dwrite d7/dc/d2a/f30 [0,4194304] 0 2024-09-16T00:25:56.507 INFO:tasks.workunit.client.1.smithi169.stdout:3/180: symlink d7/d1a/l3e 0 2024-09-16T00:25:56.508 INFO:tasks.workunit.client.1.smithi169.stdout:4/161: write d2/f3 [7074010,29830] 0 2024-09-16T00:25:56.509 INFO:tasks.workunit.client.1.smithi169.stdout:1/198: rmdir dc/df/d29/d2a/d2e 0 2024-09-16T00:25:56.511 INFO:tasks.workunit.client.1.smithi169.stdout:1/199: fsync dc/f15 0 2024-09-16T00:25:56.512 INFO:tasks.workunit.client.1.smithi169.stdout:2/138: chown d7/d2e/l2f 25 1 2024-09-16T00:25:56.518 INFO:tasks.workunit.client.1.smithi169.stdout:7/156: creat d7/f2c x:0 0 0 2024-09-16T00:25:56.518 INFO:tasks.workunit.client.1.smithi169.stdout:1/200: chown dc/df/f23 3519 1 2024-09-16T00:25:56.518 INFO:tasks.workunit.client.1.smithi169.stdout:5/167: write d2/fd [135060,108799] 0 2024-09-16T00:25:56.518 INFO:tasks.workunit.client.1.smithi169.stdout:2/139: readlink d7/d1c/d26/l28 0 2024-09-16T00:25:56.519 INFO:tasks.workunit.client.1.smithi169.stdout:4/162: truncate d2/d12/d13/d1f/f20 221382 0 2024-09-16T00:25:56.520 INFO:tasks.workunit.client.1.smithi169.stdout:8/152: dwrite d7/f10 [0,4194304] 0 2024-09-16T00:25:56.522 INFO:tasks.workunit.client.1.smithi169.stdout:7/157: symlink d7/da/l2d 0 2024-09-16T00:25:56.524 INFO:tasks.workunit.client.1.smithi169.stdout:5/168: creat d2/da/f38 x:0 0 0 2024-09-16T00:25:56.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:25:56 smithi169 ceph-mon[34653]: pgmap v251: 65 pgs: 65 active+clean; 624 MiB data, 3.3 GiB used, 533 GiB / 536 GiB avail; 5.6 MiB/s rd, 54 MiB/s wr, 284 op/s 2024-09-16T00:25:56.527 INFO:tasks.workunit.client.1.smithi169.stdout:3/181: dwrite d7/d1a/d1c/d23/f3c [0,4194304] 0 2024-09-16T00:25:56.530 INFO:tasks.workunit.client.1.smithi169.stdout:2/140: mknod d7/db/d20/d21/c31 0 2024-09-16T00:25:56.531 INFO:tasks.workunit.client.1.smithi169.stdout:7/158: mkdir d7/da/d2e 0 2024-09-16T00:25:56.531 INFO:tasks.workunit.client.1.smithi169.stdout:8/153: mknod d7/dc/d26/c31 0 2024-09-16T00:25:56.533 INFO:tasks.workunit.client.1.smithi169.stdout:7/159: write d7/df/f11 [5176359,119140] 0 2024-09-16T00:25:56.534 INFO:tasks.workunit.client.1.smithi169.stdout:3/182: mknod d7/d1a/d1c/d23/c3f 0 2024-09-16T00:25:56.537 INFO:tasks.workunit.client.1.smithi169.stdout:5/169: dwrite d2/fd [0,4194304] 0 2024-09-16T00:25:56.539 INFO:tasks.workunit.client.1.smithi169.stdout:8/154: mkdir d7/dc/d17/d24/d32 0 2024-09-16T00:25:56.540 INFO:tasks.workunit.client.1.smithi169.stdout:3/183: creat d7/d1a/d1c/f40 x:0 0 0 2024-09-16T00:25:56.540 INFO:tasks.workunit.client.1.smithi169.stdout:5/170: stat d2/da/d10 0 2024-09-16T00:25:56.540 INFO:tasks.workunit.client.1.smithi169.stdout:2/141: dread d7/db/f13 [0,4194304] 0 2024-09-16T00:25:56.541 INFO:tasks.workunit.client.1.smithi169.stdout:8/155: stat d7/dc/d26/c2c 0 2024-09-16T00:25:56.542 INFO:tasks.workunit.client.1.smithi169.stdout:7/160: dread d7/da/f23 [0,4194304] 0 2024-09-16T00:25:56.546 INFO:tasks.workunit.client.1.smithi169.stdout:1/201: truncate dc/d18/f20 1202229 0 2024-09-16T00:25:56.546 INFO:tasks.workunit.client.1.smithi169.stdout:4/163: truncate d2/f3 28804 0 2024-09-16T00:25:56.547 INFO:tasks.workunit.client.1.smithi169.stdout:8/156: rename d7/dc/d26/l2d to d7/dc/d26/l33 0 2024-09-16T00:25:56.547 INFO:tasks.workunit.client.1.smithi169.stdout:3/184: dread d7/d1a/f1b [0,4194304] 0 2024-09-16T00:25:56.549 INFO:tasks.workunit.client.1.smithi169.stdout:2/142: rename d7/c1b to d7/db/c32 0 2024-09-16T00:25:56.549 INFO:tasks.workunit.client.1.smithi169.stdout:7/161: rename d7 to d7/df/d2f 22 2024-09-16T00:25:56.550 INFO:tasks.workunit.client.1.smithi169.stdout:5/171: mknod d2/da/d37/c39 0 2024-09-16T00:25:56.550 INFO:tasks.workunit.client.1.smithi169.stdout:1/202: symlink dc/d26/l35 0 2024-09-16T00:25:56.551 INFO:tasks.workunit.client.1.smithi169.stdout:5/172: dread d2/f29 [0,4194304] 0 2024-09-16T00:25:56.552 INFO:tasks.workunit.client.1.smithi169.stdout:8/157: mknod d7/dc/d1d/c34 0 2024-09-16T00:25:56.554 INFO:tasks.workunit.client.1.smithi169.stdout:3/185: unlink d7/d1a/d1c/d23/f34 0 2024-09-16T00:25:56.554 INFO:tasks.workunit.client.1.smithi169.stdout:5/173: chown d2/ff 4 1 2024-09-16T00:25:56.557 INFO:tasks.workunit.client.1.smithi169.stdout:5/174: truncate d2/da/d10/d13/f35 382860 0 2024-09-16T00:25:56.558 INFO:tasks.workunit.client.1.smithi169.stdout:7/162: symlink d7/da/d29/l30 0 2024-09-16T00:25:56.560 INFO:tasks.workunit.client.1.smithi169.stdout:1/203: mkdir dc/d36 0 2024-09-16T00:25:56.562 INFO:tasks.workunit.client.1.smithi169.stdout:1/204: fdatasync dc/f34 0 2024-09-16T00:25:56.563 INFO:tasks.workunit.client.1.smithi169.stdout:7/163: creat d7/f31 x:0 0 0 2024-09-16T00:25:56.563 INFO:tasks.workunit.client.1.smithi169.stdout:4/164: dwrite d2/d12/f1e [0,4194304] 0 2024-09-16T00:25:56.566 INFO:tasks.workunit.client.1.smithi169.stdout:5/175: creat d2/da/d10/d13/d24/f3a x:0 0 0 2024-09-16T00:25:56.567 INFO:tasks.workunit.client.1.smithi169.stdout:3/186: dwrite d7/f9 [0,4194304] 0 2024-09-16T00:25:56.571 INFO:tasks.workunit.client.1.smithi169.stdout:3/187: fdatasync d7/d1a/d1c/d23/f3c 0 2024-09-16T00:25:56.575 INFO:tasks.workunit.client.1.smithi169.stdout:4/165: mknod d2/c37 0 2024-09-16T00:25:56.578 INFO:tasks.workunit.client.1.smithi169.stdout:3/188: chown d7/d1a/l2c 291282563 1 2024-09-16T00:25:56.580 INFO:tasks.workunit.client.1.smithi169.stdout:2/143: dwrite d7/db/f13 [0,4194304] 0 2024-09-16T00:25:56.580 INFO:tasks.workunit.client.1.smithi169.stdout:3/189: stat d7/d1a/d1c 0 2024-09-16T00:25:56.581 INFO:tasks.workunit.client.1.smithi169.stdout:5/176: creat d2/da/d10/d13/d1c/f3b x:0 0 0 2024-09-16T00:25:56.583 INFO:tasks.workunit.client.1.smithi169.stdout:5/177: chown d2/da/d10/d13 532149 1 2024-09-16T00:25:56.585 INFO:tasks.workunit.client.1.smithi169.stdout:2/144: write d7/db/d20/d21/f22 [728354,112606] 0 2024-09-16T00:25:56.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:25:56 smithi112 ceph-mon[27869]: pgmap v251: 65 pgs: 65 active+clean; 624 MiB data, 3.3 GiB used, 533 GiB / 536 GiB avail; 5.6 MiB/s rd, 54 MiB/s wr, 284 op/s 2024-09-16T00:25:56.589 INFO:tasks.workunit.client.1.smithi169.stdout:8/158: dwrite d7/dc/d17/f28 [0,4194304] 0 2024-09-16T00:25:56.591 INFO:tasks.workunit.client.1.smithi169.stdout:5/178: symlink d2/da/d10/l3c 0 2024-09-16T00:25:56.591 INFO:tasks.workunit.client.1.smithi169.stdout:2/145: creat d7/db/d20/d21/f33 x:0 0 0 2024-09-16T00:25:56.592 INFO:tasks.workunit.client.1.smithi169.stdout:8/159: symlink d7/dc/d17/l35 0 2024-09-16T00:25:56.596 INFO:tasks.workunit.client.1.smithi169.stdout:2/146: dread d7/f24 [0,4194304] 0 2024-09-16T00:25:56.596 INFO:tasks.workunit.client.1.smithi169.stdout:2/147: chown f3 735 1 2024-09-16T00:25:56.597 INFO:tasks.workunit.client.1.smithi169.stdout:8/160: unlink l3 0 2024-09-16T00:25:56.598 INFO:tasks.workunit.client.1.smithi169.stdout:8/161: chown d7/c9 32580 1 2024-09-16T00:25:56.599 INFO:tasks.workunit.client.1.smithi169.stdout:3/190: fsync d7/f9 0 2024-09-16T00:25:56.600 INFO:tasks.workunit.client.1.smithi169.stdout:8/162: write d7/dc/d2a/f30 [2679746,85743] 0 2024-09-16T00:25:56.602 INFO:tasks.workunit.client.1.smithi169.stdout:5/179: mknod d2/da/d10/d13/d1c/d2d/c3d 0 2024-09-16T00:25:56.603 INFO:tasks.workunit.client.1.smithi169.stdout:7/164: dread d7/da/fc [0,4194304] 0 2024-09-16T00:25:56.603 INFO:tasks.workunit.client.0.smithi112.stdout:1/277: sync 2024-09-16T00:25:56.603 INFO:tasks.workunit.client.0.smithi112.stdout:3/404: sync 2024-09-16T00:25:56.603 INFO:tasks.workunit.client.0.smithi112.stdout:4/293: sync 2024-09-16T00:25:56.605 INFO:tasks.workunit.client.1.smithi169.stdout:3/191: dread - d7/d1a/d1c/f40 zero size 2024-09-16T00:25:56.606 INFO:tasks.workunit.client.1.smithi169.stdout:4/166: dread d2/fb [0,4194304] 0 2024-09-16T00:25:56.608 INFO:tasks.workunit.client.1.smithi169.stdout:8/163: symlink d7/dc/d1d/l36 0 2024-09-16T00:25:56.609 INFO:tasks.workunit.client.0.smithi112.stdout:3/405: dread d2/f1a [4194304,4194304] 0 2024-09-16T00:25:56.610 INFO:tasks.workunit.client.0.smithi112.stdout:4/294: dread d6/d39/d1c/f2f [0,4194304] 0 2024-09-16T00:25:56.610 INFO:tasks.workunit.client.1.smithi169.stdout:5/180: getdents d2/d30 0 2024-09-16T00:25:56.611 INFO:tasks.workunit.client.0.smithi112.stdout:1/278: dwrite d0/d14/df/d28/f32 [0,4194304] 0 2024-09-16T00:25:56.611 INFO:tasks.workunit.client.0.smithi112.stdout:3/406: chown d2/d9/d2b/d2c/d69/f6f 11 1 2024-09-16T00:25:56.611 INFO:tasks.workunit.client.1.smithi169.stdout:7/165: symlink d7/da/d2e/l32 0 2024-09-16T00:25:56.613 INFO:tasks.workunit.client.0.smithi112.stdout:4/295: mkdir d6/d39/d1c/d2c/d69 0 2024-09-16T00:25:56.613 INFO:tasks.workunit.client.1.smithi169.stdout:2/148: dwrite d7/db/d20/d21/f33 [0,4194304] 0 2024-09-16T00:25:56.613 INFO:tasks.workunit.client.0.smithi112.stdout:1/279: mkdir d0/d14/df/d28/d45/d46/d58 0 2024-09-16T00:25:56.614 INFO:tasks.workunit.client.1.smithi169.stdout:3/192: symlink d7/d1a/d1c/d23/d2a/d37/l41 0 2024-09-16T00:25:56.615 INFO:tasks.workunit.client.0.smithi112.stdout:3/407: rename d2/d9/d2b/d47/d52/f40 to d2/d9/d2b/d2c/d69/f8d 0 2024-09-16T00:25:56.615 INFO:tasks.workunit.client.0.smithi112.stdout:4/296: chown d6/l58 5957 1 2024-09-16T00:25:56.617 INFO:tasks.workunit.client.0.smithi112.stdout:1/280: creat d0/d14/d1c/d26/d35/f59 x:0 0 0 2024-09-16T00:25:56.618 INFO:tasks.workunit.client.1.smithi169.stdout:7/166: mkdir d7/da/d29/d33 0 2024-09-16T00:25:56.618 INFO:tasks.workunit.client.0.smithi112.stdout:3/408: read d2/d9/d2b/d39/f67 [3384281,54467] 0 2024-09-16T00:25:56.619 INFO:tasks.workunit.client.0.smithi112.stdout:4/297: getdents d6/d39 0 2024-09-16T00:25:56.620 INFO:tasks.workunit.client.0.smithi112.stdout:1/281: getdents d0/d14/d1c/d20 0 2024-09-16T00:25:56.621 INFO:tasks.workunit.client.0.smithi112.stdout:1/282: readlink d0/d14/d1c/d20/l39 0 2024-09-16T00:25:56.622 INFO:tasks.workunit.client.1.smithi169.stdout:4/167: dwrite d2/d12/d13/f2e [0,4194304] 0 2024-09-16T00:25:56.622 INFO:tasks.workunit.client.0.smithi112.stdout:3/409: mkdir d2/d9/d2b/d47/d52/d7a/d80/d83/d76/d8e 0 2024-09-16T00:25:56.623 INFO:tasks.workunit.client.0.smithi112.stdout:4/298: unlink d6/db/d1f/d41/d49/f22 0 2024-09-16T00:25:56.624 INFO:tasks.workunit.client.0.smithi112.stdout:1/283: mknod d0/d14/d1c/c5a 0 2024-09-16T00:25:56.625 INFO:tasks.workunit.client.1.smithi169.stdout:2/149: symlink d7/db/d27/l34 0 2024-09-16T00:25:56.625 INFO:tasks.workunit.client.1.smithi169.stdout:4/168: stat d2/d12/d13/d27/d33 0 2024-09-16T00:25:56.626 INFO:tasks.workunit.client.0.smithi112.stdout:1/284: read d0/d14/f8 [6050265,29166] 0 2024-09-16T00:25:56.626 INFO:tasks.workunit.client.1.smithi169.stdout:8/164: dread d7/fd [4194304,4194304] 0 2024-09-16T00:25:56.626 INFO:tasks.workunit.client.1.smithi169.stdout:1/205: truncate dc/d18/f20 2060200 0 2024-09-16T00:25:56.627 INFO:tasks.workunit.client.0.smithi112.stdout:3/410: creat d2/d9/d2b/f8f x:0 0 0 2024-09-16T00:25:56.628 INFO:tasks.workunit.client.1.smithi169.stdout:7/167: mknod d7/c34 0 2024-09-16T00:25:56.628 INFO:tasks.workunit.client.0.smithi112.stdout:4/299: mkdir d6/db/df/d5e/d61/d64/d6a 0 2024-09-16T00:25:56.628 INFO:tasks.workunit.client.1.smithi169.stdout:2/150: read d7/db/f13 [1723281,59203] 0 2024-09-16T00:25:56.630 INFO:tasks.workunit.client.0.smithi112.stdout:4/300: write d6/db/f42 [3580110,73341] 0 2024-09-16T00:25:56.630 INFO:tasks.workunit.client.0.smithi112.stdout:3/411: chown d2/d9/d2b/d2c/f51 2 1 2024-09-16T00:25:56.630 INFO:tasks.workunit.client.1.smithi169.stdout:8/165: mkdir d7/d37 0 2024-09-16T00:25:56.631 INFO:tasks.workunit.client.0.smithi112.stdout:1/285: creat d0/d14/df/f5b x:0 0 0 2024-09-16T00:25:56.632 INFO:tasks.workunit.client.1.smithi169.stdout:7/168: fdatasync d7/df/f11 0 2024-09-16T00:25:56.632 INFO:tasks.workunit.client.0.smithi112.stdout:3/412: read - d2/d9/d2b/d2c/f51 zero size 2024-09-16T00:25:56.633 INFO:tasks.workunit.client.1.smithi169.stdout:1/206: creat dc/d26/f37 x:0 0 0 2024-09-16T00:25:56.634 INFO:tasks.workunit.client.0.smithi112.stdout:1/286: creat d0/d14/d1c/f5c x:0 0 0 2024-09-16T00:25:56.636 INFO:tasks.workunit.client.1.smithi169.stdout:2/151: mknod d7/db/d27/d29/c35 0 2024-09-16T00:25:56.636 INFO:tasks.workunit.client.0.smithi112.stdout:4/301: rename d6/da/d47/c4b to d6/d39/d1c/c6b 0 2024-09-16T00:25:56.636 INFO:tasks.workunit.client.1.smithi169.stdout:8/166: symlink d7/dc/d17/d24/d32/l38 0 2024-09-16T00:25:56.637 INFO:tasks.workunit.client.1.smithi169.stdout:5/181: dwrite d2/da/d10/d13/f35 [0,4194304] 0 2024-09-16T00:25:56.638 INFO:tasks.workunit.client.0.smithi112.stdout:3/413: mkdir d2/d9/d2b/d47/d52/d7a/d80/d83/d68/d90 0 2024-09-16T00:25:56.638 INFO:tasks.workunit.client.1.smithi169.stdout:1/207: creat dc/df/d29/d2a/f38 x:0 0 0 2024-09-16T00:25:56.639 INFO:tasks.workunit.client.0.smithi112.stdout:3/414: readlink d2/d9/d2b/d39/l44 0 2024-09-16T00:25:56.640 INFO:tasks.workunit.client.0.smithi112.stdout:3/415: chown d2/d9/d2b/f48 5542 1 2024-09-16T00:25:56.641 INFO:tasks.workunit.client.0.smithi112.stdout:4/302: rename d6/da/dc/c68 to d6/db/df/d5e/d61/c6c 0 2024-09-16T00:25:56.643 INFO:tasks.workunit.client.0.smithi112.stdout:1/287: getdents d0/d14/d1c/d26/d47 0 2024-09-16T00:25:56.644 INFO:tasks.workunit.client.1.smithi169.stdout:8/167: creat d7/dc/d17/f39 x:0 0 0 2024-09-16T00:25:56.644 INFO:tasks.workunit.client.0.smithi112.stdout:4/303: truncate d6/db/d1f/d41/d49/f17 1477194 0 2024-09-16T00:25:56.645 INFO:tasks.workunit.client.1.smithi169.stdout:4/169: dwrite d2/fb [0,4194304] 0 2024-09-16T00:25:56.645 INFO:tasks.workunit.client.0.smithi112.stdout:4/304: fsync d6/db/df/f5f 0 2024-09-16T00:25:56.646 INFO:tasks.workunit.client.1.smithi169.stdout:2/152: rename d7/db/d27/f2a to d7/d1c/d26/f36 0 2024-09-16T00:25:56.648 INFO:tasks.workunit.client.0.smithi112.stdout:3/416: rename d2/d9/d2b/d39/f67 to d2/f91 0 2024-09-16T00:25:56.650 INFO:tasks.workunit.client.1.smithi169.stdout:7/169: dwrite f1 [8388608,4194304] 0 2024-09-16T00:25:56.650 INFO:tasks.workunit.client.1.smithi169.stdout:8/168: symlink d7/dc/d26/l3a 0 2024-09-16T00:25:56.650 INFO:tasks.workunit.client.0.smithi112.stdout:1/288: truncate d0/d14/df/d28/d45/d30/f3a 507429 0 2024-09-16T00:25:56.650 INFO:tasks.workunit.client.0.smithi112.stdout:3/417: fsync d2/f1a 0 2024-09-16T00:25:56.652 INFO:tasks.workunit.client.0.smithi112.stdout:1/289: creat d0/d14/df/d28/f5d x:0 0 0 2024-09-16T00:25:56.652 INFO:tasks.workunit.client.1.smithi169.stdout:8/169: write f6 [6285593,21727] 0 2024-09-16T00:25:56.653 INFO:tasks.workunit.client.0.smithi112.stdout:3/418: mkdir d2/d9/d2b/d2c/d69/d92 0 2024-09-16T00:25:56.653 INFO:tasks.workunit.client.1.smithi169.stdout:3/193: dwrite d7/d1a/d1c/d23/f2f [0,4194304] 0 2024-09-16T00:25:56.654 INFO:tasks.workunit.client.1.smithi169.stdout:4/170: symlink d2/dd/d1a/l38 0 2024-09-16T00:25:56.654 INFO:tasks.workunit.client.0.smithi112.stdout:1/290: mkdir d0/d14/d5e 0 2024-09-16T00:25:56.656 INFO:tasks.workunit.client.0.smithi112.stdout:3/419: creat d2/d9/d2b/d47/d52/d7a/d80/f93 x:0 0 0 2024-09-16T00:25:56.657 INFO:tasks.workunit.client.0.smithi112.stdout:1/291: rmdir d0/d14/df/d28/d45/d46 39 2024-09-16T00:25:56.658 INFO:tasks.workunit.client.1.smithi169.stdout:8/170: chown d7/dc/l15 61067086 1 2024-09-16T00:25:56.658 INFO:tasks.workunit.client.1.smithi169.stdout:4/171: read - d2/d12/d13/d27/f32 zero size 2024-09-16T00:25:56.661 INFO:tasks.workunit.client.1.smithi169.stdout:3/194: symlink d7/d1a/d1c/d23/l42 0 2024-09-16T00:25:56.661 INFO:tasks.workunit.client.0.smithi112.stdout:3/420: creat d2/d9/f94 x:0 0 0 2024-09-16T00:25:56.662 INFO:tasks.workunit.client.0.smithi112.stdout:1/292: dread d0/d14/df/d28/f2d [0,4194304] 0 2024-09-16T00:25:56.664 INFO:tasks.workunit.client.0.smithi112.stdout:1/293: write d0/d14/f11 [8191528,2873] 0 2024-09-16T00:25:56.664 INFO:tasks.workunit.client.1.smithi169.stdout:3/195: write d7/d1a/f1b [42318,71843] 0 2024-09-16T00:25:56.666 INFO:tasks.workunit.client.0.smithi112.stdout:3/421: getdents d2/d9/d2b/d2c/d69/d92 0 2024-09-16T00:25:56.667 INFO:tasks.workunit.client.0.smithi112.stdout:1/294: mknod d0/d14/d1c/c5f 0 2024-09-16T00:25:56.667 INFO:tasks.workunit.client.1.smithi169.stdout:4/172: link d2/d12/d13/f25 d2/d12/f39 0 2024-09-16T00:25:56.668 INFO:tasks.workunit.client.0.smithi112.stdout:3/422: chown d2/d9/d2b/d47/d52/d7a/d80/d83/d76 771792702 1 2024-09-16T00:25:56.668 INFO:tasks.workunit.client.0.smithi112.stdout:1/295: write d0/d14/d1c/d26/d35/f59 [226843,1914] 0 2024-09-16T00:25:56.669 INFO:tasks.workunit.client.0.smithi112.stdout:3/423: fdatasync d2/d9/d2b/d39/f85 0 2024-09-16T00:25:56.669 INFO:tasks.workunit.client.1.smithi169.stdout:4/173: chown d2/dd/d1a/l38 60769996 1 2024-09-16T00:25:56.670 INFO:tasks.workunit.client.1.smithi169.stdout:3/196: rename d7/d1a/l21 to d7/d1a/d1c/l43 0 2024-09-16T00:25:56.670 INFO:tasks.workunit.client.0.smithi112.stdout:3/424: stat d2/c4a 0 2024-09-16T00:25:56.671 INFO:tasks.workunit.client.0.smithi112.stdout:1/296: write d0/d14/df/d28/d45/d30/f4a [180912,79515] 0 2024-09-16T00:25:56.671 INFO:tasks.workunit.client.1.smithi169.stdout:4/174: creat d2/d12/d13/f3a x:0 0 0 2024-09-16T00:25:56.672 INFO:tasks.workunit.client.1.smithi169.stdout:3/197: symlink d7/l44 0 2024-09-16T00:25:56.673 INFO:tasks.workunit.client.0.smithi112.stdout:3/425: mkdir d2/d9/d2b/d47/d52/d7a/d80/d95 0 2024-09-16T00:25:56.677 INFO:tasks.workunit.client.0.smithi112.stdout:3/426: rmdir d2/d9/d8c 0 2024-09-16T00:25:56.680 INFO:tasks.workunit.client.0.smithi112.stdout:3/427: creat d2/d9/d2b/d2c/d69/d92/f96 x:0 0 0 2024-09-16T00:25:56.681 INFO:tasks.workunit.client.1.smithi169.stdout:5/182: truncate d2/da/d10/f27 1618920 0 2024-09-16T00:25:56.684 INFO:tasks.workunit.client.1.smithi169.stdout:7/170: truncate f1 6319708 0 2024-09-16T00:25:56.685 INFO:tasks.workunit.client.1.smithi169.stdout:3/198: dwrite d7/d1a/d1c/d23/f3a [0,4194304] 0 2024-09-16T00:25:56.686 INFO:tasks.workunit.client.1.smithi169.stdout:8/171: truncate f6 8686347 0 2024-09-16T00:25:56.686 INFO:tasks.workunit.client.1.smithi169.stdout:7/171: read d7/f17 [3614265,113913] 0 2024-09-16T00:25:56.687 INFO:tasks.workunit.client.0.smithi112.stdout:3/428: dwrite d2/f91 [0,4194304] 0 2024-09-16T00:25:56.693 INFO:tasks.workunit.client.1.smithi169.stdout:7/172: rename d7/f8 to d7/da/d2e/f35 0 2024-09-16T00:25:56.694 INFO:tasks.workunit.client.1.smithi169.stdout:3/199: dread d7/d1a/f32 [4194304,4194304] 0 2024-09-16T00:25:56.697 INFO:tasks.workunit.client.1.smithi169.stdout:3/200: chown d7/d1a/d1c/d23/f2f 67 1 2024-09-16T00:25:56.697 INFO:tasks.workunit.client.1.smithi169.stdout:4/175: dwrite d2/d12/d13/f25 [0,4194304] 0 2024-09-16T00:25:56.699 INFO:tasks.workunit.client.1.smithi169.stdout:7/173: truncate d7/da/d2e/f35 8407187 0 2024-09-16T00:25:56.700 INFO:tasks.workunit.client.0.smithi112.stdout:3/429: truncate d2/d9/f1c 2852110 0 2024-09-16T00:25:56.700 INFO:tasks.workunit.client.1.smithi169.stdout:3/201: creat d7/d1a/d2b/f45 x:0 0 0 2024-09-16T00:25:56.701 INFO:tasks.workunit.client.1.smithi169.stdout:3/202: read - d7/d1a/d1c/f25 zero size 2024-09-16T00:25:56.702 INFO:tasks.workunit.client.0.smithi112.stdout:3/430: creat d2/f97 x:0 0 0 2024-09-16T00:25:56.704 INFO:tasks.workunit.client.1.smithi169.stdout:7/174: unlink d7/df/c1e 0 2024-09-16T00:25:56.705 INFO:tasks.workunit.client.1.smithi169.stdout:4/176: rmdir d2/d12/d13/d27 39 2024-09-16T00:25:56.706 INFO:tasks.workunit.client.0.smithi112.stdout:3/431: rename d2/d9/d2b/d47/d52/l5a to d2/d9/d2b/d47/d52/d7a/d80/l98 0 2024-09-16T00:25:56.706 INFO:tasks.workunit.client.1.smithi169.stdout:4/177: read - d2/d12/f31 zero size 2024-09-16T00:25:56.706 INFO:tasks.workunit.client.1.smithi169.stdout:4/178: chown d2/l7 109 1 2024-09-16T00:25:56.707 INFO:tasks.workunit.client.1.smithi169.stdout:3/203: creat d7/d1a/d2b/f46 x:0 0 0 2024-09-16T00:25:56.709 INFO:tasks.workunit.client.0.smithi112.stdout:3/432: creat d2/d9/d2b/d47/d52/d7a/d80/d83/d76/f99 x:0 0 0 2024-09-16T00:25:56.709 INFO:tasks.workunit.client.1.smithi169.stdout:7/175: rename d7/da/f2a to d7/da/f36 0 2024-09-16T00:25:56.710 INFO:tasks.workunit.client.1.smithi169.stdout:3/204: symlink d7/d1a/d1c/d23/d2a/d37/l47 0 2024-09-16T00:25:56.712 INFO:tasks.workunit.client.1.smithi169.stdout:4/179: symlink d2/dd/l3b 0 2024-09-16T00:25:56.712 INFO:tasks.workunit.client.0.smithi112.stdout:3/433: link d2/d9/d2b/f4b d2/d9/d2b/f9a 0 2024-09-16T00:25:56.713 INFO:tasks.workunit.client.0.smithi112.stdout:3/434: write d2/d9/d2b/d2c/f42 [7814291,230] 0 2024-09-16T00:25:56.714 INFO:tasks.workunit.client.1.smithi169.stdout:3/205: symlink d7/d1a/l48 0 2024-09-16T00:25:56.715 INFO:tasks.workunit.client.1.smithi169.stdout:7/176: dread f5 [0,4194304] 0 2024-09-16T00:25:56.716 INFO:tasks.workunit.client.1.smithi169.stdout:3/206: dread - d7/d1a/d2b/f46 zero size 2024-09-16T00:25:56.718 INFO:tasks.workunit.client.0.smithi112.stdout:1/297: dread d0/f9 [0,4194304] 0 2024-09-16T00:25:56.718 INFO:tasks.workunit.client.1.smithi169.stdout:7/177: write d7/f17 [5781588,18730] 0 2024-09-16T00:25:56.719 INFO:tasks.workunit.client.0.smithi112.stdout:3/435: dwrite d2/d9/d2b/d39/f3c [0,4194304] 0 2024-09-16T00:25:56.719 INFO:tasks.workunit.client.0.smithi112.stdout:1/298: fdatasync d0/d14/d1c/d26/d35/f59 0 2024-09-16T00:25:56.720 INFO:tasks.workunit.client.0.smithi112.stdout:1/299: write d0/d14/fa [759884,27433] 0 2024-09-16T00:25:56.722 INFO:tasks.workunit.client.1.smithi169.stdout:4/180: dwrite d2/d12/f1e [4194304,4194304] 0 2024-09-16T00:25:56.722 INFO:tasks.workunit.client.0.smithi112.stdout:3/436: mknod d2/d9/d2b/d2c/d69/c9b 0 2024-09-16T00:25:56.723 INFO:tasks.workunit.client.0.smithi112.stdout:3/437: dread - d2/d9/d2b/d47/f8a zero size 2024-09-16T00:25:56.723 INFO:tasks.workunit.client.0.smithi112.stdout:1/300: mknod d0/d14/df/c60 0 2024-09-16T00:25:56.724 INFO:tasks.workunit.client.1.smithi169.stdout:7/178: unlink d7/d16/l22 0 2024-09-16T00:25:56.725 INFO:tasks.workunit.client.1.smithi169.stdout:3/207: creat d7/d1a/d1c/d35/f49 x:0 0 0 2024-09-16T00:25:56.726 INFO:tasks.workunit.client.0.smithi112.stdout:7/384: sync 2024-09-16T00:25:56.727 INFO:tasks.workunit.client.1.smithi169.stdout:3/208: chown d7/d1a/c38 1 1 2024-09-16T00:25:56.727 INFO:tasks.workunit.client.0.smithi112.stdout:7/385: write d6/d10/d16/f59 [324266,119644] 0 2024-09-16T00:25:56.728 INFO:tasks.workunit.client.0.smithi112.stdout:3/438: rename d2/d4d/f60 to d2/d9/d2b/d47/d52/d7a/d80/d83/d49/f9c 0 2024-09-16T00:25:56.728 INFO:tasks.workunit.client.1.smithi169.stdout:4/181: rmdir d2/dd 39 2024-09-16T00:25:56.729 INFO:tasks.workunit.client.0.smithi112.stdout:7/386: stat d6/d10/d1b 0 2024-09-16T00:25:56.730 INFO:tasks.workunit.client.1.smithi169.stdout:7/179: creat d7/da/f37 x:0 0 0 2024-09-16T00:25:56.730 INFO:tasks.workunit.client.0.smithi112.stdout:7/387: stat d6/d10/d1b 0 2024-09-16T00:25:56.730 INFO:tasks.workunit.client.0.smithi112.stdout:3/439: dread - d2/d9/d2b/d2c/d69/f72 zero size 2024-09-16T00:25:56.731 INFO:tasks.workunit.client.0.smithi112.stdout:1/301: dwrite d0/d14/f11 [4194304,4194304] 0 2024-09-16T00:25:56.732 INFO:tasks.workunit.client.1.smithi169.stdout:4/182: write d2/f3 [424421,34988] 0 2024-09-16T00:25:56.732 INFO:tasks.workunit.client.0.smithi112.stdout:3/440: dread - d2/f97 zero size 2024-09-16T00:25:56.733 INFO:tasks.workunit.client.1.smithi169.stdout:3/209: dread d7/fd [4194304,4194304] 0 2024-09-16T00:25:56.733 INFO:tasks.workunit.client.1.smithi169.stdout:4/183: readlink d2/d12/d13/l2a 0 2024-09-16T00:25:56.740 INFO:tasks.workunit.client.1.smithi169.stdout:7/180: dwrite d7/da/d2e/f35 [4194304,4194304] 0 2024-09-16T00:25:56.741 INFO:tasks.workunit.client.1.smithi169.stdout:4/184: getdents d2/dd/d1a 0 2024-09-16T00:25:56.742 INFO:tasks.workunit.client.0.smithi112.stdout:1/302: mknod d0/d14/df/d28/d45/d46/c61 0 2024-09-16T00:25:56.743 INFO:tasks.workunit.client.0.smithi112.stdout:7/388: dwrite d6/d10/d1b/f7d [0,4194304] 0 2024-09-16T00:25:56.744 INFO:tasks.workunit.client.1.smithi169.stdout:4/185: write d2/d12/f39 [5203281,22872] 0 2024-09-16T00:25:56.745 INFO:tasks.workunit.client.1.smithi169.stdout:7/181: write f1 [1471139,45590] 0 2024-09-16T00:25:56.745 INFO:tasks.workunit.client.0.smithi112.stdout:7/389: fdatasync d6/d7f/f89 0 2024-09-16T00:25:56.747 INFO:tasks.workunit.client.0.smithi112.stdout:1/303: fdatasync d0/d14/f22 0 2024-09-16T00:25:56.747 INFO:tasks.workunit.client.0.smithi112.stdout:1/304: fsync d0/d14/df/f5b 0 2024-09-16T00:25:56.747 INFO:tasks.workunit.client.0.smithi112.stdout:7/390: truncate d6/d10/d16/d49/d6c/f5b 764699 0 2024-09-16T00:25:56.749 INFO:tasks.workunit.client.1.smithi169.stdout:3/210: dwrite d7/d1a/d1c/d23/f3c [0,4194304] 0 2024-09-16T00:25:56.749 INFO:tasks.workunit.client.1.smithi169.stdout:7/182: unlink d7/c13 0 2024-09-16T00:25:56.750 INFO:tasks.workunit.client.1.smithi169.stdout:7/183: dread - d7/f2c zero size 2024-09-16T00:25:56.751 INFO:tasks.workunit.client.0.smithi112.stdout:1/305: unlink d0/d14/fa 0 2024-09-16T00:25:56.751 INFO:tasks.workunit.client.0.smithi112.stdout:7/391: dread d6/f22 [0,4194304] 0 2024-09-16T00:25:56.752 INFO:tasks.workunit.client.0.smithi112.stdout:1/306: write d0/d14/df/d28/f2e [879194,16820] 0 2024-09-16T00:25:56.753 INFO:tasks.workunit.client.0.smithi112.stdout:7/392: read d6/d10/f12 [3021141,90547] 0 2024-09-16T00:25:56.753 INFO:tasks.workunit.client.1.smithi169.stdout:7/184: dread d7/da/f23 [0,4194304] 0 2024-09-16T00:25:56.754 INFO:tasks.workunit.client.0.smithi112.stdout:1/307: chown d0/d14/df/f4b 21602785 1 2024-09-16T00:25:56.757 INFO:tasks.workunit.client.0.smithi112.stdout:7/393: unlink d6/d10/d16/c51 0 2024-09-16T00:25:56.757 INFO:tasks.workunit.client.0.smithi112.stdout:7/394: readlink d6/lf 0 2024-09-16T00:25:56.757 INFO:tasks.workunit.client.0.smithi112.stdout:1/308: mkdir d0/d14/df/d28/d45/d46/d58/d62 0 2024-09-16T00:25:56.758 INFO:tasks.workunit.client.1.smithi169.stdout:7/185: creat d7/da/d2e/f38 x:0 0 0 2024-09-16T00:25:56.760 INFO:tasks.workunit.client.1.smithi169.stdout:7/186: creat d7/d16/f39 x:0 0 0 2024-09-16T00:25:56.760 INFO:tasks.workunit.client.0.smithi112.stdout:7/395: mkdir d6/d10/d75/d8f 0 2024-09-16T00:25:56.761 INFO:tasks.workunit.client.0.smithi112.stdout:1/309: unlink d0/f1b 0 2024-09-16T00:25:56.761 INFO:tasks.workunit.client.1.smithi169.stdout:7/187: dread d7/da/f23 [0,4194304] 0 2024-09-16T00:25:56.762 INFO:tasks.workunit.client.0.smithi112.stdout:1/310: chown d0/d14/df/d28/d45/d46 725449 1 2024-09-16T00:25:56.763 INFO:tasks.workunit.client.0.smithi112.stdout:1/311: write d0/f18 [1028121,36254] 0 2024-09-16T00:25:56.764 INFO:tasks.workunit.client.0.smithi112.stdout:1/312: fsync d0/d14/df/f36 0 2024-09-16T00:25:56.764 INFO:tasks.workunit.client.1.smithi169.stdout:7/188: unlink d7/df/f15 0 2024-09-16T00:25:56.765 INFO:tasks.workunit.client.0.smithi112.stdout:1/313: readlink d0/d14/d1c/d26/d41/l55 0 2024-09-16T00:25:56.767 INFO:tasks.workunit.client.0.smithi112.stdout:1/314: creat d0/d14/d1c/d20/f63 x:0 0 0 2024-09-16T00:25:56.769 INFO:tasks.workunit.client.0.smithi112.stdout:1/315: symlink d0/d14/df/d28/d45/d46/d58/d62/l64 0 2024-09-16T00:25:56.770 INFO:tasks.workunit.client.1.smithi169.stdout:7/189: dwrite f5 [0,4194304] 0 2024-09-16T00:25:56.771 INFO:tasks.workunit.client.1.smithi169.stdout:7/190: chown d7/d16/f25 193 1 2024-09-16T00:25:56.778 INFO:tasks.workunit.client.1.smithi169.stdout:3/211: rmdir d7/d1a/d2b 39 2024-09-16T00:25:56.781 INFO:tasks.workunit.client.1.smithi169.stdout:3/212: creat d7/d1a/d1c/d23/d2a/f4a x:0 0 0 2024-09-16T00:25:56.782 INFO:tasks.workunit.client.0.smithi112.stdout:1/316: getdents d0/d14/df/d28/d45/d46 0 2024-09-16T00:25:56.783 INFO:tasks.workunit.client.1.smithi169.stdout:3/213: fdatasync d7/d1a/d1c/f25 0 2024-09-16T00:25:56.783 INFO:tasks.workunit.client.1.smithi169.stdout:3/214: readlink d7/l2d 0 2024-09-16T00:25:56.784 INFO:tasks.workunit.client.0.smithi112.stdout:1/317: creat d0/d14/d1c/d20/f65 x:0 0 0 2024-09-16T00:25:56.786 INFO:tasks.workunit.client.0.smithi112.stdout:1/318: write d0/f48 [34062,123202] 0 2024-09-16T00:25:56.787 INFO:tasks.workunit.client.0.smithi112.stdout:1/319: chown d0/d14/d1c/d20/l54 862881761 1 2024-09-16T00:25:56.787 INFO:tasks.workunit.client.1.smithi169.stdout:7/191: getdents d7 0 2024-09-16T00:25:56.788 INFO:tasks.workunit.client.1.smithi169.stdout:4/186: truncate d2/d12/f1e 3141628 0 2024-09-16T00:25:56.788 INFO:tasks.workunit.client.0.smithi112.stdout:1/320: write d0/d14/f22 [3665839,14244] 0 2024-09-16T00:25:56.789 INFO:tasks.workunit.client.0.smithi112.stdout:1/321: chown d0/d14/d1c/d26/d35/c3b 8 1 2024-09-16T00:25:56.790 INFO:tasks.workunit.client.0.smithi112.stdout:1/322: fdatasync d0/d14/f23 0 2024-09-16T00:25:56.790 INFO:tasks.workunit.client.1.smithi169.stdout:4/187: write d2/dd/f26 [205472,113017] 0 2024-09-16T00:25:56.792 INFO:tasks.workunit.client.0.smithi112.stdout:1/323: readlink d0/d14/d1c/d26/d41/l55 0 2024-09-16T00:25:56.792 INFO:tasks.workunit.client.1.smithi169.stdout:3/215: link d7/d1a/l2c d7/d1a/d1c/d23/d2a/d37/l4b 0 2024-09-16T00:25:56.795 INFO:tasks.workunit.client.0.smithi112.stdout:1/324: write d0/d14/d1c/d26/f52 [35674,85334] 0 2024-09-16T00:25:56.795 INFO:tasks.workunit.client.0.smithi112.stdout:7/396: dwrite d6/f22 [0,4194304] 0 2024-09-16T00:25:56.798 INFO:tasks.workunit.client.1.smithi169.stdout:3/216: readlink d7/d1a/d1c/l43 0 2024-09-16T00:25:56.798 INFO:tasks.workunit.client.1.smithi169.stdout:7/192: unlink d7/da/d2e/f38 0 2024-09-16T00:25:56.799 INFO:tasks.workunit.client.0.smithi112.stdout:1/325: mknod d0/d14/d44/c66 0 2024-09-16T00:25:56.800 INFO:tasks.workunit.client.1.smithi169.stdout:4/188: rename d2/d12/d13/l15 to d2/d12/d13/d1f/d2f/l3c 0 2024-09-16T00:25:56.801 INFO:tasks.workunit.client.1.smithi169.stdout:7/193: read d7/df/f2b [128872,127591] 0 2024-09-16T00:25:56.801 INFO:tasks.workunit.client.0.smithi112.stdout:7/397: mkdir d6/d10/d75/d8f/d90 0 2024-09-16T00:25:56.805 INFO:tasks.workunit.client.1.smithi169.stdout:3/217: rename d7/d1a/f30 to d7/d1a/d1c/d23/f4c 0 2024-09-16T00:25:56.807 INFO:tasks.workunit.client.0.smithi112.stdout:1/326: dwrite d0/d14/d1c/d20/f65 [0,4194304] 0 2024-09-16T00:25:56.807 INFO:tasks.workunit.client.0.smithi112.stdout:7/398: link d6/d10/f6e d6/d10/d75/d88/f91 0 2024-09-16T00:25:56.808 INFO:tasks.workunit.client.1.smithi169.stdout:4/189: dread d2/dd/f24 [0,4194304] 0 2024-09-16T00:25:56.809 INFO:tasks.workunit.client.1.smithi169.stdout:7/194: dread d7/da/fc [4194304,4194304] 0 2024-09-16T00:25:56.810 INFO:tasks.workunit.client.0.smithi112.stdout:7/399: creat d6/d10/d1b/d39/f92 x:0 0 0 2024-09-16T00:25:56.810 INFO:tasks.workunit.client.1.smithi169.stdout:3/218: write d7/d1a/d1c/d23/f2f [3231524,117718] 0 2024-09-16T00:25:56.812 INFO:tasks.workunit.client.0.smithi112.stdout:7/400: dread - d6/d10/d1b/d39/f92 zero size 2024-09-16T00:25:56.812 INFO:tasks.workunit.client.1.smithi169.stdout:7/195: mkdir d7/da/d3a 0 2024-09-16T00:25:56.814 INFO:tasks.workunit.client.0.smithi112.stdout:1/327: dwrite d0/d14/fd [4194304,4194304] 0 2024-09-16T00:25:56.814 INFO:tasks.workunit.client.1.smithi169.stdout:3/219: symlink d7/d1a/d1c/l4d 0 2024-09-16T00:25:56.815 INFO:tasks.workunit.client.0.smithi112.stdout:7/401: mknod d6/d10/d16/d49/d6c/c93 0 2024-09-16T00:25:56.817 INFO:tasks.workunit.client.1.smithi169.stdout:4/190: dread d2/fb [0,4194304] 0 2024-09-16T00:25:56.818 INFO:tasks.workunit.client.0.smithi112.stdout:1/328: link d0/f18 d0/f67 0 2024-09-16T00:25:56.820 INFO:tasks.workunit.client.0.smithi112.stdout:1/329: mknod d0/d14/d1c/d20/c68 0 2024-09-16T00:25:56.821 INFO:tasks.workunit.client.0.smithi112.stdout:7/402: dwrite d6/d10/f12 [4194304,4194304] 0 2024-09-16T00:25:56.822 INFO:tasks.workunit.client.1.smithi169.stdout:7/196: dwrite d7/f31 [0,4194304] 0 2024-09-16T00:25:56.822 INFO:tasks.workunit.client.1.smithi169.stdout:4/191: chown d2/d12/d13/d27/d33 369 1 2024-09-16T00:25:56.823 INFO:tasks.workunit.client.0.smithi112.stdout:1/330: creat d0/d14/d1c/d26/f69 x:0 0 0 2024-09-16T00:25:56.824 INFO:tasks.workunit.client.1.smithi169.stdout:4/192: chown d2/d12/d13/c1d 26713 1 2024-09-16T00:25:56.824 INFO:tasks.workunit.client.1.smithi169.stdout:7/197: fdatasync d7/da/f23 0 2024-09-16T00:25:56.824 INFO:tasks.workunit.client.0.smithi112.stdout:7/403: mknod d6/d10/d1b/c94 0 2024-09-16T00:25:56.825 INFO:tasks.workunit.client.0.smithi112.stdout:1/331: truncate d0/d14/d1c/d20/f33 1026674 0 2024-09-16T00:25:56.828 INFO:tasks.workunit.client.0.smithi112.stdout:7/404: link d6/d10/d16/f4e d6/d10/d16/d49/d6c/f95 0 2024-09-16T00:25:56.828 INFO:tasks.workunit.client.1.smithi169.stdout:7/198: symlink d7/da/d29/l3b 0 2024-09-16T00:25:56.831 INFO:tasks.workunit.client.1.smithi169.stdout:4/193: mkdir d2/d12/d13/d3d 0 2024-09-16T00:25:56.831 INFO:tasks.workunit.client.1.smithi169.stdout:4/194: chown d2/d12/l14 412 1 2024-09-16T00:25:56.832 INFO:tasks.workunit.client.0.smithi112.stdout:8/250: sync 2024-09-16T00:25:56.832 INFO:tasks.workunit.client.0.smithi112.stdout:1/332: dwrite d0/f27 [0,4194304] 0 2024-09-16T00:25:56.833 INFO:tasks.workunit.client.0.smithi112.stdout:1/333: chown d0/ce 7281 1 2024-09-16T00:25:56.835 INFO:tasks.workunit.client.0.smithi112.stdout:8/251: chown d10/d12/c2d 1 1 2024-09-16T00:25:56.836 INFO:tasks.workunit.client.0.smithi112.stdout:1/334: mkdir d0/d14/d6a 0 2024-09-16T00:25:56.840 INFO:tasks.workunit.client.0.smithi112.stdout:1/335: mkdir d0/d14/df/d28/d45/d46/d6b 0 2024-09-16T00:25:56.841 INFO:tasks.workunit.client.0.smithi112.stdout:1/336: fdatasync d0/d14/f8 0 2024-09-16T00:25:56.842 INFO:tasks.workunit.client.0.smithi112.stdout:8/252: dwrite d10/d12/d1c/f38 [0,4194304] 0 2024-09-16T00:25:56.842 INFO:tasks.workunit.client.0.smithi112.stdout:1/337: chown d0/ce 1927 1 2024-09-16T00:25:56.844 INFO:tasks.workunit.client.0.smithi112.stdout:1/338: write d0/d14/d1c/f5c [376996,80352] 0 2024-09-16T00:25:56.845 INFO:tasks.workunit.client.0.smithi112.stdout:8/253: creat d10/d12/d1c/d3b/f53 x:0 0 0 2024-09-16T00:25:56.845 INFO:tasks.workunit.client.0.smithi112.stdout:1/339: dread - d0/d14/d1c/d20/f31 zero size 2024-09-16T00:25:56.847 INFO:tasks.workunit.client.0.smithi112.stdout:7/405: dread d6/d10/f3d [0,4194304] 0 2024-09-16T00:25:56.847 INFO:tasks.workunit.client.0.smithi112.stdout:8/254: truncate d10/f21 2868658 0 2024-09-16T00:25:56.848 INFO:tasks.workunit.client.0.smithi112.stdout:1/340: fsync d0/d14/df/d28/f2d 0 2024-09-16T00:25:56.849 INFO:tasks.workunit.client.0.smithi112.stdout:8/255: creat d10/d12/d1c/f54 x:0 0 0 2024-09-16T00:25:56.850 INFO:tasks.workunit.client.0.smithi112.stdout:8/256: truncate d10/f34 1521261 0 2024-09-16T00:25:56.853 INFO:tasks.workunit.client.1.smithi169.stdout:7/199: truncate d7/da/f23 769031 0 2024-09-16T00:25:56.854 INFO:tasks.workunit.client.1.smithi169.stdout:7/200: write d7/df/f21 [46115,109992] 0 2024-09-16T00:25:56.854 INFO:tasks.workunit.client.1.smithi169.stdout:4/195: mkdir d2/d12/d13/d1f/d3e 0 2024-09-16T00:25:56.855 INFO:tasks.workunit.client.0.smithi112.stdout:8/257: creat d10/f55 x:0 0 0 2024-09-16T00:25:56.855 INFO:tasks.workunit.client.0.smithi112.stdout:8/258: stat d10/d12/d24/f2e 0 2024-09-16T00:25:56.856 INFO:tasks.workunit.client.1.smithi169.stdout:4/196: dread - d2/dd/d1a/f1b zero size 2024-09-16T00:25:56.856 INFO:tasks.workunit.client.1.smithi169.stdout:4/197: stat d2/fb 0 2024-09-16T00:25:56.856 INFO:tasks.workunit.client.1.smithi169.stdout:7/201: write d7/d16/f25 [441808,107541] 0 2024-09-16T00:25:56.857 INFO:tasks.workunit.client.1.smithi169.stdout:7/202: fsync f5 0 2024-09-16T00:25:56.858 INFO:tasks.workunit.client.0.smithi112.stdout:8/259: mknod d10/d12/d24/c56 0 2024-09-16T00:25:56.858 INFO:tasks.workunit.client.1.smithi169.stdout:4/198: chown d2/d12/d13/f2e 1567683 1 2024-09-16T00:25:56.858 INFO:tasks.workunit.client.1.smithi169.stdout:7/203: chown d7/da/d3a 62261042 1 2024-09-16T00:25:56.859 INFO:tasks.workunit.client.1.smithi169.stdout:7/204: fdatasync f1 0 2024-09-16T00:25:56.859 INFO:tasks.workunit.client.1.smithi169.stdout:4/199: write d2/dd/f26 [609816,7389] 0 2024-09-16T00:25:56.864 INFO:tasks.workunit.client.0.smithi112.stdout:8/260: link d10/d12/d1c/f31 d10/d45/f57 0 2024-09-16T00:25:56.866 INFO:tasks.workunit.client.1.smithi169.stdout:7/205: dread d7/f17 [0,4194304] 0 2024-09-16T00:25:56.866 INFO:tasks.workunit.client.0.smithi112.stdout:8/261: dread d10/f1b [0,4194304] 0 2024-09-16T00:25:56.868 INFO:tasks.workunit.client.0.smithi112.stdout:8/262: mkdir d10/d4e/d58 0 2024-09-16T00:25:56.869 INFO:tasks.workunit.client.1.smithi169.stdout:4/200: dwrite d2/d12/d13/f25 [4194304,4194304] 0 2024-09-16T00:25:56.872 INFO:tasks.workunit.client.0.smithi112.stdout:8/263: link d10/d45/f57 d10/d12/d1c/f59 0 2024-09-16T00:25:56.872 INFO:tasks.workunit.client.1.smithi169.stdout:7/206: creat d7/da/d3a/f3c x:0 0 0 2024-09-16T00:25:56.873 INFO:tasks.workunit.client.1.smithi169.stdout:7/207: dread - d7/d16/f39 zero size 2024-09-16T00:25:56.876 INFO:tasks.workunit.client.1.smithi169.stdout:7/208: symlink d7/da/d3a/l3d 0 2024-09-16T00:25:56.880 INFO:tasks.workunit.client.1.smithi169.stdout:4/201: rename d2/d12/d13/c2d to d2/d12/c3f 0 2024-09-16T00:25:56.881 INFO:tasks.workunit.client.1.smithi169.stdout:4/202: write d2/f9 [2498617,77013] 0 2024-09-16T00:25:56.881 INFO:tasks.workunit.client.0.smithi112.stdout:8/264: dwrite d10/d45/f52 [0,4194304] 0 2024-09-16T00:25:56.884 INFO:tasks.workunit.client.0.smithi112.stdout:8/265: mknod d10/d4e/d58/c5a 0 2024-09-16T00:25:56.884 INFO:tasks.workunit.client.0.smithi112.stdout:8/266: dread - d10/f46 zero size 2024-09-16T00:25:56.886 INFO:tasks.workunit.client.1.smithi169.stdout:7/209: rename d7/c12 to d7/df/c3e 0 2024-09-16T00:25:56.896 INFO:tasks.workunit.client.0.smithi112.stdout:1/341: truncate d0/d14/d1c/d20/f33 679422 0 2024-09-16T00:25:56.899 INFO:tasks.workunit.client.0.smithi112.stdout:1/342: mknod d0/d14/d1c/d26/d47/c6c 0 2024-09-16T00:25:56.902 INFO:tasks.workunit.client.0.smithi112.stdout:8/267: write d10/f21 [174212,130067] 0 2024-09-16T00:25:56.902 INFO:tasks.workunit.client.1.smithi169.stdout:4/203: truncate d2/dd/f24 235441 0 2024-09-16T00:25:56.902 INFO:tasks.workunit.client.0.smithi112.stdout:8/268: fdatasync d10/d12/d24/f2b 0 2024-09-16T00:25:56.903 INFO:tasks.workunit.client.1.smithi169.stdout:7/210: write d7/d16/f39 [344793,102796] 0 2024-09-16T00:25:56.903 INFO:tasks.workunit.client.1.smithi169.stdout:4/204: creat d2/dd/d1a/f40 x:0 0 0 2024-09-16T00:25:56.904 INFO:tasks.workunit.client.0.smithi112.stdout:8/269: symlink d10/d12/l5b 0 2024-09-16T00:25:56.904 INFO:tasks.workunit.client.1.smithi169.stdout:4/205: fdatasync d2/dd/f26 0 2024-09-16T00:25:56.905 INFO:tasks.workunit.client.0.smithi112.stdout:8/270: write d10/d12/f16 [3353750,11215] 0 2024-09-16T00:25:56.906 INFO:tasks.workunit.client.0.smithi112.stdout:8/271: mknod d10/d12/d1c/c5c 0 2024-09-16T00:25:56.908 INFO:tasks.workunit.client.0.smithi112.stdout:8/272: truncate d10/d12/d24/f29 665309 0 2024-09-16T00:25:56.910 INFO:tasks.workunit.client.1.smithi169.stdout:7/211: dwrite d7/d16/f39 [0,4194304] 0 2024-09-16T00:25:56.911 INFO:tasks.workunit.client.1.smithi169.stdout:4/206: rmdir d2/dd/d1a 39 2024-09-16T00:25:56.911 INFO:tasks.workunit.client.0.smithi112.stdout:8/273: getdents d10/d45 0 2024-09-16T00:25:56.913 INFO:tasks.workunit.client.0.smithi112.stdout:8/274: chown d10/d12/d24/c3a 993890903 1 2024-09-16T00:25:56.919 INFO:tasks.workunit.client.0.smithi112.stdout:8/275: dwrite d10/d45/f3f [0,4194304] 0 2024-09-16T00:25:56.919 INFO:tasks.workunit.client.1.smithi169.stdout:7/212: dread d7/da/fc [4194304,4194304] 0 2024-09-16T00:25:56.920 INFO:tasks.workunit.client.1.smithi169.stdout:7/213: getdents d7/da/d29/d33 0 2024-09-16T00:25:56.921 INFO:tasks.workunit.client.0.smithi112.stdout:8/276: mknod d10/d45/c5d 0 2024-09-16T00:25:56.921 INFO:tasks.workunit.client.1.smithi169.stdout:7/214: fsync d7/d16/f1d 0 2024-09-16T00:25:56.941 INFO:tasks.workunit.client.1.smithi169.stdout:4/207: dread d2/dd/f18 [0,4194304] 0 2024-09-16T00:25:56.945 INFO:tasks.workunit.client.1.smithi169.stdout:4/208: rmdir d2/d12/d13/d3d 0 2024-09-16T00:25:56.946 INFO:tasks.workunit.client.1.smithi169.stdout:7/215: truncate d7/da/fc 2777696 0 2024-09-16T00:25:56.947 INFO:tasks.workunit.client.0.smithi112.stdout:8/277: dwrite d10/d12/d1c/f31 [4194304,4194304] 0 2024-09-16T00:25:56.947 INFO:tasks.workunit.client.0.smithi112.stdout:8/278: stat d10/f55 0 2024-09-16T00:25:56.949 INFO:tasks.workunit.client.1.smithi169.stdout:4/209: dread d2/f3 [0,4194304] 0 2024-09-16T00:25:56.949 INFO:tasks.workunit.client.1.smithi169.stdout:7/216: link d7/da/d2e/l32 d7/d16/l3f 0 2024-09-16T00:25:56.952 INFO:tasks.workunit.client.1.smithi169.stdout:7/217: write d7/df/f2b [436952,76613] 0 2024-09-16T00:25:56.959 INFO:tasks.workunit.client.0.smithi112.stdout:4/305: sync 2024-09-16T00:25:56.959 INFO:tasks.workunit.client.0.smithi112.stdout:0/332: sync 2024-09-16T00:25:56.959 INFO:tasks.workunit.client.0.smithi112.stdout:5/332: sync 2024-09-16T00:25:56.959 INFO:tasks.workunit.client.0.smithi112.stdout:6/312: sync 2024-09-16T00:25:56.959 INFO:tasks.workunit.client.0.smithi112.stdout:9/500: sync 2024-09-16T00:25:56.959 INFO:tasks.workunit.client.0.smithi112.stdout:2/352: sync 2024-09-16T00:25:56.959 INFO:tasks.workunit.client.0.smithi112.stdout:3/441: sync 2024-09-16T00:25:56.962 INFO:tasks.workunit.client.0.smithi112.stdout:0/333: creat d5/da/d18/f62 x:0 0 0 2024-09-16T00:25:56.962 INFO:tasks.workunit.client.0.smithi112.stdout:5/333: creat d4/d7/d26/d36/f74 x:0 0 0 2024-09-16T00:25:56.967 INFO:tasks.workunit.client.0.smithi112.stdout:9/501: creat d1/d5/d20/d81/d1a/fb7 x:0 0 0 2024-09-16T00:25:56.968 INFO:tasks.workunit.client.0.smithi112.stdout:2/353: creat d2/dd/d48/d22/d2b/d30/f6f x:0 0 0 2024-09-16T00:25:56.969 INFO:tasks.workunit.client.0.smithi112.stdout:4/306: mkdir d6/db/df/d5e/d61/d6d 0 2024-09-16T00:25:56.970 INFO:tasks.workunit.client.0.smithi112.stdout:2/354: write d2/f19 [3463912,44281] 0 2024-09-16T00:25:56.970 INFO:tasks.workunit.client.0.smithi112.stdout:3/442: rename d2/d9/d2b/d39/f85 to d2/d9/d2b/d47/f9d 0 2024-09-16T00:25:56.972 INFO:tasks.workunit.client.0.smithi112.stdout:0/334: dwrite f0 [0,4194304] 0 2024-09-16T00:25:56.973 INFO:tasks.workunit.client.0.smithi112.stdout:2/355: truncate d2/dd/d48/d22/f6a 913003 0 2024-09-16T00:25:56.974 INFO:tasks.workunit.client.0.smithi112.stdout:9/502: unlink d1/d5/d83/c85 0 2024-09-16T00:25:56.976 INFO:tasks.workunit.client.0.smithi112.stdout:4/307: mkdir d6/d39/d1c/d6e 0 2024-09-16T00:25:56.976 INFO:tasks.workunit.client.0.smithi112.stdout:6/313: getdents dd/df/d13/d18 0 2024-09-16T00:25:56.978 INFO:tasks.workunit.client.0.smithi112.stdout:3/443: mknod d2/d9/d2b/d47/d52/c9e 0 2024-09-16T00:25:56.978 INFO:tasks.workunit.client.0.smithi112.stdout:6/314: write dd/d15/f29 [955731,37525] 0 2024-09-16T00:25:56.980 INFO:tasks.workunit.client.0.smithi112.stdout:0/335: mkdir d5/da/d18/d63 0 2024-09-16T00:25:56.981 INFO:tasks.workunit.client.0.smithi112.stdout:2/356: creat d2/dd/d48/d22/d2b/d54/f70 x:0 0 0 2024-09-16T00:25:56.981 INFO:tasks.workunit.client.0.smithi112.stdout:5/334: link d4/dd/d12/l31 d4/dd/d6f/l75 0 2024-09-16T00:25:56.982 INFO:tasks.workunit.client.0.smithi112.stdout:2/357: fsync d2/dd/d48/f2a 0 2024-09-16T00:25:56.982 INFO:tasks.workunit.client.0.smithi112.stdout:3/444: fdatasync d2/d9/d2b/d47/f30 0 2024-09-16T00:25:56.983 INFO:tasks.workunit.client.0.smithi112.stdout:2/358: stat d2 0 2024-09-16T00:25:56.984 INFO:tasks.workunit.client.0.smithi112.stdout:6/315: mknod dd/d23/c6f 0 2024-09-16T00:25:56.984 INFO:tasks.workunit.client.0.smithi112.stdout:5/335: dread - d4/d7/d26/d36/f41 zero size 2024-09-16T00:25:56.987 INFO:tasks.workunit.client.0.smithi112.stdout:5/336: fsync d4/dd/d12/d56/f6c 0 2024-09-16T00:25:56.989 INFO:tasks.workunit.client.0.smithi112.stdout:6/316: symlink dd/d15/d5a/d37/d62/l70 0 2024-09-16T00:25:56.990 INFO:tasks.workunit.client.0.smithi112.stdout:9/503: dwrite d1/d5/d6/d2d/d39/f65 [0,4194304] 0 2024-09-16T00:25:56.990 INFO:tasks.workunit.client.0.smithi112.stdout:3/445: getdents d2/d9/d2b/d47/d52/d7a/d80/d83/d68/d90 0 2024-09-16T00:25:56.990 INFO:tasks.workunit.client.0.smithi112.stdout:2/359: dread d2/dd/d48/f2f [0,4194304] 0 2024-09-16T00:25:56.993 INFO:tasks.workunit.client.0.smithi112.stdout:2/360: chown d2/dd/d48/d22/d2b/d54/d60 5 1 2024-09-16T00:25:56.995 INFO:tasks.workunit.client.0.smithi112.stdout:6/317: link dd/df/f21 dd/d15/d34/f71 0 2024-09-16T00:25:56.997 INFO:tasks.workunit.client.0.smithi112.stdout:5/337: dwrite d4/dd/d6f/f72 [0,4194304] 0 2024-09-16T00:25:57.032 INFO:tasks.workunit.client.1.smithi169.stdout:4/210: truncate d2/d12/f39 7472558 0 2024-09-16T00:25:57.033 INFO:tasks.workunit.client.1.smithi169.stdout:7/218: truncate d7/f31 1580796 0 2024-09-16T00:25:57.037 INFO:tasks.workunit.client.1.smithi169.stdout:7/219: rmdir d7/da/d29/d33 0 2024-09-16T00:25:57.038 INFO:tasks.workunit.client.1.smithi169.stdout:7/220: creat d7/d16/f40 x:0 0 0 2024-09-16T00:25:57.040 INFO:tasks.workunit.client.1.smithi169.stdout:7/221: mknod d7/df/c41 0 2024-09-16T00:25:57.040 INFO:tasks.workunit.client.1.smithi169.stdout:7/222: stat d7/da/l1b 0 2024-09-16T00:25:57.041 INFO:tasks.workunit.client.1.smithi169.stdout:7/223: read d7/df/f2b [232770,93734] 0 2024-09-16T00:25:57.043 INFO:tasks.workunit.client.1.smithi169.stdout:7/224: rename d7/df/f11 to d7/d16/f42 0 2024-09-16T00:25:57.045 INFO:tasks.workunit.client.1.smithi169.stdout:7/225: unlink d7/df/f2b 0 2024-09-16T00:25:57.046 INFO:tasks.workunit.client.1.smithi169.stdout:7/226: creat d7/d16/f43 x:0 0 0 2024-09-16T00:25:57.051 INFO:tasks.workunit.client.0.smithi112.stdout:2/361: link d2/dd/f15 d2/dd/d48/d22/d2b/d3b/d49/f71 0 2024-09-16T00:25:57.051 INFO:tasks.workunit.client.0.smithi112.stdout:8/279: truncate d10/d12/f1a 6082974 0 2024-09-16T00:25:57.053 INFO:tasks.workunit.client.0.smithi112.stdout:8/280: truncate d10/f55 804278 0 2024-09-16T00:25:57.053 INFO:tasks.workunit.client.0.smithi112.stdout:2/362: write f1 [900638,108202] 0 2024-09-16T00:25:57.054 INFO:tasks.workunit.client.0.smithi112.stdout:4/308: truncate d6/d39/d1c/d2c/f29 411843 0 2024-09-16T00:25:57.054 INFO:tasks.workunit.client.0.smithi112.stdout:0/336: truncate d5/da/f2e 339532 0 2024-09-16T00:25:57.054 INFO:tasks.workunit.client.0.smithi112.stdout:8/281: fsync d10/d12/d1c/f38 0 2024-09-16T00:25:57.055 INFO:tasks.workunit.client.0.smithi112.stdout:2/363: fdatasync d2/dd/d48/d22/d2b/f57 0 2024-09-16T00:25:57.057 INFO:tasks.workunit.client.0.smithi112.stdout:0/337: write d5/da/f27 [258860,9077] 0 2024-09-16T00:25:57.057 INFO:tasks.workunit.client.1.smithi169.stdout:7/227: write d7/f31 [2238410,98134] 0 2024-09-16T00:25:57.057 INFO:tasks.workunit.client.1.smithi169.stdout:7/228: dread - d7/da/f36 zero size 2024-09-16T00:25:57.059 INFO:tasks.workunit.client.0.smithi112.stdout:5/338: creat d4/d4f/f76 x:0 0 0 2024-09-16T00:25:57.059 INFO:tasks.workunit.client.0.smithi112.stdout:2/364: unlink d2/dd/c26 0 2024-09-16T00:25:57.063 INFO:tasks.workunit.client.0.smithi112.stdout:8/282: dwrite d10/d45/f3f [0,4194304] 0 2024-09-16T00:25:57.064 INFO:tasks.workunit.client.1.smithi169.stdout:7/229: dwrite d7/f2c [0,4194304] 0 2024-09-16T00:25:57.065 INFO:tasks.workunit.client.1.smithi169.stdout:0/139: sync 2024-09-16T00:25:57.066 INFO:tasks.workunit.client.0.smithi112.stdout:0/338: getdents d5/da/dc 0 2024-09-16T00:25:57.069 INFO:tasks.workunit.client.0.smithi112.stdout:0/339: write d5/da/d18/f62 [311588,50146] 0 2024-09-16T00:25:57.070 INFO:tasks.workunit.client.0.smithi112.stdout:5/339: rename d4/dd/d12/d56 to d4/d4f/d77 0 2024-09-16T00:25:57.071 INFO:tasks.workunit.client.1.smithi169.stdout:0/140: creat d0/d2/f32 x:0 0 0 2024-09-16T00:25:57.073 INFO:tasks.workunit.client.1.smithi169.stdout:0/141: symlink d0/d2/d8/d5/d14/d16/l33 0 2024-09-16T00:25:57.076 INFO:tasks.workunit.client.1.smithi169.stdout:0/142: mknod d0/d2/c34 0 2024-09-16T00:25:57.076 INFO:tasks.workunit.client.0.smithi112.stdout:8/283: rename d10/d45/c37 to d10/d4e/d58/c5e 0 2024-09-16T00:25:57.077 INFO:tasks.workunit.client.1.smithi169.stdout:0/143: fsync d0/d2/d8/d5/f24 0 2024-09-16T00:25:57.077 INFO:tasks.workunit.client.1.smithi169.stdout:7/230: truncate d7/f26 4009689 0 2024-09-16T00:25:57.078 INFO:tasks.workunit.client.0.smithi112.stdout:8/284: write d10/f1f [454269,39686] 0 2024-09-16T00:25:57.078 INFO:tasks.workunit.client.0.smithi112.stdout:5/340: truncate d4/dd/f27 4488403 0 2024-09-16T00:25:57.080 INFO:tasks.workunit.client.0.smithi112.stdout:6/318: dread dd/d15/d34/f71 [0,4194304] 0 2024-09-16T00:25:57.082 INFO:tasks.workunit.client.0.smithi112.stdout:6/319: chown dd/d15/d5a/d37/d62/f58 3044238 1 2024-09-16T00:25:57.083 INFO:tasks.workunit.client.0.smithi112.stdout:0/340: mkdir d5/da/d34/d64 0 2024-09-16T00:25:57.083 INFO:tasks.workunit.client.1.smithi169.stdout:7/231: dread d7/da/fc [0,4194304] 0 2024-09-16T00:25:57.084 INFO:tasks.workunit.client.1.smithi169.stdout:0/144: dwrite d0/d2/d8/d5/d14/d21/f2f [0,4194304] 0 2024-09-16T00:25:57.084 INFO:tasks.workunit.client.1.smithi169.stdout:0/145: fdatasync d0/f23 0 2024-09-16T00:25:57.085 INFO:tasks.workunit.client.0.smithi112.stdout:8/285: dwrite d10/f34 [0,4194304] 0 2024-09-16T00:25:57.086 INFO:tasks.workunit.client.1.smithi169.stdout:7/232: unlink d7/d16/f43 0 2024-09-16T00:25:57.087 INFO:tasks.workunit.client.1.smithi169.stdout:0/146: mknod d0/d2/d8/d5/d14/d16/d1a/c35 0 2024-09-16T00:25:57.088 INFO:tasks.workunit.client.0.smithi112.stdout:6/320: rename dd/d15/f29 to dd/df/d10/d47/d55/f72 0 2024-09-16T00:25:57.089 INFO:tasks.workunit.client.0.smithi112.stdout:0/341: creat d5/da/d18/d63/f65 x:0 0 0 2024-09-16T00:25:57.089 INFO:tasks.workunit.client.1.smithi169.stdout:0/147: write d0/d2/f32 [407340,15346] 0 2024-09-16T00:25:57.090 INFO:tasks.workunit.client.1.smithi169.stdout:7/233: truncate f1 297365 0 2024-09-16T00:25:57.091 INFO:tasks.workunit.client.1.smithi169.stdout:0/148: creat d0/d2/d8/d5/d14/f36 x:0 0 0 2024-09-16T00:25:57.092 INFO:tasks.workunit.client.1.smithi169.stdout:7/234: creat d7/df/f44 x:0 0 0 2024-09-16T00:25:57.096 INFO:tasks.workunit.client.1.smithi169.stdout:7/235: creat d7/da/d29/f45 x:0 0 0 2024-09-16T00:25:57.096 INFO:tasks.workunit.client.1.smithi169.stdout:7/236: chown d7/d16/f1d 0 1 2024-09-16T00:25:57.097 INFO:tasks.workunit.client.1.smithi169.stdout:0/149: creat d0/d2/d8/f37 x:0 0 0 2024-09-16T00:25:57.097 INFO:tasks.workunit.client.0.smithi112.stdout:8/286: unlink c6 0 2024-09-16T00:25:57.097 INFO:tasks.workunit.client.0.smithi112.stdout:8/287: readlink d10/l3c 0 2024-09-16T00:25:57.101 INFO:tasks.workunit.client.1.smithi169.stdout:7/237: rename d7/d16/f1a to d7/f46 0 2024-09-16T00:25:57.101 INFO:tasks.workunit.client.1.smithi169.stdout:0/150: mkdir d0/d2c/d38 0 2024-09-16T00:25:57.101 INFO:tasks.workunit.client.0.smithi112.stdout:2/365: dread d2/dd/d48/d22/d2b/f34 [0,4194304] 0 2024-09-16T00:25:57.101 INFO:tasks.workunit.client.0.smithi112.stdout:2/366: chown d2/dd/d48/f2f 18071614 1 2024-09-16T00:25:57.101 INFO:tasks.workunit.client.0.smithi112.stdout:0/342: rename d5/da/d34/d48 to d5/da/d66 0 2024-09-16T00:25:57.102 INFO:tasks.workunit.client.1.smithi169.stdout:0/151: creat d0/d2c/f39 x:0 0 0 2024-09-16T00:25:57.102 INFO:tasks.workunit.client.0.smithi112.stdout:0/343: chown d5/da/d66/f5d 195537 1 2024-09-16T00:25:57.104 INFO:tasks.workunit.client.1.smithi169.stdout:0/152: write d0/f10 [2511548,27975] 0 2024-09-16T00:25:57.104 INFO:tasks.workunit.client.1.smithi169.stdout:0/153: read - d0/f23 zero size 2024-09-16T00:25:57.104 INFO:tasks.workunit.client.0.smithi112.stdout:0/344: write d5/da/dc/f10 [3808759,59360] 0 2024-09-16T00:25:57.105 INFO:tasks.workunit.client.0.smithi112.stdout:2/367: mknod d2/dd/d48/d22/d2b/d30/c72 0 2024-09-16T00:25:57.105 INFO:tasks.workunit.client.0.smithi112.stdout:8/288: link d10/c17 d10/d4e/c5f 0 2024-09-16T00:25:57.105 INFO:tasks.workunit.client.0.smithi112.stdout:0/345: symlink d5/da/d66/l67 0 2024-09-16T00:25:57.109 INFO:tasks.workunit.client.0.smithi112.stdout:2/368: truncate d2/dd/d48/f11 1202602 0 2024-09-16T00:25:57.109 INFO:tasks.workunit.client.1.smithi169.stdout:0/154: dwrite d0/d2/f32 [0,4194304] 0 2024-09-16T00:25:57.110 INFO:tasks.workunit.client.1.smithi169.stdout:0/155: chown d0/d2/d8/d5/d14/d21/d31 578023 1 2024-09-16T00:25:57.111 INFO:tasks.workunit.client.1.smithi169.stdout:0/156: dread - d0/d2/d8/fe zero size 2024-09-16T00:25:57.111 INFO:tasks.workunit.client.0.smithi112.stdout:0/346: rename d5/da/dc/d4e/f4f to d5/f68 0 2024-09-16T00:25:57.113 INFO:tasks.workunit.client.0.smithi112.stdout:8/289: dwrite d10/d45/f57 [4194304,4194304] 0 2024-09-16T00:25:57.114 INFO:tasks.workunit.client.1.smithi169.stdout:0/157: rename d0/d2/d8/d5/d14/d16/d1a/c35 to d0/c3a 0 2024-09-16T00:25:57.115 INFO:tasks.workunit.client.0.smithi112.stdout:7/406: sync 2024-09-16T00:25:57.115 INFO:tasks.workunit.client.0.smithi112.stdout:8/290: write d10/d12/d1c/f54 [923225,33608] 0 2024-09-16T00:25:57.116 INFO:tasks.workunit.client.0.smithi112.stdout:2/369: creat d2/dd/d48/d22/d2b/f73 x:0 0 0 2024-09-16T00:25:57.116 INFO:tasks.workunit.client.1.smithi169.stdout:0/158: unlink d0/d2/d15/f27 0 2024-09-16T00:25:57.118 INFO:tasks.workunit.client.0.smithi112.stdout:0/347: rename d5/da/f2b to d5/da/d66/f69 0 2024-09-16T00:25:57.119 INFO:tasks.workunit.client.0.smithi112.stdout:6/321: dread dd/df/d13/f1c [0,4194304] 0 2024-09-16T00:25:57.119 INFO:tasks.workunit.client.0.smithi112.stdout:7/407: fsync d6/d10/d16/f3a 0 2024-09-16T00:25:57.121 INFO:tasks.workunit.client.0.smithi112.stdout:8/291: truncate d10/d12/d1c/f54 1608938 0 2024-09-16T00:25:57.123 INFO:tasks.workunit.client.0.smithi112.stdout:2/370: dread - d2/dd/d48/d22/d2b/d3b/f66 zero size 2024-09-16T00:25:57.124 INFO:tasks.workunit.client.0.smithi112.stdout:7/408: mkdir d6/d10/d16/d96 0 2024-09-16T00:25:57.125 INFO:tasks.workunit.client.1.smithi169.stdout:0/159: dwrite d0/d2/f6 [4194304,4194304] 0 2024-09-16T00:25:57.126 INFO:tasks.workunit.client.0.smithi112.stdout:2/371: write d2/dd/f5b [785946,55913] 0 2024-09-16T00:25:57.127 INFO:tasks.workunit.client.1.smithi169.stdout:0/160: write d0/f23 [1001816,85382] 0 2024-09-16T00:25:57.128 INFO:tasks.workunit.client.0.smithi112.stdout:6/322: unlink l2 0 2024-09-16T00:25:57.128 INFO:tasks.workunit.client.0.smithi112.stdout:7/409: mknod d6/d10/d75/d8f/c97 0 2024-09-16T00:25:57.129 INFO:tasks.workunit.client.0.smithi112.stdout:6/323: stat l5 0 2024-09-16T00:25:57.131 INFO:tasks.workunit.client.0.smithi112.stdout:7/410: mknod d6/d10/d16/c98 0 2024-09-16T00:25:57.133 INFO:tasks.workunit.client.1.smithi169.stdout:0/161: dwrite d0/d2/d8/d5/d14/f2b [0,4194304] 0 2024-09-16T00:25:57.135 INFO:tasks.workunit.client.0.smithi112.stdout:6/324: chown dd/df/d17/d27/d54 181619 1 2024-09-16T00:25:57.137 INFO:tasks.workunit.client.1.smithi169.stdout:0/162: getdents d0/d2/d8/d5/d14/d21/d31 0 2024-09-16T00:25:57.140 INFO:tasks.workunit.client.0.smithi112.stdout:2/372: dwrite d2/dd/d48/f2f [0,4194304] 0 2024-09-16T00:25:57.140 INFO:tasks.workunit.client.1.smithi169.stdout:0/163: creat d0/d2/d8/d5/d14/f3b x:0 0 0 2024-09-16T00:25:57.141 INFO:tasks.workunit.client.1.smithi169.stdout:0/164: write d0/d2/d8/fe [840702,97288] 0 2024-09-16T00:25:57.142 INFO:tasks.workunit.client.1.smithi169.stdout:0/165: read d0/d2/d8/d5/d14/f2b [3342223,76899] 0 2024-09-16T00:25:57.144 INFO:tasks.workunit.client.0.smithi112.stdout:2/373: truncate d2/dd/d48/f2d 422777 0 2024-09-16T00:25:57.144 INFO:tasks.workunit.client.1.smithi169.stdout:0/166: truncate d0/d2/d15/f22 417180 0 2024-09-16T00:25:57.145 INFO:tasks.workunit.client.1.smithi169.stdout:0/167: write d0/d2/d8/d5/f2e [273728,9963] 0 2024-09-16T00:25:57.148 INFO:tasks.workunit.client.1.smithi169.stdout:0/168: mknod d0/d2/c3c 0 2024-09-16T00:25:57.162 INFO:tasks.workunit.client.0.smithi112.stdout:0/348: dread d5/da/dc/f10 [0,4194304] 0 2024-09-16T00:25:57.166 INFO:tasks.workunit.client.1.smithi169.stdout:7/238: write d7/f26 [604631,33148] 0 2024-09-16T00:25:57.167 INFO:tasks.workunit.client.0.smithi112.stdout:0/349: dwrite d5/da/dc/fe [0,4194304] 0 2024-09-16T00:25:57.167 INFO:tasks.workunit.client.1.smithi169.stdout:7/239: readlink d7/l18 0 2024-09-16T00:25:57.168 INFO:tasks.workunit.client.0.smithi112.stdout:0/350: chown d5/da/d34/d4c/f50 14551322 1 2024-09-16T00:25:57.169 INFO:tasks.workunit.client.1.smithi169.stdout:7/240: creat d7/da/d29/f47 x:0 0 0 2024-09-16T00:25:57.171 INFO:tasks.workunit.client.1.smithi169.stdout:7/241: mknod d7/d16/c48 0 2024-09-16T00:25:57.174 INFO:tasks.workunit.client.0.smithi112.stdout:2/374: dread d2/dd/f5e [0,4194304] 0 2024-09-16T00:25:57.174 INFO:tasks.workunit.client.1.smithi169.stdout:7/242: rename d7/da/d2e to d7/d16/d49 0 2024-09-16T00:25:57.176 INFO:tasks.workunit.client.0.smithi112.stdout:0/351: dwrite d5/da/d34/f4d [0,4194304] 0 2024-09-16T00:25:57.177 INFO:tasks.workunit.client.0.smithi112.stdout:2/375: rmdir d2/dd/d48/d22/d2b/d3b/d49 39 2024-09-16T00:25:57.178 INFO:tasks.workunit.client.0.smithi112.stdout:2/376: fsync d2/f14 0 2024-09-16T00:25:57.180 INFO:tasks.workunit.client.1.smithi169.stdout:7/243: dread d7/f2c [0,4194304] 0 2024-09-16T00:25:57.181 INFO:tasks.workunit.client.0.smithi112.stdout:2/377: creat d2/dd/d48/d22/d67/f74 x:0 0 0 2024-09-16T00:25:57.182 INFO:tasks.workunit.client.0.smithi112.stdout:2/378: readlink d2/dd/d48/d22/l6e 0 2024-09-16T00:25:57.183 INFO:tasks.workunit.client.0.smithi112.stdout:2/379: creat d2/dd/f75 x:0 0 0 2024-09-16T00:25:57.184 INFO:tasks.workunit.client.0.smithi112.stdout:2/380: chown d2/dd/d48/l1e 292434981 1 2024-09-16T00:25:57.184 INFO:tasks.workunit.client.0.smithi112.stdout:0/352: truncate d5/da/dc/d12/f17 3283678 0 2024-09-16T00:25:57.185 INFO:tasks.workunit.client.0.smithi112.stdout:2/381: fsync d2/dd/d48/d22/d2b/f57 0 2024-09-16T00:25:57.186 INFO:tasks.workunit.client.1.smithi169.stdout:7/244: rename d7/da/l1b to d7/l4a 0 2024-09-16T00:25:57.186 INFO:tasks.workunit.client.0.smithi112.stdout:2/382: read - d2/dd/d48/d22/d2b/d54/f70 zero size 2024-09-16T00:25:57.187 INFO:tasks.workunit.client.0.smithi112.stdout:0/353: link d5/da/d21/f28 d5/da/f6a 0 2024-09-16T00:25:57.188 INFO:tasks.workunit.client.1.smithi169.stdout:7/245: symlink d7/l4b 0 2024-09-16T00:25:57.188 INFO:tasks.workunit.client.0.smithi112.stdout:2/383: fdatasync d2/dd/d48/d22/d2b/d3b/d49/f5c 0 2024-09-16T00:25:57.188 INFO:tasks.workunit.client.0.smithi112.stdout:0/354: dread - d5/da/dc/d12/f14 zero size 2024-09-16T00:25:57.190 INFO:tasks.workunit.client.0.smithi112.stdout:2/384: write d2/dd/d48/f1d [1018490,120687] 0 2024-09-16T00:25:57.190 INFO:tasks.workunit.client.1.smithi169.stdout:7/246: rename d7/d16/d49/f35 to d7/f4c 0 2024-09-16T00:25:57.190 INFO:tasks.workunit.client.1.smithi169.stdout:7/247: chown d7/d16 23 1 2024-09-16T00:25:57.192 INFO:tasks.workunit.client.1.smithi169.stdout:7/248: chown d7/c34 6637 1 2024-09-16T00:25:57.193 INFO:tasks.workunit.client.0.smithi112.stdout:0/355: rename d5/da/d18/d24/f41 to d5/da/dc/d12/f6b 0 2024-09-16T00:25:57.193 INFO:tasks.workunit.client.1.smithi169.stdout:7/249: creat d7/d16/d49/f4d x:0 0 0 2024-09-16T00:25:57.194 INFO:tasks.workunit.client.0.smithi112.stdout:2/385: dread d2/dd/f1a [0,4194304] 0 2024-09-16T00:25:57.195 INFO:tasks.workunit.client.1.smithi169.stdout:7/250: rename d7/da/l2d to d7/d16/l4e 0 2024-09-16T00:25:57.195 INFO:tasks.workunit.client.0.smithi112.stdout:2/386: stat d2/dd/d48/f2d 0 2024-09-16T00:25:57.197 INFO:tasks.workunit.client.0.smithi112.stdout:2/387: unlink d2/dd/d48/d22/l6e 0 2024-09-16T00:25:57.201 INFO:tasks.workunit.client.1.smithi169.stdout:7/251: dwrite d7/da/d29/f47 [0,4194304] 0 2024-09-16T00:25:57.201 INFO:tasks.workunit.client.0.smithi112.stdout:2/388: dread d2/f19 [0,4194304] 0 2024-09-16T00:25:57.202 INFO:tasks.workunit.client.0.smithi112.stdout:0/356: write d5/da/dc/d12/f17 [1151388,59798] 0 2024-09-16T00:25:57.203 INFO:tasks.workunit.client.0.smithi112.stdout:0/357: fdatasync d5/f2a 0 2024-09-16T00:25:57.205 INFO:tasks.workunit.client.0.smithi112.stdout:2/389: dread d2/dd/d48/f2d [0,4194304] 0 2024-09-16T00:25:57.206 INFO:tasks.workunit.client.0.smithi112.stdout:0/358: chown d5/da/f2e 186658938 1 2024-09-16T00:25:57.208 INFO:tasks.workunit.client.1.smithi169.stdout:7/252: dwrite d7/d16/f24 [0,4194304] 0 2024-09-16T00:25:57.208 INFO:tasks.workunit.client.0.smithi112.stdout:2/390: write d2/dd/d48/d22/d2b/f31 [1147372,61981] 0 2024-09-16T00:25:57.209 INFO:tasks.workunit.client.0.smithi112.stdout:0/359: rmdir d5/da/d34/d4c 39 2024-09-16T00:25:57.211 INFO:tasks.workunit.client.0.smithi112.stdout:2/391: rmdir d2/dd/d48/d22/d2b/d54/d60 39 2024-09-16T00:25:57.212 INFO:tasks.workunit.client.0.smithi112.stdout:0/360: creat d5/da/d18/d24/d5a/f6c x:0 0 0 2024-09-16T00:25:57.215 INFO:tasks.workunit.client.0.smithi112.stdout:2/392: mknod d2/dd/d48/d22/d2b/d54/d60/d6c/c76 0 2024-09-16T00:25:57.216 INFO:tasks.workunit.client.1.smithi169.stdout:7/253: link d7/d16/c48 d7/d16/c4f 0 2024-09-16T00:25:57.216 INFO:tasks.workunit.client.0.smithi112.stdout:0/361: mknod d5/da/d34/c6d 0 2024-09-16T00:25:57.218 INFO:tasks.workunit.client.1.smithi169.stdout:7/254: creat d7/f50 x:0 0 0 2024-09-16T00:25:57.218 INFO:tasks.workunit.client.0.smithi112.stdout:2/393: rmdir d2/dd/d48/d22/d2b/d54/d60/d6c 39 2024-09-16T00:25:57.218 INFO:tasks.workunit.client.0.smithi112.stdout:2/394: stat d2/dd/d48/d22/c69 0 2024-09-16T00:25:57.219 INFO:tasks.workunit.client.1.smithi169.stdout:7/255: mknod d7/da/d3a/c51 0 2024-09-16T00:25:57.219 INFO:tasks.workunit.client.0.smithi112.stdout:0/362: creat d5/da/d57/f6e x:0 0 0 2024-09-16T00:25:57.220 INFO:tasks.workunit.client.1.smithi169.stdout:7/256: write d7/da/f37 [459070,71159] 0 2024-09-16T00:25:57.220 INFO:tasks.workunit.client.0.smithi112.stdout:2/395: write d2/dd/d48/d22/d2b/f73 [441965,107733] 0 2024-09-16T00:25:57.221 INFO:tasks.workunit.client.1.smithi169.stdout:7/257: mknod d7/da/c52 0 2024-09-16T00:25:57.221 INFO:tasks.workunit.client.0.smithi112.stdout:0/363: mkdir d5/da/d21/d6f 0 2024-09-16T00:25:57.222 INFO:tasks.workunit.client.0.smithi112.stdout:0/364: chown d5/da/d18/f54 15561906 1 2024-09-16T00:25:57.222 INFO:tasks.workunit.client.0.smithi112.stdout:0/365: chown d5/f23 49114 1 2024-09-16T00:25:57.222 INFO:tasks.workunit.client.1.smithi169.stdout:7/258: chown d7/le 14411890 1 2024-09-16T00:25:57.228 INFO:tasks.workunit.client.0.smithi112.stdout:0/366: truncate d5/da/d34/f4d 881885 0 2024-09-16T00:25:57.229 INFO:tasks.workunit.client.0.smithi112.stdout:0/367: unlink d5/da/d21/f28 0 2024-09-16T00:25:57.232 INFO:tasks.workunit.client.0.smithi112.stdout:0/368: rename d5/da/dc/c29 to d5/da/d21/d6f/c70 0 2024-09-16T00:25:57.232 INFO:tasks.workunit.client.0.smithi112.stdout:2/396: dread f1 [0,4194304] 0 2024-09-16T00:25:57.233 INFO:tasks.workunit.client.0.smithi112.stdout:0/369: stat d5/da/d34/c6d 0 2024-09-16T00:25:57.233 INFO:tasks.workunit.client.0.smithi112.stdout:2/397: write d2/f46 [1581561,43346] 0 2024-09-16T00:25:57.236 INFO:tasks.workunit.client.0.smithi112.stdout:0/370: rename d5/da/d34 to d5/da/d1d/d71 0 2024-09-16T00:25:57.239 INFO:tasks.workunit.client.0.smithi112.stdout:0/371: mknod d5/da/d1d/c72 0 2024-09-16T00:25:57.246 INFO:tasks.workunit.client.0.smithi112.stdout:0/372: dread - d5/da/d1d/d71/f5c zero size 2024-09-16T00:25:57.248 INFO:tasks.workunit.client.0.smithi112.stdout:0/373: chown d5/da/dc/d12/f3c 26496 1 2024-09-16T00:25:57.249 INFO:tasks.workunit.client.0.smithi112.stdout:0/374: truncate d5/da/d1d/f56 952122 0 2024-09-16T00:25:57.251 INFO:tasks.workunit.client.0.smithi112.stdout:2/398: dread d2/f14 [0,4194304] 0 2024-09-16T00:25:57.251 INFO:tasks.workunit.client.0.smithi112.stdout:0/375: creat d5/da/d57/f73 x:0 0 0 2024-09-16T00:25:57.252 INFO:tasks.workunit.client.0.smithi112.stdout:0/376: mkdir d5/da/dc/d12/d74 0 2024-09-16T00:25:57.254 INFO:tasks.workunit.client.0.smithi112.stdout:2/399: link d2/dd/d48/f2d d2/dd/d48/d22/d67/f77 0 2024-09-16T00:25:57.255 INFO:tasks.workunit.client.0.smithi112.stdout:0/377: mknod d5/da/d1d/d71/d64/c75 0 2024-09-16T00:25:57.256 INFO:tasks.workunit.client.0.smithi112.stdout:2/400: unlink d2/f8 0 2024-09-16T00:25:57.258 INFO:tasks.workunit.client.0.smithi112.stdout:2/401: unlink d2/dd/d48/d22/d2b/f59 0 2024-09-16T00:25:57.260 INFO:tasks.workunit.client.0.smithi112.stdout:0/378: rename d5/da/dc/d12 to d5/da/d57/d76 0 2024-09-16T00:25:57.261 INFO:tasks.workunit.client.0.smithi112.stdout:2/402: dread d2/dd/d48/d22/d2b/d3b/d49/f5c [0,4194304] 0 2024-09-16T00:25:57.265 INFO:tasks.workunit.client.0.smithi112.stdout:2/403: mknod d2/dd/d48/d22/d2b/d54/d60/c78 0 2024-09-16T00:25:57.272 INFO:tasks.workunit.client.0.smithi112.stdout:0/379: dread d5/da/d57/d76/f17 [0,4194304] 0 2024-09-16T00:25:57.273 INFO:tasks.workunit.client.1.smithi169.stdout:9/193: sync 2024-09-16T00:25:57.273 INFO:tasks.workunit.client.1.smithi169.stdout:6/85: sync 2024-09-16T00:25:57.274 INFO:tasks.workunit.client.0.smithi112.stdout:2/404: mkdir d2/dd/d48/d22/d2b/d54/d79 0 2024-09-16T00:25:57.275 INFO:tasks.workunit.client.1.smithi169.stdout:9/194: mkdir d5/da/d2e/d47 0 2024-09-16T00:25:57.276 INFO:tasks.workunit.client.1.smithi169.stdout:9/195: fsync d5/d12/f13 0 2024-09-16T00:25:57.278 INFO:tasks.workunit.client.1.smithi169.stdout:9/196: write d5/da/fc [732071,73888] 0 2024-09-16T00:25:57.278 INFO:tasks.workunit.client.0.smithi112.stdout:0/380: dread d5/ff [0,4194304] 0 2024-09-16T00:25:57.279 INFO:tasks.workunit.client.0.smithi112.stdout:0/381: chown d5/cb 194859 1 2024-09-16T00:25:57.280 INFO:tasks.workunit.client.1.smithi169.stdout:6/86: dwrite d2/d15/f1b [0,4194304] 0 2024-09-16T00:25:57.281 INFO:tasks.workunit.client.0.smithi112.stdout:0/382: creat d5/da/f77 x:0 0 0 2024-09-16T00:25:57.281 INFO:tasks.workunit.client.0.smithi112.stdout:1/343: sync 2024-09-16T00:25:57.284 INFO:tasks.workunit.client.0.smithi112.stdout:1/344: creat d0/d14/df/d28/d45/d46/f6d x:0 0 0 2024-09-16T00:25:57.286 INFO:tasks.workunit.client.0.smithi112.stdout:1/345: symlink d0/d14/d44/l6e 0 2024-09-16T00:25:57.287 INFO:tasks.workunit.client.1.smithi169.stdout:6/87: dwrite d2/f17 [0,4194304] 0 2024-09-16T00:25:57.289 INFO:tasks.workunit.client.0.smithi112.stdout:0/383: dwrite d5/da/dc/f10 [0,4194304] 0 2024-09-16T00:25:57.290 INFO:tasks.workunit.client.0.smithi112.stdout:0/384: chown d5/da/dc/l55 0 1 2024-09-16T00:25:57.292 INFO:tasks.workunit.client.1.smithi169.stdout:6/88: rename d2/ca to d2/c1c 0 2024-09-16T00:25:57.296 INFO:tasks.workunit.client.0.smithi112.stdout:1/346: write d0/f67 [38367,2923] 0 2024-09-16T00:25:57.296 INFO:tasks.workunit.client.0.smithi112.stdout:0/385: fsync d5/da/d18/f62 0 2024-09-16T00:25:57.299 INFO:tasks.workunit.client.1.smithi169.stdout:9/197: dread d5/f8 [4194304,4194304] 0 2024-09-16T00:25:57.302 INFO:tasks.workunit.client.1.smithi169.stdout:9/198: rename d5/da/c1f to d5/d12/c48 0 2024-09-16T00:25:57.303 INFO:tasks.workunit.client.0.smithi112.stdout:0/386: dwrite d5/da/d1d/d71/d4c/f50 [0,4194304] 0 2024-09-16T00:25:57.304 INFO:tasks.workunit.client.1.smithi169.stdout:9/199: mkdir d5/d49 0 2024-09-16T00:25:57.306 INFO:tasks.workunit.client.1.smithi169.stdout:9/200: creat d5/f4a x:0 0 0 2024-09-16T00:25:57.307 INFO:tasks.workunit.client.0.smithi112.stdout:0/387: unlink d5/da/d21/d6f/c70 0 2024-09-16T00:25:57.307 INFO:tasks.workunit.client.1.smithi169.stdout:9/201: symlink d5/da/d21/d23/l4b 0 2024-09-16T00:25:57.308 INFO:tasks.workunit.client.1.smithi169.stdout:9/202: stat d5/da/d21/d23/f3b 0 2024-09-16T00:25:57.311 INFO:tasks.workunit.client.1.smithi169.stdout:9/203: creat d5/d12/f4c x:0 0 0 2024-09-16T00:25:57.312 INFO:tasks.workunit.client.0.smithi112.stdout:1/347: truncate d0/f67 2249906 0 2024-09-16T00:25:57.313 INFO:tasks.workunit.client.1.smithi169.stdout:6/89: getdents d2 0 2024-09-16T00:25:57.314 INFO:tasks.workunit.client.0.smithi112.stdout:0/388: dwrite d5/da/dc/fe [0,4194304] 0 2024-09-16T00:25:57.314 INFO:tasks.workunit.client.1.smithi169.stdout:6/90: fsync d2/d15/f18 0 2024-09-16T00:25:57.314 INFO:tasks.workunit.client.0.smithi112.stdout:1/348: truncate d0/f48 172896 0 2024-09-16T00:25:57.315 INFO:tasks.workunit.client.1.smithi169.stdout:9/204: creat d5/da/f4d x:0 0 0 2024-09-16T00:25:57.318 INFO:tasks.workunit.client.0.smithi112.stdout:0/389: dread d5/f2a [0,4194304] 0 2024-09-16T00:25:57.319 INFO:tasks.workunit.client.0.smithi112.stdout:1/349: link d0/d14/d1c/f5c d0/d14/d6a/f6f 0 2024-09-16T00:25:57.320 INFO:tasks.workunit.client.0.smithi112.stdout:0/390: chown d5/da/d57/d76/l60 53815424 1 2024-09-16T00:25:57.322 INFO:tasks.workunit.client.0.smithi112.stdout:0/391: write d5/da/dc/f10 [4965837,46734] 0 2024-09-16T00:25:57.323 INFO:tasks.workunit.client.0.smithi112.stdout:1/350: unlink d0/d14/d1c/d26/f29 0 2024-09-16T00:25:57.324 INFO:tasks.workunit.client.0.smithi112.stdout:1/351: chown d0/d14/d44/l4d 23096 1 2024-09-16T00:25:57.324 INFO:tasks.workunit.client.0.smithi112.stdout:0/392: symlink d5/da/d18/d63/l78 0 2024-09-16T00:25:57.325 INFO:tasks.workunit.client.0.smithi112.stdout:1/352: truncate d0/d14/d1c/d20/f63 929168 0 2024-09-16T00:25:57.325 INFO:tasks.workunit.client.1.smithi169.stdout:9/205: stat d5/d12/c48 0 2024-09-16T00:25:57.327 INFO:tasks.workunit.client.0.smithi112.stdout:0/393: creat d5/da/d57/d76/d74/f79 x:0 0 0 2024-09-16T00:25:57.328 INFO:tasks.workunit.client.0.smithi112.stdout:1/353: stat d0/d14/df/d28/d45/d30/f3a 0 2024-09-16T00:25:57.329 INFO:tasks.workunit.client.0.smithi112.stdout:0/394: mkdir d5/da/d18/d24/d46/d5f/d7a 0 2024-09-16T00:25:57.330 INFO:tasks.workunit.client.1.smithi169.stdout:9/206: dwrite d5/da/d21/f30 [0,4194304] 0 2024-09-16T00:25:57.332 INFO:tasks.workunit.client.0.smithi112.stdout:1/354: rename d0/d14/df/f3d to d0/d14/d1c/d26/f70 0 2024-09-16T00:25:57.333 INFO:tasks.workunit.client.1.smithi169.stdout:9/207: mkdir d5/da/d4e 0 2024-09-16T00:25:57.334 INFO:tasks.workunit.client.0.smithi112.stdout:0/395: link d5/da/dc/f13 d5/f7b 0 2024-09-16T00:25:57.335 INFO:tasks.workunit.client.1.smithi169.stdout:9/208: symlink d5/da/d4e/l4f 0 2024-09-16T00:25:57.336 INFO:tasks.workunit.client.0.smithi112.stdout:0/396: chown d5/da/d66 8 1 2024-09-16T00:25:57.337 INFO:tasks.workunit.client.1.smithi169.stdout:9/209: write d5/da/d21/d23/f2d [802739,14426] 0 2024-09-16T00:25:57.340 INFO:tasks.workunit.client.0.smithi112.stdout:1/355: dwrite d0/d14/df/d28/d45/d30/f57 [8388608,4194304] 0 2024-09-16T00:25:57.342 INFO:tasks.workunit.client.1.smithi169.stdout:9/210: dwrite d5/da/d21/d23/f2d [0,4194304] 0 2024-09-16T00:25:57.343 INFO:tasks.workunit.client.0.smithi112.stdout:0/397: dwrite d5/da/d66/f5d [0,4194304] 0 2024-09-16T00:25:57.346 INFO:tasks.workunit.client.0.smithi112.stdout:1/356: rename d0/d14/df/l10 to d0/d14/d44/l71 0 2024-09-16T00:25:57.348 INFO:tasks.workunit.client.0.smithi112.stdout:1/357: write d0/d14/d1c/d20/f65 [290624,95538] 0 2024-09-16T00:25:57.349 INFO:tasks.workunit.client.0.smithi112.stdout:0/398: creat d5/da/d21/f7c x:0 0 0 2024-09-16T00:25:57.352 INFO:tasks.workunit.client.0.smithi112.stdout:1/358: dread d0/d14/df/d28/d45/d30/f4a [0,4194304] 0 2024-09-16T00:25:57.353 INFO:tasks.workunit.client.0.smithi112.stdout:0/399: creat d5/da/dc/f7d x:0 0 0 2024-09-16T00:25:57.355 INFO:tasks.workunit.client.0.smithi112.stdout:1/359: write d0/d14/d6a/f6f [1317301,108685] 0 2024-09-16T00:25:57.356 INFO:tasks.workunit.client.0.smithi112.stdout:0/400: link d5/f7b d5/da/d57/d76/d47/f7e 0 2024-09-16T00:25:57.357 INFO:tasks.workunit.client.0.smithi112.stdout:1/360: mknod d0/d14/df/d28/c72 0 2024-09-16T00:25:57.358 INFO:tasks.workunit.client.0.smithi112.stdout:1/361: fdatasync d0/d14/d1c/d20/f53 0 2024-09-16T00:25:57.359 INFO:tasks.workunit.client.0.smithi112.stdout:1/362: chown d0/d14/d1c/d20/f65 454 1 2024-09-16T00:25:57.362 INFO:tasks.workunit.client.0.smithi112.stdout:1/363: creat d0/d14/d1c/d26/f73 x:0 0 0 2024-09-16T00:25:57.367 INFO:tasks.workunit.client.1.smithi169.stdout:9/211: dwrite d5/da/f10 [0,4194304] 0 2024-09-16T00:25:57.368 INFO:tasks.workunit.client.1.smithi169.stdout:9/212: fdatasync d5/d12/f31 0 2024-09-16T00:25:57.371 INFO:tasks.workunit.client.1.smithi169.stdout:9/213: getdents d5/da/d4e 0 2024-09-16T00:25:57.372 INFO:tasks.workunit.client.0.smithi112.stdout:1/364: dread d0/d14/d6a/f6f [0,4194304] 0 2024-09-16T00:25:57.372 INFO:tasks.workunit.client.0.smithi112.stdout:0/401: getdents d5/da/d57/d76/d74 0 2024-09-16T00:25:57.376 INFO:tasks.workunit.client.0.smithi112.stdout:0/402: mknod d5/da/d18/c7f 0 2024-09-16T00:25:57.377 INFO:tasks.workunit.client.0.smithi112.stdout:0/403: readlink d5/da/d21/l58 0 2024-09-16T00:25:57.379 INFO:tasks.workunit.client.1.smithi169.stdout:9/214: dwrite d5/da/ff [0,4194304] 0 2024-09-16T00:25:57.379 INFO:tasks.workunit.client.0.smithi112.stdout:0/404: unlink d5/da/d18/c31 0 2024-09-16T00:25:57.380 INFO:tasks.workunit.client.0.smithi112.stdout:1/365: getdents d0/d14/df/d28/d45/d46/d58/d62 0 2024-09-16T00:25:57.381 INFO:tasks.workunit.client.1.smithi169.stdout:9/215: symlink d5/da/d21/d23/l50 0 2024-09-16T00:25:57.381 INFO:tasks.workunit.client.0.smithi112.stdout:1/366: dread - d0/d14/d1c/d20/f53 zero size 2024-09-16T00:25:57.381 INFO:tasks.workunit.client.0.smithi112.stdout:0/405: mkdir d5/da/d57/d76/d74/d80 0 2024-09-16T00:25:57.382 INFO:tasks.workunit.client.1.smithi169.stdout:9/216: mknod d5/da/d2e/c51 0 2024-09-16T00:25:57.382 INFO:tasks.workunit.client.0.smithi112.stdout:0/406: chown d5/da/d21/f7c 7483 1 2024-09-16T00:25:57.384 INFO:tasks.workunit.client.0.smithi112.stdout:0/407: creat d5/da/dc/f81 x:0 0 0 2024-09-16T00:25:57.386 INFO:tasks.workunit.client.0.smithi112.stdout:0/408: truncate d5/da/d57/f6e 1021442 0 2024-09-16T00:25:57.387 INFO:tasks.workunit.client.0.smithi112.stdout:1/367: read d0/f2 [337430,1716] 0 2024-09-16T00:25:57.389 INFO:tasks.workunit.client.1.smithi169.stdout:9/217: truncate d5/da/d21/d23/f2d 1198181 0 2024-09-16T00:25:57.390 INFO:tasks.workunit.client.1.smithi169.stdout:9/218: unlink d5/da/d2e/c3d 0 2024-09-16T00:25:57.392 INFO:tasks.workunit.client.1.smithi169.stdout:9/219: symlink d5/d49/l52 0 2024-09-16T00:25:57.398 INFO:tasks.workunit.client.1.smithi169.stdout:9/220: dwrite d5/da/d21/d23/f41 [0,4194304] 0 2024-09-16T00:25:57.400 INFO:tasks.workunit.client.1.smithi169.stdout:9/221: creat d5/da/d4e/f53 x:0 0 0 2024-09-16T00:25:57.402 INFO:tasks.workunit.client.1.smithi169.stdout:8/172: sync 2024-09-16T00:25:57.402 INFO:tasks.workunit.client.1.smithi169.stdout:1/208: sync 2024-09-16T00:25:57.402 INFO:tasks.workunit.client.1.smithi169.stdout:3/220: sync 2024-09-16T00:25:57.402 INFO:tasks.workunit.client.1.smithi169.stdout:5/183: sync 2024-09-16T00:25:57.402 INFO:tasks.workunit.client.1.smithi169.stdout:2/153: sync 2024-09-16T00:25:57.402 INFO:tasks.workunit.client.1.smithi169.stdout:0/169: sync 2024-09-16T00:25:57.402 INFO:tasks.workunit.client.1.smithi169.stdout:4/211: sync 2024-09-16T00:25:57.405 INFO:tasks.workunit.client.1.smithi169.stdout:3/221: dread - d7/d1a/d1c/f40 zero size 2024-09-16T00:25:57.407 INFO:tasks.workunit.client.1.smithi169.stdout:2/154: dread d7/f24 [0,4194304] 0 2024-09-16T00:25:57.407 INFO:tasks.workunit.client.1.smithi169.stdout:5/184: mknod d2/da/d10/d13/c3e 0 2024-09-16T00:25:57.409 INFO:tasks.workunit.client.1.smithi169.stdout:9/222: write d5/f14 [2995147,21455] 0 2024-09-16T00:25:57.409 INFO:tasks.workunit.client.1.smithi169.stdout:2/155: dread d7/fe [0,4194304] 0 2024-09-16T00:25:57.410 INFO:tasks.workunit.client.1.smithi169.stdout:8/173: getdents d7/d37 0 2024-09-16T00:25:57.410 INFO:tasks.workunit.client.1.smithi169.stdout:5/185: write d2/f8 [3005791,23650] 0 2024-09-16T00:25:57.411 INFO:tasks.workunit.client.1.smithi169.stdout:0/170: mknod d0/d2/d8/c3d 0 2024-09-16T00:25:57.411 INFO:tasks.workunit.client.1.smithi169.stdout:8/174: stat d7/dc/d17/d24/f27 0 2024-09-16T00:25:57.412 INFO:tasks.workunit.client.1.smithi169.stdout:4/212: creat d2/dd/d1a/f41 x:0 0 0 2024-09-16T00:25:57.412 INFO:tasks.workunit.client.1.smithi169.stdout:8/175: dread - d7/f2e zero size 2024-09-16T00:25:57.412 INFO:tasks.workunit.client.1.smithi169.stdout:0/171: truncate d0/d2/d8/fe 1899403 0 2024-09-16T00:25:57.413 INFO:tasks.workunit.client.1.smithi169.stdout:3/222: rename d7/l44 to d7/d1a/d1c/d23/l4e 0 2024-09-16T00:25:57.416 INFO:tasks.workunit.client.1.smithi169.stdout:5/186: fsync d2/f8 0 2024-09-16T00:25:57.416 INFO:tasks.workunit.client.1.smithi169.stdout:0/172: stat d0/d2/d8/d5/d14/d16 0 2024-09-16T00:25:57.416 INFO:tasks.workunit.client.1.smithi169.stdout:9/223: unlink d5/da/f10 0 2024-09-16T00:25:57.418 INFO:tasks.workunit.client.1.smithi169.stdout:8/176: mkdir d7/dc/d17/d3b 0 2024-09-16T00:25:57.421 INFO:tasks.workunit.client.1.smithi169.stdout:3/223: dwrite d7/d1a/d1c/d23/f3c [0,4194304] 0 2024-09-16T00:25:57.421 INFO:tasks.workunit.client.1.smithi169.stdout:2/156: rename d7/d2e to d7/db/d27/d29/d37 0 2024-09-16T00:25:57.426 INFO:tasks.workunit.client.1.smithi169.stdout:0/173: unlink d0/f10 0 2024-09-16T00:25:57.427 INFO:tasks.workunit.client.1.smithi169.stdout:9/224: symlink d5/da/d21/d23/l54 0 2024-09-16T00:25:57.428 INFO:tasks.workunit.client.1.smithi169.stdout:1/209: dread f7 [0,4194304] 0 2024-09-16T00:25:57.429 INFO:tasks.workunit.client.1.smithi169.stdout:6/91: sync 2024-09-16T00:25:57.429 INFO:tasks.workunit.client.1.smithi169.stdout:7/259: sync 2024-09-16T00:25:57.431 INFO:tasks.workunit.client.1.smithi169.stdout:8/177: rename d7/dc/d17/d3b to d7/dc/d1d/d3c 0 2024-09-16T00:25:57.431 INFO:tasks.workunit.client.1.smithi169.stdout:7/260: chown d7/da/c52 42 1 2024-09-16T00:25:57.431 INFO:tasks.workunit.client.0.smithi112.stdout:3/446: sync 2024-09-16T00:25:57.431 INFO:tasks.workunit.client.0.smithi112.stdout:9/504: sync 2024-09-16T00:25:57.432 INFO:tasks.workunit.client.1.smithi169.stdout:7/261: fsync d7/f50 0 2024-09-16T00:25:57.432 INFO:tasks.workunit.client.1.smithi169.stdout:2/157: mknod d7/d1c/d26/c38 0 2024-09-16T00:25:57.434 INFO:tasks.workunit.client.1.smithi169.stdout:3/224: write d7/d1a/d1c/d23/f28 [2743129,55690] 0 2024-09-16T00:25:57.434 INFO:tasks.workunit.client.1.smithi169.stdout:7/262: chown d7/da 7 1 2024-09-16T00:25:57.434 INFO:tasks.workunit.client.0.smithi112.stdout:3/447: readlink d2/d9/d2b/l36 0 2024-09-16T00:25:57.435 INFO:tasks.workunit.client.0.smithi112.stdout:9/505: chown d1/d5/d20/d81/d1a/l56 0 1 2024-09-16T00:25:57.436 INFO:tasks.workunit.client.1.smithi169.stdout:9/225: mknod d5/da/d4e/c55 0 2024-09-16T00:25:57.437 INFO:tasks.workunit.client.0.smithi112.stdout:0/409: dwrite d5/da/d66/f69 [0,4194304] 0 2024-09-16T00:25:57.438 INFO:tasks.workunit.client.0.smithi112.stdout:0/410: stat d5/da/d18/d63 0 2024-09-16T00:25:57.438 INFO:tasks.workunit.client.1.smithi169.stdout:7/263: readlink d7/da/l1f 0 2024-09-16T00:25:57.438 INFO:tasks.workunit.client.1.smithi169.stdout:6/92: mkdir d2/d15/d1d 0 2024-09-16T00:25:57.438 INFO:tasks.workunit.client.1.smithi169.stdout:0/174: dwrite d0/d2/d8/d5/d14/f28 [0,4194304] 0 2024-09-16T00:25:57.439 INFO:tasks.workunit.client.1.smithi169.stdout:5/187: creat d2/da/d10/f3f x:0 0 0 2024-09-16T00:25:57.440 INFO:tasks.workunit.client.1.smithi169.stdout:1/210: mkdir dc/d36/d39 0 2024-09-16T00:25:57.442 INFO:tasks.workunit.client.1.smithi169.stdout:0/175: chown d0/d2/d15/l19 3 1 2024-09-16T00:25:57.443 INFO:tasks.workunit.client.1.smithi169.stdout:2/158: symlink d7/d1c/d26/l39 0 2024-09-16T00:25:57.444 INFO:tasks.workunit.client.1.smithi169.stdout:1/211: dread - dc/d26/f37 zero size 2024-09-16T00:25:57.444 INFO:tasks.workunit.client.1.smithi169.stdout:3/225: creat d7/d1a/d1c/f4f x:0 0 0 2024-09-16T00:25:57.444 INFO:tasks.workunit.client.1.smithi169.stdout:8/178: symlink d7/dc/l3d 0 2024-09-16T00:25:57.445 INFO:tasks.workunit.client.1.smithi169.stdout:0/176: read d0/d2/d8/fe [1591145,54123] 0 2024-09-16T00:25:57.449 INFO:tasks.workunit.client.1.smithi169.stdout:4/213: write d2/d12/d13/f25 [3006204,55069] 0 2024-09-16T00:25:57.450 INFO:tasks.workunit.client.1.smithi169.stdout:3/226: mkdir d7/d50 0 2024-09-16T00:25:57.452 INFO:tasks.workunit.client.1.smithi169.stdout:7/264: link f5 d7/d16/d49/f53 0 2024-09-16T00:25:57.453 INFO:tasks.workunit.client.1.smithi169.stdout:8/179: fdatasync d7/d20/f21 0 2024-09-16T00:25:57.455 INFO:tasks.workunit.client.1.smithi169.stdout:1/212: mkdir dc/d36/d3a 0 2024-09-16T00:25:57.456 INFO:tasks.workunit.client.1.smithi169.stdout:1/213: dread - dc/d26/f37 zero size 2024-09-16T00:25:57.457 INFO:tasks.workunit.client.0.smithi112.stdout:9/506: link d1/d5/d6/d59/d64/l99 d1/lb8 0 2024-09-16T00:25:57.458 INFO:tasks.workunit.client.0.smithi112.stdout:9/507: stat d1/d5/d6/d2d/f97 0 2024-09-16T00:25:57.458 INFO:tasks.workunit.client.1.smithi169.stdout:3/227: symlink d7/d1a/d1c/d23/d2a/l51 0 2024-09-16T00:25:57.459 INFO:tasks.workunit.client.1.smithi169.stdout:7/265: symlink d7/d16/l54 0 2024-09-16T00:25:57.460 INFO:tasks.workunit.client.0.smithi112.stdout:9/508: creat d1/d5/d83/fb9 x:0 0 0 2024-09-16T00:25:57.460 INFO:tasks.workunit.client.1.smithi169.stdout:4/214: dwrite d2/d12/d13/d27/f32 [0,4194304] 0 2024-09-16T00:25:57.460 INFO:tasks.workunit.client.1.smithi169.stdout:9/226: link d5/da/l20 d5/da/d21/d23/l56 0 2024-09-16T00:25:57.462 INFO:tasks.workunit.client.1.smithi169.stdout:5/188: rename d2/d30 to d2/da/d10/d13/d1c/d40 0 2024-09-16T00:25:57.463 INFO:tasks.workunit.client.1.smithi169.stdout:1/214: fsync dc/df/f16 0 2024-09-16T00:25:57.463 INFO:tasks.workunit.client.0.smithi112.stdout:0/411: rmdir d5/da/d18 39 2024-09-16T00:25:57.464 INFO:tasks.workunit.client.1.smithi169.stdout:8/180: mknod d7/dc/c3e 0 2024-09-16T00:25:57.464 INFO:tasks.workunit.client.1.smithi169.stdout:9/227: chown d5/da/d2e/d47 31391023 1 2024-09-16T00:25:57.465 INFO:tasks.workunit.client.0.smithi112.stdout:3/448: write d2/d9/d2b/f4b [1514530,113496] 0 2024-09-16T00:25:57.465 INFO:tasks.workunit.client.0.smithi112.stdout:1/368: write d0/d14/d1c/d20/f33 [1306756,65092] 0 2024-09-16T00:25:57.466 INFO:tasks.workunit.client.1.smithi169.stdout:1/215: chown dc/df 63801958 1 2024-09-16T00:25:57.469 INFO:tasks.workunit.client.0.smithi112.stdout:0/412: dread d5/da/d1d/d71/f4d [0,4194304] 0 2024-09-16T00:25:57.469 INFO:tasks.workunit.client.1.smithi169.stdout:8/181: truncate f2 4874033 0 2024-09-16T00:25:57.470 INFO:tasks.workunit.client.0.smithi112.stdout:1/369: write d0/f18 [2656976,46700] 0 2024-09-16T00:25:57.472 INFO:tasks.workunit.client.0.smithi112.stdout:0/413: rmdir d5/da/d1d/d71/d4c 39 2024-09-16T00:25:57.472 INFO:tasks.workunit.client.1.smithi169.stdout:8/182: fsync d7/f8 0 2024-09-16T00:25:57.475 INFO:tasks.workunit.client.1.smithi169.stdout:1/216: rename dc/df/f33 to dc/d18/f3b 0 2024-09-16T00:25:57.476 INFO:tasks.workunit.client.1.smithi169.stdout:6/93: truncate d2/fb 3878758 0 2024-09-16T00:25:57.478 INFO:tasks.workunit.client.0.smithi112.stdout:3/449: dwrite d2/d9/d2b/d47/d52/d7a/d80/d83/d49/f77 [0,4194304] 0 2024-09-16T00:25:57.479 INFO:tasks.workunit.client.1.smithi169.stdout:6/94: fdatasync d2/f19 0 2024-09-16T00:25:57.479 INFO:tasks.workunit.client.0.smithi112.stdout:1/370: dwrite d0/d14/d1c/f51 [0,4194304] 0 2024-09-16T00:25:57.480 INFO:tasks.workunit.client.0.smithi112.stdout:9/509: dread d1/d5/d6/d18/f4b [0,4194304] 0 2024-09-16T00:25:57.481 INFO:tasks.workunit.client.0.smithi112.stdout:0/414: rename d5/da/dc/f5b to d5/da/d18/d24/d5a/f82 0 2024-09-16T00:25:57.482 INFO:tasks.workunit.client.1.smithi169.stdout:3/228: dwrite d7/d1a/d1c/d23/d2a/f4a [0,4194304] 0 2024-09-16T00:25:57.484 INFO:tasks.workunit.client.1.smithi169.stdout:6/95: read d2/d15/f1b [1147499,118871] 0 2024-09-16T00:25:57.486 INFO:tasks.workunit.client.1.smithi169.stdout:8/183: mknod d7/dc/d2a/c3f 0 2024-09-16T00:25:57.486 INFO:tasks.workunit.client.1.smithi169.stdout:4/215: dwrite d2/d12/f2c [0,4194304] 0 2024-09-16T00:25:57.487 INFO:tasks.workunit.client.0.smithi112.stdout:1/371: dread d0/f9 [0,4194304] 0 2024-09-16T00:25:57.487 INFO:tasks.workunit.client.0.smithi112.stdout:0/415: unlink d5/da/d18/c52 0 2024-09-16T00:25:57.488 INFO:tasks.workunit.client.0.smithi112.stdout:0/416: chown d5/da/d66/f5d 135026359 1 2024-09-16T00:25:57.489 INFO:tasks.workunit.client.0.smithi112.stdout:9/510: truncate d1/d5/d2c/f5b 1198280 0 2024-09-16T00:25:57.490 INFO:tasks.workunit.client.0.smithi112.stdout:0/417: stat d5/da/d21/f7c 0 2024-09-16T00:25:57.491 INFO:tasks.workunit.client.0.smithi112.stdout:9/511: write d1/d5/d6/d2d/f92 [7512761,37119] 0 2024-09-16T00:25:57.492 INFO:tasks.workunit.client.0.smithi112.stdout:9/512: chown d1/d5/d6/d2d/f3e 635 1 2024-09-16T00:25:57.492 INFO:tasks.workunit.client.1.smithi169.stdout:9/228: symlink d5/da/d2e/d47/l57 0 2024-09-16T00:25:57.493 INFO:tasks.workunit.client.1.smithi169.stdout:5/189: creat d2/da/d10/d13/d1c/f41 x:0 0 0 2024-09-16T00:25:57.493 INFO:tasks.workunit.client.0.smithi112.stdout:9/513: dread - d1/d5/d6/fa4 zero size 2024-09-16T00:25:57.494 INFO:tasks.workunit.client.0.smithi112.stdout:0/418: write d5/da/d1d/d71/f3b [4919226,76454] 0 2024-09-16T00:25:57.495 INFO:tasks.workunit.client.1.smithi169.stdout:1/217: rename c6 to dc/df/d29/d2a/c3c 0 2024-09-16T00:25:57.495 INFO:tasks.workunit.client.1.smithi169.stdout:2/159: dwrite d7/d1c/d26/f36 [0,4194304] 0 2024-09-16T00:25:57.496 INFO:tasks.workunit.client.1.smithi169.stdout:5/190: fsync d2/f8 0 2024-09-16T00:25:57.502 INFO:tasks.workunit.client.0.smithi112.stdout:9/514: dread d1/d5/d6/d2d/d39/f65 [0,4194304] 0 2024-09-16T00:25:57.502 INFO:tasks.workunit.client.0.smithi112.stdout:9/515: write d1/d5/d6/d2d/f3e [4413218,2077] 0 2024-09-16T00:25:57.503 INFO:tasks.workunit.client.1.smithi169.stdout:6/96: creat d2/d15/f1e x:0 0 0 2024-09-16T00:25:57.503 INFO:tasks.workunit.client.1.smithi169.stdout:3/229: truncate d7/d1a/d1c/f4f 422329 0 2024-09-16T00:25:57.503 INFO:tasks.workunit.client.1.smithi169.stdout:0/177: dread d0/d2/fd [0,4194304] 0 2024-09-16T00:25:57.504 INFO:tasks.workunit.client.1.smithi169.stdout:4/216: rmdir d2/dd/d1a 39 2024-09-16T00:25:57.504 INFO:tasks.workunit.client.1.smithi169.stdout:2/160: write d7/db/f19 [5169141,69636] 0 2024-09-16T00:25:57.505 INFO:tasks.workunit.client.1.smithi169.stdout:8/184: readlink d7/dc/d26/l3a 0 2024-09-16T00:25:57.505 INFO:tasks.workunit.client.1.smithi169.stdout:6/97: write d2/f17 [377474,84683] 0 2024-09-16T00:25:57.506 INFO:tasks.workunit.client.1.smithi169.stdout:7/266: getdents d7/df 0 2024-09-16T00:25:57.506 INFO:tasks.workunit.client.1.smithi169.stdout:1/218: mkdir dc/d2c/d3d 0 2024-09-16T00:25:57.506 INFO:tasks.workunit.client.1.smithi169.stdout:2/161: unlink d7/db/l1e 0 2024-09-16T00:25:57.507 INFO:tasks.workunit.client.1.smithi169.stdout:1/219: fsync dc/df/f16 0 2024-09-16T00:25:57.508 INFO:tasks.workunit.client.1.smithi169.stdout:8/185: mknod d7/dc/d1d/d3c/c40 0 2024-09-16T00:25:57.510 INFO:tasks.workunit.client.1.smithi169.stdout:8/186: write d7/f8 [116910,40137] 0 2024-09-16T00:25:57.510 INFO:tasks.workunit.client.1.smithi169.stdout:7/267: mknod d7/d16/d49/c55 0 2024-09-16T00:25:57.510 INFO:tasks.workunit.client.1.smithi169.stdout:3/230: mknod d7/d1a/d1c/d23/d2a/d37/c52 0 2024-09-16T00:25:57.512 INFO:tasks.workunit.client.1.smithi169.stdout:8/187: dread - d7/dc/d17/f39 zero size 2024-09-16T00:25:57.512 INFO:tasks.workunit.client.1.smithi169.stdout:7/268: fsync d7/da/d3a/f3c 0 2024-09-16T00:25:57.512 INFO:tasks.workunit.client.0.smithi112.stdout:3/450: truncate d2/d9/d2b/d2c/f7e 2043605 0 2024-09-16T00:25:57.513 INFO:tasks.workunit.client.0.smithi112.stdout:3/451: write d2/d9/d2b/d47/f8a [567620,22914] 0 2024-09-16T00:25:57.514 INFO:tasks.workunit.client.0.smithi112.stdout:9/516: write d1/d5/d20/d25/d66/f9d [952757,64797] 0 2024-09-16T00:25:57.516 INFO:tasks.workunit.client.0.smithi112.stdout:9/517: rmdir d1/d5/d6/d59/d64 39 2024-09-16T00:25:57.518 INFO:tasks.workunit.client.0.smithi112.stdout:3/452: link d2/d9/d2b/c65 d2/d9/d2b/d39/c9f 0 2024-09-16T00:25:57.520 INFO:tasks.workunit.client.1.smithi169.stdout:3/231: unlink d7/d1a/f1b 0 2024-09-16T00:25:57.521 INFO:tasks.workunit.client.0.smithi112.stdout:9/518: getdents d1/d5/d20/d25 0 2024-09-16T00:25:57.521 INFO:tasks.workunit.client.1.smithi169.stdout:7/269: mknod d7/da/c56 0 2024-09-16T00:25:57.521 INFO:tasks.workunit.client.1.smithi169.stdout:8/188: dread - d7/dc/d17/d24/f27 zero size 2024-09-16T00:25:57.522 INFO:tasks.workunit.client.0.smithi112.stdout:9/519: write d1/d5/d6/fb4 [3273183,65066] 0 2024-09-16T00:25:57.522 INFO:tasks.workunit.client.1.smithi169.stdout:8/189: stat d7/dc/d2a/f30 0 2024-09-16T00:25:57.524 INFO:tasks.workunit.client.1.smithi169.stdout:3/232: creat d7/d1a/d1c/d23/d2a/f53 x:0 0 0 2024-09-16T00:25:57.525 INFO:tasks.workunit.client.1.smithi169.stdout:5/191: dwrite d2/da/d10/d13/f14 [4194304,4194304] 0 2024-09-16T00:25:57.525 INFO:tasks.workunit.client.0.smithi112.stdout:9/520: mkdir d1/d5/d20/d25/da9/dba 0 2024-09-16T00:25:57.528 INFO:tasks.workunit.client.1.smithi169.stdout:8/190: mkdir d7/dc/d26/d41 0 2024-09-16T00:25:57.529 INFO:tasks.workunit.client.1.smithi169.stdout:7/270: dread d7/f2c [0,4194304] 0 2024-09-16T00:25:57.530 INFO:tasks.workunit.client.1.smithi169.stdout:0/178: write d0/d2/fd [1154560,116868] 0 2024-09-16T00:25:57.532 INFO:tasks.workunit.client.1.smithi169.stdout:5/192: mknod d2/da/d10/d13/d1c/d32/c42 0 2024-09-16T00:25:57.533 INFO:tasks.workunit.client.0.smithi112.stdout:9/521: dwrite d1/d5/d2c/d6f/f91 [0,4194304] 0 2024-09-16T00:25:57.534 INFO:tasks.workunit.client.1.smithi169.stdout:3/233: rename d7/d1a/d1c/d23/f2f to d7/d1a/d1c/f54 0 2024-09-16T00:25:57.534 INFO:tasks.workunit.client.0.smithi112.stdout:3/453: dread d2/d9/f16 [0,4194304] 0 2024-09-16T00:25:57.535 INFO:tasks.workunit.client.1.smithi169.stdout:9/229: write d5/da/d21/d23/f2d [280685,94509] 0 2024-09-16T00:25:57.537 INFO:tasks.workunit.client.1.smithi169.stdout:0/179: dread d0/d2/d15/f22 [0,4194304] 0 2024-09-16T00:25:57.538 INFO:tasks.workunit.client.0.smithi112.stdout:3/454: creat d2/d9/d2b/fa0 x:0 0 0 2024-09-16T00:25:57.540 INFO:tasks.workunit.client.1.smithi169.stdout:5/193: mkdir d2/da/d10/d13/d1c/d40/d43 0 2024-09-16T00:25:57.540 INFO:tasks.workunit.client.0.smithi112.stdout:9/522: dread d1/d5/d20/d25/f2e [0,4194304] 0 2024-09-16T00:25:57.541 INFO:tasks.workunit.client.1.smithi169.stdout:7/271: rename d7/da/c52 to d7/d16/d49/c57 0 2024-09-16T00:25:57.542 INFO:tasks.workunit.client.1.smithi169.stdout:2/162: dwrite d7/db/f23 [4194304,4194304] 0 2024-09-16T00:25:57.543 INFO:tasks.workunit.client.1.smithi169.stdout:9/230: mkdir d5/d49/d58 0 2024-09-16T00:25:57.544 INFO:tasks.workunit.client.0.smithi112.stdout:9/523: mknod d1/d5/d6/d2d/d6a/d89/cbb 0 2024-09-16T00:25:57.545 INFO:tasks.workunit.client.1.smithi169.stdout:7/272: fsync f5 0 2024-09-16T00:25:57.546 INFO:tasks.workunit.client.1.smithi169.stdout:5/194: read d2/da/d10/d13/f35 [3087527,127551] 0 2024-09-16T00:25:57.547 INFO:tasks.workunit.client.1.smithi169.stdout:0/180: mknod d0/d2/d8/d5/c3e 0 2024-09-16T00:25:57.548 INFO:tasks.workunit.client.0.smithi112.stdout:9/524: unlink d1/d5/d6/c52 0 2024-09-16T00:25:57.549 INFO:tasks.workunit.client.0.smithi112.stdout:9/525: readlink d1/d5/d6/d18/l37 0 2024-09-16T00:25:57.551 INFO:tasks.workunit.client.1.smithi169.stdout:5/195: chown d2/da/f38 2 1 2024-09-16T00:25:57.551 INFO:tasks.workunit.client.1.smithi169.stdout:7/273: creat d7/d16/d49/f58 x:0 0 0 2024-09-16T00:25:57.553 INFO:tasks.workunit.client.0.smithi112.stdout:9/526: creat d1/d5/d6/d2d/fbc x:0 0 0 2024-09-16T00:25:57.553 INFO:tasks.workunit.client.1.smithi169.stdout:8/191: dwrite d7/dc/d17/f1b [0,4194304] 0 2024-09-16T00:25:57.554 INFO:tasks.workunit.client.0.smithi112.stdout:9/527: unlink d1/d5/d6/fb4 0 2024-09-16T00:25:57.555 INFO:tasks.workunit.client.0.smithi112.stdout:9/528: chown d1/d5/d20/d25/d66 11 1 2024-09-16T00:25:57.556 INFO:tasks.workunit.client.1.smithi169.stdout:0/181: creat d0/d2/d8/d5/d14/d21/f3f x:0 0 0 2024-09-16T00:25:57.559 INFO:tasks.workunit.client.1.smithi169.stdout:2/163: dread d7/db/f23 [4194304,4194304] 0 2024-09-16T00:25:57.560 INFO:tasks.workunit.client.1.smithi169.stdout:9/231: dwrite d5/f4a [0,4194304] 0 2024-09-16T00:25:57.564 INFO:tasks.workunit.client.1.smithi169.stdout:9/232: write d5/da/d21/f30 [277954,70996] 0 2024-09-16T00:25:57.569 INFO:tasks.workunit.client.1.smithi169.stdout:7/274: dread d7/f17 [0,4194304] 0 2024-09-16T00:25:57.572 INFO:tasks.workunit.client.1.smithi169.stdout:9/233: creat d5/d49/f59 x:0 0 0 2024-09-16T00:25:57.573 INFO:tasks.workunit.client.0.smithi112.stdout:3/455: write d2/d9/d2b/d2c/d69/f8d [2771548,6647] 0 2024-09-16T00:25:57.574 INFO:tasks.workunit.client.1.smithi169.stdout:3/234: dread d7/d1a/d1c/f54 [4194304,4194304] 0 2024-09-16T00:25:57.574 INFO:tasks.workunit.client.0.smithi112.stdout:9/529: fsync d1/d5/d6/d2d/fbc 0 2024-09-16T00:25:57.575 INFO:tasks.workunit.client.1.smithi169.stdout:0/182: dwrite d0/d2/d8/d5/d14/d21/f2a [0,4194304] 0 2024-09-16T00:25:57.576 INFO:tasks.workunit.client.0.smithi112.stdout:3/456: symlink d2/d9/d2b/d2c/d69/la1 0 2024-09-16T00:25:57.577 INFO:tasks.workunit.client.1.smithi169.stdout:9/234: dread - d5/da/d21/f44 zero size 2024-09-16T00:25:57.579 INFO:tasks.workunit.client.1.smithi169.stdout:7/275: truncate f5 2243932 0 2024-09-16T00:25:57.581 INFO:tasks.workunit.client.1.smithi169.stdout:2/164: dwrite d7/db/f13 [0,4194304] 0 2024-09-16T00:25:57.582 INFO:tasks.workunit.client.1.smithi169.stdout:2/165: chown d7/l25 673244 1 2024-09-16T00:25:57.582 INFO:tasks.workunit.client.0.smithi112.stdout:3/457: dwrite d2/d9/f4e [0,4194304] 0 2024-09-16T00:25:57.583 INFO:tasks.workunit.client.1.smithi169.stdout:0/183: chown d0/d2/d8/d5/d14/f28 38 1 2024-09-16T00:25:57.584 INFO:tasks.workunit.client.0.smithi112.stdout:3/458: mknod d2/d9/ca2 0 2024-09-16T00:25:57.585 INFO:tasks.workunit.client.1.smithi169.stdout:9/235: rename d5/d12/f13 to d5/d49/f5a 0 2024-09-16T00:25:57.585 INFO:tasks.workunit.client.1.smithi169.stdout:2/166: symlink d7/d1c/d26/l3a 0 2024-09-16T00:25:57.586 INFO:tasks.workunit.client.0.smithi112.stdout:3/459: truncate d2/f97 193462 0 2024-09-16T00:25:57.587 INFO:tasks.workunit.client.0.smithi112.stdout:3/460: dread - d2/d9/d2b/fa0 zero size 2024-09-16T00:25:57.589 INFO:tasks.workunit.client.1.smithi169.stdout:2/167: read d7/f24 [741837,115278] 0 2024-09-16T00:25:57.589 INFO:tasks.workunit.client.1.smithi169.stdout:3/235: mknod d7/d50/c55 0 2024-09-16T00:25:57.589 INFO:tasks.workunit.client.1.smithi169.stdout:0/184: creat d0/d2/d8/d5/d14/d16/f40 x:0 0 0 2024-09-16T00:25:57.590 INFO:tasks.workunit.client.0.smithi112.stdout:3/461: creat d2/d9/d2b/d47/d52/fa3 x:0 0 0 2024-09-16T00:25:57.591 INFO:tasks.workunit.client.1.smithi169.stdout:9/236: rmdir d5/da/d21/d23 39 2024-09-16T00:25:57.591 INFO:tasks.workunit.client.1.smithi169.stdout:2/168: write d7/db/d20/d21/f33 [4999414,25288] 0 2024-09-16T00:25:57.592 INFO:tasks.workunit.client.0.smithi112.stdout:9/530: creat d1/d5/d20/d81/d1a/fbd x:0 0 0 2024-09-16T00:25:57.593 INFO:tasks.workunit.client.1.smithi169.stdout:3/236: readlink d7/d1a/d2b/l3b 0 2024-09-16T00:25:57.594 INFO:tasks.workunit.client.1.smithi169.stdout:2/169: truncate d7/d1c/d26/f2c 35612 0 2024-09-16T00:25:57.594 INFO:tasks.workunit.client.1.smithi169.stdout:3/237: stat d7/d1a/d2b/l3b 0 2024-09-16T00:25:57.595 INFO:tasks.workunit.client.1.smithi169.stdout:0/185: mkdir d0/d2/d15/d41 0 2024-09-16T00:25:57.596 INFO:tasks.workunit.client.1.smithi169.stdout:3/238: write d7/d1a/d1c/d23/d2a/f4a [1293181,55165] 0 2024-09-16T00:25:57.597 INFO:tasks.workunit.client.1.smithi169.stdout:7/276: rename d7/d16/d49/l32 to d7/da/d3a/l59 0 2024-09-16T00:25:57.598 INFO:tasks.workunit.client.0.smithi112.stdout:5/341: sync 2024-09-16T00:25:57.598 INFO:tasks.workunit.client.0.smithi112.stdout:4/309: sync 2024-09-16T00:25:57.598 INFO:tasks.workunit.client.0.smithi112.stdout:9/531: dwrite d1/d5/d20/f31 [4194304,4194304] 0 2024-09-16T00:25:57.599 INFO:tasks.workunit.client.0.smithi112.stdout:9/532: fsync d1/f82 0 2024-09-16T00:25:57.600 INFO:tasks.workunit.client.0.smithi112.stdout:3/462: dread d2/d9/d2b/f4b [0,4194304] 0 2024-09-16T00:25:57.600 INFO:tasks.workunit.client.1.smithi169.stdout:9/237: dread - d5/da/d21/d23/f3b zero size 2024-09-16T00:25:57.600 INFO:tasks.workunit.client.1.smithi169.stdout:7/277: symlink d7/da/l5a 0 2024-09-16T00:25:57.602 INFO:tasks.workunit.client.1.smithi169.stdout:3/239: rename d7/c17 to d7/d1a/d1c/d23/d2a/c56 0 2024-09-16T00:25:57.603 INFO:tasks.workunit.client.0.smithi112.stdout:9/533: mkdir d1/d5/d2c/d6f/dbe 0 2024-09-16T00:25:57.605 INFO:tasks.workunit.client.0.smithi112.stdout:4/310: mkdir d6/d39/d1c/d6e/d6f 0 2024-09-16T00:25:57.606 INFO:tasks.workunit.client.0.smithi112.stdout:3/463: rename d2/c4a to d2/d9/d2b/d47/d52/d7a/d80/d83/d7b/ca4 0 2024-09-16T00:25:57.606 INFO:tasks.workunit.client.1.smithi169.stdout:2/170: dwrite f5 [0,4194304] 0 2024-09-16T00:25:57.607 INFO:tasks.workunit.client.0.smithi112.stdout:9/534: mkdir d1/d5/d2c/d6f/dbf 0 2024-09-16T00:25:57.607 INFO:tasks.workunit.client.0.smithi112.stdout:3/464: read d2/f97 [117152,24968] 0 2024-09-16T00:25:57.608 INFO:tasks.workunit.client.1.smithi169.stdout:3/240: chown d7/d1a/d1c/d35/f49 7552 1 2024-09-16T00:25:57.609 INFO:tasks.workunit.client.0.smithi112.stdout:5/342: dwrite d4/dd/f24 [0,4194304] 0 2024-09-16T00:25:57.610 INFO:tasks.workunit.client.0.smithi112.stdout:4/311: truncate d6/d39/d1c/f57 74537 0 2024-09-16T00:25:57.612 INFO:tasks.workunit.client.1.smithi169.stdout:2/171: chown d7/db/c32 7 1 2024-09-16T00:25:57.612 INFO:tasks.workunit.client.0.smithi112.stdout:5/343: mkdir d4/d4f/d78 0 2024-09-16T00:25:57.614 INFO:tasks.workunit.client.1.smithi169.stdout:7/278: dread d7/f4c [4194304,4194304] 0 2024-09-16T00:25:57.614 INFO:tasks.workunit.client.0.smithi112.stdout:9/535: chown d1/d5/d20/d81/lb 1340236 1 2024-09-16T00:25:57.616 INFO:tasks.workunit.client.1.smithi169.stdout:0/186: dwrite d0/d2/d8/d5/d14/f28 [0,4194304] 0 2024-09-16T00:25:57.616 INFO:tasks.workunit.client.0.smithi112.stdout:4/312: unlink d6/d39/d1c/d2c/c46 0 2024-09-16T00:25:57.617 INFO:tasks.workunit.client.1.smithi169.stdout:7/279: write d7/d16/f1d [1076692,79951] 0 2024-09-16T00:25:57.618 INFO:tasks.workunit.client.0.smithi112.stdout:5/344: mkdir d4/dd/d12/d79 0 2024-09-16T00:25:57.618 INFO:tasks.workunit.client.1.smithi169.stdout:9/238: rename d5/d49/d58 to d5/da/d5b 0 2024-09-16T00:25:57.618 INFO:tasks.workunit.client.1.smithi169.stdout:0/187: write d0/d2/d8/f37 [332368,92866] 0 2024-09-16T00:25:57.619 INFO:tasks.workunit.client.0.smithi112.stdout:3/465: creat d2/d9/fa5 x:0 0 0 2024-09-16T00:25:57.619 INFO:tasks.workunit.client.0.smithi112.stdout:5/345: chown d4/d1c 437141 1 2024-09-16T00:25:57.622 INFO:tasks.workunit.client.1.smithi169.stdout:2/172: creat d7/db/d20/f3b x:0 0 0 2024-09-16T00:25:57.624 INFO:tasks.workunit.client.1.smithi169.stdout:0/188: rmdir d0/d2/d15/d41 0 2024-09-16T00:25:57.626 INFO:tasks.workunit.client.1.smithi169.stdout:7/280: dread d7/df/f21 [0,4194304] 0 2024-09-16T00:25:57.626 INFO:tasks.workunit.client.1.smithi169.stdout:2/173: mkdir d7/d1c/d3c 0 2024-09-16T00:25:57.627 INFO:tasks.workunit.client.0.smithi112.stdout:3/466: unlink d2/d9/d2b/d2c/d69/f86 0 2024-09-16T00:25:57.629 INFO:tasks.workunit.client.0.smithi112.stdout:9/536: dwrite d1/d5/d6/d38/f7b [0,4194304] 0 2024-09-16T00:25:57.631 INFO:tasks.workunit.client.1.smithi169.stdout:2/174: creat d7/db/f3d x:0 0 0 2024-09-16T00:25:57.631 INFO:tasks.workunit.client.1.smithi169.stdout:9/239: dwrite d5/d49/f59 [0,4194304] 0 2024-09-16T00:25:57.633 INFO:tasks.workunit.client.1.smithi169.stdout:0/189: symlink d0/d2/d8/d5/d14/d16/d1a/l42 0 2024-09-16T00:25:57.633 INFO:tasks.workunit.client.0.smithi112.stdout:5/346: link d4/dd/d6f/l75 d4/d7/d26/d4c/l7a 0 2024-09-16T00:25:57.633 INFO:tasks.workunit.client.0.smithi112.stdout:4/313: dwrite d6/d39/d36/d5c/f63 [0,4194304] 0 2024-09-16T00:25:57.634 INFO:tasks.workunit.client.1.smithi169.stdout:9/240: mknod d5/da/d2e/c5c 0 2024-09-16T00:25:57.636 INFO:tasks.workunit.client.0.smithi112.stdout:9/537: write d1/d5/d6/d2d/fbc [79487,14868] 0 2024-09-16T00:25:57.637 INFO:tasks.workunit.client.1.smithi169.stdout:3/241: dwrite d7/f10 [0,4194304] 0 2024-09-16T00:25:57.639 INFO:tasks.workunit.client.1.smithi169.stdout:0/190: creat d0/d2/d8/d5/d14/d16/f43 x:0 0 0 2024-09-16T00:25:57.639 INFO:tasks.workunit.client.0.smithi112.stdout:5/347: symlink d4/d1c/l7b 0 2024-09-16T00:25:57.641 INFO:tasks.workunit.client.0.smithi112.stdout:9/538: dread - d1/d5/d20/d25/d66/faf zero size 2024-09-16T00:25:57.642 INFO:tasks.workunit.client.1.smithi169.stdout:9/241: creat d5/d12/f5d x:0 0 0 2024-09-16T00:25:57.644 INFO:tasks.workunit.client.0.smithi112.stdout:3/467: dwrite d2/f2d [0,4194304] 0 2024-09-16T00:25:57.644 INFO:tasks.workunit.client.1.smithi169.stdout:3/242: unlink d7/d1a/d1c/c27 0 2024-09-16T00:25:57.645 INFO:tasks.workunit.client.1.smithi169.stdout:0/191: creat d0/d2/d8/d5/f44 x:0 0 0 2024-09-16T00:25:57.645 INFO:tasks.workunit.client.1.smithi169.stdout:9/242: mknod d5/da/d21/d23/c5e 0 2024-09-16T00:25:57.645 INFO:tasks.workunit.client.0.smithi112.stdout:9/539: fsync d1/d5/d6/d2d/d6a/f6e 0 2024-09-16T00:25:57.646 INFO:tasks.workunit.client.0.smithi112.stdout:5/348: mkdir d4/d7/d26/d3e/d7c 0 2024-09-16T00:25:57.650 INFO:tasks.workunit.client.0.smithi112.stdout:4/314: write d6/db/d1f/d41/d49/f17 [1048485,77980] 0 2024-09-16T00:25:57.651 INFO:tasks.workunit.client.0.smithi112.stdout:3/468: symlink d2/d9/d2b/d47/la6 0 2024-09-16T00:25:57.652 INFO:tasks.workunit.client.1.smithi169.stdout:3/243: link d7/d1a/d1c/d23/d2a/d37/l41 d7/d1a/d2b/l57 0 2024-09-16T00:25:57.653 INFO:tasks.workunit.client.1.smithi169.stdout:0/192: dwrite d0/d2/f6 [0,4194304] 0 2024-09-16T00:25:57.653 INFO:tasks.workunit.client.0.smithi112.stdout:9/540: mknod d1/d5/d6/d18/cc0 0 2024-09-16T00:25:57.655 INFO:tasks.workunit.client.1.smithi169.stdout:3/244: write d7/d1a/d1c/d23/d2a/f53 [610439,17881] 0 2024-09-16T00:25:57.655 INFO:tasks.workunit.client.0.smithi112.stdout:4/315: fdatasync d6/db/d1f/d41/d49/f4c 0 2024-09-16T00:25:57.657 INFO:tasks.workunit.client.0.smithi112.stdout:5/349: dwrite d4/d7/f4d [0,4194304] 0 2024-09-16T00:25:57.657 INFO:tasks.workunit.client.0.smithi112.stdout:4/316: dread - d6/d39/f4a zero size 2024-09-16T00:25:57.658 INFO:tasks.workunit.client.0.smithi112.stdout:4/317: chown d6/da 995507081 1 2024-09-16T00:25:57.659 INFO:tasks.workunit.client.1.smithi169.stdout:3/245: mknod d7/d50/c58 0 2024-09-16T00:25:57.660 INFO:tasks.workunit.client.0.smithi112.stdout:5/350: symlink d4/d7/d26/l7d 0 2024-09-16T00:25:57.661 INFO:tasks.workunit.client.0.smithi112.stdout:9/541: symlink d1/d5/d6/d2d/lc1 0 2024-09-16T00:25:57.661 INFO:tasks.workunit.client.1.smithi169.stdout:0/193: creat d0/d2/f45 x:0 0 0 2024-09-16T00:25:57.665 INFO:tasks.workunit.client.1.smithi169.stdout:3/246: rename d7/d1a/d1c/d23/f28 to d7/d1a/d1c/d23/d2a/d37/f59 0 2024-09-16T00:25:57.665 INFO:tasks.workunit.client.0.smithi112.stdout:4/318: rename d6/db/df/d5e/d61/c6c to d6/db/d1f/d41/d49/d31/c70 0 2024-09-16T00:25:57.668 INFO:tasks.workunit.client.1.smithi169.stdout:0/194: link d0/d2/d8/d5/d14/d16/l33 d0/d2c/d38/l46 0 2024-09-16T00:25:57.669 INFO:tasks.workunit.client.0.smithi112.stdout:5/351: dwrite d4/f5c [0,4194304] 0 2024-09-16T00:25:57.671 INFO:tasks.workunit.client.1.smithi169.stdout:0/195: rmdir d0/d2/d8/d5/d14/d21 39 2024-09-16T00:25:57.673 INFO:tasks.workunit.client.0.smithi112.stdout:4/319: getdents d6/db/df/d5e 0 2024-09-16T00:25:57.673 INFO:tasks.workunit.client.1.smithi169.stdout:3/247: rename d7/d50/c58 to d7/d1a/c5a 0 2024-09-16T00:25:57.674 INFO:tasks.workunit.client.0.smithi112.stdout:9/542: dread d1/d5/d20/d81/f1d [0,4194304] 0 2024-09-16T00:25:57.674 INFO:tasks.workunit.client.0.smithi112.stdout:5/352: rename d4/d4f/d77/f68 to d4/dd/d49/d70/f7e 0 2024-09-16T00:25:57.676 INFO:tasks.workunit.client.1.smithi169.stdout:3/248: symlink d7/d1a/d1c/l5b 0 2024-09-16T00:25:57.676 INFO:tasks.workunit.client.0.smithi112.stdout:4/320: creat d6/db/d1f/f71 x:0 0 0 2024-09-16T00:25:57.677 INFO:tasks.workunit.client.1.smithi169.stdout:3/249: stat d7/d1a/l48 0 2024-09-16T00:25:57.678 INFO:tasks.workunit.client.0.smithi112.stdout:5/353: chown d4/dd/d6f/l75 6604599 1 2024-09-16T00:25:57.678 INFO:tasks.workunit.client.1.smithi169.stdout:0/196: dread d0/d2/d8/d5/d14/f28 [0,4194304] 0 2024-09-16T00:25:57.680 INFO:tasks.workunit.client.1.smithi169.stdout:3/250: chown d7/d1a/d1c/d23/d2a/d37/l4b 4 1 2024-09-16T00:25:57.680 INFO:tasks.workunit.client.0.smithi112.stdout:9/543: dwrite d1/d5/d6/d38/f7b [0,4194304] 0 2024-09-16T00:25:57.681 INFO:tasks.workunit.client.1.smithi169.stdout:3/251: chown d7/d1a/f32 70440552 1 2024-09-16T00:25:57.681 INFO:tasks.workunit.client.1.smithi169.stdout:0/197: truncate d0/d2/d15/f22 1229580 0 2024-09-16T00:25:57.682 INFO:tasks.workunit.client.1.smithi169.stdout:3/252: fsync d7/d1a/d1c/d23/f4c 0 2024-09-16T00:25:57.684 INFO:tasks.workunit.client.0.smithi112.stdout:9/544: mkdir d1/d5/d6/d59/d64/dc2 0 2024-09-16T00:25:57.685 INFO:tasks.workunit.client.0.smithi112.stdout:9/545: chown d1/d5/d6/l51 62207 1 2024-09-16T00:25:57.686 INFO:tasks.workunit.client.1.smithi169.stdout:3/253: rename d7/d50/c55 to d7/c5c 0 2024-09-16T00:25:57.687 INFO:tasks.workunit.client.1.smithi169.stdout:0/198: creat d0/d2/d8/f47 x:0 0 0 2024-09-16T00:25:57.689 INFO:tasks.workunit.client.1.smithi169.stdout:2/175: dwrite f3 [0,4194304] 0 2024-09-16T00:25:57.691 INFO:tasks.workunit.client.0.smithi112.stdout:9/546: dwrite d1/d5/d6/f34 [0,4194304] 0 2024-09-16T00:25:57.694 INFO:tasks.workunit.client.0.smithi112.stdout:9/547: fsync d1/d5/d20/d81/fb6 0 2024-09-16T00:25:57.695 INFO:tasks.workunit.client.1.smithi169.stdout:9/243: truncate d5/da/d21/d23/f27 1860060 0 2024-09-16T00:25:57.697 INFO:tasks.workunit.client.0.smithi112.stdout:9/548: write d1/d5/d2c/d6f/f91 [4596691,108782] 0 2024-09-16T00:25:57.697 INFO:tasks.workunit.client.1.smithi169.stdout:3/254: dwrite d7/f10 [0,4194304] 0 2024-09-16T00:25:57.698 INFO:tasks.workunit.client.0.smithi112.stdout:9/549: write d1/f41 [1215908,78151] 0 2024-09-16T00:25:57.698 INFO:tasks.workunit.client.1.smithi169.stdout:7/281: write d7/d16/d49/f53 [3078144,54126] 0 2024-09-16T00:25:57.698 INFO:tasks.workunit.client.1.smithi169.stdout:0/199: creat d0/d2/d8/f48 x:0 0 0 2024-09-16T00:25:57.701 INFO:tasks.workunit.client.0.smithi112.stdout:9/550: creat d1/d5/d2c/fc3 x:0 0 0 2024-09-16T00:25:57.704 INFO:tasks.workunit.client.1.smithi169.stdout:3/255: symlink d7/d1a/d2b/l5d 0 2024-09-16T00:25:57.710 INFO:tasks.workunit.client.1.smithi169.stdout:3/256: fdatasync d7/d1a/d1c/f54 0 2024-09-16T00:25:57.712 INFO:tasks.workunit.client.1.smithi169.stdout:9/244: dwrite d5/da/d21/d23/f3b [0,4194304] 0 2024-09-16T00:25:57.712 INFO:tasks.workunit.client.1.smithi169.stdout:2/176: dwrite d7/db/d20/d21/f22 [0,4194304] 0 2024-09-16T00:25:57.712 INFO:tasks.workunit.client.1.smithi169.stdout:7/282: mkdir d7/d5b 0 2024-09-16T00:25:57.712 INFO:tasks.workunit.client.0.smithi112.stdout:4/321: dread d6/db/f18 [0,4194304] 0 2024-09-16T00:25:57.712 INFO:tasks.workunit.client.0.smithi112.stdout:4/322: stat d6/da/c30 0 2024-09-16T00:25:57.713 INFO:tasks.workunit.client.0.smithi112.stdout:3/469: write d2/f2d [4880734,100201] 0 2024-09-16T00:25:57.714 INFO:tasks.workunit.client.1.smithi169.stdout:3/257: read d7/f10 [3837746,16138] 0 2024-09-16T00:25:57.715 INFO:tasks.workunit.client.1.smithi169.stdout:0/200: dread d0/d2/fd [0,4194304] 0 2024-09-16T00:25:57.716 INFO:tasks.workunit.client.0.smithi112.stdout:4/323: creat d6/d39/d36/f72 x:0 0 0 2024-09-16T00:25:57.716 INFO:tasks.workunit.client.1.smithi169.stdout:2/177: creat d7/d1c/f3e x:0 0 0 2024-09-16T00:25:57.717 INFO:tasks.workunit.client.0.smithi112.stdout:3/470: write d2/d9/d2b/f9a [2315317,7754] 0 2024-09-16T00:25:57.720 INFO:tasks.workunit.client.1.smithi169.stdout:0/201: read d0/d2/d15/f22 [191391,41274] 0 2024-09-16T00:25:57.720 INFO:tasks.workunit.client.0.smithi112.stdout:4/324: creat d6/db/df/d3c/f73 x:0 0 0 2024-09-16T00:25:57.720 INFO:tasks.workunit.client.1.smithi169.stdout:7/283: creat d7/d5b/f5c x:0 0 0 2024-09-16T00:25:57.724 INFO:tasks.workunit.client.1.smithi169.stdout:2/178: dread d7/db/d20/d21/f22 [0,4194304] 0 2024-09-16T00:25:57.725 INFO:tasks.workunit.client.1.smithi169.stdout:9/245: creat d5/d12/f5f x:0 0 0 2024-09-16T00:25:57.726 INFO:tasks.workunit.client.1.smithi169.stdout:2/179: chown d7/d1c/d26/c38 890701898 1 2024-09-16T00:25:57.734 INFO:tasks.workunit.client.1.smithi169.stdout:0/202: creat d0/d2/d8/d5/d14/d21/f49 x:0 0 0 2024-09-16T00:25:57.734 INFO:tasks.workunit.client.1.smithi169.stdout:7/284: link d7/d16/f1d d7/df/f5d 0 2024-09-16T00:25:57.735 INFO:tasks.workunit.client.1.smithi169.stdout:2/180: mknod d7/db/d27/d29/c3f 0 2024-09-16T00:25:57.736 INFO:tasks.workunit.client.0.smithi112.stdout:9/551: dread d1/d5/d20/f31 [0,4194304] 0 2024-09-16T00:25:57.738 INFO:tasks.workunit.client.1.smithi169.stdout:2/181: mknod d7/d1c/c40 0 2024-09-16T00:25:57.739 INFO:tasks.workunit.client.0.smithi112.stdout:9/552: rmdir d1/d5/d6/d59/d64/dc2 0 2024-09-16T00:25:57.740 INFO:tasks.workunit.client.0.smithi112.stdout:9/553: write d1/d5/d6/d18/f4b [3733412,27523] 0 2024-09-16T00:25:57.741 INFO:tasks.workunit.client.0.smithi112.stdout:9/554: chown d1/d5/d6/d59/c95 37969304 1 2024-09-16T00:25:57.743 INFO:tasks.workunit.client.1.smithi169.stdout:2/182: write d7/f24 [1575235,92062] 0 2024-09-16T00:25:57.743 INFO:tasks.workunit.client.0.smithi112.stdout:9/555: readlink d1/d5/l5e 0 2024-09-16T00:25:57.745 INFO:tasks.workunit.client.0.smithi112.stdout:9/556: fsync d1/d5/d20/d81/f98 0 2024-09-16T00:25:57.746 INFO:tasks.workunit.client.1.smithi169.stdout:7/285: dread d7/da/d29/f47 [0,4194304] 0 2024-09-16T00:25:57.748 INFO:tasks.workunit.client.1.smithi169.stdout:0/203: dwrite d0/d2/d8/d5/d14/d16/f40 [0,4194304] 0 2024-09-16T00:25:57.750 INFO:tasks.workunit.client.1.smithi169.stdout:3/258: dread d7/d1a/d1c/d23/d2a/d37/f59 [0,4194304] 0 2024-09-16T00:25:57.760 INFO:tasks.workunit.client.1.smithi169.stdout:0/204: rmdir d0/d2/d8 39 2024-09-16T00:25:57.760 INFO:tasks.workunit.client.1.smithi169.stdout:3/259: rename d7/d1a/d1c/d23/d2a/d37/c52 to d7/d1a/d1c/d23/d2a/d37/c5e 0 2024-09-16T00:25:57.762 INFO:tasks.workunit.client.1.smithi169.stdout:0/205: chown d0/c12 99372 1 2024-09-16T00:25:57.764 INFO:tasks.workunit.client.1.smithi169.stdout:3/260: creat d7/d1a/d1c/d23/d2a/f5f x:0 0 0 2024-09-16T00:25:57.766 INFO:tasks.workunit.client.1.smithi169.stdout:0/206: creat d0/d2c/f4a x:0 0 0 2024-09-16T00:25:57.767 INFO:tasks.workunit.client.1.smithi169.stdout:3/261: symlink d7/d50/l60 0 2024-09-16T00:25:57.768 INFO:tasks.workunit.client.1.smithi169.stdout:7/286: dwrite d7/d16/f1d [0,4194304] 0 2024-09-16T00:25:57.770 INFO:tasks.workunit.client.1.smithi169.stdout:0/207: dread - d0/d2/d8/d5/f44 zero size 2024-09-16T00:25:57.770 INFO:tasks.workunit.client.1.smithi169.stdout:7/287: dread d7/df/f21 [0,4194304] 0 2024-09-16T00:25:57.771 INFO:tasks.workunit.client.1.smithi169.stdout:3/262: creat d7/d1a/d2b/f61 x:0 0 0 2024-09-16T00:25:57.777 INFO:tasks.workunit.client.1.smithi169.stdout:9/246: dread d5/da/f34 [0,4194304] 0 2024-09-16T00:25:57.777 INFO:tasks.workunit.client.1.smithi169.stdout:0/208: mkdir d0/d2/d4b 0 2024-09-16T00:25:57.779 INFO:tasks.workunit.client.1.smithi169.stdout:9/247: write d5/d12/f1c [2022789,101413] 0 2024-09-16T00:25:57.779 INFO:tasks.workunit.client.1.smithi169.stdout:0/209: mkdir d0/d4c 0 2024-09-16T00:25:57.780 INFO:tasks.workunit.client.1.smithi169.stdout:3/263: link d7/d1a/c1d d7/d1a/d2b/c62 0 2024-09-16T00:25:57.780 INFO:tasks.workunit.client.1.smithi169.stdout:7/288: dwrite d7/d16/f39 [0,4194304] 0 2024-09-16T00:25:57.782 INFO:tasks.workunit.client.1.smithi169.stdout:7/289: dread - d7/f50 zero size 2024-09-16T00:25:57.783 INFO:tasks.workunit.client.1.smithi169.stdout:0/210: mknod d0/d2/d8/c4d 0 2024-09-16T00:25:57.784 INFO:tasks.workunit.client.1.smithi169.stdout:7/290: write d7/d16/d49/f4d [439859,91626] 0 2024-09-16T00:25:57.787 INFO:tasks.workunit.client.1.smithi169.stdout:9/248: symlink d5/da/l60 0 2024-09-16T00:25:57.789 INFO:tasks.workunit.client.1.smithi169.stdout:3/264: rename d7/d1a/d1c/d23/d2a to d7/d1a/d1c/d35/d63 0 2024-09-16T00:25:57.790 INFO:tasks.workunit.client.1.smithi169.stdout:0/211: symlink d0/l4e 0 2024-09-16T00:25:57.790 INFO:tasks.workunit.client.1.smithi169.stdout:3/265: fsync d7/d1a/d1c/f54 0 2024-09-16T00:25:57.790 INFO:tasks.workunit.client.0.smithi112.stdout:9/557: dread d1/d5/d6/d2d/d39/f63 [4194304,4194304] 0 2024-09-16T00:25:57.793 INFO:tasks.workunit.client.1.smithi169.stdout:3/266: symlink d7/d1a/d2b/l64 0 2024-09-16T00:25:57.796 INFO:tasks.workunit.client.1.smithi169.stdout:9/249: unlink d5/d12/c3f 0 2024-09-16T00:25:57.797 INFO:tasks.workunit.client.1.smithi169.stdout:0/212: dwrite d0/d2c/f39 [0,4194304] 0 2024-09-16T00:25:57.798 INFO:tasks.workunit.client.0.smithi112.stdout:8/292: sync 2024-09-16T00:25:57.798 INFO:tasks.workunit.client.0.smithi112.stdout:7/411: sync 2024-09-16T00:25:57.799 INFO:tasks.workunit.client.0.smithi112.stdout:8/293: chown d10/d12/d1c/f31 2147661 1 2024-09-16T00:25:57.799 INFO:tasks.workunit.client.1.smithi169.stdout:3/267: creat d7/d1a/d1c/d23/f65 x:0 0 0 2024-09-16T00:25:57.800 INFO:tasks.workunit.client.0.smithi112.stdout:7/412: unlink d6/c72 0 2024-09-16T00:25:57.801 INFO:tasks.workunit.client.0.smithi112.stdout:7/413: write d6/d10/d16/f20 [2830301,83155] 0 2024-09-16T00:25:57.801 INFO:tasks.workunit.client.0.smithi112.stdout:8/294: symlink d10/d12/d24/l60 0 2024-09-16T00:25:57.802 INFO:tasks.workunit.client.1.smithi169.stdout:3/268: fsync d7/d1a/d2b/f45 0 2024-09-16T00:25:57.802 INFO:tasks.workunit.client.0.smithi112.stdout:8/295: stat d10/l35 0 2024-09-16T00:25:57.804 INFO:tasks.workunit.client.0.smithi112.stdout:7/414: creat d6/d10/d75/d8f/f99 x:0 0 0 2024-09-16T00:25:57.804 INFO:tasks.workunit.client.1.smithi169.stdout:7/291: link d7/d16/l3f d7/df/l5e 0 2024-09-16T00:25:57.804 INFO:tasks.workunit.client.1.smithi169.stdout:0/213: dread d0/d2/f6 [0,4194304] 0 2024-09-16T00:25:57.805 INFO:tasks.workunit.client.1.smithi169.stdout:9/250: mkdir d5/d12/d61 0 2024-09-16T00:25:57.805 INFO:tasks.workunit.client.1.smithi169.stdout:7/292: stat d7/d16/d49 0 2024-09-16T00:25:57.806 INFO:tasks.workunit.client.1.smithi169.stdout:3/269: mknod d7/d1a/d1c/d23/c66 0 2024-09-16T00:25:57.806 INFO:tasks.workunit.client.1.smithi169.stdout:9/251: dread - d5/d12/f31 zero size 2024-09-16T00:25:57.807 INFO:tasks.workunit.client.0.smithi112.stdout:7/415: creat d6/d10/d1b/d39/f9a x:0 0 0 2024-09-16T00:25:57.809 INFO:tasks.workunit.client.0.smithi112.stdout:8/296: dwrite d10/d12/f1a [4194304,4194304] 0 2024-09-16T00:25:57.809 INFO:tasks.workunit.client.0.smithi112.stdout:7/416: mknod d6/d10/d16/d3b/c9b 0 2024-09-16T00:25:57.810 INFO:tasks.workunit.client.1.smithi169.stdout:9/252: fsync d5/da/d21/d23/f40 0 2024-09-16T00:25:57.812 INFO:tasks.workunit.client.1.smithi169.stdout:9/253: chown d5/da/f4d 359 1 2024-09-16T00:25:57.814 INFO:tasks.workunit.client.1.smithi169.stdout:9/254: dread - d5/d12/f5d zero size 2024-09-16T00:25:57.817 INFO:tasks.workunit.client.0.smithi112.stdout:8/297: dwrite d10/f46 [0,4194304] 0 2024-09-16T00:25:57.817 INFO:tasks.workunit.client.1.smithi169.stdout:7/293: dread f1 [0,4194304] 0 2024-09-16T00:25:57.819 INFO:tasks.workunit.client.0.smithi112.stdout:9/558: dread d1/d5/d6/d2d/f3e [0,4194304] 0 2024-09-16T00:25:57.819 INFO:tasks.workunit.client.1.smithi169.stdout:3/270: dwrite d7/d1a/d1c/d35/d63/f5f [0,4194304] 0 2024-09-16T00:25:57.820 INFO:tasks.workunit.client.0.smithi112.stdout:8/298: chown d10/c17 1438670742 1 2024-09-16T00:25:57.820 INFO:tasks.workunit.client.1.smithi169.stdout:0/214: dwrite d0/d2/d8/d5/d14/f1c [0,4194304] 0 2024-09-16T00:25:57.820 INFO:tasks.workunit.client.1.smithi169.stdout:7/294: fsync d7/d16/d49/f58 0 2024-09-16T00:25:57.821 INFO:tasks.workunit.client.0.smithi112.stdout:9/559: mkdir d1/d5/d20/d81/dc4 0 2024-09-16T00:25:57.821 INFO:tasks.workunit.client.0.smithi112.stdout:8/299: write d10/d12/d24/f2e [570224,109894] 0 2024-09-16T00:25:57.824 INFO:tasks.workunit.client.0.smithi112.stdout:8/300: mkdir d10/d12/d1c/d61 0 2024-09-16T00:25:57.826 INFO:tasks.workunit.client.0.smithi112.stdout:9/560: rename d1/d5/d2c/f57 to d1/d5/d6/d18/fc5 0 2024-09-16T00:25:57.826 INFO:tasks.workunit.client.1.smithi169.stdout:0/215: mkdir d0/d2c/d4f 0 2024-09-16T00:25:57.828 INFO:tasks.workunit.client.0.smithi112.stdout:8/301: mknod d10/d45/c62 0 2024-09-16T00:25:57.828 INFO:tasks.workunit.client.1.smithi169.stdout:3/271: rename d7/d1a/c38 to d7/d1a/d2b/c67 0 2024-09-16T00:25:57.830 INFO:tasks.workunit.client.0.smithi112.stdout:9/561: mknod d1/d5/d20/d25/d66/cc6 0 2024-09-16T00:25:57.831 INFO:tasks.workunit.client.1.smithi169.stdout:7/295: symlink d7/l5f 0 2024-09-16T00:25:57.831 INFO:tasks.workunit.client.1.smithi169.stdout:7/296: fdatasync d7/d16/f40 0 2024-09-16T00:25:57.832 INFO:tasks.workunit.client.1.smithi169.stdout:0/216: mkdir d0/d50 0 2024-09-16T00:25:57.833 INFO:tasks.workunit.client.1.smithi169.stdout:7/297: chown d7/da/c56 3325 1 2024-09-16T00:25:57.833 INFO:tasks.workunit.client.0.smithi112.stdout:8/302: creat d10/f63 x:0 0 0 2024-09-16T00:25:57.834 INFO:tasks.workunit.client.1.smithi169.stdout:0/217: read d0/f23 [334867,99123] 0 2024-09-16T00:25:57.834 INFO:tasks.workunit.client.1.smithi169.stdout:7/298: dread - d7/da/d29/f45 zero size 2024-09-16T00:25:57.834 INFO:tasks.workunit.client.1.smithi169.stdout:3/272: creat d7/f68 x:0 0 0 2024-09-16T00:25:57.835 INFO:tasks.workunit.client.0.smithi112.stdout:9/562: mknod d1/d5/d6/d2d/d6a/cc7 0 2024-09-16T00:25:57.837 INFO:tasks.workunit.client.0.smithi112.stdout:9/563: chown d1/d5/d20/d25/c22 5441 1 2024-09-16T00:25:57.837 INFO:tasks.workunit.client.1.smithi169.stdout:0/218: creat d0/f51 x:0 0 0 2024-09-16T00:25:57.838 INFO:tasks.workunit.client.1.smithi169.stdout:3/273: creat d7/d1a/d2b/f69 x:0 0 0 2024-09-16T00:25:57.839 INFO:tasks.workunit.client.0.smithi112.stdout:8/303: dwrite d10/d12/f16 [0,4194304] 0 2024-09-16T00:25:57.839 INFO:tasks.workunit.client.0.smithi112.stdout:7/417: rmdir d6/d10/d16/d3b 39 2024-09-16T00:25:57.840 INFO:tasks.workunit.client.0.smithi112.stdout:9/564: write d1/d5/d20/d81/d1a/d88/f7e [4810835,110229] 0 2024-09-16T00:25:57.841 INFO:tasks.workunit.client.0.smithi112.stdout:8/304: fdatasync d10/d12/d1c/d3b/f4d 0 2024-09-16T00:25:57.841 INFO:tasks.workunit.client.1.smithi169.stdout:9/255: rename d5/d12/f1c to d5/da/d2e/f62 0 2024-09-16T00:25:57.841 INFO:tasks.workunit.client.1.smithi169.stdout:7/299: creat d7/df/f60 x:0 0 0 2024-09-16T00:25:57.842 INFO:tasks.workunit.client.0.smithi112.stdout:8/305: fdatasync d10/d12/d24/f2b 0 2024-09-16T00:25:57.844 INFO:tasks.workunit.client.0.smithi112.stdout:8/306: readlink d10/d12/l5b 0 2024-09-16T00:25:57.844 INFO:tasks.workunit.client.1.smithi169.stdout:3/274: creat d7/d1a/d1c/d35/d63/f6a x:0 0 0 2024-09-16T00:25:57.844 INFO:tasks.workunit.client.1.smithi169.stdout:3/275: chown d7/f68 4141904 1 2024-09-16T00:25:57.845 INFO:tasks.workunit.client.1.smithi169.stdout:7/300: rmdir d7/da/d3a 39 2024-09-16T00:25:57.846 INFO:tasks.workunit.client.1.smithi169.stdout:3/276: truncate d7/d1a/d1c/d23/f65 366248 0 2024-09-16T00:25:57.846 INFO:tasks.workunit.client.0.smithi112.stdout:8/307: write d10/f15 [2084931,121535] 0 2024-09-16T00:25:57.847 INFO:tasks.workunit.client.1.smithi169.stdout:3/277: chown d7/d1a/d2b/l3b 1 1 2024-09-16T00:25:57.848 INFO:tasks.workunit.client.0.smithi112.stdout:9/565: dwrite d1/d5/d20/f69 [4194304,4194304] 0 2024-09-16T00:25:57.849 INFO:tasks.workunit.client.0.smithi112.stdout:8/308: creat d10/d4e/f64 x:0 0 0 2024-09-16T00:25:57.849 INFO:tasks.workunit.client.1.smithi169.stdout:0/219: rename d0/d2/d8/d5/d14/d21/f2a to d0/d2/d4b/f52 0 2024-09-16T00:25:57.851 INFO:tasks.workunit.client.1.smithi169.stdout:3/278: unlink d7/d1a/d1c/d23/l42 0 2024-09-16T00:25:57.852 INFO:tasks.workunit.client.0.smithi112.stdout:8/309: rename d10/d45/l4c to d10/d45/l65 0 2024-09-16T00:25:57.852 INFO:tasks.workunit.client.0.smithi112.stdout:9/566: creat d1/d5/d20/d81/d1a/d88/da8/fc8 x:0 0 0 2024-09-16T00:25:57.853 INFO:tasks.workunit.client.0.smithi112.stdout:8/310: write d10/d12/d1c/f54 [2330912,128907] 0 2024-09-16T00:25:57.854 INFO:tasks.workunit.client.1.smithi169.stdout:3/279: symlink d7/d1a/d1c/d35/d63/l6b 0 2024-09-16T00:25:57.856 INFO:tasks.workunit.client.0.smithi112.stdout:9/567: getdents d1/d5/d20/d81/dc4 0 2024-09-16T00:25:57.857 INFO:tasks.workunit.client.0.smithi112.stdout:8/311: dread d10/d12/d24/f28 [0,4194304] 0 2024-09-16T00:25:57.857 INFO:tasks.workunit.client.1.smithi169.stdout:3/280: truncate d7/d1a/d1c/d23/f3d 512974 0 2024-09-16T00:25:57.857 INFO:tasks.workunit.client.0.smithi112.stdout:9/568: stat d1/d5/d6/d18/f4b 0 2024-09-16T00:25:57.858 INFO:tasks.workunit.client.0.smithi112.stdout:8/312: write d10/d12/d24/f29 [209671,101797] 0 2024-09-16T00:25:57.859 INFO:tasks.workunit.client.1.smithi169.stdout:3/281: fdatasync d7/d1a/d1c/d35/d63/f5f 0 2024-09-16T00:25:57.859 INFO:tasks.workunit.client.0.smithi112.stdout:8/313: chown d10/d4e 153892438 1 2024-09-16T00:25:57.861 INFO:tasks.workunit.client.0.smithi112.stdout:8/314: creat d10/d4e/d58/f66 x:0 0 0 2024-09-16T00:25:57.862 INFO:tasks.workunit.client.1.smithi169.stdout:3/282: mkdir d7/d1a/d1c/d6c 0 2024-09-16T00:25:57.863 INFO:tasks.workunit.client.0.smithi112.stdout:8/315: chown d10/d12/d1c/d3b/f4d 6830 1 2024-09-16T00:25:57.865 INFO:tasks.workunit.client.1.smithi169.stdout:3/283: rename d7/f1e to d7/d1a/d1c/d6c/f6d 0 2024-09-16T00:25:57.866 INFO:tasks.workunit.client.0.smithi112.stdout:8/316: mknod d10/d12/c67 0 2024-09-16T00:25:57.866 INFO:tasks.workunit.client.0.smithi112.stdout:9/569: dwrite d1/d5/d6/d2d/d6a/fad [0,4194304] 0 2024-09-16T00:25:57.867 INFO:tasks.workunit.client.0.smithi112.stdout:9/570: readlink d1/d5/d20/d81/lb 0 2024-09-16T00:25:57.867 INFO:tasks.workunit.client.1.smithi169.stdout:3/284: write d7/fd [5904370,31060] 0 2024-09-16T00:25:57.868 INFO:tasks.workunit.client.0.smithi112.stdout:8/317: write d10/d12/d1c/f59 [6599574,113026] 0 2024-09-16T00:25:57.868 INFO:tasks.workunit.client.1.smithi169.stdout:0/220: getdents d0/d2/d8/d5/d14/d21 0 2024-09-16T00:25:57.868 INFO:tasks.workunit.client.1.smithi169.stdout:3/285: readlink d7/d1a/l2c 0 2024-09-16T00:25:57.870 INFO:tasks.workunit.client.0.smithi112.stdout:6/325: sync 2024-09-16T00:25:57.870 INFO:tasks.workunit.client.0.smithi112.stdout:1/372: sync 2024-09-16T00:25:57.870 INFO:tasks.workunit.client.0.smithi112.stdout:2/405: sync 2024-09-16T00:25:57.870 INFO:tasks.workunit.client.0.smithi112.stdout:0/419: sync 2024-09-16T00:25:57.870 INFO:tasks.workunit.client.1.smithi169.stdout:0/221: write d0/d2/d8/d5/d14/d21/f2f [2627320,22317] 0 2024-09-16T00:25:57.871 INFO:tasks.workunit.client.1.smithi169.stdout:7/301: truncate d7/d16/f39 4113150 0 2024-09-16T00:25:57.871 INFO:tasks.workunit.client.1.smithi169.stdout:0/222: stat d0/d2/f32 0 2024-09-16T00:25:57.872 INFO:tasks.workunit.client.1.smithi169.stdout:7/302: fdatasync d7/d16/f25 0 2024-09-16T00:25:57.872 INFO:tasks.workunit.client.0.smithi112.stdout:6/326: fsync dd/df/f6e 0 2024-09-16T00:25:57.872 INFO:tasks.workunit.client.0.smithi112.stdout:9/571: mknod d1/d5/d20/d25/cc9 0 2024-09-16T00:25:57.873 INFO:tasks.workunit.client.0.smithi112.stdout:8/318: creat d10/d4e/f68 x:0 0 0 2024-09-16T00:25:57.873 INFO:tasks.workunit.client.1.smithi169.stdout:3/286: symlink d7/d1a/d1c/d35/l6e 0 2024-09-16T00:25:57.874 INFO:tasks.workunit.client.0.smithi112.stdout:1/373: truncate d0/f48 665878 0 2024-09-16T00:25:57.874 INFO:tasks.workunit.client.1.smithi169.stdout:0/223: truncate d0/d2/d8/d5/d14/d21/f49 710819 0 2024-09-16T00:25:57.874 INFO:tasks.workunit.client.1.smithi169.stdout:7/303: fsync d7/df/f60 0 2024-09-16T00:25:57.875 INFO:tasks.workunit.client.0.smithi112.stdout:0/420: write d5/da/f53 [4692654,9670] 0 2024-09-16T00:25:57.876 INFO:tasks.workunit.client.0.smithi112.stdout:2/406: fdatasync d2/dd/d48/d22/d2b/d3b/d49/f5c 0 2024-09-16T00:25:57.876 INFO:tasks.workunit.client.0.smithi112.stdout:1/374: chown d0/d14/df/d28/l40 2 1 2024-09-16T00:25:57.876 INFO:tasks.workunit.client.1.smithi169.stdout:3/287: mkdir d7/d50/d6f 0 2024-09-16T00:25:57.876 INFO:tasks.workunit.client.1.smithi169.stdout:9/256: dread d5/da/f2b [0,4194304] 0 2024-09-16T00:25:57.877 INFO:tasks.workunit.client.1.smithi169.stdout:7/304: mkdir d7/d5b/d61 0 2024-09-16T00:25:57.878 INFO:tasks.workunit.client.1.smithi169.stdout:3/288: write d7/d1a/d1c/f4f [1322207,21139] 0 2024-09-16T00:25:57.878 INFO:tasks.workunit.client.1.smithi169.stdout:0/224: mknod d0/d2/d8/d5/d14/d16/d1a/c53 0 2024-09-16T00:25:57.878 INFO:tasks.workunit.client.1.smithi169.stdout:9/257: truncate d5/f14 4461611 0 2024-09-16T00:25:57.879 INFO:tasks.workunit.client.1.smithi169.stdout:7/305: readlink d7/df/l5e 0 2024-09-16T00:25:57.879 INFO:tasks.workunit.client.1.smithi169.stdout:3/289: chown d7/d1a/d1c/d23/f65 200991349 1 2024-09-16T00:25:57.880 INFO:tasks.workunit.client.0.smithi112.stdout:9/572: creat d1/d5/d83/fca x:0 0 0 2024-09-16T00:25:57.881 INFO:tasks.workunit.client.1.smithi169.stdout:9/258: creat d5/da/d5b/f63 x:0 0 0 2024-09-16T00:25:57.881 INFO:tasks.workunit.client.0.smithi112.stdout:0/421: write d5/da/d57/d76/f14 [90069,77629] 0 2024-09-16T00:25:57.883 INFO:tasks.workunit.client.1.smithi169.stdout:7/306: mknod d7/d16/c62 0 2024-09-16T00:25:57.884 INFO:tasks.workunit.client.0.smithi112.stdout:2/407: write d2/dd/d48/d22/d2b/f58 [1129515,97979] 0 2024-09-16T00:25:57.884 INFO:tasks.workunit.client.0.smithi112.stdout:9/573: write d1/d5/d6/d2d/f3e [1263091,114959] 0 2024-09-16T00:25:57.885 INFO:tasks.workunit.client.0.smithi112.stdout:1/375: creat d0/d14/df/d28/d45/d46/d58/d62/f74 x:0 0 0 2024-09-16T00:25:57.887 INFO:tasks.workunit.client.0.smithi112.stdout:6/327: rename dd/d15/d5a/l4f to dd/df/d17/d27/d54/l73 0 2024-09-16T00:25:57.888 INFO:tasks.workunit.client.0.smithi112.stdout:1/376: read d0/f48 [52961,125448] 0 2024-09-16T00:25:57.888 INFO:tasks.workunit.client.0.smithi112.stdout:2/408: unlink d2/dd/d48/d22/c4d 0 2024-09-16T00:25:57.890 INFO:tasks.workunit.client.1.smithi169.stdout:9/259: getdents d5 0 2024-09-16T00:25:57.890 INFO:tasks.workunit.client.1.smithi169.stdout:3/290: dwrite d7/d1a/d1c/d23/f3a [4194304,4194304] 0 2024-09-16T00:25:57.890 INFO:tasks.workunit.client.0.smithi112.stdout:6/328: creat dd/d15/d5a/d37/f74 x:0 0 0 2024-09-16T00:25:57.891 INFO:tasks.workunit.client.0.smithi112.stdout:1/377: creat d0/d14/df/d28/d45/d46/f75 x:0 0 0 2024-09-16T00:25:57.891 INFO:tasks.workunit.client.1.smithi169.stdout:0/225: rename d0/d2/d8/d5/d14 to d0/d2c/d54 0 2024-09-16T00:25:57.892 INFO:tasks.workunit.client.1.smithi169.stdout:0/226: dread - d0/f51 zero size 2024-09-16T00:25:57.892 INFO:tasks.workunit.client.1.smithi169.stdout:9/260: mkdir d5/d49/d64 0 2024-09-16T00:25:57.893 INFO:tasks.workunit.client.0.smithi112.stdout:9/574: link d1/c72 d1/d5/d20/ccb 0 2024-09-16T00:25:57.893 INFO:tasks.workunit.client.0.smithi112.stdout:2/409: dread d2/f14 [0,4194304] 0 2024-09-16T00:25:57.894 INFO:tasks.workunit.client.1.smithi169.stdout:9/261: dread - d5/da/d21/f46 zero size 2024-09-16T00:25:57.895 INFO:tasks.workunit.client.0.smithi112.stdout:9/575: dread - d1/d5/d20/d81/f98 zero size 2024-09-16T00:25:57.895 INFO:tasks.workunit.client.0.smithi112.stdout:1/378: creat d0/d14/d1c/d26/f76 x:0 0 0 2024-09-16T00:25:57.895 INFO:tasks.workunit.client.1.smithi169.stdout:0/227: write d0/f51 [611395,73054] 0 2024-09-16T00:25:57.896 INFO:tasks.workunit.client.1.smithi169.stdout:7/307: rename d7/d16/l3f to d7/d5b/d61/l63 0 2024-09-16T00:25:57.898 INFO:tasks.workunit.client.1.smithi169.stdout:7/308: unlink d7/le 0 2024-09-16T00:25:57.899 INFO:tasks.workunit.client.1.smithi169.stdout:7/309: read - d7/d5b/f5c zero size 2024-09-16T00:25:57.899 INFO:tasks.workunit.client.1.smithi169.stdout:7/310: fdatasync d7/d16/d49/f58 0 2024-09-16T00:25:57.900 INFO:tasks.workunit.client.1.smithi169.stdout:0/228: unlink d0/d2/d8/d5/f24 0 2024-09-16T00:25:57.901 INFO:tasks.workunit.client.1.smithi169.stdout:9/262: creat d5/d49/d64/f65 x:0 0 0 2024-09-16T00:25:57.902 INFO:tasks.workunit.client.0.smithi112.stdout:2/410: write d2/dd/d48/d22/d2b/f31 [1749571,12160] 0 2024-09-16T00:25:57.903 INFO:tasks.workunit.client.1.smithi169.stdout:0/229: mknod d0/d2/d15/c55 0 2024-09-16T00:25:57.904 INFO:tasks.workunit.client.0.smithi112.stdout:6/329: dwrite dd/df/d10/d47/f5b [0,4194304] 0 2024-09-16T00:25:57.905 INFO:tasks.workunit.client.1.smithi169.stdout:7/311: symlink d7/l64 0 2024-09-16T00:25:57.906 INFO:tasks.workunit.client.1.smithi169.stdout:7/312: write d7/df/f5d [3100110,109632] 0 2024-09-16T00:25:57.907 INFO:tasks.workunit.client.0.smithi112.stdout:6/330: fdatasync fa 0 2024-09-16T00:25:57.907 INFO:tasks.workunit.client.1.smithi169.stdout:9/263: mknod d5/da/d2e/c66 0 2024-09-16T00:25:57.907 INFO:tasks.workunit.client.1.smithi169.stdout:7/313: fdatasync d7/df/f44 0 2024-09-16T00:25:57.908 INFO:tasks.workunit.client.0.smithi112.stdout:9/576: getdents d1/d76 0 2024-09-16T00:25:57.910 INFO:tasks.workunit.client.0.smithi112.stdout:2/411: creat d2/dd/d48/d22/d2b/d54/d79/f7a x:0 0 0 2024-09-16T00:25:57.912 INFO:tasks.workunit.client.0.smithi112.stdout:9/577: unlink d1/d5/d6/d18/f9a 0 2024-09-16T00:25:57.913 INFO:tasks.workunit.client.0.smithi112.stdout:1/379: dwrite d0/d14/df/d28/d45/d30/f4a [0,4194304] 0 2024-09-16T00:25:57.915 INFO:tasks.workunit.client.0.smithi112.stdout:1/380: readlink d0/d14/df/d28/l40 0 2024-09-16T00:25:57.915 INFO:tasks.workunit.client.0.smithi112.stdout:9/578: chown d1/d5/d6/d2d/d6a/fad 1992 1 2024-09-16T00:25:57.916 INFO:tasks.workunit.client.0.smithi112.stdout:1/381: dread - d0/d14/d1c/d20/f53 zero size 2024-09-16T00:25:57.916 INFO:tasks.workunit.client.1.smithi169.stdout:9/264: dwrite d5/da/d21/f33 [4194304,4194304] 0 2024-09-16T00:25:57.916 INFO:tasks.workunit.client.0.smithi112.stdout:6/331: mknod dd/d15/c75 0 2024-09-16T00:25:57.918 INFO:tasks.workunit.client.1.smithi169.stdout:7/314: getdents d7/d16 0 2024-09-16T00:25:57.918 INFO:tasks.workunit.client.1.smithi169.stdout:7/315: readlink d7/l64 0 2024-09-16T00:25:57.918 INFO:tasks.workunit.client.0.smithi112.stdout:6/332: read dd/d15/d5a/f26 [336181,27753] 0 2024-09-16T00:25:57.919 INFO:tasks.workunit.client.0.smithi112.stdout:6/333: chown dd/df/d4b 43374 1 2024-09-16T00:25:57.922 INFO:tasks.workunit.client.0.smithi112.stdout:2/412: rename d2/dd/f5b to d2/dd/d48/d22/d2b/d54/d60/d6c/f7b 0 2024-09-16T00:25:57.924 INFO:tasks.workunit.client.0.smithi112.stdout:9/579: mknod d1/d5/d6/d2d/ccc 0 2024-09-16T00:25:57.924 INFO:tasks.workunit.client.1.smithi169.stdout:9/265: dread d5/da/ff [0,4194304] 0 2024-09-16T00:25:57.928 INFO:tasks.workunit.client.0.smithi112.stdout:9/580: creat d1/d5/d6/d2d/d6a/fcd x:0 0 0 2024-09-16T00:25:57.929 INFO:tasks.workunit.client.0.smithi112.stdout:1/382: dread d0/f67 [0,4194304] 0 2024-09-16T00:25:57.931 INFO:tasks.workunit.client.1.smithi169.stdout:4/217: sync 2024-09-16T00:25:57.931 INFO:tasks.workunit.client.1.smithi169.stdout:9/266: dwrite d5/da/d21/d23/f40 [0,4194304] 0 2024-09-16T00:25:57.932 INFO:tasks.workunit.client.1.smithi169.stdout:6/98: sync 2024-09-16T00:25:57.932 INFO:tasks.workunit.client.0.smithi112.stdout:6/334: dwrite dd/df/f21 [0,4194304] 0 2024-09-16T00:25:57.934 INFO:tasks.workunit.client.1.smithi169.stdout:9/267: creat d5/da/d21/d23/f67 x:0 0 0 2024-09-16T00:25:57.936 INFO:tasks.workunit.client.1.smithi169.stdout:6/99: truncate d2/d15/f1b 4914758 0 2024-09-16T00:25:57.936 INFO:tasks.workunit.client.0.smithi112.stdout:6/335: creat dd/df/d17/d27/f76 x:0 0 0 2024-09-16T00:25:57.936 INFO:tasks.workunit.client.0.smithi112.stdout:1/383: link d0/d14/d1c/c1f d0/d14/df/d28/c77 0 2024-09-16T00:25:57.937 INFO:tasks.workunit.client.1.smithi169.stdout:9/268: mknod d5/c68 0 2024-09-16T00:25:57.940 INFO:tasks.workunit.client.0.smithi112.stdout:1/384: symlink d0/d14/df/d28/d45/d46/l78 0 2024-09-16T00:25:57.940 INFO:tasks.workunit.client.1.smithi169.stdout:6/100: creat d2/f1f x:0 0 0 2024-09-16T00:25:57.941 INFO:tasks.workunit.client.0.smithi112.stdout:6/336: creat dd/df/d17/f77 x:0 0 0 2024-09-16T00:25:57.941 INFO:tasks.workunit.client.1.smithi169.stdout:4/218: dwrite d2/d12/d13/d1f/f35 [0,4194304] 0 2024-09-16T00:25:57.942 INFO:tasks.workunit.client.0.smithi112.stdout:1/385: write d0/d14/df/d28/d45/d46/f6d [751266,110300] 0 2024-09-16T00:25:57.943 INFO:tasks.workunit.client.0.smithi112.stdout:2/413: truncate d2/dd/d48/d22/d2b/d54/d60/d6c/f7b 111784 0 2024-09-16T00:25:57.944 INFO:tasks.workunit.client.1.smithi169.stdout:7/316: write d7/da/f23 [1805441,20305] 0 2024-09-16T00:25:57.947 INFO:tasks.workunit.client.0.smithi112.stdout:6/337: read dd/df/d10/d47/d55/f72 [3423386,65189] 0 2024-09-16T00:25:57.947 INFO:tasks.workunit.client.0.smithi112.stdout:2/414: unlink d2/dd/d48/d22/d2b/d3b/f66 0 2024-09-16T00:25:57.949 INFO:tasks.workunit.client.1.smithi169.stdout:7/317: mknod d7/da/c65 0 2024-09-16T00:25:57.949 INFO:tasks.workunit.client.0.smithi112.stdout:2/415: stat d2/dd/d48/d22/d2b/f34 0 2024-09-16T00:25:57.950 INFO:tasks.workunit.client.0.smithi112.stdout:6/338: rmdir dd/d15/d5a 39 2024-09-16T00:25:57.953 INFO:tasks.workunit.client.1.smithi169.stdout:6/101: dwrite d2/d15/f18 [0,4194304] 0 2024-09-16T00:25:57.953 INFO:tasks.workunit.client.0.smithi112.stdout:6/339: fdatasync dd/df/d10/f1b 0 2024-09-16T00:25:57.954 INFO:tasks.workunit.client.0.smithi112.stdout:1/386: dwrite d0/d14/d1c/f5c [0,4194304] 0 2024-09-16T00:25:57.954 INFO:tasks.workunit.client.1.smithi169.stdout:6/102: read d2/f17 [2597451,64609] 0 2024-09-16T00:25:57.956 INFO:tasks.workunit.client.0.smithi112.stdout:2/416: unlink d2/dd/d48/d22/d2b/d54/f5a 0 2024-09-16T00:25:57.957 INFO:tasks.workunit.client.1.smithi169.stdout:6/103: symlink d2/l20 0 2024-09-16T00:25:57.960 INFO:tasks.workunit.client.1.smithi169.stdout:6/104: rename d2/d15/c1a to d2/d15/d1d/c21 0 2024-09-16T00:25:57.963 INFO:tasks.workunit.client.0.smithi112.stdout:9/581: dread d1/d5/d6/d18/f40 [0,4194304] 0 2024-09-16T00:25:57.963 INFO:tasks.workunit.client.1.smithi169.stdout:6/105: mkdir d2/d22 0 2024-09-16T00:25:57.964 INFO:tasks.workunit.client.1.smithi169.stdout:9/269: dwrite d5/da/f28 [0,4194304] 0 2024-09-16T00:25:57.965 INFO:tasks.workunit.client.0.smithi112.stdout:6/340: dread dd/df/d10/d47/f57 [0,4194304] 0 2024-09-16T00:25:57.968 INFO:tasks.workunit.client.1.smithi169.stdout:9/270: chown d5/d49/d64/f65 8487018 1 2024-09-16T00:25:57.969 INFO:tasks.workunit.client.0.smithi112.stdout:6/341: read dd/f2a [2348596,50322] 0 2024-09-16T00:25:57.969 INFO:tasks.workunit.client.0.smithi112.stdout:9/582: unlink d1/d5/d20/d81/f67 0 2024-09-16T00:25:57.969 INFO:tasks.workunit.client.0.smithi112.stdout:2/417: dwrite d2/dd/d48/d22/f29 [0,4194304] 0 2024-09-16T00:25:57.970 INFO:tasks.workunit.client.0.smithi112.stdout:1/387: dwrite d0/d14/df/d28/d45/d30/f4a [4194304,4194304] 0 2024-09-16T00:25:57.971 INFO:tasks.workunit.client.1.smithi169.stdout:6/106: link d2/d15/f1e d2/d22/f23 0 2024-09-16T00:25:57.971 INFO:tasks.workunit.client.0.smithi112.stdout:6/342: dread - dd/df/f6e zero size 2024-09-16T00:25:57.972 INFO:tasks.workunit.client.1.smithi169.stdout:4/219: write d2/d12/d13/f2e [4317539,20040] 0 2024-09-16T00:25:57.972 INFO:tasks.workunit.client.1.smithi169.stdout:6/107: chown d2/f17 1105108524 1 2024-09-16T00:25:57.976 INFO:tasks.workunit.client.1.smithi169.stdout:4/220: write d2/d12/f31 [9084,81546] 0 2024-09-16T00:25:57.977 INFO:tasks.workunit.client.1.smithi169.stdout:9/271: dwrite d5/d49/f59 [0,4194304] 0 2024-09-16T00:25:57.980 INFO:tasks.workunit.client.0.smithi112.stdout:6/343: creat dd/df/d10/d47/d55/f78 x:0 0 0 2024-09-16T00:25:57.980 INFO:tasks.workunit.client.1.smithi169.stdout:6/108: truncate d2/d15/f1b 2059929 0 2024-09-16T00:25:57.982 INFO:tasks.workunit.client.0.smithi112.stdout:6/344: chown dd/df/d11/f67 241304410 1 2024-09-16T00:25:57.982 INFO:tasks.workunit.client.0.smithi112.stdout:1/388: symlink d0/d14/d1c/d26/d47/l79 0 2024-09-16T00:25:57.984 INFO:tasks.workunit.client.1.smithi169.stdout:6/109: write d2/d15/f1e [628333,109680] 0 2024-09-16T00:25:57.984 INFO:tasks.workunit.client.0.smithi112.stdout:2/418: mknod d2/c7c 0 2024-09-16T00:25:57.985 INFO:tasks.workunit.client.1.smithi169.stdout:9/272: rename d5/da/d21/d23/l3c to d5/da/l69 0 2024-09-16T00:25:57.986 INFO:tasks.workunit.client.1.smithi169.stdout:6/110: rename d2/c8 to d2/c24 0 2024-09-16T00:25:57.987 INFO:tasks.workunit.client.0.smithi112.stdout:1/389: rmdir d0/d14/df 39 2024-09-16T00:25:57.987 INFO:tasks.workunit.client.1.smithi169.stdout:9/273: truncate d5/da/fc 1596989 0 2024-09-16T00:25:57.989 INFO:tasks.workunit.client.0.smithi112.stdout:6/345: mkdir dd/d15/d5a/d37/d79 0 2024-09-16T00:25:57.990 INFO:tasks.workunit.client.0.smithi112.stdout:2/419: chown d2/dd/d48/d22/d2b/d30/f36 3134 1 2024-09-16T00:25:57.992 INFO:tasks.workunit.client.0.smithi112.stdout:6/346: write dd/d15/d5a/d37/d62/f4a [452200,89936] 0 2024-09-16T00:25:57.994 INFO:tasks.workunit.client.0.smithi112.stdout:6/347: chown dd/l33 0 1 2024-09-16T00:25:57.997 INFO:tasks.workunit.client.0.smithi112.stdout:2/420: creat d2/dd/d48/d22/d2b/d3b/d49/f7d x:0 0 0 2024-09-16T00:25:57.998 INFO:tasks.workunit.client.0.smithi112.stdout:1/390: dwrite d0/f4e [0,4194304] 0 2024-09-16T00:25:57.998 INFO:tasks.workunit.client.1.smithi169.stdout:9/274: rename d5/f15 to d5/da/d4e/f6a 0 2024-09-16T00:25:58.000 INFO:tasks.workunit.client.1.smithi169.stdout:7/318: dread d7/f26 [0,4194304] 0 2024-09-16T00:25:58.001 INFO:tasks.workunit.client.0.smithi112.stdout:2/421: truncate d2/dd/d48/d22/d2b/d3b/d49/f5c 265533 0 2024-09-16T00:25:58.002 INFO:tasks.workunit.client.0.smithi112.stdout:6/348: link dd/d23/f63 dd/df/d10/d47/d55/f7a 0 2024-09-16T00:25:58.004 INFO:tasks.workunit.client.0.smithi112.stdout:2/422: unlink d2/dd/f75 0 2024-09-16T00:25:58.006 INFO:tasks.workunit.client.0.smithi112.stdout:2/423: symlink d2/dd/d48/d22/d2b/l7e 0 2024-09-16T00:25:58.007 INFO:tasks.workunit.client.0.smithi112.stdout:2/424: creat d2/d68/f7f x:0 0 0 2024-09-16T00:25:58.009 INFO:tasks.workunit.client.0.smithi112.stdout:2/425: symlink d2/dd/d48/d22/d2b/d3b/d49/l80 0 2024-09-16T00:25:58.012 INFO:tasks.workunit.client.0.smithi112.stdout:2/426: dread d2/dd/d48/d22/d2b/f73 [0,4194304] 0 2024-09-16T00:25:58.012 INFO:tasks.workunit.client.0.smithi112.stdout:6/349: dread dd/df/f4c [0,4194304] 0 2024-09-16T00:25:58.013 INFO:tasks.workunit.client.0.smithi112.stdout:6/350: stat dd/df/f6e 0 2024-09-16T00:25:58.017 INFO:tasks.workunit.client.0.smithi112.stdout:5/354: sync 2024-09-16T00:25:58.018 INFO:tasks.workunit.client.0.smithi112.stdout:6/351: mknod dd/df/d11/d44/c7b 0 2024-09-16T00:25:58.018 INFO:tasks.workunit.client.0.smithi112.stdout:5/355: fdatasync d4/dd/f66 0 2024-09-16T00:25:58.019 INFO:tasks.workunit.client.0.smithi112.stdout:6/352: write fa [4905028,86046] 0 2024-09-16T00:25:58.020 INFO:tasks.workunit.client.0.smithi112.stdout:5/356: stat d4/f19 0 2024-09-16T00:25:58.020 INFO:tasks.workunit.client.0.smithi112.stdout:6/353: readlink dd/d23/l30 0 2024-09-16T00:25:58.020 INFO:tasks.workunit.client.0.smithi112.stdout:5/357: chown d4/d1c/d43 22483997 1 2024-09-16T00:25:58.020 INFO:tasks.workunit.client.0.smithi112.stdout:9/583: dread d1/d5/d6/f4a [0,4194304] 0 2024-09-16T00:25:58.021 INFO:tasks.workunit.client.0.smithi112.stdout:9/584: read d1/f82 [4070837,8576] 0 2024-09-16T00:25:58.022 INFO:tasks.workunit.client.0.smithi112.stdout:5/358: creat d4/d4f/d77/f7f x:0 0 0 2024-09-16T00:25:58.024 INFO:tasks.workunit.client.0.smithi112.stdout:5/359: unlink d4/d1c/f38 0 2024-09-16T00:25:58.026 INFO:tasks.workunit.client.0.smithi112.stdout:9/585: unlink d1/d5/d6/d2d/d39/c94 0 2024-09-16T00:25:58.029 INFO:tasks.workunit.client.0.smithi112.stdout:6/354: rename dd/d15/d5a/d37/d62 to dd/df/d13/d7c 0 2024-09-16T00:25:58.030 INFO:tasks.workunit.client.0.smithi112.stdout:6/355: readlink dd/d15/d5a/l3c 0 2024-09-16T00:25:58.031 INFO:tasks.workunit.client.1.smithi169.stdout:7/319: write f1 [1137686,40559] 0 2024-09-16T00:25:58.031 INFO:tasks.workunit.client.0.smithi112.stdout:5/360: dwrite d4/f57 [0,4194304] 0 2024-09-16T00:25:58.032 INFO:tasks.workunit.client.1.smithi169.stdout:4/221: dwrite d2/f4 [0,4194304] 0 2024-09-16T00:25:58.035 INFO:tasks.workunit.client.0.smithi112.stdout:2/427: rmdir d2/dd/d48/d22 39 2024-09-16T00:25:58.036 INFO:tasks.workunit.client.1.smithi169.stdout:7/320: write d7/f17 [3718447,101223] 0 2024-09-16T00:25:58.036 INFO:tasks.workunit.client.0.smithi112.stdout:5/361: read d4/dd/d12/f11 [1907510,61591] 0 2024-09-16T00:25:58.038 INFO:tasks.workunit.client.1.smithi169.stdout:4/222: dread d2/d12/f36 [0,4194304] 0 2024-09-16T00:25:58.039 INFO:tasks.workunit.client.1.smithi169.stdout:7/321: rmdir d7/df 39 2024-09-16T00:25:58.041 INFO:tasks.workunit.client.0.smithi112.stdout:9/586: dwrite d1/d5/d20/d25/d66/f9d [0,4194304] 0 2024-09-16T00:25:58.042 INFO:tasks.workunit.client.1.smithi169.stdout:6/111: dwrite d2/d15/f1b [0,4194304] 0 2024-09-16T00:25:58.043 INFO:tasks.workunit.client.1.smithi169.stdout:6/112: stat d2/l16 0 2024-09-16T00:25:58.044 INFO:tasks.workunit.client.1.smithi169.stdout:7/322: symlink d7/d16/l66 0 2024-09-16T00:25:58.044 INFO:tasks.workunit.client.1.smithi169.stdout:6/113: dread - d2/f19 zero size 2024-09-16T00:25:58.045 INFO:tasks.workunit.client.1.smithi169.stdout:9/275: dread d5/da/d21/d23/f3b [0,4194304] 0 2024-09-16T00:25:58.045 INFO:tasks.workunit.client.0.smithi112.stdout:2/428: symlink d2/dd/d48/d22/d2b/d3b/d49/l81 0 2024-09-16T00:25:58.046 INFO:tasks.workunit.client.0.smithi112.stdout:6/356: dread dd/df/d11/f49 [0,4194304] 0 2024-09-16T00:25:58.047 INFO:tasks.workunit.client.0.smithi112.stdout:2/429: fsync d2/dd/d48/d22/f29 0 2024-09-16T00:25:58.049 INFO:tasks.workunit.client.0.smithi112.stdout:6/357: dread dd/df/f4c [0,4194304] 0 2024-09-16T00:25:58.051 INFO:tasks.workunit.client.0.smithi112.stdout:5/362: dwrite d4/d7/d26/f46 [0,4194304] 0 2024-09-16T00:25:58.053 INFO:tasks.workunit.client.0.smithi112.stdout:6/358: dread dd/df/f4c [0,4194304] 0 2024-09-16T00:25:58.054 INFO:tasks.workunit.client.0.smithi112.stdout:2/430: symlink d2/dd/d48/d22/d67/l82 0 2024-09-16T00:25:58.054 INFO:tasks.workunit.client.1.smithi169.stdout:4/223: link d2/d12/d13/d1f/d2f/l3c d2/d12/d13/d27/d33/l42 0 2024-09-16T00:25:58.054 INFO:tasks.workunit.client.0.smithi112.stdout:5/363: mknod d4/d4f/d77/c80 0 2024-09-16T00:25:58.055 INFO:tasks.workunit.client.0.smithi112.stdout:6/359: creat dd/df/d10/d47/d55/f7d x:0 0 0 2024-09-16T00:25:58.057 INFO:tasks.workunit.client.0.smithi112.stdout:2/431: readlink d2/dd/l41 0 2024-09-16T00:25:58.057 INFO:tasks.workunit.client.0.smithi112.stdout:6/360: readlink dd/df/d10/l16 0 2024-09-16T00:25:58.058 INFO:tasks.workunit.client.0.smithi112.stdout:6/361: chown dd/df/f66 7758 1 2024-09-16T00:25:58.058 INFO:tasks.workunit.client.1.smithi169.stdout:9/276: dwrite d5/f14 [0,4194304] 0 2024-09-16T00:25:58.060 INFO:tasks.workunit.client.0.smithi112.stdout:6/362: creat dd/f7e x:0 0 0 2024-09-16T00:25:58.060 INFO:tasks.workunit.client.1.smithi169.stdout:4/224: creat d2/d12/d13/d1f/d2f/f43 x:0 0 0 2024-09-16T00:25:58.061 INFO:tasks.workunit.client.0.smithi112.stdout:2/432: unlink d2/dd/d48/f23 0 2024-09-16T00:25:58.063 INFO:tasks.workunit.client.1.smithi169.stdout:4/225: symlink d2/d12/l44 0 2024-09-16T00:25:58.064 INFO:tasks.workunit.client.0.smithi112.stdout:2/433: mknod d2/dd/d48/d22/d67/c83 0 2024-09-16T00:25:58.066 INFO:tasks.workunit.client.1.smithi169.stdout:9/277: creat d5/da/d2e/d47/f6b x:0 0 0 2024-09-16T00:25:58.067 INFO:tasks.workunit.client.1.smithi169.stdout:4/226: unlink d2/f4 0 2024-09-16T00:25:58.068 INFO:tasks.workunit.client.0.smithi112.stdout:2/434: symlink d2/dd/d48/d22/d2b/d3b/l84 0 2024-09-16T00:25:58.070 INFO:tasks.workunit.client.1.smithi169.stdout:9/278: mkdir d5/da/d5b/d6c 0 2024-09-16T00:25:58.073 INFO:tasks.workunit.client.1.smithi169.stdout:4/227: write d2/dd/d1a/f41 [521414,55446] 0 2024-09-16T00:25:58.075 INFO:tasks.workunit.client.1.smithi169.stdout:6/114: dwrite d2/f17 [0,4194304] 0 2024-09-16T00:25:58.077 INFO:tasks.workunit.client.1.smithi169.stdout:7/323: dread d7/f31 [0,4194304] 0 2024-09-16T00:25:58.078 INFO:tasks.workunit.client.1.smithi169.stdout:4/228: creat d2/d12/d13/d1f/d2f/f45 x:0 0 0 2024-09-16T00:25:58.083 INFO:tasks.workunit.client.1.smithi169.stdout:9/279: dwrite d5/da/d4e/f53 [0,4194304] 0 2024-09-16T00:25:58.084 INFO:tasks.workunit.client.1.smithi169.stdout:7/324: truncate d7/da/fc 3728842 0 2024-09-16T00:25:58.085 INFO:tasks.workunit.client.0.smithi112.stdout:5/364: dread d4/d7/d26/d36/f3b [0,4194304] 0 2024-09-16T00:25:58.085 INFO:tasks.workunit.client.1.smithi169.stdout:9/280: stat d5/da/d2e/c66 0 2024-09-16T00:25:58.085 INFO:tasks.workunit.client.1.smithi169.stdout:4/229: chown d2/d12/d13/d1f/f35 807 1 2024-09-16T00:25:58.091 INFO:tasks.workunit.client.1.smithi169.stdout:7/325: link d7/d16/f42 d7/f67 0 2024-09-16T00:25:58.091 INFO:tasks.workunit.client.0.smithi112.stdout:5/365: dwrite d4/dd/d12/d52/f63 [0,4194304] 0 2024-09-16T00:25:58.092 INFO:tasks.workunit.client.1.smithi169.stdout:7/326: fdatasync d7/da/f23 0 2024-09-16T00:25:58.093 INFO:tasks.workunit.client.0.smithi112.stdout:5/366: creat d4/d7/d26/f81 x:0 0 0 2024-09-16T00:25:58.095 INFO:tasks.workunit.client.1.smithi169.stdout:4/230: dwrite d2/d12/f36 [0,4194304] 0 2024-09-16T00:25:58.097 INFO:tasks.workunit.client.0.smithi112.stdout:5/367: unlink d4/d7/l60 0 2024-09-16T00:25:58.098 INFO:tasks.workunit.client.1.smithi169.stdout:4/231: symlink d2/d12/d13/d27/l46 0 2024-09-16T00:25:58.099 INFO:tasks.workunit.client.0.smithi112.stdout:5/368: mknod d4/d7/d26/d36/c82 0 2024-09-16T00:25:58.100 INFO:tasks.workunit.client.0.smithi112.stdout:5/369: write d4/dd/d12/f2f [1300117,62734] 0 2024-09-16T00:25:58.101 INFO:tasks.workunit.client.0.smithi112.stdout:5/370: write d4/d7/d26/d4c/f6a [54354,68687] 0 2024-09-16T00:25:58.106 INFO:tasks.workunit.client.0.smithi112.stdout:5/371: dread d4/dd/d12/d52/f63 [0,4194304] 0 2024-09-16T00:25:58.106 INFO:tasks.workunit.client.1.smithi169.stdout:6/115: truncate d2/f17 3865412 0 2024-09-16T00:25:58.107 INFO:tasks.workunit.client.0.smithi112.stdout:5/372: truncate d4/dd/d6f/f72 5048551 0 2024-09-16T00:25:58.107 INFO:tasks.workunit.client.0.smithi112.stdout:5/373: chown d4/d7/d26/d3e 9516887 1 2024-09-16T00:25:58.108 INFO:tasks.workunit.client.0.smithi112.stdout:5/374: dread - d4/d1c/f73 zero size 2024-09-16T00:25:58.112 INFO:tasks.workunit.client.0.smithi112.stdout:5/375: read d4/d1c/d43/f6d [42844,67688] 0 2024-09-16T00:25:58.112 INFO:tasks.workunit.client.0.smithi112.stdout:6/363: write dd/df/f4c [413822,24332] 0 2024-09-16T00:25:58.112 INFO:tasks.workunit.client.0.smithi112.stdout:5/376: readlink d4/d7/d26/l5e 0 2024-09-16T00:25:58.113 INFO:tasks.workunit.client.0.smithi112.stdout:2/435: truncate d2/f46 2319964 0 2024-09-16T00:25:58.114 INFO:tasks.workunit.client.0.smithi112.stdout:5/377: mkdir d4/dd/d49/d70/d83 0 2024-09-16T00:25:58.115 INFO:tasks.workunit.client.0.smithi112.stdout:6/364: rename dd/df/d17/l2b to dd/df/d13/d18/l7f 0 2024-09-16T00:25:58.117 INFO:tasks.workunit.client.0.smithi112.stdout:6/365: creat dd/df/d11/f80 x:0 0 0 2024-09-16T00:25:58.118 INFO:tasks.workunit.client.0.smithi112.stdout:2/436: creat d2/dd/d48/f85 x:0 0 0 2024-09-16T00:25:58.121 INFO:tasks.workunit.client.1.smithi169.stdout:4/232: truncate d2/d12/d13/f25 4659385 0 2024-09-16T00:25:58.122 INFO:tasks.workunit.client.0.smithi112.stdout:2/437: symlink d2/dd/l86 0 2024-09-16T00:25:58.123 INFO:tasks.workunit.client.0.smithi112.stdout:2/438: fdatasync d2/dd/d48/d22/d2b/f43 0 2024-09-16T00:25:58.123 INFO:tasks.workunit.client.1.smithi169.stdout:2/183: sync 2024-09-16T00:25:58.123 INFO:tasks.workunit.client.1.smithi169.stdout:8/192: sync 2024-09-16T00:25:58.123 INFO:tasks.workunit.client.1.smithi169.stdout:1/220: sync 2024-09-16T00:25:58.123 INFO:tasks.workunit.client.1.smithi169.stdout:5/196: sync 2024-09-16T00:25:58.123 INFO:tasks.workunit.client.0.smithi112.stdout:6/366: link dd/d15/d5a/l3c dd/df/d17/l81 0 2024-09-16T00:25:58.124 INFO:tasks.workunit.client.0.smithi112.stdout:6/367: fdatasync dd/df/f21 0 2024-09-16T00:25:58.124 INFO:tasks.workunit.client.1.smithi169.stdout:9/281: dwrite d5/d12/f26 [0,4194304] 0 2024-09-16T00:25:58.125 INFO:tasks.workunit.client.0.smithi112.stdout:2/439: creat d2/dd/f87 x:0 0 0 2024-09-16T00:25:58.126 INFO:tasks.workunit.client.0.smithi112.stdout:6/368: write dd/df/d17/d27/d54/f56 [4328666,54536] 0 2024-09-16T00:25:58.127 INFO:tasks.workunit.client.1.smithi169.stdout:5/197: stat d2/da/d37/c39 0 2024-09-16T00:25:58.129 INFO:tasks.workunit.client.1.smithi169.stdout:4/233: rmdir d2/d12/d13/d1f/d2f 39 2024-09-16T00:25:58.129 INFO:tasks.workunit.client.1.smithi169.stdout:5/198: chown d2/f29 85799056 1 2024-09-16T00:25:58.133 INFO:tasks.workunit.client.1.smithi169.stdout:4/234: stat d2/d12/l19 0 2024-09-16T00:25:58.133 INFO:tasks.workunit.client.1.smithi169.stdout:8/193: rename d7/dc/f14 to d7/dc/d2a/f42 0 2024-09-16T00:25:58.134 INFO:tasks.workunit.client.0.smithi112.stdout:2/440: dwrite d2/dd/d48/f61 [0,4194304] 0 2024-09-16T00:25:58.135 INFO:tasks.workunit.client.0.smithi112.stdout:6/369: dwrite dd/df/d10/d47/d55/f78 [0,4194304] 0 2024-09-16T00:25:58.136 INFO:tasks.workunit.client.1.smithi169.stdout:7/327: dwrite d7/da/fc [0,4194304] 0 2024-09-16T00:25:58.136 INFO:tasks.workunit.client.1.smithi169.stdout:5/199: symlink d2/da/l44 0 2024-09-16T00:25:58.137 INFO:tasks.workunit.client.0.smithi112.stdout:6/370: fdatasync dd/df/d17/f41 0 2024-09-16T00:25:58.137 INFO:tasks.workunit.client.0.smithi112.stdout:6/371: write dd/df/f4c [709496,13647] 0 2024-09-16T00:25:58.138 INFO:tasks.workunit.client.1.smithi169.stdout:7/328: fdatasync d7/d5b/f5c 0 2024-09-16T00:25:58.138 INFO:tasks.workunit.client.1.smithi169.stdout:6/116: dread d2/fb [0,4194304] 0 2024-09-16T00:25:58.138 INFO:tasks.workunit.client.1.smithi169.stdout:5/200: dread - d2/da/d10/f3f zero size 2024-09-16T00:25:58.139 INFO:tasks.workunit.client.1.smithi169.stdout:6/117: dread - d2/f1f zero size 2024-09-16T00:25:58.139 INFO:tasks.workunit.client.1.smithi169.stdout:8/194: symlink d7/d20/l43 0 2024-09-16T00:25:58.141 INFO:tasks.workunit.client.1.smithi169.stdout:4/235: creat d2/dd/d1a/f47 x:0 0 0 2024-09-16T00:25:58.142 INFO:tasks.workunit.client.1.smithi169.stdout:7/329: write d7/d16/d49/f58 [813578,128078] 0 2024-09-16T00:25:58.143 INFO:tasks.workunit.client.1.smithi169.stdout:1/221: link dc/d18/l1b dc/df/d29/d2a/l3e 0 2024-09-16T00:25:58.143 INFO:tasks.workunit.client.0.smithi112.stdout:2/441: mkdir d2/dd/d48/d22/d2b/d54/d60/d6c/d88 0 2024-09-16T00:25:58.144 INFO:tasks.workunit.client.1.smithi169.stdout:6/118: mkdir d2/d15/d1d/d25 0 2024-09-16T00:25:58.145 INFO:tasks.workunit.client.1.smithi169.stdout:6/119: rename d2/d15 to d2/d15/d26 22 2024-09-16T00:25:58.147 INFO:tasks.workunit.client.1.smithi169.stdout:5/201: unlink d2/da/l44 0 2024-09-16T00:25:58.150 INFO:tasks.workunit.client.1.smithi169.stdout:6/120: getdents d2/d15/d1d/d25 0 2024-09-16T00:25:58.150 INFO:tasks.workunit.client.0.smithi112.stdout:2/442: symlink d2/dd/d48/d22/d2b/d54/d60/d6c/d88/l89 0 2024-09-16T00:25:58.151 INFO:tasks.workunit.client.1.smithi169.stdout:7/330: link d7/d16/f1d d7/d16/d49/f68 0 2024-09-16T00:25:58.152 INFO:tasks.workunit.client.1.smithi169.stdout:8/195: getdents d7/dc/d17/d24 0 2024-09-16T00:25:58.153 INFO:tasks.workunit.client.1.smithi169.stdout:1/222: dread dc/df/f16 [0,4194304] 0 2024-09-16T00:25:58.153 INFO:tasks.workunit.client.1.smithi169.stdout:4/236: creat d2/d12/d13/d1f/f48 x:0 0 0 2024-09-16T00:25:58.154 INFO:tasks.workunit.client.0.smithi112.stdout:6/372: dwrite dd/df/d11/f49 [0,4194304] 0 2024-09-16T00:25:58.154 INFO:tasks.workunit.client.1.smithi169.stdout:4/237: dread - d2/dd/d1a/f1b zero size 2024-09-16T00:25:58.154 INFO:tasks.workunit.client.1.smithi169.stdout:8/196: rmdir d7/dc/d1d 39 2024-09-16T00:25:58.155 INFO:tasks.workunit.client.1.smithi169.stdout:6/121: mkdir d2/d15/d1d/d25/d27 0 2024-09-16T00:25:58.157 INFO:tasks.workunit.client.1.smithi169.stdout:6/122: write d2/d15/f18 [1251445,35684] 0 2024-09-16T00:25:58.157 INFO:tasks.workunit.client.1.smithi169.stdout:7/331: mknod d7/c69 0 2024-09-16T00:25:58.158 INFO:tasks.workunit.client.0.smithi112.stdout:6/373: mknod dd/df/d17/d27/d54/c82 0 2024-09-16T00:25:58.159 INFO:tasks.workunit.client.1.smithi169.stdout:8/197: write d7/fd [2109577,48257] 0 2024-09-16T00:25:58.159 INFO:tasks.workunit.client.1.smithi169.stdout:5/202: getdents d2/da/d10 0 2024-09-16T00:25:58.160 INFO:tasks.workunit.client.1.smithi169.stdout:1/223: creat dc/d2c/d3d/f3f x:0 0 0 2024-09-16T00:25:58.162 INFO:tasks.workunit.client.0.smithi112.stdout:2/443: dwrite d2/dd/d48/f1d [0,4194304] 0 2024-09-16T00:25:58.163 INFO:tasks.workunit.client.1.smithi169.stdout:7/332: creat d7/d16/f6a x:0 0 0 2024-09-16T00:25:58.163 INFO:tasks.workunit.client.0.smithi112.stdout:2/444: chown d2/dd/d48/d22/d2b/d54/d60/d6c/d88 1 1 2024-09-16T00:25:58.165 INFO:tasks.workunit.client.1.smithi169.stdout:4/238: creat d2/f49 x:0 0 0 2024-09-16T00:25:58.166 INFO:tasks.workunit.client.1.smithi169.stdout:5/203: dread d2/da/d10/f11 [0,4194304] 0 2024-09-16T00:25:58.167 INFO:tasks.workunit.client.1.smithi169.stdout:8/198: symlink d7/dc/d17/d24/d32/l44 0 2024-09-16T00:25:58.168 INFO:tasks.workunit.client.1.smithi169.stdout:4/239: dread - d2/dd/d1a/f1b zero size 2024-09-16T00:25:58.169 INFO:tasks.workunit.client.1.smithi169.stdout:5/204: readlink d2/da/d10/l18 0 2024-09-16T00:25:58.169 INFO:tasks.workunit.client.1.smithi169.stdout:6/123: dwrite d2/f19 [0,4194304] 0 2024-09-16T00:25:58.171 INFO:tasks.workunit.client.0.smithi112.stdout:2/445: dwrite d2/f14 [4194304,4194304] 0 2024-09-16T00:25:58.171 INFO:tasks.workunit.client.1.smithi169.stdout:7/333: link f5 d7/d5b/f6b 0 2024-09-16T00:25:58.172 INFO:tasks.workunit.client.1.smithi169.stdout:8/199: chown d7/dc/d17/l19 47595 1 2024-09-16T00:25:58.173 INFO:tasks.workunit.client.1.smithi169.stdout:1/224: dread dc/d18/f3b [0,4194304] 0 2024-09-16T00:25:58.176 INFO:tasks.workunit.client.1.smithi169.stdout:2/184: truncate d7/db/f23 5366887 0 2024-09-16T00:25:58.176 INFO:tasks.workunit.client.1.smithi169.stdout:8/200: write d7/fa [4644796,56757] 0 2024-09-16T00:25:58.176 INFO:tasks.workunit.client.1.smithi169.stdout:6/124: creat d2/f28 x:0 0 0 2024-09-16T00:25:58.177 INFO:tasks.workunit.client.1.smithi169.stdout:1/225: dread - dc/df/d29/d2a/f38 zero size 2024-09-16T00:25:58.180 INFO:tasks.workunit.client.0.smithi112.stdout:2/446: dwrite d2/dd/d48/d22/d2b/d30/f44 [0,4194304] 0 2024-09-16T00:25:58.183 INFO:tasks.workunit.client.1.smithi169.stdout:4/240: chown d2/d12/d13/d1f/d2f/f43 1861309 1 2024-09-16T00:25:58.184 INFO:tasks.workunit.client.0.smithi112.stdout:2/447: unlink d2/c33 0 2024-09-16T00:25:58.186 INFO:tasks.workunit.client.0.smithi112.stdout:2/448: fsync d2/dd/d48/d22/d2b/d3b/d49/f5c 0 2024-09-16T00:25:58.186 INFO:tasks.workunit.client.1.smithi169.stdout:2/185: rename d7/d1c/d26/l39 to d7/l41 0 2024-09-16T00:25:58.188 INFO:tasks.workunit.client.0.smithi112.stdout:2/449: rmdir d2/dd/d48/d22 39 2024-09-16T00:25:58.191 INFO:tasks.workunit.client.1.smithi169.stdout:5/205: dread d2/f29 [0,4194304] 0 2024-09-16T00:25:58.192 INFO:tasks.workunit.client.1.smithi169.stdout:4/241: write d2/dd/f18 [3930548,7815] 0 2024-09-16T00:25:58.193 INFO:tasks.workunit.client.0.smithi112.stdout:2/450: unlink d2/dd/d48/d22/d2b/d3b/c3d 0 2024-09-16T00:25:58.194 INFO:tasks.workunit.client.0.smithi112.stdout:2/451: chown d2/c5f 4107908 1 2024-09-16T00:25:58.195 INFO:tasks.workunit.client.1.smithi169.stdout:9/282: dwrite d5/da/d21/d23/f27 [0,4194304] 0 2024-09-16T00:25:58.195 INFO:tasks.workunit.client.0.smithi112.stdout:2/452: stat d2 0 2024-09-16T00:25:58.198 INFO:tasks.workunit.client.1.smithi169.stdout:1/226: symlink dc/d26/l40 0 2024-09-16T00:25:58.198 INFO:tasks.workunit.client.0.smithi112.stdout:2/453: rmdir d2/dd/d48/d22/d2b/d54/d60 39 2024-09-16T00:25:58.198 INFO:tasks.workunit.client.0.smithi112.stdout:6/374: dwrite dd/df/d13/f1c [0,4194304] 0 2024-09-16T00:25:58.200 INFO:tasks.workunit.client.1.smithi169.stdout:9/283: unlink d5/d12/c48 0 2024-09-16T00:25:58.202 INFO:tasks.workunit.client.1.smithi169.stdout:7/334: dwrite d7/da/d3a/f3c [0,4194304] 0 2024-09-16T00:25:58.202 INFO:tasks.workunit.client.1.smithi169.stdout:1/227: creat dc/d26/f41 x:0 0 0 2024-09-16T00:25:58.203 INFO:tasks.workunit.client.1.smithi169.stdout:8/201: dwrite d7/f10 [0,4194304] 0 2024-09-16T00:25:58.204 INFO:tasks.workunit.client.0.smithi112.stdout:6/375: rmdir dd/df/d13 39 2024-09-16T00:25:58.205 INFO:tasks.workunit.client.1.smithi169.stdout:1/228: stat dc/f21 0 2024-09-16T00:25:58.205 INFO:tasks.workunit.client.0.smithi112.stdout:2/454: mkdir d2/dd/d48/d8a 0 2024-09-16T00:25:58.207 INFO:tasks.workunit.client.0.smithi112.stdout:6/376: write dd/df/d11/f49 [3867732,123704] 0 2024-09-16T00:25:58.207 INFO:tasks.workunit.client.1.smithi169.stdout:9/284: fsync d5/d49/f59 0 2024-09-16T00:25:58.207 INFO:tasks.workunit.client.1.smithi169.stdout:8/202: write d7/dc/d17/f28 [3400788,66950] 0 2024-09-16T00:25:58.208 INFO:tasks.workunit.client.0.smithi112.stdout:6/377: chown dd/df/d13/f48 3 1 2024-09-16T00:25:58.209 INFO:tasks.workunit.client.1.smithi169.stdout:8/203: chown d7/dc/d17/f39 65306495 1 2024-09-16T00:25:58.209 INFO:tasks.workunit.client.1.smithi169.stdout:4/242: mknod d2/d12/d13/d1f/c4a 0 2024-09-16T00:25:58.210 INFO:tasks.workunit.client.0.smithi112.stdout:6/378: readlink dd/df/d10/l16 0 2024-09-16T00:25:58.210 INFO:tasks.workunit.client.1.smithi169.stdout:5/206: symlink d2/da/d10/d13/d1c/d40/d43/l45 0 2024-09-16T00:25:58.210 INFO:tasks.workunit.client.1.smithi169.stdout:9/285: read - d5/da/d21/f3e zero size 2024-09-16T00:25:58.211 INFO:tasks.workunit.client.0.smithi112.stdout:2/455: creat d2/f8b x:0 0 0 2024-09-16T00:25:58.211 INFO:tasks.workunit.client.1.smithi169.stdout:2/186: rmdir d7/d1c 39 2024-09-16T00:25:58.212 INFO:tasks.workunit.client.0.smithi112.stdout:6/379: chown dd/df/d11/f1e 935814203 1 2024-09-16T00:25:58.214 INFO:tasks.workunit.client.0.smithi112.stdout:6/380: write dd/df/d17/d27/f76 [703578,31275] 0 2024-09-16T00:25:58.214 INFO:tasks.workunit.client.1.smithi169.stdout:6/125: write d2/fb [2043762,25860] 0 2024-09-16T00:25:58.214 INFO:tasks.workunit.client.0.smithi112.stdout:2/456: chown d2/c21 11615942 1 2024-09-16T00:25:58.218 INFO:tasks.workunit.client.1.smithi169.stdout:1/229: rmdir dc/d2c 39 2024-09-16T00:25:58.221 INFO:tasks.workunit.client.1.smithi169.stdout:9/286: stat d5/l2a 0 2024-09-16T00:25:58.222 INFO:tasks.workunit.client.1.smithi169.stdout:6/126: unlink d2/l20 0 2024-09-16T00:25:58.222 INFO:tasks.workunit.client.0.smithi112.stdout:2/457: dwrite d2/f64 [0,4194304] 0 2024-09-16T00:25:58.222 INFO:tasks.workunit.client.0.smithi112.stdout:6/381: dwrite dd/df/d10/f60 [0,4194304] 0 2024-09-16T00:25:58.223 INFO:tasks.workunit.client.1.smithi169.stdout:5/207: rename d2/da/d10/d13/d31 to d2/da/d10/d13/d1c/d2d/d46 0 2024-09-16T00:25:58.224 INFO:tasks.workunit.client.0.smithi112.stdout:2/458: fsync f1 0 2024-09-16T00:25:58.226 INFO:tasks.workunit.client.0.smithi112.stdout:6/382: truncate dd/df/d13/d7c/f58 692261 0 2024-09-16T00:25:58.227 INFO:tasks.workunit.client.1.smithi169.stdout:9/287: creat d5/da/d4e/f6d x:0 0 0 2024-09-16T00:25:58.230 INFO:tasks.workunit.client.1.smithi169.stdout:6/127: chown d2/c1c 301347 1 2024-09-16T00:25:58.230 INFO:tasks.workunit.client.0.smithi112.stdout:6/383: read dd/f2a [333635,26756] 0 2024-09-16T00:25:58.230 INFO:tasks.workunit.client.0.smithi112.stdout:2/459: link d2/dd/d48/c18 d2/dd/c8c 0 2024-09-16T00:25:58.232 INFO:tasks.workunit.client.1.smithi169.stdout:7/335: getdents d7/d16/d49 0 2024-09-16T00:25:58.234 INFO:tasks.workunit.client.1.smithi169.stdout:5/208: dwrite d2/da/f38 [0,4194304] 0 2024-09-16T00:25:58.235 INFO:tasks.workunit.client.1.smithi169.stdout:9/288: symlink d5/d12/d61/l6e 0 2024-09-16T00:25:58.239 INFO:tasks.workunit.client.1.smithi169.stdout:9/289: mkdir d5/da/d5b/d6f 0 2024-09-16T00:25:58.240 INFO:tasks.workunit.client.0.smithi112.stdout:3/471: sync 2024-09-16T00:25:58.240 INFO:tasks.workunit.client.0.smithi112.stdout:4/325: sync 2024-09-16T00:25:58.241 INFO:tasks.workunit.client.1.smithi169.stdout:9/290: fsync d5/da/d21/d23/f40 0 2024-09-16T00:25:58.242 INFO:tasks.workunit.client.1.smithi169.stdout:6/128: dwrite d2/d22/f23 [0,4194304] 0 2024-09-16T00:25:58.243 INFO:tasks.workunit.client.1.smithi169.stdout:6/129: chown d2/d15/d1d/d25/d27 0 1 2024-09-16T00:25:58.243 INFO:tasks.workunit.client.0.smithi112.stdout:4/326: symlink d6/d39/d1c/l74 0 2024-09-16T00:25:58.244 INFO:tasks.workunit.client.1.smithi169.stdout:6/130: fdatasync d2/d15/f1b 0 2024-09-16T00:25:58.244 INFO:tasks.workunit.client.1.smithi169.stdout:9/291: mkdir d5/d49/d70 0 2024-09-16T00:25:58.245 INFO:tasks.workunit.client.0.smithi112.stdout:3/472: mknod d2/d9/d2b/d47/d52/d7a/d80/d83/d68/ca7 0 2024-09-16T00:25:58.247 INFO:tasks.workunit.client.0.smithi112.stdout:3/473: chown d2/d9/d2b/d47/d52/d7a/d80/d83/d68/d90 185 1 2024-09-16T00:25:58.248 INFO:tasks.workunit.client.0.smithi112.stdout:4/327: unlink d6/db/d1f/c33 0 2024-09-16T00:25:58.249 INFO:tasks.workunit.client.0.smithi112.stdout:3/474: chown d2/d9/d2b/d47/d52/f38 39 1 2024-09-16T00:25:58.251 INFO:tasks.workunit.client.0.smithi112.stdout:3/475: write d2/d9/d2b/f8f [55046,73123] 0 2024-09-16T00:25:58.251 INFO:tasks.workunit.client.0.smithi112.stdout:4/328: chown d6/db/df/c55 1769 1 2024-09-16T00:25:58.252 INFO:tasks.workunit.client.0.smithi112.stdout:3/476: chown d2/d9/d2b/d47/f30 106950 1 2024-09-16T00:25:58.252 INFO:tasks.workunit.client.1.smithi169.stdout:9/292: dwrite d5/da/d4e/f6d [0,4194304] 0 2024-09-16T00:25:58.253 INFO:tasks.workunit.client.0.smithi112.stdout:3/477: fdatasync d2/d9/f94 0 2024-09-16T00:25:58.254 INFO:tasks.workunit.client.0.smithi112.stdout:4/329: creat d6/db/d1f/d41/d49/f75 x:0 0 0 2024-09-16T00:25:58.254 INFO:tasks.workunit.client.0.smithi112.stdout:3/478: readlink d2/d9/d2b/d47/l6d 0 2024-09-16T00:25:58.256 INFO:tasks.workunit.client.1.smithi169.stdout:9/293: creat d5/d49/d70/f71 x:0 0 0 2024-09-16T00:25:58.257 INFO:tasks.workunit.client.0.smithi112.stdout:4/330: creat d6/d39/d1c/d66/f76 x:0 0 0 2024-09-16T00:25:58.257 INFO:tasks.workunit.client.1.smithi169.stdout:2/187: write f0 [1109152,36710] 0 2024-09-16T00:25:58.259 INFO:tasks.workunit.client.1.smithi169.stdout:8/204: truncate f2 2812920 0 2024-09-16T00:25:58.260 INFO:tasks.workunit.client.1.smithi169.stdout:2/188: truncate d7/db/d20/d21/f22 4213798 0 2024-09-16T00:25:58.262 INFO:tasks.workunit.client.1.smithi169.stdout:8/205: unlink d7/f8 0 2024-09-16T00:25:58.266 INFO:tasks.workunit.client.1.smithi169.stdout:8/206: symlink d7/dc/d1d/l45 0 2024-09-16T00:25:58.266 INFO:tasks.workunit.client.1.smithi169.stdout:2/189: dwrite f0 [0,4194304] 0 2024-09-16T00:25:58.266 INFO:tasks.workunit.client.1.smithi169.stdout:8/207: dread - d7/f2e zero size 2024-09-16T00:25:58.269 INFO:tasks.workunit.client.1.smithi169.stdout:2/190: unlink d7/db/d27/l34 0 2024-09-16T00:25:58.275 INFO:tasks.workunit.client.1.smithi169.stdout:2/191: dwrite f0 [0,4194304] 0 2024-09-16T00:25:58.277 INFO:tasks.workunit.client.1.smithi169.stdout:2/192: symlink d7/db/d27/l42 0 2024-09-16T00:25:58.279 INFO:tasks.workunit.client.0.smithi112.stdout:4/331: fsync d6/d39/d1c/d66/f76 0 2024-09-16T00:25:58.281 INFO:tasks.workunit.client.1.smithi169.stdout:4/243: write d2/dd/f24 [353941,61379] 0 2024-09-16T00:25:58.281 INFO:tasks.workunit.client.1.smithi169.stdout:4/244: dread - d2/f49 zero size 2024-09-16T00:25:58.283 INFO:tasks.workunit.client.0.smithi112.stdout:2/460: dwrite d2/dd/d48/d22/d2b/d54/d60/d6c/f7b [0,4194304] 0 2024-09-16T00:25:58.286 INFO:tasks.workunit.client.1.smithi169.stdout:4/245: mkdir d2/d12/d13/d4b 0 2024-09-16T00:25:58.286 INFO:tasks.workunit.client.1.smithi169.stdout:4/246: fdatasync d2/f49 0 2024-09-16T00:25:58.287 INFO:tasks.workunit.client.0.smithi112.stdout:2/461: truncate d2/dd/d48/d22/d2b/d54/d60/d6c/f7b 4569006 0 2024-09-16T00:25:58.287 INFO:tasks.workunit.client.1.smithi169.stdout:4/247: write d2/d12/d13/f2e [2656181,10164] 0 2024-09-16T00:25:58.290 INFO:tasks.workunit.client.1.smithi169.stdout:5/209: truncate d2/fb 11130510 0 2024-09-16T00:25:58.291 INFO:tasks.workunit.client.0.smithi112.stdout:2/462: rename d2/dd/d48/f2f to d2/f8d 0 2024-09-16T00:25:58.293 INFO:tasks.workunit.client.0.smithi112.stdout:2/463: chown d2/dd/d48/d22/d2b/d54/c65 48 1 2024-09-16T00:25:58.293 INFO:tasks.workunit.client.1.smithi169.stdout:6/131: truncate d2/f19 2616633 0 2024-09-16T00:25:58.294 INFO:tasks.workunit.client.0.smithi112.stdout:2/464: chown d2/dd/d48/cf 2 1 2024-09-16T00:25:58.296 INFO:tasks.workunit.client.0.smithi112.stdout:2/465: mknod d2/dd/d48/d22/d2b/d54/d60/c8e 0 2024-09-16T00:25:58.297 INFO:tasks.workunit.client.1.smithi169.stdout:7/336: dwrite d7/d16/f1d [0,4194304] 0 2024-09-16T00:25:58.297 INFO:tasks.workunit.client.1.smithi169.stdout:9/294: truncate d5/da/d4e/f6a 904333 0 2024-09-16T00:25:58.297 INFO:tasks.workunit.client.1.smithi169.stdout:5/210: truncate d2/da/d10/d13/d1c/f3b 131218 0 2024-09-16T00:25:58.299 INFO:tasks.workunit.client.0.smithi112.stdout:2/466: rename d2/dd/d48/d22/d2b/f57 to d2/dd/d48/d22/f8f 0 2024-09-16T00:25:58.300 INFO:tasks.workunit.client.1.smithi169.stdout:8/208: truncate d7/fd 5480498 0 2024-09-16T00:25:58.305 INFO:tasks.workunit.client.1.smithi169.stdout:5/211: creat d2/da/d10/d13/d24/f47 x:0 0 0 2024-09-16T00:25:58.307 INFO:tasks.workunit.client.1.smithi169.stdout:2/193: write f5 [5642998,25974] 0 2024-09-16T00:25:58.312 INFO:tasks.workunit.client.1.smithi169.stdout:6/132: dwrite d2/fb [0,4194304] 0 2024-09-16T00:25:58.316 INFO:tasks.workunit.client.0.smithi112.stdout:2/467: dread d2/dd/d48/d22/d2b/d3b/d49/f71 [0,4194304] 0 2024-09-16T00:25:58.316 INFO:tasks.workunit.client.1.smithi169.stdout:4/248: write d2/d12/f39 [1044719,43030] 0 2024-09-16T00:25:58.317 INFO:tasks.workunit.client.1.smithi169.stdout:5/212: mknod d2/da/d10/d13/d1c/d40/c48 0 2024-09-16T00:25:58.317 INFO:tasks.workunit.client.0.smithi112.stdout:2/468: mknod d2/dd/d48/d22/d2b/d30/c90 0 2024-09-16T00:25:58.319 INFO:tasks.workunit.client.0.smithi112.stdout:4/332: dread d6/db/d1f/d41/d49/d31/f51 [0,4194304] 0 2024-09-16T00:25:58.320 INFO:tasks.workunit.client.1.smithi169.stdout:9/295: dwrite d5/da/d21/f46 [0,4194304] 0 2024-09-16T00:25:58.321 INFO:tasks.workunit.client.1.smithi169.stdout:2/194: stat d7/d1c/d26/c38 0 2024-09-16T00:25:58.322 INFO:tasks.workunit.client.1.smithi169.stdout:7/337: dwrite d7/d16/d49/f68 [4194304,4194304] 0 2024-09-16T00:25:58.323 INFO:tasks.workunit.client.0.smithi112.stdout:4/333: creat d6/d39/d36/d5c/f77 x:0 0 0 2024-09-16T00:25:58.324 INFO:tasks.workunit.client.1.smithi169.stdout:8/209: dwrite d7/dc/d17/f39 [0,4194304] 0 2024-09-16T00:25:58.324 INFO:tasks.workunit.client.0.smithi112.stdout:2/469: truncate d2/dd/d48/d22/d67/f77 267562 0 2024-09-16T00:25:58.325 INFO:tasks.workunit.client.1.smithi169.stdout:6/133: chown d2/c24 175505 1 2024-09-16T00:25:58.326 INFO:tasks.workunit.client.1.smithi169.stdout:2/195: chown d7/d1c/d26/f2c 3024 1 2024-09-16T00:25:58.329 INFO:tasks.workunit.client.0.smithi112.stdout:2/470: unlink d2/dd/d48/d22/d2b/d30/l32 0 2024-09-16T00:25:58.331 INFO:tasks.workunit.client.1.smithi169.stdout:8/210: write d7/dc/d17/f39 [1001306,96533] 0 2024-09-16T00:25:58.331 INFO:tasks.workunit.client.1.smithi169.stdout:2/196: unlink d7/d1c/d26/l28 0 2024-09-16T00:25:58.331 INFO:tasks.workunit.client.1.smithi169.stdout:7/338: truncate d7/df/f60 22636 0 2024-09-16T00:25:58.332 INFO:tasks.workunit.client.0.smithi112.stdout:4/334: dwrite d6/db/f60 [0,4194304] 0 2024-09-16T00:25:58.333 INFO:tasks.workunit.client.1.smithi169.stdout:9/296: mknod d5/d12/d61/c72 0 2024-09-16T00:25:58.336 INFO:tasks.workunit.client.1.smithi169.stdout:7/339: creat d7/d5b/f6c x:0 0 0 2024-09-16T00:25:58.338 INFO:tasks.workunit.client.0.smithi112.stdout:2/471: dread d2/dd/d48/d22/d2b/d30/f44 [0,4194304] 0 2024-09-16T00:25:58.339 INFO:tasks.workunit.client.1.smithi169.stdout:4/249: link d2/ca d2/d12/d13/d1f/d2f/c4c 0 2024-09-16T00:25:58.339 INFO:tasks.workunit.client.1.smithi169.stdout:2/197: symlink d7/d1c/d3c/l43 0 2024-09-16T00:25:58.340 INFO:tasks.workunit.client.1.smithi169.stdout:9/297: unlink d5/da/d21/d23/f27 0 2024-09-16T00:25:58.343 INFO:tasks.workunit.client.1.smithi169.stdout:4/250: fsync d2/d12/d13/f25 0 2024-09-16T00:25:58.343 INFO:tasks.workunit.client.1.smithi169.stdout:9/298: chown d5/da/f16 5470 1 2024-09-16T00:25:58.346 INFO:tasks.workunit.client.1.smithi169.stdout:2/198: creat d7/db/d20/f44 x:0 0 0 2024-09-16T00:25:58.347 INFO:tasks.workunit.client.1.smithi169.stdout:2/199: read f5 [3219961,86279] 0 2024-09-16T00:25:58.348 INFO:tasks.workunit.client.1.smithi169.stdout:5/213: link d2/da/d10/d13/d1c/d2d/l2e d2/da/d10/d13/d1c/d40/d43/l49 0 2024-09-16T00:25:58.348 INFO:tasks.workunit.client.1.smithi169.stdout:7/340: mknod d7/da/c6d 0 2024-09-16T00:25:58.351 INFO:tasks.workunit.client.1.smithi169.stdout:5/214: fsync d2/da/d10/d13/d24/f47 0 2024-09-16T00:25:58.352 INFO:tasks.workunit.client.1.smithi169.stdout:9/299: mkdir d5/da/d73 0 2024-09-16T00:25:58.353 INFO:tasks.workunit.client.1.smithi169.stdout:4/251: mkdir d2/d12/d13/d4d 0 2024-09-16T00:25:58.354 INFO:tasks.workunit.client.1.smithi169.stdout:4/252: write d2/d12/f39 [3274242,66002] 0 2024-09-16T00:25:58.356 INFO:tasks.workunit.client.1.smithi169.stdout:2/200: rename d7/db/d20/d21/f33 to d7/db/f45 0 2024-09-16T00:25:58.357 INFO:tasks.workunit.client.1.smithi169.stdout:2/201: chown d7/db/f17 25 1 2024-09-16T00:25:58.359 INFO:tasks.workunit.client.1.smithi169.stdout:7/341: dread d7/d16/f25 [0,4194304] 0 2024-09-16T00:25:58.360 INFO:tasks.workunit.client.1.smithi169.stdout:7/342: chown d7/d16/d49 277 1 2024-09-16T00:25:58.363 INFO:tasks.workunit.client.1.smithi169.stdout:4/253: mkdir d2/d12/d13/d4e 0 2024-09-16T00:25:58.367 INFO:tasks.workunit.client.1.smithi169.stdout:2/202: unlink d7/db/d27/d29/c3f 0 2024-09-16T00:25:58.368 INFO:tasks.workunit.client.0.smithi112.stdout:4/335: dwrite d6/db/d1f/d41/d49/d31/f51 [0,4194304] 0 2024-09-16T00:25:58.368 INFO:tasks.workunit.client.1.smithi169.stdout:2/203: dread - d7/db/f17 zero size 2024-09-16T00:25:58.370 INFO:tasks.workunit.client.1.smithi169.stdout:4/254: mknod d2/dd/d1a/c4f 0 2024-09-16T00:25:58.371 INFO:tasks.workunit.client.0.smithi112.stdout:4/336: creat d6/db/df/d5e/f78 x:0 0 0 2024-09-16T00:25:58.372 INFO:tasks.workunit.client.1.smithi169.stdout:5/215: link d2/da/d10/l3c d2/da/d10/d13/d1c/d40/l4a 0 2024-09-16T00:25:58.373 INFO:tasks.workunit.client.0.smithi112.stdout:4/337: dread - d6/db/df/d5e/f78 zero size 2024-09-16T00:25:58.373 INFO:tasks.workunit.client.1.smithi169.stdout:7/343: dwrite d7/d5b/f5c [0,4194304] 0 2024-09-16T00:25:58.373 INFO:tasks.workunit.client.0.smithi112.stdout:4/338: chown d6/db/l1d 46908763 1 2024-09-16T00:25:58.374 INFO:tasks.workunit.client.1.smithi169.stdout:3/291: sync 2024-09-16T00:25:58.374 INFO:tasks.workunit.client.1.smithi169.stdout:7/344: chown d7/da 12 1 2024-09-16T00:25:58.375 INFO:tasks.workunit.client.1.smithi169.stdout:7/345: chown d7/f67 4 1 2024-09-16T00:25:58.376 INFO:tasks.workunit.client.1.smithi169.stdout:3/292: readlink d7/d1a/d1c/d35/d63/l51 0 2024-09-16T00:25:58.376 INFO:tasks.workunit.client.0.smithi112.stdout:4/339: mknod d6/db/df/d5e/d61/d6d/c79 0 2024-09-16T00:25:58.377 INFO:tasks.workunit.client.1.smithi169.stdout:2/204: mkdir d7/db/d27/d29/d46 0 2024-09-16T00:25:58.378 INFO:tasks.workunit.client.1.smithi169.stdout:8/211: truncate d7/dc/d2a/f30 1361359 0 2024-09-16T00:25:58.380 INFO:tasks.workunit.client.1.smithi169.stdout:2/205: readlink d7/l16 0 2024-09-16T00:25:58.383 INFO:tasks.workunit.client.1.smithi169.stdout:2/206: write d7/db/f3d [231638,49135] 0 2024-09-16T00:25:58.383 INFO:tasks.workunit.client.1.smithi169.stdout:8/212: truncate d7/f10 4294001 0 2024-09-16T00:25:58.385 INFO:tasks.workunit.client.0.smithi112.stdout:4/340: dwrite d6/da/f40 [0,4194304] 0 2024-09-16T00:25:58.386 INFO:tasks.workunit.client.1.smithi169.stdout:4/255: dwrite d2/f9 [0,4194304] 0 2024-09-16T00:25:58.388 INFO:tasks.workunit.client.1.smithi169.stdout:9/300: write d5/da/d4e/f6a [286023,120082] 0 2024-09-16T00:25:58.388 INFO:tasks.workunit.client.1.smithi169.stdout:8/213: stat d7/fd 0 2024-09-16T00:25:58.388 INFO:tasks.workunit.client.1.smithi169.stdout:2/207: truncate d7/db/f45 5607125 0 2024-09-16T00:25:58.388 INFO:tasks.workunit.client.1.smithi169.stdout:6/134: write d2/f19 [1330176,91709] 0 2024-09-16T00:25:58.391 INFO:tasks.workunit.client.1.smithi169.stdout:3/293: getdents d7/d1a/d1c/d35/d63 0 2024-09-16T00:25:58.393 INFO:tasks.workunit.client.1.smithi169.stdout:7/346: creat d7/d5b/d61/f6e x:0 0 0 2024-09-16T00:25:58.396 INFO:tasks.workunit.client.1.smithi169.stdout:4/256: chown d2/d12/d13/d27/d33/l42 1397811 1 2024-09-16T00:25:58.400 INFO:tasks.workunit.client.1.smithi169.stdout:8/214: mkdir d7/dc/d26/d46 0 2024-09-16T00:25:58.402 INFO:tasks.workunit.client.1.smithi169.stdout:3/294: mkdir d7/d1a/d1c/d23/d70 0 2024-09-16T00:25:58.403 INFO:tasks.workunit.client.1.smithi169.stdout:9/301: symlink d5/da/d73/l74 0 2024-09-16T00:25:58.403 INFO:tasks.workunit.client.1.smithi169.stdout:3/295: fdatasync d7/d1a/d1c/d35/d63/f53 0 2024-09-16T00:25:58.405 INFO:tasks.workunit.client.1.smithi169.stdout:2/208: dread d7/f24 [0,4194304] 0 2024-09-16T00:25:58.406 INFO:tasks.workunit.client.1.smithi169.stdout:3/296: write d7/f16 [1149144,72224] 0 2024-09-16T00:25:58.407 INFO:tasks.workunit.client.1.smithi169.stdout:7/347: rename d7/d16/f42 to d7/da/d3a/f6f 0 2024-09-16T00:25:58.408 INFO:tasks.workunit.client.1.smithi169.stdout:4/257: fdatasync d2/d12/f1e 0 2024-09-16T00:25:58.409 INFO:tasks.workunit.client.1.smithi169.stdout:2/209: rename d7/db to d7/db/d20/d47 22 2024-09-16T00:25:58.410 INFO:tasks.workunit.client.1.smithi169.stdout:4/258: write d2/f9 [69409,80719] 0 2024-09-16T00:25:58.410 INFO:tasks.workunit.client.1.smithi169.stdout:5/216: dread d2/da/d10/f27 [0,4194304] 0 2024-09-16T00:25:58.411 INFO:tasks.workunit.client.1.smithi169.stdout:2/210: chown d7/db/d20/d21 49 1 2024-09-16T00:25:58.411 INFO:tasks.workunit.client.1.smithi169.stdout:5/217: fsync d2/fe 0 2024-09-16T00:25:58.412 INFO:tasks.workunit.client.1.smithi169.stdout:7/348: mkdir d7/d16/d70 0 2024-09-16T00:25:58.414 INFO:tasks.workunit.client.1.smithi169.stdout:3/297: creat d7/d50/d6f/f71 x:0 0 0 2024-09-16T00:25:58.415 INFO:tasks.workunit.client.1.smithi169.stdout:1/230: sync 2024-09-16T00:25:58.415 INFO:tasks.workunit.client.1.smithi169.stdout:0/230: sync 2024-09-16T00:25:58.417 INFO:tasks.workunit.client.1.smithi169.stdout:9/302: link d5/da/d21/d23/f2d d5/da/d73/f75 0 2024-09-16T00:25:58.417 INFO:tasks.workunit.client.0.smithi112.stdout:7/418: sync 2024-09-16T00:25:58.418 INFO:tasks.workunit.client.0.smithi112.stdout:8/319: sync 2024-09-16T00:25:58.418 INFO:tasks.workunit.client.1.smithi169.stdout:6/135: link d2/d15/d1d/c21 d2/d15/d1d/d25/c29 0 2024-09-16T00:25:58.418 INFO:tasks.workunit.client.1.smithi169.stdout:7/349: fdatasync d7/f4c 0 2024-09-16T00:25:58.419 INFO:tasks.workunit.client.1.smithi169.stdout:6/136: chown d2/c10 0 1 2024-09-16T00:25:58.420 INFO:tasks.workunit.client.0.smithi112.stdout:7/419: unlink d6/d10/d16/f25 0 2024-09-16T00:25:58.421 INFO:tasks.workunit.client.1.smithi169.stdout:1/231: mkdir dc/df/d29/d42 0 2024-09-16T00:25:58.421 INFO:tasks.workunit.client.0.smithi112.stdout:8/320: dread d10/d12/f16 [0,4194304] 0 2024-09-16T00:25:58.422 INFO:tasks.workunit.client.1.smithi169.stdout:3/298: mknod d7/c72 0 2024-09-16T00:25:58.423 INFO:tasks.workunit.client.0.smithi112.stdout:7/420: write d6/f11 [2088820,103018] 0 2024-09-16T00:25:58.423 INFO:tasks.workunit.client.0.smithi112.stdout:8/321: symlink d10/d4e/d58/l69 0 2024-09-16T00:25:58.424 INFO:tasks.workunit.client.1.smithi169.stdout:4/259: mkdir d2/d12/d13/d4e/d50 0 2024-09-16T00:25:58.426 INFO:tasks.workunit.client.1.smithi169.stdout:6/137: creat d2/d15/d1d/d25/f2a x:0 0 0 2024-09-16T00:25:58.426 INFO:tasks.workunit.client.0.smithi112.stdout:8/322: creat d10/d45/f6a x:0 0 0 2024-09-16T00:25:58.428 INFO:tasks.workunit.client.0.smithi112.stdout:8/323: write d10/d45/f6a [932892,90627] 0 2024-09-16T00:25:58.428 INFO:tasks.workunit.client.0.smithi112.stdout:2/472: rename d2/dd/d48/f11 to d2/dd/f91 0 2024-09-16T00:25:58.430 INFO:tasks.workunit.client.1.smithi169.stdout:8/215: truncate d7/fa 2625464 0 2024-09-16T00:25:58.431 INFO:tasks.workunit.client.1.smithi169.stdout:3/299: creat d7/d1a/d1c/f73 x:0 0 0 2024-09-16T00:25:58.431 INFO:tasks.workunit.client.0.smithi112.stdout:4/341: rename d6/da/dc to d6/db/d1f/d41/d49/d7a 0 2024-09-16T00:25:58.432 INFO:tasks.workunit.client.0.smithi112.stdout:7/421: getdents d6/d10/d75/d8f 0 2024-09-16T00:25:58.433 INFO:tasks.workunit.client.0.smithi112.stdout:7/422: chown d6/d10/f6e 3 1 2024-09-16T00:25:58.433 INFO:tasks.workunit.client.0.smithi112.stdout:7/423: fsync d6/f23 0 2024-09-16T00:25:58.434 INFO:tasks.workunit.client.0.smithi112.stdout:7/424: readlink d6/l37 0 2024-09-16T00:25:58.434 INFO:tasks.workunit.client.1.smithi169.stdout:3/300: write d7/d1a/d1c/f40 [268999,11249] 0 2024-09-16T00:25:58.436 INFO:tasks.workunit.client.0.smithi112.stdout:8/324: rmdir d10/d4e/d58 39 2024-09-16T00:25:58.437 INFO:tasks.workunit.client.0.smithi112.stdout:8/325: fsync d10/d45/f6a 0 2024-09-16T00:25:58.438 INFO:tasks.workunit.client.1.smithi169.stdout:6/138: dwrite d2/d15/f1e [0,4194304] 0 2024-09-16T00:25:58.438 INFO:tasks.workunit.client.1.smithi169.stdout:3/301: chown d7/d1a/d1c/d35/d63/d37/l47 123 1 2024-09-16T00:25:58.439 INFO:tasks.workunit.client.1.smithi169.stdout:1/232: mknod dc/df/c43 0 2024-09-16T00:25:58.439 INFO:tasks.workunit.client.1.smithi169.stdout:5/218: dread d2/da/d10/d13/f35 [0,4194304] 0 2024-09-16T00:25:58.439 INFO:tasks.workunit.client.0.smithi112.stdout:7/425: mknod d6/d10/d1b/d39/c9c 0 2024-09-16T00:25:58.439 INFO:tasks.workunit.client.1.smithi169.stdout:6/139: write d2/f19 [291364,81834] 0 2024-09-16T00:25:58.441 INFO:tasks.workunit.client.1.smithi169.stdout:0/231: rename d0/d2c/d54/d16/d1a to d0/d2c/d54/d56 0 2024-09-16T00:25:58.441 INFO:tasks.workunit.client.0.smithi112.stdout:4/342: mkdir d6/d39/d1c/d2c/d69/d7b 0 2024-09-16T00:25:58.441 INFO:tasks.workunit.client.1.smithi169.stdout:2/211: truncate d7/db/d20/d21/f22 3225286 0 2024-09-16T00:25:58.443 INFO:tasks.workunit.client.1.smithi169.stdout:6/140: dread - d2/d15/d1d/d25/f2a zero size 2024-09-16T00:25:58.443 INFO:tasks.workunit.client.1.smithi169.stdout:3/302: truncate d7/d1a/d1c/f25 963046 0 2024-09-16T00:25:58.445 INFO:tasks.workunit.client.0.smithi112.stdout:8/326: unlink d10/c3e 0 2024-09-16T00:25:58.446 INFO:tasks.workunit.client.0.smithi112.stdout:8/327: readlink d10/d12/l5b 0 2024-09-16T00:25:58.446 INFO:tasks.workunit.client.0.smithi112.stdout:8/328: readlink d10/d12/d24/l2f 0 2024-09-16T00:25:58.448 INFO:tasks.workunit.client.1.smithi169.stdout:7/350: truncate d7/da/fc 3309546 0 2024-09-16T00:25:58.449 INFO:tasks.workunit.client.0.smithi112.stdout:2/473: link d2/dd/d48/d22/d2b/d54/d60/d6c/c76 d2/dd/d48/d22/d2b/d54/c92 0 2024-09-16T00:25:58.450 INFO:tasks.workunit.client.0.smithi112.stdout:2/474: fsync d2/dd/d48/d22/f6a 0 2024-09-16T00:25:58.451 INFO:tasks.workunit.client.0.smithi112.stdout:2/475: chown d2/dd/d48/d22/d2b/l3a 77474 1 2024-09-16T00:25:58.452 INFO:tasks.workunit.client.1.smithi169.stdout:3/303: creat d7/d50/d6f/f74 x:0 0 0 2024-09-16T00:25:58.453 INFO:tasks.workunit.client.0.smithi112.stdout:7/426: link d6/d10/d16/d49/d6c/f95 d6/d10/d16/f9d 0 2024-09-16T00:25:58.454 INFO:tasks.workunit.client.1.smithi169.stdout:1/233: link dc/d26/f41 dc/df/d29/d2a/f44 0 2024-09-16T00:25:58.456 INFO:tasks.workunit.client.0.smithi112.stdout:7/427: mknod d6/d10/d16/d96/c9e 0 2024-09-16T00:25:58.457 INFO:tasks.workunit.client.1.smithi169.stdout:4/260: dread d2/d12/d13/d27/f32 [0,4194304] 0 2024-09-16T00:25:58.458 INFO:tasks.workunit.client.0.smithi112.stdout:7/428: creat d6/d10/d16/d96/f9f x:0 0 0 2024-09-16T00:25:58.458 INFO:tasks.workunit.client.1.smithi169.stdout:5/219: dwrite d2/da/d10/d13/d24/f47 [0,4194304] 0 2024-09-16T00:25:58.458 INFO:tasks.workunit.client.1.smithi169.stdout:6/141: mknod d2/c2b 0 2024-09-16T00:25:58.459 INFO:tasks.workunit.client.1.smithi169.stdout:0/232: symlink d0/d50/l57 0 2024-09-16T00:25:58.460 INFO:tasks.workunit.client.1.smithi169.stdout:9/303: link d5/da/d2e/l38 d5/da/d21/l76 0 2024-09-16T00:25:58.460 INFO:tasks.workunit.client.0.smithi112.stdout:7/429: write d6/d10/d16/f4e [1054033,23290] 0 2024-09-16T00:25:58.463 INFO:tasks.workunit.client.0.smithi112.stdout:7/430: truncate d6/d10/d16/d49/d6c/f33 1116024 0 2024-09-16T00:25:58.464 INFO:tasks.workunit.client.1.smithi169.stdout:7/351: write d7/f31 [1052680,60830] 0 2024-09-16T00:25:58.464 INFO:tasks.workunit.client.0.smithi112.stdout:4/343: write f4 [918632,22396] 0 2024-09-16T00:25:58.466 INFO:tasks.workunit.client.1.smithi169.stdout:8/216: dwrite d7/dc/d2a/f42 [0,4194304] 0 2024-09-16T00:25:58.466 INFO:tasks.workunit.client.1.smithi169.stdout:2/212: dwrite d7/d1c/d26/f36 [0,4194304] 0 2024-09-16T00:25:58.467 INFO:tasks.workunit.client.0.smithi112.stdout:4/344: mkdir d6/d7c 0 2024-09-16T00:25:58.470 INFO:tasks.workunit.client.1.smithi169.stdout:3/304: creat d7/d50/d6f/f75 x:0 0 0 2024-09-16T00:25:58.471 INFO:tasks.workunit.client.0.smithi112.stdout:8/329: truncate d10/d12/d24/f2e 504055 0 2024-09-16T00:25:58.471 INFO:tasks.workunit.client.1.smithi169.stdout:0/233: read d0/d2c/d54/d21/f49 [332303,88084] 0 2024-09-16T00:25:58.473 INFO:tasks.workunit.client.0.smithi112.stdout:0/422: sync 2024-09-16T00:25:58.477 INFO:tasks.workunit.client.0.smithi112.stdout:4/345: dwrite d6/d39/d1c/f2f [0,4194304] 0 2024-09-16T00:25:58.479 INFO:tasks.workunit.client.1.smithi169.stdout:7/352: write d7/da/d3a/f3c [3292285,4786] 0 2024-09-16T00:25:58.479 INFO:tasks.workunit.client.0.smithi112.stdout:8/330: dread d10/f46 [0,4194304] 0 2024-09-16T00:25:58.479 INFO:tasks.workunit.client.0.smithi112.stdout:4/346: chown d6/db/df/d5e/d61/c4e 17937 1 2024-09-16T00:25:58.481 INFO:tasks.workunit.client.1.smithi169.stdout:6/142: dwrite d2/d15/f1b [0,4194304] 0 2024-09-16T00:25:58.483 INFO:tasks.workunit.client.0.smithi112.stdout:2/476: dread d2/dd/d48/d22/d2b/f34 [0,4194304] 0 2024-09-16T00:25:58.484 INFO:tasks.workunit.client.1.smithi169.stdout:4/261: mknod d2/d12/d13/d4e/d50/c51 0 2024-09-16T00:25:58.484 INFO:tasks.workunit.client.1.smithi169.stdout:5/220: write d2/da/d10/f27 [893562,7284] 0 2024-09-16T00:25:58.485 INFO:tasks.workunit.client.1.smithi169.stdout:9/304: creat d5/da/d5b/d6f/f77 x:0 0 0 2024-09-16T00:25:58.485 INFO:tasks.workunit.client.1.smithi169.stdout:3/305: read d7/d1a/d1c/f54 [5420413,24209] 0 2024-09-16T00:25:58.487 INFO:tasks.workunit.client.0.smithi112.stdout:4/347: creat d6/d39/d1c/d66/f7d x:0 0 0 2024-09-16T00:25:58.489 INFO:tasks.workunit.client.1.smithi169.stdout:6/143: fsync d2/f28 0 2024-09-16T00:25:58.492 INFO:tasks.workunit.client.0.smithi112.stdout:2/477: fdatasync d2/dd/d48/d22/d2b/d54/d60/d6c/f7b 0 2024-09-16T00:25:58.493 INFO:tasks.workunit.client.0.smithi112.stdout:0/423: link d5/da/d18/c7f d5/da/d57/d76/d47/c83 0 2024-09-16T00:25:58.493 INFO:tasks.workunit.client.1.smithi169.stdout:1/234: dwrite dc/d2c/d3d/f3f [0,4194304] 0 2024-09-16T00:25:58.494 INFO:tasks.workunit.client.0.smithi112.stdout:0/424: dread - d5/da/f77 zero size 2024-09-16T00:25:58.496 INFO:tasks.workunit.client.1.smithi169.stdout:0/234: creat d0/d2c/d54/d21/d31/f58 x:0 0 0 2024-09-16T00:25:58.496 INFO:tasks.workunit.client.0.smithi112.stdout:8/331: dwrite d10/d12/d1c/d3b/f4d [0,4194304] 0 2024-09-16T00:25:58.503 INFO:tasks.workunit.client.1.smithi169.stdout:8/217: link d7/f12 d7/dc/d1d/f47 0 2024-09-16T00:25:58.504 INFO:tasks.workunit.client.0.smithi112.stdout:8/332: mknod d10/d12/d1c/d3b/c6b 0 2024-09-16T00:25:58.505 INFO:tasks.workunit.client.1.smithi169.stdout:6/144: mknod d2/d15/c2c 0 2024-09-16T00:25:58.509 INFO:tasks.workunit.client.1.smithi169.stdout:2/213: dwrite d7/db/f45 [4194304,4194304] 0 2024-09-16T00:25:58.510 INFO:tasks.workunit.client.0.smithi112.stdout:2/478: dwrite f1 [0,4194304] 0 2024-09-16T00:25:58.511 INFO:tasks.workunit.client.1.smithi169.stdout:9/305: creat d5/da/d5b/d6f/f78 x:0 0 0 2024-09-16T00:25:58.511 INFO:tasks.workunit.client.1.smithi169.stdout:0/235: creat d0/d2c/d54/d56/f59 x:0 0 0 2024-09-16T00:25:58.513 INFO:tasks.workunit.client.0.smithi112.stdout:4/348: dwrite d6/db/df/d5e/d61/f3e [0,4194304] 0 2024-09-16T00:25:58.515 INFO:tasks.workunit.client.0.smithi112.stdout:0/425: dwrite d5/f23 [0,4194304] 0 2024-09-16T00:25:58.517 INFO:tasks.workunit.client.0.smithi112.stdout:7/431: dread d6/d10/d16/f4e [4194304,4194304] 0 2024-09-16T00:25:58.517 INFO:tasks.workunit.client.0.smithi112.stdout:8/333: getdents d10/d12 0 2024-09-16T00:25:58.518 INFO:tasks.workunit.client.1.smithi169.stdout:3/306: dwrite d7/d1a/d1c/d35/d63/f5f [0,4194304] 0 2024-09-16T00:25:58.520 INFO:tasks.workunit.client.0.smithi112.stdout:8/334: chown d10/c3d 2450 1 2024-09-16T00:25:58.520 INFO:tasks.workunit.client.0.smithi112.stdout:4/349: mkdir d6/d39/d36/d5c/d7e 0 2024-09-16T00:25:58.521 INFO:tasks.workunit.client.0.smithi112.stdout:0/426: unlink d5/cb 0 2024-09-16T00:25:58.522 INFO:tasks.workunit.client.0.smithi112.stdout:4/350: read - d6/db/df/f5f zero size 2024-09-16T00:25:58.523 INFO:tasks.workunit.client.0.smithi112.stdout:7/432: creat d6/fa0 x:0 0 0 2024-09-16T00:25:58.523 INFO:tasks.workunit.client.0.smithi112.stdout:9/587: sync 2024-09-16T00:25:58.523 INFO:tasks.workunit.client.0.smithi112.stdout:3/479: sync 2024-09-16T00:25:58.523 INFO:tasks.workunit.client.0.smithi112.stdout:6/384: sync 2024-09-16T00:25:58.523 INFO:tasks.workunit.client.0.smithi112.stdout:5/378: sync 2024-09-16T00:25:58.524 INFO:tasks.workunit.client.0.smithi112.stdout:1/391: sync 2024-09-16T00:25:58.526 INFO:tasks.workunit.client.0.smithi112.stdout:8/335: mkdir d10/d12/d1c/d61/d6c 0 2024-09-16T00:25:58.527 INFO:tasks.workunit.client.0.smithi112.stdout:2/479: rename d2/c37 to d2/c93 0 2024-09-16T00:25:58.528 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:25:58 smithi169 ceph-mon[34653]: pgmap v252: 65 pgs: 65 active+clean; 713 MiB data, 3.6 GiB used, 533 GiB / 536 GiB avail; 7.2 MiB/s rd, 66 MiB/s wr, 198 op/s 2024-09-16T00:25:58.529 INFO:tasks.workunit.client.1.smithi169.stdout:9/306: unlink d5/da/d2e/c66 0 2024-09-16T00:25:58.529 INFO:tasks.workunit.client.1.smithi169.stdout:0/236: write d0/d2/fd [3595101,105108] 0 2024-09-16T00:25:58.529 INFO:tasks.workunit.client.1.smithi169.stdout:2/214: dread f5 [0,4194304] 0 2024-09-16T00:25:58.529 INFO:tasks.workunit.client.1.smithi169.stdout:0/237: chown d0/d2c 507 1 2024-09-16T00:25:58.531 INFO:tasks.workunit.client.0.smithi112.stdout:0/427: rename d5/da/d1d/f56 to d5/da/d21/d6f/f84 0 2024-09-16T00:25:58.532 INFO:tasks.workunit.client.0.smithi112.stdout:7/433: fsync d6/d10/d16/f9d 0 2024-09-16T00:25:58.533 INFO:tasks.workunit.client.0.smithi112.stdout:2/480: dread - d2/d68/f7f zero size 2024-09-16T00:25:58.533 INFO:tasks.workunit.client.0.smithi112.stdout:1/392: dread - d0/d14/d1c/d26/f73 zero size 2024-09-16T00:25:58.534 INFO:tasks.workunit.client.1.smithi169.stdout:8/218: dread d7/dc/d2a/f42 [0,4194304] 0 2024-09-16T00:25:58.534 INFO:tasks.workunit.client.1.smithi169.stdout:8/219: chown d7/dc/l15 731353 1 2024-09-16T00:25:58.534 INFO:tasks.workunit.client.0.smithi112.stdout:8/336: unlink d10/d12/f33 0 2024-09-16T00:25:58.535 INFO:tasks.workunit.client.0.smithi112.stdout:4/351: truncate d6/f9 9975667 0 2024-09-16T00:25:58.535 INFO:tasks.workunit.client.0.smithi112.stdout:6/385: dread dd/df/f4c [0,4194304] 0 2024-09-16T00:25:58.535 INFO:tasks.workunit.client.0.smithi112.stdout:1/393: stat d0/f4e 0 2024-09-16T00:25:58.536 INFO:tasks.workunit.client.0.smithi112.stdout:5/379: mknod d4/d4f/c84 0 2024-09-16T00:25:58.539 INFO:tasks.workunit.client.1.smithi169.stdout:6/145: link d2/c24 d2/d22/c2d 0 2024-09-16T00:25:58.540 INFO:tasks.workunit.client.1.smithi169.stdout:3/307: symlink d7/d1a/d1c/l76 0 2024-09-16T00:25:58.541 INFO:tasks.workunit.client.0.smithi112.stdout:6/386: fsync dd/df/d13/d7c/f58 0 2024-09-16T00:25:58.542 INFO:tasks.workunit.client.0.smithi112.stdout:4/352: readlink d6/d39/l3a 0 2024-09-16T00:25:58.542 INFO:tasks.workunit.client.0.smithi112.stdout:7/434: mkdir d6/d10/da1 0 2024-09-16T00:25:58.542 INFO:tasks.workunit.client.0.smithi112.stdout:3/480: creat d2/d9/d2b/d47/d52/d7a/d80/d83/d49/fa8 x:0 0 0 2024-09-16T00:25:58.542 INFO:tasks.workunit.client.1.smithi169.stdout:2/215: creat d7/db/d20/d21/f48 x:0 0 0 2024-09-16T00:25:58.543 INFO:tasks.workunit.client.0.smithi112.stdout:2/481: write d2/dd/d48/d22/f6a [1740813,45837] 0 2024-09-16T00:25:58.544 INFO:tasks.workunit.client.1.smithi169.stdout:6/146: mkdir d2/d2e 0 2024-09-16T00:25:58.545 INFO:tasks.workunit.client.0.smithi112.stdout:9/588: dwrite d1/f82 [0,4194304] 0 2024-09-16T00:25:58.546 INFO:tasks.workunit.client.0.smithi112.stdout:5/380: mknod d4/d7/d26/d4c/c85 0 2024-09-16T00:25:58.547 INFO:tasks.workunit.client.0.smithi112.stdout:7/435: fdatasync d6/d10/d1b/d39/f9a 0 2024-09-16T00:25:58.549 INFO:tasks.workunit.client.1.smithi169.stdout:2/216: chown f2 24 1 2024-09-16T00:25:58.550 INFO:tasks.workunit.client.0.smithi112.stdout:0/428: dread d5/da/d57/d76/f3c [0,4194304] 0 2024-09-16T00:25:58.550 INFO:tasks.workunit.client.0.smithi112.stdout:5/381: chown d4/d4f/f76 132 1 2024-09-16T00:25:58.550 INFO:tasks.workunit.client.0.smithi112.stdout:1/394: link d0/d14/d44/l4d d0/d14/d1c/l7a 0 2024-09-16T00:25:58.550 INFO:tasks.workunit.client.1.smithi169.stdout:2/217: stat d7/d1c 0 2024-09-16T00:25:58.550 INFO:tasks.workunit.client.1.smithi169.stdout:8/220: dread d7/dc/d17/f39 [0,4194304] 0 2024-09-16T00:25:58.551 INFO:tasks.workunit.client.0.smithi112.stdout:6/387: symlink dd/df/d13/l83 0 2024-09-16T00:25:58.552 INFO:tasks.workunit.client.0.smithi112.stdout:4/353: truncate d6/db/d1f/d41/d49/f17 1705122 0 2024-09-16T00:25:58.552 INFO:tasks.workunit.client.1.smithi169.stdout:7/353: dread d7/da/fc [0,4194304] 0 2024-09-16T00:25:58.552 INFO:tasks.workunit.client.0.smithi112.stdout:2/482: fsync d2/dd/d48/d22/d2b/d54/f55 0 2024-09-16T00:25:58.553 INFO:tasks.workunit.client.1.smithi169.stdout:6/147: rename d2/f28 to d2/d2e/f2f 0 2024-09-16T00:25:58.553 INFO:tasks.workunit.client.0.smithi112.stdout:6/388: stat dd/df/d10/d47/f57 0 2024-09-16T00:25:58.557 INFO:tasks.workunit.client.1.smithi169.stdout:4/262: dread d2/d12/f39 [0,4194304] 0 2024-09-16T00:25:58.557 INFO:tasks.workunit.client.0.smithi112.stdout:7/436: mkdir d6/d10/da1/da2 0 2024-09-16T00:25:58.557 INFO:tasks.workunit.client.0.smithi112.stdout:6/389: dread - dd/df/d17/f59 zero size 2024-09-16T00:25:58.558 INFO:tasks.workunit.client.1.smithi169.stdout:9/307: link d5/da/ff d5/da/d2e/f79 0 2024-09-16T00:25:58.558 INFO:tasks.workunit.client.0.smithi112.stdout:0/429: rmdir d5/da/d57/d76/d74 39 2024-09-16T00:25:58.559 INFO:tasks.workunit.client.0.smithi112.stdout:5/382: write d4/dd/d12/f11 [1051523,74183] 0 2024-09-16T00:25:58.560 INFO:tasks.workunit.client.0.smithi112.stdout:4/354: creat d6/db/df/d5e/d61/d6d/f7f x:0 0 0 2024-09-16T00:25:58.563 INFO:tasks.workunit.client.1.smithi169.stdout:6/148: rmdir d2/d15/d1d/d25 39 2024-09-16T00:25:58.564 INFO:tasks.workunit.client.1.smithi169.stdout:7/354: mkdir d7/d71 0 2024-09-16T00:25:58.564 INFO:tasks.workunit.client.1.smithi169.stdout:2/218: getdents d7/db/d27/d29/d46 0 2024-09-16T00:25:58.565 INFO:tasks.workunit.client.0.smithi112.stdout:7/437: creat d6/d10/d1b/d39/fa3 x:0 0 0 2024-09-16T00:25:58.565 INFO:tasks.workunit.client.0.smithi112.stdout:4/355: dread - d6/d39/d1c/d66/f76 zero size 2024-09-16T00:25:58.566 INFO:tasks.workunit.client.1.smithi169.stdout:1/235: dwrite dc/df/d29/d2a/f44 [0,4194304] 0 2024-09-16T00:25:58.566 INFO:tasks.workunit.client.0.smithi112.stdout:3/481: dwrite d2/d9/d2b/d47/d52/d7a/d80/f93 [0,4194304] 0 2024-09-16T00:25:58.567 INFO:tasks.workunit.client.0.smithi112.stdout:2/483: rename d2/dd/d48/d22/d2b/d3b to d2/dd/d48/d94 0 2024-09-16T00:25:58.567 INFO:tasks.workunit.client.0.smithi112.stdout:9/589: dwrite d1/d5/fa6 [0,4194304] 0 2024-09-16T00:25:58.570 INFO:tasks.workunit.client.0.smithi112.stdout:0/430: unlink d5/da/d21/l58 0 2024-09-16T00:25:58.571 INFO:tasks.workunit.client.1.smithi169.stdout:7/355: write d7/d16/f6a [609500,127944] 0 2024-09-16T00:25:58.574 INFO:tasks.workunit.client.0.smithi112.stdout:4/356: mknod d6/db/d1f/c80 0 2024-09-16T00:25:58.574 INFO:tasks.workunit.client.0.smithi112.stdout:3/482: chown d2/d9/d2b/f8f 29972325 1 2024-09-16T00:25:58.574 INFO:tasks.workunit.client.1.smithi169.stdout:4/263: creat d2/d12/d13/d1f/f52 x:0 0 0 2024-09-16T00:25:58.575 INFO:tasks.workunit.client.0.smithi112.stdout:5/383: write d4/dd/d49/d70/f7e [466130,92909] 0 2024-09-16T00:25:58.575 INFO:tasks.workunit.client.0.smithi112.stdout:7/438: write d6/d10/d16/f20 [1338931,13258] 0 2024-09-16T00:25:58.576 INFO:tasks.workunit.client.1.smithi169.stdout:5/221: dread d2/da/f28 [0,4194304] 0 2024-09-16T00:25:58.576 INFO:tasks.workunit.client.0.smithi112.stdout:9/590: mknod d1/d5/cce 0 2024-09-16T00:25:58.579 INFO:tasks.workunit.client.0.smithi112.stdout:6/390: dwrite dd/df/f3d [0,4194304] 0 2024-09-16T00:25:58.581 INFO:tasks.workunit.client.1.smithi169.stdout:9/308: dwrite d5/da/d21/f30 [0,4194304] 0 2024-09-16T00:25:58.581 INFO:tasks.workunit.client.0.smithi112.stdout:2/484: fsync d2/dd/d48/f2d 0 2024-09-16T00:25:58.582 INFO:tasks.workunit.client.0.smithi112.stdout:7/439: truncate d6/d10/f6e 87861 0 2024-09-16T00:25:58.583 INFO:tasks.workunit.client.0.smithi112.stdout:4/357: creat d6/db/df/d5e/d61/d64/f81 x:0 0 0 2024-09-16T00:25:58.583 INFO:tasks.workunit.client.0.smithi112.stdout:7/440: chown d6/f22 285676405 1 2024-09-16T00:25:58.585 INFO:tasks.workunit.client.0.smithi112.stdout:3/483: mkdir d2/d9/d2b/d47/d78/da9 0 2024-09-16T00:25:58.586 INFO:tasks.workunit.client.1.smithi169.stdout:6/149: dread d2/fb [0,4194304] 0 2024-09-16T00:25:58.586 INFO:tasks.workunit.client.1.smithi169.stdout:2/219: mknod d7/db/d20/c49 0 2024-09-16T00:25:58.587 INFO:tasks.workunit.client.1.smithi169.stdout:0/238: dread d0/d2/d4b/f52 [0,4194304] 0 2024-09-16T00:25:58.587 INFO:tasks.workunit.client.0.smithi112.stdout:6/391: unlink dd/df/d17/l43 0 2024-09-16T00:25:58.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:25:58 smithi112 ceph-mon[27869]: pgmap v252: 65 pgs: 65 active+clean; 713 MiB data, 3.6 GiB used, 533 GiB / 536 GiB avail; 7.2 MiB/s rd, 66 MiB/s wr, 198 op/s 2024-09-16T00:25:58.589 INFO:tasks.workunit.client.0.smithi112.stdout:9/591: symlink d1/d5/d2c/d6f/dbf/lcf 0 2024-09-16T00:25:58.590 INFO:tasks.workunit.client.1.smithi169.stdout:1/236: rename dc/df to dc/d36/d3a/d45 0 2024-09-16T00:25:58.592 INFO:tasks.workunit.client.0.smithi112.stdout:7/441: symlink d6/d10/d1b/d39/la4 0 2024-09-16T00:25:58.594 INFO:tasks.workunit.client.0.smithi112.stdout:2/485: dwrite d2/dd/d48/d22/d2b/f31 [0,4194304] 0 2024-09-16T00:25:58.594 INFO:tasks.workunit.client.0.smithi112.stdout:4/358: creat d6/db/df/d3c/f82 x:0 0 0 2024-09-16T00:25:58.596 INFO:tasks.workunit.client.0.smithi112.stdout:2/486: write d2/dd/f87 [690591,54250] 0 2024-09-16T00:25:58.596 INFO:tasks.workunit.client.1.smithi169.stdout:9/309: creat d5/d12/d61/f7a x:0 0 0 2024-09-16T00:25:58.597 INFO:tasks.workunit.client.0.smithi112.stdout:6/392: rename dd/d15/d5a to dd/df/d17/d27/d84 0 2024-09-16T00:25:58.598 INFO:tasks.workunit.client.0.smithi112.stdout:6/393: dread - dd/df/d11/f80 zero size 2024-09-16T00:25:58.599 INFO:tasks.workunit.client.0.smithi112.stdout:9/592: rmdir d1/d5/d6/d2d/d6a 39 2024-09-16T00:25:58.599 INFO:tasks.workunit.client.1.smithi169.stdout:4/264: dwrite d2/f3 [0,4194304] 0 2024-09-16T00:25:58.599 INFO:tasks.workunit.client.0.smithi112.stdout:7/442: read d6/d10/f32 [919330,16629] 0 2024-09-16T00:25:58.600 INFO:tasks.workunit.client.0.smithi112.stdout:7/443: fsync d6/f2a 0 2024-09-16T00:25:58.602 INFO:tasks.workunit.client.1.smithi169.stdout:4/265: stat d2/d12/l19 0 2024-09-16T00:25:58.604 INFO:tasks.workunit.client.0.smithi112.stdout:4/359: symlink d6/db/d1f/l83 0 2024-09-16T00:25:58.605 INFO:tasks.workunit.client.0.smithi112.stdout:0/431: dread f0 [0,4194304] 0 2024-09-16T00:25:58.605 INFO:tasks.workunit.client.0.smithi112.stdout:2/487: mknod d2/dd/c95 0 2024-09-16T00:25:58.606 INFO:tasks.workunit.client.1.smithi169.stdout:6/150: creat d2/f30 x:0 0 0 2024-09-16T00:25:58.607 INFO:tasks.workunit.client.0.smithi112.stdout:0/432: fdatasync d5/da/f53 0 2024-09-16T00:25:58.608 INFO:tasks.workunit.client.1.smithi169.stdout:6/151: stat d2/d15 0 2024-09-16T00:25:58.609 INFO:tasks.workunit.client.0.smithi112.stdout:6/394: symlink dd/df/d13/d18/l85 0 2024-09-16T00:25:58.609 INFO:tasks.workunit.client.0.smithi112.stdout:8/337: truncate d10/f34 1525147 0 2024-09-16T00:25:58.611 INFO:tasks.workunit.client.1.smithi169.stdout:2/220: rename d7/f24 to d7/db/d20/d21/f4a 0 2024-09-16T00:25:58.611 INFO:tasks.workunit.client.0.smithi112.stdout:1/395: rmdir d0/d14 39 2024-09-16T00:25:58.613 INFO:tasks.workunit.client.1.smithi169.stdout:2/221: readlink d7/db/d27/l30 0 2024-09-16T00:25:58.615 INFO:tasks.workunit.client.1.smithi169.stdout:1/237: creat dc/d26/f46 x:0 0 0 2024-09-16T00:25:58.615 INFO:tasks.workunit.client.0.smithi112.stdout:3/484: dread d2/fb [0,4194304] 0 2024-09-16T00:25:58.615 INFO:tasks.workunit.client.0.smithi112.stdout:7/444: dwrite d6/d10/d1b/f52 [0,4194304] 0 2024-09-16T00:25:58.617 INFO:tasks.workunit.client.1.smithi169.stdout:3/308: truncate d7/d1a/f32 1307326 0 2024-09-16T00:25:58.618 INFO:tasks.workunit.client.0.smithi112.stdout:3/485: readlink d2/d9/d2b/l36 0 2024-09-16T00:25:58.618 INFO:tasks.workunit.client.0.smithi112.stdout:4/360: creat d6/da/f84 x:0 0 0 2024-09-16T00:25:58.620 INFO:tasks.workunit.client.1.smithi169.stdout:7/356: rmdir d7/d16/d70 0 2024-09-16T00:25:58.620 INFO:tasks.workunit.client.0.smithi112.stdout:2/488: mkdir d2/dd/d96 0 2024-09-16T00:25:58.620 INFO:tasks.workunit.client.0.smithi112.stdout:7/445: chown d6/d10/c50 830265 1 2024-09-16T00:25:58.622 INFO:tasks.workunit.client.0.smithi112.stdout:3/486: chown d2/d9/d2b/d47/d52/d7a/d80 513145230 1 2024-09-16T00:25:58.624 INFO:tasks.workunit.client.1.smithi169.stdout:8/221: dread d7/dc/d1d/f47 [0,4194304] 0 2024-09-16T00:25:58.625 INFO:tasks.workunit.client.1.smithi169.stdout:2/222: dwrite d7/db/f17 [0,4194304] 0 2024-09-16T00:25:58.625 INFO:tasks.workunit.client.0.smithi112.stdout:6/395: dwrite dd/df/f6e [0,4194304] 0 2024-09-16T00:25:58.625 INFO:tasks.workunit.client.0.smithi112.stdout:4/361: mknod d6/d39/d1c/d6e/c85 0 2024-09-16T00:25:58.627 INFO:tasks.workunit.client.0.smithi112.stdout:1/396: unlink d0/d14/fd 0 2024-09-16T00:25:58.627 INFO:tasks.workunit.client.1.smithi169.stdout:0/239: link d0/d2c/d54/d21/d31/f58 d0/d2c/d54/d16/f5a 0 2024-09-16T00:25:58.631 INFO:tasks.workunit.client.1.smithi169.stdout:7/357: creat d7/d16/d49/f72 x:0 0 0 2024-09-16T00:25:58.631 INFO:tasks.workunit.client.0.smithi112.stdout:7/446: creat d6/d10/d75/fa5 x:0 0 0 2024-09-16T00:25:58.632 INFO:tasks.workunit.client.0.smithi112.stdout:2/489: creat d2/dd/d48/d22/d2b/d30/f97 x:0 0 0 2024-09-16T00:25:58.633 INFO:tasks.workunit.client.0.smithi112.stdout:3/487: creat d2/d9/faa x:0 0 0 2024-09-16T00:25:58.634 INFO:tasks.workunit.client.0.smithi112.stdout:4/362: creat d6/d39/d36/d5c/d7e/f86 x:0 0 0 2024-09-16T00:25:58.634 INFO:tasks.workunit.client.0.smithi112.stdout:7/447: rmdir d6/d10 39 2024-09-16T00:25:58.636 INFO:tasks.workunit.client.0.smithi112.stdout:3/488: stat d2/d9/d2b/d2c/d69/d92/f96 0 2024-09-16T00:25:58.637 INFO:tasks.workunit.client.1.smithi169.stdout:6/152: link d2/d22/f23 d2/d15/d1d/d25/f31 0 2024-09-16T00:25:58.638 INFO:tasks.workunit.client.0.smithi112.stdout:8/338: dwrite d10/f1b [0,4194304] 0 2024-09-16T00:25:58.638 INFO:tasks.workunit.client.0.smithi112.stdout:4/363: chown d6/db/d1f/l83 0 1 2024-09-16T00:25:58.638 INFO:tasks.workunit.client.0.smithi112.stdout:9/593: dread d1/d5/d6/d2d/d39/f96 [0,4194304] 0 2024-09-16T00:25:58.639 INFO:tasks.workunit.client.0.smithi112.stdout:2/490: symlink d2/l98 0 2024-09-16T00:25:58.640 INFO:tasks.workunit.client.1.smithi169.stdout:5/222: dread d2/da/d10/d13/f14 [0,4194304] 0 2024-09-16T00:25:58.640 INFO:tasks.workunit.client.1.smithi169.stdout:9/310: rename d5/d12/f31 to d5/da/d2e/f7b 0 2024-09-16T00:25:58.640 INFO:tasks.workunit.client.0.smithi112.stdout:8/339: write d10/d4e/f64 [87726,93421] 0 2024-09-16T00:25:58.641 INFO:tasks.workunit.client.1.smithi169.stdout:3/309: symlink d7/d1a/l77 0 2024-09-16T00:25:58.641 INFO:tasks.workunit.client.1.smithi169.stdout:3/310: readlink d7/l15 0 2024-09-16T00:25:58.642 INFO:tasks.workunit.client.1.smithi169.stdout:7/358: mkdir d7/df/d73 0 2024-09-16T00:25:58.643 INFO:tasks.workunit.client.1.smithi169.stdout:3/311: fdatasync d7/d1a/d1c/d35/d63/f6a 0 2024-09-16T00:25:58.644 INFO:tasks.workunit.client.1.smithi169.stdout:0/240: mknod d0/d2c/d38/c5b 0 2024-09-16T00:25:58.646 INFO:tasks.workunit.client.0.smithi112.stdout:3/489: mknod d2/d9/d2b/d47/d52/d7a/d80/d83/d68/cab 0 2024-09-16T00:25:58.647 INFO:tasks.workunit.client.1.smithi169.stdout:3/312: truncate d7/d1a/d1c/d23/f3c 5095741 0 2024-09-16T00:25:58.647 INFO:tasks.workunit.client.1.smithi169.stdout:4/266: rename d2/dd/d1a/f1b to d2/d12/d13/d1f/d3e/f53 0 2024-09-16T00:25:58.648 INFO:tasks.workunit.client.0.smithi112.stdout:2/491: fsync d2/dd/f5e 0 2024-09-16T00:25:58.648 INFO:tasks.workunit.client.1.smithi169.stdout:3/313: dread - d7/d50/d6f/f74 zero size 2024-09-16T00:25:58.648 INFO:tasks.workunit.client.1.smithi169.stdout:7/359: read d7/d16/d49/f53 [2853561,84650] 0 2024-09-16T00:25:58.650 INFO:tasks.workunit.client.1.smithi169.stdout:3/314: chown d7/d1a/d2b/f45 54376 1 2024-09-16T00:25:58.650 INFO:tasks.workunit.client.0.smithi112.stdout:4/364: dwrite d6/f1b [4194304,4194304] 0 2024-09-16T00:25:58.651 INFO:tasks.workunit.client.1.smithi169.stdout:6/153: mkdir d2/d22/d32 0 2024-09-16T00:25:58.651 INFO:tasks.workunit.client.1.smithi169.stdout:0/241: mkdir d0/d2c/d54/d56/d5c 0 2024-09-16T00:25:58.652 INFO:tasks.workunit.client.0.smithi112.stdout:3/490: truncate d2/d9/d2b/fa0 748833 0 2024-09-16T00:25:58.653 INFO:tasks.workunit.client.1.smithi169.stdout:6/154: truncate d2/d15/f18 4779910 0 2024-09-16T00:25:58.653 INFO:tasks.workunit.client.0.smithi112.stdout:7/448: stat d6/d10/d75/d88/c48 0 2024-09-16T00:25:58.654 INFO:tasks.workunit.client.0.smithi112.stdout:6/396: link dd/df/d17/c24 dd/df/d17/d27/d84/d37/c86 0 2024-09-16T00:25:58.654 INFO:tasks.workunit.client.1.smithi169.stdout:6/155: stat d2/d22 0 2024-09-16T00:25:58.655 INFO:tasks.workunit.client.0.smithi112.stdout:8/340: symlink d10/d4e/d58/l6d 0 2024-09-16T00:25:58.655 INFO:tasks.workunit.client.0.smithi112.stdout:4/365: write d6/da/f84 [453051,117275] 0 2024-09-16T00:25:58.658 INFO:tasks.workunit.client.0.smithi112.stdout:9/594: dwrite d1/d5/d20/d81/d1a/d88/da8/fc8 [0,4194304] 0 2024-09-16T00:25:58.663 INFO:tasks.workunit.client.1.smithi169.stdout:4/267: dwrite d2/d12/d13/d27/f32 [0,4194304] 0 2024-09-16T00:25:58.664 INFO:tasks.workunit.client.0.smithi112.stdout:7/449: write d6/d10/d16/f34 [1004058,55433] 0 2024-09-16T00:25:58.664 INFO:tasks.workunit.client.0.smithi112.stdout:8/341: stat d10/d45/f52 0 2024-09-16T00:25:58.665 INFO:tasks.workunit.client.1.smithi169.stdout:5/223: link d2/da/d10/c16 d2/da/d10/d13/d1c/d32/c4b 0 2024-09-16T00:25:58.665 INFO:tasks.workunit.client.1.smithi169.stdout:4/268: write d2/d12/d13/f2e [1368548,128648] 0 2024-09-16T00:25:58.665 INFO:tasks.workunit.client.1.smithi169.stdout:9/311: getdents d5/da/d5b/d6f 0 2024-09-16T00:25:58.667 INFO:tasks.workunit.client.1.smithi169.stdout:9/312: truncate d5/da/d2e/d47/f6b 666371 0 2024-09-16T00:25:58.668 INFO:tasks.workunit.client.0.smithi112.stdout:2/492: dwrite d2/dd/d48/d22/f6a [0,4194304] 0 2024-09-16T00:25:58.669 INFO:tasks.workunit.client.0.smithi112.stdout:6/397: dread dd/df/d13/d7c/f58 [0,4194304] 0 2024-09-16T00:25:58.670 INFO:tasks.workunit.client.0.smithi112.stdout:4/366: rename d6/db/l2e to d6/d39/d1c/d6e/l87 0 2024-09-16T00:25:58.676 INFO:tasks.workunit.client.1.smithi169.stdout:5/224: chown d2/l5 524206 1 2024-09-16T00:25:58.676 INFO:tasks.workunit.client.0.smithi112.stdout:6/398: write dd/df/d10/d47/d55/f7d [252612,35027] 0 2024-09-16T00:25:58.676 INFO:tasks.workunit.client.1.smithi169.stdout:9/313: fdatasync d5/da/ff 0 2024-09-16T00:25:58.678 INFO:tasks.workunit.client.0.smithi112.stdout:7/450: chown d6/d10/d1b/c5f 29636764 1 2024-09-16T00:25:58.678 INFO:tasks.workunit.client.1.smithi169.stdout:4/269: mkdir d2/d54 0 2024-09-16T00:25:58.679 INFO:tasks.workunit.client.0.smithi112.stdout:9/595: creat d1/d5/d6/d2d/d39/fd0 x:0 0 0 2024-09-16T00:25:58.681 INFO:tasks.workunit.client.0.smithi112.stdout:8/342: chown d10/d12/d1c/d61 3389 1 2024-09-16T00:25:58.683 INFO:tasks.workunit.client.0.smithi112.stdout:0/433: dwrite d5/da/f6a [0,4194304] 0 2024-09-16T00:25:58.686 INFO:tasks.workunit.client.0.smithi112.stdout:0/434: write d5/da/d66/f69 [1346901,80144] 0 2024-09-16T00:25:58.688 INFO:tasks.workunit.client.1.smithi169.stdout:1/238: dwrite dc/d18/f20 [0,4194304] 0 2024-09-16T00:25:58.690 INFO:tasks.workunit.client.0.smithi112.stdout:2/493: truncate d2/dd/d48/d22/d2b/d54/f55 2664996 0 2024-09-16T00:25:58.690 INFO:tasks.workunit.client.0.smithi112.stdout:0/435: write d5/da/f53 [2564535,26383] 0 2024-09-16T00:25:58.690 INFO:tasks.workunit.client.1.smithi169.stdout:4/270: creat d2/dd/d1a/f55 x:0 0 0 2024-09-16T00:25:58.691 INFO:tasks.workunit.client.1.smithi169.stdout:8/222: dwrite d7/fd [4194304,4194304] 0 2024-09-16T00:25:58.694 INFO:tasks.workunit.client.0.smithi112.stdout:9/596: link d1/d5/d2c/d6f/lab d1/d5/d20/d81/d1a/ld1 0 2024-09-16T00:25:58.694 INFO:tasks.workunit.client.0.smithi112.stdout:6/399: dwrite dd/d15/d34/f71 [0,4194304] 0 2024-09-16T00:25:58.695 INFO:tasks.workunit.client.1.smithi169.stdout:4/271: write d2/d12/f2c [945110,11049] 0 2024-09-16T00:25:58.696 INFO:tasks.workunit.client.1.smithi169.stdout:4/272: write d2/d12/d13/d1f/f35 [1450500,73562] 0 2024-09-16T00:25:58.697 INFO:tasks.workunit.client.1.smithi169.stdout:0/242: truncate d0/d2c/d54/f1c 2604930 0 2024-09-16T00:25:58.697 INFO:tasks.workunit.client.1.smithi169.stdout:6/156: truncate d2/f17 2752587 0 2024-09-16T00:25:58.698 INFO:tasks.workunit.client.0.smithi112.stdout:0/436: creat d5/da/d21/d6f/f85 x:0 0 0 2024-09-16T00:25:58.699 INFO:tasks.workunit.client.1.smithi169.stdout:6/157: fdatasync d2/f30 0 2024-09-16T00:25:58.700 INFO:tasks.workunit.client.0.smithi112.stdout:8/343: getdents d10/d12/d1c/d3b 0 2024-09-16T00:25:58.700 INFO:tasks.workunit.client.1.smithi169.stdout:9/314: rmdir d5/d49/d70 39 2024-09-16T00:25:58.701 INFO:tasks.workunit.client.0.smithi112.stdout:6/400: rmdir dd/df/d10/d47 39 2024-09-16T00:25:58.701 INFO:tasks.workunit.client.0.smithi112.stdout:8/344: readlink la 0 2024-09-16T00:25:58.703 INFO:tasks.workunit.client.1.smithi169.stdout:1/239: rmdir dc/d36/d3a/d45 39 2024-09-16T00:25:58.704 INFO:tasks.workunit.client.0.smithi112.stdout:9/597: mknod d1/d76/cd2 0 2024-09-16T00:25:58.704 INFO:tasks.workunit.client.0.smithi112.stdout:0/437: symlink d5/da/d18/d24/d5a/l86 0 2024-09-16T00:25:58.705 INFO:tasks.workunit.client.0.smithi112.stdout:6/401: mkdir dd/df/d11/d87 0 2024-09-16T00:25:58.706 INFO:tasks.workunit.client.0.smithi112.stdout:9/598: fdatasync d1/d5/d20/d81/d1a/f9c 0 2024-09-16T00:25:58.708 INFO:tasks.workunit.client.0.smithi112.stdout:8/345: dread d10/f1b [0,4194304] 0 2024-09-16T00:25:58.709 INFO:tasks.workunit.client.0.smithi112.stdout:6/402: dread dd/df/d17/f41 [0,4194304] 0 2024-09-16T00:25:58.709 INFO:tasks.workunit.client.1.smithi169.stdout:4/273: mknod d2/d12/d13/d27/c56 0 2024-09-16T00:25:58.710 INFO:tasks.workunit.client.0.smithi112.stdout:6/403: stat dd/df/l39 0 2024-09-16T00:25:58.710 INFO:tasks.workunit.client.0.smithi112.stdout:0/438: link d5/da/d1d/c72 d5/da/d1d/c87 0 2024-09-16T00:25:58.712 INFO:tasks.workunit.client.0.smithi112.stdout:8/346: write d10/d12/d1c/d3b/f4d [763565,955] 0 2024-09-16T00:25:58.713 INFO:tasks.workunit.client.1.smithi169.stdout:4/274: read d2/dd/f24 [114345,24102] 0 2024-09-16T00:25:58.713 INFO:tasks.workunit.client.0.smithi112.stdout:0/439: read - d5/da/d57/f73 zero size 2024-09-16T00:25:58.717 INFO:tasks.workunit.client.1.smithi169.stdout:5/225: link d2/da/d10/d13/d1c/f41 d2/da/d10/d13/d1c/d40/d43/f4c 0 2024-09-16T00:25:58.717 INFO:tasks.workunit.client.1.smithi169.stdout:9/315: creat d5/da/d2e/d47/f7c x:0 0 0 2024-09-16T00:25:58.718 INFO:tasks.workunit.client.0.smithi112.stdout:8/347: creat d10/d12/d24/f6e x:0 0 0 2024-09-16T00:25:58.719 INFO:tasks.workunit.client.0.smithi112.stdout:9/599: dwrite d1/d5/d20/d81/d1a/f7d [0,4194304] 0 2024-09-16T00:25:58.719 INFO:tasks.workunit.client.1.smithi169.stdout:6/158: truncate d2/d15/f1e 2244709 0 2024-09-16T00:25:58.719 INFO:tasks.workunit.client.1.smithi169.stdout:1/240: creat dc/d2c/d3d/f47 x:0 0 0 2024-09-16T00:25:58.719 INFO:tasks.workunit.client.1.smithi169.stdout:8/223: dwrite f6 [4194304,4194304] 0 2024-09-16T00:25:58.721 INFO:tasks.workunit.client.0.smithi112.stdout:6/404: getdents dd/df/d13 0 2024-09-16T00:25:58.721 INFO:tasks.workunit.client.0.smithi112.stdout:0/440: rename d5/da/d57/d76/d47 to d5/da/d18/d24/d46/d88 0 2024-09-16T00:25:58.722 INFO:tasks.workunit.client.0.smithi112.stdout:6/405: chown dd/df/d4b/c5f 0 1 2024-09-16T00:25:58.724 INFO:tasks.workunit.client.0.smithi112.stdout:8/348: dread d10/d12/d1c/d3b/f4d [0,4194304] 0 2024-09-16T00:25:58.725 INFO:tasks.workunit.client.1.smithi169.stdout:8/224: mkdir d7/dc/d17/d48 0 2024-09-16T00:25:58.725 INFO:tasks.workunit.client.0.smithi112.stdout:0/441: mkdir d5/da/d18/d89 0 2024-09-16T00:25:58.726 INFO:tasks.workunit.client.0.smithi112.stdout:6/406: mkdir dd/df/d13/d7c/d88 0 2024-09-16T00:25:58.727 INFO:tasks.workunit.client.0.smithi112.stdout:6/407: dread - dd/df/d17/d27/d54/f6a zero size 2024-09-16T00:25:58.728 INFO:tasks.workunit.client.1.smithi169.stdout:9/316: rename d5/da/d5b/d6f/f78 to d5/da/d2e/d47/f7d 0 2024-09-16T00:25:58.730 INFO:tasks.workunit.client.0.smithi112.stdout:0/442: unlink d5/l35 0 2024-09-16T00:25:58.731 INFO:tasks.workunit.client.1.smithi169.stdout:5/226: link d2/da/d10/d13/f2f d2/da/d37/f4d 0 2024-09-16T00:25:58.733 INFO:tasks.workunit.client.1.smithi169.stdout:8/225: chown d7/dc/c23 34 1 2024-09-16T00:25:58.733 INFO:tasks.workunit.client.0.smithi112.stdout:8/349: getdents d10/d12 0 2024-09-16T00:25:58.733 INFO:tasks.workunit.client.0.smithi112.stdout:9/600: link d1/d5/d2c/d6f/lab d1/d5/d20/d25/d66/ld3 0 2024-09-16T00:25:58.733 INFO:tasks.workunit.client.1.smithi169.stdout:9/317: stat d5/da/d2e/l38 0 2024-09-16T00:25:58.734 INFO:tasks.workunit.client.0.smithi112.stdout:0/443: creat d5/da/d57/f8a x:0 0 0 2024-09-16T00:25:58.734 INFO:tasks.workunit.client.1.smithi169.stdout:4/275: dwrite d2/dd/f24 [0,4194304] 0 2024-09-16T00:25:58.735 INFO:tasks.workunit.client.0.smithi112.stdout:6/408: unlink dd/df/d10/l16 0 2024-09-16T00:25:58.736 INFO:tasks.workunit.client.0.smithi112.stdout:9/601: creat d1/d5/d2c/fd4 x:0 0 0 2024-09-16T00:25:58.736 INFO:tasks.workunit.client.1.smithi169.stdout:6/159: creat d2/d15/d1d/f33 x:0 0 0 2024-09-16T00:25:58.736 INFO:tasks.workunit.client.1.smithi169.stdout:9/318: write d5/da/d21/f44 [146385,98261] 0 2024-09-16T00:25:58.737 INFO:tasks.workunit.client.0.smithi112.stdout:0/444: creat d5/da/d1d/d71/d64/f8b x:0 0 0 2024-09-16T00:25:58.737 INFO:tasks.workunit.client.1.smithi169.stdout:5/227: mkdir d2/da/d37/d4e 0 2024-09-16T00:25:58.738 INFO:tasks.workunit.client.0.smithi112.stdout:4/367: truncate d6/f1b 4266471 0 2024-09-16T00:25:58.740 INFO:tasks.workunit.client.0.smithi112.stdout:4/368: dread - d6/db/df/f5f zero size 2024-09-16T00:25:58.740 INFO:tasks.workunit.client.1.smithi169.stdout:4/276: chown d2/d12/l14 0 1 2024-09-16T00:25:58.740 INFO:tasks.workunit.client.1.smithi169.stdout:1/241: link dc/d36/d3a/d45/d29/d2a/f44 dc/d36/d3a/d45/f48 0 2024-09-16T00:25:58.740 INFO:tasks.workunit.client.0.smithi112.stdout:9/602: write d1/d5/d20/d25/f8a [544760,74580] 0 2024-09-16T00:25:58.741 INFO:tasks.workunit.client.0.smithi112.stdout:8/350: creat d10/d4e/f6f x:0 0 0 2024-09-16T00:25:58.741 INFO:tasks.workunit.client.1.smithi169.stdout:8/226: mknod d7/dc/d26/c49 0 2024-09-16T00:25:58.742 INFO:tasks.workunit.client.0.smithi112.stdout:6/409: mkdir dd/df/d10/d47/d89 0 2024-09-16T00:25:58.742 INFO:tasks.workunit.client.0.smithi112.stdout:8/351: write d10/d4e/f6f [854465,64461] 0 2024-09-16T00:25:58.744 INFO:tasks.workunit.client.1.smithi169.stdout:6/160: creat d2/d22/f34 x:0 0 0 2024-09-16T00:25:58.744 INFO:tasks.workunit.client.0.smithi112.stdout:6/410: chown dd/f7e 6400265 1 2024-09-16T00:25:58.745 INFO:tasks.workunit.client.0.smithi112.stdout:4/369: creat d6/db/f88 x:0 0 0 2024-09-16T00:25:58.745 INFO:tasks.workunit.client.1.smithi169.stdout:5/228: creat d2/da/d10/d13/d1c/d2d/f4f x:0 0 0 2024-09-16T00:25:58.745 INFO:tasks.workunit.client.1.smithi169.stdout:9/319: truncate d5/f35 800360 0 2024-09-16T00:25:58.746 INFO:tasks.workunit.client.0.smithi112.stdout:9/603: write d1/d5/d20/d81/d1a/d88/da8/fb2 [160270,111956] 0 2024-09-16T00:25:58.747 INFO:tasks.workunit.client.1.smithi169.stdout:0/243: dread d0/d2c/d54/f2b [0,4194304] 0 2024-09-16T00:25:58.747 INFO:tasks.workunit.client.1.smithi169.stdout:8/227: rename d7/dc/d17/l35 to d7/d37/l4a 0 2024-09-16T00:25:58.749 INFO:tasks.workunit.client.1.smithi169.stdout:6/161: stat d2/c24 0 2024-09-16T00:25:58.751 INFO:tasks.workunit.client.0.smithi112.stdout:6/411: getdents dd 0 2024-09-16T00:25:58.751 INFO:tasks.workunit.client.1.smithi169.stdout:8/228: readlink d7/dc/d17/l19 0 2024-09-16T00:25:58.751 INFO:tasks.workunit.client.1.smithi169.stdout:1/242: creat dc/d36/d3a/d45/d29/d42/f49 x:0 0 0 2024-09-16T00:25:58.753 INFO:tasks.workunit.client.0.smithi112.stdout:0/445: dwrite d5/da/d21/d6f/f84 [0,4194304] 0 2024-09-16T00:25:58.753 INFO:tasks.workunit.client.1.smithi169.stdout:8/229: read - d7/f2e zero size 2024-09-16T00:25:58.754 INFO:tasks.workunit.client.0.smithi112.stdout:2/494: write d2/f46 [141196,24153] 0 2024-09-16T00:25:58.755 INFO:tasks.workunit.client.1.smithi169.stdout:0/244: creat d0/d2c/f5d x:0 0 0 2024-09-16T00:25:58.756 INFO:tasks.workunit.client.0.smithi112.stdout:4/370: rename d6/db/d1f/l83 to d6/d39/d1c/d66/l89 0 2024-09-16T00:25:58.756 INFO:tasks.workunit.client.1.smithi169.stdout:9/320: rename d5/da/d21/f3e to d5/da/d4e/f7e 0 2024-09-16T00:25:58.757 INFO:tasks.workunit.client.1.smithi169.stdout:1/243: truncate dc/d18/f2b 977979 0 2024-09-16T00:25:58.759 INFO:tasks.workunit.client.0.smithi112.stdout:4/371: truncate d6/d39/d36/d5c/d7e/f86 325027 0 2024-09-16T00:25:58.759 INFO:tasks.workunit.client.0.smithi112.stdout:6/412: mkdir dd/df/d11/d44/d8a 0 2024-09-16T00:25:58.760 INFO:tasks.workunit.client.0.smithi112.stdout:0/446: creat d5/da/d18/d24/d46/d88/f8c x:0 0 0 2024-09-16T00:25:58.761 INFO:tasks.workunit.client.0.smithi112.stdout:4/372: dread - d6/d39/d36/d5c/f77 zero size 2024-09-16T00:25:58.761 INFO:tasks.workunit.client.0.smithi112.stdout:0/447: chown d5/da/d1d 28 1 2024-09-16T00:25:58.763 INFO:tasks.workunit.client.0.smithi112.stdout:2/495: write d2/dd/d48/d22/f8f [1033274,120365] 0 2024-09-16T00:25:58.763 INFO:tasks.workunit.client.0.smithi112.stdout:9/604: link d1/d5/d20/d25/f1c d1/d5/d20/d81/fd5 0 2024-09-16T00:25:58.766 INFO:tasks.workunit.client.0.smithi112.stdout:9/605: readlink d1/d5/d6/d2d/lc1 0 2024-09-16T00:25:58.767 INFO:tasks.workunit.client.0.smithi112.stdout:2/496: symlink d2/dd/d48/l99 0 2024-09-16T00:25:58.768 INFO:tasks.workunit.client.0.smithi112.stdout:0/448: mkdir d5/da/d18/d24/d46/d5f/d7a/d8d 0 2024-09-16T00:25:58.771 INFO:tasks.workunit.client.0.smithi112.stdout:6/413: dwrite dd/d23/f64 [0,4194304] 0 2024-09-16T00:25:58.771 INFO:tasks.workunit.client.0.smithi112.stdout:2/497: write d2/dd/f1a [3860697,74131] 0 2024-09-16T00:25:58.772 INFO:tasks.workunit.client.1.smithi169.stdout:4/277: dwrite d2/d12/d13/d1f/d2f/f45 [0,4194304] 0 2024-09-16T00:25:58.772 INFO:tasks.workunit.client.1.smithi169.stdout:9/321: fdatasync d5/da/d21/d23/f67 0 2024-09-16T00:25:58.773 INFO:tasks.workunit.client.0.smithi112.stdout:4/373: dwrite d6/db/d1f/d41/d49/f4c [0,4194304] 0 2024-09-16T00:25:58.775 INFO:tasks.workunit.client.1.smithi169.stdout:6/162: creat d2/d22/d32/f35 x:0 0 0 2024-09-16T00:25:58.775 INFO:tasks.workunit.client.0.smithi112.stdout:0/449: mknod d5/da/d57/d76/d74/d80/c8e 0 2024-09-16T00:25:58.776 INFO:tasks.workunit.client.1.smithi169.stdout:5/229: dwrite d2/ff [4194304,4194304] 0 2024-09-16T00:25:58.777 INFO:tasks.workunit.client.0.smithi112.stdout:2/498: mkdir d2/dd/d48/d22/d67/d9a 0 2024-09-16T00:25:58.777 INFO:tasks.workunit.client.1.smithi169.stdout:4/278: stat d2/d12/d13/d1f/d3e/f53 0 2024-09-16T00:25:58.779 INFO:tasks.workunit.client.1.smithi169.stdout:0/245: creat d0/d2c/d38/f5e x:0 0 0 2024-09-16T00:25:58.782 INFO:tasks.workunit.client.0.smithi112.stdout:0/450: mkdir d5/da/d1d/d71/d64/d8f 0 2024-09-16T00:25:58.785 INFO:tasks.workunit.client.0.smithi112.stdout:4/374: symlink d6/db/df/d5e/d61/d64/d6a/l8a 0 2024-09-16T00:25:58.785 INFO:tasks.workunit.client.1.smithi169.stdout:4/279: readlink d2/d12/l44 0 2024-09-16T00:25:58.787 INFO:tasks.workunit.client.0.smithi112.stdout:0/451: chown d5/da/f2e 1246920 1 2024-09-16T00:25:58.787 INFO:tasks.workunit.client.1.smithi169.stdout:9/322: dread d5/d12/f26 [0,4194304] 0 2024-09-16T00:25:58.788 INFO:tasks.workunit.client.1.smithi169.stdout:1/244: truncate dc/d36/d3a/d45/d29/d2a/f38 567242 0 2024-09-16T00:25:58.791 INFO:tasks.workunit.client.0.smithi112.stdout:0/452: write d5/da/d57/d76/f17 [3505057,21868] 0 2024-09-16T00:25:58.791 INFO:tasks.workunit.client.1.smithi169.stdout:5/230: creat d2/da/f50 x:0 0 0 2024-09-16T00:25:58.795 INFO:tasks.workunit.client.0.smithi112.stdout:0/453: mkdir d5/d90 0 2024-09-16T00:25:58.801 INFO:tasks.workunit.client.1.smithi169.stdout:6/163: rename d2/d15/d1d/d25 to d2/d36 0 2024-09-16T00:25:58.804 INFO:tasks.workunit.client.0.smithi112.stdout:4/375: dread f4 [0,4194304] 0 2024-09-16T00:25:58.804 INFO:tasks.workunit.client.0.smithi112.stdout:0/454: dwrite d5/da/d18/d24/d46/d88/f4a [0,4194304] 0 2024-09-16T00:25:58.804 INFO:tasks.workunit.client.1.smithi169.stdout:1/245: symlink dc/d36/d3a/d45/d29/d42/l4a 0 2024-09-16T00:25:58.806 INFO:tasks.workunit.client.0.smithi112.stdout:4/376: write d6/da/f40 [2996175,112329] 0 2024-09-16T00:25:58.811 INFO:tasks.workunit.client.1.smithi169.stdout:0/246: creat d0/d2/f5f x:0 0 0 2024-09-16T00:25:58.812 INFO:tasks.workunit.client.0.smithi112.stdout:0/455: rename d5/da/d57/d76/d74/d80/c8e to d5/c91 0 2024-09-16T00:25:58.813 INFO:tasks.workunit.client.1.smithi169.stdout:6/164: rmdir d2/d36/d27 0 2024-09-16T00:25:58.814 INFO:tasks.workunit.client.1.smithi169.stdout:9/323: dread d5/da/fc [0,4194304] 0 2024-09-16T00:25:58.814 INFO:tasks.workunit.client.1.smithi169.stdout:1/246: dwrite dc/d26/f37 [0,4194304] 0 2024-09-16T00:25:58.815 INFO:tasks.workunit.client.1.smithi169.stdout:4/280: getdents d2/dd 0 2024-09-16T00:25:58.816 INFO:tasks.workunit.client.1.smithi169.stdout:5/231: getdents d2/da/d10/d13/d1c 0 2024-09-16T00:25:58.816 INFO:tasks.workunit.client.0.smithi112.stdout:0/456: mknod d5/da/d18/d24/d46/d5f/d7a/d8d/c92 0 2024-09-16T00:25:58.817 INFO:tasks.workunit.client.1.smithi169.stdout:9/324: write d5/f35 [1448641,7223] 0 2024-09-16T00:25:58.822 INFO:tasks.workunit.client.1.smithi169.stdout:1/247: write dc/f21 [743620,15555] 0 2024-09-16T00:25:58.823 INFO:tasks.workunit.client.0.smithi112.stdout:9/606: write d1/d5/d20/d25/d66/f9d [4286036,3243] 0 2024-09-16T00:25:58.824 INFO:tasks.workunit.client.0.smithi112.stdout:9/607: fdatasync d1/d5/fa6 0 2024-09-16T00:25:58.825 INFO:tasks.workunit.client.0.smithi112.stdout:6/414: truncate dd/df/d10/d47/d55/f7d 167157 0 2024-09-16T00:25:58.828 INFO:tasks.workunit.client.0.smithi112.stdout:9/608: creat d1/d5/d20/d25/fd6 x:0 0 0 2024-09-16T00:25:58.829 INFO:tasks.workunit.client.1.smithi169.stdout:0/247: rename d0/d2c/d54/d56/d5c to d0/d2/d8/d5/d60 0 2024-09-16T00:25:58.829 INFO:tasks.workunit.client.0.smithi112.stdout:8/352: dread d10/d45/f57 [0,4194304] 0 2024-09-16T00:25:58.829 INFO:tasks.workunit.client.1.smithi169.stdout:5/232: rename d2/da/d10/d13 to d2/da/d10/d13/d1c/d51 22 2024-09-16T00:25:58.829 INFO:tasks.workunit.client.0.smithi112.stdout:9/609: chown d1/f41 59 1 2024-09-16T00:25:58.830 INFO:tasks.workunit.client.1.smithi169.stdout:5/233: chown d2/da/d10/f11 40 1 2024-09-16T00:25:58.830 INFO:tasks.workunit.client.0.smithi112.stdout:6/415: creat dd/df/d10/f8b x:0 0 0 2024-09-16T00:25:58.831 INFO:tasks.workunit.client.0.smithi112.stdout:9/610: chown d1/d5/d20/d25/l6b 22 1 2024-09-16T00:25:58.831 INFO:tasks.workunit.client.1.smithi169.stdout:0/248: write d0/d2/d8/f47 [281996,14247] 0 2024-09-16T00:25:58.833 INFO:tasks.workunit.client.1.smithi169.stdout:6/165: link d2/c11 d2/d15/c37 0 2024-09-16T00:25:58.833 INFO:tasks.workunit.client.1.smithi169.stdout:0/249: truncate d0/d2c/f5d 383488 0 2024-09-16T00:25:58.834 INFO:tasks.workunit.client.1.smithi169.stdout:9/325: creat d5/da/f7f x:0 0 0 2024-09-16T00:25:58.834 INFO:tasks.workunit.client.0.smithi112.stdout:6/416: creat dd/df/d4b/f8c x:0 0 0 2024-09-16T00:25:58.835 INFO:tasks.workunit.client.1.smithi169.stdout:6/166: write d2/d22/f34 [596773,30868] 0 2024-09-16T00:25:58.836 INFO:tasks.workunit.client.0.smithi112.stdout:8/353: link d10/d4e/d58/f66 d10/d12/d1c/d61/d6c/f70 0 2024-09-16T00:25:58.838 INFO:tasks.workunit.client.1.smithi169.stdout:9/326: dread d5/da/d2e/d47/f6b [0,4194304] 0 2024-09-16T00:25:58.839 INFO:tasks.workunit.client.0.smithi112.stdout:6/417: creat dd/df/d17/d27/d84/d37/d79/f8d x:0 0 0 2024-09-16T00:25:58.840 INFO:tasks.workunit.client.0.smithi112.stdout:9/611: dwrite d1/d5/d83/fca [0,4194304] 0 2024-09-16T00:25:58.840 INFO:tasks.workunit.client.1.smithi169.stdout:0/250: creat d0/d2c/d54/f61 x:0 0 0 2024-09-16T00:25:58.841 INFO:tasks.workunit.client.1.smithi169.stdout:6/167: mkdir d2/d22/d32/d38 0 2024-09-16T00:25:58.842 INFO:tasks.workunit.client.0.smithi112.stdout:8/354: mkdir d10/d12/d1c/d61/d6c/d71 0 2024-09-16T00:25:58.843 INFO:tasks.workunit.client.1.smithi169.stdout:9/327: unlink d5/da/f34 0 2024-09-16T00:25:58.844 INFO:tasks.workunit.client.1.smithi169.stdout:9/328: stat d5/d12/f26 0 2024-09-16T00:25:58.844 INFO:tasks.workunit.client.1.smithi169.stdout:8/230: truncate d7/dc/d17/f28 2514236 0 2024-09-16T00:25:58.847 INFO:tasks.workunit.client.0.smithi112.stdout:6/418: fsync dd/df/d10/f1a 0 2024-09-16T00:25:58.849 INFO:tasks.workunit.client.0.smithi112.stdout:9/612: dread d1/d5/d20/d81/d1a/d88/da8/fc8 [0,4194304] 0 2024-09-16T00:25:58.849 INFO:tasks.workunit.client.0.smithi112.stdout:6/419: chown dd/d23/c6b 5816 1 2024-09-16T00:25:58.850 INFO:tasks.workunit.client.0.smithi112.stdout:0/457: dwrite d5/da/d1d/d71/f4d [0,4194304] 0 2024-09-16T00:25:58.851 INFO:tasks.workunit.client.0.smithi112.stdout:8/355: creat d10/d4e/f72 x:0 0 0 2024-09-16T00:25:58.853 INFO:tasks.workunit.client.0.smithi112.stdout:9/613: creat d1/d5/d2c/fd7 x:0 0 0 2024-09-16T00:25:58.853 INFO:tasks.workunit.client.1.smithi169.stdout:6/168: creat d2/d2e/f39 x:0 0 0 2024-09-16T00:25:58.854 INFO:tasks.workunit.client.0.smithi112.stdout:6/420: symlink dd/df/d13/l8e 0 2024-09-16T00:25:58.854 INFO:tasks.workunit.client.0.smithi112.stdout:8/356: truncate d10/d12/d24/f2b 907154 0 2024-09-16T00:25:58.855 INFO:tasks.workunit.client.1.smithi169.stdout:0/251: dwrite d0/d2/f6 [4194304,4194304] 0 2024-09-16T00:25:58.855 INFO:tasks.workunit.client.0.smithi112.stdout:6/421: dread - dd/df/d11/f80 zero size 2024-09-16T00:25:58.857 INFO:tasks.workunit.client.0.smithi112.stdout:8/357: creat d10/d4e/d58/f73 x:0 0 0 2024-09-16T00:25:58.859 INFO:tasks.workunit.client.1.smithi169.stdout:9/329: rename d5 to d5/da/d80 22 2024-09-16T00:25:58.862 INFO:tasks.workunit.client.0.smithi112.stdout:8/358: mkdir d10/d12/d1c/d3b/d74 0 2024-09-16T00:25:58.862 INFO:tasks.workunit.client.1.smithi169.stdout:9/330: chown d5/d12/d61/c72 60 1 2024-09-16T00:25:58.863 INFO:tasks.workunit.client.0.smithi112.stdout:9/614: link d1/d5/d6/d2d/c9b d1/d5/d6/d59/d64/cd8 0 2024-09-16T00:25:58.864 INFO:tasks.workunit.client.1.smithi169.stdout:4/281: truncate d2/d12/d13/d27/f32 3710842 0 2024-09-16T00:25:58.865 INFO:tasks.workunit.client.1.smithi169.stdout:9/331: stat d5/da/d73/l74 0 2024-09-16T00:25:58.868 INFO:tasks.workunit.client.1.smithi169.stdout:6/169: rename d2/l6 to d2/d36/l3a 0 2024-09-16T00:25:58.868 INFO:tasks.workunit.client.0.smithi112.stdout:8/359: fdatasync d10/d12/f1a 0 2024-09-16T00:25:58.868 INFO:tasks.workunit.client.1.smithi169.stdout:0/252: write d0/d2c/d54/f2b [4074717,75637] 0 2024-09-16T00:25:58.869 INFO:tasks.workunit.client.0.smithi112.stdout:6/422: dwrite dd/df/f4c [0,4194304] 0 2024-09-16T00:25:58.874 INFO:tasks.workunit.client.1.smithi169.stdout:5/234: dread d2/f4 [0,4194304] 0 2024-09-16T00:25:58.875 INFO:tasks.workunit.client.1.smithi169.stdout:1/248: dwrite dc/d36/d3a/d45/f48 [0,4194304] 0 2024-09-16T00:25:58.876 INFO:tasks.workunit.client.0.smithi112.stdout:9/615: dwrite d1/d5/f16 [0,4194304] 0 2024-09-16T00:25:58.876 INFO:tasks.workunit.client.0.smithi112.stdout:8/360: rename d10/d45 to d10/d12/d1c/d75 0 2024-09-16T00:25:58.881 INFO:tasks.workunit.client.1.smithi169.stdout:4/282: dwrite d2/fb [0,4194304] 0 2024-09-16T00:25:58.884 INFO:tasks.workunit.client.0.smithi112.stdout:6/423: rename dd/df/d4b to dd/d23/d8f 0 2024-09-16T00:25:58.884 INFO:tasks.workunit.client.0.smithi112.stdout:9/616: truncate d1/d5/d20/f36 1577532 0 2024-09-16T00:25:58.884 INFO:tasks.workunit.client.0.smithi112.stdout:6/424: stat dd/df/d11/f49 0 2024-09-16T00:25:58.885 INFO:tasks.workunit.client.0.smithi112.stdout:9/617: write d1/d5/d20/fae [4678228,41397] 0 2024-09-16T00:25:58.886 INFO:tasks.workunit.client.0.smithi112.stdout:6/425: rmdir dd/df/d11 39 2024-09-16T00:25:58.886 INFO:tasks.workunit.client.0.smithi112.stdout:9/618: chown d1/d5/d6/d18/ca0 7 1 2024-09-16T00:25:58.890 INFO:tasks.workunit.client.0.smithi112.stdout:9/619: creat d1/d5/d6/d59/fd9 x:0 0 0 2024-09-16T00:25:58.891 INFO:tasks.workunit.client.0.smithi112.stdout:9/620: readlink d1/d5/d20/d25/l6b 0 2024-09-16T00:25:58.892 INFO:tasks.workunit.client.1.smithi169.stdout:8/231: dwrite d7/fa [0,4194304] 0 2024-09-16T00:25:58.893 INFO:tasks.workunit.client.1.smithi169.stdout:8/232: write d7/fd [4777377,26957] 0 2024-09-16T00:25:58.893 INFO:tasks.workunit.client.0.smithi112.stdout:9/621: write d1/d5/d20/d81/d1a/f9c [951394,3418] 0 2024-09-16T00:25:58.894 INFO:tasks.workunit.client.1.smithi169.stdout:0/253: mknod d0/d2c/c62 0 2024-09-16T00:25:58.904 INFO:tasks.workunit.client.1.smithi169.stdout:5/235: mknod d2/da/d10/d13/d24/c52 0 2024-09-16T00:25:58.905 INFO:tasks.workunit.client.1.smithi169.stdout:5/236: chown d2/da/d10/d13/d1c/f3b 531 1 2024-09-16T00:25:58.908 INFO:tasks.workunit.client.0.smithi112.stdout:9/622: dwrite d1/d5/d6/d2d/d6a/d89/f90 [0,4194304] 0 2024-09-16T00:25:58.908 INFO:tasks.workunit.client.0.smithi112.stdout:0/458: dread d5/da/d18/d24/d46/d88/f7e [0,4194304] 0 2024-09-16T00:25:58.908 INFO:tasks.workunit.client.0.smithi112.stdout:6/426: truncate dd/df/d17/d27/d54/f56 3171004 0 2024-09-16T00:25:58.909 INFO:tasks.workunit.client.1.smithi169.stdout:0/254: mknod d0/d2/d8/d5/d60/c63 0 2024-09-16T00:25:58.909 INFO:tasks.workunit.client.1.smithi169.stdout:1/249: symlink dc/d36/d3a/d45/l4b 0 2024-09-16T00:25:58.909 INFO:tasks.workunit.client.1.smithi169.stdout:0/255: dread - d0/d2c/d54/d56/f59 zero size 2024-09-16T00:25:58.910 INFO:tasks.workunit.client.1.smithi169.stdout:1/250: readlink dc/l14 0 2024-09-16T00:25:58.912 INFO:tasks.workunit.client.1.smithi169.stdout:4/283: link d2/d12/f39 d2/d12/f57 0 2024-09-16T00:25:58.913 INFO:tasks.workunit.client.1.smithi169.stdout:9/332: getdents d5/da/d73 0 2024-09-16T00:25:58.914 INFO:tasks.workunit.client.1.smithi169.stdout:9/333: stat d5/d49/d64 0 2024-09-16T00:25:58.915 INFO:tasks.workunit.client.1.smithi169.stdout:0/256: symlink d0/d2/d15/l64 0 2024-09-16T00:25:58.916 INFO:tasks.workunit.client.1.smithi169.stdout:9/334: readlink d5/da/d21/l76 0 2024-09-16T00:25:58.918 INFO:tasks.workunit.client.0.smithi112.stdout:8/361: dread d10/d12/d1c/f38 [0,4194304] 0 2024-09-16T00:25:58.918 INFO:tasks.workunit.client.0.smithi112.stdout:0/459: dwrite d5/da/d21/d6f/f84 [0,4194304] 0 2024-09-16T00:25:58.919 INFO:tasks.workunit.client.1.smithi169.stdout:4/284: rename d2/d12/f39 to d2/d12/d13/d1f/d3e/f58 0 2024-09-16T00:25:58.920 INFO:tasks.workunit.client.0.smithi112.stdout:9/623: getdents d1/d5/d20/d81/d1a/d88 0 2024-09-16T00:25:58.921 INFO:tasks.workunit.client.0.smithi112.stdout:8/362: creat d10/d12/d1c/d61/d6c/f76 x:0 0 0 2024-09-16T00:25:58.922 INFO:tasks.workunit.client.0.smithi112.stdout:6/427: rename dd/df/d17/c65 to dd/df/d13/c90 0 2024-09-16T00:25:58.922 INFO:tasks.workunit.client.1.smithi169.stdout:9/335: creat d5/da/d4e/f81 x:0 0 0 2024-09-16T00:25:58.924 INFO:tasks.workunit.client.1.smithi169.stdout:0/257: creat d0/d2/d8/f65 x:0 0 0 2024-09-16T00:25:58.925 INFO:tasks.workunit.client.1.smithi169.stdout:9/336: write d5/da/d21/f44 [380021,39111] 0 2024-09-16T00:25:58.929 INFO:tasks.workunit.client.1.smithi169.stdout:4/285: dwrite d2/d12/f31 [0,4194304] 0 2024-09-16T00:25:58.932 INFO:tasks.workunit.client.1.smithi169.stdout:9/337: write d5/d12/f26 [4861667,71204] 0 2024-09-16T00:25:58.935 INFO:tasks.workunit.client.1.smithi169.stdout:8/233: write d7/dc/d1d/f47 [4922271,18742] 0 2024-09-16T00:25:58.936 INFO:tasks.workunit.client.1.smithi169.stdout:4/286: write d2/d12/d13/d1f/f35 [2922515,4969] 0 2024-09-16T00:25:58.937 INFO:tasks.workunit.client.1.smithi169.stdout:0/258: link d0/d2c/d54/d21/d31/f58 d0/d2c/d38/f66 0 2024-09-16T00:25:58.938 INFO:tasks.workunit.client.1.smithi169.stdout:4/287: fsync d2/d12/f2c 0 2024-09-16T00:25:58.939 INFO:tasks.workunit.client.1.smithi169.stdout:6/170: dwrite d2/d36/f31 [0,4194304] 0 2024-09-16T00:25:58.941 INFO:tasks.workunit.client.1.smithi169.stdout:0/259: chown d0/d2/c1f 4 1 2024-09-16T00:25:58.944 INFO:tasks.workunit.client.1.smithi169.stdout:6/171: rename d2/d22/c2d to d2/d22/d32/d38/c3b 0 2024-09-16T00:25:58.947 INFO:tasks.workunit.client.1.smithi169.stdout:8/234: dwrite d7/dc/fe [4194304,4194304] 0 2024-09-16T00:25:58.949 INFO:tasks.workunit.client.1.smithi169.stdout:6/172: readlink d2/d36/l3a 0 2024-09-16T00:25:58.949 INFO:tasks.workunit.client.1.smithi169.stdout:4/288: getdents d2/d12/d13/d1f 0 2024-09-16T00:25:58.951 INFO:tasks.workunit.client.1.smithi169.stdout:9/338: dread d5/d49/f5a [0,4194304] 0 2024-09-16T00:25:58.953 INFO:tasks.workunit.client.1.smithi169.stdout:4/289: write d2/f9 [5057365,119595] 0 2024-09-16T00:25:58.953 INFO:tasks.workunit.client.1.smithi169.stdout:5/237: dread d2/fb [8388608,4194304] 0 2024-09-16T00:25:58.954 INFO:tasks.workunit.client.1.smithi169.stdout:6/173: mknod d2/d36/c3c 0 2024-09-16T00:25:58.957 INFO:tasks.workunit.client.1.smithi169.stdout:9/339: symlink d5/da/d4e/l82 0 2024-09-16T00:25:58.958 INFO:tasks.workunit.client.1.smithi169.stdout:8/235: rename f2 to d7/dc/d17/d24/f4b 0 2024-09-16T00:25:58.959 INFO:tasks.workunit.client.1.smithi169.stdout:8/236: chown d7/dc/d17/d24/d32 61927739 1 2024-09-16T00:25:58.960 INFO:tasks.workunit.client.1.smithi169.stdout:9/340: mknod d5/da/d21/d23/c83 0 2024-09-16T00:25:58.961 INFO:tasks.workunit.client.1.smithi169.stdout:9/341: chown d5/d12/l1d 1 1 2024-09-16T00:25:58.962 INFO:tasks.workunit.client.1.smithi169.stdout:5/238: rename d2/da/d37 to d2/da/d10/d13/d1c/d32/d53 0 2024-09-16T00:25:58.964 INFO:tasks.workunit.client.1.smithi169.stdout:8/237: creat d7/d20/f4c x:0 0 0 2024-09-16T00:25:58.965 INFO:tasks.workunit.client.1.smithi169.stdout:9/342: rmdir d5/da/d2e 39 2024-09-16T00:25:58.967 INFO:tasks.workunit.client.1.smithi169.stdout:4/290: dread d2/dd/f26 [0,4194304] 0 2024-09-16T00:25:58.969 INFO:tasks.workunit.client.1.smithi169.stdout:4/291: chown d2/d12/d13/l2a 0 1 2024-09-16T00:25:58.969 INFO:tasks.workunit.client.1.smithi169.stdout:8/238: rmdir d7/dc/d1d 39 2024-09-16T00:25:58.970 INFO:tasks.workunit.client.1.smithi169.stdout:0/260: write d0/d2c/d54/d16/f5a [565981,20478] 0 2024-09-16T00:25:58.970 INFO:tasks.workunit.client.1.smithi169.stdout:9/343: fdatasync d5/da/d73/f75 0 2024-09-16T00:25:58.972 INFO:tasks.workunit.client.1.smithi169.stdout:6/174: link d2/l13 d2/d15/d1d/l3d 0 2024-09-16T00:25:58.972 INFO:tasks.workunit.client.1.smithi169.stdout:9/344: dread - d5/da/d5b/f63 zero size 2024-09-16T00:25:58.973 INFO:tasks.workunit.client.1.smithi169.stdout:4/292: chown d2/d12/d13/d27/f32 29826487 1 2024-09-16T00:25:58.974 INFO:tasks.workunit.client.1.smithi169.stdout:8/239: mkdir d7/dc/d17/d24/d32/d4d 0 2024-09-16T00:25:58.976 INFO:tasks.workunit.client.1.smithi169.stdout:0/261: mkdir d0/d2c/d54/d56/d67 0 2024-09-16T00:25:58.978 INFO:tasks.workunit.client.1.smithi169.stdout:9/345: mknod d5/da/d5b/d6f/c84 0 2024-09-16T00:25:58.979 INFO:tasks.workunit.client.1.smithi169.stdout:0/262: read d0/d2c/d54/d21/d31/f58 [85847,85555] 0 2024-09-16T00:25:58.981 INFO:tasks.workunit.client.1.smithi169.stdout:4/293: mkdir d2/d12/d13/d4d/d59 0 2024-09-16T00:25:58.982 INFO:tasks.workunit.client.1.smithi169.stdout:0/263: stat d0/d2c/d54/f28 0 2024-09-16T00:25:58.982 INFO:tasks.workunit.client.1.smithi169.stdout:8/240: mkdir d7/dc/d26/d41/d4e 0 2024-09-16T00:25:58.983 INFO:tasks.workunit.client.1.smithi169.stdout:4/294: fsync d2/f49 0 2024-09-16T00:25:58.985 INFO:tasks.workunit.client.1.smithi169.stdout:9/346: fdatasync d5/da/d2e/f43 0 2024-09-16T00:25:58.985 INFO:tasks.workunit.client.1.smithi169.stdout:5/239: dwrite d2/f29 [0,4194304] 0 2024-09-16T00:25:58.988 INFO:tasks.workunit.client.1.smithi169.stdout:6/175: dwrite d2/fb [0,4194304] 0 2024-09-16T00:25:58.990 INFO:tasks.workunit.client.1.smithi169.stdout:6/176: write d2/d2e/f39 [430056,54819] 0 2024-09-16T00:25:58.990 INFO:tasks.workunit.client.1.smithi169.stdout:8/241: rmdir d7/dc/d17/d24/d32 39 2024-09-16T00:25:58.991 INFO:tasks.workunit.client.1.smithi169.stdout:8/242: chown d7/d20/l43 2002393 1 2024-09-16T00:25:58.991 INFO:tasks.workunit.client.1.smithi169.stdout:4/295: rename d2/dd/c17 to d2/d54/c5a 0 2024-09-16T00:25:58.991 INFO:tasks.workunit.client.1.smithi169.stdout:6/177: dread - d2/d15/d1d/f33 zero size 2024-09-16T00:25:58.993 INFO:tasks.workunit.client.1.smithi169.stdout:9/347: write d5/da/d21/d23/f3b [1834261,48628] 0 2024-09-16T00:25:58.995 INFO:tasks.workunit.client.1.smithi169.stdout:8/243: write d7/dc/d17/f39 [3238804,85717] 0 2024-09-16T00:25:58.996 INFO:tasks.workunit.client.1.smithi169.stdout:6/178: mknod d2/d22/d32/c3e 0 2024-09-16T00:25:58.998 INFO:tasks.workunit.client.1.smithi169.stdout:6/179: write d2/d22/d32/f35 [691266,10154] 0 2024-09-16T00:25:58.999 INFO:tasks.workunit.client.1.smithi169.stdout:9/348: mknod d5/da/d5b/c85 0 2024-09-16T00:25:59.000 INFO:tasks.workunit.client.1.smithi169.stdout:9/349: write d5/d49/f59 [3637401,362] 0 2024-09-16T00:25:59.001 INFO:tasks.workunit.client.1.smithi169.stdout:8/244: mknod d7/dc/d26/c4f 0 2024-09-16T00:25:59.002 INFO:tasks.workunit.client.1.smithi169.stdout:6/180: mkdir d2/d3f 0 2024-09-16T00:25:59.004 INFO:tasks.workunit.client.1.smithi169.stdout:9/350: readlink d5/l3a 0 2024-09-16T00:25:59.006 INFO:tasks.workunit.client.1.smithi169.stdout:6/181: write d2/d15/f1e [4003741,92609] 0 2024-09-16T00:25:59.007 INFO:tasks.workunit.client.1.smithi169.stdout:4/296: rename d2/dd to d2/d12/d13/d1f/d3e/d5b 0 2024-09-16T00:25:59.009 INFO:tasks.workunit.client.1.smithi169.stdout:6/182: write d2/f30 [59283,111557] 0 2024-09-16T00:25:59.010 INFO:tasks.workunit.client.1.smithi169.stdout:9/351: rmdir d5/da/d21 39 2024-09-16T00:25:59.014 INFO:tasks.workunit.client.1.smithi169.stdout:5/240: dwrite d2/da/d10/d13/f35 [4194304,4194304] 0 2024-09-16T00:25:59.017 INFO:tasks.workunit.client.1.smithi169.stdout:8/245: dwrite d7/dc/d17/f39 [0,4194304] 0 2024-09-16T00:25:59.025 INFO:tasks.workunit.client.1.smithi169.stdout:8/246: symlink d7/dc/d2a/l50 0 2024-09-16T00:25:59.025 INFO:tasks.workunit.client.1.smithi169.stdout:5/241: chown d2/da/d10/d13/d1c/d40/c48 184490 1 2024-09-16T00:25:59.026 INFO:tasks.workunit.client.1.smithi169.stdout:6/183: dwrite d2/d15/f1b [4194304,4194304] 0 2024-09-16T00:25:59.028 INFO:tasks.workunit.client.1.smithi169.stdout:8/247: chown d7/dc/d17/d48 823 1 2024-09-16T00:25:59.028 INFO:tasks.workunit.client.1.smithi169.stdout:4/297: dread d2/d12/d13/d1f/d3e/d5b/f24 [0,4194304] 0 2024-09-16T00:25:59.028 INFO:tasks.workunit.client.1.smithi169.stdout:0/264: dread d0/d2/f32 [0,4194304] 0 2024-09-16T00:25:59.029 INFO:tasks.workunit.client.0.smithi112.stdout:0/460: dwrite d5/da/dc/f13 [4194304,4194304] 0 2024-09-16T00:25:59.031 INFO:tasks.workunit.client.1.smithi169.stdout:5/242: mkdir d2/da/d10/d13/d1c/d40/d54 0 2024-09-16T00:25:59.034 INFO:tasks.workunit.client.1.smithi169.stdout:8/248: symlink d7/dc/d26/d46/l51 0 2024-09-16T00:25:59.037 INFO:tasks.workunit.client.1.smithi169.stdout:0/265: creat d0/d2/d8/f68 x:0 0 0 2024-09-16T00:25:59.038 INFO:tasks.workunit.client.0.smithi112.stdout:9/624: rename d1/d5/d6/d59/fd9 to d1/d5/d6/d59/fda 0 2024-09-16T00:25:59.039 INFO:tasks.workunit.client.0.smithi112.stdout:8/363: link d10/f21 d10/d12/f77 0 2024-09-16T00:25:59.040 INFO:tasks.workunit.client.1.smithi169.stdout:5/243: mkdir d2/da/d10/d13/d1c/d32/d53/d4e/d55 0 2024-09-16T00:25:59.040 INFO:tasks.workunit.client.1.smithi169.stdout:6/184: dwrite d2/d15/f1b [0,4194304] 0 2024-09-16T00:25:59.041 INFO:tasks.workunit.client.0.smithi112.stdout:6/428: mkdir dd/df/d91 0 2024-09-16T00:25:59.045 INFO:tasks.workunit.client.0.smithi112.stdout:9/625: symlink d1/d5/d20/d25/da9/ldb 0 2024-09-16T00:25:59.047 INFO:tasks.workunit.client.0.smithi112.stdout:9/626: stat d1/d5/d20/c44 0 2024-09-16T00:25:59.047 INFO:tasks.workunit.client.1.smithi169.stdout:6/185: mknod d2/d22/d32/c40 0 2024-09-16T00:25:59.048 INFO:tasks.workunit.client.1.smithi169.stdout:4/298: dread d2/d12/d13/d1f/d2f/f45 [0,4194304] 0 2024-09-16T00:25:59.050 INFO:tasks.workunit.client.0.smithi112.stdout:6/429: dwrite dd/df/d17/f77 [0,4194304] 0 2024-09-16T00:25:59.051 INFO:tasks.workunit.client.1.smithi169.stdout:9/352: dread d5/da/d2e/f62 [0,4194304] 0 2024-09-16T00:25:59.053 INFO:tasks.workunit.client.1.smithi169.stdout:5/244: getdents d2/da/d10/d13/d1c 0 2024-09-16T00:25:59.053 INFO:tasks.workunit.client.1.smithi169.stdout:6/186: truncate d2/d15/d1d/f33 159101 0 2024-09-16T00:25:59.053 INFO:tasks.workunit.client.1.smithi169.stdout:0/266: dwrite d0/d2c/d38/f5e [0,4194304] 0 2024-09-16T00:25:59.054 INFO:tasks.workunit.client.0.smithi112.stdout:9/627: creat d1/d76/fdc x:0 0 0 2024-09-16T00:25:59.056 INFO:tasks.workunit.client.1.smithi169.stdout:9/353: read - d5/da/d5b/f63 zero size 2024-09-16T00:25:59.057 INFO:tasks.workunit.client.0.smithi112.stdout:0/461: dwrite d5/da/f27 [0,4194304] 0 2024-09-16T00:25:59.057 INFO:tasks.workunit.client.0.smithi112.stdout:9/628: stat d1/d5/d6/d2d/d6a/cc7 0 2024-09-16T00:25:59.058 INFO:tasks.workunit.client.0.smithi112.stdout:0/462: fdatasync d5/f2c 0 2024-09-16T00:25:59.059 INFO:tasks.workunit.client.1.smithi169.stdout:0/267: write d0/d2c/d54/d56/f59 [757893,87601] 0 2024-09-16T00:25:59.059 INFO:tasks.workunit.client.1.smithi169.stdout:6/187: mkdir d2/d3f/d41 0 2024-09-16T00:25:59.059 INFO:tasks.workunit.client.0.smithi112.stdout:6/430: dread dd/df/f3d [0,4194304] 0 2024-09-16T00:25:59.061 INFO:tasks.workunit.client.0.smithi112.stdout:0/463: chown d5/da/d18/d24/d46/d88/f4a 32235909 1 2024-09-16T00:25:59.062 INFO:tasks.workunit.client.1.smithi169.stdout:4/299: truncate d2/d12/d13/d1f/d3e/d5b/d1a/f55 780486 0 2024-09-16T00:25:59.063 INFO:tasks.workunit.client.1.smithi169.stdout:5/245: truncate d2/da/d10/d13/d1c/d32/d53/f4d 539911 0 2024-09-16T00:25:59.065 INFO:tasks.workunit.client.1.smithi169.stdout:0/268: mknod d0/d2/d8/d5/c69 0 2024-09-16T00:25:59.065 INFO:tasks.workunit.client.0.smithi112.stdout:0/464: write d5/da/d57/d76/d74/f79 [798392,90806] 0 2024-09-16T00:25:59.065 INFO:tasks.workunit.client.0.smithi112.stdout:5/384: sync 2024-09-16T00:25:59.069 INFO:tasks.workunit.client.1.smithi169.stdout:6/188: rename d2/c4 to d2/c42 0 2024-09-16T00:25:59.069 INFO:tasks.workunit.client.0.smithi112.stdout:0/465: rmdir d5/da 39 2024-09-16T00:25:59.070 INFO:tasks.workunit.client.1.smithi169.stdout:0/269: fdatasync d0/d2c/d38/f66 0 2024-09-16T00:25:59.070 INFO:tasks.workunit.client.0.smithi112.stdout:6/431: dwrite dd/f7e [0,4194304] 0 2024-09-16T00:25:59.072 INFO:tasks.workunit.client.0.smithi112.stdout:8/364: creat d10/d12/d1c/d75/f78 x:0 0 0 2024-09-16T00:25:59.074 INFO:tasks.workunit.client.1.smithi169.stdout:0/270: mknod d0/d2/d4b/c6a 0 2024-09-16T00:25:59.076 INFO:tasks.workunit.client.0.smithi112.stdout:0/466: symlink d5/da/d18/l93 0 2024-09-16T00:25:59.076 INFO:tasks.workunit.client.1.smithi169.stdout:6/189: rename d2/d2e to d2/d22/d32/d43 0 2024-09-16T00:25:59.077 INFO:tasks.workunit.client.1.smithi169.stdout:8/249: dread f5 [0,4194304] 0 2024-09-16T00:25:59.077 INFO:tasks.workunit.client.1.smithi169.stdout:5/246: getdents d2/da/d10/d13/d1c/d40/d43 0 2024-09-16T00:25:59.078 INFO:tasks.workunit.client.1.smithi169.stdout:6/190: chown d2/f19 1249462333 1 2024-09-16T00:25:59.078 INFO:tasks.workunit.client.1.smithi169.stdout:0/271: symlink d0/d2/d8/d5/d60/l6b 0 2024-09-16T00:25:59.079 INFO:tasks.workunit.client.0.smithi112.stdout:0/467: symlink d5/da/d21/d6f/l94 0 2024-09-16T00:25:59.080 INFO:tasks.workunit.client.1.smithi169.stdout:5/247: chown d2/da/d10/d13/f2f 147903974 1 2024-09-16T00:25:59.080 INFO:tasks.workunit.client.1.smithi169.stdout:0/272: chown d0/d2/c34 1356392766 1 2024-09-16T00:25:59.081 INFO:tasks.workunit.client.1.smithi169.stdout:5/248: chown d2/da/d10 887171 1 2024-09-16T00:25:59.081 INFO:tasks.workunit.client.1.smithi169.stdout:6/191: creat d2/d36/f44 x:0 0 0 2024-09-16T00:25:59.082 INFO:tasks.workunit.client.0.smithi112.stdout:0/468: creat d5/f95 x:0 0 0 2024-09-16T00:25:59.082 INFO:tasks.workunit.client.1.smithi169.stdout:0/273: readlink d0/d2/d15/l26 0 2024-09-16T00:25:59.083 INFO:tasks.workunit.client.0.smithi112.stdout:0/469: chown d5/da/d57/f73 2203 1 2024-09-16T00:25:59.084 INFO:tasks.workunit.client.0.smithi112.stdout:8/365: dread d10/d12/d24/f29 [0,4194304] 0 2024-09-16T00:25:59.085 INFO:tasks.workunit.client.1.smithi169.stdout:5/249: creat d2/da/d10/d13/d1c/d40/d54/f56 x:0 0 0 2024-09-16T00:25:59.085 INFO:tasks.workunit.client.1.smithi169.stdout:0/274: mkdir d0/d50/d6c 0 2024-09-16T00:25:59.085 INFO:tasks.workunit.client.0.smithi112.stdout:0/470: rmdir d5/da/d57/d76/d74/d80 0 2024-09-16T00:25:59.086 INFO:tasks.workunit.client.0.smithi112.stdout:0/471: dread - d5/da/d57/f73 zero size 2024-09-16T00:25:59.087 INFO:tasks.workunit.client.1.smithi169.stdout:0/275: write d0/d2/f6 [6472754,5035] 0 2024-09-16T00:25:59.087 INFO:tasks.workunit.client.1.smithi169.stdout:5/250: rmdir d2/da/d10/d13/d1c 39 2024-09-16T00:25:59.087 INFO:tasks.workunit.client.0.smithi112.stdout:8/366: creat d10/d12/d1c/d3b/d74/f79 x:0 0 0 2024-09-16T00:25:59.088 INFO:tasks.workunit.client.1.smithi169.stdout:6/192: getdents d2/d15/d1d 0 2024-09-16T00:25:59.088 INFO:tasks.workunit.client.0.smithi112.stdout:0/472: mkdir d5/da/d18/d24/d96 0 2024-09-16T00:25:59.088 INFO:tasks.workunit.client.0.smithi112.stdout:8/367: dread - d10/d12/d1c/d61/d6c/f76 zero size 2024-09-16T00:25:59.089 INFO:tasks.workunit.client.1.smithi169.stdout:0/276: write d0/d2c/d54/d16/f40 [711504,72658] 0 2024-09-16T00:25:59.089 INFO:tasks.workunit.client.1.smithi169.stdout:5/251: fsync d2/da/d10/d13/d24/f2c 0 2024-09-16T00:25:59.090 INFO:tasks.workunit.client.0.smithi112.stdout:8/368: creat d10/d12/d1c/d61/d6c/f7a x:0 0 0 2024-09-16T00:25:59.090 INFO:tasks.workunit.client.1.smithi169.stdout:6/193: write d2/f1f [447554,14658] 0 2024-09-16T00:25:59.091 INFO:tasks.workunit.client.0.smithi112.stdout:8/369: write fe [3191096,88137] 0 2024-09-16T00:25:59.093 INFO:tasks.workunit.client.1.smithi169.stdout:0/277: creat d0/d4c/f6d x:0 0 0 2024-09-16T00:25:59.093 INFO:tasks.workunit.client.0.smithi112.stdout:9/629: dread d1/d5/d6/d38/f7b [0,4194304] 0 2024-09-16T00:25:59.093 INFO:tasks.workunit.client.0.smithi112.stdout:8/370: write d10/d4e/f6f [571105,18056] 0 2024-09-16T00:25:59.093 INFO:tasks.workunit.client.0.smithi112.stdout:0/473: rename d5/da/dc/fe to d5/da/d57/d76/f97 0 2024-09-16T00:25:59.094 INFO:tasks.workunit.client.1.smithi169.stdout:0/278: chown d0/d2c/d54/d21 251583658 1 2024-09-16T00:25:59.095 INFO:tasks.workunit.client.0.smithi112.stdout:0/474: readlink d5/da/dc/l55 0 2024-09-16T00:25:59.096 INFO:tasks.workunit.client.1.smithi169.stdout:6/194: dread d2/d36/f31 [0,4194304] 0 2024-09-16T00:25:59.096 INFO:tasks.workunit.client.1.smithi169.stdout:5/252: creat d2/da/d10/d13/d1c/d2d/f57 x:0 0 0 2024-09-16T00:25:59.097 INFO:tasks.workunit.client.1.smithi169.stdout:0/279: truncate d0/d2/f45 21959 0 2024-09-16T00:25:59.097 INFO:tasks.workunit.client.0.smithi112.stdout:8/371: truncate d10/d12/d1c/d3b/d74/f79 879880 0 2024-09-16T00:25:59.097 INFO:tasks.workunit.client.0.smithi112.stdout:8/372: readlink d10/l35 0 2024-09-16T00:25:59.097 INFO:tasks.workunit.client.0.smithi112.stdout:9/630: stat d1/d5/d20/d25/d66/ld3 0 2024-09-16T00:25:59.098 INFO:tasks.workunit.client.1.smithi169.stdout:0/280: chown d0/d2/fd 0 1 2024-09-16T00:25:59.099 INFO:tasks.workunit.client.1.smithi169.stdout:6/195: dread d2/f1f [0,4194304] 0 2024-09-16T00:25:59.099 INFO:tasks.workunit.client.0.smithi112.stdout:0/475: creat d5/da/d18/d89/f98 x:0 0 0 2024-09-16T00:25:59.100 INFO:tasks.workunit.client.1.smithi169.stdout:8/250: truncate d7/f12 1378584 0 2024-09-16T00:25:59.100 INFO:tasks.workunit.client.0.smithi112.stdout:9/631: mkdir d1/d5/d20/d25/da9/ddd 0 2024-09-16T00:25:59.100 INFO:tasks.workunit.client.1.smithi169.stdout:6/196: dread - d2/d36/f2a zero size 2024-09-16T00:25:59.101 INFO:tasks.workunit.client.0.smithi112.stdout:0/476: unlink d5/da/dc/c40 0 2024-09-16T00:25:59.103 INFO:tasks.workunit.client.1.smithi169.stdout:8/251: mknod d7/dc/d1d/c52 0 2024-09-16T00:25:59.104 INFO:tasks.workunit.client.0.smithi112.stdout:8/373: dwrite d10/d12/f1a [4194304,4194304] 0 2024-09-16T00:25:59.104 INFO:tasks.workunit.client.0.smithi112.stdout:0/477: creat d5/da/d66/f99 x:0 0 0 2024-09-16T00:25:59.104 INFO:tasks.workunit.client.1.smithi169.stdout:6/197: creat d2/f45 x:0 0 0 2024-09-16T00:25:59.105 INFO:tasks.workunit.client.1.smithi169.stdout:6/198: chown d2/d22/d32/d43/f2f 944199707 1 2024-09-16T00:25:59.105 INFO:tasks.workunit.client.1.smithi169.stdout:0/281: dwrite d0/d2/d8/f68 [0,4194304] 0 2024-09-16T00:25:59.106 INFO:tasks.workunit.client.0.smithi112.stdout:0/478: fdatasync d5/da/dc/f81 0 2024-09-16T00:25:59.106 INFO:tasks.workunit.client.0.smithi112.stdout:8/374: stat d10/d12/f16 0 2024-09-16T00:25:59.106 INFO:tasks.workunit.client.1.smithi169.stdout:8/252: mkdir d7/dc/d53 0 2024-09-16T00:25:59.109 INFO:tasks.workunit.client.0.smithi112.stdout:9/632: dwrite d1/d5/d6/d38/f7b [0,4194304] 0 2024-09-16T00:25:59.109 INFO:tasks.workunit.client.0.smithi112.stdout:0/479: creat d5/da/d18/f9a x:0 0 0 2024-09-16T00:25:59.110 INFO:tasks.workunit.client.1.smithi169.stdout:8/253: truncate d7/dc/d17/f39 4637830 0 2024-09-16T00:25:59.110 INFO:tasks.workunit.client.1.smithi169.stdout:0/282: truncate d0/d2/d8/f47 1302308 0 2024-09-16T00:25:59.111 INFO:tasks.workunit.client.1.smithi169.stdout:8/254: stat d7/dc/d1d/l36 0 2024-09-16T00:25:59.111 INFO:tasks.workunit.client.0.smithi112.stdout:9/633: chown d1/d5/d6/d2d 0 1 2024-09-16T00:25:59.111 INFO:tasks.workunit.client.1.smithi169.stdout:6/199: dread d2/d22/d32/d43/f39 [0,4194304] 0 2024-09-16T00:25:59.113 INFO:tasks.workunit.client.1.smithi169.stdout:0/283: rename d0/c12 to d0/d50/d6c/c6e 0 2024-09-16T00:25:59.114 INFO:tasks.workunit.client.0.smithi112.stdout:0/480: write d5/da/d57/d76/f6b [342178,100151] 0 2024-09-16T00:25:59.114 INFO:tasks.workunit.client.1.smithi169.stdout:8/255: rmdir d7/dc/d17/d24 39 2024-09-16T00:25:59.115 INFO:tasks.workunit.client.0.smithi112.stdout:0/481: write d5/da/d66/f69 [1665145,129436] 0 2024-09-16T00:25:59.116 INFO:tasks.workunit.client.1.smithi169.stdout:6/200: write d2/d22/d32/d43/f39 [1121574,120192] 0 2024-09-16T00:25:59.118 INFO:tasks.workunit.client.0.smithi112.stdout:0/482: chown d5/da/d57/d76/f17 102023415 1 2024-09-16T00:25:59.120 INFO:tasks.workunit.client.0.smithi112.stdout:0/483: chown d5/da/d1d/d71 6843285 1 2024-09-16T00:25:59.120 INFO:tasks.workunit.client.0.smithi112.stdout:0/484: readlink d5/da/d18/d63/l78 0 2024-09-16T00:25:59.121 INFO:tasks.workunit.client.1.smithi169.stdout:8/256: dread d7/dc/d17/f39 [0,4194304] 0 2024-09-16T00:25:59.121 INFO:tasks.workunit.client.0.smithi112.stdout:0/485: fdatasync d5/da/f27 0 2024-09-16T00:25:59.121 INFO:tasks.workunit.client.1.smithi169.stdout:6/201: rename d2/d22/d32/d43/f39 to d2/d3f/d41/f46 0 2024-09-16T00:25:59.122 INFO:tasks.workunit.client.0.smithi112.stdout:0/486: read - d5/da/f77 zero size 2024-09-16T00:25:59.125 INFO:tasks.workunit.client.0.smithi112.stdout:0/487: link d5/da/f2e d5/da/d18/d89/f9b 0 2024-09-16T00:25:59.126 INFO:tasks.workunit.client.0.smithi112.stdout:0/488: mknod d5/da/d21/c9c 0 2024-09-16T00:25:59.128 INFO:tasks.workunit.client.0.smithi112.stdout:0/489: mknod d5/da/d18/d24/d46/d5f/d7a/c9d 0 2024-09-16T00:25:59.133 INFO:tasks.workunit.client.0.smithi112.stdout:0/490: unlink d5/da/d18/c2d 0 2024-09-16T00:25:59.136 INFO:tasks.workunit.client.0.smithi112.stdout:0/491: dwrite d5/da/d18/d63/f65 [0,4194304] 0 2024-09-16T00:25:59.144 INFO:tasks.workunit.client.0.smithi112.stdout:0/492: dwrite d5/da/d21/f7c [0,4194304] 0 2024-09-16T00:25:59.145 INFO:tasks.workunit.client.0.smithi112.stdout:9/634: dread d1/d5/d20/f69 [0,4194304] 0 2024-09-16T00:25:59.147 INFO:tasks.workunit.client.0.smithi112.stdout:0/493: creat d5/da/d57/d76/d74/f9e x:0 0 0 2024-09-16T00:25:59.148 INFO:tasks.workunit.client.0.smithi112.stdout:9/635: creat d1/d5/d20/d25/da9/dba/fde x:0 0 0 2024-09-16T00:25:59.150 INFO:tasks.workunit.client.0.smithi112.stdout:9/636: creat d1/d5/d6/d38/fdf x:0 0 0 2024-09-16T00:25:59.152 INFO:tasks.workunit.client.0.smithi112.stdout:9/637: creat d1/d5/d20/d25/fe0 x:0 0 0 2024-09-16T00:25:59.154 INFO:tasks.workunit.client.0.smithi112.stdout:0/494: dwrite d5/da/d18/f9a [0,4194304] 0 2024-09-16T00:25:59.155 INFO:tasks.workunit.client.0.smithi112.stdout:9/638: unlink d1/d5/d20/d25/fd6 0 2024-09-16T00:25:59.156 INFO:tasks.workunit.client.0.smithi112.stdout:0/495: readlink d5/da/d18/d24/d5a/l86 0 2024-09-16T00:25:59.158 INFO:tasks.workunit.client.0.smithi112.stdout:0/496: chown d5/da/d57/d76/d74/f9e 469867 1 2024-09-16T00:25:59.159 INFO:tasks.workunit.client.0.smithi112.stdout:9/639: mkdir d1/d5/d20/d81/dc4/de1 0 2024-09-16T00:25:59.160 INFO:tasks.workunit.client.0.smithi112.stdout:9/640: creat d1/d76/fe2 x:0 0 0 2024-09-16T00:25:59.161 INFO:tasks.workunit.client.0.smithi112.stdout:9/641: read - d1/d5/d6/d2d/f97 zero size 2024-09-16T00:25:59.162 INFO:tasks.workunit.client.0.smithi112.stdout:0/497: rename d5/da/f77 to d5/da/d57/d76/f9f 0 2024-09-16T00:25:59.164 INFO:tasks.workunit.client.0.smithi112.stdout:0/498: write d5/da/d18/d89/f98 [747571,37195] 0 2024-09-16T00:25:59.165 INFO:tasks.workunit.client.0.smithi112.stdout:0/499: readlink d5/da/d18/d24/d5a/l86 0 2024-09-16T00:25:59.167 INFO:tasks.workunit.client.0.smithi112.stdout:0/500: symlink d5/da/dc/la0 0 2024-09-16T00:25:59.167 INFO:tasks.workunit.client.0.smithi112.stdout:9/642: dwrite d1/d5/d83/fca [0,4194304] 0 2024-09-16T00:25:59.168 INFO:tasks.workunit.client.0.smithi112.stdout:9/643: write d1/d5/d20/d25/d66/faf [179329,52200] 0 2024-09-16T00:25:59.170 INFO:tasks.workunit.client.0.smithi112.stdout:0/501: creat d5/da/d18/d24/fa1 x:0 0 0 2024-09-16T00:25:59.172 INFO:tasks.workunit.client.0.smithi112.stdout:9/644: creat d1/d5/d20/d81/d1a/d88/da8/fe3 x:0 0 0 2024-09-16T00:25:59.175 INFO:tasks.workunit.client.0.smithi112.stdout:0/502: dread d5/da/f27 [0,4194304] 0 2024-09-16T00:25:59.175 INFO:tasks.workunit.client.0.smithi112.stdout:9/645: symlink d1/d5/d20/d81/d1a/d88/le4 0 2024-09-16T00:25:59.178 INFO:tasks.workunit.client.1.smithi169.stdout:6/202: mknod d2/d3f/c47 0 2024-09-16T00:25:59.178 INFO:tasks.workunit.client.0.smithi112.stdout:9/646: creat d1/d5/d20/d81/fe5 x:0 0 0 2024-09-16T00:25:59.178 INFO:tasks.workunit.client.0.smithi112.stdout:0/503: dread d5/ff [0,4194304] 0 2024-09-16T00:25:59.179 INFO:tasks.workunit.client.0.smithi112.stdout:9/647: chown d1/d5/d20/d25/da9/lb1 206748425 1 2024-09-16T00:25:59.180 INFO:tasks.workunit.client.1.smithi169.stdout:8/257: truncate d7/dc/d17/f28 3226495 0 2024-09-16T00:25:59.181 INFO:tasks.workunit.client.0.smithi112.stdout:0/504: creat d5/da/d18/d24/d46/d5f/d7a/fa2 x:0 0 0 2024-09-16T00:25:59.182 INFO:tasks.workunit.client.0.smithi112.stdout:9/648: write d1/d5/d6/d18/f40 [4999751,61634] 0 2024-09-16T00:25:59.182 INFO:tasks.workunit.client.1.smithi169.stdout:8/258: dread - d7/f2e zero size 2024-09-16T00:25:59.184 INFO:tasks.workunit.client.1.smithi169.stdout:0/284: dwrite d0/d2c/d54/f1c [0,4194304] 0 2024-09-16T00:25:59.185 INFO:tasks.workunit.client.0.smithi112.stdout:9/649: truncate d1/d5/d6/d18/f28 220255 0 2024-09-16T00:25:59.188 INFO:tasks.workunit.client.0.smithi112.stdout:1/397: sync 2024-09-16T00:25:59.188 INFO:tasks.workunit.client.0.smithi112.stdout:3/491: sync 2024-09-16T00:25:59.189 INFO:tasks.workunit.client.0.smithi112.stdout:3/492: stat d2/d9/f4e 0 2024-09-16T00:25:59.189 INFO:tasks.workunit.client.0.smithi112.stdout:0/505: dwrite d5/f68 [0,4194304] 0 2024-09-16T00:25:59.190 INFO:tasks.workunit.client.0.smithi112.stdout:3/493: stat d2/d9/f21 0 2024-09-16T00:25:59.191 INFO:tasks.workunit.client.0.smithi112.stdout:9/650: getdents d1/d5/d6/d2d/d6a/d89 0 2024-09-16T00:25:59.192 INFO:tasks.workunit.client.0.smithi112.stdout:0/506: fdatasync d5/da/d57/d76/f9f 0 2024-09-16T00:25:59.195 INFO:tasks.workunit.client.0.smithi112.stdout:0/507: readlink d5/da/d57/d76/l60 0 2024-09-16T00:25:59.196 INFO:tasks.workunit.client.0.smithi112.stdout:0/508: fsync d5/da/f27 0 2024-09-16T00:25:59.196 INFO:tasks.workunit.client.0.smithi112.stdout:3/494: getdents d2/d9 0 2024-09-16T00:25:59.198 INFO:tasks.workunit.client.0.smithi112.stdout:0/509: creat d5/da/d18/d63/fa3 x:0 0 0 2024-09-16T00:25:59.199 INFO:tasks.workunit.client.0.smithi112.stdout:0/510: write d5/f23 [3866542,54496] 0 2024-09-16T00:25:59.201 INFO:tasks.workunit.client.0.smithi112.stdout:0/511: unlink d5/da/d57/d76/f3c 0 2024-09-16T00:25:59.203 INFO:tasks.workunit.client.0.smithi112.stdout:0/512: mknod d5/da/d57/d76/ca4 0 2024-09-16T00:25:59.204 INFO:tasks.workunit.client.0.smithi112.stdout:0/513: write d5/f68 [1048497,8981] 0 2024-09-16T00:25:59.207 INFO:tasks.workunit.client.0.smithi112.stdout:0/514: mkdir d5/da/d21/da5 0 2024-09-16T00:25:59.212 INFO:tasks.workunit.client.0.smithi112.stdout:0/515: read d5/da/d66/f5d [1669176,63192] 0 2024-09-16T00:25:59.214 INFO:tasks.workunit.client.1.smithi169.stdout:0/285: dread d0/d2c/d54/f28 [0,4194304] 0 2024-09-16T00:25:59.216 INFO:tasks.workunit.client.1.smithi169.stdout:0/286: mkdir d0/d4c/d6f 0 2024-09-16T00:25:59.216 INFO:tasks.workunit.client.0.smithi112.stdout:0/516: rename d5/da/d1d/d71/d4c to d5/da/d18/d24/d46/d5f/da6 0 2024-09-16T00:25:59.218 INFO:tasks.workunit.client.1.smithi169.stdout:8/259: dread d7/f10 [0,4194304] 0 2024-09-16T00:25:59.218 INFO:tasks.workunit.client.0.smithi112.stdout:0/517: stat d5/c2f 0 2024-09-16T00:25:59.219 INFO:tasks.workunit.client.1.smithi169.stdout:0/287: rename d0/d2c/d54/f2b to d0/d50/f70 0 2024-09-16T00:25:59.220 INFO:tasks.workunit.client.1.smithi169.stdout:0/288: stat d0/d2/d8/d5/d60/l6b 0 2024-09-16T00:25:59.220 INFO:tasks.workunit.client.1.smithi169.stdout:8/260: symlink d7/dc/d26/d41/d4e/l54 0 2024-09-16T00:25:59.221 INFO:tasks.workunit.client.0.smithi112.stdout:0/518: dread d5/da/d18/f9a [0,4194304] 0 2024-09-16T00:25:59.221 INFO:tasks.workunit.client.1.smithi169.stdout:8/261: chown d7/l2f 229784755 1 2024-09-16T00:25:59.221 INFO:tasks.workunit.client.1.smithi169.stdout:8/262: fsync d7/fa 0 2024-09-16T00:25:59.223 INFO:tasks.workunit.client.1.smithi169.stdout:0/289: rename d0/l4e to d0/d2c/d54/d21/l71 0 2024-09-16T00:25:59.225 INFO:tasks.workunit.client.1.smithi169.stdout:6/203: write d2/f17 [912164,62887] 0 2024-09-16T00:25:59.226 INFO:tasks.workunit.client.0.smithi112.stdout:0/519: link d5/da/d18/d24/c30 d5/da/d66/ca7 0 2024-09-16T00:25:59.227 INFO:tasks.workunit.client.0.smithi112.stdout:0/520: dread - d5/da/dc/f81 zero size 2024-09-16T00:25:59.227 INFO:tasks.workunit.client.1.smithi169.stdout:8/263: dread d7/dc/d2a/f42 [0,4194304] 0 2024-09-16T00:25:59.228 INFO:tasks.workunit.client.1.smithi169.stdout:6/204: mkdir d2/d36/d48 0 2024-09-16T00:25:59.229 INFO:tasks.workunit.client.0.smithi112.stdout:0/521: dread - d5/da/d18/d24/d5a/f82 zero size 2024-09-16T00:25:59.229 INFO:tasks.workunit.client.0.smithi112.stdout:1/398: dread d0/f4e [0,4194304] 0 2024-09-16T00:25:59.230 INFO:tasks.workunit.client.0.smithi112.stdout:9/651: write d1/d5/d6/d38/f7f [245933,124537] 0 2024-09-16T00:25:59.231 INFO:tasks.workunit.client.1.smithi169.stdout:8/264: unlink d7/c9 0 2024-09-16T00:25:59.231 INFO:tasks.workunit.client.0.smithi112.stdout:0/522: mknod d5/da/d1d/ca8 0 2024-09-16T00:25:59.232 INFO:tasks.workunit.client.0.smithi112.stdout:0/523: chown d5/da/d18/d89/f98 391764 1 2024-09-16T00:25:59.232 INFO:tasks.workunit.client.1.smithi169.stdout:0/290: dwrite d0/d2/d8/f65 [0,4194304] 0 2024-09-16T00:25:59.233 INFO:tasks.workunit.client.0.smithi112.stdout:0/524: chown d5/f2a 34163 1 2024-09-16T00:25:59.233 INFO:tasks.workunit.client.0.smithi112.stdout:9/652: write d1/d5/d6/d2d/d39/f63 [3136141,109824] 0 2024-09-16T00:25:59.234 INFO:tasks.workunit.client.0.smithi112.stdout:1/399: rmdir d0/d14/df/d28/d45/d46/d58/d62 39 2024-09-16T00:25:59.236 INFO:tasks.workunit.client.0.smithi112.stdout:0/525: write d5/da/d18/f9a [3735448,120941] 0 2024-09-16T00:25:59.237 INFO:tasks.workunit.client.1.smithi169.stdout:0/291: write d0/d50/f70 [3730442,40758] 0 2024-09-16T00:25:59.240 INFO:tasks.workunit.client.0.smithi112.stdout:9/653: creat d1/d5/d6/d2d/d6a/fe6 x:0 0 0 2024-09-16T00:25:59.240 INFO:tasks.workunit.client.1.smithi169.stdout:0/292: symlink d0/d2c/d54/l72 0 2024-09-16T00:25:59.244 INFO:tasks.workunit.client.0.smithi112.stdout:1/400: link d0/d14/d1c/f51 d0/d14/df/d28/d45/d46/d58/f7b 0 2024-09-16T00:25:59.245 INFO:tasks.workunit.client.1.smithi169.stdout:6/205: dread d2/f17 [0,4194304] 0 2024-09-16T00:25:59.246 INFO:tasks.workunit.client.0.smithi112.stdout:0/526: dwrite d5/da/d57/d76/d74/f79 [0,4194304] 0 2024-09-16T00:25:59.246 INFO:tasks.workunit.client.0.smithi112.stdout:9/654: mknod d1/d5/d20/d81/d1a/d88/ce7 0 2024-09-16T00:25:59.247 INFO:tasks.workunit.client.0.smithi112.stdout:9/655: chown d1/d5/d20 259922 1 2024-09-16T00:25:59.247 INFO:tasks.workunit.client.1.smithi169.stdout:8/265: dwrite d7/dc/d1d/f47 [0,4194304] 0 2024-09-16T00:25:59.247 INFO:tasks.workunit.client.1.smithi169.stdout:6/206: chown d2/l13 91 1 2024-09-16T00:25:59.248 INFO:tasks.workunit.client.0.smithi112.stdout:1/401: mknod d0/d14/df/d28/d45/d46/d6b/c7c 0 2024-09-16T00:25:59.250 INFO:tasks.workunit.client.0.smithi112.stdout:9/656: mknod d1/d5/d2c/ce8 0 2024-09-16T00:25:59.250 INFO:tasks.workunit.client.1.smithi169.stdout:6/207: creat d2/d15/d1d/f49 x:0 0 0 2024-09-16T00:25:59.250 INFO:tasks.workunit.client.1.smithi169.stdout:8/266: dread f5 [0,4194304] 0 2024-09-16T00:25:59.251 INFO:tasks.workunit.client.0.smithi112.stdout:1/402: stat d0/d14/df/d28/d45/d46/d6b 0 2024-09-16T00:25:59.251 INFO:tasks.workunit.client.1.smithi169.stdout:6/208: stat d2/d22/d32/c40 0 2024-09-16T00:25:59.252 INFO:tasks.workunit.client.1.smithi169.stdout:6/209: stat d2/d3f/d41 0 2024-09-16T00:25:59.252 INFO:tasks.workunit.client.1.smithi169.stdout:6/210: fdatasync d2/d36/f2a 0 2024-09-16T00:25:59.253 INFO:tasks.workunit.client.0.smithi112.stdout:0/527: creat d5/da/d1d/d71/fa9 x:0 0 0 2024-09-16T00:25:59.253 INFO:tasks.workunit.client.1.smithi169.stdout:8/267: unlink d7/dc/d26/c49 0 2024-09-16T00:25:59.254 INFO:tasks.workunit.client.0.smithi112.stdout:1/403: read d0/d14/d1c/f51 [4068701,34252] 0 2024-09-16T00:25:59.255 INFO:tasks.workunit.client.1.smithi169.stdout:6/211: creat d2/d22/d32/d43/f4a x:0 0 0 2024-09-16T00:25:59.257 INFO:tasks.workunit.client.0.smithi112.stdout:0/528: unlink d5/da/d21/d6f/f85 0 2024-09-16T00:25:59.257 INFO:tasks.workunit.client.0.smithi112.stdout:0/529: write d5/f23 [2813831,24412] 0 2024-09-16T00:25:59.258 INFO:tasks.workunit.client.1.smithi169.stdout:8/268: mknod d7/dc/d17/c55 0 2024-09-16T00:25:59.258 INFO:tasks.workunit.client.1.smithi169.stdout:6/212: mknod d2/d22/c4b 0 2024-09-16T00:25:59.259 INFO:tasks.workunit.client.1.smithi169.stdout:6/213: fdatasync d2/d22/f34 0 2024-09-16T00:25:59.261 INFO:tasks.workunit.client.1.smithi169.stdout:6/214: mkdir d2/d4c 0 2024-09-16T00:25:59.266 INFO:tasks.workunit.client.0.smithi112.stdout:1/404: dread d0/d14/d1c/d26/f52 [0,4194304] 0 2024-09-16T00:25:59.267 INFO:tasks.workunit.client.1.smithi169.stdout:6/215: truncate d2/d22/f34 476181 0 2024-09-16T00:25:59.269 INFO:tasks.workunit.client.1.smithi169.stdout:8/269: dwrite d7/dc/d17/f39 [0,4194304] 0 2024-09-16T00:25:59.271 INFO:tasks.workunit.client.1.smithi169.stdout:6/216: write d2/d15/d1d/f33 [969713,112480] 0 2024-09-16T00:25:59.271 INFO:tasks.workunit.client.1.smithi169.stdout:6/217: chown d2/d22/d32/d43 62147401 1 2024-09-16T00:25:59.273 INFO:tasks.workunit.client.1.smithi169.stdout:8/270: mknod d7/dc/d17/d48/c56 0 2024-09-16T00:25:59.273 INFO:tasks.workunit.client.1.smithi169.stdout:6/218: dread - d2/d15/d1d/f49 zero size 2024-09-16T00:25:59.273 INFO:tasks.workunit.client.1.smithi169.stdout:8/271: dread - d7/f2e zero size 2024-09-16T00:25:59.273 INFO:tasks.workunit.client.1.smithi169.stdout:8/272: chown d7/dc/d17/d48 7983 1 2024-09-16T00:25:59.274 INFO:tasks.workunit.client.1.smithi169.stdout:6/219: rmdir d2/d15 39 2024-09-16T00:25:59.277 INFO:tasks.workunit.client.1.smithi169.stdout:8/273: truncate d7/dc/d17/d24/f27 829408 0 2024-09-16T00:25:59.279 INFO:tasks.workunit.client.1.smithi169.stdout:8/274: truncate d7/dc/d17/f1b 4270539 0 2024-09-16T00:25:59.281 INFO:tasks.workunit.client.1.smithi169.stdout:8/275: truncate d7/dc/d17/d24/f27 953310 0 2024-09-16T00:25:59.282 INFO:tasks.workunit.client.1.smithi169.stdout:6/220: dwrite d2/d36/f31 [4194304,4194304] 0 2024-09-16T00:25:59.288 INFO:tasks.workunit.client.1.smithi169.stdout:8/276: link d7/dc/d26/l29 d7/d20/l57 0 2024-09-16T00:25:59.290 INFO:tasks.workunit.client.0.smithi112.stdout:1/405: dread d0/d14/d1c/d20/f21 [0,4194304] 0 2024-09-16T00:25:59.292 INFO:tasks.workunit.client.1.smithi169.stdout:6/221: dwrite d2/d15/f1e [0,4194304] 0 2024-09-16T00:25:59.294 INFO:tasks.workunit.client.1.smithi169.stdout:8/277: dwrite d7/fa [0,4194304] 0 2024-09-16T00:25:59.299 INFO:tasks.workunit.client.1.smithi169.stdout:6/222: rename d2/d15/d1d to d2/d4d 0 2024-09-16T00:25:59.303 INFO:tasks.workunit.client.1.smithi169.stdout:8/278: rename d7/dc/d1d/c34 to d7/dc/d26/d46/c58 0 2024-09-16T00:25:59.305 INFO:tasks.workunit.client.1.smithi169.stdout:8/279: write f5 [1740789,4357] 0 2024-09-16T00:25:59.306 INFO:tasks.workunit.client.1.smithi169.stdout:8/280: write d7/fd [4281927,124572] 0 2024-09-16T00:25:59.309 INFO:tasks.workunit.client.0.smithi112.stdout:1/406: dread d0/f2 [0,4194304] 0 2024-09-16T00:25:59.311 INFO:tasks.workunit.client.0.smithi112.stdout:1/407: creat d0/d14/d1c/d26/d35/f7d x:0 0 0 2024-09-16T00:25:59.316 INFO:tasks.workunit.client.1.smithi169.stdout:6/223: rmdir d2 39 2024-09-16T00:25:59.316 INFO:tasks.workunit.client.0.smithi112.stdout:1/408: dwrite d0/d14/d1c/d26/d35/f59 [0,4194304] 0 2024-09-16T00:25:59.318 INFO:tasks.workunit.client.0.smithi112.stdout:1/409: write d0/d14/df/d28/f32 [3999333,30978] 0 2024-09-16T00:25:59.318 INFO:tasks.workunit.client.1.smithi169.stdout:8/281: write d7/dc/d2a/f30 [2258893,38906] 0 2024-09-16T00:25:59.325 INFO:tasks.workunit.client.0.smithi112.stdout:1/410: dwrite d0/d14/df/d28/f2e [0,4194304] 0 2024-09-16T00:25:59.326 INFO:tasks.workunit.client.0.smithi112.stdout:1/411: readlink d0/d14/d1c/d26/d41/l55 0 2024-09-16T00:25:59.326 INFO:tasks.workunit.client.1.smithi169.stdout:6/224: dwrite d2/d4d/f49 [0,4194304] 0 2024-09-16T00:25:59.328 INFO:tasks.workunit.client.0.smithi112.stdout:1/412: write d0/f18 [2213831,22634] 0 2024-09-16T00:25:59.329 INFO:tasks.workunit.client.1.smithi169.stdout:8/282: dwrite f5 [0,4194304] 0 2024-09-16T00:25:59.330 INFO:tasks.workunit.client.1.smithi169.stdout:6/225: mknod d2/d4c/c4e 0 2024-09-16T00:25:59.332 INFO:tasks.workunit.client.1.smithi169.stdout:6/226: rmdir d2/d4c 39 2024-09-16T00:25:59.337 INFO:tasks.workunit.client.1.smithi169.stdout:6/227: creat d2/d22/d32/d38/f4f x:0 0 0 2024-09-16T00:25:59.340 INFO:tasks.workunit.client.1.smithi169.stdout:8/283: link d7/d37/l4a d7/dc/d17/d24/l59 0 2024-09-16T00:25:59.341 INFO:tasks.workunit.client.1.smithi169.stdout:6/228: dread d2/d22/d32/f35 [0,4194304] 0 2024-09-16T00:25:59.346 INFO:tasks.workunit.client.1.smithi169.stdout:8/284: link d7/dc/d17/d24/f4b d7/dc/d17/d48/f5a 0 2024-09-16T00:25:59.347 INFO:tasks.workunit.client.1.smithi169.stdout:6/229: getdents d2 0 2024-09-16T00:25:59.348 INFO:tasks.workunit.client.1.smithi169.stdout:6/230: mkdir d2/d3f/d50 0 2024-09-16T00:25:59.349 INFO:tasks.workunit.client.1.smithi169.stdout:6/231: dread d2/f30 [0,4194304] 0 2024-09-16T00:25:59.350 INFO:tasks.workunit.client.1.smithi169.stdout:6/232: chown d2/d22 8885148 1 2024-09-16T00:25:59.353 INFO:tasks.workunit.client.1.smithi169.stdout:6/233: dread d2/d4d/f49 [0,4194304] 0 2024-09-16T00:25:59.354 INFO:tasks.workunit.client.0.smithi112.stdout:2/499: sync 2024-09-16T00:25:59.354 INFO:tasks.workunit.client.0.smithi112.stdout:4/377: sync 2024-09-16T00:25:59.354 INFO:tasks.workunit.client.0.smithi112.stdout:7/451: sync 2024-09-16T00:25:59.354 INFO:tasks.workunit.client.0.smithi112.stdout:5/385: sync 2024-09-16T00:25:59.355 INFO:tasks.workunit.client.0.smithi112.stdout:4/378: symlink d6/d39/l8b 0 2024-09-16T00:25:59.357 INFO:tasks.workunit.client.0.smithi112.stdout:4/379: dread - d6/db/df/d5e/d61/d64/f81 zero size 2024-09-16T00:25:59.358 INFO:tasks.workunit.client.0.smithi112.stdout:7/452: creat d6/d10/d16/d3b/fa6 x:0 0 0 2024-09-16T00:25:59.359 INFO:tasks.workunit.client.1.smithi169.stdout:6/234: dwrite d2/f45 [0,4194304] 0 2024-09-16T00:25:59.360 INFO:tasks.workunit.client.0.smithi112.stdout:5/386: link d4/f5c d4/dd/d49/d70/d83/f86 0 2024-09-16T00:25:59.360 INFO:tasks.workunit.client.0.smithi112.stdout:4/380: creat d6/d39/d36/f8c x:0 0 0 2024-09-16T00:25:59.361 INFO:tasks.workunit.client.1.smithi169.stdout:6/235: dread d2/f30 [0,4194304] 0 2024-09-16T00:25:59.361 INFO:tasks.workunit.client.0.smithi112.stdout:4/381: fdatasync d6/d39/d36/f8c 0 2024-09-16T00:25:59.363 INFO:tasks.workunit.client.1.smithi169.stdout:6/236: mkdir d2/d22/d51 0 2024-09-16T00:25:59.366 INFO:tasks.workunit.client.1.smithi169.stdout:6/237: rename d2/d15 to d2/d3f/d41/d52 0 2024-09-16T00:25:59.366 INFO:tasks.workunit.client.0.smithi112.stdout:2/500: dread d2/dd/d48/d94/d49/f71 [0,4194304] 0 2024-09-16T00:25:59.376 INFO:tasks.workunit.client.0.smithi112.stdout:7/453: dwrite d6/d10/d1b/f52 [4194304,4194304] 0 2024-09-16T00:25:59.376 INFO:tasks.workunit.client.0.smithi112.stdout:4/382: dwrite d6/db/d1f/d41/d49/f67 [0,4194304] 0 2024-09-16T00:25:59.377 INFO:tasks.workunit.client.1.smithi169.stdout:6/238: creat d2/d4d/f53 x:0 0 0 2024-09-16T00:25:59.377 INFO:tasks.workunit.client.0.smithi112.stdout:7/454: fsync d6/d10/d1b/f7d 0 2024-09-16T00:25:59.379 INFO:tasks.workunit.client.1.smithi169.stdout:6/239: symlink d2/d22/d32/d43/l54 0 2024-09-16T00:25:59.379 INFO:tasks.workunit.client.0.smithi112.stdout:7/455: dread - d6/d10/f87 zero size 2024-09-16T00:25:59.381 INFO:tasks.workunit.client.1.smithi169.stdout:6/240: write d2/f19 [1188888,60613] 0 2024-09-16T00:25:59.382 INFO:tasks.workunit.client.0.smithi112.stdout:7/456: truncate d6/d10/d16/f34 2124341 0 2024-09-16T00:25:59.383 INFO:tasks.workunit.client.1.smithi169.stdout:6/241: dread d2/d22/d32/f35 [0,4194304] 0 2024-09-16T00:25:59.383 INFO:tasks.workunit.client.0.smithi112.stdout:7/457: stat d6/l37 0 2024-09-16T00:25:59.384 INFO:tasks.workunit.client.0.smithi112.stdout:4/383: dwrite d6/db/f88 [0,4194304] 0 2024-09-16T00:25:59.385 INFO:tasks.workunit.client.0.smithi112.stdout:4/384: dread - d6/db/d1f/d41/d49/f75 zero size 2024-09-16T00:25:59.388 INFO:tasks.workunit.client.0.smithi112.stdout:4/385: mknod d6/d39/c8d 0 2024-09-16T00:25:59.390 INFO:tasks.workunit.client.0.smithi112.stdout:4/386: write d6/db/f25 [193650,64637] 0 2024-09-16T00:25:59.392 INFO:tasks.workunit.client.0.smithi112.stdout:4/387: symlink d6/db/d1f/d41/d49/l8e 0 2024-09-16T00:25:59.394 INFO:tasks.workunit.client.1.smithi169.stdout:6/242: dwrite d2/d4d/f53 [0,4194304] 0 2024-09-16T00:25:59.395 INFO:tasks.workunit.client.0.smithi112.stdout:4/388: mkdir d6/d39/d36/d5c/d7e/d8f 0 2024-09-16T00:25:59.398 INFO:tasks.workunit.client.0.smithi112.stdout:4/389: creat d6/d39/d36/d5c/d7e/d8f/f90 x:0 0 0 2024-09-16T00:25:59.399 INFO:tasks.workunit.client.0.smithi112.stdout:4/390: chown d6/d39/d1c/d66/l89 1147616995 1 2024-09-16T00:25:59.399 INFO:tasks.workunit.client.1.smithi169.stdout:6/243: write d2/f30 [1076916,43892] 0 2024-09-16T00:25:59.400 INFO:tasks.workunit.client.0.smithi112.stdout:4/391: readlink d6/db/df/d5e/d61/d64/d6a/l8a 0 2024-09-16T00:25:59.401 INFO:tasks.workunit.client.1.smithi169.stdout:6/244: dread - d2/d36/f44 zero size 2024-09-16T00:25:59.404 INFO:tasks.workunit.client.1.smithi169.stdout:6/245: unlink d2/c2b 0 2024-09-16T00:25:59.405 INFO:tasks.workunit.client.0.smithi112.stdout:2/501: dread d2/f64 [0,4194304] 0 2024-09-16T00:25:59.406 INFO:tasks.workunit.client.0.smithi112.stdout:4/392: dwrite d6/d39/d1c/f57 [0,4194304] 0 2024-09-16T00:25:59.410 INFO:tasks.workunit.client.0.smithi112.stdout:2/502: getdents d2/dd/d48/d22/d2b/d54/d60/d6c 0 2024-09-16T00:25:59.413 INFO:tasks.workunit.client.0.smithi112.stdout:4/393: dwrite d6/db/df/f5f [0,4194304] 0 2024-09-16T00:25:59.415 INFO:tasks.workunit.client.0.smithi112.stdout:2/503: read d2/f14 [7167168,125033] 0 2024-09-16T00:25:59.415 INFO:tasks.workunit.client.0.smithi112.stdout:4/394: fdatasync d6/f1b 0 2024-09-16T00:25:59.417 INFO:tasks.workunit.client.0.smithi112.stdout:4/395: mknod d6/d39/d36/d5c/c91 0 2024-09-16T00:25:59.423 INFO:tasks.workunit.client.0.smithi112.stdout:2/504: dwrite d2/dd/d48/d22/d2b/f58 [0,4194304] 0 2024-09-16T00:25:59.425 INFO:tasks.workunit.client.0.smithi112.stdout:4/396: dwrite d6/db/df/d5e/d61/f3f [0,4194304] 0 2024-09-16T00:25:59.426 INFO:tasks.workunit.client.0.smithi112.stdout:2/505: mkdir d2/dd/d48/d8a/d9b 0 2024-09-16T00:25:59.427 INFO:tasks.workunit.client.0.smithi112.stdout:4/397: symlink d6/d39/d36/d5c/d7e/l92 0 2024-09-16T00:25:59.428 INFO:tasks.workunit.client.0.smithi112.stdout:2/506: chown d2/dd/d48/d94/d49/f71 43 1 2024-09-16T00:25:59.429 INFO:tasks.workunit.client.0.smithi112.stdout:2/507: dread - d2/dd/d48/d22/d2b/d30/f6f zero size 2024-09-16T00:25:59.430 INFO:tasks.workunit.client.0.smithi112.stdout:4/398: creat d6/db/f93 x:0 0 0 2024-09-16T00:25:59.435 INFO:tasks.workunit.client.0.smithi112.stdout:4/399: rename d6/db/d1f/d41/d49/d7a to d6/db/d1f/d94 0 2024-09-16T00:25:59.438 INFO:tasks.workunit.client.0.smithi112.stdout:2/508: dwrite d2/d68/f7f [0,4194304] 0 2024-09-16T00:25:59.438 INFO:tasks.workunit.client.0.smithi112.stdout:4/400: creat d6/d39/d36/d5c/d7e/f95 x:0 0 0 2024-09-16T00:25:59.442 INFO:tasks.workunit.client.0.smithi112.stdout:4/401: fdatasync d6/f9 0 2024-09-16T00:25:59.443 INFO:tasks.workunit.client.0.smithi112.stdout:2/509: getdents d2/dd/d48/d8a/d9b 0 2024-09-16T00:25:59.449 INFO:tasks.workunit.client.0.smithi112.stdout:4/402: dwrite d6/d39/d36/d5c/f77 [0,4194304] 0 2024-09-16T00:25:59.451 INFO:tasks.workunit.client.0.smithi112.stdout:4/403: symlink d6/d39/d1c/d2c/d69/d7b/l96 0 2024-09-16T00:25:59.452 INFO:tasks.workunit.client.0.smithi112.stdout:4/404: readlink d6/d39/d1c/l44 0 2024-09-16T00:25:59.452 INFO:tasks.workunit.client.0.smithi112.stdout:4/405: fsync d6/d39/f4a 0 2024-09-16T00:25:59.458 INFO:tasks.workunit.client.1.smithi169.stdout:6/246: link d2/l14 d2/d22/l55 0 2024-09-16T00:25:59.461 INFO:tasks.workunit.client.1.smithi169.stdout:6/247: dread d2/d22/f23 [4194304,4194304] 0 2024-09-16T00:25:59.465 INFO:tasks.workunit.client.1.smithi169.stdout:6/248: dread d2/d4d/f53 [0,4194304] 0 2024-09-16T00:25:59.467 INFO:tasks.workunit.client.1.smithi169.stdout:6/249: chown d2/c24 161835838 1 2024-09-16T00:25:59.469 INFO:tasks.workunit.client.1.smithi169.stdout:6/250: creat d2/d22/d32/d43/f56 x:0 0 0 2024-09-16T00:25:59.471 INFO:tasks.workunit.client.0.smithi112.stdout:4/406: dread d6/f9 [4194304,4194304] 0 2024-09-16T00:25:59.472 INFO:tasks.workunit.client.1.smithi169.stdout:6/251: rename d2/f30 to d2/d4d/f57 0 2024-09-16T00:25:59.473 INFO:tasks.workunit.client.1.smithi169.stdout:6/252: read d2/f1f [351302,67236] 0 2024-09-16T00:25:59.475 INFO:tasks.workunit.client.1.smithi169.stdout:6/253: chown d2/d36/f2a 1208134306 1 2024-09-16T00:25:59.476 INFO:tasks.workunit.client.1.smithi169.stdout:6/254: write d2/d4d/f49 [38260,104914] 0 2024-09-16T00:25:59.476 INFO:tasks.workunit.client.1.smithi169.stdout:6/255: dread - d2/d36/f2a zero size 2024-09-16T00:25:59.477 INFO:tasks.workunit.client.0.smithi112.stdout:4/407: dwrite d6/db/d1f/d41/d49/f67 [0,4194304] 0 2024-09-16T00:25:59.478 INFO:tasks.workunit.client.1.smithi169.stdout:6/256: creat d2/d36/f58 x:0 0 0 2024-09-16T00:25:59.482 INFO:tasks.workunit.client.0.smithi112.stdout:4/408: truncate d6/db/d1f/d41/d49/f67 2475469 0 2024-09-16T00:25:59.483 INFO:tasks.workunit.client.0.smithi112.stdout:4/409: write d6/db/f18 [3433893,71012] 0 2024-09-16T00:25:59.484 INFO:tasks.workunit.client.0.smithi112.stdout:4/410: dread - d6/db/df/d5e/d61/d6d/f7f zero size 2024-09-16T00:25:59.485 INFO:tasks.workunit.client.0.smithi112.stdout:4/411: read d6/db/df/d5e/d61/f3f [3248601,58651] 0 2024-09-16T00:25:59.486 INFO:tasks.workunit.client.1.smithi169.stdout:6/257: dwrite d2/d22/d32/d38/f4f [0,4194304] 0 2024-09-16T00:25:59.488 INFO:tasks.workunit.client.0.smithi112.stdout:4/412: fdatasync d6/db/df/d5e/d61/f3f 0 2024-09-16T00:25:59.488 INFO:tasks.workunit.client.0.smithi112.stdout:4/413: stat d6/db/d1f/d41/d49/c21 0 2024-09-16T00:25:59.489 INFO:tasks.workunit.client.0.smithi112.stdout:6/432: sync 2024-09-16T00:25:59.491 INFO:tasks.workunit.client.0.smithi112.stdout:6/433: symlink dd/df/d10/d47/d89/l92 0 2024-09-16T00:25:59.492 INFO:tasks.workunit.client.0.smithi112.stdout:4/414: creat d6/d39/d1c/d6e/d6f/f97 x:0 0 0 2024-09-16T00:25:59.501 INFO:tasks.workunit.client.1.smithi169.stdout:6/258: truncate d2/d3f/d41/d52/f1b 1848285 0 2024-09-16T00:25:59.504 INFO:tasks.workunit.client.0.smithi112.stdout:4/415: dread d6/db/d1f/d41/d49/f17 [0,4194304] 0 2024-09-16T00:25:59.504 INFO:tasks.workunit.client.0.smithi112.stdout:4/416: fsync d6/d39/d1c/f57 0 2024-09-16T00:25:59.505 INFO:tasks.workunit.client.1.smithi169.stdout:6/259: rename d2/d22/d32 to d2/d4d/d59 0 2024-09-16T00:25:59.506 INFO:tasks.workunit.client.0.smithi112.stdout:4/417: write d6/d39/d36/d5c/f63 [1102323,128723] 0 2024-09-16T00:25:59.507 INFO:tasks.workunit.client.0.smithi112.stdout:4/418: dread - d6/db/d1f/d41/d49/f75 zero size 2024-09-16T00:25:59.511 INFO:tasks.workunit.client.1.smithi169.stdout:6/260: dwrite d2/d4d/d59/d38/f4f [0,4194304] 0 2024-09-16T00:25:59.511 INFO:tasks.workunit.client.1.smithi169.stdout:6/261: stat d2/d4d/d59/d38 0 2024-09-16T00:25:59.513 INFO:tasks.workunit.client.0.smithi112.stdout:4/419: dwrite d6/d39/d1c/f27 [0,4194304] 0 2024-09-16T00:25:59.514 INFO:tasks.workunit.client.1.smithi169.stdout:6/262: creat d2/d4d/d59/d38/f5a x:0 0 0 2024-09-16T00:25:59.516 INFO:tasks.workunit.client.0.smithi112.stdout:4/420: creat d6/d39/d36/d5c/d7e/d8f/f98 x:0 0 0 2024-09-16T00:25:59.521 INFO:tasks.workunit.client.0.smithi112.stdout:6/434: dread dd/d23/f35 [0,4194304] 0 2024-09-16T00:25:59.522 INFO:tasks.workunit.client.1.smithi169.stdout:6/263: truncate d2/d3f/d41/d52/f18 1412266 0 2024-09-16T00:25:59.523 INFO:tasks.workunit.client.1.smithi169.stdout:6/264: readlink d2/l14 0 2024-09-16T00:25:59.524 INFO:tasks.workunit.client.0.smithi112.stdout:4/421: dwrite d6/d39/d36/f72 [0,4194304] 0 2024-09-16T00:25:59.526 INFO:tasks.workunit.client.0.smithi112.stdout:6/435: mknod dd/df/d11/d44/d8a/c93 0 2024-09-16T00:25:59.528 INFO:tasks.workunit.client.1.smithi169.stdout:6/265: creat d2/d3f/d50/f5b x:0 0 0 2024-09-16T00:25:59.528 INFO:tasks.workunit.client.0.smithi112.stdout:6/436: stat dd/df/d10/c4e 0 2024-09-16T00:25:59.530 INFO:tasks.workunit.client.1.smithi169.stdout:6/266: mkdir d2/d3f/d41/d5c 0 2024-09-16T00:25:59.530 INFO:tasks.workunit.client.0.smithi112.stdout:6/437: write dd/df/d13/d7c/f4a [1119448,39307] 0 2024-09-16T00:25:59.535 INFO:tasks.workunit.client.1.smithi169.stdout:6/267: dread d2/d4d/f57 [0,4194304] 0 2024-09-16T00:25:59.551 INFO:tasks.workunit.client.0.smithi112.stdout:4/422: dread d6/d39/d1c/f2b [0,4194304] 0 2024-09-16T00:25:59.552 INFO:tasks.workunit.client.0.smithi112.stdout:4/423: symlink d6/d39/d1c/d6e/l99 0 2024-09-16T00:25:59.553 INFO:tasks.workunit.client.0.smithi112.stdout:4/424: write d6/d39/d1c/d66/f7d [1008550,20334] 0 2024-09-16T00:25:59.557 INFO:tasks.workunit.client.0.smithi112.stdout:4/425: rename d6/db/c4f to d6/db/d1f/d41/c9a 0 2024-09-16T00:25:59.557 INFO:tasks.workunit.client.0.smithi112.stdout:4/426: chown d6/db/df/d5e/f78 10567 1 2024-09-16T00:25:59.558 INFO:tasks.workunit.client.0.smithi112.stdout:4/427: dread - d6/db/df/d3c/f73 zero size 2024-09-16T00:25:59.559 INFO:tasks.workunit.client.0.smithi112.stdout:4/428: mknod d6/db/d1f/d41/d49/d31/c9b 0 2024-09-16T00:25:59.560 INFO:tasks.workunit.client.0.smithi112.stdout:4/429: symlink d6/d39/d36/d5c/l9c 0 2024-09-16T00:25:59.563 INFO:tasks.workunit.client.0.smithi112.stdout:4/430: rename d6/d39/c8d to d6/c9d 0 2024-09-16T00:25:59.563 INFO:tasks.workunit.client.0.smithi112.stdout:4/431: write d6/db/f25 [945839,107201] 0 2024-09-16T00:25:59.564 INFO:tasks.workunit.client.0.smithi112.stdout:4/432: readlink d6/d39/l8b 0 2024-09-16T00:25:59.565 INFO:tasks.workunit.client.0.smithi112.stdout:4/433: creat d6/db/df/d5e/d61/d64/f9e x:0 0 0 2024-09-16T00:25:59.566 INFO:tasks.workunit.client.0.smithi112.stdout:4/434: truncate d6/d39/d36/d5c/d7e/f86 643011 0 2024-09-16T00:25:59.587 INFO:tasks.workunit.client.0.smithi112.stdout:8/375: sync 2024-09-16T00:25:59.587 INFO:tasks.workunit.client.0.smithi112.stdout:3/495: sync 2024-09-16T00:25:59.589 INFO:tasks.workunit.client.0.smithi112.stdout:8/376: chown d10/l4b 80072 1 2024-09-16T00:25:59.589 INFO:tasks.workunit.client.0.smithi112.stdout:3/496: creat d2/d9/d2b/d47/fac x:0 0 0 2024-09-16T00:25:59.590 INFO:tasks.workunit.client.0.smithi112.stdout:3/497: fsync d2/f2d 0 2024-09-16T00:25:59.590 INFO:tasks.workunit.client.0.smithi112.stdout:8/377: creat d10/d12/d1c/d3b/d74/f7b x:0 0 0 2024-09-16T00:25:59.660 INFO:tasks.workunit.client.1.smithi169.stdout:6/268: truncate d2/d4d/f49 1153957 0 2024-09-16T00:25:59.661 INFO:tasks.workunit.client.0.smithi112.stdout:6/438: truncate dd/df/d17/f77 3562384 0 2024-09-16T00:25:59.662 INFO:tasks.workunit.client.0.smithi112.stdout:4/435: getdents d6/d39/d1c/d6e 0 2024-09-16T00:25:59.662 INFO:tasks.workunit.client.1.smithi169.stdout:6/269: getdents d2/d4c 0 2024-09-16T00:25:59.664 INFO:tasks.workunit.client.0.smithi112.stdout:6/439: mknod dd/df/d10/d47/c94 0 2024-09-16T00:25:59.665 INFO:tasks.workunit.client.0.smithi112.stdout:6/440: fdatasync dd/df/d10/d47/d55/f78 0 2024-09-16T00:25:59.666 INFO:tasks.workunit.client.1.smithi169.stdout:6/270: rmdir d2/d4d 39 2024-09-16T00:25:59.666 INFO:tasks.workunit.client.0.smithi112.stdout:4/436: rename d6/d39/l8b to d6/db/df/d5e/d61/d64/d6a/l9f 0 2024-09-16T00:25:59.667 INFO:tasks.workunit.client.0.smithi112.stdout:4/437: dread - d6/db/df/d5e/d61/d6d/f7f zero size 2024-09-16T00:25:59.668 INFO:tasks.workunit.client.0.smithi112.stdout:4/438: readlink d6/d39/d1c/d66/l89 0 2024-09-16T00:25:59.669 INFO:tasks.workunit.client.0.smithi112.stdout:6/441: dread - dd/df/d13/f48 zero size 2024-09-16T00:25:59.670 INFO:tasks.workunit.client.1.smithi169.stdout:6/271: rename d2/f17 to d2/d4c/f5d 0 2024-09-16T00:25:59.671 INFO:tasks.workunit.client.0.smithi112.stdout:4/439: mkdir d6/d7c/da0 0 2024-09-16T00:25:59.672 INFO:tasks.workunit.client.0.smithi112.stdout:4/440: chown d6/d39/d1c/d2c/d69 40 1 2024-09-16T00:25:59.674 INFO:tasks.workunit.client.0.smithi112.stdout:8/378: write d10/d12/d1c/d61/d6c/f70 [223933,89878] 0 2024-09-16T00:25:59.675 INFO:tasks.workunit.client.0.smithi112.stdout:6/442: symlink dd/df/d13/d7c/d88/l95 0 2024-09-16T00:25:59.675 INFO:tasks.workunit.client.0.smithi112.stdout:4/441: readlink d6/d39/d36/d5c/d7e/l92 0 2024-09-16T00:25:59.676 INFO:tasks.workunit.client.0.smithi112.stdout:6/443: fdatasync dd/d23/f64 0 2024-09-16T00:25:59.676 INFO:tasks.workunit.client.0.smithi112.stdout:8/379: chown d10/d12/d1c/d61/d6c 106351439 1 2024-09-16T00:25:59.679 INFO:tasks.workunit.client.0.smithi112.stdout:8/380: mkdir d10/d12/d1c/d61/d6c/d71/d7c 0 2024-09-16T00:25:59.682 INFO:tasks.workunit.client.0.smithi112.stdout:8/381: mkdir d10/d12/d1c/d3b/d7d 0 2024-09-16T00:25:59.683 INFO:tasks.workunit.client.0.smithi112.stdout:6/444: dwrite dd/df/d17/f59 [0,4194304] 0 2024-09-16T00:25:59.689 INFO:tasks.workunit.client.0.smithi112.stdout:6/445: rename dd/df/d13/d18/c6d to dd/d15/d34/c96 0 2024-09-16T00:25:59.689 INFO:tasks.workunit.client.0.smithi112.stdout:8/382: link d10/d12/d1c/d3b/c6b d10/c7e 0 2024-09-16T00:25:59.691 INFO:tasks.workunit.client.0.smithi112.stdout:8/383: symlink d10/d4e/l7f 0 2024-09-16T00:25:59.691 INFO:tasks.workunit.client.0.smithi112.stdout:8/384: readlink d10/d4e/l7f 0 2024-09-16T00:25:59.692 INFO:tasks.workunit.client.0.smithi112.stdout:6/446: symlink dd/df/d10/l97 0 2024-09-16T00:25:59.692 INFO:tasks.workunit.client.0.smithi112.stdout:6/447: fsync dd/f2a 0 2024-09-16T00:25:59.693 INFO:tasks.workunit.client.0.smithi112.stdout:8/385: creat d10/d12/d1c/d75/f80 x:0 0 0 2024-09-16T00:25:59.693 INFO:tasks.workunit.client.0.smithi112.stdout:6/448: stat dd/df/d10/f3a 0 2024-09-16T00:25:59.694 INFO:tasks.workunit.client.0.smithi112.stdout:8/386: truncate d10/d12/d24/f2b 1918735 0 2024-09-16T00:25:59.695 INFO:tasks.workunit.client.0.smithi112.stdout:8/387: fdatasync d10/d12/d24/f6e 0 2024-09-16T00:25:59.696 INFO:tasks.workunit.client.0.smithi112.stdout:6/449: getdents dd/df/d10/d47/d55 0 2024-09-16T00:25:59.697 INFO:tasks.workunit.client.0.smithi112.stdout:8/388: write d10/d12/d1c/d61/d6c/f70 [77685,14290] 0 2024-09-16T00:25:59.697 INFO:tasks.workunit.client.0.smithi112.stdout:8/389: readlink d10/l35 0 2024-09-16T00:25:59.700 INFO:tasks.workunit.client.0.smithi112.stdout:6/450: creat dd/df/d13/d7c/f98 x:0 0 0 2024-09-16T00:25:59.707 INFO:tasks.workunit.client.0.smithi112.stdout:8/390: dread fe [0,4194304] 0 2024-09-16T00:25:59.707 INFO:tasks.workunit.client.0.smithi112.stdout:8/391: chown d10/l4b 10685947 1 2024-09-16T00:25:59.708 INFO:tasks.workunit.client.0.smithi112.stdout:8/392: write d10/f1f [2237996,119427] 0 2024-09-16T00:25:59.713 INFO:tasks.workunit.client.0.smithi112.stdout:8/393: dwrite d10/d12/d1c/f31 [4194304,4194304] 0 2024-09-16T00:25:59.713 INFO:tasks.workunit.client.0.smithi112.stdout:8/394: dread - d10/d4e/f72 zero size 2024-09-16T00:25:59.716 INFO:tasks.workunit.client.0.smithi112.stdout:8/395: link d10/d12/d24/l27 d10/d12/d1c/d75/l81 0 2024-09-16T00:25:59.717 INFO:tasks.workunit.client.0.smithi112.stdout:8/396: creat d10/d12/d24/f82 x:0 0 0 2024-09-16T00:25:59.720 INFO:tasks.workunit.client.1.smithi169.stdout:6/272: dwrite d2/d4c/f5d [0,4194304] 0 2024-09-16T00:25:59.722 INFO:tasks.workunit.client.1.smithi169.stdout:6/273: rmdir d2/d22 39 2024-09-16T00:25:59.726 INFO:tasks.workunit.client.1.smithi169.stdout:6/274: symlink d2/d22/l5e 0 2024-09-16T00:25:59.726 INFO:tasks.workunit.client.0.smithi112.stdout:8/397: dread d10/d12/f77 [0,4194304] 0 2024-09-16T00:25:59.731 INFO:tasks.workunit.client.0.smithi112.stdout:8/398: dwrite d10/d12/d24/f6e [0,4194304] 0 2024-09-16T00:25:59.735 INFO:tasks.workunit.client.0.smithi112.stdout:8/399: getdents d10/d12/d1c/d61 0 2024-09-16T00:25:59.737 INFO:tasks.workunit.client.0.smithi112.stdout:8/400: rmdir d10/d12/d1c 39 2024-09-16T00:25:59.741 INFO:tasks.workunit.client.0.smithi112.stdout:8/401: rename d10/d12/d1c/c5c to d10/d12/d1c/d61/d6c/c83 0 2024-09-16T00:25:59.741 INFO:tasks.workunit.client.0.smithi112.stdout:8/402: fsync d10/d12/d1c/d3b/d74/f79 0 2024-09-16T00:25:59.742 INFO:tasks.workunit.client.0.smithi112.stdout:8/403: stat d10/d4e/f6f 0 2024-09-16T00:25:59.742 INFO:tasks.workunit.client.0.smithi112.stdout:8/404: chown d10/d12/d1c/d75/f78 399 1 2024-09-16T00:25:59.745 INFO:tasks.workunit.client.0.smithi112.stdout:8/405: creat d10/d12/d1c/d61/f84 x:0 0 0 2024-09-16T00:25:59.745 INFO:tasks.workunit.client.0.smithi112.stdout:8/406: readlink d10/l11 0 2024-09-16T00:25:59.745 INFO:tasks.workunit.client.0.smithi112.stdout:8/407: dread - d10/d4e/f68 zero size 2024-09-16T00:25:59.749 INFO:tasks.workunit.client.0.smithi112.stdout:8/408: creat d10/d12/d1c/d61/d6c/d71/f85 x:0 0 0 2024-09-16T00:25:59.749 INFO:tasks.workunit.client.0.smithi112.stdout:8/409: write d10/d4e/f68 [714027,79259] 0 2024-09-16T00:25:59.750 INFO:tasks.workunit.client.0.smithi112.stdout:8/410: rename d10/c3d to d10/d12/d1c/d61/d6c/c86 0 2024-09-16T00:25:59.755 INFO:tasks.workunit.client.0.smithi112.stdout:9/657: sync 2024-09-16T00:25:59.755 INFO:tasks.workunit.client.0.smithi112.stdout:0/530: sync 2024-09-16T00:25:59.756 INFO:tasks.workunit.client.0.smithi112.stdout:9/658: write d1/d5/d6/d2d/fbc [876971,120607] 0 2024-09-16T00:25:59.760 INFO:tasks.workunit.client.0.smithi112.stdout:9/659: mkdir d1/d5/d20/d25/de9 0 2024-09-16T00:25:59.767 INFO:tasks.workunit.client.0.smithi112.stdout:9/660: dwrite d1/d5/d6/d18/fc5 [0,4194304] 0 2024-09-16T00:25:59.768 INFO:tasks.workunit.client.0.smithi112.stdout:0/531: truncate d5/da/d18/d24/d46/d5f/da6/f50 2003359 0 2024-09-16T00:25:59.770 INFO:tasks.workunit.client.0.smithi112.stdout:0/532: chown d5/da/d18/c26 617 1 2024-09-16T00:25:59.771 INFO:tasks.workunit.client.0.smithi112.stdout:0/533: read - d5/da/d18/d63/fa3 zero size 2024-09-16T00:25:59.775 INFO:tasks.workunit.client.0.smithi112.stdout:0/534: rename d5/da/f6a to d5/da/d21/da5/faa 0 2024-09-16T00:25:59.777 INFO:tasks.workunit.client.0.smithi112.stdout:9/661: creat d1/d5/d6/d2d/fea x:0 0 0 2024-09-16T00:25:59.778 INFO:tasks.workunit.client.0.smithi112.stdout:0/535: creat d5/da/d18/d89/fab x:0 0 0 2024-09-16T00:25:59.780 INFO:tasks.workunit.client.0.smithi112.stdout:9/662: mkdir d1/d5/d6/d2d/deb 0 2024-09-16T00:25:59.786 INFO:tasks.workunit.client.0.smithi112.stdout:9/663: dwrite d1/d5/d20/d81/fb6 [0,4194304] 0 2024-09-16T00:25:59.789 INFO:tasks.workunit.client.0.smithi112.stdout:9/664: creat d1/d5/d6/d59/fec x:0 0 0 2024-09-16T00:25:59.791 INFO:tasks.workunit.client.0.smithi112.stdout:9/665: unlink d1/d5/d2c/c70 0 2024-09-16T00:25:59.793 INFO:tasks.workunit.client.0.smithi112.stdout:0/536: truncate d5/da/d57/d76/d74/f79 1704319 0 2024-09-16T00:25:59.795 INFO:tasks.workunit.client.0.smithi112.stdout:0/537: mknod d5/da/d57/cac 0 2024-09-16T00:25:59.795 INFO:tasks.workunit.client.0.smithi112.stdout:0/538: chown d5/da/d18/d89 841 1 2024-09-16T00:25:59.798 INFO:tasks.workunit.client.0.smithi112.stdout:0/539: rename d5/da/d66/f69 to d5/da/d18/d63/fad 0 2024-09-16T00:25:59.799 INFO:tasks.workunit.client.0.smithi112.stdout:0/540: write d5/ff [13971,30074] 0 2024-09-16T00:25:59.803 INFO:tasks.workunit.client.0.smithi112.stdout:0/541: rmdir d5/da/d18/d24/d96 0 2024-09-16T00:25:59.804 INFO:tasks.workunit.client.0.smithi112.stdout:0/542: mknod d5/da/d1d/d71/d64/d8f/cae 0 2024-09-16T00:25:59.810 INFO:tasks.workunit.client.0.smithi112.stdout:9/666: dread d1/d5/d6/d2d/f3e [0,4194304] 0 2024-09-16T00:25:59.811 INFO:tasks.workunit.client.0.smithi112.stdout:9/667: truncate d1/d5/d6/d38/f5a 727538 0 2024-09-16T00:25:59.843 INFO:tasks.workunit.client.1.smithi169.stdout:7/360: sync 2024-09-16T00:25:59.843 INFO:tasks.workunit.client.1.smithi169.stdout:2/223: sync 2024-09-16T00:25:59.844 INFO:tasks.workunit.client.1.smithi169.stdout:3/315: sync 2024-09-16T00:25:59.844 INFO:tasks.workunit.client.1.smithi169.stdout:2/224: chown d7/db/d20/d21/f4a 198 1 2024-09-16T00:25:59.845 INFO:tasks.workunit.client.1.smithi169.stdout:7/361: mkdir d7/d16/d49/d74 0 2024-09-16T00:25:59.846 INFO:tasks.workunit.client.1.smithi169.stdout:3/316: truncate d7/fd 1286990 0 2024-09-16T00:25:59.850 INFO:tasks.workunit.client.1.smithi169.stdout:3/317: link d7/f14 d7/d1a/d1c/f78 0 2024-09-16T00:25:59.852 INFO:tasks.workunit.client.1.smithi169.stdout:7/362: dwrite d7/f50 [0,4194304] 0 2024-09-16T00:25:59.853 INFO:tasks.workunit.client.1.smithi169.stdout:3/318: creat d7/d50/f79 x:0 0 0 2024-09-16T00:25:59.858 INFO:tasks.workunit.client.1.smithi169.stdout:3/319: unlink d7/d1a/d1c/d35/d63/f5f 0 2024-09-16T00:25:59.860 INFO:tasks.workunit.client.1.smithi169.stdout:7/363: rename d7/d16/f39 to d7/d16/f75 0 2024-09-16T00:25:59.861 INFO:tasks.workunit.client.1.smithi169.stdout:2/225: truncate d7/db/f19 5133901 0 2024-09-16T00:25:59.862 INFO:tasks.workunit.client.1.smithi169.stdout:2/226: write d7/db/f45 [5432366,25188] 0 2024-09-16T00:25:59.863 INFO:tasks.workunit.client.1.smithi169.stdout:3/320: creat d7/d1a/d1c/d6c/f7a x:0 0 0 2024-09-16T00:25:59.864 INFO:tasks.workunit.client.1.smithi169.stdout:3/321: write d7/d50/d6f/f75 [929708,34171] 0 2024-09-16T00:25:59.865 INFO:tasks.workunit.client.1.smithi169.stdout:7/364: rename d7/da to d7/d71/d76 0 2024-09-16T00:25:59.867 INFO:tasks.workunit.client.1.smithi169.stdout:2/227: mknod d7/db/d27/c4b 0 2024-09-16T00:25:59.867 INFO:tasks.workunit.client.1.smithi169.stdout:2/228: fsync f0 0 2024-09-16T00:25:59.867 INFO:tasks.workunit.client.1.smithi169.stdout:3/322: creat d7/d50/f7b x:0 0 0 2024-09-16T00:25:59.869 INFO:tasks.workunit.client.1.smithi169.stdout:2/229: dread d7/d1c/d26/f2c [0,4194304] 0 2024-09-16T00:25:59.869 INFO:tasks.workunit.client.1.smithi169.stdout:7/365: rename d7/d71/d76/d3a/f6f to d7/d5b/f77 0 2024-09-16T00:25:59.870 INFO:tasks.workunit.client.1.smithi169.stdout:7/366: chown d7/d5b 29313 1 2024-09-16T00:25:59.873 INFO:tasks.workunit.client.1.smithi169.stdout:3/323: rename d7/d1a/d1c/d23/f3c to d7/d1a/d1c/d6c/f7c 0 2024-09-16T00:25:59.877 INFO:tasks.workunit.client.0.smithi112.stdout:1/413: sync 2024-09-16T00:25:59.878 INFO:tasks.workunit.client.1.smithi169.stdout:3/324: truncate d7/d1a/d1c/f4f 1500076 0 2024-09-16T00:25:59.880 INFO:tasks.workunit.client.1.smithi169.stdout:2/230: dwrite d7/d1c/f3e [0,4194304] 0 2024-09-16T00:25:59.880 INFO:tasks.workunit.client.0.smithi112.stdout:1/414: rename d0/f27 to d0/d14/d44/f7e 0 2024-09-16T00:25:59.880 INFO:tasks.workunit.client.0.smithi112.stdout:1/415: fdatasync d0/d14/d1c/d20/f21 0 2024-09-16T00:25:59.884 INFO:tasks.workunit.client.0.smithi112.stdout:1/416: link d0/c13 d0/d14/df/d28/d45/c7f 0 2024-09-16T00:25:59.884 INFO:tasks.workunit.client.1.smithi169.stdout:7/367: dwrite d7/d16/d49/f72 [0,4194304] 0 2024-09-16T00:25:59.884 INFO:tasks.workunit.client.0.smithi112.stdout:1/417: fsync d0/f18 0 2024-09-16T00:25:59.887 INFO:tasks.workunit.client.0.smithi112.stdout:1/418: creat d0/d14/d6a/f80 x:0 0 0 2024-09-16T00:25:59.887 INFO:tasks.workunit.client.1.smithi169.stdout:7/368: write d7/d71/d76/f23 [100618,115800] 0 2024-09-16T00:25:59.888 INFO:tasks.workunit.client.0.smithi112.stdout:1/419: write d0/f9 [7220899,81082] 0 2024-09-16T00:25:59.892 INFO:tasks.workunit.client.1.smithi169.stdout:7/369: chown d7/c28 3 1 2024-09-16T00:25:59.892 INFO:tasks.workunit.client.1.smithi169.stdout:3/325: dwrite d7/d1a/d2b/f69 [0,4194304] 0 2024-09-16T00:25:59.893 INFO:tasks.workunit.client.0.smithi112.stdout:1/420: link d0/f1e d0/d14/df/d28/d42/f81 0 2024-09-16T00:25:59.896 INFO:tasks.workunit.client.1.smithi169.stdout:3/326: read d7/d1a/d1c/d35/d63/f53 [174136,23408] 0 2024-09-16T00:25:59.897 INFO:tasks.workunit.client.1.smithi169.stdout:3/327: stat d7/d1a/d1c/d23/f4c 0 2024-09-16T00:25:59.899 INFO:tasks.workunit.client.1.smithi169.stdout:7/370: mkdir d7/d71/d76/d29/d78 0 2024-09-16T00:25:59.899 INFO:tasks.workunit.client.0.smithi112.stdout:1/421: dwrite d0/d14/d6a/f6f [0,4194304] 0 2024-09-16T00:25:59.900 INFO:tasks.workunit.client.1.smithi169.stdout:2/231: dread d7/db/d20/d21/f4a [0,4194304] 0 2024-09-16T00:25:59.902 INFO:tasks.workunit.client.1.smithi169.stdout:7/371: read d7/d5b/f77 [842518,3461] 0 2024-09-16T00:25:59.903 INFO:tasks.workunit.client.1.smithi169.stdout:2/232: read f3 [3573288,53430] 0 2024-09-16T00:25:59.909 INFO:tasks.workunit.client.1.smithi169.stdout:3/328: link d7/d1a/d1c/d23/l4e d7/d1a/d1c/d23/l7d 0 2024-09-16T00:25:59.909 INFO:tasks.workunit.client.1.smithi169.stdout:2/233: mkdir d7/db/d20/d4c 0 2024-09-16T00:25:59.912 INFO:tasks.workunit.client.1.smithi169.stdout:7/372: dwrite d7/d5b/d61/f6e [0,4194304] 0 2024-09-16T00:25:59.914 INFO:tasks.workunit.client.1.smithi169.stdout:2/234: truncate d7/db/d20/f3b 619770 0 2024-09-16T00:25:59.915 INFO:tasks.workunit.client.1.smithi169.stdout:2/235: read - d7/db/d20/f44 zero size 2024-09-16T00:25:59.916 INFO:tasks.workunit.client.1.smithi169.stdout:3/329: mknod d7/d1a/d1c/d35/d63/c7e 0 2024-09-16T00:25:59.917 INFO:tasks.workunit.client.0.smithi112.stdout:1/422: truncate d0/d14/df/d28/d45/d30/f4a 4206008 0 2024-09-16T00:25:59.918 INFO:tasks.workunit.client.1.smithi169.stdout:1/251: sync 2024-09-16T00:25:59.918 INFO:tasks.workunit.client.1.smithi169.stdout:4/300: sync 2024-09-16T00:25:59.918 INFO:tasks.workunit.client.1.smithi169.stdout:7/373: rename d7/f2c to d7/d16/f79 0 2024-09-16T00:25:59.919 INFO:tasks.workunit.client.0.smithi112.stdout:1/423: symlink d0/d14/d5e/l82 0 2024-09-16T00:25:59.919 INFO:tasks.workunit.client.1.smithi169.stdout:7/374: read d7/d5b/f77 [5233289,50759] 0 2024-09-16T00:25:59.919 INFO:tasks.workunit.client.1.smithi169.stdout:1/252: fdatasync dc/d2c/f32 0 2024-09-16T00:25:59.921 INFO:tasks.workunit.client.1.smithi169.stdout:2/236: creat d7/db/d27/f4d x:0 0 0 2024-09-16T00:25:59.921 INFO:tasks.workunit.client.1.smithi169.stdout:3/330: truncate d7/d1a/d1c/d35/d63/f53 948232 0 2024-09-16T00:25:59.927 INFO:tasks.workunit.client.1.smithi169.stdout:4/301: mkdir d2/d12/d13/d5c 0 2024-09-16T00:25:59.929 INFO:tasks.workunit.client.1.smithi169.stdout:2/237: mknod d7/c4e 0 2024-09-16T00:25:59.932 INFO:tasks.workunit.client.1.smithi169.stdout:1/253: getdents dc/d18 0 2024-09-16T00:25:59.933 INFO:tasks.workunit.client.1.smithi169.stdout:4/302: creat d2/d12/f5d x:0 0 0 2024-09-16T00:25:59.934 INFO:tasks.workunit.client.1.smithi169.stdout:2/238: truncate d7/db/d20/d21/f4a 2102748 0 2024-09-16T00:25:59.935 INFO:tasks.workunit.client.0.smithi112.stdout:1/424: dread d0/f9 [4194304,4194304] 0 2024-09-16T00:25:59.935 INFO:tasks.workunit.client.1.smithi169.stdout:3/331: dwrite d7/d1a/d1c/d6c/f7c [0,4194304] 0 2024-09-16T00:25:59.936 INFO:tasks.workunit.client.1.smithi169.stdout:4/303: readlink d2/d12/d13/d1f/d3e/d5b/d1a/l22 0 2024-09-16T00:25:59.937 INFO:tasks.workunit.client.1.smithi169.stdout:2/239: dread d7/db/d20/f3b [0,4194304] 0 2024-09-16T00:25:59.937 INFO:tasks.workunit.client.0.smithi112.stdout:1/425: creat d0/d14/df/d28/d42/f83 x:0 0 0 2024-09-16T00:25:59.938 INFO:tasks.workunit.client.1.smithi169.stdout:3/332: chown d7/l1f 29321810 1 2024-09-16T00:25:59.939 INFO:tasks.workunit.client.1.smithi169.stdout:1/254: read dc/d2c/d3d/f3f [1726841,115413] 0 2024-09-16T00:25:59.939 INFO:tasks.workunit.client.1.smithi169.stdout:4/304: symlink d2/d12/d13/d1f/d3e/d5b/d1a/l5e 0 2024-09-16T00:25:59.940 INFO:tasks.workunit.client.1.smithi169.stdout:4/305: fsync d2/f3 0 2024-09-16T00:25:59.944 INFO:tasks.workunit.client.1.smithi169.stdout:1/255: symlink dc/d36/d39/l4c 0 2024-09-16T00:25:59.944 INFO:tasks.workunit.client.1.smithi169.stdout:4/306: dread - d2/d12/d13/d1f/d3e/d5b/d1a/f47 zero size 2024-09-16T00:25:59.945 INFO:tasks.workunit.client.1.smithi169.stdout:7/375: rmdir d7/d16 39 2024-09-16T00:25:59.948 INFO:tasks.workunit.client.1.smithi169.stdout:3/333: dwrite d7/d1a/d2b/f46 [0,4194304] 0 2024-09-16T00:25:59.949 INFO:tasks.workunit.client.1.smithi169.stdout:3/334: fsync d7/d1a/d1c/f25 0 2024-09-16T00:25:59.951 INFO:tasks.workunit.client.1.smithi169.stdout:3/335: readlink d7/d1a/d1c/d35/d63/d37/l47 0 2024-09-16T00:25:59.952 INFO:tasks.workunit.client.1.smithi169.stdout:1/256: dread dc/d18/f3b [0,4194304] 0 2024-09-16T00:25:59.953 INFO:tasks.workunit.client.1.smithi169.stdout:3/336: mknod d7/d1a/d1c/d23/c7f 0 2024-09-16T00:25:59.954 INFO:tasks.workunit.client.1.smithi169.stdout:3/337: readlink d7/d1a/d1c/l2e 0 2024-09-16T00:25:59.956 INFO:tasks.workunit.client.1.smithi169.stdout:3/338: fdatasync d7/d50/d6f/f75 0 2024-09-16T00:25:59.957 INFO:tasks.workunit.client.1.smithi169.stdout:1/257: mkdir dc/d36/d3a/d45/d29/d4d 0 2024-09-16T00:25:59.957 INFO:tasks.workunit.client.1.smithi169.stdout:3/339: dread - d7/d50/f79 zero size 2024-09-16T00:25:59.958 INFO:tasks.workunit.client.1.smithi169.stdout:8/285: sync 2024-09-16T00:25:59.958 INFO:tasks.workunit.client.1.smithi169.stdout:0/293: sync 2024-09-16T00:25:59.958 INFO:tasks.workunit.client.1.smithi169.stdout:6/275: sync 2024-09-16T00:25:59.958 INFO:tasks.workunit.client.1.smithi169.stdout:9/354: sync 2024-09-16T00:25:59.958 INFO:tasks.workunit.client.1.smithi169.stdout:5/253: sync 2024-09-16T00:25:59.958 INFO:tasks.workunit.client.1.smithi169.stdout:1/258: mkdir dc/d36/d4e 0 2024-09-16T00:25:59.962 INFO:tasks.workunit.client.0.smithi112.stdout:1/426: dwrite d0/d14/df/d28/d45/d30/f4a [4194304,4194304] 0 2024-09-16T00:25:59.962 INFO:tasks.workunit.client.1.smithi169.stdout:7/376: write d7/d5b/f77 [950747,53046] 0 2024-09-16T00:25:59.963 INFO:tasks.workunit.client.1.smithi169.stdout:5/254: read d2/ff [1741172,27481] 0 2024-09-16T00:25:59.963 INFO:tasks.workunit.client.1.smithi169.stdout:1/259: rmdir dc/d2c 39 2024-09-16T00:25:59.964 INFO:tasks.workunit.client.1.smithi169.stdout:9/355: truncate d5/da/f7f 739056 0 2024-09-16T00:25:59.965 INFO:tasks.workunit.client.0.smithi112.stdout:1/427: creat d0/d14/d1c/d26/d35/f84 x:0 0 0 2024-09-16T00:25:59.965 INFO:tasks.workunit.client.1.smithi169.stdout:6/276: dread - d2/d4d/d59/d38/f5a zero size 2024-09-16T00:25:59.966 INFO:tasks.workunit.client.1.smithi169.stdout:0/294: read d0/d2/f6 [326408,59676] 0 2024-09-16T00:25:59.966 INFO:tasks.workunit.client.1.smithi169.stdout:8/286: mknod d7/dc/d26/c5b 0 2024-09-16T00:25:59.967 INFO:tasks.workunit.client.0.smithi112.stdout:1/428: mkdir d0/d14/d1c/d26/d47/d85 0 2024-09-16T00:25:59.967 INFO:tasks.workunit.client.1.smithi169.stdout:1/260: chown dc/d36/d39 129 1 2024-09-16T00:25:59.969 INFO:tasks.workunit.client.0.smithi112.stdout:1/429: read d0/f9 [167169,60805] 0 2024-09-16T00:25:59.971 INFO:tasks.workunit.client.1.smithi169.stdout:9/356: fdatasync d5/d12/f5d 0 2024-09-16T00:25:59.974 INFO:tasks.workunit.client.1.smithi169.stdout:0/295: read d0/d2/d8/f37 [11323,120071] 0 2024-09-16T00:25:59.974 INFO:tasks.workunit.client.0.smithi112.stdout:1/430: dwrite d0/d14/df/d28/d45/d30/f4a [8388608,4194304] 0 2024-09-16T00:25:59.976 INFO:tasks.workunit.client.1.smithi169.stdout:1/261: creat dc/d36/d4e/f4f x:0 0 0 2024-09-16T00:25:59.976 INFO:tasks.workunit.client.0.smithi112.stdout:1/431: symlink d0/d14/d6a/l86 0 2024-09-16T00:25:59.978 INFO:tasks.workunit.client.1.smithi169.stdout:3/340: dwrite d7/d1a/d1c/d6c/f7c [0,4194304] 0 2024-09-16T00:25:59.978 INFO:tasks.workunit.client.1.smithi169.stdout:6/277: mkdir d2/d36/d48/d5f 0 2024-09-16T00:25:59.979 INFO:tasks.workunit.client.0.smithi112.stdout:1/432: creat d0/f87 x:0 0 0 2024-09-16T00:25:59.980 INFO:tasks.workunit.client.1.smithi169.stdout:2/240: dwrite d7/db/f23 [0,4194304] 0 2024-09-16T00:25:59.984 INFO:tasks.workunit.client.0.smithi112.stdout:1/433: dwrite d0/d14/d1c/d26/d35/f59 [0,4194304] 0 2024-09-16T00:25:59.985 INFO:tasks.workunit.client.1.smithi169.stdout:2/241: chown d7/d1c/c1f 470 1 2024-09-16T00:25:59.986 INFO:tasks.workunit.client.1.smithi169.stdout:3/341: truncate d7/f68 73842 0 2024-09-16T00:25:59.988 INFO:tasks.workunit.client.1.smithi169.stdout:5/255: dread d2/da/f28 [0,4194304] 0 2024-09-16T00:25:59.989 INFO:tasks.workunit.client.1.smithi169.stdout:7/377: link d7/d5b/f6b d7/d71/d76/d3a/f7a 0 2024-09-16T00:25:59.989 INFO:tasks.workunit.client.1.smithi169.stdout:0/296: chown d0/d2c/d54/d21/l71 26881 1 2024-09-16T00:25:59.990 INFO:tasks.workunit.client.1.smithi169.stdout:6/278: mkdir d2/d3f/d50/d60 0 2024-09-16T00:25:59.990 INFO:tasks.workunit.client.0.smithi112.stdout:1/434: creat d0/d14/df/d28/d45/d46/d58/d62/f88 x:0 0 0 2024-09-16T00:25:59.992 INFO:tasks.workunit.client.0.smithi112.stdout:5/387: sync 2024-09-16T00:25:59.993 INFO:tasks.workunit.client.1.smithi169.stdout:9/357: rename d5/da/f28 to d5/d12/f86 0 2024-09-16T00:25:59.993 INFO:tasks.workunit.client.0.smithi112.stdout:1/435: rename d0/d14/df/d28/d45/d46/d6b to d0/d14/df/d28/d89 0 2024-09-16T00:25:59.994 INFO:tasks.workunit.client.0.smithi112.stdout:5/388: mkdir d4/d7/d87 0 2024-09-16T00:25:59.994 INFO:tasks.workunit.client.1.smithi169.stdout:6/279: fdatasync d2/d36/f31 0 2024-09-16T00:25:59.996 INFO:tasks.workunit.client.0.smithi112.stdout:1/436: rename d0/d14/lb to d0/d14/d44/l8a 0 2024-09-16T00:25:59.996 INFO:tasks.workunit.client.1.smithi169.stdout:7/378: symlink d7/d71/d76/d3a/l7b 0 2024-09-16T00:25:59.998 INFO:tasks.workunit.client.0.smithi112.stdout:5/389: creat d4/d7/d26/d3e/d7c/f88 x:0 0 0 2024-09-16T00:26:00.000 INFO:tasks.workunit.client.0.smithi112.stdout:1/437: symlink d0/d14/d1c/d26/l8b 0 2024-09-16T00:26:00.001 INFO:tasks.workunit.client.0.smithi112.stdout:5/390: write d4/dd/d12/d52/f63 [1294628,66852] 0 2024-09-16T00:26:00.002 INFO:tasks.workunit.client.1.smithi169.stdout:6/280: fdatasync d2/d4d/f57 0 2024-09-16T00:26:00.003 INFO:tasks.workunit.client.1.smithi169.stdout:9/358: fdatasync d5/da/d21/f2c 0 2024-09-16T00:26:00.003 INFO:tasks.workunit.client.0.smithi112.stdout:1/438: creat d0/d14/df/d28/d45/d30/f8c x:0 0 0 2024-09-16T00:26:00.005 INFO:tasks.workunit.client.1.smithi169.stdout:3/342: rename d7/d1a/d1c/d35/d63/d37/l41 to d7/d50/d6f/l80 0 2024-09-16T00:26:00.006 INFO:tasks.workunit.client.1.smithi169.stdout:8/287: write d7/dc/d17/d48/f5a [1910483,105317] 0 2024-09-16T00:26:00.007 INFO:tasks.workunit.client.1.smithi169.stdout:3/343: chown d7/d1a/d1c/d35/f49 186024491 1 2024-09-16T00:26:00.007 INFO:tasks.workunit.client.1.smithi169.stdout:6/281: symlink d2/d3f/d50/l61 0 2024-09-16T00:26:00.008 INFO:tasks.workunit.client.0.smithi112.stdout:1/439: rename d0/d14/f23 to d0/d14/d1c/d26/f8d 0 2024-09-16T00:26:00.008 INFO:tasks.workunit.client.1.smithi169.stdout:2/242: dread f2 [0,4194304] 0 2024-09-16T00:26:00.009 INFO:tasks.workunit.client.0.smithi112.stdout:1/440: rmdir d0/d14/d1c/d26 39 2024-09-16T00:26:00.009 INFO:tasks.workunit.client.1.smithi169.stdout:9/359: read d5/da/d4e/f6d [2956996,113582] 0 2024-09-16T00:26:00.010 INFO:tasks.workunit.client.1.smithi169.stdout:5/256: creat d2/f58 x:0 0 0 2024-09-16T00:26:00.010 INFO:tasks.workunit.client.1.smithi169.stdout:8/288: fsync d7/dc/d17/f39 0 2024-09-16T00:26:00.011 INFO:tasks.workunit.client.1.smithi169.stdout:3/344: creat d7/d1a/d1c/d35/d63/d37/f81 x:0 0 0 2024-09-16T00:26:00.012 INFO:tasks.workunit.client.1.smithi169.stdout:7/379: creat d7/d16/d49/d74/f7c x:0 0 0 2024-09-16T00:26:00.012 INFO:tasks.workunit.client.0.smithi112.stdout:1/441: link d0/d14/d1c/c1f d0/d14/df/d28/d42/c8e 0 2024-09-16T00:26:00.013 INFO:tasks.workunit.client.1.smithi169.stdout:6/282: creat d2/d4d/d59/d38/f62 x:0 0 0 2024-09-16T00:26:00.013 INFO:tasks.workunit.client.0.smithi112.stdout:1/442: write d0/d14/df/d28/f2e [3581578,52573] 0 2024-09-16T00:26:00.014 INFO:tasks.workunit.client.1.smithi169.stdout:7/380: fdatasync d7/d5b/f6c 0 2024-09-16T00:26:00.017 INFO:tasks.workunit.client.1.smithi169.stdout:8/289: creat d7/dc/d26/d41/f5c x:0 0 0 2024-09-16T00:26:00.017 INFO:tasks.workunit.client.1.smithi169.stdout:3/345: mknod d7/d1a/d1c/d35/d63/c82 0 2024-09-16T00:26:00.017 INFO:tasks.workunit.client.1.smithi169.stdout:2/243: dread d7/d1c/d26/f2c [0,4194304] 0 2024-09-16T00:26:00.018 INFO:tasks.workunit.client.0.smithi112.stdout:5/391: truncate d4/dd/d12/f2f 19399 0 2024-09-16T00:26:00.018 INFO:tasks.workunit.client.0.smithi112.stdout:5/392: chown d4/d1c 820578 1 2024-09-16T00:26:00.019 INFO:tasks.workunit.client.1.smithi169.stdout:9/360: unlink d5/da/d21/c29 0 2024-09-16T00:26:00.019 INFO:tasks.workunit.client.1.smithi169.stdout:8/290: truncate d7/f2e 781531 0 2024-09-16T00:26:00.021 INFO:tasks.workunit.client.1.smithi169.stdout:6/283: unlink d2/l14 0 2024-09-16T00:26:00.022 INFO:tasks.workunit.client.0.smithi112.stdout:1/443: truncate d0/d14/df/d28/d45/d30/f57 2243124 0 2024-09-16T00:26:00.024 INFO:tasks.workunit.client.0.smithi112.stdout:1/444: chown d0/d14/df/d28/f2d 174467747 1 2024-09-16T00:26:00.024 INFO:tasks.workunit.client.1.smithi169.stdout:6/284: write d2/d3f/d50/f5b [369005,115421] 0 2024-09-16T00:26:00.025 INFO:tasks.workunit.client.1.smithi169.stdout:7/381: mknod d7/d71/d76/d29/c7d 0 2024-09-16T00:26:00.026 INFO:tasks.workunit.client.0.smithi112.stdout:5/393: dwrite d4/dd/f6b [4194304,4194304] 0 2024-09-16T00:26:00.027 INFO:tasks.workunit.client.0.smithi112.stdout:1/445: chown d0/d14/d1c/d26/d47 3 1 2024-09-16T00:26:00.028 INFO:tasks.workunit.client.1.smithi169.stdout:9/361: creat d5/da/d4e/f87 x:0 0 0 2024-09-16T00:26:00.028 INFO:tasks.workunit.client.1.smithi169.stdout:0/297: dread d0/d2/d4b/f52 [0,4194304] 0 2024-09-16T00:26:00.029 INFO:tasks.workunit.client.1.smithi169.stdout:5/257: rename d2/da/d10/d13/d1c/d40/l4a to d2/da/d10/d13/d1c/d32/d53/d4e/l59 0 2024-09-16T00:26:00.033 INFO:tasks.workunit.client.0.smithi112.stdout:9/668: sync 2024-09-16T00:26:00.033 INFO:tasks.workunit.client.0.smithi112.stdout:4/442: sync 2024-09-16T00:26:00.034 INFO:tasks.workunit.client.0.smithi112.stdout:7/458: sync 2024-09-16T00:26:00.034 INFO:tasks.workunit.client.0.smithi112.stdout:3/498: sync 2024-09-16T00:26:00.034 INFO:tasks.workunit.client.0.smithi112.stdout:6/451: sync 2024-09-16T00:26:00.034 INFO:tasks.workunit.client.0.smithi112.stdout:2/510: sync 2024-09-16T00:26:00.034 INFO:tasks.workunit.client.0.smithi112.stdout:1/446: creat d0/d14/df/f8f x:0 0 0 2024-09-16T00:26:00.035 INFO:tasks.workunit.client.0.smithi112.stdout:0/543: sync 2024-09-16T00:26:00.036 INFO:tasks.workunit.client.0.smithi112.stdout:8/411: sync 2024-09-16T00:26:00.036 INFO:tasks.workunit.client.0.smithi112.stdout:9/669: write d1/d76/fdc [46518,29943] 0 2024-09-16T00:26:00.037 INFO:tasks.workunit.client.1.smithi169.stdout:3/346: dread d7/f14 [0,4194304] 0 2024-09-16T00:26:00.038 INFO:tasks.workunit.client.1.smithi169.stdout:8/291: rename d7/dc/d17/d48/f5a to d7/dc/d26/d46/f5d 0 2024-09-16T00:26:00.039 INFO:tasks.workunit.client.1.smithi169.stdout:5/258: dread d2/f1f [0,4194304] 0 2024-09-16T00:26:00.039 INFO:tasks.workunit.client.0.smithi112.stdout:3/499: read d2/d9/f1c [1199328,113444] 0 2024-09-16T00:26:00.039 INFO:tasks.workunit.client.0.smithi112.stdout:6/452: chown dd/df/d17/d27/d84/l1f 139 1 2024-09-16T00:26:00.039 INFO:tasks.workunit.client.0.smithi112.stdout:1/447: chown d0/d14/df/d28/d45/d46/c61 521586 1 2024-09-16T00:26:00.040 INFO:tasks.workunit.client.0.smithi112.stdout:3/500: readlink d2/d9/d2b/d47/l5c 0 2024-09-16T00:26:00.040 INFO:tasks.workunit.client.1.smithi169.stdout:8/292: stat d7/dc/d26/c2c 0 2024-09-16T00:26:00.041 INFO:tasks.workunit.client.0.smithi112.stdout:1/448: fsync d0/d14/df/d28/f2e 0 2024-09-16T00:26:00.041 INFO:tasks.workunit.client.0.smithi112.stdout:2/511: symlink d2/dd/d48/d94/d49/l9c 0 2024-09-16T00:26:00.041 INFO:tasks.workunit.client.1.smithi169.stdout:6/285: dwrite d2/fb [4194304,4194304] 0 2024-09-16T00:26:00.042 INFO:tasks.workunit.client.0.smithi112.stdout:9/670: mkdir d1/d5/d2c/ded 0 2024-09-16T00:26:00.043 INFO:tasks.workunit.client.1.smithi169.stdout:9/362: mknod d5/da/d2e/c88 0 2024-09-16T00:26:00.044 INFO:tasks.workunit.client.0.smithi112.stdout:2/512: chown d2/dd/d48/d22/d2b/f34 1081 1 2024-09-16T00:26:00.044 INFO:tasks.workunit.client.0.smithi112.stdout:8/412: mkdir d10/d4e/d58/d87 0 2024-09-16T00:26:00.045 INFO:tasks.workunit.client.1.smithi169.stdout:3/347: stat d7/d50/d6f/l80 0 2024-09-16T00:26:00.045 INFO:tasks.workunit.client.0.smithi112.stdout:9/671: truncate d1/d5/d6/d38/f7f 1413373 0 2024-09-16T00:26:00.045 INFO:tasks.workunit.client.0.smithi112.stdout:0/544: creat d5/da/d18/d24/faf x:0 0 0 2024-09-16T00:26:00.046 INFO:tasks.workunit.client.1.smithi169.stdout:7/382: creat d7/f7e x:0 0 0 2024-09-16T00:26:00.046 INFO:tasks.workunit.client.1.smithi169.stdout:5/259: symlink d2/da/d10/d13/d1c/d32/d53/l5a 0 2024-09-16T00:26:00.048 INFO:tasks.workunit.client.0.smithi112.stdout:1/449: unlink d0/f2 0 2024-09-16T00:26:00.050 INFO:tasks.workunit.client.1.smithi169.stdout:0/298: rename d0/d2/d8/f65 to d0/d2/d8/f73 0 2024-09-16T00:26:00.051 INFO:tasks.workunit.client.0.smithi112.stdout:4/443: link d6/d39/d1c/l74 d6/da/la1 0 2024-09-16T00:26:00.051 INFO:tasks.workunit.client.0.smithi112.stdout:8/413: stat d10/c7e 0 2024-09-16T00:26:00.053 INFO:tasks.workunit.client.1.smithi169.stdout:6/286: unlink d2/d36/c29 0 2024-09-16T00:26:00.054 INFO:tasks.workunit.client.1.smithi169.stdout:5/260: dread d2/da/d10/d13/f2f [0,4194304] 0 2024-09-16T00:26:00.054 INFO:tasks.workunit.client.1.smithi169.stdout:7/383: creat d7/d16/d49/d74/f7f x:0 0 0 2024-09-16T00:26:00.055 INFO:tasks.workunit.client.0.smithi112.stdout:2/513: rename d2/dd/l6b to d2/dd/d48/d22/l9d 0 2024-09-16T00:26:00.056 INFO:tasks.workunit.client.1.smithi169.stdout:7/384: fsync d7/d71/d76/f23 0 2024-09-16T00:26:00.056 INFO:tasks.workunit.client.0.smithi112.stdout:1/450: chown d0/d14/df/d28/d42/f83 10 1 2024-09-16T00:26:00.056 INFO:tasks.workunit.client.0.smithi112.stdout:6/453: creat dd/df/d13/f99 x:0 0 0 2024-09-16T00:26:00.058 INFO:tasks.workunit.client.1.smithi169.stdout:9/363: dwrite d5/d12/d61/f7a [0,4194304] 0 2024-09-16T00:26:00.058 INFO:tasks.workunit.client.1.smithi169.stdout:3/348: dread d7/d1a/d1c/d35/d63/f53 [0,4194304] 0 2024-09-16T00:26:00.058 INFO:tasks.workunit.client.0.smithi112.stdout:9/672: dwrite d1/d5/d6/d59/fec [0,4194304] 0 2024-09-16T00:26:00.059 INFO:tasks.workunit.client.0.smithi112.stdout:5/394: dread d4/dd/d12/d52/f63 [0,4194304] 0 2024-09-16T00:26:00.060 INFO:tasks.workunit.client.1.smithi169.stdout:5/261: write d2/f2a [603608,35497] 0 2024-09-16T00:26:00.062 INFO:tasks.workunit.client.0.smithi112.stdout:6/454: write dd/df/d10/d47/f5b [1647510,97341] 0 2024-09-16T00:26:00.062 INFO:tasks.workunit.client.0.smithi112.stdout:1/451: dread - d0/d14/d1c/d26/d35/f84 zero size 2024-09-16T00:26:00.062 INFO:tasks.workunit.client.0.smithi112.stdout:4/444: unlink d6/db/f25 0 2024-09-16T00:26:00.063 INFO:tasks.workunit.client.0.smithi112.stdout:2/514: readlink d2/dd/d48/d22/d2b/d54/d60/d6c/d88/l89 0 2024-09-16T00:26:00.064 INFO:tasks.workunit.client.0.smithi112.stdout:3/501: getdents d2/d9/d2b/d47/d52/d7a/d80/d83/d68 0 2024-09-16T00:26:00.064 INFO:tasks.workunit.client.1.smithi169.stdout:6/287: symlink d2/d36/d48/l63 0 2024-09-16T00:26:00.065 INFO:tasks.workunit.client.0.smithi112.stdout:9/673: write d1/d5/d6/d59/fec [534747,68902] 0 2024-09-16T00:26:00.065 INFO:tasks.workunit.client.0.smithi112.stdout:6/455: truncate dd/d23/d8f/f8c 413650 0 2024-09-16T00:26:00.067 INFO:tasks.workunit.client.0.smithi112.stdout:0/545: rename d5/da/d57/d76/d74 to d5/da/d57/db0 0 2024-09-16T00:26:00.067 INFO:tasks.workunit.client.1.smithi169.stdout:6/288: chown d2/fb 63 1 2024-09-16T00:26:00.067 INFO:tasks.workunit.client.1.smithi169.stdout:0/299: dwrite d0/d2/d8/f47 [0,4194304] 0 2024-09-16T00:26:00.069 INFO:tasks.workunit.client.1.smithi169.stdout:8/293: getdents d7/dc/d17/d24 0 2024-09-16T00:26:00.071 INFO:tasks.workunit.client.0.smithi112.stdout:8/414: mknod d10/d4e/d58/d87/c88 0 2024-09-16T00:26:00.071 INFO:tasks.workunit.client.0.smithi112.stdout:3/502: chown d2/d9/f1e 199212 1 2024-09-16T00:26:00.071 INFO:tasks.workunit.client.1.smithi169.stdout:8/294: fsync d7/dc/d17/f1b 0 2024-09-16T00:26:00.071 INFO:tasks.workunit.client.1.smithi169.stdout:0/300: write d0/d2c/d54/f3b [956798,55756] 0 2024-09-16T00:26:00.072 INFO:tasks.workunit.client.0.smithi112.stdout:5/395: fdatasync f2 0 2024-09-16T00:26:00.073 INFO:tasks.workunit.client.0.smithi112.stdout:1/452: creat d0/d14/df/d28/d45/d46/f90 x:0 0 0 2024-09-16T00:26:00.074 INFO:tasks.workunit.client.0.smithi112.stdout:7/459: dread d6/d10/d16/d49/d6c/f95 [0,4194304] 0 2024-09-16T00:26:00.075 INFO:tasks.workunit.client.1.smithi169.stdout:5/262: creat d2/da/d10/d13/d1c/d40/f5b x:0 0 0 2024-09-16T00:26:00.080 INFO:tasks.workunit.client.0.smithi112.stdout:4/445: truncate d6/db/d1f/d41/d49/f75 253518 0 2024-09-16T00:26:00.082 INFO:tasks.workunit.client.0.smithi112.stdout:7/460: write d6/d10/d1b/f7d [4395777,74688] 0 2024-09-16T00:26:00.083 INFO:tasks.workunit.client.1.smithi169.stdout:0/301: unlink d0/d2/c34 0 2024-09-16T00:26:00.084 INFO:tasks.workunit.client.0.smithi112.stdout:2/515: dread d2/f64 [0,4194304] 0 2024-09-16T00:26:00.086 INFO:tasks.workunit.client.1.smithi169.stdout:3/349: link d7/d1a/l77 d7/d1a/d2b/l83 0 2024-09-16T00:26:00.086 INFO:tasks.workunit.client.0.smithi112.stdout:4/446: symlink d6/db/df/la2 0 2024-09-16T00:26:00.087 INFO:tasks.workunit.client.1.smithi169.stdout:0/302: rmdir d0/d2c/d38 39 2024-09-16T00:26:00.087 INFO:tasks.workunit.client.0.smithi112.stdout:6/456: dwrite dd/df/d17/d27/d84/d37/d79/f8d [0,4194304] 0 2024-09-16T00:26:00.088 INFO:tasks.workunit.client.1.smithi169.stdout:8/295: rename d7/dc/c25 to d7/dc/d1d/c5e 0 2024-09-16T00:26:00.089 INFO:tasks.workunit.client.0.smithi112.stdout:3/503: mkdir d2/d9/d2b/d47/d52/d7a/d80/d83/d7b/dad 0 2024-09-16T00:26:00.091 INFO:tasks.workunit.client.0.smithi112.stdout:8/415: dwrite d10/d12/d1c/f31 [8388608,4194304] 0 2024-09-16T00:26:00.091 INFO:tasks.workunit.client.0.smithi112.stdout:6/457: stat dd/df/d10/d47/d55/f72 0 2024-09-16T00:26:00.092 INFO:tasks.workunit.client.0.smithi112.stdout:5/396: mknod d4/d7/d87/c89 0 2024-09-16T00:26:00.092 INFO:tasks.workunit.client.1.smithi169.stdout:6/289: creat d2/f64 x:0 0 0 2024-09-16T00:26:00.092 INFO:tasks.workunit.client.1.smithi169.stdout:7/385: getdents d7 0 2024-09-16T00:26:00.092 INFO:tasks.workunit.client.1.smithi169.stdout:3/350: mknod d7/d1a/d1c/d35/c84 0 2024-09-16T00:26:00.092 INFO:tasks.workunit.client.1.smithi169.stdout:8/296: write d7/dc/d17/f1b [1442308,91139] 0 2024-09-16T00:26:00.093 INFO:tasks.workunit.client.0.smithi112.stdout:1/453: creat d0/d14/d1c/f91 x:0 0 0 2024-09-16T00:26:00.095 INFO:tasks.workunit.client.1.smithi169.stdout:7/386: write d7/d71/d76/f23 [990902,129818] 0 2024-09-16T00:26:00.096 INFO:tasks.workunit.client.0.smithi112.stdout:0/546: getdents d5/da/dc/d4e 0 2024-09-16T00:26:00.098 INFO:tasks.workunit.client.0.smithi112.stdout:3/504: write d2/d9/d2b/d2c/d69/f72 [856169,106444] 0 2024-09-16T00:26:00.099 INFO:tasks.workunit.client.1.smithi169.stdout:8/297: creat d7/dc/d1d/f5f x:0 0 0 2024-09-16T00:26:00.100 INFO:tasks.workunit.client.0.smithi112.stdout:5/397: fdatasync d4/dd/d49/d70/d83/f86 0 2024-09-16T00:26:00.100 INFO:tasks.workunit.client.0.smithi112.stdout:8/416: write d10/d12/d1c/d3b/f4d [1656404,120550] 0 2024-09-16T00:26:00.101 INFO:tasks.workunit.client.0.smithi112.stdout:6/458: truncate dd/df/d11/f1e 871190 0 2024-09-16T00:26:00.104 INFO:tasks.workunit.client.0.smithi112.stdout:4/447: dread d6/db/df/f5f [0,4194304] 0 2024-09-16T00:26:00.107 INFO:tasks.workunit.client.0.smithi112.stdout:1/454: symlink d0/d14/d1c/d26/d47/l92 0 2024-09-16T00:26:00.108 INFO:tasks.workunit.client.1.smithi169.stdout:3/351: dwrite d7/d1a/d1c/d35/d63/f6a [0,4194304] 0 2024-09-16T00:26:00.109 INFO:tasks.workunit.client.0.smithi112.stdout:2/516: dwrite d2/dd/d48/d22/d2b/f43 [0,4194304] 0 2024-09-16T00:26:00.112 INFO:tasks.workunit.client.1.smithi169.stdout:6/290: dread d2/d4d/f53 [0,4194304] 0 2024-09-16T00:26:00.112 INFO:tasks.workunit.client.0.smithi112.stdout:6/459: write dd/df/d11/f67 [204046,45891] 0 2024-09-16T00:26:00.113 INFO:tasks.workunit.client.1.smithi169.stdout:5/263: dread d2/da/d10/d13/f35 [0,4194304] 0 2024-09-16T00:26:00.115 INFO:tasks.workunit.client.1.smithi169.stdout:3/352: symlink d7/d1a/l85 0 2024-09-16T00:26:00.115 INFO:tasks.workunit.client.0.smithi112.stdout:4/448: chown d6/db/d1f/d41/d49/l8e 29858814 1 2024-09-16T00:26:00.116 INFO:tasks.workunit.client.0.smithi112.stdout:1/455: mkdir d0/d14/d1c/d20/d93 0 2024-09-16T00:26:00.118 INFO:tasks.workunit.client.0.smithi112.stdout:5/398: mknod d4/dd/c8a 0 2024-09-16T00:26:00.118 INFO:tasks.workunit.client.0.smithi112.stdout:6/460: stat dd/df/d13/l8e 0 2024-09-16T00:26:00.118 INFO:tasks.workunit.client.1.smithi169.stdout:8/298: dwrite d7/f12 [0,4194304] 0 2024-09-16T00:26:00.118 INFO:tasks.workunit.client.1.smithi169.stdout:7/387: dread d7/d71/d76/f37 [0,4194304] 0 2024-09-16T00:26:00.120 INFO:tasks.workunit.client.1.smithi169.stdout:3/353: dread d7/f68 [0,4194304] 0 2024-09-16T00:26:00.120 INFO:tasks.workunit.client.0.smithi112.stdout:0/547: link d5/da/d66/f5d d5/da/d21/da5/fb1 0 2024-09-16T00:26:00.123 INFO:tasks.workunit.client.0.smithi112.stdout:2/517: creat d2/dd/d48/d22/d2b/d54/d79/f9e x:0 0 0 2024-09-16T00:26:00.125 INFO:tasks.workunit.client.0.smithi112.stdout:9/674: dwrite d1/d5/d20/f93 [0,4194304] 0 2024-09-16T00:26:00.125 INFO:tasks.workunit.client.1.smithi169.stdout:3/354: creat d7/d50/d6f/f86 x:0 0 0 2024-09-16T00:26:00.127 INFO:tasks.workunit.client.0.smithi112.stdout:3/505: mknod d2/d9/d2b/d47/d52/d7a/d80/d83/d68/d90/cae 0 2024-09-16T00:26:00.128 INFO:tasks.workunit.client.1.smithi169.stdout:0/303: truncate d0/f13 2577498 0 2024-09-16T00:26:00.128 INFO:tasks.workunit.client.1.smithi169.stdout:6/291: dread d2/d36/f31 [4194304,4194304] 0 2024-09-16T00:26:00.128 INFO:tasks.workunit.client.1.smithi169.stdout:7/388: symlink d7/d16/l80 0 2024-09-16T00:26:00.130 INFO:tasks.workunit.client.0.smithi112.stdout:4/449: unlink d6/da/f40 0 2024-09-16T00:26:00.130 INFO:tasks.workunit.client.0.smithi112.stdout:1/456: truncate d0/d14/d1c/d26/f76 992503 0 2024-09-16T00:26:00.131 INFO:tasks.workunit.client.1.smithi169.stdout:3/355: rename d7/d1a/d1c/d35/f49 to d7/d50/f87 0 2024-09-16T00:26:00.131 INFO:tasks.workunit.client.1.smithi169.stdout:6/292: read - d2/d36/f2a zero size 2024-09-16T00:26:00.131 INFO:tasks.workunit.client.1.smithi169.stdout:0/304: write d0/d2/d8/d5/f44 [785970,69151] 0 2024-09-16T00:26:00.132 INFO:tasks.workunit.client.0.smithi112.stdout:8/417: dwrite d10/f21 [0,4194304] 0 2024-09-16T00:26:00.133 INFO:tasks.workunit.client.1.smithi169.stdout:5/264: creat d2/da/d10/d13/d1c/d32/d53/d4e/d55/f5c x:0 0 0 2024-09-16T00:26:00.133 INFO:tasks.workunit.client.0.smithi112.stdout:6/461: chown dd/df/d17/d27/d54/f56 113317 1 2024-09-16T00:26:00.135 INFO:tasks.workunit.client.1.smithi169.stdout:0/305: chown d0/d2c/d54/d21/c29 2513931 1 2024-09-16T00:26:00.136 INFO:tasks.workunit.client.1.smithi169.stdout:8/299: dwrite d7/dc/d26/d46/f5d [0,4194304] 0 2024-09-16T00:26:00.136 INFO:tasks.workunit.client.1.smithi169.stdout:3/356: mkdir d7/d1a/d1c/d23/d88 0 2024-09-16T00:26:00.138 INFO:tasks.workunit.client.0.smithi112.stdout:0/548: write d5/da/dc/f7d [586996,17693] 0 2024-09-16T00:26:00.141 INFO:tasks.workunit.client.1.smithi169.stdout:0/306: fsync d0/d2c/d54/f28 0 2024-09-16T00:26:00.141 INFO:tasks.workunit.client.1.smithi169.stdout:8/300: creat d7/dc/d17/d48/f60 x:0 0 0 2024-09-16T00:26:00.143 INFO:tasks.workunit.client.1.smithi169.stdout:5/265: symlink d2/da/d10/d13/d1c/d40/d54/l5d 0 2024-09-16T00:26:00.144 INFO:tasks.workunit.client.0.smithi112.stdout:8/418: truncate d10/d4e/f6f 1254178 0 2024-09-16T00:26:00.144 INFO:tasks.workunit.client.0.smithi112.stdout:7/461: dread d6/d10/f15 [0,4194304] 0 2024-09-16T00:26:00.144 INFO:tasks.workunit.client.0.smithi112.stdout:1/457: stat d0/d14/df/d28/d45/d30/f4a 0 2024-09-16T00:26:00.145 INFO:tasks.workunit.client.1.smithi169.stdout:6/293: creat d2/d3f/d41/d5c/f65 x:0 0 0 2024-09-16T00:26:00.146 INFO:tasks.workunit.client.1.smithi169.stdout:3/357: creat d7/d1a/d1c/d23/f89 x:0 0 0 2024-09-16T00:26:00.147 INFO:tasks.workunit.client.0.smithi112.stdout:8/419: fsync d10/d4e/f68 0 2024-09-16T00:26:00.147 INFO:tasks.workunit.client.0.smithi112.stdout:5/399: rename d4/d7/d26/l5e to d4/d4f/d77/l8b 0 2024-09-16T00:26:00.147 INFO:tasks.workunit.client.0.smithi112.stdout:3/506: creat d2/d9/d2b/d47/d52/d7a/d80/d83/d76/faf x:0 0 0 2024-09-16T00:26:00.147 INFO:tasks.workunit.client.0.smithi112.stdout:4/450: mknod d6/d39/d36/d5c/d7e/ca3 0 2024-09-16T00:26:00.148 INFO:tasks.workunit.client.1.smithi169.stdout:7/389: dread d7/d16/d49/f58 [0,4194304] 0 2024-09-16T00:26:00.148 INFO:tasks.workunit.client.1.smithi169.stdout:5/266: chown d2/ff 1 1 2024-09-16T00:26:00.149 INFO:tasks.workunit.client.1.smithi169.stdout:6/294: write d2/f19 [1575828,72774] 0 2024-09-16T00:26:00.149 INFO:tasks.workunit.client.0.smithi112.stdout:3/507: rmdir d2/d9/d2b/d47/d78 39 2024-09-16T00:26:00.151 INFO:tasks.workunit.client.0.smithi112.stdout:3/508: fdatasync d2/d9/fa5 0 2024-09-16T00:26:00.151 INFO:tasks.workunit.client.0.smithi112.stdout:7/462: write d6/d10/d1b/d39/f92 [816190,118978] 0 2024-09-16T00:26:00.152 INFO:tasks.workunit.client.1.smithi169.stdout:0/307: dwrite d0/d2c/d54/f28 [0,4194304] 0 2024-09-16T00:26:00.156 INFO:tasks.workunit.client.0.smithi112.stdout:8/420: truncate d10/d12/d1c/d61/d6c/f7a 490825 0 2024-09-16T00:26:00.157 INFO:tasks.workunit.client.0.smithi112.stdout:6/462: link dd/df/d10/d47/d89/l92 dd/df/d11/d44/d8a/l9a 0 2024-09-16T00:26:00.157 INFO:tasks.workunit.client.1.smithi169.stdout:7/390: creat d7/d16/d49/f81 x:0 0 0 2024-09-16T00:26:00.158 INFO:tasks.workunit.client.1.smithi169.stdout:6/295: symlink d2/d3f/l66 0 2024-09-16T00:26:00.158 INFO:tasks.workunit.client.1.smithi169.stdout:5/267: rename d2/da/d10/d13/d1c/d40/d54/l5d to d2/da/d10/d13/d24/l5e 0 2024-09-16T00:26:00.158 INFO:tasks.workunit.client.1.smithi169.stdout:6/296: mkdir d2/d4d/d59/d43/d67 0 2024-09-16T00:26:00.159 INFO:tasks.workunit.client.0.smithi112.stdout:4/451: rename d6/d39/d1c/d2c/c62 to d6/db/d1f/d41/d49/d31/ca4 0 2024-09-16T00:26:00.159 INFO:tasks.workunit.client.1.smithi169.stdout:6/297: read d2/d3f/d41/d52/f1e [5877244,74711] 0 2024-09-16T00:26:00.159 INFO:tasks.workunit.client.1.smithi169.stdout:0/308: symlink d0/d2c/d54/l74 0 2024-09-16T00:26:00.159 INFO:tasks.workunit.client.1.smithi169.stdout:7/391: mknod d7/d71/d76/d29/d78/c82 0 2024-09-16T00:26:00.159 INFO:tasks.workunit.client.1.smithi169.stdout:0/309: readlink d0/d2c/d54/l72 0 2024-09-16T00:26:00.160 INFO:tasks.workunit.client.0.smithi112.stdout:7/463: unlink d6/d10/d16/c3f 0 2024-09-16T00:26:00.160 INFO:tasks.workunit.client.0.smithi112.stdout:8/421: chown d10/d12/d24/c56 143 1 2024-09-16T00:26:00.160 INFO:tasks.workunit.client.1.smithi169.stdout:5/268: getdents d2/da/d10/d13/d1c/d2d/d46 0 2024-09-16T00:26:00.161 INFO:tasks.workunit.client.1.smithi169.stdout:7/392: dread d7/d71/d76/f37 [0,4194304] 0 2024-09-16T00:26:00.162 INFO:tasks.workunit.client.0.smithi112.stdout:0/549: dwrite d5/da/d57/f73 [0,4194304] 0 2024-09-16T00:26:00.162 INFO:tasks.workunit.client.0.smithi112.stdout:2/518: dread d2/dd/d48/d22/d2b/d30/f36 [0,4194304] 0 2024-09-16T00:26:00.164 INFO:tasks.workunit.client.0.smithi112.stdout:6/463: truncate dd/df/f21 5159653 0 2024-09-16T00:26:00.164 INFO:tasks.workunit.client.0.smithi112.stdout:1/458: rename d0/d14/d1c/d26/d47/l92 to d0/d14/d1c/d26/d35/l94 0 2024-09-16T00:26:00.168 INFO:tasks.workunit.client.0.smithi112.stdout:7/464: unlink d6/f11 0 2024-09-16T00:26:00.168 INFO:tasks.workunit.client.1.smithi169.stdout:5/269: creat d2/da/d10/d13/d1c/d2d/f5f x:0 0 0 2024-09-16T00:26:00.169 INFO:tasks.workunit.client.0.smithi112.stdout:8/422: write d10/d4e/f64 [753084,55732] 0 2024-09-16T00:26:00.170 INFO:tasks.workunit.client.0.smithi112.stdout:2/519: truncate d2/dd/d48/d94/d49/f7d 457132 0 2024-09-16T00:26:00.171 INFO:tasks.workunit.client.1.smithi169.stdout:7/393: symlink d7/d71/d76/d29/l83 0 2024-09-16T00:26:00.172 INFO:tasks.workunit.client.0.smithi112.stdout:9/675: dwrite d1/d5/d2c/d6f/f8e [0,4194304] 0 2024-09-16T00:26:00.172 INFO:tasks.workunit.client.1.smithi169.stdout:0/310: creat d0/d2c/d54/f75 x:0 0 0 2024-09-16T00:26:00.174 INFO:tasks.workunit.client.0.smithi112.stdout:0/550: write d5/da/dc/f10 [2165309,17749] 0 2024-09-16T00:26:00.175 INFO:tasks.workunit.client.1.smithi169.stdout:3/358: write d7/fd [2087351,123323] 0 2024-09-16T00:26:00.176 INFO:tasks.workunit.client.1.smithi169.stdout:7/394: mkdir d7/d16/d49/d84 0 2024-09-16T00:26:00.179 INFO:tasks.workunit.client.1.smithi169.stdout:5/270: dwrite d2/f4 [0,4194304] 0 2024-09-16T00:26:00.179 INFO:tasks.workunit.client.1.smithi169.stdout:0/311: symlink d0/d50/d6c/l76 0 2024-09-16T00:26:00.179 INFO:tasks.workunit.client.0.smithi112.stdout:5/400: write d4/dd/d49/d70/d83/f86 [5184213,14209] 0 2024-09-16T00:26:00.182 INFO:tasks.workunit.client.0.smithi112.stdout:0/551: unlink d5/da/d1d/ca8 0 2024-09-16T00:26:00.183 INFO:tasks.workunit.client.1.smithi169.stdout:3/359: creat d7/d1a/d1c/d35/f8a x:0 0 0 2024-09-16T00:26:00.183 INFO:tasks.workunit.client.0.smithi112.stdout:0/552: stat d5/f33 0 2024-09-16T00:26:00.183 INFO:tasks.workunit.client.0.smithi112.stdout:9/676: truncate d1/d5/d6/f8f 4930919 0 2024-09-16T00:26:00.185 INFO:tasks.workunit.client.0.smithi112.stdout:5/401: dread - d4/d7/d26/d3e/d7c/f88 zero size 2024-09-16T00:26:00.185 INFO:tasks.workunit.client.1.smithi169.stdout:6/298: truncate d2/d4c/f5d 1239454 0 2024-09-16T00:26:00.185 INFO:tasks.workunit.client.0.smithi112.stdout:1/459: rename d0/d14/df/d28/d45/d30/f8c to d0/d14/d1c/d26/d41/f95 0 2024-09-16T00:26:00.189 INFO:tasks.workunit.client.1.smithi169.stdout:3/360: symlink d7/d1a/d2b/l8b 0 2024-09-16T00:26:00.189 INFO:tasks.workunit.client.1.smithi169.stdout:0/312: symlink d0/d4c/d6f/l77 0 2024-09-16T00:26:00.191 INFO:tasks.workunit.client.0.smithi112.stdout:6/464: truncate dd/d23/f64 2987858 0 2024-09-16T00:26:00.191 INFO:tasks.workunit.client.0.smithi112.stdout:2/520: dwrite d2/f8d [0,4194304] 0 2024-09-16T00:26:00.192 INFO:tasks.workunit.client.1.smithi169.stdout:7/395: getdents d7/d71 0 2024-09-16T00:26:00.192 INFO:tasks.workunit.client.1.smithi169.stdout:6/299: chown d2/c42 1 1 2024-09-16T00:26:00.192 INFO:tasks.workunit.client.1.smithi169.stdout:7/396: fdatasync d7/d71/d76/f23 0 2024-09-16T00:26:00.192 INFO:tasks.workunit.client.1.smithi169.stdout:0/313: chown d0/d2c 1837848 1 2024-09-16T00:26:00.193 INFO:tasks.workunit.client.1.smithi169.stdout:5/271: dwrite d2/da/d10/d13/d1c/d32/d53/d4e/d55/f5c [0,4194304] 0 2024-09-16T00:26:00.195 INFO:tasks.workunit.client.1.smithi169.stdout:7/397: dread - d7/d71/d76/d29/f45 zero size 2024-09-16T00:26:00.199 INFO:tasks.workunit.client.1.smithi169.stdout:0/314: symlink d0/d50/d6c/l78 0 2024-09-16T00:26:00.202 INFO:tasks.workunit.client.1.smithi169.stdout:5/272: chown d2/da/d10/d13/d24/l5e 12068203 1 2024-09-16T00:26:00.204 INFO:tasks.workunit.client.1.smithi169.stdout:6/300: unlink d2/d22/l55 0 2024-09-16T00:26:00.206 INFO:tasks.workunit.client.1.smithi169.stdout:0/315: mkdir d0/d2/d8/d5/d79 0 2024-09-16T00:26:00.208 INFO:tasks.workunit.client.1.smithi169.stdout:7/398: dwrite d7/df/f60 [0,4194304] 0 2024-09-16T00:26:00.208 INFO:tasks.workunit.client.1.smithi169.stdout:0/316: creat d0/d2/d15/f7a x:0 0 0 2024-09-16T00:26:00.209 INFO:tasks.workunit.client.1.smithi169.stdout:0/317: chown d0/d2c/d54/d21 207839915 1 2024-09-16T00:26:00.210 INFO:tasks.workunit.client.1.smithi169.stdout:5/273: rename d2/l25 to d2/da/d10/d13/d24/l60 0 2024-09-16T00:26:00.214 INFO:tasks.workunit.client.0.smithi112.stdout:8/423: creat d10/d12/d1c/d3b/f89 x:0 0 0 2024-09-16T00:26:00.214 INFO:tasks.workunit.client.0.smithi112.stdout:9/677: rename d1/d5/d20/f31 to d1/d5/d2c/d6f/dbf/fee 0 2024-09-16T00:26:00.217 INFO:tasks.workunit.client.0.smithi112.stdout:2/521: stat d2/dd/d48/d22/d2b/d54/d60/c8e 0 2024-09-16T00:26:00.217 INFO:tasks.workunit.client.0.smithi112.stdout:6/465: chown dd/df/d10/d47/d55/f72 4293 1 2024-09-16T00:26:00.218 INFO:tasks.workunit.client.0.smithi112.stdout:4/452: dread d6/f1b [0,4194304] 0 2024-09-16T00:26:00.218 INFO:tasks.workunit.client.1.smithi169.stdout:7/399: creat d7/d71/d76/d29/f85 x:0 0 0 2024-09-16T00:26:00.218 INFO:tasks.workunit.client.1.smithi169.stdout:5/274: fdatasync d2/da/f28 0 2024-09-16T00:26:00.219 INFO:tasks.workunit.client.0.smithi112.stdout:9/678: dread - d1/d5/d2c/fd4 zero size 2024-09-16T00:26:00.219 INFO:tasks.workunit.client.1.smithi169.stdout:7/400: write d7/d16/f40 [918096,120913] 0 2024-09-16T00:26:00.220 INFO:tasks.workunit.client.0.smithi112.stdout:1/460: symlink d0/d14/d1c/d20/d93/l96 0 2024-09-16T00:26:00.220 INFO:tasks.workunit.client.1.smithi169.stdout:7/401: chown d7/df 204 1 2024-09-16T00:26:00.220 INFO:tasks.workunit.client.1.smithi169.stdout:0/318: symlink d0/d2c/d38/l7b 0 2024-09-16T00:26:00.221 INFO:tasks.workunit.client.0.smithi112.stdout:8/424: mknod d10/d4e/d58/d87/c8a 0 2024-09-16T00:26:00.223 INFO:tasks.workunit.client.0.smithi112.stdout:2/522: creat d2/dd/d48/d94/d49/f9f x:0 0 0 2024-09-16T00:26:00.223 INFO:tasks.workunit.client.0.smithi112.stdout:3/509: dread d2/d9/d2b/d39/f3c [0,4194304] 0 2024-09-16T00:26:00.223 INFO:tasks.workunit.client.0.smithi112.stdout:4/453: creat d6/d39/d36/d5c/d7e/fa5 x:0 0 0 2024-09-16T00:26:00.223 INFO:tasks.workunit.client.0.smithi112.stdout:1/461: write d0/d14/d1c/d26/f73 [390269,32000] 0 2024-09-16T00:26:00.225 INFO:tasks.workunit.client.1.smithi169.stdout:5/275: creat d2/f61 x:0 0 0 2024-09-16T00:26:00.226 INFO:tasks.workunit.client.1.smithi169.stdout:0/319: dread d0/d2c/d38/f5e [0,4194304] 0 2024-09-16T00:26:00.228 INFO:tasks.workunit.client.1.smithi169.stdout:0/320: write d0/d2/f45 [479650,906] 0 2024-09-16T00:26:00.229 INFO:tasks.workunit.client.0.smithi112.stdout:7/465: dread d6/d10/d75/d88/f27 [0,4194304] 0 2024-09-16T00:26:00.229 INFO:tasks.workunit.client.0.smithi112.stdout:9/679: rename d1/d5/d20/d25 to d1/d5/d20/d81/d1a/d88/def 0 2024-09-16T00:26:00.229 INFO:tasks.workunit.client.1.smithi169.stdout:5/276: stat d2/da/d10/d13/d1c/d32/d53/d4e/d55 0 2024-09-16T00:26:00.231 INFO:tasks.workunit.client.1.smithi169.stdout:3/361: dread d7/d1a/d1c/d35/d63/f4a [0,4194304] 0 2024-09-16T00:26:00.231 INFO:tasks.workunit.client.1.smithi169.stdout:0/321: unlink d0/d2c/d54/f36 0 2024-09-16T00:26:00.232 INFO:tasks.workunit.client.1.smithi169.stdout:5/277: dread d2/da/d10/d13/d1c/d32/d53/f4d [0,4194304] 0 2024-09-16T00:26:00.234 INFO:tasks.workunit.client.0.smithi112.stdout:7/466: write d6/d10/d75/d88/f70 [632391,72828] 0 2024-09-16T00:26:00.235 INFO:tasks.workunit.client.1.smithi169.stdout:3/362: creat d7/d1a/d2b/f8c x:0 0 0 2024-09-16T00:26:00.235 INFO:tasks.workunit.client.0.smithi112.stdout:1/462: rename d0/d14/df/f8f to d0/d14/d1c/d20/d93/f97 0 2024-09-16T00:26:00.235 INFO:tasks.workunit.client.0.smithi112.stdout:7/467: read - d6/f23 zero size 2024-09-16T00:26:00.236 INFO:tasks.workunit.client.0.smithi112.stdout:3/510: dwrite d2/d9/d2b/d2c/d69/f8d [4194304,4194304] 0 2024-09-16T00:26:00.238 INFO:tasks.workunit.client.0.smithi112.stdout:7/468: readlink d6/l80 0 2024-09-16T00:26:00.238 INFO:tasks.workunit.client.1.smithi169.stdout:5/278: mkdir d2/da/d10/d13/d1c/d40/d43/d62 0 2024-09-16T00:26:00.238 INFO:tasks.workunit.client.0.smithi112.stdout:1/463: stat d0/d14/d1c/d26/d47/l79 0 2024-09-16T00:26:00.238 INFO:tasks.workunit.client.1.smithi169.stdout:0/322: mkdir d0/d2/d8/d5/d79/d7c 0 2024-09-16T00:26:00.239 INFO:tasks.workunit.client.0.smithi112.stdout:2/523: creat d2/dd/d48/d22/d2b/fa0 x:0 0 0 2024-09-16T00:26:00.240 INFO:tasks.workunit.client.0.smithi112.stdout:3/511: write d2/d9/d2b/d2c/d69/f8d [3293094,18380] 0 2024-09-16T00:26:00.241 INFO:tasks.workunit.client.0.smithi112.stdout:4/454: getdents d6/d39/d1c/d6e/d6f 0 2024-09-16T00:26:00.241 INFO:tasks.workunit.client.0.smithi112.stdout:2/524: read d2/dd/f15 [535335,117069] 0 2024-09-16T00:26:00.242 INFO:tasks.workunit.client.1.smithi169.stdout:0/323: chown d0/f51 0 1 2024-09-16T00:26:00.242 INFO:tasks.workunit.client.0.smithi112.stdout:7/469: creat d6/d10/d1b/d39/fa7 x:0 0 0 2024-09-16T00:26:00.243 INFO:tasks.workunit.client.0.smithi112.stdout:4/455: write d6/db/f60 [4004635,4521] 0 2024-09-16T00:26:00.245 INFO:tasks.workunit.client.0.smithi112.stdout:7/470: write d6/d10/d16/d49/d6c/f95 [3760364,102894] 0 2024-09-16T00:26:00.246 INFO:tasks.workunit.client.0.smithi112.stdout:1/464: creat d0/d14/df/d28/d45/f98 x:0 0 0 2024-09-16T00:26:00.246 INFO:tasks.workunit.client.1.smithi169.stdout:0/324: mkdir d0/d7d 0 2024-09-16T00:26:00.247 INFO:tasks.workunit.client.0.smithi112.stdout:3/512: symlink d2/d9/d2b/d2c/lb0 0 2024-09-16T00:26:00.247 INFO:tasks.workunit.client.1.smithi169.stdout:3/363: truncate d7/f9 1415081 0 2024-09-16T00:26:00.248 INFO:tasks.workunit.client.1.smithi169.stdout:7/402: write d7/d71/d76/d3a/f7a [1386629,10021] 0 2024-09-16T00:26:00.248 INFO:tasks.workunit.client.1.smithi169.stdout:0/325: chown d0/d2c/f4a 63 1 2024-09-16T00:26:00.250 INFO:tasks.workunit.client.0.smithi112.stdout:2/525: symlink d2/dd/d48/d22/d2b/d54/d79/la1 0 2024-09-16T00:26:00.250 INFO:tasks.workunit.client.0.smithi112.stdout:5/402: dread d4/dd/d49/d70/d83/f86 [0,4194304] 0 2024-09-16T00:26:00.250 INFO:tasks.workunit.client.1.smithi169.stdout:0/326: write d0/d2/f6 [6426773,49299] 0 2024-09-16T00:26:00.251 INFO:tasks.workunit.client.1.smithi169.stdout:7/403: write d7/d16/d49/f58 [1043439,44721] 0 2024-09-16T00:26:00.254 INFO:tasks.workunit.client.0.smithi112.stdout:3/513: rmdir d2/d9/d2b/d2c/d69 39 2024-09-16T00:26:00.254 INFO:tasks.workunit.client.0.smithi112.stdout:6/466: dread fa [0,4194304] 0 2024-09-16T00:26:00.257 INFO:tasks.workunit.client.0.smithi112.stdout:2/526: mknod d2/dd/d48/d94/ca2 0 2024-09-16T00:26:00.257 INFO:tasks.workunit.client.0.smithi112.stdout:5/403: write d4/dd/d49/d70/d83/f86 [1189018,35910] 0 2024-09-16T00:26:00.257 INFO:tasks.workunit.client.0.smithi112.stdout:6/467: readlink dd/df/d17/d27/d84/d37/l45 0 2024-09-16T00:26:00.257 INFO:tasks.workunit.client.0.smithi112.stdout:1/465: creat d0/d14/df/d28/d45/d46/d58/f99 x:0 0 0 2024-09-16T00:26:00.258 INFO:tasks.workunit.client.0.smithi112.stdout:5/404: chown d4/fb 40 1 2024-09-16T00:26:00.260 INFO:tasks.workunit.client.1.smithi169.stdout:0/327: dwrite d0/d50/f70 [0,4194304] 0 2024-09-16T00:26:00.260 INFO:tasks.workunit.client.0.smithi112.stdout:6/468: dread dd/d23/d8f/f8c [0,4194304] 0 2024-09-16T00:26:00.260 INFO:tasks.workunit.client.0.smithi112.stdout:2/527: symlink d2/dd/d48/d22/d67/la3 0 2024-09-16T00:26:00.261 INFO:tasks.workunit.client.0.smithi112.stdout:1/466: chown d0/d14/df/d28/f2e 0 1 2024-09-16T00:26:00.262 INFO:tasks.workunit.client.1.smithi169.stdout:7/404: dwrite d7/f31 [0,4194304] 0 2024-09-16T00:26:00.263 INFO:tasks.workunit.client.0.smithi112.stdout:6/469: fsync dd/df/d10/d47/f5b 0 2024-09-16T00:26:00.265 INFO:tasks.workunit.client.1.smithi169.stdout:0/328: mknod d0/d2c/c7e 0 2024-09-16T00:26:00.269 INFO:tasks.workunit.client.1.smithi169.stdout:0/329: creat d0/d50/f7f x:0 0 0 2024-09-16T00:26:00.270 INFO:tasks.workunit.client.0.smithi112.stdout:3/514: dwrite d2/d9/d2b/d47/f9d [0,4194304] 0 2024-09-16T00:26:00.271 INFO:tasks.workunit.client.1.smithi169.stdout:6/301: truncate d2/d4c/f5d 1888469 0 2024-09-16T00:26:00.273 INFO:tasks.workunit.client.0.smithi112.stdout:2/528: mkdir d2/dd/d48/d22/d2b/d54/d60/d6c/d88/da4 0 2024-09-16T00:26:00.273 INFO:tasks.workunit.client.0.smithi112.stdout:8/425: dread d10/d12/d1c/d75/f52 [0,4194304] 0 2024-09-16T00:26:00.274 INFO:tasks.workunit.client.1.smithi169.stdout:7/405: dwrite d7/d16/d49/f4d [0,4194304] 0 2024-09-16T00:26:00.274 INFO:tasks.workunit.client.1.smithi169.stdout:6/302: chown d2/d36/c3c 16718 1 2024-09-16T00:26:00.275 INFO:tasks.workunit.client.0.smithi112.stdout:3/515: creat d2/d9/d2b/d39/fb1 x:0 0 0 2024-09-16T00:26:00.276 INFO:tasks.workunit.client.1.smithi169.stdout:5/279: dread d2/da/d10/d13/f1d [4194304,4194304] 0 2024-09-16T00:26:00.278 INFO:tasks.workunit.client.0.smithi112.stdout:8/426: unlink d10/d4e/d58/l69 0 2024-09-16T00:26:00.282 INFO:tasks.workunit.client.1.smithi169.stdout:7/406: mknod d7/d5b/d61/c86 0 2024-09-16T00:26:00.282 INFO:tasks.workunit.client.0.smithi112.stdout:3/516: creat d2/d9/d2b/d2c/fb2 x:0 0 0 2024-09-16T00:26:00.283 INFO:tasks.workunit.client.1.smithi169.stdout:5/280: creat d2/da/d10/d13/d1c/d2d/f63 x:0 0 0 2024-09-16T00:26:00.284 INFO:tasks.workunit.client.0.smithi112.stdout:2/529: getdents d2/dd 0 2024-09-16T00:26:00.286 INFO:tasks.workunit.client.0.smithi112.stdout:2/530: read d2/dd/f15 [2028851,81228] 0 2024-09-16T00:26:00.288 INFO:tasks.workunit.client.0.smithi112.stdout:8/427: dwrite d10/d12/d24/f82 [0,4194304] 0 2024-09-16T00:26:00.288 INFO:tasks.workunit.client.0.smithi112.stdout:3/517: chown d2/d9/d2b/d47/d52/d7a/d80/d83/d49/f77 1001086137 1 2024-09-16T00:26:00.289 INFO:tasks.workunit.client.1.smithi169.stdout:7/407: creat d7/d5b/f87 x:0 0 0 2024-09-16T00:26:00.290 INFO:tasks.workunit.client.0.smithi112.stdout:4/456: dread d6/d39/d1c/f2f [0,4194304] 0 2024-09-16T00:26:00.291 INFO:tasks.workunit.client.0.smithi112.stdout:2/531: write d2/d68/f7f [3930016,118587] 0 2024-09-16T00:26:00.292 INFO:tasks.workunit.client.0.smithi112.stdout:8/428: symlink d10/d12/d1c/d3b/l8b 0 2024-09-16T00:26:00.294 INFO:tasks.workunit.client.1.smithi169.stdout:7/408: creat d7/d71/f88 x:0 0 0 2024-09-16T00:26:00.294 INFO:tasks.workunit.client.1.smithi169.stdout:6/303: dwrite d2/f1f [0,4194304] 0 2024-09-16T00:26:00.295 INFO:tasks.workunit.client.0.smithi112.stdout:4/457: chown d6/db/df/d5e/d61/d64/f9e 25 1 2024-09-16T00:26:00.295 INFO:tasks.workunit.client.1.smithi169.stdout:5/281: dwrite d2/f29 [0,4194304] 0 2024-09-16T00:26:00.296 INFO:tasks.workunit.client.1.smithi169.stdout:5/282: readlink d2/da/d10/l18 0 2024-09-16T00:26:00.297 INFO:tasks.workunit.client.1.smithi169.stdout:7/409: creat d7/d5b/d61/f89 x:0 0 0 2024-09-16T00:26:00.297 INFO:tasks.workunit.client.1.smithi169.stdout:6/304: mknod d2/d3f/d41/d5c/c68 0 2024-09-16T00:26:00.298 INFO:tasks.workunit.client.1.smithi169.stdout:5/283: chown d2/f4 4553747 1 2024-09-16T00:26:00.298 INFO:tasks.workunit.client.1.smithi169.stdout:7/410: chown d7/f7e 48 1 2024-09-16T00:26:00.302 INFO:tasks.workunit.client.1.smithi169.stdout:6/305: unlink d2/c24 0 2024-09-16T00:26:00.303 INFO:tasks.workunit.client.1.smithi169.stdout:5/284: fdatasync d2/da/d10/d13/f14 0 2024-09-16T00:26:00.304 INFO:tasks.workunit.client.1.smithi169.stdout:6/306: write d2/d4d/f53 [4590664,22319] 0 2024-09-16T00:26:00.305 INFO:tasks.workunit.client.0.smithi112.stdout:6/470: dread dd/df/d10/f1b [0,4194304] 0 2024-09-16T00:26:00.307 INFO:tasks.workunit.client.1.smithi169.stdout:6/307: dread - d2/d36/f2a zero size 2024-09-16T00:26:00.308 INFO:tasks.workunit.client.0.smithi112.stdout:5/405: dread d4/dd/d12/f48 [0,4194304] 0 2024-09-16T00:26:00.308 INFO:tasks.workunit.client.1.smithi169.stdout:5/285: rmdir d2/da/d10/d13/d1c/d32 39 2024-09-16T00:26:00.309 INFO:tasks.workunit.client.0.smithi112.stdout:7/471: dread d6/d10/d16/f3a [0,4194304] 0 2024-09-16T00:26:00.309 INFO:tasks.workunit.client.0.smithi112.stdout:7/472: chown d6/d10/d16/d49/d6c/f33 53 1 2024-09-16T00:26:00.311 INFO:tasks.workunit.client.0.smithi112.stdout:5/406: creat d4/d7/d14/f8c x:0 0 0 2024-09-16T00:26:00.313 INFO:tasks.workunit.client.1.smithi169.stdout:7/411: dwrite d7/d71/d76/f36 [0,4194304] 0 2024-09-16T00:26:00.315 INFO:tasks.workunit.client.0.smithi112.stdout:7/473: creat d6/d10/d1b/d39/fa8 x:0 0 0 2024-09-16T00:26:00.315 INFO:tasks.workunit.client.0.smithi112.stdout:1/467: truncate d0/d14/df/d28/f32 3504305 0 2024-09-16T00:26:00.317 INFO:tasks.workunit.client.1.smithi169.stdout:0/330: dread d0/d2c/d54/d21/f2f [0,4194304] 0 2024-09-16T00:26:00.317 INFO:tasks.workunit.client.1.smithi169.stdout:5/286: truncate d2/da/f28 3279211 0 2024-09-16T00:26:00.318 INFO:tasks.workunit.client.0.smithi112.stdout:6/471: getdents dd/df/d17/d27 0 2024-09-16T00:26:00.321 INFO:tasks.workunit.client.1.smithi169.stdout:7/412: write d7/d16/d49/d74/f7f [92038,91014] 0 2024-09-16T00:26:00.321 INFO:tasks.workunit.client.1.smithi169.stdout:5/287: readlink d2/da/d10/l18 0 2024-09-16T00:26:00.322 INFO:tasks.workunit.client.1.smithi169.stdout:6/308: dread d2/f45 [0,4194304] 0 2024-09-16T00:26:00.323 INFO:tasks.workunit.client.0.smithi112.stdout:1/468: link d0/d14/d1c/d26/f52 d0/f9a 0 2024-09-16T00:26:00.325 INFO:tasks.workunit.client.0.smithi112.stdout:1/469: truncate d0/d14/d6a/f80 480416 0 2024-09-16T00:26:00.327 INFO:tasks.workunit.client.1.smithi169.stdout:6/309: dread - d2/d4d/d59/d43/f2f zero size 2024-09-16T00:26:00.328 INFO:tasks.workunit.client.1.smithi169.stdout:6/310: chown d2/d4c 74599677 1 2024-09-16T00:26:00.328 INFO:tasks.workunit.client.0.smithi112.stdout:1/470: rename d0/d14/d1c/d26/d41/f95 to d0/d14/df/d28/d45/d30/f9b 0 2024-09-16T00:26:00.330 INFO:tasks.workunit.client.0.smithi112.stdout:1/471: mknod d0/d14/df/d28/d45/d46/c9c 0 2024-09-16T00:26:00.330 INFO:tasks.workunit.client.1.smithi169.stdout:7/413: link d7/d16/f1d d7/d16/d49/d84/f8a 0 2024-09-16T00:26:00.331 INFO:tasks.workunit.client.0.smithi112.stdout:8/429: dread d10/f1b [0,4194304] 0 2024-09-16T00:26:00.331 INFO:tasks.workunit.client.0.smithi112.stdout:1/472: truncate d0/f1e 1855874 0 2024-09-16T00:26:00.333 INFO:tasks.workunit.client.1.smithi169.stdout:0/331: dwrite d0/d2/d4b/f52 [0,4194304] 0 2024-09-16T00:26:00.334 INFO:tasks.workunit.client.0.smithi112.stdout:8/430: getdents d10/d12/d1c/d61/d6c/d71 0 2024-09-16T00:26:00.339 INFO:tasks.workunit.client.1.smithi169.stdout:6/311: dread d2/fb [4194304,4194304] 0 2024-09-16T00:26:00.340 INFO:tasks.workunit.client.0.smithi112.stdout:2/532: truncate d2/dd/f87 107185 0 2024-09-16T00:26:00.340 INFO:tasks.workunit.client.0.smithi112.stdout:4/458: dread d6/db/d1f/d41/d49/d31/f51 [0,4194304] 0 2024-09-16T00:26:00.340 INFO:tasks.workunit.client.0.smithi112.stdout:5/407: write d4/d7/d14/f33 [110546,31857] 0 2024-09-16T00:26:00.340 INFO:tasks.workunit.client.0.smithi112.stdout:7/474: write d6/d10/f6e [810212,116127] 0 2024-09-16T00:26:00.340 INFO:tasks.workunit.client.0.smithi112.stdout:4/459: write d6/db/f18 [4339081,48093] 0 2024-09-16T00:26:00.341 INFO:tasks.workunit.client.1.smithi169.stdout:7/414: rename d7/d71/d76/f23 to d7/f8b 0 2024-09-16T00:26:00.341 INFO:tasks.workunit.client.0.smithi112.stdout:1/473: dwrite d0/d14/d1c/d20/f53 [0,4194304] 0 2024-09-16T00:26:00.343 INFO:tasks.workunit.client.0.smithi112.stdout:5/408: chown d4/d4f/d77/f7f 5104 1 2024-09-16T00:26:00.343 INFO:tasks.workunit.client.0.smithi112.stdout:2/533: chown d2/l1f 119211414 1 2024-09-16T00:26:00.343 INFO:tasks.workunit.client.1.smithi169.stdout:4/307: sync 2024-09-16T00:26:00.344 INFO:tasks.workunit.client.0.smithi112.stdout:7/475: read d6/d10/d16/f34 [162156,60743] 0 2024-09-16T00:26:00.344 INFO:tasks.workunit.client.1.smithi169.stdout:4/308: chown d2/d12/d13/d1f/d3e/d5b 1 1 2024-09-16T00:26:00.346 INFO:tasks.workunit.client.0.smithi112.stdout:8/431: getdents d10/d4e/d58 0 2024-09-16T00:26:00.346 INFO:tasks.workunit.client.1.smithi169.stdout:4/309: dread - d2/d12/d13/d1f/d3e/d5b/d1a/f47 zero size 2024-09-16T00:26:00.347 INFO:tasks.workunit.client.1.smithi169.stdout:0/332: rename d0/d2/d8/c3d to d0/d4c/d6f/c80 0 2024-09-16T00:26:00.349 INFO:tasks.workunit.client.0.smithi112.stdout:5/409: write d4/d7/d26/f44 [297395,124107] 0 2024-09-16T00:26:00.349 INFO:tasks.workunit.client.0.smithi112.stdout:7/476: fdatasync d6/d10/d16/d3b/f4b 0 2024-09-16T00:26:00.352 INFO:tasks.workunit.client.0.smithi112.stdout:2/534: chown d2/dd/d48/d22/d2b/d54/d60/c78 72957191 1 2024-09-16T00:26:00.352 INFO:tasks.workunit.client.1.smithi169.stdout:4/310: symlink d2/l5f 0 2024-09-16T00:26:00.353 INFO:tasks.workunit.client.1.smithi169.stdout:7/415: creat d7/df/d73/f8c x:0 0 0 2024-09-16T00:26:00.354 INFO:tasks.workunit.client.0.smithi112.stdout:6/472: dwrite dd/d23/d8f/f8c [0,4194304] 0 2024-09-16T00:26:00.355 INFO:tasks.workunit.client.0.smithi112.stdout:2/535: chown d2/dd/d48/d22/d2b/d54 155456308 1 2024-09-16T00:26:00.357 INFO:tasks.workunit.client.1.smithi169.stdout:7/416: write d7/d71/d76/d3a/f3c [5064383,128547] 0 2024-09-16T00:26:00.358 INFO:tasks.workunit.client.1.smithi169.stdout:4/311: creat d2/d12/d13/f60 x:0 0 0 2024-09-16T00:26:00.360 INFO:tasks.workunit.client.1.smithi169.stdout:0/333: dwrite d0/d4c/f6d [0,4194304] 0 2024-09-16T00:26:00.361 INFO:tasks.workunit.client.0.smithi112.stdout:1/474: dwrite d0/d14/df/d28/f2e [4194304,4194304] 0 2024-09-16T00:26:00.361 INFO:tasks.workunit.client.0.smithi112.stdout:8/432: rename d10/d12/d1c/d61/d6c/d71 to d10/d4e/d8c 0 2024-09-16T00:26:00.363 INFO:tasks.workunit.client.0.smithi112.stdout:5/410: rename d4 to d4/d7/d87/d8d 22 2024-09-16T00:26:00.366 INFO:tasks.workunit.client.1.smithi169.stdout:5/288: dread d2/da/d10/d13/f14 [0,4194304] 0 2024-09-16T00:26:00.366 INFO:tasks.workunit.client.0.smithi112.stdout:2/536: mknod d2/dd/d48/d22/d2b/d54/d60/d6c/ca5 0 2024-09-16T00:26:00.366 INFO:tasks.workunit.client.0.smithi112.stdout:4/460: write d6/db/d1f/d41/d49/f17 [1444679,74070] 0 2024-09-16T00:26:00.369 INFO:tasks.workunit.client.0.smithi112.stdout:6/473: dwrite dd/d23/d8f/f8c [0,4194304] 0 2024-09-16T00:26:00.369 INFO:tasks.workunit.client.0.smithi112.stdout:2/537: readlink d2/dd/d48/l99 0 2024-09-16T00:26:00.369 INFO:tasks.workunit.client.1.smithi169.stdout:7/417: dwrite d7/d16/d49/f58 [0,4194304] 0 2024-09-16T00:26:00.369 INFO:tasks.workunit.client.0.smithi112.stdout:7/477: getdents d6/d10/d16/d49/d6c 0 2024-09-16T00:26:00.372 INFO:tasks.workunit.client.1.smithi169.stdout:4/312: creat d2/d12/d13/d5c/f61 x:0 0 0 2024-09-16T00:26:00.373 INFO:tasks.workunit.client.0.smithi112.stdout:5/411: symlink d4/dd/d12/d79/l8e 0 2024-09-16T00:26:00.373 INFO:tasks.workunit.client.1.smithi169.stdout:7/418: read d7/d71/d76/f37 [287814,33133] 0 2024-09-16T00:26:00.375 INFO:tasks.workunit.client.1.smithi169.stdout:0/334: symlink d0/d2/d15/l81 0 2024-09-16T00:26:00.376 INFO:tasks.workunit.client.0.smithi112.stdout:2/538: dread - d2/dd/d48/d94/d49/f9f zero size 2024-09-16T00:26:00.377 INFO:tasks.workunit.client.0.smithi112.stdout:4/461: creat d6/db/df/d5e/d61/fa6 x:0 0 0 2024-09-16T00:26:00.378 INFO:tasks.workunit.client.0.smithi112.stdout:1/475: dwrite d0/d14/df/d28/d42/f83 [0,4194304] 0 2024-09-16T00:26:00.378 INFO:tasks.workunit.client.1.smithi169.stdout:5/289: mknod d2/da/d10/d13/d1c/d32/d53/d4e/d55/c64 0 2024-09-16T00:26:00.379 INFO:tasks.workunit.client.1.smithi169.stdout:5/290: readlink d2/da/d10/l18 0 2024-09-16T00:26:00.379 INFO:tasks.workunit.client.1.smithi169.stdout:7/419: dread d7/d16/f40 [0,4194304] 0 2024-09-16T00:26:00.380 INFO:tasks.workunit.client.1.smithi169.stdout:4/313: mknod d2/d12/d13/d4e/c62 0 2024-09-16T00:26:00.381 INFO:tasks.workunit.client.1.smithi169.stdout:0/335: mkdir d0/d4c/d82 0 2024-09-16T00:26:00.382 INFO:tasks.workunit.client.1.smithi169.stdout:4/314: write d2/f49 [243344,74983] 0 2024-09-16T00:26:00.383 INFO:tasks.workunit.client.1.smithi169.stdout:6/312: getdents d2/d4d/d59/d38 0 2024-09-16T00:26:00.383 INFO:tasks.workunit.client.1.smithi169.stdout:5/291: truncate d2/da/f50 626666 0 2024-09-16T00:26:00.384 INFO:tasks.workunit.client.0.smithi112.stdout:5/412: mknod d4/dd/d12/d52/c8f 0 2024-09-16T00:26:00.385 INFO:tasks.workunit.client.1.smithi169.stdout:7/420: write d7/d71/d76/d29/f47 [2117299,130076] 0 2024-09-16T00:26:00.386 INFO:tasks.workunit.client.0.smithi112.stdout:6/474: unlink dd/df/d17/f77 0 2024-09-16T00:26:00.387 INFO:tasks.workunit.client.1.smithi169.stdout:4/315: symlink d2/d12/d13/d27/l63 0 2024-09-16T00:26:00.390 INFO:tasks.workunit.client.1.smithi169.stdout:7/421: creat d7/d16/d49/d84/f8d x:0 0 0 2024-09-16T00:26:00.391 INFO:tasks.workunit.client.0.smithi112.stdout:5/413: mkdir d4/d7/d14/d90 0 2024-09-16T00:26:00.391 INFO:tasks.workunit.client.0.smithi112.stdout:2/539: rename d2/dd/d48/l1e to d2/dd/d48/d22/la6 0 2024-09-16T00:26:00.392 INFO:tasks.workunit.client.0.smithi112.stdout:4/462: dwrite d6/d39/d1c/f27 [0,4194304] 0 2024-09-16T00:26:00.392 INFO:tasks.workunit.client.0.smithi112.stdout:1/476: dread d0/d14/df/d28/d42/f83 [0,4194304] 0 2024-09-16T00:26:00.392 INFO:tasks.workunit.client.1.smithi169.stdout:0/336: getdents d0/d50/d6c 0 2024-09-16T00:26:00.393 INFO:tasks.workunit.client.1.smithi169.stdout:4/316: mknod d2/d12/d13/d1f/c64 0 2024-09-16T00:26:00.394 INFO:tasks.workunit.client.1.smithi169.stdout:6/313: symlink d2/d22/l69 0 2024-09-16T00:26:00.394 INFO:tasks.workunit.client.1.smithi169.stdout:7/422: mkdir d7/d16/d49/d84/d8e 0 2024-09-16T00:26:00.394 INFO:tasks.workunit.client.1.smithi169.stdout:5/292: dread d2/da/f28 [0,4194304] 0 2024-09-16T00:26:00.396 INFO:tasks.workunit.client.0.smithi112.stdout:5/414: rename l0 to d4/dd/l91 0 2024-09-16T00:26:00.396 INFO:tasks.workunit.client.0.smithi112.stdout:1/477: read d0/d14/df/d28/d42/f83 [1136785,130506] 0 2024-09-16T00:26:00.397 INFO:tasks.workunit.client.1.smithi169.stdout:0/337: write d0/d2/d8/f68 [480951,82459] 0 2024-09-16T00:26:00.397 INFO:tasks.workunit.client.1.smithi169.stdout:5/293: stat d2/da/d10/d13/d1c/d32/d53/d4e/d55/f5c 0 2024-09-16T00:26:00.398 INFO:tasks.workunit.client.1.smithi169.stdout:7/423: mknod d7/d16/d49/d84/c8f 0 2024-09-16T00:26:00.399 INFO:tasks.workunit.client.1.smithi169.stdout:7/424: fdatasync d7/d5b/d61/f6e 0 2024-09-16T00:26:00.400 INFO:tasks.workunit.client.0.smithi112.stdout:5/415: chown d4/d7/d26/d3e/d7c 295693 1 2024-09-16T00:26:00.400 INFO:tasks.workunit.client.0.smithi112.stdout:4/463: chown d6/db/df/d5e/d61/d64/d6a/l9f 115 1 2024-09-16T00:26:00.401 INFO:tasks.workunit.client.0.smithi112.stdout:6/475: dwrite dd/df/d10/d47/f5b [0,4194304] 0 2024-09-16T00:26:00.402 INFO:tasks.workunit.client.1.smithi169.stdout:5/294: mknod d2/da/d10/d13/d1c/d40/d54/c65 0 2024-09-16T00:26:00.403 INFO:tasks.workunit.client.0.smithi112.stdout:5/416: chown d4/dd/d12/f11 7 1 2024-09-16T00:26:00.403 INFO:tasks.workunit.client.0.smithi112.stdout:1/478: truncate d0/f67 744513 0 2024-09-16T00:26:00.403 INFO:tasks.workunit.client.0.smithi112.stdout:4/464: write d6/d39/d36/d5c/d7e/d8f/f98 [882783,76288] 0 2024-09-16T00:26:00.404 INFO:tasks.workunit.client.1.smithi169.stdout:7/425: getdents d7/d16/d49/d84/d8e 0 2024-09-16T00:26:00.404 INFO:tasks.workunit.client.1.smithi169.stdout:5/295: write d2/da/d10/d13/d24/f2c [1425928,55921] 0 2024-09-16T00:26:00.406 INFO:tasks.workunit.client.0.smithi112.stdout:4/465: chown d6/db/d43/l5a 21678 1 2024-09-16T00:26:00.406 INFO:tasks.workunit.client.1.smithi169.stdout:5/296: dread - d2/da/d10/d13/d1c/d40/f5b zero size 2024-09-16T00:26:00.407 INFO:tasks.workunit.client.1.smithi169.stdout:0/338: getdents d0/d2 0 2024-09-16T00:26:00.407 INFO:tasks.workunit.client.1.smithi169.stdout:6/314: creat d2/d22/d51/f6a x:0 0 0 2024-09-16T00:26:00.407 INFO:tasks.workunit.client.0.smithi112.stdout:5/417: rename d4/d1c/l3c to d4/dd/d49/d70/l92 0 2024-09-16T00:26:00.408 INFO:tasks.workunit.client.1.smithi169.stdout:7/426: creat d7/d16/f90 x:0 0 0 2024-09-16T00:26:00.408 INFO:tasks.workunit.client.0.smithi112.stdout:4/466: mknod d6/db/df/d5e/d61/d64/d6a/ca7 0 2024-09-16T00:26:00.411 INFO:tasks.workunit.client.0.smithi112.stdout:5/418: creat d4/d7/d14/d90/f93 x:0 0 0 2024-09-16T00:26:00.412 INFO:tasks.workunit.client.1.smithi169.stdout:6/315: write d2/d3f/d41/f46 [677384,22686] 0 2024-09-16T00:26:00.413 INFO:tasks.workunit.client.0.smithi112.stdout:5/419: write d4/d7/d26/d36/f41 [705930,119877] 0 2024-09-16T00:26:00.413 INFO:tasks.workunit.client.1.smithi169.stdout:7/427: rename d7/f31 to d7/f91 0 2024-09-16T00:26:00.413 INFO:tasks.workunit.client.1.smithi169.stdout:7/428: readlink d7/d16/l66 0 2024-09-16T00:26:00.414 INFO:tasks.workunit.client.1.smithi169.stdout:0/339: mkdir d0/d2c/d54/d21/d31/d83 0 2024-09-16T00:26:00.416 INFO:tasks.workunit.client.0.smithi112.stdout:4/467: link d6/db/df/d5e/d61/d6d/f7f d6/d39/d1c/d2c/d69/d7b/fa8 0 2024-09-16T00:26:00.416 INFO:tasks.workunit.client.1.smithi169.stdout:7/429: dread - d7/d16/f90 zero size 2024-09-16T00:26:00.417 INFO:tasks.workunit.client.0.smithi112.stdout:6/476: dread dd/df/d13/d7c/f4a [0,4194304] 0 2024-09-16T00:26:00.419 INFO:tasks.workunit.client.1.smithi169.stdout:6/316: mknod d2/d4d/d59/d43/d67/c6b 0 2024-09-16T00:26:00.419 INFO:tasks.workunit.client.1.smithi169.stdout:5/297: dwrite d2/da/d10/d13/d1c/d40/d54/f56 [0,4194304] 0 2024-09-16T00:26:00.420 INFO:tasks.workunit.client.0.smithi112.stdout:7/478: dread d6/d10/d1b/d39/f43 [0,4194304] 0 2024-09-16T00:26:00.421 INFO:tasks.workunit.client.0.smithi112.stdout:5/420: rename d4/dd/d12/l31 to d4/dd/d49/d70/d83/l94 0 2024-09-16T00:26:00.422 INFO:tasks.workunit.client.0.smithi112.stdout:7/479: write d6/d10/d16/f3a [1202598,73312] 0 2024-09-16T00:26:00.422 INFO:tasks.workunit.client.0.smithi112.stdout:2/540: fdatasync d2/dd/f87 0 2024-09-16T00:26:00.423 INFO:tasks.workunit.client.1.smithi169.stdout:0/340: rename d0/d2/d8/f37 to d0/d2c/d38/f84 0 2024-09-16T00:26:00.425 INFO:tasks.workunit.client.1.smithi169.stdout:5/298: dread d2/da/d10/d13/d1c/f3b [0,4194304] 0 2024-09-16T00:26:00.428 INFO:tasks.workunit.client.1.smithi169.stdout:6/317: chown d2/d22/c4b 0 1 2024-09-16T00:26:00.428 INFO:tasks.workunit.client.1.smithi169.stdout:0/341: creat d0/d2c/d54/d21/d31/f85 x:0 0 0 2024-09-16T00:26:00.429 INFO:tasks.workunit.client.0.smithi112.stdout:2/541: creat d2/dd/d48/d22/d67/fa7 x:0 0 0 2024-09-16T00:26:00.429 INFO:tasks.workunit.client.0.smithi112.stdout:1/479: write d0/d14/df/d28/f32 [1012979,15922] 0 2024-09-16T00:26:00.430 INFO:tasks.workunit.client.1.smithi169.stdout:7/430: dwrite d7/d71/d76/d3a/f7a [0,4194304] 0 2024-09-16T00:26:00.431 INFO:tasks.workunit.client.0.smithi112.stdout:2/542: chown d2/dd/d48/d22/d2b/d54/d79/f9e 0 1 2024-09-16T00:26:00.432 INFO:tasks.workunit.client.1.smithi169.stdout:6/318: write d2/f1f [5075412,68635] 0 2024-09-16T00:26:00.434 INFO:tasks.workunit.client.0.smithi112.stdout:7/480: rename d6/d10/d16/d49/l4f to d6/d10/d16/d96/la9 0 2024-09-16T00:26:00.434 INFO:tasks.workunit.client.1.smithi169.stdout:7/431: dread d7/d71/d76/f37 [0,4194304] 0 2024-09-16T00:26:00.435 INFO:tasks.workunit.client.0.smithi112.stdout:1/480: creat d0/d14/df/d28/f9d x:0 0 0 2024-09-16T00:26:00.436 INFO:tasks.workunit.client.0.smithi112.stdout:5/421: creat d4/dd/f95 x:0 0 0 2024-09-16T00:26:00.436 INFO:tasks.workunit.client.0.smithi112.stdout:1/481: dread - d0/d14/df/d28/d45/d46/f90 zero size 2024-09-16T00:26:00.437 INFO:tasks.workunit.client.0.smithi112.stdout:2/543: unlink d2/lb 0 2024-09-16T00:26:00.437 INFO:tasks.workunit.client.0.smithi112.stdout:4/468: getdents d6/db/df/d5e/d61/d64/d6a 0 2024-09-16T00:26:00.438 INFO:tasks.workunit.client.1.smithi169.stdout:0/342: rename d0/d2c/d54/d16 to d0/d2/d4b/d86 0 2024-09-16T00:26:00.438 INFO:tasks.workunit.client.0.smithi112.stdout:7/481: mknod d6/d10/d16/d49/d6c/caa 0 2024-09-16T00:26:00.439 INFO:tasks.workunit.client.0.smithi112.stdout:4/469: write d6/db/df/d5e/d61/f3f [1412809,59184] 0 2024-09-16T00:26:00.440 INFO:tasks.workunit.client.1.smithi169.stdout:5/299: getdents d2/da/d10 0 2024-09-16T00:26:00.440 INFO:tasks.workunit.client.0.smithi112.stdout:4/470: chown d6/d39/d1c/d6e/d6f 165452 1 2024-09-16T00:26:00.441 INFO:tasks.workunit.client.0.smithi112.stdout:5/422: creat d4/dd/d49/f96 x:0 0 0 2024-09-16T00:26:00.442 INFO:tasks.workunit.client.1.smithi169.stdout:0/343: write d0/d2c/d54/f3b [1957452,50862] 0 2024-09-16T00:26:00.443 INFO:tasks.workunit.client.0.smithi112.stdout:7/482: creat d6/d10/d1b/d39/fab x:0 0 0 2024-09-16T00:26:00.443 INFO:tasks.workunit.client.0.smithi112.stdout:2/544: dread - d2/f39 zero size 2024-09-16T00:26:00.444 INFO:tasks.workunit.client.1.smithi169.stdout:6/319: dread d2/d3f/d41/d52/f1e [0,4194304] 0 2024-09-16T00:26:00.445 INFO:tasks.workunit.client.1.smithi169.stdout:5/300: write d2/da/d10/d13/d24/f3a [834576,117127] 0 2024-09-16T00:26:00.446 INFO:tasks.workunit.client.1.smithi169.stdout:4/317: dwrite d2/d12/d13/d1f/f20 [0,4194304] 0 2024-09-16T00:26:00.448 INFO:tasks.workunit.client.0.smithi112.stdout:4/471: link d6/db/df/d5e/d61/l48 d6/d39/d36/d5c/la9 0 2024-09-16T00:26:00.449 INFO:tasks.workunit.client.0.smithi112.stdout:4/472: write d6/db/df/d3c/f73 [1005086,119807] 0 2024-09-16T00:26:00.450 INFO:tasks.workunit.client.1.smithi169.stdout:6/320: write d2/d4d/d59/d38/f4f [3365246,51576] 0 2024-09-16T00:26:00.455 INFO:tasks.workunit.client.0.smithi112.stdout:7/483: write d6/d10/f3d [1931451,93875] 0 2024-09-16T00:26:00.456 INFO:tasks.workunit.client.0.smithi112.stdout:2/545: write d2/dd/d48/d22/d2b/d30/f44 [2033030,72802] 0 2024-09-16T00:26:00.456 INFO:tasks.workunit.client.0.smithi112.stdout:4/473: dwrite d6/f1b [0,4194304] 0 2024-09-16T00:26:00.457 INFO:tasks.workunit.client.1.smithi169.stdout:5/301: dwrite d2/da/d10/d13/d1c/d2d/f63 [0,4194304] 0 2024-09-16T00:26:00.458 INFO:tasks.workunit.client.1.smithi169.stdout:7/432: write d7/d16/f40 [1439465,99846] 0 2024-09-16T00:26:00.460 INFO:tasks.workunit.client.1.smithi169.stdout:4/318: symlink d2/d12/d13/d1f/d2f/l65 0 2024-09-16T00:26:00.460 INFO:tasks.workunit.client.1.smithi169.stdout:0/344: write d0/d2/d8/f47 [4487207,51134] 0 2024-09-16T00:26:00.460 INFO:tasks.workunit.client.1.smithi169.stdout:6/321: creat d2/d36/d48/d5f/f6c x:0 0 0 2024-09-16T00:26:00.462 INFO:tasks.workunit.client.1.smithi169.stdout:7/433: rmdir d7/df 39 2024-09-16T00:26:00.463 INFO:tasks.workunit.client.1.smithi169.stdout:5/302: mknod d2/da/d10/d13/d1c/d2d/d46/c66 0 2024-09-16T00:26:00.463 INFO:tasks.workunit.client.0.smithi112.stdout:4/474: mkdir d6/db/df/d5e/d61/d6d/daa 0 2024-09-16T00:26:00.464 INFO:tasks.workunit.client.1.smithi169.stdout:7/434: stat d7/d16/f79 0 2024-09-16T00:26:00.464 INFO:tasks.workunit.client.1.smithi169.stdout:5/303: chown d2/da/d10/f27 26146 1 2024-09-16T00:26:00.464 INFO:tasks.workunit.client.0.smithi112.stdout:2/546: write d2/dd/d48/d22/d2b/d54/d79/f9e [670415,33971] 0 2024-09-16T00:26:00.465 INFO:tasks.workunit.client.1.smithi169.stdout:6/322: symlink d2/d3f/d41/l6d 0 2024-09-16T00:26:00.467 INFO:tasks.workunit.client.0.smithi112.stdout:1/482: dwrite d0/f48 [0,4194304] 0 2024-09-16T00:26:00.469 INFO:tasks.workunit.client.0.smithi112.stdout:4/475: mkdir d6/d39/d36/d5c/d7e/dab 0 2024-09-16T00:26:00.471 INFO:tasks.workunit.client.0.smithi112.stdout:5/423: dread d4/d7/d14/f1a [0,4194304] 0 2024-09-16T00:26:00.471 INFO:tasks.workunit.client.1.smithi169.stdout:5/304: write d2/da/d10/d13/d1c/d2d/f4f [44853,61047] 0 2024-09-16T00:26:00.472 INFO:tasks.workunit.client.1.smithi169.stdout:6/323: rmdir d2/d4d/d59/d43 39 2024-09-16T00:26:00.474 INFO:tasks.workunit.client.0.smithi112.stdout:2/547: dread d2/dd/d48/d94/d49/f71 [0,4194304] 0 2024-09-16T00:26:00.475 INFO:tasks.workunit.client.1.smithi169.stdout:0/345: write d0/d2c/d38/f66 [1211864,101029] 0 2024-09-16T00:26:00.476 INFO:tasks.workunit.client.0.smithi112.stdout:5/424: creat d4/d7/d26/d4c/f97 x:0 0 0 2024-09-16T00:26:00.476 INFO:tasks.workunit.client.0.smithi112.stdout:4/476: dread d6/d39/d1c/f2f [0,4194304] 0 2024-09-16T00:26:00.477 INFO:tasks.workunit.client.1.smithi169.stdout:6/324: rmdir d2 39 2024-09-16T00:26:00.477 INFO:tasks.workunit.client.0.smithi112.stdout:2/548: symlink d2/dd/d48/d8a/d9b/la8 0 2024-09-16T00:26:00.478 INFO:tasks.workunit.client.1.smithi169.stdout:7/435: dwrite d7/d16/d49/f58 [0,4194304] 0 2024-09-16T00:26:00.478 INFO:tasks.workunit.client.0.smithi112.stdout:5/425: mkdir d4/dd/d49/d70/d98 0 2024-09-16T00:26:00.481 INFO:tasks.workunit.client.1.smithi169.stdout:4/319: dwrite d2/d12/d13/d1f/f52 [0,4194304] 0 2024-09-16T00:26:00.481 INFO:tasks.workunit.client.0.smithi112.stdout:4/477: getdents d6 0 2024-09-16T00:26:00.482 INFO:tasks.workunit.client.0.smithi112.stdout:5/426: mkdir d4/d7/d26/d3e/d7c/d99 0 2024-09-16T00:26:00.485 INFO:tasks.workunit.client.0.smithi112.stdout:4/478: unlink d6/db/d1f/d41/d49/d31/f51 0 2024-09-16T00:26:00.487 INFO:tasks.workunit.client.0.smithi112.stdout:7/484: dread d6/f9 [0,4194304] 0 2024-09-16T00:26:00.488 INFO:tasks.workunit.client.0.smithi112.stdout:4/479: creat d6/d39/d1c/d66/fac x:0 0 0 2024-09-16T00:26:00.491 INFO:tasks.workunit.client.0.smithi112.stdout:5/427: dwrite d4/d7/d26/d4c/f6a [0,4194304] 0 2024-09-16T00:26:00.491 INFO:tasks.workunit.client.0.smithi112.stdout:4/480: truncate d6/db/f60 4761539 0 2024-09-16T00:26:00.494 INFO:tasks.workunit.client.0.smithi112.stdout:7/485: getdents d6/d10 0 2024-09-16T00:26:00.495 INFO:tasks.workunit.client.0.smithi112.stdout:1/483: write d0/d14/d1c/d20/f33 [1831060,11257] 0 2024-09-16T00:26:00.497 INFO:tasks.workunit.client.0.smithi112.stdout:7/486: symlink d6/d10/da1/da2/lac 0 2024-09-16T00:26:00.501 INFO:tasks.workunit.client.0.smithi112.stdout:4/481: dwrite d6/db/df/d3c/f73 [0,4194304] 0 2024-09-16T00:26:00.502 INFO:tasks.workunit.client.0.smithi112.stdout:7/487: write d6/d10/d75/d88/f27 [2868349,76618] 0 2024-09-16T00:26:00.504 INFO:tasks.workunit.client.0.smithi112.stdout:4/482: fdatasync d6/db/df/d3c/f82 0 2024-09-16T00:26:00.505 INFO:tasks.workunit.client.0.smithi112.stdout:7/488: write d6/d7f/f89 [415163,27627] 0 2024-09-16T00:26:00.506 INFO:tasks.workunit.client.0.smithi112.stdout:1/484: dwrite d0/d14/f11 [4194304,4194304] 0 2024-09-16T00:26:00.508 INFO:tasks.workunit.client.0.smithi112.stdout:2/549: dread d2/dd/d48/d22/d2b/d30/f44 [0,4194304] 0 2024-09-16T00:26:00.509 INFO:tasks.workunit.client.0.smithi112.stdout:1/485: chown d0/d14/d1c/f51 221939210 1 2024-09-16T00:26:00.512 INFO:tasks.workunit.client.0.smithi112.stdout:4/483: rmdir d6/da/d47 0 2024-09-16T00:26:00.513 INFO:tasks.workunit.client.0.smithi112.stdout:1/486: mkdir d0/d14/d6a/d9e 0 2024-09-16T00:26:00.513 INFO:tasks.workunit.client.0.smithi112.stdout:7/489: truncate d6/d10/d16/d49/d6c/f4a 825084 0 2024-09-16T00:26:00.514 INFO:tasks.workunit.client.0.smithi112.stdout:4/484: read - d6/d39/d1c/d66/f76 zero size 2024-09-16T00:26:00.514 INFO:tasks.workunit.client.0.smithi112.stdout:7/490: chown d6/d10/d1b/d39/c6f 4124977 1 2024-09-16T00:26:00.516 INFO:tasks.workunit.client.0.smithi112.stdout:4/485: mkdir d6/d7c/dad 0 2024-09-16T00:26:00.519 INFO:tasks.workunit.client.0.smithi112.stdout:1/487: dread d0/d14/d1c/d26/f3e [0,4194304] 0 2024-09-16T00:26:00.522 INFO:tasks.workunit.client.0.smithi112.stdout:1/488: creat d0/d14/df/d28/d45/f9f x:0 0 0 2024-09-16T00:26:00.524 INFO:tasks.workunit.client.0.smithi112.stdout:1/489: creat d0/d14/df/d28/d45/d46/d58/d62/fa0 x:0 0 0 2024-09-16T00:26:00.525 INFO:tasks.workunit.client.0.smithi112.stdout:1/490: chown d0/f9 140218 1 2024-09-16T00:26:00.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:00 smithi169 ceph-mon[34653]: pgmap v253: 65 pgs: 65 active+clean; 1.2 GiB data, 5.3 GiB used, 531 GiB / 536 GiB avail; 27 MiB/s rd, 153 MiB/s wr, 349 op/s 2024-09-16T00:26:00.527 INFO:tasks.workunit.client.0.smithi112.stdout:1/491: write d0/d14/d1c/d26/f8d [1018673,83786] 0 2024-09-16T00:26:00.529 INFO:tasks.workunit.client.0.smithi112.stdout:1/492: creat d0/d14/d1c/d20/fa1 x:0 0 0 2024-09-16T00:26:00.535 INFO:tasks.workunit.client.0.smithi112.stdout:7/491: dread d6/d10/d16/f59 [0,4194304] 0 2024-09-16T00:26:00.539 INFO:tasks.workunit.client.0.smithi112.stdout:1/493: write d0/d14/d1c/f51 [850084,71039] 0 2024-09-16T00:26:00.540 INFO:tasks.workunit.client.0.smithi112.stdout:7/492: truncate d6/f7b 531976 0 2024-09-16T00:26:00.541 INFO:tasks.workunit.client.0.smithi112.stdout:7/493: creat d6/d10/d1b/d39/fad x:0 0 0 2024-09-16T00:26:00.542 INFO:tasks.workunit.client.0.smithi112.stdout:7/494: chown d6/d10/d16/l17 61627 1 2024-09-16T00:26:00.542 INFO:tasks.workunit.client.0.smithi112.stdout:1/494: creat d0/d14/d1c/d26/fa2 x:0 0 0 2024-09-16T00:26:00.543 INFO:tasks.workunit.client.0.smithi112.stdout:7/495: write d6/d10/d75/d8f/f99 [962435,30954] 0 2024-09-16T00:26:00.545 INFO:tasks.workunit.client.0.smithi112.stdout:1/495: rename d0/d14/f11 to d0/d14/d1c/d20/d93/fa3 0 2024-09-16T00:26:00.546 INFO:tasks.workunit.client.0.smithi112.stdout:7/496: unlink d6/f2a 0 2024-09-16T00:26:00.546 INFO:tasks.workunit.client.0.smithi112.stdout:1/496: write d0/d14/d1c/d20/f21 [3929631,89473] 0 2024-09-16T00:26:00.549 INFO:tasks.workunit.client.0.smithi112.stdout:7/497: rmdir d6/d7f 39 2024-09-16T00:26:00.549 INFO:tasks.workunit.client.0.smithi112.stdout:0/553: sync 2024-09-16T00:26:00.549 INFO:tasks.workunit.client.0.smithi112.stdout:1/497: chown d0/d14/df/d28/d42/c8e 2928182 1 2024-09-16T00:26:00.553 INFO:tasks.workunit.client.0.smithi112.stdout:7/498: creat d6/fae x:0 0 0 2024-09-16T00:26:00.553 INFO:tasks.workunit.client.1.smithi169.stdout:5/305: write d2/da/d10/d13/d1c/d32/d53/d4e/d55/f5c [2369671,52777] 0 2024-09-16T00:26:00.554 INFO:tasks.workunit.client.0.smithi112.stdout:1/498: symlink d0/d14/la4 0 2024-09-16T00:26:00.556 INFO:tasks.workunit.client.1.smithi169.stdout:7/436: read d7/d16/d49/d74/f7f [45193,98340] 0 2024-09-16T00:26:00.557 INFO:tasks.workunit.client.0.smithi112.stdout:0/554: dwrite d5/da/d1d/d71/f5c [0,4194304] 0 2024-09-16T00:26:00.557 INFO:tasks.workunit.client.1.smithi169.stdout:5/306: stat d2/da/f38 0 2024-09-16T00:26:00.557 INFO:tasks.workunit.client.0.smithi112.stdout:1/499: write d0/d14/d1c/d26/fa2 [741857,22489] 0 2024-09-16T00:26:00.564 INFO:tasks.workunit.client.0.smithi112.stdout:0/555: mkdir d5/da/d18/d63/db2 0 2024-09-16T00:26:00.564 INFO:tasks.workunit.client.0.smithi112.stdout:1/500: symlink d0/d14/d1c/d20/d93/la5 0 2024-09-16T00:26:00.564 INFO:tasks.workunit.client.1.smithi169.stdout:6/325: dwrite d2/d4d/d59/d38/f62 [0,4194304] 0 2024-09-16T00:26:00.566 INFO:tasks.workunit.client.1.smithi169.stdout:0/346: truncate d0/d2c/d54/d21/d31/f58 227658 0 2024-09-16T00:26:00.566 INFO:tasks.workunit.client.0.smithi112.stdout:0/556: dread - d5/da/d1d/d71/d64/f8b zero size 2024-09-16T00:26:00.567 INFO:tasks.workunit.client.1.smithi169.stdout:7/437: creat d7/f92 x:0 0 0 2024-09-16T00:26:00.568 INFO:tasks.workunit.client.0.smithi112.stdout:0/557: readlink d5/da/d21/d6f/l94 0 2024-09-16T00:26:00.568 INFO:tasks.workunit.client.1.smithi169.stdout:0/347: chown d0/f23 1613842 1 2024-09-16T00:26:00.569 INFO:tasks.workunit.client.1.smithi169.stdout:5/307: dread d2/da/d10/d13/d24/f3a [0,4194304] 0 2024-09-16T00:26:00.569 INFO:tasks.workunit.client.1.smithi169.stdout:4/320: dwrite d2/f3 [0,4194304] 0 2024-09-16T00:26:00.569 INFO:tasks.workunit.client.1.smithi169.stdout:6/326: creat d2/d3f/d41/d52/f6e x:0 0 0 2024-09-16T00:26:00.571 INFO:tasks.workunit.client.1.smithi169.stdout:0/348: creat d0/d4c/f87 x:0 0 0 2024-09-16T00:26:00.571 INFO:tasks.workunit.client.0.smithi112.stdout:7/499: dwrite d6/d10/f32 [0,4194304] 0 2024-09-16T00:26:00.572 INFO:tasks.workunit.client.0.smithi112.stdout:0/558: mknod d5/da/d18/d24/d46/d5f/da6/cb3 0 2024-09-16T00:26:00.572 INFO:tasks.workunit.client.1.smithi169.stdout:5/308: mkdir d2/da/d67 0 2024-09-16T00:26:00.573 INFO:tasks.workunit.client.0.smithi112.stdout:0/559: write d5/da/dc/f81 [84749,41886] 0 2024-09-16T00:26:00.574 INFO:tasks.workunit.client.1.smithi169.stdout:5/309: readlink d2/l12 0 2024-09-16T00:26:00.574 INFO:tasks.workunit.client.1.smithi169.stdout:7/438: mkdir d7/df/d93 0 2024-09-16T00:26:00.575 INFO:tasks.workunit.client.1.smithi169.stdout:6/327: rmdir d2/d4d/d59/d43/d67 39 2024-09-16T00:26:00.576 INFO:tasks.workunit.client.0.smithi112.stdout:0/560: chown d5/da/d66/ca7 28341 1 2024-09-16T00:26:00.577 INFO:tasks.workunit.client.1.smithi169.stdout:5/310: write d2/da/f19 [731565,118611] 0 2024-09-16T00:26:00.577 INFO:tasks.workunit.client.0.smithi112.stdout:7/500: creat d6/d10/d16/faf x:0 0 0 2024-09-16T00:26:00.586 INFO:tasks.workunit.client.1.smithi169.stdout:7/439: dwrite d7/d16/d49/d74/f7c [0,4194304] 0 2024-09-16T00:26:00.587 INFO:tasks.workunit.client.0.smithi112.stdout:7/501: read d6/d10/f53 [3785,104611] 0 2024-09-16T00:26:00.589 INFO:tasks.workunit.client.1.smithi169.stdout:6/328: dread d2/d36/f31 [4194304,4194304] 0 2024-09-16T00:26:00.589 INFO:tasks.workunit.client.0.smithi112.stdout:7/502: fsync d6/d10/f87 0 2024-09-16T00:26:00.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:00 smithi112 ceph-mon[27869]: pgmap v253: 65 pgs: 65 active+clean; 1.2 GiB data, 5.3 GiB used, 531 GiB / 536 GiB avail; 27 MiB/s rd, 153 MiB/s wr, 349 op/s 2024-09-16T00:26:00.589 INFO:tasks.workunit.client.1.smithi169.stdout:5/311: mkdir d2/da/d10/d13/d1c/d40/d68 0 2024-09-16T00:26:00.590 INFO:tasks.workunit.client.1.smithi169.stdout:7/440: write d7/d5b/f6b [971101,9880] 0 2024-09-16T00:26:00.595 INFO:tasks.workunit.client.1.smithi169.stdout:7/441: mknod d7/d5b/d61/c94 0 2024-09-16T00:26:00.596 INFO:tasks.workunit.client.1.smithi169.stdout:0/349: truncate d0/d2/d8/d5/f44 632868 0 2024-09-16T00:26:00.596 INFO:tasks.workunit.client.1.smithi169.stdout:7/442: truncate d7/f7e 422674 0 2024-09-16T00:26:00.597 INFO:tasks.workunit.client.0.smithi112.stdout:7/503: dwrite d6/d10/d1b/d39/fab [0,4194304] 0 2024-09-16T00:26:00.597 INFO:tasks.workunit.client.1.smithi169.stdout:6/329: creat d2/d4d/f6f x:0 0 0 2024-09-16T00:26:00.597 INFO:tasks.workunit.client.1.smithi169.stdout:7/443: chown d7/c34 200273217 1 2024-09-16T00:26:00.598 INFO:tasks.workunit.client.0.smithi112.stdout:7/504: truncate d6/fae 580206 0 2024-09-16T00:26:00.598 INFO:tasks.workunit.client.1.smithi169.stdout:5/312: creat d2/da/d10/d13/d1c/d40/f69 x:0 0 0 2024-09-16T00:26:00.599 INFO:tasks.workunit.client.1.smithi169.stdout:0/350: creat d0/d2/d4b/f88 x:0 0 0 2024-09-16T00:26:00.602 INFO:tasks.workunit.client.0.smithi112.stdout:7/505: getdents d6/d10/d75/d88 0 2024-09-16T00:26:00.603 INFO:tasks.workunit.client.0.smithi112.stdout:0/561: dread d5/da/d66/f5d [0,4194304] 0 2024-09-16T00:26:00.603 INFO:tasks.workunit.client.1.smithi169.stdout:5/313: creat d2/da/d10/d13/d1c/f6a x:0 0 0 2024-09-16T00:26:00.604 INFO:tasks.workunit.client.1.smithi169.stdout:7/444: dread d7/d16/f40 [0,4194304] 0 2024-09-16T00:26:00.604 INFO:tasks.workunit.client.0.smithi112.stdout:0/562: readlink d5/da/d21/d6f/l94 0 2024-09-16T00:26:00.607 INFO:tasks.workunit.client.1.smithi169.stdout:0/351: dread d0/d2/f45 [0,4194304] 0 2024-09-16T00:26:00.610 INFO:tasks.workunit.client.1.smithi169.stdout:6/330: dwrite d2/d4d/d59/d38/f4f [4194304,4194304] 0 2024-09-16T00:26:00.610 INFO:tasks.workunit.client.0.smithi112.stdout:7/506: rmdir d6 39 2024-09-16T00:26:00.612 INFO:tasks.workunit.client.1.smithi169.stdout:0/352: truncate d0/d2/d8/d5/f2e 661836 0 2024-09-16T00:26:00.613 INFO:tasks.workunit.client.0.smithi112.stdout:1/501: dwrite d0/f9 [0,4194304] 0 2024-09-16T00:26:00.614 INFO:tasks.workunit.client.1.smithi169.stdout:5/314: dwrite d2/fe [0,4194304] 0 2024-09-16T00:26:00.615 INFO:tasks.workunit.client.0.smithi112.stdout:7/507: creat d6/d10/d75/d88/fb0 x:0 0 0 2024-09-16T00:26:00.616 INFO:tasks.workunit.client.0.smithi112.stdout:1/502: symlink d0/d14/d1c/d26/d35/la6 0 2024-09-16T00:26:00.617 INFO:tasks.workunit.client.0.smithi112.stdout:1/503: fdatasync d0/d14/df/d28/f2e 0 2024-09-16T00:26:00.618 INFO:tasks.workunit.client.0.smithi112.stdout:7/508: creat d6/fb1 x:0 0 0 2024-09-16T00:26:00.619 INFO:tasks.workunit.client.0.smithi112.stdout:1/504: write d0/d14/d6a/f80 [675469,113762] 0 2024-09-16T00:26:00.619 INFO:tasks.workunit.client.0.smithi112.stdout:7/509: dread - d6/d10/d1b/f61 zero size 2024-09-16T00:26:00.620 INFO:tasks.workunit.client.1.smithi169.stdout:6/331: getdents d2/d22 0 2024-09-16T00:26:00.621 INFO:tasks.workunit.client.0.smithi112.stdout:7/510: write d6/d10/d1b/d39/fa3 [382467,101525] 0 2024-09-16T00:26:00.623 INFO:tasks.workunit.client.0.smithi112.stdout:1/505: mkdir d0/d14/df/d28/d45/d46/da7 0 2024-09-16T00:26:00.623 INFO:tasks.workunit.client.0.smithi112.stdout:1/506: readlink d0/d14/d6a/l86 0 2024-09-16T00:26:00.626 INFO:tasks.workunit.client.1.smithi169.stdout:0/353: dwrite d0/d2c/d54/f3b [0,4194304] 0 2024-09-16T00:26:00.627 INFO:tasks.workunit.client.1.smithi169.stdout:5/315: dwrite d2/da/f50 [0,4194304] 0 2024-09-16T00:26:00.629 INFO:tasks.workunit.client.1.smithi169.stdout:0/354: chown d0/d2c/d54/d56/l25 115442869 1 2024-09-16T00:26:00.629 INFO:tasks.workunit.client.1.smithi169.stdout:6/332: rmdir d2/d3f/d50/d60 0 2024-09-16T00:26:00.629 INFO:tasks.workunit.client.1.smithi169.stdout:5/316: write d2/da/d10/d13/f14 [4898194,2563] 0 2024-09-16T00:26:00.631 INFO:tasks.workunit.client.1.smithi169.stdout:6/333: chown d2/d4d/f57 65264566 1 2024-09-16T00:26:00.632 INFO:tasks.workunit.client.0.smithi112.stdout:1/507: dwrite d0/f87 [0,4194304] 0 2024-09-16T00:26:00.632 INFO:tasks.workunit.client.1.smithi169.stdout:6/334: stat d2/d36/l3a 0 2024-09-16T00:26:00.633 INFO:tasks.workunit.client.1.smithi169.stdout:0/355: fsync d0/d2/d4b/f52 0 2024-09-16T00:26:00.633 INFO:tasks.workunit.client.0.smithi112.stdout:7/511: fsync d6/d10/d75/d88/f70 0 2024-09-16T00:26:00.634 INFO:tasks.workunit.client.1.smithi169.stdout:5/317: dread d2/da/d10/d13/d1c/d32/d53/f4d [0,4194304] 0 2024-09-16T00:26:00.635 INFO:tasks.workunit.client.0.smithi112.stdout:7/512: chown d6/d10/d16/d49/d6c/c54 172878970 1 2024-09-16T00:26:00.640 INFO:tasks.workunit.client.0.smithi112.stdout:7/513: creat d6/fb2 x:0 0 0 2024-09-16T00:26:00.641 INFO:tasks.workunit.client.0.smithi112.stdout:7/514: stat d6/d10/d16/d49/d6c/l82 0 2024-09-16T00:26:00.641 INFO:tasks.workunit.client.1.smithi169.stdout:5/318: creat d2/f6b x:0 0 0 2024-09-16T00:26:00.642 INFO:tasks.workunit.client.1.smithi169.stdout:0/356: dwrite d0/d2/d4b/d86/f43 [0,4194304] 0 2024-09-16T00:26:00.642 INFO:tasks.workunit.client.1.smithi169.stdout:5/319: readlink d2/da/l1b 0 2024-09-16T00:26:00.642 INFO:tasks.workunit.client.0.smithi112.stdout:7/515: write d6/d10/d1b/d39/fa3 [453,82433] 0 2024-09-16T00:26:00.644 INFO:tasks.workunit.client.0.smithi112.stdout:7/516: write d6/d10/f87 [192468,42141] 0 2024-09-16T00:26:00.645 INFO:tasks.workunit.client.0.smithi112.stdout:7/517: stat d6/d10/d16/f59 0 2024-09-16T00:26:00.645 INFO:tasks.workunit.client.0.smithi112.stdout:7/518: chown d6/l37 5751571 1 2024-09-16T00:26:00.647 INFO:tasks.workunit.client.0.smithi112.stdout:7/519: symlink d6/d7f/lb3 0 2024-09-16T00:26:00.648 INFO:tasks.workunit.client.1.smithi169.stdout:5/320: symlink d2/da/d10/d13/d1c/d32/d53/d4e/d55/l6c 0 2024-09-16T00:26:00.649 INFO:tasks.workunit.client.0.smithi112.stdout:7/520: mkdir d6/d10/d16/db4 0 2024-09-16T00:26:00.649 INFO:tasks.workunit.client.0.smithi112.stdout:7/521: chown d6 463716 1 2024-09-16T00:26:00.649 INFO:tasks.workunit.client.1.smithi169.stdout:0/357: symlink d0/d2c/d54/l89 0 2024-09-16T00:26:00.650 INFO:tasks.workunit.client.0.smithi112.stdout:7/522: chown d6/d10/f76 0 1 2024-09-16T00:26:00.650 INFO:tasks.workunit.client.1.smithi169.stdout:0/358: chown d0/d2/d4b/f52 120431303 1 2024-09-16T00:26:00.651 INFO:tasks.workunit.client.1.smithi169.stdout:5/321: write d2/da/d10/d13/d1c/f6a [696262,1957] 0 2024-09-16T00:26:00.652 INFO:tasks.workunit.client.0.smithi112.stdout:7/523: creat d6/d7f/d85/fb5 x:0 0 0 2024-09-16T00:26:00.653 INFO:tasks.workunit.client.1.smithi169.stdout:0/359: rmdir d0/d4c 39 2024-09-16T00:26:00.656 INFO:tasks.workunit.client.1.smithi169.stdout:0/360: creat d0/d2c/d54/d21/d31/d83/f8a x:0 0 0 2024-09-16T00:26:00.660 INFO:tasks.workunit.client.1.smithi169.stdout:0/361: link d0/d2/d8/d5/d60/c63 d0/d4c/d82/c8b 0 2024-09-16T00:26:00.662 INFO:tasks.workunit.client.1.smithi169.stdout:0/362: dread d0/d2/d8/f47 [4194304,4194304] 0 2024-09-16T00:26:00.670 INFO:tasks.workunit.client.0.smithi112.stdout:3/518: sync 2024-09-16T00:26:00.670 INFO:tasks.workunit.client.0.smithi112.stdout:9/680: sync 2024-09-16T00:26:00.671 INFO:tasks.workunit.client.0.smithi112.stdout:3/519: fsync d2/d9/faa 0 2024-09-16T00:26:00.672 INFO:tasks.workunit.client.0.smithi112.stdout:9/681: mknod d1/d5/d6/cf0 0 2024-09-16T00:26:00.674 INFO:tasks.workunit.client.0.smithi112.stdout:3/520: link d2/d9/faa d2/d4d/fb3 0 2024-09-16T00:26:00.674 INFO:tasks.workunit.client.0.smithi112.stdout:9/682: mknod d1/d5/d20/d81/d1a/d88/cf1 0 2024-09-16T00:26:00.676 INFO:tasks.workunit.client.0.smithi112.stdout:9/683: dread d1/d5/d6/d59/d64/fa1 [0,4194304] 0 2024-09-16T00:26:00.678 INFO:tasks.workunit.client.0.smithi112.stdout:7/524: write d6/d10/d16/d3b/f5e [1520664,122029] 0 2024-09-16T00:26:00.679 INFO:tasks.workunit.client.0.smithi112.stdout:7/525: chown d6/d10 2181 1 2024-09-16T00:26:00.679 INFO:tasks.workunit.client.0.smithi112.stdout:9/684: mknod d1/d5/d20/d81/d1a/d88/def/de9/cf2 0 2024-09-16T00:26:00.682 INFO:tasks.workunit.client.0.smithi112.stdout:1/508: dwrite d0/d14/df/d28/d45/d30/f3a [0,4194304] 0 2024-09-16T00:26:00.683 INFO:tasks.workunit.client.0.smithi112.stdout:9/685: read d1/d5/d6/d2d/f3e [1004994,97060] 0 2024-09-16T00:26:00.685 INFO:tasks.workunit.client.0.smithi112.stdout:7/526: creat d6/d10/d1b/fb6 x:0 0 0 2024-09-16T00:26:00.692 INFO:tasks.workunit.client.0.smithi112.stdout:7/527: rename d6/d7f/lb3 to d6/d10/d75/d8f/d90/lb7 0 2024-09-16T00:26:00.692 INFO:tasks.workunit.client.0.smithi112.stdout:1/509: dwrite d0/f9 [0,4194304] 0 2024-09-16T00:26:00.696 INFO:tasks.workunit.client.0.smithi112.stdout:1/510: creat d0/d14/d1c/d20/fa8 x:0 0 0 2024-09-16T00:26:00.696 INFO:tasks.workunit.client.0.smithi112.stdout:9/686: dwrite d1/d5/d20/d81/faa [0,4194304] 0 2024-09-16T00:26:00.703 INFO:tasks.workunit.client.0.smithi112.stdout:1/511: unlink d0/d14/df/d28/d45/d46/d58/f7b 0 2024-09-16T00:26:00.705 INFO:tasks.workunit.client.0.smithi112.stdout:1/512: chown d0/d14/d6a/f80 2327505 1 2024-09-16T00:26:00.705 INFO:tasks.workunit.client.0.smithi112.stdout:9/687: dwrite d1/d5/d20/d81/d1a/d88/def/f8a [4194304,4194304] 0 2024-09-16T00:26:00.706 INFO:tasks.workunit.client.0.smithi112.stdout:1/513: chown d0/d14/df/d28 7620 1 2024-09-16T00:26:00.716 INFO:tasks.workunit.client.0.smithi112.stdout:7/528: truncate d6/d10/d16/f4e 6119530 0 2024-09-16T00:26:00.716 INFO:tasks.workunit.client.0.smithi112.stdout:1/514: rename d0/d14/d1c/d26/f3e to d0/d14/d1c/d20/fa9 0 2024-09-16T00:26:00.718 INFO:tasks.workunit.client.0.smithi112.stdout:1/515: stat d0/d14/df/d28/c77 0 2024-09-16T00:26:00.719 INFO:tasks.workunit.client.0.smithi112.stdout:9/688: dwrite d1/d5/d20/d81/d1a/d88/def/f43 [0,4194304] 0 2024-09-16T00:26:00.720 INFO:tasks.workunit.client.0.smithi112.stdout:7/529: creat d6/d10/d75/d8f/d90/fb8 x:0 0 0 2024-09-16T00:26:00.723 INFO:tasks.workunit.client.1.smithi169.stdout:1/262: sync 2024-09-16T00:26:00.723 INFO:tasks.workunit.client.0.smithi112.stdout:9/689: rmdir d1/d5/d6/d2d/d6a 39 2024-09-16T00:26:00.724 INFO:tasks.workunit.client.1.smithi169.stdout:1/263: chown dc/d26/l35 0 1 2024-09-16T00:26:00.728 INFO:tasks.workunit.client.0.smithi112.stdout:1/516: dwrite d0/d14/d1c/d20/f31 [0,4194304] 0 2024-09-16T00:26:00.731 INFO:tasks.workunit.client.1.smithi169.stdout:1/264: dwrite dc/d18/f3b [0,4194304] 0 2024-09-16T00:26:00.732 INFO:tasks.workunit.client.0.smithi112.stdout:1/517: rmdir d0/d14/df/d28/d45/d46/d58/d62 39 2024-09-16T00:26:00.734 INFO:tasks.workunit.client.0.smithi112.stdout:1/518: rmdir d0/d14/d1c/d26 39 2024-09-16T00:26:00.742 INFO:tasks.workunit.client.0.smithi112.stdout:1/519: dwrite d0/d14/df/d28/f5d [0,4194304] 0 2024-09-16T00:26:00.744 INFO:tasks.workunit.client.0.smithi112.stdout:9/690: write d1/d5/d20/d81/d1a/d88/def/f1c [532543,87720] 0 2024-09-16T00:26:00.746 INFO:tasks.workunit.client.0.smithi112.stdout:1/520: write d0/d14/df/d28/d45/d30/f9b [242027,24450] 0 2024-09-16T00:26:00.748 INFO:tasks.workunit.client.0.smithi112.stdout:1/521: dread - d0/d14/df/d28/d45/f9f zero size 2024-09-16T00:26:00.751 INFO:tasks.workunit.client.0.smithi112.stdout:1/522: chown d0/d14/d1c/f51 154832517 1 2024-09-16T00:26:00.754 INFO:tasks.workunit.client.0.smithi112.stdout:1/523: unlink d0/d14/d1c/d20/f21 0 2024-09-16T00:26:00.757 INFO:tasks.workunit.client.0.smithi112.stdout:1/524: mknod d0/d14/d44/caa 0 2024-09-16T00:26:00.760 INFO:tasks.workunit.client.0.smithi112.stdout:1/525: creat d0/d14/d6a/d9e/fab x:0 0 0 2024-09-16T00:26:00.763 INFO:tasks.workunit.client.0.smithi112.stdout:1/526: link d0/d14/d1c/c1f d0/d14/d1c/d20/cac 0 2024-09-16T00:26:00.765 INFO:tasks.workunit.client.0.smithi112.stdout:1/527: mknod d0/d14/df/d28/d89/cad 0 2024-09-16T00:26:00.766 INFO:tasks.workunit.client.0.smithi112.stdout:1/528: chown d0/d14/df/d28/d45/d46/c61 1 1 2024-09-16T00:26:00.769 INFO:tasks.workunit.client.1.smithi169.stdout:1/265: dread dc/d36/d3a/d45/f23 [0,4194304] 0 2024-09-16T00:26:00.769 INFO:tasks.workunit.client.0.smithi112.stdout:1/529: write d0/d14/df/d28/d45/d46/d58/d62/f74 [334297,83923] 0 2024-09-16T00:26:00.770 INFO:tasks.workunit.client.1.smithi169.stdout:1/266: mkdir dc/d36/d39/d50 0 2024-09-16T00:26:00.770 INFO:tasks.workunit.client.0.smithi112.stdout:1/530: mknod d0/d14/df/d28/d89/cae 0 2024-09-16T00:26:00.772 INFO:tasks.workunit.client.0.smithi112.stdout:1/531: rmdir d0/d14/d5e 39 2024-09-16T00:26:00.773 INFO:tasks.workunit.client.1.smithi169.stdout:1/267: rename dc/d36/d3a/d45/f16 to dc/d36/d4e/f51 0 2024-09-16T00:26:00.774 INFO:tasks.workunit.client.1.smithi169.stdout:1/268: creat dc/d36/d3a/d45/d29/f52 x:0 0 0 2024-09-16T00:26:00.776 INFO:tasks.workunit.client.1.smithi169.stdout:1/269: mknod dc/d26/c53 0 2024-09-16T00:26:00.777 INFO:tasks.workunit.client.0.smithi112.stdout:1/532: rename d0/d14/d1c/d26/d35 to d0/d14/d1c/d20/daf 0 2024-09-16T00:26:00.777 INFO:tasks.workunit.client.1.smithi169.stdout:1/270: read dc/d18/f2b [529373,108977] 0 2024-09-16T00:26:00.778 INFO:tasks.workunit.client.0.smithi112.stdout:1/533: fsync d0/d14/d1c/f51 0 2024-09-16T00:26:00.779 INFO:tasks.workunit.client.1.smithi169.stdout:1/271: creat dc/d36/d3a/d45/d29/d4d/f54 x:0 0 0 2024-09-16T00:26:00.779 INFO:tasks.workunit.client.0.smithi112.stdout:1/534: getdents d0/d14/df/d28/d45/d46/da7 0 2024-09-16T00:26:00.781 INFO:tasks.workunit.client.0.smithi112.stdout:1/535: mkdir d0/d14/df/d28/d45/d46/d58/d62/db0 0 2024-09-16T00:26:00.781 INFO:tasks.workunit.client.0.smithi112.stdout:1/536: chown d0/d14/d1c/d20/d93 61913 1 2024-09-16T00:26:00.785 INFO:tasks.workunit.client.1.smithi169.stdout:1/272: dwrite f7 [4194304,4194304] 0 2024-09-16T00:26:00.788 INFO:tasks.workunit.client.1.smithi169.stdout:1/273: getdents dc/d36/d3a 0 2024-09-16T00:26:00.797 INFO:tasks.workunit.client.1.smithi169.stdout:1/274: dwrite dc/d36/d3a/d45/f1c [0,4194304] 0 2024-09-16T00:26:00.814 INFO:tasks.workunit.client.1.smithi169.stdout:1/275: dread dc/d26/f41 [0,4194304] 0 2024-09-16T00:26:00.818 INFO:tasks.workunit.client.1.smithi169.stdout:1/276: dread dc/d36/d3a/d45/f1c [0,4194304] 0 2024-09-16T00:26:00.819 INFO:tasks.workunit.client.1.smithi169.stdout:1/277: creat dc/d26/f55 x:0 0 0 2024-09-16T00:26:00.820 INFO:tasks.workunit.client.1.smithi169.stdout:1/278: chown dc/d36/d4e/f51 224100 1 2024-09-16T00:26:00.820 INFO:tasks.workunit.client.1.smithi169.stdout:1/279: stat dc/d26 0 2024-09-16T00:26:00.821 INFO:tasks.workunit.client.1.smithi169.stdout:1/280: rmdir dc/d36/d3a 39 2024-09-16T00:26:00.824 INFO:tasks.workunit.client.1.smithi169.stdout:1/281: getdents dc/d36/d3a/d45/d29/d4d 0 2024-09-16T00:26:00.828 INFO:tasks.workunit.client.1.smithi169.stdout:1/282: dread dc/d36/d4e/f51 [0,4194304] 0 2024-09-16T00:26:00.829 INFO:tasks.workunit.client.1.smithi169.stdout:1/283: dread dc/d36/d4e/f51 [0,4194304] 0 2024-09-16T00:26:00.830 INFO:tasks.workunit.client.1.smithi169.stdout:1/284: read dc/f15 [3253907,27759] 0 2024-09-16T00:26:00.836 INFO:tasks.workunit.client.1.smithi169.stdout:1/285: dwrite dc/d26/f55 [0,4194304] 0 2024-09-16T00:26:00.836 INFO:tasks.workunit.client.1.smithi169.stdout:1/286: chown dc 266 1 2024-09-16T00:26:00.836 INFO:tasks.workunit.client.1.smithi169.stdout:1/287: chown dc/d36/d3a/d45/f1c 125845 1 2024-09-16T00:26:00.837 INFO:tasks.workunit.client.1.smithi169.stdout:2/244: sync 2024-09-16T00:26:00.841 INFO:tasks.workunit.client.1.smithi169.stdout:1/288: link dc/d36/d3a/d45/l1a dc/d36/d3a/d45/d29/d4d/l56 0 2024-09-16T00:26:00.843 INFO:tasks.workunit.client.1.smithi169.stdout:2/245: dwrite d7/db/f17 [0,4194304] 0 2024-09-16T00:26:00.844 INFO:tasks.workunit.client.1.smithi169.stdout:2/246: dread - d7/db/d27/f4d zero size 2024-09-16T00:26:00.847 INFO:tasks.workunit.client.1.smithi169.stdout:2/247: mkdir d7/db/d27/d4f 0 2024-09-16T00:26:00.848 INFO:tasks.workunit.client.1.smithi169.stdout:2/248: chown d7/db/d20/d4c 23597240 1 2024-09-16T00:26:00.849 INFO:tasks.workunit.client.1.smithi169.stdout:1/289: dwrite dc/f15 [4194304,4194304] 0 2024-09-16T00:26:00.852 INFO:tasks.workunit.client.1.smithi169.stdout:1/290: write f7 [8989367,66759] 0 2024-09-16T00:26:00.853 INFO:tasks.workunit.client.1.smithi169.stdout:1/291: readlink la 0 2024-09-16T00:26:00.859 INFO:tasks.workunit.client.1.smithi169.stdout:2/249: dwrite d7/db/f13 [4194304,4194304] 0 2024-09-16T00:26:00.861 INFO:tasks.workunit.client.1.smithi169.stdout:1/292: dread dc/d36/d3a/d45/f48 [0,4194304] 0 2024-09-16T00:26:00.863 INFO:tasks.workunit.client.1.smithi169.stdout:1/293: mknod dc/d36/d3a/d45/c57 0 2024-09-16T00:26:00.865 INFO:tasks.workunit.client.1.smithi169.stdout:1/294: chown dc/d36/d3a/d45/d29/d42/f49 205 1 2024-09-16T00:26:00.866 INFO:tasks.workunit.client.1.smithi169.stdout:2/250: dwrite d7/db/f17 [0,4194304] 0 2024-09-16T00:26:00.867 INFO:tasks.workunit.client.1.smithi169.stdout:1/295: symlink dc/d26/l58 0 2024-09-16T00:26:00.868 INFO:tasks.workunit.client.1.smithi169.stdout:2/251: symlink d7/db/d27/d4f/l50 0 2024-09-16T00:26:00.870 INFO:tasks.workunit.client.1.smithi169.stdout:2/252: mknod d7/d1c/d26/c51 0 2024-09-16T00:26:00.871 INFO:tasks.workunit.client.1.smithi169.stdout:1/296: dread dc/d18/f3b [0,4194304] 0 2024-09-16T00:26:00.873 INFO:tasks.workunit.client.1.smithi169.stdout:2/253: unlink d7/f1a 0 2024-09-16T00:26:00.876 INFO:tasks.workunit.client.1.smithi169.stdout:2/254: unlink d7/db/d27/f4d 0 2024-09-16T00:26:00.877 INFO:tasks.workunit.client.1.smithi169.stdout:1/297: link dc/d18/f3b dc/d18/f59 0 2024-09-16T00:26:00.879 INFO:tasks.workunit.client.1.smithi169.stdout:2/255: symlink d7/db/d27/d29/l52 0 2024-09-16T00:26:00.951 INFO:tasks.workunit.client.1.smithi169.stdout:9/364: sync 2024-09-16T00:26:00.951 INFO:tasks.workunit.client.1.smithi169.stdout:9/365: dread - d5/da/d2e/d47/f7d zero size 2024-09-16T00:26:00.952 INFO:tasks.workunit.client.1.smithi169.stdout:9/366: chown d5/da/d21/d23/f41 160986659 1 2024-09-16T00:26:00.952 INFO:tasks.workunit.client.1.smithi169.stdout:9/367: chown d5/da/d21/f2c 65 1 2024-09-16T00:26:00.957 INFO:tasks.workunit.client.1.smithi169.stdout:9/368: dwrite d5/d12/f5f [0,4194304] 0 2024-09-16T00:26:00.959 INFO:tasks.workunit.client.1.smithi169.stdout:9/369: chown d5/da/d21/d23/f67 1949466 1 2024-09-16T00:26:00.959 INFO:tasks.workunit.client.1.smithi169.stdout:9/370: write d5/f35 [128437,93283] 0 2024-09-16T00:26:00.961 INFO:tasks.workunit.client.1.smithi169.stdout:9/371: write d5/f14 [5185456,117586] 0 2024-09-16T00:26:00.969 INFO:tasks.workunit.client.1.smithi169.stdout:9/372: dwrite d5/da/d2e/f43 [0,4194304] 0 2024-09-16T00:26:00.972 INFO:tasks.workunit.client.1.smithi169.stdout:9/373: write d5/da/d2e/d47/f7d [364112,75423] 0 2024-09-16T00:26:00.979 INFO:tasks.workunit.client.1.smithi169.stdout:9/374: dwrite d5/da/f2b [0,4194304] 0 2024-09-16T00:26:01.000 INFO:tasks.workunit.client.1.smithi169.stdout:9/375: dread d5/da/d21/d23/f41 [0,4194304] 0 2024-09-16T00:26:01.003 INFO:tasks.workunit.client.1.smithi169.stdout:9/376: write d5/da/d21/d23/f2d [101555,87685] 0 2024-09-16T00:26:01.004 INFO:tasks.workunit.client.1.smithi169.stdout:4/321: sync 2024-09-16T00:26:01.004 INFO:tasks.workunit.client.1.smithi169.stdout:3/364: sync 2024-09-16T00:26:01.004 INFO:tasks.workunit.client.1.smithi169.stdout:8/301: sync 2024-09-16T00:26:01.004 INFO:tasks.workunit.client.1.smithi169.stdout:9/377: write d5/da/d21/d23/f67 [1022142,8401] 0 2024-09-16T00:26:01.007 INFO:tasks.workunit.client.1.smithi169.stdout:3/365: mkdir d7/d1a/d2b/d8d 0 2024-09-16T00:26:01.010 INFO:tasks.workunit.client.1.smithi169.stdout:4/322: link d2/d12/d13/d1f/d3e/f53 d2/d12/d13/d5c/f66 0 2024-09-16T00:26:01.011 INFO:tasks.workunit.client.1.smithi169.stdout:4/323: readlink d2/d12/l19 0 2024-09-16T00:26:01.014 INFO:tasks.workunit.client.1.smithi169.stdout:8/302: dwrite d7/dc/d2a/f42 [0,4194304] 0 2024-09-16T00:26:01.019 INFO:tasks.workunit.client.1.smithi169.stdout:3/366: dwrite d7/d50/f87 [0,4194304] 0 2024-09-16T00:26:01.021 INFO:tasks.workunit.client.1.smithi169.stdout:9/378: dread d5/f4a [0,4194304] 0 2024-09-16T00:26:01.022 INFO:tasks.workunit.client.1.smithi169.stdout:8/303: truncate d7/dc/d17/f28 3398250 0 2024-09-16T00:26:01.022 INFO:tasks.workunit.client.1.smithi169.stdout:9/379: chown d5/da/d2e/c88 25400265 1 2024-09-16T00:26:01.023 INFO:tasks.workunit.client.1.smithi169.stdout:8/304: dread d7/f2e [0,4194304] 0 2024-09-16T00:26:01.028 INFO:tasks.workunit.client.0.smithi112.stdout:6/477: sync 2024-09-16T00:26:01.028 INFO:tasks.workunit.client.0.smithi112.stdout:8/433: sync 2024-09-16T00:26:01.028 INFO:tasks.workunit.client.1.smithi169.stdout:9/380: rename d5/d49/f59 to d5/da/d4e/f89 0 2024-09-16T00:26:01.032 INFO:tasks.workunit.client.1.smithi169.stdout:8/305: creat d7/dc/d53/f61 x:0 0 0 2024-09-16T00:26:01.032 INFO:tasks.workunit.client.0.smithi112.stdout:8/434: creat d10/d12/d1c/d75/f8d x:0 0 0 2024-09-16T00:26:01.033 INFO:tasks.workunit.client.1.smithi169.stdout:8/306: chown d7/dc/d26/d46/f5d 1313189 1 2024-09-16T00:26:01.033 INFO:tasks.workunit.client.0.smithi112.stdout:8/435: fsync d10/d12/d1c/f31 0 2024-09-16T00:26:01.034 INFO:tasks.workunit.client.0.smithi112.stdout:8/436: chown d10/d4e/c5f 6 1 2024-09-16T00:26:01.035 INFO:tasks.workunit.client.0.smithi112.stdout:6/478: dwrite dd/d15/d34/f71 [0,4194304] 0 2024-09-16T00:26:01.039 INFO:tasks.workunit.client.1.smithi169.stdout:8/307: dwrite d7/f12 [0,4194304] 0 2024-09-16T00:26:01.041 INFO:tasks.workunit.client.1.smithi169.stdout:2/256: sync 2024-09-16T00:26:01.041 INFO:tasks.workunit.client.1.smithi169.stdout:6/335: sync 2024-09-16T00:26:01.041 INFO:tasks.workunit.client.1.smithi169.stdout:1/298: sync 2024-09-16T00:26:01.041 INFO:tasks.workunit.client.1.smithi169.stdout:5/322: sync 2024-09-16T00:26:01.041 INFO:tasks.workunit.client.1.smithi169.stdout:7/445: sync 2024-09-16T00:26:01.041 INFO:tasks.workunit.client.1.smithi169.stdout:0/363: sync 2024-09-16T00:26:01.042 INFO:tasks.workunit.client.0.smithi112.stdout:8/437: dwrite d10/d4e/d58/f66 [0,4194304] 0 2024-09-16T00:26:01.043 INFO:tasks.workunit.client.1.smithi169.stdout:9/381: rmdir d5/da/d4e 39 2024-09-16T00:26:01.048 INFO:tasks.workunit.client.1.smithi169.stdout:0/364: chown d0/d2c/d54/l72 1758273 1 2024-09-16T00:26:01.052 INFO:tasks.workunit.client.0.smithi112.stdout:8/438: dwrite d10/d12/d24/f29 [0,4194304] 0 2024-09-16T00:26:01.054 INFO:tasks.workunit.client.1.smithi169.stdout:4/324: dread d2/f9 [0,4194304] 0 2024-09-16T00:26:01.054 INFO:tasks.workunit.client.1.smithi169.stdout:1/299: chown dc/d36/d3a/d45/c57 5036293 1 2024-09-16T00:26:01.054 INFO:tasks.workunit.client.1.smithi169.stdout:8/308: rename d7/dc/l3d to d7/dc/d17/l62 0 2024-09-16T00:26:01.054 INFO:tasks.workunit.client.0.smithi112.stdout:8/439: chown d10/d12/d1c/d75/f3f 4693 1 2024-09-16T00:26:01.055 INFO:tasks.workunit.client.1.smithi169.stdout:9/382: rmdir d5/da/d21 39 2024-09-16T00:26:01.057 INFO:tasks.workunit.client.1.smithi169.stdout:2/257: creat d7/db/d27/d4f/f53 x:0 0 0 2024-09-16T00:26:01.058 INFO:tasks.workunit.client.1.smithi169.stdout:5/323: mkdir d2/da/d10/d13/d1c/d40/d54/d6d 0 2024-09-16T00:26:01.058 INFO:tasks.workunit.client.0.smithi112.stdout:8/440: creat d10/d4e/f8e x:0 0 0 2024-09-16T00:26:01.058 INFO:tasks.workunit.client.0.smithi112.stdout:8/441: fdatasync d10/d4e/f8e 0 2024-09-16T00:26:01.061 INFO:tasks.workunit.client.0.smithi112.stdout:8/442: creat d10/d12/f8f x:0 0 0 2024-09-16T00:26:01.061 INFO:tasks.workunit.client.0.smithi112.stdout:8/443: readlink d10/d12/d1c/d75/l47 0 2024-09-16T00:26:01.062 INFO:tasks.workunit.client.1.smithi169.stdout:8/309: symlink d7/dc/d1d/d3c/l63 0 2024-09-16T00:26:01.065 INFO:tasks.workunit.client.1.smithi169.stdout:4/325: dread - d2/d12/d13/d1f/d3e/d5b/d1a/f47 zero size 2024-09-16T00:26:01.065 INFO:tasks.workunit.client.1.smithi169.stdout:0/365: write d0/d2/f45 [857271,52865] 0 2024-09-16T00:26:01.065 INFO:tasks.workunit.client.1.smithi169.stdout:8/310: chown d7/dc/d1d 1 1 2024-09-16T00:26:01.066 INFO:tasks.workunit.client.1.smithi169.stdout:2/258: creat d7/db/d27/d29/f54 x:0 0 0 2024-09-16T00:26:01.066 INFO:tasks.workunit.client.1.smithi169.stdout:3/367: dread d7/d1a/d1c/d6c/f7c [0,4194304] 0 2024-09-16T00:26:01.068 INFO:tasks.workunit.client.1.smithi169.stdout:6/336: dwrite d2/d22/d51/f6a [0,4194304] 0 2024-09-16T00:26:01.070 INFO:tasks.workunit.client.1.smithi169.stdout:9/383: creat d5/da/d2e/f8a x:0 0 0 2024-09-16T00:26:01.071 INFO:tasks.workunit.client.1.smithi169.stdout:4/326: mknod d2/d12/d13/d4e/c67 0 2024-09-16T00:26:01.072 INFO:tasks.workunit.client.1.smithi169.stdout:6/337: truncate d2/d36/f2a 352315 0 2024-09-16T00:26:01.072 INFO:tasks.workunit.client.1.smithi169.stdout:1/300: getdents dc/d36/d3a/d45/d29/d4d 0 2024-09-16T00:26:01.073 INFO:tasks.workunit.client.1.smithi169.stdout:0/366: dread d0/d2c/d54/d21/d31/f58 [0,4194304] 0 2024-09-16T00:26:01.076 INFO:tasks.workunit.client.1.smithi169.stdout:5/324: rename d2/da/d10/d13/c3e to d2/da/d10/d13/d1c/d32/d53/d4e/d55/c6e 0 2024-09-16T00:26:01.077 INFO:tasks.workunit.client.1.smithi169.stdout:9/384: mknod d5/da/d2e/d47/c8b 0 2024-09-16T00:26:01.077 INFO:tasks.workunit.client.1.smithi169.stdout:5/325: write d2/f2a [49025,16678] 0 2024-09-16T00:26:01.078 INFO:tasks.workunit.client.1.smithi169.stdout:3/368: creat d7/d1a/d1c/d23/d70/f8e x:0 0 0 2024-09-16T00:26:01.080 INFO:tasks.workunit.client.1.smithi169.stdout:5/326: fdatasync d2/da/d10/d13/d1c/d2d/f4f 0 2024-09-16T00:26:01.080 INFO:tasks.workunit.client.1.smithi169.stdout:3/369: chown d7/d1a/d1c/d35/d63/d37 54 1 2024-09-16T00:26:01.080 INFO:tasks.workunit.client.1.smithi169.stdout:6/338: creat d2/d36/f70 x:0 0 0 2024-09-16T00:26:01.082 INFO:tasks.workunit.client.1.smithi169.stdout:4/327: rmdir d2/d12/d13/d1f/d3e/d5b 39 2024-09-16T00:26:01.083 INFO:tasks.workunit.client.1.smithi169.stdout:0/367: mkdir d0/d2c/d54/d21/d31/d8c 0 2024-09-16T00:26:01.083 INFO:tasks.workunit.client.1.smithi169.stdout:7/446: dread d7/df/f5d [0,4194304] 0 2024-09-16T00:26:01.084 INFO:tasks.workunit.client.1.smithi169.stdout:9/385: rmdir d5/da/d73 39 2024-09-16T00:26:01.085 INFO:tasks.workunit.client.1.smithi169.stdout:0/368: chown d0/d2/fd 12702228 1 2024-09-16T00:26:01.085 INFO:tasks.workunit.client.1.smithi169.stdout:0/369: chown d0/d2 1 1 2024-09-16T00:26:01.086 INFO:tasks.workunit.client.1.smithi169.stdout:5/327: mknod d2/da/c6f 0 2024-09-16T00:26:01.087 INFO:tasks.workunit.client.1.smithi169.stdout:3/370: unlink d7/d1a/d1c/d23/f65 0 2024-09-16T00:26:01.088 INFO:tasks.workunit.client.1.smithi169.stdout:3/371: readlink d7/d50/d6f/l80 0 2024-09-16T00:26:01.090 INFO:tasks.workunit.client.1.smithi169.stdout:4/328: unlink d2/d12/d13/d4e/d50/c51 0 2024-09-16T00:26:01.091 INFO:tasks.workunit.client.1.smithi169.stdout:4/329: chown d2/d12/d13/f60 7995 1 2024-09-16T00:26:01.093 INFO:tasks.workunit.client.1.smithi169.stdout:6/339: symlink d2/d4d/d59/l71 0 2024-09-16T00:26:01.093 INFO:tasks.workunit.client.1.smithi169.stdout:0/370: rmdir d0/d2c/d38 39 2024-09-16T00:26:01.095 INFO:tasks.workunit.client.1.smithi169.stdout:2/259: rename d7/db/d20/f44 to d7/f55 0 2024-09-16T00:26:01.099 INFO:tasks.workunit.client.1.smithi169.stdout:6/340: chown d2/d3f/d41/d52/c37 18985 1 2024-09-16T00:26:01.101 INFO:tasks.workunit.client.1.smithi169.stdout:2/260: write d7/db/d27/d4f/f53 [308157,127079] 0 2024-09-16T00:26:01.101 INFO:tasks.workunit.client.1.smithi169.stdout:3/372: dread d7/d1a/d1c/d6c/f7c [0,4194304] 0 2024-09-16T00:26:01.101 INFO:tasks.workunit.client.1.smithi169.stdout:8/311: rename d7/dc/d17/l2b to d7/dc/d26/d41/l64 0 2024-09-16T00:26:01.107 INFO:tasks.workunit.client.1.smithi169.stdout:9/386: dwrite d5/da/d21/f2c [0,4194304] 0 2024-09-16T00:26:01.107 INFO:tasks.workunit.client.1.smithi169.stdout:7/447: dread d7/f17 [0,4194304] 0 2024-09-16T00:26:01.109 INFO:tasks.workunit.client.1.smithi169.stdout:0/371: write d0/d2/d8/f47 [3831911,87599] 0 2024-09-16T00:26:01.110 INFO:tasks.workunit.client.1.smithi169.stdout:4/330: getdents d2/d12 0 2024-09-16T00:26:01.110 INFO:tasks.workunit.client.1.smithi169.stdout:7/448: write d7/d5b/d61/f6e [3238429,60853] 0 2024-09-16T00:26:01.110 INFO:tasks.workunit.client.1.smithi169.stdout:3/373: creat d7/f8f x:0 0 0 2024-09-16T00:26:01.111 INFO:tasks.workunit.client.1.smithi169.stdout:5/328: dread d2/fb [0,4194304] 0 2024-09-16T00:26:01.113 INFO:tasks.workunit.client.1.smithi169.stdout:1/301: rename c2 to dc/d36/d39/c5a 0 2024-09-16T00:26:01.117 INFO:tasks.workunit.client.1.smithi169.stdout:9/387: write d5/da/d21/d23/f41 [4420505,120904] 0 2024-09-16T00:26:01.118 INFO:tasks.workunit.client.1.smithi169.stdout:5/329: symlink d2/da/d10/d13/d1c/d32/l70 0 2024-09-16T00:26:01.121 INFO:tasks.workunit.client.1.smithi169.stdout:6/341: rename d2/d36/d48/d5f to d2/d3f/d41/d72 0 2024-09-16T00:26:01.121 INFO:tasks.workunit.client.1.smithi169.stdout:2/261: dread d7/db/f19 [4194304,4194304] 0 2024-09-16T00:26:01.124 INFO:tasks.workunit.client.1.smithi169.stdout:3/374: dwrite d7/d50/d6f/f71 [0,4194304] 0 2024-09-16T00:26:01.126 INFO:tasks.workunit.client.1.smithi169.stdout:9/388: creat d5/d12/d61/f8c x:0 0 0 2024-09-16T00:26:01.127 INFO:tasks.workunit.client.1.smithi169.stdout:8/312: rename d7/l1e to d7/dc/d1d/d3c/l65 0 2024-09-16T00:26:01.128 INFO:tasks.workunit.client.1.smithi169.stdout:5/330: readlink d2/da/d10/d13/d1c/d40/d43/l49 0 2024-09-16T00:26:01.129 INFO:tasks.workunit.client.1.smithi169.stdout:9/389: fdatasync d5/da/d2e/f43 0 2024-09-16T00:26:01.131 INFO:tasks.workunit.client.1.smithi169.stdout:3/375: mkdir d7/d50/d90 0 2024-09-16T00:26:01.131 INFO:tasks.workunit.client.1.smithi169.stdout:5/331: readlink d2/da/l1b 0 2024-09-16T00:26:01.134 INFO:tasks.workunit.client.1.smithi169.stdout:1/302: creat dc/d36/f5b x:0 0 0 2024-09-16T00:26:01.137 INFO:tasks.workunit.client.1.smithi169.stdout:0/372: write d0/d2/d4b/d86/f5a [873977,42139] 0 2024-09-16T00:26:01.143 INFO:tasks.workunit.client.1.smithi169.stdout:4/331: rename d2/d12/d13/d1f/d3e/d5b/ce to d2/d12/c68 0 2024-09-16T00:26:01.144 INFO:tasks.workunit.client.1.smithi169.stdout:2/262: dwrite f0 [0,4194304] 0 2024-09-16T00:26:01.149 INFO:tasks.workunit.client.1.smithi169.stdout:6/342: dwrite d2/d36/f2a [0,4194304] 0 2024-09-16T00:26:01.150 INFO:tasks.workunit.client.1.smithi169.stdout:9/390: chown d5/da/l69 1025626 1 2024-09-16T00:26:01.153 INFO:tasks.workunit.client.1.smithi169.stdout:9/391: stat d5/d12 0 2024-09-16T00:26:01.153 INFO:tasks.workunit.client.1.smithi169.stdout:3/376: dwrite d7/d50/d6f/f74 [0,4194304] 0 2024-09-16T00:26:01.155 INFO:tasks.workunit.client.1.smithi169.stdout:8/313: dwrite d7/dc/d2a/f30 [0,4194304] 0 2024-09-16T00:26:01.156 INFO:tasks.workunit.client.1.smithi169.stdout:6/343: write d2/f64 [389369,45360] 0 2024-09-16T00:26:01.157 INFO:tasks.workunit.client.1.smithi169.stdout:9/392: stat d5/d12/f5f 0 2024-09-16T00:26:01.157 INFO:tasks.workunit.client.1.smithi169.stdout:5/332: mkdir d2/da/d10/d13/d1c/d71 0 2024-09-16T00:26:01.158 INFO:tasks.workunit.client.1.smithi169.stdout:2/263: mkdir d7/db/d27/d4f/d56 0 2024-09-16T00:26:01.158 INFO:tasks.workunit.client.1.smithi169.stdout:6/344: fdatasync d2/d4d/f6f 0 2024-09-16T00:26:01.159 INFO:tasks.workunit.client.1.smithi169.stdout:8/314: dread - d7/dc/d1d/f5f zero size 2024-09-16T00:26:01.161 INFO:tasks.workunit.client.0.smithi112.stdout:2/550: sync 2024-09-16T00:26:01.161 INFO:tasks.workunit.client.0.smithi112.stdout:0/563: sync 2024-09-16T00:26:01.161 INFO:tasks.workunit.client.0.smithi112.stdout:3/521: sync 2024-09-16T00:26:01.161 INFO:tasks.workunit.client.0.smithi112.stdout:5/428: sync 2024-09-16T00:26:01.161 INFO:tasks.workunit.client.0.smithi112.stdout:4/486: sync 2024-09-16T00:26:01.162 INFO:tasks.workunit.client.0.smithi112.stdout:7/530: sync 2024-09-16T00:26:01.162 INFO:tasks.workunit.client.0.smithi112.stdout:9/691: sync 2024-09-16T00:26:01.162 INFO:tasks.workunit.client.0.smithi112.stdout:1/537: sync 2024-09-16T00:26:01.162 INFO:tasks.workunit.client.1.smithi169.stdout:0/373: symlink d0/d2c/d54/d21/d31/d8c/l8d 0 2024-09-16T00:26:01.162 INFO:tasks.workunit.client.1.smithi169.stdout:3/377: write d7/f16 [6051288,43737] 0 2024-09-16T00:26:01.163 INFO:tasks.workunit.client.0.smithi112.stdout:4/487: chown d6/c9d 3009 1 2024-09-16T00:26:01.165 INFO:tasks.workunit.client.0.smithi112.stdout:7/531: chown d6/d10/d1b/d39/fad 250 1 2024-09-16T00:26:01.166 INFO:tasks.workunit.client.0.smithi112.stdout:3/522: chown d2/d9/d2b/f29 15816 1 2024-09-16T00:26:01.166 INFO:tasks.workunit.client.0.smithi112.stdout:5/429: chown d4/dd/d12/l53 102054 1 2024-09-16T00:26:01.166 INFO:tasks.workunit.client.1.smithi169.stdout:1/303: dwrite dc/d36/d3a/d45/d29/d42/f49 [0,4194304] 0 2024-09-16T00:26:01.168 INFO:tasks.workunit.client.0.smithi112.stdout:9/692: creat d1/d5/d2c/d6f/ff3 x:0 0 0 2024-09-16T00:26:01.170 INFO:tasks.workunit.client.0.smithi112.stdout:1/538: write d0/d14/d1c/f5c [57152,124692] 0 2024-09-16T00:26:01.170 INFO:tasks.workunit.client.1.smithi169.stdout:2/264: write d7/db/d20/f3b [882262,72826] 0 2024-09-16T00:26:01.170 INFO:tasks.workunit.client.1.smithi169.stdout:0/374: creat d0/d2c/d54/d21/f8e x:0 0 0 2024-09-16T00:26:01.171 INFO:tasks.workunit.client.1.smithi169.stdout:5/333: mknod d2/da/d10/d13/d1c/d71/c72 0 2024-09-16T00:26:01.172 INFO:tasks.workunit.client.0.smithi112.stdout:3/523: readlink d2/d9/d2b/d2c/d69/l81 0 2024-09-16T00:26:01.172 INFO:tasks.workunit.client.1.smithi169.stdout:1/304: creat dc/d36/d3a/d45/d29/f5c x:0 0 0 2024-09-16T00:26:01.177 INFO:tasks.workunit.client.0.smithi112.stdout:5/430: read d4/d7/d14/f33 [281144,71585] 0 2024-09-16T00:26:01.177 INFO:tasks.workunit.client.1.smithi169.stdout:8/315: dread d7/f12 [0,4194304] 0 2024-09-16T00:26:01.179 INFO:tasks.workunit.client.1.smithi169.stdout:4/332: rename d2/d12/d13/d1f/d3e/f58 to d2/d12/d13/d1f/d3e/f69 0 2024-09-16T00:26:01.183 INFO:tasks.workunit.client.0.smithi112.stdout:3/524: mkdir d2/d9/d2b/d47/d52/d7a/d80/db4 0 2024-09-16T00:26:01.183 INFO:tasks.workunit.client.1.smithi169.stdout:3/378: truncate d7/d1a/d1c/f78 3923550 0 2024-09-16T00:26:01.184 INFO:tasks.workunit.client.0.smithi112.stdout:5/431: read - d4/d7/d26/f81 zero size 2024-09-16T00:26:01.185 INFO:tasks.workunit.client.0.smithi112.stdout:4/488: dwrite d6/d39/d36/d5c/d7e/f95 [0,4194304] 0 2024-09-16T00:26:01.186 INFO:tasks.workunit.client.1.smithi169.stdout:6/345: dwrite d2/d3f/d41/d5c/f65 [0,4194304] 0 2024-09-16T00:26:01.186 INFO:tasks.workunit.client.1.smithi169.stdout:0/375: chown d0/d2c/d38/l7b 27009 1 2024-09-16T00:26:01.187 INFO:tasks.workunit.client.0.smithi112.stdout:0/564: dwrite d5/da/d1d/d71/f4d [0,4194304] 0 2024-09-16T00:26:01.188 INFO:tasks.workunit.client.0.smithi112.stdout:7/532: dwrite d6/d10/d1b/d39/f9a [0,4194304] 0 2024-09-16T00:26:01.189 INFO:tasks.workunit.client.1.smithi169.stdout:6/346: write d2/f64 [265378,122220] 0 2024-09-16T00:26:01.190 INFO:tasks.workunit.client.0.smithi112.stdout:4/489: fsync d6/db/df/d3c/f73 0 2024-09-16T00:26:01.190 INFO:tasks.workunit.client.0.smithi112.stdout:0/565: chown d5/da/dc/f7d 201637623 1 2024-09-16T00:26:01.191 INFO:tasks.workunit.client.1.smithi169.stdout:6/347: fsync d2/d36/f70 0 2024-09-16T00:26:01.193 INFO:tasks.workunit.client.0.smithi112.stdout:0/566: chown d5/f7b 10193 1 2024-09-16T00:26:01.193 INFO:tasks.workunit.client.0.smithi112.stdout:7/533: write d6/d10/d16/d49/d6c/f5b [132025,103760] 0 2024-09-16T00:26:01.193 INFO:tasks.workunit.client.0.smithi112.stdout:5/432: rename d4/d7/d26/d3e to d4/d7/d14/d90/d9a 0 2024-09-16T00:26:01.194 INFO:tasks.workunit.client.0.smithi112.stdout:5/433: chown d4/d7/d14 2781300 1 2024-09-16T00:26:01.194 INFO:tasks.workunit.client.0.smithi112.stdout:4/490: rmdir d6/d39 39 2024-09-16T00:26:01.195 INFO:tasks.workunit.client.0.smithi112.stdout:5/434: stat d4/fb 0 2024-09-16T00:26:01.196 INFO:tasks.workunit.client.1.smithi169.stdout:3/379: mknod d7/d1a/d1c/d23/d88/c91 0 2024-09-16T00:26:01.196 INFO:tasks.workunit.client.1.smithi169.stdout:4/333: read d2/d12/f1e [3102533,14902] 0 2024-09-16T00:26:01.197 INFO:tasks.workunit.client.1.smithi169.stdout:5/334: dwrite d2/da/d10/d13/d1c/d32/d53/d4e/d55/f5c [0,4194304] 0 2024-09-16T00:26:01.197 INFO:tasks.workunit.client.1.smithi169.stdout:0/376: unlink d0/d2c/d54/l1b 0 2024-09-16T00:26:01.198 INFO:tasks.workunit.client.0.smithi112.stdout:0/567: rename d5/f2a to d5/da/d18/d24/d46/d88/fb4 0 2024-09-16T00:26:01.200 INFO:tasks.workunit.client.0.smithi112.stdout:9/693: write d1/d5/d6/d18/f28 [1050301,74449] 0 2024-09-16T00:26:01.201 INFO:tasks.workunit.client.0.smithi112.stdout:4/491: write d6/d39/d36/d5c/d7e/f86 [407954,50033] 0 2024-09-16T00:26:01.201 INFO:tasks.workunit.client.0.smithi112.stdout:0/568: chown d5/da/d18/c7f 478 1 2024-09-16T00:26:01.202 INFO:tasks.workunit.client.1.smithi169.stdout:8/316: creat d7/dc/d17/f66 x:0 0 0 2024-09-16T00:26:01.203 INFO:tasks.workunit.client.1.smithi169.stdout:4/334: write d2/d12/d13/d1f/d3e/d5b/d1a/f47 [700084,305] 0 2024-09-16T00:26:01.203 INFO:tasks.workunit.client.0.smithi112.stdout:7/534: rename d6/d10/f1c to d6/d10/d75/fb9 0 2024-09-16T00:26:01.208 INFO:tasks.workunit.client.0.smithi112.stdout:9/694: creat d1/ff4 x:0 0 0 2024-09-16T00:26:01.208 INFO:tasks.workunit.client.0.smithi112.stdout:4/492: symlink d6/db/df/d5e/d61/lae 0 2024-09-16T00:26:01.212 INFO:tasks.workunit.client.1.smithi169.stdout:9/393: dread d5/d12/f86 [0,4194304] 0 2024-09-16T00:26:01.213 INFO:tasks.workunit.client.0.smithi112.stdout:4/493: creat d6/d39/d1c/d66/faf x:0 0 0 2024-09-16T00:26:01.213 INFO:tasks.workunit.client.0.smithi112.stdout:0/569: dwrite d5/da/d18/d24/faf [0,4194304] 0 2024-09-16T00:26:01.215 INFO:tasks.workunit.client.1.smithi169.stdout:4/335: truncate d2/d12/d13/d1f/d3e/d5b/d1a/f40 259851 0 2024-09-16T00:26:01.216 INFO:tasks.workunit.client.1.smithi169.stdout:3/380: rename d7/d1a/d1c/d23/c66 to d7/d1a/d2b/d8d/c92 0 2024-09-16T00:26:01.217 INFO:tasks.workunit.client.1.smithi169.stdout:3/381: fdatasync d7/d50/f79 0 2024-09-16T00:26:01.217 INFO:tasks.workunit.client.0.smithi112.stdout:4/494: readlink d6/db/d1f/d94/l32 0 2024-09-16T00:26:01.219 INFO:tasks.workunit.client.0.smithi112.stdout:0/570: unlink d5/da/d18/c39 0 2024-09-16T00:26:01.222 INFO:tasks.workunit.client.0.smithi112.stdout:9/695: dwrite d1/d5/d2c/d6f/f8e [0,4194304] 0 2024-09-16T00:26:01.222 INFO:tasks.workunit.client.1.smithi169.stdout:4/336: write d2/d12/f31 [2637446,50528] 0 2024-09-16T00:26:01.223 INFO:tasks.workunit.client.1.smithi169.stdout:8/317: dread d7/f12 [0,4194304] 0 2024-09-16T00:26:01.224 INFO:tasks.workunit.client.1.smithi169.stdout:5/335: rename d2/da/d10/d13/d1c/d40/d43/l49 to d2/da/d10/d13/d1c/d32/d53/d4e/d55/l73 0 2024-09-16T00:26:01.225 INFO:tasks.workunit.client.1.smithi169.stdout:3/382: creat d7/d1a/d1c/d35/f93 x:0 0 0 2024-09-16T00:26:01.225 INFO:tasks.workunit.client.0.smithi112.stdout:4/495: rename d6/db/d1f/d41/d49/f75 to d6/d39/d1c/d6e/fb0 0 2024-09-16T00:26:01.227 INFO:tasks.workunit.client.1.smithi169.stdout:1/305: getdents dc/d2c/d3d 0 2024-09-16T00:26:01.228 INFO:tasks.workunit.client.1.smithi169.stdout:2/265: dread d7/db/f23 [0,4194304] 0 2024-09-16T00:26:01.230 INFO:tasks.workunit.client.1.smithi169.stdout:3/383: chown d7/d1a/d2b/f8c 3 1 2024-09-16T00:26:01.231 INFO:tasks.workunit.client.0.smithi112.stdout:9/696: unlink d1/d5/d6/f8f 0 2024-09-16T00:26:01.231 INFO:tasks.workunit.client.1.smithi169.stdout:8/318: dread d7/dc/d17/d24/f27 [0,4194304] 0 2024-09-16T00:26:01.233 INFO:tasks.workunit.client.1.smithi169.stdout:5/336: mkdir d2/da/d10/d13/d1c/d74 0 2024-09-16T00:26:01.234 INFO:tasks.workunit.client.1.smithi169.stdout:3/384: stat d7/d1a/d1c/f40 0 2024-09-16T00:26:01.235 INFO:tasks.workunit.client.0.smithi112.stdout:4/496: dread d6/d39/d1c/d66/f7d [0,4194304] 0 2024-09-16T00:26:01.237 INFO:tasks.workunit.client.1.smithi169.stdout:5/337: dread - d2/f61 zero size 2024-09-16T00:26:01.237 INFO:tasks.workunit.client.1.smithi169.stdout:9/394: dwrite d5/da/d21/d23/f41 [4194304,4194304] 0 2024-09-16T00:26:01.238 INFO:tasks.workunit.client.1.smithi169.stdout:8/319: chown d7/d20/l43 5873641 1 2024-09-16T00:26:01.238 INFO:tasks.workunit.client.0.smithi112.stdout:0/571: dread f0 [4194304,4194304] 0 2024-09-16T00:26:01.240 INFO:tasks.workunit.client.0.smithi112.stdout:4/497: stat d6/db/df/d5e/d61/l65 0 2024-09-16T00:26:01.240 INFO:tasks.workunit.client.0.smithi112.stdout:8/444: sync 2024-09-16T00:26:01.240 INFO:tasks.workunit.client.0.smithi112.stdout:6/479: sync 2024-09-16T00:26:01.240 INFO:tasks.workunit.client.0.smithi112.stdout:2/551: dread d2/fa [0,4194304] 0 2024-09-16T00:26:01.241 INFO:tasks.workunit.client.1.smithi169.stdout:1/306: stat dc/d36/d3a/d45/d29/d4d/l56 0 2024-09-16T00:26:01.241 INFO:tasks.workunit.client.0.smithi112.stdout:0/572: creat d5/da/d21/fb5 x:0 0 0 2024-09-16T00:26:01.241 INFO:tasks.workunit.client.0.smithi112.stdout:8/445: stat d10/d4e/d8c/f85 0 2024-09-16T00:26:01.242 INFO:tasks.workunit.client.0.smithi112.stdout:6/480: stat dd/df/d10/d47/d89/l92 0 2024-09-16T00:26:01.243 INFO:tasks.workunit.client.1.smithi169.stdout:5/338: dread - d2/da/d10/d13/d1c/d2d/f57 zero size 2024-09-16T00:26:01.243 INFO:tasks.workunit.client.1.smithi169.stdout:4/337: creat d2/d12/d13/d1f/f6a x:0 0 0 2024-09-16T00:26:01.244 INFO:tasks.workunit.client.0.smithi112.stdout:8/446: truncate d10/d4e/d58/f73 23116 0 2024-09-16T00:26:01.244 INFO:tasks.workunit.client.1.smithi169.stdout:3/385: creat d7/d1a/d2b/d8d/f94 x:0 0 0 2024-09-16T00:26:01.244 INFO:tasks.workunit.client.1.smithi169.stdout:8/320: fdatasync d7/f10 0 2024-09-16T00:26:01.245 INFO:tasks.workunit.client.1.smithi169.stdout:6/348: dread d2/d3f/d41/d52/f18 [0,4194304] 0 2024-09-16T00:26:01.246 INFO:tasks.workunit.client.1.smithi169.stdout:1/307: read dc/d18/f2b [81802,117243] 0 2024-09-16T00:26:01.247 INFO:tasks.workunit.client.0.smithi112.stdout:2/552: mkdir d2/dd/d48/d22/d2b/d54/d60/da9 0 2024-09-16T00:26:01.247 INFO:tasks.workunit.client.1.smithi169.stdout:8/321: readlink d7/dc/d1d/d3c/l63 0 2024-09-16T00:26:01.247 INFO:tasks.workunit.client.1.smithi169.stdout:5/339: symlink d2/da/l75 0 2024-09-16T00:26:01.248 INFO:tasks.workunit.client.0.smithi112.stdout:4/498: dwrite d6/db/df/d5e/d61/f3e [0,4194304] 0 2024-09-16T00:26:01.249 INFO:tasks.workunit.client.1.smithi169.stdout:5/340: write d2/f61 [238563,17947] 0 2024-09-16T00:26:01.251 INFO:tasks.workunit.client.1.smithi169.stdout:4/338: write d2/d12/f5d [887576,79968] 0 2024-09-16T00:26:01.252 INFO:tasks.workunit.client.0.smithi112.stdout:0/573: truncate d5/da/d57/db0/f79 657493 0 2024-09-16T00:26:01.253 INFO:tasks.workunit.client.0.smithi112.stdout:3/525: dread d2/d9/d2b/d47/d52/f38 [0,4194304] 0 2024-09-16T00:26:01.255 INFO:tasks.workunit.client.0.smithi112.stdout:8/447: creat d10/d12/d1c/d3b/d74/f90 x:0 0 0 2024-09-16T00:26:01.257 INFO:tasks.workunit.client.0.smithi112.stdout:2/553: creat d2/dd/d48/d22/d67/faa x:0 0 0 2024-09-16T00:26:01.263 INFO:tasks.workunit.client.0.smithi112.stdout:1/539: dread d0/d14/d1c/d20/daf/f59 [0,4194304] 0 2024-09-16T00:26:01.263 INFO:tasks.workunit.client.0.smithi112.stdout:3/526: dread - d2/d9/d2b/d47/d52/d7a/d80/d83/d76/faf zero size 2024-09-16T00:26:01.265 INFO:tasks.workunit.client.1.smithi169.stdout:6/349: dread d2/d3f/d41/d5c/f65 [0,4194304] 0 2024-09-16T00:26:01.266 INFO:tasks.workunit.client.1.smithi169.stdout:4/339: dread - d2/d12/d13/d1f/f6a zero size 2024-09-16T00:26:01.268 INFO:tasks.workunit.client.1.smithi169.stdout:8/322: dwrite f6 [8388608,4194304] 0 2024-09-16T00:26:01.268 INFO:tasks.workunit.client.0.smithi112.stdout:1/540: rmdir d0/d14/d1c/d26/d41 39 2024-09-16T00:26:01.268 INFO:tasks.workunit.client.0.smithi112.stdout:8/448: rename d10/d12/d1c/d75/f8d to d10/d4e/d58/f91 0 2024-09-16T00:26:01.268 INFO:tasks.workunit.client.0.smithi112.stdout:3/527: chown d2/d9/d2b/d39/fb1 180 1 2024-09-16T00:26:01.269 INFO:tasks.workunit.client.0.smithi112.stdout:5/435: dwrite d4/f19 [4194304,4194304] 0 2024-09-16T00:26:01.269 INFO:tasks.workunit.client.0.smithi112.stdout:2/554: read d2/dd/d48/f2a [2008729,89574] 0 2024-09-16T00:26:01.270 INFO:tasks.workunit.client.0.smithi112.stdout:7/535: symlink d6/d10/d16/d49/d6c/lba 0 2024-09-16T00:26:01.272 INFO:tasks.workunit.client.1.smithi169.stdout:1/308: dwrite dc/d36/d3a/d45/d29/d42/f49 [4194304,4194304] 0 2024-09-16T00:26:01.273 INFO:tasks.workunit.client.0.smithi112.stdout:9/697: write d1/d5/d6/d18/fb0 [757324,34447] 0 2024-09-16T00:26:01.275 INFO:tasks.workunit.client.0.smithi112.stdout:5/436: rmdir d4/dd 39 2024-09-16T00:26:01.275 INFO:tasks.workunit.client.0.smithi112.stdout:3/528: chown d2/d9/d2b/c5d 0 1 2024-09-16T00:26:01.275 INFO:tasks.workunit.client.0.smithi112.stdout:6/481: dread dd/df/d11/f49 [0,4194304] 0 2024-09-16T00:26:01.276 INFO:tasks.workunit.client.0.smithi112.stdout:4/499: dwrite d6/d39/d1c/d66/f7d [0,4194304] 0 2024-09-16T00:26:01.276 INFO:tasks.workunit.client.1.smithi169.stdout:2/266: dwrite d7/db/f19 [0,4194304] 0 2024-09-16T00:26:01.279 INFO:tasks.workunit.client.0.smithi112.stdout:8/449: mknod d10/d12/d1c/d3b/c92 0 2024-09-16T00:26:01.279 INFO:tasks.workunit.client.0.smithi112.stdout:9/698: chown d1/d5/d6/d38/f7b 0 1 2024-09-16T00:26:01.280 INFO:tasks.workunit.client.0.smithi112.stdout:8/450: stat d10/d4e/f6f 0 2024-09-16T00:26:01.280 INFO:tasks.workunit.client.1.smithi169.stdout:9/395: dread d5/da/ff [0,4194304] 0 2024-09-16T00:26:01.280 INFO:tasks.workunit.client.1.smithi169.stdout:3/386: dread d7/f9 [0,4194304] 0 2024-09-16T00:26:01.283 INFO:tasks.workunit.client.0.smithi112.stdout:5/437: mkdir d4/d7/d9b 0 2024-09-16T00:26:01.284 INFO:tasks.workunit.client.1.smithi169.stdout:4/340: dread d2/f49 [0,4194304] 0 2024-09-16T00:26:01.284 INFO:tasks.workunit.client.1.smithi169.stdout:9/396: readlink d5/d12/l1d 0 2024-09-16T00:26:01.284 INFO:tasks.workunit.client.1.smithi169.stdout:1/309: write dc/d2c/d3d/f47 [918345,94445] 0 2024-09-16T00:26:01.284 INFO:tasks.workunit.client.0.smithi112.stdout:2/555: rename d2/dd/d48/d22/f6a to d2/dd/d96/fab 0 2024-09-16T00:26:01.284 INFO:tasks.workunit.client.0.smithi112.stdout:5/438: read - d4/d7/d26/d4c/f97 zero size 2024-09-16T00:26:01.284 INFO:tasks.workunit.client.0.smithi112.stdout:7/536: creat d6/d10/d16/d49/fbb x:0 0 0 2024-09-16T00:26:01.287 INFO:tasks.workunit.client.0.smithi112.stdout:7/537: read d6/d10/d16/f3e [2055768,122941] 0 2024-09-16T00:26:01.289 INFO:tasks.workunit.client.0.smithi112.stdout:8/451: creat d10/d12/d1c/d61/d6c/f93 x:0 0 0 2024-09-16T00:26:01.290 INFO:tasks.workunit.client.0.smithi112.stdout:8/452: chown lb 0 1 2024-09-16T00:26:01.291 INFO:tasks.workunit.client.1.smithi169.stdout:8/323: mkdir d7/dc/d26/d41/d67 0 2024-09-16T00:26:01.294 INFO:tasks.workunit.client.0.smithi112.stdout:0/574: truncate d5/da/d21/da5/faa 1271852 0 2024-09-16T00:26:01.295 INFO:tasks.workunit.client.0.smithi112.stdout:4/500: dwrite d6/d39/d1c/d6e/fb0 [0,4194304] 0 2024-09-16T00:26:01.296 INFO:tasks.workunit.client.1.smithi169.stdout:2/267: chown d7/db/d20/d21/f22 9788918 1 2024-09-16T00:26:01.297 INFO:tasks.workunit.client.1.smithi169.stdout:6/350: write d2/d3f/d41/d52/f1b [2440248,61208] 0 2024-09-16T00:26:01.300 INFO:tasks.workunit.client.0.smithi112.stdout:2/556: symlink d2/dd/d48/d22/d2b/d54/d79/lac 0 2024-09-16T00:26:01.302 INFO:tasks.workunit.client.1.smithi169.stdout:9/397: mkdir d5/da/d21/d8d 0 2024-09-16T00:26:01.302 INFO:tasks.workunit.client.1.smithi169.stdout:8/324: dwrite d7/dc/d53/f61 [0,4194304] 0 2024-09-16T00:26:01.302 INFO:tasks.workunit.client.1.smithi169.stdout:1/310: truncate dc/d36/d3a/d45/f17 2008643 0 2024-09-16T00:26:01.303 INFO:tasks.workunit.client.0.smithi112.stdout:1/541: link d0/d14/d1c/d26/d47/l3f d0/d14/d1c/d20/daf/lb1 0 2024-09-16T00:26:01.304 INFO:tasks.workunit.client.1.smithi169.stdout:2/268: fsync d7/db/f23 0 2024-09-16T00:26:01.305 INFO:tasks.workunit.client.0.smithi112.stdout:9/699: creat d1/d5/d20/d81/d1a/d88/def/da9/ddd/ff5 x:0 0 0 2024-09-16T00:26:01.305 INFO:tasks.workunit.client.0.smithi112.stdout:1/542: read d0/f87 [1018815,86990] 0 2024-09-16T00:26:01.305 INFO:tasks.workunit.client.0.smithi112.stdout:0/575: mkdir d5/da/d1d/d71/d64/db6 0 2024-09-16T00:26:01.307 INFO:tasks.workunit.client.0.smithi112.stdout:6/482: rmdir dd/df/d50 0 2024-09-16T00:26:01.308 INFO:tasks.workunit.client.1.smithi169.stdout:6/351: rename d2/d3f/d50 to d2/d3f/d41/d5c/d73 0 2024-09-16T00:26:01.310 INFO:tasks.workunit.client.1.smithi169.stdout:6/352: chown d2/d4d/d59/d43/f4a 574464621 1 2024-09-16T00:26:01.310 INFO:tasks.workunit.client.0.smithi112.stdout:3/529: creat d2/d9/d2b/d47/d52/d7a/d80/d83/fb5 x:0 0 0 2024-09-16T00:26:01.310 INFO:tasks.workunit.client.0.smithi112.stdout:0/576: write d5/f95 [220792,26114] 0 2024-09-16T00:26:01.310 INFO:tasks.workunit.client.0.smithi112.stdout:1/543: chown d0/f87 245575 1 2024-09-16T00:26:01.311 INFO:tasks.workunit.client.1.smithi169.stdout:9/398: mknod d5/da/d5b/d6f/c8e 0 2024-09-16T00:26:01.313 INFO:tasks.workunit.client.1.smithi169.stdout:4/341: creat d2/d12/d13/d1f/d3e/d5b/f6b x:0 0 0 2024-09-16T00:26:01.313 INFO:tasks.workunit.client.0.smithi112.stdout:2/557: dwrite d2/dd/d48/d22/d2b/fa0 [0,4194304] 0 2024-09-16T00:26:01.314 INFO:tasks.workunit.client.1.smithi169.stdout:6/353: read d2/d4d/f33 [301313,87570] 0 2024-09-16T00:26:01.314 INFO:tasks.workunit.client.0.smithi112.stdout:8/453: rename d10/d12/d1c/d75/l81 to d10/d4e/d58/d87/l94 0 2024-09-16T00:26:01.315 INFO:tasks.workunit.client.1.smithi169.stdout:1/311: getdents dc/d2c 0 2024-09-16T00:26:01.316 INFO:tasks.workunit.client.0.smithi112.stdout:6/483: dread - dd/df/d17/d27/d84/d37/f74 zero size 2024-09-16T00:26:01.316 INFO:tasks.workunit.client.0.smithi112.stdout:9/700: fsync d1/d5/d20/d81/f1d 0 2024-09-16T00:26:01.317 INFO:tasks.workunit.client.1.smithi169.stdout:9/399: creat d5/f8f x:0 0 0 2024-09-16T00:26:01.318 INFO:tasks.workunit.client.0.smithi112.stdout:7/538: dread d6/d10/d16/d49/d6c/f33 [0,4194304] 0 2024-09-16T00:26:01.318 INFO:tasks.workunit.client.0.smithi112.stdout:5/439: getdents d4/d4f/d77 0 2024-09-16T00:26:01.321 INFO:tasks.workunit.client.0.smithi112.stdout:9/701: truncate d1/f82 4452194 0 2024-09-16T00:26:01.322 INFO:tasks.workunit.client.0.smithi112.stdout:4/501: dwrite f4 [0,4194304] 0 2024-09-16T00:26:01.322 INFO:tasks.workunit.client.0.smithi112.stdout:1/544: mknod d0/d14/d1c/d26/d47/d85/cb2 0 2024-09-16T00:26:01.324 INFO:tasks.workunit.client.0.smithi112.stdout:6/484: creat dd/df/d13/d7c/d88/f9b x:0 0 0 2024-09-16T00:26:01.324 INFO:tasks.workunit.client.1.smithi169.stdout:1/312: dwrite dc/d26/f55 [0,4194304] 0 2024-09-16T00:26:01.325 INFO:tasks.workunit.client.0.smithi112.stdout:5/440: truncate d4/d4f/d77/f7f 636729 0 2024-09-16T00:26:01.327 INFO:tasks.workunit.client.1.smithi169.stdout:5/341: dread d2/da/d10/d13/f35 [4194304,4194304] 0 2024-09-16T00:26:01.329 INFO:tasks.workunit.client.0.smithi112.stdout:4/502: write d6/d39/d1c/d66/faf [91366,109470] 0 2024-09-16T00:26:01.330 INFO:tasks.workunit.client.0.smithi112.stdout:3/530: rename d2/d9/d2b/d47/f30 to d2/d9/d2b/d47/d52/d7a/d80/d83/d76/d8e/fb6 0 2024-09-16T00:26:01.330 INFO:tasks.workunit.client.1.smithi169.stdout:5/342: creat d2/da/d10/d13/d1c/d40/d54/d6d/f76 x:0 0 0 2024-09-16T00:26:01.333 INFO:tasks.workunit.client.1.smithi169.stdout:5/343: mknod d2/da/d67/c77 0 2024-09-16T00:26:01.334 INFO:tasks.workunit.client.0.smithi112.stdout:7/539: mkdir d6/d10/d75/d8f/d90/dbc 0 2024-09-16T00:26:01.335 INFO:tasks.workunit.client.1.smithi169.stdout:1/313: dread dc/d2c/d3d/f47 [0,4194304] 0 2024-09-16T00:26:01.336 INFO:tasks.workunit.client.0.smithi112.stdout:5/441: write d4/dd/d49/d70/d83/f86 [4145950,98187] 0 2024-09-16T00:26:01.336 INFO:tasks.workunit.client.1.smithi169.stdout:5/344: mknod d2/da/d67/c78 0 2024-09-16T00:26:01.336 INFO:tasks.workunit.client.0.smithi112.stdout:4/503: chown l5 22 1 2024-09-16T00:26:01.337 INFO:tasks.workunit.client.0.smithi112.stdout:2/558: getdents d2/dd/d48/d22/d2b/d54 0 2024-09-16T00:26:01.338 INFO:tasks.workunit.client.1.smithi169.stdout:1/314: mkdir dc/d26/d5d 0 2024-09-16T00:26:01.339 INFO:tasks.workunit.client.1.smithi169.stdout:1/315: truncate dc/d36/d4e/f4f 650806 0 2024-09-16T00:26:01.341 INFO:tasks.workunit.client.0.smithi112.stdout:1/545: creat d0/d14/d1c/d26/fb3 x:0 0 0 2024-09-16T00:26:01.341 INFO:tasks.workunit.client.1.smithi169.stdout:5/345: unlink d2/da/d10/d13/d1c/d32/d53/d4e/d55/c6e 0 2024-09-16T00:26:01.341 INFO:tasks.workunit.client.0.smithi112.stdout:1/546: fsync d0/d14/df/d28/f5d 0 2024-09-16T00:26:01.341 INFO:tasks.workunit.client.0.smithi112.stdout:4/504: mkdir d6/db/d1f/d41/d49/db1 0 2024-09-16T00:26:01.342 INFO:tasks.workunit.client.1.smithi169.stdout:1/316: creat dc/d2c/d3d/f5e x:0 0 0 2024-09-16T00:26:01.343 INFO:tasks.workunit.client.0.smithi112.stdout:2/559: read d2/dd/d48/d22/f29 [1409910,110514] 0 2024-09-16T00:26:01.343 INFO:tasks.workunit.client.1.smithi169.stdout:1/317: chown dc/d26/f37 331 1 2024-09-16T00:26:01.344 INFO:tasks.workunit.client.1.smithi169.stdout:5/346: symlink d2/da/d10/d13/d1c/d40/d54/d6d/l79 0 2024-09-16T00:26:01.345 INFO:tasks.workunit.client.1.smithi169.stdout:1/318: write dc/d26/f46 [199146,126643] 0 2024-09-16T00:26:01.345 INFO:tasks.workunit.client.0.smithi112.stdout:7/540: rename d6/l8b to d6/d10/d1b/lbd 0 2024-09-16T00:26:01.346 INFO:tasks.workunit.client.0.smithi112.stdout:7/541: stat d6/d10/d16/d3b/l58 0 2024-09-16T00:26:01.346 INFO:tasks.workunit.client.0.smithi112.stdout:1/547: truncate d0/d14/df/d28/d45/d46/f49 52958 0 2024-09-16T00:26:01.347 INFO:tasks.workunit.client.1.smithi169.stdout:5/347: write d2/fd [985329,26596] 0 2024-09-16T00:26:01.347 INFO:tasks.workunit.client.0.smithi112.stdout:2/560: mkdir d2/dd/d48/d22/d2b/d54/d60/d6c/d88/dad 0 2024-09-16T00:26:01.350 INFO:tasks.workunit.client.1.smithi169.stdout:5/348: chown d2/l9 3062362 1 2024-09-16T00:26:01.351 INFO:tasks.workunit.client.1.smithi169.stdout:1/319: dread dc/d36/d3a/d45/f1c [0,4194304] 0 2024-09-16T00:26:01.354 INFO:tasks.workunit.client.0.smithi112.stdout:2/561: fdatasync d2/dd/d48/d94/d49/f71 0 2024-09-16T00:26:01.355 INFO:tasks.workunit.client.0.smithi112.stdout:4/505: dwrite d6/d39/d1c/f2f [0,4194304] 0 2024-09-16T00:26:01.355 INFO:tasks.workunit.client.1.smithi169.stdout:8/325: truncate f5 138637 0 2024-09-16T00:26:01.356 INFO:tasks.workunit.client.1.smithi169.stdout:4/342: dread d2/d12/f31 [0,4194304] 0 2024-09-16T00:26:01.358 INFO:tasks.workunit.client.1.smithi169.stdout:6/354: truncate d2/d22/d51/f6a 3456162 0 2024-09-16T00:26:01.360 INFO:tasks.workunit.client.1.smithi169.stdout:9/400: truncate d5/d12/f86 319071 0 2024-09-16T00:26:01.360 INFO:tasks.workunit.client.1.smithi169.stdout:8/326: read d7/dc/fe [8242439,110937] 0 2024-09-16T00:26:01.360 INFO:tasks.workunit.client.1.smithi169.stdout:6/355: read d2/d4d/f33 [174437,78946] 0 2024-09-16T00:26:01.361 INFO:tasks.workunit.client.1.smithi169.stdout:1/320: symlink dc/d36/l5f 0 2024-09-16T00:26:01.361 INFO:tasks.workunit.client.0.smithi112.stdout:1/548: getdents d0 0 2024-09-16T00:26:01.361 INFO:tasks.workunit.client.0.smithi112.stdout:7/542: dwrite d6/d10/f3d [4194304,4194304] 0 2024-09-16T00:26:01.363 INFO:tasks.workunit.client.1.smithi169.stdout:1/321: fdatasync dc/f21 0 2024-09-16T00:26:01.363 INFO:tasks.workunit.client.0.smithi112.stdout:9/702: dread d1/d5/d2c/d6f/f91 [0,4194304] 0 2024-09-16T00:26:01.366 INFO:tasks.workunit.client.0.smithi112.stdout:8/454: getdents d10/d4e/d58/d87 0 2024-09-16T00:26:01.368 INFO:tasks.workunit.client.0.smithi112.stdout:8/455: write d10/d4e/d8c/f85 [387039,82143] 0 2024-09-16T00:26:01.370 INFO:tasks.workunit.client.0.smithi112.stdout:1/549: symlink d0/d14/d1c/d20/d93/lb4 0 2024-09-16T00:26:01.370 INFO:tasks.workunit.client.0.smithi112.stdout:8/456: chown d10/d12/d1c/d75/l51 4 1 2024-09-16T00:26:01.371 INFO:tasks.workunit.client.0.smithi112.stdout:9/703: unlink d1/d5/f30 0 2024-09-16T00:26:01.371 INFO:tasks.workunit.client.1.smithi169.stdout:6/356: mkdir d2/d4d/d59/d74 0 2024-09-16T00:26:01.372 INFO:tasks.workunit.client.1.smithi169.stdout:8/327: mknod d7/dc/d26/d41/d67/c68 0 2024-09-16T00:26:01.373 INFO:tasks.workunit.client.0.smithi112.stdout:9/704: write d1/d5/d20/d81/d1a/f7d [3212493,38679] 0 2024-09-16T00:26:01.374 INFO:tasks.workunit.client.1.smithi169.stdout:1/322: mkdir dc/d36/d60 0 2024-09-16T00:26:01.375 INFO:tasks.workunit.client.1.smithi169.stdout:8/328: read d7/dc/d2a/f30 [3906713,129891] 0 2024-09-16T00:26:01.376 INFO:tasks.workunit.client.1.smithi169.stdout:6/357: mknod d2/d36/c75 0 2024-09-16T00:26:01.377 INFO:tasks.workunit.client.1.smithi169.stdout:8/329: stat d7/dc/d26/c31 0 2024-09-16T00:26:01.378 INFO:tasks.workunit.client.0.smithi112.stdout:6/485: write dd/d23/f64 [498203,77327] 0 2024-09-16T00:26:01.380 INFO:tasks.workunit.client.1.smithi169.stdout:6/358: symlink d2/d3f/d41/d52/l76 0 2024-09-16T00:26:01.381 INFO:tasks.workunit.client.0.smithi112.stdout:1/550: rename d0/d14/df/d28/d45/c34 to d0/d14/df/d28/d89/cb5 0 2024-09-16T00:26:01.381 INFO:tasks.workunit.client.0.smithi112.stdout:3/531: write d2/d9/d2b/d47/f8b [349454,110628] 0 2024-09-16T00:26:01.381 INFO:tasks.workunit.client.0.smithi112.stdout:1/551: stat d0/d14/d1c/d20/d93/la5 0 2024-09-16T00:26:01.382 INFO:tasks.workunit.client.1.smithi169.stdout:1/323: link dc/f24 dc/d36/d3a/d45/f61 0 2024-09-16T00:26:01.383 INFO:tasks.workunit.client.1.smithi169.stdout:9/401: dwrite d5/da/d4e/f6d [0,4194304] 0 2024-09-16T00:26:01.383 INFO:tasks.workunit.client.0.smithi112.stdout:7/543: link d6/f9 d6/d10/d1b/fbe 0 2024-09-16T00:26:01.383 INFO:tasks.workunit.client.0.smithi112.stdout:5/442: dread d4/dd/f27 [0,4194304] 0 2024-09-16T00:26:01.384 INFO:tasks.workunit.client.0.smithi112.stdout:8/457: mknod d10/d12/c95 0 2024-09-16T00:26:01.385 INFO:tasks.workunit.client.0.smithi112.stdout:4/506: link d6/db/df/d5e/d61/d6d/f7f d6/db/d1f/d41/fb2 0 2024-09-16T00:26:01.386 INFO:tasks.workunit.client.0.smithi112.stdout:5/443: read d4/d1c/d43/f54 [3364358,35073] 0 2024-09-16T00:26:01.388 INFO:tasks.workunit.client.0.smithi112.stdout:6/486: chown dd/df/d10/l31 240944 1 2024-09-16T00:26:01.391 INFO:tasks.workunit.client.0.smithi112.stdout:3/532: creat d2/d9/d2b/d47/d52/d7a/d80/fb7 x:0 0 0 2024-09-16T00:26:01.391 INFO:tasks.workunit.client.1.smithi169.stdout:1/324: rename dc/d26/f41 to dc/d36/d3a/d45/d29/d2a/f62 0 2024-09-16T00:26:01.391 INFO:tasks.workunit.client.1.smithi169.stdout:9/402: symlink d5/d49/d64/l90 0 2024-09-16T00:26:01.392 INFO:tasks.workunit.client.0.smithi112.stdout:7/544: chown d6/d10/c68 117 1 2024-09-16T00:26:01.393 INFO:tasks.workunit.client.1.smithi169.stdout:1/325: creat dc/d26/f63 x:0 0 0 2024-09-16T00:26:01.393 INFO:tasks.workunit.client.1.smithi169.stdout:1/326: stat dc/d26/f46 0 2024-09-16T00:26:01.396 INFO:tasks.workunit.client.0.smithi112.stdout:5/444: symlink d4/d7/l9c 0 2024-09-16T00:26:01.397 INFO:tasks.workunit.client.0.smithi112.stdout:4/507: symlink d6/d39/d36/lb3 0 2024-09-16T00:26:01.397 INFO:tasks.workunit.client.1.smithi169.stdout:1/327: symlink dc/d36/d3a/d45/d29/d2a/l64 0 2024-09-16T00:26:01.397 INFO:tasks.workunit.client.1.smithi169.stdout:1/328: chown dc/d36/d3a/d45/l4b 201457 1 2024-09-16T00:26:01.398 INFO:tasks.workunit.client.0.smithi112.stdout:8/458: dwrite d10/d12/d1c/f54 [0,4194304] 0 2024-09-16T00:26:01.398 INFO:tasks.workunit.client.0.smithi112.stdout:1/552: dwrite d0/d14/df/d28/d45/d46/d58/d62/f74 [0,4194304] 0 2024-09-16T00:26:01.401 INFO:tasks.workunit.client.1.smithi169.stdout:4/343: dread d2/d12/d13/d1f/f35 [0,4194304] 0 2024-09-16T00:26:01.401 INFO:tasks.workunit.client.1.smithi169.stdout:1/329: write dc/d36/d3a/d45/d29/d2a/f44 [3560318,79102] 0 2024-09-16T00:26:01.404 INFO:tasks.workunit.client.0.smithi112.stdout:5/445: mknod d4/dd/d12/d79/c9d 0 2024-09-16T00:26:01.404 INFO:tasks.workunit.client.0.smithi112.stdout:4/508: truncate d6/d39/f4a 175638 0 2024-09-16T00:26:01.405 INFO:tasks.workunit.client.1.smithi169.stdout:1/330: rename dc/l14 to dc/d26/l65 0 2024-09-16T00:26:01.405 INFO:tasks.workunit.client.0.smithi112.stdout:8/459: dread - d10/d4e/d58/f91 zero size 2024-09-16T00:26:01.406 INFO:tasks.workunit.client.0.smithi112.stdout:2/562: truncate d2/dd/d48/d22/d2b/d54/d79/f9e 444457 0 2024-09-16T00:26:01.407 INFO:tasks.workunit.client.1.smithi169.stdout:4/344: creat d2/d12/d13/f6c x:0 0 0 2024-09-16T00:26:01.408 INFO:tasks.workunit.client.0.smithi112.stdout:7/545: mkdir d6/d10/d1b/dbf 0 2024-09-16T00:26:01.411 INFO:tasks.workunit.client.0.smithi112.stdout:9/705: write d1/d76/f79 [653307,33617] 0 2024-09-16T00:26:01.412 INFO:tasks.workunit.client.0.smithi112.stdout:7/546: chown d6/d10/d16/d96/f9f 457589483 1 2024-09-16T00:26:01.413 INFO:tasks.workunit.client.0.smithi112.stdout:4/509: stat d6/da/c30 0 2024-09-16T00:26:01.413 INFO:tasks.workunit.client.1.smithi169.stdout:1/331: dwrite dc/fd [4194304,4194304] 0 2024-09-16T00:26:01.414 INFO:tasks.workunit.client.0.smithi112.stdout:8/460: mkdir d10/d12/d1c/d96 0 2024-09-16T00:26:01.414 INFO:tasks.workunit.client.0.smithi112.stdout:0/577: dwrite d5/da/d21/da5/faa [0,4194304] 0 2024-09-16T00:26:01.414 INFO:tasks.workunit.client.1.smithi169.stdout:4/345: getdents d2/d54 0 2024-09-16T00:26:01.416 INFO:tasks.workunit.client.0.smithi112.stdout:5/446: creat d4/dd/d12/d52/d64/f9e x:0 0 0 2024-09-16T00:26:01.417 INFO:tasks.workunit.client.0.smithi112.stdout:0/578: chown d5/da/d18/d63/l78 165379801 1 2024-09-16T00:26:01.418 INFO:tasks.workunit.client.0.smithi112.stdout:8/461: dread d10/d12/d24/f2b [0,4194304] 0 2024-09-16T00:26:01.420 INFO:tasks.workunit.client.0.smithi112.stdout:7/547: fsync d6/f9 0 2024-09-16T00:26:01.420 INFO:tasks.workunit.client.1.smithi169.stdout:6/359: write d2/d22/f34 [1104766,123383] 0 2024-09-16T00:26:01.422 INFO:tasks.workunit.client.1.smithi169.stdout:8/330: truncate d7/fd 3950397 0 2024-09-16T00:26:01.423 INFO:tasks.workunit.client.1.smithi169.stdout:9/403: truncate d5/da/d2e/f62 1257560 0 2024-09-16T00:26:01.426 INFO:tasks.workunit.client.0.smithi112.stdout:7/548: truncate d6/d10/d1b/f74 896456 0 2024-09-16T00:26:01.428 INFO:tasks.workunit.client.0.smithi112.stdout:4/510: unlink d6/d39/d1c/d2c/f29 0 2024-09-16T00:26:01.429 INFO:tasks.workunit.client.1.smithi169.stdout:9/404: write d5/da/d21/d23/f41 [3580281,29472] 0 2024-09-16T00:26:01.430 INFO:tasks.workunit.client.0.smithi112.stdout:3/533: dwrite d2/f24 [0,4194304] 0 2024-09-16T00:26:01.430 INFO:tasks.workunit.client.0.smithi112.stdout:8/462: mkdir d10/d4e/d8c/d7c/d97 0 2024-09-16T00:26:01.432 INFO:tasks.workunit.client.0.smithi112.stdout:7/549: fsync d6/d10/d75/fa5 0 2024-09-16T00:26:01.432 INFO:tasks.workunit.client.1.smithi169.stdout:4/346: mkdir d2/d12/d13/d1f/d6d 0 2024-09-16T00:26:01.434 INFO:tasks.workunit.client.0.smithi112.stdout:5/447: getdents d4/d1c/d43 0 2024-09-16T00:26:01.435 INFO:tasks.workunit.client.0.smithi112.stdout:4/511: write d6/d39/d36/d5c/d7e/f86 [953663,78475] 0 2024-09-16T00:26:01.436 INFO:tasks.workunit.client.0.smithi112.stdout:8/463: unlink d10/d12/d1c/d3b/f4d 0 2024-09-16T00:26:01.436 INFO:tasks.workunit.client.0.smithi112.stdout:5/448: write d4/dd/d49/f96 [527332,14018] 0 2024-09-16T00:26:01.436 INFO:tasks.workunit.client.1.smithi169.stdout:8/331: dwrite d7/fa [4194304,4194304] 0 2024-09-16T00:26:01.436 INFO:tasks.workunit.client.1.smithi169.stdout:1/332: link dc/d36/d3a/d45/f1c dc/f66 0 2024-09-16T00:26:01.437 INFO:tasks.workunit.client.1.smithi169.stdout:6/360: link d2/d3f/d41/d52/f1e d2/d22/d51/f77 0 2024-09-16T00:26:01.437 INFO:tasks.workunit.client.0.smithi112.stdout:7/550: unlink d6/d10/d1b/d39/c9c 0 2024-09-16T00:26:01.437 INFO:tasks.workunit.client.1.smithi169.stdout:9/405: mkdir d5/da/d5b/d91 0 2024-09-16T00:26:01.439 INFO:tasks.workunit.client.0.smithi112.stdout:7/551: chown d6/d7f/d85/fb5 1843009623 1 2024-09-16T00:26:01.441 INFO:tasks.workunit.client.0.smithi112.stdout:8/464: mkdir d10/d4e/d8c/d98 0 2024-09-16T00:26:01.442 INFO:tasks.workunit.client.1.smithi169.stdout:8/332: mknod d7/dc/d26/c69 0 2024-09-16T00:26:01.442 INFO:tasks.workunit.client.1.smithi169.stdout:6/361: dread d2/d3f/d41/d52/f18 [0,4194304] 0 2024-09-16T00:26:01.442 INFO:tasks.workunit.client.0.smithi112.stdout:7/552: stat d6/d10/d16/d3b/fa6 0 2024-09-16T00:26:01.442 INFO:tasks.workunit.client.0.smithi112.stdout:5/449: read d4/d1c/d43/f54 [2009084,37367] 0 2024-09-16T00:26:01.444 INFO:tasks.workunit.client.1.smithi169.stdout:9/406: mkdir d5/da/d5b/d92 0 2024-09-16T00:26:01.445 INFO:tasks.workunit.client.0.smithi112.stdout:4/512: link l5 d6/db/df/d5e/d61/lb4 0 2024-09-16T00:26:01.447 INFO:tasks.workunit.client.1.smithi169.stdout:1/333: rename dc/d36/d3a/d45/d29/d42 to dc/d18/d67 0 2024-09-16T00:26:01.448 INFO:tasks.workunit.client.1.smithi169.stdout:4/347: link d2/d12/d13/d1f/f35 d2/d12/d13/d1f/d6d/f6e 0 2024-09-16T00:26:01.449 INFO:tasks.workunit.client.0.smithi112.stdout:5/450: write d4/f19 [3374621,40480] 0 2024-09-16T00:26:01.449 INFO:tasks.workunit.client.0.smithi112.stdout:0/579: dread d5/f2c [0,4194304] 0 2024-09-16T00:26:01.451 INFO:tasks.workunit.client.0.smithi112.stdout:1/553: dread d0/d14/d44/f7e [0,4194304] 0 2024-09-16T00:26:01.452 INFO:tasks.workunit.client.1.smithi169.stdout:9/407: mknod d5/da/d21/d23/c93 0 2024-09-16T00:26:01.452 INFO:tasks.workunit.client.0.smithi112.stdout:9/706: dread d1/f41 [0,4194304] 0 2024-09-16T00:26:01.453 INFO:tasks.workunit.client.1.smithi169.stdout:4/348: symlink d2/d12/d13/d5c/l6f 0 2024-09-16T00:26:01.454 INFO:tasks.workunit.client.0.smithi112.stdout:6/487: dread dd/df/d10/d47/d55/f7a [0,4194304] 0 2024-09-16T00:26:01.456 INFO:tasks.workunit.client.0.smithi112.stdout:0/580: rmdir d5/da/d18/d24 39 2024-09-16T00:26:01.457 INFO:tasks.workunit.client.1.smithi169.stdout:4/349: write d2/d12/f1e [3023724,87446] 0 2024-09-16T00:26:01.457 INFO:tasks.workunit.client.1.smithi169.stdout:9/408: unlink d5/da/d21/d23/c93 0 2024-09-16T00:26:01.460 INFO:tasks.workunit.client.0.smithi112.stdout:1/554: creat d0/d14/df/d28/d45/d46/fb6 x:0 0 0 2024-09-16T00:26:01.460 INFO:tasks.workunit.client.1.smithi169.stdout:1/334: dwrite dc/d36/d3a/d45/d29/f5c [0,4194304] 0 2024-09-16T00:26:01.461 INFO:tasks.workunit.client.0.smithi112.stdout:4/513: mkdir d6/d7c/dad/db5 0 2024-09-16T00:26:01.461 INFO:tasks.workunit.client.0.smithi112.stdout:6/488: chown dd/df/d11/d44/d8a/l9a 145306 1 2024-09-16T00:26:01.462 INFO:tasks.workunit.client.1.smithi169.stdout:1/335: dread - dc/f34 zero size 2024-09-16T00:26:01.462 INFO:tasks.workunit.client.1.smithi169.stdout:9/409: creat d5/da/d21/d23/f94 x:0 0 0 2024-09-16T00:26:01.465 INFO:tasks.workunit.client.0.smithi112.stdout:0/581: write d5/da/d18/d63/fad [4316750,116854] 0 2024-09-16T00:26:01.467 INFO:tasks.workunit.client.1.smithi169.stdout:1/336: unlink dc/f24 0 2024-09-16T00:26:01.467 INFO:tasks.workunit.client.0.smithi112.stdout:5/451: dwrite d4/dd/d6f/f72 [0,4194304] 0 2024-09-16T00:26:01.467 INFO:tasks.workunit.client.0.smithi112.stdout:4/514: dread - d6/d39/d36/d5c/d7e/d8f/f90 zero size 2024-09-16T00:26:01.467 INFO:tasks.workunit.client.0.smithi112.stdout:1/555: readlink d0/d14/d1c/d26/d47/l3f 0 2024-09-16T00:26:01.468 INFO:tasks.workunit.client.1.smithi169.stdout:1/337: readlink dc/d36/d3a/d45/l4b 0 2024-09-16T00:26:01.470 INFO:tasks.workunit.client.0.smithi112.stdout:4/515: chown d6/da/f35 1698 1 2024-09-16T00:26:01.470 INFO:tasks.workunit.client.0.smithi112.stdout:1/556: dread - d0/d14/df/f36 zero size 2024-09-16T00:26:01.471 INFO:tasks.workunit.client.1.smithi169.stdout:9/410: rename d5/da/d5b/d6f/f77 to d5/d12/d61/f95 0 2024-09-16T00:26:01.471 INFO:tasks.workunit.client.1.smithi169.stdout:1/338: chown dc/d36/d3a/d45/l1a 0 1 2024-09-16T00:26:01.472 INFO:tasks.workunit.client.0.smithi112.stdout:6/489: read dd/df/d10/d47/d55/f72 [2317590,24035] 0 2024-09-16T00:26:01.473 INFO:tasks.workunit.client.0.smithi112.stdout:0/582: creat d5/da/d66/fb7 x:0 0 0 2024-09-16T00:26:01.473 INFO:tasks.workunit.client.0.smithi112.stdout:5/452: mknod d4/d7/d26/d4c/c9f 0 2024-09-16T00:26:01.475 INFO:tasks.workunit.client.0.smithi112.stdout:8/465: truncate d10/d12/d24/f29 1093067 0 2024-09-16T00:26:01.476 INFO:tasks.workunit.client.0.smithi112.stdout:5/453: chown d4/d1c 0 1 2024-09-16T00:26:01.477 INFO:tasks.workunit.client.0.smithi112.stdout:4/516: creat d6/d7c/dad/fb6 x:0 0 0 2024-09-16T00:26:01.479 INFO:tasks.workunit.client.0.smithi112.stdout:7/553: write d6/d10/d16/d49/d6c/f36 [4037991,59933] 0 2024-09-16T00:26:01.479 INFO:tasks.workunit.client.1.smithi169.stdout:1/339: rename dc/d36/d39/c5a to dc/d2c/d3d/c68 0 2024-09-16T00:26:01.481 INFO:tasks.workunit.client.1.smithi169.stdout:1/340: chown dc/d2c/d3d/f47 1 1 2024-09-16T00:26:01.481 INFO:tasks.workunit.client.1.smithi169.stdout:9/411: creat d5/d12/f96 x:0 0 0 2024-09-16T00:26:01.482 INFO:tasks.workunit.client.0.smithi112.stdout:8/466: dread - d10/d12/d1c/d61/f84 zero size 2024-09-16T00:26:01.482 INFO:tasks.workunit.client.0.smithi112.stdout:3/534: dwrite d2/d9/d2b/d47/d52/d7a/d80/d83/d49/f4c [4194304,4194304] 0 2024-09-16T00:26:01.484 INFO:tasks.workunit.client.0.smithi112.stdout:2/563: dread d2/f64 [0,4194304] 0 2024-09-16T00:26:01.485 INFO:tasks.workunit.client.1.smithi169.stdout:9/412: mknod d5/da/d5b/d6c/c97 0 2024-09-16T00:26:01.488 INFO:tasks.workunit.client.0.smithi112.stdout:4/517: creat d6/db/d1f/d94/fb7 x:0 0 0 2024-09-16T00:26:01.489 INFO:tasks.workunit.client.1.smithi169.stdout:1/341: dwrite dc/d2c/f32 [0,4194304] 0 2024-09-16T00:26:01.490 INFO:tasks.workunit.client.1.smithi169.stdout:9/413: mknod d5/d12/d61/c98 0 2024-09-16T00:26:01.490 INFO:tasks.workunit.client.0.smithi112.stdout:8/467: chown d10/d12/d1c/d3b/c6b 817 1 2024-09-16T00:26:01.491 INFO:tasks.workunit.client.0.smithi112.stdout:6/490: link dd/df/f6e dd/df/d10/d47/f9c 0 2024-09-16T00:26:01.491 INFO:tasks.workunit.client.0.smithi112.stdout:0/583: creat d5/da/d1d/d71/fb8 x:0 0 0 2024-09-16T00:26:01.492 INFO:tasks.workunit.client.0.smithi112.stdout:2/564: fdatasync d2/dd/d48/d22/f8f 0 2024-09-16T00:26:01.494 INFO:tasks.workunit.client.0.smithi112.stdout:7/554: symlink d6/d10/d75/d8f/d90/dbc/lc0 0 2024-09-16T00:26:01.494 INFO:tasks.workunit.client.1.smithi169.stdout:6/362: dread d2/d4d/d59/d38/f62 [0,4194304] 0 2024-09-16T00:26:01.495 INFO:tasks.workunit.client.0.smithi112.stdout:1/557: dwrite d0/d14/df/d28/d45/d46/f90 [0,4194304] 0 2024-09-16T00:26:01.497 INFO:tasks.workunit.client.1.smithi169.stdout:4/350: write d2/d12/f57 [4467387,47188] 0 2024-09-16T00:26:01.498 INFO:tasks.workunit.client.0.smithi112.stdout:4/518: write d6/db/f88 [5156259,14707] 0 2024-09-16T00:26:01.499 INFO:tasks.workunit.client.1.smithi169.stdout:6/363: rename d2/d4d/f6f to d2/d22/f78 0 2024-09-16T00:26:01.503 INFO:tasks.workunit.client.0.smithi112.stdout:0/584: write d5/da/f27 [3368242,85221] 0 2024-09-16T00:26:01.503 INFO:tasks.workunit.client.0.smithi112.stdout:3/535: symlink d2/d9/d2b/d47/d52/d7a/d80/d83/d7b/lb8 0 2024-09-16T00:26:01.503 INFO:tasks.workunit.client.0.smithi112.stdout:2/565: unlink d2/dd/d48/d22/d2b/c51 0 2024-09-16T00:26:01.504 INFO:tasks.workunit.client.1.smithi169.stdout:9/414: mkdir d5/da/d2e/d99 0 2024-09-16T00:26:01.504 INFO:tasks.workunit.client.0.smithi112.stdout:9/707: dread d1/d5/d6/d2d/f92 [4194304,4194304] 0 2024-09-16T00:26:01.504 INFO:tasks.workunit.client.1.smithi169.stdout:1/342: truncate dc/d36/d3a/d45/f17 840325 0 2024-09-16T00:26:01.507 INFO:tasks.workunit.client.0.smithi112.stdout:5/454: dwrite d4/d7/d14/f33 [0,4194304] 0 2024-09-16T00:26:01.508 INFO:tasks.workunit.client.1.smithi169.stdout:4/351: truncate d2/d12/d13/d1f/f52 5006793 0 2024-09-16T00:26:01.511 INFO:tasks.workunit.client.1.smithi169.stdout:1/343: mkdir dc/d18/d67/d69 0 2024-09-16T00:26:01.511 INFO:tasks.workunit.client.0.smithi112.stdout:6/491: mknod dd/df/d91/c9d 0 2024-09-16T00:26:01.514 INFO:tasks.workunit.client.0.smithi112.stdout:2/566: fdatasync d2/dd/d48/d22/d67/fa7 0 2024-09-16T00:26:01.514 INFO:tasks.workunit.client.1.smithi169.stdout:4/352: mkdir d2/d12/d13/d1f/d3e/d5b/d70 0 2024-09-16T00:26:01.516 INFO:tasks.workunit.client.0.smithi112.stdout:5/455: chown d4/dd/d12/d52/c8f 5850 1 2024-09-16T00:26:01.518 INFO:tasks.workunit.client.1.smithi169.stdout:4/353: getdents d2/d12/d13/d4d/d59 0 2024-09-16T00:26:01.521 INFO:tasks.workunit.client.0.smithi112.stdout:8/468: dwrite d10/d12/d1c/d3b/d74/f90 [0,4194304] 0 2024-09-16T00:26:01.521 INFO:tasks.workunit.client.0.smithi112.stdout:3/536: chown d2/d9/d2b/d47/d52/d7a/d80/d83/d68/l89 3368 1 2024-09-16T00:26:01.522 INFO:tasks.workunit.client.0.smithi112.stdout:0/585: creat d5/da/d21/fb9 x:0 0 0 2024-09-16T00:26:01.522 INFO:tasks.workunit.client.1.smithi169.stdout:6/364: write d2/d36/f31 [8188000,101345] 0 2024-09-16T00:26:01.524 INFO:tasks.workunit.client.1.smithi169.stdout:6/365: truncate d2/d4d/d59/f35 1571854 0 2024-09-16T00:26:01.525 INFO:tasks.workunit.client.0.smithi112.stdout:6/492: symlink dd/df/d10/d47/d89/l9e 0 2024-09-16T00:26:01.526 INFO:tasks.workunit.client.0.smithi112.stdout:5/456: write d4/dd/d49/d70/f7e [241706,97515] 0 2024-09-16T00:26:01.526 INFO:tasks.workunit.client.0.smithi112.stdout:9/708: creat d1/d5/d6/d2d/d6a/d89/ff6 x:0 0 0 2024-09-16T00:26:01.527 INFO:tasks.workunit.client.0.smithi112.stdout:3/537: unlink d2/d9/d2b/c5d 0 2024-09-16T00:26:01.529 INFO:tasks.workunit.client.0.smithi112.stdout:1/558: dwrite d0/d14/d1c/d26/f70 [0,4194304] 0 2024-09-16T00:26:01.530 INFO:tasks.workunit.client.0.smithi112.stdout:0/586: dread - d5/da/d18/d89/fab zero size 2024-09-16T00:26:01.533 INFO:tasks.workunit.client.1.smithi169.stdout:6/366: dwrite d2/d3f/d41/d52/f1e [0,4194304] 0 2024-09-16T00:26:01.533 INFO:tasks.workunit.client.0.smithi112.stdout:2/567: creat d2/dd/d48/d22/d2b/d54/d60/d6c/d88/fae x:0 0 0 2024-09-16T00:26:01.538 INFO:tasks.workunit.client.0.smithi112.stdout:7/555: dread d6/d10/d75/fb9 [0,4194304] 0 2024-09-16T00:26:01.539 INFO:tasks.workunit.client.1.smithi169.stdout:4/354: dwrite d2/d12/d13/d1f/f35 [0,4194304] 0 2024-09-16T00:26:01.541 INFO:tasks.workunit.client.0.smithi112.stdout:2/568: write d2/dd/d48/d22/d2b/f31 [1276906,9589] 0 2024-09-16T00:26:01.545 INFO:tasks.workunit.client.0.smithi112.stdout:8/469: dwrite d10/d12/d1c/d75/f6a [0,4194304] 0 2024-09-16T00:26:01.545 INFO:tasks.workunit.client.0.smithi112.stdout:9/709: creat d1/d5/d20/d81/d1a/d88/def/da9/dba/ff7 x:0 0 0 2024-09-16T00:26:01.546 INFO:tasks.workunit.client.0.smithi112.stdout:3/538: symlink d2/d9/d2b/d47/d52/d7a/d80/d83/d68/lb9 0 2024-09-16T00:26:01.547 INFO:tasks.workunit.client.0.smithi112.stdout:7/556: fdatasync d6/d7f/f89 0 2024-09-16T00:26:01.552 INFO:tasks.workunit.client.0.smithi112.stdout:4/519: dread d6/db/df/f5f [0,4194304] 0 2024-09-16T00:26:01.554 INFO:tasks.workunit.client.0.smithi112.stdout:1/559: link d0/d14/d1c/d26/d47/l79 d0/lb7 0 2024-09-16T00:26:01.554 INFO:tasks.workunit.client.0.smithi112.stdout:0/587: truncate d5/da/d57/db0/f79 1457233 0 2024-09-16T00:26:01.556 INFO:tasks.workunit.client.0.smithi112.stdout:9/710: chown d1/d5/d6/d59/d64/cd8 30051633 1 2024-09-16T00:26:01.556 INFO:tasks.workunit.client.0.smithi112.stdout:7/557: creat d6/d10/d75/d8f/d90/dbc/fc1 x:0 0 0 2024-09-16T00:26:01.556 INFO:tasks.workunit.client.0.smithi112.stdout:6/493: link dd/df/d10/l31 dd/df/d17/d27/d84/d37/d79/l9f 0 2024-09-16T00:26:01.559 INFO:tasks.workunit.client.0.smithi112.stdout:4/520: mkdir d6/db/df/d5e/d61/d64/d6a/db8 0 2024-09-16T00:26:01.559 INFO:tasks.workunit.client.1.smithi169.stdout:1/344: dread dc/f1d [0,4194304] 0 2024-09-16T00:26:01.560 INFO:tasks.workunit.client.0.smithi112.stdout:0/588: symlink d5/da/d1d/d71/d64/lba 0 2024-09-16T00:26:01.561 INFO:tasks.workunit.client.0.smithi112.stdout:4/521: chown d6/d39/d1c/d2c/d69/d7b 1797878 1 2024-09-16T00:26:01.561 INFO:tasks.workunit.client.0.smithi112.stdout:3/539: creat d2/d9/d2b/d47/d78/da9/fba x:0 0 0 2024-09-16T00:26:01.562 INFO:tasks.workunit.client.1.smithi169.stdout:4/355: truncate d2/d12/d13/d1f/f52 230380 0 2024-09-16T00:26:01.563 INFO:tasks.workunit.client.1.smithi169.stdout:1/345: getdents dc/d18/d67/d69 0 2024-09-16T00:26:01.563 INFO:tasks.workunit.client.0.smithi112.stdout:6/494: truncate dd/df/d17/d27/d84/f36 687143 0 2024-09-16T00:26:01.564 INFO:tasks.workunit.client.0.smithi112.stdout:0/589: stat d5/da/c11 0 2024-09-16T00:26:01.565 INFO:tasks.workunit.client.1.smithi169.stdout:4/356: creat d2/d12/d13/d27/d33/f71 x:0 0 0 2024-09-16T00:26:01.565 INFO:tasks.workunit.client.1.smithi169.stdout:6/367: dread d2/d4d/f49 [0,4194304] 0 2024-09-16T00:26:01.566 INFO:tasks.workunit.client.0.smithi112.stdout:7/558: mkdir d6/d10/d75/d88/dc2 0 2024-09-16T00:26:01.567 INFO:tasks.workunit.client.0.smithi112.stdout:4/522: fdatasync d6/db/df/f5f 0 2024-09-16T00:26:01.568 INFO:tasks.workunit.client.1.smithi169.stdout:4/357: creat d2/d12/d13/d27/d33/f72 x:0 0 0 2024-09-16T00:26:01.569 INFO:tasks.workunit.client.0.smithi112.stdout:3/540: dread d2/f97 [0,4194304] 0 2024-09-16T00:26:01.573 INFO:tasks.workunit.client.1.smithi169.stdout:1/346: dwrite dc/f15 [0,4194304] 0 2024-09-16T00:26:01.573 INFO:tasks.workunit.client.0.smithi112.stdout:0/590: link d5/da/d57/d76/c37 d5/da/d1d/d71/d64/d8f/cbb 0 2024-09-16T00:26:01.576 INFO:tasks.workunit.client.0.smithi112.stdout:3/541: creat d2/d9/d2b/d39/fbb x:0 0 0 2024-09-16T00:26:01.576 INFO:tasks.workunit.client.0.smithi112.stdout:5/457: dread d4/d7/d26/d36/f3b [0,4194304] 0 2024-09-16T00:26:01.577 INFO:tasks.workunit.client.1.smithi169.stdout:6/368: dwrite d2/d3f/d41/d52/f1b [0,4194304] 0 2024-09-16T00:26:01.578 INFO:tasks.workunit.client.0.smithi112.stdout:5/458: read - d4/d7/d26/d4c/f97 zero size 2024-09-16T00:26:01.579 INFO:tasks.workunit.client.1.smithi169.stdout:1/347: write dc/f66 [2005611,92396] 0 2024-09-16T00:26:01.581 INFO:tasks.workunit.client.1.smithi169.stdout:6/369: symlink d2/d4d/d59/d74/l79 0 2024-09-16T00:26:01.581 INFO:tasks.workunit.client.0.smithi112.stdout:3/542: fsync d2/f2d 0 2024-09-16T00:26:01.584 INFO:tasks.workunit.client.1.smithi169.stdout:1/348: truncate dc/d18/f2b 800033 0 2024-09-16T00:26:01.584 INFO:tasks.workunit.client.1.smithi169.stdout:6/370: symlink d2/d36/l7a 0 2024-09-16T00:26:01.585 INFO:tasks.workunit.client.0.smithi112.stdout:4/523: dwrite d6/d39/d1c/d6e/fb0 [0,4194304] 0 2024-09-16T00:26:01.585 INFO:tasks.workunit.client.0.smithi112.stdout:7/559: dread d6/d10/d75/d8f/f99 [0,4194304] 0 2024-09-16T00:26:01.585 INFO:tasks.workunit.client.0.smithi112.stdout:8/470: truncate d10/f55 688150 0 2024-09-16T00:26:01.585 INFO:tasks.workunit.client.0.smithi112.stdout:2/569: dread f1 [0,4194304] 0 2024-09-16T00:26:01.586 INFO:tasks.workunit.client.1.smithi169.stdout:1/349: dread - dc/d36/d3a/d45/f61 zero size 2024-09-16T00:26:01.586 INFO:tasks.workunit.client.1.smithi169.stdout:6/371: readlink d2/d22/l5e 0 2024-09-16T00:26:01.586 INFO:tasks.workunit.client.1.smithi169.stdout:1/350: stat la 0 2024-09-16T00:26:01.589 INFO:tasks.workunit.client.1.smithi169.stdout:6/372: write d2/f1f [3466386,4618] 0 2024-09-16T00:26:01.591 INFO:tasks.workunit.client.0.smithi112.stdout:6/495: write dd/df/d10/f1a [1199781,116828] 0 2024-09-16T00:26:01.592 INFO:tasks.workunit.client.0.smithi112.stdout:6/496: chown dd/df/d91 241 1 2024-09-16T00:26:01.593 INFO:tasks.workunit.client.1.smithi169.stdout:4/358: dread d2/d12/f2c [0,4194304] 0 2024-09-16T00:26:01.593 INFO:tasks.workunit.client.0.smithi112.stdout:4/524: mknod d6/d39/cb9 0 2024-09-16T00:26:01.594 INFO:tasks.workunit.client.1.smithi169.stdout:6/373: symlink d2/d3f/d41/d5c/d73/l7b 0 2024-09-16T00:26:01.594 INFO:tasks.workunit.client.0.smithi112.stdout:7/560: fdatasync d6/d10/d75/d88/f27 0 2024-09-16T00:26:01.595 INFO:tasks.workunit.client.1.smithi169.stdout:1/351: link dc/d26/c53 dc/d18/d67/c6a 0 2024-09-16T00:26:01.595 INFO:tasks.workunit.client.1.smithi169.stdout:4/359: chown d2/d12/d13/d1f/f35 18 1 2024-09-16T00:26:01.595 INFO:tasks.workunit.client.1.smithi169.stdout:1/352: write dc/d26/f46 [737588,92028] 0 2024-09-16T00:26:01.596 INFO:tasks.workunit.client.0.smithi112.stdout:6/497: symlink dd/d23/d8f/la0 0 2024-09-16T00:26:01.596 INFO:tasks.workunit.client.0.smithi112.stdout:2/570: read d2/dd/d48/d22/d2b/d54/d79/f9e [239082,28321] 0 2024-09-16T00:26:01.597 INFO:tasks.workunit.client.1.smithi169.stdout:6/374: chown d2/d22/d51/f77 108085980 1 2024-09-16T00:26:01.597 INFO:tasks.workunit.client.0.smithi112.stdout:8/471: chown d10/f55 40 1 2024-09-16T00:26:01.598 INFO:tasks.workunit.client.1.smithi169.stdout:1/353: write dc/d36/d4e/f4f [825135,129514] 0 2024-09-16T00:26:01.599 INFO:tasks.workunit.client.0.smithi112.stdout:1/560: write d0/d14/df/d28/d45/d46/d58/f99 [582627,104075] 0 2024-09-16T00:26:01.600 INFO:tasks.workunit.client.0.smithi112.stdout:7/561: readlink d6/d10/d16/d49/d6c/l8c 0 2024-09-16T00:26:01.601 INFO:tasks.workunit.client.0.smithi112.stdout:3/543: rename d2/d9/d2b/d47/d52/c59 to d2/d9/d2b/d47/d52/d7a/d80/d83/d7b/dad/cbc 0 2024-09-16T00:26:01.601 INFO:tasks.workunit.client.0.smithi112.stdout:9/711: dread d1/d5/d6/d2d/d6a/fad [0,4194304] 0 2024-09-16T00:26:01.602 INFO:tasks.workunit.client.1.smithi169.stdout:6/375: mknod d2/d3f/d41/d72/c7c 0 2024-09-16T00:26:01.602 INFO:tasks.workunit.client.1.smithi169.stdout:4/360: fsync d2/d12/d13/d1f/d3e/d5b/f18 0 2024-09-16T00:26:01.603 INFO:tasks.workunit.client.1.smithi169.stdout:1/354: write dc/d2c/d3d/f47 [955138,30235] 0 2024-09-16T00:26:01.603 INFO:tasks.workunit.client.0.smithi112.stdout:9/712: dread - d1/ff4 zero size 2024-09-16T00:26:01.603 INFO:tasks.workunit.client.0.smithi112.stdout:7/562: mkdir d6/d7f/dc3 0 2024-09-16T00:26:01.605 INFO:tasks.workunit.client.0.smithi112.stdout:6/498: symlink dd/df/d10/la1 0 2024-09-16T00:26:01.605 INFO:tasks.workunit.client.1.smithi169.stdout:4/361: dread d2/d12/d13/d1f/d3e/d5b/d1a/f55 [0,4194304] 0 2024-09-16T00:26:01.607 INFO:tasks.workunit.client.0.smithi112.stdout:1/561: creat d0/d14/df/d28/d45/d46/d58/d62/db0/fb8 x:0 0 0 2024-09-16T00:26:01.607 INFO:tasks.workunit.client.1.smithi169.stdout:1/355: creat dc/d26/d5d/f6b x:0 0 0 2024-09-16T00:26:01.607 INFO:tasks.workunit.client.0.smithi112.stdout:3/544: write d2/d9/d2b/d39/fb1 [163451,41940] 0 2024-09-16T00:26:01.611 INFO:tasks.workunit.client.0.smithi112.stdout:2/571: dwrite d2/f8d [4194304,4194304] 0 2024-09-16T00:26:01.611 INFO:tasks.workunit.client.0.smithi112.stdout:9/713: truncate d1/d5/d6/d2d/fbc 1315489 0 2024-09-16T00:26:01.613 INFO:tasks.workunit.client.0.smithi112.stdout:1/562: fsync d0/d14/df/d28/d45/d30/f3a 0 2024-09-16T00:26:01.613 INFO:tasks.workunit.client.0.smithi112.stdout:8/472: dread d10/d4e/f64 [0,4194304] 0 2024-09-16T00:26:01.614 INFO:tasks.workunit.client.1.smithi169.stdout:4/362: creat d2/d12/d13/d1f/d3e/d5b/f73 x:0 0 0 2024-09-16T00:26:01.614 INFO:tasks.workunit.client.0.smithi112.stdout:7/563: rename d6/d10/d16/d49/d6c/f4a to d6/d10/d16/db4/fc4 0 2024-09-16T00:26:01.615 INFO:tasks.workunit.client.0.smithi112.stdout:3/545: readlink d2/d9/d2b/d2c/d69/l81 0 2024-09-16T00:26:01.615 INFO:tasks.workunit.client.1.smithi169.stdout:1/356: dread dc/d36/d3a/d45/d29/f5c [0,4194304] 0 2024-09-16T00:26:01.618 INFO:tasks.workunit.client.1.smithi169.stdout:4/363: read d2/fb [2708808,129598] 0 2024-09-16T00:26:01.619 INFO:tasks.workunit.client.0.smithi112.stdout:0/591: dread d5/da/f53 [0,4194304] 0 2024-09-16T00:26:01.620 INFO:tasks.workunit.client.0.smithi112.stdout:2/572: mkdir d2/dd/d48/d22/d2b/d54/d60/daf 0 2024-09-16T00:26:01.621 INFO:tasks.workunit.client.0.smithi112.stdout:1/563: creat d0/d14/d1c/d26/fb9 x:0 0 0 2024-09-16T00:26:01.622 INFO:tasks.workunit.client.0.smithi112.stdout:7/564: fdatasync d6/d10/d16/f4e 0 2024-09-16T00:26:01.624 INFO:tasks.workunit.client.1.smithi169.stdout:7/449: sync 2024-09-16T00:26:01.624 INFO:tasks.workunit.client.1.smithi169.stdout:0/377: sync 2024-09-16T00:26:01.626 INFO:tasks.workunit.client.0.smithi112.stdout:3/546: mknod d2/d9/d2b/d47/d52/d7a/d80/d83/d68/cbd 0 2024-09-16T00:26:01.628 INFO:tasks.workunit.client.1.smithi169.stdout:6/376: dread d2/f45 [0,4194304] 0 2024-09-16T00:26:01.630 INFO:tasks.workunit.client.0.smithi112.stdout:7/565: dwrite d6/d10/d16/d49/d6c/f5d [0,4194304] 0 2024-09-16T00:26:01.630 INFO:tasks.workunit.client.1.smithi169.stdout:4/364: creat d2/d12/d13/d1f/d6d/f74 x:0 0 0 2024-09-16T00:26:01.631 INFO:tasks.workunit.client.0.smithi112.stdout:8/473: rename d10/c17 to d10/d4e/c99 0 2024-09-16T00:26:01.633 INFO:tasks.workunit.client.0.smithi112.stdout:8/474: chown d10/d12/d1c/d3b/f89 3340 1 2024-09-16T00:26:01.634 INFO:tasks.workunit.client.1.smithi169.stdout:0/378: read d0/d2/d8/fe [796854,122040] 0 2024-09-16T00:26:01.635 INFO:tasks.workunit.client.0.smithi112.stdout:7/566: creat d6/d10/d75/d8f/fc5 x:0 0 0 2024-09-16T00:26:01.635 INFO:tasks.workunit.client.0.smithi112.stdout:0/592: creat d5/da/d1d/d71/fbc x:0 0 0 2024-09-16T00:26:01.637 INFO:tasks.workunit.client.0.smithi112.stdout:7/567: chown d6/d10/d16/d49/d6c/f33 44008804 1 2024-09-16T00:26:01.637 INFO:tasks.workunit.client.0.smithi112.stdout:0/593: fdatasync d5/da/d18/f62 0 2024-09-16T00:26:01.638 INFO:tasks.workunit.client.1.smithi169.stdout:6/377: rename d2/d3f/d41/d52/f6e to d2/d4d/f7d 0 2024-09-16T00:26:01.639 INFO:tasks.workunit.client.0.smithi112.stdout:7/568: write d6/d10/d16/d96/f9f [68257,97929] 0 2024-09-16T00:26:01.639 INFO:tasks.workunit.client.1.smithi169.stdout:6/378: dread - d2/d4d/d59/d43/f2f zero size 2024-09-16T00:26:01.639 INFO:tasks.workunit.client.0.smithi112.stdout:8/475: getdents d10/d12/d1c/d3b/d7d 0 2024-09-16T00:26:01.641 INFO:tasks.workunit.client.1.smithi169.stdout:4/365: creat d2/d12/d13/d4d/d59/f75 x:0 0 0 2024-09-16T00:26:01.642 INFO:tasks.workunit.client.0.smithi112.stdout:9/714: rename d1/d5/d20/d81/d1a/d88/def/da9 to d1/d5/d6/df8 0 2024-09-16T00:26:01.643 INFO:tasks.workunit.client.1.smithi169.stdout:6/379: write d2/d4d/f53 [3624030,74411] 0 2024-09-16T00:26:01.643 INFO:tasks.workunit.client.0.smithi112.stdout:7/569: fdatasync d6/d10/f1d 0 2024-09-16T00:26:01.644 INFO:tasks.workunit.client.1.smithi169.stdout:0/379: rename d0/d2/d8/d5/l18 to d0/d2/d4b/l8f 0 2024-09-16T00:26:01.644 INFO:tasks.workunit.client.0.smithi112.stdout:0/594: creat d5/da/d1d/d71/d64/db6/fbd x:0 0 0 2024-09-16T00:26:01.645 INFO:tasks.workunit.client.1.smithi169.stdout:0/380: dread - d0/d2c/f4a zero size 2024-09-16T00:26:01.646 INFO:tasks.workunit.client.1.smithi169.stdout:1/357: dread dc/d26/f37 [0,4194304] 0 2024-09-16T00:26:01.646 INFO:tasks.workunit.client.0.smithi112.stdout:1/564: rename d0/d14/d1c/d26/f70 to d0/d14/df/d28/d42/fba 0 2024-09-16T00:26:01.647 INFO:tasks.workunit.client.0.smithi112.stdout:9/715: unlink d1/d5/d6/l51 0 2024-09-16T00:26:01.649 INFO:tasks.workunit.client.1.smithi169.stdout:6/380: creat d2/d36/d48/f7e x:0 0 0 2024-09-16T00:26:01.649 INFO:tasks.workunit.client.0.smithi112.stdout:9/716: write d1/d5/d6/d18/fb0 [1615740,83899] 0 2024-09-16T00:26:01.649 INFO:tasks.workunit.client.1.smithi169.stdout:1/358: write dc/d36/d3a/d45/d29/d4d/f54 [795231,115350] 0 2024-09-16T00:26:01.650 INFO:tasks.workunit.client.1.smithi169.stdout:1/359: read f7 [4686684,34313] 0 2024-09-16T00:26:01.650 INFO:tasks.workunit.client.0.smithi112.stdout:2/573: dread d2/dd/d48/d22/d2b/f43 [0,4194304] 0 2024-09-16T00:26:01.651 INFO:tasks.workunit.client.1.smithi169.stdout:0/381: mknod d0/d2/d4b/c90 0 2024-09-16T00:26:01.651 INFO:tasks.workunit.client.0.smithi112.stdout:3/547: write d2/d9/d2b/d47/d52/f38 [4003317,76472] 0 2024-09-16T00:26:01.651 INFO:tasks.workunit.client.1.smithi169.stdout:1/360: chown dc/d2c/d3d 202 1 2024-09-16T00:26:01.653 INFO:tasks.workunit.client.1.smithi169.stdout:4/366: link d2/d12/d13/d27/f32 d2/d12/d13/d4e/f76 0 2024-09-16T00:26:01.654 INFO:tasks.workunit.client.0.smithi112.stdout:7/570: creat d6/d10/d16/d49/d6c/fc6 x:0 0 0 2024-09-16T00:26:01.654 INFO:tasks.workunit.client.0.smithi112.stdout:1/565: dread - d0/d14/df/f4b zero size 2024-09-16T00:26:01.655 INFO:tasks.workunit.client.1.smithi169.stdout:0/382: creat d0/d4c/d6f/f91 x:0 0 0 2024-09-16T00:26:01.655 INFO:tasks.workunit.client.1.smithi169.stdout:1/361: fdatasync dc/d26/f37 0 2024-09-16T00:26:01.655 INFO:tasks.workunit.client.0.smithi112.stdout:1/566: chown d0/d14/d1c/d26/fa2 0 1 2024-09-16T00:26:01.656 INFO:tasks.workunit.client.1.smithi169.stdout:1/362: fdatasync dc/d2c/f32 0 2024-09-16T00:26:01.656 INFO:tasks.workunit.client.0.smithi112.stdout:1/567: read - d0/d14/df/f5b zero size 2024-09-16T00:26:01.657 INFO:tasks.workunit.client.0.smithi112.stdout:9/717: mknod d1/d5/d6/df8/ddd/cf9 0 2024-09-16T00:26:01.658 INFO:tasks.workunit.client.1.smithi169.stdout:6/381: creat d2/d4d/d59/f7f x:0 0 0 2024-09-16T00:26:01.659 INFO:tasks.workunit.client.0.smithi112.stdout:0/595: rmdir d5/da/d18/d24/d46/d88 39 2024-09-16T00:26:01.660 INFO:tasks.workunit.client.0.smithi112.stdout:8/476: link d10/d12/d24/f29 d10/d4e/d8c/f9a 0 2024-09-16T00:26:01.660 INFO:tasks.workunit.client.1.smithi169.stdout:1/363: rmdir dc/d36/d3a/d45/d29 39 2024-09-16T00:26:01.661 INFO:tasks.workunit.client.1.smithi169.stdout:7/450: dread d7/d16/d49/f4d [0,4194304] 0 2024-09-16T00:26:01.661 INFO:tasks.workunit.client.1.smithi169.stdout:4/367: fdatasync d2/d12/d13/d1f/d2f/f45 0 2024-09-16T00:26:01.662 INFO:tasks.workunit.client.0.smithi112.stdout:3/548: symlink d2/lbe 0 2024-09-16T00:26:01.662 INFO:tasks.workunit.client.0.smithi112.stdout:2/574: symlink d2/dd/d48/d22/d67/d9a/lb0 0 2024-09-16T00:26:01.663 INFO:tasks.workunit.client.0.smithi112.stdout:9/718: rmdir d1/d5/d6/df8 39 2024-09-16T00:26:01.663 INFO:tasks.workunit.client.0.smithi112.stdout:8/477: dread d10/d4e/d8c/f85 [0,4194304] 0 2024-09-16T00:26:01.664 INFO:tasks.workunit.client.0.smithi112.stdout:3/549: write d2/d9/d2b/d2c/d69/f72 [428546,123775] 0 2024-09-16T00:26:01.665 INFO:tasks.workunit.client.0.smithi112.stdout:8/478: write d10/d12/d1c/d75/f6a [656811,54221] 0 2024-09-16T00:26:01.666 INFO:tasks.workunit.client.0.smithi112.stdout:1/568: creat d0/d14/df/d28/d45/d46/da7/fbb x:0 0 0 2024-09-16T00:26:01.667 INFO:tasks.workunit.client.1.smithi169.stdout:4/368: fsync d2/d12/f31 0 2024-09-16T00:26:01.668 INFO:tasks.workunit.client.0.smithi112.stdout:2/575: creat d2/dd/d48/d22/fb1 x:0 0 0 2024-09-16T00:26:01.668 INFO:tasks.workunit.client.0.smithi112.stdout:8/479: write d10/d12/d24/f82 [729184,128529] 0 2024-09-16T00:26:01.668 INFO:tasks.workunit.client.1.smithi169.stdout:6/382: dwrite d2/d4d/d59/f7f [0,4194304] 0 2024-09-16T00:26:01.671 INFO:tasks.workunit.client.1.smithi169.stdout:7/451: rename d7/d71/d76/d3a to d7/df/d95 0 2024-09-16T00:26:01.671 INFO:tasks.workunit.client.1.smithi169.stdout:6/383: fdatasync d2/d3f/d41/d5c/d73/f5b 0 2024-09-16T00:26:01.671 INFO:tasks.workunit.client.1.smithi169.stdout:7/452: chown d7/d16/d49/d84 5275 1 2024-09-16T00:26:01.673 INFO:tasks.workunit.client.0.smithi112.stdout:1/569: chown d0/d14/df/d28/d45/d46/d58/d62/db0/fb8 1 1 2024-09-16T00:26:01.674 INFO:tasks.workunit.client.0.smithi112.stdout:7/571: getdents d6/d10/d1b/d39 0 2024-09-16T00:26:01.674 INFO:tasks.workunit.client.1.smithi169.stdout:4/369: rmdir d2/d12/d13/d1f 39 2024-09-16T00:26:01.674 INFO:tasks.workunit.client.1.smithi169.stdout:6/384: write d2/d4d/f53 [4917982,122570] 0 2024-09-16T00:26:01.675 INFO:tasks.workunit.client.0.smithi112.stdout:0/596: link d5/da/d57/d76/c1a d5/da/d18/d24/d5a/cbe 0 2024-09-16T00:26:01.676 INFO:tasks.workunit.client.1.smithi169.stdout:6/385: dread - d2/d4d/d59/d43/f56 zero size 2024-09-16T00:26:01.676 INFO:tasks.workunit.client.0.smithi112.stdout:3/550: dwrite d2/f1a [4194304,4194304] 0 2024-09-16T00:26:01.677 INFO:tasks.workunit.client.0.smithi112.stdout:2/576: creat d2/dd/d48/d22/d67/fb2 x:0 0 0 2024-09-16T00:26:01.678 INFO:tasks.workunit.client.1.smithi169.stdout:1/364: truncate dc/f1d 55970 0 2024-09-16T00:26:01.679 INFO:tasks.workunit.client.0.smithi112.stdout:0/597: dread d5/da/d18/f54 [0,4194304] 0 2024-09-16T00:26:01.681 INFO:tasks.workunit.client.0.smithi112.stdout:1/570: creat d0/d14/df/d28/d45/d46/da7/fbc x:0 0 0 2024-09-16T00:26:01.681 INFO:tasks.workunit.client.0.smithi112.stdout:2/577: dread - d2/dd/d48/d22/d2b/d30/f6f zero size 2024-09-16T00:26:01.681 INFO:tasks.workunit.client.1.smithi169.stdout:7/453: dwrite f1 [0,4194304] 0 2024-09-16T00:26:01.682 INFO:tasks.workunit.client.1.smithi169.stdout:6/386: unlink d2/d3f/d41/d72/f6c 0 2024-09-16T00:26:01.682 INFO:tasks.workunit.client.0.smithi112.stdout:9/719: rename d1/d5/d6/d59/c95 to d1/d5/d6/cfa 0 2024-09-16T00:26:01.686 INFO:tasks.workunit.client.0.smithi112.stdout:2/578: truncate d2/dd/d48/f85 227838 0 2024-09-16T00:26:01.686 INFO:tasks.workunit.client.0.smithi112.stdout:8/480: rename d10/d12/d1c/d3b/d74/f79 to d10/d4e/d58/f9b 0 2024-09-16T00:26:01.690 INFO:tasks.workunit.client.1.smithi169.stdout:4/370: link d2/d12/d13/d1f/f35 d2/d12/d13/d4e/f77 0 2024-09-16T00:26:01.690 INFO:tasks.workunit.client.1.smithi169.stdout:6/387: creat d2/d3f/d41/f80 x:0 0 0 2024-09-16T00:26:01.690 INFO:tasks.workunit.client.1.smithi169.stdout:7/454: mkdir d7/df/d73/d96 0 2024-09-16T00:26:01.691 INFO:tasks.workunit.client.0.smithi112.stdout:0/598: truncate d5/da/d18/d89/f9b 14766 0 2024-09-16T00:26:01.691 INFO:tasks.workunit.client.0.smithi112.stdout:1/571: dread d0/d14/df/d28/d45/d46/d58/f99 [0,4194304] 0 2024-09-16T00:26:01.691 INFO:tasks.workunit.client.0.smithi112.stdout:7/572: dwrite d6/d10/f87 [0,4194304] 0 2024-09-16T00:26:01.692 INFO:tasks.workunit.client.1.smithi169.stdout:4/371: chown d2/d12/d13/f6c 262661 1 2024-09-16T00:26:01.692 INFO:tasks.workunit.client.1.smithi169.stdout:6/388: read d2/d36/f2a [1882431,119615] 0 2024-09-16T00:26:01.692 INFO:tasks.workunit.client.1.smithi169.stdout:4/372: chown d2/d12/d13/d4d 63541 1 2024-09-16T00:26:01.693 INFO:tasks.workunit.client.1.smithi169.stdout:6/389: readlink d2/d22/l5e 0 2024-09-16T00:26:01.694 INFO:tasks.workunit.client.0.smithi112.stdout:8/481: dread d10/d12/d1c/d61/d6c/f7a [0,4194304] 0 2024-09-16T00:26:01.694 INFO:tasks.workunit.client.1.smithi169.stdout:4/373: fsync d2/d12/d13/d1f/d3e/d5b/d1a/f41 0 2024-09-16T00:26:01.694 INFO:tasks.workunit.client.0.smithi112.stdout:2/579: mknod d2/dd/d48/d94/d49/cb3 0 2024-09-16T00:26:01.695 INFO:tasks.workunit.client.1.smithi169.stdout:6/390: write d2/d36/f31 [5267642,105580] 0 2024-09-16T00:26:01.696 INFO:tasks.workunit.client.0.smithi112.stdout:1/572: unlink d0/d14/d1c/d20/d93/la5 0 2024-09-16T00:26:01.699 INFO:tasks.workunit.client.0.smithi112.stdout:0/599: unlink d5/da/d18/d24/d46/d88/f7e 0 2024-09-16T00:26:01.700 INFO:tasks.workunit.client.0.smithi112.stdout:8/482: mknod d10/d4e/d8c/c9c 0 2024-09-16T00:26:01.700 INFO:tasks.workunit.client.1.smithi169.stdout:6/391: mkdir d2/d3f/d41/d72/d81 0 2024-09-16T00:26:01.701 INFO:tasks.workunit.client.0.smithi112.stdout:8/483: readlink d10/l3c 0 2024-09-16T00:26:01.701 INFO:tasks.workunit.client.0.smithi112.stdout:2/580: unlink d2/dd/d48/d22/d2b/d54/d60/d6c/d88/fae 0 2024-09-16T00:26:01.702 INFO:tasks.workunit.client.0.smithi112.stdout:8/484: dread d10/d4e/d58/f73 [0,4194304] 0 2024-09-16T00:26:01.702 INFO:tasks.workunit.client.0.smithi112.stdout:1/573: creat d0/d14/d6a/fbd x:0 0 0 2024-09-16T00:26:01.705 INFO:tasks.workunit.client.0.smithi112.stdout:0/600: creat d5/da/d18/d24/d5a/fbf x:0 0 0 2024-09-16T00:26:01.706 INFO:tasks.workunit.client.1.smithi169.stdout:4/374: unlink d2/d12/l19 0 2024-09-16T00:26:01.706 INFO:tasks.workunit.client.1.smithi169.stdout:7/455: getdents d7/d16/d49/d84 0 2024-09-16T00:26:01.707 INFO:tasks.workunit.client.1.smithi169.stdout:6/392: symlink d2/d4d/d59/l82 0 2024-09-16T00:26:01.709 INFO:tasks.workunit.client.0.smithi112.stdout:2/581: symlink d2/dd/d48/lb4 0 2024-09-16T00:26:01.710 INFO:tasks.workunit.client.0.smithi112.stdout:1/574: dread d0/f67 [0,4194304] 0 2024-09-16T00:26:01.710 INFO:tasks.workunit.client.0.smithi112.stdout:0/601: dread d5/da/d21/da5/faa [0,4194304] 0 2024-09-16T00:26:01.711 INFO:tasks.workunit.client.1.smithi169.stdout:7/456: link d7/d16/c62 d7/d16/d49/d84/c97 0 2024-09-16T00:26:01.711 INFO:tasks.workunit.client.1.smithi169.stdout:7/457: fsync d7/d16/f24 0 2024-09-16T00:26:01.712 INFO:tasks.workunit.client.0.smithi112.stdout:8/485: getdents d10/d12/d1c 0 2024-09-16T00:26:01.712 INFO:tasks.workunit.client.0.smithi112.stdout:0/602: stat d5/da/d18/d63/db2 0 2024-09-16T00:26:01.714 INFO:tasks.workunit.client.0.smithi112.stdout:2/582: truncate d2/dd/d48/d22/d2b/d30/f44 4209339 0 2024-09-16T00:26:01.714 INFO:tasks.workunit.client.0.smithi112.stdout:1/575: write d0/d14/d1c/d20/daf/f59 [4853499,84542] 0 2024-09-16T00:26:01.714 INFO:tasks.workunit.client.0.smithi112.stdout:8/486: rmdir d10/d12/d24 39 2024-09-16T00:26:01.716 INFO:tasks.workunit.client.1.smithi169.stdout:7/458: rmdir d7/df/d93 0 2024-09-16T00:26:01.716 INFO:tasks.workunit.client.0.smithi112.stdout:0/603: mkdir d5/da/d18/d24/d5a/dc0 0 2024-09-16T00:26:01.716 INFO:tasks.workunit.client.1.smithi169.stdout:7/459: chown d7/d16/d49/f68 1 1 2024-09-16T00:26:01.717 INFO:tasks.workunit.client.0.smithi112.stdout:1/576: creat d0/d14/d6a/fbe x:0 0 0 2024-09-16T00:26:01.719 INFO:tasks.workunit.client.0.smithi112.stdout:0/604: stat d5/da/d18/d24/d46/d5f/d7a/d8d/c92 0 2024-09-16T00:26:01.722 INFO:tasks.workunit.client.0.smithi112.stdout:1/577: write d0/d14/d6a/f80 [1453079,125416] 0 2024-09-16T00:26:01.723 INFO:tasks.workunit.client.1.smithi169.stdout:1/365: dread dc/d36/d3a/d45/d29/d2a/f62 [0,4194304] 0 2024-09-16T00:26:01.723 INFO:tasks.workunit.client.0.smithi112.stdout:2/583: dwrite d2/dd/f1a [4194304,4194304] 0 2024-09-16T00:26:01.725 INFO:tasks.workunit.client.0.smithi112.stdout:8/487: link d10/d4e/d58/c5e d10/d4e/d8c/d98/c9d 0 2024-09-16T00:26:01.725 INFO:tasks.workunit.client.1.smithi169.stdout:6/393: dread d2/f19 [0,4194304] 0 2024-09-16T00:26:01.726 INFO:tasks.workunit.client.0.smithi112.stdout:1/578: chown d0/d14/df/d28/f2d 469801 1 2024-09-16T00:26:01.728 INFO:tasks.workunit.client.1.smithi169.stdout:6/394: stat d2/d4d/d59 0 2024-09-16T00:26:01.728 INFO:tasks.workunit.client.0.smithi112.stdout:0/605: creat d5/da/d1d/d71/d64/db6/fc1 x:0 0 0 2024-09-16T00:26:01.730 INFO:tasks.workunit.client.1.smithi169.stdout:1/366: dread dc/d18/f2b [0,4194304] 0 2024-09-16T00:26:01.732 INFO:tasks.workunit.client.1.smithi169.stdout:6/395: mknod d2/d3f/d41/c83 0 2024-09-16T00:26:01.733 INFO:tasks.workunit.client.0.smithi112.stdout:7/573: write d6/d10/d1b/fbe [3560074,80759] 0 2024-09-16T00:26:01.735 INFO:tasks.workunit.client.1.smithi169.stdout:1/367: mkdir dc/d6c 0 2024-09-16T00:26:01.735 INFO:tasks.workunit.client.0.smithi112.stdout:0/606: unlink d5/da/d57/d76/f6b 0 2024-09-16T00:26:01.736 INFO:tasks.workunit.client.0.smithi112.stdout:2/584: dwrite d2/f19 [4194304,4194304] 0 2024-09-16T00:26:01.736 INFO:tasks.workunit.client.1.smithi169.stdout:6/396: creat d2/d3f/d41/d72/f84 x:0 0 0 2024-09-16T00:26:01.738 INFO:tasks.workunit.client.0.smithi112.stdout:1/579: symlink d0/d14/df/d28/d45/d46/d58/d62/db0/lbf 0 2024-09-16T00:26:01.741 INFO:tasks.workunit.client.0.smithi112.stdout:0/607: write d5/da/f53 [1996269,11338] 0 2024-09-16T00:26:01.742 INFO:tasks.workunit.client.0.smithi112.stdout:9/720: dwrite d1/d5/d20/f36 [0,4194304] 0 2024-09-16T00:26:01.743 INFO:tasks.workunit.client.0.smithi112.stdout:7/574: write d6/d10/d75/d8f/d90/fb8 [10905,12236] 0 2024-09-16T00:26:01.744 INFO:tasks.workunit.client.1.smithi169.stdout:6/397: write d2/d3f/d41/d5c/f65 [4584415,100997] 0 2024-09-16T00:26:01.745 INFO:tasks.workunit.client.0.smithi112.stdout:8/488: dread d10/f1b [0,4194304] 0 2024-09-16T00:26:01.746 INFO:tasks.workunit.client.1.smithi169.stdout:1/368: dwrite dc/d18/f20 [0,4194304] 0 2024-09-16T00:26:01.747 INFO:tasks.workunit.client.0.smithi112.stdout:9/721: fsync d1/d5/d6/fa4 0 2024-09-16T00:26:01.748 INFO:tasks.workunit.client.0.smithi112.stdout:0/608: write d5/da/d57/d76/f97 [3100884,23999] 0 2024-09-16T00:26:01.749 INFO:tasks.workunit.client.0.smithi112.stdout:1/580: creat d0/d14/d1c/d26/d41/fc0 x:0 0 0 2024-09-16T00:26:01.750 INFO:tasks.workunit.client.1.smithi169.stdout:4/375: dread d2/d12/d13/d1f/d3e/d5b/f24 [0,4194304] 0 2024-09-16T00:26:01.752 INFO:tasks.workunit.client.1.smithi169.stdout:4/376: chown d2/d12/d13/d1f/c64 8 1 2024-09-16T00:26:01.753 INFO:tasks.workunit.client.1.smithi169.stdout:4/377: readlink d2/d12/d13/d5c/l6f 0 2024-09-16T00:26:01.754 INFO:tasks.workunit.client.1.smithi169.stdout:6/398: creat d2/d3f/f85 x:0 0 0 2024-09-16T00:26:01.755 INFO:tasks.workunit.client.1.smithi169.stdout:1/369: fsync dc/d18/f59 0 2024-09-16T00:26:01.757 INFO:tasks.workunit.client.0.smithi112.stdout:2/585: rename d2/dd/d48/d94/l63 to d2/dd/d48/d22/d2b/d54/d60/d6c/d88/da4/lb5 0 2024-09-16T00:26:01.757 INFO:tasks.workunit.client.0.smithi112.stdout:0/609: write d5/da/d57/d76/f14 [266891,35626] 0 2024-09-16T00:26:01.759 INFO:tasks.workunit.client.0.smithi112.stdout:8/489: symlink d10/d12/d24/l9e 0 2024-09-16T00:26:01.760 INFO:tasks.workunit.client.1.smithi169.stdout:1/370: stat dc/d36/d3a/d45/c30 0 2024-09-16T00:26:01.762 INFO:tasks.workunit.client.1.smithi169.stdout:4/378: creat d2/d12/d13/d1f/d3e/d5b/d70/f78 x:0 0 0 2024-09-16T00:26:01.766 INFO:tasks.workunit.client.0.smithi112.stdout:8/490: unlink d10/d12/d24/f6e 0 2024-09-16T00:26:01.766 INFO:tasks.workunit.client.0.smithi112.stdout:7/575: dwrite d6/d10/d1b/d39/f92 [0,4194304] 0 2024-09-16T00:26:01.767 INFO:tasks.workunit.client.0.smithi112.stdout:9/722: rename d1/d5/d6/cf0 to d1/d5/d6/d2d/deb/cfb 0 2024-09-16T00:26:01.767 INFO:tasks.workunit.client.1.smithi169.stdout:6/399: dwrite d2/d4d/d59/d43/f4a [0,4194304] 0 2024-09-16T00:26:01.768 INFO:tasks.workunit.client.1.smithi169.stdout:4/379: fdatasync d2/d12/d13/d1f/f48 0 2024-09-16T00:26:01.768 INFO:tasks.workunit.client.0.smithi112.stdout:8/491: dread - d10/d12/d1c/d61/d6c/f76 zero size 2024-09-16T00:26:01.770 INFO:tasks.workunit.client.0.smithi112.stdout:0/610: symlink d5/da/d1d/d71/d64/d8f/lc2 0 2024-09-16T00:26:01.771 INFO:tasks.workunit.client.0.smithi112.stdout:9/723: dread - d1/d76/fe2 zero size 2024-09-16T00:26:01.772 INFO:tasks.workunit.client.0.smithi112.stdout:1/581: dwrite d0/d14/d6a/f6f [0,4194304] 0 2024-09-16T00:26:01.773 INFO:tasks.workunit.client.1.smithi169.stdout:6/400: rmdir d2/d3f/d41/d72 39 2024-09-16T00:26:01.774 INFO:tasks.workunit.client.0.smithi112.stdout:0/611: chown d5/da/d18/d24/d5a/fbf 373124290 1 2024-09-16T00:26:01.775 INFO:tasks.workunit.client.1.smithi169.stdout:1/371: dwrite dc/d26/f37 [0,4194304] 0 2024-09-16T00:26:01.775 INFO:tasks.workunit.client.0.smithi112.stdout:8/492: rename d10/d4e/d58/d87/c88 to d10/d4e/d58/d87/c9f 0 2024-09-16T00:26:01.777 INFO:tasks.workunit.client.0.smithi112.stdout:1/582: stat d0/d14/d6a/d9e 0 2024-09-16T00:26:01.780 INFO:tasks.workunit.client.0.smithi112.stdout:7/576: rename d6/d10/d1b to d6/d10/d16/d96/dc7 0 2024-09-16T00:26:01.780 INFO:tasks.workunit.client.0.smithi112.stdout:9/724: fsync d1/d5/f2f 0 2024-09-16T00:26:01.780 INFO:tasks.workunit.client.0.smithi112.stdout:8/493: dread - d10/d12/d1c/d61/d6c/f93 zero size 2024-09-16T00:26:01.781 INFO:tasks.workunit.client.0.smithi112.stdout:2/586: dwrite d2/dd/d48/d22/d67/fb2 [0,4194304] 0 2024-09-16T00:26:01.782 INFO:tasks.workunit.client.1.smithi169.stdout:1/372: dread - dc/d36/d3a/d45/d29/f52 zero size 2024-09-16T00:26:01.785 INFO:tasks.workunit.client.0.smithi112.stdout:8/494: chown d10/d12/d24/l60 581625865 1 2024-09-16T00:26:01.785 INFO:tasks.workunit.client.0.smithi112.stdout:0/612: chown d5/da/d57/d76/c1a 11415651 1 2024-09-16T00:26:01.786 INFO:tasks.workunit.client.0.smithi112.stdout:0/613: stat d5/da/d1d 0 2024-09-16T00:26:01.787 INFO:tasks.workunit.client.1.smithi169.stdout:1/373: rename dc/d2c/d3d/f5e to dc/d26/d5d/f6d 0 2024-09-16T00:26:01.788 INFO:tasks.workunit.client.0.smithi112.stdout:0/614: chown d5/da/d57/db0 35298 1 2024-09-16T00:26:01.790 INFO:tasks.workunit.client.0.smithi112.stdout:1/583: unlink d0/d14/d44/l8a 0 2024-09-16T00:26:01.790 INFO:tasks.workunit.client.1.smithi169.stdout:1/374: unlink dc/d18/f3b 0 2024-09-16T00:26:01.791 INFO:tasks.workunit.client.1.smithi169.stdout:6/401: dwrite d2/d3f/d41/f80 [0,4194304] 0 2024-09-16T00:26:01.797 INFO:tasks.workunit.client.0.smithi112.stdout:1/584: stat d0/d14/d1c/d20/f33 0 2024-09-16T00:26:01.798 INFO:tasks.workunit.client.1.smithi169.stdout:1/375: truncate dc/d36/d3a/d45/f23 4936428 0 2024-09-16T00:26:01.798 INFO:tasks.workunit.client.1.smithi169.stdout:1/376: fsync dc/f66 0 2024-09-16T00:26:01.799 INFO:tasks.workunit.client.0.smithi112.stdout:0/615: dwrite d5/da/d18/d89/f98 [0,4194304] 0 2024-09-16T00:26:01.799 INFO:tasks.workunit.client.0.smithi112.stdout:2/587: creat d2/fb6 x:0 0 0 2024-09-16T00:26:01.801 INFO:tasks.workunit.client.1.smithi169.stdout:6/402: dwrite d2/d3f/d41/d5c/d73/f5b [0,4194304] 0 2024-09-16T00:26:01.801 INFO:tasks.workunit.client.0.smithi112.stdout:1/585: write d0/d14/df/f36 [1035520,72178] 0 2024-09-16T00:26:01.802 INFO:tasks.workunit.client.0.smithi112.stdout:8/495: rename d10/d12/f1a to d10/d12/d1c/d75/fa0 0 2024-09-16T00:26:01.802 INFO:tasks.workunit.client.1.smithi169.stdout:1/377: creat dc/d26/d5d/f6e x:0 0 0 2024-09-16T00:26:01.806 INFO:tasks.workunit.client.1.smithi169.stdout:6/403: mknod d2/d3f/d41/d72/c86 0 2024-09-16T00:26:01.807 INFO:tasks.workunit.client.0.smithi112.stdout:2/588: unlink d2/dd/d48/d22/d2b/f73 0 2024-09-16T00:26:01.807 INFO:tasks.workunit.client.0.smithi112.stdout:1/586: chown d0/d14/d44/caa 3 1 2024-09-16T00:26:01.811 INFO:tasks.workunit.client.0.smithi112.stdout:0/616: dwrite d5/da/d21/da5/faa [0,4194304] 0 2024-09-16T00:26:01.812 INFO:tasks.workunit.client.1.smithi169.stdout:6/404: mkdir d2/d4d/d59/d43/d67/d87 0 2024-09-16T00:26:01.812 INFO:tasks.workunit.client.1.smithi169.stdout:1/378: dwrite dc/d26/f46 [0,4194304] 0 2024-09-16T00:26:01.814 INFO:tasks.workunit.client.1.smithi169.stdout:6/405: dread - d2/d36/f58 zero size 2024-09-16T00:26:01.814 INFO:tasks.workunit.client.0.smithi112.stdout:8/496: creat d10/d4e/d58/fa1 x:0 0 0 2024-09-16T00:26:01.816 INFO:tasks.workunit.client.1.smithi169.stdout:1/379: dread dc/f1d [0,4194304] 0 2024-09-16T00:26:01.817 INFO:tasks.workunit.client.0.smithi112.stdout:0/617: write d5/da/d57/d76/f17 [409789,866] 0 2024-09-16T00:26:01.817 INFO:tasks.workunit.client.1.smithi169.stdout:1/380: chown f7 57568948 1 2024-09-16T00:26:01.819 INFO:tasks.workunit.client.1.smithi169.stdout:1/381: fdatasync f7 0 2024-09-16T00:26:01.819 INFO:tasks.workunit.client.0.smithi112.stdout:0/618: readlink d5/da/d18/d63/l78 0 2024-09-16T00:26:01.819 INFO:tasks.workunit.client.0.smithi112.stdout:8/497: creat d10/d12/d1c/d61/fa2 x:0 0 0 2024-09-16T00:26:01.823 INFO:tasks.workunit.client.1.smithi169.stdout:1/382: dread dc/f1d [0,4194304] 0 2024-09-16T00:26:01.823 INFO:tasks.workunit.client.1.smithi169.stdout:1/383: chown dc/d26 1 1 2024-09-16T00:26:01.824 INFO:tasks.workunit.client.0.smithi112.stdout:0/619: chown d5/da/d18/d24/d5a/f6c 24876395 1 2024-09-16T00:26:01.825 INFO:tasks.workunit.client.0.smithi112.stdout:1/587: dread d0/d14/df/d28/f2d [0,4194304] 0 2024-09-16T00:26:01.825 INFO:tasks.workunit.client.0.smithi112.stdout:2/589: dwrite d2/dd/d48/d22/f29 [0,4194304] 0 2024-09-16T00:26:01.825 INFO:tasks.workunit.client.0.smithi112.stdout:8/498: truncate d10/d4e/d58/f91 812177 0 2024-09-16T00:26:01.825 INFO:tasks.workunit.client.1.smithi169.stdout:1/384: write dc/d36/d3a/d45/f23 [3879053,9136] 0 2024-09-16T00:26:01.827 INFO:tasks.workunit.client.1.smithi169.stdout:4/380: write d2/d12/d13/d4e/f77 [4297534,62688] 0 2024-09-16T00:26:01.829 INFO:tasks.workunit.client.0.smithi112.stdout:0/620: truncate d5/da/d66/fb7 53080 0 2024-09-16T00:26:01.829 INFO:tasks.workunit.client.0.smithi112.stdout:8/499: creat d10/d12/d1c/d61/fa3 x:0 0 0 2024-09-16T00:26:01.831 INFO:tasks.workunit.client.1.smithi169.stdout:4/381: readlink d2/d12/l23 0 2024-09-16T00:26:01.833 INFO:tasks.workunit.client.0.smithi112.stdout:9/725: write d1/f62 [2095245,87131] 0 2024-09-16T00:26:01.834 INFO:tasks.workunit.client.0.smithi112.stdout:2/590: getdents d2/dd/d48/d22 0 2024-09-16T00:26:01.835 INFO:tasks.workunit.client.0.smithi112.stdout:0/621: mkdir d5/da/d21/dc3 0 2024-09-16T00:26:01.835 INFO:tasks.workunit.client.1.smithi169.stdout:1/385: creat dc/f6f x:0 0 0 2024-09-16T00:26:01.837 INFO:tasks.workunit.client.0.smithi112.stdout:1/588: dwrite d0/d14/df/d28/d45/d30/f9b [0,4194304] 0 2024-09-16T00:26:01.838 INFO:tasks.workunit.client.1.smithi169.stdout:1/386: fsync dc/d18/f20 0 2024-09-16T00:26:01.841 INFO:tasks.workunit.client.1.smithi169.stdout:4/382: dwrite d2/d12/d13/f6c [0,4194304] 0 2024-09-16T00:26:01.842 INFO:tasks.workunit.client.1.smithi169.stdout:3/387: sync 2024-09-16T00:26:01.844 INFO:tasks.workunit.client.1.smithi169.stdout:3/388: fsync d7/d50/f79 0 2024-09-16T00:26:01.844 INFO:tasks.workunit.client.0.smithi112.stdout:8/500: dwrite d10/d12/f8f [0,4194304] 0 2024-09-16T00:26:01.846 INFO:tasks.workunit.client.1.smithi169.stdout:1/387: symlink dc/d36/d39/d50/l70 0 2024-09-16T00:26:01.846 INFO:tasks.workunit.client.1.smithi169.stdout:4/383: stat d2/d12/d13/d27/l63 0 2024-09-16T00:26:01.847 INFO:tasks.workunit.client.1.smithi169.stdout:3/389: dread - d7/d50/f79 zero size 2024-09-16T00:26:01.847 INFO:tasks.workunit.client.1.smithi169.stdout:4/384: chown d2/d12/d13/d4d 273 1 2024-09-16T00:26:01.848 INFO:tasks.workunit.client.0.smithi112.stdout:0/622: dwrite d5/da/dc/f7d [0,4194304] 0 2024-09-16T00:26:01.849 INFO:tasks.workunit.client.1.smithi169.stdout:3/390: creat d7/d1a/d1c/d23/d70/f95 x:0 0 0 2024-09-16T00:26:01.849 INFO:tasks.workunit.client.1.smithi169.stdout:1/388: symlink dc/d36/d3a/d45/l71 0 2024-09-16T00:26:01.852 INFO:tasks.workunit.client.1.smithi169.stdout:4/385: creat d2/d12/d13/d4e/d50/f79 x:0 0 0 2024-09-16T00:26:01.853 INFO:tasks.workunit.client.1.smithi169.stdout:1/389: mkdir dc/d26/d5d/d72 0 2024-09-16T00:26:01.855 INFO:tasks.workunit.client.1.smithi169.stdout:3/391: symlink d7/d50/d90/l96 0 2024-09-16T00:26:01.856 INFO:tasks.workunit.client.1.smithi169.stdout:4/386: creat d2/d54/f7a x:0 0 0 2024-09-16T00:26:01.860 INFO:tasks.workunit.client.1.smithi169.stdout:3/392: creat d7/d50/f97 x:0 0 0 2024-09-16T00:26:01.861 INFO:tasks.workunit.client.1.smithi169.stdout:1/390: mkdir dc/d26/d5d/d72/d73 0 2024-09-16T00:26:01.864 INFO:tasks.workunit.client.1.smithi169.stdout:4/387: creat d2/d12/d13/d1f/d2f/f7b x:0 0 0 2024-09-16T00:26:01.864 INFO:tasks.workunit.client.1.smithi169.stdout:1/391: write dc/d36/d3a/d45/d29/d4d/f54 [1771002,110940] 0 2024-09-16T00:26:01.865 INFO:tasks.workunit.client.1.smithi169.stdout:4/388: stat d2/d12/l23 0 2024-09-16T00:26:01.865 INFO:tasks.workunit.client.1.smithi169.stdout:6/406: dread d2/d22/d51/f77 [4194304,4194304] 0 2024-09-16T00:26:01.867 INFO:tasks.workunit.client.1.smithi169.stdout:1/392: fdatasync dc/fd 0 2024-09-16T00:26:01.869 INFO:tasks.workunit.client.1.smithi169.stdout:6/407: write d2/d4d/d59/d38/f4f [2524415,96012] 0 2024-09-16T00:26:01.869 INFO:tasks.workunit.client.1.smithi169.stdout:4/389: mknod d2/d12/d13/d1f/d3e/d5b/d1a/c7c 0 2024-09-16T00:26:01.870 INFO:tasks.workunit.client.0.smithi112.stdout:9/726: creat d1/d5/d6/d2d/d39/ffc x:0 0 0 2024-09-16T00:26:01.873 INFO:tasks.workunit.client.1.smithi169.stdout:6/408: chown d2/d3f/d41/d72/d81 7451 1 2024-09-16T00:26:01.873 INFO:tasks.workunit.client.0.smithi112.stdout:1/589: mknod d0/d14/df/d28/d89/cc1 0 2024-09-16T00:26:01.876 INFO:tasks.workunit.client.0.smithi112.stdout:8/501: rmdir d10 39 2024-09-16T00:26:01.877 INFO:tasks.workunit.client.0.smithi112.stdout:9/727: truncate d1/d5/d20/d81/d1a/d88/def/f2e 1199840 0 2024-09-16T00:26:01.877 INFO:tasks.workunit.client.1.smithi169.stdout:6/409: rmdir d2/d4d/d59/d43/d67 39 2024-09-16T00:26:01.877 INFO:tasks.workunit.client.1.smithi169.stdout:3/393: dwrite d7/d50/f87 [4194304,4194304] 0 2024-09-16T00:26:01.878 INFO:tasks.workunit.client.1.smithi169.stdout:4/390: write d2/d12/d13/d1f/d3e/d5b/f24 [2002801,121166] 0 2024-09-16T00:26:01.879 INFO:tasks.workunit.client.1.smithi169.stdout:3/394: chown d7/d1a/d2b/l5d 179 1 2024-09-16T00:26:01.880 INFO:tasks.workunit.client.0.smithi112.stdout:2/591: link d2/dd/d48/lb4 d2/dd/d48/d22/d2b/d54/lb7 0 2024-09-16T00:26:01.880 INFO:tasks.workunit.client.0.smithi112.stdout:9/728: creat d1/d5/d6/d38/ffd x:0 0 0 2024-09-16T00:26:01.881 INFO:tasks.workunit.client.0.smithi112.stdout:8/502: write d10/d4e/d58/f66 [4114299,7887] 0 2024-09-16T00:26:01.881 INFO:tasks.workunit.client.0.smithi112.stdout:1/590: mknod d0/d14/cc2 0 2024-09-16T00:26:01.883 INFO:tasks.workunit.client.0.smithi112.stdout:0/623: creat d5/da/d18/fc4 x:0 0 0 2024-09-16T00:26:01.883 INFO:tasks.workunit.client.0.smithi112.stdout:8/503: write d10/d12/f8f [1208148,65255] 0 2024-09-16T00:26:01.885 INFO:tasks.workunit.client.0.smithi112.stdout:8/504: stat d10/d12/d1c/l36 0 2024-09-16T00:26:01.887 INFO:tasks.workunit.client.1.smithi169.stdout:1/393: rename dc/d18/f20 to dc/d26/d5d/d72/d73/f74 0 2024-09-16T00:26:01.888 INFO:tasks.workunit.client.0.smithi112.stdout:9/729: mknod d1/d5/d6/d2d/cfe 0 2024-09-16T00:26:01.889 INFO:tasks.workunit.client.1.smithi169.stdout:3/395: dwrite d7/d1a/d1c/f54 [4194304,4194304] 0 2024-09-16T00:26:01.889 INFO:tasks.workunit.client.0.smithi112.stdout:0/624: mkdir d5/d90/dc5 0 2024-09-16T00:26:01.891 INFO:tasks.workunit.client.1.smithi169.stdout:3/396: write d7/d1a/d2b/f8c [650891,19227] 0 2024-09-16T00:26:01.892 INFO:tasks.workunit.client.0.smithi112.stdout:8/505: unlink d10/d12/d1c/d61/d6c/f70 0 2024-09-16T00:26:01.893 INFO:tasks.workunit.client.1.smithi169.stdout:1/394: mkdir dc/d36/d39/d50/d75 0 2024-09-16T00:26:01.893 INFO:tasks.workunit.client.0.smithi112.stdout:9/730: chown d1/d5/d20/d81/faa 56 1 2024-09-16T00:26:01.893 INFO:tasks.workunit.client.1.smithi169.stdout:4/391: mkdir d2/d12/d13/d1f/d3e/d5b/d70/d7d 0 2024-09-16T00:26:01.894 INFO:tasks.workunit.client.0.smithi112.stdout:2/592: dread d2/dd/d48/d22/d67/fb2 [0,4194304] 0 2024-09-16T00:26:01.894 INFO:tasks.workunit.client.1.smithi169.stdout:1/395: fdatasync dc/d36/d3a/d45/f1c 0 2024-09-16T00:26:01.895 INFO:tasks.workunit.client.1.smithi169.stdout:1/396: chown dc/d26/d5d 4 1 2024-09-16T00:26:01.896 INFO:tasks.workunit.client.1.smithi169.stdout:3/397: creat d7/d1a/d1c/d23/d70/f98 x:0 0 0 2024-09-16T00:26:01.897 INFO:tasks.workunit.client.0.smithi112.stdout:7/577: dwrite d6/d10/d16/f3e [4194304,4194304] 0 2024-09-16T00:26:01.899 INFO:tasks.workunit.client.0.smithi112.stdout:1/591: dwrite d0/d14/d1c/d26/d41/fc0 [0,4194304] 0 2024-09-16T00:26:01.900 INFO:tasks.workunit.client.0.smithi112.stdout:8/506: creat d10/d12/d1c/fa4 x:0 0 0 2024-09-16T00:26:01.901 INFO:tasks.workunit.client.0.smithi112.stdout:9/731: unlink d1/d5/d20/d81/d1a/d88/fb3 0 2024-09-16T00:26:01.901 INFO:tasks.workunit.client.0.smithi112.stdout:2/593: stat d2/dd/d48/d22/d2b/d30 0 2024-09-16T00:26:01.902 INFO:tasks.workunit.client.1.smithi169.stdout:3/398: mknod d7/d50/d6f/c99 0 2024-09-16T00:26:01.903 INFO:tasks.workunit.client.0.smithi112.stdout:9/732: stat d1 0 2024-09-16T00:26:01.905 INFO:tasks.workunit.client.1.smithi169.stdout:6/410: dwrite d2/d36/f31 [4194304,4194304] 0 2024-09-16T00:26:01.907 INFO:tasks.workunit.client.1.smithi169.stdout:4/392: dwrite d2/d12/f31 [0,4194304] 0 2024-09-16T00:26:01.908 INFO:tasks.workunit.client.0.smithi112.stdout:7/578: write d6/d10/d75/d8f/fc5 [320917,56404] 0 2024-09-16T00:26:01.908 INFO:tasks.workunit.client.0.smithi112.stdout:1/592: mknod d0/d14/d1c/cc3 0 2024-09-16T00:26:01.908 INFO:tasks.workunit.client.0.smithi112.stdout:0/625: dwrite d5/da/d18/d24/d46/d88/f8c [0,4194304] 0 2024-09-16T00:26:01.909 INFO:tasks.workunit.client.0.smithi112.stdout:9/733: write d1/d5/d20/d81/d1a/d88/def/f43 [3677638,52881] 0 2024-09-16T00:26:01.910 INFO:tasks.workunit.client.1.smithi169.stdout:3/399: unlink d7/d1a/d2b/f45 0 2024-09-16T00:26:01.911 INFO:tasks.workunit.client.1.smithi169.stdout:4/393: write d2/d12/d13/d4e/d50/f79 [316643,111910] 0 2024-09-16T00:26:01.911 INFO:tasks.workunit.client.0.smithi112.stdout:2/594: unlink d2/dd/d48/d22/d67/fa7 0 2024-09-16T00:26:01.912 INFO:tasks.workunit.client.0.smithi112.stdout:9/734: write d1/d5/d6/d2d/fbc [1525689,6296] 0 2024-09-16T00:26:01.912 INFO:tasks.workunit.client.0.smithi112.stdout:0/626: write d5/f23 [3167029,39660] 0 2024-09-16T00:26:01.914 INFO:tasks.workunit.client.1.smithi169.stdout:6/411: write d2/d22/d51/f77 [8228119,57404] 0 2024-09-16T00:26:01.915 INFO:tasks.workunit.client.1.smithi169.stdout:4/394: fdatasync d2/d12/d13/d1f/d3e/d5b/d70/f78 0 2024-09-16T00:26:01.921 INFO:tasks.workunit.client.1.smithi169.stdout:4/395: chown d2/l7 104762048 1 2024-09-16T00:26:01.923 INFO:tasks.workunit.client.0.smithi112.stdout:0/627: write d5/da/d18/f62 [990899,4426] 0 2024-09-16T00:26:01.926 INFO:tasks.workunit.client.1.smithi169.stdout:4/396: unlink d2/d12/f57 0 2024-09-16T00:26:01.927 INFO:tasks.workunit.client.0.smithi112.stdout:7/579: dwrite d6/f9 [0,4194304] 0 2024-09-16T00:26:01.927 INFO:tasks.workunit.client.0.smithi112.stdout:2/595: read d2/f46 [1500853,58636] 0 2024-09-16T00:26:01.928 INFO:tasks.workunit.client.1.smithi169.stdout:6/412: rename d2/d22/f78 to d2/d3f/d41/d5c/f88 0 2024-09-16T00:26:01.929 INFO:tasks.workunit.client.0.smithi112.stdout:7/580: dread - d6/d10/d16/d96/dc7/fb6 zero size 2024-09-16T00:26:01.931 INFO:tasks.workunit.client.1.smithi169.stdout:6/413: truncate d2/d36/f58 957806 0 2024-09-16T00:26:01.931 INFO:tasks.workunit.client.1.smithi169.stdout:4/397: mknod d2/d12/d13/d1f/d3e/c7e 0 2024-09-16T00:26:01.933 INFO:tasks.workunit.client.0.smithi112.stdout:1/593: mkdir d0/d14/d44/dc4 0 2024-09-16T00:26:01.933 INFO:tasks.workunit.client.1.smithi169.stdout:6/414: dread - d2/d36/f70 zero size 2024-09-16T00:26:01.933 INFO:tasks.workunit.client.1.smithi169.stdout:6/415: fsync d2/d36/f70 0 2024-09-16T00:26:01.934 INFO:tasks.workunit.client.1.smithi169.stdout:6/416: fdatasync d2/d4d/d59/d43/f4a 0 2024-09-16T00:26:01.936 INFO:tasks.workunit.client.0.smithi112.stdout:7/581: unlink d6/d10/d16/d96/dc7/d39/fad 0 2024-09-16T00:26:01.937 INFO:tasks.workunit.client.0.smithi112.stdout:8/507: dread d10/f1f [0,4194304] 0 2024-09-16T00:26:01.938 INFO:tasks.workunit.client.1.smithi169.stdout:4/398: rmdir d2/d12/d13/d4b 0 2024-09-16T00:26:01.941 INFO:tasks.workunit.client.0.smithi112.stdout:7/582: mknod d6/d10/d16/d49/d6c/cc8 0 2024-09-16T00:26:01.943 INFO:tasks.workunit.client.0.smithi112.stdout:8/508: mkdir d10/d4e/d8c/d7c/da5 0 2024-09-16T00:26:01.945 INFO:tasks.workunit.client.0.smithi112.stdout:2/596: creat d2/dd/d48/d22/d2b/d54/d60/d6c/d88/dad/fb8 x:0 0 0 2024-09-16T00:26:01.946 INFO:tasks.workunit.client.0.smithi112.stdout:8/509: mknod d10/d4e/d8c/ca6 0 2024-09-16T00:26:01.948 INFO:tasks.workunit.client.0.smithi112.stdout:1/594: dwrite d0/f1e [0,4194304] 0 2024-09-16T00:26:01.948 INFO:tasks.workunit.client.0.smithi112.stdout:9/735: dread d1/d5/d20/f69 [4194304,4194304] 0 2024-09-16T00:26:01.949 INFO:tasks.workunit.client.0.smithi112.stdout:7/583: rename d6/d10/d16/d49/d6c/lba to d6/lc9 0 2024-09-16T00:26:01.949 INFO:tasks.workunit.client.0.smithi112.stdout:2/597: write d2/dd/d48/d94/d49/f7d [1213870,13190] 0 2024-09-16T00:26:01.951 INFO:tasks.workunit.client.0.smithi112.stdout:1/595: chown d0/d14/d1c/d20/daf/lb1 0 1 2024-09-16T00:26:01.951 INFO:tasks.workunit.client.0.smithi112.stdout:7/584: write d6/fa0 [122024,111140] 0 2024-09-16T00:26:01.954 INFO:tasks.workunit.client.0.smithi112.stdout:1/596: chown d0/d14/d1c/d20/c68 906 1 2024-09-16T00:26:01.955 INFO:tasks.workunit.client.0.smithi112.stdout:7/585: truncate d6/d10/d16/d96/f9f 1036892 0 2024-09-16T00:26:01.956 INFO:tasks.workunit.client.0.smithi112.stdout:7/586: stat d6/d10/f87 0 2024-09-16T00:26:01.958 INFO:tasks.workunit.client.0.smithi112.stdout:9/736: rename d1/d5/d20/d81/dc4/de1 to d1/d5/d20/d81/dff 0 2024-09-16T00:26:01.959 INFO:tasks.workunit.client.0.smithi112.stdout:9/737: chown d1/d5/d20/d81/d1a/d88/def/l53 217022 1 2024-09-16T00:26:01.960 INFO:tasks.workunit.client.0.smithi112.stdout:7/587: rename d6/d10/d75/l8e to d6/d10/d75/lca 0 2024-09-16T00:26:01.963 INFO:tasks.workunit.client.1.smithi169.stdout:3/400: rmdir d7/d50/d6f 39 2024-09-16T00:26:01.964 INFO:tasks.workunit.client.0.smithi112.stdout:9/738: symlink d1/d5/d6/d59/d64/l100 0 2024-09-16T00:26:01.965 INFO:tasks.workunit.client.1.smithi169.stdout:3/401: chown d7/d1a/d1c/d23/f4c 22 1 2024-09-16T00:26:01.966 INFO:tasks.workunit.client.0.smithi112.stdout:7/588: unlink d6/d10/d16/d96/dc7/f74 0 2024-09-16T00:26:01.967 INFO:tasks.workunit.client.0.smithi112.stdout:7/589: chown d6/d10/d16/f3e 448 1 2024-09-16T00:26:01.968 INFO:tasks.workunit.client.0.smithi112.stdout:9/739: mknod d1/d5/d6/df8/dba/c101 0 2024-09-16T00:26:01.971 INFO:tasks.workunit.client.1.smithi169.stdout:6/417: truncate d2/d3f/d41/d52/f1b 3439768 0 2024-09-16T00:26:01.971 INFO:tasks.workunit.client.0.smithi112.stdout:9/740: unlink d1/d5/d6/d2d/d6a/fcd 0 2024-09-16T00:26:01.973 INFO:tasks.workunit.client.0.smithi112.stdout:9/741: write d1/d5/d20/d81/d1a/d88/def/f43 [5030596,42605] 0 2024-09-16T00:26:01.973 INFO:tasks.workunit.client.1.smithi169.stdout:6/418: truncate d2/d4d/d59/d38/f62 4964677 0 2024-09-16T00:26:01.973 INFO:tasks.workunit.client.1.smithi169.stdout:6/419: chown d2/d3f/c47 384 1 2024-09-16T00:26:01.975 INFO:tasks.workunit.client.0.smithi112.stdout:9/742: unlink d1/d5/d20/fae 0 2024-09-16T00:26:01.976 INFO:tasks.workunit.client.1.smithi169.stdout:6/420: unlink d2/ce 0 2024-09-16T00:26:01.977 INFO:tasks.workunit.client.0.smithi112.stdout:9/743: write d1/d5/d20/d81/d1a/d88/def/fe0 [974550,6358] 0 2024-09-16T00:26:01.977 INFO:tasks.workunit.client.1.smithi169.stdout:6/421: readlink d2/d4d/d59/l82 0 2024-09-16T00:26:01.978 INFO:tasks.workunit.client.1.smithi169.stdout:6/422: read d2/fb [7691528,1254] 0 2024-09-16T00:26:01.978 INFO:tasks.workunit.client.0.smithi112.stdout:9/744: creat d1/d5/d2c/f102 x:0 0 0 2024-09-16T00:26:01.980 INFO:tasks.workunit.client.0.smithi112.stdout:9/745: mknod d1/d5/d6/d2d/d6a/c103 0 2024-09-16T00:26:01.981 INFO:tasks.workunit.client.1.smithi169.stdout:6/423: unlink d2/d4d/d59/d74/l79 0 2024-09-16T00:26:01.982 INFO:tasks.workunit.client.0.smithi112.stdout:9/746: rename d1/d5/d20/f36 to d1/d5/d2c/ded/f104 0 2024-09-16T00:26:01.983 INFO:tasks.workunit.client.1.smithi169.stdout:6/424: creat d2/d3f/d41/d52/f89 x:0 0 0 2024-09-16T00:26:01.983 INFO:tasks.workunit.client.0.smithi112.stdout:9/747: readlink d1/d5/d20/d81/d1a/d88/def/l6b 0 2024-09-16T00:26:01.984 INFO:tasks.workunit.client.0.smithi112.stdout:9/748: truncate d1/d5/d6/fa4 171441 0 2024-09-16T00:26:01.984 INFO:tasks.workunit.client.0.smithi112.stdout:0/628: write d5/da/f2e [508646,73706] 0 2024-09-16T00:26:01.986 INFO:tasks.workunit.client.0.smithi112.stdout:0/629: truncate d5/da/d18/d63/f65 4781218 0 2024-09-16T00:26:01.988 INFO:tasks.workunit.client.1.smithi169.stdout:6/425: symlink d2/d4d/d59/d43/d67/d87/l8a 0 2024-09-16T00:26:01.988 INFO:tasks.workunit.client.1.smithi169.stdout:6/426: readlink d2/d3f/l66 0 2024-09-16T00:26:01.988 INFO:tasks.workunit.client.0.smithi112.stdout:0/630: chown d5/da/d66 3206337 1 2024-09-16T00:26:01.989 INFO:tasks.workunit.client.0.smithi112.stdout:1/597: truncate d0/d14/d1c/d20/daf/f59 4041826 0 2024-09-16T00:26:01.991 INFO:tasks.workunit.client.0.smithi112.stdout:7/590: write d6/d10/f12 [2153118,8738] 0 2024-09-16T00:26:01.992 INFO:tasks.workunit.client.1.smithi169.stdout:6/427: unlink d2/d22/d51/f6a 0 2024-09-16T00:26:01.993 INFO:tasks.workunit.client.1.smithi169.stdout:3/402: dread d7/d50/f87 [0,4194304] 0 2024-09-16T00:26:01.994 INFO:tasks.workunit.client.0.smithi112.stdout:0/631: chown d5/da/d18/d24/d5a/dc0 20133067 1 2024-09-16T00:26:01.995 INFO:tasks.workunit.client.1.smithi169.stdout:3/403: readlink d7/d1a/d1c/d35/d63/l6b 0 2024-09-16T00:26:01.996 INFO:tasks.workunit.client.0.smithi112.stdout:1/598: write d0/d14/df/f5b [51350,107792] 0 2024-09-16T00:26:01.996 INFO:tasks.workunit.client.0.smithi112.stdout:7/591: fsync d6/d10/f12 0 2024-09-16T00:26:01.996 INFO:tasks.workunit.client.1.smithi169.stdout:3/404: write d7/d1a/d1c/d23/d70/f8e [521571,112933] 0 2024-09-16T00:26:01.997 INFO:tasks.workunit.client.0.smithi112.stdout:9/749: dwrite d1/d5/fa6 [0,4194304] 0 2024-09-16T00:26:01.997 INFO:tasks.workunit.client.1.smithi169.stdout:3/405: readlink d7/d1a/d1c/d35/d63/l51 0 2024-09-16T00:26:01.998 INFO:tasks.workunit.client.1.smithi169.stdout:3/406: readlink d7/d1a/d2b/l3b 0 2024-09-16T00:26:01.999 INFO:tasks.workunit.client.0.smithi112.stdout:8/510: dwrite d10/d12/d24/f2e [0,4194304] 0 2024-09-16T00:26:01.999 INFO:tasks.workunit.client.1.smithi169.stdout:3/407: chown d7/d1a/d1c 195 1 2024-09-16T00:26:02.002 INFO:tasks.workunit.client.0.smithi112.stdout:2/598: dwrite d2/dd/f15 [0,4194304] 0 2024-09-16T00:26:02.003 INFO:tasks.workunit.client.0.smithi112.stdout:1/599: unlink d0/d14/df/f4b 0 2024-09-16T00:26:02.005 INFO:tasks.workunit.client.1.smithi169.stdout:3/408: rename d7/fd to d7/d1a/d1c/d35/d63/d37/f9a 0 2024-09-16T00:26:02.010 INFO:tasks.workunit.client.0.smithi112.stdout:2/599: dread - d2/dd/d48/d22/d2b/d54/f70 zero size 2024-09-16T00:26:02.012 INFO:tasks.workunit.client.1.smithi169.stdout:3/409: dread d7/d50/f87 [0,4194304] 0 2024-09-16T00:26:02.013 INFO:tasks.workunit.client.0.smithi112.stdout:0/632: write d5/da/d66/f5d [3116040,8813] 0 2024-09-16T00:26:02.015 INFO:tasks.workunit.client.0.smithi112.stdout:2/600: mkdir d2/dd/d48/d8a/db9 0 2024-09-16T00:26:02.015 INFO:tasks.workunit.client.1.smithi169.stdout:3/410: mkdir d7/d1a/d1c/d23/d88/d9b 0 2024-09-16T00:26:02.016 INFO:tasks.workunit.client.1.smithi169.stdout:6/428: dread d2/d3f/d41/d52/f1b [0,4194304] 0 2024-09-16T00:26:02.016 INFO:tasks.workunit.client.0.smithi112.stdout:0/633: mknod d5/da/d18/d63/cc6 0 2024-09-16T00:26:02.017 INFO:tasks.workunit.client.0.smithi112.stdout:2/601: write d2/dd/d96/fab [1086205,41639] 0 2024-09-16T00:26:02.019 INFO:tasks.workunit.client.1.smithi169.stdout:6/429: rmdir d2/d4d/d59/d43/d67/d87 39 2024-09-16T00:26:02.021 INFO:tasks.workunit.client.0.smithi112.stdout:9/750: dwrite d1/d5/d6/d18/f19 [4194304,4194304] 0 2024-09-16T00:26:02.022 INFO:tasks.workunit.client.1.smithi169.stdout:3/411: creat d7/d1a/d1c/d35/d63/f9c x:0 0 0 2024-09-16T00:26:02.022 INFO:tasks.workunit.client.0.smithi112.stdout:2/602: rename d2/f19 to d2/dd/d48/d94/d49/fba 0 2024-09-16T00:26:02.023 INFO:tasks.workunit.client.0.smithi112.stdout:9/751: chown d1/d5/d6/d59/d64/fa1 28 1 2024-09-16T00:26:02.024 INFO:tasks.workunit.client.1.smithi169.stdout:6/430: mknod d2/d4d/d59/d43/d67/d87/c8b 0 2024-09-16T00:26:02.025 INFO:tasks.workunit.client.0.smithi112.stdout:2/603: mknod d2/dd/d48/d94/d49/cbb 0 2024-09-16T00:26:02.032 INFO:tasks.workunit.client.0.smithi112.stdout:2/604: dwrite d2/dd/f15 [0,4194304] 0 2024-09-16T00:26:02.033 INFO:tasks.workunit.client.1.smithi169.stdout:3/412: dread d7/d1a/d1c/d23/d70/f8e [0,4194304] 0 2024-09-16T00:26:02.034 INFO:tasks.workunit.client.0.smithi112.stdout:2/605: readlink d2/dd/d48/l4b 0 2024-09-16T00:26:02.035 INFO:tasks.workunit.client.1.smithi169.stdout:3/413: stat d7/d1a/d1c/d35/d63/c82 0 2024-09-16T00:26:02.035 INFO:tasks.workunit.client.1.smithi169.stdout:3/414: read - d7/d1a/d1c/d6c/f7a zero size 2024-09-16T00:26:02.036 INFO:tasks.workunit.client.0.smithi112.stdout:7/592: dread d6/d10/d16/d49/d6c/f36 [0,4194304] 0 2024-09-16T00:26:02.037 INFO:tasks.workunit.client.1.smithi169.stdout:3/415: symlink d7/d1a/d2b/l9d 0 2024-09-16T00:26:02.038 INFO:tasks.workunit.client.1.smithi169.stdout:6/431: write d2/d3f/d41/d52/f18 [820043,11960] 0 2024-09-16T00:26:02.038 INFO:tasks.workunit.client.1.smithi169.stdout:2/269: sync 2024-09-16T00:26:02.038 INFO:tasks.workunit.client.1.smithi169.stdout:3/416: stat d7/d1a/d2b/f8c 0 2024-09-16T00:26:02.039 INFO:tasks.workunit.client.0.smithi112.stdout:1/600: dread d0/d14/df/d28/f2e [0,4194304] 0 2024-09-16T00:26:02.040 INFO:tasks.workunit.client.1.smithi169.stdout:3/417: fsync d7/d1a/f32 0 2024-09-16T00:26:02.041 INFO:tasks.workunit.client.1.smithi169.stdout:2/270: creat d7/db/d27/d4f/d56/f57 x:0 0 0 2024-09-16T00:26:02.042 INFO:tasks.workunit.client.1.smithi169.stdout:2/271: readlink d7/l16 0 2024-09-16T00:26:02.042 INFO:tasks.workunit.client.1.smithi169.stdout:6/432: read d2/fb [5318773,64980] 0 2024-09-16T00:26:02.043 INFO:tasks.workunit.client.1.smithi169.stdout:3/418: rmdir d7/d1a/d1c/d35/d63 39 2024-09-16T00:26:02.044 INFO:tasks.workunit.client.0.smithi112.stdout:1/601: dread d0/d14/d1c/d26/f52 [0,4194304] 0 2024-09-16T00:26:02.050 INFO:tasks.workunit.client.1.smithi169.stdout:2/272: getdents d7/db/d20 0 2024-09-16T00:26:02.051 INFO:tasks.workunit.client.0.smithi112.stdout:1/602: dwrite d0/d14/df/d28/d45/d46/d58/d62/f74 [0,4194304] 0 2024-09-16T00:26:02.052 INFO:tasks.workunit.client.1.smithi169.stdout:2/273: chown d7/db/c32 35 1 2024-09-16T00:26:02.052 INFO:tasks.workunit.client.1.smithi169.stdout:6/433: dwrite d2/d4d/f53 [0,4194304] 0 2024-09-16T00:26:02.054 INFO:tasks.workunit.client.0.smithi112.stdout:1/603: mknod d0/d14/d1c/d20/daf/cc5 0 2024-09-16T00:26:02.054 INFO:tasks.workunit.client.1.smithi169.stdout:3/419: rename d7/d1a/d2b/l8b to d7/d50/d6f/l9e 0 2024-09-16T00:26:02.055 INFO:tasks.workunit.client.1.smithi169.stdout:6/434: write d2/d4d/d59/d43/f56 [105684,115933] 0 2024-09-16T00:26:02.057 INFO:tasks.workunit.client.1.smithi169.stdout:2/274: mknod d7/db/d20/d4c/c58 0 2024-09-16T00:26:02.060 INFO:tasks.workunit.client.1.smithi169.stdout:3/420: mknod d7/d50/d6f/c9f 0 2024-09-16T00:26:02.061 INFO:tasks.workunit.client.1.smithi169.stdout:2/275: symlink d7/db/d27/d29/l59 0 2024-09-16T00:26:02.062 INFO:tasks.workunit.client.1.smithi169.stdout:6/435: symlink d2/d4d/d59/d43/l8c 0 2024-09-16T00:26:02.064 INFO:tasks.workunit.client.1.smithi169.stdout:6/436: mknod d2/d4c/c8d 0 2024-09-16T00:26:02.065 INFO:tasks.workunit.client.1.smithi169.stdout:3/421: symlink d7/d1a/d1c/d35/d63/la0 0 2024-09-16T00:26:02.066 INFO:tasks.workunit.client.1.smithi169.stdout:2/276: dread d7/db/f13 [4194304,4194304] 0 2024-09-16T00:26:02.070 INFO:tasks.workunit.client.0.smithi112.stdout:2/606: dread d2/dd/f12 [4194304,4194304] 0 2024-09-16T00:26:02.070 INFO:tasks.workunit.client.1.smithi169.stdout:2/277: symlink d7/db/d27/d29/d37/l5a 0 2024-09-16T00:26:02.071 INFO:tasks.workunit.client.0.smithi112.stdout:2/607: write d2/f8d [7883947,110762] 0 2024-09-16T00:26:02.071 INFO:tasks.workunit.client.1.smithi169.stdout:6/437: getdents d2/d4d/d59/d43/d67/d87 0 2024-09-16T00:26:02.071 INFO:tasks.workunit.client.0.smithi112.stdout:2/608: chown d2/f46 932 1 2024-09-16T00:26:02.073 INFO:tasks.workunit.client.1.smithi169.stdout:2/278: mkdir d7/db/d27/d29/d37/d5b 0 2024-09-16T00:26:02.074 INFO:tasks.workunit.client.0.smithi112.stdout:2/609: creat d2/dd/d48/d22/d2b/d54/d60/daf/fbc x:0 0 0 2024-09-16T00:26:02.074 INFO:tasks.workunit.client.1.smithi169.stdout:6/438: unlink d2/d4d/d59/c40 0 2024-09-16T00:26:02.074 INFO:tasks.workunit.client.0.smithi112.stdout:2/610: read - d2/dd/d48/d22/d2b/d30/f6f zero size 2024-09-16T00:26:02.079 INFO:tasks.workunit.client.0.smithi112.stdout:2/611: write d2/dd/f87 [959265,66000] 0 2024-09-16T00:26:02.083 INFO:tasks.workunit.client.1.smithi169.stdout:6/439: dwrite d2/d4d/f7d [0,4194304] 0 2024-09-16T00:26:02.084 INFO:tasks.workunit.client.1.smithi169.stdout:6/440: dread - d2/d3f/f85 zero size 2024-09-16T00:26:02.085 INFO:tasks.workunit.client.0.smithi112.stdout:2/612: dwrite d2/dd/d48/d22/d2b/d54/d60/daf/fbc [0,4194304] 0 2024-09-16T00:26:02.086 INFO:tasks.workunit.client.0.smithi112.stdout:2/613: stat d2/dd/d48/d22 0 2024-09-16T00:26:02.086 INFO:tasks.workunit.client.0.smithi112.stdout:1/604: dread d0/f87 [0,4194304] 0 2024-09-16T00:26:02.087 INFO:tasks.workunit.client.1.smithi169.stdout:2/279: truncate f0 2052144 0 2024-09-16T00:26:02.088 INFO:tasks.workunit.client.0.smithi112.stdout:2/614: chown d2/dd/d48/c18 1312493 1 2024-09-16T00:26:02.089 INFO:tasks.workunit.client.0.smithi112.stdout:1/605: symlink d0/lc6 0 2024-09-16T00:26:02.091 INFO:tasks.workunit.client.1.smithi169.stdout:6/441: creat d2/d4d/f8e x:0 0 0 2024-09-16T00:26:02.092 INFO:tasks.workunit.client.1.smithi169.stdout:3/422: dwrite d7/d1a/d1c/d35/d63/f53 [0,4194304] 0 2024-09-16T00:26:02.092 INFO:tasks.workunit.client.0.smithi112.stdout:2/615: rename d2/dd/d48/d22/d67/f77 to d2/dd/d48/d8a/d9b/fbd 0 2024-09-16T00:26:02.095 INFO:tasks.workunit.client.0.smithi112.stdout:2/616: creat d2/dd/d48/d94/fbe x:0 0 0 2024-09-16T00:26:02.097 INFO:tasks.workunit.client.0.smithi112.stdout:2/617: creat d2/fbf x:0 0 0 2024-09-16T00:26:02.098 INFO:tasks.workunit.client.0.smithi112.stdout:2/618: truncate d2/f46 3162686 0 2024-09-16T00:26:02.101 INFO:tasks.workunit.client.0.smithi112.stdout:2/619: creat d2/dd/d48/d22/d2b/d54/d60/d6c/d88/da4/fc0 x:0 0 0 2024-09-16T00:26:02.102 INFO:tasks.workunit.client.0.smithi112.stdout:2/620: rmdir d2/dd/d96 39 2024-09-16T00:26:02.103 INFO:tasks.workunit.client.1.smithi169.stdout:2/280: dwrite d7/db/f17 [0,4194304] 0 2024-09-16T00:26:02.105 INFO:tasks.workunit.client.1.smithi169.stdout:6/442: dread d2/d22/f34 [0,4194304] 0 2024-09-16T00:26:02.105 INFO:tasks.workunit.client.1.smithi169.stdout:2/281: readlink d7/db/d27/l42 0 2024-09-16T00:26:02.105 INFO:tasks.workunit.client.0.smithi112.stdout:2/621: dread d2/dd/d48/d94/d49/f5c [0,4194304] 0 2024-09-16T00:26:02.108 INFO:tasks.workunit.client.1.smithi169.stdout:6/443: rmdir d2/d4d/d59/d43/d67/d87 39 2024-09-16T00:26:02.110 INFO:tasks.workunit.client.0.smithi112.stdout:2/622: link d2/dd/d48/d22/d2b/f31 d2/dd/d48/d22/d2b/d54/d60/daf/fc1 0 2024-09-16T00:26:02.112 INFO:tasks.workunit.client.1.smithi169.stdout:6/444: mkdir d2/d36/d48/d8f 0 2024-09-16T00:26:02.115 INFO:tasks.workunit.client.1.smithi169.stdout:6/445: chown d2/d22/f34 35414951 1 2024-09-16T00:26:02.117 INFO:tasks.workunit.client.0.smithi112.stdout:2/623: dwrite d2/dd/d96/fab [0,4194304] 0 2024-09-16T00:26:02.117 INFO:tasks.workunit.client.1.smithi169.stdout:2/282: dwrite d7/db/d20/f3b [0,4194304] 0 2024-09-16T00:26:02.120 INFO:tasks.workunit.client.1.smithi169.stdout:6/446: creat d2/d36/d48/f90 x:0 0 0 2024-09-16T00:26:02.120 INFO:tasks.workunit.client.1.smithi169.stdout:2/283: stat d7/db/d27/d29/d37/d5b 0 2024-09-16T00:26:02.121 INFO:tasks.workunit.client.1.smithi169.stdout:3/423: dwrite d7/d1a/d1c/d23/f4c [4194304,4194304] 0 2024-09-16T00:26:02.122 INFO:tasks.workunit.client.0.smithi112.stdout:2/624: link d2/dd/d48/l99 d2/dd/d48/d22/d2b/d54/d60/d6c/d88/da4/lc2 0 2024-09-16T00:26:02.124 INFO:tasks.workunit.client.1.smithi169.stdout:3/424: dread - d7/f8f zero size 2024-09-16T00:26:02.124 INFO:tasks.workunit.client.1.smithi169.stdout:6/447: truncate d2/d4d/d59/d38/f5a 897190 0 2024-09-16T00:26:02.135 INFO:tasks.workunit.client.1.smithi169.stdout:2/284: dwrite d7/db/d27/d4f/d56/f57 [0,4194304] 0 2024-09-16T00:26:02.138 INFO:tasks.workunit.client.1.smithi169.stdout:2/285: write d7/db/f19 [4565418,19187] 0 2024-09-16T00:26:02.139 INFO:tasks.workunit.client.1.smithi169.stdout:6/448: dread d2/d4d/d59/d43/f4a [0,4194304] 0 2024-09-16T00:26:02.140 INFO:tasks.workunit.client.0.smithi112.stdout:2/625: truncate d2/dd/d48/d22/f29 1988098 0 2024-09-16T00:26:02.141 INFO:tasks.workunit.client.1.smithi169.stdout:6/449: readlink d2/d3f/d41/d52/l76 0 2024-09-16T00:26:02.142 INFO:tasks.workunit.client.1.smithi169.stdout:6/450: fsync d2/d4d/d59/d43/f56 0 2024-09-16T00:26:02.144 INFO:tasks.workunit.client.1.smithi169.stdout:2/286: rename d7/db/d27/d29/d37/d5b to d7/db/d20/d5c 0 2024-09-16T00:26:02.145 INFO:tasks.workunit.client.1.smithi169.stdout:6/451: mkdir d2/d3f/d41/d5c/d91 0 2024-09-16T00:26:02.149 INFO:tasks.workunit.client.1.smithi169.stdout:2/287: dread d7/db/f19 [0,4194304] 0 2024-09-16T00:26:02.151 INFO:tasks.workunit.client.1.smithi169.stdout:6/452: link d2/l16 d2/d4d/d59/d74/l92 0 2024-09-16T00:26:02.151 INFO:tasks.workunit.client.1.smithi169.stdout:2/288: symlink d7/d1c/l5d 0 2024-09-16T00:26:02.152 INFO:tasks.workunit.client.1.smithi169.stdout:2/289: fdatasync d7/db/d27/d4f/f53 0 2024-09-16T00:26:02.154 INFO:tasks.workunit.client.1.smithi169.stdout:6/453: rmdir d2/d36/d48/d8f 0 2024-09-16T00:26:02.155 INFO:tasks.workunit.client.1.smithi169.stdout:2/290: unlink d7/db/d20/d21/f22 0 2024-09-16T00:26:02.159 INFO:tasks.workunit.client.1.smithi169.stdout:6/454: rename d2/d3f/d41/d52/c37 to d2/d3f/d41/c93 0 2024-09-16T00:26:02.159 INFO:tasks.workunit.client.1.smithi169.stdout:5/349: sync 2024-09-16T00:26:02.160 INFO:tasks.workunit.client.1.smithi169.stdout:5/350: chown d2/da/d10/d13/d1c/d40/d43 53 1 2024-09-16T00:26:02.161 INFO:tasks.workunit.client.1.smithi169.stdout:6/455: creat d2/d3f/d41/d5c/f94 x:0 0 0 2024-09-16T00:26:02.163 INFO:tasks.workunit.client.1.smithi169.stdout:5/351: dread - d2/da/d10/d13/d1c/d2d/f5f zero size 2024-09-16T00:26:02.167 INFO:tasks.workunit.client.1.smithi169.stdout:5/352: creat d2/da/d10/f7a x:0 0 0 2024-09-16T00:26:02.167 INFO:tasks.workunit.client.0.smithi112.stdout:2/626: dwrite d2/dd/d48/d22/d2b/d54/d79/f9e [0,4194304] 0 2024-09-16T00:26:02.168 INFO:tasks.workunit.client.1.smithi169.stdout:6/456: dread d2/d36/f31 [4194304,4194304] 0 2024-09-16T00:26:02.171 INFO:tasks.workunit.client.0.smithi112.stdout:2/627: creat d2/dd/d48/d22/d67/d9a/fc3 x:0 0 0 2024-09-16T00:26:02.174 INFO:tasks.workunit.client.0.smithi112.stdout:2/628: rmdir d2/dd/d48/d22/d2b/d54/d60/d6c/d88/da4 39 2024-09-16T00:26:02.174 INFO:tasks.workunit.client.1.smithi169.stdout:3/425: dread d7/d1a/d1c/d35/d63/d37/f59 [0,4194304] 0 2024-09-16T00:26:02.174 INFO:tasks.workunit.client.1.smithi169.stdout:5/353: truncate d2/da/d10/f11 1381 0 2024-09-16T00:26:02.176 INFO:tasks.workunit.client.1.smithi169.stdout:2/291: rmdir d7/db/d20/d21 39 2024-09-16T00:26:02.177 INFO:tasks.workunit.client.0.smithi112.stdout:2/629: rename d2/fb6 to d2/dd/fc4 0 2024-09-16T00:26:02.177 INFO:tasks.workunit.client.1.smithi169.stdout:6/457: dwrite d2/d36/f70 [0,4194304] 0 2024-09-16T00:26:02.179 INFO:tasks.workunit.client.0.smithi112.stdout:2/630: creat d2/dd/d48/d22/d2b/d54/d60/fc5 x:0 0 0 2024-09-16T00:26:02.180 INFO:tasks.workunit.client.1.smithi169.stdout:6/458: readlink d2/d4d/d59/l82 0 2024-09-16T00:26:02.183 INFO:tasks.workunit.client.0.smithi112.stdout:2/631: rename d2/dd/d48/d94/d49/l4c to d2/lc6 0 2024-09-16T00:26:02.185 INFO:tasks.workunit.client.1.smithi169.stdout:5/354: dwrite d2/f8 [0,4194304] 0 2024-09-16T00:26:02.187 INFO:tasks.workunit.client.0.smithi112.stdout:2/632: getdents d2/dd/d48/d22 0 2024-09-16T00:26:02.187 INFO:tasks.workunit.client.1.smithi169.stdout:5/355: chown d2/da/d10/d13/d1c/d40/d54 62529 1 2024-09-16T00:26:02.191 INFO:tasks.workunit.client.0.smithi112.stdout:5/459: sync 2024-09-16T00:26:02.192 INFO:tasks.workunit.client.1.smithi169.stdout:2/292: getdents d7/db/d20/d21 0 2024-09-16T00:26:02.193 INFO:tasks.workunit.client.1.smithi169.stdout:0/383: sync 2024-09-16T00:26:02.193 INFO:tasks.workunit.client.1.smithi169.stdout:8/333: sync 2024-09-16T00:26:02.193 INFO:tasks.workunit.client.1.smithi169.stdout:4/399: sync 2024-09-16T00:26:02.193 INFO:tasks.workunit.client.1.smithi169.stdout:7/460: sync 2024-09-16T00:26:02.193 INFO:tasks.workunit.client.1.smithi169.stdout:9/415: sync 2024-09-16T00:26:02.193 INFO:tasks.workunit.client.1.smithi169.stdout:1/397: sync 2024-09-16T00:26:02.194 INFO:tasks.workunit.client.0.smithi112.stdout:5/460: chown d4/d7/d26/d4c/f97 434928 1 2024-09-16T00:26:02.194 INFO:tasks.workunit.client.1.smithi169.stdout:9/416: dread - d5/d12/f5d zero size 2024-09-16T00:26:02.195 INFO:tasks.workunit.client.1.smithi169.stdout:7/461: write d7/d5b/f5c [3126146,104796] 0 2024-09-16T00:26:02.201 INFO:tasks.workunit.client.1.smithi169.stdout:6/459: dwrite d2/d3f/d41/d52/f1e [0,4194304] 0 2024-09-16T00:26:02.203 INFO:tasks.workunit.client.1.smithi169.stdout:4/400: rmdir d2/d12/d13 39 2024-09-16T00:26:02.204 INFO:tasks.workunit.client.1.smithi169.stdout:2/293: symlink d7/db/d20/d5c/l5e 0 2024-09-16T00:26:02.207 INFO:tasks.workunit.client.1.smithi169.stdout:3/426: dwrite d7/f24 [0,4194304] 0 2024-09-16T00:26:02.208 INFO:tasks.workunit.client.1.smithi169.stdout:2/294: chown d7/db/f19 617912198 1 2024-09-16T00:26:02.210 INFO:tasks.workunit.client.1.smithi169.stdout:9/417: rmdir d5/d12 39 2024-09-16T00:26:02.212 INFO:tasks.workunit.client.1.smithi169.stdout:7/462: readlink d7/df/d95/l59 0 2024-09-16T00:26:02.213 INFO:tasks.workunit.client.1.smithi169.stdout:6/460: rmdir d2/d36/d48 39 2024-09-16T00:26:02.214 INFO:tasks.workunit.client.0.smithi112.stdout:5/461: truncate d4/d7/d26/f44 929388 0 2024-09-16T00:26:02.214 INFO:tasks.workunit.client.1.smithi169.stdout:7/463: readlink d7/l4a 0 2024-09-16T00:26:02.215 INFO:tasks.workunit.client.1.smithi169.stdout:9/418: dread - d5/d49/d64/f65 zero size 2024-09-16T00:26:02.216 INFO:tasks.workunit.client.1.smithi169.stdout:2/295: unlink d7/db/d27/d4f/l50 0 2024-09-16T00:26:02.216 INFO:tasks.workunit.client.1.smithi169.stdout:0/384: dwrite d0/d2/f45 [0,4194304] 0 2024-09-16T00:26:02.216 INFO:tasks.workunit.client.1.smithi169.stdout:1/398: dwrite dc/d2c/d3d/f47 [0,4194304] 0 2024-09-16T00:26:02.217 INFO:tasks.workunit.client.0.smithi112.stdout:5/462: stat d4/d7/d26/d4c/c85 0 2024-09-16T00:26:02.217 INFO:tasks.workunit.client.0.smithi112.stdout:2/633: dread d2/f14 [0,4194304] 0 2024-09-16T00:26:02.217 INFO:tasks.workunit.client.0.smithi112.stdout:5/463: chown d4/dd/d49/d70/d98 2130461 1 2024-09-16T00:26:02.217 INFO:tasks.workunit.client.1.smithi169.stdout:5/356: write d2/da/d10/d13/d1c/f3b [1075587,73441] 0 2024-09-16T00:26:02.219 INFO:tasks.workunit.client.1.smithi169.stdout:2/296: fdatasync d7/db/f23 0 2024-09-16T00:26:02.221 INFO:tasks.workunit.client.0.smithi112.stdout:5/464: creat d4/d4f/d78/fa0 x:0 0 0 2024-09-16T00:26:02.221 INFO:tasks.workunit.client.0.smithi112.stdout:5/465: readlink d4/d7/l25 0 2024-09-16T00:26:02.222 INFO:tasks.workunit.client.1.smithi169.stdout:3/427: readlink d7/d1a/d1c/d23/l4e 0 2024-09-16T00:26:02.222 INFO:tasks.workunit.client.1.smithi169.stdout:4/401: symlink d2/d12/d13/d4e/d50/l7f 0 2024-09-16T00:26:02.222 INFO:tasks.workunit.client.0.smithi112.stdout:5/466: chown d4/d7/d26/d4c/f6a 19960 1 2024-09-16T00:26:02.224 INFO:tasks.workunit.client.1.smithi169.stdout:6/461: mkdir d2/d3f/d95 0 2024-09-16T00:26:02.224 INFO:tasks.workunit.client.1.smithi169.stdout:5/357: chown d2/da/d10/d13/d1c/d40/c48 4464950 1 2024-09-16T00:26:02.226 INFO:tasks.workunit.client.1.smithi169.stdout:1/399: creat dc/d36/d3a/f76 x:0 0 0 2024-09-16T00:26:02.226 INFO:tasks.workunit.client.1.smithi169.stdout:9/419: dread - d5/da/d4e/f87 zero size 2024-09-16T00:26:02.226 INFO:tasks.workunit.client.1.smithi169.stdout:6/462: stat d2/d3f/d41/f80 0 2024-09-16T00:26:02.227 INFO:tasks.workunit.client.1.smithi169.stdout:7/464: write d7/d16/d49/f68 [1206189,120881] 0 2024-09-16T00:26:02.229 INFO:tasks.workunit.client.1.smithi169.stdout:2/297: rmdir d7/db/d27/d29 39 2024-09-16T00:26:02.230 INFO:tasks.workunit.client.0.smithi112.stdout:5/467: dwrite d4/dd/d6f/f72 [4194304,4194304] 0 2024-09-16T00:26:02.233 INFO:tasks.workunit.client.1.smithi169.stdout:1/400: read - dc/d36/d3a/d45/f61 zero size 2024-09-16T00:26:02.233 INFO:tasks.workunit.client.1.smithi169.stdout:3/428: unlink d7/d50/d6f/l80 0 2024-09-16T00:26:02.234 INFO:tasks.workunit.client.1.smithi169.stdout:5/358: unlink d2/da/d10/d13/d24/f3a 0 2024-09-16T00:26:02.236 INFO:tasks.workunit.client.0.smithi112.stdout:2/634: truncate d2/f8d 1960438 0 2024-09-16T00:26:02.237 INFO:tasks.workunit.client.1.smithi169.stdout:7/465: mkdir d7/d5b/d61/d98 0 2024-09-16T00:26:02.241 INFO:tasks.workunit.client.0.smithi112.stdout:2/635: rmdir d2/dd/d48/d22/d2b/d54/d60/d6c/d88/da4 39 2024-09-16T00:26:02.242 INFO:tasks.workunit.client.0.smithi112.stdout:2/636: chown d2/dd/d48/c4a 903 1 2024-09-16T00:26:02.242 INFO:tasks.workunit.client.1.smithi169.stdout:3/429: symlink d7/d1a/d2b/d8d/la1 0 2024-09-16T00:26:02.243 INFO:tasks.workunit.client.0.smithi112.stdout:2/637: readlink d2/dd/d48/d22/d2b/d54/d60/d6c/d88/l89 0 2024-09-16T00:26:02.244 INFO:tasks.workunit.client.1.smithi169.stdout:0/385: dwrite d0/d2c/d38/f5e [4194304,4194304] 0 2024-09-16T00:26:02.245 INFO:tasks.workunit.client.1.smithi169.stdout:7/466: mkdir d7/d5b/d61/d98/d99 0 2024-09-16T00:26:02.246 INFO:tasks.workunit.client.0.smithi112.stdout:2/638: creat d2/dd/d48/d22/d2b/d54/d60/fc7 x:0 0 0 2024-09-16T00:26:02.248 INFO:tasks.workunit.client.0.smithi112.stdout:2/639: mknod d2/dd/d48/d94/d49/cc8 0 2024-09-16T00:26:02.249 INFO:tasks.workunit.client.1.smithi169.stdout:2/298: dwrite d7/db/f13 [4194304,4194304] 0 2024-09-16T00:26:02.249 INFO:tasks.workunit.client.0.smithi112.stdout:2/640: chown d2/dd/d48/l99 449 1 2024-09-16T00:26:02.250 INFO:tasks.workunit.client.1.smithi169.stdout:6/463: dread d2/d22/f23 [0,4194304] 0 2024-09-16T00:26:02.251 INFO:tasks.workunit.client.1.smithi169.stdout:7/467: rename d7/d16/f40 to d7/d5b/f9a 0 2024-09-16T00:26:02.257 INFO:tasks.workunit.client.1.smithi169.stdout:3/430: dwrite d7/d50/f79 [0,4194304] 0 2024-09-16T00:26:02.263 INFO:tasks.workunit.client.0.smithi112.stdout:5/468: dread d4/dd/d12/d52/f63 [0,4194304] 0 2024-09-16T00:26:02.264 INFO:tasks.workunit.client.1.smithi169.stdout:2/299: creat d7/db/d20/d4c/f5f x:0 0 0 2024-09-16T00:26:02.265 INFO:tasks.workunit.client.0.smithi112.stdout:5/469: write d4/d1c/d43/f6d [470768,54520] 0 2024-09-16T00:26:02.266 INFO:tasks.workunit.client.1.smithi169.stdout:6/464: symlink d2/d3f/d95/l96 0 2024-09-16T00:26:02.271 INFO:tasks.workunit.client.1.smithi169.stdout:0/386: getdents d0/d2c/d54/d21/d31/d8c 0 2024-09-16T00:26:02.272 INFO:tasks.workunit.client.1.smithi169.stdout:6/465: stat d2/cd 0 2024-09-16T00:26:02.273 INFO:tasks.workunit.client.1.smithi169.stdout:8/334: dread d7/dc/d2a/f42 [0,4194304] 0 2024-09-16T00:26:02.273 INFO:tasks.workunit.client.1.smithi169.stdout:0/387: chown d0/d2/c3c 109 1 2024-09-16T00:26:02.274 INFO:tasks.workunit.client.1.smithi169.stdout:3/431: creat d7/d50/fa2 x:0 0 0 2024-09-16T00:26:02.275 INFO:tasks.workunit.client.1.smithi169.stdout:8/335: fdatasync d7/dc/d26/d46/f5d 0 2024-09-16T00:26:02.276 INFO:tasks.workunit.client.1.smithi169.stdout:5/359: truncate d2/da/d10/d13/f1d 2472527 0 2024-09-16T00:26:02.278 INFO:tasks.workunit.client.1.smithi169.stdout:0/388: mknod d0/d2/d8/c92 0 2024-09-16T00:26:02.281 INFO:tasks.workunit.client.1.smithi169.stdout:3/432: mkdir d7/d1a/d1c/d35/d63/da3 0 2024-09-16T00:26:02.281 INFO:tasks.workunit.client.1.smithi169.stdout:4/402: dread d2/d12/d13/d4e/f76 [0,4194304] 0 2024-09-16T00:26:02.282 INFO:tasks.workunit.client.1.smithi169.stdout:6/466: dread d2/f45 [0,4194304] 0 2024-09-16T00:26:02.282 INFO:tasks.workunit.client.0.smithi112.stdout:2/641: write d2/dd/f12 [4030593,107191] 0 2024-09-16T00:26:02.283 INFO:tasks.workunit.client.0.smithi112.stdout:2/642: fdatasync d2/f8b 0 2024-09-16T00:26:02.285 INFO:tasks.workunit.client.1.smithi169.stdout:8/336: mkdir d7/dc/d17/d6a 0 2024-09-16T00:26:02.288 INFO:tasks.workunit.client.1.smithi169.stdout:4/403: readlink d2/d12/d13/d1f/d3e/d5b/d1a/l5e 0 2024-09-16T00:26:02.289 INFO:tasks.workunit.client.1.smithi169.stdout:8/337: chown d7/dc/d26/c5b 108 1 2024-09-16T00:26:02.291 INFO:tasks.workunit.client.1.smithi169.stdout:5/360: mknod d2/da/d10/d13/d1c/d2d/c7b 0 2024-09-16T00:26:02.292 INFO:tasks.workunit.client.1.smithi169.stdout:3/433: mknod d7/d50/ca4 0 2024-09-16T00:26:02.292 INFO:tasks.workunit.client.1.smithi169.stdout:1/401: dread dc/fd [0,4194304] 0 2024-09-16T00:26:02.293 INFO:tasks.workunit.client.1.smithi169.stdout:4/404: chown d2/d12/d13/d27/l63 977 1 2024-09-16T00:26:02.295 INFO:tasks.workunit.client.1.smithi169.stdout:3/434: write d7/d1a/d1c/f78 [6012999,75189] 0 2024-09-16T00:26:02.297 INFO:tasks.workunit.client.1.smithi169.stdout:0/389: dwrite d0/d2c/d54/d21/f3f [0,4194304] 0 2024-09-16T00:26:02.298 INFO:tasks.workunit.client.0.smithi112.stdout:2/643: write d2/f8d [2268963,130795] 0 2024-09-16T00:26:02.299 INFO:tasks.workunit.client.0.smithi112.stdout:2/644: write d2/dd/d48/d22/d67/d9a/fc3 [10972,125139] 0 2024-09-16T00:26:02.300 INFO:tasks.workunit.client.1.smithi169.stdout:6/467: creat d2/d4d/d59/d38/f97 x:0 0 0 2024-09-16T00:26:02.301 INFO:tasks.workunit.client.1.smithi169.stdout:0/390: chown d0/d2/d8/f73 5499014 1 2024-09-16T00:26:02.301 INFO:tasks.workunit.client.0.smithi112.stdout:2/645: mkdir d2/dd/d48/d22/d2b/d54/d60/d6c/dc9 0 2024-09-16T00:26:02.302 INFO:tasks.workunit.client.0.smithi112.stdout:2/646: write d2/dd/d48/f85 [316426,46266] 0 2024-09-16T00:26:02.307 INFO:tasks.workunit.client.1.smithi169.stdout:4/405: symlink d2/d12/d13/d4d/d59/l80 0 2024-09-16T00:26:02.309 INFO:tasks.workunit.client.1.smithi169.stdout:8/338: read d7/dc/d17/f28 [2589237,47294] 0 2024-09-16T00:26:02.310 INFO:tasks.workunit.client.1.smithi169.stdout:5/361: mkdir d2/d7c 0 2024-09-16T00:26:02.311 INFO:tasks.workunit.client.1.smithi169.stdout:0/391: creat d0/d2/f93 x:0 0 0 2024-09-16T00:26:02.312 INFO:tasks.workunit.client.1.smithi169.stdout:1/402: dwrite dc/d36/d3a/d45/d29/d4d/f54 [0,4194304] 0 2024-09-16T00:26:02.313 INFO:tasks.workunit.client.1.smithi169.stdout:5/362: readlink d2/da/l75 0 2024-09-16T00:26:02.315 INFO:tasks.workunit.client.1.smithi169.stdout:6/468: dwrite d2/d3f/d41/d72/f84 [0,4194304] 0 2024-09-16T00:26:02.316 INFO:tasks.workunit.client.1.smithi169.stdout:0/392: dread - d0/d2c/d54/d21/f8e zero size 2024-09-16T00:26:02.318 INFO:tasks.workunit.client.1.smithi169.stdout:6/469: fsync d2/d3f/d41/f80 0 2024-09-16T00:26:02.318 INFO:tasks.workunit.client.1.smithi169.stdout:3/435: creat d7/d1a/d1c/d35/fa5 x:0 0 0 2024-09-16T00:26:02.318 INFO:tasks.workunit.client.1.smithi169.stdout:1/403: mknod dc/c77 0 2024-09-16T00:26:02.321 INFO:tasks.workunit.client.1.smithi169.stdout:6/470: fsync d2/d3f/d41/f80 0 2024-09-16T00:26:02.324 INFO:tasks.workunit.client.1.smithi169.stdout:4/406: symlink d2/d12/d13/d1f/d3e/d5b/d70/d7d/l81 0 2024-09-16T00:26:02.325 INFO:tasks.workunit.client.1.smithi169.stdout:8/339: dwrite d7/dc/d17/d24/f4b [0,4194304] 0 2024-09-16T00:26:02.326 INFO:tasks.workunit.client.1.smithi169.stdout:5/363: mkdir d2/da/d10/d13/d1c/d40/d54/d6d/d7d 0 2024-09-16T00:26:02.329 INFO:tasks.workunit.client.1.smithi169.stdout:3/436: mkdir d7/d50/d90/da6 0 2024-09-16T00:26:02.333 INFO:tasks.workunit.client.1.smithi169.stdout:6/471: rmdir d2/d22 39 2024-09-16T00:26:02.334 INFO:tasks.workunit.client.1.smithi169.stdout:5/364: mkdir d2/da/d10/d13/d1c/d32/d53/d7e 0 2024-09-16T00:26:02.335 INFO:tasks.workunit.client.1.smithi169.stdout:3/437: fsync d7/d1a/d1c/d35/d63/d37/f9a 0 2024-09-16T00:26:02.336 INFO:tasks.workunit.client.1.smithi169.stdout:8/340: creat d7/dc/d26/d46/f6b x:0 0 0 2024-09-16T00:26:02.337 INFO:tasks.workunit.client.1.smithi169.stdout:4/407: dread d2/d12/d13/d1f/f35 [0,4194304] 0 2024-09-16T00:26:02.338 INFO:tasks.workunit.client.1.smithi169.stdout:3/438: chown d7/d1a/d1c/d23/d70 53370436 1 2024-09-16T00:26:02.339 INFO:tasks.workunit.client.1.smithi169.stdout:1/404: getdents dc/d36/d39 0 2024-09-16T00:26:02.339 INFO:tasks.workunit.client.1.smithi169.stdout:0/393: creat d0/d2/d8/d5/d79/d7c/f94 x:0 0 0 2024-09-16T00:26:02.340 INFO:tasks.workunit.client.1.smithi169.stdout:5/365: truncate d2/da/d10/d13/d1c/f41 163443 0 2024-09-16T00:26:02.342 INFO:tasks.workunit.client.1.smithi169.stdout:6/472: getdents d2/d3f/d41/d52 0 2024-09-16T00:26:02.345 INFO:tasks.workunit.client.1.smithi169.stdout:6/473: read d2/f64 [124216,100619] 0 2024-09-16T00:26:02.349 INFO:tasks.workunit.client.1.smithi169.stdout:4/408: dread d2/d12/f2c [0,4194304] 0 2024-09-16T00:26:02.350 INFO:tasks.workunit.client.1.smithi169.stdout:5/366: mkdir d2/da/d10/d13/d1c/d40/d54/d7f 0 2024-09-16T00:26:02.351 INFO:tasks.workunit.client.1.smithi169.stdout:6/474: symlink d2/d3f/d95/l98 0 2024-09-16T00:26:02.351 INFO:tasks.workunit.client.1.smithi169.stdout:4/409: write d2/d12/d13/d1f/f48 [224869,34547] 0 2024-09-16T00:26:02.353 INFO:tasks.workunit.client.1.smithi169.stdout:8/341: dwrite d7/dc/d26/d46/f6b [0,4194304] 0 2024-09-16T00:26:02.354 INFO:tasks.workunit.client.0.smithi112.stdout:3/551: sync 2024-09-16T00:26:02.354 INFO:tasks.workunit.client.0.smithi112.stdout:4/525: sync 2024-09-16T00:26:02.354 INFO:tasks.workunit.client.0.smithi112.stdout:6/499: sync 2024-09-16T00:26:02.355 INFO:tasks.workunit.client.0.smithi112.stdout:6/500: chown dd/df/f3d 494271 1 2024-09-16T00:26:02.355 INFO:tasks.workunit.client.1.smithi169.stdout:1/405: getdents dc/d36/d4e 0 2024-09-16T00:26:02.356 INFO:tasks.workunit.client.0.smithi112.stdout:4/526: symlink d6/d39/d1c/lba 0 2024-09-16T00:26:02.357 INFO:tasks.workunit.client.1.smithi169.stdout:4/410: chown d2/d12/l44 423982 1 2024-09-16T00:26:02.358 INFO:tasks.workunit.client.1.smithi169.stdout:6/475: mknod d2/d4d/d59/d74/c99 0 2024-09-16T00:26:02.359 INFO:tasks.workunit.client.0.smithi112.stdout:4/527: creat d6/d39/d1c/fbb x:0 0 0 2024-09-16T00:26:02.361 INFO:tasks.workunit.client.0.smithi112.stdout:4/528: chown d6/d39/d1c/fbb 59 1 2024-09-16T00:26:02.361 INFO:tasks.workunit.client.1.smithi169.stdout:1/406: rmdir dc/d36/d39 39 2024-09-16T00:26:02.362 INFO:tasks.workunit.client.0.smithi112.stdout:2/647: truncate d2/dd/d48/d94/d49/fba 1541500 0 2024-09-16T00:26:02.362 INFO:tasks.workunit.client.0.smithi112.stdout:3/552: dwrite d2/d9/d2b/d47/d52/d7a/d80/fb7 [0,4194304] 0 2024-09-16T00:26:02.364 INFO:tasks.workunit.client.1.smithi169.stdout:1/407: write dc/f34 [269972,107062] 0 2024-09-16T00:26:02.364 INFO:tasks.workunit.client.0.smithi112.stdout:2/648: read d2/dd/d48/d94/d49/f5c [114313,123703] 0 2024-09-16T00:26:02.364 INFO:tasks.workunit.client.0.smithi112.stdout:4/529: creat d6/db/d1f/fbc x:0 0 0 2024-09-16T00:26:02.365 INFO:tasks.workunit.client.1.smithi169.stdout:0/394: dwrite d0/d2/d8/fe [0,4194304] 0 2024-09-16T00:26:02.367 INFO:tasks.workunit.client.0.smithi112.stdout:2/649: write d2/dd/d48/d22/d2b/d54/d60/fc5 [80092,106365] 0 2024-09-16T00:26:02.367 INFO:tasks.workunit.client.0.smithi112.stdout:4/530: mknod d6/d39/d1c/d2c/d69/cbd 0 2024-09-16T00:26:02.370 INFO:tasks.workunit.client.0.smithi112.stdout:4/531: creat d6/d39/d1c/d6e/fbe x:0 0 0 2024-09-16T00:26:02.371 INFO:tasks.workunit.client.0.smithi112.stdout:2/650: truncate d2/dd/f91 573869 0 2024-09-16T00:26:02.373 INFO:tasks.workunit.client.0.smithi112.stdout:2/651: unlink d2/dd/d48/l99 0 2024-09-16T00:26:02.374 INFO:tasks.workunit.client.1.smithi169.stdout:5/367: write d2/da/d10/f11 [216465,75584] 0 2024-09-16T00:26:02.374 INFO:tasks.workunit.client.1.smithi169.stdout:1/408: creat dc/d36/d3a/f78 x:0 0 0 2024-09-16T00:26:02.374 INFO:tasks.workunit.client.1.smithi169.stdout:8/342: write d7/dc/d2a/f30 [2952568,66551] 0 2024-09-16T00:26:02.377 INFO:tasks.workunit.client.0.smithi112.stdout:3/553: write d2/f97 [618852,64022] 0 2024-09-16T00:26:02.378 INFO:tasks.workunit.client.1.smithi169.stdout:0/395: creat d0/d2/d8/d5/d60/f95 x:0 0 0 2024-09-16T00:26:02.378 INFO:tasks.workunit.client.0.smithi112.stdout:4/532: dwrite d6/d39/d1c/d66/f7d [0,4194304] 0 2024-09-16T00:26:02.379 INFO:tasks.workunit.client.1.smithi169.stdout:0/396: chown d0/d2c/d54/d21/f2f 653768807 1 2024-09-16T00:26:02.380 INFO:tasks.workunit.client.0.smithi112.stdout:4/533: chown d6/d39/d36/d5c/d7e/dab 231 1 2024-09-16T00:26:02.380 INFO:tasks.workunit.client.0.smithi112.stdout:3/554: dread - d2/d9/d2b/d47/d52/d7a/d80/d83/d49/fa8 zero size 2024-09-16T00:26:02.381 INFO:tasks.workunit.client.1.smithi169.stdout:6/476: symlink d2/d4d/d59/d43/d67/d87/l9a 0 2024-09-16T00:26:02.381 INFO:tasks.workunit.client.1.smithi169.stdout:1/409: rmdir dc/d26 39 2024-09-16T00:26:02.382 INFO:tasks.workunit.client.0.smithi112.stdout:4/534: rmdir d6/db/df/d5e/d61/d64 39 2024-09-16T00:26:02.383 INFO:tasks.workunit.client.1.smithi169.stdout:1/410: dread - dc/d36/f5b zero size 2024-09-16T00:26:02.384 INFO:tasks.workunit.client.0.smithi112.stdout:4/535: mknod d6/d7c/da0/cbf 0 2024-09-16T00:26:02.385 INFO:tasks.workunit.client.1.smithi169.stdout:6/477: mkdir d2/d4c/d9b 0 2024-09-16T00:26:02.385 INFO:tasks.workunit.client.1.smithi169.stdout:6/478: fdatasync d2/d4d/f7d 0 2024-09-16T00:26:02.386 INFO:tasks.workunit.client.0.smithi112.stdout:4/536: creat d6/d39/d36/d5c/fc0 x:0 0 0 2024-09-16T00:26:02.388 INFO:tasks.workunit.client.0.smithi112.stdout:4/537: rename d6/db/d1f/d94/fb7 to d6/db/d1f/fc1 0 2024-09-16T00:26:02.390 INFO:tasks.workunit.client.1.smithi169.stdout:4/411: link d2/d12/d13/f25 d2/d12/d13/d1f/d2f/f82 0 2024-09-16T00:26:02.390 INFO:tasks.workunit.client.1.smithi169.stdout:0/397: dwrite d0/d2c/d54/f61 [0,4194304] 0 2024-09-16T00:26:02.390 INFO:tasks.workunit.client.1.smithi169.stdout:1/411: symlink dc/l79 0 2024-09-16T00:26:02.391 INFO:tasks.workunit.client.0.smithi112.stdout:4/538: getdents d6/db/d1f/d41/d49/db1 0 2024-09-16T00:26:02.392 INFO:tasks.workunit.client.0.smithi112.stdout:4/539: creat d6/db/d1f/d41/d49/db1/fc2 x:0 0 0 2024-09-16T00:26:02.393 INFO:tasks.workunit.client.1.smithi169.stdout:0/398: fsync d0/d2c/d54/d56/f59 0 2024-09-16T00:26:02.394 INFO:tasks.workunit.client.0.smithi112.stdout:4/540: mknod d6/db/d1f/d94/cc3 0 2024-09-16T00:26:02.395 INFO:tasks.workunit.client.0.smithi112.stdout:4/541: fdatasync d6/db/df/f59 0 2024-09-16T00:26:02.397 INFO:tasks.workunit.client.0.smithi112.stdout:4/542: symlink d6/db/df/d5e/d61/d6d/daa/lc4 0 2024-09-16T00:26:02.397 INFO:tasks.workunit.client.1.smithi169.stdout:1/412: unlink dc/d36/d3a/d45/l71 0 2024-09-16T00:26:02.401 INFO:tasks.workunit.client.1.smithi169.stdout:1/413: read - dc/f6f zero size 2024-09-16T00:26:02.401 INFO:tasks.workunit.client.0.smithi112.stdout:4/543: truncate d6/db/d1f/d41/d49/db1/fc2 869113 0 2024-09-16T00:26:02.401 INFO:tasks.workunit.client.0.smithi112.stdout:4/544: mknod d6/db/df/d5e/d61/d64/d6a/cc5 0 2024-09-16T00:26:02.402 INFO:tasks.workunit.client.1.smithi169.stdout:0/399: truncate d0/d2/d15/f22 679779 0 2024-09-16T00:26:02.408 INFO:tasks.workunit.client.1.smithi169.stdout:8/343: dwrite d7/dc/d17/d24/f27 [0,4194304] 0 2024-09-16T00:26:02.409 INFO:tasks.workunit.client.0.smithi112.stdout:4/545: dread d6/d39/d36/d5c/d7e/f86 [0,4194304] 0 2024-09-16T00:26:02.409 INFO:tasks.workunit.client.1.smithi169.stdout:0/400: dread d0/d2c/d38/f66 [0,4194304] 0 2024-09-16T00:26:02.411 INFO:tasks.workunit.client.1.smithi169.stdout:8/344: chown d7/dc/d17/f39 355861412 1 2024-09-16T00:26:02.414 INFO:tasks.workunit.client.0.smithi112.stdout:3/555: dread d2/d9/d2b/f48 [0,4194304] 0 2024-09-16T00:26:02.415 INFO:tasks.workunit.client.1.smithi169.stdout:8/345: unlink d7/dc/d26/c31 0 2024-09-16T00:26:02.418 INFO:tasks.workunit.client.1.smithi169.stdout:0/401: read d0/f51 [532731,123739] 0 2024-09-16T00:26:02.418 INFO:tasks.workunit.client.1.smithi169.stdout:4/412: dwrite d2/d12/d13/d27/f32 [0,4194304] 0 2024-09-16T00:26:02.419 INFO:tasks.workunit.client.1.smithi169.stdout:8/346: mknod d7/dc/d53/c6c 0 2024-09-16T00:26:02.420 INFO:tasks.workunit.client.1.smithi169.stdout:6/479: dread d2/fb [0,4194304] 0 2024-09-16T00:26:02.420 INFO:tasks.workunit.client.1.smithi169.stdout:0/402: stat d0/d2c/d54/d21/d31/f58 0 2024-09-16T00:26:02.423 INFO:tasks.workunit.client.1.smithi169.stdout:6/480: readlink d2/d3f/d41/d5c/d73/l7b 0 2024-09-16T00:26:02.427 INFO:tasks.workunit.client.1.smithi169.stdout:4/413: dwrite d2/d12/d13/d4e/d50/f79 [0,4194304] 0 2024-09-16T00:26:02.428 INFO:tasks.workunit.client.1.smithi169.stdout:8/347: rename d7/dc/d1d/c52 to d7/dc/c6d 0 2024-09-16T00:26:02.436 INFO:tasks.workunit.client.1.smithi169.stdout:0/403: rename d0/d2/d4b/d86/f43 to d0/d2/d8/d5/d79/f96 0 2024-09-16T00:26:02.441 INFO:tasks.workunit.client.1.smithi169.stdout:0/404: getdents d0/d2c/d54/d21/d31/d83 0 2024-09-16T00:26:02.443 INFO:tasks.workunit.client.1.smithi169.stdout:0/405: mknod d0/d2c/d54/d21/d31/d83/c97 0 2024-09-16T00:26:02.451 INFO:tasks.workunit.client.1.smithi169.stdout:6/481: dwrite d2/f64 [0,4194304] 0 2024-09-16T00:26:02.453 INFO:tasks.workunit.client.1.smithi169.stdout:4/414: truncate d2/fb 2349912 0 2024-09-16T00:26:02.454 INFO:tasks.workunit.client.1.smithi169.stdout:0/406: write d0/f13 [669622,11466] 0 2024-09-16T00:26:02.458 INFO:tasks.workunit.client.1.smithi169.stdout:8/348: dwrite d7/dc/d17/f28 [0,4194304] 0 2024-09-16T00:26:02.458 INFO:tasks.workunit.client.1.smithi169.stdout:0/407: truncate d0/d2/f93 183443 0 2024-09-16T00:26:02.461 INFO:tasks.workunit.client.1.smithi169.stdout:8/349: symlink d7/dc/d1d/d3c/l6e 0 2024-09-16T00:26:02.461 INFO:tasks.workunit.client.1.smithi169.stdout:6/482: link d2/d4d/d59/d43/d67/c6b d2/d3f/d41/d5c/d73/c9c 0 2024-09-16T00:26:02.462 INFO:tasks.workunit.client.1.smithi169.stdout:8/350: dread - d7/dc/d26/d41/f5c zero size 2024-09-16T00:26:02.463 INFO:tasks.workunit.client.1.smithi169.stdout:8/351: fsync d7/dc/d26/d46/f6b 0 2024-09-16T00:26:02.464 INFO:tasks.workunit.client.1.smithi169.stdout:0/408: mknod d0/d2c/c98 0 2024-09-16T00:26:02.464 INFO:tasks.workunit.client.1.smithi169.stdout:8/352: stat d7/dc/d1d/d3c/l63 0 2024-09-16T00:26:02.465 INFO:tasks.workunit.client.1.smithi169.stdout:0/409: dread - d0/d2/d15/f7a zero size 2024-09-16T00:26:02.466 INFO:tasks.workunit.client.1.smithi169.stdout:4/415: rename d2/d12/d13/d1f/d6d to d2/d12/d13/d1f/d3e/d83 0 2024-09-16T00:26:02.468 INFO:tasks.workunit.client.1.smithi169.stdout:8/353: creat d7/dc/d17/d24/d32/f6f x:0 0 0 2024-09-16T00:26:02.469 INFO:tasks.workunit.client.1.smithi169.stdout:0/410: write d0/d2/d8/d5/d79/d7c/f94 [178721,45054] 0 2024-09-16T00:26:02.472 INFO:tasks.workunit.client.1.smithi169.stdout:8/354: mknod d7/d37/c70 0 2024-09-16T00:26:02.473 INFO:tasks.workunit.client.1.smithi169.stdout:6/483: dread d2/d22/d51/f77 [0,4194304] 0 2024-09-16T00:26:02.474 INFO:tasks.workunit.client.1.smithi169.stdout:0/411: mknod d0/d2/c99 0 2024-09-16T00:26:02.475 INFO:tasks.workunit.client.1.smithi169.stdout:4/416: dread d2/d12/d13/d1f/d3e/d5b/f24 [0,4194304] 0 2024-09-16T00:26:02.480 INFO:tasks.workunit.client.1.smithi169.stdout:5/368: read d2/f2a [595686,93366] 0 2024-09-16T00:26:02.482 INFO:tasks.workunit.client.1.smithi169.stdout:0/412: chown d0/d2/d15/f7a 1 1 2024-09-16T00:26:02.483 INFO:tasks.workunit.client.1.smithi169.stdout:8/355: dwrite d7/dc/d2a/f30 [0,4194304] 0 2024-09-16T00:26:02.487 INFO:tasks.workunit.client.0.smithi112.stdout:2/652: write d2/dd/d48/d22/f29 [1848407,92048] 0 2024-09-16T00:26:02.490 INFO:tasks.workunit.client.0.smithi112.stdout:2/653: creat d2/dd/d48/d94/d49/fca x:0 0 0 2024-09-16T00:26:02.492 INFO:tasks.workunit.client.1.smithi169.stdout:0/413: write d0/d4c/d6f/f91 [183244,105265] 0 2024-09-16T00:26:02.493 INFO:tasks.workunit.client.1.smithi169.stdout:5/369: fdatasync d2/da/d10/d13/d1c/f41 0 2024-09-16T00:26:02.495 INFO:tasks.workunit.client.0.smithi112.stdout:4/546: dread d6/d39/d36/d5c/f63 [0,4194304] 0 2024-09-16T00:26:02.495 INFO:tasks.workunit.client.0.smithi112.stdout:2/654: creat d2/dd/d48/d22/d2b/d54/fcb x:0 0 0 2024-09-16T00:26:02.496 INFO:tasks.workunit.client.1.smithi169.stdout:4/417: dwrite d2/d12/d13/d1f/d3e/d5b/d1a/f47 [0,4194304] 0 2024-09-16T00:26:02.497 INFO:tasks.workunit.client.1.smithi169.stdout:6/484: dwrite d2/d4d/d59/d38/f62 [0,4194304] 0 2024-09-16T00:26:02.499 INFO:tasks.workunit.client.0.smithi112.stdout:3/556: dwrite d2/d9/faa [0,4194304] 0 2024-09-16T00:26:02.501 INFO:tasks.workunit.client.1.smithi169.stdout:5/370: symlink d2/da/d67/l80 0 2024-09-16T00:26:02.502 INFO:tasks.workunit.client.0.smithi112.stdout:4/547: rename d6/d7c/dad to d6/d39/d36/d5c/dc6 0 2024-09-16T00:26:02.504 INFO:tasks.workunit.client.1.smithi169.stdout:5/371: chown d2/da/d10/d13 10693552 1 2024-09-16T00:26:02.504 INFO:tasks.workunit.client.0.smithi112.stdout:4/548: write d6/db/df/d3c/f73 [2596366,100304] 0 2024-09-16T00:26:02.505 INFO:tasks.workunit.client.0.smithi112.stdout:2/655: rename d2/dd/d48/d22/d2b/f43 to d2/dd/d48/d22/d2b/d54/d60/da9/fcc 0 2024-09-16T00:26:02.508 INFO:tasks.workunit.client.0.smithi112.stdout:2/656: rmdir d2/d68 39 2024-09-16T00:26:02.509 INFO:tasks.workunit.client.1.smithi169.stdout:8/356: dwrite d7/dc/d17/d48/f60 [0,4194304] 0 2024-09-16T00:26:02.510 INFO:tasks.workunit.client.0.smithi112.stdout:4/549: mknod d6/cc7 0 2024-09-16T00:26:02.510 INFO:tasks.workunit.client.0.smithi112.stdout:8/511: sync 2024-09-16T00:26:02.510 INFO:tasks.workunit.client.1.smithi169.stdout:5/372: rmdir d2/da/d10 39 2024-09-16T00:26:02.511 INFO:tasks.workunit.client.0.smithi112.stdout:8/512: read - d10/f63 zero size 2024-09-16T00:26:02.512 INFO:tasks.workunit.client.0.smithi112.stdout:8/513: read - d10/d12/d1c/d75/f80 zero size 2024-09-16T00:26:02.512 INFO:tasks.workunit.client.0.smithi112.stdout:2/657: link d2/dd/d48/d22/fb1 d2/dd/d96/fcd 0 2024-09-16T00:26:02.515 INFO:tasks.workunit.client.1.smithi169.stdout:4/418: read d2/f49 [167870,126321] 0 2024-09-16T00:26:02.515 INFO:tasks.workunit.client.0.smithi112.stdout:2/658: mknod d2/dd/d48/d22/cce 0 2024-09-16T00:26:02.516 INFO:tasks.workunit.client.1.smithi169.stdout:6/485: rename d2/d22/f34 to d2/d3f/d41/d72/f9d 0 2024-09-16T00:26:02.516 INFO:tasks.workunit.client.1.smithi169.stdout:8/357: readlink d7/d37/l4a 0 2024-09-16T00:26:02.516 INFO:tasks.workunit.client.0.smithi112.stdout:2/659: stat d2/dd/d48/d22/d67/c83 0 2024-09-16T00:26:02.517 INFO:tasks.workunit.client.0.smithi112.stdout:8/514: creat d10/d12/d1c/d96/fa7 x:0 0 0 2024-09-16T00:26:02.518 INFO:tasks.workunit.client.1.smithi169.stdout:5/373: rename d2/da/d10 to d2/da/d10/d13/d1c/d40/d68/d81 22 2024-09-16T00:26:02.518 INFO:tasks.workunit.client.0.smithi112.stdout:8/515: dread - d10/d12/d1c/d61/d6c/f93 zero size 2024-09-16T00:26:02.519 INFO:tasks.workunit.client.1.smithi169.stdout:4/419: chown d2/d12/d13/d1f/d3e/d5b/d1a 109647 1 2024-09-16T00:26:02.520 INFO:tasks.workunit.client.0.smithi112.stdout:2/660: creat d2/dd/d48/d22/d67/d9a/fcf x:0 0 0 2024-09-16T00:26:02.522 INFO:tasks.workunit.client.0.smithi112.stdout:3/557: dread d2/d9/d2b/d47/f54 [0,4194304] 0 2024-09-16T00:26:02.522 INFO:tasks.workunit.client.1.smithi169.stdout:5/374: creat d2/da/d67/f82 x:0 0 0 2024-09-16T00:26:02.523 INFO:tasks.workunit.client.1.smithi169.stdout:6/486: mkdir d2/d3f/d41/d72/d9e 0 2024-09-16T00:26:02.524 INFO:tasks.workunit.client.0.smithi112.stdout:3/558: chown d2/f2d 491028 1 2024-09-16T00:26:02.524 INFO:tasks.workunit.client.0.smithi112.stdout:2/661: dread d2/dd/d48/d22/f29 [0,4194304] 0 2024-09-16T00:26:02.525 INFO:tasks.workunit.client.1.smithi169.stdout:5/375: stat d2/da/d10/d13/d1c/d2d 0 2024-09-16T00:26:02.525 INFO:tasks.workunit.client.1.smithi169.stdout:6/487: fdatasync d2/d3f/f85 0 2024-09-16T00:26:02.526 INFO:tasks.workunit.client.1.smithi169.stdout:5/376: write d2/f58 [133106,110167] 0 2024-09-16T00:26:02.528 INFO:tasks.workunit.client.1.smithi169.stdout:5/377: write d2/f8 [2946704,121651] 0 2024-09-16T00:26:02.529 INFO:tasks.workunit.client.0.smithi112.stdout:2/662: creat d2/dd/d48/d22/d2b/d54/d60/d6c/d88/da4/fd0 x:0 0 0 2024-09-16T00:26:02.529 INFO:tasks.workunit.client.1.smithi169.stdout:4/420: dread d2/d12/d13/d1f/f35 [0,4194304] 0 2024-09-16T00:26:02.530 INFO:tasks.workunit.client.1.smithi169.stdout:6/488: mkdir d2/d4c/d9f 0 2024-09-16T00:26:02.530 INFO:tasks.workunit.client.1.smithi169.stdout:4/421: write d2/d12/d13/d4e/f76 [1964345,59418] 0 2024-09-16T00:26:02.531 INFO:tasks.workunit.client.0.smithi112.stdout:2/663: mkdir d2/d68/dd1 0 2024-09-16T00:26:02.532 INFO:tasks.workunit.client.1.smithi169.stdout:4/422: fsync d2/d12/d13/f60 0 2024-09-16T00:26:02.534 INFO:tasks.workunit.client.1.smithi169.stdout:5/378: rename d2/da/d10/d13/d1c/d40/d54/d6d to d2/da/d10/d13/d1c/d32/d53/d83 0 2024-09-16T00:26:02.536 INFO:tasks.workunit.client.1.smithi169.stdout:5/379: chown d2/da/d10/d13/d1c/d32/c42 167 1 2024-09-16T00:26:02.537 INFO:tasks.workunit.client.1.smithi169.stdout:6/489: getdents d2/d3f/d95 0 2024-09-16T00:26:02.537 INFO:tasks.workunit.client.1.smithi169.stdout:4/423: creat d2/d12/d13/d4d/f84 x:0 0 0 2024-09-16T00:26:02.539 INFO:tasks.workunit.client.1.smithi169.stdout:6/490: dread - d2/d4d/f8e zero size 2024-09-16T00:26:02.541 INFO:tasks.workunit.client.1.smithi169.stdout:5/380: write d2/da/d67/f82 [635255,38388] 0 2024-09-16T00:26:02.541 INFO:tasks.workunit.client.0.smithi112.stdout:8/516: rmdir d10/d12/d1c 39 2024-09-16T00:26:02.542 INFO:tasks.workunit.client.1.smithi169.stdout:0/414: dread d0/d50/f70 [0,4194304] 0 2024-09-16T00:26:02.543 INFO:tasks.workunit.client.1.smithi169.stdout:6/491: stat d2/d4d/l3d 0 2024-09-16T00:26:02.543 INFO:tasks.workunit.client.0.smithi112.stdout:3/559: write d2/d9/d2b/d2c/d69/d92/f96 [81861,38434] 0 2024-09-16T00:26:02.544 INFO:tasks.workunit.client.1.smithi169.stdout:0/415: chown d0/d2c/d54/l74 1114 1 2024-09-16T00:26:02.544 INFO:tasks.workunit.client.1.smithi169.stdout:4/424: creat d2/d12/d13/d4e/d50/f85 x:0 0 0 2024-09-16T00:26:02.545 INFO:tasks.workunit.client.0.smithi112.stdout:2/664: dread d2/dd/d48/f61 [0,4194304] 0 2024-09-16T00:26:02.546 INFO:tasks.workunit.client.1.smithi169.stdout:8/358: truncate d7/dc/d17/f1b 1624415 0 2024-09-16T00:26:02.546 INFO:tasks.workunit.client.1.smithi169.stdout:5/381: dread d2/da/d10/d13/d1c/d40/d43/f4c [0,4194304] 0 2024-09-16T00:26:02.546 INFO:tasks.workunit.client.0.smithi112.stdout:8/517: creat d10/d12/d1c/d3b/fa8 x:0 0 0 2024-09-16T00:26:02.547 INFO:tasks.workunit.client.1.smithi169.stdout:6/492: write d2/d3f/d41/d52/f89 [652650,28567] 0 2024-09-16T00:26:02.548 INFO:tasks.workunit.client.1.smithi169.stdout:0/416: write d0/d2c/d54/f1c [4854101,704] 0 2024-09-16T00:26:02.548 INFO:tasks.workunit.client.0.smithi112.stdout:2/665: dread - d2/fbf zero size 2024-09-16T00:26:02.548 INFO:tasks.workunit.client.0.smithi112.stdout:4/550: dread d6/d39/d1c/f2b [0,4194304] 0 2024-09-16T00:26:02.548 INFO:tasks.workunit.client.0.smithi112.stdout:3/560: creat d2/d9/d2b/d2c/d69/fbf x:0 0 0 2024-09-16T00:26:02.548 INFO:tasks.workunit.client.0.smithi112.stdout:2/666: stat d2/l1f 0 2024-09-16T00:26:02.548 INFO:tasks.workunit.client.0.smithi112.stdout:8/518: mknod d10/d12/d24/ca9 0 2024-09-16T00:26:02.549 INFO:tasks.workunit.client.1.smithi169.stdout:4/425: chown d2/d12/d13/d5c 13906 1 2024-09-16T00:26:02.550 INFO:tasks.workunit.client.0.smithi112.stdout:3/561: unlink d2/d9/d2b/d47/l5c 0 2024-09-16T00:26:02.551 INFO:tasks.workunit.client.1.smithi169.stdout:4/426: fdatasync d2/d12/f1e 0 2024-09-16T00:26:02.551 INFO:tasks.workunit.client.1.smithi169.stdout:8/359: unlink d7/dc/d53/f61 0 2024-09-16T00:26:02.551 INFO:tasks.workunit.client.0.smithi112.stdout:3/562: dread - d2/d9/f94 zero size 2024-09-16T00:26:02.552 INFO:tasks.workunit.client.1.smithi169.stdout:0/417: write d0/f51 [118188,79889] 0 2024-09-16T00:26:02.553 INFO:tasks.workunit.client.0.smithi112.stdout:2/667: mknod d2/dd/d48/d22/d2b/d54/d60/d6c/dc9/cd2 0 2024-09-16T00:26:02.553 INFO:tasks.workunit.client.0.smithi112.stdout:8/519: dread d10/d12/d24/f2b [0,4194304] 0 2024-09-16T00:26:02.555 INFO:tasks.workunit.client.0.smithi112.stdout:3/563: rename d2/f91 to d2/d9/d2b/d47/d78/fc0 0 2024-09-16T00:26:02.556 INFO:tasks.workunit.client.0.smithi112.stdout:8/520: symlink d10/d12/d1c/d3b/d74/laa 0 2024-09-16T00:26:02.557 INFO:tasks.workunit.client.1.smithi169.stdout:5/382: creat d2/da/d10/d13/d1c/d40/d43/f84 x:0 0 0 2024-09-16T00:26:02.558 INFO:tasks.workunit.client.0.smithi112.stdout:3/564: fsync d2/d9/d2b/d47/d52/d7a/d80/d83/d76/faf 0 2024-09-16T00:26:02.558 INFO:tasks.workunit.client.0.smithi112.stdout:8/521: chown d10/d12/d1c/f59 1412 1 2024-09-16T00:26:02.561 INFO:tasks.workunit.client.1.smithi169.stdout:6/493: mkdir d2/d4d/da0 0 2024-09-16T00:26:02.561 INFO:tasks.workunit.client.0.smithi112.stdout:2/668: dwrite d2/dd/d48/d22/d67/d9a/fc3 [0,4194304] 0 2024-09-16T00:26:02.562 INFO:tasks.workunit.client.0.smithi112.stdout:4/551: dread f4 [0,4194304] 0 2024-09-16T00:26:02.563 INFO:tasks.workunit.client.1.smithi169.stdout:9/420: sync 2024-09-16T00:26:02.565 INFO:tasks.workunit.client.1.smithi169.stdout:4/427: dwrite d2/d12/d13/d1f/d2f/f43 [0,4194304] 0 2024-09-16T00:26:02.565 INFO:tasks.workunit.client.0.smithi112.stdout:4/552: mkdir d6/d39/dc8 0 2024-09-16T00:26:02.568 INFO:tasks.workunit.client.0.smithi112.stdout:3/565: dwrite d2/d9/d2b/d47/d52/d7a/d80/d83/d49/f77 [4194304,4194304] 0 2024-09-16T00:26:02.569 INFO:tasks.workunit.client.0.smithi112.stdout:8/522: link d10/f34 d10/d4e/d8c/fab 0 2024-09-16T00:26:02.571 INFO:tasks.workunit.client.0.smithi112.stdout:8/523: dread - d10/d12/d1c/d61/fa2 zero size 2024-09-16T00:26:02.571 INFO:tasks.workunit.client.0.smithi112.stdout:8/524: stat d10/d4e/d8c/c9c 0 2024-09-16T00:26:02.572 INFO:tasks.workunit.client.0.smithi112.stdout:4/553: rename d6/d39/d1c/f2f to d6/d39/d1c/d2c/fc9 0 2024-09-16T00:26:02.572 INFO:tasks.workunit.client.0.smithi112.stdout:8/525: fsync d10/d12/d1c/d75/f78 0 2024-09-16T00:26:02.574 INFO:tasks.workunit.client.1.smithi169.stdout:8/360: rmdir d7/dc/d17/d6a 0 2024-09-16T00:26:02.577 INFO:tasks.workunit.client.0.smithi112.stdout:8/526: creat d10/d12/d1c/d96/fac x:0 0 0 2024-09-16T00:26:02.578 INFO:tasks.workunit.client.0.smithi112.stdout:4/554: creat d6/db/d1f/d41/d49/fca x:0 0 0 2024-09-16T00:26:02.578 INFO:tasks.workunit.client.1.smithi169.stdout:9/421: rmdir d5 39 2024-09-16T00:26:02.580 INFO:tasks.workunit.client.0.smithi112.stdout:0/634: sync 2024-09-16T00:26:02.581 INFO:tasks.workunit.client.0.smithi112.stdout:2/669: rename d2/dd/d48/d94/d49/cc8 to d2/dd/d48/d22/d2b/d54/d60/d6c/d88/da4/cd3 0 2024-09-16T00:26:02.582 INFO:tasks.workunit.client.1.smithi169.stdout:4/428: symlink d2/d12/d13/d1f/d3e/d5b/l86 0 2024-09-16T00:26:02.582 INFO:tasks.workunit.client.0.smithi112.stdout:4/555: creat d6/db/df/d5e/fcb x:0 0 0 2024-09-16T00:26:02.582 INFO:tasks.workunit.client.1.smithi169.stdout:5/383: creat d2/f85 x:0 0 0 2024-09-16T00:26:02.583 INFO:tasks.workunit.client.1.smithi169.stdout:6/494: creat d2/d22/fa1 x:0 0 0 2024-09-16T00:26:02.583 INFO:tasks.workunit.client.0.smithi112.stdout:3/566: rename d2/d9/d2b/d2c/d69/la1 to d2/d9/d2b/d2c/d69/d92/lc1 0 2024-09-16T00:26:02.585 INFO:tasks.workunit.client.1.smithi169.stdout:8/361: link d7/dc/d17/d24/d32/l44 d7/dc/d1d/d3c/l71 0 2024-09-16T00:26:02.585 INFO:tasks.workunit.client.0.smithi112.stdout:2/670: truncate d2/dd/f5e 3412082 0 2024-09-16T00:26:02.586 INFO:tasks.workunit.client.1.smithi169.stdout:9/422: mkdir d5/da/d5b/d91/d9a 0 2024-09-16T00:26:02.586 INFO:tasks.workunit.client.1.smithi169.stdout:6/495: mkdir d2/d4d/d59/d43/d67/d87/da2 0 2024-09-16T00:26:02.587 INFO:tasks.workunit.client.0.smithi112.stdout:3/567: mknod d2/d4d/cc2 0 2024-09-16T00:26:02.587 INFO:tasks.workunit.client.0.smithi112.stdout:2/671: chown d2/f39 0 1 2024-09-16T00:26:02.588 INFO:tasks.workunit.client.1.smithi169.stdout:8/362: unlink d7/d20/f4c 0 2024-09-16T00:26:02.588 INFO:tasks.workunit.client.1.smithi169.stdout:5/384: read d2/da/f28 [1766227,86946] 0 2024-09-16T00:26:02.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:02 smithi112 ceph-mon[27869]: pgmap v254: 65 pgs: 65 active+clean; 1.4 GiB data, 5.6 GiB used, 531 GiB / 536 GiB avail; 33 MiB/s rd, 177 MiB/s wr, 301 op/s 2024-09-16T00:26:02.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:02 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:26:02.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:02 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:26:02.589 INFO:tasks.workunit.client.0.smithi112.stdout:3/568: stat d2/d9/d2b/d47/d78 0 2024-09-16T00:26:02.589 INFO:tasks.workunit.client.1.smithi169.stdout:5/385: chown d2/l5 534839 1 2024-09-16T00:26:02.590 INFO:tasks.workunit.client.0.smithi112.stdout:0/635: dwrite f0 [4194304,4194304] 0 2024-09-16T00:26:02.593 INFO:tasks.workunit.client.0.smithi112.stdout:2/672: creat d2/dd/d48/d22/fd4 x:0 0 0 2024-09-16T00:26:02.593 INFO:tasks.workunit.client.0.smithi112.stdout:4/556: dwrite d6/d39/d36/d5c/d7e/d8f/f98 [0,4194304] 0 2024-09-16T00:26:02.595 INFO:tasks.workunit.client.0.smithi112.stdout:0/636: mknod d5/da/d18/d24/d5a/cc7 0 2024-09-16T00:26:02.595 INFO:tasks.workunit.client.0.smithi112.stdout:4/557: fdatasync d6/d39/d1c/d66/faf 0 2024-09-16T00:26:02.600 INFO:tasks.workunit.client.0.smithi112.stdout:4/558: symlink d6/d39/d36/d5c/d7e/lcc 0 2024-09-16T00:26:02.601 INFO:tasks.workunit.client.0.smithi112.stdout:4/559: chown d6/d39/d1c/d6e/fbe 8850 1 2024-09-16T00:26:02.601 INFO:tasks.workunit.client.0.smithi112.stdout:2/673: mkdir d2/dd/d48/d22/d2b/d54/d60/d6c/dc9/dd5 0 2024-09-16T00:26:02.602 INFO:tasks.workunit.client.0.smithi112.stdout:0/637: truncate d5/da/dc/f13 2570551 0 2024-09-16T00:26:02.606 INFO:tasks.workunit.client.0.smithi112.stdout:0/638: creat d5/da/d57/fc8 x:0 0 0 2024-09-16T00:26:02.606 INFO:tasks.workunit.client.0.smithi112.stdout:4/560: dread d6/d39/d1c/f2b [0,4194304] 0 2024-09-16T00:26:02.609 INFO:tasks.workunit.client.0.smithi112.stdout:0/639: creat d5/da/d66/fc9 x:0 0 0 2024-09-16T00:26:02.609 INFO:tasks.workunit.client.0.smithi112.stdout:3/569: write d2/d9/d2b/f48 [442314,87079] 0 2024-09-16T00:26:02.611 INFO:tasks.workunit.client.0.smithi112.stdout:4/561: rename d6/db/df/d5e/d61/d6d to d6/d39/d36/d5c/d7e/dcd 0 2024-09-16T00:26:02.613 INFO:tasks.workunit.client.0.smithi112.stdout:0/640: creat d5/da/d18/d63/db2/fca x:0 0 0 2024-09-16T00:26:02.614 INFO:tasks.workunit.client.0.smithi112.stdout:3/570: rename d2/d9/d2b/f48 to d2/d9/d2b/d47/d52/d7a/fc3 0 2024-09-16T00:26:02.615 INFO:tasks.workunit.client.0.smithi112.stdout:4/562: creat d6/d39/fce x:0 0 0 2024-09-16T00:26:02.615 INFO:tasks.workunit.client.0.smithi112.stdout:4/563: read - d6/db/df/d3c/f82 zero size 2024-09-16T00:26:02.616 INFO:tasks.workunit.client.0.smithi112.stdout:3/571: stat d2/d9/d2b/c7f 0 2024-09-16T00:26:02.617 INFO:tasks.workunit.client.0.smithi112.stdout:0/641: symlink d5/da/d21/dc3/lcb 0 2024-09-16T00:26:02.617 INFO:tasks.workunit.client.0.smithi112.stdout:3/572: dread - d2/d9/d2b/d47/fac zero size 2024-09-16T00:26:02.618 INFO:tasks.workunit.client.0.smithi112.stdout:4/564: write d6/da/f84 [904572,86265] 0 2024-09-16T00:26:02.620 INFO:tasks.workunit.client.0.smithi112.stdout:3/573: write d2/d9/faa [5172539,98707] 0 2024-09-16T00:26:02.623 INFO:tasks.workunit.client.0.smithi112.stdout:0/642: symlink d5/da/d18/d24/d46/d5f/d7a/d8d/lcc 0 2024-09-16T00:26:02.624 INFO:tasks.workunit.client.0.smithi112.stdout:3/574: symlink d2/d4d/lc4 0 2024-09-16T00:26:02.626 INFO:tasks.workunit.client.0.smithi112.stdout:4/565: dwrite f4 [0,4194304] 0 2024-09-16T00:26:02.631 INFO:tasks.workunit.client.0.smithi112.stdout:2/674: dwrite d2/dd/f5e [0,4194304] 0 2024-09-16T00:26:02.632 INFO:tasks.workunit.client.0.smithi112.stdout:4/566: unlink d6/d39/d1c/d66/l89 0 2024-09-16T00:26:02.636 INFO:tasks.workunit.client.0.smithi112.stdout:4/567: mknod d6/db/d43/ccf 0 2024-09-16T00:26:02.638 INFO:tasks.workunit.client.0.smithi112.stdout:3/575: link d2/d9/d2b/d2c/f51 d2/d9/d2b/d47/d52/d7a/d80/d83/d68/fc5 0 2024-09-16T00:26:02.640 INFO:tasks.workunit.client.0.smithi112.stdout:0/643: dwrite d5/da/d18/d24/d46/d88/fb4 [0,4194304] 0 2024-09-16T00:26:02.641 INFO:tasks.workunit.client.0.smithi112.stdout:3/576: fsync d2/d9/f16 0 2024-09-16T00:26:02.643 INFO:tasks.workunit.client.0.smithi112.stdout:4/568: rename d6/db/d1f/f71 to d6/db/df/d5e/d61/d64/d6a/fd0 0 2024-09-16T00:26:02.646 INFO:tasks.workunit.client.0.smithi112.stdout:2/675: dwrite d2/dd/d48/d22/d2b/d54/d60/daf/fbc [0,4194304] 0 2024-09-16T00:26:02.646 INFO:tasks.workunit.client.0.smithi112.stdout:2/676: fsync d2/fbf 0 2024-09-16T00:26:02.649 INFO:tasks.workunit.client.0.smithi112.stdout:4/569: getdents d6/d39/d36/d5c/d7e/dab 0 2024-09-16T00:26:02.652 INFO:tasks.workunit.client.0.smithi112.stdout:7/593: sync 2024-09-16T00:26:02.652 INFO:tasks.workunit.client.0.smithi112.stdout:6/501: sync 2024-09-16T00:26:02.652 INFO:tasks.workunit.client.0.smithi112.stdout:9/752: sync 2024-09-16T00:26:02.652 INFO:tasks.workunit.client.0.smithi112.stdout:1/606: sync 2024-09-16T00:26:02.652 INFO:tasks.workunit.client.0.smithi112.stdout:5/470: sync 2024-09-16T00:26:02.654 INFO:tasks.workunit.client.0.smithi112.stdout:4/570: mkdir d6/d39/d36/d5c/d7e/dd1 0 2024-09-16T00:26:02.655 INFO:tasks.workunit.client.0.smithi112.stdout:5/471: dread - d4/d4f/d78/fa0 zero size 2024-09-16T00:26:02.655 INFO:tasks.workunit.client.0.smithi112.stdout:3/577: getdents d2/d9/d2b/d47/d52/d7a/d80/d83/d76/d8e 0 2024-09-16T00:26:02.658 INFO:tasks.workunit.client.0.smithi112.stdout:4/571: chown d6/db/d1f/d41/d49/c21 3052 1 2024-09-16T00:26:02.659 INFO:tasks.workunit.client.0.smithi112.stdout:6/502: fdatasync dd/df/d10/d47/d55/f72 0 2024-09-16T00:26:02.659 INFO:tasks.workunit.client.0.smithi112.stdout:3/578: stat d2/d9/d2b/d47/d52/fa3 0 2024-09-16T00:26:02.660 INFO:tasks.workunit.client.0.smithi112.stdout:9/753: unlink d1/d5/d20/d81/d1a/d88/def/cc9 0 2024-09-16T00:26:02.661 INFO:tasks.workunit.client.0.smithi112.stdout:2/677: dwrite d2/fa [0,4194304] 0 2024-09-16T00:26:02.662 INFO:tasks.workunit.client.0.smithi112.stdout:1/607: readlink d0/d14/df/d28/d45/d46/d58/d62/db0/lbf 0 2024-09-16T00:26:02.662 INFO:tasks.workunit.client.0.smithi112.stdout:2/678: stat d2/f64 0 2024-09-16T00:26:02.663 INFO:tasks.workunit.client.0.smithi112.stdout:6/503: mkdir dd/df/d17/da2 0 2024-09-16T00:26:02.664 INFO:tasks.workunit.client.0.smithi112.stdout:9/754: rmdir d1/d5/d20 39 2024-09-16T00:26:02.666 INFO:tasks.workunit.client.0.smithi112.stdout:7/594: dwrite d6/d10/d16/f3e [0,4194304] 0 2024-09-16T00:26:02.670 INFO:tasks.workunit.client.0.smithi112.stdout:2/679: dread - d2/dd/d48/d22/d2b/d54/d60/fc7 zero size 2024-09-16T00:26:02.672 INFO:tasks.workunit.client.0.smithi112.stdout:1/608: creat d0/d14/df/d28/d45/d46/d58/d62/db0/fc7 x:0 0 0 2024-09-16T00:26:02.673 INFO:tasks.workunit.client.0.smithi112.stdout:5/472: link d4/d7/l21 d4/dd/d12/d52/d64/la1 0 2024-09-16T00:26:02.674 INFO:tasks.workunit.client.0.smithi112.stdout:6/504: creat dd/df/d10/d47/d55/fa3 x:0 0 0 2024-09-16T00:26:02.677 INFO:tasks.workunit.client.0.smithi112.stdout:1/609: mknod d0/d14/d1c/d20/d93/cc8 0 2024-09-16T00:26:02.681 INFO:tasks.workunit.client.0.smithi112.stdout:6/505: truncate dd/df/f66 4549229 0 2024-09-16T00:26:02.684 INFO:tasks.workunit.client.0.smithi112.stdout:7/595: dwrite d6/d10/f6e [0,4194304] 0 2024-09-16T00:26:02.684 INFO:tasks.workunit.client.0.smithi112.stdout:4/572: truncate d6/db/df/d5e/d61/f3f 2990576 0 2024-09-16T00:26:02.685 INFO:tasks.workunit.client.0.smithi112.stdout:0/644: dread d5/da/d1d/d71/f3b [0,4194304] 0 2024-09-16T00:26:02.686 INFO:tasks.workunit.client.0.smithi112.stdout:1/610: write d0/d14/df/d28/f2e [7613289,111643] 0 2024-09-16T00:26:02.692 INFO:tasks.workunit.client.0.smithi112.stdout:2/680: dwrite d2/dd/d48/d22/fd4 [0,4194304] 0 2024-09-16T00:26:02.693 INFO:tasks.workunit.client.0.smithi112.stdout:5/473: dread d4/d7/d26/f44 [0,4194304] 0 2024-09-16T00:26:02.693 INFO:tasks.workunit.client.0.smithi112.stdout:2/681: stat d2/dd/d48/d94/d49/f5c 0 2024-09-16T00:26:02.695 INFO:tasks.workunit.client.0.smithi112.stdout:9/755: truncate d1/d5/d20/d81/d1a/d88/def/f1c 255130 0 2024-09-16T00:26:02.697 INFO:tasks.workunit.client.0.smithi112.stdout:4/573: mknod d6/db/df/cd2 0 2024-09-16T00:26:02.698 INFO:tasks.workunit.client.0.smithi112.stdout:6/506: rmdir dd/df/d11/d87 0 2024-09-16T00:26:02.701 INFO:tasks.workunit.client.0.smithi112.stdout:1/611: rename d0/d14/df/d28/d45/d46/f90 to d0/d14/df/d28/d45/d30/fc9 0 2024-09-16T00:26:02.703 INFO:tasks.workunit.client.0.smithi112.stdout:6/507: chown dd/df/d11/d44/l68 22865462 1 2024-09-16T00:26:02.703 INFO:tasks.workunit.client.0.smithi112.stdout:4/574: stat d6/d39/d36/d5c/fc0 0 2024-09-16T00:26:02.705 INFO:tasks.workunit.client.0.smithi112.stdout:1/612: stat d0/d14/df/d28/d45/d46/d58/d62/l64 0 2024-09-16T00:26:02.707 INFO:tasks.workunit.client.0.smithi112.stdout:6/508: write dd/df/d13/f48 [25689,33437] 0 2024-09-16T00:26:02.708 INFO:tasks.workunit.client.0.smithi112.stdout:2/682: mknod d2/dd/d48/d22/d2b/d54/d60/d6c/d88/dad/cd6 0 2024-09-16T00:26:02.713 INFO:tasks.workunit.client.1.smithi169.stdout:8/363: mknod d7/dc/d26/d41/c72 0 2024-09-16T00:26:02.714 INFO:tasks.workunit.client.0.smithi112.stdout:0/645: dread d5/da/d18/d24/d46/d88/f4a [0,4194304] 0 2024-09-16T00:26:02.716 INFO:tasks.workunit.client.0.smithi112.stdout:5/474: dwrite d4/d7/d14/f8c [0,4194304] 0 2024-09-16T00:26:02.717 INFO:tasks.workunit.client.1.smithi169.stdout:6/496: getdents d2/d36 0 2024-09-16T00:26:02.721 INFO:tasks.workunit.client.0.smithi112.stdout:4/575: rename d6/l58 to d6/d39/d36/d5c/d7e/dcd/ld3 0 2024-09-16T00:26:02.722 INFO:tasks.workunit.client.1.smithi169.stdout:5/386: symlink d2/da/d10/d13/d1c/d32/d53/d83/d7d/l86 0 2024-09-16T00:26:02.722 INFO:tasks.workunit.client.1.smithi169.stdout:9/423: dwrite d5/d12/f26 [0,4194304] 0 2024-09-16T00:26:02.723 INFO:tasks.workunit.client.0.smithi112.stdout:7/596: dwrite d6/d10/d16/d96/dc7/f60 [0,4194304] 0 2024-09-16T00:26:02.724 INFO:tasks.workunit.client.0.smithi112.stdout:2/683: fsync d2/dd/d48/d22/d2b/d54/d60/da9/fcc 0 2024-09-16T00:26:02.724 INFO:tasks.workunit.client.0.smithi112.stdout:1/613: creat d0/d14/df/d28/d45/d46/d58/d62/db0/fca x:0 0 0 2024-09-16T00:26:02.724 INFO:tasks.workunit.client.1.smithi169.stdout:5/387: truncate d2/f85 754301 0 2024-09-16T00:26:02.725 INFO:tasks.workunit.client.1.smithi169.stdout:9/424: truncate d5/da/d21/f44 468221 0 2024-09-16T00:26:02.726 INFO:tasks.workunit.client.0.smithi112.stdout:5/475: creat d4/d4f/d77/fa2 x:0 0 0 2024-09-16T00:26:02.726 INFO:tasks.workunit.client.1.smithi169.stdout:5/388: creat d2/da/d10/f87 x:0 0 0 2024-09-16T00:26:02.727 INFO:tasks.workunit.client.0.smithi112.stdout:0/646: mknod d5/da/d57/d76/ccd 0 2024-09-16T00:26:02.728 INFO:tasks.workunit.client.1.smithi169.stdout:9/425: read d5/da/d21/d23/f67 [238082,128336] 0 2024-09-16T00:26:02.729 INFO:tasks.workunit.client.1.smithi169.stdout:5/389: write d2/da/f19 [415275,58695] 0 2024-09-16T00:26:02.729 INFO:tasks.workunit.client.0.smithi112.stdout:4/576: chown d6/db/d1f/d94/cc3 43832 1 2024-09-16T00:26:02.730 INFO:tasks.workunit.client.0.smithi112.stdout:5/476: stat d4/d1c/f73 0 2024-09-16T00:26:02.730 INFO:tasks.workunit.client.0.smithi112.stdout:0/647: write d5/da/d57/d76/f17 [3913175,23993] 0 2024-09-16T00:26:02.733 INFO:tasks.workunit.client.0.smithi112.stdout:0/648: read - d5/da/d66/fc9 zero size 2024-09-16T00:26:02.734 INFO:tasks.workunit.client.0.smithi112.stdout:2/684: readlink d2/dd/d48/d22/l9d 0 2024-09-16T00:26:02.735 INFO:tasks.workunit.client.1.smithi169.stdout:9/426: creat d5/da/d73/f9b x:0 0 0 2024-09-16T00:26:02.735 INFO:tasks.workunit.client.0.smithi112.stdout:6/509: dwrite dd/d23/f35 [0,4194304] 0 2024-09-16T00:26:02.735 INFO:tasks.workunit.client.1.smithi169.stdout:5/390: creat d2/f88 x:0 0 0 2024-09-16T00:26:02.737 INFO:tasks.workunit.client.0.smithi112.stdout:7/597: creat d6/d10/fcb x:0 0 0 2024-09-16T00:26:02.738 INFO:tasks.workunit.client.1.smithi169.stdout:5/391: readlink d2/da/d10/l18 0 2024-09-16T00:26:02.740 INFO:tasks.workunit.client.1.smithi169.stdout:6/497: dwrite d2/d36/f2a [4194304,4194304] 0 2024-09-16T00:26:02.740 INFO:tasks.workunit.client.1.smithi169.stdout:8/364: dwrite d7/fd [4194304,4194304] 0 2024-09-16T00:26:02.741 INFO:tasks.workunit.client.1.smithi169.stdout:9/427: creat d5/d12/f9c x:0 0 0 2024-09-16T00:26:02.744 INFO:tasks.workunit.client.0.smithi112.stdout:9/756: dread d1/d5/d6/d2d/f92 [0,4194304] 0 2024-09-16T00:26:02.744 INFO:tasks.workunit.client.0.smithi112.stdout:2/685: rmdir d2/dd/d48/d22/d2b/d54/d60/d6c 39 2024-09-16T00:26:02.745 INFO:tasks.workunit.client.0.smithi112.stdout:7/598: fsync d6/d10/d75/fa5 0 2024-09-16T00:26:02.746 INFO:tasks.workunit.client.1.smithi169.stdout:9/428: write d5/da/d2e/f7b [419797,88545] 0 2024-09-16T00:26:02.746 INFO:tasks.workunit.client.0.smithi112.stdout:4/577: symlink d6/db/df/d5e/d61/d64/d6a/db8/ld4 0 2024-09-16T00:26:02.747 INFO:tasks.workunit.client.1.smithi169.stdout:5/392: mkdir d2/da/d10/d13/d1c/d32/d53/d4e/d89 0 2024-09-16T00:26:02.748 INFO:tasks.workunit.client.0.smithi112.stdout:1/614: dwrite d0/f4e [0,4194304] 0 2024-09-16T00:26:02.748 INFO:tasks.workunit.client.1.smithi169.stdout:5/393: fsync d2/fd 0 2024-09-16T00:26:02.752 INFO:tasks.workunit.client.1.smithi169.stdout:6/498: dread d2/d3f/d41/d52/f1e [0,4194304] 0 2024-09-16T00:26:02.752 INFO:tasks.workunit.client.0.smithi112.stdout:0/649: dread f0 [4194304,4194304] 0 2024-09-16T00:26:02.754 INFO:tasks.workunit.client.0.smithi112.stdout:5/477: getdents d4/d1c 0 2024-09-16T00:26:02.754 INFO:tasks.workunit.client.1.smithi169.stdout:8/365: rename d7/dc/d26/c5b to d7/dc/c73 0 2024-09-16T00:26:02.756 INFO:tasks.workunit.client.0.smithi112.stdout:0/650: dread - d5/da/d1d/d71/d64/db6/fbd zero size 2024-09-16T00:26:02.757 INFO:tasks.workunit.client.0.smithi112.stdout:9/757: creat d1/d5/d20/d81/d1a/d88/da8/f105 x:0 0 0 2024-09-16T00:26:02.757 INFO:tasks.workunit.client.0.smithi112.stdout:1/615: dread - d0/d14/d6a/d9e/fab zero size 2024-09-16T00:26:02.758 INFO:tasks.workunit.client.1.smithi169.stdout:6/499: chown d2/d4d/d59/d43/d67/c6b 18970 1 2024-09-16T00:26:02.758 INFO:tasks.workunit.client.0.smithi112.stdout:7/599: truncate d6/d7f/d85/fb5 1045866 0 2024-09-16T00:26:02.759 INFO:tasks.workunit.client.0.smithi112.stdout:0/651: dread - d5/da/d18/fc4 zero size 2024-09-16T00:26:02.762 INFO:tasks.workunit.client.1.smithi169.stdout:5/394: rename d2/f85 to d2/da/d10/f8a 0 2024-09-16T00:26:02.762 INFO:tasks.workunit.client.1.smithi169.stdout:9/429: dwrite d5/d12/f26 [4194304,4194304] 0 2024-09-16T00:26:02.762 INFO:tasks.workunit.client.0.smithi112.stdout:4/578: dwrite d6/d39/d36/d5c/f77 [0,4194304] 0 2024-09-16T00:26:02.763 INFO:tasks.workunit.client.0.smithi112.stdout:4/579: fsync d6/db/df/d5e/fcb 0 2024-09-16T00:26:02.765 INFO:tasks.workunit.client.0.smithi112.stdout:5/478: creat d4/d7/d14/d90/d9a/d7c/fa3 x:0 0 0 2024-09-16T00:26:02.766 INFO:tasks.workunit.client.0.smithi112.stdout:2/686: mkdir d2/dd/d48/d22/d2b/d54/d60/d6c/d88/dd7 0 2024-09-16T00:26:02.766 INFO:tasks.workunit.client.1.smithi169.stdout:9/430: chown d5/da/d2e 11012 1 2024-09-16T00:26:02.767 INFO:tasks.workunit.client.1.smithi169.stdout:3/439: sync 2024-09-16T00:26:02.767 INFO:tasks.workunit.client.1.smithi169.stdout:7/468: sync 2024-09-16T00:26:02.767 INFO:tasks.workunit.client.0.smithi112.stdout:5/479: fdatasync d4/d7/d26/f81 0 2024-09-16T00:26:02.767 INFO:tasks.workunit.client.1.smithi169.stdout:5/395: truncate d2/f6b 537748 0 2024-09-16T00:26:02.769 INFO:tasks.workunit.client.1.smithi169.stdout:8/366: mknod d7/dc/d17/d24/d32/d4d/c74 0 2024-09-16T00:26:02.769 INFO:tasks.workunit.client.0.smithi112.stdout:4/580: rmdir d6/d39/d36/d5c/dc6 39 2024-09-16T00:26:02.769 INFO:tasks.workunit.client.1.smithi169.stdout:6/500: dread d2/d3f/d41/d72/f9d [0,4194304] 0 2024-09-16T00:26:02.770 INFO:tasks.workunit.client.1.smithi169.stdout:9/431: truncate d5/da/f2b 4691844 0 2024-09-16T00:26:02.771 INFO:tasks.workunit.client.1.smithi169.stdout:7/469: truncate d7/d5b/f6c 89747 0 2024-09-16T00:26:02.771 INFO:tasks.workunit.client.0.smithi112.stdout:5/480: dread - d4/d7/d26/d36/f74 zero size 2024-09-16T00:26:02.771 INFO:tasks.workunit.client.0.smithi112.stdout:4/581: readlink d6/db/d1f/d94/l32 0 2024-09-16T00:26:02.772 INFO:tasks.workunit.client.0.smithi112.stdout:0/652: symlink d5/da/d1d/d71/lce 0 2024-09-16T00:26:02.772 INFO:tasks.workunit.client.1.smithi169.stdout:7/470: chown d7/d16 1 1 2024-09-16T00:26:02.773 INFO:tasks.workunit.client.1.smithi169.stdout:1/414: sync 2024-09-16T00:26:02.773 INFO:tasks.workunit.client.1.smithi169.stdout:2/300: sync 2024-09-16T00:26:02.773 INFO:tasks.workunit.client.0.smithi112.stdout:7/600: read d6/d10/d16/d96/dc7/d39/fa3 [55183,78827] 0 2024-09-16T00:26:02.775 INFO:tasks.workunit.client.0.smithi112.stdout:5/481: creat d4/d7/d14/d90/fa4 x:0 0 0 2024-09-16T00:26:02.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:02 smithi169 ceph-mon[34653]: pgmap v254: 65 pgs: 65 active+clean; 1.4 GiB data, 5.6 GiB used, 531 GiB / 536 GiB avail; 33 MiB/s rd, 177 MiB/s wr, 301 op/s 2024-09-16T00:26:02.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:02 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:26:02.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:02 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:26:02.777 INFO:tasks.workunit.client.0.smithi112.stdout:0/653: chown d5/da/d66/fc9 949785 1 2024-09-16T00:26:02.777 INFO:tasks.workunit.client.0.smithi112.stdout:5/482: chown d4/d1c/d43/f54 184 1 2024-09-16T00:26:02.777 INFO:tasks.workunit.client.0.smithi112.stdout:0/654: fdatasync d5/da/d18/d63/db2/fca 0 2024-09-16T00:26:02.779 INFO:tasks.workunit.client.0.smithi112.stdout:4/582: symlink d6/d39/d36/d5c/d7e/dd1/ld5 0 2024-09-16T00:26:02.779 INFO:tasks.workunit.client.0.smithi112.stdout:5/483: stat d4/d4f/d77/fa2 0 2024-09-16T00:26:02.779 INFO:tasks.workunit.client.1.smithi169.stdout:3/440: symlink d7/d1a/d1c/d35/la7 0 2024-09-16T00:26:02.779 INFO:tasks.workunit.client.1.smithi169.stdout:9/432: unlink d5/d49/d64/f65 0 2024-09-16T00:26:02.780 INFO:tasks.workunit.client.0.smithi112.stdout:2/687: link d2/dd/d48/d22/d2b/d30/c3f d2/dd/d48/d22/d67/d9a/cd8 0 2024-09-16T00:26:02.781 INFO:tasks.workunit.client.0.smithi112.stdout:5/484: dread - d4/d4f/d78/fa0 zero size 2024-09-16T00:26:02.784 INFO:tasks.workunit.client.0.smithi112.stdout:6/510: dread dd/d23/f64 [0,4194304] 0 2024-09-16T00:26:02.784 INFO:tasks.workunit.client.0.smithi112.stdout:4/583: getdents d6/db/d1f/d94 0 2024-09-16T00:26:02.784 INFO:tasks.workunit.client.1.smithi169.stdout:2/301: unlink d7/db/d27/l42 0 2024-09-16T00:26:02.785 INFO:tasks.workunit.client.0.smithi112.stdout:5/485: write d4/d7/d26/d36/f74 [765009,89903] 0 2024-09-16T00:26:02.785 INFO:tasks.workunit.client.0.smithi112.stdout:4/584: chown d6/db/df 7130 1 2024-09-16T00:26:02.787 INFO:tasks.workunit.client.0.smithi112.stdout:4/585: stat d6/db/df/d5e/d61/d64/d6a 0 2024-09-16T00:26:02.787 INFO:tasks.workunit.client.0.smithi112.stdout:6/511: mkdir dd/df/d11/d44/da4 0 2024-09-16T00:26:02.788 INFO:tasks.workunit.client.0.smithi112.stdout:2/688: read d2/dd/d48/f2a [1613024,70127] 0 2024-09-16T00:26:02.790 INFO:tasks.workunit.client.1.smithi169.stdout:6/501: dwrite d2/d3f/d41/d52/f89 [0,4194304] 0 2024-09-16T00:26:02.790 INFO:tasks.workunit.client.0.smithi112.stdout:4/586: write d6/db/df/d5e/f78 [1042103,65333] 0 2024-09-16T00:26:02.790 INFO:tasks.workunit.client.0.smithi112.stdout:5/486: rename d4/d7/d26/d4c/c9f to d4/d7/d14/d90/ca5 0 2024-09-16T00:26:02.793 INFO:tasks.workunit.client.0.smithi112.stdout:5/487: fdatasync d4/d7/d26/d4c/f97 0 2024-09-16T00:26:02.794 INFO:tasks.workunit.client.1.smithi169.stdout:9/433: dwrite d5/da/d21/d23/f94 [0,4194304] 0 2024-09-16T00:26:02.795 INFO:tasks.workunit.client.0.smithi112.stdout:5/488: write d4/dd/f6b [4014833,40422] 0 2024-09-16T00:26:02.795 INFO:tasks.workunit.client.1.smithi169.stdout:6/502: write d2/d4d/d59/d38/f5a [546285,80812] 0 2024-09-16T00:26:02.796 INFO:tasks.workunit.client.1.smithi169.stdout:9/434: fdatasync d5/da/d2e/f7b 0 2024-09-16T00:26:02.798 INFO:tasks.workunit.client.0.smithi112.stdout:4/587: symlink d6/db/df/d5e/d61/ld6 0 2024-09-16T00:26:02.798 INFO:tasks.workunit.client.1.smithi169.stdout:2/302: dwrite d7/db/f17 [0,4194304] 0 2024-09-16T00:26:02.799 INFO:tasks.workunit.client.1.smithi169.stdout:2/303: chown d7/db/d27/c4b 287 1 2024-09-16T00:26:02.800 INFO:tasks.workunit.client.1.smithi169.stdout:9/435: write d5/da/d4e/f6d [337281,9636] 0 2024-09-16T00:26:02.800 INFO:tasks.workunit.client.1.smithi169.stdout:0/418: sync 2024-09-16T00:26:02.801 INFO:tasks.workunit.client.1.smithi169.stdout:4/429: sync 2024-09-16T00:26:02.801 INFO:tasks.workunit.client.0.smithi112.stdout:1/616: dread d0/f48 [0,4194304] 0 2024-09-16T00:26:02.804 INFO:tasks.workunit.client.1.smithi169.stdout:4/430: read - d2/d12/d13/d27/d33/f71 zero size 2024-09-16T00:26:02.806 INFO:tasks.workunit.client.1.smithi169.stdout:2/304: creat d7/db/f60 x:0 0 0 2024-09-16T00:26:02.808 INFO:tasks.workunit.client.0.smithi112.stdout:6/512: rename dd/df/d17/d27/d84/d37 to dd/d15/da5 0 2024-09-16T00:26:02.808 INFO:tasks.workunit.client.0.smithi112.stdout:7/601: write d6/d10/f71 [1353376,114084] 0 2024-09-16T00:26:02.809 INFO:tasks.workunit.client.1.smithi169.stdout:8/367: rename d7/dc/c73 to d7/dc/d2a/c75 0 2024-09-16T00:26:02.810 INFO:tasks.workunit.client.0.smithi112.stdout:4/588: dwrite d6/db/df/d3c/f82 [0,4194304] 0 2024-09-16T00:26:02.811 INFO:tasks.workunit.client.1.smithi169.stdout:6/503: getdents d2/d4d/da0 0 2024-09-16T00:26:02.811 INFO:tasks.workunit.client.0.smithi112.stdout:2/689: rename d2/dd/d48/d22/d2b/d30/c47 to d2/dd/d48/d8a/d9b/cd9 0 2024-09-16T00:26:02.812 INFO:tasks.workunit.client.0.smithi112.stdout:1/617: truncate d0/d14/df/d28/d45/d30/f4a 13410545 0 2024-09-16T00:26:02.814 INFO:tasks.workunit.client.0.smithi112.stdout:5/489: link d4/d7/c10 d4/dd/d12/d52/d64/ca6 0 2024-09-16T00:26:02.815 INFO:tasks.workunit.client.0.smithi112.stdout:6/513: truncate dd/df/d10/f60 5043022 0 2024-09-16T00:26:02.816 INFO:tasks.workunit.client.1.smithi169.stdout:5/396: dwrite d2/da/d10/d13/f2f [0,4194304] 0 2024-09-16T00:26:02.818 INFO:tasks.workunit.client.0.smithi112.stdout:2/690: creat d2/dd/d96/fda x:0 0 0 2024-09-16T00:26:02.818 INFO:tasks.workunit.client.0.smithi112.stdout:1/618: mknod d0/d14/df/d28/d45/d46/da7/ccb 0 2024-09-16T00:26:02.820 INFO:tasks.workunit.client.0.smithi112.stdout:2/691: write d2/dd/d96/fab [953106,30632] 0 2024-09-16T00:26:02.820 INFO:tasks.workunit.client.0.smithi112.stdout:6/514: unlink dd/d15/d34/f71 0 2024-09-16T00:26:02.820 INFO:tasks.workunit.client.1.smithi169.stdout:3/441: truncate d7/d1a/d1c/d35/d63/f53 267313 0 2024-09-16T00:26:02.822 INFO:tasks.workunit.client.1.smithi169.stdout:0/419: write d0/d2/f32 [342256,55996] 0 2024-09-16T00:26:02.826 INFO:tasks.workunit.client.1.smithi169.stdout:8/368: dread d7/fd [4194304,4194304] 0 2024-09-16T00:26:02.827 INFO:tasks.workunit.client.0.smithi112.stdout:4/589: dwrite d6/f1b [0,4194304] 0 2024-09-16T00:26:02.827 INFO:tasks.workunit.client.1.smithi169.stdout:7/471: rename d7/d5b/f6c to d7/d5b/d61/d98/d99/f9b 0 2024-09-16T00:26:02.828 INFO:tasks.workunit.client.0.smithi112.stdout:0/655: dread d5/f2c [0,4194304] 0 2024-09-16T00:26:02.828 INFO:tasks.workunit.client.0.smithi112.stdout:6/515: readlink dd/df/d10/l14 0 2024-09-16T00:26:02.829 INFO:tasks.workunit.client.0.smithi112.stdout:1/619: mknod d0/d14/d6a/d9e/ccc 0 2024-09-16T00:26:02.829 INFO:tasks.workunit.client.1.smithi169.stdout:2/305: symlink d7/db/d27/d29/d37/l61 0 2024-09-16T00:26:02.830 INFO:tasks.workunit.client.1.smithi169.stdout:2/306: readlink d7/db/d27/l30 0 2024-09-16T00:26:02.830 INFO:tasks.workunit.client.1.smithi169.stdout:6/504: truncate d2/d4d/f49 1890125 0 2024-09-16T00:26:02.832 INFO:tasks.workunit.client.1.smithi169.stdout:5/397: unlink d2/da/d10/d13/d1c/d40/f69 0 2024-09-16T00:26:02.832 INFO:tasks.workunit.client.1.smithi169.stdout:3/442: rmdir d7/d1a/d1c/d23/d88 39 2024-09-16T00:26:02.837 INFO:tasks.workunit.client.1.smithi169.stdout:4/431: creat d2/d12/d13/d1f/f87 x:0 0 0 2024-09-16T00:26:02.837 INFO:tasks.workunit.client.0.smithi112.stdout:7/602: write d6/d10/d16/f66 [389006,101696] 0 2024-09-16T00:26:02.838 INFO:tasks.workunit.client.1.smithi169.stdout:1/415: rename dc/f6f to dc/d36/d39/d50/f7a 0 2024-09-16T00:26:02.839 INFO:tasks.workunit.client.0.smithi112.stdout:1/620: rename d0/d14/d1c/d20/d93/f97 to d0/d14/d1c/fcd 0 2024-09-16T00:26:02.840 INFO:tasks.workunit.client.1.smithi169.stdout:2/307: dwrite d7/d1c/f3e [0,4194304] 0 2024-09-16T00:26:02.841 INFO:tasks.workunit.client.0.smithi112.stdout:4/590: dwrite d6/d39/d1c/d66/f76 [0,4194304] 0 2024-09-16T00:26:02.843 INFO:tasks.workunit.client.0.smithi112.stdout:6/516: rename dd/df/d11/d44/l68 to dd/df/d11/d44/da4/la6 0 2024-09-16T00:26:02.844 INFO:tasks.workunit.client.1.smithi169.stdout:6/505: chown d2/d22/c4b 2459565 1 2024-09-16T00:26:02.845 INFO:tasks.workunit.client.1.smithi169.stdout:7/472: mkdir d7/d71/d76/d9c 0 2024-09-16T00:26:02.845 INFO:tasks.workunit.client.1.smithi169.stdout:5/398: creat d2/da/d10/d13/d1c/d32/d53/d83/d7d/f8b x:0 0 0 2024-09-16T00:26:02.847 INFO:tasks.workunit.client.0.smithi112.stdout:7/603: rename d6/d10/d16/d96/dc7/f81 to d6/d10/d16/d96/fcc 0 2024-09-16T00:26:02.847 INFO:tasks.workunit.client.1.smithi169.stdout:8/369: symlink d7/dc/l76 0 2024-09-16T00:26:02.850 INFO:tasks.workunit.client.1.smithi169.stdout:7/473: write d7/d71/d76/d29/f45 [567383,107635] 0 2024-09-16T00:26:02.850 INFO:tasks.workunit.client.1.smithi169.stdout:9/436: rename d5/d12/f5f to d5/da/d5b/d6c/f9d 0 2024-09-16T00:26:02.851 INFO:tasks.workunit.client.0.smithi112.stdout:7/604: write d6/d10/d75/d8f/d90/fb8 [951533,41198] 0 2024-09-16T00:26:02.851 INFO:tasks.workunit.client.0.smithi112.stdout:6/517: mknod dd/df/d17/d27/d84/ca7 0 2024-09-16T00:26:02.852 INFO:tasks.workunit.client.0.smithi112.stdout:6/518: read dd/d23/f64 [705841,2675] 0 2024-09-16T00:26:02.854 INFO:tasks.workunit.client.1.smithi169.stdout:3/443: dwrite d7/d1a/f32 [0,4194304] 0 2024-09-16T00:26:02.856 INFO:tasks.workunit.client.1.smithi169.stdout:7/474: write d7/d71/d76/d29/f85 [387275,6447] 0 2024-09-16T00:26:02.857 INFO:tasks.workunit.client.1.smithi169.stdout:2/308: truncate d7/db/d27/d4f/d56/f57 5092322 0 2024-09-16T00:26:02.857 INFO:tasks.workunit.client.1.smithi169.stdout:6/506: unlink d2/d3f/d41/d72/f84 0 2024-09-16T00:26:02.858 INFO:tasks.workunit.client.1.smithi169.stdout:0/420: rename d0/d2c/d54/d21/d31/d8c/l8d to d0/d4c/d82/l9a 0 2024-09-16T00:26:02.859 INFO:tasks.workunit.client.0.smithi112.stdout:4/591: rename d6/db/d1f/d41/c54 to d6/db/df/cd7 0 2024-09-16T00:26:02.861 INFO:tasks.workunit.client.0.smithi112.stdout:1/621: getdents d0/d14/df/d28/d45/d30 0 2024-09-16T00:26:02.862 INFO:tasks.workunit.client.0.smithi112.stdout:5/490: dwrite d4/d7/f35 [0,4194304] 0 2024-09-16T00:26:02.865 INFO:tasks.workunit.client.1.smithi169.stdout:8/370: mkdir d7/dc/d1d/d3c/d77 0 2024-09-16T00:26:02.867 INFO:tasks.workunit.client.0.smithi112.stdout:1/622: truncate d0/d14/df/d28/f5d 5064294 0 2024-09-16T00:26:02.868 INFO:tasks.workunit.client.1.smithi169.stdout:9/437: symlink d5/d49/l9e 0 2024-09-16T00:26:02.869 INFO:tasks.workunit.client.1.smithi169.stdout:2/309: dwrite d7/db/f3d [0,4194304] 0 2024-09-16T00:26:02.871 INFO:tasks.workunit.client.0.smithi112.stdout:4/592: fdatasync d6/d39/d1c/d2c/d69/d7b/fa8 0 2024-09-16T00:26:02.871 INFO:tasks.workunit.client.0.smithi112.stdout:1/623: unlink d0/d14/d1c/d26/fa2 0 2024-09-16T00:26:02.871 INFO:tasks.workunit.client.0.smithi112.stdout:5/491: dread d4/d7/d26/f44 [0,4194304] 0 2024-09-16T00:26:02.872 INFO:tasks.workunit.client.1.smithi169.stdout:1/416: write dc/d36/d3a/d45/f17 [386318,19106] 0 2024-09-16T00:26:02.874 INFO:tasks.workunit.client.0.smithi112.stdout:0/656: dread d5/da/d18/d24/faf [0,4194304] 0 2024-09-16T00:26:02.874 INFO:tasks.workunit.client.0.smithi112.stdout:5/492: dread - d4/dd/f95 zero size 2024-09-16T00:26:02.875 INFO:tasks.workunit.client.1.smithi169.stdout:1/417: read - dc/d36/d3a/f76 zero size 2024-09-16T00:26:02.875 INFO:tasks.workunit.client.1.smithi169.stdout:0/421: mkdir d0/d50/d6c/d9b 0 2024-09-16T00:26:02.875 INFO:tasks.workunit.client.0.smithi112.stdout:4/593: creat d6/d7c/fd8 x:0 0 0 2024-09-16T00:26:02.876 INFO:tasks.workunit.client.1.smithi169.stdout:6/507: mknod d2/d4c/d9b/ca3 0 2024-09-16T00:26:02.877 INFO:tasks.workunit.client.0.smithi112.stdout:5/493: symlink d4/dd/d12/d79/la7 0 2024-09-16T00:26:02.879 INFO:tasks.workunit.client.0.smithi112.stdout:5/494: stat d4/d7/d14/d90/f93 0 2024-09-16T00:26:02.880 INFO:tasks.workunit.client.0.smithi112.stdout:5/495: fsync d4/d4f/f76 0 2024-09-16T00:26:02.880 INFO:tasks.workunit.client.0.smithi112.stdout:4/594: dread - d6/d39/d36/d5c/d7e/d8f/f90 zero size 2024-09-16T00:26:02.881 INFO:tasks.workunit.client.1.smithi169.stdout:1/418: readlink dc/d18/l1b 0 2024-09-16T00:26:02.883 INFO:tasks.workunit.client.1.smithi169.stdout:7/475: dwrite d7/d16/d49/d84/f8d [0,4194304] 0 2024-09-16T00:26:02.885 INFO:tasks.workunit.client.1.smithi169.stdout:4/432: rename c1 to d2/c88 0 2024-09-16T00:26:02.886 INFO:tasks.workunit.client.1.smithi169.stdout:6/508: write d2/d22/f23 [7117918,12208] 0 2024-09-16T00:26:02.887 INFO:tasks.workunit.client.0.smithi112.stdout:0/657: getdents d5/da/d18/d24/d46/d5f/d7a/d8d 0 2024-09-16T00:26:02.887 INFO:tasks.workunit.client.0.smithi112.stdout:1/624: link d0/d14/df/d28/c77 d0/d14/df/d28/d45/d30/cce 0 2024-09-16T00:26:02.890 INFO:tasks.workunit.client.1.smithi169.stdout:2/310: readlink d7/d1c/d26/l3a 0 2024-09-16T00:26:02.891 INFO:tasks.workunit.client.1.smithi169.stdout:5/399: rename d2/da/d67/c77 to d2/da/d10/d13/d1c/d74/c8c 0 2024-09-16T00:26:02.891 INFO:tasks.workunit.client.0.smithi112.stdout:4/595: dwrite d6/db/d1f/d41/d49/f4c [0,4194304] 0 2024-09-16T00:26:02.892 INFO:tasks.workunit.client.1.smithi169.stdout:6/509: write d2/d3f/d41/d5c/f65 [4486286,24314] 0 2024-09-16T00:26:02.893 INFO:tasks.workunit.client.0.smithi112.stdout:7/605: truncate d6/f9 1919980 0 2024-09-16T00:26:02.893 INFO:tasks.workunit.client.0.smithi112.stdout:6/519: dread dd/df/d17/d27/d54/f56 [0,4194304] 0 2024-09-16T00:26:02.893 INFO:tasks.workunit.client.0.smithi112.stdout:0/658: creat d5/da/d18/d63/fcf x:0 0 0 2024-09-16T00:26:02.897 INFO:tasks.workunit.client.0.smithi112.stdout:0/659: chown d5/da/d18/d24/d5a/cc7 95 1 2024-09-16T00:26:02.897 INFO:tasks.workunit.client.0.smithi112.stdout:1/625: rename d0/d14/d44/l71 to d0/d14/d1c/d20/lcf 0 2024-09-16T00:26:02.897 INFO:tasks.workunit.client.1.smithi169.stdout:8/371: rename d7/dc/d17/d24/l59 to d7/l78 0 2024-09-16T00:26:02.897 INFO:tasks.workunit.client.1.smithi169.stdout:6/510: chown d2/d4d/d59/d43/l54 85151 1 2024-09-16T00:26:02.898 INFO:tasks.workunit.client.0.smithi112.stdout:6/520: creat dd/d23/d8f/fa8 x:0 0 0 2024-09-16T00:26:02.899 INFO:tasks.workunit.client.0.smithi112.stdout:1/626: write d0/d14/d6a/f6f [2185439,38328] 0 2024-09-16T00:26:02.900 INFO:tasks.workunit.client.0.smithi112.stdout:4/596: rename d6/d39/d1c/d6e/c85 to d6/db/df/d5e/d61/d64/d6a/db8/cd9 0 2024-09-16T00:26:02.900 INFO:tasks.workunit.client.0.smithi112.stdout:6/521: write dd/d23/f63 [2507843,126347] 0 2024-09-16T00:26:02.901 INFO:tasks.workunit.client.1.smithi169.stdout:1/419: mkdir dc/d18/d67/d69/d7b 0 2024-09-16T00:26:02.901 INFO:tasks.workunit.client.1.smithi169.stdout:3/444: dwrite d7/d1a/d2b/f61 [0,4194304] 0 2024-09-16T00:26:02.903 INFO:tasks.workunit.client.0.smithi112.stdout:4/597: truncate d6/d39/d36/d5c/d7e/f86 1839547 0 2024-09-16T00:26:02.904 INFO:tasks.workunit.client.1.smithi169.stdout:3/445: write d7/d1a/d1c/d35/d63/d37/f9a [3866750,95614] 0 2024-09-16T00:26:02.904 INFO:tasks.workunit.client.0.smithi112.stdout:0/660: symlink d5/da/d57/d76/ld0 0 2024-09-16T00:26:02.905 INFO:tasks.workunit.client.0.smithi112.stdout:1/627: read d0/d14/df/d28/d45/d30/f9b [1692182,92383] 0 2024-09-16T00:26:02.906 INFO:tasks.workunit.client.1.smithi169.stdout:4/433: creat d2/d12/d13/d1f/d3e/d5b/f89 x:0 0 0 2024-09-16T00:26:02.909 INFO:tasks.workunit.client.0.smithi112.stdout:7/606: rename d6/d10/d75/d88/dc2 to d6/d10/da1/dcd 0 2024-09-16T00:26:02.912 INFO:tasks.workunit.client.0.smithi112.stdout:4/598: mkdir d6/d39/d1c/d6e/d6f/dda 0 2024-09-16T00:26:02.914 INFO:tasks.workunit.client.0.smithi112.stdout:4/599: fdatasync d6/db/f93 0 2024-09-16T00:26:02.914 INFO:tasks.workunit.client.1.smithi169.stdout:6/511: mkdir d2/da4 0 2024-09-16T00:26:02.916 INFO:tasks.workunit.client.1.smithi169.stdout:3/446: dwrite d7/d1a/d1c/d23/f89 [0,4194304] 0 2024-09-16T00:26:02.917 INFO:tasks.workunit.client.1.smithi169.stdout:1/420: rename dc/d26/l35 to dc/d2c/l7c 0 2024-09-16T00:26:02.918 INFO:tasks.workunit.client.0.smithi112.stdout:7/607: dwrite d6/d10/d16/d96/dc7/f7d [0,4194304] 0 2024-09-16T00:26:02.919 INFO:tasks.workunit.client.1.smithi169.stdout:6/512: symlink d2/d36/la5 0 2024-09-16T00:26:02.919 INFO:tasks.workunit.client.0.smithi112.stdout:7/608: chown d6/d10/d16/f3a 1090012 1 2024-09-16T00:26:02.920 INFO:tasks.workunit.client.1.smithi169.stdout:1/421: chown dc/d36/d3a 574 1 2024-09-16T00:26:02.926 INFO:tasks.workunit.client.1.smithi169.stdout:2/311: dread f0 [0,4194304] 0 2024-09-16T00:26:02.926 INFO:tasks.workunit.client.1.smithi169.stdout:0/422: truncate d0/d2c/f39 393794 0 2024-09-16T00:26:02.928 INFO:tasks.workunit.client.0.smithi112.stdout:5/496: dread d4/f16 [0,4194304] 0 2024-09-16T00:26:02.928 INFO:tasks.workunit.client.0.smithi112.stdout:4/600: rename d6/d39/l3a to d6/d7c/ldb 0 2024-09-16T00:26:02.930 INFO:tasks.workunit.client.0.smithi112.stdout:7/609: creat d6/d10/d16/d49/fce x:0 0 0 2024-09-16T00:26:02.931 INFO:tasks.workunit.client.0.smithi112.stdout:4/601: truncate d6/d39/d1c/d66/f76 4363670 0 2024-09-16T00:26:02.931 INFO:tasks.workunit.client.1.smithi169.stdout:7/476: truncate d7/d5b/f77 2574095 0 2024-09-16T00:26:02.933 INFO:tasks.workunit.client.1.smithi169.stdout:1/422: write dc/d36/d39/d50/f7a [959822,11221] 0 2024-09-16T00:26:02.934 INFO:tasks.workunit.client.1.smithi169.stdout:0/423: dread - d0/d2/d15/f7a zero size 2024-09-16T00:26:02.934 INFO:tasks.workunit.client.1.smithi169.stdout:6/513: dwrite d2/f1f [0,4194304] 0 2024-09-16T00:26:02.936 INFO:tasks.workunit.client.1.smithi169.stdout:2/312: chown d7/db/d27/d29/d37/l2f 553912 1 2024-09-16T00:26:02.937 INFO:tasks.workunit.client.0.smithi112.stdout:4/602: dwrite d6/d39/d36/f72 [0,4194304] 0 2024-09-16T00:26:02.939 INFO:tasks.workunit.client.1.smithi169.stdout:3/447: rename d7/d1a/d1c/d35/l6e to d7/d1a/d1c/d6c/la8 0 2024-09-16T00:26:02.940 INFO:tasks.workunit.client.1.smithi169.stdout:8/372: mkdir d7/dc/d17/d24/d32/d4d/d79 0 2024-09-16T00:26:02.944 INFO:tasks.workunit.client.1.smithi169.stdout:1/423: fsync dc/d36/d3a/d45/d29/d2a/f38 0 2024-09-16T00:26:02.944 INFO:tasks.workunit.client.1.smithi169.stdout:9/438: dwrite d5/da/d2e/f62 [0,4194304] 0 2024-09-16T00:26:02.945 INFO:tasks.workunit.client.0.smithi112.stdout:6/522: getdents dd/d15/da5/d79 0 2024-09-16T00:26:02.947 INFO:tasks.workunit.client.0.smithi112.stdout:5/497: readlink d4/d1c/l3d 0 2024-09-16T00:26:02.947 INFO:tasks.workunit.client.0.smithi112.stdout:7/610: rmdir d6/d7f 39 2024-09-16T00:26:02.948 INFO:tasks.workunit.client.0.smithi112.stdout:5/498: chown d4/dd/d12/d79/c9d 1 1 2024-09-16T00:26:02.950 INFO:tasks.workunit.client.0.smithi112.stdout:7/611: write d6/d10/d16/d96/f9f [435520,73267] 0 2024-09-16T00:26:02.951 INFO:tasks.workunit.client.1.smithi169.stdout:0/424: creat d0/d2c/d54/d21/d31/d83/f9c x:0 0 0 2024-09-16T00:26:02.952 INFO:tasks.workunit.client.1.smithi169.stdout:7/477: link d7/d5b/f5c d7/d71/f9d 0 2024-09-16T00:26:02.952 INFO:tasks.workunit.client.1.smithi169.stdout:1/424: write dc/d36/d3a/f78 [672332,104691] 0 2024-09-16T00:26:02.953 INFO:tasks.workunit.client.0.smithi112.stdout:6/523: chown dd/df/d17/l81 49 1 2024-09-16T00:26:02.953 INFO:tasks.workunit.client.1.smithi169.stdout:8/373: symlink d7/dc/d26/d41/d4e/l7a 0 2024-09-16T00:26:02.954 INFO:tasks.workunit.client.0.smithi112.stdout:1/628: write d0/f67 [1769854,13479] 0 2024-09-16T00:26:02.956 INFO:tasks.workunit.client.0.smithi112.stdout:6/524: write dd/df/d17/f41 [1505999,12074] 0 2024-09-16T00:26:02.956 INFO:tasks.workunit.client.0.smithi112.stdout:4/603: getdents d6/d39/d36/d5c/dc6/db5 0 2024-09-16T00:26:02.957 INFO:tasks.workunit.client.1.smithi169.stdout:1/425: dread dc/f1d [0,4194304] 0 2024-09-16T00:26:02.957 INFO:tasks.workunit.client.1.smithi169.stdout:3/448: creat d7/d1a/d1c/d23/d88/fa9 x:0 0 0 2024-09-16T00:26:02.958 INFO:tasks.workunit.client.0.smithi112.stdout:7/612: dread d6/d10/d16/f66 [0,4194304] 0 2024-09-16T00:26:02.962 INFO:tasks.workunit.client.0.smithi112.stdout:7/613: truncate d6/d10/d16/d96/dc7/f61 645507 0 2024-09-16T00:26:02.965 INFO:tasks.workunit.client.0.smithi112.stdout:4/604: dwrite d6/d39/d36/f72 [4194304,4194304] 0 2024-09-16T00:26:02.965 INFO:tasks.workunit.client.0.smithi112.stdout:4/605: readlink d6/d39/d36/d5c/d7e/dd1/ld5 0 2024-09-16T00:26:02.967 INFO:tasks.workunit.client.1.smithi169.stdout:5/400: dread d2/da/d10/d13/f35 [0,4194304] 0 2024-09-16T00:26:02.967 INFO:tasks.workunit.client.1.smithi169.stdout:6/514: unlink d2/d36/la5 0 2024-09-16T00:26:02.968 INFO:tasks.workunit.client.1.smithi169.stdout:0/425: dwrite d0/d2/d8/f68 [0,4194304] 0 2024-09-16T00:26:02.970 INFO:tasks.workunit.client.0.smithi112.stdout:4/606: getdents d6/d7c 0 2024-09-16T00:26:02.974 INFO:tasks.workunit.client.0.smithi112.stdout:4/607: rmdir d6/d39/dc8 0 2024-09-16T00:26:02.975 INFO:tasks.workunit.client.1.smithi169.stdout:9/439: mknod d5/da/d21/d8d/c9f 0 2024-09-16T00:26:02.976 INFO:tasks.workunit.client.1.smithi169.stdout:8/374: creat d7/dc/d1d/d3c/d77/f7b x:0 0 0 2024-09-16T00:26:02.977 INFO:tasks.workunit.client.1.smithi169.stdout:3/449: creat d7/d50/d90/da6/faa x:0 0 0 2024-09-16T00:26:02.978 INFO:tasks.workunit.client.1.smithi169.stdout:6/515: write d2/d4d/f33 [67892,97463] 0 2024-09-16T00:26:02.981 INFO:tasks.workunit.client.1.smithi169.stdout:5/401: dread d2/da/d10/d13/d1c/d2d/f4f [0,4194304] 0 2024-09-16T00:26:02.982 INFO:tasks.workunit.client.1.smithi169.stdout:1/426: dwrite dc/d36/d39/d50/f7a [0,4194304] 0 2024-09-16T00:26:02.984 INFO:tasks.workunit.client.1.smithi169.stdout:9/440: write d5/da/d21/d23/f94 [4220066,29411] 0 2024-09-16T00:26:02.989 INFO:tasks.workunit.client.1.smithi169.stdout:8/375: write d7/dc/d1d/d3c/d77/f7b [173018,104668] 0 2024-09-16T00:26:02.990 INFO:tasks.workunit.client.1.smithi169.stdout:0/426: dwrite d0/d2/d8/d5/f2e [0,4194304] 0 2024-09-16T00:26:02.993 INFO:tasks.workunit.client.1.smithi169.stdout:5/402: truncate d2/da/d10/d13/d1c/f41 809450 0 2024-09-16T00:26:02.993 INFO:tasks.workunit.client.0.smithi112.stdout:4/608: truncate d6/db/d1f/d41/d49/db1/fc2 838238 0 2024-09-16T00:26:02.993 INFO:tasks.workunit.client.1.smithi169.stdout:1/427: creat dc/d18/d67/d69/f7d x:0 0 0 2024-09-16T00:26:02.993 INFO:tasks.workunit.client.0.smithi112.stdout:4/609: symlink d6/d7c/da0/ldc 0 2024-09-16T00:26:02.995 INFO:tasks.workunit.client.0.smithi112.stdout:4/610: mkdir d6/d39/d1c/d6e/d6f/ddd 0 2024-09-16T00:26:02.995 INFO:tasks.workunit.client.1.smithi169.stdout:0/427: creat d0/d50/d6c/f9d x:0 0 0 2024-09-16T00:26:02.996 INFO:tasks.workunit.client.0.smithi112.stdout:4/611: write d6/db/f42 [1906640,55706] 0 2024-09-16T00:26:02.999 INFO:tasks.workunit.client.0.smithi112.stdout:4/612: truncate d6/db/df/f59 2377726 0 2024-09-16T00:26:03.000 INFO:tasks.workunit.client.1.smithi169.stdout:5/403: stat d2/da/d10/d13/d1c/d32/d53/d83/l79 0 2024-09-16T00:26:03.000 INFO:tasks.workunit.client.0.smithi112.stdout:4/613: write d6/d39/d36/f8c [545897,57785] 0 2024-09-16T00:26:03.002 INFO:tasks.workunit.client.1.smithi169.stdout:8/376: read d7/dc/d17/d24/f4b [1696572,37752] 0 2024-09-16T00:26:03.003 INFO:tasks.workunit.client.0.smithi112.stdout:9/758: sync 2024-09-16T00:26:03.003 INFO:tasks.workunit.client.0.smithi112.stdout:3/579: sync 2024-09-16T00:26:03.003 INFO:tasks.workunit.client.0.smithi112.stdout:8/527: sync 2024-09-16T00:26:03.004 INFO:tasks.workunit.client.1.smithi169.stdout:6/516: mkdir d2/d3f/d41/d5c/d91/da6 0 2024-09-16T00:26:03.004 INFO:tasks.workunit.client.0.smithi112.stdout:4/614: mknod d6/d39/d36/d5c/dc6/db5/cde 0 2024-09-16T00:26:03.004 INFO:tasks.workunit.client.1.smithi169.stdout:6/517: stat d2/d3f/d41/d5c/d73 0 2024-09-16T00:26:03.006 INFO:tasks.workunit.client.0.smithi112.stdout:3/580: fsync d2/d9/d2b/f29 0 2024-09-16T00:26:03.008 INFO:tasks.workunit.client.1.smithi169.stdout:1/428: fdatasync dc/d36/d39/d50/f7a 0 2024-09-16T00:26:03.009 INFO:tasks.workunit.client.1.smithi169.stdout:6/518: chown d2/fb 41 1 2024-09-16T00:26:03.009 INFO:tasks.workunit.client.0.smithi112.stdout:6/525: dread dd/df/d10/f1a [0,4194304] 0 2024-09-16T00:26:03.009 INFO:tasks.workunit.client.1.smithi169.stdout:9/441: dwrite d5/da/d4e/f7e [0,4194304] 0 2024-09-16T00:26:03.010 INFO:tasks.workunit.client.0.smithi112.stdout:4/615: mkdir d6/db/d1f/d41/ddf 0 2024-09-16T00:26:03.012 INFO:tasks.workunit.client.0.smithi112.stdout:8/528: link d10/d12/d1c/d3b/d74/f7b d10/d4e/d8c/d7c/d97/fad 0 2024-09-16T00:26:03.012 INFO:tasks.workunit.client.1.smithi169.stdout:0/428: dwrite d0/f13 [0,4194304] 0 2024-09-16T00:26:03.012 INFO:tasks.workunit.client.1.smithi169.stdout:5/404: unlink d2/da/c1e 0 2024-09-16T00:26:03.014 INFO:tasks.workunit.client.0.smithi112.stdout:9/759: dwrite d1/d5/d2c/d6f/ff3 [0,4194304] 0 2024-09-16T00:26:03.014 INFO:tasks.workunit.client.0.smithi112.stdout:3/581: truncate d2/d9/d2b/f8f 792442 0 2024-09-16T00:26:03.016 INFO:tasks.workunit.client.1.smithi169.stdout:6/519: write d2/d4d/d59/d38/f62 [5334628,17410] 0 2024-09-16T00:26:03.016 INFO:tasks.workunit.client.1.smithi169.stdout:8/377: creat d7/dc/d26/d41/d4e/f7c x:0 0 0 2024-09-16T00:26:03.018 INFO:tasks.workunit.client.0.smithi112.stdout:8/529: symlink d10/d12/d1c/d96/lae 0 2024-09-16T00:26:03.022 INFO:tasks.workunit.client.0.smithi112.stdout:9/760: symlink d1/d5/d2c/d6f/dbf/l106 0 2024-09-16T00:26:03.023 INFO:tasks.workunit.client.0.smithi112.stdout:4/616: dwrite d6/db/df/d3c/f82 [0,4194304] 0 2024-09-16T00:26:03.024 INFO:tasks.workunit.client.1.smithi169.stdout:0/429: chown d0/d2/d8/f68 14189105 1 2024-09-16T00:26:03.026 INFO:tasks.workunit.client.1.smithi169.stdout:5/405: read d2/da/d10/d13/d1c/d40/d43/f4c [36726,23600] 0 2024-09-16T00:26:03.026 INFO:tasks.workunit.client.0.smithi112.stdout:8/530: write d10/d12/d1c/f31 [1544735,79542] 0 2024-09-16T00:26:03.027 INFO:tasks.workunit.client.1.smithi169.stdout:1/429: mknod dc/d2c/c7e 0 2024-09-16T00:26:03.029 INFO:tasks.workunit.client.0.smithi112.stdout:9/761: mkdir d1/d5/d6/d107 0 2024-09-16T00:26:03.030 INFO:tasks.workunit.client.1.smithi169.stdout:7/478: dread d7/f17 [0,4194304] 0 2024-09-16T00:26:03.032 INFO:tasks.workunit.client.1.smithi169.stdout:6/520: write d2/d36/f44 [529904,24785] 0 2024-09-16T00:26:03.033 INFO:tasks.workunit.client.1.smithi169.stdout:9/442: dwrite d5/da/d4e/f7e [0,4194304] 0 2024-09-16T00:26:03.034 INFO:tasks.workunit.client.0.smithi112.stdout:8/531: rename d10/d12/d24/l27 to d10/d4e/d8c/d7c/d97/laf 0 2024-09-16T00:26:03.035 INFO:tasks.workunit.client.1.smithi169.stdout:6/521: read - d2/d3f/d41/d5c/f94 zero size 2024-09-16T00:26:03.035 INFO:tasks.workunit.client.0.smithi112.stdout:8/532: stat d10/d12/d1c/l36 0 2024-09-16T00:26:03.037 INFO:tasks.workunit.client.0.smithi112.stdout:4/617: getdents d6/db/df/d3c 0 2024-09-16T00:26:03.038 INFO:tasks.workunit.client.1.smithi169.stdout:5/406: symlink d2/d7c/l8d 0 2024-09-16T00:26:03.038 INFO:tasks.workunit.client.0.smithi112.stdout:9/762: creat d1/f108 x:0 0 0 2024-09-16T00:26:03.039 INFO:tasks.workunit.client.0.smithi112.stdout:8/533: symlink d10/d12/d24/lb0 0 2024-09-16T00:26:03.042 INFO:tasks.workunit.client.0.smithi112.stdout:4/618: mkdir d6/d39/d1c/d2c/d69/d7b/de0 0 2024-09-16T00:26:03.043 INFO:tasks.workunit.client.0.smithi112.stdout:8/534: truncate d10/d12/d1c/fa4 825251 0 2024-09-16T00:26:03.043 INFO:tasks.workunit.client.1.smithi169.stdout:1/430: mkdir dc/d36/d3a/d7f 0 2024-09-16T00:26:03.044 INFO:tasks.workunit.client.0.smithi112.stdout:3/582: dwrite d2/d9/d2b/d47/f8a [0,4194304] 0 2024-09-16T00:26:03.044 INFO:tasks.workunit.client.0.smithi112.stdout:9/763: symlink d1/d5/d6/d2d/d6a/d89/l109 0 2024-09-16T00:26:03.045 INFO:tasks.workunit.client.0.smithi112.stdout:9/764: chown d1/d5/d6/d18 158 1 2024-09-16T00:26:03.046 INFO:tasks.workunit.client.1.smithi169.stdout:1/431: chown dc/d26/d5d/f6e 25 1 2024-09-16T00:26:03.048 INFO:tasks.workunit.client.0.smithi112.stdout:8/535: creat d10/d12/d1c/d3b/d7d/fb1 x:0 0 0 2024-09-16T00:26:03.049 INFO:tasks.workunit.client.0.smithi112.stdout:8/536: chown d10/d12/d1c/f54 339 1 2024-09-16T00:26:03.051 INFO:tasks.workunit.client.1.smithi169.stdout:9/443: dwrite d5/da/d73/f75 [0,4194304] 0 2024-09-16T00:26:03.052 INFO:tasks.workunit.client.0.smithi112.stdout:3/583: write d2/d9/d2b/d47/d52/d7a/d80/d83/d49/f4c [2842336,21069] 0 2024-09-16T00:26:03.052 INFO:tasks.workunit.client.0.smithi112.stdout:4/619: getdents d6/db/df 0 2024-09-16T00:26:03.055 INFO:tasks.workunit.client.1.smithi169.stdout:5/407: truncate d2/da/f28 2948549 0 2024-09-16T00:26:03.056 INFO:tasks.workunit.client.0.smithi112.stdout:9/765: dwrite d1/d5/d2c/d6f/f91 [0,4194304] 0 2024-09-16T00:26:03.056 INFO:tasks.workunit.client.1.smithi169.stdout:9/444: readlink d5/da/d21/d23/l54 0 2024-09-16T00:26:03.057 INFO:tasks.workunit.client.1.smithi169.stdout:1/432: readlink dc/d36/d3a/d45/l1a 0 2024-09-16T00:26:03.061 INFO:tasks.workunit.client.0.smithi112.stdout:9/766: creat d1/d5/d20/d81/d1a/d88/def/f10a x:0 0 0 2024-09-16T00:26:03.063 INFO:tasks.workunit.client.0.smithi112.stdout:4/620: getdents d6/d39/d1c/d6e/d6f 0 2024-09-16T00:26:03.066 INFO:tasks.workunit.client.1.smithi169.stdout:1/433: creat dc/d18/d67/d69/f80 x:0 0 0 2024-09-16T00:26:03.067 INFO:tasks.workunit.client.0.smithi112.stdout:9/767: link d1/d5/d2c/d6f/f8e d1/d5/d6/d107/f10b 0 2024-09-16T00:26:03.067 INFO:tasks.workunit.client.0.smithi112.stdout:4/621: mknod d6/db/df/d3c/ce1 0 2024-09-16T00:26:03.070 INFO:tasks.workunit.client.1.smithi169.stdout:1/434: mknod dc/d2c/d3d/c81 0 2024-09-16T00:26:03.070 INFO:tasks.workunit.client.1.smithi169.stdout:5/408: dwrite d2/da/d10/d13/d1c/f3b [0,4194304] 0 2024-09-16T00:26:03.071 INFO:tasks.workunit.client.0.smithi112.stdout:4/622: rename d6/cc7 to d6/d39/d1c/d6e/d6f/ddd/ce2 0 2024-09-16T00:26:03.074 INFO:tasks.workunit.client.1.smithi169.stdout:1/435: rmdir dc/d18 39 2024-09-16T00:26:03.075 INFO:tasks.workunit.client.1.smithi169.stdout:9/445: dwrite d5/da/d73/f9b [0,4194304] 0 2024-09-16T00:26:03.079 INFO:tasks.workunit.client.1.smithi169.stdout:9/446: write d5/da/d21/f30 [4120294,3110] 0 2024-09-16T00:26:03.082 INFO:tasks.workunit.client.1.smithi169.stdout:5/409: rmdir d2/da/d10/d13/d1c/d40/d43/d62 0 2024-09-16T00:26:03.084 INFO:tasks.workunit.client.1.smithi169.stdout:9/447: fsync d5/da/d21/d23/f40 0 2024-09-16T00:26:03.084 INFO:tasks.workunit.client.1.smithi169.stdout:8/378: dread d7/f10 [0,4194304] 0 2024-09-16T00:26:03.085 INFO:tasks.workunit.client.1.smithi169.stdout:5/410: fdatasync d2/fb 0 2024-09-16T00:26:03.087 INFO:tasks.workunit.client.1.smithi169.stdout:9/448: write d5/f14 [5322655,62753] 0 2024-09-16T00:26:03.088 INFO:tasks.workunit.client.0.smithi112.stdout:6/526: dread dd/df/d10/d47/f5b [0,4194304] 0 2024-09-16T00:26:03.089 INFO:tasks.workunit.client.1.smithi169.stdout:8/379: unlink d7/l16 0 2024-09-16T00:26:03.089 INFO:tasks.workunit.client.1.smithi169.stdout:0/430: write d0/d2/d15/f22 [1636599,99555] 0 2024-09-16T00:26:03.090 INFO:tasks.workunit.client.0.smithi112.stdout:6/527: creat dd/df/d13/d7c/fa9 x:0 0 0 2024-09-16T00:26:03.092 INFO:tasks.workunit.client.1.smithi169.stdout:5/411: symlink d2/da/d10/d13/d1c/d40/l8e 0 2024-09-16T00:26:03.092 INFO:tasks.workunit.client.0.smithi112.stdout:6/528: rename dd/d15/da5/l45 to dd/df/d13/d7c/laa 0 2024-09-16T00:26:03.094 INFO:tasks.workunit.client.1.smithi169.stdout:9/449: rename d5/d12/d61 to d5/da/d5b/d6c/da0 0 2024-09-16T00:26:03.095 INFO:tasks.workunit.client.0.smithi112.stdout:6/529: dread dd/df/d17/d27/d84/f36 [0,4194304] 0 2024-09-16T00:26:03.096 INFO:tasks.workunit.client.1.smithi169.stdout:9/450: readlink d5/d12/l1d 0 2024-09-16T00:26:03.096 INFO:tasks.workunit.client.1.smithi169.stdout:8/380: readlink d7/dc/d26/l33 0 2024-09-16T00:26:03.097 INFO:tasks.workunit.client.0.smithi112.stdout:6/530: getdents dd/df/d11/d44/d8a 0 2024-09-16T00:26:03.098 INFO:tasks.workunit.client.1.smithi169.stdout:8/381: creat d7/d20/f7d x:0 0 0 2024-09-16T00:26:03.098 INFO:tasks.workunit.client.1.smithi169.stdout:0/431: dwrite d0/d50/d6c/f9d [0,4194304] 0 2024-09-16T00:26:03.100 INFO:tasks.workunit.client.1.smithi169.stdout:5/412: getdents d2/d7c 0 2024-09-16T00:26:03.110 INFO:tasks.workunit.client.1.smithi169.stdout:0/432: write d0/d2/d8/d5/fa [4128553,41574] 0 2024-09-16T00:26:03.111 INFO:tasks.workunit.client.1.smithi169.stdout:6/522: truncate d2/f45 3786334 0 2024-09-16T00:26:03.111 INFO:tasks.workunit.client.0.smithi112.stdout:9/768: write d1/d5/d6/d18/f4b [4319277,47434] 0 2024-09-16T00:26:03.114 INFO:tasks.workunit.client.1.smithi169.stdout:9/451: dwrite d5/da/d2e/f7b [0,4194304] 0 2024-09-16T00:26:03.115 INFO:tasks.workunit.client.0.smithi112.stdout:3/584: dwrite d2/d9/d2b/d2c/f51 [0,4194304] 0 2024-09-16T00:26:03.115 INFO:tasks.workunit.client.0.smithi112.stdout:9/769: symlink d1/d5/d6/d59/l10c 0 2024-09-16T00:26:03.115 INFO:tasks.workunit.client.0.smithi112.stdout:4/623: dread d6/d39/d1c/f57 [0,4194304] 0 2024-09-16T00:26:03.116 INFO:tasks.workunit.client.0.smithi112.stdout:3/585: write d2/f2d [1303766,105239] 0 2024-09-16T00:26:03.118 INFO:tasks.workunit.client.0.smithi112.stdout:3/586: stat d2/f97 0 2024-09-16T00:26:03.119 INFO:tasks.workunit.client.0.smithi112.stdout:3/587: fsync d2/d9/d2b/d47/f8b 0 2024-09-16T00:26:03.119 INFO:tasks.workunit.client.1.smithi169.stdout:8/382: truncate d7/fd 560461 0 2024-09-16T00:26:03.120 INFO:tasks.workunit.client.0.smithi112.stdout:4/624: symlink d6/d39/d36/le3 0 2024-09-16T00:26:03.121 INFO:tasks.workunit.client.1.smithi169.stdout:9/452: creat d5/da/d5b/d6f/fa1 x:0 0 0 2024-09-16T00:26:03.123 INFO:tasks.workunit.client.1.smithi169.stdout:6/523: mkdir d2/d22/da7 0 2024-09-16T00:26:03.124 INFO:tasks.workunit.client.1.smithi169.stdout:7/479: dread d7/d16/d49/f68 [0,4194304] 0 2024-09-16T00:26:03.126 INFO:tasks.workunit.client.0.smithi112.stdout:3/588: creat d2/d9/d2b/d47/d78/fc6 x:0 0 0 2024-09-16T00:26:03.127 INFO:tasks.workunit.client.0.smithi112.stdout:9/770: getdents d1/d5/d6/d2d 0 2024-09-16T00:26:03.129 INFO:tasks.workunit.client.0.smithi112.stdout:4/625: rename d6/d39/d1c/d6e/d6f/ddd to d6/d39/d36/d5c/d7e/dab/de4 0 2024-09-16T00:26:03.130 INFO:tasks.workunit.client.0.smithi112.stdout:3/589: creat d2/d9/d2b/d47/d52/fc7 x:0 0 0 2024-09-16T00:26:03.130 INFO:tasks.workunit.client.0.smithi112.stdout:9/771: mknod d1/d5/d6/d18/c10d 0 2024-09-16T00:26:03.130 INFO:tasks.workunit.client.1.smithi169.stdout:5/413: dwrite d2/da/d10/f8a [0,4194304] 0 2024-09-16T00:26:03.131 INFO:tasks.workunit.client.0.smithi112.stdout:4/626: creat d6/d39/d1c/d6e/d6f/fe5 x:0 0 0 2024-09-16T00:26:03.133 INFO:tasks.workunit.client.0.smithi112.stdout:9/772: symlink d1/d5/d6/df8/dba/l10e 0 2024-09-16T00:26:03.134 INFO:tasks.workunit.client.1.smithi169.stdout:7/480: rmdir d7/df/d95 39 2024-09-16T00:26:03.135 INFO:tasks.workunit.client.1.smithi169.stdout:0/433: dwrite d0/d2/d8/f68 [4194304,4194304] 0 2024-09-16T00:26:03.135 INFO:tasks.workunit.client.0.smithi112.stdout:9/773: link d1/f62 d1/d5/d20/d81/d1a/d88/f10f 0 2024-09-16T00:26:03.139 INFO:tasks.workunit.client.0.smithi112.stdout:9/774: unlink d1/d5/d6/d2d/d39/c5d 0 2024-09-16T00:26:03.142 INFO:tasks.workunit.client.0.smithi112.stdout:9/775: mkdir d1/d5/d110 0 2024-09-16T00:26:03.145 INFO:tasks.workunit.client.1.smithi169.stdout:5/414: creat d2/da/d10/d13/d1c/d32/f8f x:0 0 0 2024-09-16T00:26:03.149 INFO:tasks.workunit.client.1.smithi169.stdout:6/524: dread d2/d4d/d59/d38/f62 [0,4194304] 0 2024-09-16T00:26:03.150 INFO:tasks.workunit.client.1.smithi169.stdout:8/383: dwrite d7/d20/f21 [0,4194304] 0 2024-09-16T00:26:03.150 INFO:tasks.workunit.client.1.smithi169.stdout:6/525: dread - d2/d22/fa1 zero size 2024-09-16T00:26:03.151 INFO:tasks.workunit.client.1.smithi169.stdout:9/453: dread d5/da/d4e/f6a [0,4194304] 0 2024-09-16T00:26:03.152 INFO:tasks.workunit.client.1.smithi169.stdout:6/526: chown d2/d22/d51 19 1 2024-09-16T00:26:03.153 INFO:tasks.workunit.client.1.smithi169.stdout:7/481: link d7/d5b/f6b d7/d71/d76/f9e 0 2024-09-16T00:26:03.156 INFO:tasks.workunit.client.1.smithi169.stdout:5/415: unlink d2/da/d10/d13/d1c/d32/d53/d4e/d55/f5c 0 2024-09-16T00:26:03.158 INFO:tasks.workunit.client.1.smithi169.stdout:7/482: truncate d7/df/f21 1175890 0 2024-09-16T00:26:03.159 INFO:tasks.workunit.client.1.smithi169.stdout:5/416: chown d2/da/d67/f82 2 1 2024-09-16T00:26:03.160 INFO:tasks.workunit.client.1.smithi169.stdout:0/434: dwrite d0/d2/d4b/f52 [0,4194304] 0 2024-09-16T00:26:03.161 INFO:tasks.workunit.client.1.smithi169.stdout:6/527: creat d2/d3f/d41/d5c/d73/fa8 x:0 0 0 2024-09-16T00:26:03.162 INFO:tasks.workunit.client.1.smithi169.stdout:9/454: mknod d5/ca2 0 2024-09-16T00:26:03.164 INFO:tasks.workunit.client.0.smithi112.stdout:4/627: dread d6/d39/d1c/f27 [0,4194304] 0 2024-09-16T00:26:03.166 INFO:tasks.workunit.client.0.smithi112.stdout:4/628: readlink d6/d39/d1c/d2c/d69/d7b/l96 0 2024-09-16T00:26:03.167 INFO:tasks.workunit.client.0.smithi112.stdout:4/629: chown d6/db/f18 928175342 1 2024-09-16T00:26:03.168 INFO:tasks.workunit.client.1.smithi169.stdout:0/435: truncate d0/d2/d8/d5/d60/f95 230533 0 2024-09-16T00:26:03.169 INFO:tasks.workunit.client.0.smithi112.stdout:4/630: symlink d6/db/df/le6 0 2024-09-16T00:26:03.169 INFO:tasks.workunit.client.0.smithi112.stdout:4/631: dread - d6/d39/d1c/d6e/fbe zero size 2024-09-16T00:26:03.171 INFO:tasks.workunit.client.1.smithi169.stdout:9/455: chown d5/ce 14256352 1 2024-09-16T00:26:03.171 INFO:tasks.workunit.client.1.smithi169.stdout:8/384: rmdir d7/dc/d17/d24/d32/d4d/d79 0 2024-09-16T00:26:03.172 INFO:tasks.workunit.client.0.smithi112.stdout:4/632: unlink d6/d39/d1c/d2c/fc9 0 2024-09-16T00:26:03.173 INFO:tasks.workunit.client.1.smithi169.stdout:8/385: chown d7/dc/d17 798942118 1 2024-09-16T00:26:03.174 INFO:tasks.workunit.client.0.smithi112.stdout:9/776: dread d1/d5/f8c [0,4194304] 0 2024-09-16T00:26:03.174 INFO:tasks.workunit.client.1.smithi169.stdout:6/528: creat d2/d3f/d41/d72/fa9 x:0 0 0 2024-09-16T00:26:03.174 INFO:tasks.workunit.client.1.smithi169.stdout:9/456: dread - d5/d12/f4c zero size 2024-09-16T00:26:03.175 INFO:tasks.workunit.client.1.smithi169.stdout:6/529: stat d2/d3f/d41/d52/f1b 0 2024-09-16T00:26:03.176 INFO:tasks.workunit.client.1.smithi169.stdout:7/483: rename d7/f4c to d7/f9f 0 2024-09-16T00:26:03.176 INFO:tasks.workunit.client.1.smithi169.stdout:7/484: chown d7/d71/d76/l5a 1 1 2024-09-16T00:26:03.177 INFO:tasks.workunit.client.1.smithi169.stdout:7/485: chown d7/f92 662743 1 2024-09-16T00:26:03.177 INFO:tasks.workunit.client.0.smithi112.stdout:4/633: dwrite d6/db/f60 [0,4194304] 0 2024-09-16T00:26:03.178 INFO:tasks.workunit.client.1.smithi169.stdout:8/386: creat d7/d20/f7e x:0 0 0 2024-09-16T00:26:03.182 INFO:tasks.workunit.client.1.smithi169.stdout:7/486: dread d7/d16/f25 [0,4194304] 0 2024-09-16T00:26:03.185 INFO:tasks.workunit.client.1.smithi169.stdout:4/434: sync 2024-09-16T00:26:03.193 INFO:tasks.workunit.client.1.smithi169.stdout:6/530: rmdir d2/d4d/d59/d43/d67/d87/da2 0 2024-09-16T00:26:03.196 INFO:tasks.workunit.client.1.smithi169.stdout:9/457: getdents d5/d12 0 2024-09-16T00:26:03.197 INFO:tasks.workunit.client.1.smithi169.stdout:9/458: fdatasync d5/da/d4e/f6d 0 2024-09-16T00:26:03.197 INFO:tasks.workunit.client.1.smithi169.stdout:6/531: getdents d2/d4d/da0 0 2024-09-16T00:26:03.199 INFO:tasks.workunit.client.1.smithi169.stdout:9/459: creat d5/da/d5b/d91/d9a/fa3 x:0 0 0 2024-09-16T00:26:03.201 INFO:tasks.workunit.client.1.smithi169.stdout:6/532: unlink d2/d4c/c8d 0 2024-09-16T00:26:03.201 INFO:tasks.workunit.client.1.smithi169.stdout:5/417: dread d2/da/d10/d13/d1c/d2d/f63 [0,4194304] 0 2024-09-16T00:26:03.204 INFO:tasks.workunit.client.1.smithi169.stdout:8/387: write d7/dc/d17/f1b [777518,2953] 0 2024-09-16T00:26:03.206 INFO:tasks.workunit.client.1.smithi169.stdout:6/533: rename d2/d4c/d9b to d2/da4/daa 0 2024-09-16T00:26:03.211 INFO:tasks.workunit.client.1.smithi169.stdout:4/435: dread d2/d12/d13/d27/f32 [0,4194304] 0 2024-09-16T00:26:03.211 INFO:tasks.workunit.client.1.smithi169.stdout:5/418: symlink d2/da/d10/d13/d1c/d32/d53/d4e/d55/l90 0 2024-09-16T00:26:03.216 INFO:tasks.workunit.client.1.smithi169.stdout:7/487: dwrite d7/d5b/f6b [0,4194304] 0 2024-09-16T00:26:03.220 INFO:tasks.workunit.client.1.smithi169.stdout:0/436: dread d0/d2/d8/fe [0,4194304] 0 2024-09-16T00:26:03.222 INFO:tasks.workunit.client.1.smithi169.stdout:0/437: chown d0/d2c 416 1 2024-09-16T00:26:03.222 INFO:tasks.workunit.client.1.smithi169.stdout:8/388: truncate d7/dc/d17/f66 510422 0 2024-09-16T00:26:03.224 INFO:tasks.workunit.client.1.smithi169.stdout:5/419: symlink d2/da/d10/d13/d1c/d2d/l91 0 2024-09-16T00:26:03.225 INFO:tasks.workunit.client.1.smithi169.stdout:2/313: sync 2024-09-16T00:26:03.225 INFO:tasks.workunit.client.1.smithi169.stdout:3/450: sync 2024-09-16T00:26:03.226 INFO:tasks.workunit.client.1.smithi169.stdout:0/438: readlink d0/d2c/d54/d56/l42 0 2024-09-16T00:26:03.227 INFO:tasks.workunit.client.0.smithi112.stdout:2/692: sync 2024-09-16T00:26:03.227 INFO:tasks.workunit.client.1.smithi169.stdout:0/439: chown d0/d4c/f87 7 1 2024-09-16T00:26:03.228 INFO:tasks.workunit.client.1.smithi169.stdout:7/488: unlink d7/d16/d49/f4d 0 2024-09-16T00:26:03.228 INFO:tasks.workunit.client.1.smithi169.stdout:6/534: dwrite d2/d4d/f8e [0,4194304] 0 2024-09-16T00:26:03.230 INFO:tasks.workunit.client.0.smithi112.stdout:4/634: rename d6/db/d1f/d41/d49/db1/fc2 to d6/d39/d36/d5c/fe7 0 2024-09-16T00:26:03.231 INFO:tasks.workunit.client.0.smithi112.stdout:9/777: dwrite d1/d5/d20/d81/d1a/d88/f10f [0,4194304] 0 2024-09-16T00:26:03.232 INFO:tasks.workunit.client.1.smithi169.stdout:9/460: dwrite d5/da/d5b/d91/d9a/fa3 [0,4194304] 0 2024-09-16T00:26:03.232 INFO:tasks.workunit.client.1.smithi169.stdout:0/440: write d0/d2c/d38/f66 [1687528,30634] 0 2024-09-16T00:26:03.232 INFO:tasks.workunit.client.1.smithi169.stdout:8/389: dread - d7/dc/d26/d41/f5c zero size 2024-09-16T00:26:03.236 INFO:tasks.workunit.client.0.smithi112.stdout:4/635: dwrite d6/d39/d1c/d6e/fb0 [0,4194304] 0 2024-09-16T00:26:03.239 INFO:tasks.workunit.client.1.smithi169.stdout:0/441: write d0/d50/f7f [75991,43439] 0 2024-09-16T00:26:03.241 INFO:tasks.workunit.client.1.smithi169.stdout:4/436: dwrite d2/d12/d13/d1f/f87 [0,4194304] 0 2024-09-16T00:26:03.246 INFO:tasks.workunit.client.1.smithi169.stdout:9/461: chown d5/da/d21/f45 172559 1 2024-09-16T00:26:03.246 INFO:tasks.workunit.client.1.smithi169.stdout:0/442: chown d0/d2/d8/d5/d60/l6b 1777039347 1 2024-09-16T00:26:03.247 INFO:tasks.workunit.client.1.smithi169.stdout:5/420: unlink d2/da/d10/d13/f2f 0 2024-09-16T00:26:03.251 INFO:tasks.workunit.client.1.smithi169.stdout:0/443: chown d0/d2/d8/d5/c69 2 1 2024-09-16T00:26:03.256 INFO:tasks.workunit.client.1.smithi169.stdout:7/489: dread d7/df/f21 [0,4194304] 0 2024-09-16T00:26:03.259 INFO:tasks.workunit.client.1.smithi169.stdout:0/444: creat d0/d2c/d54/d21/d31/f9e x:0 0 0 2024-09-16T00:26:03.261 INFO:tasks.workunit.client.1.smithi169.stdout:2/314: getdents d7/d1c/d3c 0 2024-09-16T00:26:03.261 INFO:tasks.workunit.client.0.smithi112.stdout:2/693: rmdir d2/d68/dd1 0 2024-09-16T00:26:03.261 INFO:tasks.workunit.client.1.smithi169.stdout:3/451: dwrite d7/d50/f87 [4194304,4194304] 0 2024-09-16T00:26:03.263 INFO:tasks.workunit.client.1.smithi169.stdout:7/490: unlink d7/d5b/d61/f89 0 2024-09-16T00:26:03.263 INFO:tasks.workunit.client.0.smithi112.stdout:2/694: chown d2/dd/d48/d22/d2b/d54/d60/d6c/ca5 229156552 1 2024-09-16T00:26:03.263 INFO:tasks.workunit.client.1.smithi169.stdout:4/437: getdents d2/d54 0 2024-09-16T00:26:03.263 INFO:tasks.workunit.client.0.smithi112.stdout:2/695: chown d2/dd/d48/d22/d67/c83 36168 1 2024-09-16T00:26:03.264 INFO:tasks.workunit.client.1.smithi169.stdout:9/462: getdents d5/da/d73 0 2024-09-16T00:26:03.264 INFO:tasks.workunit.client.1.smithi169.stdout:4/438: stat d2/d12/d13/d1f 0 2024-09-16T00:26:03.266 INFO:tasks.workunit.client.1.smithi169.stdout:0/445: creat d0/d50/f9f x:0 0 0 2024-09-16T00:26:03.269 INFO:tasks.workunit.client.1.smithi169.stdout:4/439: symlink d2/d12/d13/d4e/d50/l8a 0 2024-09-16T00:26:03.270 INFO:tasks.workunit.client.0.smithi112.stdout:9/778: dwrite d1/d5/d6/d2d/f92 [0,4194304] 0 2024-09-16T00:26:03.271 INFO:tasks.workunit.client.1.smithi169.stdout:3/452: creat d7/d1a/d2b/d8d/fab x:0 0 0 2024-09-16T00:26:03.274 INFO:tasks.workunit.client.0.smithi112.stdout:2/696: dwrite d2/dd/d48/d22/d67/fb2 [4194304,4194304] 0 2024-09-16T00:26:03.275 INFO:tasks.workunit.client.1.smithi169.stdout:0/446: truncate d0/d2/d4b/f88 97610 0 2024-09-16T00:26:03.275 INFO:tasks.workunit.client.1.smithi169.stdout:4/440: chown d2/d12/d13/d5c/f61 15100 1 2024-09-16T00:26:03.276 INFO:tasks.workunit.client.1.smithi169.stdout:3/453: unlink d7/d1a/d1c/l76 0 2024-09-16T00:26:03.276 INFO:tasks.workunit.client.0.smithi112.stdout:9/779: mknod d1/d5/d20/d81/d1a/d88/def/d66/c111 0 2024-09-16T00:26:03.277 INFO:tasks.workunit.client.0.smithi112.stdout:2/697: symlink d2/dd/d48/d8a/db9/ldb 0 2024-09-16T00:26:03.278 INFO:tasks.workunit.client.1.smithi169.stdout:7/491: creat d7/fa0 x:0 0 0 2024-09-16T00:26:03.281 INFO:tasks.workunit.client.0.smithi112.stdout:9/780: link d1/d5/d6/d18/f4b d1/d5/d6/d18/f112 0 2024-09-16T00:26:03.282 INFO:tasks.workunit.client.1.smithi169.stdout:6/535: dread d2/d4d/f49 [0,4194304] 0 2024-09-16T00:26:03.283 INFO:tasks.workunit.client.0.smithi112.stdout:9/781: truncate d1/d5/d6/d59/fda 967413 0 2024-09-16T00:26:03.283 INFO:tasks.workunit.client.1.smithi169.stdout:4/441: rmdir d2/d12/d13/d1f/d2f 39 2024-09-16T00:26:03.284 INFO:tasks.workunit.client.1.smithi169.stdout:8/390: write d7/dc/d17/d24/f4b [4143057,36320] 0 2024-09-16T00:26:03.285 INFO:tasks.workunit.client.0.smithi112.stdout:9/782: unlink d1/d5/d6/d18/f4b 0 2024-09-16T00:26:03.285 INFO:tasks.workunit.client.1.smithi169.stdout:6/536: chown d2/d4d/da0 188 1 2024-09-16T00:26:03.288 INFO:tasks.workunit.client.1.smithi169.stdout:9/463: dwrite d5/da/d2e/d47/f7d [0,4194304] 0 2024-09-16T00:26:03.294 INFO:tasks.workunit.client.1.smithi169.stdout:3/454: creat d7/d1a/d1c/d35/fac x:0 0 0 2024-09-16T00:26:03.294 INFO:tasks.workunit.client.1.smithi169.stdout:0/447: truncate d0/d2c/d54/d21/f8e 491114 0 2024-09-16T00:26:03.298 INFO:tasks.workunit.client.1.smithi169.stdout:3/455: truncate d7/f24 4862289 0 2024-09-16T00:26:03.298 INFO:tasks.workunit.client.1.smithi169.stdout:4/442: mknod d2/d12/c8b 0 2024-09-16T00:26:03.299 INFO:tasks.workunit.client.1.smithi169.stdout:9/464: mkdir d5/da/d21/d8d/da4 0 2024-09-16T00:26:03.300 INFO:tasks.workunit.client.1.smithi169.stdout:8/391: link d7/dc/d17/c55 d7/dc/d1d/d3c/c7f 0 2024-09-16T00:26:03.301 INFO:tasks.workunit.client.1.smithi169.stdout:7/492: dread d7/d16/f1d [0,4194304] 0 2024-09-16T00:26:03.302 INFO:tasks.workunit.client.1.smithi169.stdout:5/421: dread d2/da/f38 [0,4194304] 0 2024-09-16T00:26:03.302 INFO:tasks.workunit.client.0.smithi112.stdout:9/783: dread d1/d5/d6/d2d/d6a/d89/f90 [0,4194304] 0 2024-09-16T00:26:03.305 INFO:tasks.workunit.client.1.smithi169.stdout:0/448: creat d0/d2c/d54/d21/fa0 x:0 0 0 2024-09-16T00:26:03.305 INFO:tasks.workunit.client.0.smithi112.stdout:9/784: creat d1/d5/d6/d107/f113 x:0 0 0 2024-09-16T00:26:03.306 INFO:tasks.workunit.client.0.smithi112.stdout:9/785: chown d1/d5/d2c/d6f/ff3 4021960 1 2024-09-16T00:26:03.307 INFO:tasks.workunit.client.1.smithi169.stdout:3/456: truncate d7/d1a/d1c/f25 1205030 0 2024-09-16T00:26:03.308 INFO:tasks.workunit.client.1.smithi169.stdout:9/465: rmdir d5/da/d5b/d92 0 2024-09-16T00:26:03.309 INFO:tasks.workunit.client.0.smithi112.stdout:9/786: mkdir d1/d5/d6/d114 0 2024-09-16T00:26:03.313 INFO:tasks.workunit.client.0.smithi112.stdout:9/787: mknod d1/d5/d20/d81/d1a/d88/c115 0 2024-09-16T00:26:03.313 INFO:tasks.workunit.client.1.smithi169.stdout:7/493: truncate d7/d16/f25 919903 0 2024-09-16T00:26:03.314 INFO:tasks.workunit.client.1.smithi169.stdout:8/392: dread f5 [0,4194304] 0 2024-09-16T00:26:03.314 INFO:tasks.workunit.client.1.smithi169.stdout:6/537: dwrite d2/d4d/f33 [0,4194304] 0 2024-09-16T00:26:03.315 INFO:tasks.workunit.client.1.smithi169.stdout:2/315: dread d7/db/f19 [0,4194304] 0 2024-09-16T00:26:03.316 INFO:tasks.workunit.client.0.smithi112.stdout:2/698: dread d2/dd/d48/d22/d2b/f34 [0,4194304] 0 2024-09-16T00:26:03.317 INFO:tasks.workunit.client.1.smithi169.stdout:4/443: creat d2/d12/d13/d1f/d3e/f8c x:0 0 0 2024-09-16T00:26:03.318 INFO:tasks.workunit.client.1.smithi169.stdout:2/316: stat d7/db/f23 0 2024-09-16T00:26:03.319 INFO:tasks.workunit.client.1.smithi169.stdout:7/494: write d7/d71/d76/d29/f85 [802996,98064] 0 2024-09-16T00:26:03.319 INFO:tasks.workunit.client.0.smithi112.stdout:9/788: mkdir d1/d5/d6/d114/d116 0 2024-09-16T00:26:03.320 INFO:tasks.workunit.client.1.smithi169.stdout:0/449: rename d0/d2/f32 to d0/d2c/d54/d56/d67/fa1 0 2024-09-16T00:26:03.321 INFO:tasks.workunit.client.0.smithi112.stdout:2/699: write d2/dd/d48/d22/f29 [2750034,79664] 0 2024-09-16T00:26:03.325 INFO:tasks.workunit.client.1.smithi169.stdout:1/436: sync 2024-09-16T00:26:03.325 INFO:tasks.workunit.client.0.smithi112.stdout:2/700: creat d2/dd/d48/d8a/d9b/fdc x:0 0 0 2024-09-16T00:26:03.326 INFO:tasks.workunit.client.1.smithi169.stdout:8/393: mkdir d7/dc/d1d/d80 0 2024-09-16T00:26:03.327 INFO:tasks.workunit.client.1.smithi169.stdout:8/394: fdatasync d7/dc/d17/f1b 0 2024-09-16T00:26:03.332 INFO:tasks.workunit.client.1.smithi169.stdout:8/395: dread d7/f10 [0,4194304] 0 2024-09-16T00:26:03.337 INFO:tasks.workunit.client.1.smithi169.stdout:1/437: write dc/d26/d5d/d72/d73/f74 [490219,60748] 0 2024-09-16T00:26:03.338 INFO:tasks.workunit.client.1.smithi169.stdout:4/444: link d2/d12/f5d d2/d12/d13/d4e/f8d 0 2024-09-16T00:26:03.340 INFO:tasks.workunit.client.1.smithi169.stdout:8/396: mkdir d7/dc/d26/d41/d4e/d81 0 2024-09-16T00:26:03.344 INFO:tasks.workunit.client.1.smithi169.stdout:5/422: link d2/f6b d2/da/d10/d13/d1c/d32/d53/f92 0 2024-09-16T00:26:03.346 INFO:tasks.workunit.client.1.smithi169.stdout:3/457: getdents d7/d1a/d1c/d23/d70 0 2024-09-16T00:26:03.349 INFO:tasks.workunit.client.1.smithi169.stdout:9/466: dread d5/da/d21/d23/f40 [0,4194304] 0 2024-09-16T00:26:03.349 INFO:tasks.workunit.client.1.smithi169.stdout:8/397: mkdir d7/dc/d1d/d3c/d82 0 2024-09-16T00:26:03.350 INFO:tasks.workunit.client.1.smithi169.stdout:9/467: write d5/d12/f5d [257851,124936] 0 2024-09-16T00:26:03.352 INFO:tasks.workunit.client.1.smithi169.stdout:9/468: fsync d5/da/d2e/f62 0 2024-09-16T00:26:03.352 INFO:tasks.workunit.client.1.smithi169.stdout:1/438: mknod dc/d6c/c82 0 2024-09-16T00:26:03.352 INFO:tasks.workunit.client.1.smithi169.stdout:5/423: rmdir d2/da/d10/d13/d1c/d40/d43 39 2024-09-16T00:26:03.355 INFO:tasks.workunit.client.1.smithi169.stdout:9/469: rmdir d5/d49 39 2024-09-16T00:26:03.358 INFO:tasks.workunit.client.1.smithi169.stdout:3/458: mknod d7/d1a/d2b/cad 0 2024-09-16T00:26:03.359 INFO:tasks.workunit.client.1.smithi169.stdout:5/424: symlink d2/da/d10/d13/d24/l93 0 2024-09-16T00:26:03.360 INFO:tasks.workunit.client.1.smithi169.stdout:0/450: dread d0/d2c/d54/f61 [0,4194304] 0 2024-09-16T00:26:03.361 INFO:tasks.workunit.client.1.smithi169.stdout:8/398: symlink d7/dc/d17/d24/l83 0 2024-09-16T00:26:03.363 INFO:tasks.workunit.client.1.smithi169.stdout:9/470: fsync d5/da/d4e/f6a 0 2024-09-16T00:26:03.363 INFO:tasks.workunit.client.1.smithi169.stdout:2/317: truncate d7/db/f19 4146126 0 2024-09-16T00:26:03.366 INFO:tasks.workunit.client.1.smithi169.stdout:1/439: mkdir dc/d36/d3a/d45/d29/d2a/d83 0 2024-09-16T00:26:03.366 INFO:tasks.workunit.client.1.smithi169.stdout:7/495: truncate d7/df/f21 1104713 0 2024-09-16T00:26:03.366 INFO:tasks.workunit.client.1.smithi169.stdout:2/318: chown d7/db 76174 1 2024-09-16T00:26:03.367 INFO:tasks.workunit.client.0.smithi112.stdout:9/789: dwrite d1/d5/d2c/fc3 [0,4194304] 0 2024-09-16T00:26:03.368 INFO:tasks.workunit.client.0.smithi112.stdout:9/790: write d1/d5/d6/fa4 [395786,24292] 0 2024-09-16T00:26:03.371 INFO:tasks.workunit.client.0.smithi112.stdout:9/791: truncate d1/d5/f2f 765629 0 2024-09-16T00:26:03.371 INFO:tasks.workunit.client.1.smithi169.stdout:6/538: dwrite d2/d4c/f5d [0,4194304] 0 2024-09-16T00:26:03.373 INFO:tasks.workunit.client.1.smithi169.stdout:9/471: fsync d5/da/d2e/f7b 0 2024-09-16T00:26:03.374 INFO:tasks.workunit.client.0.smithi112.stdout:9/792: creat d1/d5/d20/d81/d1a/d88/def/de9/f117 x:0 0 0 2024-09-16T00:26:03.380 INFO:tasks.workunit.client.0.smithi112.stdout:9/793: link d1/d5/d20/d81/d1a/d88/def/d66/ld3 d1/d5/d6/d2d/d39/l118 0 2024-09-16T00:26:03.389 INFO:tasks.workunit.client.1.smithi169.stdout:4/445: dread d2/d12/d13/d4e/d50/f79 [0,4194304] 0 2024-09-16T00:26:03.390 INFO:tasks.workunit.client.1.smithi169.stdout:1/440: dread dc/f21 [0,4194304] 0 2024-09-16T00:26:03.393 INFO:tasks.workunit.client.1.smithi169.stdout:2/319: symlink d7/db/d27/d4f/l62 0 2024-09-16T00:26:03.397 INFO:tasks.workunit.client.1.smithi169.stdout:3/459: dwrite d7/d1a/d1c/d6c/f7c [4194304,4194304] 0 2024-09-16T00:26:03.399 INFO:tasks.workunit.client.1.smithi169.stdout:6/539: creat d2/d3f/d41/d72/d9e/fab x:0 0 0 2024-09-16T00:26:03.403 INFO:tasks.workunit.client.1.smithi169.stdout:2/320: stat d7/db/d20/d5c/l5e 0 2024-09-16T00:26:03.403 INFO:tasks.workunit.client.0.smithi112.stdout:0/661: sync 2024-09-16T00:26:03.405 INFO:tasks.workunit.client.1.smithi169.stdout:0/451: getdents d0/d2c/d54 0 2024-09-16T00:26:03.405 INFO:tasks.workunit.client.1.smithi169.stdout:9/472: dwrite d5/da/d2e/f62 [0,4194304] 0 2024-09-16T00:26:03.406 INFO:tasks.workunit.client.1.smithi169.stdout:8/399: rename d7/dc/d1d/d3c/l71 to d7/dc/d17/d24/d32/d4d/l84 0 2024-09-16T00:26:03.411 INFO:tasks.workunit.client.0.smithi112.stdout:0/662: rename d5/da/d18/d63/fad to d5/da/fd1 0 2024-09-16T00:26:03.412 INFO:tasks.workunit.client.1.smithi169.stdout:5/425: dwrite d2/da/d10/d13/f35 [8388608,4194304] 0 2024-09-16T00:26:03.413 INFO:tasks.workunit.client.0.smithi112.stdout:0/663: write d5/da/d57/d76/f14 [716875,34711] 0 2024-09-16T00:26:03.416 INFO:tasks.workunit.client.1.smithi169.stdout:4/446: creat d2/d12/d13/d1f/f8e x:0 0 0 2024-09-16T00:26:03.416 INFO:tasks.workunit.client.1.smithi169.stdout:8/400: write d7/dc/d26/d46/f5d [2752919,46685] 0 2024-09-16T00:26:03.420 INFO:tasks.workunit.client.0.smithi112.stdout:0/664: dread f0 [4194304,4194304] 0 2024-09-16T00:26:03.421 INFO:tasks.workunit.client.1.smithi169.stdout:7/496: dread d7/d16/d49/f58 [0,4194304] 0 2024-09-16T00:26:03.421 INFO:tasks.workunit.client.1.smithi169.stdout:0/452: truncate d0/d2/d4b/d86/f5a 2724457 0 2024-09-16T00:26:03.421 INFO:tasks.workunit.client.1.smithi169.stdout:9/473: symlink d5/da/d5b/d91/la5 0 2024-09-16T00:26:03.422 INFO:tasks.workunit.client.1.smithi169.stdout:1/441: dread dc/f34 [0,4194304] 0 2024-09-16T00:26:03.423 INFO:tasks.workunit.client.0.smithi112.stdout:0/665: getdents d5/da/d21/da5 0 2024-09-16T00:26:03.424 INFO:tasks.workunit.client.1.smithi169.stdout:1/442: read - dc/d36/d3a/f76 zero size 2024-09-16T00:26:03.429 INFO:tasks.workunit.client.1.smithi169.stdout:7/497: unlink d7/f92 0 2024-09-16T00:26:03.430 INFO:tasks.workunit.client.0.smithi112.stdout:0/666: dwrite d5/da/dc/f81 [0,4194304] 0 2024-09-16T00:26:03.431 INFO:tasks.workunit.client.0.smithi112.stdout:0/667: dread - d5/da/d18/d24/d5a/fbf zero size 2024-09-16T00:26:03.431 INFO:tasks.workunit.client.1.smithi169.stdout:3/460: rename d7/d1a/d1c/d35/d63/c56 to d7/d1a/d2b/cae 0 2024-09-16T00:26:03.436 INFO:tasks.workunit.client.1.smithi169.stdout:4/447: creat d2/d12/d13/f8f x:0 0 0 2024-09-16T00:26:03.436 INFO:tasks.workunit.client.0.smithi112.stdout:0/668: getdents d5/da/d18/d24/d46/d5f/da6 0 2024-09-16T00:26:03.439 INFO:tasks.workunit.client.0.smithi112.stdout:0/669: getdents d5 0 2024-09-16T00:26:03.441 INFO:tasks.workunit.client.1.smithi169.stdout:9/474: rename d5/ca2 to d5/da/d2e/d99/ca6 0 2024-09-16T00:26:03.441 INFO:tasks.workunit.client.0.smithi112.stdout:0/670: write d5/da/d18/d24/d46/d88/f4a [3434298,9505] 0 2024-09-16T00:26:03.445 INFO:tasks.workunit.client.1.smithi169.stdout:6/540: truncate d2/d4c/f5d 270605 0 2024-09-16T00:26:03.447 INFO:tasks.workunit.client.1.smithi169.stdout:2/321: dwrite d7/db/f19 [0,4194304] 0 2024-09-16T00:26:03.448 INFO:tasks.workunit.client.1.smithi169.stdout:1/443: creat dc/d36/f84 x:0 0 0 2024-09-16T00:26:03.448 INFO:tasks.workunit.client.1.smithi169.stdout:3/461: symlink d7/d1a/d1c/d23/laf 0 2024-09-16T00:26:03.448 INFO:tasks.workunit.client.1.smithi169.stdout:4/448: rmdir d2/d12/d13/d1f/d2f 39 2024-09-16T00:26:03.449 INFO:tasks.workunit.client.1.smithi169.stdout:7/498: dwrite d7/f26 [0,4194304] 0 2024-09-16T00:26:03.451 INFO:tasks.workunit.client.1.smithi169.stdout:9/475: creat d5/da/d5b/d6c/da0/fa7 x:0 0 0 2024-09-16T00:26:03.452 INFO:tasks.workunit.client.0.smithi112.stdout:9/794: mknod d1/d5/c119 0 2024-09-16T00:26:03.454 INFO:tasks.workunit.client.1.smithi169.stdout:2/322: write d7/db/f60 [437411,23070] 0 2024-09-16T00:26:03.455 INFO:tasks.workunit.client.1.smithi169.stdout:6/541: mkdir d2/d3f/d41/d5c/dac 0 2024-09-16T00:26:03.456 INFO:tasks.workunit.client.1.smithi169.stdout:3/462: symlink d7/d50/d90/lb0 0 2024-09-16T00:26:03.458 INFO:tasks.workunit.client.1.smithi169.stdout:7/499: creat d7/d71/d76/d29/fa1 x:0 0 0 2024-09-16T00:26:03.458 INFO:tasks.workunit.client.1.smithi169.stdout:3/463: chown d7/d50/d90/da6/faa 30765 1 2024-09-16T00:26:03.459 INFO:tasks.workunit.client.1.smithi169.stdout:4/449: fdatasync d2/d12/d13/d4e/f77 0 2024-09-16T00:26:03.460 INFO:tasks.workunit.client.1.smithi169.stdout:2/323: creat d7/db/d27/d29/d37/f63 x:0 0 0 2024-09-16T00:26:03.461 INFO:tasks.workunit.client.1.smithi169.stdout:9/476: symlink d5/la8 0 2024-09-16T00:26:03.464 INFO:tasks.workunit.client.1.smithi169.stdout:6/542: creat d2/da4/daa/fad x:0 0 0 2024-09-16T00:26:03.467 INFO:tasks.workunit.client.1.smithi169.stdout:7/500: mknod d7/ca2 0 2024-09-16T00:26:03.468 INFO:tasks.workunit.client.1.smithi169.stdout:3/464: mknod d7/d1a/d1c/d35/cb1 0 2024-09-16T00:26:03.470 INFO:tasks.workunit.client.1.smithi169.stdout:2/324: dwrite d7/db/d27/d4f/f53 [0,4194304] 0 2024-09-16T00:26:03.471 INFO:tasks.workunit.client.1.smithi169.stdout:1/444: getdents dc/d36/d3a 0 2024-09-16T00:26:03.472 INFO:tasks.workunit.client.1.smithi169.stdout:8/401: dread d7/dc/d17/f28 [0,4194304] 0 2024-09-16T00:26:03.473 INFO:tasks.workunit.client.1.smithi169.stdout:6/543: chown d2/d36/d48/f7e 0 1 2024-09-16T00:26:03.474 INFO:tasks.workunit.client.1.smithi169.stdout:7/501: fdatasync d7/f9f 0 2024-09-16T00:26:03.476 INFO:tasks.workunit.client.1.smithi169.stdout:1/445: creat dc/d36/d39/d50/f85 x:0 0 0 2024-09-16T00:26:03.481 INFO:tasks.workunit.client.1.smithi169.stdout:6/544: rmdir d2/d4d/d59 39 2024-09-16T00:26:03.483 INFO:tasks.workunit.client.1.smithi169.stdout:7/502: chown d7/d16/c4f 6321224 1 2024-09-16T00:26:03.485 INFO:tasks.workunit.client.1.smithi169.stdout:2/325: rename d7/db/cd to d7/db/d27/d29/c64 0 2024-09-16T00:26:03.485 INFO:tasks.workunit.client.1.smithi169.stdout:8/402: dwrite d7/dc/d1d/f5f [0,4194304] 0 2024-09-16T00:26:03.491 INFO:tasks.workunit.client.1.smithi169.stdout:9/477: write d5/da/f4d [652302,93973] 0 2024-09-16T00:26:03.494 INFO:tasks.workunit.client.1.smithi169.stdout:1/446: rename dc/d36/d4e to dc/d2c/d3d/d86 0 2024-09-16T00:26:03.495 INFO:tasks.workunit.client.1.smithi169.stdout:8/403: creat d7/dc/d2a/f85 x:0 0 0 2024-09-16T00:26:03.497 INFO:tasks.workunit.client.1.smithi169.stdout:2/326: symlink d7/l65 0 2024-09-16T00:26:03.498 INFO:tasks.workunit.client.1.smithi169.stdout:2/327: stat d7/db/d20 0 2024-09-16T00:26:03.499 INFO:tasks.workunit.client.1.smithi169.stdout:9/478: write d5/da/d4e/f89 [128532,82397] 0 2024-09-16T00:26:03.501 INFO:tasks.workunit.client.1.smithi169.stdout:1/447: mknod dc/d36/d3a/d45/d29/d4d/c87 0 2024-09-16T00:26:03.501 INFO:tasks.workunit.client.1.smithi169.stdout:3/465: truncate d7/d1a/d1c/d35/d63/f6a 3906129 0 2024-09-16T00:26:03.503 INFO:tasks.workunit.client.1.smithi169.stdout:3/466: dread - d7/d50/f97 zero size 2024-09-16T00:26:03.504 INFO:tasks.workunit.client.1.smithi169.stdout:2/328: mknod d7/db/d27/d29/d37/c66 0 2024-09-16T00:26:03.505 INFO:tasks.workunit.client.1.smithi169.stdout:2/329: chown d7/db/d27/d4f/l62 167220983 1 2024-09-16T00:26:03.506 INFO:tasks.workunit.client.1.smithi169.stdout:6/545: creat d2/d4d/d59/d43/d67/fae x:0 0 0 2024-09-16T00:26:03.507 INFO:tasks.workunit.client.1.smithi169.stdout:9/479: dwrite d5/da/d21/d23/f3b [0,4194304] 0 2024-09-16T00:26:03.508 INFO:tasks.workunit.client.1.smithi169.stdout:1/448: mkdir dc/d2c/d3d/d88 0 2024-09-16T00:26:03.509 INFO:tasks.workunit.client.1.smithi169.stdout:3/467: fsync d7/d1a/d2b/f8c 0 2024-09-16T00:26:03.509 INFO:tasks.workunit.client.1.smithi169.stdout:2/330: unlink d7/db/d27/d4f/l62 0 2024-09-16T00:26:03.510 INFO:tasks.workunit.client.1.smithi169.stdout:1/449: dread f7 [0,4194304] 0 2024-09-16T00:26:03.510 INFO:tasks.workunit.client.1.smithi169.stdout:8/404: creat d7/dc/d17/f86 x:0 0 0 2024-09-16T00:26:03.511 INFO:tasks.workunit.client.1.smithi169.stdout:8/405: chown d7/dc/c23 7365260 1 2024-09-16T00:26:03.512 INFO:tasks.workunit.client.1.smithi169.stdout:1/450: read dc/d26/f46 [2255876,79845] 0 2024-09-16T00:26:03.513 INFO:tasks.workunit.client.1.smithi169.stdout:3/468: chown d7/d1a/d2b/l9d 2038 1 2024-09-16T00:26:03.516 INFO:tasks.workunit.client.1.smithi169.stdout:6/546: rename d2/d4d/d59/d74/l92 to d2/d3f/laf 0 2024-09-16T00:26:03.517 INFO:tasks.workunit.client.1.smithi169.stdout:8/406: mkdir d7/d87 0 2024-09-16T00:26:03.520 INFO:tasks.workunit.client.1.smithi169.stdout:2/331: mknod d7/db/d20/c67 0 2024-09-16T00:26:03.521 INFO:tasks.workunit.client.1.smithi169.stdout:2/332: chown d7/db/d27/d29 8 1 2024-09-16T00:26:03.523 INFO:tasks.workunit.client.1.smithi169.stdout:3/469: mknod d7/d1a/d1c/d6c/cb2 0 2024-09-16T00:26:03.526 INFO:tasks.workunit.client.0.smithi112.stdout:0/671: dread d5/da/fd1 [0,4194304] 0 2024-09-16T00:26:03.526 INFO:tasks.workunit.client.1.smithi169.stdout:6/547: rmdir d2/da4/daa 39 2024-09-16T00:26:03.527 INFO:tasks.workunit.client.0.smithi112.stdout:9/795: dread d1/d5/d20/d81/d1a/d88/da8/fc8 [0,4194304] 0 2024-09-16T00:26:03.530 INFO:tasks.workunit.client.1.smithi169.stdout:9/480: getdents d5/da/d21/d23 0 2024-09-16T00:26:03.531 INFO:tasks.workunit.client.0.smithi112.stdout:9/796: dread d1/d5/d20/d81/d1a/d88/def/d66/faf [0,4194304] 0 2024-09-16T00:26:03.531 INFO:tasks.workunit.client.1.smithi169.stdout:9/481: fdatasync d5/d12/f5d 0 2024-09-16T00:26:03.533 INFO:tasks.workunit.client.0.smithi112.stdout:0/672: dwrite d5/da/fd1 [0,4194304] 0 2024-09-16T00:26:03.534 INFO:tasks.workunit.client.0.smithi112.stdout:0/673: dread - d5/da/d1d/d71/d64/db6/fc1 zero size 2024-09-16T00:26:03.534 INFO:tasks.workunit.client.0.smithi112.stdout:9/797: creat d1/d5/d6/df8/ddd/f11a x:0 0 0 2024-09-16T00:26:03.536 INFO:tasks.workunit.client.0.smithi112.stdout:0/674: truncate d5/da/d1d/d71/d64/db6/fc1 252098 0 2024-09-16T00:26:03.537 INFO:tasks.workunit.client.0.smithi112.stdout:9/798: mkdir d1/d5/d20/d81/d1a/d88/def/d11b 0 2024-09-16T00:26:03.538 INFO:tasks.workunit.client.0.smithi112.stdout:0/675: write d5/da/d18/f54 [1017796,63642] 0 2024-09-16T00:26:03.538 INFO:tasks.workunit.client.0.smithi112.stdout:9/799: readlink d1/d5/d2c/d6f/dbf/l106 0 2024-09-16T00:26:03.542 INFO:tasks.workunit.client.0.smithi112.stdout:0/676: link d5/f33 d5/da/d1d/d71/d64/db6/fd2 0 2024-09-16T00:26:03.542 INFO:tasks.workunit.client.1.smithi169.stdout:1/451: dread dc/d18/d67/f49 [0,4194304] 0 2024-09-16T00:26:03.543 INFO:tasks.workunit.client.1.smithi169.stdout:1/452: rename dc/d36/d3a to dc/d36/d3a/d45/d29/d2a/d89 22 2024-09-16T00:26:03.545 INFO:tasks.workunit.client.0.smithi112.stdout:0/677: rename d5/da/dc/d4e to d5/da/d21/dc3/dd3 0 2024-09-16T00:26:03.549 INFO:tasks.workunit.client.0.smithi112.stdout:0/678: dread d5/da/d57/d76/f97 [0,4194304] 0 2024-09-16T00:26:03.551 INFO:tasks.workunit.client.1.smithi169.stdout:6/548: mknod d2/d4c/d9f/cb0 0 2024-09-16T00:26:03.554 INFO:tasks.workunit.client.1.smithi169.stdout:8/407: creat d7/dc/d17/d24/f88 x:0 0 0 2024-09-16T00:26:03.555 INFO:tasks.workunit.client.1.smithi169.stdout:2/333: link d7/db/d27/d29/l59 d7/d1c/d3c/l68 0 2024-09-16T00:26:03.555 INFO:tasks.workunit.client.0.smithi112.stdout:0/679: fsync d5/da/d18/d24/d46/d88/f4a 0 2024-09-16T00:26:03.556 INFO:tasks.workunit.client.1.smithi169.stdout:3/470: creat d7/d1a/d2b/fb3 x:0 0 0 2024-09-16T00:26:03.557 INFO:tasks.workunit.client.1.smithi169.stdout:1/453: mknod dc/d36/d3a/c8a 0 2024-09-16T00:26:03.558 INFO:tasks.workunit.client.0.smithi112.stdout:0/680: unlink d5/da/d57/d76/f97 0 2024-09-16T00:26:03.558 INFO:tasks.workunit.client.1.smithi169.stdout:2/334: rmdir d7/d1c/d26 39 2024-09-16T00:26:03.559 INFO:tasks.workunit.client.0.smithi112.stdout:0/681: write f0 [8491287,64764] 0 2024-09-16T00:26:03.559 INFO:tasks.workunit.client.1.smithi169.stdout:2/335: write f5 [7714527,123435] 0 2024-09-16T00:26:03.559 INFO:tasks.workunit.client.1.smithi169.stdout:1/454: write dc/d36/d3a/d45/d29/f52 [170221,80302] 0 2024-09-16T00:26:03.566 INFO:tasks.workunit.client.1.smithi169.stdout:6/549: rename d2/d4d/da0 to d2/da4/db1 0 2024-09-16T00:26:03.566 INFO:tasks.workunit.client.1.smithi169.stdout:8/408: creat d7/dc/d17/f89 x:0 0 0 2024-09-16T00:26:03.567 INFO:tasks.workunit.client.1.smithi169.stdout:2/336: write d7/db/d27/d4f/d56/f57 [2137485,99465] 0 2024-09-16T00:26:03.575 INFO:tasks.workunit.client.0.smithi112.stdout:9/800: dread d1/d5/d20/d81/d1a/f7d [0,4194304] 0 2024-09-16T00:26:03.575 INFO:tasks.workunit.client.1.smithi169.stdout:3/471: truncate d7/d1a/d1c/d23/f4c 5108526 0 2024-09-16T00:26:03.576 INFO:tasks.workunit.client.0.smithi112.stdout:9/801: mknod d1/d5/d6/d2d/d6a/c11c 0 2024-09-16T00:26:03.577 INFO:tasks.workunit.client.0.smithi112.stdout:9/802: fsync d1/d5/d83/fca 0 2024-09-16T00:26:03.578 INFO:tasks.workunit.client.1.smithi169.stdout:8/409: symlink d7/dc/d1d/d3c/d77/l8a 0 2024-09-16T00:26:03.578 INFO:tasks.workunit.client.1.smithi169.stdout:6/550: fdatasync d2/d3f/d41/d5c/f88 0 2024-09-16T00:26:03.581 INFO:tasks.workunit.client.1.smithi169.stdout:1/455: truncate dc/d18/f59 504287 0 2024-09-16T00:26:03.583 INFO:tasks.workunit.client.1.smithi169.stdout:8/410: rmdir d7/dc/d26/d41/d67 39 2024-09-16T00:26:03.585 INFO:tasks.workunit.client.1.smithi169.stdout:6/551: dread d2/d3f/d41/d72/f9d [0,4194304] 0 2024-09-16T00:26:03.586 INFO:tasks.workunit.client.1.smithi169.stdout:6/552: readlink d2/d4d/d59/d43/l8c 0 2024-09-16T00:26:03.587 INFO:tasks.workunit.client.1.smithi169.stdout:2/337: getdents d7/db/d27/d29/d37 0 2024-09-16T00:26:03.587 INFO:tasks.workunit.client.0.smithi112.stdout:9/803: dwrite d1/d5/d6/d2d/d39/f65 [4194304,4194304] 0 2024-09-16T00:26:03.590 INFO:tasks.workunit.client.0.smithi112.stdout:9/804: mkdir d1/d5/d20/d81/d11d 0 2024-09-16T00:26:03.591 INFO:tasks.workunit.client.0.smithi112.stdout:9/805: read d1/f41 [2902363,80404] 0 2024-09-16T00:26:03.591 INFO:tasks.workunit.client.1.smithi169.stdout:8/411: link d7/dc/d2a/f85 d7/dc/d1d/d80/f8b 0 2024-09-16T00:26:03.592 INFO:tasks.workunit.client.0.smithi112.stdout:9/806: chown d1/d5/d6/d114/d116 0 1 2024-09-16T00:26:03.592 INFO:tasks.workunit.client.0.smithi112.stdout:9/807: dread - d1/d5/d6/d38/fdf zero size 2024-09-16T00:26:03.593 INFO:tasks.workunit.client.1.smithi169.stdout:1/456: dwrite dc/d36/d3a/d45/d29/d2a/f44 [0,4194304] 0 2024-09-16T00:26:03.594 INFO:tasks.workunit.client.0.smithi112.stdout:9/808: mknod d1/d5/d20/d81/d11d/c11e 0 2024-09-16T00:26:03.594 INFO:tasks.workunit.client.1.smithi169.stdout:6/553: unlink d2/l16 0 2024-09-16T00:26:03.595 INFO:tasks.workunit.client.0.smithi112.stdout:9/809: mkdir d1/d5/d83/d11f 0 2024-09-16T00:26:03.596 INFO:tasks.workunit.client.1.smithi169.stdout:6/554: fdatasync d2/d3f/d41/f80 0 2024-09-16T00:26:03.596 INFO:tasks.workunit.client.0.smithi112.stdout:9/810: write d1/d5/d6/d2d/d6a/fe6 [896502,92016] 0 2024-09-16T00:26:03.599 INFO:tasks.workunit.client.1.smithi169.stdout:2/338: mkdir d7/db/d27/d29/d46/d69 0 2024-09-16T00:26:03.601 INFO:tasks.workunit.client.1.smithi169.stdout:1/457: symlink dc/d26/l8b 0 2024-09-16T00:26:03.601 INFO:tasks.workunit.client.1.smithi169.stdout:2/339: fdatasync d7/db/d20/d4c/f5f 0 2024-09-16T00:26:03.603 INFO:tasks.workunit.client.1.smithi169.stdout:1/458: chown dc/d36/d3a/d45/d29/d4d/f54 30531110 1 2024-09-16T00:26:03.604 INFO:tasks.workunit.client.1.smithi169.stdout:6/555: creat d2/d3f/d95/fb2 x:0 0 0 2024-09-16T00:26:03.604 INFO:tasks.workunit.client.0.smithi112.stdout:5/499: sync 2024-09-16T00:26:03.606 INFO:tasks.workunit.client.1.smithi169.stdout:2/340: mknod d7/db/d20/d21/c6a 0 2024-09-16T00:26:03.606 INFO:tasks.workunit.client.1.smithi169.stdout:6/556: fsync d2/f64 0 2024-09-16T00:26:03.610 INFO:tasks.workunit.client.1.smithi169.stdout:1/459: symlink dc/d36/d3a/d7f/l8c 0 2024-09-16T00:26:03.610 INFO:tasks.workunit.client.0.smithi112.stdout:5/500: dwrite d4/d7/d14/d90/d9a/d7c/f88 [0,4194304] 0 2024-09-16T00:26:03.611 INFO:tasks.workunit.client.1.smithi169.stdout:6/557: chown d2/d36/c75 28763 1 2024-09-16T00:26:03.612 INFO:tasks.workunit.client.1.smithi169.stdout:2/341: rmdir d7/db/d27/d29/d37 39 2024-09-16T00:26:03.612 INFO:tasks.workunit.client.0.smithi112.stdout:5/501: truncate d4/d7/d26/f46 4845977 0 2024-09-16T00:26:03.614 INFO:tasks.workunit.client.1.smithi169.stdout:1/460: creat dc/d36/d3a/d45/d29/f8d x:0 0 0 2024-09-16T00:26:03.615 INFO:tasks.workunit.client.0.smithi112.stdout:5/502: creat d4/d4f/d78/fa8 x:0 0 0 2024-09-16T00:26:03.616 INFO:tasks.workunit.client.0.smithi112.stdout:5/503: fsync d4/fb 0 2024-09-16T00:26:03.617 INFO:tasks.workunit.client.1.smithi169.stdout:6/558: symlink d2/da4/daa/lb3 0 2024-09-16T00:26:03.619 INFO:tasks.workunit.client.0.smithi112.stdout:5/504: rename d4/dd/d12/d79 to d4/dd/d49/da9 0 2024-09-16T00:26:03.619 INFO:tasks.workunit.client.1.smithi169.stdout:2/342: mkdir d7/d6b 0 2024-09-16T00:26:03.620 INFO:tasks.workunit.client.0.smithi112.stdout:9/811: write d1/d5/d6/d2d/d6a/f6e [2998772,107415] 0 2024-09-16T00:26:03.621 INFO:tasks.workunit.client.0.smithi112.stdout:5/505: symlink d4/dd/d12/d52/d64/laa 0 2024-09-16T00:26:03.622 INFO:tasks.workunit.client.1.smithi169.stdout:1/461: readlink dc/d36/d3a/d45/l13 0 2024-09-16T00:26:03.624 INFO:tasks.workunit.client.1.smithi169.stdout:2/343: symlink d7/db/d20/d21/l6c 0 2024-09-16T00:26:03.625 INFO:tasks.workunit.client.0.smithi112.stdout:9/812: rename d1/d5/d20/d81/d1a/d88/l87 to d1/d5/d6/d59/d64/l120 0 2024-09-16T00:26:03.625 INFO:tasks.workunit.client.1.smithi169.stdout:2/344: write d7/db/f23 [5537000,81122] 0 2024-09-16T00:26:03.627 INFO:tasks.workunit.client.0.smithi112.stdout:5/506: dwrite d4/d7/d14/d90/d9a/d7c/f88 [0,4194304] 0 2024-09-16T00:26:03.628 INFO:tasks.workunit.client.1.smithi169.stdout:6/559: creat d2/d3f/d41/d72/d81/fb4 x:0 0 0 2024-09-16T00:26:03.631 INFO:tasks.workunit.client.0.smithi112.stdout:5/507: link d4/d7/f35 d4/dd/d6f/fab 0 2024-09-16T00:26:03.633 INFO:tasks.workunit.client.1.smithi169.stdout:3/472: dread d7/d1a/d1c/d23/f4c [0,4194304] 0 2024-09-16T00:26:03.633 INFO:tasks.workunit.client.1.smithi169.stdout:8/412: dwrite d7/f2e [0,4194304] 0 2024-09-16T00:26:03.636 INFO:tasks.workunit.client.1.smithi169.stdout:8/413: truncate d7/dc/d17/d48/f60 4432225 0 2024-09-16T00:26:03.638 INFO:tasks.workunit.client.1.smithi169.stdout:2/345: mkdir d7/db/d27/d4f/d56/d6d 0 2024-09-16T00:26:03.639 INFO:tasks.workunit.client.0.smithi112.stdout:5/508: dwrite d4/dd/d12/f48 [0,4194304] 0 2024-09-16T00:26:03.639 INFO:tasks.workunit.client.0.smithi112.stdout:9/813: truncate d1/d5/d6/d59/fec 1075742 0 2024-09-16T00:26:03.640 INFO:tasks.workunit.client.0.smithi112.stdout:5/509: fsync d4/d7/d26/f46 0 2024-09-16T00:26:03.642 INFO:tasks.workunit.client.1.smithi169.stdout:8/414: mkdir d7/d8c 0 2024-09-16T00:26:03.643 INFO:tasks.workunit.client.1.smithi169.stdout:1/462: write dc/d36/d3a/d45/f61 [561213,125721] 0 2024-09-16T00:26:03.644 INFO:tasks.workunit.client.1.smithi169.stdout:3/473: creat d7/d50/d6f/fb4 x:0 0 0 2024-09-16T00:26:03.644 INFO:tasks.workunit.client.0.smithi112.stdout:9/814: mknod d1/d5/d20/d81/d1a/d88/def/d11b/c121 0 2024-09-16T00:26:03.645 INFO:tasks.workunit.client.0.smithi112.stdout:9/815: chown d1/d5/d6/f4a 517189424 1 2024-09-16T00:26:03.647 INFO:tasks.workunit.client.1.smithi169.stdout:8/415: dread - d7/d20/f7d zero size 2024-09-16T00:26:03.647 INFO:tasks.workunit.client.1.smithi169.stdout:6/560: fsync d2/d3f/d41/d52/f89 0 2024-09-16T00:26:03.648 INFO:tasks.workunit.client.0.smithi112.stdout:5/510: dwrite d4/dd/d49/d70/f7e [0,4194304] 0 2024-09-16T00:26:03.649 INFO:tasks.workunit.client.1.smithi169.stdout:1/463: truncate dc/d2c/d3d/d86/f4f 1038539 0 2024-09-16T00:26:03.649 INFO:tasks.workunit.client.1.smithi169.stdout:8/416: fdatasync d7/dc/d17/f66 0 2024-09-16T00:26:03.652 INFO:tasks.workunit.client.0.smithi112.stdout:5/511: mknod d4/d7/d14/d90/cac 0 2024-09-16T00:26:03.652 INFO:tasks.workunit.client.0.smithi112.stdout:9/816: dread d1/d5/d6/d2d/f92 [0,4194304] 0 2024-09-16T00:26:03.653 INFO:tasks.workunit.client.1.smithi169.stdout:2/346: link d7/db/d20/d4c/c58 d7/db/d27/d29/c6e 0 2024-09-16T00:26:03.654 INFO:tasks.workunit.client.1.smithi169.stdout:3/474: mknod d7/d1a/d1c/d35/d63/da3/cb5 0 2024-09-16T00:26:03.654 INFO:tasks.workunit.client.0.smithi112.stdout:5/512: creat d4/d1c/fad x:0 0 0 2024-09-16T00:26:03.655 INFO:tasks.workunit.client.1.smithi169.stdout:8/417: mkdir d7/dc/d17/d24/d32/d8d 0 2024-09-16T00:26:03.655 INFO:tasks.workunit.client.1.smithi169.stdout:2/347: fsync d7/db/f45 0 2024-09-16T00:26:03.656 INFO:tasks.workunit.client.1.smithi169.stdout:6/561: symlink d2/d3f/d41/d72/d81/lb5 0 2024-09-16T00:26:03.658 INFO:tasks.workunit.client.1.smithi169.stdout:3/475: creat d7/d1a/d1c/fb6 x:0 0 0 2024-09-16T00:26:03.658 INFO:tasks.workunit.client.0.smithi112.stdout:5/513: link d4/d1c/l7b d4/d7/d14/d90/lae 0 2024-09-16T00:26:03.659 INFO:tasks.workunit.client.1.smithi169.stdout:6/562: truncate d2/d36/d48/f90 191980 0 2024-09-16T00:26:03.660 INFO:tasks.workunit.client.1.smithi169.stdout:8/418: mknod d7/dc/d26/d41/d4e/d81/c8e 0 2024-09-16T00:26:03.661 INFO:tasks.workunit.client.1.smithi169.stdout:2/348: symlink d7/d1c/d26/l6f 0 2024-09-16T00:26:03.661 INFO:tasks.workunit.client.0.smithi112.stdout:5/514: creat d4/d7/d9b/faf x:0 0 0 2024-09-16T00:26:03.662 INFO:tasks.workunit.client.1.smithi169.stdout:2/349: stat d7/db/f19 0 2024-09-16T00:26:03.662 INFO:tasks.workunit.client.0.smithi112.stdout:9/817: dwrite d1/d5/d6/d18/fb0 [0,4194304] 0 2024-09-16T00:26:03.662 INFO:tasks.workunit.client.1.smithi169.stdout:6/563: mknod d2/d4c/cb6 0 2024-09-16T00:26:03.664 INFO:tasks.workunit.client.1.smithi169.stdout:8/419: unlink d7/dc/d26/c69 0 2024-09-16T00:26:03.664 INFO:tasks.workunit.client.0.smithi112.stdout:9/818: creat d1/d76/f122 x:0 0 0 2024-09-16T00:26:03.664 INFO:tasks.workunit.client.1.smithi169.stdout:8/420: dread - d7/dc/d17/d24/f88 zero size 2024-09-16T00:26:03.665 INFO:tasks.workunit.client.1.smithi169.stdout:2/350: mkdir d7/d6b/d70 0 2024-09-16T00:26:03.666 INFO:tasks.workunit.client.1.smithi169.stdout:2/351: write f5 [5549704,50889] 0 2024-09-16T00:26:03.667 INFO:tasks.workunit.client.0.smithi112.stdout:9/819: link d1/d5/d20/l58 d1/d5/d6/d114/l123 0 2024-09-16T00:26:03.671 INFO:tasks.workunit.client.0.smithi112.stdout:9/820: dread d1/d5/d6/d2d/d39/f65 [4194304,4194304] 0 2024-09-16T00:26:03.671 INFO:tasks.workunit.client.1.smithi169.stdout:8/421: mkdir d7/dc/d17/d24/d32/d8d/d8f 0 2024-09-16T00:26:03.672 INFO:tasks.workunit.client.1.smithi169.stdout:8/422: read - d7/dc/d17/f89 zero size 2024-09-16T00:26:03.673 INFO:tasks.workunit.client.1.smithi169.stdout:2/352: link d7/db/f17 d7/db/d27/d4f/d56/d6d/f71 0 2024-09-16T00:26:03.674 INFO:tasks.workunit.client.0.smithi112.stdout:9/821: truncate d1/d5/d6/d107/f10b 1023377 0 2024-09-16T00:26:03.675 INFO:tasks.workunit.client.1.smithi169.stdout:8/423: fdatasync d7/dc/d1d/f47 0 2024-09-16T00:26:03.675 INFO:tasks.workunit.client.0.smithi112.stdout:9/822: dread - d1/d5/d6/d2d/d39/ffc zero size 2024-09-16T00:26:03.676 INFO:tasks.workunit.client.1.smithi169.stdout:2/353: creat d7/d1c/f72 x:0 0 0 2024-09-16T00:26:03.676 INFO:tasks.workunit.client.1.smithi169.stdout:8/424: rmdir d7/d37 39 2024-09-16T00:26:03.677 INFO:tasks.workunit.client.1.smithi169.stdout:2/354: chown d7/db/d27 28 1 2024-09-16T00:26:03.678 INFO:tasks.workunit.client.1.smithi169.stdout:8/425: fsync d7/dc/d17/f28 0 2024-09-16T00:26:03.680 INFO:tasks.workunit.client.1.smithi169.stdout:2/355: mknod d7/db/d27/d29/d37/c73 0 2024-09-16T00:26:03.681 INFO:tasks.workunit.client.0.smithi112.stdout:9/823: dwrite d1/d5/f16 [0,4194304] 0 2024-09-16T00:26:03.681 INFO:tasks.workunit.client.1.smithi169.stdout:8/426: mknod d7/d37/c90 0 2024-09-16T00:26:03.683 INFO:tasks.workunit.client.1.smithi169.stdout:1/464: dread dc/f15 [0,4194304] 0 2024-09-16T00:26:03.683 INFO:tasks.workunit.client.1.smithi169.stdout:8/427: fsync d7/dc/d17/f1b 0 2024-09-16T00:26:03.685 INFO:tasks.workunit.client.1.smithi169.stdout:2/356: symlink d7/d1c/l74 0 2024-09-16T00:26:03.685 INFO:tasks.workunit.client.1.smithi169.stdout:1/465: rmdir dc/d6c 39 2024-09-16T00:26:03.685 INFO:tasks.workunit.client.1.smithi169.stdout:8/428: write d7/dc/d26/d41/f5c [534986,35900] 0 2024-09-16T00:26:03.686 INFO:tasks.workunit.client.1.smithi169.stdout:6/564: dread d2/f19 [0,4194304] 0 2024-09-16T00:26:03.688 INFO:tasks.workunit.client.1.smithi169.stdout:6/565: chown d2/d3f/d41 404913 1 2024-09-16T00:26:03.692 INFO:tasks.workunit.client.1.smithi169.stdout:1/466: creat dc/d36/d3a/d45/d29/d2a/f8e x:0 0 0 2024-09-16T00:26:03.693 INFO:tasks.workunit.client.0.smithi112.stdout:9/824: write d1/d5/d83/fb9 [109705,104327] 0 2024-09-16T00:26:03.694 INFO:tasks.workunit.client.1.smithi169.stdout:1/467: write dc/d26/f63 [334738,19669] 0 2024-09-16T00:26:03.695 INFO:tasks.workunit.client.0.smithi112.stdout:9/825: symlink d1/d76/l124 0 2024-09-16T00:26:03.697 INFO:tasks.workunit.client.0.smithi112.stdout:9/826: symlink d1/d5/d6/d59/d64/l125 0 2024-09-16T00:26:03.698 INFO:tasks.workunit.client.1.smithi169.stdout:1/468: write dc/d2c/d3d/d86/f4f [1722354,31743] 0 2024-09-16T00:26:03.700 INFO:tasks.workunit.client.0.smithi112.stdout:7/614: sync 2024-09-16T00:26:03.700 INFO:tasks.workunit.client.0.smithi112.stdout:1/629: sync 2024-09-16T00:26:03.700 INFO:tasks.workunit.client.0.smithi112.stdout:7/615: readlink d6/lc 0 2024-09-16T00:26:03.701 INFO:tasks.workunit.client.1.smithi169.stdout:1/469: write dc/d36/d39/d50/f7a [4075370,113494] 0 2024-09-16T00:26:03.702 INFO:tasks.workunit.client.1.smithi169.stdout:2/357: write f2 [1360356,1979] 0 2024-09-16T00:26:03.704 INFO:tasks.workunit.client.0.smithi112.stdout:7/616: creat d6/d10/d16/fcf x:0 0 0 2024-09-16T00:26:03.707 INFO:tasks.workunit.client.1.smithi169.stdout:2/358: creat d7/d1c/d26/f75 x:0 0 0 2024-09-16T00:26:03.707 INFO:tasks.workunit.client.0.smithi112.stdout:7/617: read d6/d10/d16/d49/d6c/f36 [3761545,129962] 0 2024-09-16T00:26:03.708 INFO:tasks.workunit.client.0.smithi112.stdout:7/618: mknod d6/d10/da1/dcd/cd0 0 2024-09-16T00:26:03.709 INFO:tasks.workunit.client.1.smithi169.stdout:6/566: dwrite d2/d4d/f49 [0,4194304] 0 2024-09-16T00:26:03.709 INFO:tasks.workunit.client.0.smithi112.stdout:7/619: chown d6/d10/d16/l21 247707 1 2024-09-16T00:26:03.709 INFO:tasks.workunit.client.0.smithi112.stdout:7/620: dread - d6/d10/d16/d3b/fa6 zero size 2024-09-16T00:26:03.712 INFO:tasks.workunit.client.0.smithi112.stdout:7/621: write d6/d10/f87 [3250522,77599] 0 2024-09-16T00:26:03.713 INFO:tasks.workunit.client.1.smithi169.stdout:2/359: creat d7/d1c/d26/f76 x:0 0 0 2024-09-16T00:26:03.715 INFO:tasks.workunit.client.1.smithi169.stdout:6/567: mkdir d2/d3f/d41/d5c/d91/db7 0 2024-09-16T00:26:03.719 INFO:tasks.workunit.client.1.smithi169.stdout:6/568: symlink d2/da4/db1/lb8 0 2024-09-16T00:26:03.720 INFO:tasks.workunit.client.0.smithi112.stdout:7/622: rename d6/d10/d75/d88/l28 to d6/d10/d16/ld1 0 2024-09-16T00:26:03.722 INFO:tasks.workunit.client.1.smithi169.stdout:2/360: mkdir d7/db/d27/d29/d77 0 2024-09-16T00:26:03.723 INFO:tasks.workunit.client.0.smithi112.stdout:7/623: mkdir d6/d10/d16/d96/dc7/dd2 0 2024-09-16T00:26:03.723 INFO:tasks.workunit.client.0.smithi112.stdout:7/624: stat d6/d10/d16/d96/dc7/dd2 0 2024-09-16T00:26:03.725 INFO:tasks.workunit.client.0.smithi112.stdout:7/625: rmdir d6/d10/d16/d3b 39 2024-09-16T00:26:03.726 INFO:tasks.workunit.client.1.smithi169.stdout:2/361: mkdir d7/db/d27/d29/d78 0 2024-09-16T00:26:03.727 INFO:tasks.workunit.client.0.smithi112.stdout:7/626: mkdir d6/d10/d75/d8f/d90/dd3 0 2024-09-16T00:26:03.730 INFO:tasks.workunit.client.1.smithi169.stdout:6/569: truncate d2/d3f/d41/d52/f18 1022862 0 2024-09-16T00:26:03.730 INFO:tasks.workunit.client.1.smithi169.stdout:2/362: dread d7/db/f60 [0,4194304] 0 2024-09-16T00:26:03.733 INFO:tasks.workunit.client.1.smithi169.stdout:6/570: dread d2/d4d/d59/d43/f56 [0,4194304] 0 2024-09-16T00:26:03.733 INFO:tasks.workunit.client.1.smithi169.stdout:6/571: chown d2/d4d/f49 70349 1 2024-09-16T00:26:03.735 INFO:tasks.workunit.client.1.smithi169.stdout:6/572: mkdir d2/d3f/d41/d5c/db9 0 2024-09-16T00:26:03.736 INFO:tasks.workunit.client.0.smithi112.stdout:7/627: dwrite d6/d10/d16/d49/d6c/f62 [0,4194304] 0 2024-09-16T00:26:03.736 INFO:tasks.workunit.client.1.smithi169.stdout:6/573: write d2/d4d/d59/d43/d67/fae [373912,77603] 0 2024-09-16T00:26:03.738 INFO:tasks.workunit.client.1.smithi169.stdout:6/574: write d2/d3f/f85 [610544,37973] 0 2024-09-16T00:26:03.742 INFO:tasks.workunit.client.0.smithi112.stdout:7/628: dwrite d6/d10/d16/f4e [0,4194304] 0 2024-09-16T00:26:03.743 INFO:tasks.workunit.client.1.smithi169.stdout:6/575: link d2/d4d/d59/c3e d2/d4d/d59/cba 0 2024-09-16T00:26:03.745 INFO:tasks.workunit.client.1.smithi169.stdout:6/576: write d2/d22/d51/f77 [8581263,84408] 0 2024-09-16T00:26:03.745 INFO:tasks.workunit.client.1.smithi169.stdout:6/577: readlink d2/d22/l5e 0 2024-09-16T00:26:03.748 INFO:tasks.workunit.client.0.smithi112.stdout:7/629: getdents d6/d10/d75/d8f/d90/dbc 0 2024-09-16T00:26:03.750 INFO:tasks.workunit.client.1.smithi169.stdout:2/363: dread d7/db/f23 [4194304,4194304] 0 2024-09-16T00:26:03.752 INFO:tasks.workunit.client.1.smithi169.stdout:2/364: mkdir d7/d6b/d70/d79 0 2024-09-16T00:26:03.753 INFO:tasks.workunit.client.0.smithi112.stdout:7/630: getdents d6/d7f 0 2024-09-16T00:26:03.755 INFO:tasks.workunit.client.1.smithi169.stdout:6/578: dread d2/d4d/d59/d38/f5a [0,4194304] 0 2024-09-16T00:26:03.757 INFO:tasks.workunit.client.1.smithi169.stdout:2/365: rename d7/d1c/d26/c38 to d7/db/d20/d21/c7a 0 2024-09-16T00:26:03.758 INFO:tasks.workunit.client.1.smithi169.stdout:6/579: mkdir d2/d3f/d41/d5c/dac/dbb 0 2024-09-16T00:26:03.758 INFO:tasks.workunit.client.0.smithi112.stdout:7/631: dwrite d6/d10/d16/f34 [0,4194304] 0 2024-09-16T00:26:03.759 INFO:tasks.workunit.client.1.smithi169.stdout:2/366: creat d7/d6b/d70/d79/f7b x:0 0 0 2024-09-16T00:26:03.762 INFO:tasks.workunit.client.1.smithi169.stdout:6/580: rename d2/d4d/d59/d43/d67 to d2/da4/db1/dbc 0 2024-09-16T00:26:03.763 INFO:tasks.workunit.client.0.smithi112.stdout:7/632: read d6/d10/d16/d96/dc7/fbe [329522,76152] 0 2024-09-16T00:26:03.764 INFO:tasks.workunit.client.1.smithi169.stdout:2/367: symlink d7/db/d20/d5c/l7c 0 2024-09-16T00:26:03.765 INFO:tasks.workunit.client.0.smithi112.stdout:7/633: read d6/d10/d75/d8f/fc5 [167884,17094] 0 2024-09-16T00:26:03.770 INFO:tasks.workunit.client.0.smithi112.stdout:7/634: truncate d6/d10/d75/d88/f70 976426 0 2024-09-16T00:26:03.772 INFO:tasks.workunit.client.1.smithi169.stdout:2/368: fdatasync d7/fe 0 2024-09-16T00:26:03.773 INFO:tasks.workunit.client.1.smithi169.stdout:6/581: symlink d2/d3f/d41/d52/lbd 0 2024-09-16T00:26:03.773 INFO:tasks.workunit.client.1.smithi169.stdout:6/582: fdatasync d2/f64 0 2024-09-16T00:26:03.777 INFO:tasks.workunit.client.1.smithi169.stdout:6/583: truncate d2/d36/f58 1478478 0 2024-09-16T00:26:03.778 INFO:tasks.workunit.client.1.smithi169.stdout:6/584: truncate d2/d36/d48/f7e 620407 0 2024-09-16T00:26:03.783 INFO:tasks.workunit.client.0.smithi112.stdout:7/635: truncate d6/d10/d16/d49/d6c/f5b 668236 0 2024-09-16T00:26:03.783 INFO:tasks.workunit.client.1.smithi169.stdout:6/585: read d2/d4d/d59/d38/f4f [1474918,52966] 0 2024-09-16T00:26:03.784 INFO:tasks.workunit.client.0.smithi112.stdout:7/636: chown d6/d10/f32 223692660 1 2024-09-16T00:26:03.786 INFO:tasks.workunit.client.0.smithi112.stdout:7/637: creat d6/d10/d16/d96/dc7/d39/fd4 x:0 0 0 2024-09-16T00:26:03.786 INFO:tasks.workunit.client.1.smithi169.stdout:6/586: mknod d2/da4/cbe 0 2024-09-16T00:26:03.786 INFO:tasks.workunit.client.1.smithi169.stdout:6/587: readlink d2/d36/d48/l63 0 2024-09-16T00:26:03.788 INFO:tasks.workunit.client.0.smithi112.stdout:7/638: mknod d6/d10/d16/d96/dc7/dd2/cd5 0 2024-09-16T00:26:03.789 INFO:tasks.workunit.client.1.smithi169.stdout:6/588: creat d2/da4/fbf x:0 0 0 2024-09-16T00:26:03.789 INFO:tasks.workunit.client.0.smithi112.stdout:7/639: getdents d6/d10/da1 0 2024-09-16T00:26:03.790 INFO:tasks.workunit.client.0.smithi112.stdout:7/640: chown d6/d10/d16/d96/dc7/c56 81604060 1 2024-09-16T00:26:03.793 INFO:tasks.workunit.client.1.smithi169.stdout:6/589: getdents d2/d4c 0 2024-09-16T00:26:03.793 INFO:tasks.workunit.client.0.smithi112.stdout:7/641: dread d6/d10/d16/d96/dc7/d39/fa3 [0,4194304] 0 2024-09-16T00:26:03.796 INFO:tasks.workunit.client.1.smithi169.stdout:2/369: dread d7/db/d20/f3b [0,4194304] 0 2024-09-16T00:26:03.796 INFO:tasks.workunit.client.1.smithi169.stdout:6/590: write d2/fb [143185,10820] 0 2024-09-16T00:26:03.796 INFO:tasks.workunit.client.1.smithi169.stdout:6/591: fsync d2/d4d/f57 0 2024-09-16T00:26:03.796 INFO:tasks.workunit.client.1.smithi169.stdout:2/370: chown d7/db/d27/d29/l52 233376 1 2024-09-16T00:26:03.798 INFO:tasks.workunit.client.1.smithi169.stdout:2/371: write d7/db/d20/d4c/f5f [109309,8738] 0 2024-09-16T00:26:03.798 INFO:tasks.workunit.client.1.smithi169.stdout:6/592: mkdir d2/d3f/d41/d72/d9e/dc0 0 2024-09-16T00:26:03.800 INFO:tasks.workunit.client.1.smithi169.stdout:6/593: mknod d2/d4d/cc1 0 2024-09-16T00:26:03.801 INFO:tasks.workunit.client.1.smithi169.stdout:2/372: mkdir d7/db/d27/d29/d7d 0 2024-09-16T00:26:03.801 INFO:tasks.workunit.client.1.smithi169.stdout:2/373: read - d7/d1c/d26/f76 zero size 2024-09-16T00:26:03.802 INFO:tasks.workunit.client.1.smithi169.stdout:6/594: read d2/d4d/f7d [118726,90603] 0 2024-09-16T00:26:03.803 INFO:tasks.workunit.client.1.smithi169.stdout:6/595: write d2/d22/fa1 [636538,93050] 0 2024-09-16T00:26:03.809 INFO:tasks.workunit.client.1.smithi169.stdout:2/374: dread d7/db/d20/d21/f4a [0,4194304] 0 2024-09-16T00:26:03.810 INFO:tasks.workunit.client.1.smithi169.stdout:6/596: dwrite d2/d36/f44 [0,4194304] 0 2024-09-16T00:26:03.811 INFO:tasks.workunit.client.1.smithi169.stdout:2/375: dread - d7/d1c/d26/f75 zero size 2024-09-16T00:26:03.819 INFO:tasks.workunit.client.1.smithi169.stdout:2/376: dread d7/db/d27/d4f/f53 [0,4194304] 0 2024-09-16T00:26:03.821 INFO:tasks.workunit.client.1.smithi169.stdout:6/597: dwrite d2/d3f/d41/d72/d81/fb4 [0,4194304] 0 2024-09-16T00:26:03.823 INFO:tasks.workunit.client.1.smithi169.stdout:6/598: truncate d2/d3f/d41/d5c/f88 517675 0 2024-09-16T00:26:03.825 INFO:tasks.workunit.client.1.smithi169.stdout:6/599: truncate d2/d3f/d41/d5c/d73/fa8 5372 0 2024-09-16T00:26:03.829 INFO:tasks.workunit.client.1.smithi169.stdout:6/600: read d2/d4d/d59/d43/f4a [3680350,26701] 0 2024-09-16T00:26:03.829 INFO:tasks.workunit.client.1.smithi169.stdout:2/377: write d7/f11 [1496688,96050] 0 2024-09-16T00:26:03.830 INFO:tasks.workunit.client.1.smithi169.stdout:2/378: chown d7/d6b 49 1 2024-09-16T00:26:03.831 INFO:tasks.workunit.client.1.smithi169.stdout:6/601: dread d2/d36/d48/f7e [0,4194304] 0 2024-09-16T00:26:03.832 INFO:tasks.workunit.client.1.smithi169.stdout:2/379: symlink d7/db/d27/d4f/d56/l7e 0 2024-09-16T00:26:03.833 INFO:tasks.workunit.client.1.smithi169.stdout:6/602: fsync d2/da4/db1/dbc/fae 0 2024-09-16T00:26:03.834 INFO:tasks.workunit.client.1.smithi169.stdout:2/380: chown d7/db/c18 1519 1 2024-09-16T00:26:03.836 INFO:tasks.workunit.client.1.smithi169.stdout:6/603: rename d2/da4/daa/fad to d2/d22/fc2 0 2024-09-16T00:26:03.837 INFO:tasks.workunit.client.1.smithi169.stdout:2/381: symlink d7/db/d20/d21/l7f 0 2024-09-16T00:26:03.839 INFO:tasks.workunit.client.1.smithi169.stdout:6/604: mknod d2/d3f/d41/d5c/d91/db7/cc3 0 2024-09-16T00:26:03.839 INFO:tasks.workunit.client.0.smithi112.stdout:8/537: sync 2024-09-16T00:26:03.839 INFO:tasks.workunit.client.0.smithi112.stdout:6/531: sync 2024-09-16T00:26:03.841 INFO:tasks.workunit.client.0.smithi112.stdout:8/538: mknod d10/d12/d1c/d3b/d7d/cb2 0 2024-09-16T00:26:03.841 INFO:tasks.workunit.client.1.smithi169.stdout:6/605: symlink d2/d3f/d41/d5c/db9/lc4 0 2024-09-16T00:26:03.843 INFO:tasks.workunit.client.1.smithi169.stdout:6/606: fsync d2/d36/d48/f7e 0 2024-09-16T00:26:03.844 INFO:tasks.workunit.client.0.smithi112.stdout:6/532: rename dd/df/d10/f1b to dd/df/d13/fab 0 2024-09-16T00:26:03.845 INFO:tasks.workunit.client.1.smithi169.stdout:2/382: write d7/db/d27/d4f/d56/d6d/f71 [457599,110027] 0 2024-09-16T00:26:03.846 INFO:tasks.workunit.client.0.smithi112.stdout:8/539: rename d10/d12/d1c/f31 to d10/d12/d1c/d61/d6c/fb3 0 2024-09-16T00:26:03.847 INFO:tasks.workunit.client.1.smithi169.stdout:6/607: creat d2/d3f/d41/d5c/d91/da6/fc5 x:0 0 0 2024-09-16T00:26:03.848 INFO:tasks.workunit.client.1.smithi169.stdout:2/383: creat d7/db/d27/d4f/f80 x:0 0 0 2024-09-16T00:26:03.848 INFO:tasks.workunit.client.0.smithi112.stdout:6/533: mknod dd/df/d11/d44/da4/cac 0 2024-09-16T00:26:03.849 INFO:tasks.workunit.client.1.smithi169.stdout:6/608: readlink d2/d3f/laf 0 2024-09-16T00:26:03.849 INFO:tasks.workunit.client.0.smithi112.stdout:6/534: chown dd/df/d13/d7c/d88 33944 1 2024-09-16T00:26:03.849 INFO:tasks.workunit.client.0.smithi112.stdout:6/535: chown dd/df/d10/c4e 50 1 2024-09-16T00:26:03.851 INFO:tasks.workunit.client.1.smithi169.stdout:2/384: rename d7/db/d27/d29/d46 to d7/d6b/d70/d81 0 2024-09-16T00:26:03.853 INFO:tasks.workunit.client.0.smithi112.stdout:8/540: dwrite d10/f46 [0,4194304] 0 2024-09-16T00:26:03.853 INFO:tasks.workunit.client.1.smithi169.stdout:2/385: write d7/db/f13 [6725142,12329] 0 2024-09-16T00:26:03.854 INFO:tasks.workunit.client.0.smithi112.stdout:8/541: dread - d10/d12/d1c/d75/f80 zero size 2024-09-16T00:26:03.854 INFO:tasks.workunit.client.0.smithi112.stdout:6/536: link dd/df/d10/d47/f57 dd/df/d11/fad 0 2024-09-16T00:26:03.856 INFO:tasks.workunit.client.0.smithi112.stdout:8/542: write d10/d4e/f8e [78607,69917] 0 2024-09-16T00:26:03.857 INFO:tasks.workunit.client.1.smithi169.stdout:6/609: truncate d2/f1f 1332923 0 2024-09-16T00:26:03.858 INFO:tasks.workunit.client.1.smithi169.stdout:6/610: dread d2/d36/d48/f90 [0,4194304] 0 2024-09-16T00:26:03.861 INFO:tasks.workunit.client.1.smithi169.stdout:2/386: truncate f2 1101455 0 2024-09-16T00:26:03.862 INFO:tasks.workunit.client.0.smithi112.stdout:6/537: dwrite dd/df/d10/d47/d55/f7a [4194304,4194304] 0 2024-09-16T00:26:03.864 INFO:tasks.workunit.client.1.smithi169.stdout:2/387: unlink d7/fe 0 2024-09-16T00:26:03.867 INFO:tasks.workunit.client.1.smithi169.stdout:6/611: mknod d2/d3f/d41/d5c/dac/dbb/cc6 0 2024-09-16T00:26:03.867 INFO:tasks.workunit.client.0.smithi112.stdout:6/538: mkdir dd/d15/da5/dae 0 2024-09-16T00:26:03.868 INFO:tasks.workunit.client.1.smithi169.stdout:6/612: write d2/d3f/d41/d52/f1e [808780,44400] 0 2024-09-16T00:26:03.870 INFO:tasks.workunit.client.1.smithi169.stdout:2/388: unlink d7/db/d27/d29/c6e 0 2024-09-16T00:26:03.874 INFO:tasks.workunit.client.0.smithi112.stdout:8/543: write d10/d4e/d58/f66 [733632,120875] 0 2024-09-16T00:26:03.874 INFO:tasks.workunit.client.1.smithi169.stdout:2/389: symlink d7/d6b/d70/l82 0 2024-09-16T00:26:03.875 INFO:tasks.workunit.client.0.smithi112.stdout:6/539: dwrite dd/df/d13/d7c/fa9 [0,4194304] 0 2024-09-16T00:26:03.876 INFO:tasks.workunit.client.0.smithi112.stdout:6/540: chown dd/df 127272 1 2024-09-16T00:26:03.878 INFO:tasks.workunit.client.1.smithi169.stdout:6/613: dwrite d2/d3f/d41/d5c/f65 [0,4194304] 0 2024-09-16T00:26:03.879 INFO:tasks.workunit.client.0.smithi112.stdout:6/541: symlink dd/d15/d34/laf 0 2024-09-16T00:26:03.880 INFO:tasks.workunit.client.0.smithi112.stdout:8/544: rename d10/f55 to d10/d12/d1c/d61/fb4 0 2024-09-16T00:26:03.881 INFO:tasks.workunit.client.0.smithi112.stdout:6/542: creat dd/d15/da5/fb0 x:0 0 0 2024-09-16T00:26:03.882 INFO:tasks.workunit.client.0.smithi112.stdout:8/545: rmdir d10/d12/d24 39 2024-09-16T00:26:03.883 INFO:tasks.workunit.client.0.smithi112.stdout:6/543: symlink dd/df/d10/lb1 0 2024-09-16T00:26:03.886 INFO:tasks.workunit.client.0.smithi112.stdout:6/544: symlink dd/df/d10/lb2 0 2024-09-16T00:26:03.888 INFO:tasks.workunit.client.0.smithi112.stdout:6/545: rmdir dd/df/d10/d47/d55 39 2024-09-16T00:26:03.908 INFO:tasks.workunit.client.0.smithi112.stdout:8/546: truncate d10/d12/d1c/d61/fb4 546374 0 2024-09-16T00:26:03.909 INFO:tasks.workunit.client.0.smithi112.stdout:6/546: truncate dd/d23/d8f/f8c 1801778 0 2024-09-16T00:26:03.911 INFO:tasks.workunit.client.0.smithi112.stdout:8/547: creat d10/d12/d1c/d61/d6c/fb5 x:0 0 0 2024-09-16T00:26:03.913 INFO:tasks.workunit.client.0.smithi112.stdout:6/547: rename dd/df/d10/f1a to dd/d15/da5/d79/fb3 0 2024-09-16T00:26:03.915 INFO:tasks.workunit.client.0.smithi112.stdout:8/548: mknod d10/d4e/d8c/d7c/da5/cb6 0 2024-09-16T00:26:03.916 INFO:tasks.workunit.client.1.smithi169.stdout:6/614: dread d2/d22/d51/f77 [0,4194304] 0 2024-09-16T00:26:03.918 INFO:tasks.workunit.client.0.smithi112.stdout:8/549: mknod d10/d12/d24/cb7 0 2024-09-16T00:26:03.920 INFO:tasks.workunit.client.0.smithi112.stdout:8/550: read - d10/d12/d1c/d61/fa2 zero size 2024-09-16T00:26:03.920 INFO:tasks.workunit.client.1.smithi169.stdout:6/615: chown d2/da4/db1 776734 1 2024-09-16T00:26:03.920 INFO:tasks.workunit.client.0.smithi112.stdout:8/551: chown d10/d12/d24/f2e 165158 1 2024-09-16T00:26:03.921 INFO:tasks.workunit.client.1.smithi169.stdout:6/616: write d2/d36/f44 [993370,273] 0 2024-09-16T00:26:03.930 INFO:tasks.workunit.client.1.smithi169.stdout:6/617: getdents d2/d3f/d41/d5c 0 2024-09-16T00:26:03.932 INFO:tasks.workunit.client.1.smithi169.stdout:6/618: write d2/da4/db1/dbc/fae [17301,78044] 0 2024-09-16T00:26:03.953 INFO:tasks.workunit.client.0.smithi112.stdout:3/590: sync 2024-09-16T00:26:03.953 INFO:tasks.workunit.client.0.smithi112.stdout:4/636: sync 2024-09-16T00:26:03.953 INFO:tasks.workunit.client.1.smithi169.stdout:6/619: symlink d2/d22/lc7 0 2024-09-16T00:26:03.961 INFO:tasks.workunit.client.1.smithi169.stdout:6/620: link d2/d4d/d59/d38/f62 d2/d3f/d95/fc8 0 2024-09-16T00:26:03.964 INFO:tasks.workunit.client.1.smithi169.stdout:2/390: creat d7/db/d20/f83 x:0 0 0 2024-09-16T00:26:03.968 INFO:tasks.workunit.client.1.smithi169.stdout:6/621: dread d2/d4d/f49 [0,4194304] 0 2024-09-16T00:26:03.969 INFO:tasks.workunit.client.1.smithi169.stdout:2/391: rename d7/db/d27/d29/d78 to d7/d6b/d84 0 2024-09-16T00:26:03.969 INFO:tasks.workunit.client.0.smithi112.stdout:6/548: truncate dd/df/d13/f1c 3568285 0 2024-09-16T00:26:03.971 INFO:tasks.workunit.client.1.smithi169.stdout:6/622: symlink d2/d3f/d41/d5c/db9/lc9 0 2024-09-16T00:26:03.972 INFO:tasks.workunit.client.0.smithi112.stdout:6/549: write dd/df/d10/d47/d55/f78 [561863,50198] 0 2024-09-16T00:26:03.973 INFO:tasks.workunit.client.1.smithi169.stdout:2/392: readlink d7/d1c/d26/l2d 0 2024-09-16T00:26:03.974 INFO:tasks.workunit.client.0.smithi112.stdout:6/550: dread - dd/d23/d8f/fa8 zero size 2024-09-16T00:26:03.975 INFO:tasks.workunit.client.0.smithi112.stdout:8/552: dwrite d10/f15 [0,4194304] 0 2024-09-16T00:26:03.978 INFO:tasks.workunit.client.1.smithi169.stdout:2/393: mknod d7/db/d27/d29/d7d/c85 0 2024-09-16T00:26:03.978 INFO:tasks.workunit.client.0.smithi112.stdout:3/591: write d2/f3f [2675028,9264] 0 2024-09-16T00:26:03.980 INFO:tasks.workunit.client.0.smithi112.stdout:4/637: write d6/da/f35 [520191,84909] 0 2024-09-16T00:26:03.980 INFO:tasks.workunit.client.0.smithi112.stdout:4/638: stat d6/db/df 0 2024-09-16T00:26:03.981 INFO:tasks.workunit.client.0.smithi112.stdout:4/639: stat d6/d7c/da0 0 2024-09-16T00:26:03.981 INFO:tasks.workunit.client.0.smithi112.stdout:3/592: mknod d2/d9/d2b/d47/d52/d7a/d80/d83/d68/d90/cc8 0 2024-09-16T00:26:03.983 INFO:tasks.workunit.client.1.smithi169.stdout:2/394: creat d7/db/d27/d29/d77/f86 x:0 0 0 2024-09-16T00:26:03.984 INFO:tasks.workunit.client.0.smithi112.stdout:6/551: write dd/df/d17/d27/d54/f6a [314761,98542] 0 2024-09-16T00:26:03.985 INFO:tasks.workunit.client.0.smithi112.stdout:4/640: rename d6/d39/l5b to d6/d39/d36/d5c/dc6/le8 0 2024-09-16T00:26:03.986 INFO:tasks.workunit.client.1.smithi169.stdout:2/395: stat d7/db/d27/d29/c64 0 2024-09-16T00:26:03.986 INFO:tasks.workunit.client.0.smithi112.stdout:3/593: mknod d2/d9/d2b/d47/d52/d7a/cc9 0 2024-09-16T00:26:03.987 INFO:tasks.workunit.client.1.smithi169.stdout:6/623: rmdir d2/d36 39 2024-09-16T00:26:03.989 INFO:tasks.workunit.client.0.smithi112.stdout:4/641: mknod d6/d39/d36/d5c/d7e/d8f/ce9 0 2024-09-16T00:26:03.992 INFO:tasks.workunit.client.1.smithi169.stdout:2/396: unlink d7/lc 0 2024-09-16T00:26:03.993 INFO:tasks.workunit.client.0.smithi112.stdout:4/642: creat d6/db/d43/fea x:0 0 0 2024-09-16T00:26:03.994 INFO:tasks.workunit.client.0.smithi112.stdout:4/643: dread - d6/db/df/d5e/fcb zero size 2024-09-16T00:26:03.995 INFO:tasks.workunit.client.1.smithi169.stdout:6/624: dwrite d2/d3f/f85 [0,4194304] 0 2024-09-16T00:26:03.997 INFO:tasks.workunit.client.1.smithi169.stdout:6/625: mknod d2/da4/db1/cca 0 2024-09-16T00:26:03.997 INFO:tasks.workunit.client.0.smithi112.stdout:4/644: creat d6/db/d1f/d94/feb x:0 0 0 2024-09-16T00:26:03.997 INFO:tasks.workunit.client.0.smithi112.stdout:6/552: write dd/df/f6e [4146195,90870] 0 2024-09-16T00:26:03.998 INFO:tasks.workunit.client.0.smithi112.stdout:3/594: write d2/d9/d2b/d47/d52/d7a/d80/d83/f50 [7015951,112094] 0 2024-09-16T00:26:03.999 INFO:tasks.workunit.client.1.smithi169.stdout:6/626: symlink d2/d3f/d41/d5c/d91/da6/lcb 0 2024-09-16T00:26:04.001 INFO:tasks.workunit.client.1.smithi169.stdout:6/627: fsync d2/d4d/d59/d38/f4f 0 2024-09-16T00:26:04.002 INFO:tasks.workunit.client.1.smithi169.stdout:2/397: getdents d7/db/d27/d29/d7d 0 2024-09-16T00:26:04.003 INFO:tasks.workunit.client.1.smithi169.stdout:6/628: fdatasync d2/d22/f23 0 2024-09-16T00:26:04.006 INFO:tasks.workunit.client.0.smithi112.stdout:4/645: dwrite d6/db/f60 [0,4194304] 0 2024-09-16T00:26:04.007 INFO:tasks.workunit.client.0.smithi112.stdout:6/553: dwrite dd/df/d13/f48 [0,4194304] 0 2024-09-16T00:26:04.008 INFO:tasks.workunit.client.1.smithi169.stdout:6/629: link d2/d4d/d59/d43/f2f d2/da4/fcc 0 2024-09-16T00:26:04.009 INFO:tasks.workunit.client.0.smithi112.stdout:6/554: chown dd/df/d17/d27/d54/l73 29 1 2024-09-16T00:26:04.010 INFO:tasks.workunit.client.0.smithi112.stdout:4/646: symlink d6/db/d1f/d41/d49/d31/lec 0 2024-09-16T00:26:04.010 INFO:tasks.workunit.client.1.smithi169.stdout:6/630: rmdir d2/d4c 39 2024-09-16T00:26:04.010 INFO:tasks.workunit.client.0.smithi112.stdout:4/647: read - d6/d39/d1c/d6e/d6f/f97 zero size 2024-09-16T00:26:04.011 INFO:tasks.workunit.client.1.smithi169.stdout:2/398: truncate d7/db/d27/d4f/f53 1637021 0 2024-09-16T00:26:04.011 INFO:tasks.workunit.client.0.smithi112.stdout:6/555: mknod dd/d23/cb4 0 2024-09-16T00:26:04.012 INFO:tasks.workunit.client.1.smithi169.stdout:2/399: dread - d7/d1c/f72 zero size 2024-09-16T00:26:04.012 INFO:tasks.workunit.client.1.smithi169.stdout:6/631: rmdir d2/d22/d51 39 2024-09-16T00:26:04.013 INFO:tasks.workunit.client.0.smithi112.stdout:6/556: symlink dd/d15/lb5 0 2024-09-16T00:26:04.015 INFO:tasks.workunit.client.1.smithi169.stdout:2/400: symlink d7/d1c/d26/l87 0 2024-09-16T00:26:04.017 INFO:tasks.workunit.client.0.smithi112.stdout:4/648: link d6/d39/d36/d5c/d7e/f95 d6/db/d1f/d41/d49/d31/fed 0 2024-09-16T00:26:04.018 INFO:tasks.workunit.client.0.smithi112.stdout:3/595: write d2/d9/f1e [8039867,88223] 0 2024-09-16T00:26:04.019 INFO:tasks.workunit.client.0.smithi112.stdout:6/557: getdents dd/df/d91 0 2024-09-16T00:26:04.020 INFO:tasks.workunit.client.0.smithi112.stdout:6/558: readlink dd/df/d13/l8e 0 2024-09-16T00:26:04.021 INFO:tasks.workunit.client.0.smithi112.stdout:4/649: creat d6/db/df/d5e/d61/d64/fee x:0 0 0 2024-09-16T00:26:04.023 INFO:tasks.workunit.client.0.smithi112.stdout:4/650: truncate d6/db/df/f5f 4475672 0 2024-09-16T00:26:04.023 INFO:tasks.workunit.client.0.smithi112.stdout:3/596: rename d2/d9/c64 to d2/d9/d2b/d47/d52/d7a/d80/d83/d68/d90/cca 0 2024-09-16T00:26:04.024 INFO:tasks.workunit.client.1.smithi169.stdout:2/401: dwrite d7/db/d27/d4f/d56/d6d/f71 [0,4194304] 0 2024-09-16T00:26:04.027 INFO:tasks.workunit.client.0.smithi112.stdout:4/651: write d6/d39/d36/d5c/d7e/f95 [4438165,91040] 0 2024-09-16T00:26:04.027 INFO:tasks.workunit.client.0.smithi112.stdout:6/559: dwrite dd/df/d13/fab [0,4194304] 0 2024-09-16T00:26:04.030 INFO:tasks.workunit.client.1.smithi169.stdout:2/402: rename d7/db/c1d to d7/db/d27/d29/d77/c88 0 2024-09-16T00:26:04.031 INFO:tasks.workunit.client.1.smithi169.stdout:6/632: dwrite d2/d36/d48/f90 [0,4194304] 0 2024-09-16T00:26:04.032 INFO:tasks.workunit.client.0.smithi112.stdout:6/560: mkdir dd/df/d13/d7c/d88/db6 0 2024-09-16T00:26:04.033 INFO:tasks.workunit.client.1.smithi169.stdout:2/403: mkdir d7/db/d89 0 2024-09-16T00:26:04.036 INFO:tasks.workunit.client.0.smithi112.stdout:6/561: chown dd/df/d17/d27/d84/l3c 0 1 2024-09-16T00:26:04.037 INFO:tasks.workunit.client.1.smithi169.stdout:2/404: truncate d7/db/d27/d29/f54 15324 0 2024-09-16T00:26:04.038 INFO:tasks.workunit.client.0.smithi112.stdout:4/652: dwrite d6/db/d1f/d94/feb [0,4194304] 0 2024-09-16T00:26:04.039 INFO:tasks.workunit.client.0.smithi112.stdout:6/562: mknod dd/df/d91/cb7 0 2024-09-16T00:26:04.039 INFO:tasks.workunit.client.1.smithi169.stdout:2/405: write d7/db/f45 [9420334,1148] 0 2024-09-16T00:26:04.041 INFO:tasks.workunit.client.1.smithi169.stdout:6/633: dwrite d2/da4/fbf [0,4194304] 0 2024-09-16T00:26:04.043 INFO:tasks.workunit.client.0.smithi112.stdout:4/653: mknod d6/db/d1f/d94/cef 0 2024-09-16T00:26:04.044 INFO:tasks.workunit.client.1.smithi169.stdout:2/406: symlink d7/db/d27/d4f/l8a 0 2024-09-16T00:26:04.045 INFO:tasks.workunit.client.1.smithi169.stdout:2/407: readlink d7/db/d27/d29/l52 0 2024-09-16T00:26:04.046 INFO:tasks.workunit.client.1.smithi169.stdout:6/634: creat d2/d22/da7/fcd x:0 0 0 2024-09-16T00:26:04.047 INFO:tasks.workunit.client.1.smithi169.stdout:2/408: dread - d7/d1c/d26/f76 zero size 2024-09-16T00:26:04.048 INFO:tasks.workunit.client.0.smithi112.stdout:4/654: rename d6/d7c/fd8 to d6/db/df/d5e/d61/ff0 0 2024-09-16T00:26:04.049 INFO:tasks.workunit.client.0.smithi112.stdout:6/563: read dd/df/d10/d47/d55/f72 [2223828,122965] 0 2024-09-16T00:26:04.050 INFO:tasks.workunit.client.1.smithi169.stdout:6/635: symlink d2/da4/db1/dbc/lce 0 2024-09-16T00:26:04.050 INFO:tasks.workunit.client.0.smithi112.stdout:6/564: stat dd/df/d10 0 2024-09-16T00:26:04.051 INFO:tasks.workunit.client.1.smithi169.stdout:2/409: mknod d7/db/d27/c8b 0 2024-09-16T00:26:04.052 INFO:tasks.workunit.client.1.smithi169.stdout:6/636: chown d2/d4d/f49 276877232 1 2024-09-16T00:26:04.052 INFO:tasks.workunit.client.0.smithi112.stdout:6/565: write dd/df/d13/fab [7232768,53638] 0 2024-09-16T00:26:04.052 INFO:tasks.workunit.client.0.smithi112.stdout:6/566: read - dd/d23/d8f/fa8 zero size 2024-09-16T00:26:04.055 INFO:tasks.workunit.client.0.smithi112.stdout:4/655: link d6/db/df/d5e/d61/lae d6/d39/d1c/d66/lf1 0 2024-09-16T00:26:04.056 INFO:tasks.workunit.client.1.smithi169.stdout:6/637: dread d2/d22/f23 [4194304,4194304] 0 2024-09-16T00:26:04.057 INFO:tasks.workunit.client.0.smithi112.stdout:6/567: rename dd/df/d13/l8e to dd/d15/da5/lb8 0 2024-09-16T00:26:04.058 INFO:tasks.workunit.client.0.smithi112.stdout:4/656: creat d6/d39/d1c/d2c/ff2 x:0 0 0 2024-09-16T00:26:04.059 INFO:tasks.workunit.client.0.smithi112.stdout:6/568: truncate dd/df/d10/d47/f9c 4751539 0 2024-09-16T00:26:04.059 INFO:tasks.workunit.client.1.smithi169.stdout:6/638: truncate d2/d22/d51/f77 6746455 0 2024-09-16T00:26:04.060 INFO:tasks.workunit.client.0.smithi112.stdout:4/657: write d6/d39/d1c/d66/faf [308748,3727] 0 2024-09-16T00:26:04.064 INFO:tasks.workunit.client.0.smithi112.stdout:4/658: truncate d6/db/d1f/d41/fb2 968503 0 2024-09-16T00:26:04.065 INFO:tasks.workunit.client.0.smithi112.stdout:3/597: mkdir d2/d9/d2b/d47/d52/d7a/d80/d83/d68/d90/dcb 0 2024-09-16T00:26:04.066 INFO:tasks.workunit.client.0.smithi112.stdout:4/659: unlink d6/d39/d1c/d66/faf 0 2024-09-16T00:26:04.068 INFO:tasks.workunit.client.0.smithi112.stdout:3/598: write d2/fb [1804944,42757] 0 2024-09-16T00:26:04.070 INFO:tasks.workunit.client.0.smithi112.stdout:4/660: creat d6/ff3 x:0 0 0 2024-09-16T00:26:04.070 INFO:tasks.workunit.client.0.smithi112.stdout:3/599: dread - d2/d9/d2b/d47/d52/d7a/d80/d83/d76/f99 zero size 2024-09-16T00:26:04.071 INFO:tasks.workunit.client.0.smithi112.stdout:3/600: fsync d2/d9/d2b/d47/d78/da9/fba 0 2024-09-16T00:26:04.072 INFO:tasks.workunit.client.0.smithi112.stdout:4/661: creat d6/d39/d1c/d2c/ff4 x:0 0 0 2024-09-16T00:26:04.075 INFO:tasks.workunit.client.0.smithi112.stdout:4/662: rmdir d6/d39/d36/d5c/dc6 39 2024-09-16T00:26:04.075 INFO:tasks.workunit.client.0.smithi112.stdout:4/663: dread - d6/d39/d1c/d6e/fbe zero size 2024-09-16T00:26:04.076 INFO:tasks.workunit.client.0.smithi112.stdout:4/664: stat d6/db/df/d5e/d61/d64/f81 0 2024-09-16T00:26:04.077 INFO:tasks.workunit.client.0.smithi112.stdout:3/601: dread d2/d9/d2b/d2c/d69/d92/f96 [0,4194304] 0 2024-09-16T00:26:04.077 INFO:tasks.workunit.client.0.smithi112.stdout:4/665: fdatasync d6/d39/d1c/d2c/ff4 0 2024-09-16T00:26:04.078 INFO:tasks.workunit.client.0.smithi112.stdout:3/602: write d2/f1a [43391,81834] 0 2024-09-16T00:26:04.081 INFO:tasks.workunit.client.0.smithi112.stdout:3/603: creat d2/d9/d2b/d47/d52/d7a/d80/d83/d7b/fcc x:0 0 0 2024-09-16T00:26:04.082 INFO:tasks.workunit.client.0.smithi112.stdout:4/666: dread d6/db/f42 [0,4194304] 0 2024-09-16T00:26:04.082 INFO:tasks.workunit.client.1.smithi169.stdout:6/639: dwrite d2/d4d/d59/d38/f62 [4194304,4194304] 0 2024-09-16T00:26:04.085 INFO:tasks.workunit.client.0.smithi112.stdout:3/604: mknod d2/d9/d2b/d2c/d69/ccd 0 2024-09-16T00:26:04.085 INFO:tasks.workunit.client.1.smithi169.stdout:6/640: fdatasync d2/d4d/d59/d38/f97 0 2024-09-16T00:26:04.088 INFO:tasks.workunit.client.1.smithi169.stdout:6/641: truncate d2/d4d/d59/d38/f4f 216579 0 2024-09-16T00:26:04.088 INFO:tasks.workunit.client.0.smithi112.stdout:3/605: creat d2/d9/d2b/d2c/fce x:0 0 0 2024-09-16T00:26:04.089 INFO:tasks.workunit.client.0.smithi112.stdout:3/606: write d2/d9/d2b/d39/fb1 [109920,2557] 0 2024-09-16T00:26:04.091 INFO:tasks.workunit.client.1.smithi169.stdout:6/642: mknod d2/d4d/d59/ccf 0 2024-09-16T00:26:04.092 INFO:tasks.workunit.client.0.smithi112.stdout:3/607: mkdir d2/d9/d2b/d47/dcf 0 2024-09-16T00:26:04.092 INFO:tasks.workunit.client.1.smithi169.stdout:6/643: mknod d2/d36/cd0 0 2024-09-16T00:26:04.093 INFO:tasks.workunit.client.1.smithi169.stdout:6/644: readlink d2/d3f/d41/d52/lbd 0 2024-09-16T00:26:04.094 INFO:tasks.workunit.client.1.smithi169.stdout:6/645: rmdir d2/d3f/d41/d52 39 2024-09-16T00:26:04.097 INFO:tasks.workunit.client.0.smithi112.stdout:4/667: truncate d6/db/d1f/d41/fb2 873616 0 2024-09-16T00:26:04.100 INFO:tasks.workunit.client.0.smithi112.stdout:3/608: dwrite d2/d9/faa [4194304,4194304] 0 2024-09-16T00:26:04.101 INFO:tasks.workunit.client.1.smithi169.stdout:6/646: dwrite d2/d3f/d95/fc8 [0,4194304] 0 2024-09-16T00:26:04.101 INFO:tasks.workunit.client.0.smithi112.stdout:4/668: creat d6/d39/d1c/d2c/d69/ff5 x:0 0 0 2024-09-16T00:26:04.101 INFO:tasks.workunit.client.0.smithi112.stdout:0/682: sync 2024-09-16T00:26:04.101 INFO:tasks.workunit.client.0.smithi112.stdout:2/701: sync 2024-09-16T00:26:04.101 INFO:tasks.workunit.client.0.smithi112.stdout:9/827: sync 2024-09-16T00:26:04.102 INFO:tasks.workunit.client.0.smithi112.stdout:7/642: sync 2024-09-16T00:26:04.102 INFO:tasks.workunit.client.0.smithi112.stdout:5/515: sync 2024-09-16T00:26:04.102 INFO:tasks.workunit.client.0.smithi112.stdout:1/630: sync 2024-09-16T00:26:04.102 INFO:tasks.workunit.client.1.smithi169.stdout:6/647: stat d2/f45 0 2024-09-16T00:26:04.103 INFO:tasks.workunit.client.1.smithi169.stdout:6/648: dread - d2/d22/fc2 zero size 2024-09-16T00:26:04.106 INFO:tasks.workunit.client.1.smithi169.stdout:6/649: rename d2/da4/db1/dbc/d87/l9a to d2/d3f/d41/d5c/d73/ld1 0 2024-09-16T00:26:04.109 INFO:tasks.workunit.client.0.smithi112.stdout:0/683: read d5/da/d57/f73 [1319420,44575] 0 2024-09-16T00:26:04.110 INFO:tasks.workunit.client.0.smithi112.stdout:3/609: creat d2/d9/d2b/d47/d78/da9/fd0 x:0 0 0 2024-09-16T00:26:04.110 INFO:tasks.workunit.client.0.smithi112.stdout:7/643: creat d6/d10/da1/fd6 x:0 0 0 2024-09-16T00:26:04.113 INFO:tasks.workunit.client.0.smithi112.stdout:1/631: truncate d0/d14/d1c/d26/f69 711557 0 2024-09-16T00:26:04.115 INFO:tasks.workunit.client.0.smithi112.stdout:2/702: unlink d2/dd/d48/d22/d2b/d30/c3f 0 2024-09-16T00:26:04.118 INFO:tasks.workunit.client.0.smithi112.stdout:9/828: dwrite d1/d5/d6/d2d/d39/f96 [0,4194304] 0 2024-09-16T00:26:04.118 INFO:tasks.workunit.client.0.smithi112.stdout:7/644: mkdir d6/d10/d75/d8f/d90/dd7 0 2024-09-16T00:26:04.120 INFO:tasks.workunit.client.0.smithi112.stdout:2/703: chown d2/dd/d48/c4a 14775409 1 2024-09-16T00:26:04.121 INFO:tasks.workunit.client.1.smithi169.stdout:6/650: write d2/d4d/d59/d38/f4f [1259083,76990] 0 2024-09-16T00:26:04.121 INFO:tasks.workunit.client.0.smithi112.stdout:5/516: dwrite d4/dd/d12/f48 [4194304,4194304] 0 2024-09-16T00:26:04.124 INFO:tasks.workunit.client.0.smithi112.stdout:1/632: creat d0/d14/fd0 x:0 0 0 2024-09-16T00:26:04.124 INFO:tasks.workunit.client.1.smithi169.stdout:6/651: creat d2/d3f/d41/d52/fd2 x:0 0 0 2024-09-16T00:26:04.128 INFO:tasks.workunit.client.0.smithi112.stdout:3/610: truncate d2/d9/d2b/d47/d52/d7a/d80/d83/fb5 655103 0 2024-09-16T00:26:04.128 INFO:tasks.workunit.client.0.smithi112.stdout:9/829: symlink d1/d5/d20/d81/dc4/l126 0 2024-09-16T00:26:04.128 INFO:tasks.workunit.client.0.smithi112.stdout:5/517: rmdir d4/d7/d26/d4c 39 2024-09-16T00:26:04.130 INFO:tasks.workunit.client.0.smithi112.stdout:7/645: mknod d6/d10/d75/d8f/d90/dbc/cd8 0 2024-09-16T00:26:04.132 INFO:tasks.workunit.client.0.smithi112.stdout:0/684: dwrite d5/da/d1d/d71/d64/db6/fbd [0,4194304] 0 2024-09-16T00:26:04.133 INFO:tasks.workunit.client.0.smithi112.stdout:9/830: dread - d1/d5/f8b zero size 2024-09-16T00:26:04.134 INFO:tasks.workunit.client.0.smithi112.stdout:3/611: mknod d2/d9/d2b/d47/d52/d7a/d80/d83/d49/cd1 0 2024-09-16T00:26:04.136 INFO:tasks.workunit.client.0.smithi112.stdout:9/831: fdatasync d1/d76/fe2 0 2024-09-16T00:26:04.136 INFO:tasks.workunit.client.0.smithi112.stdout:7/646: unlink d6/d10/d16/d49/fbb 0 2024-09-16T00:26:04.139 INFO:tasks.workunit.client.0.smithi112.stdout:9/832: fsync d1/d5/d6/fa4 0 2024-09-16T00:26:04.140 INFO:tasks.workunit.client.0.smithi112.stdout:1/633: link d0/d14/d6a/fbe d0/d14/d1c/d26/d41/fd1 0 2024-09-16T00:26:04.144 INFO:tasks.workunit.client.0.smithi112.stdout:9/833: chown d1/d5/d20/l58 0 1 2024-09-16T00:26:04.146 INFO:tasks.workunit.client.0.smithi112.stdout:3/612: creat d2/d9/d2b/d47/d52/d7a/d80/d83/d68/d90/dcb/fd2 x:0 0 0 2024-09-16T00:26:04.146 INFO:tasks.workunit.client.0.smithi112.stdout:5/518: dwrite d4/dd/f95 [0,4194304] 0 2024-09-16T00:26:04.151 INFO:tasks.workunit.client.0.smithi112.stdout:9/834: unlink d1/d5/d6/d2d/f97 0 2024-09-16T00:26:04.152 INFO:tasks.workunit.client.0.smithi112.stdout:5/519: unlink d4/d7/d26/l7d 0 2024-09-16T00:26:04.153 INFO:tasks.workunit.client.0.smithi112.stdout:0/685: dwrite d5/da/d1d/d71/f5c [0,4194304] 0 2024-09-16T00:26:04.155 INFO:tasks.workunit.client.0.smithi112.stdout:1/634: truncate d0/d14/d1c/d26/f52 504037 0 2024-09-16T00:26:04.159 INFO:tasks.workunit.client.0.smithi112.stdout:7/647: dwrite d6/d10/d75/d8f/d90/dbc/fc1 [0,4194304] 0 2024-09-16T00:26:04.159 INFO:tasks.workunit.client.0.smithi112.stdout:1/635: dread - d0/d14/d1c/d26/fb9 zero size 2024-09-16T00:26:04.160 INFO:tasks.workunit.client.0.smithi112.stdout:3/613: rename d2/d9/d2b/d2c/d69/f71 to d2/d9/d2b/d47/d52/fd3 0 2024-09-16T00:26:04.162 INFO:tasks.workunit.client.0.smithi112.stdout:5/520: truncate d4/d4f/d78/fa8 549857 0 2024-09-16T00:26:04.165 INFO:tasks.workunit.client.0.smithi112.stdout:5/521: truncate d4/d1c/fad 740381 0 2024-09-16T00:26:04.166 INFO:tasks.workunit.client.0.smithi112.stdout:9/835: truncate d1/d5/d20/d81/f21 664133 0 2024-09-16T00:26:04.170 INFO:tasks.workunit.client.0.smithi112.stdout:1/636: dwrite d0/d14/df/d28/d42/f81 [0,4194304] 0 2024-09-16T00:26:04.177 INFO:tasks.workunit.client.0.smithi112.stdout:2/704: dwrite d2/f46 [0,4194304] 0 2024-09-16T00:26:04.177 INFO:tasks.workunit.client.0.smithi112.stdout:5/522: creat d4/d7/d14/fb0 x:0 0 0 2024-09-16T00:26:04.182 INFO:tasks.workunit.client.0.smithi112.stdout:1/637: dread d0/d14/d1c/d26/f73 [0,4194304] 0 2024-09-16T00:26:04.182 INFO:tasks.workunit.client.0.smithi112.stdout:2/705: read d2/dd/d48/d22/d2b/d54/d79/f9e [3524109,58706] 0 2024-09-16T00:26:04.184 INFO:tasks.workunit.client.0.smithi112.stdout:3/614: symlink d2/d9/d2b/d47/d52/d7a/d80/d95/ld4 0 2024-09-16T00:26:04.186 INFO:tasks.workunit.client.0.smithi112.stdout:5/523: creat d4/d4f/fb1 x:0 0 0 2024-09-16T00:26:04.194 INFO:tasks.workunit.client.0.smithi112.stdout:3/615: dread - d2/d9/d2b/d47/d52/d7a/d80/d83/d68/f88 zero size 2024-09-16T00:26:04.196 INFO:tasks.workunit.client.0.smithi112.stdout:0/686: truncate d5/da/d66/f5d 2737339 0 2024-09-16T00:26:04.197 INFO:tasks.workunit.client.0.smithi112.stdout:5/524: mknod d4/d7/d14/d90/d9a/d7c/cb2 0 2024-09-16T00:26:04.198 INFO:tasks.workunit.client.0.smithi112.stdout:7/648: truncate d6/d10/d16/f4e 218954 0 2024-09-16T00:26:04.199 INFO:tasks.workunit.client.1.smithi169.stdout:5/426: sync 2024-09-16T00:26:04.200 INFO:tasks.workunit.client.0.smithi112.stdout:7/649: write d6/d10/d75/fa5 [906990,70698] 0 2024-09-16T00:26:04.201 INFO:tasks.workunit.client.0.smithi112.stdout:7/650: write f4 [4167464,14843] 0 2024-09-16T00:26:04.202 INFO:tasks.workunit.client.0.smithi112.stdout:0/687: dread d5/da/fd1 [0,4194304] 0 2024-09-16T00:26:04.202 INFO:tasks.workunit.client.0.smithi112.stdout:9/836: rename d1/d5/d6/d2d/d6a/d89/c35 to d1/d5/d110/c127 0 2024-09-16T00:26:04.204 INFO:tasks.workunit.client.1.smithi169.stdout:5/427: write d2/da/d10/d13/d24/f47 [2116984,69095] 0 2024-09-16T00:26:04.204 INFO:tasks.workunit.client.0.smithi112.stdout:9/837: chown d1/d5/d2c/f102 1931079 1 2024-09-16T00:26:04.204 INFO:tasks.workunit.client.0.smithi112.stdout:7/651: stat d6/d10 0 2024-09-16T00:26:04.206 INFO:tasks.workunit.client.0.smithi112.stdout:5/525: read d4/d7/f40 [2998604,47955] 0 2024-09-16T00:26:04.209 INFO:tasks.workunit.client.0.smithi112.stdout:9/838: fdatasync d1/d5/d6/d2d/d6a/fe6 0 2024-09-16T00:26:04.210 INFO:tasks.workunit.client.0.smithi112.stdout:0/688: dread - d5/da/d21/f51 zero size 2024-09-16T00:26:04.212 INFO:tasks.workunit.client.0.smithi112.stdout:5/526: truncate d4/d7/d14/d90/fa4 19248 0 2024-09-16T00:26:04.213 INFO:tasks.workunit.client.0.smithi112.stdout:0/689: chown d5/da/f2e 984 1 2024-09-16T00:26:04.215 INFO:tasks.workunit.client.0.smithi112.stdout:7/652: symlink d6/d10/da1/dcd/ld9 0 2024-09-16T00:26:04.216 INFO:tasks.workunit.client.0.smithi112.stdout:1/638: dread d0/d14/d1c/d26/d41/fc0 [0,4194304] 0 2024-09-16T00:26:04.217 INFO:tasks.workunit.client.0.smithi112.stdout:2/706: dwrite d2/dd/d48/d22/d2b/d54/d79/f7a [0,4194304] 0 2024-09-16T00:26:04.218 INFO:tasks.workunit.client.0.smithi112.stdout:5/527: chown d4/d4f/d77/c80 4 1 2024-09-16T00:26:04.221 INFO:tasks.workunit.client.0.smithi112.stdout:3/616: dread d2/d9/d2b/d47/d52/d7a/d80/fb7 [0,4194304] 0 2024-09-16T00:26:04.221 INFO:tasks.workunit.client.0.smithi112.stdout:2/707: rmdir d2/dd/d96 39 2024-09-16T00:26:04.223 INFO:tasks.workunit.client.0.smithi112.stdout:9/839: dread d1/d5/d20/d81/fd5 [0,4194304] 0 2024-09-16T00:26:04.223 INFO:tasks.workunit.client.0.smithi112.stdout:0/690: dread - d5/da/d18/d63/db2/fca zero size 2024-09-16T00:26:04.224 INFO:tasks.workunit.client.0.smithi112.stdout:1/639: stat d0/d14/d1c/d20/cac 0 2024-09-16T00:26:04.227 INFO:tasks.workunit.client.0.smithi112.stdout:3/617: unlink d2/d9/d2b/l87 0 2024-09-16T00:26:04.230 INFO:tasks.workunit.client.0.smithi112.stdout:9/840: write d1/d5/d6/d2d/fea [650090,71367] 0 2024-09-16T00:26:04.230 INFO:tasks.workunit.client.0.smithi112.stdout:7/653: creat d6/fda x:0 0 0 2024-09-16T00:26:04.232 INFO:tasks.workunit.client.0.smithi112.stdout:1/640: dwrite d0/d14/d1c/d20/f53 [0,4194304] 0 2024-09-16T00:26:04.232 INFO:tasks.workunit.client.0.smithi112.stdout:0/691: mkdir d5/da/d18/d63/db2/dd4 0 2024-09-16T00:26:04.232 INFO:tasks.workunit.client.0.smithi112.stdout:5/528: mkdir d4/d7/d26/d4c/db3 0 2024-09-16T00:26:04.234 INFO:tasks.workunit.client.0.smithi112.stdout:3/618: read d2/d9/d2b/d47/d52/d7a/fc3 [374101,130668] 0 2024-09-16T00:26:04.234 INFO:tasks.workunit.client.0.smithi112.stdout:9/841: chown d1/d5/d20/d81/d1a/d88/f7e 455792 1 2024-09-16T00:26:04.235 INFO:tasks.workunit.client.0.smithi112.stdout:0/692: write d5/da/d57/db0/f9e [968303,50714] 0 2024-09-16T00:26:04.235 INFO:tasks.workunit.client.0.smithi112.stdout:7/654: truncate d6/d10/d16/d96/dc7/fb6 274537 0 2024-09-16T00:26:04.235 INFO:tasks.workunit.client.0.smithi112.stdout:5/529: unlink d4/dd/d12/l53 0 2024-09-16T00:26:04.236 INFO:tasks.workunit.client.0.smithi112.stdout:1/641: write d0/d14/df/d28/d42/f81 [1181044,104268] 0 2024-09-16T00:26:04.237 INFO:tasks.workunit.client.0.smithi112.stdout:3/619: chown d2/d9/d2b/d2c/f42 127207352 1 2024-09-16T00:26:04.237 INFO:tasks.workunit.client.0.smithi112.stdout:5/530: readlink d4/d1c/d43/l50 0 2024-09-16T00:26:04.241 INFO:tasks.workunit.client.0.smithi112.stdout:9/842: write d1/d5/d20/d81/fe5 [329186,25925] 0 2024-09-16T00:26:04.241 INFO:tasks.workunit.client.0.smithi112.stdout:1/642: stat d0/d14/d1c/d26/fb3 0 2024-09-16T00:26:04.241 INFO:tasks.workunit.client.0.smithi112.stdout:5/531: readlink d4/d7/l55 0 2024-09-16T00:26:04.243 INFO:tasks.workunit.client.0.smithi112.stdout:0/693: read d5/f95 [173497,91526] 0 2024-09-16T00:26:04.246 INFO:tasks.workunit.client.0.smithi112.stdout:7/655: dwrite d6/fb2 [0,4194304] 0 2024-09-16T00:26:04.247 INFO:tasks.workunit.client.0.smithi112.stdout:9/843: getdents d1/d5/d2c 0 2024-09-16T00:26:04.248 INFO:tasks.workunit.client.0.smithi112.stdout:1/643: write d0/d14/df/d28/d45/d46/d58/d62/db0/fc7 [1013086,20702] 0 2024-09-16T00:26:04.249 INFO:tasks.workunit.client.0.smithi112.stdout:0/694: mkdir d5/da/d21/dc3/dd3/dd5 0 2024-09-16T00:26:04.256 INFO:tasks.workunit.client.0.smithi112.stdout:5/532: dwrite d4/d7/d14/d90/fa4 [0,4194304] 0 2024-09-16T00:26:04.258 INFO:tasks.workunit.client.0.smithi112.stdout:9/844: link d1/d5/d20/d81/fe5 d1/d5/d20/d81/dff/f128 0 2024-09-16T00:26:04.259 INFO:tasks.workunit.client.0.smithi112.stdout:3/620: truncate d2/d9/d2b/d47/d52/f38 327744 0 2024-09-16T00:26:04.263 INFO:tasks.workunit.client.0.smithi112.stdout:0/695: creat d5/da/d18/d24/d46/d5f/d7a/d8d/fd6 x:0 0 0 2024-09-16T00:26:04.263 INFO:tasks.workunit.client.0.smithi112.stdout:1/644: rename d0/f18 to d0/d14/d1c/d26/d47/d85/fd2 0 2024-09-16T00:26:04.265 INFO:tasks.workunit.client.0.smithi112.stdout:3/621: chown d2/d9/d2b/d47/d78/fc0 470779 1 2024-09-16T00:26:04.269 INFO:tasks.workunit.client.0.smithi112.stdout:1/645: creat d0/d14/df/d28/d45/d46/fd3 x:0 0 0 2024-09-16T00:26:04.270 INFO:tasks.workunit.client.0.smithi112.stdout:1/646: chown d0/d14/d1c/d20 53644 1 2024-09-16T00:26:04.270 INFO:tasks.workunit.client.0.smithi112.stdout:3/622: mkdir d2/d9/d2b/d2c/d69/dd5 0 2024-09-16T00:26:04.271 INFO:tasks.workunit.client.0.smithi112.stdout:0/696: dread d5/da/d18/d89/f9b [0,4194304] 0 2024-09-16T00:26:04.274 INFO:tasks.workunit.client.0.smithi112.stdout:0/697: read d5/da/dc/f7d [2414616,27961] 0 2024-09-16T00:26:04.276 INFO:tasks.workunit.client.0.smithi112.stdout:0/698: mkdir d5/d90/dc5/dd7 0 2024-09-16T00:26:04.278 INFO:tasks.workunit.client.0.smithi112.stdout:0/699: truncate d5/da/d18/d24/d5a/f82 578998 0 2024-09-16T00:26:04.279 INFO:tasks.workunit.client.0.smithi112.stdout:0/700: truncate d5/da/d21/f7c 4908152 0 2024-09-16T00:26:04.280 INFO:tasks.workunit.client.0.smithi112.stdout:0/701: chown d5/da/f2e 228 1 2024-09-16T00:26:04.283 INFO:tasks.workunit.client.0.smithi112.stdout:5/533: dread d4/dd/d12/f11 [0,4194304] 0 2024-09-16T00:26:04.284 INFO:tasks.workunit.client.0.smithi112.stdout:1/647: rename d0/d14/d1c/d26 to d0/d14/df/d28/d45/dd4 0 2024-09-16T00:26:04.284 INFO:tasks.workunit.client.0.smithi112.stdout:5/534: write d4/d4f/f76 [452779,23304] 0 2024-09-16T00:26:04.288 INFO:tasks.workunit.client.0.smithi112.stdout:1/648: creat d0/fd5 x:0 0 0 2024-09-16T00:26:04.291 INFO:tasks.workunit.client.0.smithi112.stdout:9/845: dwrite d1/d5/d2c/d6f/f8e [0,4194304] 0 2024-09-16T00:26:04.293 INFO:tasks.workunit.client.0.smithi112.stdout:5/535: getdents d4/dd 0 2024-09-16T00:26:04.293 INFO:tasks.workunit.client.0.smithi112.stdout:3/623: dread d2/d9/d2b/d47/d52/d7a/d80/d83/d76/d8e/fb6 [0,4194304] 0 2024-09-16T00:26:04.298 INFO:tasks.workunit.client.0.smithi112.stdout:5/536: unlink d4/d1c/l3d 0 2024-09-16T00:26:04.298 INFO:tasks.workunit.client.0.smithi112.stdout:9/846: write d1/d5/d6/d18/fc5 [4235005,127223] 0 2024-09-16T00:26:04.300 INFO:tasks.workunit.client.0.smithi112.stdout:3/624: fdatasync d2/d9/d2b/d47/d52/d7a/d80/d83/d49/f4c 0 2024-09-16T00:26:04.301 INFO:tasks.workunit.client.0.smithi112.stdout:1/649: dwrite d0/d14/df/d28/d45/f98 [0,4194304] 0 2024-09-16T00:26:04.301 INFO:tasks.workunit.client.0.smithi112.stdout:5/537: read - d4/d1c/f73 zero size 2024-09-16T00:26:04.303 INFO:tasks.workunit.client.0.smithi112.stdout:9/847: chown d1/d5/d20/d81/d1a/d88/da8/fc8 14869077 1 2024-09-16T00:26:04.303 INFO:tasks.workunit.client.0.smithi112.stdout:1/650: chown d0/d14/df/d28/d45/dd4/d41/l55 0 1 2024-09-16T00:26:04.306 INFO:tasks.workunit.client.0.smithi112.stdout:3/625: truncate d2/d9/d2b/d47/f54 2876371 0 2024-09-16T00:26:04.307 INFO:tasks.workunit.client.0.smithi112.stdout:1/651: truncate d0/d14/df/d28/f32 4287611 0 2024-09-16T00:26:04.307 INFO:tasks.workunit.client.0.smithi112.stdout:5/538: rename d4/d1c/l7b to d4/dd/d12/lb4 0 2024-09-16T00:26:04.308 INFO:tasks.workunit.client.0.smithi112.stdout:9/848: mkdir d1/d5/d20/d81/d1a/d88/def/d129 0 2024-09-16T00:26:04.309 INFO:tasks.workunit.client.0.smithi112.stdout:5/539: creat d4/d7/d14/d90/fb5 x:0 0 0 2024-09-16T00:26:04.310 INFO:tasks.workunit.client.0.smithi112.stdout:1/652: chown d0/d14/df/d28/d45/dd4/d47/d85 183726 1 2024-09-16T00:26:04.310 INFO:tasks.workunit.client.0.smithi112.stdout:9/849: chown d1/d76/f79 63438 1 2024-09-16T00:26:04.311 INFO:tasks.workunit.client.0.smithi112.stdout:3/626: symlink d2/d9/d2b/ld6 0 2024-09-16T00:26:04.312 INFO:tasks.workunit.client.0.smithi112.stdout:1/653: write d0/d14/d6a/fbd [119203,70809] 0 2024-09-16T00:26:04.312 INFO:tasks.workunit.client.0.smithi112.stdout:3/627: fsync d2/d4d/fb3 0 2024-09-16T00:26:04.313 INFO:tasks.workunit.client.0.smithi112.stdout:5/540: mkdir d4/d7/d26/d4c/db3/db6 0 2024-09-16T00:26:04.314 INFO:tasks.workunit.client.0.smithi112.stdout:1/654: write d0/d14/df/d28/d45/d30/f4a [1536134,3835] 0 2024-09-16T00:26:04.315 INFO:tasks.workunit.client.0.smithi112.stdout:1/655: readlink d0/d14/d1c/d20/l54 0 2024-09-16T00:26:04.316 INFO:tasks.workunit.client.0.smithi112.stdout:1/656: read - d0/d14/df/d28/d45/d46/d58/d62/fa0 zero size 2024-09-16T00:26:04.317 INFO:tasks.workunit.client.0.smithi112.stdout:1/657: read d0/d14/d6a/f80 [835363,30090] 0 2024-09-16T00:26:04.318 INFO:tasks.workunit.client.0.smithi112.stdout:1/658: fdatasync d0/d14/df/d28/d45/d46/d58/d62/f88 0 2024-09-16T00:26:04.320 INFO:tasks.workunit.client.1.smithi169.stdout:4/450: sync 2024-09-16T00:26:04.320 INFO:tasks.workunit.client.1.smithi169.stdout:0/453: sync 2024-09-16T00:26:04.323 INFO:tasks.workunit.client.1.smithi169.stdout:4/451: mkdir d2/d12/d13/d90 0 2024-09-16T00:26:04.325 INFO:tasks.workunit.client.1.smithi169.stdout:0/454: link d0/d2/c3c d0/ca2 0 2024-09-16T00:26:04.326 INFO:tasks.workunit.client.1.smithi169.stdout:4/452: mknod d2/d12/c91 0 2024-09-16T00:26:04.326 INFO:tasks.workunit.client.1.smithi169.stdout:0/455: mknod d0/d2c/d38/ca3 0 2024-09-16T00:26:04.327 INFO:tasks.workunit.client.1.smithi169.stdout:0/456: readlink d0/d2/d15/l19 0 2024-09-16T00:26:04.328 INFO:tasks.workunit.client.1.smithi169.stdout:4/453: symlink d2/d12/d13/d1f/d3e/d5b/d70/d7d/l92 0 2024-09-16T00:26:04.328 INFO:tasks.workunit.client.1.smithi169.stdout:0/457: write d0/d2c/d54/f3b [3786195,84336] 0 2024-09-16T00:26:04.331 INFO:tasks.workunit.client.1.smithi169.stdout:0/458: mkdir d0/d2c/d54/d21/d31/da4 0 2024-09-16T00:26:04.333 INFO:tasks.workunit.client.1.smithi169.stdout:0/459: fdatasync d0/d2/f6 0 2024-09-16T00:26:04.338 INFO:tasks.workunit.client.1.smithi169.stdout:0/460: mkdir d0/d2c/d4f/da5 0 2024-09-16T00:26:04.338 INFO:tasks.workunit.client.1.smithi169.stdout:4/454: link d2/d12/l14 d2/d12/d13/d1f/d3e/d5b/l93 0 2024-09-16T00:26:04.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:04 smithi112 ceph-mon[27869]: pgmap v255: 65 pgs: 65 active+clean; 2.0 GiB data, 7.3 GiB used, 529 GiB / 536 GiB avail; 63 MiB/s rd, 263 MiB/s wr, 484 op/s 2024-09-16T00:26:04.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:04 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:26:04.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:04 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:26:04.344 INFO:tasks.workunit.client.1.smithi169.stdout:4/455: truncate d2/d12/d13/f2e 635008 0 2024-09-16T00:26:04.348 INFO:tasks.workunit.client.1.smithi169.stdout:4/456: dwrite d2/d12/d13/d1f/f48 [0,4194304] 0 2024-09-16T00:26:04.359 INFO:tasks.workunit.client.0.smithi112.stdout:8/553: sync 2024-09-16T00:26:04.374 INFO:tasks.workunit.client.0.smithi112.stdout:3/628: fsync d2/d9/d2b/d2c/d69/d92/f96 0 2024-09-16T00:26:04.377 INFO:tasks.workunit.client.0.smithi112.stdout:9/850: getdents d1/d5/d20/d81/dc4 0 2024-09-16T00:26:04.380 INFO:tasks.workunit.client.0.smithi112.stdout:5/541: write d4/dd/d6f/fab [3094595,111210] 0 2024-09-16T00:26:04.381 INFO:tasks.workunit.client.0.smithi112.stdout:5/542: chown d4/d7/d87/c89 122 1 2024-09-16T00:26:04.381 INFO:tasks.workunit.client.0.smithi112.stdout:8/554: symlink d10/d4e/d8c/d98/lb8 0 2024-09-16T00:26:04.381 INFO:tasks.workunit.client.0.smithi112.stdout:1/659: write d0/d14/df/d28/d45/d46/f6d [1157022,41643] 0 2024-09-16T00:26:04.382 INFO:tasks.workunit.client.0.smithi112.stdout:5/543: chown d4/dd/f24 1673528479 1 2024-09-16T00:26:04.384 INFO:tasks.workunit.client.0.smithi112.stdout:3/629: write d2/d9/d2b/d47/d52/d7a/d80/d83/d68/f88 [268908,38165] 0 2024-09-16T00:26:04.385 INFO:tasks.workunit.client.0.smithi112.stdout:9/851: creat d1/d5/d6/f12a x:0 0 0 2024-09-16T00:26:04.386 INFO:tasks.workunit.client.0.smithi112.stdout:5/544: read d4/dd/d6f/fab [3618705,130909] 0 2024-09-16T00:26:04.387 INFO:tasks.workunit.client.0.smithi112.stdout:1/660: rename d0/d14/d44/l6e to d0/d14/df/d28/d89/ld6 0 2024-09-16T00:26:04.388 INFO:tasks.workunit.client.0.smithi112.stdout:8/555: creat d10/d12/fb9 x:0 0 0 2024-09-16T00:26:04.388 INFO:tasks.workunit.client.0.smithi112.stdout:9/852: mkdir d1/d5/d20/d81/d11d/d12b 0 2024-09-16T00:26:04.389 INFO:tasks.workunit.client.0.smithi112.stdout:5/545: mkdir d4/dd/db7 0 2024-09-16T00:26:04.390 INFO:tasks.workunit.client.0.smithi112.stdout:9/853: fdatasync d1/d5/d6/d2d/d6a/fe6 0 2024-09-16T00:26:04.390 INFO:tasks.workunit.client.0.smithi112.stdout:1/661: mknod d0/d14/df/d28/d42/cd7 0 2024-09-16T00:26:04.392 INFO:tasks.workunit.client.0.smithi112.stdout:3/630: getdents d2/d9/d2b 0 2024-09-16T00:26:04.393 INFO:tasks.workunit.client.0.smithi112.stdout:9/854: creat d1/d5/d6/df8/ddd/f12c x:0 0 0 2024-09-16T00:26:04.394 INFO:tasks.workunit.client.0.smithi112.stdout:1/662: mkdir d0/d14/df/d28/d42/dd8 0 2024-09-16T00:26:04.395 INFO:tasks.workunit.client.0.smithi112.stdout:1/663: dread - d0/d14/d1c/d20/daf/f84 zero size 2024-09-16T00:26:04.397 INFO:tasks.workunit.client.0.smithi112.stdout:1/664: dread - d0/d14/df/d28/d45/d46/d58/d62/f88 zero size 2024-09-16T00:26:04.401 INFO:tasks.workunit.client.0.smithi112.stdout:8/556: write d10/d4e/d58/f73 [662255,13429] 0 2024-09-16T00:26:04.401 INFO:tasks.workunit.client.0.smithi112.stdout:5/546: chown d4/dd/f95 508 1 2024-09-16T00:26:04.404 INFO:tasks.workunit.client.0.smithi112.stdout:9/855: write d1/d5/d20/d81/f1d [3065241,60136] 0 2024-09-16T00:26:04.404 INFO:tasks.workunit.client.0.smithi112.stdout:8/557: mknod d10/d12/cba 0 2024-09-16T00:26:04.405 INFO:tasks.workunit.client.0.smithi112.stdout:5/547: read d4/dd/f27 [3949141,36152] 0 2024-09-16T00:26:04.411 INFO:tasks.workunit.client.0.smithi112.stdout:5/548: write d4/d7/f4d [1765954,34016] 0 2024-09-16T00:26:04.411 INFO:tasks.workunit.client.0.smithi112.stdout:3/631: dwrite d2/d9/d2b/d47/d52/fa3 [0,4194304] 0 2024-09-16T00:26:04.413 INFO:tasks.workunit.client.0.smithi112.stdout:1/665: dread d0/d14/df/d28/f32 [0,4194304] 0 2024-09-16T00:26:04.413 INFO:tasks.workunit.client.0.smithi112.stdout:5/549: chown d4/d1c/d43 221466386 1 2024-09-16T00:26:04.421 INFO:tasks.workunit.client.0.smithi112.stdout:3/632: write d2/d9/d2b/d47/d52/d7a/d80/d83/d68/f88 [1223389,47251] 0 2024-09-16T00:26:04.421 INFO:tasks.workunit.client.0.smithi112.stdout:9/856: dread d1/d5/d6/d2d/d6a/d89/f90 [0,4194304] 0 2024-09-16T00:26:04.421 INFO:tasks.workunit.client.0.smithi112.stdout:3/633: chown d2/d9/l27 6 1 2024-09-16T00:26:04.421 INFO:tasks.workunit.client.0.smithi112.stdout:1/666: mknod d0/d14/df/d28/d45/d46/da7/cd9 0 2024-09-16T00:26:04.422 INFO:tasks.workunit.client.0.smithi112.stdout:3/634: write d2/d9/d2b/d47/d52/d7a/d80/d83/d49/f4c [9387573,68371] 0 2024-09-16T00:26:04.422 INFO:tasks.workunit.client.0.smithi112.stdout:9/857: mkdir d1/d5/d20/d81/d1a/d88/def/d66/d12d 0 2024-09-16T00:26:04.423 INFO:tasks.workunit.client.0.smithi112.stdout:1/667: mkdir d0/d14/df/d28/d45/d46/da7/dda 0 2024-09-16T00:26:04.425 INFO:tasks.workunit.client.0.smithi112.stdout:9/858: mknod d1/d5/d6/d114/d116/c12e 0 2024-09-16T00:26:04.427 INFO:tasks.workunit.client.0.smithi112.stdout:8/558: dread d10/d12/f16 [0,4194304] 0 2024-09-16T00:26:04.427 INFO:tasks.workunit.client.0.smithi112.stdout:1/668: creat d0/d14/df/d28/d45/fdb x:0 0 0 2024-09-16T00:26:04.428 INFO:tasks.workunit.client.0.smithi112.stdout:9/859: creat d1/d5/d2c/f12f x:0 0 0 2024-09-16T00:26:04.428 INFO:tasks.workunit.client.0.smithi112.stdout:8/559: rmdir d10/d4e/d8c 39 2024-09-16T00:26:04.429 INFO:tasks.workunit.client.0.smithi112.stdout:1/669: chown d0/d14/df/d28/d45/d46/da7/cd9 282753597 1 2024-09-16T00:26:04.429 INFO:tasks.workunit.client.0.smithi112.stdout:6/569: sync 2024-09-16T00:26:04.429 INFO:tasks.workunit.client.0.smithi112.stdout:4/669: sync 2024-09-16T00:26:04.433 INFO:tasks.workunit.client.0.smithi112.stdout:1/670: chown d0/d14/df/d28/d45/d46/c9c 1689935 1 2024-09-16T00:26:04.433 INFO:tasks.workunit.client.0.smithi112.stdout:4/670: creat d6/db/d1f/d41/d49/db1/ff6 x:0 0 0 2024-09-16T00:26:04.435 INFO:tasks.workunit.client.0.smithi112.stdout:5/550: truncate d4/d7/d14/f33 2894764 0 2024-09-16T00:26:04.436 INFO:tasks.workunit.client.0.smithi112.stdout:8/560: truncate d10/d12/d1c/d61/fb4 1163325 0 2024-09-16T00:26:04.437 INFO:tasks.workunit.client.0.smithi112.stdout:9/860: dwrite d1/d5/d2c/fc3 [4194304,4194304] 0 2024-09-16T00:26:04.438 INFO:tasks.workunit.client.0.smithi112.stdout:8/561: chown d10/d12/d24 5948679 1 2024-09-16T00:26:04.439 INFO:tasks.workunit.client.0.smithi112.stdout:6/570: rmdir dd/df/d13/d7c/d88/db6 0 2024-09-16T00:26:04.439 INFO:tasks.workunit.client.0.smithi112.stdout:1/671: truncate d0/d14/df/d28/d45/dd4/f8d 2026826 0 2024-09-16T00:26:04.441 INFO:tasks.workunit.client.0.smithi112.stdout:5/551: chown d4/dd/d49/d70/f7e 14994 1 2024-09-16T00:26:04.444 INFO:tasks.workunit.client.0.smithi112.stdout:4/671: getdents d6/d39/d1c/d2c/d69/d7b/de0 0 2024-09-16T00:26:04.445 INFO:tasks.workunit.client.0.smithi112.stdout:8/562: mkdir d10/d12/d1c/d75/dbb 0 2024-09-16T00:26:04.445 INFO:tasks.workunit.client.0.smithi112.stdout:9/861: chown d1/d5/d20/d81/d1a/f7d 516533113 1 2024-09-16T00:26:04.445 INFO:tasks.workunit.client.0.smithi112.stdout:1/672: chown d0/d14/df/d28/d45/d46/da7/fbb 300722 1 2024-09-16T00:26:04.446 INFO:tasks.workunit.client.0.smithi112.stdout:9/862: chown d1/d5/cce 8 1 2024-09-16T00:26:04.447 INFO:tasks.workunit.client.0.smithi112.stdout:8/563: fdatasync d10/d12/d1c/d3b/f89 0 2024-09-16T00:26:04.448 INFO:tasks.workunit.client.0.smithi112.stdout:3/635: dwrite d2/f19 [0,4194304] 0 2024-09-16T00:26:04.448 INFO:tasks.workunit.client.0.smithi112.stdout:1/673: write d0/fd5 [174517,97974] 0 2024-09-16T00:26:04.450 INFO:tasks.workunit.client.0.smithi112.stdout:8/564: chown d10/d12/d1c/d3b/fa8 131016529 1 2024-09-16T00:26:04.452 INFO:tasks.workunit.client.0.smithi112.stdout:3/636: readlink d2/d9/l61 0 2024-09-16T00:26:04.455 INFO:tasks.workunit.client.0.smithi112.stdout:1/674: symlink d0/d14/df/ldc 0 2024-09-16T00:26:04.455 INFO:tasks.workunit.client.0.smithi112.stdout:8/565: readlink d10/d4e/d8c/d98/lb8 0 2024-09-16T00:26:04.456 INFO:tasks.workunit.client.0.smithi112.stdout:3/637: read d2/fb [8163087,46083] 0 2024-09-16T00:26:04.460 INFO:tasks.workunit.client.0.smithi112.stdout:4/672: dwrite d6/d39/d1c/d66/fac [0,4194304] 0 2024-09-16T00:26:04.461 INFO:tasks.workunit.client.0.smithi112.stdout:1/675: write d0/d14/d1c/d20/f31 [4222436,39665] 0 2024-09-16T00:26:04.464 INFO:tasks.workunit.client.0.smithi112.stdout:8/566: rmdir d10/d4e/d8c/d7c/da5 39 2024-09-16T00:26:04.467 INFO:tasks.workunit.client.0.smithi112.stdout:3/638: truncate d2/d9/d2b/d2c/d69/f6f 198161 0 2024-09-16T00:26:04.470 INFO:tasks.workunit.client.0.smithi112.stdout:9/863: link d1/d5/d6/d18/ca0 d1/d5/d20/d81/d1a/d88/c130 0 2024-09-16T00:26:04.470 INFO:tasks.workunit.client.0.smithi112.stdout:3/639: readlink d2/d4d/lc4 0 2024-09-16T00:26:04.470 INFO:tasks.workunit.client.0.smithi112.stdout:8/567: symlink d10/d4e/d58/lbc 0 2024-09-16T00:26:04.471 INFO:tasks.workunit.client.0.smithi112.stdout:5/552: dwrite d4/d7/d26/f44 [0,4194304] 0 2024-09-16T00:26:04.472 INFO:tasks.workunit.client.0.smithi112.stdout:3/640: chown d2/d9/d2b/f82 5 1 2024-09-16T00:26:04.474 INFO:tasks.workunit.client.0.smithi112.stdout:8/568: fdatasync d10/d12/d1c/d61/f84 0 2024-09-16T00:26:04.475 INFO:tasks.workunit.client.0.smithi112.stdout:6/571: dread dd/df/f66 [0,4194304] 0 2024-09-16T00:26:04.476 INFO:tasks.workunit.client.0.smithi112.stdout:3/641: mknod d2/d4d/cd7 0 2024-09-16T00:26:04.479 INFO:tasks.workunit.client.0.smithi112.stdout:6/572: symlink dd/d15/da5/lb9 0 2024-09-16T00:26:04.480 INFO:tasks.workunit.client.0.smithi112.stdout:1/676: dwrite d0/d14/d1c/d20/d93/fa3 [4194304,4194304] 0 2024-09-16T00:26:04.485 INFO:tasks.workunit.client.0.smithi112.stdout:5/553: creat d4/d7/d26/d4c/db3/db6/fb8 x:0 0 0 2024-09-16T00:26:04.486 INFO:tasks.workunit.client.0.smithi112.stdout:8/569: symlink d10/d12/d1c/d75/dbb/lbd 0 2024-09-16T00:26:04.487 INFO:tasks.workunit.client.0.smithi112.stdout:6/573: symlink dd/df/d10/d47/d55/lba 0 2024-09-16T00:26:04.491 INFO:tasks.workunit.client.0.smithi112.stdout:8/570: readlink d10/d12/l5b 0 2024-09-16T00:26:04.492 INFO:tasks.workunit.client.0.smithi112.stdout:4/673: dread d6/db/df/f59 [0,4194304] 0 2024-09-16T00:26:04.494 INFO:tasks.workunit.client.0.smithi112.stdout:9/864: dwrite d1/d5/d6/d59/fec [0,4194304] 0 2024-09-16T00:26:04.495 INFO:tasks.workunit.client.0.smithi112.stdout:1/677: link d0/d14/d1c/c3c d0/d14/d44/cdd 0 2024-09-16T00:26:04.496 INFO:tasks.workunit.client.0.smithi112.stdout:9/865: chown d1/d5/d20/d81/d1a/d88/ce7 8 1 2024-09-16T00:26:04.498 INFO:tasks.workunit.client.0.smithi112.stdout:6/574: link dd/df/d13/f48 dd/df/d10/d47/d89/fbb 0 2024-09-16T00:26:04.499 INFO:tasks.workunit.client.0.smithi112.stdout:3/642: getdents d2/d9/d2b/d47/d52/d7a/d80/d83/d49 0 2024-09-16T00:26:04.502 INFO:tasks.workunit.client.0.smithi112.stdout:1/678: mknod d0/d14/df/d28/cde 0 2024-09-16T00:26:04.503 INFO:tasks.workunit.client.0.smithi112.stdout:1/679: chown d0/d14/d1c/l50 403809811 1 2024-09-16T00:26:04.504 INFO:tasks.workunit.client.0.smithi112.stdout:6/575: write dd/d15/da5/d79/fb3 [4163172,81955] 0 2024-09-16T00:26:04.505 INFO:tasks.workunit.client.0.smithi112.stdout:1/680: chown d0/d14/d1c/c5f 48 1 2024-09-16T00:26:04.505 INFO:tasks.workunit.client.0.smithi112.stdout:4/674: unlink d6/d39/d1c/d66/lf1 0 2024-09-16T00:26:04.508 INFO:tasks.workunit.client.0.smithi112.stdout:3/643: rename d2/d9/d2b/d47/d78/fc0 to d2/d9/d2b/d2c/d69/dd5/fd8 0 2024-09-16T00:26:04.508 INFO:tasks.workunit.client.0.smithi112.stdout:5/554: write d4/f16 [87774,110980] 0 2024-09-16T00:26:04.510 INFO:tasks.workunit.client.1.smithi169.stdout:7/503: sync 2024-09-16T00:26:04.510 INFO:tasks.workunit.client.1.smithi169.stdout:7/504: read - d7/d71/f88 zero size 2024-09-16T00:26:04.511 INFO:tasks.workunit.client.0.smithi112.stdout:6/576: unlink dd/df/d11/f67 0 2024-09-16T00:26:04.512 INFO:tasks.workunit.client.0.smithi112.stdout:3/644: chown d2/d9/d2b/d47/d78/da9/fd0 24919 1 2024-09-16T00:26:04.512 INFO:tasks.workunit.client.0.smithi112.stdout:4/675: symlink d6/d39/lf7 0 2024-09-16T00:26:04.512 INFO:tasks.workunit.client.1.smithi169.stdout:7/505: unlink d7/d71/d76/d29/c7d 0 2024-09-16T00:26:04.513 INFO:tasks.workunit.client.0.smithi112.stdout:5/555: mkdir d4/d7/d26/d4c/db9 0 2024-09-16T00:26:04.515 INFO:tasks.workunit.client.0.smithi112.stdout:1/681: dwrite d0/d14/d1c/d20/fa1 [0,4194304] 0 2024-09-16T00:26:04.518 INFO:tasks.workunit.client.0.smithi112.stdout:6/577: fsync dd/df/d10/f8b 0 2024-09-16T00:26:04.518 INFO:tasks.workunit.client.0.smithi112.stdout:4/676: write d6/d39/d1c/d6e/d6f/fe5 [430387,87619] 0 2024-09-16T00:26:04.519 INFO:tasks.workunit.client.1.smithi169.stdout:7/506: truncate d7/f26 875916 0 2024-09-16T00:26:04.520 INFO:tasks.workunit.client.0.smithi112.stdout:4/677: fsync d6/d39/d36/f8c 0 2024-09-16T00:26:04.521 INFO:tasks.workunit.client.1.smithi169.stdout:7/507: creat d7/d71/d76/d29/d78/fa3 x:0 0 0 2024-09-16T00:26:04.521 INFO:tasks.workunit.client.1.smithi169.stdout:7/508: chown d7/d16/d49/d84/f8d 1 1 2024-09-16T00:26:04.522 INFO:tasks.workunit.client.0.smithi112.stdout:9/866: write d1/d5/d20/d81/d1a/d88/def/f1c [1251458,12834] 0 2024-09-16T00:26:04.523 INFO:tasks.workunit.client.1.smithi169.stdout:7/509: symlink d7/d5b/d61/d98/la4 0 2024-09-16T00:26:04.525 INFO:tasks.workunit.client.0.smithi112.stdout:5/556: mknod d4/dd/d49/d70/d98/cba 0 2024-09-16T00:26:04.525 INFO:tasks.workunit.client.0.smithi112.stdout:8/571: dread d10/d12/f8f [0,4194304] 0 2024-09-16T00:26:04.525 INFO:tasks.workunit.client.1.smithi169.stdout:7/510: write d7/df/d95/f7a [3836397,105709] 0 2024-09-16T00:26:04.527 INFO:tasks.workunit.client.1.smithi169.stdout:7/511: chown d7/d16/l54 132391 1 2024-09-16T00:26:04.527 INFO:tasks.workunit.client.1.smithi169.stdout:7/512: readlink d7/d71/d76/d29/l3b 0 2024-09-16T00:26:04.528 INFO:tasks.workunit.client.0.smithi112.stdout:6/578: getdents dd/df/d17/da2 0 2024-09-16T00:26:04.528 INFO:tasks.workunit.client.0.smithi112.stdout:1/682: link d0/d14/fd0 d0/d14/df/fdf 0 2024-09-16T00:26:04.531 INFO:tasks.workunit.client.1.smithi169.stdout:7/513: creat d7/df/fa5 x:0 0 0 2024-09-16T00:26:04.531 INFO:tasks.workunit.client.1.smithi169.stdout:7/514: stat d7/c34 0 2024-09-16T00:26:04.532 INFO:tasks.workunit.client.0.smithi112.stdout:3/645: dwrite d2/d9/d2b/d47/d52/d7a/d80/fb7 [0,4194304] 0 2024-09-16T00:26:04.532 INFO:tasks.workunit.client.1.smithi169.stdout:7/515: write d7/d16/d49/d84/f8d [1455904,79851] 0 2024-09-16T00:26:04.535 INFO:tasks.workunit.client.0.smithi112.stdout:2/708: sync 2024-09-16T00:26:04.536 INFO:tasks.workunit.client.1.smithi169.stdout:7/516: getdents d7/df/d73 0 2024-09-16T00:26:04.537 INFO:tasks.workunit.client.0.smithi112.stdout:8/572: truncate d10/d12/d1c/d75/f80 903409 0 2024-09-16T00:26:04.538 INFO:tasks.workunit.client.0.smithi112.stdout:4/678: dwrite d6/d39/d1c/d66/f76 [4194304,4194304] 0 2024-09-16T00:26:04.538 INFO:tasks.workunit.client.1.smithi169.stdout:7/517: unlink d7/d16/d49/d84/c8f 0 2024-09-16T00:26:04.538 INFO:tasks.workunit.client.1.smithi169.stdout:7/518: chown d7/d71/d76/c65 51 1 2024-09-16T00:26:04.539 INFO:tasks.workunit.client.1.smithi169.stdout:7/519: truncate d7/d71/f88 805053 0 2024-09-16T00:26:04.541 INFO:tasks.workunit.client.1.smithi169.stdout:7/520: creat d7/df/fa6 x:0 0 0 2024-09-16T00:26:04.542 INFO:tasks.workunit.client.1.smithi169.stdout:7/521: dread - d7/d71/d76/d29/d78/fa3 zero size 2024-09-16T00:26:04.543 INFO:tasks.workunit.client.1.smithi169.stdout:7/522: dread - d7/df/f44 zero size 2024-09-16T00:26:04.547 INFO:tasks.workunit.client.0.smithi112.stdout:4/679: chown d6/db/df/f59 1007766919 1 2024-09-16T00:26:04.547 INFO:tasks.workunit.client.1.smithi169.stdout:7/523: creat d7/d71/d76/d9c/fa7 x:0 0 0 2024-09-16T00:26:04.547 INFO:tasks.workunit.client.1.smithi169.stdout:7/524: chown d7/d71/d76/fc 256 1 2024-09-16T00:26:04.548 INFO:tasks.workunit.client.0.smithi112.stdout:9/867: dwrite d1/d5/d2c/d6f/ff3 [4194304,4194304] 0 2024-09-16T00:26:04.548 INFO:tasks.workunit.client.0.smithi112.stdout:8/573: creat d10/d4e/d58/fbe x:0 0 0 2024-09-16T00:26:04.551 INFO:tasks.workunit.client.1.smithi169.stdout:7/525: creat d7/d5b/d61/d98/d99/fa8 x:0 0 0 2024-09-16T00:26:04.551 INFO:tasks.workunit.client.0.smithi112.stdout:2/709: symlink d2/dd/d48/d8a/db9/ldd 0 2024-09-16T00:26:04.552 INFO:tasks.workunit.client.0.smithi112.stdout:5/557: dwrite d4/d7/f35 [0,4194304] 0 2024-09-16T00:26:04.553 INFO:tasks.workunit.client.0.smithi112.stdout:3/646: symlink d2/d9/d2b/d47/d52/d7a/d80/d83/d68/ld9 0 2024-09-16T00:26:04.554 INFO:tasks.workunit.client.1.smithi169.stdout:7/526: creat d7/d71/d76/d9c/fa9 x:0 0 0 2024-09-16T00:26:04.556 INFO:tasks.workunit.client.0.smithi112.stdout:6/579: dwrite dd/df/d13/d7c/f98 [0,4194304] 0 2024-09-16T00:26:04.557 INFO:tasks.workunit.client.0.smithi112.stdout:4/680: write d6/d39/d1c/d66/f76 [5553911,55662] 0 2024-09-16T00:26:04.557 INFO:tasks.workunit.client.0.smithi112.stdout:1/683: dread d0/d14/df/d28/d42/f81 [0,4194304] 0 2024-09-16T00:26:04.561 INFO:tasks.workunit.client.0.smithi112.stdout:5/558: readlink d4/d7/d26/d4c/l7a 0 2024-09-16T00:26:04.562 INFO:tasks.workunit.client.0.smithi112.stdout:2/710: mknod d2/dd/d48/d94/cde 0 2024-09-16T00:26:04.563 INFO:tasks.workunit.client.0.smithi112.stdout:3/647: mkdir d2/d9/d2b/d2c/d69/d92/dda 0 2024-09-16T00:26:04.564 INFO:tasks.workunit.client.0.smithi112.stdout:4/681: mkdir d6/db/df/d5e/d61/d64/df8 0 2024-09-16T00:26:04.566 INFO:tasks.workunit.client.0.smithi112.stdout:5/559: mkdir d4/d7/d14/dbb 0 2024-09-16T00:26:04.567 INFO:tasks.workunit.client.0.smithi112.stdout:8/574: truncate d10/d12/d24/f29 1445634 0 2024-09-16T00:26:04.569 INFO:tasks.workunit.client.0.smithi112.stdout:5/560: fdatasync d4/f57 0 2024-09-16T00:26:04.570 INFO:tasks.workunit.client.0.smithi112.stdout:1/684: rename d0/d14/df/d28/d89 to d0/d14/df/d28/d45/dd4/d41/de0 0 2024-09-16T00:26:04.572 INFO:tasks.workunit.client.0.smithi112.stdout:4/682: creat d6/db/df/d5e/d61/d64/d6a/ff9 x:0 0 0 2024-09-16T00:26:04.572 INFO:tasks.workunit.client.0.smithi112.stdout:1/685: chown d0/d14/d1c/d20/c2a 17292038 1 2024-09-16T00:26:04.574 INFO:tasks.workunit.client.0.smithi112.stdout:5/561: write d4/d7/d14/d90/d9a/d7c/f88 [3163290,95979] 0 2024-09-16T00:26:04.575 INFO:tasks.workunit.client.0.smithi112.stdout:4/683: readlink d6/ld 0 2024-09-16T00:26:04.575 INFO:tasks.workunit.client.0.smithi112.stdout:3/648: fsync d2/d9/d2b/d47/d52/f38 0 2024-09-16T00:26:04.577 INFO:tasks.workunit.client.0.smithi112.stdout:1/686: write d0/d14/d1c/f91 [112084,106388] 0 2024-09-16T00:26:04.579 INFO:tasks.workunit.client.0.smithi112.stdout:2/711: dwrite d2/f64 [0,4194304] 0 2024-09-16T00:26:04.581 INFO:tasks.workunit.client.0.smithi112.stdout:8/575: rename d10/d4e/d8c/d7c/da5/cb6 to d10/d4e/cbf 0 2024-09-16T00:26:04.585 INFO:tasks.workunit.client.1.smithi169.stdout:7/527: write d7/d71/d76/fc [3124850,40793] 0 2024-09-16T00:26:04.586 INFO:tasks.workunit.client.0.smithi112.stdout:5/562: rename d4/d7/d26/f44 to d4/d7/d26/d4c/db3/db6/fbc 0 2024-09-16T00:26:04.587 INFO:tasks.workunit.client.0.smithi112.stdout:1/687: creat d0/d14/fe1 x:0 0 0 2024-09-16T00:26:04.590 INFO:tasks.workunit.client.0.smithi112.stdout:2/712: link d2/dd/d48/d22/d2b/d54/d60/d6c/d88/dad/fb8 d2/dd/d48/d8a/fdf 0 2024-09-16T00:26:04.593 INFO:tasks.workunit.client.0.smithi112.stdout:0/702: sync 2024-09-16T00:26:04.593 INFO:tasks.workunit.client.0.smithi112.stdout:7/656: sync 2024-09-16T00:26:04.595 INFO:tasks.workunit.client.0.smithi112.stdout:1/688: creat d0/d14/df/d28/d45/dd4/d41/de0/fe2 x:0 0 0 2024-09-16T00:26:04.595 INFO:tasks.workunit.client.0.smithi112.stdout:9/868: write d1/d5/d6/d18/f28 [4810043,39125] 0 2024-09-16T00:26:04.597 INFO:tasks.workunit.client.0.smithi112.stdout:1/689: write d0/d14/df/d28/d45/dd4/fb3 [387840,12378] 0 2024-09-16T00:26:04.598 INFO:tasks.workunit.client.0.smithi112.stdout:0/703: rmdir d5/da/d21 39 2024-09-16T00:26:04.600 INFO:tasks.workunit.client.0.smithi112.stdout:4/684: rename d6/d39/d1c/d6e/fb0 to d6/d39/d36/d5c/ffa 0 2024-09-16T00:26:04.601 INFO:tasks.workunit.client.0.smithi112.stdout:7/657: truncate d6/d10/d16/f20 41296 0 2024-09-16T00:26:04.604 INFO:tasks.workunit.client.0.smithi112.stdout:6/580: dwrite dd/df/d17/d27/d54/f56 [0,4194304] 0 2024-09-16T00:26:04.605 INFO:tasks.workunit.client.0.smithi112.stdout:4/685: truncate d6/d39/d1c/d6e/d6f/f97 425366 0 2024-09-16T00:26:04.606 INFO:tasks.workunit.client.0.smithi112.stdout:4/686: fsync d6/d39/d36/d5c/fc0 0 2024-09-16T00:26:04.610 INFO:tasks.workunit.client.0.smithi112.stdout:3/649: dread d2/f24 [0,4194304] 0 2024-09-16T00:26:04.613 INFO:tasks.workunit.client.0.smithi112.stdout:6/581: mkdir dd/df/d10/d47/d89/dbc 0 2024-09-16T00:26:04.613 INFO:tasks.workunit.client.0.smithi112.stdout:8/576: write d10/d12/d1c/f38 [1202646,90139] 0 2024-09-16T00:26:04.615 INFO:tasks.workunit.client.0.smithi112.stdout:4/687: unlink d6/d39/d1c/d2c/d69/d7b/l96 0 2024-09-16T00:26:04.617 INFO:tasks.workunit.client.0.smithi112.stdout:9/869: creat d1/d5/d6/d59/f131 x:0 0 0 2024-09-16T00:26:04.619 INFO:tasks.workunit.client.0.smithi112.stdout:3/650: creat d2/d9/d2b/d47/d52/d7a/d80/d83/d68/d90/fdb x:0 0 0 2024-09-16T00:26:04.620 INFO:tasks.workunit.client.0.smithi112.stdout:6/582: fdatasync dd/df/d17/d27/d54/f56 0 2024-09-16T00:26:04.620 INFO:tasks.workunit.client.1.smithi169.stdout:9/482: sync 2024-09-16T00:26:04.620 INFO:tasks.workunit.client.1.smithi169.stdout:3/476: sync 2024-09-16T00:26:04.621 INFO:tasks.workunit.client.1.smithi169.stdout:9/483: write d5/da/d5b/d6c/da0/f8c [965244,1383] 0 2024-09-16T00:26:04.622 INFO:tasks.workunit.client.0.smithi112.stdout:8/577: creat d10/d12/d1c/d75/dbb/fc0 x:0 0 0 2024-09-16T00:26:04.623 INFO:tasks.workunit.client.1.smithi169.stdout:9/484: readlink d5/da/d5b/d91/la5 0 2024-09-16T00:26:04.623 INFO:tasks.workunit.client.0.smithi112.stdout:1/690: link d0/d14/df/d28/d45/dd4/d47/l3f d0/d14/d6a/le3 0 2024-09-16T00:26:04.624 INFO:tasks.workunit.client.1.smithi169.stdout:3/477: rename d7/d50/f97 to d7/d1a/d1c/d23/d70/fb7 0 2024-09-16T00:26:04.625 INFO:tasks.workunit.client.0.smithi112.stdout:5/563: dwrite d4/dd/f27 [0,4194304] 0 2024-09-16T00:26:04.626 INFO:tasks.workunit.client.1.smithi169.stdout:9/485: rename d5/da/d21/d23/f41 to d5/da/d21/d23/fa9 0 2024-09-16T00:26:04.627 INFO:tasks.workunit.client.0.smithi112.stdout:2/713: truncate d2/dd/d48/d22/d2b/f34 335382 0 2024-09-16T00:26:04.627 INFO:tasks.workunit.client.1.smithi169.stdout:3/478: getdents d7/d1a/d1c/d23/d88/d9b 0 2024-09-16T00:26:04.628 INFO:tasks.workunit.client.0.smithi112.stdout:7/658: creat d6/d10/d16/d96/dc7/fdb x:0 0 0 2024-09-16T00:26:04.629 INFO:tasks.workunit.client.0.smithi112.stdout:9/870: symlink d1/d5/d20/d81/d1a/l132 0 2024-09-16T00:26:04.629 INFO:tasks.workunit.client.1.smithi169.stdout:9/486: rename d5/da/d21/d23/f3b to d5/da/d5b/d6f/faa 0 2024-09-16T00:26:04.630 INFO:tasks.workunit.client.0.smithi112.stdout:2/714: write d2/dd/d48/d22/d2b/fa0 [4044914,126926] 0 2024-09-16T00:26:04.631 INFO:tasks.workunit.client.0.smithi112.stdout:0/704: truncate d5/ff 1245993 0 2024-09-16T00:26:04.632 INFO:tasks.workunit.client.0.smithi112.stdout:6/583: mkdir dd/df/d10/d47/d55/dbd 0 2024-09-16T00:26:04.633 INFO:tasks.workunit.client.0.smithi112.stdout:8/578: rmdir d10 39 2024-09-16T00:26:04.634 INFO:tasks.workunit.client.1.smithi169.stdout:9/487: getdents d5/da/d21/d8d 0 2024-09-16T00:26:04.636 INFO:tasks.workunit.client.0.smithi112.stdout:1/691: dwrite d0/d14/d1c/d20/fa1 [4194304,4194304] 0 2024-09-16T00:26:04.637 INFO:tasks.workunit.client.0.smithi112.stdout:5/564: mkdir d4/dd/d49/d70/d83/dbd 0 2024-09-16T00:26:04.638 INFO:tasks.workunit.client.1.smithi169.stdout:9/488: mknod d5/da/d2e/cab 0 2024-09-16T00:26:04.638 INFO:tasks.workunit.client.1.smithi169.stdout:9/489: truncate d5/da/f4d 791034 0 2024-09-16T00:26:04.640 INFO:tasks.workunit.client.1.smithi169.stdout:9/490: chown d5/da/d21/d23/f40 2982 1 2024-09-16T00:26:04.641 INFO:tasks.workunit.client.0.smithi112.stdout:7/659: mkdir d6/d10/d75/d8f/d90/ddc 0 2024-09-16T00:26:04.641 INFO:tasks.workunit.client.0.smithi112.stdout:1/692: chown d0/d14/df/d28/d45/dd4/fb9 229 1 2024-09-16T00:26:04.641 INFO:tasks.workunit.client.1.smithi169.stdout:9/491: write d5/da/d5b/d6f/fa1 [7826,16186] 0 2024-09-16T00:26:04.641 INFO:tasks.workunit.client.1.smithi169.stdout:9/492: readlink d5/la8 0 2024-09-16T00:26:04.646 INFO:tasks.workunit.client.0.smithi112.stdout:6/584: rmdir dd/df/d11/d44/da4 39 2024-09-16T00:26:04.646 INFO:tasks.workunit.client.0.smithi112.stdout:7/660: fsync d6/d10/d75/d8f/d90/dbc/fc1 0 2024-09-16T00:26:04.647 INFO:tasks.workunit.client.0.smithi112.stdout:9/871: dwrite d1/d5/d6/d107/f113 [0,4194304] 0 2024-09-16T00:26:04.647 INFO:tasks.workunit.client.0.smithi112.stdout:5/565: creat d4/d1c/fbe x:0 0 0 2024-09-16T00:26:04.648 INFO:tasks.workunit.client.1.smithi169.stdout:9/493: link d5/da/d2e/d47/f6b d5/da/d5b/fac 0 2024-09-16T00:26:04.648 INFO:tasks.workunit.client.0.smithi112.stdout:2/715: rmdir d2/dd/d48/d22/d67/d9a 39 2024-09-16T00:26:04.651 INFO:tasks.workunit.client.0.smithi112.stdout:0/705: rmdir d5/da/d21 39 2024-09-16T00:26:04.655 INFO:tasks.workunit.client.1.smithi169.stdout:3/479: dread d7/d1a/f32 [0,4194304] 0 2024-09-16T00:26:04.656 INFO:tasks.workunit.client.0.smithi112.stdout:7/661: rmdir d6/d10/da1/dcd 39 2024-09-16T00:26:04.657 INFO:tasks.workunit.client.1.smithi169.stdout:3/480: write d7/d1a/d2b/f8c [70452,83719] 0 2024-09-16T00:26:04.658 INFO:tasks.workunit.client.1.smithi169.stdout:3/481: fsync d7/d1a/d1c/d23/d70/f95 0 2024-09-16T00:26:04.660 INFO:tasks.workunit.client.0.smithi112.stdout:4/688: getdents d6/d39/d36/d5c/dc6 0 2024-09-16T00:26:04.662 INFO:tasks.workunit.client.0.smithi112.stdout:8/579: truncate d10/d12/d1c/d61/d6c/f7a 98608 0 2024-09-16T00:26:04.662 INFO:tasks.workunit.client.0.smithi112.stdout:3/651: dread d2/d9/d2b/d2c/f63 [0,4194304] 0 2024-09-16T00:26:04.663 INFO:tasks.workunit.client.0.smithi112.stdout:3/652: chown d2/d9/d2b/d47/la6 11 1 2024-09-16T00:26:04.665 INFO:tasks.workunit.client.0.smithi112.stdout:1/693: dread d0/d14/df/d28/d45/d46/d58/d62/f74 [0,4194304] 0 2024-09-16T00:26:04.666 INFO:tasks.workunit.client.0.smithi112.stdout:1/694: chown d0/d14/d1c/f5c 2 1 2024-09-16T00:26:04.667 INFO:tasks.workunit.client.1.smithi169.stdout:9/494: dwrite d5/da/d21/d23/f2d [4194304,4194304] 0 2024-09-16T00:26:04.668 INFO:tasks.workunit.client.0.smithi112.stdout:1/695: chown d0/d14/d1c/cc3 468 1 2024-09-16T00:26:04.668 INFO:tasks.workunit.client.0.smithi112.stdout:8/580: symlink d10/d4e/d58/lc1 0 2024-09-16T00:26:04.669 INFO:tasks.workunit.client.1.smithi169.stdout:9/495: write d5/da/d73/f75 [9379538,56640] 0 2024-09-16T00:26:04.674 INFO:tasks.workunit.client.0.smithi112.stdout:3/653: rmdir d2/d9/d2b/d47/d52/d7a/d80 39 2024-09-16T00:26:04.674 INFO:tasks.workunit.client.1.smithi169.stdout:9/496: rename d5/da/d21/f44 to d5/da/d2e/d47/fad 0 2024-09-16T00:26:04.677 INFO:tasks.workunit.client.1.smithi169.stdout:9/497: dread - d5/d49/d70/f71 zero size 2024-09-16T00:26:04.678 INFO:tasks.workunit.client.0.smithi112.stdout:2/716: truncate d2/dd/d48/d22/d2b/d54/d60/daf/fbc 2746768 0 2024-09-16T00:26:04.678 INFO:tasks.workunit.client.0.smithi112.stdout:6/585: getdents dd/df/d13/d7c 0 2024-09-16T00:26:04.681 INFO:tasks.workunit.client.0.smithi112.stdout:8/581: mkdir d10/d12/d1c/d75/dc2 0 2024-09-16T00:26:04.685 INFO:tasks.workunit.client.0.smithi112.stdout:2/717: creat d2/dd/d48/d8a/fe0 x:0 0 0 2024-09-16T00:26:04.689 INFO:tasks.workunit.client.0.smithi112.stdout:0/706: dwrite d5/da/dc/f13 [0,4194304] 0 2024-09-16T00:26:04.690 INFO:tasks.workunit.client.0.smithi112.stdout:8/582: write d10/d12/d1c/d75/f78 [357510,7958] 0 2024-09-16T00:26:04.691 INFO:tasks.workunit.client.0.smithi112.stdout:3/654: dwrite d2/d9/d2b/d2c/f51 [0,4194304] 0 2024-09-16T00:26:04.694 INFO:tasks.workunit.client.0.smithi112.stdout:2/718: mkdir d2/dd/d96/de1 0 2024-09-16T00:26:04.696 INFO:tasks.workunit.client.0.smithi112.stdout:4/689: dwrite d6/db/df/f59 [0,4194304] 0 2024-09-16T00:26:04.697 INFO:tasks.workunit.client.0.smithi112.stdout:7/662: dread d6/d10/d16/d96/dc7/f24 [0,4194304] 0 2024-09-16T00:26:04.700 INFO:tasks.workunit.client.1.smithi169.stdout:9/498: dread d5/da/d4e/f89 [0,4194304] 0 2024-09-16T00:26:04.701 INFO:tasks.workunit.client.0.smithi112.stdout:6/586: rename dd/df/d11/d44/l6c to dd/df/d17/d27/d54/lbe 0 2024-09-16T00:26:04.701 INFO:tasks.workunit.client.0.smithi112.stdout:8/583: creat d10/d4e/d58/d87/fc3 x:0 0 0 2024-09-16T00:26:04.702 INFO:tasks.workunit.client.0.smithi112.stdout:1/696: dwrite d0/d14/d1c/d20/fa9 [0,4194304] 0 2024-09-16T00:26:04.702 INFO:tasks.workunit.client.0.smithi112.stdout:2/719: chown d2/dd/d48/d94/d49/f7d 709823550 1 2024-09-16T00:26:04.704 INFO:tasks.workunit.client.1.smithi169.stdout:9/499: truncate d5/da/d5b/fac 434338 0 2024-09-16T00:26:04.704 INFO:tasks.workunit.client.1.smithi169.stdout:9/500: chown d5/da/d4e/f6d 55475 1 2024-09-16T00:26:04.705 INFO:tasks.workunit.client.0.smithi112.stdout:4/690: dread - d6/d39/d36/d5c/d7e/fa5 zero size 2024-09-16T00:26:04.706 INFO:tasks.workunit.client.1.smithi169.stdout:9/501: write d5/da/d21/d23/f2d [10021943,6341] 0 2024-09-16T00:26:04.708 INFO:tasks.workunit.client.0.smithi112.stdout:6/587: read dd/df/d17/d27/d54/f6a [99274,113534] 0 2024-09-16T00:26:04.710 INFO:tasks.workunit.client.0.smithi112.stdout:2/720: read - d2/dd/d96/fcd zero size 2024-09-16T00:26:04.712 INFO:tasks.workunit.client.0.smithi112.stdout:0/707: getdents d5/da/d21/dc3/dd3/dd5 0 2024-09-16T00:26:04.716 INFO:tasks.workunit.client.0.smithi112.stdout:4/691: symlink d6/d39/d1c/d2c/d69/d7b/lfb 0 2024-09-16T00:26:04.717 INFO:tasks.workunit.client.0.smithi112.stdout:0/708: stat d5/da/d18/d24/d46/d5f 0 2024-09-16T00:26:04.719 INFO:tasks.workunit.client.0.smithi112.stdout:1/697: rmdir d0/d14/d44/dc4 0 2024-09-16T00:26:04.719 INFO:tasks.workunit.client.0.smithi112.stdout:7/663: dwrite d6/fae [0,4194304] 0 2024-09-16T00:26:04.720 INFO:tasks.workunit.client.0.smithi112.stdout:2/721: rename d2/dd/l4f to d2/dd/d48/d22/le2 0 2024-09-16T00:26:04.722 INFO:tasks.workunit.client.0.smithi112.stdout:1/698: stat d0/d14/d44 0 2024-09-16T00:26:04.722 INFO:tasks.workunit.client.0.smithi112.stdout:4/692: dread d6/db/d1f/d41/fb2 [0,4194304] 0 2024-09-16T00:26:04.723 INFO:tasks.workunit.client.0.smithi112.stdout:0/709: mknod d5/da/dc/cd8 0 2024-09-16T00:26:04.723 INFO:tasks.workunit.client.0.smithi112.stdout:1/699: chown d0/d14/df/d28/cde 2006 1 2024-09-16T00:26:04.727 INFO:tasks.workunit.client.0.smithi112.stdout:0/710: stat d5/da/f27 0 2024-09-16T00:26:04.727 INFO:tasks.workunit.client.0.smithi112.stdout:0/711: stat d5/da 0 2024-09-16T00:26:04.730 INFO:tasks.workunit.client.0.smithi112.stdout:4/693: symlink d6/db/d1f/d41/lfc 0 2024-09-16T00:26:04.730 INFO:tasks.workunit.client.0.smithi112.stdout:4/694: fsync d6/db/f60 0 2024-09-16T00:26:04.731 INFO:tasks.workunit.client.0.smithi112.stdout:1/700: mknod d0/d14/df/d28/d45/d46/d58/ce4 0 2024-09-16T00:26:04.734 INFO:tasks.workunit.client.0.smithi112.stdout:1/701: write d0/d14/df/d28/d45/dd4/fb3 [947594,38676] 0 2024-09-16T00:26:04.734 INFO:tasks.workunit.client.0.smithi112.stdout:4/695: write d6/d39/d36/d5c/d7e/fa5 [310261,97408] 0 2024-09-16T00:26:04.736 INFO:tasks.workunit.client.0.smithi112.stdout:1/702: dread - d0/d14/d6a/d9e/fab zero size 2024-09-16T00:26:04.742 INFO:tasks.workunit.client.0.smithi112.stdout:1/703: creat d0/fe5 x:0 0 0 2024-09-16T00:26:04.742 INFO:tasks.workunit.client.0.smithi112.stdout:8/584: dread d10/d12/d24/f2e [0,4194304] 0 2024-09-16T00:26:04.744 INFO:tasks.workunit.client.0.smithi112.stdout:3/655: truncate d2/d9/d2b/d47/d52/d7a/d80/d83/d49/f77 4840066 0 2024-09-16T00:26:04.745 INFO:tasks.workunit.client.0.smithi112.stdout:7/664: write d6/d10/d75/d8f/fc5 [763144,98785] 0 2024-09-16T00:26:04.745 INFO:tasks.workunit.client.0.smithi112.stdout:2/722: truncate d2/f8d 1063590 0 2024-09-16T00:26:04.747 INFO:tasks.workunit.client.0.smithi112.stdout:8/585: dread d10/d12/d1c/d75/f80 [0,4194304] 0 2024-09-16T00:26:04.748 INFO:tasks.workunit.client.0.smithi112.stdout:7/665: write d6/fae [4051758,59745] 0 2024-09-16T00:26:04.750 INFO:tasks.workunit.client.0.smithi112.stdout:1/704: read d0/d14/f8 [6237390,86171] 0 2024-09-16T00:26:04.752 INFO:tasks.workunit.client.0.smithi112.stdout:0/712: truncate d5/f7b 1153457 0 2024-09-16T00:26:04.753 INFO:tasks.workunit.client.0.smithi112.stdout:6/588: dread dd/df/f3d [0,4194304] 0 2024-09-16T00:26:04.753 INFO:tasks.workunit.client.0.smithi112.stdout:1/705: fsync d0/d14/d1c/d20/f31 0 2024-09-16T00:26:04.754 INFO:tasks.workunit.client.0.smithi112.stdout:3/656: creat d2/d9/d2b/d47/d52/d7a/d80/d83/d76/fdc x:0 0 0 2024-09-16T00:26:04.755 INFO:tasks.workunit.client.0.smithi112.stdout:8/586: rmdir d10/d4e/d58/d87 39 2024-09-16T00:26:04.756 INFO:tasks.workunit.client.0.smithi112.stdout:3/657: truncate d2/f2d 5787278 0 2024-09-16T00:26:04.758 INFO:tasks.workunit.client.0.smithi112.stdout:7/666: creat d6/d10/d16/d96/dc7/d39/fdd x:0 0 0 2024-09-16T00:26:04.761 INFO:tasks.workunit.client.0.smithi112.stdout:6/589: write dd/df/d10/d47/f5b [3392515,91843] 0 2024-09-16T00:26:04.763 INFO:tasks.workunit.client.0.smithi112.stdout:0/713: unlink d5/da/d18/d24/d46/d5f/d7a/d8d/c92 0 2024-09-16T00:26:04.764 INFO:tasks.workunit.client.0.smithi112.stdout:8/587: creat d10/d12/d1c/d61/d6c/fc4 x:0 0 0 2024-09-16T00:26:04.768 INFO:tasks.workunit.client.1.smithi169.stdout:8/429: sync 2024-09-16T00:26:04.768 INFO:tasks.workunit.client.1.smithi169.stdout:1/470: sync 2024-09-16T00:26:04.768 INFO:tasks.workunit.client.0.smithi112.stdout:1/706: mknod d0/d14/df/ce6 0 2024-09-16T00:26:04.769 INFO:tasks.workunit.client.1.smithi169.stdout:1/471: write dc/fd [5294898,107031] 0 2024-09-16T00:26:04.770 INFO:tasks.workunit.client.1.smithi169.stdout:8/430: rmdir d7/dc/d17/d24 39 2024-09-16T00:26:04.771 INFO:tasks.workunit.client.0.smithi112.stdout:2/723: dwrite d2/dd/d48/d22/d67/f74 [0,4194304] 0 2024-09-16T00:26:04.771 INFO:tasks.workunit.client.0.smithi112.stdout:3/658: symlink d2/d9/ldd 0 2024-09-16T00:26:04.774 INFO:tasks.workunit.client.0.smithi112.stdout:7/667: dwrite d6/d10/d16/d96/dc7/d39/f92 [0,4194304] 0 2024-09-16T00:26:04.774 INFO:tasks.workunit.client.0.smithi112.stdout:6/590: rename dd/d23/cb4 to dd/df/d17/d27/d54/cbf 0 2024-09-16T00:26:04.775 INFO:tasks.workunit.client.1.smithi169.stdout:1/472: unlink dc/d36/d3a/d45/c57 0 2024-09-16T00:26:04.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:04 smithi169 ceph-mon[34653]: pgmap v255: 65 pgs: 65 active+clean; 2.0 GiB data, 7.3 GiB used, 529 GiB / 536 GiB avail; 63 MiB/s rd, 263 MiB/s wr, 484 op/s 2024-09-16T00:26:04.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:04 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:26:04.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:04 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:26:04.777 INFO:tasks.workunit.client.1.smithi169.stdout:1/473: dread - dc/d18/d67/d69/f7d zero size 2024-09-16T00:26:04.778 INFO:tasks.workunit.client.1.smithi169.stdout:8/431: dwrite d7/dc/d17/d48/f60 [0,4194304] 0 2024-09-16T00:26:04.779 INFO:tasks.workunit.client.0.smithi112.stdout:8/588: dread d10/d12/f16 [0,4194304] 0 2024-09-16T00:26:04.782 INFO:tasks.workunit.client.0.smithi112.stdout:6/591: creat dd/d23/fc0 x:0 0 0 2024-09-16T00:26:04.782 INFO:tasks.workunit.client.1.smithi169.stdout:8/432: creat d7/dc/d1d/f91 x:0 0 0 2024-09-16T00:26:04.784 INFO:tasks.workunit.client.0.smithi112.stdout:0/714: getdents d5/da/d18/d63/db2/dd4 0 2024-09-16T00:26:04.786 INFO:tasks.workunit.client.1.smithi169.stdout:8/433: getdents d7/d8c 0 2024-09-16T00:26:04.788 INFO:tasks.workunit.client.1.smithi169.stdout:1/474: dwrite dc/d18/d67/d69/f80 [0,4194304] 0 2024-09-16T00:26:04.788 INFO:tasks.workunit.client.0.smithi112.stdout:0/715: truncate d5/da/d1d/d71/d64/f8b 186122 0 2024-09-16T00:26:04.790 INFO:tasks.workunit.client.1.smithi169.stdout:8/434: read d7/dc/d1d/d3c/d77/f7b [5166,27313] 0 2024-09-16T00:26:04.791 INFO:tasks.workunit.client.0.smithi112.stdout:6/592: dwrite dd/d23/f35 [0,4194304] 0 2024-09-16T00:26:04.792 INFO:tasks.workunit.client.1.smithi169.stdout:8/435: write d7/dc/d1d/f47 [1395441,16439] 0 2024-09-16T00:26:04.793 INFO:tasks.workunit.client.0.smithi112.stdout:7/668: symlink d6/d10/d16/d96/dc7/dbf/lde 0 2024-09-16T00:26:04.793 INFO:tasks.workunit.client.0.smithi112.stdout:6/593: chown dd/df/d17/d27/d84 211792 1 2024-09-16T00:26:04.795 INFO:tasks.workunit.client.0.smithi112.stdout:6/594: write dd/df/f6e [3241496,111240] 0 2024-09-16T00:26:04.795 INFO:tasks.workunit.client.0.smithi112.stdout:7/669: chown d6/d10/d16/f34 1308253 1 2024-09-16T00:26:04.797 INFO:tasks.workunit.client.1.smithi169.stdout:8/436: mkdir d7/dc/d26/d41/d92 0 2024-09-16T00:26:04.797 INFO:tasks.workunit.client.1.smithi169.stdout:1/475: dwrite dc/d36/d3a/d45/d29/d2a/f62 [4194304,4194304] 0 2024-09-16T00:26:04.798 INFO:tasks.workunit.client.0.smithi112.stdout:8/589: fsync d10/d4e/d8c/fab 0 2024-09-16T00:26:04.799 INFO:tasks.workunit.client.1.smithi169.stdout:1/476: chown dc/d36/d3a/d45/f23 246054 1 2024-09-16T00:26:04.799 INFO:tasks.workunit.client.1.smithi169.stdout:1/477: dread - dc/d26/d5d/f6d zero size 2024-09-16T00:26:04.804 INFO:tasks.workunit.client.1.smithi169.stdout:1/478: rename dc/d36/d3a/d45/l13 to dc/d18/d67/l8f 0 2024-09-16T00:26:04.805 INFO:tasks.workunit.client.0.smithi112.stdout:0/716: mknod d5/d90/dc5/dd7/cd9 0 2024-09-16T00:26:04.807 INFO:tasks.workunit.client.1.smithi169.stdout:1/479: mkdir dc/d26/d90 0 2024-09-16T00:26:04.807 INFO:tasks.workunit.client.0.smithi112.stdout:2/724: dwrite d2/dd/d48/f1d [0,4194304] 0 2024-09-16T00:26:04.808 INFO:tasks.workunit.client.1.smithi169.stdout:1/480: chown dc/d36/d60 4071 1 2024-09-16T00:26:04.808 INFO:tasks.workunit.client.0.smithi112.stdout:1/707: write d0/d14/d6a/f80 [615917,75226] 0 2024-09-16T00:26:04.809 INFO:tasks.workunit.client.0.smithi112.stdout:7/670: getdents d6/d10/d75/d8f/d90/dd7 0 2024-09-16T00:26:04.811 INFO:tasks.workunit.client.0.smithi112.stdout:1/708: truncate d0/d14/d6a/d9e/fab 953743 0 2024-09-16T00:26:04.811 INFO:tasks.workunit.client.0.smithi112.stdout:3/659: write d2/d9/d2b/d47/d52/f38 [1315677,40838] 0 2024-09-16T00:26:04.812 INFO:tasks.workunit.client.0.smithi112.stdout:6/595: link dd/d15/da5/fb0 dd/df/d13/fc1 0 2024-09-16T00:26:04.812 INFO:tasks.workunit.client.1.smithi169.stdout:8/437: write d7/dc/d1d/d80/f8b [954263,46991] 0 2024-09-16T00:26:04.813 INFO:tasks.workunit.client.0.smithi112.stdout:2/725: truncate d2/dd/d48/d22/f29 3432322 0 2024-09-16T00:26:04.813 INFO:tasks.workunit.client.1.smithi169.stdout:1/481: rmdir dc/d36/d60 0 2024-09-16T00:26:04.815 INFO:tasks.workunit.client.0.smithi112.stdout:0/717: symlink d5/da/d57/d76/lda 0 2024-09-16T00:26:04.816 INFO:tasks.workunit.client.1.smithi169.stdout:8/438: unlink d7/dc/d1d/f47 0 2024-09-16T00:26:04.816 INFO:tasks.workunit.client.0.smithi112.stdout:7/671: mkdir d6/d10/d75/d88/ddf 0 2024-09-16T00:26:04.816 INFO:tasks.workunit.client.0.smithi112.stdout:1/709: chown d0/d14/df/c4c 97821 1 2024-09-16T00:26:04.817 INFO:tasks.workunit.client.1.smithi169.stdout:8/439: write d7/d20/f7d [438813,12228] 0 2024-09-16T00:26:04.818 INFO:tasks.workunit.client.0.smithi112.stdout:2/726: truncate d2/dd/d48/d22/d2b/d54/d60/fc7 674556 0 2024-09-16T00:26:04.819 INFO:tasks.workunit.client.0.smithi112.stdout:7/672: creat d6/d10/d16/d49/fe0 x:0 0 0 2024-09-16T00:26:04.820 INFO:tasks.workunit.client.0.smithi112.stdout:3/660: write d2/d9/d2b/d47/d52/d7a/d80/d83/d68/d90/fdb [963711,33046] 0 2024-09-16T00:26:04.820 INFO:tasks.workunit.client.1.smithi169.stdout:1/482: dwrite dc/d2c/f32 [0,4194304] 0 2024-09-16T00:26:04.822 INFO:tasks.workunit.client.1.smithi169.stdout:1/483: chown dc/d2c/d3d/c81 957 1 2024-09-16T00:26:04.822 INFO:tasks.workunit.client.1.smithi169.stdout:1/484: stat dc/d36/d3a/d45/d29/f8d 0 2024-09-16T00:26:04.823 INFO:tasks.workunit.client.1.smithi169.stdout:1/485: write dc/d36/f5b [765770,95812] 0 2024-09-16T00:26:04.824 INFO:tasks.workunit.client.0.smithi112.stdout:3/661: rmdir d2/d9/d2b/d47/d52/d7a/d80/d83/d7b/dad 39 2024-09-16T00:26:04.824 INFO:tasks.workunit.client.0.smithi112.stdout:7/673: creat d6/d10/da1/dcd/fe1 x:0 0 0 2024-09-16T00:26:04.827 INFO:tasks.workunit.client.1.smithi169.stdout:1/486: truncate dc/d18/f59 1548271 0 2024-09-16T00:26:04.827 INFO:tasks.workunit.client.0.smithi112.stdout:2/727: mknod d2/dd/d48/d22/d2b/d54/d60/d6c/d88/ce3 0 2024-09-16T00:26:04.827 INFO:tasks.workunit.client.0.smithi112.stdout:1/710: dread d0/d14/df/d28/d45/d46/d58/d62/f74 [0,4194304] 0 2024-09-16T00:26:04.828 INFO:tasks.workunit.client.1.smithi169.stdout:1/487: symlink dc/d36/d3a/d7f/l91 0 2024-09-16T00:26:04.829 INFO:tasks.workunit.client.1.smithi169.stdout:1/488: chown la 172 1 2024-09-16T00:26:04.830 INFO:tasks.workunit.client.0.smithi112.stdout:3/662: symlink d2/d9/d2b/d47/d52/d7a/d80/d83/d68/lde 0 2024-09-16T00:26:04.830 INFO:tasks.workunit.client.0.smithi112.stdout:7/674: write d6/d7f/d85/fb5 [899265,16695] 0 2024-09-16T00:26:04.831 INFO:tasks.workunit.client.1.smithi169.stdout:1/489: mkdir dc/d18/d67/d69/d7b/d92 0 2024-09-16T00:26:04.831 INFO:tasks.workunit.client.0.smithi112.stdout:2/728: write d2/dd/d48/d94/fbe [357794,13792] 0 2024-09-16T00:26:04.831 INFO:tasks.workunit.client.0.smithi112.stdout:1/711: fsync d0/d14/df/d28/d45/f98 0 2024-09-16T00:26:04.834 INFO:tasks.workunit.client.0.smithi112.stdout:7/675: creat d6/d10/d16/d96/fe2 x:0 0 0 2024-09-16T00:26:04.834 INFO:tasks.workunit.client.0.smithi112.stdout:1/712: creat d0/d14/df/d28/fe7 x:0 0 0 2024-09-16T00:26:04.835 INFO:tasks.workunit.client.0.smithi112.stdout:1/713: dread - d0/d14/d1c/fcd zero size 2024-09-16T00:26:04.837 INFO:tasks.workunit.client.0.smithi112.stdout:3/663: mkdir d2/d9/d2b/d47/d52/d7a/d80/d83/d68/d90/dcb/ddf 0 2024-09-16T00:26:04.837 INFO:tasks.workunit.client.1.smithi169.stdout:8/440: read d7/d20/f7d [169472,125469] 0 2024-09-16T00:26:04.838 INFO:tasks.workunit.client.0.smithi112.stdout:3/664: readlink d2/d9/d2b/d2c/d69/l81 0 2024-09-16T00:26:04.839 INFO:tasks.workunit.client.1.smithi169.stdout:8/441: write d7/dc/d17/d48/f60 [5305533,22611] 0 2024-09-16T00:26:04.840 INFO:tasks.workunit.client.0.smithi112.stdout:3/665: write d2/f19 [3972292,79711] 0 2024-09-16T00:26:04.841 INFO:tasks.workunit.client.0.smithi112.stdout:2/729: creat d2/dd/d48/d22/d67/fe4 x:0 0 0 2024-09-16T00:26:04.842 INFO:tasks.workunit.client.1.smithi169.stdout:8/442: dread d7/dc/d1d/f5f [0,4194304] 0 2024-09-16T00:26:04.844 INFO:tasks.workunit.client.0.smithi112.stdout:1/714: mkdir d0/d14/df/d28/d42/dd8/de8 0 2024-09-16T00:26:04.846 INFO:tasks.workunit.client.0.smithi112.stdout:8/590: dwrite d10/d12/d24/f2b [0,4194304] 0 2024-09-16T00:26:04.846 INFO:tasks.workunit.client.0.smithi112.stdout:7/676: link d6/d10/d16/d96/dc7/l67 d6/d10/d16/db4/le3 0 2024-09-16T00:26:04.848 INFO:tasks.workunit.client.0.smithi112.stdout:6/596: truncate dd/df/d10/d47/f9c 4079144 0 2024-09-16T00:26:04.848 INFO:tasks.workunit.client.1.smithi169.stdout:8/443: dwrite d7/d20/f7e [0,4194304] 0 2024-09-16T00:26:04.853 INFO:tasks.workunit.client.1.smithi169.stdout:1/490: write dc/f1d [1067866,81923] 0 2024-09-16T00:26:04.853 INFO:tasks.workunit.client.1.smithi169.stdout:8/444: readlink d7/d20/l57 0 2024-09-16T00:26:04.854 INFO:tasks.workunit.client.0.smithi112.stdout:3/666: fdatasync d2/d9/d2b/d47/d52/d7a/d80/d83/d76/f99 0 2024-09-16T00:26:04.854 INFO:tasks.workunit.client.0.smithi112.stdout:0/718: dwrite d5/da/d21/f51 [0,4194304] 0 2024-09-16T00:26:04.856 INFO:tasks.workunit.client.0.smithi112.stdout:0/719: dread d5/f95 [0,4194304] 0 2024-09-16T00:26:04.856 INFO:tasks.workunit.client.0.smithi112.stdout:8/591: write d10/d12/d1c/f59 [1367522,99821] 0 2024-09-16T00:26:04.860 INFO:tasks.workunit.client.1.smithi169.stdout:1/491: dwrite dc/d2c/d3d/f3f [0,4194304] 0 2024-09-16T00:26:04.860 INFO:tasks.workunit.client.1.smithi169.stdout:2/410: sync 2024-09-16T00:26:04.860 INFO:tasks.workunit.client.1.smithi169.stdout:1/492: chown f7 0 1 2024-09-16T00:26:04.862 INFO:tasks.workunit.client.0.smithi112.stdout:7/677: dwrite d6/d10/f1d [0,4194304] 0 2024-09-16T00:26:04.863 INFO:tasks.workunit.client.0.smithi112.stdout:2/730: creat d2/dd/d48/d94/fe5 x:0 0 0 2024-09-16T00:26:04.864 INFO:tasks.workunit.client.0.smithi112.stdout:7/678: fdatasync d6/d10/f71 0 2024-09-16T00:26:04.864 INFO:tasks.workunit.client.1.smithi169.stdout:1/493: truncate dc/d18/d67/d69/f80 4321185 0 2024-09-16T00:26:04.865 INFO:tasks.workunit.client.0.smithi112.stdout:1/715: mkdir d0/d14/d1c/de9 0 2024-09-16T00:26:04.865 INFO:tasks.workunit.client.1.smithi169.stdout:2/411: dread d7/db/d20/d21/f4a [0,4194304] 0 2024-09-16T00:26:04.867 INFO:tasks.workunit.client.0.smithi112.stdout:2/731: fsync d2/dd/d48/d22/d2b/d54/d60/d6c/d88/da4/fd0 0 2024-09-16T00:26:04.868 INFO:tasks.workunit.client.0.smithi112.stdout:3/667: mknod d2/d9/d2b/d2c/ce0 0 2024-09-16T00:26:04.868 INFO:tasks.workunit.client.0.smithi112.stdout:1/716: fsync d0/fe5 0 2024-09-16T00:26:04.868 INFO:tasks.workunit.client.0.smithi112.stdout:8/592: write fe [1873989,26357] 0 2024-09-16T00:26:04.871 INFO:tasks.workunit.client.0.smithi112.stdout:6/597: creat dd/df/fc2 x:0 0 0 2024-09-16T00:26:04.873 INFO:tasks.workunit.client.0.smithi112.stdout:3/668: write d2/d9/d2b/d2c/fce [674759,665] 0 2024-09-16T00:26:04.876 INFO:tasks.workunit.client.1.smithi169.stdout:1/494: dwrite dc/d36/d3a/d45/d29/f5c [0,4194304] 0 2024-09-16T00:26:04.877 INFO:tasks.workunit.client.0.smithi112.stdout:6/598: creat dd/d15/da5/d79/fc3 x:0 0 0 2024-09-16T00:26:04.879 INFO:tasks.workunit.client.0.smithi112.stdout:8/593: symlink d10/d12/lc5 0 2024-09-16T00:26:04.883 INFO:tasks.workunit.client.0.smithi112.stdout:0/720: dwrite d5/da/d1d/d71/f3b [0,4194304] 0 2024-09-16T00:26:04.887 INFO:tasks.workunit.client.0.smithi112.stdout:3/669: dread d2/f19 [0,4194304] 0 2024-09-16T00:26:04.889 INFO:tasks.workunit.client.0.smithi112.stdout:6/599: rename dd/d15/d34 to dd/df/d10/d47/d55/dc4 0 2024-09-16T00:26:04.891 INFO:tasks.workunit.client.0.smithi112.stdout:1/717: dwrite d0/d14/df/d28/f32 [0,4194304] 0 2024-09-16T00:26:04.892 INFO:tasks.workunit.client.0.smithi112.stdout:0/721: symlink d5/da/d1d/d71/d64/db6/ldb 0 2024-09-16T00:26:04.893 INFO:tasks.workunit.client.0.smithi112.stdout:6/600: readlink dd/df/d17/d27/d84/l3c 0 2024-09-16T00:26:04.895 INFO:tasks.workunit.client.0.smithi112.stdout:0/722: readlink d5/da/dc/l55 0 2024-09-16T00:26:04.895 INFO:tasks.workunit.client.0.smithi112.stdout:8/594: dwrite d10/d12/d1c/d61/d6c/fb5 [0,4194304] 0 2024-09-16T00:26:04.901 INFO:tasks.workunit.client.0.smithi112.stdout:0/723: truncate d5/da/d57/f73 4704795 0 2024-09-16T00:26:04.902 INFO:tasks.workunit.client.0.smithi112.stdout:8/595: unlink d10/d4e/d58/d87/fc3 0 2024-09-16T00:26:04.906 INFO:tasks.workunit.client.0.smithi112.stdout:1/718: link d0/d14/df/d28/d45/dd4/l8b d0/d14/df/d28/d45/d46/da7/lea 0 2024-09-16T00:26:04.908 INFO:tasks.workunit.client.1.smithi169.stdout:2/412: write d7/d1c/d26/f2c [162782,123907] 0 2024-09-16T00:26:04.909 INFO:tasks.workunit.client.0.smithi112.stdout:8/596: creat d10/d4e/d58/fc6 x:0 0 0 2024-09-16T00:26:04.911 INFO:tasks.workunit.client.0.smithi112.stdout:2/732: dread d2/dd/f12 [0,4194304] 0 2024-09-16T00:26:04.911 INFO:tasks.workunit.client.1.smithi169.stdout:2/413: symlink d7/d1c/l8c 0 2024-09-16T00:26:04.911 INFO:tasks.workunit.client.0.smithi112.stdout:1/719: unlink d0/d14/d1c/d20/daf/lb1 0 2024-09-16T00:26:04.913 INFO:tasks.workunit.client.0.smithi112.stdout:1/720: chown d0/d14/df/d28/d45/d46/l78 502571775 1 2024-09-16T00:26:04.913 INFO:tasks.workunit.client.0.smithi112.stdout:2/733: mkdir d2/de6 0 2024-09-16T00:26:04.915 INFO:tasks.workunit.client.0.smithi112.stdout:8/597: rename d10/d12/d1c/d75/l47 to d10/d4e/lc7 0 2024-09-16T00:26:04.915 INFO:tasks.workunit.client.1.smithi169.stdout:2/414: dwrite d7/f11 [0,4194304] 0 2024-09-16T00:26:04.917 INFO:tasks.workunit.client.0.smithi112.stdout:2/734: mkdir d2/dd/d48/d8a/db9/de7 0 2024-09-16T00:26:04.918 INFO:tasks.workunit.client.0.smithi112.stdout:8/598: creat d10/d4e/d8c/d7c/d97/fc8 x:0 0 0 2024-09-16T00:26:04.919 INFO:tasks.workunit.client.0.smithi112.stdout:1/721: fdatasync d0/d14/d1c/d20/daf/f59 0 2024-09-16T00:26:04.922 INFO:tasks.workunit.client.0.smithi112.stdout:8/599: unlink d10/d12/d1c/f54 0 2024-09-16T00:26:04.923 INFO:tasks.workunit.client.1.smithi169.stdout:2/415: dwrite d7/db/f45 [4194304,4194304] 0 2024-09-16T00:26:04.925 INFO:tasks.workunit.client.0.smithi112.stdout:8/600: truncate d10/d12/fb9 432519 0 2024-09-16T00:26:04.926 INFO:tasks.workunit.client.1.smithi169.stdout:2/416: mknod d7/db/d20/c8d 0 2024-09-16T00:26:04.930 INFO:tasks.workunit.client.0.smithi112.stdout:1/722: dwrite d0/d14/df/d28/d45/d46/d58/d62/f88 [0,4194304] 0 2024-09-16T00:26:04.941 INFO:tasks.workunit.client.0.smithi112.stdout:1/723: stat d0/d14/d6a/f80 0 2024-09-16T00:26:04.942 INFO:tasks.workunit.client.0.smithi112.stdout:8/601: dwrite d10/f15 [0,4194304] 0 2024-09-16T00:26:04.944 INFO:tasks.workunit.client.0.smithi112.stdout:8/602: chown lb 351093 1 2024-09-16T00:26:04.944 INFO:tasks.workunit.client.1.smithi169.stdout:2/417: write d7/d1c/d26/f36 [1508612,128536] 0 2024-09-16T00:26:04.945 INFO:tasks.workunit.client.0.smithi112.stdout:7/679: dwrite d6/d10/d16/d96/dc7/d39/f43 [4194304,4194304] 0 2024-09-16T00:26:04.946 INFO:tasks.workunit.client.1.smithi169.stdout:2/418: write d7/d1c/f72 [368176,99523] 0 2024-09-16T00:26:04.950 INFO:tasks.workunit.client.1.smithi169.stdout:2/419: unlink d7/db/d27/d29/d7d/c85 0 2024-09-16T00:26:04.951 INFO:tasks.workunit.client.0.smithi112.stdout:3/670: truncate d2/d9/d2b/d47/d52/d7a/d80/d83/d49/f4c 7780958 0 2024-09-16T00:26:04.954 INFO:tasks.workunit.client.1.smithi169.stdout:2/420: dread d7/db/d20/f3b [0,4194304] 0 2024-09-16T00:26:04.954 INFO:tasks.workunit.client.0.smithi112.stdout:6/601: dwrite dd/df/d10/d47/d55/f7d [0,4194304] 0 2024-09-16T00:26:04.960 INFO:tasks.workunit.client.1.smithi169.stdout:2/421: dwrite f5 [4194304,4194304] 0 2024-09-16T00:26:04.967 INFO:tasks.workunit.client.1.smithi169.stdout:2/422: dwrite d7/db/f17 [0,4194304] 0 2024-09-16T00:26:04.967 INFO:tasks.workunit.client.1.smithi169.stdout:2/423: chown d7/db 302 1 2024-09-16T00:26:04.968 INFO:tasks.workunit.client.1.smithi169.stdout:2/424: write d7/d6b/d70/d79/f7b [150377,121525] 0 2024-09-16T00:26:04.971 INFO:tasks.workunit.client.0.smithi112.stdout:1/724: creat d0/d14/d1c/d20/daf/feb x:0 0 0 2024-09-16T00:26:04.975 INFO:tasks.workunit.client.0.smithi112.stdout:2/735: write d2/dd/d48/d22/d67/d9a/fc3 [4500689,35519] 0 2024-09-16T00:26:04.975 INFO:tasks.workunit.client.1.smithi169.stdout:2/425: dwrite f5 [4194304,4194304] 0 2024-09-16T00:26:04.976 INFO:tasks.workunit.client.0.smithi112.stdout:7/680: mkdir d6/d10/d16/db4/de4 0 2024-09-16T00:26:04.983 INFO:tasks.workunit.client.1.smithi169.stdout:2/426: dwrite d7/d1c/d26/f75 [0,4194304] 0 2024-09-16T00:26:04.983 INFO:tasks.workunit.client.0.smithi112.stdout:3/671: mknod d2/d9/d2b/d2c/d69/d92/dda/ce1 0 2024-09-16T00:26:04.985 INFO:tasks.workunit.client.1.smithi169.stdout:2/427: creat d7/db/d27/d29/d7d/f8e x:0 0 0 2024-09-16T00:26:04.986 INFO:tasks.workunit.client.0.smithi112.stdout:3/672: chown d2/d9/d2b/d2c/f42 6420 1 2024-09-16T00:26:04.986 INFO:tasks.workunit.client.0.smithi112.stdout:2/736: mkdir d2/dd/d48/d22/d2b/d54/d60/d6c/d88/dad/de8 0 2024-09-16T00:26:04.988 INFO:tasks.workunit.client.1.smithi169.stdout:2/428: creat d7/db/d27/f8f x:0 0 0 2024-09-16T00:26:04.989 INFO:tasks.workunit.client.0.smithi112.stdout:3/673: chown d2/d9/d2b/d39/f3c 55 1 2024-09-16T00:26:04.991 INFO:tasks.workunit.client.0.smithi112.stdout:3/674: stat d2/d9/d2b/d47/d52/d7a/d80/d83/d76 0 2024-09-16T00:26:04.992 INFO:tasks.workunit.client.0.smithi112.stdout:7/681: dwrite d6/d10/f32 [0,4194304] 0 2024-09-16T00:26:04.993 INFO:tasks.workunit.client.0.smithi112.stdout:8/603: creat d10/d4e/fc9 x:0 0 0 2024-09-16T00:26:04.994 INFO:tasks.workunit.client.1.smithi169.stdout:2/429: dwrite d7/d1c/d26/f75 [0,4194304] 0 2024-09-16T00:26:04.995 INFO:tasks.workunit.client.1.smithi169.stdout:4/457: sync 2024-09-16T00:26:04.995 INFO:tasks.workunit.client.1.smithi169.stdout:3/482: sync 2024-09-16T00:26:04.995 INFO:tasks.workunit.client.1.smithi169.stdout:5/428: sync 2024-09-16T00:26:04.995 INFO:tasks.workunit.client.1.smithi169.stdout:9/502: sync 2024-09-16T00:26:04.995 INFO:tasks.workunit.client.1.smithi169.stdout:7/528: sync 2024-09-16T00:26:04.995 INFO:tasks.workunit.client.1.smithi169.stdout:0/461: sync 2024-09-16T00:26:04.995 INFO:tasks.workunit.client.1.smithi169.stdout:6/652: sync 2024-09-16T00:26:04.996 INFO:tasks.workunit.client.0.smithi112.stdout:3/675: readlink d2/d9/d2b/d47/d52/l75 0 2024-09-16T00:26:05.001 INFO:tasks.workunit.client.1.smithi169.stdout:4/458: chown d2/c88 299563 1 2024-09-16T00:26:05.002 INFO:tasks.workunit.client.1.smithi169.stdout:3/483: truncate d7/d50/d6f/f86 993440 0 2024-09-16T00:26:05.002 INFO:tasks.workunit.client.1.smithi169.stdout:2/430: rename d7/db/f3d to d7/db/d27/f90 0 2024-09-16T00:26:05.003 INFO:tasks.workunit.client.1.smithi169.stdout:5/429: dread d2/da/d10/d13/d1c/f41 [0,4194304] 0 2024-09-16T00:26:05.006 INFO:tasks.workunit.client.1.smithi169.stdout:3/484: rmdir d7 39 2024-09-16T00:26:05.008 INFO:tasks.workunit.client.1.smithi169.stdout:7/529: creat d7/d71/d76/faa x:0 0 0 2024-09-16T00:26:05.008 INFO:tasks.workunit.client.0.smithi112.stdout:2/737: rename d2/dd/d48/d22/d2b/d54/f70 to d2/dd/d48/d94/d49/fe9 0 2024-09-16T00:26:05.009 INFO:tasks.workunit.client.1.smithi169.stdout:9/503: rename d5/d12/l1d to d5/da/d21/d23/lae 0 2024-09-16T00:26:05.009 INFO:tasks.workunit.client.1.smithi169.stdout:2/431: fsync f3 0 2024-09-16T00:26:05.010 INFO:tasks.workunit.client.0.smithi112.stdout:7/682: creat d6/d10/d75/d88/ddf/fe5 x:0 0 0 2024-09-16T00:26:05.010 INFO:tasks.workunit.client.1.smithi169.stdout:5/430: write d2/da/d10/f8a [63449,32495] 0 2024-09-16T00:26:05.013 INFO:tasks.workunit.client.1.smithi169.stdout:1/495: sync 2024-09-16T00:26:05.013 INFO:tasks.workunit.client.1.smithi169.stdout:8/445: sync 2024-09-16T00:26:05.014 INFO:tasks.workunit.client.1.smithi169.stdout:6/653: dwrite d2/d4d/d59/d38/f97 [0,4194304] 0 2024-09-16T00:26:05.014 INFO:tasks.workunit.client.0.smithi112.stdout:2/738: mknod d2/dd/d48/d22/d2b/d54/d60/da9/cea 0 2024-09-16T00:26:05.015 INFO:tasks.workunit.client.1.smithi169.stdout:6/654: chown d2/d3f/d41/d5c/d91/da6 89 1 2024-09-16T00:26:05.016 INFO:tasks.workunit.client.0.smithi112.stdout:7/683: stat d6/d10/f76 0 2024-09-16T00:26:05.019 INFO:tasks.workunit.client.1.smithi169.stdout:0/462: dwrite d0/d4c/d6f/f91 [0,4194304] 0 2024-09-16T00:26:05.020 INFO:tasks.workunit.client.1.smithi169.stdout:4/459: getdents d2/d12/d13/d90 0 2024-09-16T00:26:05.022 INFO:tasks.workunit.client.0.smithi112.stdout:2/739: rename d2/dd/d96 to d2/dd/d48/d22/d2b/d54/d60/d6c/d88/dad/deb 0 2024-09-16T00:26:05.023 INFO:tasks.workunit.client.1.smithi169.stdout:6/655: write d2/d22/da7/fcd [712479,86290] 0 2024-09-16T00:26:05.023 INFO:tasks.workunit.client.1.smithi169.stdout:8/446: creat d7/dc/d53/f93 x:0 0 0 2024-09-16T00:26:05.023 INFO:tasks.workunit.client.1.smithi169.stdout:4/460: fdatasync d2/d54/f7a 0 2024-09-16T00:26:05.024 INFO:tasks.workunit.client.0.smithi112.stdout:7/684: creat d6/d10/d75/d8f/d90/dbc/fe6 x:0 0 0 2024-09-16T00:26:05.026 INFO:tasks.workunit.client.1.smithi169.stdout:8/447: stat d7/dc/d26/c4f 0 2024-09-16T00:26:05.026 INFO:tasks.workunit.client.1.smithi169.stdout:6/656: readlink d2/d3f/d41/d5c/db9/lc9 0 2024-09-16T00:26:05.026 INFO:tasks.workunit.client.0.smithi112.stdout:6/602: truncate dd/df/d10/d47/d55/f7d 3305040 0 2024-09-16T00:26:05.026 INFO:tasks.workunit.client.1.smithi169.stdout:4/461: read - d2/d12/d13/d1f/d3e/f53 zero size 2024-09-16T00:26:05.027 INFO:tasks.workunit.client.0.smithi112.stdout:6/603: chown dd/df/d91/c9d 48553 1 2024-09-16T00:26:05.028 INFO:tasks.workunit.client.1.smithi169.stdout:3/485: rename d7/d1a/d1c/d35/cb1 to d7/d1a/d1c/cb8 0 2024-09-16T00:26:05.029 INFO:tasks.workunit.client.0.smithi112.stdout:1/725: write d0/f9a [203938,89499] 0 2024-09-16T00:26:05.030 INFO:tasks.workunit.client.1.smithi169.stdout:5/431: truncate d2/da/f28 3539996 0 2024-09-16T00:26:05.031 INFO:tasks.workunit.client.0.smithi112.stdout:1/726: chown d0/d14/df/d28/c77 2559502 1 2024-09-16T00:26:05.032 INFO:tasks.workunit.client.0.smithi112.stdout:1/727: write d0/f9a [79463,22910] 0 2024-09-16T00:26:05.033 INFO:tasks.workunit.client.1.smithi169.stdout:2/432: symlink d7/db/d27/d4f/d56/d6d/l91 0 2024-09-16T00:26:05.034 INFO:tasks.workunit.client.1.smithi169.stdout:0/463: creat d0/d2c/d4f/da5/fa6 x:0 0 0 2024-09-16T00:26:05.034 INFO:tasks.workunit.client.0.smithi112.stdout:7/685: creat d6/d7f/fe7 x:0 0 0 2024-09-16T00:26:05.035 INFO:tasks.workunit.client.1.smithi169.stdout:1/496: rename dc/d26/d5d to dc/d36/d3a/d7f/d93 0 2024-09-16T00:26:05.036 INFO:tasks.workunit.client.0.smithi112.stdout:6/604: fsync dd/df/d17/d27/d84/f36 0 2024-09-16T00:26:05.037 INFO:tasks.workunit.client.1.smithi169.stdout:9/504: getdents d5/da/d21/d8d 0 2024-09-16T00:26:05.037 INFO:tasks.workunit.client.1.smithi169.stdout:3/486: mkdir d7/d1a/d1c/d23/d70/db9 0 2024-09-16T00:26:05.037 INFO:tasks.workunit.client.0.smithi112.stdout:6/605: dread - dd/df/d10/f8b zero size 2024-09-16T00:26:05.038 INFO:tasks.workunit.client.0.smithi112.stdout:2/740: rename d2/lc6 to d2/dd/d48/d22/d2b/d54/d60/d6c/d88/dad/de8/lec 0 2024-09-16T00:26:05.040 INFO:tasks.workunit.client.0.smithi112.stdout:6/606: write dd/df/d10/d47/d55/fa3 [912714,34534] 0 2024-09-16T00:26:05.040 INFO:tasks.workunit.client.1.smithi169.stdout:7/530: getdents d7/d5b/d61/d98 0 2024-09-16T00:26:05.041 INFO:tasks.workunit.client.1.smithi169.stdout:6/657: dwrite d2/d3f/d41/d52/fd2 [0,4194304] 0 2024-09-16T00:26:05.042 INFO:tasks.workunit.client.1.smithi169.stdout:9/505: stat d5/da/d2e/l38 0 2024-09-16T00:26:05.042 INFO:tasks.workunit.client.1.smithi169.stdout:0/464: chown d0/d2c/c62 9928774 1 2024-09-16T00:26:05.042 INFO:tasks.workunit.client.0.smithi112.stdout:1/728: unlink d0/d14/df/d28/d45/dd4/d41/fd1 0 2024-09-16T00:26:05.043 INFO:tasks.workunit.client.0.smithi112.stdout:8/604: truncate d10/f15 1287491 0 2024-09-16T00:26:05.044 INFO:tasks.workunit.client.1.smithi169.stdout:3/487: write d7/d1a/d2b/fb3 [1013600,90634] 0 2024-09-16T00:26:05.045 INFO:tasks.workunit.client.1.smithi169.stdout:4/462: rename d2/d12/d13/d1f/d3e/d5b/d1a/f40 to d2/d12/d13/d5c/f94 0 2024-09-16T00:26:05.048 INFO:tasks.workunit.client.1.smithi169.stdout:0/465: write d0/d2c/d54/d56/f59 [823587,32158] 0 2024-09-16T00:26:05.049 INFO:tasks.workunit.client.1.smithi169.stdout:9/506: mknod d5/da/d5b/d91/d9a/caf 0 2024-09-16T00:26:05.049 INFO:tasks.workunit.client.0.smithi112.stdout:7/686: rename d6/d10/d16/d96/dc7/d39/l57 to d6/d10/d75/d8f/d90/ddc/le8 0 2024-09-16T00:26:05.049 INFO:tasks.workunit.client.0.smithi112.stdout:3/676: dread d2/d4d/fb3 [4194304,4194304] 0 2024-09-16T00:26:05.050 INFO:tasks.workunit.client.1.smithi169.stdout:5/432: rename d2/da/d10/d13/d1c/d74 to d2/da/d10/d13/d1c/d94 0 2024-09-16T00:26:05.051 INFO:tasks.workunit.client.1.smithi169.stdout:1/497: truncate dc/d36/d3a/d7f/d93/f6e 758647 0 2024-09-16T00:26:05.051 INFO:tasks.workunit.client.1.smithi169.stdout:6/658: write d2/d3f/d41/d5c/d73/f5b [3779601,32688] 0 2024-09-16T00:26:05.057 INFO:tasks.workunit.client.1.smithi169.stdout:4/463: readlink d2/d12/l14 0 2024-09-16T00:26:05.057 INFO:tasks.workunit.client.1.smithi169.stdout:1/498: chown dc/d36/d39/l4c 65574034 1 2024-09-16T00:26:05.057 INFO:tasks.workunit.client.0.smithi112.stdout:1/729: mknod d0/d14/d1c/d20/daf/cec 0 2024-09-16T00:26:05.058 INFO:tasks.workunit.client.0.smithi112.stdout:8/605: unlink d10/d12/f8f 0 2024-09-16T00:26:05.058 INFO:tasks.workunit.client.0.smithi112.stdout:7/687: write d6/d10/d16/d3b/f4b [358960,62690] 0 2024-09-16T00:26:05.058 INFO:tasks.workunit.client.1.smithi169.stdout:0/466: stat d0/d2/d8/d5/d79/f96 0 2024-09-16T00:26:05.058 INFO:tasks.workunit.client.1.smithi169.stdout:5/433: write d2/da/f19 [2349316,61262] 0 2024-09-16T00:26:05.059 INFO:tasks.workunit.client.0.smithi112.stdout:2/741: mknod d2/dd/d48/d22/d67/d9a/ced 0 2024-09-16T00:26:05.060 INFO:tasks.workunit.client.1.smithi169.stdout:2/433: getdents d7/db/d27/d29/d77 0 2024-09-16T00:26:05.063 INFO:tasks.workunit.client.1.smithi169.stdout:8/448: dwrite d7/dc/d17/d24/f27 [0,4194304] 0 2024-09-16T00:26:05.063 INFO:tasks.workunit.client.1.smithi169.stdout:6/659: dread - d2/d3f/d41/d5c/f94 zero size 2024-09-16T00:26:05.064 INFO:tasks.workunit.client.0.smithi112.stdout:7/688: chown l3 8329641 1 2024-09-16T00:26:05.064 INFO:tasks.workunit.client.0.smithi112.stdout:6/607: link dd/df/d10/d47/f57 dd/df/d10/d47/d89/dbc/fc5 0 2024-09-16T00:26:05.064 INFO:tasks.workunit.client.1.smithi169.stdout:7/531: link d7/d71/d76/d29/l83 d7/d5b/d61/d98/d99/lab 0 2024-09-16T00:26:05.065 INFO:tasks.workunit.client.0.smithi112.stdout:3/677: symlink d2/d9/d2b/d47/d52/d7a/d80/db4/le2 0 2024-09-16T00:26:05.066 INFO:tasks.workunit.client.1.smithi169.stdout:7/532: chown d7/l64 280923 1 2024-09-16T00:26:05.067 INFO:tasks.workunit.client.1.smithi169.stdout:3/488: dwrite d7/d1a/d1c/d23/f89 [0,4194304] 0 2024-09-16T00:26:05.067 INFO:tasks.workunit.client.0.smithi112.stdout:3/678: stat d2/d9/d2b/d47/d52/d7a/d80/d83/d49/cd1 0 2024-09-16T00:26:05.067 INFO:tasks.workunit.client.0.smithi112.stdout:5/566: sync 2024-09-16T00:26:05.068 INFO:tasks.workunit.client.0.smithi112.stdout:8/606: link d10/d12/d24/f2b d10/d12/d1c/d75/dbb/fca 0 2024-09-16T00:26:05.070 INFO:tasks.workunit.client.1.smithi169.stdout:8/449: chown d7/dc/d26/c4f 341 1 2024-09-16T00:26:05.070 INFO:tasks.workunit.client.1.smithi169.stdout:2/434: write d7/db/d20/f83 [950774,13367] 0 2024-09-16T00:26:05.070 INFO:tasks.workunit.client.0.smithi112.stdout:7/689: chown d6/d10/f53 0 1 2024-09-16T00:26:05.071 INFO:tasks.workunit.client.0.smithi112.stdout:6/608: rename dd/df/d13/fab to dd/d15/da5/dae/fc6 0 2024-09-16T00:26:05.072 INFO:tasks.workunit.client.1.smithi169.stdout:4/464: mknod d2/d12/d13/d1f/d3e/c95 0 2024-09-16T00:26:05.073 INFO:tasks.workunit.client.0.smithi112.stdout:5/567: creat d4/d1c/d43/fbf x:0 0 0 2024-09-16T00:26:05.074 INFO:tasks.workunit.client.0.smithi112.stdout:5/568: stat d4/d7/d26 0 2024-09-16T00:26:05.074 INFO:tasks.workunit.client.1.smithi169.stdout:6/660: rmdir d2/d3f/d41/d5c/d91/db7 39 2024-09-16T00:26:05.074 INFO:tasks.workunit.client.1.smithi169.stdout:9/507: dwrite d5/da/f2b [0,4194304] 0 2024-09-16T00:26:05.075 INFO:tasks.workunit.client.1.smithi169.stdout:8/450: chown d7/dc/d17/d24/d32 14907841 1 2024-09-16T00:26:05.075 INFO:tasks.workunit.client.1.smithi169.stdout:7/533: read d7/f17 [342603,35795] 0 2024-09-16T00:26:05.079 INFO:tasks.workunit.client.0.smithi112.stdout:2/742: dread d2/dd/d48/d22/d2b/d54/d79/f7a [0,4194304] 0 2024-09-16T00:26:05.079 INFO:tasks.workunit.client.1.smithi169.stdout:5/434: write d2/da/d10/d13/d1c/d40/d43/f84 [488730,3791] 0 2024-09-16T00:26:05.079 INFO:tasks.workunit.client.1.smithi169.stdout:2/435: write d7/db/d20/d4c/f5f [980561,12404] 0 2024-09-16T00:26:05.079 INFO:tasks.workunit.client.1.smithi169.stdout:3/489: mkdir d7/d50/d90/dba 0 2024-09-16T00:26:05.081 INFO:tasks.workunit.client.0.smithi112.stdout:6/609: unlink dd/df/d10/d47/d55/fa3 0 2024-09-16T00:26:05.082 INFO:tasks.workunit.client.1.smithi169.stdout:7/534: fsync d7/df/d73/f8c 0 2024-09-16T00:26:05.085 INFO:tasks.workunit.client.0.smithi112.stdout:3/679: dread d2/d9/d2b/d2c/d69/f72 [0,4194304] 0 2024-09-16T00:26:05.086 INFO:tasks.workunit.client.1.smithi169.stdout:4/465: mknod d2/d12/d13/d1f/c96 0 2024-09-16T00:26:05.086 INFO:tasks.workunit.client.1.smithi169.stdout:9/508: creat d5/d49/d64/fb0 x:0 0 0 2024-09-16T00:26:05.088 INFO:tasks.workunit.client.0.smithi112.stdout:8/607: creat d10/d4e/d58/fcb x:0 0 0 2024-09-16T00:26:05.088 INFO:tasks.workunit.client.1.smithi169.stdout:5/435: mknod d2/da/c95 0 2024-09-16T00:26:05.089 INFO:tasks.workunit.client.1.smithi169.stdout:8/451: mknod d7/dc/d1d/d3c/d82/c94 0 2024-09-16T00:26:05.090 INFO:tasks.workunit.client.1.smithi169.stdout:2/436: mkdir d7/d6b/d70/d81/d69/d92 0 2024-09-16T00:26:05.090 INFO:tasks.workunit.client.0.smithi112.stdout:5/569: rmdir d4/d7/d26/d4c/db9 0 2024-09-16T00:26:05.091 INFO:tasks.workunit.client.0.smithi112.stdout:5/570: write d4/dd/f27 [3721444,29377] 0 2024-09-16T00:26:05.091 INFO:tasks.workunit.client.1.smithi169.stdout:8/452: fsync d7/fa 0 2024-09-16T00:26:05.092 INFO:tasks.workunit.client.0.smithi112.stdout:3/680: unlink d2/d9/d2b/d47/d52/d7a/fc3 0 2024-09-16T00:26:05.092 INFO:tasks.workunit.client.1.smithi169.stdout:7/535: link d7/f50 d7/d71/d76/d29/fac 0 2024-09-16T00:26:05.092 INFO:tasks.workunit.client.0.smithi112.stdout:3/681: readlink d2/d9/d2b/d47/d52/l3e 0 2024-09-16T00:26:05.092 INFO:tasks.workunit.client.1.smithi169.stdout:9/509: creat d5/da/d5b/d6c/fb1 x:0 0 0 2024-09-16T00:26:05.093 INFO:tasks.workunit.client.0.smithi112.stdout:1/730: truncate d0/d14/d1c/d20/f33 82916 0 2024-09-16T00:26:05.093 INFO:tasks.workunit.client.1.smithi169.stdout:4/466: truncate d2/d12/d13/f60 458483 0 2024-09-16T00:26:05.094 INFO:tasks.workunit.client.0.smithi112.stdout:6/610: mknod dd/df/d10/d47/cc7 0 2024-09-16T00:26:05.094 INFO:tasks.workunit.client.0.smithi112.stdout:3/682: chown d2/d9/d2b/d47/d52/f38 50590316 1 2024-09-16T00:26:05.095 INFO:tasks.workunit.client.1.smithi169.stdout:3/490: dwrite d7/d1a/d1c/f54 [4194304,4194304] 0 2024-09-16T00:26:05.095 INFO:tasks.workunit.client.0.smithi112.stdout:8/608: creat d10/d12/d24/fcc x:0 0 0 2024-09-16T00:26:05.096 INFO:tasks.workunit.client.1.smithi169.stdout:8/453: unlink d7/dc/d17/f39 0 2024-09-16T00:26:05.097 INFO:tasks.workunit.client.1.smithi169.stdout:3/491: dread - d7/d50/d90/da6/faa zero size 2024-09-16T00:26:05.097 INFO:tasks.workunit.client.1.smithi169.stdout:4/467: truncate d2/d12/d13/d1f/f8e 947837 0 2024-09-16T00:26:05.099 INFO:tasks.workunit.client.1.smithi169.stdout:8/454: fsync d7/d20/f7e 0 2024-09-16T00:26:05.103 INFO:tasks.workunit.client.0.smithi112.stdout:8/609: mknod d10/d4e/d8c/d7c/da5/ccd 0 2024-09-16T00:26:05.104 INFO:tasks.workunit.client.1.smithi169.stdout:1/499: truncate dc/d18/d67/d69/f80 3429353 0 2024-09-16T00:26:05.105 INFO:tasks.workunit.client.0.smithi112.stdout:3/683: symlink d2/d9/d2b/d2c/le3 0 2024-09-16T00:26:05.105 INFO:tasks.workunit.client.1.smithi169.stdout:3/492: chown d7/d1a/d2b/l64 3549644 1 2024-09-16T00:26:05.105 INFO:tasks.workunit.client.1.smithi169.stdout:4/468: mknod d2/d12/d13/d1f/d3e/d5b/d1a/c97 0 2024-09-16T00:26:05.106 INFO:tasks.workunit.client.1.smithi169.stdout:0/467: truncate d0/d2c/d54/d21/d31/f58 482132 0 2024-09-16T00:26:05.107 INFO:tasks.workunit.client.0.smithi112.stdout:6/611: link dd/d15/da5/d79/fc3 dd/df/d11/fc8 0 2024-09-16T00:26:05.109 INFO:tasks.workunit.client.0.smithi112.stdout:8/610: symlink d10/d12/d1c/d75/lce 0 2024-09-16T00:26:05.109 INFO:tasks.workunit.client.1.smithi169.stdout:4/469: readlink d2/d12/d13/d4e/d50/l7f 0 2024-09-16T00:26:05.111 INFO:tasks.workunit.client.0.smithi112.stdout:1/731: rename d0/d14/d1c/cc3 to d0/d14/df/d28/d45/d46/d58/ced 0 2024-09-16T00:26:05.111 INFO:tasks.workunit.client.0.smithi112.stdout:2/743: write d2/dd/d48/d22/d2b/d54/d60/d6c/d88/dad/fb8 [692271,114919] 0 2024-09-16T00:26:05.112 INFO:tasks.workunit.client.1.smithi169.stdout:3/493: creat d7/d50/d90/da6/fbb x:0 0 0 2024-09-16T00:26:05.113 INFO:tasks.workunit.client.1.smithi169.stdout:6/661: dwrite d2/f45 [4194304,4194304] 0 2024-09-16T00:26:05.115 INFO:tasks.workunit.client.0.smithi112.stdout:5/571: dread d4/d7/f4d [0,4194304] 0 2024-09-16T00:26:05.115 INFO:tasks.workunit.client.0.smithi112.stdout:3/684: rename d2/d9/d2b/d47/d52/d7a/d80/d83/d49/cd1 to d2/d9/d2b/d47/d52/d7a/d80/d83/d76/ce4 0 2024-09-16T00:26:05.116 INFO:tasks.workunit.client.0.smithi112.stdout:2/744: symlink d2/dd/d48/d8a/lee 0 2024-09-16T00:26:05.116 INFO:tasks.workunit.client.0.smithi112.stdout:2/745: stat d2/d68 0 2024-09-16T00:26:05.117 INFO:tasks.workunit.client.1.smithi169.stdout:0/468: write d0/d2c/d38/f84 [192554,31627] 0 2024-09-16T00:26:05.119 INFO:tasks.workunit.client.1.smithi169.stdout:5/436: rmdir d2/da/d10/d13/d1c/d32/d53/d7e 0 2024-09-16T00:26:05.120 INFO:tasks.workunit.client.1.smithi169.stdout:9/510: getdents d5/da 0 2024-09-16T00:26:05.121 INFO:tasks.workunit.client.1.smithi169.stdout:4/470: fsync d2/d12/d13/d1f/f20 0 2024-09-16T00:26:05.121 INFO:tasks.workunit.client.1.smithi169.stdout:3/494: rmdir d7 39 2024-09-16T00:26:05.122 INFO:tasks.workunit.client.1.smithi169.stdout:2/437: truncate d7/db/f17 2811115 0 2024-09-16T00:26:05.123 INFO:tasks.workunit.client.0.smithi112.stdout:3/685: dwrite d2/f3f [0,4194304] 0 2024-09-16T00:26:05.123 INFO:tasks.workunit.client.1.smithi169.stdout:6/662: symlink d2/da4/db1/ld3 0 2024-09-16T00:26:05.124 INFO:tasks.workunit.client.0.smithi112.stdout:5/572: rmdir d4/dd/d12/d52 39 2024-09-16T00:26:05.124 INFO:tasks.workunit.client.0.smithi112.stdout:3/686: readlink d2/d9/d2b/d47/la6 0 2024-09-16T00:26:05.125 INFO:tasks.workunit.client.1.smithi169.stdout:6/663: write d2/d3f/d41/d5c/f65 [2414087,22717] 0 2024-09-16T00:26:05.126 INFO:tasks.workunit.client.0.smithi112.stdout:2/746: creat d2/dd/d48/fef x:0 0 0 2024-09-16T00:26:05.126 INFO:tasks.workunit.client.1.smithi169.stdout:0/469: mknod d0/d2c/d54/d21/d31/d8c/ca7 0 2024-09-16T00:26:05.130 INFO:tasks.workunit.client.0.smithi112.stdout:5/573: write d4/dd/d6f/f72 [4465320,98458] 0 2024-09-16T00:26:05.130 INFO:tasks.workunit.client.0.smithi112.stdout:2/747: write d2/dd/d48/d94/d49/fe9 [54826,88140] 0 2024-09-16T00:26:05.130 INFO:tasks.workunit.client.1.smithi169.stdout:5/437: readlink d2/da/d10/d13/d1c/d32/d53/l5a 0 2024-09-16T00:26:05.134 INFO:tasks.workunit.client.0.smithi112.stdout:5/574: mknod d4/d1c/cc0 0 2024-09-16T00:26:05.137 INFO:tasks.workunit.client.0.smithi112.stdout:5/575: unlink d4/d7/d26/d4c/f97 0 2024-09-16T00:26:05.138 INFO:tasks.workunit.client.1.smithi169.stdout:2/438: mknod d7/db/c93 0 2024-09-16T00:26:05.139 INFO:tasks.workunit.client.0.smithi112.stdout:2/748: dread d2/dd/d48/d22/f8f [0,4194304] 0 2024-09-16T00:26:05.140 INFO:tasks.workunit.client.1.smithi169.stdout:8/455: getdents d7/dc/d1d/d3c/d77 0 2024-09-16T00:26:05.140 INFO:tasks.workunit.client.0.smithi112.stdout:5/576: creat d4/d7/d14/d90/d9a/fc1 x:0 0 0 2024-09-16T00:26:05.142 INFO:tasks.workunit.client.1.smithi169.stdout:3/495: dread d7/d1a/d1c/d23/f89 [0,4194304] 0 2024-09-16T00:26:05.144 INFO:tasks.workunit.client.1.smithi169.stdout:6/664: rename d2/da4/db1/lb8 to d2/d3f/d41/d5c/db9/ld4 0 2024-09-16T00:26:05.144 INFO:tasks.workunit.client.0.smithi112.stdout:2/749: creat d2/dd/d48/d22/d2b/d54/d60/d6c/d88/dad/ff0 x:0 0 0 2024-09-16T00:26:05.144 INFO:tasks.workunit.client.1.smithi169.stdout:3/496: write d7/d1a/d1c/d6c/f7a [923545,87727] 0 2024-09-16T00:26:05.145 INFO:tasks.workunit.client.0.smithi112.stdout:6/612: write dd/df/f6e [1490890,11080] 0 2024-09-16T00:26:05.146 INFO:tasks.workunit.client.1.smithi169.stdout:3/497: chown d7/d1a/d1c/d35/d63/f4a 2762709 1 2024-09-16T00:26:05.148 INFO:tasks.workunit.client.1.smithi169.stdout:9/511: read d5/d12/f5d [247836,85281] 0 2024-09-16T00:26:05.148 INFO:tasks.workunit.client.0.smithi112.stdout:8/611: dread d10/f46 [0,4194304] 0 2024-09-16T00:26:05.152 INFO:tasks.workunit.client.1.smithi169.stdout:4/471: creat d2/f98 x:0 0 0 2024-09-16T00:26:05.153 INFO:tasks.workunit.client.0.smithi112.stdout:6/613: rename dd/df/d17/c3f to dd/df/d17/d27/cc9 0 2024-09-16T00:26:05.153 INFO:tasks.workunit.client.0.smithi112.stdout:3/687: dread d2/d9/d2b/d47/d52/d7a/d80/d83/f50 [0,4194304] 0 2024-09-16T00:26:05.153 INFO:tasks.workunit.client.0.smithi112.stdout:5/577: dread d4/d1c/d43/f6d [0,4194304] 0 2024-09-16T00:26:05.153 INFO:tasks.workunit.client.0.smithi112.stdout:8/612: rmdir d10/d12/d1c/d61 39 2024-09-16T00:26:05.154 INFO:tasks.workunit.client.1.smithi169.stdout:5/438: rename d2/da/d10/d13/d1c/d2d/d46 to d2/da/d10/d13/d1c/d32/d53/d83/d7d/d96 0 2024-09-16T00:26:05.154 INFO:tasks.workunit.client.1.smithi169.stdout:3/498: dwrite d7/d50/f87 [4194304,4194304] 0 2024-09-16T00:26:05.157 INFO:tasks.workunit.client.0.smithi112.stdout:2/750: dwrite d2/f46 [0,4194304] 0 2024-09-16T00:26:05.157 INFO:tasks.workunit.client.0.smithi112.stdout:8/613: write d10/d4e/f8e [163344,30290] 0 2024-09-16T00:26:05.158 INFO:tasks.workunit.client.0.smithi112.stdout:1/732: dwrite d0/d14/df/d28/d45/d46/f49 [0,4194304] 0 2024-09-16T00:26:05.159 INFO:tasks.workunit.client.0.smithi112.stdout:8/614: stat d10/d4e/f6f 0 2024-09-16T00:26:05.160 INFO:tasks.workunit.client.0.smithi112.stdout:3/688: rmdir d2/d9/d2b/d47/d52 39 2024-09-16T00:26:05.164 INFO:tasks.workunit.client.1.smithi169.stdout:6/665: truncate d2/f64 2485427 0 2024-09-16T00:26:05.165 INFO:tasks.workunit.client.1.smithi169.stdout:2/439: truncate d7/db/d27/d4f/f53 533591 0 2024-09-16T00:26:05.166 INFO:tasks.workunit.client.1.smithi169.stdout:9/512: rename d5/d49/l9e to d5/da/d73/lb2 0 2024-09-16T00:26:05.171 INFO:tasks.workunit.client.1.smithi169.stdout:3/499: mkdir d7/dbc 0 2024-09-16T00:26:05.174 INFO:tasks.workunit.client.1.smithi169.stdout:6/666: rmdir d2/d22 39 2024-09-16T00:26:05.176 INFO:tasks.workunit.client.1.smithi169.stdout:4/472: truncate d2/d12/f2c 1444359 0 2024-09-16T00:26:05.177 INFO:tasks.workunit.client.1.smithi169.stdout:9/513: dwrite d5/da/d21/d23/f40 [4194304,4194304] 0 2024-09-16T00:26:05.177 INFO:tasks.workunit.client.1.smithi169.stdout:0/470: truncate d0/d2c/d38/f84 368393 0 2024-09-16T00:26:05.179 INFO:tasks.workunit.client.1.smithi169.stdout:2/440: creat d7/db/d27/d29/d7d/f94 x:0 0 0 2024-09-16T00:26:05.179 INFO:tasks.workunit.client.1.smithi169.stdout:5/439: mknod d2/da/d10/d13/d1c/d32/d53/d4e/d55/c97 0 2024-09-16T00:26:05.184 INFO:tasks.workunit.client.1.smithi169.stdout:2/441: read d7/d1c/d26/f36 [3371931,87590] 0 2024-09-16T00:26:05.185 INFO:tasks.workunit.client.1.smithi169.stdout:4/473: dread d2/d12/f5d [0,4194304] 0 2024-09-16T00:26:05.187 INFO:tasks.workunit.client.1.smithi169.stdout:2/442: fsync d7/d1c/d26/f2c 0 2024-09-16T00:26:05.188 INFO:tasks.workunit.client.1.smithi169.stdout:2/443: stat d7/db/d27/d29/d7d/f94 0 2024-09-16T00:26:05.189 INFO:tasks.workunit.client.1.smithi169.stdout:8/456: write d7/f12 [1978695,9914] 0 2024-09-16T00:26:05.190 INFO:tasks.workunit.client.1.smithi169.stdout:9/514: rename d5/d49/d70 to d5/da/d5b/d91/db3 0 2024-09-16T00:26:05.190 INFO:tasks.workunit.client.1.smithi169.stdout:3/500: mknod d7/d1a/d1c/d23/cbd 0 2024-09-16T00:26:05.191 INFO:tasks.workunit.client.1.smithi169.stdout:9/515: fsync d5/f14 0 2024-09-16T00:26:05.196 INFO:tasks.workunit.client.1.smithi169.stdout:9/516: write d5/da/d21/f2c [4388223,111286] 0 2024-09-16T00:26:05.197 INFO:tasks.workunit.client.1.smithi169.stdout:4/474: rename d2/d12/d13/d1f/d3e/d5b/d1a/c97 to d2/d12/d13/d27/c99 0 2024-09-16T00:26:05.201 INFO:tasks.workunit.client.1.smithi169.stdout:3/501: mkdir d7/d1a/d1c/d23/d88/dbe 0 2024-09-16T00:26:05.203 INFO:tasks.workunit.client.1.smithi169.stdout:6/667: dread d2/fb [0,4194304] 0 2024-09-16T00:26:05.204 INFO:tasks.workunit.client.1.smithi169.stdout:3/502: stat d7/d1a/d1c/fb6 0 2024-09-16T00:26:05.204 INFO:tasks.workunit.client.1.smithi169.stdout:8/457: creat d7/dc/d17/d24/d32/d8d/f95 x:0 0 0 2024-09-16T00:26:05.205 INFO:tasks.workunit.client.1.smithi169.stdout:2/444: dwrite d7/db/d27/f90 [4194304,4194304] 0 2024-09-16T00:26:05.209 INFO:tasks.workunit.client.1.smithi169.stdout:6/668: write d2/d4d/d59/d38/f97 [209613,42824] 0 2024-09-16T00:26:05.213 INFO:tasks.workunit.client.1.smithi169.stdout:4/475: readlink d2/d12/d13/d1f/d2f/l3c 0 2024-09-16T00:26:05.214 INFO:tasks.workunit.client.1.smithi169.stdout:9/517: fsync d5/da/d2e/d47/f7c 0 2024-09-16T00:26:05.216 INFO:tasks.workunit.client.1.smithi169.stdout:5/440: dread d2/da/d10/d13/d1c/f3b [0,4194304] 0 2024-09-16T00:26:05.218 INFO:tasks.workunit.client.0.smithi112.stdout:1/733: stat d0/d14/df/d28/d45/d46 0 2024-09-16T00:26:05.220 INFO:tasks.workunit.client.1.smithi169.stdout:0/471: dread d0/d2/d8/f68 [4194304,4194304] 0 2024-09-16T00:26:05.221 INFO:tasks.workunit.client.1.smithi169.stdout:8/458: dwrite d7/dc/d17/d24/f27 [0,4194304] 0 2024-09-16T00:26:05.221 INFO:tasks.workunit.client.0.smithi112.stdout:2/751: rename d2/de6 to d2/dd/d48/d8a/db9/de7/df1 0 2024-09-16T00:26:05.222 INFO:tasks.workunit.client.0.smithi112.stdout:8/615: symlink d10/d4e/d58/lcf 0 2024-09-16T00:26:05.222 INFO:tasks.workunit.client.1.smithi169.stdout:6/669: mknod d2/d3f/d41/d5c/d73/cd5 0 2024-09-16T00:26:05.222 INFO:tasks.workunit.client.1.smithi169.stdout:0/472: write d0/d2/d8/d5/f2e [2149246,86982] 0 2024-09-16T00:26:05.223 INFO:tasks.workunit.client.0.smithi112.stdout:6/614: link dd/df/d17/d27/d54/f6a dd/fca 0 2024-09-16T00:26:05.223 INFO:tasks.workunit.client.0.smithi112.stdout:6/615: stat dd/df/d10/d47 0 2024-09-16T00:26:05.225 INFO:tasks.workunit.client.1.smithi169.stdout:6/670: write d2/d3f/d41/d52/fd2 [2974067,61593] 0 2024-09-16T00:26:05.225 INFO:tasks.workunit.client.1.smithi169.stdout:4/476: rmdir d2/d12/d13/d4d 39 2024-09-16T00:26:05.225 INFO:tasks.workunit.client.0.smithi112.stdout:1/734: mknod d0/d14/cee 0 2024-09-16T00:26:05.227 INFO:tasks.workunit.client.1.smithi169.stdout:6/671: chown d2/d4d/d59/d43/f2f 3 1 2024-09-16T00:26:05.228 INFO:tasks.workunit.client.0.smithi112.stdout:6/616: mkdir dd/df/d10/d47/d55/dbd/dcb 0 2024-09-16T00:26:05.229 INFO:tasks.workunit.client.0.smithi112.stdout:1/735: truncate d0/d14/df/d28/d45/dd4/d41/de0/fe2 621357 0 2024-09-16T00:26:05.231 INFO:tasks.workunit.client.1.smithi169.stdout:2/445: truncate f2 1016155 0 2024-09-16T00:26:05.231 INFO:tasks.workunit.client.0.smithi112.stdout:8/616: rename d10/d12/d1c/d75 to d10/d12/d1c/dd0 0 2024-09-16T00:26:05.232 INFO:tasks.workunit.client.1.smithi169.stdout:8/459: dwrite d7/dc/d2a/f85 [0,4194304] 0 2024-09-16T00:26:05.232 INFO:tasks.workunit.client.0.smithi112.stdout:8/617: readlink d10/d4e/d58/lbc 0 2024-09-16T00:26:05.232 INFO:tasks.workunit.client.1.smithi169.stdout:3/503: creat d7/d50/d90/dba/fbf x:0 0 0 2024-09-16T00:26:05.233 INFO:tasks.workunit.client.0.smithi112.stdout:6/617: mkdir dd/dcc 0 2024-09-16T00:26:05.233 INFO:tasks.workunit.client.0.smithi112.stdout:2/752: link d2/dd/d48/f2d d2/dd/d48/ff2 0 2024-09-16T00:26:05.235 INFO:tasks.workunit.client.0.smithi112.stdout:2/753: dread - d2/dd/d48/d94/d49/f9f zero size 2024-09-16T00:26:05.237 INFO:tasks.workunit.client.0.smithi112.stdout:2/754: write d2/dd/d48/d22/d2b/d30/f6f [65418,55389] 0 2024-09-16T00:26:05.237 INFO:tasks.workunit.client.1.smithi169.stdout:9/518: unlink d5/da/d2e/d47/f6b 0 2024-09-16T00:26:05.240 INFO:tasks.workunit.client.0.smithi112.stdout:6/618: getdents dd/d15/da5 0 2024-09-16T00:26:05.240 INFO:tasks.workunit.client.1.smithi169.stdout:5/441: unlink d2/da/d10/l3c 0 2024-09-16T00:26:05.241 INFO:tasks.workunit.client.0.smithi112.stdout:3/689: truncate d2/d9/d2b/d2c/f51 2631666 0 2024-09-16T00:26:05.242 INFO:tasks.workunit.client.0.smithi112.stdout:2/755: rename d2/dd/d48/d22/d2b/l7e to d2/dd/d48/d8a/lf3 0 2024-09-16T00:26:05.243 INFO:tasks.workunit.client.1.smithi169.stdout:8/460: unlink d7/dc/fe 0 2024-09-16T00:26:05.244 INFO:tasks.workunit.client.0.smithi112.stdout:3/690: mknod d2/d4d/ce5 0 2024-09-16T00:26:05.245 INFO:tasks.workunit.client.1.smithi169.stdout:3/504: mkdir d7/d50/d90/dba/dc0 0 2024-09-16T00:26:05.246 INFO:tasks.workunit.client.1.smithi169.stdout:0/473: creat d0/d7d/fa8 x:0 0 0 2024-09-16T00:26:05.247 INFO:tasks.workunit.client.1.smithi169.stdout:9/519: chown d5/da/d2e/c37 12 1 2024-09-16T00:26:05.248 INFO:tasks.workunit.client.1.smithi169.stdout:9/520: chown d5/da/d21/d8d/da4 785373768 1 2024-09-16T00:26:05.248 INFO:tasks.workunit.client.0.smithi112.stdout:6/619: dread dd/df/d10/d47/d89/dbc/fc5 [0,4194304] 0 2024-09-16T00:26:05.252 INFO:tasks.workunit.client.1.smithi169.stdout:8/461: unlink d7/dc/d1d/f5f 0 2024-09-16T00:26:05.252 INFO:tasks.workunit.client.1.smithi169.stdout:3/505: dread - d7/d1a/d1c/f73 zero size 2024-09-16T00:26:05.253 INFO:tasks.workunit.client.0.smithi112.stdout:3/691: dread d2/d9/d2b/d47/f8b [0,4194304] 0 2024-09-16T00:26:05.253 INFO:tasks.workunit.client.0.smithi112.stdout:6/620: dread dd/df/d10/f3a [0,4194304] 0 2024-09-16T00:26:05.254 INFO:tasks.workunit.client.1.smithi169.stdout:6/672: rename d2/d3f/d41/d52/c2c to d2/d4d/d59/d43/cd6 0 2024-09-16T00:26:05.255 INFO:tasks.workunit.client.1.smithi169.stdout:9/521: mkdir d5/da/d21/d8d/db4 0 2024-09-16T00:26:05.255 INFO:tasks.workunit.client.1.smithi169.stdout:0/474: dwrite d0/d2c/d54/d21/fa0 [0,4194304] 0 2024-09-16T00:26:05.256 INFO:tasks.workunit.client.1.smithi169.stdout:6/673: stat d2/d4d/f53 0 2024-09-16T00:26:05.257 INFO:tasks.workunit.client.1.smithi169.stdout:2/446: creat d7/d1c/f95 x:0 0 0 2024-09-16T00:26:05.257 INFO:tasks.workunit.client.0.smithi112.stdout:6/621: mknod dd/d15/ccd 0 2024-09-16T00:26:05.260 INFO:tasks.workunit.client.1.smithi169.stdout:0/475: write d0/d2/d4b/f52 [927462,45020] 0 2024-09-16T00:26:05.260 INFO:tasks.workunit.client.1.smithi169.stdout:8/462: symlink d7/dc/d17/d24/d32/d4d/l96 0 2024-09-16T00:26:05.262 INFO:tasks.workunit.client.1.smithi169.stdout:5/442: rename d2/f61 to d2/da/d10/f98 0 2024-09-16T00:26:05.262 INFO:tasks.workunit.client.1.smithi169.stdout:2/447: truncate d7/db/d20/d21/f4a 3074458 0 2024-09-16T00:26:05.263 INFO:tasks.workunit.client.0.smithi112.stdout:3/692: creat d2/d9/d2b/d47/d52/d7a/d80/d83/d7b/fe6 x:0 0 0 2024-09-16T00:26:05.264 INFO:tasks.workunit.client.0.smithi112.stdout:2/756: dwrite d2/dd/f1a [0,4194304] 0 2024-09-16T00:26:05.265 INFO:tasks.workunit.client.0.smithi112.stdout:6/622: getdents dd/df/d13/d7c/d88 0 2024-09-16T00:26:05.266 INFO:tasks.workunit.client.1.smithi169.stdout:2/448: chown d7/db/d27/d29/d77 1 1 2024-09-16T00:26:05.267 INFO:tasks.workunit.client.0.smithi112.stdout:6/623: rmdir dd/df/d17 39 2024-09-16T00:26:05.269 INFO:tasks.workunit.client.0.smithi112.stdout:3/693: rename d2/d9/d2b/d47/d52/d7a/d80/d83/d7b/fe6 to d2/d9/d2b/d47/d52/d7a/d80/d83/d68/d90/fe7 0 2024-09-16T00:26:05.270 INFO:tasks.workunit.client.1.smithi169.stdout:8/463: dread d7/dc/d1d/d3c/d77/f7b [0,4194304] 0 2024-09-16T00:26:05.271 INFO:tasks.workunit.client.1.smithi169.stdout:0/476: dwrite d0/d2c/f4a [0,4194304] 0 2024-09-16T00:26:05.271 INFO:tasks.workunit.client.0.smithi112.stdout:6/624: write dd/df/d10/d47/d55/f7d [4148350,87350] 0 2024-09-16T00:26:05.274 INFO:tasks.workunit.client.0.smithi112.stdout:2/757: dwrite f1 [0,4194304] 0 2024-09-16T00:26:05.277 INFO:tasks.workunit.client.0.smithi112.stdout:6/625: rename dd/d15/da5/d79/l9f to dd/d15/da5/lce 0 2024-09-16T00:26:05.281 INFO:tasks.workunit.client.1.smithi169.stdout:2/449: unlink d7/db/d20/f3b 0 2024-09-16T00:26:05.284 INFO:tasks.workunit.client.1.smithi169.stdout:5/443: dwrite d2/da/d10/d13/d1c/f41 [0,4194304] 0 2024-09-16T00:26:05.285 INFO:tasks.workunit.client.1.smithi169.stdout:3/506: link d7/d1a/d1c/d35/d63/d37/l4b d7/dbc/lc1 0 2024-09-16T00:26:05.286 INFO:tasks.workunit.client.1.smithi169.stdout:3/507: fsync d7/f8f 0 2024-09-16T00:26:05.287 INFO:tasks.workunit.client.1.smithi169.stdout:8/464: symlink d7/dc/d1d/d3c/d77/l97 0 2024-09-16T00:26:05.290 INFO:tasks.workunit.client.1.smithi169.stdout:8/465: dread - d7/dc/d26/d41/d4e/f7c zero size 2024-09-16T00:26:05.290 INFO:tasks.workunit.client.1.smithi169.stdout:9/522: rename d5/da/d21/d23/l50 to d5/da/d2e/lb5 0 2024-09-16T00:26:05.292 INFO:tasks.workunit.client.1.smithi169.stdout:8/466: write d7/dc/d17/f28 [1101222,18410] 0 2024-09-16T00:26:05.293 INFO:tasks.workunit.client.1.smithi169.stdout:0/477: mknod d0/d2/ca9 0 2024-09-16T00:26:05.294 INFO:tasks.workunit.client.1.smithi169.stdout:8/467: write d7/dc/d2a/f85 [754172,85964] 0 2024-09-16T00:26:05.298 INFO:tasks.workunit.client.0.smithi112.stdout:9/872: sync 2024-09-16T00:26:05.298 INFO:tasks.workunit.client.0.smithi112.stdout:4/696: sync 2024-09-16T00:26:05.298 INFO:tasks.workunit.client.1.smithi169.stdout:8/468: mknod d7/dc/d26/c98 0 2024-09-16T00:26:05.299 INFO:tasks.workunit.client.1.smithi169.stdout:0/478: symlink d0/d2c/d4f/da5/laa 0 2024-09-16T00:26:05.299 INFO:tasks.workunit.client.1.smithi169.stdout:8/469: chown d7/dc/d17/l19 3 1 2024-09-16T00:26:05.300 INFO:tasks.workunit.client.0.smithi112.stdout:2/758: mknod d2/dd/d48/d22/d67/cf4 0 2024-09-16T00:26:05.302 INFO:tasks.workunit.client.0.smithi112.stdout:9/873: creat d1/d5/d6/df8/ddd/f133 x:0 0 0 2024-09-16T00:26:05.303 INFO:tasks.workunit.client.0.smithi112.stdout:2/759: write d2/dd/d48/d22/d2b/d54/d60/fc5 [1148642,59628] 0 2024-09-16T00:26:05.306 INFO:tasks.workunit.client.1.smithi169.stdout:8/470: truncate f5 1178606 0 2024-09-16T00:26:05.307 INFO:tasks.workunit.client.1.smithi169.stdout:9/523: mkdir d5/da/d5b/d6c/da0/db6 0 2024-09-16T00:26:05.307 INFO:tasks.workunit.client.0.smithi112.stdout:9/874: creat d1/d5/d20/d81/d1a/d88/da8/f134 x:0 0 0 2024-09-16T00:26:05.308 INFO:tasks.workunit.client.1.smithi169.stdout:6/674: rename d2/d4d/d59/cba to d2/d4d/d59/d38/cd7 0 2024-09-16T00:26:05.311 INFO:tasks.workunit.client.0.smithi112.stdout:4/697: dwrite d6/db/d1f/d41/d49/d31/fed [0,4194304] 0 2024-09-16T00:26:05.311 INFO:tasks.workunit.client.1.smithi169.stdout:8/471: symlink d7/dc/d1d/l99 0 2024-09-16T00:26:05.313 INFO:tasks.workunit.client.1.smithi169.stdout:9/524: creat d5/da/d73/fb7 x:0 0 0 2024-09-16T00:26:05.314 INFO:tasks.workunit.client.0.smithi112.stdout:2/760: rename d2/dd/l41 to d2/dd/d48/d22/d2b/d54/lf5 0 2024-09-16T00:26:05.314 INFO:tasks.workunit.client.0.smithi112.stdout:3/694: dwrite d2/d9/d2b/fa0 [0,4194304] 0 2024-09-16T00:26:05.317 INFO:tasks.workunit.client.0.smithi112.stdout:3/695: dread - d2/d9/d2b/d47/d52/d7a/d80/d83/d49/fa8 zero size 2024-09-16T00:26:05.317 INFO:tasks.workunit.client.1.smithi169.stdout:9/525: fdatasync d5/da/d5b/d6c/da0/f7a 0 2024-09-16T00:26:05.317 INFO:tasks.workunit.client.1.smithi169.stdout:2/450: rename d7/db/d27/d29/d7d to d7/d1c/d26/d96 0 2024-09-16T00:26:05.319 INFO:tasks.workunit.client.0.smithi112.stdout:4/698: creat d6/d39/d1c/d6e/ffd x:0 0 0 2024-09-16T00:26:05.320 INFO:tasks.workunit.client.0.smithi112.stdout:2/761: rmdir d2/dd/d48/d22/d2b/d54/d60/d6c/dc9 39 2024-09-16T00:26:05.321 INFO:tasks.workunit.client.1.smithi169.stdout:6/675: dwrite d2/d4d/f49 [0,4194304] 0 2024-09-16T00:26:05.321 INFO:tasks.workunit.client.0.smithi112.stdout:6/626: truncate dd/df/d13/d7c/f98 828105 0 2024-09-16T00:26:05.323 INFO:tasks.workunit.client.0.smithi112.stdout:4/699: creat d6/d39/d36/d5c/d7e/dcd/ffe x:0 0 0 2024-09-16T00:26:05.323 INFO:tasks.workunit.client.1.smithi169.stdout:2/451: rename d7/db/d20/d21/f48 to d7/db/d27/d29/d37/f97 0 2024-09-16T00:26:05.327 INFO:tasks.workunit.client.0.smithi112.stdout:6/627: read dd/d23/f63 [5827555,47610] 0 2024-09-16T00:26:05.330 INFO:tasks.workunit.client.0.smithi112.stdout:6/628: creat dd/d23/fcf x:0 0 0 2024-09-16T00:26:05.330 INFO:tasks.workunit.client.0.smithi112.stdout:3/696: link d2/d9/l27 d2/d9/d2b/d47/d52/d7a/d80/d83/d68/d90/dcb/le8 0 2024-09-16T00:26:05.331 INFO:tasks.workunit.client.1.smithi169.stdout:2/452: symlink d7/db/d20/d5c/l98 0 2024-09-16T00:26:05.332 INFO:tasks.workunit.client.0.smithi112.stdout:3/697: write d2/d9/d2b/d2c/d69/fbf [631148,33531] 0 2024-09-16T00:26:05.334 INFO:tasks.workunit.client.0.smithi112.stdout:6/629: mknod dd/df/cd0 0 2024-09-16T00:26:05.335 INFO:tasks.workunit.client.1.smithi169.stdout:0/479: dwrite d0/d2c/d38/f66 [0,4194304] 0 2024-09-16T00:26:05.336 INFO:tasks.workunit.client.0.smithi112.stdout:6/630: stat dd/d15/da5/d79/f8d 0 2024-09-16T00:26:05.337 INFO:tasks.workunit.client.0.smithi112.stdout:9/875: dread d1/d5/d20/d81/d1a/d88/def/f2e [0,4194304] 0 2024-09-16T00:26:05.337 INFO:tasks.workunit.client.1.smithi169.stdout:2/453: creat d7/d6b/d70/d81/d69/f99 x:0 0 0 2024-09-16T00:26:05.338 INFO:tasks.workunit.client.0.smithi112.stdout:3/698: creat d2/d9/d2b/d47/d52/d7a/d80/d83/d68/d90/fe9 x:0 0 0 2024-09-16T00:26:05.338 INFO:tasks.workunit.client.0.smithi112.stdout:6/631: truncate dd/d23/d8f/fa8 95003 0 2024-09-16T00:26:05.339 INFO:tasks.workunit.client.0.smithi112.stdout:9/876: creat d1/d5/d6/df8/dba/f135 x:0 0 0 2024-09-16T00:26:05.341 INFO:tasks.workunit.client.0.smithi112.stdout:6/632: rmdir dd/df/d13/d7c/d88 39 2024-09-16T00:26:05.341 INFO:tasks.workunit.client.1.smithi169.stdout:0/480: fsync d0/d2/d8/f48 0 2024-09-16T00:26:05.342 INFO:tasks.workunit.client.0.smithi112.stdout:9/877: dread - d1/d5/d6/d38/ffd zero size 2024-09-16T00:26:05.342 INFO:tasks.workunit.client.0.smithi112.stdout:6/633: readlink dd/df/l39 0 2024-09-16T00:26:05.342 INFO:tasks.workunit.client.1.smithi169.stdout:3/508: dread d7/d1a/d1c/d35/d63/f6a [0,4194304] 0 2024-09-16T00:26:05.343 INFO:tasks.workunit.client.1.smithi169.stdout:6/676: link d2/d4d/d59/d38/c3b d2/d22/cd8 0 2024-09-16T00:26:05.343 INFO:tasks.workunit.client.1.smithi169.stdout:5/444: dread d2/da/d10/d13/d24/f47 [0,4194304] 0 2024-09-16T00:26:05.344 INFO:tasks.workunit.client.1.smithi169.stdout:0/481: creat d0/d2c/d54/fab x:0 0 0 2024-09-16T00:26:05.346 INFO:tasks.workunit.client.1.smithi169.stdout:2/454: rename d7/db/d27/c4b to d7/db/d20/d4c/c9a 0 2024-09-16T00:26:05.346 INFO:tasks.workunit.client.0.smithi112.stdout:9/878: fdatasync d1/d5/d6/d2d/d6a/fad 0 2024-09-16T00:26:05.347 INFO:tasks.workunit.client.0.smithi112.stdout:2/762: write d2/dd/d48/d22/fb1 [386658,92259] 0 2024-09-16T00:26:05.347 INFO:tasks.workunit.client.1.smithi169.stdout:3/509: fsync d7/d1a/d2b/f61 0 2024-09-16T00:26:05.348 INFO:tasks.workunit.client.0.smithi112.stdout:9/879: dread - d1/ff4 zero size 2024-09-16T00:26:05.349 INFO:tasks.workunit.client.0.smithi112.stdout:6/634: rename dd/df/d13/d7c to dd/df/d10/d47/d55/dc4/dd1 0 2024-09-16T00:26:05.350 INFO:tasks.workunit.client.1.smithi169.stdout:0/482: truncate d0/d2/d4b/f88 930188 0 2024-09-16T00:26:05.350 INFO:tasks.workunit.client.1.smithi169.stdout:6/677: rmdir d2/d3f/d41/d5c/d91/da6 39 2024-09-16T00:26:05.351 INFO:tasks.workunit.client.0.smithi112.stdout:3/699: dwrite d2/d9/f1e [0,4194304] 0 2024-09-16T00:26:05.352 INFO:tasks.workunit.client.0.smithi112.stdout:9/880: truncate d1/d5/d6/df8/ddd/f133 586562 0 2024-09-16T00:26:05.353 INFO:tasks.workunit.client.0.smithi112.stdout:3/700: stat d2/d9/d2b/d47/d52/d7a/d80/d83/d49/fa8 0 2024-09-16T00:26:05.353 INFO:tasks.workunit.client.0.smithi112.stdout:9/881: chown d1/d76/fdc 3114 1 2024-09-16T00:26:05.354 INFO:tasks.workunit.client.1.smithi169.stdout:2/455: symlink d7/d1c/d3c/l9b 0 2024-09-16T00:26:05.357 INFO:tasks.workunit.client.1.smithi169.stdout:3/510: fdatasync d7/f10 0 2024-09-16T00:26:05.358 INFO:tasks.workunit.client.0.smithi112.stdout:3/701: read d2/d9/d2b/d47/d52/fd3 [5511021,74095] 0 2024-09-16T00:26:05.358 INFO:tasks.workunit.client.0.smithi112.stdout:4/700: dread d6/db/df/d3c/f82 [0,4194304] 0 2024-09-16T00:26:05.359 INFO:tasks.workunit.client.1.smithi169.stdout:5/445: getdents d2/da/d10/d13/d24 0 2024-09-16T00:26:05.360 INFO:tasks.workunit.client.1.smithi169.stdout:6/678: dwrite d2/d3f/d41/d72/fa9 [0,4194304] 0 2024-09-16T00:26:05.360 INFO:tasks.workunit.client.0.smithi112.stdout:6/635: link c4 dd/df/cd2 0 2024-09-16T00:26:05.361 INFO:tasks.workunit.client.1.smithi169.stdout:5/446: stat d2/f6b 0 2024-09-16T00:26:05.361 INFO:tasks.workunit.client.0.smithi112.stdout:3/702: dread - d2/d9/d2b/d47/d52/d7a/d80/d83/d76/faf zero size 2024-09-16T00:26:05.362 INFO:tasks.workunit.client.0.smithi112.stdout:9/882: creat d1/d5/d6/d59/f136 x:0 0 0 2024-09-16T00:26:05.364 INFO:tasks.workunit.client.0.smithi112.stdout:2/763: truncate d2/dd/d48/d22/d2b/fa0 1330849 0 2024-09-16T00:26:05.366 INFO:tasks.workunit.client.1.smithi169.stdout:3/511: mkdir d7/d1a/d1c/d35/d63/dc2 0 2024-09-16T00:26:05.366 INFO:tasks.workunit.client.1.smithi169.stdout:2/456: dread d7/d1c/d26/f2c [0,4194304] 0 2024-09-16T00:26:05.367 INFO:tasks.workunit.client.0.smithi112.stdout:4/701: rename d6/d39/d36/d5c/d7e/f86 to d6/fff 0 2024-09-16T00:26:05.367 INFO:tasks.workunit.client.1.smithi169.stdout:6/679: rmdir d2/d3f/d41/d52 39 2024-09-16T00:26:05.368 INFO:tasks.workunit.client.0.smithi112.stdout:2/764: chown d2/dd/d48/d22/d2b/d54/lf5 852 1 2024-09-16T00:26:05.368 INFO:tasks.workunit.client.0.smithi112.stdout:4/702: read - d6/db/df/d5e/d61/fa6 zero size 2024-09-16T00:26:05.369 INFO:tasks.workunit.client.0.smithi112.stdout:2/765: write d2/dd/d48/d94/d49/f7d [954109,9084] 0 2024-09-16T00:26:05.370 INFO:tasks.workunit.client.1.smithi169.stdout:6/680: write d2/d3f/d41/d72/d9e/fab [819558,68485] 0 2024-09-16T00:26:05.371 INFO:tasks.workunit.client.0.smithi112.stdout:4/703: mknod d6/db/d1f/d41/d49/c100 0 2024-09-16T00:26:05.371 INFO:tasks.workunit.client.0.smithi112.stdout:2/766: fsync d2/dd/d48/d22/d2b/d54/d60/da9/fcc 0 2024-09-16T00:26:05.372 INFO:tasks.workunit.client.1.smithi169.stdout:2/457: rmdir d7/db/d20/d4c 39 2024-09-16T00:26:05.372 INFO:tasks.workunit.client.1.smithi169.stdout:0/483: mkdir d0/d2c/d54/d21/d31/d83/dac 0 2024-09-16T00:26:05.373 INFO:tasks.workunit.client.0.smithi112.stdout:2/767: write d2/dd/d48/d94/d49/fe9 [185266,112953] 0 2024-09-16T00:26:05.374 INFO:tasks.workunit.client.1.smithi169.stdout:5/447: link d2/da/d10/d13/d1c/d2d/f57 d2/da/d10/d13/d1c/d94/f99 0 2024-09-16T00:26:05.374 INFO:tasks.workunit.client.0.smithi112.stdout:2/768: rename d2 to d2/dd/df6 22 2024-09-16T00:26:05.376 INFO:tasks.workunit.client.0.smithi112.stdout:4/704: link d6/db/df/d5e/fcb d6/db/df/d3c/f101 0 2024-09-16T00:26:05.376 INFO:tasks.workunit.client.1.smithi169.stdout:2/458: symlink d7/db/d27/d29/d37/l9c 0 2024-09-16T00:26:05.378 INFO:tasks.workunit.client.1.smithi169.stdout:6/681: mknod d2/d3f/d41/d5c/d91/da6/cd9 0 2024-09-16T00:26:05.378 INFO:tasks.workunit.client.0.smithi112.stdout:4/705: creat d6/db/d1f/d41/d49/d31/f102 x:0 0 0 2024-09-16T00:26:05.379 INFO:tasks.workunit.client.1.smithi169.stdout:2/459: mkdir d7/d6b/d70/d81/d9d 0 2024-09-16T00:26:05.381 INFO:tasks.workunit.client.0.smithi112.stdout:4/706: creat d6/db/df/f103 x:0 0 0 2024-09-16T00:26:05.381 INFO:tasks.workunit.client.1.smithi169.stdout:3/512: getdents d7/d1a/d1c/d23/d70 0 2024-09-16T00:26:05.381 INFO:tasks.workunit.client.0.smithi112.stdout:2/769: dwrite d2/dd/d48/d22/d2b/d54/d60/d6c/d88/dad/deb/fcd [0,4194304] 0 2024-09-16T00:26:05.382 INFO:tasks.workunit.client.0.smithi112.stdout:4/707: fsync d6/db/d43/fea 0 2024-09-16T00:26:05.383 INFO:tasks.workunit.client.0.smithi112.stdout:6/636: truncate dd/d23/f64 1751699 0 2024-09-16T00:26:05.384 INFO:tasks.workunit.client.1.smithi169.stdout:0/484: rename d0/d2/l2d to d0/d2c/d54/d56/d67/lad 0 2024-09-16T00:26:05.384 INFO:tasks.workunit.client.0.smithi112.stdout:2/770: creat d2/dd/d48/d8a/ff7 x:0 0 0 2024-09-16T00:26:05.385 INFO:tasks.workunit.client.1.smithi169.stdout:0/485: stat d0/d2/d8/f47 0 2024-09-16T00:26:05.386 INFO:tasks.workunit.client.1.smithi169.stdout:2/460: symlink d7/d6b/d70/d81/d9d/l9e 0 2024-09-16T00:26:05.386 INFO:tasks.workunit.client.1.smithi169.stdout:2/461: write d7/f11 [1761708,84741] 0 2024-09-16T00:26:05.388 INFO:tasks.workunit.client.1.smithi169.stdout:6/682: link d2/d3f/d41/f46 d2/d22/fda 0 2024-09-16T00:26:05.389 INFO:tasks.workunit.client.1.smithi169.stdout:5/448: mkdir d2/da/d10/d13/d1c/d32/d53/d4e/d89/d9a 0 2024-09-16T00:26:05.390 INFO:tasks.workunit.client.0.smithi112.stdout:2/771: symlink d2/dd/d48/d22/d2b/d54/d60/d6c/d88/dad/lf8 0 2024-09-16T00:26:05.390 INFO:tasks.workunit.client.1.smithi169.stdout:3/513: getdents d7/d1a/d1c/d23/d88/dbe 0 2024-09-16T00:26:05.390 INFO:tasks.workunit.client.0.smithi112.stdout:6/637: dwrite fa [0,4194304] 0 2024-09-16T00:26:05.392 INFO:tasks.workunit.client.1.smithi169.stdout:2/462: creat d7/d6b/d70/d79/f9f x:0 0 0 2024-09-16T00:26:05.393 INFO:tasks.workunit.client.1.smithi169.stdout:5/449: mknod d2/da/d10/c9b 0 2024-09-16T00:26:05.394 INFO:tasks.workunit.client.1.smithi169.stdout:2/463: readlink d7/d1c/d26/l87 0 2024-09-16T00:26:05.395 INFO:tasks.workunit.client.1.smithi169.stdout:6/683: write d2/d3f/d41/d52/f89 [4109912,14507] 0 2024-09-16T00:26:05.396 INFO:tasks.workunit.client.0.smithi112.stdout:2/772: creat d2/dd/d48/d22/d2b/d54/d60/d6c/dc9/ff9 x:0 0 0 2024-09-16T00:26:05.396 INFO:tasks.workunit.client.0.smithi112.stdout:2/773: rename d2 to d2/dd/d48/d8a/dfa 22 2024-09-16T00:26:05.397 INFO:tasks.workunit.client.0.smithi112.stdout:2/774: write d2/dd/d48/d22/d67/f74 [608411,51178] 0 2024-09-16T00:26:05.400 INFO:tasks.workunit.client.1.smithi169.stdout:0/486: dwrite d0/d2c/d54/d21/f2f [0,4194304] 0 2024-09-16T00:26:05.400 INFO:tasks.workunit.client.0.smithi112.stdout:2/775: chown d2/dd/d48/d22/d67/d9a/fcf 565 1 2024-09-16T00:26:05.401 INFO:tasks.workunit.client.0.smithi112.stdout:2/776: write d2/dd/d48/d22/fd4 [1628069,53976] 0 2024-09-16T00:26:05.403 INFO:tasks.workunit.client.1.smithi169.stdout:0/487: write d0/d2c/d54/d21/f3f [4173253,43662] 0 2024-09-16T00:26:05.403 INFO:tasks.workunit.client.1.smithi169.stdout:5/450: unlink d2/da/d10/d13/d24/f2c 0 2024-09-16T00:26:05.406 INFO:tasks.workunit.client.1.smithi169.stdout:6/684: unlink d2/d36/cd0 0 2024-09-16T00:26:05.406 INFO:tasks.workunit.client.1.smithi169.stdout:2/464: creat d7/fa0 x:0 0 0 2024-09-16T00:26:05.408 INFO:tasks.workunit.client.1.smithi169.stdout:0/488: creat d0/d2/d8/d5/d79/fae x:0 0 0 2024-09-16T00:26:05.409 INFO:tasks.workunit.client.1.smithi169.stdout:3/514: dwrite d7/d1a/d1c/d35/d63/d37/f81 [0,4194304] 0 2024-09-16T00:26:05.410 INFO:tasks.workunit.client.1.smithi169.stdout:5/451: mkdir d2/da/d10/d13/d1c/d40/d54/d9c 0 2024-09-16T00:26:05.412 INFO:tasks.workunit.client.0.smithi112.stdout:2/777: rename d2/dd/d48/d8a/d9b/cd9 to d2/dd/d48/d22/d2b/d54/d60/d6c/d88/da4/cfb 0 2024-09-16T00:26:05.414 INFO:tasks.workunit.client.0.smithi112.stdout:6/638: dwrite dd/df/d17/d27/d84/f36 [0,4194304] 0 2024-09-16T00:26:05.440 INFO:tasks.workunit.client.0.smithi112.stdout:2/778: truncate d2/dd/d48/d22/d2b/d30/f44 183747 0 2024-09-16T00:26:05.442 INFO:tasks.workunit.client.0.smithi112.stdout:2/779: read d2/dd/d48/d94/d49/fe9 [63999,44384] 0 2024-09-16T00:26:05.442 INFO:tasks.workunit.client.0.smithi112.stdout:2/780: chown d2/dd/c45 2097 1 2024-09-16T00:26:05.443 INFO:tasks.workunit.client.0.smithi112.stdout:2/781: symlink d2/dd/d48/lfc 0 2024-09-16T00:26:05.446 INFO:tasks.workunit.client.0.smithi112.stdout:2/782: rename d2/c5f to d2/dd/cfd 0 2024-09-16T00:26:05.450 INFO:tasks.workunit.client.0.smithi112.stdout:6/639: dread dd/df/d10/d47/d55/f7a [4194304,4194304] 0 2024-09-16T00:26:05.450 INFO:tasks.workunit.client.0.smithi112.stdout:2/783: rmdir d2/dd/d48/d22/d2b/d54/d60/d6c/d88/dd7 0 2024-09-16T00:26:05.452 INFO:tasks.workunit.client.0.smithi112.stdout:6/640: creat dd/df/d13/d18/fd3 x:0 0 0 2024-09-16T00:26:05.455 INFO:tasks.workunit.client.0.smithi112.stdout:2/784: dread d2/dd/d48/d22/d2b/d54/d60/fc5 [0,4194304] 0 2024-09-16T00:26:05.460 INFO:tasks.workunit.client.0.smithi112.stdout:2/785: dwrite d2/dd/d48/f1d [0,4194304] 0 2024-09-16T00:26:05.460 INFO:tasks.workunit.client.1.smithi169.stdout:0/489: mkdir d0/d2c/d54/d56/d67/daf 0 2024-09-16T00:26:05.461 INFO:tasks.workunit.client.1.smithi169.stdout:3/515: dread - d7/d1a/d2b/d8d/f94 zero size 2024-09-16T00:26:05.461 INFO:tasks.workunit.client.1.smithi169.stdout:2/465: truncate d7/db/d27/d4f/f80 1046525 0 2024-09-16T00:26:05.465 INFO:tasks.workunit.client.0.smithi112.stdout:2/786: rmdir d2/dd/d48/d22/d2b/d54/d60/d6c/d88/dad 39 2024-09-16T00:26:05.465 INFO:tasks.workunit.client.0.smithi112.stdout:2/787: stat d2/dd/d48/d22/d67/d9a/fcf 0 2024-09-16T00:26:05.465 INFO:tasks.workunit.client.1.smithi169.stdout:5/452: fsync d2/da/d10/d13/f35 0 2024-09-16T00:26:05.465 INFO:tasks.workunit.client.1.smithi169.stdout:6/685: creat d2/fdb x:0 0 0 2024-09-16T00:26:05.465 INFO:tasks.workunit.client.1.smithi169.stdout:3/516: write d7/d1a/d1c/d35/f93 [816329,70032] 0 2024-09-16T00:26:05.472 INFO:tasks.workunit.client.0.smithi112.stdout:2/788: truncate d2/dd/d48/d22/d2b/d54/d60/daf/fbc 3516407 0 2024-09-16T00:26:05.477 INFO:tasks.workunit.client.0.smithi112.stdout:2/789: dwrite d2/dd/d48/d94/fe5 [0,4194304] 0 2024-09-16T00:26:05.481 INFO:tasks.workunit.client.0.smithi112.stdout:2/790: dread d2/dd/d48/d22/d67/f74 [0,4194304] 0 2024-09-16T00:26:05.488 INFO:tasks.workunit.client.1.smithi169.stdout:3/517: creat d7/d1a/d2b/fc3 x:0 0 0 2024-09-16T00:26:05.488 INFO:tasks.workunit.client.1.smithi169.stdout:6/686: dwrite d2/d3f/d95/fb2 [0,4194304] 0 2024-09-16T00:26:05.489 INFO:tasks.workunit.client.1.smithi169.stdout:0/490: link d0/d2/d8/d5/f44 d0/d2c/d4f/da5/fb0 0 2024-09-16T00:26:05.490 INFO:tasks.workunit.client.1.smithi169.stdout:5/453: truncate d2/da/f19 1948170 0 2024-09-16T00:26:05.491 INFO:tasks.workunit.client.0.smithi112.stdout:2/791: truncate d2/dd/d48/d94/d49/f71 126733 0 2024-09-16T00:26:05.492 INFO:tasks.workunit.client.0.smithi112.stdout:2/792: fsync d2/dd/d48/f85 0 2024-09-16T00:26:05.495 INFO:tasks.workunit.client.0.smithi112.stdout:2/793: rename d2/dd/l86 to d2/d68/lfe 0 2024-09-16T00:26:05.495 INFO:tasks.workunit.client.1.smithi169.stdout:0/491: truncate d0/d2c/d54/d21/d31/f85 1046254 0 2024-09-16T00:26:05.497 INFO:tasks.workunit.client.0.smithi112.stdout:2/794: creat d2/d68/fff x:0 0 0 2024-09-16T00:26:05.497 INFO:tasks.workunit.client.1.smithi169.stdout:2/466: dwrite f0 [0,4194304] 0 2024-09-16T00:26:05.499 INFO:tasks.workunit.client.0.smithi112.stdout:2/795: fdatasync d2/dd/d48/d22/d2b/d54/d60/d6c/d88/da4/fc0 0 2024-09-16T00:26:05.499 INFO:tasks.workunit.client.1.smithi169.stdout:0/492: creat d0/d7d/fb1 x:0 0 0 2024-09-16T00:26:05.501 INFO:tasks.workunit.client.1.smithi169.stdout:2/467: mknod d7/db/d27/d4f/ca1 0 2024-09-16T00:26:05.502 INFO:tasks.workunit.client.0.smithi112.stdout:2/796: dread d2/dd/d48/f1d [0,4194304] 0 2024-09-16T00:26:05.504 INFO:tasks.workunit.client.1.smithi169.stdout:2/468: truncate f2 1879346 0 2024-09-16T00:26:05.508 INFO:tasks.workunit.client.1.smithi169.stdout:2/469: mkdir d7/d6b/d70/d81/d69/da2 0 2024-09-16T00:26:05.508 INFO:tasks.workunit.client.1.smithi169.stdout:0/493: dwrite d0/d2c/d38/f66 [0,4194304] 0 2024-09-16T00:26:05.509 INFO:tasks.workunit.client.1.smithi169.stdout:2/470: stat d7/d1c/d3c/l68 0 2024-09-16T00:26:05.510 INFO:tasks.workunit.client.1.smithi169.stdout:0/494: symlink d0/d2c/d4f/lb2 0 2024-09-16T00:26:05.510 INFO:tasks.workunit.client.1.smithi169.stdout:2/471: chown d7/db/d27/d4f/d56/d6d/l91 18682 1 2024-09-16T00:26:05.515 INFO:tasks.workunit.client.1.smithi169.stdout:2/472: link d7/d1c/d3c/l68 d7/db/d27/d4f/d56/d6d/la3 0 2024-09-16T00:26:05.517 INFO:tasks.workunit.client.0.smithi112.stdout:0/724: sync 2024-09-16T00:26:05.517 INFO:tasks.workunit.client.1.smithi169.stdout:6/687: truncate d2/d22/fa1 502513 0 2024-09-16T00:26:05.518 INFO:tasks.workunit.client.1.smithi169.stdout:2/473: rename d7/d6b/d70/d81/d69/d92 to d7/d6b/d70/d81/d69/d92/da4 22 2024-09-16T00:26:05.519 INFO:tasks.workunit.client.0.smithi112.stdout:0/725: truncate d5/f33 559907 0 2024-09-16T00:26:05.522 INFO:tasks.workunit.client.1.smithi169.stdout:6/688: rename d2/d3f/d41/d5c/d73/f5b to d2/d3f/d41/d72/d9e/dc0/fdc 0 2024-09-16T00:26:05.527 INFO:tasks.workunit.client.1.smithi169.stdout:3/518: dread d7/d1a/d1c/d35/d63/f4a [0,4194304] 0 2024-09-16T00:26:05.529 INFO:tasks.workunit.client.1.smithi169.stdout:2/474: link d7/db/d20/d5c/l5e d7/d1c/la5 0 2024-09-16T00:26:05.530 INFO:tasks.workunit.client.1.smithi169.stdout:6/689: dwrite d2/d3f/f85 [0,4194304] 0 2024-09-16T00:26:05.532 INFO:tasks.workunit.client.0.smithi112.stdout:0/726: dwrite d5/da/d18/d24/d46/d5f/d7a/fa2 [0,4194304] 0 2024-09-16T00:26:05.532 INFO:tasks.workunit.client.1.smithi169.stdout:3/519: symlink d7/d50/d90/lc4 0 2024-09-16T00:26:05.533 INFO:tasks.workunit.client.0.smithi112.stdout:0/727: chown d5/da/d1d/d71/l36 3773004 1 2024-09-16T00:26:05.534 INFO:tasks.workunit.client.1.smithi169.stdout:6/690: creat d2/d3f/d41/d72/d9e/fdd x:0 0 0 2024-09-16T00:26:05.536 INFO:tasks.workunit.client.0.smithi112.stdout:0/728: dread - d5/da/d1d/d71/fa9 zero size 2024-09-16T00:26:05.543 INFO:tasks.workunit.client.1.smithi169.stdout:2/475: getdents d7/d6b 0 2024-09-16T00:26:05.544 INFO:tasks.workunit.client.1.smithi169.stdout:2/476: stat d7/db/d20/c49 0 2024-09-16T00:26:05.545 INFO:tasks.workunit.client.1.smithi169.stdout:6/691: creat d2/d3f/fde x:0 0 0 2024-09-16T00:26:05.548 INFO:tasks.workunit.client.1.smithi169.stdout:2/477: symlink d7/d6b/d70/d81/la6 0 2024-09-16T00:26:05.551 INFO:tasks.workunit.client.1.smithi169.stdout:2/478: creat d7/db/d20/fa7 x:0 0 0 2024-09-16T00:26:05.552 INFO:tasks.workunit.client.1.smithi169.stdout:6/692: dread d2/d4d/d59/f35 [0,4194304] 0 2024-09-16T00:26:05.554 INFO:tasks.workunit.client.1.smithi169.stdout:2/479: symlink d7/d1c/d26/la8 0 2024-09-16T00:26:05.554 INFO:tasks.workunit.client.1.smithi169.stdout:6/693: chown d2/d22/da7/fcd 210 1 2024-09-16T00:26:05.554 INFO:tasks.workunit.client.0.smithi112.stdout:0/729: dwrite d5/da/d66/f5d [0,4194304] 0 2024-09-16T00:26:05.556 INFO:tasks.workunit.client.1.smithi169.stdout:2/480: unlink d7/d6b/d70/d79/f7b 0 2024-09-16T00:26:05.558 INFO:tasks.workunit.client.1.smithi169.stdout:6/694: creat d2/d4c/d9f/fdf x:0 0 0 2024-09-16T00:26:05.560 INFO:tasks.workunit.client.1.smithi169.stdout:3/520: dread d7/d50/d6f/f74 [0,4194304] 0 2024-09-16T00:26:05.560 INFO:tasks.workunit.client.1.smithi169.stdout:2/481: rename d7/d1c/d3c/l9b to d7/db/d27/d29/d77/la9 0 2024-09-16T00:26:05.561 INFO:tasks.workunit.client.1.smithi169.stdout:6/695: read d2/d3f/d41/f80 [82904,86366] 0 2024-09-16T00:26:05.562 INFO:tasks.workunit.client.1.smithi169.stdout:3/521: mkdir d7/d1a/d1c/d23/d70/dc5 0 2024-09-16T00:26:05.564 INFO:tasks.workunit.client.0.smithi112.stdout:0/730: dwrite d5/da/d18/d24/d5a/f6c [0,4194304] 0 2024-09-16T00:26:05.564 INFO:tasks.workunit.client.1.smithi169.stdout:2/482: dread d7/db/f17 [0,4194304] 0 2024-09-16T00:26:05.566 INFO:tasks.workunit.client.1.smithi169.stdout:2/483: write d7/db/d27/f90 [4029903,125208] 0 2024-09-16T00:26:05.566 INFO:tasks.workunit.client.1.smithi169.stdout:6/696: creat d2/d36/fe0 x:0 0 0 2024-09-16T00:26:05.567 INFO:tasks.workunit.client.1.smithi169.stdout:2/484: chown f5 19 1 2024-09-16T00:26:05.567 INFO:tasks.workunit.client.1.smithi169.stdout:2/485: chown d7/d1c/c1f 0 1 2024-09-16T00:26:05.569 INFO:tasks.workunit.client.0.smithi112.stdout:0/731: getdents d5/da/d66 0 2024-09-16T00:26:05.569 INFO:tasks.workunit.client.1.smithi169.stdout:3/522: rmdir d7/d1a/d1c/d23/d70/db9 0 2024-09-16T00:26:05.571 INFO:tasks.workunit.client.0.smithi112.stdout:0/732: truncate d5/da/d18/d24/d46/d5f/da6/f50 1640027 0 2024-09-16T00:26:05.572 INFO:tasks.workunit.client.0.smithi112.stdout:0/733: chown d5/da/d1d/d71/fbc 2612 1 2024-09-16T00:26:05.573 INFO:tasks.workunit.client.0.smithi112.stdout:0/734: symlink d5/da/d18/d89/ldc 0 2024-09-16T00:26:05.580 INFO:tasks.workunit.client.1.smithi169.stdout:3/523: mkdir d7/d1a/d2b/dc6 0 2024-09-16T00:26:05.581 INFO:tasks.workunit.client.1.smithi169.stdout:3/524: write d7/d1a/d2b/f8c [325729,70257] 0 2024-09-16T00:26:05.582 INFO:tasks.workunit.client.1.smithi169.stdout:2/486: rename d7/db/d20/d4c to d7/daa 0 2024-09-16T00:26:05.584 INFO:tasks.workunit.client.1.smithi169.stdout:2/487: mkdir d7/d1c/d26/d96/dab 0 2024-09-16T00:26:05.584 INFO:tasks.workunit.client.1.smithi169.stdout:2/488: fdatasync d7/db/f23 0 2024-09-16T00:26:05.586 INFO:tasks.workunit.client.1.smithi169.stdout:2/489: creat d7/d6b/d70/d81/d69/fac x:0 0 0 2024-09-16T00:26:05.588 INFO:tasks.workunit.client.0.smithi112.stdout:0/735: dread d5/da/d1d/d71/f5c [0,4194304] 0 2024-09-16T00:26:05.588 INFO:tasks.workunit.client.1.smithi169.stdout:2/490: creat d7/db/d27/d29/d77/fad x:0 0 0 2024-09-16T00:26:05.591 INFO:tasks.workunit.client.1.smithi169.stdout:2/491: dread d7/db/f23 [4194304,4194304] 0 2024-09-16T00:26:05.591 INFO:tasks.workunit.client.0.smithi112.stdout:0/736: link d5/da/d1d/c87 d5/da/d1d/d71/d64/cdd 0 2024-09-16T00:26:05.592 INFO:tasks.workunit.client.1.smithi169.stdout:2/492: truncate d7/d6b/d70/d79/f9f 570953 0 2024-09-16T00:26:05.593 INFO:tasks.workunit.client.1.smithi169.stdout:2/493: write d7/db/d27/d29/f54 [506616,99758] 0 2024-09-16T00:26:05.595 INFO:tasks.workunit.client.1.smithi169.stdout:2/494: mkdir d7/d6b/d70/d79/dae 0 2024-09-16T00:26:05.597 INFO:tasks.workunit.client.1.smithi169.stdout:2/495: mkdir d7/db/d27/d4f/d56/daf 0 2024-09-16T00:26:05.598 INFO:tasks.workunit.client.0.smithi112.stdout:0/737: truncate d5/da/d18/d24/d46/d88/f8c 3193993 0 2024-09-16T00:26:05.601 INFO:tasks.workunit.client.0.smithi112.stdout:0/738: unlink d5/da/d18/d89/f98 0 2024-09-16T00:26:05.603 INFO:tasks.workunit.client.1.smithi169.stdout:2/496: dwrite d7/d6b/d70/d81/d69/fac [0,4194304] 0 2024-09-16T00:26:05.608 INFO:tasks.workunit.client.0.smithi112.stdout:0/739: dwrite d5/da/d18/d24/d46/d5f/d7a/fa2 [0,4194304] 0 2024-09-16T00:26:05.609 INFO:tasks.workunit.client.0.smithi112.stdout:0/740: chown d5/da/d18/c26 420489926 1 2024-09-16T00:26:05.610 INFO:tasks.workunit.client.1.smithi169.stdout:2/497: dwrite d7/f11 [0,4194304] 0 2024-09-16T00:26:05.614 INFO:tasks.workunit.client.0.smithi112.stdout:0/741: rename d5/da/d18/d24/d5a/f82 to d5/da/d18/d24/d5a/dc0/fde 0 2024-09-16T00:26:05.614 INFO:tasks.workunit.client.0.smithi112.stdout:0/742: dread - d5/da/d21/fb9 zero size 2024-09-16T00:26:05.616 INFO:tasks.workunit.client.0.smithi112.stdout:0/743: write d5/da/d21/da5/faa [5114088,11646] 0 2024-09-16T00:26:05.617 INFO:tasks.workunit.client.0.smithi112.stdout:0/744: creat d5/da/d21/fdf x:0 0 0 2024-09-16T00:26:05.620 INFO:tasks.workunit.client.0.smithi112.stdout:0/745: creat d5/da/d66/fe0 x:0 0 0 2024-09-16T00:26:05.622 INFO:tasks.workunit.client.0.smithi112.stdout:0/746: truncate d5/da/d57/fc8 31985 0 2024-09-16T00:26:05.623 INFO:tasks.workunit.client.1.smithi169.stdout:2/498: link d7/db/d27/l30 d7/db/d27/d29/d37/lb0 0 2024-09-16T00:26:05.624 INFO:tasks.workunit.client.0.smithi112.stdout:0/747: symlink d5/d90/dc5/le1 0 2024-09-16T00:26:05.627 INFO:tasks.workunit.client.0.smithi112.stdout:0/748: getdents d5/da/d21/dc3 0 2024-09-16T00:26:05.627 INFO:tasks.workunit.client.1.smithi169.stdout:2/499: getdents d7/d6b/d70/d81/d69/da2 0 2024-09-16T00:26:05.628 INFO:tasks.workunit.client.0.smithi112.stdout:0/749: chown d5/da/d57/f73 32149 1 2024-09-16T00:26:05.629 INFO:tasks.workunit.client.0.smithi112.stdout:0/750: mkdir d5/d90/dc5/de2 0 2024-09-16T00:26:05.633 INFO:tasks.workunit.client.1.smithi169.stdout:2/500: dwrite d7/d1c/f95 [0,4194304] 0 2024-09-16T00:26:05.635 INFO:tasks.workunit.client.1.smithi169.stdout:2/501: rmdir d7/daa 39 2024-09-16T00:26:05.636 INFO:tasks.workunit.client.0.smithi112.stdout:0/751: dread d5/da/d57/db0/f9e [0,4194304] 0 2024-09-16T00:26:05.637 INFO:tasks.workunit.client.1.smithi169.stdout:2/502: write d7/d1c/f3e [3648059,104513] 0 2024-09-16T00:26:05.645 INFO:tasks.workunit.client.1.smithi169.stdout:2/503: dwrite d7/d1c/d26/f76 [0,4194304] 0 2024-09-16T00:26:05.649 INFO:tasks.workunit.client.1.smithi169.stdout:2/504: symlink d7/db/d27/d4f/d56/daf/lb1 0 2024-09-16T00:26:05.652 INFO:tasks.workunit.client.1.smithi169.stdout:2/505: truncate d7/db/f13 972616 0 2024-09-16T00:26:05.654 INFO:tasks.workunit.client.1.smithi169.stdout:2/506: rmdir d7/d1c/d26 39 2024-09-16T00:26:05.656 INFO:tasks.workunit.client.1.smithi169.stdout:2/507: mkdir d7/db/d20/d5c/db2 0 2024-09-16T00:26:05.657 INFO:tasks.workunit.client.1.smithi169.stdout:2/508: mknod d7/d6b/d70/d81/d69/da2/cb3 0 2024-09-16T00:26:05.658 INFO:tasks.workunit.client.1.smithi169.stdout:2/509: chown d7/d6b/d70/d81/d69/d92 183890 1 2024-09-16T00:26:05.659 INFO:tasks.workunit.client.1.smithi169.stdout:2/510: fsync d7/d6b/d70/d81/d69/fac 0 2024-09-16T00:26:05.668 INFO:tasks.workunit.client.1.smithi169.stdout:2/511: rename d7/d6b/d70/d81 to d7/db/d89/db4 0 2024-09-16T00:26:05.668 INFO:tasks.workunit.client.1.smithi169.stdout:2/512: write d7/db/d20/fa7 [6634,74642] 0 2024-09-16T00:26:05.668 INFO:tasks.workunit.client.1.smithi169.stdout:2/513: unlink d7/d1c/f3e 0 2024-09-16T00:26:05.669 INFO:tasks.workunit.client.0.smithi112.stdout:7/690: sync 2024-09-16T00:26:05.670 INFO:tasks.workunit.client.1.smithi169.stdout:2/514: rename d7/db/d27/d4f/d56/l7e to d7/db/d20/d5c/lb5 0 2024-09-16T00:26:05.675 INFO:tasks.workunit.client.1.smithi169.stdout:2/515: readlink d7/db/d20/d5c/lb5 0 2024-09-16T00:26:05.676 INFO:tasks.workunit.client.1.smithi169.stdout:2/516: truncate d7/d1c/d26/d96/f8e 1010054 0 2024-09-16T00:26:05.678 INFO:tasks.workunit.client.1.smithi169.stdout:2/517: mknod d7/db/d89/db4/d69/d92/cb6 0 2024-09-16T00:26:05.680 INFO:tasks.workunit.client.1.smithi169.stdout:2/518: creat d7/d1c/fb7 x:0 0 0 2024-09-16T00:26:05.680 INFO:tasks.workunit.client.1.smithi169.stdout:2/519: chown d7/db/c93 7162814 1 2024-09-16T00:26:05.683 INFO:tasks.workunit.client.1.smithi169.stdout:2/520: rename d7/db/d27/d4f/d56 to d7/db/db8 0 2024-09-16T00:26:05.684 INFO:tasks.workunit.client.0.smithi112.stdout:0/752: truncate d5/da/dc/f81 282274 0 2024-09-16T00:26:05.684 INFO:tasks.workunit.client.0.smithi112.stdout:7/691: fdatasync d6/d10/d16/f59 0 2024-09-16T00:26:05.685 INFO:tasks.workunit.client.0.smithi112.stdout:0/753: write d5/da/d21/fb5 [968657,100822] 0 2024-09-16T00:26:05.685 INFO:tasks.workunit.client.1.smithi169.stdout:2/521: mkdir d7/db/d89/db4/d9d/db9 0 2024-09-16T00:26:05.687 INFO:tasks.workunit.client.0.smithi112.stdout:7/692: unlink d6/d10/d16/d3b/l58 0 2024-09-16T00:26:05.688 INFO:tasks.workunit.client.1.smithi169.stdout:2/522: truncate d7/db/d27/d29/d37/f97 504749 0 2024-09-16T00:26:05.689 INFO:tasks.workunit.client.1.smithi169.stdout:2/523: rmdir d7/db/db8/daf 39 2024-09-16T00:26:05.690 INFO:tasks.workunit.client.1.smithi169.stdout:2/524: write d7/d1c/d26/d96/f8e [800559,118371] 0 2024-09-16T00:26:05.695 INFO:tasks.workunit.client.1.smithi169.stdout:2/525: dread d7/db/d20/f83 [0,4194304] 0 2024-09-16T00:26:05.695 INFO:tasks.workunit.client.0.smithi112.stdout:7/693: dwrite d6/d10/f15 [4194304,4194304] 0 2024-09-16T00:26:05.697 INFO:tasks.workunit.client.1.smithi169.stdout:2/526: creat d7/db/db8/d6d/fba x:0 0 0 2024-09-16T00:26:05.697 INFO:tasks.workunit.client.0.smithi112.stdout:7/694: fsync d6/fa0 0 2024-09-16T00:26:05.697 INFO:tasks.workunit.client.0.smithi112.stdout:0/754: dwrite d5/da/d66/f99 [0,4194304] 0 2024-09-16T00:26:05.698 INFO:tasks.workunit.client.0.smithi112.stdout:7/695: write d6/d10/f71 [2208789,38215] 0 2024-09-16T00:26:05.698 INFO:tasks.workunit.client.1.smithi169.stdout:2/527: symlink d7/db/d27/lbb 0 2024-09-16T00:26:05.699 INFO:tasks.workunit.client.1.smithi169.stdout:2/528: fdatasync d7/d1c/f72 0 2024-09-16T00:26:05.700 INFO:tasks.workunit.client.1.smithi169.stdout:2/529: chown d7/d6b/d70/d79 1914928 1 2024-09-16T00:26:05.701 INFO:tasks.workunit.client.0.smithi112.stdout:0/755: mkdir d5/d90/dc5/de3 0 2024-09-16T00:26:05.702 INFO:tasks.workunit.client.1.smithi169.stdout:2/530: symlink d7/db/d20/d5c/db2/lbc 0 2024-09-16T00:26:05.702 INFO:tasks.workunit.client.0.smithi112.stdout:7/696: read - d6/d10/d16/d96/fcc zero size 2024-09-16T00:26:05.703 INFO:tasks.workunit.client.1.smithi169.stdout:2/531: rmdir d7/db/d20/d5c/db2 39 2024-09-16T00:26:05.703 INFO:tasks.workunit.client.0.smithi112.stdout:0/756: chown d5/da/d66/fb7 1039 1 2024-09-16T00:26:05.704 INFO:tasks.workunit.client.0.smithi112.stdout:0/757: fdatasync d5/da/d18/d24/d5a/fbf 0 2024-09-16T00:26:05.705 INFO:tasks.workunit.client.0.smithi112.stdout:0/758: creat d5/da/d21/fe4 x:0 0 0 2024-09-16T00:26:05.706 INFO:tasks.workunit.client.1.smithi169.stdout:2/532: truncate d7/db/f23 869944 0 2024-09-16T00:26:05.706 INFO:tasks.workunit.client.0.smithi112.stdout:0/759: chown d5/da/d66/fb7 484622 1 2024-09-16T00:26:05.708 INFO:tasks.workunit.client.0.smithi112.stdout:0/760: mknod d5/da/dc/ce5 0 2024-09-16T00:26:05.710 INFO:tasks.workunit.client.0.smithi112.stdout:0/761: mkdir d5/da/dc/de6 0 2024-09-16T00:26:05.711 INFO:tasks.workunit.client.0.smithi112.stdout:0/762: rmdir d5/da/d21/dc3 39 2024-09-16T00:26:05.712 INFO:tasks.workunit.client.0.smithi112.stdout:0/763: chown d5/da/d66/fb7 413835 1 2024-09-16T00:26:05.713 INFO:tasks.workunit.client.1.smithi169.stdout:2/533: rename d7/db/d27/d4f to d7/d1c/d26/d96/dbd 0 2024-09-16T00:26:05.713 INFO:tasks.workunit.client.0.smithi112.stdout:7/697: write d6/d10/d75/d88/f2d [5098537,57404] 0 2024-09-16T00:26:05.714 INFO:tasks.workunit.client.0.smithi112.stdout:0/764: chown d5/da/d57/db0/f79 299469 1 2024-09-16T00:26:05.717 INFO:tasks.workunit.client.0.smithi112.stdout:0/765: unlink d5/da/d1d/d71/fb8 0 2024-09-16T00:26:05.717 INFO:tasks.workunit.client.0.smithi112.stdout:0/766: chown d5/da/dc/la0 7673488 1 2024-09-16T00:26:05.720 INFO:tasks.workunit.client.1.smithi169.stdout:2/534: rmdir d7/db/d27 39 2024-09-16T00:26:05.720 INFO:tasks.workunit.client.0.smithi112.stdout:0/767: link d5/da/d66/fc9 d5/da/d21/fe7 0 2024-09-16T00:26:05.722 INFO:tasks.workunit.client.1.smithi169.stdout:2/535: mkdir d7/db/d89/db4/d9d/dbe 0 2024-09-16T00:26:05.722 INFO:tasks.workunit.client.1.smithi169.stdout:2/536: chown d7/db/d89/db4/d69/fac 612 1 2024-09-16T00:26:05.723 INFO:tasks.workunit.client.0.smithi112.stdout:0/768: truncate d5/da/d18/f1f 1546090 0 2024-09-16T00:26:05.723 INFO:tasks.workunit.client.0.smithi112.stdout:0/769: dread - d5/da/d21/fe7 zero size 2024-09-16T00:26:05.723 INFO:tasks.workunit.client.1.smithi169.stdout:2/537: write f3 [2836931,122546] 0 2024-09-16T00:26:05.724 INFO:tasks.workunit.client.0.smithi112.stdout:0/770: readlink d5/da/d18/d24/l44 0 2024-09-16T00:26:05.725 INFO:tasks.workunit.client.1.smithi169.stdout:2/538: creat d7/db/d89/fbf x:0 0 0 2024-09-16T00:26:05.725 INFO:tasks.workunit.client.0.smithi112.stdout:0/771: symlink d5/da/d18/d24/d5a/le8 0 2024-09-16T00:26:05.727 INFO:tasks.workunit.client.0.smithi112.stdout:0/772: truncate d5/da/d18/d89/fab 565805 0 2024-09-16T00:26:05.728 INFO:tasks.workunit.client.1.smithi169.stdout:2/539: dread d7/d1c/f95 [0,4194304] 0 2024-09-16T00:26:05.729 INFO:tasks.workunit.client.0.smithi112.stdout:0/773: symlink d5/d90/dc5/de3/le9 0 2024-09-16T00:26:05.730 INFO:tasks.workunit.client.0.smithi112.stdout:0/774: mkdir d5/da/d1d/d71/d64/dea 0 2024-09-16T00:26:05.731 INFO:tasks.workunit.client.0.smithi112.stdout:0/775: chown d5/da/d21/dc3/dd3/dd5 151208 1 2024-09-16T00:26:05.732 INFO:tasks.workunit.client.1.smithi169.stdout:2/540: creat d7/db/d27/d29/fc0 x:0 0 0 2024-09-16T00:26:05.732 INFO:tasks.workunit.client.1.smithi169.stdout:2/541: stat d7/db/db8/d6d 0 2024-09-16T00:26:05.735 INFO:tasks.workunit.client.1.smithi169.stdout:2/542: write f5 [1101221,70574] 0 2024-09-16T00:26:05.737 INFO:tasks.workunit.client.1.smithi169.stdout:2/543: dread d7/db/d20/f83 [0,4194304] 0 2024-09-16T00:26:05.741 INFO:tasks.workunit.client.0.smithi112.stdout:0/776: dread d5/da/d57/d76/f17 [0,4194304] 0 2024-09-16T00:26:05.746 INFO:tasks.workunit.client.0.smithi112.stdout:0/777: write d5/da/d1d/d71/fa9 [86230,21168] 0 2024-09-16T00:26:05.747 INFO:tasks.workunit.client.0.smithi112.stdout:0/778: rmdir d5/da/d18/d24 39 2024-09-16T00:26:05.759 INFO:tasks.workunit.client.1.smithi169.stdout:7/536: sync 2024-09-16T00:26:05.763 INFO:tasks.workunit.client.1.smithi169.stdout:7/537: dwrite f5 [0,4194304] 0 2024-09-16T00:26:05.779 INFO:tasks.workunit.client.1.smithi169.stdout:7/538: dread d7/d71/d76/d29/f47 [0,4194304] 0 2024-09-16T00:26:05.782 INFO:tasks.workunit.client.1.smithi169.stdout:7/539: truncate d7/f9f 608212 0 2024-09-16T00:26:05.783 INFO:tasks.workunit.client.1.smithi169.stdout:7/540: symlink d7/d16/d49/lad 0 2024-09-16T00:26:05.784 INFO:tasks.workunit.client.1.smithi169.stdout:7/541: read f1 [3921668,6156] 0 2024-09-16T00:26:05.786 INFO:tasks.workunit.client.1.smithi169.stdout:7/542: creat d7/d71/fae x:0 0 0 2024-09-16T00:26:05.788 INFO:tasks.workunit.client.1.smithi169.stdout:7/543: dread - d7/d5b/f87 zero size 2024-09-16T00:26:05.804 INFO:tasks.workunit.client.1.smithi169.stdout:7/544: dread d7/d5b/d61/f6e [0,4194304] 0 2024-09-16T00:26:05.810 INFO:tasks.workunit.client.1.smithi169.stdout:7/545: dwrite d7/d16/f6a [0,4194304] 0 2024-09-16T00:26:05.813 INFO:tasks.workunit.client.0.smithi112.stdout:5/578: sync 2024-09-16T00:26:05.814 INFO:tasks.workunit.client.0.smithi112.stdout:5/579: chown d4/dd/db7 119297027 1 2024-09-16T00:26:05.814 INFO:tasks.workunit.client.0.smithi112.stdout:5/580: chown d4/d7/d9b 343080334 1 2024-09-16T00:26:05.815 INFO:tasks.workunit.client.0.smithi112.stdout:5/581: write d4/dd/f95 [2355960,91946] 0 2024-09-16T00:26:05.815 INFO:tasks.workunit.client.1.smithi169.stdout:7/546: creat d7/df/faf x:0 0 0 2024-09-16T00:26:05.815 INFO:tasks.workunit.client.1.smithi169.stdout:7/547: truncate d7/d5b/f9a 1859038 0 2024-09-16T00:26:05.816 INFO:tasks.workunit.client.0.smithi112.stdout:5/582: readlink d4/d7/l21 0 2024-09-16T00:26:05.823 INFO:tasks.workunit.client.1.smithi169.stdout:7/548: dwrite d7/d5b/d61/f6e [0,4194304] 0 2024-09-16T00:26:05.824 INFO:tasks.workunit.client.0.smithi112.stdout:5/583: dwrite d4/d4f/d77/fa2 [0,4194304] 0 2024-09-16T00:26:05.831 INFO:tasks.workunit.client.1.smithi169.stdout:7/549: truncate d7/d16/f1d 7890556 0 2024-09-16T00:26:05.833 INFO:tasks.workunit.client.1.smithi169.stdout:7/550: mknod d7/d16/cb0 0 2024-09-16T00:26:05.836 INFO:tasks.workunit.client.1.smithi169.stdout:7/551: link d7/d71/d76/l5a d7/d5b/d61/d98/d99/lb1 0 2024-09-16T00:26:05.837 INFO:tasks.workunit.client.1.smithi169.stdout:7/552: fdatasync d7/d5b/f87 0 2024-09-16T00:26:05.839 INFO:tasks.workunit.client.1.smithi169.stdout:7/553: mkdir d7/d16/d49/db2 0 2024-09-16T00:26:05.839 INFO:tasks.workunit.client.1.smithi169.stdout:7/554: readlink d7/l5f 0 2024-09-16T00:26:05.849 INFO:tasks.workunit.client.1.smithi169.stdout:7/555: truncate f1 2055745 0 2024-09-16T00:26:05.851 INFO:tasks.workunit.client.1.smithi169.stdout:7/556: rename d7/d71/d76/f36 to d7/d16/d49/d74/fb3 0 2024-09-16T00:26:05.852 INFO:tasks.workunit.client.1.smithi169.stdout:7/557: mkdir d7/d16/d49/db2/db4 0 2024-09-16T00:26:05.852 INFO:tasks.workunit.client.1.smithi169.stdout:1/500: sync 2024-09-16T00:26:05.853 INFO:tasks.workunit.client.1.smithi169.stdout:1/501: write dc/d36/d3a/d45/f61 [124720,83438] 0 2024-09-16T00:26:05.856 INFO:tasks.workunit.client.1.smithi169.stdout:1/502: fdatasync dc/d36/d3a/d45/f61 0 2024-09-16T00:26:05.861 INFO:tasks.workunit.client.1.smithi169.stdout:1/503: creat dc/d18/d67/f94 x:0 0 0 2024-09-16T00:26:05.861 INFO:tasks.workunit.client.1.smithi169.stdout:1/504: chown dc/d36/d3a/d45/d29/f52 1 1 2024-09-16T00:26:05.864 INFO:tasks.workunit.client.1.smithi169.stdout:7/558: dwrite d7/d16/d49/f72 [0,4194304] 0 2024-09-16T00:26:05.864 INFO:tasks.workunit.client.1.smithi169.stdout:1/505: dread dc/d2c/d3d/f3f [0,4194304] 0 2024-09-16T00:26:05.865 INFO:tasks.workunit.client.1.smithi169.stdout:7/559: creat d7/d5b/fb5 x:0 0 0 2024-09-16T00:26:05.867 INFO:tasks.workunit.client.1.smithi169.stdout:7/560: fdatasync d7/df/d73/f8c 0 2024-09-16T00:26:05.868 INFO:tasks.workunit.client.1.smithi169.stdout:7/561: stat d7/df/d95/f3c 0 2024-09-16T00:26:05.869 INFO:tasks.workunit.client.1.smithi169.stdout:7/562: truncate d7/d71/d76/d9c/fa7 628179 0 2024-09-16T00:26:05.872 INFO:tasks.workunit.client.1.smithi169.stdout:7/563: mknod d7/d16/d49/d74/cb6 0 2024-09-16T00:26:05.880 INFO:tasks.workunit.client.0.smithi112.stdout:5/584: write d4/dd/d12/f2f [224411,85078] 0 2024-09-16T00:26:05.880 INFO:tasks.workunit.client.0.smithi112.stdout:5/585: read - d4/d7/d26/f81 zero size 2024-09-16T00:26:05.882 INFO:tasks.workunit.client.0.smithi112.stdout:5/586: truncate d4/dd/d12/f11 4563737 0 2024-09-16T00:26:05.889 INFO:tasks.workunit.client.0.smithi112.stdout:5/587: dwrite d4/d7/d14/f1a [0,4194304] 0 2024-09-16T00:26:05.892 INFO:tasks.workunit.client.0.smithi112.stdout:5/588: symlink d4/d7/d87/lc2 0 2024-09-16T00:26:05.892 INFO:tasks.workunit.client.0.smithi112.stdout:5/589: stat d4/d1c/fad 0 2024-09-16T00:26:05.899 INFO:tasks.workunit.client.0.smithi112.stdout:5/590: dwrite d4/d1c/d43/f6d [0,4194304] 0 2024-09-16T00:26:05.903 INFO:tasks.workunit.client.0.smithi112.stdout:5/591: read d4/dd/d6f/f72 [7311448,108358] 0 2024-09-16T00:26:05.908 INFO:tasks.workunit.client.0.smithi112.stdout:5/592: write d4/d1c/f73 [387002,108177] 0 2024-09-16T00:26:05.911 INFO:tasks.workunit.client.0.smithi112.stdout:5/593: fsync d4/dd/d12/d52/f63 0 2024-09-16T00:26:05.914 INFO:tasks.workunit.client.0.smithi112.stdout:5/594: rmdir d4/dd/db7 0 2024-09-16T00:26:05.916 INFO:tasks.workunit.client.0.smithi112.stdout:5/595: fsync d4/dd/f24 0 2024-09-16T00:26:05.920 INFO:tasks.workunit.client.0.smithi112.stdout:5/596: rmdir d4/d7/d14/d90/d9a/d7c 39 2024-09-16T00:26:05.925 INFO:tasks.workunit.client.0.smithi112.stdout:5/597: dwrite d4/d7/d14/f1a [0,4194304] 0 2024-09-16T00:26:05.925 INFO:tasks.workunit.client.0.smithi112.stdout:1/736: sync 2024-09-16T00:26:05.925 INFO:tasks.workunit.client.0.smithi112.stdout:8/618: sync 2024-09-16T00:26:05.927 INFO:tasks.workunit.client.0.smithi112.stdout:8/619: readlink d10/d4e/d8c/d7c/d97/laf 0 2024-09-16T00:26:05.928 INFO:tasks.workunit.client.0.smithi112.stdout:8/620: fsync d10/d12/d1c/d96/fac 0 2024-09-16T00:26:05.929 INFO:tasks.workunit.client.0.smithi112.stdout:5/598: write d4/d7/d14/d90/d9a/d7c/f88 [1091811,28673] 0 2024-09-16T00:26:05.934 INFO:tasks.workunit.client.0.smithi112.stdout:5/599: rename d4/dd/d49/da9/l8e to d4/d7/d14/d90/lc3 0 2024-09-16T00:26:05.935 INFO:tasks.workunit.client.0.smithi112.stdout:5/600: chown d4/d4f/d77/f7f 1478 1 2024-09-16T00:26:05.936 INFO:tasks.workunit.client.0.smithi112.stdout:5/601: write d4/f16 [1505854,79538] 0 2024-09-16T00:26:05.937 INFO:tasks.workunit.client.0.smithi112.stdout:5/602: readlink d4/d1c/l5d 0 2024-09-16T00:26:05.937 INFO:tasks.workunit.client.0.smithi112.stdout:5/603: fsync d4/d4f/d77/f7f 0 2024-09-16T00:26:05.944 INFO:tasks.workunit.client.0.smithi112.stdout:1/737: dread d0/d14/df/d28/d45/dd4/d47/d85/fd2 [0,4194304] 0 2024-09-16T00:26:05.948 INFO:tasks.workunit.client.0.smithi112.stdout:5/604: dwrite d4/d7/d26/d36/f41 [0,4194304] 0 2024-09-16T00:26:05.954 INFO:tasks.workunit.client.0.smithi112.stdout:1/738: dwrite d0/d14/df/d28/d45/d46/da7/fbc [0,4194304] 0 2024-09-16T00:26:05.955 INFO:tasks.workunit.client.0.smithi112.stdout:1/739: fsync d0/d14/df/f36 0 2024-09-16T00:26:05.957 INFO:tasks.workunit.client.1.smithi169.stdout:1/506: write dc/d18/f59 [1062216,101703] 0 2024-09-16T00:26:05.958 INFO:tasks.workunit.client.1.smithi169.stdout:1/507: fsync dc/d36/d3a/d45/f61 0 2024-09-16T00:26:05.960 INFO:tasks.workunit.client.1.smithi169.stdout:7/564: getdents d7/d71/d76 0 2024-09-16T00:26:05.960 INFO:tasks.workunit.client.0.smithi112.stdout:8/621: dread d10/d12/d1c/f59 [8388608,4194304] 0 2024-09-16T00:26:05.960 INFO:tasks.workunit.client.1.smithi169.stdout:7/565: stat d7/df 0 2024-09-16T00:26:05.969 INFO:tasks.workunit.client.1.smithi169.stdout:7/566: dwrite d7/d5b/d61/d98/d99/f9b [0,4194304] 0 2024-09-16T00:26:05.970 INFO:tasks.workunit.client.1.smithi169.stdout:7/567: write f5 [4080020,90513] 0 2024-09-16T00:26:05.973 INFO:tasks.workunit.client.1.smithi169.stdout:7/568: creat d7/d5b/d61/d98/fb7 x:0 0 0 2024-09-16T00:26:05.984 INFO:tasks.workunit.client.0.smithi112.stdout:8/622: dread d10/d12/d1c/d3b/d74/f90 [0,4194304] 0 2024-09-16T00:26:05.990 INFO:tasks.workunit.client.0.smithi112.stdout:8/623: rename d10/d4e to d10/d12/d1c/d61/d6c/dd1 0 2024-09-16T00:26:05.991 INFO:tasks.workunit.client.0.smithi112.stdout:8/624: creat d10/d12/d1c/d3b/fd2 x:0 0 0 2024-09-16T00:26:05.993 INFO:tasks.workunit.client.0.smithi112.stdout:8/625: chown d10/f15 523151 1 2024-09-16T00:26:05.993 INFO:tasks.workunit.client.0.smithi112.stdout:5/605: truncate d4/d7/d14/f8c 2239644 0 2024-09-16T00:26:05.994 INFO:tasks.workunit.client.0.smithi112.stdout:5/606: read - d4/d7/d14/fb0 zero size 2024-09-16T00:26:05.997 INFO:tasks.workunit.client.0.smithi112.stdout:5/607: rmdir d4/dd/d6f 39 2024-09-16T00:26:06.000 INFO:tasks.workunit.client.0.smithi112.stdout:1/740: dwrite d0/d14/df/d28/d45/dd4/f69 [0,4194304] 0 2024-09-16T00:26:06.004 INFO:tasks.workunit.client.1.smithi169.stdout:7/569: symlink d7/d16/d49/d84/d8e/lb8 0 2024-09-16T00:26:06.005 INFO:tasks.workunit.client.0.smithi112.stdout:8/626: dwrite d10/d12/f77 [0,4194304] 0 2024-09-16T00:26:06.006 INFO:tasks.workunit.client.0.smithi112.stdout:5/608: getdents d4/d7/d14/d90/d9a/d7c 0 2024-09-16T00:26:06.008 INFO:tasks.workunit.client.1.smithi169.stdout:7/570: rename d7/d16/f75 to d7/d16/d49/d74/fb9 0 2024-09-16T00:26:06.008 INFO:tasks.workunit.client.0.smithi112.stdout:5/609: chown d4/d7/d14/d90/d9a/d7c 63360 1 2024-09-16T00:26:06.008 INFO:tasks.workunit.client.0.smithi112.stdout:8/627: read - d10/d12/d1c/d61/d6c/dd1/d58/fcb zero size 2024-09-16T00:26:06.008 INFO:tasks.workunit.client.0.smithi112.stdout:3/703: sync 2024-09-16T00:26:06.008 INFO:tasks.workunit.client.0.smithi112.stdout:4/708: sync 2024-09-16T00:26:06.009 INFO:tasks.workunit.client.1.smithi169.stdout:7/571: write d7/d16/d49/f58 [4399757,8299] 0 2024-09-16T00:26:06.010 INFO:tasks.workunit.client.0.smithi112.stdout:5/610: symlink d4/dd/d49/d70/lc4 0 2024-09-16T00:26:06.011 INFO:tasks.workunit.client.0.smithi112.stdout:1/741: write d0/f9 [3165103,15534] 0 2024-09-16T00:26:06.015 INFO:tasks.workunit.client.0.smithi112.stdout:5/611: mkdir d4/dd/d49/d70/dc5 0 2024-09-16T00:26:06.016 INFO:tasks.workunit.client.1.smithi169.stdout:7/572: dwrite d7/d5b/f6b [0,4194304] 0 2024-09-16T00:26:06.016 INFO:tasks.workunit.client.0.smithi112.stdout:8/628: rmdir d10/d12/d1c/dd0/dc2 0 2024-09-16T00:26:06.017 INFO:tasks.workunit.client.1.smithi169.stdout:7/573: stat d7/d71/d76/c65 0 2024-09-16T00:26:06.018 INFO:tasks.workunit.client.0.smithi112.stdout:1/742: creat d0/d14/df/d28/d45/dd4/d47/d85/fef x:0 0 0 2024-09-16T00:26:06.018 INFO:tasks.workunit.client.0.smithi112.stdout:4/709: dwrite d6/d39/d1c/d6e/d6f/f97 [0,4194304] 0 2024-09-16T00:26:06.019 INFO:tasks.workunit.client.1.smithi169.stdout:7/574: mkdir d7/d71/d76/d29/d78/dba 0 2024-09-16T00:26:06.019 INFO:tasks.workunit.client.0.smithi112.stdout:8/629: chown d10/d12/d1c/d96/lae 0 1 2024-09-16T00:26:06.020 INFO:tasks.workunit.client.0.smithi112.stdout:5/612: creat d4/d7/d9b/fc6 x:0 0 0 2024-09-16T00:26:06.021 INFO:tasks.workunit.client.1.smithi169.stdout:7/575: chown d7/d71/d76/d29/f45 129404 1 2024-09-16T00:26:06.021 INFO:tasks.workunit.client.0.smithi112.stdout:3/704: getdents d2/d9/d2b 0 2024-09-16T00:26:06.023 INFO:tasks.workunit.client.0.smithi112.stdout:1/743: chown d0/d14/d1c/l2c 84815 1 2024-09-16T00:26:06.023 INFO:tasks.workunit.client.1.smithi169.stdout:7/576: symlink d7/d5b/d61/d98/d99/lbb 0 2024-09-16T00:26:06.025 INFO:tasks.workunit.client.0.smithi112.stdout:8/630: creat d10/d12/fd3 x:0 0 0 2024-09-16T00:26:06.026 INFO:tasks.workunit.client.1.smithi169.stdout:7/577: write d7/f8b [459786,100341] 0 2024-09-16T00:26:06.029 INFO:tasks.workunit.client.0.smithi112.stdout:3/705: write d2/d9/d2b/d47/d52/d7a/d80/d83/d76/d8e/fb6 [1910956,99536] 0 2024-09-16T00:26:06.029 INFO:tasks.workunit.client.0.smithi112.stdout:1/744: write d0/d14/df/d28/d45/d46/da7/fbc [4655555,82018] 0 2024-09-16T00:26:06.029 INFO:tasks.workunit.client.0.smithi112.stdout:5/613: rename d4/d7/d14/d90/d9a/fc1 to d4/d7/d14/d90/d9a/d7c/fc7 0 2024-09-16T00:26:06.031 INFO:tasks.workunit.client.0.smithi112.stdout:3/706: chown d2/d9/d2b/d47/d52/d7a/d80/d83/d68/d90/dcb/fd2 77 1 2024-09-16T00:26:06.036 INFO:tasks.workunit.client.0.smithi112.stdout:8/631: getdents d10/d12/d1c/d61/d6c/dd1/d8c/d98 0 2024-09-16T00:26:06.037 INFO:tasks.workunit.client.0.smithi112.stdout:1/745: dwrite d0/d14/d1c/f91 [0,4194304] 0 2024-09-16T00:26:06.038 INFO:tasks.workunit.client.0.smithi112.stdout:3/707: link d2/d9/d2b/f4b d2/d9/d2b/d47/d52/d7a/d80/d83/d76/d8e/fea 0 2024-09-16T00:26:06.039 INFO:tasks.workunit.client.0.smithi112.stdout:8/632: dread - d10/d12/d1c/d61/d6c/dd1/d8c/d7c/d97/fad zero size 2024-09-16T00:26:06.040 INFO:tasks.workunit.client.1.smithi169.stdout:7/578: getdents d7/d5b/d61/d98 0 2024-09-16T00:26:06.040 INFO:tasks.workunit.client.0.smithi112.stdout:8/633: chown d10/d12/d1c/d3b/d74 1 1 2024-09-16T00:26:06.044 INFO:tasks.workunit.client.1.smithi169.stdout:7/579: truncate d7/d16/f24 2319632 0 2024-09-16T00:26:06.046 INFO:tasks.workunit.client.0.smithi112.stdout:8/634: unlink d10/d12/d1c/l36 0 2024-09-16T00:26:06.047 INFO:tasks.workunit.client.0.smithi112.stdout:1/746: symlink d0/d14/df/d28/d45/dd4/lf0 0 2024-09-16T00:26:06.049 INFO:tasks.workunit.client.0.smithi112.stdout:4/710: dwrite d6/d39/d36/d5c/d7e/dcd/f7f [0,4194304] 0 2024-09-16T00:26:06.049 INFO:tasks.workunit.client.0.smithi112.stdout:3/708: chown d2/d9/d2b/d47/d52/d7a/d80/d83/d7b/dad/cbc 0 1 2024-09-16T00:26:06.050 INFO:tasks.workunit.client.0.smithi112.stdout:1/747: dread - d0/d14/fe1 zero size 2024-09-16T00:26:06.050 INFO:tasks.workunit.client.0.smithi112.stdout:8/635: dread - d10/d12/d1c/d61/d6c/dd1/d58/fa1 zero size 2024-09-16T00:26:06.052 INFO:tasks.workunit.client.0.smithi112.stdout:4/711: chown d6/db/d1f/d41/d49 454 1 2024-09-16T00:26:06.053 INFO:tasks.workunit.client.0.smithi112.stdout:3/709: chown d2/d9/ca2 15189521 1 2024-09-16T00:26:06.053 INFO:tasks.workunit.client.0.smithi112.stdout:1/748: symlink d0/d14/df/d28/d45/d46/lf1 0 2024-09-16T00:26:06.055 INFO:tasks.workunit.client.0.smithi112.stdout:1/749: chown d0/d14/d1c/d20/daf/cec 3 1 2024-09-16T00:26:06.056 INFO:tasks.workunit.client.0.smithi112.stdout:8/636: dread d10/d12/d1c/d61/fb4 [0,4194304] 0 2024-09-16T00:26:06.057 INFO:tasks.workunit.client.0.smithi112.stdout:4/712: creat d6/db/d1f/d41/ddf/f104 x:0 0 0 2024-09-16T00:26:06.058 INFO:tasks.workunit.client.0.smithi112.stdout:8/637: chown d10/d12/d1c/d61/d6c/dd1/f8e 64554693 1 2024-09-16T00:26:06.058 INFO:tasks.workunit.client.0.smithi112.stdout:4/713: chown d6/db/df/d5e/f78 7655954 1 2024-09-16T00:26:06.059 INFO:tasks.workunit.client.0.smithi112.stdout:3/710: symlink d2/d9/d2b/d47/d52/d7a/d80/d83/d68/d90/dcb/leb 0 2024-09-16T00:26:06.060 INFO:tasks.workunit.client.0.smithi112.stdout:1/750: symlink d0/d14/df/d28/lf2 0 2024-09-16T00:26:06.060 INFO:tasks.workunit.client.0.smithi112.stdout:4/714: readlink d6/db/df/d5e/d61/ld6 0 2024-09-16T00:26:06.061 INFO:tasks.workunit.client.1.smithi169.stdout:7/580: dread d7/d16/f79 [0,4194304] 0 2024-09-16T00:26:06.061 INFO:tasks.workunit.client.0.smithi112.stdout:1/751: readlink d0/d14/d1c/l2c 0 2024-09-16T00:26:06.062 INFO:tasks.workunit.client.0.smithi112.stdout:4/715: truncate d6/da/f35 1129214 0 2024-09-16T00:26:06.063 INFO:tasks.workunit.client.0.smithi112.stdout:1/752: dread - d0/d14/d1c/fcd zero size 2024-09-16T00:26:06.068 INFO:tasks.workunit.client.0.smithi112.stdout:1/753: write d0/d14/df/d28/d45/dd4/d41/fc0 [2255085,104962] 0 2024-09-16T00:26:06.071 INFO:tasks.workunit.client.0.smithi112.stdout:3/711: dwrite d2/d9/d2b/d2c/d69/f72 [0,4194304] 0 2024-09-16T00:26:06.071 INFO:tasks.workunit.client.0.smithi112.stdout:8/638: write d10/d12/d1c/d61/d6c/dd1/f6f [1719418,3006] 0 2024-09-16T00:26:06.072 INFO:tasks.workunit.client.0.smithi112.stdout:1/754: chown d0/d14/df/d28/d42/f81 57 1 2024-09-16T00:26:06.075 INFO:tasks.workunit.client.1.smithi169.stdout:7/581: dread d7/f9f [0,4194304] 0 2024-09-16T00:26:06.076 INFO:tasks.workunit.client.0.smithi112.stdout:3/712: fdatasync d2/d4d/fb3 0 2024-09-16T00:26:06.076 INFO:tasks.workunit.client.0.smithi112.stdout:1/755: chown d0/d14/df/d28/d45/d46/lf1 9 1 2024-09-16T00:26:06.076 INFO:tasks.workunit.client.0.smithi112.stdout:8/639: creat d10/d12/d1c/d61/d6c/fd4 x:0 0 0 2024-09-16T00:26:06.076 INFO:tasks.workunit.client.0.smithi112.stdout:8/640: chown d10/d12/d1c/d3b/d74 3 1 2024-09-16T00:26:06.078 INFO:tasks.workunit.client.1.smithi169.stdout:7/582: link d7/df/d95/l3d d7/d16/d49/d84/lbc 0 2024-09-16T00:26:06.080 INFO:tasks.workunit.client.1.smithi169.stdout:7/583: dread - d7/d16/d49/f81 zero size 2024-09-16T00:26:06.080 INFO:tasks.workunit.client.0.smithi112.stdout:3/713: link d2/d9/d2b/f82 d2/d4d/fec 0 2024-09-16T00:26:06.080 INFO:tasks.workunit.client.0.smithi112.stdout:3/714: chown d2 57798147 1 2024-09-16T00:26:06.081 INFO:tasks.workunit.client.0.smithi112.stdout:1/756: write d0/d14/d1c/d20/f65 [4890024,88916] 0 2024-09-16T00:26:06.083 INFO:tasks.workunit.client.0.smithi112.stdout:1/757: stat d0/d14/df/d28/d45/dd4/d47/d85/fef 0 2024-09-16T00:26:06.083 INFO:tasks.workunit.client.1.smithi169.stdout:7/584: dread d7/d16/f6a [0,4194304] 0 2024-09-16T00:26:06.084 INFO:tasks.workunit.client.0.smithi112.stdout:1/758: read - d0/d14/d1c/d20/daf/feb zero size 2024-09-16T00:26:06.090 INFO:tasks.workunit.client.1.smithi169.stdout:4/477: sync 2024-09-16T00:26:06.091 INFO:tasks.workunit.client.0.smithi112.stdout:3/715: dwrite d2/d9/d2b/d47/d52/d7a/d80/d83/d68/d90/fe7 [0,4194304] 0 2024-09-16T00:26:06.093 INFO:tasks.workunit.client.0.smithi112.stdout:1/759: dwrite d0/d14/df/f36 [0,4194304] 0 2024-09-16T00:26:06.094 INFO:tasks.workunit.client.0.smithi112.stdout:4/716: dread d6/db/df/d5e/d61/f3e [0,4194304] 0 2024-09-16T00:26:06.095 INFO:tasks.workunit.client.0.smithi112.stdout:1/760: write d0/f4e [1579065,78006] 0 2024-09-16T00:26:06.095 INFO:tasks.workunit.client.1.smithi169.stdout:4/478: link d2/d12/f2c d2/d12/d13/d5c/f9a 0 2024-09-16T00:26:06.096 INFO:tasks.workunit.client.1.smithi169.stdout:4/479: stat d2/d12/d13/f6c 0 2024-09-16T00:26:06.096 INFO:tasks.workunit.client.0.smithi112.stdout:3/716: unlink d2/d9/d2b/d2c/d69/f72 0 2024-09-16T00:26:06.100 INFO:tasks.workunit.client.0.smithi112.stdout:3/717: fdatasync d2/fb 0 2024-09-16T00:26:06.103 INFO:tasks.workunit.client.0.smithi112.stdout:3/718: stat d2/d9/d2b/d47/d52/d7a/d80/fb7 0 2024-09-16T00:26:06.105 INFO:tasks.workunit.client.1.smithi169.stdout:4/480: truncate d2/f49 316085 0 2024-09-16T00:26:06.105 INFO:tasks.workunit.client.0.smithi112.stdout:4/717: dwrite d6/d39/d1c/d66/f76 [0,4194304] 0 2024-09-16T00:26:06.106 INFO:tasks.workunit.client.0.smithi112.stdout:3/719: creat d2/d9/d2b/d47/d52/d7a/d80/d83/d76/fed x:0 0 0 2024-09-16T00:26:06.107 INFO:tasks.workunit.client.1.smithi169.stdout:4/481: mknod d2/d12/d13/c9b 0 2024-09-16T00:26:06.107 INFO:tasks.workunit.client.1.smithi169.stdout:4/482: truncate d2/f98 818071 0 2024-09-16T00:26:06.108 INFO:tasks.workunit.client.1.smithi169.stdout:4/483: dread - d2/d12/d13/d1f/f6a zero size 2024-09-16T00:26:06.112 INFO:tasks.workunit.client.0.smithi112.stdout:4/718: dread d6/d39/d1c/d6e/d6f/fe5 [0,4194304] 0 2024-09-16T00:26:06.113 INFO:tasks.workunit.client.0.smithi112.stdout:3/720: rename d2/d9/d2b/f8f to d2/d9/d2b/d47/d52/d7a/d80/d83/d49/fee 0 2024-09-16T00:26:06.116 INFO:tasks.workunit.client.1.smithi169.stdout:4/484: rmdir d2 39 2024-09-16T00:26:06.117 INFO:tasks.workunit.client.0.smithi112.stdout:1/761: dwrite d0/d14/df/d28/d45/dd4/f73 [0,4194304] 0 2024-09-16T00:26:06.119 INFO:tasks.workunit.client.1.smithi169.stdout:4/485: mkdir d2/d12/d9c 0 2024-09-16T00:26:06.120 INFO:tasks.workunit.client.0.smithi112.stdout:3/721: truncate d2/d4d/fb3 7999465 0 2024-09-16T00:26:06.123 INFO:tasks.workunit.client.0.smithi112.stdout:3/722: symlink d2/d9/d2b/lef 0 2024-09-16T00:26:06.124 INFO:tasks.workunit.client.0.smithi112.stdout:3/723: write d2/f1a [6210197,68242] 0 2024-09-16T00:26:06.125 INFO:tasks.workunit.client.0.smithi112.stdout:1/762: dwrite d0/d14/df/d28/d45/d46/da7/fbc [0,4194304] 0 2024-09-16T00:26:06.127 INFO:tasks.workunit.client.1.smithi169.stdout:4/486: dread d2/d12/d13/d1f/f52 [0,4194304] 0 2024-09-16T00:26:06.127 INFO:tasks.workunit.client.0.smithi112.stdout:3/724: truncate d2/d9/d2b/d2c/fce 1693964 0 2024-09-16T00:26:06.133 INFO:tasks.workunit.client.0.smithi112.stdout:3/725: rename d2/d9/d2b/d47/d78/fc6 to d2/d9/d2b/d47/d52/d7a/d80/d83/d76/d8e/ff0 0 2024-09-16T00:26:06.136 INFO:tasks.workunit.client.0.smithi112.stdout:3/726: fdatasync d2/d9/d2b/d47/d52/d7a/d80/d83/d76/d8e/fea 0 2024-09-16T00:26:06.139 INFO:tasks.workunit.client.0.smithi112.stdout:3/727: unlink d2/d9/d2b/d47/d52/d7a/d80/d83/d68/cab 0 2024-09-16T00:26:06.144 INFO:tasks.workunit.client.1.smithi169.stdout:4/487: dwrite d2/d12/f5d [0,4194304] 0 2024-09-16T00:26:06.145 INFO:tasks.workunit.client.0.smithi112.stdout:4/719: dread d6/db/d1f/d41/d49/f67 [0,4194304] 0 2024-09-16T00:26:06.146 INFO:tasks.workunit.client.0.smithi112.stdout:3/728: rename d2/d9/d2b/d47/d78/da9/fd0 to d2/d9/d2b/d2c/ff1 0 2024-09-16T00:26:06.148 INFO:tasks.workunit.client.0.smithi112.stdout:1/763: dwrite d0/d14/df/d28/d45/f9f [0,4194304] 0 2024-09-16T00:26:06.149 INFO:tasks.workunit.client.1.smithi169.stdout:4/488: getdents d2/d12/d13/d27/d33 0 2024-09-16T00:26:06.149 INFO:tasks.workunit.client.0.smithi112.stdout:9/883: sync 2024-09-16T00:26:06.149 INFO:tasks.workunit.client.0.smithi112.stdout:6/641: sync 2024-09-16T00:26:06.149 INFO:tasks.workunit.client.0.smithi112.stdout:2/797: sync 2024-09-16T00:26:06.151 INFO:tasks.workunit.client.0.smithi112.stdout:4/720: symlink d6/db/l105 0 2024-09-16T00:26:06.151 INFO:tasks.workunit.client.1.smithi169.stdout:4/489: creat d2/f9d x:0 0 0 2024-09-16T00:26:06.152 INFO:tasks.workunit.client.1.smithi169.stdout:4/490: stat d2/d12/d13/d4e 0 2024-09-16T00:26:06.152 INFO:tasks.workunit.client.0.smithi112.stdout:3/729: mknod d2/d9/d2b/d47/d52/d7a/d80/cf2 0 2024-09-16T00:26:06.153 INFO:tasks.workunit.client.0.smithi112.stdout:1/764: symlink d0/d14/df/d28/d42/dd8/lf3 0 2024-09-16T00:26:06.154 INFO:tasks.workunit.client.0.smithi112.stdout:1/765: truncate d0/fe5 979470 0 2024-09-16T00:26:06.159 INFO:tasks.workunit.client.1.smithi169.stdout:4/491: dwrite d2/d12/d13/d4e/f8d [0,4194304] 0 2024-09-16T00:26:06.159 INFO:tasks.workunit.client.0.smithi112.stdout:3/730: chown d2/d9/d2b/c65 492 1 2024-09-16T00:26:06.161 INFO:tasks.workunit.client.0.smithi112.stdout:2/798: fdatasync d2/dd/d48/d22/d2b/d54/d60/d6c/f7b 0 2024-09-16T00:26:06.163 INFO:tasks.workunit.client.1.smithi169.stdout:4/492: getdents d2/d12/d13/d1f/d2f 0 2024-09-16T00:26:06.164 INFO:tasks.workunit.client.0.smithi112.stdout:1/766: dwrite d0/d14/df/d28/d45/dd4/f8d [0,4194304] 0 2024-09-16T00:26:06.166 INFO:tasks.workunit.client.1.smithi169.stdout:4/493: fdatasync d2/d12/d13/d1f/d3e/d5b/d1a/f55 0 2024-09-16T00:26:06.167 INFO:tasks.workunit.client.0.smithi112.stdout:1/767: fsync d0/d14/df/d28/d45/dd4/d41/de0/fe2 0 2024-09-16T00:26:06.168 INFO:tasks.workunit.client.0.smithi112.stdout:2/799: mkdir d2/dd/d48/d94/d49/d100 0 2024-09-16T00:26:06.169 INFO:tasks.workunit.client.1.smithi169.stdout:4/494: rename d2/d12/d13/d27/d33 to d2/d12/d13/d1f/d3e/d5b/d1a/d9e 0 2024-09-16T00:26:06.171 INFO:tasks.workunit.client.0.smithi112.stdout:9/884: link d1/lb8 d1/d5/d20/l137 0 2024-09-16T00:26:06.172 INFO:tasks.workunit.client.1.smithi169.stdout:4/495: write d2/d12/f36 [2747843,5190] 0 2024-09-16T00:26:06.174 INFO:tasks.workunit.client.1.smithi169.stdout:4/496: rmdir d2/d12/d13/d4d/d59 39 2024-09-16T00:26:06.175 INFO:tasks.workunit.client.0.smithi112.stdout:2/800: mkdir d2/dd/d48/d8a/d101 0 2024-09-16T00:26:06.176 INFO:tasks.workunit.client.0.smithi112.stdout:9/885: getdents d1/d5/d20/d81/dc4 0 2024-09-16T00:26:06.177 INFO:tasks.workunit.client.1.smithi169.stdout:4/497: fdatasync d2/d12/f2c 0 2024-09-16T00:26:06.178 INFO:tasks.workunit.client.1.smithi169.stdout:4/498: write d2/d12/d13/d1f/d3e/f8c [744332,21704] 0 2024-09-16T00:26:06.179 INFO:tasks.workunit.client.0.smithi112.stdout:1/768: dwrite d0/d14/df/d28/d45/dd4/f69 [0,4194304] 0 2024-09-16T00:26:06.179 INFO:tasks.workunit.client.0.smithi112.stdout:2/801: creat d2/dd/d48/d94/d49/d100/f102 x:0 0 0 2024-09-16T00:26:06.180 INFO:tasks.workunit.client.1.smithi169.stdout:4/499: fsync d2/d12/d13/d1f/f6a 0 2024-09-16T00:26:06.180 INFO:tasks.workunit.client.1.smithi169.stdout:4/500: stat d2/d12/d13/d27 0 2024-09-16T00:26:06.181 INFO:tasks.workunit.client.1.smithi169.stdout:4/501: write d2/d12/f31 [1627025,115230] 0 2024-09-16T00:26:06.183 INFO:tasks.workunit.client.1.smithi169.stdout:8/472: sync 2024-09-16T00:26:06.183 INFO:tasks.workunit.client.1.smithi169.stdout:9/526: sync 2024-09-16T00:26:06.183 INFO:tasks.workunit.client.1.smithi169.stdout:0/495: sync 2024-09-16T00:26:06.184 INFO:tasks.workunit.client.1.smithi169.stdout:9/527: chown d5/c68 47085934 1 2024-09-16T00:26:06.186 INFO:tasks.workunit.client.0.smithi112.stdout:3/731: write d2/d4d/fb3 [6952201,122656] 0 2024-09-16T00:26:06.186 INFO:tasks.workunit.client.1.smithi169.stdout:0/496: mknod d0/cb3 0 2024-09-16T00:26:06.187 INFO:tasks.workunit.client.0.smithi112.stdout:1/769: read d0/d14/d44/f7e [661756,15036] 0 2024-09-16T00:26:06.188 INFO:tasks.workunit.client.1.smithi169.stdout:4/502: creat d2/d12/d13/d1f/d3e/d5b/f9f x:0 0 0 2024-09-16T00:26:06.188 INFO:tasks.workunit.client.0.smithi112.stdout:9/886: dread d1/d5/d20/f32 [0,4194304] 0 2024-09-16T00:26:06.188 INFO:tasks.workunit.client.0.smithi112.stdout:6/642: dwrite dd/df/d11/f80 [0,4194304] 0 2024-09-16T00:26:06.190 INFO:tasks.workunit.client.1.smithi169.stdout:4/503: chown d2/d12/d13/d1f/d3e/d5b/d1a/d9e/f71 17045 1 2024-09-16T00:26:06.190 INFO:tasks.workunit.client.1.smithi169.stdout:4/504: stat d2/d12/d13/d5c 0 2024-09-16T00:26:06.191 INFO:tasks.workunit.client.1.smithi169.stdout:9/528: creat d5/fb8 x:0 0 0 2024-09-16T00:26:06.192 INFO:tasks.workunit.client.1.smithi169.stdout:4/505: fdatasync d2/d12/d13/d4d/f84 0 2024-09-16T00:26:06.192 INFO:tasks.workunit.client.1.smithi169.stdout:8/473: rmdir d7/dc/d17/d24/d32/d8d/d8f 0 2024-09-16T00:26:06.193 INFO:tasks.workunit.client.1.smithi169.stdout:4/506: stat d2/d12/d13/d1f/d3e/d5b/f18 0 2024-09-16T00:26:06.194 INFO:tasks.workunit.client.1.smithi169.stdout:9/529: getdents d5/d49 0 2024-09-16T00:26:06.196 INFO:tasks.workunit.client.1.smithi169.stdout:8/474: rename d7/dc/d26/c4f to d7/d20/c9a 0 2024-09-16T00:26:06.197 INFO:tasks.workunit.client.0.smithi112.stdout:4/721: dwrite d6/db/df/d5e/d61/d64/f81 [0,4194304] 0 2024-09-16T00:26:06.197 INFO:tasks.workunit.client.1.smithi169.stdout:8/475: chown d7/dc/d26/c98 37 1 2024-09-16T00:26:06.198 INFO:tasks.workunit.client.0.smithi112.stdout:3/732: rmdir d2/d9/d2b/d47/d52/d7a/d80/d83/d49 39 2024-09-16T00:26:06.198 INFO:tasks.workunit.client.0.smithi112.stdout:9/887: read d1/d5/d20/d81/d1a/d88/def/f2e [330366,11239] 0 2024-09-16T00:26:06.198 INFO:tasks.workunit.client.0.smithi112.stdout:2/802: dwrite d2/dd/d48/f61 [0,4194304] 0 2024-09-16T00:26:06.199 INFO:tasks.workunit.client.0.smithi112.stdout:1/770: read d0/d14/df/d28/d45/d30/f4a [5177972,8528] 0 2024-09-16T00:26:06.199 INFO:tasks.workunit.client.1.smithi169.stdout:4/507: creat d2/d12/d13/d4d/d59/fa0 x:0 0 0 2024-09-16T00:26:06.200 INFO:tasks.workunit.client.1.smithi169.stdout:9/530: link d5/d49/d64/fb0 d5/da/d4e/fb9 0 2024-09-16T00:26:06.200 INFO:tasks.workunit.client.1.smithi169.stdout:0/497: write d0/d2/d4b/d86/f40 [3788857,8127] 0 2024-09-16T00:26:06.201 INFO:tasks.workunit.client.0.smithi112.stdout:6/643: getdents dd/df/d10/d47/d55/dbd/dcb 0 2024-09-16T00:26:06.201 INFO:tasks.workunit.client.1.smithi169.stdout:9/531: readlink d5/da/d4e/l4f 0 2024-09-16T00:26:06.202 INFO:tasks.workunit.client.1.smithi169.stdout:8/476: creat d7/d37/f9b x:0 0 0 2024-09-16T00:26:06.203 INFO:tasks.workunit.client.1.smithi169.stdout:0/498: rmdir d0/d2/d4b 39 2024-09-16T00:26:06.205 INFO:tasks.workunit.client.0.smithi112.stdout:9/888: creat d1/d5/d2c/d6f/f138 x:0 0 0 2024-09-16T00:26:06.205 INFO:tasks.workunit.client.1.smithi169.stdout:9/532: write d5/da/d5b/d6c/da0/f95 [276147,121508] 0 2024-09-16T00:26:06.206 INFO:tasks.workunit.client.1.smithi169.stdout:4/508: creat d2/d12/d13/d1f/d3e/fa1 x:0 0 0 2024-09-16T00:26:06.207 INFO:tasks.workunit.client.0.smithi112.stdout:6/644: fsync dd/df/d11/f49 0 2024-09-16T00:26:06.210 INFO:tasks.workunit.client.1.smithi169.stdout:9/533: creat d5/da/d5b/d91/d9a/fba x:0 0 0 2024-09-16T00:26:06.211 INFO:tasks.workunit.client.0.smithi112.stdout:4/722: rename d6/db/f42 to d6/db/d1f/d94/f106 0 2024-09-16T00:26:06.212 INFO:tasks.workunit.client.1.smithi169.stdout:9/534: chown d5/da/d73/lb2 27663 1 2024-09-16T00:26:06.213 INFO:tasks.workunit.client.0.smithi112.stdout:1/771: creat d0/d14/df/d28/d45/d46/d58/d62/ff4 x:0 0 0 2024-09-16T00:26:06.214 INFO:tasks.workunit.client.1.smithi169.stdout:9/535: symlink d5/da/d4e/lbb 0 2024-09-16T00:26:06.214 INFO:tasks.workunit.client.0.smithi112.stdout:6/645: unlink dd/d15/da5/fb0 0 2024-09-16T00:26:06.214 INFO:tasks.workunit.client.1.smithi169.stdout:4/509: link d2/d12/d13/d4e/c67 d2/d12/d13/d27/ca2 0 2024-09-16T00:26:06.215 INFO:tasks.workunit.client.0.smithi112.stdout:4/723: read - d6/db/df/d5e/d61/fa6 zero size 2024-09-16T00:26:06.216 INFO:tasks.workunit.client.0.smithi112.stdout:2/803: dwrite d2/dd/d48/f61 [4194304,4194304] 0 2024-09-16T00:26:06.217 INFO:tasks.workunit.client.0.smithi112.stdout:3/733: dwrite d2/d9/d2b/d47/d52/d7a/d80/d83/f50 [0,4194304] 0 2024-09-16T00:26:06.217 INFO:tasks.workunit.client.1.smithi169.stdout:9/536: truncate d5/da/d2e/d47/f7c 379349 0 2024-09-16T00:26:06.217 INFO:tasks.workunit.client.0.smithi112.stdout:9/889: truncate d1/d5/d20/d81/d1a/d88/da8/f134 501325 0 2024-09-16T00:26:06.222 INFO:tasks.workunit.client.0.smithi112.stdout:3/734: chown d2/d9/d2b/d47/d52/d7a/d80/d83/d68/f88 280427 1 2024-09-16T00:26:06.222 INFO:tasks.workunit.client.1.smithi169.stdout:4/510: dwrite d2/d12/d13/d1f/f87 [0,4194304] 0 2024-09-16T00:26:06.223 INFO:tasks.workunit.client.0.smithi112.stdout:4/724: rmdir d6/db/df/d5e/d61/d64/d6a 39 2024-09-16T00:26:06.224 INFO:tasks.workunit.client.0.smithi112.stdout:2/804: creat d2/dd/d48/d8a/d9b/f103 x:0 0 0 2024-09-16T00:26:06.225 INFO:tasks.workunit.client.0.smithi112.stdout:9/890: truncate d1/d5/d2c/f102 371594 0 2024-09-16T00:26:06.226 INFO:tasks.workunit.client.1.smithi169.stdout:9/537: creat d5/da/d21/d8d/db4/fbc x:0 0 0 2024-09-16T00:26:06.228 INFO:tasks.workunit.client.1.smithi169.stdout:4/511: creat d2/d12/d13/d5c/fa3 x:0 0 0 2024-09-16T00:26:06.230 INFO:tasks.workunit.client.0.smithi112.stdout:4/725: write d6/d39/d36/d5c/ffa [4336830,82334] 0 2024-09-16T00:26:06.231 INFO:tasks.workunit.client.0.smithi112.stdout:6/646: link dd/df/d13/d18/l5d dd/df/d10/d47/d55/dc4/dd1/ld4 0 2024-09-16T00:26:06.231 INFO:tasks.workunit.client.1.smithi169.stdout:9/538: write d5/da/d21/d23/fa9 [4353895,67408] 0 2024-09-16T00:26:06.232 INFO:tasks.workunit.client.1.smithi169.stdout:8/477: getdents d7/d37 0 2024-09-16T00:26:06.233 INFO:tasks.workunit.client.0.smithi112.stdout:1/772: dwrite d0/d14/d1c/d20/fa9 [4194304,4194304] 0 2024-09-16T00:26:06.233 INFO:tasks.workunit.client.0.smithi112.stdout:3/735: chown d2/d9/d2b/d47/d52/d7a/d80/d83/d68/fc5 238837 1 2024-09-16T00:26:06.234 INFO:tasks.workunit.client.0.smithi112.stdout:2/805: fsync d2/dd/d48/d22/d2b/d54/d60/d6c/d88/da4/fd0 0 2024-09-16T00:26:06.236 INFO:tasks.workunit.client.1.smithi169.stdout:9/539: read d5/f4a [1531613,67218] 0 2024-09-16T00:26:06.236 INFO:tasks.workunit.client.0.smithi112.stdout:4/726: fsync d6/db/d1f/d41/d49/db1/ff6 0 2024-09-16T00:26:06.238 INFO:tasks.workunit.client.0.smithi112.stdout:3/736: chown d2/d9/d2b/d47/d52/d7a/d80/d83/fb5 14984 1 2024-09-16T00:26:06.239 INFO:tasks.workunit.client.1.smithi169.stdout:9/540: write d5/da/d2e/f62 [2124549,127484] 0 2024-09-16T00:26:06.240 INFO:tasks.workunit.client.1.smithi169.stdout:8/478: symlink d7/dc/d17/d24/l9c 0 2024-09-16T00:26:06.241 INFO:tasks.workunit.client.0.smithi112.stdout:2/806: fsync d2/dd/d48/d22/d2b/d54/d60/daf/fbc 0 2024-09-16T00:26:06.242 INFO:tasks.workunit.client.1.smithi169.stdout:0/499: dwrite d0/d2/f5f [0,4194304] 0 2024-09-16T00:26:06.242 INFO:tasks.workunit.client.1.smithi169.stdout:8/479: chown d7/dc/d2a/l50 1588018942 1 2024-09-16T00:26:06.243 INFO:tasks.workunit.client.0.smithi112.stdout:3/737: write d2/d9/f16 [1066383,109396] 0 2024-09-16T00:26:06.243 INFO:tasks.workunit.client.0.smithi112.stdout:6/647: dread dd/df/d17/f41 [0,4194304] 0 2024-09-16T00:26:06.244 INFO:tasks.workunit.client.1.smithi169.stdout:0/500: chown d0/d50/d6c/l78 0 1 2024-09-16T00:26:06.245 INFO:tasks.workunit.client.1.smithi169.stdout:9/541: creat d5/da/d5b/d91/fbd x:0 0 0 2024-09-16T00:26:06.245 INFO:tasks.workunit.client.1.smithi169.stdout:0/501: stat d0/d2/d8/f48 0 2024-09-16T00:26:06.247 INFO:tasks.workunit.client.0.smithi112.stdout:6/648: chown dd/df/d13/c53 499 1 2024-09-16T00:26:06.248 INFO:tasks.workunit.client.1.smithi169.stdout:8/480: creat d7/dc/d26/f9d x:0 0 0 2024-09-16T00:26:06.249 INFO:tasks.workunit.client.0.smithi112.stdout:3/738: mkdir d2/d9/d2b/d47/d52/d7a/df3 0 2024-09-16T00:26:06.250 INFO:tasks.workunit.client.0.smithi112.stdout:2/807: symlink d2/dd/d48/d22/d2b/d54/d60/d6c/d88/dad/deb/de1/l104 0 2024-09-16T00:26:06.250 INFO:tasks.workunit.client.1.smithi169.stdout:9/542: mknod d5/da/d4e/cbe 0 2024-09-16T00:26:06.251 INFO:tasks.workunit.client.1.smithi169.stdout:8/481: unlink d7/dc/d1d/l99 0 2024-09-16T00:26:06.253 INFO:tasks.workunit.client.1.smithi169.stdout:9/543: dread - d5/d12/f96 zero size 2024-09-16T00:26:06.254 INFO:tasks.workunit.client.1.smithi169.stdout:8/482: unlink d7/dc/d26/l3a 0 2024-09-16T00:26:06.254 INFO:tasks.workunit.client.0.smithi112.stdout:3/739: dread - d2/d9/d2b/d47/d52/d7a/d80/d83/d68/d90/dcb/fd2 zero size 2024-09-16T00:26:06.255 INFO:tasks.workunit.client.1.smithi169.stdout:9/544: creat d5/d49/fbf x:0 0 0 2024-09-16T00:26:06.255 INFO:tasks.workunit.client.0.smithi112.stdout:2/808: mkdir d2/dd/d48/d94/d49/d100/d105 0 2024-09-16T00:26:06.256 INFO:tasks.workunit.client.0.smithi112.stdout:6/649: dread dd/df/d10/d47/d55/f7a [4194304,4194304] 0 2024-09-16T00:26:06.257 INFO:tasks.workunit.client.1.smithi169.stdout:8/483: mkdir d7/dc/d26/d41/d4e/d81/d9e 0 2024-09-16T00:26:06.257 INFO:tasks.workunit.client.0.smithi112.stdout:2/809: write d2/dd/f1a [1857640,40645] 0 2024-09-16T00:26:06.258 INFO:tasks.workunit.client.0.smithi112.stdout:2/810: write f1 [2394049,29515] 0 2024-09-16T00:26:06.259 INFO:tasks.workunit.client.0.smithi112.stdout:2/811: dread - d2/dd/d48/d94/d49/fca zero size 2024-09-16T00:26:06.260 INFO:tasks.workunit.client.1.smithi169.stdout:8/484: mkdir d7/dc/d17/d24/d32/d8d/d9f 0 2024-09-16T00:26:06.261 INFO:tasks.workunit.client.0.smithi112.stdout:1/773: write d0/d14/d1c/d20/f33 [256596,3432] 0 2024-09-16T00:26:06.261 INFO:tasks.workunit.client.0.smithi112.stdout:3/740: rmdir d2/d9/d2b/d47/d52/d7a/d80/d95 39 2024-09-16T00:26:06.262 INFO:tasks.workunit.client.1.smithi169.stdout:8/485: mkdir d7/dc/d53/da0 0 2024-09-16T00:26:06.262 INFO:tasks.workunit.client.0.smithi112.stdout:5/614: sync 2024-09-16T00:26:06.262 INFO:tasks.workunit.client.0.smithi112.stdout:0/779: sync 2024-09-16T00:26:06.262 INFO:tasks.workunit.client.0.smithi112.stdout:7/698: sync 2024-09-16T00:26:06.267 INFO:tasks.workunit.client.0.smithi112.stdout:9/891: dwrite d1/d5/d2c/d6f/dbf/fee [4194304,4194304] 0 2024-09-16T00:26:06.269 INFO:tasks.workunit.client.0.smithi112.stdout:4/727: truncate d6/d39/d36/d5c/f63 3985460 0 2024-09-16T00:26:06.270 INFO:tasks.workunit.client.1.smithi169.stdout:8/486: dwrite d7/dc/d17/f89 [0,4194304] 0 2024-09-16T00:26:06.271 INFO:tasks.workunit.client.0.smithi112.stdout:5/615: chown d4/d7/d14/d90/d9a/d7c/fa3 426879500 1 2024-09-16T00:26:06.271 INFO:tasks.workunit.client.1.smithi169.stdout:8/487: stat d7/dc/d26/c98 0 2024-09-16T00:26:06.271 INFO:tasks.workunit.client.1.smithi169.stdout:9/545: write d5/da/ff [2697997,74864] 0 2024-09-16T00:26:06.274 INFO:tasks.workunit.client.1.smithi169.stdout:8/488: symlink d7/dc/d26/d41/d4e/la1 0 2024-09-16T00:26:06.275 INFO:tasks.workunit.client.0.smithi112.stdout:9/892: readlink d1/d5/d20/l58 0 2024-09-16T00:26:06.275 INFO:tasks.workunit.client.0.smithi112.stdout:7/699: creat d6/d10/d16/fe9 x:0 0 0 2024-09-16T00:26:06.278 INFO:tasks.workunit.client.1.smithi169.stdout:8/489: write d7/dc/d17/f89 [979445,7878] 0 2024-09-16T00:26:06.282 INFO:tasks.workunit.client.1.smithi169.stdout:0/502: dread d0/d2c/d54/f28 [0,4194304] 0 2024-09-16T00:26:06.282 INFO:tasks.workunit.client.1.smithi169.stdout:9/546: dwrite d5/da/d2e/d47/f7d [0,4194304] 0 2024-09-16T00:26:06.282 INFO:tasks.workunit.client.0.smithi112.stdout:0/780: dread d5/da/d18/d24/d46/d5f/d7a/fa2 [0,4194304] 0 2024-09-16T00:26:06.282 INFO:tasks.workunit.client.0.smithi112.stdout:2/812: dwrite d2/dd/f12 [0,4194304] 0 2024-09-16T00:26:06.283 INFO:tasks.workunit.client.0.smithi112.stdout:6/650: link dd/df/d10/d47/d55/dc4/dd1/d88/l95 dd/df/d10/d47/d55/ld5 0 2024-09-16T00:26:06.285 INFO:tasks.workunit.client.0.smithi112.stdout:1/774: rename d0/d14/df/d28/d45/dd4/f73 to d0/d14/df/d28/d45/dd4/d41/de0/ff5 0 2024-09-16T00:26:06.287 INFO:tasks.workunit.client.1.smithi169.stdout:8/490: getdents d7/dc/d26/d41/d92 0 2024-09-16T00:26:06.288 INFO:tasks.workunit.client.0.smithi112.stdout:4/728: symlink d6/d39/d36/d5c/d7e/dd1/l107 0 2024-09-16T00:26:06.288 INFO:tasks.workunit.client.0.smithi112.stdout:3/741: truncate d2/d9/d2b/d47/d52/d7a/d80/d83/d76/d8e/fea 1233070 0 2024-09-16T00:26:06.289 INFO:tasks.workunit.client.1.smithi169.stdout:9/547: mkdir d5/d49/dc0 0 2024-09-16T00:26:06.291 INFO:tasks.workunit.client.1.smithi169.stdout:8/491: mknod d7/dc/d1d/ca2 0 2024-09-16T00:26:06.291 INFO:tasks.workunit.client.0.smithi112.stdout:0/781: mknod d5/da/d21/dc3/ceb 0 2024-09-16T00:26:06.292 INFO:tasks.workunit.client.1.smithi169.stdout:0/503: dwrite d0/d2/d8/f68 [4194304,4194304] 0 2024-09-16T00:26:06.293 INFO:tasks.workunit.client.0.smithi112.stdout:7/700: getdents d6/d10/d16/db4/de4 0 2024-09-16T00:26:06.294 INFO:tasks.workunit.client.1.smithi169.stdout:9/548: rmdir d5/da/d21/d8d 39 2024-09-16T00:26:06.296 INFO:tasks.workunit.client.0.smithi112.stdout:6/651: mkdir dd/df/d10/d47/d89/dbc/dd6 0 2024-09-16T00:26:06.296 INFO:tasks.workunit.client.0.smithi112.stdout:8/641: sync 2024-09-16T00:26:06.296 INFO:tasks.workunit.client.0.smithi112.stdout:7/701: write d6/fae [2168832,69616] 0 2024-09-16T00:26:06.298 INFO:tasks.workunit.client.0.smithi112.stdout:9/893: rename d1/d5/d6/d18/ca0 to d1/d5/c139 0 2024-09-16T00:26:06.299 INFO:tasks.workunit.client.0.smithi112.stdout:2/813: link d2/dd/d48/d22/d67/la3 d2/dd/d48/d22/d2b/d54/l106 0 2024-09-16T00:26:06.302 INFO:tasks.workunit.client.1.smithi169.stdout:0/504: rename d0/d2/ca9 to d0/d2/cb4 0 2024-09-16T00:26:06.302 INFO:tasks.workunit.client.0.smithi112.stdout:9/894: readlink d1/d5/d6/d2d/d6a/d89/l7a 0 2024-09-16T00:26:06.304 INFO:tasks.workunit.client.1.smithi169.stdout:0/505: dread - d0/d2c/d4f/da5/fa6 zero size 2024-09-16T00:26:06.304 INFO:tasks.workunit.client.1.smithi169.stdout:9/549: mkdir d5/da/d21/d8d/da4/dc1 0 2024-09-16T00:26:06.304 INFO:tasks.workunit.client.1.smithi169.stdout:8/492: dwrite d7/dc/d17/d48/f60 [4194304,4194304] 0 2024-09-16T00:26:06.305 INFO:tasks.workunit.client.0.smithi112.stdout:4/729: dwrite d6/d39/d1c/d66/fac [0,4194304] 0 2024-09-16T00:26:06.305 INFO:tasks.workunit.client.0.smithi112.stdout:0/782: rename d5/da/d18/d89/ldc to d5/da/d21/dc3/lec 0 2024-09-16T00:26:06.307 INFO:tasks.workunit.client.0.smithi112.stdout:4/730: fsync d6/d39/d36/d5c/fc0 0 2024-09-16T00:26:06.308 INFO:tasks.workunit.client.0.smithi112.stdout:8/642: mknod d10/d12/d1c/d61/d6c/dd1/d8c/d98/cd5 0 2024-09-16T00:26:06.308 INFO:tasks.workunit.client.0.smithi112.stdout:0/783: fsync d5/da/d1d/d71/d64/f8b 0 2024-09-16T00:26:06.308 INFO:tasks.workunit.client.1.smithi169.stdout:8/493: symlink d7/dc/d17/d24/d32/d8d/la3 0 2024-09-16T00:26:06.309 INFO:tasks.workunit.client.0.smithi112.stdout:3/742: link d2/d9/d2b/d47/d52/d7a/d80/d83/d68/ld9 d2/d9/d2b/d47/d52/d7a/d80/d83/d68/lf4 0 2024-09-16T00:26:06.310 INFO:tasks.workunit.client.1.smithi169.stdout:8/494: chown d7/d20/f7e 7607 1 2024-09-16T00:26:06.310 INFO:tasks.workunit.client.1.smithi169.stdout:0/506: rmdir d0/d2c/d54/d21/d31/da4 0 2024-09-16T00:26:06.312 INFO:tasks.workunit.client.1.smithi169.stdout:0/507: dread - d0/d2c/d54/d21/d31/f9e zero size 2024-09-16T00:26:06.315 INFO:tasks.workunit.client.0.smithi112.stdout:0/784: truncate d5/f95 487259 0 2024-09-16T00:26:06.315 INFO:tasks.workunit.client.0.smithi112.stdout:9/895: chown d1/d5/d6/d18/f1f 416082763 1 2024-09-16T00:26:06.315 INFO:tasks.workunit.client.1.smithi169.stdout:9/550: dread d5/da/f16 [0,4194304] 0 2024-09-16T00:26:06.316 INFO:tasks.workunit.client.0.smithi112.stdout:7/702: rename d6/d10/d16/d96/dc7/dd2 to d6/d10/d75/dea 0 2024-09-16T00:26:06.318 INFO:tasks.workunit.client.0.smithi112.stdout:2/814: dwrite d2/dd/d48/d22/d2b/d54/d60/d6c/d88/dad/deb/fab [0,4194304] 0 2024-09-16T00:26:06.318 INFO:tasks.workunit.client.1.smithi169.stdout:9/551: rename d5/c68 to d5/da/d2e/d47/cc2 0 2024-09-16T00:26:06.320 INFO:tasks.workunit.client.0.smithi112.stdout:8/643: creat d10/d12/d1c/d96/fd6 x:0 0 0 2024-09-16T00:26:06.321 INFO:tasks.workunit.client.1.smithi169.stdout:9/552: mkdir d5/da/d5b/d91/d9a/dc3 0 2024-09-16T00:26:06.321 INFO:tasks.workunit.client.1.smithi169.stdout:9/553: stat d5/da/d4e/l82 0 2024-09-16T00:26:06.323 INFO:tasks.workunit.client.0.smithi112.stdout:8/644: readlink d10/d12/d1c/d61/d6c/dd1/d58/lc1 0 2024-09-16T00:26:06.323 INFO:tasks.workunit.client.0.smithi112.stdout:9/896: unlink d1/d5/d6/df8/ddd/f12c 0 2024-09-16T00:26:06.323 INFO:tasks.workunit.client.0.smithi112.stdout:8/645: chown d10/d12/d1c 7 1 2024-09-16T00:26:06.323 INFO:tasks.workunit.client.0.smithi112.stdout:7/703: chown d6/f7b 46 1 2024-09-16T00:26:06.324 INFO:tasks.workunit.client.1.smithi169.stdout:9/554: creat d5/da/d21/d8d/db4/fc4 x:0 0 0 2024-09-16T00:26:06.324 INFO:tasks.workunit.client.0.smithi112.stdout:4/731: mknod d6/d39/d1c/d2c/d69/d7b/de0/c108 0 2024-09-16T00:26:06.325 INFO:tasks.workunit.client.0.smithi112.stdout:8/646: write d10/d12/d1c/d61/fa2 [381154,64305] 0 2024-09-16T00:26:06.326 INFO:tasks.workunit.client.1.smithi169.stdout:9/555: creat d5/da/d5b/d6c/da0/fc5 x:0 0 0 2024-09-16T00:26:06.327 INFO:tasks.workunit.client.0.smithi112.stdout:7/704: write d6/d10/d16/d49/d6c/f62 [3126450,24765] 0 2024-09-16T00:26:06.327 INFO:tasks.workunit.client.0.smithi112.stdout:3/743: creat d2/d9/d2b/d47/dcf/ff5 x:0 0 0 2024-09-16T00:26:06.328 INFO:tasks.workunit.client.0.smithi112.stdout:7/705: dread - d6/d10/d16/d3b/fa6 zero size 2024-09-16T00:26:06.329 INFO:tasks.workunit.client.1.smithi169.stdout:9/556: chown d5/da/d5b/f63 29 1 2024-09-16T00:26:06.330 INFO:tasks.workunit.client.0.smithi112.stdout:3/744: fsync d2/d9/d2b/fa0 0 2024-09-16T00:26:06.332 INFO:tasks.workunit.client.1.smithi169.stdout:9/557: read d5/da/d5b/d6c/da0/f95 [166789,16228] 0 2024-09-16T00:26:06.332 INFO:tasks.workunit.client.0.smithi112.stdout:2/815: chown d2/dd/d48/d22/d2b/d54/l106 98 1 2024-09-16T00:26:06.332 INFO:tasks.workunit.client.0.smithi112.stdout:9/897: mkdir d1/d5/d20/d81/d1a/d13a 0 2024-09-16T00:26:06.333 INFO:tasks.workunit.client.0.smithi112.stdout:4/732: chown d6/d39/d1c/d2c/d69/d7b/de0/c108 1789617 1 2024-09-16T00:26:06.333 INFO:tasks.workunit.client.0.smithi112.stdout:0/785: rename d5/da/d57/f8a to d5/da/d1d/d71/fed 0 2024-09-16T00:26:06.334 INFO:tasks.workunit.client.0.smithi112.stdout:8/647: truncate d10/d12/d1c/d61/d6c/fc4 306870 0 2024-09-16T00:26:06.335 INFO:tasks.workunit.client.0.smithi112.stdout:7/706: mknod d6/d10/d75/dea/ceb 0 2024-09-16T00:26:06.337 INFO:tasks.workunit.client.0.smithi112.stdout:4/733: mknod d6/db/d1f/d41/d49/c109 0 2024-09-16T00:26:06.337 INFO:tasks.workunit.client.0.smithi112.stdout:8/648: dread - d10/d12/d1c/d61/d6c/dd1/d58/fc6 zero size 2024-09-16T00:26:06.339 INFO:tasks.workunit.client.1.smithi169.stdout:0/508: truncate d0/d2/d15/f22 1702589 0 2024-09-16T00:26:06.340 INFO:tasks.workunit.client.0.smithi112.stdout:2/816: read d2/dd/f5e [1995562,33515] 0 2024-09-16T00:26:06.341 INFO:tasks.workunit.client.0.smithi112.stdout:5/616: dwrite d4/dd/d12/f11 [0,4194304] 0 2024-09-16T00:26:06.342 INFO:tasks.workunit.client.0.smithi112.stdout:0/786: mknod d5/da/d18/d24/d46/d88/cee 0 2024-09-16T00:26:06.345 INFO:tasks.workunit.client.0.smithi112.stdout:0/787: write d5/da/d1d/d71/fbc [924684,86249] 0 2024-09-16T00:26:06.346 INFO:tasks.workunit.client.0.smithi112.stdout:7/707: creat d6/d10/d16/d49/d6c/fec x:0 0 0 2024-09-16T00:26:06.348 INFO:tasks.workunit.client.1.smithi169.stdout:9/558: dwrite d5/d12/f86 [0,4194304] 0 2024-09-16T00:26:06.348 INFO:tasks.workunit.client.1.smithi169.stdout:8/495: dread d7/dc/d26/d46/f6b [0,4194304] 0 2024-09-16T00:26:06.348 INFO:tasks.workunit.client.0.smithi112.stdout:0/788: chown d5/da/d21/dc3/dd3 1384 1 2024-09-16T00:26:06.350 INFO:tasks.workunit.client.0.smithi112.stdout:4/734: creat d6/d39/d1c/d2c/d69/f10a x:0 0 0 2024-09-16T00:26:06.351 INFO:tasks.workunit.client.0.smithi112.stdout:5/617: dwrite d4/d4f/d77/fa2 [4194304,4194304] 0 2024-09-16T00:26:06.351 INFO:tasks.workunit.client.1.smithi169.stdout:0/509: truncate d0/d2/d4b/d86/f5a 1377288 0 2024-09-16T00:26:06.352 INFO:tasks.workunit.client.0.smithi112.stdout:6/652: write dd/df/d13/f48 [5083563,92935] 0 2024-09-16T00:26:06.352 INFO:tasks.workunit.client.1.smithi169.stdout:0/510: stat d0/d2/d8/c92 0 2024-09-16T00:26:06.356 INFO:tasks.workunit.client.1.smithi169.stdout:9/559: creat d5/fc6 x:0 0 0 2024-09-16T00:26:06.356 INFO:tasks.workunit.client.1.smithi169.stdout:8/496: creat d7/dc/d26/d41/fa4 x:0 0 0 2024-09-16T00:26:06.358 INFO:tasks.workunit.client.1.smithi169.stdout:8/497: chown d7/dc/d26/d41/f5c 76075 1 2024-09-16T00:26:06.358 INFO:tasks.workunit.client.1.smithi169.stdout:8/498: stat d7/dc/d26 0 2024-09-16T00:26:06.359 INFO:tasks.workunit.client.1.smithi169.stdout:9/560: creat d5/da/d21/d23/fc7 x:0 0 0 2024-09-16T00:26:06.359 INFO:tasks.workunit.client.1.smithi169.stdout:8/499: fsync d7/d20/f21 0 2024-09-16T00:26:06.360 INFO:tasks.workunit.client.1.smithi169.stdout:0/511: rename d0/d2/fd to d0/d2/fb5 0 2024-09-16T00:26:06.363 INFO:tasks.workunit.client.0.smithi112.stdout:9/898: link d1/d5/d6/df8/dba/c101 d1/d5/d2c/c13b 0 2024-09-16T00:26:06.364 INFO:tasks.workunit.client.0.smithi112.stdout:9/899: chown d1/d5/d6/df8/dba 3 1 2024-09-16T00:26:06.364 INFO:tasks.workunit.client.0.smithi112.stdout:2/817: symlink d2/dd/d48/d22/d2b/d54/d60/l107 0 2024-09-16T00:26:06.365 INFO:tasks.workunit.client.0.smithi112.stdout:4/735: dwrite d6/d39/d1c/d66/f7d [4194304,4194304] 0 2024-09-16T00:26:06.365 INFO:tasks.workunit.client.1.smithi169.stdout:6/697: sync 2024-09-16T00:26:06.366 INFO:tasks.workunit.client.1.smithi169.stdout:5/454: sync 2024-09-16T00:26:06.366 INFO:tasks.workunit.client.1.smithi169.stdout:0/512: rmdir d0/d2/d8/d5/d79/d7c 39 2024-09-16T00:26:06.367 INFO:tasks.workunit.client.1.smithi169.stdout:8/500: mkdir d7/dc/d26/d41/d4e/d81/d9e/da5 0 2024-09-16T00:26:06.368 INFO:tasks.workunit.client.0.smithi112.stdout:8/649: creat d10/d12/fd7 x:0 0 0 2024-09-16T00:26:06.369 INFO:tasks.workunit.client.1.smithi169.stdout:9/561: dwrite d5/d12/f96 [0,4194304] 0 2024-09-16T00:26:06.369 INFO:tasks.workunit.client.0.smithi112.stdout:4/736: mknod d6/db/df/d5e/c10b 0 2024-09-16T00:26:06.371 INFO:tasks.workunit.client.0.smithi112.stdout:7/708: rename d6/d10/d16/d96/dc7/c5f to d6/d10/da1/ced 0 2024-09-16T00:26:06.371 INFO:tasks.workunit.client.0.smithi112.stdout:8/650: creat d10/d12/d1c/d61/d6c/dd1/d8c/fd8 x:0 0 0 2024-09-16T00:26:06.372 INFO:tasks.workunit.client.0.smithi112.stdout:4/737: mkdir d6/db/d1f/d94/d10c 0 2024-09-16T00:26:06.374 INFO:tasks.workunit.client.0.smithi112.stdout:4/738: write d6/db/d1f/fbc [818961,16177] 0 2024-09-16T00:26:06.375 INFO:tasks.workunit.client.0.smithi112.stdout:5/618: rename d4/dd/c22 to d4/d4f/d78/cc8 0 2024-09-16T00:26:06.379 INFO:tasks.workunit.client.0.smithi112.stdout:5/619: symlink d4/d7/d26/d4c/lc9 0 2024-09-16T00:26:06.381 INFO:tasks.workunit.client.0.smithi112.stdout:9/900: rename d1/d5/d2c/c13b to d1/d5/d6/df8/c13c 0 2024-09-16T00:26:06.382 INFO:tasks.workunit.client.1.smithi169.stdout:8/501: creat d7/dc/d26/d41/d4e/d81/d9e/da5/fa6 x:0 0 0 2024-09-16T00:26:06.384 INFO:tasks.workunit.client.0.smithi112.stdout:5/620: fdatasync d4/dd/f27 0 2024-09-16T00:26:06.384 INFO:tasks.workunit.client.0.smithi112.stdout:9/901: write d1/d5/d2c/fc3 [2052177,12876] 0 2024-09-16T00:26:06.384 INFO:tasks.workunit.client.1.smithi169.stdout:9/562: dwrite d5/da/d5b/d6f/fa1 [0,4194304] 0 2024-09-16T00:26:06.384 INFO:tasks.workunit.client.1.smithi169.stdout:8/502: write d7/d20/f7e [2363800,53788] 0 2024-09-16T00:26:06.386 INFO:tasks.workunit.client.0.smithi112.stdout:4/739: rename d6/db/df/d5e/d61/d64/d6a/db8/ld4 to d6/db/df/d5e/l10d 0 2024-09-16T00:26:06.386 INFO:tasks.workunit.client.1.smithi169.stdout:0/513: rename d0/d2/d4b/l8f to d0/d2c/d54/d21/d31/d83/dac/lb6 0 2024-09-16T00:26:06.386 INFO:tasks.workunit.client.1.smithi169.stdout:9/563: write d5/da/d21/d23/f40 [2572515,98053] 0 2024-09-16T00:26:06.386 INFO:tasks.workunit.client.1.smithi169.stdout:8/503: dread - d7/d37/f9b zero size 2024-09-16T00:26:06.387 INFO:tasks.workunit.client.0.smithi112.stdout:9/902: write d1/d5/d20/d81/d1a/d88/f10f [1781646,121370] 0 2024-09-16T00:26:06.392 INFO:tasks.workunit.client.1.smithi169.stdout:9/564: rename d5/da/d73/lb2 to d5/da/d21/d8d/db4/lc8 0 2024-09-16T00:26:06.393 INFO:tasks.workunit.client.0.smithi112.stdout:5/621: rename d4/l39 to d4/d7/d14/d90/d9a/d7c/lca 0 2024-09-16T00:26:06.393 INFO:tasks.workunit.client.1.smithi169.stdout:5/455: write d2/da/d10/d13/d1c/d32/d53/f4d [2374118,88584] 0 2024-09-16T00:26:06.393 INFO:tasks.workunit.client.1.smithi169.stdout:6/698: truncate d2/d36/f58 837097 0 2024-09-16T00:26:06.397 INFO:tasks.workunit.client.0.smithi112.stdout:9/903: mknod d1/d5/d6/d18/c13d 0 2024-09-16T00:26:06.397 INFO:tasks.workunit.client.0.smithi112.stdout:4/740: dwrite d6/db/d43/fea [0,4194304] 0 2024-09-16T00:26:06.398 INFO:tasks.workunit.client.1.smithi169.stdout:8/504: dwrite d7/dc/d17/f89 [4194304,4194304] 0 2024-09-16T00:26:06.399 INFO:tasks.workunit.client.0.smithi112.stdout:4/741: chown d6/db/df/d5e/d61/c4e 69 1 2024-09-16T00:26:06.399 INFO:tasks.workunit.client.0.smithi112.stdout:5/622: rename d4/d7/d26/d4c/db3/db6/fbc to d4/d7/d87/fcb 0 2024-09-16T00:26:06.400 INFO:tasks.workunit.client.0.smithi112.stdout:6/653: write dd/f2a [430097,30201] 0 2024-09-16T00:26:06.401 INFO:tasks.workunit.client.1.smithi169.stdout:9/565: rename d5/da/d4e to d5/da/d5b/d91/dc9 0 2024-09-16T00:26:06.402 INFO:tasks.workunit.client.0.smithi112.stdout:4/742: write d6/d39/d1c/d2c/ff2 [955688,45133] 0 2024-09-16T00:26:06.402 INFO:tasks.workunit.client.0.smithi112.stdout:9/904: chown d1/d5/d6/d18/c10d 121671 1 2024-09-16T00:26:06.402 INFO:tasks.workunit.client.1.smithi169.stdout:0/514: dwrite d0/d2/d8/d5/d79/fae [0,4194304] 0 2024-09-16T00:26:06.402 INFO:tasks.workunit.client.1.smithi169.stdout:6/699: mkdir d2/d3f/d41/d72/d9e/de1 0 2024-09-16T00:26:06.404 INFO:tasks.workunit.client.1.smithi169.stdout:9/566: stat d5/da/d21/d8d/db4/fbc 0 2024-09-16T00:26:06.405 INFO:tasks.workunit.client.0.smithi112.stdout:4/743: write d6/d39/d1c/d6e/ffd [740999,82212] 0 2024-09-16T00:26:06.405 INFO:tasks.workunit.client.0.smithi112.stdout:6/654: unlink dd/df/f66 0 2024-09-16T00:26:06.407 INFO:tasks.workunit.client.1.smithi169.stdout:0/515: creat d0/d2/d8/fb7 x:0 0 0 2024-09-16T00:26:06.407 INFO:tasks.workunit.client.1.smithi169.stdout:5/456: dread d2/f2a [0,4194304] 0 2024-09-16T00:26:06.408 INFO:tasks.workunit.client.1.smithi169.stdout:8/505: dwrite d7/fa [4194304,4194304] 0 2024-09-16T00:26:06.409 INFO:tasks.workunit.client.1.smithi169.stdout:0/516: dread - d0/d2/d8/f48 zero size 2024-09-16T00:26:06.409 INFO:tasks.workunit.client.0.smithi112.stdout:4/744: chown d6/db/l105 56 1 2024-09-16T00:26:06.410 INFO:tasks.workunit.client.0.smithi112.stdout:6/655: mkdir dd/df/d91/dd7 0 2024-09-16T00:26:06.411 INFO:tasks.workunit.client.0.smithi112.stdout:4/745: creat d6/d39/d1c/d6e/f10e x:0 0 0 2024-09-16T00:26:06.412 INFO:tasks.workunit.client.0.smithi112.stdout:7/709: dread d6/d10/d16/f3e [0,4194304] 0 2024-09-16T00:26:06.413 INFO:tasks.workunit.client.0.smithi112.stdout:6/656: stat dd/fca 0 2024-09-16T00:26:06.425 INFO:tasks.workunit.client.0.smithi112.stdout:6/657: rmdir dd/df/d11/d44/d8a 39 2024-09-16T00:26:06.429 INFO:tasks.workunit.client.0.smithi112.stdout:5/623: rename d4/d7/d14/d90/d9a to d4/d4f/d77/dcc 0 2024-09-16T00:26:06.430 INFO:tasks.workunit.client.0.smithi112.stdout:5/624: stat d4/dd/d12/d52/c65 0 2024-09-16T00:26:06.431 INFO:tasks.workunit.client.0.smithi112.stdout:5/625: chown d4/d7/d26/d4c/db3 2839 1 2024-09-16T00:26:06.433 INFO:tasks.workunit.client.0.smithi112.stdout:4/746: link d6/db/df/d5e/d61/d64/d6a/cc5 d6/c10f 0 2024-09-16T00:26:06.436 INFO:tasks.workunit.client.0.smithi112.stdout:4/747: truncate d6/db/d1f/fc1 238012 0 2024-09-16T00:26:06.438 INFO:tasks.workunit.client.0.smithi112.stdout:5/626: dwrite d4/dd/d12/f48 [0,4194304] 0 2024-09-16T00:26:06.440 INFO:tasks.workunit.client.0.smithi112.stdout:4/748: symlink d6/db/df/d5e/d61/d64/d6a/db8/l110 0 2024-09-16T00:26:06.441 INFO:tasks.workunit.client.0.smithi112.stdout:9/905: dwrite d1/d76/fdc [0,4194304] 0 2024-09-16T00:26:06.448 INFO:tasks.workunit.client.0.smithi112.stdout:4/749: mknod d6/db/d1f/d41/d49/d31/c111 0 2024-09-16T00:26:06.452 INFO:tasks.workunit.client.0.smithi112.stdout:9/906: dwrite d1/ff4 [0,4194304] 0 2024-09-16T00:26:06.457 INFO:tasks.workunit.client.0.smithi112.stdout:6/658: write dd/df/d13/d18/f28 [264222,16573] 0 2024-09-16T00:26:06.457 INFO:tasks.workunit.client.0.smithi112.stdout:7/710: dwrite d6/d10/f76 [0,4194304] 0 2024-09-16T00:26:06.459 INFO:tasks.workunit.client.0.smithi112.stdout:9/907: symlink d1/d5/d6/d59/d64/l13e 0 2024-09-16T00:26:06.460 INFO:tasks.workunit.client.0.smithi112.stdout:7/711: mkdir d6/d10/d75/d8f/d90/ddc/dee 0 2024-09-16T00:26:06.462 INFO:tasks.workunit.client.0.smithi112.stdout:9/908: fsync d1/d5/d6/d2d/d39/f65 0 2024-09-16T00:26:06.462 INFO:tasks.workunit.client.0.smithi112.stdout:7/712: dread - d6/d10/f6b zero size 2024-09-16T00:26:06.464 INFO:tasks.workunit.client.0.smithi112.stdout:5/627: truncate d4/d1c/d43/f6d 3675663 0 2024-09-16T00:26:06.465 INFO:tasks.workunit.client.0.smithi112.stdout:9/909: rmdir d1/d5/d6/d59/d64 39 2024-09-16T00:26:06.466 INFO:tasks.workunit.client.0.smithi112.stdout:5/628: creat d4/d7/d14/fcd x:0 0 0 2024-09-16T00:26:06.467 INFO:tasks.workunit.client.0.smithi112.stdout:5/629: chown d4/f19 1 1 2024-09-16T00:26:06.468 INFO:tasks.workunit.client.0.smithi112.stdout:7/713: creat d6/d10/d16/d96/dc7/fef x:0 0 0 2024-09-16T00:26:06.470 INFO:tasks.workunit.client.0.smithi112.stdout:7/714: rmdir d6/d10/d75 39 2024-09-16T00:26:06.471 INFO:tasks.workunit.client.0.smithi112.stdout:5/630: truncate d4/d1c/f32 1165631 0 2024-09-16T00:26:06.472 INFO:tasks.workunit.client.0.smithi112.stdout:9/910: dwrite d1/d5/d6/d2d/d39/f65 [0,4194304] 0 2024-09-16T00:26:06.473 INFO:tasks.workunit.client.0.smithi112.stdout:4/750: rmdir d6/db 39 2024-09-16T00:26:06.474 INFO:tasks.workunit.client.0.smithi112.stdout:7/715: creat d6/d10/d75/d88/ddf/ff0 x:0 0 0 2024-09-16T00:26:06.477 INFO:tasks.workunit.client.0.smithi112.stdout:5/631: rename d4/f16 to d4/d7/d87/fce 0 2024-09-16T00:26:06.479 INFO:tasks.workunit.client.0.smithi112.stdout:4/751: chown d6/db/df/d5e/d61/d64/d6a/fd0 1 1 2024-09-16T00:26:06.481 INFO:tasks.workunit.client.0.smithi112.stdout:6/659: write dd/f7e [1745021,129157] 0 2024-09-16T00:26:06.485 INFO:tasks.workunit.client.0.smithi112.stdout:5/632: chown d4/dd/c17 359 1 2024-09-16T00:26:06.485 INFO:tasks.workunit.client.0.smithi112.stdout:5/633: dread - d4/d7/d14/fcd zero size 2024-09-16T00:26:06.485 INFO:tasks.workunit.client.0.smithi112.stdout:9/911: rename d1/d5/d6/d2d/f3b to d1/d5/d6/d114/d116/f13f 0 2024-09-16T00:26:06.486 INFO:tasks.workunit.client.0.smithi112.stdout:9/912: dread - d1/f108 zero size 2024-09-16T00:26:06.486 INFO:tasks.workunit.client.0.smithi112.stdout:5/634: chown d4/d7/d26/d4c/l7a 5 1 2024-09-16T00:26:06.486 INFO:tasks.workunit.client.0.smithi112.stdout:6/660: symlink dd/df/d10/d47/ld8 0 2024-09-16T00:26:06.487 INFO:tasks.workunit.client.0.smithi112.stdout:6/661: stat dd/fca 0 2024-09-16T00:26:06.487 INFO:tasks.workunit.client.0.smithi112.stdout:4/752: rename d6/d39/fce to d6/d39/d36/d5c/d7e/f112 0 2024-09-16T00:26:06.488 INFO:tasks.workunit.client.0.smithi112.stdout:9/913: creat d1/d5/d6/d107/f140 x:0 0 0 2024-09-16T00:26:06.495 INFO:tasks.workunit.client.0.smithi112.stdout:4/753: dwrite d6/d39/d1c/d6e/fbe [0,4194304] 0 2024-09-16T00:26:06.503 INFO:tasks.workunit.client.0.smithi112.stdout:9/914: creat d1/d5/d6/d2d/d6a/d89/f141 x:0 0 0 2024-09-16T00:26:06.503 INFO:tasks.workunit.client.0.smithi112.stdout:4/754: link d6/db/d43/ccf d6/d39/d1c/d6e/d6f/c113 0 2024-09-16T00:26:06.504 INFO:tasks.workunit.client.0.smithi112.stdout:6/662: truncate dd/d23/f63 3397685 0 2024-09-16T00:26:06.506 INFO:tasks.workunit.client.0.smithi112.stdout:7/716: dread d6/d10/d75/d88/f91 [0,4194304] 0 2024-09-16T00:26:06.507 INFO:tasks.workunit.client.0.smithi112.stdout:4/755: mknod d6/d7c/da0/c114 0 2024-09-16T00:26:06.507 INFO:tasks.workunit.client.0.smithi112.stdout:6/663: write dd/df/fc2 [479613,24591] 0 2024-09-16T00:26:06.508 INFO:tasks.workunit.client.0.smithi112.stdout:9/915: truncate d1/f50 992719 0 2024-09-16T00:26:06.508 INFO:tasks.workunit.client.0.smithi112.stdout:7/717: chown d6/fae 124552874 1 2024-09-16T00:26:06.508 INFO:tasks.workunit.client.0.smithi112.stdout:5/635: dwrite d4/fb [0,4194304] 0 2024-09-16T00:26:06.510 INFO:tasks.workunit.client.0.smithi112.stdout:7/718: dread - d6/d10/d16/d49/d6c/fec zero size 2024-09-16T00:26:06.511 INFO:tasks.workunit.client.0.smithi112.stdout:5/636: mkdir d4/d7/d26/d36/dcf 0 2024-09-16T00:26:06.512 INFO:tasks.workunit.client.0.smithi112.stdout:5/637: chown d4/d1c 3 1 2024-09-16T00:26:06.512 INFO:tasks.workunit.client.0.smithi112.stdout:7/719: readlink d6/lc9 0 2024-09-16T00:26:06.515 INFO:tasks.workunit.client.0.smithi112.stdout:9/916: dread d1/d5/d20/d81/d1a/d88/def/f2e [0,4194304] 0 2024-09-16T00:26:06.515 INFO:tasks.workunit.client.0.smithi112.stdout:5/638: write d4/d7/d26/d36/f41 [3835255,75266] 0 2024-09-16T00:26:06.516 INFO:tasks.workunit.client.1.smithi169.stdout:8/506: creat d7/dc/d26/fa7 x:0 0 0 2024-09-16T00:26:06.518 INFO:tasks.workunit.client.0.smithi112.stdout:4/756: dwrite d6/d39/d1c/d2c/ff4 [0,4194304] 0 2024-09-16T00:26:06.518 INFO:tasks.workunit.client.1.smithi169.stdout:5/457: symlink d2/da/d10/d13/d1c/d40/d54/d9c/l9d 0 2024-09-16T00:26:06.519 INFO:tasks.workunit.client.1.smithi169.stdout:0/517: dread d0/d2/d8/f47 [4194304,4194304] 0 2024-09-16T00:26:06.519 INFO:tasks.workunit.client.0.smithi112.stdout:9/917: symlink d1/d5/d6/d38/l142 0 2024-09-16T00:26:06.522 INFO:tasks.workunit.client.1.smithi169.stdout:0/518: creat d0/d2c/d54/d21/d31/d83/dac/fb8 x:0 0 0 2024-09-16T00:26:06.523 INFO:tasks.workunit.client.0.smithi112.stdout:4/757: unlink d6/db/df/d5e/d61/d64/fee 0 2024-09-16T00:26:06.523 INFO:tasks.workunit.client.1.smithi169.stdout:8/507: creat d7/dc/d17/fa8 x:0 0 0 2024-09-16T00:26:06.524 INFO:tasks.workunit.client.1.smithi169.stdout:5/458: truncate d2/da/d10/d13/f1d 1342391 0 2024-09-16T00:26:06.525 INFO:tasks.workunit.client.1.smithi169.stdout:8/508: dread - d7/dc/d53/f93 zero size 2024-09-16T00:26:06.525 INFO:tasks.workunit.client.1.smithi169.stdout:0/519: symlink d0/d2c/d4f/lb9 0 2024-09-16T00:26:06.526 INFO:tasks.workunit.client.0.smithi112.stdout:4/758: creat d6/db/d1f/d41/d49/f115 x:0 0 0 2024-09-16T00:26:06.527 INFO:tasks.workunit.client.1.smithi169.stdout:0/520: write d0/d50/f9f [950636,84601] 0 2024-09-16T00:26:06.527 INFO:tasks.workunit.client.1.smithi169.stdout:8/509: mknod d7/dc/d26/ca9 0 2024-09-16T00:26:06.528 INFO:tasks.workunit.client.0.smithi112.stdout:9/918: link d1/d5/d6/d18/fb0 d1/d5/d6/d107/f143 0 2024-09-16T00:26:06.529 INFO:tasks.workunit.client.0.smithi112.stdout:4/759: mkdir d6/d39/d36/d5c/d7e/dcd/d116 0 2024-09-16T00:26:06.529 INFO:tasks.workunit.client.1.smithi169.stdout:0/521: fsync d0/d2c/d54/f28 0 2024-09-16T00:26:06.531 INFO:tasks.workunit.client.0.smithi112.stdout:9/919: creat d1/d5/d6/d107/f144 x:0 0 0 2024-09-16T00:26:06.539 INFO:tasks.workunit.client.0.smithi112.stdout:4/760: dwrite d6/d39/d36/d5c/dc6/fb6 [0,4194304] 0 2024-09-16T00:26:06.541 INFO:tasks.workunit.client.0.smithi112.stdout:7/720: dwrite d6/d10/d16/faf [0,4194304] 0 2024-09-16T00:26:06.541 INFO:tasks.workunit.client.0.smithi112.stdout:4/761: chown d6/d39/d36/d5c/c91 30 1 2024-09-16T00:26:06.546 INFO:tasks.workunit.client.1.smithi169.stdout:0/522: write d0/d2c/d54/f28 [3599207,12051] 0 2024-09-16T00:26:06.550 INFO:tasks.workunit.client.0.smithi112.stdout:4/762: link d6/d39/d1c/d6e/ffd d6/d7c/da0/f117 0 2024-09-16T00:26:06.550 INFO:tasks.workunit.client.1.smithi169.stdout:8/510: dread d7/dc/d26/d46/f5d [0,4194304] 0 2024-09-16T00:26:06.551 INFO:tasks.workunit.client.0.smithi112.stdout:9/920: getdents d1/d5/d6/d107 0 2024-09-16T00:26:06.551 INFO:tasks.workunit.client.0.smithi112.stdout:7/721: dwrite d6/d10/d16/f34 [0,4194304] 0 2024-09-16T00:26:06.555 INFO:tasks.workunit.client.1.smithi169.stdout:5/459: dread d2/f29 [0,4194304] 0 2024-09-16T00:26:06.557 INFO:tasks.workunit.client.1.smithi169.stdout:8/511: dread d7/dc/d1d/d80/f8b [0,4194304] 0 2024-09-16T00:26:06.558 INFO:tasks.workunit.client.0.smithi112.stdout:4/763: link d6/d39/d1c/d6e/d6f/c113 d6/d39/d1c/d2c/d69/d7b/de0/c118 0 2024-09-16T00:26:06.559 INFO:tasks.workunit.client.1.smithi169.stdout:8/512: stat d7/dc/d17/d24/d32/d4d 0 2024-09-16T00:26:06.560 INFO:tasks.workunit.client.1.smithi169.stdout:5/460: mkdir d2/da/d10/d13/d1c/d9e 0 2024-09-16T00:26:06.560 INFO:tasks.workunit.client.0.smithi112.stdout:4/764: write d6/db/d1f/d41/ddf/f104 [741645,42080] 0 2024-09-16T00:26:06.561 INFO:tasks.workunit.client.1.smithi169.stdout:8/513: readlink d7/dc/d17/d24/d32/d4d/l84 0 2024-09-16T00:26:06.562 INFO:tasks.workunit.client.1.smithi169.stdout:0/523: truncate d0/f13 4119042 0 2024-09-16T00:26:06.563 INFO:tasks.workunit.client.1.smithi169.stdout:5/461: fsync d2/da/f38 0 2024-09-16T00:26:06.563 INFO:tasks.workunit.client.0.smithi112.stdout:4/765: mkdir d6/d39/d1c/d6e/d119 0 2024-09-16T00:26:06.565 INFO:tasks.workunit.client.0.smithi112.stdout:4/766: readlink d6/d39/d36/le3 0 2024-09-16T00:26:06.565 INFO:tasks.workunit.client.1.smithi169.stdout:0/524: fsync d0/d2c/d54/d21/f49 0 2024-09-16T00:26:06.567 INFO:tasks.workunit.client.1.smithi169.stdout:5/462: mknod d2/da/d10/d13/d1c/d40/d54/d7f/c9f 0 2024-09-16T00:26:06.568 INFO:tasks.workunit.client.1.smithi169.stdout:5/463: fdatasync d2/da/d10/d13/d1c/d40/d43/f4c 0 2024-09-16T00:26:06.569 INFO:tasks.workunit.client.0.smithi112.stdout:4/767: rename d6/d39/d1c/d2c/ff4 to d6/d39/d1c/d2c/d69/d7b/f11a 0 2024-09-16T00:26:06.574 INFO:tasks.workunit.client.0.smithi112.stdout:7/722: dread d6/f22 [0,4194304] 0 2024-09-16T00:26:06.576 INFO:tasks.workunit.client.0.smithi112.stdout:9/921: dwrite d1/d5/d6/d18/fb0 [0,4194304] 0 2024-09-16T00:26:06.580 INFO:tasks.workunit.client.0.smithi112.stdout:9/922: mknod d1/d5/d20/d81/d11d/c145 0 2024-09-16T00:26:06.583 INFO:tasks.workunit.client.0.smithi112.stdout:7/723: dwrite d6/d10/f76 [0,4194304] 0 2024-09-16T00:26:06.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:06 smithi112 ceph-mon[27869]: pgmap v256: 65 pgs: 65 active+clean; 2.3 GiB data, 8.0 GiB used, 528 GiB / 536 GiB avail; 87 MiB/s rd, 301 MiB/s wr, 514 op/s 2024-09-16T00:26:06.593 INFO:tasks.workunit.client.0.smithi112.stdout:9/923: dwrite d1/d5/d2c/fd4 [0,4194304] 0 2024-09-16T00:26:06.594 INFO:tasks.workunit.client.1.smithi169.stdout:5/464: dwrite d2/f29 [0,4194304] 0 2024-09-16T00:26:06.597 INFO:tasks.workunit.client.0.smithi112.stdout:7/724: dwrite d6/d10/d16/d96/dc7/d39/fa8 [0,4194304] 0 2024-09-16T00:26:06.598 INFO:tasks.workunit.client.0.smithi112.stdout:9/924: symlink d1/d5/d20/d81/d1a/d88/l146 0 2024-09-16T00:26:06.599 INFO:tasks.workunit.client.1.smithi169.stdout:5/465: creat d2/da/d10/d13/d24/fa0 x:0 0 0 2024-09-16T00:26:06.602 INFO:tasks.workunit.client.1.smithi169.stdout:5/466: link d2/f88 d2/da/d10/d13/d1c/d32/d53/d83/d7d/fa1 0 2024-09-16T00:26:06.603 INFO:tasks.workunit.client.0.smithi112.stdout:7/725: symlink d6/lf1 0 2024-09-16T00:26:06.605 INFO:tasks.workunit.client.0.smithi112.stdout:9/925: dread d1/d5/d6/d59/d64/fa1 [0,4194304] 0 2024-09-16T00:26:06.607 INFO:tasks.workunit.client.1.smithi169.stdout:8/514: dread d7/f12 [0,4194304] 0 2024-09-16T00:26:06.608 INFO:tasks.workunit.client.1.smithi169.stdout:8/515: fsync d7/dc/d26/d41/f5c 0 2024-09-16T00:26:06.608 INFO:tasks.workunit.client.1.smithi169.stdout:3/525: sync 2024-09-16T00:26:06.608 INFO:tasks.workunit.client.0.smithi112.stdout:9/926: mknod d1/d5/d2c/c147 0 2024-09-16T00:26:06.610 INFO:tasks.workunit.client.1.smithi169.stdout:5/467: dread d2/f29 [0,4194304] 0 2024-09-16T00:26:06.611 INFO:tasks.workunit.client.0.smithi112.stdout:9/927: creat d1/d5/d20/d81/d1a/d88/def/f148 x:0 0 0 2024-09-16T00:26:06.613 INFO:tasks.workunit.client.0.smithi112.stdout:9/928: chown d1/d76/cd2 0 1 2024-09-16T00:26:06.613 INFO:tasks.workunit.client.0.smithi112.stdout:7/726: dwrite d6/d10/f6e [0,4194304] 0 2024-09-16T00:26:06.613 INFO:tasks.workunit.client.1.smithi169.stdout:3/526: mkdir d7/d1a/d1c/d35/d63/dc2/dc7 0 2024-09-16T00:26:06.614 INFO:tasks.workunit.client.1.smithi169.stdout:5/468: mknod d2/da/d10/d13/d1c/d40/d54/ca2 0 2024-09-16T00:26:06.615 INFO:tasks.workunit.client.0.smithi112.stdout:7/727: dread - d6/d10/d16/d3b/fa6 zero size 2024-09-16T00:26:06.616 INFO:tasks.workunit.client.1.smithi169.stdout:8/516: dwrite d7/dc/d17/d24/f27 [0,4194304] 0 2024-09-16T00:26:06.618 INFO:tasks.workunit.client.1.smithi169.stdout:5/469: mkdir d2/da/da3 0 2024-09-16T00:26:06.619 INFO:tasks.workunit.client.1.smithi169.stdout:3/527: mkdir d7/d1a/d1c/d23/d88/d9b/dc8 0 2024-09-16T00:26:06.619 INFO:tasks.workunit.client.1.smithi169.stdout:8/517: fdatasync d7/dc/d26/d46/f6b 0 2024-09-16T00:26:06.622 INFO:tasks.workunit.client.1.smithi169.stdout:8/518: creat d7/dc/d17/d24/d32/d4d/faa x:0 0 0 2024-09-16T00:26:06.623 INFO:tasks.workunit.client.1.smithi169.stdout:3/528: creat d7/d1a/d2b/fc9 x:0 0 0 2024-09-16T00:26:06.627 INFO:tasks.workunit.client.0.smithi112.stdout:1/775: sync 2024-09-16T00:26:06.628 INFO:tasks.workunit.client.1.smithi169.stdout:5/470: dwrite d2/da/d67/f82 [0,4194304] 0 2024-09-16T00:26:06.629 INFO:tasks.workunit.client.1.smithi169.stdout:3/529: getdents d7 0 2024-09-16T00:26:06.629 INFO:tasks.workunit.client.0.smithi112.stdout:1/776: getdents d0/d14/d1c/de9 0 2024-09-16T00:26:06.630 INFO:tasks.workunit.client.0.smithi112.stdout:7/728: dread d6/d10/d16/f4e [0,4194304] 0 2024-09-16T00:26:06.631 INFO:tasks.workunit.client.0.smithi112.stdout:1/777: symlink d0/d14/d44/lf6 0 2024-09-16T00:26:06.632 INFO:tasks.workunit.client.0.smithi112.stdout:7/729: write d6/d10/d16/d96/dc7/f7d [5094021,39132] 0 2024-09-16T00:26:06.634 INFO:tasks.workunit.client.1.smithi169.stdout:8/519: dwrite d7/dc/d17/f1b [0,4194304] 0 2024-09-16T00:26:06.635 INFO:tasks.workunit.client.0.smithi112.stdout:1/778: unlink d0/d14/df/d28/d45/dd4/f52 0 2024-09-16T00:26:06.636 INFO:tasks.workunit.client.1.smithi169.stdout:3/530: chown d7/d1a/d1c/d35/d63/d37/f9a 16172906 1 2024-09-16T00:26:06.636 INFO:tasks.workunit.client.1.smithi169.stdout:8/520: truncate d7/dc/d2a/f85 4313196 0 2024-09-16T00:26:06.637 INFO:tasks.workunit.client.0.smithi112.stdout:1/779: fdatasync d0/d14/df/d28/d45/d46/d58/d62/db0/fb8 0 2024-09-16T00:26:06.638 INFO:tasks.workunit.client.0.smithi112.stdout:7/730: creat d6/d10/d16/d49/d6c/ff2 x:0 0 0 2024-09-16T00:26:06.638 INFO:tasks.workunit.client.1.smithi169.stdout:8/521: chown d7/dc/d17/d48/f60 3701167 1 2024-09-16T00:26:06.642 INFO:tasks.workunit.client.0.smithi112.stdout:1/780: dread d0/d14/df/d28/d45/dd4/d47/d85/fd2 [4194304,4194304] 0 2024-09-16T00:26:06.644 INFO:tasks.workunit.client.1.smithi169.stdout:3/531: dwrite d7/d1a/d1c/fb6 [0,4194304] 0 2024-09-16T00:26:06.645 INFO:tasks.workunit.client.1.smithi169.stdout:3/532: chown d7/d50/d90/lc4 1 1 2024-09-16T00:26:06.646 INFO:tasks.workunit.client.0.smithi112.stdout:9/929: dread d1/d5/d6/d107/f113 [0,4194304] 0 2024-09-16T00:26:06.647 INFO:tasks.workunit.client.0.smithi112.stdout:1/781: link d0/d14/df/d28/d45/d46/f49 d0/d14/df/d28/d45/d46/d58/ff7 0 2024-09-16T00:26:06.649 INFO:tasks.workunit.client.0.smithi112.stdout:1/782: chown d0/d14/df/d28/d42/c8e 94238 1 2024-09-16T00:26:06.651 INFO:tasks.workunit.client.0.smithi112.stdout:9/930: getdents d1/d5/d83 0 2024-09-16T00:26:06.651 INFO:tasks.workunit.client.0.smithi112.stdout:7/731: dread d6/d10/d75/d8f/fc5 [0,4194304] 0 2024-09-16T00:26:06.651 INFO:tasks.workunit.client.0.smithi112.stdout:9/931: chown d1/ff4 32 1 2024-09-16T00:26:06.652 INFO:tasks.workunit.client.1.smithi169.stdout:3/533: dwrite d7/d50/f87 [4194304,4194304] 0 2024-09-16T00:26:06.652 INFO:tasks.workunit.client.1.smithi169.stdout:3/534: rename d7/d50/d90/dba to d7/d50/d90/dba/dca 22 2024-09-16T00:26:06.656 INFO:tasks.workunit.client.0.smithi112.stdout:9/932: creat d1/d5/d20/d81/f149 x:0 0 0 2024-09-16T00:26:06.657 INFO:tasks.workunit.client.1.smithi169.stdout:3/535: creat d7/d1a/d1c/d35/d63/fcb x:0 0 0 2024-09-16T00:26:06.658 INFO:tasks.workunit.client.0.smithi112.stdout:1/783: dwrite d0/d14/df/d28/d45/d46/f6d [0,4194304] 0 2024-09-16T00:26:06.662 INFO:tasks.workunit.client.0.smithi112.stdout:1/784: mkdir d0/d14/d5e/df8 0 2024-09-16T00:26:06.665 INFO:tasks.workunit.client.1.smithi169.stdout:3/536: dread d7/d1a/d1c/f40 [0,4194304] 0 2024-09-16T00:26:06.666 INFO:tasks.workunit.client.0.smithi112.stdout:1/785: dread d0/d14/df/d28/d45/d46/da7/fbc [0,4194304] 0 2024-09-16T00:26:06.668 INFO:tasks.workunit.client.0.smithi112.stdout:7/732: getdents d6/d10/d16/d49/d6c 0 2024-09-16T00:26:06.669 INFO:tasks.workunit.client.1.smithi169.stdout:3/537: mkdir d7/d1a/d1c/d23/d88/d9b/dc8/dcc 0 2024-09-16T00:26:06.671 INFO:tasks.workunit.client.1.smithi169.stdout:3/538: mknod d7/ccd 0 2024-09-16T00:26:06.672 INFO:tasks.workunit.client.0.smithi112.stdout:7/733: creat d6/d10/d16/d49/ff3 x:0 0 0 2024-09-16T00:26:06.672 INFO:tasks.workunit.client.1.smithi169.stdout:8/522: fsync d7/dc/d17/d24/d32/d4d/faa 0 2024-09-16T00:26:06.673 INFO:tasks.workunit.client.1.smithi169.stdout:5/471: write d2/da/d10/d13/d1c/d2d/f57 [1013145,105090] 0 2024-09-16T00:26:06.676 INFO:tasks.workunit.client.0.smithi112.stdout:7/734: rename d6/d10/f3d to d6/d10/d75/d8f/d90/dd3/ff4 0 2024-09-16T00:26:06.676 INFO:tasks.workunit.client.0.smithi112.stdout:7/735: chown d6/d10/d16/d96/c9e 1654664 1 2024-09-16T00:26:06.677 INFO:tasks.workunit.client.0.smithi112.stdout:7/736: read d6/d10/f6e [2149737,6726] 0 2024-09-16T00:26:06.678 INFO:tasks.workunit.client.0.smithi112.stdout:7/737: write d6/d10/d16/d96/fe2 [798044,44274] 0 2024-09-16T00:26:06.679 INFO:tasks.workunit.client.1.smithi169.stdout:5/472: creat d2/da/d10/d13/d1c/d32/d53/d83/fa4 x:0 0 0 2024-09-16T00:26:06.680 INFO:tasks.workunit.client.1.smithi169.stdout:3/539: dwrite d7/d1a/d1c/d35/d63/fcb [0,4194304] 0 2024-09-16T00:26:06.680 INFO:tasks.workunit.client.0.smithi112.stdout:7/738: truncate d6/d10/d16/d49/d6c/ff2 962175 0 2024-09-16T00:26:06.680 INFO:tasks.workunit.client.0.smithi112.stdout:7/739: stat d6/d10/f6e 0 2024-09-16T00:26:06.683 INFO:tasks.workunit.client.1.smithi169.stdout:3/540: dread - d7/d1a/d1c/d35/d63/f9c zero size 2024-09-16T00:26:06.683 INFO:tasks.workunit.client.0.smithi112.stdout:7/740: link d6/d10/da1/dcd/cd0 d6/d10/d75/d88/cf5 0 2024-09-16T00:26:06.684 INFO:tasks.workunit.client.1.smithi169.stdout:8/523: dwrite d7/fa [4194304,4194304] 0 2024-09-16T00:26:06.684 INFO:tasks.workunit.client.1.smithi169.stdout:5/473: unlink d2/da/c95 0 2024-09-16T00:26:06.688 INFO:tasks.workunit.client.0.smithi112.stdout:7/741: dread d6/d7f/f89 [0,4194304] 0 2024-09-16T00:26:06.688 INFO:tasks.workunit.client.0.smithi112.stdout:7/742: mkdir d6/d7f/df6 0 2024-09-16T00:26:06.688 INFO:tasks.workunit.client.1.smithi169.stdout:3/541: read d7/d1a/d1c/d23/f3d [311290,128987] 0 2024-09-16T00:26:06.689 INFO:tasks.workunit.client.0.smithi112.stdout:7/743: readlink d6/d10/d75/d8f/d90/dbc/lc0 0 2024-09-16T00:26:06.690 INFO:tasks.workunit.client.0.smithi112.stdout:7/744: dread - d6/d10/d16/d96/fcc zero size 2024-09-16T00:26:06.690 INFO:tasks.workunit.client.1.smithi169.stdout:5/474: creat d2/da/d10/d13/d1c/d32/d53/d4e/d89/d9a/fa5 x:0 0 0 2024-09-16T00:26:06.692 INFO:tasks.workunit.client.1.smithi169.stdout:8/524: dwrite d7/dc/d17/d48/f60 [4194304,4194304] 0 2024-09-16T00:26:06.695 INFO:tasks.workunit.client.1.smithi169.stdout:8/525: mkdir d7/dc/d26/dab 0 2024-09-16T00:26:06.697 INFO:tasks.workunit.client.0.smithi112.stdout:7/745: dwrite d6/d10/f15 [4194304,4194304] 0 2024-09-16T00:26:06.697 INFO:tasks.workunit.client.1.smithi169.stdout:8/526: write d7/dc/d1d/d80/f8b [4809050,26973] 0 2024-09-16T00:26:06.699 INFO:tasks.workunit.client.0.smithi112.stdout:7/746: write d6/d10/f1d [3537327,35078] 0 2024-09-16T00:26:06.701 INFO:tasks.workunit.client.0.smithi112.stdout:7/747: readlink d6/d10/da1/dcd/ld9 0 2024-09-16T00:26:06.702 INFO:tasks.workunit.client.0.smithi112.stdout:9/933: dread d1/d5/d6/d114/d116/f13f [0,4194304] 0 2024-09-16T00:26:06.705 INFO:tasks.workunit.client.0.smithi112.stdout:7/748: symlink d6/d10/d75/d8f/d90/dbc/lf7 0 2024-09-16T00:26:06.708 INFO:tasks.workunit.client.1.smithi169.stdout:8/527: dwrite d7/dc/d26/d41/fa4 [0,4194304] 0 2024-09-16T00:26:06.709 INFO:tasks.workunit.client.0.smithi112.stdout:1/786: truncate d0/d14/df/d28/d45/d46/f75 488597 0 2024-09-16T00:26:06.710 INFO:tasks.workunit.client.0.smithi112.stdout:7/749: mkdir d6/d7f/dc3/df8 0 2024-09-16T00:26:06.711 INFO:tasks.workunit.client.1.smithi169.stdout:8/528: mkdir d7/dc/d53/da0/dac 0 2024-09-16T00:26:06.712 INFO:tasks.workunit.client.0.smithi112.stdout:7/750: fsync d6/d10/f87 0 2024-09-16T00:26:06.712 INFO:tasks.workunit.client.0.smithi112.stdout:9/934: dwrite d1/d5/d20/d81/d1a/d88/da8/f105 [0,4194304] 0 2024-09-16T00:26:06.713 INFO:tasks.workunit.client.1.smithi169.stdout:8/529: write d7/dc/d17/d24/d32/f6f [984942,23936] 0 2024-09-16T00:26:06.715 INFO:tasks.workunit.client.1.smithi169.stdout:8/530: mkdir d7/dc/d17/d24/d32/d8d/dad 0 2024-09-16T00:26:06.717 INFO:tasks.workunit.client.1.smithi169.stdout:8/531: write d7/f2e [921511,32283] 0 2024-09-16T00:26:06.717 INFO:tasks.workunit.client.0.smithi112.stdout:1/787: readlink d0/d14/df/d28/d45/dd4/d41/de0/ld6 0 2024-09-16T00:26:06.717 INFO:tasks.workunit.client.0.smithi112.stdout:9/935: creat d1/d5/d6/d114/d116/f14a x:0 0 0 2024-09-16T00:26:06.717 INFO:tasks.workunit.client.1.smithi169.stdout:8/532: stat d7/dc/d17/f86 0 2024-09-16T00:26:06.721 INFO:tasks.workunit.client.1.smithi169.stdout:8/533: symlink d7/dc/d53/da0/dac/lae 0 2024-09-16T00:26:06.721 INFO:tasks.workunit.client.0.smithi112.stdout:9/936: unlink d1/d5/fa6 0 2024-09-16T00:26:06.721 INFO:tasks.workunit.client.0.smithi112.stdout:1/788: mkdir d0/d14/df/d28/d45/d46/d58/df9 0 2024-09-16T00:26:06.722 INFO:tasks.workunit.client.1.smithi169.stdout:5/475: write d2/fe [1287359,64037] 0 2024-09-16T00:26:06.724 INFO:tasks.workunit.client.1.smithi169.stdout:5/476: creat d2/da/d10/d13/d1c/d32/d53/d83/fa6 x:0 0 0 2024-09-16T00:26:06.725 INFO:tasks.workunit.client.0.smithi112.stdout:9/937: dread d1/d5/d6/d107/f143 [0,4194304] 0 2024-09-16T00:26:06.725 INFO:tasks.workunit.client.0.smithi112.stdout:1/789: rename d0/d14/df/d28/d45/d46/d58/f99 to d0/d14/d6a/ffa 0 2024-09-16T00:26:06.727 INFO:tasks.workunit.client.0.smithi112.stdout:9/938: fdatasync d1/d5/d20/d81/d1a/d88/def/fe0 0 2024-09-16T00:26:06.727 INFO:tasks.workunit.client.1.smithi169.stdout:8/534: dwrite d7/dc/d2a/f42 [0,4194304] 0 2024-09-16T00:26:06.729 INFO:tasks.workunit.client.1.smithi169.stdout:5/477: truncate d2/da/d10/d13/d1c/f6a 1431990 0 2024-09-16T00:26:06.731 INFO:tasks.workunit.client.1.smithi169.stdout:3/542: dread d7/d1a/d1c/d6c/f7c [0,4194304] 0 2024-09-16T00:26:06.732 INFO:tasks.workunit.client.1.smithi169.stdout:8/535: creat d7/dc/d1d/d3c/d77/faf x:0 0 0 2024-09-16T00:26:06.734 INFO:tasks.workunit.client.0.smithi112.stdout:1/790: dwrite d0/d14/df/d28/d42/f83 [0,4194304] 0 2024-09-16T00:26:06.735 INFO:tasks.workunit.client.1.smithi169.stdout:3/543: mknod d7/d1a/d1c/d35/d63/d37/cce 0 2024-09-16T00:26:06.735 INFO:tasks.workunit.client.1.smithi169.stdout:8/536: creat d7/dc/d17/d24/fb0 x:0 0 0 2024-09-16T00:26:06.740 INFO:tasks.workunit.client.0.smithi112.stdout:1/791: rename d0/d14/d44/l4d to d0/d14/d6a/lfb 0 2024-09-16T00:26:06.741 INFO:tasks.workunit.client.1.smithi169.stdout:8/537: creat d7/dc/d1d/d3c/d82/fb1 x:0 0 0 2024-09-16T00:26:06.741 INFO:tasks.workunit.client.1.smithi169.stdout:5/478: dread d2/da/d10/d13/d1c/f3b [0,4194304] 0 2024-09-16T00:26:06.743 INFO:tasks.workunit.client.1.smithi169.stdout:8/538: dread - d7/dc/d17/d24/fb0 zero size 2024-09-16T00:26:06.743 INFO:tasks.workunit.client.1.smithi169.stdout:5/479: write d2/da/d10/f7a [128823,55586] 0 2024-09-16T00:26:06.745 INFO:tasks.workunit.client.0.smithi112.stdout:7/751: write d6/d10/d16/db4/fc4 [70345,25907] 0 2024-09-16T00:26:06.750 INFO:tasks.workunit.client.1.smithi169.stdout:5/480: creat d2/da/d10/d13/d24/fa7 x:0 0 0 2024-09-16T00:26:06.750 INFO:tasks.workunit.client.1.smithi169.stdout:3/544: write d7/f68 [907285,467] 0 2024-09-16T00:26:06.751 INFO:tasks.workunit.client.1.smithi169.stdout:5/481: chown d2/da/l33 102828520 1 2024-09-16T00:26:06.752 INFO:tasks.workunit.client.1.smithi169.stdout:8/539: dread d7/dc/d17/d48/f60 [4194304,4194304] 0 2024-09-16T00:26:06.754 INFO:tasks.workunit.client.1.smithi169.stdout:8/540: dread - d7/dc/d17/d24/f88 zero size 2024-09-16T00:26:06.758 INFO:tasks.workunit.client.1.smithi169.stdout:5/482: dwrite d2/fd [0,4194304] 0 2024-09-16T00:26:06.760 INFO:tasks.workunit.client.1.smithi169.stdout:2/544: sync 2024-09-16T00:26:06.760 INFO:tasks.workunit.client.1.smithi169.stdout:1/508: sync 2024-09-16T00:26:06.763 INFO:tasks.workunit.client.1.smithi169.stdout:8/541: rename d7/dc/d2a/c75 to d7/dc/d26/d41/d67/cb2 0 2024-09-16T00:26:06.765 INFO:tasks.workunit.client.1.smithi169.stdout:1/509: rmdir dc/d2c/d3d 39 2024-09-16T00:26:06.766 INFO:tasks.workunit.client.0.smithi112.stdout:1/792: rmdir d0/d14/df/d28/d45/d46 39 2024-09-16T00:26:06.768 INFO:tasks.workunit.client.1.smithi169.stdout:2/545: symlink d7/d6b/d70/d79/dae/lc1 0 2024-09-16T00:26:06.770 INFO:tasks.workunit.client.1.smithi169.stdout:2/546: fdatasync d7/db/db8/f57 0 2024-09-16T00:26:06.771 INFO:tasks.workunit.client.0.smithi112.stdout:1/793: write d0/d14/df/d28/d45/dd4/d47/d85/fd2 [713738,129287] 0 2024-09-16T00:26:06.771 INFO:tasks.workunit.client.1.smithi169.stdout:1/510: creat dc/d36/d3a/d7f/d93/d72/d73/f95 x:0 0 0 2024-09-16T00:26:06.772 INFO:tasks.workunit.client.0.smithi112.stdout:1/794: dread - d0/d14/d1c/d20/daf/f7d zero size 2024-09-16T00:26:06.773 INFO:tasks.workunit.client.1.smithi169.stdout:8/542: dwrite d7/dc/d17/d24/d32/f6f [0,4194304] 0 2024-09-16T00:26:06.773 INFO:tasks.workunit.client.0.smithi112.stdout:1/795: fsync d0/d14/d1c/d20/fa1 0 2024-09-16T00:26:06.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:06 smithi169 ceph-mon[34653]: pgmap v256: 65 pgs: 65 active+clean; 2.3 GiB data, 8.0 GiB used, 528 GiB / 536 GiB avail; 87 MiB/s rd, 301 MiB/s wr, 514 op/s 2024-09-16T00:26:06.776 INFO:tasks.workunit.client.0.smithi112.stdout:1/796: rmdir d0/d14/df/d28 39 2024-09-16T00:26:06.777 INFO:tasks.workunit.client.1.smithi169.stdout:2/547: mkdir d7/db/d89/dc2 0 2024-09-16T00:26:06.779 INFO:tasks.workunit.client.1.smithi169.stdout:8/543: mkdir d7/dc/d1d/d3c/d82/db3 0 2024-09-16T00:26:06.782 INFO:tasks.workunit.client.1.smithi169.stdout:8/544: fdatasync f6 0 2024-09-16T00:26:06.783 INFO:tasks.workunit.client.1.smithi169.stdout:3/545: dwrite d7/d1a/d2b/f46 [0,4194304] 0 2024-09-16T00:26:06.785 INFO:tasks.workunit.client.0.smithi112.stdout:1/797: dwrite d0/d14/df/d28/d45/dd4/d47/d85/fd2 [4194304,4194304] 0 2024-09-16T00:26:06.785 INFO:tasks.workunit.client.0.smithi112.stdout:7/752: dread d6/d10/d16/d49/d6c/f36 [0,4194304] 0 2024-09-16T00:26:06.786 INFO:tasks.workunit.client.1.smithi169.stdout:3/546: stat d7/d50/d6f/c99 0 2024-09-16T00:26:06.786 INFO:tasks.workunit.client.1.smithi169.stdout:1/511: dwrite dc/d36/d3a/d7f/d93/f6e [0,4194304] 0 2024-09-16T00:26:06.792 INFO:tasks.workunit.client.1.smithi169.stdout:1/512: getdents dc/d36/d3a/d45 0 2024-09-16T00:26:06.802 INFO:tasks.workunit.client.0.smithi112.stdout:7/753: dwrite d6/d10/d16/d96/dc7/f47 [0,4194304] 0 2024-09-16T00:26:06.803 INFO:tasks.workunit.client.1.smithi169.stdout:8/545: write f6 [1223727,27942] 0 2024-09-16T00:26:06.804 INFO:tasks.workunit.client.1.smithi169.stdout:2/548: truncate d7/db/f45 6483347 0 2024-09-16T00:26:06.804 INFO:tasks.workunit.client.0.smithi112.stdout:7/754: creat d6/d10/d75/d8f/d90/dd3/ff9 x:0 0 0 2024-09-16T00:26:06.805 INFO:tasks.workunit.client.1.smithi169.stdout:3/547: truncate d7/d1a/d1c/d6c/f7c 306259 0 2024-09-16T00:26:06.806 INFO:tasks.workunit.client.1.smithi169.stdout:2/549: write d7/d1c/d26/f76 [3860375,59375] 0 2024-09-16T00:26:06.807 INFO:tasks.workunit.client.0.smithi112.stdout:7/755: rmdir d6/d10/d16/d96/dc7/dbf 39 2024-09-16T00:26:06.808 INFO:tasks.workunit.client.0.smithi112.stdout:7/756: stat d6/d10/d16/d3b/f5e 0 2024-09-16T00:26:06.809 INFO:tasks.workunit.client.1.smithi169.stdout:3/548: unlink d7/d1a/d1c/d23/laf 0 2024-09-16T00:26:06.811 INFO:tasks.workunit.client.1.smithi169.stdout:8/546: truncate d7/dc/d26/d46/f6b 3026234 0 2024-09-16T00:26:06.811 INFO:tasks.workunit.client.1.smithi169.stdout:2/550: mkdir d7/d1c/dc3 0 2024-09-16T00:26:06.812 INFO:tasks.workunit.client.1.smithi169.stdout:2/551: chown d7/db/d20/f83 1958291 1 2024-09-16T00:26:06.814 INFO:tasks.workunit.client.1.smithi169.stdout:8/547: mkdir d7/dc/d1d/d3c/d82/db3/db4 0 2024-09-16T00:26:06.815 INFO:tasks.workunit.client.1.smithi169.stdout:8/548: chown d7/d37/f9b 213539 1 2024-09-16T00:26:06.815 INFO:tasks.workunit.client.1.smithi169.stdout:2/552: mknod d7/d1c/d26/d96/dab/cc4 0 2024-09-16T00:26:06.818 INFO:tasks.workunit.client.1.smithi169.stdout:8/549: mknod d7/dc/d17/d24/cb5 0 2024-09-16T00:26:06.819 INFO:tasks.workunit.client.1.smithi169.stdout:8/550: write d7/dc/d26/d41/f5c [584027,118063] 0 2024-09-16T00:26:06.822 INFO:tasks.workunit.client.1.smithi169.stdout:8/551: read d7/d20/f7e [2458602,118305] 0 2024-09-16T00:26:06.822 INFO:tasks.workunit.client.1.smithi169.stdout:8/552: readlink d7/d20/l43 0 2024-09-16T00:26:06.824 INFO:tasks.workunit.client.1.smithi169.stdout:8/553: rename d7/dc/d17/d24/l9c to d7/d37/lb6 0 2024-09-16T00:26:06.827 INFO:tasks.workunit.client.0.smithi112.stdout:1/798: dread d0/d14/df/d28/f5d [0,4194304] 0 2024-09-16T00:26:06.836 INFO:tasks.workunit.client.0.smithi112.stdout:7/757: dwrite d6/d10/d16/d96/dc7/f24 [0,4194304] 0 2024-09-16T00:26:06.837 INFO:tasks.workunit.client.1.smithi169.stdout:8/554: truncate d7/f2e 3282696 0 2024-09-16T00:26:06.838 INFO:tasks.workunit.client.0.smithi112.stdout:1/799: truncate d0/f67 4576302 0 2024-09-16T00:26:06.838 INFO:tasks.workunit.client.0.smithi112.stdout:3/745: sync 2024-09-16T00:26:06.838 INFO:tasks.workunit.client.0.smithi112.stdout:0/789: sync 2024-09-16T00:26:06.838 INFO:tasks.workunit.client.0.smithi112.stdout:2/818: sync 2024-09-16T00:26:06.839 INFO:tasks.workunit.client.0.smithi112.stdout:8/651: sync 2024-09-16T00:26:06.839 INFO:tasks.workunit.client.1.smithi169.stdout:2/553: write d7/db/f45 [1954148,70524] 0 2024-09-16T00:26:06.840 INFO:tasks.workunit.client.0.smithi112.stdout:1/800: chown d0/d14/df/c4c 424832935 1 2024-09-16T00:26:06.841 INFO:tasks.workunit.client.0.smithi112.stdout:8/652: dread - d10/d12/d24/fcc zero size 2024-09-16T00:26:06.843 INFO:tasks.workunit.client.1.smithi169.stdout:3/549: dread d7/d1a/d1c/d23/f89 [0,4194304] 0 2024-09-16T00:26:06.844 INFO:tasks.workunit.client.0.smithi112.stdout:0/790: symlink d5/d90/dc5/dd7/lef 0 2024-09-16T00:26:06.844 INFO:tasks.workunit.client.1.smithi169.stdout:8/555: getdents d7/dc/d53/da0 0 2024-09-16T00:26:06.845 INFO:tasks.workunit.client.1.smithi169.stdout:3/550: fsync d7/d1a/d1c/d23/f3d 0 2024-09-16T00:26:06.846 INFO:tasks.workunit.client.0.smithi112.stdout:8/653: read d10/d12/d1c/dd0/f80 [408849,10516] 0 2024-09-16T00:26:06.846 INFO:tasks.workunit.client.0.smithi112.stdout:0/791: read - d5/da/d21/fb9 zero size 2024-09-16T00:26:06.847 INFO:tasks.workunit.client.0.smithi112.stdout:1/801: symlink d0/d14/df/d28/d42/lfc 0 2024-09-16T00:26:06.847 INFO:tasks.workunit.client.1.smithi169.stdout:3/551: dread - d7/d1a/d1c/d35/fac zero size 2024-09-16T00:26:06.848 INFO:tasks.workunit.client.1.smithi169.stdout:8/556: mknod d7/dc/d26/d41/d92/cb7 0 2024-09-16T00:26:06.850 INFO:tasks.workunit.client.0.smithi112.stdout:7/758: dwrite d6/d10/d16/faf [0,4194304] 0 2024-09-16T00:26:06.850 INFO:tasks.workunit.client.1.smithi169.stdout:8/557: mknod d7/dc/d1d/cb8 0 2024-09-16T00:26:06.851 INFO:tasks.workunit.client.0.smithi112.stdout:3/746: dwrite d2/d9/d2b/fa0 [0,4194304] 0 2024-09-16T00:26:06.852 INFO:tasks.workunit.client.1.smithi169.stdout:8/558: write d7/d20/f7d [298543,569] 0 2024-09-16T00:26:06.854 INFO:tasks.workunit.client.0.smithi112.stdout:8/654: symlink d10/d12/d1c/d3b/d74/ld9 0 2024-09-16T00:26:06.864 INFO:tasks.workunit.client.1.smithi169.stdout:3/552: dwrite d7/d1a/d1c/d23/f3a [8388608,4194304] 0 2024-09-16T00:26:06.866 INFO:tasks.workunit.client.1.smithi169.stdout:8/559: dwrite d7/dc/d17/d24/f4b [0,4194304] 0 2024-09-16T00:26:06.866 INFO:tasks.workunit.client.1.smithi169.stdout:3/553: readlink d7/d50/d90/lb0 0 2024-09-16T00:26:06.868 INFO:tasks.workunit.client.0.smithi112.stdout:0/792: dwrite d5/da/d66/fc9 [0,4194304] 0 2024-09-16T00:26:06.868 INFO:tasks.workunit.client.0.smithi112.stdout:7/759: link d6/d10/d75/d8f/d90/lb7 d6/d10/d75/d88/ddf/lfa 0 2024-09-16T00:26:06.870 INFO:tasks.workunit.client.0.smithi112.stdout:8/655: link d10/d12/d1c/d61/d6c/c83 d10/d12/d1c/d61/d6c/dd1/d8c/d7c/da5/cda 0 2024-09-16T00:26:06.870 INFO:tasks.workunit.client.0.smithi112.stdout:7/760: chown d6/d10/f6b 124 1 2024-09-16T00:26:06.871 INFO:tasks.workunit.client.1.smithi169.stdout:3/554: creat d7/d1a/d1c/d23/d70/dc5/fcf x:0 0 0 2024-09-16T00:26:06.872 INFO:tasks.workunit.client.1.smithi169.stdout:3/555: chown d7/d1a/d1c/fb6 42409389 1 2024-09-16T00:26:06.873 INFO:tasks.workunit.client.0.smithi112.stdout:3/747: getdents d2/d9/d2b/d47/d52/d7a/d80/d95 0 2024-09-16T00:26:06.873 INFO:tasks.workunit.client.0.smithi112.stdout:7/761: symlink d6/d7f/d85/lfb 0 2024-09-16T00:26:06.873 INFO:tasks.workunit.client.0.smithi112.stdout:1/802: dread d0/d14/d1c/f51 [0,4194304] 0 2024-09-16T00:26:06.874 INFO:tasks.workunit.client.1.smithi169.stdout:3/556: write d7/d1a/d2b/fc3 [727660,53320] 0 2024-09-16T00:26:06.875 INFO:tasks.workunit.client.0.smithi112.stdout:0/793: symlink d5/da/d57/lf0 0 2024-09-16T00:26:06.876 INFO:tasks.workunit.client.1.smithi169.stdout:3/557: dread - d7/d1a/d1c/d23/d70/f98 zero size 2024-09-16T00:26:06.877 INFO:tasks.workunit.client.0.smithi112.stdout:0/794: read - d5/da/d21/fe4 zero size 2024-09-16T00:26:06.877 INFO:tasks.workunit.client.0.smithi112.stdout:8/656: link fe d10/d12/d1c/d61/d6c/fdb 0 2024-09-16T00:26:06.879 INFO:tasks.workunit.client.1.smithi169.stdout:3/558: dread - d7/d50/f7b zero size 2024-09-16T00:26:06.880 INFO:tasks.workunit.client.0.smithi112.stdout:1/803: write d0/d14/df/d28/d45/d46/d58/d62/fa0 [590775,17880] 0 2024-09-16T00:26:06.881 INFO:tasks.workunit.client.0.smithi112.stdout:7/762: creat d6/ffc x:0 0 0 2024-09-16T00:26:06.888 INFO:tasks.workunit.client.1.smithi169.stdout:6/700: sync 2024-09-16T00:26:06.888 INFO:tasks.workunit.client.1.smithi169.stdout:7/585: sync 2024-09-16T00:26:06.889 INFO:tasks.workunit.client.0.smithi112.stdout:7/763: link d6/d10/d16/fe9 d6/d10/da1/da2/ffd 0 2024-09-16T00:26:06.892 INFO:tasks.workunit.client.1.smithi169.stdout:6/701: fdatasync d2/d3f/d41/d5c/f65 0 2024-09-16T00:26:06.892 INFO:tasks.workunit.client.1.smithi169.stdout:7/586: dread - d7/d5b/d61/d98/fb7 zero size 2024-09-16T00:26:06.892 INFO:tasks.workunit.client.1.smithi169.stdout:7/587: readlink d7/l64 0 2024-09-16T00:26:06.892 INFO:tasks.workunit.client.1.smithi169.stdout:6/702: chown d2/d3f/d95/l96 6880 1 2024-09-16T00:26:06.892 INFO:tasks.workunit.client.1.smithi169.stdout:7/588: fsync d7/d71/f9d 0 2024-09-16T00:26:06.893 INFO:tasks.workunit.client.1.smithi169.stdout:6/703: stat d2/c12 0 2024-09-16T00:26:06.893 INFO:tasks.workunit.client.0.smithi112.stdout:7/764: readlink d6/lf 0 2024-09-16T00:26:06.893 INFO:tasks.workunit.client.0.smithi112.stdout:2/819: dread d2/dd/d48/d22/d2b/d54/d60/daf/fc1 [0,4194304] 0 2024-09-16T00:26:06.893 INFO:tasks.workunit.client.0.smithi112.stdout:7/765: chown d6/fa0 702706 1 2024-09-16T00:26:06.893 INFO:tasks.workunit.client.0.smithi112.stdout:7/766: chown d6/d10/d16 265223 1 2024-09-16T00:26:06.894 INFO:tasks.workunit.client.1.smithi169.stdout:6/704: fsync d2/d4d/d59/d38/f4f 0 2024-09-16T00:26:06.894 INFO:tasks.workunit.client.1.smithi169.stdout:9/567: sync 2024-09-16T00:26:06.894 INFO:tasks.workunit.client.1.smithi169.stdout:0/525: sync 2024-09-16T00:26:06.894 INFO:tasks.workunit.client.1.smithi169.stdout:4/512: sync 2024-09-16T00:26:06.896 INFO:tasks.workunit.client.1.smithi169.stdout:6/705: mkdir d2/d3f/d41/d5c/d91/da6/de2 0 2024-09-16T00:26:06.897 INFO:tasks.workunit.client.0.smithi112.stdout:7/767: dread d6/d10/f6e [0,4194304] 0 2024-09-16T00:26:06.897 INFO:tasks.workunit.client.0.smithi112.stdout:2/820: dwrite d2/dd/d48/d22/d2b/d54/d79/f7a [0,4194304] 0 2024-09-16T00:26:06.898 INFO:tasks.workunit.client.1.smithi169.stdout:0/526: rmdir d0/d2/d4b 39 2024-09-16T00:26:06.898 INFO:tasks.workunit.client.1.smithi169.stdout:9/568: write d5/d12/f9c [906821,9148] 0 2024-09-16T00:26:06.899 INFO:tasks.workunit.client.0.smithi112.stdout:7/768: dread - d6/d10/d16/d49/fce zero size 2024-09-16T00:26:06.899 INFO:tasks.workunit.client.1.smithi169.stdout:7/589: link d7/df/l5e d7/d71/d76/d29/d78/dba/lbd 0 2024-09-16T00:26:06.900 INFO:tasks.workunit.client.1.smithi169.stdout:6/706: creat d2/d36/d48/fe3 x:0 0 0 2024-09-16T00:26:06.901 INFO:tasks.workunit.client.1.smithi169.stdout:0/527: creat d0/d2c/d38/fba x:0 0 0 2024-09-16T00:26:06.901 INFO:tasks.workunit.client.0.smithi112.stdout:6/664: sync 2024-09-16T00:26:06.901 INFO:tasks.workunit.client.0.smithi112.stdout:5/639: sync 2024-09-16T00:26:06.901 INFO:tasks.workunit.client.0.smithi112.stdout:4/768: sync 2024-09-16T00:26:06.902 INFO:tasks.workunit.client.1.smithi169.stdout:4/513: mkdir d2/d12/d9c/da4 0 2024-09-16T00:26:06.902 INFO:tasks.workunit.client.0.smithi112.stdout:7/769: fsync d6/d10/d16/f1e 0 2024-09-16T00:26:06.903 INFO:tasks.workunit.client.0.smithi112.stdout:6/665: readlink dd/df/d10/lb2 0 2024-09-16T00:26:06.903 INFO:tasks.workunit.client.0.smithi112.stdout:2/821: read d2/dd/d48/d94/d49/f71 [77720,86401] 0 2024-09-16T00:26:06.903 INFO:tasks.workunit.client.1.smithi169.stdout:4/514: fdatasync d2/d12/d13/f60 0 2024-09-16T00:26:06.904 INFO:tasks.workunit.client.1.smithi169.stdout:7/590: rename d7/l5f to d7/d71/d76/d29/d78/lbe 0 2024-09-16T00:26:06.905 INFO:tasks.workunit.client.0.smithi112.stdout:7/770: dread - d6/d7f/fe7 zero size 2024-09-16T00:26:06.906 INFO:tasks.workunit.client.1.smithi169.stdout:6/707: unlink d2/da4/cbe 0 2024-09-16T00:26:06.908 INFO:tasks.workunit.client.0.smithi112.stdout:5/640: read d4/dd/f95 [3657283,49812] 0 2024-09-16T00:26:06.909 INFO:tasks.workunit.client.0.smithi112.stdout:8/657: write d10/d12/d1c/d61/d6c/dd1/f72 [603272,113723] 0 2024-09-16T00:26:06.910 INFO:tasks.workunit.client.1.smithi169.stdout:8/560: truncate d7/dc/d17/d24/f27 2222044 0 2024-09-16T00:26:06.912 INFO:tasks.workunit.client.1.smithi169.stdout:8/561: read d7/dc/d26/d46/f5d [2959510,94386] 0 2024-09-16T00:26:06.913 INFO:tasks.workunit.client.1.smithi169.stdout:0/528: dwrite d0/d2c/d54/d21/f2f [0,4194304] 0 2024-09-16T00:26:06.913 INFO:tasks.workunit.client.1.smithi169.stdout:4/515: creat d2/d12/d13/d1f/d3e/d83/fa5 x:0 0 0 2024-09-16T00:26:06.913 INFO:tasks.workunit.client.1.smithi169.stdout:9/569: dread d5/da/d2e/d47/fad [0,4194304] 0 2024-09-16T00:26:06.914 INFO:tasks.workunit.client.0.smithi112.stdout:3/748: dread d2/d9/d2b/d47/d52/d7a/d80/f93 [0,4194304] 0 2024-09-16T00:26:06.914 INFO:tasks.workunit.client.0.smithi112.stdout:7/771: rename d6/d10/d75/d88/f27 to d6/d10/d75/d88/ddf/ffe 0 2024-09-16T00:26:06.915 INFO:tasks.workunit.client.0.smithi112.stdout:8/658: write d10/d12/d24/fcc [1022488,122032] 0 2024-09-16T00:26:06.916 INFO:tasks.workunit.client.0.smithi112.stdout:5/641: mkdir d4/d7/d26/d4c/db3/db6/dd0 0 2024-09-16T00:26:06.918 INFO:tasks.workunit.client.0.smithi112.stdout:1/804: dwrite d0/d14/f22 [0,4194304] 0 2024-09-16T00:26:06.920 INFO:tasks.workunit.client.1.smithi169.stdout:0/529: chown d0/d2c/d54/d56/c53 5728331 1 2024-09-16T00:26:06.922 INFO:tasks.workunit.client.0.smithi112.stdout:1/805: readlink d0/d14/d1c/d20/d93/l96 0 2024-09-16T00:26:06.922 INFO:tasks.workunit.client.0.smithi112.stdout:7/772: mkdir d6/d10/d75/d88/dff 0 2024-09-16T00:26:06.923 INFO:tasks.workunit.client.1.smithi169.stdout:0/530: chown d0/d2c/d4f/lb2 16 1 2024-09-16T00:26:06.923 INFO:tasks.workunit.client.1.smithi169.stdout:6/708: truncate d2/d22/f23 4993191 0 2024-09-16T00:26:06.924 INFO:tasks.workunit.client.0.smithi112.stdout:8/659: truncate d10/f15 375890 0 2024-09-16T00:26:06.925 INFO:tasks.workunit.client.1.smithi169.stdout:0/531: readlink d0/d2/d15/l81 0 2024-09-16T00:26:06.926 INFO:tasks.workunit.client.1.smithi169.stdout:3/559: dwrite d7/d1a/d1c/d35/d63/f6a [0,4194304] 0 2024-09-16T00:26:06.926 INFO:tasks.workunit.client.1.smithi169.stdout:4/516: mkdir d2/d12/d13/d5c/da6 0 2024-09-16T00:26:06.927 INFO:tasks.workunit.client.0.smithi112.stdout:2/822: truncate d2/dd/f12 3539452 0 2024-09-16T00:26:06.928 INFO:tasks.workunit.client.0.smithi112.stdout:7/773: mknod d6/d10/d75/d8f/c100 0 2024-09-16T00:26:06.930 INFO:tasks.workunit.client.0.smithi112.stdout:1/806: read - d0/d14/df/d28/d45/d46/fd3 zero size 2024-09-16T00:26:06.930 INFO:tasks.workunit.client.1.smithi169.stdout:0/532: chown d0/d2c/d54/d56 777201 1 2024-09-16T00:26:06.933 INFO:tasks.workunit.client.0.smithi112.stdout:5/642: dwrite d4/d7/d26/d36/f74 [0,4194304] 0 2024-09-16T00:26:06.933 INFO:tasks.workunit.client.1.smithi169.stdout:9/570: link d5/d12/f96 d5/fca 0 2024-09-16T00:26:06.933 INFO:tasks.workunit.client.0.smithi112.stdout:0/795: dread d5/da/dc/f7d [0,4194304] 0 2024-09-16T00:26:06.934 INFO:tasks.workunit.client.0.smithi112.stdout:3/749: dwrite d2/d9/d2b/d47/d52/f38 [0,4194304] 0 2024-09-16T00:26:06.934 INFO:tasks.workunit.client.0.smithi112.stdout:7/774: write d6/fb2 [153507,84511] 0 2024-09-16T00:26:06.937 INFO:tasks.workunit.client.1.smithi169.stdout:9/571: write d5/da/d21/d23/fa9 [288688,32163] 0 2024-09-16T00:26:06.938 INFO:tasks.workunit.client.1.smithi169.stdout:6/709: mkdir d2/d3f/d41/d72/d9e/de1/de4 0 2024-09-16T00:26:06.938 INFO:tasks.workunit.client.1.smithi169.stdout:3/560: fsync d7/d1a/d1c/d23/d70/fb7 0 2024-09-16T00:26:06.939 INFO:tasks.workunit.client.1.smithi169.stdout:0/533: rename d0/d2c/d38/fba to d0/d7d/fbb 0 2024-09-16T00:26:06.941 INFO:tasks.workunit.client.0.smithi112.stdout:5/643: write d4/d7/d14/fcd [426776,8495] 0 2024-09-16T00:26:06.942 INFO:tasks.workunit.client.0.smithi112.stdout:0/796: chown d5/da/d66/f5d 233450726 1 2024-09-16T00:26:06.942 INFO:tasks.workunit.client.0.smithi112.stdout:1/807: write d0/d14/df/d28/d45/d46/d58/d62/fa0 [51917,91627] 0 2024-09-16T00:26:06.946 INFO:tasks.workunit.client.1.smithi169.stdout:7/591: dwrite d7/f67 [0,4194304] 0 2024-09-16T00:26:06.948 INFO:tasks.workunit.client.1.smithi169.stdout:3/561: write d7/d50/d90/dba/fbf [69863,110721] 0 2024-09-16T00:26:06.951 INFO:tasks.workunit.client.1.smithi169.stdout:9/572: rename d5/da/d21/d8d to d5/da/d5b/d6c/dcb 0 2024-09-16T00:26:06.953 INFO:tasks.workunit.client.1.smithi169.stdout:7/592: fdatasync d7/d71/f9d 0 2024-09-16T00:26:06.954 INFO:tasks.workunit.client.1.smithi169.stdout:0/534: rmdir d0/d2/d8/d5/d79/d7c 39 2024-09-16T00:26:06.955 INFO:tasks.workunit.client.1.smithi169.stdout:7/593: fsync d7/df/d73/f8c 0 2024-09-16T00:26:06.955 INFO:tasks.workunit.client.1.smithi169.stdout:3/562: creat d7/d50/d90/dba/fd0 x:0 0 0 2024-09-16T00:26:06.955 INFO:tasks.workunit.client.0.smithi112.stdout:1/808: fsync d0/d14/d1c/f5c 0 2024-09-16T00:26:06.955 INFO:tasks.workunit.client.0.smithi112.stdout:7/775: rename d6/d10/da1 to d6/d10/da1/da2/d101 22 2024-09-16T00:26:06.957 INFO:tasks.workunit.client.1.smithi169.stdout:0/535: creat d0/d2c/d54/d21/d31/fbc x:0 0 0 2024-09-16T00:26:06.958 INFO:tasks.workunit.client.0.smithi112.stdout:6/666: dread dd/df/d13/f1c [0,4194304] 0 2024-09-16T00:26:06.959 INFO:tasks.workunit.client.1.smithi169.stdout:7/594: creat d7/df/d73/d96/fbf x:0 0 0 2024-09-16T00:26:06.963 INFO:tasks.workunit.client.1.smithi169.stdout:7/595: symlink d7/d5b/lc0 0 2024-09-16T00:26:06.967 INFO:tasks.workunit.client.1.smithi169.stdout:9/573: dwrite d5/da/d21/d23/f94 [4194304,4194304] 0 2024-09-16T00:26:06.968 INFO:tasks.workunit.client.1.smithi169.stdout:6/710: dread d2/d22/fda [0,4194304] 0 2024-09-16T00:26:06.972 INFO:tasks.workunit.client.1.smithi169.stdout:0/536: dwrite d0/d50/f70 [0,4194304] 0 2024-09-16T00:26:06.973 INFO:tasks.workunit.client.1.smithi169.stdout:8/562: write d7/dc/d17/d24/f27 [207676,39717] 0 2024-09-16T00:26:06.974 INFO:tasks.workunit.client.0.smithi112.stdout:3/750: dread d2/d9/faa [4194304,4194304] 0 2024-09-16T00:26:06.974 INFO:tasks.workunit.client.0.smithi112.stdout:2/823: mkdir d2/dd/d48/d22/d2b/d54/d60/d6c/dc9/dd5/d108 0 2024-09-16T00:26:06.976 INFO:tasks.workunit.client.1.smithi169.stdout:8/563: stat d7/dc/d1d/l45 0 2024-09-16T00:26:06.976 INFO:tasks.workunit.client.1.smithi169.stdout:4/517: dread d2/f3 [0,4194304] 0 2024-09-16T00:26:06.977 INFO:tasks.workunit.client.1.smithi169.stdout:4/518: chown d2/l5f 13160 1 2024-09-16T00:26:06.978 INFO:tasks.workunit.client.0.smithi112.stdout:0/797: mkdir d5/da/d18/d24/d46/d5f/da6/df1 0 2024-09-16T00:26:06.978 INFO:tasks.workunit.client.1.smithi169.stdout:0/537: mknod d0/d50/d6c/cbd 0 2024-09-16T00:26:06.979 INFO:tasks.workunit.client.1.smithi169.stdout:6/711: symlink d2/d36/le5 0 2024-09-16T00:26:06.979 INFO:tasks.workunit.client.0.smithi112.stdout:1/809: mknod d0/d14/df/d28/d45/dd4/d41/cfd 0 2024-09-16T00:26:06.980 INFO:tasks.workunit.client.0.smithi112.stdout:2/824: dread d2/dd/d48/f85 [0,4194304] 0 2024-09-16T00:26:06.983 INFO:tasks.workunit.client.1.smithi169.stdout:8/564: fsync d7/dc/d1d/d3c/d77/f7b 0 2024-09-16T00:26:06.983 INFO:tasks.workunit.client.1.smithi169.stdout:4/519: mkdir d2/d12/da7 0 2024-09-16T00:26:06.983 INFO:tasks.workunit.client.0.smithi112.stdout:3/751: read - d2/d9/d2b/d2c/fb2 zero size 2024-09-16T00:26:06.985 INFO:tasks.workunit.client.1.smithi169.stdout:9/574: dwrite d5/da/d5b/d6f/faa [0,4194304] 0 2024-09-16T00:26:06.985 INFO:tasks.workunit.client.1.smithi169.stdout:6/712: creat d2/d3f/d41/d5c/d91/da6/de2/fe6 x:0 0 0 2024-09-16T00:26:06.985 INFO:tasks.workunit.client.1.smithi169.stdout:0/538: truncate d0/d2c/d54/d21/d31/fbc 810771 0 2024-09-16T00:26:06.987 INFO:tasks.workunit.client.1.smithi169.stdout:4/520: read - d2/d12/d13/d4d/d59/f75 zero size 2024-09-16T00:26:06.987 INFO:tasks.workunit.client.1.smithi169.stdout:7/596: rename d7/df/d73 to d7/d5b/dc1 0 2024-09-16T00:26:06.988 INFO:tasks.workunit.client.0.smithi112.stdout:1/810: creat d0/d14/df/d28/ffe x:0 0 0 2024-09-16T00:26:06.989 INFO:tasks.workunit.client.0.smithi112.stdout:1/811: rename d0/d14/df to d0/d14/df/d28/d45/d46/dff 22 2024-09-16T00:26:06.989 INFO:tasks.workunit.client.1.smithi169.stdout:8/565: symlink d7/d37/lb9 0 2024-09-16T00:26:06.989 INFO:tasks.workunit.client.1.smithi169.stdout:9/575: dread - d5/da/d5b/d91/db3/f71 zero size 2024-09-16T00:26:06.990 INFO:tasks.workunit.client.0.smithi112.stdout:2/825: mknod d2/dd/d48/d22/d2b/d54/d60/d6c/d88/dad/deb/de1/c109 0 2024-09-16T00:26:06.990 INFO:tasks.workunit.client.1.smithi169.stdout:0/539: creat d0/d2c/d54/d21/d31/fbe x:0 0 0 2024-09-16T00:26:06.991 INFO:tasks.workunit.client.1.smithi169.stdout:4/521: dread - d2/d12/d13/d1f/f6a zero size 2024-09-16T00:26:06.994 INFO:tasks.workunit.client.0.smithi112.stdout:5/644: getdents d4/dd/d6f 0 2024-09-16T00:26:06.994 INFO:tasks.workunit.client.1.smithi169.stdout:0/540: mknod d0/d50/d6c/cbf 0 2024-09-16T00:26:06.994 INFO:tasks.workunit.client.1.smithi169.stdout:4/522: dread - d2/d12/d13/d4d/f84 zero size 2024-09-16T00:26:06.996 INFO:tasks.workunit.client.1.smithi169.stdout:7/597: getdents d7 0 2024-09-16T00:26:06.998 INFO:tasks.workunit.client.1.smithi169.stdout:8/566: rename d7/dc/d17/d24/fb0 to d7/dc/d2a/fba 0 2024-09-16T00:26:06.999 INFO:tasks.workunit.client.0.smithi112.stdout:1/812: creat d0/d14/df/d28/d45/d46/d58/df9/f100 x:0 0 0 2024-09-16T00:26:07.000 INFO:tasks.workunit.client.1.smithi169.stdout:9/576: creat d5/da/d5b/fcc x:0 0 0 2024-09-16T00:26:07.000 INFO:tasks.workunit.client.0.smithi112.stdout:2/826: creat d2/dd/d48/d22/d2b/f10a x:0 0 0 2024-09-16T00:26:07.001 INFO:tasks.workunit.client.0.smithi112.stdout:2/827: fdatasync d2/dd/d48/d22/d67/d9a/fc3 0 2024-09-16T00:26:07.002 INFO:tasks.workunit.client.1.smithi169.stdout:7/598: write d7/d71/d76/d29/f45 [414002,67345] 0 2024-09-16T00:26:07.003 INFO:tasks.workunit.client.1.smithi169.stdout:8/567: write d7/dc/d2a/f30 [3632909,36611] 0 2024-09-16T00:26:07.004 INFO:tasks.workunit.client.0.smithi112.stdout:1/813: read d0/f48 [2220881,122380] 0 2024-09-16T00:26:07.004 INFO:tasks.workunit.client.1.smithi169.stdout:3/563: dread d7/d1a/f32 [0,4194304] 0 2024-09-16T00:26:07.005 INFO:tasks.workunit.client.0.smithi112.stdout:2/828: dread d2/dd/d48/ff2 [0,4194304] 0 2024-09-16T00:26:07.006 INFO:tasks.workunit.client.1.smithi169.stdout:6/713: dwrite d2/d3f/d41/d72/d9e/dc0/fdc [4194304,4194304] 0 2024-09-16T00:26:07.007 INFO:tasks.workunit.client.0.smithi112.stdout:5/645: dread f2 [0,4194304] 0 2024-09-16T00:26:07.010 INFO:tasks.workunit.client.0.smithi112.stdout:5/646: write d4/d7/d14/d90/fb5 [934364,45541] 0 2024-09-16T00:26:07.010 INFO:tasks.workunit.client.1.smithi169.stdout:8/568: rmdir d7/dc/d26/d41/d92 39 2024-09-16T00:26:07.011 INFO:tasks.workunit.client.0.smithi112.stdout:1/814: rename d0/d14/df/d28/d45/d30/f57 to d0/d14/df/d28/d42/dd8/de8/f101 0 2024-09-16T00:26:07.012 INFO:tasks.workunit.client.0.smithi112.stdout:1/815: fdatasync d0/d14/d1c/d20/f33 0 2024-09-16T00:26:07.013 INFO:tasks.workunit.client.1.smithi169.stdout:0/541: dwrite d0/d7d/fa8 [0,4194304] 0 2024-09-16T00:26:07.014 INFO:tasks.workunit.client.1.smithi169.stdout:6/714: read d2/d4d/f57 [306277,9103] 0 2024-09-16T00:26:07.015 INFO:tasks.workunit.client.0.smithi112.stdout:7/776: truncate d6/d10/d16/d96/fe2 574316 0 2024-09-16T00:26:07.015 INFO:tasks.workunit.client.0.smithi112.stdout:6/667: write dd/df/d10/d47/d55/dc4/dd1/f98 [1317664,101018] 0 2024-09-16T00:26:07.017 INFO:tasks.workunit.client.0.smithi112.stdout:1/816: rmdir d0/d14/d1c 39 2024-09-16T00:26:07.019 INFO:tasks.workunit.client.1.smithi169.stdout:6/715: write d2/fb [4180427,103976] 0 2024-09-16T00:26:07.019 INFO:tasks.workunit.client.0.smithi112.stdout:7/777: creat d6/d10/d75/d8f/f102 x:0 0 0 2024-09-16T00:26:07.020 INFO:tasks.workunit.client.0.smithi112.stdout:0/798: truncate d5/da/d1d/d71/f3b 7343776 0 2024-09-16T00:26:07.021 INFO:tasks.workunit.client.0.smithi112.stdout:3/752: write d2/d9/d2b/d47/f54 [815692,32904] 0 2024-09-16T00:26:07.022 INFO:tasks.workunit.client.0.smithi112.stdout:7/778: write d6/d7f/d85/fb5 [195210,92903] 0 2024-09-16T00:26:07.023 INFO:tasks.workunit.client.1.smithi169.stdout:7/599: dwrite d7/d5b/dc1/f8c [0,4194304] 0 2024-09-16T00:26:07.023 INFO:tasks.workunit.client.1.smithi169.stdout:8/569: symlink d7/dc/d1d/d3c/d82/db3/lbb 0 2024-09-16T00:26:07.025 INFO:tasks.workunit.client.1.smithi169.stdout:0/542: unlink d0/d2c/d54/d21/d31/d83/c97 0 2024-09-16T00:26:07.026 INFO:tasks.workunit.client.0.smithi112.stdout:0/799: chown d5/da/d21/dc3/lec 2738 1 2024-09-16T00:26:07.026 INFO:tasks.workunit.client.1.smithi169.stdout:6/716: mknod d2/d4c/ce7 0 2024-09-16T00:26:07.027 INFO:tasks.workunit.client.0.smithi112.stdout:2/829: write d2/dd/d48/d94/d49/f71 [728748,19130] 0 2024-09-16T00:26:07.027 INFO:tasks.workunit.client.1.smithi169.stdout:3/564: rename d7/d1a/d1c/d23/d70/f98 to d7/d1a/fd1 0 2024-09-16T00:26:07.029 INFO:tasks.workunit.client.0.smithi112.stdout:3/753: creat d2/d9/d2b/d47/d52/d7a/d80/d83/d76/d8e/ff6 x:0 0 0 2024-09-16T00:26:07.029 INFO:tasks.workunit.client.1.smithi169.stdout:0/543: write d0/d2c/d54/d21/d31/d83/f9c [175447,116151] 0 2024-09-16T00:26:07.029 INFO:tasks.workunit.client.1.smithi169.stdout:6/717: fsync d2/d4d/f49 0 2024-09-16T00:26:07.030 INFO:tasks.workunit.client.0.smithi112.stdout:1/817: symlink d0/d14/d1c/de9/l102 0 2024-09-16T00:26:07.030 INFO:tasks.workunit.client.1.smithi169.stdout:0/544: write d0/f51 [1233824,29362] 0 2024-09-16T00:26:07.032 INFO:tasks.workunit.client.0.smithi112.stdout:6/668: getdents dd/df/d11/d44 0 2024-09-16T00:26:07.033 INFO:tasks.workunit.client.1.smithi169.stdout:0/545: fsync d0/d2/f6 0 2024-09-16T00:26:07.033 INFO:tasks.workunit.client.1.smithi169.stdout:6/718: symlink d2/d3f/d41/d72/d9e/de1/le8 0 2024-09-16T00:26:07.034 INFO:tasks.workunit.client.1.smithi169.stdout:8/570: link d7/dc/d1d/d3c/d77/f7b d7/dc/d26/fbc 0 2024-09-16T00:26:07.034 INFO:tasks.workunit.client.0.smithi112.stdout:0/800: fsync d5/f23 0 2024-09-16T00:26:07.035 INFO:tasks.workunit.client.0.smithi112.stdout:0/801: readlink d5/l9 0 2024-09-16T00:26:07.036 INFO:tasks.workunit.client.0.smithi112.stdout:7/779: creat d6/d10/d75/d8f/f103 x:0 0 0 2024-09-16T00:26:07.037 INFO:tasks.workunit.client.1.smithi169.stdout:9/577: truncate d5/da/d21/d23/f94 1016470 0 2024-09-16T00:26:07.037 INFO:tasks.workunit.client.0.smithi112.stdout:2/830: creat d2/d68/f10b x:0 0 0 2024-09-16T00:26:07.037 INFO:tasks.workunit.client.0.smithi112.stdout:0/802: readlink d5/da/d1d/d71/d64/db6/ldb 0 2024-09-16T00:26:07.039 INFO:tasks.workunit.client.0.smithi112.stdout:7/780: read d6/d10/d16/d96/f9f [301312,34926] 0 2024-09-16T00:26:07.039 INFO:tasks.workunit.client.1.smithi169.stdout:6/719: truncate d2/d4d/d59/d38/f4f 2085507 0 2024-09-16T00:26:07.041 INFO:tasks.workunit.client.1.smithi169.stdout:7/600: dread d7/f26 [0,4194304] 0 2024-09-16T00:26:07.041 INFO:tasks.workunit.client.1.smithi169.stdout:0/546: symlink d0/lc0 0 2024-09-16T00:26:07.043 INFO:tasks.workunit.client.0.smithi112.stdout:0/803: symlink d5/da/d18/lf2 0 2024-09-16T00:26:07.043 INFO:tasks.workunit.client.1.smithi169.stdout:3/565: dwrite d7/d1a/d1c/d35/d63/d37/f59 [0,4194304] 0 2024-09-16T00:26:07.044 INFO:tasks.workunit.client.0.smithi112.stdout:1/818: getdents d0/d14/df/d28/d45/d46/d58/d62/db0 0 2024-09-16T00:26:07.045 INFO:tasks.workunit.client.0.smithi112.stdout:0/804: readlink d5/da/d21/d6f/l94 0 2024-09-16T00:26:07.047 INFO:tasks.workunit.client.0.smithi112.stdout:5/647: dread d4/f19 [4194304,4194304] 0 2024-09-16T00:26:07.047 INFO:tasks.workunit.client.1.smithi169.stdout:8/571: mkdir d7/dc/d1d/d80/dbd 0 2024-09-16T00:26:07.047 INFO:tasks.workunit.client.1.smithi169.stdout:9/578: creat d5/d49/d64/fcd x:0 0 0 2024-09-16T00:26:07.047 INFO:tasks.workunit.client.0.smithi112.stdout:1/819: mkdir d0/d14/df/d28/d45/d103 0 2024-09-16T00:26:07.048 INFO:tasks.workunit.client.0.smithi112.stdout:5/648: fdatasync d4/d1c/f73 0 2024-09-16T00:26:07.050 INFO:tasks.workunit.client.0.smithi112.stdout:5/649: truncate d4/d4f/fb1 37516 0 2024-09-16T00:26:07.050 INFO:tasks.workunit.client.1.smithi169.stdout:3/566: readlink d7/d1a/d2b/d8d/la1 0 2024-09-16T00:26:07.050 INFO:tasks.workunit.client.1.smithi169.stdout:6/720: dwrite d2/d4d/f8e [0,4194304] 0 2024-09-16T00:26:07.051 INFO:tasks.workunit.client.0.smithi112.stdout:1/820: unlink d0/d14/d1c/d20/f65 0 2024-09-16T00:26:07.054 INFO:tasks.workunit.client.0.smithi112.stdout:6/669: dread dd/df/d10/d47/d55/f7a [0,4194304] 0 2024-09-16T00:26:07.054 INFO:tasks.workunit.client.1.smithi169.stdout:7/601: creat d7/d71/d76/d29/d78/dba/fc2 x:0 0 0 2024-09-16T00:26:07.055 INFO:tasks.workunit.client.1.smithi169.stdout:6/721: write d2/d36/f70 [3760125,99243] 0 2024-09-16T00:26:07.057 INFO:tasks.workunit.client.1.smithi169.stdout:4/523: dread d2/d12/d13/d27/f32 [0,4194304] 0 2024-09-16T00:26:07.057 INFO:tasks.workunit.client.0.smithi112.stdout:1/821: creat d0/d14/df/d28/d45/dd4/d47/d85/f104 x:0 0 0 2024-09-16T00:26:07.058 INFO:tasks.workunit.client.0.smithi112.stdout:6/670: unlink dd/d23/c6f 0 2024-09-16T00:26:07.060 INFO:tasks.workunit.client.0.smithi112.stdout:5/650: fsync d4/d1c/d43/f6d 0 2024-09-16T00:26:07.060 INFO:tasks.workunit.client.0.smithi112.stdout:3/754: rmdir d2/d9/d2b/d47/d52/d7a 39 2024-09-16T00:26:07.062 INFO:tasks.workunit.client.1.smithi169.stdout:4/524: chown d2/d12/d13/d4d/d59/l80 31116895 1 2024-09-16T00:26:07.062 INFO:tasks.workunit.client.0.smithi112.stdout:5/651: write d4/dd/d12/f2f [1144803,12256] 0 2024-09-16T00:26:07.062 INFO:tasks.workunit.client.0.smithi112.stdout:1/822: mknod d0/d14/df/d28/d45/d46/da7/c105 0 2024-09-16T00:26:07.063 INFO:tasks.workunit.client.0.smithi112.stdout:5/652: stat d4/d4f/d78 0 2024-09-16T00:26:07.064 INFO:tasks.workunit.client.1.smithi169.stdout:8/572: dread d7/dc/d17/d24/f27 [0,4194304] 0 2024-09-16T00:26:07.065 INFO:tasks.workunit.client.0.smithi112.stdout:9/939: sync 2024-09-16T00:26:07.066 INFO:tasks.workunit.client.0.smithi112.stdout:6/671: dread dd/df/d13/f1c [0,4194304] 0 2024-09-16T00:26:07.068 INFO:tasks.workunit.client.0.smithi112.stdout:3/755: truncate d2/d9/d2b/d2c/f63 202049 0 2024-09-16T00:26:07.068 INFO:tasks.workunit.client.1.smithi169.stdout:8/573: chown d7/dc/d2a/c3f 31 1 2024-09-16T00:26:07.069 INFO:tasks.workunit.client.1.smithi169.stdout:0/547: write d0/d2c/d54/d21/f49 [1534632,127876] 0 2024-09-16T00:26:07.069 INFO:tasks.workunit.client.1.smithi169.stdout:9/579: creat d5/da/d2e/d47/fce x:0 0 0 2024-09-16T00:26:07.071 INFO:tasks.workunit.client.0.smithi112.stdout:5/653: unlink d4/d4f/d77/dcc/d7c/cb2 0 2024-09-16T00:26:07.071 INFO:tasks.workunit.client.0.smithi112.stdout:5/654: dread - d4/d7/d9b/faf zero size 2024-09-16T00:26:07.072 INFO:tasks.workunit.client.1.smithi169.stdout:8/574: truncate d7/dc/d2a/f42 5221967 0 2024-09-16T00:26:07.072 INFO:tasks.workunit.client.1.smithi169.stdout:4/525: fdatasync d2/d12/d13/d5c/f66 0 2024-09-16T00:26:07.072 INFO:tasks.workunit.client.0.smithi112.stdout:1/823: rename d0/d14/df/d28/d45/d46/d58/ced to d0/d14/d5e/df8/c106 0 2024-09-16T00:26:07.075 INFO:tasks.workunit.client.0.smithi112.stdout:3/756: creat d2/d9/d2b/d47/d78/da9/ff7 x:0 0 0 2024-09-16T00:26:07.076 INFO:tasks.workunit.client.1.smithi169.stdout:0/548: creat d0/d2/d15/fc1 x:0 0 0 2024-09-16T00:26:07.076 INFO:tasks.workunit.client.1.smithi169.stdout:4/526: chown d2/d12/d13/l2a 1 1 2024-09-16T00:26:07.077 INFO:tasks.workunit.client.1.smithi169.stdout:6/722: truncate d2/d4d/d59/d43/f4a 1362789 0 2024-09-16T00:26:07.077 INFO:tasks.workunit.client.1.smithi169.stdout:3/567: write d7/d50/d6f/f71 [3227203,122963] 0 2024-09-16T00:26:07.078 INFO:tasks.workunit.client.0.smithi112.stdout:0/805: truncate d5/da/d21/da5/faa 2699217 0 2024-09-16T00:26:07.078 INFO:tasks.workunit.client.0.smithi112.stdout:0/806: chown d5/da/d21/f7c 6143630 1 2024-09-16T00:26:07.080 INFO:tasks.workunit.client.1.smithi169.stdout:9/580: rename d5/da/d21/d23/c83 to d5/da/d5b/d6c/dcb/ccf 0 2024-09-16T00:26:07.081 INFO:tasks.workunit.client.0.smithi112.stdout:5/655: symlink d4/d7/d26/d4c/db3/db6/dd0/ld1 0 2024-09-16T00:26:07.081 INFO:tasks.workunit.client.1.smithi169.stdout:8/575: symlink d7/dc/d53/da0/dac/lbe 0 2024-09-16T00:26:07.082 INFO:tasks.workunit.client.1.smithi169.stdout:8/576: stat d7/d8c 0 2024-09-16T00:26:07.082 INFO:tasks.workunit.client.1.smithi169.stdout:0/549: fdatasync d0/d2c/d54/f3b 0 2024-09-16T00:26:07.083 INFO:tasks.workunit.client.0.smithi112.stdout:3/757: mknod d2/d9/d2b/d47/d52/d7a/d80/d83/d68/d90/cf8 0 2024-09-16T00:26:07.084 INFO:tasks.workunit.client.0.smithi112.stdout:5/656: mknod d4/dd/d49/da9/cd2 0 2024-09-16T00:26:07.085 INFO:tasks.workunit.client.1.smithi169.stdout:4/527: mknod d2/d12/d13/d1f/d3e/d5b/d70/d7d/ca8 0 2024-09-16T00:26:07.085 INFO:tasks.workunit.client.0.smithi112.stdout:1/824: link d0/d14/df/d28/d45/d30/cce d0/d14/df/d28/d45/d46/c107 0 2024-09-16T00:26:07.086 INFO:tasks.workunit.client.1.smithi169.stdout:4/528: readlink d2/d12/d13/d1f/d3e/d5b/d1a/l22 0 2024-09-16T00:26:07.087 INFO:tasks.workunit.client.0.smithi112.stdout:3/758: rmdir d2/d9/d2b/d47/dcf 39 2024-09-16T00:26:07.087 INFO:tasks.workunit.client.1.smithi169.stdout:9/581: mkdir d5/da/dd0 0 2024-09-16T00:26:07.088 INFO:tasks.workunit.client.1.smithi169.stdout:8/577: creat d7/dc/d1d/d3c/d77/fbf x:0 0 0 2024-09-16T00:26:07.089 INFO:tasks.workunit.client.0.smithi112.stdout:3/759: chown d2/d9/d2b/d47/d52/d7a/d80/d83/d76/d8e/ff0 257 1 2024-09-16T00:26:07.089 INFO:tasks.workunit.client.0.smithi112.stdout:3/760: chown d2 2596826 1 2024-09-16T00:26:07.089 INFO:tasks.workunit.client.1.smithi169.stdout:6/723: symlink d2/da4/db1/dbc/le9 0 2024-09-16T00:26:07.090 INFO:tasks.workunit.client.0.smithi112.stdout:5/657: unlink d4/d7/d87/fce 0 2024-09-16T00:26:07.090 INFO:tasks.workunit.client.0.smithi112.stdout:9/940: write d1/d5/f8b [707534,60089] 0 2024-09-16T00:26:07.091 INFO:tasks.workunit.client.1.smithi169.stdout:8/578: write d7/dc/d26/d41/fa4 [51681,102922] 0 2024-09-16T00:26:07.091 INFO:tasks.workunit.client.0.smithi112.stdout:1/825: symlink d0/d14/df/d28/d45/d46/d58/d62/l108 0 2024-09-16T00:26:07.092 INFO:tasks.workunit.client.1.smithi169.stdout:8/579: chown d7/dc/d1d/d3c 245578503 1 2024-09-16T00:26:07.092 INFO:tasks.workunit.client.1.smithi169.stdout:0/550: write d0/d7d/fbb [614380,66157] 0 2024-09-16T00:26:07.093 INFO:tasks.workunit.client.1.smithi169.stdout:3/568: fdatasync d7/d1a/d1c/d6c/f7c 0 2024-09-16T00:26:07.096 INFO:tasks.workunit.client.1.smithi169.stdout:4/529: rename d2/d12/d13/d4e/f8d to d2/d12/d13/d5c/da6/fa9 0 2024-09-16T00:26:07.096 INFO:tasks.workunit.client.1.smithi169.stdout:3/569: chown d7/d1a/d1c/f54 5 1 2024-09-16T00:26:07.097 INFO:tasks.workunit.client.0.smithi112.stdout:9/941: creat d1/d5/d20/d81/d1a/d88/f14b x:0 0 0 2024-09-16T00:26:07.097 INFO:tasks.workunit.client.1.smithi169.stdout:6/724: symlink d2/d3f/d41/d5c/dac/dbb/lea 0 2024-09-16T00:26:07.099 INFO:tasks.workunit.client.1.smithi169.stdout:6/725: chown d2/da4/db1/ld3 311 1 2024-09-16T00:26:07.099 INFO:tasks.workunit.client.1.smithi169.stdout:0/551: chown d0/d2/lb 23464482 1 2024-09-16T00:26:07.099 INFO:tasks.workunit.client.0.smithi112.stdout:1/826: creat d0/d14/d1c/de9/f109 x:0 0 0 2024-09-16T00:26:07.102 INFO:tasks.workunit.client.0.smithi112.stdout:1/827: chown d0/d14/d1c/d20/d93/fa3 0 1 2024-09-16T00:26:07.103 INFO:tasks.workunit.client.0.smithi112.stdout:5/658: dwrite d4/dd/f6b [4194304,4194304] 0 2024-09-16T00:26:07.104 INFO:tasks.workunit.client.1.smithi169.stdout:9/582: fsync d5/f8 0 2024-09-16T00:26:07.105 INFO:tasks.workunit.client.1.smithi169.stdout:3/570: creat d7/fd2 x:0 0 0 2024-09-16T00:26:07.105 INFO:tasks.workunit.client.0.smithi112.stdout:6/672: dwrite dd/df/d10/d47/d89/dbc/fc5 [0,4194304] 0 2024-09-16T00:26:07.106 INFO:tasks.workunit.client.1.smithi169.stdout:4/530: creat d2/d12/d9c/faa x:0 0 0 2024-09-16T00:26:07.106 INFO:tasks.workunit.client.0.smithi112.stdout:3/761: rmdir d2/d9/d2b/d47/d52/d7a/d80/d83/d68/d90/dcb/ddf 0 2024-09-16T00:26:07.107 INFO:tasks.workunit.client.1.smithi169.stdout:4/531: write d2/d12/d13/d1f/d3e/d5b/d1a/f47 [1524485,91254] 0 2024-09-16T00:26:07.109 INFO:tasks.workunit.client.0.smithi112.stdout:9/942: dread - d1/d5/d6/d2d/d39/fd0 zero size 2024-09-16T00:26:07.111 INFO:tasks.workunit.client.1.smithi169.stdout:0/552: rmdir d0/d2c/d54/d21/d31/d83 39 2024-09-16T00:26:07.112 INFO:tasks.workunit.client.1.smithi169.stdout:8/580: getdents d7/dc/d1d/d80/dbd 0 2024-09-16T00:26:07.112 INFO:tasks.workunit.client.1.smithi169.stdout:0/553: write d0/d50/f7f [98537,60210] 0 2024-09-16T00:26:07.113 INFO:tasks.workunit.client.0.smithi112.stdout:5/659: mkdir d4/d7/d14/d90/dd3 0 2024-09-16T00:26:07.113 INFO:tasks.workunit.client.0.smithi112.stdout:9/943: write d1/d5/d20/d81/d1a/d88/f10f [3575810,49086] 0 2024-09-16T00:26:07.116 INFO:tasks.workunit.client.0.smithi112.stdout:6/673: rmdir dd/d15/da5 39 2024-09-16T00:26:07.118 INFO:tasks.workunit.client.1.smithi169.stdout:3/571: fdatasync d7/d1a/d1c/d23/d70/f8e 0 2024-09-16T00:26:07.120 INFO:tasks.workunit.client.1.smithi169.stdout:0/554: symlink d0/d50/d6c/lc2 0 2024-09-16T00:26:07.120 INFO:tasks.workunit.client.1.smithi169.stdout:3/572: chown d7/d1a/d1c/d35/d63/c82 40 1 2024-09-16T00:26:07.122 INFO:tasks.workunit.client.1.smithi169.stdout:9/583: symlink d5/da/d2e/ld1 0 2024-09-16T00:26:07.122 INFO:tasks.workunit.client.1.smithi169.stdout:4/532: unlink d2/d12/d13/d1f/d3e/d5b/f26 0 2024-09-16T00:26:07.123 INFO:tasks.workunit.client.0.smithi112.stdout:0/807: dread d5/da/d21/da5/faa [0,4194304] 0 2024-09-16T00:26:07.124 INFO:tasks.workunit.client.0.smithi112.stdout:6/674: creat dd/df/d91/fd9 x:0 0 0 2024-09-16T00:26:07.124 INFO:tasks.workunit.client.1.smithi169.stdout:4/533: dread - d2/d12/d13/d1f/d3e/d83/fa5 zero size 2024-09-16T00:26:07.124 INFO:tasks.workunit.client.1.smithi169.stdout:3/573: mkdir d7/d1a/d1c/d23/d88/dd3 0 2024-09-16T00:26:07.125 INFO:tasks.workunit.client.1.smithi169.stdout:0/555: mknod d0/d2c/d54/d21/d31/cc3 0 2024-09-16T00:26:07.125 INFO:tasks.workunit.client.1.smithi169.stdout:4/534: dread - d2/d12/d13/d1f/d3e/d5b/f89 zero size 2024-09-16T00:26:07.125 INFO:tasks.workunit.client.0.smithi112.stdout:6/675: chown dd/df/d10/d47/d55/dc4/dd1/f58 46879 1 2024-09-16T00:26:07.126 INFO:tasks.workunit.client.0.smithi112.stdout:1/828: rename d0/d14/d1c/de9 to d0/d14/df/d28/d45/dd4/d47/d10a 0 2024-09-16T00:26:07.128 INFO:tasks.workunit.client.1.smithi169.stdout:0/556: rmdir d0/d2/d8/d5 39 2024-09-16T00:26:07.133 INFO:tasks.workunit.client.0.smithi112.stdout:9/944: link d1/d5/d20/d81/d1a/d88/f10f d1/d5/d6/df8/f14c 0 2024-09-16T00:26:07.135 INFO:tasks.workunit.client.0.smithi112.stdout:1/829: creat d0/d14/df/d28/d42/dd8/f10b x:0 0 0 2024-09-16T00:26:07.136 INFO:tasks.workunit.client.0.smithi112.stdout:1/830: chown d0/d14/df/d28/d45 4035666 1 2024-09-16T00:26:07.138 INFO:tasks.workunit.client.0.smithi112.stdout:3/762: link d2/f97 d2/d9/d2b/d47/d52/d7a/ff9 0 2024-09-16T00:26:07.140 INFO:tasks.workunit.client.0.smithi112.stdout:9/945: write d1/d5/d6/d2d/d6a/d89/f90 [3611162,61301] 0 2024-09-16T00:26:07.142 INFO:tasks.workunit.client.0.smithi112.stdout:3/763: write d2/d9/faa [290762,54839] 0 2024-09-16T00:26:07.144 INFO:tasks.workunit.client.1.smithi169.stdout:6/726: dread d2/d22/d51/f77 [0,4194304] 0 2024-09-16T00:26:07.144 INFO:tasks.workunit.client.1.smithi169.stdout:9/584: dwrite d5/da/d5b/fac [0,4194304] 0 2024-09-16T00:26:07.146 INFO:tasks.workunit.client.1.smithi169.stdout:8/581: dread d7/d20/f7e [0,4194304] 0 2024-09-16T00:26:07.146 INFO:tasks.workunit.client.0.smithi112.stdout:5/660: dread d4/d7/f40 [0,4194304] 0 2024-09-16T00:26:07.147 INFO:tasks.workunit.client.0.smithi112.stdout:1/831: creat d0/d14/df/d28/d45/d46/da7/dda/f10c x:0 0 0 2024-09-16T00:26:07.149 INFO:tasks.workunit.client.1.smithi169.stdout:3/574: dwrite d7/d1a/d1c/d35/d63/f53 [0,4194304] 0 2024-09-16T00:26:07.152 INFO:tasks.workunit.client.1.smithi169.stdout:4/535: dread d2/d12/d13/d1f/f20 [0,4194304] 0 2024-09-16T00:26:07.153 INFO:tasks.workunit.client.0.smithi112.stdout:1/832: mknod d0/d14/d6a/d9e/c10d 0 2024-09-16T00:26:07.153 INFO:tasks.workunit.client.0.smithi112.stdout:0/808: dwrite d5/da/d57/f73 [0,4194304] 0 2024-09-16T00:26:07.154 INFO:tasks.workunit.client.1.smithi169.stdout:4/536: stat d2/d12/d13/d4d/d59/fa0 0 2024-09-16T00:26:07.155 INFO:tasks.workunit.client.1.smithi169.stdout:3/575: dread d7/d50/d90/dba/fbf [0,4194304] 0 2024-09-16T00:26:07.155 INFO:tasks.workunit.client.1.smithi169.stdout:4/537: dread - d2/d12/d9c/faa zero size 2024-09-16T00:26:07.156 INFO:tasks.workunit.client.0.smithi112.stdout:5/661: rename d4/dd/d6f/f72 to d4/dd/d12/d52/d64/fd4 0 2024-09-16T00:26:07.156 INFO:tasks.workunit.client.0.smithi112.stdout:6/676: dread dd/df/d10/d47/f9c [0,4194304] 0 2024-09-16T00:26:07.158 INFO:tasks.workunit.client.1.smithi169.stdout:9/585: link d5/d12/f9c d5/da/dd0/fd2 0 2024-09-16T00:26:07.159 INFO:tasks.workunit.client.1.smithi169.stdout:6/727: creat d2/d3f/d41/d5c/d91/feb x:0 0 0 2024-09-16T00:26:07.160 INFO:tasks.workunit.client.0.smithi112.stdout:5/662: fdatasync d4/d4f/d78/fa8 0 2024-09-16T00:26:07.160 INFO:tasks.workunit.client.1.smithi169.stdout:8/582: rmdir d7/dc/d17/d24/d32/d8d/d9f 0 2024-09-16T00:26:07.161 INFO:tasks.workunit.client.1.smithi169.stdout:6/728: truncate d2/d22/fc2 135723 0 2024-09-16T00:26:07.162 INFO:tasks.workunit.client.1.smithi169.stdout:4/538: creat d2/d12/d13/d1f/d3e/d5b/d70/d7d/fab x:0 0 0 2024-09-16T00:26:07.162 INFO:tasks.workunit.client.0.smithi112.stdout:0/809: dread d5/da/d66/fc9 [0,4194304] 0 2024-09-16T00:26:07.163 INFO:tasks.workunit.client.1.smithi169.stdout:8/583: chown d7/dc/d17/d48/c56 139 1 2024-09-16T00:26:07.163 INFO:tasks.workunit.client.1.smithi169.stdout:3/576: mknod d7/d1a/d1c/d35/d63/d37/cd4 0 2024-09-16T00:26:07.163 INFO:tasks.workunit.client.0.smithi112.stdout:6/677: creat dd/df/d91/fda x:0 0 0 2024-09-16T00:26:07.165 INFO:tasks.workunit.client.1.smithi169.stdout:9/586: mknod d5/da/d5b/d91/d9a/dc3/cd3 0 2024-09-16T00:26:07.165 INFO:tasks.workunit.client.0.smithi112.stdout:9/946: truncate d1/d76/fdc 2583248 0 2024-09-16T00:26:07.166 INFO:tasks.workunit.client.0.smithi112.stdout:5/663: dread - d4/d7/d14/d90/f93 zero size 2024-09-16T00:26:07.166 INFO:tasks.workunit.client.0.smithi112.stdout:6/678: rmdir dd/d15 39 2024-09-16T00:26:07.168 INFO:tasks.workunit.client.0.smithi112.stdout:0/810: rename d5/da/d21/f51 to d5/da/d57/db0/ff3 0 2024-09-16T00:26:07.168 INFO:tasks.workunit.client.0.smithi112.stdout:9/947: write d1/d5/d6/df8/ddd/f11a [883479,23702] 0 2024-09-16T00:26:07.169 INFO:tasks.workunit.client.1.smithi169.stdout:3/577: rename d7/d1a/d2b/fc9 to d7/d50/fd5 0 2024-09-16T00:26:07.170 INFO:tasks.workunit.client.1.smithi169.stdout:9/587: mknod d5/da/d73/cd4 0 2024-09-16T00:26:07.171 INFO:tasks.workunit.client.0.smithi112.stdout:0/811: creat d5/ff4 x:0 0 0 2024-09-16T00:26:07.171 INFO:tasks.workunit.client.0.smithi112.stdout:9/948: write d1/d5/d6/df8/dba/f135 [971739,73641] 0 2024-09-16T00:26:07.173 INFO:tasks.workunit.client.0.smithi112.stdout:6/679: link dd/df/d10/d47/d55/f72 dd/df/d10/d47/d89/dbc/fdb 0 2024-09-16T00:26:07.174 INFO:tasks.workunit.client.1.smithi169.stdout:3/578: symlink d7/d50/d90/dba/ld6 0 2024-09-16T00:26:07.174 INFO:tasks.workunit.client.1.smithi169.stdout:4/539: dwrite d2/d12/d13/d1f/d3e/f8c [0,4194304] 0 2024-09-16T00:26:07.175 INFO:tasks.workunit.client.0.smithi112.stdout:3/764: dread d2/d9/f1e [0,4194304] 0 2024-09-16T00:26:07.176 INFO:tasks.workunit.client.1.smithi169.stdout:6/729: rename d2/d3f/d41/f46 to d2/d36/fec 0 2024-09-16T00:26:07.177 INFO:tasks.workunit.client.0.smithi112.stdout:9/949: mkdir d1/d5/d6/d2d/d6a/d89/d14d 0 2024-09-16T00:26:07.177 INFO:tasks.workunit.client.0.smithi112.stdout:3/765: write d2/d9/f16 [3218014,27795] 0 2024-09-16T00:26:07.178 INFO:tasks.workunit.client.0.smithi112.stdout:5/664: rename d4/d1c/l5d to d4/dd/ld5 0 2024-09-16T00:26:07.179 INFO:tasks.workunit.client.0.smithi112.stdout:3/766: write d2/d9/d2b/d47/d52/d7a/d80/d83/d68/d90/fdb [90942,124021] 0 2024-09-16T00:26:07.181 INFO:tasks.workunit.client.1.smithi169.stdout:6/730: read d2/d3f/d41/d52/f1e [3280852,41314] 0 2024-09-16T00:26:07.182 INFO:tasks.workunit.client.0.smithi112.stdout:9/950: unlink d1/d5/d20/l58 0 2024-09-16T00:26:07.185 INFO:tasks.workunit.client.0.smithi112.stdout:5/665: read d4/d7/d14/f8c [1375025,12957] 0 2024-09-16T00:26:07.185 INFO:tasks.workunit.client.1.smithi169.stdout:8/584: rename d7/dc/d1d/d3c/d77/faf to d7/dc/d17/d24/d32/fc0 0 2024-09-16T00:26:07.187 INFO:tasks.workunit.client.1.smithi169.stdout:9/588: dwrite d5/da/d5b/d6c/fb1 [0,4194304] 0 2024-09-16T00:26:07.189 INFO:tasks.workunit.client.1.smithi169.stdout:6/731: read d2/d36/d48/f7e [101086,86059] 0 2024-09-16T00:26:07.189 INFO:tasks.workunit.client.0.smithi112.stdout:9/951: mkdir d1/d5/d6/d38/d14e 0 2024-09-16T00:26:07.189 INFO:tasks.workunit.client.0.smithi112.stdout:1/833: dread d0/d14/df/d28/d45/dd4/f8d [0,4194304] 0 2024-09-16T00:26:07.189 INFO:tasks.workunit.client.1.smithi169.stdout:4/540: mkdir d2/d12/da7/dac 0 2024-09-16T00:26:07.190 INFO:tasks.workunit.client.0.smithi112.stdout:3/767: fsync d2/d9/d2b/d47/d52/d7a/d80/d83/d49/f9c 0 2024-09-16T00:26:07.192 INFO:tasks.workunit.client.1.smithi169.stdout:9/589: mknod d5/da/d5b/d6c/dcb/da4/cd5 0 2024-09-16T00:26:07.192 INFO:tasks.workunit.client.1.smithi169.stdout:3/579: dwrite d7/d1a/d1c/d6c/f6d [0,4194304] 0 2024-09-16T00:26:07.193 INFO:tasks.workunit.client.0.smithi112.stdout:1/834: symlink d0/d14/df/d28/l10e 0 2024-09-16T00:26:07.194 INFO:tasks.workunit.client.1.smithi169.stdout:6/732: rmdir d2/d36/d48 39 2024-09-16T00:26:07.194 INFO:tasks.workunit.client.1.smithi169.stdout:4/541: dread - d2/d12/d13/d1f/d3e/d5b/f6b zero size 2024-09-16T00:26:07.195 INFO:tasks.workunit.client.1.smithi169.stdout:9/590: mkdir d5/d49/dc0/dd6 0 2024-09-16T00:26:07.198 INFO:tasks.workunit.client.1.smithi169.stdout:6/733: symlink d2/d22/d51/led 0 2024-09-16T00:26:07.198 INFO:tasks.workunit.client.0.smithi112.stdout:3/768: creat d2/d9/d2b/d47/d52/d7a/d80/d83/d49/ffa x:0 0 0 2024-09-16T00:26:07.198 INFO:tasks.workunit.client.0.smithi112.stdout:0/812: write d5/da/d1d/d71/f4d [2740455,96030] 0 2024-09-16T00:26:07.198 INFO:tasks.workunit.client.1.smithi169.stdout:4/542: dread d2/d12/d13/d1f/f8e [0,4194304] 0 2024-09-16T00:26:07.199 INFO:tasks.workunit.client.1.smithi169.stdout:3/580: chown d7/d1a/d1c/d35/d63/d37/l4b 29 1 2024-09-16T00:26:07.200 INFO:tasks.workunit.client.0.smithi112.stdout:1/835: symlink d0/d14/l10f 0 2024-09-16T00:26:07.201 INFO:tasks.workunit.client.1.smithi169.stdout:1/513: sync 2024-09-16T00:26:07.201 INFO:tasks.workunit.client.1.smithi169.stdout:5/483: sync 2024-09-16T00:26:07.202 INFO:tasks.workunit.client.0.smithi112.stdout:3/769: truncate d2/d9/d2b/d47/d52/d7a/d80/d83/d49/ffa 759699 0 2024-09-16T00:26:07.204 INFO:tasks.workunit.client.1.smithi169.stdout:3/581: write d7/d1a/d1c/d35/d63/d37/f9a [5131620,51451] 0 2024-09-16T00:26:07.204 INFO:tasks.workunit.client.1.smithi169.stdout:1/514: chown dc/d36/d39/d50/d75 382949088 1 2024-09-16T00:26:07.205 INFO:tasks.workunit.client.0.smithi112.stdout:1/836: creat d0/d14/df/d28/d45/dd4/d47/f110 x:0 0 0 2024-09-16T00:26:07.207 INFO:tasks.workunit.client.0.smithi112.stdout:0/813: link d5/da/dc/cd8 d5/da/d1d/d71/d64/dea/cf5 0 2024-09-16T00:26:07.208 INFO:tasks.workunit.client.0.smithi112.stdout:3/770: mkdir d2/d9/dfb 0 2024-09-16T00:26:07.208 INFO:tasks.workunit.client.1.smithi169.stdout:1/515: dread dc/f34 [0,4194304] 0 2024-09-16T00:26:07.209 INFO:tasks.workunit.client.1.smithi169.stdout:6/734: creat d2/d4d/d59/d43/fee x:0 0 0 2024-09-16T00:26:07.209 INFO:tasks.workunit.client.0.smithi112.stdout:6/680: dwrite dd/df/d17/d27/f76 [0,4194304] 0 2024-09-16T00:26:07.212 INFO:tasks.workunit.client.0.smithi112.stdout:6/681: chown dd/df/d10/d47/d89/dbc/fdb 37752 1 2024-09-16T00:26:07.214 INFO:tasks.workunit.client.0.smithi112.stdout:0/814: rename d5/da/d18/c26 to d5/da/d1d/d71/d64/dea/cf6 0 2024-09-16T00:26:07.215 INFO:tasks.workunit.client.1.smithi169.stdout:8/585: write d7/fd [954070,40679] 0 2024-09-16T00:26:07.216 INFO:tasks.workunit.client.0.smithi112.stdout:9/952: dwrite d1/d5/d20/d81/faa [0,4194304] 0 2024-09-16T00:26:07.217 INFO:tasks.workunit.client.1.smithi169.stdout:1/516: rename dc/d26/d90 to dc/d6c/d96 0 2024-09-16T00:26:07.223 INFO:tasks.workunit.client.0.smithi112.stdout:6/682: symlink dd/d15/da5/d79/ldc 0 2024-09-16T00:26:07.225 INFO:tasks.workunit.client.1.smithi169.stdout:9/591: truncate d5/da/f4d 494163 0 2024-09-16T00:26:07.225 INFO:tasks.workunit.client.0.smithi112.stdout:9/953: truncate d1/d5/d6/d18/f112 197612 0 2024-09-16T00:26:07.225 INFO:tasks.workunit.client.0.smithi112.stdout:6/683: truncate dd/d15/da5/d79/fc3 315772 0 2024-09-16T00:26:07.227 INFO:tasks.workunit.client.1.smithi169.stdout:9/592: write d5/da/d5b/d6c/dcb/db4/fc4 [841014,19209] 0 2024-09-16T00:26:07.227 INFO:tasks.workunit.client.1.smithi169.stdout:5/484: write d2/da/f19 [245723,101407] 0 2024-09-16T00:26:07.228 INFO:tasks.workunit.client.0.smithi112.stdout:9/954: write d1/d5/d20/d81/d1a/f7d [1703490,109343] 0 2024-09-16T00:26:07.229 INFO:tasks.workunit.client.0.smithi112.stdout:3/771: write d2/d9/d2b/f9a [1147827,108116] 0 2024-09-16T00:26:07.230 INFO:tasks.workunit.client.1.smithi169.stdout:3/582: truncate d7/d1a/d1c/d35/d63/d37/f9a 5120712 0 2024-09-16T00:26:07.232 INFO:tasks.workunit.client.0.smithi112.stdout:6/684: rename dd/df/d10/d47/d55/dc4/c96 to dd/df/d91/cdd 0 2024-09-16T00:26:07.232 INFO:tasks.workunit.client.0.smithi112.stdout:0/815: read d5/da/d57/d76/f14 [223912,47325] 0 2024-09-16T00:26:07.232 INFO:tasks.workunit.client.0.smithi112.stdout:9/955: getdents d1/d5/d6/d38/d14e 0 2024-09-16T00:26:07.234 INFO:tasks.workunit.client.0.smithi112.stdout:0/816: readlink d5/da/d1d/d71/l36 0 2024-09-16T00:26:07.235 INFO:tasks.workunit.client.0.smithi112.stdout:6/685: mknod dd/df/d10/d47/d89/dbc/cde 0 2024-09-16T00:26:07.236 INFO:tasks.workunit.client.1.smithi169.stdout:3/583: mkdir d7/d1a/d1c/d23/d88/dd7 0 2024-09-16T00:26:07.237 INFO:tasks.workunit.client.0.smithi112.stdout:0/817: write d5/da/d18/d24/d5a/fbf [229707,108666] 0 2024-09-16T00:26:07.238 INFO:tasks.workunit.client.0.smithi112.stdout:3/772: dwrite d2/d9/d2b/d2c/fce [0,4194304] 0 2024-09-16T00:26:07.238 INFO:tasks.workunit.client.0.smithi112.stdout:9/956: getdents d1/d5/d6/df8/dba 0 2024-09-16T00:26:07.239 INFO:tasks.workunit.client.1.smithi169.stdout:4/543: dread d2/d12/d13/d1f/f35 [0,4194304] 0 2024-09-16T00:26:07.239 INFO:tasks.workunit.client.1.smithi169.stdout:1/517: dwrite dc/d2c/d3d/d86/f4f [0,4194304] 0 2024-09-16T00:26:07.240 INFO:tasks.workunit.client.0.smithi112.stdout:3/773: readlink d2/d9/d2b/d2c/lb0 0 2024-09-16T00:26:07.241 INFO:tasks.workunit.client.0.smithi112.stdout:9/957: stat d1/d5/d6/df8/c13c 0 2024-09-16T00:26:07.241 INFO:tasks.workunit.client.0.smithi112.stdout:0/818: truncate d5/da/d18/d63/fcf 766447 0 2024-09-16T00:26:07.242 INFO:tasks.workunit.client.0.smithi112.stdout:6/686: mkdir dd/df/d10/d47/d55/dc4/dd1/ddf 0 2024-09-16T00:26:07.244 INFO:tasks.workunit.client.0.smithi112.stdout:6/687: readlink dd/df/d17/l81 0 2024-09-16T00:26:07.245 INFO:tasks.workunit.client.0.smithi112.stdout:9/958: read d1/d76/fdc [1492899,45504] 0 2024-09-16T00:26:07.245 INFO:tasks.workunit.client.0.smithi112.stdout:3/774: chown d2/d9/d2b/f82 398 1 2024-09-16T00:26:07.246 INFO:tasks.workunit.client.1.smithi169.stdout:9/593: dwrite d5/fb8 [0,4194304] 0 2024-09-16T00:26:07.247 INFO:tasks.workunit.client.0.smithi112.stdout:6/688: symlink dd/df/d17/le0 0 2024-09-16T00:26:07.250 INFO:tasks.workunit.client.1.smithi169.stdout:9/594: stat c4 0 2024-09-16T00:26:07.251 INFO:tasks.workunit.client.1.smithi169.stdout:4/544: write d2/d12/d13/d4e/f76 [4821229,101678] 0 2024-09-16T00:26:07.252 INFO:tasks.workunit.client.0.smithi112.stdout:0/819: dread d5/f95 [0,4194304] 0 2024-09-16T00:26:07.253 INFO:tasks.workunit.client.1.smithi169.stdout:4/545: read d2/d12/d13/d5c/f94 [149912,97241] 0 2024-09-16T00:26:07.253 INFO:tasks.workunit.client.1.smithi169.stdout:1/518: chown dc/d36/d3a/d7f/d93/f6b 44715 1 2024-09-16T00:26:07.253 INFO:tasks.workunit.client.1.smithi169.stdout:8/586: dwrite d7/dc/d26/d46/f5d [0,4194304] 0 2024-09-16T00:26:07.253 INFO:tasks.workunit.client.1.smithi169.stdout:1/519: readlink dc/l79 0 2024-09-16T00:26:07.257 INFO:tasks.workunit.client.0.smithi112.stdout:3/775: dwrite d2/d9/d2b/d47/d52/d7a/d80/d83/d76/d8e/ff0 [0,4194304] 0 2024-09-16T00:26:07.261 INFO:tasks.workunit.client.1.smithi169.stdout:1/520: mknod dc/d26/c97 0 2024-09-16T00:26:07.262 INFO:tasks.workunit.client.0.smithi112.stdout:3/776: creat d2/d9/d2b/d47/d78/da9/ffc x:0 0 0 2024-09-16T00:26:07.263 INFO:tasks.workunit.client.0.smithi112.stdout:3/777: chown d2/c31 2 1 2024-09-16T00:26:07.265 INFO:tasks.workunit.client.0.smithi112.stdout:3/778: read - d2/d9/fa5 zero size 2024-09-16T00:26:07.267 INFO:tasks.workunit.client.0.smithi112.stdout:3/779: mknod d2/d9/d2b/d2c/d69/dd5/cfd 0 2024-09-16T00:26:07.268 INFO:tasks.workunit.client.0.smithi112.stdout:3/780: fdatasync d2/f24 0 2024-09-16T00:26:07.271 INFO:tasks.workunit.client.1.smithi169.stdout:9/595: dwrite d5/da/d5b/d6c/da0/f95 [0,4194304] 0 2024-09-16T00:26:07.271 INFO:tasks.workunit.client.1.smithi169.stdout:1/521: unlink dc/d36/d3a/d45/d29/d2a/c3c 0 2024-09-16T00:26:07.272 INFO:tasks.workunit.client.1.smithi169.stdout:8/587: dwrite d7/dc/d2a/fba [0,4194304] 0 2024-09-16T00:26:07.274 INFO:tasks.workunit.client.0.smithi112.stdout:3/781: creat d2/d9/d2b/d47/d52/d7a/d80/d83/d76/d8e/ffe x:0 0 0 2024-09-16T00:26:07.275 INFO:tasks.workunit.client.1.smithi169.stdout:9/596: symlink d5/da/ld7 0 2024-09-16T00:26:07.277 INFO:tasks.workunit.client.0.smithi112.stdout:3/782: getdents d2/d9/dfb 0 2024-09-16T00:26:07.277 INFO:tasks.workunit.client.1.smithi169.stdout:1/522: truncate dc/d36/d3a/d45/f17 944279 0 2024-09-16T00:26:07.278 INFO:tasks.workunit.client.1.smithi169.stdout:9/597: write d5/da/d21/d23/f2d [4414264,109856] 0 2024-09-16T00:26:07.279 INFO:tasks.workunit.client.0.smithi112.stdout:3/783: mknod d2/d4d/cff 0 2024-09-16T00:26:07.279 INFO:tasks.workunit.client.0.smithi112.stdout:3/784: stat d2/d9/d2b/d2c/d69/l6a 0 2024-09-16T00:26:07.288 INFO:tasks.workunit.client.1.smithi169.stdout:3/584: dread d7/d1a/d1c/f54 [0,4194304] 0 2024-09-16T00:26:07.288 INFO:tasks.workunit.client.0.smithi112.stdout:6/689: write dd/d15/da5/d79/fb3 [1289781,50218] 0 2024-09-16T00:26:07.289 INFO:tasks.workunit.client.1.smithi169.stdout:1/523: read dc/d36/d3a/d45/d29/d2a/f62 [7528909,93668] 0 2024-09-16T00:26:07.291 INFO:tasks.workunit.client.1.smithi169.stdout:9/598: dwrite d5/da/d5b/d91/dc9/f7e [0,4194304] 0 2024-09-16T00:26:07.292 INFO:tasks.workunit.client.1.smithi169.stdout:5/485: dread d2/da/d10/d13/d1c/d40/d54/f56 [0,4194304] 0 2024-09-16T00:26:07.296 INFO:tasks.workunit.client.1.smithi169.stdout:9/599: symlink d5/da/d5b/d6c/da0/ld8 0 2024-09-16T00:26:07.300 INFO:tasks.workunit.client.0.smithi112.stdout:0/820: truncate d5/da/d18/d24/faf 3669994 0 2024-09-16T00:26:07.300 INFO:tasks.workunit.client.1.smithi169.stdout:3/585: dread d7/d1a/d1c/d23/d70/f8e [0,4194304] 0 2024-09-16T00:26:07.301 INFO:tasks.workunit.client.0.smithi112.stdout:0/821: mkdir d5/df7 0 2024-09-16T00:26:07.302 INFO:tasks.workunit.client.0.smithi112.stdout:3/785: write d2/d9/d2b/d47/d52/d7a/d80/d83/d49/f4c [8046959,59321] 0 2024-09-16T00:26:07.302 INFO:tasks.workunit.client.0.smithi112.stdout:0/822: chown d5/da/d18/d24/d46 6137 1 2024-09-16T00:26:07.303 INFO:tasks.workunit.client.1.smithi169.stdout:9/600: creat d5/da/d5b/d91/dc9/fd9 x:0 0 0 2024-09-16T00:26:07.303 INFO:tasks.workunit.client.0.smithi112.stdout:3/786: fsync d2/d9/d2b/d47/d52/d7a/d80/d83/d68/d90/fdb 0 2024-09-16T00:26:07.303 INFO:tasks.workunit.client.0.smithi112.stdout:0/823: creat d5/da/d18/d24/d46/d5f/da6/ff8 x:0 0 0 2024-09-16T00:26:07.304 INFO:tasks.workunit.client.1.smithi169.stdout:1/524: rmdir dc/d36/d3a/d45/d29/d2a/d83 0 2024-09-16T00:26:07.306 INFO:tasks.workunit.client.1.smithi169.stdout:4/546: dread d2/d12/d13/d1f/d2f/f45 [0,4194304] 0 2024-09-16T00:26:07.307 INFO:tasks.workunit.client.1.smithi169.stdout:3/586: mkdir d7/d50/d90/da6/dd8 0 2024-09-16T00:26:07.308 INFO:tasks.workunit.client.1.smithi169.stdout:9/601: read d5/fca [1562304,104781] 0 2024-09-16T00:26:07.309 INFO:tasks.workunit.client.1.smithi169.stdout:5/486: creat d2/da/d10/d13/d1c/d32/d53/d4e/fa8 x:0 0 0 2024-09-16T00:26:07.311 INFO:tasks.workunit.client.1.smithi169.stdout:1/525: write dc/d36/d3a/d7f/d93/f6d [253356,41189] 0 2024-09-16T00:26:07.313 INFO:tasks.workunit.client.1.smithi169.stdout:8/588: truncate d7/dc/d17/d24/f27 692610 0 2024-09-16T00:26:07.313 INFO:tasks.workunit.client.1.smithi169.stdout:5/487: creat d2/da/d10/fa9 x:0 0 0 2024-09-16T00:26:07.315 INFO:tasks.workunit.client.1.smithi169.stdout:1/526: chown dc/d36/d3a/d45/d29/d4d/c87 7933 1 2024-09-16T00:26:07.317 INFO:tasks.workunit.client.1.smithi169.stdout:3/587: dwrite d7/d1a/d1c/d35/d63/f9c [0,4194304] 0 2024-09-16T00:26:07.317 INFO:tasks.workunit.client.1.smithi169.stdout:1/527: stat dc/d36/d3a/d45/c2f 0 2024-09-16T00:26:07.317 INFO:tasks.workunit.client.1.smithi169.stdout:8/589: creat d7/dc/d1d/fc1 x:0 0 0 2024-09-16T00:26:07.319 INFO:tasks.workunit.client.1.smithi169.stdout:5/488: write d2/da/d10/d13/d1c/d40/d43/f84 [450780,124683] 0 2024-09-16T00:26:07.320 INFO:tasks.workunit.client.1.smithi169.stdout:1/528: truncate dc/d36/f84 211322 0 2024-09-16T00:26:07.321 INFO:tasks.workunit.client.1.smithi169.stdout:3/588: fsync d7/f14 0 2024-09-16T00:26:07.325 INFO:tasks.workunit.client.1.smithi169.stdout:8/590: unlink d7/dc/d17/d24/d32/d8d/f95 0 2024-09-16T00:26:07.327 INFO:tasks.workunit.client.1.smithi169.stdout:3/589: creat d7/d50/d90/da6/fd9 x:0 0 0 2024-09-16T00:26:07.327 INFO:tasks.workunit.client.0.smithi112.stdout:0/824: creat d5/da/d21/d6f/ff9 x:0 0 0 2024-09-16T00:26:07.328 INFO:tasks.workunit.client.1.smithi169.stdout:1/529: mkdir dc/d36/d3a/d7f/d93/d72/d98 0 2024-09-16T00:26:07.329 INFO:tasks.workunit.client.0.smithi112.stdout:3/787: mknod d2/d9/d2b/d47/d52/d7a/d80/d83/d7b/dad/c100 0 2024-09-16T00:26:07.331 INFO:tasks.workunit.client.0.smithi112.stdout:3/788: symlink d2/d9/d2b/d2c/d69/d92/l101 0 2024-09-16T00:26:07.331 INFO:tasks.workunit.client.1.smithi169.stdout:1/530: read dc/d36/d3a/d45/f61 [423169,50729] 0 2024-09-16T00:26:07.334 INFO:tasks.workunit.client.0.smithi112.stdout:6/690: dread dd/df/d10/d47/d55/f78 [0,4194304] 0 2024-09-16T00:26:07.334 INFO:tasks.workunit.client.1.smithi169.stdout:8/591: dwrite d7/dc/d26/d41/f5c [0,4194304] 0 2024-09-16T00:26:07.334 INFO:tasks.workunit.client.1.smithi169.stdout:3/590: rename d7/d1a/d1c/d23/f89 to d7/d1a/d1c/d35/fda 0 2024-09-16T00:26:07.336 INFO:tasks.workunit.client.1.smithi169.stdout:1/531: read dc/d36/d3a/d45/f23 [4121989,45478] 0 2024-09-16T00:26:07.336 INFO:tasks.workunit.client.1.smithi169.stdout:3/591: symlink d7/d50/d90/ldb 0 2024-09-16T00:26:07.338 INFO:tasks.workunit.client.1.smithi169.stdout:3/592: write d7/d1a/d2b/f8c [733136,127203] 0 2024-09-16T00:26:07.339 INFO:tasks.workunit.client.1.smithi169.stdout:3/593: stat d7/d1a/d1c/d35/f8a 0 2024-09-16T00:26:07.339 INFO:tasks.workunit.client.1.smithi169.stdout:1/532: mknod dc/d18/d67/d69/c99 0 2024-09-16T00:26:07.340 INFO:tasks.workunit.client.1.smithi169.stdout:8/592: rmdir d7/dc/d1d 39 2024-09-16T00:26:07.341 INFO:tasks.workunit.client.1.smithi169.stdout:3/594: write d7/d1a/d1c/fb6 [2613494,77445] 0 2024-09-16T00:26:07.345 INFO:tasks.workunit.client.1.smithi169.stdout:1/533: symlink dc/d2c/d3d/d86/l9a 0 2024-09-16T00:26:07.345 INFO:tasks.workunit.client.1.smithi169.stdout:3/595: stat d7/d1a/d2b/c67 0 2024-09-16T00:26:07.347 INFO:tasks.workunit.client.1.smithi169.stdout:8/593: unlink d7/dc/d17/d24/d32/d4d/faa 0 2024-09-16T00:26:07.348 INFO:tasks.workunit.client.1.smithi169.stdout:1/534: symlink dc/d26/l9b 0 2024-09-16T00:26:07.350 INFO:tasks.workunit.client.1.smithi169.stdout:5/489: truncate d2/da/d10/d13/d1c/f3b 30630 0 2024-09-16T00:26:07.351 INFO:tasks.workunit.client.1.smithi169.stdout:8/594: creat d7/d37/fc2 x:0 0 0 2024-09-16T00:26:07.353 INFO:tasks.workunit.client.1.smithi169.stdout:1/535: creat dc/d36/d3a/d45/d29/d4d/f9c x:0 0 0 2024-09-16T00:26:07.357 INFO:tasks.workunit.client.1.smithi169.stdout:8/595: rmdir d7/dc/d1d/d3c/d82/db3 39 2024-09-16T00:26:07.361 INFO:tasks.workunit.client.1.smithi169.stdout:1/536: dwrite dc/d36/d3a/d45/d29/d4d/f9c [0,4194304] 0 2024-09-16T00:26:07.363 INFO:tasks.workunit.client.1.smithi169.stdout:8/596: getdents d7/dc/d26/d46 0 2024-09-16T00:26:07.369 INFO:tasks.workunit.client.1.smithi169.stdout:1/537: creat dc/d18/d67/d69/d7b/d92/f9d x:0 0 0 2024-09-16T00:26:07.369 INFO:tasks.workunit.client.1.smithi169.stdout:3/596: truncate d7/d50/f79 2249434 0 2024-09-16T00:26:07.370 INFO:tasks.workunit.client.1.smithi169.stdout:5/490: write d2/da/d10/d13/d1c/f6a [2246076,87745] 0 2024-09-16T00:26:07.373 INFO:tasks.workunit.client.1.smithi169.stdout:8/597: dwrite d7/fd [0,4194304] 0 2024-09-16T00:26:07.374 INFO:tasks.workunit.client.1.smithi169.stdout:8/598: write d7/d37/fc2 [208543,6680] 0 2024-09-16T00:26:07.375 INFO:tasks.workunit.client.1.smithi169.stdout:3/597: symlink d7/d1a/d1c/d23/d88/dbe/ldc 0 2024-09-16T00:26:07.379 INFO:tasks.workunit.client.1.smithi169.stdout:5/491: truncate d2/da/d10/d13/d1c/d32/d53/f92 349418 0 2024-09-16T00:26:07.380 INFO:tasks.workunit.client.1.smithi169.stdout:8/599: fdatasync d7/dc/d2a/f42 0 2024-09-16T00:26:07.381 INFO:tasks.workunit.client.1.smithi169.stdout:3/598: creat d7/d1a/d1c/d23/d88/d9b/dc8/dcc/fdd x:0 0 0 2024-09-16T00:26:07.384 INFO:tasks.workunit.client.1.smithi169.stdout:8/600: mknod d7/dc/d26/d41/d4e/d81/d9e/da5/cc3 0 2024-09-16T00:26:07.388 INFO:tasks.workunit.client.1.smithi169.stdout:3/599: getdents d7/d50/d90/da6 0 2024-09-16T00:26:07.388 INFO:tasks.workunit.client.1.smithi169.stdout:8/601: symlink d7/dc/d1d/d3c/d82/lc4 0 2024-09-16T00:26:07.389 INFO:tasks.workunit.client.1.smithi169.stdout:8/602: fsync d7/dc/d53/f93 0 2024-09-16T00:26:07.394 INFO:tasks.workunit.client.1.smithi169.stdout:1/538: dread dc/d36/d3a/d7f/d93/d72/d73/f74 [0,4194304] 0 2024-09-16T00:26:07.397 INFO:tasks.workunit.client.1.smithi169.stdout:1/539: mknod dc/d18/d67/d69/c9e 0 2024-09-16T00:26:07.401 INFO:tasks.workunit.client.1.smithi169.stdout:1/540: creat dc/d36/d3a/f9f x:0 0 0 2024-09-16T00:26:07.404 INFO:tasks.workunit.client.1.smithi169.stdout:5/492: dwrite d2/da/d10/d13/d1c/d32/d53/d83/f76 [0,4194304] 0 2024-09-16T00:26:07.406 INFO:tasks.workunit.client.1.smithi169.stdout:3/600: dread d7/d1a/d1c/d35/d63/d37/f81 [0,4194304] 0 2024-09-16T00:26:07.406 INFO:tasks.workunit.client.1.smithi169.stdout:5/493: dread - d2/da/d10/d13/d1c/d32/f8f zero size 2024-09-16T00:26:07.407 INFO:tasks.workunit.client.1.smithi169.stdout:1/541: creat dc/d26/fa0 x:0 0 0 2024-09-16T00:26:07.409 INFO:tasks.workunit.client.1.smithi169.stdout:8/603: dread d7/dc/d17/f89 [0,4194304] 0 2024-09-16T00:26:07.410 INFO:tasks.workunit.client.1.smithi169.stdout:1/542: write dc/d18/d67/f49 [7183652,19705] 0 2024-09-16T00:26:07.410 INFO:tasks.workunit.client.1.smithi169.stdout:5/494: creat d2/da/d10/d13/d1c/d94/faa x:0 0 0 2024-09-16T00:26:07.416 INFO:tasks.workunit.client.1.smithi169.stdout:1/543: rename dc/d26/f46 to dc/fa1 0 2024-09-16T00:26:07.416 INFO:tasks.workunit.client.1.smithi169.stdout:8/604: getdents d7/dc/d26/d41/d4e/d81 0 2024-09-16T00:26:07.418 INFO:tasks.workunit.client.1.smithi169.stdout:1/544: creat dc/fa2 x:0 0 0 2024-09-16T00:26:07.418 INFO:tasks.workunit.client.1.smithi169.stdout:8/605: symlink d7/dc/d17/d24/d32/lc5 0 2024-09-16T00:26:07.419 INFO:tasks.workunit.client.1.smithi169.stdout:1/545: truncate dc/d2c/d3d/d86/f4f 4233771 0 2024-09-16T00:26:07.420 INFO:tasks.workunit.client.1.smithi169.stdout:8/606: write d7/dc/d26/d41/d4e/f7c [157968,34402] 0 2024-09-16T00:26:07.421 INFO:tasks.workunit.client.1.smithi169.stdout:1/546: rmdir dc/d36/d3a/d45/d29/d4d 39 2024-09-16T00:26:07.423 INFO:tasks.workunit.client.1.smithi169.stdout:1/547: write dc/d36/d3a/f76 [219649,106777] 0 2024-09-16T00:26:07.428 INFO:tasks.workunit.client.1.smithi169.stdout:8/607: truncate d7/dc/d17/d48/f60 4427345 0 2024-09-16T00:26:07.433 INFO:tasks.workunit.client.1.smithi169.stdout:1/548: dwrite dc/d18/f59 [0,4194304] 0 2024-09-16T00:26:07.435 INFO:tasks.workunit.client.1.smithi169.stdout:5/495: dwrite d2/da/d10/f27 [0,4194304] 0 2024-09-16T00:26:07.436 INFO:tasks.workunit.client.1.smithi169.stdout:8/608: creat d7/dc/d26/d41/d4e/d81/fc6 x:0 0 0 2024-09-16T00:26:07.436 INFO:tasks.workunit.client.1.smithi169.stdout:3/601: dwrite d7/d1a/d2b/f69 [0,4194304] 0 2024-09-16T00:26:07.443 INFO:tasks.workunit.client.1.smithi169.stdout:5/496: creat d2/d7c/fab x:0 0 0 2024-09-16T00:26:07.444 INFO:tasks.workunit.client.1.smithi169.stdout:3/602: symlink d7/d1a/d1c/d6c/lde 0 2024-09-16T00:26:07.445 INFO:tasks.workunit.client.1.smithi169.stdout:1/549: link dc/d36/d3a/d45/f23 dc/d36/d3a/d7f/d93/fa3 0 2024-09-16T00:26:07.446 INFO:tasks.workunit.client.1.smithi169.stdout:3/603: rmdir d7/d1a/d2b/d8d 39 2024-09-16T00:26:07.450 INFO:tasks.workunit.client.1.smithi169.stdout:8/609: fsync d7/dc/d26/d41/d4e/d81/fc6 0 2024-09-16T00:26:07.450 INFO:tasks.workunit.client.1.smithi169.stdout:3/604: rmdir d7/d1a/d1c/d23/d88/d9b 39 2024-09-16T00:26:07.451 INFO:tasks.workunit.client.1.smithi169.stdout:5/497: dwrite d2/da/d10/d13/d24/fa7 [0,4194304] 0 2024-09-16T00:26:07.453 INFO:tasks.workunit.client.1.smithi169.stdout:1/550: read dc/d36/d3a/d7f/d93/fa3 [1434835,53550] 0 2024-09-16T00:26:07.455 INFO:tasks.workunit.client.1.smithi169.stdout:1/551: chown dc/d36/d3a/d7f/l8c 301693 1 2024-09-16T00:26:07.456 INFO:tasks.workunit.client.1.smithi169.stdout:3/605: fdatasync d7/d50/d6f/f75 0 2024-09-16T00:26:07.460 INFO:tasks.workunit.client.0.smithi112.stdout:3/789: unlink d2/d9/d2b/c65 0 2024-09-16T00:26:07.461 INFO:tasks.workunit.client.1.smithi169.stdout:3/606: chown d7/d1a/d1c/d23/d88/d9b 12 1 2024-09-16T00:26:07.462 INFO:tasks.workunit.client.0.smithi112.stdout:6/691: symlink dd/df/d10/d47/d55/dc4/dd1/ddf/le1 0 2024-09-16T00:26:07.464 INFO:tasks.workunit.client.1.smithi169.stdout:1/552: dwrite dc/d2c/d3d/d86/f4f [0,4194304] 0 2024-09-16T00:26:07.464 INFO:tasks.workunit.client.0.smithi112.stdout:3/790: creat d2/d9/d2b/d2c/d69/d92/dda/f102 x:0 0 0 2024-09-16T00:26:07.465 INFO:tasks.workunit.client.1.smithi169.stdout:3/607: write d7/d50/d90/dba/fd0 [307445,87369] 0 2024-09-16T00:26:07.467 INFO:tasks.workunit.client.1.smithi169.stdout:1/553: creat dc/d18/d67/d69/d7b/d92/fa4 x:0 0 0 2024-09-16T00:26:07.471 INFO:tasks.workunit.client.1.smithi169.stdout:1/554: mkdir dc/d2c/d3d/d88/da5 0 2024-09-16T00:26:07.471 INFO:tasks.workunit.client.0.smithi112.stdout:0/825: dwrite d5/da/d1d/d71/f3b [0,4194304] 0 2024-09-16T00:26:07.472 INFO:tasks.workunit.client.1.smithi169.stdout:1/555: chown dc/fd 2 1 2024-09-16T00:26:07.472 INFO:tasks.workunit.client.1.smithi169.stdout:3/608: dwrite d7/f10 [0,4194304] 0 2024-09-16T00:26:07.472 INFO:tasks.workunit.client.1.smithi169.stdout:1/556: stat dc 0 2024-09-16T00:26:07.474 INFO:tasks.workunit.client.1.smithi169.stdout:1/557: write dc/d36/d3a/d45/f61 [986557,40309] 0 2024-09-16T00:26:07.474 INFO:tasks.workunit.client.0.smithi112.stdout:3/791: dwrite d2/d9/d2b/d47/d52/d7a/d80/d83/d76/d8e/fea [0,4194304] 0 2024-09-16T00:26:07.474 INFO:tasks.workunit.client.0.smithi112.stdout:0/826: symlink d5/da/d18/d63/lfa 0 2024-09-16T00:26:07.475 INFO:tasks.workunit.client.0.smithi112.stdout:3/792: chown d2/d9/d2b/d2c/d69/c9b 18107656 1 2024-09-16T00:26:07.476 INFO:tasks.workunit.client.0.smithi112.stdout:3/793: chown d2/d9/d2b/d47/l6d 84729888 1 2024-09-16T00:26:07.482 INFO:tasks.workunit.client.1.smithi169.stdout:5/498: write d2/da/d10/d13/d1c/d32/d53/f92 [1128231,37509] 0 2024-09-16T00:26:07.487 INFO:tasks.workunit.client.1.smithi169.stdout:3/609: dwrite d7/d1a/d1c/f25 [0,4194304] 0 2024-09-16T00:26:07.489 INFO:tasks.workunit.client.1.smithi169.stdout:1/558: dwrite dc/fd [4194304,4194304] 0 2024-09-16T00:26:07.491 INFO:tasks.workunit.client.1.smithi169.stdout:2/554: sync 2024-09-16T00:26:07.491 INFO:tasks.workunit.client.1.smithi169.stdout:5/499: symlink d2/da/d10/d13/d1c/d94/lac 0 2024-09-16T00:26:07.493 INFO:tasks.workunit.client.1.smithi169.stdout:2/555: fsync d7/d1c/fb7 0 2024-09-16T00:26:07.494 INFO:tasks.workunit.client.1.smithi169.stdout:3/610: creat d7/d1a/d1c/d6c/fdf x:0 0 0 2024-09-16T00:26:07.495 INFO:tasks.workunit.client.1.smithi169.stdout:5/500: write d2/d7c/fab [417121,121742] 0 2024-09-16T00:26:07.495 INFO:tasks.workunit.client.0.smithi112.stdout:6/692: dread dd/df/d17/d27/d54/f56 [0,4194304] 0 2024-09-16T00:26:07.499 INFO:tasks.workunit.client.0.smithi112.stdout:6/693: symlink dd/df/d10/d47/d55/dbd/le2 0 2024-09-16T00:26:07.499 INFO:tasks.workunit.client.1.smithi169.stdout:7/602: sync 2024-09-16T00:26:07.501 INFO:tasks.workunit.client.1.smithi169.stdout:3/611: unlink d7/d50/d90/lc4 0 2024-09-16T00:26:07.501 INFO:tasks.workunit.client.1.smithi169.stdout:1/559: dwrite dc/fa2 [0,4194304] 0 2024-09-16T00:26:07.501 INFO:tasks.workunit.client.0.smithi112.stdout:6/694: chown dd/df/d11/d44/da4/la6 1623953 1 2024-09-16T00:26:07.506 INFO:tasks.workunit.client.0.smithi112.stdout:6/695: rmdir dd/df/d17/d27/d84 39 2024-09-16T00:26:07.506 INFO:tasks.workunit.client.0.smithi112.stdout:4/769: sync 2024-09-16T00:26:07.506 INFO:tasks.workunit.client.0.smithi112.stdout:8/660: sync 2024-09-16T00:26:07.506 INFO:tasks.workunit.client.0.smithi112.stdout:2/831: sync 2024-09-16T00:26:07.506 INFO:tasks.workunit.client.0.smithi112.stdout:8/661: fsync d10/d12/d1c/d3b/f89 0 2024-09-16T00:26:07.506 INFO:tasks.workunit.client.1.smithi169.stdout:2/556: creat d7/db/d89/db4/d9d/fc5 x:0 0 0 2024-09-16T00:26:07.506 INFO:tasks.workunit.client.1.smithi169.stdout:7/603: unlink d7/d5b/d61/f6e 0 2024-09-16T00:26:07.506 INFO:tasks.workunit.client.1.smithi169.stdout:3/612: creat d7/d1a/d1c/d23/d70/fe0 x:0 0 0 2024-09-16T00:26:07.506 INFO:tasks.workunit.client.1.smithi169.stdout:7/604: chown d7/d71/f88 100 1 2024-09-16T00:26:07.507 INFO:tasks.workunit.client.0.smithi112.stdout:2/832: fdatasync d2/dd/d48/d22/d67/fb2 0 2024-09-16T00:26:07.508 INFO:tasks.workunit.client.1.smithi169.stdout:1/560: rename dc/d18/d67/d69/d7b/d92/fa4 to dc/d36/d39/d50/fa6 0 2024-09-16T00:26:07.510 INFO:tasks.workunit.client.1.smithi169.stdout:3/613: mknod d7/d50/d90/da6/dd8/ce1 0 2024-09-16T00:26:07.511 INFO:tasks.workunit.client.0.smithi112.stdout:8/662: read - d10/d12/d1c/d61/d6c/dd1/fc9 zero size 2024-09-16T00:26:07.512 INFO:tasks.workunit.client.0.smithi112.stdout:6/696: dwrite dd/df/f3d [0,4194304] 0 2024-09-16T00:26:07.514 INFO:tasks.workunit.client.1.smithi169.stdout:1/561: write dc/d36/d3a/d45/d29/d2a/f44 [1507838,65544] 0 2024-09-16T00:26:07.514 INFO:tasks.workunit.client.1.smithi169.stdout:3/614: write d7/d1a/d1c/d6c/f6d [1444257,88916] 0 2024-09-16T00:26:07.514 INFO:tasks.workunit.client.1.smithi169.stdout:1/562: chown dc/d2c 201 1 2024-09-16T00:26:07.515 INFO:tasks.workunit.client.0.smithi112.stdout:2/833: rename d2/dd/d48/d8a/d9b/la8 to d2/dd/d48/d22/d2b/d54/d79/l10c 0 2024-09-16T00:26:07.517 INFO:tasks.workunit.client.0.smithi112.stdout:6/697: truncate dd/df/d10/d47/d55/dc4/dd1/d88/f9b 5007 0 2024-09-16T00:26:07.517 INFO:tasks.workunit.client.0.smithi112.stdout:3/794: dread d2/d9/d2b/d47/d52/d7a/d80/d83/d49/f77 [0,4194304] 0 2024-09-16T00:26:07.517 INFO:tasks.workunit.client.1.smithi169.stdout:1/563: link dc/d36/d3a/f9f dc/d36/d3a/d45/fa7 0 2024-09-16T00:26:07.518 INFO:tasks.workunit.client.1.smithi169.stdout:1/564: readlink dc/d26/l9b 0 2024-09-16T00:26:07.521 INFO:tasks.workunit.client.0.smithi112.stdout:2/834: rmdir d2/dd/d48/d22/d2b/d30 39 2024-09-16T00:26:07.525 INFO:tasks.workunit.client.1.smithi169.stdout:1/565: creat dc/d36/d39/d50/d75/fa8 x:0 0 0 2024-09-16T00:26:07.525 INFO:tasks.workunit.client.0.smithi112.stdout:3/795: dwrite d2/f2d [4194304,4194304] 0 2024-09-16T00:26:07.527 INFO:tasks.workunit.client.1.smithi169.stdout:3/615: dread d7/d50/d6f/f74 [0,4194304] 0 2024-09-16T00:26:07.527 INFO:tasks.workunit.client.0.smithi112.stdout:2/835: dread d2/dd/d48/d94/d49/f5c [0,4194304] 0 2024-09-16T00:26:07.528 INFO:tasks.workunit.client.1.smithi169.stdout:3/616: write d7/d1a/d2b/f46 [3685527,112334] 0 2024-09-16T00:26:07.531 INFO:tasks.workunit.client.0.smithi112.stdout:6/698: creat dd/fe3 x:0 0 0 2024-09-16T00:26:07.532 INFO:tasks.workunit.client.1.smithi169.stdout:3/617: mkdir d7/d1a/d1c/d23/d88/d9b/de2 0 2024-09-16T00:26:07.533 INFO:tasks.workunit.client.1.smithi169.stdout:7/605: write d7/d16/f79 [4020254,56673] 0 2024-09-16T00:26:07.535 INFO:tasks.workunit.client.0.smithi112.stdout:0/827: dwrite d5/da/d21/d6f/f84 [0,4194304] 0 2024-09-16T00:26:07.536 INFO:tasks.workunit.client.0.smithi112.stdout:6/699: fdatasync dd/df/d10/f3a 0 2024-09-16T00:26:07.536 INFO:tasks.workunit.client.1.smithi169.stdout:3/618: symlink d7/d1a/d2b/d8d/le3 0 2024-09-16T00:26:07.537 INFO:tasks.workunit.client.1.smithi169.stdout:7/606: mknod d7/d5b/d61/cc3 0 2024-09-16T00:26:07.537 INFO:tasks.workunit.client.0.smithi112.stdout:2/836: dwrite d2/dd/d48/d94/fe5 [0,4194304] 0 2024-09-16T00:26:07.538 INFO:tasks.workunit.client.0.smithi112.stdout:0/828: write d5/da/d21/d6f/f84 [4251350,2258] 0 2024-09-16T00:26:07.541 INFO:tasks.workunit.client.0.smithi112.stdout:2/837: chown d2/dd/d48/f2a 16082 1 2024-09-16T00:26:07.542 INFO:tasks.workunit.client.1.smithi169.stdout:3/619: mkdir d7/d1a/d1c/d35/d63/da3/de4 0 2024-09-16T00:26:07.543 INFO:tasks.workunit.client.0.smithi112.stdout:8/663: dread d10/d12/d1c/d61/d6c/dd1/d8c/f9a [0,4194304] 0 2024-09-16T00:26:07.546 INFO:tasks.workunit.client.1.smithi169.stdout:2/557: dread d7/db/f19 [0,4194304] 0 2024-09-16T00:26:07.548 INFO:tasks.workunit.client.0.smithi112.stdout:6/700: dwrite dd/df/d11/fad [0,4194304] 0 2024-09-16T00:26:07.550 INFO:tasks.workunit.client.1.smithi169.stdout:7/607: dwrite d7/d5b/dc1/d96/fbf [0,4194304] 0 2024-09-16T00:26:07.551 INFO:tasks.workunit.client.0.smithi112.stdout:0/829: link d5/da/d57/d76/ca4 d5/da/d18/d63/db2/cfb 0 2024-09-16T00:26:07.553 INFO:tasks.workunit.client.1.smithi169.stdout:1/566: chown dc/d36/d3a/d45/d29/f52 59911336 1 2024-09-16T00:26:07.553 INFO:tasks.workunit.client.1.smithi169.stdout:5/501: dread d2/da/d10/d13/d1c/d2d/f63 [0,4194304] 0 2024-09-16T00:26:07.554 INFO:tasks.workunit.client.0.smithi112.stdout:2/838: dwrite d2/dd/d48/f85 [0,4194304] 0 2024-09-16T00:26:07.557 INFO:tasks.workunit.client.0.smithi112.stdout:6/701: symlink dd/dcc/le4 0 2024-09-16T00:26:07.557 INFO:tasks.workunit.client.0.smithi112.stdout:2/839: chown d2/dd/d48/d22/d2b/d54/d60/d6c/dc9/dd5/d108 0 1 2024-09-16T00:26:07.559 INFO:tasks.workunit.client.1.smithi169.stdout:2/558: dread d7/d1c/d26/d96/dbd/f80 [0,4194304] 0 2024-09-16T00:26:07.559 INFO:tasks.workunit.client.1.smithi169.stdout:1/567: creat dc/d2c/d3d/d88/fa9 x:0 0 0 2024-09-16T00:26:07.560 INFO:tasks.workunit.client.1.smithi169.stdout:5/502: truncate d2/da/d10/f3f 188715 0 2024-09-16T00:26:07.561 INFO:tasks.workunit.client.0.smithi112.stdout:0/830: rename d5/da/d21/fdf to d5/da/d18/d24/d5a/ffc 0 2024-09-16T00:26:07.562 INFO:tasks.workunit.client.0.smithi112.stdout:2/840: mkdir d2/dd/d48/d22/d2b/d54/d60/d6c/dc9/d10d 0 2024-09-16T00:26:07.563 INFO:tasks.workunit.client.0.smithi112.stdout:0/831: mknod d5/da/d57/cfd 0 2024-09-16T00:26:07.566 INFO:tasks.workunit.client.0.smithi112.stdout:6/702: read dd/df/d10/d47/d89/dbc/fdb [3331483,110546] 0 2024-09-16T00:26:07.566 INFO:tasks.workunit.client.1.smithi169.stdout:7/608: dwrite d7/d71/d76/d29/d78/fa3 [0,4194304] 0 2024-09-16T00:26:07.566 INFO:tasks.workunit.client.1.smithi169.stdout:5/503: rmdir d2/da/d10/d13/d1c/d32/d53/d4e 39 2024-09-16T00:26:07.567 INFO:tasks.workunit.client.0.smithi112.stdout:2/841: mkdir d2/dd/d48/d22/d2b/d54/d79/d10e 0 2024-09-16T00:26:07.567 INFO:tasks.workunit.client.0.smithi112.stdout:0/832: getdents d5/df7 0 2024-09-16T00:26:07.569 INFO:tasks.workunit.client.0.smithi112.stdout:0/833: write d5/da/d21/fb5 [453807,85253] 0 2024-09-16T00:26:07.569 INFO:tasks.workunit.client.1.smithi169.stdout:2/559: mknod d7/d6b/d84/cc6 0 2024-09-16T00:26:07.570 INFO:tasks.workunit.client.0.smithi112.stdout:6/703: mknod dd/df/d11/d44/da4/ce5 0 2024-09-16T00:26:07.572 INFO:tasks.workunit.client.0.smithi112.stdout:0/834: creat d5/da/d18/d24/d46/d5f/ffe x:0 0 0 2024-09-16T00:26:07.573 INFO:tasks.workunit.client.1.smithi169.stdout:2/560: stat d7/d1c/d26/l3a 0 2024-09-16T00:26:07.573 INFO:tasks.workunit.client.0.smithi112.stdout:6/704: mkdir dd/dcc/de6 0 2024-09-16T00:26:07.574 INFO:tasks.workunit.client.1.smithi169.stdout:7/609: creat d7/fc4 x:0 0 0 2024-09-16T00:26:07.574 INFO:tasks.workunit.client.0.smithi112.stdout:2/842: rmdir d2/dd/d48/d8a/db9/de7/df1 0 2024-09-16T00:26:07.576 INFO:tasks.workunit.client.0.smithi112.stdout:6/705: stat dd/df/d17/d27/d84/f36 0 2024-09-16T00:26:07.582 INFO:tasks.workunit.client.0.smithi112.stdout:6/706: link dd/df/d17/d27/f76 dd/df/d11/d44/d8a/fe7 0 2024-09-16T00:26:07.582 INFO:tasks.workunit.client.0.smithi112.stdout:2/843: dwrite d2/dd/f87 [0,4194304] 0 2024-09-16T00:26:07.584 INFO:tasks.workunit.client.0.smithi112.stdout:2/844: chown d2/dd/d48/f2a 630 1 2024-09-16T00:26:07.586 INFO:tasks.workunit.client.0.smithi112.stdout:8/664: dwrite d10/d12/d1c/d61/fb4 [0,4194304] 0 2024-09-16T00:26:07.590 INFO:tasks.workunit.client.0.smithi112.stdout:2/845: creat d2/dd/d48/d22/d2b/d54/d60/d6c/dc9/d10d/f10f x:0 0 0 2024-09-16T00:26:07.591 INFO:tasks.workunit.client.0.smithi112.stdout:6/707: dwrite dd/d23/f35 [0,4194304] 0 2024-09-16T00:26:07.593 INFO:tasks.workunit.client.0.smithi112.stdout:8/665: dread d10/d12/d1c/d61/fb4 [0,4194304] 0 2024-09-16T00:26:07.594 INFO:tasks.workunit.client.0.smithi112.stdout:8/666: readlink d10/d12/d1c/dd0/lce 0 2024-09-16T00:26:07.595 INFO:tasks.workunit.client.0.smithi112.stdout:6/708: mkdir dd/df/d10/d47/d55/dbd/dcb/de8 0 2024-09-16T00:26:07.597 INFO:tasks.workunit.client.1.smithi169.stdout:3/620: truncate d7/d1a/d1c/d23/f3a 9426207 0 2024-09-16T00:26:07.597 INFO:tasks.workunit.client.0.smithi112.stdout:8/667: mkdir d10/d12/d1c/d61/d6c/ddc 0 2024-09-16T00:26:07.598 INFO:tasks.workunit.client.0.smithi112.stdout:6/709: chown dd/df/d17/l81 1080 1 2024-09-16T00:26:07.598 INFO:tasks.workunit.client.0.smithi112.stdout:3/796: write d2/d9/d2b/d39/fbb [541948,126722] 0 2024-09-16T00:26:07.601 INFO:tasks.workunit.client.0.smithi112.stdout:8/668: creat d10/d12/d24/fdd x:0 0 0 2024-09-16T00:26:07.601 INFO:tasks.workunit.client.0.smithi112.stdout:6/710: stat dd/df/d11/d44/da4/la6 0 2024-09-16T00:26:07.601 INFO:tasks.workunit.client.1.smithi169.stdout:2/561: dread d7/d1c/d26/d96/dbd/f53 [0,4194304] 0 2024-09-16T00:26:07.602 INFO:tasks.workunit.client.0.smithi112.stdout:8/669: chown d10/d12/d1c/dd0/c62 455518 1 2024-09-16T00:26:07.605 INFO:tasks.workunit.client.0.smithi112.stdout:0/835: write d5/da/d18/d63/fa3 [383045,1843] 0 2024-09-16T00:26:07.605 INFO:tasks.workunit.client.1.smithi169.stdout:1/568: dread dc/f15 [0,4194304] 0 2024-09-16T00:26:07.606 INFO:tasks.workunit.client.1.smithi169.stdout:5/504: write d2/f1f [791786,41024] 0 2024-09-16T00:26:07.606 INFO:tasks.workunit.client.0.smithi112.stdout:3/797: dwrite d2/d9/d2b/d47/d78/da9/ffc [0,4194304] 0 2024-09-16T00:26:07.607 INFO:tasks.workunit.client.0.smithi112.stdout:0/836: truncate d5/f95 1270223 0 2024-09-16T00:26:07.608 INFO:tasks.workunit.client.1.smithi169.stdout:1/569: chown dc/d36/d3a/d7f/d93/d72/d73 715 1 2024-09-16T00:26:07.609 INFO:tasks.workunit.client.1.smithi169.stdout:5/505: write d2/da/d10/d13/d1c/d32/f8f [432775,120704] 0 2024-09-16T00:26:07.609 INFO:tasks.workunit.client.0.smithi112.stdout:3/798: readlink d2/d9/d2b/d47/d52/l75 0 2024-09-16T00:26:07.610 INFO:tasks.workunit.client.1.smithi169.stdout:1/570: chown dc/d2c/d3d/d88/da5 6657 1 2024-09-16T00:26:07.610 INFO:tasks.workunit.client.1.smithi169.stdout:3/621: mkdir d7/de5 0 2024-09-16T00:26:07.611 INFO:tasks.workunit.client.0.smithi112.stdout:0/837: mkdir d5/da/d18/d63/db2/dd4/dff 0 2024-09-16T00:26:07.616 INFO:tasks.workunit.client.1.smithi169.stdout:7/610: rmdir d7/d16/d49/db2 39 2024-09-16T00:26:07.616 INFO:tasks.workunit.client.1.smithi169.stdout:5/506: mkdir d2/da/d10/d13/d1c/d32/d53/d4e/d89/d9a/dad 0 2024-09-16T00:26:07.617 INFO:tasks.workunit.client.0.smithi112.stdout:3/799: dwrite d2/d9/d2b/d47/d52/d7a/d80/d83/d76/fed [0,4194304] 0 2024-09-16T00:26:07.620 INFO:tasks.workunit.client.0.smithi112.stdout:3/800: mknod d2/d9/d2b/d47/d52/d7a/d80/d83/d7b/dad/c103 0 2024-09-16T00:26:07.622 INFO:tasks.workunit.client.1.smithi169.stdout:3/622: dread d7/d50/d6f/f75 [0,4194304] 0 2024-09-16T00:26:07.625 INFO:tasks.workunit.client.1.smithi169.stdout:3/623: write d7/d1a/d1c/f78 [258373,31141] 0 2024-09-16T00:26:07.631 INFO:tasks.workunit.client.1.smithi169.stdout:7/611: dwrite d7/d5b/d61/d98/d99/fa8 [0,4194304] 0 2024-09-16T00:26:07.633 INFO:tasks.workunit.client.0.smithi112.stdout:7/781: sync 2024-09-16T00:26:07.633 INFO:tasks.workunit.client.0.smithi112.stdout:5/666: sync 2024-09-16T00:26:07.635 INFO:tasks.workunit.client.1.smithi169.stdout:7/612: creat d7/d16/d49/d74/fc5 x:0 0 0 2024-09-16T00:26:07.636 INFO:tasks.workunit.client.0.smithi112.stdout:5/667: creat d4/dd/d49/d70/fd6 x:0 0 0 2024-09-16T00:26:07.636 INFO:tasks.workunit.client.0.smithi112.stdout:7/782: mknod d6/d10/d16/d49/c104 0 2024-09-16T00:26:07.637 INFO:tasks.workunit.client.1.smithi169.stdout:3/624: dwrite d7/d1a/d2b/f8c [0,4194304] 0 2024-09-16T00:26:07.641 INFO:tasks.workunit.client.1.smithi169.stdout:3/625: stat d7/d50/d90/da6/fbb 0 2024-09-16T00:26:07.641 INFO:tasks.workunit.client.0.smithi112.stdout:5/668: link d4/d1c/f73 d4/d7/fd7 0 2024-09-16T00:26:07.645 INFO:tasks.workunit.client.0.smithi112.stdout:5/669: dwrite d4/d4f/d77/f7f [0,4194304] 0 2024-09-16T00:26:07.646 INFO:tasks.workunit.client.1.smithi169.stdout:7/613: dwrite d7/d5b/d61/d98/d99/fa8 [0,4194304] 0 2024-09-16T00:26:07.647 INFO:tasks.workunit.client.1.smithi169.stdout:3/626: unlink d7/d1a/d2b/f61 0 2024-09-16T00:26:07.649 INFO:tasks.workunit.client.0.smithi112.stdout:2/846: write d2/d68/f7f [2816732,70895] 0 2024-09-16T00:26:07.650 INFO:tasks.workunit.client.1.smithi169.stdout:3/627: write d7/d50/d6f/fb4 [515380,117897] 0 2024-09-16T00:26:07.651 INFO:tasks.workunit.client.1.smithi169.stdout:7/614: mkdir d7/d16/d49/d84/dc6 0 2024-09-16T00:26:07.652 INFO:tasks.workunit.client.0.smithi112.stdout:5/670: truncate d4/d7/d14/f33 1500963 0 2024-09-16T00:26:07.652 INFO:tasks.workunit.client.1.smithi169.stdout:2/562: dread d7/f11 [0,4194304] 0 2024-09-16T00:26:07.652 INFO:tasks.workunit.client.0.smithi112.stdout:6/711: write dd/d15/da5/f74 [1036689,105579] 0 2024-09-16T00:26:07.655 INFO:tasks.workunit.client.0.smithi112.stdout:2/847: rename d2/dd/d48/d94 to d2/dd/d48/d22/d2b/d54/d79/d110 0 2024-09-16T00:26:07.658 INFO:tasks.workunit.client.0.smithi112.stdout:5/671: mknod d4/d7/d26/d4c/db3/db6/dd0/cd8 0 2024-09-16T00:26:07.659 INFO:tasks.workunit.client.0.smithi112.stdout:0/838: write d5/da/d66/fb7 [985279,58004] 0 2024-09-16T00:26:07.661 INFO:tasks.workunit.client.1.smithi169.stdout:3/628: dwrite d7/d50/d90/da6/fbb [0,4194304] 0 2024-09-16T00:26:07.661 INFO:tasks.workunit.client.0.smithi112.stdout:2/848: creat d2/dd/d48/d22/d2b/d54/d60/d6c/f111 x:0 0 0 2024-09-16T00:26:07.663 INFO:tasks.workunit.client.0.smithi112.stdout:8/670: dwrite d10/d12/d1c/d61/d6c/fdb [0,4194304] 0 2024-09-16T00:26:07.664 INFO:tasks.workunit.client.1.smithi169.stdout:3/629: fsync d7/d50/fd5 0 2024-09-16T00:26:07.664 INFO:tasks.workunit.client.1.smithi169.stdout:7/615: getdents d7/d5b/dc1/d96 0 2024-09-16T00:26:07.665 INFO:tasks.workunit.client.0.smithi112.stdout:8/671: write d10/d12/d1c/d61/d6c/dd1/d58/fcb [997840,13050] 0 2024-09-16T00:26:07.668 INFO:tasks.workunit.client.0.smithi112.stdout:2/849: symlink d2/dd/d48/d22/d67/d9a/l112 0 2024-09-16T00:26:07.668 INFO:tasks.workunit.client.1.smithi169.stdout:7/616: chown d7/d5b/d61/d98/d99/lab 0 1 2024-09-16T00:26:07.672 INFO:tasks.workunit.client.1.smithi169.stdout:7/617: mknod d7/d5b/dc1/cc7 0 2024-09-16T00:26:07.672 INFO:tasks.workunit.client.0.smithi112.stdout:8/672: link d10/d12/d1c/dd0/fa0 d10/d12/d1c/d96/fde 0 2024-09-16T00:26:07.674 INFO:tasks.workunit.client.0.smithi112.stdout:3/801: dread d2/f19 [0,4194304] 0 2024-09-16T00:26:07.674 INFO:tasks.workunit.client.1.smithi169.stdout:7/618: mkdir d7/d16/d49/d84/dc8 0 2024-09-16T00:26:07.676 INFO:tasks.workunit.client.0.smithi112.stdout:3/802: readlink d2/d9/d2b/d47/d52/d7a/d80/d83/d68/lb9 0 2024-09-16T00:26:07.677 INFO:tasks.workunit.client.0.smithi112.stdout:2/850: dwrite d2/dd/d48/d22/d2b/f31 [4194304,4194304] 0 2024-09-16T00:26:07.679 INFO:tasks.workunit.client.0.smithi112.stdout:3/803: unlink d2/lbe 0 2024-09-16T00:26:07.682 INFO:tasks.workunit.client.0.smithi112.stdout:2/851: creat d2/dd/d48/d8a/db9/de7/f113 x:0 0 0 2024-09-16T00:26:07.683 INFO:tasks.workunit.client.0.smithi112.stdout:2/852: write d2/dd/d48/d8a/d9b/fdc [895045,89207] 0 2024-09-16T00:26:07.686 INFO:tasks.workunit.client.0.smithi112.stdout:3/804: link d2/d9/f4e d2/d9/d2b/d47/d52/d7a/d80/d83/d76/d8e/f104 0 2024-09-16T00:26:07.687 INFO:tasks.workunit.client.0.smithi112.stdout:2/853: creat d2/dd/d48/f114 x:0 0 0 2024-09-16T00:26:07.687 INFO:tasks.workunit.client.0.smithi112.stdout:3/805: fdatasync d2/d9/d2b/d2c/f51 0 2024-09-16T00:26:07.688 INFO:tasks.workunit.client.0.smithi112.stdout:2/854: write d2/dd/d48/d8a/d9b/fdc [473158,81719] 0 2024-09-16T00:26:07.690 INFO:tasks.workunit.client.0.smithi112.stdout:3/806: mkdir d2/d9/d2b/d2c/d69/dd5/d105 0 2024-09-16T00:26:07.693 INFO:tasks.workunit.client.0.smithi112.stdout:2/855: getdents d2/dd/d48/d22/d2b/d30 0 2024-09-16T00:26:07.695 INFO:tasks.workunit.client.0.smithi112.stdout:2/856: mknod d2/dd/d48/c115 0 2024-09-16T00:26:07.696 INFO:tasks.workunit.client.0.smithi112.stdout:2/857: rename d2/dd/d48 to d2/dd/d48/d22/d2b/d54/d60/da9/d116 22 2024-09-16T00:26:07.696 INFO:tasks.workunit.client.0.smithi112.stdout:2/858: stat d2/dd/d48/d22/d2b/f10a 0 2024-09-16T00:26:07.699 INFO:tasks.workunit.client.0.smithi112.stdout:2/859: creat d2/dd/d48/d22/d2b/d54/d60/d6c/d88/da4/f117 x:0 0 0 2024-09-16T00:26:07.705 INFO:tasks.workunit.client.0.smithi112.stdout:2/860: dwrite d2/dd/d48/d22/d2b/d54/d60/d6c/f111 [0,4194304] 0 2024-09-16T00:26:07.706 INFO:tasks.workunit.client.0.smithi112.stdout:2/861: symlink d2/d68/l118 0 2024-09-16T00:26:07.707 INFO:tasks.workunit.client.0.smithi112.stdout:2/862: stat d2/dd/d48 0 2024-09-16T00:26:07.710 INFO:tasks.workunit.client.0.smithi112.stdout:7/783: write d6/d10/d16/d49/d6c/f95 [554841,56373] 0 2024-09-16T00:26:07.711 INFO:tasks.workunit.client.0.smithi112.stdout:0/839: write d5/da/d18/d24/d5a/ffc [556,82962] 0 2024-09-16T00:26:07.712 INFO:tasks.workunit.client.0.smithi112.stdout:2/863: truncate d2/dd/d48/d8a/ff7 761929 0 2024-09-16T00:26:07.715 INFO:tasks.workunit.client.0.smithi112.stdout:5/672: write d4/d7/d26/d36/f3b [2793447,9633] 0 2024-09-16T00:26:07.718 INFO:tasks.workunit.client.0.smithi112.stdout:8/673: write d10/f34 [2269265,99651] 0 2024-09-16T00:26:07.718 INFO:tasks.workunit.client.0.smithi112.stdout:9/959: sync 2024-09-16T00:26:07.718 INFO:tasks.workunit.client.0.smithi112.stdout:4/770: sync 2024-09-16T00:26:07.718 INFO:tasks.workunit.client.0.smithi112.stdout:1/837: sync 2024-09-16T00:26:07.721 INFO:tasks.workunit.client.0.smithi112.stdout:7/784: dread d6/d10/d16/d96/dc7/fbe [0,4194304] 0 2024-09-16T00:26:07.725 INFO:tasks.workunit.client.0.smithi112.stdout:6/712: dwrite dd/df/d10/f8b [0,4194304] 0 2024-09-16T00:26:07.728 INFO:tasks.workunit.client.0.smithi112.stdout:0/840: dread d5/da/d18/f62 [0,4194304] 0 2024-09-16T00:26:07.729 INFO:tasks.workunit.client.0.smithi112.stdout:2/864: rename d2/dd/d48/d22/d2b/d54/d79/d110/ca2 to d2/dd/d48/d22/d2b/c119 0 2024-09-16T00:26:07.729 INFO:tasks.workunit.client.0.smithi112.stdout:1/838: dread - d0/d14/df/d28/d45/d46/d58/d62/ff4 zero size 2024-09-16T00:26:07.730 INFO:tasks.workunit.client.0.smithi112.stdout:4/771: unlink d6/d39/d36/d5c/d7e/fa5 0 2024-09-16T00:26:07.732 INFO:tasks.workunit.client.0.smithi112.stdout:5/673: creat d4/d7/d26/d36/dcf/fd9 x:0 0 0 2024-09-16T00:26:07.732 INFO:tasks.workunit.client.0.smithi112.stdout:0/841: fdatasync d5/da/d18/d89/fab 0 2024-09-16T00:26:07.733 INFO:tasks.workunit.client.0.smithi112.stdout:8/674: mknod d10/d12/d1c/d3b/cdf 0 2024-09-16T00:26:07.733 INFO:tasks.workunit.client.0.smithi112.stdout:3/807: dread d2/d9/d2b/d2c/f42 [0,4194304] 0 2024-09-16T00:26:07.733 INFO:tasks.workunit.client.0.smithi112.stdout:9/960: symlink d1/d5/d20/d81/d1a/d88/da8/l14f 0 2024-09-16T00:26:07.734 INFO:tasks.workunit.client.0.smithi112.stdout:5/674: dread - d4/d1c/fbe zero size 2024-09-16T00:26:07.738 INFO:tasks.workunit.client.0.smithi112.stdout:2/865: mkdir d2/dd/d48/d22/d2b/d54/d60/daf/d11a 0 2024-09-16T00:26:07.739 INFO:tasks.workunit.client.1.smithi169.stdout:1/571: dwrite dc/d36/d3a/d7f/d93/fa3 [0,4194304] 0 2024-09-16T00:26:07.740 INFO:tasks.workunit.client.0.smithi112.stdout:2/866: dread - d2/dd/d48/d22/d2b/d54/d60/d6c/dc9/d10d/f10f zero size 2024-09-16T00:26:07.741 INFO:tasks.workunit.client.0.smithi112.stdout:6/713: rename dd/df/d10/d47/d89/dbc/dd6 to dd/df/d10/d47/d55/dbd/de9 0 2024-09-16T00:26:07.742 INFO:tasks.workunit.client.1.smithi169.stdout:3/630: dwrite d7/d1a/d1c/d35/f8a [0,4194304] 0 2024-09-16T00:26:07.742 INFO:tasks.workunit.client.0.smithi112.stdout:0/842: mkdir d5/da/d1d/d71/d64/d100 0 2024-09-16T00:26:07.744 INFO:tasks.workunit.client.1.smithi169.stdout:1/572: unlink dc/d36/d39/d50/fa6 0 2024-09-16T00:26:07.745 INFO:tasks.workunit.client.0.smithi112.stdout:5/675: symlink d4/lda 0 2024-09-16T00:26:07.745 INFO:tasks.workunit.client.0.smithi112.stdout:9/961: symlink d1/d5/d6/d59/l150 0 2024-09-16T00:26:07.746 INFO:tasks.workunit.client.0.smithi112.stdout:3/808: fdatasync d2/d9/f4e 0 2024-09-16T00:26:07.748 INFO:tasks.workunit.client.0.smithi112.stdout:1/839: creat d0/d14/df/d28/d45/d46/d58/f111 x:0 0 0 2024-09-16T00:26:07.749 INFO:tasks.workunit.client.1.smithi169.stdout:1/573: creat dc/d2c/d3d/d88/da5/faa x:0 0 0 2024-09-16T00:26:07.751 INFO:tasks.workunit.client.1.smithi169.stdout:3/631: getdents d7/d1a 0 2024-09-16T00:26:07.752 INFO:tasks.workunit.client.1.smithi169.stdout:7/619: dread d7/d16/d49/d74/fb9 [0,4194304] 0 2024-09-16T00:26:07.752 INFO:tasks.workunit.client.1.smithi169.stdout:3/632: readlink d7/d50/d90/ldb 0 2024-09-16T00:26:07.753 INFO:tasks.workunit.client.0.smithi112.stdout:4/772: rename d6/d39/d36/d5c/d7e/dcd to d6/d39/d1c/d6e/d6f/d11b 0 2024-09-16T00:26:07.756 INFO:tasks.workunit.client.1.smithi169.stdout:3/633: truncate d7/d1a/d1c/f73 539901 0 2024-09-16T00:26:07.756 INFO:tasks.workunit.client.0.smithi112.stdout:4/773: dread d6/d39/d1c/d6e/d6f/fe5 [0,4194304] 0 2024-09-16T00:26:07.758 INFO:tasks.workunit.client.1.smithi169.stdout:3/634: stat d7/d1a/d1c/d35/d63/fcb 0 2024-09-16T00:26:07.759 INFO:tasks.workunit.client.0.smithi112.stdout:2/867: symlink d2/dd/d48/d22/d2b/d54/d60/d6c/d88/dad/deb/de1/l11b 0 2024-09-16T00:26:07.760 INFO:tasks.workunit.client.0.smithi112.stdout:5/676: chown d4/l42 0 1 2024-09-16T00:26:07.760 INFO:tasks.workunit.client.1.smithi169.stdout:3/635: stat d7/d1a/d1c/d23/c3f 0 2024-09-16T00:26:07.761 INFO:tasks.workunit.client.1.smithi169.stdout:1/574: dwrite dc/d36/d3a/d45/f17 [0,4194304] 0 2024-09-16T00:26:07.761 INFO:tasks.workunit.client.1.smithi169.stdout:3/636: stat d7/d1a 0 2024-09-16T00:26:07.762 INFO:tasks.workunit.client.1.smithi169.stdout:7/620: dwrite d7/d5b/f77 [0,4194304] 0 2024-09-16T00:26:07.763 INFO:tasks.workunit.client.0.smithi112.stdout:3/809: symlink d2/d9/d2b/d47/d52/d7a/d80/d83/d68/d90/l106 0 2024-09-16T00:26:07.764 INFO:tasks.workunit.client.0.smithi112.stdout:3/810: chown d2/d4d/lc4 2 1 2024-09-16T00:26:07.765 INFO:tasks.workunit.client.0.smithi112.stdout:3/811: write d2/d4d/fb3 [2936064,69949] 0 2024-09-16T00:26:07.766 INFO:tasks.workunit.client.0.smithi112.stdout:6/714: creat dd/df/d17/da2/fea x:0 0 0 2024-09-16T00:26:07.766 INFO:tasks.workunit.client.1.smithi169.stdout:3/637: symlink d7/d50/d90/dba/dc0/le6 0 2024-09-16T00:26:07.767 INFO:tasks.workunit.client.0.smithi112.stdout:6/715: readlink dd/d15/lb5 0 2024-09-16T00:26:07.768 INFO:tasks.workunit.client.0.smithi112.stdout:6/716: readlink dd/df/d11/d44/d8a/l9a 0 2024-09-16T00:26:07.768 INFO:tasks.workunit.client.0.smithi112.stdout:0/843: unlink d5/da/d18/d63/db2/cfb 0 2024-09-16T00:26:07.769 INFO:tasks.workunit.client.0.smithi112.stdout:6/717: readlink dd/df/d11/d44/d8a/l9a 0 2024-09-16T00:26:07.770 INFO:tasks.workunit.client.1.smithi169.stdout:1/575: dwrite dc/f1d [0,4194304] 0 2024-09-16T00:26:07.770 INFO:tasks.workunit.client.0.smithi112.stdout:4/774: creat d6/d39/d1c/d2c/f11c x:0 0 0 2024-09-16T00:26:07.773 INFO:tasks.workunit.client.1.smithi169.stdout:1/576: symlink dc/d2c/lab 0 2024-09-16T00:26:07.775 INFO:tasks.workunit.client.1.smithi169.stdout:3/638: dwrite d7/d1a/f32 [4194304,4194304] 0 2024-09-16T00:26:07.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:07 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:26:07.777 INFO:tasks.workunit.client.0.smithi112.stdout:1/840: truncate d0/f9a 539097 0 2024-09-16T00:26:07.777 INFO:tasks.workunit.client.0.smithi112.stdout:5/677: truncate d4/d4f/d77/f6c 903280 0 2024-09-16T00:26:07.779 INFO:tasks.workunit.client.0.smithi112.stdout:8/675: truncate d10/d12/d1c/f38 1462384 0 2024-09-16T00:26:07.782 INFO:tasks.workunit.client.0.smithi112.stdout:0/844: symlink d5/da/d18/d24/l101 0 2024-09-16T00:26:07.785 INFO:tasks.workunit.client.1.smithi169.stdout:3/639: creat d7/d1a/d1c/d23/fe7 x:0 0 0 2024-09-16T00:26:07.785 INFO:tasks.workunit.client.0.smithi112.stdout:5/678: mkdir d4/dd/d49/ddb 0 2024-09-16T00:26:07.786 INFO:tasks.workunit.client.1.smithi169.stdout:7/621: truncate d7/d71/d76/d9c/fa7 192293 0 2024-09-16T00:26:07.786 INFO:tasks.workunit.client.0.smithi112.stdout:1/841: rmdir d0/d14/df/d28/d45/d46 39 2024-09-16T00:26:07.787 INFO:tasks.workunit.client.1.smithi169.stdout:3/640: symlink d7/d1a/le8 0 2024-09-16T00:26:07.788 INFO:tasks.workunit.client.1.smithi169.stdout:7/622: creat d7/d71/fc9 x:0 0 0 2024-09-16T00:26:07.789 INFO:tasks.workunit.client.0.smithi112.stdout:4/775: dwrite d6/d39/d1c/d66/fac [4194304,4194304] 0 2024-09-16T00:26:07.790 INFO:tasks.workunit.client.0.smithi112.stdout:0/845: creat d5/da/d1d/d71/d64/dea/f102 x:0 0 0 2024-09-16T00:26:07.792 INFO:tasks.workunit.client.0.smithi112.stdout:6/718: creat dd/df/d10/feb x:0 0 0 2024-09-16T00:26:07.792 INFO:tasks.workunit.client.0.smithi112.stdout:5/679: readlink d4/dd/l91 0 2024-09-16T00:26:07.794 INFO:tasks.workunit.client.0.smithi112.stdout:2/868: getdents d2/dd/d48/d8a/db9/de7 0 2024-09-16T00:26:07.796 INFO:tasks.workunit.client.0.smithi112.stdout:3/812: getdents d2/d9/d2b/d2c/d69/dd5 0 2024-09-16T00:26:07.797 INFO:tasks.workunit.client.0.smithi112.stdout:4/776: write d6/d39/d1c/d66/fac [7918090,90776] 0 2024-09-16T00:26:07.797 INFO:tasks.workunit.client.0.smithi112.stdout:1/842: creat d0/d14/d1c/d20/f112 x:0 0 0 2024-09-16T00:26:07.798 INFO:tasks.workunit.client.0.smithi112.stdout:8/676: dwrite d10/d12/d1c/d61/d6c/dd1/d58/f66 [4194304,4194304] 0 2024-09-16T00:26:07.799 INFO:tasks.workunit.client.0.smithi112.stdout:0/846: creat d5/da/d18/d24/d46/d5f/d7a/f103 x:0 0 0 2024-09-16T00:26:07.801 INFO:tasks.workunit.client.0.smithi112.stdout:3/813: chown d2/d9/d2b/d47/f9d 80 1 2024-09-16T00:26:07.802 INFO:tasks.workunit.client.0.smithi112.stdout:5/680: symlink d4/dd/d49/ddb/ldc 0 2024-09-16T00:26:07.811 INFO:tasks.workunit.client.0.smithi112.stdout:8/677: creat d10/d12/d1c/d61/d6c/dd1/d8c/d98/fe0 x:0 0 0 2024-09-16T00:26:07.814 INFO:tasks.workunit.client.0.smithi112.stdout:1/843: truncate d0/d14/df/d28/d45/d30/f9b 2311051 0 2024-09-16T00:26:07.814 INFO:tasks.workunit.client.0.smithi112.stdout:0/847: creat d5/da/d1d/d71/d64/db6/f104 x:0 0 0 2024-09-16T00:26:07.815 INFO:tasks.workunit.client.0.smithi112.stdout:6/719: dwrite dd/df/d10/d47/d55/f78 [0,4194304] 0 2024-09-16T00:26:07.816 INFO:tasks.workunit.client.0.smithi112.stdout:9/962: dread d1/d5/d6/d2d/f92 [0,4194304] 0 2024-09-16T00:26:07.819 INFO:tasks.workunit.client.1.smithi169.stdout:3/641: write d7/d1a/d2b/f69 [463939,117237] 0 2024-09-16T00:26:07.820 INFO:tasks.workunit.client.0.smithi112.stdout:3/814: rename d2/d9/d2b/d2c to d2/d9/d2b/d47/d52/d7a/d80/d83/d7b/dad/d107 0 2024-09-16T00:26:07.820 INFO:tasks.workunit.client.0.smithi112.stdout:2/869: dwrite d2/dd/d48/d8a/ff7 [0,4194304] 0 2024-09-16T00:26:07.822 INFO:tasks.workunit.client.1.smithi169.stdout:3/642: rmdir d7/d50/d6f 39 2024-09-16T00:26:07.823 INFO:tasks.workunit.client.0.smithi112.stdout:0/848: dread - d5/da/d21/d6f/ff9 zero size 2024-09-16T00:26:07.825 INFO:tasks.workunit.client.1.smithi169.stdout:3/643: write d7/d50/d6f/f74 [1488595,86926] 0 2024-09-16T00:26:07.827 INFO:tasks.workunit.client.0.smithi112.stdout:9/963: fdatasync d1/f62 0 2024-09-16T00:26:07.828 INFO:tasks.workunit.client.1.smithi169.stdout:3/644: dread d7/d1a/d1c/d23/f3d [0,4194304] 0 2024-09-16T00:26:07.829 INFO:tasks.workunit.client.0.smithi112.stdout:6/720: creat dd/df/d17/d27/d54/fec x:0 0 0 2024-09-16T00:26:07.830 INFO:tasks.workunit.client.1.smithi169.stdout:3/645: write d7/d1a/d1c/f25 [4975054,5979] 0 2024-09-16T00:26:07.831 INFO:tasks.workunit.client.0.smithi112.stdout:0/849: mkdir d5/d90/d105 0 2024-09-16T00:26:07.832 INFO:tasks.workunit.client.1.smithi169.stdout:7/623: dread d7/d16/f6a [0,4194304] 0 2024-09-16T00:26:07.833 INFO:tasks.workunit.client.0.smithi112.stdout:5/681: rename d4/dd/d6f/fab to d4/d7/d14/d90/dd3/fdd 0 2024-09-16T00:26:07.836 INFO:tasks.workunit.client.1.smithi169.stdout:3/646: getdents d7/d1a/d1c/d35 0 2024-09-16T00:26:07.836 INFO:tasks.workunit.client.1.smithi169.stdout:7/624: truncate d7/d16/d49/d74/f7c 3515582 0 2024-09-16T00:26:07.836 INFO:tasks.workunit.client.1.smithi169.stdout:7/625: readlink d7/d16/l80 0 2024-09-16T00:26:07.837 INFO:tasks.workunit.client.0.smithi112.stdout:8/678: creat d10/d12/d1c/fe1 x:0 0 0 2024-09-16T00:26:07.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:07 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:26:07.839 INFO:tasks.workunit.client.0.smithi112.stdout:2/870: dwrite d2/dd/d48/d22/d2b/d54/d79/d110/d49/fba [0,4194304] 0 2024-09-16T00:26:07.839 INFO:tasks.workunit.client.1.smithi169.stdout:7/626: read d7/d5b/dc1/f8c [2047052,25217] 0 2024-09-16T00:26:07.840 INFO:tasks.workunit.client.0.smithi112.stdout:2/871: chown d2/dd/d48/d22/d2b/d54/d60 461396 1 2024-09-16T00:26:07.841 INFO:tasks.workunit.client.1.smithi169.stdout:3/647: link d7/d1a/d1c/d35/d63/da3/cb5 d7/d1a/d1c/d23/d88/dbe/ce9 0 2024-09-16T00:26:07.841 INFO:tasks.workunit.client.0.smithi112.stdout:6/721: mknod dd/d15/da5/dae/ced 0 2024-09-16T00:26:07.841 INFO:tasks.workunit.client.0.smithi112.stdout:2/872: chown d2/dd/c95 8634 1 2024-09-16T00:26:07.842 INFO:tasks.workunit.client.1.smithi169.stdout:7/627: mknod d7/d16/d49/d84/dc8/cca 0 2024-09-16T00:26:07.843 INFO:tasks.workunit.client.0.smithi112.stdout:4/777: write d6/d39/d36/d5c/fe7 [334128,68048] 0 2024-09-16T00:26:07.843 INFO:tasks.workunit.client.1.smithi169.stdout:3/648: mkdir d7/d1a/d1c/d35/d63/dea 0 2024-09-16T00:26:07.845 INFO:tasks.workunit.client.0.smithi112.stdout:3/815: getdents d2/d9/d2b/d47/d52/d7a/df3 0 2024-09-16T00:26:07.847 INFO:tasks.workunit.client.1.smithi169.stdout:3/649: creat d7/d1a/feb x:0 0 0 2024-09-16T00:26:07.849 INFO:tasks.workunit.client.0.smithi112.stdout:2/873: read d2/f64 [1806683,109106] 0 2024-09-16T00:26:07.850 INFO:tasks.workunit.client.1.smithi169.stdout:7/628: dwrite d7/d71/fae [0,4194304] 0 2024-09-16T00:26:07.851 INFO:tasks.workunit.client.0.smithi112.stdout:5/682: mknod d4/dd/d49/d70/d83/dbd/cde 0 2024-09-16T00:26:07.852 INFO:tasks.workunit.client.0.smithi112.stdout:6/722: dwrite dd/df/d13/d18/f28 [0,4194304] 0 2024-09-16T00:26:07.854 INFO:tasks.workunit.client.1.smithi169.stdout:3/650: mknod d7/d1a/d1c/d23/d88/cec 0 2024-09-16T00:26:07.857 INFO:tasks.workunit.client.0.smithi112.stdout:9/964: link d1/d5/d6/d2d/ccc d1/d5/d6/d2d/d6a/c151 0 2024-09-16T00:26:07.861 INFO:tasks.workunit.client.0.smithi112.stdout:4/778: link d6/db/f60 d6/db/f11d 0 2024-09-16T00:26:07.862 INFO:tasks.workunit.client.0.smithi112.stdout:5/683: creat d4/dd/d49/d70/fdf x:0 0 0 2024-09-16T00:26:07.863 INFO:tasks.workunit.client.1.smithi169.stdout:3/651: dwrite d7/d50/f87 [4194304,4194304] 0 2024-09-16T00:26:07.864 INFO:tasks.workunit.client.1.smithi169.stdout:0/557: sync 2024-09-16T00:26:07.865 INFO:tasks.workunit.client.0.smithi112.stdout:4/779: write d6/db/d1f/d41/ddf/f104 [611076,115933] 0 2024-09-16T00:26:07.866 INFO:tasks.workunit.client.1.smithi169.stdout:0/558: dread - d0/d2/d8/fb7 zero size 2024-09-16T00:26:07.867 INFO:tasks.workunit.client.0.smithi112.stdout:6/723: creat dd/d15/da5/d79/fee x:0 0 0 2024-09-16T00:26:07.867 INFO:tasks.workunit.client.1.smithi169.stdout:3/652: mknod d7/d50/d90/dba/ced 0 2024-09-16T00:26:07.870 INFO:tasks.workunit.client.0.smithi112.stdout:3/816: dwrite d2/d9/d2b/d47/d52/d7a/d80/d83/d7b/dad/d107/ff1 [0,4194304] 0 2024-09-16T00:26:07.870 INFO:tasks.workunit.client.0.smithi112.stdout:9/965: truncate d1/d5/d20/d81/f98 575592 0 2024-09-16T00:26:07.871 INFO:tasks.workunit.client.1.smithi169.stdout:0/559: symlink d0/d2/d8/d5/lc4 0 2024-09-16T00:26:07.872 INFO:tasks.workunit.client.0.smithi112.stdout:9/966: chown d1/d5/d6/d59/d64/l100 1423 1 2024-09-16T00:26:07.873 INFO:tasks.workunit.client.1.smithi169.stdout:3/653: symlink d7/d1a/d1c/d23/d88/d9b/de2/lee 0 2024-09-16T00:26:07.874 INFO:tasks.workunit.client.0.smithi112.stdout:3/817: chown d2/d9/d2b/d47/d52/d7a/d80/d83/d7b/dad/d107/d69/d92/l101 1448232 1 2024-09-16T00:26:07.875 INFO:tasks.workunit.client.1.smithi169.stdout:0/560: mkdir d0/d2/d15/dc5 0 2024-09-16T00:26:07.876 INFO:tasks.workunit.client.1.smithi169.stdout:3/654: rmdir d7/dbc 39 2024-09-16T00:26:07.876 INFO:tasks.workunit.client.0.smithi112.stdout:2/874: creat d2/f11c x:0 0 0 2024-09-16T00:26:07.877 INFO:tasks.workunit.client.1.smithi169.stdout:3/655: chown d7/d1a/d1c/d23/cbd 420 1 2024-09-16T00:26:07.878 INFO:tasks.workunit.client.0.smithi112.stdout:4/780: mkdir d6/d39/d36/d5c/d7e/d8f/d11e 0 2024-09-16T00:26:07.879 INFO:tasks.workunit.client.0.smithi112.stdout:2/875: write d2/dd/d48/d22/f8f [168234,61157] 0 2024-09-16T00:26:07.880 INFO:tasks.workunit.client.1.smithi169.stdout:0/561: rename d0/d2/d8/fb7 to d0/d2c/d38/fc6 0 2024-09-16T00:26:07.880 INFO:tasks.workunit.client.0.smithi112.stdout:5/684: link d4/d7/d14/d90/fb5 d4/dd/d49/d70/d83/dbd/fe0 0 2024-09-16T00:26:07.880 INFO:tasks.workunit.client.0.smithi112.stdout:0/850: truncate f0 3732532 0 2024-09-16T00:26:07.882 INFO:tasks.workunit.client.1.smithi169.stdout:0/562: unlink d0/d2c/d54/d21/d31/cc3 0 2024-09-16T00:26:07.882 INFO:tasks.workunit.client.0.smithi112.stdout:8/679: dwrite d10/d12/d1c/d3b/d74/f7b [0,4194304] 0 2024-09-16T00:26:07.885 INFO:tasks.workunit.client.1.smithi169.stdout:0/563: rmdir d0/d2/d8/d5/d79 39 2024-09-16T00:26:07.887 INFO:tasks.workunit.client.0.smithi112.stdout:2/876: dread d2/dd/f87 [0,4194304] 0 2024-09-16T00:26:07.888 INFO:tasks.workunit.client.0.smithi112.stdout:3/818: mkdir d2/d9/d2b/d47/d52/d7a/d80/d83/d7b/dad/d107/d108 0 2024-09-16T00:26:07.888 INFO:tasks.workunit.client.0.smithi112.stdout:2/877: fsync d2/dd/d48/f85 0 2024-09-16T00:26:07.889 INFO:tasks.workunit.client.1.smithi169.stdout:3/656: dwrite d7/d1a/d2b/d8d/fab [0,4194304] 0 2024-09-16T00:26:07.889 INFO:tasks.workunit.client.1.smithi169.stdout:7/629: dread d7/df/d95/f7a [0,4194304] 0 2024-09-16T00:26:07.891 INFO:tasks.workunit.client.0.smithi112.stdout:0/851: rename d5/da/d18/d24/d5a to d5/da/d1d/d71/d64/d106 0 2024-09-16T00:26:07.892 INFO:tasks.workunit.client.0.smithi112.stdout:8/680: mknod d10/d12/d1c/d3b/d7d/ce2 0 2024-09-16T00:26:07.893 INFO:tasks.workunit.client.0.smithi112.stdout:9/967: creat d1/d5/d6/d59/f152 x:0 0 0 2024-09-16T00:26:07.893 INFO:tasks.workunit.client.1.smithi169.stdout:0/564: truncate d0/d2c/d54/d21/d31/fbe 472185 0 2024-09-16T00:26:07.894 INFO:tasks.workunit.client.0.smithi112.stdout:6/724: getdents dd 0 2024-09-16T00:26:07.895 INFO:tasks.workunit.client.1.smithi169.stdout:0/565: fsync d0/d50/f70 0 2024-09-16T00:26:07.897 INFO:tasks.workunit.client.0.smithi112.stdout:5/685: creat d4/dd/d49/d70/dc5/fe1 x:0 0 0 2024-09-16T00:26:07.897 INFO:tasks.workunit.client.1.smithi169.stdout:3/657: symlink d7/d1a/d2b/dc6/lef 0 2024-09-16T00:26:07.898 INFO:tasks.workunit.client.0.smithi112.stdout:3/819: mknod d2/d9/d2b/d47/d52/d7a/d80/db4/c109 0 2024-09-16T00:26:07.898 INFO:tasks.workunit.client.0.smithi112.stdout:2/878: dwrite d2/dd/d48/d22/d2b/d54/d79/f7a [0,4194304] 0 2024-09-16T00:26:07.899 INFO:tasks.workunit.client.1.smithi169.stdout:0/566: dread - d0/d7d/fb1 zero size 2024-09-16T00:26:07.900 INFO:tasks.workunit.client.0.smithi112.stdout:0/852: mknod d5/d90/dc5/c107 0 2024-09-16T00:26:07.900 INFO:tasks.workunit.client.1.smithi169.stdout:7/630: dwrite d7/d16/d49/f58 [4194304,4194304] 0 2024-09-16T00:26:07.903 INFO:tasks.workunit.client.0.smithi112.stdout:8/681: symlink d10/d12/d1c/d61/d6c/dd1/d58/d87/le3 0 2024-09-16T00:26:07.905 INFO:tasks.workunit.client.0.smithi112.stdout:4/781: getdents d6/db/d1f/d41/d49 0 2024-09-16T00:26:07.907 INFO:tasks.workunit.client.0.smithi112.stdout:5/686: creat d4/d7/d26/d4c/fe2 x:0 0 0 2024-09-16T00:26:07.907 INFO:tasks.workunit.client.0.smithi112.stdout:4/782: chown d6/d39 1378 1 2024-09-16T00:26:07.909 INFO:tasks.workunit.client.0.smithi112.stdout:3/820: fdatasync d2/d9/d2b/d47/d52/d7a/ff9 0 2024-09-16T00:26:07.910 INFO:tasks.workunit.client.0.smithi112.stdout:6/725: creat dd/df/d10/d47/d55/dbd/dcb/de8/fef x:0 0 0 2024-09-16T00:26:07.911 INFO:tasks.workunit.client.0.smithi112.stdout:9/968: creat d1/d5/d6/d2d/f153 x:0 0 0 2024-09-16T00:26:07.911 INFO:tasks.workunit.client.0.smithi112.stdout:0/853: dwrite d5/f2c [4194304,4194304] 0 2024-09-16T00:26:07.913 INFO:tasks.workunit.client.0.smithi112.stdout:8/682: mkdir d10/d12/d1c/d3b/d7d/de4 0 2024-09-16T00:26:07.916 INFO:tasks.workunit.client.0.smithi112.stdout:6/726: symlink dd/df/d10/d47/d55/dbd/lf0 0 2024-09-16T00:26:07.917 INFO:tasks.workunit.client.0.smithi112.stdout:8/683: read - d10/d12/d1c/d3b/fa8 zero size 2024-09-16T00:26:07.919 INFO:tasks.workunit.client.0.smithi112.stdout:4/783: rename d6/db/f88 to d6/d39/d1c/d6e/d119/f11f 0 2024-09-16T00:26:07.921 INFO:tasks.workunit.client.0.smithi112.stdout:0/854: symlink d5/da/l108 0 2024-09-16T00:26:07.926 INFO:tasks.workunit.client.0.smithi112.stdout:4/784: unlink d6/db/d1f/d94/cc3 0 2024-09-16T00:26:07.930 INFO:tasks.workunit.client.0.smithi112.stdout:5/687: dwrite d4/d7/d26/d4c/fe2 [0,4194304] 0 2024-09-16T00:26:07.930 INFO:tasks.workunit.client.0.smithi112.stdout:9/969: link d1/d5/d6/d18/c10d d1/d5/d6/d107/c154 0 2024-09-16T00:26:07.932 INFO:tasks.workunit.client.0.smithi112.stdout:4/785: chown d6/db/f60 27843122 1 2024-09-16T00:26:07.932 INFO:tasks.workunit.client.0.smithi112.stdout:3/821: dwrite d2/d9/d2b/d47/d52/d7a/d80/d83/d76/d8e/ff0 [0,4194304] 0 2024-09-16T00:26:07.933 INFO:tasks.workunit.client.0.smithi112.stdout:6/727: dwrite dd/d15/da5/d79/fee [0,4194304] 0 2024-09-16T00:26:07.934 INFO:tasks.workunit.client.1.smithi169.stdout:7/631: dwrite d7/f9f [0,4194304] 0 2024-09-16T00:26:07.934 INFO:tasks.workunit.client.1.smithi169.stdout:0/567: dwrite d0/d2c/f5d [0,4194304] 0 2024-09-16T00:26:07.935 INFO:tasks.workunit.client.1.smithi169.stdout:7/632: stat d7/d16/d49/c55 0 2024-09-16T00:26:07.939 INFO:tasks.workunit.client.1.smithi169.stdout:3/658: dread d7/d1a/d1c/fb6 [0,4194304] 0 2024-09-16T00:26:07.940 INFO:tasks.workunit.client.0.smithi112.stdout:8/684: creat d10/d12/d1c/d61/d6c/dd1/fe5 x:0 0 0 2024-09-16T00:26:07.940 INFO:tasks.workunit.client.1.smithi169.stdout:7/633: creat d7/d71/d76/d29/d78/dba/fcb x:0 0 0 2024-09-16T00:26:07.942 INFO:tasks.workunit.client.0.smithi112.stdout:8/685: readlink d10/d12/d1c/dd0/l44 0 2024-09-16T00:26:07.943 INFO:tasks.workunit.client.0.smithi112.stdout:3/822: truncate d2/d9/d2b/d47/dcf/ff5 351371 0 2024-09-16T00:26:07.944 INFO:tasks.workunit.client.0.smithi112.stdout:8/686: dread d10/d12/fb9 [0,4194304] 0 2024-09-16T00:26:07.945 INFO:tasks.workunit.client.0.smithi112.stdout:6/728: creat dd/d23/ff1 x:0 0 0 2024-09-16T00:26:07.946 INFO:tasks.workunit.client.1.smithi169.stdout:0/568: dwrite d0/d2/d8/f68 [4194304,4194304] 0 2024-09-16T00:26:07.947 INFO:tasks.workunit.client.0.smithi112.stdout:4/786: rename d6/db/d1f/d94/feb to d6/d39/d36/d5c/d7e/dab/de4/f120 0 2024-09-16T00:26:07.951 INFO:tasks.workunit.client.0.smithi112.stdout:3/823: mkdir d2/d9/d2b/d47/d52/d7a/d80/d83/d68/d10a 0 2024-09-16T00:26:07.952 INFO:tasks.workunit.client.1.smithi169.stdout:3/659: dwrite d7/d1a/d1c/d35/d63/d37/f9a [0,4194304] 0 2024-09-16T00:26:07.952 INFO:tasks.workunit.client.1.smithi169.stdout:0/569: write d0/d2c/f5d [423090,45775] 0 2024-09-16T00:26:07.954 INFO:tasks.workunit.client.1.smithi169.stdout:3/660: write d7/d50/d6f/fb4 [1196059,6781] 0 2024-09-16T00:26:07.956 INFO:tasks.workunit.client.1.smithi169.stdout:3/661: readlink d7/d50/d90/lb0 0 2024-09-16T00:26:07.960 INFO:tasks.workunit.client.0.smithi112.stdout:8/687: creat d10/d12/d1c/d61/d6c/dd1/d8c/d98/fe6 x:0 0 0 2024-09-16T00:26:07.961 INFO:tasks.workunit.client.0.smithi112.stdout:6/729: creat dd/df/d17/d27/d84/ff2 x:0 0 0 2024-09-16T00:26:07.962 INFO:tasks.workunit.client.1.smithi169.stdout:3/662: chown d7/d1a/d2b/l57 600181 1 2024-09-16T00:26:07.962 INFO:tasks.workunit.client.0.smithi112.stdout:2/879: truncate d2/dd/d48/d8a/d9b/fdc 614274 0 2024-09-16T00:26:07.963 INFO:tasks.workunit.client.0.smithi112.stdout:2/880: stat d2/dd/d48/f85 0 2024-09-16T00:26:07.963 INFO:tasks.workunit.client.0.smithi112.stdout:4/787: dwrite d6/d39/d1c/f57 [0,4194304] 0 2024-09-16T00:26:07.964 INFO:tasks.workunit.client.1.smithi169.stdout:3/663: dread - d7/d50/d90/da6/fd9 zero size 2024-09-16T00:26:07.966 INFO:tasks.workunit.client.1.smithi169.stdout:0/570: link d0/d2c/d54/f1c d0/d2c/d54/d21/d31/fc7 0 2024-09-16T00:26:07.967 INFO:tasks.workunit.client.1.smithi169.stdout:3/664: mkdir d7/d50/d90/dba/df0 0 2024-09-16T00:26:07.969 INFO:tasks.workunit.client.1.smithi169.stdout:3/665: write d7/d1a/d2b/f69 [476624,92767] 0 2024-09-16T00:26:07.970 INFO:tasks.workunit.client.0.smithi112.stdout:6/730: dread dd/df/d11/fad [0,4194304] 0 2024-09-16T00:26:07.971 INFO:tasks.workunit.client.1.smithi169.stdout:0/571: rename d0/d4c/f87 to d0/d2c/d54/d21/d31/d83/fc8 0 2024-09-16T00:26:07.973 INFO:tasks.workunit.client.0.smithi112.stdout:0/855: dread d5/da/d57/d76/f14 [0,4194304] 0 2024-09-16T00:26:07.976 INFO:tasks.workunit.client.1.smithi169.stdout:0/572: creat d0/d2/d8/d5/fc9 x:0 0 0 2024-09-16T00:26:07.977 INFO:tasks.workunit.client.0.smithi112.stdout:6/731: dwrite dd/df/d10/d47/f57 [0,4194304] 0 2024-09-16T00:26:07.979 INFO:tasks.workunit.client.0.smithi112.stdout:6/732: truncate dd/df/d91/fda 1044998 0 2024-09-16T00:26:07.980 INFO:tasks.workunit.client.0.smithi112.stdout:6/733: fdatasync dd/df/d10/d47/d55/dbd/dcb/de8/fef 0 2024-09-16T00:26:07.980 INFO:tasks.workunit.client.1.smithi169.stdout:0/573: write d0/d2c/d54/f61 [942489,10296] 0 2024-09-16T00:26:07.981 INFO:tasks.workunit.client.1.smithi169.stdout:0/574: read - d0/d2c/d54/d21/d31/f9e zero size 2024-09-16T00:26:07.986 INFO:tasks.workunit.client.1.smithi169.stdout:0/575: creat d0/d2/d4b/d86/fca x:0 0 0 2024-09-16T00:26:07.991 INFO:tasks.workunit.client.0.smithi112.stdout:8/688: unlink d10/l14 0 2024-09-16T00:26:07.994 INFO:tasks.workunit.client.0.smithi112.stdout:6/734: stat dd/df/d13/d18/l5d 0 2024-09-16T00:26:07.994 INFO:tasks.workunit.client.0.smithi112.stdout:3/824: mkdir d2/d9/d2b/d47/d52/d7a/d80/d83/d10b 0 2024-09-16T00:26:07.996 INFO:tasks.workunit.client.0.smithi112.stdout:0/856: symlink d5/d90/d105/l109 0 2024-09-16T00:26:07.999 INFO:tasks.workunit.client.0.smithi112.stdout:8/689: dwrite d10/d12/d1c/d61/d6c/fc4 [0,4194304] 0 2024-09-16T00:26:08.001 INFO:tasks.workunit.client.0.smithi112.stdout:0/857: unlink d5/da/d66/fc9 0 2024-09-16T00:26:08.002 INFO:tasks.workunit.client.0.smithi112.stdout:8/690: creat d10/d12/d1c/dd0/fe7 x:0 0 0 2024-09-16T00:26:08.002 INFO:tasks.workunit.client.0.smithi112.stdout:3/825: mknod d2/d9/d2b/d47/d52/d7a/d80/d83/d7b/dad/d107/d69/dd5/d105/c10c 0 2024-09-16T00:26:08.003 INFO:tasks.workunit.client.0.smithi112.stdout:0/858: mkdir d5/da/d1d/d10a 0 2024-09-16T00:26:08.005 INFO:tasks.workunit.client.0.smithi112.stdout:0/859: write d5/da/d1d/d71/f3b [5774406,95078] 0 2024-09-16T00:26:08.009 INFO:tasks.workunit.client.0.smithi112.stdout:3/826: dread d2/d9/d2b/d47/d52/d7a/d80/d83/d49/f77 [0,4194304] 0 2024-09-16T00:26:08.011 INFO:tasks.workunit.client.0.smithi112.stdout:8/691: dwrite d10/d12/d1c/d3b/fa8 [0,4194304] 0 2024-09-16T00:26:08.014 INFO:tasks.workunit.client.0.smithi112.stdout:3/827: mkdir d2/d9/d2b/d47/d52/d7a/d80/db4/d10d 0 2024-09-16T00:26:08.016 INFO:tasks.workunit.client.0.smithi112.stdout:0/860: dwrite d5/da/d18/d63/db2/fca [0,4194304] 0 2024-09-16T00:26:08.018 INFO:tasks.workunit.client.0.smithi112.stdout:0/861: symlink d5/d90/dc5/de3/l10b 0 2024-09-16T00:26:08.019 INFO:tasks.workunit.client.0.smithi112.stdout:0/862: chown d5/da/d66 1 1 2024-09-16T00:26:08.020 INFO:tasks.workunit.client.0.smithi112.stdout:0/863: read - d5/da/d1d/d71/fed zero size 2024-09-16T00:26:08.021 INFO:tasks.workunit.client.0.smithi112.stdout:0/864: write d5/da/d18/d89/fab [376497,70018] 0 2024-09-16T00:26:08.023 INFO:tasks.workunit.client.0.smithi112.stdout:0/865: write d5/da/d18/d24/d46/d5f/d7a/fa2 [435900,115256] 0 2024-09-16T00:26:08.025 INFO:tasks.workunit.client.0.smithi112.stdout:0/866: write d5/da/d1d/d71/fbc [1541683,96936] 0 2024-09-16T00:26:08.026 INFO:tasks.workunit.client.0.smithi112.stdout:0/867: dread d5/da/d1d/d71/d64/f8b [0,4194304] 0 2024-09-16T00:26:08.030 INFO:tasks.workunit.client.0.smithi112.stdout:0/868: unlink d5/da/d1d/d71/d64/dea/cf5 0 2024-09-16T00:26:08.031 INFO:tasks.workunit.client.0.smithi112.stdout:0/869: symlink d5/da/d1d/d71/d64/db6/l10c 0 2024-09-16T00:26:08.033 INFO:tasks.workunit.client.0.smithi112.stdout:0/870: symlink d5/da/d18/d63/db2/dd4/dff/l10d 0 2024-09-16T00:26:08.035 INFO:tasks.workunit.client.0.smithi112.stdout:0/871: mknod d5/da/dc/c10e 0 2024-09-16T00:26:08.037 INFO:tasks.workunit.client.0.smithi112.stdout:0/872: getdents d5/da/d57/d76 0 2024-09-16T00:26:08.038 INFO:tasks.workunit.client.0.smithi112.stdout:0/873: symlink d5/d90/d105/l10f 0 2024-09-16T00:26:08.040 INFO:tasks.workunit.client.0.smithi112.stdout:0/874: mkdir d5/da/d66/d110 0 2024-09-16T00:26:08.040 INFO:tasks.workunit.client.0.smithi112.stdout:0/875: dread - d5/da/d18/d24/d46/d5f/d7a/f103 zero size 2024-09-16T00:26:08.046 INFO:tasks.workunit.client.0.smithi112.stdout:0/876: dwrite d5/da/d18/d89/f9b [0,4194304] 0 2024-09-16T00:26:08.049 INFO:tasks.workunit.client.0.smithi112.stdout:0/877: creat d5/da/d21/f111 x:0 0 0 2024-09-16T00:26:08.050 INFO:tasks.workunit.client.0.smithi112.stdout:0/878: creat d5/da/d1d/d71/d64/d106/f112 x:0 0 0 2024-09-16T00:26:08.052 INFO:tasks.workunit.client.0.smithi112.stdout:0/879: creat d5/d90/dc5/de2/f113 x:0 0 0 2024-09-16T00:26:08.059 INFO:tasks.workunit.client.0.smithi112.stdout:0/880: dwrite d5/da/d1d/d71/d64/d106/ffc [0,4194304] 0 2024-09-16T00:26:08.061 INFO:tasks.workunit.client.0.smithi112.stdout:0/881: creat d5/d90/dc5/dd7/f114 x:0 0 0 2024-09-16T00:26:08.062 INFO:tasks.workunit.client.0.smithi112.stdout:0/882: chown d5/da/l108 0 1 2024-09-16T00:26:08.065 INFO:tasks.workunit.client.0.smithi112.stdout:0/883: link d5/d90/dc5/de3/l10b d5/da/d18/d24/d46/d5f/da6/l115 0 2024-09-16T00:26:08.065 INFO:tasks.workunit.client.0.smithi112.stdout:0/884: stat d5/da/d18/d63/fa3 0 2024-09-16T00:26:08.068 INFO:tasks.workunit.client.1.smithi169.stdout:7/634: write d7/d16/d49/d74/fb3 [4246805,106061] 0 2024-09-16T00:26:08.073 INFO:tasks.workunit.client.0.smithi112.stdout:9/970: write d1/d5/d6/d2d/d6a/d89/ff6 [52570,90145] 0 2024-09-16T00:26:08.075 INFO:tasks.workunit.client.0.smithi112.stdout:9/971: dread - d1/d5/d20/d81/d1a/d88/f14b zero size 2024-09-16T00:26:08.076 INFO:tasks.workunit.client.1.smithi169.stdout:7/635: link d7/d16/d49/d84/d8e/lb8 d7/df/d95/lcc 0 2024-09-16T00:26:08.076 INFO:tasks.workunit.client.1.smithi169.stdout:3/666: write d7/d1a/d1c/d23/f3a [8359004,77831] 0 2024-09-16T00:26:08.086 INFO:tasks.workunit.client.1.smithi169.stdout:0/576: dwrite d0/d2/d15/f22 [0,4194304] 0 2024-09-16T00:26:08.086 INFO:tasks.workunit.client.1.smithi169.stdout:3/667: dwrite d7/d50/fd5 [0,4194304] 0 2024-09-16T00:26:08.089 INFO:tasks.workunit.client.1.smithi169.stdout:0/577: read d0/d2c/d54/d21/f49 [1286718,92287] 0 2024-09-16T00:26:08.092 INFO:tasks.workunit.client.1.smithi169.stdout:0/578: truncate d0/d2c/d4f/da5/fa6 578699 0 2024-09-16T00:26:08.096 INFO:tasks.workunit.client.1.smithi169.stdout:3/668: dwrite d7/d1a/d1c/d35/d63/fcb [0,4194304] 0 2024-09-16T00:26:08.102 INFO:tasks.workunit.client.1.smithi169.stdout:7/636: dwrite d7/d16/f90 [0,4194304] 0 2024-09-16T00:26:08.103 INFO:tasks.workunit.client.0.smithi112.stdout:4/788: dwrite d6/db/d1f/d41/d49/f17 [0,4194304] 0 2024-09-16T00:26:08.104 INFO:tasks.workunit.client.1.smithi169.stdout:3/669: creat d7/d50/d90/dba/df0/ff1 x:0 0 0 2024-09-16T00:26:08.105 INFO:tasks.workunit.client.0.smithi112.stdout:4/789: chown d6/d39/cb9 2303 1 2024-09-16T00:26:08.106 INFO:tasks.workunit.client.1.smithi169.stdout:3/670: fsync d7/d50/d6f/f74 0 2024-09-16T00:26:08.106 INFO:tasks.workunit.client.0.smithi112.stdout:6/735: dwrite dd/df/d10/d47/d55/dc4/dd1/f58 [0,4194304] 0 2024-09-16T00:26:08.108 INFO:tasks.workunit.client.1.smithi169.stdout:7/637: rename d7/d16/d49/d74/f7f to d7/d71/d76/d29/d78/dba/fcd 0 2024-09-16T00:26:08.110 INFO:tasks.workunit.client.1.smithi169.stdout:3/671: creat d7/d1a/d2b/dc6/ff2 x:0 0 0 2024-09-16T00:26:08.111 INFO:tasks.workunit.client.0.smithi112.stdout:4/790: mknod d6/d39/d36/d5c/dc6/c121 0 2024-09-16T00:26:08.111 INFO:tasks.workunit.client.1.smithi169.stdout:3/672: chown d7/d50/f87 67850705 1 2024-09-16T00:26:08.115 INFO:tasks.workunit.client.0.smithi112.stdout:6/736: mknod dd/df/d91/dd7/cf3 0 2024-09-16T00:26:08.121 INFO:tasks.workunit.client.0.smithi112.stdout:4/791: mknod d6/db/df/d3c/c122 0 2024-09-16T00:26:08.121 INFO:tasks.workunit.client.0.smithi112.stdout:0/885: dwrite d5/da/d18/d63/db2/fca [4194304,4194304] 0 2024-09-16T00:26:08.121 INFO:tasks.workunit.client.0.smithi112.stdout:3/828: dwrite d2/d9/f4e [0,4194304] 0 2024-09-16T00:26:08.121 INFO:tasks.workunit.client.1.smithi169.stdout:7/638: rename d7/d71/d76/c19 to d7/d16/d49/db2/db4/cce 0 2024-09-16T00:26:08.123 INFO:tasks.workunit.client.1.smithi169.stdout:3/673: dwrite d7/d1a/d1c/d35/d63/f4a [0,4194304] 0 2024-09-16T00:26:08.129 INFO:tasks.workunit.client.1.smithi169.stdout:0/579: dread d0/d2c/d38/f66 [0,4194304] 0 2024-09-16T00:26:08.131 INFO:tasks.workunit.client.1.smithi169.stdout:7/639: dread d7/d71/d76/d29/f85 [0,4194304] 0 2024-09-16T00:26:08.132 INFO:tasks.workunit.client.0.smithi112.stdout:6/737: dwrite dd/df/d10/d47/d55/f7d [0,4194304] 0 2024-09-16T00:26:08.134 INFO:tasks.workunit.client.0.smithi112.stdout:8/692: dread d10/d12/d1c/f59 [0,4194304] 0 2024-09-16T00:26:08.135 INFO:tasks.workunit.client.1.smithi169.stdout:7/640: write d7/f91 [3369676,4766] 0 2024-09-16T00:26:08.135 INFO:tasks.workunit.client.0.smithi112.stdout:4/792: mkdir d6/db/d1f/d94/d10c/d123 0 2024-09-16T00:26:08.136 INFO:tasks.workunit.client.1.smithi169.stdout:7/641: stat d7/d71/d76/faa 0 2024-09-16T00:26:08.136 INFO:tasks.workunit.client.0.smithi112.stdout:9/972: write d1/d5/d20/f32 [245573,76336] 0 2024-09-16T00:26:08.136 INFO:tasks.workunit.client.1.smithi169.stdout:0/580: rename d0/d2c/d54/d21/d31/f9e to d0/d2c/d54/fcb 0 2024-09-16T00:26:08.139 INFO:tasks.workunit.client.0.smithi112.stdout:8/693: readlink d10/d12/d1c/d3b/d74/ld9 0 2024-09-16T00:26:08.140 INFO:tasks.workunit.client.0.smithi112.stdout:0/886: link d5/da/d1d/d71/d64/d106/ffc d5/da/d18/d63/f116 0 2024-09-16T00:26:08.141 INFO:tasks.workunit.client.0.smithi112.stdout:4/793: mknod d6/d39/d1c/d6e/c124 0 2024-09-16T00:26:08.142 INFO:tasks.workunit.client.0.smithi112.stdout:0/887: fdatasync d5/da/d66/f99 0 2024-09-16T00:26:08.144 INFO:tasks.workunit.client.0.smithi112.stdout:6/738: dwrite dd/df/d17/d27/d84/ff2 [0,4194304] 0 2024-09-16T00:26:08.146 INFO:tasks.workunit.client.1.smithi169.stdout:0/581: dwrite d0/d2c/d4f/da5/fa6 [0,4194304] 0 2024-09-16T00:26:08.147 INFO:tasks.workunit.client.1.smithi169.stdout:7/642: dwrite d7/d16/d49/d74/fc5 [0,4194304] 0 2024-09-16T00:26:08.148 INFO:tasks.workunit.client.0.smithi112.stdout:6/739: mkdir dd/df/d11/d44/df4 0 2024-09-16T00:26:08.151 INFO:tasks.workunit.client.0.smithi112.stdout:8/694: dread d10/d12/d1c/d61/d6c/dd1/f68 [0,4194304] 0 2024-09-16T00:26:08.154 INFO:tasks.workunit.client.0.smithi112.stdout:8/695: getdents d10/d12/d1c/d96 0 2024-09-16T00:26:08.156 INFO:tasks.workunit.client.1.smithi169.stdout:0/582: creat d0/d2c/d54/d21/d31/d8c/fcc x:0 0 0 2024-09-16T00:26:08.158 INFO:tasks.workunit.client.1.smithi169.stdout:0/583: symlink d0/d7d/lcd 0 2024-09-16T00:26:08.159 INFO:tasks.workunit.client.1.smithi169.stdout:0/584: dread - d0/d7d/fb1 zero size 2024-09-16T00:26:08.162 INFO:tasks.workunit.client.1.smithi169.stdout:0/585: rmdir d0/d2/d8/d5/d79/d7c 39 2024-09-16T00:26:08.164 INFO:tasks.workunit.client.1.smithi169.stdout:0/586: unlink d0/d2/f5f 0 2024-09-16T00:26:08.166 INFO:tasks.workunit.client.1.smithi169.stdout:0/587: fdatasync d0/d2c/d54/d21/d31/f58 0 2024-09-16T00:26:08.167 INFO:tasks.workunit.client.1.smithi169.stdout:0/588: write d0/d2c/d54/d21/f2f [4412180,31845] 0 2024-09-16T00:26:08.171 INFO:tasks.workunit.client.1.smithi169.stdout:0/589: rmdir d0/d2c/d54/d56/d67/daf 0 2024-09-16T00:26:08.172 INFO:tasks.workunit.client.1.smithi169.stdout:0/590: write d0/d50/f70 [670022,107484] 0 2024-09-16T00:26:08.195 INFO:tasks.workunit.client.1.smithi169.stdout:3/674: truncate d7/d1a/d1c/d23/f3a 3048837 0 2024-09-16T00:26:08.195 INFO:tasks.workunit.client.0.smithi112.stdout:1/844: sync 2024-09-16T00:26:08.195 INFO:tasks.workunit.client.0.smithi112.stdout:7/785: sync 2024-09-16T00:26:08.196 INFO:tasks.workunit.client.0.smithi112.stdout:3/829: write d2/d9/fa5 [233173,57833] 0 2024-09-16T00:26:08.196 INFO:tasks.workunit.client.0.smithi112.stdout:0/888: write d5/da/d21/fb9 [477794,67436] 0 2024-09-16T00:26:08.196 INFO:tasks.workunit.client.1.smithi169.stdout:3/675: stat d7/d1a/d1c/d23/d70/fb7 0 2024-09-16T00:26:08.197 INFO:tasks.workunit.client.1.smithi169.stdout:3/676: read d7/d1a/d1c/d6c/f7a [645879,37090] 0 2024-09-16T00:26:08.198 INFO:tasks.workunit.client.0.smithi112.stdout:7/786: dread - d6/d10/d75/d8f/d90/dbc/fe6 zero size 2024-09-16T00:26:08.199 INFO:tasks.workunit.client.0.smithi112.stdout:1/845: dread d0/fe5 [0,4194304] 0 2024-09-16T00:26:08.200 INFO:tasks.workunit.client.1.smithi169.stdout:3/677: mkdir d7/d1a/d1c/d35/d63/dea/df3 0 2024-09-16T00:26:08.200 INFO:tasks.workunit.client.0.smithi112.stdout:3/830: fdatasync d2/d9/d2b/f29 0 2024-09-16T00:26:08.201 INFO:tasks.workunit.client.0.smithi112.stdout:0/889: fdatasync d5/da/d18/d24/fa1 0 2024-09-16T00:26:08.203 INFO:tasks.workunit.client.0.smithi112.stdout:7/787: write d6/d10/d16/d96/dc7/f61 [94856,39049] 0 2024-09-16T00:26:08.203 INFO:tasks.workunit.client.1.smithi169.stdout:3/678: getdents d7/dbc 0 2024-09-16T00:26:08.204 INFO:tasks.workunit.client.0.smithi112.stdout:6/740: write dd/df/d10/d47/d55/dc4/dd1/d88/f9b [523568,21368] 0 2024-09-16T00:26:08.205 INFO:tasks.workunit.client.0.smithi112.stdout:8/696: write d10/d12/d1c/dd0/dbb/fca [3933031,51296] 0 2024-09-16T00:26:08.207 INFO:tasks.workunit.client.1.smithi169.stdout:0/591: dwrite d0/f23 [0,4194304] 0 2024-09-16T00:26:08.207 INFO:tasks.workunit.client.0.smithi112.stdout:1/846: chown d0/d14/df/d28/d45/d46/d58/d62/db0/fb8 138082 1 2024-09-16T00:26:08.207 INFO:tasks.workunit.client.1.smithi169.stdout:3/679: creat d7/d1a/d1c/d35/d63/d37/ff4 x:0 0 0 2024-09-16T00:26:08.209 INFO:tasks.workunit.client.0.smithi112.stdout:3/831: rename d2/d4d/cd7 to d2/d9/d2b/d47/d78/c10e 0 2024-09-16T00:26:08.212 INFO:tasks.workunit.client.0.smithi112.stdout:7/788: rmdir d6/d10/d75/d8f 39 2024-09-16T00:26:08.214 INFO:tasks.workunit.client.0.smithi112.stdout:6/741: mknod dd/df/d10/d47/d55/dbd/de9/cf5 0 2024-09-16T00:26:08.215 INFO:tasks.workunit.client.0.smithi112.stdout:8/697: symlink d10/d12/d1c/d3b/le8 0 2024-09-16T00:26:08.215 INFO:tasks.workunit.client.1.smithi169.stdout:0/592: getdents d0/d7d 0 2024-09-16T00:26:08.216 INFO:tasks.workunit.client.1.smithi169.stdout:3/680: link d7/d50/d6f/c9f d7/d1a/d1c/d23/d70/dc5/cf5 0 2024-09-16T00:26:08.217 INFO:tasks.workunit.client.1.smithi169.stdout:0/593: fsync d0/d2c/d54/d21/d31/fbe 0 2024-09-16T00:26:08.217 INFO:tasks.workunit.client.1.smithi169.stdout:3/681: readlink d7/d1a/d1c/l4d 0 2024-09-16T00:26:08.218 INFO:tasks.workunit.client.0.smithi112.stdout:0/890: truncate d5/da/d18/f9a 656694 0 2024-09-16T00:26:08.218 INFO:tasks.workunit.client.0.smithi112.stdout:8/698: write d10/d12/d1c/d61/d6c/dd1/f6f [134450,69675] 0 2024-09-16T00:26:08.219 INFO:tasks.workunit.client.1.smithi169.stdout:3/682: mkdir d7/d50/d90/da6/df6 0 2024-09-16T00:26:08.219 INFO:tasks.workunit.client.0.smithi112.stdout:7/789: read d6/d10/d16/d96/f9f [185591,72861] 0 2024-09-16T00:26:08.219 INFO:tasks.workunit.client.0.smithi112.stdout:6/742: symlink dd/df/d10/d47/d55/dbd/dcb/de8/lf6 0 2024-09-16T00:26:08.220 INFO:tasks.workunit.client.1.smithi169.stdout:3/683: fsync d7/d1a/d2b/fb3 0 2024-09-16T00:26:08.220 INFO:tasks.workunit.client.0.smithi112.stdout:8/699: write d10/d12/d24/fdd [943915,129657] 0 2024-09-16T00:26:08.221 INFO:tasks.workunit.client.0.smithi112.stdout:3/832: mknod d2/d9/d2b/d47/d52/d7a/d80/d83/d7b/c10f 0 2024-09-16T00:26:08.221 INFO:tasks.workunit.client.1.smithi169.stdout:3/684: creat d7/d1a/d1c/d23/d88/d9b/dc8/ff7 x:0 0 0 2024-09-16T00:26:08.222 INFO:tasks.workunit.client.0.smithi112.stdout:8/700: fdatasync d10/d12/fd7 0 2024-09-16T00:26:08.224 INFO:tasks.workunit.client.1.smithi169.stdout:3/685: mkdir d7/d1a/d1c/d23/d88/d9b/dc8/df8 0 2024-09-16T00:26:08.224 INFO:tasks.workunit.client.0.smithi112.stdout:0/891: mknod d5/da/d18/d24/d46/c117 0 2024-09-16T00:26:08.225 INFO:tasks.workunit.client.1.smithi169.stdout:3/686: truncate d7/f16 8902111 0 2024-09-16T00:26:08.227 INFO:tasks.workunit.client.0.smithi112.stdout:8/701: mkdir d10/de9 0 2024-09-16T00:26:08.229 INFO:tasks.workunit.client.1.smithi169.stdout:3/687: dwrite d7/d50/d6f/f74 [0,4194304] 0 2024-09-16T00:26:08.232 INFO:tasks.workunit.client.0.smithi112.stdout:8/702: unlink d10/d12/d24/cb7 0 2024-09-16T00:26:08.235 INFO:tasks.workunit.client.0.smithi112.stdout:6/743: link dd/df/d17/d27/d54/l73 dd/d15/da5/d79/lf7 0 2024-09-16T00:26:08.237 INFO:tasks.workunit.client.0.smithi112.stdout:8/703: write d10/d12/d1c/d61/d6c/fb5 [461979,90965] 0 2024-09-16T00:26:08.237 INFO:tasks.workunit.client.0.smithi112.stdout:1/847: dwrite d0/d14/df/d28/f9d [0,4194304] 0 2024-09-16T00:26:08.238 INFO:tasks.workunit.client.1.smithi169.stdout:3/688: truncate d7/d1a/d1c/d35/f93 12497 0 2024-09-16T00:26:08.238 INFO:tasks.workunit.client.0.smithi112.stdout:7/790: write d6/d10/d16/d96/dc7/fbe [983047,14627] 0 2024-09-16T00:26:08.239 INFO:tasks.workunit.client.0.smithi112.stdout:6/744: write dd/df/d10/f8b [5178025,30641] 0 2024-09-16T00:26:08.242 INFO:tasks.workunit.client.0.smithi112.stdout:0/892: rename d5/da/d57/d76/c3e to d5/da/d21/da5/c118 0 2024-09-16T00:26:08.246 INFO:tasks.workunit.client.0.smithi112.stdout:6/745: dread dd/df/d10/d47/d55/f7d [0,4194304] 0 2024-09-16T00:26:08.246 INFO:tasks.workunit.client.1.smithi169.stdout:3/689: write d7/d50/fa2 [151306,61754] 0 2024-09-16T00:26:08.247 INFO:tasks.workunit.client.0.smithi112.stdout:6/746: dread - dd/df/d10/feb zero size 2024-09-16T00:26:08.248 INFO:tasks.workunit.client.1.smithi169.stdout:3/690: read d7/f14 [1618607,124241] 0 2024-09-16T00:26:08.249 INFO:tasks.workunit.client.0.smithi112.stdout:3/833: truncate d2/d9/d2b/d47/d52/d7a/d80/d83/d76/d8e/f104 3780931 0 2024-09-16T00:26:08.249 INFO:tasks.workunit.client.0.smithi112.stdout:6/747: write dd/df/d10/f8b [2055497,78254] 0 2024-09-16T00:26:08.251 INFO:tasks.workunit.client.1.smithi169.stdout:3/691: unlink d7/d1a/d1c/d23/c36 0 2024-09-16T00:26:08.252 INFO:tasks.workunit.client.0.smithi112.stdout:8/704: dwrite d10/d12/d1c/d61/fb4 [0,4194304] 0 2024-09-16T00:26:08.254 INFO:tasks.workunit.client.1.smithi169.stdout:6/735: sync 2024-09-16T00:26:08.256 INFO:tasks.workunit.client.0.smithi112.stdout:0/893: creat d5/da/d1d/d10a/f119 x:0 0 0 2024-09-16T00:26:08.256 INFO:tasks.workunit.client.1.smithi169.stdout:6/736: mkdir d2/d4d/d59/d74/def 0 2024-09-16T00:26:08.257 INFO:tasks.workunit.client.0.smithi112.stdout:7/791: creat d6/d10/d75/d8f/d90/dd7/f105 x:0 0 0 2024-09-16T00:26:08.259 INFO:tasks.workunit.client.0.smithi112.stdout:7/792: readlink d6/d10/d16/d49/d6c/l8c 0 2024-09-16T00:26:08.260 INFO:tasks.workunit.client.0.smithi112.stdout:6/748: dwrite dd/df/d10/d47/d89/fbb [0,4194304] 0 2024-09-16T00:26:08.261 INFO:tasks.workunit.client.0.smithi112.stdout:6/749: chown dd/df/d13/d18/l85 1381753 1 2024-09-16T00:26:08.261 INFO:tasks.workunit.client.0.smithi112.stdout:7/793: chown d6/d10/d75/dea/cd5 8398 1 2024-09-16T00:26:08.262 INFO:tasks.workunit.client.0.smithi112.stdout:3/834: creat d2/d9/d2b/d47/d52/d7a/d80/d83/d7b/dad/d107/f110 x:0 0 0 2024-09-16T00:26:08.263 INFO:tasks.workunit.client.0.smithi112.stdout:6/750: write dd/df/d10/d47/d55/dc4/dd1/f98 [53686,47010] 0 2024-09-16T00:26:08.265 INFO:tasks.workunit.client.0.smithi112.stdout:6/751: write dd/f7e [863671,128984] 0 2024-09-16T00:26:08.265 INFO:tasks.workunit.client.0.smithi112.stdout:3/835: chown d2/d9/d2b/d47/d52/d7a/d80/d83/f50 12525209 1 2024-09-16T00:26:08.265 INFO:tasks.workunit.client.0.smithi112.stdout:7/794: creat d6/d10/d75/d8f/d90/ddc/f106 x:0 0 0 2024-09-16T00:26:08.269 INFO:tasks.workunit.client.0.smithi112.stdout:7/795: creat d6/d10/d75/d8f/d90/f107 x:0 0 0 2024-09-16T00:26:08.270 INFO:tasks.workunit.client.1.smithi169.stdout:3/692: truncate d7/d1a/d2b/f46 2115381 0 2024-09-16T00:26:08.271 INFO:tasks.workunit.client.0.smithi112.stdout:7/796: fdatasync d6/d10/da1/fd6 0 2024-09-16T00:26:08.271 INFO:tasks.workunit.client.0.smithi112.stdout:6/752: chown dd/df/d11/d44/c7b 3196 1 2024-09-16T00:26:08.272 INFO:tasks.workunit.client.1.smithi169.stdout:3/693: write d7/d1a/d1c/d23/d88/d9b/dc8/ff7 [25175,22609] 0 2024-09-16T00:26:08.274 INFO:tasks.workunit.client.0.smithi112.stdout:3/836: dwrite d2/d9/d2b/d47/f8a [4194304,4194304] 0 2024-09-16T00:26:08.276 INFO:tasks.workunit.client.0.smithi112.stdout:7/797: mkdir d6/d10/d16/d96/dc7/dbf/d108 0 2024-09-16T00:26:08.276 INFO:tasks.workunit.client.1.smithi169.stdout:6/737: dwrite d2/d4d/d59/d43/f4a [0,4194304] 0 2024-09-16T00:26:08.278 INFO:tasks.workunit.client.0.smithi112.stdout:6/753: rename dd/df/d17/d27/d84/f26 to dd/ff8 0 2024-09-16T00:26:08.278 INFO:tasks.workunit.client.0.smithi112.stdout:7/798: write d6/d10/d75/d8f/d90/f107 [178267,66570] 0 2024-09-16T00:26:08.280 INFO:tasks.workunit.client.0.smithi112.stdout:3/837: creat d2/d9/d2b/d47/d52/d7a/d80/d83/d7b/dad/d107/d69/d92/dda/f111 x:0 0 0 2024-09-16T00:26:08.281 INFO:tasks.workunit.client.0.smithi112.stdout:7/799: creat d6/d10/d16/d96/f109 x:0 0 0 2024-09-16T00:26:08.282 INFO:tasks.workunit.client.0.smithi112.stdout:3/838: fdatasync d2/d9/d2b/d47/fac 0 2024-09-16T00:26:08.283 INFO:tasks.workunit.client.1.smithi169.stdout:3/694: dwrite d7/f24 [4194304,4194304] 0 2024-09-16T00:26:08.284 INFO:tasks.workunit.client.0.smithi112.stdout:7/800: unlink d6/d10/d16/l2e 0 2024-09-16T00:26:08.284 INFO:tasks.workunit.client.1.smithi169.stdout:6/738: mknod d2/d36/d48/cf0 0 2024-09-16T00:26:08.286 INFO:tasks.workunit.client.1.smithi169.stdout:6/739: write d2/d4d/f49 [235564,22703] 0 2024-09-16T00:26:08.287 INFO:tasks.workunit.client.1.smithi169.stdout:3/695: truncate d7/d1a/d1c/d23/d70/f8e 1633889 0 2024-09-16T00:26:08.287 INFO:tasks.workunit.client.0.smithi112.stdout:8/705: dread d10/d12/d1c/f38 [0,4194304] 0 2024-09-16T00:26:08.288 INFO:tasks.workunit.client.0.smithi112.stdout:7/801: dread d6/d10/d16/d96/dc7/f61 [0,4194304] 0 2024-09-16T00:26:08.289 INFO:tasks.workunit.client.1.smithi169.stdout:6/740: creat d2/d22/da7/ff1 x:0 0 0 2024-09-16T00:26:08.291 INFO:tasks.workunit.client.1.smithi169.stdout:6/741: write d2/d22/da7/ff1 [398162,18186] 0 2024-09-16T00:26:08.293 INFO:tasks.workunit.client.0.smithi112.stdout:8/706: mknod d10/d12/d1c/d61/d6c/dd1/d8c/d7c/da5/cea 0 2024-09-16T00:26:08.294 INFO:tasks.workunit.client.1.smithi169.stdout:3/696: creat d7/d1a/d1c/d23/d88/ff9 x:0 0 0 2024-09-16T00:26:08.294 INFO:tasks.workunit.client.1.smithi169.stdout:3/697: fdatasync d7/f68 0 2024-09-16T00:26:08.295 INFO:tasks.workunit.client.0.smithi112.stdout:7/802: mknod d6/d7f/dc3/df8/c10a 0 2024-09-16T00:26:08.295 INFO:tasks.workunit.client.0.smithi112.stdout:8/707: dread - d10/d12/d1c/d96/fac zero size 2024-09-16T00:26:08.297 INFO:tasks.workunit.client.1.smithi169.stdout:6/742: link d2/d3f/d41/d5c/d73/l61 d2/d3f/d41/d52/lf2 0 2024-09-16T00:26:08.298 INFO:tasks.workunit.client.0.smithi112.stdout:8/708: rename d10/d12/d24/c56 to d10/d12/d1c/d61/d6c/ddc/ceb 0 2024-09-16T00:26:08.299 INFO:tasks.workunit.client.1.smithi169.stdout:3/698: creat d7/d1a/d1c/d23/d88/ffa x:0 0 0 2024-09-16T00:26:08.300 INFO:tasks.workunit.client.0.smithi112.stdout:8/709: symlink d10/d12/d1c/d96/lec 0 2024-09-16T00:26:08.302 INFO:tasks.workunit.client.1.smithi169.stdout:3/699: mknod d7/d1a/d1c/d23/cfb 0 2024-09-16T00:26:08.303 INFO:tasks.workunit.client.0.smithi112.stdout:8/710: unlink d10/d12/d1c/c39 0 2024-09-16T00:26:08.304 INFO:tasks.workunit.client.1.smithi169.stdout:3/700: rename d7/d50/d90/da6/dd8/ce1 to d7/d1a/d1c/d23/d88/d9b/de2/cfc 0 2024-09-16T00:26:08.305 INFO:tasks.workunit.client.0.smithi112.stdout:0/894: dwrite d5/da/d18/d63/f65 [0,4194304] 0 2024-09-16T00:26:08.307 INFO:tasks.workunit.client.0.smithi112.stdout:0/895: creat d5/da/d1d/d71/d64/db6/f11a x:0 0 0 2024-09-16T00:26:08.309 INFO:tasks.workunit.client.0.smithi112.stdout:0/896: write d5/da/d1d/d71/d64/dea/f102 [892421,118338] 0 2024-09-16T00:26:08.311 INFO:tasks.workunit.client.0.smithi112.stdout:0/897: write d5/da/d1d/d71/fbc [540441,99365] 0 2024-09-16T00:26:08.313 INFO:tasks.workunit.client.1.smithi169.stdout:6/743: write d2/d3f/d41/d52/f18 [1424930,107571] 0 2024-09-16T00:26:08.318 INFO:tasks.workunit.client.0.smithi112.stdout:0/898: creat d5/da/d21/f11b x:0 0 0 2024-09-16T00:26:08.319 INFO:tasks.workunit.client.1.smithi169.stdout:6/744: read d2/d36/f2a [4172085,47793] 0 2024-09-16T00:26:08.321 INFO:tasks.workunit.client.1.smithi169.stdout:3/701: dwrite d7/d1a/d1c/f73 [0,4194304] 0 2024-09-16T00:26:08.321 INFO:tasks.workunit.client.0.smithi112.stdout:6/754: dwrite dd/d23/f63 [0,4194304] 0 2024-09-16T00:26:08.324 INFO:tasks.workunit.client.1.smithi169.stdout:3/702: chown d7/c12 95187 1 2024-09-16T00:26:08.324 INFO:tasks.workunit.client.0.smithi112.stdout:6/755: stat dd/dcc/le4 0 2024-09-16T00:26:08.325 INFO:tasks.workunit.client.1.smithi169.stdout:3/703: rename d7/d50/fa2 to d7/d1a/d1c/d23/d88/d9b/dc8/df8/ffd 0 2024-09-16T00:26:08.325 INFO:tasks.workunit.client.1.smithi169.stdout:6/745: getdents d2/d4c/d9f 0 2024-09-16T00:26:08.329 INFO:tasks.workunit.client.1.smithi169.stdout:6/746: symlink d2/d4d/lf3 0 2024-09-16T00:26:08.330 INFO:tasks.workunit.client.1.smithi169.stdout:3/704: symlink d7/d1a/d1c/d23/d88/lfe 0 2024-09-16T00:26:08.330 INFO:tasks.workunit.client.0.smithi112.stdout:6/756: symlink dd/df/d11/lf9 0 2024-09-16T00:26:08.331 INFO:tasks.workunit.client.0.smithi112.stdout:7/803: dwrite d6/f23 [0,4194304] 0 2024-09-16T00:26:08.331 INFO:tasks.workunit.client.0.smithi112.stdout:0/899: dwrite d5/da/d66/f99 [4194304,4194304] 0 2024-09-16T00:26:08.331 INFO:tasks.workunit.client.1.smithi169.stdout:3/705: dread - d7/d1a/d1c/d23/d70/dc5/fcf zero size 2024-09-16T00:26:08.334 INFO:tasks.workunit.client.0.smithi112.stdout:7/804: read - d6/d10/d16/d49/d6c/fec zero size 2024-09-16T00:26:08.335 INFO:tasks.workunit.client.1.smithi169.stdout:3/706: mkdir d7/d1a/d2b/dc6/dff 0 2024-09-16T00:26:08.337 INFO:tasks.workunit.client.1.smithi169.stdout:6/747: dwrite d2/d3f/d95/fb2 [0,4194304] 0 2024-09-16T00:26:08.339 INFO:tasks.workunit.client.0.smithi112.stdout:7/805: truncate d6/d10/da1/dcd/fe1 523973 0 2024-09-16T00:26:08.340 INFO:tasks.workunit.client.1.smithi169.stdout:6/748: truncate d2/d36/d48/f7e 795375 0 2024-09-16T00:26:08.341 INFO:tasks.workunit.client.1.smithi169.stdout:3/707: read d7/d1a/d1c/d35/d63/d37/f81 [2931912,5964] 0 2024-09-16T00:26:08.342 INFO:tasks.workunit.client.1.smithi169.stdout:3/708: stat d7/d1a/d2b/l9d 0 2024-09-16T00:26:08.345 INFO:tasks.workunit.client.0.smithi112.stdout:0/900: dwrite d5/da/d21/f11b [0,4194304] 0 2024-09-16T00:26:08.347 INFO:tasks.workunit.client.1.smithi169.stdout:6/749: getdents d2/d4d/d59/d43 0 2024-09-16T00:26:08.353 INFO:tasks.workunit.client.1.smithi169.stdout:6/750: getdents d2/d3f/d41/d5c/db9 0 2024-09-16T00:26:08.356 INFO:tasks.workunit.client.1.smithi169.stdout:6/751: creat d2/d3f/d41/d72/d81/ff4 x:0 0 0 2024-09-16T00:26:08.359 INFO:tasks.workunit.client.1.smithi169.stdout:6/752: symlink d2/d22/da7/lf5 0 2024-09-16T00:26:08.366 INFO:tasks.workunit.client.1.smithi169.stdout:4/547: sync 2024-09-16T00:26:08.367 INFO:tasks.workunit.client.1.smithi169.stdout:6/753: dwrite d2/d3f/d41/d52/f89 [0,4194304] 0 2024-09-16T00:26:08.371 INFO:tasks.workunit.client.1.smithi169.stdout:4/548: rmdir d2/d12/d13 39 2024-09-16T00:26:08.373 INFO:tasks.workunit.client.1.smithi169.stdout:6/754: creat d2/d22/d51/ff6 x:0 0 0 2024-09-16T00:26:08.373 INFO:tasks.workunit.client.1.smithi169.stdout:6/755: fdatasync d2/d4d/f33 0 2024-09-16T00:26:08.374 INFO:tasks.workunit.client.1.smithi169.stdout:4/549: dread - d2/d12/d13/d4d/d59/f75 zero size 2024-09-16T00:26:08.375 INFO:tasks.workunit.client.1.smithi169.stdout:6/756: fdatasync d2/d36/f70 0 2024-09-16T00:26:08.376 INFO:tasks.workunit.client.1.smithi169.stdout:6/757: chown d2/d4c/cb6 1 1 2024-09-16T00:26:08.378 INFO:tasks.workunit.client.1.smithi169.stdout:6/758: dread d2/d36/d48/f7e [0,4194304] 0 2024-09-16T00:26:08.383 INFO:tasks.workunit.client.1.smithi169.stdout:4/550: dwrite d2/f3 [0,4194304] 0 2024-09-16T00:26:08.404 INFO:tasks.workunit.client.1.smithi169.stdout:4/551: write d2/d12/d13/d5c/f61 [898625,112216] 0 2024-09-16T00:26:08.408 INFO:tasks.workunit.client.1.smithi169.stdout:4/552: creat d2/d12/d13/d1f/d3e/d5b/d1a/fad x:0 0 0 2024-09-16T00:26:08.410 INFO:tasks.workunit.client.1.smithi169.stdout:6/759: dwrite d2/d4c/f5d [0,4194304] 0 2024-09-16T00:26:08.410 INFO:tasks.workunit.client.1.smithi169.stdout:4/553: write d2/d12/d13/d5c/f61 [525053,98664] 0 2024-09-16T00:26:08.411 INFO:tasks.workunit.client.1.smithi169.stdout:6/760: chown d2/cd 118 1 2024-09-16T00:26:08.417 INFO:tasks.workunit.client.1.smithi169.stdout:6/761: creat d2/d4d/ff7 x:0 0 0 2024-09-16T00:26:08.420 INFO:tasks.workunit.client.1.smithi169.stdout:6/762: unlink d2/d3f/d41/d5c/d73/ld1 0 2024-09-16T00:26:08.423 INFO:tasks.workunit.client.1.smithi169.stdout:6/763: unlink d2/l13 0 2024-09-16T00:26:08.426 INFO:tasks.workunit.client.1.smithi169.stdout:4/554: dwrite d2/d12/d13/d5c/f9a [0,4194304] 0 2024-09-16T00:26:08.426 INFO:tasks.workunit.client.1.smithi169.stdout:6/764: creat d2/d3f/d41/d5c/ff8 x:0 0 0 2024-09-16T00:26:08.427 INFO:tasks.workunit.client.1.smithi169.stdout:4/555: chown d2/d12/d9c/da4 15 1 2024-09-16T00:26:08.430 INFO:tasks.workunit.client.1.smithi169.stdout:6/765: mknod d2/d4d/d59/d74/cf9 0 2024-09-16T00:26:08.432 INFO:tasks.workunit.client.1.smithi169.stdout:4/556: link d2/d12/d13/d5c/f61 d2/d12/d13/d90/fae 0 2024-09-16T00:26:08.437 INFO:tasks.workunit.client.1.smithi169.stdout:6/766: dwrite d2/d3f/d41/d5c/ff8 [0,4194304] 0 2024-09-16T00:26:08.439 INFO:tasks.workunit.client.1.smithi169.stdout:4/557: rename d2/d54 to d2/d12/d13/d1f/d3e/d83/daf 0 2024-09-16T00:26:08.439 INFO:tasks.workunit.client.1.smithi169.stdout:6/767: dread d2/d3f/d41/d5c/f88 [0,4194304] 0 2024-09-16T00:26:08.440 INFO:tasks.workunit.client.1.smithi169.stdout:4/558: dread - d2/d12/d13/d1f/d3e/fa1 zero size 2024-09-16T00:26:08.444 INFO:tasks.workunit.client.1.smithi169.stdout:4/559: creat d2/d12/d13/d5c/da6/fb0 x:0 0 0 2024-09-16T00:26:08.446 INFO:tasks.workunit.client.1.smithi169.stdout:6/768: stat d2/d4d/d59/c3e 0 2024-09-16T00:26:08.447 INFO:tasks.workunit.client.1.smithi169.stdout:4/560: mknod d2/d12/d13/d4e/d50/cb1 0 2024-09-16T00:26:08.447 INFO:tasks.workunit.client.1.smithi169.stdout:6/769: write d2/d3f/d41/d5c/d73/fa8 [995560,129459] 0 2024-09-16T00:26:08.449 INFO:tasks.workunit.client.1.smithi169.stdout:6/770: write d2/d3f/d41/d5c/f94 [542254,37337] 0 2024-09-16T00:26:08.450 INFO:tasks.workunit.client.1.smithi169.stdout:4/561: dread - d2/d12/d13/d1f/d3e/d5b/f73 zero size 2024-09-16T00:26:08.452 INFO:tasks.workunit.client.1.smithi169.stdout:4/562: symlink d2/d12/lb2 0 2024-09-16T00:26:08.454 INFO:tasks.workunit.client.1.smithi169.stdout:6/771: dread d2/d3f/d41/d52/f89 [0,4194304] 0 2024-09-16T00:26:08.455 INFO:tasks.workunit.client.1.smithi169.stdout:4/563: rename d2/d12/f36 to d2/d12/da7/fb3 0 2024-09-16T00:26:08.457 INFO:tasks.workunit.client.1.smithi169.stdout:4/564: write d2/d12/d13/d1f/d3e/d83/f6e [957473,21892] 0 2024-09-16T00:26:08.458 INFO:tasks.workunit.client.1.smithi169.stdout:6/772: fsync d2/d4d/d59/d38/f5a 0 2024-09-16T00:26:08.459 INFO:tasks.workunit.client.1.smithi169.stdout:6/773: dread - d2/d36/fe0 zero size 2024-09-16T00:26:08.460 INFO:tasks.workunit.client.1.smithi169.stdout:6/774: dread - d2/d22/d51/ff6 zero size 2024-09-16T00:26:08.464 INFO:tasks.workunit.client.0.smithi112.stdout:6/757: symlink dd/df/d17/lfa 0 2024-09-16T00:26:08.466 INFO:tasks.workunit.client.1.smithi169.stdout:4/565: dwrite d2/d12/f5d [0,4194304] 0 2024-09-16T00:26:08.469 INFO:tasks.workunit.client.1.smithi169.stdout:6/775: dwrite d2/da4/fbf [0,4194304] 0 2024-09-16T00:26:08.472 INFO:tasks.workunit.client.1.smithi169.stdout:4/566: dread - d2/d12/d13/d1f/d3e/d83/daf/f7a zero size 2024-09-16T00:26:08.474 INFO:tasks.workunit.client.1.smithi169.stdout:4/567: rmdir d2/d12/d13/d1f 39 2024-09-16T00:26:08.475 INFO:tasks.workunit.client.1.smithi169.stdout:6/776: link d2/d4d/d59/l82 d2/d3f/d41/d52/lfa 0 2024-09-16T00:26:08.476 INFO:tasks.workunit.client.0.smithi112.stdout:6/758: dwrite dd/df/d10/d47/d89/dbc/fc5 [4194304,4194304] 0 2024-09-16T00:26:08.477 INFO:tasks.workunit.client.1.smithi169.stdout:6/777: write d2/d4d/d59/d38/f5a [467087,739] 0 2024-09-16T00:26:08.477 INFO:tasks.workunit.client.1.smithi169.stdout:4/568: fdatasync d2/d12/d13/d1f/f87 0 2024-09-16T00:26:08.480 INFO:tasks.workunit.client.1.smithi169.stdout:6/778: symlink d2/d36/lfb 0 2024-09-16T00:26:08.482 INFO:tasks.workunit.client.1.smithi169.stdout:4/569: mkdir d2/d12/d13/db4 0 2024-09-16T00:26:08.483 INFO:tasks.workunit.client.1.smithi169.stdout:6/779: mkdir d2/d3f/d41/d72/dfc 0 2024-09-16T00:26:08.484 INFO:tasks.workunit.client.0.smithi112.stdout:6/759: mkdir dd/df/d11/d44/d8a/dfb 0 2024-09-16T00:26:08.487 INFO:tasks.workunit.client.0.smithi112.stdout:6/760: creat dd/df/d13/ffc x:0 0 0 2024-09-16T00:26:08.494 INFO:tasks.workunit.client.0.smithi112.stdout:6/761: dwrite dd/df/d10/d47/d89/fbb [4194304,4194304] 0 2024-09-16T00:26:08.495 INFO:tasks.workunit.client.1.smithi169.stdout:2/563: sync 2024-09-16T00:26:08.495 INFO:tasks.workunit.client.1.smithi169.stdout:7/643: sync 2024-09-16T00:26:08.495 INFO:tasks.workunit.client.1.smithi169.stdout:1/577: sync 2024-09-16T00:26:08.496 INFO:tasks.workunit.client.1.smithi169.stdout:5/507: sync 2024-09-16T00:26:08.496 INFO:tasks.workunit.client.1.smithi169.stdout:8/610: sync 2024-09-16T00:26:08.496 INFO:tasks.workunit.client.1.smithi169.stdout:0/594: sync 2024-09-16T00:26:08.496 INFO:tasks.workunit.client.1.smithi169.stdout:9/602: sync 2024-09-16T00:26:08.499 INFO:tasks.workunit.client.1.smithi169.stdout:1/578: symlink dc/d6c/lac 0 2024-09-16T00:26:08.499 INFO:tasks.workunit.client.1.smithi169.stdout:0/595: fdatasync d0/d2/d15/f7a 0 2024-09-16T00:26:08.501 INFO:tasks.workunit.client.0.smithi112.stdout:6/762: dwrite dd/df/d17/d27/d54/f56 [0,4194304] 0 2024-09-16T00:26:08.501 INFO:tasks.workunit.client.1.smithi169.stdout:7/644: chown d7/d16/d49/f72 1591085 1 2024-09-16T00:26:08.502 INFO:tasks.workunit.client.1.smithi169.stdout:8/611: truncate d7/dc/d1d/f91 949230 0 2024-09-16T00:26:08.504 INFO:tasks.workunit.client.1.smithi169.stdout:0/596: chown d0/d2/d15/f22 2829820 1 2024-09-16T00:26:08.507 INFO:tasks.workunit.client.1.smithi169.stdout:5/508: mkdir d2/da/d10/d13/d1c/d32/d53/dae 0 2024-09-16T00:26:08.508 INFO:tasks.workunit.client.1.smithi169.stdout:2/564: mknod d7/db/d89/db4/d9d/dbe/cc7 0 2024-09-16T00:26:08.509 INFO:tasks.workunit.client.0.smithi112.stdout:6/763: dread dd/d15/da5/d79/fb3 [0,4194304] 0 2024-09-16T00:26:08.511 INFO:tasks.workunit.client.0.smithi112.stdout:6/764: fsync dd/df/d13/f1c 0 2024-09-16T00:26:08.514 INFO:tasks.workunit.client.0.smithi112.stdout:6/765: write dd/df/d17/f41 [1680272,99786] 0 2024-09-16T00:26:08.515 INFO:tasks.workunit.client.1.smithi169.stdout:3/709: sync 2024-09-16T00:26:08.515 INFO:tasks.workunit.client.1.smithi169.stdout:8/612: rmdir d7/dc/d53/da0/dac 39 2024-09-16T00:26:08.518 INFO:tasks.workunit.client.0.smithi112.stdout:7/806: truncate d6/d10/d16/f20 124046 0 2024-09-16T00:26:08.518 INFO:tasks.workunit.client.1.smithi169.stdout:3/710: fdatasync d7/d50/d6f/f74 0 2024-09-16T00:26:08.518 INFO:tasks.workunit.client.1.smithi169.stdout:0/597: creat d0/d2c/d54/d21/d31/d83/fce x:0 0 0 2024-09-16T00:26:08.518 INFO:tasks.workunit.client.0.smithi112.stdout:0/901: write d5/da/dc/f13 [1835607,46536] 0 2024-09-16T00:26:08.519 INFO:tasks.workunit.client.1.smithi169.stdout:2/565: dwrite d7/db/db8/d6d/fba [0,4194304] 0 2024-09-16T00:26:08.519 INFO:tasks.workunit.client.1.smithi169.stdout:4/570: write d2/d12/d13/d1f/d3e/d5b/d1a/d9e/f72 [148049,114328] 0 2024-09-16T00:26:08.520 INFO:tasks.workunit.client.0.smithi112.stdout:7/807: truncate d6/d10/d75/d88/f91 1516880 0 2024-09-16T00:26:08.520 INFO:tasks.workunit.client.1.smithi169.stdout:5/509: symlink d2/da/d10/d13/d24/laf 0 2024-09-16T00:26:08.521 INFO:tasks.workunit.client.0.smithi112.stdout:0/902: mkdir d5/da/d57/db0/d11c 0 2024-09-16T00:26:08.522 INFO:tasks.workunit.client.1.smithi169.stdout:6/780: truncate d2/d22/da7/fcd 641323 0 2024-09-16T00:26:08.522 INFO:tasks.workunit.client.0.smithi112.stdout:0/903: chown d5/da/d1d/d71/d64/d106/l86 295017801 1 2024-09-16T00:26:08.524 INFO:tasks.workunit.client.1.smithi169.stdout:7/645: unlink d7/df/c3e 0 2024-09-16T00:26:08.529 INFO:tasks.workunit.client.1.smithi169.stdout:1/579: link dc/d2c/d3d/d86/l9a dc/d6c/d96/lad 0 2024-09-16T00:26:08.529 INFO:tasks.workunit.client.1.smithi169.stdout:7/646: truncate d7/d16/f6a 4821298 0 2024-09-16T00:26:08.529 INFO:tasks.workunit.client.1.smithi169.stdout:9/603: write d5/f8f [705613,87800] 0 2024-09-16T00:26:08.530 INFO:tasks.workunit.client.1.smithi169.stdout:3/711: chown d7/d1a/d2b/l3b 890628566 1 2024-09-16T00:26:08.531 INFO:tasks.workunit.client.1.smithi169.stdout:1/580: dread - dc/d18/d67/d69/d7b/d92/f9d zero size 2024-09-16T00:26:08.532 INFO:tasks.workunit.client.1.smithi169.stdout:3/712: fdatasync d7/d1a/d1c/d35/d63/f6a 0 2024-09-16T00:26:08.535 INFO:tasks.workunit.client.1.smithi169.stdout:8/613: symlink d7/dc/d26/d41/lc7 0 2024-09-16T00:26:08.535 INFO:tasks.workunit.client.1.smithi169.stdout:9/604: rmdir d5/da/d5b/d6c 39 2024-09-16T00:26:08.535 INFO:tasks.workunit.client.1.smithi169.stdout:8/614: readlink d7/dc/d2a/l50 0 2024-09-16T00:26:08.537 INFO:tasks.workunit.client.1.smithi169.stdout:6/781: fsync d2/d36/f31 0 2024-09-16T00:26:08.538 INFO:tasks.workunit.client.1.smithi169.stdout:3/713: creat d7/d50/d6f/f100 x:0 0 0 2024-09-16T00:26:08.541 INFO:tasks.workunit.client.1.smithi169.stdout:9/605: creat d5/da/d73/fda x:0 0 0 2024-09-16T00:26:08.541 INFO:tasks.workunit.client.1.smithi169.stdout:0/598: creat d0/d2c/d54/fcf x:0 0 0 2024-09-16T00:26:08.542 INFO:tasks.workunit.client.1.smithi169.stdout:6/782: read d2/d3f/d41/d72/f9d [663803,86165] 0 2024-09-16T00:26:08.544 INFO:tasks.workunit.client.1.smithi169.stdout:3/714: creat d7/d1a/d1c/d35/d63/d37/f101 x:0 0 0 2024-09-16T00:26:08.547 INFO:tasks.workunit.client.1.smithi169.stdout:8/615: link d7/dc/d17/d24/d32/f6f d7/dc/d17/d24/d32/d8d/fc8 0 2024-09-16T00:26:08.549 INFO:tasks.workunit.client.1.smithi169.stdout:9/606: symlink d5/d49/dc0/ldb 0 2024-09-16T00:26:08.551 INFO:tasks.workunit.client.1.smithi169.stdout:9/607: chown d5/d49/dc0 8875 1 2024-09-16T00:26:08.551 INFO:tasks.workunit.client.0.smithi112.stdout:6/766: dread dd/df/d10/d47/d89/dbc/fdb [0,4194304] 0 2024-09-16T00:26:08.552 INFO:tasks.workunit.client.1.smithi169.stdout:6/783: mknod d2/d3f/d41/d5c/cfd 0 2024-09-16T00:26:08.554 INFO:tasks.workunit.client.1.smithi169.stdout:2/566: dread d7/d1c/f95 [0,4194304] 0 2024-09-16T00:26:08.557 INFO:tasks.workunit.client.1.smithi169.stdout:9/608: write d5/da/d21/f2c [2389700,12621] 0 2024-09-16T00:26:08.557 INFO:tasks.workunit.client.1.smithi169.stdout:4/571: dread d2/d12/d13/d27/f32 [0,4194304] 0 2024-09-16T00:26:08.557 INFO:tasks.workunit.client.1.smithi169.stdout:5/510: write d2/da/d10/d13/d1c/d40/f5b [829815,125340] 0 2024-09-16T00:26:08.560 INFO:tasks.workunit.client.1.smithi169.stdout:3/715: rename d7/d1a/d1c/d35/d63/da3/cb5 to d7/d1a/d1c/d23/d88/d9b/dc8/c102 0 2024-09-16T00:26:08.560 INFO:tasks.workunit.client.1.smithi169.stdout:8/616: dwrite d7/d37/fc2 [0,4194304] 0 2024-09-16T00:26:08.561 INFO:tasks.workunit.client.1.smithi169.stdout:0/599: creat d0/fd0 x:0 0 0 2024-09-16T00:26:08.561 INFO:tasks.workunit.client.1.smithi169.stdout:5/511: stat d2/da/d10/d13/d1c/d40/d54/d9c 0 2024-09-16T00:26:08.562 INFO:tasks.workunit.client.1.smithi169.stdout:4/572: chown d2/d12/d13/d4e/f77 5 1 2024-09-16T00:26:08.564 INFO:tasks.workunit.client.1.smithi169.stdout:7/647: write f1 [118911,60822] 0 2024-09-16T00:26:08.564 INFO:tasks.workunit.client.1.smithi169.stdout:6/784: mknod d2/d3f/d41/d72/d9e/dc0/cfe 0 2024-09-16T00:26:08.568 INFO:tasks.workunit.client.1.smithi169.stdout:6/785: readlink d2/d22/d51/led 0 2024-09-16T00:26:08.570 INFO:tasks.workunit.client.1.smithi169.stdout:5/512: unlink d2/fe 0 2024-09-16T00:26:08.573 INFO:tasks.workunit.client.1.smithi169.stdout:7/648: stat d7/df/d95/l3d 0 2024-09-16T00:26:08.575 INFO:tasks.workunit.client.1.smithi169.stdout:3/716: creat d7/de5/f103 x:0 0 0 2024-09-16T00:26:08.575 INFO:tasks.workunit.client.1.smithi169.stdout:9/609: symlink d5/da/d2e/ldc 0 2024-09-16T00:26:08.577 INFO:tasks.workunit.client.1.smithi169.stdout:2/567: dwrite d7/db/f60 [0,4194304] 0 2024-09-16T00:26:08.578 INFO:tasks.workunit.client.1.smithi169.stdout:4/573: mknod d2/cb5 0 2024-09-16T00:26:08.579 INFO:tasks.workunit.client.1.smithi169.stdout:9/610: chown d5/da/d5b 3874949 1 2024-09-16T00:26:08.580 INFO:tasks.workunit.client.1.smithi169.stdout:3/717: dread - d7/d1a/d2b/dc6/ff2 zero size 2024-09-16T00:26:08.581 INFO:tasks.workunit.client.1.smithi169.stdout:6/786: dwrite d2/d4c/f5d [0,4194304] 0 2024-09-16T00:26:08.582 INFO:tasks.workunit.client.1.smithi169.stdout:2/568: chown d7/f11 35525 1 2024-09-16T00:26:08.584 INFO:tasks.workunit.client.1.smithi169.stdout:3/718: chown d7/d1a/d1c/d35/d63/dea 2 1 2024-09-16T00:26:08.587 INFO:tasks.workunit.client.1.smithi169.stdout:3/719: chown d7/d50/d6f/fb4 1610 1 2024-09-16T00:26:08.594 INFO:tasks.workunit.client.1.smithi169.stdout:6/787: read d2/d3f/d41/f80 [3593663,13835] 0 2024-09-16T00:26:08.594 INFO:tasks.workunit.client.1.smithi169.stdout:2/569: read d7/db/d27/d29/f54 [139426,108] 0 2024-09-16T00:26:08.598 INFO:tasks.workunit.client.1.smithi169.stdout:0/600: dread d0/d2/d8/d5/fa [0,4194304] 0 2024-09-16T00:26:08.598 INFO:tasks.workunit.client.1.smithi169.stdout:2/570: write d7/db/f13 [1973148,50701] 0 2024-09-16T00:26:08.600 INFO:tasks.workunit.client.0.smithi112.stdout:2/881: sync 2024-09-16T00:26:08.600 INFO:tasks.workunit.client.0.smithi112.stdout:9/973: sync 2024-09-16T00:26:08.600 INFO:tasks.workunit.client.0.smithi112.stdout:5/688: sync 2024-09-16T00:26:08.601 INFO:tasks.workunit.client.1.smithi169.stdout:7/649: dwrite d7/fa0 [0,4194304] 0 2024-09-16T00:26:08.601 INFO:tasks.workunit.client.1.smithi169.stdout:0/601: fsync d0/d2c/d54/d21/d31/d83/dac/fb8 0 2024-09-16T00:26:08.602 INFO:tasks.workunit.client.0.smithi112.stdout:0/904: stat d5/da/d18/fc4 0 2024-09-16T00:26:08.603 INFO:tasks.workunit.client.0.smithi112.stdout:2/882: write d2/f46 [406595,7699] 0 2024-09-16T00:26:08.604 INFO:tasks.workunit.client.0.smithi112.stdout:9/974: truncate d1/d5/d6/d107/f113 4702730 0 2024-09-16T00:26:08.605 INFO:tasks.workunit.client.0.smithi112.stdout:5/689: unlink d4/d7/d14/f1a 0 2024-09-16T00:26:08.606 INFO:tasks.workunit.client.0.smithi112.stdout:9/975: dread d1/d5/d6/df8/ddd/f133 [0,4194304] 0 2024-09-16T00:26:08.607 INFO:tasks.workunit.client.0.smithi112.stdout:9/976: chown d1/d5/d20/d81/d1a/d88/def/d11b 3455 1 2024-09-16T00:26:08.608 INFO:tasks.workunit.client.0.smithi112.stdout:9/977: readlink d1/d5/d20/d81/d1a/d88/def/l6b 0 2024-09-16T00:26:08.609 INFO:tasks.workunit.client.1.smithi169.stdout:8/617: dread d7/dc/d2a/f42 [0,4194304] 0 2024-09-16T00:26:08.611 INFO:tasks.workunit.client.0.smithi112.stdout:5/690: creat d4/dd/d49/d70/d83/fe3 x:0 0 0 2024-09-16T00:26:08.612 INFO:tasks.workunit.client.1.smithi169.stdout:6/788: unlink d2/d4d/d59/d43/f2f 0 2024-09-16T00:26:08.613 INFO:tasks.workunit.client.1.smithi169.stdout:2/571: mkdir d7/db/d89/db4/dc8 0 2024-09-16T00:26:08.614 INFO:tasks.workunit.client.1.smithi169.stdout:2/572: fdatasync d7/db/f60 0 2024-09-16T00:26:08.615 INFO:tasks.workunit.client.0.smithi112.stdout:5/691: creat d4/d7/d87/fe4 x:0 0 0 2024-09-16T00:26:08.615 INFO:tasks.workunit.client.1.smithi169.stdout:7/650: creat d7/d16/fcf x:0 0 0 2024-09-16T00:26:08.616 INFO:tasks.workunit.client.0.smithi112.stdout:5/692: rmdir d4/d7/d26/d4c/db3/db6/dd0 39 2024-09-16T00:26:08.617 INFO:tasks.workunit.client.0.smithi112.stdout:5/693: readlink d4/l42 0 2024-09-16T00:26:08.617 INFO:tasks.workunit.client.0.smithi112.stdout:5/694: chown d4/d1c/fad 62803098 1 2024-09-16T00:26:08.618 INFO:tasks.workunit.client.1.smithi169.stdout:6/789: unlink d2/d3f/d41/d5c/db9/lc4 0 2024-09-16T00:26:08.620 INFO:tasks.workunit.client.0.smithi112.stdout:5/695: fdatasync d4/d7/d14/d90/dd3/fdd 0 2024-09-16T00:26:08.622 INFO:tasks.workunit.client.0.smithi112.stdout:7/808: truncate d6/d10/d16/d96/dc7/d39/f43 5644414 0 2024-09-16T00:26:08.623 INFO:tasks.workunit.client.1.smithi169.stdout:6/790: chown d2/d36/l3a 1 1 2024-09-16T00:26:08.623 INFO:tasks.workunit.client.0.smithi112.stdout:7/809: write d6/d10/f76 [4811542,39636] 0 2024-09-16T00:26:08.625 INFO:tasks.workunit.client.0.smithi112.stdout:6/767: truncate dd/df/d13/d18/f28 4167641 0 2024-09-16T00:26:08.627 INFO:tasks.workunit.client.1.smithi169.stdout:6/791: creat d2/d4d/d59/d74/fff x:0 0 0 2024-09-16T00:26:08.628 INFO:tasks.workunit.client.0.smithi112.stdout:6/768: symlink dd/df/d11/lfd 0 2024-09-16T00:26:08.628 INFO:tasks.workunit.client.1.smithi169.stdout:2/573: creat d7/db/d27/d29/fc9 x:0 0 0 2024-09-16T00:26:08.629 INFO:tasks.workunit.client.0.smithi112.stdout:0/905: write d5/da/d57/db0/f79 [2095392,56044] 0 2024-09-16T00:26:08.629 INFO:tasks.workunit.client.0.smithi112.stdout:9/978: dread d1/d5/d2c/fc3 [0,4194304] 0 2024-09-16T00:26:08.631 INFO:tasks.workunit.client.0.smithi112.stdout:7/810: dwrite d6/d10/d16/d96/dc7/f24 [0,4194304] 0 2024-09-16T00:26:08.633 INFO:tasks.workunit.client.0.smithi112.stdout:0/906: write d5/d90/dc5/de2/f113 [518948,52791] 0 2024-09-16T00:26:08.634 INFO:tasks.workunit.client.1.smithi169.stdout:9/611: truncate d5/da/d5b/d6c/da0/fa7 216761 0 2024-09-16T00:26:08.636 INFO:tasks.workunit.client.1.smithi169.stdout:6/792: dwrite d2/d36/d48/fe3 [0,4194304] 0 2024-09-16T00:26:08.636 INFO:tasks.workunit.client.0.smithi112.stdout:0/907: getdents d5/da/dc/de6 0 2024-09-16T00:26:08.639 INFO:tasks.workunit.client.0.smithi112.stdout:0/908: dread - d5/da/d18/d24/d46/d5f/d7a/f103 zero size 2024-09-16T00:26:08.640 INFO:tasks.workunit.client.1.smithi169.stdout:8/618: dread d7/dc/d26/d41/f5c [0,4194304] 0 2024-09-16T00:26:08.642 INFO:tasks.workunit.client.0.smithi112.stdout:0/909: readlink d5/da/d57/d76/ld0 0 2024-09-16T00:26:08.642 INFO:tasks.workunit.client.0.smithi112.stdout:7/811: dwrite d6/d10/d16/f34 [0,4194304] 0 2024-09-16T00:26:08.643 INFO:tasks.workunit.client.0.smithi112.stdout:7/812: write d6/f23 [1652656,88728] 0 2024-09-16T00:26:08.645 INFO:tasks.workunit.client.0.smithi112.stdout:7/813: write d6/fae [2274676,47015] 0 2024-09-16T00:26:08.646 INFO:tasks.workunit.client.0.smithi112.stdout:0/910: mknod d5/df7/c11d 0 2024-09-16T00:26:08.649 INFO:tasks.workunit.client.0.smithi112.stdout:0/911: write d5/da/d1d/d71/d64/d106/fbf [841835,9825] 0 2024-09-16T00:26:08.649 INFO:tasks.workunit.client.0.smithi112.stdout:7/814: creat d6/d10/d75/d8f/d90/dd3/f10b x:0 0 0 2024-09-16T00:26:08.651 INFO:tasks.workunit.client.1.smithi169.stdout:6/793: dwrite d2/d36/fe0 [0,4194304] 0 2024-09-16T00:26:08.654 INFO:tasks.workunit.client.0.smithi112.stdout:7/815: mkdir d6/d10/da1/da2/d10c 0 2024-09-16T00:26:08.655 INFO:tasks.workunit.client.1.smithi169.stdout:0/602: truncate d0/d7d/fa8 3103268 0 2024-09-16T00:26:08.656 INFO:tasks.workunit.client.0.smithi112.stdout:7/816: creat d6/d10/d75/f10d x:0 0 0 2024-09-16T00:26:08.657 INFO:tasks.workunit.client.0.smithi112.stdout:7/817: dread - d6/d10/d16/d49/fce zero size 2024-09-16T00:26:08.659 INFO:tasks.workunit.client.1.smithi169.stdout:7/651: write d7/df/f44 [236994,15045] 0 2024-09-16T00:26:08.659 INFO:tasks.workunit.client.1.smithi169.stdout:8/619: rename d7/dc/d2a/l50 to d7/dc/d1d/d3c/d82/db3/lc9 0 2024-09-16T00:26:08.660 INFO:tasks.workunit.client.0.smithi112.stdout:7/818: mknod d6/d10/d16/db4/c10e 0 2024-09-16T00:26:08.660 INFO:tasks.workunit.client.0.smithi112.stdout:9/979: dread d1/d5/d6/d107/f113 [0,4194304] 0 2024-09-16T00:26:08.661 INFO:tasks.workunit.client.1.smithi169.stdout:2/574: truncate d7/d1c/d26/d96/f8e 82155 0 2024-09-16T00:26:08.661 INFO:tasks.workunit.client.1.smithi169.stdout:8/620: write d7/dc/d17/f28 [2273157,25218] 0 2024-09-16T00:26:08.662 INFO:tasks.workunit.client.0.smithi112.stdout:7/819: fdatasync d6/d10/d16/d96/dc7/d39/fa8 0 2024-09-16T00:26:08.663 INFO:tasks.workunit.client.1.smithi169.stdout:9/612: write d5/da/d21/f45 [411892,130466] 0 2024-09-16T00:26:08.664 INFO:tasks.workunit.client.1.smithi169.stdout:2/575: chown d7/db/d89/db4/d9d/dbe/cc7 10288 1 2024-09-16T00:26:08.664 INFO:tasks.workunit.client.0.smithi112.stdout:5/696: dread d4/dd/d12/d52/d64/fd4 [0,4194304] 0 2024-09-16T00:26:08.666 INFO:tasks.workunit.client.1.smithi169.stdout:0/603: rename d0/d50 to d0/d2c/d54/d56/d67/dd1 0 2024-09-16T00:26:08.668 INFO:tasks.workunit.client.0.smithi112.stdout:7/820: mknod d6/d10/d16/d3b/c10f 0 2024-09-16T00:26:08.669 INFO:tasks.workunit.client.0.smithi112.stdout:5/697: mkdir d4/d7/d14/d90/dd3/de5 0 2024-09-16T00:26:08.669 INFO:tasks.workunit.client.1.smithi169.stdout:0/604: write d0/d2c/d54/d21/d31/d83/fce [545660,79398] 0 2024-09-16T00:26:08.669 INFO:tasks.workunit.client.1.smithi169.stdout:8/621: stat d7/dc/d17/d48/f60 0 2024-09-16T00:26:08.671 INFO:tasks.workunit.client.0.smithi112.stdout:9/980: dwrite d1/d5/d6/d2d/d39/f96 [0,4194304] 0 2024-09-16T00:26:08.671 INFO:tasks.workunit.client.0.smithi112.stdout:5/698: fsync d4/dd/d12/d52/f63 0 2024-09-16T00:26:08.673 INFO:tasks.workunit.client.1.smithi169.stdout:9/613: dwrite d5/da/d2e/f62 [0,4194304] 0 2024-09-16T00:26:08.675 INFO:tasks.workunit.client.1.smithi169.stdout:0/605: creat d0/d2c/d54/d56/fd2 x:0 0 0 2024-09-16T00:26:08.676 INFO:tasks.workunit.client.0.smithi112.stdout:9/981: rmdir d1/d5/d20/d81/d1a/d88 39 2024-09-16T00:26:08.677 INFO:tasks.workunit.client.0.smithi112.stdout:7/821: dread d6/d10/d16/db4/fc4 [0,4194304] 0 2024-09-16T00:26:08.677 INFO:tasks.workunit.client.1.smithi169.stdout:0/606: write d0/d2c/d54/d56/f59 [19824,11259] 0 2024-09-16T00:26:08.678 INFO:tasks.workunit.client.0.smithi112.stdout:5/699: unlink d4/d7/d26/d36/dcf/fd9 0 2024-09-16T00:26:08.678 INFO:tasks.workunit.client.0.smithi112.stdout:7/822: write d6/d7f/d85/fb5 [1794870,54404] 0 2024-09-16T00:26:08.679 INFO:tasks.workunit.client.1.smithi169.stdout:2/576: symlink d7/db/d20/d5c/db2/lca 0 2024-09-16T00:26:08.679 INFO:tasks.workunit.client.0.smithi112.stdout:5/700: read d4/d1c/fad [566627,92600] 0 2024-09-16T00:26:08.684 INFO:tasks.workunit.client.1.smithi169.stdout:2/577: symlink d7/db/d89/db4/d69/da2/lcb 0 2024-09-16T00:26:08.685 INFO:tasks.workunit.client.0.smithi112.stdout:5/701: mkdir d4/dd/d49/de6 0 2024-09-16T00:26:08.688 INFO:tasks.workunit.client.0.smithi112.stdout:7/823: creat d6/d7f/df6/f110 x:0 0 0 2024-09-16T00:26:08.690 INFO:tasks.workunit.client.0.smithi112.stdout:0/912: dwrite d5/da/d57/f6e [0,4194304] 0 2024-09-16T00:26:08.690 INFO:tasks.workunit.client.0.smithi112.stdout:5/702: mknod d4/d4f/d77/dcc/d7c/d99/ce7 0 2024-09-16T00:26:08.693 INFO:tasks.workunit.client.0.smithi112.stdout:0/913: chown d5/da/d18/c7f 1 1 2024-09-16T00:26:08.694 INFO:tasks.workunit.client.0.smithi112.stdout:5/703: rename d4/d1c/l29 to d4/d4f/le8 0 2024-09-16T00:26:08.697 INFO:tasks.workunit.client.1.smithi169.stdout:2/578: dwrite d7/db/d27/d29/d77/f86 [0,4194304] 0 2024-09-16T00:26:08.699 INFO:tasks.workunit.client.0.smithi112.stdout:7/824: dwrite d6/d10/d16/f66 [0,4194304] 0 2024-09-16T00:26:08.700 INFO:tasks.workunit.client.1.smithi169.stdout:2/579: fsync f0 0 2024-09-16T00:26:08.701 INFO:tasks.workunit.client.1.smithi169.stdout:6/794: dwrite d2/d22/fa1 [0,4194304] 0 2024-09-16T00:26:08.704 INFO:tasks.workunit.client.1.smithi169.stdout:2/580: chown d7/db/db8/d6d/la3 578739347 1 2024-09-16T00:26:08.705 INFO:tasks.workunit.client.0.smithi112.stdout:5/704: dwrite d4/d7/d26/d36/f3b [0,4194304] 0 2024-09-16T00:26:08.707 INFO:tasks.workunit.client.1.smithi169.stdout:6/795: symlink d2/d3f/d41/d5c/d91/l100 0 2024-09-16T00:26:08.708 INFO:tasks.workunit.client.1.smithi169.stdout:2/581: creat d7/db/db8/d6d/fcc x:0 0 0 2024-09-16T00:26:08.709 INFO:tasks.workunit.client.0.smithi112.stdout:5/705: read d4/d7/d26/d36/f74 [2434748,46367] 0 2024-09-16T00:26:08.713 INFO:tasks.workunit.client.0.smithi112.stdout:0/914: dread d5/da/d18/d24/d46/d88/f8c [0,4194304] 0 2024-09-16T00:26:08.714 INFO:tasks.workunit.client.1.smithi169.stdout:6/796: unlink d2/d3f/d41/d72/d81/ff4 0 2024-09-16T00:26:08.714 INFO:tasks.workunit.client.0.smithi112.stdout:5/706: symlink d4/le9 0 2024-09-16T00:26:08.714 INFO:tasks.workunit.client.1.smithi169.stdout:2/582: creat d7/db/d20/d5c/db2/fcd x:0 0 0 2024-09-16T00:26:08.718 INFO:tasks.workunit.client.0.smithi112.stdout:5/707: chown d4/dd/d12/d52/d64/ca6 2096727 1 2024-09-16T00:26:08.720 INFO:tasks.workunit.client.1.smithi169.stdout:6/797: chown d2/d3f/d41/d52/l76 4764069 1 2024-09-16T00:26:08.723 INFO:tasks.workunit.client.0.smithi112.stdout:9/982: dread d1/d5/d6/d2d/f3e [0,4194304] 0 2024-09-16T00:26:08.725 INFO:tasks.workunit.client.0.smithi112.stdout:0/915: dwrite d5/da/d1d/d71/d64/db6/fbd [0,4194304] 0 2024-09-16T00:26:08.726 INFO:tasks.workunit.client.0.smithi112.stdout:0/916: chown d5/d90 37417232 1 2024-09-16T00:26:08.727 INFO:tasks.workunit.client.0.smithi112.stdout:9/983: creat d1/d5/d6/df8/f155 x:0 0 0 2024-09-16T00:26:08.727 INFO:tasks.workunit.client.0.smithi112.stdout:0/917: chown d5/da/d1d/d71/d64/db6 11392452 1 2024-09-16T00:26:08.728 INFO:tasks.workunit.client.0.smithi112.stdout:7/825: write d6/d10/d16/d96/dc7/f60 [2692550,49933] 0 2024-09-16T00:26:08.728 INFO:tasks.workunit.client.1.smithi169.stdout:6/798: fdatasync d2/f19 0 2024-09-16T00:26:08.729 INFO:tasks.workunit.client.1.smithi169.stdout:2/583: truncate d7/db/f60 118157 0 2024-09-16T00:26:08.729 INFO:tasks.workunit.client.1.smithi169.stdout:8/622: dwrite d7/dc/d26/fbc [0,4194304] 0 2024-09-16T00:26:08.732 INFO:tasks.workunit.client.0.smithi112.stdout:0/918: truncate d5/da/d66/fe0 763381 0 2024-09-16T00:26:08.733 INFO:tasks.workunit.client.1.smithi169.stdout:9/614: dread d5/f8 [4194304,4194304] 0 2024-09-16T00:26:08.734 INFO:tasks.workunit.client.0.smithi112.stdout:9/984: mknod d1/d5/d20/d81/d1a/d88/def/c156 0 2024-09-16T00:26:08.734 INFO:tasks.workunit.client.1.smithi169.stdout:7/652: dwrite d7/df/f21 [0,4194304] 0 2024-09-16T00:26:08.737 INFO:tasks.workunit.client.1.smithi169.stdout:2/584: write d7/d1c/fb7 [113048,36987] 0 2024-09-16T00:26:08.737 INFO:tasks.workunit.client.0.smithi112.stdout:9/985: fdatasync d1/d5/d6/d38/f4f 0 2024-09-16T00:26:08.738 INFO:tasks.workunit.client.1.smithi169.stdout:8/623: readlink d7/dc/d17/d24/l83 0 2024-09-16T00:26:08.739 INFO:tasks.workunit.client.1.smithi169.stdout:6/799: chown d2/d4d/d59/d43 35272 1 2024-09-16T00:26:08.739 INFO:tasks.workunit.client.0.smithi112.stdout:9/986: symlink d1/d5/d6/d107/l157 0 2024-09-16T00:26:08.740 INFO:tasks.workunit.client.0.smithi112.stdout:7/826: dwrite d6/d10/d16/d96/dc7/d39/fdd [0,4194304] 0 2024-09-16T00:26:08.742 INFO:tasks.workunit.client.1.smithi169.stdout:2/585: mkdir d7/d6b/d70/d79/dce 0 2024-09-16T00:26:08.742 INFO:tasks.workunit.client.1.smithi169.stdout:0/607: dwrite d0/d2c/d54/d21/d31/d83/f8a [0,4194304] 0 2024-09-16T00:26:08.743 INFO:tasks.workunit.client.1.smithi169.stdout:2/586: readlink d7/d1c/d26/l6f 0 2024-09-16T00:26:08.745 INFO:tasks.workunit.client.1.smithi169.stdout:8/624: write d7/dc/d17/f89 [1058721,51825] 0 2024-09-16T00:26:08.751 INFO:tasks.workunit.client.1.smithi169.stdout:8/625: chown d7/dc/d2a/fba 1301966609 1 2024-09-16T00:26:08.753 INFO:tasks.workunit.client.1.smithi169.stdout:8/626: write d7/dc/d2a/f30 [4342424,480] 0 2024-09-16T00:26:08.753 INFO:tasks.workunit.client.1.smithi169.stdout:0/608: creat d0/d4c/d82/fd3 x:0 0 0 2024-09-16T00:26:08.754 INFO:tasks.workunit.client.1.smithi169.stdout:2/587: rename d7/db/d27/d29/d37/l9c to d7/d1c/d26/d96/dab/lcf 0 2024-09-16T00:26:08.755 INFO:tasks.workunit.client.1.smithi169.stdout:9/615: dwrite d5/da/d5b/d91/fbd [0,4194304] 0 2024-09-16T00:26:08.755 INFO:tasks.workunit.client.0.smithi112.stdout:4/794: sync 2024-09-16T00:26:08.756 INFO:tasks.workunit.client.1.smithi169.stdout:2/588: chown d7/db/d20/d5c/db2 1686533 1 2024-09-16T00:26:08.757 INFO:tasks.workunit.client.1.smithi169.stdout:2/589: write d7/db/d27/d29/fc9 [86489,64587] 0 2024-09-16T00:26:08.759 INFO:tasks.workunit.client.1.smithi169.stdout:0/609: chown d0/c3a 6270787 1 2024-09-16T00:26:08.763 INFO:tasks.workunit.client.0.smithi112.stdout:9/987: chown d1/d5/d20/d81/d1a/l56 2266 1 2024-09-16T00:26:08.765 INFO:tasks.workunit.client.0.smithi112.stdout:4/795: chown d6/db/df/d3c/l50 441653 1 2024-09-16T00:26:08.765 INFO:tasks.workunit.client.1.smithi169.stdout:0/610: chown d0/d2/d8/d5/d79/f96 10731931 1 2024-09-16T00:26:08.766 INFO:tasks.workunit.client.1.smithi169.stdout:9/616: truncate d5/da/d2e/d47/f7c 312599 0 2024-09-16T00:26:08.769 INFO:tasks.workunit.client.0.smithi112.stdout:4/796: symlink d6/db/df/d5e/l125 0 2024-09-16T00:26:08.769 INFO:tasks.workunit.client.0.smithi112.stdout:5/708: dwrite d4/d1c/fad [0,4194304] 0 2024-09-16T00:26:08.769 INFO:tasks.workunit.client.0.smithi112.stdout:9/988: truncate d1/f50 958892 0 2024-09-16T00:26:08.770 INFO:tasks.workunit.client.1.smithi169.stdout:2/590: dwrite d7/db/f45 [0,4194304] 0 2024-09-16T00:26:08.770 INFO:tasks.workunit.client.0.smithi112.stdout:5/709: chown d4/dd/d12/c67 2045937194 1 2024-09-16T00:26:08.771 INFO:tasks.workunit.client.0.smithi112.stdout:9/989: readlink d1/d5/d20/d81/d1a/l56 0 2024-09-16T00:26:08.771 INFO:tasks.workunit.client.1.smithi169.stdout:6/800: dread d2/d22/d51/f77 [0,4194304] 0 2024-09-16T00:26:08.771 INFO:tasks.workunit.client.1.smithi169.stdout:9/617: write d5/da/d5b/d6f/fa1 [1410728,130418] 0 2024-09-16T00:26:08.771 INFO:tasks.workunit.client.0.smithi112.stdout:4/797: creat d6/d7c/f126 x:0 0 0 2024-09-16T00:26:08.772 INFO:tasks.workunit.client.0.smithi112.stdout:5/710: creat d4/d1c/fea x:0 0 0 2024-09-16T00:26:08.773 INFO:tasks.workunit.client.1.smithi169.stdout:2/591: write d7/db/d27/d29/d77/f86 [3711465,130977] 0 2024-09-16T00:26:08.773 INFO:tasks.workunit.client.0.smithi112.stdout:4/798: stat d6/d39/d1c/d6e/fbe 0 2024-09-16T00:26:08.774 INFO:tasks.workunit.client.0.smithi112.stdout:4/799: rmdir d6/d39 39 2024-09-16T00:26:08.774 INFO:tasks.workunit.client.1.smithi169.stdout:6/801: mkdir d2/d4c/d101 0 2024-09-16T00:26:08.775 INFO:tasks.workunit.client.1.smithi169.stdout:6/802: fdatasync d2/d36/fe0 0 2024-09-16T00:26:08.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:08 smithi169 ceph-mon[34653]: pgmap v257: 65 pgs: 65 active+clean; 2.4 GiB data, 8.4 GiB used, 528 GiB / 536 GiB avail; 94 MiB/s rd, 290 MiB/s wr, 510 op/s 2024-09-16T00:26:08.776 INFO:tasks.workunit.client.0.smithi112.stdout:4/800: stat d6/db/df/d5e/d61/lb4 0 2024-09-16T00:26:08.777 INFO:tasks.workunit.client.0.smithi112.stdout:9/990: truncate d1/d5/d20/d81/d1a/d88/def/f8a 8702987 0 2024-09-16T00:26:08.778 INFO:tasks.workunit.client.1.smithi169.stdout:6/803: write d2/d4d/d59/d74/fff [863129,73004] 0 2024-09-16T00:26:08.778 INFO:tasks.workunit.client.1.smithi169.stdout:2/592: creat d7/db/d89/db4/d69/d92/fd0 x:0 0 0 2024-09-16T00:26:08.780 INFO:tasks.workunit.client.1.smithi169.stdout:9/618: creat d5/da/d5b/fdd x:0 0 0 2024-09-16T00:26:08.781 INFO:tasks.workunit.client.1.smithi169.stdout:6/804: chown d2/d3f/d41/d5c/d73/cd5 0 1 2024-09-16T00:26:08.781 INFO:tasks.workunit.client.1.smithi169.stdout:2/593: readlink d7/db/d27/d29/d37/l2f 0 2024-09-16T00:26:08.782 INFO:tasks.workunit.client.1.smithi169.stdout:9/619: chown d5/da/c18 7 1 2024-09-16T00:26:08.783 INFO:tasks.workunit.client.1.smithi169.stdout:9/620: chown d5/da/c18 84 1 2024-09-16T00:26:08.786 INFO:tasks.workunit.client.0.smithi112.stdout:9/991: dread d1/d5/d6/d2d/d6a/fe6 [0,4194304] 0 2024-09-16T00:26:08.787 INFO:tasks.workunit.client.1.smithi169.stdout:7/653: dread d7/d71/f9d [0,4194304] 0 2024-09-16T00:26:08.789 INFO:tasks.workunit.client.1.smithi169.stdout:6/805: rename d2/d4d/lf3 to d2/d3f/d41/d5c/d91/da6/l102 0 2024-09-16T00:26:08.790 INFO:tasks.workunit.client.0.smithi112.stdout:9/992: symlink d1/d5/d20/d81/d11d/d12b/l158 0 2024-09-16T00:26:08.797 INFO:tasks.workunit.client.0.smithi112.stdout:7/827: dwrite d6/d10/d75/fb9 [0,4194304] 0 2024-09-16T00:26:08.798 INFO:tasks.workunit.client.1.smithi169.stdout:9/621: link d5/d49/f5a d5/da/d2e/fde 0 2024-09-16T00:26:08.799 INFO:tasks.workunit.client.1.smithi169.stdout:6/806: dwrite d2/f19 [0,4194304] 0 2024-09-16T00:26:08.799 INFO:tasks.workunit.client.0.smithi112.stdout:7/828: fdatasync d6/d10/d75/d88/f2d 0 2024-09-16T00:26:08.800 INFO:tasks.workunit.client.1.smithi169.stdout:7/654: dwrite d7/fa0 [0,4194304] 0 2024-09-16T00:26:08.800 INFO:tasks.workunit.client.0.smithi112.stdout:5/711: write d4/dd/f24 [1858436,5940] 0 2024-09-16T00:26:08.801 INFO:tasks.workunit.client.1.smithi169.stdout:7/655: fdatasync f1 0 2024-09-16T00:26:08.802 INFO:tasks.workunit.client.0.smithi112.stdout:7/829: creat d6/d7f/d85/f111 x:0 0 0 2024-09-16T00:26:08.803 INFO:tasks.workunit.client.1.smithi169.stdout:9/622: chown d5/da/d5b/d6c/da0/ld8 1 1 2024-09-16T00:26:08.808 INFO:tasks.workunit.client.0.smithi112.stdout:4/801: write d6/db/f18 [4352212,90160] 0 2024-09-16T00:26:08.811 INFO:tasks.workunit.client.1.smithi169.stdout:7/656: unlink d7/ca2 0 2024-09-16T00:26:08.812 INFO:tasks.workunit.client.0.smithi112.stdout:5/712: link d4/dd/d49/d70/f7e d4/d7/d26/d4c/db3/db6/dd0/feb 0 2024-09-16T00:26:08.813 INFO:tasks.workunit.client.1.smithi169.stdout:6/807: mkdir d2/d4d/d59/d43/d103 0 2024-09-16T00:26:08.814 INFO:tasks.workunit.client.1.smithi169.stdout:6/808: chown d2/d22/d51/led 1427617 1 2024-09-16T00:26:08.815 INFO:tasks.workunit.client.0.smithi112.stdout:7/830: getdents d6/d10/da1/dcd 0 2024-09-16T00:26:08.816 INFO:tasks.workunit.client.0.smithi112.stdout:5/713: symlink d4/d7/d87/lec 0 2024-09-16T00:26:08.817 INFO:tasks.workunit.client.0.smithi112.stdout:5/714: chown d4/dd/d12 1494586 1 2024-09-16T00:26:08.818 INFO:tasks.workunit.client.1.smithi169.stdout:0/611: dread d0/d2/d8/f47 [0,4194304] 0 2024-09-16T00:26:08.818 INFO:tasks.workunit.client.1.smithi169.stdout:9/623: truncate d5/da/d21/d23/f94 149055 0 2024-09-16T00:26:08.818 INFO:tasks.workunit.client.0.smithi112.stdout:7/831: creat d6/d10/d75/dea/f112 x:0 0 0 2024-09-16T00:26:08.819 INFO:tasks.workunit.client.0.smithi112.stdout:9/993: write d1/f62 [771494,68836] 0 2024-09-16T00:26:08.822 INFO:tasks.workunit.client.0.smithi112.stdout:5/715: write d4/dd/d12/d52/d64/fd4 [606750,101684] 0 2024-09-16T00:26:08.822 INFO:tasks.workunit.client.0.smithi112.stdout:7/832: readlink d6/d10/d75/d8f/d90/dbc/lf7 0 2024-09-16T00:26:08.822 INFO:tasks.workunit.client.1.smithi169.stdout:0/612: stat d0/d2c/d54/f28 0 2024-09-16T00:26:08.823 INFO:tasks.workunit.client.0.smithi112.stdout:4/802: dwrite d6/d39/d1c/d2c/d69/f10a [0,4194304] 0 2024-09-16T00:26:08.823 INFO:tasks.workunit.client.1.smithi169.stdout:6/809: truncate d2/d4d/d59/d43/fee 28611 0 2024-09-16T00:26:08.824 INFO:tasks.workunit.client.0.smithi112.stdout:5/716: dread - d4/dd/d49/d70/fd6 zero size 2024-09-16T00:26:08.825 INFO:tasks.workunit.client.1.smithi169.stdout:0/613: rmdir d0/d4c/d82 39 2024-09-16T00:26:08.826 INFO:tasks.workunit.client.1.smithi169.stdout:9/624: dread d5/da/d5b/d91/fbd [0,4194304] 0 2024-09-16T00:26:08.827 INFO:tasks.workunit.client.0.smithi112.stdout:4/803: mknod d6/d39/d36/d5c/dc6/db5/c127 0 2024-09-16T00:26:08.829 INFO:tasks.workunit.client.1.smithi169.stdout:6/810: mknod d2/d4d/d59/c104 0 2024-09-16T00:26:08.829 INFO:tasks.workunit.client.0.smithi112.stdout:5/717: creat d4/dd/d49/de6/fed x:0 0 0 2024-09-16T00:26:08.830 INFO:tasks.workunit.client.0.smithi112.stdout:4/804: rmdir d6/db/d1f/d94/d10c 39 2024-09-16T00:26:08.830 INFO:tasks.workunit.client.0.smithi112.stdout:7/833: read d6/d10/d75/d88/f2d [4985879,65361] 0 2024-09-16T00:26:08.832 INFO:tasks.workunit.client.0.smithi112.stdout:5/718: mkdir d4/dee 0 2024-09-16T00:26:08.832 INFO:tasks.workunit.client.0.smithi112.stdout:8/711: sync 2024-09-16T00:26:08.833 INFO:tasks.workunit.client.0.smithi112.stdout:3/839: sync 2024-09-16T00:26:08.833 INFO:tasks.workunit.client.0.smithi112.stdout:1/848: sync 2024-09-16T00:26:08.834 INFO:tasks.workunit.client.1.smithi169.stdout:6/811: symlink d2/d3f/d41/d72/d9e/de1/de4/l105 0 2024-09-16T00:26:08.834 INFO:tasks.workunit.client.1.smithi169.stdout:0/614: dwrite d0/d2c/d54/d21/fa0 [0,4194304] 0 2024-09-16T00:26:08.835 INFO:tasks.workunit.client.0.smithi112.stdout:7/834: unlink d6/d10/d16/l17 0 2024-09-16T00:26:08.836 INFO:tasks.workunit.client.0.smithi112.stdout:5/719: creat d4/d7/d26/d4c/fef x:0 0 0 2024-09-16T00:26:08.836 INFO:tasks.workunit.client.0.smithi112.stdout:1/849: fsync d0/d14/df/d28/d45/d46/da7/dda/f10c 0 2024-09-16T00:26:08.837 INFO:tasks.workunit.client.0.smithi112.stdout:8/712: chown d10/d12/d1c/d61/d6c/c83 116738824 1 2024-09-16T00:26:08.837 INFO:tasks.workunit.client.0.smithi112.stdout:3/840: chown d2/d9/d2b/d47/d52/d7a/d80/d83/d68 2060669 1 2024-09-16T00:26:08.837 INFO:tasks.workunit.client.0.smithi112.stdout:5/720: fsync d4/d4f/fb1 0 2024-09-16T00:26:08.838 INFO:tasks.workunit.client.0.smithi112.stdout:1/850: chown d0/d14/fd0 3452 1 2024-09-16T00:26:08.838 INFO:tasks.workunit.client.0.smithi112.stdout:7/835: write d6/d10/d16/d49/d6c/f5d [225176,68777] 0 2024-09-16T00:26:08.839 INFO:tasks.workunit.client.1.smithi169.stdout:6/812: creat d2/d3f/d41/d5c/d91/da6/de2/f106 x:0 0 0 2024-09-16T00:26:08.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:08 smithi112 ceph-mon[27869]: pgmap v257: 65 pgs: 65 active+clean; 2.4 GiB data, 8.4 GiB used, 528 GiB / 536 GiB avail; 94 MiB/s rd, 290 MiB/s wr, 510 op/s 2024-09-16T00:26:08.841 INFO:tasks.workunit.client.1.smithi169.stdout:6/813: write d2/d3f/d41/d5c/d91/feb [469445,103963] 0 2024-09-16T00:26:08.841 INFO:tasks.workunit.client.0.smithi112.stdout:1/851: write d0/d14/d1c/f5c [4401258,5048] 0 2024-09-16T00:26:08.842 INFO:tasks.workunit.client.0.smithi112.stdout:8/713: unlink d10/d12/d1c/d61/d6c/dd1/f64 0 2024-09-16T00:26:08.843 INFO:tasks.workunit.client.0.smithi112.stdout:7/836: mknod d6/c113 0 2024-09-16T00:26:08.845 INFO:tasks.workunit.client.0.smithi112.stdout:3/841: creat d2/d9/d2b/d47/d52/d7a/d80/d83/f112 x:0 0 0 2024-09-16T00:26:08.849 INFO:tasks.workunit.client.1.smithi169.stdout:0/615: truncate d0/d2/d8/f73 184871 0 2024-09-16T00:26:08.849 INFO:tasks.workunit.client.1.smithi169.stdout:6/814: dwrite d2/fb [8388608,4194304] 0 2024-09-16T00:26:08.850 INFO:tasks.workunit.client.0.smithi112.stdout:7/837: mkdir d6/d10/d16/d49/d6c/d114 0 2024-09-16T00:26:08.851 INFO:tasks.workunit.client.1.smithi169.stdout:6/815: read d2/d4d/d59/d74/fff [27059,97973] 0 2024-09-16T00:26:08.852 INFO:tasks.workunit.client.0.smithi112.stdout:9/994: dread d1/d5/d20/d81/fd5 [0,4194304] 0 2024-09-16T00:26:08.853 INFO:tasks.workunit.client.0.smithi112.stdout:3/842: unlink d2/d9/d2b/d47/d52/d7a/d80/d83/d49/ffa 0 2024-09-16T00:26:08.854 INFO:tasks.workunit.client.1.smithi169.stdout:0/616: unlink d0/d2c/d54/d56/d67/dd1/d6c/f9d 0 2024-09-16T00:26:08.854 INFO:tasks.workunit.client.0.smithi112.stdout:8/714: dread d10/d12/d1c/d61/d6c/dd1/f6f [0,4194304] 0 2024-09-16T00:26:08.855 INFO:tasks.workunit.client.0.smithi112.stdout:5/721: getdents d4/d7/d14 0 2024-09-16T00:26:08.857 INFO:tasks.workunit.client.0.smithi112.stdout:1/852: dwrite d0/d14/df/d28/d45/d30/fc9 [4194304,4194304] 0 2024-09-16T00:26:08.857 INFO:tasks.workunit.client.1.smithi169.stdout:0/617: symlink d0/d2c/d54/d56/d67/dd1/ld4 0 2024-09-16T00:26:08.857 INFO:tasks.workunit.client.1.smithi169.stdout:6/816: fdatasync d2/d3f/d41/d52/f1b 0 2024-09-16T00:26:08.858 INFO:tasks.workunit.client.1.smithi169.stdout:7/657: dread d7/d5b/f77 [0,4194304] 0 2024-09-16T00:26:08.858 INFO:tasks.workunit.client.0.smithi112.stdout:9/995: readlink d1/d5/d20/d81/d1a/d88/def/l53 0 2024-09-16T00:26:08.859 INFO:tasks.workunit.client.0.smithi112.stdout:3/843: unlink d2/d4d/l6b 0 2024-09-16T00:26:08.862 INFO:tasks.workunit.client.0.smithi112.stdout:7/838: creat d6/d10/d75/d8f/d90/ddc/dee/f115 x:0 0 0 2024-09-16T00:26:08.862 INFO:tasks.workunit.client.1.smithi169.stdout:7/658: write d7/d71/fae [4425879,43048] 0 2024-09-16T00:26:08.862 INFO:tasks.workunit.client.1.smithi169.stdout:0/618: symlink d0/d2c/ld5 0 2024-09-16T00:26:08.863 INFO:tasks.workunit.client.0.smithi112.stdout:6/769: sync 2024-09-16T00:26:08.863 INFO:tasks.workunit.client.0.smithi112.stdout:2/883: sync 2024-09-16T00:26:08.865 INFO:tasks.workunit.client.0.smithi112.stdout:9/996: symlink d1/d5/d20/d81/d11d/l159 0 2024-09-16T00:26:08.866 INFO:tasks.workunit.client.0.smithi112.stdout:4/805: dwrite d6/d39/d36/d5c/d7e/d8f/f90 [0,4194304] 0 2024-09-16T00:26:08.866 INFO:tasks.workunit.client.0.smithi112.stdout:5/722: rmdir d4/d7/d14/d90/dd3 39 2024-09-16T00:26:08.866 INFO:tasks.workunit.client.0.smithi112.stdout:7/839: symlink d6/l116 0 2024-09-16T00:26:08.867 INFO:tasks.workunit.client.1.smithi169.stdout:6/817: dwrite d2/f45 [8388608,4194304] 0 2024-09-16T00:26:08.868 INFO:tasks.workunit.client.0.smithi112.stdout:2/884: readlink d2/dd/d48/d22/d2b/d54/d79/d110/d49/l81 0 2024-09-16T00:26:08.869 INFO:tasks.workunit.client.1.smithi169.stdout:7/659: mknod d7/d16/d49/db2/cd0 0 2024-09-16T00:26:08.869 INFO:tasks.workunit.client.0.smithi112.stdout:1/853: rename d0/d14/df/d28/f5d to d0/d14/df/d28/d45/dd4/d41/f113 0 2024-09-16T00:26:08.872 INFO:tasks.workunit.client.0.smithi112.stdout:1/854: truncate d0/d14/df/d28/d42/dd8/f10b 226944 0 2024-09-16T00:26:08.877 INFO:tasks.workunit.client.1.smithi169.stdout:6/818: dread d2/d3f/d41/d5c/ff8 [0,4194304] 0 2024-09-16T00:26:08.880 INFO:tasks.workunit.client.1.smithi169.stdout:6/819: write d2/d36/fe0 [1953370,79130] 0 2024-09-16T00:26:08.911 INFO:tasks.workunit.client.1.smithi169.stdout:7/660: readlink d7/df/d95/lcc 0 2024-09-16T00:26:08.913 INFO:tasks.workunit.client.1.smithi169.stdout:7/661: chown d7/d16/d49/d84/d8e/lb8 23462 1 2024-09-16T00:26:08.913 INFO:tasks.workunit.client.1.smithi169.stdout:6/820: mknod d2/d3f/d41/d72/d9e/de1/c107 0 2024-09-16T00:26:08.916 INFO:tasks.workunit.client.1.smithi169.stdout:6/821: creat d2/d3f/d41/d72/d9e/dc0/f108 x:0 0 0 2024-09-16T00:26:08.918 INFO:tasks.workunit.client.1.smithi169.stdout:7/662: mknod d7/cd1 0 2024-09-16T00:26:08.921 INFO:tasks.workunit.client.1.smithi169.stdout:0/619: write d0/d2c/d54/d21/d31/fc7 [1840114,30675] 0 2024-09-16T00:26:08.937 INFO:tasks.workunit.client.1.smithi169.stdout:7/663: link d7/d71/d76/d29/d78/dba/lbd d7/d5b/d61/d98/ld2 0 2024-09-16T00:26:08.937 INFO:tasks.workunit.client.1.smithi169.stdout:6/822: rmdir d2/d4c/d101 0 2024-09-16T00:26:08.942 INFO:tasks.workunit.client.1.smithi169.stdout:7/664: getdents d7/d5b/d61 0 2024-09-16T00:26:08.951 INFO:tasks.workunit.client.1.smithi169.stdout:6/823: dread d2/d36/f44 [0,4194304] 0 2024-09-16T00:26:08.953 INFO:tasks.workunit.client.1.smithi169.stdout:6/824: creat d2/d4d/d59/d74/f109 x:0 0 0 2024-09-16T00:26:08.954 INFO:tasks.workunit.client.1.smithi169.stdout:6/825: stat d2/d3f 0 2024-09-16T00:26:08.956 INFO:tasks.workunit.client.1.smithi169.stdout:6/826: mknod d2/d3f/d41/d72/d9e/de1/de4/c10a 0 2024-09-16T00:26:08.962 INFO:tasks.workunit.client.1.smithi169.stdout:7/665: rmdir d7/df/d95 39 2024-09-16T00:26:08.963 INFO:tasks.workunit.client.1.smithi169.stdout:0/620: rename d0/d2/d15/l64 to d0/d2c/d54/d56/ld6 0 2024-09-16T00:26:08.965 INFO:tasks.workunit.client.1.smithi169.stdout:7/666: truncate d7/f26 1575728 0 2024-09-16T00:26:08.966 INFO:tasks.workunit.client.0.smithi112.stdout:9/997: creat d1/d5/d20/d81/dc4/f15a x:0 0 0 2024-09-16T00:26:08.967 INFO:tasks.workunit.client.1.smithi169.stdout:0/621: link d0/d2/f93 d0/d4c/d82/fd7 0 2024-09-16T00:26:08.968 INFO:tasks.workunit.client.1.smithi169.stdout:6/827: write d2/d4d/d59/f7f [2667911,113206] 0 2024-09-16T00:26:08.968 INFO:tasks.workunit.client.0.smithi112.stdout:4/806: creat d6/d39/d36/f128 x:0 0 0 2024-09-16T00:26:08.969 INFO:tasks.workunit.client.0.smithi112.stdout:4/807: chown d6/db/df/d5e/d61 599385 1 2024-09-16T00:26:08.969 INFO:tasks.workunit.client.1.smithi169.stdout:0/622: write d0/d2/d8/f47 [4219192,101237] 0 2024-09-16T00:26:08.970 INFO:tasks.workunit.client.0.smithi112.stdout:5/723: creat d4/dd/d49/d70/d83/ff0 x:0 0 0 2024-09-16T00:26:08.971 INFO:tasks.workunit.client.0.smithi112.stdout:5/724: chown d4/d7/d14/f33 8222 1 2024-09-16T00:26:08.972 INFO:tasks.workunit.client.0.smithi112.stdout:7/840: truncate d6/d10/d75/d8f/f99 1785222 0 2024-09-16T00:26:08.974 INFO:tasks.workunit.client.0.smithi112.stdout:8/715: rename d10/d12/d1c/d61/d6c/dd1/d8c/d98/fe0 to d10/d12/d1c/d3b/d7d/de4/fed 0 2024-09-16T00:26:08.980 INFO:tasks.workunit.client.0.smithi112.stdout:8/716: dwrite d10/d12/d24/fcc [0,4194304] 0 2024-09-16T00:26:08.989 INFO:tasks.workunit.client.0.smithi112.stdout:9/998: chown d1/d5/l5e 52571454 1 2024-09-16T00:26:08.989 INFO:tasks.workunit.client.0.smithi112.stdout:4/808: rmdir d6/d39/d36/d5c/d7e/dab/de4 39 2024-09-16T00:26:08.991 INFO:tasks.workunit.client.0.smithi112.stdout:5/725: symlink d4/d7/d26/d4c/lf1 0 2024-09-16T00:26:08.994 INFO:tasks.workunit.client.0.smithi112.stdout:7/841: rename d6/d10/d16/d49 to d6/d10/d16/d96/dc7/dbf/d117 0 2024-09-16T00:26:08.996 INFO:tasks.workunit.client.0.smithi112.stdout:2/885: link d2/f11c d2/dd/d48/d8a/db9/de7/f11d 0 2024-09-16T00:26:08.998 INFO:tasks.workunit.client.0.smithi112.stdout:3/844: getdents d2/d9/d2b/d47/d52/d7a/d80/d83/d7b 0 2024-09-16T00:26:09.000 INFO:tasks.workunit.client.0.smithi112.stdout:9/999: rmdir d1/d5/d20/d81/d1a/d88/def/d11b 39 2024-09-16T00:26:09.001 INFO:tasks.workunit.client.0.smithi112.stdout:5/726: creat d4/d1c/d43/ff2 x:0 0 0 2024-09-16T00:26:09.002 INFO:tasks.workunit.client.0.smithi112.stdout:7/842: creat d6/d10/da1/da2/f118 x:0 0 0 2024-09-16T00:26:09.003 INFO:tasks.workunit.client.0.smithi112.stdout:5/727: write d4/dd/d49/d70/dc5/fe1 [824964,37000] 0 2024-09-16T00:26:09.004 INFO:tasks.workunit.client.0.smithi112.stdout:5/728: fdatasync d4/d7/d26/d36/f41 0 2024-09-16T00:26:09.004 INFO:tasks.workunit.client.0.smithi112.stdout:4/809: symlink d6/d39/d36/d5c/d7e/d8f/d11e/l129 0 2024-09-16T00:26:09.006 INFO:tasks.workunit.client.0.smithi112.stdout:3/845: mknod d2/d9/d2b/d47/d52/d7a/d80/d83/d7b/dad/d107/d69/dd5/d105/c113 0 2024-09-16T00:26:09.009 INFO:tasks.workunit.client.0.smithi112.stdout:2/886: dread d2/dd/d48/d22/d67/fb2 [4194304,4194304] 0 2024-09-16T00:26:09.010 INFO:tasks.workunit.client.0.smithi112.stdout:2/887: stat d2/f64 0 2024-09-16T00:26:09.010 INFO:tasks.workunit.client.0.smithi112.stdout:5/729: symlink d4/d4f/d78/lf3 0 2024-09-16T00:26:09.014 INFO:tasks.workunit.client.0.smithi112.stdout:3/846: symlink d2/d9/d2b/d47/d52/d7a/d80/d83/d7b/dad/d107/d108/l114 0 2024-09-16T00:26:09.014 INFO:tasks.workunit.client.0.smithi112.stdout:4/810: creat d6/db/d1f/d94/d10c/d123/f12a x:0 0 0 2024-09-16T00:26:09.016 INFO:tasks.workunit.client.0.smithi112.stdout:2/888: mkdir d2/dd/d48/d22/d2b/d54/d79/d110/d49/d100/d105/d11e 0 2024-09-16T00:26:09.017 INFO:tasks.workunit.client.0.smithi112.stdout:3/847: symlink d2/d9/d2b/d47/d52/d7a/d80/d83/d7b/dad/d107/d69/dd5/l115 0 2024-09-16T00:26:09.019 INFO:tasks.workunit.client.0.smithi112.stdout:4/811: creat d6/d39/d1c/d2c/d69/d7b/de0/f12b x:0 0 0 2024-09-16T00:26:09.021 INFO:tasks.workunit.client.0.smithi112.stdout:3/848: creat d2/d9/d2b/f116 x:0 0 0 2024-09-16T00:26:09.022 INFO:tasks.workunit.client.0.smithi112.stdout:2/889: mknod d2/dd/d48/d22/d2b/d54/d79/d110/c11f 0 2024-09-16T00:26:09.023 INFO:tasks.workunit.client.0.smithi112.stdout:4/812: mkdir d6/d39/d36/d5c/d7e/d8f/d12c 0 2024-09-16T00:26:09.024 INFO:tasks.workunit.client.0.smithi112.stdout:4/813: dread - d6/db/df/d3c/f101 zero size 2024-09-16T00:26:09.024 INFO:tasks.workunit.client.0.smithi112.stdout:5/730: dread d4/f19 [4194304,4194304] 0 2024-09-16T00:26:09.026 INFO:tasks.workunit.client.0.smithi112.stdout:3/849: mkdir d2/d9/d2b/d47/d52/d117 0 2024-09-16T00:26:09.026 INFO:tasks.workunit.client.0.smithi112.stdout:4/814: dread - d6/d39/d36/d5c/d7e/f112 zero size 2024-09-16T00:26:09.030 INFO:tasks.workunit.client.0.smithi112.stdout:4/815: mkdir d6/d39/d1c/d66/d12d 0 2024-09-16T00:26:09.032 INFO:tasks.workunit.client.0.smithi112.stdout:3/850: dwrite d2/d9/d2b/d47/f54 [0,4194304] 0 2024-09-16T00:26:09.033 INFO:tasks.workunit.client.0.smithi112.stdout:5/731: mknod d4/d7/d26/d4c/db3/db6/cf4 0 2024-09-16T00:26:09.033 INFO:tasks.workunit.client.0.smithi112.stdout:2/890: getdents d2/dd/d48/d22/d2b/d54/d60/d6c/dc9 0 2024-09-16T00:26:09.034 INFO:tasks.workunit.client.0.smithi112.stdout:3/851: chown d2/d9/d2b/d47/d52/d7a/d80/d83/l41 1 1 2024-09-16T00:26:09.038 INFO:tasks.workunit.client.0.smithi112.stdout:3/852: mknod d2/d9/d2b/d47/d78/c118 0 2024-09-16T00:26:09.039 INFO:tasks.workunit.client.0.smithi112.stdout:5/732: mknod d4/dee/cf5 0 2024-09-16T00:26:09.041 INFO:tasks.workunit.client.0.smithi112.stdout:1/855: truncate d0/d14/df/f36 552559 0 2024-09-16T00:26:09.042 INFO:tasks.workunit.client.0.smithi112.stdout:8/717: write d10/d12/f16 [1681834,65396] 0 2024-09-16T00:26:09.045 INFO:tasks.workunit.client.0.smithi112.stdout:0/919: sync 2024-09-16T00:26:09.046 INFO:tasks.workunit.client.0.smithi112.stdout:1/856: symlink d0/d14/df/d28/d45/d46/d58/df9/l114 0 2024-09-16T00:26:09.047 INFO:tasks.workunit.client.0.smithi112.stdout:7/843: write d6/d7f/f8d [628542,93718] 0 2024-09-16T00:26:09.049 INFO:tasks.workunit.client.0.smithi112.stdout:3/853: creat d2/d9/d2b/d47/d52/f119 x:0 0 0 2024-09-16T00:26:09.050 INFO:tasks.workunit.client.0.smithi112.stdout:8/718: unlink d10/d12/d1c/d61/d6c/dd1/d8c/d98/lb8 0 2024-09-16T00:26:09.051 INFO:tasks.workunit.client.0.smithi112.stdout:5/733: dwrite d4/d7/f4d [0,4194304] 0 2024-09-16T00:26:09.051 INFO:tasks.workunit.client.0.smithi112.stdout:2/891: getdents d2/dd/d48/d22/d2b/d54/d60/d6c/d88/dad 0 2024-09-16T00:26:09.052 INFO:tasks.workunit.client.0.smithi112.stdout:8/719: stat d10/d12/d1c/d61/fb4 0 2024-09-16T00:26:09.053 INFO:tasks.workunit.client.0.smithi112.stdout:5/734: chown d4/d1c/d43/f54 32095742 1 2024-09-16T00:26:09.053 INFO:tasks.workunit.client.0.smithi112.stdout:0/920: mknod d5/da/dc/c11e 0 2024-09-16T00:26:09.055 INFO:tasks.workunit.client.0.smithi112.stdout:7/844: dread - d6/d10/d16/d96/dc7/d39/fa7 zero size 2024-09-16T00:26:09.055 INFO:tasks.workunit.client.0.smithi112.stdout:0/921: write d5/da/d18/d63/fa3 [767295,36639] 0 2024-09-16T00:26:09.057 INFO:tasks.workunit.client.0.smithi112.stdout:1/857: mknod d0/d14/df/d28/d42/dd8/de8/c115 0 2024-09-16T00:26:09.059 INFO:tasks.workunit.client.0.smithi112.stdout:3/854: symlink d2/d9/d2b/d47/d52/d7a/d80/d83/d7b/dad/l11a 0 2024-09-16T00:26:09.059 INFO:tasks.workunit.client.1.smithi169.stdout:6/828: creat d2/d4d/d59/d43/f10b x:0 0 0 2024-09-16T00:26:09.061 INFO:tasks.workunit.client.1.smithi169.stdout:0/623: mknod d0/d2/d15/dc5/cd8 0 2024-09-16T00:26:09.062 INFO:tasks.workunit.client.0.smithi112.stdout:3/855: dread - d2/d9/d2b/f116 zero size 2024-09-16T00:26:09.063 INFO:tasks.workunit.client.1.smithi169.stdout:0/624: truncate d0/d2/d15/fc1 719943 0 2024-09-16T00:26:09.063 INFO:tasks.workunit.client.0.smithi112.stdout:2/892: dwrite d2/dd/d48/d22/d2b/d54/d60/daf/fc1 [4194304,4194304] 0 2024-09-16T00:26:09.065 INFO:tasks.workunit.client.0.smithi112.stdout:3/856: fsync d2/d9/d2b/d47/d52/d7a/d80/d83/d76/d8e/fb6 0 2024-09-16T00:26:09.065 INFO:tasks.workunit.client.0.smithi112.stdout:8/720: rename d10/d12/d1c/dd0/dbb/fc0 to d10/d12/d1c/d61/d6c/ddc/fee 0 2024-09-16T00:26:09.066 INFO:tasks.workunit.client.0.smithi112.stdout:2/893: fdatasync d2/dd/d48/fef 0 2024-09-16T00:26:09.069 INFO:tasks.workunit.client.0.smithi112.stdout:1/858: symlink d0/d14/d6a/d9e/l116 0 2024-09-16T00:26:09.074 INFO:tasks.workunit.client.0.smithi112.stdout:0/922: mkdir d5/da/d18/d24/d46/d11f 0 2024-09-16T00:26:09.076 INFO:tasks.workunit.client.0.smithi112.stdout:5/735: dread d4/d7/d14/d90/fa4 [0,4194304] 0 2024-09-16T00:26:09.077 INFO:tasks.workunit.client.0.smithi112.stdout:2/894: unlink d2/dd/d48/d22/d2b/d54/d60/l107 0 2024-09-16T00:26:09.077 INFO:tasks.workunit.client.0.smithi112.stdout:4/816: dwrite d6/db/d1f/d41/d49/f4c [0,4194304] 0 2024-09-16T00:26:09.078 INFO:tasks.workunit.client.0.smithi112.stdout:5/736: dread - d4/dd/d49/d70/fd6 zero size 2024-09-16T00:26:09.079 INFO:tasks.workunit.client.0.smithi112.stdout:3/857: creat d2/d9/d2b/d47/d52/d7a/d80/d83/d68/f11b x:0 0 0 2024-09-16T00:26:09.080 INFO:tasks.workunit.client.0.smithi112.stdout:0/923: read d5/da/d1d/d71/f4d [2469429,108215] 0 2024-09-16T00:26:09.082 INFO:tasks.workunit.client.0.smithi112.stdout:8/721: rename d10/d12/d1c/d61/d6c/dd1/d8c/d98 to d10/de9/def 0 2024-09-16T00:26:09.085 INFO:tasks.workunit.client.0.smithi112.stdout:8/722: write d10/d12/d1c/d61/d6c/fdb [3005335,82161] 0 2024-09-16T00:26:09.087 INFO:tasks.workunit.client.0.smithi112.stdout:8/723: chown d10/c7e 27521184 1 2024-09-16T00:26:09.088 INFO:tasks.workunit.client.0.smithi112.stdout:8/724: fdatasync d10/d12/fd7 0 2024-09-16T00:26:09.089 INFO:tasks.workunit.client.0.smithi112.stdout:2/895: dwrite d2/dd/d48/d22/d2b/d54/d79/d110/d49/f7d [0,4194304] 0 2024-09-16T00:26:09.090 INFO:tasks.workunit.client.0.smithi112.stdout:0/924: chown d5/da/d21/c9c 240205421 1 2024-09-16T00:26:09.091 INFO:tasks.workunit.client.0.smithi112.stdout:8/725: chown d10/d12/d1c/d3b/le8 0 1 2024-09-16T00:26:09.096 INFO:tasks.workunit.client.0.smithi112.stdout:4/817: mkdir d6/d39/d1c/d6e/d6f/dda/d12e 0 2024-09-16T00:26:09.097 INFO:tasks.workunit.client.0.smithi112.stdout:3/858: symlink d2/d9/d2b/d47/d52/d7a/df3/l11c 0 2024-09-16T00:26:09.097 INFO:tasks.workunit.client.0.smithi112.stdout:4/818: chown d6/db/l105 22658 1 2024-09-16T00:26:09.101 INFO:tasks.workunit.client.0.smithi112.stdout:8/726: mkdir d10/d12/d1c/d61/d6c/dd1/d58/d87/df0 0 2024-09-16T00:26:09.103 INFO:tasks.workunit.client.0.smithi112.stdout:3/859: creat d2/d9/d2b/d47/d52/d7a/d80/d83/d7b/dad/d107/d69/f11d x:0 0 0 2024-09-16T00:26:09.104 INFO:tasks.workunit.client.0.smithi112.stdout:0/925: mknod d5/da/d21/dc3/c120 0 2024-09-16T00:26:09.106 INFO:tasks.workunit.client.0.smithi112.stdout:4/819: mknod d6/d39/d36/d5c/d7e/d8f/d12c/c12f 0 2024-09-16T00:26:09.106 INFO:tasks.workunit.client.0.smithi112.stdout:0/926: dread - d5/da/d1d/d71/d64/db6/f104 zero size 2024-09-16T00:26:09.108 INFO:tasks.workunit.client.0.smithi112.stdout:3/860: dread d2/d9/d2b/d47/f8a [4194304,4194304] 0 2024-09-16T00:26:09.108 INFO:tasks.workunit.client.0.smithi112.stdout:2/896: creat d2/dd/d48/d22/d2b/d54/d60/d6c/f120 x:0 0 0 2024-09-16T00:26:09.109 INFO:tasks.workunit.client.0.smithi112.stdout:8/727: symlink d10/d12/d1c/d61/d6c/dd1/d8c/d7c/lf1 0 2024-09-16T00:26:09.109 INFO:tasks.workunit.client.0.smithi112.stdout:3/861: readlink d2/d9/ldd 0 2024-09-16T00:26:09.111 INFO:tasks.workunit.client.0.smithi112.stdout:4/820: creat d6/db/d1f/d94/d10c/d123/f130 x:0 0 0 2024-09-16T00:26:09.111 INFO:tasks.workunit.client.0.smithi112.stdout:8/728: write d10/d12/d1c/d3b/fd2 [494007,44350] 0 2024-09-16T00:26:09.112 INFO:tasks.workunit.client.0.smithi112.stdout:0/927: mknod d5/da/d1d/d71/d64/db6/c121 0 2024-09-16T00:26:09.112 INFO:tasks.workunit.client.0.smithi112.stdout:5/737: truncate d4/d4f/d77/fa2 4428789 0 2024-09-16T00:26:09.113 INFO:tasks.workunit.client.0.smithi112.stdout:2/897: creat d2/dd/d48/d22/d2b/d54/d60/daf/f121 x:0 0 0 2024-09-16T00:26:09.114 INFO:tasks.workunit.client.0.smithi112.stdout:2/898: dread - d2/dd/d48/fef zero size 2024-09-16T00:26:09.114 INFO:tasks.workunit.client.0.smithi112.stdout:4/821: symlink d6/db/df/d5e/l131 0 2024-09-16T00:26:09.118 INFO:tasks.workunit.client.0.smithi112.stdout:0/928: read d5/da/d21/d6f/f84 [530422,91143] 0 2024-09-16T00:26:09.118 INFO:tasks.workunit.client.0.smithi112.stdout:3/862: mknod d2/d9/d2b/d47/d52/d7a/d80/d83/d76/c11e 0 2024-09-16T00:26:09.119 INFO:tasks.workunit.client.0.smithi112.stdout:8/729: mkdir d10/d12/d1c/df2 0 2024-09-16T00:26:09.120 INFO:tasks.workunit.client.0.smithi112.stdout:3/863: stat d2/d9/d2b/d47/d52/d7a/d80/cf2 0 2024-09-16T00:26:09.120 INFO:tasks.workunit.client.0.smithi112.stdout:5/738: mkdir d4/dd/df6 0 2024-09-16T00:26:09.121 INFO:tasks.workunit.client.1.smithi169.stdout:0/625: rename d0/d2/fb5 to d0/d2/d8/d5/d60/fd9 0 2024-09-16T00:26:09.122 INFO:tasks.workunit.client.0.smithi112.stdout:2/899: mkdir d2/dd/d48/d8a/d101/d122 0 2024-09-16T00:26:09.122 INFO:tasks.workunit.client.1.smithi169.stdout:0/626: fsync d0/d2c/d54/d21/d31/fbc 0 2024-09-16T00:26:09.123 INFO:tasks.workunit.client.0.smithi112.stdout:5/739: mknod d4/dd/d49/cf7 0 2024-09-16T00:26:09.125 INFO:tasks.workunit.client.0.smithi112.stdout:8/730: getdents d10/d12/d1c/df2 0 2024-09-16T00:26:09.125 INFO:tasks.workunit.client.0.smithi112.stdout:5/740: stat d4/d7/d26/d4c/db3/db6/dd0/cd8 0 2024-09-16T00:26:09.125 INFO:tasks.workunit.client.1.smithi169.stdout:0/627: truncate d0/d7d/fa8 374831 0 2024-09-16T00:26:09.126 INFO:tasks.workunit.client.0.smithi112.stdout:5/741: dread - d4/d1c/fea zero size 2024-09-16T00:26:09.126 INFO:tasks.workunit.client.1.smithi169.stdout:6/829: dwrite d2/d36/f31 [0,4194304] 0 2024-09-16T00:26:09.127 INFO:tasks.workunit.client.0.smithi112.stdout:2/900: mkdir d2/dd/d48/d22/d2b/d54/d60/daf/d11a/d123 0 2024-09-16T00:26:09.127 INFO:tasks.workunit.client.0.smithi112.stdout:8/731: fdatasync d10/d12/d1c/d61/d6c/fb5 0 2024-09-16T00:26:09.127 INFO:tasks.workunit.client.1.smithi169.stdout:0/628: read d0/d2/d4b/d86/f5a [393283,85480] 0 2024-09-16T00:26:09.129 INFO:tasks.workunit.client.0.smithi112.stdout:3/864: dread d2/d9/d2b/d47/d52/d7a/d80/d83/d76/d8e/f104 [0,4194304] 0 2024-09-16T00:26:09.130 INFO:tasks.workunit.client.0.smithi112.stdout:2/901: mknod d2/dd/d48/d22/c124 0 2024-09-16T00:26:09.131 INFO:tasks.workunit.client.0.smithi112.stdout:2/902: fsync d2/dd/d48/d8a/ff7 0 2024-09-16T00:26:09.132 INFO:tasks.workunit.client.1.smithi169.stdout:6/830: write d2/d3f/d41/d52/f18 [1952086,125723] 0 2024-09-16T00:26:09.132 INFO:tasks.workunit.client.1.smithi169.stdout:0/629: creat d0/d4c/d6f/fda x:0 0 0 2024-09-16T00:26:09.132 INFO:tasks.workunit.client.1.smithi169.stdout:6/831: mknod d2/d3f/d41/d5c/db9/c10c 0 2024-09-16T00:26:09.134 INFO:tasks.workunit.client.1.smithi169.stdout:0/630: truncate d0/d2c/d38/f84 1363840 0 2024-09-16T00:26:09.134 INFO:tasks.workunit.client.1.smithi169.stdout:0/631: chown d0/d4c/d6f 1625 1 2024-09-16T00:26:09.135 INFO:tasks.workunit.client.1.smithi169.stdout:0/632: write d0/fd0 [562011,114399] 0 2024-09-16T00:26:09.136 INFO:tasks.workunit.client.1.smithi169.stdout:0/633: dread - d0/d2c/d38/fc6 zero size 2024-09-16T00:26:09.140 INFO:tasks.workunit.client.0.smithi112.stdout:3/865: readlink d2/d9/l27 0 2024-09-16T00:26:09.140 INFO:tasks.workunit.client.1.smithi169.stdout:0/634: getdents d0/d2c/d54/d56/d67/dd1/d6c 0 2024-09-16T00:26:09.141 INFO:tasks.workunit.client.0.smithi112.stdout:3/866: chown d2/d9/d2b/d47/d52/d7a/d80/d83/d49/f9c 4971786 1 2024-09-16T00:26:09.142 INFO:tasks.workunit.client.0.smithi112.stdout:8/732: mknod d10/d12/d1c/d61/d6c/dd1/d58/d87/df0/cf3 0 2024-09-16T00:26:09.143 INFO:tasks.workunit.client.1.smithi169.stdout:0/635: mkdir d0/d2/d8/d5/d60/ddb 0 2024-09-16T00:26:09.143 INFO:tasks.workunit.client.0.smithi112.stdout:5/742: dread d4/d4f/d77/dcc/d7c/f88 [0,4194304] 0 2024-09-16T00:26:09.143 INFO:tasks.workunit.client.1.smithi169.stdout:0/636: chown d0/d2/d8/d5/d60/ddb 75759973 1 2024-09-16T00:26:09.144 INFO:tasks.workunit.client.0.smithi112.stdout:0/929: dread d5/da/d18/d24/d46/d88/fb4 [0,4194304] 0 2024-09-16T00:26:09.145 INFO:tasks.workunit.client.0.smithi112.stdout:5/743: dread - d4/d1c/d43/ff2 zero size 2024-09-16T00:26:09.145 INFO:tasks.workunit.client.0.smithi112.stdout:8/733: rmdir d10/d12/d24 39 2024-09-16T00:26:09.146 INFO:tasks.workunit.client.0.smithi112.stdout:0/930: stat d5/da/d18/d24/d46/d88/f8c 0 2024-09-16T00:26:09.146 INFO:tasks.workunit.client.0.smithi112.stdout:2/903: getdents d2/dd/d48/d22/d2b/d54/d60/daf 0 2024-09-16T00:26:09.148 INFO:tasks.workunit.client.1.smithi169.stdout:0/637: creat d0/d2c/fdc x:0 0 0 2024-09-16T00:26:09.149 INFO:tasks.workunit.client.1.smithi169.stdout:0/638: write d0/d2/d15/f7a [261623,24806] 0 2024-09-16T00:26:09.150 INFO:tasks.workunit.client.0.smithi112.stdout:5/744: creat d4/dee/ff8 x:0 0 0 2024-09-16T00:26:09.150 INFO:tasks.workunit.client.0.smithi112.stdout:8/734: truncate d10/d12/d24/f2b 5154012 0 2024-09-16T00:26:09.151 INFO:tasks.workunit.client.1.smithi169.stdout:0/639: dread d0/d2c/d54/d21/f8e [0,4194304] 0 2024-09-16T00:26:09.151 INFO:tasks.workunit.client.0.smithi112.stdout:4/822: truncate d6/d39/d36/d5c/d7e/dab/de4/f120 2167127 0 2024-09-16T00:26:09.153 INFO:tasks.workunit.client.1.smithi169.stdout:0/640: mknod d0/d2c/d54/cdd 0 2024-09-16T00:26:09.153 INFO:tasks.workunit.client.0.smithi112.stdout:2/904: symlink d2/dd/d48/d22/d2b/d54/d79/d110/d49/d100/d105/l125 0 2024-09-16T00:26:09.154 INFO:tasks.workunit.client.0.smithi112.stdout:2/905: chown d2/dd/d48 3404 1 2024-09-16T00:26:09.154 INFO:tasks.workunit.client.1.smithi169.stdout:0/641: creat d0/d7d/fde x:0 0 0 2024-09-16T00:26:09.155 INFO:tasks.workunit.client.0.smithi112.stdout:3/867: link d2/f97 d2/d9/d2b/d47/d52/d7a/d80/d83/d10b/f11f 0 2024-09-16T00:26:09.155 INFO:tasks.workunit.client.0.smithi112.stdout:2/906: dread - d2/f11c zero size 2024-09-16T00:26:09.157 INFO:tasks.workunit.client.0.smithi112.stdout:2/907: dread - d2/dd/d48/d22/d2b/d54/d60/d6c/f120 zero size 2024-09-16T00:26:09.160 INFO:tasks.workunit.client.1.smithi169.stdout:0/642: write d0/d2/d8/fe [3209039,64862] 0 2024-09-16T00:26:09.162 INFO:tasks.workunit.client.1.smithi169.stdout:6/832: dread d2/d3f/d41/d52/fd2 [0,4194304] 0 2024-09-16T00:26:09.164 INFO:tasks.workunit.client.1.smithi169.stdout:0/643: mknod d0/d2/d4b/d86/cdf 0 2024-09-16T00:26:09.165 INFO:tasks.workunit.client.0.smithi112.stdout:0/931: getdents d5/da/d1d/d71/d64/d106/dc0 0 2024-09-16T00:26:09.165 INFO:tasks.workunit.client.1.smithi169.stdout:0/644: chown d0/d2c/d54/d21/d31/d83/f8a 7487 1 2024-09-16T00:26:09.166 INFO:tasks.workunit.client.1.smithi169.stdout:6/833: mkdir d2/da4/d10d 0 2024-09-16T00:26:09.167 INFO:tasks.workunit.client.1.smithi169.stdout:0/645: creat d0/d2c/d54/d21/d31/d83/fe0 x:0 0 0 2024-09-16T00:26:09.167 INFO:tasks.workunit.client.0.smithi112.stdout:2/908: unlink d2/dd/d48/d22/d2b/d54/d60/d6c/d88/da4/cfb 0 2024-09-16T00:26:09.169 INFO:tasks.workunit.client.0.smithi112.stdout:3/868: creat d2/d9/d2b/d47/d52/d7a/d80/f120 x:0 0 0 2024-09-16T00:26:09.171 INFO:tasks.workunit.client.0.smithi112.stdout:5/745: dwrite d4/f57 [0,4194304] 0 2024-09-16T00:26:09.174 INFO:tasks.workunit.client.0.smithi112.stdout:0/932: creat d5/da/d18/d63/db2/dd4/f122 x:0 0 0 2024-09-16T00:26:09.174 INFO:tasks.workunit.client.0.smithi112.stdout:8/735: dread d10/d12/d1c/dd0/fa0 [8388608,4194304] 0 2024-09-16T00:26:09.175 INFO:tasks.workunit.client.1.smithi169.stdout:6/834: dwrite d2/d3f/d41/d52/f1b [0,4194304] 0 2024-09-16T00:26:09.176 INFO:tasks.workunit.client.0.smithi112.stdout:4/823: dwrite d6/d39/d36/d5c/f63 [0,4194304] 0 2024-09-16T00:26:09.176 INFO:tasks.workunit.client.0.smithi112.stdout:0/933: chown d5/da/d1d/d71/l36 254 1 2024-09-16T00:26:09.177 INFO:tasks.workunit.client.1.smithi169.stdout:6/835: fdatasync d2/d36/d48/f90 0 2024-09-16T00:26:09.182 INFO:tasks.workunit.client.0.smithi112.stdout:6/770: sync 2024-09-16T00:26:09.184 INFO:tasks.workunit.client.0.smithi112.stdout:0/934: truncate d5/da/d57/fc8 666280 0 2024-09-16T00:26:09.185 INFO:tasks.workunit.client.1.smithi169.stdout:6/836: dread d2/d4d/f57 [0,4194304] 0 2024-09-16T00:26:09.186 INFO:tasks.workunit.client.1.smithi169.stdout:6/837: truncate d2/f19 5076456 0 2024-09-16T00:26:09.187 INFO:tasks.workunit.client.0.smithi112.stdout:5/746: dwrite d4/dd/d49/d70/d83/ff0 [0,4194304] 0 2024-09-16T00:26:09.188 INFO:tasks.workunit.client.1.smithi169.stdout:6/838: dread - d2/d3f/d41/d5c/d91/da6/fc5 zero size 2024-09-16T00:26:09.188 INFO:tasks.workunit.client.1.smithi169.stdout:6/839: stat d2/d3f/d41/d52/f18 0 2024-09-16T00:26:09.189 INFO:tasks.workunit.client.1.smithi169.stdout:6/840: chown d2/d3f/d41/d5c/d91/da6/lcb 12 1 2024-09-16T00:26:09.190 INFO:tasks.workunit.client.0.smithi112.stdout:6/771: write dd/df/d10/d47/f57 [5965214,96646] 0 2024-09-16T00:26:09.191 INFO:tasks.workunit.client.1.smithi169.stdout:6/841: stat d2/d4d/c21 0 2024-09-16T00:26:09.191 INFO:tasks.workunit.client.0.smithi112.stdout:5/747: creat d4/d4f/d78/ff9 x:0 0 0 2024-09-16T00:26:09.192 INFO:tasks.workunit.client.0.smithi112.stdout:6/772: write dd/df/d13/ffc [772311,78472] 0 2024-09-16T00:26:09.194 INFO:tasks.workunit.client.0.smithi112.stdout:8/736: dread d10/d12/d24/f29 [0,4194304] 0 2024-09-16T00:26:09.195 INFO:tasks.workunit.client.0.smithi112.stdout:0/935: mkdir d5/da/d57/d123 0 2024-09-16T00:26:09.195 INFO:tasks.workunit.client.0.smithi112.stdout:5/748: stat d4/d7/d26/d4c/db3/db6/cf4 0 2024-09-16T00:26:09.198 INFO:tasks.workunit.client.0.smithi112.stdout:2/909: dwrite d2/dd/d48/d22/d2b/f58 [0,4194304] 0 2024-09-16T00:26:09.199 INFO:tasks.workunit.client.0.smithi112.stdout:6/773: mknod dd/df/d10/d47/d89/cfe 0 2024-09-16T00:26:09.199 INFO:tasks.workunit.client.0.smithi112.stdout:5/749: chown d4/d7/d26/d36/f41 129 1 2024-09-16T00:26:09.200 INFO:tasks.workunit.client.0.smithi112.stdout:0/936: mknod d5/da/d18/d24/d46/d5f/c124 0 2024-09-16T00:26:09.201 INFO:tasks.workunit.client.0.smithi112.stdout:2/910: write d2/dd/d48/d22/d2b/d54/d60/da9/fcc [3114012,5150] 0 2024-09-16T00:26:09.201 INFO:tasks.workunit.client.0.smithi112.stdout:8/737: symlink d10/d12/d1c/d3b/d7d/de4/lf4 0 2024-09-16T00:26:09.203 INFO:tasks.workunit.client.0.smithi112.stdout:6/774: read - dd/df/d13/fc1 zero size 2024-09-16T00:26:09.206 INFO:tasks.workunit.client.0.smithi112.stdout:8/738: rmdir d10/d12/d1c/d61/d6c/dd1 39 2024-09-16T00:26:09.207 INFO:tasks.workunit.client.0.smithi112.stdout:3/869: dread d2/d9/d2b/f82 [0,4194304] 0 2024-09-16T00:26:09.208 INFO:tasks.workunit.client.0.smithi112.stdout:8/739: read d10/d12/d1c/d61/fa2 [355568,6526] 0 2024-09-16T00:26:09.208 INFO:tasks.workunit.client.0.smithi112.stdout:2/911: truncate d2/dd/d48/d22/d2b/d30/f36 755733 0 2024-09-16T00:26:09.210 INFO:tasks.workunit.client.0.smithi112.stdout:2/912: chown d2/f11c 1370002 1 2024-09-16T00:26:09.210 INFO:tasks.workunit.client.0.smithi112.stdout:4/824: truncate d6/d39/d1c/d6e/d6f/f97 422803 0 2024-09-16T00:26:09.212 INFO:tasks.workunit.client.0.smithi112.stdout:0/937: dread d5/da/d1d/d71/d64/d106/fbf [0,4194304] 0 2024-09-16T00:26:09.213 INFO:tasks.workunit.client.1.smithi169.stdout:6/842: write d2/d22/fda [973205,39218] 0 2024-09-16T00:26:09.216 INFO:tasks.workunit.client.0.smithi112.stdout:3/870: symlink d2/d9/dfb/l121 0 2024-09-16T00:26:09.216 INFO:tasks.workunit.client.0.smithi112.stdout:0/938: readlink d5/da/d1d/d71/l36 0 2024-09-16T00:26:09.218 INFO:tasks.workunit.client.0.smithi112.stdout:2/913: creat d2/dd/d48/d22/d2b/d54/d60/d6c/dc9/f126 x:0 0 0 2024-09-16T00:26:09.218 INFO:tasks.workunit.client.0.smithi112.stdout:3/871: read d2/d9/d2b/f29 [2675169,83485] 0 2024-09-16T00:26:09.219 INFO:tasks.workunit.client.1.smithi169.stdout:6/843: truncate d2/d22/d51/f77 2784192 0 2024-09-16T00:26:09.219 INFO:tasks.workunit.client.0.smithi112.stdout:4/825: rename d6/d39/d1c/d2c/d69/d7b/de0/f12b to d6/db/df/d5e/f132 0 2024-09-16T00:26:09.220 INFO:tasks.workunit.client.0.smithi112.stdout:5/750: dread d4/d7/d14/f33 [0,4194304] 0 2024-09-16T00:26:09.221 INFO:tasks.workunit.client.0.smithi112.stdout:0/939: creat d5/da/d1d/d71/d64/f125 x:0 0 0 2024-09-16T00:26:09.223 INFO:tasks.workunit.client.0.smithi112.stdout:4/826: chown d6/db/d1f/d41/d49/d31/c111 186 1 2024-09-16T00:26:09.227 INFO:tasks.workunit.client.0.smithi112.stdout:8/740: rename d10/d12/d1c/d61/d6c/dd1/d8c/fab to d10/d12/d1c/ff5 0 2024-09-16T00:26:09.227 INFO:tasks.workunit.client.0.smithi112.stdout:3/872: dwrite d2/f24 [0,4194304] 0 2024-09-16T00:26:09.228 INFO:tasks.workunit.client.0.smithi112.stdout:2/914: dread d2/dd/d48/d22/d2b/d54/d79/d110/fbe [0,4194304] 0 2024-09-16T00:26:09.230 INFO:tasks.workunit.client.0.smithi112.stdout:5/751: symlink d4/d7/d14/d90/dd3/lfa 0 2024-09-16T00:26:09.239 INFO:tasks.workunit.client.0.smithi112.stdout:5/752: readlink d4/d7/d26/l34 0 2024-09-16T00:26:09.240 INFO:tasks.workunit.client.0.smithi112.stdout:6/775: dread dd/df/d11/f49 [0,4194304] 0 2024-09-16T00:26:09.240 INFO:tasks.workunit.client.0.smithi112.stdout:3/873: dwrite d2/d9/d2b/d47/d52/d7a/d80/d83/f50 [0,4194304] 0 2024-09-16T00:26:09.241 INFO:tasks.workunit.client.0.smithi112.stdout:2/915: mkdir d2/dd/d48/d8a/d101/d122/d127 0 2024-09-16T00:26:09.242 INFO:tasks.workunit.client.0.smithi112.stdout:8/741: dread d10/d12/d1c/ff5 [0,4194304] 0 2024-09-16T00:26:09.247 INFO:tasks.workunit.client.0.smithi112.stdout:0/940: rename d5/da/d57/d76/l60 to d5/da/d18/l126 0 2024-09-16T00:26:09.248 INFO:tasks.workunit.client.0.smithi112.stdout:3/874: dread - d2/d9/d2b/d47/d52/d7a/d80/d83/d7b/dad/d107/d69/d92/dda/f111 zero size 2024-09-16T00:26:09.248 INFO:tasks.workunit.client.0.smithi112.stdout:5/753: fdatasync d4/d7/fd7 0 2024-09-16T00:26:09.249 INFO:tasks.workunit.client.0.smithi112.stdout:8/742: mknod d10/d12/d1c/d96/cf6 0 2024-09-16T00:26:09.249 INFO:tasks.workunit.client.0.smithi112.stdout:2/916: write d2/dd/d48/d22/f8f [1587410,15564] 0 2024-09-16T00:26:09.250 INFO:tasks.workunit.client.0.smithi112.stdout:0/941: chown d5/da/d18/d24/d46/c117 765795 1 2024-09-16T00:26:09.253 INFO:tasks.workunit.client.0.smithi112.stdout:5/754: creat d4/d4f/d78/ffb x:0 0 0 2024-09-16T00:26:09.253 INFO:tasks.workunit.client.0.smithi112.stdout:4/827: dread d6/d39/d36/d5c/d7e/d8f/f98 [0,4194304] 0 2024-09-16T00:26:09.254 INFO:tasks.workunit.client.0.smithi112.stdout:8/743: unlink d10/d12/d1c/d61/f84 0 2024-09-16T00:26:09.255 INFO:tasks.workunit.client.0.smithi112.stdout:6/776: dread dd/df/d10/d47/f57 [4194304,4194304] 0 2024-09-16T00:26:09.255 INFO:tasks.workunit.client.0.smithi112.stdout:4/828: chown d6/db/df/le6 379350 1 2024-09-16T00:26:09.257 INFO:tasks.workunit.client.0.smithi112.stdout:2/917: creat d2/dd/d48/d22/d2b/d54/d60/daf/d11a/f128 x:0 0 0 2024-09-16T00:26:09.258 INFO:tasks.workunit.client.0.smithi112.stdout:5/755: write d4/d7/d26/d36/f74 [3814479,100174] 0 2024-09-16T00:26:09.261 INFO:tasks.workunit.client.0.smithi112.stdout:3/875: link d2/d9/d2b/c55 d2/d9/d2b/d47/d78/c122 0 2024-09-16T00:26:09.262 INFO:tasks.workunit.client.0.smithi112.stdout:0/942: getdents d5/d90/dc5/dd7 0 2024-09-16T00:26:09.265 INFO:tasks.workunit.client.0.smithi112.stdout:5/756: mkdir d4/d7/d26/d4c/db3/db6/dd0/dfc 0 2024-09-16T00:26:09.268 INFO:tasks.workunit.client.0.smithi112.stdout:0/943: mkdir d5/da/d1d/d71/d64/db6/d127 0 2024-09-16T00:26:09.268 INFO:tasks.workunit.client.0.smithi112.stdout:6/777: link dd/fca dd/df/d10/d47/d55/dbd/dcb/fff 0 2024-09-16T00:26:09.269 INFO:tasks.workunit.client.0.smithi112.stdout:3/876: dread - d2/d9/d2b/d47/d52/d7a/d80/d83/d7b/fcc zero size 2024-09-16T00:26:09.271 INFO:tasks.workunit.client.0.smithi112.stdout:0/944: chown d5/f68 4383 1 2024-09-16T00:26:09.272 INFO:tasks.workunit.client.0.smithi112.stdout:8/744: dwrite d10/d12/d1c/d61/d6c/dd1/f6f [0,4194304] 0 2024-09-16T00:26:09.272 INFO:tasks.workunit.client.0.smithi112.stdout:4/829: dwrite d6/db/df/d5e/d61/d64/d6a/fd0 [0,4194304] 0 2024-09-16T00:26:09.275 INFO:tasks.workunit.client.0.smithi112.stdout:4/830: fsync d6/db/f18 0 2024-09-16T00:26:09.275 INFO:tasks.workunit.client.1.smithi169.stdout:1/581: sync 2024-09-16T00:26:09.276 INFO:tasks.workunit.client.0.smithi112.stdout:0/945: dread - d5/da/d18/d24/d46/d5f/ffe zero size 2024-09-16T00:26:09.277 INFO:tasks.workunit.client.0.smithi112.stdout:2/918: dread d2/dd/d48/d22/d67/f74 [0,4194304] 0 2024-09-16T00:26:09.279 INFO:tasks.workunit.client.0.smithi112.stdout:5/757: link d4/d7/d14/d90/dd3/fdd d4/d4f/d78/ffd 0 2024-09-16T00:26:09.279 INFO:tasks.workunit.client.0.smithi112.stdout:7/845: sync 2024-09-16T00:26:09.280 INFO:tasks.workunit.client.1.smithi169.stdout:1/582: rename dc/d18/f2b to dc/d18/d67/d69/fae 0 2024-09-16T00:26:09.283 INFO:tasks.workunit.client.0.smithi112.stdout:0/946: stat d5/da/d21/f111 0 2024-09-16T00:26:09.283 INFO:tasks.workunit.client.0.smithi112.stdout:4/831: rmdir d6/d39/d1c/d6e/d6f 39 2024-09-16T00:26:09.286 INFO:tasks.workunit.client.0.smithi112.stdout:5/758: symlink d4/dd/d49/da9/lfe 0 2024-09-16T00:26:09.286 INFO:tasks.workunit.client.0.smithi112.stdout:2/919: rename d2/dd/d48/d22/d2b/d54/d60/c78 to d2/dd/d48/d22/d2b/d54/d60/daf/d11a/d123/c129 0 2024-09-16T00:26:09.288 INFO:tasks.workunit.client.0.smithi112.stdout:0/947: mknod d5/d90/d105/c128 0 2024-09-16T00:26:09.289 INFO:tasks.workunit.client.0.smithi112.stdout:2/920: read - d2/dd/d48/d22/d2b/d54/d60/d6c/dc9/ff9 zero size 2024-09-16T00:26:09.290 INFO:tasks.workunit.client.0.smithi112.stdout:7/846: mknod d6/d10/d16/c119 0 2024-09-16T00:26:09.293 INFO:tasks.workunit.client.0.smithi112.stdout:5/759: mknod d4/dd/d49/de6/cff 0 2024-09-16T00:26:09.296 INFO:tasks.workunit.client.0.smithi112.stdout:0/948: mknod d5/da/d18/d63/db2/c129 0 2024-09-16T00:26:09.297 INFO:tasks.workunit.client.0.smithi112.stdout:2/921: dread - d2/dd/d48/d22/d2b/d54/d60/d6c/d88/dad/deb/fda zero size 2024-09-16T00:26:09.297 INFO:tasks.workunit.client.0.smithi112.stdout:5/760: creat d4/dee/f100 x:0 0 0 2024-09-16T00:26:09.299 INFO:tasks.workunit.client.0.smithi112.stdout:3/877: dread d2/d9/d2b/d47/d52/fd3 [4194304,4194304] 0 2024-09-16T00:26:09.302 INFO:tasks.workunit.client.0.smithi112.stdout:5/761: write d4/d7/d26/d4c/db3/db6/fb8 [38476,3303] 0 2024-09-16T00:26:09.302 INFO:tasks.workunit.client.0.smithi112.stdout:0/949: creat d5/da/d18/d24/d46/d5f/d7a/f12a x:0 0 0 2024-09-16T00:26:09.303 INFO:tasks.workunit.client.0.smithi112.stdout:7/847: creat d6/d10/da1/f11a x:0 0 0 2024-09-16T00:26:09.305 INFO:tasks.workunit.client.0.smithi112.stdout:2/922: write d2/dd/d48/d22/d2b/d54/d79/f7a [247987,97278] 0 2024-09-16T00:26:09.306 INFO:tasks.workunit.client.0.smithi112.stdout:8/745: dread fe [0,4194304] 0 2024-09-16T00:26:09.309 INFO:tasks.workunit.client.1.smithi169.stdout:1/583: dread dc/d36/d3a/d45/d29/d4d/f54 [0,4194304] 0 2024-09-16T00:26:09.312 INFO:tasks.workunit.client.1.smithi169.stdout:1/584: rename dc/d2c/d3d to dc/d36/d39/d50/daf 0 2024-09-16T00:26:09.314 INFO:tasks.workunit.client.0.smithi112.stdout:5/762: creat d4/d7/d14/d90/dd3/f101 x:0 0 0 2024-09-16T00:26:09.316 INFO:tasks.workunit.client.0.smithi112.stdout:2/923: dread - d2/dd/d48/d22/d67/faa zero size 2024-09-16T00:26:09.320 INFO:tasks.workunit.client.0.smithi112.stdout:4/832: dread d6/d39/d1c/d2c/d69/d7b/fa8 [0,4194304] 0 2024-09-16T00:26:09.320 INFO:tasks.workunit.client.1.smithi169.stdout:1/585: write dc/d36/d3a/d7f/d93/f6b [719467,109633] 0 2024-09-16T00:26:09.322 INFO:tasks.workunit.client.0.smithi112.stdout:6/778: dwrite dd/df/d13/fc1 [0,4194304] 0 2024-09-16T00:26:09.322 INFO:tasks.workunit.client.0.smithi112.stdout:5/763: dread - d4/dd/d49/d70/fd6 zero size 2024-09-16T00:26:09.323 INFO:tasks.workunit.client.1.smithi169.stdout:1/586: rename dc/d18/f59 to dc/d36/d39/fb0 0 2024-09-16T00:26:09.325 INFO:tasks.workunit.client.0.smithi112.stdout:5/764: readlink d4/dd/d6f/l75 0 2024-09-16T00:26:09.326 INFO:tasks.workunit.client.0.smithi112.stdout:0/950: dwrite d5/da/d18/d63/f65 [0,4194304] 0 2024-09-16T00:26:09.326 INFO:tasks.workunit.client.1.smithi169.stdout:1/587: getdents dc/d36/d3a/d7f 0 2024-09-16T00:26:09.332 INFO:tasks.workunit.client.0.smithi112.stdout:3/878: dwrite d2/d9/d2b/d47/d52/d7a/d80/d83/d49/f77 [0,4194304] 0 2024-09-16T00:26:09.332 INFO:tasks.workunit.client.0.smithi112.stdout:2/924: write d2/fa [724863,33037] 0 2024-09-16T00:26:09.334 INFO:tasks.workunit.client.0.smithi112.stdout:4/833: write f4 [2676260,101437] 0 2024-09-16T00:26:09.334 INFO:tasks.workunit.client.0.smithi112.stdout:7/848: link d6/d10/d75/d8f/d90/ddc/dee/f115 d6/d10/d75/d88/dff/f11b 0 2024-09-16T00:26:09.335 INFO:tasks.workunit.client.0.smithi112.stdout:1/859: sync 2024-09-16T00:26:09.337 INFO:tasks.workunit.client.1.smithi169.stdout:1/588: dwrite dc/d36/d39/d50/daf/d86/f51 [0,4194304] 0 2024-09-16T00:26:09.340 INFO:tasks.workunit.client.1.smithi169.stdout:1/589: write dc/d36/d39/d50/daf/f3f [2790634,10846] 0 2024-09-16T00:26:09.345 INFO:tasks.workunit.client.0.smithi112.stdout:6/779: symlink dd/df/d10/d47/d55/l100 0 2024-09-16T00:26:09.345 INFO:tasks.workunit.client.0.smithi112.stdout:0/951: rename d5/da/d21/dc3/lcb to d5/da/d66/l12b 0 2024-09-16T00:26:09.345 INFO:tasks.workunit.client.0.smithi112.stdout:5/765: symlink d4/d7/d14/d90/dd3/de5/l102 0 2024-09-16T00:26:09.345 INFO:tasks.workunit.client.0.smithi112.stdout:3/879: chown d2/d9/d2b/d47/d52/d7a/d80/f120 0 1 2024-09-16T00:26:09.345 INFO:tasks.workunit.client.0.smithi112.stdout:8/746: dwrite d10/d12/d1c/d3b/d7d/fb1 [0,4194304] 0 2024-09-16T00:26:09.346 INFO:tasks.workunit.client.0.smithi112.stdout:2/925: mkdir d2/dd/d48/d8a/d101/d122/d127/d12a 0 2024-09-16T00:26:09.347 INFO:tasks.workunit.client.0.smithi112.stdout:0/952: write d5/da/d18/d24/d46/d5f/d7a/f12a [781070,82493] 0 2024-09-16T00:26:09.355 INFO:tasks.workunit.client.0.smithi112.stdout:7/849: write d6/f7b [422047,120033] 0 2024-09-16T00:26:09.355 INFO:tasks.workunit.client.0.smithi112.stdout:5/766: rename d4/d7/d14/fcd to d4/d7/d26/d36/f103 0 2024-09-16T00:26:09.358 INFO:tasks.workunit.client.0.smithi112.stdout:0/953: mknod d5/da/d18/d24/d46/d5f/d7a/d8d/c12c 0 2024-09-16T00:26:09.361 INFO:tasks.workunit.client.0.smithi112.stdout:3/880: read d2/d9/d2b/d47/d52/d7a/d80/d83/d7b/dad/d107/d69/f6f [105345,59118] 0 2024-09-16T00:26:09.362 INFO:tasks.workunit.client.1.smithi169.stdout:5/513: sync 2024-09-16T00:26:09.362 INFO:tasks.workunit.client.1.smithi169.stdout:4/574: sync 2024-09-16T00:26:09.363 INFO:tasks.workunit.client.0.smithi112.stdout:2/926: dread d2/dd/d48/d22/d2b/f34 [0,4194304] 0 2024-09-16T00:26:09.363 INFO:tasks.workunit.client.1.smithi169.stdout:4/575: chown d2/d12/l44 29612384 1 2024-09-16T00:26:09.363 INFO:tasks.workunit.client.0.smithi112.stdout:1/860: creat d0/d14/df/d28/d45/f117 x:0 0 0 2024-09-16T00:26:09.365 INFO:tasks.workunit.client.0.smithi112.stdout:3/881: fsync d2/d9/f4e 0 2024-09-16T00:26:09.365 INFO:tasks.workunit.client.0.smithi112.stdout:7/850: dread d6/d10/d16/d96/fe2 [0,4194304] 0 2024-09-16T00:26:09.369 INFO:tasks.workunit.client.1.smithi169.stdout:5/514: creat d2/da/d10/d13/d1c/d32/d53/fb0 x:0 0 0 2024-09-16T00:26:09.369 INFO:tasks.workunit.client.0.smithi112.stdout:4/834: getdents d6/d39/d1c/d6e/d6f/d11b/daa 0 2024-09-16T00:26:09.370 INFO:tasks.workunit.client.0.smithi112.stdout:2/927: chown d2/l62 419254 1 2024-09-16T00:26:09.371 INFO:tasks.workunit.client.1.smithi169.stdout:5/515: mkdir d2/da/d10/d13/d1c/db1 0 2024-09-16T00:26:09.372 INFO:tasks.workunit.client.0.smithi112.stdout:1/861: rename d0/d14/d1c/d20/c68 to d0/d14/d1c/d20/c118 0 2024-09-16T00:26:09.373 INFO:tasks.workunit.client.0.smithi112.stdout:4/835: truncate d6/db/df/d5e/f132 716430 0 2024-09-16T00:26:09.374 INFO:tasks.workunit.client.0.smithi112.stdout:7/851: mkdir d6/d10/d16/d96/dc7/dbf/d11c 0 2024-09-16T00:26:09.375 INFO:tasks.workunit.client.0.smithi112.stdout:1/862: chown d0/d14/df/fdf 202 1 2024-09-16T00:26:09.376 INFO:tasks.workunit.client.0.smithi112.stdout:8/747: dwrite d10/d12/d1c/d61/d6c/f7a [0,4194304] 0 2024-09-16T00:26:09.376 INFO:tasks.workunit.client.0.smithi112.stdout:3/882: rename d2/d9/d2b/d47/d52/d7a/d80/f93 to d2/d9/d2b/d47/d52/d7a/d80/d83/d7b/dad/d107/d69/f123 0 2024-09-16T00:26:09.378 INFO:tasks.workunit.client.0.smithi112.stdout:2/928: mkdir d2/dd/d48/d22/d2b/d54/d60/da9/d12b 0 2024-09-16T00:26:09.379 INFO:tasks.workunit.client.1.smithi169.stdout:4/576: write d2/d12/d13/d1f/d3e/d5b/d1a/f41 [1318208,47961] 0 2024-09-16T00:26:09.380 INFO:tasks.workunit.client.1.smithi169.stdout:5/516: write d2/da/d10/f3f [367379,90781] 0 2024-09-16T00:26:09.381 INFO:tasks.workunit.client.1.smithi169.stdout:1/590: dread dc/d36/d3a/d45/d29/f5c [0,4194304] 0 2024-09-16T00:26:09.383 INFO:tasks.workunit.client.1.smithi169.stdout:4/577: rmdir d2/d12/d13/db4 0 2024-09-16T00:26:09.384 INFO:tasks.workunit.client.1.smithi169.stdout:5/517: mkdir d2/da/d10/d13/d1c/d2d/db2 0 2024-09-16T00:26:09.387 INFO:tasks.workunit.client.0.smithi112.stdout:7/852: creat d6/d10/d16/d96/dc7/f11d x:0 0 0 2024-09-16T00:26:09.388 INFO:tasks.workunit.client.1.smithi169.stdout:5/518: mknod d2/da/d10/d13/d1c/d32/d53/d83/d7d/cb3 0 2024-09-16T00:26:09.388 INFO:tasks.workunit.client.1.smithi169.stdout:1/591: dwrite dc/d36/d3a/d45/d29/d2a/f44 [0,4194304] 0 2024-09-16T00:26:09.389 INFO:tasks.workunit.client.1.smithi169.stdout:5/519: stat d2/da/d10/d13/d1c 0 2024-09-16T00:26:09.389 INFO:tasks.workunit.client.0.smithi112.stdout:2/929: mknod d2/dd/d48/d22/d2b/d54/d60/daf/d11a/d123/c12c 0 2024-09-16T00:26:09.390 INFO:tasks.workunit.client.0.smithi112.stdout:3/883: write d2/d9/d2b/d47/d52/d7a/d80/d83/d76/fdc [581131,65206] 0 2024-09-16T00:26:09.390 INFO:tasks.workunit.client.0.smithi112.stdout:0/954: dwrite d5/da/d18/f54 [0,4194304] 0 2024-09-16T00:26:09.392 INFO:tasks.workunit.client.1.smithi169.stdout:5/520: mkdir d2/da/d10/d13/d1c/d94/db4 0 2024-09-16T00:26:09.392 INFO:tasks.workunit.client.1.smithi169.stdout:1/592: truncate dc/d36/d3a/d45/f1c 1778517 0 2024-09-16T00:26:09.396 INFO:tasks.workunit.client.1.smithi169.stdout:1/593: mknod dc/d36/d39/d50/daf/d88/cb1 0 2024-09-16T00:26:09.399 INFO:tasks.workunit.client.0.smithi112.stdout:2/930: creat d2/dd/d48/d8a/f12d x:0 0 0 2024-09-16T00:26:09.400 INFO:tasks.workunit.client.1.smithi169.stdout:5/521: creat d2/da/d10/d13/d1c/d40/fb5 x:0 0 0 2024-09-16T00:26:09.400 INFO:tasks.workunit.client.0.smithi112.stdout:8/748: dwrite d10/d12/d1c/d61/d6c/fd4 [0,4194304] 0 2024-09-16T00:26:09.404 INFO:tasks.workunit.client.0.smithi112.stdout:3/884: getdents d2/d9/d2b/d47/d52/d7a/d80/db4/d10d 0 2024-09-16T00:26:09.404 INFO:tasks.workunit.client.0.smithi112.stdout:1/863: dwrite d0/d14/df/d28/d45/dd4/d47/d10a/f109 [0,4194304] 0 2024-09-16T00:26:09.406 INFO:tasks.workunit.client.1.smithi169.stdout:5/522: creat d2/da/d10/d13/d1c/d94/db4/fb6 x:0 0 0 2024-09-16T00:26:09.406 INFO:tasks.workunit.client.0.smithi112.stdout:7/853: link d6/d10/d16/d96/dc7/dbf/d117/c104 d6/d10/d16/d96/dc7/d39/c11e 0 2024-09-16T00:26:09.408 INFO:tasks.workunit.client.1.smithi169.stdout:4/578: dwrite d2/d12/d13/d1f/d2f/f7b [0,4194304] 0 2024-09-16T00:26:09.408 INFO:tasks.workunit.client.0.smithi112.stdout:8/749: creat d10/d12/d1c/d61/ff7 x:0 0 0 2024-09-16T00:26:09.409 INFO:tasks.workunit.client.0.smithi112.stdout:2/931: creat d2/dd/d48/d22/d2b/d54/d79/d110/d49/d100/d105/f12e x:0 0 0 2024-09-16T00:26:09.410 INFO:tasks.workunit.client.0.smithi112.stdout:3/885: chown d2/d9/d2b/d47/d52/d7a/d80/d83/f112 0 1 2024-09-16T00:26:09.411 INFO:tasks.workunit.client.1.smithi169.stdout:1/594: dwrite dc/d36/d3a/d45/d29/f5c [0,4194304] 0 2024-09-16T00:26:09.411 INFO:tasks.workunit.client.1.smithi169.stdout:5/523: mknod d2/da/d10/d13/d1c/d32/d53/d4e/cb7 0 2024-09-16T00:26:09.413 INFO:tasks.workunit.client.0.smithi112.stdout:7/854: chown d6/d10/d16/d96/fcc 1946255 1 2024-09-16T00:26:09.413 INFO:tasks.workunit.client.1.smithi169.stdout:4/579: creat d2/d12/d13/d5c/da6/fb6 x:0 0 0 2024-09-16T00:26:09.414 INFO:tasks.workunit.client.0.smithi112.stdout:8/750: creat d10/de9/ff8 x:0 0 0 2024-09-16T00:26:09.414 INFO:tasks.workunit.client.1.smithi169.stdout:2/594: sync 2024-09-16T00:26:09.415 INFO:tasks.workunit.client.1.smithi169.stdout:0/646: sync 2024-09-16T00:26:09.415 INFO:tasks.workunit.client.1.smithi169.stdout:3/720: sync 2024-09-16T00:26:09.415 INFO:tasks.workunit.client.1.smithi169.stdout:6/844: sync 2024-09-16T00:26:09.415 INFO:tasks.workunit.client.1.smithi169.stdout:8/627: sync 2024-09-16T00:26:09.415 INFO:tasks.workunit.client.1.smithi169.stdout:9/625: sync 2024-09-16T00:26:09.415 INFO:tasks.workunit.client.1.smithi169.stdout:7/667: sync 2024-09-16T00:26:09.416 INFO:tasks.workunit.client.0.smithi112.stdout:0/955: dwrite d5/da/d1d/d71/d64/d106/dc0/fde [0,4194304] 0 2024-09-16T00:26:09.416 INFO:tasks.workunit.client.0.smithi112.stdout:1/864: truncate d0/d14/d1c/f51 4590479 0 2024-09-16T00:26:09.418 INFO:tasks.workunit.client.0.smithi112.stdout:2/932: chown d2/dd/d48/d22/d2b/d54/d79/f9e 1 1 2024-09-16T00:26:09.419 INFO:tasks.workunit.client.0.smithi112.stdout:4/836: dread d6/f1b [0,4194304] 0 2024-09-16T00:26:09.421 INFO:tasks.workunit.client.1.smithi169.stdout:8/628: chown d7/dc/d1d/d80 168 1 2024-09-16T00:26:09.422 INFO:tasks.workunit.client.1.smithi169.stdout:1/595: read dc/fa1 [3246671,55703] 0 2024-09-16T00:26:09.422 INFO:tasks.workunit.client.1.smithi169.stdout:5/524: creat d2/da/fb8 x:0 0 0 2024-09-16T00:26:09.426 INFO:tasks.workunit.client.0.smithi112.stdout:3/886: creat d2/d9/f124 x:0 0 0 2024-09-16T00:26:09.427 INFO:tasks.workunit.client.0.smithi112.stdout:0/956: mknod d5/da/d18/d24/d46/d5f/da6/c12d 0 2024-09-16T00:26:09.429 INFO:tasks.workunit.client.0.smithi112.stdout:2/933: mkdir d2/dd/d48/d8a/db9/de7/d12f 0 2024-09-16T00:26:09.430 INFO:tasks.workunit.client.0.smithi112.stdout:0/957: dread - d5/da/d18/fc4 zero size 2024-09-16T00:26:09.433 INFO:tasks.workunit.client.0.smithi112.stdout:2/934: fdatasync d2/dd/d48/d22/d2b/d54/d79/d110/d49/d100/d105/f12e 0 2024-09-16T00:26:09.436 INFO:tasks.workunit.client.1.smithi169.stdout:2/595: dread d7/db/d27/d29/d77/f86 [0,4194304] 0 2024-09-16T00:26:09.436 INFO:tasks.workunit.client.1.smithi169.stdout:9/626: dread - d5/da/d5b/d91/dc9/f81 zero size 2024-09-16T00:26:09.436 INFO:tasks.workunit.client.0.smithi112.stdout:8/751: dwrite d10/d12/fd7 [0,4194304] 0 2024-09-16T00:26:09.438 INFO:tasks.workunit.client.0.smithi112.stdout:3/887: creat d2/d9/d2b/d47/d52/d7a/d80/d83/d7b/dad/d107/f125 x:0 0 0 2024-09-16T00:26:09.438 INFO:tasks.workunit.client.0.smithi112.stdout:1/865: dread d0/f67 [0,4194304] 0 2024-09-16T00:26:09.439 INFO:tasks.workunit.client.0.smithi112.stdout:0/958: unlink d5/da/d21/da5/c118 0 2024-09-16T00:26:09.440 INFO:tasks.workunit.client.0.smithi112.stdout:3/888: dread - d2/d9/d2b/d47/fac zero size 2024-09-16T00:26:09.440 INFO:tasks.workunit.client.0.smithi112.stdout:2/935: creat d2/dd/d48/d22/d2b/d54/d60/da9/f130 x:0 0 0 2024-09-16T00:26:09.441 INFO:tasks.workunit.client.0.smithi112.stdout:8/752: mknod d10/d12/d1c/d61/d6c/ddc/cf9 0 2024-09-16T00:26:09.443 INFO:tasks.workunit.client.1.smithi169.stdout:7/668: creat d7/d5b/d61/d98/fd3 x:0 0 0 2024-09-16T00:26:09.443 INFO:tasks.workunit.client.1.smithi169.stdout:8/629: unlink d7/dc/d26/c2c 0 2024-09-16T00:26:09.443 INFO:tasks.workunit.client.1.smithi169.stdout:1/596: creat dc/d36/d39/d50/daf/d86/fb2 x:0 0 0 2024-09-16T00:26:09.443 INFO:tasks.workunit.client.1.smithi169.stdout:4/580: dwrite d2/d12/d13/d27/f32 [0,4194304] 0 2024-09-16T00:26:09.443 INFO:tasks.workunit.client.0.smithi112.stdout:2/936: fdatasync d2/f64 0 2024-09-16T00:26:09.444 INFO:tasks.workunit.client.1.smithi169.stdout:5/525: write d2/da/d10/d13/d1c/d2d/f63 [2472674,14460] 0 2024-09-16T00:26:09.445 INFO:tasks.workunit.client.1.smithi169.stdout:8/630: stat d7/dc/d1d/l45 0 2024-09-16T00:26:09.447 INFO:tasks.workunit.client.0.smithi112.stdout:3/889: creat d2/d9/d2b/d47/d78/da9/f126 x:0 0 0 2024-09-16T00:26:09.447 INFO:tasks.workunit.client.1.smithi169.stdout:2/596: rename d7/d6b/d84 to d7/db/d27/d29/d37/dd1 0 2024-09-16T00:26:09.448 INFO:tasks.workunit.client.0.smithi112.stdout:8/753: mknod d10/d12/d1c/cfa 0 2024-09-16T00:26:09.450 INFO:tasks.workunit.client.1.smithi169.stdout:1/597: write dc/d36/d39/d50/daf/f3f [5227723,29292] 0 2024-09-16T00:26:09.450 INFO:tasks.workunit.client.0.smithi112.stdout:1/866: dwrite d0/d14/fe1 [0,4194304] 0 2024-09-16T00:26:09.453 INFO:tasks.workunit.client.1.smithi169.stdout:7/669: mkdir d7/dd4 0 2024-09-16T00:26:09.455 INFO:tasks.workunit.client.0.smithi112.stdout:0/959: dwrite d5/f2c [0,4194304] 0 2024-09-16T00:26:09.456 INFO:tasks.workunit.client.1.smithi169.stdout:3/721: creat d7/d1a/d2b/f104 x:0 0 0 2024-09-16T00:26:09.459 INFO:tasks.workunit.client.0.smithi112.stdout:3/890: creat d2/d9/d2b/d47/d52/d7a/d80/d83/d7b/dad/f127 x:0 0 0 2024-09-16T00:26:09.461 INFO:tasks.workunit.client.0.smithi112.stdout:0/960: fdatasync d5/da/d1d/d71/d64/db6/fd2 0 2024-09-16T00:26:09.462 INFO:tasks.workunit.client.1.smithi169.stdout:2/597: rename d7/l25 to d7/d6b/d70/d79/dae/ld2 0 2024-09-16T00:26:09.465 INFO:tasks.workunit.client.0.smithi112.stdout:2/937: link d2/dd/d48/d22/d2b/d54/d60/da9/f130 d2/dd/d48/d22/d2b/d54/d60/d6c/d88/dad/deb/f131 0 2024-09-16T00:26:09.465 INFO:tasks.workunit.client.1.smithi169.stdout:3/722: fsync d7/d1a/d1c/fb6 0 2024-09-16T00:26:09.466 INFO:tasks.workunit.client.0.smithi112.stdout:0/961: symlink d5/da/d18/d24/l12e 0 2024-09-16T00:26:09.467 INFO:tasks.workunit.client.0.smithi112.stdout:2/938: chown d2/f8d 0 1 2024-09-16T00:26:09.467 INFO:tasks.workunit.client.0.smithi112.stdout:3/891: creat d2/d9/f128 x:0 0 0 2024-09-16T00:26:09.469 INFO:tasks.workunit.client.1.smithi169.stdout:4/581: dwrite d2/d12/d13/d1f/d3e/d5b/f89 [0,4194304] 0 2024-09-16T00:26:09.469 INFO:tasks.workunit.client.0.smithi112.stdout:0/962: unlink d5/da/d57/d76/f9f 0 2024-09-16T00:26:09.470 INFO:tasks.workunit.client.0.smithi112.stdout:2/939: read - d2/dd/d48/d22/d2b/d54/d60/d6c/d88/dad/deb/f131 zero size 2024-09-16T00:26:09.470 INFO:tasks.workunit.client.1.smithi169.stdout:4/582: dread - d2/d12/d9c/faa zero size 2024-09-16T00:26:09.473 INFO:tasks.workunit.client.0.smithi112.stdout:2/940: creat d2/f132 x:0 0 0 2024-09-16T00:26:09.474 INFO:tasks.workunit.client.1.smithi169.stdout:8/631: rename d7/dc/d26/d41/lc7 to d7/dc/d26/d41/d4e/d81/lca 0 2024-09-16T00:26:09.476 INFO:tasks.workunit.client.1.smithi169.stdout:0/647: dwrite d0/d2c/d38/f5e [0,4194304] 0 2024-09-16T00:26:09.477 INFO:tasks.workunit.client.0.smithi112.stdout:7/855: dread d6/d10/d16/d3b/f4b [0,4194304] 0 2024-09-16T00:26:09.477 INFO:tasks.workunit.client.0.smithi112.stdout:2/941: mkdir d2/dd/d48/d22/d2b/d54/d60/d6c/d133 0 2024-09-16T00:26:09.478 INFO:tasks.workunit.client.1.smithi169.stdout:5/526: link d2/da/d10/d13/d1c/d32/d53/d4e/d89/d9a/fa5 d2/da/d10/d13/d1c/d40/d68/fb9 0 2024-09-16T00:26:09.479 INFO:tasks.workunit.client.0.smithi112.stdout:7/856: chown d6/d10/d16/d3b 13788273 1 2024-09-16T00:26:09.480 INFO:tasks.workunit.client.1.smithi169.stdout:2/598: fdatasync d7/db/d27/d29/d37/f97 0 2024-09-16T00:26:09.481 INFO:tasks.workunit.client.1.smithi169.stdout:4/583: mknod d2/d12/d13/d1f/d3e/d5b/d1a/d9e/cb7 0 2024-09-16T00:26:09.484 INFO:tasks.workunit.client.0.smithi112.stdout:2/942: link d2/d68/f10b d2/dd/d48/d22/d2b/d54/d60/d6c/d88/dad/f134 0 2024-09-16T00:26:09.486 INFO:tasks.workunit.client.1.smithi169.stdout:5/527: chown d2/f4 127535 1 2024-09-16T00:26:09.487 INFO:tasks.workunit.client.1.smithi169.stdout:8/632: write d7/dc/d1d/d3c/d77/f7b [5047350,56545] 0 2024-09-16T00:26:09.487 INFO:tasks.workunit.client.1.smithi169.stdout:7/670: dwrite d7/d71/d76/d29/f45 [0,4194304] 0 2024-09-16T00:26:09.490 INFO:tasks.workunit.client.1.smithi169.stdout:6/845: truncate d2/d4d/d59/f35 1483604 0 2024-09-16T00:26:09.490 INFO:tasks.workunit.client.1.smithi169.stdout:4/584: chown d2/d12/da7/dac 32254489 1 2024-09-16T00:26:09.496 INFO:tasks.workunit.client.1.smithi169.stdout:5/528: fsync d2/da/d10/d13/d1c/d32/d53/d83/fa4 0 2024-09-16T00:26:09.497 INFO:tasks.workunit.client.1.smithi169.stdout:0/648: dread d0/d2c/d54/d21/d31/f85 [0,4194304] 0 2024-09-16T00:26:09.497 INFO:tasks.workunit.client.1.smithi169.stdout:8/633: chown d7/dc/d1d/ca2 6227073 1 2024-09-16T00:26:09.497 INFO:tasks.workunit.client.1.smithi169.stdout:3/723: dwrite d7/d1a/d1c/d23/d88/d9b/dc8/ff7 [0,4194304] 0 2024-09-16T00:26:09.497 INFO:tasks.workunit.client.0.smithi112.stdout:2/943: dwrite d2/dd/d48/d22/d2b/d54/d60/d6c/d88/dad/f134 [0,4194304] 0 2024-09-16T00:26:09.498 INFO:tasks.workunit.client.0.smithi112.stdout:3/892: write d2/d9/d2b/f29 [4725635,14495] 0 2024-09-16T00:26:09.501 INFO:tasks.workunit.client.0.smithi112.stdout:2/944: chown d2/dd/d48/d8a/d101 55578688 1 2024-09-16T00:26:09.503 INFO:tasks.workunit.client.1.smithi169.stdout:2/599: rename d7/db/d20/d5c/l98 to d7/db/d89/db4/d9d/ld3 0 2024-09-16T00:26:09.503 INFO:tasks.workunit.client.1.smithi169.stdout:9/627: dread d5/da/d21/f2c [0,4194304] 0 2024-09-16T00:26:09.507 INFO:tasks.workunit.client.0.smithi112.stdout:1/867: dwrite d0/f67 [0,4194304] 0 2024-09-16T00:26:09.507 INFO:tasks.workunit.client.1.smithi169.stdout:1/598: dwrite dc/d18/d67/d69/f80 [0,4194304] 0 2024-09-16T00:26:09.508 INFO:tasks.workunit.client.1.smithi169.stdout:9/628: write d5/da/d2e/f79 [3402723,26245] 0 2024-09-16T00:26:09.508 INFO:tasks.workunit.client.1.smithi169.stdout:8/634: write d7/d20/f7e [4913365,14338] 0 2024-09-16T00:26:09.509 INFO:tasks.workunit.client.1.smithi169.stdout:2/600: creat d7/db/d89/dc2/fd4 x:0 0 0 2024-09-16T00:26:09.515 INFO:tasks.workunit.client.0.smithi112.stdout:7/857: dwrite d6/d10/fcb [0,4194304] 0 2024-09-16T00:26:09.515 INFO:tasks.workunit.client.1.smithi169.stdout:9/629: chown d5/da/d5b/d6c/c97 0 1 2024-09-16T00:26:09.515 INFO:tasks.workunit.client.1.smithi169.stdout:2/601: chown d7/d1c/la5 1 1 2024-09-16T00:26:09.517 INFO:tasks.workunit.client.1.smithi169.stdout:1/599: rmdir dc/d36/d3a/d7f 39 2024-09-16T00:26:09.518 INFO:tasks.workunit.client.0.smithi112.stdout:2/945: write d2/dd/d48/d22/d2b/d54/d60/daf/d11a/f128 [347810,120472] 0 2024-09-16T00:26:09.518 INFO:tasks.workunit.client.0.smithi112.stdout:1/868: creat d0/d14/df/d28/d45/dd4/d41/de0/f119 x:0 0 0 2024-09-16T00:26:09.519 INFO:tasks.workunit.client.1.smithi169.stdout:9/630: chown d5/da/d21/d23/c5e 222 1 2024-09-16T00:26:09.521 INFO:tasks.workunit.client.1.smithi169.stdout:3/724: dwrite d7/d50/d6f/f100 [0,4194304] 0 2024-09-16T00:26:09.521 INFO:tasks.workunit.client.1.smithi169.stdout:2/602: chown d7/db/d89/db4/d69/d92/cb6 215614529 1 2024-09-16T00:26:09.521 INFO:tasks.workunit.client.0.smithi112.stdout:3/893: rename d2/d9/d2b/d47/d52/d7a/d80/d83/d76/c11e to d2/d9/d2b/c129 0 2024-09-16T00:26:09.522 INFO:tasks.workunit.client.1.smithi169.stdout:2/603: stat d7/db/d20/d5c/lb5 0 2024-09-16T00:26:09.522 INFO:tasks.workunit.client.1.smithi169.stdout:1/600: read - dc/d36/d39/d50/d75/fa8 zero size 2024-09-16T00:26:09.524 INFO:tasks.workunit.client.0.smithi112.stdout:0/963: dread d5/da/d21/f11b [0,4194304] 0 2024-09-16T00:26:09.524 INFO:tasks.workunit.client.0.smithi112.stdout:7/858: rmdir d6/d10/d16/d96/dc7/dbf/d117/d6c 39 2024-09-16T00:26:09.524 INFO:tasks.workunit.client.0.smithi112.stdout:1/869: symlink d0/d14/df/d28/d42/dd8/de8/l11a 0 2024-09-16T00:26:09.526 INFO:tasks.workunit.client.0.smithi112.stdout:2/946: mkdir d2/dd/d48/d8a/d101/d122/d135 0 2024-09-16T00:26:09.526 INFO:tasks.workunit.client.1.smithi169.stdout:3/725: readlink d7/d1a/d2b/l5d 0 2024-09-16T00:26:09.528 INFO:tasks.workunit.client.0.smithi112.stdout:0/964: write d5/da/d66/fb7 [792551,54283] 0 2024-09-16T00:26:09.532 INFO:tasks.workunit.client.1.smithi169.stdout:5/529: dread d2/da/d10/d13/f35 [4194304,4194304] 0 2024-09-16T00:26:09.533 INFO:tasks.workunit.client.0.smithi112.stdout:2/947: write d2/dd/d48/d22/d2b/d54/d79/d110/d49/d100/f102 [172236,84572] 0 2024-09-16T00:26:09.534 INFO:tasks.workunit.client.0.smithi112.stdout:0/965: mknod d5/d90/dc5/de2/c12f 0 2024-09-16T00:26:09.535 INFO:tasks.workunit.client.1.smithi169.stdout:3/726: fsync d7/d50/d6f/f74 0 2024-09-16T00:26:09.535 INFO:tasks.workunit.client.0.smithi112.stdout:3/894: dwrite d2/d9/d2b/d47/d52/d7a/d80/f120 [0,4194304] 0 2024-09-16T00:26:09.536 INFO:tasks.workunit.client.0.smithi112.stdout:3/895: fdatasync d2/d9/d2b/d47/d78/da9/ff7 0 2024-09-16T00:26:09.538 INFO:tasks.workunit.client.1.smithi169.stdout:2/604: mkdir d7/db/d89/db4/d9d/db9/dd5 0 2024-09-16T00:26:09.538 INFO:tasks.workunit.client.0.smithi112.stdout:0/966: creat d5/da/d18/d89/f130 x:0 0 0 2024-09-16T00:26:09.539 INFO:tasks.workunit.client.1.smithi169.stdout:5/530: readlink d2/da/d10/d13/d1c/d2d/l2e 0 2024-09-16T00:26:09.540 INFO:tasks.workunit.client.0.smithi112.stdout:0/967: fsync d5/da/d18/d24/d46/d5f/d7a/d8d/fd6 0 2024-09-16T00:26:09.541 INFO:tasks.workunit.client.1.smithi169.stdout:5/531: creat d2/d7c/fba x:0 0 0 2024-09-16T00:26:09.543 INFO:tasks.workunit.client.0.smithi112.stdout:2/948: link d2/dd/d48/d22/d2b/d54/d79/d110/fbe d2/dd/d48/d22/d2b/d54/d60/da9/f136 0 2024-09-16T00:26:09.544 INFO:tasks.workunit.client.1.smithi169.stdout:5/532: unlink d2/f8 0 2024-09-16T00:26:09.544 INFO:tasks.workunit.client.1.smithi169.stdout:2/605: truncate d7/d1c/d26/d96/f8e 891113 0 2024-09-16T00:26:09.545 INFO:tasks.workunit.client.1.smithi169.stdout:3/727: link d7/d1a/d1c/d23/d88/dbe/ce9 d7/d1a/d1c/d35/d63/c105 0 2024-09-16T00:26:09.549 INFO:tasks.workunit.client.0.smithi112.stdout:2/949: dwrite d2/dd/d48/d22/fb1 [0,4194304] 0 2024-09-16T00:26:09.551 INFO:tasks.workunit.client.1.smithi169.stdout:5/533: read d2/da/d10/d13/d1c/d40/d43/f4c [963719,9820] 0 2024-09-16T00:26:09.553 INFO:tasks.workunit.client.0.smithi112.stdout:2/950: creat d2/dd/d48/d22/d2b/d54/d60/daf/d11a/f137 x:0 0 0 2024-09-16T00:26:09.554 INFO:tasks.workunit.client.1.smithi169.stdout:2/606: creat d7/db/d27/d29/fd6 x:0 0 0 2024-09-16T00:26:09.556 INFO:tasks.workunit.client.0.smithi112.stdout:2/951: symlink d2/dd/d48/d22/d2b/d54/d60/d6c/dc9/dd5/l138 0 2024-09-16T00:26:09.558 INFO:tasks.workunit.client.1.smithi169.stdout:5/534: symlink d2/da/d10/d13/d1c/d32/d53/d4e/d89/lbb 0 2024-09-16T00:26:09.559 INFO:tasks.workunit.client.1.smithi169.stdout:7/671: dwrite d7/f7e [0,4194304] 0 2024-09-16T00:26:09.560 INFO:tasks.workunit.client.0.smithi112.stdout:2/952: mknod d2/dd/d48/d8a/db9/c139 0 2024-09-16T00:26:09.562 INFO:tasks.workunit.client.1.smithi169.stdout:0/649: dread d0/d4c/d6f/f91 [0,4194304] 0 2024-09-16T00:26:09.570 INFO:tasks.workunit.client.1.smithi169.stdout:8/635: write d7/dc/d17/d24/f27 [802788,37141] 0 2024-09-16T00:26:09.570 INFO:tasks.workunit.client.1.smithi169.stdout:4/585: dread d2/d12/d13/d1f/d2f/f43 [0,4194304] 0 2024-09-16T00:26:09.571 INFO:tasks.workunit.client.0.smithi112.stdout:0/968: dread d5/f7b [0,4194304] 0 2024-09-16T00:26:09.571 INFO:tasks.workunit.client.1.smithi169.stdout:7/672: rename d7/df/l5e to d7/ld5 0 2024-09-16T00:26:09.573 INFO:tasks.workunit.client.1.smithi169.stdout:8/636: symlink d7/dc/d17/d24/d32/d8d/lcb 0 2024-09-16T00:26:09.574 INFO:tasks.workunit.client.1.smithi169.stdout:4/586: fdatasync d2/d12/d13/d1f/d3e/f53 0 2024-09-16T00:26:09.576 INFO:tasks.workunit.client.1.smithi169.stdout:8/637: chown d7/dc/d26/d41/d4e/l54 1189963 1 2024-09-16T00:26:09.576 INFO:tasks.workunit.client.1.smithi169.stdout:9/631: dread d5/da/d73/f75 [4194304,4194304] 0 2024-09-16T00:26:09.577 INFO:tasks.workunit.client.1.smithi169.stdout:7/673: rename d7/f26 to d7/d5b/dc1/fd6 0 2024-09-16T00:26:09.578 INFO:tasks.workunit.client.1.smithi169.stdout:7/674: write d7/df/f44 [254290,43020] 0 2024-09-16T00:26:09.582 INFO:tasks.workunit.client.1.smithi169.stdout:9/632: truncate d5/f14 226057 0 2024-09-16T00:26:09.583 INFO:tasks.workunit.client.1.smithi169.stdout:3/728: dread d7/d1a/d2b/f8c [0,4194304] 0 2024-09-16T00:26:09.584 INFO:tasks.workunit.client.0.smithi112.stdout:1/870: dread d0/d14/d44/f7e [0,4194304] 0 2024-09-16T00:26:09.585 INFO:tasks.workunit.client.0.smithi112.stdout:7/859: write d6/d10/d16/d96/dc7/dbf/d117/d6c/f5b [419811,44539] 0 2024-09-16T00:26:09.586 INFO:tasks.workunit.client.1.smithi169.stdout:7/675: write d7/fa0 [1487347,90895] 0 2024-09-16T00:26:09.588 INFO:tasks.workunit.client.0.smithi112.stdout:7/860: write d6/d10/d16/d96/dc7/dbf/d117/d6c/f62 [2578244,117372] 0 2024-09-16T00:26:09.589 INFO:tasks.workunit.client.0.smithi112.stdout:1/871: creat d0/d14/df/d28/d45/f11b x:0 0 0 2024-09-16T00:26:09.590 INFO:tasks.workunit.client.1.smithi169.stdout:3/729: read - d7/d1a/fd1 zero size 2024-09-16T00:26:09.591 INFO:tasks.workunit.client.1.smithi169.stdout:9/633: truncate d5/da/d5b/d91/dc9/f6a 665017 0 2024-09-16T00:26:09.591 INFO:tasks.workunit.client.0.smithi112.stdout:3/896: truncate d2/d9/d2b/d47/d52/d7a/d80/d83/d49/f4c 3889370 0 2024-09-16T00:26:09.591 INFO:tasks.workunit.client.1.smithi169.stdout:8/638: link d7/dc/d17/d24/d32/d8d/la3 d7/dc/d17/d24/d32/d8d/dad/lcc 0 2024-09-16T00:26:09.593 INFO:tasks.workunit.client.0.smithi112.stdout:7/861: link d6/d10/d16/d96/dc7/f61 d6/d10/d16/db4/de4/f11f 0 2024-09-16T00:26:09.594 INFO:tasks.workunit.client.1.smithi169.stdout:7/676: rename d7/d16/c4f to d7/d5b/dc1/d96/cd7 0 2024-09-16T00:26:09.598 INFO:tasks.workunit.client.0.smithi112.stdout:1/872: dwrite d0/d14/df/d28/d45/dd4/fb9 [0,4194304] 0 2024-09-16T00:26:09.598 INFO:tasks.workunit.client.0.smithi112.stdout:0/969: dread d5/da/d21/da5/faa [0,4194304] 0 2024-09-16T00:26:09.599 INFO:tasks.workunit.client.1.smithi169.stdout:1/601: dwrite dc/d18/d67/d69/fae [0,4194304] 0 2024-09-16T00:26:09.599 INFO:tasks.workunit.client.1.smithi169.stdout:5/535: write d2/da/d10/d13/f1d [2186742,64154] 0 2024-09-16T00:26:09.602 INFO:tasks.workunit.client.1.smithi169.stdout:8/639: dread - d7/dc/d53/f93 zero size 2024-09-16T00:26:09.607 INFO:tasks.workunit.client.0.smithi112.stdout:2/953: dwrite d2/dd/d48/d8a/db9/de7/f11d [0,4194304] 0 2024-09-16T00:26:09.607 INFO:tasks.workunit.client.1.smithi169.stdout:4/587: write d2/d12/d13/d1f/f8e [1970939,48582] 0 2024-09-16T00:26:09.608 INFO:tasks.workunit.client.0.smithi112.stdout:0/970: chown d5/da/d18/d24/d46/d5f/da6/l115 4049 1 2024-09-16T00:26:09.612 INFO:tasks.workunit.client.0.smithi112.stdout:1/873: write d0/d14/df/d28/f2e [3361665,86118] 0 2024-09-16T00:26:09.612 INFO:tasks.workunit.client.1.smithi169.stdout:0/650: dread d0/d2c/f5d [0,4194304] 0 2024-09-16T00:26:09.613 INFO:tasks.workunit.client.0.smithi112.stdout:3/897: dwrite d2/d9/f94 [0,4194304] 0 2024-09-16T00:26:09.615 INFO:tasks.workunit.client.1.smithi169.stdout:3/730: dwrite d7/d1a/d1c/d23/d70/fb7 [0,4194304] 0 2024-09-16T00:26:09.616 INFO:tasks.workunit.client.0.smithi112.stdout:3/898: dread - d2/d9/d2b/d47/fac zero size 2024-09-16T00:26:09.617 INFO:tasks.workunit.client.0.smithi112.stdout:1/874: creat d0/d14/d6a/f11c x:0 0 0 2024-09-16T00:26:09.620 INFO:tasks.workunit.client.0.smithi112.stdout:1/875: creat d0/d14/d5e/f11d x:0 0 0 2024-09-16T00:26:09.621 INFO:tasks.workunit.client.0.smithi112.stdout:0/971: dwrite d5/da/d1d/d71/fa9 [0,4194304] 0 2024-09-16T00:26:09.622 INFO:tasks.workunit.client.1.smithi169.stdout:9/634: creat d5/da/d2e/d47/fdf x:0 0 0 2024-09-16T00:26:09.625 INFO:tasks.workunit.client.1.smithi169.stdout:9/635: dread - d5/da/d5b/d91/dc9/fd9 zero size 2024-09-16T00:26:09.626 INFO:tasks.workunit.client.0.smithi112.stdout:1/876: truncate d0/d14/df/d28/d42/f81 663809 0 2024-09-16T00:26:09.626 INFO:tasks.workunit.client.1.smithi169.stdout:7/677: unlink d7/df/f60 0 2024-09-16T00:26:09.628 INFO:tasks.workunit.client.0.smithi112.stdout:1/877: chown d0/d14/df/d28/d45/f98 7568 1 2024-09-16T00:26:09.632 INFO:tasks.workunit.client.0.smithi112.stdout:3/899: dwrite d2/d9/d2b/d47/d52/d7a/d80/d83/d76/fdc [0,4194304] 0 2024-09-16T00:26:09.633 INFO:tasks.workunit.client.0.smithi112.stdout:1/878: mknod d0/d14/d6a/d9e/c11e 0 2024-09-16T00:26:09.634 INFO:tasks.workunit.client.1.smithi169.stdout:8/640: rename d7/dc/d53/f93 to d7/dc/d26/d41/d4e/d81/d9e/da5/fcd 0 2024-09-16T00:26:09.634 INFO:tasks.workunit.client.1.smithi169.stdout:9/636: dwrite d5/d12/f86 [0,4194304] 0 2024-09-16T00:26:09.635 INFO:tasks.workunit.client.0.smithi112.stdout:1/879: stat d0/d14/d1c/fcd 0 2024-09-16T00:26:09.635 INFO:tasks.workunit.client.0.smithi112.stdout:0/972: dread d5/f33 [0,4194304] 0 2024-09-16T00:26:09.636 INFO:tasks.workunit.client.1.smithi169.stdout:4/588: mkdir d2/d12/d13/d4e/d50/db8 0 2024-09-16T00:26:09.637 INFO:tasks.workunit.client.1.smithi169.stdout:3/731: symlink d7/d1a/d1c/d23/d88/d9b/l106 0 2024-09-16T00:26:09.638 INFO:tasks.workunit.client.1.smithi169.stdout:4/589: truncate d2/d12/d13/d4d/f84 1024005 0 2024-09-16T00:26:09.641 INFO:tasks.workunit.client.0.smithi112.stdout:0/973: mkdir d5/d131 0 2024-09-16T00:26:09.642 INFO:tasks.workunit.client.0.smithi112.stdout:3/900: write d2/d9/d2b/d39/fb1 [71594,118307] 0 2024-09-16T00:26:09.643 INFO:tasks.workunit.client.0.smithi112.stdout:1/880: read d0/d14/df/d28/d45/d46/d58/d62/f74 [617230,119168] 0 2024-09-16T00:26:09.645 INFO:tasks.workunit.client.0.smithi112.stdout:0/974: creat d5/da/d1d/f132 x:0 0 0 2024-09-16T00:26:09.647 INFO:tasks.workunit.client.1.smithi169.stdout:3/732: dwrite d7/d1a/d1c/d23/d88/ffa [0,4194304] 0 2024-09-16T00:26:09.647 INFO:tasks.workunit.client.1.smithi169.stdout:9/637: rename d5/da/d5b/d6c/da0/db6 to d5/de0 0 2024-09-16T00:26:09.649 INFO:tasks.workunit.client.1.smithi169.stdout:0/651: getdents d0/d2c/d4f/da5 0 2024-09-16T00:26:09.651 INFO:tasks.workunit.client.1.smithi169.stdout:4/590: creat d2/d12/d13/fb9 x:0 0 0 2024-09-16T00:26:09.653 INFO:tasks.workunit.client.1.smithi169.stdout:0/652: write d0/d2/f6 [637972,55585] 0 2024-09-16T00:26:09.653 INFO:tasks.workunit.client.0.smithi112.stdout:3/901: read d2/d9/d2b/d47/d52/d7a/d80/d83/d68/d90/fe7 [3293604,41869] 0 2024-09-16T00:26:09.653 INFO:tasks.workunit.client.1.smithi169.stdout:9/638: mkdir d5/de1 0 2024-09-16T00:26:09.654 INFO:tasks.workunit.client.1.smithi169.stdout:9/639: stat d5/de0 0 2024-09-16T00:26:09.654 INFO:tasks.workunit.client.1.smithi169.stdout:0/653: stat d0/d2c/d54/cdd 0 2024-09-16T00:26:09.656 INFO:tasks.workunit.client.0.smithi112.stdout:3/902: chown d2/d9/d2b/d47/d52/d7a/d80/d83/d7b 75542736 1 2024-09-16T00:26:09.656 INFO:tasks.workunit.client.1.smithi169.stdout:9/640: readlink d5/da/d5b/d91/la5 0 2024-09-16T00:26:09.657 INFO:tasks.workunit.client.1.smithi169.stdout:9/641: write d5/da/f2b [2163389,10513] 0 2024-09-16T00:26:09.658 INFO:tasks.workunit.client.0.smithi112.stdout:3/903: write d2/d9/d2b/d47/d52/d7a/d80/d83/d68/d90/fe9 [767119,103156] 0 2024-09-16T00:26:09.659 INFO:tasks.workunit.client.1.smithi169.stdout:9/642: write d5/da/ff [4689551,93864] 0 2024-09-16T00:26:09.659 INFO:tasks.workunit.client.1.smithi169.stdout:8/641: dread d7/dc/d17/d24/f27 [0,4194304] 0 2024-09-16T00:26:09.661 INFO:tasks.workunit.client.1.smithi169.stdout:0/654: read d0/d2c/d54/f28 [2472629,16968] 0 2024-09-16T00:26:09.661 INFO:tasks.workunit.client.0.smithi112.stdout:2/954: dread d2/dd/d48/d22/d2b/d54/d60/d6c/d88/dad/deb/fab [0,4194304] 0 2024-09-16T00:26:09.661 INFO:tasks.workunit.client.1.smithi169.stdout:9/643: chown d5/da/d5b/d91/dc9/f53 9910718 1 2024-09-16T00:26:09.663 INFO:tasks.workunit.client.1.smithi169.stdout:4/591: link d2/d12/d13/f25 d2/d12/fba 0 2024-09-16T00:26:09.664 INFO:tasks.workunit.client.0.smithi112.stdout:3/904: unlink d2/d9/d2b/d47/d78/c10e 0 2024-09-16T00:26:09.664 INFO:tasks.workunit.client.1.smithi169.stdout:3/733: dread d7/d50/d90/dba/fbf [0,4194304] 0 2024-09-16T00:26:09.664 INFO:tasks.workunit.client.1.smithi169.stdout:0/655: symlink d0/d2/d4b/le1 0 2024-09-16T00:26:09.666 INFO:tasks.workunit.client.0.smithi112.stdout:1/881: truncate d0/d14/d1c/f91 3038048 0 2024-09-16T00:26:09.669 INFO:tasks.workunit.client.0.smithi112.stdout:3/905: dwrite d2/f1a [4194304,4194304] 0 2024-09-16T00:26:09.671 INFO:tasks.workunit.client.1.smithi169.stdout:9/644: mknod d5/da/d2e/ce2 0 2024-09-16T00:26:09.671 INFO:tasks.workunit.client.1.smithi169.stdout:8/642: dwrite d7/dc/d17/d24/f88 [0,4194304] 0 2024-09-16T00:26:09.671 INFO:tasks.workunit.client.1.smithi169.stdout:8/643: readlink d7/l2f 0 2024-09-16T00:26:09.671 INFO:tasks.workunit.client.0.smithi112.stdout:2/955: write d2/dd/d48/d22/d2b/d54/d60/da9/f136 [1196425,19812] 0 2024-09-16T00:26:09.672 INFO:tasks.workunit.client.1.smithi169.stdout:9/645: chown d5/da/d5b/d91/d9a/fba 104420 1 2024-09-16T00:26:09.676 INFO:tasks.workunit.client.0.smithi112.stdout:2/956: creat d2/dd/d48/d22/d2b/d54/d60/d6c/d133/f13a x:0 0 0 2024-09-16T00:26:09.676 INFO:tasks.workunit.client.1.smithi169.stdout:4/592: rename d2/d12/d13/d4d to d2/dbb 0 2024-09-16T00:26:09.678 INFO:tasks.workunit.client.0.smithi112.stdout:0/975: dread d5/f23 [0,4194304] 0 2024-09-16T00:26:09.678 INFO:tasks.workunit.client.0.smithi112.stdout:3/906: dread d2/d9/d2b/d39/fbb [0,4194304] 0 2024-09-16T00:26:09.681 INFO:tasks.workunit.client.0.smithi112.stdout:0/976: fsync d5/f7b 0 2024-09-16T00:26:09.681 INFO:tasks.workunit.client.1.smithi169.stdout:9/646: symlink d5/d49/dc0/dd6/le3 0 2024-09-16T00:26:09.683 INFO:tasks.workunit.client.1.smithi169.stdout:3/734: rename d7/l2d to d7/d1a/d1c/d23/d88/d9b/dc8/df8/l107 0 2024-09-16T00:26:09.684 INFO:tasks.workunit.client.1.smithi169.stdout:4/593: mknod d2/cbc 0 2024-09-16T00:26:09.686 INFO:tasks.workunit.client.0.smithi112.stdout:2/957: dwrite d2/dd/f1a [0,4194304] 0 2024-09-16T00:26:09.687 INFO:tasks.workunit.client.1.smithi169.stdout:4/594: readlink d2/d12/d13/d1f/d3e/d5b/d1a/d9e/l42 0 2024-09-16T00:26:09.687 INFO:tasks.workunit.client.0.smithi112.stdout:2/958: chown d2/d68/fff 2680808 1 2024-09-16T00:26:09.691 INFO:tasks.workunit.client.1.smithi169.stdout:9/647: rename d5/d12 to d5/da/d21/de4 0 2024-09-16T00:26:09.692 INFO:tasks.workunit.client.0.smithi112.stdout:2/959: dread d2/dd/d48/d22/d2b/d54/d60/d6c/d88/dad/deb/fcd [0,4194304] 0 2024-09-16T00:26:09.693 INFO:tasks.workunit.client.1.smithi169.stdout:9/648: chown d5/da/d2e/d47/fdf 248 1 2024-09-16T00:26:09.693 INFO:tasks.workunit.client.1.smithi169.stdout:3/735: truncate d7/d1a/d1c/d35/f93 663254 0 2024-09-16T00:26:09.693 INFO:tasks.workunit.client.0.smithi112.stdout:2/960: dread - d2/dd/fc4 zero size 2024-09-16T00:26:09.695 INFO:tasks.workunit.client.0.smithi112.stdout:0/977: dread d5/da/d21/fb5 [0,4194304] 0 2024-09-16T00:26:09.696 INFO:tasks.workunit.client.1.smithi169.stdout:4/595: rename d2/d12/d13/c9b to d2/d12/d9c/da4/cbd 0 2024-09-16T00:26:09.699 INFO:tasks.workunit.client.0.smithi112.stdout:0/978: unlink d5/da/d1d/c72 0 2024-09-16T00:26:09.701 INFO:tasks.workunit.client.1.smithi169.stdout:1/602: dwrite dc/d36/d3a/d45/d29/f52 [0,4194304] 0 2024-09-16T00:26:09.701 INFO:tasks.workunit.client.1.smithi169.stdout:9/649: write d5/da/d5b/d91/dc9/f89 [1774490,29256] 0 2024-09-16T00:26:09.701 INFO:tasks.workunit.client.1.smithi169.stdout:5/536: truncate d2/da/d10/d13/d1c/d32/d53/f4d 167674 0 2024-09-16T00:26:09.701 INFO:tasks.workunit.client.0.smithi112.stdout:2/961: dwrite d2/dd/d48/fef [0,4194304] 0 2024-09-16T00:26:09.702 INFO:tasks.workunit.client.1.smithi169.stdout:8/644: dread d7/dc/d17/f89 [4194304,4194304] 0 2024-09-16T00:26:09.703 INFO:tasks.workunit.client.0.smithi112.stdout:0/979: creat d5/da/d66/d110/f133 x:0 0 0 2024-09-16T00:26:09.704 INFO:tasks.workunit.client.1.smithi169.stdout:7/678: write d7/d16/d49/d74/f7c [2058452,222] 0 2024-09-16T00:26:09.704 INFO:tasks.workunit.client.0.smithi112.stdout:1/882: dread d0/d14/d1c/d20/daf/f59 [0,4194304] 0 2024-09-16T00:26:09.707 INFO:tasks.workunit.client.1.smithi169.stdout:3/736: creat d7/d1a/d1c/d35/d63/da3/de4/f108 x:0 0 0 2024-09-16T00:26:09.708 INFO:tasks.workunit.client.1.smithi169.stdout:9/650: chown d5/da/d2e/d47/cc2 3967741 1 2024-09-16T00:26:09.711 INFO:tasks.workunit.client.1.smithi169.stdout:1/603: creat dc/d26/fb3 x:0 0 0 2024-09-16T00:26:09.711 INFO:tasks.workunit.client.0.smithi112.stdout:1/883: creat d0/d14/df/d28/d42/dd8/de8/f11f x:0 0 0 2024-09-16T00:26:09.712 INFO:tasks.workunit.client.1.smithi169.stdout:3/737: readlink d7/d1a/d1c/d35/d63/d37/l4b 0 2024-09-16T00:26:09.713 INFO:tasks.workunit.client.1.smithi169.stdout:4/596: creat d2/d12/fbe x:0 0 0 2024-09-16T00:26:09.713 INFO:tasks.workunit.client.0.smithi112.stdout:2/962: mknod d2/dd/d48/d22/d2b/d54/d60/d6c/d88/c13b 0 2024-09-16T00:26:09.716 INFO:tasks.workunit.client.0.smithi112.stdout:1/884: mkdir d0/d14/d5e/d120 0 2024-09-16T00:26:09.718 INFO:tasks.workunit.client.1.smithi169.stdout:7/679: rename d7/d16/d49/d74/fb3 to d7/d71/d76/d29/d78/dba/fd8 0 2024-09-16T00:26:09.718 INFO:tasks.workunit.client.0.smithi112.stdout:3/907: dwrite d2/d9/f4e [0,4194304] 0 2024-09-16T00:26:09.718 INFO:tasks.workunit.client.0.smithi112.stdout:2/963: symlink d2/dd/d48/d22/d2b/d54/d60/daf/d11a/d123/l13c 0 2024-09-16T00:26:09.721 INFO:tasks.workunit.client.1.smithi169.stdout:4/597: truncate d2/d12/d13/d1f/d2f/f43 3871441 0 2024-09-16T00:26:09.722 INFO:tasks.workunit.client.0.smithi112.stdout:0/980: write f0 [1610510,23205] 0 2024-09-16T00:26:09.725 INFO:tasks.workunit.client.0.smithi112.stdout:1/885: mkdir d0/d14/df/d121 0 2024-09-16T00:26:09.726 INFO:tasks.workunit.client.1.smithi169.stdout:0/656: dwrite d0/d2/d8/f48 [0,4194304] 0 2024-09-16T00:26:09.729 INFO:tasks.workunit.client.0.smithi112.stdout:3/908: dread d2/d9/fa5 [0,4194304] 0 2024-09-16T00:26:09.731 INFO:tasks.workunit.client.0.smithi112.stdout:3/909: fsync d2/d9/d2b/d47/d78/da9/ff7 0 2024-09-16T00:26:09.732 INFO:tasks.workunit.client.1.smithi169.stdout:9/651: link d5/da/d5b/d6f/c84 d5/da/d2e/d47/ce5 0 2024-09-16T00:26:09.732 INFO:tasks.workunit.client.0.smithi112.stdout:1/886: symlink d0/d14/d6a/l122 0 2024-09-16T00:26:09.733 INFO:tasks.workunit.client.0.smithi112.stdout:2/964: dwrite d2/dd/d48/d22/d2b/d54/d60/d6c/d88/da4/f117 [0,4194304] 0 2024-09-16T00:26:09.739 INFO:tasks.workunit.client.1.smithi169.stdout:1/604: creat dc/d36/d3a/d45/d29/fb4 x:0 0 0 2024-09-16T00:26:09.739 INFO:tasks.workunit.client.0.smithi112.stdout:1/887: mkdir d0/d14/df/d28/d45/d46/da7/dda/d123 0 2024-09-16T00:26:09.739 INFO:tasks.workunit.client.1.smithi169.stdout:5/537: write d2/da/d10/d13/d1c/d40/d68/fb9 [263374,106398] 0 2024-09-16T00:26:09.740 INFO:tasks.workunit.client.1.smithi169.stdout:9/652: read d5/da/f7f [678807,99006] 0 2024-09-16T00:26:09.740 INFO:tasks.workunit.client.1.smithi169.stdout:0/657: mknod d0/d2c/d54/d56/d67/dd1/d6c/d9b/ce2 0 2024-09-16T00:26:09.741 INFO:tasks.workunit.client.0.smithi112.stdout:1/888: write d0/f67 [3045876,74448] 0 2024-09-16T00:26:09.742 INFO:tasks.workunit.client.1.smithi169.stdout:5/538: mknod d2/da/d67/cbc 0 2024-09-16T00:26:09.742 INFO:tasks.workunit.client.1.smithi169.stdout:1/605: write dc/d36/d3a/d7f/d93/fa3 [4767665,66310] 0 2024-09-16T00:26:09.743 INFO:tasks.workunit.client.0.smithi112.stdout:0/981: truncate d5/da/d18/d24/d46/d5f/d7a/f12a 629046 0 2024-09-16T00:26:09.744 INFO:tasks.workunit.client.1.smithi169.stdout:9/653: creat d5/d49/dc0/fe6 x:0 0 0 2024-09-16T00:26:09.745 INFO:tasks.workunit.client.1.smithi169.stdout:1/606: fsync dc/f34 0 2024-09-16T00:26:09.747 INFO:tasks.workunit.client.1.smithi169.stdout:9/654: write d5/d49/d64/fcd [675796,125987] 0 2024-09-16T00:26:09.747 INFO:tasks.workunit.client.0.smithi112.stdout:1/889: creat d0/d14/d44/f124 x:0 0 0 2024-09-16T00:26:09.747 INFO:tasks.workunit.client.1.smithi169.stdout:5/539: read d2/da/d10/d13/d1c/d94/f99 [666512,106132] 0 2024-09-16T00:26:09.747 INFO:tasks.workunit.client.1.smithi169.stdout:1/607: readlink dc/d36/d3a/d7f/l91 0 2024-09-16T00:26:09.748 INFO:tasks.workunit.client.0.smithi112.stdout:3/910: dwrite d2/d9/f16 [4194304,4194304] 0 2024-09-16T00:26:09.749 INFO:tasks.workunit.client.0.smithi112.stdout:0/982: chown d5/da/d1d/d71/d64/db6 1065 1 2024-09-16T00:26:09.750 INFO:tasks.workunit.client.1.smithi169.stdout:9/655: unlink d5/da/d5b/d91/dc9/l4f 0 2024-09-16T00:26:09.752 INFO:tasks.workunit.client.1.smithi169.stdout:1/608: symlink dc/d36/d39/d50/daf/d86/lb5 0 2024-09-16T00:26:09.753 INFO:tasks.workunit.client.1.smithi169.stdout:9/656: symlink d5/da/d5b/d6c/dcb/le7 0 2024-09-16T00:26:09.753 INFO:tasks.workunit.client.0.smithi112.stdout:2/965: dwrite d2/dd/d48/d22/d67/fb2 [0,4194304] 0 2024-09-16T00:26:09.753 INFO:tasks.workunit.client.0.smithi112.stdout:1/890: mkdir d0/d125 0 2024-09-16T00:26:09.754 INFO:tasks.workunit.client.1.smithi169.stdout:1/609: mkdir dc/d36/d3a/d45/d29/d4d/db6 0 2024-09-16T00:26:09.756 INFO:tasks.workunit.client.0.smithi112.stdout:2/966: chown d2/dd/c35 154039196 1 2024-09-16T00:26:09.758 INFO:tasks.workunit.client.1.smithi169.stdout:3/738: write d7/d50/f7b [996797,42679] 0 2024-09-16T00:26:09.758 INFO:tasks.workunit.client.1.smithi169.stdout:1/610: stat dc/le 0 2024-09-16T00:26:09.759 INFO:tasks.workunit.client.0.smithi112.stdout:0/983: rename d5/da/d18/d24/c4b to d5/d90/c134 0 2024-09-16T00:26:09.760 INFO:tasks.workunit.client.1.smithi169.stdout:4/598: dread d2/d12/d13/d1f/d2f/f82 [0,4194304] 0 2024-09-16T00:26:09.761 INFO:tasks.workunit.client.0.smithi112.stdout:3/911: creat d2/d9/d2b/d47/d52/d7a/d80/d83/d68/d90/f12a x:0 0 0 2024-09-16T00:26:09.761 INFO:tasks.workunit.client.0.smithi112.stdout:1/891: dread d0/d14/d6a/fbd [0,4194304] 0 2024-09-16T00:26:09.762 INFO:tasks.workunit.client.1.smithi169.stdout:7/680: truncate d7/d16/f79 118767 0 2024-09-16T00:26:09.762 INFO:tasks.workunit.client.1.smithi169.stdout:7/681: chown d7/d16/d49/f81 4399253 1 2024-09-16T00:26:09.763 INFO:tasks.workunit.client.0.smithi112.stdout:3/912: write d2/d9/d2b/d47/d52/d7a/d80/fb7 [4211727,35094] 0 2024-09-16T00:26:09.763 INFO:tasks.workunit.client.0.smithi112.stdout:2/967: creat d2/dd/d48/d8a/db9/de7/d12f/f13d x:0 0 0 2024-09-16T00:26:09.764 INFO:tasks.workunit.client.1.smithi169.stdout:3/739: unlink d7/d1a/l77 0 2024-09-16T00:26:09.767 INFO:tasks.workunit.client.0.smithi112.stdout:6/780: sync 2024-09-16T00:26:09.767 INFO:tasks.workunit.client.0.smithi112.stdout:3/913: symlink d2/d9/d2b/d47/d52/d7a/d80/d83/d68/l12b 0 2024-09-16T00:26:09.767 INFO:tasks.workunit.client.1.smithi169.stdout:1/611: symlink dc/d36/d3a/d45/d29/d4d/db6/lb7 0 2024-09-16T00:26:09.768 INFO:tasks.workunit.client.1.smithi169.stdout:3/740: rmdir d7/d1a/d1c/d23/d88/d9b 39 2024-09-16T00:26:09.768 INFO:tasks.workunit.client.1.smithi169.stdout:1/612: stat dc/fa1 0 2024-09-16T00:26:09.769 INFO:tasks.workunit.client.0.smithi112.stdout:2/968: read d2/f14 [4207544,406] 0 2024-09-16T00:26:09.769 INFO:tasks.workunit.client.1.smithi169.stdout:7/682: dread d7/f7e [0,4194304] 0 2024-09-16T00:26:09.769 INFO:tasks.workunit.client.0.smithi112.stdout:6/781: creat dd/df/d10/d47/d55/dbd/f101 x:0 0 0 2024-09-16T00:26:09.770 INFO:tasks.workunit.client.1.smithi169.stdout:7/683: rename d7 to d7/d5b/d61/dd9 22 2024-09-16T00:26:09.771 INFO:tasks.workunit.client.0.smithi112.stdout:3/914: creat d2/d9/d2b/d47/d52/d7a/d80/d83/d7b/dad/d107/d108/f12c x:0 0 0 2024-09-16T00:26:09.771 INFO:tasks.workunit.client.1.smithi169.stdout:1/613: unlink dc/d26/l40 0 2024-09-16T00:26:09.772 INFO:tasks.workunit.client.0.smithi112.stdout:6/782: stat dd/df/d17/d27/cc9 0 2024-09-16T00:26:09.773 INFO:tasks.workunit.client.1.smithi169.stdout:7/684: readlink d7/d71/d76/l5a 0 2024-09-16T00:26:09.773 INFO:tasks.workunit.client.0.smithi112.stdout:6/783: chown dd/d23/l30 563213 1 2024-09-16T00:26:09.775 INFO:tasks.workunit.client.1.smithi169.stdout:9/657: dread d5/da/f16 [0,4194304] 0 2024-09-16T00:26:09.775 INFO:tasks.workunit.client.0.smithi112.stdout:2/969: getdents d2/dd/d48/d22/d2b/d54/d79/d10e 0 2024-09-16T00:26:09.775 INFO:tasks.workunit.client.1.smithi169.stdout:7/685: chown d7/d16/d49/d84/dc8/cca 1 1 2024-09-16T00:26:09.776 INFO:tasks.workunit.client.1.smithi169.stdout:3/741: dwrite d7/d1a/d1c/d35/d63/f9c [0,4194304] 0 2024-09-16T00:26:09.778 INFO:tasks.workunit.client.0.smithi112.stdout:3/915: rename d2/d4d/ce5 to d2/d9/d2b/d47/d52/d7a/c12d 0 2024-09-16T00:26:09.780 INFO:tasks.workunit.client.0.smithi112.stdout:0/984: write d5/da/d1d/d71/d64/db6/fd2 [993703,106168] 0 2024-09-16T00:26:09.781 INFO:tasks.workunit.client.0.smithi112.stdout:2/970: creat d2/dd/d48/d22/d2b/d54/d60/d6c/d133/f13e x:0 0 0 2024-09-16T00:26:09.782 INFO:tasks.workunit.client.0.smithi112.stdout:2/971: chown d2/c21 272 1 2024-09-16T00:26:09.783 INFO:tasks.workunit.client.0.smithi112.stdout:2/972: read - d2/dd/fc4 zero size 2024-09-16T00:26:09.784 INFO:tasks.workunit.client.0.smithi112.stdout:6/784: getdents dd/df/d10/d47/d55/dbd/de9 0 2024-09-16T00:26:09.784 INFO:tasks.workunit.client.1.smithi169.stdout:1/614: dwrite dc/d26/fb3 [0,4194304] 0 2024-09-16T00:26:09.785 INFO:tasks.workunit.client.0.smithi112.stdout:0/985: fdatasync d5/da/d57/db0/ff3 0 2024-09-16T00:26:09.786 INFO:tasks.workunit.client.1.smithi169.stdout:3/742: truncate d7/d50/d6f/f74 2715991 0 2024-09-16T00:26:09.789 INFO:tasks.workunit.client.1.smithi169.stdout:1/615: creat dc/d36/d3a/d7f/fb8 x:0 0 0 2024-09-16T00:26:09.790 INFO:tasks.workunit.client.0.smithi112.stdout:2/973: creat d2/dd/d48/d22/d2b/d30/f13f x:0 0 0 2024-09-16T00:26:09.790 INFO:tasks.workunit.client.1.smithi169.stdout:1/616: dread - dc/d26/fa0 zero size 2024-09-16T00:26:09.792 INFO:tasks.workunit.client.1.smithi169.stdout:1/617: dread - dc/d36/d3a/d45/fa7 zero size 2024-09-16T00:26:09.793 INFO:tasks.workunit.client.1.smithi169.stdout:5/540: dread d2/f29 [0,4194304] 0 2024-09-16T00:26:09.796 INFO:tasks.workunit.client.0.smithi112.stdout:6/785: mkdir dd/df/d17/d27/d102 0 2024-09-16T00:26:09.797 INFO:tasks.workunit.client.0.smithi112.stdout:2/974: dwrite d2/f46 [0,4194304] 0 2024-09-16T00:26:09.799 INFO:tasks.workunit.client.1.smithi169.stdout:3/743: creat d7/d1a/d1c/d23/d88/d9b/dc8/df8/f109 x:0 0 0 2024-09-16T00:26:09.799 INFO:tasks.workunit.client.0.smithi112.stdout:2/975: read - d2/dd/d48/d22/d2b/d54/d60/daf/d11a/f137 zero size 2024-09-16T00:26:09.799 INFO:tasks.workunit.client.0.smithi112.stdout:3/916: write d2/d9/d2b/d47/d52/d7a/d80/d83/d7b/dad/d107/fb2 [651820,100476] 0 2024-09-16T00:26:09.800 INFO:tasks.workunit.client.0.smithi112.stdout:6/786: write dd/f2a [3431766,85473] 0 2024-09-16T00:26:09.800 INFO:tasks.workunit.client.1.smithi169.stdout:3/744: read d7/d1a/d1c/d23/d70/f8e [152104,59323] 0 2024-09-16T00:26:09.801 INFO:tasks.workunit.client.0.smithi112.stdout:2/976: readlink d2/dd/d48/d22/d2b/d54/lf5 0 2024-09-16T00:26:09.801 INFO:tasks.workunit.client.0.smithi112.stdout:3/917: fdatasync d2/d9/d2b/f9a 0 2024-09-16T00:26:09.802 INFO:tasks.workunit.client.0.smithi112.stdout:6/787: chown dd/df/d10/d47/d89/l92 158799575 1 2024-09-16T00:26:09.805 INFO:tasks.workunit.client.0.smithi112.stdout:6/788: truncate dd/fe3 728732 0 2024-09-16T00:26:09.810 INFO:tasks.workunit.client.0.smithi112.stdout:6/789: mknod dd/df/d11/d44/d8a/c103 0 2024-09-16T00:26:09.810 INFO:tasks.workunit.client.1.smithi169.stdout:5/541: truncate d2/da/d10/d13/f35 5293711 0 2024-09-16T00:26:09.810 INFO:tasks.workunit.client.1.smithi169.stdout:9/658: dread d5/da/d5b/d6c/f9d [0,4194304] 0 2024-09-16T00:26:09.811 INFO:tasks.workunit.client.0.smithi112.stdout:3/918: dwrite d2/d9/d2b/d47/d52/d7a/d80/d83/d68/d90/dcb/fd2 [0,4194304] 0 2024-09-16T00:26:09.812 INFO:tasks.workunit.client.0.smithi112.stdout:6/790: rmdir dd/d15 39 2024-09-16T00:26:09.814 INFO:tasks.workunit.client.1.smithi169.stdout:1/618: rename dc/d18/c27 to dc/d36/d3a/d45/d29/d4d/cb9 0 2024-09-16T00:26:09.814 INFO:tasks.workunit.client.1.smithi169.stdout:9/659: chown d5/da/d73/fb7 0 1 2024-09-16T00:26:09.814 INFO:tasks.workunit.client.1.smithi169.stdout:5/542: fsync d2/da/d10/d13/d1c/d32/d53/f92 0 2024-09-16T00:26:09.814 INFO:tasks.workunit.client.0.smithi112.stdout:3/919: readlink d2/d9/d2b/d47/d52/l75 0 2024-09-16T00:26:09.814 INFO:tasks.workunit.client.0.smithi112.stdout:0/986: truncate d5/da/d66/fe0 24680 0 2024-09-16T00:26:09.815 INFO:tasks.workunit.client.1.smithi169.stdout:3/745: symlink d7/d50/l10a 0 2024-09-16T00:26:09.815 INFO:tasks.workunit.client.0.smithi112.stdout:3/920: truncate d2/d9/f4e 4848141 0 2024-09-16T00:26:09.816 INFO:tasks.workunit.client.1.smithi169.stdout:4/599: dread d2/d12/d13/d1f/d3e/d5b/f18 [0,4194304] 0 2024-09-16T00:26:09.817 INFO:tasks.workunit.client.0.smithi112.stdout:3/921: stat d2/d9/d2b/d47/d78/da9/ff7 0 2024-09-16T00:26:09.818 INFO:tasks.workunit.client.0.smithi112.stdout:0/987: unlink d5/da/d18/d89/f130 0 2024-09-16T00:26:09.820 INFO:tasks.workunit.client.0.smithi112.stdout:3/922: rmdir d2 39 2024-09-16T00:26:09.821 INFO:tasks.workunit.client.1.smithi169.stdout:7/686: truncate d7/f8b 1672396 0 2024-09-16T00:26:09.823 INFO:tasks.workunit.client.1.smithi169.stdout:1/619: creat dc/d18/d67/d69/d7b/fba x:0 0 0 2024-09-16T00:26:09.823 INFO:tasks.workunit.client.1.smithi169.stdout:3/746: creat d7/de5/f10b x:0 0 0 2024-09-16T00:26:09.825 INFO:tasks.workunit.client.1.smithi169.stdout:3/747: dread - d7/d1a/d1c/d23/d70/fe0 zero size 2024-09-16T00:26:09.825 INFO:tasks.workunit.client.1.smithi169.stdout:1/620: write dc/d36/d39/d50/daf/f3f [2120562,59118] 0 2024-09-16T00:26:09.826 INFO:tasks.workunit.client.0.smithi112.stdout:0/988: dwrite d5/da/d1d/d71/d64/db6/fd2 [0,4194304] 0 2024-09-16T00:26:09.827 INFO:tasks.workunit.client.0.smithi112.stdout:3/923: symlink d2/d9/d2b/d47/d52/d7a/d80/d83/d7b/dad/d107/d69/d92/dda/l12e 0 2024-09-16T00:26:09.828 INFO:tasks.workunit.client.1.smithi169.stdout:7/687: creat d7/d16/fda x:0 0 0 2024-09-16T00:26:09.830 INFO:tasks.workunit.client.0.smithi112.stdout:3/924: chown d2/d9/d2b/d47/la6 236152751 1 2024-09-16T00:26:09.832 INFO:tasks.workunit.client.0.smithi112.stdout:6/791: write dd/df/d10/f3a [752952,34081] 0 2024-09-16T00:26:09.835 INFO:tasks.workunit.client.1.smithi169.stdout:3/748: symlink d7/d1a/d1c/d23/d70/l10c 0 2024-09-16T00:26:09.838 INFO:tasks.workunit.client.1.smithi169.stdout:4/600: dwrite d2/d12/d13/d5c/f94 [0,4194304] 0 2024-09-16T00:26:09.839 INFO:tasks.workunit.client.1.smithi169.stdout:1/621: dwrite dc/d36/f84 [0,4194304] 0 2024-09-16T00:26:09.840 INFO:tasks.workunit.client.0.smithi112.stdout:0/989: write d5/ff [587815,7430] 0 2024-09-16T00:26:09.842 INFO:tasks.workunit.client.0.smithi112.stdout:0/990: creat d5/da/d18/d24/d46/d5f/da6/f135 x:0 0 0 2024-09-16T00:26:09.843 INFO:tasks.workunit.client.1.smithi169.stdout:7/688: dread d7/d5b/f77 [0,4194304] 0 2024-09-16T00:26:09.846 INFO:tasks.workunit.client.0.smithi112.stdout:0/991: dread d5/da/d18/f1f [0,4194304] 0 2024-09-16T00:26:09.847 INFO:tasks.workunit.client.1.smithi169.stdout:3/749: write d7/d1a/d1c/d23/d70/fe0 [875927,113610] 0 2024-09-16T00:26:09.848 INFO:tasks.workunit.client.1.smithi169.stdout:9/660: write d5/da/dd0/fd2 [1110091,106024] 0 2024-09-16T00:26:09.851 INFO:tasks.workunit.client.0.smithi112.stdout:0/992: creat d5/da/f136 x:0 0 0 2024-09-16T00:26:09.851 INFO:tasks.workunit.client.0.smithi112.stdout:3/925: write d2/d9/d2b/d47/f9d [422933,34548] 0 2024-09-16T00:26:09.851 INFO:tasks.workunit.client.1.smithi169.stdout:3/750: chown d7/d50/d6f 114 1 2024-09-16T00:26:09.851 INFO:tasks.workunit.client.1.smithi169.stdout:4/601: symlink d2/d12/d13/d1f/d2f/lbf 0 2024-09-16T00:26:09.851 INFO:tasks.workunit.client.1.smithi169.stdout:4/602: chown d2/d12/d9c 11658152 1 2024-09-16T00:26:09.852 INFO:tasks.workunit.client.0.smithi112.stdout:0/993: fdatasync d5/da/d21/d6f/ff9 0 2024-09-16T00:26:09.854 INFO:tasks.workunit.client.0.smithi112.stdout:0/994: dread - d5/da/d1d/d71/fed zero size 2024-09-16T00:26:09.854 INFO:tasks.workunit.client.0.smithi112.stdout:6/792: write dd/fca [1003499,107763] 0 2024-09-16T00:26:09.855 INFO:tasks.workunit.client.1.smithi169.stdout:4/603: read d2/d12/d13/d1f/d3e/d5b/d1a/d9e/f72 [82836,8715] 0 2024-09-16T00:26:09.855 INFO:tasks.workunit.client.1.smithi169.stdout:3/751: dread - d7/d1a/d1c/d23/d88/fa9 zero size 2024-09-16T00:26:09.860 INFO:tasks.workunit.client.0.smithi112.stdout:6/793: rename dd/df/d11/fad to dd/d23/f104 0 2024-09-16T00:26:09.860 INFO:tasks.workunit.client.0.smithi112.stdout:0/995: mkdir d5/da/d18/d24/d46/d5f/da6/df1/d137 0 2024-09-16T00:26:09.861 INFO:tasks.workunit.client.0.smithi112.stdout:6/794: readlink dd/df/d10/l14 0 2024-09-16T00:26:09.861 INFO:tasks.workunit.client.1.smithi169.stdout:9/661: creat d5/da/d5b/fe8 x:0 0 0 2024-09-16T00:26:09.862 INFO:tasks.workunit.client.1.smithi169.stdout:4/604: mknod d2/d12/d13/d1f/d3e/d5b/d1a/d9e/cc0 0 2024-09-16T00:26:09.863 INFO:tasks.workunit.client.0.smithi112.stdout:6/795: rename dd/df/d17/lfa to dd/df/d17/l105 0 2024-09-16T00:26:09.863 INFO:tasks.workunit.client.1.smithi169.stdout:3/752: dread - d7/d1a/d2b/dc6/ff2 zero size 2024-09-16T00:26:09.866 INFO:tasks.workunit.client.1.smithi169.stdout:4/605: unlink d2/f9 0 2024-09-16T00:26:09.867 INFO:tasks.workunit.client.0.smithi112.stdout:6/796: getdents dd/df/d10/d47 0 2024-09-16T00:26:09.867 INFO:tasks.workunit.client.1.smithi169.stdout:3/753: creat d7/d1a/d1c/d23/d88/dd7/f10d x:0 0 0 2024-09-16T00:26:09.868 INFO:tasks.workunit.client.1.smithi169.stdout:4/606: chown d2/d12/d13/d1f/d2f/f7b 41561705 1 2024-09-16T00:26:09.868 INFO:tasks.workunit.client.1.smithi169.stdout:3/754: fsync d7/f8f 0 2024-09-16T00:26:09.868 INFO:tasks.workunit.client.0.smithi112.stdout:6/797: rmdir dd/d23 39 2024-09-16T00:26:09.869 INFO:tasks.workunit.client.1.smithi169.stdout:4/607: truncate d2/d12/d13/fb9 653716 0 2024-09-16T00:26:09.870 INFO:tasks.workunit.client.1.smithi169.stdout:3/755: mknod d7/d1a/d1c/d6c/c10e 0 2024-09-16T00:26:09.871 INFO:tasks.workunit.client.0.smithi112.stdout:6/798: truncate dd/d23/ff1 465732 0 2024-09-16T00:26:09.871 INFO:tasks.workunit.client.1.smithi169.stdout:9/662: write d5/da/d21/de4/f96 [2689181,112165] 0 2024-09-16T00:26:09.872 INFO:tasks.workunit.client.1.smithi169.stdout:4/608: creat d2/d12/d13/d4e/fc1 x:0 0 0 2024-09-16T00:26:09.875 INFO:tasks.workunit.client.1.smithi169.stdout:3/756: creat d7/d1a/d2b/dc6/dff/f10f x:0 0 0 2024-09-16T00:26:09.876 INFO:tasks.workunit.client.1.smithi169.stdout:9/663: rmdir d5/da/d5b/d91/dc9 39 2024-09-16T00:26:09.878 INFO:tasks.workunit.client.1.smithi169.stdout:3/757: read d7/d1a/d1c/fb6 [2071657,82822] 0 2024-09-16T00:26:09.878 INFO:tasks.workunit.client.1.smithi169.stdout:4/609: mkdir d2/d12/d9c/dc2 0 2024-09-16T00:26:09.879 INFO:tasks.workunit.client.1.smithi169.stdout:1/622: dread dc/d36/d39/d50/daf/f47 [0,4194304] 0 2024-09-16T00:26:09.883 INFO:tasks.workunit.client.1.smithi169.stdout:4/610: fdatasync d2/d12/d13/d1f/d3e/d83/f74 0 2024-09-16T00:26:09.884 INFO:tasks.workunit.client.0.smithi112.stdout:4/837: sync 2024-09-16T00:26:09.884 INFO:tasks.workunit.client.0.smithi112.stdout:5/767: sync 2024-09-16T00:26:09.885 INFO:tasks.workunit.client.1.smithi169.stdout:1/623: mkdir dc/d36/dbb 0 2024-09-16T00:26:09.885 INFO:tasks.workunit.client.0.smithi112.stdout:3/926: dwrite d2/d9/d2b/d47/d52/d7a/d80/d83/d7b/dad/d107/d69/f6f [0,4194304] 0 2024-09-16T00:26:09.887 INFO:tasks.workunit.client.1.smithi169.stdout:7/689: dwrite d7/f8b [0,4194304] 0 2024-09-16T00:26:09.887 INFO:tasks.workunit.client.1.smithi169.stdout:9/664: dwrite d5/da/d5b/d6c/da0/f95 [0,4194304] 0 2024-09-16T00:26:09.887 INFO:tasks.workunit.client.1.smithi169.stdout:1/624: readlink dc/d36/d3a/d7f/l8c 0 2024-09-16T00:26:09.890 INFO:tasks.workunit.client.0.smithi112.stdout:0/996: dread d5/da/d18/d24/d46/d88/f8c [0,4194304] 0 2024-09-16T00:26:09.891 INFO:tasks.workunit.client.0.smithi112.stdout:4/838: truncate d6/d39/d1c/d66/f7d 9319149 0 2024-09-16T00:26:09.893 INFO:tasks.workunit.client.0.smithi112.stdout:5/768: rename d4/d4f/d77/f7f to d4/dd/d49/da9/f104 0 2024-09-16T00:26:09.893 INFO:tasks.workunit.client.0.smithi112.stdout:6/799: dwrite dd/df/d10/d47/d55/dbd/dcb/fff [0,4194304] 0 2024-09-16T00:26:09.893 INFO:tasks.workunit.client.1.smithi169.stdout:1/625: dread dc/d36/d3a/d45/d29/d2a/f38 [0,4194304] 0 2024-09-16T00:26:09.895 INFO:tasks.workunit.client.0.smithi112.stdout:6/800: write dd/d23/f35 [1019407,42179] 0 2024-09-16T00:26:09.896 INFO:tasks.workunit.client.1.smithi169.stdout:4/611: rename d2/d12/d13/d1f/d3e/d5b/lf to d2/d12/d9c/da4/lc3 0 2024-09-16T00:26:09.896 INFO:tasks.workunit.client.1.smithi169.stdout:7/690: unlink d7/d71/d76/d29/d78/lbe 0 2024-09-16T00:26:09.899 INFO:tasks.workunit.client.1.smithi169.stdout:1/626: chown dc/d18/d67/l8f 262239078 1 2024-09-16T00:26:09.901 INFO:tasks.workunit.client.0.smithi112.stdout:5/769: creat d4/d7/d26/d4c/db3/db6/dd0/f105 x:0 0 0 2024-09-16T00:26:09.902 INFO:tasks.workunit.client.1.smithi169.stdout:4/612: dread - d2/d12/d13/d1f/d3e/d5b/d1a/d9e/f71 zero size 2024-09-16T00:26:09.902 INFO:tasks.workunit.client.1.smithi169.stdout:9/665: dwrite d5/da/d5b/d6f/fa1 [0,4194304] 0 2024-09-16T00:26:09.903 INFO:tasks.workunit.client.0.smithi112.stdout:5/770: chown d4/d7/d14/d90/fa4 11562140 1 2024-09-16T00:26:09.906 INFO:tasks.workunit.client.1.smithi169.stdout:7/691: rename d7/d71/d76/d9c/fa7 to d7/fdb 0 2024-09-16T00:26:09.908 INFO:tasks.workunit.client.0.smithi112.stdout:5/771: truncate d4/dd/d12/d52/d64/f9e 121554 0 2024-09-16T00:26:09.909 INFO:tasks.workunit.client.1.smithi169.stdout:9/666: unlink d5/da/d5b/d6c/dcb/db4/fbc 0 2024-09-16T00:26:09.910 INFO:tasks.workunit.client.0.smithi112.stdout:5/772: creat d4/dd/d12/d52/f106 x:0 0 0 2024-09-16T00:26:09.912 INFO:tasks.workunit.client.0.smithi112.stdout:0/997: truncate d5/d90/dc5/de2/f113 207941 0 2024-09-16T00:26:09.913 INFO:tasks.workunit.client.0.smithi112.stdout:0/998: chown d5/da/d57/d123 228 1 2024-09-16T00:26:09.915 INFO:tasks.workunit.client.1.smithi169.stdout:1/627: dwrite dc/d26/fa0 [0,4194304] 0 2024-09-16T00:26:09.916 INFO:tasks.workunit.client.1.smithi169.stdout:9/667: mkdir d5/da/d21/de9 0 2024-09-16T00:26:09.917 INFO:tasks.workunit.client.0.smithi112.stdout:4/839: dwrite d6/db/f93 [0,4194304] 0 2024-09-16T00:26:09.923 INFO:tasks.workunit.client.0.smithi112.stdout:5/773: dwrite d4/dd/f27 [4194304,4194304] 0 2024-09-16T00:26:09.923 INFO:tasks.workunit.client.0.smithi112.stdout:4/840: symlink d6/db/df/d5e/l133 0 2024-09-16T00:26:09.923 INFO:tasks.workunit.client.1.smithi169.stdout:4/613: dread d2/d12/d13/d1f/f8e [0,4194304] 0 2024-09-16T00:26:09.923 INFO:tasks.workunit.client.1.smithi169.stdout:1/628: dread dc/d18/d67/d69/fae [0,4194304] 0 2024-09-16T00:26:09.924 INFO:tasks.workunit.client.0.smithi112.stdout:0/999: link d5/da/d18/d24/c30 d5/da/d57/db0/d11c/c138 0 2024-09-16T00:26:09.926 INFO:tasks.workunit.client.0.smithi112.stdout:5/774: dread - d4/d4f/d77/dcc/d7c/fa3 zero size 2024-09-16T00:26:09.927 INFO:tasks.workunit.client.1.smithi169.stdout:1/629: creat dc/d36/d39/d50/d75/fbc x:0 0 0 2024-09-16T00:26:09.928 INFO:tasks.workunit.client.1.smithi169.stdout:9/668: dwrite d5/da/d21/d23/f2d [4194304,4194304] 0 2024-09-16T00:26:09.930 INFO:tasks.workunit.client.1.smithi169.stdout:4/614: creat d2/d12/d13/d1f/d3e/fc4 x:0 0 0 2024-09-16T00:26:09.933 INFO:tasks.workunit.client.1.smithi169.stdout:9/669: dread - d5/da/d5b/d91/dc9/fd9 zero size 2024-09-16T00:26:09.934 INFO:tasks.workunit.client.0.smithi112.stdout:5/775: dwrite d4/d7/d26/d36/f103 [0,4194304] 0 2024-09-16T00:26:09.935 INFO:tasks.workunit.client.1.smithi169.stdout:4/615: creat d2/d12/d9c/da4/fc5 x:0 0 0 2024-09-16T00:26:09.936 INFO:tasks.workunit.client.0.smithi112.stdout:5/776: chown d4/dd/d49/de6/fed 19 1 2024-09-16T00:26:09.937 INFO:tasks.workunit.client.0.smithi112.stdout:4/841: dread d6/fff [0,4194304] 0 2024-09-16T00:26:09.937 INFO:tasks.workunit.client.1.smithi169.stdout:1/630: dwrite dc/d36/d39/d50/daf/d86/f4f [4194304,4194304] 0 2024-09-16T00:26:09.940 INFO:tasks.workunit.client.0.smithi112.stdout:5/777: getdents d4/dd/d49/d70/d98 0 2024-09-16T00:26:09.944 INFO:tasks.workunit.client.0.smithi112.stdout:6/801: dread dd/df/d10/f60 [0,4194304] 0 2024-09-16T00:26:09.944 INFO:tasks.workunit.client.0.smithi112.stdout:5/778: truncate d4/d4f/d77/f6c 552330 0 2024-09-16T00:26:09.945 INFO:tasks.workunit.client.1.smithi169.stdout:9/670: dread d5/da/d5b/d6c/da0/f8c [0,4194304] 0 2024-09-16T00:26:09.948 INFO:tasks.workunit.client.1.smithi169.stdout:9/671: creat d5/da/d2e/d99/fea x:0 0 0 2024-09-16T00:26:09.949 INFO:tasks.workunit.client.0.smithi112.stdout:5/779: rename d4/dd/d49/de6/fed to d4/dd/f107 0 2024-09-16T00:26:09.949 INFO:tasks.workunit.client.1.smithi169.stdout:9/672: mknod d5/ceb 0 2024-09-16T00:26:09.951 INFO:tasks.workunit.client.1.smithi169.stdout:9/673: fdatasync d5/da/d2e/f8a 0 2024-09-16T00:26:09.952 INFO:tasks.workunit.client.0.smithi112.stdout:6/802: rename dd/d15/da5/lce to dd/df/d10/l106 0 2024-09-16T00:26:09.953 INFO:tasks.workunit.client.0.smithi112.stdout:6/803: chown dd/df/d10/d47/d55/dbd/f101 3770 1 2024-09-16T00:26:09.955 INFO:tasks.workunit.client.0.smithi112.stdout:6/804: mknod dd/d15/c107 0 2024-09-16T00:26:09.957 INFO:tasks.workunit.client.0.smithi112.stdout:6/805: symlink dd/df/d10/d47/d55/dbd/l108 0 2024-09-16T00:26:09.957 INFO:tasks.workunit.client.0.smithi112.stdout:6/806: readlink l5 0 2024-09-16T00:26:09.957 INFO:tasks.workunit.client.1.smithi169.stdout:9/674: dwrite d5/d49/d64/fcd [0,4194304] 0 2024-09-16T00:26:09.960 INFO:tasks.workunit.client.0.smithi112.stdout:6/807: write dd/df/d17/d27/d54/f6a [612129,43806] 0 2024-09-16T00:26:09.973 INFO:tasks.workunit.client.0.smithi112.stdout:4/842: dread d6/db/df/d5e/d61/f3e [0,4194304] 0 2024-09-16T00:26:09.974 INFO:tasks.workunit.client.0.smithi112.stdout:4/843: chown d6/d39/d1c/d2c/d69/ff5 22932 1 2024-09-16T00:26:09.989 INFO:tasks.workunit.client.0.smithi112.stdout:5/780: dread d4/d7/f35 [0,4194304] 0 2024-09-16T00:26:09.997 INFO:tasks.workunit.client.1.smithi169.stdout:1/631: write dc/d26/f55 [2781019,36695] 0 2024-09-16T00:26:09.999 INFO:tasks.workunit.client.1.smithi169.stdout:9/675: write d5/da/d21/f30 [3401502,71098] 0 2024-09-16T00:26:09.999 INFO:tasks.workunit.client.0.smithi112.stdout:5/781: write d4/dd/d49/da9/f104 [5020246,83420] 0 2024-09-16T00:26:10.000 INFO:tasks.workunit.client.0.smithi112.stdout:5/782: readlink d4/dd/d6f/l75 0 2024-09-16T00:26:10.001 INFO:tasks.workunit.client.0.smithi112.stdout:6/808: truncate dd/df/d17/d27/d84/f36 4018179 0 2024-09-16T00:26:10.001 INFO:tasks.workunit.client.1.smithi169.stdout:7/692: dwrite d7/d16/f79 [0,4194304] 0 2024-09-16T00:26:10.002 INFO:tasks.workunit.client.0.smithi112.stdout:6/809: stat dd/df/d10/d47/d55/dbd/le2 0 2024-09-16T00:26:10.002 INFO:tasks.workunit.client.0.smithi112.stdout:5/783: readlink d4/d7/d14/d90/lc3 0 2024-09-16T00:26:10.003 INFO:tasks.workunit.client.1.smithi169.stdout:1/632: creat dc/d6c/d96/fbd x:0 0 0 2024-09-16T00:26:10.004 INFO:tasks.workunit.client.0.smithi112.stdout:5/784: write d4/dd/f107 [386966,72247] 0 2024-09-16T00:26:10.006 INFO:tasks.workunit.client.1.smithi169.stdout:7/693: creat d7/d16/d49/d84/d8e/fdc x:0 0 0 2024-09-16T00:26:10.007 INFO:tasks.workunit.client.0.smithi112.stdout:5/785: truncate d4/dd/d49/f96 315046 0 2024-09-16T00:26:10.007 INFO:tasks.workunit.client.1.smithi169.stdout:1/633: creat dc/d36/d3a/d45/d29/d4d/fbe x:0 0 0 2024-09-16T00:26:10.008 INFO:tasks.workunit.client.0.smithi112.stdout:5/786: chown d4/dd/d12/d52/d64 239971380 1 2024-09-16T00:26:10.009 INFO:tasks.workunit.client.1.smithi169.stdout:9/676: dwrite d5/da/d21/de4/f86 [0,4194304] 0 2024-09-16T00:26:10.009 INFO:tasks.workunit.client.0.smithi112.stdout:5/787: write d4/d7/d14/d90/fa4 [4465646,118637] 0 2024-09-16T00:26:10.011 INFO:tasks.workunit.client.1.smithi169.stdout:9/677: write d5/da/d5b/d91/dc9/fd9 [574738,21869] 0 2024-09-16T00:26:10.013 INFO:tasks.workunit.client.0.smithi112.stdout:5/788: read d4/d7/d26/f46 [3682302,10395] 0 2024-09-16T00:26:10.015 INFO:tasks.workunit.client.1.smithi169.stdout:7/694: link d7/d16/f25 d7/d16/d49/d84/d8e/fdd 0 2024-09-16T00:26:10.022 INFO:tasks.workunit.client.0.smithi112.stdout:5/789: dwrite d4/d1c/d43/ff2 [0,4194304] 0 2024-09-16T00:26:10.022 INFO:tasks.workunit.client.1.smithi169.stdout:2/607: sync 2024-09-16T00:26:10.022 INFO:tasks.workunit.client.1.smithi169.stdout:6/846: sync 2024-09-16T00:26:10.023 INFO:tasks.workunit.client.1.smithi169.stdout:9/678: dwrite d5/da/d5b/d6c/dcb/db4/fc4 [0,4194304] 0 2024-09-16T00:26:10.029 INFO:tasks.workunit.client.1.smithi169.stdout:1/634: dwrite dc/fa1 [4194304,4194304] 0 2024-09-16T00:26:10.029 INFO:tasks.workunit.client.1.smithi169.stdout:6/847: write d2/d36/fe0 [4807318,27779] 0 2024-09-16T00:26:10.029 INFO:tasks.workunit.client.0.smithi112.stdout:6/810: write dd/df/d10/d47/d55/dc4/dd1/fa9 [2866305,114658] 0 2024-09-16T00:26:10.030 INFO:tasks.workunit.client.1.smithi169.stdout:9/679: chown d5/da/d5b/d91/dc9/lbb 12407658 1 2024-09-16T00:26:10.030 INFO:tasks.workunit.client.1.smithi169.stdout:2/608: stat d7/db/d89/db4/d9d/ld3 0 2024-09-16T00:26:10.034 INFO:tasks.workunit.client.0.smithi112.stdout:4/844: dread d6/db/df/f5f [0,4194304] 0 2024-09-16T00:26:10.035 INFO:tasks.workunit.client.1.smithi169.stdout:7/695: dwrite d7/d5b/d61/d98/fd3 [0,4194304] 0 2024-09-16T00:26:10.036 INFO:tasks.workunit.client.1.smithi169.stdout:9/680: dread d5/da/d2e/d47/f7c [0,4194304] 0 2024-09-16T00:26:10.037 INFO:tasks.workunit.client.1.smithi169.stdout:2/609: dread d7/d6b/d70/d79/f9f [0,4194304] 0 2024-09-16T00:26:10.037 INFO:tasks.workunit.client.0.smithi112.stdout:4/845: read d6/db/df/d3c/f73 [4158569,101218] 0 2024-09-16T00:26:10.041 INFO:tasks.workunit.client.1.smithi169.stdout:6/848: truncate d2/d3f/d95/fb2 5178220 0 2024-09-16T00:26:10.042 INFO:tasks.workunit.client.0.smithi112.stdout:7/862: sync 2024-09-16T00:26:10.042 INFO:tasks.workunit.client.0.smithi112.stdout:8/754: sync 2024-09-16T00:26:10.042 INFO:tasks.workunit.client.1.smithi169.stdout:9/681: fdatasync d5/da/d2e/d99/fea 0 2024-09-16T00:26:10.044 INFO:tasks.workunit.client.0.smithi112.stdout:7/863: chown d6/d10/d16/d96/dc7/dbf/d117/d6c/cc8 1203286035 1 2024-09-16T00:26:10.046 INFO:tasks.workunit.client.1.smithi169.stdout:6/849: creat d2/d22/da7/f10e x:0 0 0 2024-09-16T00:26:10.046 INFO:tasks.workunit.client.1.smithi169.stdout:2/610: chown d7/db/d27/lbb 3958245 1 2024-09-16T00:26:10.047 INFO:tasks.workunit.client.1.smithi169.stdout:1/635: dwrite dc/d18/d67/d69/f80 [0,4194304] 0 2024-09-16T00:26:10.048 INFO:tasks.workunit.client.0.smithi112.stdout:7/864: rename d6/fb2 to d6/d10/d75/f120 0 2024-09-16T00:26:10.049 INFO:tasks.workunit.client.0.smithi112.stdout:8/755: dwrite d10/d12/d1c/d61/d6c/fb5 [0,4194304] 0 2024-09-16T00:26:10.050 INFO:tasks.workunit.client.1.smithi169.stdout:9/682: truncate d5/da/d2e/d47/fdf 768023 0 2024-09-16T00:26:10.051 INFO:tasks.workunit.client.0.smithi112.stdout:8/756: rename d10/d12 to d10/d12/d24/dfb 22 2024-09-16T00:26:10.051 INFO:tasks.workunit.client.0.smithi112.stdout:7/865: dread - d6/d10/d75/d8f/d90/ddc/dee/f115 zero size 2024-09-16T00:26:10.053 INFO:tasks.workunit.client.1.smithi169.stdout:7/696: dread d7/f67 [0,4194304] 0 2024-09-16T00:26:10.055 INFO:tasks.workunit.client.0.smithi112.stdout:8/757: creat d10/d12/d1c/df2/ffc x:0 0 0 2024-09-16T00:26:10.056 INFO:tasks.workunit.client.0.smithi112.stdout:8/758: chown d10/d12/d1c/ff5 1706942 1 2024-09-16T00:26:10.058 INFO:tasks.workunit.client.1.smithi169.stdout:7/697: symlink d7/d71/lde 0 2024-09-16T00:26:10.058 INFO:tasks.workunit.client.0.smithi112.stdout:8/759: stat d10/d12/d1c/d61/d6c/dd1/d58/l6d 0 2024-09-16T00:26:10.061 INFO:tasks.workunit.client.0.smithi112.stdout:8/760: creat d10/d12/d1c/dd0/dbb/ffd x:0 0 0 2024-09-16T00:26:10.061 INFO:tasks.workunit.client.1.smithi169.stdout:7/698: chown d7/d71/lde 1036155 1 2024-09-16T00:26:10.062 INFO:tasks.workunit.client.1.smithi169.stdout:6/850: dwrite d2/d3f/d41/d5c/d73/fa8 [0,4194304] 0 2024-09-16T00:26:10.062 INFO:tasks.workunit.client.1.smithi169.stdout:1/636: dread dc/d36/d3a/d7f/d93/fa3 [4194304,4194304] 0 2024-09-16T00:26:10.063 INFO:tasks.workunit.client.1.smithi169.stdout:7/699: stat d7/d16/d49/db2 0 2024-09-16T00:26:10.064 INFO:tasks.workunit.client.1.smithi169.stdout:6/851: chown d2/d3f/d41/d5c/f88 135550477 1 2024-09-16T00:26:10.065 INFO:tasks.workunit.client.0.smithi112.stdout:8/761: creat d10/d12/d1c/d61/d6c/ffe x:0 0 0 2024-09-16T00:26:10.067 INFO:tasks.workunit.client.0.smithi112.stdout:8/762: dread - d10/d12/d1c/d61/fa3 zero size 2024-09-16T00:26:10.067 INFO:tasks.workunit.client.1.smithi169.stdout:1/637: write dc/d36/d3a/d7f/d93/f6b [312758,15335] 0 2024-09-16T00:26:10.067 INFO:tasks.workunit.client.0.smithi112.stdout:4/846: write d6/d39/d1c/f27 [4626498,67620] 0 2024-09-16T00:26:10.070 INFO:tasks.workunit.client.1.smithi169.stdout:7/700: mkdir d7/d71/d76/d9c/ddf 0 2024-09-16T00:26:10.071 INFO:tasks.workunit.client.0.smithi112.stdout:8/763: mkdir d10/de9/def/dff 0 2024-09-16T00:26:10.073 INFO:tasks.workunit.client.1.smithi169.stdout:7/701: symlink d7/d5b/le0 0 2024-09-16T00:26:10.073 INFO:tasks.workunit.client.0.smithi112.stdout:7/866: write d6/d10/f53 [16149,80627] 0 2024-09-16T00:26:10.074 INFO:tasks.workunit.client.0.smithi112.stdout:6/811: dwrite dd/df/d11/d44/d8a/fe7 [0,4194304] 0 2024-09-16T00:26:10.074 INFO:tasks.workunit.client.1.smithi169.stdout:6/852: dwrite d2/f45 [8388608,4194304] 0 2024-09-16T00:26:10.075 INFO:tasks.workunit.client.0.smithi112.stdout:5/790: dread d4/dd/d49/da9/f104 [0,4194304] 0 2024-09-16T00:26:10.078 INFO:tasks.workunit.client.0.smithi112.stdout:5/791: dread - d4/d1c/fea zero size 2024-09-16T00:26:10.078 INFO:tasks.workunit.client.0.smithi112.stdout:4/847: creat d6/db/df/d5e/d61/f134 x:0 0 0 2024-09-16T00:26:10.078 INFO:tasks.workunit.client.1.smithi169.stdout:6/853: mkdir d2/d3f/d41/d72/d9e/de1/d10f 0 2024-09-16T00:26:10.079 INFO:tasks.workunit.client.0.smithi112.stdout:7/867: creat d6/d10/d75/d8f/d90/dd7/f121 x:0 0 0 2024-09-16T00:26:10.080 INFO:tasks.workunit.client.1.smithi169.stdout:6/854: write d2/f19 [4684833,15203] 0 2024-09-16T00:26:10.081 INFO:tasks.workunit.client.0.smithi112.stdout:5/792: creat d4/d7/d14/d90/f108 x:0 0 0 2024-09-16T00:26:10.086 INFO:tasks.workunit.client.0.smithi112.stdout:7/868: link d6/d10/f76 d6/d7f/dc3/f122 0 2024-09-16T00:26:10.091 INFO:tasks.workunit.client.1.smithi169.stdout:6/855: dwrite d2/d3f/d41/d72/d9e/dc0/f108 [0,4194304] 0 2024-09-16T00:26:10.092 INFO:tasks.workunit.client.1.smithi169.stdout:9/683: dread d5/da/d73/f9b [0,4194304] 0 2024-09-16T00:26:10.094 INFO:tasks.workunit.client.1.smithi169.stdout:9/684: readlink d5/da/d5b/d91/dc9/l82 0 2024-09-16T00:26:10.101 INFO:tasks.workunit.client.1.smithi169.stdout:2/611: dwrite f2 [0,4194304] 0 2024-09-16T00:26:10.101 INFO:tasks.workunit.client.1.smithi169.stdout:1/638: write dc/d36/d3a/d45/d29/f8d [898296,116934] 0 2024-09-16T00:26:10.106 INFO:tasks.workunit.client.0.smithi112.stdout:1/892: sync 2024-09-16T00:26:10.107 INFO:tasks.workunit.client.1.smithi169.stdout:2/612: rmdir d7/d1c/d26/d96/dab 39 2024-09-16T00:26:10.107 INFO:tasks.workunit.client.1.smithi169.stdout:6/856: dwrite d2/d3f/d41/d5c/f94 [0,4194304] 0 2024-09-16T00:26:10.109 INFO:tasks.workunit.client.1.smithi169.stdout:1/639: symlink dc/d36/lbf 0 2024-09-16T00:26:10.110 INFO:tasks.workunit.client.1.smithi169.stdout:9/685: dwrite d5/fc6 [0,4194304] 0 2024-09-16T00:26:10.111 INFO:tasks.workunit.client.1.smithi169.stdout:2/613: dread - d7/db/d27/d29/fc0 zero size 2024-09-16T00:26:10.112 INFO:tasks.workunit.client.0.smithi112.stdout:1/893: dwrite d0/d14/df/d28/d42/fba [0,4194304] 0 2024-09-16T00:26:10.118 INFO:tasks.workunit.client.1.smithi169.stdout:6/857: symlink d2/d3f/d41/d72/d9e/de1/de4/l110 0 2024-09-16T00:26:10.118 INFO:tasks.workunit.client.1.smithi169.stdout:9/686: mknod d5/da/d5b/d6c/dcb/da4/dc1/cec 0 2024-09-16T00:26:10.121 INFO:tasks.workunit.client.0.smithi112.stdout:6/812: truncate dd/f7e 3096100 0 2024-09-16T00:26:10.121 INFO:tasks.workunit.client.1.smithi169.stdout:2/614: getdents d7/db/d27/d29 0 2024-09-16T00:26:10.123 INFO:tasks.workunit.client.1.smithi169.stdout:9/687: rename c4 to d5/d49/dc0/ced 0 2024-09-16T00:26:10.124 INFO:tasks.workunit.client.1.smithi169.stdout:7/702: dread d7/d16/d49/f53 [0,4194304] 0 2024-09-16T00:26:10.124 INFO:tasks.workunit.client.1.smithi169.stdout:2/615: fdatasync d7/db/d20/d5c/db2/fcd 0 2024-09-16T00:26:10.124 INFO:tasks.workunit.client.1.smithi169.stdout:9/688: stat d5/da/d5b/d91/d9a/fa3 0 2024-09-16T00:26:10.126 INFO:tasks.workunit.client.0.smithi112.stdout:8/764: dwrite d10/d12/d1c/d61/d6c/dd1/f68 [0,4194304] 0 2024-09-16T00:26:10.126 INFO:tasks.workunit.client.1.smithi169.stdout:9/689: fdatasync d5/da/d5b/d91/dc9/fd9 0 2024-09-16T00:26:10.126 INFO:tasks.workunit.client.1.smithi169.stdout:2/616: readlink d7/db/d27/d29/d37/l2f 0 2024-09-16T00:26:10.127 INFO:tasks.workunit.client.1.smithi169.stdout:9/690: chown d5/da/ff 26309457 1 2024-09-16T00:26:10.128 INFO:tasks.workunit.client.1.smithi169.stdout:7/703: truncate d7/d5b/f87 365641 0 2024-09-16T00:26:10.128 INFO:tasks.workunit.client.1.smithi169.stdout:2/617: rmdir d7/db/d27/d29/d77 39 2024-09-16T00:26:10.129 INFO:tasks.workunit.client.0.smithi112.stdout:4/848: dread d6/d39/d36/d5c/f77 [0,4194304] 0 2024-09-16T00:26:10.129 INFO:tasks.workunit.client.0.smithi112.stdout:5/793: dwrite f2 [0,4194304] 0 2024-09-16T00:26:10.130 INFO:tasks.workunit.client.1.smithi169.stdout:9/691: creat d5/da/d5b/d6c/dcb/da4/dc1/fee x:0 0 0 2024-09-16T00:26:10.133 INFO:tasks.workunit.client.0.smithi112.stdout:6/813: rename dd/d15/da5/d79/lf7 to dd/df/d10/d47/d89/dbc/l109 0 2024-09-16T00:26:10.133 INFO:tasks.workunit.client.0.smithi112.stdout:4/849: chown d6/d39/d36/d5c/d7e/dab 953096611 1 2024-09-16T00:26:10.134 INFO:tasks.workunit.client.0.smithi112.stdout:6/814: stat dd/df/d13/d18/f28 0 2024-09-16T00:26:10.135 INFO:tasks.workunit.client.1.smithi169.stdout:9/692: link d5/f35 d5/d49/fef 0 2024-09-16T00:26:10.136 INFO:tasks.workunit.client.0.smithi112.stdout:5/794: mknod d4/d7/d26/d4c/db3/db6/dd0/dfc/c109 0 2024-09-16T00:26:10.136 INFO:tasks.workunit.client.0.smithi112.stdout:6/815: write dd/df/d10/d47/d55/f78 [2941080,20591] 0 2024-09-16T00:26:10.136 INFO:tasks.workunit.client.0.smithi112.stdout:4/850: rmdir d6/db/d1f 39 2024-09-16T00:26:10.137 INFO:tasks.workunit.client.1.smithi169.stdout:9/693: symlink d5/da/lf0 0 2024-09-16T00:26:10.138 INFO:tasks.workunit.client.0.smithi112.stdout:4/851: stat d6/db/df/d5e/d61/d64 0 2024-09-16T00:26:10.138 INFO:tasks.workunit.client.0.smithi112.stdout:6/816: write dd/df/d13/f48 [1435678,90366] 0 2024-09-16T00:26:10.140 INFO:tasks.workunit.client.0.smithi112.stdout:4/852: readlink d6/db/df/d5e/d61/d64/d6a/l9f 0 2024-09-16T00:26:10.141 INFO:tasks.workunit.client.0.smithi112.stdout:5/795: getdents d4/d4f/d77/dcc/d7c 0 2024-09-16T00:26:10.141 INFO:tasks.workunit.client.0.smithi112.stdout:6/817: read dd/d15/da5/dae/fc6 [7890634,9816] 0 2024-09-16T00:26:10.143 INFO:tasks.workunit.client.0.smithi112.stdout:4/853: mknod d6/db/df/d5e/d61/c135 0 2024-09-16T00:26:10.144 INFO:tasks.workunit.client.0.smithi112.stdout:5/796: creat d4/dd/df6/f10a x:0 0 0 2024-09-16T00:26:10.144 INFO:tasks.workunit.client.1.smithi169.stdout:9/694: dread d5/da/f4d [0,4194304] 0 2024-09-16T00:26:10.147 INFO:tasks.workunit.client.1.smithi169.stdout:9/695: truncate d5/da/d73/f9b 4606546 0 2024-09-16T00:26:10.149 INFO:tasks.workunit.client.1.smithi169.stdout:9/696: rename d5/da/d2e/l38 to d5/de0/lf1 0 2024-09-16T00:26:10.154 INFO:tasks.workunit.client.1.smithi169.stdout:1/640: dread dc/d36/d3a/d45/f1c [0,4194304] 0 2024-09-16T00:26:10.155 INFO:tasks.workunit.client.0.smithi112.stdout:7/869: dwrite d6/d10/d16/d96/fcc [0,4194304] 0 2024-09-16T00:26:10.156 INFO:tasks.workunit.client.1.smithi169.stdout:9/697: dwrite d5/f8f [0,4194304] 0 2024-09-16T00:26:10.156 INFO:tasks.workunit.client.0.smithi112.stdout:1/894: dwrite d0/d14/d1c/d20/daf/f59 [0,4194304] 0 2024-09-16T00:26:10.157 INFO:tasks.workunit.client.1.smithi169.stdout:9/698: fdatasync d5/da/d5b/d91/d9a/fba 0 2024-09-16T00:26:10.158 INFO:tasks.workunit.client.1.smithi169.stdout:1/641: symlink dc/d18/d67/d69/d7b/d92/lc0 0 2024-09-16T00:26:10.158 INFO:tasks.workunit.client.0.smithi112.stdout:4/854: dwrite d6/db/d1f/d41/fb2 [0,4194304] 0 2024-09-16T00:26:10.165 INFO:tasks.workunit.client.1.smithi169.stdout:9/699: creat d5/da/d5b/d6c/ff2 x:0 0 0 2024-09-16T00:26:10.165 INFO:tasks.workunit.client.1.smithi169.stdout:9/700: truncate d5/da/d21/f45 979213 0 2024-09-16T00:26:10.166 INFO:tasks.workunit.client.1.smithi169.stdout:2/618: dread d7/d1c/d26/f76 [0,4194304] 0 2024-09-16T00:26:10.169 INFO:tasks.workunit.client.1.smithi169.stdout:9/701: mkdir d5/da/df3 0 2024-09-16T00:26:10.171 INFO:tasks.workunit.client.1.smithi169.stdout:9/702: readlink d5/da/d5b/d91/dc9/lbb 0 2024-09-16T00:26:10.172 INFO:tasks.workunit.client.0.smithi112.stdout:1/895: symlink d0/d14/df/d28/d45/d46/d58/df9/l126 0 2024-09-16T00:26:10.173 INFO:tasks.workunit.client.0.smithi112.stdout:8/765: dwrite d10/d12/d24/f82 [0,4194304] 0 2024-09-16T00:26:10.174 INFO:tasks.workunit.client.0.smithi112.stdout:7/870: write d6/d10/d16/d96/fe2 [867929,2039] 0 2024-09-16T00:26:10.175 INFO:tasks.workunit.client.1.smithi169.stdout:2/619: link d7/db/f13 d7/db/d89/db4/d9d/dbe/fd7 0 2024-09-16T00:26:10.181 INFO:tasks.workunit.client.1.smithi169.stdout:9/703: dwrite d5/da/d2e/d47/fdf [0,4194304] 0 2024-09-16T00:26:10.183 INFO:tasks.workunit.client.0.smithi112.stdout:7/871: truncate d6/d10/d16/d96/dc7/d39/f9a 3049153 0 2024-09-16T00:26:10.184 INFO:tasks.workunit.client.1.smithi169.stdout:9/704: mknod d5/d49/cf4 0 2024-09-16T00:26:10.186 INFO:tasks.workunit.client.0.smithi112.stdout:5/797: dwrite d4/d7/d14/f8c [0,4194304] 0 2024-09-16T00:26:10.188 INFO:tasks.workunit.client.0.smithi112.stdout:1/896: dwrite d0/f4e [0,4194304] 0 2024-09-16T00:26:10.188 INFO:tasks.workunit.client.1.smithi169.stdout:1/642: truncate dc/d2c/f32 1598978 0 2024-09-16T00:26:10.190 INFO:tasks.workunit.client.0.smithi112.stdout:5/798: read - d4/dd/d49/d70/fd6 zero size 2024-09-16T00:26:10.191 INFO:tasks.workunit.client.1.smithi169.stdout:9/705: dwrite d5/da/d21/d23/f40 [0,4194304] 0 2024-09-16T00:26:10.195 INFO:tasks.workunit.client.0.smithi112.stdout:7/872: truncate d6/d10/d16/d96/dc7/fdb 82483 0 2024-09-16T00:26:10.197 INFO:tasks.workunit.client.1.smithi169.stdout:8/645: sync 2024-09-16T00:26:10.198 INFO:tasks.workunit.client.1.smithi169.stdout:9/706: read d5/da/d5b/d6c/da0/f7a [812318,14815] 0 2024-09-16T00:26:10.198 INFO:tasks.workunit.client.0.smithi112.stdout:8/766: dwrite d10/d12/d1c/d3b/fa8 [4194304,4194304] 0 2024-09-16T00:26:10.200 INFO:tasks.workunit.client.1.smithi169.stdout:8/646: mkdir d7/dc/d17/d48/dce 0 2024-09-16T00:26:10.202 INFO:tasks.workunit.client.1.smithi169.stdout:9/707: mknod d5/da/d5b/d91/db3/cf5 0 2024-09-16T00:26:10.203 INFO:tasks.workunit.client.0.smithi112.stdout:8/767: write d10/f46 [2442898,79428] 0 2024-09-16T00:26:10.204 INFO:tasks.workunit.client.0.smithi112.stdout:7/873: mkdir d6/d10/d75/d8f/d123 0 2024-09-16T00:26:10.205 INFO:tasks.workunit.client.0.smithi112.stdout:8/768: stat d10/d12/fd3 0 2024-09-16T00:26:10.205 INFO:tasks.workunit.client.1.smithi169.stdout:9/708: rename d5/ceb to d5/da/df3/cf6 0 2024-09-16T00:26:10.207 INFO:tasks.workunit.client.0.smithi112.stdout:8/769: write d10/d12/d1c/d61/fb4 [5207978,37618] 0 2024-09-16T00:26:10.210 INFO:tasks.workunit.client.0.smithi112.stdout:4/855: dread d6/d39/d36/d5c/f63 [0,4194304] 0 2024-09-16T00:26:10.210 INFO:tasks.workunit.client.0.smithi112.stdout:5/799: dwrite d4/dd/d12/d52/f106 [0,4194304] 0 2024-09-16T00:26:10.212 INFO:tasks.workunit.client.1.smithi169.stdout:8/647: dwrite d7/dc/d1d/f91 [0,4194304] 0 2024-09-16T00:26:10.212 INFO:tasks.workunit.client.0.smithi112.stdout:7/874: rmdir d6/d10/d75/d8f/d90/dd3 39 2024-09-16T00:26:10.214 INFO:tasks.workunit.client.0.smithi112.stdout:8/770: creat d10/de9/f100 x:0 0 0 2024-09-16T00:26:10.215 INFO:tasks.workunit.client.0.smithi112.stdout:4/856: mknod d6/db/d1f/d94/d10c/d123/c136 0 2024-09-16T00:26:10.216 INFO:tasks.workunit.client.0.smithi112.stdout:1/897: getdents d0/d14/df/d28/d45/dd4/d47/d10a 0 2024-09-16T00:26:10.216 INFO:tasks.workunit.client.0.smithi112.stdout:5/800: dread d4/dd/f107 [0,4194304] 0 2024-09-16T00:26:10.217 INFO:tasks.workunit.client.0.smithi112.stdout:5/801: dread - d4/d7/d14/d90/f108 zero size 2024-09-16T00:26:10.218 INFO:tasks.workunit.client.0.smithi112.stdout:7/875: unlink d6/d10/d75/d88/f2d 0 2024-09-16T00:26:10.218 INFO:tasks.workunit.client.0.smithi112.stdout:8/771: write d10/d12/d1c/d61/d6c/dd1/d8c/d7c/d97/fad [3942658,42797] 0 2024-09-16T00:26:10.220 INFO:tasks.workunit.client.0.smithi112.stdout:5/802: write d4/dd/d12/d52/f63 [3215214,76072] 0 2024-09-16T00:26:10.220 INFO:tasks.workunit.client.0.smithi112.stdout:7/876: write d6/fa0 [347529,83274] 0 2024-09-16T00:26:10.222 INFO:tasks.workunit.client.0.smithi112.stdout:4/857: symlink d6/db/d1f/d41/l137 0 2024-09-16T00:26:10.222 INFO:tasks.workunit.client.0.smithi112.stdout:5/803: dread - d4/d7/d9b/faf zero size 2024-09-16T00:26:10.222 INFO:tasks.workunit.client.1.smithi169.stdout:9/709: link d5/da/d5b/d91/dc9/f87 d5/da/d5b/d91/d9a/ff7 0 2024-09-16T00:26:10.223 INFO:tasks.workunit.client.1.smithi169.stdout:8/648: rename d7/dc/d1d to d7/dc/d26/d41/d92/dcf 0 2024-09-16T00:26:10.223 INFO:tasks.workunit.client.0.smithi112.stdout:8/772: readlink d10/d12/d1c/d61/d6c/dd1/d8c/d7c/lf1 0 2024-09-16T00:26:10.223 INFO:tasks.workunit.client.0.smithi112.stdout:7/877: write d6/d10/da1/da2/f118 [636210,105255] 0 2024-09-16T00:26:10.224 INFO:tasks.workunit.client.0.smithi112.stdout:5/804: symlink d4/d4f/d78/l10b 0 2024-09-16T00:26:10.225 INFO:tasks.workunit.client.0.smithi112.stdout:8/773: truncate d10/d12/d1c/d96/fd6 561118 0 2024-09-16T00:26:10.226 INFO:tasks.workunit.client.0.smithi112.stdout:4/858: symlink d6/db/d1f/d41/d49/db1/l138 0 2024-09-16T00:26:10.226 INFO:tasks.workunit.client.0.smithi112.stdout:7/878: dread - d6/d10/d16/fcf zero size 2024-09-16T00:26:10.227 INFO:tasks.workunit.client.1.smithi169.stdout:9/710: rename d5/da/d5b/d6c/dcb/da4 to d5/da/df3/df8 0 2024-09-16T00:26:10.229 INFO:tasks.workunit.client.0.smithi112.stdout:5/805: rename d4/d7/d26/d36/c4b to d4/d7/c10c 0 2024-09-16T00:26:10.229 INFO:tasks.workunit.client.1.smithi169.stdout:9/711: symlink d5/da/d2e/lf9 0 2024-09-16T00:26:10.230 INFO:tasks.workunit.client.0.smithi112.stdout:4/859: creat d6/d39/d36/d5c/d7e/d8f/d11e/f139 x:0 0 0 2024-09-16T00:26:10.230 INFO:tasks.workunit.client.1.smithi169.stdout:9/712: chown d5/da/df3/df8/cd5 2 1 2024-09-16T00:26:10.231 INFO:tasks.workunit.client.1.smithi169.stdout:2/620: truncate d7/db/f13 643630 0 2024-09-16T00:26:10.231 INFO:tasks.workunit.client.0.smithi112.stdout:5/806: creat d4/dd/d49/d70/dc5/f10d x:0 0 0 2024-09-16T00:26:10.232 INFO:tasks.workunit.client.0.smithi112.stdout:5/807: readlink d4/d7/d26/l34 0 2024-09-16T00:26:10.232 INFO:tasks.workunit.client.1.smithi169.stdout:9/713: mknod d5/da/d5b/d91/d9a/dc3/cfa 0 2024-09-16T00:26:10.232 INFO:tasks.workunit.client.0.smithi112.stdout:7/879: creat d6/d10/d75/d8f/d90/f124 x:0 0 0 2024-09-16T00:26:10.233 INFO:tasks.workunit.client.0.smithi112.stdout:5/808: stat d4/dd/d12/d52/f63 0 2024-09-16T00:26:10.233 INFO:tasks.workunit.client.1.smithi169.stdout:9/714: chown d5/da/d5b/d6f/faa 2095209205 1 2024-09-16T00:26:10.234 INFO:tasks.workunit.client.0.smithi112.stdout:4/860: mkdir d6/d39/d1c/d6e/d13a 0 2024-09-16T00:26:10.234 INFO:tasks.workunit.client.0.smithi112.stdout:7/880: fsync d6/d10/d16/d96/dc7/fef 0 2024-09-16T00:26:10.234 INFO:tasks.workunit.client.1.smithi169.stdout:9/715: chown d5/da/d5b/d6c/da0/ld8 116 1 2024-09-16T00:26:10.236 INFO:tasks.workunit.client.0.smithi112.stdout:5/809: unlink d4/dd/d12/d52/f63 0 2024-09-16T00:26:10.238 INFO:tasks.workunit.client.1.smithi169.stdout:9/716: rename d5/da/d2e/lb5 to d5/da/d21/lfb 0 2024-09-16T00:26:10.238 INFO:tasks.workunit.client.0.smithi112.stdout:7/881: rename d6/d10/d16/d96/dc7/f7d to d6/d10/d16/d96/f125 0 2024-09-16T00:26:10.239 INFO:tasks.workunit.client.0.smithi112.stdout:4/861: mkdir d6/d39/d1c/d6e/d6f/d11b/d13b 0 2024-09-16T00:26:10.239 INFO:tasks.workunit.client.1.smithi169.stdout:9/717: write d5/da/d5b/d6f/fa1 [3423948,32673] 0 2024-09-16T00:26:10.240 INFO:tasks.workunit.client.0.smithi112.stdout:7/882: chown d6/d10/d75/d8f/d90/dbc/fc1 189919636 1 2024-09-16T00:26:10.241 INFO:tasks.workunit.client.0.smithi112.stdout:5/810: mkdir d4/d7/d26/d4c/d10e 0 2024-09-16T00:26:10.241 INFO:tasks.workunit.client.1.smithi169.stdout:9/718: unlink d5/da/d73/fb7 0 2024-09-16T00:26:10.243 INFO:tasks.workunit.client.1.smithi169.stdout:9/719: symlink d5/de0/lfc 0 2024-09-16T00:26:10.243 INFO:tasks.workunit.client.0.smithi112.stdout:7/883: chown d6/f9 0 1 2024-09-16T00:26:10.243 INFO:tasks.workunit.client.0.smithi112.stdout:4/862: dread d6/db/f93 [0,4194304] 0 2024-09-16T00:26:10.245 INFO:tasks.workunit.client.0.smithi112.stdout:5/811: creat d4/dd/f10f x:0 0 0 2024-09-16T00:26:10.245 INFO:tasks.workunit.client.0.smithi112.stdout:4/863: read d6/d39/d1c/d6e/d6f/fe5 [35580,56850] 0 2024-09-16T00:26:10.246 INFO:tasks.workunit.client.0.smithi112.stdout:7/884: fsync d6/d10/f6e 0 2024-09-16T00:26:10.247 INFO:tasks.workunit.client.0.smithi112.stdout:4/864: dread - d6/db/d1f/d41/d49/f115 zero size 2024-09-16T00:26:10.247 INFO:tasks.workunit.client.0.smithi112.stdout:5/812: write d4/d1c/d43/f54 [195998,466] 0 2024-09-16T00:26:10.248 INFO:tasks.workunit.client.1.smithi169.stdout:9/720: link d5/da/d21/d23/f94 d5/da/d5b/d6c/ffd 0 2024-09-16T00:26:10.249 INFO:tasks.workunit.client.0.smithi112.stdout:4/865: write d6/db/df/f103 [392293,130521] 0 2024-09-16T00:26:10.249 INFO:tasks.workunit.client.0.smithi112.stdout:7/885: creat d6/d7f/dc3/df8/f126 x:0 0 0 2024-09-16T00:26:10.251 INFO:tasks.workunit.client.1.smithi169.stdout:9/721: getdents d5/da/df3/df8 0 2024-09-16T00:26:10.252 INFO:tasks.workunit.client.0.smithi112.stdout:7/886: write d6/d10/d16/db4/fc4 [1231840,12175] 0 2024-09-16T00:26:10.253 INFO:tasks.workunit.client.1.smithi169.stdout:1/643: truncate dc/d36/d3a/d45/f17 1650688 0 2024-09-16T00:26:10.255 INFO:tasks.workunit.client.0.smithi112.stdout:2/977: sync 2024-09-16T00:26:10.255 INFO:tasks.workunit.client.0.smithi112.stdout:3/927: sync 2024-09-16T00:26:10.256 INFO:tasks.workunit.client.0.smithi112.stdout:5/813: rename d4/d7/d26/d4c/db3/db6/dd0/feb to d4/d4f/d77/dcc/d7c/d99/f110 0 2024-09-16T00:26:10.257 INFO:tasks.workunit.client.1.smithi169.stdout:1/644: stat dc/d36/d3a/d45/c22 0 2024-09-16T00:26:10.258 INFO:tasks.workunit.client.0.smithi112.stdout:2/978: chown d2/dd/d48/d22/d2b/d30/l42 3 1 2024-09-16T00:26:10.258 INFO:tasks.workunit.client.0.smithi112.stdout:4/866: symlink d6/db/df/d5e/d61/d64/l13c 0 2024-09-16T00:26:10.258 INFO:tasks.workunit.client.0.smithi112.stdout:4/867: readlink d6/db/d1f/d41/d49/db1/l138 0 2024-09-16T00:26:10.258 INFO:tasks.workunit.client.0.smithi112.stdout:7/887: chown d6/d10/da1/dcd/cd0 177243676 1 2024-09-16T00:26:10.259 INFO:tasks.workunit.client.1.smithi169.stdout:9/722: rename d5/da/d5b/d6f/c84 to d5/da/d21/de4/cfe 0 2024-09-16T00:26:10.259 INFO:tasks.workunit.client.0.smithi112.stdout:5/814: creat d4/d1c/f111 x:0 0 0 2024-09-16T00:26:10.260 INFO:tasks.workunit.client.0.smithi112.stdout:2/979: dread - d2/dd/d48/d22/d67/d9a/fcf zero size 2024-09-16T00:26:10.261 INFO:tasks.workunit.client.1.smithi169.stdout:1/645: rename la to dc/d36/dbb/lc1 0 2024-09-16T00:26:10.261 INFO:tasks.workunit.client.1.smithi169.stdout:1/646: stat dc/d36/d3a/d7f/d93/d72 0 2024-09-16T00:26:10.262 INFO:tasks.workunit.client.0.smithi112.stdout:4/868: write d6/d39/d36/d5c/d7e/f112 [604074,130102] 0 2024-09-16T00:26:10.263 INFO:tasks.workunit.client.1.smithi169.stdout:8/649: truncate d7/dc/d17/d24/f88 1244990 0 2024-09-16T00:26:10.264 INFO:tasks.workunit.client.0.smithi112.stdout:3/928: dwrite d2/d9/d2b/d47/d52/d7a/d80/d83/f112 [0,4194304] 0 2024-09-16T00:26:10.264 INFO:tasks.workunit.client.0.smithi112.stdout:5/815: rmdir d4/dd/d49/d70 39 2024-09-16T00:26:10.265 INFO:tasks.workunit.client.0.smithi112.stdout:2/980: write d2/dd/d48/d22/d2b/d54/d60/d6c/dc9/ff9 [869217,128974] 0 2024-09-16T00:26:10.265 INFO:tasks.workunit.client.1.smithi169.stdout:9/723: dread d5/d49/f5a [0,4194304] 0 2024-09-16T00:26:10.266 INFO:tasks.workunit.client.0.smithi112.stdout:5/816: stat d4/d4f/d78/lf3 0 2024-09-16T00:26:10.267 INFO:tasks.workunit.client.1.smithi169.stdout:9/724: chown d5/da/d5b/d6c/da0/c72 86332838 1 2024-09-16T00:26:10.267 INFO:tasks.workunit.client.0.smithi112.stdout:7/888: symlink d6/d10/d16/db4/de4/l127 0 2024-09-16T00:26:10.267 INFO:tasks.workunit.client.1.smithi169.stdout:1/647: mkdir dc/d36/d3a/d45/d29/dc2 0 2024-09-16T00:26:10.267 INFO:tasks.workunit.client.1.smithi169.stdout:9/725: stat d5/da/d5b/d6f/c8e 0 2024-09-16T00:26:10.268 INFO:tasks.workunit.client.1.smithi169.stdout:9/726: write d5/fb8 [1168430,77694] 0 2024-09-16T00:26:10.269 INFO:tasks.workunit.client.0.smithi112.stdout:2/981: readlink d2/dd/d48/d8a/lee 0 2024-09-16T00:26:10.269 INFO:tasks.workunit.client.1.smithi169.stdout:2/621: write d7/db/d27/d29/f54 [1631849,5634] 0 2024-09-16T00:26:10.269 INFO:tasks.workunit.client.0.smithi112.stdout:4/869: chown d6/db/df/d5e/d61/d64/d6a/ff9 3297983 1 2024-09-16T00:26:10.271 INFO:tasks.workunit.client.0.smithi112.stdout:8/774: write d10/d12/d1c/fa4 [418409,96550] 0 2024-09-16T00:26:10.271 INFO:tasks.workunit.client.0.smithi112.stdout:5/817: fsync d4/dd/d12/d52/f106 0 2024-09-16T00:26:10.272 INFO:tasks.workunit.client.1.smithi169.stdout:8/650: creat d7/dc/d17/fd0 x:0 0 0 2024-09-16T00:26:10.272 INFO:tasks.workunit.client.0.smithi112.stdout:2/982: rmdir d2/dd/d48/d22/d67 39 2024-09-16T00:26:10.273 INFO:tasks.workunit.client.1.smithi169.stdout:2/622: dread d7/d1c/fb7 [0,4194304] 0 2024-09-16T00:26:10.273 INFO:tasks.workunit.client.0.smithi112.stdout:2/983: read - d2/f132 zero size 2024-09-16T00:26:10.273 INFO:tasks.workunit.client.1.smithi169.stdout:2/623: read - d7/db/d89/db4/d69/d92/fd0 zero size 2024-09-16T00:26:10.274 INFO:tasks.workunit.client.0.smithi112.stdout:7/889: mknod d6/d10/d16/d96/c128 0 2024-09-16T00:26:10.274 INFO:tasks.workunit.client.1.smithi169.stdout:1/648: unlink dc/d36/d39/d50/daf/d88/fa9 0 2024-09-16T00:26:10.276 INFO:tasks.workunit.client.0.smithi112.stdout:4/870: mknod d6/d39/d36/c13d 0 2024-09-16T00:26:10.276 INFO:tasks.workunit.client.0.smithi112.stdout:8/775: write d10/d12/d1c/d61/d6c/dd1/fc9 [849297,38998] 0 2024-09-16T00:26:10.276 INFO:tasks.workunit.client.1.smithi169.stdout:9/727: symlink d5/da/d5b/d91/dc9/lff 0 2024-09-16T00:26:10.278 INFO:tasks.workunit.client.0.smithi112.stdout:5/818: truncate d4/d1c/d43/f6d 2280510 0 2024-09-16T00:26:10.279 INFO:tasks.workunit.client.1.smithi169.stdout:8/651: mknod d7/dc/d26/d41/d92/dcf/d80/cd1 0 2024-09-16T00:26:10.280 INFO:tasks.workunit.client.0.smithi112.stdout:5/819: write d4/f57 [4366262,102525] 0 2024-09-16T00:26:10.281 INFO:tasks.workunit.client.1.smithi169.stdout:1/649: dread dc/d18/d67/d69/fae [0,4194304] 0 2024-09-16T00:26:10.282 INFO:tasks.workunit.client.0.smithi112.stdout:4/871: mknod d6/d39/d1c/d6e/d6f/c13e 0 2024-09-16T00:26:10.282 INFO:tasks.workunit.client.1.smithi169.stdout:2/624: dwrite f0 [0,4194304] 0 2024-09-16T00:26:10.284 INFO:tasks.workunit.client.1.smithi169.stdout:9/728: truncate d5/da/d5b/d91/dc9/f53 1567216 0 2024-09-16T00:26:10.284 INFO:tasks.workunit.client.0.smithi112.stdout:3/929: truncate d2/d9/d2b/f9a 525151 0 2024-09-16T00:26:10.287 INFO:tasks.workunit.client.0.smithi112.stdout:8/776: rename d10/d12/d1c/dd0/fe7 to d10/d12/d1c/dd0/dbb/f101 0 2024-09-16T00:26:10.287 INFO:tasks.workunit.client.1.smithi169.stdout:9/729: truncate d5/da/d5b/d6f/faa 4733873 0 2024-09-16T00:26:10.287 INFO:tasks.workunit.client.1.smithi169.stdout:8/652: rename d7/dc/d17/c55 to d7/dc/d17/d48/dce/cd2 0 2024-09-16T00:26:10.288 INFO:tasks.workunit.client.0.smithi112.stdout:5/820: creat d4/d7/d14/f112 x:0 0 0 2024-09-16T00:26:10.291 INFO:tasks.workunit.client.1.smithi169.stdout:1/650: creat dc/d36/d3a/d45/d29/d4d/db6/fc3 x:0 0 0 2024-09-16T00:26:10.294 INFO:tasks.workunit.client.1.smithi169.stdout:9/730: mkdir d5/da/d2e/d99/d100 0 2024-09-16T00:26:10.294 INFO:tasks.workunit.client.0.smithi112.stdout:7/890: dread d6/fae [0,4194304] 0 2024-09-16T00:26:10.294 INFO:tasks.workunit.client.0.smithi112.stdout:3/930: mkdir d2/d9/d2b/d47/d52/d7a/d80/d83/d7b/dad/d107/d108/d12f 0 2024-09-16T00:26:10.295 INFO:tasks.workunit.client.0.smithi112.stdout:7/891: readlink d6/lc 0 2024-09-16T00:26:10.295 INFO:tasks.workunit.client.1.smithi169.stdout:9/731: chown d5/da/d21/de4 14816 1 2024-09-16T00:26:10.296 INFO:tasks.workunit.client.0.smithi112.stdout:3/931: write d2/d9/f128 [643140,108712] 0 2024-09-16T00:26:10.296 INFO:tasks.workunit.client.1.smithi169.stdout:9/732: write d5/d49/d64/fcd [1743529,51794] 0 2024-09-16T00:26:10.297 INFO:tasks.workunit.client.0.smithi112.stdout:5/821: mkdir d4/dd/d49/de6/d113 0 2024-09-16T00:26:10.297 INFO:tasks.workunit.client.1.smithi169.stdout:9/733: chown d5/da/d5b/d91/db3 86266980 1 2024-09-16T00:26:10.298 INFO:tasks.workunit.client.1.smithi169.stdout:2/625: link d7/db/d89/fbf d7/d6b/d70/d79/dce/fd8 0 2024-09-16T00:26:10.298 INFO:tasks.workunit.client.0.smithi112.stdout:2/984: link d2/dd/d48/d8a/d9b/fbd d2/dd/d48/d22/d2b/d54/d60/d6c/d88/dad/f140 0 2024-09-16T00:26:10.299 INFO:tasks.workunit.client.1.smithi169.stdout:8/653: symlink d7/dc/d53/da0/ld3 0 2024-09-16T00:26:10.300 INFO:tasks.workunit.client.0.smithi112.stdout:7/892: symlink d6/d10/d16/d96/dc7/dbf/l129 0 2024-09-16T00:26:10.300 INFO:tasks.workunit.client.1.smithi169.stdout:8/654: write d7/d37/fc2 [1776383,32579] 0 2024-09-16T00:26:10.304 INFO:tasks.workunit.client.0.smithi112.stdout:8/777: dwrite fe [0,4194304] 0 2024-09-16T00:26:10.305 INFO:tasks.workunit.client.1.smithi169.stdout:9/734: rename d5/da/d5b/d6c/f9d to d5/da/d5b/d91/db3/f101 0 2024-09-16T00:26:10.306 INFO:tasks.workunit.client.1.smithi169.stdout:9/735: read - d5/da/d5b/d91/dc9/fb9 zero size 2024-09-16T00:26:10.307 INFO:tasks.workunit.client.0.smithi112.stdout:4/872: link d6/db/df/d5e/d61/d64/d6a/db8/cd9 d6/d39/d36/d5c/d7e/c13f 0 2024-09-16T00:26:10.312 INFO:tasks.workunit.client.0.smithi112.stdout:3/932: dwrite d2/d9/d2b/d47/d52/d7a/d80/d83/d68/d90/f12a [0,4194304] 0 2024-09-16T00:26:10.312 INFO:tasks.workunit.client.1.smithi169.stdout:9/736: symlink d5/da/d5b/d91/l102 0 2024-09-16T00:26:10.314 INFO:tasks.workunit.client.1.smithi169.stdout:9/737: read d5/da/d5b/d6f/fa1 [2392914,91786] 0 2024-09-16T00:26:10.314 INFO:tasks.workunit.client.0.smithi112.stdout:2/985: symlink d2/dd/d48/d22/d2b/d54/d79/d110/d49/d100/d105/d11e/l141 0 2024-09-16T00:26:10.314 INFO:tasks.workunit.client.1.smithi169.stdout:9/738: readlink d5/da/d2e/lf9 0 2024-09-16T00:26:10.318 INFO:tasks.workunit.client.1.smithi169.stdout:8/655: dread d7/d20/f21 [0,4194304] 0 2024-09-16T00:26:10.319 INFO:tasks.workunit.client.0.smithi112.stdout:4/873: getdents d6/d39/d1c/d6e/d6f/d11b/daa 0 2024-09-16T00:26:10.321 INFO:tasks.workunit.client.0.smithi112.stdout:7/893: link d6/lf1 d6/d10/d75/d8f/d90/ddc/l12a 0 2024-09-16T00:26:10.321 INFO:tasks.workunit.client.1.smithi169.stdout:1/651: write dc/d36/d3a/d45/f1c [2534016,13770] 0 2024-09-16T00:26:10.322 INFO:tasks.workunit.client.0.smithi112.stdout:4/874: creat d6/d39/d36/d5c/d7e/d8f/f140 x:0 0 0 2024-09-16T00:26:10.322 INFO:tasks.workunit.client.1.smithi169.stdout:2/626: rmdir d7/d6b/d70/d79 39 2024-09-16T00:26:10.323 INFO:tasks.workunit.client.0.smithi112.stdout:5/822: getdents d4/dd/d49/d70/d83/dbd 0 2024-09-16T00:26:10.324 INFO:tasks.workunit.client.1.smithi169.stdout:1/652: symlink dc/d36/d3a/d7f/lc4 0 2024-09-16T00:26:10.324 INFO:tasks.workunit.client.0.smithi112.stdout:7/894: unlink d6/d10/d16/d96/dc7/c7e 0 2024-09-16T00:26:10.327 INFO:tasks.workunit.client.0.smithi112.stdout:2/986: creat d2/dd/d48/d22/d2b/d54/d60/f142 x:0 0 0 2024-09-16T00:26:10.329 INFO:tasks.workunit.client.1.smithi169.stdout:1/653: rename dc/d36/d3a/d45/d29/d2a/f62 to dc/d36/d39/d50/d75/fc5 0 2024-09-16T00:26:10.331 INFO:tasks.workunit.client.0.smithi112.stdout:7/895: readlink d6/d10/l84 0 2024-09-16T00:26:10.331 INFO:tasks.workunit.client.0.smithi112.stdout:5/823: dwrite d4/d4f/fb1 [0,4194304] 0 2024-09-16T00:26:10.332 INFO:tasks.workunit.client.1.smithi169.stdout:2/627: dwrite d7/d1c/f95 [0,4194304] 0 2024-09-16T00:26:10.333 INFO:tasks.workunit.client.0.smithi112.stdout:3/933: link d2/d9/d2b/d39/c9f d2/d9/c130 0 2024-09-16T00:26:10.334 INFO:tasks.workunit.client.0.smithi112.stdout:4/875: symlink d6/d39/d1c/d6e/d6f/d11b/d116/l141 0 2024-09-16T00:26:10.335 INFO:tasks.workunit.client.0.smithi112.stdout:8/778: truncate d10/f1b 2486781 0 2024-09-16T00:26:10.336 INFO:tasks.workunit.client.0.smithi112.stdout:2/987: dread d2/f11c [0,4194304] 0 2024-09-16T00:26:10.336 INFO:tasks.workunit.client.0.smithi112.stdout:8/779: readlink d10/d12/d24/l60 0 2024-09-16T00:26:10.337 INFO:tasks.workunit.client.1.smithi169.stdout:2/628: mknod d7/daa/cd9 0 2024-09-16T00:26:10.339 INFO:tasks.workunit.client.0.smithi112.stdout:7/896: symlink d6/d10/d75/d8f/d90/ddc/l12b 0 2024-09-16T00:26:10.340 INFO:tasks.workunit.client.0.smithi112.stdout:5/824: creat d4/d7/d26/d36/dcf/f114 x:0 0 0 2024-09-16T00:26:10.341 INFO:tasks.workunit.client.1.smithi169.stdout:8/656: dread d7/fd [0,4194304] 0 2024-09-16T00:26:10.341 INFO:tasks.workunit.client.1.smithi169.stdout:2/629: creat d7/db/d20/d21/fda x:0 0 0 2024-09-16T00:26:10.343 INFO:tasks.workunit.client.0.smithi112.stdout:2/988: chown d2/dd/f91 43517708 1 2024-09-16T00:26:10.344 INFO:tasks.workunit.client.1.smithi169.stdout:1/654: write dc/d36/d3a/d45/f17 [1807138,70930] 0 2024-09-16T00:26:10.344 INFO:tasks.workunit.client.0.smithi112.stdout:3/934: mkdir d2/d9/d2b/d47/d52/d7a/d80/db4/d131 0 2024-09-16T00:26:10.345 INFO:tasks.workunit.client.1.smithi169.stdout:2/630: creat d7/d1c/d3c/fdb x:0 0 0 2024-09-16T00:26:10.345 INFO:tasks.workunit.client.1.smithi169.stdout:8/657: mknod d7/d8c/cd4 0 2024-09-16T00:26:10.347 INFO:tasks.workunit.client.0.smithi112.stdout:3/935: write d2/d9/d2b/d47/d52/d7a/d80/d83/d76/d8e/ff0 [3604300,8112] 0 2024-09-16T00:26:10.349 INFO:tasks.workunit.client.0.smithi112.stdout:6/818: sync 2024-09-16T00:26:10.349 INFO:tasks.workunit.client.1.smithi169.stdout:8/658: rename d7/d20 to d7/d87/dd5 0 2024-09-16T00:26:10.350 INFO:tasks.workunit.client.0.smithi112.stdout:2/989: symlink d2/dd/d48/d22/d2b/d54/d60/d6c/d88/dad/deb/l143 0 2024-09-16T00:26:10.350 INFO:tasks.workunit.client.0.smithi112.stdout:7/897: getdents d6/d10/d16/d96/dc7/dbf/d11c 0 2024-09-16T00:26:10.351 INFO:tasks.workunit.client.1.smithi169.stdout:2/631: creat d7/d6b/d70/d79/fdc x:0 0 0 2024-09-16T00:26:10.351 INFO:tasks.workunit.client.0.smithi112.stdout:5/825: mknod d4/dd/d49/c115 0 2024-09-16T00:26:10.351 INFO:tasks.workunit.client.1.smithi169.stdout:2/632: readlink d7/l16 0 2024-09-16T00:26:10.352 INFO:tasks.workunit.client.1.smithi169.stdout:1/655: getdents dc/d36/dbb 0 2024-09-16T00:26:10.353 INFO:tasks.workunit.client.0.smithi112.stdout:3/936: chown d2/d9/d2b/f9a 13415421 1 2024-09-16T00:26:10.353 INFO:tasks.workunit.client.0.smithi112.stdout:5/826: chown d4/dd/c17 716905231 1 2024-09-16T00:26:10.354 INFO:tasks.workunit.client.1.smithi169.stdout:8/659: mkdir d7/dc/d53/da0/dac/dd6 0 2024-09-16T00:26:10.355 INFO:tasks.workunit.client.1.smithi169.stdout:8/660: chown d7/dc/d26/c98 3 1 2024-09-16T00:26:10.358 INFO:tasks.workunit.client.0.smithi112.stdout:5/827: creat d4/dd/d6f/f116 x:0 0 0 2024-09-16T00:26:10.359 INFO:tasks.workunit.client.1.smithi169.stdout:1/656: mkdir dc/d36/dbb/dc6 0 2024-09-16T00:26:10.359 INFO:tasks.workunit.client.1.smithi169.stdout:1/657: fsync f7 0 2024-09-16T00:26:10.359 INFO:tasks.workunit.client.1.smithi169.stdout:2/633: symlink d7/d1c/d26/d96/ldd 0 2024-09-16T00:26:10.359 INFO:tasks.workunit.client.1.smithi169.stdout:1/658: stat dc/d18/d67/d69/d7b/d92 0 2024-09-16T00:26:10.360 INFO:tasks.workunit.client.0.smithi112.stdout:5/828: read - d4/d7/d9b/fc6 zero size 2024-09-16T00:26:10.360 INFO:tasks.workunit.client.0.smithi112.stdout:5/829: stat d4/d4f/d77/dcc/d7c/lca 0 2024-09-16T00:26:10.360 INFO:tasks.workunit.client.0.smithi112.stdout:5/830: chown d4/d7/f40 24896 1 2024-09-16T00:26:10.360 INFO:tasks.workunit.client.0.smithi112.stdout:2/990: rename d2/dd/d48/d22/d2b/d54/d60/daf/fc1 to d2/dd/d48/d22/d2b/d54/d79/f144 0 2024-09-16T00:26:10.361 INFO:tasks.workunit.client.1.smithi169.stdout:2/634: mknod d7/d1c/d26/d96/cde 0 2024-09-16T00:26:10.362 INFO:tasks.workunit.client.0.smithi112.stdout:8/780: dread d10/f21 [0,4194304] 0 2024-09-16T00:26:10.362 INFO:tasks.workunit.client.0.smithi112.stdout:6/819: link dd/df/d10/lb1 dd/d15/da5/l10a 0 2024-09-16T00:26:10.362 INFO:tasks.workunit.client.1.smithi169.stdout:8/661: rename d7/dc/d26/d41/d92/dcf/d3c/d82/fb1 to d7/dc/d26/d41/d92/dcf/d3c/d82/fd7 0 2024-09-16T00:26:10.363 INFO:tasks.workunit.client.0.smithi112.stdout:3/937: truncate d2/d9/d2b/d47/d52/fa3 380294 0 2024-09-16T00:26:10.363 INFO:tasks.workunit.client.0.smithi112.stdout:4/876: write d6/f9 [7710277,112424] 0 2024-09-16T00:26:10.364 INFO:tasks.workunit.client.1.smithi169.stdout:8/662: truncate d7/d37/fc2 4844289 0 2024-09-16T00:26:10.364 INFO:tasks.workunit.client.1.smithi169.stdout:8/663: rename d7/dc/d26 to d7/dc/d26/dab/dd8 22 2024-09-16T00:26:10.365 INFO:tasks.workunit.client.1.smithi169.stdout:2/635: creat d7/db/d89/db4/d9d/dbe/fdf x:0 0 0 2024-09-16T00:26:10.365 INFO:tasks.workunit.client.1.smithi169.stdout:1/659: mkdir dc/d36/d3a/d45/d29/dc2/dc7 0 2024-09-16T00:26:10.366 INFO:tasks.workunit.client.1.smithi169.stdout:1/660: stat dc/d36/d3a/d7f/lc4 0 2024-09-16T00:26:10.367 INFO:tasks.workunit.client.1.smithi169.stdout:8/664: mknod d7/dc/d53/cd9 0 2024-09-16T00:26:10.367 INFO:tasks.workunit.client.1.smithi169.stdout:1/661: fdatasync dc/d26/f55 0 2024-09-16T00:26:10.367 INFO:tasks.workunit.client.0.smithi112.stdout:5/831: creat d4/d7/d14/d90/dd3/f117 x:0 0 0 2024-09-16T00:26:10.368 INFO:tasks.workunit.client.0.smithi112.stdout:7/898: write d6/d10/d75/d88/f70 [1018992,38815] 0 2024-09-16T00:26:10.368 INFO:tasks.workunit.client.0.smithi112.stdout:6/820: mknod dd/df/d10/d47/d55/dbd/dcb/de8/c10b 0 2024-09-16T00:26:10.369 INFO:tasks.workunit.client.1.smithi169.stdout:8/665: truncate d7/dc/d26/fa7 122635 0 2024-09-16T00:26:10.372 INFO:tasks.workunit.client.0.smithi112.stdout:4/877: creat d6/db/d1f/d41/ddf/f142 x:0 0 0 2024-09-16T00:26:10.373 INFO:tasks.workunit.client.0.smithi112.stdout:7/899: stat d6/d10/d16/c98 0 2024-09-16T00:26:10.373 INFO:tasks.workunit.client.1.smithi169.stdout:8/666: creat d7/dc/d17/d24/d32/fda x:0 0 0 2024-09-16T00:26:10.373 INFO:tasks.workunit.client.1.smithi169.stdout:1/662: dwrite dc/f1d [0,4194304] 0 2024-09-16T00:26:10.376 INFO:tasks.workunit.client.1.smithi169.stdout:1/663: creat dc/d18/d67/fc8 x:0 0 0 2024-09-16T00:26:10.376 INFO:tasks.workunit.client.0.smithi112.stdout:2/991: link d2/dd/d48/d22/d67/f74 d2/dd/d48/d22/d67/f145 0 2024-09-16T00:26:10.377 INFO:tasks.workunit.client.0.smithi112.stdout:8/781: dwrite d10/d12/d1c/d61/d6c/dd1/f68 [0,4194304] 0 2024-09-16T00:26:10.377 INFO:tasks.workunit.client.1.smithi169.stdout:1/664: rmdir dc/d36/d3a/d7f/d93/d72/d73 39 2024-09-16T00:26:10.378 INFO:tasks.workunit.client.1.smithi169.stdout:8/667: link d7/dc/d26/d41/d92/dcf/d3c/d82/db3/lbb d7/dc/d26/d41/d92/dcf/d3c/d77/ldb 0 2024-09-16T00:26:10.378 INFO:tasks.workunit.client.1.smithi169.stdout:1/665: fdatasync dc/d36/d3a/d45/d29/f8d 0 2024-09-16T00:26:10.379 INFO:tasks.workunit.client.1.smithi169.stdout:1/666: write dc/d36/d3a/d7f/d93/f6d [99451,32325] 0 2024-09-16T00:26:10.381 INFO:tasks.workunit.client.1.smithi169.stdout:8/668: mkdir d7/dc/d26/d41/d92/dcf/ddc 0 2024-09-16T00:26:10.382 INFO:tasks.workunit.client.1.smithi169.stdout:1/667: rename dc/d36/d3a/d7f/d93/d72/d98 to dc/d36/d3a/d7f/d93/d72/d98/dc9 22 2024-09-16T00:26:10.383 INFO:tasks.workunit.client.1.smithi169.stdout:1/668: write dc/d18/d67/fc8 [376859,105730] 0 2024-09-16T00:26:10.385 INFO:tasks.workunit.client.0.smithi112.stdout:6/821: mkdir dd/df/d11/d44/da4/d10c 0 2024-09-16T00:26:10.386 INFO:tasks.workunit.client.0.smithi112.stdout:4/878: rename d6/db/df/d5e/d61/ld6 to d6/db/df/d3c/l143 0 2024-09-16T00:26:10.389 INFO:tasks.workunit.client.0.smithi112.stdout:2/992: dwrite d2/dd/d48/d22/d2b/d54/d60/d6c/d88/dad/ff0 [0,4194304] 0 2024-09-16T00:26:10.389 INFO:tasks.workunit.client.0.smithi112.stdout:3/938: link d2/d9/d2b/d47/d52/d7a/d80/d83/d7b/dad/d107/d69/dd5/d105/c113 d2/d9/d2b/c132 0 2024-09-16T00:26:10.390 INFO:tasks.workunit.client.0.smithi112.stdout:2/993: stat d2/c7c 0 2024-09-16T00:26:10.390 INFO:tasks.workunit.client.0.smithi112.stdout:8/782: symlink d10/d12/d24/l102 0 2024-09-16T00:26:10.391 INFO:tasks.workunit.client.1.smithi169.stdout:8/669: dwrite d7/dc/d26/d41/d4e/d81/d9e/da5/fa6 [0,4194304] 0 2024-09-16T00:26:10.391 INFO:tasks.workunit.client.0.smithi112.stdout:6/822: write dd/d23/f35 [490836,89076] 0 2024-09-16T00:26:10.392 INFO:tasks.workunit.client.0.smithi112.stdout:5/832: getdents d4/d4f/d78 0 2024-09-16T00:26:10.394 INFO:tasks.workunit.client.0.smithi112.stdout:4/879: unlink d6/db/df/d5e/f78 0 2024-09-16T00:26:10.394 INFO:tasks.workunit.client.1.smithi169.stdout:1/669: dwrite dc/d26/f55 [0,4194304] 0 2024-09-16T00:26:10.394 INFO:tasks.workunit.client.1.smithi169.stdout:8/670: fsync d7/dc/d17/f89 0 2024-09-16T00:26:10.396 INFO:tasks.workunit.client.0.smithi112.stdout:4/880: fsync d6/d39/d36/d5c/d7e/d8f/f90 0 2024-09-16T00:26:10.398 INFO:tasks.workunit.client.1.smithi169.stdout:1/670: rename dc/c77 to dc/d36/d39/d50/daf/d88/da5/cca 0 2024-09-16T00:26:10.399 INFO:tasks.workunit.client.1.smithi169.stdout:1/671: stat dc/d36/d3a/d45/d29/f52 0 2024-09-16T00:26:10.404 INFO:tasks.workunit.client.0.smithi112.stdout:2/994: fdatasync d2/dd/d48/d22/d2b/d30/f97 0 2024-09-16T00:26:10.410 INFO:tasks.workunit.client.0.smithi112.stdout:6/823: creat dd/df/d13/f10d x:0 0 0 2024-09-16T00:26:10.414 INFO:tasks.workunit.client.0.smithi112.stdout:8/783: creat d10/d12/d1c/d61/d6c/dd1/d58/f103 x:0 0 0 2024-09-16T00:26:10.414 INFO:tasks.workunit.client.0.smithi112.stdout:6/824: mknod dd/d15/da5/dae/c10e 0 2024-09-16T00:26:10.416 INFO:tasks.workunit.client.0.smithi112.stdout:4/881: creat d6/db/d1f/d41/f144 x:0 0 0 2024-09-16T00:26:10.420 INFO:tasks.workunit.client.0.smithi112.stdout:3/939: dwrite d2/fb [4194304,4194304] 0 2024-09-16T00:26:10.422 INFO:tasks.workunit.client.0.smithi112.stdout:4/882: truncate d6/d39/d1c/d2c/d69/ff5 908208 0 2024-09-16T00:26:10.423 INFO:tasks.workunit.client.0.smithi112.stdout:3/940: readlink d2/d9/d2b/d47/d52/d7a/d80/db4/le2 0 2024-09-16T00:26:10.425 INFO:tasks.workunit.client.0.smithi112.stdout:3/941: readlink d2/d9/d2b/d47/d52/d7a/d80/d83/d7b/dad/l11a 0 2024-09-16T00:26:10.426 INFO:tasks.workunit.client.0.smithi112.stdout:4/883: mknod d6/db/d1f/d41/d49/c145 0 2024-09-16T00:26:10.427 INFO:tasks.workunit.client.0.smithi112.stdout:2/995: dread d2/f8d [0,4194304] 0 2024-09-16T00:26:10.427 INFO:tasks.workunit.client.0.smithi112.stdout:8/784: dwrite d10/d12/f77 [0,4194304] 0 2024-09-16T00:26:10.430 INFO:tasks.workunit.client.0.smithi112.stdout:6/825: truncate dd/df/d10/d47/d55/f7a 1229591 0 2024-09-16T00:26:10.430 INFO:tasks.workunit.client.0.smithi112.stdout:8/785: mkdir d10/de9/def/d104 0 2024-09-16T00:26:10.431 INFO:tasks.workunit.client.0.smithi112.stdout:4/884: write d6/db/d1f/d41/ddf/f104 [702055,24196] 0 2024-09-16T00:26:10.431 INFO:tasks.workunit.client.0.smithi112.stdout:5/833: dread d4/f19 [0,4194304] 0 2024-09-16T00:26:10.436 INFO:tasks.workunit.client.0.smithi112.stdout:4/885: unlink d6/d39/d1c/f57 0 2024-09-16T00:26:10.438 INFO:tasks.workunit.client.0.smithi112.stdout:5/834: creat d4/d1c/d43/f118 x:0 0 0 2024-09-16T00:26:10.441 INFO:tasks.workunit.client.0.smithi112.stdout:5/835: fsync d4/d4f/d77/dcc/d7c/fa3 0 2024-09-16T00:26:10.441 INFO:tasks.workunit.client.0.smithi112.stdout:8/786: dread d10/d12/d1c/d61/d6c/dd1/f72 [0,4194304] 0 2024-09-16T00:26:10.441 INFO:tasks.workunit.client.0.smithi112.stdout:6/826: getdents dd/df/d91/dd7 0 2024-09-16T00:26:10.442 INFO:tasks.workunit.client.1.smithi169.stdout:8/671: write d7/dc/d17/d24/d32/d8d/fc8 [1910922,2241] 0 2024-09-16T00:26:10.442 INFO:tasks.workunit.client.1.smithi169.stdout:1/672: write dc/d36/d3a/f9f [262357,84538] 0 2024-09-16T00:26:10.444 INFO:tasks.workunit.client.0.smithi112.stdout:5/836: truncate d4/dd/d49/d70/dc5/f10d 1010685 0 2024-09-16T00:26:10.445 INFO:tasks.workunit.client.0.smithi112.stdout:6/827: mknod dd/df/d17/d27/d54/c10f 0 2024-09-16T00:26:10.445 INFO:tasks.workunit.client.1.smithi169.stdout:8/672: creat d7/dc/d17/d48/dce/fdd x:0 0 0 2024-09-16T00:26:10.446 INFO:tasks.workunit.client.0.smithi112.stdout:5/837: rename d4/d7/d14/d90/dd3/de5 to d4/d7/d14/d90/dd3/de5/d119 22 2024-09-16T00:26:10.448 INFO:tasks.workunit.client.1.smithi169.stdout:8/673: getdents d7/dc 0 2024-09-16T00:26:10.451 INFO:tasks.workunit.client.1.smithi169.stdout:8/674: mkdir d7/dc/d26/d41/d92/dcf/d3c/d77/dde 0 2024-09-16T00:26:10.451 INFO:tasks.workunit.client.0.smithi112.stdout:5/838: creat d4/d4f/d77/dcc/f11a x:0 0 0 2024-09-16T00:26:10.452 INFO:tasks.workunit.client.1.smithi169.stdout:8/675: chown d7/dc/d26/d46/c58 48 1 2024-09-16T00:26:10.453 INFO:tasks.workunit.client.0.smithi112.stdout:6/828: dwrite dd/df/d10/f8b [4194304,4194304] 0 2024-09-16T00:26:10.454 INFO:tasks.workunit.client.1.smithi169.stdout:8/676: creat d7/dc/d26/d41/d92/dcf/d3c/d82/fdf x:0 0 0 2024-09-16T00:26:10.455 INFO:tasks.workunit.client.0.smithi112.stdout:8/787: write d10/d12/d1c/d61/d6c/dd1/d8c/f9a [456204,47208] 0 2024-09-16T00:26:10.457 INFO:tasks.workunit.client.0.smithi112.stdout:5/839: mknod d4/d7/d26/d36/dcf/c11b 0 2024-09-16T00:26:10.458 INFO:tasks.workunit.client.0.smithi112.stdout:6/829: creat dd/df/d91/f110 x:0 0 0 2024-09-16T00:26:10.460 INFO:tasks.workunit.client.0.smithi112.stdout:6/830: fdatasync dd/df/d10/d47/d55/dbd/dcb/fff 0 2024-09-16T00:26:10.460 INFO:tasks.workunit.client.0.smithi112.stdout:8/788: fsync d10/d12/d1c/d61/d6c/dd1/d8c/f85 0 2024-09-16T00:26:10.461 INFO:tasks.workunit.client.0.smithi112.stdout:6/831: chown dd 23106 1 2024-09-16T00:26:10.461 INFO:tasks.workunit.client.1.smithi169.stdout:1/673: dwrite dc/d36/d3a/d45/d29/d2a/f38 [0,4194304] 0 2024-09-16T00:26:10.462 INFO:tasks.workunit.client.0.smithi112.stdout:4/886: dread d6/d39/d1c/d66/f76 [0,4194304] 0 2024-09-16T00:26:10.466 INFO:tasks.workunit.client.0.smithi112.stdout:4/887: mknod d6/db/d1f/d94/d10c/d123/c146 0 2024-09-16T00:26:10.466 INFO:tasks.workunit.client.0.smithi112.stdout:8/789: creat d10/d12/d1c/d61/d6c/dd1/d58/f105 x:0 0 0 2024-09-16T00:26:10.467 INFO:tasks.workunit.client.1.smithi169.stdout:8/677: write d7/f10 [4865981,118677] 0 2024-09-16T00:26:10.468 INFO:tasks.workunit.client.0.smithi112.stdout:8/790: truncate d10/d12/fd3 1040605 0 2024-09-16T00:26:10.468 INFO:tasks.workunit.client.1.smithi169.stdout:8/678: write d7/dc/d26/fbc [1260471,122580] 0 2024-09-16T00:26:10.471 INFO:tasks.workunit.client.0.smithi112.stdout:4/888: getdents d6/d39/d36/d5c/dc6 0 2024-09-16T00:26:10.472 INFO:tasks.workunit.client.1.smithi169.stdout:8/679: write d7/d87/dd5/f7d [1184865,10892] 0 2024-09-16T00:26:10.472 INFO:tasks.workunit.client.1.smithi169.stdout:1/674: dwrite dc/f15 [0,4194304] 0 2024-09-16T00:26:10.473 INFO:tasks.workunit.client.1.smithi169.stdout:8/680: readlink d7/dc/d17/d24/d32/d4d/l84 0 2024-09-16T00:26:10.473 INFO:tasks.workunit.client.0.smithi112.stdout:6/832: write dd/df/d11/fc8 [402862,62721] 0 2024-09-16T00:26:10.479 INFO:tasks.workunit.client.0.smithi112.stdout:6/833: mkdir dd/df/d10/d47/d89/d111 0 2024-09-16T00:26:10.479 INFO:tasks.workunit.client.1.smithi169.stdout:8/681: dwrite d7/dc/d2a/fba [0,4194304] 0 2024-09-16T00:26:10.480 INFO:tasks.workunit.client.1.smithi169.stdout:8/682: fsync d7/dc/d17/f89 0 2024-09-16T00:26:10.481 INFO:tasks.workunit.client.0.smithi112.stdout:6/834: chown dd/df/d13/c90 2 1 2024-09-16T00:26:10.482 INFO:tasks.workunit.client.1.smithi169.stdout:8/683: chown d7/dc/d26/f9d 1 1 2024-09-16T00:26:10.484 INFO:tasks.workunit.client.0.smithi112.stdout:6/835: truncate dd/d15/da5/d79/f8d 1096908 0 2024-09-16T00:26:10.485 INFO:tasks.workunit.client.1.smithi169.stdout:8/684: write d7/fd [2326163,94438] 0 2024-09-16T00:26:10.487 INFO:tasks.workunit.client.0.smithi112.stdout:6/836: mkdir dd/df/d11/d44/da4/d10c/d112 0 2024-09-16T00:26:10.489 INFO:tasks.workunit.client.1.smithi169.stdout:8/685: mknod d7/dc/d26/d41/d4e/d81/ce0 0 2024-09-16T00:26:10.490 INFO:tasks.workunit.client.0.smithi112.stdout:6/837: getdents dd/df/d10/d47/d89/dbc 0 2024-09-16T00:26:10.490 INFO:tasks.workunit.client.0.smithi112.stdout:6/838: dread - dd/df/d13/d18/fd3 zero size 2024-09-16T00:26:10.491 INFO:tasks.workunit.client.1.smithi169.stdout:8/686: chown d7/dc/d26/d41/d92/dcf/c1f 13711889 1 2024-09-16T00:26:10.491 INFO:tasks.workunit.client.0.smithi112.stdout:6/839: fsync dd/df/d13/f48 0 2024-09-16T00:26:10.492 INFO:tasks.workunit.client.0.smithi112.stdout:6/840: write dd/df/d13/fc1 [2480259,122718] 0 2024-09-16T00:26:10.492 INFO:tasks.workunit.client.1.smithi169.stdout:0/658: sync 2024-09-16T00:26:10.494 INFO:tasks.workunit.client.1.smithi169.stdout:8/687: symlink d7/dc/d26/d41/d92/dcf/d3c/d77/dde/le1 0 2024-09-16T00:26:10.494 INFO:tasks.workunit.client.1.smithi169.stdout:0/659: dread - d0/d2c/d54/d21/d31/d83/dac/fb8 zero size 2024-09-16T00:26:10.495 INFO:tasks.workunit.client.0.smithi112.stdout:6/841: unlink dd/df/d17/d27/d54/fec 0 2024-09-16T00:26:10.496 INFO:tasks.workunit.client.1.smithi169.stdout:8/688: readlink d7/dc/d26/d41/d92/dcf/d3c/d82/lc4 0 2024-09-16T00:26:10.497 INFO:tasks.workunit.client.0.smithi112.stdout:6/842: mknod dd/df/d13/c113 0 2024-09-16T00:26:10.497 INFO:tasks.workunit.client.1.smithi169.stdout:0/660: write d0/d2c/d54/f28 [5181158,107822] 0 2024-09-16T00:26:10.498 INFO:tasks.workunit.client.1.smithi169.stdout:8/689: unlink d7/d8c/cd4 0 2024-09-16T00:26:10.498 INFO:tasks.workunit.client.1.smithi169.stdout:0/661: chown d0/d2c/d54/d21/c30 34 1 2024-09-16T00:26:10.498 INFO:tasks.workunit.client.0.smithi112.stdout:6/843: mknod dd/df/d11/d44/d8a/dfb/c114 0 2024-09-16T00:26:10.500 INFO:tasks.workunit.client.1.smithi169.stdout:0/662: creat d0/d2/d15/fe3 x:0 0 0 2024-09-16T00:26:10.503 INFO:tasks.workunit.client.1.smithi169.stdout:8/690: creat d7/dc/d26/d41/d92/dcf/d80/dbd/fe2 x:0 0 0 2024-09-16T00:26:10.504 INFO:tasks.workunit.client.0.smithi112.stdout:4/889: dread d6/d39/d1c/d2c/d69/d7b/f11a [0,4194304] 0 2024-09-16T00:26:10.505 INFO:tasks.workunit.client.0.smithi112.stdout:4/890: chown d6/db/l105 192 1 2024-09-16T00:26:10.506 INFO:tasks.workunit.client.1.smithi169.stdout:0/663: rename d0/d2c/d54 to d0/d2c/d38/de4 0 2024-09-16T00:26:10.507 INFO:tasks.workunit.client.0.smithi112.stdout:4/891: mkdir d6/db/d1f/d94/d10c/d147 0 2024-09-16T00:26:10.508 INFO:tasks.workunit.client.0.smithi112.stdout:4/892: fdatasync d6/d39/d36/d5c/ffa 0 2024-09-16T00:26:10.509 INFO:tasks.workunit.client.1.smithi169.stdout:0/664: stat d0/d2/d8/l9 0 2024-09-16T00:26:10.511 INFO:tasks.workunit.client.1.smithi169.stdout:0/665: read d0/d2/d8/f47 [4096795,82215] 0 2024-09-16T00:26:10.511 INFO:tasks.workunit.client.0.smithi112.stdout:6/844: write dd/d15/da5/d79/f8d [495456,71928] 0 2024-09-16T00:26:10.512 INFO:tasks.workunit.client.1.smithi169.stdout:0/666: readlink d0/d2c/d4f/lb2 0 2024-09-16T00:26:10.513 INFO:tasks.workunit.client.0.smithi112.stdout:6/845: chown dd/dcc/de6 5562554 1 2024-09-16T00:26:10.513 INFO:tasks.workunit.client.1.smithi169.stdout:8/691: getdents d7/dc/d26/d41/d92/dcf 0 2024-09-16T00:26:10.514 INFO:tasks.workunit.client.0.smithi112.stdout:4/893: dwrite d6/db/d1f/d41/fb2 [0,4194304] 0 2024-09-16T00:26:10.515 INFO:tasks.workunit.client.1.smithi169.stdout:8/692: write d7/dc/d2a/f30 [3596461,49990] 0 2024-09-16T00:26:10.515 INFO:tasks.workunit.client.1.smithi169.stdout:0/667: unlink d0/lc0 0 2024-09-16T00:26:10.516 INFO:tasks.workunit.client.1.smithi169.stdout:0/668: write d0/fd0 [510156,25158] 0 2024-09-16T00:26:10.518 INFO:tasks.workunit.client.1.smithi169.stdout:8/693: read d7/dc/d2a/f30 [934549,87693] 0 2024-09-16T00:26:10.519 INFO:tasks.workunit.client.1.smithi169.stdout:0/669: creat d0/d2/d4b/fe5 x:0 0 0 2024-09-16T00:26:10.527 INFO:tasks.workunit.client.0.smithi112.stdout:6/846: dread dd/df/d10/d47/d55/f7d [0,4194304] 0 2024-09-16T00:26:10.528 INFO:tasks.workunit.client.0.smithi112.stdout:6/847: chown dd/df/d11/d44/d8a/dfb 1535068 1 2024-09-16T00:26:10.533 INFO:tasks.workunit.client.0.smithi112.stdout:6/848: dwrite dd/df/d11/d44/d8a/fe7 [0,4194304] 0 2024-09-16T00:26:10.534 INFO:tasks.workunit.client.0.smithi112.stdout:6/849: stat dd/df/d17/d27/f76 0 2024-09-16T00:26:10.538 INFO:tasks.workunit.client.0.smithi112.stdout:1/898: sync 2024-09-16T00:26:10.539 INFO:tasks.workunit.client.0.smithi112.stdout:1/899: dread - d0/d14/d5e/f11d zero size 2024-09-16T00:26:10.541 INFO:tasks.workunit.client.0.smithi112.stdout:1/900: fsync d0/d14/df/d28/d45/dd4/f76 0 2024-09-16T00:26:10.541 INFO:tasks.workunit.client.1.smithi169.stdout:0/670: dread d0/d4c/d6f/f91 [0,4194304] 0 2024-09-16T00:26:10.544 INFO:tasks.workunit.client.1.smithi169.stdout:0/671: creat d0/d2c/d4f/fe6 x:0 0 0 2024-09-16T00:26:10.546 INFO:tasks.workunit.client.1.smithi169.stdout:0/672: creat d0/d2c/d38/de4/fe7 x:0 0 0 2024-09-16T00:26:10.547 INFO:tasks.workunit.client.0.smithi112.stdout:1/901: dwrite d0/d14/df/d28/d45/d30/f4a [0,4194304] 0 2024-09-16T00:26:10.550 INFO:tasks.workunit.client.0.smithi112.stdout:1/902: mkdir d0/d14/df/d28/d45/d103/d127 0 2024-09-16T00:26:10.552 INFO:tasks.workunit.client.1.smithi169.stdout:0/673: rename d0/d2/d15/l81 to d0/d4c/le8 0 2024-09-16T00:26:10.554 INFO:tasks.workunit.client.0.smithi112.stdout:1/903: rename d0/d14/d44 to d0/d14/df/d28/d45/d46/d58/d128 0 2024-09-16T00:26:10.555 INFO:tasks.workunit.client.0.smithi112.stdout:1/904: chown d0/d14/d5e/df8/c106 30 1 2024-09-16T00:26:10.557 INFO:tasks.workunit.client.0.smithi112.stdout:1/905: truncate d0/d14/df/d28/d45/dd4/fb9 4228922 0 2024-09-16T00:26:10.559 INFO:tasks.workunit.client.0.smithi112.stdout:1/906: write d0/d14/df/d28/d45/d46/d58/d62/f74 [1170108,123635] 0 2024-09-16T00:26:10.562 INFO:tasks.workunit.client.0.smithi112.stdout:1/907: symlink d0/d14/df/d28/d45/d46/d58/d128/l129 0 2024-09-16T00:26:10.566 INFO:tasks.workunit.client.0.smithi112.stdout:1/908: rename d0/d14/d5e/d120 to d0/d14/df/d28/d45/d46/d58/d12a 0 2024-09-16T00:26:10.572 INFO:tasks.workunit.client.0.smithi112.stdout:6/850: write dd/df/d17/d27/d84/f36 [4586414,32543] 0 2024-09-16T00:26:10.572 INFO:tasks.workunit.client.1.smithi169.stdout:0/674: dwrite d0/d2c/d38/f84 [0,4194304] 0 2024-09-16T00:26:10.577 INFO:tasks.workunit.client.0.smithi112.stdout:4/894: dwrite d6/db/d1f/d41/d49/fca [0,4194304] 0 2024-09-16T00:26:10.579 INFO:tasks.workunit.client.0.smithi112.stdout:6/851: link cb dd/df/c115 0 2024-09-16T00:26:10.581 INFO:tasks.workunit.client.0.smithi112.stdout:6/852: fsync dd/df/d11/f80 0 2024-09-16T00:26:10.586 INFO:tasks.workunit.client.0.smithi112.stdout:4/895: dwrite d6/d39/d1c/d6e/fbe [4194304,4194304] 0 2024-09-16T00:26:10.588 INFO:tasks.workunit.client.0.smithi112.stdout:6/853: dread dd/df/d17/d27/d54/f6a [0,4194304] 0 2024-09-16T00:26:10.588 INFO:tasks.workunit.client.0.smithi112.stdout:4/896: getdents d6/d39/d1c/d66/d12d 0 2024-09-16T00:26:10.592 INFO:tasks.workunit.client.0.smithi112.stdout:4/897: mkdir d6/db/d1f/d148 0 2024-09-16T00:26:10.592 INFO:tasks.workunit.client.0.smithi112.stdout:6/854: fdatasync dd/df/d10/d47/d55/f72 0 2024-09-16T00:26:10.594 INFO:tasks.workunit.client.0.smithi112.stdout:6/855: chown dd/df/d11/f1e 0 1 2024-09-16T00:26:10.595 INFO:tasks.workunit.client.0.smithi112.stdout:6/856: chown dd/d15/da5/d79 0 1 2024-09-16T00:26:10.595 INFO:tasks.workunit.client.0.smithi112.stdout:4/898: creat d6/db/f149 x:0 0 0 2024-09-16T00:26:10.596 INFO:tasks.workunit.client.0.smithi112.stdout:4/899: symlink d6/d39/d1c/d6e/d119/l14a 0 2024-09-16T00:26:10.600 INFO:tasks.workunit.client.0.smithi112.stdout:4/900: getdents d6/d39/d36/d5c/d7e 0 2024-09-16T00:26:10.601 INFO:tasks.workunit.client.0.smithi112.stdout:4/901: fsync d6/d39/d36/d5c/d7e/d8f/d11e/f139 0 2024-09-16T00:26:10.601 INFO:tasks.workunit.client.0.smithi112.stdout:4/902: readlink d6/db/df/d5e/l133 0 2024-09-16T00:26:10.603 INFO:tasks.workunit.client.0.smithi112.stdout:4/903: mknod d6/d39/d36/d5c/dc6/db5/c14b 0 2024-09-16T00:26:10.603 INFO:tasks.workunit.client.0.smithi112.stdout:4/904: chown d6/db/f60 0 1 2024-09-16T00:26:10.605 INFO:tasks.workunit.client.0.smithi112.stdout:4/905: creat d6/db/df/d5e/d61/d64/d6a/f14c x:0 0 0 2024-09-16T00:26:10.606 INFO:tasks.workunit.client.0.smithi112.stdout:4/906: chown d6/db/d1f/d94/d10c 3744468 1 2024-09-16T00:26:10.606 INFO:tasks.workunit.client.0.smithi112.stdout:4/907: chown d6/db/df/d5e/d61/d64/d6a/db8/l110 1889796 1 2024-09-16T00:26:10.607 INFO:tasks.workunit.client.0.smithi112.stdout:4/908: write d6/d39/d36/d5c/d7e/d8f/d11e/f139 [845966,6506] 0 2024-09-16T00:26:10.609 INFO:tasks.workunit.client.0.smithi112.stdout:4/909: fdatasync d6/d39/d36/f72 0 2024-09-16T00:26:10.610 INFO:tasks.workunit.client.0.smithi112.stdout:4/910: chown d6/d39/d36/d5c/d7e/dd1/ld5 27316739 1 2024-09-16T00:26:10.611 INFO:tasks.workunit.client.0.smithi112.stdout:4/911: getdents d6/d39/d1c/d66/d12d 0 2024-09-16T00:26:10.612 INFO:tasks.workunit.client.0.smithi112.stdout:4/912: write d6/d39/d1c/d2c/d69/f10a [3823858,122404] 0 2024-09-16T00:26:10.617 INFO:tasks.workunit.client.0.smithi112.stdout:4/913: dwrite d6/d39/d1c/d66/fac [4194304,4194304] 0 2024-09-16T00:26:10.622 INFO:tasks.workunit.client.0.smithi112.stdout:4/914: getdents d6/db/df/d5e 0 2024-09-16T00:26:10.623 INFO:tasks.workunit.client.0.smithi112.stdout:4/915: readlink d6/db/df/d3c/l143 0 2024-09-16T00:26:10.629 INFO:tasks.workunit.client.0.smithi112.stdout:4/916: dwrite d6/d39/d1c/d6e/d6f/d11b/f7f [0,4194304] 0 2024-09-16T00:26:10.634 INFO:tasks.workunit.client.0.smithi112.stdout:4/917: rmdir d6/db/d1f/d94/d10c/d147 0 2024-09-16T00:26:10.635 INFO:tasks.workunit.client.0.smithi112.stdout:4/918: mkdir d6/d39/d1c/d2c/d69/d7b/de0/d14d 0 2024-09-16T00:26:10.638 INFO:tasks.workunit.client.0.smithi112.stdout:4/919: rename d6/d39/d36/d5c/d7e/f95 to d6/d39/d36/d5c/dc6/db5/f14e 0 2024-09-16T00:26:10.642 INFO:tasks.workunit.client.0.smithi112.stdout:4/920: rename d6/db/d1f/d41/ddf/f142 to d6/d39/d36/d5c/d7e/f14f 0 2024-09-16T00:26:10.643 INFO:tasks.workunit.client.0.smithi112.stdout:4/921: creat d6/d39/d1c/d2c/d69/d7b/de0/f150 x:0 0 0 2024-09-16T00:26:10.645 INFO:tasks.workunit.client.0.smithi112.stdout:4/922: symlink d6/d39/d1c/d66/d12d/l151 0 2024-09-16T00:26:10.646 INFO:tasks.workunit.client.0.smithi112.stdout:4/923: chown d6/d39/d36/c13d 23598 1 2024-09-16T00:26:10.647 INFO:tasks.workunit.client.0.smithi112.stdout:4/924: mkdir d6/da/d152 0 2024-09-16T00:26:10.648 INFO:tasks.workunit.client.0.smithi112.stdout:4/925: chown d6/db/f60 945 1 2024-09-16T00:26:10.650 INFO:tasks.workunit.client.0.smithi112.stdout:4/926: unlink d6/f1b 0 2024-09-16T00:26:10.653 INFO:tasks.workunit.client.0.smithi112.stdout:4/927: link d6/d39/d1c/d2c/d69/f10a d6/d39/f153 0 2024-09-16T00:26:10.655 INFO:tasks.workunit.client.0.smithi112.stdout:4/928: creat d6/d39/d36/d5c/d7e/d8f/d11e/f154 x:0 0 0 2024-09-16T00:26:10.657 INFO:tasks.workunit.client.0.smithi112.stdout:4/929: creat d6/d39/d1c/d6e/d13a/f155 x:0 0 0 2024-09-16T00:26:10.736 INFO:tasks.workunit.client.0.smithi112.stdout:7/900: sync 2024-09-16T00:26:10.747 INFO:tasks.workunit.client.0.smithi112.stdout:7/901: dread d6/d10/d75/d8f/d90/dbc/fc1 [0,4194304] 0 2024-09-16T00:26:10.748 INFO:tasks.workunit.client.0.smithi112.stdout:7/902: write d6/d10/d75/fb9 [5172663,73913] 0 2024-09-16T00:26:10.755 INFO:tasks.workunit.client.0.smithi112.stdout:7/903: dread d6/d7f/d85/fb5 [0,4194304] 0 2024-09-16T00:26:10.757 INFO:tasks.workunit.client.0.smithi112.stdout:7/904: getdents d6/d10/d16/d96/dc7/dbf/d108 0 2024-09-16T00:26:10.760 INFO:tasks.workunit.client.0.smithi112.stdout:7/905: link d6/d10/d16/f66 d6/d10/d16/d96/dc7/dbf/f12c 0 2024-09-16T00:26:10.761 INFO:tasks.workunit.client.0.smithi112.stdout:7/906: write d6/d10/d16/d3b/f4b [1291795,110702] 0 2024-09-16T00:26:10.762 INFO:tasks.workunit.client.0.smithi112.stdout:7/907: chown d6/d10/d16/d96/f9f 68 1 2024-09-16T00:26:10.766 INFO:tasks.workunit.client.0.smithi112.stdout:7/908: rename d6/d10/d16/d96/dc7/f47 to d6/d7f/dc3/f12d 0 2024-09-16T00:26:10.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:10 smithi169 ceph-mon[34653]: pgmap v258: 65 pgs: 65 active+clean; 2.8 GiB data, 9.7 GiB used, 527 GiB / 536 GiB avail; 123 MiB/s rd, 368 MiB/s wr, 701 op/s 2024-09-16T00:26:10.781 INFO:tasks.workunit.client.0.smithi112.stdout:7/909: dread d6/d10/d16/f1e [0,4194304] 0 2024-09-16T00:26:10.782 INFO:tasks.workunit.client.0.smithi112.stdout:7/910: stat d6/d10/c50 0 2024-09-16T00:26:10.786 INFO:tasks.workunit.client.0.smithi112.stdout:7/911: rename d6/d10/d75/d8f/d90/dd3/ff4 to d6/d10/d75/d8f/d90/dd3/f12e 0 2024-09-16T00:26:10.789 INFO:tasks.workunit.client.0.smithi112.stdout:7/912: link d6/d10/d16/db4/c10e d6/d10/da1/da2/d10c/c12f 0 2024-09-16T00:26:10.791 INFO:tasks.workunit.client.0.smithi112.stdout:7/913: creat d6/d10/d16/d96/dc7/f130 x:0 0 0 2024-09-16T00:26:10.792 INFO:tasks.workunit.client.0.smithi112.stdout:7/914: chown d6/d10/l45 687 1 2024-09-16T00:26:10.804 INFO:tasks.workunit.client.0.smithi112.stdout:7/915: dread d6/d10/d16/d96/dc7/dbf/d117/d6c/f36 [0,4194304] 0 2024-09-16T00:26:10.809 INFO:tasks.workunit.client.0.smithi112.stdout:7/916: write d6/f22 [350177,72546] 0 2024-09-16T00:26:10.809 INFO:tasks.workunit.client.0.smithi112.stdout:7/917: chown d6/d10/f87 304964 1 2024-09-16T00:26:10.822 INFO:tasks.workunit.client.0.smithi112.stdout:7/918: dread d6/d10/f32 [0,4194304] 0 2024-09-16T00:26:10.824 INFO:tasks.workunit.client.0.smithi112.stdout:7/919: getdents d6/d10/da1/dcd 0 2024-09-16T00:26:10.826 INFO:tasks.workunit.client.0.smithi112.stdout:7/920: rename d6/d10/da1/dcd/cd0 to d6/d10/d16/d96/c131 0 2024-09-16T00:26:10.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:10 smithi112 ceph-mon[27869]: pgmap v258: 65 pgs: 65 active+clean; 2.8 GiB data, 9.7 GiB used, 527 GiB / 536 GiB avail; 123 MiB/s rd, 368 MiB/s wr, 701 op/s 2024-09-16T00:26:10.846 INFO:tasks.workunit.client.1.smithi169.stdout:2/636: sync 2024-09-16T00:26:10.846 INFO:tasks.workunit.client.1.smithi169.stdout:1/675: sync 2024-09-16T00:26:10.846 INFO:tasks.workunit.client.1.smithi169.stdout:9/739: sync 2024-09-16T00:26:10.847 INFO:tasks.workunit.client.1.smithi169.stdout:6/858: sync 2024-09-16T00:26:10.847 INFO:tasks.workunit.client.1.smithi169.stdout:4/616: sync 2024-09-16T00:26:10.847 INFO:tasks.workunit.client.1.smithi169.stdout:7/704: sync 2024-09-16T00:26:10.847 INFO:tasks.workunit.client.1.smithi169.stdout:5/543: sync 2024-09-16T00:26:10.847 INFO:tasks.workunit.client.1.smithi169.stdout:3/758: sync 2024-09-16T00:26:10.847 INFO:tasks.workunit.client.1.smithi169.stdout:0/675: sync 2024-09-16T00:26:10.847 INFO:tasks.workunit.client.1.smithi169.stdout:8/694: sync 2024-09-16T00:26:10.848 INFO:tasks.workunit.client.1.smithi169.stdout:3/759: readlink d7/d1a/d1c/l43 0 2024-09-16T00:26:10.849 INFO:tasks.workunit.client.1.smithi169.stdout:1/676: creat dc/d36/d39/d50/fcb x:0 0 0 2024-09-16T00:26:10.850 INFO:tasks.workunit.client.1.smithi169.stdout:2/637: mkdir d7/d6b/d70/de0 0 2024-09-16T00:26:10.854 INFO:tasks.workunit.client.1.smithi169.stdout:6/859: symlink d2/d3f/d41/d5c/d91/da6/de2/l111 0 2024-09-16T00:26:10.855 INFO:tasks.workunit.client.1.smithi169.stdout:1/677: symlink dc/d6c/d96/lcc 0 2024-09-16T00:26:10.856 INFO:tasks.workunit.client.1.smithi169.stdout:4/617: symlink d2/d12/d13/d1f/d3e/d5b/d70/d7d/lc6 0 2024-09-16T00:26:10.862 INFO:tasks.workunit.client.1.smithi169.stdout:5/544: dwrite d2/da/d10/d13/d1c/d94/faa [0,4194304] 0 2024-09-16T00:26:10.864 INFO:tasks.workunit.client.1.smithi169.stdout:7/705: dwrite d7/d16/d49/f72 [0,4194304] 0 2024-09-16T00:26:10.864 INFO:tasks.workunit.client.1.smithi169.stdout:8/695: getdents d7/dc/d17/d24 0 2024-09-16T00:26:10.866 INFO:tasks.workunit.client.1.smithi169.stdout:1/678: creat dc/d36/fcd x:0 0 0 2024-09-16T00:26:10.876 INFO:tasks.workunit.client.1.smithi169.stdout:9/740: dread d5/da/d2e/f62 [0,4194304] 0 2024-09-16T00:26:10.877 INFO:tasks.workunit.client.1.smithi169.stdout:7/706: rmdir d7/df/d95 39 2024-09-16T00:26:10.877 INFO:tasks.workunit.client.1.smithi169.stdout:2/638: dwrite d7/f55 [0,4194304] 0 2024-09-16T00:26:10.879 INFO:tasks.workunit.client.1.smithi169.stdout:2/639: chown d7/db/d89/dc2/fd4 785676 1 2024-09-16T00:26:10.882 INFO:tasks.workunit.client.1.smithi169.stdout:7/707: fsync d7/f91 0 2024-09-16T00:26:10.882 INFO:tasks.workunit.client.1.smithi169.stdout:4/618: dwrite d2/d12/d13/d1f/d3e/d5b/d70/d7d/fab [0,4194304] 0 2024-09-16T00:26:10.886 INFO:tasks.workunit.client.1.smithi169.stdout:0/676: write d0/d2c/d4f/da5/fb0 [283975,56607] 0 2024-09-16T00:26:10.887 INFO:tasks.workunit.client.1.smithi169.stdout:4/619: symlink d2/d12/d13/d27/lc7 0 2024-09-16T00:26:10.889 INFO:tasks.workunit.client.1.smithi169.stdout:7/708: creat d7/d71/d76/d29/d78/dba/fe1 x:0 0 0 2024-09-16T00:26:10.890 INFO:tasks.workunit.client.1.smithi169.stdout:4/620: write d2/d12/d13/d1f/d3e/d5b/f9f [974045,107670] 0 2024-09-16T00:26:10.891 INFO:tasks.workunit.client.1.smithi169.stdout:9/741: rename d5/d49/dc0/ced to d5/da/d2e/d99/d100/c103 0 2024-09-16T00:26:10.896 INFO:tasks.workunit.client.1.smithi169.stdout:0/677: mkdir d0/d2c/d38/de4/d56/d67/dd1/de9 0 2024-09-16T00:26:10.897 INFO:tasks.workunit.client.1.smithi169.stdout:9/742: fdatasync d5/da/d5b/d91/d9a/fba 0 2024-09-16T00:26:10.898 INFO:tasks.workunit.client.1.smithi169.stdout:7/709: creat d7/d16/d49/d84/fe2 x:0 0 0 2024-09-16T00:26:10.900 INFO:tasks.workunit.client.1.smithi169.stdout:2/640: rename d7/db/d27/d29/f54 to d7/db/d89/db4/dc8/fe1 0 2024-09-16T00:26:10.901 INFO:tasks.workunit.client.1.smithi169.stdout:4/621: mkdir d2/d12/d13/d1f/d3e/d83/dc8 0 2024-09-16T00:26:10.901 INFO:tasks.workunit.client.1.smithi169.stdout:6/860: dwrite d2/d3f/d41/d5c/f88 [0,4194304] 0 2024-09-16T00:26:10.905 INFO:tasks.workunit.client.0.smithi112.stdout:3/942: sync 2024-09-16T00:26:10.908 INFO:tasks.workunit.client.0.smithi112.stdout:3/943: mknod d2/d9/d2b/d47/d52/d7a/d80/db4/d10d/c133 0 2024-09-16T00:26:10.908 INFO:tasks.workunit.client.1.smithi169.stdout:1/679: dread dc/d2c/f32 [0,4194304] 0 2024-09-16T00:26:10.913 INFO:tasks.workunit.client.1.smithi169.stdout:2/641: link f2 d7/db/db8/daf/fe2 0 2024-09-16T00:26:10.913 INFO:tasks.workunit.client.1.smithi169.stdout:5/545: dwrite d2/da/d10/d13/d1c/d32/d53/d83/d7d/fa1 [0,4194304] 0 2024-09-16T00:26:10.915 INFO:tasks.workunit.client.0.smithi112.stdout:3/944: dwrite d2/d9/d2b/d47/d52/d7a/d80/d83/d7b/dad/d107/d69/fbf [0,4194304] 0 2024-09-16T00:26:10.916 INFO:tasks.workunit.client.1.smithi169.stdout:8/696: dwrite d7/dc/d26/d46/f6b [0,4194304] 0 2024-09-16T00:26:10.918 INFO:tasks.workunit.client.0.smithi112.stdout:3/945: symlink d2/d9/d2b/d47/d52/d7a/d80/d83/d7b/dad/d107/d108/l134 0 2024-09-16T00:26:10.925 INFO:tasks.workunit.client.0.smithi112.stdout:3/946: truncate d2/d9/d2b/d47/d52/d7a/d80/d83/d7b/dad/d107/f7e 1838949 0 2024-09-16T00:26:10.928 INFO:tasks.workunit.client.0.smithi112.stdout:3/947: creat d2/d9/d2b/d47/d52/d7a/d80/d83/d68/d90/dcb/f135 x:0 0 0 2024-09-16T00:26:10.929 INFO:tasks.workunit.client.0.smithi112.stdout:3/948: readlink d2/d9/d2b/lef 0 2024-09-16T00:26:10.929 INFO:tasks.workunit.client.1.smithi169.stdout:7/710: dwrite d7/d71/d76/d29/d78/dba/fe1 [0,4194304] 0 2024-09-16T00:26:10.929 INFO:tasks.workunit.client.1.smithi169.stdout:8/697: read d7/dc/d17/f1b [3877856,37034] 0 2024-09-16T00:26:10.929 INFO:tasks.workunit.client.1.smithi169.stdout:4/622: dread d2/d12/d13/d1f/d3e/d5b/d1a/d9e/f72 [0,4194304] 0 2024-09-16T00:26:10.929 INFO:tasks.workunit.client.1.smithi169.stdout:2/642: creat d7/daa/fe3 x:0 0 0 2024-09-16T00:26:10.930 INFO:tasks.workunit.client.1.smithi169.stdout:6/861: dread d2/f64 [0,4194304] 0 2024-09-16T00:26:10.934 INFO:tasks.workunit.client.1.smithi169.stdout:0/678: write d0/d2c/d38/de4/d21/d31/f85 [1645036,95102] 0 2024-09-16T00:26:10.934 INFO:tasks.workunit.client.1.smithi169.stdout:1/680: mkdir dc/d36/d3a/d45/dce 0 2024-09-16T00:26:10.934 INFO:tasks.workunit.client.1.smithi169.stdout:6/862: fdatasync d2/d4d/d59/f7f 0 2024-09-16T00:26:10.935 INFO:tasks.workunit.client.0.smithi112.stdout:3/949: link d2/d9/d2b/d47/d52/d7a/d80/d83/d7b/dad/d107/fb2 d2/d9/d2b/d47/d52/d7a/d80/f136 0 2024-09-16T00:26:10.935 INFO:tasks.workunit.client.1.smithi169.stdout:5/546: symlink d2/da/d10/d13/d1c/d40/d54/d7f/lbd 0 2024-09-16T00:26:10.935 INFO:tasks.workunit.client.1.smithi169.stdout:7/711: creat d7/d5b/d61/fe3 x:0 0 0 2024-09-16T00:26:10.938 INFO:tasks.workunit.client.1.smithi169.stdout:2/643: readlink d7/db/d27/d29/l59 0 2024-09-16T00:26:10.940 INFO:tasks.workunit.client.1.smithi169.stdout:9/743: truncate d5/da/d21/f30 2714933 0 2024-09-16T00:26:10.941 INFO:tasks.workunit.client.1.smithi169.stdout:8/698: symlink d7/dc/d26/d41/d92/dcf/d3c/d82/db3/le3 0 2024-09-16T00:26:10.943 INFO:tasks.workunit.client.0.smithi112.stdout:5/840: sync 2024-09-16T00:26:10.943 INFO:tasks.workunit.client.0.smithi112.stdout:8/791: sync 2024-09-16T00:26:10.943 INFO:tasks.workunit.client.0.smithi112.stdout:1/909: sync 2024-09-16T00:26:10.943 INFO:tasks.workunit.client.0.smithi112.stdout:4/930: sync 2024-09-16T00:26:10.943 INFO:tasks.workunit.client.0.smithi112.stdout:6/857: sync 2024-09-16T00:26:10.943 INFO:tasks.workunit.client.0.smithi112.stdout:7/921: sync 2024-09-16T00:26:10.943 INFO:tasks.workunit.client.0.smithi112.stdout:2/996: sync 2024-09-16T00:26:10.944 INFO:tasks.workunit.client.1.smithi169.stdout:1/681: dwrite dc/f15 [4194304,4194304] 0 2024-09-16T00:26:10.944 INFO:tasks.workunit.client.1.smithi169.stdout:9/744: chown d5/da/d5b/d6c/c97 360924329 1 2024-09-16T00:26:10.944 INFO:tasks.workunit.client.0.smithi112.stdout:6/858: dread - dd/d23/fc0 zero size 2024-09-16T00:26:10.945 INFO:tasks.workunit.client.0.smithi112.stdout:5/841: truncate d4/d7/d14/fb0 76849 0 2024-09-16T00:26:10.946 INFO:tasks.workunit.client.0.smithi112.stdout:2/997: mkdir d2/dd/d48/d8a/d101/d122/d146 0 2024-09-16T00:26:10.947 INFO:tasks.workunit.client.0.smithi112.stdout:2/998: stat d2/dd/d48/d22/d2b/d54/d60/d6c/dc9 0 2024-09-16T00:26:10.947 INFO:tasks.workunit.client.1.smithi169.stdout:8/699: read d7/f10 [667647,38380] 0 2024-09-16T00:26:10.947 INFO:tasks.workunit.client.1.smithi169.stdout:7/712: mknod d7/d16/d49/d84/dc8/ce4 0 2024-09-16T00:26:10.948 INFO:tasks.workunit.client.0.smithi112.stdout:8/792: rename d10/d12/d1c/d61/d6c/dd1/d58/c5a to d10/de9/c106 0 2024-09-16T00:26:10.948 INFO:tasks.workunit.client.1.smithi169.stdout:4/623: mknod d2/dbb/cc9 0 2024-09-16T00:26:10.948 INFO:tasks.workunit.client.1.smithi169.stdout:7/713: chown c4 21 1 2024-09-16T00:26:10.949 INFO:tasks.workunit.client.0.smithi112.stdout:2/999: chown d2/dd/d48/d22/d2b/d54/d60/d6c/d88/da4/fc0 968400 1 2024-09-16T00:26:10.949 INFO:tasks.workunit.client.0.smithi112.stdout:6/859: mknod dd/df/d11/d44/df4/c116 0 2024-09-16T00:26:10.949 INFO:tasks.workunit.client.1.smithi169.stdout:4/624: chown d2 1072 1 2024-09-16T00:26:10.951 INFO:tasks.workunit.client.0.smithi112.stdout:4/931: rename d6/d39/d36/d5c to d6/d39/d1c/d6e/d6f/d11b/daa/d156 0 2024-09-16T00:26:10.954 INFO:tasks.workunit.client.0.smithi112.stdout:4/932: creat d6/d39/d36/f157 x:0 0 0 2024-09-16T00:26:10.955 INFO:tasks.workunit.client.0.smithi112.stdout:4/933: chown d6/d7c/da0/cbf 14 1 2024-09-16T00:26:10.956 INFO:tasks.workunit.client.0.smithi112.stdout:6/860: link dd/df/d10/d47/d55/dc4/dd1/f4a dd/df/d10/d47/d55/dbd/dcb/de8/f117 0 2024-09-16T00:26:10.956 INFO:tasks.workunit.client.0.smithi112.stdout:4/934: chown d6/d39/d1c/d2c 1 1 2024-09-16T00:26:10.958 INFO:tasks.workunit.client.1.smithi169.stdout:8/700: write d7/dc/d26/d41/d4e/d81/d9e/da5/fa6 [725509,45117] 0 2024-09-16T00:26:10.958 INFO:tasks.workunit.client.1.smithi169.stdout:7/714: mknod d7/d71/d76/d9c/ddf/ce5 0 2024-09-16T00:26:10.958 INFO:tasks.workunit.client.0.smithi112.stdout:4/935: chown d6/d39/d1c/d6e/d6f/d11b/daa/d156/d7e/d8f/d11e 3221 1 2024-09-16T00:26:10.959 INFO:tasks.workunit.client.1.smithi169.stdout:9/745: dwrite d5/da/d5b/d91/d9a/fba [0,4194304] 0 2024-09-16T00:26:10.959 INFO:tasks.workunit.client.1.smithi169.stdout:2/644: getdents d7/db/d20 0 2024-09-16T00:26:10.960 INFO:tasks.workunit.client.0.smithi112.stdout:4/936: symlink d6/db/df/d5e/d61/d64/d6a/l158 0 2024-09-16T00:26:10.965 INFO:tasks.workunit.client.1.smithi169.stdout:9/746: mknod d5/da/d2e/d47/c104 0 2024-09-16T00:26:10.969 INFO:tasks.workunit.client.1.smithi169.stdout:8/701: truncate d7/dc/d26/d41/d4e/f7c 622305 0 2024-09-16T00:26:10.969 INFO:tasks.workunit.client.1.smithi169.stdout:2/645: mkdir d7/db/d89/db4/d9d/dbe/de4 0 2024-09-16T00:26:10.970 INFO:tasks.workunit.client.0.smithi112.stdout:8/793: dread d10/d12/d1c/d61/d6c/dd1/d58/f66 [4194304,4194304] 0 2024-09-16T00:26:10.970 INFO:tasks.workunit.client.1.smithi169.stdout:4/625: dwrite d2/dbb/d59/f75 [0,4194304] 0 2024-09-16T00:26:10.971 INFO:tasks.workunit.client.0.smithi112.stdout:3/950: write d2/d9/d2b/d47/d52/d7a/d80/d83/d76/d8e/fea [1442541,1854] 0 2024-09-16T00:26:10.971 INFO:tasks.workunit.client.1.smithi169.stdout:1/682: dwrite dc/d36/d3a/d7f/d93/fa3 [4194304,4194304] 0 2024-09-16T00:26:10.974 INFO:tasks.workunit.client.1.smithi169.stdout:7/715: truncate d7/d5b/dc1/fd6 520596 0 2024-09-16T00:26:10.976 INFO:tasks.workunit.client.1.smithi169.stdout:8/702: mknod d7/d87/dd5/ce4 0 2024-09-16T00:26:10.977 INFO:tasks.workunit.client.1.smithi169.stdout:4/626: dread - d2/d12/d13/d4e/d50/f85 zero size 2024-09-16T00:26:10.980 INFO:tasks.workunit.client.0.smithi112.stdout:8/794: mkdir d10/d12/d1c/dd0/dbb/d107 0 2024-09-16T00:26:10.980 INFO:tasks.workunit.client.0.smithi112.stdout:3/951: write d2/d9/d2b/d47/d52/f119 [380288,123343] 0 2024-09-16T00:26:10.981 INFO:tasks.workunit.client.0.smithi112.stdout:8/795: write d10/d12/d1c/d61/d6c/dd1/d58/f103 [691667,108358] 0 2024-09-16T00:26:10.981 INFO:tasks.workunit.client.1.smithi169.stdout:5/547: dwrite d2/da/d10/d13/f14 [0,4194304] 0 2024-09-16T00:26:10.981 INFO:tasks.workunit.client.1.smithi169.stdout:4/627: write d2/d12/d13/d1f/d2f/f7b [1301169,115870] 0 2024-09-16T00:26:10.982 INFO:tasks.workunit.client.1.smithi169.stdout:2/646: rename d7/db/d20/c49 to d7/d1c/dc3/ce5 0 2024-09-16T00:26:10.982 INFO:tasks.workunit.client.0.smithi112.stdout:8/796: stat d10/l4b 0 2024-09-16T00:26:10.983 INFO:tasks.workunit.client.1.smithi169.stdout:9/747: creat d5/da/d2e/f105 x:0 0 0 2024-09-16T00:26:10.983 INFO:tasks.workunit.client.1.smithi169.stdout:6/863: write d2/da4/db1/dbc/fae [1081783,22408] 0 2024-09-16T00:26:10.983 INFO:tasks.workunit.client.1.smithi169.stdout:2/647: dread - d7/db/d89/db4/d9d/dbe/fdf zero size 2024-09-16T00:26:10.985 INFO:tasks.workunit.client.0.smithi112.stdout:6/861: rename dd/df/d10/d47/d55/dc4/dd1 to dd/df/d17/d27/d84/d118 0 2024-09-16T00:26:10.986 INFO:tasks.workunit.client.1.smithi169.stdout:8/703: unlink d7/dc/d53/cd9 0 2024-09-16T00:26:10.990 INFO:tasks.workunit.client.1.smithi169.stdout:8/704: dread - d7/dc/d26/d41/d92/dcf/fc1 zero size 2024-09-16T00:26:10.991 INFO:tasks.workunit.client.1.smithi169.stdout:1/683: creat dc/d36/d3a/d45/d29/dc2/fcf x:0 0 0 2024-09-16T00:26:10.996 INFO:tasks.workunit.client.1.smithi169.stdout:2/648: dwrite f3 [0,4194304] 0 2024-09-16T00:26:10.998 INFO:tasks.workunit.client.1.smithi169.stdout:4/628: fdatasync d2/d12/d13/d1f/d2f/f7b 0 2024-09-16T00:26:11.000 INFO:tasks.workunit.client.1.smithi169.stdout:6/864: creat d2/d3f/d41/d5c/db9/f112 x:0 0 0 2024-09-16T00:26:11.001 INFO:tasks.workunit.client.0.smithi112.stdout:8/797: truncate d10/d12/d1c/dd0/f52 944211 0 2024-09-16T00:26:11.002 INFO:tasks.workunit.client.1.smithi169.stdout:4/629: chown d2/d12/d13/d4e/d50/l8a 230 1 2024-09-16T00:26:11.002 INFO:tasks.workunit.client.1.smithi169.stdout:8/705: mknod d7/dc/d26/d41/d67/ce5 0 2024-09-16T00:26:11.003 INFO:tasks.workunit.client.0.smithi112.stdout:6/862: mknod dd/df/d10/d47/d55/dbd/de9/c119 0 2024-09-16T00:26:11.004 INFO:tasks.workunit.client.1.smithi169.stdout:7/716: creat d7/fe6 x:0 0 0 2024-09-16T00:26:11.004 INFO:tasks.workunit.client.1.smithi169.stdout:6/865: truncate d2/d22/da7/f10e 43600 0 2024-09-16T00:26:11.005 INFO:tasks.workunit.client.0.smithi112.stdout:4/937: rename d6/db/d1f/d148 to d6/d39/d1c/d6e/d159 0 2024-09-16T00:26:11.007 INFO:tasks.workunit.client.0.smithi112.stdout:6/863: fdatasync dd/df/d13/d18/f28 0 2024-09-16T00:26:11.009 INFO:tasks.workunit.client.0.smithi112.stdout:4/938: mknod d6/db/d1f/d41/c15a 0 2024-09-16T00:26:11.011 INFO:tasks.workunit.client.0.smithi112.stdout:8/798: dwrite d10/f34 [0,4194304] 0 2024-09-16T00:26:11.011 INFO:tasks.workunit.client.1.smithi169.stdout:1/684: dread dc/f21 [0,4194304] 0 2024-09-16T00:26:11.014 INFO:tasks.workunit.client.0.smithi112.stdout:8/799: symlink d10/de9/def/l108 0 2024-09-16T00:26:11.016 INFO:tasks.workunit.client.1.smithi169.stdout:7/717: creat d7/df/fe7 x:0 0 0 2024-09-16T00:26:11.018 INFO:tasks.workunit.client.1.smithi169.stdout:7/718: creat d7/d71/fe8 x:0 0 0 2024-09-16T00:26:11.019 INFO:tasks.workunit.client.1.smithi169.stdout:1/685: getdents dc/d36/d3a/d45/d29/d2a 0 2024-09-16T00:26:11.020 INFO:tasks.workunit.client.1.smithi169.stdout:3/760: sync 2024-09-16T00:26:11.020 INFO:tasks.workunit.client.0.smithi112.stdout:6/864: dwrite dd/df/d11/fc8 [0,4194304] 0 2024-09-16T00:26:11.022 INFO:tasks.workunit.client.1.smithi169.stdout:3/761: getdents d7/d1a/d1c/d35 0 2024-09-16T00:26:11.023 INFO:tasks.workunit.client.0.smithi112.stdout:6/865: getdents dd/df/d10/d47/d89/dbc 0 2024-09-16T00:26:11.024 INFO:tasks.workunit.client.0.smithi112.stdout:8/800: mknod d10/d12/d1c/d61/d6c/dd1/d8c/d7c/c109 0 2024-09-16T00:26:11.025 INFO:tasks.workunit.client.1.smithi169.stdout:3/762: rmdir d7/d1a/d2b/dc6 39 2024-09-16T00:26:11.025 INFO:tasks.workunit.client.0.smithi112.stdout:6/866: chown dd/df/d13/d18/c3b 0 1 2024-09-16T00:26:11.025 INFO:tasks.workunit.client.1.smithi169.stdout:3/763: chown d7 3177450 1 2024-09-16T00:26:11.026 INFO:tasks.workunit.client.1.smithi169.stdout:3/764: chown d7/c12 726274496 1 2024-09-16T00:26:11.026 INFO:tasks.workunit.client.0.smithi112.stdout:8/801: symlink d10/d12/d1c/d3b/l10a 0 2024-09-16T00:26:11.027 INFO:tasks.workunit.client.1.smithi169.stdout:5/548: dread d2/fd [0,4194304] 0 2024-09-16T00:26:11.028 INFO:tasks.workunit.client.0.smithi112.stdout:3/952: dread d2/f2d [4194304,4194304] 0 2024-09-16T00:26:11.029 INFO:tasks.workunit.client.1.smithi169.stdout:4/630: truncate d2/d12/d13/d1f/d2f/f7b 2915729 0 2024-09-16T00:26:11.029 INFO:tasks.workunit.client.0.smithi112.stdout:8/802: write d10/d12/d1c/d3b/d7d/de4/fed [572605,2094] 0 2024-09-16T00:26:11.030 INFO:tasks.workunit.client.1.smithi169.stdout:4/631: chown d2/d12/d13/c1d 581220 1 2024-09-16T00:26:11.031 INFO:tasks.workunit.client.1.smithi169.stdout:3/765: creat d7/d1a/d1c/d23/d88/dbe/f110 x:0 0 0 2024-09-16T00:26:11.033 INFO:tasks.workunit.client.0.smithi112.stdout:3/953: symlink d2/d9/d2b/d47/d78/l137 0 2024-09-16T00:26:11.035 INFO:tasks.workunit.client.1.smithi169.stdout:3/766: symlink d7/d1a/d1c/d35/d63/da3/de4/l111 0 2024-09-16T00:26:11.036 INFO:tasks.workunit.client.1.smithi169.stdout:5/549: creat d2/da/d10/d13/d1c/d40/fbe x:0 0 0 2024-09-16T00:26:11.041 INFO:tasks.workunit.client.1.smithi169.stdout:8/706: truncate d7/dc/d17/d24/f4b 2707210 0 2024-09-16T00:26:11.042 INFO:tasks.workunit.client.1.smithi169.stdout:5/550: mkdir d2/da/d10/d13/d1c/d32/d53/dae/dbf 0 2024-09-16T00:26:11.042 INFO:tasks.workunit.client.0.smithi112.stdout:3/954: dwrite d2/d9/d2b/d47/d52/d7a/d80/f120 [4194304,4194304] 0 2024-09-16T00:26:11.044 INFO:tasks.workunit.client.1.smithi169.stdout:1/686: truncate dc/d18/d67/f49 2482867 0 2024-09-16T00:26:11.045 INFO:tasks.workunit.client.0.smithi112.stdout:8/803: dread d10/f1b [0,4194304] 0 2024-09-16T00:26:11.048 INFO:tasks.workunit.client.0.smithi112.stdout:3/955: write d2/d9/d2b/d47/f9d [4413867,80251] 0 2024-09-16T00:26:11.050 INFO:tasks.workunit.client.1.smithi169.stdout:8/707: creat d7/dc/d26/d41/d67/fe6 x:0 0 0 2024-09-16T00:26:11.051 INFO:tasks.workunit.client.1.smithi169.stdout:3/767: dwrite d7/d1a/d1c/f78 [0,4194304] 0 2024-09-16T00:26:11.051 INFO:tasks.workunit.client.0.smithi112.stdout:4/939: dread d6/d39/f153 [0,4194304] 0 2024-09-16T00:26:11.051 INFO:tasks.workunit.client.0.smithi112.stdout:8/804: mknod d10/c10b 0 2024-09-16T00:26:11.053 INFO:tasks.workunit.client.0.smithi112.stdout:3/956: creat d2/d9/dfb/f138 x:0 0 0 2024-09-16T00:26:11.053 INFO:tasks.workunit.client.1.smithi169.stdout:5/551: chown d2/da/d10/d13/d1c/d94/lac 49 1 2024-09-16T00:26:11.054 INFO:tasks.workunit.client.1.smithi169.stdout:9/748: dwrite d5/da/d21/de4/f4c [0,4194304] 0 2024-09-16T00:26:11.054 INFO:tasks.workunit.client.0.smithi112.stdout:4/940: creat d6/d39/d1c/d6e/d119/f15b x:0 0 0 2024-09-16T00:26:11.055 INFO:tasks.workunit.client.1.smithi169.stdout:3/768: chown d7/d50/d90/dba/fbf 5265 1 2024-09-16T00:26:11.056 INFO:tasks.workunit.client.0.smithi112.stdout:3/957: rmdir d2/d9/d2b/d47/d52/d7a/d80/d83/d68 39 2024-09-16T00:26:11.061 INFO:tasks.workunit.client.0.smithi112.stdout:4/941: rename d6/db/df/l56 to d6/d39/d1c/d6e/d6f/d11b/daa/d156/d7e/dd1/l15c 0 2024-09-16T00:26:11.062 INFO:tasks.workunit.client.0.smithi112.stdout:6/867: dread dd/df/d13/d18/f28 [0,4194304] 0 2024-09-16T00:26:11.063 INFO:tasks.workunit.client.1.smithi169.stdout:6/866: dwrite d2/d4d/d59/d38/f62 [4194304,4194304] 0 2024-09-16T00:26:11.065 INFO:tasks.workunit.client.1.smithi169.stdout:5/552: mkdir d2/da/d10/d13/dc0 0 2024-09-16T00:26:11.066 INFO:tasks.workunit.client.0.smithi112.stdout:6/868: creat dd/f11a x:0 0 0 2024-09-16T00:26:11.068 INFO:tasks.workunit.client.1.smithi169.stdout:8/708: dread d7/dc/d26/d41/d92/dcf/f91 [0,4194304] 0 2024-09-16T00:26:11.068 INFO:tasks.workunit.client.0.smithi112.stdout:3/958: dwrite d2/d9/d2b/d47/d52/d7a/d80/d83/d68/d90/fe9 [0,4194304] 0 2024-09-16T00:26:11.069 INFO:tasks.workunit.client.0.smithi112.stdout:6/869: creat dd/dcc/f11b x:0 0 0 2024-09-16T00:26:11.070 INFO:tasks.workunit.client.1.smithi169.stdout:4/632: dread d2/d12/f1e [0,4194304] 0 2024-09-16T00:26:11.073 INFO:tasks.workunit.client.1.smithi169.stdout:8/709: dread - d7/dc/d26/d41/d92/dcf/fc1 zero size 2024-09-16T00:26:11.073 INFO:tasks.workunit.client.1.smithi169.stdout:3/769: rename d7/d50/d90/dba/dc0/le6 to d7/d1a/d1c/d6c/l112 0 2024-09-16T00:26:11.074 INFO:tasks.workunit.client.1.smithi169.stdout:3/770: dread - d7/de5/f103 zero size 2024-09-16T00:26:11.075 INFO:tasks.workunit.client.0.smithi112.stdout:3/959: dwrite d2/d9/d2b/d47/d52/d7a/d80/d83/d49/f77 [0,4194304] 0 2024-09-16T00:26:11.076 INFO:tasks.workunit.client.0.smithi112.stdout:3/960: fsync d2/d9/d2b/d47/d52/f119 0 2024-09-16T00:26:11.078 INFO:tasks.workunit.client.1.smithi169.stdout:6/867: read d2/d3f/d41/d72/fa9 [2372300,24064] 0 2024-09-16T00:26:11.079 INFO:tasks.workunit.client.1.smithi169.stdout:8/710: creat d7/dc/d2a/fe7 x:0 0 0 2024-09-16T00:26:11.080 INFO:tasks.workunit.client.1.smithi169.stdout:9/749: link d5/da/d5b/fcc d5/da/d5b/d6c/da0/f106 0 2024-09-16T00:26:11.081 INFO:tasks.workunit.client.0.smithi112.stdout:3/961: creat d2/d9/f139 x:0 0 0 2024-09-16T00:26:11.081 INFO:tasks.workunit.client.1.smithi169.stdout:6/868: rename d2/d3f/d41/d72/d81 to d2/d3f/d41/d72/d81/d113 22 2024-09-16T00:26:11.081 INFO:tasks.workunit.client.1.smithi169.stdout:8/711: stat d7/dc/d17/d24/d32/l44 0 2024-09-16T00:26:11.082 INFO:tasks.workunit.client.0.smithi112.stdout:4/942: dread d6/db/d43/fea [0,4194304] 0 2024-09-16T00:26:11.082 INFO:tasks.workunit.client.1.smithi169.stdout:9/750: chown d5/da/d5b/d6c/dcb/db4/fc4 10 1 2024-09-16T00:26:11.083 INFO:tasks.workunit.client.1.smithi169.stdout:5/553: link d2/da/d10/fa9 d2/da/d10/d13/d1c/d40/d68/fc1 0 2024-09-16T00:26:11.084 INFO:tasks.workunit.client.1.smithi169.stdout:6/869: read d2/d3f/d41/f80 [1663969,985] 0 2024-09-16T00:26:11.084 INFO:tasks.workunit.client.0.smithi112.stdout:3/962: symlink d2/d9/d2b/d47/d52/d7a/d80/d83/d49/l13a 0 2024-09-16T00:26:11.085 INFO:tasks.workunit.client.1.smithi169.stdout:1/687: link dc/d36/d3a/d45/d29/d4d/c87 dc/d36/d3a/d45/d29/dc2/dc7/cd0 0 2024-09-16T00:26:11.086 INFO:tasks.workunit.client.1.smithi169.stdout:8/712: rmdir d7 39 2024-09-16T00:26:11.086 INFO:tasks.workunit.client.0.smithi112.stdout:3/963: symlink d2/d9/d2b/d47/d52/d7a/d80/db4/d131/l13b 0 2024-09-16T00:26:11.087 INFO:tasks.workunit.client.0.smithi112.stdout:3/964: write d2/d9/faa [1565004,113685] 0 2024-09-16T00:26:11.090 INFO:tasks.workunit.client.1.smithi169.stdout:5/554: symlink d2/da/d10/d13/d1c/d71/lc2 0 2024-09-16T00:26:11.091 INFO:tasks.workunit.client.1.smithi169.stdout:1/688: mkdir dc/d36/d39/d50/daf/d88/dd1 0 2024-09-16T00:26:11.093 INFO:tasks.workunit.client.1.smithi169.stdout:6/870: truncate d2/d3f/d41/d72/d81/fb4 2329956 0 2024-09-16T00:26:11.093 INFO:tasks.workunit.client.1.smithi169.stdout:4/633: link d2/d12/d13/d4e/c67 d2/dbb/d59/cca 0 2024-09-16T00:26:11.093 INFO:tasks.workunit.client.1.smithi169.stdout:8/713: readlink d7/dc/d26/d41/d4e/d81/lca 0 2024-09-16T00:26:11.095 INFO:tasks.workunit.client.1.smithi169.stdout:1/689: write dc/d18/d67/d69/fae [1033079,10727] 0 2024-09-16T00:26:11.096 INFO:tasks.workunit.client.1.smithi169.stdout:1/690: fdatasync dc/f66 0 2024-09-16T00:26:11.096 INFO:tasks.workunit.client.1.smithi169.stdout:4/634: creat d2/dbb/d59/fcb x:0 0 0 2024-09-16T00:26:11.097 INFO:tasks.workunit.client.1.smithi169.stdout:4/635: fdatasync d2/d12/d9c/da4/fc5 0 2024-09-16T00:26:11.098 INFO:tasks.workunit.client.1.smithi169.stdout:1/691: truncate dc/d18/d67/d69/d7b/fba 701622 0 2024-09-16T00:26:11.099 INFO:tasks.workunit.client.1.smithi169.stdout:6/871: creat d2/d4d/d59/d43/f114 x:0 0 0 2024-09-16T00:26:11.100 INFO:tasks.workunit.client.1.smithi169.stdout:5/555: getdents d2/d7c 0 2024-09-16T00:26:11.101 INFO:tasks.workunit.client.1.smithi169.stdout:4/636: fdatasync d2/d12/d13/d90/fae 0 2024-09-16T00:26:11.102 INFO:tasks.workunit.client.1.smithi169.stdout:6/872: chown d2/da4/fbf 2 1 2024-09-16T00:26:11.102 INFO:tasks.workunit.client.1.smithi169.stdout:5/556: fdatasync d2/da/d10/f8a 0 2024-09-16T00:26:11.105 INFO:tasks.workunit.client.1.smithi169.stdout:6/873: mkdir d2/d36/d115 0 2024-09-16T00:26:11.106 INFO:tasks.workunit.client.1.smithi169.stdout:5/557: fsync d2/da/d10/d13/d1c/d32/d53/d83/d7d/f8b 0 2024-09-16T00:26:11.109 INFO:tasks.workunit.client.1.smithi169.stdout:4/637: rename d2/d12/d13/d1f/d3e/d5b/d1a to d2/d12/dcc 0 2024-09-16T00:26:11.110 INFO:tasks.workunit.client.0.smithi112.stdout:4/943: dread d6/d39/d1c/d6e/d6f/d11b/daa/d156/dc6/db5/f14e [0,4194304] 0 2024-09-16T00:26:11.111 INFO:tasks.workunit.client.1.smithi169.stdout:5/558: mkdir d2/da/d10/d13/dc0/dc3 0 2024-09-16T00:26:11.111 INFO:tasks.workunit.client.1.smithi169.stdout:4/638: chown d2/d12/d13/d4e/c62 9383273 1 2024-09-16T00:26:11.115 INFO:tasks.workunit.client.1.smithi169.stdout:4/639: dread - d2/d12/d13/f8f zero size 2024-09-16T00:26:11.118 INFO:tasks.workunit.client.1.smithi169.stdout:8/714: dread d7/dc/d26/d41/d92/dcf/d80/f8b [0,4194304] 0 2024-09-16T00:26:11.121 INFO:tasks.workunit.client.1.smithi169.stdout:5/559: dwrite d2/da/d10/d13/d24/fa0 [0,4194304] 0 2024-09-16T00:26:11.121 INFO:tasks.workunit.client.0.smithi112.stdout:6/870: dwrite dd/df/d10/f60 [4194304,4194304] 0 2024-09-16T00:26:11.123 INFO:tasks.workunit.client.1.smithi169.stdout:9/751: write d5/da/d5b/d6f/fa1 [381413,7913] 0 2024-09-16T00:26:11.123 INFO:tasks.workunit.client.0.smithi112.stdout:3/965: dwrite d2/d9/d2b/d47/d52/d7a/d80/d83/d68/d90/fe9 [4194304,4194304] 0 2024-09-16T00:26:11.126 INFO:tasks.workunit.client.1.smithi169.stdout:8/715: creat d7/fe8 x:0 0 0 2024-09-16T00:26:11.127 INFO:tasks.workunit.client.1.smithi169.stdout:6/874: dread d2/d22/f23 [0,4194304] 0 2024-09-16T00:26:11.130 INFO:tasks.workunit.client.0.smithi112.stdout:6/871: symlink dd/df/d13/l11c 0 2024-09-16T00:26:11.131 INFO:tasks.workunit.client.0.smithi112.stdout:4/944: dread d6/db/df/d5e/d61/d64/d6a/fd0 [0,4194304] 0 2024-09-16T00:26:11.132 INFO:tasks.workunit.client.1.smithi169.stdout:4/640: rename d2/d12/l28 to d2/d12/d13/d1f/d3e/lcd 0 2024-09-16T00:26:11.132 INFO:tasks.workunit.client.1.smithi169.stdout:3/771: dwrite d7/d1a/d1c/d35/f93 [0,4194304] 0 2024-09-16T00:26:11.134 INFO:tasks.workunit.client.1.smithi169.stdout:8/716: write d7/d87/dd5/f7d [476047,90419] 0 2024-09-16T00:26:11.135 INFO:tasks.workunit.client.0.smithi112.stdout:3/966: rename d2/d9/d2b/d47/d52/d7a/d80/d83/d7b/dad/d107/f7e to d2/d9/d2b/d47/d52/d7a/d80/d83/d7b/dad/d107/d108/f13c 0 2024-09-16T00:26:11.137 INFO:tasks.workunit.client.1.smithi169.stdout:6/875: rmdir d2/d3f/d41/d72/d9e/dc0 39 2024-09-16T00:26:11.137 INFO:tasks.workunit.client.1.smithi169.stdout:8/717: write d7/dc/d26/d41/d4e/f7c [875396,15304] 0 2024-09-16T00:26:11.138 INFO:tasks.workunit.client.0.smithi112.stdout:3/967: readlink d2/d9/d2b/d47/d52/d7a/d80/d83/d68/lb9 0 2024-09-16T00:26:11.139 INFO:tasks.workunit.client.0.smithi112.stdout:4/945: creat d6/db/d1f/d41/d49/d31/f15d x:0 0 0 2024-09-16T00:26:11.140 INFO:tasks.workunit.client.1.smithi169.stdout:3/772: mkdir d7/d50/d6f/d113 0 2024-09-16T00:26:11.141 INFO:tasks.workunit.client.0.smithi112.stdout:4/946: fsync d6/d39/d1c/d6e/d6f/d11b/daa/d156/ffa 0 2024-09-16T00:26:11.141 INFO:tasks.workunit.client.1.smithi169.stdout:6/876: dread d2/d22/f23 [0,4194304] 0 2024-09-16T00:26:11.142 INFO:tasks.workunit.client.1.smithi169.stdout:3/773: write d7/d1a/d1c/d35/d63/f4a [3287370,12179] 0 2024-09-16T00:26:11.145 INFO:tasks.workunit.client.1.smithi169.stdout:4/641: fdatasync d2/d12/d13/d1f/d2f/f82 0 2024-09-16T00:26:11.147 INFO:tasks.workunit.client.0.smithi112.stdout:4/947: dwrite d6/db/f149 [0,4194304] 0 2024-09-16T00:26:11.148 INFO:tasks.workunit.client.1.smithi169.stdout:8/718: creat d7/dc/d17/d48/fe9 x:0 0 0 2024-09-16T00:26:11.149 INFO:tasks.workunit.client.0.smithi112.stdout:4/948: creat d6/d39/d1c/d66/d12d/f15e x:0 0 0 2024-09-16T00:26:11.151 INFO:tasks.workunit.client.0.smithi112.stdout:4/949: creat d6/d39/d1c/d6e/d6f/dda/f15f x:0 0 0 2024-09-16T00:26:11.153 INFO:tasks.workunit.client.1.smithi169.stdout:5/560: getdents d2/da/d10/d13/d1c 0 2024-09-16T00:26:11.153 INFO:tasks.workunit.client.1.smithi169.stdout:3/774: chown d7/d1a/l3e 511704957 1 2024-09-16T00:26:11.153 INFO:tasks.workunit.client.0.smithi112.stdout:3/968: dread d2/d9/d2b/d47/d52/d7a/d80/d83/d68/d90/fdb [0,4194304] 0 2024-09-16T00:26:11.153 INFO:tasks.workunit.client.0.smithi112.stdout:4/950: read - d6/d39/d1c/d6e/d6f/d11b/daa/d156/d7e/f14f zero size 2024-09-16T00:26:11.154 INFO:tasks.workunit.client.1.smithi169.stdout:8/719: creat d7/dc/d26/fea x:0 0 0 2024-09-16T00:26:11.155 INFO:tasks.workunit.client.0.smithi112.stdout:4/951: chown d6/d39/d1c/d6e/c124 10172 1 2024-09-16T00:26:11.156 INFO:tasks.workunit.client.0.smithi112.stdout:3/969: creat d2/d9/d2b/d39/f13d x:0 0 0 2024-09-16T00:26:11.157 INFO:tasks.workunit.client.1.smithi169.stdout:9/752: getdents d5 0 2024-09-16T00:26:11.158 INFO:tasks.workunit.client.1.smithi169.stdout:3/775: dread - d7/d1a/d2b/dc6/dff/f10f zero size 2024-09-16T00:26:11.158 INFO:tasks.workunit.client.1.smithi169.stdout:9/753: chown d5/da/l69 1 1 2024-09-16T00:26:11.158 INFO:tasks.workunit.client.1.smithi169.stdout:5/561: creat d2/da/d10/d13/d1c/d32/d53/dae/fc4 x:0 0 0 2024-09-16T00:26:11.159 INFO:tasks.workunit.client.0.smithi112.stdout:3/970: write d2/d9/f128 [1017128,99677] 0 2024-09-16T00:26:11.159 INFO:tasks.workunit.client.1.smithi169.stdout:3/776: chown d7/d50/ca4 9952299 1 2024-09-16T00:26:11.161 INFO:tasks.workunit.client.0.smithi112.stdout:3/971: chown d2/d9/d2b/d47/d52/d7a/d80/db4/c109 178039685 1 2024-09-16T00:26:11.162 INFO:tasks.workunit.client.1.smithi169.stdout:9/754: creat d5/da/d5b/d6c/f107 x:0 0 0 2024-09-16T00:26:11.162 INFO:tasks.workunit.client.0.smithi112.stdout:4/952: dwrite f4 [4194304,4194304] 0 2024-09-16T00:26:11.163 INFO:tasks.workunit.client.1.smithi169.stdout:8/720: dread d7/d87/dd5/f7d [0,4194304] 0 2024-09-16T00:26:11.163 INFO:tasks.workunit.client.1.smithi169.stdout:3/777: symlink d7/d1a/d1c/l114 0 2024-09-16T00:26:11.164 INFO:tasks.workunit.client.0.smithi112.stdout:4/953: dread - d6/d39/d1c/fbb zero size 2024-09-16T00:26:11.165 INFO:tasks.workunit.client.0.smithi112.stdout:1/910: sync 2024-09-16T00:26:11.165 INFO:tasks.workunit.client.1.smithi169.stdout:9/755: symlink d5/da/d2e/l108 0 2024-09-16T00:26:11.167 INFO:tasks.workunit.client.1.smithi169.stdout:8/721: symlink d7/dc/d26/d41/d92/dcf/d3c/d82/db3/leb 0 2024-09-16T00:26:11.169 INFO:tasks.workunit.client.0.smithi112.stdout:4/954: creat d6/da/d152/f160 x:0 0 0 2024-09-16T00:26:11.170 INFO:tasks.workunit.client.1.smithi169.stdout:9/756: creat d5/de0/f109 x:0 0 0 2024-09-16T00:26:11.171 INFO:tasks.workunit.client.0.smithi112.stdout:1/911: mknod d0/d14/df/c12b 0 2024-09-16T00:26:11.171 INFO:tasks.workunit.client.0.smithi112.stdout:4/955: chown d6/db/d1f/d41/d49/fca 66554236 1 2024-09-16T00:26:11.172 INFO:tasks.workunit.client.0.smithi112.stdout:6/872: write dd/df/d11/f1e [1908659,3227] 0 2024-09-16T00:26:11.172 INFO:tasks.workunit.client.1.smithi169.stdout:8/722: creat d7/fec x:0 0 0 2024-09-16T00:26:11.173 INFO:tasks.workunit.client.0.smithi112.stdout:1/912: mknod d0/d14/df/d28/d45/d30/c12c 0 2024-09-16T00:26:11.174 INFO:tasks.workunit.client.0.smithi112.stdout:4/956: truncate d6/d39/d1c/d6e/d6f/d11b/daa/d156/fc0 338208 0 2024-09-16T00:26:11.175 INFO:tasks.workunit.client.1.smithi169.stdout:8/723: mkdir d7/d87/dd5/ded 0 2024-09-16T00:26:11.175 INFO:tasks.workunit.client.1.smithi169.stdout:9/757: getdents d5/da/d2e/d99 0 2024-09-16T00:26:11.176 INFO:tasks.workunit.client.1.smithi169.stdout:9/758: chown d5/d49/d64/fcd 4972457 1 2024-09-16T00:26:11.178 INFO:tasks.workunit.client.1.smithi169.stdout:4/642: truncate d2/d12/d13/d4e/f77 4272337 0 2024-09-16T00:26:11.178 INFO:tasks.workunit.client.1.smithi169.stdout:8/724: rmdir d7/dc/d17/d24/d32/d8d/dad 39 2024-09-16T00:26:11.178 INFO:tasks.workunit.client.0.smithi112.stdout:3/972: write d2/d9/d2b/d47/d52/d7a/d80/d83/d7b/dad/d107/d69/f123 [135984,27496] 0 2024-09-16T00:26:11.178 INFO:tasks.workunit.client.0.smithi112.stdout:6/873: truncate dd/df/d10/d47/d55/f7a 423722 0 2024-09-16T00:26:11.179 INFO:tasks.workunit.client.1.smithi169.stdout:4/643: chown d2/d12/d13/d1f/d3e/d5b/f18 1 1 2024-09-16T00:26:11.180 INFO:tasks.workunit.client.0.smithi112.stdout:4/957: creat d6/d39/d1c/d6e/d6f/d11b/daa/d156/d7e/dd1/f161 x:0 0 0 2024-09-16T00:26:11.180 INFO:tasks.workunit.client.1.smithi169.stdout:9/759: mknod d5/da/d2e/d99/c10a 0 2024-09-16T00:26:11.184 INFO:tasks.workunit.client.1.smithi169.stdout:5/562: write d2/da/d10/d13/d24/f47 [4921123,28209] 0 2024-09-16T00:26:11.184 INFO:tasks.workunit.client.1.smithi169.stdout:6/877: dread d2/d4d/d59/d38/f97 [0,4194304] 0 2024-09-16T00:26:11.185 INFO:tasks.workunit.client.0.smithi112.stdout:4/958: creat d6/d39/d36/f162 x:0 0 0 2024-09-16T00:26:11.185 INFO:tasks.workunit.client.0.smithi112.stdout:4/959: symlink d6/d39/l163 0 2024-09-16T00:26:11.187 INFO:tasks.workunit.client.0.smithi112.stdout:1/913: dwrite d0/d14/fd0 [0,4194304] 0 2024-09-16T00:26:11.189 INFO:tasks.workunit.client.1.smithi169.stdout:6/878: creat d2/d3f/d41/d5c/d73/f116 x:0 0 0 2024-09-16T00:26:11.189 INFO:tasks.workunit.client.1.smithi169.stdout:5/563: write d2/da/d10/d13/d1c/d2d/f57 [1784097,30232] 0 2024-09-16T00:26:11.190 INFO:tasks.workunit.client.1.smithi169.stdout:8/725: dread d7/dc/d2a/f85 [0,4194304] 0 2024-09-16T00:26:11.194 INFO:tasks.workunit.client.1.smithi169.stdout:6/879: chown d2/d22/da7 19 1 2024-09-16T00:26:11.194 INFO:tasks.workunit.client.1.smithi169.stdout:5/564: write d2/f6b [561138,59587] 0 2024-09-16T00:26:11.194 INFO:tasks.workunit.client.1.smithi169.stdout:4/644: truncate d2/d12/d13/d1f/d2f/f45 2887295 0 2024-09-16T00:26:11.194 INFO:tasks.workunit.client.0.smithi112.stdout:4/960: write d6/d39/d1c/fbb [877298,73037] 0 2024-09-16T00:26:11.195 INFO:tasks.workunit.client.0.smithi112.stdout:3/973: dwrite d2/d9/d2b/d47/f8b [0,4194304] 0 2024-09-16T00:26:11.198 INFO:tasks.workunit.client.1.smithi169.stdout:6/880: stat d2/d36/fec 0 2024-09-16T00:26:11.198 INFO:tasks.workunit.client.0.smithi112.stdout:6/874: dwrite dd/f7e [0,4194304] 0 2024-09-16T00:26:11.200 INFO:tasks.workunit.client.1.smithi169.stdout:9/760: dwrite d5/da/d2e/d47/fdf [4194304,4194304] 0 2024-09-16T00:26:11.201 INFO:tasks.workunit.client.1.smithi169.stdout:0/679: sync 2024-09-16T00:26:11.201 INFO:tasks.workunit.client.0.smithi112.stdout:5/842: sync 2024-09-16T00:26:11.201 INFO:tasks.workunit.client.0.smithi112.stdout:7/922: sync 2024-09-16T00:26:11.202 INFO:tasks.workunit.client.1.smithi169.stdout:3/778: dwrite d7/d1a/d1c/d23/d70/f8e [0,4194304] 0 2024-09-16T00:26:11.204 INFO:tasks.workunit.client.1.smithi169.stdout:8/726: symlink d7/dc/d53/da0/dac/lee 0 2024-09-16T00:26:11.205 INFO:tasks.workunit.client.0.smithi112.stdout:4/961: rename d6/db/d1f/d94/d10c/d123/f130 to d6/d39/d1c/d6e/d6f/d11b/d13b/f164 0 2024-09-16T00:26:11.205 INFO:tasks.workunit.client.1.smithi169.stdout:4/645: fdatasync d2/d12/d13/d5c/f66 0 2024-09-16T00:26:11.207 INFO:tasks.workunit.client.0.smithi112.stdout:7/923: creat d6/d10/d75/d8f/d90/ddc/f132 x:0 0 0 2024-09-16T00:26:11.208 INFO:tasks.workunit.client.0.smithi112.stdout:3/974: creat d2/d9/d2b/d47/d52/d7a/d80/d83/d7b/dad/d107/d108/d12f/f13e x:0 0 0 2024-09-16T00:26:11.208 INFO:tasks.workunit.client.1.smithi169.stdout:8/727: fsync d7/dc/d17/d24/d32/d8d/fc8 0 2024-09-16T00:26:11.209 INFO:tasks.workunit.client.0.smithi112.stdout:4/962: creat d6/d39/d36/f165 x:0 0 0 2024-09-16T00:26:11.209 INFO:tasks.workunit.client.0.smithi112.stdout:5/843: creat d4/d4f/d77/f11c x:0 0 0 2024-09-16T00:26:11.210 INFO:tasks.workunit.client.1.smithi169.stdout:4/646: chown d2/d12/d13/d27/lc7 26148 1 2024-09-16T00:26:11.210 INFO:tasks.workunit.client.0.smithi112.stdout:7/924: chown d6/lf 6 1 2024-09-16T00:26:11.210 INFO:tasks.workunit.client.0.smithi112.stdout:6/875: dread dd/df/d13/d18/f28 [0,4194304] 0 2024-09-16T00:26:11.211 INFO:tasks.workunit.client.1.smithi169.stdout:9/761: dread d5/da/f7f [0,4194304] 0 2024-09-16T00:26:11.211 INFO:tasks.workunit.client.1.smithi169.stdout:6/881: symlink d2/da4/d10d/l117 0 2024-09-16T00:26:11.212 INFO:tasks.workunit.client.0.smithi112.stdout:5/844: dread - d4/dd/d49/d70/d83/fe3 zero size 2024-09-16T00:26:11.212 INFO:tasks.workunit.client.1.smithi169.stdout:3/779: mknod d7/d1a/d1c/d23/d88/d9b/de2/c115 0 2024-09-16T00:26:11.212 INFO:tasks.workunit.client.1.smithi169.stdout:5/565: dread d2/da/d10/d13/f14 [0,4194304] 0 2024-09-16T00:26:11.212 INFO:tasks.workunit.client.1.smithi169.stdout:0/680: creat d0/d2/d4b/fea x:0 0 0 2024-09-16T00:26:11.213 INFO:tasks.workunit.client.0.smithi112.stdout:1/914: dread d0/d14/df/d28/d45/dd4/fb3 [0,4194304] 0 2024-09-16T00:26:11.215 INFO:tasks.workunit.client.0.smithi112.stdout:6/876: mknod dd/df/d10/d47/c11d 0 2024-09-16T00:26:11.216 INFO:tasks.workunit.client.1.smithi169.stdout:4/647: chown d2/d12/d13/d1f/d2f/c4c 940 1 2024-09-16T00:26:11.216 INFO:tasks.workunit.client.0.smithi112.stdout:3/975: readlink d2/d9/d2b/d47/d52/d7a/d80/d83/d7b/dad/d107/d108/l114 0 2024-09-16T00:26:11.217 INFO:tasks.workunit.client.1.smithi169.stdout:5/566: rename d2/da/d10/d13 to d2/da/d10/d13/d1c/d40/d68/dc5 22 2024-09-16T00:26:11.218 INFO:tasks.workunit.client.0.smithi112.stdout:1/915: symlink d0/d14/d5e/l12d 0 2024-09-16T00:26:11.218 INFO:tasks.workunit.client.0.smithi112.stdout:7/925: symlink d6/d10/d16/d96/dc7/l133 0 2024-09-16T00:26:11.219 INFO:tasks.workunit.client.0.smithi112.stdout:3/976: symlink d2/d9/d2b/l13f 0 2024-09-16T00:26:11.219 INFO:tasks.workunit.client.0.smithi112.stdout:1/916: readlink d0/d14/df/d28/l10e 0 2024-09-16T00:26:11.220 INFO:tasks.workunit.client.1.smithi169.stdout:0/681: creat d0/d2/d8/d5/d79/feb x:0 0 0 2024-09-16T00:26:11.221 INFO:tasks.workunit.client.0.smithi112.stdout:5/845: link d4/d4f/d77/fa2 d4/d7/d26/d4c/f11d 0 2024-09-16T00:26:11.222 INFO:tasks.workunit.client.1.smithi169.stdout:9/762: truncate d5/da/d5b/d91/dc9/f53 857461 0 2024-09-16T00:26:11.223 INFO:tasks.workunit.client.0.smithi112.stdout:3/977: stat d2/d9/d2b/d47/d52/d7a/d80/d83/d7b/ca4 0 2024-09-16T00:26:11.224 INFO:tasks.workunit.client.1.smithi169.stdout:5/567: fsync d2/da/d10/d13/d1c/d32/d53/d83/fa4 0 2024-09-16T00:26:11.225 INFO:tasks.workunit.client.0.smithi112.stdout:5/846: write d4/d7/d26/d4c/fe2 [2923283,108701] 0 2024-09-16T00:26:11.226 INFO:tasks.workunit.client.0.smithi112.stdout:1/917: write d0/d14/df/d28/d45/dd4/f8d [3797791,8706] 0 2024-09-16T00:26:11.227 INFO:tasks.workunit.client.1.smithi169.stdout:9/763: dread d5/d49/f5a [0,4194304] 0 2024-09-16T00:26:11.228 INFO:tasks.workunit.client.1.smithi169.stdout:3/780: dwrite d7/f10 [0,4194304] 0 2024-09-16T00:26:11.228 INFO:tasks.workunit.client.0.smithi112.stdout:3/978: creat d2/d9/d2b/d47/d52/d7a/d80/db4/f140 x:0 0 0 2024-09-16T00:26:11.229 INFO:tasks.workunit.client.0.smithi112.stdout:7/926: dwrite d6/d10/d16/d3b/f5e [0,4194304] 0 2024-09-16T00:26:11.229 INFO:tasks.workunit.client.0.smithi112.stdout:1/918: symlink d0/d14/df/d28/d45/d30/l12e 0 2024-09-16T00:26:11.232 INFO:tasks.workunit.client.1.smithi169.stdout:6/882: rename d2/d3f/d41/d72/d9e/fdd to d2/d3f/d41/d72/d9e/f118 0 2024-09-16T00:26:11.232 INFO:tasks.workunit.client.0.smithi112.stdout:7/927: write d6/d7f/dc3/f12d [2725213,2678] 0 2024-09-16T00:26:11.233 INFO:tasks.workunit.client.1.smithi169.stdout:5/568: write d2/da/d10/d13/d1c/d94/faa [1753261,117858] 0 2024-09-16T00:26:11.233 INFO:tasks.workunit.client.1.smithi169.stdout:9/764: symlink d5/d49/d64/l10b 0 2024-09-16T00:26:11.234 INFO:tasks.workunit.client.0.smithi112.stdout:7/928: stat d6/d10/d16/l73 0 2024-09-16T00:26:11.234 INFO:tasks.workunit.client.1.smithi169.stdout:6/883: read d2/f64 [421972,114860] 0 2024-09-16T00:26:11.235 INFO:tasks.workunit.client.1.smithi169.stdout:0/682: creat d0/d2c/d38/de4/fec x:0 0 0 2024-09-16T00:26:11.237 INFO:tasks.workunit.client.0.smithi112.stdout:5/847: link d4/dd/f10f d4/d7/d14/d90/dd3/de5/f11e 0 2024-09-16T00:26:11.237 INFO:tasks.workunit.client.0.smithi112.stdout:1/919: mknod d0/d14/df/d28/d45/d46/d58/d62/db0/c12f 0 2024-09-16T00:26:11.238 INFO:tasks.workunit.client.0.smithi112.stdout:7/929: dread - d6/d10/d75/d8f/d90/dbc/fe6 zero size 2024-09-16T00:26:11.238 INFO:tasks.workunit.client.1.smithi169.stdout:5/569: write d2/da/f19 [1427925,42886] 0 2024-09-16T00:26:11.239 INFO:tasks.workunit.client.0.smithi112.stdout:3/979: dwrite d2/d9/d2b/d47/d52/d7a/d80/d83/d76/d8e/f104 [0,4194304] 0 2024-09-16T00:26:11.242 INFO:tasks.workunit.client.1.smithi169.stdout:0/683: write d0/d2c/d38/f5e [984239,74231] 0 2024-09-16T00:26:11.243 INFO:tasks.workunit.client.1.smithi169.stdout:9/765: truncate d5/f35 276909 0 2024-09-16T00:26:11.245 INFO:tasks.workunit.client.1.smithi169.stdout:3/781: dread d7/d50/d90/dba/fbf [0,4194304] 0 2024-09-16T00:26:11.248 INFO:tasks.workunit.client.1.smithi169.stdout:6/884: rename d2/d22/l69 to d2/d3f/d41/d5c/d91/l119 0 2024-09-16T00:26:11.249 INFO:tasks.workunit.client.0.smithi112.stdout:7/930: creat d6/d10/d75/d8f/d90/ddc/dee/f134 x:0 0 0 2024-09-16T00:26:11.249 INFO:tasks.workunit.client.0.smithi112.stdout:3/980: rename d2/d9/d2b/d47/d52/d7a/d80/d83/d68/d90/dcb/leb to d2/d9/d2b/d47/d52/d7a/d80/d83/d49/l141 0 2024-09-16T00:26:11.251 INFO:tasks.workunit.client.1.smithi169.stdout:6/885: chown d2/d3f/d41/d52/f1b 5266091 1 2024-09-16T00:26:11.253 INFO:tasks.workunit.client.1.smithi169.stdout:0/684: truncate d0/d2c/d38/de4/f3b 3140403 0 2024-09-16T00:26:11.253 INFO:tasks.workunit.client.0.smithi112.stdout:4/963: write d6/d7c/da0/f117 [1532632,99167] 0 2024-09-16T00:26:11.254 INFO:tasks.workunit.client.0.smithi112.stdout:7/931: symlink d6/d7f/l135 0 2024-09-16T00:26:11.254 INFO:tasks.workunit.client.1.smithi169.stdout:8/728: dread d7/dc/d17/d24/d32/d8d/fc8 [0,4194304] 0 2024-09-16T00:26:11.255 INFO:tasks.workunit.client.1.smithi169.stdout:3/782: getdents d7/d1a/d1c/d23/d88/dd7 0 2024-09-16T00:26:11.256 INFO:tasks.workunit.client.1.smithi169.stdout:0/685: chown d0/d2/d8/d5/f44 194457 1 2024-09-16T00:26:11.256 INFO:tasks.workunit.client.0.smithi112.stdout:1/920: dwrite d0/d14/df/d28/d45/dd4/d41/de0/ff5 [0,4194304] 0 2024-09-16T00:26:11.258 INFO:tasks.workunit.client.1.smithi169.stdout:8/729: chown d7/dc/d17/fa8 121 1 2024-09-16T00:26:11.258 INFO:tasks.workunit.client.0.smithi112.stdout:4/964: chown d6/db/df/c2d 261 1 2024-09-16T00:26:11.258 INFO:tasks.workunit.client.0.smithi112.stdout:5/848: dwrite d4/d7/d9b/fc6 [0,4194304] 0 2024-09-16T00:26:11.259 INFO:tasks.workunit.client.1.smithi169.stdout:3/783: creat d7/d1a/d1c/d35/d63/da3/f116 x:0 0 0 2024-09-16T00:26:11.260 INFO:tasks.workunit.client.1.smithi169.stdout:8/730: write d7/dc/d26/f9d [809663,122973] 0 2024-09-16T00:26:11.262 INFO:tasks.workunit.client.1.smithi169.stdout:5/570: dread d2/da/d10/d13/d1c/d32/f8f [0,4194304] 0 2024-09-16T00:26:11.263 INFO:tasks.workunit.client.0.smithi112.stdout:4/965: dread - d6/d39/d36/f128 zero size 2024-09-16T00:26:11.264 INFO:tasks.workunit.client.0.smithi112.stdout:3/981: dread d2/d9/d2b/d39/fbb [0,4194304] 0 2024-09-16T00:26:11.264 INFO:tasks.workunit.client.0.smithi112.stdout:5/849: symlink d4/d1c/d43/l11f 0 2024-09-16T00:26:11.266 INFO:tasks.workunit.client.1.smithi169.stdout:5/571: chown d2/da/d10/d13/d24/laf 21 1 2024-09-16T00:26:11.266 INFO:tasks.workunit.client.0.smithi112.stdout:8/805: sync 2024-09-16T00:26:11.267 INFO:tasks.workunit.client.1.smithi169.stdout:3/784: read d7/d1a/d1c/d35/d63/d37/f59 [1549257,129725] 0 2024-09-16T00:26:11.269 INFO:tasks.workunit.client.1.smithi169.stdout:4/648: dread d2/d12/d13/d1f/f48 [0,4194304] 0 2024-09-16T00:26:11.269 INFO:tasks.workunit.client.1.smithi169.stdout:0/686: truncate d0/d7d/fa8 855948 0 2024-09-16T00:26:11.270 INFO:tasks.workunit.client.0.smithi112.stdout:5/850: mknod d4/dd/d49/d70/d98/c120 0 2024-09-16T00:26:11.270 INFO:tasks.workunit.client.0.smithi112.stdout:7/932: dwrite d6/d10/da1/f11a [0,4194304] 0 2024-09-16T00:26:11.270 INFO:tasks.workunit.client.1.smithi169.stdout:3/785: write d7/d1a/d2b/dc6/dff/f10f [251426,105614] 0 2024-09-16T00:26:11.271 INFO:tasks.workunit.client.0.smithi112.stdout:4/966: readlink d6/db/d1f/d41/d49/l8e 0 2024-09-16T00:26:11.272 INFO:tasks.workunit.client.1.smithi169.stdout:5/572: truncate d2/da/d10/d13/d1c/d32/f8f 1448027 0 2024-09-16T00:26:11.272 INFO:tasks.workunit.client.1.smithi169.stdout:0/687: chown d0/d2c/d38/de4/f28 3806 1 2024-09-16T00:26:11.273 INFO:tasks.workunit.client.1.smithi169.stdout:9/766: write d5/da/d2e/f43 [3487183,31626] 0 2024-09-16T00:26:11.274 INFO:tasks.workunit.client.1.smithi169.stdout:3/786: dread - d7/d1a/d1c/d35/d63/da3/f116 zero size 2024-09-16T00:26:11.274 INFO:tasks.workunit.client.1.smithi169.stdout:4/649: write d2/d12/d13/d4e/fc1 [268116,89765] 0 2024-09-16T00:26:11.275 INFO:tasks.workunit.client.0.smithi112.stdout:8/806: dread d10/d12/d1c/d61/d6c/dd1/d58/f9b [0,4194304] 0 2024-09-16T00:26:11.277 INFO:tasks.workunit.client.0.smithi112.stdout:4/967: creat d6/db/d1f/d94/d10c/f166 x:0 0 0 2024-09-16T00:26:11.278 INFO:tasks.workunit.client.1.smithi169.stdout:0/688: truncate d0/d2/d15/f7a 848055 0 2024-09-16T00:26:11.278 INFO:tasks.workunit.client.0.smithi112.stdout:8/807: truncate d10/d12/d1c/d61/d6c/fc4 5135003 0 2024-09-16T00:26:11.278 INFO:tasks.workunit.client.0.smithi112.stdout:4/968: write d6/d39/d1c/d6e/d13a/f155 [243329,115760] 0 2024-09-16T00:26:11.281 INFO:tasks.workunit.client.1.smithi169.stdout:0/689: rmdir d0/d2c/d38/de4 39 2024-09-16T00:26:11.282 INFO:tasks.workunit.client.0.smithi112.stdout:3/982: rename d2/d9/d2b/d47/d52/d7a/d80/d83/d10b/f11f to d2/d9/d2b/d47/d52/d7a/df3/f142 0 2024-09-16T00:26:11.282 INFO:tasks.workunit.client.1.smithi169.stdout:9/767: mknod d5/da/d21/de4/c10c 0 2024-09-16T00:26:11.283 INFO:tasks.workunit.client.0.smithi112.stdout:8/808: creat d10/de9/f10c x:0 0 0 2024-09-16T00:26:11.284 INFO:tasks.workunit.client.1.smithi169.stdout:4/650: creat d2/d12/d13/d4e/d50/fce x:0 0 0 2024-09-16T00:26:11.285 INFO:tasks.workunit.client.1.smithi169.stdout:5/573: rmdir d2/da/da3 0 2024-09-16T00:26:11.285 INFO:tasks.workunit.client.0.smithi112.stdout:4/969: mkdir d6/db/d1f/d94/d167 0 2024-09-16T00:26:11.286 INFO:tasks.workunit.client.1.smithi169.stdout:9/768: write d5/da/d5b/d6c/fb1 [1128497,46332] 0 2024-09-16T00:26:11.287 INFO:tasks.workunit.client.1.smithi169.stdout:4/651: stat d2/d12/d13/d1f/d3e/f53 0 2024-09-16T00:26:11.287 INFO:tasks.workunit.client.1.smithi169.stdout:3/787: rename d7/d1a/d1c/d35/fa5 to d7/d1a/d1c/d35/d63/d37/f117 0 2024-09-16T00:26:11.288 INFO:tasks.workunit.client.0.smithi112.stdout:8/809: symlink d10/d12/d1c/l10d 0 2024-09-16T00:26:11.289 INFO:tasks.workunit.client.1.smithi169.stdout:3/788: write d7/d1a/d1c/f78 [2840013,123298] 0 2024-09-16T00:26:11.291 INFO:tasks.workunit.client.1.smithi169.stdout:9/769: mknod d5/d49/d64/c10d 0 2024-09-16T00:26:11.292 INFO:tasks.workunit.client.0.smithi112.stdout:8/810: mkdir d10/d12/d1c/d61/d6c/dd1/d8c/d7c/da5/d10e 0 2024-09-16T00:26:11.293 INFO:tasks.workunit.client.1.smithi169.stdout:9/770: dread - d5/da/d5b/fe8 zero size 2024-09-16T00:26:11.293 INFO:tasks.workunit.client.0.smithi112.stdout:3/983: dwrite d2/d9/d2b/d47/d52/d7a/d80/d83/d7b/fcc [0,4194304] 0 2024-09-16T00:26:11.294 INFO:tasks.workunit.client.1.smithi169.stdout:0/690: rename d0/l11 to d0/d2c/d38/de4/d56/d67/dd1/d6c/led 0 2024-09-16T00:26:11.294 INFO:tasks.workunit.client.0.smithi112.stdout:8/811: read - d10/d12/d1c/d96/fac zero size 2024-09-16T00:26:11.294 INFO:tasks.workunit.client.0.smithi112.stdout:3/984: fdatasync d2/d9/d2b/f4b 0 2024-09-16T00:26:11.298 INFO:tasks.workunit.client.1.smithi169.stdout:0/691: stat d0/d2c/d38/de4/d21/fa0 0 2024-09-16T00:26:11.299 INFO:tasks.workunit.client.1.smithi169.stdout:9/771: dread - d5/da/d5b/d91/d9a/ff7 zero size 2024-09-16T00:26:11.299 INFO:tasks.workunit.client.1.smithi169.stdout:5/574: dwrite d2/da/d10/d13/d1c/d32/d53/dae/fc4 [0,4194304] 0 2024-09-16T00:26:11.300 INFO:tasks.workunit.client.0.smithi112.stdout:1/921: dread d0/d14/df/d28/d45/d30/fc9 [4194304,4194304] 0 2024-09-16T00:26:11.300 INFO:tasks.workunit.client.1.smithi169.stdout:6/886: dread d2/d3f/d41/d5c/f94 [0,4194304] 0 2024-09-16T00:26:11.301 INFO:tasks.workunit.client.1.smithi169.stdout:3/789: rename d7/d1a/d2b to d7/d50/d6f/d113/d118 0 2024-09-16T00:26:11.302 INFO:tasks.workunit.client.0.smithi112.stdout:1/922: read - d0/d14/df/d28/d45/dd4/d41/de0/f119 zero size 2024-09-16T00:26:11.302 INFO:tasks.workunit.client.0.smithi112.stdout:5/851: dread d4/d7/d26/d36/f74 [0,4194304] 0 2024-09-16T00:26:11.303 INFO:tasks.workunit.client.1.smithi169.stdout:9/772: chown d5/l2a 174973 1 2024-09-16T00:26:11.303 INFO:tasks.workunit.client.0.smithi112.stdout:5/852: stat d4/d4f/d78/ff9 0 2024-09-16T00:26:11.304 INFO:tasks.workunit.client.0.smithi112.stdout:3/985: getdents d2/d9/d2b/d47/d52/d7a/d80/d83/d76/d8e 0 2024-09-16T00:26:11.306 INFO:tasks.workunit.client.0.smithi112.stdout:5/853: truncate d4/dd/df6/f10a 1017536 0 2024-09-16T00:26:11.308 INFO:tasks.workunit.client.0.smithi112.stdout:8/812: link d10/d12/d1c/d61/d6c/f76 d10/d12/d1c/d61/d6c/dd1/f10f 0 2024-09-16T00:26:11.309 INFO:tasks.workunit.client.1.smithi169.stdout:3/790: creat d7/d1a/d1c/d23/d88/d9b/f119 x:0 0 0 2024-09-16T00:26:11.310 INFO:tasks.workunit.client.0.smithi112.stdout:5/854: mknod d4/d7/d14/dbb/c121 0 2024-09-16T00:26:11.311 INFO:tasks.workunit.client.0.smithi112.stdout:6/877: dread dd/df/d10/d47/d89/dbc/fc5 [0,4194304] 0 2024-09-16T00:26:11.311 INFO:tasks.workunit.client.0.smithi112.stdout:5/855: stat d4/d7/f4d 0 2024-09-16T00:26:11.311 INFO:tasks.workunit.client.1.smithi169.stdout:9/773: readlink d5/da/l20 0 2024-09-16T00:26:11.312 INFO:tasks.workunit.client.0.smithi112.stdout:8/813: unlink d10/d12/c67 0 2024-09-16T00:26:11.312 INFO:tasks.workunit.client.1.smithi169.stdout:0/692: rename d0/d2c/d4f/da5/laa to d0/d2c/d38/de4/d56/d67/dd1/d6c/d9b/lee 0 2024-09-16T00:26:11.314 INFO:tasks.workunit.client.0.smithi112.stdout:5/856: mkdir d4/dd/d49/d70/d122 0 2024-09-16T00:26:11.315 INFO:tasks.workunit.client.1.smithi169.stdout:3/791: write d7/d50/d90/da6/fbb [2291194,56066] 0 2024-09-16T00:26:11.315 INFO:tasks.workunit.client.0.smithi112.stdout:5/857: chown d4/dd/d49/d70/d122 28723 1 2024-09-16T00:26:11.316 INFO:tasks.workunit.client.0.smithi112.stdout:8/814: rmdir d10/d12/d1c/d61/d6c/dd1/d58/d87/df0 39 2024-09-16T00:26:11.316 INFO:tasks.workunit.client.1.smithi169.stdout:6/887: creat d2/f11a x:0 0 0 2024-09-16T00:26:11.317 INFO:tasks.workunit.client.1.smithi169.stdout:5/575: dwrite d2/da/d10/f98 [0,4194304] 0 2024-09-16T00:26:11.317 INFO:tasks.workunit.client.0.smithi112.stdout:5/858: chown d4/d7/d26/d4c/c6e 16 1 2024-09-16T00:26:11.318 INFO:tasks.workunit.client.1.smithi169.stdout:5/576: stat d2/l12 0 2024-09-16T00:26:11.318 INFO:tasks.workunit.client.0.smithi112.stdout:5/859: dread - d4/dee/ff8 zero size 2024-09-16T00:26:11.320 INFO:tasks.workunit.client.0.smithi112.stdout:8/815: truncate d10/d12/d1c/d61/ff7 923997 0 2024-09-16T00:26:11.322 INFO:tasks.workunit.client.0.smithi112.stdout:7/933: dwrite d6/d10/d16/d96/dc7/d39/f92 [4194304,4194304] 0 2024-09-16T00:26:11.322 INFO:tasks.workunit.client.1.smithi169.stdout:9/774: rename d5/da/d5b/d91/dc9/lbb to d5/da/d5b/d6c/dcb/db4/l10e 0 2024-09-16T00:26:11.323 INFO:tasks.workunit.client.1.smithi169.stdout:3/792: mkdir d7/d1a/d1c/d35/d63/dea/d11a 0 2024-09-16T00:26:11.325 INFO:tasks.workunit.client.0.smithi112.stdout:7/934: chown d6/d10/d16/d96/dc7/d39/c6f 8394 1 2024-09-16T00:26:11.325 INFO:tasks.workunit.client.1.smithi169.stdout:5/577: unlink d2/da/d67/cbc 0 2024-09-16T00:26:11.326 INFO:tasks.workunit.client.0.smithi112.stdout:7/935: fsync d6/d10/da1/fd6 0 2024-09-16T00:26:11.329 INFO:tasks.workunit.client.1.smithi169.stdout:3/793: symlink d7/d1a/d1c/d35/d63/d37/l11b 0 2024-09-16T00:26:11.331 INFO:tasks.workunit.client.1.smithi169.stdout:3/794: dread - d7/d50/d6f/d113/d118/dc6/ff2 zero size 2024-09-16T00:26:11.331 INFO:tasks.workunit.client.1.smithi169.stdout:9/775: dread d5/da/d21/de4/f9c [0,4194304] 0 2024-09-16T00:26:11.333 INFO:tasks.workunit.client.1.smithi169.stdout:5/578: creat d2/da/d10/d13/d1c/d32/d53/d4e/d89/d9a/dad/fc6 x:0 0 0 2024-09-16T00:26:11.334 INFO:tasks.workunit.client.0.smithi112.stdout:7/936: chown d6/d10/d75/d88/c3c 56 1 2024-09-16T00:26:11.334 INFO:tasks.workunit.client.0.smithi112.stdout:1/923: write d0/d14/d1c/f51 [2983495,78755] 0 2024-09-16T00:26:11.335 INFO:tasks.workunit.client.0.smithi112.stdout:8/816: dwrite d10/d12/d1c/fa4 [0,4194304] 0 2024-09-16T00:26:11.336 INFO:tasks.workunit.client.1.smithi169.stdout:3/795: truncate d7/d1a/d1c/d23/d88/d9b/dc8/dcc/fdd 691950 0 2024-09-16T00:26:11.336 INFO:tasks.workunit.client.1.smithi169.stdout:5/579: readlink d2/da/d10/d13/d1c/d32/d53/d83/d7d/l86 0 2024-09-16T00:26:11.337 INFO:tasks.workunit.client.0.smithi112.stdout:6/878: write dd/d23/fcf [107315,119302] 0 2024-09-16T00:26:11.338 INFO:tasks.workunit.client.0.smithi112.stdout:3/986: dread d2/fb [4194304,4194304] 0 2024-09-16T00:26:11.339 INFO:tasks.workunit.client.0.smithi112.stdout:5/860: write d4/d1c/f32 [842500,51182] 0 2024-09-16T00:26:11.340 INFO:tasks.workunit.client.1.smithi169.stdout:5/580: creat d2/da/d10/d13/d1c/d40/d54/d7f/fc7 x:0 0 0 2024-09-16T00:26:11.340 INFO:tasks.workunit.client.0.smithi112.stdout:7/937: dread - d6/d10/d75/d8f/d90/dd3/ff9 zero size 2024-09-16T00:26:11.343 INFO:tasks.workunit.client.1.smithi169.stdout:5/581: mkdir d2/da/d10/d13/d1c/d94/dc8 0 2024-09-16T00:26:11.344 INFO:tasks.workunit.client.1.smithi169.stdout:3/796: dwrite d7/d1a/d1c/d35/d63/f9c [0,4194304] 0 2024-09-16T00:26:11.344 INFO:tasks.workunit.client.0.smithi112.stdout:4/970: dwrite d6/db/d1f/d41/d49/d31/f102 [0,4194304] 0 2024-09-16T00:26:11.344 INFO:tasks.workunit.client.0.smithi112.stdout:1/924: symlink d0/d14/df/d28/d45/d46/d58/d62/l130 0 2024-09-16T00:26:11.345 INFO:tasks.workunit.client.1.smithi169.stdout:5/582: read d2/da/d10/f7a [128784,13420] 0 2024-09-16T00:26:11.348 INFO:tasks.workunit.client.0.smithi112.stdout:3/987: dread d2/d9/d2b/d39/fbb [0,4194304] 0 2024-09-16T00:26:11.351 INFO:tasks.workunit.client.0.smithi112.stdout:8/817: link d10/d12/d1c/df2/ffc d10/d12/d1c/dd0/f110 0 2024-09-16T00:26:11.351 INFO:tasks.workunit.client.1.smithi169.stdout:5/583: mkdir d2/da/d10/d13/d24/dc9 0 2024-09-16T00:26:11.352 INFO:tasks.workunit.client.1.smithi169.stdout:0/693: dread d0/d2c/d38/de4/d21/d31/fc7 [0,4194304] 0 2024-09-16T00:26:11.352 INFO:tasks.workunit.client.0.smithi112.stdout:3/988: fsync d2/d9/d2b/d47/d52/f38 0 2024-09-16T00:26:11.353 INFO:tasks.workunit.client.0.smithi112.stdout:7/938: rename d6/d10/fcb to d6/d10/d16/db4/f136 0 2024-09-16T00:26:11.355 INFO:tasks.workunit.client.0.smithi112.stdout:6/879: dread dd/df/d10/d47/d55/f7d [0,4194304] 0 2024-09-16T00:26:11.358 INFO:tasks.workunit.client.0.smithi112.stdout:8/818: mknod d10/d12/d1c/d61/d6c/dd1/d8c/d7c/c111 0 2024-09-16T00:26:11.359 INFO:tasks.workunit.client.1.smithi169.stdout:5/584: truncate d2/da/d10/d13/d1c/d32/d53/f92 1359064 0 2024-09-16T00:26:11.359 INFO:tasks.workunit.client.0.smithi112.stdout:7/939: mkdir d6/d10/d16/d96/dc7/dbf/d108/d137 0 2024-09-16T00:26:11.360 INFO:tasks.workunit.client.1.smithi169.stdout:6/888: write d2/d3f/d41/d72/d81/fb4 [2253533,112303] 0 2024-09-16T00:26:11.362 INFO:tasks.workunit.client.0.smithi112.stdout:7/940: mknod d6/d7f/d85/c138 0 2024-09-16T00:26:11.363 INFO:tasks.workunit.client.1.smithi169.stdout:6/889: truncate d2/d4d/d59/d43/fee 790682 0 2024-09-16T00:26:11.363 INFO:tasks.workunit.client.1.smithi169.stdout:5/585: mknod d2/d7c/cca 0 2024-09-16T00:26:11.364 INFO:tasks.workunit.client.1.smithi169.stdout:9/776: write d5/da/d21/de4/f26 [6218625,68971] 0 2024-09-16T00:26:11.364 INFO:tasks.workunit.client.0.smithi112.stdout:6/880: dwrite dd/df/d10/feb [0,4194304] 0 2024-09-16T00:26:11.365 INFO:tasks.workunit.client.0.smithi112.stdout:8/819: rename d10/d12/d1c/dd0/fa0 to d10/de9/f112 0 2024-09-16T00:26:11.367 INFO:tasks.workunit.client.0.smithi112.stdout:6/881: fsync dd/df/d10/d47/d89/fbb 0 2024-09-16T00:26:11.367 INFO:tasks.workunit.client.0.smithi112.stdout:7/941: chown d6/d10/d75/dea/cd5 53641 1 2024-09-16T00:26:11.367 INFO:tasks.workunit.client.0.smithi112.stdout:6/882: stat dd/df/d17/d27/f76 0 2024-09-16T00:26:11.368 INFO:tasks.workunit.client.1.smithi169.stdout:9/777: mkdir d5/da/d2e/d99/d100/d10f 0 2024-09-16T00:26:11.369 INFO:tasks.workunit.client.0.smithi112.stdout:8/820: creat d10/d12/d1c/d61/d6c/dd1/d8c/d7c/da5/d10e/f113 x:0 0 0 2024-09-16T00:26:11.369 INFO:tasks.workunit.client.1.smithi169.stdout:9/778: fsync d5/da/f2b 0 2024-09-16T00:26:11.369 INFO:tasks.workunit.client.0.smithi112.stdout:6/883: mkdir dd/dcc/d11e 0 2024-09-16T00:26:11.369 INFO:tasks.workunit.client.0.smithi112.stdout:7/942: write d6/d10/d75/d8f/d90/dd3/ff9 [266145,8339] 0 2024-09-16T00:26:11.371 INFO:tasks.workunit.client.0.smithi112.stdout:6/884: write dd/d23/fcf [965292,37422] 0 2024-09-16T00:26:11.371 INFO:tasks.workunit.client.0.smithi112.stdout:7/943: stat d6/d10/d75/d8f/d90/ddc/f132 0 2024-09-16T00:26:11.371 INFO:tasks.workunit.client.0.smithi112.stdout:8/821: chown d10/d12/d1c/d61/d6c/c86 664 1 2024-09-16T00:26:11.372 INFO:tasks.workunit.client.0.smithi112.stdout:6/885: write dd/df/d91/f110 [187596,44707] 0 2024-09-16T00:26:11.373 INFO:tasks.workunit.client.1.smithi169.stdout:5/586: dwrite d2/ff [0,4194304] 0 2024-09-16T00:26:11.374 INFO:tasks.workunit.client.0.smithi112.stdout:7/944: mkdir d6/d7f/d85/d139 0 2024-09-16T00:26:11.376 INFO:tasks.workunit.client.0.smithi112.stdout:8/822: mkdir d10/d12/d1c/d114 0 2024-09-16T00:26:11.378 INFO:tasks.workunit.client.0.smithi112.stdout:5/861: write d4/dd/d49/d70/d83/f86 [1889451,38749] 0 2024-09-16T00:26:11.380 INFO:tasks.workunit.client.1.smithi169.stdout:0/694: truncate d0/d2c/d38/de4/d56/d67/fa1 435136 0 2024-09-16T00:26:11.380 INFO:tasks.workunit.client.1.smithi169.stdout:3/797: dwrite d7/d50/d6f/d113/d118/d8d/f94 [0,4194304] 0 2024-09-16T00:26:11.382 INFO:tasks.workunit.client.0.smithi112.stdout:4/971: truncate d6/db/df/d5e/f132 566631 0 2024-09-16T00:26:11.385 INFO:tasks.workunit.client.1.smithi169.stdout:9/779: creat d5/d49/f110 x:0 0 0 2024-09-16T00:26:11.385 INFO:tasks.workunit.client.0.smithi112.stdout:6/886: dwrite dd/df/d13/d18/fd3 [0,4194304] 0 2024-09-16T00:26:11.388 INFO:tasks.workunit.client.1.smithi169.stdout:0/695: read d0/d2/f93 [26290,101197] 0 2024-09-16T00:26:11.389 INFO:tasks.workunit.client.1.smithi169.stdout:5/587: dwrite d2/fd [0,4194304] 0 2024-09-16T00:26:11.389 INFO:tasks.workunit.client.1.smithi169.stdout:9/780: write d5/da/d5b/d6c/fb1 [734638,14689] 0 2024-09-16T00:26:11.389 INFO:tasks.workunit.client.0.smithi112.stdout:6/887: chown dd/df/d11/d44/da4/la6 127826 1 2024-09-16T00:26:11.389 INFO:tasks.workunit.client.0.smithi112.stdout:8/823: write d10/d12/d1c/d61/d6c/dd1/d58/f9b [1554660,62735] 0 2024-09-16T00:26:11.389 INFO:tasks.workunit.client.0.smithi112.stdout:5/862: creat d4/dd/d49/de6/d113/f123 x:0 0 0 2024-09-16T00:26:11.390 INFO:tasks.workunit.client.1.smithi169.stdout:9/781: stat d5/d49/dc0 0 2024-09-16T00:26:11.390 INFO:tasks.workunit.client.1.smithi169.stdout:3/798: unlink d7/d1a/d1c/d23/d88/d9b/dc8/c102 0 2024-09-16T00:26:11.391 INFO:tasks.workunit.client.0.smithi112.stdout:5/863: chown d4/d7/d14/f8c 3 1 2024-09-16T00:26:11.392 INFO:tasks.workunit.client.0.smithi112.stdout:7/945: link d6/d10/d75/d8f/d90/ddc/l12b d6/d10/d75/d8f/d90/l13a 0 2024-09-16T00:26:11.393 INFO:tasks.workunit.client.1.smithi169.stdout:3/799: fsync d7/d50/d90/da6/faa 0 2024-09-16T00:26:11.394 INFO:tasks.workunit.client.0.smithi112.stdout:6/888: getdents dd/dcc/d11e 0 2024-09-16T00:26:11.395 INFO:tasks.workunit.client.1.smithi169.stdout:3/800: write d7/de5/f10b [79340,16843] 0 2024-09-16T00:26:11.395 INFO:tasks.workunit.client.1.smithi169.stdout:5/588: truncate d2/da/d10/d13/d1c/d40/d54/f56 4259804 0 2024-09-16T00:26:11.397 INFO:tasks.workunit.client.0.smithi112.stdout:3/989: dread d2/f24 [0,4194304] 0 2024-09-16T00:26:11.398 INFO:tasks.workunit.client.1.smithi169.stdout:3/801: creat d7/d50/d6f/f11c x:0 0 0 2024-09-16T00:26:11.398 INFO:tasks.workunit.client.0.smithi112.stdout:4/972: mkdir d6/d39/d1c/d6e/d6f/d11b/daa/d156/dc6/d168 0 2024-09-16T00:26:11.398 INFO:tasks.workunit.client.0.smithi112.stdout:5/864: creat d4/dd/d49/de6/f124 x:0 0 0 2024-09-16T00:26:11.399 INFO:tasks.workunit.client.0.smithi112.stdout:8/824: creat d10/d12/f115 x:0 0 0 2024-09-16T00:26:11.400 INFO:tasks.workunit.client.0.smithi112.stdout:5/865: write f2 [1586468,67047] 0 2024-09-16T00:26:11.402 INFO:tasks.workunit.client.0.smithi112.stdout:4/973: mknod d6/db/d1f/c169 0 2024-09-16T00:26:11.402 INFO:tasks.workunit.client.0.smithi112.stdout:8/825: write d10/d12/d24/f2b [1323342,19220] 0 2024-09-16T00:26:11.402 INFO:tasks.workunit.client.0.smithi112.stdout:7/946: read d6/d7f/f89 [161473,52874] 0 2024-09-16T00:26:11.403 INFO:tasks.workunit.client.1.smithi169.stdout:3/802: truncate d7/d1a/d1c/d23/d70/f95 104190 0 2024-09-16T00:26:11.405 INFO:tasks.workunit.client.0.smithi112.stdout:5/866: chown d4/dd/d12/d52/f106 201 1 2024-09-16T00:26:11.406 INFO:tasks.workunit.client.0.smithi112.stdout:4/974: dread - d6/d39/d1c/d6e/d6f/dda/f15f zero size 2024-09-16T00:26:11.406 INFO:tasks.workunit.client.1.smithi169.stdout:3/803: unlink d7/d50/d6f/d113/d118/cad 0 2024-09-16T00:26:11.408 INFO:tasks.workunit.client.0.smithi112.stdout:5/867: write d4/d1c/fea [304206,59841] 0 2024-09-16T00:26:11.408 INFO:tasks.workunit.client.1.smithi169.stdout:3/804: truncate d7/d1a/d1c/d23/d88/dbe/f110 71757 0 2024-09-16T00:26:11.408 INFO:tasks.workunit.client.0.smithi112.stdout:4/975: chown d6/db/d1f/d94/d10c 1564 1 2024-09-16T00:26:11.409 INFO:tasks.workunit.client.0.smithi112.stdout:5/868: fsync d4/d4f/d77/dcc/f11a 0 2024-09-16T00:26:11.410 INFO:tasks.workunit.client.0.smithi112.stdout:3/990: dwrite d2/d9/d2b/d47/d52/d7a/d80/d83/d49/f77 [0,4194304] 0 2024-09-16T00:26:11.411 INFO:tasks.workunit.client.1.smithi169.stdout:3/805: rename d7/d50/d6f/d113/d118/l9d to d7/d1a/d1c/d23/d70/l11d 0 2024-09-16T00:26:11.412 INFO:tasks.workunit.client.0.smithi112.stdout:4/976: symlink d6/d39/d1c/d66/d12d/l16a 0 2024-09-16T00:26:11.413 INFO:tasks.workunit.client.0.smithi112.stdout:3/991: truncate d2/d9/d2b/d39/fb1 906410 0 2024-09-16T00:26:11.414 INFO:tasks.workunit.client.1.smithi169.stdout:3/806: creat d7/d1a/d1c/d35/d63/dc2/dc7/f11e x:0 0 0 2024-09-16T00:26:11.415 INFO:tasks.workunit.client.1.smithi169.stdout:3/807: dread - d7/f8f zero size 2024-09-16T00:26:11.418 INFO:tasks.workunit.client.1.smithi169.stdout:3/808: creat d7/d50/d90/dba/f11f x:0 0 0 2024-09-16T00:26:11.418 INFO:tasks.workunit.client.0.smithi112.stdout:4/977: truncate d6/d39/d36/f162 528601 0 2024-09-16T00:26:11.419 INFO:tasks.workunit.client.0.smithi112.stdout:3/992: rmdir d2/d9/d2b/d47/d52/d7a/d80/d95 39 2024-09-16T00:26:11.420 INFO:tasks.workunit.client.1.smithi169.stdout:9/782: write d5/da/d5b/d91/dc9/f6d [2264474,93008] 0 2024-09-16T00:26:11.421 INFO:tasks.workunit.client.0.smithi112.stdout:5/869: dwrite d4/dee/f100 [0,4194304] 0 2024-09-16T00:26:11.423 INFO:tasks.workunit.client.0.smithi112.stdout:7/947: dread d6/d10/d16/d96/f9f [0,4194304] 0 2024-09-16T00:26:11.424 INFO:tasks.workunit.client.1.smithi169.stdout:9/783: fsync d5/da/d5b/fdd 0 2024-09-16T00:26:11.425 INFO:tasks.workunit.client.0.smithi112.stdout:5/870: write d4/d7/d26/d4c/fe2 [4072578,114964] 0 2024-09-16T00:26:11.430 INFO:tasks.workunit.client.1.smithi169.stdout:5/589: fsync d2/da/d10/d13/d1c/d32/f8f 0 2024-09-16T00:26:11.430 INFO:tasks.workunit.client.0.smithi112.stdout:3/993: symlink d2/d9/d2b/d47/d52/d7a/d80/d83/d7b/dad/d107/d69/dd5/l143 0 2024-09-16T00:26:11.431 INFO:tasks.workunit.client.1.smithi169.stdout:0/696: dwrite d0/d2c/d38/de4/d21/f8e [0,4194304] 0 2024-09-16T00:26:11.432 INFO:tasks.workunit.client.1.smithi169.stdout:9/784: creat d5/da/df3/df8/f111 x:0 0 0 2024-09-16T00:26:11.432 INFO:tasks.workunit.client.1.smithi169.stdout:0/697: dread - d0/d2c/d38/de4/fe7 zero size 2024-09-16T00:26:11.433 INFO:tasks.workunit.client.1.smithi169.stdout:3/809: read d7/d50/d6f/d113/d118/f8c [545165,27174] 0 2024-09-16T00:26:11.434 INFO:tasks.workunit.client.1.smithi169.stdout:9/785: dread - d5/da/d5b/f63 zero size 2024-09-16T00:26:11.435 INFO:tasks.workunit.client.0.smithi112.stdout:7/948: dwrite d6/fae [0,4194304] 0 2024-09-16T00:26:11.436 INFO:tasks.workunit.client.1.smithi169.stdout:9/786: fsync d5/da/d5b/d6c/fb1 0 2024-09-16T00:26:11.439 INFO:tasks.workunit.client.1.smithi169.stdout:5/590: truncate d2/f58 41619 0 2024-09-16T00:26:11.439 INFO:tasks.workunit.client.0.smithi112.stdout:6/889: dread dd/df/d10/d47/d89/fbb [0,4194304] 0 2024-09-16T00:26:11.440 INFO:tasks.workunit.client.0.smithi112.stdout:3/994: symlink d2/d9/d2b/d47/d52/d7a/d80/d83/l144 0 2024-09-16T00:26:11.445 INFO:tasks.workunit.client.1.smithi169.stdout:9/787: symlink d5/da/d21/d23/l112 0 2024-09-16T00:26:11.445 INFO:tasks.workunit.client.1.smithi169.stdout:3/810: write d7/d1a/fd1 [531583,122664] 0 2024-09-16T00:26:11.446 INFO:tasks.workunit.client.0.smithi112.stdout:7/949: symlink d6/d10/d16/d96/dc7/dbf/d108/d137/l13b 0 2024-09-16T00:26:11.446 INFO:tasks.workunit.client.1.smithi169.stdout:5/591: rename d2/da/c17 to d2/da/d10/d13/d1c/d94/dc8/ccb 0 2024-09-16T00:26:11.446 INFO:tasks.workunit.client.1.smithi169.stdout:9/788: chown d5/da/d2e/d47/fdf 954889825 1 2024-09-16T00:26:11.447 INFO:tasks.workunit.client.1.smithi169.stdout:0/698: dread d0/d2c/d38/de4/d21/d31/d83/fce [0,4194304] 0 2024-09-16T00:26:11.449 INFO:tasks.workunit.client.1.smithi169.stdout:0/699: chown d0/d2/d4b/f52 91 1 2024-09-16T00:26:11.449 INFO:tasks.workunit.client.0.smithi112.stdout:7/950: stat d6/d7f/d85/c138 0 2024-09-16T00:26:11.450 INFO:tasks.workunit.client.0.smithi112.stdout:3/995: fsync d2/d9/d2b/d39/f3c 0 2024-09-16T00:26:11.450 INFO:tasks.workunit.client.1.smithi169.stdout:3/811: symlink d7/d50/d6f/d113/d118/dc6/dff/l120 0 2024-09-16T00:26:11.450 INFO:tasks.workunit.client.0.smithi112.stdout:7/951: rmdir d6/d10/d75/d8f/d90/dbc 39 2024-09-16T00:26:11.451 INFO:tasks.workunit.client.1.smithi169.stdout:9/789: creat d5/da/dd0/f113 x:0 0 0 2024-09-16T00:26:11.452 INFO:tasks.workunit.client.1.smithi169.stdout:9/790: stat d5/da/d21/d23/fc7 0 2024-09-16T00:26:11.453 INFO:tasks.workunit.client.1.smithi169.stdout:5/592: unlink d2/da/d10/d13/d1c/d40/c48 0 2024-09-16T00:26:11.454 INFO:tasks.workunit.client.0.smithi112.stdout:4/978: dread d6/db/f93 [0,4194304] 0 2024-09-16T00:26:11.455 INFO:tasks.workunit.client.1.smithi169.stdout:3/812: creat d7/d50/d6f/d113/d118/d8d/f121 x:0 0 0 2024-09-16T00:26:11.457 INFO:tasks.workunit.client.0.smithi112.stdout:4/979: fsync d6/ff3 0 2024-09-16T00:26:11.458 INFO:tasks.workunit.client.1.smithi169.stdout:9/791: unlink d5/da/d21/lfb 0 2024-09-16T00:26:11.460 INFO:tasks.workunit.client.0.smithi112.stdout:3/996: dwrite d2/d9/d2b/d47/d78/da9/ff7 [0,4194304] 0 2024-09-16T00:26:11.462 INFO:tasks.workunit.client.0.smithi112.stdout:4/980: symlink d6/d39/d1c/d2c/d69/d7b/l16b 0 2024-09-16T00:26:11.462 INFO:tasks.workunit.client.1.smithi169.stdout:9/792: creat d5/da/d5b/d91/db3/f114 x:0 0 0 2024-09-16T00:26:11.463 INFO:tasks.workunit.client.1.smithi169.stdout:5/593: dwrite d2/da/d10/d13/d1c/d40/fb5 [0,4194304] 0 2024-09-16T00:26:11.466 INFO:tasks.workunit.client.1.smithi169.stdout:9/793: mkdir d5/da/d5b/d91/d115 0 2024-09-16T00:26:11.467 INFO:tasks.workunit.client.1.smithi169.stdout:5/594: read d2/da/d10/d13/d1c/d32/d53/d83/d7d/fa1 [3963369,128956] 0 2024-09-16T00:26:11.468 INFO:tasks.workunit.client.1.smithi169.stdout:9/794: mknod d5/c116 0 2024-09-16T00:26:11.469 INFO:tasks.workunit.client.0.smithi112.stdout:3/997: creat d2/d9/d2b/d47/d52/d7a/d80/d83/d68/f145 x:0 0 0 2024-09-16T00:26:11.470 INFO:tasks.workunit.client.1.smithi169.stdout:9/795: chown d5/da/d5b/d91/dc9/f81 0 1 2024-09-16T00:26:11.473 INFO:tasks.workunit.client.1.smithi169.stdout:9/796: rmdir d5/da/d2e/d47 39 2024-09-16T00:26:11.473 INFO:tasks.workunit.client.0.smithi112.stdout:4/981: dwrite d6/d39/d36/f162 [0,4194304] 0 2024-09-16T00:26:11.475 INFO:tasks.workunit.client.1.smithi169.stdout:5/595: dread d2/da/d10/d13/d1c/d32/d53/d83/d7d/fa1 [0,4194304] 0 2024-09-16T00:26:11.476 INFO:tasks.workunit.client.1.smithi169.stdout:9/797: mknod d5/da/d73/c117 0 2024-09-16T00:26:11.477 INFO:tasks.workunit.client.0.smithi112.stdout:7/952: write d6/d10/d16/d96/dc7/dbf/d117/d6c/f38 [524463,94513] 0 2024-09-16T00:26:11.477 INFO:tasks.workunit.client.1.smithi169.stdout:5/596: read d2/da/d10/d13/d1c/d2d/f57 [1534807,99834] 0 2024-09-16T00:26:11.478 INFO:tasks.workunit.client.0.smithi112.stdout:6/890: dread dd/d23/f35 [0,4194304] 0 2024-09-16T00:26:11.479 INFO:tasks.workunit.client.0.smithi112.stdout:8/826: dread d10/d12/d1c/d61/d6c/dd1/d58/f66 [0,4194304] 0 2024-09-16T00:26:11.479 INFO:tasks.workunit.client.1.smithi169.stdout:9/798: creat d5/da/d73/f118 x:0 0 0 2024-09-16T00:26:11.482 INFO:tasks.workunit.client.1.smithi169.stdout:0/700: dread d0/d2c/d38/de4/d21/fa0 [0,4194304] 0 2024-09-16T00:26:11.483 INFO:tasks.workunit.client.0.smithi112.stdout:5/871: dwrite d4/d7/d26/d4c/f11d [0,4194304] 0 2024-09-16T00:26:11.485 INFO:tasks.workunit.client.1.smithi169.stdout:3/813: write d7/d1a/d1c/d23/f3a [1112,76818] 0 2024-09-16T00:26:11.489 INFO:tasks.workunit.client.1.smithi169.stdout:9/799: creat d5/da/d2e/d47/f119 x:0 0 0 2024-09-16T00:26:11.490 INFO:tasks.workunit.client.0.smithi112.stdout:3/998: rename d2/d9/d2b/d47/d52/d7a/d80/d83/d7b/dad/d107 to d2/d9/d2b/d47/d52/d7a/d80/d83/d68/d90/dcb/d146 0 2024-09-16T00:26:11.491 INFO:tasks.workunit.client.0.smithi112.stdout:5/872: creat d4/d7/d87/f125 x:0 0 0 2024-09-16T00:26:11.492 INFO:tasks.workunit.client.1.smithi169.stdout:0/701: mknod d0/d2c/d38/de4/d56/d67/dd1/d6c/cef 0 2024-09-16T00:26:11.492 INFO:tasks.workunit.client.0.smithi112.stdout:3/999: write d2/d9/f4e [5600990,48300] 0 2024-09-16T00:26:11.492 INFO:tasks.workunit.client.1.smithi169.stdout:0/702: chown d0/d2/d4b/le1 1714 1 2024-09-16T00:26:11.493 INFO:tasks.workunit.client.0.smithi112.stdout:7/953: creat d6/d10/d75/d88/f13c x:0 0 0 2024-09-16T00:26:11.494 INFO:tasks.workunit.client.0.smithi112.stdout:5/873: write d4/d7/d14/fb0 [334777,13577] 0 2024-09-16T00:26:11.494 INFO:tasks.workunit.client.1.smithi169.stdout:3/814: creat d7/d50/d6f/d113/d118/d8d/f122 x:0 0 0 2024-09-16T00:26:11.496 INFO:tasks.workunit.client.0.smithi112.stdout:7/954: creat d6/d10/da1/da2/f13d x:0 0 0 2024-09-16T00:26:11.497 INFO:tasks.workunit.client.1.smithi169.stdout:9/800: rename d5/da/d5b/d6f to d5/d49/dc0/d11a 0 2024-09-16T00:26:11.497 INFO:tasks.workunit.client.0.smithi112.stdout:4/982: dread d6/da/f84 [0,4194304] 0 2024-09-16T00:26:11.498 INFO:tasks.workunit.client.0.smithi112.stdout:5/874: rename d4/dd/d6f/f116 to d4/dd/d6f/f126 0 2024-09-16T00:26:11.498 INFO:tasks.workunit.client.0.smithi112.stdout:6/891: write dd/df/d17/f59 [972937,66651] 0 2024-09-16T00:26:11.499 INFO:tasks.workunit.client.1.smithi169.stdout:0/703: creat d0/d2c/d38/de4/d21/d31/d83/dac/ff0 x:0 0 0 2024-09-16T00:26:11.500 INFO:tasks.workunit.client.0.smithi112.stdout:5/875: fdatasync d4/d1c/f111 0 2024-09-16T00:26:11.500 INFO:tasks.workunit.client.0.smithi112.stdout:4/983: chown d6/d39/d1c/d6e/fbe 10589 1 2024-09-16T00:26:11.501 INFO:tasks.workunit.client.0.smithi112.stdout:7/955: unlink d6/d10/d16/d96/dc7/dbf/d117/d6c/f95 0 2024-09-16T00:26:11.503 INFO:tasks.workunit.client.0.smithi112.stdout:4/984: mkdir d6/d39/d16c 0 2024-09-16T00:26:11.504 INFO:tasks.workunit.client.1.smithi169.stdout:9/801: dwrite d5/da/d5b/fac [4194304,4194304] 0 2024-09-16T00:26:11.504 INFO:tasks.workunit.client.0.smithi112.stdout:6/892: mkdir dd/df/d10/d47/d55/dbd/d11f 0 2024-09-16T00:26:11.504 INFO:tasks.workunit.client.0.smithi112.stdout:5/876: rmdir d4/dd/d49/d70/d83 39 2024-09-16T00:26:11.506 INFO:tasks.workunit.client.0.smithi112.stdout:7/956: symlink d6/d10/da1/dcd/l13e 0 2024-09-16T00:26:11.507 INFO:tasks.workunit.client.0.smithi112.stdout:8/827: dwrite d10/d12/d1c/dd0/f78 [0,4194304] 0 2024-09-16T00:26:11.510 INFO:tasks.workunit.client.1.smithi169.stdout:5/597: write d2/da/d10/d13/f35 [4487276,57667] 0 2024-09-16T00:26:11.513 INFO:tasks.workunit.client.1.smithi169.stdout:5/598: write d2/da/d10/d13/d1c/d94/db4/fb6 [193434,85007] 0 2024-09-16T00:26:11.514 INFO:tasks.workunit.client.0.smithi112.stdout:7/957: symlink d6/d10/d16/d96/dc7/d39/l13f 0 2024-09-16T00:26:11.514 INFO:tasks.workunit.client.1.smithi169.stdout:0/704: write d0/d2c/d38/de4/fab [421704,21909] 0 2024-09-16T00:26:11.516 INFO:tasks.workunit.client.0.smithi112.stdout:6/893: link dd/df/d11/fc8 dd/d15/da5/dae/f120 0 2024-09-16T00:26:11.517 INFO:tasks.workunit.client.0.smithi112.stdout:5/877: dwrite d4/d7/d87/fe4 [0,4194304] 0 2024-09-16T00:26:11.517 INFO:tasks.workunit.client.0.smithi112.stdout:8/828: truncate d10/f1f 1567686 0 2024-09-16T00:26:11.518 INFO:tasks.workunit.client.1.smithi169.stdout:5/599: symlink d2/da/d10/d13/d1c/d40/d54/lcc 0 2024-09-16T00:26:11.518 INFO:tasks.workunit.client.1.smithi169.stdout:9/802: dread d5/da/d21/f45 [0,4194304] 0 2024-09-16T00:26:11.521 INFO:tasks.workunit.client.0.smithi112.stdout:6/894: symlink dd/df/d10/d47/d55/l121 0 2024-09-16T00:26:11.522 INFO:tasks.workunit.client.1.smithi169.stdout:3/815: dwrite d7/d1a/d1c/d35/d63/d37/f117 [0,4194304] 0 2024-09-16T00:26:11.523 INFO:tasks.workunit.client.0.smithi112.stdout:5/878: symlink d4/dd/d49/ddb/l127 0 2024-09-16T00:26:11.523 INFO:tasks.workunit.client.1.smithi169.stdout:0/705: unlink d0/d2c/d38/de4/d56/d67/dd1/d6c/led 0 2024-09-16T00:26:11.526 INFO:tasks.workunit.client.1.smithi169.stdout:5/600: getdents d2/da/d10/d13/d1c/d2d/db2 0 2024-09-16T00:26:11.527 INFO:tasks.workunit.client.0.smithi112.stdout:7/958: creat d6/d10/d16/d96/dc7/dbf/d117/f140 x:0 0 0 2024-09-16T00:26:11.527 INFO:tasks.workunit.client.0.smithi112.stdout:6/895: mknod dd/df/d10/d47/c122 0 2024-09-16T00:26:11.530 INFO:tasks.workunit.client.0.smithi112.stdout:6/896: creat dd/df/d91/f123 x:0 0 0 2024-09-16T00:26:11.532 INFO:tasks.workunit.client.0.smithi112.stdout:6/897: mknod dd/df/d10/d47/d55/dbd/c124 0 2024-09-16T00:26:11.533 INFO:tasks.workunit.client.0.smithi112.stdout:6/898: chown dd/df/d10/d47/d89 230 1 2024-09-16T00:26:11.533 INFO:tasks.workunit.client.0.smithi112.stdout:6/899: stat dd/d15/ccd 0 2024-09-16T00:26:11.536 INFO:tasks.workunit.client.0.smithi112.stdout:6/900: truncate dd/df/d10/d47/d55/f7a 894359 0 2024-09-16T00:26:11.536 INFO:tasks.workunit.client.0.smithi112.stdout:6/901: chown dd/df/cd0 18 1 2024-09-16T00:26:11.537 INFO:tasks.workunit.client.0.smithi112.stdout:6/902: mkdir dd/d23/d8f/d125 0 2024-09-16T00:26:11.538 INFO:tasks.workunit.client.0.smithi112.stdout:6/903: chown dd/df/d10/f60 190128311 1 2024-09-16T00:26:11.540 INFO:tasks.workunit.client.0.smithi112.stdout:6/904: getdents dd/d15/da5 0 2024-09-16T00:26:11.541 INFO:tasks.workunit.client.0.smithi112.stdout:6/905: chown dd/df/d10/d47/d55/dbd/dcb/de8 145 1 2024-09-16T00:26:11.542 INFO:tasks.workunit.client.0.smithi112.stdout:6/906: mkdir dd/df/d126 0 2024-09-16T00:26:11.543 INFO:tasks.workunit.client.0.smithi112.stdout:1/925: sync 2024-09-16T00:26:11.548 INFO:tasks.workunit.client.0.smithi112.stdout:7/959: read d6/d10/d16/d96/dc7/dbf/d117/d6c/f33 [319384,67192] 0 2024-09-16T00:26:11.548 INFO:tasks.workunit.client.1.smithi169.stdout:9/803: rename d5/da/d5b/d91/dc9 to d5/da/d2e/d47/d11b 0 2024-09-16T00:26:11.549 INFO:tasks.workunit.client.1.smithi169.stdout:0/706: mknod d0/d2c/d38/de4/d21/d31/d83/cf1 0 2024-09-16T00:26:11.549 INFO:tasks.workunit.client.1.smithi169.stdout:3/816: creat d7/d1a/d1c/d35/d63/d37/f123 x:0 0 0 2024-09-16T00:26:11.553 INFO:tasks.workunit.client.0.smithi112.stdout:7/960: dwrite d6/d10/d16/d3b/f5e [0,4194304] 0 2024-09-16T00:26:11.553 INFO:tasks.workunit.client.1.smithi169.stdout:5/601: creat d2/da/fcd x:0 0 0 2024-09-16T00:26:11.554 INFO:tasks.workunit.client.1.smithi169.stdout:3/817: mkdir d7/d1a/d124 0 2024-09-16T00:26:11.556 INFO:tasks.workunit.client.0.smithi112.stdout:7/961: creat d6/d10/d16/d96/dc7/dbf/d108/d137/f141 x:0 0 0 2024-09-16T00:26:11.558 INFO:tasks.workunit.client.0.smithi112.stdout:7/962: creat d6/f142 x:0 0 0 2024-09-16T00:26:11.560 INFO:tasks.workunit.client.1.smithi169.stdout:3/818: symlink d7/d1a/d1c/d35/d63/dea/df3/l125 0 2024-09-16T00:26:11.560 INFO:tasks.workunit.client.1.smithi169.stdout:0/707: dwrite d0/d2c/d38/de4/d21/d31/fc7 [0,4194304] 0 2024-09-16T00:26:11.562 INFO:tasks.workunit.client.1.smithi169.stdout:5/602: rename d2/da/d10/d13/d1c/d40/d54/d7f/fc7 to d2/da/d10/d13/d1c/d40/d54/fce 0 2024-09-16T00:26:11.564 INFO:tasks.workunit.client.1.smithi169.stdout:0/708: truncate d0/d2/d8/d5/f44 1037852 0 2024-09-16T00:26:11.565 INFO:tasks.workunit.client.1.smithi169.stdout:3/819: mkdir d7/d50/d90/da6/df6/d126 0 2024-09-16T00:26:11.566 INFO:tasks.workunit.client.1.smithi169.stdout:5/603: fsync d2/f58 0 2024-09-16T00:26:11.570 INFO:tasks.workunit.client.1.smithi169.stdout:5/604: symlink d2/da/d10/d13/lcf 0 2024-09-16T00:26:11.602 INFO:tasks.workunit.client.1.smithi169.stdout:3/820: dread d7/d1a/d1c/d23/f3a [0,4194304] 0 2024-09-16T00:26:11.603 INFO:tasks.workunit.client.0.smithi112.stdout:7/963: truncate d6/d10/d75/dea/f112 602015 0 2024-09-16T00:26:11.607 INFO:tasks.workunit.client.0.smithi112.stdout:4/985: write d6/db/df/d5e/d61/d64/d6a/ff9 [675463,48337] 0 2024-09-16T00:26:11.609 INFO:tasks.workunit.client.0.smithi112.stdout:7/964: dwrite d6/d10/d16/d96/dc7/dbf/d108/d137/f141 [0,4194304] 0 2024-09-16T00:26:11.609 INFO:tasks.workunit.client.1.smithi169.stdout:3/821: symlink d7/d1a/d1c/d6c/l127 0 2024-09-16T00:26:11.616 INFO:tasks.workunit.client.1.smithi169.stdout:3/822: mknod d7/d1a/d1c/d35/d63/dea/c128 0 2024-09-16T00:26:11.619 INFO:tasks.workunit.client.1.smithi169.stdout:3/823: mknod d7/d50/d6f/d113/d118/dc6/dff/c129 0 2024-09-16T00:26:11.624 INFO:tasks.workunit.client.1.smithi169.stdout:3/824: rename d7/d1a/d1c/d35/f8a to d7/d1a/d1c/d23/d88/dd3/f12a 0 2024-09-16T00:26:11.625 INFO:tasks.workunit.client.1.smithi169.stdout:3/825: symlink d7/l12b 0 2024-09-16T00:26:11.627 INFO:tasks.workunit.client.1.smithi169.stdout:3/826: write d7/d50/d6f/d113/d118/f8c [3864770,112706] 0 2024-09-16T00:26:11.637 INFO:tasks.workunit.client.1.smithi169.stdout:9/804: write d5/f8f [5197143,88981] 0 2024-09-16T00:26:11.641 INFO:tasks.workunit.client.1.smithi169.stdout:9/805: rename d5/da/d21/d23 to d5/da/d2e/d99/d100/d10f/d11c 0 2024-09-16T00:26:11.643 INFO:tasks.workunit.client.1.smithi169.stdout:9/806: mknod d5/da/d2e/d99/d100/d10f/d11c/c11d 0 2024-09-16T00:26:11.644 INFO:tasks.workunit.client.1.smithi169.stdout:5/605: rmdir d2 39 2024-09-16T00:26:11.645 INFO:tasks.workunit.client.1.smithi169.stdout:0/709: truncate d0/d2c/d38/de4/d56/d67/dd1/f70 256045 0 2024-09-16T00:26:11.646 INFO:tasks.workunit.client.1.smithi169.stdout:9/807: mkdir d5/d49/dc0/dd6/d11e 0 2024-09-16T00:26:11.648 INFO:tasks.workunit.client.1.smithi169.stdout:9/808: mkdir d5/da/dd0/d11f 0 2024-09-16T00:26:11.651 INFO:tasks.workunit.client.1.smithi169.stdout:9/809: dread d5/da/d21/de4/f9c [0,4194304] 0 2024-09-16T00:26:11.651 INFO:tasks.workunit.client.1.smithi169.stdout:5/606: rmdir d2/da/d10/d13/d1c/d40/d54/d7f 39 2024-09-16T00:26:11.652 INFO:tasks.workunit.client.0.smithi112.stdout:8/829: write d10/d12/d1c/d61/fa3 [990454,85699] 0 2024-09-16T00:26:11.653 INFO:tasks.workunit.client.0.smithi112.stdout:6/907: truncate dd/d15/da5/d79/fee 1967261 0 2024-09-16T00:26:11.654 INFO:tasks.workunit.client.0.smithi112.stdout:6/908: readlink dd/df/d10/l14 0 2024-09-16T00:26:11.654 INFO:tasks.workunit.client.1.smithi169.stdout:9/810: stat d5/da/d21/l76 0 2024-09-16T00:26:11.655 INFO:tasks.workunit.client.1.smithi169.stdout:5/607: dread d2/f88 [0,4194304] 0 2024-09-16T00:26:11.657 INFO:tasks.workunit.client.0.smithi112.stdout:6/909: read dd/df/d17/d27/f76 [2344807,90161] 0 2024-09-16T00:26:11.658 INFO:tasks.workunit.client.0.smithi112.stdout:7/965: truncate d6/d10/d16/d96/dc7/dbf/f12c 1300996 0 2024-09-16T00:26:11.658 INFO:tasks.workunit.client.0.smithi112.stdout:4/986: dread d6/d39/d1c/d6e/ffd [0,4194304] 0 2024-09-16T00:26:11.661 INFO:tasks.workunit.client.1.smithi169.stdout:5/608: creat d2/da/d10/d13/d1c/d2d/db2/fd0 x:0 0 0 2024-09-16T00:26:11.661 INFO:tasks.workunit.client.0.smithi112.stdout:5/879: dread d4/dd/d49/d70/d83/f86 [0,4194304] 0 2024-09-16T00:26:11.661 INFO:tasks.workunit.client.0.smithi112.stdout:4/987: readlink d6/d39/d1c/d2c/d69/d7b/l16b 0 2024-09-16T00:26:11.661 INFO:tasks.workunit.client.0.smithi112.stdout:6/910: creat dd/df/d91/f127 x:0 0 0 2024-09-16T00:26:11.662 INFO:tasks.workunit.client.0.smithi112.stdout:8/830: dread d10/d12/d1c/d61/d6c/dd1/d58/fcb [0,4194304] 0 2024-09-16T00:26:11.663 INFO:tasks.workunit.client.1.smithi169.stdout:5/609: rename d2/da/d10/d13/d1c/f3b to d2/da/d10/d13/d1c/d94/fd1 0 2024-09-16T00:26:11.664 INFO:tasks.workunit.client.0.smithi112.stdout:8/831: dread - d10/de9/def/fe6 zero size 2024-09-16T00:26:11.665 INFO:tasks.workunit.client.0.smithi112.stdout:4/988: mkdir d6/d39/d1c/d6e/d6f/d11b/d16d 0 2024-09-16T00:26:11.668 INFO:tasks.workunit.client.0.smithi112.stdout:7/966: dwrite d6/d10/d75/d88/f13c [0,4194304] 0 2024-09-16T00:26:11.671 INFO:tasks.workunit.client.0.smithi112.stdout:5/880: rename d4/d4f/d77/dcc/l5b to d4/d7/d26/d4c/db3/db6/l128 0 2024-09-16T00:26:11.671 INFO:tasks.workunit.client.1.smithi169.stdout:0/710: truncate d0/d2/d8/f48 1496858 0 2024-09-16T00:26:11.672 INFO:tasks.workunit.client.0.smithi112.stdout:8/832: mkdir d10/d12/d1c/d3b/d7d/de4/d116 0 2024-09-16T00:26:11.673 INFO:tasks.workunit.client.1.smithi169.stdout:3/827: dwrite d7/d1a/d1c/d23/d70/f95 [0,4194304] 0 2024-09-16T00:26:11.673 INFO:tasks.workunit.client.0.smithi112.stdout:6/911: link dd/df/d10/lb2 dd/df/d91/l128 0 2024-09-16T00:26:11.674 INFO:tasks.workunit.client.1.smithi169.stdout:9/811: dread d5/da/d5b/d6c/dcb/db4/fc4 [0,4194304] 0 2024-09-16T00:26:11.675 INFO:tasks.workunit.client.0.smithi112.stdout:7/967: creat d6/d7f/df6/f143 x:0 0 0 2024-09-16T00:26:11.678 INFO:tasks.workunit.client.0.smithi112.stdout:4/989: symlink d6/d39/d1c/d6e/d6f/d11b/daa/d156/l16e 0 2024-09-16T00:26:11.679 INFO:tasks.workunit.client.0.smithi112.stdout:8/833: dread - d10/d12/d1c/d61/d6c/dd1/d58/fbe zero size 2024-09-16T00:26:11.679 INFO:tasks.workunit.client.1.smithi169.stdout:0/711: symlink d0/d2c/d38/de4/d56/d67/dd1/d6c/lf2 0 2024-09-16T00:26:11.680 INFO:tasks.workunit.client.0.smithi112.stdout:6/912: symlink dd/df/d91/dd7/l129 0 2024-09-16T00:26:11.681 INFO:tasks.workunit.client.1.smithi169.stdout:0/712: write d0/d2c/d38/de4/d21/d31/d83/f8a [3062696,9151] 0 2024-09-16T00:26:11.684 INFO:tasks.workunit.client.1.smithi169.stdout:5/610: creat d2/da/d10/d13/d1c/d32/d53/dae/dbf/fd2 x:0 0 0 2024-09-16T00:26:11.685 INFO:tasks.workunit.client.0.smithi112.stdout:8/834: dread d10/d12/d1c/d61/fa3 [0,4194304] 0 2024-09-16T00:26:11.687 INFO:tasks.workunit.client.0.smithi112.stdout:5/881: creat d4/d7/d26/f129 x:0 0 0 2024-09-16T00:26:11.689 INFO:tasks.workunit.client.1.smithi169.stdout:2/649: sync 2024-09-16T00:26:11.690 INFO:tasks.workunit.client.1.smithi169.stdout:0/713: dwrite d0/d2c/d38/de4/f28 [0,4194304] 0 2024-09-16T00:26:11.690 INFO:tasks.workunit.client.0.smithi112.stdout:5/882: creat d4/d4f/f12a x:0 0 0 2024-09-16T00:26:11.691 INFO:tasks.workunit.client.1.smithi169.stdout:5/611: rename d2/da/f28 to d2/da/d10/d13/d1c/d32/d53/d4e/d89/fd3 0 2024-09-16T00:26:11.692 INFO:tasks.workunit.client.0.smithi112.stdout:7/968: creat d6/f144 x:0 0 0 2024-09-16T00:26:11.694 INFO:tasks.workunit.client.1.smithi169.stdout:5/612: truncate d2/da/d10/d13/d1c/d32/d53/fb0 613896 0 2024-09-16T00:26:11.694 INFO:tasks.workunit.client.1.smithi169.stdout:5/613: read d2/f88 [1061737,29075] 0 2024-09-16T00:26:11.695 INFO:tasks.workunit.client.0.smithi112.stdout:5/883: creat d4/dd/d49/de6/d113/f12b x:0 0 0 2024-09-16T00:26:11.695 INFO:tasks.workunit.client.0.smithi112.stdout:4/990: write d6/d39/d1c/d6e/d6f/d11b/daa/d156/d7e/d8f/f98 [4709328,38687] 0 2024-09-16T00:26:11.695 INFO:tasks.workunit.client.1.smithi169.stdout:2/650: mkdir d7/d1c/de6 0 2024-09-16T00:26:11.696 INFO:tasks.workunit.client.1.smithi169.stdout:3/828: truncate d7/d50/d6f/f100 3696068 0 2024-09-16T00:26:11.697 INFO:tasks.workunit.client.1.smithi169.stdout:2/651: truncate d7/db/d89/fbf 1031330 0 2024-09-16T00:26:11.697 INFO:tasks.workunit.client.0.smithi112.stdout:8/835: link d10/d12/d1c/d61/d6c/dd1/d58/fa1 d10/d12/d1c/d61/d6c/dd1/f117 0 2024-09-16T00:26:11.697 INFO:tasks.workunit.client.1.smithi169.stdout:0/714: creat d0/d4c/ff3 x:0 0 0 2024-09-16T00:26:11.699 INFO:tasks.workunit.client.0.smithi112.stdout:5/884: creat d4/dd/d49/d70/d83/dbd/f12c x:0 0 0 2024-09-16T00:26:11.700 INFO:tasks.workunit.client.0.smithi112.stdout:8/836: mkdir d10/d12/d1c/d3b/d74/d118 0 2024-09-16T00:26:11.702 INFO:tasks.workunit.client.1.smithi169.stdout:3/829: mkdir d7/d1a/d1c/d35/d63/dc2/dc7/d12c 0 2024-09-16T00:26:11.705 INFO:tasks.workunit.client.1.smithi169.stdout:9/812: dwrite d5/da/d2e/d99/d100/d10f/d11c/f67 [0,4194304] 0 2024-09-16T00:26:11.705 INFO:tasks.workunit.client.0.smithi112.stdout:4/991: dwrite d6/db/d1f/d41/d49/d31/fed [4194304,4194304] 0 2024-09-16T00:26:11.706 INFO:tasks.workunit.client.1.smithi169.stdout:3/830: mknod d7/de5/c12d 0 2024-09-16T00:26:11.708 INFO:tasks.workunit.client.0.smithi112.stdout:4/992: mkdir d6/d39/d1c/d6e/d6f/d11b/daa/d156/d7e/d8f/d16f 0 2024-09-16T00:26:11.709 INFO:tasks.workunit.client.1.smithi169.stdout:3/831: chown d7/d50/d6f/d113/d118/cae 5181917 1 2024-09-16T00:26:11.710 INFO:tasks.workunit.client.1.smithi169.stdout:0/715: dread d0/d2/d8/d5/d79/d7c/f94 [0,4194304] 0 2024-09-16T00:26:11.715 INFO:tasks.workunit.client.1.smithi169.stdout:9/813: link d5/da/d2e/d47/c104 d5/da/d2e/c120 0 2024-09-16T00:26:11.718 INFO:tasks.workunit.client.1.smithi169.stdout:3/832: symlink d7/d50/d90/da6/df6/d126/l12e 0 2024-09-16T00:26:11.719 INFO:tasks.workunit.client.1.smithi169.stdout:3/833: fdatasync d7/f8f 0 2024-09-16T00:26:11.721 INFO:tasks.workunit.client.1.smithi169.stdout:9/814: mkdir d5/da/dd0/d11f/d121 0 2024-09-16T00:26:11.721 INFO:tasks.workunit.client.1.smithi169.stdout:3/834: fsync d7/d50/d6f/f71 0 2024-09-16T00:26:11.721 INFO:tasks.workunit.client.0.smithi112.stdout:7/969: truncate d6/d10/d16/faf 747787 0 2024-09-16T00:26:11.722 INFO:tasks.workunit.client.1.smithi169.stdout:0/716: dread d0/d2c/d38/de4/d21/d31/f58 [0,4194304] 0 2024-09-16T00:26:11.724 INFO:tasks.workunit.client.1.smithi169.stdout:9/815: creat d5/da/f122 x:0 0 0 2024-09-16T00:26:11.725 INFO:tasks.workunit.client.1.smithi169.stdout:7/719: sync 2024-09-16T00:26:11.727 INFO:tasks.workunit.client.0.smithi112.stdout:7/970: dwrite d6/d10/d75/d8f/d90/dd7/f121 [0,4194304] 0 2024-09-16T00:26:11.729 INFO:tasks.workunit.client.1.smithi169.stdout:5/614: truncate d2/f29 3222508 0 2024-09-16T00:26:11.730 INFO:tasks.workunit.client.0.smithi112.stdout:7/971: truncate d6/d10/d16/d96/f9f 109499 0 2024-09-16T00:26:11.730 INFO:tasks.workunit.client.1.smithi169.stdout:3/835: dread d7/d1a/d1c/d23/d88/dbe/f110 [0,4194304] 0 2024-09-16T00:26:11.733 INFO:tasks.workunit.client.0.smithi112.stdout:7/972: symlink d6/d10/d16/d96/dc7/d39/l145 0 2024-09-16T00:26:11.733 INFO:tasks.workunit.client.1.smithi169.stdout:7/720: rmdir d7/df/d95 39 2024-09-16T00:26:11.734 INFO:tasks.workunit.client.1.smithi169.stdout:3/836: mkdir d7/d50/d6f/d113/d12f 0 2024-09-16T00:26:11.736 INFO:tasks.workunit.client.0.smithi112.stdout:7/973: rename d6/d10/d75/f120 to d6/d7f/f146 0 2024-09-16T00:26:11.736 INFO:tasks.workunit.client.1.smithi169.stdout:9/816: creat d5/da/d21/de4/f123 x:0 0 0 2024-09-16T00:26:11.736 INFO:tasks.workunit.client.1.smithi169.stdout:3/837: dread d7/de5/f10b [0,4194304] 0 2024-09-16T00:26:11.738 INFO:tasks.workunit.client.1.smithi169.stdout:9/817: chown d5/da/df3/df8/dc1/cec 78 1 2024-09-16T00:26:11.739 INFO:tasks.workunit.client.1.smithi169.stdout:3/838: write d7/d1a/d1c/d35/d63/da3/f116 [52601,55741] 0 2024-09-16T00:26:11.739 INFO:tasks.workunit.client.1.smithi169.stdout:7/721: symlink d7/df/le9 0 2024-09-16T00:26:11.741 INFO:tasks.workunit.client.0.smithi112.stdout:5/885: dread d4/d1c/d43/f6d [0,4194304] 0 2024-09-16T00:26:11.741 INFO:tasks.workunit.client.1.smithi169.stdout:9/818: symlink d5/da/d2e/d47/l124 0 2024-09-16T00:26:11.741 INFO:tasks.workunit.client.1.smithi169.stdout:2/652: dread d7/d1c/f95 [0,4194304] 0 2024-09-16T00:26:11.742 INFO:tasks.workunit.client.1.smithi169.stdout:5/615: link d2/da/d10/d13/d1c/d32/d53/d4e/d55/l6c d2/da/d10/d13/d1c/d32/ld4 0 2024-09-16T00:26:11.743 INFO:tasks.workunit.client.0.smithi112.stdout:8/837: truncate d10/d12/d1c/d96/fd6 338265 0 2024-09-16T00:26:11.746 INFO:tasks.workunit.client.0.smithi112.stdout:8/838: rmdir d10/d12/d1c/d114 0 2024-09-16T00:26:11.746 INFO:tasks.workunit.client.1.smithi169.stdout:2/653: symlink d7/d6b/d70/de0/le7 0 2024-09-16T00:26:11.747 INFO:tasks.workunit.client.1.smithi169.stdout:5/616: fsync d2/da/d10/d13/d1c/d40/d43/f84 0 2024-09-16T00:26:11.749 INFO:tasks.workunit.client.0.smithi112.stdout:8/839: dread d10/d12/d1c/dd0/f78 [0,4194304] 0 2024-09-16T00:26:11.750 INFO:tasks.workunit.client.0.smithi112.stdout:4/993: truncate d6/db/df/f5f 2734468 0 2024-09-16T00:26:11.750 INFO:tasks.workunit.client.1.smithi169.stdout:9/819: link d5/da/d2e/d47/d11b/fd9 d5/da/d21/de9/f125 0 2024-09-16T00:26:11.751 INFO:tasks.workunit.client.1.smithi169.stdout:3/839: dwrite d7/d1a/d1c/d35/d63/dc2/dc7/f11e [0,4194304] 0 2024-09-16T00:26:11.754 INFO:tasks.workunit.client.1.smithi169.stdout:5/617: rename d2/f2a to d2/da/d10/fd5 0 2024-09-16T00:26:11.756 INFO:tasks.workunit.client.0.smithi112.stdout:8/840: rename d10/d12/c18 to d10/d12/d1c/dd0/dbb/c119 0 2024-09-16T00:26:11.758 INFO:tasks.workunit.client.0.smithi112.stdout:4/994: dwrite d6/d39/d36/f128 [0,4194304] 0 2024-09-16T00:26:11.760 INFO:tasks.workunit.client.1.smithi169.stdout:3/840: symlink d7/d50/d90/l130 0 2024-09-16T00:26:11.763 INFO:tasks.workunit.client.1.smithi169.stdout:3/841: write d7/d1a/d1c/fb6 [1756408,43687] 0 2024-09-16T00:26:11.765 INFO:tasks.workunit.client.0.smithi112.stdout:8/841: link lb d10/d12/d1c/d61/l11a 0 2024-09-16T00:26:11.766 INFO:tasks.workunit.client.0.smithi112.stdout:8/842: stat d10/d12/d1c/d3b/d74/f7b 0 2024-09-16T00:26:11.770 INFO:tasks.workunit.client.1.smithi169.stdout:0/717: truncate d0/d2c/d38/de4/d21/d31/d83/fce 325368 0 2024-09-16T00:26:11.774 INFO:tasks.workunit.client.1.smithi169.stdout:3/842: dwrite d7/d50/d6f/fb4 [0,4194304] 0 2024-09-16T00:26:11.777 INFO:tasks.workunit.client.1.smithi169.stdout:3/843: mknod d7/d1a/d1c/d35/d63/d37/c131 0 2024-09-16T00:26:11.779 INFO:tasks.workunit.client.1.smithi169.stdout:0/718: creat d0/d2/d8/d5/d60/ddb/ff4 x:0 0 0 2024-09-16T00:26:11.779 INFO:tasks.workunit.client.0.smithi112.stdout:5/886: dread d4/d4f/d78/ffd [0,4194304] 0 2024-09-16T00:26:11.782 INFO:tasks.workunit.client.1.smithi169.stdout:3/844: creat d7/d1a/d1c/d23/d88/d9b/de2/f132 x:0 0 0 2024-09-16T00:26:11.783 INFO:tasks.workunit.client.1.smithi169.stdout:7/722: dwrite d7/f50 [0,4194304] 0 2024-09-16T00:26:11.783 INFO:tasks.workunit.client.0.smithi112.stdout:5/887: rename d4/d4f/d77/dcc/d7c/fa3 to d4/d7/f12d 0 2024-09-16T00:26:11.785 INFO:tasks.workunit.client.1.smithi169.stdout:7/723: chown d7/d16/d49/d74 15 1 2024-09-16T00:26:11.791 INFO:tasks.workunit.client.1.smithi169.stdout:7/724: rename d7/d16/d49/f72 to d7/dd4/fea 0 2024-09-16T00:26:11.796 INFO:tasks.workunit.client.1.smithi169.stdout:3/845: link d7/d50/d6f/d113/d118/f8c d7/d50/f133 0 2024-09-16T00:26:11.796 INFO:tasks.workunit.client.1.smithi169.stdout:3/846: chown d7/d1a/d1c/d35/la7 16961887 1 2024-09-16T00:26:11.800 INFO:tasks.workunit.client.1.smithi169.stdout:2/654: dread d7/db/db8/daf/fe2 [0,4194304] 0 2024-09-16T00:26:11.800 INFO:tasks.workunit.client.1.smithi169.stdout:3/847: symlink d7/d1a/d1c/d23/d88/d9b/de2/l134 0 2024-09-16T00:26:11.803 INFO:tasks.workunit.client.1.smithi169.stdout:2/655: mknod d7/db/d89/db4/dc8/ce8 0 2024-09-16T00:26:11.804 INFO:tasks.workunit.client.0.smithi112.stdout:4/995: dwrite d6/db/df/d3c/f82 [0,4194304] 0 2024-09-16T00:26:11.805 INFO:tasks.workunit.client.1.smithi169.stdout:9/820: dread d5/da/f2b [0,4194304] 0 2024-09-16T00:26:11.806 INFO:tasks.workunit.client.1.smithi169.stdout:3/848: write d7/d50/d6f/d113/d118/f104 [315609,111700] 0 2024-09-16T00:26:11.808 INFO:tasks.workunit.client.0.smithi112.stdout:8/843: dwrite d10/d12/d1c/dd0/f3f [0,4194304] 0 2024-09-16T00:26:11.809 INFO:tasks.workunit.client.1.smithi169.stdout:2/656: truncate d7/daa/f5f 1028663 0 2024-09-16T00:26:11.809 INFO:tasks.workunit.client.1.smithi169.stdout:0/719: write d0/d2c/d38/f66 [1666881,73830] 0 2024-09-16T00:26:11.810 INFO:tasks.workunit.client.0.smithi112.stdout:8/844: creat d10/de9/f11b x:0 0 0 2024-09-16T00:26:11.810 INFO:tasks.workunit.client.1.smithi169.stdout:9/821: mknod d5/da/dd0/c126 0 2024-09-16T00:26:11.811 INFO:tasks.workunit.client.1.smithi169.stdout:3/849: mkdir d7/d1a/d1c/d23/d88/dd7/d135 0 2024-09-16T00:26:11.812 INFO:tasks.workunit.client.0.smithi112.stdout:8/845: write d10/d12/d24/f82 [677912,97940] 0 2024-09-16T00:26:11.813 INFO:tasks.workunit.client.0.smithi112.stdout:8/846: stat d10/d12/d1c/d61/d6c/f93 0 2024-09-16T00:26:11.814 INFO:tasks.workunit.client.0.smithi112.stdout:8/847: stat d10/d12/d24/f29 0 2024-09-16T00:26:11.816 INFO:tasks.workunit.client.0.smithi112.stdout:8/848: write d10/d12/d1c/d61/d6c/dd1/d58/fcb [1816956,88672] 0 2024-09-16T00:26:11.817 INFO:tasks.workunit.client.1.smithi169.stdout:0/720: creat d0/d2/ff5 x:0 0 0 2024-09-16T00:26:11.818 INFO:tasks.workunit.client.0.smithi112.stdout:8/849: write d10/d12/d1c/d61/d6c/dd1/d58/f105 [611052,82452] 0 2024-09-16T00:26:11.819 INFO:tasks.workunit.client.1.smithi169.stdout:0/721: fsync d0/d2c/d4f/fe6 0 2024-09-16T00:26:11.820 INFO:tasks.workunit.client.0.smithi112.stdout:8/850: creat d10/d12/d1c/d96/f11c x:0 0 0 2024-09-16T00:26:11.821 INFO:tasks.workunit.client.1.smithi169.stdout:3/850: dread d7/d50/f7b [0,4194304] 0 2024-09-16T00:26:11.822 INFO:tasks.workunit.client.1.smithi169.stdout:2/657: dwrite d7/db/db8/daf/fe2 [4194304,4194304] 0 2024-09-16T00:26:11.824 INFO:tasks.workunit.client.0.smithi112.stdout:8/851: creat d10/d12/d1c/d61/d6c/dd1/d58/f11d x:0 0 0 2024-09-16T00:26:11.827 INFO:tasks.workunit.client.1.smithi169.stdout:2/658: creat d7/d1c/d26/fe9 x:0 0 0 2024-09-16T00:26:11.827 INFO:tasks.workunit.client.1.smithi169.stdout:2/659: readlink d7/d1c/d26/l6f 0 2024-09-16T00:26:11.828 INFO:tasks.workunit.client.1.smithi169.stdout:2/660: write d7/db/d89/db4/d69/d92/fd0 [728610,14085] 0 2024-09-16T00:26:11.831 INFO:tasks.workunit.client.1.smithi169.stdout:2/661: symlink d7/d6b/d70/lea 0 2024-09-16T00:26:11.831 INFO:tasks.workunit.client.0.smithi112.stdout:8/852: dwrite d10/d12/d1c/d61/d6c/dd1/fc9 [0,4194304] 0 2024-09-16T00:26:11.833 INFO:tasks.workunit.client.1.smithi169.stdout:2/662: mknod d7/d6b/d70/ceb 0 2024-09-16T00:26:11.837 INFO:tasks.workunit.client.1.smithi169.stdout:2/663: rename d7/d1c/l8c to d7/d1c/d26/d96/dbd/lec 0 2024-09-16T00:26:11.837 INFO:tasks.workunit.client.0.smithi112.stdout:8/853: dwrite d10/d12/d24/f2b [4194304,4194304] 0 2024-09-16T00:26:11.839 INFO:tasks.workunit.client.1.smithi169.stdout:0/722: dread d0/d7d/fa8 [0,4194304] 0 2024-09-16T00:26:11.840 INFO:tasks.workunit.client.0.smithi112.stdout:8/854: dread d10/d12/fb9 [0,4194304] 0 2024-09-16T00:26:11.840 INFO:tasks.workunit.client.1.smithi169.stdout:2/664: mkdir d7/db/d27/ded 0 2024-09-16T00:26:11.842 INFO:tasks.workunit.client.0.smithi112.stdout:8/855: stat d10/d12/d1c/dd0/f52 0 2024-09-16T00:26:11.844 INFO:tasks.workunit.client.1.smithi169.stdout:0/723: mknod d0/d2c/d38/de4/cf6 0 2024-09-16T00:26:11.845 INFO:tasks.workunit.client.0.smithi112.stdout:8/856: creat d10/de9/def/dff/f11e x:0 0 0 2024-09-16T00:26:11.846 INFO:tasks.workunit.client.1.smithi169.stdout:2/665: readlink d7/db/d20/d5c/lb5 0 2024-09-16T00:26:11.847 INFO:tasks.workunit.client.0.smithi112.stdout:5/888: dread d4/dd/d49/da9/f104 [0,4194304] 0 2024-09-16T00:26:11.848 INFO:tasks.workunit.client.0.smithi112.stdout:5/889: fdatasync d4/dd/d49/d70/d83/ff0 0 2024-09-16T00:26:11.848 INFO:tasks.workunit.client.1.smithi169.stdout:2/666: readlink d7/l41 0 2024-09-16T00:26:11.850 INFO:tasks.workunit.client.0.smithi112.stdout:5/890: unlink d4/d7/d26/d36/dcf/f114 0 2024-09-16T00:26:11.852 INFO:tasks.workunit.client.1.smithi169.stdout:7/725: dread d7/d71/d76/d29/d78/fa3 [0,4194304] 0 2024-09-16T00:26:11.853 INFO:tasks.workunit.client.1.smithi169.stdout:0/724: dwrite d0/d2c/f5d [0,4194304] 0 2024-09-16T00:26:11.855 INFO:tasks.workunit.client.1.smithi169.stdout:7/726: symlink d7/d71/d76/d9c/leb 0 2024-09-16T00:26:11.855 INFO:tasks.workunit.client.1.smithi169.stdout:2/667: read d7/d1c/fb7 [119249,91998] 0 2024-09-16T00:26:11.856 INFO:tasks.workunit.client.0.smithi112.stdout:5/891: dread d4/d7/d14/d90/fb5 [0,4194304] 0 2024-09-16T00:26:11.856 INFO:tasks.workunit.client.1.smithi169.stdout:0/725: mknod d0/d2/d8/d5/d60/cf7 0 2024-09-16T00:26:11.857 INFO:tasks.workunit.client.0.smithi112.stdout:5/892: read d4/dd/d49/d70/d83/f86 [1130553,96984] 0 2024-09-16T00:26:11.858 INFO:tasks.workunit.client.0.smithi112.stdout:5/893: rmdir d4/dd/d49 39 2024-09-16T00:26:11.858 INFO:tasks.workunit.client.1.smithi169.stdout:0/726: chown d0/d2c/d38/de4/d21/d31/d83/dac 150497 1 2024-09-16T00:26:11.858 INFO:tasks.workunit.client.0.smithi112.stdout:5/894: truncate d4/dd/f10f 223297 0 2024-09-16T00:26:11.861 INFO:tasks.workunit.client.1.smithi169.stdout:2/668: creat d7/db/d20/d21/fee x:0 0 0 2024-09-16T00:26:11.861 INFO:tasks.workunit.client.1.smithi169.stdout:0/727: chown d0/d4c/d6f/c80 50 1 2024-09-16T00:26:11.861 INFO:tasks.workunit.client.0.smithi112.stdout:5/895: unlink d4/dd/d49/d70/d98/cba 0 2024-09-16T00:26:11.862 INFO:tasks.workunit.client.1.smithi169.stdout:7/727: read d7/d71/d76/fc [2523662,58110] 0 2024-09-16T00:26:11.864 INFO:tasks.workunit.client.1.smithi169.stdout:2/669: creat d7/db/d27/d29/d77/fef x:0 0 0 2024-09-16T00:26:11.864 INFO:tasks.workunit.client.1.smithi169.stdout:7/728: rmdir d7/d5b/dc1 39 2024-09-16T00:26:11.867 INFO:tasks.workunit.client.1.smithi169.stdout:7/729: symlink d7/d71/d76/d29/lec 0 2024-09-16T00:26:11.868 INFO:tasks.workunit.client.1.smithi169.stdout:0/728: dwrite d0/f51 [0,4194304] 0 2024-09-16T00:26:11.869 INFO:tasks.workunit.client.1.smithi169.stdout:2/670: truncate d7/d6b/d70/d79/dce/fd8 1860354 0 2024-09-16T00:26:11.871 INFO:tasks.workunit.client.1.smithi169.stdout:7/730: mknod d7/d71/d76/d9c/ced 0 2024-09-16T00:26:11.871 INFO:tasks.workunit.client.1.smithi169.stdout:0/729: truncate d0/d2c/d38/de4/fab 1214575 0 2024-09-16T00:26:11.873 INFO:tasks.workunit.client.1.smithi169.stdout:2/671: unlink d7/d1c/d26/d96/dab/cc4 0 2024-09-16T00:26:11.876 INFO:tasks.workunit.client.1.smithi169.stdout:7/731: dwrite f1 [0,4194304] 0 2024-09-16T00:26:11.879 INFO:tasks.workunit.client.1.smithi169.stdout:2/672: getdents d7/d6b/d70/de0 0 2024-09-16T00:26:11.880 INFO:tasks.workunit.client.1.smithi169.stdout:7/732: mkdir d7/df/d95/dee 0 2024-09-16T00:26:11.885 INFO:tasks.workunit.client.1.smithi169.stdout:7/733: mknod d7/d5b/dc1/d96/cef 0 2024-09-16T00:26:11.887 INFO:tasks.workunit.client.1.smithi169.stdout:2/673: link d7/d1c/d26/f75 d7/db/d27/d29/d37/dd1/ff0 0 2024-09-16T00:26:11.888 INFO:tasks.workunit.client.1.smithi169.stdout:7/734: rename d7/d16/d49/c55 to d7/cf0 0 2024-09-16T00:26:11.889 INFO:tasks.workunit.client.1.smithi169.stdout:2/674: stat d7/db/d89/db4/d9d/dbe/fd7 0 2024-09-16T00:26:11.891 INFO:tasks.workunit.client.1.smithi169.stdout:2/675: mkdir d7/db/db8/d6d/df1 0 2024-09-16T00:26:11.893 INFO:tasks.workunit.client.1.smithi169.stdout:2/676: creat d7/d1c/d26/d96/dbd/ff2 x:0 0 0 2024-09-16T00:26:11.895 INFO:tasks.workunit.client.1.smithi169.stdout:2/677: creat d7/d1c/dc3/ff3 x:0 0 0 2024-09-16T00:26:11.896 INFO:tasks.workunit.client.1.smithi169.stdout:7/735: dwrite d7/d16/d49/d84/d8e/fdc [0,4194304] 0 2024-09-16T00:26:11.925 INFO:tasks.workunit.client.1.smithi169.stdout:9/822: write d5/f4a [2665891,67305] 0 2024-09-16T00:26:11.927 INFO:tasks.workunit.client.1.smithi169.stdout:9/823: creat d5/da/d5b/d6c/dcb/f127 x:0 0 0 2024-09-16T00:26:11.928 INFO:tasks.workunit.client.1.smithi169.stdout:9/824: chown d5/da/d2e/d99/d100/c103 213909597 1 2024-09-16T00:26:11.930 INFO:tasks.workunit.client.1.smithi169.stdout:9/825: rename d5/da/d2e/c88 to d5/d49/dc0/dd6/c128 0 2024-09-16T00:26:11.943 INFO:tasks.workunit.client.1.smithi169.stdout:3/851: fsync d7/d1a/d1c/f54 0 2024-09-16T00:26:11.944 INFO:tasks.workunit.client.1.smithi169.stdout:3/852: chown d7/l1f 0 1 2024-09-16T00:26:11.945 INFO:tasks.workunit.client.1.smithi169.stdout:3/853: chown d7/d1a/d1c/d6c/lde 239 1 2024-09-16T00:26:11.950 INFO:tasks.workunit.client.0.smithi112.stdout:8/857: write d10/d12/d1c/fa4 [4671133,126424] 0 2024-09-16T00:26:11.951 INFO:tasks.workunit.client.1.smithi169.stdout:8/731: sync 2024-09-16T00:26:11.951 INFO:tasks.workunit.client.1.smithi169.stdout:1/692: sync 2024-09-16T00:26:11.951 INFO:tasks.workunit.client.1.smithi169.stdout:6/890: sync 2024-09-16T00:26:11.951 INFO:tasks.workunit.client.1.smithi169.stdout:4/652: sync 2024-09-16T00:26:11.951 INFO:tasks.workunit.client.0.smithi112.stdout:8/858: write d10/d12/d24/fdd [1137460,23776] 0 2024-09-16T00:26:11.952 INFO:tasks.workunit.client.0.smithi112.stdout:4/996: dwrite d6/db/df/d3c/f73 [0,4194304] 0 2024-09-16T00:26:11.953 INFO:tasks.workunit.client.1.smithi169.stdout:9/826: rmdir d5/da/d5b/d6c 39 2024-09-16T00:26:11.954 INFO:tasks.workunit.client.1.smithi169.stdout:6/891: write d2/d36/fe0 [614891,45950] 0 2024-09-16T00:26:11.956 INFO:tasks.workunit.client.1.smithi169.stdout:4/653: creat d2/d12/da7/fcf x:0 0 0 2024-09-16T00:26:11.957 INFO:tasks.workunit.client.0.smithi112.stdout:4/997: rmdir d6/d39/d1c 39 2024-09-16T00:26:11.958 INFO:tasks.workunit.client.1.smithi169.stdout:3/854: write d7/d1a/d1c/f40 [873222,20640] 0 2024-09-16T00:26:11.958 INFO:tasks.workunit.client.1.smithi169.stdout:8/732: mknod d7/dc/d17/d24/d32/d8d/dad/cef 0 2024-09-16T00:26:11.959 INFO:tasks.workunit.client.0.smithi112.stdout:8/859: symlink d10/d12/d1c/d61/d6c/ddc/l11f 0 2024-09-16T00:26:11.959 INFO:tasks.workunit.client.1.smithi169.stdout:9/827: mkdir d5/da/d5b/d91/db3/d129 0 2024-09-16T00:26:11.959 INFO:tasks.workunit.client.1.smithi169.stdout:6/892: write d2/d3f/d95/fc8 [8820452,11263] 0 2024-09-16T00:26:11.961 INFO:tasks.workunit.client.0.smithi112.stdout:4/998: stat d6/d39/d1c/d6e/d6f/d11b/daa/d156/d7e/d8f/f90 0 2024-09-16T00:26:11.964 INFO:tasks.workunit.client.0.smithi112.stdout:4/999: creat d6/d39/d1c/d6e/d6f/d11b/daa/d156/dc6/db5/f170 x:0 0 0 2024-09-16T00:26:11.964 INFO:tasks.workunit.client.1.smithi169.stdout:9/828: mkdir d5/da/d2e/d99/d100/d12a 0 2024-09-16T00:26:11.965 INFO:tasks.workunit.client.1.smithi169.stdout:4/654: truncate d2/dbb/d59/f75 4681137 0 2024-09-16T00:26:11.965 INFO:tasks.workunit.client.1.smithi169.stdout:7/736: dwrite d7/d71/d76/d9c/fa9 [0,4194304] 0 2024-09-16T00:26:11.965 INFO:tasks.workunit.client.1.smithi169.stdout:1/693: getdents dc/d18/d67/d69/d7b 0 2024-09-16T00:26:11.967 INFO:tasks.workunit.client.1.smithi169.stdout:6/893: fsync d2/d4d/d59/f35 0 2024-09-16T00:26:11.969 INFO:tasks.workunit.client.1.smithi169.stdout:4/655: chown d2/d12/d13/d1f/d3e/d5b/f18 711 1 2024-09-16T00:26:11.971 INFO:tasks.workunit.client.1.smithi169.stdout:7/737: chown d7/d71/d76/d29/d78/dba/fcd 148308 1 2024-09-16T00:26:11.973 INFO:tasks.workunit.client.1.smithi169.stdout:1/694: creat dc/d18/d67/d69/d7b/d92/fd2 x:0 0 0 2024-09-16T00:26:11.973 INFO:tasks.workunit.client.1.smithi169.stdout:8/733: dread d7/dc/d2a/f85 [0,4194304] 0 2024-09-16T00:26:11.973 INFO:tasks.workunit.client.1.smithi169.stdout:6/894: creat d2/da4/db1/dbc/d87/f11b x:0 0 0 2024-09-16T00:26:11.974 INFO:tasks.workunit.client.0.smithi112.stdout:8/860: write d10/f63 [394887,54305] 0 2024-09-16T00:26:11.975 INFO:tasks.workunit.client.0.smithi112.stdout:8/861: fsync d10/de9/def/fe6 0 2024-09-16T00:26:11.975 INFO:tasks.workunit.client.1.smithi169.stdout:9/829: creat d5/da/d2e/d99/d100/d12a/f12b x:0 0 0 2024-09-16T00:26:11.976 INFO:tasks.workunit.client.1.smithi169.stdout:3/855: dread d7/d1a/d1c/d23/d70/fe0 [0,4194304] 0 2024-09-16T00:26:11.979 INFO:tasks.workunit.client.1.smithi169.stdout:8/734: write d7/dc/d26/fa7 [364098,31857] 0 2024-09-16T00:26:11.979 INFO:tasks.workunit.client.1.smithi169.stdout:4/656: truncate d2/d12/dcc/d9e/f71 865717 0 2024-09-16T00:26:11.981 INFO:tasks.workunit.client.1.smithi169.stdout:4/657: chown d2/d12/dcc 3906907 1 2024-09-16T00:26:11.982 INFO:tasks.workunit.client.1.smithi169.stdout:1/695: read dc/d18/d67/f49 [254464,70445] 0 2024-09-16T00:26:11.984 INFO:tasks.workunit.client.0.smithi112.stdout:8/862: rename d10/d12/d1c/dd0/f80 to d10/d12/d1c/d61/d6c/dd1/f120 0 2024-09-16T00:26:11.984 INFO:tasks.workunit.client.1.smithi169.stdout:3/856: symlink d7/d1a/d1c/d35/d63/d37/l136 0 2024-09-16T00:26:11.986 INFO:tasks.workunit.client.1.smithi169.stdout:4/658: symlink d2/d12/d9c/ld0 0 2024-09-16T00:26:11.986 INFO:tasks.workunit.client.1.smithi169.stdout:7/738: dwrite d7/d16/fcf [0,4194304] 0 2024-09-16T00:26:11.987 INFO:tasks.workunit.client.1.smithi169.stdout:8/735: mkdir d7/dc/d26/d41/d92/dcf/df0 0 2024-09-16T00:26:11.994 INFO:tasks.workunit.client.1.smithi169.stdout:1/696: mknod dc/d36/d3a/d45/d29/d2a/cd3 0 2024-09-16T00:26:11.994 INFO:tasks.workunit.client.1.smithi169.stdout:7/739: mknod d7/d71/d76/cf1 0 2024-09-16T00:26:11.995 INFO:tasks.workunit.client.1.smithi169.stdout:6/895: dwrite d2/d3f/d41/d5c/d91/da6/de2/fe6 [0,4194304] 0 2024-09-16T00:26:11.995 INFO:tasks.workunit.client.1.smithi169.stdout:9/830: dwrite d5/da/d21/de4/f86 [0,4194304] 0 2024-09-16T00:26:11.996 INFO:tasks.workunit.client.1.smithi169.stdout:3/857: creat d7/d1a/d1c/d35/d63/da3/f137 x:0 0 0 2024-09-16T00:26:12.000 INFO:tasks.workunit.client.1.smithi169.stdout:8/736: creat d7/dc/d53/da0/dac/dd6/ff1 x:0 0 0 2024-09-16T00:26:12.000 INFO:tasks.workunit.client.1.smithi169.stdout:7/740: stat d7/d5b/d61/d98/ld2 0 2024-09-16T00:26:12.003 INFO:tasks.workunit.client.0.smithi112.stdout:1/926: sync 2024-09-16T00:26:12.003 INFO:tasks.workunit.client.0.smithi112.stdout:6/913: sync 2024-09-16T00:26:12.003 INFO:tasks.workunit.client.1.smithi169.stdout:8/737: fdatasync d7/dc/d26/d41/d92/dcf/d3c/d82/fdf 0 2024-09-16T00:26:12.004 INFO:tasks.workunit.client.1.smithi169.stdout:1/697: symlink dc/d36/d3a/d7f/d93/d72/d98/ld4 0 2024-09-16T00:26:12.005 INFO:tasks.workunit.client.0.smithi112.stdout:1/927: chown d0/d14/df/d28/d45/dd4/d47/d10a/l102 0 1 2024-09-16T00:26:12.007 INFO:tasks.workunit.client.0.smithi112.stdout:1/928: write d0/d14/df/d28/d45/d46/da7/dda/f10c [597349,32724] 0 2024-09-16T00:26:12.007 INFO:tasks.workunit.client.1.smithi169.stdout:9/831: write d5/da/d5b/d91/db3/f101 [2561813,98980] 0 2024-09-16T00:26:12.007 INFO:tasks.workunit.client.1.smithi169.stdout:3/858: dwrite d7/d1a/d1c/d23/fe7 [0,4194304] 0 2024-09-16T00:26:12.009 INFO:tasks.workunit.client.0.smithi112.stdout:1/929: chown d0/d14/df/d28/d42/dd8/de8 153819 1 2024-09-16T00:26:12.009 INFO:tasks.workunit.client.1.smithi169.stdout:7/741: creat d7/d71/d76/d29/ff2 x:0 0 0 2024-09-16T00:26:12.010 INFO:tasks.workunit.client.1.smithi169.stdout:7/742: fdatasync d7/fc4 0 2024-09-16T00:26:12.010 INFO:tasks.workunit.client.0.smithi112.stdout:1/930: chown d0/d14/df/d28/d45/d46/c61 440995449 1 2024-09-16T00:26:12.012 INFO:tasks.workunit.client.1.smithi169.stdout:8/738: mknod d7/dc/d26/d41/d92/dcf/d3c/d82/db3/cf2 0 2024-09-16T00:26:12.012 INFO:tasks.workunit.client.1.smithi169.stdout:8/739: chown d7/dc/d2a 413304455 1 2024-09-16T00:26:12.013 INFO:tasks.workunit.client.1.smithi169.stdout:1/698: stat dc/d6c/d96/lad 0 2024-09-16T00:26:12.015 INFO:tasks.workunit.client.1.smithi169.stdout:6/896: creat d2/d4d/d59/d43/f11c x:0 0 0 2024-09-16T00:26:12.016 INFO:tasks.workunit.client.1.smithi169.stdout:4/659: write d2/d12/dcc/d9e/f72 [691830,116736] 0 2024-09-16T00:26:12.019 INFO:tasks.workunit.client.1.smithi169.stdout:3/859: readlink d7/d1a/d1c/d23/d88/d9b/dc8/df8/l107 0 2024-09-16T00:26:12.019 INFO:tasks.workunit.client.0.smithi112.stdout:7/974: sync 2024-09-16T00:26:12.020 INFO:tasks.workunit.client.0.smithi112.stdout:7/975: dread - d6/d7f/df6/f143 zero size 2024-09-16T00:26:12.020 INFO:tasks.workunit.client.1.smithi169.stdout:7/743: unlink d7/d5b/d61/d98/fb7 0 2024-09-16T00:26:12.021 INFO:tasks.workunit.client.1.smithi169.stdout:8/740: dread d7/dc/d17/d24/d32/d8d/fc8 [0,4194304] 0 2024-09-16T00:26:12.021 INFO:tasks.workunit.client.1.smithi169.stdout:4/660: readlink d2/d12/d13/d1f/d3e/d5b/d70/d7d/l81 0 2024-09-16T00:26:12.022 INFO:tasks.workunit.client.1.smithi169.stdout:3/860: dread - d7/d50/d90/dba/f11f zero size 2024-09-16T00:26:12.022 INFO:tasks.workunit.client.1.smithi169.stdout:6/897: dread - d2/d4d/ff7 zero size 2024-09-16T00:26:12.022 INFO:tasks.workunit.client.0.smithi112.stdout:8/863: mknod d10/d12/d1c/d61/d6c/dd1/d58/d87/c121 0 2024-09-16T00:26:12.023 INFO:tasks.workunit.client.1.smithi169.stdout:1/699: symlink dc/d36/d3a/d45/dce/ld5 0 2024-09-16T00:26:12.023 INFO:tasks.workunit.client.0.smithi112.stdout:1/931: mknod d0/d14/df/d28/d45/d46/da7/dda/d123/c131 0 2024-09-16T00:26:12.024 INFO:tasks.workunit.client.0.smithi112.stdout:6/914: read dd/d23/d8f/f8c [1116774,47817] 0 2024-09-16T00:26:12.024 INFO:tasks.workunit.client.0.smithi112.stdout:7/976: unlink d6/d10/d75/dea/ceb 0 2024-09-16T00:26:12.025 INFO:tasks.workunit.client.0.smithi112.stdout:1/932: chown d0/d14/df/d28/d45/fdb 417240 1 2024-09-16T00:26:12.025 INFO:tasks.workunit.client.1.smithi169.stdout:3/861: dread - d7/d1a/d1c/d23/d88/ff9 zero size 2024-09-16T00:26:12.026 INFO:tasks.workunit.client.1.smithi169.stdout:6/898: write d2/d4d/d59/d38/f4f [2319291,72381] 0 2024-09-16T00:26:12.026 INFO:tasks.workunit.client.1.smithi169.stdout:3/862: read d7/f10 [1369309,110224] 0 2024-09-16T00:26:12.026 INFO:tasks.workunit.client.1.smithi169.stdout:1/700: mkdir dc/d18/dd6 0 2024-09-16T00:26:12.027 INFO:tasks.workunit.client.0.smithi112.stdout:6/915: unlink dd/dcc/le4 0 2024-09-16T00:26:12.029 INFO:tasks.workunit.client.1.smithi169.stdout:1/701: write dc/fa2 [2265502,93460] 0 2024-09-16T00:26:12.029 INFO:tasks.workunit.client.0.smithi112.stdout:8/864: dwrite d10/de9/def/fe6 [0,4194304] 0 2024-09-16T00:26:12.031 INFO:tasks.workunit.client.1.smithi169.stdout:8/741: rename d7/dc/d26/l29 to d7/dc/d26/d41/d92/dcf/lf3 0 2024-09-16T00:26:12.034 INFO:tasks.workunit.client.0.smithi112.stdout:8/865: symlink d10/d12/d1c/df2/l122 0 2024-09-16T00:26:12.034 INFO:tasks.workunit.client.0.smithi112.stdout:6/916: symlink dd/dcc/de6/l12a 0 2024-09-16T00:26:12.034 INFO:tasks.workunit.client.1.smithi169.stdout:6/899: symlink d2/d3f/d41/d52/l11d 0 2024-09-16T00:26:12.035 INFO:tasks.workunit.client.1.smithi169.stdout:6/900: fsync d2/d3f/d41/d5c/f88 0 2024-09-16T00:26:12.035 INFO:tasks.workunit.client.1.smithi169.stdout:3/863: creat d7/d1a/d1c/d23/d88/d9b/dc8/dcc/f138 x:0 0 0 2024-09-16T00:26:12.037 INFO:tasks.workunit.client.1.smithi169.stdout:6/901: read d2/d22/d51/f77 [2088119,27248] 0 2024-09-16T00:26:12.037 INFO:tasks.workunit.client.1.smithi169.stdout:8/742: rmdir d7/d87/dd5 39 2024-09-16T00:26:12.038 INFO:tasks.workunit.client.0.smithi112.stdout:7/977: dwrite d6/d10/d16/d96/dc7/dbf/d117/d6c/fec [0,4194304] 0 2024-09-16T00:26:12.039 INFO:tasks.workunit.client.1.smithi169.stdout:8/743: chown d7/d87 36358369 1 2024-09-16T00:26:12.040 INFO:tasks.workunit.client.1.smithi169.stdout:8/744: chown d7/dc/d17/d24/l83 198078 1 2024-09-16T00:26:12.040 INFO:tasks.workunit.client.0.smithi112.stdout:8/866: symlink d10/d12/d1c/d61/d6c/ddc/l123 0 2024-09-16T00:26:12.041 INFO:tasks.workunit.client.0.smithi112.stdout:6/917: read dd/df/d10/d47/d55/dbd/dcb/de8/f117 [873914,80787] 0 2024-09-16T00:26:12.042 INFO:tasks.workunit.client.0.smithi112.stdout:6/918: readlink dd/df/d10/l31 0 2024-09-16T00:26:12.043 INFO:tasks.workunit.client.1.smithi169.stdout:6/902: creat d2/d4d/f11e x:0 0 0 2024-09-16T00:26:12.044 INFO:tasks.workunit.client.1.smithi169.stdout:6/903: chown d2/d4d/f11e 4 1 2024-09-16T00:26:12.044 INFO:tasks.workunit.client.1.smithi169.stdout:8/745: rmdir d7/dc/d53/da0/dac/dd6 39 2024-09-16T00:26:12.045 INFO:tasks.workunit.client.0.smithi112.stdout:6/919: mknod dd/df/d10/d47/d55/dbd/dcb/c12b 0 2024-09-16T00:26:12.045 INFO:tasks.workunit.client.1.smithi169.stdout:6/904: dread - d2/da4/db1/dbc/d87/f11b zero size 2024-09-16T00:26:12.045 INFO:tasks.workunit.client.0.smithi112.stdout:6/920: read dd/f7e [3187041,53971] 0 2024-09-16T00:26:12.046 INFO:tasks.workunit.client.1.smithi169.stdout:8/746: chown d7/dc/d26/d41/d92/dcf/ddc 12465 1 2024-09-16T00:26:12.047 INFO:tasks.workunit.client.0.smithi112.stdout:6/921: creat dd/df/d10/d47/d55/f12c x:0 0 0 2024-09-16T00:26:12.049 INFO:tasks.workunit.client.0.smithi112.stdout:6/922: write dd/df/d10/f3a [1180067,103324] 0 2024-09-16T00:26:12.052 INFO:tasks.workunit.client.0.smithi112.stdout:6/923: mkdir dd/df/d10/d47/d55/dc4/d12d 0 2024-09-16T00:26:12.053 INFO:tasks.workunit.client.1.smithi169.stdout:6/905: dread d2/d36/fec [0,4194304] 0 2024-09-16T00:26:12.054 INFO:tasks.workunit.client.1.smithi169.stdout:6/906: chown d2/d3f/d41/d5c/cfd 1 1 2024-09-16T00:26:12.055 INFO:tasks.workunit.client.0.smithi112.stdout:6/924: mknod dd/df/d10/d47/d89/dbc/c12e 0 2024-09-16T00:26:12.055 INFO:tasks.workunit.client.1.smithi169.stdout:6/907: write d2/d36/f44 [5177923,80230] 0 2024-09-16T00:26:12.056 INFO:tasks.workunit.client.0.smithi112.stdout:6/925: stat dd/df/d10/d47/ld8 0 2024-09-16T00:26:12.057 INFO:tasks.workunit.client.0.smithi112.stdout:6/926: creat dd/df/d10/d47/d55/f12f x:0 0 0 2024-09-16T00:26:12.058 INFO:tasks.workunit.client.1.smithi169.stdout:4/661: dread d2/d12/d13/d1f/f87 [0,4194304] 0 2024-09-16T00:26:12.058 INFO:tasks.workunit.client.1.smithi169.stdout:6/908: creat d2/d3f/d41/d5c/d91/da6/de2/f11f x:0 0 0 2024-09-16T00:26:12.059 INFO:tasks.workunit.client.0.smithi112.stdout:6/927: write dd/df/d13/d18/fd3 [1671142,72799] 0 2024-09-16T00:26:12.059 INFO:tasks.workunit.client.1.smithi169.stdout:4/662: chown d2/d12/d13/d5c/da6/fa9 26 1 2024-09-16T00:26:12.060 INFO:tasks.workunit.client.1.smithi169.stdout:6/909: mkdir d2/d36/d120 0 2024-09-16T00:26:12.061 INFO:tasks.workunit.client.1.smithi169.stdout:4/663: symlink d2/d12/da7/ld1 0 2024-09-16T00:26:12.061 INFO:tasks.workunit.client.1.smithi169.stdout:6/910: dread - d2/d4d/d59/d74/f109 zero size 2024-09-16T00:26:12.062 INFO:tasks.workunit.client.0.smithi112.stdout:6/928: rename dd/df/d13/f1c to dd/df/d10/f130 0 2024-09-16T00:26:12.063 INFO:tasks.workunit.client.1.smithi169.stdout:4/664: unlink d2/cbc 0 2024-09-16T00:26:12.063 INFO:tasks.workunit.client.0.smithi112.stdout:6/929: creat dd/df/d10/d47/d55/dbd/f131 x:0 0 0 2024-09-16T00:26:12.065 INFO:tasks.workunit.client.0.smithi112.stdout:6/930: mkdir dd/df/d10/d47/d55/dc4/d132 0 2024-09-16T00:26:12.066 INFO:tasks.workunit.client.1.smithi169.stdout:4/665: getdents d2/d12/d13/d4e/d50/db8 0 2024-09-16T00:26:12.066 INFO:tasks.workunit.client.1.smithi169.stdout:6/911: dread d2/d3f/d41/d5c/d91/da6/de2/fe6 [0,4194304] 0 2024-09-16T00:26:12.068 INFO:tasks.workunit.client.1.smithi169.stdout:4/666: write d2/d12/d13/d1f/d3e/fc4 [265517,27936] 0 2024-09-16T00:26:12.069 INFO:tasks.workunit.client.1.smithi169.stdout:4/667: truncate d2/d12/d13/f60 1308889 0 2024-09-16T00:26:12.073 INFO:tasks.workunit.client.1.smithi169.stdout:6/912: dwrite d2/d3f/d41/d5c/d91/da6/de2/f11f [0,4194304] 0 2024-09-16T00:26:12.075 INFO:tasks.workunit.client.1.smithi169.stdout:4/668: unlink d2/d12/d9c/da4/lc3 0 2024-09-16T00:26:12.075 INFO:tasks.workunit.client.1.smithi169.stdout:4/669: chown d2/d12/d13/d1f/d2f/c4c 0 1 2024-09-16T00:26:12.079 INFO:tasks.workunit.client.1.smithi169.stdout:4/670: fdatasync d2/d12/d13/d1f/f48 0 2024-09-16T00:26:12.081 INFO:tasks.workunit.client.1.smithi169.stdout:4/671: write d2/d12/d13/d1f/d3e/d83/fa5 [446701,27661] 0 2024-09-16T00:26:12.082 INFO:tasks.workunit.client.0.smithi112.stdout:8/867: rmdir d10/d12/d1c/dd0 39 2024-09-16T00:26:12.082 INFO:tasks.workunit.client.1.smithi169.stdout:7/744: write d7/df/fa5 [213715,74898] 0 2024-09-16T00:26:12.084 INFO:tasks.workunit.client.0.smithi112.stdout:7/978: dread d6/d10/f87 [0,4194304] 0 2024-09-16T00:26:12.085 INFO:tasks.workunit.client.1.smithi169.stdout:4/672: dread - d2/d12/d13/d5c/fa3 zero size 2024-09-16T00:26:12.087 INFO:tasks.workunit.client.1.smithi169.stdout:1/702: write dc/d18/d67/f94 [630464,1327] 0 2024-09-16T00:26:12.087 INFO:tasks.workunit.client.0.smithi112.stdout:8/868: rename d10/d12/d1c/d61/d6c/dd1/f117 to d10/d12/d1c/d61/f124 0 2024-09-16T00:26:12.088 INFO:tasks.workunit.client.0.smithi112.stdout:8/869: fsync d10/d12/d1c/d61/d6c/dd1/f68 0 2024-09-16T00:26:12.088 INFO:tasks.workunit.client.1.smithi169.stdout:3/864: write d7/d50/d6f/d113/d118/f8c [1023835,73745] 0 2024-09-16T00:26:12.089 INFO:tasks.workunit.client.1.smithi169.stdout:4/673: write d2/d12/d13/f60 [1412535,123414] 0 2024-09-16T00:26:12.090 INFO:tasks.workunit.client.1.smithi169.stdout:1/703: stat dc/d36/d39/d50/daf/d88 0 2024-09-16T00:26:12.090 INFO:tasks.workunit.client.1.smithi169.stdout:9/832: dwrite d5/da/f4d [0,4194304] 0 2024-09-16T00:26:12.092 INFO:tasks.workunit.client.0.smithi112.stdout:5/896: sync 2024-09-16T00:26:12.093 INFO:tasks.workunit.client.0.smithi112.stdout:5/897: chown d4/dd/d6f/f126 7 1 2024-09-16T00:26:12.095 INFO:tasks.workunit.client.0.smithi112.stdout:7/979: dwrite d6/d10/f1d [4194304,4194304] 0 2024-09-16T00:26:12.095 INFO:tasks.workunit.client.1.smithi169.stdout:3/865: chown d7/d1a/d1c/d23/d70/dc5/cf5 22712569 1 2024-09-16T00:26:12.096 INFO:tasks.workunit.client.0.smithi112.stdout:7/980: write d6/d7f/dc3/df8/f126 [673061,64777] 0 2024-09-16T00:26:12.099 INFO:tasks.workunit.client.0.smithi112.stdout:8/870: getdents d10/d12/d1c/d61/d6c/dd1/d58/d87/df0 0 2024-09-16T00:26:12.101 INFO:tasks.workunit.client.0.smithi112.stdout:7/981: mkdir d6/d7f/dc3/df8/d147 0 2024-09-16T00:26:12.102 INFO:tasks.workunit.client.1.smithi169.stdout:6/913: dread d2/d36/fe0 [0,4194304] 0 2024-09-16T00:26:12.102 INFO:tasks.workunit.client.1.smithi169.stdout:4/674: symlink d2/d12/d13/d90/ld2 0 2024-09-16T00:26:12.103 INFO:tasks.workunit.client.1.smithi169.stdout:4/675: write d2/d12/d13/d5c/f94 [5045967,49602] 0 2024-09-16T00:26:12.103 INFO:tasks.workunit.client.0.smithi112.stdout:5/898: creat d4/dd/d49/d70/f12e x:0 0 0 2024-09-16T00:26:12.104 INFO:tasks.workunit.client.0.smithi112.stdout:5/899: stat d4/d7/d14/d90/lc3 0 2024-09-16T00:26:12.106 INFO:tasks.workunit.client.1.smithi169.stdout:6/914: mkdir d2/d3f/d41/d52/d121 0 2024-09-16T00:26:12.106 INFO:tasks.workunit.client.0.smithi112.stdout:7/982: symlink d6/d10/d75/d8f/d90/l148 0 2024-09-16T00:26:12.107 INFO:tasks.workunit.client.1.smithi169.stdout:7/745: truncate d7/dd4/fea 2461855 0 2024-09-16T00:26:12.108 INFO:tasks.workunit.client.1.smithi169.stdout:3/866: symlink d7/d1a/d1c/d35/d63/dea/d11a/l139 0 2024-09-16T00:26:12.108 INFO:tasks.workunit.client.1.smithi169.stdout:3/867: chown d7/d1a/feb 184603890 1 2024-09-16T00:26:12.110 INFO:tasks.workunit.client.0.smithi112.stdout:8/871: unlink d10/d12/d1c/dd0/l65 0 2024-09-16T00:26:12.111 INFO:tasks.workunit.client.1.smithi169.stdout:1/704: getdents dc/d36/d3a/d7f/d93 0 2024-09-16T00:26:12.112 INFO:tasks.workunit.client.0.smithi112.stdout:8/872: symlink d10/d12/d1c/d3b/d7d/de4/l125 0 2024-09-16T00:26:12.113 INFO:tasks.workunit.client.1.smithi169.stdout:1/705: chown dc/d18/d67/d69/d7b/fba 1053661 1 2024-09-16T00:26:12.113 INFO:tasks.workunit.client.1.smithi169.stdout:6/915: unlink d2/d3f/d41/d5c/d73/cd5 0 2024-09-16T00:26:12.113 INFO:tasks.workunit.client.0.smithi112.stdout:7/983: dread d6/d10/d75/fb9 [4194304,4194304] 0 2024-09-16T00:26:12.113 INFO:tasks.workunit.client.1.smithi169.stdout:7/746: mknod d7/d71/d76/d9c/cf3 0 2024-09-16T00:26:12.114 INFO:tasks.workunit.client.0.smithi112.stdout:8/873: creat d10/de9/def/f126 x:0 0 0 2024-09-16T00:26:12.115 INFO:tasks.workunit.client.1.smithi169.stdout:9/833: getdents d5/da 0 2024-09-16T00:26:12.115 INFO:tasks.workunit.client.1.smithi169.stdout:6/916: creat d2/d22/d51/f122 x:0 0 0 2024-09-16T00:26:12.116 INFO:tasks.workunit.client.1.smithi169.stdout:1/706: symlink dc/d36/d3a/d45/d29/d2a/ld7 0 2024-09-16T00:26:12.116 INFO:tasks.workunit.client.1.smithi169.stdout:9/834: write d5/da/ff [1592417,69510] 0 2024-09-16T00:26:12.117 INFO:tasks.workunit.client.0.smithi112.stdout:7/984: dread d6/d10/f87 [0,4194304] 0 2024-09-16T00:26:12.118 INFO:tasks.workunit.client.0.smithi112.stdout:8/874: read d10/d12/d1c/d61/d6c/dd1/d8c/f85 [327786,120206] 0 2024-09-16T00:26:12.121 INFO:tasks.workunit.client.0.smithi112.stdout:8/875: read d10/d12/d1c/d61/d6c/fb3 [4617001,97724] 0 2024-09-16T00:26:12.122 INFO:tasks.workunit.client.0.smithi112.stdout:8/876: chown d10/d12/fd7 6847212 1 2024-09-16T00:26:12.123 INFO:tasks.workunit.client.1.smithi169.stdout:7/747: rename d7/l64 to d7/d5b/dc1/d96/lf4 0 2024-09-16T00:26:12.124 INFO:tasks.workunit.client.0.smithi112.stdout:1/933: write d0/d14/df/d28/d45/d46/f49 [110571,44509] 0 2024-09-16T00:26:12.124 INFO:tasks.workunit.client.1.smithi169.stdout:6/917: symlink d2/d3f/d41/l123 0 2024-09-16T00:26:12.128 INFO:tasks.workunit.client.1.smithi169.stdout:1/707: dwrite dc/d36/f84 [0,4194304] 0 2024-09-16T00:26:12.128 INFO:tasks.workunit.client.1.smithi169.stdout:7/748: write f5 [5034539,39485] 0 2024-09-16T00:26:12.130 INFO:tasks.workunit.client.1.smithi169.stdout:9/835: rename d5/da/d5b/d6c/dcb/db4 to d5/da/d2e/d99/d12c 0 2024-09-16T00:26:12.130 INFO:tasks.workunit.client.1.smithi169.stdout:4/676: dwrite d2/d12/d13/d1f/f52 [0,4194304] 0 2024-09-16T00:26:12.130 INFO:tasks.workunit.client.1.smithi169.stdout:6/918: truncate d2/d22/fc2 1028980 0 2024-09-16T00:26:12.134 INFO:tasks.workunit.client.1.smithi169.stdout:3/868: dread d7/d50/d90/da6/fbb [0,4194304] 0 2024-09-16T00:26:12.137 INFO:tasks.workunit.client.1.smithi169.stdout:9/836: unlink d5/da/f4d 0 2024-09-16T00:26:12.138 INFO:tasks.workunit.client.1.smithi169.stdout:7/749: symlink d7/d71/d76/d29/lf5 0 2024-09-16T00:26:12.139 INFO:tasks.workunit.client.1.smithi169.stdout:6/919: write d2/d36/f2a [7805694,73161] 0 2024-09-16T00:26:12.140 INFO:tasks.workunit.client.0.smithi112.stdout:6/931: write dd/d15/da5/d79/fc3 [2754794,18516] 0 2024-09-16T00:26:12.140 INFO:tasks.workunit.client.0.smithi112.stdout:6/932: readlink dd/d23/l30 0 2024-09-16T00:26:12.140 INFO:tasks.workunit.client.0.smithi112.stdout:7/985: dread d6/d7f/dc3/df8/f126 [0,4194304] 0 2024-09-16T00:26:12.140 INFO:tasks.workunit.client.1.smithi169.stdout:4/677: mkdir d2/d12/d13/d5c/da6/dd3 0 2024-09-16T00:26:12.140 INFO:tasks.workunit.client.1.smithi169.stdout:9/837: chown d5/d49/d64/c10d 233036 1 2024-09-16T00:26:12.140 INFO:tasks.workunit.client.1.smithi169.stdout:9/838: readlink d5/de0/lfc 0 2024-09-16T00:26:12.141 INFO:tasks.workunit.client.1.smithi169.stdout:9/839: readlink d5/da/d73/l74 0 2024-09-16T00:26:12.141 INFO:tasks.workunit.client.1.smithi169.stdout:3/869: rename d7/d50/d6f/d113/d118 to d7/d50/d90/d13a 0 2024-09-16T00:26:12.142 INFO:tasks.workunit.client.0.smithi112.stdout:5/900: write d4/d4f/f76 [89550,48842] 0 2024-09-16T00:26:12.142 INFO:tasks.workunit.client.1.smithi169.stdout:1/708: dread dc/d36/d3a/d7f/d93/f6d [0,4194304] 0 2024-09-16T00:26:12.142 INFO:tasks.workunit.client.1.smithi169.stdout:7/750: truncate d7/d71/f9d 5093106 0 2024-09-16T00:26:12.142 INFO:tasks.workunit.client.0.smithi112.stdout:7/986: dread - d6/d10/d75/d8f/d90/dd7/f105 zero size 2024-09-16T00:26:12.144 INFO:tasks.workunit.client.1.smithi169.stdout:3/870: chown d7/d1a/d1c/f4f 90412183 1 2024-09-16T00:26:12.144 INFO:tasks.workunit.client.1.smithi169.stdout:6/920: unlink d2/d3f/d41/d72/d9e/de1/le8 0 2024-09-16T00:26:12.145 INFO:tasks.workunit.client.1.smithi169.stdout:9/840: creat d5/da/d5b/d91/d9a/f12d x:0 0 0 2024-09-16T00:26:12.146 INFO:tasks.workunit.client.1.smithi169.stdout:4/678: fdatasync d2/d12/d13/d1f/d3e/d83/f6e 0 2024-09-16T00:26:12.146 INFO:tasks.workunit.client.0.smithi112.stdout:8/877: truncate d10/d12/d24/f82 3252115 0 2024-09-16T00:26:12.147 INFO:tasks.workunit.client.1.smithi169.stdout:3/871: dread d7/d1a/d1c/d23/f3d [0,4194304] 0 2024-09-16T00:26:12.147 INFO:tasks.workunit.client.0.smithi112.stdout:1/934: write d0/d14/df/d28/d45/dd4/fb3 [1092098,49415] 0 2024-09-16T00:26:12.149 INFO:tasks.workunit.client.1.smithi169.stdout:6/921: getdents d2/d4d/d59/d74/def 0 2024-09-16T00:26:12.149 INFO:tasks.workunit.client.1.smithi169.stdout:3/872: write d7/f24 [539018,107434] 0 2024-09-16T00:26:12.149 INFO:tasks.workunit.client.0.smithi112.stdout:5/901: read d4/d7/d26/d4c/f6a [2960857,9444] 0 2024-09-16T00:26:12.150 INFO:tasks.workunit.client.1.smithi169.stdout:4/679: mknod d2/d12/d13/d5c/cd4 0 2024-09-16T00:26:12.151 INFO:tasks.workunit.client.0.smithi112.stdout:8/878: read - d10/d12/d1c/d61/f124 zero size 2024-09-16T00:26:12.153 INFO:tasks.workunit.client.0.smithi112.stdout:7/987: dwrite d6/f142 [0,4194304] 0 2024-09-16T00:26:12.153 INFO:tasks.workunit.client.0.smithi112.stdout:1/935: unlink d0/d14/d1c/d20/fa9 0 2024-09-16T00:26:12.154 INFO:tasks.workunit.client.1.smithi169.stdout:6/922: unlink d2/d4d/ff7 0 2024-09-16T00:26:12.155 INFO:tasks.workunit.client.0.smithi112.stdout:8/879: readlink d10/d12/d1c/d61/d6c/dd1/d58/lcf 0 2024-09-16T00:26:12.156 INFO:tasks.workunit.client.1.smithi169.stdout:4/680: creat d2/dbb/d59/fd5 x:0 0 0 2024-09-16T00:26:12.158 INFO:tasks.workunit.client.1.smithi169.stdout:4/681: chown d2/d12/d13/d1f/d3e/d5b/d70/d7d/lc6 1090120 1 2024-09-16T00:26:12.159 INFO:tasks.workunit.client.0.smithi112.stdout:1/936: fsync d0/d14/df/d28/d45/d46/da7/fbb 0 2024-09-16T00:26:12.159 INFO:tasks.workunit.client.0.smithi112.stdout:8/880: chown d10/d12/d1c/d3b/d74/laa 3771269 1 2024-09-16T00:26:12.159 INFO:tasks.workunit.client.0.smithi112.stdout:7/988: creat d6/d10/d75/d8f/d90/dbc/f149 x:0 0 0 2024-09-16T00:26:12.161 INFO:tasks.workunit.client.0.smithi112.stdout:5/902: dwrite d4/d7/d26/f129 [0,4194304] 0 2024-09-16T00:26:12.163 INFO:tasks.workunit.client.0.smithi112.stdout:5/903: rmdir d4/dd/d49/d70/d83 39 2024-09-16T00:26:12.163 INFO:tasks.workunit.client.1.smithi169.stdout:4/682: creat d2/d12/d13/d4e/fd6 x:0 0 0 2024-09-16T00:26:12.164 INFO:tasks.workunit.client.1.smithi169.stdout:3/873: rename d7/d50/d90/d13a/l83 to d7/l13b 0 2024-09-16T00:26:12.165 INFO:tasks.workunit.client.1.smithi169.stdout:4/683: dread - d2/f9d zero size 2024-09-16T00:26:12.168 INFO:tasks.workunit.client.0.smithi112.stdout:5/904: write d4/dd/d49/da9/f104 [64044,66220] 0 2024-09-16T00:26:12.169 INFO:tasks.workunit.client.1.smithi169.stdout:4/684: creat d2/d12/d13/d1f/d3e/d5b/fd7 x:0 0 0 2024-09-16T00:26:12.170 INFO:tasks.workunit.client.0.smithi112.stdout:8/881: link d10/d12/d1c/d61/d6c/dd1/fe5 d10/f127 0 2024-09-16T00:26:12.170 INFO:tasks.workunit.client.0.smithi112.stdout:7/989: dwrite d6/d10/d16/d96/f125 [0,4194304] 0 2024-09-16T00:26:12.176 INFO:tasks.workunit.client.1.smithi169.stdout:7/751: dread d7/d71/d76/fc [0,4194304] 0 2024-09-16T00:26:12.177 INFO:tasks.workunit.client.1.smithi169.stdout:3/874: dwrite d7/d50/d90/d13a/fb3 [0,4194304] 0 2024-09-16T00:26:12.178 INFO:tasks.workunit.client.0.smithi112.stdout:7/990: rmdir d6/d10/d16/d96/dc7/dbf/d11c 0 2024-09-16T00:26:12.178 INFO:tasks.workunit.client.0.smithi112.stdout:5/905: rename d4/dd/c8a to d4/dd/d49/d70/c12f 0 2024-09-16T00:26:12.179 INFO:tasks.workunit.client.0.smithi112.stdout:7/991: dread - d6/d10/d75/d8f/d90/ddc/f106 zero size 2024-09-16T00:26:12.180 INFO:tasks.workunit.client.0.smithi112.stdout:7/992: read - d6/d10/d75/d8f/d90/ddc/dee/f115 zero size 2024-09-16T00:26:12.181 INFO:tasks.workunit.client.0.smithi112.stdout:5/906: unlink d4/d4f/d77/f11c 0 2024-09-16T00:26:12.182 INFO:tasks.workunit.client.0.smithi112.stdout:7/993: readlink d6/d10/d16/d96/dc7/l67 0 2024-09-16T00:26:12.183 INFO:tasks.workunit.client.0.smithi112.stdout:5/907: rmdir d4/d7/d26/d4c/db3/db6/dd0/dfc 39 2024-09-16T00:26:12.184 INFO:tasks.workunit.client.1.smithi169.stdout:7/752: creat d7/d71/d76/d9c/ddf/ff6 x:0 0 0 2024-09-16T00:26:12.184 INFO:tasks.workunit.client.1.smithi169.stdout:3/875: symlink d7/d1a/d1c/d6c/l13c 0 2024-09-16T00:26:12.186 INFO:tasks.workunit.client.0.smithi112.stdout:5/908: fdatasync d4/dd/d49/d70/f12e 0 2024-09-16T00:26:12.186 INFO:tasks.workunit.client.1.smithi169.stdout:7/753: mknod d7/d71/d76/d9c/ddf/cf7 0 2024-09-16T00:26:12.187 INFO:tasks.workunit.client.0.smithi112.stdout:7/994: rename d6/d10/d16/d96/dc7/f11d to d6/d10/d75/d8f/d90/dd7/f14a 0 2024-09-16T00:26:12.187 INFO:tasks.workunit.client.1.smithi169.stdout:3/876: creat d7/d50/d90/da6/df6/f13d x:0 0 0 2024-09-16T00:26:12.188 INFO:tasks.workunit.client.1.smithi169.stdout:7/754: symlink d7/d5b/dc1/d96/lf8 0 2024-09-16T00:26:12.189 INFO:tasks.workunit.client.1.smithi169.stdout:3/877: stat d7/d1a/d1c/c31 0 2024-09-16T00:26:12.189 INFO:tasks.workunit.client.0.smithi112.stdout:5/909: creat d4/dd/d12/d52/d64/f130 x:0 0 0 2024-09-16T00:26:12.189 INFO:tasks.workunit.client.1.smithi169.stdout:7/755: chown d7/df/f44 216908 1 2024-09-16T00:26:12.190 INFO:tasks.workunit.client.1.smithi169.stdout:3/878: write d7/f8f [1038434,103005] 0 2024-09-16T00:26:12.191 INFO:tasks.workunit.client.1.smithi169.stdout:6/923: dread d2/d4d/f8e [0,4194304] 0 2024-09-16T00:26:12.192 INFO:tasks.workunit.client.1.smithi169.stdout:7/756: rmdir d7/d16/d49/d74 39 2024-09-16T00:26:12.194 INFO:tasks.workunit.client.0.smithi112.stdout:8/882: dwrite d10/d12/d1c/d61/d6c/dd1/f120 [0,4194304] 0 2024-09-16T00:26:12.194 INFO:tasks.workunit.client.0.smithi112.stdout:7/995: dread d6/d10/da1/f11a [0,4194304] 0 2024-09-16T00:26:12.198 INFO:tasks.workunit.client.0.smithi112.stdout:8/883: creat d10/d12/d1c/dd0/dbb/d107/f128 x:0 0 0 2024-09-16T00:26:12.198 INFO:tasks.workunit.client.1.smithi169.stdout:3/879: dwrite d7/f8f [0,4194304] 0 2024-09-16T00:26:12.199 INFO:tasks.workunit.client.0.smithi112.stdout:5/910: dwrite d4/d7/d14/d90/dd3/fdd [0,4194304] 0 2024-09-16T00:26:12.199 INFO:tasks.workunit.client.0.smithi112.stdout:5/911: stat d4/le9 0 2024-09-16T00:26:12.203 INFO:tasks.workunit.client.1.smithi169.stdout:3/880: readlink d7/d1a/l2c 0 2024-09-16T00:26:12.203 INFO:tasks.workunit.client.1.smithi169.stdout:7/757: dwrite d7/d71/d76/d29/d78/dba/fd8 [0,4194304] 0 2024-09-16T00:26:12.208 INFO:tasks.workunit.client.0.smithi112.stdout:5/912: dwrite d4/d7/d9b/fc6 [4194304,4194304] 0 2024-09-16T00:26:12.209 INFO:tasks.workunit.client.1.smithi169.stdout:8/747: sync 2024-09-16T00:26:12.209 INFO:tasks.workunit.client.1.smithi169.stdout:0/730: sync 2024-09-16T00:26:12.209 INFO:tasks.workunit.client.1.smithi169.stdout:5/618: sync 2024-09-16T00:26:12.209 INFO:tasks.workunit.client.1.smithi169.stdout:2/678: sync 2024-09-16T00:26:12.210 INFO:tasks.workunit.client.0.smithi112.stdout:5/913: dread d4/d4f/d77/f6c [0,4194304] 0 2024-09-16T00:26:12.215 INFO:tasks.workunit.client.0.smithi112.stdout:5/914: symlink d4/d7/d14/d90/dd3/l131 0 2024-09-16T00:26:12.219 INFO:tasks.workunit.client.1.smithi169.stdout:7/758: dwrite f1 [4194304,4194304] 0 2024-09-16T00:26:12.219 INFO:tasks.workunit.client.1.smithi169.stdout:8/748: creat d7/d8c/ff4 x:0 0 0 2024-09-16T00:26:12.220 INFO:tasks.workunit.client.1.smithi169.stdout:3/881: read d7/d50/d90/d13a/f69 [2924318,47182] 0 2024-09-16T00:26:12.221 INFO:tasks.workunit.client.0.smithi112.stdout:5/915: getdents d4/d7/d14 0 2024-09-16T00:26:12.223 INFO:tasks.workunit.client.0.smithi112.stdout:5/916: fdatasync d4/f5c 0 2024-09-16T00:26:12.225 INFO:tasks.workunit.client.1.smithi169.stdout:5/619: rename d2/da/d10/d13/d1c/d40/d54/lcc to d2/da/d10/ld6 0 2024-09-16T00:26:12.227 INFO:tasks.workunit.client.1.smithi169.stdout:0/731: creat d0/d4c/ff8 x:0 0 0 2024-09-16T00:26:12.227 INFO:tasks.workunit.client.0.smithi112.stdout:5/917: unlink d4/dd/d12/d52/d64/f9e 0 2024-09-16T00:26:12.228 INFO:tasks.workunit.client.1.smithi169.stdout:0/732: write d0/d2c/d38/de4/d21/d31/f85 [1924484,52048] 0 2024-09-16T00:26:12.230 INFO:tasks.workunit.client.0.smithi112.stdout:5/918: getdents d4/dd/d49/d70/dc5 0 2024-09-16T00:26:12.230 INFO:tasks.workunit.client.1.smithi169.stdout:7/759: rename d7/d16/d49/d84 to d7/d5b/d61/d98/d99/df9 0 2024-09-16T00:26:12.231 INFO:tasks.workunit.client.0.smithi112.stdout:5/919: chown d4/d4f/d77/c80 1013224 1 2024-09-16T00:26:12.232 INFO:tasks.workunit.client.1.smithi169.stdout:5/620: symlink d2/da/d10/ld7 0 2024-09-16T00:26:12.232 INFO:tasks.workunit.client.1.smithi169.stdout:7/760: read f5 [1930308,44976] 0 2024-09-16T00:26:12.234 INFO:tasks.workunit.client.0.smithi112.stdout:5/920: mkdir d4/dd/d49/d70/d83/d132 0 2024-09-16T00:26:12.236 INFO:tasks.workunit.client.0.smithi112.stdout:5/921: symlink d4/d4f/l133 0 2024-09-16T00:26:12.238 INFO:tasks.workunit.client.1.smithi169.stdout:1/709: write dc/d36/d3a/d45/d29/d2a/f8e [658895,112649] 0 2024-09-16T00:26:12.238 INFO:tasks.workunit.client.0.smithi112.stdout:7/996: write d6/d10/d16/f9d [1000864,112907] 0 2024-09-16T00:26:12.239 INFO:tasks.workunit.client.1.smithi169.stdout:8/749: mknod d7/d87/dd5/ded/cf5 0 2024-09-16T00:26:12.239 INFO:tasks.workunit.client.0.smithi112.stdout:8/884: write d10/d12/d1c/df2/ffc [500479,74793] 0 2024-09-16T00:26:12.240 INFO:tasks.workunit.client.1.smithi169.stdout:0/733: truncate d0/d2c/d38/de4/d21/d31/d83/fc8 738729 0 2024-09-16T00:26:12.241 INFO:tasks.workunit.client.1.smithi169.stdout:7/761: creat d7/d5b/d61/d98/d99/df9/dc6/ffa x:0 0 0 2024-09-16T00:26:12.242 INFO:tasks.workunit.client.0.smithi112.stdout:8/885: rmdir d10/d12/d1c/d61/d6c/dd1 39 2024-09-16T00:26:12.242 INFO:tasks.workunit.client.1.smithi169.stdout:7/762: chown d7/d71/d76/d29/d78/dba/fe1 17153 1 2024-09-16T00:26:12.243 INFO:tasks.workunit.client.1.smithi169.stdout:1/710: unlink dc/d18/d67/d69/d7b/fba 0 2024-09-16T00:26:12.243 INFO:tasks.workunit.client.0.smithi112.stdout:7/997: truncate d6/d10/d16/d96/f9f 851092 0 2024-09-16T00:26:12.244 INFO:tasks.workunit.client.0.smithi112.stdout:5/922: dwrite d4/d7/d14/d90/dd3/fdd [0,4194304] 0 2024-09-16T00:26:12.245 INFO:tasks.workunit.client.0.smithi112.stdout:8/886: unlink d10/d12/d1c/fa4 0 2024-09-16T00:26:12.246 INFO:tasks.workunit.client.1.smithi169.stdout:1/711: write dc/d36/d3a/d45/d29/d4d/fbe [1008815,13086] 0 2024-09-16T00:26:12.246 INFO:tasks.workunit.client.1.smithi169.stdout:0/734: creat d0/d2/d4b/ff9 x:0 0 0 2024-09-16T00:26:12.247 INFO:tasks.workunit.client.1.smithi169.stdout:7/763: rename d7/d16/d49/lad to d7/d5b/dc1/d96/lfb 0 2024-09-16T00:26:12.248 INFO:tasks.workunit.client.0.smithi112.stdout:7/998: write d6/d10/d16/d96/dc7/dbf/d117/f140 [4833,6960] 0 2024-09-16T00:26:12.248 INFO:tasks.workunit.client.1.smithi169.stdout:0/735: chown d0/d2c/d38/de4/d56/d67/dd1/f7f 75861633 1 2024-09-16T00:26:12.249 INFO:tasks.workunit.client.1.smithi169.stdout:1/712: fdatasync dc/d36/d39/d50/f7a 0 2024-09-16T00:26:12.251 INFO:tasks.workunit.client.0.smithi112.stdout:5/923: rename d4/d7/d26/l34 to d4/dd/d49/d70/dc5/l134 0 2024-09-16T00:26:12.252 INFO:tasks.workunit.client.1.smithi169.stdout:7/764: dread - d7/d71/d76/d29/d78/dba/fcb zero size 2024-09-16T00:26:12.253 INFO:tasks.workunit.client.0.smithi112.stdout:5/924: write d4/dd/d49/da9/f104 [5296160,44434] 0 2024-09-16T00:26:12.254 INFO:tasks.workunit.client.1.smithi169.stdout:1/713: mknod dc/d36/d3a/d45/dce/cd8 0 2024-09-16T00:26:12.257 INFO:tasks.workunit.client.0.smithi112.stdout:5/925: creat d4/dd/df6/f135 x:0 0 0 2024-09-16T00:26:12.258 INFO:tasks.workunit.client.0.smithi112.stdout:8/887: getdents d10/d12/d1c/d61/d6c/dd1/d58 0 2024-09-16T00:26:12.259 INFO:tasks.workunit.client.1.smithi169.stdout:0/736: link d0/d4c/d82/fd7 d0/d2/d8/d5/ffa 0 2024-09-16T00:26:12.259 INFO:tasks.workunit.client.0.smithi112.stdout:6/933: dread dd/df/d17/d27/d54/f6a [0,4194304] 0 2024-09-16T00:26:12.261 INFO:tasks.workunit.client.0.smithi112.stdout:5/926: creat d4/dd/f136 x:0 0 0 2024-09-16T00:26:12.261 INFO:tasks.workunit.client.0.smithi112.stdout:6/934: write dd/df/d17/d27/d84/ff2 [4319100,75934] 0 2024-09-16T00:26:12.262 INFO:tasks.workunit.client.0.smithi112.stdout:8/888: mknod d10/c129 0 2024-09-16T00:26:12.265 INFO:tasks.workunit.client.0.smithi112.stdout:6/935: mknod dd/df/d10/d47/d55/dbd/de9/c133 0 2024-09-16T00:26:12.266 INFO:tasks.workunit.client.1.smithi169.stdout:0/737: link d0/d2c/d38/de4/l72 d0/d7d/lfb 0 2024-09-16T00:26:12.267 INFO:tasks.workunit.client.1.smithi169.stdout:0/738: chown d0/d2c/ld5 868335783 1 2024-09-16T00:26:12.268 INFO:tasks.workunit.client.0.smithi112.stdout:6/936: creat dd/d15/da5/dae/f134 x:0 0 0 2024-09-16T00:26:12.270 INFO:tasks.workunit.client.1.smithi169.stdout:0/739: mknod d0/d4c/cfc 0 2024-09-16T00:26:12.270 INFO:tasks.workunit.client.0.smithi112.stdout:8/889: dwrite d10/d12/d1c/d3b/d74/f7b [0,4194304] 0 2024-09-16T00:26:12.270 INFO:tasks.workunit.client.1.smithi169.stdout:4/685: write d2/d12/fba [2737803,17137] 0 2024-09-16T00:26:12.272 INFO:tasks.workunit.client.0.smithi112.stdout:7/999: truncate d6/d10/d75/dea/f112 239543 0 2024-09-16T00:26:12.273 INFO:tasks.workunit.client.0.smithi112.stdout:8/890: write d10/d12/d1c/d61/d6c/fc4 [5484854,28203] 0 2024-09-16T00:26:12.274 INFO:tasks.workunit.client.1.smithi169.stdout:6/924: write d2/d22/d51/f77 [585285,97824] 0 2024-09-16T00:26:12.274 INFO:tasks.workunit.client.1.smithi169.stdout:2/679: write d7/d1c/fb7 [96728,8917] 0 2024-09-16T00:26:12.275 INFO:tasks.workunit.client.0.smithi112.stdout:5/927: write d4/d1c/d43/fbf [504138,75409] 0 2024-09-16T00:26:12.275 INFO:tasks.workunit.client.1.smithi169.stdout:2/680: chown d7/d1c/dc3/ff3 266762 1 2024-09-16T00:26:12.277 INFO:tasks.workunit.client.0.smithi112.stdout:8/891: write d10/f34 [4699568,82810] 0 2024-09-16T00:26:12.277 INFO:tasks.workunit.client.1.smithi169.stdout:0/740: creat d0/d4c/d82/ffd x:0 0 0 2024-09-16T00:26:12.279 INFO:tasks.workunit.client.0.smithi112.stdout:8/892: read - d10/d12/d1c/d3b/f89 zero size 2024-09-16T00:26:12.279 INFO:tasks.workunit.client.0.smithi112.stdout:6/937: dwrite dd/df/d10/d47/d55/dbd/f101 [0,4194304] 0 2024-09-16T00:26:12.280 INFO:tasks.workunit.client.0.smithi112.stdout:6/938: dread - dd/dcc/f11b zero size 2024-09-16T00:26:12.280 INFO:tasks.workunit.client.0.smithi112.stdout:5/928: creat d4/d4f/d77/dcc/d7c/d99/f137 x:0 0 0 2024-09-16T00:26:12.280 INFO:tasks.workunit.client.1.smithi169.stdout:9/841: dread d5/da/d2e/d99/d100/d10f/d11c/f40 [0,4194304] 0 2024-09-16T00:26:12.281 INFO:tasks.workunit.client.0.smithi112.stdout:8/893: readlink d10/d12/d1c/dd0/l51 0 2024-09-16T00:26:12.281 INFO:tasks.workunit.client.0.smithi112.stdout:6/939: chown dd/df/d10/d47/d55/dc4 1822570 1 2024-09-16T00:26:12.281 INFO:tasks.workunit.client.0.smithi112.stdout:5/929: fdatasync d4/d7/d14/fb0 0 2024-09-16T00:26:12.283 INFO:tasks.workunit.client.0.smithi112.stdout:5/930: read - d4/dd/d49/de6/d113/f12b zero size 2024-09-16T00:26:12.284 INFO:tasks.workunit.client.0.smithi112.stdout:5/931: stat d4/fb 0 2024-09-16T00:26:12.285 INFO:tasks.workunit.client.1.smithi169.stdout:0/741: read d0/fd0 [27390,82925] 0 2024-09-16T00:26:12.286 INFO:tasks.workunit.client.0.smithi112.stdout:8/894: rename d10/c7e to d10/d12/d1c/d61/d6c/dd1/d8c/c12a 0 2024-09-16T00:26:12.287 INFO:tasks.workunit.client.1.smithi169.stdout:6/925: stat d2/d4d/f53 0 2024-09-16T00:26:12.289 INFO:tasks.workunit.client.1.smithi169.stdout:8/750: dread d7/dc/d17/f28 [0,4194304] 0 2024-09-16T00:26:12.289 INFO:tasks.workunit.client.0.smithi112.stdout:6/940: dwrite dd/df/d91/f123 [0,4194304] 0 2024-09-16T00:26:12.290 INFO:tasks.workunit.client.0.smithi112.stdout:5/932: symlink d4/l138 0 2024-09-16T00:26:12.292 INFO:tasks.workunit.client.0.smithi112.stdout:5/933: write d4/d1c/d43/f54 [2400644,54876] 0 2024-09-16T00:26:12.293 INFO:tasks.workunit.client.1.smithi169.stdout:1/714: dread dc/d36/d3a/d45/d29/d4d/f9c [0,4194304] 0 2024-09-16T00:26:12.296 INFO:tasks.workunit.client.0.smithi112.stdout:5/934: mknod d4/d7/d26/d4c/c139 0 2024-09-16T00:26:12.297 INFO:tasks.workunit.client.1.smithi169.stdout:9/842: unlink d5/da/d2e/d47/fad 0 2024-09-16T00:26:12.297 INFO:tasks.workunit.client.1.smithi169.stdout:4/686: dwrite d2/d12/d13/d1f/d3e/d83/daf/f7a [0,4194304] 0 2024-09-16T00:26:12.298 INFO:tasks.workunit.client.0.smithi112.stdout:6/941: getdents dd/df/d11/d44/d8a/dfb 0 2024-09-16T00:26:12.298 INFO:tasks.workunit.client.0.smithi112.stdout:5/935: dread - d4/d4f/d77/dcc/d7c/d99/f137 zero size 2024-09-16T00:26:12.298 INFO:tasks.workunit.client.1.smithi169.stdout:6/926: write d2/d4d/f57 [1819920,105227] 0 2024-09-16T00:26:12.300 INFO:tasks.workunit.client.1.smithi169.stdout:2/681: symlink d7/db/d27/d29/d37/dd1/lf4 0 2024-09-16T00:26:12.301 INFO:tasks.workunit.client.0.smithi112.stdout:5/936: symlink d4/d7/d14/d90/l13a 0 2024-09-16T00:26:12.301 INFO:tasks.workunit.client.0.smithi112.stdout:6/942: getdents dd/df/d17/d27/d102 0 2024-09-16T00:26:12.302 INFO:tasks.workunit.client.0.smithi112.stdout:5/937: fsync d4/d7/f4d 0 2024-09-16T00:26:12.303 INFO:tasks.workunit.client.0.smithi112.stdout:6/943: chown dd/df/d17/d27/d54/lbe 0 1 2024-09-16T00:26:12.303 INFO:tasks.workunit.client.0.smithi112.stdout:5/938: chown d4/le9 23 1 2024-09-16T00:26:12.303 INFO:tasks.workunit.client.0.smithi112.stdout:6/944: dread - dd/dcc/f11b zero size 2024-09-16T00:26:12.304 INFO:tasks.workunit.client.1.smithi169.stdout:7/765: truncate d7/d5b/d61/d98/d99/df9/d8e/fdc 3146764 0 2024-09-16T00:26:12.306 INFO:tasks.workunit.client.1.smithi169.stdout:0/742: dwrite d0/d4c/d82/ffd [0,4194304] 0 2024-09-16T00:26:12.307 INFO:tasks.workunit.client.0.smithi112.stdout:6/945: symlink dd/df/d10/d47/d55/dbd/de9/l135 0 2024-09-16T00:26:12.309 INFO:tasks.workunit.client.0.smithi112.stdout:6/946: mknod dd/df/d11/d44/c136 0 2024-09-16T00:26:12.312 INFO:tasks.workunit.client.1.smithi169.stdout:4/687: mknod d2/d12/d13/d4e/d50/cd8 0 2024-09-16T00:26:12.312 INFO:tasks.workunit.client.1.smithi169.stdout:5/621: dwrite d2/da/f38 [0,4194304] 0 2024-09-16T00:26:12.312 INFO:tasks.workunit.client.1.smithi169.stdout:1/715: creat dc/d36/d3a/d7f/fd9 x:0 0 0 2024-09-16T00:26:12.317 INFO:tasks.workunit.client.1.smithi169.stdout:8/751: mkdir d7/dc/d17/df6 0 2024-09-16T00:26:12.318 INFO:tasks.workunit.client.1.smithi169.stdout:4/688: symlink d2/d12/d13/d5c/da6/ld9 0 2024-09-16T00:26:12.320 INFO:tasks.workunit.client.1.smithi169.stdout:4/689: write d2/d12/d13/f25 [4837907,128729] 0 2024-09-16T00:26:12.320 INFO:tasks.workunit.client.1.smithi169.stdout:1/716: rename dc/d36/f5b to dc/d6c/fda 0 2024-09-16T00:26:12.320 INFO:tasks.workunit.client.1.smithi169.stdout:0/743: mkdir d0/d2c/d38/de4/d21/d31/dfe 0 2024-09-16T00:26:12.322 INFO:tasks.workunit.client.1.smithi169.stdout:8/752: mkdir d7/dc/d26/d41/d4e/d81/d9e/df7 0 2024-09-16T00:26:12.325 INFO:tasks.workunit.client.1.smithi169.stdout:1/717: read dc/d26/f37 [429341,95836] 0 2024-09-16T00:26:12.327 INFO:tasks.workunit.client.1.smithi169.stdout:1/718: fsync dc/f66 0 2024-09-16T00:26:12.328 INFO:tasks.workunit.client.1.smithi169.stdout:0/744: chown d0/d2/d8/d5/ffa 0 1 2024-09-16T00:26:12.328 INFO:tasks.workunit.client.1.smithi169.stdout:6/927: getdents d2/d3f/d41/d72/d9e/dc0 0 2024-09-16T00:26:12.328 INFO:tasks.workunit.client.1.smithi169.stdout:5/622: getdents d2/da/d10/d13/dc0 0 2024-09-16T00:26:12.328 INFO:tasks.workunit.client.1.smithi169.stdout:7/766: write d7/d16/d49/f81 [511743,75798] 0 2024-09-16T00:26:12.331 INFO:tasks.workunit.client.0.smithi112.stdout:6/947: dwrite dd/d23/fc0 [0,4194304] 0 2024-09-16T00:26:12.331 INFO:tasks.workunit.client.1.smithi169.stdout:8/753: truncate d7/dc/d17/d24/f88 1741037 0 2024-09-16T00:26:12.332 INFO:tasks.workunit.client.0.smithi112.stdout:6/948: chown dd/dcc 816110761 1 2024-09-16T00:26:12.334 INFO:tasks.workunit.client.1.smithi169.stdout:7/767: read - d7/d5b/d61/d98/d99/df9/fe2 zero size 2024-09-16T00:26:12.334 INFO:tasks.workunit.client.0.smithi112.stdout:5/939: dread d4/dd/d12/f48 [0,4194304] 0 2024-09-16T00:26:12.335 INFO:tasks.workunit.client.1.smithi169.stdout:1/719: write dc/d36/d3a/d45/d29/d4d/fbe [1530354,36059] 0 2024-09-16T00:26:12.340 INFO:tasks.workunit.client.1.smithi169.stdout:2/682: dwrite d7/db/d20/f83 [0,4194304] 0 2024-09-16T00:26:12.340 INFO:tasks.workunit.client.1.smithi169.stdout:9/843: dwrite d5/da/d2e/d47/f7d [0,4194304] 0 2024-09-16T00:26:12.341 INFO:tasks.workunit.client.0.smithi112.stdout:5/940: dwrite d4/dd/f27 [0,4194304] 0 2024-09-16T00:26:12.341 INFO:tasks.workunit.client.0.smithi112.stdout:5/941: chown d4/dd/c17 181266 1 2024-09-16T00:26:12.342 INFO:tasks.workunit.client.1.smithi169.stdout:7/768: chown d7/d16/d49 60 1 2024-09-16T00:26:12.345 INFO:tasks.workunit.client.0.smithi112.stdout:5/942: truncate d4/d7/d26/f46 2625553 0 2024-09-16T00:26:12.346 INFO:tasks.workunit.client.1.smithi169.stdout:8/754: write d7/dc/d17/f28 [2506762,77818] 0 2024-09-16T00:26:12.347 INFO:tasks.workunit.client.1.smithi169.stdout:8/755: chown d7/dc/d17/f1b 1391576 1 2024-09-16T00:26:12.352 INFO:tasks.workunit.client.0.smithi112.stdout:5/943: dwrite d4/d1c/fad [0,4194304] 0 2024-09-16T00:26:12.353 INFO:tasks.workunit.client.1.smithi169.stdout:1/720: rmdir dc/d36/d3a/d45/d29/dc2/dc7 39 2024-09-16T00:26:12.353 INFO:tasks.workunit.client.0.smithi112.stdout:5/944: chown d4/dd/d49/d70/d83/d132 94388783 1 2024-09-16T00:26:12.354 INFO:tasks.workunit.client.1.smithi169.stdout:4/690: truncate d2/d12/d13/d1f/f8e 1119607 0 2024-09-16T00:26:12.355 INFO:tasks.workunit.client.1.smithi169.stdout:4/691: read d2/d12/d13/d1f/f87 [3420816,6589] 0 2024-09-16T00:26:12.356 INFO:tasks.workunit.client.0.smithi112.stdout:5/945: mkdir d4/d7/d26/d4c/d10e/d13b 0 2024-09-16T00:26:12.357 INFO:tasks.workunit.client.0.smithi112.stdout:5/946: stat d4/d7/d26/d4c/l7a 0 2024-09-16T00:26:12.360 INFO:tasks.workunit.client.0.smithi112.stdout:5/947: mkdir d4/dd/df6/d13c 0 2024-09-16T00:26:12.360 INFO:tasks.workunit.client.1.smithi169.stdout:0/745: rename d0/d2c/d38/l46 to d0/d2/d8/d5/lff 0 2024-09-16T00:26:12.361 INFO:tasks.workunit.client.0.smithi112.stdout:5/948: getdents d4/dd/df6/d13c 0 2024-09-16T00:26:12.363 INFO:tasks.workunit.client.1.smithi169.stdout:6/928: truncate d2/fb 7853129 0 2024-09-16T00:26:12.363 INFO:tasks.workunit.client.1.smithi169.stdout:0/746: readlink d0/d2c/d38/de4/d56/d67/dd1/d6c/l76 0 2024-09-16T00:26:12.364 INFO:tasks.workunit.client.1.smithi169.stdout:1/721: mkdir dc/d36/d3a/d7f/ddb 0 2024-09-16T00:26:12.365 INFO:tasks.workunit.client.0.smithi112.stdout:6/949: dread dd/df/d10/d47/d55/dbd/dcb/de8/f117 [0,4194304] 0 2024-09-16T00:26:12.366 INFO:tasks.workunit.client.1.smithi169.stdout:6/929: write d2/d3f/d41/d5c/d91/da6/de2/f106 [752627,51039] 0 2024-09-16T00:26:12.366 INFO:tasks.workunit.client.1.smithi169.stdout:1/722: truncate dc/d36/d3a/d45/f1c 2933677 0 2024-09-16T00:26:12.367 INFO:tasks.workunit.client.1.smithi169.stdout:5/623: rename d2/da/d10/d13/f14 to d2/da/d10/d13/d1c/d32/d53/d4e/d89/d9a/dad/fd8 0 2024-09-16T00:26:12.368 INFO:tasks.workunit.client.1.smithi169.stdout:6/930: chown d2/cd 170338 1 2024-09-16T00:26:12.369 INFO:tasks.workunit.client.1.smithi169.stdout:4/692: creat d2/d12/d13/d4e/d50/db8/fda x:0 0 0 2024-09-16T00:26:12.369 INFO:tasks.workunit.client.0.smithi112.stdout:6/950: link dd/df/d11/f49 dd/d23/d8f/f137 0 2024-09-16T00:26:12.370 INFO:tasks.workunit.client.1.smithi169.stdout:8/756: creat d7/dc/d53/da0/ff8 x:0 0 0 2024-09-16T00:26:12.370 INFO:tasks.workunit.client.1.smithi169.stdout:1/723: chown dc/d36/d39/d50/daf/d86/f4f 3293489 1 2024-09-16T00:26:12.371 INFO:tasks.workunit.client.1.smithi169.stdout:2/683: rename d7/d1c/d26/d96/dbd to d7/daa/df5 0 2024-09-16T00:26:12.373 INFO:tasks.workunit.client.0.smithi112.stdout:6/951: getdents dd/df/d11/d44/da4/d10c 0 2024-09-16T00:26:12.373 INFO:tasks.workunit.client.1.smithi169.stdout:6/931: unlink d2/f11a 0 2024-09-16T00:26:12.374 INFO:tasks.workunit.client.1.smithi169.stdout:6/932: stat d2/d4d/d59/d74/cf9 0 2024-09-16T00:26:12.375 INFO:tasks.workunit.client.1.smithi169.stdout:4/693: symlink d2/d12/d13/d1f/d3e/d5b/d70/d7d/ldb 0 2024-09-16T00:26:12.376 INFO:tasks.workunit.client.0.smithi112.stdout:6/952: read dd/df/d13/d18/fd3 [3657164,116409] 0 2024-09-16T00:26:12.376 INFO:tasks.workunit.client.1.smithi169.stdout:5/624: creat d2/da/d10/d13/d1c/d40/d54/d7f/fd9 x:0 0 0 2024-09-16T00:26:12.377 INFO:tasks.workunit.client.0.smithi112.stdout:6/953: symlink dd/df/d17/l138 0 2024-09-16T00:26:12.378 INFO:tasks.workunit.client.1.smithi169.stdout:2/684: mkdir d7/d6b/d70/d79/dae/df6 0 2024-09-16T00:26:12.378 INFO:tasks.workunit.client.0.smithi112.stdout:6/954: chown dd/df/d17/l138 231467040 1 2024-09-16T00:26:12.378 INFO:tasks.workunit.client.0.smithi112.stdout:6/955: fdatasync dd/df/d10/f60 0 2024-09-16T00:26:12.380 INFO:tasks.workunit.client.0.smithi112.stdout:6/956: mknod dd/df/d10/d47/d55/dbd/c139 0 2024-09-16T00:26:12.382 INFO:tasks.workunit.client.1.smithi169.stdout:0/747: link d0/d2/d8/d5/d60/c63 d0/d4c/d82/c100 0 2024-09-16T00:26:12.382 INFO:tasks.workunit.client.0.smithi112.stdout:6/957: mknod dd/df/d91/c13a 0 2024-09-16T00:26:12.384 INFO:tasks.workunit.client.1.smithi169.stdout:9/844: rename d5/da/d2e/d99/d100/d10f/d11c/c2f to d5/da/d2e/c12e 0 2024-09-16T00:26:12.386 INFO:tasks.workunit.client.1.smithi169.stdout:6/933: creat d2/d4d/d59/d74/def/f124 x:0 0 0 2024-09-16T00:26:12.390 INFO:tasks.workunit.client.0.smithi112.stdout:5/949: dwrite d4/d1c/d43/f6d [0,4194304] 0 2024-09-16T00:26:12.391 INFO:tasks.workunit.client.1.smithi169.stdout:5/625: creat d2/da/d10/d13/d1c/d40/d68/fda x:0 0 0 2024-09-16T00:26:12.392 INFO:tasks.workunit.client.0.smithi112.stdout:6/958: getdents dd/df/d17 0 2024-09-16T00:26:12.393 INFO:tasks.workunit.client.1.smithi169.stdout:0/748: dread d0/d2c/d38/de4/d56/f59 [0,4194304] 0 2024-09-16T00:26:12.394 INFO:tasks.workunit.client.0.smithi112.stdout:5/950: rmdir d4/d7/d14/d90 39 2024-09-16T00:26:12.395 INFO:tasks.workunit.client.0.smithi112.stdout:6/959: rmdir dd/dcc/de6 39 2024-09-16T00:26:12.396 INFO:tasks.workunit.client.1.smithi169.stdout:8/757: dread d7/dc/d26/d41/f5c [0,4194304] 0 2024-09-16T00:26:12.396 INFO:tasks.workunit.client.1.smithi169.stdout:7/769: dwrite d7/d71/d76/f9e [4194304,4194304] 0 2024-09-16T00:26:12.396 INFO:tasks.workunit.client.0.smithi112.stdout:6/960: write dd/df/d91/fd9 [535377,40523] 0 2024-09-16T00:26:12.397 INFO:tasks.workunit.client.1.smithi169.stdout:7/770: chown d7/f91 6876258 1 2024-09-16T00:26:12.398 INFO:tasks.workunit.client.0.smithi112.stdout:5/951: mkdir d4/d7/d26/d13d 0 2024-09-16T00:26:12.399 INFO:tasks.workunit.client.1.smithi169.stdout:7/771: dread - d7/fe6 zero size 2024-09-16T00:26:12.401 INFO:tasks.workunit.client.0.smithi112.stdout:5/952: rmdir d4/d7/d26/d4c 39 2024-09-16T00:26:12.403 INFO:tasks.workunit.client.1.smithi169.stdout:1/724: write dc/d2c/f32 [2190720,75605] 0 2024-09-16T00:26:12.403 INFO:tasks.workunit.client.0.smithi112.stdout:6/961: link dd/df/d13/f10d dd/df/d10/d47/d55/dbd/dcb/f13b 0 2024-09-16T00:26:12.408 INFO:tasks.workunit.client.0.smithi112.stdout:6/962: fdatasync dd/df/d10/d47/f57 0 2024-09-16T00:26:12.409 INFO:tasks.workunit.client.1.smithi169.stdout:8/758: unlink d7/dc/d26/d41/d92/dcf/d3c/d77/l97 0 2024-09-16T00:26:12.409 INFO:tasks.workunit.client.0.smithi112.stdout:5/953: dwrite d4/dd/df6/f135 [0,4194304] 0 2024-09-16T00:26:12.410 INFO:tasks.workunit.client.1.smithi169.stdout:8/759: fdatasync d7/fa 0 2024-09-16T00:26:12.410 INFO:tasks.workunit.client.0.smithi112.stdout:6/963: write dd/d15/da5/dae/f134 [877773,95002] 0 2024-09-16T00:26:12.412 INFO:tasks.workunit.client.1.smithi169.stdout:8/760: stat d7/dc/d26/d41/d92/dcf/d3c/d77/f7b 0 2024-09-16T00:26:12.416 INFO:tasks.workunit.client.1.smithi169.stdout:8/761: readlink d7/dc/d53/da0/dac/lbe 0 2024-09-16T00:26:12.416 INFO:tasks.workunit.client.1.smithi169.stdout:1/725: rename dc/d36/dbb/dc6 to dc/d36/d3a/d7f/ddc 0 2024-09-16T00:26:12.417 INFO:tasks.workunit.client.1.smithi169.stdout:9/845: rmdir d5/da/d2e 39 2024-09-16T00:26:12.418 INFO:tasks.workunit.client.1.smithi169.stdout:3/882: sync 2024-09-16T00:26:12.419 INFO:tasks.workunit.client.1.smithi169.stdout:7/772: symlink d7/d5b/lfc 0 2024-09-16T00:26:12.422 INFO:tasks.workunit.client.0.smithi112.stdout:5/954: write d4/d7/d14/f33 [1745732,38011] 0 2024-09-16T00:26:12.422 INFO:tasks.workunit.client.1.smithi169.stdout:4/694: dwrite d2/d12/d13/d1f/d3e/d5b/f18 [0,4194304] 0 2024-09-16T00:26:12.424 INFO:tasks.workunit.client.1.smithi169.stdout:2/685: truncate d7/db/db8/d6d/fba 1840018 0 2024-09-16T00:26:12.426 INFO:tasks.workunit.client.1.smithi169.stdout:1/726: write dc/d36/d3a/d45/d29/d2a/f44 [1521113,74796] 0 2024-09-16T00:26:12.426 INFO:tasks.workunit.client.1.smithi169.stdout:3/883: rmdir d7/d50/d6f/d113 39 2024-09-16T00:26:12.428 INFO:tasks.workunit.client.1.smithi169.stdout:1/727: readlink dc/d18/l1b 0 2024-09-16T00:26:12.428 INFO:tasks.workunit.client.0.smithi112.stdout:6/964: dwrite dd/df/d13/f99 [0,4194304] 0 2024-09-16T00:26:12.429 INFO:tasks.workunit.client.1.smithi169.stdout:2/686: symlink d7/d6b/lf7 0 2024-09-16T00:26:12.433 INFO:tasks.workunit.client.1.smithi169.stdout:4/695: symlink d2/d12/d13/d1f/d3e/d5b/ldc 0 2024-09-16T00:26:12.433 INFO:tasks.workunit.client.1.smithi169.stdout:6/934: dread d2/d3f/d41/d72/d9e/dc0/fdc [4194304,4194304] 0 2024-09-16T00:26:12.434 INFO:tasks.workunit.client.0.smithi112.stdout:5/955: dwrite d4/d7/d14/d90/dd3/fdd [0,4194304] 0 2024-09-16T00:26:12.435 INFO:tasks.workunit.client.1.smithi169.stdout:7/773: symlink d7/d16/d49/db2/db4/lfd 0 2024-09-16T00:26:12.436 INFO:tasks.workunit.client.0.smithi112.stdout:8/895: sync 2024-09-16T00:26:12.436 INFO:tasks.workunit.client.0.smithi112.stdout:1/937: sync 2024-09-16T00:26:12.436 INFO:tasks.workunit.client.0.smithi112.stdout:6/965: unlink dd/df/d17/d27/d54/lbe 0 2024-09-16T00:26:12.438 INFO:tasks.workunit.client.1.smithi169.stdout:2/687: mknod d7/db/d89/db4/d69/d92/cf8 0 2024-09-16T00:26:12.441 INFO:tasks.workunit.client.0.smithi112.stdout:5/956: mkdir d4/dd/d49/d70/d98/d13e 0 2024-09-16T00:26:12.441 INFO:tasks.workunit.client.1.smithi169.stdout:2/688: fdatasync d7/d6b/d70/d79/fdc 0 2024-09-16T00:26:12.441 INFO:tasks.workunit.client.1.smithi169.stdout:0/749: dread d0/d2/d4b/d86/f40 [0,4194304] 0 2024-09-16T00:26:12.442 INFO:tasks.workunit.client.0.smithi112.stdout:6/966: mkdir dd/df/d10/d47/d55/d13c 0 2024-09-16T00:26:12.442 INFO:tasks.workunit.client.0.smithi112.stdout:5/957: dread - d4/dd/d49/d70/f12e zero size 2024-09-16T00:26:12.443 INFO:tasks.workunit.client.0.smithi112.stdout:5/958: write d4/d7/d14/f33 [1734974,93152] 0 2024-09-16T00:26:12.443 INFO:tasks.workunit.client.1.smithi169.stdout:1/728: creat dc/d36/d39/d50/daf/d88/dd1/fdd x:0 0 0 2024-09-16T00:26:12.444 INFO:tasks.workunit.client.0.smithi112.stdout:1/938: getdents d0/d14/df/d28 0 2024-09-16T00:26:12.446 INFO:tasks.workunit.client.1.smithi169.stdout:3/884: dwrite d7/d1a/d1c/d35/d63/dc2/dc7/f11e [0,4194304] 0 2024-09-16T00:26:12.448 INFO:tasks.workunit.client.0.smithi112.stdout:1/939: symlink d0/d14/d1c/d20/l132 0 2024-09-16T00:26:12.448 INFO:tasks.workunit.client.1.smithi169.stdout:4/696: creat d2/d12/d13/d5c/da6/dd3/fdd x:0 0 0 2024-09-16T00:26:12.449 INFO:tasks.workunit.client.1.smithi169.stdout:0/750: chown d0/d2c/c62 323159271 1 2024-09-16T00:26:12.449 INFO:tasks.workunit.client.0.smithi112.stdout:5/959: creat d4/dd/d49/d70/d83/dbd/f13f x:0 0 0 2024-09-16T00:26:12.449 INFO:tasks.workunit.client.0.smithi112.stdout:6/967: link dd/df/fc2 dd/df/d11/d44/d8a/dfb/f13d 0 2024-09-16T00:26:12.450 INFO:tasks.workunit.client.0.smithi112.stdout:5/960: dread - d4/d7/d14/d90/dd3/f101 zero size 2024-09-16T00:26:12.450 INFO:tasks.workunit.client.1.smithi169.stdout:2/689: stat d7/db/d89/db4/d69/d92/cf8 0 2024-09-16T00:26:12.451 INFO:tasks.workunit.client.0.smithi112.stdout:6/968: write dd/f2a [4003669,60891] 0 2024-09-16T00:26:12.452 INFO:tasks.workunit.client.1.smithi169.stdout:6/935: creat d2/d3f/f125 x:0 0 0 2024-09-16T00:26:12.453 INFO:tasks.workunit.client.0.smithi112.stdout:6/969: write dd/df/d17/d27/d54/f6a [3372175,88794] 0 2024-09-16T00:26:12.454 INFO:tasks.workunit.client.1.smithi169.stdout:5/626: truncate d2/da/d10/d13/d1c/f6a 802706 0 2024-09-16T00:26:12.456 INFO:tasks.workunit.client.0.smithi112.stdout:1/940: link d0/d14/d6a/fbe d0/d14/df/d28/d45/d46/da7/dda/d123/f133 0 2024-09-16T00:26:12.456 INFO:tasks.workunit.client.0.smithi112.stdout:8/896: write d10/f1b [108026,4300] 0 2024-09-16T00:26:12.457 INFO:tasks.workunit.client.1.smithi169.stdout:5/627: fdatasync d2/da/d10/d13/d1c/d32/d53/d4e/fa8 0 2024-09-16T00:26:12.458 INFO:tasks.workunit.client.1.smithi169.stdout:3/885: rename d7/d1a/d1c/d23/c7f to d7/dbc/c13e 0 2024-09-16T00:26:12.458 INFO:tasks.workunit.client.1.smithi169.stdout:0/751: creat d0/d2c/d38/de4/d56/d67/dd1/d6c/f101 x:0 0 0 2024-09-16T00:26:12.459 INFO:tasks.workunit.client.0.smithi112.stdout:8/897: mknod d10/d12/d1c/d61/d6c/ddc/c12b 0 2024-09-16T00:26:12.459 INFO:tasks.workunit.client.1.smithi169.stdout:6/936: creat d2/d3f/d41/d5c/d91/da6/de2/f126 x:0 0 0 2024-09-16T00:26:12.459 INFO:tasks.workunit.client.1.smithi169.stdout:4/697: mkdir d2/d12/d13/d1f/d3e/dde 0 2024-09-16T00:26:12.460 INFO:tasks.workunit.client.0.smithi112.stdout:1/941: read d0/f9a [11095,58120] 0 2024-09-16T00:26:12.460 INFO:tasks.workunit.client.1.smithi169.stdout:1/729: creat dc/d36/d3a/fde x:0 0 0 2024-09-16T00:26:12.461 INFO:tasks.workunit.client.0.smithi112.stdout:8/898: chown d10/d12/d1c/d61/d6c/dd1/d58/d87/c8a 52775 1 2024-09-16T00:26:12.461 INFO:tasks.workunit.client.1.smithi169.stdout:5/628: chown d2/da/d10/d13/d1c/d32/d53/d83/fa4 60 1 2024-09-16T00:26:12.461 INFO:tasks.workunit.client.1.smithi169.stdout:3/886: fdatasync d7/d50/d90/da6/faa 0 2024-09-16T00:26:12.462 INFO:tasks.workunit.client.0.smithi112.stdout:8/899: readlink d10/d12/d1c/d96/lae 0 2024-09-16T00:26:12.463 INFO:tasks.workunit.client.1.smithi169.stdout:4/698: dread - d2/d12/d13/d1f/d3e/d5b/fd7 zero size 2024-09-16T00:26:12.464 INFO:tasks.workunit.client.0.smithi112.stdout:1/942: read d0/f67 [699773,24820] 0 2024-09-16T00:26:12.465 INFO:tasks.workunit.client.0.smithi112.stdout:8/900: symlink d10/d12/d1c/d61/d6c/ddc/l12c 0 2024-09-16T00:26:12.465 INFO:tasks.workunit.client.1.smithi169.stdout:6/937: write d2/f45 [7446603,111088] 0 2024-09-16T00:26:12.465 INFO:tasks.workunit.client.1.smithi169.stdout:2/690: creat d7/d1c/d26/d96/ff9 x:0 0 0 2024-09-16T00:26:12.466 INFO:tasks.workunit.client.1.smithi169.stdout:8/762: dread d7/f2e [0,4194304] 0 2024-09-16T00:26:12.467 INFO:tasks.workunit.client.0.smithi112.stdout:8/901: rmdir d10/de9 39 2024-09-16T00:26:12.467 INFO:tasks.workunit.client.0.smithi112.stdout:1/943: truncate d0/d14/d1c/d20/f112 296504 0 2024-09-16T00:26:12.468 INFO:tasks.workunit.client.1.smithi169.stdout:9/846: write d5/da/d2e/d47/d11b/fd9 [1552700,26419] 0 2024-09-16T00:26:12.468 INFO:tasks.workunit.client.0.smithi112.stdout:1/944: chown d0/d14/d1c/f51 31 1 2024-09-16T00:26:12.469 INFO:tasks.workunit.client.1.smithi169.stdout:0/752: write d0/d2/d8/d5/fa [1723188,88911] 0 2024-09-16T00:26:12.470 INFO:tasks.workunit.client.1.smithi169.stdout:8/763: write d7/d87/dd5/f7e [5606896,46306] 0 2024-09-16T00:26:12.470 INFO:tasks.workunit.client.1.smithi169.stdout:3/887: mkdir d7/d50/d90/d13a/dc6/d13f 0 2024-09-16T00:26:12.472 INFO:tasks.workunit.client.0.smithi112.stdout:8/902: rename d10/d12/d1c/d61/d6c/dd1/d8c/ca6 to d10/d12/d1c/d96/c12d 0 2024-09-16T00:26:12.475 INFO:tasks.workunit.client.1.smithi169.stdout:0/753: chown d0/d2/d4b/c90 2990 1 2024-09-16T00:26:12.475 INFO:tasks.workunit.client.1.smithi169.stdout:5/629: rmdir d2/da/d10/d13/d1c/d32/d53/dae 39 2024-09-16T00:26:12.475 INFO:tasks.workunit.client.0.smithi112.stdout:1/945: dread d0/f1e [0,4194304] 0 2024-09-16T00:26:12.476 INFO:tasks.workunit.client.1.smithi169.stdout:7/774: write d7/d16/f1d [5545373,69141] 0 2024-09-16T00:26:12.476 INFO:tasks.workunit.client.1.smithi169.stdout:1/730: dwrite dc/d36/d3a/d7f/d93/f6e [0,4194304] 0 2024-09-16T00:26:12.476 INFO:tasks.workunit.client.0.smithi112.stdout:8/903: mkdir d10/d12/d1c/d61/d6c/dd1/d8c/d7c/da5/d10e/d12e 0 2024-09-16T00:26:12.477 INFO:tasks.workunit.client.0.smithi112.stdout:1/946: read d0/d14/d6a/f6f [4180490,124583] 0 2024-09-16T00:26:12.478 INFO:tasks.workunit.client.1.smithi169.stdout:9/847: mknod d5/da/d2e/d99/d100/d12a/c12f 0 2024-09-16T00:26:12.479 INFO:tasks.workunit.client.0.smithi112.stdout:1/947: dread d0/d14/df/d28/d42/dd8/f10b [0,4194304] 0 2024-09-16T00:26:12.480 INFO:tasks.workunit.client.0.smithi112.stdout:5/961: dread d4/dd/d49/d70/f7e [0,4194304] 0 2024-09-16T00:26:12.484 INFO:tasks.workunit.client.0.smithi112.stdout:5/962: creat d4/dd/d49/d70/f140 x:0 0 0 2024-09-16T00:26:12.484 INFO:tasks.workunit.client.1.smithi169.stdout:8/764: creat d7/dc/d17/d48/ff9 x:0 0 0 2024-09-16T00:26:12.486 INFO:tasks.workunit.client.0.smithi112.stdout:5/963: rename d4/d7/d26/d4c/d10e to d4/d7/d26/d4c/d141 0 2024-09-16T00:26:12.487 INFO:tasks.workunit.client.1.smithi169.stdout:9/848: dwrite d5/de0/f109 [0,4194304] 0 2024-09-16T00:26:12.487 INFO:tasks.workunit.client.1.smithi169.stdout:7/775: unlink d7/d5b/fb5 0 2024-09-16T00:26:12.487 INFO:tasks.workunit.client.1.smithi169.stdout:8/765: write d7/dc/d17/d24/d32/d8d/fc8 [3946401,16873] 0 2024-09-16T00:26:12.488 INFO:tasks.workunit.client.0.smithi112.stdout:5/964: symlink d4/d7/d26/d36/dcf/l142 0 2024-09-16T00:26:12.490 INFO:tasks.workunit.client.0.smithi112.stdout:5/965: creat d4/d7/d14/d90/dd3/f143 x:0 0 0 2024-09-16T00:26:12.491 INFO:tasks.workunit.client.1.smithi169.stdout:6/938: write d2/d4c/d9f/fdf [218267,42817] 0 2024-09-16T00:26:12.492 INFO:tasks.workunit.client.1.smithi169.stdout:0/754: creat d0/d2c/d38/de4/d56/d67/dd1/d6c/d9b/f102 x:0 0 0 2024-09-16T00:26:12.493 INFO:tasks.workunit.client.0.smithi112.stdout:5/966: rename d4/dd/d12/d52/d64/la1 to d4/d7/d9b/l144 0 2024-09-16T00:26:12.494 INFO:tasks.workunit.client.1.smithi169.stdout:0/755: chown d0/d2c/d38/de4/fe7 9 1 2024-09-16T00:26:12.495 INFO:tasks.workunit.client.1.smithi169.stdout:5/630: mkdir d2/da/d10/d13/d1c/d2d/ddb 0 2024-09-16T00:26:12.496 INFO:tasks.workunit.client.1.smithi169.stdout:7/776: creat d7/d5b/d61/d98/ffe x:0 0 0 2024-09-16T00:26:12.496 INFO:tasks.workunit.client.1.smithi169.stdout:9/849: creat d5/da/d5b/d91/d9a/f130 x:0 0 0 2024-09-16T00:26:12.499 INFO:tasks.workunit.client.1.smithi169.stdout:5/631: chown d2/da/d10/d13/d1c/d40/f5b 0 1 2024-09-16T00:26:12.500 INFO:tasks.workunit.client.0.smithi112.stdout:1/948: truncate d0/d14/d1c/d20/daf/f59 1215702 0 2024-09-16T00:26:12.501 INFO:tasks.workunit.client.1.smithi169.stdout:5/632: chown d2/da/d10/l18 661 1 2024-09-16T00:26:12.502 INFO:tasks.workunit.client.1.smithi169.stdout:7/777: fsync d7/d16/d49/d74/fc5 0 2024-09-16T00:26:12.503 INFO:tasks.workunit.client.0.smithi112.stdout:1/949: creat d0/d14/df/d28/d45/d46/da7/dda/d123/f134 x:0 0 0 2024-09-16T00:26:12.504 INFO:tasks.workunit.client.1.smithi169.stdout:0/756: mkdir d0/d2c/d103 0 2024-09-16T00:26:12.505 INFO:tasks.workunit.client.1.smithi169.stdout:0/757: chown d0/d2/f6 794485 1 2024-09-16T00:26:12.505 INFO:tasks.workunit.client.1.smithi169.stdout:9/850: mknod d5/d49/dc0/dd6/d11e/c131 0 2024-09-16T00:26:12.506 INFO:tasks.workunit.client.1.smithi169.stdout:3/888: write d7/d50/d90/d13a/f46 [457904,99335] 0 2024-09-16T00:26:12.506 INFO:tasks.workunit.client.1.smithi169.stdout:0/758: chown d0/c4 4432551 1 2024-09-16T00:26:12.507 INFO:tasks.workunit.client.0.smithi112.stdout:5/967: truncate d4/d1c/d43/f6d 3229465 0 2024-09-16T00:26:12.508 INFO:tasks.workunit.client.0.smithi112.stdout:5/968: readlink d4/d7/d26/d4c/db3/db6/dd0/ld1 0 2024-09-16T00:26:12.509 INFO:tasks.workunit.client.0.smithi112.stdout:1/950: link d0/d14/d1c/c5f d0/d14/df/d28/d45/d46/d58/d12a/c135 0 2024-09-16T00:26:12.511 INFO:tasks.workunit.client.0.smithi112.stdout:1/951: fsync d0/d14/df/d28/d45/d46/d58/d62/db0/fc7 0 2024-09-16T00:26:12.511 INFO:tasks.workunit.client.1.smithi169.stdout:9/851: symlink d5/da/d2e/d99/d100/l132 0 2024-09-16T00:26:12.512 INFO:tasks.workunit.client.0.smithi112.stdout:1/952: write d0/d14/df/d28/d45/d46/f49 [3711200,29507] 0 2024-09-16T00:26:12.513 INFO:tasks.workunit.client.1.smithi169.stdout:9/852: readlink d5/da/l69 0 2024-09-16T00:26:12.513 INFO:tasks.workunit.client.1.smithi169.stdout:1/731: dwrite dc/d26/f37 [0,4194304] 0 2024-09-16T00:26:12.514 INFO:tasks.workunit.client.1.smithi169.stdout:1/732: stat dc/d36/d39/d50/f85 0 2024-09-16T00:26:12.518 INFO:tasks.workunit.client.1.smithi169.stdout:5/633: rename d2/da/d10/d13/d1c/d40/d43/l45 to d2/da/d10/d13/d1c/d71/ldc 0 2024-09-16T00:26:12.518 INFO:tasks.workunit.client.0.smithi112.stdout:5/969: write d4/dd/f10f [1181506,116744] 0 2024-09-16T00:26:12.519 INFO:tasks.workunit.client.1.smithi169.stdout:9/853: creat d5/d49/dc0/dd6/d11e/f133 x:0 0 0 2024-09-16T00:26:12.519 INFO:tasks.workunit.client.1.smithi169.stdout:7/778: write d7/d5b/d61/d98/d99/df9/d8e/fdc [1145965,118823] 0 2024-09-16T00:26:12.521 INFO:tasks.workunit.client.1.smithi169.stdout:1/733: creat dc/d36/d39/d50/d75/fdf x:0 0 0 2024-09-16T00:26:12.522 INFO:tasks.workunit.client.0.smithi112.stdout:1/953: dwrite d0/d14/df/d28/d42/fba [4194304,4194304] 0 2024-09-16T00:26:12.522 INFO:tasks.workunit.client.0.smithi112.stdout:5/970: fdatasync d4/d7/fd7 0 2024-09-16T00:26:12.524 INFO:tasks.workunit.client.0.smithi112.stdout:5/971: symlink d4/dd/d12/d52/l145 0 2024-09-16T00:26:12.526 INFO:tasks.workunit.client.1.smithi169.stdout:6/939: dread d2/d36/f44 [0,4194304] 0 2024-09-16T00:26:12.528 INFO:tasks.workunit.client.1.smithi169.stdout:5/634: creat d2/da/d10/d13/d1c/d32/d53/fdd x:0 0 0 2024-09-16T00:26:12.529 INFO:tasks.workunit.client.0.smithi112.stdout:5/972: symlink d4/dd/d49/d70/d83/d132/l146 0 2024-09-16T00:26:12.529 INFO:tasks.workunit.client.1.smithi169.stdout:8/766: dread d7/dc/d26/d46/f5d [0,4194304] 0 2024-09-16T00:26:12.529 INFO:tasks.workunit.client.0.smithi112.stdout:5/973: fdatasync d4/d4f/d78/fa8 0 2024-09-16T00:26:12.529 INFO:tasks.workunit.client.1.smithi169.stdout:0/759: rename d0/d2c/d38/de4/d56 to d0/d2c/d104 0 2024-09-16T00:26:12.532 INFO:tasks.workunit.client.1.smithi169.stdout:1/734: rename dc/d36/d39/d50/l70 to dc/d36/d39/d50/le0 0 2024-09-16T00:26:12.533 INFO:tasks.workunit.client.0.smithi112.stdout:5/974: mknod d4/d7/d26/c147 0 2024-09-16T00:26:12.534 INFO:tasks.workunit.client.1.smithi169.stdout:3/889: dwrite d7/d1a/d1c/d35/d63/d37/f81 [0,4194304] 0 2024-09-16T00:26:12.534 INFO:tasks.workunit.client.0.smithi112.stdout:1/954: write d0/fe5 [649771,105733] 0 2024-09-16T00:26:12.535 INFO:tasks.workunit.client.0.smithi112.stdout:5/975: symlink d4/dd/d49/de6/d113/l148 0 2024-09-16T00:26:12.535 INFO:tasks.workunit.client.0.smithi112.stdout:5/976: readlink d4/d7/d26/d36/dcf/l142 0 2024-09-16T00:26:12.535 INFO:tasks.workunit.client.0.smithi112.stdout:1/955: truncate d0/d14/df/d28/d45/f117 253354 0 2024-09-16T00:26:12.536 INFO:tasks.workunit.client.1.smithi169.stdout:9/854: creat d5/da/dd0/d11f/d121/f134 x:0 0 0 2024-09-16T00:26:12.537 INFO:tasks.workunit.client.0.smithi112.stdout:1/956: rmdir d0/d14/df/d28/d45/dd4/d41/de0 39 2024-09-16T00:26:12.539 INFO:tasks.workunit.client.0.smithi112.stdout:1/957: mkdir d0/d14/d136 0 2024-09-16T00:26:12.539 INFO:tasks.workunit.client.1.smithi169.stdout:8/767: rmdir d7/d87/dd5 39 2024-09-16T00:26:12.540 INFO:tasks.workunit.client.1.smithi169.stdout:0/760: creat d0/d2c/d38/de4/d21/f105 x:0 0 0 2024-09-16T00:26:12.540 INFO:tasks.workunit.client.1.smithi169.stdout:5/635: read d2/da/d10/d13/d1c/d40/d43/f84 [442422,15727] 0 2024-09-16T00:26:12.541 INFO:tasks.workunit.client.0.smithi112.stdout:5/977: rename d4/dd/d49/d70/d83/f86 to d4/dd/d12/f149 0 2024-09-16T00:26:12.542 INFO:tasks.workunit.client.0.smithi112.stdout:1/958: mkdir d0/d14/df/d121/d137 0 2024-09-16T00:26:12.543 INFO:tasks.workunit.client.0.smithi112.stdout:5/978: mkdir d4/d7/d14a 0 2024-09-16T00:26:12.543 INFO:tasks.workunit.client.1.smithi169.stdout:7/779: dwrite d7/d5b/dc1/d96/fbf [0,4194304] 0 2024-09-16T00:26:12.544 INFO:tasks.workunit.client.0.smithi112.stdout:5/979: stat d4/d4f/d77/dcc/f11a 0 2024-09-16T00:26:12.544 INFO:tasks.workunit.client.1.smithi169.stdout:6/940: dread d2/d22/da7/fcd [0,4194304] 0 2024-09-16T00:26:12.546 INFO:tasks.workunit.client.1.smithi169.stdout:8/768: mkdir d7/dc/d26/dfa 0 2024-09-16T00:26:12.547 INFO:tasks.workunit.client.1.smithi169.stdout:0/761: mkdir d0/d4c/d6f/d106 0 2024-09-16T00:26:12.547 INFO:tasks.workunit.client.1.smithi169.stdout:3/890: mknod d7/d1a/d1c/d35/d63/dc2/dc7/d12c/c140 0 2024-09-16T00:26:12.549 INFO:tasks.workunit.client.0.smithi112.stdout:5/980: creat d4/d7/d14/f14b x:0 0 0 2024-09-16T00:26:12.553 INFO:tasks.workunit.client.0.smithi112.stdout:5/981: rename d4/d4f/d77/dcc/d7c/lca to d4/d4f/d77/dcc/d7c/l14c 0 2024-09-16T00:26:12.553 INFO:tasks.workunit.client.1.smithi169.stdout:6/941: creat d2/d3f/d41/d52/f127 x:0 0 0 2024-09-16T00:26:12.554 INFO:tasks.workunit.client.0.smithi112.stdout:5/982: stat d4/d7/d26/d4c/c85 0 2024-09-16T00:26:12.557 INFO:tasks.workunit.client.1.smithi169.stdout:8/769: truncate d7/dc/d26/d41/d92/dcf/d80/dbd/fe2 275409 0 2024-09-16T00:26:12.557 INFO:tasks.workunit.client.1.smithi169.stdout:1/735: dwrite dc/d36/d3a/d7f/d93/d72/d73/f95 [0,4194304] 0 2024-09-16T00:26:12.559 INFO:tasks.workunit.client.0.smithi112.stdout:1/959: write d0/d14/df/d28/d45/f98 [2493877,76771] 0 2024-09-16T00:26:12.559 INFO:tasks.workunit.client.0.smithi112.stdout:5/983: dwrite d4/d7/d14/f112 [0,4194304] 0 2024-09-16T00:26:12.562 INFO:tasks.workunit.client.0.smithi112.stdout:5/984: write d4/d7/d14/fb0 [484131,8778] 0 2024-09-16T00:26:12.563 INFO:tasks.workunit.client.0.smithi112.stdout:5/985: write d4/dd/d49/d70/d83/fe3 [493548,85398] 0 2024-09-16T00:26:12.566 INFO:tasks.workunit.client.0.smithi112.stdout:1/960: rename d0/d14/df/d28/c77 to d0/c138 0 2024-09-16T00:26:12.566 INFO:tasks.workunit.client.0.smithi112.stdout:5/986: symlink d4/dd/df6/l14d 0 2024-09-16T00:26:12.567 INFO:tasks.workunit.client.0.smithi112.stdout:5/987: readlink d4/dd/d49/ddb/ldc 0 2024-09-16T00:26:12.568 INFO:tasks.workunit.client.0.smithi112.stdout:1/961: symlink d0/d14/df/d28/d45/d46/d58/d62/db0/l139 0 2024-09-16T00:26:12.569 INFO:tasks.workunit.client.0.smithi112.stdout:5/988: fdatasync d4/dd/f95 0 2024-09-16T00:26:12.570 INFO:tasks.workunit.client.0.smithi112.stdout:5/989: write d4/dd/f10f [64341,49982] 0 2024-09-16T00:26:12.572 INFO:tasks.workunit.client.0.smithi112.stdout:1/962: creat d0/d14/d1c/f13a x:0 0 0 2024-09-16T00:26:12.572 INFO:tasks.workunit.client.1.smithi169.stdout:8/770: dread - d7/dc/d17/d48/dce/fdd zero size 2024-09-16T00:26:12.573 INFO:tasks.workunit.client.0.smithi112.stdout:5/990: creat d4/d7/d26/d4c/f14e x:0 0 0 2024-09-16T00:26:12.574 INFO:tasks.workunit.client.0.smithi112.stdout:5/991: mkdir d4/dd/d12/d52/d14f 0 2024-09-16T00:26:12.576 INFO:tasks.workunit.client.1.smithi169.stdout:3/891: dwrite d7/d1a/d1c/d35/d63/d37/f81 [4194304,4194304] 0 2024-09-16T00:26:12.578 INFO:tasks.workunit.client.0.smithi112.stdout:5/992: truncate d4/dd/d12/d52/f106 2123463 0 2024-09-16T00:26:12.579 INFO:tasks.workunit.client.0.smithi112.stdout:5/993: chown d4/d1c/d43/fbf 313555 1 2024-09-16T00:26:12.583 INFO:tasks.workunit.client.1.smithi169.stdout:9/855: dwrite d5/da/d21/f45 [0,4194304] 0 2024-09-16T00:26:12.583 INFO:tasks.workunit.client.0.smithi112.stdout:5/994: creat d4/dd/d49/d70/d98/f150 x:0 0 0 2024-09-16T00:26:12.586 INFO:tasks.workunit.client.1.smithi169.stdout:3/892: mkdir d7/d1a/d124/d141 0 2024-09-16T00:26:12.586 INFO:tasks.workunit.client.1.smithi169.stdout:7/780: dwrite d7/d16/d49/d74/fb9 [0,4194304] 0 2024-09-16T00:26:12.590 INFO:tasks.workunit.client.0.smithi112.stdout:5/995: dwrite d4/d4f/d77/dcc/d7c/fc7 [0,4194304] 0 2024-09-16T00:26:12.591 INFO:tasks.workunit.client.1.smithi169.stdout:9/856: mkdir d5/da/d73/d135 0 2024-09-16T00:26:12.593 INFO:tasks.workunit.client.1.smithi169.stdout:1/736: dwrite dc/d36/d3a/d45/fa7 [0,4194304] 0 2024-09-16T00:26:12.595 INFO:tasks.workunit.client.0.smithi112.stdout:5/996: link d4/dd/d12/f3a d4/d7/d26/d4c/db3/db6/f151 0 2024-09-16T00:26:12.599 INFO:tasks.workunit.client.1.smithi169.stdout:5/636: dread d2/da/d10/d13/d1c/d32/d53/d83/d7d/fa1 [0,4194304] 0 2024-09-16T00:26:12.601 INFO:tasks.workunit.client.1.smithi169.stdout:8/771: dwrite d7/dc/d17/fd0 [0,4194304] 0 2024-09-16T00:26:12.601 INFO:tasks.workunit.client.1.smithi169.stdout:1/737: rmdir dc/d36/d39/d50/daf/d86 39 2024-09-16T00:26:12.611 INFO:tasks.workunit.client.1.smithi169.stdout:5/637: fdatasync d2/da/d10/d13/d1c/d40/d43/f4c 0 2024-09-16T00:26:12.621 INFO:tasks.workunit.client.1.smithi169.stdout:8/772: write d7/dc/d17/d48/ff9 [70212,115705] 0 2024-09-16T00:26:12.621 INFO:tasks.workunit.client.1.smithi169.stdout:7/781: rename d7/cd1 to d7/dd4/cff 0 2024-09-16T00:26:12.621 INFO:tasks.workunit.client.0.smithi112.stdout:1/963: getdents d0/d14/d1c 0 2024-09-16T00:26:12.623 INFO:tasks.workunit.client.1.smithi169.stdout:9/857: creat d5/d49/dc0/d11a/f136 x:0 0 0 2024-09-16T00:26:12.624 INFO:tasks.workunit.client.0.smithi112.stdout:1/964: chown d0/d14/df/d28/d45/dd4/d41/f113 2019 1 2024-09-16T00:26:12.625 INFO:tasks.workunit.client.0.smithi112.stdout:1/965: fsync d0/d14/d1c/d20/f53 0 2024-09-16T00:26:12.627 INFO:tasks.workunit.client.1.smithi169.stdout:0/762: dread d0/d2/d8/f68 [4194304,4194304] 0 2024-09-16T00:26:12.628 INFO:tasks.workunit.client.1.smithi169.stdout:6/942: dread d2/f45 [0,4194304] 0 2024-09-16T00:26:12.630 INFO:tasks.workunit.client.1.smithi169.stdout:1/738: dread dc/d36/d3a/d7f/d93/f6e [0,4194304] 0 2024-09-16T00:26:12.630 INFO:tasks.workunit.client.1.smithi169.stdout:3/893: dwrite d7/d50/d90/d13a/d8d/f121 [0,4194304] 0 2024-09-16T00:26:12.631 INFO:tasks.workunit.client.1.smithi169.stdout:9/858: chown d5/da/d5b/d91/d9a/dc3/cd3 14327 1 2024-09-16T00:26:12.640 INFO:tasks.workunit.client.1.smithi169.stdout:8/773: creat d7/dc/d26/d41/d4e/d81/d9e/da5/ffb x:0 0 0 2024-09-16T00:26:12.641 INFO:tasks.workunit.client.1.smithi169.stdout:7/782: unlink d7/d71/d76/d29/l3b 0 2024-09-16T00:26:12.642 INFO:tasks.workunit.client.1.smithi169.stdout:5/638: symlink d2/da/d10/d13/d1c/d71/lde 0 2024-09-16T00:26:12.643 INFO:tasks.workunit.client.1.smithi169.stdout:1/739: symlink dc/d36/d3a/d7f/ddc/le1 0 2024-09-16T00:26:12.643 INFO:tasks.workunit.client.1.smithi169.stdout:5/639: chown d2/da/d10/fa9 1 1 2024-09-16T00:26:12.647 INFO:tasks.workunit.client.1.smithi169.stdout:9/859: creat d5/da/d21/de9/f137 x:0 0 0 2024-09-16T00:26:12.648 INFO:tasks.workunit.client.1.smithi169.stdout:5/640: mkdir d2/da/d10/d13/d1c/d40/d54/d9c/ddf 0 2024-09-16T00:26:12.650 INFO:tasks.workunit.client.1.smithi169.stdout:8/774: truncate d7/dc/d17/f89 941098 0 2024-09-16T00:26:12.651 INFO:tasks.workunit.client.1.smithi169.stdout:8/775: chown d7/dc 757 1 2024-09-16T00:26:12.651 INFO:tasks.workunit.client.1.smithi169.stdout:6/943: creat d2/d3f/d41/d72/d9e/f128 x:0 0 0 2024-09-16T00:26:12.652 INFO:tasks.workunit.client.1.smithi169.stdout:1/740: unlink dc/d18/d67/c6a 0 2024-09-16T00:26:12.654 INFO:tasks.workunit.client.1.smithi169.stdout:0/763: rename d0/d2/d15/fe3 to d0/d2/d8/d5/f107 0 2024-09-16T00:26:12.655 INFO:tasks.workunit.client.1.smithi169.stdout:5/641: creat d2/da/d10/d13/d1c/d71/fe0 x:0 0 0 2024-09-16T00:26:12.656 INFO:tasks.workunit.client.1.smithi169.stdout:9/860: dwrite d5/da/d5b/d91/d9a/f12d [0,4194304] 0 2024-09-16T00:26:12.658 INFO:tasks.workunit.client.1.smithi169.stdout:6/944: dread - d2/d3f/fde zero size 2024-09-16T00:26:12.658 INFO:tasks.workunit.client.0.smithi112.stdout:1/966: dwrite d0/d14/df/d28/d45/d46/da7/dda/d123/f133 [0,4194304] 0 2024-09-16T00:26:12.659 INFO:tasks.workunit.client.1.smithi169.stdout:5/642: chown d2/da/d10/d13 999 1 2024-09-16T00:26:12.664 INFO:tasks.workunit.client.1.smithi169.stdout:0/764: symlink d0/l108 0 2024-09-16T00:26:12.664 INFO:tasks.workunit.client.1.smithi169.stdout:9/861: readlink d5/da/d5b/d6c/da0/ld8 0 2024-09-16T00:26:12.664 INFO:tasks.workunit.client.1.smithi169.stdout:5/643: fdatasync d2/da/d10/d13/d1c/d2d/f63 0 2024-09-16T00:26:12.666 INFO:tasks.workunit.client.1.smithi169.stdout:8/776: link d7/d37/fc2 d7/dc/d17/d24/ffc 0 2024-09-16T00:26:12.667 INFO:tasks.workunit.client.1.smithi169.stdout:6/945: write d2/d36/d48/fe3 [4580905,52600] 0 2024-09-16T00:26:12.668 INFO:tasks.workunit.client.0.smithi112.stdout:1/967: dwrite d0/d14/d1c/d20/f31 [0,4194304] 0 2024-09-16T00:26:12.670 INFO:tasks.workunit.client.1.smithi169.stdout:1/741: dwrite dc/d36/d39/d50/d75/fdf [0,4194304] 0 2024-09-16T00:26:12.673 INFO:tasks.workunit.client.1.smithi169.stdout:3/894: rename d7/d50/d90/d13a/d8d/f121 to d7/d50/d90/d13a/f142 0 2024-09-16T00:26:12.674 INFO:tasks.workunit.client.1.smithi169.stdout:1/742: stat dc/d36/d3a/d45/d29/d4d 0 2024-09-16T00:26:12.675 INFO:tasks.workunit.client.1.smithi169.stdout:8/777: write d7/dc/d26/d41/d4e/d81/d9e/da5/fa6 [2970529,3984] 0 2024-09-16T00:26:12.676 INFO:tasks.workunit.client.1.smithi169.stdout:3/895: chown d7/d1a/d1c/d35/d63/d37/f123 0 1 2024-09-16T00:26:12.679 INFO:tasks.workunit.client.1.smithi169.stdout:0/765: chown d0/d2c/d104/d67/dd1/ld4 929 1 2024-09-16T00:26:12.681 INFO:tasks.workunit.client.1.smithi169.stdout:9/862: mknod d5/da/dd0/d11f/c138 0 2024-09-16T00:26:12.692 INFO:tasks.workunit.client.1.smithi169.stdout:0/766: chown d0/d2c/d38/de4/l74 3 1 2024-09-16T00:26:12.694 INFO:tasks.workunit.client.1.smithi169.stdout:9/863: rename d5/d49/cf4 to d5/da/d5b/d91/db3/c139 0 2024-09-16T00:26:12.696 INFO:tasks.workunit.client.1.smithi169.stdout:8/778: symlink d7/lfd 0 2024-09-16T00:26:12.699 INFO:tasks.workunit.client.1.smithi169.stdout:1/743: dwrite dc/d36/d39/d50/daf/d86/f4f [4194304,4194304] 0 2024-09-16T00:26:12.703 INFO:tasks.workunit.client.1.smithi169.stdout:3/896: mkdir d7/d1a/d1c/d23/d88/dbe/d143 0 2024-09-16T00:26:12.704 INFO:tasks.workunit.client.1.smithi169.stdout:3/897: readlink d7/d50/d90/d13a/d8d/la1 0 2024-09-16T00:26:12.706 INFO:tasks.workunit.client.1.smithi169.stdout:7/783: dwrite d7/d5b/f5c [0,4194304] 0 2024-09-16T00:26:12.711 INFO:tasks.workunit.client.1.smithi169.stdout:8/779: symlink d7/dc/d26/d41/d4e/d81/d9e/lfe 0 2024-09-16T00:26:12.712 INFO:tasks.workunit.client.1.smithi169.stdout:6/946: getdents d2/d4c 0 2024-09-16T00:26:12.713 INFO:tasks.workunit.client.1.smithi169.stdout:0/767: fdatasync d0/d2c/d104/d67/fa1 0 2024-09-16T00:26:12.714 INFO:tasks.workunit.client.1.smithi169.stdout:9/864: mkdir d5/da/d2e/d13a 0 2024-09-16T00:26:12.717 INFO:tasks.workunit.client.1.smithi169.stdout:3/898: dread d7/d1a/d1c/d6c/f7c [0,4194304] 0 2024-09-16T00:26:12.722 INFO:tasks.workunit.client.1.smithi169.stdout:8/780: fsync d7/dc/d26/d41/f5c 0 2024-09-16T00:26:12.729 INFO:tasks.workunit.client.1.smithi169.stdout:8/781: fdatasync d7/dc/d2a/fe7 0 2024-09-16T00:26:12.731 INFO:tasks.workunit.client.1.smithi169.stdout:6/947: mknod d2/d3f/d41/d72/dfc/c129 0 2024-09-16T00:26:12.734 INFO:tasks.workunit.client.1.smithi169.stdout:0/768: dwrite d0/d2/d8/d5/d79/feb [0,4194304] 0 2024-09-16T00:26:12.735 INFO:tasks.workunit.client.1.smithi169.stdout:5/644: dread d2/da/d10/f98 [0,4194304] 0 2024-09-16T00:26:12.735 INFO:tasks.workunit.client.1.smithi169.stdout:7/784: dwrite d7/d16/f6a [0,4194304] 0 2024-09-16T00:26:12.740 INFO:tasks.workunit.client.1.smithi169.stdout:8/782: fdatasync d7/d87/dd5/f7e 0 2024-09-16T00:26:12.743 INFO:tasks.workunit.client.1.smithi169.stdout:3/899: dwrite d7/d50/d90/d13a/dc6/dff/f10f [0,4194304] 0 2024-09-16T00:26:12.747 INFO:tasks.workunit.client.1.smithi169.stdout:0/769: rmdir d0/d2c/d104/d67/dd1/d6c/d9b 39 2024-09-16T00:26:12.747 INFO:tasks.workunit.client.1.smithi169.stdout:8/783: fsync d7/d8c/ff4 0 2024-09-16T00:26:12.748 INFO:tasks.workunit.client.1.smithi169.stdout:9/865: dwrite d5/da/d2e/d99/d100/d10f/d11c/fc7 [0,4194304] 0 2024-09-16T00:26:12.748 INFO:tasks.workunit.client.1.smithi169.stdout:1/744: dread dc/d26/fa0 [0,4194304] 0 2024-09-16T00:26:12.750 INFO:tasks.workunit.client.1.smithi169.stdout:7/785: creat d7/d71/d76/d29/d78/f100 x:0 0 0 2024-09-16T00:26:12.760 INFO:tasks.workunit.client.1.smithi169.stdout:3/900: chown d7/d1a/d1c/d23/d88/dbe/ldc 788355 1 2024-09-16T00:26:12.761 INFO:tasks.workunit.client.1.smithi169.stdout:0/770: creat d0/d2/d15/dc5/f109 x:0 0 0 2024-09-16T00:26:12.763 INFO:tasks.workunit.client.1.smithi169.stdout:5/645: mknod d2/da/d10/d13/d1c/d32/d53/d4e/d89/d9a/dad/ce1 0 2024-09-16T00:26:12.763 INFO:tasks.workunit.client.1.smithi169.stdout:1/745: write dc/d26/f37 [1317975,51387] 0 2024-09-16T00:26:12.769 INFO:tasks.workunit.client.1.smithi169.stdout:9/866: dwrite d5/d49/dc0/d11a/fa1 [0,4194304] 0 2024-09-16T00:26:12.770 INFO:tasks.workunit.client.1.smithi169.stdout:1/746: fdatasync dc/d36/d39/d50/daf/d86/fb2 0 2024-09-16T00:26:12.772 INFO:tasks.workunit.client.1.smithi169.stdout:8/784: dwrite d7/d8c/ff4 [0,4194304] 0 2024-09-16T00:26:12.772 INFO:tasks.workunit.client.1.smithi169.stdout:7/786: dwrite d7/d5b/d61/d98/ffe [0,4194304] 0 2024-09-16T00:26:12.774 INFO:tasks.workunit.client.1.smithi169.stdout:0/771: mknod d0/d2/d8/d5/d79/d7c/c10a 0 2024-09-16T00:26:12.776 INFO:tasks.workunit.client.1.smithi169.stdout:9/867: read - d5/da/d5b/d6c/dcb/f127 zero size 2024-09-16T00:26:12.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:12 smithi169 ceph-mon[34653]: pgmap v259: 65 pgs: 65 active+clean; 2.9 GiB data, 9.9 GiB used, 526 GiB / 536 GiB avail; 115 MiB/s rd, 303 MiB/s wr, 571 op/s 2024-09-16T00:26:12.779 INFO:tasks.workunit.client.1.smithi169.stdout:0/772: write d0/d2c/d38/de4/d21/d31/fc7 [2920265,111511] 0 2024-09-16T00:26:12.780 INFO:tasks.workunit.client.1.smithi169.stdout:5/646: mkdir d2/da/d10/d13/d1c/d32/d53/dae/de2 0 2024-09-16T00:26:12.782 INFO:tasks.workunit.client.1.smithi169.stdout:7/787: mknod d7/d5b/d61/d98/d99/df9/dc8/c101 0 2024-09-16T00:26:12.789 INFO:tasks.workunit.client.1.smithi169.stdout:7/788: dread d7/d5b/d61/d98/ffe [0,4194304] 0 2024-09-16T00:26:12.790 INFO:tasks.workunit.client.1.smithi169.stdout:6/948: dread d2/f19 [0,4194304] 0 2024-09-16T00:26:12.791 INFO:tasks.workunit.client.1.smithi169.stdout:7/789: write d7/df/f21 [4507484,124063] 0 2024-09-16T00:26:12.791 INFO:tasks.workunit.client.1.smithi169.stdout:1/747: creat dc/d36/d3a/d45/fe2 x:0 0 0 2024-09-16T00:26:12.800 INFO:tasks.workunit.client.1.smithi169.stdout:6/949: rename d2/d3f/d41/c83 to d2/da4/daa/c12a 0 2024-09-16T00:26:12.800 INFO:tasks.workunit.client.1.smithi169.stdout:1/748: dwrite dc/f1d [0,4194304] 0 2024-09-16T00:26:12.801 INFO:tasks.workunit.client.1.smithi169.stdout:5/647: creat d2/da/d10/d13/d1c/d32/d53/d4e/fe3 x:0 0 0 2024-09-16T00:26:12.802 INFO:tasks.workunit.client.1.smithi169.stdout:7/790: symlink d7/d5b/l102 0 2024-09-16T00:26:12.806 INFO:tasks.workunit.client.1.smithi169.stdout:5/648: unlink d2/da/d10/d13/d1c/d2d/c7b 0 2024-09-16T00:26:12.808 INFO:tasks.workunit.client.1.smithi169.stdout:6/950: rename d2/d3f/d41/d5c/d91/db7 to d2/d4d/d59/d43/d103/d12b 0 2024-09-16T00:26:12.811 INFO:tasks.workunit.client.1.smithi169.stdout:7/791: creat d7/d5b/d61/d98/f103 x:0 0 0 2024-09-16T00:26:12.811 INFO:tasks.workunit.client.1.smithi169.stdout:8/785: truncate d7/dc/d26/fbc 1693667 0 2024-09-16T00:26:12.813 INFO:tasks.workunit.client.1.smithi169.stdout:5/649: unlink d2/f6b 0 2024-09-16T00:26:12.814 INFO:tasks.workunit.client.1.smithi169.stdout:9/868: dwrite d5/da/d2e/f7b [0,4194304] 0 2024-09-16T00:26:12.816 INFO:tasks.workunit.client.1.smithi169.stdout:1/749: dwrite dc/d36/d39/d50/d75/fc5 [4194304,4194304] 0 2024-09-16T00:26:12.816 INFO:tasks.workunit.client.1.smithi169.stdout:6/951: symlink d2/d36/d120/l12c 0 2024-09-16T00:26:12.817 INFO:tasks.workunit.client.1.smithi169.stdout:0/773: dwrite d0/d2/d4b/d86/f40 [0,4194304] 0 2024-09-16T00:26:12.821 INFO:tasks.workunit.client.0.smithi112.stdout:6/970: sync 2024-09-16T00:26:12.821 INFO:tasks.workunit.client.0.smithi112.stdout:8/904: sync 2024-09-16T00:26:12.821 INFO:tasks.workunit.client.1.smithi169.stdout:8/786: fdatasync d7/dc/d26/d41/fa4 0 2024-09-16T00:26:12.823 INFO:tasks.workunit.client.0.smithi112.stdout:8/905: mknod d10/d12/d1c/d61/d6c/dd1/d8c/d7c/da5/c12f 0 2024-09-16T00:26:12.824 INFO:tasks.workunit.client.1.smithi169.stdout:8/787: unlink d7/dc/d26/c98 0 2024-09-16T00:26:12.825 INFO:tasks.workunit.client.0.smithi112.stdout:6/971: rename dd/df/d17/l81 to dd/df/d17/d27/d84/d118/ddf/l13e 0 2024-09-16T00:26:12.826 INFO:tasks.workunit.client.0.smithi112.stdout:8/906: creat d10/d12/d1c/d3b/f130 x:0 0 0 2024-09-16T00:26:12.828 INFO:tasks.workunit.client.1.smithi169.stdout:9/869: truncate d5/da/f16 1457845 0 2024-09-16T00:26:12.830 INFO:tasks.workunit.client.0.smithi112.stdout:8/907: rename d10/d12/d24/lb0 to d10/d12/d1c/d61/d6c/dd1/d8c/d7c/d97/l131 0 2024-09-16T00:26:12.830 INFO:tasks.workunit.client.1.smithi169.stdout:1/750: mknod dc/d26/ce3 0 2024-09-16T00:26:12.834 INFO:tasks.workunit.client.0.smithi112.stdout:8/908: creat d10/de9/def/d104/f132 x:0 0 0 2024-09-16T00:26:12.835 INFO:tasks.workunit.client.1.smithi169.stdout:5/650: mknod d2/da/d10/d13/d1c/d9e/ce4 0 2024-09-16T00:26:12.836 INFO:tasks.workunit.client.1.smithi169.stdout:9/870: chown d5/da/d5b/c85 2835200 1 2024-09-16T00:26:12.836 INFO:tasks.workunit.client.1.smithi169.stdout:1/751: mknod dc/d36/d3a/d7f/ddc/ce4 0 2024-09-16T00:26:12.837 INFO:tasks.workunit.client.1.smithi169.stdout:9/871: fdatasync d5/d49/dc0/d11a/faa 0 2024-09-16T00:26:12.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:12 smithi112 ceph-mon[27869]: pgmap v259: 65 pgs: 65 active+clean; 2.9 GiB data, 9.9 GiB used, 526 GiB / 536 GiB avail; 115 MiB/s rd, 303 MiB/s wr, 571 op/s 2024-09-16T00:26:12.840 INFO:tasks.workunit.client.1.smithi169.stdout:5/651: creat d2/da/d10/d13/d1c/d40/d43/fe5 x:0 0 0 2024-09-16T00:26:12.840 INFO:tasks.workunit.client.0.smithi112.stdout:8/909: dwrite d10/d12/d1c/d3b/f130 [0,4194304] 0 2024-09-16T00:26:12.841 INFO:tasks.workunit.client.1.smithi169.stdout:0/774: link d0/d2c/d104/d67/dd1/d6c/lc2 d0/d2c/d103/l10b 0 2024-09-16T00:26:12.843 INFO:tasks.workunit.client.0.smithi112.stdout:8/910: symlink d10/d12/d1c/d61/d6c/dd1/d8c/d7c/d97/l133 0 2024-09-16T00:26:12.844 INFO:tasks.workunit.client.1.smithi169.stdout:0/775: stat d0/d2/cb4 0 2024-09-16T00:26:12.845 INFO:tasks.workunit.client.0.smithi112.stdout:8/911: fdatasync d10/d12/d1c/d61/fb4 0 2024-09-16T00:26:12.845 INFO:tasks.workunit.client.0.smithi112.stdout:6/972: dread dd/df/d10/f3a [0,4194304] 0 2024-09-16T00:26:12.845 INFO:tasks.workunit.client.1.smithi169.stdout:9/872: mkdir d5/da/d21/de4/d13b 0 2024-09-16T00:26:12.845 INFO:tasks.workunit.client.1.smithi169.stdout:0/776: chown d0/d2c/d104/d67/dd1/f9f 46568 1 2024-09-16T00:26:12.846 INFO:tasks.workunit.client.0.smithi112.stdout:8/912: chown d10/d12/d24/f2b 678233671 1 2024-09-16T00:26:12.847 INFO:tasks.workunit.client.1.smithi169.stdout:8/788: mknod d7/dc/d17/d24/d32/d8d/cff 0 2024-09-16T00:26:12.848 INFO:tasks.workunit.client.0.smithi112.stdout:6/973: mkdir dd/df/d10/d47/d89/d13f 0 2024-09-16T00:26:12.849 INFO:tasks.workunit.client.1.smithi169.stdout:8/789: write d7/dc/d26/d41/d92/dcf/fc1 [432984,44647] 0 2024-09-16T00:26:12.849 INFO:tasks.workunit.client.1.smithi169.stdout:1/752: creat dc/fe5 x:0 0 0 2024-09-16T00:26:12.849 INFO:tasks.workunit.client.0.smithi112.stdout:8/913: truncate d10/d12/d1c/d61/d6c/dd1/d58/f73 1208162 0 2024-09-16T00:26:12.850 INFO:tasks.workunit.client.1.smithi169.stdout:0/777: rmdir d0/d7d 39 2024-09-16T00:26:12.851 INFO:tasks.workunit.client.0.smithi112.stdout:6/974: creat dd/df/d10/d47/d89/d13f/f140 x:0 0 0 2024-09-16T00:26:12.853 INFO:tasks.workunit.client.0.smithi112.stdout:8/914: mkdir d10/d12/d1c/dd0/dbb/d107/d134 0 2024-09-16T00:26:12.854 INFO:tasks.workunit.client.0.smithi112.stdout:8/915: chown d10/d12/d1c/d61/d6c/dd1/d8c/fd8 13644 1 2024-09-16T00:26:12.855 INFO:tasks.workunit.client.1.smithi169.stdout:8/790: creat d7/d87/dd5/ded/f100 x:0 0 0 2024-09-16T00:26:12.856 INFO:tasks.workunit.client.0.smithi112.stdout:6/975: fsync dd/d23/f63 0 2024-09-16T00:26:12.857 INFO:tasks.workunit.client.0.smithi112.stdout:8/916: creat d10/d12/d1c/d61/f135 x:0 0 0 2024-09-16T00:26:12.858 INFO:tasks.workunit.client.0.smithi112.stdout:6/976: unlink dd/df/d13/d18/c3b 0 2024-09-16T00:26:12.859 INFO:tasks.workunit.client.1.smithi169.stdout:7/792: dread d7/d5b/d61/d98/d99/f9b [0,4194304] 0 2024-09-16T00:26:12.859 INFO:tasks.workunit.client.1.smithi169.stdout:0/778: chown d0/d2c/d38/de4/d21/d31/d83/fc8 44985331 1 2024-09-16T00:26:12.859 INFO:tasks.workunit.client.1.smithi169.stdout:7/793: fsync d7/df/f5d 0 2024-09-16T00:26:12.860 INFO:tasks.workunit.client.0.smithi112.stdout:8/917: unlink d10/d12/d1c/d61/d6c/c86 0 2024-09-16T00:26:12.860 INFO:tasks.workunit.client.1.smithi169.stdout:8/791: dread d7/dc/d17/fd0 [0,4194304] 0 2024-09-16T00:26:12.861 INFO:tasks.workunit.client.1.smithi169.stdout:9/873: dread d5/da/d2e/d47/d11b/f6a [0,4194304] 0 2024-09-16T00:26:12.862 INFO:tasks.workunit.client.1.smithi169.stdout:7/794: write d7/d71/d76/d29/d78/f100 [607123,46582] 0 2024-09-16T00:26:12.865 INFO:tasks.workunit.client.1.smithi169.stdout:1/753: rename dc/d36/d39/d50/daf/c81 to dc/d36/d3a/d7f/d93/d72/ce6 0 2024-09-16T00:26:12.865 INFO:tasks.workunit.client.1.smithi169.stdout:8/792: creat d7/dc/d17/d24/d32/f101 x:0 0 0 2024-09-16T00:26:12.866 INFO:tasks.workunit.client.0.smithi112.stdout:6/977: dwrite dd/df/d17/d27/d54/f6a [4194304,4194304] 0 2024-09-16T00:26:12.866 INFO:tasks.workunit.client.1.smithi169.stdout:1/754: stat dc/d36/d3a/d7f/d93/d72/d98 0 2024-09-16T00:26:12.867 INFO:tasks.workunit.client.1.smithi169.stdout:1/755: chown dc/d18/dd6 14977 1 2024-09-16T00:26:12.867 INFO:tasks.workunit.client.1.smithi169.stdout:9/874: mkdir d5/da/d5b/d91/d13c 0 2024-09-16T00:26:12.869 INFO:tasks.workunit.client.1.smithi169.stdout:0/779: creat d0/d2c/d38/de4/d21/d31/f10c x:0 0 0 2024-09-16T00:26:12.870 INFO:tasks.workunit.client.0.smithi112.stdout:5/997: sync 2024-09-16T00:26:12.870 INFO:tasks.workunit.client.0.smithi112.stdout:1/968: sync 2024-09-16T00:26:12.870 INFO:tasks.workunit.client.0.smithi112.stdout:5/998: stat d4/d4f/d77/dcc 0 2024-09-16T00:26:12.871 INFO:tasks.workunit.client.0.smithi112.stdout:5/999: fdatasync d4/d7/d14/f8c 0 2024-09-16T00:26:12.872 INFO:tasks.workunit.client.0.smithi112.stdout:8/918: link d10/d12/d24/f2b d10/d12/d1c/d61/d6c/dd1/d8c/d7c/da5/d10e/f136 0 2024-09-16T00:26:12.873 INFO:tasks.workunit.client.0.smithi112.stdout:6/978: mknod dd/df/d17/d27/c141 0 2024-09-16T00:26:12.874 INFO:tasks.workunit.client.0.smithi112.stdout:8/919: truncate d10/d12/d24/f29 1856474 0 2024-09-16T00:26:12.875 INFO:tasks.workunit.client.1.smithi169.stdout:0/780: mkdir d0/d10d 0 2024-09-16T00:26:12.877 INFO:tasks.workunit.client.0.smithi112.stdout:8/920: mkdir d10/d12/d1c/d3b/d7d/d137 0 2024-09-16T00:26:12.877 INFO:tasks.workunit.client.1.smithi169.stdout:9/875: getdents d5/d49/d64 0 2024-09-16T00:26:12.878 INFO:tasks.workunit.client.0.smithi112.stdout:6/979: creat dd/df/d10/d47/d55/dc4/d12d/f142 x:0 0 0 2024-09-16T00:26:12.878 INFO:tasks.workunit.client.1.smithi169.stdout:8/793: dwrite d7/dc/d2a/f85 [4194304,4194304] 0 2024-09-16T00:26:12.880 INFO:tasks.workunit.client.0.smithi112.stdout:8/921: creat d10/d12/d24/f138 x:0 0 0 2024-09-16T00:26:12.881 INFO:tasks.workunit.client.1.smithi169.stdout:1/756: write dc/d36/d3a/f78 [47278,1278] 0 2024-09-16T00:26:12.881 INFO:tasks.workunit.client.0.smithi112.stdout:8/922: read - d10/d12/d24/f138 zero size 2024-09-16T00:26:12.883 INFO:tasks.workunit.client.0.smithi112.stdout:1/969: getdents d0/d14/df/d28/d45/dd4/d47/d85 0 2024-09-16T00:26:12.883 INFO:tasks.workunit.client.1.smithi169.stdout:9/876: mkdir d5/da/d2e/d99/d100/d12a/d13d 0 2024-09-16T00:26:12.885 INFO:tasks.workunit.client.1.smithi169.stdout:9/877: stat d5/f8f 0 2024-09-16T00:26:12.885 INFO:tasks.workunit.client.1.smithi169.stdout:1/757: chown dc/d36/f84 2 1 2024-09-16T00:26:12.887 INFO:tasks.workunit.client.1.smithi169.stdout:8/794: rename d7/dc/d53 to d7/dc/d26/d41/d4e/d102 0 2024-09-16T00:26:12.887 INFO:tasks.workunit.client.1.smithi169.stdout:9/878: fsync d5/da/d2e/d47/fdf 0 2024-09-16T00:26:12.887 INFO:tasks.workunit.client.0.smithi112.stdout:8/923: symlink d10/d12/d1c/d3b/d7d/d137/l139 0 2024-09-16T00:26:12.888 INFO:tasks.workunit.client.1.smithi169.stdout:1/758: stat dc/d36/d39/d50/d75/fa8 0 2024-09-16T00:26:12.889 INFO:tasks.workunit.client.0.smithi112.stdout:1/970: creat d0/d14/df/d28/d45/dd4/d47/d10a/f13b x:0 0 0 2024-09-16T00:26:12.889 INFO:tasks.workunit.client.1.smithi169.stdout:1/759: chown dc/d36/d3a/d7f 0 1 2024-09-16T00:26:12.889 INFO:tasks.workunit.client.1.smithi169.stdout:0/781: dwrite d0/d2c/d38/de4/d21/f8e [0,4194304] 0 2024-09-16T00:26:12.890 INFO:tasks.workunit.client.0.smithi112.stdout:6/980: creat dd/df/d13/f143 x:0 0 0 2024-09-16T00:26:12.890 INFO:tasks.workunit.client.1.smithi169.stdout:7/795: dread d7/d71/fae [0,4194304] 0 2024-09-16T00:26:12.890 INFO:tasks.workunit.client.1.smithi169.stdout:9/879: dread - d5/da/d5b/d91/db3/f114 zero size 2024-09-16T00:26:12.892 INFO:tasks.workunit.client.1.smithi169.stdout:1/760: chown dc/d36/d39/d50/daf/d86/l9a 0 1 2024-09-16T00:26:12.892 INFO:tasks.workunit.client.0.smithi112.stdout:1/971: truncate d0/d14/df/d28/d45/d46/da7/fbb 245287 0 2024-09-16T00:26:12.895 INFO:tasks.workunit.client.0.smithi112.stdout:1/972: unlink d0/d14/df/d28/d45/d46/da7/c105 0 2024-09-16T00:26:12.899 INFO:tasks.workunit.client.1.smithi169.stdout:1/761: rename dc/d36/d3a/d45/d29/d4d/f54 to dc/d18/d67/d69/fe7 0 2024-09-16T00:26:12.901 INFO:tasks.workunit.client.0.smithi112.stdout:1/973: dwrite d0/d14/d6a/f11c [0,4194304] 0 2024-09-16T00:26:12.901 INFO:tasks.workunit.client.1.smithi169.stdout:9/880: creat d5/da/d5b/d6c/da0/f13e x:0 0 0 2024-09-16T00:26:12.902 INFO:tasks.workunit.client.1.smithi169.stdout:1/762: chown dc/d36/d3a/d7f/d93/f6d 0 1 2024-09-16T00:26:12.904 INFO:tasks.workunit.client.1.smithi169.stdout:7/796: dread d7/d5b/f6b [0,4194304] 0 2024-09-16T00:26:12.905 INFO:tasks.workunit.client.1.smithi169.stdout:9/881: dread - d5/da/d2e/d47/fce zero size 2024-09-16T00:26:12.905 INFO:tasks.workunit.client.1.smithi169.stdout:8/795: mknod d7/dc/d26/d41/d92/c103 0 2024-09-16T00:26:12.906 INFO:tasks.workunit.client.0.smithi112.stdout:1/974: dread d0/d14/df/d28/f2d [0,4194304] 0 2024-09-16T00:26:12.906 INFO:tasks.workunit.client.0.smithi112.stdout:1/975: stat d0/d14/d6a/fbd 0 2024-09-16T00:26:12.908 INFO:tasks.workunit.client.0.smithi112.stdout:1/976: rmdir d0/d14/d6a/d9e 39 2024-09-16T00:26:12.911 INFO:tasks.workunit.client.0.smithi112.stdout:1/977: symlink d0/d14/df/d28/d45/d46/d58/d62/db0/l13c 0 2024-09-16T00:26:12.914 INFO:tasks.workunit.client.1.smithi169.stdout:1/763: link dc/d36/d3a/d45/d29/fb4 dc/d36/d39/d50/daf/fe8 0 2024-09-16T00:26:12.915 INFO:tasks.workunit.client.1.smithi169.stdout:8/796: creat d7/dc/d26/d41/d4e/d102/da0/dac/dd6/f104 x:0 0 0 2024-09-16T00:26:12.916 INFO:tasks.workunit.client.1.smithi169.stdout:9/882: dwrite d5/da/d5b/d91/db3/f101 [0,4194304] 0 2024-09-16T00:26:12.918 INFO:tasks.workunit.client.1.smithi169.stdout:1/764: write dc/d36/d39/d50/d75/fbc [171437,104215] 0 2024-09-16T00:26:12.921 INFO:tasks.workunit.client.1.smithi169.stdout:0/782: dread d0/f51 [0,4194304] 0 2024-09-16T00:26:12.924 INFO:tasks.workunit.client.1.smithi169.stdout:9/883: symlink d5/da/d73/d135/l13f 0 2024-09-16T00:26:12.924 INFO:tasks.workunit.client.1.smithi169.stdout:8/797: mkdir d7/dc/d17/d24/d32/d105 0 2024-09-16T00:26:12.926 INFO:tasks.workunit.client.1.smithi169.stdout:8/798: symlink d7/dc/d26/d41/d92/dcf/l106 0 2024-09-16T00:26:12.927 INFO:tasks.workunit.client.0.smithi112.stdout:8/924: dread d10/de9/f112 [0,4194304] 0 2024-09-16T00:26:12.929 INFO:tasks.workunit.client.1.smithi169.stdout:8/799: getdents d7/d8c 0 2024-09-16T00:26:12.930 INFO:tasks.workunit.client.1.smithi169.stdout:1/765: dread dc/d36/d3a/f78 [0,4194304] 0 2024-09-16T00:26:12.931 INFO:tasks.workunit.client.0.smithi112.stdout:1/978: dwrite d0/d14/d1c/d20/f63 [0,4194304] 0 2024-09-16T00:26:12.931 INFO:tasks.workunit.client.1.smithi169.stdout:8/800: chown d7/dc/d26/d41/d4e/d102/da0/dac 3200965 1 2024-09-16T00:26:12.931 INFO:tasks.workunit.client.1.smithi169.stdout:1/766: dread - dc/d36/d3a/d7f/fb8 zero size 2024-09-16T00:26:12.932 INFO:tasks.workunit.client.0.smithi112.stdout:8/925: mkdir d10/d12/d1c/d61/d6c/dd1/d8c/d7c/da5/d10e/d13a 0 2024-09-16T00:26:12.932 INFO:tasks.workunit.client.1.smithi169.stdout:1/767: chown dc/d36/d3a/d45/d29/d2a/l64 2308492 1 2024-09-16T00:26:12.934 INFO:tasks.workunit.client.1.smithi169.stdout:7/797: dread d7/d5b/d61/d98/d99/df9/d8e/fdc [0,4194304] 0 2024-09-16T00:26:12.934 INFO:tasks.workunit.client.0.smithi112.stdout:1/979: unlink d0/lb7 0 2024-09-16T00:26:12.935 INFO:tasks.workunit.client.1.smithi169.stdout:1/768: write dc/d36/d3a/d45/d29/d4d/db6/fc3 [729209,85534] 0 2024-09-16T00:26:12.936 INFO:tasks.workunit.client.0.smithi112.stdout:8/926: dread d10/d12/d1c/df2/ffc [0,4194304] 0 2024-09-16T00:26:12.937 INFO:tasks.workunit.client.0.smithi112.stdout:1/980: unlink d0/d14/d6a/ffa 0 2024-09-16T00:26:12.937 INFO:tasks.workunit.client.1.smithi169.stdout:8/801: creat d7/dc/d26/d41/d92/dcf/d3c/d82/db3/f107 x:0 0 0 2024-09-16T00:26:12.939 INFO:tasks.workunit.client.0.smithi112.stdout:8/927: mkdir d10/d12/d1c/d13b 0 2024-09-16T00:26:12.940 INFO:tasks.workunit.client.0.smithi112.stdout:8/928: chown d10/d12/d1c/d61/d6c/ffe 13163 1 2024-09-16T00:26:12.940 INFO:tasks.workunit.client.1.smithi169.stdout:8/802: fsync d7/d8c/ff4 0 2024-09-16T00:26:12.940 INFO:tasks.workunit.client.1.smithi169.stdout:7/798: rename d7/d16/d49/db2/cd0 to d7/d5b/d61/d98/c104 0 2024-09-16T00:26:12.940 INFO:tasks.workunit.client.1.smithi169.stdout:9/884: dread d5/da/fc [0,4194304] 0 2024-09-16T00:26:12.942 INFO:tasks.workunit.client.0.smithi112.stdout:1/981: link d0/d14/fe1 d0/d14/df/d28/d45/dd4/f13d 0 2024-09-16T00:26:12.942 INFO:tasks.workunit.client.1.smithi169.stdout:8/803: dread - d7/dc/d26/d41/d92/dcf/d3c/d82/db3/f107 zero size 2024-09-16T00:26:12.942 INFO:tasks.workunit.client.1.smithi169.stdout:9/885: chown d5/da/d5b/d91/d9a/fa3 137 1 2024-09-16T00:26:12.942 INFO:tasks.workunit.client.1.smithi169.stdout:7/799: creat d7/d71/f105 x:0 0 0 2024-09-16T00:26:12.943 INFO:tasks.workunit.client.0.smithi112.stdout:1/982: chown d0/d14/d1c/d20/d93 0 1 2024-09-16T00:26:12.943 INFO:tasks.workunit.client.0.smithi112.stdout:8/929: dread d10/f63 [0,4194304] 0 2024-09-16T00:26:12.943 INFO:tasks.workunit.client.1.smithi169.stdout:8/804: chown d7/dc/d26/d41/d92/dcf/d3c/d77/dde 1021587 1 2024-09-16T00:26:12.944 INFO:tasks.workunit.client.0.smithi112.stdout:1/983: stat d0/d14/df/d28/d45/fdb 0 2024-09-16T00:26:12.946 INFO:tasks.workunit.client.0.smithi112.stdout:8/930: getdents d10/d12/d1c/d61/d6c/dd1/d8c/d7c/da5/d10e/d12e 0 2024-09-16T00:26:12.947 INFO:tasks.workunit.client.1.smithi169.stdout:8/805: read d7/dc/d2a/f30 [1722052,76627] 0 2024-09-16T00:26:12.948 INFO:tasks.workunit.client.1.smithi169.stdout:7/800: truncate d7/d5b/d61/d98/d99/df9/d8e/fdd 450859 0 2024-09-16T00:26:12.949 INFO:tasks.workunit.client.0.smithi112.stdout:8/931: rename d10/d12/d1c/d61/d6c/dd1/d58/lc1 to d10/d12/d1c/d96/l13c 0 2024-09-16T00:26:12.950 INFO:tasks.workunit.client.0.smithi112.stdout:8/932: fsync d10/d12/d1c/d61/d6c/dd1/f8e 0 2024-09-16T00:26:12.952 INFO:tasks.workunit.client.1.smithi169.stdout:8/806: dwrite d7/dc/d26/d41/f5c [0,4194304] 0 2024-09-16T00:26:12.953 INFO:tasks.workunit.client.0.smithi112.stdout:8/933: symlink d10/d12/d1c/d61/d6c/dd1/d8c/d7c/da5/d10e/d13a/l13d 0 2024-09-16T00:26:12.955 INFO:tasks.workunit.client.1.smithi169.stdout:8/807: rmdir d7/dc/d26/d41/d92/dcf/d3c/d77/dde 39 2024-09-16T00:26:12.958 INFO:tasks.workunit.client.1.smithi169.stdout:8/808: rename d7/lb to d7/dc/d17/d24/d32/l108 0 2024-09-16T00:26:12.960 INFO:tasks.workunit.client.1.smithi169.stdout:8/809: creat d7/dc/d17/d24/d32/d105/f109 x:0 0 0 2024-09-16T00:26:12.961 INFO:tasks.workunit.client.0.smithi112.stdout:1/984: dread d0/d14/df/d28/d45/d46/d58/d62/f88 [0,4194304] 0 2024-09-16T00:26:12.963 INFO:tasks.workunit.client.1.smithi169.stdout:8/810: rmdir d7/dc/d17/df6 0 2024-09-16T00:26:12.963 INFO:tasks.workunit.client.0.smithi112.stdout:1/985: chown d0/d14/df/d28/d45/dd4/d47/d85/fd2 28936 1 2024-09-16T00:26:12.965 INFO:tasks.workunit.client.1.smithi169.stdout:8/811: mkdir d7/dc/d26/d41/d92/dcf/d3c/d82/db3/d10a 0 2024-09-16T00:26:12.965 INFO:tasks.workunit.client.1.smithi169.stdout:8/812: dread - d7/fe8 zero size 2024-09-16T00:26:12.966 INFO:tasks.workunit.client.1.smithi169.stdout:8/813: chown d7/dc/d2a/fe7 486 1 2024-09-16T00:26:12.968 INFO:tasks.workunit.client.1.smithi169.stdout:8/814: creat d7/dc/d26/d41/d92/dcf/d80/dbd/f10b x:0 0 0 2024-09-16T00:26:12.968 INFO:tasks.workunit.client.1.smithi169.stdout:8/815: fdatasync d7/dc/d26/d41/d92/dcf/d80/dbd/f10b 0 2024-09-16T00:26:12.970 INFO:tasks.workunit.client.0.smithi112.stdout:1/986: truncate d0/fe5 844096 0 2024-09-16T00:26:12.972 INFO:tasks.workunit.client.0.smithi112.stdout:1/987: symlink d0/d14/d136/l13e 0 2024-09-16T00:26:12.972 INFO:tasks.workunit.client.0.smithi112.stdout:1/988: chown d0/d14/df/d28/d45/d46/da7/dda/d123 76 1 2024-09-16T00:26:12.973 INFO:tasks.workunit.client.0.smithi112.stdout:1/989: readlink d0/d14/df/d28/d45/d46/d58/d62/l64 0 2024-09-16T00:26:12.976 INFO:tasks.workunit.client.1.smithi169.stdout:8/816: truncate d7/dc/d2a/f30 2786908 0 2024-09-16T00:26:12.977 INFO:tasks.workunit.client.1.smithi169.stdout:8/817: write d7/dc/d26/d41/f5c [2239770,65089] 0 2024-09-16T00:26:12.978 INFO:tasks.workunit.client.1.smithi169.stdout:8/818: write d7/dc/d2a/fba [820024,20035] 0 2024-09-16T00:26:12.979 INFO:tasks.workunit.client.1.smithi169.stdout:8/819: write d7/dc/d17/f28 [1217079,104577] 0 2024-09-16T00:26:12.982 INFO:tasks.workunit.client.1.smithi169.stdout:8/820: creat d7/dc/d26/dfa/f10c x:0 0 0 2024-09-16T00:26:12.983 INFO:tasks.workunit.client.1.smithi169.stdout:8/821: mkdir d7/dc/d26/d41/d92/dcf/df0/d10d 0 2024-09-16T00:26:12.993 INFO:tasks.workunit.client.1.smithi169.stdout:8/822: dread d7/dc/d2a/fba [0,4194304] 0 2024-09-16T00:26:12.997 INFO:tasks.workunit.client.1.smithi169.stdout:8/823: link d7/d87/dd5/ded/cf5 d7/dc/d26/d41/d4e/d102/da0/dac/c10e 0 2024-09-16T00:26:12.998 INFO:tasks.workunit.client.1.smithi169.stdout:8/824: chown d7/dc/c3e 124975 1 2024-09-16T00:26:12.998 INFO:tasks.workunit.client.1.smithi169.stdout:8/825: stat d7/fd 0 2024-09-16T00:26:13.000 INFO:tasks.workunit.client.1.smithi169.stdout:8/826: mknod d7/dc/d17/d24/d32/d8d/c10f 0 2024-09-16T00:26:13.001 INFO:tasks.workunit.client.1.smithi169.stdout:8/827: creat d7/dc/d17/d24/d32/d105/f110 x:0 0 0 2024-09-16T00:26:13.002 INFO:tasks.workunit.client.1.smithi169.stdout:8/828: mkdir d7/dc/d17/d24/d32/d4d/d111 0 2024-09-16T00:26:13.004 INFO:tasks.workunit.client.1.smithi169.stdout:8/829: write d7/d87/dd5/f7d [335289,127782] 0 2024-09-16T00:26:13.053 INFO:tasks.workunit.client.0.smithi112.stdout:1/990: mknod d0/d14/df/d28/d45/dd4/d47/d10a/c13f 0 2024-09-16T00:26:13.053 INFO:tasks.workunit.client.0.smithi112.stdout:1/991: chown d0/d14/d1c/d20/daf/feb 2605 1 2024-09-16T00:26:13.055 INFO:tasks.workunit.client.1.smithi169.stdout:4/699: sync 2024-09-16T00:26:13.070 INFO:tasks.workunit.client.0.smithi112.stdout:1/992: getdents d0/d14/df/d28/d45/dd4/d47/d10a 0 2024-09-16T00:26:13.071 INFO:tasks.workunit.client.0.smithi112.stdout:1/993: read d0/d14/df/d28/f9d [1733978,18087] 0 2024-09-16T00:26:13.074 INFO:tasks.workunit.client.0.smithi112.stdout:1/994: dread d0/d14/d6a/d9e/fab [0,4194304] 0 2024-09-16T00:26:13.075 INFO:tasks.workunit.client.0.smithi112.stdout:1/995: read d0/d14/df/fdf [2881943,128140] 0 2024-09-16T00:26:13.077 INFO:tasks.workunit.client.0.smithi112.stdout:1/996: creat d0/d14/d1c/d20/daf/f140 x:0 0 0 2024-09-16T00:26:13.079 INFO:tasks.workunit.client.0.smithi112.stdout:6/981: sync 2024-09-16T00:26:13.079 INFO:tasks.workunit.client.0.smithi112.stdout:1/997: mkdir d0/d14/df/d28/d141 0 2024-09-16T00:26:13.084 INFO:tasks.workunit.client.0.smithi112.stdout:6/982: dwrite dd/df/d10/d47/d55/dc4/d12d/f142 [0,4194304] 0 2024-09-16T00:26:13.090 INFO:tasks.workunit.client.0.smithi112.stdout:6/983: dwrite dd/d15/da5/d79/fc3 [4194304,4194304] 0 2024-09-16T00:26:13.101 INFO:tasks.workunit.client.0.smithi112.stdout:6/984: dwrite dd/df/d10/f130 [4194304,4194304] 0 2024-09-16T00:26:13.105 INFO:tasks.workunit.client.1.smithi169.stdout:8/830: dwrite d7/dc/d17/f89 [0,4194304] 0 2024-09-16T00:26:13.108 INFO:tasks.workunit.client.1.smithi169.stdout:8/831: mkdir d7/dc/d17/d48/dce/d112 0 2024-09-16T00:26:13.109 INFO:tasks.workunit.client.0.smithi112.stdout:6/985: dwrite dd/df/d10/f130 [4194304,4194304] 0 2024-09-16T00:26:13.111 INFO:tasks.workunit.client.0.smithi112.stdout:6/986: creat dd/df/d17/d27/d54/f144 x:0 0 0 2024-09-16T00:26:13.114 INFO:tasks.workunit.client.1.smithi169.stdout:8/832: rename d7/dc/d17/d24/d32/d8d/dad to d7/dc/d26/dab/d113 0 2024-09-16T00:26:13.118 INFO:tasks.workunit.client.1.smithi169.stdout:8/833: truncate d7/dc/d26/fbc 1893665 0 2024-09-16T00:26:13.119 INFO:tasks.workunit.client.1.smithi169.stdout:8/834: write d7/dc/d2a/f42 [3029949,124517] 0 2024-09-16T00:26:13.124 INFO:tasks.workunit.client.1.smithi169.stdout:8/835: symlink d7/dc/d26/d41/d4e/d102/da0/dac/l114 0 2024-09-16T00:26:13.124 INFO:tasks.workunit.client.1.smithi169.stdout:4/700: dread d2/d12/d13/d1f/d2f/f45 [0,4194304] 0 2024-09-16T00:26:13.126 INFO:tasks.workunit.client.0.smithi112.stdout:6/987: truncate dd/df/d10/f8b 8283865 0 2024-09-16T00:26:13.129 INFO:tasks.workunit.client.0.smithi112.stdout:6/988: unlink dd/df/d11/d44/d8a/c93 0 2024-09-16T00:26:13.130 INFO:tasks.workunit.client.0.smithi112.stdout:6/989: getdents dd/df/d17/d27/d102 0 2024-09-16T00:26:13.131 INFO:tasks.workunit.client.0.smithi112.stdout:6/990: stat dd/d23/c6b 0 2024-09-16T00:26:13.133 INFO:tasks.workunit.client.0.smithi112.stdout:6/991: rename dd/df/d13/d18 to dd/df/d17/d27/d54/d145 0 2024-09-16T00:26:13.136 INFO:tasks.workunit.client.0.smithi112.stdout:6/992: rename dd/df/d10/d47/d55/f7a to dd/d15/da5/f146 0 2024-09-16T00:26:13.140 INFO:tasks.workunit.client.0.smithi112.stdout:6/993: link dd/df/d17/d27/d84/d118/ddf/le1 dd/df/d17/l147 0 2024-09-16T00:26:13.141 INFO:tasks.workunit.client.1.smithi169.stdout:8/836: write d7/dc/d17/d24/f88 [1595148,57318] 0 2024-09-16T00:26:13.141 INFO:tasks.workunit.client.1.smithi169.stdout:4/701: write d2/d12/dcc/d9e/f71 [756914,48586] 0 2024-09-16T00:26:13.142 INFO:tasks.workunit.client.1.smithi169.stdout:4/702: chown d2/d12/d13/d4e/f77 1 1 2024-09-16T00:26:13.142 INFO:tasks.workunit.client.1.smithi169.stdout:8/837: stat d7/dc/d26/d41/d4e/d102/da0/dac/l114 0 2024-09-16T00:26:13.143 INFO:tasks.workunit.client.0.smithi112.stdout:6/994: symlink dd/dcc/de6/l148 0 2024-09-16T00:26:13.143 INFO:tasks.workunit.client.1.smithi169.stdout:4/703: chown d2/d12/d13/d1f/d3e/f53 22 1 2024-09-16T00:26:13.144 INFO:tasks.workunit.client.1.smithi169.stdout:4/704: dread - d2/d12/d9c/da4/fc5 zero size 2024-09-16T00:26:13.144 INFO:tasks.workunit.client.1.smithi169.stdout:8/838: creat d7/dc/d17/d24/d32/f115 x:0 0 0 2024-09-16T00:26:13.145 INFO:tasks.workunit.client.0.smithi112.stdout:6/995: creat dd/df/d10/d47/d55/dbd/f149 x:0 0 0 2024-09-16T00:26:13.145 INFO:tasks.workunit.client.1.smithi169.stdout:4/705: dread d2/d12/dcc/f55 [0,4194304] 0 2024-09-16T00:26:13.145 INFO:tasks.workunit.client.0.smithi112.stdout:6/996: stat dd/df/d10/l14 0 2024-09-16T00:26:13.147 INFO:tasks.workunit.client.0.smithi112.stdout:6/997: mknod dd/df/d17/d27/d54/d145/c14a 0 2024-09-16T00:26:13.147 INFO:tasks.workunit.client.1.smithi169.stdout:4/706: fdatasync d2/d12/dcc/f55 0 2024-09-16T00:26:13.149 INFO:tasks.workunit.client.0.smithi112.stdout:6/998: mkdir dd/dcc/d14b 0 2024-09-16T00:26:13.149 INFO:tasks.workunit.client.0.smithi112.stdout:6/999: chown dd/d15/da5/dae/f120 231 1 2024-09-16T00:26:13.152 INFO:tasks.workunit.client.1.smithi169.stdout:8/839: dwrite d7/f12 [0,4194304] 0 2024-09-16T00:26:13.154 INFO:tasks.workunit.client.1.smithi169.stdout:8/840: fdatasync d7/dc/d17/f89 0 2024-09-16T00:26:13.158 INFO:tasks.workunit.client.1.smithi169.stdout:4/707: dread d2/d12/d13/d1f/d2f/f43 [0,4194304] 0 2024-09-16T00:26:13.163 INFO:tasks.workunit.client.1.smithi169.stdout:8/841: dwrite d7/dc/d17/f1b [0,4194304] 0 2024-09-16T00:26:13.166 INFO:tasks.workunit.client.1.smithi169.stdout:8/842: mknod d7/dc/d17/d24/c116 0 2024-09-16T00:26:13.169 INFO:tasks.workunit.client.1.smithi169.stdout:8/843: fsync d7/dc/d26/d41/d92/dcf/f91 0 2024-09-16T00:26:13.172 INFO:tasks.workunit.client.1.smithi169.stdout:8/844: mkdir d7/dc/d26/d41/d117 0 2024-09-16T00:26:13.175 INFO:tasks.workunit.client.1.smithi169.stdout:8/845: mknod d7/dc/d17/d24/d32/d8d/c118 0 2024-09-16T00:26:13.179 INFO:tasks.workunit.client.1.smithi169.stdout:4/708: dread d2/f3 [0,4194304] 0 2024-09-16T00:26:13.182 INFO:tasks.workunit.client.1.smithi169.stdout:4/709: creat d2/fdf x:0 0 0 2024-09-16T00:26:13.189 INFO:tasks.workunit.client.1.smithi169.stdout:8/846: dwrite d7/f10 [0,4194304] 0 2024-09-16T00:26:13.192 INFO:tasks.workunit.client.1.smithi169.stdout:8/847: unlink d7/dc/d2a/fe7 0 2024-09-16T00:26:13.196 INFO:tasks.workunit.client.0.smithi112.stdout:8/934: sync 2024-09-16T00:26:13.198 INFO:tasks.workunit.client.0.smithi112.stdout:8/935: mknod d10/d12/d1c/d3b/d74/d118/c13e 0 2024-09-16T00:26:13.202 INFO:tasks.workunit.client.1.smithi169.stdout:4/710: dread d2/d12/d13/d1f/d3e/f8c [0,4194304] 0 2024-09-16T00:26:13.203 INFO:tasks.workunit.client.0.smithi112.stdout:8/936: dwrite d10/d12/f115 [0,4194304] 0 2024-09-16T00:26:13.207 INFO:tasks.workunit.client.0.smithi112.stdout:8/937: symlink d10/d12/d1c/l13f 0 2024-09-16T00:26:13.210 INFO:tasks.workunit.client.1.smithi169.stdout:4/711: dwrite d2/dbb/d59/fcb [0,4194304] 0 2024-09-16T00:26:13.211 INFO:tasks.workunit.client.1.smithi169.stdout:4/712: write d2/d12/d13/d5c/f94 [2504264,37691] 0 2024-09-16T00:26:13.218 INFO:tasks.workunit.client.1.smithi169.stdout:4/713: link d2/l7 d2/d12/d13/d1f/d3e/d83/daf/le0 0 2024-09-16T00:26:13.219 INFO:tasks.workunit.client.0.smithi112.stdout:8/938: truncate d10/d12/d1c/d61/d6c/fc4 2173322 0 2024-09-16T00:26:13.219 INFO:tasks.workunit.client.1.smithi169.stdout:4/714: dread - d2/d12/fbe zero size 2024-09-16T00:26:13.220 INFO:tasks.workunit.client.0.smithi112.stdout:8/939: mknod d10/d12/d1c/dd0/dbb/d107/c140 0 2024-09-16T00:26:13.221 INFO:tasks.workunit.client.0.smithi112.stdout:1/998: sync 2024-09-16T00:26:13.224 INFO:tasks.workunit.client.0.smithi112.stdout:8/940: dread d10/de9/def/fe6 [0,4194304] 0 2024-09-16T00:26:13.224 INFO:tasks.workunit.client.0.smithi112.stdout:1/999: truncate d0/fe5 1622108 0 2024-09-16T00:26:13.229 INFO:tasks.workunit.client.0.smithi112.stdout:8/941: dread - d10/d12/d1c/d61/d6c/dd1/d8c/d7c/da5/d10e/f113 zero size 2024-09-16T00:26:13.229 INFO:tasks.workunit.client.0.smithi112.stdout:8/942: unlink d10/d12/d1c/d61/d6c/ddc/l12c 0 2024-09-16T00:26:13.230 INFO:tasks.workunit.client.1.smithi169.stdout:4/715: dwrite d2/d12/d13/d1f/d3e/d5b/f9f [0,4194304] 0 2024-09-16T00:26:13.230 INFO:tasks.workunit.client.1.smithi169.stdout:4/716: symlink d2/le1 0 2024-09-16T00:26:13.230 INFO:tasks.workunit.client.1.smithi169.stdout:4/717: mkdir d2/d12/d9c/de2 0 2024-09-16T00:26:13.231 INFO:tasks.workunit.client.0.smithi112.stdout:8/943: symlink d10/d12/d1c/d3b/d7d/de4/l141 0 2024-09-16T00:26:13.231 INFO:tasks.workunit.client.1.smithi169.stdout:4/718: write d2/d12/d13/d1f/d3e/d5b/f9f [1677997,72346] 0 2024-09-16T00:26:13.234 INFO:tasks.workunit.client.1.smithi169.stdout:4/719: truncate d2/d12/d13/d1f/d3e/d83/f6e 4482423 0 2024-09-16T00:26:13.236 INFO:tasks.workunit.client.1.smithi169.stdout:2/691: sync 2024-09-16T00:26:13.239 INFO:tasks.workunit.client.1.smithi169.stdout:2/692: creat d7/db/db8/d6d/df1/ffa x:0 0 0 2024-09-16T00:26:13.256 INFO:tasks.workunit.client.1.smithi169.stdout:2/693: dread d7/db/d27/d29/d77/f86 [0,4194304] 0 2024-09-16T00:26:13.258 INFO:tasks.workunit.client.1.smithi169.stdout:2/694: chown d7/db/f60 8 1 2024-09-16T00:26:13.268 INFO:tasks.workunit.client.1.smithi169.stdout:2/695: dread d7/db/f17 [0,4194304] 0 2024-09-16T00:26:13.273 INFO:tasks.workunit.client.1.smithi169.stdout:2/696: dwrite f2 [8388608,4194304] 0 2024-09-16T00:26:13.276 INFO:tasks.workunit.client.1.smithi169.stdout:2/697: mkdir d7/db/d89/dfb 0 2024-09-16T00:26:13.280 INFO:tasks.workunit.client.1.smithi169.stdout:2/698: read d7/d1c/f72 [279564,109277] 0 2024-09-16T00:26:13.283 INFO:tasks.workunit.client.1.smithi169.stdout:2/699: symlink d7/db/d89/db4/d9d/dbe/de4/lfc 0 2024-09-16T00:26:13.284 INFO:tasks.workunit.client.1.smithi169.stdout:2/700: mknod d7/db/db8/d6d/df1/cfd 0 2024-09-16T00:26:13.339 INFO:tasks.workunit.client.0.smithi112.stdout:8/944: sync 2024-09-16T00:26:13.385 INFO:tasks.workunit.client.0.smithi112.stdout:8/945: dwrite d10/d12/d1c/d61/d6c/ddc/fee [0,4194304] 0 2024-09-16T00:26:13.387 INFO:tasks.workunit.client.0.smithi112.stdout:8/946: read - d10/d12/d1c/d61/d6c/dd1/d8c/fd8 zero size 2024-09-16T00:26:13.390 INFO:tasks.workunit.client.0.smithi112.stdout:8/947: chown d10/d12/c2d 154 1 2024-09-16T00:26:13.392 INFO:tasks.workunit.client.0.smithi112.stdout:8/948: write d10/d12/d1c/d61/d6c/dd1/f6f [1069554,27873] 0 2024-09-16T00:26:13.393 INFO:tasks.workunit.client.0.smithi112.stdout:8/949: chown d10/d12/d24/f28 86019 1 2024-09-16T00:26:13.396 INFO:tasks.workunit.client.0.smithi112.stdout:8/950: rename d10/d12/fd7 to d10/d12/d1c/d13b/f142 0 2024-09-16T00:26:13.400 INFO:tasks.workunit.client.0.smithi112.stdout:8/951: mkdir d10/d12/d1c/d61/d6c/dd1/d58/d143 0 2024-09-16T00:26:13.401 INFO:tasks.workunit.client.0.smithi112.stdout:8/952: write d10/d12/d1c/d61/d6c/ffe [1033687,17148] 0 2024-09-16T00:26:13.405 INFO:tasks.workunit.client.0.smithi112.stdout:8/953: creat d10/d12/d1c/d61/d6c/dd1/d8c/d7c/f144 x:0 0 0 2024-09-16T00:26:13.408 INFO:tasks.workunit.client.0.smithi112.stdout:8/954: creat d10/d12/d1c/d61/d6c/dd1/d8c/d7c/da5/f145 x:0 0 0 2024-09-16T00:26:13.410 INFO:tasks.workunit.client.0.smithi112.stdout:8/955: creat d10/d12/d1c/d13b/f146 x:0 0 0 2024-09-16T00:26:13.410 INFO:tasks.workunit.client.0.smithi112.stdout:8/956: stat d10/d12/d1c/d3b/d74/f90 0 2024-09-16T00:26:13.412 INFO:tasks.workunit.client.0.smithi112.stdout:8/957: creat d10/d12/d1c/dd0/dbb/d107/f147 x:0 0 0 2024-09-16T00:26:13.415 INFO:tasks.workunit.client.0.smithi112.stdout:8/958: rename d10/d12/d1c/d3b/c92 to d10/d12/d1c/df2/c148 0 2024-09-16T00:26:13.416 INFO:tasks.workunit.client.0.smithi112.stdout:8/959: chown d10/d12/d1c/d61/d6c/fdb 96 1 2024-09-16T00:26:13.417 INFO:tasks.workunit.client.1.smithi169.stdout:3/901: sync 2024-09-16T00:26:13.420 INFO:tasks.workunit.client.1.smithi169.stdout:3/902: mkdir d7/d1a/d1c/d23/d88/dbe/d144 0 2024-09-16T00:26:13.423 INFO:tasks.workunit.client.1.smithi169.stdout:3/903: unlink d7/d50/d90/dba/f11f 0 2024-09-16T00:26:13.425 INFO:tasks.workunit.client.1.smithi169.stdout:3/904: stat d7/d50/d6f/d113/d12f 0 2024-09-16T00:26:13.426 INFO:tasks.workunit.client.1.smithi169.stdout:3/905: write d7/d1a/d1c/d23/d70/f8e [4114666,87865] 0 2024-09-16T00:26:13.428 INFO:tasks.workunit.client.1.smithi169.stdout:3/906: stat d7/d50/d90/da6/fd9 0 2024-09-16T00:26:13.430 INFO:tasks.workunit.client.1.smithi169.stdout:3/907: creat d7/d1a/d1c/d35/d63/f145 x:0 0 0 2024-09-16T00:26:13.436 INFO:tasks.workunit.client.0.smithi112.stdout:8/960: symlink d10/d12/d1c/d61/d6c/dd1/d58/d87/df0/l149 0 2024-09-16T00:26:13.438 INFO:tasks.workunit.client.0.smithi112.stdout:8/961: creat d10/d12/d1c/d61/d6c/dd1/d8c/d7c/da5/f14a x:0 0 0 2024-09-16T00:26:13.442 INFO:tasks.workunit.client.0.smithi112.stdout:8/962: dread d10/d12/f115 [0,4194304] 0 2024-09-16T00:26:13.445 INFO:tasks.workunit.client.0.smithi112.stdout:8/963: mknod d10/d12/d1c/d61/d6c/dd1/d58/d143/c14b 0 2024-09-16T00:26:13.446 INFO:tasks.workunit.client.1.smithi169.stdout:3/908: dread d7/d1a/d1c/d23/d88/d9b/dc8/ff7 [0,4194304] 0 2024-09-16T00:26:13.452 INFO:tasks.workunit.client.1.smithi169.stdout:3/909: truncate d7/d50/d90/d13a/f69 1112730 0 2024-09-16T00:26:13.454 INFO:tasks.workunit.client.1.smithi169.stdout:3/910: rmdir d7/d1a/d1c/d35/d63/da3 39 2024-09-16T00:26:13.456 INFO:tasks.workunit.client.1.smithi169.stdout:3/911: fdatasync d7/d1a/d1c/d23/d88/d9b/dc8/df8/f109 0 2024-09-16T00:26:13.461 INFO:tasks.workunit.client.1.smithi169.stdout:3/912: dwrite d7/d1a/d1c/d35/d63/d37/f101 [0,4194304] 0 2024-09-16T00:26:13.463 INFO:tasks.workunit.client.1.smithi169.stdout:3/913: write d7/d1a/d1c/d23/d88/d9b/dc8/dcc/fdd [632835,130296] 0 2024-09-16T00:26:13.466 INFO:tasks.workunit.client.1.smithi169.stdout:3/914: dread d7/d1a/d1c/d23/d88/dbe/f110 [0,4194304] 0 2024-09-16T00:26:13.471 INFO:tasks.workunit.client.1.smithi169.stdout:3/915: dwrite d7/d1a/d1c/d23/d88/ffa [0,4194304] 0 2024-09-16T00:26:13.472 INFO:tasks.workunit.client.1.smithi169.stdout:3/916: readlink d7/d50/l10a 0 2024-09-16T00:26:13.491 INFO:tasks.workunit.client.1.smithi169.stdout:3/917: dread d7/d50/d90/d13a/f46 [0,4194304] 0 2024-09-16T00:26:13.492 INFO:tasks.workunit.client.1.smithi169.stdout:3/918: rmdir d7/d1a/d1c/d35/d63/dea/df3 39 2024-09-16T00:26:13.579 INFO:tasks.workunit.client.1.smithi169.stdout:6/952: sync 2024-09-16T00:26:13.579 INFO:tasks.workunit.client.1.smithi169.stdout:9/886: sync 2024-09-16T00:26:13.581 INFO:tasks.workunit.client.1.smithi169.stdout:9/887: stat d5/da/d21/de4/cfe 0 2024-09-16T00:26:13.582 INFO:tasks.workunit.client.1.smithi169.stdout:9/888: write d5/d49/f110 [431536,66645] 0 2024-09-16T00:26:13.582 INFO:tasks.workunit.client.0.smithi112.stdout:8/964: sync 2024-09-16T00:26:13.583 INFO:tasks.workunit.client.1.smithi169.stdout:6/953: rename d2/d4d/d59/d38/c3b to d2/d36/d115/c12d 0 2024-09-16T00:26:13.585 INFO:tasks.workunit.client.0.smithi112.stdout:8/965: mknod d10/d12/d1c/d61/d6c/dd1/c14c 0 2024-09-16T00:26:13.585 INFO:tasks.workunit.client.1.smithi169.stdout:9/889: mkdir d5/da/d21/de4/d140 0 2024-09-16T00:26:13.585 INFO:tasks.workunit.client.1.smithi169.stdout:9/890: write d5/da/d21/de9/f125 [1319747,80179] 0 2024-09-16T00:26:13.586 INFO:tasks.workunit.client.1.smithi169.stdout:9/891: readlink d5/l3a 0 2024-09-16T00:26:13.589 INFO:tasks.workunit.client.1.smithi169.stdout:6/954: rename d2/d4d/d59/d38 to d2/d4c/d12e 0 2024-09-16T00:26:13.591 INFO:tasks.workunit.client.1.smithi169.stdout:9/892: getdents d5/da/d2e/d99/d100/d12a 0 2024-09-16T00:26:13.592 INFO:tasks.workunit.client.1.smithi169.stdout:9/893: chown d5/da/d5b/d6c/da0/l6e 21855 1 2024-09-16T00:26:13.593 INFO:tasks.workunit.client.1.smithi169.stdout:9/894: read d5/f14 [36278,17582] 0 2024-09-16T00:26:13.595 INFO:tasks.workunit.client.1.smithi169.stdout:6/955: dread d2/d22/fda [0,4194304] 0 2024-09-16T00:26:13.595 INFO:tasks.workunit.client.0.smithi112.stdout:8/966: dread d10/d12/d1c/dd0/f78 [0,4194304] 0 2024-09-16T00:26:13.597 INFO:tasks.workunit.client.1.smithi169.stdout:6/956: creat d2/d4d/d59/d43/f12f x:0 0 0 2024-09-16T00:26:13.598 INFO:tasks.workunit.client.1.smithi169.stdout:9/895: truncate d5/da/d21/de4/f9c 729043 0 2024-09-16T00:26:13.599 INFO:tasks.workunit.client.1.smithi169.stdout:9/896: chown d5/da/d5b 107 1 2024-09-16T00:26:13.600 INFO:tasks.workunit.client.0.smithi112.stdout:8/967: dwrite d10/d12/f16 [0,4194304] 0 2024-09-16T00:26:13.601 INFO:tasks.workunit.client.0.smithi112.stdout:8/968: fdatasync d10/d12/d1c/d61/d6c/ffe 0 2024-09-16T00:26:13.604 INFO:tasks.workunit.client.1.smithi169.stdout:9/897: dwrite d5/da/f122 [0,4194304] 0 2024-09-16T00:26:13.604 INFO:tasks.workunit.client.1.smithi169.stdout:7/801: sync 2024-09-16T00:26:13.604 INFO:tasks.workunit.client.1.smithi169.stdout:2/701: sync 2024-09-16T00:26:13.604 INFO:tasks.workunit.client.1.smithi169.stdout:0/783: sync 2024-09-16T00:26:13.604 INFO:tasks.workunit.client.1.smithi169.stdout:4/720: sync 2024-09-16T00:26:13.604 INFO:tasks.workunit.client.1.smithi169.stdout:5/652: sync 2024-09-16T00:26:13.604 INFO:tasks.workunit.client.1.smithi169.stdout:8/848: sync 2024-09-16T00:26:13.605 INFO:tasks.workunit.client.1.smithi169.stdout:1/769: sync 2024-09-16T00:26:13.605 INFO:tasks.workunit.client.1.smithi169.stdout:3/919: sync 2024-09-16T00:26:13.605 INFO:tasks.workunit.client.0.smithi112.stdout:8/969: creat d10/d12/f14d x:0 0 0 2024-09-16T00:26:13.608 INFO:tasks.workunit.client.1.smithi169.stdout:9/898: write d5/da/d73/f118 [270047,62900] 0 2024-09-16T00:26:13.609 INFO:tasks.workunit.client.0.smithi112.stdout:8/970: dread d10/d12/d1c/d61/d6c/dd1/f8e [0,4194304] 0 2024-09-16T00:26:13.610 INFO:tasks.workunit.client.1.smithi169.stdout:8/849: mknod d7/dc/d2a/c119 0 2024-09-16T00:26:13.614 INFO:tasks.workunit.client.1.smithi169.stdout:9/899: dread - d5/da/d5b/d6c/da0/f106 zero size 2024-09-16T00:26:13.615 INFO:tasks.workunit.client.1.smithi169.stdout:3/920: creat d7/d1a/d1c/d23/f146 x:0 0 0 2024-09-16T00:26:13.616 INFO:tasks.workunit.client.1.smithi169.stdout:1/770: mkdir dc/d36/d3a/d7f/d93/d72/de9 0 2024-09-16T00:26:13.617 INFO:tasks.workunit.client.1.smithi169.stdout:4/721: rmdir d2/d12/da7/dac 0 2024-09-16T00:26:13.620 INFO:tasks.workunit.client.1.smithi169.stdout:4/722: chown d2/d12/c91 1395 1 2024-09-16T00:26:13.620 INFO:tasks.workunit.client.1.smithi169.stdout:7/802: dwrite d7/f7e [0,4194304] 0 2024-09-16T00:26:13.627 INFO:tasks.workunit.client.1.smithi169.stdout:8/850: write d7/dc/d17/d24/d32/fc0 [243367,121464] 0 2024-09-16T00:26:13.627 INFO:tasks.workunit.client.1.smithi169.stdout:9/900: mkdir d5/da/d21/d141 0 2024-09-16T00:26:13.629 INFO:tasks.workunit.client.0.smithi112.stdout:8/971: dread d10/f1f [0,4194304] 0 2024-09-16T00:26:13.630 INFO:tasks.workunit.client.0.smithi112.stdout:8/972: dread - d10/d12/d1c/d96/fa7 zero size 2024-09-16T00:26:13.632 INFO:tasks.workunit.client.1.smithi169.stdout:5/653: write d2/da/d10/f7a [149780,16289] 0 2024-09-16T00:26:13.632 INFO:tasks.workunit.client.1.smithi169.stdout:2/702: write d7/db/f60 [991705,129600] 0 2024-09-16T00:26:13.632 INFO:tasks.workunit.client.1.smithi169.stdout:0/784: dwrite d0/d2c/d38/de4/d21/d31/f58 [0,4194304] 0 2024-09-16T00:26:13.636 INFO:tasks.workunit.client.1.smithi169.stdout:7/803: fsync d7/d71/d76/d29/fac 0 2024-09-16T00:26:13.638 INFO:tasks.workunit.client.1.smithi169.stdout:2/703: write d7/db/d89/db4/d9d/dbe/fdf [455423,89790] 0 2024-09-16T00:26:13.638 INFO:tasks.workunit.client.1.smithi169.stdout:0/785: dread d0/d2/d15/fc1 [0,4194304] 0 2024-09-16T00:26:13.640 INFO:tasks.workunit.client.1.smithi169.stdout:7/804: chown d7/d71/d76/d29/lf5 87783 1 2024-09-16T00:26:13.641 INFO:tasks.workunit.client.1.smithi169.stdout:1/771: dwrite dc/d36/d3a/d45/d29/f52 [0,4194304] 0 2024-09-16T00:26:13.642 INFO:tasks.workunit.client.1.smithi169.stdout:7/805: readlink d7/d71/d76/d9c/leb 0 2024-09-16T00:26:13.643 INFO:tasks.workunit.client.1.smithi169.stdout:1/772: dread - dc/d36/d39/d50/fcb zero size 2024-09-16T00:26:13.644 INFO:tasks.workunit.client.0.smithi112.stdout:8/973: write d10/d12/d1c/d61/d6c/dd1/fe5 [819773,54206] 0 2024-09-16T00:26:13.646 INFO:tasks.workunit.client.0.smithi112.stdout:8/974: fdatasync d10/d12/d1c/d61/d6c/f93 0 2024-09-16T00:26:13.647 INFO:tasks.workunit.client.0.smithi112.stdout:8/975: stat d10/d12/d1c/d3b/d74/f7b 0 2024-09-16T00:26:13.650 INFO:tasks.workunit.client.0.smithi112.stdout:8/976: symlink d10/d12/d1c/dd0/dbb/d107/d134/l14e 0 2024-09-16T00:26:13.651 INFO:tasks.workunit.client.0.smithi112.stdout:8/977: fdatasync d10/d12/d1c/d61/d6c/dd1/d8c/f9a 0 2024-09-16T00:26:13.656 INFO:tasks.workunit.client.0.smithi112.stdout:8/978: dwrite d10/d12/d1c/d3b/d7d/de4/fed [0,4194304] 0 2024-09-16T00:26:13.659 INFO:tasks.workunit.client.0.smithi112.stdout:8/979: mknod d10/d12/d1c/dd0/dbb/c14f 0 2024-09-16T00:26:13.660 INFO:tasks.workunit.client.0.smithi112.stdout:8/980: dread - d10/de9/def/dff/f11e zero size 2024-09-16T00:26:13.662 INFO:tasks.workunit.client.0.smithi112.stdout:8/981: creat d10/d12/d1c/d3b/f150 x:0 0 0 2024-09-16T00:26:13.663 INFO:tasks.workunit.client.1.smithi169.stdout:3/921: creat d7/d50/d6f/f147 x:0 0 0 2024-09-16T00:26:13.663 INFO:tasks.workunit.client.1.smithi169.stdout:3/922: stat d7/d1a/l3e 0 2024-09-16T00:26:13.665 INFO:tasks.workunit.client.1.smithi169.stdout:3/923: write d7/d50/d6f/fb4 [4048929,94403] 0 2024-09-16T00:26:13.667 INFO:tasks.workunit.client.0.smithi112.stdout:8/982: link d10/d12/d1c/d61/d6c/dd1/d8c/fd8 d10/d12/d1c/d61/d6c/dd1/d8c/f151 0 2024-09-16T00:26:13.668 INFO:tasks.workunit.client.1.smithi169.stdout:2/704: symlink d7/db/d27/d29/d37/lfe 0 2024-09-16T00:26:13.669 INFO:tasks.workunit.client.1.smithi169.stdout:0/786: creat d0/d2c/d104/d67/dd1/f10e x:0 0 0 2024-09-16T00:26:13.669 INFO:tasks.workunit.client.1.smithi169.stdout:7/806: creat d7/d71/d76/d9c/ddf/f106 x:0 0 0 2024-09-16T00:26:13.669 INFO:tasks.workunit.client.0.smithi112.stdout:8/983: symlink d10/d12/d1c/d61/d6c/dd1/d8c/d7c/d97/l152 0 2024-09-16T00:26:13.670 INFO:tasks.workunit.client.0.smithi112.stdout:8/984: stat d10/d12/d1c/d3b/d7d/ce2 0 2024-09-16T00:26:13.670 INFO:tasks.workunit.client.1.smithi169.stdout:0/787: write d0/d2c/d104/d67/dd1/f10e [226171,89220] 0 2024-09-16T00:26:13.670 INFO:tasks.workunit.client.1.smithi169.stdout:7/807: dread - d7/d5b/d61/fe3 zero size 2024-09-16T00:26:13.671 INFO:tasks.workunit.client.1.smithi169.stdout:0/788: chown d0/d2/d8/d5/d60/ddb 0 1 2024-09-16T00:26:13.673 INFO:tasks.workunit.client.1.smithi169.stdout:4/723: symlink d2/d12/d13/d1f/d3e/dde/le3 0 2024-09-16T00:26:13.675 INFO:tasks.workunit.client.1.smithi169.stdout:4/724: stat d2/d12/d13/d1f/d2f/f43 0 2024-09-16T00:26:13.676 INFO:tasks.workunit.client.1.smithi169.stdout:1/773: mkdir dc/d36/d3a/d7f/ddb/dea 0 2024-09-16T00:26:13.678 INFO:tasks.workunit.client.1.smithi169.stdout:5/654: mknod d2/da/d10/d13/d1c/d32/d53/d83/ce6 0 2024-09-16T00:26:13.681 INFO:tasks.workunit.client.0.smithi112.stdout:8/985: dwrite d10/d12/d1c/d61/d6c/dd1/d8c/fd8 [0,4194304] 0 2024-09-16T00:26:13.682 INFO:tasks.workunit.client.1.smithi169.stdout:7/808: dwrite d7/d5b/d61/d98/ffe [0,4194304] 0 2024-09-16T00:26:13.682 INFO:tasks.workunit.client.1.smithi169.stdout:2/705: mkdir d7/db/d27/dff 0 2024-09-16T00:26:13.683 INFO:tasks.workunit.client.0.smithi112.stdout:8/986: mknod d10/de9/def/dff/c153 0 2024-09-16T00:26:13.685 INFO:tasks.workunit.client.0.smithi112.stdout:8/987: symlink d10/l154 0 2024-09-16T00:26:13.686 INFO:tasks.workunit.client.1.smithi169.stdout:5/655: mkdir d2/da/d10/d13/d1c/d2d/db2/de7 0 2024-09-16T00:26:13.686 INFO:tasks.workunit.client.0.smithi112.stdout:8/988: creat d10/d12/d1c/d3b/f155 x:0 0 0 2024-09-16T00:26:13.687 INFO:tasks.workunit.client.1.smithi169.stdout:7/809: creat d7/d16/f107 x:0 0 0 2024-09-16T00:26:13.689 INFO:tasks.workunit.client.1.smithi169.stdout:6/957: sync 2024-09-16T00:26:13.689 INFO:tasks.workunit.client.1.smithi169.stdout:2/706: mkdir d7/d6b/d70/d79/dae/d100 0 2024-09-16T00:26:13.689 INFO:tasks.workunit.client.0.smithi112.stdout:8/989: mknod d10/c156 0 2024-09-16T00:26:13.690 INFO:tasks.workunit.client.1.smithi169.stdout:1/774: mknod dc/d18/dd6/ceb 0 2024-09-16T00:26:13.695 INFO:tasks.workunit.client.1.smithi169.stdout:1/775: write dc/d18/d67/fc8 [1138849,63002] 0 2024-09-16T00:26:13.695 INFO:tasks.workunit.client.1.smithi169.stdout:5/656: creat d2/da/d10/d13/d1c/fe8 x:0 0 0 2024-09-16T00:26:13.696 INFO:tasks.workunit.client.0.smithi112.stdout:8/990: dread d10/d12/d1c/d61/d6c/dd1/d58/f9b [0,4194304] 0 2024-09-16T00:26:13.697 INFO:tasks.workunit.client.1.smithi169.stdout:6/958: unlink d2/da4/db1/cca 0 2024-09-16T00:26:13.698 INFO:tasks.workunit.client.0.smithi112.stdout:8/991: readlink d10/d12/d1c/d61/d6c/dd1/d8c/d7c/d97/l131 0 2024-09-16T00:26:13.699 INFO:tasks.workunit.client.0.smithi112.stdout:8/992: read d10/d12/d1c/f59 [6866474,81276] 0 2024-09-16T00:26:13.701 INFO:tasks.workunit.client.0.smithi112.stdout:8/993: dread - d10/d12/d1c/d3b/f53 zero size 2024-09-16T00:26:13.703 INFO:tasks.workunit.client.0.smithi112.stdout:8/994: creat d10/d12/d1c/dd0/dbb/d107/f157 x:0 0 0 2024-09-16T00:26:13.703 INFO:tasks.workunit.client.1.smithi169.stdout:2/707: dwrite d7/daa/fe3 [0,4194304] 0 2024-09-16T00:26:13.707 INFO:tasks.workunit.client.1.smithi169.stdout:6/959: mknod d2/d4c/d12e/c130 0 2024-09-16T00:26:13.707 INFO:tasks.workunit.client.1.smithi169.stdout:5/657: dread d2/da/d10/d13/d1c/d32/d53/fb0 [0,4194304] 0 2024-09-16T00:26:13.708 INFO:tasks.workunit.client.1.smithi169.stdout:7/810: dwrite d7/d5b/d61/d98/d99/df9/dc6/ffa [0,4194304] 0 2024-09-16T00:26:13.709 INFO:tasks.workunit.client.1.smithi169.stdout:9/901: write d5/da/d5b/fcc [334335,18904] 0 2024-09-16T00:26:13.709 INFO:tasks.workunit.client.1.smithi169.stdout:1/776: dread dc/d26/fa0 [0,4194304] 0 2024-09-16T00:26:13.713 INFO:tasks.workunit.client.1.smithi169.stdout:1/777: write dc/d36/d3a/d7f/fd9 [575468,65874] 0 2024-09-16T00:26:13.715 INFO:tasks.workunit.client.1.smithi169.stdout:7/811: symlink d7/d71/l108 0 2024-09-16T00:26:13.715 INFO:tasks.workunit.client.1.smithi169.stdout:9/902: write d5/da/d2e/d47/f119 [721043,95733] 0 2024-09-16T00:26:13.716 INFO:tasks.workunit.client.1.smithi169.stdout:0/789: rename d0/d2c/d104/d67/dd1 to d0/d4c/d82/d10f 0 2024-09-16T00:26:13.717 INFO:tasks.workunit.client.0.smithi112.stdout:8/995: dwrite d10/d12/d1c/d96/fd6 [0,4194304] 0 2024-09-16T00:26:13.719 INFO:tasks.workunit.client.1.smithi169.stdout:5/658: read d2/da/d10/d13/d1c/f6a [679468,92643] 0 2024-09-16T00:26:13.720 INFO:tasks.workunit.client.1.smithi169.stdout:4/725: dread d2/d12/d13/d4e/f77 [0,4194304] 0 2024-09-16T00:26:13.723 INFO:tasks.workunit.client.0.smithi112.stdout:8/996: dwrite d10/d12/d1c/d61/d6c/dd1/fc9 [0,4194304] 0 2024-09-16T00:26:13.724 INFO:tasks.workunit.client.0.smithi112.stdout:8/997: chown d10/d12/d1c/d13b 6 1 2024-09-16T00:26:13.726 INFO:tasks.workunit.client.1.smithi169.stdout:7/812: symlink d7/d5b/d61/d98/d99/df9/dc6/l109 0 2024-09-16T00:26:13.728 INFO:tasks.workunit.client.1.smithi169.stdout:6/960: dwrite d2/d36/d48/f90 [0,4194304] 0 2024-09-16T00:26:13.733 INFO:tasks.workunit.client.1.smithi169.stdout:9/903: fsync d5/f35 0 2024-09-16T00:26:13.738 INFO:tasks.workunit.client.1.smithi169.stdout:3/924: truncate d7/d1a/d1c/d23/d88/ffa 1535065 0 2024-09-16T00:26:13.739 INFO:tasks.workunit.client.1.smithi169.stdout:1/778: dwrite dc/f66 [0,4194304] 0 2024-09-16T00:26:13.743 INFO:tasks.workunit.client.1.smithi169.stdout:5/659: fsync d2/da/d10/d13/d1c/d40/d54/d7f/fd9 0 2024-09-16T00:26:13.743 INFO:tasks.workunit.client.0.smithi112.stdout:8/998: dwrite d10/d12/d1c/fe1 [0,4194304] 0 2024-09-16T00:26:13.746 INFO:tasks.workunit.client.0.smithi112.stdout:8/999: mkdir d10/d12/d1c/d61/d6c/ddc/d158 0 2024-09-16T00:26:13.746 INFO:tasks.workunit.client.1.smithi169.stdout:0/790: rename d0/d2/d15/dc5 to d0/d2c/d110 0 2024-09-16T00:26:13.747 INFO:tasks.workunit.client.1.smithi169.stdout:8/851: dwrite d7/dc/d2a/f30 [0,4194304] 0 2024-09-16T00:26:13.750 INFO:tasks.workunit.client.1.smithi169.stdout:6/961: truncate d2/fb 8232668 0 2024-09-16T00:26:13.751 INFO:tasks.workunit.client.0.smithi112.stderr:+ rm -rf -- ./tmp.ZUfysPtAcL 2024-09-16T00:26:13.752 INFO:tasks.workunit.client.1.smithi169.stdout:1/779: write dc/d36/d3a/d45/f23 [6477155,30924] 0 2024-09-16T00:26:13.756 INFO:tasks.workunit.client.1.smithi169.stdout:4/726: dread d2/d12/d13/d4e/f77 [0,4194304] 0 2024-09-16T00:26:13.759 INFO:tasks.workunit.client.1.smithi169.stdout:2/708: dwrite d7/db/d20/d21/f4a [0,4194304] 0 2024-09-16T00:26:13.760 INFO:tasks.workunit.client.1.smithi169.stdout:0/791: creat d0/f111 x:0 0 0 2024-09-16T00:26:13.760 INFO:tasks.workunit.client.1.smithi169.stdout:6/962: read d2/d3f/d95/fb2 [1818977,96646] 0 2024-09-16T00:26:13.760 INFO:tasks.workunit.client.1.smithi169.stdout:7/813: rmdir d7/d5b/d61/d98/d99 39 2024-09-16T00:26:13.763 INFO:tasks.workunit.client.1.smithi169.stdout:5/660: rename d2/da/d10/d13/d1c/d40/d54/d9c to d2/da/d10/d13/d24/de9 0 2024-09-16T00:26:13.769 INFO:tasks.workunit.client.1.smithi169.stdout:4/727: truncate d2/d12/d13/d5c/da6/fb6 763059 0 2024-09-16T00:26:13.769 INFO:tasks.workunit.client.1.smithi169.stdout:2/709: creat d7/db/d89/f101 x:0 0 0 2024-09-16T00:26:13.770 INFO:tasks.workunit.client.1.smithi169.stdout:8/852: mkdir d7/dc/d26/d41/d92/dcf/ddc/d11a 0 2024-09-16T00:26:13.771 INFO:tasks.workunit.client.1.smithi169.stdout:1/780: rename dc/d36/d39/d50/daf/fe8 to dc/d6c/fec 0 2024-09-16T00:26:13.772 INFO:tasks.workunit.client.1.smithi169.stdout:5/661: rmdir d2/da/d10/d13/d1c/d32/d53/d83/d7d/d96 39 2024-09-16T00:26:13.772 INFO:tasks.workunit.client.1.smithi169.stdout:0/792: write d0/d4c/d6f/f91 [5088290,1232] 0 2024-09-16T00:26:13.775 INFO:tasks.workunit.client.1.smithi169.stdout:8/853: mknod d7/dc/d26/d41/d92/dcf/df0/c11b 0 2024-09-16T00:26:13.775 INFO:tasks.workunit.client.1.smithi169.stdout:1/781: readlink dc/d26/l58 0 2024-09-16T00:26:13.776 INFO:tasks.workunit.client.1.smithi169.stdout:2/710: creat d7/db/d89/f102 x:0 0 0 2024-09-16T00:26:13.779 INFO:tasks.workunit.client.1.smithi169.stdout:5/662: write d2/da/d10/d13/d1c/d32/d53/d4e/fa8 [575539,78272] 0 2024-09-16T00:26:13.779 INFO:tasks.workunit.client.1.smithi169.stdout:4/728: creat d2/d12/d13/d1f/fe4 x:0 0 0 2024-09-16T00:26:13.782 INFO:tasks.workunit.client.1.smithi169.stdout:1/782: creat dc/d36/d39/d50/daf/d88/da5/fed x:0 0 0 2024-09-16T00:26:13.785 INFO:tasks.workunit.client.1.smithi169.stdout:2/711: chown d7/db/d27/d29/d77/la9 606 1 2024-09-16T00:26:13.788 INFO:tasks.workunit.client.1.smithi169.stdout:0/793: dwrite d0/d2/ff5 [0,4194304] 0 2024-09-16T00:26:13.788 INFO:tasks.workunit.client.1.smithi169.stdout:8/854: symlink d7/dc/d26/d41/d92/dcf/d3c/d77/l11c 0 2024-09-16T00:26:13.789 INFO:tasks.workunit.client.1.smithi169.stdout:9/904: dread d5/da/d21/de4/f4c [0,4194304] 0 2024-09-16T00:26:13.790 INFO:tasks.workunit.client.1.smithi169.stdout:3/925: write d7/d1a/d1c/f73 [4888195,21000] 0 2024-09-16T00:26:13.794 INFO:tasks.workunit.client.1.smithi169.stdout:5/663: mknod d2/da/d10/d13/d1c/d32/d53/d4e/d89/d9a/dad/cea 0 2024-09-16T00:26:13.794 INFO:tasks.workunit.client.1.smithi169.stdout:4/729: symlink d2/d12/d9c/dc2/le5 0 2024-09-16T00:26:13.795 INFO:tasks.workunit.client.1.smithi169.stdout:2/712: mknod d7/db/d89/db4/d9d/dbe/de4/c103 0 2024-09-16T00:26:13.796 INFO:tasks.workunit.client.1.smithi169.stdout:4/730: fdatasync d2/d12/d13/d5c/f94 0 2024-09-16T00:26:13.798 INFO:tasks.workunit.client.1.smithi169.stdout:8/855: creat d7/dc/d26/d41/d4e/d81/f11d x:0 0 0 2024-09-16T00:26:13.799 INFO:tasks.workunit.client.1.smithi169.stdout:5/664: write d2/da/d10/d13/d1c/d32/d53/d4e/d89/d9a/fa5 [283528,64425] 0 2024-09-16T00:26:13.800 INFO:tasks.workunit.client.1.smithi169.stdout:3/926: creat d7/d50/d90/dba/df0/f148 x:0 0 0 2024-09-16T00:26:13.801 INFO:tasks.workunit.client.1.smithi169.stdout:2/713: read d7/f55 [4023764,77574] 0 2024-09-16T00:26:13.801 INFO:tasks.workunit.client.1.smithi169.stdout:3/927: stat d7/d50/d90 0 2024-09-16T00:26:13.805 INFO:tasks.workunit.client.1.smithi169.stdout:7/814: write d7/d71/d76/fc [3236963,107782] 0 2024-09-16T00:26:13.805 INFO:tasks.workunit.client.1.smithi169.stdout:9/905: mknod d5/da/d2e/d13a/c142 0 2024-09-16T00:26:13.809 INFO:tasks.workunit.client.1.smithi169.stdout:9/906: stat d5/da/dd0/c126 0 2024-09-16T00:26:13.809 INFO:tasks.workunit.client.1.smithi169.stdout:2/714: dread - d7/db/db8/d6d/fcc zero size 2024-09-16T00:26:13.813 INFO:tasks.workunit.client.1.smithi169.stdout:4/731: dwrite d2/d12/d13/d4e/d50/db8/fda [0,4194304] 0 2024-09-16T00:26:13.816 INFO:tasks.workunit.client.1.smithi169.stdout:0/794: link d0/d2c/d38/de4/d21/d31/d83/dac/ff0 d0/d4c/d82/d10f/d6c/f112 0 2024-09-16T00:26:13.817 INFO:tasks.workunit.client.1.smithi169.stdout:6/963: dwrite d2/d3f/f85 [0,4194304] 0 2024-09-16T00:26:13.817 INFO:tasks.workunit.client.1.smithi169.stdout:1/783: dread dc/d36/d39/d50/daf/d86/f51 [0,4194304] 0 2024-09-16T00:26:13.818 INFO:tasks.workunit.client.1.smithi169.stdout:3/928: truncate d7/f10 990276 0 2024-09-16T00:26:13.819 INFO:tasks.workunit.client.1.smithi169.stdout:0/795: read d0/d4c/d82/ffd [3229017,113199] 0 2024-09-16T00:26:13.821 INFO:tasks.workunit.client.1.smithi169.stdout:5/665: mknod d2/da/d10/d13/d1c/d2d/ceb 0 2024-09-16T00:26:13.822 INFO:tasks.workunit.client.1.smithi169.stdout:9/907: mknod d5/da/d2e/d99/d100/d10f/c143 0 2024-09-16T00:26:13.825 INFO:tasks.workunit.client.1.smithi169.stdout:4/732: creat d2/dbb/d59/fe6 x:0 0 0 2024-09-16T00:26:13.826 INFO:tasks.workunit.client.1.smithi169.stdout:8/856: write d7/d37/f9b [380811,50170] 0 2024-09-16T00:26:13.826 INFO:tasks.workunit.client.1.smithi169.stdout:1/784: rmdir dc/d2c 39 2024-09-16T00:26:13.827 INFO:tasks.workunit.client.1.smithi169.stdout:2/715: dwrite d7/d1c/f95 [0,4194304] 0 2024-09-16T00:26:13.828 INFO:tasks.workunit.client.1.smithi169.stdout:1/785: chown dc/d36/l5f 1066 1 2024-09-16T00:26:13.829 INFO:tasks.workunit.client.1.smithi169.stdout:6/964: symlink d2/d3f/d95/l131 0 2024-09-16T00:26:13.830 INFO:tasks.workunit.client.1.smithi169.stdout:3/929: mkdir d7/d50/d90/d13a/d8d/d149 0 2024-09-16T00:26:13.836 INFO:tasks.workunit.client.1.smithi169.stdout:7/815: creat d7/f10a x:0 0 0 2024-09-16T00:26:13.841 INFO:tasks.workunit.client.1.smithi169.stdout:4/733: mkdir d2/d12/d13/d1f/d2f/de7 0 2024-09-16T00:26:13.841 INFO:tasks.workunit.client.1.smithi169.stdout:5/666: rename d2/da/d10/d13/d1c/d71/lc2 to d2/da/d67/lec 0 2024-09-16T00:26:13.841 INFO:tasks.workunit.client.1.smithi169.stdout:8/857: dread d7/dc/d17/f66 [0,4194304] 0 2024-09-16T00:26:13.842 INFO:tasks.workunit.client.1.smithi169.stdout:6/965: chown d2/d36/l7a 90 1 2024-09-16T00:26:13.843 INFO:tasks.workunit.client.1.smithi169.stdout:9/908: getdents d5/de1 0 2024-09-16T00:26:13.844 INFO:tasks.workunit.client.1.smithi169.stdout:1/786: write dc/d2c/f32 [1051605,59044] 0 2024-09-16T00:26:13.847 INFO:tasks.workunit.client.1.smithi169.stdout:6/966: readlink d2/d4d/d59/d43/l8c 0 2024-09-16T00:26:13.847 INFO:tasks.workunit.client.1.smithi169.stdout:3/930: dwrite d7/d50/d90/d13a/f104 [0,4194304] 0 2024-09-16T00:26:13.850 INFO:tasks.workunit.client.1.smithi169.stdout:5/667: creat d2/da/d10/d13/d1c/d2d/fed x:0 0 0 2024-09-16T00:26:13.850 INFO:tasks.workunit.client.1.smithi169.stdout:3/931: readlink d7/d1a/d1c/d35/d63/d37/l136 0 2024-09-16T00:26:13.850 INFO:tasks.workunit.client.1.smithi169.stdout:6/967: read - d2/d3f/d41/d5c/d73/f116 zero size 2024-09-16T00:26:13.854 INFO:tasks.workunit.client.1.smithi169.stdout:3/932: stat d7/d1a/d1c/d35/d63/dc2/dc7/d12c 0 2024-09-16T00:26:13.854 INFO:tasks.workunit.client.1.smithi169.stdout:8/858: symlink d7/dc/d17/l11e 0 2024-09-16T00:26:13.856 INFO:tasks.workunit.client.1.smithi169.stdout:4/734: truncate d2/d12/d13/d4e/d50/f79 2525804 0 2024-09-16T00:26:13.857 INFO:tasks.workunit.client.1.smithi169.stdout:0/796: rmdir d0/d2c/d38 39 2024-09-16T00:26:13.859 INFO:tasks.workunit.client.1.smithi169.stdout:1/787: mkdir dc/d36/d3a/d45/d29/dc2/dee 0 2024-09-16T00:26:13.864 INFO:tasks.workunit.client.1.smithi169.stdout:4/735: dread d2/f98 [0,4194304] 0 2024-09-16T00:26:13.866 INFO:tasks.workunit.client.1.smithi169.stdout:6/968: mknod d2/d3f/c132 0 2024-09-16T00:26:13.866 INFO:tasks.workunit.client.1.smithi169.stdout:4/736: write d2/d12/d13/d1f/fe4 [171826,107572] 0 2024-09-16T00:26:13.868 INFO:tasks.workunit.client.1.smithi169.stdout:1/788: creat dc/d6c/d96/fef x:0 0 0 2024-09-16T00:26:13.870 INFO:tasks.workunit.client.1.smithi169.stdout:0/797: mkdir d0/d2/d8/d5/d79/d113 0 2024-09-16T00:26:13.872 INFO:tasks.workunit.client.1.smithi169.stdout:2/716: truncate d7/daa/fe3 3704948 0 2024-09-16T00:26:13.874 INFO:tasks.workunit.client.1.smithi169.stdout:5/668: creat d2/da/d10/d13/d1c/d32/d53/d4e/fee x:0 0 0 2024-09-16T00:26:13.877 INFO:tasks.workunit.client.1.smithi169.stdout:7/816: dwrite d7/d16/f90 [0,4194304] 0 2024-09-16T00:26:13.892 INFO:tasks.workunit.client.1.smithi169.stdout:1/789: dread dc/d26/f55 [0,4194304] 0 2024-09-16T00:26:13.900 INFO:tasks.workunit.client.1.smithi169.stdout:4/737: creat d2/d12/d13/d27/fe8 x:0 0 0 2024-09-16T00:26:13.929 INFO:tasks.workunit.client.1.smithi169.stdout:9/909: write d5/da/d73/fda [232969,16913] 0 2024-09-16T00:26:13.932 INFO:tasks.workunit.client.1.smithi169.stdout:9/910: write d5/da/d5b/d6c/da0/f13e [339757,59353] 0 2024-09-16T00:26:13.937 INFO:tasks.workunit.client.1.smithi169.stdout:2/717: creat d7/d6b/d70/de0/f104 x:0 0 0 2024-09-16T00:26:13.941 INFO:tasks.workunit.client.1.smithi169.stdout:3/933: write d7/d1a/d1c/d35/fac [939596,86654] 0 2024-09-16T00:26:13.944 INFO:tasks.workunit.client.1.smithi169.stdout:6/969: creat d2/d3f/d41/f133 x:0 0 0 2024-09-16T00:26:13.946 INFO:tasks.workunit.client.1.smithi169.stdout:8/859: dwrite f6 [0,4194304] 0 2024-09-16T00:26:13.950 INFO:tasks.workunit.client.1.smithi169.stdout:2/718: mkdir d7/db/d20/d21/d105 0 2024-09-16T00:26:13.952 INFO:tasks.workunit.client.1.smithi169.stdout:5/669: symlink d2/da/d10/d13/d1c/d9e/lef 0 2024-09-16T00:26:13.954 INFO:tasks.workunit.client.1.smithi169.stdout:5/670: dread d2/f58 [0,4194304] 0 2024-09-16T00:26:13.958 INFO:tasks.workunit.client.1.smithi169.stdout:3/934: read - d7/d1a/d1c/d6c/fdf zero size 2024-09-16T00:26:13.964 INFO:tasks.workunit.client.1.smithi169.stdout:6/970: creat d2/d3f/d41/d72/d9e/de1/f134 x:0 0 0 2024-09-16T00:26:13.967 INFO:tasks.workunit.client.1.smithi169.stdout:8/860: dread d7/fa [0,4194304] 0 2024-09-16T00:26:13.970 INFO:tasks.workunit.client.1.smithi169.stdout:8/861: truncate d7/dc/d17/d48/ff9 197630 0 2024-09-16T00:26:13.970 INFO:tasks.workunit.client.1.smithi169.stdout:6/971: dread - d2/fdb zero size 2024-09-16T00:26:13.972 INFO:tasks.workunit.client.1.smithi169.stdout:8/862: readlink d7/dc/d26/d41/d4e/l54 0 2024-09-16T00:26:13.972 INFO:tasks.workunit.client.1.smithi169.stdout:6/972: write d2/d3f/d41/d5c/d73/f116 [616962,54889] 0 2024-09-16T00:26:13.973 INFO:tasks.workunit.client.1.smithi169.stdout:8/863: fsync d7/dc/d26/fea 0 2024-09-16T00:26:13.976 INFO:tasks.workunit.client.1.smithi169.stdout:3/935: link d7/d1a/d1c/d35/d63/da3/f116 d7/d1a/d1c/d35/d63/da3/f14a 0 2024-09-16T00:26:13.982 INFO:tasks.workunit.client.1.smithi169.stdout:5/671: dread d2/da/d10/d13/d1c/d32/d53/d4e/d89/d9a/dad/fd8 [0,4194304] 0 2024-09-16T00:26:13.984 INFO:tasks.workunit.client.1.smithi169.stdout:8/864: link d7/dc/d26/d41/d4e/l7a d7/dc/d26/d41/d92/dcf/d80/dbd/l11f 0 2024-09-16T00:26:13.985 INFO:tasks.workunit.client.1.smithi169.stdout:8/865: chown d7/dc/d17/d24/d32/d105 33011833 1 2024-09-16T00:26:13.988 INFO:tasks.workunit.client.1.smithi169.stdout:3/936: getdents d7/d50/d90/d13a 0 2024-09-16T00:26:13.988 INFO:tasks.workunit.client.1.smithi169.stdout:5/672: symlink d2/da/d10/d13/d1c/d40/d54/d7f/lf0 0 2024-09-16T00:26:13.994 INFO:tasks.workunit.client.1.smithi169.stdout:8/866: dwrite d7/dc/d17/d48/ff9 [0,4194304] 0 2024-09-16T00:26:13.997 INFO:tasks.workunit.client.1.smithi169.stdout:8/867: fdatasync d7/dc/d26/d41/d92/dcf/d3c/d82/db3/f107 0 2024-09-16T00:26:14.000 INFO:tasks.workunit.client.1.smithi169.stdout:5/673: link d2/da/d10/d13/d1c/d40/d54/c65 d2/da/d10/d13/d24/de9/cf1 0 2024-09-16T00:26:14.006 INFO:tasks.workunit.client.1.smithi169.stdout:6/973: symlink d2/d4d/l135 0 2024-09-16T00:26:14.011 INFO:tasks.workunit.client.1.smithi169.stdout:6/974: creat d2/d3f/d41/d52/d121/f136 x:0 0 0 2024-09-16T00:26:14.016 INFO:tasks.workunit.client.1.smithi169.stdout:6/975: truncate d2/d3f/d41/d52/f89 1053183 0 2024-09-16T00:26:14.016 INFO:tasks.workunit.client.1.smithi169.stdout:3/937: dread d7/d1a/d1c/f54 [0,4194304] 0 2024-09-16T00:26:14.016 INFO:tasks.workunit.client.1.smithi169.stdout:3/938: chown d7/d1a/d1c/d23/d88/d9b/f119 4193 1 2024-09-16T00:26:14.017 INFO:tasks.workunit.client.1.smithi169.stdout:6/976: symlink d2/d4d/l137 0 2024-09-16T00:26:14.024 INFO:tasks.workunit.client.1.smithi169.stdout:6/977: dwrite d2/d4d/d59/d43/f114 [0,4194304] 0 2024-09-16T00:26:14.028 INFO:tasks.workunit.client.1.smithi169.stdout:6/978: mkdir d2/da4/db1/dbc/d138 0 2024-09-16T00:26:14.029 INFO:tasks.workunit.client.1.smithi169.stdout:3/939: symlink d7/d1a/d1c/d23/d88/d9b/dc8/l14b 0 2024-09-16T00:26:14.031 INFO:tasks.workunit.client.1.smithi169.stdout:6/979: chown d2/d36/d115/c12d 1403 1 2024-09-16T00:26:14.037 INFO:tasks.workunit.client.1.smithi169.stdout:6/980: getdents d2/d3f/d95 0 2024-09-16T00:26:14.047 INFO:tasks.workunit.client.1.smithi169.stdout:7/817: dwrite d7/d71/d76/d29/d78/dba/fcd [0,4194304] 0 2024-09-16T00:26:14.050 INFO:tasks.workunit.client.1.smithi169.stdout:1/790: truncate dc/d36/d3a/d45/d29/f5c 1199071 0 2024-09-16T00:26:14.053 INFO:tasks.workunit.client.1.smithi169.stdout:1/791: symlink dc/d18/dd6/lf0 0 2024-09-16T00:26:14.057 INFO:tasks.workunit.client.1.smithi169.stdout:9/911: dwrite d5/da/d5b/d6c/ffd [0,4194304] 0 2024-09-16T00:26:14.058 INFO:tasks.workunit.client.1.smithi169.stdout:2/719: write d7/d1c/d26/d96/f8e [1835324,89105] 0 2024-09-16T00:26:14.059 INFO:tasks.workunit.client.1.smithi169.stdout:9/912: write d5/da/d5b/d91/db3/f101 [2258007,105721] 0 2024-09-16T00:26:14.063 INFO:tasks.workunit.client.1.smithi169.stdout:9/913: symlink d5/da/df3/df8/l144 0 2024-09-16T00:26:14.064 INFO:tasks.workunit.client.1.smithi169.stdout:9/914: chown d5/da/d21/f2c 7516675 1 2024-09-16T00:26:14.066 INFO:tasks.workunit.client.1.smithi169.stdout:9/915: mknod d5/de0/c145 0 2024-09-16T00:26:14.067 INFO:tasks.workunit.client.1.smithi169.stdout:9/916: readlink d5/da/d2e/ld1 0 2024-09-16T00:26:14.068 INFO:tasks.workunit.client.1.smithi169.stdout:8/868: truncate d7/dc/d26/d41/d92/dcf/d80/dbd/fe2 232133 0 2024-09-16T00:26:14.070 INFO:tasks.workunit.client.1.smithi169.stdout:5/674: write d2/da/d10/d13/d1c/d32/d53/fb0 [945958,40366] 0 2024-09-16T00:26:14.071 INFO:tasks.workunit.client.1.smithi169.stdout:8/869: chown d7/dc/d17/d24/f4b 4083995 1 2024-09-16T00:26:14.071 INFO:tasks.workunit.client.1.smithi169.stdout:5/675: read - d2/da/fcd zero size 2024-09-16T00:26:14.073 INFO:tasks.workunit.client.1.smithi169.stdout:8/870: chown d7/dc/d26/d41/d4e/d81/f11d 0 1 2024-09-16T00:26:14.074 INFO:tasks.workunit.client.1.smithi169.stdout:5/676: creat d2/da/d10/d13/dc0/ff2 x:0 0 0 2024-09-16T00:26:14.075 INFO:tasks.workunit.client.1.smithi169.stdout:4/738: rmdir d2/d12/d13/d1f/d2f 39 2024-09-16T00:26:14.076 INFO:tasks.workunit.client.1.smithi169.stdout:3/940: rename d7/d1a/d1c/d23/c3f to d7/d50/d90/c14c 0 2024-09-16T00:26:14.081 INFO:tasks.workunit.client.1.smithi169.stdout:4/739: unlink d2/d12/d9c/da4/cbd 0 2024-09-16T00:26:14.083 INFO:tasks.workunit.client.1.smithi169.stdout:4/740: write d2/d12/d13/f60 [1767379,67337] 0 2024-09-16T00:26:14.084 INFO:tasks.workunit.client.1.smithi169.stdout:3/941: dread d7/d50/d90/dba/fd0 [0,4194304] 0 2024-09-16T00:26:14.085 INFO:tasks.workunit.client.1.smithi169.stdout:4/741: truncate d2/d12/d13/f25 5191805 0 2024-09-16T00:26:14.086 INFO:tasks.workunit.client.1.smithi169.stdout:1/792: rename dc/d26/l8b to dc/d36/d3a/d45/lf1 0 2024-09-16T00:26:14.088 INFO:tasks.workunit.client.1.smithi169.stdout:3/942: creat d7/d1a/d1c/d23/d88/d9b/de2/f14d x:0 0 0 2024-09-16T00:26:14.089 INFO:tasks.workunit.client.1.smithi169.stdout:4/742: fdatasync d2/d12/d13/d5c/f66 0 2024-09-16T00:26:14.090 INFO:tasks.workunit.client.1.smithi169.stdout:3/943: stat d7/d1a/d1c/d23/d70/f8e 0 2024-09-16T00:26:14.090 INFO:tasks.workunit.client.1.smithi169.stdout:3/944: stat d7/f68 0 2024-09-16T00:26:14.091 INFO:tasks.workunit.client.1.smithi169.stdout:8/871: dread d7/dc/d2a/f42 [0,4194304] 0 2024-09-16T00:26:14.092 INFO:tasks.workunit.client.1.smithi169.stdout:8/872: dread - d7/dc/d17/d24/d32/f101 zero size 2024-09-16T00:26:14.095 INFO:tasks.workunit.client.1.smithi169.stdout:3/945: creat d7/d1a/d1c/d35/d63/da3/f14e x:0 0 0 2024-09-16T00:26:14.099 INFO:tasks.workunit.client.1.smithi169.stdout:7/818: write d7/d5b/f9a [1964890,130652] 0 2024-09-16T00:26:14.101 INFO:tasks.workunit.client.1.smithi169.stdout:8/873: dwrite d7/dc/d17/d24/d32/d8d/fc8 [0,4194304] 0 2024-09-16T00:26:14.102 INFO:tasks.workunit.client.1.smithi169.stdout:6/981: dwrite d2/d3f/d41/d5c/f65 [0,4194304] 0 2024-09-16T00:26:14.104 INFO:tasks.workunit.client.1.smithi169.stdout:2/720: rename d7/db/d89/f101 to d7/db/d20/d21/d105/f106 0 2024-09-16T00:26:14.107 INFO:tasks.workunit.client.1.smithi169.stdout:7/819: chown d7/d5b/d61/d98/d99/df9/f8a 488 1 2024-09-16T00:26:14.110 INFO:tasks.workunit.client.1.smithi169.stdout:8/874: creat d7/dc/d2a/f120 x:0 0 0 2024-09-16T00:26:14.113 INFO:tasks.workunit.client.1.smithi169.stdout:8/875: dread - d7/dc/d17/d24/d32/fda zero size 2024-09-16T00:26:14.115 INFO:tasks.workunit.client.1.smithi169.stdout:9/917: rename d5/da/d2e/c37 to d5/d49/dc0/c146 0 2024-09-16T00:26:14.116 INFO:tasks.workunit.client.1.smithi169.stdout:5/677: write d2/da/d10/f27 [2542439,11196] 0 2024-09-16T00:26:14.119 INFO:tasks.workunit.client.1.smithi169.stdout:7/820: creat d7/df/f10b x:0 0 0 2024-09-16T00:26:14.120 INFO:tasks.workunit.client.1.smithi169.stdout:9/918: mknod d5/da/d2e/d99/d100/d10f/d11c/c147 0 2024-09-16T00:26:14.120 INFO:tasks.workunit.client.1.smithi169.stdout:7/821: readlink d7/d71/lde 0 2024-09-16T00:26:14.123 INFO:tasks.workunit.client.1.smithi169.stdout:4/743: write d2/d12/d13/d1f/d3e/f8c [450237,3450] 0 2024-09-16T00:26:14.126 INFO:tasks.workunit.client.1.smithi169.stdout:7/822: rename d7/d16/d49/f68 to d7/dd4/f10c 0 2024-09-16T00:26:14.126 INFO:tasks.workunit.client.1.smithi169.stdout:4/744: write d2/d12/d13/f25 [2505922,125475] 0 2024-09-16T00:26:14.129 INFO:tasks.workunit.client.1.smithi169.stdout:5/678: dwrite d2/da/d10/d13/d1c/d32/d53/d4e/d89/d9a/fa5 [0,4194304] 0 2024-09-16T00:26:14.129 INFO:tasks.workunit.client.1.smithi169.stdout:7/823: dread - d7/d71/d76/d9c/ddf/ff6 zero size 2024-09-16T00:26:14.134 INFO:tasks.workunit.client.1.smithi169.stdout:4/745: chown d2/d12/d13/d27/c99 20 1 2024-09-16T00:26:14.134 INFO:tasks.workunit.client.1.smithi169.stdout:9/919: truncate d5/da/d5b/d6c/da0/f7a 2991210 0 2024-09-16T00:26:14.135 INFO:tasks.workunit.client.1.smithi169.stdout:8/876: write d7/dc/d17/f66 [725322,112503] 0 2024-09-16T00:26:14.136 INFO:tasks.workunit.client.1.smithi169.stdout:7/824: creat d7/df/d95/f10d x:0 0 0 2024-09-16T00:26:14.136 INFO:tasks.workunit.client.1.smithi169.stdout:4/746: read d2/f3 [3255927,75676] 0 2024-09-16T00:26:14.138 INFO:tasks.workunit.client.1.smithi169.stdout:2/721: dwrite d7/f55 [0,4194304] 0 2024-09-16T00:26:14.139 INFO:tasks.workunit.client.1.smithi169.stdout:9/920: fsync d5/da/dd0/f113 0 2024-09-16T00:26:14.141 INFO:tasks.workunit.client.1.smithi169.stdout:9/921: write d5/da/d2e/f7b [817957,101083] 0 2024-09-16T00:26:14.142 INFO:tasks.workunit.client.1.smithi169.stdout:5/679: truncate d2/da/d10/d13/d1c/d40/d43/f4c 970511 0 2024-09-16T00:26:14.143 INFO:tasks.workunit.client.1.smithi169.stdout:4/747: rmdir d2/d12/d9c/dc2 39 2024-09-16T00:26:14.145 INFO:tasks.workunit.client.1.smithi169.stdout:4/748: chown d2/d12/da7 917846 1 2024-09-16T00:26:14.149 INFO:tasks.workunit.client.1.smithi169.stdout:9/922: mkdir d5/da/d73/d135/d148 0 2024-09-16T00:26:14.151 INFO:tasks.workunit.client.1.smithi169.stdout:8/877: dwrite d7/dc/d2a/f30 [0,4194304] 0 2024-09-16T00:26:14.154 INFO:tasks.workunit.client.1.smithi169.stdout:5/680: dwrite d2/ff [0,4194304] 0 2024-09-16T00:26:14.159 INFO:tasks.workunit.client.1.smithi169.stdout:5/681: write d2/da/d10/d13/f35 [5263220,46482] 0 2024-09-16T00:26:14.163 INFO:tasks.workunit.client.1.smithi169.stdout:8/878: creat d7/dc/d26/d41/d4e/d102/f121 x:0 0 0 2024-09-16T00:26:14.166 INFO:tasks.workunit.client.1.smithi169.stdout:9/923: unlink d5/da/d2e/d99/d12c/lc8 0 2024-09-16T00:26:14.166 INFO:tasks.workunit.client.1.smithi169.stdout:2/722: dwrite d7/db/d27/d29/d37/f97 [0,4194304] 0 2024-09-16T00:26:14.167 INFO:tasks.workunit.client.1.smithi169.stdout:4/749: getdents d2/d12/d13/d4e 0 2024-09-16T00:26:14.171 INFO:tasks.workunit.client.1.smithi169.stdout:5/682: mknod d2/da/d10/d13/d1c/d32/d53/d83/d7d/cf3 0 2024-09-16T00:26:14.176 INFO:tasks.workunit.client.1.smithi169.stdout:8/879: dwrite d7/dc/d17/d48/dce/fdd [0,4194304] 0 2024-09-16T00:26:14.176 INFO:tasks.workunit.client.1.smithi169.stdout:4/750: getdents d2/d12/d13/d1f/d3e/d5b/d70 0 2024-09-16T00:26:14.177 INFO:tasks.workunit.client.1.smithi169.stdout:5/683: fsync d2/da/d10/d13/d1c/d32/d53/d83/fa6 0 2024-09-16T00:26:14.179 INFO:tasks.workunit.client.1.smithi169.stdout:8/880: chown d7/dc/d26/d41/d92/dcf/cb8 176588495 1 2024-09-16T00:26:14.181 INFO:tasks.workunit.client.1.smithi169.stdout:8/881: write f6 [5494319,117592] 0 2024-09-16T00:26:14.183 INFO:tasks.workunit.client.1.smithi169.stdout:8/882: write d7/d37/f9b [1364125,82747] 0 2024-09-16T00:26:14.185 INFO:tasks.workunit.client.1.smithi169.stdout:5/684: getdents d2/da/d10/d13/d1c/d94 0 2024-09-16T00:26:14.186 INFO:tasks.workunit.client.1.smithi169.stdout:8/883: mknod d7/d87/dd5/c122 0 2024-09-16T00:26:14.187 INFO:tasks.workunit.client.1.smithi169.stdout:8/884: write d7/d87/dd5/f7e [598045,52437] 0 2024-09-16T00:26:14.189 INFO:tasks.workunit.client.1.smithi169.stdout:8/885: read - d7/dc/d17/d24/d32/f101 zero size 2024-09-16T00:26:14.193 INFO:tasks.workunit.client.1.smithi169.stdout:8/886: unlink d7/dc/d26/d41/d4e/d81/ce0 0 2024-09-16T00:26:14.194 INFO:tasks.workunit.client.1.smithi169.stdout:5/685: dwrite d2/da/d10/d13/d24/f47 [4194304,4194304] 0 2024-09-16T00:26:14.201 INFO:tasks.workunit.client.1.smithi169.stdout:8/887: link d7/dc/d26/d41/d4e/d81/d9e/da5/cc3 d7/dc/d17/d48/dce/d112/c123 0 2024-09-16T00:26:14.204 INFO:tasks.workunit.client.1.smithi169.stdout:8/888: link d7/dc/d17/d24/d32/lc5 d7/dc/d17/d48/dce/l124 0 2024-09-16T00:26:14.204 INFO:tasks.workunit.client.1.smithi169.stdout:5/686: dread d2/da/d10/d13/d1c/d32/d53/f4d [0,4194304] 0 2024-09-16T00:26:14.209 INFO:tasks.workunit.client.1.smithi169.stdout:8/889: dread d7/dc/d26/d41/d4e/f7c [0,4194304] 0 2024-09-16T00:26:14.214 INFO:tasks.workunit.client.1.smithi169.stdout:5/687: dread d2/da/d10/d13/d1c/d40/d43/f4c [0,4194304] 0 2024-09-16T00:26:14.218 INFO:tasks.workunit.client.1.smithi169.stdout:2/723: dwrite d7/db/d27/d29/d77/fad [0,4194304] 0 2024-09-16T00:26:14.219 INFO:tasks.workunit.client.1.smithi169.stdout:9/924: dwrite d5/da/d2e/d99/d100/d10f/d11c/fa9 [0,4194304] 0 2024-09-16T00:26:14.226 INFO:tasks.workunit.client.1.smithi169.stdout:8/890: dwrite d7/dc/d26/fa7 [0,4194304] 0 2024-09-16T00:26:14.226 INFO:tasks.workunit.client.1.smithi169.stdout:9/925: creat d5/da/d73/d135/f149 x:0 0 0 2024-09-16T00:26:14.228 INFO:tasks.workunit.client.1.smithi169.stdout:8/891: chown d7/d37/c70 0 1 2024-09-16T00:26:14.232 INFO:tasks.workunit.client.1.smithi169.stdout:2/724: link d7/db/d27/d29/d77/c88 d7/db/d20/d21/c107 0 2024-09-16T00:26:14.235 INFO:tasks.workunit.client.1.smithi169.stdout:2/725: fdatasync f2 0 2024-09-16T00:26:14.236 INFO:tasks.workunit.client.1.smithi169.stdout:8/892: dread - d7/dc/d26/d41/d92/dcf/d3c/d82/fd7 zero size 2024-09-16T00:26:14.239 INFO:tasks.workunit.client.1.smithi169.stdout:4/751: dwrite d2/d12/d13/d4e/d50/f79 [0,4194304] 0 2024-09-16T00:26:14.241 INFO:tasks.workunit.client.1.smithi169.stdout:9/926: dwrite d5/da/d2e/d47/f119 [0,4194304] 0 2024-09-16T00:26:14.241 INFO:tasks.workunit.client.1.smithi169.stdout:2/726: fdatasync d7/daa/fe3 0 2024-09-16T00:26:14.246 INFO:tasks.workunit.client.1.smithi169.stdout:4/752: mknod d2/d12/d13/d5c/da6/dd3/ce9 0 2024-09-16T00:26:14.252 INFO:tasks.workunit.client.1.smithi169.stdout:8/893: creat d7/dc/d26/d41/d92/dcf/d3c/f125 x:0 0 0 2024-09-16T00:26:14.252 INFO:tasks.workunit.client.1.smithi169.stdout:5/688: dwrite d2/da/d10/d13/d1c/d40/f5b [0,4194304] 0 2024-09-16T00:26:14.254 INFO:tasks.workunit.client.1.smithi169.stdout:9/927: mkdir d5/da/d2e/d14a 0 2024-09-16T00:26:14.255 INFO:tasks.workunit.client.1.smithi169.stdout:8/894: fsync d7/dc/d26/d41/d4e/d81/d9e/da5/fcd 0 2024-09-16T00:26:14.258 INFO:tasks.workunit.client.1.smithi169.stdout:5/689: mkdir d2/da/d10/d13/d1c/d2d/db2/de7/df4 0 2024-09-16T00:26:14.259 INFO:tasks.workunit.client.1.smithi169.stdout:9/928: symlink d5/da/d5b/d91/d13c/l14b 0 2024-09-16T00:26:14.261 INFO:tasks.workunit.client.1.smithi169.stdout:8/895: symlink d7/dc/d26/d41/d92/dcf/l126 0 2024-09-16T00:26:14.261 INFO:tasks.workunit.client.1.smithi169.stdout:5/690: write d2/da/d10/d13/d1c/d2d/f63 [27372,22207] 0 2024-09-16T00:26:14.262 INFO:tasks.workunit.client.1.smithi169.stdout:9/929: mknod d5/de0/c14c 0 2024-09-16T00:26:14.263 INFO:tasks.workunit.client.1.smithi169.stdout:2/727: truncate f2 3185542 0 2024-09-16T00:26:14.266 INFO:tasks.workunit.client.1.smithi169.stdout:2/728: mkdir d7/d6b/d70/de0/d108 0 2024-09-16T00:26:14.268 INFO:tasks.workunit.client.1.smithi169.stdout:9/930: getdents d5 0 2024-09-16T00:26:14.270 INFO:tasks.workunit.client.1.smithi169.stdout:8/896: dwrite d7/dc/d26/d41/d92/dcf/f91 [0,4194304] 0 2024-09-16T00:26:14.272 INFO:tasks.workunit.client.1.smithi169.stdout:9/931: getdents d5/da/d5b/d91/db3/d129 0 2024-09-16T00:26:14.274 INFO:tasks.workunit.client.1.smithi169.stdout:8/897: mknod d7/d87/c127 0 2024-09-16T00:26:14.275 INFO:tasks.workunit.client.1.smithi169.stdout:2/729: truncate d7/db/d27/d29/d37/dd1/ff0 4568766 0 2024-09-16T00:26:14.276 INFO:tasks.workunit.client.1.smithi169.stdout:2/730: dread - d7/db/d89/dc2/fd4 zero size 2024-09-16T00:26:14.280 INFO:tasks.workunit.client.1.smithi169.stdout:8/898: rename d7/dc/d17/d24/d32/fda to d7/dc/f128 0 2024-09-16T00:26:14.280 INFO:tasks.workunit.client.1.smithi169.stdout:4/753: write d2/d12/d13/d1f/d3e/d5b/f73 [634051,24872] 0 2024-09-16T00:26:14.281 INFO:tasks.workunit.client.1.smithi169.stdout:2/731: creat d7/db/d89/db4/d9d/db9/f109 x:0 0 0 2024-09-16T00:26:14.284 INFO:tasks.workunit.client.1.smithi169.stdout:2/732: truncate d7/db/d27/d29/d77/fef 85965 0 2024-09-16T00:26:14.285 INFO:tasks.workunit.client.1.smithi169.stdout:4/754: rename d2/d12/d13/d5c/da6/dd3/fdd to d2/dbb/d59/fea 0 2024-09-16T00:26:14.288 INFO:tasks.workunit.client.1.smithi169.stdout:2/733: write d7/d1c/d26/f76 [1200661,65023] 0 2024-09-16T00:26:14.289 INFO:tasks.workunit.client.1.smithi169.stdout:4/755: creat d2/d12/d13/d1f/d3e/dde/feb x:0 0 0 2024-09-16T00:26:14.291 INFO:tasks.workunit.client.1.smithi169.stdout:2/734: mkdir d7/db/db8/daf/d10a 0 2024-09-16T00:26:14.293 INFO:tasks.workunit.client.1.smithi169.stdout:2/735: chown d7/db/d27/lbb 400599 1 2024-09-16T00:26:14.295 INFO:tasks.workunit.client.1.smithi169.stdout:2/736: stat d7/d1c/d26/d96/f8e 0 2024-09-16T00:26:14.298 INFO:tasks.workunit.client.1.smithi169.stdout:5/691: dwrite d2/da/d10/d13/d1c/d32/d53/d83/fa6 [0,4194304] 0 2024-09-16T00:26:14.298 INFO:tasks.workunit.client.1.smithi169.stdout:2/737: write d7/d6b/d70/de0/f104 [187152,57400] 0 2024-09-16T00:26:14.302 INFO:tasks.workunit.client.1.smithi169.stdout:4/756: write d2/d12/d13/d5c/da6/fb0 [809888,87584] 0 2024-09-16T00:26:14.303 INFO:tasks.workunit.client.1.smithi169.stdout:5/692: symlink d2/da/d10/d13/d1c/d40/d43/lf5 0 2024-09-16T00:26:14.303 INFO:tasks.workunit.client.1.smithi169.stdout:4/757: rmdir d2 39 2024-09-16T00:26:14.306 INFO:tasks.workunit.client.1.smithi169.stdout:2/738: getdents d7/db/d89/db4/d69/da2 0 2024-09-16T00:26:14.307 INFO:tasks.workunit.client.1.smithi169.stdout:4/758: chown d2/d12/l44 1472 1 2024-09-16T00:26:14.307 INFO:tasks.workunit.client.1.smithi169.stdout:2/739: stat d7/db/db8/d6d/f71 0 2024-09-16T00:26:14.309 INFO:tasks.workunit.client.1.smithi169.stdout:4/759: chown d2/d12/d13/d1f/d3e/dde/le3 2 1 2024-09-16T00:26:14.310 INFO:tasks.workunit.client.1.smithi169.stdout:5/693: dwrite d2/fd [0,4194304] 0 2024-09-16T00:26:14.310 INFO:tasks.workunit.client.1.smithi169.stdout:2/740: truncate d7/d1c/d26/d96/ff9 788065 0 2024-09-16T00:26:14.311 INFO:tasks.workunit.client.1.smithi169.stdout:2/741: chown d7/d1c/c1f 661155 1 2024-09-16T00:26:14.313 INFO:tasks.workunit.client.1.smithi169.stdout:2/742: dread - d7/db/d89/dc2/fd4 zero size 2024-09-16T00:26:14.317 INFO:tasks.workunit.client.1.smithi169.stdout:8/899: dread d7/dc/d26/d46/f6b [0,4194304] 0 2024-09-16T00:26:14.319 INFO:tasks.workunit.client.1.smithi169.stdout:4/760: dread d2/d12/d13/d1f/f8e [0,4194304] 0 2024-09-16T00:26:14.320 INFO:tasks.workunit.client.1.smithi169.stdout:4/761: dread - d2/d12/fbe zero size 2024-09-16T00:26:14.322 INFO:tasks.workunit.client.1.smithi169.stdout:8/900: unlink d7/dc/d17/d24/d32/d8d/fc8 0 2024-09-16T00:26:14.324 INFO:tasks.workunit.client.1.smithi169.stdout:8/901: truncate d7/dc/d26/d41/d4e/d81/d9e/da5/ffb 858899 0 2024-09-16T00:26:14.325 INFO:tasks.workunit.client.1.smithi169.stdout:4/762: rename d2/d12/da7/fcf to d2/d12/d9c/fec 0 2024-09-16T00:26:14.326 INFO:tasks.workunit.client.1.smithi169.stdout:2/743: dread d7/db/d89/db4/dc8/fe1 [0,4194304] 0 2024-09-16T00:26:14.327 INFO:tasks.workunit.client.1.smithi169.stdout:8/902: creat d7/dc/d26/d41/d92/dcf/d80/dbd/f129 x:0 0 0 2024-09-16T00:26:14.327 INFO:tasks.workunit.client.1.smithi169.stdout:2/744: chown d7/d1c/d26/fe9 2 1 2024-09-16T00:26:14.330 INFO:tasks.workunit.client.1.smithi169.stdout:4/763: getdents d2/d12/d13/d5c/da6/dd3 0 2024-09-16T00:26:14.330 INFO:tasks.workunit.client.1.smithi169.stdout:4/764: dread - d2/d12/d9c/da4/fc5 zero size 2024-09-16T00:26:14.331 INFO:tasks.workunit.client.1.smithi169.stdout:8/903: unlink d7/dc/d26/d41/d92/dcf/d3c/d82/db3/le3 0 2024-09-16T00:26:14.331 INFO:tasks.workunit.client.1.smithi169.stdout:4/765: chown d2/d12/d9c/faa 232838203 1 2024-09-16T00:26:14.332 INFO:tasks.workunit.client.1.smithi169.stdout:2/745: creat d7/d1c/d26/d96/dab/f10b x:0 0 0 2024-09-16T00:26:14.334 INFO:tasks.workunit.client.1.smithi169.stdout:2/746: symlink d7/d1c/dc3/l10c 0 2024-09-16T00:26:14.337 INFO:tasks.workunit.client.1.smithi169.stdout:2/747: unlink f3 0 2024-09-16T00:26:14.338 INFO:tasks.workunit.client.1.smithi169.stdout:8/904: dwrite d7/d87/dd5/ded/f100 [0,4194304] 0 2024-09-16T00:26:14.343 INFO:tasks.workunit.client.1.smithi169.stdout:8/905: symlink d7/dc/d26/d41/d67/l12a 0 2024-09-16T00:26:14.343 INFO:tasks.workunit.client.1.smithi169.stdout:5/694: dwrite d2/da/d10/d13/d1c/d32/d53/f92 [0,4194304] 0 2024-09-16T00:26:14.346 INFO:tasks.workunit.client.1.smithi169.stdout:8/906: truncate d7/dc/d17/d24/d32/f115 39179 0 2024-09-16T00:26:14.348 INFO:tasks.workunit.client.1.smithi169.stdout:4/766: truncate d2/d12/d13/d5c/da6/fb0 293630 0 2024-09-16T00:26:14.350 INFO:tasks.workunit.client.1.smithi169.stdout:5/695: symlink d2/da/d10/d13/d24/de9/ddf/lf6 0 2024-09-16T00:26:14.351 INFO:tasks.workunit.client.1.smithi169.stdout:8/907: mknod d7/c12b 0 2024-09-16T00:26:14.353 INFO:tasks.workunit.client.1.smithi169.stdout:2/748: write f2 [1249016,3127] 0 2024-09-16T00:26:14.354 INFO:tasks.workunit.client.1.smithi169.stdout:2/749: chown d7/d1c/d26 4 1 2024-09-16T00:26:14.355 INFO:tasks.workunit.client.1.smithi169.stdout:5/696: creat d2/da/d10/d13/d1c/d2d/db2/de7/ff7 x:0 0 0 2024-09-16T00:26:14.355 INFO:tasks.workunit.client.1.smithi169.stdout:4/767: creat d2/dbb/fed x:0 0 0 2024-09-16T00:26:14.356 INFO:tasks.workunit.client.1.smithi169.stdout:5/697: chown d2/fd 751 1 2024-09-16T00:26:14.357 INFO:tasks.workunit.client.1.smithi169.stdout:8/908: unlink d7/d8c/ff4 0 2024-09-16T00:26:14.360 INFO:tasks.workunit.client.1.smithi169.stdout:5/698: creat d2/da/d10/d13/d1c/d32/ff8 x:0 0 0 2024-09-16T00:26:14.361 INFO:tasks.workunit.client.1.smithi169.stdout:8/909: symlink d7/dc/d26/l12c 0 2024-09-16T00:26:14.362 INFO:tasks.workunit.client.1.smithi169.stdout:2/750: creat d7/db/d27/d29/f10d x:0 0 0 2024-09-16T00:26:14.363 INFO:tasks.workunit.client.1.smithi169.stdout:4/768: dwrite d2/d12/d13/d4e/fc1 [0,4194304] 0 2024-09-16T00:26:14.364 INFO:tasks.workunit.client.1.smithi169.stdout:5/699: chown d2/da/d10/d13/d1c/d2d/db2/fd0 9442757 1 2024-09-16T00:26:14.364 INFO:tasks.workunit.client.1.smithi169.stdout:8/910: chown d7/dc/d17/d24/d32/d4d/d111 17354829 1 2024-09-16T00:26:14.366 INFO:tasks.workunit.client.1.smithi169.stdout:4/769: truncate d2/d12/dcc/d9e/f72 1376007 0 2024-09-16T00:26:14.368 INFO:tasks.workunit.client.1.smithi169.stdout:8/911: truncate d7/dc/d17/d24/d32/d105/f110 1011322 0 2024-09-16T00:26:14.368 INFO:tasks.workunit.client.1.smithi169.stdout:4/770: mknod d2/d12/cee 0 2024-09-16T00:26:14.369 INFO:tasks.workunit.client.1.smithi169.stdout:5/700: getdents d2/da/d10/d13/d1c/d32/d53 0 2024-09-16T00:26:14.371 INFO:tasks.workunit.client.1.smithi169.stdout:5/701: fsync d2/da/d10/d13/dc0/ff2 0 2024-09-16T00:26:14.372 INFO:tasks.workunit.client.1.smithi169.stdout:4/771: mkdir d2/d12/d13/d1f/d3e/d5b/d70/def 0 2024-09-16T00:26:14.373 INFO:tasks.workunit.client.1.smithi169.stdout:4/772: readlink d2/d12/d13/d27/lc7 0 2024-09-16T00:26:14.373 INFO:tasks.workunit.client.1.smithi169.stdout:4/773: chown d2/dbb/d59 133375262 1 2024-09-16T00:26:14.375 INFO:tasks.workunit.client.1.smithi169.stdout:5/702: link d2/da/d10/d13/d1c/d71/fe0 d2/da/d10/d13/d1c/d32/d53/dae/ff9 0 2024-09-16T00:26:14.376 INFO:tasks.workunit.client.1.smithi169.stdout:4/774: creat d2/d12/d9c/da4/ff0 x:0 0 0 2024-09-16T00:26:14.377 INFO:tasks.workunit.client.1.smithi169.stdout:5/703: chown d2/da/d10/d13/d1c/d94/c8c 1771400325 1 2024-09-16T00:26:14.377 INFO:tasks.workunit.client.1.smithi169.stdout:4/775: write d2/f3 [1793604,65731] 0 2024-09-16T00:26:14.381 INFO:tasks.workunit.client.1.smithi169.stdout:4/776: rename d2/d12/cee to d2/cf1 0 2024-09-16T00:26:14.383 INFO:tasks.workunit.client.1.smithi169.stdout:4/777: mkdir d2/d12/d13/d4e/d50/df2 0 2024-09-16T00:26:14.385 INFO:tasks.workunit.client.1.smithi169.stdout:4/778: getdents d2/d12/dcc 0 2024-09-16T00:26:14.387 INFO:tasks.workunit.client.1.smithi169.stdout:4/779: read d2/d12/d13/d5c/f9a [178592,61281] 0 2024-09-16T00:26:14.393 INFO:tasks.workunit.client.1.smithi169.stdout:2/751: write d7/d1c/d26/f2c [411643,57234] 0 2024-09-16T00:26:14.395 INFO:tasks.workunit.client.1.smithi169.stdout:4/780: dwrite d2/dbb/d59/fea [0,4194304] 0 2024-09-16T00:26:14.397 INFO:tasks.workunit.client.1.smithi169.stdout:5/704: truncate d2/da/d10/d13/f1d 3200781 0 2024-09-16T00:26:14.400 INFO:tasks.workunit.client.1.smithi169.stdout:5/705: write d2/da/d10/d13/d1c/d40/d68/fb9 [3595756,130742] 0 2024-09-16T00:26:14.402 INFO:tasks.workunit.client.1.smithi169.stdout:8/912: dwrite d7/d37/fc2 [0,4194304] 0 2024-09-16T00:26:14.403 INFO:tasks.workunit.client.1.smithi169.stdout:4/781: creat d2/d12/d13/d1f/d3e/d83/dc8/ff3 x:0 0 0 2024-09-16T00:26:14.405 INFO:tasks.workunit.client.1.smithi169.stdout:2/752: dread d7/db/d27/d29/d77/fad [0,4194304] 0 2024-09-16T00:26:14.408 INFO:tasks.workunit.client.1.smithi169.stdout:4/782: truncate d2/d12/d13/d1f/d3e/f8c 5154359 0 2024-09-16T00:26:14.410 INFO:tasks.workunit.client.1.smithi169.stdout:2/753: mkdir d7/db/d27/d29/d37/d10e 0 2024-09-16T00:26:14.411 INFO:tasks.workunit.client.1.smithi169.stdout:8/913: stat d7/dc/d26/d41/d92/dcf/d3c/d82/lc4 0 2024-09-16T00:26:14.412 INFO:tasks.workunit.client.1.smithi169.stdout:5/706: creat d2/da/d10/d13/d1c/d2d/db2/de7/df4/ffa x:0 0 0 2024-09-16T00:26:14.414 INFO:tasks.workunit.client.1.smithi169.stdout:8/914: unlink d7/dc/d17/d48/ff9 0 2024-09-16T00:26:14.415 INFO:tasks.workunit.client.1.smithi169.stdout:8/915: write d7/dc/d26/fa7 [2698302,10519] 0 2024-09-16T00:26:14.415 INFO:tasks.workunit.client.1.smithi169.stdout:2/754: symlink d7/db/l10f 0 2024-09-16T00:26:14.417 INFO:tasks.workunit.client.1.smithi169.stdout:4/783: creat d2/d12/d9c/dc2/ff4 x:0 0 0 2024-09-16T00:26:14.419 INFO:tasks.workunit.client.1.smithi169.stdout:2/755: mknod d7/d1c/dc3/c110 0 2024-09-16T00:26:14.419 INFO:tasks.workunit.client.1.smithi169.stdout:5/707: dread d2/da/d10/d13/d1c/d32/d53/d4e/d89/d9a/fa5 [0,4194304] 0 2024-09-16T00:26:14.420 INFO:tasks.workunit.client.1.smithi169.stdout:8/916: truncate f5 1764120 0 2024-09-16T00:26:14.422 INFO:tasks.workunit.client.1.smithi169.stdout:2/756: creat d7/db/d89/db4/d9d/f111 x:0 0 0 2024-09-16T00:26:14.423 INFO:tasks.workunit.client.1.smithi169.stdout:2/757: chown d7/db 313 1 2024-09-16T00:26:14.423 INFO:tasks.workunit.client.1.smithi169.stdout:5/708: truncate d2/da/d10/d13/d1c/d32/d53/dae/ff9 286603 0 2024-09-16T00:26:14.424 INFO:tasks.workunit.client.1.smithi169.stdout:8/917: symlink d7/d87/dd5/ded/l12d 0 2024-09-16T00:26:14.425 INFO:tasks.workunit.client.1.smithi169.stdout:4/784: getdents d2/d12/d13/d1f/d3e/dde 0 2024-09-16T00:26:14.425 INFO:tasks.workunit.client.1.smithi169.stdout:2/758: mkdir d7/d6b/d70/de0/d112 0 2024-09-16T00:26:14.426 INFO:tasks.workunit.client.1.smithi169.stdout:8/918: fsync d7/dc/d26/d41/d92/dcf/d80/dbd/f10b 0 2024-09-16T00:26:14.427 INFO:tasks.workunit.client.1.smithi169.stdout:2/759: chown d7/d1c/d26/f76 11612 1 2024-09-16T00:26:14.428 INFO:tasks.workunit.client.1.smithi169.stdout:5/709: mkdir d2/da/d10/d13/d1c/d40/d43/dfb 0 2024-09-16T00:26:14.428 INFO:tasks.workunit.client.1.smithi169.stdout:2/760: stat d7/db/d89/db4/d69/d92 0 2024-09-16T00:26:14.428 INFO:tasks.workunit.client.1.smithi169.stdout:4/785: rmdir d2/d12/d13/d5c/da6/dd3 39 2024-09-16T00:26:14.428 INFO:tasks.workunit.client.1.smithi169.stdout:2/761: readlink d7/db/d27/lbb 0 2024-09-16T00:26:14.429 INFO:tasks.workunit.client.1.smithi169.stdout:8/919: mknod d7/dc/d26/dfa/c12e 0 2024-09-16T00:26:14.431 INFO:tasks.workunit.client.1.smithi169.stdout:2/762: creat d7/db/d89/db4/f113 x:0 0 0 2024-09-16T00:26:14.434 INFO:tasks.workunit.client.1.smithi169.stdout:8/920: truncate d7/dc/d26/d41/d4e/d81/fc6 929348 0 2024-09-16T00:26:14.435 INFO:tasks.workunit.client.1.smithi169.stdout:2/763: mkdir d7/db/d89/db4/d69/da2/d114 0 2024-09-16T00:26:14.525 INFO:tasks.workunit.client.1.smithi169.stdout:8/921: readlink d7/dc/d26/d41/d92/dcf/d80/dbd/l11f 0 2024-09-16T00:26:14.526 INFO:tasks.workunit.client.1.smithi169.stdout:2/764: getdents d7/db/db8/daf/d10a 0 2024-09-16T00:26:14.529 INFO:tasks.workunit.client.1.smithi169.stdout:2/765: write d7/db/d27/d29/f10d [973602,127616] 0 2024-09-16T00:26:14.530 INFO:tasks.workunit.client.1.smithi169.stdout:8/922: dread d7/dc/d26/f9d [0,4194304] 0 2024-09-16T00:26:14.532 INFO:tasks.workunit.client.1.smithi169.stdout:4/786: dwrite d2/d12/d13/d1f/d3e/d83/f6e [0,4194304] 0 2024-09-16T00:26:14.533 INFO:tasks.workunit.client.1.smithi169.stdout:2/766: mknod d7/db/d89/db4/d69/d92/c115 0 2024-09-16T00:26:14.533 INFO:tasks.workunit.client.1.smithi169.stdout:8/923: truncate d7/dc/d2a/f120 914468 0 2024-09-16T00:26:14.534 INFO:tasks.workunit.client.1.smithi169.stdout:4/787: symlink d2/d12/d13/d5c/lf5 0 2024-09-16T00:26:14.536 INFO:tasks.workunit.client.1.smithi169.stdout:4/788: write d2/d12/d13/d1f/fe4 [1196822,91930] 0 2024-09-16T00:26:14.542 INFO:tasks.workunit.client.1.smithi169.stdout:2/767: dwrite d7/db/d89/db4/d9d/f111 [0,4194304] 0 2024-09-16T00:26:14.547 INFO:tasks.workunit.client.1.smithi169.stdout:4/789: dwrite d2/d12/d13/fb9 [0,4194304] 0 2024-09-16T00:26:14.548 INFO:tasks.workunit.client.1.smithi169.stdout:4/790: chown d2/f9d 932 1 2024-09-16T00:26:14.558 INFO:tasks.workunit.client.1.smithi169.stdout:4/791: creat d2/d12/d13/d4e/d50/df2/ff6 x:0 0 0 2024-09-16T00:26:14.559 INFO:tasks.workunit.client.1.smithi169.stdout:8/924: link d7/dc/d26/d41/d92/dcf/l126 d7/dc/d26/d41/d92/dcf/ddc/d11a/l12f 0 2024-09-16T00:26:14.559 INFO:tasks.workunit.client.1.smithi169.stdout:8/925: chown d7/dc/d26/d41/d92/c103 3392965 1 2024-09-16T00:26:14.561 INFO:tasks.workunit.client.1.smithi169.stdout:8/926: truncate d7/dc/d26/dfa/f10c 986449 0 2024-09-16T00:26:14.562 INFO:tasks.workunit.client.1.smithi169.stdout:8/927: creat d7/dc/d26/d46/f130 x:0 0 0 2024-09-16T00:26:14.563 INFO:tasks.workunit.client.1.smithi169.stdout:8/928: fdatasync d7/dc/d17/d48/fe9 0 2024-09-16T00:26:14.564 INFO:tasks.workunit.client.1.smithi169.stdout:8/929: creat d7/dc/d26/dfa/f131 x:0 0 0 2024-09-16T00:26:14.565 INFO:tasks.workunit.client.1.smithi169.stdout:0/798: sync 2024-09-16T00:26:14.566 INFO:tasks.workunit.client.1.smithi169.stdout:8/930: creat d7/dc/d26/d41/d4e/d102/f132 x:0 0 0 2024-09-16T00:26:14.571 INFO:tasks.workunit.client.1.smithi169.stdout:0/799: dwrite d0/d2/d8/fe [0,4194304] 0 2024-09-16T00:26:14.573 INFO:tasks.workunit.client.1.smithi169.stdout:8/931: dwrite d7/d87/dd5/f21 [0,4194304] 0 2024-09-16T00:26:14.576 INFO:tasks.workunit.client.1.smithi169.stdout:8/932: write d7/dc/d17/d24/d32/f101 [546086,27020] 0 2024-09-16T00:26:14.582 INFO:tasks.workunit.client.1.smithi169.stdout:8/933: mkdir d7/dc/d26/d41/d92/dcf/df0/d10d/d133 0 2024-09-16T00:26:14.585 INFO:tasks.workunit.client.1.smithi169.stdout:0/800: dwrite d0/d2c/f5d [0,4194304] 0 2024-09-16T00:26:14.587 INFO:tasks.workunit.client.1.smithi169.stdout:2/768: dwrite d7/db/f13 [0,4194304] 0 2024-09-16T00:26:14.588 INFO:tasks.workunit.client.1.smithi169.stdout:8/934: creat d7/dc/d17/d24/d32/f134 x:0 0 0 2024-09-16T00:26:14.590 INFO:tasks.workunit.client.1.smithi169.stdout:0/801: chown d0/d2c/d38/de4/d21/f3f 327243245 1 2024-09-16T00:26:14.595 INFO:tasks.workunit.client.1.smithi169.stdout:0/802: creat d0/d2c/d38/de4/d21/d31/d83/f114 x:0 0 0 2024-09-16T00:26:14.598 INFO:tasks.workunit.client.1.smithi169.stdout:0/803: mkdir d0/d2/d8/d5/d60/d115 0 2024-09-16T00:26:14.601 INFO:tasks.workunit.client.1.smithi169.stdout:0/804: unlink d0/d2c/d38/de4/d21/d31/d83/fce 0 2024-09-16T00:26:14.602 INFO:tasks.workunit.client.1.smithi169.stdout:0/805: write d0/d2/d8/d5/fa [3097726,53944] 0 2024-09-16T00:26:14.603 INFO:tasks.workunit.client.1.smithi169.stdout:0/806: chown d0/d4c/d82/ffd 12301 1 2024-09-16T00:26:14.605 INFO:tasks.workunit.client.1.smithi169.stdout:0/807: creat d0/d2/d8/d5/d79/f116 x:0 0 0 2024-09-16T00:26:14.607 INFO:tasks.workunit.client.1.smithi169.stdout:0/808: symlink d0/d2c/d38/de4/d21/d31/l117 0 2024-09-16T00:26:14.618 INFO:tasks.workunit.client.1.smithi169.stdout:2/769: write d7/db/d27/f8f [152920,10278] 0 2024-09-16T00:26:14.620 INFO:tasks.workunit.client.1.smithi169.stdout:8/935: dread d7/f2e [0,4194304] 0 2024-09-16T00:26:14.621 INFO:tasks.workunit.client.1.smithi169.stdout:2/770: creat d7/d6b/f116 x:0 0 0 2024-09-16T00:26:14.622 INFO:tasks.workunit.client.1.smithi169.stdout:0/809: write d0/d2c/d38/de4/d21/d31/d8c/fcc [202062,43338] 0 2024-09-16T00:26:14.624 INFO:tasks.workunit.client.1.smithi169.stdout:8/936: dread d7/dc/d26/d41/d4e/d81/fc6 [0,4194304] 0 2024-09-16T00:26:14.626 INFO:tasks.workunit.client.1.smithi169.stdout:0/810: rmdir d0/d4c/d6f/d106 0 2024-09-16T00:26:14.635 INFO:tasks.workunit.client.1.smithi169.stdout:8/937: truncate d7/dc/d17/fd0 3870028 0 2024-09-16T00:26:14.635 INFO:tasks.workunit.client.1.smithi169.stdout:8/938: stat d7/d37/c70 0 2024-09-16T00:26:14.643 INFO:tasks.workunit.client.1.smithi169.stdout:2/771: dread d7/db/d27/f90 [4194304,4194304] 0 2024-09-16T00:26:14.644 INFO:tasks.workunit.client.1.smithi169.stdout:2/772: read - d7/d1c/dc3/ff3 zero size 2024-09-16T00:26:14.646 INFO:tasks.workunit.client.1.smithi169.stdout:2/773: creat d7/db/db8/daf/d10a/f117 x:0 0 0 2024-09-16T00:26:14.743 INFO:tasks.workunit.client.1.smithi169.stdout:2/774: dread - d7/db/d27/d29/fd6 zero size 2024-09-16T00:26:14.744 INFO:tasks.workunit.client.1.smithi169.stdout:2/775: readlink d7/db/d20/d5c/db2/lca 0 2024-09-16T00:26:14.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:14 smithi169 ceph-mon[34653]: pgmap v260: 65 pgs: 65 active+clean; 3.3 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 141 MiB/s rd, 351 MiB/s wr, 739 op/s 2024-09-16T00:26:14.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:14 smithi112 ceph-mon[27869]: pgmap v260: 65 pgs: 65 active+clean; 3.3 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 141 MiB/s rd, 351 MiB/s wr, 739 op/s 2024-09-16T00:26:14.957 INFO:tasks.workunit.client.1.smithi169.stdout:1/793: sync 2024-09-16T00:26:14.957 INFO:tasks.workunit.client.1.smithi169.stdout:1/794: dread - dc/fe5 zero size 2024-09-16T00:26:14.958 INFO:tasks.workunit.client.1.smithi169.stdout:1/795: read - dc/d6c/fec zero size 2024-09-16T00:26:14.959 INFO:tasks.workunit.client.1.smithi169.stdout:1/796: write dc/d36/d3a/d7f/d93/d72/d73/f95 [3486192,73194] 0 2024-09-16T00:26:14.960 INFO:tasks.workunit.client.1.smithi169.stdout:1/797: readlink dc/d26/l58 0 2024-09-16T00:26:14.961 INFO:tasks.workunit.client.1.smithi169.stdout:1/798: chown dc/d36/d3a/d45/d29/d4d/l56 678661 1 2024-09-16T00:26:14.963 INFO:tasks.workunit.client.1.smithi169.stdout:1/799: chown dc/d36/d3a/d7f/d93/d72/ce6 0 1 2024-09-16T00:26:14.965 INFO:tasks.workunit.client.1.smithi169.stdout:1/800: creat dc/d36/d39/d50/daf/d86/ff2 x:0 0 0 2024-09-16T00:26:14.968 INFO:tasks.workunit.client.1.smithi169.stdout:1/801: creat dc/d36/ff3 x:0 0 0 2024-09-16T00:26:14.969 INFO:tasks.workunit.client.1.smithi169.stdout:1/802: fsync dc/d36/d39/d50/daf/d88/dd1/fdd 0 2024-09-16T00:26:14.969 INFO:tasks.workunit.client.1.smithi169.stdout:1/803: readlink dc/d18/l1b 0 2024-09-16T00:26:14.970 INFO:tasks.workunit.client.1.smithi169.stdout:1/804: readlink dc/d18/l31 0 2024-09-16T00:26:14.979 INFO:tasks.workunit.client.1.smithi169.stdout:1/805: dwrite dc/d18/d67/d69/d7b/d92/f9d [0,4194304] 0 2024-09-16T00:26:15.001 INFO:tasks.workunit.client.1.smithi169.stdout:1/806: dread dc/d36/d3a/d45/d29/d2a/f38 [0,4194304] 0 2024-09-16T00:26:15.001 INFO:tasks.workunit.client.1.smithi169.stdout:1/807: chown dc/d6c/d96/lad 60430 1 2024-09-16T00:26:15.017 INFO:tasks.workunit.client.1.smithi169.stdout:1/808: dread dc/f15 [4194304,4194304] 0 2024-09-16T00:26:15.017 INFO:tasks.workunit.client.1.smithi169.stdout:1/809: write dc/d18/d67/f94 [620996,66774] 0 2024-09-16T00:26:15.079 INFO:tasks.workunit.client.1.smithi169.stdout:6/982: sync 2024-09-16T00:26:15.079 INFO:tasks.workunit.client.1.smithi169.stdout:3/946: sync 2024-09-16T00:26:15.079 INFO:tasks.workunit.client.1.smithi169.stdout:7/825: sync 2024-09-16T00:26:15.081 INFO:tasks.workunit.client.1.smithi169.stdout:6/983: truncate d2/d3f/d41/d5c/d73/f116 886692 0 2024-09-16T00:26:15.085 INFO:tasks.workunit.client.1.smithi169.stdout:6/984: rename d2/d4d/d59/d43/d103/d12b to d2/d3f/d41/d5c/d73/d139 0 2024-09-16T00:26:15.086 INFO:tasks.workunit.client.1.smithi169.stdout:6/985: chown d2/d3f 49 1 2024-09-16T00:26:15.087 INFO:tasks.workunit.client.1.smithi169.stdout:7/826: write d7/d71/d76/d29/fa1 [585810,31532] 0 2024-09-16T00:26:15.092 INFO:tasks.workunit.client.1.smithi169.stdout:3/947: dwrite d7/d50/d90/d13a/d8d/fab [0,4194304] 0 2024-09-16T00:26:15.093 INFO:tasks.workunit.client.1.smithi169.stdout:6/986: getdents d2/d3f/d41/d5c/d73 0 2024-09-16T00:26:15.096 INFO:tasks.workunit.client.1.smithi169.stdout:3/948: creat d7/d1a/d1c/d23/d70/f14f x:0 0 0 2024-09-16T00:26:15.098 INFO:tasks.workunit.client.1.smithi169.stdout:6/987: mknod d2/da4/db1/dbc/d87/c13a 0 2024-09-16T00:26:15.100 INFO:tasks.workunit.client.1.smithi169.stdout:3/949: symlink d7/d1a/d1c/d35/d63/da3/de4/l150 0 2024-09-16T00:26:15.103 INFO:tasks.workunit.client.1.smithi169.stdout:3/950: mkdir d7/d50/d90/dba/df0/d151 0 2024-09-16T00:26:15.104 INFO:tasks.workunit.client.1.smithi169.stdout:6/988: getdents d2/d3f/d41/d52 0 2024-09-16T00:26:15.105 INFO:tasks.workunit.client.1.smithi169.stdout:7/827: dread d7/d5b/d61/d98/ffe [0,4194304] 0 2024-09-16T00:26:15.105 INFO:tasks.workunit.client.1.smithi169.stdout:3/951: creat d7/d1a/d1c/d6c/f152 x:0 0 0 2024-09-16T00:26:15.107 INFO:tasks.workunit.client.1.smithi169.stdout:6/989: unlink d2/d4d/d59/c104 0 2024-09-16T00:26:15.109 INFO:tasks.workunit.client.1.smithi169.stdout:7/828: creat d7/d5b/d61/d98/d99/f10e x:0 0 0 2024-09-16T00:26:15.110 INFO:tasks.workunit.client.1.smithi169.stdout:3/952: creat d7/d50/d90/d13a/d8d/d149/f153 x:0 0 0 2024-09-16T00:26:15.111 INFO:tasks.workunit.client.1.smithi169.stdout:7/829: mknod d7/d71/d76/d29/c10f 0 2024-09-16T00:26:15.113 INFO:tasks.workunit.client.1.smithi169.stdout:3/953: creat d7/d50/d90/d13a/d8d/d149/f154 x:0 0 0 2024-09-16T00:26:15.114 INFO:tasks.workunit.client.1.smithi169.stdout:6/990: getdents d2/d4d 0 2024-09-16T00:26:15.118 INFO:tasks.workunit.client.1.smithi169.stdout:7/830: dwrite d7/d71/d76/d9c/fa9 [0,4194304] 0 2024-09-16T00:26:15.121 INFO:tasks.workunit.client.1.smithi169.stdout:3/954: rename d7/d50/d90/d13a/d8d/d149 to d7/d1a/d1c/d23/d88/dd7/d155 0 2024-09-16T00:26:15.122 INFO:tasks.workunit.client.1.smithi169.stdout:7/831: creat d7/df/d95/f110 x:0 0 0 2024-09-16T00:26:15.124 INFO:tasks.workunit.client.1.smithi169.stdout:7/832: chown d7/d71/fc9 118400 1 2024-09-16T00:26:15.124 INFO:tasks.workunit.client.1.smithi169.stdout:3/955: symlink d7/d1a/d1c/d23/d88/dbe/d143/l156 0 2024-09-16T00:26:15.127 INFO:tasks.workunit.client.1.smithi169.stdout:7/833: rmdir d7/d71/d76/d29/d78/dba 39 2024-09-16T00:26:15.128 INFO:tasks.workunit.client.1.smithi169.stdout:6/991: dwrite d2/d4d/d59/d43/f10b [0,4194304] 0 2024-09-16T00:26:15.129 INFO:tasks.workunit.client.1.smithi169.stdout:7/834: write f5 [5304850,66053] 0 2024-09-16T00:26:15.129 INFO:tasks.workunit.client.1.smithi169.stdout:5/710: sync 2024-09-16T00:26:15.129 INFO:tasks.workunit.client.1.smithi169.stdout:9/932: sync 2024-09-16T00:26:15.131 INFO:tasks.workunit.client.1.smithi169.stdout:7/835: chown d7/d71/d76/d29/d78/f100 58585686 1 2024-09-16T00:26:15.133 INFO:tasks.workunit.client.1.smithi169.stdout:3/956: getdents d7/d1a/d1c/d35/d63 0 2024-09-16T00:26:15.135 INFO:tasks.workunit.client.1.smithi169.stdout:9/933: mkdir d5/de1/d14d 0 2024-09-16T00:26:15.135 INFO:tasks.workunit.client.1.smithi169.stdout:7/836: rmdir d7/d71/d76/d29/d78 39 2024-09-16T00:26:15.136 INFO:tasks.workunit.client.1.smithi169.stdout:7/837: truncate d7/df/f10b 235294 0 2024-09-16T00:26:15.137 INFO:tasks.workunit.client.1.smithi169.stdout:7/838: stat d7/df/d95/l3d 0 2024-09-16T00:26:15.137 INFO:tasks.workunit.client.1.smithi169.stdout:6/992: symlink d2/d3f/d41/d72/d9e/l13b 0 2024-09-16T00:26:15.138 INFO:tasks.workunit.client.1.smithi169.stdout:3/957: write d7/d50/d90/d13a/f46 [369835,87537] 0 2024-09-16T00:26:15.140 INFO:tasks.workunit.client.1.smithi169.stdout:5/711: rename d2/d7c/fab to d2/da/d10/d13/d1c/d32/d53/d4e/d89/d9a/ffc 0 2024-09-16T00:26:15.141 INFO:tasks.workunit.client.1.smithi169.stdout:3/958: dread d7/d1a/d1c/d35/d63/da3/f14a [0,4194304] 0 2024-09-16T00:26:15.142 INFO:tasks.workunit.client.1.smithi169.stdout:6/993: mknod d2/d36/d48/c13c 0 2024-09-16T00:26:15.144 INFO:tasks.workunit.client.1.smithi169.stdout:7/839: rename d7/d5b/lc0 to d7/d5b/d61/d98/d99/df9/dc8/l111 0 2024-09-16T00:26:15.145 INFO:tasks.workunit.client.1.smithi169.stdout:9/934: truncate d5/da/dd0/fd2 1693953 0 2024-09-16T00:26:15.146 INFO:tasks.workunit.client.1.smithi169.stdout:6/994: mkdir d2/d22/d51/d13d 0 2024-09-16T00:26:15.146 INFO:tasks.workunit.client.1.smithi169.stdout:5/712: fsync d2/da/d10/d13/d1c/d40/d54/fce 0 2024-09-16T00:26:15.147 INFO:tasks.workunit.client.1.smithi169.stdout:9/935: fdatasync d5/d49/f110 0 2024-09-16T00:26:15.147 INFO:tasks.workunit.client.1.smithi169.stdout:7/840: truncate d7/d71/fc9 666382 0 2024-09-16T00:26:15.150 INFO:tasks.workunit.client.1.smithi169.stdout:7/841: symlink d7/d71/l112 0 2024-09-16T00:26:15.150 INFO:tasks.workunit.client.1.smithi169.stdout:7/842: chown d7/d5b/lfc 6303967 1 2024-09-16T00:26:15.152 INFO:tasks.workunit.client.1.smithi169.stdout:7/843: write d7/f50 [3528636,123566] 0 2024-09-16T00:26:15.153 INFO:tasks.workunit.client.1.smithi169.stdout:9/936: rmdir d5/da/d2e/d99/d12c 39 2024-09-16T00:26:15.157 INFO:tasks.workunit.client.1.smithi169.stdout:7/844: fsync d7/d71/d76/d29/f85 0 2024-09-16T00:26:15.158 INFO:tasks.workunit.client.1.smithi169.stdout:7/845: fsync d7/df/fe7 0 2024-09-16T00:26:15.158 INFO:tasks.workunit.client.1.smithi169.stdout:9/937: unlink d5/da/d2e/d99/d100/d10f/d11c/l4b 0 2024-09-16T00:26:15.159 INFO:tasks.workunit.client.1.smithi169.stdout:3/959: dread d7/d1a/d1c/d35/d63/d37/f59 [0,4194304] 0 2024-09-16T00:26:15.162 INFO:tasks.workunit.client.1.smithi169.stdout:9/938: creat d5/de0/f14e x:0 0 0 2024-09-16T00:26:15.163 INFO:tasks.workunit.client.1.smithi169.stdout:3/960: readlink d7/d1a/d1c/d6c/l112 0 2024-09-16T00:26:15.166 INFO:tasks.workunit.client.1.smithi169.stdout:3/961: read d7/d1a/fd1 [523458,61659] 0 2024-09-16T00:26:15.168 INFO:tasks.workunit.client.1.smithi169.stdout:7/846: creat d7/d5b/d61/d98/d99/df9/d8e/f113 x:0 0 0 2024-09-16T00:26:15.168 INFO:tasks.workunit.client.1.smithi169.stdout:5/713: dread d2/da/d10/d13/d24/f47 [0,4194304] 0 2024-09-16T00:26:15.169 INFO:tasks.workunit.client.1.smithi169.stdout:6/995: write d2/d4d/d59/d43/f56 [810209,23003] 0 2024-09-16T00:26:15.171 INFO:tasks.workunit.client.1.smithi169.stdout:5/714: readlink d2/da/d10/d13/d24/l93 0 2024-09-16T00:26:15.172 INFO:tasks.workunit.client.1.smithi169.stdout:9/939: dwrite d5/da/d21/de4/f86 [4194304,4194304] 0 2024-09-16T00:26:15.174 INFO:tasks.workunit.client.1.smithi169.stdout:7/847: creat d7/d5b/d61/d98/d99/df9/dc8/f114 x:0 0 0 2024-09-16T00:26:15.175 INFO:tasks.workunit.client.1.smithi169.stdout:7/848: readlink d7/d5b/d61/d98/d99/lb1 0 2024-09-16T00:26:15.176 INFO:tasks.workunit.client.1.smithi169.stdout:3/962: getdents d7/d50/d90/da6 0 2024-09-16T00:26:15.176 INFO:tasks.workunit.client.1.smithi169.stdout:5/715: creat d2/da/d10/d13/d1c/d32/d53/dae/ffd x:0 0 0 2024-09-16T00:26:15.182 INFO:tasks.workunit.client.1.smithi169.stdout:7/849: chown d7/d5b/d61/d98/d99/df9/c97 187482 1 2024-09-16T00:26:15.183 INFO:tasks.workunit.client.1.smithi169.stdout:3/963: unlink d7/f10 0 2024-09-16T00:26:15.184 INFO:tasks.workunit.client.1.smithi169.stdout:5/716: dread d2/da/d10/d13/f1d [0,4194304] 0 2024-09-16T00:26:15.185 INFO:tasks.workunit.client.1.smithi169.stdout:7/850: unlink d7/d16/f6a 0 2024-09-16T00:26:15.190 INFO:tasks.workunit.client.1.smithi169.stdout:5/717: rename d2/da/d67/c78 to d2/da/d10/d13/d24/de9/ddf/cfe 0 2024-09-16T00:26:15.192 INFO:tasks.workunit.client.1.smithi169.stdout:3/964: dwrite d7/d50/d6f/f71 [0,4194304] 0 2024-09-16T00:26:15.195 INFO:tasks.workunit.client.1.smithi169.stdout:9/940: dwrite d5/da/d2e/d47/fce [0,4194304] 0 2024-09-16T00:26:15.202 INFO:tasks.workunit.client.1.smithi169.stdout:5/718: dread d2/da/d10/f11 [0,4194304] 0 2024-09-16T00:26:15.205 INFO:tasks.workunit.client.1.smithi169.stdout:7/851: dread d7/d5b/d61/d98/d99/f9b [0,4194304] 0 2024-09-16T00:26:15.205 INFO:tasks.workunit.client.1.smithi169.stdout:9/941: rename d5/da/d73/cd4 to d5/d49/dc0/d11a/c14f 0 2024-09-16T00:26:15.206 INFO:tasks.workunit.client.1.smithi169.stdout:5/719: readlink d2/da/d10/d13/d1c/d32/d53/d4e/d55/l90 0 2024-09-16T00:26:15.212 INFO:tasks.workunit.client.1.smithi169.stdout:5/720: creat d2/da/d10/d13/d1c/d40/d43/dfb/fff x:0 0 0 2024-09-16T00:26:15.216 INFO:tasks.workunit.client.1.smithi169.stdout:5/721: truncate d2/da/d67/f82 2562557 0 2024-09-16T00:26:15.216 INFO:tasks.workunit.client.1.smithi169.stdout:3/965: dwrite d7/d1a/d1c/d35/d63/da3/f14a [0,4194304] 0 2024-09-16T00:26:15.218 INFO:tasks.workunit.client.1.smithi169.stdout:5/722: chown d2/da/d10/d13/d1c/d40/d43/fe5 401779860 1 2024-09-16T00:26:15.223 INFO:tasks.workunit.client.1.smithi169.stdout:7/852: dread d7/d71/d76/f9e [0,4194304] 0 2024-09-16T00:26:15.224 INFO:tasks.workunit.client.1.smithi169.stdout:9/942: truncate d5/da/d5b/fac 2415561 0 2024-09-16T00:26:15.226 INFO:tasks.workunit.client.1.smithi169.stdout:5/723: creat d2/da/d10/d13/d1c/d71/f100 x:0 0 0 2024-09-16T00:26:15.228 INFO:tasks.workunit.client.1.smithi169.stdout:7/853: fsync d7/d5b/d61/d98/d99/df9/f8d 0 2024-09-16T00:26:15.229 INFO:tasks.workunit.client.1.smithi169.stdout:9/943: write d5/da/d5b/d91/db3/f114 [516956,18131] 0 2024-09-16T00:26:15.229 INFO:tasks.workunit.client.1.smithi169.stdout:3/966: dwrite d7/d1a/d1c/d23/d88/ff9 [0,4194304] 0 2024-09-16T00:26:15.232 INFO:tasks.workunit.client.1.smithi169.stdout:5/724: write d2/da/d10/d13/d1c/fe8 [735368,83043] 0 2024-09-16T00:26:15.232 INFO:tasks.workunit.client.1.smithi169.stdout:3/967: truncate d7/d1a/fd1 1445699 0 2024-09-16T00:26:15.235 INFO:tasks.workunit.client.1.smithi169.stdout:7/854: unlink d7/fdb 0 2024-09-16T00:26:15.236 INFO:tasks.workunit.client.1.smithi169.stdout:7/855: stat d7/d16/l4e 0 2024-09-16T00:26:15.242 INFO:tasks.workunit.client.1.smithi169.stdout:3/968: dread d7/d1a/d1c/d6c/f7a [0,4194304] 0 2024-09-16T00:26:15.243 INFO:tasks.workunit.client.1.smithi169.stdout:7/856: mknod d7/c115 0 2024-09-16T00:26:15.243 INFO:tasks.workunit.client.1.smithi169.stdout:5/725: dwrite d2/da/d10/d13/d1c/d2d/db2/de7/df4/ffa [0,4194304] 0 2024-09-16T00:26:15.247 INFO:tasks.workunit.client.1.smithi169.stdout:3/969: unlink d7/d1a/d1c/d6c/cb2 0 2024-09-16T00:26:15.251 INFO:tasks.workunit.client.1.smithi169.stdout:5/726: truncate d2/fb 1020286 0 2024-09-16T00:26:15.252 INFO:tasks.workunit.client.1.smithi169.stdout:7/857: read d7/d71/d76/d29/d78/fa3 [2491840,119501] 0 2024-09-16T00:26:15.253 INFO:tasks.workunit.client.1.smithi169.stdout:7/858: fsync d7/f50 0 2024-09-16T00:26:15.253 INFO:tasks.workunit.client.1.smithi169.stdout:7/859: stat d7/d71/d76/d29/fac 0 2024-09-16T00:26:15.254 INFO:tasks.workunit.client.1.smithi169.stdout:5/727: fsync d2/f29 0 2024-09-16T00:26:15.256 INFO:tasks.workunit.client.1.smithi169.stdout:5/728: chown d2/da/d10/d13/d1c/d71/fe0 78002554 1 2024-09-16T00:26:15.257 INFO:tasks.workunit.client.1.smithi169.stdout:9/944: dread d5/da/d5b/d6c/fb1 [0,4194304] 0 2024-09-16T00:26:15.257 INFO:tasks.workunit.client.1.smithi169.stdout:3/970: dread d7/f68 [0,4194304] 0 2024-09-16T00:26:15.262 INFO:tasks.workunit.client.1.smithi169.stdout:9/945: read d5/d49/f110 [110469,11121] 0 2024-09-16T00:26:15.263 INFO:tasks.workunit.client.1.smithi169.stdout:7/860: dread d7/d71/d76/fc [0,4194304] 0 2024-09-16T00:26:15.263 INFO:tasks.workunit.client.1.smithi169.stdout:9/946: creat d5/d49/dc0/dd6/d11e/f150 x:0 0 0 2024-09-16T00:26:15.264 INFO:tasks.workunit.client.1.smithi169.stdout:7/861: chown d7/d71/fae 391358451 1 2024-09-16T00:26:15.265 INFO:tasks.workunit.client.1.smithi169.stdout:5/729: dread d2/da/d10/d13/d1c/d94/db4/fb6 [0,4194304] 0 2024-09-16T00:26:15.265 INFO:tasks.workunit.client.1.smithi169.stdout:9/947: rmdir d5/da/d21/de9 39 2024-09-16T00:26:15.268 INFO:tasks.workunit.client.1.smithi169.stdout:7/862: mknod d7/d5b/c116 0 2024-09-16T00:26:15.273 INFO:tasks.workunit.client.1.smithi169.stdout:7/863: mkdir d7/d16/d49/db2/db4/d117 0 2024-09-16T00:26:15.273 INFO:tasks.workunit.client.1.smithi169.stdout:5/730: write d2/da/d10/d13/d1c/d32/d53/dae/ff9 [244198,108329] 0 2024-09-16T00:26:15.274 INFO:tasks.workunit.client.1.smithi169.stdout:5/731: stat d2/f58 0 2024-09-16T00:26:15.281 INFO:tasks.workunit.client.1.smithi169.stdout:5/732: symlink d2/l101 0 2024-09-16T00:26:15.281 INFO:tasks.workunit.client.1.smithi169.stdout:4/792: sync 2024-09-16T00:26:15.281 INFO:tasks.workunit.client.1.smithi169.stdout:0/811: sync 2024-09-16T00:26:15.281 INFO:tasks.workunit.client.1.smithi169.stdout:1/810: sync 2024-09-16T00:26:15.281 INFO:tasks.workunit.client.1.smithi169.stdout:8/939: sync 2024-09-16T00:26:15.281 INFO:tasks.workunit.client.1.smithi169.stdout:2/776: sync 2024-09-16T00:26:15.284 INFO:tasks.workunit.client.1.smithi169.stdout:1/811: symlink dc/d36/d3a/d7f/d93/lf4 0 2024-09-16T00:26:15.284 INFO:tasks.workunit.client.1.smithi169.stdout:0/812: readlink d0/d2/d8/d5/lc4 0 2024-09-16T00:26:15.286 INFO:tasks.workunit.client.1.smithi169.stdout:9/948: dread d5/da/d21/f2c [0,4194304] 0 2024-09-16T00:26:15.287 INFO:tasks.workunit.client.1.smithi169.stdout:8/940: truncate d7/dc/f128 752428 0 2024-09-16T00:26:15.287 INFO:tasks.workunit.client.1.smithi169.stdout:1/812: fdatasync dc/f21 0 2024-09-16T00:26:15.288 INFO:tasks.workunit.client.1.smithi169.stdout:3/971: dread d7/d1a/d1c/d23/d70/fb7 [0,4194304] 0 2024-09-16T00:26:15.290 INFO:tasks.workunit.client.1.smithi169.stdout:2/777: mkdir d7/db/d20/d5c/d118 0 2024-09-16T00:26:15.291 INFO:tasks.workunit.client.1.smithi169.stdout:0/813: mknod d0/d2c/d38/de4/d21/d31/d8c/c118 0 2024-09-16T00:26:15.292 INFO:tasks.workunit.client.1.smithi169.stdout:9/949: write d5/da/ff [2114238,48816] 0 2024-09-16T00:26:15.293 INFO:tasks.workunit.client.1.smithi169.stdout:7/864: dwrite d7/d71/d76/d29/d78/dba/fc2 [0,4194304] 0 2024-09-16T00:26:15.293 INFO:tasks.workunit.client.1.smithi169.stdout:5/733: rename d2/da/d67/f82 to d2/da/d10/d13/d1c/d94/db4/f102 0 2024-09-16T00:26:15.299 INFO:tasks.workunit.client.1.smithi169.stdout:9/950: write d5/da/dd0/f113 [185975,23214] 0 2024-09-16T00:26:15.299 INFO:tasks.workunit.client.1.smithi169.stdout:3/972: creat d7/d50/d90/dba/dc0/f157 x:0 0 0 2024-09-16T00:26:15.301 INFO:tasks.workunit.client.1.smithi169.stdout:2/778: fdatasync d7/d1c/d26/f75 0 2024-09-16T00:26:15.301 INFO:tasks.workunit.client.1.smithi169.stdout:4/793: dwrite d2/d12/d13/d1f/d3e/f8c [0,4194304] 0 2024-09-16T00:26:15.303 INFO:tasks.workunit.client.1.smithi169.stdout:9/951: stat d5/da/d2e/ldc 0 2024-09-16T00:26:15.307 INFO:tasks.workunit.client.1.smithi169.stdout:8/941: mknod d7/dc/d17/d24/d32/d4d/d111/c135 0 2024-09-16T00:26:15.307 INFO:tasks.workunit.client.1.smithi169.stdout:5/734: truncate d2/da/d10/d13/d1c/d40/fbe 509054 0 2024-09-16T00:26:15.308 INFO:tasks.workunit.client.1.smithi169.stdout:9/952: chown d5/da/d5b/d6c/c97 26441 1 2024-09-16T00:26:15.311 INFO:tasks.workunit.client.1.smithi169.stdout:7/865: mknod d7/d16/d49/db2/db4/d117/c118 0 2024-09-16T00:26:15.313 INFO:tasks.workunit.client.1.smithi169.stdout:6/996: sync 2024-09-16T00:26:15.316 INFO:tasks.workunit.client.1.smithi169.stdout:0/814: write d0/d2c/d38/de4/d21/d31/d83/dac/ff0 [735686,129966] 0 2024-09-16T00:26:15.318 INFO:tasks.workunit.client.1.smithi169.stdout:7/866: dwrite d7/d71/d76/d29/d78/dba/fcd [0,4194304] 0 2024-09-16T00:26:15.318 INFO:tasks.workunit.client.1.smithi169.stdout:0/815: write d0/d2c/d38/f84 [1962966,55772] 0 2024-09-16T00:26:15.318 INFO:tasks.workunit.client.1.smithi169.stdout:4/794: symlink d2/d12/d9c/de2/lf7 0 2024-09-16T00:26:15.321 INFO:tasks.workunit.client.1.smithi169.stdout:3/973: creat d7/d50/d90/dba/df0/d151/f158 x:0 0 0 2024-09-16T00:26:15.321 INFO:tasks.workunit.client.1.smithi169.stdout:4/795: chown d2/d12/d13/f6c 536511284 1 2024-09-16T00:26:15.322 INFO:tasks.workunit.client.1.smithi169.stdout:0/816: write d0/d2c/f5d [1881049,16748] 0 2024-09-16T00:26:15.322 INFO:tasks.workunit.client.1.smithi169.stdout:2/779: rmdir d7/db/d27/dff 0 2024-09-16T00:26:15.331 INFO:tasks.workunit.client.1.smithi169.stdout:9/953: link d5/da/l69 d5/da/d2e/d99/d100/l151 0 2024-09-16T00:26:15.331 INFO:tasks.workunit.client.1.smithi169.stdout:4/796: rename d2/d12/l14 to d2/d12/dcc/lf8 0 2024-09-16T00:26:15.334 INFO:tasks.workunit.client.1.smithi169.stdout:1/813: truncate dc/d36/d3a/d7f/d93/fa3 2798829 0 2024-09-16T00:26:15.335 INFO:tasks.workunit.client.1.smithi169.stdout:3/974: truncate d7/d50/d6f/f100 2339221 0 2024-09-16T00:26:15.338 INFO:tasks.workunit.client.1.smithi169.stdout:0/817: getdents d0/d2c/d104/d67 0 2024-09-16T00:26:15.339 INFO:tasks.workunit.client.1.smithi169.stdout:3/975: chown d7/d1a/d1c/f54 1971466 1 2024-09-16T00:26:15.341 INFO:tasks.workunit.client.1.smithi169.stdout:7/867: dwrite d7/d5b/f77 [4194304,4194304] 0 2024-09-16T00:26:15.341 INFO:tasks.workunit.client.1.smithi169.stdout:3/976: write d7/d1a/d1c/d35/d63/f9c [41175,21865] 0 2024-09-16T00:26:15.343 INFO:tasks.workunit.client.1.smithi169.stdout:6/997: dread d2/d3f/d41/d5c/d91/feb [0,4194304] 0 2024-09-16T00:26:15.344 INFO:tasks.workunit.client.1.smithi169.stdout:1/814: unlink dc/d36/d3a/d45/c2f 0 2024-09-16T00:26:15.346 INFO:tasks.workunit.client.1.smithi169.stdout:1/815: chown dc/d36/ff3 1325 1 2024-09-16T00:26:15.348 INFO:tasks.workunit.client.1.smithi169.stdout:1/816: stat dc/d36/d3a/d45/fa7 0 2024-09-16T00:26:15.350 INFO:tasks.workunit.client.1.smithi169.stdout:7/868: creat d7/d71/f119 x:0 0 0 2024-09-16T00:26:15.352 INFO:tasks.workunit.client.1.smithi169.stdout:8/942: dread d7/dc/d26/d41/fa4 [0,4194304] 0 2024-09-16T00:26:15.352 INFO:tasks.workunit.client.1.smithi169.stdout:6/998: mkdir d2/da4/db1/dbc/d138/d13e 0 2024-09-16T00:26:15.353 INFO:tasks.workunit.client.1.smithi169.stdout:3/977: dwrite d7/d1a/d1c/d23/fe7 [0,4194304] 0 2024-09-16T00:26:15.353 INFO:tasks.workunit.client.1.smithi169.stdout:2/780: write d7/db/f17 [2682559,61764] 0 2024-09-16T00:26:15.358 INFO:tasks.workunit.client.1.smithi169.stdout:9/954: write d5/da/d2e/f8a [645525,3344] 0 2024-09-16T00:26:15.359 INFO:tasks.workunit.client.1.smithi169.stdout:2/781: write d7/d1c/f95 [3300460,110719] 0 2024-09-16T00:26:15.364 INFO:tasks.workunit.client.1.smithi169.stdout:8/943: read d7/dc/d17/d48/dce/fdd [1891908,100434] 0 2024-09-16T00:26:15.366 INFO:tasks.workunit.client.1.smithi169.stdout:2/782: stat d7/d6b/d70/de0/le7 0 2024-09-16T00:26:15.367 INFO:tasks.workunit.client.1.smithi169.stdout:9/955: fsync d5/d49/fbf 0 2024-09-16T00:26:15.377 INFO:tasks.workunit.client.1.smithi169.stdout:7/869: dwrite d7/d5b/d61/d98/d99/df9/d8e/f113 [0,4194304] 0 2024-09-16T00:26:15.377 INFO:tasks.workunit.client.1.smithi169.stdout:1/817: dwrite dc/d36/d3a/d45/d29/f8d [0,4194304] 0 2024-09-16T00:26:15.378 INFO:tasks.workunit.client.1.smithi169.stdout:2/783: rename d7/db/f45 to d7/db/d89/f119 0 2024-09-16T00:26:15.379 INFO:tasks.workunit.client.1.smithi169.stdout:4/797: dwrite d2/d12/d13/d5c/da6/fa9 [0,4194304] 0 2024-09-16T00:26:15.382 INFO:tasks.workunit.client.1.smithi169.stdout:1/818: unlink dc/d36/ff3 0 2024-09-16T00:26:15.383 INFO:tasks.workunit.client.1.smithi169.stdout:0/818: dwrite d0/d2/d8/d5/d79/d7c/f94 [0,4194304] 0 2024-09-16T00:26:15.387 INFO:tasks.workunit.client.1.smithi169.stdout:9/956: rmdir d5/da/d2e/d99/d100/d10f/d11c 39 2024-09-16T00:26:15.387 INFO:tasks.workunit.client.1.smithi169.stdout:6/999: truncate d2/d3f/d41/d5c/f65 2046963 0 2024-09-16T00:26:15.389 INFO:tasks.workunit.client.1.smithi169.stdout:7/870: creat d7/dd4/f11a x:0 0 0 2024-09-16T00:26:15.391 INFO:tasks.workunit.client.1.smithi169.stdout:4/798: dread d2/d12/d13/fb9 [0,4194304] 0 2024-09-16T00:26:15.392 INFO:tasks.workunit.client.1.smithi169.stdout:7/871: fsync d7/d5b/f5c 0 2024-09-16T00:26:15.395 INFO:tasks.workunit.client.1.smithi169.stdout:8/944: rename d7/dc/d17/d24/ffc to d7/dc/d26/d41/d92/dcf/d3c/d77/dde/f136 0 2024-09-16T00:26:15.396 INFO:tasks.workunit.client.1.smithi169.stdout:9/957: mkdir d5/da/df3/d152 0 2024-09-16T00:26:15.397 INFO:tasks.workunit.client.1.smithi169.stdout:3/978: truncate d7/d1a/d1c/d23/d70/f95 3766594 0 2024-09-16T00:26:15.397 INFO:tasks.workunit.client.1.smithi169.stdout:9/958: stat d5/da/d2e/d47/fce 0 2024-09-16T00:26:15.400 INFO:tasks.workunit.client.1.smithi169.stdout:4/799: symlink d2/d12/d13/d4e/d50/db8/lf9 0 2024-09-16T00:26:15.401 INFO:tasks.workunit.client.1.smithi169.stdout:7/872: dwrite d7/df/f10b [0,4194304] 0 2024-09-16T00:26:15.402 INFO:tasks.workunit.client.1.smithi169.stdout:0/819: mkdir d0/d10d/d119 0 2024-09-16T00:26:15.408 INFO:tasks.workunit.client.1.smithi169.stdout:4/800: rename d2/d12/d13/l2a to d2/d12/da7/lfa 0 2024-09-16T00:26:15.409 INFO:tasks.workunit.client.1.smithi169.stdout:4/801: stat d2/l34 0 2024-09-16T00:26:15.413 INFO:tasks.workunit.client.1.smithi169.stdout:7/873: write d7/d5b/d61/d98/d99/df9/d8e/fdc [1548284,44465] 0 2024-09-16T00:26:15.415 INFO:tasks.workunit.client.1.smithi169.stdout:9/959: creat d5/da/f153 x:0 0 0 2024-09-16T00:26:15.415 INFO:tasks.workunit.client.1.smithi169.stdout:2/784: write d7/db/f19 [2916814,113982] 0 2024-09-16T00:26:15.416 INFO:tasks.workunit.client.1.smithi169.stdout:9/960: chown d5/d49/dc0/d11a 0 1 2024-09-16T00:26:15.417 INFO:tasks.workunit.client.1.smithi169.stdout:4/802: truncate d2/d12/d13/d1f/f6a 971115 0 2024-09-16T00:26:15.418 INFO:tasks.workunit.client.1.smithi169.stdout:2/785: readlink d7/db/d20/d5c/db2/lbc 0 2024-09-16T00:26:15.421 INFO:tasks.workunit.client.1.smithi169.stdout:8/945: link d7/dc/d26/d41/d92/dcf/c5e d7/dc/d26/d41/d92/dcf/df0/c137 0 2024-09-16T00:26:15.424 INFO:tasks.workunit.client.1.smithi169.stdout:1/819: dread dc/fd [0,4194304] 0 2024-09-16T00:26:15.426 INFO:tasks.workunit.client.1.smithi169.stdout:3/979: write d7/d1a/d1c/d35/d63/f6a [1831974,93467] 0 2024-09-16T00:26:15.430 INFO:tasks.workunit.client.1.smithi169.stdout:1/820: write dc/d36/d3a/d45/f17 [200127,89152] 0 2024-09-16T00:26:15.431 INFO:tasks.workunit.client.1.smithi169.stdout:9/961: rename d5/da/d5b/d6c/dcb/le7 to d5/da/d2e/d99/l154 0 2024-09-16T00:26:15.434 INFO:tasks.workunit.client.1.smithi169.stdout:7/874: getdents d7/d71/d76/d29/d78/dba 0 2024-09-16T00:26:15.437 INFO:tasks.workunit.client.1.smithi169.stdout:2/786: write d7/db/db8/d6d/fba [2531171,42769] 0 2024-09-16T00:26:15.438 INFO:tasks.workunit.client.1.smithi169.stdout:9/962: creat d5/da/df3/f155 x:0 0 0 2024-09-16T00:26:15.440 INFO:tasks.workunit.client.1.smithi169.stdout:8/946: link d7/f2e d7/dc/d26/d41/d92/dcf/d3c/d82/db3/db4/f138 0 2024-09-16T00:26:15.443 INFO:tasks.workunit.client.1.smithi169.stdout:9/963: creat d5/da/d2e/d47/f156 x:0 0 0 2024-09-16T00:26:15.444 INFO:tasks.workunit.client.1.smithi169.stdout:1/821: creat dc/ff5 x:0 0 0 2024-09-16T00:26:15.445 INFO:tasks.workunit.client.1.smithi169.stdout:3/980: link d7/d1a/d1c/d23/d88/dbe/ce9 d7/d50/d90/d13a/dc6/c159 0 2024-09-16T00:26:15.447 INFO:tasks.workunit.client.1.smithi169.stdout:9/964: mknod d5/da/d5b/d6c/dcb/c157 0 2024-09-16T00:26:15.451 INFO:tasks.workunit.client.1.smithi169.stdout:7/875: getdents d7/d16 0 2024-09-16T00:26:15.452 INFO:tasks.workunit.client.1.smithi169.stdout:1/822: dwrite dc/d36/d3a/d45/fe2 [0,4194304] 0 2024-09-16T00:26:15.452 INFO:tasks.workunit.client.1.smithi169.stdout:2/787: rename d7/d6b/d70/d79/dae/lc1 to d7/d6b/d70/l11a 0 2024-09-16T00:26:15.454 INFO:tasks.workunit.client.1.smithi169.stdout:3/981: fdatasync d7/d1a/d1c/d35/d63/f53 0 2024-09-16T00:26:15.456 INFO:tasks.workunit.client.1.smithi169.stdout:9/965: creat d5/da/d2e/d13a/f158 x:0 0 0 2024-09-16T00:26:15.458 INFO:tasks.workunit.client.1.smithi169.stdout:7/876: mknod d7/d5b/d61/d98/d99/c11b 0 2024-09-16T00:26:15.460 INFO:tasks.workunit.client.1.smithi169.stdout:1/823: symlink dc/d36/d3a/d7f/d93/d72/d98/lf6 0 2024-09-16T00:26:15.462 INFO:tasks.workunit.client.1.smithi169.stdout:1/824: stat dc/d36/d3a/d45/fe2 0 2024-09-16T00:26:15.463 INFO:tasks.workunit.client.1.smithi169.stdout:2/788: rename d7/db/d89/db4/d69/da2 to d7/db/db8/daf/d10a/d11b 0 2024-09-16T00:26:15.465 INFO:tasks.workunit.client.1.smithi169.stdout:8/947: write d7/dc/d26/d41/d92/dcf/d80/dbd/fe2 [612551,108687] 0 2024-09-16T00:26:15.466 INFO:tasks.workunit.client.1.smithi169.stdout:7/877: dwrite d7/d71/d76/d29/ff2 [0,4194304] 0 2024-09-16T00:26:15.469 INFO:tasks.workunit.client.1.smithi169.stdout:7/878: chown d7/d71/d76/d29 156 1 2024-09-16T00:26:15.470 INFO:tasks.workunit.client.1.smithi169.stdout:8/948: dread d7/dc/d26/d41/d4e/d81/fc6 [0,4194304] 0 2024-09-16T00:26:15.471 INFO:tasks.workunit.client.1.smithi169.stdout:9/966: symlink d5/da/df3/d152/l159 0 2024-09-16T00:26:15.472 INFO:tasks.workunit.client.1.smithi169.stdout:2/789: mknod d7/d6b/d70/d79/c11c 0 2024-09-16T00:26:15.473 INFO:tasks.workunit.client.1.smithi169.stdout:1/825: dread dc/d36/d3a/d45/d29/d4d/fbe [0,4194304] 0 2024-09-16T00:26:15.474 INFO:tasks.workunit.client.1.smithi169.stdout:8/949: chown d7/dc/d26/l12c 846547 1 2024-09-16T00:26:15.476 INFO:tasks.workunit.client.1.smithi169.stdout:1/826: write dc/d36/d39/d50/daf/d86/fb2 [92850,75550] 0 2024-09-16T00:26:15.477 INFO:tasks.workunit.client.1.smithi169.stdout:9/967: mkdir d5/da/d5b/d6c/d15a 0 2024-09-16T00:26:15.477 INFO:tasks.workunit.client.1.smithi169.stdout:2/790: symlink d7/d1c/d3c/l11d 0 2024-09-16T00:26:15.484 INFO:tasks.workunit.client.1.smithi169.stdout:7/879: rename d7/f9f to d7/d16/d49/f11c 0 2024-09-16T00:26:15.485 INFO:tasks.workunit.client.1.smithi169.stdout:1/827: mkdir dc/d36/d3a/d7f/d93/d72/df7 0 2024-09-16T00:26:15.486 INFO:tasks.workunit.client.1.smithi169.stdout:8/950: getdents d7/dc/d26/d41/d92/dcf/d3c/d77 0 2024-09-16T00:26:15.487 INFO:tasks.workunit.client.1.smithi169.stdout:3/982: dwrite d7/d1a/d1c/d23/d88/ffa [0,4194304] 0 2024-09-16T00:26:15.488 INFO:tasks.workunit.client.1.smithi169.stdout:1/828: chown dc/d36/d3a/d7f/ddc 109 1 2024-09-16T00:26:15.489 INFO:tasks.workunit.client.1.smithi169.stdout:8/951: chown d7/dc/d26/d41/d117 2497 1 2024-09-16T00:26:15.490 INFO:tasks.workunit.client.1.smithi169.stdout:2/791: dwrite d7/db/d20/f83 [0,4194304] 0 2024-09-16T00:26:15.491 INFO:tasks.workunit.client.1.smithi169.stdout:7/880: dread f5 [4194304,4194304] 0 2024-09-16T00:26:15.494 INFO:tasks.workunit.client.1.smithi169.stdout:8/952: dread d7/dc/d17/d24/d32/f115 [0,4194304] 0 2024-09-16T00:26:15.497 INFO:tasks.workunit.client.1.smithi169.stdout:1/829: rename dc/d36/d39/fb0 to dc/d36/d39/d50/daf/d88/ff8 0 2024-09-16T00:26:15.497 INFO:tasks.workunit.client.1.smithi169.stdout:3/983: write d7/d1a/d1c/d23/d88/fa9 [481352,73773] 0 2024-09-16T00:26:15.498 INFO:tasks.workunit.client.1.smithi169.stdout:3/984: readlink d7/d1a/d1c/d6c/l112 0 2024-09-16T00:26:15.500 INFO:tasks.workunit.client.1.smithi169.stdout:8/953: symlink d7/dc/d17/d24/d32/l139 0 2024-09-16T00:26:15.500 INFO:tasks.workunit.client.1.smithi169.stdout:2/792: dread d7/d6b/d70/de0/f104 [0,4194304] 0 2024-09-16T00:26:15.503 INFO:tasks.workunit.client.1.smithi169.stdout:7/881: mkdir d7/d16/d49/db2/db4/d11d 0 2024-09-16T00:26:15.505 INFO:tasks.workunit.client.1.smithi169.stdout:1/830: mkdir dc/d36/d3a/d7f/d93/d72/df9 0 2024-09-16T00:26:15.513 INFO:tasks.workunit.client.1.smithi169.stdout:7/882: rename d7/df/fe7 to d7/d5b/d61/d98/d99/f11e 0 2024-09-16T00:26:15.513 INFO:tasks.workunit.client.1.smithi169.stdout:2/793: dwrite d7/d1c/fb7 [0,4194304] 0 2024-09-16T00:26:15.514 INFO:tasks.workunit.client.1.smithi169.stdout:1/831: fsync dc/d36/d3a/d45/d29/d4d/f9c 0 2024-09-16T00:26:15.515 INFO:tasks.workunit.client.1.smithi169.stdout:9/968: dread d5/da/d5b/d91/d9a/fa3 [0,4194304] 0 2024-09-16T00:26:15.516 INFO:tasks.workunit.client.1.smithi169.stdout:1/832: write dc/f66 [332766,76100] 0 2024-09-16T00:26:15.517 INFO:tasks.workunit.client.1.smithi169.stdout:7/883: symlink d7/l11f 0 2024-09-16T00:26:15.518 INFO:tasks.workunit.client.1.smithi169.stdout:2/794: symlink d7/d1c/dc3/l11e 0 2024-09-16T00:26:15.519 INFO:tasks.workunit.client.1.smithi169.stdout:3/985: dread d7/d50/d90/d13a/f8c [0,4194304] 0 2024-09-16T00:26:15.521 INFO:tasks.workunit.client.1.smithi169.stdout:2/795: dread - d7/db/d89/db4/d9d/db9/f109 zero size 2024-09-16T00:26:15.522 INFO:tasks.workunit.client.1.smithi169.stdout:3/986: write d7/d1a/d1c/d23/d88/fa9 [733717,88358] 0 2024-09-16T00:26:15.523 INFO:tasks.workunit.client.1.smithi169.stdout:1/833: fsync dc/d36/d3a/d45/d29/d2a/f38 0 2024-09-16T00:26:15.523 INFO:tasks.workunit.client.1.smithi169.stdout:9/969: mknod d5/da/d2e/d99/d100/c15b 0 2024-09-16T00:26:15.525 INFO:tasks.workunit.client.1.smithi169.stdout:2/796: read d7/daa/df5/f80 [321752,58682] 0 2024-09-16T00:26:15.527 INFO:tasks.workunit.client.1.smithi169.stdout:2/797: write d7/d1c/fb7 [2723295,77118] 0 2024-09-16T00:26:15.527 INFO:tasks.workunit.client.1.smithi169.stdout:3/987: mkdir d7/d50/d90/d13a/d15a 0 2024-09-16T00:26:15.528 INFO:tasks.workunit.client.1.smithi169.stdout:1/834: mkdir dc/d36/d39/d50/d75/dfa 0 2024-09-16T00:26:15.529 INFO:tasks.workunit.client.1.smithi169.stdout:8/954: dread d7/d37/fc2 [0,4194304] 0 2024-09-16T00:26:15.530 INFO:tasks.workunit.client.1.smithi169.stdout:1/835: readlink dc/d36/d3a/d7f/d93/d72/d98/lf6 0 2024-09-16T00:26:15.531 INFO:tasks.workunit.client.1.smithi169.stdout:8/955: fsync d7/dc/d26/d41/d4e/d81/d9e/da5/fcd 0 2024-09-16T00:26:15.537 INFO:tasks.workunit.client.1.smithi169.stdout:1/836: rmdir dc/d36/d3a/d7f/d93/d72/df9 0 2024-09-16T00:26:15.538 INFO:tasks.workunit.client.1.smithi169.stdout:1/837: stat dc/d18/d67/d69/c9e 0 2024-09-16T00:26:15.540 INFO:tasks.workunit.client.1.smithi169.stdout:7/884: dread d7/d71/d76/d29/f47 [0,4194304] 0 2024-09-16T00:26:15.541 INFO:tasks.workunit.client.1.smithi169.stdout:7/885: write d7/f50 [402364,96714] 0 2024-09-16T00:26:15.542 INFO:tasks.workunit.client.1.smithi169.stdout:7/886: chown d7/df/c27 39609 1 2024-09-16T00:26:15.544 INFO:tasks.workunit.client.1.smithi169.stdout:9/970: write d5/d49/f5a [499473,115752] 0 2024-09-16T00:26:15.546 INFO:tasks.workunit.client.1.smithi169.stdout:7/887: symlink d7/d71/d76/d9c/l120 0 2024-09-16T00:26:15.548 INFO:tasks.workunit.client.1.smithi169.stdout:7/888: chown d7/d5b 6914259 1 2024-09-16T00:26:15.549 INFO:tasks.workunit.client.1.smithi169.stdout:9/971: unlink d5/da/d2e/c12e 0 2024-09-16T00:26:15.550 INFO:tasks.workunit.client.1.smithi169.stdout:1/838: truncate dc/d36/d3a/f9f 2629210 0 2024-09-16T00:26:15.552 INFO:tasks.workunit.client.1.smithi169.stdout:2/798: dwrite d7/db/d20/fa7 [0,4194304] 0 2024-09-16T00:26:15.552 INFO:tasks.workunit.client.1.smithi169.stdout:8/956: dread d7/dc/d17/f28 [0,4194304] 0 2024-09-16T00:26:15.552 INFO:tasks.workunit.client.1.smithi169.stdout:9/972: write d5/da/d2e/fde [18825,48408] 0 2024-09-16T00:26:15.558 INFO:tasks.workunit.client.1.smithi169.stdout:2/799: mkdir d7/db/d20/d21/d105/d11f 0 2024-09-16T00:26:15.562 INFO:tasks.workunit.client.1.smithi169.stdout:8/957: symlink d7/dc/d26/d41/d4e/d81/d9e/l13a 0 2024-09-16T00:26:15.564 INFO:tasks.workunit.client.1.smithi169.stdout:1/839: getdents dc/d18/d67/d69 0 2024-09-16T00:26:15.566 INFO:tasks.workunit.client.1.smithi169.stdout:9/973: dwrite d5/da/d5b/d6c/dcb/f127 [0,4194304] 0 2024-09-16T00:26:15.573 INFO:tasks.workunit.client.1.smithi169.stdout:7/889: dwrite d7/d16/f24 [0,4194304] 0 2024-09-16T00:26:15.573 INFO:tasks.workunit.client.1.smithi169.stdout:1/840: creat dc/d36/d3a/d45/d29/d4d/db6/ffb x:0 0 0 2024-09-16T00:26:15.575 INFO:tasks.workunit.client.1.smithi169.stdout:2/800: dwrite d7/db/f19 [4194304,4194304] 0 2024-09-16T00:26:15.575 INFO:tasks.workunit.client.1.smithi169.stdout:7/890: fsync d7/f10a 0 2024-09-16T00:26:15.578 INFO:tasks.workunit.client.1.smithi169.stdout:8/958: getdents d7/dc/d26/d41/d92/dcf/d80 0 2024-09-16T00:26:15.579 INFO:tasks.workunit.client.1.smithi169.stdout:9/974: unlink d5/da/d2e/d47/d11b/fd9 0 2024-09-16T00:26:15.581 INFO:tasks.workunit.client.1.smithi169.stdout:9/975: creat d5/da/d2e/d13a/f15c x:0 0 0 2024-09-16T00:26:15.583 INFO:tasks.workunit.client.1.smithi169.stdout:8/959: creat d7/dc/d17/d24/d32/d4d/f13b x:0 0 0 2024-09-16T00:26:15.583 INFO:tasks.workunit.client.1.smithi169.stdout:7/891: mkdir d7/d16/d49/d121 0 2024-09-16T00:26:15.590 INFO:tasks.workunit.client.1.smithi169.stdout:8/960: dread d7/dc/d26/d41/fa4 [0,4194304] 0 2024-09-16T00:26:15.592 INFO:tasks.workunit.client.1.smithi169.stdout:8/961: dread - d7/dc/d26/d41/d92/dcf/d3c/d77/fbf zero size 2024-09-16T00:26:15.592 INFO:tasks.workunit.client.1.smithi169.stdout:9/976: rename d5/da/d2e/d99/d100/d10f/d11c/c147 to d5/da/d2e/d47/d11b/c15d 0 2024-09-16T00:26:15.595 INFO:tasks.workunit.client.1.smithi169.stdout:7/892: link d7/c69 d7/d5b/d61/d98/d99/df9/c122 0 2024-09-16T00:26:15.599 INFO:tasks.workunit.client.1.smithi169.stdout:7/893: rename d7/d5b/dc1/d96/cef to d7/d16/d49/db2/c123 0 2024-09-16T00:26:15.600 INFO:tasks.workunit.client.1.smithi169.stdout:2/801: truncate d7/d1c/f95 2185748 0 2024-09-16T00:26:15.605 INFO:tasks.workunit.client.1.smithi169.stdout:7/894: fdatasync d7/d5b/d61/d98/d99/df9/d8e/fdd 0 2024-09-16T00:26:15.606 INFO:tasks.workunit.client.1.smithi169.stdout:1/841: dread dc/d36/d39/d50/daf/d88/ff8 [0,4194304] 0 2024-09-16T00:26:15.609 INFO:tasks.workunit.client.1.smithi169.stdout:9/977: truncate d5/da/d2e/d99/d100/d10f/d11c/fc7 2139909 0 2024-09-16T00:26:15.609 INFO:tasks.workunit.client.1.smithi169.stdout:7/895: truncate d7/d71/d76/d29/d78/dba/fcd 4295709 0 2024-09-16T00:26:15.613 INFO:tasks.workunit.client.1.smithi169.stdout:8/962: dread d7/fd [0,4194304] 0 2024-09-16T00:26:15.615 INFO:tasks.workunit.client.1.smithi169.stdout:9/978: creat d5/da/d2e/d47/f15e x:0 0 0 2024-09-16T00:26:15.616 INFO:tasks.workunit.client.1.smithi169.stdout:8/963: truncate d7/dc/d26/d41/d4e/f7c 919085 0 2024-09-16T00:26:15.618 INFO:tasks.workunit.client.1.smithi169.stdout:2/802: rename d7/d6b/d70/de0/d108 to d7/db/d20/d5c/d120 0 2024-09-16T00:26:15.622 INFO:tasks.workunit.client.1.smithi169.stdout:1/842: link dc/d26/ce3 dc/d36/d3a/d45/d29/dc2/dc7/cfc 0 2024-09-16T00:26:15.622 INFO:tasks.workunit.client.1.smithi169.stdout:9/979: dwrite d5/da/d2e/f105 [0,4194304] 0 2024-09-16T00:26:15.624 INFO:tasks.workunit.client.1.smithi169.stdout:7/896: creat d7/d71/d76/d29/d78/f124 x:0 0 0 2024-09-16T00:26:15.629 INFO:tasks.workunit.client.1.smithi169.stdout:1/843: creat dc/d36/d39/d50/daf/d88/dd1/ffd x:0 0 0 2024-09-16T00:26:15.630 INFO:tasks.workunit.client.1.smithi169.stdout:7/897: symlink d7/d71/l125 0 2024-09-16T00:26:15.631 INFO:tasks.workunit.client.1.smithi169.stdout:2/803: mknod d7/d6b/d70/d79/dce/c121 0 2024-09-16T00:26:15.632 INFO:tasks.workunit.client.1.smithi169.stdout:8/964: rename d7/dc/d26/d41/d4e/d81/d9e/da5 to d7/dc/d17/d24/d32/d8d/d13c 0 2024-09-16T00:26:15.634 INFO:tasks.workunit.client.1.smithi169.stdout:2/804: mkdir d7/d1c/d3c/d122 0 2024-09-16T00:26:15.634 INFO:tasks.workunit.client.1.smithi169.stdout:1/844: mknod dc/d36/d3a/d7f/d93/d72/cfe 0 2024-09-16T00:26:15.640 INFO:tasks.workunit.client.1.smithi169.stdout:9/980: rename d5/da/d21/de4/d140 to d5/d49/dc0/d11a/d15f 0 2024-09-16T00:26:15.640 INFO:tasks.workunit.client.1.smithi169.stdout:8/965: unlink d7/d87/dd5/ded/l12d 0 2024-09-16T00:26:15.645 INFO:tasks.workunit.client.1.smithi169.stdout:7/898: truncate d7/d5b/f77 493747 0 2024-09-16T00:26:15.646 INFO:tasks.workunit.client.1.smithi169.stdout:9/981: dwrite d5/da/d21/f45 [0,4194304] 0 2024-09-16T00:26:15.655 INFO:tasks.workunit.client.1.smithi169.stdout:7/899: dread d7/d71/d76/f37 [0,4194304] 0 2024-09-16T00:26:15.656 INFO:tasks.workunit.client.1.smithi169.stdout:9/982: dread d5/da/d2e/d47/d11b/f6a [0,4194304] 0 2024-09-16T00:26:15.659 INFO:tasks.workunit.client.1.smithi169.stdout:2/805: mkdir d7/db/d89/db4/d9d/d123 0 2024-09-16T00:26:15.662 INFO:tasks.workunit.client.1.smithi169.stdout:8/966: rename d7/dc/d26/d41/d92/dcf/d3c/d77/dde/le1 to d7/dc/d26/d41/d92/dcf/d3c/l13d 0 2024-09-16T00:26:15.663 INFO:tasks.workunit.client.1.smithi169.stdout:1/845: fdatasync dc/d36/d3a/d45/f61 0 2024-09-16T00:26:15.664 INFO:tasks.workunit.client.1.smithi169.stdout:1/846: chown dc/d36/d3a/d45/f61 11 1 2024-09-16T00:26:15.665 INFO:tasks.workunit.client.1.smithi169.stdout:2/806: unlink d7/db/d27/d29/d77/la9 0 2024-09-16T00:26:15.666 INFO:tasks.workunit.client.1.smithi169.stdout:8/967: creat d7/dc/d26/d41/d92/dcf/f13e x:0 0 0 2024-09-16T00:26:15.666 INFO:tasks.workunit.client.1.smithi169.stdout:2/807: chown d7/db/db8/d6d/df1/ffa 2517368 1 2024-09-16T00:26:15.667 INFO:tasks.workunit.client.1.smithi169.stdout:1/847: chown dc/d36/d3a/d45/d29/f5c 7983 1 2024-09-16T00:26:15.668 INFO:tasks.workunit.client.1.smithi169.stdout:1/848: stat dc/d6c/c82 0 2024-09-16T00:26:15.668 INFO:tasks.workunit.client.1.smithi169.stdout:8/968: truncate d7/dc/d26/fa7 4487945 0 2024-09-16T00:26:15.671 INFO:tasks.workunit.client.1.smithi169.stdout:8/969: mknod d7/dc/d26/d41/d4e/c13f 0 2024-09-16T00:26:15.672 INFO:tasks.workunit.client.1.smithi169.stdout:1/849: creat dc/d36/fff x:0 0 0 2024-09-16T00:26:15.673 INFO:tasks.workunit.client.1.smithi169.stdout:8/970: symlink d7/dc/d26/d41/d92/dcf/d80/dbd/l140 0 2024-09-16T00:26:15.675 INFO:tasks.workunit.client.1.smithi169.stdout:1/850: symlink dc/d36/d3a/d45/d29/l100 0 2024-09-16T00:26:15.680 INFO:tasks.workunit.client.1.smithi169.stdout:1/851: mknod dc/d36/d3a/d45/d29/dc2/c101 0 2024-09-16T00:26:15.682 INFO:tasks.workunit.client.1.smithi169.stdout:1/852: mknod dc/d36/d3a/d45/c102 0 2024-09-16T00:26:15.688 INFO:tasks.workunit.client.1.smithi169.stdout:1/853: creat dc/d36/d3a/d45/d29/dc2/f103 x:0 0 0 2024-09-16T00:26:15.688 INFO:tasks.workunit.client.1.smithi169.stdout:1/854: stat dc/d18/d67/d69/fe7 0 2024-09-16T00:26:15.688 INFO:tasks.workunit.client.1.smithi169.stdout:7/900: dread d7/d5b/f5c [0,4194304] 0 2024-09-16T00:26:15.689 INFO:tasks.workunit.client.1.smithi169.stdout:1/855: stat dc/d36/d3a/d45/fa7 0 2024-09-16T00:26:15.690 INFO:tasks.workunit.client.1.smithi169.stdout:9/983: dread d5/d49/dc0/d11a/fa1 [0,4194304] 0 2024-09-16T00:26:15.691 INFO:tasks.workunit.client.1.smithi169.stdout:9/984: readlink d5/da/d2e/ldc 0 2024-09-16T00:26:15.691 INFO:tasks.workunit.client.1.smithi169.stdout:7/901: mknod d7/d5b/d61/d98/d99/df9/dc6/c126 0 2024-09-16T00:26:15.692 INFO:tasks.workunit.client.1.smithi169.stdout:9/985: fdatasync d5/da/d5b/d91/db3/f101 0 2024-09-16T00:26:15.697 INFO:tasks.workunit.client.1.smithi169.stdout:7/902: dwrite d7/fc4 [0,4194304] 0 2024-09-16T00:26:15.699 INFO:tasks.workunit.client.1.smithi169.stdout:7/903: write d7/f91 [5228868,130393] 0 2024-09-16T00:26:15.700 INFO:tasks.workunit.client.1.smithi169.stdout:9/986: mknod d5/da/d5b/d6c/d15a/c160 0 2024-09-16T00:26:15.700 INFO:tasks.workunit.client.1.smithi169.stdout:1/856: link dc/d2c/lab dc/d36/d3a/l104 0 2024-09-16T00:26:15.701 INFO:tasks.workunit.client.1.smithi169.stdout:1/857: chown dc/ff5 27 1 2024-09-16T00:26:15.702 INFO:tasks.workunit.client.1.smithi169.stdout:1/858: fsync dc/d36/d3a/d45/f17 0 2024-09-16T00:26:15.703 INFO:tasks.workunit.client.1.smithi169.stdout:9/987: fsync d5/da/d21/f46 0 2024-09-16T00:26:15.704 INFO:tasks.workunit.client.1.smithi169.stdout:5/735: sync 2024-09-16T00:26:15.704 INFO:tasks.workunit.client.1.smithi169.stdout:7/904: rename d7/df/d95 to d7/d71/d76/d9c/ddf/d127 0 2024-09-16T00:26:15.705 INFO:tasks.workunit.client.1.smithi169.stdout:9/988: read d5/da/d2e/f7b [1717411,104691] 0 2024-09-16T00:26:15.711 INFO:tasks.workunit.client.1.smithi169.stdout:9/989: mknod d5/de0/c161 0 2024-09-16T00:26:15.712 INFO:tasks.workunit.client.1.smithi169.stdout:5/736: creat d2/da/d10/d13/d1c/d32/d53/d4e/d89/f103 x:0 0 0 2024-09-16T00:26:15.715 INFO:tasks.workunit.client.1.smithi169.stdout:5/737: truncate d2/da/d10/d13/d1c/d2d/f4f 64626 0 2024-09-16T00:26:15.716 INFO:tasks.workunit.client.1.smithi169.stdout:1/859: dread dc/d26/fa0 [0,4194304] 0 2024-09-16T00:26:15.716 INFO:tasks.workunit.client.1.smithi169.stdout:9/990: mknod d5/da/d21/de4/c162 0 2024-09-16T00:26:15.718 INFO:tasks.workunit.client.1.smithi169.stdout:5/738: write d2/da/d10/d13/d24/f47 [7607671,17660] 0 2024-09-16T00:26:15.720 INFO:tasks.workunit.client.1.smithi169.stdout:1/860: creat dc/d18/d67/d69/d7b/d92/f105 x:0 0 0 2024-09-16T00:26:15.721 INFO:tasks.workunit.client.1.smithi169.stdout:5/739: mknod d2/da/d10/d13/d1c/d32/c104 0 2024-09-16T00:26:15.722 INFO:tasks.workunit.client.1.smithi169.stdout:1/861: symlink dc/d36/d39/l106 0 2024-09-16T00:26:15.723 INFO:tasks.workunit.client.1.smithi169.stdout:1/862: chown dc/d36/d3a/d45 45 1 2024-09-16T00:26:15.726 INFO:tasks.workunit.client.1.smithi169.stdout:5/740: dread d2/da/d10/d13/d1c/d40/d43/f84 [0,4194304] 0 2024-09-16T00:26:15.726 INFO:tasks.workunit.client.1.smithi169.stdout:1/863: write dc/d36/d39/d50/daf/d86/f4f [7808497,44000] 0 2024-09-16T00:26:15.729 INFO:tasks.workunit.client.1.smithi169.stdout:1/864: chown dc/d36/d39/d50/fcb 3362 1 2024-09-16T00:26:15.732 INFO:tasks.workunit.client.1.smithi169.stdout:7/905: dwrite d7/df/fa6 [0,4194304] 0 2024-09-16T00:26:15.735 INFO:tasks.workunit.client.1.smithi169.stdout:1/865: creat dc/d36/dbb/f107 x:0 0 0 2024-09-16T00:26:15.735 INFO:tasks.workunit.client.1.smithi169.stdout:9/991: dread d5/f4a [0,4194304] 0 2024-09-16T00:26:15.738 INFO:tasks.workunit.client.1.smithi169.stdout:9/992: write d5/da/f122 [1457044,81027] 0 2024-09-16T00:26:15.743 INFO:tasks.workunit.client.1.smithi169.stdout:1/866: dread dc/d18/d67/f94 [0,4194304] 0 2024-09-16T00:26:15.745 INFO:tasks.workunit.client.1.smithi169.stdout:1/867: write dc/d36/dbb/f107 [11808,106004] 0 2024-09-16T00:26:15.751 INFO:tasks.workunit.client.1.smithi169.stdout:9/993: write d5/da/fc [954554,74028] 0 2024-09-16T00:26:15.757 INFO:tasks.workunit.client.1.smithi169.stdout:7/906: dwrite d7/d16/d49/d74/f7c [0,4194304] 0 2024-09-16T00:26:15.757 INFO:tasks.workunit.client.1.smithi169.stdout:9/994: mknod d5/da/d2e/d99/c163 0 2024-09-16T00:26:15.758 INFO:tasks.workunit.client.1.smithi169.stdout:1/868: dwrite dc/d36/d39/d50/daf/d88/da5/faa [0,4194304] 0 2024-09-16T00:26:15.759 INFO:tasks.workunit.client.1.smithi169.stdout:5/741: dread d2/da/d10/d13/d1c/d32/d53/d4e/d89/d9a/fa5 [0,4194304] 0 2024-09-16T00:26:15.760 INFO:tasks.workunit.client.1.smithi169.stdout:1/869: chown dc/d18/dd6/lf0 119 1 2024-09-16T00:26:15.762 INFO:tasks.workunit.client.1.smithi169.stdout:7/907: read d7/d5b/d61/d98/d99/f9b [626197,47283] 0 2024-09-16T00:26:15.770 INFO:tasks.workunit.client.1.smithi169.stdout:1/870: rename dc/d36/d39/d50/daf/d86/l9a to dc/d36/d39/d50/daf/l108 0 2024-09-16T00:26:15.770 INFO:tasks.workunit.client.1.smithi169.stdout:5/742: getdents d2/da/d10/d13/d1c/d9e 0 2024-09-16T00:26:15.771 INFO:tasks.workunit.client.1.smithi169.stdout:9/995: truncate d5/da/d21/de9/f125 1152119 0 2024-09-16T00:26:15.773 INFO:tasks.workunit.client.1.smithi169.stdout:5/743: rmdir d2/da/d10/d13/d1c/d40/d68 39 2024-09-16T00:26:15.775 INFO:tasks.workunit.client.1.smithi169.stdout:9/996: creat d5/da/dd0/d11f/d121/f164 x:0 0 0 2024-09-16T00:26:15.777 INFO:tasks.workunit.client.1.smithi169.stdout:5/744: chown d2/f4 3057 1 2024-09-16T00:26:15.779 INFO:tasks.workunit.client.1.smithi169.stdout:5/745: creat d2/d7c/f105 x:0 0 0 2024-09-16T00:26:15.781 INFO:tasks.workunit.client.1.smithi169.stdout:7/908: dread d7/d5b/f9a [0,4194304] 0 2024-09-16T00:26:15.781 INFO:tasks.workunit.client.1.smithi169.stdout:1/871: dwrite dc/d36/d39/d50/daf/d88/dd1/fdd [0,4194304] 0 2024-09-16T00:26:15.782 INFO:tasks.workunit.client.1.smithi169.stdout:5/746: stat d2/da/d10/d13/d1c/d32/ff8 0 2024-09-16T00:26:15.782 INFO:tasks.workunit.client.1.smithi169.stdout:9/997: creat d5/da/d2e/d99/d100/d10f/f165 x:0 0 0 2024-09-16T00:26:15.785 INFO:tasks.workunit.client.1.smithi169.stdout:1/872: write dc/d18/d67/d69/fae [4321447,93232] 0 2024-09-16T00:26:15.787 INFO:tasks.workunit.client.1.smithi169.stdout:7/909: dread d7/d5b/f87 [0,4194304] 0 2024-09-16T00:26:15.792 INFO:tasks.workunit.client.1.smithi169.stdout:7/910: dread d7/d5b/d61/d98/d99/df9/d8e/fdd [0,4194304] 0 2024-09-16T00:26:15.793 INFO:tasks.workunit.client.1.smithi169.stdout:1/873: mknod dc/d36/d3a/d45/dce/c109 0 2024-09-16T00:26:15.795 INFO:tasks.workunit.client.1.smithi169.stdout:1/874: fdatasync dc/d18/d67/d69/f80 0 2024-09-16T00:26:15.796 INFO:tasks.workunit.client.1.smithi169.stdout:7/911: getdents d7/d16 0 2024-09-16T00:26:15.797 INFO:tasks.workunit.client.1.smithi169.stdout:1/875: read dc/d36/d3a/f78 [758982,120489] 0 2024-09-16T00:26:15.806 INFO:tasks.workunit.client.1.smithi169.stdout:1/876: write dc/d36/d39/d50/daf/d88/da5/faa [1564306,82509] 0 2024-09-16T00:26:15.809 INFO:tasks.workunit.client.1.smithi169.stdout:9/998: dwrite d5/da/d21/de4/f96 [4194304,4194304] 0 2024-09-16T00:26:15.810 INFO:tasks.workunit.client.1.smithi169.stdout:9/999: write d5/f8f [3465050,19647] 0 2024-09-16T00:26:15.811 INFO:tasks.workunit.client.1.smithi169.stdout:7/912: truncate d7/d71/d76/d9c/fa9 3444940 0 2024-09-16T00:26:15.818 INFO:tasks.workunit.client.1.smithi169.stdout:7/913: creat d7/d5b/d61/d98/f128 x:0 0 0 2024-09-16T00:26:15.818 INFO:tasks.workunit.client.1.smithi169.stdout:7/914: stat d7/d16/f25 0 2024-09-16T00:26:15.821 INFO:tasks.workunit.client.1.smithi169.stdout:1/877: dwrite dc/d18/d67/d69/fe7 [0,4194304] 0 2024-09-16T00:26:15.821 INFO:tasks.workunit.client.1.smithi169.stdout:1/878: stat dc/d6c/c82 0 2024-09-16T00:26:15.822 INFO:tasks.workunit.client.1.smithi169.stdout:7/915: rmdir d7/d71/d76/d9c/ddf/d127/dee 0 2024-09-16T00:26:15.824 INFO:tasks.workunit.client.1.smithi169.stdout:7/916: write d7/d5b/d61/d98/d99/df9/d8e/fdc [2056554,117745] 0 2024-09-16T00:26:15.826 INFO:tasks.workunit.client.1.smithi169.stdout:7/917: readlink d7/d71/d76/d29/l30 0 2024-09-16T00:26:15.827 INFO:tasks.workunit.client.1.smithi169.stdout:7/918: stat d7/d16/d49/d74/f7c 0 2024-09-16T00:26:15.829 INFO:tasks.workunit.client.1.smithi169.stdout:1/879: dwrite dc/d36/d3a/d45/d29/d4d/db6/ffb [0,4194304] 0 2024-09-16T00:26:15.830 INFO:tasks.workunit.client.1.smithi169.stdout:7/919: mkdir d7/d71/d76/d129 0 2024-09-16T00:26:15.833 INFO:tasks.workunit.client.1.smithi169.stdout:7/920: symlink d7/d16/d49/db2/l12a 0 2024-09-16T00:26:15.834 INFO:tasks.workunit.client.1.smithi169.stdout:1/880: mkdir dc/d36/d39/d50/daf/d88/dd1/d10a 0 2024-09-16T00:26:15.834 INFO:tasks.workunit.client.1.smithi169.stdout:7/921: chown d7/fa0 11662512 1 2024-09-16T00:26:15.838 INFO:tasks.workunit.client.1.smithi169.stdout:1/881: rename dc/d36/d3a/d45/d29/d4d/fbe to dc/d36/d39/d50/daf/d88/f10b 0 2024-09-16T00:26:15.845 INFO:tasks.workunit.client.1.smithi169.stdout:1/882: write dc/d36/d39/d50/daf/f3f [5773014,40149] 0 2024-09-16T00:26:15.859 INFO:tasks.workunit.client.1.smithi169.stdout:7/922: dread d7/d5b/d61/d98/d99/df9/d8e/fdc [0,4194304] 0 2024-09-16T00:26:15.860 INFO:tasks.workunit.client.1.smithi169.stdout:7/923: rmdir d7/d5b/d61/d98/d99/df9 39 2024-09-16T00:26:15.861 INFO:tasks.workunit.client.1.smithi169.stdout:7/924: chown d7/cf0 117749 1 2024-09-16T00:26:15.862 INFO:tasks.workunit.client.1.smithi169.stdout:7/925: dread - d7/d71/fe8 zero size 2024-09-16T00:26:15.865 INFO:tasks.workunit.client.1.smithi169.stdout:7/926: dread d7/d71/d76/d29/fa1 [0,4194304] 0 2024-09-16T00:26:15.873 INFO:tasks.workunit.client.1.smithi169.stdout:7/927: dwrite d7/d71/d76/d29/f45 [4194304,4194304] 0 2024-09-16T00:26:15.877 INFO:tasks.workunit.client.1.smithi169.stdout:7/928: mknod d7/d16/d49/db2/db4/d11d/c12b 0 2024-09-16T00:26:15.878 INFO:tasks.workunit.client.1.smithi169.stdout:7/929: dread d7/d71/d76/f37 [0,4194304] 0 2024-09-16T00:26:15.879 INFO:tasks.workunit.client.1.smithi169.stdout:7/930: chown d7/d71/l125 3729 1 2024-09-16T00:26:15.887 INFO:tasks.workunit.client.1.smithi169.stdout:7/931: dwrite d7/d5b/d61/d98/d99/fa8 [0,4194304] 0 2024-09-16T00:26:15.888 INFO:tasks.workunit.client.1.smithi169.stdout:7/932: chown d7/d71/d76/d9c 9 1 2024-09-16T00:26:16.011 INFO:tasks.workunit.client.1.smithi169.stdout:4/803: sync 2024-09-16T00:26:16.012 INFO:tasks.workunit.client.1.smithi169.stdout:0/820: sync 2024-09-16T00:26:16.012 INFO:tasks.workunit.client.1.smithi169.stdout:4/804: fsync d2/fdf 0 2024-09-16T00:26:16.013 INFO:tasks.workunit.client.1.smithi169.stdout:0/821: creat d0/d10d/d119/f11a x:0 0 0 2024-09-16T00:26:16.014 INFO:tasks.workunit.client.1.smithi169.stdout:0/822: dread - d0/d2/d4b/ff9 zero size 2024-09-16T00:26:16.015 INFO:tasks.workunit.client.1.smithi169.stdout:0/823: stat d0/d2/d4b/f52 0 2024-09-16T00:26:16.016 INFO:tasks.workunit.client.1.smithi169.stdout:4/805: rename d2/d12/d13/d1f/d3e/dde to d2/d12/d13/d1f/d3e/dfb 0 2024-09-16T00:26:16.017 INFO:tasks.workunit.client.1.smithi169.stdout:0/824: write d0/d2c/d38/de4/f1c [46491,94158] 0 2024-09-16T00:26:16.018 INFO:tasks.workunit.client.1.smithi169.stdout:4/806: write d2/d12/d13/d4e/d50/fce [955254,17916] 0 2024-09-16T00:26:16.020 INFO:tasks.workunit.client.1.smithi169.stdout:4/807: truncate d2/d12/d9c/da4/ff0 213202 0 2024-09-16T00:26:16.022 INFO:tasks.workunit.client.1.smithi169.stdout:4/808: symlink d2/d12/d13/d5c/da6/dd3/lfc 0 2024-09-16T00:26:16.024 INFO:tasks.workunit.client.1.smithi169.stdout:4/809: mkdir d2/d12/d13/d27/dfd 0 2024-09-16T00:26:16.025 INFO:tasks.workunit.client.1.smithi169.stdout:4/810: symlink d2/lfe 0 2024-09-16T00:26:16.027 INFO:tasks.workunit.client.1.smithi169.stdout:4/811: creat d2/d12/d9c/fff x:0 0 0 2024-09-16T00:26:16.030 INFO:tasks.workunit.client.1.smithi169.stdout:4/812: read d2/d12/d13/d1f/d2f/f43 [2772189,68445] 0 2024-09-16T00:26:16.034 INFO:tasks.workunit.client.1.smithi169.stdout:0/825: dread d0/d2c/d38/de4/d21/d31/d83/f8a [0,4194304] 0 2024-09-16T00:26:16.036 INFO:tasks.workunit.client.1.smithi169.stdout:4/813: dwrite d2/d12/d13/d1f/d3e/d5b/f73 [0,4194304] 0 2024-09-16T00:26:16.038 INFO:tasks.workunit.client.1.smithi169.stdout:0/826: creat d0/d2/d8/d5/d79/d7c/f11b x:0 0 0 2024-09-16T00:26:16.041 INFO:tasks.workunit.client.1.smithi169.stdout:4/814: truncate d2/d12/d13/d1f/d3e/d5b/d70/f78 1047796 0 2024-09-16T00:26:16.042 INFO:tasks.workunit.client.1.smithi169.stdout:0/827: mkdir d0/d7d/d11c 0 2024-09-16T00:26:16.043 INFO:tasks.workunit.client.1.smithi169.stdout:4/815: creat d2/d12/d13/d4e/d50/f100 x:0 0 0 2024-09-16T00:26:16.045 INFO:tasks.workunit.client.1.smithi169.stdout:4/816: truncate d2/d12/d13/f8f 438806 0 2024-09-16T00:26:16.060 INFO:tasks.workunit.client.1.smithi169.stdout:4/817: write d2/d12/dcc/f47 [241242,50894] 0 2024-09-16T00:26:16.062 INFO:tasks.workunit.client.1.smithi169.stdout:4/818: rmdir d2/d12/d13/d4e/d50/df2 39 2024-09-16T00:26:16.064 INFO:tasks.workunit.client.1.smithi169.stdout:0/828: dwrite d0/d2/d15/fc1 [0,4194304] 0 2024-09-16T00:26:16.068 INFO:tasks.workunit.client.1.smithi169.stdout:0/829: mkdir d0/d2c/d38/de4/d11d 0 2024-09-16T00:26:16.069 INFO:tasks.workunit.client.1.smithi169.stdout:0/830: readlink d0/d7d/lfb 0 2024-09-16T00:26:16.071 INFO:tasks.workunit.client.1.smithi169.stdout:0/831: mknod d0/d2c/d38/de4/d11d/c11e 0 2024-09-16T00:26:16.088 INFO:tasks.workunit.client.1.smithi169.stdout:0/832: dread d0/d2/d8/fe [0,4194304] 0 2024-09-16T00:26:16.090 INFO:tasks.workunit.client.1.smithi169.stdout:0/833: fdatasync d0/d4c/d82/d10f/f9f 0 2024-09-16T00:26:16.091 INFO:tasks.workunit.client.1.smithi169.stdout:0/834: read d0/d2/d8/d5/ffa [155215,89083] 0 2024-09-16T00:26:16.093 INFO:tasks.workunit.client.1.smithi169.stdout:0/835: mknod d0/d2/d8/d5/c11f 0 2024-09-16T00:26:16.094 INFO:tasks.workunit.client.1.smithi169.stdout:0/836: mknod d0/d2c/d103/c120 0 2024-09-16T00:26:16.210 INFO:tasks.workunit.client.1.smithi169.stdout:3/988: sync 2024-09-16T00:26:16.214 INFO:tasks.workunit.client.1.smithi169.stdout:3/989: dwrite d7/f14 [0,4194304] 0 2024-09-16T00:26:16.215 INFO:tasks.workunit.client.1.smithi169.stdout:3/990: stat d7/d1a/d1c/d23/cbd 0 2024-09-16T00:26:16.219 INFO:tasks.workunit.client.1.smithi169.stdout:3/991: rmdir d7/d50/d90/d13a/dc6/dff 39 2024-09-16T00:26:16.220 INFO:tasks.workunit.client.1.smithi169.stdout:3/992: dread - d7/d1a/d1c/d23/d88/d9b/de2/f14d zero size 2024-09-16T00:26:16.222 INFO:tasks.workunit.client.1.smithi169.stdout:3/993: rmdir d7/d1a/d1c/d35 39 2024-09-16T00:26:16.225 INFO:tasks.workunit.client.1.smithi169.stdout:3/994: creat d7/f15b x:0 0 0 2024-09-16T00:26:16.227 INFO:tasks.workunit.client.1.smithi169.stdout:3/995: symlink d7/d50/l15c 0 2024-09-16T00:26:16.228 INFO:tasks.workunit.client.1.smithi169.stdout:3/996: creat d7/d1a/d1c/d23/d88/dd3/f15d x:0 0 0 2024-09-16T00:26:16.241 INFO:tasks.workunit.client.1.smithi169.stdout:3/997: dwrite d7/fd2 [0,4194304] 0 2024-09-16T00:26:16.241 INFO:tasks.workunit.client.1.smithi169.stdout:3/998: fsync d7/d1a/d1c/f78 0 2024-09-16T00:26:16.243 INFO:tasks.workunit.client.1.smithi169.stdout:3/999: symlink d7/d1a/d1c/d23/d88/d9b/l15e 0 2024-09-16T00:26:16.353 INFO:tasks.workunit.client.1.smithi169.stdout:2/808: sync 2024-09-16T00:26:16.354 INFO:tasks.workunit.client.1.smithi169.stdout:2/809: dread - d7/db/d89/db4/d69/f99 zero size 2024-09-16T00:26:16.356 INFO:tasks.workunit.client.1.smithi169.stdout:2/810: mknod d7/db/db8/daf/d10a/d11b/c124 0 2024-09-16T00:26:16.424 INFO:tasks.workunit.client.1.smithi169.stdout:5/747: sync 2024-09-16T00:26:16.424 INFO:tasks.workunit.client.1.smithi169.stdout:8/971: sync 2024-09-16T00:26:16.424 INFO:tasks.workunit.client.1.smithi169.stdout:1/883: sync 2024-09-16T00:26:16.424 INFO:tasks.workunit.client.1.smithi169.stdout:1/884: chown dc/d26/l58 9 1 2024-09-16T00:26:16.425 INFO:tasks.workunit.client.1.smithi169.stdout:5/748: write d2/da/d10/d13/d24/fa0 [4599631,14946] 0 2024-09-16T00:26:16.427 INFO:tasks.workunit.client.1.smithi169.stdout:5/749: dread - d2/da/d10/d13/d1c/d2d/f5f zero size 2024-09-16T00:26:16.430 INFO:tasks.workunit.client.1.smithi169.stdout:1/885: rename dc/d36/d3a/d45/d29/f52 to dc/d36/d3a/d7f/d93/f10c 0 2024-09-16T00:26:16.436 INFO:tasks.workunit.client.1.smithi169.stdout:5/750: rename d2/da/d10/fd5 to d2/da/d10/d13/d24/de9/ddf/f106 0 2024-09-16T00:26:16.438 INFO:tasks.workunit.client.1.smithi169.stdout:1/886: rename dc/d26/l58 to dc/d36/d3a/d7f/d93/l10d 0 2024-09-16T00:26:16.439 INFO:tasks.workunit.client.1.smithi169.stdout:1/887: write dc/d18/d67/d69/d7b/d92/fd2 [234055,56489] 0 2024-09-16T00:26:16.441 INFO:tasks.workunit.client.1.smithi169.stdout:5/751: rename d2/da/d10/d13/d1c/d94/db4/fb6 to d2/da/d10/d13/d24/f107 0 2024-09-16T00:26:16.442 INFO:tasks.workunit.client.1.smithi169.stdout:8/972: dread d7/fa [4194304,4194304] 0 2024-09-16T00:26:16.443 INFO:tasks.workunit.client.1.smithi169.stdout:1/888: unlink dc/d36/d3a/d7f/d93/d72/cfe 0 2024-09-16T00:26:16.446 INFO:tasks.workunit.client.1.smithi169.stdout:5/752: creat d2/da/d10/d13/d1c/d32/d53/dae/de2/f108 x:0 0 0 2024-09-16T00:26:16.448 INFO:tasks.workunit.client.1.smithi169.stdout:8/973: mkdir d7/dc/d17/d24/d32/d8d/d141 0 2024-09-16T00:26:16.450 INFO:tasks.workunit.client.1.smithi169.stdout:5/753: symlink d2/da/l109 0 2024-09-16T00:26:16.451 INFO:tasks.workunit.client.1.smithi169.stdout:8/974: write d7/dc/d26/fa7 [3267048,65927] 0 2024-09-16T00:26:16.451 INFO:tasks.workunit.client.1.smithi169.stdout:1/889: dwrite dc/d26/fa0 [0,4194304] 0 2024-09-16T00:26:16.459 INFO:tasks.workunit.client.1.smithi169.stdout:7/933: sync 2024-09-16T00:26:16.459 INFO:tasks.workunit.client.1.smithi169.stdout:4/819: sync 2024-09-16T00:26:16.459 INFO:tasks.workunit.client.1.smithi169.stdout:0/837: sync 2024-09-16T00:26:16.459 INFO:tasks.workunit.client.1.smithi169.stdout:2/811: sync 2024-09-16T00:26:16.459 INFO:tasks.workunit.client.1.smithi169.stdout:8/975: mkdir d7/dc/d26/d41/d92/dcf/d3c/d82/db3/db4/d142 0 2024-09-16T00:26:16.464 INFO:tasks.workunit.client.1.smithi169.stdout:2/812: fsync d7/db/d20/d5c/db2/fcd 0 2024-09-16T00:26:16.464 INFO:tasks.workunit.client.1.smithi169.stdout:7/934: symlink d7/d16/d49/d121/l12c 0 2024-09-16T00:26:16.466 INFO:tasks.workunit.client.1.smithi169.stdout:4/820: mknod d2/d12/d13/d1f/d3e/d5b/c101 0 2024-09-16T00:26:16.466 INFO:tasks.workunit.client.1.smithi169.stdout:7/935: chown d7/d71/d76/d29/f85 2102721538 1 2024-09-16T00:26:16.468 INFO:tasks.workunit.client.1.smithi169.stdout:2/813: rename d7/f11 to d7/db/db8/daf/d10a/d11b/f125 0 2024-09-16T00:26:16.468 INFO:tasks.workunit.client.1.smithi169.stdout:7/936: rmdir d7/d5b/dc1 39 2024-09-16T00:26:16.469 INFO:tasks.workunit.client.1.smithi169.stdout:0/838: read d0/d4c/d82/d10f/f10e [186641,117437] 0 2024-09-16T00:26:16.469 INFO:tasks.workunit.client.1.smithi169.stdout:4/821: mkdir d2/d12/d13/d1f/d3e/d5b/d70/d7d/d102 0 2024-09-16T00:26:16.476 INFO:tasks.workunit.client.1.smithi169.stdout:4/822: rename d2/d12/d13/d90/fae to d2/dbb/d59/f103 0 2024-09-16T00:26:16.476 INFO:tasks.workunit.client.1.smithi169.stdout:8/976: dread d7/dc/d26/fbc [0,4194304] 0 2024-09-16T00:26:16.477 INFO:tasks.workunit.client.1.smithi169.stdout:4/823: chown d2/d12/d13/d1f/d3e/dfb 80969936 1 2024-09-16T00:26:16.479 INFO:tasks.workunit.client.1.smithi169.stdout:8/977: mkdir d7/d37/d143 0 2024-09-16T00:26:16.481 INFO:tasks.workunit.client.1.smithi169.stdout:4/824: mkdir d2/d12/d13/d5c/da6/d104 0 2024-09-16T00:26:16.483 INFO:tasks.workunit.client.1.smithi169.stdout:7/937: dread d7/d5b/d61/d98/d99/f9b [0,4194304] 0 2024-09-16T00:26:16.484 INFO:tasks.workunit.client.1.smithi169.stdout:8/978: mknod d7/dc/c144 0 2024-09-16T00:26:16.485 INFO:tasks.workunit.client.1.smithi169.stdout:7/938: dread - d7/d71/d76/d29/d78/dba/fcb zero size 2024-09-16T00:26:16.486 INFO:tasks.workunit.client.1.smithi169.stdout:4/825: symlink d2/d12/d13/d1f/d3e/d5b/d70/def/l105 0 2024-09-16T00:26:16.490 INFO:tasks.workunit.client.1.smithi169.stdout:7/939: rename d7/d71/d76/d9c/ddf to d7/d12d 0 2024-09-16T00:26:16.491 INFO:tasks.workunit.client.1.smithi169.stdout:4/826: truncate d2/dbb/f84 1243533 0 2024-09-16T00:26:16.495 INFO:tasks.workunit.client.1.smithi169.stdout:4/827: readlink d2/d12/d9c/de2/lf7 0 2024-09-16T00:26:16.495 INFO:tasks.workunit.client.1.smithi169.stdout:5/754: dwrite d2/fb [0,4194304] 0 2024-09-16T00:26:16.499 INFO:tasks.workunit.client.1.smithi169.stdout:0/839: write d0/d2c/d38/de4/d21/d31/fbc [500156,113496] 0 2024-09-16T00:26:16.502 INFO:tasks.workunit.client.1.smithi169.stdout:4/828: dread - d2/d12/d13/d4e/d50/f100 zero size 2024-09-16T00:26:16.502 INFO:tasks.workunit.client.1.smithi169.stdout:1/890: dwrite dc/d36/d3a/d45/d29/fb4 [0,4194304] 0 2024-09-16T00:26:16.503 INFO:tasks.workunit.client.1.smithi169.stdout:5/755: write d2/da/d10/d13/d1c/d32/d53/dae/ffd [740289,37869] 0 2024-09-16T00:26:16.505 INFO:tasks.workunit.client.1.smithi169.stdout:7/940: rename d7/d16/d49/d74/f7c to d7/d5b/d61/d98/d99/df9/dc6/f12e 0 2024-09-16T00:26:16.512 INFO:tasks.workunit.client.1.smithi169.stdout:2/814: dwrite d7/db/d89/fbf [0,4194304] 0 2024-09-16T00:26:16.514 INFO:tasks.workunit.client.1.smithi169.stdout:4/829: link d2/d12/d9c/fec d2/d12/d13/d1f/d3e/d5b/d70/d7d/d102/f106 0 2024-09-16T00:26:16.515 INFO:tasks.workunit.client.1.smithi169.stdout:1/891: write dc/d36/d3a/d7f/d93/f6d [966850,23794] 0 2024-09-16T00:26:16.516 INFO:tasks.workunit.client.1.smithi169.stdout:7/941: creat d7/d16/d49/d74/f12f x:0 0 0 2024-09-16T00:26:16.517 INFO:tasks.workunit.client.1.smithi169.stdout:0/840: creat d0/d2/d8/d5/d79/d113/f121 x:0 0 0 2024-09-16T00:26:16.519 INFO:tasks.workunit.client.1.smithi169.stdout:0/841: chown d0/d2/f93 6052 1 2024-09-16T00:26:16.520 INFO:tasks.workunit.client.1.smithi169.stdout:4/830: unlink d2/d12/d13/d1f/d2f/c4c 0 2024-09-16T00:26:16.521 INFO:tasks.workunit.client.1.smithi169.stdout:0/842: fdatasync d0/d2c/d4f/da5/fb0 0 2024-09-16T00:26:16.521 INFO:tasks.workunit.client.1.smithi169.stdout:4/831: chown d2/d12/d13/d4e/d50 134 1 2024-09-16T00:26:16.524 INFO:tasks.workunit.client.1.smithi169.stdout:1/892: getdents dc/d36 0 2024-09-16T00:26:16.526 INFO:tasks.workunit.client.1.smithi169.stdout:7/942: dread d7/df/f44 [0,4194304] 0 2024-09-16T00:26:16.528 INFO:tasks.workunit.client.1.smithi169.stdout:1/893: fsync dc/f34 0 2024-09-16T00:26:16.530 INFO:tasks.workunit.client.1.smithi169.stdout:2/815: write d7/d6b/d70/d79/f9f [934666,3473] 0 2024-09-16T00:26:16.532 INFO:tasks.workunit.client.1.smithi169.stdout:5/756: dwrite d2/da/d10/f11 [0,4194304] 0 2024-09-16T00:26:16.533 INFO:tasks.workunit.client.1.smithi169.stdout:7/943: write d7/d5b/d61/d98/d99/df9/d8e/f113 [1203274,72668] 0 2024-09-16T00:26:16.534 INFO:tasks.workunit.client.1.smithi169.stdout:1/894: unlink dc/d2c/f32 0 2024-09-16T00:26:16.534 INFO:tasks.workunit.client.1.smithi169.stdout:7/944: chown d7/d71/l108 2 1 2024-09-16T00:26:16.537 INFO:tasks.workunit.client.1.smithi169.stdout:0/843: rename d0/d4c/d82/d10f/d6c/d9b/ce2 to d0/c122 0 2024-09-16T00:26:16.537 INFO:tasks.workunit.client.1.smithi169.stdout:5/757: dread - d2/da/d10/d13/d1c/d32/d53/d4e/d89/f103 zero size 2024-09-16T00:26:16.540 INFO:tasks.workunit.client.1.smithi169.stdout:1/895: mkdir dc/d18/d67/d69/d10e 0 2024-09-16T00:26:16.542 INFO:tasks.workunit.client.1.smithi169.stdout:5/758: mknod d2/da/d10/d13/d1c/d32/d53/c10a 0 2024-09-16T00:26:16.542 INFO:tasks.workunit.client.1.smithi169.stdout:1/896: chown dc/d36/d39/d50/d75/dfa 67444 1 2024-09-16T00:26:16.543 INFO:tasks.workunit.client.1.smithi169.stdout:5/759: fsync d2/da/f19 0 2024-09-16T00:26:16.544 INFO:tasks.workunit.client.1.smithi169.stdout:4/832: dread d2/d12/d13/fb9 [0,4194304] 0 2024-09-16T00:26:16.656 INFO:tasks.workunit.client.1.smithi169.stdout:1/897: mkdir dc/d36/d3a/d45/d29/d4d/db6/d10f 0 2024-09-16T00:26:16.657 INFO:tasks.workunit.client.1.smithi169.stdout:7/945: write d7/d16/d49/f11c [4133738,121148] 0 2024-09-16T00:26:16.665 INFO:tasks.workunit.client.1.smithi169.stdout:2/816: dread d7/db/db8/daf/d10a/d11b/f125 [0,4194304] 0 2024-09-16T00:26:16.673 INFO:tasks.workunit.client.1.smithi169.stdout:4/833: mknod d2/d12/d13/d1f/d3e/c107 0 2024-09-16T00:26:16.673 INFO:tasks.workunit.client.1.smithi169.stdout:1/898: dread dc/d36/d39/d50/f7a [0,4194304] 0 2024-09-16T00:26:16.673 INFO:tasks.workunit.client.1.smithi169.stdout:7/946: mknod d7/d5b/d61/c130 0 2024-09-16T00:26:16.675 INFO:tasks.workunit.client.1.smithi169.stdout:2/817: fsync f0 0 2024-09-16T00:26:16.677 INFO:tasks.workunit.client.1.smithi169.stdout:5/760: write d2/da/d10/d13/d1c/d32/d53/d4e/d89/d9a/dad/fc6 [499783,35074] 0 2024-09-16T00:26:16.679 INFO:tasks.workunit.client.1.smithi169.stdout:4/834: mkdir d2/d12/d13/d4e/d50/db8/d108 0 2024-09-16T00:26:16.680 INFO:tasks.workunit.client.1.smithi169.stdout:2/818: mkdir d7/d6b/d70/d79/dae/d126 0 2024-09-16T00:26:16.682 INFO:tasks.workunit.client.1.smithi169.stdout:7/947: dwrite d7/df/f10b [0,4194304] 0 2024-09-16T00:26:16.682 INFO:tasks.workunit.client.1.smithi169.stdout:5/761: fdatasync d2/da/d10/d13/d24/f107 0 2024-09-16T00:26:16.684 INFO:tasks.workunit.client.1.smithi169.stdout:4/835: rmdir d2/d12/d13/d5c/da6 39 2024-09-16T00:26:16.686 INFO:tasks.workunit.client.1.smithi169.stdout:5/762: write d2/da/d10/d13/d1c/d32/d53/dae/ffd [1626351,21124] 0 2024-09-16T00:26:16.688 INFO:tasks.workunit.client.1.smithi169.stdout:5/763: chown d2/da/d10/d13/d1c/d2d/db2/de7/df4/ffa 38166653 1 2024-09-16T00:26:16.689 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:16 smithi169 ceph-mon[34653]: pgmap v261: 65 pgs: 65 active+clean; 3.7 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 137 MiB/s rd, 317 MiB/s wr, 631 op/s 2024-09-16T00:26:16.693 INFO:tasks.workunit.client.1.smithi169.stdout:4/836: mkdir d2/d12/d13/d5c/d109 0 2024-09-16T00:26:16.694 INFO:tasks.workunit.client.1.smithi169.stdout:2/819: creat d7/db/d27/d29/f127 x:0 0 0 2024-09-16T00:26:16.695 INFO:tasks.workunit.client.1.smithi169.stdout:2/820: read d7/d1c/fb7 [4103492,21685] 0 2024-09-16T00:26:16.696 INFO:tasks.workunit.client.1.smithi169.stdout:2/821: read - d7/db/d20/d5c/db2/fcd zero size 2024-09-16T00:26:16.696 INFO:tasks.workunit.client.1.smithi169.stdout:5/764: rename d2/da/d10/d13/d1c/d32/d53/d4e/d89/d9a/dad/cea to d2/da/d10/d13/d1c/d94/c10b 0 2024-09-16T00:26:16.698 INFO:tasks.workunit.client.1.smithi169.stdout:4/837: fdatasync d2/d12/d13/f6c 0 2024-09-16T00:26:16.698 INFO:tasks.workunit.client.1.smithi169.stdout:5/765: chown d2/da/d10/d13/d1c/d2d/db2/de7/df4/ffa 729 1 2024-09-16T00:26:16.701 INFO:tasks.workunit.client.1.smithi169.stdout:2/822: symlink d7/db/d89/db4/d69/l128 0 2024-09-16T00:26:16.706 INFO:tasks.workunit.client.1.smithi169.stdout:2/823: symlink d7/db/d20/d5c/db2/l129 0 2024-09-16T00:26:16.708 INFO:tasks.workunit.client.1.smithi169.stdout:2/824: rmdir d7/db/db8 39 2024-09-16T00:26:16.712 INFO:tasks.workunit.client.1.smithi169.stdout:2/825: mkdir d7/db/d89/db4/d69/d92/d12a 0 2024-09-16T00:26:16.712 INFO:tasks.workunit.client.1.smithi169.stdout:2/826: mkdir d7/db/d20/d12b 0 2024-09-16T00:26:16.717 INFO:tasks.workunit.client.1.smithi169.stdout:1/899: write dc/d26/f63 [1161614,81806] 0 2024-09-16T00:26:16.719 INFO:tasks.workunit.client.1.smithi169.stdout:1/900: readlink dc/d2c/l7c 0 2024-09-16T00:26:16.719 INFO:tasks.workunit.client.1.smithi169.stdout:7/948: truncate d7/df/f5d 2309184 0 2024-09-16T00:26:16.720 INFO:tasks.workunit.client.1.smithi169.stdout:7/949: write d7/f91 [2543467,3382] 0 2024-09-16T00:26:16.727 INFO:tasks.workunit.client.1.smithi169.stdout:7/950: chown d7/d5b/dc1/d96/lf4 38323452 1 2024-09-16T00:26:16.728 INFO:tasks.workunit.client.1.smithi169.stdout:5/766: write d2/da/d10/d13/d24/fa7 [3882188,59006] 0 2024-09-16T00:26:16.734 INFO:tasks.workunit.client.1.smithi169.stdout:4/838: dwrite d2/d12/d13/f3a [0,4194304] 0 2024-09-16T00:26:16.734 INFO:tasks.workunit.client.1.smithi169.stdout:2/827: write d7/d1c/d26/f36 [3029955,115043] 0 2024-09-16T00:26:16.736 INFO:tasks.workunit.client.1.smithi169.stdout:2/828: chown d7/d1c/fb7 1 1 2024-09-16T00:26:16.739 INFO:tasks.workunit.client.1.smithi169.stdout:1/901: truncate dc/d26/fa0 2568906 0 2024-09-16T00:26:16.740 INFO:tasks.workunit.client.1.smithi169.stdout:5/767: mknod d2/da/d10/d13/d1c/d71/c10c 0 2024-09-16T00:26:16.741 INFO:tasks.workunit.client.1.smithi169.stdout:7/951: write d7/d16/fda [852649,129206] 0 2024-09-16T00:26:16.749 INFO:tasks.workunit.client.1.smithi169.stdout:2/829: dread d7/daa/f5f [0,4194304] 0 2024-09-16T00:26:16.792 INFO:tasks.workunit.client.1.smithi169.stdout:8/979: sync 2024-09-16T00:26:16.794 INFO:tasks.workunit.client.1.smithi169.stdout:1/902: fsync dc/d36/d3a/d45/fa7 0 2024-09-16T00:26:16.794 INFO:tasks.workunit.client.1.smithi169.stdout:8/980: dread - d7/dc/d17/d24/d32/d105/f109 zero size 2024-09-16T00:26:16.795 INFO:tasks.workunit.client.1.smithi169.stdout:8/981: read d7/fd [2849670,115851] 0 2024-09-16T00:26:16.795 INFO:tasks.workunit.client.1.smithi169.stdout:5/768: fdatasync d2/da/fb8 0 2024-09-16T00:26:16.795 INFO:tasks.workunit.client.1.smithi169.stdout:8/982: chown d7/fd 629 1 2024-09-16T00:26:16.798 INFO:tasks.workunit.client.1.smithi169.stdout:5/769: truncate d2/da/d10/d13/d24/f107 1152242 0 2024-09-16T00:26:16.803 INFO:tasks.workunit.client.1.smithi169.stdout:7/952: getdents d7/d16 0 2024-09-16T00:26:16.805 INFO:tasks.workunit.client.1.smithi169.stdout:2/830: creat d7/db/db8/daf/d10a/d11b/d114/f12c x:0 0 0 2024-09-16T00:26:16.808 INFO:tasks.workunit.client.1.smithi169.stdout:7/953: creat d7/d16/d49/d121/f131 x:0 0 0 2024-09-16T00:26:16.809 INFO:tasks.workunit.client.1.smithi169.stdout:2/831: symlink d7/db/db8/d6d/df1/l12d 0 2024-09-16T00:26:16.810 INFO:tasks.workunit.client.1.smithi169.stdout:8/983: dread d7/dc/d17/d24/f4b [0,4194304] 0 2024-09-16T00:26:16.813 INFO:tasks.workunit.client.1.smithi169.stdout:2/832: dread - d7/db/d27/d29/f127 zero size 2024-09-16T00:26:16.815 INFO:tasks.workunit.client.1.smithi169.stdout:5/770: dwrite d2/da/fb8 [0,4194304] 0 2024-09-16T00:26:16.816 INFO:tasks.workunit.client.1.smithi169.stdout:2/833: dread - d7/db/d89/dc2/fd4 zero size 2024-09-16T00:26:16.818 INFO:tasks.workunit.client.1.smithi169.stdout:8/984: symlink d7/dc/d26/d41/d92/dcf/d80/dbd/l145 0 2024-09-16T00:26:16.820 INFO:tasks.workunit.client.1.smithi169.stdout:1/903: dwrite dc/d6c/fda [0,4194304] 0 2024-09-16T00:26:16.821 INFO:tasks.workunit.client.1.smithi169.stdout:2/834: mknod d7/db/db8/daf/c12e 0 2024-09-16T00:26:16.823 INFO:tasks.workunit.client.1.smithi169.stdout:8/985: rmdir d7/dc/d17/d24/d32/d4d/d111 39 2024-09-16T00:26:16.827 INFO:tasks.workunit.client.1.smithi169.stdout:2/835: rmdir d7/db/d27/d29 39 2024-09-16T00:26:16.829 INFO:tasks.workunit.client.1.smithi169.stdout:5/771: dread d2/da/f19 [0,4194304] 0 2024-09-16T00:26:16.831 INFO:tasks.workunit.client.1.smithi169.stdout:1/904: creat dc/d36/d3a/d45/d29/d4d/db6/d10f/f110 x:0 0 0 2024-09-16T00:26:16.832 INFO:tasks.workunit.client.1.smithi169.stdout:1/905: read dc/d36/d3a/d45/d29/fb4 [1127541,20730] 0 2024-09-16T00:26:16.835 INFO:tasks.workunit.client.1.smithi169.stdout:2/836: mknod d7/db/d89/dfb/c12f 0 2024-09-16T00:26:16.838 INFO:tasks.workunit.client.1.smithi169.stdout:8/986: dread f5 [0,4194304] 0 2024-09-16T00:26:16.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:16 smithi112 ceph-mon[27869]: pgmap v261: 65 pgs: 65 active+clean; 3.7 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 137 MiB/s rd, 317 MiB/s wr, 631 op/s 2024-09-16T00:26:16.840 INFO:tasks.workunit.client.1.smithi169.stdout:7/954: dread d7/d16/f90 [0,4194304] 0 2024-09-16T00:26:16.841 INFO:tasks.workunit.client.1.smithi169.stdout:2/837: creat d7/db/db8/d6d/f130 x:0 0 0 2024-09-16T00:26:16.843 INFO:tasks.workunit.client.1.smithi169.stdout:5/772: getdents d2/da/d10/d13/d1c/d32/d53/d83/d7d 0 2024-09-16T00:26:16.843 INFO:tasks.workunit.client.1.smithi169.stdout:1/906: dwrite dc/d26/f37 [0,4194304] 0 2024-09-16T00:26:16.853 INFO:tasks.workunit.client.1.smithi169.stdout:8/987: dwrite d7/dc/d26/d41/d4e/d81/f11d [0,4194304] 0 2024-09-16T00:26:16.855 INFO:tasks.workunit.client.1.smithi169.stdout:8/988: chown d7/dc/d17/d24 18 1 2024-09-16T00:26:16.857 INFO:tasks.workunit.client.1.smithi169.stdout:5/773: mknod d2/d7c/c10d 0 2024-09-16T00:26:16.857 INFO:tasks.workunit.client.1.smithi169.stdout:1/907: symlink dc/d18/d67/d69/d7b/l111 0 2024-09-16T00:26:16.857 INFO:tasks.workunit.client.1.smithi169.stdout:1/908: chown dc/d36/d39/d50/d75 0 1 2024-09-16T00:26:16.857 INFO:tasks.workunit.client.1.smithi169.stdout:2/838: dread d7/db/db8/d6d/f71 [0,4194304] 0 2024-09-16T00:26:16.860 INFO:tasks.workunit.client.1.smithi169.stdout:1/909: fsync dc/d18/d67/d69/d7b/d92/f105 0 2024-09-16T00:26:16.864 INFO:tasks.workunit.client.1.smithi169.stdout:5/774: fdatasync d2/da/d10/d13/d1c/d2d/f57 0 2024-09-16T00:26:16.865 INFO:tasks.workunit.client.1.smithi169.stdout:5/775: fdatasync d2/da/f19 0 2024-09-16T00:26:16.867 INFO:tasks.workunit.client.1.smithi169.stdout:8/989: symlink d7/dc/d26/d41/d92/dcf/d3c/d82/db3/db4/l146 0 2024-09-16T00:26:16.869 INFO:tasks.workunit.client.1.smithi169.stdout:5/776: creat d2/da/d10/d13/d1c/d2d/db2/f10e x:0 0 0 2024-09-16T00:26:16.874 INFO:tasks.workunit.client.1.smithi169.stdout:1/910: rename dc/d36/d3a/d45/c43 to dc/d36/d3a/d45/d29/dc2/c112 0 2024-09-16T00:26:16.874 INFO:tasks.workunit.client.1.smithi169.stdout:8/990: truncate d7/dc/d26/d41/d67/fe6 706481 0 2024-09-16T00:26:16.875 INFO:tasks.workunit.client.1.smithi169.stdout:5/777: rmdir d2/da/d10/d13/d1c/d32/d53/d4e/d89 39 2024-09-16T00:26:16.877 INFO:tasks.workunit.client.1.smithi169.stdout:1/911: chown dc/d36/d3a/d45/d29/dc2/f103 7507 1 2024-09-16T00:26:16.880 INFO:tasks.workunit.client.1.smithi169.stdout:8/991: read d7/dc/d26/d41/d92/dcf/d80/f8b [2977801,61037] 0 2024-09-16T00:26:16.881 INFO:tasks.workunit.client.1.smithi169.stdout:8/992: chown d7/dc/d17 125125 1 2024-09-16T00:26:16.882 INFO:tasks.workunit.client.1.smithi169.stdout:2/839: write d7/db/d20/d21/d105/f106 [287038,117690] 0 2024-09-16T00:26:16.884 INFO:tasks.workunit.client.1.smithi169.stdout:1/912: symlink dc/d36/d39/d50/daf/l113 0 2024-09-16T00:26:16.885 INFO:tasks.workunit.client.1.smithi169.stdout:5/778: creat d2/da/f10f x:0 0 0 2024-09-16T00:26:16.911 INFO:tasks.workunit.client.1.smithi169.stdout:0/844: sync 2024-09-16T00:26:16.917 INFO:tasks.workunit.client.1.smithi169.stdout:0/845: dread d0/d2/d15/fc1 [0,4194304] 0 2024-09-16T00:26:16.917 INFO:tasks.workunit.client.1.smithi169.stdout:0/846: stat d0/d4c/d82/d10f/l57 0 2024-09-16T00:26:16.919 INFO:tasks.workunit.client.1.smithi169.stdout:8/993: symlink d7/dc/d17/l147 0 2024-09-16T00:26:16.923 INFO:tasks.workunit.client.1.smithi169.stdout:2/840: mknod d7/db/d89/db4/d69/d92/d12a/c131 0 2024-09-16T00:26:16.925 INFO:tasks.workunit.client.1.smithi169.stdout:1/913: mknod dc/d36/d39/d50/daf/c114 0 2024-09-16T00:26:16.931 INFO:tasks.workunit.client.1.smithi169.stdout:2/841: unlink d7/d1c/d26/fe9 0 2024-09-16T00:26:16.932 INFO:tasks.workunit.client.1.smithi169.stdout:2/842: chown d7/db/d27 2 1 2024-09-16T00:26:16.932 INFO:tasks.workunit.client.1.smithi169.stdout:2/843: chown d7/d1c/dc3/l11e 2698367 1 2024-09-16T00:26:16.937 INFO:tasks.workunit.client.1.smithi169.stdout:5/779: getdents d2/da/d10/d13/d1c/d32/d53/d83/d7d/d96 0 2024-09-16T00:26:16.941 INFO:tasks.workunit.client.1.smithi169.stdout:0/847: write d0/d2/f6 [8749540,65799] 0 2024-09-16T00:26:16.942 INFO:tasks.workunit.client.1.smithi169.stdout:0/848: fsync d0/d2/d8/d5/d60/ddb/ff4 0 2024-09-16T00:26:16.942 INFO:tasks.workunit.client.1.smithi169.stdout:2/844: unlink d7/daa/df5/lec 0 2024-09-16T00:26:16.944 INFO:tasks.workunit.client.1.smithi169.stdout:2/845: write d7/db/d20/d21/f4a [619000,30663] 0 2024-09-16T00:26:16.950 INFO:tasks.workunit.client.1.smithi169.stdout:8/994: dread d7/dc/d17/d24/d32/d8d/d13c/fa6 [0,4194304] 0 2024-09-16T00:26:16.952 INFO:tasks.workunit.client.1.smithi169.stdout:8/995: dread - d7/dc/d26/d41/d4e/d102/f132 zero size 2024-09-16T00:26:16.954 INFO:tasks.workunit.client.1.smithi169.stdout:5/780: getdents d2/da/d10/d13/d1c/d32/d53/d4e/d55 0 2024-09-16T00:26:16.955 INFO:tasks.workunit.client.1.smithi169.stdout:2/846: fdatasync d7/db/db8/f57 0 2024-09-16T00:26:16.956 INFO:tasks.workunit.client.1.smithi169.stdout:8/996: mknod d7/dc/d26/d41/d4e/d81/d9e/c148 0 2024-09-16T00:26:16.961 INFO:tasks.workunit.client.1.smithi169.stdout:5/781: creat d2/da/d10/d13/d1c/d32/d53/d4e/d89/d9a/f110 x:0 0 0 2024-09-16T00:26:16.964 INFO:tasks.workunit.client.1.smithi169.stdout:8/997: truncate d7/dc/d26/d41/d4e/d81/fc6 1461283 0 2024-09-16T00:26:16.967 INFO:tasks.workunit.client.1.smithi169.stdout:2/847: mkdir d7/db/d27/d29/d132 0 2024-09-16T00:26:16.967 INFO:tasks.workunit.client.1.smithi169.stdout:8/998: dread f5 [0,4194304] 0 2024-09-16T00:26:16.970 INFO:tasks.workunit.client.1.smithi169.stdout:5/782: dwrite d2/da/fcd [0,4194304] 0 2024-09-16T00:26:16.973 INFO:tasks.workunit.client.1.smithi169.stdout:8/999: dread - d7/dc/d17/d24/d32/d105/f109 zero size 2024-09-16T00:26:16.978 INFO:tasks.workunit.client.1.smithi169.stdout:5/783: unlink d2/da/d10/d13/d1c/fe8 0 2024-09-16T00:26:16.979 INFO:tasks.workunit.client.1.smithi169.stdout:2/848: creat d7/db/d20/d21/d105/d11f/f133 x:0 0 0 2024-09-16T00:26:16.980 INFO:tasks.workunit.client.1.smithi169.stdout:0/849: dwrite d0/d4c/d6f/fda [0,4194304] 0 2024-09-16T00:26:16.981 INFO:tasks.workunit.client.1.smithi169.stdout:5/784: rmdir d2/da/d10/d13/d24/de9 39 2024-09-16T00:26:16.982 INFO:tasks.workunit.client.1.smithi169.stdout:5/785: dread - d2/da/d10/fa9 zero size 2024-09-16T00:26:16.988 INFO:tasks.workunit.client.1.smithi169.stdout:5/786: symlink d2/da/d10/d13/d1c/d32/l111 0 2024-09-16T00:26:16.988 INFO:tasks.workunit.client.1.smithi169.stdout:0/850: getdents d0/d2c/d38/de4/d21/d31/dfe 0 2024-09-16T00:26:16.992 INFO:tasks.workunit.client.1.smithi169.stdout:2/849: dwrite d7/d1c/d26/d96/dab/f10b [0,4194304] 0 2024-09-16T00:26:16.996 INFO:tasks.workunit.client.1.smithi169.stdout:0/851: rename d0/d2c/d38/de4/d11d/c11e to d0/d2c/d38/de4/d21/d31/d83/c123 0 2024-09-16T00:26:16.997 INFO:tasks.workunit.client.1.smithi169.stdout:2/850: truncate d7/db/db8/daf/d10a/d11b/d114/f12c 892861 0 2024-09-16T00:26:16.998 INFO:tasks.workunit.client.1.smithi169.stdout:5/787: symlink d2/da/d10/d13/d1c/d32/d53/dae/l112 0 2024-09-16T00:26:17.001 INFO:tasks.workunit.client.1.smithi169.stdout:2/851: unlink d7/db/db8/d6d/l91 0 2024-09-16T00:26:17.002 INFO:tasks.workunit.client.1.smithi169.stdout:0/852: symlink d0/d2c/d38/de4/d21/d31/l124 0 2024-09-16T00:26:17.003 INFO:tasks.workunit.client.1.smithi169.stdout:5/788: creat d2/da/d10/d13/d1c/d32/d53/dae/de2/f113 x:0 0 0 2024-09-16T00:26:17.004 INFO:tasks.workunit.client.1.smithi169.stdout:5/789: stat d2/da/d10/d13/d1c/d40/d54 0 2024-09-16T00:26:17.007 INFO:tasks.workunit.client.1.smithi169.stdout:2/852: rename d7/db/f19 to d7/d1c/d3c/f134 0 2024-09-16T00:26:17.010 INFO:tasks.workunit.client.1.smithi169.stdout:0/853: creat d0/d2c/d38/f125 x:0 0 0 2024-09-16T00:26:17.014 INFO:tasks.workunit.client.1.smithi169.stdout:2/853: rename d7/db/d20/d21 to d7/d1c/de6/d135 0 2024-09-16T00:26:17.015 INFO:tasks.workunit.client.1.smithi169.stdout:5/790: dwrite d2/da/d10/d13/d1c/d40/d54/d7f/fd9 [0,4194304] 0 2024-09-16T00:26:17.018 INFO:tasks.workunit.client.1.smithi169.stdout:0/854: dread - d0/d2c/d38/fc6 zero size 2024-09-16T00:26:17.024 INFO:tasks.workunit.client.1.smithi169.stdout:2/854: read d7/db/f23 [1277176,20770] 0 2024-09-16T00:26:17.025 INFO:tasks.workunit.client.1.smithi169.stdout:5/791: write d2/da/d10/d13/d1c/d32/d53/fdd [681285,105301] 0 2024-09-16T00:26:17.025 INFO:tasks.workunit.client.1.smithi169.stdout:5/792: chown d2/da/d10/d13/d1c/d32/d53/dae/dbf/fd2 939027730 1 2024-09-16T00:26:17.031 INFO:tasks.workunit.client.1.smithi169.stdout:5/793: mkdir d2/da/d10/d13/d1c/d2d/d114 0 2024-09-16T00:26:17.036 INFO:tasks.workunit.client.1.smithi169.stdout:2/855: link d7/d1c/f95 d7/db/d89/db4/dc8/f136 0 2024-09-16T00:26:17.040 INFO:tasks.workunit.client.1.smithi169.stdout:2/856: mknod d7/d6b/d70/d79/dce/c137 0 2024-09-16T00:26:17.041 INFO:tasks.workunit.client.1.smithi169.stdout:5/794: dwrite d2/da/d10/d13/d1c/d32/d53/dae/de2/f108 [0,4194304] 0 2024-09-16T00:26:17.047 INFO:tasks.workunit.client.1.smithi169.stdout:0/855: dread d0/d2/d4b/d86/f40 [0,4194304] 0 2024-09-16T00:26:17.050 INFO:tasks.workunit.client.1.smithi169.stdout:0/856: mknod d0/d2c/d38/de4/d11d/c126 0 2024-09-16T00:26:17.052 INFO:tasks.workunit.client.1.smithi169.stdout:2/857: creat d7/db/db8/f138 x:0 0 0 2024-09-16T00:26:17.056 INFO:tasks.workunit.client.1.smithi169.stdout:2/858: truncate d7/db/d89/db4/d9d/dbe/fdf 1073451 0 2024-09-16T00:26:17.064 INFO:tasks.workunit.client.1.smithi169.stdout:2/859: dwrite d7/db/db8/d6d/f71 [0,4194304] 0 2024-09-16T00:26:17.067 INFO:tasks.workunit.client.1.smithi169.stdout:2/860: rmdir d7/db 39 2024-09-16T00:26:17.070 INFO:tasks.workunit.client.1.smithi169.stdout:0/857: dwrite d0/d2c/d38/de4/d21/f2f [0,4194304] 0 2024-09-16T00:26:17.072 INFO:tasks.workunit.client.1.smithi169.stdout:0/858: dread - d0/d2/d8/d5/d79/f116 zero size 2024-09-16T00:26:17.076 INFO:tasks.workunit.client.1.smithi169.stdout:2/861: rename d7/db/d89/db4 to d7/d1c/d26/d96/d139 0 2024-09-16T00:26:17.077 INFO:tasks.workunit.client.1.smithi169.stdout:2/862: dread - d7/d1c/d26/d96/d139/f113 zero size 2024-09-16T00:26:17.080 INFO:tasks.workunit.client.1.smithi169.stdout:2/863: stat d7/d6b/d70/d79/dce/c137 0 2024-09-16T00:26:17.080 INFO:tasks.workunit.client.1.smithi169.stdout:2/864: readlink d7/d1c/d26/l6f 0 2024-09-16T00:26:17.083 INFO:tasks.workunit.client.1.smithi169.stdout:2/865: read d7/d1c/d26/f2c [319799,97176] 0 2024-09-16T00:26:17.087 INFO:tasks.workunit.client.1.smithi169.stdout:2/866: dread d7/d6b/d70/d79/dce/fd8 [0,4194304] 0 2024-09-16T00:26:17.092 INFO:tasks.workunit.client.1.smithi169.stdout:2/867: dwrite d7/db/f23 [0,4194304] 0 2024-09-16T00:26:17.097 INFO:tasks.workunit.client.1.smithi169.stdout:2/868: unlink d7/d1c/d26/d96/d139/dc8/ce8 0 2024-09-16T00:26:17.099 INFO:tasks.workunit.client.1.smithi169.stdout:2/869: fdatasync d7/db/d27/d29/d37/f97 0 2024-09-16T00:26:17.099 INFO:tasks.workunit.client.1.smithi169.stdout:0/859: write d0/d7d/fde [419829,6833] 0 2024-09-16T00:26:17.102 INFO:tasks.workunit.client.1.smithi169.stdout:2/870: chown d7/db/d27/ded 125047 1 2024-09-16T00:26:17.105 INFO:tasks.workunit.client.1.smithi169.stdout:2/871: creat d7/d1c/d26/d96/d139/f13a x:0 0 0 2024-09-16T00:26:17.106 INFO:tasks.workunit.client.1.smithi169.stdout:2/872: readlink d7/db/db8/d6d/df1/l12d 0 2024-09-16T00:26:17.110 INFO:tasks.workunit.client.1.smithi169.stdout:2/873: dread d7/d1c/de6/d135/d105/f106 [0,4194304] 0 2024-09-16T00:26:17.114 INFO:tasks.workunit.client.1.smithi169.stdout:2/874: dread - d7/db/d27/d29/fc0 zero size 2024-09-16T00:26:17.118 INFO:tasks.workunit.client.1.smithi169.stdout:0/860: dwrite d0/d2c/d38/de4/d21/f3f [0,4194304] 0 2024-09-16T00:26:17.120 INFO:tasks.workunit.client.1.smithi169.stdout:2/875: getdents d7/d6b/d70/d79/dae 0 2024-09-16T00:26:17.122 INFO:tasks.workunit.client.1.smithi169.stdout:0/861: chown d0/d2c/d38/de4/fcf 71 1 2024-09-16T00:26:17.125 INFO:tasks.workunit.client.1.smithi169.stdout:2/876: symlink d7/db/d27/d29/d37/l13b 0 2024-09-16T00:26:17.126 INFO:tasks.workunit.client.1.smithi169.stdout:2/877: fsync d7/db/d20/fa7 0 2024-09-16T00:26:17.130 INFO:tasks.workunit.client.1.smithi169.stdout:2/878: rename d7/db/db8/daf/lb1 to d7/db/db8/d6d/l13c 0 2024-09-16T00:26:17.131 INFO:tasks.workunit.client.1.smithi169.stdout:2/879: fdatasync d7/db/d20/f83 0 2024-09-16T00:26:17.142 INFO:tasks.workunit.client.1.smithi169.stdout:0/862: write d0/d2c/d38/de4/d21/d31/d83/fe0 [745436,118220] 0 2024-09-16T00:26:17.160 INFO:tasks.workunit.client.1.smithi169.stdout:2/880: dread d7/db/d20/fa7 [0,4194304] 0 2024-09-16T00:26:17.162 INFO:tasks.workunit.client.1.smithi169.stdout:2/881: symlink d7/d1c/d3c/l13d 0 2024-09-16T00:26:17.164 INFO:tasks.workunit.client.1.smithi169.stdout:2/882: mknod d7/c13e 0 2024-09-16T00:26:17.167 INFO:tasks.workunit.client.1.smithi169.stdout:2/883: symlink d7/d1c/d26/d96/d139/d9d/dbe/de4/l13f 0 2024-09-16T00:26:17.169 INFO:tasks.workunit.client.1.smithi169.stdout:2/884: rmdir d7/daa 39 2024-09-16T00:26:17.170 INFO:tasks.workunit.client.1.smithi169.stdout:2/885: chown d7/d1c/d26/l2d 41112 1 2024-09-16T00:26:17.172 INFO:tasks.workunit.client.1.smithi169.stdout:2/886: creat d7/db/d27/d29/d37/f140 x:0 0 0 2024-09-16T00:26:17.207 INFO:tasks.workunit.client.1.smithi169.stdout:2/887: fdatasync d7/d1c/d26/d96/d139/d69/f99 0 2024-09-16T00:26:17.208 INFO:tasks.workunit.client.1.smithi169.stdout:2/888: truncate d7/d1c/d26/d96/d139/f113 61022 0 2024-09-16T00:26:17.210 INFO:tasks.workunit.client.1.smithi169.stdout:2/889: truncate d7/fa0 268934 0 2024-09-16T00:26:17.212 INFO:tasks.workunit.client.1.smithi169.stdout:2/890: creat d7/d1c/d26/d96/d139/dc8/f141 x:0 0 0 2024-09-16T00:26:17.215 INFO:tasks.workunit.client.1.smithi169.stdout:2/891: rename d7/d1c/d26/d96/d139/d9d/db9 to d7/d1c/de6/d135/d142 0 2024-09-16T00:26:17.217 INFO:tasks.workunit.client.1.smithi169.stdout:2/892: creat d7/d6b/d70/d79/dae/f143 x:0 0 0 2024-09-16T00:26:17.218 INFO:tasks.workunit.client.1.smithi169.stdout:2/893: read d7/d1c/d26/f76 [1327737,23144] 0 2024-09-16T00:26:17.219 INFO:tasks.workunit.client.1.smithi169.stdout:2/894: fsync d7/d1c/de6/d135/fee 0 2024-09-16T00:26:17.220 INFO:tasks.workunit.client.1.smithi169.stdout:2/895: write d7/db/db8/d6d/f130 [92594,51720] 0 2024-09-16T00:26:17.222 INFO:tasks.workunit.client.1.smithi169.stdout:2/896: mknod d7/d1c/d3c/d122/c144 0 2024-09-16T00:26:17.234 INFO:tasks.workunit.client.1.smithi169.stdout:2/897: dread d7/d1c/d26/d96/d139/d9d/f111 [0,4194304] 0 2024-09-16T00:26:17.237 INFO:tasks.workunit.client.1.smithi169.stdout:2/898: creat d7/d1c/d26/d96/d139/d9d/dbe/f145 x:0 0 0 2024-09-16T00:26:17.316 INFO:tasks.workunit.client.1.smithi169.stdout:4/839: sync 2024-09-16T00:26:17.316 INFO:tasks.workunit.client.1.smithi169.stdout:4/840: dread - d2/d12/d13/d1f/d3e/dfb/feb zero size 2024-09-16T00:26:17.318 INFO:tasks.workunit.client.1.smithi169.stdout:4/841: mkdir d2/d12/d13/d5c/d10a 0 2024-09-16T00:26:17.320 INFO:tasks.workunit.client.1.smithi169.stdout:4/842: mknod d2/d12/d13/d90/c10b 0 2024-09-16T00:26:17.322 INFO:tasks.workunit.client.1.smithi169.stdout:4/843: mknod d2/d12/d13/d1f/d3e/d5b/d70/c10c 0 2024-09-16T00:26:17.419 INFO:tasks.workunit.client.1.smithi169.stdout:4/844: dwrite d2/d12/d13/d1f/d3e/d83/f74 [0,4194304] 0 2024-09-16T00:26:17.421 INFO:tasks.workunit.client.1.smithi169.stdout:4/845: creat d2/d12/d13/d90/f10d x:0 0 0 2024-09-16T00:26:17.423 INFO:tasks.workunit.client.1.smithi169.stdout:4/846: symlink d2/d12/d13/d4e/l10e 0 2024-09-16T00:26:17.426 INFO:tasks.workunit.client.1.smithi169.stdout:4/847: creat d2/d12/d13/d5c/f10f x:0 0 0 2024-09-16T00:26:17.428 INFO:tasks.workunit.client.1.smithi169.stdout:4/848: symlink d2/d12/l110 0 2024-09-16T00:26:17.430 INFO:tasks.workunit.client.1.smithi169.stdout:4/849: truncate d2/d12/d9c/fff 700905 0 2024-09-16T00:26:17.433 INFO:tasks.workunit.client.1.smithi169.stdout:4/850: creat d2/d12/d13/d1f/d3e/f111 x:0 0 0 2024-09-16T00:26:17.434 INFO:tasks.workunit.client.1.smithi169.stdout:4/851: dread d2/d12/dcc/f55 [0,4194304] 0 2024-09-16T00:26:17.435 INFO:tasks.workunit.client.1.smithi169.stdout:4/852: write d2/d12/fbe [566450,88086] 0 2024-09-16T00:26:17.439 INFO:tasks.workunit.client.1.smithi169.stdout:4/853: rename d2/d12/d13/d1f/d3e/d5b/d70/d7d/l81 to d2/d12/d13/d1f/d3e/d5b/d70/d7d/l112 0 2024-09-16T00:26:17.441 INFO:tasks.workunit.client.1.smithi169.stdout:4/854: mknod d2/d12/d13/d1f/d2f/de7/c113 0 2024-09-16T00:26:17.442 INFO:tasks.workunit.client.1.smithi169.stdout:4/855: chown d2/d12/d13/d1f/d3e/d83/fa5 93372 1 2024-09-16T00:26:17.444 INFO:tasks.workunit.client.1.smithi169.stdout:4/856: chown d2/l7 21562412 1 2024-09-16T00:26:17.448 INFO:tasks.workunit.client.1.smithi169.stdout:4/857: link d2/d12/d13/d1f/d2f/f7b d2/d12/d13/d4e/d50/f114 0 2024-09-16T00:26:17.449 INFO:tasks.workunit.client.1.smithi169.stdout:4/858: read - d2/d12/d13/d1f/d3e/d5b/f6b zero size 2024-09-16T00:26:17.451 INFO:tasks.workunit.client.1.smithi169.stdout:4/859: mkdir d2/d12/d13/d1f/d3e/d5b/d70/d115 0 2024-09-16T00:26:17.455 INFO:tasks.workunit.client.1.smithi169.stdout:4/860: link d2/d12/d13/d27/c99 d2/d12/d13/d27/dfd/c116 0 2024-09-16T00:26:17.458 INFO:tasks.workunit.client.1.smithi169.stdout:4/861: rename d2/d12/d13/d4e/fd6 to d2/d12/d13/d5c/f117 0 2024-09-16T00:26:17.459 INFO:tasks.workunit.client.1.smithi169.stdout:4/862: readlink d2/d12/d13/d4e/d50/l8a 0 2024-09-16T00:26:17.460 INFO:tasks.workunit.client.1.smithi169.stdout:4/863: write d2/d12/d13/d5c/f94 [2241504,121472] 0 2024-09-16T00:26:17.473 INFO:tasks.workunit.client.1.smithi169.stdout:4/864: dread d2/d12/d13/f25 [0,4194304] 0 2024-09-16T00:26:17.475 INFO:tasks.workunit.client.1.smithi169.stdout:4/865: creat d2/d12/d13/d1f/f118 x:0 0 0 2024-09-16T00:26:17.476 INFO:tasks.workunit.client.1.smithi169.stdout:4/866: readlink d2/d12/d13/d4e/d50/db8/lf9 0 2024-09-16T00:26:17.487 INFO:tasks.workunit.client.1.smithi169.stdout:7/955: sync 2024-09-16T00:26:17.488 INFO:tasks.workunit.client.1.smithi169.stdout:4/867: dwrite d2/d12/d13/d1f/d3e/d5b/f89 [0,4194304] 0 2024-09-16T00:26:17.490 INFO:tasks.workunit.client.1.smithi169.stdout:4/868: write d2/d12/dcc/f47 [3468155,123262] 0 2024-09-16T00:26:17.499 INFO:tasks.workunit.client.1.smithi169.stdout:7/956: dwrite d7/fa0 [4194304,4194304] 0 2024-09-16T00:26:17.501 INFO:tasks.workunit.client.1.smithi169.stdout:4/869: write d2/d12/d13/d1f/f8e [2025143,123718] 0 2024-09-16T00:26:17.502 INFO:tasks.workunit.client.1.smithi169.stdout:7/957: mkdir d7/d16/d49/db2/d132 0 2024-09-16T00:26:17.504 INFO:tasks.workunit.client.1.smithi169.stdout:4/870: creat d2/d12/dcc/d9e/f119 x:0 0 0 2024-09-16T00:26:17.511 INFO:tasks.workunit.client.1.smithi169.stdout:4/871: rmdir d2/d12/dcc/d9e 39 2024-09-16T00:26:17.512 INFO:tasks.workunit.client.1.smithi169.stdout:4/872: dread - d2/dbb/fed zero size 2024-09-16T00:26:17.516 INFO:tasks.workunit.client.1.smithi169.stdout:4/873: mkdir d2/d12/d13/d1f/d3e/d5b/d70/def/d11a 0 2024-09-16T00:26:17.518 INFO:tasks.workunit.client.1.smithi169.stdout:4/874: symlink d2/dbb/d59/l11b 0 2024-09-16T00:26:17.520 INFO:tasks.workunit.client.1.smithi169.stdout:4/875: creat d2/dbb/d59/f11c x:0 0 0 2024-09-16T00:26:17.527 INFO:tasks.workunit.client.1.smithi169.stdout:4/876: dwrite d2/d12/d13/d1f/d3e/d83/f74 [0,4194304] 0 2024-09-16T00:26:17.532 INFO:tasks.workunit.client.1.smithi169.stdout:4/877: symlink d2/d12/d13/d4e/l11d 0 2024-09-16T00:26:17.533 INFO:tasks.workunit.client.1.smithi169.stdout:7/958: dread d7/d5b/f6b [4194304,4194304] 0 2024-09-16T00:26:17.535 INFO:tasks.workunit.client.1.smithi169.stdout:4/878: symlink d2/d12/d13/d1f/d3e/d5b/l11e 0 2024-09-16T00:26:17.535 INFO:tasks.workunit.client.1.smithi169.stdout:7/959: creat d7/d71/d76/f133 x:0 0 0 2024-09-16T00:26:17.540 INFO:tasks.workunit.client.1.smithi169.stdout:7/960: stat d7/d71/d76/d29/d78/fa3 0 2024-09-16T00:26:17.541 INFO:tasks.workunit.client.1.smithi169.stdout:7/961: creat d7/d16/d49/db2/db4/f134 x:0 0 0 2024-09-16T00:26:17.543 INFO:tasks.workunit.client.1.smithi169.stdout:4/879: write d2/d12/f2c [4748352,84807] 0 2024-09-16T00:26:17.546 INFO:tasks.workunit.client.1.smithi169.stdout:7/962: creat d7/d16/d49/f135 x:0 0 0 2024-09-16T00:26:17.548 INFO:tasks.workunit.client.1.smithi169.stdout:4/880: mkdir d2/d12/d13/d1f/d3e/d5b/d70/d7d/d102/d11f 0 2024-09-16T00:26:17.552 INFO:tasks.workunit.client.1.smithi169.stdout:4/881: read d2/d12/d13/d1f/d3e/d5b/f24 [4167254,92032] 0 2024-09-16T00:26:17.558 INFO:tasks.workunit.client.1.smithi169.stdout:4/882: write d2/d12/d13/d27/f32 [2040452,127351] 0 2024-09-16T00:26:17.561 INFO:tasks.workunit.client.1.smithi169.stdout:4/883: mkdir d2/d12/d13/d5c/d10a/d120 0 2024-09-16T00:26:17.565 INFO:tasks.workunit.client.1.smithi169.stdout:4/884: link d2/d12/dcc/fad d2/d12/dcc/d9e/f121 0 2024-09-16T00:26:17.570 INFO:tasks.workunit.client.1.smithi169.stdout:7/963: dread d7/d71/d76/d29/d78/dba/fcd [0,4194304] 0 2024-09-16T00:26:17.570 INFO:tasks.workunit.client.1.smithi169.stdout:4/885: truncate d2/d12/dcc/fad 980568 0 2024-09-16T00:26:17.571 INFO:tasks.workunit.client.1.smithi169.stdout:7/964: rmdir d7/d5b/dc1 39 2024-09-16T00:26:17.572 INFO:tasks.workunit.client.1.smithi169.stdout:4/886: write d2/d12/d13/d1f/f35 [4642083,7866] 0 2024-09-16T00:26:17.574 INFO:tasks.workunit.client.1.smithi169.stdout:7/965: mkdir d7/d71/d76/d136 0 2024-09-16T00:26:17.575 INFO:tasks.workunit.client.1.smithi169.stdout:7/966: write d7/d16/d49/d74/fc5 [3441556,38488] 0 2024-09-16T00:26:17.576 INFO:tasks.workunit.client.1.smithi169.stdout:4/887: rename d2/d12/d13/d1f/f20 to d2/d12/d13/d1f/f122 0 2024-09-16T00:26:17.578 INFO:tasks.workunit.client.1.smithi169.stdout:7/967: rename d7/d71/fae to d7/d16/d49/d74/f137 0 2024-09-16T00:26:17.579 INFO:tasks.workunit.client.1.smithi169.stdout:4/888: symlink d2/d12/l123 0 2024-09-16T00:26:17.586 INFO:tasks.workunit.client.1.smithi169.stdout:4/889: dwrite d2/d12/dcc/f47 [0,4194304] 0 2024-09-16T00:26:17.589 INFO:tasks.workunit.client.1.smithi169.stdout:7/968: write d7/d5b/f77 [1320075,129649] 0 2024-09-16T00:26:17.592 INFO:tasks.workunit.client.1.smithi169.stdout:4/890: creat d2/d12/d13/d1f/d3e/d5b/d70/d7d/d102/f124 x:0 0 0 2024-09-16T00:26:17.602 INFO:tasks.workunit.client.1.smithi169.stdout:7/969: write d7/d71/f88 [923345,127640] 0 2024-09-16T00:26:17.611 INFO:tasks.workunit.client.1.smithi169.stdout:1/914: sync 2024-09-16T00:26:17.615 INFO:tasks.workunit.client.1.smithi169.stdout:4/891: dread d2/d12/d13/d1f/d3e/d5b/f9f [0,4194304] 0 2024-09-16T00:26:17.618 INFO:tasks.workunit.client.1.smithi169.stdout:1/915: link dc/d36/d3a/d45/d29/dc2/dc7/cfc dc/d36/d3a/d7f/ddb/c115 0 2024-09-16T00:26:17.619 INFO:tasks.workunit.client.1.smithi169.stdout:4/892: write d2/d12/d13/d5c/f117 [803627,19870] 0 2024-09-16T00:26:17.619 INFO:tasks.workunit.client.1.smithi169.stdout:7/970: dread d7/f91 [0,4194304] 0 2024-09-16T00:26:17.621 INFO:tasks.workunit.client.1.smithi169.stdout:7/971: stat d7/d5b/d61 0 2024-09-16T00:26:17.624 INFO:tasks.workunit.client.1.smithi169.stdout:7/972: creat d7/d5b/d61/d98/d99/df9/dc8/f138 x:0 0 0 2024-09-16T00:26:17.625 INFO:tasks.workunit.client.1.smithi169.stdout:7/973: dread - d7/d12d/f106 zero size 2024-09-16T00:26:17.626 INFO:tasks.workunit.client.1.smithi169.stdout:7/974: write d7/d16/fda [1787183,50963] 0 2024-09-16T00:26:17.630 INFO:tasks.workunit.client.1.smithi169.stdout:7/975: mknod d7/d5b/d61/c139 0 2024-09-16T00:26:17.633 INFO:tasks.workunit.client.1.smithi169.stdout:4/893: dwrite d2/d12/d13/d4e/d50/f85 [0,4194304] 0 2024-09-16T00:26:17.633 INFO:tasks.workunit.client.1.smithi169.stdout:1/916: truncate dc/d36/d3a/d45/f48 6093414 0 2024-09-16T00:26:17.634 INFO:tasks.workunit.client.1.smithi169.stdout:4/894: write d2/fdf [728997,32104] 0 2024-09-16T00:26:17.635 INFO:tasks.workunit.client.1.smithi169.stdout:7/976: write d7/d71/d76/d29/d78/f100 [1022509,78091] 0 2024-09-16T00:26:17.640 INFO:tasks.workunit.client.1.smithi169.stdout:1/917: rename dc/d36/d39/d50/daf/d86/lb5 to dc/d36/l116 0 2024-09-16T00:26:17.641 INFO:tasks.workunit.client.1.smithi169.stdout:4/895: fdatasync d2/d12/d13/d1f/d2f/f82 0 2024-09-16T00:26:17.643 INFO:tasks.workunit.client.1.smithi169.stdout:1/918: creat dc/d18/d67/f117 x:0 0 0 2024-09-16T00:26:17.644 INFO:tasks.workunit.client.1.smithi169.stdout:4/896: symlink d2/d12/d13/d1f/d3e/d83/dc8/l125 0 2024-09-16T00:26:17.648 INFO:tasks.workunit.client.1.smithi169.stdout:4/897: symlink d2/d12/d13/d5c/da6/dd3/l126 0 2024-09-16T00:26:17.652 INFO:tasks.workunit.client.1.smithi169.stdout:4/898: symlink d2/d12/d13/d4e/d50/db8/l127 0 2024-09-16T00:26:17.663 INFO:tasks.workunit.client.1.smithi169.stdout:4/899: write d2/d12/f31 [3684393,113916] 0 2024-09-16T00:26:17.665 INFO:tasks.workunit.client.1.smithi169.stdout:7/977: dwrite d7/f8b [4194304,4194304] 0 2024-09-16T00:26:17.667 INFO:tasks.workunit.client.1.smithi169.stdout:7/978: chown d7/d5b/d61/d98/d99/df9/dc8/c101 922013927 1 2024-09-16T00:26:17.667 INFO:tasks.workunit.client.1.smithi169.stdout:7/979: chown d7 203232 1 2024-09-16T00:26:17.668 INFO:tasks.workunit.client.1.smithi169.stdout:5/795: sync 2024-09-16T00:26:17.669 INFO:tasks.workunit.client.1.smithi169.stdout:0/863: sync 2024-09-16T00:26:17.669 INFO:tasks.workunit.client.1.smithi169.stdout:2/899: sync 2024-09-16T00:26:17.671 INFO:tasks.workunit.client.1.smithi169.stdout:0/864: readlink d0/d2c/d38/de4/l72 0 2024-09-16T00:26:17.731 INFO:tasks.workunit.client.1.smithi169.stdout:4/900: fsync d2/d12/d13/d1f/d3e/d5b/fd7 0 2024-09-16T00:26:17.732 INFO:tasks.workunit.client.1.smithi169.stdout:4/901: readlink d2/d12/d13/d5c/lf5 0 2024-09-16T00:26:17.734 INFO:tasks.workunit.client.1.smithi169.stdout:5/796: creat d2/da/d10/d13/d1c/d94/dc8/f115 x:0 0 0 2024-09-16T00:26:17.736 INFO:tasks.workunit.client.1.smithi169.stdout:2/900: rmdir d7/d1c/dc3 39 2024-09-16T00:26:17.740 INFO:tasks.workunit.client.1.smithi169.stdout:5/797: dread d2/da/fcd [0,4194304] 0 2024-09-16T00:26:17.745 INFO:tasks.workunit.client.1.smithi169.stdout:5/798: truncate d2/da/d10/d13/d1c/d32/d53/dae/ffd 2480689 0 2024-09-16T00:26:17.745 INFO:tasks.workunit.client.1.smithi169.stdout:2/901: fsync d7/d6b/d70/d79/dce/fd8 0 2024-09-16T00:26:17.750 INFO:tasks.workunit.client.1.smithi169.stdout:0/865: creat d0/d2/d8/d5/d79/f127 x:0 0 0 2024-09-16T00:26:17.751 INFO:tasks.workunit.client.1.smithi169.stdout:5/799: creat d2/da/d10/d13/d1c/d94/dc8/f116 x:0 0 0 2024-09-16T00:26:17.751 INFO:tasks.workunit.client.1.smithi169.stdout:0/866: rename d0/d2/d8/d5/d60/ddb to d0/d2/d8/d5/d60/ddb/d128 22 2024-09-16T00:26:17.752 INFO:tasks.workunit.client.1.smithi169.stdout:7/980: creat d7/d16/d49/db2/db4/f13a x:0 0 0 2024-09-16T00:26:17.753 INFO:tasks.workunit.client.1.smithi169.stdout:5/800: chown d2/da/d10/d13/d24/l60 269322 1 2024-09-16T00:26:17.753 INFO:tasks.workunit.client.1.smithi169.stdout:2/902: rmdir d7/d1c/d26/d96/d139/d9d/dbe/de4 39 2024-09-16T00:26:17.755 INFO:tasks.workunit.client.1.smithi169.stdout:2/903: dread - d7/db/d27/d29/fc0 zero size 2024-09-16T00:26:17.758 INFO:tasks.workunit.client.1.smithi169.stdout:0/867: dread d0/d2c/d38/de4/d21/d31/d83/fe0 [0,4194304] 0 2024-09-16T00:26:17.761 INFO:tasks.workunit.client.1.smithi169.stdout:2/904: creat d7/db/db8/daf/d10a/d11b/f146 x:0 0 0 2024-09-16T00:26:17.761 INFO:tasks.workunit.client.1.smithi169.stdout:2/905: dread - d7/d1c/de6/d135/fee zero size 2024-09-16T00:26:17.765 INFO:tasks.workunit.client.1.smithi169.stdout:5/801: getdents d2/da/d10/d13/d1c/d32/d53/d83 0 2024-09-16T00:26:17.769 INFO:tasks.workunit.client.1.smithi169.stdout:2/906: dwrite d7/db/f60 [0,4194304] 0 2024-09-16T00:26:17.773 INFO:tasks.workunit.client.1.smithi169.stdout:5/802: truncate d2/da/d10/d13/d1c/d40/d68/fc1 256796 0 2024-09-16T00:26:17.774 INFO:tasks.workunit.client.1.smithi169.stdout:2/907: rename d7/d1c/dc3/c110 to d7/db/d27/ded/c147 0 2024-09-16T00:26:17.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:17 smithi169 ceph-mon[34653]: pgmap v262: 65 pgs: 65 active+clean; 3.7 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 118 MiB/s rd, 270 MiB/s wr, 570 op/s 2024-09-16T00:26:17.776 INFO:tasks.workunit.client.1.smithi169.stdout:2/908: write d7/d6b/d70/d79/fdc [122881,119557] 0 2024-09-16T00:26:17.777 INFO:tasks.workunit.client.1.smithi169.stdout:2/909: stat d7/d6b/d70/d79/dae 0 2024-09-16T00:26:17.782 INFO:tasks.workunit.client.1.smithi169.stdout:2/910: unlink d7/db/d27/d29/d37/lb0 0 2024-09-16T00:26:17.785 INFO:tasks.workunit.client.1.smithi169.stdout:2/911: creat d7/db/d20/d12b/f148 x:0 0 0 2024-09-16T00:26:17.786 INFO:tasks.workunit.client.1.smithi169.stdout:2/912: stat d7/db/d27/d29/d37/l61 0 2024-09-16T00:26:17.788 INFO:tasks.workunit.client.1.smithi169.stdout:2/913: symlink d7/db/d20/d12b/l149 0 2024-09-16T00:26:17.795 INFO:tasks.workunit.client.1.smithi169.stdout:2/914: dwrite d7/d1c/de6/d135/fee [0,4194304] 0 2024-09-16T00:26:17.798 INFO:tasks.workunit.client.1.smithi169.stdout:2/915: mkdir d7/db/d27/d29/d37/dd1/d14a 0 2024-09-16T00:26:17.801 INFO:tasks.workunit.client.1.smithi169.stdout:2/916: symlink d7/daa/df5/l14b 0 2024-09-16T00:26:17.802 INFO:tasks.workunit.client.1.smithi169.stdout:4/902: truncate d2/d12/d13/d1f/f8e 2077930 0 2024-09-16T00:26:17.803 INFO:tasks.workunit.client.1.smithi169.stdout:5/803: dread d2/da/f50 [0,4194304] 0 2024-09-16T00:26:17.806 INFO:tasks.workunit.client.1.smithi169.stdout:7/981: truncate d7/df/fa6 291330 0 2024-09-16T00:26:17.807 INFO:tasks.workunit.client.1.smithi169.stdout:4/903: chown d2/d12/d13/d5c/da6/fa9 15994829 1 2024-09-16T00:26:17.809 INFO:tasks.workunit.client.1.smithi169.stdout:1/919: dwrite dc/d36/d3a/d7f/d93/fa3 [0,4194304] 0 2024-09-16T00:26:17.810 INFO:tasks.workunit.client.1.smithi169.stdout:0/868: write d0/d4c/d82/d10f/f9f [1787795,5403] 0 2024-09-16T00:26:17.810 INFO:tasks.workunit.client.1.smithi169.stdout:2/917: mkdir d7/db/d20/d5c/d14c 0 2024-09-16T00:26:17.812 INFO:tasks.workunit.client.1.smithi169.stdout:2/918: chown d7/d1c/l5d 65160834 1 2024-09-16T00:26:17.817 INFO:tasks.workunit.client.1.smithi169.stdout:0/869: fdatasync d0/d2/d8/d5/d79/d113/f121 0 2024-09-16T00:26:17.817 INFO:tasks.workunit.client.1.smithi169.stdout:0/870: chown d0/d10d 574180 1 2024-09-16T00:26:17.822 INFO:tasks.workunit.client.1.smithi169.stdout:7/982: dread d7/d12d/d127/f7a [4194304,4194304] 0 2024-09-16T00:26:17.824 INFO:tasks.workunit.client.1.smithi169.stdout:7/983: chown d7/d12d/d127/c51 18 1 2024-09-16T00:26:17.824 INFO:tasks.workunit.client.1.smithi169.stdout:1/920: dread dc/d36/d3a/d45/d29/fb4 [0,4194304] 0 2024-09-16T00:26:17.824 INFO:tasks.workunit.client.1.smithi169.stdout:2/919: unlink d7/db/c15 0 2024-09-16T00:26:17.829 INFO:tasks.workunit.client.1.smithi169.stdout:0/871: dread - d0/d2c/d38/de4/d21/d31/d83/dac/fb8 zero size 2024-09-16T00:26:17.830 INFO:tasks.workunit.client.1.smithi169.stdout:5/804: getdents d2/da/d10/d13/d1c/d40/d43 0 2024-09-16T00:26:17.830 INFO:tasks.workunit.client.1.smithi169.stdout:7/984: symlink d7/d5b/d61/l13b 0 2024-09-16T00:26:17.832 INFO:tasks.workunit.client.1.smithi169.stdout:2/920: rename d7/db/d27/d29/fd6 to d7/db/db8/daf/d10a/f14d 0 2024-09-16T00:26:17.833 INFO:tasks.workunit.client.1.smithi169.stdout:1/921: creat dc/d36/d3a/d45/d29/d4d/db6/f118 x:0 0 0 2024-09-16T00:26:17.834 INFO:tasks.workunit.client.1.smithi169.stdout:2/921: chown d7/db/d27/d29/d37/dd1/ff0 2647931 1 2024-09-16T00:26:17.834 INFO:tasks.workunit.client.1.smithi169.stdout:7/985: creat d7/dd4/f13c x:0 0 0 2024-09-16T00:26:17.835 INFO:tasks.workunit.client.1.smithi169.stdout:0/872: mknod d0/d2c/d38/c129 0 2024-09-16T00:26:17.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:17 smithi112 ceph-mon[27869]: pgmap v262: 65 pgs: 65 active+clean; 3.7 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 118 MiB/s rd, 270 MiB/s wr, 570 op/s 2024-09-16T00:26:17.844 INFO:tasks.workunit.client.1.smithi169.stdout:2/922: dwrite d7/db/d27/d29/f127 [0,4194304] 0 2024-09-16T00:26:17.849 INFO:tasks.workunit.client.1.smithi169.stdout:0/873: creat d0/d2c/d104/f12a x:0 0 0 2024-09-16T00:26:17.852 INFO:tasks.workunit.client.1.smithi169.stdout:7/986: symlink d7/d71/d76/d136/l13d 0 2024-09-16T00:26:17.859 INFO:tasks.workunit.client.1.smithi169.stdout:4/904: dwrite d2/d12/d13/d5c/f66 [0,4194304] 0 2024-09-16T00:26:17.861 INFO:tasks.workunit.client.1.smithi169.stdout:7/987: symlink d7/d71/d76/l13e 0 2024-09-16T00:26:17.861 INFO:tasks.workunit.client.1.smithi169.stdout:5/805: link d2/da/d10/d13/d1c/d9e/lef d2/da/d10/d13/d1c/d2d/d114/l117 0 2024-09-16T00:26:17.863 INFO:tasks.workunit.client.1.smithi169.stdout:2/923: link d7/d1c/d26/d96/d139/d9d/dbe/fdf d7/db/d20/d5c/d118/f14e 0 2024-09-16T00:26:17.865 INFO:tasks.workunit.client.1.smithi169.stdout:2/924: chown d7/d1c/c1f 50846 1 2024-09-16T00:26:17.866 INFO:tasks.workunit.client.1.smithi169.stdout:4/905: mkdir d2/d12/d13/d1f/d3e/d5b/d70/def/d11a/d128 0 2024-09-16T00:26:17.867 INFO:tasks.workunit.client.1.smithi169.stdout:4/906: stat d2/d12/d13/d27/f32 0 2024-09-16T00:26:17.869 INFO:tasks.workunit.client.1.smithi169.stdout:2/925: fsync d7/db/d20/d12b/f148 0 2024-09-16T00:26:17.869 INFO:tasks.workunit.client.1.smithi169.stdout:7/988: symlink d7/d5b/d61/d98/l13f 0 2024-09-16T00:26:17.870 INFO:tasks.workunit.client.1.smithi169.stdout:2/926: chown d7/d1c/d26/d96/cde 7974 1 2024-09-16T00:26:17.873 INFO:tasks.workunit.client.1.smithi169.stdout:4/907: write d2/d12/d13/d1f/d3e/d83/dc8/ff3 [102660,110748] 0 2024-09-16T00:26:17.877 INFO:tasks.workunit.client.1.smithi169.stdout:7/989: fsync d7/d5b/f6b 0 2024-09-16T00:26:17.880 INFO:tasks.workunit.client.1.smithi169.stdout:4/908: fdatasync d2/d12/d13/d1f/f52 0 2024-09-16T00:26:17.883 INFO:tasks.workunit.client.1.smithi169.stdout:2/927: mkdir d7/db/d20/d5c/d118/d14f 0 2024-09-16T00:26:17.887 INFO:tasks.workunit.client.1.smithi169.stdout:0/874: write d0/d2/d4b/fe5 [372288,120925] 0 2024-09-16T00:26:17.887 INFO:tasks.workunit.client.1.smithi169.stdout:2/928: fdatasync d7/db/d27/d29/f127 0 2024-09-16T00:26:17.888 INFO:tasks.workunit.client.1.smithi169.stdout:1/922: dwrite dc/d6c/d96/fbd [0,4194304] 0 2024-09-16T00:26:17.892 INFO:tasks.workunit.client.1.smithi169.stdout:4/909: mkdir d2/d12/d13/d1f/d3e/d5b/d70/def/d11a/d128/d129 0 2024-09-16T00:26:17.894 INFO:tasks.workunit.client.1.smithi169.stdout:1/923: read dc/d36/d3a/d45/d29/fb4 [569478,56190] 0 2024-09-16T00:26:17.897 INFO:tasks.workunit.client.1.smithi169.stdout:4/910: creat d2/d12/d13/d4e/f12a x:0 0 0 2024-09-16T00:26:17.902 INFO:tasks.workunit.client.1.smithi169.stdout:1/924: fdatasync dc/d36/d3a/d45/d29/dc2/fcf 0 2024-09-16T00:26:17.905 INFO:tasks.workunit.client.1.smithi169.stdout:0/875: link d0/d2/d8/d5/d79/d113/f121 d0/d2/d8/d5/d60/d115/f12b 0 2024-09-16T00:26:17.907 INFO:tasks.workunit.client.1.smithi169.stdout:5/806: dwrite d2/da/d10/d13/d1c/d32/d53/d83/f76 [4194304,4194304] 0 2024-09-16T00:26:17.909 INFO:tasks.workunit.client.1.smithi169.stdout:4/911: unlink d2/d12/dcc/f41 0 2024-09-16T00:26:17.915 INFO:tasks.workunit.client.1.smithi169.stdout:4/912: dread d2/d12/fba [0,4194304] 0 2024-09-16T00:26:17.916 INFO:tasks.workunit.client.1.smithi169.stdout:1/925: symlink dc/d36/d3a/d45/d29/d2a/l119 0 2024-09-16T00:26:17.926 INFO:tasks.workunit.client.1.smithi169.stdout:7/990: dread d7/d71/d76/d9c/fa9 [0,4194304] 0 2024-09-16T00:26:17.927 INFO:tasks.workunit.client.1.smithi169.stdout:5/807: rmdir d2/da 39 2024-09-16T00:26:17.928 INFO:tasks.workunit.client.1.smithi169.stdout:0/876: mkdir d0/d4c/d12c 0 2024-09-16T00:26:17.930 INFO:tasks.workunit.client.1.smithi169.stdout:4/913: fsync d2/d12/d13/d1f/d3e/d5b/d70/d7d/d102/f106 0 2024-09-16T00:26:17.935 INFO:tasks.workunit.client.1.smithi169.stdout:0/877: read d0/d2c/f4a [2745786,7824] 0 2024-09-16T00:26:17.937 INFO:tasks.workunit.client.1.smithi169.stdout:4/914: creat d2/d12/d13/d1f/d3e/d5b/d70/d115/f12b x:0 0 0 2024-09-16T00:26:17.938 INFO:tasks.workunit.client.1.smithi169.stdout:1/926: rename dc/d36/d3a/d45/d29/d4d/db6/fc3 to dc/d36/d3a/d45/f11a 0 2024-09-16T00:26:17.938 INFO:tasks.workunit.client.1.smithi169.stdout:4/915: fdatasync d2/d12/d9c/da4/fc5 0 2024-09-16T00:26:17.941 INFO:tasks.workunit.client.1.smithi169.stdout:5/808: fdatasync d2/da/d10/d13/d1c/d40/d54/f56 0 2024-09-16T00:26:17.943 INFO:tasks.workunit.client.1.smithi169.stdout:0/878: read d0/fd0 [260142,82544] 0 2024-09-16T00:26:17.952 INFO:tasks.workunit.client.1.smithi169.stdout:7/991: truncate d7/d16/d49/d74/fc5 709649 0 2024-09-16T00:26:17.952 INFO:tasks.workunit.client.1.smithi169.stdout:0/879: mkdir d0/d10d/d119/d12d 0 2024-09-16T00:26:17.953 INFO:tasks.workunit.client.1.smithi169.stdout:7/992: read d7/d16/f24 [2191675,14583] 0 2024-09-16T00:26:17.954 INFO:tasks.workunit.client.1.smithi169.stdout:1/927: mknod dc/d36/d3a/d7f/d93/d72/de9/c11b 0 2024-09-16T00:26:17.954 INFO:tasks.workunit.client.1.smithi169.stdout:7/993: stat d7/dd4 0 2024-09-16T00:26:17.958 INFO:tasks.workunit.client.1.smithi169.stdout:5/809: mknod d2/da/d10/d13/d24/de9/ddf/c118 0 2024-09-16T00:26:17.959 INFO:tasks.workunit.client.1.smithi169.stdout:1/928: dread dc/d26/f37 [0,4194304] 0 2024-09-16T00:26:17.960 INFO:tasks.workunit.client.1.smithi169.stdout:4/916: truncate d2/d12/d13/d1f/d3e/d5b/d70/f78 1434615 0 2024-09-16T00:26:17.963 INFO:tasks.workunit.client.1.smithi169.stdout:4/917: dread - d2/d12/d13/d1f/d3e/d5b/f6b zero size 2024-09-16T00:26:17.964 INFO:tasks.workunit.client.1.smithi169.stdout:5/810: truncate d2/da/d10/d13/d1c/d32/f8f 1108535 0 2024-09-16T00:26:17.965 INFO:tasks.workunit.client.1.smithi169.stdout:5/811: dread - d2/da/d10/d13/d1c/d32/ff8 zero size 2024-09-16T00:26:17.970 INFO:tasks.workunit.client.1.smithi169.stdout:1/929: read dc/fa1 [2630365,38908] 0 2024-09-16T00:26:17.972 INFO:tasks.workunit.client.1.smithi169.stdout:0/880: write d0/d4c/d82/fd7 [177476,14961] 0 2024-09-16T00:26:17.973 INFO:tasks.workunit.client.1.smithi169.stdout:4/918: mknod d2/d12/d13/d4e/c12c 0 2024-09-16T00:26:17.975 INFO:tasks.workunit.client.1.smithi169.stdout:5/812: read d2/da/d10/f3f [158678,63766] 0 2024-09-16T00:26:17.976 INFO:tasks.workunit.client.1.smithi169.stdout:7/994: creat d7/d71/d76/d29/d78/f140 x:0 0 0 2024-09-16T00:26:17.976 INFO:tasks.workunit.client.1.smithi169.stdout:1/930: mknod dc/d18/d67/d69/d7b/c11c 0 2024-09-16T00:26:17.982 INFO:tasks.workunit.client.1.smithi169.stdout:5/813: creat d2/da/d10/d13/d1c/d94/db4/f119 x:0 0 0 2024-09-16T00:26:17.982 INFO:tasks.workunit.client.1.smithi169.stdout:0/881: dread d0/d2/d8/f73 [0,4194304] 0 2024-09-16T00:26:17.985 INFO:tasks.workunit.client.1.smithi169.stdout:0/882: chown d0/d2c/d38/de4/d21/d31/d8c/fcc 682924 1 2024-09-16T00:26:17.986 INFO:tasks.workunit.client.1.smithi169.stdout:4/919: link d2/d12/d13/d4e/d50/db8/fda d2/d12/d13/d1f/d3e/d5b/d70/d115/f12d 0 2024-09-16T00:26:17.989 INFO:tasks.workunit.client.1.smithi169.stdout:5/814: mkdir d2/da/d10/d13/d1c/d32/d53/dae/d11a 0 2024-09-16T00:26:17.991 INFO:tasks.workunit.client.1.smithi169.stdout:0/883: mkdir d0/d2c/d38/de4/d21/d12e 0 2024-09-16T00:26:17.994 INFO:tasks.workunit.client.1.smithi169.stdout:5/815: creat d2/da/d67/f11b x:0 0 0 2024-09-16T00:26:17.997 INFO:tasks.workunit.client.1.smithi169.stdout:5/816: write d2/da/d10/d13/d1c/d94/dc8/f115 [247965,100067] 0 2024-09-16T00:26:18.003 INFO:tasks.workunit.client.1.smithi169.stdout:4/920: link d2/d12/d13/d1f/d3e/d5b/d70/d7d/d102/f124 d2/d12/d13/d1f/d3e/d5b/d70/d7d/f12e 0 2024-09-16T00:26:18.004 INFO:tasks.workunit.client.1.smithi169.stdout:4/921: stat d2/d12/d13/d1f/d2f/lbf 0 2024-09-16T00:26:18.008 INFO:tasks.workunit.client.1.smithi169.stdout:5/817: rename d2/da/l1b to d2/da/d10/d13/d1c/d32/d53/d4e/d89/d9a/dad/l11c 0 2024-09-16T00:26:18.009 INFO:tasks.workunit.client.1.smithi169.stdout:0/884: write d0/d2/d8/d5/d79/fae [3736119,45304] 0 2024-09-16T00:26:18.012 INFO:tasks.workunit.client.1.smithi169.stdout:4/922: dwrite d2/d12/d13/d5c/f66 [0,4194304] 0 2024-09-16T00:26:18.016 INFO:tasks.workunit.client.1.smithi169.stdout:0/885: creat d0/d2c/d38/de4/d21/f12f x:0 0 0 2024-09-16T00:26:18.019 INFO:tasks.workunit.client.1.smithi169.stdout:4/923: mknod d2/d12/d13/d5c/d10a/d120/c12f 0 2024-09-16T00:26:18.023 INFO:tasks.workunit.client.1.smithi169.stdout:5/818: dwrite d2/da/d10/d13/d1c/d2d/db2/f10e [0,4194304] 0 2024-09-16T00:26:18.029 INFO:tasks.workunit.client.1.smithi169.stdout:5/819: symlink d2/da/d10/d13/d1c/d32/d53/l11d 0 2024-09-16T00:26:18.030 INFO:tasks.workunit.client.1.smithi169.stdout:0/886: link d0/d2/d8/d5/d79/d7c/c10a d0/d2/d8/d5/d60/ddb/c130 0 2024-09-16T00:26:18.030 INFO:tasks.workunit.client.1.smithi169.stdout:5/820: stat d2/da/d10/d13/lcf 0 2024-09-16T00:26:18.033 INFO:tasks.workunit.client.1.smithi169.stdout:5/821: write d2/da/f10f [415447,33589] 0 2024-09-16T00:26:18.033 INFO:tasks.workunit.client.1.smithi169.stdout:0/887: symlink d0/d2/d8/d5/d79/d7c/l131 0 2024-09-16T00:26:18.039 INFO:tasks.workunit.client.1.smithi169.stdout:4/924: dwrite d2/d12/d13/d1f/f48 [4194304,4194304] 0 2024-09-16T00:26:18.042 INFO:tasks.workunit.client.1.smithi169.stdout:5/822: getdents d2/da/d10/d13/d1c/d2d 0 2024-09-16T00:26:18.044 INFO:tasks.workunit.client.1.smithi169.stdout:0/888: write d0/d2c/d38/de4/fe7 [916709,63328] 0 2024-09-16T00:26:18.044 INFO:tasks.workunit.client.1.smithi169.stdout:4/925: fdatasync d2/d12/d13/fb9 0 2024-09-16T00:26:18.046 INFO:tasks.workunit.client.1.smithi169.stdout:0/889: chown d0/d2/d4b/ff9 5 1 2024-09-16T00:26:18.048 INFO:tasks.workunit.client.1.smithi169.stdout:0/890: write d0/d7d/fde [553987,31762] 0 2024-09-16T00:26:18.052 INFO:tasks.workunit.client.1.smithi169.stdout:4/926: dwrite d2/d12/f2c [0,4194304] 0 2024-09-16T00:26:18.056 INFO:tasks.workunit.client.1.smithi169.stdout:5/823: dwrite d2/f4 [0,4194304] 0 2024-09-16T00:26:18.061 INFO:tasks.workunit.client.1.smithi169.stdout:5/824: dread - d2/da/d10/d13/d1c/d32/d53/d4e/fee zero size 2024-09-16T00:26:18.065 INFO:tasks.workunit.client.1.smithi169.stdout:5/825: truncate d2/da/d10/d13/d1c/d2d/f5f 497661 0 2024-09-16T00:26:18.067 INFO:tasks.workunit.client.1.smithi169.stdout:4/927: truncate d2/d12/d13/d5c/f94 312800 0 2024-09-16T00:26:18.070 INFO:tasks.workunit.client.1.smithi169.stdout:5/826: getdents d2/da/d10/d13/d1c/db1 0 2024-09-16T00:26:18.071 INFO:tasks.workunit.client.1.smithi169.stdout:4/928: creat d2/d12/d13/d5c/da6/f130 x:0 0 0 2024-09-16T00:26:18.072 INFO:tasks.workunit.client.1.smithi169.stdout:5/827: stat d2/da/d10/d13/dc0 0 2024-09-16T00:26:18.075 INFO:tasks.workunit.client.1.smithi169.stdout:5/828: mknod d2/da/d10/d13/d1c/d32/d53/d83/d7d/c11e 0 2024-09-16T00:26:18.076 INFO:tasks.workunit.client.1.smithi169.stdout:4/929: dread d2/d12/fba [0,4194304] 0 2024-09-16T00:26:18.077 INFO:tasks.workunit.client.1.smithi169.stdout:5/829: rmdir d2/da/d10/d13/d24/de9 39 2024-09-16T00:26:18.081 INFO:tasks.workunit.client.1.smithi169.stdout:4/930: dread d2/d12/d13/d5c/da6/fb0 [0,4194304] 0 2024-09-16T00:26:18.082 INFO:tasks.workunit.client.1.smithi169.stdout:4/931: write d2/dbb/d59/f11c [441937,49333] 0 2024-09-16T00:26:18.083 INFO:tasks.workunit.client.1.smithi169.stdout:4/932: fsync d2/d12/d13/d4e/f77 0 2024-09-16T00:26:18.088 INFO:tasks.workunit.client.1.smithi169.stdout:4/933: unlink d2/d12/d13/d1f/d3e/d83/daf/c5a 0 2024-09-16T00:26:18.090 INFO:tasks.workunit.client.1.smithi169.stdout:4/934: write d2/dbb/d59/f11c [340024,129247] 0 2024-09-16T00:26:18.091 INFO:tasks.workunit.client.1.smithi169.stdout:5/830: dwrite d2/da/d10/d13/d1c/d32/d53/dae/fc4 [0,4194304] 0 2024-09-16T00:26:18.100 INFO:tasks.workunit.client.1.smithi169.stdout:4/935: write d2/d12/dcc/d9e/f121 [1389005,47264] 0 2024-09-16T00:26:18.106 INFO:tasks.workunit.client.1.smithi169.stdout:5/831: dwrite d2/da/d10/d13/d1c/f6a [0,4194304] 0 2024-09-16T00:26:18.108 INFO:tasks.workunit.client.1.smithi169.stdout:5/832: chown d2/da/d10/d13/d24/f47 930 1 2024-09-16T00:26:18.110 INFO:tasks.workunit.client.1.smithi169.stdout:5/833: mknod d2/da/d10/d13/d1c/d32/d53/d83/c11f 0 2024-09-16T00:26:18.110 INFO:tasks.workunit.client.1.smithi169.stdout:4/936: write d2/d12/d13/d1f/f8e [1730629,117568] 0 2024-09-16T00:26:18.111 INFO:tasks.workunit.client.1.smithi169.stdout:5/834: chown d2/da/d10/f87 626232 1 2024-09-16T00:26:18.113 INFO:tasks.workunit.client.1.smithi169.stdout:5/835: readlink d2/da/d10/d13/d24/de9/l9d 0 2024-09-16T00:26:18.114 INFO:tasks.workunit.client.1.smithi169.stdout:4/937: mknod d2/d12/d13/d4e/d50/c131 0 2024-09-16T00:26:18.128 INFO:tasks.workunit.client.1.smithi169.stdout:5/836: dread d2/da/d10/d13/d1c/d32/d53/d4e/d89/d9a/dad/fd8 [4194304,4194304] 0 2024-09-16T00:26:18.130 INFO:tasks.workunit.client.1.smithi169.stdout:5/837: creat d2/da/d10/d13/d1c/db1/f120 x:0 0 0 2024-09-16T00:26:18.132 INFO:tasks.workunit.client.1.smithi169.stdout:5/838: stat d2/da/d10/d13/d1c/d2d/f5f 0 2024-09-16T00:26:18.133 INFO:tasks.workunit.client.1.smithi169.stdout:5/839: readlink d2/da/d10/d13/d1c/d32/d53/l5a 0 2024-09-16T00:26:18.135 INFO:tasks.workunit.client.1.smithi169.stdout:5/840: chown d2/da/d67/lec 2 1 2024-09-16T00:26:18.137 INFO:tasks.workunit.client.1.smithi169.stdout:5/841: write d2/da/d10/d13/d1c/d40/fbe [1058423,66881] 0 2024-09-16T00:26:18.138 INFO:tasks.workunit.client.1.smithi169.stdout:5/842: chown d2/da/d10/d13/d1c/d32/d53/d4e/fee 155947 1 2024-09-16T00:26:18.141 INFO:tasks.workunit.client.1.smithi169.stdout:5/843: creat d2/da/d10/d13/d1c/d94/f121 x:0 0 0 2024-09-16T00:26:18.144 INFO:tasks.workunit.client.1.smithi169.stdout:5/844: creat d2/da/d10/d13/d24/dc9/f122 x:0 0 0 2024-09-16T00:26:18.147 INFO:tasks.workunit.client.1.smithi169.stdout:5/845: fdatasync d2/da/d10/d13/d1c/d2d/f4f 0 2024-09-16T00:26:18.148 INFO:tasks.workunit.client.1.smithi169.stdout:5/846: chown d2/da/d67/l80 8607 1 2024-09-16T00:26:18.156 INFO:tasks.workunit.client.1.smithi169.stdout:5/847: dwrite d2/da/d10/d13/d1c/d40/d54/fce [0,4194304] 0 2024-09-16T00:26:18.158 INFO:tasks.workunit.client.1.smithi169.stdout:5/848: truncate d2/da/d10/d13/d1c/d40/d68/fb9 4682361 0 2024-09-16T00:26:18.169 INFO:tasks.workunit.client.1.smithi169.stdout:5/849: dwrite d2/da/f50 [0,4194304] 0 2024-09-16T00:26:18.173 INFO:tasks.workunit.client.1.smithi169.stdout:5/850: creat d2/f123 x:0 0 0 2024-09-16T00:26:18.177 INFO:tasks.workunit.client.1.smithi169.stdout:5/851: link d2/ff d2/da/d10/d13/d1c/d40/d54/f124 0 2024-09-16T00:26:18.181 INFO:tasks.workunit.client.1.smithi169.stdout:5/852: dread d2/da/d10/d13/d1c/d32/d53/dae/ff9 [0,4194304] 0 2024-09-16T00:26:18.184 INFO:tasks.workunit.client.1.smithi169.stdout:5/853: mknod d2/da/d10/d13/d1c/d40/c125 0 2024-09-16T00:26:18.186 INFO:tasks.workunit.client.1.smithi169.stdout:5/854: rmdir d2/da/d10/d13/d1c/d32/d53/d4e 39 2024-09-16T00:26:18.187 INFO:tasks.workunit.client.1.smithi169.stdout:5/855: mknod d2/da/d10/c126 0 2024-09-16T00:26:18.190 INFO:tasks.workunit.client.1.smithi169.stdout:5/856: rmdir d2/da/d10/d13/d1c/d32/d53/d4e/d89/d9a 39 2024-09-16T00:26:18.196 INFO:tasks.workunit.client.1.smithi169.stdout:5/857: dread d2/da/d10/d13/d1c/d32/d53/d4e/d89/d9a/ffc [0,4194304] 0 2024-09-16T00:26:18.206 INFO:tasks.workunit.client.1.smithi169.stdout:5/858: dwrite d2/da/d10/d13/d1c/d40/fb5 [0,4194304] 0 2024-09-16T00:26:18.207 INFO:tasks.workunit.client.1.smithi169.stdout:5/859: chown d2/da/d10/d13/d1c/d40/c125 20 1 2024-09-16T00:26:18.209 INFO:tasks.workunit.client.1.smithi169.stdout:5/860: dread - d2/da/d10/d13/d1c/d94/db4/f119 zero size 2024-09-16T00:26:18.213 INFO:tasks.workunit.client.1.smithi169.stdout:5/861: fdatasync d2/da/d10/d13/d1c/d32/d53/d4e/d89/fd3 0 2024-09-16T00:26:18.214 INFO:tasks.workunit.client.1.smithi169.stdout:5/862: write d2/f123 [638106,20731] 0 2024-09-16T00:26:18.215 INFO:tasks.workunit.client.1.smithi169.stdout:5/863: chown d2/da/l75 239 1 2024-09-16T00:26:18.217 INFO:tasks.workunit.client.1.smithi169.stdout:5/864: dread d2/da/d10/d13/d1c/d32/d53/dae/ff9 [0,4194304] 0 2024-09-16T00:26:18.219 INFO:tasks.workunit.client.1.smithi169.stdout:5/865: stat d2/da/d10/d13/d1c/d32/ld4 0 2024-09-16T00:26:18.224 INFO:tasks.workunit.client.1.smithi169.stdout:5/866: link d2/da/d10/d13/lcf d2/da/d10/d13/d1c/d2d/db2/de7/l127 0 2024-09-16T00:26:18.225 INFO:tasks.workunit.client.1.smithi169.stdout:5/867: chown d2/da/d10/d13/d1c/d94 224661 1 2024-09-16T00:26:18.231 INFO:tasks.workunit.client.1.smithi169.stdout:5/868: dwrite d2/da/d10/d13/d24/fa0 [4194304,4194304] 0 2024-09-16T00:26:18.233 INFO:tasks.workunit.client.1.smithi169.stdout:5/869: chown d2/da/d10/d13/d1c/d32/d53/d83/d7d/cf3 59 1 2024-09-16T00:26:18.367 INFO:tasks.workunit.client.1.smithi169.stdout:2/929: sync 2024-09-16T00:26:18.547 INFO:tasks.workunit.client.1.smithi169.stdout:7/995: sync 2024-09-16T00:26:18.547 INFO:tasks.workunit.client.1.smithi169.stdout:1/931: sync 2024-09-16T00:26:18.548 INFO:tasks.workunit.client.1.smithi169.stdout:7/996: chown d7/d5b/f77 24190 1 2024-09-16T00:26:18.550 INFO:tasks.workunit.client.1.smithi169.stdout:7/997: rename d7/d5b/c116 to d7/d5b/d61/c141 0 2024-09-16T00:26:18.552 INFO:tasks.workunit.client.1.smithi169.stdout:7/998: creat d7/d12d/f142 x:0 0 0 2024-09-16T00:26:18.554 INFO:tasks.workunit.client.1.smithi169.stdout:1/932: dwrite dc/f15 [4194304,4194304] 0 2024-09-16T00:26:18.555 INFO:tasks.workunit.client.1.smithi169.stdout:7/999: symlink d7/d71/d76/d136/l143 0 2024-09-16T00:26:18.556 INFO:tasks.workunit.client.1.smithi169.stdout:1/933: symlink dc/d6c/d96/l11d 0 2024-09-16T00:26:18.557 INFO:tasks.workunit.client.1.smithi169.stdout:1/934: readlink dc/d36/lbf 0 2024-09-16T00:26:18.570 INFO:tasks.workunit.client.1.smithi169.stdout:1/935: dwrite dc/d6c/fec [0,4194304] 0 2024-09-16T00:26:18.572 INFO:tasks.workunit.client.1.smithi169.stdout:1/936: truncate dc/d36/f84 3837695 0 2024-09-16T00:26:18.575 INFO:tasks.workunit.client.1.smithi169.stdout:1/937: symlink dc/d2c/l11e 0 2024-09-16T00:26:18.585 INFO:tasks.workunit.client.1.smithi169.stdout:1/938: dwrite dc/d36/d3a/fde [0,4194304] 0 2024-09-16T00:26:18.590 INFO:tasks.workunit.client.1.smithi169.stdout:1/939: link dc/d36/d3a/d45/d29/d2a/cd3 dc/d36/d39/d50/d75/c11f 0 2024-09-16T00:26:18.592 INFO:tasks.workunit.client.1.smithi169.stdout:2/930: sync 2024-09-16T00:26:18.592 INFO:tasks.workunit.client.1.smithi169.stdout:0/891: sync 2024-09-16T00:26:18.592 INFO:tasks.workunit.client.1.smithi169.stdout:5/870: sync 2024-09-16T00:26:18.593 INFO:tasks.workunit.client.1.smithi169.stdout:4/938: sync 2024-09-16T00:26:18.594 INFO:tasks.workunit.client.1.smithi169.stdout:0/892: write d0/d2c/d38/de4/d21/d31/d8c/fcc [677472,11752] 0 2024-09-16T00:26:18.595 INFO:tasks.workunit.client.1.smithi169.stdout:2/931: creat d7/db/d89/f150 x:0 0 0 2024-09-16T00:26:18.595 INFO:tasks.workunit.client.1.smithi169.stdout:5/871: dread - d2/da/d10/f87 zero size 2024-09-16T00:26:18.597 INFO:tasks.workunit.client.1.smithi169.stdout:4/939: symlink d2/d12/d13/d1f/d3e/d5b/d70/def/d11a/d128/d129/l132 0 2024-09-16T00:26:18.597 INFO:tasks.workunit.client.1.smithi169.stdout:0/893: readlink d0/d2/d8/d5/lff 0 2024-09-16T00:26:18.598 INFO:tasks.workunit.client.1.smithi169.stdout:2/932: creat d7/db/d27/d29/d37/f151 x:0 0 0 2024-09-16T00:26:18.600 INFO:tasks.workunit.client.1.smithi169.stdout:4/940: rmdir d2/d12/d13/d27 39 2024-09-16T00:26:18.603 INFO:tasks.workunit.client.1.smithi169.stdout:2/933: creat d7/db/db8/f152 x:0 0 0 2024-09-16T00:26:18.606 INFO:tasks.workunit.client.1.smithi169.stdout:5/872: write d2/da/d10/d13/d1c/d2d/f5f [744210,116831] 0 2024-09-16T00:26:18.606 INFO:tasks.workunit.client.1.smithi169.stdout:0/894: write d0/d2/d8/f48 [148056,16713] 0 2024-09-16T00:26:18.608 INFO:tasks.workunit.client.1.smithi169.stdout:4/941: dwrite d2/d12/d13/d1f/d3e/d83/dc8/ff3 [0,4194304] 0 2024-09-16T00:26:18.611 INFO:tasks.workunit.client.1.smithi169.stdout:0/895: mkdir d0/d4c/d82/d10f/d132 0 2024-09-16T00:26:18.611 INFO:tasks.workunit.client.1.smithi169.stdout:2/934: link d7/d1c/l74 d7/d1c/de6/d135/d105/l153 0 2024-09-16T00:26:18.613 INFO:tasks.workunit.client.1.smithi169.stdout:0/896: chown d0/d2c/d38/de4/d21/f12f 26220 1 2024-09-16T00:26:18.613 INFO:tasks.workunit.client.1.smithi169.stdout:5/873: creat d2/da/d10/d13/d1c/d71/f128 x:0 0 0 2024-09-16T00:26:18.614 INFO:tasks.workunit.client.1.smithi169.stdout:2/935: rmdir d7/db/d89/dfb 39 2024-09-16T00:26:18.616 INFO:tasks.workunit.client.1.smithi169.stdout:4/942: dwrite d2/d12/d13/d1f/f8e [0,4194304] 0 2024-09-16T00:26:18.622 INFO:tasks.workunit.client.1.smithi169.stdout:0/897: rename d0/d2c/d38/de4/f61 to d0/d4c/d82/d10f/f133 0 2024-09-16T00:26:18.631 INFO:tasks.workunit.client.1.smithi169.stdout:4/943: dread d2/d12/d13/d1f/d2f/f43 [0,4194304] 0 2024-09-16T00:26:18.634 INFO:tasks.workunit.client.1.smithi169.stdout:4/944: unlink d2/dbb/d59/fe6 0 2024-09-16T00:26:18.635 INFO:tasks.workunit.client.1.smithi169.stdout:2/936: dread d7/d1c/d26/f76 [0,4194304] 0 2024-09-16T00:26:18.637 INFO:tasks.workunit.client.1.smithi169.stdout:2/937: mkdir d7/d6b/d154 0 2024-09-16T00:26:18.638 INFO:tasks.workunit.client.1.smithi169.stdout:5/874: write d2/da/d10/d13/d1c/d40/d68/fc1 [313796,105229] 0 2024-09-16T00:26:18.639 INFO:tasks.workunit.client.1.smithi169.stdout:2/938: fsync d7/db/db8/f152 0 2024-09-16T00:26:18.639 INFO:tasks.workunit.client.1.smithi169.stdout:0/898: write d0/d4c/d82/d10f/f70 [319136,79695] 0 2024-09-16T00:26:18.642 INFO:tasks.workunit.client.1.smithi169.stdout:0/899: truncate d0/d2/d8/d5/ffa 1083459 0 2024-09-16T00:26:18.642 INFO:tasks.workunit.client.1.smithi169.stdout:2/939: truncate d7/d1c/d26/d96/d139/d9d/dbe/f145 567822 0 2024-09-16T00:26:18.644 INFO:tasks.workunit.client.1.smithi169.stdout:0/900: dread - d0/d2/d4b/d86/fca zero size 2024-09-16T00:26:18.644 INFO:tasks.workunit.client.1.smithi169.stdout:2/940: symlink d7/d1c/l155 0 2024-09-16T00:26:18.645 INFO:tasks.workunit.client.1.smithi169.stdout:0/901: readlink d0/d4c/d82/l9a 0 2024-09-16T00:26:18.645 INFO:tasks.workunit.client.1.smithi169.stdout:2/941: dread - d7/db/d20/d12b/f148 zero size 2024-09-16T00:26:18.646 INFO:tasks.workunit.client.1.smithi169.stdout:0/902: stat d0/d2/d8/d5/fc9 0 2024-09-16T00:26:18.648 INFO:tasks.workunit.client.1.smithi169.stdout:2/942: write d7/db/d27/d29/d77/fad [4694599,29774] 0 2024-09-16T00:26:18.650 INFO:tasks.workunit.client.1.smithi169.stdout:0/903: dread d0/fd0 [0,4194304] 0 2024-09-16T00:26:18.651 INFO:tasks.workunit.client.1.smithi169.stdout:2/943: truncate d7/d1c/dc3/ff3 603367 0 2024-09-16T00:26:18.651 INFO:tasks.workunit.client.1.smithi169.stdout:4/945: dread d2/d12/d13/d4e/d50/f114 [0,4194304] 0 2024-09-16T00:26:18.652 INFO:tasks.workunit.client.1.smithi169.stdout:2/944: fsync d7/db/db8/daf/d10a/f117 0 2024-09-16T00:26:18.653 INFO:tasks.workunit.client.1.smithi169.stdout:4/946: stat d2/d12/d13/d27/dfd 0 2024-09-16T00:26:18.656 INFO:tasks.workunit.client.1.smithi169.stdout:0/904: truncate d0/d2c/d38/de4/f3b 2541160 0 2024-09-16T00:26:18.656 INFO:tasks.workunit.client.1.smithi169.stdout:5/875: dread d2/da/d10/d13/d1c/d40/d68/fb9 [0,4194304] 0 2024-09-16T00:26:18.657 INFO:tasks.workunit.client.1.smithi169.stdout:0/905: chown d0/d4c/d82/fd7 48934 1 2024-09-16T00:26:18.657 INFO:tasks.workunit.client.1.smithi169.stdout:2/945: rmdir d7/db/db8/daf/d10a/d11b/d114 39 2024-09-16T00:26:18.660 INFO:tasks.workunit.client.1.smithi169.stdout:5/876: write d2/da/d10/d13/d1c/d94/db4/f119 [118544,13950] 0 2024-09-16T00:26:18.662 INFO:tasks.workunit.client.1.smithi169.stdout:5/877: readlink d2/da/d10/d13/d24/de9/l9d 0 2024-09-16T00:26:18.663 INFO:tasks.workunit.client.1.smithi169.stdout:4/947: dwrite d2/d12/d13/d4e/f12a [0,4194304] 0 2024-09-16T00:26:18.668 INFO:tasks.workunit.client.1.smithi169.stdout:2/946: fsync d7/db/d27/d29/fc0 0 2024-09-16T00:26:18.669 INFO:tasks.workunit.client.1.smithi169.stdout:0/906: stat d0/d2c/d38/de4/d21/d31/d83/c123 0 2024-09-16T00:26:18.673 INFO:tasks.workunit.client.1.smithi169.stdout:4/948: write d2/d12/fbe [1023657,86645] 0 2024-09-16T00:26:18.690 INFO:tasks.workunit.client.1.smithi169.stdout:2/947: chown d7/d1c/de6/d135/c7a 2 1 2024-09-16T00:26:18.690 INFO:tasks.workunit.client.1.smithi169.stdout:5/878: symlink d2/da/d10/d13/d1c/d2d/l129 0 2024-09-16T00:26:18.696 INFO:tasks.workunit.client.1.smithi169.stdout:2/948: rmdir d7/db 39 2024-09-16T00:26:18.697 INFO:tasks.workunit.client.1.smithi169.stdout:2/949: write d7/d1c/d26/d96/d139/f113 [635774,49597] 0 2024-09-16T00:26:18.698 INFO:tasks.workunit.client.1.smithi169.stdout:2/950: chown d7/daa/df5/f53 0 1 2024-09-16T00:26:18.701 INFO:tasks.workunit.client.1.smithi169.stdout:2/951: creat d7/db/d27/d29/d37/f156 x:0 0 0 2024-09-16T00:26:18.704 INFO:tasks.workunit.client.1.smithi169.stdout:2/952: truncate d7/d1c/d26/d96/d139/d69/fac 1514863 0 2024-09-16T00:26:18.704 INFO:tasks.workunit.client.1.smithi169.stdout:2/953: fdatasync d7/f55 0 2024-09-16T00:26:18.707 INFO:tasks.workunit.client.1.smithi169.stdout:2/954: creat d7/db/db8/d6d/f157 x:0 0 0 2024-09-16T00:26:18.712 INFO:tasks.workunit.client.1.smithi169.stdout:4/949: write d2/dbb/d59/fa0 [516530,8886] 0 2024-09-16T00:26:18.716 INFO:tasks.workunit.client.1.smithi169.stdout:2/955: write d7/daa/df5/ff2 [621469,101781] 0 2024-09-16T00:26:18.720 INFO:tasks.workunit.client.1.smithi169.stdout:0/907: dwrite d0/d2/d8/d5/f2e [0,4194304] 0 2024-09-16T00:26:18.720 INFO:tasks.workunit.client.1.smithi169.stdout:5/879: dwrite d2/da/d10/d13/d1c/d2d/f57 [0,4194304] 0 2024-09-16T00:26:18.721 INFO:tasks.workunit.client.1.smithi169.stdout:2/956: write d7/db/db8/daf/d10a/d11b/f146 [147772,121903] 0 2024-09-16T00:26:18.723 INFO:tasks.workunit.client.1.smithi169.stdout:4/950: link d2/d12/d13/d1f/d3e/d5b/d70/d115/f12b d2/d12/d13/d1f/d3e/dfb/f133 0 2024-09-16T00:26:18.724 INFO:tasks.workunit.client.1.smithi169.stdout:5/880: stat d2/da/d10/d13/d1c/d40/c125 0 2024-09-16T00:26:18.725 INFO:tasks.workunit.client.1.smithi169.stdout:0/908: unlink d0/d2c/d38/de4/f28 0 2024-09-16T00:26:18.725 INFO:tasks.workunit.client.1.smithi169.stdout:4/951: dread - d2/d12/d9c/da4/fc5 zero size 2024-09-16T00:26:18.725 INFO:tasks.workunit.client.1.smithi169.stdout:0/909: fdatasync d0/d2/f93 0 2024-09-16T00:26:18.730 INFO:tasks.workunit.client.1.smithi169.stdout:5/881: symlink d2/da/d10/d13/d1c/d32/d53/d4e/l12a 0 2024-09-16T00:26:18.731 INFO:tasks.workunit.client.1.smithi169.stdout:4/952: mkdir d2/d12/d13/d4e/d50/db8/d108/d134 0 2024-09-16T00:26:18.732 INFO:tasks.workunit.client.1.smithi169.stdout:4/953: dread - d2/d12/d9c/dc2/ff4 zero size 2024-09-16T00:26:18.735 INFO:tasks.workunit.client.1.smithi169.stdout:4/954: write d2/d12/d13/d1f/d3e/f111 [477369,29820] 0 2024-09-16T00:26:18.737 INFO:tasks.workunit.client.1.smithi169.stdout:4/955: chown d2/d12/d13/d1f 0 1 2024-09-16T00:26:18.738 INFO:tasks.workunit.client.1.smithi169.stdout:5/882: dread d2/da/d10/d13/d1c/d40/fb5 [0,4194304] 0 2024-09-16T00:26:18.742 INFO:tasks.workunit.client.1.smithi169.stdout:4/956: mknod d2/dbb/d59/c135 0 2024-09-16T00:26:18.742 INFO:tasks.workunit.client.1.smithi169.stdout:2/957: dread d7/db/db8/daf/d10a/d11b/f125 [0,4194304] 0 2024-09-16T00:26:18.742 INFO:tasks.workunit.client.1.smithi169.stdout:5/883: creat d2/da/d10/d13/dc0/f12b x:0 0 0 2024-09-16T00:26:18.745 INFO:tasks.workunit.client.1.smithi169.stdout:0/910: dwrite d0/d2c/d38/de4/d21/f49 [0,4194304] 0 2024-09-16T00:26:18.746 INFO:tasks.workunit.client.1.smithi169.stdout:2/958: symlink d7/db/d89/l158 0 2024-09-16T00:26:18.749 INFO:tasks.workunit.client.1.smithi169.stdout:2/959: write d7/d1c/d26/f36 [2631699,19270] 0 2024-09-16T00:26:18.752 INFO:tasks.workunit.client.1.smithi169.stdout:4/957: rename d2/d12/d13/d1f/f35 to d2/d12/d13/d1f/d2f/f136 0 2024-09-16T00:26:18.754 INFO:tasks.workunit.client.1.smithi169.stdout:5/884: mknod d2/da/d10/d13/d1c/d32/d53/dae/de2/c12c 0 2024-09-16T00:26:18.754 INFO:tasks.workunit.client.1.smithi169.stdout:5/885: write d2/da/f10f [1264252,21510] 0 2024-09-16T00:26:18.755 INFO:tasks.workunit.client.1.smithi169.stdout:0/911: unlink d0/c122 0 2024-09-16T00:26:18.756 INFO:tasks.workunit.client.1.smithi169.stdout:2/960: creat d7/daa/df5/f159 x:0 0 0 2024-09-16T00:26:18.757 INFO:tasks.workunit.client.1.smithi169.stdout:5/886: rmdir d2/da/d10/d13/d1c/d40/d68 39 2024-09-16T00:26:18.760 INFO:tasks.workunit.client.1.smithi169.stdout:5/887: symlink d2/da/d10/d13/d1c/db1/l12d 0 2024-09-16T00:26:18.761 INFO:tasks.workunit.client.1.smithi169.stdout:4/958: creat d2/d12/d13/d1f/d3e/f137 x:0 0 0 2024-09-16T00:26:18.764 INFO:tasks.workunit.client.1.smithi169.stdout:4/959: chown d2/d12/d13/d1f/d3e/d5b/l11e 0 1 2024-09-16T00:26:18.764 INFO:tasks.workunit.client.1.smithi169.stdout:0/912: write d0/d2/d8/d5/d60/d115/f12b [122619,58560] 0 2024-09-16T00:26:18.764 INFO:tasks.workunit.client.1.smithi169.stdout:2/961: dwrite d7/db/d27/d29/d77/fad [0,4194304] 0 2024-09-16T00:26:18.766 INFO:tasks.workunit.client.1.smithi169.stdout:2/962: fsync d7/d1c/d26/f36 0 2024-09-16T00:26:18.767 INFO:tasks.workunit.client.1.smithi169.stdout:5/888: symlink d2/da/d10/d13/d1c/d32/d53/d83/d7d/l12e 0 2024-09-16T00:26:18.773 INFO:tasks.workunit.client.1.smithi169.stdout:0/913: symlink d0/d2/d8/d5/d60/ddb/l134 0 2024-09-16T00:26:18.773 INFO:tasks.workunit.client.1.smithi169.stdout:5/889: read d2/da/d10/d13/d1c/d40/d54/d7f/fd9 [2004921,45919] 0 2024-09-16T00:26:18.773 INFO:tasks.workunit.client.1.smithi169.stdout:4/960: creat d2/d12/d13/d5c/da6/d104/f138 x:0 0 0 2024-09-16T00:26:18.775 INFO:tasks.workunit.client.1.smithi169.stdout:4/961: write d2/d12/d13/d5c/da6/d104/f138 [956292,57889] 0 2024-09-16T00:26:18.778 INFO:tasks.workunit.client.1.smithi169.stdout:4/962: mknod d2/d12/d13/d1f/d3e/d5b/d70/def/d11a/d128/c139 0 2024-09-16T00:26:18.779 INFO:tasks.workunit.client.1.smithi169.stdout:4/963: stat d2/d12/d13/d4e/d50/db8/d108/d134 0 2024-09-16T00:26:18.782 INFO:tasks.workunit.client.1.smithi169.stdout:2/963: write d7/d1c/f72 [1147559,57749] 0 2024-09-16T00:26:18.783 INFO:tasks.workunit.client.1.smithi169.stdout:5/890: write d2/f58 [792597,92588] 0 2024-09-16T00:26:18.790 INFO:tasks.workunit.client.1.smithi169.stdout:5/891: symlink d2/da/d10/d13/d24/l12f 0 2024-09-16T00:26:18.790 INFO:tasks.workunit.client.1.smithi169.stdout:4/964: dwrite d2/d12/d13/d5c/f10f [0,4194304] 0 2024-09-16T00:26:18.792 INFO:tasks.workunit.client.1.smithi169.stdout:0/914: dwrite d0/d2c/d104/d67/fa1 [0,4194304] 0 2024-09-16T00:26:18.793 INFO:tasks.workunit.client.1.smithi169.stdout:0/915: chown d0/l108 6 1 2024-09-16T00:26:18.795 INFO:tasks.workunit.client.1.smithi169.stdout:2/964: truncate d7/d1c/d26/d96/d139/d69/fac 176291 0 2024-09-16T00:26:18.796 INFO:tasks.workunit.client.1.smithi169.stdout:4/965: creat d2/d12/d13/d1f/d3e/d5b/d70/d7d/f13a x:0 0 0 2024-09-16T00:26:18.796 INFO:tasks.workunit.client.1.smithi169.stdout:5/892: mkdir d2/da/d10/d13/d1c/d40/d54/d7f/d130 0 2024-09-16T00:26:18.796 INFO:tasks.workunit.client.1.smithi169.stdout:0/916: write d0/d2c/d38/f84 [4926845,121702] 0 2024-09-16T00:26:18.798 INFO:tasks.workunit.client.1.smithi169.stdout:5/893: fsync d2/da/d10/d13/d1c/d40/fb5 0 2024-09-16T00:26:18.799 INFO:tasks.workunit.client.1.smithi169.stdout:2/965: dread d7/db/d27/d29/fc9 [0,4194304] 0 2024-09-16T00:26:18.802 INFO:tasks.workunit.client.1.smithi169.stdout:0/917: rename d0/d2c/d104/d67/lad to d0/d2c/d38/de4/d21/d31/dfe/l135 0 2024-09-16T00:26:18.804 INFO:tasks.workunit.client.1.smithi169.stdout:0/918: stat d0/d2/d15/f7a 0 2024-09-16T00:26:18.805 INFO:tasks.workunit.client.1.smithi169.stdout:5/894: getdents d2/da/d10/d13/d1c/db1 0 2024-09-16T00:26:18.811 INFO:tasks.workunit.client.1.smithi169.stdout:2/966: dwrite d7/db/d27/d29/d37/f156 [0,4194304] 0 2024-09-16T00:26:18.815 INFO:tasks.workunit.client.1.smithi169.stdout:2/967: unlink d7/d6b/d70/lea 0 2024-09-16T00:26:18.819 INFO:tasks.workunit.client.1.smithi169.stdout:4/966: truncate d2/dbb/d59/fea 1801137 0 2024-09-16T00:26:18.823 INFO:tasks.workunit.client.1.smithi169.stdout:5/895: dread d2/da/d10/d13/d1c/d32/d53/d4e/d89/d9a/dad/fd8 [0,4194304] 0 2024-09-16T00:26:18.824 INFO:tasks.workunit.client.1.smithi169.stdout:2/968: dwrite d7/d6b/d70/d79/dce/fd8 [0,4194304] 0 2024-09-16T00:26:18.825 INFO:tasks.workunit.client.1.smithi169.stdout:2/969: chown d7/d1c/la5 25155363 1 2024-09-16T00:26:18.826 INFO:tasks.workunit.client.1.smithi169.stdout:4/967: symlink d2/d12/d13/d1f/d3e/d5b/d70/d7d/d102/d11f/l13b 0 2024-09-16T00:26:18.827 INFO:tasks.workunit.client.1.smithi169.stdout:4/968: readlink d2/l5f 0 2024-09-16T00:26:18.829 INFO:tasks.workunit.client.1.smithi169.stdout:5/896: write d2/da/d10/d13/d1c/d32/d53/fb0 [242319,16151] 0 2024-09-16T00:26:18.836 INFO:tasks.workunit.client.1.smithi169.stdout:5/897: creat d2/da/d10/d13/d1c/d40/f131 x:0 0 0 2024-09-16T00:26:18.838 INFO:tasks.workunit.client.1.smithi169.stdout:4/969: rename d2/d12/d13/d1f/d3e/dfb/feb to d2/d12/d13/d5c/f13c 0 2024-09-16T00:26:18.838 INFO:tasks.workunit.client.1.smithi169.stdout:0/919: dread d0/d2/d8/d5/d60/fd9 [0,4194304] 0 2024-09-16T00:26:18.840 INFO:tasks.workunit.client.1.smithi169.stdout:5/898: unlink d2/f88 0 2024-09-16T00:26:18.841 INFO:tasks.workunit.client.1.smithi169.stdout:1/940: sync 2024-09-16T00:26:18.844 INFO:tasks.workunit.client.1.smithi169.stdout:0/920: dread d0/d2/d8/d5/fa [0,4194304] 0 2024-09-16T00:26:18.855 INFO:tasks.workunit.client.1.smithi169.stdout:1/941: getdents dc/d36/d3a/d45/d29/d4d 0 2024-09-16T00:26:18.857 INFO:tasks.workunit.client.1.smithi169.stdout:5/899: dwrite d2/da/d10/d13/d1c/d32/d53/d4e/d89/d9a/dad/fc6 [0,4194304] 0 2024-09-16T00:26:18.863 INFO:tasks.workunit.client.1.smithi169.stdout:0/921: link d0/d2c/d4f/fe6 d0/d2/d8/d5/d60/f136 0 2024-09-16T00:26:18.863 INFO:tasks.workunit.client.1.smithi169.stdout:1/942: chown dc/d36/d39/d50/f85 26312 1 2024-09-16T00:26:18.863 INFO:tasks.workunit.client.1.smithi169.stdout:4/970: truncate d2/d12/d13/d1f/d3e/d83/dc8/ff3 2134757 0 2024-09-16T00:26:18.867 INFO:tasks.workunit.client.1.smithi169.stdout:2/970: dwrite d7/daa/df5/f53 [0,4194304] 0 2024-09-16T00:26:18.870 INFO:tasks.workunit.client.1.smithi169.stdout:4/971: chown d2/dbb/f84 4139625 1 2024-09-16T00:26:18.873 INFO:tasks.workunit.client.1.smithi169.stdout:1/943: symlink dc/l120 0 2024-09-16T00:26:18.875 INFO:tasks.workunit.client.1.smithi169.stdout:2/971: write d7/d1c/de6/d135/d105/d11f/f133 [1026583,52324] 0 2024-09-16T00:26:18.876 INFO:tasks.workunit.client.1.smithi169.stdout:5/900: chown d2/da/d10/d13/d1c/d94/dc8/ccb 45278224 1 2024-09-16T00:26:18.877 INFO:tasks.workunit.client.1.smithi169.stdout:0/922: dwrite d0/d2/d8/d5/d79/f116 [0,4194304] 0 2024-09-16T00:26:18.877 INFO:tasks.workunit.client.1.smithi169.stdout:2/972: dread - d7/db/d89/f150 zero size 2024-09-16T00:26:18.878 INFO:tasks.workunit.client.1.smithi169.stdout:5/901: chown d2/f123 726788544 1 2024-09-16T00:26:18.879 INFO:tasks.workunit.client.1.smithi169.stdout:0/923: stat d0/d2c/d38/de4/d21/d31/d8c/ca7 0 2024-09-16T00:26:18.903 INFO:tasks.workunit.client.1.smithi169.stdout:1/944: truncate dc/d36/fff 812528 0 2024-09-16T00:26:18.907 INFO:tasks.workunit.client.1.smithi169.stdout:1/945: read dc/d36/d39/d50/daf/f47 [3946190,7035] 0 2024-09-16T00:26:18.913 INFO:tasks.workunit.client.1.smithi169.stdout:5/902: dread d2/da/d10/d13/f35 [0,4194304] 0 2024-09-16T00:26:18.955 INFO:tasks.workunit.client.1.smithi169.stdout:1/946: truncate dc/d36/d3a/d7f/fb8 383691 0 2024-09-16T00:26:18.958 INFO:tasks.workunit.client.1.smithi169.stdout:2/973: getdents d7/d6b/d70/d79/dae/d100 0 2024-09-16T00:26:18.963 INFO:tasks.workunit.client.1.smithi169.stdout:1/947: creat dc/d36/d3a/d45/d29/dc2/dc7/f121 x:0 0 0 2024-09-16T00:26:18.966 INFO:tasks.workunit.client.1.smithi169.stdout:2/974: creat d7/db/d20/d5c/d118/d14f/f15a x:0 0 0 2024-09-16T00:26:18.969 INFO:tasks.workunit.client.1.smithi169.stdout:4/972: dread d2/d12/d13/d1f/d3e/d5b/d70/d7d/fab [0,4194304] 0 2024-09-16T00:26:18.969 INFO:tasks.workunit.client.1.smithi169.stdout:2/975: truncate f0 2911814 0 2024-09-16T00:26:18.970 INFO:tasks.workunit.client.1.smithi169.stdout:2/976: read f2 [724362,46395] 0 2024-09-16T00:26:18.972 INFO:tasks.workunit.client.1.smithi169.stdout:2/977: dread d7/daa/df5/f80 [0,4194304] 0 2024-09-16T00:26:18.983 INFO:tasks.workunit.client.1.smithi169.stdout:4/973: mkdir d2/d12/dcc/d13d 0 2024-09-16T00:26:18.987 INFO:tasks.workunit.client.1.smithi169.stdout:4/974: mknod d2/d12/d13/d5c/d10a/d120/c13e 0 2024-09-16T00:26:18.990 INFO:tasks.workunit.client.1.smithi169.stdout:5/903: symlink d2/da/d10/d13/d1c/d2d/l132 0 2024-09-16T00:26:18.990 INFO:tasks.workunit.client.1.smithi169.stdout:2/978: link d7/db/d27/d29/c64 d7/db/d27/c15b 0 2024-09-16T00:26:18.993 INFO:tasks.workunit.client.1.smithi169.stdout:2/979: write d7/db/db8/d6d/f71 [2353665,13293] 0 2024-09-16T00:26:18.995 INFO:tasks.workunit.client.1.smithi169.stdout:5/904: dread d2/da/d10/d13/d1c/d94/fd1 [0,4194304] 0 2024-09-16T00:26:18.998 INFO:tasks.workunit.client.1.smithi169.stdout:5/905: truncate d2/da/d10/d13/d1c/d94/dc8/f116 959236 0 2024-09-16T00:26:18.998 INFO:tasks.workunit.client.1.smithi169.stdout:5/906: readlink d2/da/d10/l18 0 2024-09-16T00:26:19.026 INFO:tasks.workunit.client.1.smithi169.stdout:0/924: dwrite d0/d7d/fbb [0,4194304] 0 2024-09-16T00:26:19.027 INFO:tasks.workunit.client.1.smithi169.stdout:2/980: creat d7/d1c/d26/d96/d139/d9d/f15c x:0 0 0 2024-09-16T00:26:19.029 INFO:tasks.workunit.client.1.smithi169.stdout:0/925: write d0/d2/d8/d5/d79/fae [3649011,124872] 0 2024-09-16T00:26:19.030 INFO:tasks.workunit.client.1.smithi169.stdout:0/926: chown d0/d2/d8/l9 12419239 1 2024-09-16T00:26:19.035 INFO:tasks.workunit.client.1.smithi169.stdout:0/927: unlink d0/d2c/d38/de4/d21/d31/fc7 0 2024-09-16T00:26:19.036 INFO:tasks.workunit.client.1.smithi169.stdout:2/981: dwrite d7/db/f13 [0,4194304] 0 2024-09-16T00:26:19.041 INFO:tasks.workunit.client.1.smithi169.stdout:2/982: mkdir d7/d1c/d26/d96/d139/d9d/d123/d15d 0 2024-09-16T00:26:19.042 INFO:tasks.workunit.client.1.smithi169.stdout:2/983: write d7/d6b/d70/d79/dae/f143 [240279,39393] 0 2024-09-16T00:26:19.047 INFO:tasks.workunit.client.1.smithi169.stdout:2/984: creat d7/db/d20/d5c/d14c/f15e x:0 0 0 2024-09-16T00:26:19.050 INFO:tasks.workunit.client.1.smithi169.stdout:4/975: rename d2/d12/d13/d1f/d3e/f53 to d2/d12/d13/d1f/d3e/d5b/d70/def/f13f 0 2024-09-16T00:26:19.052 INFO:tasks.workunit.client.1.smithi169.stdout:2/985: mkdir d7/d1c/d15f 0 2024-09-16T00:26:19.054 INFO:tasks.workunit.client.1.smithi169.stdout:1/948: creat dc/d36/d3a/d45/d29/dc2/f122 x:0 0 0 2024-09-16T00:26:19.054 INFO:tasks.workunit.client.1.smithi169.stdout:4/976: symlink d2/d12/d13/d1f/d3e/d5b/d70/l140 0 2024-09-16T00:26:19.057 INFO:tasks.workunit.client.1.smithi169.stdout:2/986: creat d7/d1c/d26/d96/d139/d69/d92/d12a/f160 x:0 0 0 2024-09-16T00:26:19.058 INFO:tasks.workunit.client.1.smithi169.stdout:1/949: dread dc/d36/d3a/d7f/fb8 [0,4194304] 0 2024-09-16T00:26:19.060 INFO:tasks.workunit.client.1.smithi169.stdout:5/907: rename d2/f1f to d2/da/d10/d13/d1c/f133 0 2024-09-16T00:26:19.061 INFO:tasks.workunit.client.1.smithi169.stdout:4/977: mkdir d2/d12/d13/d1f/d2f/de7/d141 0 2024-09-16T00:26:19.065 INFO:tasks.workunit.client.1.smithi169.stdout:0/928: rename d0/d4c/d6f/fda to d0/d2/d15/f137 0 2024-09-16T00:26:19.065 INFO:tasks.workunit.client.1.smithi169.stdout:1/950: dread dc/d18/d67/f94 [0,4194304] 0 2024-09-16T00:26:19.066 INFO:tasks.workunit.client.1.smithi169.stdout:2/987: dwrite d7/d6b/f116 [0,4194304] 0 2024-09-16T00:26:19.067 INFO:tasks.workunit.client.1.smithi169.stdout:5/908: fsync d2/da/d10/d13/d1c/d32/d53/dae/de2/f113 0 2024-09-16T00:26:19.069 INFO:tasks.workunit.client.1.smithi169.stdout:1/951: write dc/d6c/d96/fbd [4541909,29968] 0 2024-09-16T00:26:19.070 INFO:tasks.workunit.client.1.smithi169.stdout:5/909: write d2/da/d10/d13/d1c/d32/d53/fb0 [770038,50741] 0 2024-09-16T00:26:19.071 INFO:tasks.workunit.client.1.smithi169.stdout:4/978: creat d2/d12/d13/d1f/d3e/d5b/d70/def/f142 x:0 0 0 2024-09-16T00:26:19.071 INFO:tasks.workunit.client.1.smithi169.stdout:5/910: chown d2/l12 103 1 2024-09-16T00:26:19.072 INFO:tasks.workunit.client.1.smithi169.stdout:0/929: readlink d0/d2c/d38/de4/l89 0 2024-09-16T00:26:19.073 INFO:tasks.workunit.client.1.smithi169.stdout:0/930: stat d0/d2/d4b/f88 0 2024-09-16T00:26:19.075 INFO:tasks.workunit.client.1.smithi169.stdout:2/988: truncate d7/db/db8/f57 2049294 0 2024-09-16T00:26:19.076 INFO:tasks.workunit.client.1.smithi169.stdout:2/989: chown d7/db/db8 7408870 1 2024-09-16T00:26:19.078 INFO:tasks.workunit.client.1.smithi169.stdout:2/990: fsync d7/d1c/f72 0 2024-09-16T00:26:19.079 INFO:tasks.workunit.client.1.smithi169.stdout:2/991: chown d7/db/d27/d29/d37 8355 1 2024-09-16T00:26:19.080 INFO:tasks.workunit.client.1.smithi169.stdout:4/979: dwrite d2/d12/d13/d1f/d3e/f111 [0,4194304] 0 2024-09-16T00:26:19.082 INFO:tasks.workunit.client.1.smithi169.stdout:4/980: truncate d2/d12/fbe 2154654 0 2024-09-16T00:26:19.082 INFO:tasks.workunit.client.1.smithi169.stdout:5/911: mkdir d2/d7c/d134 0 2024-09-16T00:26:19.092 INFO:tasks.workunit.client.1.smithi169.stdout:0/931: mkdir d0/d138 0 2024-09-16T00:26:19.093 INFO:tasks.workunit.client.1.smithi169.stdout:1/952: dread dc/f66 [0,4194304] 0 2024-09-16T00:26:19.095 INFO:tasks.workunit.client.1.smithi169.stdout:5/912: rename d2/da/d10/d13/d1c/d94/lac to d2/da/d10/d13/d1c/d32/d53/dae/d11a/l135 0 2024-09-16T00:26:19.096 INFO:tasks.workunit.client.1.smithi169.stdout:2/992: creat d7/d6b/d70/d79/dae/d126/f161 x:0 0 0 2024-09-16T00:26:19.097 INFO:tasks.workunit.client.1.smithi169.stdout:0/932: mkdir d0/d2/d15/d139 0 2024-09-16T00:26:19.098 INFO:tasks.workunit.client.1.smithi169.stdout:5/913: creat d2/da/d10/d13/d1c/d94/db4/f136 x:0 0 0 2024-09-16T00:26:19.099 INFO:tasks.workunit.client.1.smithi169.stdout:2/993: symlink d7/d1c/d26/d96/d139/l162 0 2024-09-16T00:26:19.100 INFO:tasks.workunit.client.1.smithi169.stdout:0/933: mkdir d0/d4c/d82/d10f/d13a 0 2024-09-16T00:26:19.101 INFO:tasks.workunit.client.1.smithi169.stdout:0/934: readlink d0/d2c/d38/de4/l72 0 2024-09-16T00:26:19.106 INFO:tasks.workunit.client.1.smithi169.stdout:2/994: dread d7/daa/df5/f53 [0,4194304] 0 2024-09-16T00:26:19.108 INFO:tasks.workunit.client.1.smithi169.stdout:0/935: rename d0/d2/d8/d5/f44 to d0/d2/d8/d5/d79/d113/f13b 0 2024-09-16T00:26:19.110 INFO:tasks.workunit.client.1.smithi169.stdout:1/953: dread dc/d36/d3a/d45/fa7 [0,4194304] 0 2024-09-16T00:26:19.110 INFO:tasks.workunit.client.1.smithi169.stdout:2/995: creat d7/d1c/de6/f163 x:0 0 0 2024-09-16T00:26:19.110 INFO:tasks.workunit.client.1.smithi169.stdout:4/981: write d2/d12/d13/d1f/d3e/d5b/f6b [457448,68579] 0 2024-09-16T00:26:19.112 INFO:tasks.workunit.client.1.smithi169.stdout:1/954: write dc/d36/d3a/d45/d29/f8d [172784,40682] 0 2024-09-16T00:26:19.117 INFO:tasks.workunit.client.1.smithi169.stdout:2/996: creat d7/d1c/d15f/f164 x:0 0 0 2024-09-16T00:26:19.118 INFO:tasks.workunit.client.1.smithi169.stdout:4/982: truncate d2/d12/d13/d1f/d2f/f45 837439 0 2024-09-16T00:26:19.120 INFO:tasks.workunit.client.1.smithi169.stdout:2/997: creat d7/d1c/de6/d135/d142/f165 x:0 0 0 2024-09-16T00:26:19.125 INFO:tasks.workunit.client.1.smithi169.stdout:0/936: dread d0/d2c/f5d [0,4194304] 0 2024-09-16T00:26:19.127 INFO:tasks.workunit.client.1.smithi169.stdout:5/914: dwrite d2/da/d10/d13/d1c/d32/d53/d83/d7d/fa1 [0,4194304] 0 2024-09-16T00:26:19.128 INFO:tasks.workunit.client.1.smithi169.stdout:5/915: stat d2/da/d10/d13 0 2024-09-16T00:26:19.129 INFO:tasks.workunit.client.1.smithi169.stdout:4/983: truncate d2/d12/d13/d4e/d50/f85 1241117 0 2024-09-16T00:26:19.133 INFO:tasks.workunit.client.1.smithi169.stdout:2/998: link d7/db/db8/daf/d10a/f117 d7/db/db8/daf/d10a/d11b/d114/f166 0 2024-09-16T00:26:19.133 INFO:tasks.workunit.client.1.smithi169.stdout:1/955: dread dc/d36/d39/d50/daf/d86/f51 [0,4194304] 0 2024-09-16T00:26:19.135 INFO:tasks.workunit.client.1.smithi169.stdout:4/984: creat d2/d12/d13/d90/f143 x:0 0 0 2024-09-16T00:26:19.136 INFO:tasks.workunit.client.1.smithi169.stdout:5/916: rename d2/da/d10/d13/d1c/d32/d53/d83 to d2/da/d10/d13/d1c/d71/d137 0 2024-09-16T00:26:19.137 INFO:tasks.workunit.client.1.smithi169.stdout:5/917: chown d2/da/d10/d13/dc0 227767 1 2024-09-16T00:26:19.138 INFO:tasks.workunit.client.1.smithi169.stdout:4/985: rmdir d2/d12/d13/d1f/d3e/dfb 39 2024-09-16T00:26:19.139 INFO:tasks.workunit.client.1.smithi169.stdout:2/999: creat d7/d1c/d26/d96/d139/d9d/d123/d15d/f167 x:0 0 0 2024-09-16T00:26:19.140 INFO:tasks.workunit.client.1.smithi169.stdout:5/918: chown d2/da/d10/d13/d1c/d71/d137/d7d/f8b 249577 1 2024-09-16T00:26:19.147 INFO:tasks.workunit.client.1.smithi169.stdout:5/919: dwrite d2/da/d10/d13/d1c/d2d/db2/f10e [0,4194304] 0 2024-09-16T00:26:19.148 INFO:tasks.workunit.client.1.smithi169.stdout:5/920: fsync d2/da/d10/d13/dc0/ff2 0 2024-09-16T00:26:19.150 INFO:tasks.workunit.client.1.smithi169.stdout:4/986: write d2/d12/dcc/d9e/f71 [1829990,127573] 0 2024-09-16T00:26:19.153 INFO:tasks.workunit.client.1.smithi169.stdout:4/987: fdatasync d2/d12/d13/d1f/f118 0 2024-09-16T00:26:19.153 INFO:tasks.workunit.client.1.smithi169.stdout:5/921: dread d2/da/d10/d13/d1c/d2d/f4f [0,4194304] 0 2024-09-16T00:26:19.157 INFO:tasks.workunit.client.1.smithi169.stdout:1/956: dwrite dc/fd [4194304,4194304] 0 2024-09-16T00:26:19.161 INFO:tasks.workunit.client.1.smithi169.stdout:5/922: dwrite d2/da/f50 [0,4194304] 0 2024-09-16T00:26:19.166 INFO:tasks.workunit.client.1.smithi169.stdout:5/923: write d2/da/d10/d13/d1c/d32/d53/dae/de2/f108 [556637,79046] 0 2024-09-16T00:26:19.169 INFO:tasks.workunit.client.1.smithi169.stdout:5/924: write d2/da/d10/d13/d1c/d71/d137/f76 [2034025,70365] 0 2024-09-16T00:26:19.171 INFO:tasks.workunit.client.1.smithi169.stdout:4/988: dwrite d2/d12/d13/d1f/d3e/d5b/f24 [0,4194304] 0 2024-09-16T00:26:19.172 INFO:tasks.workunit.client.1.smithi169.stdout:5/925: write d2/da/d10/d13/d1c/d32/d53/d4e/fee [807694,40234] 0 2024-09-16T00:26:19.174 INFO:tasks.workunit.client.1.smithi169.stdout:4/989: chown d2/d12/d13/d1f/d3e/d83/f74 10741 1 2024-09-16T00:26:19.176 INFO:tasks.workunit.client.1.smithi169.stdout:4/990: chown d2/d12/d13/d1f 690 1 2024-09-16T00:26:19.180 INFO:tasks.workunit.client.1.smithi169.stdout:5/926: rmdir d2/da/d10/d13/d1c/d40/d68 39 2024-09-16T00:26:19.180 INFO:tasks.workunit.client.1.smithi169.stdout:5/927: chown d2/da/d10/d13/d24/de9/ddf/c118 126105791 1 2024-09-16T00:26:19.181 INFO:tasks.workunit.client.1.smithi169.stdout:5/928: unlink d2/da/d10/f3f 0 2024-09-16T00:26:19.184 INFO:tasks.workunit.client.1.smithi169.stdout:5/929: creat d2/da/d10/d13/d1c/d32/d53/d4e/f138 x:0 0 0 2024-09-16T00:26:19.184 INFO:tasks.workunit.client.1.smithi169.stdout:5/930: stat d2/da/f38 0 2024-09-16T00:26:19.191 INFO:tasks.workunit.client.1.smithi169.stdout:4/991: write d2/d12/d13/d1f/d3e/d83/fa5 [1076331,64074] 0 2024-09-16T00:26:19.192 INFO:tasks.workunit.client.1.smithi169.stdout:5/931: truncate d2/da/d10/d13/d1c/d40/d54/d7f/fd9 1593557 0 2024-09-16T00:26:19.194 INFO:tasks.workunit.client.1.smithi169.stdout:4/992: unlink d2/d12/d13/d4e/d50/f100 0 2024-09-16T00:26:19.195 INFO:tasks.workunit.client.1.smithi169.stdout:4/993: chown d2/d12/d13/d1f/d3e/d5b/d70/d115/f12b 46 1 2024-09-16T00:26:19.195 INFO:tasks.workunit.client.1.smithi169.stdout:4/994: stat d2/d12/dcc/f47 0 2024-09-16T00:26:19.196 INFO:tasks.workunit.client.1.smithi169.stdout:4/995: write d2/d12/d13/d27/fe8 [823687,53520] 0 2024-09-16T00:26:19.197 INFO:tasks.workunit.client.1.smithi169.stdout:5/932: link d2/da/d10/d13/d1c/d40/d54/d7f/lbd d2/da/d10/d13/d24/de9/ddf/l139 0 2024-09-16T00:26:19.199 INFO:tasks.workunit.client.1.smithi169.stdout:4/996: creat d2/f144 x:0 0 0 2024-09-16T00:26:19.200 INFO:tasks.workunit.client.1.smithi169.stdout:4/997: chown d2/d12/d9c/de2 1 1 2024-09-16T00:26:19.202 INFO:tasks.workunit.client.1.smithi169.stdout:4/998: creat d2/d12/d13/d1f/d3e/d5b/d70/d7d/d102/f145 x:0 0 0 2024-09-16T00:26:19.204 INFO:tasks.workunit.client.1.smithi169.stdout:4/999: creat d2/d12/d13/d5c/da6/d104/f146 x:0 0 0 2024-09-16T00:26:19.205 INFO:tasks.workunit.client.1.smithi169.stdout:5/933: truncate d2/f58 180559 0 2024-09-16T00:26:19.340 INFO:tasks.workunit.client.1.smithi169.stdout:5/934: dread d2/da/d10/d13/d24/f47 [4194304,4194304] 0 2024-09-16T00:26:19.768 INFO:tasks.workunit.client.1.smithi169.stdout:0/937: sync 2024-09-16T00:26:19.770 INFO:tasks.workunit.client.1.smithi169.stdout:0/938: write d0/d2/d8/d5/d60/d115/f12b [1071916,38045] 0 2024-09-16T00:26:19.772 INFO:tasks.workunit.client.1.smithi169.stdout:0/939: truncate d0/d2/d8/d5/d79/d7c/f11b 737978 0 2024-09-16T00:26:19.775 INFO:tasks.workunit.client.1.smithi169.stdout:0/940: creat d0/d4c/f13c x:0 0 0 2024-09-16T00:26:19.777 INFO:tasks.workunit.client.1.smithi169.stdout:0/941: truncate d0/d2c/d38/fc6 573937 0 2024-09-16T00:26:19.780 INFO:tasks.workunit.client.1.smithi169.stdout:0/942: rename d0/d4c/d82/d10f/f70 to d0/d2/d8/d5/d60/ddb/f13d 0 2024-09-16T00:26:19.781 INFO:tasks.workunit.client.1.smithi169.stdout:0/943: chown d0/d2/d8/d5 3976 1 2024-09-16T00:26:19.782 INFO:tasks.workunit.client.1.smithi169.stdout:0/944: dread - d0/d4c/d82/fd3 zero size 2024-09-16T00:26:19.783 INFO:tasks.workunit.client.1.smithi169.stdout:0/945: readlink d0/d2c/d38/de4/d21/d31/d83/dac/lb6 0 2024-09-16T00:26:19.785 INFO:tasks.workunit.client.1.smithi169.stdout:0/946: symlink d0/d4c/d82/d10f/l13e 0 2024-09-16T00:26:19.792 INFO:tasks.workunit.client.1.smithi169.stdout:0/947: dwrite d0/d2/d8/d5/d79/f96 [0,4194304] 0 2024-09-16T00:26:19.793 INFO:tasks.workunit.client.1.smithi169.stdout:0/948: rename d0/d4c/d82/d10f/d6c/d9b/f102 to d0/d10d/f13f 0 2024-09-16T00:26:19.796 INFO:tasks.workunit.client.1.smithi169.stdout:0/949: creat d0/d2/d8/d5/d79/d113/f140 x:0 0 0 2024-09-16T00:26:19.797 INFO:tasks.workunit.client.1.smithi169.stdout:0/950: chown d0/d2c/d38/de4/d21/fa0 320422 1 2024-09-16T00:26:19.825 INFO:tasks.workunit.client.1.smithi169.stdout:0/951: dwrite d0/d2/d15/fc1 [4194304,4194304] 0 2024-09-16T00:26:19.825 INFO:tasks.workunit.client.1.smithi169.stdout:1/957: sync 2024-09-16T00:26:19.826 INFO:tasks.workunit.client.1.smithi169.stdout:5/935: sync 2024-09-16T00:26:19.827 INFO:tasks.workunit.client.1.smithi169.stdout:1/958: chown dc/d18/l31 1 1 2024-09-16T00:26:19.828 INFO:tasks.workunit.client.1.smithi169.stdout:0/952: mknod d0/d2/d8/d5/c141 0 2024-09-16T00:26:19.829 INFO:tasks.workunit.client.1.smithi169.stdout:0/953: chown d0/d2c/d104/d67/fa1 11634 1 2024-09-16T00:26:19.831 INFO:tasks.workunit.client.1.smithi169.stdout:5/936: creat d2/da/d10/d13/d1c/d40/d54/d7f/f13a x:0 0 0 2024-09-16T00:26:19.832 INFO:tasks.workunit.client.1.smithi169.stdout:0/954: truncate d0/d2c/d38/de4/fec 1022025 0 2024-09-16T00:26:19.833 INFO:tasks.workunit.client.1.smithi169.stdout:5/937: fsync d2/da/d10/d13/d1c/d2d/f63 0 2024-09-16T00:26:19.834 INFO:tasks.workunit.client.1.smithi169.stdout:1/959: link dc/d36/l116 dc/d6c/d96/l123 0 2024-09-16T00:26:19.835 INFO:tasks.workunit.client.1.smithi169.stdout:1/960: chown dc/d18/d67/f49 3039596 1 2024-09-16T00:26:19.836 INFO:tasks.workunit.client.1.smithi169.stdout:0/955: symlink d0/d4c/l142 0 2024-09-16T00:26:19.837 INFO:tasks.workunit.client.1.smithi169.stdout:5/938: creat d2/da/d67/f13b x:0 0 0 2024-09-16T00:26:19.838 INFO:tasks.workunit.client.1.smithi169.stdout:0/956: fdatasync d0/d4c/d82/d10f/f9f 0 2024-09-16T00:26:19.838 INFO:tasks.workunit.client.1.smithi169.stdout:1/961: symlink dc/d36/d39/d50/daf/d88/da5/l124 0 2024-09-16T00:26:19.839 INFO:tasks.workunit.client.1.smithi169.stdout:5/939: chown d2/da/d10/d13/d1c/d94/dc8/f115 2091854 1 2024-09-16T00:26:19.840 INFO:tasks.workunit.client.1.smithi169.stdout:0/957: write d0/d2c/d38/de4/d21/d31/d83/dac/ff0 [880559,115935] 0 2024-09-16T00:26:19.844 INFO:tasks.workunit.client.1.smithi169.stdout:0/958: creat d0/d2/d8/f143 x:0 0 0 2024-09-16T00:26:19.845 INFO:tasks.workunit.client.1.smithi169.stdout:0/959: readlink d0/d2c/ld5 0 2024-09-16T00:26:19.849 INFO:tasks.workunit.client.1.smithi169.stdout:5/940: dwrite d2/da/d10/d13/d1c/d40/d43/dfb/fff [0,4194304] 0 2024-09-16T00:26:19.851 INFO:tasks.workunit.client.1.smithi169.stdout:1/962: dwrite dc/d36/d39/d50/daf/d86/f51 [4194304,4194304] 0 2024-09-16T00:26:19.855 INFO:tasks.workunit.client.1.smithi169.stdout:5/941: write d2/da/d10/d13/d1c/d40/d43/dfb/fff [1142774,37697] 0 2024-09-16T00:26:19.862 INFO:tasks.workunit.client.1.smithi169.stdout:1/963: dwrite dc/d36/d3a/d45/d29/fb4 [0,4194304] 0 2024-09-16T00:26:19.865 INFO:tasks.workunit.client.1.smithi169.stdout:0/960: dwrite d0/d2/d8/d5/d60/fd9 [0,4194304] 0 2024-09-16T00:26:19.866 INFO:tasks.workunit.client.1.smithi169.stdout:5/942: fsync d2/da/d10/d13/d1c/d40/d54/d7f/fd9 0 2024-09-16T00:26:19.870 INFO:tasks.workunit.client.1.smithi169.stdout:0/961: truncate d0/d2/d8/d5/d79/d113/f13b 1261850 0 2024-09-16T00:26:19.893 INFO:tasks.workunit.client.1.smithi169.stdout:5/943: write d2/da/f38 [570191,74504] 0 2024-09-16T00:26:19.899 INFO:tasks.workunit.client.1.smithi169.stdout:5/944: symlink d2/da/d10/d13/d1c/d40/d68/l13c 0 2024-09-16T00:26:19.899 INFO:tasks.workunit.client.1.smithi169.stdout:0/962: dwrite d0/d2/f45 [0,4194304] 0 2024-09-16T00:26:19.901 INFO:tasks.workunit.client.1.smithi169.stdout:1/964: dread dc/d26/f37 [0,4194304] 0 2024-09-16T00:26:19.904 INFO:tasks.workunit.client.1.smithi169.stdout:1/965: creat dc/d6c/f125 x:0 0 0 2024-09-16T00:26:19.908 INFO:tasks.workunit.client.1.smithi169.stdout:0/963: dread d0/d2c/d38/f84 [4194304,4194304] 0 2024-09-16T00:26:19.908 INFO:tasks.workunit.client.1.smithi169.stdout:5/945: dwrite d2/da/f38 [0,4194304] 0 2024-09-16T00:26:19.912 INFO:tasks.workunit.client.1.smithi169.stdout:1/966: link dc/l79 dc/d36/d3a/d7f/ddb/l126 0 2024-09-16T00:26:19.914 INFO:tasks.workunit.client.1.smithi169.stdout:1/967: creat dc/d36/d3a/d45/dce/f127 x:0 0 0 2024-09-16T00:26:19.915 INFO:tasks.workunit.client.1.smithi169.stdout:1/968: fdatasync dc/d36/d3a/d45/d29/d4d/f9c 0 2024-09-16T00:26:19.917 INFO:tasks.workunit.client.1.smithi169.stdout:1/969: write dc/d36/d3a/d45/d29/f8d [693145,72786] 0 2024-09-16T00:26:19.919 INFO:tasks.workunit.client.1.smithi169.stdout:1/970: creat dc/d36/d3a/f128 x:0 0 0 2024-09-16T00:26:19.922 INFO:tasks.workunit.client.1.smithi169.stdout:5/946: read d2/da/d10/d13/d1c/d71/d137/f76 [1421259,126321] 0 2024-09-16T00:26:19.924 INFO:tasks.workunit.client.1.smithi169.stdout:5/947: readlink d2/da/d10/d13/d1c/d9e/lef 0 2024-09-16T00:26:19.926 INFO:tasks.workunit.client.1.smithi169.stdout:5/948: symlink d2/da/d10/d13/d24/de9/l13d 0 2024-09-16T00:26:19.927 INFO:tasks.workunit.client.1.smithi169.stdout:5/949: unlink d2/da/d10/d13/d24/c52 0 2024-09-16T00:26:19.939 INFO:tasks.workunit.client.1.smithi169.stdout:1/971: write dc/d18/d67/fc8 [964067,113658] 0 2024-09-16T00:26:19.941 INFO:tasks.workunit.client.1.smithi169.stdout:5/950: truncate d2/da/f50 2471628 0 2024-09-16T00:26:19.946 INFO:tasks.workunit.client.1.smithi169.stdout:0/964: dwrite d0/d2c/d38/de4/d21/d31/d83/f114 [0,4194304] 0 2024-09-16T00:26:19.951 INFO:tasks.workunit.client.1.smithi169.stdout:0/965: mkdir d0/d2/d8/d5/d79/d7c/d144 0 2024-09-16T00:26:19.952 INFO:tasks.workunit.client.1.smithi169.stdout:1/972: link dc/d36/d3a/d45/l2d dc/d18/d67/d69/l129 0 2024-09-16T00:26:19.955 INFO:tasks.workunit.client.1.smithi169.stdout:1/973: mknod dc/d2c/c12a 0 2024-09-16T00:26:19.955 INFO:tasks.workunit.client.1.smithi169.stdout:1/974: chown dc/f1d 55227744 1 2024-09-16T00:26:19.958 INFO:tasks.workunit.client.1.smithi169.stdout:1/975: unlink dc/d36/d3a/f76 0 2024-09-16T00:26:19.961 INFO:tasks.workunit.client.1.smithi169.stdout:1/976: creat dc/d6c/f12b x:0 0 0 2024-09-16T00:26:19.963 INFO:tasks.workunit.client.1.smithi169.stdout:1/977: symlink dc/d36/d3a/d7f/d93/l12c 0 2024-09-16T00:26:19.963 INFO:tasks.workunit.client.1.smithi169.stdout:1/978: chown dc/d18/d67/d69 57736970 1 2024-09-16T00:26:19.966 INFO:tasks.workunit.client.1.smithi169.stdout:1/979: creat dc/d36/d3a/d45/d29/dc2/dee/f12d x:0 0 0 2024-09-16T00:26:19.968 INFO:tasks.workunit.client.1.smithi169.stdout:1/980: dread - dc/ff5 zero size 2024-09-16T00:26:19.972 INFO:tasks.workunit.client.1.smithi169.stdout:5/951: dwrite d2/da/d10/d13/d1c/d32/d53/d4e/fe3 [0,4194304] 0 2024-09-16T00:26:19.978 INFO:tasks.workunit.client.1.smithi169.stdout:0/966: dwrite d0/d10d/f13f [0,4194304] 0 2024-09-16T00:26:19.980 INFO:tasks.workunit.client.1.smithi169.stdout:1/981: getdents dc/d18 0 2024-09-16T00:26:19.983 INFO:tasks.workunit.client.1.smithi169.stdout:5/952: mkdir d2/da/d10/d13/d1c/d13e 0 2024-09-16T00:26:19.988 INFO:tasks.workunit.client.1.smithi169.stdout:0/967: dread d0/d10d/f13f [0,4194304] 0 2024-09-16T00:26:19.990 INFO:tasks.workunit.client.1.smithi169.stdout:5/953: mknod d2/da/d10/d13/d1c/d40/d54/c13f 0 2024-09-16T00:26:19.993 INFO:tasks.workunit.client.1.smithi169.stdout:1/982: dwrite dc/d36/d39/d50/daf/d88/dd1/ffd [0,4194304] 0 2024-09-16T00:26:19.996 INFO:tasks.workunit.client.1.smithi169.stdout:0/968: write d0/d2c/d38/de4/d21/d31/d83/f114 [3827266,50807] 0 2024-09-16T00:26:19.998 INFO:tasks.workunit.client.1.smithi169.stdout:1/983: symlink dc/d36/d3a/d7f/d93/d72/l12e 0 2024-09-16T00:26:19.998 INFO:tasks.workunit.client.1.smithi169.stdout:1/984: chown dc/d18/d67/fc8 665 1 2024-09-16T00:26:20.000 INFO:tasks.workunit.client.1.smithi169.stdout:1/985: unlink dc/f1d 0 2024-09-16T00:26:20.003 INFO:tasks.workunit.client.1.smithi169.stdout:1/986: creat dc/d36/d3a/d45/d29/d4d/db6/d10f/f12f x:0 0 0 2024-09-16T00:26:20.018 INFO:tasks.workunit.client.1.smithi169.stdout:1/987: dwrite dc/d36/d39/d50/f7a [4194304,4194304] 0 2024-09-16T00:26:20.022 INFO:tasks.workunit.client.1.smithi169.stdout:1/988: creat dc/d36/d3a/d7f/d93/d72/df7/f130 x:0 0 0 2024-09-16T00:26:20.025 INFO:tasks.workunit.client.1.smithi169.stdout:1/989: mkdir dc/d36/d39/d50/d75/dfa/d131 0 2024-09-16T00:26:20.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:19 smithi169 ceph-mon[34653]: pgmap v263: 65 pgs: 65 active+clean; 3.7 GiB data, 13 GiB used, 524 GiB / 536 GiB avail; 128 MiB/s rd, 303 MiB/s wr, 715 op/s 2024-09-16T00:26:20.032 INFO:tasks.workunit.client.1.smithi169.stdout:0/969: dread d0/d2c/d38/de4/d21/f2f [0,4194304] 0 2024-09-16T00:26:20.032 INFO:tasks.workunit.client.1.smithi169.stdout:0/970: readlink d0/l108 0 2024-09-16T00:26:20.035 INFO:tasks.workunit.client.1.smithi169.stdout:1/990: dread dc/d36/d39/d50/daf/d86/fb2 [0,4194304] 0 2024-09-16T00:26:20.036 INFO:tasks.workunit.client.1.smithi169.stdout:1/991: write dc/d36/d39/d50/daf/d86/f51 [2000753,66706] 0 2024-09-16T00:26:20.037 INFO:tasks.workunit.client.1.smithi169.stdout:1/992: chown dc/d36/d3a/d7f/fd9 723627 1 2024-09-16T00:26:20.039 INFO:tasks.workunit.client.1.smithi169.stdout:1/993: mkdir dc/d36/d39/d50/daf/d86/d132 0 2024-09-16T00:26:20.050 INFO:tasks.workunit.client.1.smithi169.stdout:1/994: dwrite dc/d36/d39/d50/d75/fa8 [0,4194304] 0 2024-09-16T00:26:20.052 INFO:tasks.workunit.client.1.smithi169.stdout:1/995: readlink dc/l79 0 2024-09-16T00:26:20.055 INFO:tasks.workunit.client.1.smithi169.stdout:1/996: creat dc/d36/d39/d50/daf/d88/da5/f133 x:0 0 0 2024-09-16T00:26:20.056 INFO:tasks.workunit.client.1.smithi169.stdout:1/997: chown dc/d36/d3a/d45/d29/d2a/ld7 100 1 2024-09-16T00:26:20.059 INFO:tasks.workunit.client.1.smithi169.stdout:1/998: creat dc/d36/d39/d50/daf/f134 x:0 0 0 2024-09-16T00:26:20.060 INFO:tasks.workunit.client.1.smithi169.stdout:1/999: creat dc/d36/d3a/d45/d29/d2a/f135 x:0 0 0 2024-09-16T00:26:20.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:19 smithi112 ceph-mon[27869]: pgmap v263: 65 pgs: 65 active+clean; 3.7 GiB data, 13 GiB used, 524 GiB / 536 GiB avail; 128 MiB/s rd, 303 MiB/s wr, 715 op/s 2024-09-16T00:26:20.279 INFO:tasks.workunit.client.1.smithi169.stdout:5/954: sync 2024-09-16T00:26:20.279 INFO:tasks.workunit.client.1.smithi169.stdout:0/971: sync 2024-09-16T00:26:20.280 INFO:tasks.workunit.client.1.smithi169.stdout:0/972: fdatasync d0/d2c/d104/f12a 0 2024-09-16T00:26:20.286 INFO:tasks.workunit.client.1.smithi169.stdout:5/955: dwrite d2/da/d10/d13/d1c/d94/f121 [0,4194304] 0 2024-09-16T00:26:20.287 INFO:tasks.workunit.client.1.smithi169.stdout:5/956: dread - d2/d7c/f105 zero size 2024-09-16T00:26:20.290 INFO:tasks.workunit.client.1.smithi169.stdout:0/973: dwrite d0/d2/d8/f68 [4194304,4194304] 0 2024-09-16T00:26:20.292 INFO:tasks.workunit.client.1.smithi169.stdout:0/974: stat d0/d2/d8/d5/c3e 0 2024-09-16T00:26:20.296 INFO:tasks.workunit.client.1.smithi169.stdout:5/957: link d2/da/d67/f13b d2/da/d10/d13/d1c/d71/d137/d7d/d96/f140 0 2024-09-16T00:26:20.297 INFO:tasks.workunit.client.1.smithi169.stdout:0/975: rename d0/d2c/d38/de4/cdd to d0/d4c/d82/d10f/d6c/d9b/c145 0 2024-09-16T00:26:20.298 INFO:tasks.workunit.client.1.smithi169.stdout:0/976: rename d0 to d0/d2c/d4f/d146 22 2024-09-16T00:26:20.311 INFO:tasks.workunit.client.1.smithi169.stdout:0/977: getdents d0/d2c/d38/de4 0 2024-09-16T00:26:20.312 INFO:tasks.workunit.client.1.smithi169.stdout:5/958: write d2/da/d10/f98 [5014568,44327] 0 2024-09-16T00:26:20.312 INFO:tasks.workunit.client.1.smithi169.stdout:0/978: chown d0/d2/d15/f22 3531614 1 2024-09-16T00:26:20.314 INFO:tasks.workunit.client.1.smithi169.stdout:5/959: dread - d2/da/d10/d13/d1c/d40/d68/fda zero size 2024-09-16T00:26:20.316 INFO:tasks.workunit.client.1.smithi169.stdout:0/979: dread d0/d2/d8/d5/d79/f96 [0,4194304] 0 2024-09-16T00:26:20.316 INFO:tasks.workunit.client.1.smithi169.stdout:5/960: unlink d2/da/d10/d13/dc0/ff2 0 2024-09-16T00:26:20.317 INFO:tasks.workunit.client.1.smithi169.stdout:5/961: chown d2/da/d10/d13/d1c/d71/d137/fa4 0 1 2024-09-16T00:26:20.320 INFO:tasks.workunit.client.1.smithi169.stdout:0/980: truncate d0/d2c/d38/de4/f1c 368708 0 2024-09-16T00:26:20.322 INFO:tasks.workunit.client.1.smithi169.stdout:0/981: creat d0/d2/d8/d5/d79/d7c/d144/f147 x:0 0 0 2024-09-16T00:26:20.322 INFO:tasks.workunit.client.1.smithi169.stdout:5/962: dwrite d2/f4 [0,4194304] 0 2024-09-16T00:26:20.324 INFO:tasks.workunit.client.1.smithi169.stdout:5/963: dread - d2/da/d10/d13/d1c/d40/d54/d7f/f13a zero size 2024-09-16T00:26:20.327 INFO:tasks.workunit.client.1.smithi169.stdout:5/964: fsync d2/da/d10/d13/d1c/d94/fd1 0 2024-09-16T00:26:20.327 INFO:tasks.workunit.client.1.smithi169.stdout:0/982: link d0/d2c/d110/f109 d0/d10d/d119/f148 0 2024-09-16T00:26:20.330 INFO:tasks.workunit.client.1.smithi169.stdout:5/965: chown d2/da/d10/d13/d1c/d32/d53/fdd 174 1 2024-09-16T00:26:20.330 INFO:tasks.workunit.client.1.smithi169.stdout:5/966: fsync d2/da/f10f 0 2024-09-16T00:26:20.332 INFO:tasks.workunit.client.1.smithi169.stdout:5/967: chown d2/da/d10/d13/d1c/d71/d137/c11f 6347571 1 2024-09-16T00:26:20.332 INFO:tasks.workunit.client.1.smithi169.stdout:0/983: creat d0/d2c/d38/f149 x:0 0 0 2024-09-16T00:26:20.334 INFO:tasks.workunit.client.1.smithi169.stdout:0/984: dread - d0/d4c/ff3 zero size 2024-09-16T00:26:20.336 INFO:tasks.workunit.client.1.smithi169.stdout:0/985: rmdir d0/d2c 39 2024-09-16T00:26:20.341 INFO:tasks.workunit.client.1.smithi169.stdout:0/986: dread d0/d2c/f5d [0,4194304] 0 2024-09-16T00:26:20.341 INFO:tasks.workunit.client.1.smithi169.stdout:5/968: dread d2/da/d10/d13/d24/de9/ddf/f106 [0,4194304] 0 2024-09-16T00:26:20.343 INFO:tasks.workunit.client.1.smithi169.stdout:0/987: fsync d0/d2/d8/d5/d60/f95 0 2024-09-16T00:26:20.343 INFO:tasks.workunit.client.1.smithi169.stdout:0/988: fdatasync d0/d2/d15/fc1 0 2024-09-16T00:26:20.345 INFO:tasks.workunit.client.1.smithi169.stdout:0/989: creat d0/d2/d8/d5/d79/d7c/f14a x:0 0 0 2024-09-16T00:26:20.346 INFO:tasks.workunit.client.1.smithi169.stdout:5/969: write d2/da/d10/d13/d24/de9/ddf/f106 [573533,107024] 0 2024-09-16T00:26:20.347 INFO:tasks.workunit.client.1.smithi169.stdout:5/970: fdatasync d2/da/f38 0 2024-09-16T00:26:20.351 INFO:tasks.workunit.client.1.smithi169.stdout:0/990: dread d0/d2/d8/d5/d60/ddb/f13d [0,4194304] 0 2024-09-16T00:26:20.352 INFO:tasks.workunit.client.1.smithi169.stdout:0/991: rmdir d0/d2c/d104 39 2024-09-16T00:26:20.353 INFO:tasks.workunit.client.1.smithi169.stdout:0/992: symlink d0/d2c/d38/de4/d21/d12e/l14b 0 2024-09-16T00:26:20.355 INFO:tasks.workunit.client.1.smithi169.stdout:0/993: truncate d0/d2/d4b/d86/f5a 1540497 0 2024-09-16T00:26:20.358 INFO:tasks.workunit.client.1.smithi169.stdout:0/994: unlink d0/d2c/d38/de4/d21/d31/dfe/l135 0 2024-09-16T00:26:20.370 INFO:tasks.workunit.client.1.smithi169.stdout:0/995: dread d0/d2c/d38/de4/d21/fa0 [0,4194304] 0 2024-09-16T00:26:20.370 INFO:tasks.workunit.client.1.smithi169.stdout:0/996: write d0/d2/d15/fc1 [8041000,34292] 0 2024-09-16T00:26:20.373 INFO:tasks.workunit.client.1.smithi169.stdout:0/997: mkdir d0/d2/d8/d5/d79/d7c/d14c 0 2024-09-16T00:26:20.374 INFO:tasks.workunit.client.1.smithi169.stdout:0/998: symlink d0/d2c/d38/de4/d21/d31/d83/l14d 0 2024-09-16T00:26:20.375 INFO:tasks.workunit.client.1.smithi169.stdout:0/999: stat d0/d2/d8/d5/d79/d7c/f14a 0 2024-09-16T00:26:20.509 INFO:tasks.workunit.client.1.smithi169.stdout:5/971: sync 2024-09-16T00:26:20.509 INFO:tasks.workunit.client.1.smithi169.stdout:5/972: stat d2/da/d10/d13/d1c/d32/d53/dae/ff9 0 2024-09-16T00:26:20.511 INFO:tasks.workunit.client.1.smithi169.stdout:5/973: mkdir d2/da/d10/d13/d1c/d94/db4/d141 0 2024-09-16T00:26:20.512 INFO:tasks.workunit.client.1.smithi169.stdout:5/974: mknod d2/da/d10/d13/d24/de9/c142 0 2024-09-16T00:26:20.513 INFO:tasks.workunit.client.1.smithi169.stdout:5/975: chown d2/da/d10/d13/d1c/d71/f128 1 1 2024-09-16T00:26:20.516 INFO:tasks.workunit.client.1.smithi169.stdout:5/976: truncate d2/da/d10/d13/d1c/d32/d53/d4e/fa8 714524 0 2024-09-16T00:26:20.524 INFO:tasks.workunit.client.1.smithi169.stdout:5/977: rmdir d2/da/d10 39 2024-09-16T00:26:20.527 INFO:tasks.workunit.client.1.smithi169.stdout:5/978: mkdir d2/da/d10/d13/d1c/d2d/d114/d143 0 2024-09-16T00:26:20.529 INFO:tasks.workunit.client.1.smithi169.stdout:5/979: write d2/da/d10/d13/d1c/d94/f99 [1625403,113390] 0 2024-09-16T00:26:20.683 INFO:tasks.workunit.client.1.smithi169.stdout:5/980: sync 2024-09-16T00:26:20.708 INFO:tasks.workunit.client.1.smithi169.stdout:5/981: creat d2/da/d10/d13/f144 x:0 0 0 2024-09-16T00:26:20.718 INFO:tasks.workunit.client.1.smithi169.stdout:5/982: dread d2/da/d10/f27 [0,4194304] 0 2024-09-16T00:26:20.720 INFO:tasks.workunit.client.1.smithi169.stdout:5/983: mknod d2/da/d10/d13/d24/de9/c145 0 2024-09-16T00:26:20.723 INFO:tasks.workunit.client.1.smithi169.stdout:5/984: link d2/da/d10/d13/d1c/db1/l12d d2/da/d10/d13/d1c/d71/d137/d7d/d96/l146 0 2024-09-16T00:26:20.725 INFO:tasks.workunit.client.1.smithi169.stdout:5/985: mkdir d2/da/d10/d13/d1c/d94/db4/d141/d147 0 2024-09-16T00:26:20.727 INFO:tasks.workunit.client.1.smithi169.stdout:5/986: mknod d2/da/d10/d13/d1c/d2d/db2/c148 0 2024-09-16T00:26:20.727 INFO:tasks.workunit.client.1.smithi169.stdout:5/987: readlink d2/da/d10/d13/d1c/d32/ld4 0 2024-09-16T00:26:20.728 INFO:tasks.workunit.client.1.smithi169.stdout:5/988: fsync d2/da/f38 0 2024-09-16T00:26:20.729 INFO:tasks.workunit.client.1.smithi169.stdout:5/989: dread - d2/da/d10/d13/d1c/d32/d53/d4e/d89/f103 zero size 2024-09-16T00:26:20.730 INFO:tasks.workunit.client.1.smithi169.stdout:5/990: rmdir d2/da/d10/d13/d1c/d40/d43/dfb 39 2024-09-16T00:26:20.733 INFO:tasks.workunit.client.1.smithi169.stdout:5/991: creat d2/da/d10/d13/d1c/db1/f149 x:0 0 0 2024-09-16T00:26:20.736 INFO:tasks.workunit.client.1.smithi169.stdout:5/992: creat d2/da/d10/d13/d1c/d32/d53/d4e/f14a x:0 0 0 2024-09-16T00:26:20.739 INFO:tasks.workunit.client.1.smithi169.stdout:5/993: link d2/da/d10/d13/d1c/d71/ldc d2/da/d10/d13/d1c/d71/l14b 0 2024-09-16T00:26:20.744 INFO:tasks.workunit.client.1.smithi169.stdout:5/994: dwrite d2/da/f38 [0,4194304] 0 2024-09-16T00:26:20.746 INFO:tasks.workunit.client.1.smithi169.stdout:5/995: rmdir d2/da/d10/d13/d1c/d32/d53/d4e/d89 39 2024-09-16T00:26:20.781 INFO:tasks.workunit.client.1.smithi169.stdout:5/996: write d2/da/d10/d13/d1c/d32/d53/f92 [4744086,105576] 0 2024-09-16T00:26:20.784 INFO:tasks.workunit.client.1.smithi169.stdout:5/997: symlink d2/da/d10/d13/d1c/d40/d54/d7f/l14c 0 2024-09-16T00:26:20.786 INFO:tasks.workunit.client.1.smithi169.stdout:5/998: unlink d2/da/d10/d13/d1c/db1/l12d 0 2024-09-16T00:26:20.937 INFO:tasks.workunit.client.1.smithi169.stdout:5/999: sync 2024-09-16T00:26:20.941 INFO:tasks.workunit.client.1.smithi169.stderr:+ rm -rf -- ./tmp.RVqddVnjCD 2024-09-16T00:26:22.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:21 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:26:22.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:21 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:26:22.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:21 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:26:22.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:21 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:26:22.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:21 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:26:22.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:21 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:26:22.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:21 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mgr fail", "who": "smithi112.xrkkbc"}]: dispatch 2024-09-16T00:26:22.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:21 smithi112 ceph-mon[27869]: Activating manager daemon smithi169.mtrobk 2024-09-16T00:26:22.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:21 smithi112 ceph-mon[27869]: osdmap e45: 6 total, 6 up, 6 in 2024-09-16T00:26:22.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:21 smithi112 ceph-mon[27869]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "mgr fail", "who": "smithi112.xrkkbc"}]': finished 2024-09-16T00:26:22.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:21 smithi112 ceph-mon[27869]: mgrmap e21: smithi169.mtrobk(active, starting, since 0.0112959s) 2024-09-16T00:26:22.091 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:21 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "mon metadata", "id": "smithi112"}]: dispatch 2024-09-16T00:26:22.091 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:21 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "mon metadata", "id": "smithi169"}]: dispatch 2024-09-16T00:26:22.091 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:21 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi112.hxfwkh"}]: dispatch 2024-09-16T00:26:22.091 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:21 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi112.ithwzj"}]: dispatch 2024-09-16T00:26:22.091 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:21 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi169.gawwgw"}]: dispatch 2024-09-16T00:26:22.091 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:21 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi169.lgeqrj"}]: dispatch 2024-09-16T00:26:22.091 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:21 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "mgr metadata", "who": "smithi169.mtrobk", "id": "smithi169.mtrobk"}]: dispatch 2024-09-16T00:26:22.091 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:21 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:26:22.091 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:21 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:26:22.091 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:21 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:26:22.092 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:21 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:26:22.092 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:21 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:26:22.092 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:21 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:26:22.092 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:21 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T00:26:22.092 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:21 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T00:26:22.092 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:21 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T00:26:22.092 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:21 smithi112 ceph-mon[27869]: Manager daemon smithi169.mtrobk is now available 2024-09-16T00:26:22.092 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:21 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:26:22.093 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:21 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:26:22.093 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:21 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:26:22.093 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:21 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:26:22.093 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:21 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:26:22.093 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:21 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi169.mtrobk/mirror_snapshot_schedule"}]: dispatch 2024-09-16T00:26:22.093 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:21 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi169.mtrobk/trash_purge_schedule"}]: dispatch 2024-09-16T00:26:22.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:21 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:26:22.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:21 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:26:22.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:21 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:26:22.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:21 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:26:22.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:21 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:26:22.278 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:21 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:26:22.278 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:21 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mgr fail", "who": "smithi112.xrkkbc"}]: dispatch 2024-09-16T00:26:22.278 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:21 smithi169 ceph-mon[34653]: Activating manager daemon smithi169.mtrobk 2024-09-16T00:26:22.278 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:21 smithi169 ceph-mon[34653]: osdmap e45: 6 total, 6 up, 6 in 2024-09-16T00:26:22.278 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:21 smithi169 ceph-mon[34653]: from='mgr.14208 172.21.15.112:0/364431580' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "mgr fail", "who": "smithi112.xrkkbc"}]': finished 2024-09-16T00:26:22.278 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:21 smithi169 ceph-mon[34653]: mgrmap e21: smithi169.mtrobk(active, starting, since 0.0112959s) 2024-09-16T00:26:22.278 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:21 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "mon metadata", "id": "smithi112"}]: dispatch 2024-09-16T00:26:22.278 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:21 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "mon metadata", "id": "smithi169"}]: dispatch 2024-09-16T00:26:22.278 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:21 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi112.hxfwkh"}]: dispatch 2024-09-16T00:26:22.278 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:21 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi112.ithwzj"}]: dispatch 2024-09-16T00:26:22.278 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:21 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi169.gawwgw"}]: dispatch 2024-09-16T00:26:22.278 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:21 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi169.lgeqrj"}]: dispatch 2024-09-16T00:26:22.278 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:21 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "mgr metadata", "who": "smithi169.mtrobk", "id": "smithi169.mtrobk"}]: dispatch 2024-09-16T00:26:22.278 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:21 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:26:22.279 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:21 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:26:22.279 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:21 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:26:22.279 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:21 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:26:22.279 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:21 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:26:22.279 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:21 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:26:22.279 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:21 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T00:26:22.279 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:21 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T00:26:22.279 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:21 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T00:26:22.279 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:21 smithi169 ceph-mon[34653]: Manager daemon smithi169.mtrobk is now available 2024-09-16T00:26:22.279 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:21 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:26:22.279 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:21 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:26:22.279 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:21 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:26:22.279 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:21 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:26:22.279 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:21 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:26:22.279 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:21 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi169.mtrobk/mirror_snapshot_schedule"}]: dispatch 2024-09-16T00:26:22.280 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:21 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi169.mtrobk/trash_purge_schedule"}]: dispatch 2024-09-16T00:26:23.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.090+0000 7f94f0a1c640 1 -- 172.21.15.112:0/2075917967 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94e4094c40 msgr2=0x7f94e40950b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:23.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.090+0000 7f94f0a1c640 1 --2- 172.21.15.112:0/2075917967 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94e4094c40 0x7f94e40950b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f94d80048d0 tx=0x7f94d8009da0 comp rx=0 tx=0).stop 2024-09-16T00:26:23.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.090+0000 7f94f0a1c640 1 -- 172.21.15.112:0/2075917967 shutdown_connections 2024-09-16T00:26:23.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.090+0000 7f94f0a1c640 1 --2- 172.21.15.112:0/2075917967 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94e4094c40 0x7f94e40950b0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:23.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.090+0000 7f94f0a1c640 1 --2- 172.21.15.112:0/2075917967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94e4093970 0x7f94e4093d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:23.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.090+0000 7f94f0a1c640 1 -- 172.21.15.112:0/2075917967 >> 172.21.15.112:0/2075917967 conn(0x7f94e408f110 msgr2=0x7f94e4091560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:23.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.091+0000 7f94f0a1c640 1 -- 172.21.15.112:0/2075917967 shutdown_connections 2024-09-16T00:26:23.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.091+0000 7f94f0a1c640 1 -- 172.21.15.112:0/2075917967 wait complete. 2024-09-16T00:26:23.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.093+0000 7f94f0a1c640 1 Processor -- start 2024-09-16T00:26:23.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.093+0000 7f94f0a1c640 1 -- start start 2024-09-16T00:26:23.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.093+0000 7f94f0a1c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94e4093970 0x7f94e40a3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:23.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.093+0000 7f94f0a1c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94e4094c40 0x7f94e40a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:23.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.093+0000 7f94f0a1c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94e40a3e00 con 0x7f94e4093970 2024-09-16T00:26:23.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.093+0000 7f94f0a1c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94e40a67f0 con 0x7f94e4094c40 2024-09-16T00:26:23.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.094+0000 7f94eb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94e4093970 0x7f94e40a3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:23.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.094+0000 7f94eb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94e4093970 0x7f94e40a3270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54584/0 (socket says 172.21.15.112:54584) 2024-09-16T00:26:23.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.094+0000 7f94eb7fe640 1 -- 172.21.15.112:0/1609410436 learned_addr learned my addr 172.21.15.112:0/1609410436 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:26:23.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.095+0000 7f94eaffd640 1 --2- 172.21.15.112:0/1609410436 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94e4094c40 0x7f94e40a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:23.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.095+0000 7f94eb7fe640 1 -- 172.21.15.112:0/1609410436 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94e4094c40 msgr2=0x7f94e40a37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:23.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.095+0000 7f94eb7fe640 1 --2- 172.21.15.112:0/1609410436 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94e4094c40 0x7f94e40a37b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:23.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.095+0000 7f94eb7fe640 1 -- 172.21.15.112:0/1609410436 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94d8005040 con 0x7f94e4093970 2024-09-16T00:26:23.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.095+0000 7f94eb7fe640 1 --2- 172.21.15.112:0/1609410436 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94e4093970 0x7f94e40a3270 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f94e0002a50 tx=0x7f94e0004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:23.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.096+0000 7f94d7fff640 1 -- 172.21.15.112:0/1609410436 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94e0007c30 con 0x7f94e4093970 2024-09-16T00:26:23.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.097+0000 7f94f0a1c640 1 -- 172.21.15.112:0/1609410436 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94e40a6990 con 0x7f94e4093970 2024-09-16T00:26:23.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.097+0000 7f94f0a1c640 1 -- 172.21.15.112:0/1609410436 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94e40a6eb0 con 0x7f94e4093970 2024-09-16T00:26:23.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.098+0000 7f94d7fff640 1 -- 172.21.15.112:0/1609410436 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f94e000f040 con 0x7f94e4093970 2024-09-16T00:26:23.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.098+0000 7f94f0a1c640 1 -- 172.21.15.112:0/1609410436 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94e40043a0 con 0x7f94e4093970 2024-09-16T00:26:23.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.098+0000 7f94d7fff640 1 -- 172.21.15.112:0/1609410436 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94e000db50 con 0x7f94e4093970 2024-09-16T00:26:23.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.100+0000 7f94d7fff640 1 -- 172.21.15.112:0/1609410436 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 23) v1 ==== 46415+0+0 (secure 0 0 0) 0x7f94e000dcb0 con 0x7f94e4093970 2024-09-16T00:26:23.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.100+0000 7f94d7fff640 1 --2- 172.21.15.112:0/1609410436 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7f94dc039070 0x7f94dc03b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:23.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.100+0000 7f94d7fff640 1 -- 172.21.15.112:0/1609410436 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f94e004cd00 con 0x7f94e4093970 2024-09-16T00:26:23.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.102+0000 7f94eaffd640 1 --2- 172.21.15.112:0/1609410436 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7f94dc039070 0x7f94dc03b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:23.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.105+0000 7f94eaffd640 1 --2- 172.21.15.112:0/1609410436 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7f94dc039070 0x7f94dc03b520 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f94d800b020 tx=0x7f94d8009da0 comp rx=0 tx=0).ready entity=mgr.24341 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:23.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.109+0000 7f94d7fff640 1 -- 172.21.15.112:0/1609410436 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185942 (secure 0 0 0) 0x7f94e0052020 con 0x7f94e4093970 2024-09-16T00:26:23.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:23 smithi112 ceph-mon[27869]: Checking for cert/key for grafana.smithi112 2024-09-16T00:26:23.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:23 smithi112 ceph-mon[27869]: Migrating grafana.smithi112 cert to cert store 2024-09-16T00:26:23.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:23 smithi112 ceph-mon[27869]: Migrating grafana.smithi112 key to cert store 2024-09-16T00:26:23.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:23 smithi112 ceph-mon[27869]: Deploying cephadm binary to smithi112 2024-09-16T00:26:23.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:23 smithi112 ceph-mon[27869]: mgrmap e22: smithi169.mtrobk(active, since 1.00882s) 2024-09-16T00:26:23.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:23 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:26:23.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:23 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:26:23.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.398+0000 7f94f0a1c640 1 -- 172.21.15.112:0/1609410436 --> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f94e409d9a0 con 0x7f94dc039070 2024-09-16T00:26:23.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.409+0000 7f94d7fff640 1 -- 172.21.15.112:0/1609410436 <== mgr.24341 v2:172.21.15.169:6828/1220507355 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f94e409d9a0 con 0x7f94dc039070 2024-09-16T00:26:23.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.412+0000 7f94f0a1c640 1 -- 172.21.15.112:0/1609410436 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7f94dc039070 msgr2=0x7f94dc03b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:23.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.412+0000 7f94f0a1c640 1 --2- 172.21.15.112:0/1609410436 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7f94dc039070 0x7f94dc03b520 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f94d800b020 tx=0x7f94d8009da0 comp rx=0 tx=0).stop 2024-09-16T00:26:23.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.412+0000 7f94f0a1c640 1 -- 172.21.15.112:0/1609410436 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94e4093970 msgr2=0x7f94e40a3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:23.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.412+0000 7f94f0a1c640 1 --2- 172.21.15.112:0/1609410436 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94e4093970 0x7f94e40a3270 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f94e0002a50 tx=0x7f94e0004600 comp rx=0 tx=0).stop 2024-09-16T00:26:23.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.412+0000 7f94f0a1c640 1 -- 172.21.15.112:0/1609410436 shutdown_connections 2024-09-16T00:26:23.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.412+0000 7f94f0a1c640 1 --2- 172.21.15.112:0/1609410436 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7f94dc039070 0x7f94dc03b520 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:23.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.412+0000 7f94f0a1c640 1 --2- 172.21.15.112:0/1609410436 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94e4094c40 0x7f94e40a37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:23.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.412+0000 7f94f0a1c640 1 --2- 172.21.15.112:0/1609410436 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94e4093970 0x7f94e40a3270 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:23.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.412+0000 7f94f0a1c640 1 -- 172.21.15.112:0/1609410436 >> 172.21.15.112:0/1609410436 conn(0x7f94e408f110 msgr2=0x7f94e4090c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:23.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.412+0000 7f94f0a1c640 1 -- 172.21.15.112:0/1609410436 shutdown_connections 2024-09-16T00:26:23.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.412+0000 7f94f0a1c640 1 -- 172.21.15.112:0/1609410436 wait complete. 2024-09-16T00:26:23.423 INFO:teuthology.orchestra.run.smithi112.stdout:true 2024-09-16T00:26:23.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.520+0000 7f65a9312640 1 -- 172.21.15.112:0/1291968525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65a40ff690 msgr2=0x7f65a4101a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:23.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.520+0000 7f65a9312640 1 --2- 172.21.15.112:0/1291968525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65a40ff690 0x7f65a4101a70 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f65980048d0 tx=0x7f6598009da0 comp rx=0 tx=0).stop 2024-09-16T00:26:23.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.520+0000 7f65a9312640 1 -- 172.21.15.112:0/1291968525 shutdown_connections 2024-09-16T00:26:23.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.520+0000 7f65a9312640 1 --2- 172.21.15.112:0/1291968525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65a40ff690 0x7f65a4101a70 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:23.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.520+0000 7f65a9312640 1 --2- 172.21.15.112:0/1291968525 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f65a40666b0 0x7f65a40ff150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:23.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.520+0000 7f65a9312640 1 -- 172.21.15.112:0/1291968525 >> 172.21.15.112:0/1291968525 conn(0x7f65a40f6c20 msgr2=0x7f65a40f9030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:23.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.520+0000 7f65a9312640 1 -- 172.21.15.112:0/1291968525 shutdown_connections 2024-09-16T00:26:23.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.520+0000 7f65a9312640 1 -- 172.21.15.112:0/1291968525 wait complete. 2024-09-16T00:26:23.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.521+0000 7f65a9312640 1 Processor -- start 2024-09-16T00:26:23.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.521+0000 7f65a9312640 1 -- start start 2024-09-16T00:26:23.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.521+0000 7f65a9312640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65a40666b0 0x7f65a40fcb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:23.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.521+0000 7f65a9312640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f65a40ff690 0x7f65a40fb050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:23.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.521+0000 7f65a9312640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a40fb590 con 0x7f65a40666b0 2024-09-16T00:26:23.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.521+0000 7f65a2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65a40666b0 0x7f65a40fcb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:23.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.521+0000 7f65a9312640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a40fb6d0 con 0x7f65a40ff690 2024-09-16T00:26:23.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.521+0000 7f65a2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65a40666b0 0x7f65a40fcb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54616/0 (socket says 172.21.15.112:54616) 2024-09-16T00:26:23.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.521+0000 7f65a27fc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f65a40ff690 0x7f65a40fb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:23.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.521+0000 7f65a2ffd640 1 -- 172.21.15.112:0/91082566 learned_addr learned my addr 172.21.15.112:0/91082566 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:26:23.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.522+0000 7f65a27fc640 1 -- 172.21.15.112:0/91082566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65a40666b0 msgr2=0x7f65a40fcb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:23.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.522+0000 7f65a27fc640 1 --2- 172.21.15.112:0/91082566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65a40666b0 0x7f65a40fcb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:23.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.522+0000 7f65a27fc640 1 -- 172.21.15.112:0/91082566 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6598005040 con 0x7f65a40ff690 2024-09-16T00:26:23.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.522+0000 7f65a27fc640 1 --2- 172.21.15.112:0/91082566 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f65a40ff690 0x7f65a40fb050 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f65980048d0 tx=0x7f6598009da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:23.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.522+0000 7f658b7fe640 1 -- 172.21.15.112:0/91082566 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6598038070 con 0x7f65a40ff690 2024-09-16T00:26:23.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.522+0000 7f658b7fe640 1 -- 172.21.15.112:0/91082566 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f659802ae90 con 0x7f65a40ff690 2024-09-16T00:26:23.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.522+0000 7f658b7fe640 1 -- 172.21.15.112:0/91082566 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f659803e430 con 0x7f65a40ff690 2024-09-16T00:26:23.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.522+0000 7f65a9312640 1 -- 172.21.15.112:0/91082566 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65a40fb950 con 0x7f65a40ff690 2024-09-16T00:26:23.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.523+0000 7f65a9312640 1 -- 172.21.15.112:0/91082566 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65a4115190 con 0x7f65a40ff690 2024-09-16T00:26:23.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.523+0000 7f658b7fe640 1 -- 172.21.15.112:0/91082566 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 23) v1 ==== 46415+0+0 (secure 0 0 0) 0x7f6598030020 con 0x7f65a40ff690 2024-09-16T00:26:23.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.524+0000 7f658b7fe640 1 --2- 172.21.15.112:0/91082566 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7f658c039030 0x7f658c03b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:23.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.524+0000 7f658b7fe640 1 -- 172.21.15.112:0/91082566 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f659806db80 con 0x7f65a40ff690 2024-09-16T00:26:23.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.524+0000 7f65a9312640 1 -- 172.21.15.112:0/91082566 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65a40fa8a0 con 0x7f65a40ff690 2024-09-16T00:26:23.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.524+0000 7f65a2ffd640 1 --2- 172.21.15.112:0/91082566 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7f658c039030 0x7f658c03b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:23.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.525+0000 7f65a2ffd640 1 --2- 172.21.15.112:0/91082566 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7f658c039030 0x7f658c03b4e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f65a40fc8c0 tx=0x7f659400b040 comp rx=0 tx=0).ready entity=mgr.24341 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:23.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:23 smithi169 ceph-mon[34653]: Checking for cert/key for grafana.smithi112 2024-09-16T00:26:23.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:23 smithi169 ceph-mon[34653]: Migrating grafana.smithi112 cert to cert store 2024-09-16T00:26:23.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:23 smithi169 ceph-mon[34653]: Migrating grafana.smithi112 key to cert store 2024-09-16T00:26:23.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:23 smithi169 ceph-mon[34653]: Deploying cephadm binary to smithi112 2024-09-16T00:26:23.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:23 smithi169 ceph-mon[34653]: mgrmap e22: smithi169.mtrobk(active, since 1.00882s) 2024-09-16T00:26:23.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:23 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:26:23.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:23 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:26:23.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.533+0000 7f658b7fe640 1 -- 172.21.15.112:0/91082566 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185942 (secure 0 0 0) 0x7f659806de60 con 0x7f65a40ff690 2024-09-16T00:26:23.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.797+0000 7f65a9312640 1 -- 172.21.15.112:0/91082566 --> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f65a40f8710 con 0x7f658c039030 2024-09-16T00:26:23.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.810+0000 7f658b7fe640 1 -- 172.21.15.112:0/91082566 <== mgr.24341 v2:172.21.15.169:6828/1220507355 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f65a40f8710 con 0x7f658c039030 2024-09-16T00:26:23.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.812+0000 7f65a9312640 1 -- 172.21.15.112:0/91082566 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7f658c039030 msgr2=0x7f658c03b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:23.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.812+0000 7f65a9312640 1 --2- 172.21.15.112:0/91082566 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7f658c039030 0x7f658c03b4e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f65a40fc8c0 tx=0x7f659400b040 comp rx=0 tx=0).stop 2024-09-16T00:26:23.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.812+0000 7f65a9312640 1 -- 172.21.15.112:0/91082566 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f65a40ff690 msgr2=0x7f65a40fb050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:23.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.812+0000 7f65a9312640 1 --2- 172.21.15.112:0/91082566 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f65a40ff690 0x7f65a40fb050 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f65980048d0 tx=0x7f6598009da0 comp rx=0 tx=0).stop 2024-09-16T00:26:23.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.812+0000 7f65a9312640 1 -- 172.21.15.112:0/91082566 shutdown_connections 2024-09-16T00:26:23.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.812+0000 7f65a9312640 1 --2- 172.21.15.112:0/91082566 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7f658c039030 0x7f658c03b4e0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:23.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.812+0000 7f65a9312640 1 --2- 172.21.15.112:0/91082566 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f65a40ff690 0x7f65a40fb050 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:23.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.812+0000 7f65a9312640 1 --2- 172.21.15.112:0/91082566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65a40666b0 0x7f65a40fcb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:23.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.812+0000 7f65a9312640 1 -- 172.21.15.112:0/91082566 >> 172.21.15.112:0/91082566 conn(0x7f65a40f6c20 msgr2=0x7f65a40f6ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:23.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.812+0000 7f65a9312640 1 -- 172.21.15.112:0/91082566 shutdown_connections 2024-09-16T00:26:23.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.812+0000 7f65a9312640 1 -- 172.21.15.112:0/91082566 wait complete. 2024-09-16T00:26:23.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.924+0000 7f6b6eba1640 1 -- 172.21.15.112:0/3416065443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b6806e540 msgr2=0x7f6b68108730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:23.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.924+0000 7f6b6eba1640 1 --2- 172.21.15.112:0/3416065443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b6806e540 0x7f6b68108730 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6b540048d0 tx=0x7f6b54009da0 comp rx=0 tx=0).stop 2024-09-16T00:26:23.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.924+0000 7f6b6eba1640 1 -- 172.21.15.112:0/3416065443 shutdown_connections 2024-09-16T00:26:23.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.924+0000 7f6b6eba1640 1 --2- 172.21.15.112:0/3416065443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b6806e540 0x7f6b68108730 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:23.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.924+0000 7f6b6eba1640 1 --2- 172.21.15.112:0/3416065443 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b6806dbf0 0x7f6b6806e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:23.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.924+0000 7f6b6eba1640 1 -- 172.21.15.112:0/3416065443 >> 172.21.15.112:0/3416065443 conn(0x7f6b68069560 msgr2=0x7f6b6806b990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:23.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.924+0000 7f6b6eba1640 1 -- 172.21.15.112:0/3416065443 shutdown_connections 2024-09-16T00:26:23.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.924+0000 7f6b6eba1640 1 -- 172.21.15.112:0/3416065443 wait complete. 2024-09-16T00:26:23.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.925+0000 7f6b6eba1640 1 Processor -- start 2024-09-16T00:26:23.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.925+0000 7f6b6eba1640 1 -- start start 2024-09-16T00:26:23.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.926+0000 7f6b6eba1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b6806dbf0 0x7f6b68124a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:23.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.926+0000 7f6b6eba1640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b6806e540 0x7f6b68124fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:23.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.926+0000 7f6b6eba1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b681255b0 con 0x7f6b6806dbf0 2024-09-16T00:26:23.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.926+0000 7f6b6eba1640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b68125a60 con 0x7f6b6806e540 2024-09-16T00:26:23.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.926+0000 7f6b6c93d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b6806dbf0 0x7f6b68124a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:23.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.926+0000 7f6b6c93d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b6806dbf0 0x7f6b68124a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54634/0 (socket says 172.21.15.112:54634) 2024-09-16T00:26:23.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.926+0000 7f6b6c93d640 1 -- 172.21.15.112:0/924530257 learned_addr learned my addr 172.21.15.112:0/924530257 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:26:23.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.927+0000 7f6b6c93d640 1 -- 172.21.15.112:0/924530257 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b6806e540 msgr2=0x7f6b68124fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T00:26:23.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.927+0000 7f6b6c93d640 1 --2- 172.21.15.112:0/924530257 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b6806e540 0x7f6b68124fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:23.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.927+0000 7f6b6c93d640 1 -- 172.21.15.112:0/924530257 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b54005040 con 0x7f6b6806dbf0 2024-09-16T00:26:23.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.927+0000 7f6b6c93d640 1 --2- 172.21.15.112:0/924530257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b6806dbf0 0x7f6b68124a70 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c00eb70 tx=0x7f6b5c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:23.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.929+0000 7f6b657fa640 1 -- 172.21.15.112:0/924530257 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b5c007c30 con 0x7f6b6806dbf0 2024-09-16T00:26:23.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.929+0000 7f6b657fa640 1 -- 172.21.15.112:0/924530257 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f6b5c00f040 con 0x7f6b6806dbf0 2024-09-16T00:26:23.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.929+0000 7f6b6eba1640 1 -- 172.21.15.112:0/924530257 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b68125ce0 con 0x7f6b6806dbf0 2024-09-16T00:26:23.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.929+0000 7f6b6eba1640 1 -- 172.21.15.112:0/924530257 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b68126280 con 0x7f6b6806dbf0 2024-09-16T00:26:23.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.929+0000 7f6b657fa640 1 -- 172.21.15.112:0/924530257 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b5c009550 con 0x7f6b6806dbf0 2024-09-16T00:26:23.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.931+0000 7f6b657fa640 1 -- 172.21.15.112:0/924530257 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 23) v1 ==== 46415+0+0 (secure 0 0 0) 0x7f6b5c0096d0 con 0x7f6b6806dbf0 2024-09-16T00:26:23.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.931+0000 7f6b657fa640 1 --2- 172.21.15.112:0/924530257 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7f6b58039110 0x7f6b5803b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:23.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.931+0000 7f6b657fa640 1 -- 172.21.15.112:0/924530257 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f6b5c04ca70 con 0x7f6b6806dbf0 2024-09-16T00:26:23.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.932+0000 7f6b67fff640 1 --2- 172.21.15.112:0/924530257 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7f6b58039110 0x7f6b5803b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:23.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.933+0000 7f6b67fff640 1 --2- 172.21.15.112:0/924530257 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7f6b58039110 0x7f6b5803b5c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6b540043d0 tx=0x7f6b54009da0 comp rx=0 tx=0).ready entity=mgr.24341 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:23.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.933+0000 7f6b6eba1640 1 -- 172.21.15.112:0/924530257 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b4c005240 con 0x7f6b6806dbf0 2024-09-16T00:26:23.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:23.943+0000 7f6b657fa640 1 -- 172.21.15.112:0/924530257 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185942 (secure 0 0 0) 0x7f6b5c01d3b0 con 0x7f6b6806dbf0 2024-09-16T00:26:24.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.207+0000 7f6b6eba1640 1 -- 172.21.15.112:0/924530257 --> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6b4c000bf0 con 0x7f6b58039110 2024-09-16T00:26:24.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.225+0000 7f6b657fa640 1 -- 172.21.15.112:0/924530257 <== mgr.24341 v2:172.21.15.169:6828/1220507355 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f6b4c000bf0 con 0x7f6b58039110 2024-09-16T00:26:24.226 INFO:teuthology.orchestra.run.smithi112.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T00:26:24.226 INFO:teuthology.orchestra.run.smithi112.stdout:alertmanager.smithi112 smithi112 *:9093,9094 running (7m) 2m ago 8m 24.2M - 0.25.0 c8568f914cd2 89acc17eab5c 2024-09-16T00:26:24.226 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi112 smithi112 running (8m) 2m ago 8m 6891k - 17.2.7-1633-g6d025bc2 33ed8dba3996 c94220ed2776 2024-09-16T00:26:24.226 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi169 smithi169 running (7m) 22s ago 7m 6899k - 17.2.7-1633-g6d025bc2 33ed8dba3996 d0d3726280a2 2024-09-16T00:26:24.226 INFO:teuthology.orchestra.run.smithi112.stdout:grafana.smithi112 smithi112 *:3000 running (6m) 2m ago 8m 84.6M - 9.4.7 954c08fa6188 ffeb9c3bdf66 2024-09-16T00:26:24.226 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.hxfwkh smithi112 running (2m) 2m ago 2m 15.6M - 17.2.7-1633-g6d025bc2 33ed8dba3996 cf92d9461fc6 2024-09-16T00:26:24.226 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.ithwzj smithi112 running (2m) 2m ago 2m 11.8M - 17.2.7-1633-g6d025bc2 33ed8dba3996 02786c8a83c7 2024-09-16T00:26:24.226 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.gawwgw smithi169 running (2m) 22s ago 2m 139M - 17.2.7-1633-g6d025bc2 33ed8dba3996 dcf3a4dfa879 2024-09-16T00:26:24.226 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.lgeqrj smithi169 running (2m) 22s ago 2m 117M - 17.2.7-1633-g6d025bc2 33ed8dba3996 445c14e55831 2024-09-16T00:26:24.226 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi112.xrkkbc smithi112 *:9283,8765,8443 running (9m) 2m ago 9m 449M - 17.2.7-1633-g6d025bc2 33ed8dba3996 b3d3adb94a85 2024-09-16T00:26:24.226 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi169.mtrobk smithi169 *:8443,9283 running (45s) 22s ago 7m 113M - 19.1.1-326-g8921c8ef f1455bd0219c 519ed31e6dee 2024-09-16T00:26:24.226 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi112 smithi112 running (9m) 2m ago 9m 43.3M 2048M 17.2.7-1633-g6d025bc2 33ed8dba3996 ddaaced54bf8 2024-09-16T00:26:24.227 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi169 smithi169 running (7m) 22s ago 7m 37.3M 2048M 17.2.7-1633-g6d025bc2 33ed8dba3996 6c2289bc3e11 2024-09-16T00:26:24.227 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi112 smithi112 *:9100 running (8m) 2m ago 8m 15.9M - 1.5.0 0da6a335fe13 89c6f32288bc 2024-09-16T00:26:24.227 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi169 smithi169 *:9100 running (7m) 22s ago 7m 15.5M - 1.5.0 0da6a335fe13 5d6fe1dc6fa9 2024-09-16T00:26:24.227 INFO:teuthology.orchestra.run.smithi112.stdout:osd.0 smithi112 running (6m) 2m ago 6m 46.0M 4288M 17.2.7-1633-g6d025bc2 33ed8dba3996 f6b2e146ffdf 2024-09-16T00:26:24.227 INFO:teuthology.orchestra.run.smithi112.stdout:osd.1 smithi112 running (6m) 2m ago 6m 47.9M 4288M 17.2.7-1633-g6d025bc2 33ed8dba3996 e9bee87bba0f 2024-09-16T00:26:24.227 INFO:teuthology.orchestra.run.smithi112.stdout:osd.2 smithi112 running (5m) 2m ago 5m 43.3M 4288M 17.2.7-1633-g6d025bc2 33ed8dba3996 100ec4e4852a 2024-09-16T00:26:24.227 INFO:teuthology.orchestra.run.smithi112.stdout:osd.3 smithi169 running (5m) 22s ago 5m 101M 5312M 17.2.7-1633-g6d025bc2 33ed8dba3996 f88463e19f23 2024-09-16T00:26:24.227 INFO:teuthology.orchestra.run.smithi112.stdout:osd.4 smithi169 running (4m) 22s ago 4m 92.0M 5312M 17.2.7-1633-g6d025bc2 33ed8dba3996 d3e4cdb016b9 2024-09-16T00:26:24.227 INFO:teuthology.orchestra.run.smithi112.stdout:osd.5 smithi169 running (4m) 22s ago 4m 87.8M 5312M 17.2.7-1633-g6d025bc2 33ed8dba3996 8e7d708d74eb 2024-09-16T00:26:24.227 INFO:teuthology.orchestra.run.smithi112.stdout:prometheus.smithi112 smithi112 *:9095 running (6m) 2m ago 7m 55.1M - 2.43.0 a07b618ecd1d 7f5a86d91011 2024-09-16T00:26:24.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.228+0000 7f6b6eba1640 1 -- 172.21.15.112:0/924530257 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7f6b58039110 msgr2=0x7f6b5803b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:24.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.228+0000 7f6b6eba1640 1 --2- 172.21.15.112:0/924530257 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7f6b58039110 0x7f6b5803b5c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6b540043d0 tx=0x7f6b54009da0 comp rx=0 tx=0).stop 2024-09-16T00:26:24.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.228+0000 7f6b6eba1640 1 -- 172.21.15.112:0/924530257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b6806dbf0 msgr2=0x7f6b68124a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:24.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.228+0000 7f6b6eba1640 1 --2- 172.21.15.112:0/924530257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b6806dbf0 0x7f6b68124a70 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c00eb70 tx=0x7f6b5c00b040 comp rx=0 tx=0).stop 2024-09-16T00:26:24.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.228+0000 7f6b6eba1640 1 -- 172.21.15.112:0/924530257 shutdown_connections 2024-09-16T00:26:24.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.228+0000 7f6b6eba1640 1 --2- 172.21.15.112:0/924530257 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7f6b58039110 0x7f6b5803b5c0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:24.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.228+0000 7f6b6eba1640 1 --2- 172.21.15.112:0/924530257 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b6806e540 0x7f6b68124fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:24.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.228+0000 7f6b6eba1640 1 --2- 172.21.15.112:0/924530257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b6806dbf0 0x7f6b68124a70 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:24.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.228+0000 7f6b6eba1640 1 -- 172.21.15.112:0/924530257 >> 172.21.15.112:0/924530257 conn(0x7f6b68069560 msgr2=0x7f6b68106890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:24.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.228+0000 7f6b6eba1640 1 -- 172.21.15.112:0/924530257 shutdown_connections 2024-09-16T00:26:24.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.228+0000 7f6b6eba1640 1 -- 172.21.15.112:0/924530257 wait complete. 2024-09-16T00:26:24.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.329+0000 7fa8fa2ce640 1 -- 172.21.15.112:0/3151359861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8f40fb420 msgr2=0x7fa8f40fb830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:24.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.329+0000 7fa8fa2ce640 1 --2- 172.21.15.112:0/3151359861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8f40fb420 0x7fa8f40fb830 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa8e4004950 tx=0x7fa8e4009da0 comp rx=0 tx=0).stop 2024-09-16T00:26:24.330 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:24 smithi112 ceph-mon[27869]: pgmap v3: 65 pgs: 65 active+clean; 3.1 GiB data, 11 GiB used, 525 GiB / 536 GiB avail 2024-09-16T00:26:24.330 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:24 smithi112 ceph-mon[27869]: Deploying cephadm binary to smithi169 2024-09-16T00:26:24.330 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:24 smithi112 ceph-mon[27869]: mgrmap e23: smithi169.mtrobk(active, since 2s) 2024-09-16T00:26:24.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.329+0000 7fa8fa2ce640 1 -- 172.21.15.112:0/3151359861 shutdown_connections 2024-09-16T00:26:24.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.329+0000 7fa8fa2ce640 1 --2- 172.21.15.112:0/3151359861 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa8f40fc6f0 0x7fa8f40fcb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:24.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.329+0000 7fa8fa2ce640 1 --2- 172.21.15.112:0/3151359861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8f40fb420 0x7fa8f40fb830 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:24.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.329+0000 7fa8fa2ce640 1 -- 172.21.15.112:0/3151359861 >> 172.21.15.112:0/3151359861 conn(0x7fa8f40f6c00 msgr2=0x7fa8f40f9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:24.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.329+0000 7fa8fa2ce640 1 -- 172.21.15.112:0/3151359861 shutdown_connections 2024-09-16T00:26:24.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.329+0000 7fa8fa2ce640 1 -- 172.21.15.112:0/3151359861 wait complete. 2024-09-16T00:26:24.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.330+0000 7fa8fa2ce640 1 Processor -- start 2024-09-16T00:26:24.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.331+0000 7fa8fa2ce640 1 -- start start 2024-09-16T00:26:24.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.331+0000 7fa8fa2ce640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa8f40fb420 0x7fa8f4113650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:24.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.331+0000 7fa8fa2ce640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8f40fc6f0 0x7fa8f4113b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:24.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.331+0000 7fa8fa2ce640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f41141e0 con 0x7fa8f40fc6f0 2024-09-16T00:26:24.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.331+0000 7fa8f3fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa8f40fb420 0x7fa8f4113650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:24.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.331+0000 7fa8f3fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa8f40fb420 0x7fa8f4113650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.112:50940/0 (socket says 172.21.15.112:50940) 2024-09-16T00:26:24.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.331+0000 7fa8f3fff640 1 -- 172.21.15.112:0/3462150558 learned_addr learned my addr 172.21.15.112:0/3462150558 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:26:24.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.331+0000 7fa8f37fe640 1 --2- 172.21.15.112:0/3462150558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8f40fc6f0 0x7fa8f4113b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:24.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.331+0000 7fa8fa2ce640 1 -- 172.21.15.112:0/3462150558 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f4114a00 con 0x7fa8f40fb420 2024-09-16T00:26:24.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.332+0000 7fa8f37fe640 1 -- 172.21.15.112:0/3462150558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa8f40fb420 msgr2=0x7fa8f4113650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:24.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.332+0000 7fa8f37fe640 1 --2- 172.21.15.112:0/3462150558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa8f40fb420 0x7fa8f4113650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:24.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.332+0000 7fa8f37fe640 1 -- 172.21.15.112:0/3462150558 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8e4005040 con 0x7fa8f40fc6f0 2024-09-16T00:26:24.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.333+0000 7fa8f37fe640 1 --2- 172.21.15.112:0/3462150558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8f40fc6f0 0x7fa8f4113b90 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa8e80027a0 tx=0x7fa8e8009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:24.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.333+0000 7fa8f0ff9640 1 -- 172.21.15.112:0/3462150558 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa8e8002d90 con 0x7fa8f40fc6f0 2024-09-16T00:26:24.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.333+0000 7fa8fa2ce640 1 -- 172.21.15.112:0/3462150558 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8f4114c90 con 0x7fa8f40fc6f0 2024-09-16T00:26:24.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.333+0000 7fa8f0ff9640 1 -- 172.21.15.112:0/3462150558 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fa8e800f040 con 0x7fa8f40fc6f0 2024-09-16T00:26:24.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.333+0000 7fa8f0ff9640 1 -- 172.21.15.112:0/3462150558 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa8e8009470 con 0x7fa8f40fc6f0 2024-09-16T00:26:24.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.333+0000 7fa8fa2ce640 1 -- 172.21.15.112:0/3462150558 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8f4115230 con 0x7fa8f40fc6f0 2024-09-16T00:26:24.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.334+0000 7fa8f0ff9640 1 -- 172.21.15.112:0/3462150558 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 23) v1 ==== 46415+0+0 (secure 0 0 0) 0x7fa8e8009730 con 0x7fa8f40fc6f0 2024-09-16T00:26:24.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.336+0000 7fa8f0ff9640 1 --2- 172.21.15.112:0/3462150558 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7fa8dc039040 0x7fa8dc03b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:24.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.336+0000 7fa8f0ff9640 1 -- 172.21.15.112:0/3462150558 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fa8e804cf90 con 0x7fa8f40fc6f0 2024-09-16T00:26:24.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.336+0000 7fa8fa2ce640 1 -- 172.21.15.112:0/3462150558 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8f40fa880 con 0x7fa8f40fc6f0 2024-09-16T00:26:24.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.336+0000 7fa8f3fff640 1 --2- 172.21.15.112:0/3462150558 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7fa8dc039040 0x7fa8dc03b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:24.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.337+0000 7fa8f3fff640 1 --2- 172.21.15.112:0/3462150558 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7fa8dc039040 0x7fa8dc03b4f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa8e400d230 tx=0x7fa8e400b040 comp rx=0 tx=0).ready entity=mgr.24341 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:24.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.342+0000 7fa8f0ff9640 1 -- 172.21.15.112:0/3462150558 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185942 (secure 0 0 0) 0x7fa8e801ce40 con 0x7fa8f40fc6f0 2024-09-16T00:26:24.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:24 smithi169 ceph-mon[34653]: pgmap v3: 65 pgs: 65 active+clean; 3.1 GiB data, 11 GiB used, 525 GiB / 536 GiB avail 2024-09-16T00:26:24.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:24 smithi169 ceph-mon[34653]: Deploying cephadm binary to smithi169 2024-09-16T00:26:24.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:24 smithi169 ceph-mon[34653]: mgrmap e23: smithi169.mtrobk(active, since 2s) 2024-09-16T00:26:24.689 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-16T00:26:24.690 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-16T00:26:24.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.690+0000 7fa8fa2ce640 1 -- 172.21.15.112:0/3462150558 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa8f41154e0 con 0x7fa8f40fc6f0 2024-09-16T00:26:24.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.690+0000 7fa8f0ff9640 1 -- 172.21.15.112:0/3462150558 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+747 (secure 0 0 0) 0x7fa8e8016120 con 0x7fa8f40fc6f0 2024-09-16T00:26:24.691 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:26:24.691 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": { 2024-09-16T00:26:24.691 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 17.2.7-1633-g6d025bc2 (6d025bc26f3176c910707f8cb4c8d8e076640615) quincy (stable)": 2 2024-09-16T00:26:24.692 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:26:24.692 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": { 2024-09-16T00:26:24.692 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 1 2024-09-16T00:26:24.692 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:26:24.692 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": { 2024-09-16T00:26:24.692 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 17.2.7-1633-g6d025bc2 (6d025bc26f3176c910707f8cb4c8d8e076640615) quincy (stable)": 6 2024-09-16T00:26:24.692 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:26:24.692 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": { 2024-09-16T00:26:24.692 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 17.2.7-1633-g6d025bc2 (6d025bc26f3176c910707f8cb4c8d8e076640615) quincy (stable)": 4 2024-09-16T00:26:24.692 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:26:24.692 INFO:teuthology.orchestra.run.smithi112.stdout: "overall": { 2024-09-16T00:26:24.692 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 17.2.7-1633-g6d025bc2 (6d025bc26f3176c910707f8cb4c8d8e076640615) quincy (stable)": 12, 2024-09-16T00:26:24.692 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 1 2024-09-16T00:26:24.692 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-16T00:26:24.693 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:26:24.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.693+0000 7fa8fa2ce640 1 -- 172.21.15.112:0/3462150558 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7fa8dc039040 msgr2=0x7fa8dc03b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:24.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.693+0000 7fa8fa2ce640 1 --2- 172.21.15.112:0/3462150558 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7fa8dc039040 0x7fa8dc03b4f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa8e400d230 tx=0x7fa8e400b040 comp rx=0 tx=0).stop 2024-09-16T00:26:24.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.693+0000 7fa8fa2ce640 1 -- 172.21.15.112:0/3462150558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8f40fc6f0 msgr2=0x7fa8f4113b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:24.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.693+0000 7fa8fa2ce640 1 --2- 172.21.15.112:0/3462150558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8f40fc6f0 0x7fa8f4113b90 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa8e80027a0 tx=0x7fa8e8009f30 comp rx=0 tx=0).stop 2024-09-16T00:26:24.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.693+0000 7fa8fa2ce640 1 -- 172.21.15.112:0/3462150558 shutdown_connections 2024-09-16T00:26:24.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.693+0000 7fa8fa2ce640 1 --2- 172.21.15.112:0/3462150558 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7fa8dc039040 0x7fa8dc03b4f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:24.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.693+0000 7fa8fa2ce640 1 --2- 172.21.15.112:0/3462150558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8f40fc6f0 0x7fa8f4113b90 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:24.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.693+0000 7fa8fa2ce640 1 --2- 172.21.15.112:0/3462150558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa8f40fb420 0x7fa8f4113650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:24.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.693+0000 7fa8fa2ce640 1 -- 172.21.15.112:0/3462150558 >> 172.21.15.112:0/3462150558 conn(0x7fa8f40f6c00 msgr2=0x7fa8f40f8410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:24.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.693+0000 7fa8fa2ce640 1 -- 172.21.15.112:0/3462150558 shutdown_connections 2024-09-16T00:26:24.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.693+0000 7fa8fa2ce640 1 -- 172.21.15.112:0/3462150558 wait complete. 2024-09-16T00:26:24.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.816+0000 7f5cde6a8640 1 -- 172.21.15.112:0/3304554570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5cd0008570 msgr2=0x7f5cd0006aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:24.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.816+0000 7f5cde6a8640 1 --2- 172.21.15.112:0/3304554570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5cd0008570 0x7f5cd0006aa0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5cd8063010 tx=0x7f5cd8062e70 comp rx=0 tx=0).stop 2024-09-16T00:26:24.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.817+0000 7f5cde6a8640 1 -- 172.21.15.112:0/3304554570 shutdown_connections 2024-09-16T00:26:24.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.817+0000 7f5cde6a8640 1 --2- 172.21.15.112:0/3304554570 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5cd0006fe0 0x7f5cd0007450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:24.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.817+0000 7f5cde6a8640 1 --2- 172.21.15.112:0/3304554570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5cd0008570 0x7f5cd0006aa0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:24.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.817+0000 7f5cde6a8640 1 -- 172.21.15.112:0/3304554570 >> 172.21.15.112:0/3304554570 conn(0x7f5cd0090240 msgr2=0x7f5cd0092670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:24.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.818+0000 7f5cde6a8640 1 -- 172.21.15.112:0/3304554570 shutdown_connections 2024-09-16T00:26:24.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.818+0000 7f5cde6a8640 1 -- 172.21.15.112:0/3304554570 wait complete. 2024-09-16T00:26:24.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.819+0000 7f5cde6a8640 1 Processor -- start 2024-09-16T00:26:24.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.819+0000 7f5cde6a8640 1 -- start start 2024-09-16T00:26:24.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.819+0000 7f5cde6a8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5cd0006fe0 0x7f5cd00a75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:24.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.819+0000 7f5cde6a8640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5cd0008570 0x7f5cd00a7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:24.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.819+0000 7f5cde6a8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd00a8150 con 0x7f5cd0006fe0 2024-09-16T00:26:24.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.820+0000 7f5cde6a8640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd00acb50 con 0x7f5cd0008570 2024-09-16T00:26:24.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.820+0000 7f5cdd6a6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5cd0006fe0 0x7f5cd00a75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:24.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.820+0000 7f5cdd6a6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5cd0006fe0 0x7f5cd00a75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54670/0 (socket says 172.21.15.112:54670) 2024-09-16T00:26:24.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.820+0000 7f5cdd6a6640 1 -- 172.21.15.112:0/519346701 learned_addr learned my addr 172.21.15.112:0/519346701 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:26:24.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.820+0000 7f5cdcea5640 1 --2- 172.21.15.112:0/519346701 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5cd0008570 0x7f5cd00a7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:24.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.820+0000 7f5cdd6a6640 1 -- 172.21.15.112:0/519346701 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5cd0008570 msgr2=0x7f5cd00a7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:24.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.820+0000 7f5cdd6a6640 1 --2- 172.21.15.112:0/519346701 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5cd0008570 0x7f5cd00a7b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:24.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.820+0000 7f5cdd6a6640 1 -- 172.21.15.112:0/519346701 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cd804b760 con 0x7f5cd0006fe0 2024-09-16T00:26:24.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.821+0000 7f5cdd6a6640 1 --2- 172.21.15.112:0/519346701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5cd0006fe0 0x7f5cd00a75c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f5cd8092700 tx=0x7f5cd8062e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:24.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.821+0000 7f5ccdffb640 1 -- 172.21.15.112:0/519346701 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5cd8099a40 con 0x7f5cd0006fe0 2024-09-16T00:26:24.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.821+0000 7f5ccdffb640 1 -- 172.21.15.112:0/519346701 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f5cd80a3430 con 0x7f5cd0006fe0 2024-09-16T00:26:24.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.821+0000 7f5cde6a8640 1 -- 172.21.15.112:0/519346701 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cd00acc90 con 0x7f5cd0006fe0 2024-09-16T00:26:24.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.822+0000 7f5cde6a8640 1 -- 172.21.15.112:0/519346701 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cd00ad0c0 con 0x7f5cd0006fe0 2024-09-16T00:26:24.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.822+0000 7f5ccdffb640 1 -- 172.21.15.112:0/519346701 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5cd80a2390 con 0x7f5cd0006fe0 2024-09-16T00:26:24.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.823+0000 7f5ccdffb640 1 -- 172.21.15.112:0/519346701 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 23) v1 ==== 46415+0+0 (secure 0 0 0) 0x7f5cd80a1040 con 0x7f5cd0006fe0 2024-09-16T00:26:24.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.823+0000 7f5ccdffb640 1 --2- 172.21.15.112:0/519346701 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7f5cc40390c0 0x7f5cc403b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:24.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.823+0000 7f5ccdffb640 1 -- 172.21.15.112:0/519346701 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f5cd8073030 con 0x7f5cd0006fe0 2024-09-16T00:26:24.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.824+0000 7f5cdcea5640 1 --2- 172.21.15.112:0/519346701 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7f5cc40390c0 0x7f5cc403b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:24.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.824+0000 7f5cde6a8640 1 -- 172.21.15.112:0/519346701 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cd00043a0 con 0x7f5cd0006fe0 2024-09-16T00:26:24.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.825+0000 7f5cdcea5640 1 --2- 172.21.15.112:0/519346701 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7f5cc40390c0 0x7f5cc403b570 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f5cd00a8ac0 tx=0x7f5cd400b040 comp rx=0 tx=0).ready entity=mgr.24341 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:24.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:24.832+0000 7f5ccdffb640 1 -- 172.21.15.112:0/519346701 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185942 (secure 0 0 0) 0x7f5cd809f020 con 0x7f5cd0006fe0 2024-09-16T00:26:25.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:25.065+0000 7f5ccdffb640 1 -- 172.21.15.112:0/519346701 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mgrmap(e 24) v1 ==== 46471+0+0 (secure 0 0 0) 0x7f5cd80a3810 con 0x7f5cd0006fe0 2024-09-16T00:26:25.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:25.156+0000 7f5cde6a8640 1 -- 172.21.15.112:0/519346701 --> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5cd009ba70 con 0x7f5cc40390c0 2024-09-16T00:26:25.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:25.157+0000 7f5ccdffb640 1 -- 172.21.15.112:0/519346701 <== mgr.24341 v2:172.21.15.169:6828/1220507355 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f5cd009ba70 con 0x7f5cc40390c0 2024-09-16T00:26:25.158 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:26:25.158 INFO:teuthology.orchestra.run.smithi112.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80", 2024-09-16T00:26:25.158 INFO:teuthology.orchestra.run.smithi112.stdout: "in_progress": true, 2024-09-16T00:26:25.159 INFO:teuthology.orchestra.run.smithi112.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-09-16T00:26:25.159 INFO:teuthology.orchestra.run.smithi112.stdout: "services_complete": [], 2024-09-16T00:26:25.159 INFO:teuthology.orchestra.run.smithi112.stdout: "progress": "1/2 daemons upgraded", 2024-09-16T00:26:25.159 INFO:teuthology.orchestra.run.smithi112.stdout: "message": "", 2024-09-16T00:26:25.159 INFO:teuthology.orchestra.run.smithi112.stdout: "is_paused": false 2024-09-16T00:26:25.159 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:26:25.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:25.159+0000 7f5cde6a8640 1 -- 172.21.15.112:0/519346701 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7f5cc40390c0 msgr2=0x7f5cc403b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:25.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:25.159+0000 7f5cde6a8640 1 --2- 172.21.15.112:0/519346701 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7f5cc40390c0 0x7f5cc403b570 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f5cd00a8ac0 tx=0x7f5cd400b040 comp rx=0 tx=0).stop 2024-09-16T00:26:25.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:25.160+0000 7f5cde6a8640 1 -- 172.21.15.112:0/519346701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5cd0006fe0 msgr2=0x7f5cd00a75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:25.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:25.160+0000 7f5cde6a8640 1 --2- 172.21.15.112:0/519346701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5cd0006fe0 0x7f5cd00a75c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f5cd8092700 tx=0x7f5cd8062e70 comp rx=0 tx=0).stop 2024-09-16T00:26:25.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:25.160+0000 7f5cde6a8640 1 -- 172.21.15.112:0/519346701 shutdown_connections 2024-09-16T00:26:25.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:25.160+0000 7f5cde6a8640 1 --2- 172.21.15.112:0/519346701 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7f5cc40390c0 0x7f5cc403b570 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:25.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:25.160+0000 7f5cde6a8640 1 --2- 172.21.15.112:0/519346701 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5cd0008570 0x7f5cd00a7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:25.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:25.160+0000 7f5cde6a8640 1 --2- 172.21.15.112:0/519346701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5cd0006fe0 0x7f5cd00a75c0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:25.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:25.160+0000 7f5cde6a8640 1 -- 172.21.15.112:0/519346701 >> 172.21.15.112:0/519346701 conn(0x7f5cd0090240 msgr2=0x7f5cd0091890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:25.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:25.160+0000 7f5cde6a8640 1 -- 172.21.15.112:0/519346701 shutdown_connections 2024-09-16T00:26:25.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:25.160+0000 7f5cde6a8640 1 -- 172.21.15.112:0/519346701 wait complete. 2024-09-16T00:26:25.287 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:25 smithi112 ceph-mon[27869]: pgmap v4: 65 pgs: 65 active+clean; 3.1 GiB data, 11 GiB used, 525 GiB / 536 GiB avail 2024-09-16T00:26:25.287 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:25 smithi112 ceph-mon[27869]: from='client.14626 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:26:25.287 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:25 smithi112 ceph-mon[27869]: from='client.24389 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:26:25.287 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:25 smithi112 ceph-mon[27869]: [16/Sep/2024:00:26:23] ENGINE Bus STARTING 2024-09-16T00:26:25.287 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:25 smithi112 ceph-mon[27869]: [16/Sep/2024:00:26:24] ENGINE Serving on http://172.21.15.169:8765 2024-09-16T00:26:25.287 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:25 smithi112 ceph-mon[27869]: from='client.14638 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:26:25.287 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:25 smithi112 ceph-mon[27869]: [16/Sep/2024:00:26:24] ENGINE Serving on https://172.21.15.169:7150 2024-09-16T00:26:25.287 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:25 smithi112 ceph-mon[27869]: [16/Sep/2024:00:26:24] ENGINE Bus STARTED 2024-09-16T00:26:25.287 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:25 smithi112 ceph-mon[27869]: [16/Sep/2024:00:26:24] ENGINE Error in HTTPServer.serve 2024-09-16T00:26:25.287 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: Traceback (most recent call last): 2024-09-16T00:26:25.288 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T00:26:25.288 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: self._connections.run(self.expiration_interval) 2024-09-16T00:26:25.288 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T00:26:25.288 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: self._run(expiration_interval) 2024-09-16T00:26:25.288 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T00:26:25.288 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T00:26:25.288 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T00:26:25.288 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T00:26:25.288 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T00:26:25.288 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: s = self.context.wrap_socket( 2024-09-16T00:26:25.288 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T00:26:25.288 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: return self.sslsocket_class._create( 2024-09-16T00:26:25.288 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T00:26:25.289 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: self.do_handshake() 2024-09-16T00:26:25.289 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T00:26:25.289 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: self._sslobj.do_handshake() 2024-09-16T00:26:25.289 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T00:26:25.289 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:25 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/3462150558' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:26:25.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:25 smithi169 ceph-mon[34653]: pgmap v4: 65 pgs: 65 active+clean; 3.1 GiB data, 11 GiB used, 525 GiB / 536 GiB avail 2024-09-16T00:26:25.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:25 smithi169 ceph-mon[34653]: from='client.14626 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:26:25.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:25 smithi169 ceph-mon[34653]: from='client.24389 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:26:25.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:25 smithi169 ceph-mon[34653]: [16/Sep/2024:00:26:23] ENGINE Bus STARTING 2024-09-16T00:26:25.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:25 smithi169 ceph-mon[34653]: [16/Sep/2024:00:26:24] ENGINE Serving on http://172.21.15.169:8765 2024-09-16T00:26:25.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:25 smithi169 ceph-mon[34653]: from='client.14638 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:26:25.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:25 smithi169 ceph-mon[34653]: [16/Sep/2024:00:26:24] ENGINE Serving on https://172.21.15.169:7150 2024-09-16T00:26:25.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:25 smithi169 ceph-mon[34653]: [16/Sep/2024:00:26:24] ENGINE Bus STARTED 2024-09-16T00:26:25.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:25 smithi169 ceph-mon[34653]: [16/Sep/2024:00:26:24] ENGINE Error in HTTPServer.serve 2024-09-16T00:26:25.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Traceback (most recent call last): 2024-09-16T00:26:25.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T00:26:25.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: self._connections.run(self.expiration_interval) 2024-09-16T00:26:25.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T00:26:25.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: self._run(expiration_interval) 2024-09-16T00:26:25.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T00:26:25.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T00:26:25.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T00:26:25.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T00:26:25.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T00:26:25.528 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: s = self.context.wrap_socket( 2024-09-16T00:26:25.528 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T00:26:25.528 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: return self.sslsocket_class._create( 2024-09-16T00:26:25.528 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T00:26:25.528 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: self.do_handshake() 2024-09-16T00:26:25.528 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T00:26:25.528 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: self._sslobj.do_handshake() 2024-09-16T00:26:25.528 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T00:26:25.528 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:25 smithi169 ceph-mon[34653]: from='client.? 172.21.15.112:0/3462150558' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:26:26.166 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:26 smithi112 ceph-mon[27869]: pgmap v5: 65 pgs: 65 active+clean; 3.1 GiB data, 11 GiB used, 525 GiB / 536 GiB avail 2024-09-16T00:26:26.166 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:26 smithi112 ceph-mon[27869]: mgrmap e24: smithi169.mtrobk(active, since 4s) 2024-09-16T00:26:26.166 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:26 smithi112 ceph-mon[27869]: from='client.14646 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:26:26.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:26 smithi169 ceph-mon[34653]: pgmap v5: 65 pgs: 65 active+clean; 3.1 GiB data, 11 GiB used, 525 GiB / 536 GiB avail 2024-09-16T00:26:26.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:26 smithi169 ceph-mon[34653]: mgrmap e24: smithi169.mtrobk(active, since 4s) 2024-09-16T00:26:26.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:26 smithi169 ceph-mon[34653]: from='client.14646 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:26:28.700 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:28 smithi169 ceph-mon[34653]: pgmap v6: 65 pgs: 65 active+clean; 2.5 GiB data, 9.6 GiB used, 527 GiB / 536 GiB avail; 571 KiB/s rd, 434 KiB/s wr, 81 op/s 2024-09-16T00:26:28.700 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:28 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:26:28.700 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:28 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:26:28.700 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:28 smithi169 ceph-mon[34653]: Standby manager daemon smithi112.xrkkbc started 2024-09-16T00:26:28.700 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:28 smithi169 ceph-mon[34653]: from='mgr.? 172.21.15.112:0/3643405823' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi112.xrkkbc/crt"}]: dispatch 2024-09-16T00:26:28.700 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:28 smithi169 ceph-mon[34653]: from='mgr.? 172.21.15.112:0/3643405823' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T00:26:28.700 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:28 smithi169 ceph-mon[34653]: from='mgr.? 172.21.15.112:0/3643405823' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi112.xrkkbc/key"}]: dispatch 2024-09-16T00:26:28.700 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:28 smithi169 ceph-mon[34653]: from='mgr.? 172.21.15.112:0/3643405823' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T00:26:28.700 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:28 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:26:28.700 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:28 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:26:28.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:28 smithi112 ceph-mon[27869]: pgmap v6: 65 pgs: 65 active+clean; 2.5 GiB data, 9.6 GiB used, 527 GiB / 536 GiB avail; 571 KiB/s rd, 434 KiB/s wr, 81 op/s 2024-09-16T00:26:28.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:28 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:26:28.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:28 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:26:28.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:28 smithi112 ceph-mon[27869]: Standby manager daemon smithi112.xrkkbc started 2024-09-16T00:26:28.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:28 smithi112 ceph-mon[27869]: from='mgr.? 172.21.15.112:0/3643405823' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi112.xrkkbc/crt"}]: dispatch 2024-09-16T00:26:28.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:28 smithi112 ceph-mon[27869]: from='mgr.? 172.21.15.112:0/3643405823' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T00:26:28.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:28 smithi112 ceph-mon[27869]: from='mgr.? 172.21.15.112:0/3643405823' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi112.xrkkbc/key"}]: dispatch 2024-09-16T00:26:28.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:28 smithi112 ceph-mon[27869]: from='mgr.? 172.21.15.112:0/3643405823' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T00:26:28.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:28 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:26:28.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:28 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:26:29.744 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:29 smithi169 ceph-mon[34653]: mgrmap e25: smithi169.mtrobk(active, since 7s), standbys: smithi112.xrkkbc 2024-09-16T00:26:29.744 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:29 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "mgr metadata", "who": "smithi112.xrkkbc", "id": "smithi112.xrkkbc"}]: dispatch 2024-09-16T00:26:29.744 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:29 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:26:29.744 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:29 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:26:29.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:29 smithi112 ceph-mon[27869]: mgrmap e25: smithi169.mtrobk(active, since 7s), standbys: smithi112.xrkkbc 2024-09-16T00:26:29.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:29 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "mgr metadata", "who": "smithi112.xrkkbc", "id": "smithi112.xrkkbc"}]: dispatch 2024-09-16T00:26:29.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:29 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:26:29.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:29 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:26:31.253 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:30 smithi169 ceph-mon[34653]: pgmap v7: 65 pgs: 65 active+clean; 1.4 GiB data, 6.3 GiB used, 530 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 239 op/s 2024-09-16T00:26:31.253 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:30 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:26:31.254 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:30 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:26:31.254 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:30 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:26:31.254 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:30 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:26:31.254 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:30 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:26:31.254 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:30 smithi169 ceph-mon[34653]: Adjusting osd_memory_target on smithi112 to 1557M 2024-09-16T00:26:31.254 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:30 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:26:31.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:30 smithi112 ceph-mon[27869]: pgmap v7: 65 pgs: 65 active+clean; 1.4 GiB data, 6.3 GiB used, 530 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 239 op/s 2024-09-16T00:26:31.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:30 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:26:31.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:30 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:26:31.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:30 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:26:31.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:30 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:26:31.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:30 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:26:31.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:30 smithi112 ceph-mon[27869]: Adjusting osd_memory_target on smithi112 to 1557M 2024-09-16T00:26:31.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:30 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:26:31.933 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-16T00:26:31.934 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-09-16T00:26:33.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:32 smithi169 ceph-mon[34653]: pgmap v8: 65 pgs: 65 active+clean; 1.4 GiB data, 6.3 GiB used, 530 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 185 op/s 2024-09-16T00:26:33.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:32 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:26:33.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:32 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:26:33.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:32 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:26:33.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:32 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:26:33.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:32 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:26:33.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:32 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:26:33.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:32 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:26:33.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:32 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:26:33.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:32 smithi112 ceph-mon[27869]: pgmap v8: 65 pgs: 65 active+clean; 1.4 GiB data, 6.3 GiB used, 530 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 185 op/s 2024-09-16T00:26:33.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:32 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:26:33.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:32 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:26:33.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:32 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:26:33.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:32 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:26:33.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:32 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:26:33.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:32 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:26:33.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:32 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:26:33.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:32 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:26:34.077 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:33 smithi112 ceph-mon[27869]: Adjusting osd_memory_target on smithi169 to 2581M 2024-09-16T00:26:34.077 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:33 smithi112 ceph-mon[27869]: Updating smithi112:/etc/ceph/ceph.conf 2024-09-16T00:26:34.077 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:33 smithi112 ceph-mon[27869]: Updating smithi169:/etc/ceph/ceph.conf 2024-09-16T00:26:34.077 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:33 smithi112 ceph-mon[27869]: Updating smithi112:/var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:26:34.077 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:33 smithi112 ceph-mon[27869]: Updating smithi169:/var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:26:34.077 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:33 smithi112 ceph-mon[27869]: Updating smithi112:/etc/ceph/ceph.client.admin.keyring 2024-09-16T00:26:34.078 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:33 smithi112 ceph-mon[27869]: Updating smithi169:/etc/ceph/ceph.client.admin.keyring 2024-09-16T00:26:34.078 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:33 smithi112 ceph-mon[27869]: Updating smithi112:/var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T00:26:34.078 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:33 smithi112 ceph-mon[27869]: Updating smithi169:/var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T00:26:34.078 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:33 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:26:34.078 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:33 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:26:34.078 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:33 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:26:34.078 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:33 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:26:34.078 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:33 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:26:34.078 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:33 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:26:34.078 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:33 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:26:34.078 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:33 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:26:34.078 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:33 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:26:34.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:33 smithi169 ceph-mon[34653]: Adjusting osd_memory_target on smithi169 to 2581M 2024-09-16T00:26:34.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:33 smithi169 ceph-mon[34653]: Updating smithi112:/etc/ceph/ceph.conf 2024-09-16T00:26:34.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:33 smithi169 ceph-mon[34653]: Updating smithi169:/etc/ceph/ceph.conf 2024-09-16T00:26:34.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:33 smithi169 ceph-mon[34653]: Updating smithi112:/var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:26:34.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:33 smithi169 ceph-mon[34653]: Updating smithi169:/var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:26:34.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:33 smithi169 ceph-mon[34653]: Updating smithi112:/etc/ceph/ceph.client.admin.keyring 2024-09-16T00:26:34.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:33 smithi169 ceph-mon[34653]: Updating smithi169:/etc/ceph/ceph.client.admin.keyring 2024-09-16T00:26:34.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:33 smithi169 ceph-mon[34653]: Updating smithi112:/var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T00:26:34.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:33 smithi169 ceph-mon[34653]: Updating smithi169:/var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T00:26:34.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:33 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:26:34.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:33 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:26:34.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:33 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:26:34.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:33 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:26:34.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:33 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:26:34.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:33 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:26:34.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:33 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:26:34.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:33 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:26:34.278 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:33 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:26:35.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:34 smithi169 ceph-mon[34653]: pgmap v9: 65 pgs: 65 active+clean; 508 MiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 269 op/s 2024-09-16T00:26:35.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:34 smithi169 ceph-mon[34653]: Reconfiguring node-exporter.smithi112 (dependencies changed)... 2024-09-16T00:26:35.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:34 smithi169 ceph-mon[34653]: Deploying daemon node-exporter.smithi112 on smithi112 2024-09-16T00:26:35.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:34 smithi112 ceph-mon[27869]: pgmap v9: 65 pgs: 65 active+clean; 508 MiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 269 op/s 2024-09-16T00:26:35.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:34 smithi112 ceph-mon[27869]: Reconfiguring node-exporter.smithi112 (dependencies changed)... 2024-09-16T00:26:35.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:34 smithi112 ceph-mon[27869]: Deploying daemon node-exporter.smithi112 on smithi112 2024-09-16T00:26:37.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:36 smithi169 ceph-mon[34653]: pgmap v10: 65 pgs: 65 active+clean; 508 MiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 246 op/s 2024-09-16T00:26:37.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:36 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:26:37.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:36 smithi112 ceph-mon[27869]: pgmap v10: 65 pgs: 65 active+clean; 508 MiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 246 op/s 2024-09-16T00:26:37.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:36 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:26:39.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:38 smithi169 ceph-mon[34653]: pgmap v11: 65 pgs: 65 active+clean; 423 MiB data, 3.2 GiB used, 533 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 277 op/s 2024-09-16T00:26:39.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:38 smithi112 ceph-mon[27869]: pgmap v11: 65 pgs: 65 active+clean; 423 MiB data, 3.2 GiB used, 533 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 277 op/s 2024-09-16T00:26:40.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:39 smithi169 ceph-mon[34653]: fsmap cephfs:2 {0=cephfs.smithi112.hxfwkh=up:active,1=cephfs.smithi169.gawwgw=up:active} 2 up:standby-replay 2024-09-16T00:26:40.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:39 smithi112 ceph-mon[27869]: fsmap cephfs:2 {0=cephfs.smithi112.hxfwkh=up:active,1=cephfs.smithi169.gawwgw=up:active} 2 up:standby-replay 2024-09-16T00:26:41.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:40 smithi169 ceph-mon[34653]: pgmap v12: 65 pgs: 65 active+clean; 302 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 307 op/s 2024-09-16T00:26:41.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:40 smithi112 ceph-mon[27869]: pgmap v12: 65 pgs: 65 active+clean; 302 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 307 op/s 2024-09-16T00:26:43.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:42 smithi112 ceph-mon[27869]: pgmap v13: 65 pgs: 65 active+clean; 302 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 202 op/s 2024-09-16T00:26:43.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:42 smithi169 ceph-mon[34653]: pgmap v13: 65 pgs: 65 active+clean; 302 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 202 op/s 2024-09-16T00:26:45.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:44 smithi169 ceph-mon[34653]: pgmap v14: 65 pgs: 65 active+clean; 316 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 4.0 MiB/s rd, 4.1 MiB/s wr, 262 op/s 2024-09-16T00:26:45.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:44 smithi112 ceph-mon[27869]: pgmap v14: 65 pgs: 65 active+clean; 316 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 4.0 MiB/s rd, 4.1 MiB/s wr, 262 op/s 2024-09-16T00:26:47.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:46 smithi169 ceph-mon[34653]: pgmap v15: 65 pgs: 65 active+clean; 316 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 155 op/s 2024-09-16T00:26:47.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:46 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:26:47.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:46 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:26:47.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:46 smithi112 ceph-mon[27869]: pgmap v15: 65 pgs: 65 active+clean; 316 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 155 op/s 2024-09-16T00:26:47.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:46 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:26:47.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:46 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:26:48.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:47 smithi169 ceph-mon[34653]: Reconfiguring alertmanager.smithi112 (dependencies changed)... 2024-09-16T00:26:48.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:47 smithi169 ceph-mon[34653]: Deploying daemon alertmanager.smithi112 on smithi112 2024-09-16T00:26:48.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:47 smithi112 ceph-mon[27869]: Reconfiguring alertmanager.smithi112 (dependencies changed)... 2024-09-16T00:26:48.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:47 smithi112 ceph-mon[27869]: Deploying daemon alertmanager.smithi112 on smithi112 2024-09-16T00:26:49.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:48 smithi169 ceph-mon[34653]: pgmap v16: 65 pgs: 65 active+clean; 312 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.9 MiB/s rd, 3.2 MiB/s wr, 191 op/s 2024-09-16T00:26:49.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:48 smithi112 ceph-mon[27869]: pgmap v16: 65 pgs: 65 active+clean; 312 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.9 MiB/s rd, 3.2 MiB/s wr, 191 op/s 2024-09-16T00:26:51.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:50 smithi169 ceph-mon[34653]: pgmap v17: 65 pgs: 65 active+clean; 321 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 3.9 MiB/s rd, 4.1 MiB/s wr, 197 op/s 2024-09-16T00:26:51.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:50 smithi112 ceph-mon[27869]: pgmap v17: 65 pgs: 65 active+clean; 321 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 3.9 MiB/s rd, 4.1 MiB/s wr, 197 op/s 2024-09-16T00:26:52.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:51 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:26:52.288 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:51 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:26:53.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:52 smithi169 ceph-mon[34653]: pgmap v18: 65 pgs: 65 active+clean; 321 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 133 op/s 2024-09-16T00:26:53.300 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:52 smithi112 ceph-mon[27869]: pgmap v18: 65 pgs: 65 active+clean; 321 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 133 op/s 2024-09-16T00:26:55.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.272+0000 7f2203ce5640 1 -- 172.21.15.112:0/3642910312 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21fc0fc550 msgr2=0x7f21fc0fc960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:55.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.272+0000 7f2203ce5640 1 --2- 172.21.15.112:0/3642910312 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21fc0fc550 0x7f21fc0fc960 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f21f4009c50 tx=0x7f21f4004d10 comp rx=0 tx=0).stop 2024-09-16T00:26:55.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.272+0000 7f2203ce5640 1 -- 172.21.15.112:0/3642910312 shutdown_connections 2024-09-16T00:26:55.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.272+0000 7f2203ce5640 1 --2- 172.21.15.112:0/3642910312 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21fc0fd820 0x7f21fc0fdc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:55.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.272+0000 7f2203ce5640 1 --2- 172.21.15.112:0/3642910312 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21fc0fc550 0x7f21fc0fc960 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:55.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.272+0000 7f2203ce5640 1 -- 172.21.15.112:0/3642910312 >> 172.21.15.112:0/3642910312 conn(0x7f21fc0f7d30 msgr2=0x7f21fc0fa140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:55.274 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:54 smithi112 ceph-mon[27869]: pgmap v19: 65 pgs: 65 active+clean; 321 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 4.1 MiB/s rd, 4.1 MiB/s wr, 207 op/s 2024-09-16T00:26:55.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.273+0000 7f2203ce5640 1 -- 172.21.15.112:0/3642910312 shutdown_connections 2024-09-16T00:26:55.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.273+0000 7f2203ce5640 1 -- 172.21.15.112:0/3642910312 wait complete. 2024-09-16T00:26:55.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.273+0000 7f2203ce5640 1 Processor -- start 2024-09-16T00:26:55.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.273+0000 7f2203ce5640 1 -- start start 2024-09-16T00:26:55.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.274+0000 7f2203ce5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21fc0fc550 0x7f21fc10d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:55.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.274+0000 7f2203ce5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21fc0fd820 0x7f21fc10d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:55.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.274+0000 7f2203ce5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21fc10dc00 con 0x7f21fc0fc550 2024-09-16T00:26:55.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.274+0000 7f2203ce5640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21fc10e420 con 0x7f21fc0fd820 2024-09-16T00:26:55.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.274+0000 7f2201a81640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21fc0fc550 0x7f21fc10d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:55.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.274+0000 7f2201a81640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21fc0fc550 0x7f21fc10d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35820/0 (socket says 172.21.15.112:35820) 2024-09-16T00:26:55.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.274+0000 7f2201a81640 1 -- 172.21.15.112:0/1160215948 learned_addr learned my addr 172.21.15.112:0/1160215948 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:26:55.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.274+0000 7f2201280640 1 --2- 172.21.15.112:0/1160215948 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21fc0fd820 0x7f21fc10d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:55.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.275+0000 7f2201a81640 1 -- 172.21.15.112:0/1160215948 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21fc0fd820 msgr2=0x7f21fc10d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:55.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.275+0000 7f2201a81640 1 --2- 172.21.15.112:0/1160215948 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21fc0fd820 0x7f21fc10d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:55.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.275+0000 7f2201a81640 1 -- 172.21.15.112:0/1160215948 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21f40098b0 con 0x7f21fc0fc550 2024-09-16T00:26:55.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.275+0000 7f2201a81640 1 --2- 172.21.15.112:0/1160215948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21fc0fc550 0x7f21fc10d070 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f21f4004a30 tx=0x7f21f4004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:55.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:54 smithi169 ceph-mon[34653]: pgmap v19: 65 pgs: 65 active+clean; 321 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 4.1 MiB/s rd, 4.1 MiB/s wr, 207 op/s 2024-09-16T00:26:55.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.275+0000 7f21f27fc640 1 -- 172.21.15.112:0/1160215948 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f21f4032850 con 0x7f21fc0fc550 2024-09-16T00:26:55.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.276+0000 7f21f27fc640 1 -- 172.21.15.112:0/1160215948 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f21f4032e50 con 0x7f21fc0fc550 2024-09-16T00:26:55.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.276+0000 7f21f27fc640 1 -- 172.21.15.112:0/1160215948 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f21f403de40 con 0x7f21fc0fc550 2024-09-16T00:26:55.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.276+0000 7f2203ce5640 1 -- 172.21.15.112:0/1160215948 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21fc10e650 con 0x7f21fc0fc550 2024-09-16T00:26:55.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.276+0000 7f2203ce5640 1 -- 172.21.15.112:0/1160215948 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21fc10ebf0 con 0x7f21fc0fc550 2024-09-16T00:26:55.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.278+0000 7f2203ce5640 1 -- 172.21.15.112:0/1160215948 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21fc0fb9b0 con 0x7f21fc0fc550 2024-09-16T00:26:55.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.279+0000 7f21f27fc640 1 -- 172.21.15.112:0/1160215948 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) v1 ==== 91662+0+0 (secure 0 0 0) 0x7f21f402f030 con 0x7f21fc0fc550 2024-09-16T00:26:55.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.280+0000 7f21f27fc640 1 --2- 172.21.15.112:0/1160215948 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7f21e806e8d0 0x7f21e8070d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:55.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.280+0000 7f21f27fc640 1 -- 172.21.15.112:0/1160215948 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f21f40af610 con 0x7f21fc0fc550 2024-09-16T00:26:55.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.280+0000 7f2201280640 1 --2- 172.21.15.112:0/1160215948 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7f21e806e8d0 0x7f21e8070d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:55.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.281+0000 7f2201280640 1 --2- 172.21.15.112:0/1160215948 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7f21e806e8d0 0x7f21e8070d80 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f21fc10ce20 tx=0x7f21ec00b040 comp rx=0 tx=0).ready entity=mgr.24341 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:55.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.287+0000 7f21f27fc640 1 -- 172.21.15.112:0/1160215948 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185942 (secure 0 0 0) 0x7f21f4078f10 con 0x7f21fc0fc550 2024-09-16T00:26:55.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.563+0000 7f2203ce5640 1 -- 172.21.15.112:0/1160215948 --> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f21fc10f030 con 0x7f21e806e8d0 2024-09-16T00:26:55.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.565+0000 7f21f27fc640 1 -- 172.21.15.112:0/1160215948 <== mgr.24341 v2:172.21.15.169:6828/1220507355 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f21fc10f030 con 0x7f21e806e8d0 2024-09-16T00:26:55.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.568+0000 7f2203ce5640 1 -- 172.21.15.112:0/1160215948 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7f21e806e8d0 msgr2=0x7f21e8070d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:55.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.568+0000 7f2203ce5640 1 --2- 172.21.15.112:0/1160215948 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7f21e806e8d0 0x7f21e8070d80 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f21fc10ce20 tx=0x7f21ec00b040 comp rx=0 tx=0).stop 2024-09-16T00:26:55.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.568+0000 7f2203ce5640 1 -- 172.21.15.112:0/1160215948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21fc0fc550 msgr2=0x7f21fc10d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:55.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.568+0000 7f2203ce5640 1 --2- 172.21.15.112:0/1160215948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21fc0fc550 0x7f21fc10d070 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f21f4004a30 tx=0x7f21f4004d10 comp rx=0 tx=0).stop 2024-09-16T00:26:55.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.568+0000 7f2203ce5640 1 -- 172.21.15.112:0/1160215948 shutdown_connections 2024-09-16T00:26:55.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.568+0000 7f2203ce5640 1 --2- 172.21.15.112:0/1160215948 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7f21e806e8d0 0x7f21e8070d80 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:55.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.568+0000 7f2203ce5640 1 --2- 172.21.15.112:0/1160215948 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21fc0fd820 0x7f21fc10d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:55.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.568+0000 7f2203ce5640 1 --2- 172.21.15.112:0/1160215948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21fc0fc550 0x7f21fc10d070 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:55.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.568+0000 7f2203ce5640 1 -- 172.21.15.112:0/1160215948 >> 172.21.15.112:0/1160215948 conn(0x7f21fc0f7d30 msgr2=0x7f21fc0f9500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:55.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.569+0000 7f2203ce5640 1 -- 172.21.15.112:0/1160215948 shutdown_connections 2024-09-16T00:26:55.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.569+0000 7f2203ce5640 1 -- 172.21.15.112:0/1160215948 wait complete. 2024-09-16T00:26:55.583 INFO:teuthology.orchestra.run.smithi112.stdout:true 2024-09-16T00:26:55.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.675+0000 7f354f56f640 1 -- 172.21.15.112:0/3949616602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3548065c40 msgr2=0x7f35480ff140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:55.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.675+0000 7f354f56f640 1 --2- 172.21.15.112:0/3949616602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3548065c40 0x7f35480ff140 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f35440048d0 tx=0x7f3544009da0 comp rx=0 tx=0).stop 2024-09-16T00:26:55.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.675+0000 7f354f56f640 1 -- 172.21.15.112:0/3949616602 shutdown_connections 2024-09-16T00:26:55.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.675+0000 7f354f56f640 1 --2- 172.21.15.112:0/3949616602 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35480ff680 0x7f3548101a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:55.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.675+0000 7f354f56f640 1 --2- 172.21.15.112:0/3949616602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3548065c40 0x7f35480ff140 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:55.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.675+0000 7f354f56f640 1 -- 172.21.15.112:0/3949616602 >> 172.21.15.112:0/3949616602 conn(0x7f35480f6c10 msgr2=0x7f35480f9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:55.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.675+0000 7f354f56f640 1 -- 172.21.15.112:0/3949616602 shutdown_connections 2024-09-16T00:26:55.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.675+0000 7f354f56f640 1 -- 172.21.15.112:0/3949616602 wait complete. 2024-09-16T00:26:55.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.676+0000 7f354f56f640 1 Processor -- start 2024-09-16T00:26:55.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.676+0000 7f354f56f640 1 -- start start 2024-09-16T00:26:55.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.676+0000 7f354f56f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3548065c40 0x7f354810f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:55.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.676+0000 7f354f56f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35480ff680 0x7f354810f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:55.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.676+0000 7f354f56f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f354810fe50 con 0x7f3548065c40 2024-09-16T00:26:55.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.676+0000 7f354f56f640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3548110670 con 0x7f35480ff680 2024-09-16T00:26:55.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.677+0000 7f354d30b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3548065c40 0x7f354810f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:55.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.677+0000 7f354d30b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3548065c40 0x7f354810f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35836/0 (socket says 172.21.15.112:35836) 2024-09-16T00:26:55.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.677+0000 7f354d30b640 1 -- 172.21.15.112:0/3939014051 learned_addr learned my addr 172.21.15.112:0/3939014051 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:26:55.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.677+0000 7f354cb0a640 1 --2- 172.21.15.112:0/3939014051 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35480ff680 0x7f354810f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:55.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.677+0000 7f354d30b640 1 -- 172.21.15.112:0/3939014051 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35480ff680 msgr2=0x7f354810f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:55.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.677+0000 7f354d30b640 1 --2- 172.21.15.112:0/3939014051 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35480ff680 0x7f354810f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:55.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.677+0000 7f354d30b640 1 -- 172.21.15.112:0/3939014051 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3544005040 con 0x7f3548065c40 2024-09-16T00:26:55.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.677+0000 7f354d30b640 1 --2- 172.21.15.112:0/3939014051 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3548065c40 0x7f354810f2c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f354402d040 tx=0x7f3544009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:55.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.678+0000 7f353dffb640 1 -- 172.21.15.112:0/3939014051 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3544038070 con 0x7f3548065c40 2024-09-16T00:26:55.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.678+0000 7f353dffb640 1 -- 172.21.15.112:0/3939014051 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f354402ae90 con 0x7f3548065c40 2024-09-16T00:26:55.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.678+0000 7f353dffb640 1 -- 172.21.15.112:0/3939014051 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f354403e670 con 0x7f3548065c40 2024-09-16T00:26:55.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.678+0000 7f354f56f640 1 -- 172.21.15.112:0/3939014051 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35481108a0 con 0x7f3548065c40 2024-09-16T00:26:55.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.678+0000 7f354f56f640 1 -- 172.21.15.112:0/3939014051 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3548110df0 con 0x7f3548065c40 2024-09-16T00:26:55.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.679+0000 7f354f56f640 1 -- 172.21.15.112:0/3939014051 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35480fa890 con 0x7f3548065c40 2024-09-16T00:26:55.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.680+0000 7f353dffb640 1 -- 172.21.15.112:0/3939014051 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) v1 ==== 91662+0+0 (secure 0 0 0) 0x7f3544030020 con 0x7f3548065c40 2024-09-16T00:26:55.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.680+0000 7f353dffb640 1 --2- 172.21.15.112:0/3939014051 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7f353406e6e0 0x7f3534070b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:55.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.681+0000 7f353dffb640 1 -- 172.21.15.112:0/3939014051 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f35440aded0 con 0x7f3548065c40 2024-09-16T00:26:55.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.681+0000 7f354cb0a640 1 --2- 172.21.15.112:0/3939014051 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7f353406e6e0 0x7f3534070b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:55.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.682+0000 7f354cb0a640 1 --2- 172.21.15.112:0/3939014051 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7f353406e6e0 0x7f3534070b90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f354810f070 tx=0x7f353800b040 comp rx=0 tx=0).ready entity=mgr.24341 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:55.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.686+0000 7f353dffb640 1 -- 172.21.15.112:0/3939014051 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185942 (secure 0 0 0) 0x7f3544077860 con 0x7f3548065c40 2024-09-16T00:26:55.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.896+0000 7f354f56f640 1 -- 172.21.15.112:0/3939014051 --> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3548111170 con 0x7f353406e6e0 2024-09-16T00:26:55.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.897+0000 7f353dffb640 1 -- 172.21.15.112:0/3939014051 <== mgr.24341 v2:172.21.15.169:6828/1220507355 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f3548111170 con 0x7f353406e6e0 2024-09-16T00:26:55.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.900+0000 7f354f56f640 1 -- 172.21.15.112:0/3939014051 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7f353406e6e0 msgr2=0x7f3534070b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:55.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.900+0000 7f354f56f640 1 --2- 172.21.15.112:0/3939014051 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7f353406e6e0 0x7f3534070b90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f354810f070 tx=0x7f353800b040 comp rx=0 tx=0).stop 2024-09-16T00:26:55.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.900+0000 7f354f56f640 1 -- 172.21.15.112:0/3939014051 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3548065c40 msgr2=0x7f354810f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:55.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.900+0000 7f354f56f640 1 --2- 172.21.15.112:0/3939014051 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3548065c40 0x7f354810f2c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f354402d040 tx=0x7f3544009da0 comp rx=0 tx=0).stop 2024-09-16T00:26:55.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.900+0000 7f354f56f640 1 -- 172.21.15.112:0/3939014051 shutdown_connections 2024-09-16T00:26:55.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.900+0000 7f354f56f640 1 --2- 172.21.15.112:0/3939014051 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7f353406e6e0 0x7f3534070b90 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:55.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.900+0000 7f354f56f640 1 --2- 172.21.15.112:0/3939014051 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35480ff680 0x7f354810f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:55.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.900+0000 7f354f56f640 1 --2- 172.21.15.112:0/3939014051 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3548065c40 0x7f354810f2c0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:55.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.900+0000 7f354f56f640 1 -- 172.21.15.112:0/3939014051 >> 172.21.15.112:0/3939014051 conn(0x7f35480f6c10 msgr2=0x7f35480f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:55.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.900+0000 7f354f56f640 1 -- 172.21.15.112:0/3939014051 shutdown_connections 2024-09-16T00:26:55.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.900+0000 7f354f56f640 1 -- 172.21.15.112:0/3939014051 wait complete. 2024-09-16T00:26:55.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.998+0000 7f1193920640 1 -- 172.21.15.112:0/1789754987 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f118c0fa930 msgr2=0x7f118c0fad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:56.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.998+0000 7f1193920640 1 --2- 172.21.15.112:0/1789754987 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f118c0fa930 0x7f118c0fad40 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f11880048d0 tx=0x7f1188009da0 comp rx=0 tx=0).stop 2024-09-16T00:26:56.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.998+0000 7f1193920640 1 -- 172.21.15.112:0/1789754987 shutdown_connections 2024-09-16T00:26:56.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.998+0000 7f1193920640 1 --2- 172.21.15.112:0/1789754987 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f118c0fb280 0x7f118c0fdb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:56.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.998+0000 7f1193920640 1 --2- 172.21.15.112:0/1789754987 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f118c0fa930 0x7f118c0fad40 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:56.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.999+0000 7f1193920640 1 -- 172.21.15.112:0/1789754987 >> 172.21.15.112:0/1789754987 conn(0x7f118c0f6390 msgr2=0x7f118c0f87a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:56.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.999+0000 7f1193920640 1 -- 172.21.15.112:0/1789754987 shutdown_connections 2024-09-16T00:26:56.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.999+0000 7f1193920640 1 -- 172.21.15.112:0/1789754987 wait complete. 2024-09-16T00:26:56.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.999+0000 7f1193920640 1 Processor -- start 2024-09-16T00:26:56.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:55.999+0000 7f1193920640 1 -- start start 2024-09-16T00:26:56.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.000+0000 7f1193920640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f118c0fa930 0x7f118c10d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:56.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.000+0000 7f1193920640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f118c0fb280 0x7f118c10d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:56.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.000+0000 7f1193920640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f118c10de10 con 0x7f118c0fb280 2024-09-16T00:26:56.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.000+0000 7f1193920640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f118c10e630 con 0x7f118c0fa930 2024-09-16T00:26:56.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.000+0000 7f1190ebb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f118c0fb280 0x7f118c10d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:56.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.000+0000 7f11916bc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f118c0fa930 0x7f118c10d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:56.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.000+0000 7f1190ebb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f118c0fb280 0x7f118c10d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35850/0 (socket says 172.21.15.112:35850) 2024-09-16T00:26:56.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.000+0000 7f1190ebb640 1 -- 172.21.15.112:0/165644606 learned_addr learned my addr 172.21.15.112:0/165644606 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:26:56.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.000+0000 7f11916bc640 1 -- 172.21.15.112:0/165644606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f118c0fb280 msgr2=0x7f118c10d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:56.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.000+0000 7f11916bc640 1 --2- 172.21.15.112:0/165644606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f118c0fb280 0x7f118c10d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:56.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.000+0000 7f11916bc640 1 -- 172.21.15.112:0/165644606 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1188005040 con 0x7f118c0fa930 2024-09-16T00:26:56.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.000+0000 7f1190ebb640 1 --2- 172.21.15.112:0/165644606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f118c0fb280 0x7f118c10d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:26:56.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.000+0000 7f11916bc640 1 --2- 172.21.15.112:0/165644606 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f118c0fa930 0x7f118c10d280 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f118802a530 tx=0x7f1188009da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:56.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.001+0000 7f1181ffb640 1 -- 172.21.15.112:0/165644606 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1188038070 con 0x7f118c0fa930 2024-09-16T00:26:56.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.001+0000 7f1193920640 1 -- 172.21.15.112:0/165644606 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f118c10e860 con 0x7f118c0fa930 2024-09-16T00:26:56.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.001+0000 7f1181ffb640 1 -- 172.21.15.112:0/165644606 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f118802ae90 con 0x7f118c0fa930 2024-09-16T00:26:56.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.001+0000 7f1181ffb640 1 -- 172.21.15.112:0/165644606 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f118803e670 con 0x7f118c0fa930 2024-09-16T00:26:56.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.001+0000 7f1193920640 1 -- 172.21.15.112:0/165644606 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f118c10ee00 con 0x7f118c0fa930 2024-09-16T00:26:56.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.002+0000 7f1193920640 1 -- 172.21.15.112:0/165644606 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f118c0fa010 con 0x7f118c0fa930 2024-09-16T00:26:56.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.003+0000 7f1181ffb640 1 -- 172.21.15.112:0/165644606 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 25) v1 ==== 91662+0+0 (secure 0 0 0) 0x7f1188030020 con 0x7f118c0fa930 2024-09-16T00:26:56.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.003+0000 7f1181ffb640 1 --2- 172.21.15.112:0/165644606 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7f117806e6e0 0x7f1178070b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:56.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.003+0000 7f1181ffb640 1 -- 172.21.15.112:0/165644606 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f11880ade70 con 0x7f118c0fa930 2024-09-16T00:26:56.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.003+0000 7f1190ebb640 1 --2- 172.21.15.112:0/165644606 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7f117806e6e0 0x7f1178070b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:56.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.004+0000 7f1190ebb640 1 --2- 172.21.15.112:0/165644606 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7f117806e6e0 0x7f1178070b90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f118c10d030 tx=0x7f117c00b040 comp rx=0 tx=0).ready entity=mgr.24341 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:56.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.008+0000 7f1181ffb640 1 -- 172.21.15.112:0/165644606 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185942 (secure 0 0 0) 0x7f11880778c0 con 0x7f118c0fa930 2024-09-16T00:26:56.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.245+0000 7f1193920640 1 -- 172.21.15.112:0/165644606 --> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f118c10f070 con 0x7f117806e6e0 2024-09-16T00:26:56.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.266+0000 7f1181ffb640 1 -- 172.21.15.112:0/165644606 <== mgr.24341 v2:172.21.15.169:6828/1220507355 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f118c10f070 con 0x7f117806e6e0 2024-09-16T00:26:56.267 INFO:teuthology.orchestra.run.smithi112.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T00:26:56.267 INFO:teuthology.orchestra.run.smithi112.stdout:alertmanager.smithi112 smithi112 *:9093,9094 running (7m) 27s ago 8m 24.0M - 0.25.0 c8568f914cd2 89acc17eab5c 2024-09-16T00:26:56.267 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi112 smithi112 running (9m) 27s ago 9m 6891k - 17.2.7-1633-g6d025bc2 33ed8dba3996 c94220ed2776 2024-09-16T00:26:56.267 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi169 smithi169 running (8m) 28s ago 8m 6899k - 17.2.7-1633-g6d025bc2 33ed8dba3996 d0d3726280a2 2024-09-16T00:26:56.267 INFO:teuthology.orchestra.run.smithi112.stdout:grafana.smithi112 smithi112 *:3000 running (7m) 27s ago 8m 84.7M - 9.4.7 954c08fa6188 ffeb9c3bdf66 2024-09-16T00:26:56.267 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.hxfwkh smithi112 running (3m) 27s ago 3m 165M - 17.2.7-1633-g6d025bc2 33ed8dba3996 cf92d9461fc6 2024-09-16T00:26:56.267 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.ithwzj smithi112 running (3m) 27s ago 3m 138M - 17.2.7-1633-g6d025bc2 33ed8dba3996 02786c8a83c7 2024-09-16T00:26:56.267 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.gawwgw smithi169 running (3m) 28s ago 3m 169M - 17.2.7-1633-g6d025bc2 33ed8dba3996 dcf3a4dfa879 2024-09-16T00:26:56.267 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.lgeqrj smithi169 running (3m) 28s ago 3m 150M - 17.2.7-1633-g6d025bc2 33ed8dba3996 445c14e55831 2024-09-16T00:26:56.267 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi112.xrkkbc smithi112 *:9283,8765,8443 running (10m) 27s ago 10m 243M - 17.2.7-1633-g6d025bc2 33ed8dba3996 b3d3adb94a85 2024-09-16T00:26:56.268 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi169.mtrobk smithi169 *:8443,9283 running (77s) 28s ago 7m 482M - 19.1.1-326-g8921c8ef f1455bd0219c 519ed31e6dee 2024-09-16T00:26:56.268 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi112 smithi112 running (10m) 27s ago 10m 48.4M 2048M 17.2.7-1633-g6d025bc2 33ed8dba3996 ddaaced54bf8 2024-09-16T00:26:56.268 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi169 smithi169 running (7m) 28s ago 7m 42.3M 2048M 17.2.7-1633-g6d025bc2 33ed8dba3996 6c2289bc3e11 2024-09-16T00:26:56.268 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi112 smithi112 *:9100 starting - - - - 2024-09-16T00:26:56.268 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi169 smithi169 *:9100 running (8m) 28s ago 8m 15.7M - 1.5.0 0da6a335fe13 5d6fe1dc6fa9 2024-09-16T00:26:56.268 INFO:teuthology.orchestra.run.smithi112.stdout:osd.0 smithi112 running (7m) 27s ago 7m 188M 1557M 17.2.7-1633-g6d025bc2 33ed8dba3996 f6b2e146ffdf 2024-09-16T00:26:56.268 INFO:teuthology.orchestra.run.smithi112.stdout:osd.1 smithi112 running (6m) 27s ago 6m 192M 1557M 17.2.7-1633-g6d025bc2 33ed8dba3996 e9bee87bba0f 2024-09-16T00:26:56.268 INFO:teuthology.orchestra.run.smithi112.stdout:osd.2 smithi112 running (6m) 27s ago 6m 152M 1557M 17.2.7-1633-g6d025bc2 33ed8dba3996 100ec4e4852a 2024-09-16T00:26:56.268 INFO:teuthology.orchestra.run.smithi112.stdout:osd.3 smithi169 running (5m) 28s ago 5m 223M 2581M 17.2.7-1633-g6d025bc2 33ed8dba3996 f88463e19f23 2024-09-16T00:26:56.268 INFO:teuthology.orchestra.run.smithi112.stdout:osd.4 smithi169 running (5m) 28s ago 5m 194M 2581M 17.2.7-1633-g6d025bc2 33ed8dba3996 d3e4cdb016b9 2024-09-16T00:26:56.268 INFO:teuthology.orchestra.run.smithi112.stdout:osd.5 smithi169 running (5m) 28s ago 4m 229M 2581M 17.2.7-1633-g6d025bc2 33ed8dba3996 8e7d708d74eb 2024-09-16T00:26:56.268 INFO:teuthology.orchestra.run.smithi112.stdout:prometheus.smithi112 smithi112 *:9095 running (7m) 27s ago 8m 61.1M - 2.43.0 a07b618ecd1d 7f5a86d91011 2024-09-16T00:26:56.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.270+0000 7f1193920640 1 -- 172.21.15.112:0/165644606 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7f117806e6e0 msgr2=0x7f1178070b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:56.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.270+0000 7f1193920640 1 --2- 172.21.15.112:0/165644606 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7f117806e6e0 0x7f1178070b90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f118c10d030 tx=0x7f117c00b040 comp rx=0 tx=0).stop 2024-09-16T00:26:56.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.270+0000 7f1193920640 1 -- 172.21.15.112:0/165644606 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f118c0fa930 msgr2=0x7f118c10d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:56.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.270+0000 7f1193920640 1 --2- 172.21.15.112:0/165644606 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f118c0fa930 0x7f118c10d280 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f118802a530 tx=0x7f1188009da0 comp rx=0 tx=0).stop 2024-09-16T00:26:56.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.270+0000 7f1193920640 1 -- 172.21.15.112:0/165644606 shutdown_connections 2024-09-16T00:26:56.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.270+0000 7f1193920640 1 --2- 172.21.15.112:0/165644606 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7f117806e6e0 0x7f1178070b90 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:56.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.270+0000 7f1193920640 1 --2- 172.21.15.112:0/165644606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f118c0fb280 0x7f118c10d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:56.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.270+0000 7f1193920640 1 --2- 172.21.15.112:0/165644606 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f118c0fa930 0x7f118c10d280 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:56.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.270+0000 7f1193920640 1 -- 172.21.15.112:0/165644606 >> 172.21.15.112:0/165644606 conn(0x7f118c0f6390 msgr2=0x7f118c0f7b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:56.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.270+0000 7f1193920640 1 -- 172.21.15.112:0/165644606 shutdown_connections 2024-09-16T00:26:56.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.270+0000 7f1193920640 1 -- 172.21.15.112:0/165644606 wait complete. 2024-09-16T00:26:56.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.363+0000 7fdde457d640 1 -- 172.21.15.112:0/2858405845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdddc065bc0 msgr2=0x7fdddc0ff1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:56.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.363+0000 7fdde457d640 1 --2- 172.21.15.112:0/2858405845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdddc065bc0 0x7fdddc0ff1c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fddd40048d0 tx=0x7fddd4009da0 comp rx=0 tx=0).stop 2024-09-16T00:26:56.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.363+0000 7fdde457d640 1 -- 172.21.15.112:0/2858405845 shutdown_connections 2024-09-16T00:26:56.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.363+0000 7fdde457d640 1 --2- 172.21.15.112:0/2858405845 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdddc0ff700 0x7fdddc101ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:56.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.363+0000 7fdde457d640 1 --2- 172.21.15.112:0/2858405845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdddc065bc0 0x7fdddc0ff1c0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:56.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.363+0000 7fdde457d640 1 -- 172.21.15.112:0/2858405845 >> 172.21.15.112:0/2858405845 conn(0x7fdddc0f6c10 msgr2=0x7fdddc0f9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:56.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.363+0000 7fdde457d640 1 -- 172.21.15.112:0/2858405845 shutdown_connections 2024-09-16T00:26:56.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.363+0000 7fdde457d640 1 -- 172.21.15.112:0/2858405845 wait complete. 2024-09-16T00:26:56.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.363+0000 7fdde457d640 1 Processor -- start 2024-09-16T00:26:56.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.363+0000 7fdde457d640 1 -- start start 2024-09-16T00:26:56.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.363+0000 7fdde457d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdddc065bc0 0x7fdddc10d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:56.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.363+0000 7fdde457d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdddc0ff700 0x7fdddc10d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:56.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.363+0000 7fdde457d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdddc10dc90 con 0x7fdddc065bc0 2024-09-16T00:26:56.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.363+0000 7fdde457d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdddc10e460 con 0x7fdddc0ff700 2024-09-16T00:26:56.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.364+0000 7fdde2319640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdddc065bc0 0x7fdddc10d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:56.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.364+0000 7fdde2319640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdddc065bc0 0x7fdddc10d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35862/0 (socket says 172.21.15.112:35862) 2024-09-16T00:26:56.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.364+0000 7fdde2319640 1 -- 172.21.15.112:0/2595395566 learned_addr learned my addr 172.21.15.112:0/2595395566 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:26:56.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.364+0000 7fdde1b18640 1 --2- 172.21.15.112:0/2595395566 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdddc0ff700 0x7fdddc10d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:56.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.364+0000 7fdde2319640 1 -- 172.21.15.112:0/2595395566 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdddc0ff700 msgr2=0x7fdddc10d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:56.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.364+0000 7fdde2319640 1 --2- 172.21.15.112:0/2595395566 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdddc0ff700 0x7fdddc10d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:56.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.364+0000 7fdde2319640 1 -- 172.21.15.112:0/2595395566 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddd4005040 con 0x7fdddc065bc0 2024-09-16T00:26:56.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.364+0000 7fdde2319640 1 --2- 172.21.15.112:0/2595395566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdddc065bc0 0x7fdddc10d100 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fddd40043d0 tx=0x7fddd4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:56.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.364+0000 7fdddaffd640 1 -- 172.21.15.112:0/2595395566 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fddd4038070 con 0x7fdddc065bc0 2024-09-16T00:26:56.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.364+0000 7fdddaffd640 1 -- 172.21.15.112:0/2595395566 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fddd402ae90 con 0x7fdddc065bc0 2024-09-16T00:26:56.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.364+0000 7fdde457d640 1 -- 172.21.15.112:0/2595395566 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdddc10e690 con 0x7fdddc065bc0 2024-09-16T00:26:56.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.364+0000 7fdddaffd640 1 -- 172.21.15.112:0/2595395566 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fddd403e670 con 0x7fdddc065bc0 2024-09-16T00:26:56.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.364+0000 7fdde457d640 1 -- 172.21.15.112:0/2595395566 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdddc10ec30 con 0x7fdddc065bc0 2024-09-16T00:26:56.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.365+0000 7fdde457d640 1 -- 172.21.15.112:0/2595395566 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdddc0fa890 con 0x7fdddc065bc0 2024-09-16T00:26:56.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.366+0000 7fdddaffd640 1 -- 172.21.15.112:0/2595395566 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) v1 ==== 91662+0+0 (secure 0 0 0) 0x7fddd4030020 con 0x7fdddc065bc0 2024-09-16T00:26:56.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.366+0000 7fdddaffd640 1 --2- 172.21.15.112:0/2595395566 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7fddc406e6e0 0x7fddc4070b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:56.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.366+0000 7fdddaffd640 1 -- 172.21.15.112:0/2595395566 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fddd40adee0 con 0x7fdddc065bc0 2024-09-16T00:26:56.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.366+0000 7fdde1b18640 1 --2- 172.21.15.112:0/2595395566 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7fddc406e6e0 0x7fddc4070b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:56.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.367+0000 7fdde1b18640 1 --2- 172.21.15.112:0/2595395566 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7fddc406e6e0 0x7fddc4070b90 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fdddc10ceb0 tx=0x7fddd000b040 comp rx=0 tx=0).ready entity=mgr.24341 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:56.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.370+0000 7fdddaffd640 1 -- 172.21.15.112:0/2595395566 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185942 (secure 0 0 0) 0x7fddd40779a0 con 0x7fdddc065bc0 2024-09-16T00:26:56.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.683+0000 7fdde457d640 1 -- 172.21.15.112:0/2595395566 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fdddc104f80 con 0x7fdddc065bc0 2024-09-16T00:26:56.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.684+0000 7fdddaffd640 1 -- 172.21.15.112:0/2595395566 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+855 (secure 0 0 0) 0x7fddd407b060 con 0x7fdddc065bc0 2024-09-16T00:26:56.684 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:26:56.684 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": { 2024-09-16T00:26:56.684 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 17.2.7-1633-g6d025bc2 (6d025bc26f3176c910707f8cb4c8d8e076640615) quincy (stable)": 2 2024-09-16T00:26:56.684 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:26:56.685 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": { 2024-09-16T00:26:56.685 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 17.2.7-1633-g6d025bc2 (6d025bc26f3176c910707f8cb4c8d8e076640615) quincy (stable)": 1, 2024-09-16T00:26:56.685 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 1 2024-09-16T00:26:56.685 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:26:56.685 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": { 2024-09-16T00:26:56.685 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 17.2.7-1633-g6d025bc2 (6d025bc26f3176c910707f8cb4c8d8e076640615) quincy (stable)": 6 2024-09-16T00:26:56.685 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:26:56.685 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": { 2024-09-16T00:26:56.685 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 17.2.7-1633-g6d025bc2 (6d025bc26f3176c910707f8cb4c8d8e076640615) quincy (stable)": 4 2024-09-16T00:26:56.685 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:26:56.685 INFO:teuthology.orchestra.run.smithi112.stdout: "overall": { 2024-09-16T00:26:56.685 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 17.2.7-1633-g6d025bc2 (6d025bc26f3176c910707f8cb4c8d8e076640615) quincy (stable)": 13, 2024-09-16T00:26:56.685 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 1 2024-09-16T00:26:56.685 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-16T00:26:56.686 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:26:56.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.685+0000 7fdde457d640 1 -- 172.21.15.112:0/2595395566 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7fddc406e6e0 msgr2=0x7fddc4070b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:56.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.685+0000 7fdde457d640 1 --2- 172.21.15.112:0/2595395566 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7fddc406e6e0 0x7fddc4070b90 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fdddc10ceb0 tx=0x7fddd000b040 comp rx=0 tx=0).stop 2024-09-16T00:26:56.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.685+0000 7fdde457d640 1 -- 172.21.15.112:0/2595395566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdddc065bc0 msgr2=0x7fdddc10d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:56.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.685+0000 7fdde457d640 1 --2- 172.21.15.112:0/2595395566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdddc065bc0 0x7fdddc10d100 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fddd40043d0 tx=0x7fddd4009da0 comp rx=0 tx=0).stop 2024-09-16T00:26:56.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.686+0000 7fdde457d640 1 -- 172.21.15.112:0/2595395566 shutdown_connections 2024-09-16T00:26:56.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.686+0000 7fdde457d640 1 --2- 172.21.15.112:0/2595395566 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7fddc406e6e0 0x7fddc4070b90 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:56.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.686+0000 7fdde457d640 1 --2- 172.21.15.112:0/2595395566 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdddc0ff700 0x7fdddc10d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:56.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.686+0000 7fdde457d640 1 --2- 172.21.15.112:0/2595395566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdddc065bc0 0x7fdddc10d100 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:56.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.686+0000 7fdde457d640 1 -- 172.21.15.112:0/2595395566 >> 172.21.15.112:0/2595395566 conn(0x7fdddc0f6c10 msgr2=0x7fdddc0f6fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:56.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.686+0000 7fdde457d640 1 -- 172.21.15.112:0/2595395566 shutdown_connections 2024-09-16T00:26:56.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.686+0000 7fdde457d640 1 -- 172.21.15.112:0/2595395566 wait complete. 2024-09-16T00:26:56.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.765+0000 7faba12cd640 1 -- 172.21.15.112:0/2956731229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab9c0fba20 msgr2=0x7fab9c0fbe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:56.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.765+0000 7faba12cd640 1 --2- 172.21.15.112:0/2956731229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab9c0fba20 0x7fab9c0fbe30 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fab8c0048d0 tx=0x7fab8c009da0 comp rx=0 tx=0).stop 2024-09-16T00:26:56.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.765+0000 7faba12cd640 1 -- 172.21.15.112:0/2956731229 shutdown_connections 2024-09-16T00:26:56.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.765+0000 7faba12cd640 1 --2- 172.21.15.112:0/2956731229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab9c0fc370 0x7fab9c0fc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:56.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.765+0000 7faba12cd640 1 --2- 172.21.15.112:0/2956731229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab9c0fba20 0x7fab9c0fbe30 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:56.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.765+0000 7faba12cd640 1 -- 172.21.15.112:0/2956731229 >> 172.21.15.112:0/2956731229 conn(0x7fab9c0f6390 msgr2=0x7fab9c0f87a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:56.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.765+0000 7faba12cd640 1 -- 172.21.15.112:0/2956731229 shutdown_connections 2024-09-16T00:26:56.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.765+0000 7faba12cd640 1 -- 172.21.15.112:0/2956731229 wait complete. 2024-09-16T00:26:56.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.766+0000 7faba12cd640 1 Processor -- start 2024-09-16T00:26:56.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.766+0000 7faba12cd640 1 -- start start 2024-09-16T00:26:56.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.766+0000 7faba12cd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab9c0fba20 0x7fab9c10d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:56.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.766+0000 7faba12cd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab9c0fc370 0x7fab9c10d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:56.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.766+0000 7faba12cd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab9c10dd10 con 0x7fab9c0fc370 2024-09-16T00:26:56.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.766+0000 7faba12cd640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab9c10e4e0 con 0x7fab9c0fba20 2024-09-16T00:26:56.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.766+0000 7fab9a7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab9c0fc370 0x7fab9c10d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:56.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.766+0000 7fab9a7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab9c0fc370 0x7fab9c10d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35874/0 (socket says 172.21.15.112:35874) 2024-09-16T00:26:56.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.766+0000 7fab9a7fc640 1 -- 172.21.15.112:0/2956243925 learned_addr learned my addr 172.21.15.112:0/2956243925 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:26:56.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.767+0000 7fab9affd640 1 --2- 172.21.15.112:0/2956243925 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab9c0fba20 0x7fab9c10d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:56.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.767+0000 7fab9a7fc640 1 -- 172.21.15.112:0/2956243925 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab9c0fba20 msgr2=0x7fab9c10d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:56.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.767+0000 7fab9a7fc640 1 --2- 172.21.15.112:0/2956243925 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab9c0fba20 0x7fab9c10d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:56.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.767+0000 7fab9a7fc640 1 -- 172.21.15.112:0/2956243925 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab8c005040 con 0x7fab9c0fc370 2024-09-16T00:26:56.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.767+0000 7fab9a7fc640 1 --2- 172.21.15.112:0/2956243925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab9c0fc370 0x7fab9c10d6c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fab9000eb70 tx=0x7fab9000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:56.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.767+0000 7fab837fe640 1 -- 172.21.15.112:0/2956243925 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab90007c30 con 0x7fab9c0fc370 2024-09-16T00:26:56.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.767+0000 7faba12cd640 1 -- 172.21.15.112:0/2956243925 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab9c10e710 con 0x7fab9c0fc370 2024-09-16T00:26:56.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.767+0000 7fab837fe640 1 -- 172.21.15.112:0/2956243925 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fab9000f040 con 0x7fab9c0fc370 2024-09-16T00:26:56.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.767+0000 7fab837fe640 1 -- 172.21.15.112:0/2956243925 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab9000ca60 con 0x7fab9c0fc370 2024-09-16T00:26:56.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.767+0000 7faba12cd640 1 -- 172.21.15.112:0/2956243925 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab9c10ecb0 con 0x7fab9c0fc370 2024-09-16T00:26:56.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.769+0000 7faba12cd640 1 -- 172.21.15.112:0/2956243925 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab9c0fa010 con 0x7fab9c0fc370 2024-09-16T00:26:56.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.769+0000 7fab837fe640 1 -- 172.21.15.112:0/2956243925 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) v1 ==== 91662+0+0 (secure 0 0 0) 0x7fab9000cbc0 con 0x7fab9c0fc370 2024-09-16T00:26:56.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.769+0000 7fab837fe640 1 --2- 172.21.15.112:0/2956243925 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7fab8406e6e0 0x7fab84070b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:56.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.770+0000 7fab837fe640 1 -- 172.21.15.112:0/2956243925 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fab9008dc10 con 0x7fab9c0fc370 2024-09-16T00:26:56.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.770+0000 7fab9affd640 1 --2- 172.21.15.112:0/2956243925 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7fab8406e6e0 0x7fab84070b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:56.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.770+0000 7fab9affd640 1 --2- 172.21.15.112:0/2956243925 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7fab8406e6e0 0x7fab84070b90 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fab8c007ac0 tx=0x7fab8c035040 comp rx=0 tx=0).ready entity=mgr.24341 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:56.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:56.774+0000 7fab837fe640 1 -- 172.21.15.112:0/2956243925 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185942 (secure 0 0 0) 0x7fab90057600 con 0x7fab9c0fc370 2024-09-16T00:26:57.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:57.043+0000 7faba12cd640 1 -- 172.21.15.112:0/2956243925 --> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fab9c103fd0 con 0x7fab8406e6e0 2024-09-16T00:26:57.044 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:56 smithi112 ceph-mon[27869]: pgmap v20: 65 pgs: 65 active+clean; 321 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 146 op/s 2024-09-16T00:26:57.044 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:56 smithi112 ceph-mon[27869]: from='client.14650 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:26:57.044 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:56 smithi112 ceph-mon[27869]: from='client.14654 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:26:57.044 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:56 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/2595395566' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:26:57.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:57.044+0000 7fab837fe640 1 -- 172.21.15.112:0/2956243925 <== mgr.24341 v2:172.21.15.169:6828/1220507355 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7fab9c103fd0 con 0x7fab8406e6e0 2024-09-16T00:26:57.045 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:26:57.045 INFO:teuthology.orchestra.run.smithi112.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80", 2024-09-16T00:26:57.045 INFO:teuthology.orchestra.run.smithi112.stdout: "in_progress": true, 2024-09-16T00:26:57.045 INFO:teuthology.orchestra.run.smithi112.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-09-16T00:26:57.045 INFO:teuthology.orchestra.run.smithi112.stdout: "services_complete": [], 2024-09-16T00:26:57.045 INFO:teuthology.orchestra.run.smithi112.stdout: "progress": "1/2 daemons upgraded", 2024-09-16T00:26:57.045 INFO:teuthology.orchestra.run.smithi112.stdout: "message": "", 2024-09-16T00:26:57.045 INFO:teuthology.orchestra.run.smithi112.stdout: "is_paused": false 2024-09-16T00:26:57.045 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:26:57.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:57.046+0000 7faba12cd640 1 -- 172.21.15.112:0/2956243925 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7fab8406e6e0 msgr2=0x7fab84070b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:57.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:57.046+0000 7faba12cd640 1 --2- 172.21.15.112:0/2956243925 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7fab8406e6e0 0x7fab84070b90 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fab8c007ac0 tx=0x7fab8c035040 comp rx=0 tx=0).stop 2024-09-16T00:26:57.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:57.046+0000 7faba12cd640 1 -- 172.21.15.112:0/2956243925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab9c0fc370 msgr2=0x7fab9c10d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:57.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:57.046+0000 7faba12cd640 1 --2- 172.21.15.112:0/2956243925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab9c0fc370 0x7fab9c10d6c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fab9000eb70 tx=0x7fab9000b040 comp rx=0 tx=0).stop 2024-09-16T00:26:57.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:57.047+0000 7faba12cd640 1 -- 172.21.15.112:0/2956243925 shutdown_connections 2024-09-16T00:26:57.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:57.047+0000 7faba12cd640 1 --2- 172.21.15.112:0/2956243925 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7fab8406e6e0 0x7fab84070b90 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:57.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:57.047+0000 7faba12cd640 1 --2- 172.21.15.112:0/2956243925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab9c0fc370 0x7fab9c10d6c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:57.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:57.047+0000 7faba12cd640 1 --2- 172.21.15.112:0/2956243925 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab9c0fba20 0x7fab9c10d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:57.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:57.047+0000 7faba12cd640 1 -- 172.21.15.112:0/2956243925 >> 172.21.15.112:0/2956243925 conn(0x7fab9c0f6390 msgr2=0x7fab9c0fae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:57.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:57.047+0000 7faba12cd640 1 -- 172.21.15.112:0/2956243925 shutdown_connections 2024-09-16T00:26:57.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:26:57.047+0000 7faba12cd640 1 -- 172.21.15.112:0/2956243925 wait complete. 2024-09-16T00:26:57.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:56 smithi169 ceph-mon[34653]: pgmap v20: 65 pgs: 65 active+clean; 321 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 146 op/s 2024-09-16T00:26:57.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:56 smithi169 ceph-mon[34653]: from='client.14650 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:26:57.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:56 smithi169 ceph-mon[34653]: from='client.14654 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:26:57.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:56 smithi169 ceph-mon[34653]: from='client.? 172.21.15.112:0/2595395566' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:26:58.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:58 smithi169 ceph-mon[34653]: from='client.24411 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:26:58.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:58 smithi169 ceph-mon[34653]: from='client.14666 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:26:58.332 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:58 smithi112 ceph-mon[27869]: from='client.24411 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:26:58.332 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:58 smithi112 ceph-mon[27869]: from='client.14666 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:26:59.255 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:26:59 smithi112 ceph-mon[27869]: pgmap v21: 65 pgs: 65 active+clean; 319 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 184 op/s 2024-09-16T00:26:59.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:26:59 smithi169 ceph-mon[34653]: pgmap v21: 65 pgs: 65 active+clean; 319 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 184 op/s 2024-09-16T00:27:01.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:01 smithi169 ceph-mon[34653]: pgmap v22: 65 pgs: 65 active+clean; 324 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 190 op/s 2024-09-16T00:27:01.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:01 smithi112 ceph-mon[27869]: pgmap v22: 65 pgs: 65 active+clean; 324 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 190 op/s 2024-09-16T00:27:03.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:03 smithi169 ceph-mon[34653]: pgmap v23: 65 pgs: 65 active+clean; 324 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 152 op/s 2024-09-16T00:27:03.277 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:03 smithi112 ceph-mon[27869]: pgmap v23: 65 pgs: 65 active+clean; 324 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 152 op/s 2024-09-16T00:27:05.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:05 smithi169 ceph-mon[34653]: pgmap v24: 65 pgs: 65 active+clean; 329 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 264 op/s 2024-09-16T00:27:05.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:05 smithi112 ceph-mon[27869]: pgmap v24: 65 pgs: 65 active+clean; 329 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 264 op/s 2024-09-16T00:27:06.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:06 smithi112 ceph-mon[27869]: pgmap v25: 65 pgs: 65 active+clean; 329 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 190 op/s 2024-09-16T00:27:06.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:06 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:27:06.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:06 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:27:06.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:06 smithi112 ceph-mon[27869]: Reconfiguring grafana.smithi112 (dependencies changed)... 2024-09-16T00:27:06.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:06 smithi112 ceph-mon[27869]: Reconfiguring daemon grafana.smithi112 on smithi112 2024-09-16T00:27:06.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:06 smithi169 ceph-mon[34653]: pgmap v25: 65 pgs: 65 active+clean; 329 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 190 op/s 2024-09-16T00:27:06.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:06 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:27:06.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:06 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:27:06.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:06 smithi169 ceph-mon[34653]: Reconfiguring grafana.smithi112 (dependencies changed)... 2024-09-16T00:27:06.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:06 smithi169 ceph-mon[34653]: Reconfiguring daemon grafana.smithi112 on smithi112 2024-09-16T00:27:07.500 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:07 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:27:07.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:07 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:27:08.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:08 smithi112 ceph-mon[27869]: pgmap v26: 65 pgs: 65 active+clean; 332 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 252 op/s 2024-09-16T00:27:08.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:08 smithi169 ceph-mon[34653]: pgmap v26: 65 pgs: 65 active+clean; 332 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 252 op/s 2024-09-16T00:27:09.935 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.0... 2024-09-16T00:27:09.935 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-16T00:27:10.308 DEBUG:teuthology.parallel:result is None 2024-09-16T00:27:10.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:10 smithi112 ceph-mon[27869]: pgmap v27: 65 pgs: 65 active+clean; 335 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 264 op/s 2024-09-16T00:27:10.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:10 smithi169 ceph-mon[34653]: pgmap v27: 65 pgs: 65 active+clean; 335 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 264 op/s 2024-09-16T00:27:12.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:12 smithi112 ceph-mon[27869]: pgmap v28: 65 pgs: 65 active+clean; 335 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 223 op/s 2024-09-16T00:27:12.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:12 smithi169 ceph-mon[34653]: pgmap v28: 65 pgs: 65 active+clean; 335 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 223 op/s 2024-09-16T00:27:14.213 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:13 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:27:14.213 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:13 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:27:14.213 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:13 smithi112 ceph-mon[27869]: Reconfiguring prometheus.smithi112 (dependencies changed)... 2024-09-16T00:27:14.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:13 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:27:14.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:14 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:27:14.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:14 smithi169 ceph-mon[34653]: Reconfiguring prometheus.smithi112 (dependencies changed)... 2024-09-16T00:27:15.034 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:15 smithi112 ceph-mon[27869]: pgmap v29: 65 pgs: 65 active+clean; 329 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 356 op/s 2024-09-16T00:27:15.034 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:15 smithi112 ceph-mon[27869]: Deploying daemon prometheus.smithi112 on smithi112 2024-09-16T00:27:15.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:15 smithi169 ceph-mon[34653]: pgmap v29: 65 pgs: 65 active+clean; 329 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 356 op/s 2024-09-16T00:27:15.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:15 smithi169 ceph-mon[34653]: Deploying daemon prometheus.smithi112 on smithi112 2024-09-16T00:27:16.343 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.1... 2024-09-16T00:27:16.344 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.1 /home/ubuntu/cephtest/clone.client.1 2024-09-16T00:27:16.636 DEBUG:teuthology.parallel:result is None 2024-09-16T00:27:16.637 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-16T00:27:16.671 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-16T00:27:16.672 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1 2024-09-16T00:27:16.714 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.1/client.1 2024-09-16T00:27:16.714 DEBUG:teuthology.parallel:result is None 2024-09-16T00:27:17.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:17 smithi169 ceph-mon[34653]: pgmap v30: 65 pgs: 65 active+clean; 329 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 244 op/s 2024-09-16T00:27:17.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:17 smithi112 ceph-mon[27869]: pgmap v30: 65 pgs: 65 active+clean; 329 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 244 op/s 2024-09-16T00:27:19.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:19 smithi169 ceph-mon[34653]: pgmap v31: 65 pgs: 65 active+clean; 329 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 315 op/s 2024-09-16T00:27:19.289 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:19 smithi112 ceph-mon[27869]: pgmap v31: 65 pgs: 65 active+clean; 329 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 315 op/s 2024-09-16T00:27:20.733 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:20 smithi169 ceph-mon[34653]: pgmap v32: 65 pgs: 65 active+clean; 329 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 264 op/s 2024-09-16T00:27:20.733 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:20 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:27:20.733 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:20 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:27:20.733 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:20 smithi169 ceph-mon[34653]: Reconfiguring node-exporter.smithi169 (dependencies changed)... 2024-09-16T00:27:20.733 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:20 smithi169 ceph-mon[34653]: Deploying daemon node-exporter.smithi169 on smithi169 2024-09-16T00:27:20.768 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:20 smithi112 ceph-mon[27869]: pgmap v32: 65 pgs: 65 active+clean; 329 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 264 op/s 2024-09-16T00:27:20.768 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:20 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:27:20.768 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:20 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:27:20.768 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:20 smithi112 ceph-mon[27869]: Reconfiguring node-exporter.smithi169 (dependencies changed)... 2024-09-16T00:27:20.768 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:20 smithi112 ceph-mon[27869]: Deploying daemon node-exporter.smithi169 on smithi169 2024-09-16T00:27:21.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:21 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:27:21.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:21 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:27:22.760 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:22 smithi112 ceph-mon[27869]: pgmap v33: 65 pgs: 65 active+clean; 329 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 214 op/s 2024-09-16T00:27:22.760 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:22 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:27:22.760 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:22 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:27:22.760 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:22 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-16T00:27:22.761 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:22 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-16T00:27:22.761 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:22 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-16T00:27:22.761 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:22 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:27:22.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:22 smithi169 ceph-mon[34653]: pgmap v33: 65 pgs: 65 active+clean; 329 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 214 op/s 2024-09-16T00:27:22.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:22 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:27:22.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:22 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:27:22.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:22 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-16T00:27:22.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:22 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-16T00:27:22.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:22 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-16T00:27:22.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:22 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:27:23.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:23 smithi169 ceph-mon[34653]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-16T00:27:23.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:23 smithi169 ceph-mon[34653]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-16T00:27:23.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:23 smithi169 ceph-mon[34653]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-16T00:27:23.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:23 smithi169 ceph-mon[34653]: Upgrade: Need to upgrade myself (mgr.smithi169.mtrobk) 2024-09-16T00:27:23.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:23 smithi169 ceph-mon[34653]: Upgrade: Need to upgrade myself (mgr.smithi169.mtrobk) 2024-09-16T00:27:23.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:23 smithi112 ceph-mon[27869]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-16T00:27:23.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:23 smithi112 ceph-mon[27869]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-16T00:27:23.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:23 smithi112 ceph-mon[27869]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-16T00:27:23.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:23 smithi112 ceph-mon[27869]: Upgrade: Need to upgrade myself (mgr.smithi169.mtrobk) 2024-09-16T00:27:23.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:23 smithi112 ceph-mon[27869]: Upgrade: Need to upgrade myself (mgr.smithi169.mtrobk) 2024-09-16T00:27:25.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:24 smithi169 ceph-mon[34653]: pgmap v34: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 236 op/s 2024-09-16T00:27:25.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:24 smithi169 ceph-mon[34653]: Upgrade: Updating mgr.smithi112.xrkkbc 2024-09-16T00:27:25.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:24 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:27:25.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:24 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi112.xrkkbc", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T00:27:25.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:24 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T00:27:25.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:24 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:27:25.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:24 smithi169 ceph-mon[34653]: Deploying daemon mgr.smithi112.xrkkbc on smithi112 2024-09-16T00:27:25.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:24 smithi112 ceph-mon[27869]: pgmap v34: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 236 op/s 2024-09-16T00:27:25.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:24 smithi112 ceph-mon[27869]: Upgrade: Updating mgr.smithi112.xrkkbc 2024-09-16T00:27:25.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:24 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:27:25.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:24 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi112.xrkkbc", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T00:27:25.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:24 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T00:27:25.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:24 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:27:25.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:24 smithi112 ceph-mon[27869]: Deploying daemon mgr.smithi112.xrkkbc on smithi112 2024-09-16T00:27:26.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:26 smithi112 ceph-mon[27869]: pgmap v35: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 628 KiB/s rd, 550 KiB/s wr, 103 op/s 2024-09-16T00:27:27.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:26 smithi169 ceph-mon[34653]: pgmap v35: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 628 KiB/s rd, 550 KiB/s wr, 103 op/s 2024-09-16T00:27:27.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.130+0000 7f0b86153640 1 -- 172.21.15.112:0/737091972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b800fe4f0 msgr2=0x7f0b800fa650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:27.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.130+0000 7f0b86153640 1 --2- 172.21.15.112:0/737091972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b800fe4f0 0x7f0b800fa650 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f0b740048d0 tx=0x7f0b74009da0 comp rx=0 tx=0).stop 2024-09-16T00:27:27.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.130+0000 7f0b86153640 1 -- 172.21.15.112:0/737091972 shutdown_connections 2024-09-16T00:27:27.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.130+0000 7f0b86153640 1 --2- 172.21.15.112:0/737091972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b800fe4f0 0x7f0b800fa650 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:27.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.130+0000 7f0b86153640 1 --2- 172.21.15.112:0/737091972 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b800fdba0 0x7f0b800fdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:27.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.130+0000 7f0b86153640 1 -- 172.21.15.112:0/737091972 >> 172.21.15.112:0/737091972 conn(0x7f0b800f63a0 msgr2=0x7f0b800f87d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:27.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.130+0000 7f0b86153640 1 -- 172.21.15.112:0/737091972 shutdown_connections 2024-09-16T00:27:27.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.130+0000 7f0b86153640 1 -- 172.21.15.112:0/737091972 wait complete. 2024-09-16T00:27:27.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.130+0000 7f0b86153640 1 Processor -- start 2024-09-16T00:27:27.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.130+0000 7f0b86153640 1 -- start start 2024-09-16T00:27:27.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.131+0000 7f0b86153640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b800fdba0 0x7f0b8010d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:27.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.131+0000 7f0b86153640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b800fe4f0 0x7f0b8010d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:27.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.131+0000 7f0b86153640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8010e4c0 con 0x7f0b800fe4f0 2024-09-16T00:27:27.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.131+0000 7f0b86153640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8010e630 con 0x7f0b800fdba0 2024-09-16T00:27:27.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.131+0000 7f0b7effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b800fe4f0 0x7f0b8010d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:27.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.131+0000 7f0b7effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b800fe4f0 0x7f0b8010d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36200/0 (socket says 172.21.15.112:36200) 2024-09-16T00:27:27.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.131+0000 7f0b7effd640 1 -- 172.21.15.112:0/456137993 learned_addr learned my addr 172.21.15.112:0/456137993 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:27:27.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.131+0000 7f0b7f7fe640 1 --2- 172.21.15.112:0/456137993 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b800fdba0 0x7f0b8010d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:27.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.131+0000 7f0b7effd640 1 -- 172.21.15.112:0/456137993 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b800fdba0 msgr2=0x7f0b8010d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:27.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.131+0000 7f0b7effd640 1 --2- 172.21.15.112:0/456137993 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b800fdba0 0x7f0b8010d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:27.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.131+0000 7f0b7effd640 1 -- 172.21.15.112:0/456137993 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b74005040 con 0x7f0b800fe4f0 2024-09-16T00:27:27.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.131+0000 7f0b7effd640 1 --2- 172.21.15.112:0/456137993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b800fe4f0 0x7f0b8010d650 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0b740048a0 tx=0x7f0b74009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:27.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.131+0000 7f0b67fff640 1 -- 172.21.15.112:0/456137993 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b74038070 con 0x7f0b800fe4f0 2024-09-16T00:27:27.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.132+0000 7f0b86153640 1 -- 172.21.15.112:0/456137993 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b8010e8b0 con 0x7f0b800fe4f0 2024-09-16T00:27:27.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.132+0000 7f0b67fff640 1 -- 172.21.15.112:0/456137993 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f0b7402abd0 con 0x7f0b800fe4f0 2024-09-16T00:27:27.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.132+0000 7f0b67fff640 1 -- 172.21.15.112:0/456137993 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b74033640 con 0x7f0b800fe4f0 2024-09-16T00:27:27.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.132+0000 7f0b86153640 1 -- 172.21.15.112:0/456137993 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b8010edd0 con 0x7f0b800fe4f0 2024-09-16T00:27:27.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.133+0000 7f0b86153640 1 -- 172.21.15.112:0/456137993 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b800fa040 con 0x7f0b800fe4f0 2024-09-16T00:27:27.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.133+0000 7f0b67fff640 1 -- 172.21.15.112:0/456137993 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) v1 ==== 91662+0+0 (secure 0 0 0) 0x7f0b74030020 con 0x7f0b800fe4f0 2024-09-16T00:27:27.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.134+0000 7f0b67fff640 1 --2- 172.21.15.112:0/456137993 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7f0b6806e6e0 0x7f0b68070b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:27.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.134+0000 7f0b67fff640 1 -- 172.21.15.112:0/456137993 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f0b740ae1d0 con 0x7f0b800fe4f0 2024-09-16T00:27:27.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.134+0000 7f0b7f7fe640 1 --2- 172.21.15.112:0/456137993 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7f0b6806e6e0 0x7f0b68070b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:27.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.134+0000 7f0b7f7fe640 1 --2- 172.21.15.112:0/456137993 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7f0b6806e6e0 0x7f0b68070b90 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0b70006fd0 tx=0x7f0b70008040 comp rx=0 tx=0).ready entity=mgr.24341 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:27.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.138+0000 7f0b67fff640 1 -- 172.21.15.112:0/456137993 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185942 (secure 0 0 0) 0x7f0b74077b60 con 0x7f0b800fe4f0 2024-09-16T00:27:27.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.325+0000 7f0b86153640 1 -- 172.21.15.112:0/456137993 --> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0b80105160 con 0x7f0b6806e6e0 2024-09-16T00:27:27.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.327+0000 7f0b67fff640 1 -- 172.21.15.112:0/456137993 <== mgr.24341 v2:172.21.15.169:6828/1220507355 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f0b80105160 con 0x7f0b6806e6e0 2024-09-16T00:27:27.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.329+0000 7f0b86153640 1 -- 172.21.15.112:0/456137993 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7f0b6806e6e0 msgr2=0x7f0b68070b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:27.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.329+0000 7f0b86153640 1 --2- 172.21.15.112:0/456137993 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7f0b6806e6e0 0x7f0b68070b90 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0b70006fd0 tx=0x7f0b70008040 comp rx=0 tx=0).stop 2024-09-16T00:27:27.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.329+0000 7f0b86153640 1 -- 172.21.15.112:0/456137993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b800fe4f0 msgr2=0x7f0b8010d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:27.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.329+0000 7f0b86153640 1 --2- 172.21.15.112:0/456137993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b800fe4f0 0x7f0b8010d650 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0b740048a0 tx=0x7f0b74009da0 comp rx=0 tx=0).stop 2024-09-16T00:27:27.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.329+0000 7f0b86153640 1 -- 172.21.15.112:0/456137993 shutdown_connections 2024-09-16T00:27:27.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.329+0000 7f0b86153640 1 --2- 172.21.15.112:0/456137993 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7f0b6806e6e0 0x7f0b68070b90 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:27.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.329+0000 7f0b86153640 1 --2- 172.21.15.112:0/456137993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b800fe4f0 0x7f0b8010d650 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:27.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.329+0000 7f0b86153640 1 --2- 172.21.15.112:0/456137993 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b800fdba0 0x7f0b8010d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:27.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.329+0000 7f0b86153640 1 -- 172.21.15.112:0/456137993 >> 172.21.15.112:0/456137993 conn(0x7f0b800f63a0 msgr2=0x7f0b800f7b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:27.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.329+0000 7f0b86153640 1 -- 172.21.15.112:0/456137993 shutdown_connections 2024-09-16T00:27:27.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.329+0000 7f0b86153640 1 -- 172.21.15.112:0/456137993 wait complete. 2024-09-16T00:27:27.338 INFO:teuthology.orchestra.run.smithi112.stdout:true 2024-09-16T00:27:27.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.410+0000 7fa313d87640 1 -- 172.21.15.112:0/3837928106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa30c0ff680 msgr2=0x7fa30c101a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:27.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.410+0000 7fa313d87640 1 --2- 172.21.15.112:0/3837928106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa30c0ff680 0x7fa30c101a60 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc0048d0 tx=0x7fa2fc009da0 comp rx=0 tx=0).stop 2024-09-16T00:27:27.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.410+0000 7fa313d87640 1 -- 172.21.15.112:0/3837928106 shutdown_connections 2024-09-16T00:27:27.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.410+0000 7fa313d87640 1 --2- 172.21.15.112:0/3837928106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa30c0ff680 0x7fa30c101a60 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:27.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.410+0000 7fa313d87640 1 --2- 172.21.15.112:0/3837928106 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa30c065c40 0x7fa30c0ff140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:27.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.410+0000 7fa313d87640 1 -- 172.21.15.112:0/3837928106 >> 172.21.15.112:0/3837928106 conn(0x7fa30c0f6c10 msgr2=0x7fa30c0f9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:27.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.410+0000 7fa313d87640 1 -- 172.21.15.112:0/3837928106 shutdown_connections 2024-09-16T00:27:27.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.410+0000 7fa313d87640 1 -- 172.21.15.112:0/3837928106 wait complete. 2024-09-16T00:27:27.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.411+0000 7fa313d87640 1 Processor -- start 2024-09-16T00:27:27.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.411+0000 7fa313d87640 1 -- start start 2024-09-16T00:27:27.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.411+0000 7fa313d87640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa30c065c40 0x7fa30c0fcb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:27.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.411+0000 7fa313d87640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa30c0ff680 0x7fa30c0fb040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:27.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.411+0000 7fa313d87640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa30c0fb5d0 con 0x7fa30c065c40 2024-09-16T00:27:27.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.411+0000 7fa313d87640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa30c0fb710 con 0x7fa30c0ff680 2024-09-16T00:27:27.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.411+0000 7fa311b23640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa30c065c40 0x7fa30c0fcb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:27.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.411+0000 7fa311b23640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa30c065c40 0x7fa30c0fcb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36224/0 (socket says 172.21.15.112:36224) 2024-09-16T00:27:27.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.411+0000 7fa311b23640 1 -- 172.21.15.112:0/1483413634 learned_addr learned my addr 172.21.15.112:0/1483413634 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:27:27.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.411+0000 7fa311322640 1 --2- 172.21.15.112:0/1483413634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa30c0ff680 0x7fa30c0fb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:27.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.411+0000 7fa311b23640 1 -- 172.21.15.112:0/1483413634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa30c0ff680 msgr2=0x7fa30c0fb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:27.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.411+0000 7fa311b23640 1 --2- 172.21.15.112:0/1483413634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa30c0ff680 0x7fa30c0fb040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:27.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.411+0000 7fa311b23640 1 -- 172.21.15.112:0/1483413634 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2fc005040 con 0x7fa30c065c40 2024-09-16T00:27:27.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.412+0000 7fa311b23640 1 --2- 172.21.15.112:0/1483413634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa30c065c40 0x7fa30c0fcb50 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa308002840 tx=0x7fa308004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:27.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.412+0000 7fa3027fc640 1 -- 172.21.15.112:0/1483413634 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa308007c30 con 0x7fa30c065c40 2024-09-16T00:27:27.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.412+0000 7fa3027fc640 1 -- 172.21.15.112:0/1483413634 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fa30800f040 con 0x7fa30c065c40 2024-09-16T00:27:27.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.412+0000 7fa313d87640 1 -- 172.21.15.112:0/1483413634 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa30c0fb9f0 con 0x7fa30c065c40 2024-09-16T00:27:27.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.412+0000 7fa3027fc640 1 -- 172.21.15.112:0/1483413634 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa308009410 con 0x7fa30c065c40 2024-09-16T00:27:27.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.412+0000 7fa313d87640 1 -- 172.21.15.112:0/1483413634 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa30c1150c0 con 0x7fa30c065c40 2024-09-16T00:27:27.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.413+0000 7fa313d87640 1 -- 172.21.15.112:0/1483413634 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa30c04b720 con 0x7fa30c065c40 2024-09-16T00:27:27.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.413+0000 7fa3027fc640 1 -- 172.21.15.112:0/1483413634 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) v1 ==== 91662+0+0 (secure 0 0 0) 0x7fa308009700 con 0x7fa30c065c40 2024-09-16T00:27:27.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.414+0000 7fa3027fc640 1 --2- 172.21.15.112:0/1483413634 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7fa2f806e6e0 0x7fa2f8070b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:27.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.414+0000 7fa3027fc640 1 -- 172.21.15.112:0/1483413634 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fa30808de30 con 0x7fa30c065c40 2024-09-16T00:27:27.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.414+0000 7fa311322640 1 --2- 172.21.15.112:0/1483413634 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7fa2f806e6e0 0x7fa2f8070b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:27.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.415+0000 7fa311322640 1 --2- 172.21.15.112:0/1483413634 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7fa2f806e6e0 0x7fa2f8070b90 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc007ac0 tx=0x7fa2fc035040 comp rx=0 tx=0).ready entity=mgr.24341 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:27.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.418+0000 7fa3027fc640 1 -- 172.21.15.112:0/1483413634 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185942 (secure 0 0 0) 0x7fa308057730 con 0x7fa30c065c40 2024-09-16T00:27:27.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.634+0000 7fa313d87640 1 -- 172.21.15.112:0/1483413634 --> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa30c0f8700 con 0x7fa2f806e6e0 2024-09-16T00:27:27.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.636+0000 7fa3027fc640 1 -- 172.21.15.112:0/1483413634 <== mgr.24341 v2:172.21.15.169:6828/1220507355 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7fa30c0f8700 con 0x7fa2f806e6e0 2024-09-16T00:27:27.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.638+0000 7fa313d87640 1 -- 172.21.15.112:0/1483413634 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7fa2f806e6e0 msgr2=0x7fa2f8070b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:27.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.638+0000 7fa313d87640 1 --2- 172.21.15.112:0/1483413634 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7fa2f806e6e0 0x7fa2f8070b90 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc007ac0 tx=0x7fa2fc035040 comp rx=0 tx=0).stop 2024-09-16T00:27:27.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.638+0000 7fa313d87640 1 -- 172.21.15.112:0/1483413634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa30c065c40 msgr2=0x7fa30c0fcb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:27.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.638+0000 7fa313d87640 1 --2- 172.21.15.112:0/1483413634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa30c065c40 0x7fa30c0fcb50 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa308002840 tx=0x7fa308004570 comp rx=0 tx=0).stop 2024-09-16T00:27:27.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.638+0000 7fa313d87640 1 -- 172.21.15.112:0/1483413634 shutdown_connections 2024-09-16T00:27:27.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.638+0000 7fa313d87640 1 --2- 172.21.15.112:0/1483413634 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7fa2f806e6e0 0x7fa2f8070b90 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:27.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.638+0000 7fa313d87640 1 --2- 172.21.15.112:0/1483413634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa30c0ff680 0x7fa30c0fb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:27.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.638+0000 7fa313d87640 1 --2- 172.21.15.112:0/1483413634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa30c065c40 0x7fa30c0fcb50 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:27.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.638+0000 7fa313d87640 1 -- 172.21.15.112:0/1483413634 >> 172.21.15.112:0/1483413634 conn(0x7fa30c0f6c10 msgr2=0x7fa30c0f6fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:27.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.639+0000 7fa313d87640 1 -- 172.21.15.112:0/1483413634 shutdown_connections 2024-09-16T00:27:27.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.639+0000 7fa313d87640 1 -- 172.21.15.112:0/1483413634 wait complete. 2024-09-16T00:27:27.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.722+0000 7f738f671640 1 -- 172.21.15.112:0/2790174851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7388065bc0 msgr2=0x7f73880ff1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:27.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.722+0000 7f738f671640 1 --2- 172.21.15.112:0/2790174851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7388065bc0 0x7f73880ff1c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f73840048d0 tx=0x7f7384009da0 comp rx=0 tx=0).stop 2024-09-16T00:27:27.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.722+0000 7f738f671640 1 -- 172.21.15.112:0/2790174851 shutdown_connections 2024-09-16T00:27:27.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.722+0000 7f738f671640 1 --2- 172.21.15.112:0/2790174851 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73880ff700 0x7f7388101ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:27.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.722+0000 7f738f671640 1 --2- 172.21.15.112:0/2790174851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7388065bc0 0x7f73880ff1c0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:27.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.722+0000 7f738f671640 1 -- 172.21.15.112:0/2790174851 >> 172.21.15.112:0/2790174851 conn(0x7f73880f6c10 msgr2=0x7f73880f9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:27.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.722+0000 7f738f671640 1 -- 172.21.15.112:0/2790174851 shutdown_connections 2024-09-16T00:27:27.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.722+0000 7f738f671640 1 -- 172.21.15.112:0/2790174851 wait complete. 2024-09-16T00:27:27.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.722+0000 7f738f671640 1 Processor -- start 2024-09-16T00:27:27.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.723+0000 7f738f671640 1 -- start start 2024-09-16T00:27:27.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.723+0000 7f738f671640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7388065bc0 0x7f738810d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:27.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.723+0000 7f738f671640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73880ff700 0x7f738810d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:27.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.723+0000 7f738f671640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f738810dc90 con 0x7f73880ff700 2024-09-16T00:27:27.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.723+0000 7f738f671640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f738810e460 con 0x7f7388065bc0 2024-09-16T00:27:27.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.723+0000 7f738d40d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7388065bc0 0x7f738810d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:27.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.723+0000 7f738cc0c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73880ff700 0x7f738810d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:27.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.723+0000 7f738d40d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7388065bc0 0x7f738810d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.112:46230/0 (socket says 172.21.15.112:46230) 2024-09-16T00:27:27.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.723+0000 7f738d40d640 1 -- 172.21.15.112:0/4206223144 learned_addr learned my addr 172.21.15.112:0/4206223144 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:27:27.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.723+0000 7f738cc0c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73880ff700 0x7f738810d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36272/0 (socket says 172.21.15.112:36272) 2024-09-16T00:27:27.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.723+0000 7f738d40d640 1 -- 172.21.15.112:0/4206223144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73880ff700 msgr2=0x7f738810d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:27.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.723+0000 7f738d40d640 1 --2- 172.21.15.112:0/4206223144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73880ff700 0x7f738810d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:27.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.723+0000 7f738d40d640 1 -- 172.21.15.112:0/4206223144 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7384005040 con 0x7f7388065bc0 2024-09-16T00:27:27.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.724+0000 7f738d40d640 1 --2- 172.21.15.112:0/4206223144 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7388065bc0 0x7f738810d100 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f738402d040 tx=0x7f7384009da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:27.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.724+0000 7f737dffb640 1 -- 172.21.15.112:0/4206223144 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f738403c070 con 0x7f7388065bc0 2024-09-16T00:27:27.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.724+0000 7f737dffb640 1 -- 172.21.15.112:0/4206223144 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f738402ae90 con 0x7f7388065bc0 2024-09-16T00:27:27.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.724+0000 7f738f671640 1 -- 172.21.15.112:0/4206223144 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f738810e690 con 0x7f7388065bc0 2024-09-16T00:27:27.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.724+0000 7f737dffb640 1 -- 172.21.15.112:0/4206223144 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f738402c350 con 0x7f7388065bc0 2024-09-16T00:27:27.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.724+0000 7f738f671640 1 -- 172.21.15.112:0/4206223144 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f738810ec30 con 0x7f7388065bc0 2024-09-16T00:27:27.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.725+0000 7f738f671640 1 -- 172.21.15.112:0/4206223144 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73880fa890 con 0x7f7388065bc0 2024-09-16T00:27:27.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.726+0000 7f737dffb640 1 -- 172.21.15.112:0/4206223144 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 25) v1 ==== 91662+0+0 (secure 0 0 0) 0x7f7384030020 con 0x7f7388065bc0 2024-09-16T00:27:27.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.726+0000 7f737dffb640 1 --2- 172.21.15.112:0/4206223144 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7f737406e6e0 0x7f7374070b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:27.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.726+0000 7f737dffb640 1 -- 172.21.15.112:0/4206223144 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f73840b23d0 con 0x7f7388065bc0 2024-09-16T00:27:27.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.726+0000 7f738cc0c640 1 --2- 172.21.15.112:0/4206223144 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7f737406e6e0 0x7f7374070b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:27.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.727+0000 7f738cc0c640 1 --2- 172.21.15.112:0/4206223144 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7f737406e6e0 0x7f7374070b90 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f738810ceb0 tx=0x7f737800b040 comp rx=0 tx=0).ready entity=mgr.24341 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:27.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.731+0000 7f737dffb640 1 -- 172.21.15.112:0/4206223144 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185942 (secure 0 0 0) 0x7f738407bd90 con 0x7f7388065bc0 2024-09-16T00:27:27.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.918+0000 7f738f671640 1 -- 172.21.15.112:0/4206223144 --> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f738810eef0 con 0x7f737406e6e0 2024-09-16T00:27:27.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.928+0000 7f737dffb640 1 -- 172.21.15.112:0/4206223144 <== mgr.24341 v2:172.21.15.169:6828/1220507355 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f738810eef0 con 0x7f737406e6e0 2024-09-16T00:27:27.929 INFO:teuthology.orchestra.run.smithi112.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T00:27:27.929 INFO:teuthology.orchestra.run.smithi112.stdout:alertmanager.smithi112 smithi112 *:9093,9094 starting - - - - 2024-09-16T00:27:27.929 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi112 smithi112 running (9m) 58s ago 9m 6891k - 17.2.7-1633-g6d025bc2 33ed8dba3996 c94220ed2776 2024-09-16T00:27:27.929 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi169 smithi169 running (8m) 60s ago 8m 6899k - 17.2.7-1633-g6d025bc2 33ed8dba3996 d0d3726280a2 2024-09-16T00:27:27.929 INFO:teuthology.orchestra.run.smithi112.stdout:grafana.smithi112 smithi112 *:3000 starting - - - - 2024-09-16T00:27:27.929 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.hxfwkh smithi112 running (3m) 58s ago 3m 165M - 17.2.7-1633-g6d025bc2 33ed8dba3996 cf92d9461fc6 2024-09-16T00:27:27.929 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.ithwzj smithi112 running (4m) 58s ago 4m 138M - 17.2.7-1633-g6d025bc2 33ed8dba3996 02786c8a83c7 2024-09-16T00:27:27.929 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.gawwgw smithi169 running (3m) 60s ago 3m 169M - 17.2.7-1633-g6d025bc2 33ed8dba3996 dcf3a4dfa879 2024-09-16T00:27:27.929 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.lgeqrj smithi169 running (3m) 60s ago 3m 150M - 17.2.7-1633-g6d025bc2 33ed8dba3996 445c14e55831 2024-09-16T00:27:27.930 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi112.xrkkbc smithi112 *:8443,9283,8765 starting - - - - 2024-09-16T00:27:27.930 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi169.mtrobk smithi169 *:8443,9283 running (109s) 60s ago 8m 482M - 19.1.1-326-g8921c8ef f1455bd0219c 519ed31e6dee 2024-09-16T00:27:27.930 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi112 smithi112 running (10m) 58s ago 10m 48.4M 2048M 17.2.7-1633-g6d025bc2 33ed8dba3996 ddaaced54bf8 2024-09-16T00:27:27.930 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi169 smithi169 running (8m) 60s ago 8m 42.3M 2048M 17.2.7-1633-g6d025bc2 33ed8dba3996 6c2289bc3e11 2024-09-16T00:27:27.930 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi112 smithi112 *:9100 starting - - - - 2024-09-16T00:27:27.930 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi169 smithi169 *:9100 starting - - - - 2024-09-16T00:27:27.930 INFO:teuthology.orchestra.run.smithi112.stdout:osd.0 smithi112 running (7m) 58s ago 7m 188M 1557M 17.2.7-1633-g6d025bc2 33ed8dba3996 f6b2e146ffdf 2024-09-16T00:27:27.930 INFO:teuthology.orchestra.run.smithi112.stdout:osd.1 smithi112 running (7m) 58s ago 7m 192M 1557M 17.2.7-1633-g6d025bc2 33ed8dba3996 e9bee87bba0f 2024-09-16T00:27:27.930 INFO:teuthology.orchestra.run.smithi112.stdout:osd.2 smithi112 running (6m) 58s ago 6m 152M 1557M 17.2.7-1633-g6d025bc2 33ed8dba3996 100ec4e4852a 2024-09-16T00:27:27.930 INFO:teuthology.orchestra.run.smithi112.stdout:osd.3 smithi169 running (6m) 60s ago 6m 223M 2581M 17.2.7-1633-g6d025bc2 33ed8dba3996 f88463e19f23 2024-09-16T00:27:27.930 INFO:teuthology.orchestra.run.smithi112.stdout:osd.4 smithi169 running (5m) 60s ago 5m 194M 2581M 17.2.7-1633-g6d025bc2 33ed8dba3996 d3e4cdb016b9 2024-09-16T00:27:27.930 INFO:teuthology.orchestra.run.smithi112.stdout:osd.5 smithi169 running (5m) 60s ago 5m 229M 2581M 17.2.7-1633-g6d025bc2 33ed8dba3996 8e7d708d74eb 2024-09-16T00:27:27.930 INFO:teuthology.orchestra.run.smithi112.stdout:prometheus.smithi112 smithi112 *:9095 starting - - - - 2024-09-16T00:27:27.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.931+0000 7f738f671640 1 -- 172.21.15.112:0/4206223144 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7f737406e6e0 msgr2=0x7f7374070b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:27.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.931+0000 7f738f671640 1 --2- 172.21.15.112:0/4206223144 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7f737406e6e0 0x7f7374070b90 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f738810ceb0 tx=0x7f737800b040 comp rx=0 tx=0).stop 2024-09-16T00:27:27.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.931+0000 7f738f671640 1 -- 172.21.15.112:0/4206223144 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7388065bc0 msgr2=0x7f738810d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:27.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.931+0000 7f738f671640 1 --2- 172.21.15.112:0/4206223144 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7388065bc0 0x7f738810d100 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f738402d040 tx=0x7f7384009da0 comp rx=0 tx=0).stop 2024-09-16T00:27:27.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.931+0000 7f738f671640 1 -- 172.21.15.112:0/4206223144 shutdown_connections 2024-09-16T00:27:27.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.931+0000 7f738f671640 1 --2- 172.21.15.112:0/4206223144 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7f737406e6e0 0x7f7374070b90 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:27.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.931+0000 7f738f671640 1 --2- 172.21.15.112:0/4206223144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73880ff700 0x7f738810d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:27.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.931+0000 7f738f671640 1 --2- 172.21.15.112:0/4206223144 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7388065bc0 0x7f738810d100 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:27.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.931+0000 7f738f671640 1 -- 172.21.15.112:0/4206223144 >> 172.21.15.112:0/4206223144 conn(0x7f73880f6c10 msgr2=0x7f73880f6fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:27.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.931+0000 7f738f671640 1 -- 172.21.15.112:0/4206223144 shutdown_connections 2024-09-16T00:27:27.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:27.931+0000 7f738f671640 1 -- 172.21.15.112:0/4206223144 wait complete. 2024-09-16T00:27:28.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.013+0000 7f08564ab640 1 -- 172.21.15.112:0/1691699888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08500fc520 msgr2=0x7f08500fc930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:28.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.013+0000 7f08564ab640 1 --2- 172.21.15.112:0/1691699888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08500fc520 0x7f08500fc930 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f08400048d0 tx=0x7f0840009da0 comp rx=0 tx=0).stop 2024-09-16T00:27:28.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.013+0000 7f08564ab640 1 -- 172.21.15.112:0/1691699888 shutdown_connections 2024-09-16T00:27:28.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.013+0000 7f08564ab640 1 --2- 172.21.15.112:0/1691699888 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08500fd7f0 0x7f08500fdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:28.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.013+0000 7f08564ab640 1 --2- 172.21.15.112:0/1691699888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08500fc520 0x7f08500fc930 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:28.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.013+0000 7f08564ab640 1 -- 172.21.15.112:0/1691699888 >> 172.21.15.112:0/1691699888 conn(0x7f08500f7ce0 msgr2=0x7f08500fa110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:28.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.013+0000 7f08564ab640 1 -- 172.21.15.112:0/1691699888 shutdown_connections 2024-09-16T00:27:28.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.013+0000 7f08564ab640 1 -- 172.21.15.112:0/1691699888 wait complete. 2024-09-16T00:27:28.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.014+0000 7f08564ab640 1 Processor -- start 2024-09-16T00:27:28.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.014+0000 7f08564ab640 1 -- start start 2024-09-16T00:27:28.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.014+0000 7f08564ab640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08500fc520 0x7f0850071570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:28.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.014+0000 7f08564ab640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08500fd7f0 0x7f085006fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:28.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.014+0000 7f08564ab640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f085006fff0 con 0x7f08500fc520 2024-09-16T00:27:28.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.014+0000 7f08564ab640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0850070130 con 0x7f08500fd7f0 2024-09-16T00:27:28.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.014+0000 7f084ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08500fc520 0x7f0850071570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:28.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.014+0000 7f084ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08500fc520 0x7f0850071570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36292/0 (socket says 172.21.15.112:36292) 2024-09-16T00:27:28.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.014+0000 7f084ffff640 1 -- 172.21.15.112:0/3572623925 learned_addr learned my addr 172.21.15.112:0/3572623925 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:27:28.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.014+0000 7f084f7fe640 1 --2- 172.21.15.112:0/3572623925 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08500fd7f0 0x7f085006fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:28.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.015+0000 7f084ffff640 1 -- 172.21.15.112:0/3572623925 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08500fd7f0 msgr2=0x7f085006fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:28.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.015+0000 7f084ffff640 1 --2- 172.21.15.112:0/3572623925 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08500fd7f0 0x7f085006fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:28.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.015+0000 7f084ffff640 1 -- 172.21.15.112:0/3572623925 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0840005040 con 0x7f08500fc520 2024-09-16T00:27:28.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.015+0000 7f084ffff640 1 --2- 172.21.15.112:0/3572623925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08500fc520 0x7f0850071570 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f08400043d0 tx=0x7f0840009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:28.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.015+0000 7f084cff9640 1 -- 172.21.15.112:0/3572623925 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0840038070 con 0x7f08500fc520 2024-09-16T00:27:28.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.015+0000 7f084cff9640 1 -- 172.21.15.112:0/3572623925 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f084002ae90 con 0x7f08500fc520 2024-09-16T00:27:28.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.015+0000 7f08564ab640 1 -- 172.21.15.112:0/3572623925 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08500703b0 con 0x7f08500fc520 2024-09-16T00:27:28.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.015+0000 7f084cff9640 1 -- 172.21.15.112:0/3572623925 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f084003e670 con 0x7f08500fc520 2024-09-16T00:27:28.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.015+0000 7f08564ab640 1 -- 172.21.15.112:0/3572623925 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0850115100 con 0x7f08500fc520 2024-09-16T00:27:28.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.016+0000 7f08564ab640 1 -- 172.21.15.112:0/3572623925 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08500fb980 con 0x7f08500fc520 2024-09-16T00:27:28.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.017+0000 7f084cff9640 1 -- 172.21.15.112:0/3572623925 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) v1 ==== 91662+0+0 (secure 0 0 0) 0x7f0840030020 con 0x7f08500fc520 2024-09-16T00:27:28.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.018+0000 7f084cff9640 1 --2- 172.21.15.112:0/3572623925 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7f083806e6e0 0x7f0838070b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:28.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.018+0000 7f084cff9640 1 -- 172.21.15.112:0/3572623925 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f08400ae0a0 con 0x7f08500fc520 2024-09-16T00:27:28.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.018+0000 7f084f7fe640 1 --2- 172.21.15.112:0/3572623925 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7f083806e6e0 0x7f0838070b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:28.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.018+0000 7f084f7fe640 1 --2- 172.21.15.112:0/3572623925 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7f083806e6e0 0x7f0838070b90 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0844004ed0 tx=0x7f084400b040 comp rx=0 tx=0).ready entity=mgr.24341 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:28.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.025+0000 7f084cff9640 1 -- 172.21.15.112:0/3572623925 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185942 (secure 0 0 0) 0x7f0840077b60 con 0x7f08500fc520 2024-09-16T00:27:28.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.313+0000 7f08564ab640 1 -- 172.21.15.112:0/3572623925 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f0850115290 con 0x7f08500fc520 2024-09-16T00:27:28.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.314+0000 7f084cff9640 1 -- 172.21.15.112:0/3572623925 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+855 (secure 0 0 0) 0x7f084007b220 con 0x7f08500fc520 2024-09-16T00:27:28.315 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:27:28.316 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": { 2024-09-16T00:27:28.316 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 17.2.7-1633-g6d025bc2 (6d025bc26f3176c910707f8cb4c8d8e076640615) quincy (stable)": 2 2024-09-16T00:27:28.316 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:27:28.316 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": { 2024-09-16T00:27:28.316 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 17.2.7-1633-g6d025bc2 (6d025bc26f3176c910707f8cb4c8d8e076640615) quincy (stable)": 1, 2024-09-16T00:27:28.316 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 1 2024-09-16T00:27:28.316 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:27:28.316 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": { 2024-09-16T00:27:28.316 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 17.2.7-1633-g6d025bc2 (6d025bc26f3176c910707f8cb4c8d8e076640615) quincy (stable)": 6 2024-09-16T00:27:28.316 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:27:28.316 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": { 2024-09-16T00:27:28.316 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 17.2.7-1633-g6d025bc2 (6d025bc26f3176c910707f8cb4c8d8e076640615) quincy (stable)": 4 2024-09-16T00:27:28.317 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:27:28.317 INFO:teuthology.orchestra.run.smithi112.stdout: "overall": { 2024-09-16T00:27:28.317 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 17.2.7-1633-g6d025bc2 (6d025bc26f3176c910707f8cb4c8d8e076640615) quincy (stable)": 13, 2024-09-16T00:27:28.317 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 1 2024-09-16T00:27:28.317 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-16T00:27:28.317 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:27:28.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.316+0000 7f08564ab640 1 -- 172.21.15.112:0/3572623925 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7f083806e6e0 msgr2=0x7f0838070b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:28.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.316+0000 7f08564ab640 1 --2- 172.21.15.112:0/3572623925 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7f083806e6e0 0x7f0838070b90 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0844004ed0 tx=0x7f084400b040 comp rx=0 tx=0).stop 2024-09-16T00:27:28.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.316+0000 7f08564ab640 1 -- 172.21.15.112:0/3572623925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08500fc520 msgr2=0x7f0850071570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:28.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.316+0000 7f08564ab640 1 --2- 172.21.15.112:0/3572623925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08500fc520 0x7f0850071570 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f08400043d0 tx=0x7f0840009da0 comp rx=0 tx=0).stop 2024-09-16T00:27:28.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.316+0000 7f08564ab640 1 -- 172.21.15.112:0/3572623925 shutdown_connections 2024-09-16T00:27:28.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.316+0000 7f08564ab640 1 --2- 172.21.15.112:0/3572623925 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7f083806e6e0 0x7f0838070b90 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:28.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.316+0000 7f08564ab640 1 --2- 172.21.15.112:0/3572623925 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08500fd7f0 0x7f085006fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:28.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.316+0000 7f08564ab640 1 --2- 172.21.15.112:0/3572623925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08500fc520 0x7f0850071570 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:28.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.316+0000 7f08564ab640 1 -- 172.21.15.112:0/3572623925 >> 172.21.15.112:0/3572623925 conn(0x7f08500f7ce0 msgr2=0x7f08500f9500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:28.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.316+0000 7f08564ab640 1 -- 172.21.15.112:0/3572623925 shutdown_connections 2024-09-16T00:27:28.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.316+0000 7f08564ab640 1 -- 172.21.15.112:0/3572623925 wait complete. 2024-09-16T00:27:28.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.399+0000 7f5515104640 1 -- 172.21.15.112:0/2561770968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55100fc700 msgr2=0x7f55100fcb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:28.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.399+0000 7f5515104640 1 --2- 172.21.15.112:0/2561770968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55100fc700 0x7f55100fcb70 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f54f80048d0 tx=0x7f54f8009da0 comp rx=0 tx=0).stop 2024-09-16T00:27:28.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.399+0000 7f5515104640 1 -- 172.21.15.112:0/2561770968 shutdown_connections 2024-09-16T00:27:28.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.399+0000 7f5515104640 1 --2- 172.21.15.112:0/2561770968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55100fc700 0x7f55100fcb70 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:28.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.399+0000 7f5515104640 1 --2- 172.21.15.112:0/2561770968 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55100fb430 0x7f55100fb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:28.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.399+0000 7f5515104640 1 -- 172.21.15.112:0/2561770968 >> 172.21.15.112:0/2561770968 conn(0x7f55100f6c10 msgr2=0x7f55100f9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:28.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.399+0000 7f5515104640 1 -- 172.21.15.112:0/2561770968 shutdown_connections 2024-09-16T00:27:28.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.399+0000 7f5515104640 1 -- 172.21.15.112:0/2561770968 wait complete. 2024-09-16T00:27:28.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.400+0000 7f5515104640 1 Processor -- start 2024-09-16T00:27:28.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.400+0000 7f5515104640 1 -- start start 2024-09-16T00:27:28.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.400+0000 7f5515104640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55100fb430 0x7f551010bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:28.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.400+0000 7f5515104640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55100fc700 0x7f551010c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:28.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.400+0000 7f5515104640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f551010cb10 con 0x7f55100fb430 2024-09-16T00:27:28.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.400+0000 7f5515104640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f551010d330 con 0x7f55100fc700 2024-09-16T00:27:28.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.400+0000 7f550ed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55100fb430 0x7f551010bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:28.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.400+0000 7f550ed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55100fb430 0x7f551010bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36322/0 (socket says 172.21.15.112:36322) 2024-09-16T00:27:28.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.400+0000 7f550ed9d640 1 -- 172.21.15.112:0/2859250616 learned_addr learned my addr 172.21.15.112:0/2859250616 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:27:28.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.401+0000 7f550e59c640 1 --2- 172.21.15.112:0/2859250616 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55100fc700 0x7f551010c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:28.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.401+0000 7f550ed9d640 1 -- 172.21.15.112:0/2859250616 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55100fc700 msgr2=0x7f551010c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:28.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.401+0000 7f550ed9d640 1 --2- 172.21.15.112:0/2859250616 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55100fc700 0x7f551010c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:28.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.401+0000 7f550ed9d640 1 -- 172.21.15.112:0/2859250616 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54f8005040 con 0x7f55100fb430 2024-09-16T00:27:28.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.401+0000 7f550ed9d640 1 --2- 172.21.15.112:0/2859250616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55100fb430 0x7f551010bf80 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f550000eb70 tx=0x7f550000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:28.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.401+0000 7f55077fe640 1 -- 172.21.15.112:0/2859250616 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5500007c30 con 0x7f55100fb430 2024-09-16T00:27:28.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.402+0000 7f5515104640 1 -- 172.21.15.112:0/2859250616 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f551010d560 con 0x7f55100fb430 2024-09-16T00:27:28.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.401+0000 7f55077fe640 1 -- 172.21.15.112:0/2859250616 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f550000f040 con 0x7f55100fb430 2024-09-16T00:27:28.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.402+0000 7f55077fe640 1 -- 172.21.15.112:0/2859250616 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f550000ca60 con 0x7f55100fb430 2024-09-16T00:27:28.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.402+0000 7f5515104640 1 -- 172.21.15.112:0/2859250616 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f551010db00 con 0x7f55100fb430 2024-09-16T00:27:28.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.403+0000 7f5515104640 1 -- 172.21.15.112:0/2859250616 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55100fa890 con 0x7f55100fb430 2024-09-16T00:27:28.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.403+0000 7f55077fe640 1 -- 172.21.15.112:0/2859250616 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) v1 ==== 91662+0+0 (secure 0 0 0) 0x7f550000cbc0 con 0x7f55100fb430 2024-09-16T00:27:28.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.404+0000 7f55077fe640 1 --2- 172.21.15.112:0/2859250616 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7f54fc06e6e0 0x7f54fc070b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:28.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.404+0000 7f55077fe640 1 -- 172.21.15.112:0/2859250616 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f550008db40 con 0x7f55100fb430 2024-09-16T00:27:28.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.404+0000 7f550e59c640 1 --2- 172.21.15.112:0/2859250616 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7f54fc06e6e0 0x7f54fc070b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:28.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.404+0000 7f550e59c640 1 --2- 172.21.15.112:0/2859250616 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7f54fc06e6e0 0x7f54fc070b90 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f54f8007ac0 tx=0x7f54f8035040 comp rx=0 tx=0).ready entity=mgr.24341 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:28.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.408+0000 7f55077fe640 1 -- 172.21.15.112:0/2859250616 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185942 (secure 0 0 0) 0x7f55000575b0 con 0x7f55100fb430 2024-09-16T00:27:28.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:28 smithi112 ceph-mon[27869]: pgmap v36: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 629 KiB/s rd, 551 KiB/s wr, 105 op/s 2024-09-16T00:27:28.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:28 smithi112 ceph-mon[27869]: from='client.14670 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:27:28.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:28 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:27:28.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:28 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:27:28.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:28 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:27:28.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:28 smithi112 ceph-mon[27869]: from='client.14674 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:27:28.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:28 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:27:28.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:28 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:27:28.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:28 smithi112 ceph-mon[27869]: from='client.24431 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:27:28.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:28 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/3572623925' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:27:28.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.600+0000 7f5515104640 1 -- 172.21.15.112:0/2859250616 --> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f551010de80 con 0x7f54fc06e6e0 2024-09-16T00:27:28.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.602+0000 7f55077fe640 1 -- 172.21.15.112:0/2859250616 <== mgr.24341 v2:172.21.15.169:6828/1220507355 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7f551010de80 con 0x7f54fc06e6e0 2024-09-16T00:27:28.602 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:27:28.602 INFO:teuthology.orchestra.run.smithi112.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80", 2024-09-16T00:27:28.602 INFO:teuthology.orchestra.run.smithi112.stdout: "in_progress": true, 2024-09-16T00:27:28.602 INFO:teuthology.orchestra.run.smithi112.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-09-16T00:27:28.603 INFO:teuthology.orchestra.run.smithi112.stdout: "services_complete": [], 2024-09-16T00:27:28.603 INFO:teuthology.orchestra.run.smithi112.stdout: "progress": "", 2024-09-16T00:27:28.603 INFO:teuthology.orchestra.run.smithi112.stdout: "message": "Currently upgrading mgr daemons", 2024-09-16T00:27:28.603 INFO:teuthology.orchestra.run.smithi112.stdout: "is_paused": false 2024-09-16T00:27:28.603 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:27:28.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.604+0000 7f5515104640 1 -- 172.21.15.112:0/2859250616 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7f54fc06e6e0 msgr2=0x7f54fc070b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:28.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.604+0000 7f5515104640 1 --2- 172.21.15.112:0/2859250616 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7f54fc06e6e0 0x7f54fc070b90 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f54f8007ac0 tx=0x7f54f8035040 comp rx=0 tx=0).stop 2024-09-16T00:27:28.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.604+0000 7f5515104640 1 -- 172.21.15.112:0/2859250616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55100fb430 msgr2=0x7f551010bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:28.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.604+0000 7f5515104640 1 --2- 172.21.15.112:0/2859250616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55100fb430 0x7f551010bf80 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f550000eb70 tx=0x7f550000b040 comp rx=0 tx=0).stop 2024-09-16T00:27:28.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.604+0000 7f5515104640 1 -- 172.21.15.112:0/2859250616 shutdown_connections 2024-09-16T00:27:28.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.604+0000 7f5515104640 1 --2- 172.21.15.112:0/2859250616 >> [v2:172.21.15.169:6828/1220507355,v1:172.21.15.169:6829/1220507355] conn(0x7f54fc06e6e0 0x7f54fc070b90 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:28.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.604+0000 7f5515104640 1 --2- 172.21.15.112:0/2859250616 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55100fc700 0x7f551010c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:28.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.604+0000 7f5515104640 1 --2- 172.21.15.112:0/2859250616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55100fb430 0x7f551010bf80 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:28.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.604+0000 7f5515104640 1 -- 172.21.15.112:0/2859250616 >> 172.21.15.112:0/2859250616 conn(0x7f55100f6c10 msgr2=0x7f55100f83f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:28.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.604+0000 7f5515104640 1 -- 172.21.15.112:0/2859250616 shutdown_connections 2024-09-16T00:27:28.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:28.604+0000 7f5515104640 1 -- 172.21.15.112:0/2859250616 wait complete. 2024-09-16T00:27:29.012 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:28 smithi169 ceph-mon[34653]: pgmap v36: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 629 KiB/s rd, 551 KiB/s wr, 105 op/s 2024-09-16T00:27:29.012 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:28 smithi169 ceph-mon[34653]: from='client.14670 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:27:29.013 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:28 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:27:29.013 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:28 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:27:29.013 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:28 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:27:29.013 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:28 smithi169 ceph-mon[34653]: from='client.14674 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:27:29.013 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:28 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:27:29.014 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:28 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:27:29.014 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:28 smithi169 ceph-mon[34653]: from='client.24431 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:27:29.014 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:28 smithi169 ceph-mon[34653]: from='client.? 172.21.15.112:0/3572623925' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:27:29.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:29 smithi112 ceph-mon[27869]: from='client.14690 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:27:30.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:29 smithi169 ceph-mon[34653]: from='client.14690 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:27:30.789 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:30 smithi112 ceph-mon[27869]: pgmap v37: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 56 KiB/s rd, 56 KiB/s wr, 35 op/s 2024-09-16T00:27:30.789 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:30 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:27:30.789 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:30 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:27:31.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:30 smithi169 ceph-mon[34653]: pgmap v37: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 56 KiB/s rd, 56 KiB/s wr, 35 op/s 2024-09-16T00:27:31.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:30 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:27:31.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:30 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:27:32.797 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:32 smithi112 ceph-mon[27869]: pgmap v38: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 31 KiB/s rd, 30 KiB/s wr, 24 op/s 2024-09-16T00:27:32.797 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:32 smithi112 ceph-mon[27869]: Standby manager daemon smithi112.xrkkbc restarted 2024-09-16T00:27:32.797 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:32 smithi112 ceph-mon[27869]: Standby manager daemon smithi112.xrkkbc started 2024-09-16T00:27:32.797 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:32 smithi112 ceph-mon[27869]: from='mgr.? 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi112.xrkkbc/crt"}]: dispatch 2024-09-16T00:27:32.797 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:32 smithi112 ceph-mon[27869]: from='mgr.? 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T00:27:32.797 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:32 smithi112 ceph-mon[27869]: from='mgr.? 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi112.xrkkbc/key"}]: dispatch 2024-09-16T00:27:32.797 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:32 smithi112 ceph-mon[27869]: from='mgr.? 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T00:27:33.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:32 smithi169 ceph-mon[34653]: pgmap v38: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 31 KiB/s rd, 30 KiB/s wr, 24 op/s 2024-09-16T00:27:33.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:32 smithi169 ceph-mon[34653]: Standby manager daemon smithi112.xrkkbc restarted 2024-09-16T00:27:33.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:32 smithi169 ceph-mon[34653]: Standby manager daemon smithi112.xrkkbc started 2024-09-16T00:27:33.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:32 smithi169 ceph-mon[34653]: from='mgr.? 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi112.xrkkbc/crt"}]: dispatch 2024-09-16T00:27:33.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:32 smithi169 ceph-mon[34653]: from='mgr.? 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T00:27:33.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:32 smithi169 ceph-mon[34653]: from='mgr.? 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi112.xrkkbc/key"}]: dispatch 2024-09-16T00:27:33.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:32 smithi169 ceph-mon[34653]: from='mgr.? 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T00:27:33.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:33 smithi112 ceph-mon[27869]: mgrmap e26: smithi169.mtrobk(active, since 71s), standbys: smithi112.xrkkbc 2024-09-16T00:27:33.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:33 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:27:33.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:33 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:27:33.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:33 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:27:33.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:33 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:27:33.881 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:33 smithi169 ceph-mon[34653]: mgrmap e26: smithi169.mtrobk(active, since 71s), standbys: smithi112.xrkkbc 2024-09-16T00:27:33.881 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:33 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:27:33.881 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:33 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:27:33.882 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:33 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:27:33.882 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:33 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:27:35.202 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:34 smithi112 ceph-mon[27869]: pgmap v39: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 31 KiB/s rd, 30 KiB/s wr, 26 op/s 2024-09-16T00:27:35.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:34 smithi169 ceph-mon[34653]: pgmap v39: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 31 KiB/s rd, 30 KiB/s wr, 26 op/s 2024-09-16T00:27:37.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:36 smithi112 ceph-mon[27869]: pgmap v40: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.0 KiB/s rd, 597 B/s wr, 4 op/s 2024-09-16T00:27:37.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:36 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:27:37.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:36 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:27:37.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:36 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:27:37.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:36 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:27:37.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:36 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:27:37.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:36 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:27:37.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:36 smithi169 ceph-mon[34653]: pgmap v40: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.0 KiB/s rd, 597 B/s wr, 4 op/s 2024-09-16T00:27:37.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:36 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:27:37.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:36 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:27:37.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:36 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:27:37.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:36 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:27:37.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:36 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:27:37.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:36 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' 2024-09-16T00:27:38.229 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:37 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:27:38.229 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:37 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "mgr fail", "who": "smithi169.mtrobk"}]: dispatch 2024-09-16T00:27:38.229 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:37 smithi169 ceph-mon[34653]: Activating manager daemon smithi112.xrkkbc 2024-09-16T00:27:38.229 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:37 smithi169 ceph-mon[34653]: osdmap e46: 6 total, 6 up, 6 in 2024-09-16T00:27:38.229 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:37 smithi169 ceph-mon[34653]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd='[{"prefix": "mgr fail", "who": "smithi169.mtrobk"}]': finished 2024-09-16T00:27:38.230 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:37 smithi169 ceph-mon[34653]: mgrmap e27: smithi112.xrkkbc(active, starting, since 0.00616755s) 2024-09-16T00:27:38.230 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:37 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mon metadata", "id": "smithi112"}]: dispatch 2024-09-16T00:27:38.230 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:37 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mon metadata", "id": "smithi169"}]: dispatch 2024-09-16T00:27:38.230 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:37 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi112.hxfwkh"}]: dispatch 2024-09-16T00:27:38.230 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:37 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi112.ithwzj"}]: dispatch 2024-09-16T00:27:38.230 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:37 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi169.gawwgw"}]: dispatch 2024-09-16T00:27:38.230 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:37 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi169.lgeqrj"}]: dispatch 2024-09-16T00:27:38.230 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:37 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mgr metadata", "who": "smithi112.xrkkbc", "id": "smithi112.xrkkbc"}]: dispatch 2024-09-16T00:27:38.230 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:37 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:27:38.230 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:37 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:27:38.230 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:37 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:27:38.230 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:37 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:27:38.230 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:37 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:27:38.230 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:37 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:27:38.231 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:37 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T00:27:38.231 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:37 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T00:27:38.231 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:37 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T00:27:38.231 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:37 smithi169 ceph-mon[34653]: Manager daemon smithi112.xrkkbc is now available 2024-09-16T00:27:38.231 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:37 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:27:38.231 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:37 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:27:38.231 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:37 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi112.xrkkbc/mirror_snapshot_schedule"}]: dispatch 2024-09-16T00:27:38.231 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:37 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi112.xrkkbc/trash_purge_schedule"}]: dispatch 2024-09-16T00:27:38.250 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:37 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:27:38.250 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:37 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "mgr fail", "who": "smithi169.mtrobk"}]: dispatch 2024-09-16T00:27:38.250 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:37 smithi112 ceph-mon[27869]: Activating manager daemon smithi112.xrkkbc 2024-09-16T00:27:38.250 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:37 smithi112 ceph-mon[27869]: osdmap e46: 6 total, 6 up, 6 in 2024-09-16T00:27:38.250 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:37 smithi112 ceph-mon[27869]: from='mgr.24341 172.21.15.169:0/3871209471' entity='mgr.smithi169.mtrobk' cmd='[{"prefix": "mgr fail", "who": "smithi169.mtrobk"}]': finished 2024-09-16T00:27:38.250 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:37 smithi112 ceph-mon[27869]: mgrmap e27: smithi112.xrkkbc(active, starting, since 0.00616755s) 2024-09-16T00:27:38.251 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:37 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mon metadata", "id": "smithi112"}]: dispatch 2024-09-16T00:27:38.251 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:37 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mon metadata", "id": "smithi169"}]: dispatch 2024-09-16T00:27:38.251 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:37 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi112.hxfwkh"}]: dispatch 2024-09-16T00:27:38.251 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:37 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi112.ithwzj"}]: dispatch 2024-09-16T00:27:38.251 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:37 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi169.gawwgw"}]: dispatch 2024-09-16T00:27:38.251 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:37 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi169.lgeqrj"}]: dispatch 2024-09-16T00:27:38.251 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:37 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mgr metadata", "who": "smithi112.xrkkbc", "id": "smithi112.xrkkbc"}]: dispatch 2024-09-16T00:27:38.251 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:37 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:27:38.251 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:37 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:27:38.251 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:37 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:27:38.251 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:37 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:27:38.251 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:37 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:27:38.251 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:37 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:27:38.251 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:37 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T00:27:38.252 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:37 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T00:27:38.252 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:37 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T00:27:38.252 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:37 smithi112 ceph-mon[27869]: Manager daemon smithi112.xrkkbc is now available 2024-09-16T00:27:38.252 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:37 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:27:38.252 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:37 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:27:38.252 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:37 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi112.xrkkbc/mirror_snapshot_schedule"}]: dispatch 2024-09-16T00:27:38.252 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:37 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi112.xrkkbc/trash_purge_schedule"}]: dispatch 2024-09-16T00:27:39.068 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:38 smithi112 ceph-mon[27869]: mgrmap e28: smithi112.xrkkbc(active, since 1.00963s) 2024-09-16T00:27:39.068 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:38 smithi112 ceph-mon[27869]: pgmap v3: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail 2024-09-16T00:27:39.263 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:38 smithi169 ceph-mon[34653]: mgrmap e28: smithi112.xrkkbc(active, since 1.00963s) 2024-09-16T00:27:39.263 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:38 smithi169 ceph-mon[34653]: pgmap v3: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail 2024-09-16T00:27:40.006 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:39 smithi169 ceph-mon[34653]: [16/Sep/2024:00:27:38] ENGINE Bus STARTING 2024-09-16T00:27:40.006 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:39 smithi169 ceph-mon[34653]: [16/Sep/2024:00:27:38] ENGINE Serving on http://172.21.15.112:8765 2024-09-16T00:27:40.006 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:39 smithi169 ceph-mon[34653]: [16/Sep/2024:00:27:38] ENGINE Serving on https://172.21.15.112:7150 2024-09-16T00:27:40.006 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:39 smithi169 ceph-mon[34653]: [16/Sep/2024:00:27:38] ENGINE Bus STARTED 2024-09-16T00:27:40.006 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:39 smithi169 ceph-mon[34653]: [16/Sep/2024:00:27:38] ENGINE Error in HTTPServer.serve 2024-09-16T00:27:40.006 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Traceback (most recent call last): 2024-09-16T00:27:40.006 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T00:27:40.006 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: self._connections.run(self.expiration_interval) 2024-09-16T00:27:40.006 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T00:27:40.006 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: self._run(expiration_interval) 2024-09-16T00:27:40.007 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T00:27:40.007 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T00:27:40.007 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T00:27:40.007 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T00:27:40.007 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T00:27:40.007 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: s = self.context.wrap_socket( 2024-09-16T00:27:40.007 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T00:27:40.007 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: return self.sslsocket_class._create( 2024-09-16T00:27:40.007 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T00:27:40.007 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: self.do_handshake() 2024-09-16T00:27:40.007 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T00:27:40.007 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: self._sslobj.do_handshake() 2024-09-16T00:27:40.007 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T00:27:40.008 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:39 smithi169 ceph-mon[34653]: pgmap v4: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail 2024-09-16T00:27:40.008 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:39 smithi169 ceph-mon[34653]: mgrmap e29: smithi112.xrkkbc(active, since 2s) 2024-09-16T00:27:40.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:39 smithi112 ceph-mon[27869]: [16/Sep/2024:00:27:38] ENGINE Bus STARTING 2024-09-16T00:27:40.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:39 smithi112 ceph-mon[27869]: [16/Sep/2024:00:27:38] ENGINE Serving on http://172.21.15.112:8765 2024-09-16T00:27:40.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:39 smithi112 ceph-mon[27869]: [16/Sep/2024:00:27:38] ENGINE Serving on https://172.21.15.112:7150 2024-09-16T00:27:40.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:39 smithi112 ceph-mon[27869]: [16/Sep/2024:00:27:38] ENGINE Bus STARTED 2024-09-16T00:27:40.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:39 smithi112 ceph-mon[27869]: [16/Sep/2024:00:27:38] ENGINE Error in HTTPServer.serve 2024-09-16T00:27:40.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: Traceback (most recent call last): 2024-09-16T00:27:40.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T00:27:40.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: self._connections.run(self.expiration_interval) 2024-09-16T00:27:40.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T00:27:40.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: self._run(expiration_interval) 2024-09-16T00:27:40.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T00:27:40.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T00:27:40.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T00:27:40.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T00:27:40.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T00:27:40.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: s = self.context.wrap_socket( 2024-09-16T00:27:40.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T00:27:40.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: return self.sslsocket_class._create( 2024-09-16T00:27:40.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T00:27:40.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: self.do_handshake() 2024-09-16T00:27:40.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T00:27:40.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: self._sslobj.do_handshake() 2024-09-16T00:27:40.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T00:27:40.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:39 smithi112 ceph-mon[27869]: pgmap v4: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail 2024-09-16T00:27:40.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:39 smithi112 ceph-mon[27869]: mgrmap e29: smithi112.xrkkbc(active, since 2s) 2024-09-16T00:27:41.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:41 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:27:41.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:41 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:27:41.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:41 smithi169 ceph-mon[34653]: pgmap v5: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail 2024-09-16T00:27:41.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:41 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:27:41.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:41 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:27:41.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:41 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:27:41.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:41 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:27:41.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:41 smithi112 ceph-mon[27869]: pgmap v5: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail 2024-09-16T00:27:41.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:41 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:27:41.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:41 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:27:42.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:42 smithi169 ceph-mon[34653]: mgrmap e30: smithi112.xrkkbc(active, since 4s) 2024-09-16T00:27:42.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:42 smithi169 ceph-mon[34653]: Standby manager daemon smithi169.mtrobk started 2024-09-16T00:27:42.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:42 smithi169 ceph-mon[34653]: from='mgr.? 172.21.15.169:0/1092228922' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi169.mtrobk/crt"}]: dispatch 2024-09-16T00:27:42.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:42 smithi169 ceph-mon[34653]: from='mgr.? 172.21.15.169:0/1092228922' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T00:27:42.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:42 smithi169 ceph-mon[34653]: from='mgr.? 172.21.15.169:0/1092228922' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi169.mtrobk/key"}]: dispatch 2024-09-16T00:27:42.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:42 smithi169 ceph-mon[34653]: from='mgr.? 172.21.15.169:0/1092228922' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T00:27:42.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:42 smithi112 ceph-mon[27869]: mgrmap e30: smithi112.xrkkbc(active, since 4s) 2024-09-16T00:27:42.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:42 smithi112 ceph-mon[27869]: Standby manager daemon smithi169.mtrobk started 2024-09-16T00:27:42.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:42 smithi112 ceph-mon[27869]: from='mgr.? 172.21.15.169:0/1092228922' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi169.mtrobk/crt"}]: dispatch 2024-09-16T00:27:42.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:42 smithi112 ceph-mon[27869]: from='mgr.? 172.21.15.169:0/1092228922' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T00:27:42.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:42 smithi112 ceph-mon[27869]: from='mgr.? 172.21.15.169:0/1092228922' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi169.mtrobk/key"}]: dispatch 2024-09-16T00:27:42.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:42 smithi112 ceph-mon[27869]: from='mgr.? 172.21.15.169:0/1092228922' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T00:27:43.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:43 smithi169 ceph-mon[34653]: mgrmap e31: smithi112.xrkkbc(active, since 5s), standbys: smithi169.mtrobk 2024-09-16T00:27:43.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:43 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mgr metadata", "who": "smithi169.mtrobk", "id": "smithi169.mtrobk"}]: dispatch 2024-09-16T00:27:43.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:43 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:27:43.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:43 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:27:43.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:43 smithi169 ceph-mon[34653]: pgmap v6: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 205 B/s rd, 1 op/s 2024-09-16T00:27:43.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:43 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:27:43.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:43 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:27:43.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:43 smithi112 ceph-mon[27869]: mgrmap e31: smithi112.xrkkbc(active, since 5s), standbys: smithi169.mtrobk 2024-09-16T00:27:43.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:43 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mgr metadata", "who": "smithi169.mtrobk", "id": "smithi169.mtrobk"}]: dispatch 2024-09-16T00:27:43.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:43 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:27:43.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:43 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:27:43.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:43 smithi112 ceph-mon[27869]: pgmap v6: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 205 B/s rd, 1 op/s 2024-09-16T00:27:43.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:43 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:27:43.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:43 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:27:46.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:46 smithi169 ceph-mon[34653]: pgmap v7: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 31 KiB/s rd, 0 B/s wr, 19 op/s 2024-09-16T00:27:46.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:46 smithi112 ceph-mon[27869]: pgmap v7: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 31 KiB/s rd, 0 B/s wr, 19 op/s 2024-09-16T00:27:47.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:47 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:27:47.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:47 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:27:47.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:47 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:27:47.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:47 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:27:47.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:47 smithi169 ceph-mon[34653]: Updating smithi112:/etc/ceph/ceph.conf 2024-09-16T00:27:47.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:47 smithi169 ceph-mon[34653]: Updating smithi169:/etc/ceph/ceph.conf 2024-09-16T00:27:47.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:47 smithi169 ceph-mon[34653]: Updating smithi112:/var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:27:47.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:47 smithi169 ceph-mon[34653]: Updating smithi169:/var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:27:47.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:47 smithi169 ceph-mon[34653]: Updating smithi112:/etc/ceph/ceph.client.admin.keyring 2024-09-16T00:27:47.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:47 smithi169 ceph-mon[34653]: Updating smithi169:/etc/ceph/ceph.client.admin.keyring 2024-09-16T00:27:47.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:47 smithi169 ceph-mon[34653]: pgmap v8: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 24 KiB/s rd, 0 B/s wr, 15 op/s 2024-09-16T00:27:47.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:47 smithi169 ceph-mon[34653]: Updating smithi112:/var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T00:27:47.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:47 smithi169 ceph-mon[34653]: Updating smithi169:/var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T00:27:47.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:47 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:27:47.778 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:47 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:27:47.778 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:47 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:27:47.778 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:47 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:27:47.778 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:47 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:27:47.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:47 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:27:47.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:47 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:27:47.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:47 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:27:47.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:47 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:27:47.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:47 smithi112 ceph-mon[27869]: Updating smithi112:/etc/ceph/ceph.conf 2024-09-16T00:27:47.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:47 smithi112 ceph-mon[27869]: Updating smithi169:/etc/ceph/ceph.conf 2024-09-16T00:27:47.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:47 smithi112 ceph-mon[27869]: Updating smithi112:/var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:27:47.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:47 smithi112 ceph-mon[27869]: Updating smithi169:/var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:27:47.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:47 smithi112 ceph-mon[27869]: Updating smithi112:/etc/ceph/ceph.client.admin.keyring 2024-09-16T00:27:47.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:47 smithi112 ceph-mon[27869]: Updating smithi169:/etc/ceph/ceph.client.admin.keyring 2024-09-16T00:27:47.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:47 smithi112 ceph-mon[27869]: pgmap v8: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 24 KiB/s rd, 0 B/s wr, 15 op/s 2024-09-16T00:27:47.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:47 smithi112 ceph-mon[27869]: Updating smithi112:/var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T00:27:47.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:47 smithi112 ceph-mon[27869]: Updating smithi169:/var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T00:27:47.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:47 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:27:47.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:47 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:27:47.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:47 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:27:47.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:47 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:27:47.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:47 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:27:48.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:48 smithi112 ceph-mon[27869]: Reconfiguring prometheus.smithi112 (dependencies changed)... 2024-09-16T00:27:48.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:48 smithi112 ceph-mon[27869]: Reconfiguring daemon prometheus.smithi112 on smithi112 2024-09-16T00:27:48.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:48 smithi169 ceph-mon[34653]: Reconfiguring prometheus.smithi112 (dependencies changed)... 2024-09-16T00:27:48.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:48 smithi169 ceph-mon[34653]: Reconfiguring daemon prometheus.smithi112 on smithi112 2024-09-16T00:27:49.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:49 smithi169 ceph-mon[34653]: pgmap v9: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 21 KiB/s rd, 0 B/s wr, 14 op/s 2024-09-16T00:27:49.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:49 smithi112 ceph-mon[27869]: pgmap v9: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 21 KiB/s rd, 0 B/s wr, 14 op/s 2024-09-16T00:27:52.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:52 smithi112 ceph-mon[27869]: pgmap v10: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-16T00:27:52.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:52 smithi169 ceph-mon[34653]: pgmap v10: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-16T00:27:53.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:53 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:27:53.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:53 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:27:53.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:53 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:27:53.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:53 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-16T00:27:53.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:53 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:27:53.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:53 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:27:53.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:53 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:27:53.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:53 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:27:53.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:53 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-16T00:27:53.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:53 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:27:54.203 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:54 smithi112 ceph-mon[27869]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-16T00:27:54.203 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:54 smithi112 ceph-mon[27869]: pgmap v11: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 13 op/s 2024-09-16T00:27:54.204 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:54 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi169.mtrobk", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T00:27:54.204 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:54 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T00:27:54.204 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:54 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:27:54.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:54 smithi169 ceph-mon[34653]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-16T00:27:54.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:54 smithi169 ceph-mon[34653]: pgmap v11: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 13 op/s 2024-09-16T00:27:54.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:54 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi169.mtrobk", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T00:27:54.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:54 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T00:27:54.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:54 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:27:55.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:55 smithi112 ceph-mon[27869]: Upgrade: Updating mgr.smithi169.mtrobk 2024-09-16T00:27:55.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:55 smithi112 ceph-mon[27869]: Deploying daemon mgr.smithi169.mtrobk on smithi169 2024-09-16T00:27:55.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:55 smithi169 ceph-mon[34653]: Upgrade: Updating mgr.smithi169.mtrobk 2024-09-16T00:27:55.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:55 smithi169 ceph-mon[34653]: Deploying daemon mgr.smithi169.mtrobk on smithi169 2024-09-16T00:27:56.096 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:56 smithi169 ceph-mon[34653]: pgmap v12: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 20 KiB/s rd, 0 B/s wr, 14 op/s 2024-09-16T00:27:56.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:56 smithi112 ceph-mon[27869]: pgmap v12: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 20 KiB/s rd, 0 B/s wr, 14 op/s 2024-09-16T00:27:58.221 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:58 smithi169 ceph-mon[34653]: pgmap v13: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:27:58.221 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:58 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:27:58.221 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:58 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:27:58.221 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:27:58 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:27:58.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:58 smithi112 ceph-mon[27869]: pgmap v13: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:27:58.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:58 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:27:58.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:58 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:27:58.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:27:58 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:27:58.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.689+0000 7efde13be640 1 -- 172.21.15.112:0/487185096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efddc0fc720 msgr2=0x7efddc0fcb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:58.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.689+0000 7efde13be640 1 --2- 172.21.15.112:0/487185096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efddc0fc720 0x7efddc0fcb90 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7efdc40048d0 tx=0x7efdc4009da0 comp rx=0 tx=0).stop 2024-09-16T00:27:58.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.689+0000 7efde13be640 1 -- 172.21.15.112:0/487185096 shutdown_connections 2024-09-16T00:27:58.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.689+0000 7efde13be640 1 --2- 172.21.15.112:0/487185096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efddc0fc720 0x7efddc0fcb90 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:58.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.689+0000 7efde13be640 1 --2- 172.21.15.112:0/487185096 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efddc0fb450 0x7efddc0fb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:58.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.689+0000 7efde13be640 1 -- 172.21.15.112:0/487185096 >> 172.21.15.112:0/487185096 conn(0x7efddc0f6c10 msgr2=0x7efddc0f9040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:58.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.689+0000 7efde13be640 1 -- 172.21.15.112:0/487185096 shutdown_connections 2024-09-16T00:27:58.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.689+0000 7efde13be640 1 -- 172.21.15.112:0/487185096 wait complete. 2024-09-16T00:27:58.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.689+0000 7efde13be640 1 Processor -- start 2024-09-16T00:27:58.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.690+0000 7efde13be640 1 -- start start 2024-09-16T00:27:58.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.690+0000 7efde13be640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efddc0fb450 0x7efddc10f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:58.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.690+0000 7efde13be640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efddc10f9e0 0x7efddc110850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:58.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.690+0000 7efde13be640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efddc10fea0 con 0x7efddc10f9e0 2024-09-16T00:27:58.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.690+0000 7efde13be640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efddc10ffe0 con 0x7efddc0fb450 2024-09-16T00:27:58.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.690+0000 7efdd27fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efddc10f9e0 0x7efddc110850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:58.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.690+0000 7efddaffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efddc0fb450 0x7efddc10f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:58.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.690+0000 7efddaffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efddc0fb450 0x7efddc10f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.112:60500/0 (socket says 172.21.15.112:60500) 2024-09-16T00:27:58.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.690+0000 7efdd27fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efddc10f9e0 0x7efddc110850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39176/0 (socket says 172.21.15.112:39176) 2024-09-16T00:27:58.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.690+0000 7efddaffd640 1 -- 172.21.15.112:0/1976731965 learned_addr learned my addr 172.21.15.112:0/1976731965 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:27:58.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.691+0000 7efddaffd640 1 -- 172.21.15.112:0/1976731965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efddc10f9e0 msgr2=0x7efddc110850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:58.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.691+0000 7efddaffd640 1 --2- 172.21.15.112:0/1976731965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efddc10f9e0 0x7efddc110850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:58.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.691+0000 7efddaffd640 1 -- 172.21.15.112:0/1976731965 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdc4005040 con 0x7efddc0fb450 2024-09-16T00:27:58.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.691+0000 7efddaffd640 1 --2- 172.21.15.112:0/1976731965 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efddc0fb450 0x7efddc10f4a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7efdcc002a80 tx=0x7efdcc004600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:58.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.691+0000 7efdd3fff640 1 -- 172.21.15.112:0/1976731965 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efdcc007c30 con 0x7efddc0fb450 2024-09-16T00:27:58.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.691+0000 7efde13be640 1 -- 172.21.15.112:0/1976731965 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efddc110e50 con 0x7efddc0fb450 2024-09-16T00:27:58.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.691+0000 7efdd3fff640 1 -- 172.21.15.112:0/1976731965 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7efdcc00f040 con 0x7efddc0fb450 2024-09-16T00:27:58.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.691+0000 7efdd3fff640 1 -- 172.21.15.112:0/1976731965 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efdcc00dab0 con 0x7efddc0fb450 2024-09-16T00:27:58.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.691+0000 7efde13be640 1 -- 172.21.15.112:0/1976731965 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efddc111370 con 0x7efddc0fb450 2024-09-16T00:27:58.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.693+0000 7efde13be640 1 -- 172.21.15.112:0/1976731965 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efddc0fa8b0 con 0x7efddc0fb450 2024-09-16T00:27:58.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.693+0000 7efdd3fff640 1 -- 172.21.15.112:0/1976731965 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 31) v1 ==== 92377+0+0 (secure 0 0 0) 0x7efdcc00dc10 con 0x7efddc0fb450 2024-09-16T00:27:58.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.693+0000 7efdd3fff640 1 --2- 172.21.15.112:0/1976731965 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7efdc806df60 0x7efdc8070410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:58.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.693+0000 7efdd3fff640 1 -- 172.21.15.112:0/1976731965 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6152+0+0 (secure 0 0 0) 0x7efdcc08d360 con 0x7efddc0fb450 2024-09-16T00:27:58.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.693+0000 7efdd27fc640 1 --2- 172.21.15.112:0/1976731965 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7efdc806df60 0x7efdc8070410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:58.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.694+0000 7efdd27fc640 1 --2- 172.21.15.112:0/1976731965 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7efdc806df60 0x7efdc8070410 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7efdc40048a0 tx=0x7efdc4035040 comp rx=0 tx=0).ready entity=mgr.14678 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:58.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.698+0000 7efdd3fff640 1 -- 172.21.15.112:0/1976731965 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185942 (secure 0 0 0) 0x7efdcc056b30 con 0x7efddc0fb450 2024-09-16T00:27:58.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.890+0000 7efde13be640 1 -- 172.21.15.112:0/1976731965 --> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7efddc103e50 con 0x7efdc806df60 2024-09-16T00:27:58.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.891+0000 7efdd3fff640 1 -- 172.21.15.112:0/1976731965 <== mgr.14678 v2:172.21.15.112:6800/1965658616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7efddc103e50 con 0x7efdc806df60 2024-09-16T00:27:58.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.893+0000 7efde13be640 1 -- 172.21.15.112:0/1976731965 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7efdc806df60 msgr2=0x7efdc8070410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:58.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.893+0000 7efde13be640 1 --2- 172.21.15.112:0/1976731965 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7efdc806df60 0x7efdc8070410 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7efdc40048a0 tx=0x7efdc4035040 comp rx=0 tx=0).stop 2024-09-16T00:27:58.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.893+0000 7efde13be640 1 -- 172.21.15.112:0/1976731965 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efddc0fb450 msgr2=0x7efddc10f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:58.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.893+0000 7efde13be640 1 --2- 172.21.15.112:0/1976731965 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efddc0fb450 0x7efddc10f4a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7efdcc002a80 tx=0x7efdcc004600 comp rx=0 tx=0).stop 2024-09-16T00:27:58.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.893+0000 7efde13be640 1 -- 172.21.15.112:0/1976731965 shutdown_connections 2024-09-16T00:27:58.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.893+0000 7efde13be640 1 --2- 172.21.15.112:0/1976731965 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7efdc806df60 0x7efdc8070410 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:58.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.893+0000 7efde13be640 1 --2- 172.21.15.112:0/1976731965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efddc10f9e0 0x7efddc110850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:58.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.893+0000 7efde13be640 1 --2- 172.21.15.112:0/1976731965 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efddc0fb450 0x7efddc10f4a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:58.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.894+0000 7efde13be640 1 -- 172.21.15.112:0/1976731965 >> 172.21.15.112:0/1976731965 conn(0x7efddc0f6c10 msgr2=0x7efddc066340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:58.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.894+0000 7efde13be640 1 -- 172.21.15.112:0/1976731965 shutdown_connections 2024-09-16T00:27:58.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.894+0000 7efde13be640 1 -- 172.21.15.112:0/1976731965 wait complete. 2024-09-16T00:27:58.902 INFO:teuthology.orchestra.run.smithi112.stdout:true 2024-09-16T00:27:58.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.974+0000 7f596065d640 1 -- 172.21.15.112:0/775550715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59580fc520 msgr2=0x7f59580fc930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:58.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.974+0000 7f596065d640 1 --2- 172.21.15.112:0/775550715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59580fc520 0x7f59580fc930 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f59540048d0 tx=0x7f5954009da0 comp rx=0 tx=0).stop 2024-09-16T00:27:58.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.974+0000 7f596065d640 1 -- 172.21.15.112:0/775550715 shutdown_connections 2024-09-16T00:27:58.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.974+0000 7f596065d640 1 --2- 172.21.15.112:0/775550715 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59580fd7f0 0x7f59580fdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:58.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.974+0000 7f596065d640 1 --2- 172.21.15.112:0/775550715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59580fc520 0x7f59580fc930 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:58.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.974+0000 7f596065d640 1 -- 172.21.15.112:0/775550715 >> 172.21.15.112:0/775550715 conn(0x7f59580f7ce0 msgr2=0x7f59580fa110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:58.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.974+0000 7f596065d640 1 -- 172.21.15.112:0/775550715 shutdown_connections 2024-09-16T00:27:58.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.974+0000 7f596065d640 1 -- 172.21.15.112:0/775550715 wait complete. 2024-09-16T00:27:58.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.975+0000 7f596065d640 1 Processor -- start 2024-09-16T00:27:58.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.975+0000 7f596065d640 1 -- start start 2024-09-16T00:27:58.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.975+0000 7f596065d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59580fc520 0x7f595810d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:58.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.975+0000 7f596065d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59580fd7f0 0x7f595810d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:58.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.975+0000 7f596065d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595810dc10 con 0x7f59580fd7f0 2024-09-16T00:27:58.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.975+0000 7f596065d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595810e430 con 0x7f59580fc520 2024-09-16T00:27:58.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.975+0000 7f595dbf8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59580fd7f0 0x7f595810d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:58.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.975+0000 7f595e3f9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59580fc520 0x7f595810d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:58.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.975+0000 7f595dbf8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59580fd7f0 0x7f595810d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39184/0 (socket says 172.21.15.112:39184) 2024-09-16T00:27:58.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.975+0000 7f595dbf8640 1 -- 172.21.15.112:0/3333074061 learned_addr learned my addr 172.21.15.112:0/3333074061 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:27:58.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.976+0000 7f595dbf8640 1 -- 172.21.15.112:0/3333074061 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59580fc520 msgr2=0x7f595810d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:58.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.976+0000 7f595dbf8640 1 --2- 172.21.15.112:0/3333074061 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59580fc520 0x7f595810d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:58.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.976+0000 7f595dbf8640 1 -- 172.21.15.112:0/3333074061 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5954005040 con 0x7f59580fd7f0 2024-09-16T00:27:58.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.976+0000 7f595e3f9640 1 --2- 172.21.15.112:0/3333074061 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59580fc520 0x7f595810d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:27:58.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.976+0000 7f595dbf8640 1 --2- 172.21.15.112:0/3333074061 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59580fd7f0 0x7f595810d5c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f594800eb70 tx=0x7f594800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:58.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.976+0000 7f594effd640 1 -- 172.21.15.112:0/3333074061 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5948007c30 con 0x7f59580fd7f0 2024-09-16T00:27:58.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.976+0000 7f594effd640 1 -- 172.21.15.112:0/3333074061 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f594800f040 con 0x7f59580fd7f0 2024-09-16T00:27:58.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.976+0000 7f594effd640 1 -- 172.21.15.112:0/3333074061 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f59480094e0 con 0x7f59580fd7f0 2024-09-16T00:27:58.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.976+0000 7f596065d640 1 -- 172.21.15.112:0/3333074061 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f595810e660 con 0x7f59580fd7f0 2024-09-16T00:27:58.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.976+0000 7f596065d640 1 -- 172.21.15.112:0/3333074061 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f595810ec00 con 0x7f59580fd7f0 2024-09-16T00:27:58.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.977+0000 7f596065d640 1 -- 172.21.15.112:0/3333074061 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59580fb980 con 0x7f59580fd7f0 2024-09-16T00:27:58.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.978+0000 7f594effd640 1 -- 172.21.15.112:0/3333074061 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 31) v1 ==== 92377+0+0 (secure 0 0 0) 0x7f59480096a0 con 0x7f59580fd7f0 2024-09-16T00:27:58.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.978+0000 7f594effd640 1 --2- 172.21.15.112:0/3333074061 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f594406df60 0x7f5944070410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:58.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.978+0000 7f594effd640 1 -- 172.21.15.112:0/3333074061 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6152+0+0 (secure 0 0 0) 0x7f594808d5e0 con 0x7f59580fd7f0 2024-09-16T00:27:58.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.978+0000 7f595e3f9640 1 --2- 172.21.15.112:0/3333074061 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f594406df60 0x7f5944070410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:58.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.979+0000 7f595e3f9640 1 --2- 172.21.15.112:0/3333074061 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f594406df60 0x7f5944070410 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5954000c00 tx=0x7f5954035040 comp rx=0 tx=0).ready entity=mgr.14678 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:58.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:58.983+0000 7f594effd640 1 -- 172.21.15.112:0/3333074061 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185942 (secure 0 0 0) 0x7f5948056e30 con 0x7f59580fd7f0 2024-09-16T00:27:59.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.172+0000 7f596065d640 1 -- 172.21.15.112:0/3333074061 --> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f595810ef80 con 0x7f594406df60 2024-09-16T00:27:59.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.174+0000 7f594effd640 1 -- 172.21.15.112:0/3333074061 <== mgr.14678 v2:172.21.15.112:6800/1965658616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7f595810ef80 con 0x7f594406df60 2024-09-16T00:27:59.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.176+0000 7f596065d640 1 -- 172.21.15.112:0/3333074061 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f594406df60 msgr2=0x7f5944070410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:59.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.176+0000 7f596065d640 1 --2- 172.21.15.112:0/3333074061 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f594406df60 0x7f5944070410 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5954000c00 tx=0x7f5954035040 comp rx=0 tx=0).stop 2024-09-16T00:27:59.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.176+0000 7f596065d640 1 -- 172.21.15.112:0/3333074061 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59580fd7f0 msgr2=0x7f595810d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:59.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.176+0000 7f596065d640 1 --2- 172.21.15.112:0/3333074061 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59580fd7f0 0x7f595810d5c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f594800eb70 tx=0x7f594800b040 comp rx=0 tx=0).stop 2024-09-16T00:27:59.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.176+0000 7f596065d640 1 -- 172.21.15.112:0/3333074061 shutdown_connections 2024-09-16T00:27:59.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.176+0000 7f596065d640 1 --2- 172.21.15.112:0/3333074061 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f594406df60 0x7f5944070410 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:59.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.176+0000 7f596065d640 1 --2- 172.21.15.112:0/3333074061 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59580fd7f0 0x7f595810d5c0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:59.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.176+0000 7f596065d640 1 --2- 172.21.15.112:0/3333074061 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59580fc520 0x7f595810d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:59.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.176+0000 7f596065d640 1 -- 172.21.15.112:0/3333074061 >> 172.21.15.112:0/3333074061 conn(0x7f59580f7ce0 msgr2=0x7f59580f9500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:59.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.176+0000 7f596065d640 1 -- 172.21.15.112:0/3333074061 shutdown_connections 2024-09-16T00:27:59.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.176+0000 7f596065d640 1 -- 172.21.15.112:0/3333074061 wait complete. 2024-09-16T00:27:59.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.253+0000 7fb135826640 1 -- 172.21.15.112:0/4235283754 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1300fc550 msgr2=0x7fb1300fc960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:59.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.253+0000 7fb135826640 1 --2- 172.21.15.112:0/4235283754 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1300fc550 0x7fb1300fc960 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb1200048d0 tx=0x7fb120009da0 comp rx=0 tx=0).stop 2024-09-16T00:27:59.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.253+0000 7fb135826640 1 -- 172.21.15.112:0/4235283754 shutdown_connections 2024-09-16T00:27:59.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.253+0000 7fb135826640 1 --2- 172.21.15.112:0/4235283754 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1300fd820 0x7fb1300fdc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:59.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.253+0000 7fb135826640 1 --2- 172.21.15.112:0/4235283754 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1300fc550 0x7fb1300fc960 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:59.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.253+0000 7fb135826640 1 -- 172.21.15.112:0/4235283754 >> 172.21.15.112:0/4235283754 conn(0x7fb1300f7d30 msgr2=0x7fb1300fa140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:59.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.253+0000 7fb135826640 1 -- 172.21.15.112:0/4235283754 shutdown_connections 2024-09-16T00:27:59.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.253+0000 7fb135826640 1 -- 172.21.15.112:0/4235283754 wait complete. 2024-09-16T00:27:59.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.254+0000 7fb135826640 1 Processor -- start 2024-09-16T00:27:59.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.254+0000 7fb135826640 1 -- start start 2024-09-16T00:27:59.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.254+0000 7fb135826640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1300fc550 0x7fb13010d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:59.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.254+0000 7fb135826640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1300fd820 0x7fb13010d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:59.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.254+0000 7fb135826640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb13010dc10 con 0x7fb1300fc550 2024-09-16T00:27:59.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.254+0000 7fb135826640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb13010e430 con 0x7fb1300fd820 2024-09-16T00:27:59.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.254+0000 7fb12effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1300fc550 0x7fb13010d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:59.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.254+0000 7fb12effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1300fc550 0x7fb13010d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39206/0 (socket says 172.21.15.112:39206) 2024-09-16T00:27:59.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.254+0000 7fb12effd640 1 -- 172.21.15.112:0/562090705 learned_addr learned my addr 172.21.15.112:0/562090705 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:27:59.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.254+0000 7fb12e7fc640 1 --2- 172.21.15.112:0/562090705 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1300fd820 0x7fb13010d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:59.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.254+0000 7fb12effd640 1 -- 172.21.15.112:0/562090705 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1300fd820 msgr2=0x7fb13010d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:59.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.254+0000 7fb12effd640 1 --2- 172.21.15.112:0/562090705 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1300fd820 0x7fb13010d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:59.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.254+0000 7fb12effd640 1 -- 172.21.15.112:0/562090705 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb120005040 con 0x7fb1300fc550 2024-09-16T00:27:59.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.255+0000 7fb12effd640 1 --2- 172.21.15.112:0/562090705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1300fc550 0x7fb13010d080 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb120003080 tx=0x7fb120009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:59.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.255+0000 7fb1177fe640 1 -- 172.21.15.112:0/562090705 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb120038070 con 0x7fb1300fc550 2024-09-16T00:27:59.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.255+0000 7fb1177fe640 1 -- 172.21.15.112:0/562090705 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fb12002abd0 con 0x7fb1300fc550 2024-09-16T00:27:59.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.255+0000 7fb135826640 1 -- 172.21.15.112:0/562090705 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb13010e660 con 0x7fb1300fc550 2024-09-16T00:27:59.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.255+0000 7fb1177fe640 1 -- 172.21.15.112:0/562090705 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb12003e670 con 0x7fb1300fc550 2024-09-16T00:27:59.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.255+0000 7fb135826640 1 -- 172.21.15.112:0/562090705 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb13010ec00 con 0x7fb1300fc550 2024-09-16T00:27:59.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.256+0000 7fb135826640 1 -- 172.21.15.112:0/562090705 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1300fb9b0 con 0x7fb1300fc550 2024-09-16T00:27:59.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.256+0000 7fb1177fe640 1 -- 172.21.15.112:0/562090705 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 31) v1 ==== 92377+0+0 (secure 0 0 0) 0x7fb120030020 con 0x7fb1300fc550 2024-09-16T00:27:59.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.257+0000 7fb1177fe640 1 --2- 172.21.15.112:0/562090705 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7fb11806df60 0x7fb118070410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:59.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.257+0000 7fb1177fe640 1 -- 172.21.15.112:0/562090705 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6152+0+0 (secure 0 0 0) 0x7fb1200adc00 con 0x7fb1300fc550 2024-09-16T00:27:59.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.257+0000 7fb12e7fc640 1 --2- 172.21.15.112:0/562090705 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7fb11806df60 0x7fb118070410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:59.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.257+0000 7fb12e7fc640 1 --2- 172.21.15.112:0/562090705 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7fb11806df60 0x7fb118070410 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb13010ce30 tx=0x7fb124004330 comp rx=0 tx=0).ready entity=mgr.14678 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:59.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.262+0000 7fb1177fe640 1 -- 172.21.15.112:0/562090705 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185942 (secure 0 0 0) 0x7fb1200b3020 con 0x7fb1300fc550 2024-09-16T00:27:59.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.436+0000 7fb135826640 1 -- 172.21.15.112:0/562090705 --> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb13010ef30 con 0x7fb11806df60 2024-09-16T00:27:59.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.444+0000 7fb1177fe640 1 -- 172.21.15.112:0/562090705 <== mgr.14678 v2:172.21.15.112:6800/1965658616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fb13010ef30 con 0x7fb11806df60 2024-09-16T00:27:59.445 INFO:teuthology.orchestra.run.smithi112.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T00:27:59.445 INFO:teuthology.orchestra.run.smithi112.stdout:alertmanager.smithi112 smithi112 *:9093,9094 running (57s) 16s ago 9m 16.6M - 0.25.0 c8568f914cd2 d85ea0562022 2024-09-16T00:27:59.445 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi112 smithi112 running (10m) 16s ago 10m 6891k - 17.2.7-1633-g6d025bc2 33ed8dba3996 c94220ed2776 2024-09-16T00:27:59.445 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi169 smithi169 running (9m) 18s ago 9m 6899k - 17.2.7-1633-g6d025bc2 33ed8dba3996 d0d3726280a2 2024-09-16T00:27:59.445 INFO:teuthology.orchestra.run.smithi112.stdout:grafana.smithi112 smithi112 *:3000 running (49s) 16s ago 9m 80.1M - 9.4.7 954c08fa6188 1246495f739b 2024-09-16T00:27:59.445 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.hxfwkh smithi112 running (4m) 16s ago 4m 135M - 17.2.7-1633-g6d025bc2 33ed8dba3996 cf92d9461fc6 2024-09-16T00:27:59.445 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.ithwzj smithi112 running (4m) 16s ago 4m 65.5M - 17.2.7-1633-g6d025bc2 33ed8dba3996 02786c8a83c7 2024-09-16T00:27:59.446 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.gawwgw smithi169 running (4m) 18s ago 4m 114M - 17.2.7-1633-g6d025bc2 33ed8dba3996 dcf3a4dfa879 2024-09-16T00:27:59.446 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.lgeqrj smithi169 running (4m) 18s ago 4m 60.2M - 17.2.7-1633-g6d025bc2 33ed8dba3996 445c14e55831 2024-09-16T00:27:59.446 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi112.xrkkbc smithi112 *:8443,9283,8765 running (32s) 16s ago 11m 472M - 19.1.1-326-g8921c8ef f1455bd0219c a981f7fd2810 2024-09-16T00:27:59.446 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi169.mtrobk smithi169 *:8443,9283,8765 starting - - - - 2024-09-16T00:27:59.446 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi112 smithi112 running (11m) 16s ago 11m 54.9M 2048M 17.2.7-1633-g6d025bc2 33ed8dba3996 ddaaced54bf8 2024-09-16T00:27:59.446 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi169 smithi169 running (8m) 18s ago 8m 41.5M 2048M 17.2.7-1633-g6d025bc2 33ed8dba3996 6c2289bc3e11 2024-09-16T00:27:59.446 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi112 smithi112 *:9100 running (77s) 16s ago 10m 8484k - 1.5.0 0da6a335fe13 6404d1d601e2 2024-09-16T00:27:59.446 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi169 smithi169 *:9100 running (37s) 18s ago 9m 8702k - 1.5.0 0da6a335fe13 5505baf702a6 2024-09-16T00:27:59.446 INFO:teuthology.orchestra.run.smithi112.stdout:osd.0 smithi112 running (8m) 16s ago 8m 189M 1557M 17.2.7-1633-g6d025bc2 33ed8dba3996 f6b2e146ffdf 2024-09-16T00:27:59.446 INFO:teuthology.orchestra.run.smithi112.stdout:osd.1 smithi112 running (7m) 16s ago 7m 200M 1557M 17.2.7-1633-g6d025bc2 33ed8dba3996 e9bee87bba0f 2024-09-16T00:27:59.447 INFO:teuthology.orchestra.run.smithi112.stdout:osd.2 smithi112 running (7m) 16s ago 7m 157M 1557M 17.2.7-1633-g6d025bc2 33ed8dba3996 100ec4e4852a 2024-09-16T00:27:59.447 INFO:teuthology.orchestra.run.smithi112.stdout:osd.3 smithi169 running (6m) 18s ago 6m 230M 2581M 17.2.7-1633-g6d025bc2 33ed8dba3996 f88463e19f23 2024-09-16T00:27:59.447 INFO:teuthology.orchestra.run.smithi112.stdout:osd.4 smithi169 running (6m) 18s ago 6m 209M 2581M 17.2.7-1633-g6d025bc2 33ed8dba3996 d3e4cdb016b9 2024-09-16T00:27:59.447 INFO:teuthology.orchestra.run.smithi112.stdout:osd.5 smithi169 running (6m) 18s ago 6m 197M 2581M 17.2.7-1633-g6d025bc2 33ed8dba3996 8e7d708d74eb 2024-09-16T00:27:59.447 INFO:teuthology.orchestra.run.smithi112.stdout:prometheus.smithi112 smithi112 *:9095 starting - - - - 2024-09-16T00:27:59.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.446+0000 7fb135826640 1 -- 172.21.15.112:0/562090705 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7fb11806df60 msgr2=0x7fb118070410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:59.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.446+0000 7fb135826640 1 --2- 172.21.15.112:0/562090705 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7fb11806df60 0x7fb118070410 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb13010ce30 tx=0x7fb124004330 comp rx=0 tx=0).stop 2024-09-16T00:27:59.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.446+0000 7fb135826640 1 -- 172.21.15.112:0/562090705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1300fc550 msgr2=0x7fb13010d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:59.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.446+0000 7fb135826640 1 --2- 172.21.15.112:0/562090705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1300fc550 0x7fb13010d080 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb120003080 tx=0x7fb120009da0 comp rx=0 tx=0).stop 2024-09-16T00:27:59.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.447+0000 7fb135826640 1 -- 172.21.15.112:0/562090705 shutdown_connections 2024-09-16T00:27:59.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.447+0000 7fb135826640 1 --2- 172.21.15.112:0/562090705 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7fb11806df60 0x7fb118070410 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:59.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.447+0000 7fb135826640 1 --2- 172.21.15.112:0/562090705 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1300fd820 0x7fb13010d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:59.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.447+0000 7fb135826640 1 --2- 172.21.15.112:0/562090705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1300fc550 0x7fb13010d080 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:59.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.447+0000 7fb135826640 1 -- 172.21.15.112:0/562090705 >> 172.21.15.112:0/562090705 conn(0x7fb1300f7d30 msgr2=0x7fb1300f94c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:59.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.447+0000 7fb135826640 1 -- 172.21.15.112:0/562090705 shutdown_connections 2024-09-16T00:27:59.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.447+0000 7fb135826640 1 -- 172.21.15.112:0/562090705 wait complete. 2024-09-16T00:27:59.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.529+0000 7f4e6c9cb640 1 -- 172.21.15.112:0/1488351248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e680ff740 msgr2=0x7f4e68101b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:59.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.529+0000 7f4e6c9cb640 1 --2- 172.21.15.112:0/1488351248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e680ff740 0x7f4e68101b20 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c0048d0 tx=0x7f4e5c009da0 comp rx=0 tx=0).stop 2024-09-16T00:27:59.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.529+0000 7f4e6c9cb640 1 -- 172.21.15.112:0/1488351248 shutdown_connections 2024-09-16T00:27:59.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.529+0000 7f4e6c9cb640 1 --2- 172.21.15.112:0/1488351248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e680ff740 0x7f4e68101b20 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:59.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.529+0000 7f4e6c9cb640 1 --2- 172.21.15.112:0/1488351248 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e68065b80 0x7f4e680ff200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:59.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.529+0000 7f4e6c9cb640 1 -- 172.21.15.112:0/1488351248 >> 172.21.15.112:0/1488351248 conn(0x7f4e680f6c10 msgr2=0x7f4e680f9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:59.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.530+0000 7f4e6c9cb640 1 -- 172.21.15.112:0/1488351248 shutdown_connections 2024-09-16T00:27:59.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.530+0000 7f4e6c9cb640 1 -- 172.21.15.112:0/1488351248 wait complete. 2024-09-16T00:27:59.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.530+0000 7f4e6c9cb640 1 Processor -- start 2024-09-16T00:27:59.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.530+0000 7f4e6c9cb640 1 -- start start 2024-09-16T00:27:59.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.530+0000 7f4e6c9cb640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e68065b80 0x7f4e680fcbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:59.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.531+0000 7f4e6c9cb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e680ff740 0x7f4e680fb100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:59.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.531+0000 7f4e6c9cb640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e680fb640 con 0x7f4e680ff740 2024-09-16T00:27:59.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.531+0000 7f4e6c9cb640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e680fb780 con 0x7f4e68065b80 2024-09-16T00:27:59.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.531+0000 7f4e65d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e680ff740 0x7f4e680fb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:59.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.531+0000 7f4e65d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e680ff740 0x7f4e680fb100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39224/0 (socket says 172.21.15.112:39224) 2024-09-16T00:27:59.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.531+0000 7f4e65d9b640 1 -- 172.21.15.112:0/3283104105 learned_addr learned my addr 172.21.15.112:0/3283104105 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:27:59.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.531+0000 7f4e6659c640 1 --2- 172.21.15.112:0/3283104105 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e68065b80 0x7f4e680fcbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:59.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.531+0000 7f4e65d9b640 1 -- 172.21.15.112:0/3283104105 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e68065b80 msgr2=0x7f4e680fcbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:59.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.531+0000 7f4e65d9b640 1 --2- 172.21.15.112:0/3283104105 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e68065b80 0x7f4e680fcbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:59.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.531+0000 7f4e65d9b640 1 -- 172.21.15.112:0/3283104105 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e5c005040 con 0x7f4e680ff740 2024-09-16T00:27:59.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.531+0000 7f4e65d9b640 1 --2- 172.21.15.112:0/3283104105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e680ff740 0x7f4e680fb100 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c0043d0 tx=0x7f4e5c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:59.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.531+0000 7f4e56ffd640 1 -- 172.21.15.112:0/3283104105 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e5c038070 con 0x7f4e680ff740 2024-09-16T00:27:59.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.531+0000 7f4e56ffd640 1 -- 172.21.15.112:0/3283104105 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f4e5c02ae90 con 0x7f4e680ff740 2024-09-16T00:27:59.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.531+0000 7f4e6c9cb640 1 -- 172.21.15.112:0/3283104105 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e680fba00 con 0x7f4e680ff740 2024-09-16T00:27:59.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.531+0000 7f4e56ffd640 1 -- 172.21.15.112:0/3283104105 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e5c03e670 con 0x7f4e680ff740 2024-09-16T00:27:59.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.532+0000 7f4e6c9cb640 1 -- 172.21.15.112:0/3283104105 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e680fbf20 con 0x7f4e680ff740 2024-09-16T00:27:59.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.533+0000 7f4e6c9cb640 1 -- 172.21.15.112:0/3283104105 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e6804b720 con 0x7f4e680ff740 2024-09-16T00:27:59.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.533+0000 7f4e56ffd640 1 -- 172.21.15.112:0/3283104105 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 31) v1 ==== 92377+0+0 (secure 0 0 0) 0x7f4e5c030020 con 0x7f4e680ff740 2024-09-16T00:27:59.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.534+0000 7f4e56ffd640 1 --2- 172.21.15.112:0/3283104105 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f4e5006df60 0x7f4e50070410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:59.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.534+0000 7f4e56ffd640 1 -- 172.21.15.112:0/3283104105 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6152+0+0 (secure 0 0 0) 0x7f4e5c0adb40 con 0x7f4e680ff740 2024-09-16T00:27:59.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.534+0000 7f4e6659c640 1 --2- 172.21.15.112:0/3283104105 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f4e5006df60 0x7f4e50070410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:59.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.534+0000 7f4e6659c640 1 --2- 172.21.15.112:0/3283104105 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f4e5006df60 0x7f4e50070410 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4e680fc970 tx=0x7f4e5800b040 comp rx=0 tx=0).ready entity=mgr.14678 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:59.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.539+0000 7f4e56ffd640 1 -- 172.21.15.112:0/3283104105 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185942 (secure 0 0 0) 0x7f4e5c077420 con 0x7f4e680ff740 2024-09-16T00:27:59.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.778+0000 7f4e6c9cb640 1 -- 172.21.15.112:0/3283104105 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4e68115200 con 0x7f4e680ff740 2024-09-16T00:27:59.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.779+0000 7f4e56ffd640 1 -- 172.21.15.112:0/3283104105 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+747 (secure 0 0 0) 0x7f4e5c07aae0 con 0x7f4e680ff740 2024-09-16T00:27:59.779 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:27:59.779 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": { 2024-09-16T00:27:59.779 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 17.2.7-1633-g6d025bc2 (6d025bc26f3176c910707f8cb4c8d8e076640615) quincy (stable)": 2 2024-09-16T00:27:59.780 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:27:59.780 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": { 2024-09-16T00:27:59.780 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:27:59.780 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:27:59.780 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": { 2024-09-16T00:27:59.780 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 17.2.7-1633-g6d025bc2 (6d025bc26f3176c910707f8cb4c8d8e076640615) quincy (stable)": 6 2024-09-16T00:27:59.780 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:27:59.780 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": { 2024-09-16T00:27:59.780 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 17.2.7-1633-g6d025bc2 (6d025bc26f3176c910707f8cb4c8d8e076640615) quincy (stable)": 4 2024-09-16T00:27:59.780 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:27:59.780 INFO:teuthology.orchestra.run.smithi112.stdout: "overall": { 2024-09-16T00:27:59.780 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 17.2.7-1633-g6d025bc2 (6d025bc26f3176c910707f8cb4c8d8e076640615) quincy (stable)": 12, 2024-09-16T00:27:59.781 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:27:59.781 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-16T00:27:59.781 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:27:59.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.781+0000 7f4e6c9cb640 1 -- 172.21.15.112:0/3283104105 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f4e5006df60 msgr2=0x7f4e50070410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:59.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.781+0000 7f4e6c9cb640 1 --2- 172.21.15.112:0/3283104105 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f4e5006df60 0x7f4e50070410 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4e680fc970 tx=0x7f4e5800b040 comp rx=0 tx=0).stop 2024-09-16T00:27:59.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.781+0000 7f4e6c9cb640 1 -- 172.21.15.112:0/3283104105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e680ff740 msgr2=0x7f4e680fb100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:59.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.781+0000 7f4e6c9cb640 1 --2- 172.21.15.112:0/3283104105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e680ff740 0x7f4e680fb100 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c0043d0 tx=0x7f4e5c009da0 comp rx=0 tx=0).stop 2024-09-16T00:27:59.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.781+0000 7f4e6c9cb640 1 -- 172.21.15.112:0/3283104105 shutdown_connections 2024-09-16T00:27:59.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.781+0000 7f4e6c9cb640 1 --2- 172.21.15.112:0/3283104105 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f4e5006df60 0x7f4e50070410 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:59.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.781+0000 7f4e6c9cb640 1 --2- 172.21.15.112:0/3283104105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e680ff740 0x7f4e680fb100 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:59.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.781+0000 7f4e6c9cb640 1 --2- 172.21.15.112:0/3283104105 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e68065b80 0x7f4e680fcbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:59.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.781+0000 7f4e6c9cb640 1 -- 172.21.15.112:0/3283104105 >> 172.21.15.112:0/3283104105 conn(0x7f4e680f6c10 msgr2=0x7f4e680f6fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:59.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.781+0000 7f4e6c9cb640 1 -- 172.21.15.112:0/3283104105 shutdown_connections 2024-09-16T00:27:59.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.781+0000 7f4e6c9cb640 1 -- 172.21.15.112:0/3283104105 wait complete. 2024-09-16T00:27:59.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.862+0000 7fad6e175640 1 -- 172.21.15.112:0/1917833412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad68065be0 msgr2=0x7fad680ff1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:59.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.862+0000 7fad6e175640 1 --2- 172.21.15.112:0/1917833412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad68065be0 0x7fad680ff1a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fad580048d0 tx=0x7fad58009da0 comp rx=0 tx=0).stop 2024-09-16T00:27:59.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.862+0000 7fad6e175640 1 -- 172.21.15.112:0/1917833412 shutdown_connections 2024-09-16T00:27:59.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.862+0000 7fad6e175640 1 --2- 172.21.15.112:0/1917833412 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad680ff6e0 0x7fad68101ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:59.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.862+0000 7fad6e175640 1 --2- 172.21.15.112:0/1917833412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad68065be0 0x7fad680ff1a0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:59.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.862+0000 7fad6e175640 1 -- 172.21.15.112:0/1917833412 >> 172.21.15.112:0/1917833412 conn(0x7fad680f6c10 msgr2=0x7fad680f9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:59.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.862+0000 7fad6e175640 1 -- 172.21.15.112:0/1917833412 shutdown_connections 2024-09-16T00:27:59.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.862+0000 7fad6e175640 1 -- 172.21.15.112:0/1917833412 wait complete. 2024-09-16T00:27:59.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.863+0000 7fad6e175640 1 Processor -- start 2024-09-16T00:27:59.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.863+0000 7fad6e175640 1 -- start start 2024-09-16T00:27:59.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.863+0000 7fad6e175640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad68065be0 0x7fad6810d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:59.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.863+0000 7fad6e175640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad680ff6e0 0x7fad6810d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:59.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.863+0000 7fad6e175640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad6810dcb0 con 0x7fad680ff6e0 2024-09-16T00:27:59.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.863+0000 7fad6e175640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad6810e480 con 0x7fad68065be0 2024-09-16T00:27:59.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.863+0000 7fad66ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad680ff6e0 0x7fad6810d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:59.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.863+0000 7fad66ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad680ff6e0 0x7fad6810d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39234/0 (socket says 172.21.15.112:39234) 2024-09-16T00:27:59.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.863+0000 7fad66ffd640 1 -- 172.21.15.112:0/4083274046 learned_addr learned my addr 172.21.15.112:0/4083274046 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:27:59.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.863+0000 7fad677fe640 1 --2- 172.21.15.112:0/4083274046 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad68065be0 0x7fad6810d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:59.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.864+0000 7fad66ffd640 1 -- 172.21.15.112:0/4083274046 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad68065be0 msgr2=0x7fad6810d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:59.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.864+0000 7fad66ffd640 1 --2- 172.21.15.112:0/4083274046 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad68065be0 0x7fad6810d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:59.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.864+0000 7fad66ffd640 1 -- 172.21.15.112:0/4083274046 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad58005040 con 0x7fad680ff6e0 2024-09-16T00:27:59.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.864+0000 7fad66ffd640 1 --2- 172.21.15.112:0/4083274046 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad680ff6e0 0x7fad6810d660 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fad5c002a80 tx=0x7fad5c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:59.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.864+0000 7fad4ffff640 1 -- 172.21.15.112:0/4083274046 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad5c007c30 con 0x7fad680ff6e0 2024-09-16T00:27:59.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.864+0000 7fad4ffff640 1 -- 172.21.15.112:0/4083274046 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fad5c00f040 con 0x7fad680ff6e0 2024-09-16T00:27:59.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.864+0000 7fad6e175640 1 -- 172.21.15.112:0/4083274046 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad6810e710 con 0x7fad680ff6e0 2024-09-16T00:27:59.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.864+0000 7fad4ffff640 1 -- 172.21.15.112:0/4083274046 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad5c00dab0 con 0x7fad680ff6e0 2024-09-16T00:27:59.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.864+0000 7fad6e175640 1 -- 172.21.15.112:0/4083274046 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad6810ec30 con 0x7fad680ff6e0 2024-09-16T00:27:59.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.865+0000 7fad6e175640 1 -- 172.21.15.112:0/4083274046 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad680fa890 con 0x7fad680ff6e0 2024-09-16T00:27:59.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.866+0000 7fad4ffff640 1 -- 172.21.15.112:0/4083274046 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 31) v1 ==== 92377+0+0 (secure 0 0 0) 0x7fad5c00dc10 con 0x7fad680ff6e0 2024-09-16T00:27:59.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.866+0000 7fad4ffff640 1 --2- 172.21.15.112:0/4083274046 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7fad5006df60 0x7fad50070410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:59.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.866+0000 7fad4ffff640 1 -- 172.21.15.112:0/4083274046 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6152+0+0 (secure 0 0 0) 0x7fad5c08d310 con 0x7fad680ff6e0 2024-09-16T00:27:59.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.866+0000 7fad677fe640 1 --2- 172.21.15.112:0/4083274046 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7fad5006df60 0x7fad50070410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:59.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.867+0000 7fad677fe640 1 --2- 172.21.15.112:0/4083274046 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7fad5006df60 0x7fad50070410 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fad580048a0 tx=0x7fad58035040 comp rx=0 tx=0).ready entity=mgr.14678 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:59.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:27:59.870+0000 7fad4ffff640 1 -- 172.21.15.112:0/4083274046 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185942 (secure 0 0 0) 0x7fad5c056ae0 con 0x7fad680ff6e0 2024-09-16T00:28:00.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:00.058+0000 7fad6e175640 1 -- 172.21.15.112:0/4083274046 --> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fad6810efc0 con 0x7fad5006df60 2024-09-16T00:28:00.059 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:00 smithi112 ceph-mon[27869]: from='client.24473 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:28:00.059 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:00 smithi112 ceph-mon[27869]: pgmap v14: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:28:00.059 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:00 smithi112 ceph-mon[27869]: from='client.14726 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:28:00.059 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:00 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/3283104105' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:28:00.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:00.059+0000 7fad4ffff640 1 -- 172.21.15.112:0/4083274046 <== mgr.14678 v2:172.21.15.112:6800/1965658616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7fad6810efc0 con 0x7fad5006df60 2024-09-16T00:28:00.060 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:28:00.060 INFO:teuthology.orchestra.run.smithi112.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80", 2024-09-16T00:28:00.060 INFO:teuthology.orchestra.run.smithi112.stdout: "in_progress": true, 2024-09-16T00:28:00.060 INFO:teuthology.orchestra.run.smithi112.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-09-16T00:28:00.060 INFO:teuthology.orchestra.run.smithi112.stdout: "services_complete": [], 2024-09-16T00:28:00.060 INFO:teuthology.orchestra.run.smithi112.stdout: "progress": "", 2024-09-16T00:28:00.060 INFO:teuthology.orchestra.run.smithi112.stdout: "message": "Currently upgrading mgr daemons", 2024-09-16T00:28:00.060 INFO:teuthology.orchestra.run.smithi112.stdout: "is_paused": false 2024-09-16T00:28:00.060 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:28:00.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:00.061+0000 7fad6e175640 1 -- 172.21.15.112:0/4083274046 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7fad5006df60 msgr2=0x7fad50070410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:00.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:00.061+0000 7fad6e175640 1 --2- 172.21.15.112:0/4083274046 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7fad5006df60 0x7fad50070410 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fad580048a0 tx=0x7fad58035040 comp rx=0 tx=0).stop 2024-09-16T00:28:00.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:00.061+0000 7fad6e175640 1 -- 172.21.15.112:0/4083274046 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad680ff6e0 msgr2=0x7fad6810d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:00.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:00.061+0000 7fad6e175640 1 --2- 172.21.15.112:0/4083274046 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad680ff6e0 0x7fad6810d660 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fad5c002a80 tx=0x7fad5c004600 comp rx=0 tx=0).stop 2024-09-16T00:28:00.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:00.061+0000 7fad6e175640 1 -- 172.21.15.112:0/4083274046 shutdown_connections 2024-09-16T00:28:00.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:00.061+0000 7fad6e175640 1 --2- 172.21.15.112:0/4083274046 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7fad5006df60 0x7fad50070410 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:00.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:00.061+0000 7fad6e175640 1 --2- 172.21.15.112:0/4083274046 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad680ff6e0 0x7fad6810d660 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:00.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:00.061+0000 7fad6e175640 1 --2- 172.21.15.112:0/4083274046 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad68065be0 0x7fad6810d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:00.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:00.061+0000 7fad6e175640 1 -- 172.21.15.112:0/4083274046 >> 172.21.15.112:0/4083274046 conn(0x7fad680f6c10 msgr2=0x7fad680f6fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:00.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:00.062+0000 7fad6e175640 1 -- 172.21.15.112:0/4083274046 shutdown_connections 2024-09-16T00:28:00.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:00.062+0000 7fad6e175640 1 -- 172.21.15.112:0/4083274046 wait complete. 2024-09-16T00:28:00.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:00 smithi169 ceph-mon[34653]: from='client.24473 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:28:00.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:00 smithi169 ceph-mon[34653]: pgmap v14: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:28:00.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:00 smithi169 ceph-mon[34653]: from='client.14726 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:28:00.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:00 smithi169 ceph-mon[34653]: from='client.? 172.21.15.112:0/3283104105' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:28:01.582 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:01 smithi169 ceph-mon[34653]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:28:01.582 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:01 smithi169 ceph-mon[34653]: from='client.14738 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:28:01.582 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:01 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:28:01.582 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:01 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:28:01.582 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:01 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:28:01.582 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:01 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:28:01.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:01 smithi112 ceph-mon[27869]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:28:01.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:01 smithi112 ceph-mon[27869]: from='client.14738 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:28:01.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:01 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:28:01.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:01 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:28:01.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:01 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:28:01.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:01 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:28:02.563 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:02 smithi169 ceph-mon[34653]: pgmap v15: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:28:02.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:02 smithi112 ceph-mon[27869]: pgmap v15: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:28:03.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:03 smithi112 ceph-mon[27869]: Standby manager daemon smithi169.mtrobk restarted 2024-09-16T00:28:03.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:03 smithi112 ceph-mon[27869]: Standby manager daemon smithi169.mtrobk started 2024-09-16T00:28:03.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:03 smithi112 ceph-mon[27869]: from='mgr.? 172.21.15.169:0/3979125716' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi169.mtrobk/crt"}]: dispatch 2024-09-16T00:28:03.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:03 smithi112 ceph-mon[27869]: from='mgr.? 172.21.15.169:0/3979125716' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T00:28:03.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:03 smithi112 ceph-mon[27869]: from='mgr.? 172.21.15.169:0/3979125716' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi169.mtrobk/key"}]: dispatch 2024-09-16T00:28:03.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:03 smithi112 ceph-mon[27869]: from='mgr.? 172.21.15.169:0/3979125716' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T00:28:03.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:03 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:28:03.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:03 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:28:03.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:03 smithi169 ceph-mon[34653]: Standby manager daemon smithi169.mtrobk restarted 2024-09-16T00:28:03.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:03 smithi169 ceph-mon[34653]: Standby manager daemon smithi169.mtrobk started 2024-09-16T00:28:03.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:03 smithi169 ceph-mon[34653]: from='mgr.? 172.21.15.169:0/3979125716' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi169.mtrobk/crt"}]: dispatch 2024-09-16T00:28:03.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:03 smithi169 ceph-mon[34653]: from='mgr.? 172.21.15.169:0/3979125716' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T00:28:03.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:03 smithi169 ceph-mon[34653]: from='mgr.? 172.21.15.169:0/3979125716' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi169.mtrobk/key"}]: dispatch 2024-09-16T00:28:03.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:03 smithi169 ceph-mon[34653]: from='mgr.? 172.21.15.169:0/3979125716' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T00:28:03.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:03 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:28:03.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:03 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:28:04.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:04 smithi169 ceph-mon[34653]: pgmap v16: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:28:04.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:04 smithi169 ceph-mon[34653]: mgrmap e32: smithi112.xrkkbc(active, since 26s), standbys: smithi169.mtrobk 2024-09-16T00:28:04.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:04 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:28:04.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:04 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:28:04.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:04 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:28:04.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:04 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:28:04.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:04 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:28:04.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:04 smithi112 ceph-mon[27869]: pgmap v16: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:28:04.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:04 smithi112 ceph-mon[27869]: mgrmap e32: smithi112.xrkkbc(active, since 26s), standbys: smithi169.mtrobk 2024-09-16T00:28:04.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:04 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:28:04.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:04 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:28:04.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:04 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:28:04.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:04 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:28:04.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:04 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:28:05.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:05 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:28:05.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:05 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:28:05.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:05 smithi112 ceph-mon[27869]: Upgrade: Setting container_image for all mgr 2024-09-16T00:28:05.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:05 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:28:05.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:05 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi112.xrkkbc"}]: dispatch 2024-09-16T00:28:05.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:05 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi112.xrkkbc"}]': finished 2024-09-16T00:28:05.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:05 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi169.mtrobk"}]: dispatch 2024-09-16T00:28:05.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:05 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi169.mtrobk"}]': finished 2024-09-16T00:28:05.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:05 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:28:05.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:05 smithi112 ceph-mon[27869]: Upgrade: Setting container_image for all rgw 2024-09-16T00:28:05.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:05 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:28:05.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:05 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:28:05.591 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:05 smithi112 ceph-mon[27869]: Upgrade: Setting container_image for all rbd-mirror 2024-09-16T00:28:05.591 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:05 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:28:05.591 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:05 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:28:05.591 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:05 smithi112 ceph-mon[27869]: Upgrade: Setting container_image for all cephfs-mirror 2024-09-16T00:28:05.591 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:05 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:28:05.591 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:05 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:28:05.591 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:05 smithi112 ceph-mon[27869]: Upgrade: Setting container_image for all ceph-exporter 2024-09-16T00:28:05.591 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:05 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:28:05.591 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:05 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:28:05.591 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:05 smithi112 ceph-mon[27869]: Upgrade: Setting container_image for all iscsi 2024-09-16T00:28:05.591 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:05 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:28:05.591 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:05 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:28:05.591 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:05 smithi112 ceph-mon[27869]: Upgrade: Setting container_image for all nfs 2024-09-16T00:28:05.591 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:05 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:28:05.592 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:05 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:28:05.592 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:05 smithi112 ceph-mon[27869]: Upgrade: Setting container_image for all nvmeof 2024-09-16T00:28:05.592 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:05 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:28:05.592 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:05 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:28:05.592 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:05 smithi112 ceph-mon[27869]: Upgrade: Setting container_image for all node-exporter 2024-09-16T00:28:05.592 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:05 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:28:05.592 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:05 smithi112 ceph-mon[27869]: Upgrade: Setting container_image for all prometheus 2024-09-16T00:28:05.592 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:05 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:28:05.592 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:05 smithi112 ceph-mon[27869]: Upgrade: Setting container_image for all alertmanager 2024-09-16T00:28:05.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:05 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:28:05.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:05 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:28:05.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:05 smithi169 ceph-mon[34653]: Upgrade: Setting container_image for all mgr 2024-09-16T00:28:05.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:05 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:28:05.778 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:05 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi112.xrkkbc"}]: dispatch 2024-09-16T00:28:05.778 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:05 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi112.xrkkbc"}]': finished 2024-09-16T00:28:05.778 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:05 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi169.mtrobk"}]: dispatch 2024-09-16T00:28:05.778 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:05 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi169.mtrobk"}]': finished 2024-09-16T00:28:05.778 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:05 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:28:05.778 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:05 smithi169 ceph-mon[34653]: Upgrade: Setting container_image for all rgw 2024-09-16T00:28:05.778 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:05 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:28:05.778 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:05 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:28:05.778 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:05 smithi169 ceph-mon[34653]: Upgrade: Setting container_image for all rbd-mirror 2024-09-16T00:28:05.779 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:05 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:28:05.779 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:05 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:28:05.779 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:05 smithi169 ceph-mon[34653]: Upgrade: Setting container_image for all cephfs-mirror 2024-09-16T00:28:05.779 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:05 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:28:05.779 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:05 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:28:05.779 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:05 smithi169 ceph-mon[34653]: Upgrade: Setting container_image for all ceph-exporter 2024-09-16T00:28:05.779 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:05 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:28:05.779 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:05 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:28:05.780 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:05 smithi169 ceph-mon[34653]: Upgrade: Setting container_image for all iscsi 2024-09-16T00:28:05.780 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:05 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:28:05.780 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:05 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:28:05.780 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:05 smithi169 ceph-mon[34653]: Upgrade: Setting container_image for all nfs 2024-09-16T00:28:05.780 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:05 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:28:05.780 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:05 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:28:05.780 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:05 smithi169 ceph-mon[34653]: Upgrade: Setting container_image for all nvmeof 2024-09-16T00:28:05.780 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:05 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:28:05.780 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:05 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:28:05.780 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:05 smithi169 ceph-mon[34653]: Upgrade: Setting container_image for all node-exporter 2024-09-16T00:28:05.781 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:05 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:28:05.781 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:05 smithi169 ceph-mon[34653]: Upgrade: Setting container_image for all prometheus 2024-09-16T00:28:05.781 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:05 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:28:05.781 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:05 smithi169 ceph-mon[34653]: Upgrade: Setting container_image for all alertmanager 2024-09-16T00:28:06.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:06 smithi169 ceph-mon[34653]: pgmap v17: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:28:06.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:06 smithi112 ceph-mon[27869]: pgmap v17: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:28:07.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:07 smithi169 ceph-mon[34653]: Upgrade: Updating grafana.smithi112 2024-09-16T00:28:07.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:07 smithi169 ceph-mon[34653]: Deploying daemon grafana.smithi112 on smithi112 2024-09-16T00:28:07.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:07 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:28:07.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:07 smithi112 ceph-mon[27869]: Upgrade: Updating grafana.smithi112 2024-09-16T00:28:07.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:07 smithi112 ceph-mon[27869]: Deploying daemon grafana.smithi112 on smithi112 2024-09-16T00:28:07.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:07 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:28:08.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:08 smithi169 ceph-mon[34653]: pgmap v18: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:28:08.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:08 smithi112 ceph-mon[27869]: pgmap v18: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:28:10.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:10 smithi169 ceph-mon[34653]: pgmap v19: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:28:10.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:10 smithi112 ceph-mon[27869]: pgmap v19: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:28:12.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:12 smithi112 ceph-mon[27869]: pgmap v20: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:28:12.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:12 smithi169 ceph-mon[34653]: pgmap v20: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:28:14.718 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:14 smithi112 ceph-mon[27869]: pgmap v21: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:28:14.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:14 smithi169 ceph-mon[34653]: pgmap v21: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:28:16.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:16 smithi169 ceph-mon[34653]: pgmap v22: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:28:16.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:16 smithi112 ceph-mon[27869]: pgmap v22: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:28:18.701 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:18 smithi112 ceph-mon[27869]: pgmap v23: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:28:18.701 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:18 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:28:18.701 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:18 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:28:18.701 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:18 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:28:18.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:18 smithi169 ceph-mon[34653]: pgmap v23: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:28:18.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:18 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:28:18.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:18 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:28:18.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:18 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:28:19.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:19 smithi169 ceph-mon[34653]: pgmap v24: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:28:19.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:19 smithi112 ceph-mon[27869]: pgmap v24: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:28:22.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:22 smithi169 ceph-mon[34653]: pgmap v25: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:28:22.314 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:22 smithi112 ceph-mon[27869]: pgmap v25: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:28:23.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:23 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:28:23.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:23 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:28:23.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:23 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:28:23.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:23 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:28:23.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:23 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:28:23.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:23 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:28:24.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:24 smithi169 ceph-mon[34653]: pgmap v26: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:28:24.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:24 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:28:24.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:24 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:28:24.301 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:24 smithi112 ceph-mon[27869]: pgmap v26: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:28:24.301 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:24 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:28:24.301 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:24 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:28:26.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:26 smithi169 ceph-mon[34653]: pgmap v27: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:28:26.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:26 smithi112 ceph-mon[27869]: pgmap v27: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:28:28.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:27 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:28:28.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:27 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:28:28.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:27 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:28:28.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:27 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:28:28.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:27 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:28:28.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:27 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-16T00:28:28.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:27 smithi169 ceph-mon[34653]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-16T00:28:28.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:27 smithi169 ceph-mon[34653]: pgmap v28: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:28:28.278 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:27 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:28:28.278 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:27 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:28:28.278 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:27 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:28:28.278 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:27 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:28:28.278 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:27 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:28:28.278 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:27 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:28:28.278 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:27 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:28:28.278 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:27 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:28:28.278 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:27 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:28:28.278 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:27 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:28:28.278 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:27 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:28:28.278 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:27 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:28:28.279 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:27 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:28:28.279 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:27 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:28:28.279 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:27 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:28:28.279 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:27 smithi169 ceph-mon[34653]: Upgrade: Finalizing container_image settings 2024-09-16T00:28:28.279 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:27 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:28:28.279 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:27 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-09-16T00:28:28.279 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:27 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-09-16T00:28:28.279 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:27 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-16T00:28:28.279 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:27 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-09-16T00:28:28.279 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:27 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-09-16T00:28:28.279 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:27 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-09-16T00:28:28.279 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:27 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-09-16T00:28:28.279 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:27 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-09-16T00:28:28.279 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:27 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-09-16T00:28:28.279 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:27 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-09-16T00:28:28.280 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:27 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-09-16T00:28:28.280 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:27 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-16T00:28:28.280 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:27 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-09-16T00:28:28.280 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:27 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-09-16T00:28:28.280 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:27 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-09-16T00:28:28.280 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:27 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-09-16T00:28:28.280 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:27 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-09-16T00:28:28.280 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:27 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-09-16T00:28:28.280 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:27 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-09-16T00:28:28.280 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:27 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-09-16T00:28:28.280 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:27 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-16T00:28:28.280 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:27 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-16T00:28:28.280 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:27 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-16T00:28:28.280 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:27 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-16T00:28:28.281 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:27 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-16T00:28:28.281 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:27 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-16T00:28:28.281 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:27 smithi169 ceph-mon[34653]: Upgrade: Complete! 2024-09-16T00:28:28.281 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:27 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-09-16T00:28:28.281 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:27 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-09-16T00:28:28.281 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:27 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:28:28.281 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:27 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:28:28.281 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:27 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:28:28.281 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:27 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:28:28.281 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:27 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:28:28.281 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:27 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:28:28.281 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:27 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:28:28.281 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:27 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:28:28.281 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:27 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:28:28.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:27 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:28:28.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:27 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:28:28.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:27 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:28:28.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:27 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:28:28.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:27 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:28:28.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:27 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-16T00:28:28.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:27 smithi112 ceph-mon[27869]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-16T00:28:28.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:27 smithi112 ceph-mon[27869]: pgmap v28: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:28:28.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:27 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:28:28.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:27 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:28:28.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:27 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:28:28.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:27 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:28:28.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:27 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:28:28.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:27 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:28:28.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:27 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:28:28.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:27 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:28:28.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:27 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:28:28.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:27 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:28:28.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:27 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:28:28.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:27 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:28:28.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:27 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:28:28.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:27 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:28:28.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:27 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:28:28.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:27 smithi112 ceph-mon[27869]: Upgrade: Finalizing container_image settings 2024-09-16T00:28:28.342 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:27 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:28:28.342 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:27 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-09-16T00:28:28.342 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:27 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-09-16T00:28:28.342 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:27 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-16T00:28:28.342 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:27 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-09-16T00:28:28.342 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:27 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-09-16T00:28:28.342 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:27 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-09-16T00:28:28.342 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:27 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-09-16T00:28:28.342 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:27 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-09-16T00:28:28.342 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:27 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-09-16T00:28:28.342 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:27 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-09-16T00:28:28.342 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:27 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-09-16T00:28:28.343 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:27 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-16T00:28:28.343 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:27 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-09-16T00:28:28.343 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:27 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-09-16T00:28:28.343 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:27 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-09-16T00:28:28.343 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:27 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-09-16T00:28:28.343 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:27 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-09-16T00:28:28.343 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:27 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-09-16T00:28:28.343 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:27 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-09-16T00:28:28.343 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:27 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-09-16T00:28:28.343 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:27 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-16T00:28:28.343 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:27 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-16T00:28:28.343 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:27 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-16T00:28:28.343 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:27 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-16T00:28:28.343 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:27 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-16T00:28:28.344 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:27 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-16T00:28:28.344 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:27 smithi112 ceph-mon[27869]: Upgrade: Complete! 2024-09-16T00:28:28.344 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:27 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-09-16T00:28:28.344 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:27 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-09-16T00:28:28.344 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:27 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:28:28.344 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:27 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:28:28.344 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:27 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:28:28.344 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:27 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:28:28.344 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:27 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:28:28.344 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:27 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:28:28.344 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:27 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:28:28.344 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:27 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:28:28.344 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:27 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:28:30.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:30.152+0000 7fd269bb2640 1 -- 172.21.15.112:0/2783912200 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2640fb430 msgr2=0x7fd2640fb840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:30.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:30.152+0000 7fd269bb2640 1 --2- 172.21.15.112:0/2783912200 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2640fb430 0x7fd2640fb840 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fd254009c50 tx=0x7fd254004d10 comp rx=0 tx=0).stop 2024-09-16T00:28:30.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:30.152+0000 7fd269bb2640 1 -- 172.21.15.112:0/2783912200 shutdown_connections 2024-09-16T00:28:30.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:30.152+0000 7fd269bb2640 1 --2- 172.21.15.112:0/2783912200 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2640fc700 0x7fd2640fcb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:30.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:30.152+0000 7fd269bb2640 1 --2- 172.21.15.112:0/2783912200 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2640fb430 0x7fd2640fb840 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:30.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:30.152+0000 7fd269bb2640 1 -- 172.21.15.112:0/2783912200 >> 172.21.15.112:0/2783912200 conn(0x7fd2640f6c10 msgr2=0x7fd2640f9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:30.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:30.152+0000 7fd269bb2640 1 -- 172.21.15.112:0/2783912200 shutdown_connections 2024-09-16T00:28:30.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:30.152+0000 7fd269bb2640 1 -- 172.21.15.112:0/2783912200 wait complete. 2024-09-16T00:28:30.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:30.153+0000 7fd269bb2640 1 Processor -- start 2024-09-16T00:28:30.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:30.153+0000 7fd269bb2640 1 -- start start 2024-09-16T00:28:30.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:30.153+0000 7fd269bb2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2640fb430 0x7fd26410fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:30.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:30.153+0000 7fd269bb2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2640fc700 0x7fd264112310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:30.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:30.153+0000 7fd269bb2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd264065c60 con 0x7fd2640fb430 2024-09-16T00:28:30.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:30.153+0000 7fd269bb2640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd264065dd0 con 0x7fd2640fc700 2024-09-16T00:28:30.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:30.153+0000 7fd2637fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2640fb430 0x7fd26410fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:30.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:30.153+0000 7fd2637fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2640fb430 0x7fd26410fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50702/0 (socket says 172.21.15.112:50702) 2024-09-16T00:28:30.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:30.153+0000 7fd2637fe640 1 -- 172.21.15.112:0/569532577 learned_addr learned my addr 172.21.15.112:0/569532577 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:28:30.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:30.154+0000 7fd262ffd640 1 --2- 172.21.15.112:0/569532577 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2640fc700 0x7fd264112310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:30.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:30.154+0000 7fd2637fe640 1 -- 172.21.15.112:0/569532577 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2640fc700 msgr2=0x7fd264112310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:30.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:30.154+0000 7fd2637fe640 1 --2- 172.21.15.112:0/569532577 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2640fc700 0x7fd264112310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:30.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:30.154+0000 7fd2637fe640 1 -- 172.21.15.112:0/569532577 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2540098b0 con 0x7fd2640fb430 2024-09-16T00:28:30.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:30.154+0000 7fd2637fe640 1 --2- 172.21.15.112:0/569532577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2640fb430 0x7fd26410fdc0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd254004a30 tx=0x7fd254004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:30.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:30.154+0000 7fd24bfff640 1 -- 172.21.15.112:0/569532577 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd254009710 con 0x7fd2640fb430 2024-09-16T00:28:30.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:30.154+0000 7fd24bfff640 1 -- 172.21.15.112:0/569532577 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fd254032b10 con 0x7fd2640fb430 2024-09-16T00:28:30.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:30.154+0000 7fd269bb2640 1 -- 172.21.15.112:0/569532577 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd264112850 con 0x7fd2640fb430 2024-09-16T00:28:30.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:30.154+0000 7fd24bfff640 1 -- 172.21.15.112:0/569532577 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd25403db10 con 0x7fd2640fb430 2024-09-16T00:28:30.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:30.154+0000 7fd269bb2640 1 -- 172.21.15.112:0/569532577 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd264112dc0 con 0x7fd2640fb430 2024-09-16T00:28:30.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:30.155+0000 7fd269bb2640 1 -- 172.21.15.112:0/569532577 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2640fa890 con 0x7fd2640fb430 2024-09-16T00:28:30.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:30.156+0000 7fd24bfff640 1 -- 172.21.15.112:0/569532577 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 32) v1 ==== 92377+0+0 (secure 0 0 0) 0x7fd25402f030 con 0x7fd2640fb430 2024-09-16T00:28:30.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:30.156+0000 7fd24bfff640 1 --2- 172.21.15.112:0/569532577 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7fd24c06df60 0x7fd24c070410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:30.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:30.156+0000 7fd24bfff640 1 -- 172.21.15.112:0/569532577 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6152+0+0 (secure 0 0 0) 0x7fd2540af050 con 0x7fd2640fb430 2024-09-16T00:28:30.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:30.156+0000 7fd262ffd640 1 --2- 172.21.15.112:0/569532577 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7fd24c06df60 0x7fd24c070410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:30.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:30.157+0000 7fd262ffd640 1 --2- 172.21.15.112:0/569532577 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7fd24c06df60 0x7fd24c070410 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd258004ed0 tx=0x7fd25800b040 comp rx=0 tx=0).ready entity=mgr.14678 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:30.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:30.161+0000 7fd24bfff640 1 -- 172.21.15.112:0/569532577 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185942 (secure 0 0 0) 0x7fd2540788b0 con 0x7fd2640fb430 2024-09-16T00:28:30.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:30 smithi169 ceph-mon[34653]: pgmap v29: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:28:30.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:30 smithi112 ceph-mon[27869]: pgmap v29: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:28:30.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:30.341+0000 7fd269bb2640 1 -- 172.21.15.112:0/569532577 --> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd264100b60 con 0x7fd24c06df60 2024-09-16T00:28:30.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:30.342+0000 7fd24bfff640 1 -- 172.21.15.112:0/569532577 <== mgr.14678 v2:172.21.15.112:6800/1965658616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7fd264100b60 con 0x7fd24c06df60 2024-09-16T00:28:30.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:30.344+0000 7fd269bb2640 1 -- 172.21.15.112:0/569532577 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7fd24c06df60 msgr2=0x7fd24c070410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:30.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:30.344+0000 7fd269bb2640 1 --2- 172.21.15.112:0/569532577 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7fd24c06df60 0x7fd24c070410 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd258004ed0 tx=0x7fd25800b040 comp rx=0 tx=0).stop 2024-09-16T00:28:30.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:30.344+0000 7fd269bb2640 1 -- 172.21.15.112:0/569532577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2640fb430 msgr2=0x7fd26410fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:30.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:30.344+0000 7fd269bb2640 1 --2- 172.21.15.112:0/569532577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2640fb430 0x7fd26410fdc0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd254004a30 tx=0x7fd254004d10 comp rx=0 tx=0).stop 2024-09-16T00:28:30.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:30.345+0000 7fd269bb2640 1 -- 172.21.15.112:0/569532577 shutdown_connections 2024-09-16T00:28:30.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:30.345+0000 7fd269bb2640 1 --2- 172.21.15.112:0/569532577 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7fd24c06df60 0x7fd24c070410 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:30.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:30.345+0000 7fd269bb2640 1 --2- 172.21.15.112:0/569532577 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2640fc700 0x7fd264112310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:30.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:30.345+0000 7fd269bb2640 1 --2- 172.21.15.112:0/569532577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2640fb430 0x7fd26410fdc0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:30.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:30.345+0000 7fd269bb2640 1 -- 172.21.15.112:0/569532577 >> 172.21.15.112:0/569532577 conn(0x7fd2640f6c10 msgr2=0x7fd2640f83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:30.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:30.345+0000 7fd269bb2640 1 -- 172.21.15.112:0/569532577 shutdown_connections 2024-09-16T00:28:30.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:30.345+0000 7fd269bb2640 1 -- 172.21.15.112:0/569532577 wait complete. 2024-09-16T00:28:31.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:31 smithi169 ceph-mon[34653]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:28:31.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:31 smithi112 ceph-mon[27869]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:28:32.015 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -e sha1=8921c8ef947807b717db041d85f0761b7914d5fc -- bash -c 'ceph versions | jq -e '"'"'.mgr | length == 1'"'"'' 2024-09-16T00:28:32.266 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:28:32.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:32 smithi169 ceph-mon[34653]: pgmap v30: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:28:32.314 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:32 smithi112 ceph-mon[27869]: pgmap v30: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:28:33.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:33.946+0000 7f4f5e9a6640 1 -- 172.21.15.112:0/1098415704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f58102ef0 msgr2=0x7f4f580fc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:33.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:33.946+0000 7f4f5e9a6640 1 --2- 172.21.15.112:0/1098415704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f58102ef0 0x7f4f580fc5b0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4f400048d0 tx=0x7f4f40009da0 comp rx=0 tx=0).stop 2024-09-16T00:28:33.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:33.946+0000 7f4f5e9a6640 1 -- 172.21.15.112:0/1098415704 shutdown_connections 2024-09-16T00:28:33.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:33.946+0000 7f4f5e9a6640 1 --2- 172.21.15.112:0/1098415704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f58102ef0 0x7f4f580fc5b0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:33.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:33.946+0000 7f4f5e9a6640 1 --2- 172.21.15.112:0/1098415704 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f58102550 0x7f4f58102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:33.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:33.946+0000 7f4f5e9a6640 1 -- 172.21.15.112:0/1098415704 >> 172.21.15.112:0/1098415704 conn(0x7f4f580f80f0 msgr2=0x7f4f580fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:33.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:33.946+0000 7f4f5e9a6640 1 -- 172.21.15.112:0/1098415704 shutdown_connections 2024-09-16T00:28:33.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:33.946+0000 7f4f5e9a6640 1 -- 172.21.15.112:0/1098415704 wait complete. 2024-09-16T00:28:33.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:33.947+0000 7f4f5e9a6640 1 Processor -- start 2024-09-16T00:28:33.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:33.947+0000 7f4f5e9a6640 1 -- start start 2024-09-16T00:28:33.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:33.947+0000 7f4f5e9a6640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f58102550 0x7f4f58113520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:33.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:33.947+0000 7f4f5e9a6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f58102ef0 0x7f4f58113a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:33.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:33.947+0000 7f4f5e9a6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f581140b0 con 0x7f4f58102ef0 2024-09-16T00:28:33.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:33.947+0000 7f4f5e9a6640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f5810d4c0 con 0x7f4f58102550 2024-09-16T00:28:33.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:33.948+0000 7f4f4ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f58102ef0 0x7f4f58113a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:33.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:33.948+0000 7f4f4ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f58102ef0 0x7f4f58113a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44860/0 (socket says 172.21.15.112:44860) 2024-09-16T00:28:33.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:33.948+0000 7f4f4ffff640 1 -- 172.21.15.112:0/2329350865 learned_addr learned my addr 172.21.15.112:0/2329350865 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:28:33.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:33.948+0000 7f4f57fff640 1 --2- 172.21.15.112:0/2329350865 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f58102550 0x7f4f58113520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:33.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:33.948+0000 7f4f4ffff640 1 -- 172.21.15.112:0/2329350865 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f58102550 msgr2=0x7f4f58113520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:33.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:33.948+0000 7f4f4ffff640 1 --2- 172.21.15.112:0/2329350865 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f58102550 0x7f4f58113520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:33.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:33.948+0000 7f4f4ffff640 1 -- 172.21.15.112:0/2329350865 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f40005040 con 0x7f4f58102ef0 2024-09-16T00:28:33.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:33.948+0000 7f4f57fff640 1 --2- 172.21.15.112:0/2329350865 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f58102550 0x7f4f58113520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:28:33.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:33.948+0000 7f4f4ffff640 1 --2- 172.21.15.112:0/2329350865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f58102ef0 0x7f4f58113a60 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4f400043d0 tx=0x7f4f40009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:33.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:33.948+0000 7f4f557fa640 1 -- 172.21.15.112:0/2329350865 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f40038070 con 0x7f4f58102ef0 2024-09-16T00:28:33.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:33.948+0000 7f4f557fa640 1 -- 172.21.15.112:0/2329350865 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f4f400096e0 con 0x7f4f58102ef0 2024-09-16T00:28:33.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:33.948+0000 7f4f5e9a6640 1 -- 172.21.15.112:0/2329350865 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f5810d740 con 0x7f4f58102ef0 2024-09-16T00:28:33.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:33.948+0000 7f4f557fa640 1 -- 172.21.15.112:0/2329350865 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f4002c8a0 con 0x7f4f58102ef0 2024-09-16T00:28:33.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:33.948+0000 7f4f5e9a6640 1 -- 172.21.15.112:0/2329350865 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f5810dc80 con 0x7f4f58102ef0 2024-09-16T00:28:33.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:33.949+0000 7f4f5e9a6640 1 -- 172.21.15.112:0/2329350865 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f580fdcd0 con 0x7f4f58102ef0 2024-09-16T00:28:33.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:33.950+0000 7f4f557fa640 1 -- 172.21.15.112:0/2329350865 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 32) v1 ==== 92377+0+0 (secure 0 0 0) 0x7f4f40030020 con 0x7f4f58102ef0 2024-09-16T00:28:33.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:33.950+0000 7f4f557fa640 1 --2- 172.21.15.112:0/2329350865 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f4f4406df60 0x7f4f44070410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:33.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:33.950+0000 7f4f557fa640 1 -- 172.21.15.112:0/2329350865 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6152+0+0 (secure 0 0 0) 0x7f4f400ad6c0 con 0x7f4f58102ef0 2024-09-16T00:28:33.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:33.950+0000 7f4f57fff640 1 --2- 172.21.15.112:0/2329350865 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f4f4406df60 0x7f4f44070410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:33.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:33.951+0000 7f4f57fff640 1 --2- 172.21.15.112:0/2329350865 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f4f4406df60 0x7f4f44070410 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4f48007fd0 tx=0x7f4f48009e90 comp rx=0 tx=0).ready entity=mgr.14678 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:33.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:33.955+0000 7f4f557fa640 1 -- 172.21.15.112:0/2329350865 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185942 (secure 0 0 0) 0x7f4f40076ef0 con 0x7f4f58102ef0 2024-09-16T00:28:34.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:34.186+0000 7f4f5e9a6640 1 -- 172.21.15.112:0/2329350865 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4f580fbd50 con 0x7f4f58102ef0 2024-09-16T00:28:34.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:34.187+0000 7f4f557fa640 1 -- 172.21.15.112:0/2329350865 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+747 (secure 0 0 0) 0x7f4f4007a5b0 con 0x7f4f58102ef0 2024-09-16T00:28:34.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:34.189+0000 7f4f5e9a6640 1 -- 172.21.15.112:0/2329350865 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f4f4406df60 msgr2=0x7f4f44070410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:34.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:34.189+0000 7f4f5e9a6640 1 --2- 172.21.15.112:0/2329350865 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f4f4406df60 0x7f4f44070410 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4f48007fd0 tx=0x7f4f48009e90 comp rx=0 tx=0).stop 2024-09-16T00:28:34.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:34.189+0000 7f4f5e9a6640 1 -- 172.21.15.112:0/2329350865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f58102ef0 msgr2=0x7f4f58113a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:34.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:34.189+0000 7f4f5e9a6640 1 --2- 172.21.15.112:0/2329350865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f58102ef0 0x7f4f58113a60 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4f400043d0 tx=0x7f4f40009da0 comp rx=0 tx=0).stop 2024-09-16T00:28:34.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:34.189+0000 7f4f5e9a6640 1 -- 172.21.15.112:0/2329350865 shutdown_connections 2024-09-16T00:28:34.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:34.189+0000 7f4f5e9a6640 1 --2- 172.21.15.112:0/2329350865 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f4f4406df60 0x7f4f44070410 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:34.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:34.189+0000 7f4f5e9a6640 1 --2- 172.21.15.112:0/2329350865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f58102ef0 0x7f4f58113a60 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:34.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:34.189+0000 7f4f5e9a6640 1 --2- 172.21.15.112:0/2329350865 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f58102550 0x7f4f58113520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:34.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:34.189+0000 7f4f5e9a6640 1 -- 172.21.15.112:0/2329350865 >> 172.21.15.112:0/2329350865 conn(0x7f4f580f80f0 msgr2=0x7f4f580f9150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:34.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:34.189+0000 7f4f5e9a6640 1 -- 172.21.15.112:0/2329350865 shutdown_connections 2024-09-16T00:28:34.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:34.189+0000 7f4f5e9a6640 1 -- 172.21.15.112:0/2329350865 wait complete. 2024-09-16T00:28:34.199 INFO:teuthology.orchestra.run.smithi112.stdout:true 2024-09-16T00:28:34.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:34 smithi112 ceph-mon[27869]: pgmap v31: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:28:34.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:34 smithi169 ceph-mon[34653]: pgmap v31: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:28:34.918 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -e sha1=8921c8ef947807b717db041d85f0761b7914d5fc -- bash -c 'ceph versions | jq -e '"'"'.mgr | keys'"'"' | grep $sha1' 2024-09-16T00:28:35.167 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:28:35.287 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:35 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/2329350865' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:28:35.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:35 smithi169 ceph-mon[34653]: from='client.? 172.21.15.112:0/2329350865' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:28:36.262 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:36 smithi112 ceph-mon[27869]: pgmap v32: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:28:36.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:36 smithi169 ceph-mon[34653]: pgmap v32: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:28:36.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:36.871+0000 7f1cdf6a4640 1 -- 172.21.15.112:0/1362311883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1cd8102f20 msgr2=0x7f1cd80fc5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:36.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:36.871+0000 7f1cdf6a4640 1 --2- 172.21.15.112:0/1362311883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1cd8102f20 0x7f1cd80fc5e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc0048d0 tx=0x7f1ccc009da0 comp rx=0 tx=0).stop 2024-09-16T00:28:36.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:36.872+0000 7f1cdf6a4640 1 -- 172.21.15.112:0/1362311883 shutdown_connections 2024-09-16T00:28:36.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:36.872+0000 7f1cdf6a4640 1 --2- 172.21.15.112:0/1362311883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1cd8102f20 0x7f1cd80fc5e0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:36.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:36.872+0000 7f1cdf6a4640 1 --2- 172.21.15.112:0/1362311883 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1cd8102580 0x7f1cd8102950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:36.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:36.872+0000 7f1cdf6a4640 1 -- 172.21.15.112:0/1362311883 >> 172.21.15.112:0/1362311883 conn(0x7f1cd80f81a0 msgr2=0x7f1cd80fa5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:36.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:36.872+0000 7f1cdf6a4640 1 -- 172.21.15.112:0/1362311883 shutdown_connections 2024-09-16T00:28:36.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:36.872+0000 7f1cdf6a4640 1 -- 172.21.15.112:0/1362311883 wait complete. 2024-09-16T00:28:36.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:36.872+0000 7f1cdf6a4640 1 Processor -- start 2024-09-16T00:28:36.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:36.872+0000 7f1cdf6a4640 1 -- start start 2024-09-16T00:28:36.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:36.872+0000 7f1cdf6a4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1cd8102580 0x7f1cd8113270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:36.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:36.872+0000 7f1cdf6a4640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1cd8102f20 0x7f1cd81137b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:36.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:36.872+0000 7f1cdf6a4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd8113e40 con 0x7f1cd8102580 2024-09-16T00:28:36.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:36.872+0000 7f1cdf6a4640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd810d1f0 con 0x7f1cd8102f20 2024-09-16T00:28:36.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:36.872+0000 7f1cdd440640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1cd8102580 0x7f1cd8113270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:36.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:36.872+0000 7f1cdd440640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1cd8102580 0x7f1cd8113270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44890/0 (socket says 172.21.15.112:44890) 2024-09-16T00:28:36.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:36.873+0000 7f1cdd440640 1 -- 172.21.15.112:0/37627021 learned_addr learned my addr 172.21.15.112:0/37627021 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:28:36.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:36.873+0000 7f1cdcc3f640 1 --2- 172.21.15.112:0/37627021 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1cd8102f20 0x7f1cd81137b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:36.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:36.873+0000 7f1cdd440640 1 -- 172.21.15.112:0/37627021 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1cd8102f20 msgr2=0x7f1cd81137b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:36.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:36.873+0000 7f1cdd440640 1 --2- 172.21.15.112:0/37627021 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1cd8102f20 0x7f1cd81137b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:36.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:36.873+0000 7f1cdd440640 1 -- 172.21.15.112:0/37627021 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ccc005040 con 0x7f1cd8102580 2024-09-16T00:28:36.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:36.873+0000 7f1cdcc3f640 1 --2- 172.21.15.112:0/37627021 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1cd8102f20 0x7f1cd81137b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:28:36.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:36.873+0000 7f1cdd440640 1 --2- 172.21.15.112:0/37627021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1cd8102580 0x7f1cd8113270 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1cc800bbb0 tx=0x7f1cc8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:36.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:36.873+0000 7f1cd5ffb640 1 -- 172.21.15.112:0/37627021 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1cc800c9c0 con 0x7f1cd8102580 2024-09-16T00:28:36.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:36.873+0000 7f1cdf6a4640 1 -- 172.21.15.112:0/37627021 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cd810d4d0 con 0x7f1cd8102580 2024-09-16T00:28:36.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:36.873+0000 7f1cd5ffb640 1 -- 172.21.15.112:0/37627021 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f1cc80092e0 con 0x7f1cd8102580 2024-09-16T00:28:36.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:36.873+0000 7f1cd5ffb640 1 -- 172.21.15.112:0/37627021 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1cc8007640 con 0x7f1cd8102580 2024-09-16T00:28:36.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:36.873+0000 7f1cdf6a4640 1 -- 172.21.15.112:0/37627021 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cd810d9f0 con 0x7f1cd8102580 2024-09-16T00:28:36.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:36.874+0000 7f1cdf6a4640 1 -- 172.21.15.112:0/37627021 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cd80fdd00 con 0x7f1cd8102580 2024-09-16T00:28:36.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:36.875+0000 7f1cd5ffb640 1 -- 172.21.15.112:0/37627021 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 32) v1 ==== 92377+0+0 (secure 0 0 0) 0x7f1cc8012070 con 0x7f1cd8102580 2024-09-16T00:28:36.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:36.875+0000 7f1cd5ffb640 1 --2- 172.21.15.112:0/37627021 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f1cc006df60 0x7f1cc0070410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:36.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:36.875+0000 7f1cd5ffb640 1 -- 172.21.15.112:0/37627021 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6152+0+0 (secure 0 0 0) 0x7f1cc808db50 con 0x7f1cd8102580 2024-09-16T00:28:36.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:36.875+0000 7f1cdcc3f640 1 --2- 172.21.15.112:0/37627021 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f1cc006df60 0x7f1cc0070410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:36.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:36.876+0000 7f1cdcc3f640 1 --2- 172.21.15.112:0/37627021 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f1cc006df60 0x7f1cc0070410 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1cd810eb30 tx=0x7f1ccc009da0 comp rx=0 tx=0).ready entity=mgr.14678 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:36.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:36.880+0000 7f1cd5ffb640 1 -- 172.21.15.112:0/37627021 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185942 (secure 0 0 0) 0x7f1cc8057380 con 0x7f1cd8102580 2024-09-16T00:28:37.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:37.113+0000 7f1cdf6a4640 1 -- 172.21.15.112:0/37627021 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1cd804b780 con 0x7f1cd8102580 2024-09-16T00:28:37.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:37.113+0000 7f1cd5ffb640 1 -- 172.21.15.112:0/37627021 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+747 (secure 0 0 0) 0x7f1cc805aa40 con 0x7f1cd8102580 2024-09-16T00:28:37.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:37.115+0000 7f1cdf6a4640 1 -- 172.21.15.112:0/37627021 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f1cc006df60 msgr2=0x7f1cc0070410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:37.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:37.115+0000 7f1cdf6a4640 1 --2- 172.21.15.112:0/37627021 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f1cc006df60 0x7f1cc0070410 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1cd810eb30 tx=0x7f1ccc009da0 comp rx=0 tx=0).stop 2024-09-16T00:28:37.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:37.115+0000 7f1cdf6a4640 1 -- 172.21.15.112:0/37627021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1cd8102580 msgr2=0x7f1cd8113270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:37.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:37.115+0000 7f1cdf6a4640 1 --2- 172.21.15.112:0/37627021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1cd8102580 0x7f1cd8113270 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1cc800bbb0 tx=0x7f1cc8009840 comp rx=0 tx=0).stop 2024-09-16T00:28:37.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:37.115+0000 7f1cdf6a4640 1 -- 172.21.15.112:0/37627021 shutdown_connections 2024-09-16T00:28:37.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:37.115+0000 7f1cdf6a4640 1 --2- 172.21.15.112:0/37627021 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f1cc006df60 0x7f1cc0070410 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:37.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:37.115+0000 7f1cdf6a4640 1 --2- 172.21.15.112:0/37627021 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1cd8102f20 0x7f1cd81137b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:37.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:37.115+0000 7f1cdf6a4640 1 --2- 172.21.15.112:0/37627021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1cd8102580 0x7f1cd8113270 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:37.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:37.115+0000 7f1cdf6a4640 1 -- 172.21.15.112:0/37627021 >> 172.21.15.112:0/37627021 conn(0x7f1cd80f81a0 msgr2=0x7f1cd80f91f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:37.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:37.115+0000 7f1cdf6a4640 1 -- 172.21.15.112:0/37627021 shutdown_connections 2024-09-16T00:28:37.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:37.115+0000 7f1cdf6a4640 1 -- 172.21.15.112:0/37627021 wait complete. 2024-09-16T00:28:37.124 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)" 2024-09-16T00:28:37.775 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -e sha1=8921c8ef947807b717db041d85f0761b7914d5fc -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 2'"'"'' 2024-09-16T00:28:38.022 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:28:38.293 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:38 smithi112 ceph-mon[27869]: pgmap v33: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:28:38.293 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:38 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/37627021' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:28:38.293 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:38 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:28:38.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:38 smithi169 ceph-mon[34653]: pgmap v33: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:28:38.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:38 smithi169 ceph-mon[34653]: from='client.? 172.21.15.112:0/37627021' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:28:38.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:38 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:28:39.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:39.667+0000 7f03149a3640 1 -- 172.21.15.112:0/697307989 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03100fac60 msgr2=0x7f03100fb0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:39.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:39.667+0000 7f03149a3640 1 --2- 172.21.15.112:0/697307989 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03100fac60 0x7f03100fb0b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f03040048d0 tx=0x7f0304009da0 comp rx=0 tx=0).stop 2024-09-16T00:28:39.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:39.667+0000 7f03149a3640 1 -- 172.21.15.112:0/697307989 shutdown_connections 2024-09-16T00:28:39.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:39.667+0000 7f03149a3640 1 --2- 172.21.15.112:0/697307989 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03100fac60 0x7f03100fb0b0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:39.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:39.667+0000 7f03149a3640 1 --2- 172.21.15.112:0/697307989 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03101025f0 0x7f03101029c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:39.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:39.667+0000 7f03149a3640 1 -- 172.21.15.112:0/697307989 >> 172.21.15.112:0/697307989 conn(0x7f0310070c80 msgr2=0x7f0310071080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:39.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:39.667+0000 7f03149a3640 1 -- 172.21.15.112:0/697307989 shutdown_connections 2024-09-16T00:28:39.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:39.667+0000 7f03149a3640 1 -- 172.21.15.112:0/697307989 wait complete. 2024-09-16T00:28:39.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:39.668+0000 7f03149a3640 1 Processor -- start 2024-09-16T00:28:39.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:39.668+0000 7f03149a3640 1 -- start start 2024-09-16T00:28:39.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:39.668+0000 7f03149a3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03100fac60 0x7f03101131f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:39.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:39.668+0000 7f03149a3640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03101025f0 0x7f0310113730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:39.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:39.668+0000 7f03149a3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0310113dc0 con 0x7f03100fac60 2024-09-16T00:28:39.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:39.668+0000 7f03149a3640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f031010d170 con 0x7f03101025f0 2024-09-16T00:28:39.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:39.668+0000 7f030e59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03100fac60 0x7f03101131f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:39.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:39.668+0000 7f030e59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03100fac60 0x7f03101131f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44912/0 (socket says 172.21.15.112:44912) 2024-09-16T00:28:39.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:39.668+0000 7f030e59c640 1 -- 172.21.15.112:0/351937683 learned_addr learned my addr 172.21.15.112:0/351937683 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:28:39.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:39.669+0000 7f030dd9b640 1 --2- 172.21.15.112:0/351937683 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03101025f0 0x7f0310113730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:39.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:39.669+0000 7f030e59c640 1 -- 172.21.15.112:0/351937683 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03101025f0 msgr2=0x7f0310113730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:39.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:39.669+0000 7f030e59c640 1 --2- 172.21.15.112:0/351937683 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03101025f0 0x7f0310113730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:39.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:39.669+0000 7f030e59c640 1 -- 172.21.15.112:0/351937683 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0304005040 con 0x7f03100fac60 2024-09-16T00:28:39.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:39.669+0000 7f030dd9b640 1 --2- 172.21.15.112:0/351937683 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03101025f0 0x7f0310113730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:28:39.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:39.669+0000 7f030e59c640 1 --2- 172.21.15.112:0/351937683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03100fac60 0x7f03101131f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f030000b790 tx=0x7f03000091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:39.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:39.669+0000 7f02feffd640 1 -- 172.21.15.112:0/351937683 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f030000c700 con 0x7f03100fac60 2024-09-16T00:28:39.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:39.669+0000 7f03149a3640 1 -- 172.21.15.112:0/351937683 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f031010d450 con 0x7f03100fac60 2024-09-16T00:28:39.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:39.669+0000 7f02feffd640 1 -- 172.21.15.112:0/351937683 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f030000cd00 con 0x7f03100fac60 2024-09-16T00:28:39.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:39.669+0000 7f02feffd640 1 -- 172.21.15.112:0/351937683 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0300015e70 con 0x7f03100fac60 2024-09-16T00:28:39.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:39.669+0000 7f03149a3640 1 -- 172.21.15.112:0/351937683 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f031010d940 con 0x7f03100fac60 2024-09-16T00:28:39.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:39.670+0000 7f03149a3640 1 -- 172.21.15.112:0/351937683 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0310065810 con 0x7f03100fac60 2024-09-16T00:28:39.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:39.671+0000 7f02feffd640 1 -- 172.21.15.112:0/351937683 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 32) v1 ==== 92377+0+0 (secure 0 0 0) 0x7f0300012070 con 0x7f03100fac60 2024-09-16T00:28:39.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:39.671+0000 7f02feffd640 1 --2- 172.21.15.112:0/351937683 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f02f806df60 0x7f02f8070410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:39.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:39.671+0000 7f02feffd640 1 -- 172.21.15.112:0/351937683 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6152+0+0 (secure 0 0 0) 0x7f030008d590 con 0x7f03100fac60 2024-09-16T00:28:39.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:39.671+0000 7f030dd9b640 1 --2- 172.21.15.112:0/351937683 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f02f806df60 0x7f02f8070410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:39.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:39.672+0000 7f030dd9b640 1 --2- 172.21.15.112:0/351937683 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f02f806df60 0x7f02f8070410 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0310106c70 tx=0x7f0304035040 comp rx=0 tx=0).ready entity=mgr.14678 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:39.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:39.676+0000 7f02feffd640 1 -- 172.21.15.112:0/351937683 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185942 (secure 0 0 0) 0x7f0300056dc0 con 0x7f03100fac60 2024-09-16T00:28:39.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:39.905+0000 7f03149a3640 1 -- 172.21.15.112:0/351937683 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f03100ff7d0 con 0x7f03100fac60 2024-09-16T00:28:39.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:39.906+0000 7f02feffd640 1 -- 172.21.15.112:0/351937683 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+747 (secure 0 0 0) 0x7f030005a480 con 0x7f03100fac60 2024-09-16T00:28:39.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:39.908+0000 7f03149a3640 1 -- 172.21.15.112:0/351937683 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f02f806df60 msgr2=0x7f02f8070410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:39.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:39.908+0000 7f03149a3640 1 --2- 172.21.15.112:0/351937683 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f02f806df60 0x7f02f8070410 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0310106c70 tx=0x7f0304035040 comp rx=0 tx=0).stop 2024-09-16T00:28:39.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:39.908+0000 7f03149a3640 1 -- 172.21.15.112:0/351937683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03100fac60 msgr2=0x7f03101131f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:39.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:39.908+0000 7f03149a3640 1 --2- 172.21.15.112:0/351937683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03100fac60 0x7f03101131f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f030000b790 tx=0x7f03000091d0 comp rx=0 tx=0).stop 2024-09-16T00:28:39.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:39.908+0000 7f03149a3640 1 -- 172.21.15.112:0/351937683 shutdown_connections 2024-09-16T00:28:39.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:39.908+0000 7f03149a3640 1 --2- 172.21.15.112:0/351937683 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f02f806df60 0x7f02f8070410 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:39.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:39.908+0000 7f03149a3640 1 --2- 172.21.15.112:0/351937683 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03101025f0 0x7f0310113730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:39.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:39.908+0000 7f03149a3640 1 --2- 172.21.15.112:0/351937683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03100fac60 0x7f03101131f0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:39.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:39.908+0000 7f03149a3640 1 -- 172.21.15.112:0/351937683 >> 172.21.15.112:0/351937683 conn(0x7f0310070c80 msgr2=0x7f03100f9f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:39.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:39.908+0000 7f03149a3640 1 -- 172.21.15.112:0/351937683 shutdown_connections 2024-09-16T00:28:39.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:39.908+0000 7f03149a3640 1 -- 172.21.15.112:0/351937683 wait complete. 2024-09-16T00:28:39.917 INFO:teuthology.orchestra.run.smithi112.stdout:true 2024-09-16T00:28:40.205 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:40 smithi112 ceph-mon[27869]: pgmap v34: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-16T00:28:40.206 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:40 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/351937683' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:28:40.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:40 smithi169 ceph-mon[34653]: pgmap v34: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-16T00:28:40.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:40 smithi169 ceph-mon[34653]: from='client.? 172.21.15.112:0/351937683' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:28:40.649 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -e sha1=8921c8ef947807b717db041d85f0761b7914d5fc -- bash -c 'ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '"'"'.up_to_date | length == 2'"'"'' 2024-09-16T00:28:40.893 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:28:42.315 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:42 smithi112 ceph-mon[27869]: pgmap v35: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-09-16T00:28:42.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:42.462+0000 7f91129f9640 1 -- 172.21.15.112:0/1677633366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f910c0fcf80 msgr2=0x7f910c0fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:42.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:42.462+0000 7f91129f9640 1 --2- 172.21.15.112:0/1677633366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f910c0fcf80 0x7f910c0fd3d0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f90f40048d0 tx=0x7f90f4009da0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:42.462+0000 7f91129f9640 1 -- 172.21.15.112:0/1677633366 shutdown_connections 2024-09-16T00:28:42.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:42.462+0000 7f91129f9640 1 --2- 172.21.15.112:0/1677633366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f910c0fcf80 0x7f910c0fd3d0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:42.462+0000 7f91129f9640 1 --2- 172.21.15.112:0/1677633366 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f910c0fb810 0x7f910c0fbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:42.462+0000 7f91129f9640 1 -- 172.21.15.112:0/1677633366 >> 172.21.15.112:0/1677633366 conn(0x7f910c0723b0 msgr2=0x7f910c0727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:42.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:42.462+0000 7f91129f9640 1 -- 172.21.15.112:0/1677633366 shutdown_connections 2024-09-16T00:28:42.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:42.462+0000 7f91129f9640 1 -- 172.21.15.112:0/1677633366 wait complete. 2024-09-16T00:28:42.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:42.462+0000 7f91129f9640 1 Processor -- start 2024-09-16T00:28:42.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:42.462+0000 7f91129f9640 1 -- start start 2024-09-16T00:28:42.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:42.463+0000 7f91129f9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f910c0fb810 0x7f910c113170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:42.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:42.463+0000 7f91129f9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f910c0fcf80 0x7f910c1136b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:42.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:42.463+0000 7f91129f9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f910c113cb0 con 0x7f910c0fcf80 2024-09-16T00:28:42.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:42.463+0000 7f91129f9640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f910c10d0f0 con 0x7f910c0fb810 2024-09-16T00:28:42.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:42.463+0000 7f9103fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f910c0fcf80 0x7f910c1136b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:42.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:42.463+0000 7f9103fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f910c0fcf80 0x7f910c1136b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41588/0 (socket says 172.21.15.112:41588) 2024-09-16T00:28:42.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:42.463+0000 7f9103fff640 1 -- 172.21.15.112:0/3431024229 learned_addr learned my addr 172.21.15.112:0/3431024229 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:28:42.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:42.463+0000 7f910bfff640 1 --2- 172.21.15.112:0/3431024229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f910c0fb810 0x7f910c113170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:42.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:42.463+0000 7f9103fff640 1 -- 172.21.15.112:0/3431024229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f910c0fb810 msgr2=0x7f910c113170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:42.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:42.463+0000 7f9103fff640 1 --2- 172.21.15.112:0/3431024229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f910c0fb810 0x7f910c113170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:42.463+0000 7f9103fff640 1 -- 172.21.15.112:0/3431024229 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f4005040 con 0x7f910c0fcf80 2024-09-16T00:28:42.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:42.463+0000 7f9103fff640 1 --2- 172.21.15.112:0/3431024229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f910c0fcf80 0x7f910c1136b0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f90f4000c00 tx=0x7f90f4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:42.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:42.463+0000 7f91097fa640 1 -- 172.21.15.112:0/3431024229 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f90f4038070 con 0x7f910c0fcf80 2024-09-16T00:28:42.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:42.463+0000 7f91097fa640 1 -- 172.21.15.112:0/3431024229 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f90f402ade0 con 0x7f910c0fcf80 2024-09-16T00:28:42.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:42.463+0000 7f91129f9640 1 -- 172.21.15.112:0/3431024229 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f910c10d370 con 0x7f910c0fcf80 2024-09-16T00:28:42.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:42.463+0000 7f91097fa640 1 -- 172.21.15.112:0/3431024229 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f90f402c7e0 con 0x7f910c0fcf80 2024-09-16T00:28:42.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:42.463+0000 7f91129f9640 1 -- 172.21.15.112:0/3431024229 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f910c10d8b0 con 0x7f910c0fcf80 2024-09-16T00:28:42.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:42.464+0000 7f91129f9640 1 -- 172.21.15.112:0/3431024229 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f910c0fcbc0 con 0x7f910c0fcf80 2024-09-16T00:28:42.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:42.465+0000 7f91097fa640 1 -- 172.21.15.112:0/3431024229 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 32) v1 ==== 92377+0+0 (secure 0 0 0) 0x7f90f4030020 con 0x7f910c0fcf80 2024-09-16T00:28:42.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:42.465+0000 7f91097fa640 1 --2- 172.21.15.112:0/3431024229 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f90f806df60 0x7f90f8070410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:42.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:42.465+0000 7f91097fa640 1 -- 172.21.15.112:0/3431024229 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6152+0+0 (secure 0 0 0) 0x7f90f40ad600 con 0x7f910c0fcf80 2024-09-16T00:28:42.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:42.465+0000 7f910bfff640 1 --2- 172.21.15.112:0/3431024229 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f90f806df60 0x7f90f8070410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:42.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:42.466+0000 7f910bfff640 1 --2- 172.21.15.112:0/3431024229 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f90f806df60 0x7f90f8070410 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f90fc007fd0 tx=0x7f90fc009e90 comp rx=0 tx=0).ready entity=mgr.14678 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:42.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:42.470+0000 7f91097fa640 1 -- 172.21.15.112:0/3431024229 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185942 (secure 0 0 0) 0x7f90f4076ee0 con 0x7f910c0fcf80 2024-09-16T00:28:42.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:42 smithi169 ceph-mon[34653]: pgmap v35: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-09-16T00:28:42.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:42.651+0000 7f91129f9640 1 -- 172.21.15.112:0/3431024229 --> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] -- mgr_command(tid 0: {"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc", "target": ["mon-mgr", ""]}) v1 -- 0x7f910c10e880 con 0x7f90f806df60 2024-09-16T00:28:44.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:44 smithi112 ceph-mon[27869]: from='client.14758 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:28:44.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:44 smithi112 ceph-mon[27869]: pgmap v36: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-16T00:28:44.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:44 smithi169 ceph-mon[34653]: from='client.14758 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:28:44.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:44 smithi169 ceph-mon[34653]: pgmap v36: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-16T00:28:44.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:44.556+0000 7f91097fa640 1 -- 172.21.15.112:0/3431024229 <== mgr.14678 v2:172.21.15.112:6800/1965658616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+5108 (secure 0 0 0) 0x7f910c10e880 con 0x7f90f806df60 2024-09-16T00:28:44.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:44.559+0000 7f91129f9640 1 -- 172.21.15.112:0/3431024229 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f90f806df60 msgr2=0x7f90f8070410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:44.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:44.559+0000 7f91129f9640 1 --2- 172.21.15.112:0/3431024229 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f90f806df60 0x7f90f8070410 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f90fc007fd0 tx=0x7f90fc009e90 comp rx=0 tx=0).stop 2024-09-16T00:28:44.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:44.559+0000 7f91129f9640 1 -- 172.21.15.112:0/3431024229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f910c0fcf80 msgr2=0x7f910c1136b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:44.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:44.559+0000 7f91129f9640 1 --2- 172.21.15.112:0/3431024229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f910c0fcf80 0x7f910c1136b0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f90f4000c00 tx=0x7f90f4009da0 comp rx=0 tx=0).stop 2024-09-16T00:28:44.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:44.559+0000 7f91129f9640 1 -- 172.21.15.112:0/3431024229 shutdown_connections 2024-09-16T00:28:44.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:44.559+0000 7f91129f9640 1 --2- 172.21.15.112:0/3431024229 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f90f806df60 0x7f90f8070410 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:44.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:44.559+0000 7f91129f9640 1 --2- 172.21.15.112:0/3431024229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f910c0fcf80 0x7f910c1136b0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:44.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:44.559+0000 7f91129f9640 1 --2- 172.21.15.112:0/3431024229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f910c0fb810 0x7f910c113170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:44.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:44.559+0000 7f91129f9640 1 -- 172.21.15.112:0/3431024229 >> 172.21.15.112:0/3431024229 conn(0x7f910c0723b0 msgr2=0x7f910c065810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:44.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:44.559+0000 7f91129f9640 1 -- 172.21.15.112:0/3431024229 shutdown_connections 2024-09-16T00:28:44.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:44.559+0000 7f91129f9640 1 -- 172.21.15.112:0/3431024229 wait complete. 2024-09-16T00:28:44.568 INFO:teuthology.orchestra.run.smithi112.stdout:true 2024-09-16T00:28:45.223 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -e sha1=8921c8ef947807b717db041d85f0761b7914d5fc -- bash -c 'ceph orch ps' 2024-09-16T00:28:45.469 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:28:45.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:45 smithi169 ceph-mon[34653]: pgmap v37: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-09-16T00:28:45.819 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:45 smithi112 ceph-mon[27869]: pgmap v37: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-09-16T00:28:47.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:47.057+0000 7f921250f640 1 -- 172.21.15.112:0/4149970526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f920c0fcf80 msgr2=0x7f920c0fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:47.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:47.057+0000 7f921250f640 1 --2- 172.21.15.112:0/4149970526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f920c0fcf80 0x7f920c0fd3d0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f92000048d0 tx=0x7f9200009da0 comp rx=0 tx=0).stop 2024-09-16T00:28:47.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:47.057+0000 7f921250f640 1 -- 172.21.15.112:0/4149970526 shutdown_connections 2024-09-16T00:28:47.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:47.057+0000 7f921250f640 1 --2- 172.21.15.112:0/4149970526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f920c0fcf80 0x7f920c0fd3d0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:47.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:47.057+0000 7f921250f640 1 --2- 172.21.15.112:0/4149970526 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f920c0fb810 0x7f920c0fbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:47.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:47.057+0000 7f921250f640 1 -- 172.21.15.112:0/4149970526 >> 172.21.15.112:0/4149970526 conn(0x7f920c0723b0 msgr2=0x7f920c0727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:47.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:47.057+0000 7f921250f640 1 -- 172.21.15.112:0/4149970526 shutdown_connections 2024-09-16T00:28:47.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:47.057+0000 7f921250f640 1 -- 172.21.15.112:0/4149970526 wait complete. 2024-09-16T00:28:47.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:47.058+0000 7f921250f640 1 Processor -- start 2024-09-16T00:28:47.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:47.058+0000 7f921250f640 1 -- start start 2024-09-16T00:28:47.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:47.058+0000 7f921250f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f920c0fb810 0x7f920c113150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:47.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:47.058+0000 7f921250f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f920c0fcf80 0x7f920c113690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:47.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:47.058+0000 7f921250f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f920c113c90 con 0x7f920c0fcf80 2024-09-16T00:28:47.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:47.058+0000 7f921250f640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f920c10d0d0 con 0x7f920c0fb810 2024-09-16T00:28:47.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:47.058+0000 7f920b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f920c0fcf80 0x7f920c113690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:47.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:47.058+0000 7f920b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f920c0fcf80 0x7f920c113690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41604/0 (socket says 172.21.15.112:41604) 2024-09-16T00:28:47.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:47.058+0000 7f920b7fe640 1 -- 172.21.15.112:0/2333161473 learned_addr learned my addr 172.21.15.112:0/2333161473 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:28:47.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:47.058+0000 7f920bfff640 1 --2- 172.21.15.112:0/2333161473 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f920c0fb810 0x7f920c113150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:47.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:47.058+0000 7f920b7fe640 1 -- 172.21.15.112:0/2333161473 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f920c0fb810 msgr2=0x7f920c113150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:47.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:47.058+0000 7f920b7fe640 1 --2- 172.21.15.112:0/2333161473 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f920c0fb810 0x7f920c113150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:47.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:47.058+0000 7f920b7fe640 1 -- 172.21.15.112:0/2333161473 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9200005040 con 0x7f920c0fcf80 2024-09-16T00:28:47.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:47.059+0000 7f920b7fe640 1 --2- 172.21.15.112:0/2333161473 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f920c0fcf80 0x7f920c113690 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9200000c00 tx=0x7f9200009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:47.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:47.059+0000 7f9208ff9640 1 -- 172.21.15.112:0/2333161473 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9200038070 con 0x7f920c0fcf80 2024-09-16T00:28:47.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:47.059+0000 7f921250f640 1 -- 172.21.15.112:0/2333161473 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f920c10d350 con 0x7f920c0fcf80 2024-09-16T00:28:47.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:47.059+0000 7f9208ff9640 1 -- 172.21.15.112:0/2333161473 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f920002ade0 con 0x7f920c0fcf80 2024-09-16T00:28:47.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:47.059+0000 7f9208ff9640 1 -- 172.21.15.112:0/2333161473 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f920002c7e0 con 0x7f920c0fcf80 2024-09-16T00:28:47.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:47.059+0000 7f921250f640 1 -- 172.21.15.112:0/2333161473 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f920c10d890 con 0x7f920c0fcf80 2024-09-16T00:28:47.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:47.060+0000 7f921250f640 1 -- 172.21.15.112:0/2333161473 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f920c0fcbc0 con 0x7f920c0fcf80 2024-09-16T00:28:47.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:47.061+0000 7f9208ff9640 1 -- 172.21.15.112:0/2333161473 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 32) v1 ==== 92377+0+0 (secure 0 0 0) 0x7f9200030020 con 0x7f920c0fcf80 2024-09-16T00:28:47.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:47.061+0000 7f9208ff9640 1 --2- 172.21.15.112:0/2333161473 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f91f406df60 0x7f91f4070410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:47.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:47.061+0000 7f9208ff9640 1 -- 172.21.15.112:0/2333161473 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6152+0+0 (secure 0 0 0) 0x7f92000ad600 con 0x7f920c0fcf80 2024-09-16T00:28:47.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:47.061+0000 7f920bfff640 1 --2- 172.21.15.112:0/2333161473 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f91f406df60 0x7f91f4070410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:47.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:47.062+0000 7f920bfff640 1 --2- 172.21.15.112:0/2333161473 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f91f406df60 0x7f91f4070410 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f91fc004ed0 tx=0x7f91fc00b040 comp rx=0 tx=0).ready entity=mgr.14678 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:47.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:47.066+0000 7f9208ff9640 1 -- 172.21.15.112:0/2333161473 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185942 (secure 0 0 0) 0x7f9200076ee0 con 0x7f920c0fcf80 2024-09-16T00:28:47.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:47.240+0000 7f921250f640 1 -- 172.21.15.112:0/2333161473 --> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f920c10e7c0 con 0x7f91f406df60 2024-09-16T00:28:47.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:47.249+0000 7f9208ff9640 1 -- 172.21.15.112:0/2333161473 <== mgr.14678 v2:172.21.15.112:6800/1965658616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f920c10e7c0 con 0x7f91f406df60 2024-09-16T00:28:47.250 INFO:teuthology.orchestra.run.smithi112.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T00:28:47.250 INFO:teuthology.orchestra.run.smithi112.stdout:alertmanager.smithi112 smithi112 *:9093,9094 running (105s) 24s ago 10m 16.6M - 0.25.0 c8568f914cd2 d85ea0562022 2024-09-16T00:28:47.250 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi112 smithi112 running (10m) 24s ago 10m 6891k - 17.2.7-1633-g6d025bc2 33ed8dba3996 c94220ed2776 2024-09-16T00:28:47.250 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi169 smithi169 running (9m) 46s ago 9m 6899k - 17.2.7-1633-g6d025bc2 33ed8dba3996 d0d3726280a2 2024-09-16T00:28:47.250 INFO:teuthology.orchestra.run.smithi112.stdout:grafana.smithi112 smithi112 *:3000 running (29s) 24s ago 10m 45.2M - 9.4.12 2bacad6d85d8 51f9d2013da2 2024-09-16T00:28:47.250 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.hxfwkh smithi112 running (5m) 24s ago 5m 136M - 17.2.7-1633-g6d025bc2 33ed8dba3996 cf92d9461fc6 2024-09-16T00:28:47.250 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.ithwzj smithi112 running (5m) 24s ago 5m 65.6M - 17.2.7-1633-g6d025bc2 33ed8dba3996 02786c8a83c7 2024-09-16T00:28:47.250 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.gawwgw smithi169 running (5m) 46s ago 5m 114M - 17.2.7-1633-g6d025bc2 33ed8dba3996 dcf3a4dfa879 2024-09-16T00:28:47.250 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.lgeqrj smithi169 running (5m) 46s ago 5m 60.2M - 17.2.7-1633-g6d025bc2 33ed8dba3996 445c14e55831 2024-09-16T00:28:47.250 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi112.xrkkbc smithi112 *:8443,9283,8765 running (80s) 24s ago 11m 499M - 19.1.1-326-g8921c8ef f1455bd0219c a981f7fd2810 2024-09-16T00:28:47.250 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi169.mtrobk smithi169 *:8443,9283,8765 running (50s) 46s ago 9m 69.2M - 19.1.1-326-g8921c8ef f1455bd0219c 4096e5369c2d 2024-09-16T00:28:47.251 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi112 smithi112 running (11m) 24s ago 12m 55.1M 2048M 17.2.7-1633-g6d025bc2 33ed8dba3996 ddaaced54bf8 2024-09-16T00:28:47.251 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi169 smithi169 running (9m) 46s ago 9m 42.9M 2048M 17.2.7-1633-g6d025bc2 33ed8dba3996 6c2289bc3e11 2024-09-16T00:28:47.251 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi112 smithi112 *:9100 running (2m) 24s ago 10m 8451k - 1.5.0 0da6a335fe13 6404d1d601e2 2024-09-16T00:28:47.251 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi169 smithi169 *:9100 running (85s) 46s ago 9m 8094k - 1.5.0 0da6a335fe13 5505baf702a6 2024-09-16T00:28:47.251 INFO:teuthology.orchestra.run.smithi112.stdout:osd.0 smithi112 running (9m) 24s ago 8m 190M 1557M 17.2.7-1633-g6d025bc2 33ed8dba3996 f6b2e146ffdf 2024-09-16T00:28:47.251 INFO:teuthology.orchestra.run.smithi112.stdout:osd.1 smithi112 running (8m) 24s ago 8m 201M 1557M 17.2.7-1633-g6d025bc2 33ed8dba3996 e9bee87bba0f 2024-09-16T00:28:47.251 INFO:teuthology.orchestra.run.smithi112.stdout:osd.2 smithi112 running (8m) 24s ago 8m 158M 1557M 17.2.7-1633-g6d025bc2 33ed8dba3996 100ec4e4852a 2024-09-16T00:28:47.251 INFO:teuthology.orchestra.run.smithi112.stdout:osd.3 smithi169 running (7m) 46s ago 7m 230M 2581M 17.2.7-1633-g6d025bc2 33ed8dba3996 f88463e19f23 2024-09-16T00:28:47.251 INFO:teuthology.orchestra.run.smithi112.stdout:osd.4 smithi169 running (7m) 46s ago 7m 210M 2581M 17.2.7-1633-g6d025bc2 33ed8dba3996 d3e4cdb016b9 2024-09-16T00:28:47.251 INFO:teuthology.orchestra.run.smithi112.stdout:osd.5 smithi169 running (6m) 46s ago 6m 197M 2581M 17.2.7-1633-g6d025bc2 33ed8dba3996 8e7d708d74eb 2024-09-16T00:28:47.251 INFO:teuthology.orchestra.run.smithi112.stdout:prometheus.smithi112 smithi112 *:9095 running (55s) 24s ago 10m 53.2M - 2.43.0 a07b618ecd1d 7a1d42e60d2b 2024-09-16T00:28:47.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:47.251+0000 7f921250f640 1 -- 172.21.15.112:0/2333161473 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f91f406df60 msgr2=0x7f91f4070410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:47.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:47.251+0000 7f921250f640 1 --2- 172.21.15.112:0/2333161473 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f91f406df60 0x7f91f4070410 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f91fc004ed0 tx=0x7f91fc00b040 comp rx=0 tx=0).stop 2024-09-16T00:28:47.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:47.251+0000 7f921250f640 1 -- 172.21.15.112:0/2333161473 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f920c0fcf80 msgr2=0x7f920c113690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:47.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:47.251+0000 7f921250f640 1 --2- 172.21.15.112:0/2333161473 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f920c0fcf80 0x7f920c113690 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9200000c00 tx=0x7f9200009da0 comp rx=0 tx=0).stop 2024-09-16T00:28:47.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:47.251+0000 7f921250f640 1 -- 172.21.15.112:0/2333161473 shutdown_connections 2024-09-16T00:28:47.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:47.251+0000 7f921250f640 1 --2- 172.21.15.112:0/2333161473 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f91f406df60 0x7f91f4070410 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:47.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:47.251+0000 7f921250f640 1 --2- 172.21.15.112:0/2333161473 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f920c0fcf80 0x7f920c113690 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:47.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:47.251+0000 7f921250f640 1 --2- 172.21.15.112:0/2333161473 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f920c0fb810 0x7f920c113150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:47.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:47.251+0000 7f921250f640 1 -- 172.21.15.112:0/2333161473 >> 172.21.15.112:0/2333161473 conn(0x7f920c0723b0 msgr2=0x7f920c101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:47.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:47.251+0000 7f921250f640 1 -- 172.21.15.112:0/2333161473 shutdown_connections 2024-09-16T00:28:47.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:47.251+0000 7f921250f640 1 -- 172.21.15.112:0/2333161473 wait complete. 2024-09-16T00:28:47.906 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-09-16T00:28:47.907 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi112.front.sepia.ceph.com 2024-09-16T00:28:47.907 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -e sha1=8921c8ef947807b717db041d85f0761b7914d5fc -- bash -c 'ceph config set mgr mgr/orchestrator/fail_fs true' 2024-09-16T00:28:48.155 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:28:48.275 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:48 smithi112 ceph-mon[27869]: pgmap v38: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-09-16T00:28:48.275 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:48 smithi112 ceph-mon[27869]: from='client.14762 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:28:48.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:48 smithi169 ceph-mon[34653]: pgmap v38: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-09-16T00:28:48.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:48 smithi169 ceph-mon[34653]: from='client.14762 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:28:49.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:49.774+0000 7f661f98a640 1 -- 172.21.15.112:0/499512986 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6618102ed0 msgr2=0x7f66180fc590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:49.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:49.774+0000 7f661f98a640 1 --2- 172.21.15.112:0/499512986 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6618102ed0 0x7f66180fc590 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f660c0048d0 tx=0x7f660c009da0 comp rx=0 tx=0).stop 2024-09-16T00:28:49.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:49.774+0000 7f661f98a640 1 -- 172.21.15.112:0/499512986 shutdown_connections 2024-09-16T00:28:49.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:49.774+0000 7f661f98a640 1 --2- 172.21.15.112:0/499512986 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6618102ed0 0x7f66180fc590 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:49.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:49.774+0000 7f661f98a640 1 --2- 172.21.15.112:0/499512986 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6618102530 0x7f6618102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:49.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:49.774+0000 7f661f98a640 1 -- 172.21.15.112:0/499512986 >> 172.21.15.112:0/499512986 conn(0x7f66180f80f0 msgr2=0x7f66180fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:49.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:49.774+0000 7f661f98a640 1 -- 172.21.15.112:0/499512986 shutdown_connections 2024-09-16T00:28:49.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:49.774+0000 7f661f98a640 1 -- 172.21.15.112:0/499512986 wait complete. 2024-09-16T00:28:49.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:49.774+0000 7f661f98a640 1 Processor -- start 2024-09-16T00:28:49.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:49.774+0000 7f661f98a640 1 -- start start 2024-09-16T00:28:49.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:49.775+0000 7f661f98a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6618102530 0x7f66181131f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:49.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:49.775+0000 7f661f98a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6618102ed0 0x7f6618113730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:49.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:49.775+0000 7f661f98a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6618113d30 con 0x7f6618102ed0 2024-09-16T00:28:49.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:49.775+0000 7f661f98a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f661810d170 con 0x7f6618102530 2024-09-16T00:28:49.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:49.775+0000 7f661cf25640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6618102ed0 0x7f6618113730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:49.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:49.775+0000 7f661cf25640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6618102ed0 0x7f6618113730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41612/0 (socket says 172.21.15.112:41612) 2024-09-16T00:28:49.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:49.775+0000 7f661cf25640 1 -- 172.21.15.112:0/1734776855 learned_addr learned my addr 172.21.15.112:0/1734776855 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:28:49.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:49.775+0000 7f661d726640 1 --2- 172.21.15.112:0/1734776855 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6618102530 0x7f66181131f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:49.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:49.775+0000 7f661cf25640 1 -- 172.21.15.112:0/1734776855 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6618102530 msgr2=0x7f66181131f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:49.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:49.775+0000 7f661cf25640 1 --2- 172.21.15.112:0/1734776855 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6618102530 0x7f66181131f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:49.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:49.775+0000 7f661cf25640 1 -- 172.21.15.112:0/1734776855 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f660c005040 con 0x7f6618102ed0 2024-09-16T00:28:49.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:49.775+0000 7f661d726640 1 --2- 172.21.15.112:0/1734776855 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6618102530 0x7f66181131f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:28:49.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:49.775+0000 7f661cf25640 1 --2- 172.21.15.112:0/1734776855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6618102ed0 0x7f6618113730 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f660c0043d0 tx=0x7f660c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:49.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:49.775+0000 7f6615ffb640 1 -- 172.21.15.112:0/1734776855 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f660c038070 con 0x7f6618102ed0 2024-09-16T00:28:49.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:49.775+0000 7f6615ffb640 1 -- 172.21.15.112:0/1734776855 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f660c0096e0 con 0x7f6618102ed0 2024-09-16T00:28:49.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:49.775+0000 7f661f98a640 1 -- 172.21.15.112:0/1734776855 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f661810d3f0 con 0x7f6618102ed0 2024-09-16T00:28:49.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:49.776+0000 7f6615ffb640 1 -- 172.21.15.112:0/1734776855 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f660c02c8a0 con 0x7f6618102ed0 2024-09-16T00:28:49.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:49.776+0000 7f661f98a640 1 -- 172.21.15.112:0/1734776855 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f661810d930 con 0x7f6618102ed0 2024-09-16T00:28:49.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:49.777+0000 7f661f98a640 1 -- 172.21.15.112:0/1734776855 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66180fdcb0 con 0x7f6618102ed0 2024-09-16T00:28:49.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:49.777+0000 7f6615ffb640 1 -- 172.21.15.112:0/1734776855 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 32) v1 ==== 92377+0+0 (secure 0 0 0) 0x7f660c030020 con 0x7f6618102ed0 2024-09-16T00:28:49.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:49.777+0000 7f6615ffb640 1 --2- 172.21.15.112:0/1734776855 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f660006df60 0x7f6600070410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:49.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:49.778+0000 7f6615ffb640 1 -- 172.21.15.112:0/1734776855 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6152+0+0 (secure 0 0 0) 0x7f660c0ad6c0 con 0x7f6618102ed0 2024-09-16T00:28:49.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:49.778+0000 7f661d726640 1 --2- 172.21.15.112:0/1734776855 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f660006df60 0x7f6600070410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:49.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:49.778+0000 7f661d726640 1 --2- 172.21.15.112:0/1734776855 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f660006df60 0x7f6600070410 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6608007fc0 tx=0x7f6608004d00 comp rx=0 tx=0).ready entity=mgr.14678 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:49.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:49.782+0000 7f6615ffb640 1 -- 172.21.15.112:0/1734776855 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185942 (secure 0 0 0) 0x7f660c076ef0 con 0x7f6618102ed0 2024-09-16T00:28:49.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:49.946+0000 7f661f98a640 1 -- 172.21.15.112:0/1734776855 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mgr/orchestrator/fail_fs}] v 0) v1 -- 0x7f661810e860 con 0x7f6618102ed0 2024-09-16T00:28:49.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:49.950+0000 7f6615ffb640 1 -- 172.21.15.112:0/1734776855 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/orchestrator/fail_fs}]=0 v47)=0 v47) v1 ==== 125+0+0 (secure 0 0 0) 0x7f660c07a5b0 con 0x7f6618102ed0 2024-09-16T00:28:49.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:49.953+0000 7f661f98a640 1 -- 172.21.15.112:0/1734776855 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f660006df60 msgr2=0x7f6600070410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:49.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:49.953+0000 7f661f98a640 1 --2- 172.21.15.112:0/1734776855 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f660006df60 0x7f6600070410 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6608007fc0 tx=0x7f6608004d00 comp rx=0 tx=0).stop 2024-09-16T00:28:49.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:49.953+0000 7f661f98a640 1 -- 172.21.15.112:0/1734776855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6618102ed0 msgr2=0x7f6618113730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:49.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:49.953+0000 7f661f98a640 1 --2- 172.21.15.112:0/1734776855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6618102ed0 0x7f6618113730 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f660c0043d0 tx=0x7f660c009da0 comp rx=0 tx=0).stop 2024-09-16T00:28:49.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:49.953+0000 7f661f98a640 1 -- 172.21.15.112:0/1734776855 shutdown_connections 2024-09-16T00:28:49.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:49.953+0000 7f661f98a640 1 --2- 172.21.15.112:0/1734776855 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f660006df60 0x7f6600070410 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:49.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:49.953+0000 7f661f98a640 1 --2- 172.21.15.112:0/1734776855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6618102ed0 0x7f6618113730 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:49.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:49.953+0000 7f661f98a640 1 --2- 172.21.15.112:0/1734776855 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6618102530 0x7f66181131f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:49.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:49.953+0000 7f661f98a640 1 -- 172.21.15.112:0/1734776855 >> 172.21.15.112:0/1734776855 conn(0x7f66180f80f0 msgr2=0x7f66180f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:49.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:49.953+0000 7f661f98a640 1 -- 172.21.15.112:0/1734776855 shutdown_connections 2024-09-16T00:28:49.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:49.953+0000 7f661f98a640 1 -- 172.21.15.112:0/1734776855 wait complete. 2024-09-16T00:28:50.247 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:50 smithi112 ceph-mon[27869]: pgmap v39: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-16T00:28:50.247 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:50 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/1734776855' entity='client.admin' 2024-09-16T00:28:50.247 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:50 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:28:50.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:50 smithi169 ceph-mon[34653]: pgmap v39: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-16T00:28:50.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:50 smithi169 ceph-mon[34653]: from='client.? 172.21.15.112:0/1734776855' entity='client.admin' 2024-09-16T00:28:50.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:50 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:28:50.613 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-09-16T00:28:50.614 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi112.front.sepia.ceph.com 2024-09-16T00:28:50.614 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -e sha1=8921c8ef947807b717db041d85f0761b7914d5fc -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-09-16T00:28:50.860 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:28:51.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:51 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:28:51.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:51 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:28:51.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:51 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:28:51.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:51 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:28:51.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:51 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:28:51.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:51 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:28:51.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:51 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:28:51.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:51 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:28:51.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:51 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:28:51.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:51 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:28:52.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:52.475+0000 7ffbf9b3b640 1 -- 172.21.15.112:0/4009594520 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffbf40fcf80 msgr2=0x7ffbf40fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:52.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:52.475+0000 7ffbf9b3b640 1 --2- 172.21.15.112:0/4009594520 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffbf40fcf80 0x7ffbf40fd3d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ffbe80048d0 tx=0x7ffbe8009da0 comp rx=0 tx=0).stop 2024-09-16T00:28:52.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:52.475+0000 7ffbf9b3b640 1 -- 172.21.15.112:0/4009594520 shutdown_connections 2024-09-16T00:28:52.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:52.475+0000 7ffbf9b3b640 1 --2- 172.21.15.112:0/4009594520 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffbf40fcf80 0x7ffbf40fd3d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:52.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:52.475+0000 7ffbf9b3b640 1 --2- 172.21.15.112:0/4009594520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbf40fb810 0x7ffbf40fbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:52.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:52.475+0000 7ffbf9b3b640 1 -- 172.21.15.112:0/4009594520 >> 172.21.15.112:0/4009594520 conn(0x7ffbf40723b0 msgr2=0x7ffbf40727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:52.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:52.475+0000 7ffbf9b3b640 1 -- 172.21.15.112:0/4009594520 shutdown_connections 2024-09-16T00:28:52.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:52.475+0000 7ffbf9b3b640 1 -- 172.21.15.112:0/4009594520 wait complete. 2024-09-16T00:28:52.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:52.475+0000 7ffbf9b3b640 1 Processor -- start 2024-09-16T00:28:52.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:52.476+0000 7ffbf9b3b640 1 -- start start 2024-09-16T00:28:52.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:52.476+0000 7ffbf9b3b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbf40fb810 0x7ffbf410bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:52.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:52.476+0000 7ffbf9b3b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffbf40fcf80 0x7ffbf410c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:52.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:52.476+0000 7ffbf9b3b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbf410ca00 con 0x7ffbf40fb810 2024-09-16T00:28:52.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:52.476+0000 7ffbf9b3b640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbf410cb40 con 0x7ffbf40fcf80 2024-09-16T00:28:52.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:52.476+0000 7ffbf37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbf40fb810 0x7ffbf410bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:52.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:52.476+0000 7ffbf37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbf40fb810 0x7ffbf410bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58328/0 (socket says 172.21.15.112:58328) 2024-09-16T00:28:52.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:52.476+0000 7ffbf37fe640 1 -- 172.21.15.112:0/1228068866 learned_addr learned my addr 172.21.15.112:0/1228068866 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:28:52.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:52.476+0000 7ffbf37fe640 1 -- 172.21.15.112:0/1228068866 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffbf40fcf80 msgr2=0x7ffbf410c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:52.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:52.476+0000 7ffbf2ffd640 1 --2- 172.21.15.112:0/1228068866 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffbf40fcf80 0x7ffbf410c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:52.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:52.476+0000 7ffbf37fe640 1 --2- 172.21.15.112:0/1228068866 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffbf40fcf80 0x7ffbf410c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:52.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:52.476+0000 7ffbf37fe640 1 -- 172.21.15.112:0/1228068866 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbe8005040 con 0x7ffbf40fb810 2024-09-16T00:28:52.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:52.476+0000 7ffbf37fe640 1 --2- 172.21.15.112:0/1228068866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbf40fb810 0x7ffbf410bf80 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ffbe400eb70 tx=0x7ffbe400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:52.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:52.477+0000 7ffbdbfff640 1 -- 172.21.15.112:0/1228068866 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffbe4007c30 con 0x7ffbf40fb810 2024-09-16T00:28:52.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:52.477+0000 7ffbdbfff640 1 -- 172.21.15.112:0/1228068866 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7ffbe400f040 con 0x7ffbf40fb810 2024-09-16T00:28:52.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:52.477+0000 7ffbdbfff640 1 -- 172.21.15.112:0/1228068866 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffbe400ca60 con 0x7ffbf40fb810 2024-09-16T00:28:52.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:52.477+0000 7ffbf9b3b640 1 -- 172.21.15.112:0/1228068866 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbf410cdc0 con 0x7ffbf40fb810 2024-09-16T00:28:52.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:52.477+0000 7ffbf9b3b640 1 -- 172.21.15.112:0/1228068866 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbf4119920 con 0x7ffbf40fb810 2024-09-16T00:28:52.479 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:52 smithi112 ceph-mon[27869]: pgmap v40: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:28:52.480 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:52 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:28:52.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:52.478+0000 7ffbf9b3b640 1 -- 172.21.15.112:0/1228068866 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbf40fcbc0 con 0x7ffbf40fb810 2024-09-16T00:28:52.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:52.479+0000 7ffbdbfff640 1 -- 172.21.15.112:0/1228068866 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 32) v1 ==== 92377+0+0 (secure 0 0 0) 0x7ffbe400cbc0 con 0x7ffbf40fb810 2024-09-16T00:28:52.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:52.480+0000 7ffbdbfff640 1 --2- 172.21.15.112:0/1228068866 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7ffbdc06df60 0x7ffbdc070410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:52.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:52.480+0000 7ffbdbfff640 1 -- 172.21.15.112:0/1228068866 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6152+0+0 (secure 0 0 0) 0x7ffbe408d620 con 0x7ffbf40fb810 2024-09-16T00:28:52.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:52.480+0000 7ffbf2ffd640 1 --2- 172.21.15.112:0/1228068866 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7ffbdc06df60 0x7ffbdc070410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:52.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:52.480+0000 7ffbf2ffd640 1 --2- 172.21.15.112:0/1228068866 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7ffbdc06df60 0x7ffbdc070410 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ffbe8007ac0 tx=0x7ffbe8009da0 comp rx=0 tx=0).ready entity=mgr.14678 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:52.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:52.484+0000 7ffbdbfff640 1 -- 172.21.15.112:0/1228068866 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185942 (secure 0 0 0) 0x7ffbe4056dc0 con 0x7ffbf40fb810 2024-09-16T00:28:52.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:52 smithi169 ceph-mon[34653]: pgmap v40: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:28:52.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:52 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:28:52.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:52.651+0000 7ffbf9b3b640 1 -- 172.21.15.112:0/1228068866 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7ffbf4065730 con 0x7ffbf40fb810 2024-09-16T00:28:52.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:52.651+0000 7ffbdbfff640 1 -- 172.21.15.112:0/1228068866 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v47)=0 v47) v1 ==== 155+0+0 (secure 0 0 0) 0x7ffbe405a480 con 0x7ffbf40fb810 2024-09-16T00:28:52.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:52.653+0000 7ffbf9b3b640 1 -- 172.21.15.112:0/1228068866 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7ffbdc06df60 msgr2=0x7ffbdc070410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:52.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:52.653+0000 7ffbf9b3b640 1 --2- 172.21.15.112:0/1228068866 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7ffbdc06df60 0x7ffbdc070410 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ffbe8007ac0 tx=0x7ffbe8009da0 comp rx=0 tx=0).stop 2024-09-16T00:28:52.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:52.653+0000 7ffbf9b3b640 1 -- 172.21.15.112:0/1228068866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbf40fb810 msgr2=0x7ffbf410bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:52.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:52.653+0000 7ffbf9b3b640 1 --2- 172.21.15.112:0/1228068866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbf40fb810 0x7ffbf410bf80 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ffbe400eb70 tx=0x7ffbe400b040 comp rx=0 tx=0).stop 2024-09-16T00:28:52.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:52.653+0000 7ffbf9b3b640 1 -- 172.21.15.112:0/1228068866 shutdown_connections 2024-09-16T00:28:52.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:52.653+0000 7ffbf9b3b640 1 --2- 172.21.15.112:0/1228068866 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7ffbdc06df60 0x7ffbdc070410 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:52.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:52.653+0000 7ffbf9b3b640 1 --2- 172.21.15.112:0/1228068866 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffbf40fcf80 0x7ffbf410c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:52.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:52.653+0000 7ffbf9b3b640 1 --2- 172.21.15.112:0/1228068866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbf40fb810 0x7ffbf410bf80 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:52.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:52.653+0000 7ffbf9b3b640 1 -- 172.21.15.112:0/1228068866 >> 172.21.15.112:0/1228068866 conn(0x7ffbf40723b0 msgr2=0x7ffbf40fa480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:52.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:52.653+0000 7ffbf9b3b640 1 -- 172.21.15.112:0/1228068866 shutdown_connections 2024-09-16T00:28:52.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:52.653+0000 7ffbf9b3b640 1 -- 172.21.15.112:0/1228068866 wait complete. 2024-09-16T00:28:53.348 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -e sha1=8921c8ef947807b717db041d85f0761b7914d5fc -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-09-16T00:28:53.608 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:28:54.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:54 smithi169 ceph-mon[34653]: pgmap v41: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:28:54.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:54 smithi112 ceph-mon[27869]: pgmap v41: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:28:55.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:55.357+0000 7f25700c1640 1 -- 172.21.15.112:0/3019451008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25680fcf80 msgr2=0x7f25680fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:55.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:55.357+0000 7f25700c1640 1 --2- 172.21.15.112:0/3019451008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25680fcf80 0x7f25680fd3d0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f25580048d0 tx=0x7f2558009da0 comp rx=0 tx=0).stop 2024-09-16T00:28:55.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:55.357+0000 7f25700c1640 1 -- 172.21.15.112:0/3019451008 shutdown_connections 2024-09-16T00:28:55.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:55.357+0000 7f25700c1640 1 --2- 172.21.15.112:0/3019451008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25680fcf80 0x7f25680fd3d0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:55.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:55.357+0000 7f25700c1640 1 --2- 172.21.15.112:0/3019451008 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25680fb810 0x7f25680fbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:55.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:55.357+0000 7f25700c1640 1 -- 172.21.15.112:0/3019451008 >> 172.21.15.112:0/3019451008 conn(0x7f25680723b0 msgr2=0x7f25680727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:55.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:55.357+0000 7f25700c1640 1 -- 172.21.15.112:0/3019451008 shutdown_connections 2024-09-16T00:28:55.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:55.357+0000 7f25700c1640 1 -- 172.21.15.112:0/3019451008 wait complete. 2024-09-16T00:28:55.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:55.357+0000 7f25700c1640 1 Processor -- start 2024-09-16T00:28:55.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:55.357+0000 7f25700c1640 1 -- start start 2024-09-16T00:28:55.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:55.358+0000 7f25700c1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25680fb810 0x7f256810e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:55.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:55.358+0000 7f25700c1640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25680fcf80 0x7f256810e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:55.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:55.358+0000 7f25700c1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f256810ee80 con 0x7f25680fb810 2024-09-16T00:28:55.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:55.358+0000 7f25700c1640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f256810eff0 con 0x7f25680fcf80 2024-09-16T00:28:55.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:55.358+0000 7f256de5d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25680fb810 0x7f256810e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:55.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:55.358+0000 7f256de5d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25680fb810 0x7f256810e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58358/0 (socket says 172.21.15.112:58358) 2024-09-16T00:28:55.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:55.358+0000 7f256de5d640 1 -- 172.21.15.112:0/945535562 learned_addr learned my addr 172.21.15.112:0/945535562 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:28:55.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:55.358+0000 7f256d65c640 1 --2- 172.21.15.112:0/945535562 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25680fcf80 0x7f256810e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:55.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:55.358+0000 7f256de5d640 1 -- 172.21.15.112:0/945535562 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25680fcf80 msgr2=0x7f256810e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:55.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:55.358+0000 7f256de5d640 1 --2- 172.21.15.112:0/945535562 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25680fcf80 0x7f256810e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:55.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:55.358+0000 7f256de5d640 1 -- 172.21.15.112:0/945535562 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2558005040 con 0x7f25680fb810 2024-09-16T00:28:55.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:55.358+0000 7f256d65c640 1 --2- 172.21.15.112:0/945535562 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25680fcf80 0x7f256810e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:28:55.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:55.358+0000 7f256de5d640 1 --2- 172.21.15.112:0/945535562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25680fb810 0x7f256810e240 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f256400b790 tx=0x7f25640091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:55.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:55.358+0000 7f255e7fc640 1 -- 172.21.15.112:0/945535562 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f256400c700 con 0x7f25680fb810 2024-09-16T00:28:55.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:55.358+0000 7f255e7fc640 1 -- 172.21.15.112:0/945535562 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f256400cd00 con 0x7f25680fb810 2024-09-16T00:28:55.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:55.358+0000 7f25700c1640 1 -- 172.21.15.112:0/945535562 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2568117370 con 0x7f25680fb810 2024-09-16T00:28:55.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:55.358+0000 7f255e7fc640 1 -- 172.21.15.112:0/945535562 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2564015e70 con 0x7f25680fb810 2024-09-16T00:28:55.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:55.358+0000 7f25700c1640 1 -- 172.21.15.112:0/945535562 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2568117860 con 0x7f25680fb810 2024-09-16T00:28:55.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:55.360+0000 7f25700c1640 1 -- 172.21.15.112:0/945535562 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25680fcbc0 con 0x7f25680fb810 2024-09-16T00:28:55.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:55.360+0000 7f255e7fc640 1 -- 172.21.15.112:0/945535562 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 32) v1 ==== 92377+0+0 (secure 0 0 0) 0x7f2564012070 con 0x7f25680fb810 2024-09-16T00:28:55.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:55.360+0000 7f255e7fc640 1 --2- 172.21.15.112:0/945535562 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f255406df60 0x7f2554070410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:55.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:55.360+0000 7f255e7fc640 1 -- 172.21.15.112:0/945535562 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6152+0+0 (secure 0 0 0) 0x7f256408d520 con 0x7f25680fb810 2024-09-16T00:28:55.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:55.361+0000 7f256d65c640 1 --2- 172.21.15.112:0/945535562 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f255406df60 0x7f2554070410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:55.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:55.361+0000 7f256d65c640 1 --2- 172.21.15.112:0/945535562 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f255406df60 0x7f2554070410 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2568065980 tx=0x7f2558035040 comp rx=0 tx=0).ready entity=mgr.14678 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:55.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:55.365+0000 7f255e7fc640 1 -- 172.21.15.112:0/945535562 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185942 (secure 0 0 0) 0x7f2564056d50 con 0x7f25680fb810 2024-09-16T00:28:55.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:55.529+0000 7f25700c1640 1 -- 172.21.15.112:0/945535562 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7f2568072ec0 con 0x7f25680fb810 2024-09-16T00:28:55.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:55.530+0000 7f255e7fc640 1 -- 172.21.15.112:0/945535562 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v47)=0 v47) v1 ==== 163+0+0 (secure 0 0 0) 0x7f256405a410 con 0x7f25680fb810 2024-09-16T00:28:55.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:55.532+0000 7f25700c1640 1 -- 172.21.15.112:0/945535562 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f255406df60 msgr2=0x7f2554070410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:55.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:55.532+0000 7f25700c1640 1 --2- 172.21.15.112:0/945535562 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f255406df60 0x7f2554070410 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2568065980 tx=0x7f2558035040 comp rx=0 tx=0).stop 2024-09-16T00:28:55.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:55.532+0000 7f25700c1640 1 -- 172.21.15.112:0/945535562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25680fb810 msgr2=0x7f256810e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:55.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:55.532+0000 7f25700c1640 1 --2- 172.21.15.112:0/945535562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25680fb810 0x7f256810e240 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f256400b790 tx=0x7f25640091d0 comp rx=0 tx=0).stop 2024-09-16T00:28:55.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:55.533+0000 7f25700c1640 1 -- 172.21.15.112:0/945535562 shutdown_connections 2024-09-16T00:28:55.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:55.533+0000 7f25700c1640 1 --2- 172.21.15.112:0/945535562 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f255406df60 0x7f2554070410 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:55.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:55.533+0000 7f25700c1640 1 --2- 172.21.15.112:0/945535562 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25680fcf80 0x7f256810e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:55.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:55.533+0000 7f25700c1640 1 --2- 172.21.15.112:0/945535562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25680fb810 0x7f256810e240 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:55.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:55.533+0000 7f25700c1640 1 -- 172.21.15.112:0/945535562 >> 172.21.15.112:0/945535562 conn(0x7f25680723b0 msgr2=0x7f25680fa480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:55.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:55.533+0000 7f25700c1640 1 -- 172.21.15.112:0/945535562 shutdown_connections 2024-09-16T00:28:55.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:55.533+0000 7f25700c1640 1 -- 172.21.15.112:0/945535562 wait complete. 2024-09-16T00:28:56.256 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -e sha1=8921c8ef947807b717db041d85f0761b7914d5fc -- bash -c 'ceph config set global log_to_journald false --force' 2024-09-16T00:28:56.502 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:28:56.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:56 smithi112 ceph-mon[27869]: pgmap v42: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:28:56.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:56 smithi169 ceph-mon[34653]: pgmap v42: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:28:58.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:58.131+0000 7f2f10159640 1 -- 172.21.15.112:0/2935855375 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f080fb490 msgr2=0x7f2f080fb900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:58.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:58.131+0000 7f2f10159640 1 --2- 172.21.15.112:0/2935855375 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f080fb490 0x7f2f080fb900 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2ef80048d0 tx=0x7f2ef8009da0 comp rx=0 tx=0).stop 2024-09-16T00:28:58.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:58.131+0000 7f2f10159640 1 -- 172.21.15.112:0/2935855375 shutdown_connections 2024-09-16T00:28:58.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:58.131+0000 7f2f10159640 1 --2- 172.21.15.112:0/2935855375 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f080fb490 0x7f2f080fb900 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:58.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:58.131+0000 7f2f10159640 1 --2- 172.21.15.112:0/2935855375 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f081014d0 0x7f2f081018a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:58.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:58.131+0000 7f2f10159640 1 -- 172.21.15.112:0/2935855375 >> 172.21.15.112:0/2935855375 conn(0x7f2f080f7010 msgr2=0x7f2f080f9420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:58.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:58.131+0000 7f2f10159640 1 -- 172.21.15.112:0/2935855375 shutdown_connections 2024-09-16T00:28:58.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:58.131+0000 7f2f10159640 1 -- 172.21.15.112:0/2935855375 wait complete. 2024-09-16T00:28:58.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:58.131+0000 7f2f10159640 1 Processor -- start 2024-09-16T00:28:58.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:58.131+0000 7f2f10159640 1 -- start start 2024-09-16T00:28:58.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:58.132+0000 7f2f10159640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f080fb490 0x7f2f081154a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:58.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:58.132+0000 7f2f10159640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f081014d0 0x7f2f081159e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:58.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:58.132+0000 7f2f10159640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f081160c0 con 0x7f2f081014d0 2024-09-16T00:28:58.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:58.132+0000 7f2f10159640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f0810f440 con 0x7f2f080fb490 2024-09-16T00:28:58.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:58.132+0000 7f2f0d6f4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f081014d0 0x7f2f081159e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:58.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:58.132+0000 7f2f0d6f4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f081014d0 0x7f2f081159e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58380/0 (socket says 172.21.15.112:58380) 2024-09-16T00:28:58.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:58.132+0000 7f2f0d6f4640 1 -- 172.21.15.112:0/1122003963 learned_addr learned my addr 172.21.15.112:0/1122003963 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:28:58.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:58.132+0000 7f2f0def5640 1 --2- 172.21.15.112:0/1122003963 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f080fb490 0x7f2f081154a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:58.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:58.132+0000 7f2f0d6f4640 1 -- 172.21.15.112:0/1122003963 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f080fb490 msgr2=0x7f2f081154a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:58.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:58.132+0000 7f2f0d6f4640 1 --2- 172.21.15.112:0/1122003963 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f080fb490 0x7f2f081154a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:58.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:58.132+0000 7f2f0d6f4640 1 -- 172.21.15.112:0/1122003963 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ef8005040 con 0x7f2f081014d0 2024-09-16T00:28:58.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:58.132+0000 7f2f0def5640 1 --2- 172.21.15.112:0/1122003963 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f080fb490 0x7f2f081154a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:28:58.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:58.132+0000 7f2f0d6f4640 1 --2- 172.21.15.112:0/1122003963 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f081014d0 0x7f2f081159e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8003080 tx=0x7f2ef8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:58.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:58.132+0000 7f2efe7fc640 1 -- 172.21.15.112:0/1122003963 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ef8038070 con 0x7f2f081014d0 2024-09-16T00:28:58.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:58.132+0000 7f2efe7fc640 1 -- 172.21.15.112:0/1122003963 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f2ef80097e0 con 0x7f2f081014d0 2024-09-16T00:28:58.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:58.132+0000 7f2f10159640 1 -- 172.21.15.112:0/1122003963 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f0810f670 con 0x7f2f081014d0 2024-09-16T00:28:58.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:58.132+0000 7f2efe7fc640 1 -- 172.21.15.112:0/1122003963 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ef803d420 con 0x7f2f081014d0 2024-09-16T00:28:58.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:58.132+0000 7f2f10159640 1 -- 172.21.15.112:0/1122003963 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f0810fb30 con 0x7f2f081014d0 2024-09-16T00:28:58.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:58.134+0000 7f2f10159640 1 -- 172.21.15.112:0/1122003963 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f080fcbd0 con 0x7f2f081014d0 2024-09-16T00:28:58.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:58.134+0000 7f2efe7fc640 1 -- 172.21.15.112:0/1122003963 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 32) v1 ==== 92377+0+0 (secure 0 0 0) 0x7f2ef8030020 con 0x7f2f081014d0 2024-09-16T00:28:58.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:58.134+0000 7f2efe7fc640 1 --2- 172.21.15.112:0/1122003963 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f2ef406df60 0x7f2ef4070410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:58.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:58.135+0000 7f2f0def5640 1 --2- 172.21.15.112:0/1122003963 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f2ef406df60 0x7f2ef4070410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:58.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:58.135+0000 7f2efe7fc640 1 -- 172.21.15.112:0/1122003963 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6152+0+0 (secure 0 0 0) 0x7f2ef80ade00 con 0x7f2f081014d0 2024-09-16T00:28:58.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:58.135+0000 7f2f0def5640 1 --2- 172.21.15.112:0/1122003963 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f2ef406df60 0x7f2ef4070410 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2f04009a80 tx=0x7f2f04008040 comp rx=0 tx=0).ready entity=mgr.14678 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:58.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:58.139+0000 7f2efe7fc640 1 -- 172.21.15.112:0/1122003963 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185942 (secure 0 0 0) 0x7f2ef80776b0 con 0x7f2f081014d0 2024-09-16T00:28:58.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:58.304+0000 7f2f10159640 1 -- 172.21.15.112:0/1122003963 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7f2f08108540 con 0x7f2f081014d0 2024-09-16T00:28:58.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:58.305+0000 7f2efe7fc640 1 -- 172.21.15.112:0/1122003963 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v47)=0 v47) v1 ==== 135+0+0 (secure 0 0 0) 0x7f2ef807ad70 con 0x7f2f081014d0 2024-09-16T00:28:58.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:58.307+0000 7f2f10159640 1 -- 172.21.15.112:0/1122003963 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f2ef406df60 msgr2=0x7f2ef4070410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:58.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:58.307+0000 7f2f10159640 1 --2- 172.21.15.112:0/1122003963 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f2ef406df60 0x7f2ef4070410 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2f04009a80 tx=0x7f2f04008040 comp rx=0 tx=0).stop 2024-09-16T00:28:58.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:58.307+0000 7f2f10159640 1 -- 172.21.15.112:0/1122003963 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f081014d0 msgr2=0x7f2f081159e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:58.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:58.307+0000 7f2f10159640 1 --2- 172.21.15.112:0/1122003963 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f081014d0 0x7f2f081159e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8003080 tx=0x7f2ef8009da0 comp rx=0 tx=0).stop 2024-09-16T00:28:58.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:58.307+0000 7f2f10159640 1 -- 172.21.15.112:0/1122003963 shutdown_connections 2024-09-16T00:28:58.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:58.307+0000 7f2f10159640 1 --2- 172.21.15.112:0/1122003963 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f2ef406df60 0x7f2ef4070410 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:58.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:58.307+0000 7f2f10159640 1 --2- 172.21.15.112:0/1122003963 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f081014d0 0x7f2f081159e0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:58.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:58.307+0000 7f2f10159640 1 --2- 172.21.15.112:0/1122003963 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f080fb490 0x7f2f081154a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:58.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:58.307+0000 7f2f10159640 1 -- 172.21.15.112:0/1122003963 >> 172.21.15.112:0/1122003963 conn(0x7f2f080f7010 msgr2=0x7f2f080f7cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:58.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:58.307+0000 7f2f10159640 1 -- 172.21.15.112:0/1122003963 shutdown_connections 2024-09-16T00:28:58.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:28:58.307+0000 7f2f10159640 1 -- 172.21.15.112:0/1122003963 wait complete. 2024-09-16T00:28:58.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:28:58 smithi112 ceph-mon[27869]: pgmap v43: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:28:58.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:28:58 smithi169 ceph-mon[34653]: pgmap v43: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:28:59.022 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -e sha1=8921c8ef947807b717db041d85f0761b7914d5fc -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1' 2024-09-16T00:28:59.266 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:29:00.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:00 smithi112 ceph-mon[27869]: pgmap v44: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:29:00.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:00 smithi169 ceph-mon[34653]: pgmap v44: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:29:00.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:00.964+0000 7f3bdaaac640 1 -- 172.21.15.112:0/93434481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3bd4102ef0 msgr2=0x7f3bd40fc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:00.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:00.964+0000 7f3bdaaac640 1 --2- 172.21.15.112:0/93434481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3bd4102ef0 0x7f3bd40fc5b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3bc80048d0 tx=0x7f3bc8009da0 comp rx=0 tx=0).stop 2024-09-16T00:29:00.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:00.964+0000 7f3bdaaac640 1 -- 172.21.15.112:0/93434481 shutdown_connections 2024-09-16T00:29:00.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:00.964+0000 7f3bdaaac640 1 --2- 172.21.15.112:0/93434481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3bd4102ef0 0x7f3bd40fc5b0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:00.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:00.964+0000 7f3bdaaac640 1 --2- 172.21.15.112:0/93434481 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3bd4102550 0x7f3bd4102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:00.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:00.964+0000 7f3bdaaac640 1 -- 172.21.15.112:0/93434481 >> 172.21.15.112:0/93434481 conn(0x7f3bd40f80f0 msgr2=0x7f3bd40fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:00.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:00.964+0000 7f3bdaaac640 1 -- 172.21.15.112:0/93434481 shutdown_connections 2024-09-16T00:29:00.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:00.964+0000 7f3bdaaac640 1 -- 172.21.15.112:0/93434481 wait complete. 2024-09-16T00:29:00.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:00.964+0000 7f3bdaaac640 1 Processor -- start 2024-09-16T00:29:00.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:00.964+0000 7f3bdaaac640 1 -- start start 2024-09-16T00:29:00.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:00.965+0000 7f3bdaaac640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3bd4102550 0x7f3bd41135b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:00.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:00.965+0000 7f3bdaaac640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3bd4102ef0 0x7f3bd4113af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:00.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:00.965+0000 7f3bdaaac640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd41141d0 con 0x7f3bd4102550 2024-09-16T00:29:00.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:00.965+0000 7f3bdaaac640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd410d550 con 0x7f3bd4102ef0 2024-09-16T00:29:00.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:00.965+0000 7f3bd8848640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3bd4102550 0x7f3bd41135b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:00.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:00.965+0000 7f3bd8848640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3bd4102550 0x7f3bd41135b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58392/0 (socket says 172.21.15.112:58392) 2024-09-16T00:29:00.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:00.965+0000 7f3bd8848640 1 -- 172.21.15.112:0/1211347187 learned_addr learned my addr 172.21.15.112:0/1211347187 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:29:00.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:00.965+0000 7f3bd3fff640 1 --2- 172.21.15.112:0/1211347187 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3bd4102ef0 0x7f3bd4113af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:00.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:00.965+0000 7f3bd8848640 1 -- 172.21.15.112:0/1211347187 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3bd4102ef0 msgr2=0x7f3bd4113af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:00.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:00.965+0000 7f3bd8848640 1 --2- 172.21.15.112:0/1211347187 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3bd4102ef0 0x7f3bd4113af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:00.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:00.965+0000 7f3bd8848640 1 -- 172.21.15.112:0/1211347187 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bc8005040 con 0x7f3bd4102550 2024-09-16T00:29:00.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:00.965+0000 7f3bd8848640 1 --2- 172.21.15.112:0/1211347187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3bd4102550 0x7f3bd41135b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f3bc400b790 tx=0x7f3bc40091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:00.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:00.965+0000 7f3bd17fa640 1 -- 172.21.15.112:0/1211347187 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3bc400c700 con 0x7f3bd4102550 2024-09-16T00:29:00.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:00.966+0000 7f3bd17fa640 1 -- 172.21.15.112:0/1211347187 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f3bc400cd00 con 0x7f3bd4102550 2024-09-16T00:29:00.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:00.966+0000 7f3bdaaac640 1 -- 172.21.15.112:0/1211347187 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bd410d7e0 con 0x7f3bd4102550 2024-09-16T00:29:00.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:00.966+0000 7f3bd17fa640 1 -- 172.21.15.112:0/1211347187 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3bc4015e70 con 0x7f3bd4102550 2024-09-16T00:29:00.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:00.966+0000 7f3bdaaac640 1 -- 172.21.15.112:0/1211347187 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bd410dd00 con 0x7f3bd4102550 2024-09-16T00:29:00.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:00.967+0000 7f3bdaaac640 1 -- 172.21.15.112:0/1211347187 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bd40fdcd0 con 0x7f3bd4102550 2024-09-16T00:29:00.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:00.967+0000 7f3bd17fa640 1 -- 172.21.15.112:0/1211347187 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 32) v1 ==== 92377+0+0 (secure 0 0 0) 0x7f3bc4012070 con 0x7f3bd4102550 2024-09-16T00:29:00.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:00.968+0000 7f3bd17fa640 1 --2- 172.21.15.112:0/1211347187 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f3bbc06e010 0x7f3bbc0704c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:00.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:00.968+0000 7f3bd3fff640 1 --2- 172.21.15.112:0/1211347187 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f3bbc06e010 0x7f3bbc0704c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:00.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:00.968+0000 7f3bd17fa640 1 -- 172.21.15.112:0/1211347187 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6152+0+0 (secure 0 0 0) 0x7f3bc408d400 con 0x7f3bd4102550 2024-09-16T00:29:00.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:00.968+0000 7f3bd3fff640 1 --2- 172.21.15.112:0/1211347187 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f3bbc06e010 0x7f3bbc0704c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3bd410e830 tx=0x7f3bc8009da0 comp rx=0 tx=0).ready entity=mgr.14678 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:00.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:00.972+0000 7f3bd17fa640 1 -- 172.21.15.112:0/1211347187 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185942 (secure 0 0 0) 0x7f3bc4056d50 con 0x7f3bd4102550 2024-09-16T00:29:01.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:01.157+0000 7f3bdaaac640 1 -- 172.21.15.112:0/1211347187 --> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc", "target": ["mon-mgr", ""]}) v1 -- 0x7f3bd410ec20 con 0x7f3bbc06e010 2024-09-16T00:29:01.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:01.162+0000 7f3bd17fa640 1 -- 172.21.15.112:0/1211347187 <== mgr.14678 v2:172.21.15.112:6800/1965658616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7f3bd410ec20 con 0x7f3bbc06e010 2024-09-16T00:29:01.162 INFO:teuthology.orchestra.run.smithi112.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T00:29:01.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:01.164+0000 7f3bdaaac640 1 -- 172.21.15.112:0/1211347187 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f3bbc06e010 msgr2=0x7f3bbc0704c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:01.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:01.164+0000 7f3bdaaac640 1 --2- 172.21.15.112:0/1211347187 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f3bbc06e010 0x7f3bbc0704c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3bd410e830 tx=0x7f3bc8009da0 comp rx=0 tx=0).stop 2024-09-16T00:29:01.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:01.164+0000 7f3bdaaac640 1 -- 172.21.15.112:0/1211347187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3bd4102550 msgr2=0x7f3bd41135b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:01.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:01.164+0000 7f3bdaaac640 1 --2- 172.21.15.112:0/1211347187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3bd4102550 0x7f3bd41135b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f3bc400b790 tx=0x7f3bc40091d0 comp rx=0 tx=0).stop 2024-09-16T00:29:01.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:01.164+0000 7f3bdaaac640 1 -- 172.21.15.112:0/1211347187 shutdown_connections 2024-09-16T00:29:01.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:01.164+0000 7f3bdaaac640 1 --2- 172.21.15.112:0/1211347187 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f3bbc06e010 0x7f3bbc0704c0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:01.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:01.164+0000 7f3bdaaac640 1 --2- 172.21.15.112:0/1211347187 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3bd4102ef0 0x7f3bd4113af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:01.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:01.164+0000 7f3bdaaac640 1 --2- 172.21.15.112:0/1211347187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3bd4102550 0x7f3bd41135b0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:01.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:01.164+0000 7f3bdaaac640 1 -- 172.21.15.112:0/1211347187 >> 172.21.15.112:0/1211347187 conn(0x7f3bd40f80f0 msgr2=0x7f3bd40f91a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:01.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:01.164+0000 7f3bdaaac640 1 -- 172.21.15.112:0/1211347187 shutdown_connections 2024-09-16T00:29:01.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:01.164+0000 7f3bdaaac640 1 -- 172.21.15.112:0/1211347187 wait complete. 2024-09-16T00:29:01.813 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-09-16T00:29:01.814 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi112.front.sepia.ceph.com 2024-09-16T00:29:01.814 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -e sha1=8921c8ef947807b717db041d85f0761b7914d5fc -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done' 2024-09-16T00:29:02.070 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:29:02.474 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:02 smithi112 ceph-mon[27869]: pgmap v45: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:29:02.475 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:02 smithi112 ceph-mon[27869]: from='client.14782 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:29:02.475 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:02 smithi112 ceph-mon[27869]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T00:29:02.475 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:02 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:02.475 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:02 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:29:02.475 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:02 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:02.475 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:02 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:29:02.475 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:02 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:02.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:02 smithi169 ceph-mon[34653]: pgmap v45: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:29:02.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:02 smithi169 ceph-mon[34653]: from='client.14782 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:29:02.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:02 smithi169 ceph-mon[34653]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T00:29:02.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:02 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:02.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:02 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:29:02.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:02 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:02.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:02 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:29:02.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:02 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:03.479 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:03 smithi112 ceph-mon[27869]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T00:29:03.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:03 smithi169 ceph-mon[34653]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T00:29:04.454 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:04 smithi112 ceph-mon[27869]: pgmap v46: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:29:04.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:04 smithi169 ceph-mon[34653]: pgmap v46: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:29:05.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.123+0000 7f9fd68b2640 1 -- 172.21.15.112:0/3135195174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fd01014b0 msgr2=0x7f9fd0101880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:05.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.123+0000 7f9fd68b2640 1 --2- 172.21.15.112:0/3135195174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fd01014b0 0x7f9fd0101880 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f9fc00048d0 tx=0x7f9fc0009da0 comp rx=0 tx=0).stop 2024-09-16T00:29:05.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.123+0000 7f9fd68b2640 1 -- 172.21.15.112:0/3135195174 shutdown_connections 2024-09-16T00:29:05.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.123+0000 7f9fd68b2640 1 --2- 172.21.15.112:0/3135195174 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9fd00fb470 0x7f9fd00fb8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:05.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.123+0000 7f9fd68b2640 1 --2- 172.21.15.112:0/3135195174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fd01014b0 0x7f9fd0101880 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:05.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.123+0000 7f9fd68b2640 1 -- 172.21.15.112:0/3135195174 >> 172.21.15.112:0/3135195174 conn(0x7f9fd00f7030 msgr2=0x7f9fd00f9440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:05.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.123+0000 7f9fd68b2640 1 -- 172.21.15.112:0/3135195174 shutdown_connections 2024-09-16T00:29:05.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.123+0000 7f9fd68b2640 1 -- 172.21.15.112:0/3135195174 wait complete. 2024-09-16T00:29:05.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.124+0000 7f9fd68b2640 1 Processor -- start 2024-09-16T00:29:05.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.124+0000 7f9fd68b2640 1 -- start start 2024-09-16T00:29:05.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.124+0000 7f9fd68b2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fd00fb470 0x7f9fd010e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:05.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.124+0000 7f9fd68b2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9fd01014b0 0x7f9fd010e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:05.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.124+0000 7f9fd68b2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd010ed20 con 0x7f9fd00fb470 2024-09-16T00:29:05.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.124+0000 7f9fd68b2640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd010ee90 con 0x7f9fd01014b0 2024-09-16T00:29:05.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.124+0000 7f9fcffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fd00fb470 0x7f9fd010e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:05.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.124+0000 7f9fcffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fd00fb470 0x7f9fd010e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36726/0 (socket says 172.21.15.112:36726) 2024-09-16T00:29:05.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.124+0000 7f9fcffff640 1 -- 172.21.15.112:0/2182076928 learned_addr learned my addr 172.21.15.112:0/2182076928 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:29:05.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.124+0000 7f9fc7fff640 1 --2- 172.21.15.112:0/2182076928 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9fd01014b0 0x7f9fd010e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:05.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.124+0000 7f9fcffff640 1 -- 172.21.15.112:0/2182076928 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9fd01014b0 msgr2=0x7f9fd010e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:05.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.124+0000 7f9fcffff640 1 --2- 172.21.15.112:0/2182076928 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9fd01014b0 0x7f9fd010e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:05.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.124+0000 7f9fcffff640 1 -- 172.21.15.112:0/2182076928 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fc0005040 con 0x7f9fd00fb470 2024-09-16T00:29:05.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.125+0000 7f9fcffff640 1 --2- 172.21.15.112:0/2182076928 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fd00fb470 0x7f9fd010e2a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9fc0000c00 tx=0x7f9fc0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:05.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.125+0000 7f9fcd7fa640 1 -- 172.21.15.112:0/2182076928 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9fc0038070 con 0x7f9fd00fb470 2024-09-16T00:29:05.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.125+0000 7f9fcd7fa640 1 -- 172.21.15.112:0/2182076928 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f9fc002ade0 con 0x7f9fd00fb470 2024-09-16T00:29:05.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.125+0000 7f9fcd7fa640 1 -- 172.21.15.112:0/2182076928 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9fc002c7e0 con 0x7f9fd00fb470 2024-09-16T00:29:05.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.125+0000 7f9fd68b2640 1 -- 172.21.15.112:0/2182076928 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fd0117170 con 0x7f9fd00fb470 2024-09-16T00:29:05.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.125+0000 7f9fd68b2640 1 -- 172.21.15.112:0/2182076928 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fd0117630 con 0x7f9fd00fb470 2024-09-16T00:29:05.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.126+0000 7f9fd68b2640 1 -- 172.21.15.112:0/2182076928 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fd00fcbb0 con 0x7f9fd00fb470 2024-09-16T00:29:05.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.127+0000 7f9fcd7fa640 1 -- 172.21.15.112:0/2182076928 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 32) v1 ==== 92377+0+0 (secure 0 0 0) 0x7f9fc0030020 con 0x7f9fd00fb470 2024-09-16T00:29:05.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.127+0000 7f9fcd7fa640 1 --2- 172.21.15.112:0/2182076928 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f9fbc06df60 0x7f9fbc070410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:05.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.127+0000 7f9fcd7fa640 1 -- 172.21.15.112:0/2182076928 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6152+0+0 (secure 0 0 0) 0x7f9fc00ad600 con 0x7f9fd00fb470 2024-09-16T00:29:05.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.127+0000 7f9fc7fff640 1 --2- 172.21.15.112:0/2182076928 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f9fbc06df60 0x7f9fbc070410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:05.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.128+0000 7f9fc7fff640 1 --2- 172.21.15.112:0/2182076928 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f9fbc06df60 0x7f9fbc070410 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8009fd0 tx=0x7f9fb8009e50 comp rx=0 tx=0).ready entity=mgr.14678 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:05.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.131+0000 7f9fcd7fa640 1 -- 172.21.15.112:0/2182076928 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185942 (secure 0 0 0) 0x7f9fc0076ee0 con 0x7f9fd00fb470 2024-09-16T00:29:05.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.327+0000 7f9fd68b2640 1 -- 172.21.15.112:0/2182076928 --> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9fd0066020 con 0x7f9fbc06df60 2024-09-16T00:29:05.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.328+0000 7f9fcd7fa640 1 -- 172.21.15.112:0/2182076928 <== mgr.14678 v2:172.21.15.112:6800/1965658616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f9fd0066020 con 0x7f9fbc06df60 2024-09-16T00:29:05.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.330+0000 7f9fd68b2640 1 -- 172.21.15.112:0/2182076928 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f9fbc06df60 msgr2=0x7f9fbc070410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:05.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.330+0000 7f9fd68b2640 1 --2- 172.21.15.112:0/2182076928 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f9fbc06df60 0x7f9fbc070410 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8009fd0 tx=0x7f9fb8009e50 comp rx=0 tx=0).stop 2024-09-16T00:29:05.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.330+0000 7f9fd68b2640 1 -- 172.21.15.112:0/2182076928 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fd00fb470 msgr2=0x7f9fd010e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:05.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.330+0000 7f9fd68b2640 1 --2- 172.21.15.112:0/2182076928 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fd00fb470 0x7f9fd010e2a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9fc0000c00 tx=0x7f9fc0009da0 comp rx=0 tx=0).stop 2024-09-16T00:29:05.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.330+0000 7f9fd68b2640 1 -- 172.21.15.112:0/2182076928 shutdown_connections 2024-09-16T00:29:05.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.330+0000 7f9fd68b2640 1 --2- 172.21.15.112:0/2182076928 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f9fbc06df60 0x7f9fbc070410 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:05.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.330+0000 7f9fd68b2640 1 --2- 172.21.15.112:0/2182076928 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9fd01014b0 0x7f9fd010e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:05.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.330+0000 7f9fd68b2640 1 --2- 172.21.15.112:0/2182076928 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fd00fb470 0x7f9fd010e2a0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:05.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.330+0000 7f9fd68b2640 1 -- 172.21.15.112:0/2182076928 >> 172.21.15.112:0/2182076928 conn(0x7f9fd00f7030 msgr2=0x7f9fd00f88e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:05.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.330+0000 7f9fd68b2640 1 -- 172.21.15.112:0/2182076928 shutdown_connections 2024-09-16T00:29:05.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.330+0000 7f9fd68b2640 1 -- 172.21.15.112:0/2182076928 wait complete. 2024-09-16T00:29:05.339 INFO:teuthology.orchestra.run.smithi112.stdout:true 2024-09-16T00:29:05.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.410+0000 7f4658332640 1 -- 172.21.15.112:0/4100955011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4650065300 msgr2=0x7f46500656d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:05.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.410+0000 7f4658332640 1 --2- 172.21.15.112:0/4100955011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4650065300 0x7f46500656d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f46400048d0 tx=0x7f4640009da0 comp rx=0 tx=0).stop 2024-09-16T00:29:05.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.411+0000 7f4658332640 1 -- 172.21.15.112:0/4100955011 shutdown_connections 2024-09-16T00:29:05.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.411+0000 7f4658332640 1 --2- 172.21.15.112:0/4100955011 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4650065c10 0x7f46500fe0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:05.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.411+0000 7f4658332640 1 --2- 172.21.15.112:0/4100955011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4650065300 0x7f46500656d0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:05.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.411+0000 7f4658332640 1 -- 172.21.15.112:0/4100955011 >> 172.21.15.112:0/4100955011 conn(0x7f46500723b0 msgr2=0x7f46500727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:05.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.411+0000 7f4658332640 1 -- 172.21.15.112:0/4100955011 shutdown_connections 2024-09-16T00:29:05.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.411+0000 7f4658332640 1 -- 172.21.15.112:0/4100955011 wait complete. 2024-09-16T00:29:05.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.411+0000 7f4658332640 1 Processor -- start 2024-09-16T00:29:05.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.411+0000 7f4658332640 1 -- start start 2024-09-16T00:29:05.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.411+0000 7f4658332640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4650065300 0x7f46501131d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:05.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.411+0000 7f4658332640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4650065c10 0x7f4650113710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:05.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.411+0000 7f4658332640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4650113d10 con 0x7f4650065300 2024-09-16T00:29:05.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.411+0000 7f4658332640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f465010d150 con 0x7f4650065c10 2024-09-16T00:29:05.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.411+0000 7f46560ce640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4650065300 0x7f46501131d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:05.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.412+0000 7f46560ce640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4650065300 0x7f46501131d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36742/0 (socket says 172.21.15.112:36742) 2024-09-16T00:29:05.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.412+0000 7f46560ce640 1 -- 172.21.15.112:0/2276141974 learned_addr learned my addr 172.21.15.112:0/2276141974 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:29:05.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.412+0000 7f46558cd640 1 --2- 172.21.15.112:0/2276141974 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4650065c10 0x7f4650113710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:05.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.412+0000 7f46560ce640 1 -- 172.21.15.112:0/2276141974 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4650065c10 msgr2=0x7f4650113710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:05.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.412+0000 7f46560ce640 1 --2- 172.21.15.112:0/2276141974 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4650065c10 0x7f4650113710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:05.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.412+0000 7f46560ce640 1 -- 172.21.15.112:0/2276141974 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4640005040 con 0x7f4650065300 2024-09-16T00:29:05.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.412+0000 7f46560ce640 1 --2- 172.21.15.112:0/2276141974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4650065300 0x7f46501131d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4640000c00 tx=0x7f4640009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:05.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.412+0000 7f464effd640 1 -- 172.21.15.112:0/2276141974 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4640038070 con 0x7f4650065300 2024-09-16T00:29:05.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.412+0000 7f4658332640 1 -- 172.21.15.112:0/2276141974 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f465010d3d0 con 0x7f4650065300 2024-09-16T00:29:05.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.412+0000 7f464effd640 1 -- 172.21.15.112:0/2276141974 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f464002ade0 con 0x7f4650065300 2024-09-16T00:29:05.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.412+0000 7f464effd640 1 -- 172.21.15.112:0/2276141974 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f464002c7b0 con 0x7f4650065300 2024-09-16T00:29:05.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.412+0000 7f4658332640 1 -- 172.21.15.112:0/2276141974 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f465010d910 con 0x7f4650065300 2024-09-16T00:29:05.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.413+0000 7f4658332640 1 -- 172.21.15.112:0/2276141974 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46500fdce0 con 0x7f4650065300 2024-09-16T00:29:05.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.414+0000 7f464effd640 1 -- 172.21.15.112:0/2276141974 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 32) v1 ==== 92377+0+0 (secure 0 0 0) 0x7f4640030020 con 0x7f4650065300 2024-09-16T00:29:05.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.415+0000 7f464effd640 1 --2- 172.21.15.112:0/2276141974 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f463806df60 0x7f4638070410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:05.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.415+0000 7f464effd640 1 -- 172.21.15.112:0/2276141974 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6152+0+0 (secure 0 0 0) 0x7f46400addd0 con 0x7f4650065300 2024-09-16T00:29:05.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.415+0000 7f46558cd640 1 --2- 172.21.15.112:0/2276141974 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f463806df60 0x7f4638070410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:05.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.415+0000 7f46558cd640 1 --2- 172.21.15.112:0/2276141974 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f463806df60 0x7f4638070410 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f465010ea90 tx=0x7f464400b040 comp rx=0 tx=0).ready entity=mgr.14678 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:05.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.419+0000 7f464effd640 1 -- 172.21.15.112:0/2276141974 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185942 (secure 0 0 0) 0x7f4640077600 con 0x7f4650065300 2024-09-16T00:29:05.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.608+0000 7f4658332640 1 -- 172.21.15.112:0/2276141974 --> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f465004b720 con 0x7f463806df60 2024-09-16T00:29:05.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.609+0000 7f464effd640 1 -- 172.21.15.112:0/2276141974 <== mgr.14678 v2:172.21.15.112:6800/1965658616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f465004b720 con 0x7f463806df60 2024-09-16T00:29:05.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.611+0000 7f4658332640 1 -- 172.21.15.112:0/2276141974 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f463806df60 msgr2=0x7f4638070410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:05.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.611+0000 7f4658332640 1 --2- 172.21.15.112:0/2276141974 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f463806df60 0x7f4638070410 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f465010ea90 tx=0x7f464400b040 comp rx=0 tx=0).stop 2024-09-16T00:29:05.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.611+0000 7f4658332640 1 -- 172.21.15.112:0/2276141974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4650065300 msgr2=0x7f46501131d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:05.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.611+0000 7f4658332640 1 --2- 172.21.15.112:0/2276141974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4650065300 0x7f46501131d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4640000c00 tx=0x7f4640009da0 comp rx=0 tx=0).stop 2024-09-16T00:29:05.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.611+0000 7f4658332640 1 -- 172.21.15.112:0/2276141974 shutdown_connections 2024-09-16T00:29:05.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.611+0000 7f4658332640 1 --2- 172.21.15.112:0/2276141974 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f463806df60 0x7f4638070410 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:05.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.611+0000 7f4658332640 1 --2- 172.21.15.112:0/2276141974 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4650065c10 0x7f4650113710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:05.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.611+0000 7f4658332640 1 --2- 172.21.15.112:0/2276141974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4650065300 0x7f46501131d0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:05.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.611+0000 7f4658332640 1 -- 172.21.15.112:0/2276141974 >> 172.21.15.112:0/2276141974 conn(0x7f46500723b0 msgr2=0x7f4650105420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:05.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.611+0000 7f4658332640 1 -- 172.21.15.112:0/2276141974 shutdown_connections 2024-09-16T00:29:05.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.611+0000 7f4658332640 1 -- 172.21.15.112:0/2276141974 wait complete. 2024-09-16T00:29:05.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.692+0000 7f79aaf42640 1 -- 172.21.15.112:0/1510147381 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79a41014b0 msgr2=0x7f79a4101880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:05.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.692+0000 7f79aaf42640 1 --2- 172.21.15.112:0/1510147381 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79a41014b0 0x7f79a4101880 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f79980048d0 tx=0x7f7998009da0 comp rx=0 tx=0).stop 2024-09-16T00:29:05.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.693+0000 7f79aaf42640 1 -- 172.21.15.112:0/1510147381 shutdown_connections 2024-09-16T00:29:05.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.693+0000 7f79aaf42640 1 --2- 172.21.15.112:0/1510147381 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79a40fb470 0x7f79a40fb8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:05.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.693+0000 7f79aaf42640 1 --2- 172.21.15.112:0/1510147381 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79a41014b0 0x7f79a4101880 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:05.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.693+0000 7f79aaf42640 1 -- 172.21.15.112:0/1510147381 >> 172.21.15.112:0/1510147381 conn(0x7f79a40f7030 msgr2=0x7f79a40f9440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:05.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.693+0000 7f79aaf42640 1 -- 172.21.15.112:0/1510147381 shutdown_connections 2024-09-16T00:29:05.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.693+0000 7f79aaf42640 1 -- 172.21.15.112:0/1510147381 wait complete. 2024-09-16T00:29:05.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.693+0000 7f79aaf42640 1 Processor -- start 2024-09-16T00:29:05.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.693+0000 7f79aaf42640 1 -- start start 2024-09-16T00:29:05.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.694+0000 7f79aaf42640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79a40fb470 0x7f79a41120c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:05.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.694+0000 7f79aaf42640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79a41014b0 0x7f79a4112600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:05.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.694+0000 7f79aaf42640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a4112c00 con 0x7f79a40fb470 2024-09-16T00:29:05.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.694+0000 7f79aaf42640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a410c040 con 0x7f79a41014b0 2024-09-16T00:29:05.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.694+0000 7f79a8cde640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79a40fb470 0x7f79a41120c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:05.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.694+0000 7f79a8cde640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79a40fb470 0x7f79a41120c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36758/0 (socket says 172.21.15.112:36758) 2024-09-16T00:29:05.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.694+0000 7f79a8cde640 1 -- 172.21.15.112:0/3970393762 learned_addr learned my addr 172.21.15.112:0/3970393762 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:29:05.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.694+0000 7f79a3fff640 1 --2- 172.21.15.112:0/3970393762 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79a41014b0 0x7f79a4112600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:05.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.694+0000 7f79a8cde640 1 -- 172.21.15.112:0/3970393762 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79a41014b0 msgr2=0x7f79a4112600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:05.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.694+0000 7f79a8cde640 1 --2- 172.21.15.112:0/3970393762 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79a41014b0 0x7f79a4112600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:05.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.694+0000 7f79a8cde640 1 -- 172.21.15.112:0/3970393762 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7998005040 con 0x7f79a40fb470 2024-09-16T00:29:05.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.694+0000 7f79a8cde640 1 --2- 172.21.15.112:0/3970393762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79a40fb470 0x7f79a41120c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f7998003080 tx=0x7f7998009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:05.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.694+0000 7f79a17fa640 1 -- 172.21.15.112:0/3970393762 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7998038070 con 0x7f79a40fb470 2024-09-16T00:29:05.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.694+0000 7f79aaf42640 1 -- 172.21.15.112:0/3970393762 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79a410c2c0 con 0x7f79a40fb470 2024-09-16T00:29:05.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.694+0000 7f79a17fa640 1 -- 172.21.15.112:0/3970393762 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f79980097e0 con 0x7f79a40fb470 2024-09-16T00:29:05.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.694+0000 7f79a17fa640 1 -- 172.21.15.112:0/3970393762 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f799803d420 con 0x7f79a40fb470 2024-09-16T00:29:05.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.695+0000 7f79aaf42640 1 -- 172.21.15.112:0/3970393762 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79a410c800 con 0x7f79a40fb470 2024-09-16T00:29:05.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.696+0000 7f79aaf42640 1 -- 172.21.15.112:0/3970393762 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79a40fcbb0 con 0x7f79a40fb470 2024-09-16T00:29:05.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.696+0000 7f79a17fa640 1 -- 172.21.15.112:0/3970393762 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 32) v1 ==== 92377+0+0 (secure 0 0 0) 0x7f7998030020 con 0x7f79a40fb470 2024-09-16T00:29:05.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.696+0000 7f79a17fa640 1 --2- 172.21.15.112:0/3970393762 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f799406df60 0x7f7994070410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:05.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.696+0000 7f79a17fa640 1 -- 172.21.15.112:0/3970393762 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6152+0+0 (secure 0 0 0) 0x7f79980ade80 con 0x7f79a40fb470 2024-09-16T00:29:05.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.696+0000 7f79a3fff640 1 --2- 172.21.15.112:0/3970393762 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f799406df60 0x7f7994070410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:05.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.697+0000 7f79a3fff640 1 --2- 172.21.15.112:0/3970393762 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f799406df60 0x7f7994070410 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f79a410d980 tx=0x7f7990004330 comp rx=0 tx=0).ready entity=mgr.14678 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:05.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.701+0000 7f79a17fa640 1 -- 172.21.15.112:0/3970393762 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185942 (secure 0 0 0) 0x7f79980776b0 con 0x7f79a40fb470 2024-09-16T00:29:05.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.876+0000 7f79aaf42640 1 -- 172.21.15.112:0/3970393762 --> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f79a404b720 con 0x7f799406df60 2024-09-16T00:29:05.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.885+0000 7f79a17fa640 1 -- 172.21.15.112:0/3970393762 <== mgr.14678 v2:172.21.15.112:6800/1965658616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f79a404b720 con 0x7f799406df60 2024-09-16T00:29:05.885 INFO:teuthology.orchestra.run.smithi112.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T00:29:05.885 INFO:teuthology.orchestra.run.smithi112.stdout:alertmanager.smithi112 smithi112 *:9093,9094 running (2m) 43s ago 10m 16.6M - 0.25.0 c8568f914cd2 d85ea0562022 2024-09-16T00:29:05.886 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi112 smithi112 running (11m) 43s ago 11m 6891k - 17.2.7-1633-g6d025bc2 33ed8dba3996 c94220ed2776 2024-09-16T00:29:05.886 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi169 smithi169 running (10m) 65s ago 10m 6899k - 17.2.7-1633-g6d025bc2 33ed8dba3996 d0d3726280a2 2024-09-16T00:29:05.886 INFO:teuthology.orchestra.run.smithi112.stdout:grafana.smithi112 smithi112 *:3000 running (48s) 43s ago 10m 45.2M - 9.4.12 2bacad6d85d8 51f9d2013da2 2024-09-16T00:29:05.886 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.hxfwkh smithi112 running (5m) 43s ago 5m 136M - 17.2.7-1633-g6d025bc2 33ed8dba3996 cf92d9461fc6 2024-09-16T00:29:05.886 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.ithwzj smithi112 running (5m) 43s ago 5m 65.6M - 17.2.7-1633-g6d025bc2 33ed8dba3996 02786c8a83c7 2024-09-16T00:29:05.886 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.gawwgw smithi169 running (5m) 65s ago 5m 114M - 17.2.7-1633-g6d025bc2 33ed8dba3996 dcf3a4dfa879 2024-09-16T00:29:05.886 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.lgeqrj smithi169 running (5m) 65s ago 5m 60.2M - 17.2.7-1633-g6d025bc2 33ed8dba3996 445c14e55831 2024-09-16T00:29:05.886 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi112.xrkkbc smithi112 *:8443,9283,8765 running (98s) 43s ago 12m 499M - 19.1.1-326-g8921c8ef f1455bd0219c a981f7fd2810 2024-09-16T00:29:05.886 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi169.mtrobk smithi169 *:8443,9283,8765 running (68s) 65s ago 10m 69.2M - 19.1.1-326-g8921c8ef f1455bd0219c 4096e5369c2d 2024-09-16T00:29:05.886 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi112 smithi112 running (12m) 43s ago 12m 55.1M 2048M 17.2.7-1633-g6d025bc2 33ed8dba3996 ddaaced54bf8 2024-09-16T00:29:05.886 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi169 smithi169 running (10m) 65s ago 10m 42.9M 2048M 17.2.7-1633-g6d025bc2 33ed8dba3996 6c2289bc3e11 2024-09-16T00:29:05.887 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi112 smithi112 *:9100 running (2m) 43s ago 11m 8451k - 1.5.0 0da6a335fe13 6404d1d601e2 2024-09-16T00:29:05.887 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi169 smithi169 *:9100 running (104s) 65s ago 10m 8094k - 1.5.0 0da6a335fe13 5505baf702a6 2024-09-16T00:29:05.887 INFO:teuthology.orchestra.run.smithi112.stdout:osd.0 smithi112 running (9m) 43s ago 9m 190M 1557M 17.2.7-1633-g6d025bc2 33ed8dba3996 f6b2e146ffdf 2024-09-16T00:29:05.887 INFO:teuthology.orchestra.run.smithi112.stdout:osd.1 smithi112 running (8m) 43s ago 8m 201M 1557M 17.2.7-1633-g6d025bc2 33ed8dba3996 e9bee87bba0f 2024-09-16T00:29:05.887 INFO:teuthology.orchestra.run.smithi112.stdout:osd.2 smithi112 running (8m) 43s ago 8m 158M 1557M 17.2.7-1633-g6d025bc2 33ed8dba3996 100ec4e4852a 2024-09-16T00:29:05.887 INFO:teuthology.orchestra.run.smithi112.stdout:osd.3 smithi169 running (8m) 65s ago 7m 230M 2581M 17.2.7-1633-g6d025bc2 33ed8dba3996 f88463e19f23 2024-09-16T00:29:05.887 INFO:teuthology.orchestra.run.smithi112.stdout:osd.4 smithi169 running (7m) 65s ago 7m 210M 2581M 17.2.7-1633-g6d025bc2 33ed8dba3996 d3e4cdb016b9 2024-09-16T00:29:05.887 INFO:teuthology.orchestra.run.smithi112.stdout:osd.5 smithi169 running (7m) 65s ago 7m 197M 2581M 17.2.7-1633-g6d025bc2 33ed8dba3996 8e7d708d74eb 2024-09-16T00:29:05.887 INFO:teuthology.orchestra.run.smithi112.stdout:prometheus.smithi112 smithi112 *:9095 running (73s) 43s ago 10m 53.2M - 2.43.0 a07b618ecd1d 7a1d42e60d2b 2024-09-16T00:29:05.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.887+0000 7f79aaf42640 1 -- 172.21.15.112:0/3970393762 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f799406df60 msgr2=0x7f7994070410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:05.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.887+0000 7f79aaf42640 1 --2- 172.21.15.112:0/3970393762 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f799406df60 0x7f7994070410 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f79a410d980 tx=0x7f7990004330 comp rx=0 tx=0).stop 2024-09-16T00:29:05.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.887+0000 7f79aaf42640 1 -- 172.21.15.112:0/3970393762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79a40fb470 msgr2=0x7f79a41120c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:05.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.887+0000 7f79aaf42640 1 --2- 172.21.15.112:0/3970393762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79a40fb470 0x7f79a41120c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f7998003080 tx=0x7f7998009da0 comp rx=0 tx=0).stop 2024-09-16T00:29:05.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.887+0000 7f79aaf42640 1 -- 172.21.15.112:0/3970393762 shutdown_connections 2024-09-16T00:29:05.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.887+0000 7f79aaf42640 1 --2- 172.21.15.112:0/3970393762 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f799406df60 0x7f7994070410 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:05.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.887+0000 7f79aaf42640 1 --2- 172.21.15.112:0/3970393762 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79a41014b0 0x7f79a4112600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:05.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.887+0000 7f79aaf42640 1 --2- 172.21.15.112:0/3970393762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79a40fb470 0x7f79a41120c0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:05.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.887+0000 7f79aaf42640 1 -- 172.21.15.112:0/3970393762 >> 172.21.15.112:0/3970393762 conn(0x7f79a40f7030 msgr2=0x7f79a40f88e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:05.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.887+0000 7f79aaf42640 1 -- 172.21.15.112:0/3970393762 shutdown_connections 2024-09-16T00:29:05.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.887+0000 7f79aaf42640 1 -- 172.21.15.112:0/3970393762 wait complete. 2024-09-16T00:29:05.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.966+0000 7f9b2b8f4640 1 -- 172.21.15.112:0/2529388490 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b24102530 msgr2=0x7f9b24102900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:05.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.966+0000 7f9b2b8f4640 1 --2- 172.21.15.112:0/2529388490 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b24102530 0x7f9b24102900 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f9b200048d0 tx=0x7f9b20009da0 comp rx=0 tx=0).stop 2024-09-16T00:29:05.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.966+0000 7f9b2b8f4640 1 -- 172.21.15.112:0/2529388490 shutdown_connections 2024-09-16T00:29:05.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.966+0000 7f9b2b8f4640 1 --2- 172.21.15.112:0/2529388490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b24102ed0 0x7f9b240fc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:05.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.966+0000 7f9b2b8f4640 1 --2- 172.21.15.112:0/2529388490 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b24102530 0x7f9b24102900 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:05.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.966+0000 7f9b2b8f4640 1 -- 172.21.15.112:0/2529388490 >> 172.21.15.112:0/2529388490 conn(0x7f9b240f80f0 msgr2=0x7f9b240fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:05.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.966+0000 7f9b2b8f4640 1 -- 172.21.15.112:0/2529388490 shutdown_connections 2024-09-16T00:29:05.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.966+0000 7f9b2b8f4640 1 -- 172.21.15.112:0/2529388490 wait complete. 2024-09-16T00:29:05.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.966+0000 7f9b2b8f4640 1 Processor -- start 2024-09-16T00:29:05.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.966+0000 7f9b2b8f4640 1 -- start start 2024-09-16T00:29:05.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.966+0000 7f9b2b8f4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b24102530 0x7f9b2406fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:05.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.967+0000 7f9b2b8f4640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b24102ed0 0x7f9b2406ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:05.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.967+0000 7f9b2b8f4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b24070520 con 0x7f9b24102530 2024-09-16T00:29:05.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.967+0000 7f9b2b8f4640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b24070690 con 0x7f9b24102ed0 2024-09-16T00:29:05.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.967+0000 7f9b29690640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b24102530 0x7f9b2406fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:05.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.967+0000 7f9b29690640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b24102530 0x7f9b2406fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36776/0 (socket says 172.21.15.112:36776) 2024-09-16T00:29:05.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.967+0000 7f9b29690640 1 -- 172.21.15.112:0/3502975405 learned_addr learned my addr 172.21.15.112:0/3502975405 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:29:05.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.967+0000 7f9b28e8f640 1 --2- 172.21.15.112:0/3502975405 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b24102ed0 0x7f9b2406ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:05.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.967+0000 7f9b29690640 1 -- 172.21.15.112:0/3502975405 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b24102ed0 msgr2=0x7f9b2406ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:05.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.967+0000 7f9b29690640 1 --2- 172.21.15.112:0/3502975405 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b24102ed0 0x7f9b2406ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:05.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.967+0000 7f9b29690640 1 -- 172.21.15.112:0/3502975405 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b20005040 con 0x7f9b24102530 2024-09-16T00:29:05.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.967+0000 7f9b28e8f640 1 --2- 172.21.15.112:0/3502975405 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b24102ed0 0x7f9b2406ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:29:05.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.967+0000 7f9b29690640 1 --2- 172.21.15.112:0/3502975405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b24102530 0x7f9b2406fa10 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f9b20000c00 tx=0x7f9b20009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:05.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.967+0000 7f9b19ffb640 1 -- 172.21.15.112:0/3502975405 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b20038070 con 0x7f9b24102530 2024-09-16T00:29:05.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.967+0000 7f9b2b8f4640 1 -- 172.21.15.112:0/3502975405 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b24119570 con 0x7f9b24102530 2024-09-16T00:29:05.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.967+0000 7f9b19ffb640 1 -- 172.21.15.112:0/3502975405 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f9b2003daf0 con 0x7f9b24102530 2024-09-16T00:29:05.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.967+0000 7f9b19ffb640 1 -- 172.21.15.112:0/3502975405 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b20033b20 con 0x7f9b24102530 2024-09-16T00:29:05.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.968+0000 7f9b2b8f4640 1 -- 172.21.15.112:0/3502975405 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b24119950 con 0x7f9b24102530 2024-09-16T00:29:05.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.969+0000 7f9b2b8f4640 1 -- 172.21.15.112:0/3502975405 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b240fdcb0 con 0x7f9b24102530 2024-09-16T00:29:05.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.969+0000 7f9b19ffb640 1 -- 172.21.15.112:0/3502975405 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 32) v1 ==== 92377+0+0 (secure 0 0 0) 0x7f9b20030020 con 0x7f9b24102530 2024-09-16T00:29:05.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.969+0000 7f9b19ffb640 1 --2- 172.21.15.112:0/3502975405 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f9b1006df60 0x7f9b10070410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:05.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.970+0000 7f9b19ffb640 1 -- 172.21.15.112:0/3502975405 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6152+0+0 (secure 0 0 0) 0x7f9b200adbd0 con 0x7f9b24102530 2024-09-16T00:29:05.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.970+0000 7f9b28e8f640 1 --2- 172.21.15.112:0/3502975405 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f9b1006df60 0x7f9b10070410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:05.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.970+0000 7f9b28e8f640 1 --2- 172.21.15.112:0/3502975405 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f9b1006df60 0x7f9b10070410 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9b14006fd0 tx=0x7f9b14008040 comp rx=0 tx=0).ready entity=mgr.14678 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:05.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:05.974+0000 7f9b19ffb640 1 -- 172.21.15.112:0/3502975405 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185942 (secure 0 0 0) 0x7f9b20077400 con 0x7f9b24102530 2024-09-16T00:29:06.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.207+0000 7f9b2b8f4640 1 -- 172.21.15.112:0/3502975405 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f9b240fbdb0 con 0x7f9b24102530 2024-09-16T00:29:06.207 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:05 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:06.208 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:05 smithi112 ceph-mon[27869]: Upgrade: Target is version 19.1.1-326-g8921c8ef (squid) 2024-09-16T00:29:06.208 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:05 smithi112 ceph-mon[27869]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, digests ['quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80'] 2024-09-16T00:29:06.208 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:05 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:29:06.208 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:05 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:29:06.208 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:05 smithi112 ceph-mon[27869]: Upgrade: Setting container_image for all mgr 2024-09-16T00:29:06.208 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:05 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:06.209 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:05 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-09-16T00:29:06.209 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:05 smithi112 ceph-mon[27869]: pgmap v47: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:29:06.209 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:05 smithi112 ceph-mon[27869]: from='client.14786 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:29:06.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.207+0000 7f9b19ffb640 1 -- 172.21.15.112:0/3502975405 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+747 (secure 0 0 0) 0x7f9b2007aac0 con 0x7f9b24102530 2024-09-16T00:29:06.209 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:29:06.209 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": { 2024-09-16T00:29:06.209 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 17.2.7-1633-g6d025bc2 (6d025bc26f3176c910707f8cb4c8d8e076640615) quincy (stable)": 2 2024-09-16T00:29:06.210 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:29:06.210 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": { 2024-09-16T00:29:06.210 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:29:06.210 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:29:06.210 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": { 2024-09-16T00:29:06.210 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 17.2.7-1633-g6d025bc2 (6d025bc26f3176c910707f8cb4c8d8e076640615) quincy (stable)": 6 2024-09-16T00:29:06.210 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:29:06.210 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": { 2024-09-16T00:29:06.210 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 17.2.7-1633-g6d025bc2 (6d025bc26f3176c910707f8cb4c8d8e076640615) quincy (stable)": 4 2024-09-16T00:29:06.210 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:29:06.210 INFO:teuthology.orchestra.run.smithi112.stdout: "overall": { 2024-09-16T00:29:06.210 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 17.2.7-1633-g6d025bc2 (6d025bc26f3176c910707f8cb4c8d8e076640615) quincy (stable)": 12, 2024-09-16T00:29:06.211 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:29:06.211 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-16T00:29:06.211 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:29:06.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.209+0000 7f9b2b8f4640 1 -- 172.21.15.112:0/3502975405 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f9b1006df60 msgr2=0x7f9b10070410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:06.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.209+0000 7f9b2b8f4640 1 --2- 172.21.15.112:0/3502975405 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f9b1006df60 0x7f9b10070410 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9b14006fd0 tx=0x7f9b14008040 comp rx=0 tx=0).stop 2024-09-16T00:29:06.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.209+0000 7f9b2b8f4640 1 -- 172.21.15.112:0/3502975405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b24102530 msgr2=0x7f9b2406fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:06.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.209+0000 7f9b2b8f4640 1 --2- 172.21.15.112:0/3502975405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b24102530 0x7f9b2406fa10 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f9b20000c00 tx=0x7f9b20009da0 comp rx=0 tx=0).stop 2024-09-16T00:29:06.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.209+0000 7f9b2b8f4640 1 -- 172.21.15.112:0/3502975405 shutdown_connections 2024-09-16T00:29:06.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.209+0000 7f9b2b8f4640 1 --2- 172.21.15.112:0/3502975405 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f9b1006df60 0x7f9b10070410 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:06.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.209+0000 7f9b2b8f4640 1 --2- 172.21.15.112:0/3502975405 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b24102ed0 0x7f9b2406ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:06.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.209+0000 7f9b2b8f4640 1 --2- 172.21.15.112:0/3502975405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b24102530 0x7f9b2406fa10 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:06.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.209+0000 7f9b2b8f4640 1 -- 172.21.15.112:0/3502975405 >> 172.21.15.112:0/3502975405 conn(0x7f9b240f80f0 msgr2=0x7f9b240f9140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:06.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.209+0000 7f9b2b8f4640 1 -- 172.21.15.112:0/3502975405 shutdown_connections 2024-09-16T00:29:06.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.209+0000 7f9b2b8f4640 1 -- 172.21.15.112:0/3502975405 wait complete. 2024-09-16T00:29:06.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:05 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:06.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:05 smithi169 ceph-mon[34653]: Upgrade: Target is version 19.1.1-326-g8921c8ef (squid) 2024-09-16T00:29:06.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:05 smithi169 ceph-mon[34653]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, digests ['quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80'] 2024-09-16T00:29:06.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:05 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:29:06.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:05 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:29:06.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:05 smithi169 ceph-mon[34653]: Upgrade: Setting container_image for all mgr 2024-09-16T00:29:06.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:05 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:06.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:05 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-09-16T00:29:06.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:05 smithi169 ceph-mon[34653]: pgmap v47: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:29:06.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:05 smithi169 ceph-mon[34653]: from='client.14786 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:29:06.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.295+0000 7f11c81e5640 1 -- 172.21.15.112:0/3191498294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11c0065300 msgr2=0x7f11c00656d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:06.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.295+0000 7f11c81e5640 1 --2- 172.21.15.112:0/3191498294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11c0065300 0x7f11c00656d0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f11bc0048d0 tx=0x7f11bc009da0 comp rx=0 tx=0).stop 2024-09-16T00:29:06.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.295+0000 7f11c81e5640 1 -- 172.21.15.112:0/3191498294 shutdown_connections 2024-09-16T00:29:06.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.295+0000 7f11c81e5640 1 --2- 172.21.15.112:0/3191498294 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11c0065c10 0x7f11c00fe0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:06.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.295+0000 7f11c81e5640 1 --2- 172.21.15.112:0/3191498294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11c0065300 0x7f11c00656d0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:06.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.295+0000 7f11c81e5640 1 -- 172.21.15.112:0/3191498294 >> 172.21.15.112:0/3191498294 conn(0x7f11c00723b0 msgr2=0x7f11c00727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:06.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.295+0000 7f11c81e5640 1 -- 172.21.15.112:0/3191498294 shutdown_connections 2024-09-16T00:29:06.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.295+0000 7f11c81e5640 1 -- 172.21.15.112:0/3191498294 wait complete. 2024-09-16T00:29:06.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.296+0000 7f11c81e5640 1 Processor -- start 2024-09-16T00:29:06.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.296+0000 7f11c81e5640 1 -- start start 2024-09-16T00:29:06.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.296+0000 7f11c81e5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11c0065300 0x7f11c01131d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:06.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.296+0000 7f11c81e5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11c0065c10 0x7f11c0113710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:06.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.296+0000 7f11c81e5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c0113d10 con 0x7f11c0065c10 2024-09-16T00:29:06.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.296+0000 7f11c81e5640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c010d150 con 0x7f11c0065300 2024-09-16T00:29:06.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.296+0000 7f11c5780640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11c0065c10 0x7f11c0113710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:06.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.296+0000 7f11c5780640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11c0065c10 0x7f11c0113710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36796/0 (socket says 172.21.15.112:36796) 2024-09-16T00:29:06.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.296+0000 7f11c5780640 1 -- 172.21.15.112:0/1506952263 learned_addr learned my addr 172.21.15.112:0/1506952263 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:29:06.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.297+0000 7f11c5780640 1 -- 172.21.15.112:0/1506952263 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11c0065300 msgr2=0x7f11c01131d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:06.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.297+0000 7f11c5780640 1 --2- 172.21.15.112:0/1506952263 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11c0065300 0x7f11c01131d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:06.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.297+0000 7f11c5780640 1 -- 172.21.15.112:0/1506952263 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11bc005040 con 0x7f11c0065c10 2024-09-16T00:29:06.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.297+0000 7f11c5780640 1 --2- 172.21.15.112:0/1506952263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11c0065c10 0x7f11c0113710 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f11b000eb70 tx=0x7f11b000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:06.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.297+0000 7f11b67fc640 1 -- 172.21.15.112:0/1506952263 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11b0007c30 con 0x7f11c0065c10 2024-09-16T00:29:06.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.297+0000 7f11b67fc640 1 -- 172.21.15.112:0/1506952263 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f11b000f040 con 0x7f11c0065c10 2024-09-16T00:29:06.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.297+0000 7f11b67fc640 1 -- 172.21.15.112:0/1506952263 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11b00094e0 con 0x7f11c0065c10 2024-09-16T00:29:06.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.297+0000 7f11c81e5640 1 -- 172.21.15.112:0/1506952263 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11c010d3d0 con 0x7f11c0065c10 2024-09-16T00:29:06.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.297+0000 7f11c81e5640 1 -- 172.21.15.112:0/1506952263 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11c010d920 con 0x7f11c0065c10 2024-09-16T00:29:06.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.298+0000 7f11c81e5640 1 -- 172.21.15.112:0/1506952263 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11c00fdce0 con 0x7f11c0065c10 2024-09-16T00:29:06.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.299+0000 7f11b67fc640 1 -- 172.21.15.112:0/1506952263 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 32) v1 ==== 92377+0+0 (secure 0 0 0) 0x7f11b0009660 con 0x7f11c0065c10 2024-09-16T00:29:06.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.300+0000 7f11b67fc640 1 --2- 172.21.15.112:0/1506952263 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f11ac06df60 0x7f11ac070410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:06.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.300+0000 7f11b67fc640 1 -- 172.21.15.112:0/1506952263 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6152+0+0 (secure 0 0 0) 0x7f11b008d820 con 0x7f11c0065c10 2024-09-16T00:29:06.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.300+0000 7f11c5f81640 1 --2- 172.21.15.112:0/1506952263 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f11ac06df60 0x7f11ac070410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:06.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.300+0000 7f11c5f81640 1 --2- 172.21.15.112:0/1506952263 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f11ac06df60 0x7f11ac070410 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f11bc02a530 tx=0x7f11bc009da0 comp rx=0 tx=0).ready entity=mgr.14678 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:06.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.304+0000 7f11b67fc640 1 -- 172.21.15.112:0/1506952263 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185942 (secure 0 0 0) 0x7f11b0056f40 con 0x7f11c0065c10 2024-09-16T00:29:06.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.525+0000 7f11c81e5640 1 -- 172.21.15.112:0/1506952263 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f11c010e840 con 0x7f11c0065c10 2024-09-16T00:29:06.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.526+0000 7f11b67fc640 1 -- 172.21.15.112:0/1506952263 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1865 (secure 0 0 0) 0x7f11b005a600 con 0x7f11c0065c10 2024-09-16T00:29:06.527 INFO:teuthology.orchestra.run.smithi112.stdout:e21 2024-09-16T00:29:06.527 INFO:teuthology.orchestra.run.smithi112.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T00:29:06.527 INFO:teuthology.orchestra.run.smithi112.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:29:06.527 INFO:teuthology.orchestra.run.smithi112.stdout:legacy client fscid: 1 2024-09-16T00:29:06.527 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:29:06.527 INFO:teuthology.orchestra.run.smithi112.stdout:Filesystem 'cephfs' (1) 2024-09-16T00:29:06.528 INFO:teuthology.orchestra.run.smithi112.stdout:fs_name cephfs 2024-09-16T00:29:06.528 INFO:teuthology.orchestra.run.smithi112.stdout:epoch 21 2024-09-16T00:29:06.528 INFO:teuthology.orchestra.run.smithi112.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-16T00:29:06.528 INFO:teuthology.orchestra.run.smithi112.stdout:created 2024-09-16T00:23:22.328199+0000 2024-09-16T00:29:06.528 INFO:teuthology.orchestra.run.smithi112.stdout:modified 2024-09-16T00:26:38.963153+0000 2024-09-16T00:29:06.528 INFO:teuthology.orchestra.run.smithi112.stdout:tableserver 0 2024-09-16T00:29:06.528 INFO:teuthology.orchestra.run.smithi112.stdout:root 0 2024-09-16T00:29:06.528 INFO:teuthology.orchestra.run.smithi112.stdout:session_timeout 60 2024-09-16T00:29:06.528 INFO:teuthology.orchestra.run.smithi112.stdout:session_autoclose 300 2024-09-16T00:29:06.529 INFO:teuthology.orchestra.run.smithi112.stdout:max_file_size 1099511627776 2024-09-16T00:29:06.529 INFO:teuthology.orchestra.run.smithi112.stdout:required_client_features {} 2024-09-16T00:29:06.529 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure 0 2024-09-16T00:29:06.529 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure_osd_epoch 44 2024-09-16T00:29:06.529 INFO:teuthology.orchestra.run.smithi112.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:29:06.529 INFO:teuthology.orchestra.run.smithi112.stdout:max_mds 2 2024-09-16T00:29:06.529 INFO:teuthology.orchestra.run.smithi112.stdout:in 0,1 2024-09-16T00:29:06.529 INFO:teuthology.orchestra.run.smithi112.stdout:up {0=14480,1=24259} 2024-09-16T00:29:06.529 INFO:teuthology.orchestra.run.smithi112.stdout:failed 2024-09-16T00:29:06.529 INFO:teuthology.orchestra.run.smithi112.stdout:damaged 2024-09-16T00:29:06.530 INFO:teuthology.orchestra.run.smithi112.stdout:stopped 2024-09-16T00:29:06.530 INFO:teuthology.orchestra.run.smithi112.stdout:data_pools [3] 2024-09-16T00:29:06.530 INFO:teuthology.orchestra.run.smithi112.stdout:metadata_pool 2 2024-09-16T00:29:06.530 INFO:teuthology.orchestra.run.smithi112.stdout:inline_data enabled 2024-09-16T00:29:06.530 INFO:teuthology.orchestra.run.smithi112.stdout:balancer 2024-09-16T00:29:06.530 INFO:teuthology.orchestra.run.smithi112.stdout:standby_count_wanted 1 2024-09-16T00:29:06.530 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.hxfwkh{0:14480} state up:active seq 6 export targets 1 join_fscid=1 addr [v2:172.21.15.112:6828/486654885,v1:172.21.15.112:6829/486654885] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T00:29:06.530 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.ithwzj{0:14492} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.112:6826/381830565,v1:172.21.15.112:6827/381830565] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T00:29:06.530 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.gawwgw{1:24259} state up:active seq 4 export targets 0 join_fscid=1 addr [v2:172.21.15.169:6824/2892476953,v1:172.21.15.169:6825/2892476953] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T00:29:06.530 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.lgeqrj{1:24269} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.169:6826/861725728,v1:172.21.15.169:6827/861725728] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T00:29:06.530 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:29:06.530 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:29:06.531 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 21 2024-09-16T00:29:06.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.529+0000 7f11c81e5640 1 -- 172.21.15.112:0/1506952263 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f11ac06df60 msgr2=0x7f11ac070410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:06.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.529+0000 7f11c81e5640 1 --2- 172.21.15.112:0/1506952263 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f11ac06df60 0x7f11ac070410 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f11bc02a530 tx=0x7f11bc009da0 comp rx=0 tx=0).stop 2024-09-16T00:29:06.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.529+0000 7f11c81e5640 1 -- 172.21.15.112:0/1506952263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11c0065c10 msgr2=0x7f11c0113710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:06.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.529+0000 7f11c81e5640 1 --2- 172.21.15.112:0/1506952263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11c0065c10 0x7f11c0113710 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f11b000eb70 tx=0x7f11b000b040 comp rx=0 tx=0).stop 2024-09-16T00:29:06.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.529+0000 7f11c81e5640 1 -- 172.21.15.112:0/1506952263 shutdown_connections 2024-09-16T00:29:06.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.529+0000 7f11c81e5640 1 --2- 172.21.15.112:0/1506952263 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f11ac06df60 0x7f11ac070410 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:06.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.529+0000 7f11c81e5640 1 --2- 172.21.15.112:0/1506952263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11c0065c10 0x7f11c0113710 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:06.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.529+0000 7f11c81e5640 1 --2- 172.21.15.112:0/1506952263 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11c0065300 0x7f11c01131d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:06.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.529+0000 7f11c81e5640 1 -- 172.21.15.112:0/1506952263 >> 172.21.15.112:0/1506952263 conn(0x7f11c00723b0 msgr2=0x7f11c0105420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:06.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.529+0000 7f11c81e5640 1 -- 172.21.15.112:0/1506952263 shutdown_connections 2024-09-16T00:29:06.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.529+0000 7f11c81e5640 1 -- 172.21.15.112:0/1506952263 wait complete. 2024-09-16T00:29:06.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.611+0000 7f32489cc640 1 -- 172.21.15.112:0/431501270 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32440fe460 msgr2=0x7f32440fe830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:06.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.611+0000 7f32489cc640 1 --2- 172.21.15.112:0/431501270 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32440fe460 0x7f32440fe830 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f32340048d0 tx=0x7f3234009da0 comp rx=0 tx=0).stop 2024-09-16T00:29:06.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.611+0000 7f32489cc640 1 -- 172.21.15.112:0/431501270 shutdown_connections 2024-09-16T00:29:06.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.611+0000 7f32489cc640 1 --2- 172.21.15.112:0/431501270 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3244064fc0 0x7f3244065430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:06.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.611+0000 7f32489cc640 1 --2- 172.21.15.112:0/431501270 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32440fe460 0x7f32440fe830 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:06.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.611+0000 7f32489cc640 1 -- 172.21.15.112:0/431501270 >> 172.21.15.112:0/431501270 conn(0x7f32440723b0 msgr2=0x7f32440727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:06.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.611+0000 7f32489cc640 1 -- 172.21.15.112:0/431501270 shutdown_connections 2024-09-16T00:29:06.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.611+0000 7f32489cc640 1 -- 172.21.15.112:0/431501270 wait complete. 2024-09-16T00:29:06.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.611+0000 7f32489cc640 1 Processor -- start 2024-09-16T00:29:06.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.611+0000 7f32489cc640 1 -- start start 2024-09-16T00:29:06.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.612+0000 7f32489cc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3244064fc0 0x7f32441131b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:06.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.612+0000 7f32489cc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32440fe460 0x7f32441136f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:06.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.612+0000 7f32489cc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3244113cf0 con 0x7f3244064fc0 2024-09-16T00:29:06.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.612+0000 7f32489cc640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f324410d130 con 0x7f32440fe460 2024-09-16T00:29:06.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.612+0000 7f324259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3244064fc0 0x7f32441131b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:06.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.612+0000 7f324259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3244064fc0 0x7f32441131b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36818/0 (socket says 172.21.15.112:36818) 2024-09-16T00:29:06.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.612+0000 7f324259c640 1 -- 172.21.15.112:0/876152446 learned_addr learned my addr 172.21.15.112:0/876152446 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:29:06.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.612+0000 7f3241d9b640 1 --2- 172.21.15.112:0/876152446 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32440fe460 0x7f32441136f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:06.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.612+0000 7f324259c640 1 -- 172.21.15.112:0/876152446 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32440fe460 msgr2=0x7f32441136f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:06.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.612+0000 7f324259c640 1 --2- 172.21.15.112:0/876152446 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32440fe460 0x7f32441136f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:06.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.612+0000 7f324259c640 1 -- 172.21.15.112:0/876152446 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3234005040 con 0x7f3244064fc0 2024-09-16T00:29:06.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.612+0000 7f324259c640 1 --2- 172.21.15.112:0/876152446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3244064fc0 0x7f32441131b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f3234000c00 tx=0x7f3234009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:06.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.612+0000 7f3232ffd640 1 -- 172.21.15.112:0/876152446 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3234038070 con 0x7f3244064fc0 2024-09-16T00:29:06.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.612+0000 7f3232ffd640 1 -- 172.21.15.112:0/876152446 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f323402ade0 con 0x7f3244064fc0 2024-09-16T00:29:06.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.612+0000 7f32489cc640 1 -- 172.21.15.112:0/876152446 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f324410d3b0 con 0x7f3244064fc0 2024-09-16T00:29:06.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.613+0000 7f3232ffd640 1 -- 172.21.15.112:0/876152446 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f323402c7b0 con 0x7f3244064fc0 2024-09-16T00:29:06.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.613+0000 7f32489cc640 1 -- 172.21.15.112:0/876152446 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f324410d8f0 con 0x7f3244064fc0 2024-09-16T00:29:06.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.614+0000 7f32489cc640 1 -- 172.21.15.112:0/876152446 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32440fdcf0 con 0x7f3244064fc0 2024-09-16T00:29:06.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.614+0000 7f3232ffd640 1 -- 172.21.15.112:0/876152446 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 32) v1 ==== 92377+0+0 (secure 0 0 0) 0x7f3234030020 con 0x7f3244064fc0 2024-09-16T00:29:06.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.615+0000 7f3232ffd640 1 --2- 172.21.15.112:0/876152446 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f322c06df60 0x7f322c070410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:06.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.615+0000 7f3232ffd640 1 -- 172.21.15.112:0/876152446 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6152+0+0 (secure 0 0 0) 0x7f32340add90 con 0x7f3244064fc0 2024-09-16T00:29:06.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.615+0000 7f3241d9b640 1 --2- 172.21.15.112:0/876152446 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f322c06df60 0x7f322c070410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:06.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.615+0000 7f3241d9b640 1 --2- 172.21.15.112:0/876152446 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f322c06df60 0x7f322c070410 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f324410ea70 tx=0x7f323800b040 comp rx=0 tx=0).ready entity=mgr.14678 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:06.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.619+0000 7f3232ffd640 1 -- 172.21.15.112:0/876152446 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185942 (secure 0 0 0) 0x7f3234077640 con 0x7f3244064fc0 2024-09-16T00:29:06.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.812+0000 7f32489cc640 1 -- 172.21.15.112:0/876152446 --> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f324404b720 con 0x7f322c06df60 2024-09-16T00:29:06.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.813+0000 7f3232ffd640 1 -- 172.21.15.112:0/876152446 <== mgr.14678 v2:172.21.15.112:6800/1965658616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f324404b720 con 0x7f322c06df60 2024-09-16T00:29:06.813 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:29:06.813 INFO:teuthology.orchestra.run.smithi112.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80", 2024-09-16T00:29:06.813 INFO:teuthology.orchestra.run.smithi112.stdout: "in_progress": true, 2024-09-16T00:29:06.813 INFO:teuthology.orchestra.run.smithi112.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T00:29:06.814 INFO:teuthology.orchestra.run.smithi112.stdout: "services_complete": [ 2024-09-16T00:29:06.814 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr" 2024-09-16T00:29:06.814 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-16T00:29:06.814 INFO:teuthology.orchestra.run.smithi112.stdout: "progress": "2/21 daemons upgraded", 2024-09-16T00:29:06.814 INFO:teuthology.orchestra.run.smithi112.stdout: "message": "Currently upgrading mon daemons", 2024-09-16T00:29:06.814 INFO:teuthology.orchestra.run.smithi112.stdout: "is_paused": false 2024-09-16T00:29:06.814 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:29:06.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.815+0000 7f32489cc640 1 -- 172.21.15.112:0/876152446 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f322c06df60 msgr2=0x7f322c070410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:06.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.815+0000 7f32489cc640 1 --2- 172.21.15.112:0/876152446 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f322c06df60 0x7f322c070410 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f324410ea70 tx=0x7f323800b040 comp rx=0 tx=0).stop 2024-09-16T00:29:06.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.815+0000 7f32489cc640 1 -- 172.21.15.112:0/876152446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3244064fc0 msgr2=0x7f32441131b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:06.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.815+0000 7f32489cc640 1 --2- 172.21.15.112:0/876152446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3244064fc0 0x7f32441131b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f3234000c00 tx=0x7f3234009da0 comp rx=0 tx=0).stop 2024-09-16T00:29:06.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.815+0000 7f32489cc640 1 -- 172.21.15.112:0/876152446 shutdown_connections 2024-09-16T00:29:06.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.815+0000 7f32489cc640 1 --2- 172.21.15.112:0/876152446 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7f322c06df60 0x7f322c070410 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:06.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.815+0000 7f32489cc640 1 --2- 172.21.15.112:0/876152446 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32440fe460 0x7f32441136f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:06.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.815+0000 7f32489cc640 1 --2- 172.21.15.112:0/876152446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3244064fc0 0x7f32441131b0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:06.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.815+0000 7f32489cc640 1 -- 172.21.15.112:0/876152446 >> 172.21.15.112:0/876152446 conn(0x7f32440723b0 msgr2=0x7f3244105410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:06.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.815+0000 7f32489cc640 1 -- 172.21.15.112:0/876152446 shutdown_connections 2024-09-16T00:29:06.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.815+0000 7f32489cc640 1 -- 172.21.15.112:0/876152446 wait complete. 2024-09-16T00:29:06.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.894+0000 7fba0e42a640 1 -- 172.21.15.112:0/4203686052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba080fac80 msgr2=0x7fba080fb0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:06.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.894+0000 7fba0e42a640 1 --2- 172.21.15.112:0/4203686052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba080fac80 0x7fba080fb0d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc0048d0 tx=0x7fb9fc009da0 comp rx=0 tx=0).stop 2024-09-16T00:29:06.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.894+0000 7fba0e42a640 1 -- 172.21.15.112:0/4203686052 shutdown_connections 2024-09-16T00:29:06.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.894+0000 7fba0e42a640 1 --2- 172.21.15.112:0/4203686052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba080fac80 0x7fba080fb0d0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:06.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.894+0000 7fba0e42a640 1 --2- 172.21.15.112:0/4203686052 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba08102660 0x7fba08102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:06.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.894+0000 7fba0e42a640 1 -- 172.21.15.112:0/4203686052 >> 172.21.15.112:0/4203686052 conn(0x7fba08070ce0 msgr2=0x7fba080710e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:06.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.894+0000 7fba0e42a640 1 -- 172.21.15.112:0/4203686052 shutdown_connections 2024-09-16T00:29:06.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.894+0000 7fba0e42a640 1 -- 172.21.15.112:0/4203686052 wait complete. 2024-09-16T00:29:06.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.894+0000 7fba0e42a640 1 Processor -- start 2024-09-16T00:29:06.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.895+0000 7fba0e42a640 1 -- start start 2024-09-16T00:29:06.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.895+0000 7fba0e42a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba080fac80 0x7fba080fbe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:06.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.895+0000 7fba0e42a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba08102660 0x7fba080fc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:06.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.895+0000 7fba0e42a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba080fc930 con 0x7fba080fac80 2024-09-16T00:29:06.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.895+0000 7fba0e42a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba080fcaa0 con 0x7fba08102660 2024-09-16T00:29:06.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.895+0000 7fba07fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba080fac80 0x7fba080fbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:06.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.895+0000 7fba07fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba080fac80 0x7fba080fbe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36834/0 (socket says 172.21.15.112:36834) 2024-09-16T00:29:06.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.895+0000 7fba07fff640 1 -- 172.21.15.112:0/894574392 learned_addr learned my addr 172.21.15.112:0/894574392 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:29:06.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.895+0000 7fba077fe640 1 --2- 172.21.15.112:0/894574392 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba08102660 0x7fba080fc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:06.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.895+0000 7fba07fff640 1 -- 172.21.15.112:0/894574392 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba08102660 msgr2=0x7fba080fc3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:06.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.895+0000 7fba07fff640 1 --2- 172.21.15.112:0/894574392 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba08102660 0x7fba080fc3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:06.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.895+0000 7fba07fff640 1 -- 172.21.15.112:0/894574392 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9fc005040 con 0x7fba080fac80 2024-09-16T00:29:06.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.895+0000 7fba077fe640 1 --2- 172.21.15.112:0/894574392 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba08102660 0x7fba080fc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:29:06.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.895+0000 7fba07fff640 1 --2- 172.21.15.112:0/894574392 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba080fac80 0x7fba080fbe60 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8002a50 tx=0x7fb9f8004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:06.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.896+0000 7fba04ff9640 1 -- 172.21.15.112:0/894574392 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9f8007c30 con 0x7fba080fac80 2024-09-16T00:29:06.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.896+0000 7fba04ff9640 1 -- 172.21.15.112:0/894574392 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fb9f800f040 con 0x7fba080fac80 2024-09-16T00:29:06.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.896+0000 7fba0e42a640 1 -- 172.21.15.112:0/894574392 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba08119510 con 0x7fba080fac80 2024-09-16T00:29:06.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.896+0000 7fba04ff9640 1 -- 172.21.15.112:0/894574392 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9f800da90 con 0x7fba080fac80 2024-09-16T00:29:06.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.896+0000 7fba0e42a640 1 -- 172.21.15.112:0/894574392 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba08119950 con 0x7fba080fac80 2024-09-16T00:29:06.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.897+0000 7fba0e42a640 1 -- 172.21.15.112:0/894574392 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba080657d0 con 0x7fba080fac80 2024-09-16T00:29:06.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.897+0000 7fba04ff9640 1 -- 172.21.15.112:0/894574392 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 32) v1 ==== 92377+0+0 (secure 0 0 0) 0x7fb9f800dd00 con 0x7fba080fac80 2024-09-16T00:29:06.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.898+0000 7fba04ff9640 1 --2- 172.21.15.112:0/894574392 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7fb9f006df60 0x7fb9f0070410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:06.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.898+0000 7fba04ff9640 1 -- 172.21.15.112:0/894574392 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6152+0+0 (secure 0 0 0) 0x7fb9f808d380 con 0x7fba080fac80 2024-09-16T00:29:06.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.898+0000 7fba077fe640 1 --2- 172.21.15.112:0/894574392 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7fb9f006df60 0x7fb9f0070410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:06.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.899+0000 7fba077fe640 1 --2- 172.21.15.112:0/894574392 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7fb9f006df60 0x7fb9f0070410 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fba080fd7a0 tx=0x7fb9fc035040 comp rx=0 tx=0).ready entity=mgr.14678 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:06.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:06.903+0000 7fba04ff9640 1 -- 172.21.15.112:0/894574392 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185942 (secure 0 0 0) 0x7fb9f8056b10 con 0x7fba080fac80 2024-09-16T00:29:07.058 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:06 smithi112 ceph-mon[27869]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:29:07.058 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:06 smithi112 ceph-mon[27869]: from='client.14794 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:29:07.058 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:06 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/3502975405' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:29:07.058 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:06 smithi112 ceph-mon[27869]: Upgrade: Updating mon.smithi112 2024-09-16T00:29:07.058 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:06 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:07.058 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:06 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T00:29:07.058 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:06 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T00:29:07.058 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:06 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:07.058 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:06 smithi112 ceph-mon[27869]: Deploying daemon mon.smithi112 on smithi112 2024-09-16T00:29:07.059 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:06 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/1506952263' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:29:07.146 INFO:teuthology.orchestra.run.smithi112.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-09-16T00:29:07.146 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T00:29:07.146 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T00:29:07.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:07.145+0000 7fba0e42a640 1 -- 172.21.15.112:0/894574392 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fba080fd550 con 0x7fba080fac80 2024-09-16T00:29:07.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:07.146+0000 7fba04ff9640 1 -- 172.21.15.112:0/894574392 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fb9f805a1d0 con 0x7fba080fac80 2024-09-16T00:29:07.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:07.147+0000 7fba0e42a640 1 -- 172.21.15.112:0/894574392 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7fb9f006df60 msgr2=0x7fb9f0070410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:07.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:07.147+0000 7fba0e42a640 1 --2- 172.21.15.112:0/894574392 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7fb9f006df60 0x7fb9f0070410 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fba080fd7a0 tx=0x7fb9fc035040 comp rx=0 tx=0).stop 2024-09-16T00:29:07.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:07.147+0000 7fba0e42a640 1 -- 172.21.15.112:0/894574392 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba080fac80 msgr2=0x7fba080fbe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:07.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:07.147+0000 7fba0e42a640 1 --2- 172.21.15.112:0/894574392 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba080fac80 0x7fba080fbe60 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8002a50 tx=0x7fb9f8004600 comp rx=0 tx=0).stop 2024-09-16T00:29:07.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:07.147+0000 7fba0e42a640 1 -- 172.21.15.112:0/894574392 shutdown_connections 2024-09-16T00:29:07.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:07.147+0000 7fba0e42a640 1 --2- 172.21.15.112:0/894574392 >> [v2:172.21.15.112:6800/1965658616,v1:172.21.15.112:6801/1965658616] conn(0x7fb9f006df60 0x7fb9f0070410 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:07.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:07.147+0000 7fba0e42a640 1 --2- 172.21.15.112:0/894574392 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba08102660 0x7fba080fc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:07.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:07.147+0000 7fba0e42a640 1 --2- 172.21.15.112:0/894574392 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba080fac80 0x7fba080fbe60 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:07.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:07.148+0000 7fba0e42a640 1 -- 172.21.15.112:0/894574392 >> 172.21.15.112:0/894574392 conn(0x7fba08070ce0 msgr2=0x7fba080fa0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:07.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:07.148+0000 7fba0e42a640 1 -- 172.21.15.112:0/894574392 shutdown_connections 2024-09-16T00:29:07.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:07.148+0000 7fba0e42a640 1 -- 172.21.15.112:0/894574392 wait complete. 2024-09-16T00:29:07.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:06 smithi169 ceph-mon[34653]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:29:07.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:06 smithi169 ceph-mon[34653]: from='client.14794 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:29:07.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:06 smithi169 ceph-mon[34653]: from='client.? 172.21.15.112:0/3502975405' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:29:07.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:06 smithi169 ceph-mon[34653]: Upgrade: Updating mon.smithi112 2024-09-16T00:29:07.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:06 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:07.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:06 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T00:29:07.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:06 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T00:29:07.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:06 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:07.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:06 smithi169 ceph-mon[34653]: Deploying daemon mon.smithi112 on smithi112 2024-09-16T00:29:07.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:06 smithi169 ceph-mon[34653]: from='client.? 172.21.15.112:0/1506952263' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:29:08.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:07 smithi112 ceph-mon[27869]: from='client.14806 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:29:08.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:07 smithi112 ceph-mon[27869]: pgmap v48: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:29:08.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:07 smithi112 ceph-mon[27869]: from='client.? 172.21.15.112:0/894574392' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:29:08.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:07 smithi112 ceph-mon[27869]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:29:08.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:07 smithi169 ceph-mon[34653]: from='client.14806 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:29:08.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:07 smithi169 ceph-mon[34653]: pgmap v48: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:29:08.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:07 smithi169 ceph-mon[34653]: from='client.? 172.21.15.112:0/894574392' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:29:08.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:07 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:29:08.459 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:08 smithi112 systemd[1]: Stopping Ceph mon.smithi112 for d346dd56-73c0-11ef-bceb-c7b262605968... 2024-09-16T00:29:08.762 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:08 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-mon-smithi112[27865]: 2024-09-16T00:29:08.457+0000 7f0cd63b0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi112 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:29:08.762 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:08 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-mon-smithi112[27865]: 2024-09-16T00:29:08.457+0000 7f0cd63b0640 -1 mon.smithi112@0(leader) e2 *** Got Signal Terminated *** 2024-09-16T00:29:08.762 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:08 smithi112 podman[97582]: 2024-09-16 00:29:08.653115652 +0000 UTC m=+0.301184712 container died ddaaced54bf8f3f21ac04a1d448f7340d5b51a8a869c703e359c81b1e715d401 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-mon-smithi112, RELEASE=quincy-6d025bc, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-16T00:29:09.511 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:09 smithi112 podman[97582]: 2024-09-16 00:29:09.159222507 +0000 UTC m=+0.807291567 container cleanup ddaaced54bf8f3f21ac04a1d448f7340d5b51a8a869c703e359c81b1e715d401 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-mon-smithi112, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=quincy-6d025bc, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True) 2024-09-16T00:29:09.511 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:09 smithi112 bash[97582]: ceph-d346dd56-73c0-11ef-bceb-c7b262605968-mon-smithi112 2024-09-16T00:29:09.511 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:09 smithi112 podman[97594]: 2024-09-16 00:29:09.385830829 +0000 UTC m=+0.726897760 container remove ddaaced54bf8f3f21ac04a1d448f7340d5b51a8a869c703e359c81b1e715d401 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-mon-smithi112, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-6d025bc, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-16T00:29:09.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:09 smithi112 systemd[1]: ceph-d346dd56-73c0-11ef-bceb-c7b262605968@mon.smithi112.service: Deactivated successfully. 2024-09-16T00:29:09.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:09 smithi112 systemd[1]: Stopped Ceph mon.smithi112 for d346dd56-73c0-11ef-bceb-c7b262605968. 2024-09-16T00:29:09.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:09 smithi112 systemd[1]: ceph-d346dd56-73c0-11ef-bceb-c7b262605968@mon.smithi112.service: Consumed 10.929s CPU time. 2024-09-16T00:29:09.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:09 smithi112 systemd[1]: Starting Ceph mon.smithi112 for d346dd56-73c0-11ef-bceb-c7b262605968... 2024-09-16T00:29:10.214 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:09 smithi112 podman[97731]: 2024-09-16 00:29:09.848203464 +0000 UTC m=+0.015893325 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:29:10.214 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:09 smithi112 podman[97731]: 2024-09-16 00:29:09.961960116 +0000 UTC m=+0.129650026 container create 5aaec033c042c58ff4bf136853bbb4200f4a68bd6285561dcbfd6effed675cef (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-mon-smithi112, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-16T00:29:10.214 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 podman[97731]: 2024-09-16 00:29:10.190016007 +0000 UTC m=+0.357705869 container init 5aaec033c042c58ff4bf136853bbb4200f4a68bd6285561dcbfd6effed675cef (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-mon-smithi112, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:29:10.214 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 podman[97731]: 2024-09-16 00:29:10.193452183 +0000 UTC m=+0.361142044 container start 5aaec033c042c58ff4bf136853bbb4200f4a68bd6285561dcbfd6effed675cef (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-mon-smithi112, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-16T00:29:10.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: set uid:gid to 167:167 (ceph:ceph) 2024-09-16T00:29:10.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable), process ceph-mon, pid 2 2024-09-16T00:29:10.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: pidfile_write: ignore empty --pid-file 2024-09-16T00:29:10.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: load: jerasure load: lrc 2024-09-16T00:29:10.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: RocksDB version: 7.9.2 2024-09-16T00:29:10.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Git sha 0 2024-09-16T00:29:10.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Compile date 2024-09-15 15:04:35 2024-09-16T00:29:10.591 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: DB SUMMARY 2024-09-16T00:29:10.591 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: DB Session ID: F12IXZFL97QJ2NK52OFO 2024-09-16T00:29:10.591 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: CURRENT file: CURRENT 2024-09-16T00:29:10.591 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: IDENTITY file: IDENTITY 2024-09-16T00:29:10.591 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: MANIFEST file: MANIFEST-000015 size: 1180 Bytes 2024-09-16T00:29:10.591 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi112/store.db dir, Total Num: 1, files: 000033.sst 2024-09-16T00:29:10.591 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi112/store.db: 000031.log size: 293057 ; 2024-09-16T00:29:10.591 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.error_if_exists: 0 2024-09-16T00:29:10.591 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.create_if_missing: 0 2024-09-16T00:29:10.592 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.paranoid_checks: 1 2024-09-16T00:29:10.592 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-16T00:29:10.592 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-16T00:29:10.592 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-16T00:29:10.592 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.env: 0x5638c0b9d940 2024-09-16T00:29:10.592 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.fs: PosixFileSystem 2024-09-16T00:29:10.592 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.info_log: 0x5638c28cf8c0 2024-09-16T00:29:10.592 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.max_file_opening_threads: 16 2024-09-16T00:29:10.592 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.statistics: (nil) 2024-09-16T00:29:10.592 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.use_fsync: 0 2024-09-16T00:29:10.592 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.max_log_file_size: 0 2024-09-16T00:29:10.592 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-16T00:29:10.593 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-16T00:29:10.593 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.keep_log_file_num: 1000 2024-09-16T00:29:10.593 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.recycle_log_file_num: 0 2024-09-16T00:29:10.593 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.allow_fallocate: 1 2024-09-16T00:29:10.593 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.allow_mmap_reads: 0 2024-09-16T00:29:10.593 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.allow_mmap_writes: 0 2024-09-16T00:29:10.593 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.use_direct_reads: 0 2024-09-16T00:29:10.593 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-16T00:29:10.593 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.create_missing_column_families: 0 2024-09-16T00:29:10.593 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.db_log_dir: 2024-09-16T00:29:10.593 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.wal_dir: 2024-09-16T00:29:10.593 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-16T00:29:10.593 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-16T00:29:10.594 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-16T00:29:10.594 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-16T00:29:10.594 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-16T00:29:10.594 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-16T00:29:10.594 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.advise_random_on_open: 1 2024-09-16T00:29:10.594 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.db_write_buffer_size: 0 2024-09-16T00:29:10.594 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.write_buffer_manager: 0x5638c28dedc0 2024-09-16T00:29:10.594 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-16T00:29:10.594 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-16T00:29:10.595 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-16T00:29:10.595 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.rate_limiter: (nil) 2024-09-16T00:29:10.595 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-16T00:29:10.595 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.wal_recovery_mode: 2 2024-09-16T00:29:10.595 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.enable_thread_tracking: 0 2024-09-16T00:29:10.595 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.enable_pipelined_write: 0 2024-09-16T00:29:10.595 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.unordered_write: 0 2024-09-16T00:29:10.595 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-16T00:29:10.595 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-16T00:29:10.595 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-16T00:29:10.595 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-16T00:29:10.595 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.row_cache: None 2024-09-16T00:29:10.595 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.wal_filter: None 2024-09-16T00:29:10.595 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-16T00:29:10.596 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.allow_ingest_behind: 0 2024-09-16T00:29:10.596 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.two_write_queues: 0 2024-09-16T00:29:10.596 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.manual_wal_flush: 0 2024-09-16T00:29:10.596 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.wal_compression: 0 2024-09-16T00:29:10.596 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.atomic_flush: 0 2024-09-16T00:29:10.596 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-16T00:29:10.596 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-16T00:29:10.596 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-16T00:29:10.596 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.log_readahead_size: 0 2024-09-16T00:29:10.596 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-16T00:29:10.596 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.best_efforts_recovery: 0 2024-09-16T00:29:10.596 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-16T00:29:10.596 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-16T00:29:10.596 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.allow_data_in_errors: 0 2024-09-16T00:29:10.597 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.db_host_id: __hostname__ 2024-09-16T00:29:10.598 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-16T00:29:10.598 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.max_background_jobs: 2 2024-09-16T00:29:10.598 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.max_background_compactions: -1 2024-09-16T00:29:10.598 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.max_subcompactions: 1 2024-09-16T00:29:10.598 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-16T00:29:10.598 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-16T00:29:10.598 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-16T00:29:10.598 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.max_total_wal_size: 0 2024-09-16T00:29:10.598 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-16T00:29:10.598 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-16T00:29:10.599 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-16T00:29:10.599 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-16T00:29:10.599 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.max_open_files: -1 2024-09-16T00:29:10.599 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.bytes_per_sync: 0 2024-09-16T00:29:10.599 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-16T00:29:10.599 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-16T00:29:10.599 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.compaction_readahead_size: 0 2024-09-16T00:29:10.599 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.max_background_flushes: -1 2024-09-16T00:29:10.599 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Compression algorithms supported: 2024-09-16T00:29:10.599 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: kZSTD supported: 0 2024-09-16T00:29:10.599 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: kXpressCompression supported: 0 2024-09-16T00:29:10.599 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: kBZip2Compression supported: 0 2024-09-16T00:29:10.599 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-16T00:29:10.600 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: kLZ4Compression supported: 1 2024-09-16T00:29:10.600 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: kZlibCompression supported: 1 2024-09-16T00:29:10.600 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: kLZ4HCCompression supported: 1 2024-09-16T00:29:10.600 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: kSnappyCompression supported: 1 2024-09-16T00:29:10.600 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-16T00:29:10.600 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-16T00:29:10.600 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi112/store.db/MANIFEST-000015 2024-09-16T00:29:10.600 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-16T00:29:10.600 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-16T00:29:10.600 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.merge_operator: 2024-09-16T00:29:10.600 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.compaction_filter: None 2024-09-16T00:29:10.600 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.compaction_filter_factory: None 2024-09-16T00:29:10.600 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.sst_partitioner_factory: None 2024-09-16T00:29:10.600 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-16T00:29:10.600 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-16T00:29:10.601 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5638c28cf520) 2024-09-16T00:29:10.601 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: cache_index_and_filter_blocks: 1 2024-09-16T00:29:10.601 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-16T00:29:10.601 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-16T00:29:10.601 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: pin_top_level_index_and_filter: 1 2024-09-16T00:29:10.601 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: index_type: 0 2024-09-16T00:29:10.601 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: data_block_index_type: 0 2024-09-16T00:29:10.601 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: index_shortening: 1 2024-09-16T00:29:10.602 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-16T00:29:10.602 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: checksum: 4 2024-09-16T00:29:10.602 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: no_block_cache: 0 2024-09-16T00:29:10.602 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: block_cache: 0x5638c28c31f0 2024-09-16T00:29:10.602 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: block_cache_name: BinnedLRUCache 2024-09-16T00:29:10.602 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: block_cache_options: 2024-09-16T00:29:10.602 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: capacity : 536870912 2024-09-16T00:29:10.602 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: num_shard_bits : 4 2024-09-16T00:29:10.602 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: strict_capacity_limit : 0 2024-09-16T00:29:10.602 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: high_pri_pool_ratio: 0.000 2024-09-16T00:29:10.602 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: block_cache_compressed: (nil) 2024-09-16T00:29:10.602 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: persistent_cache: (nil) 2024-09-16T00:29:10.603 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: block_size: 4096 2024-09-16T00:29:10.603 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: block_size_deviation: 10 2024-09-16T00:29:10.603 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: block_restart_interval: 16 2024-09-16T00:29:10.603 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: index_block_restart_interval: 1 2024-09-16T00:29:10.603 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: metadata_block_size: 4096 2024-09-16T00:29:10.603 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: partition_filters: 0 2024-09-16T00:29:10.603 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: use_delta_encoding: 1 2024-09-16T00:29:10.603 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: filter_policy: bloomfilter 2024-09-16T00:29:10.603 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: whole_key_filtering: 1 2024-09-16T00:29:10.603 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: verify_compression: 0 2024-09-16T00:29:10.603 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: read_amp_bytes_per_bit: 0 2024-09-16T00:29:10.603 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: format_version: 5 2024-09-16T00:29:10.603 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: enable_index_compression: 1 2024-09-16T00:29:10.604 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: block_align: 0 2024-09-16T00:29:10.604 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: max_auto_readahead_size: 262144 2024-09-16T00:29:10.604 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: prepopulate_block_cache: 0 2024-09-16T00:29:10.604 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: initial_auto_readahead_size: 8192 2024-09-16T00:29:10.604 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: num_file_reads_for_auto_readahead: 2 2024-09-16T00:29:10.604 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.write_buffer_size: 33554432 2024-09-16T00:29:10.604 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.max_write_buffer_number: 2 2024-09-16T00:29:10.604 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.compression: NoCompression 2024-09-16T00:29:10.604 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.bottommost_compression: Disabled 2024-09-16T00:29:10.604 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.prefix_extractor: nullptr 2024-09-16T00:29:10.604 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-16T00:29:10.604 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.num_levels: 7 2024-09-16T00:29:10.604 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-16T00:29:10.604 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-16T00:29:10.605 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-16T00:29:10.605 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-16T00:29:10.605 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-16T00:29:10.605 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-16T00:29:10.605 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-16T00:29:10.605 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-16T00:29:10.605 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-16T00:29:10.605 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-16T00:29:10.605 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-16T00:29:10.605 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-16T00:29:10.605 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-16T00:29:10.606 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.compression_opts.level: 32767 2024-09-16T00:29:10.606 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.compression_opts.strategy: 0 2024-09-16T00:29:10.606 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-16T00:29:10.606 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-16T00:29:10.606 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-16T00:29:10.606 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-16T00:29:10.606 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.compression_opts.enabled: false 2024-09-16T00:29:10.606 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-16T00:29:10.606 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-16T00:29:10.606 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-16T00:29:10.606 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-16T00:29:10.606 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.target_file_size_base: 67108864 2024-09-16T00:29:10.606 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-16T00:29:10.606 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-16T00:29:10.607 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-16T00:29:10.607 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-16T00:29:10.607 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-16T00:29:10.607 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-16T00:29:10.607 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-16T00:29:10.607 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-16T00:29:10.607 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-16T00:29:10.607 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-16T00:29:10.607 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-16T00:29:10.607 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-16T00:29:10.607 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-16T00:29:10.607 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-16T00:29:10.607 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.arena_block_size: 1048576 2024-09-16T00:29:10.607 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-16T00:29:10.607 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-16T00:29:10.608 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.disable_auto_compactions: 0 2024-09-16T00:29:10.608 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-16T00:29:10.608 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-16T00:29:10.608 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-16T00:29:10.608 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-16T00:29:10.608 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-16T00:29:10.608 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-16T00:29:10.608 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-16T00:29:10.608 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-16T00:29:10.608 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-16T00:29:10.608 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-16T00:29:10.608 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-16T00:29:10.608 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.inplace_update_support: 0 2024-09-16T00:29:10.608 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-16T00:29:10.608 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-16T00:29:10.609 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-16T00:29:10.609 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-16T00:29:10.609 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.bloom_locality: 0 2024-09-16T00:29:10.609 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.max_successive_merges: 0 2024-09-16T00:29:10.609 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-16T00:29:10.609 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.paranoid_file_checks: 0 2024-09-16T00:29:10.609 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.force_consistency_checks: 1 2024-09-16T00:29:10.609 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.report_bg_io_stats: 0 2024-09-16T00:29:10.609 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.ttl: 2592000 2024-09-16T00:29:10.609 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-16T00:29:10.609 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-16T00:29:10.609 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-16T00:29:10.609 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.enable_blob_files: false 2024-09-16T00:29:10.609 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.min_blob_size: 0 2024-09-16T00:29:10.609 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.blob_file_size: 268435456 2024-09-16T00:29:10.609 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-16T00:29:10.610 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-16T00:29:10.610 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-16T00:29:10.610 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-16T00:29:10.610 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-16T00:29:10.610 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.blob_file_starting_level: 0 2024-09-16T00:29:10.610 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-16T00:29:10.610 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: [table/block_based/block_based_table_reader.cc:721] At least one SST file opened without unique ID to verify: 33.sst 2024-09-16T00:29:10.610 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-09-16T00:29:10.610 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi112/store.db/MANIFEST-000015 succeeded,manifest_file_number is 15, next_file_number is 35, last_sequence is 12074, log_number is 31,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-16T00:29:10.610 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 31 2024-09-16T00:29:10.610 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 398da636-9c8c-42fa-8d49-90bdbe1a55a9 2024-09-16T00:29:10.610 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726446550220204, "job": 1, "event": "recovery_started", "wal_files": [31]} 2024-09-16T00:29:10.610 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #31 mode 2 2024-09-16T00:29:10.611 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726446550222037, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 36, "file_size": 287147, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12075, "largest_seqno": 12471, "table_properties": {"data_size": 284484, "index_size": 1111, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 517, "raw_key_size": 4436, "raw_average_key_size": 23, "raw_value_size": 280458, "raw_average_value_size": 1515, "num_data_blocks": 49, "num_entries": 185, "num_filter_entries": 185, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726446550, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "398da636-9c8c-42fa-8d49-90bdbe1a55a9", "db_session_id": "F12IXZFL97QJ2NK52OFO", "orig_file_number": 36, "seqno_to_time_mapping": "N/A"}} 2024-09-16T00:29:10.611 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726446550222146, "job": 1, "event": "recovery_finished"} 2024-09-16T00:29:10.611 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: [db/version_set.cc:5047] Creating manifest 38 2024-09-16T00:29:10.611 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-09-16T00:29:10.611 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi112/store.db/000031.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-16T00:29:10.611 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5638c28f0e00 2024-09-16T00:29:10.611 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: DB pointer 0x5638c29ec000 2024-09-16T00:29:10.611 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-16T00:29:10.611 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-16T00:29:10.611 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: ** DB Stats ** 2024-09-16T00:29:10.611 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T00:29:10.611 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-16T00:29:10.611 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T00:29:10.611 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T00:29:10.611 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-16T00:29:10.612 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T00:29:10.612 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T00:29:10.612 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: 2024-09-16T00:29:10.612 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: ** Compaction Stats [default] ** 2024-09-16T00:29:10.612 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T00:29:10.612 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-16T00:29:10.612 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: L0 1/0 280.42 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 227.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-16T00:29:10.612 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: L6 1/0 6.87 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-09-16T00:29:10.612 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: Sum 2/0 7.14 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 227.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-16T00:29:10.612 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 227.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-16T00:29:10.612 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: 2024-09-16T00:29:10.612 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: ** Compaction Stats [default] ** 2024-09-16T00:29:10.612 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T00:29:10.612 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-16T00:29:10.612 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 227.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-16T00:29:10.612 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: 2024-09-16T00:29:10.613 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-16T00:29:10.613 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: 2024-09-16T00:29:10.613 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T00:29:10.613 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-16T00:29:10.613 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-16T00:29:10.613 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-16T00:29:10.613 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-16T00:29:10.613 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-16T00:29:10.613 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: Cumulative compaction: 0.00 GB write, 44.20 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T00:29:10.613 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: Interval compaction: 0.00 GB write, 44.20 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T00:29:10.614 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-16T00:29:10.614 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: Block cache BinnedLRUCache@0x5638c28c31f0#2 capacity: 512.00 MB usage: 1.80 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-16T00:29:10.614 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.59 KB,0.000113249%) IndexBlock(1,1.20 KB,0.000229478%) Misc(1,0.00 KB,0%) 2024-09-16T00:29:10.614 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: 2024-09-16T00:29:10.614 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-16T00:29:10.614 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: starting mon.smithi112 rank 0 at public addrs [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] at bind addrs [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi112 fsid d346dd56-73c0-11ef-bceb-c7b262605968 2024-09-16T00:29:10.614 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: mon.smithi112@-1(???) e2 preinit fsid d346dd56-73c0-11ef-bceb-c7b262605968 2024-09-16T00:29:10.614 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: mon.smithi112@-1(???).mds e21 new map 2024-09-16T00:29:10.614 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: mon.smithi112@-1(???).mds e21 print_map 2024-09-16T00:29:10.614 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: e21 2024-09-16T00:29:10.614 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: btime 1970-01-01T00:00:00:000000+0000 2024-09-16T00:29:10.614 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T00:29:10.614 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:29:10.614 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: legacy client fscid: 1 2024-09-16T00:29:10.614 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: 2024-09-16T00:29:10.615 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: Filesystem 'cephfs' (1) 2024-09-16T00:29:10.615 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: fs_name cephfs 2024-09-16T00:29:10.615 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: epoch 21 2024-09-16T00:29:10.615 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-16T00:29:10.615 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: created 2024-09-16T00:23:22.328199+0000 2024-09-16T00:29:10.615 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: modified 2024-09-16T00:26:38.963153+0000 2024-09-16T00:29:10.615 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: tableserver 0 2024-09-16T00:29:10.615 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: root 0 2024-09-16T00:29:10.615 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: session_timeout 60 2024-09-16T00:29:10.615 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: session_autoclose 300 2024-09-16T00:29:10.615 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: max_file_size 1099511627776 2024-09-16T00:29:10.615 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: max_xattr_size 65536 2024-09-16T00:29:10.615 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: required_client_features {} 2024-09-16T00:29:10.615 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: last_failure 0 2024-09-16T00:29:10.615 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: last_failure_osd_epoch 44 2024-09-16T00:29:10.616 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:29:10.616 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: max_mds 2 2024-09-16T00:29:10.616 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: in 0,1 2024-09-16T00:29:10.616 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: up {0=14480,1=24259} 2024-09-16T00:29:10.616 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: failed 2024-09-16T00:29:10.616 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: damaged 2024-09-16T00:29:10.616 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: stopped 2024-09-16T00:29:10.616 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: data_pools [3] 2024-09-16T00:29:10.616 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: metadata_pool 2 2024-09-16T00:29:10.616 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: inline_data enabled 2024-09-16T00:29:10.616 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: balancer 2024-09-16T00:29:10.616 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: bal_rank_mask -1 2024-09-16T00:29:10.617 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: standby_count_wanted 1 2024-09-16T00:29:10.617 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: qdb_cluster leader: 0 members: 2024-09-16T00:29:10.617 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: [mds.cephfs.smithi112.hxfwkh{0:14480} state up:active seq 6 export targets 1 join_fscid=1 addr [v2:172.21.15.112:6828/486654885,v1:172.21.15.112:6829/486654885] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T00:29:10.617 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: [mds.cephfs.smithi112.ithwzj{0:14492} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.112:6826/381830565,v1:172.21.15.112:6827/381830565] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T00:29:10.617 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: [mds.cephfs.smithi169.gawwgw{1:24259} state up:active seq 4 export targets 0 join_fscid=1 addr [v2:172.21.15.169:6824/2892476953,v1:172.21.15.169:6825/2892476953] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T00:29:10.617 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: [mds.cephfs.smithi169.lgeqrj{1:24269} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.169:6826/861725728,v1:172.21.15.169:6827/861725728] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T00:29:10.617 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: 2024-09-16T00:29:10.617 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: 2024-09-16T00:29:10.617 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: mon.smithi112@-1(???).osd e46 crush map has features 3314933000852226048, adjusting msgr requires 2024-09-16T00:29:10.617 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: mon.smithi112@-1(???).osd e46 crush map has features 288514051259236352, adjusting msgr requires 2024-09-16T00:29:10.617 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: mon.smithi112@-1(???).osd e46 crush map has features 288514051259236352, adjusting msgr requires 2024-09-16T00:29:10.617 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: mon.smithi112@-1(???).osd e46 crush map has features 288514051259236352, adjusting msgr requires 2024-09-16T00:29:10.617 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 ceph-mon[97750]: mon.smithi112@-1(???).paxosservice(auth 1..21) refresh upgraded, format 0 -> 3 2024-09-16T00:29:10.618 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 bash[97731]: 5aaec033c042c58ff4bf136853bbb4200f4a68bd6285561dcbfd6effed675cef 2024-09-16T00:29:10.618 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:10 smithi112 systemd[1]: Started Ceph mon.smithi112 for d346dd56-73c0-11ef-bceb-c7b262605968. 2024-09-16T00:29:12.009 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:11 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mon metadata", "id": "smithi112"}]: dispatch 2024-09-16T00:29:12.009 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:11 smithi169 ceph-mon[34653]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mon metadata", "id": "smithi169"}]: dispatch 2024-09-16T00:29:12.009 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:11 smithi169 ceph-mon[34653]: pgmap v49: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 511 B/s wr, 5 op/s 2024-09-16T00:29:12.009 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:11 smithi169 ceph-mon[34653]: pgmap v50: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 3 op/s 2024-09-16T00:29:12.009 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:11 smithi169 ceph-mon[34653]: mon.smithi112 calling monitor election 2024-09-16T00:29:12.009 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:11 smithi169 ceph-mon[34653]: mon.smithi112 is new leader, mons smithi112,smithi169 in quorum (ranks 0,1) 2024-09-16T00:29:12.009 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:11 smithi169 ceph-mon[34653]: monmap epoch 2 2024-09-16T00:29:12.009 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:11 smithi169 ceph-mon[34653]: fsid d346dd56-73c0-11ef-bceb-c7b262605968 2024-09-16T00:29:12.009 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:11 smithi169 ceph-mon[34653]: last_changed 2024-09-16T00:19:02.487171+0000 2024-09-16T00:29:12.009 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:11 smithi169 ceph-mon[34653]: created 2024-09-16T00:16:40.582056+0000 2024-09-16T00:29:12.010 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:11 smithi169 ceph-mon[34653]: min_mon_release 17 (quincy) 2024-09-16T00:29:12.010 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:11 smithi169 ceph-mon[34653]: election_strategy: 1 2024-09-16T00:29:12.010 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:11 smithi169 ceph-mon[34653]: 0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.smithi112 2024-09-16T00:29:12.010 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:11 smithi169 ceph-mon[34653]: 1: [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] mon.smithi169 2024-09-16T00:29:12.010 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:11 smithi169 ceph-mon[34653]: fsmap cephfs:2 {0=cephfs.smithi112.hxfwkh=up:active,1=cephfs.smithi169.gawwgw=up:active} 2 up:standby-replay 2024-09-16T00:29:12.010 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:11 smithi169 ceph-mon[34653]: osdmap e46: 6 total, 6 up, 6 in 2024-09-16T00:29:12.010 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:11 smithi169 ceph-mon[34653]: mgrmap e32: smithi112.xrkkbc(active, since 94s), standbys: smithi169.mtrobk 2024-09-16T00:29:12.010 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:11 smithi169 ceph-mon[34653]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-09-16T00:29:12.010 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:11 smithi169 ceph-mon[34653]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T00:29:12.010 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:11 smithi169 ceph-mon[34653]: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T00:29:12.010 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:11 smithi169 ceph-mon[34653]: from='mgr.14678 ' entity='' 2024-09-16T00:29:12.010 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:11 smithi169 ceph-mon[34653]: mgrmap e33: smithi112.xrkkbc(active, since 94s), standbys: smithi169.mtrobk 2024-09-16T00:29:12.019 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:11 smithi112 ceph-mon[97750]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mon metadata", "id": "smithi112"}]: dispatch 2024-09-16T00:29:12.019 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:11 smithi112 ceph-mon[97750]: from='mgr.14678 172.21.15.112:0/3459258219' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mon metadata", "id": "smithi169"}]: dispatch 2024-09-16T00:29:12.019 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:11 smithi112 ceph-mon[97750]: pgmap v49: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 511 B/s wr, 5 op/s 2024-09-16T00:29:12.019 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:11 smithi112 ceph-mon[97750]: pgmap v50: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 3 op/s 2024-09-16T00:29:12.019 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:11 smithi112 ceph-mon[97750]: mon.smithi112 calling monitor election 2024-09-16T00:29:12.019 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:11 smithi112 ceph-mon[97750]: mon.smithi112 is new leader, mons smithi112,smithi169 in quorum (ranks 0,1) 2024-09-16T00:29:12.019 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:11 smithi112 ceph-mon[97750]: monmap epoch 2 2024-09-16T00:29:12.019 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:11 smithi112 ceph-mon[97750]: fsid d346dd56-73c0-11ef-bceb-c7b262605968 2024-09-16T00:29:12.019 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:11 smithi112 ceph-mon[97750]: last_changed 2024-09-16T00:19:02.487171+0000 2024-09-16T00:29:12.019 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:11 smithi112 ceph-mon[97750]: created 2024-09-16T00:16:40.582056+0000 2024-09-16T00:29:12.019 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:11 smithi112 ceph-mon[97750]: min_mon_release 17 (quincy) 2024-09-16T00:29:12.019 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:11 smithi112 ceph-mon[97750]: election_strategy: 1 2024-09-16T00:29:12.020 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:11 smithi112 ceph-mon[97750]: 0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.smithi112 2024-09-16T00:29:12.020 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:11 smithi112 ceph-mon[97750]: 1: [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] mon.smithi169 2024-09-16T00:29:12.020 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:11 smithi112 ceph-mon[97750]: fsmap cephfs:2 {0=cephfs.smithi112.hxfwkh=up:active,1=cephfs.smithi169.gawwgw=up:active} 2 up:standby-replay 2024-09-16T00:29:12.020 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:11 smithi112 ceph-mon[97750]: osdmap e46: 6 total, 6 up, 6 in 2024-09-16T00:29:12.020 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:11 smithi112 ceph-mon[97750]: mgrmap e32: smithi112.xrkkbc(active, since 94s), standbys: smithi169.mtrobk 2024-09-16T00:29:12.020 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:11 smithi112 ceph-mon[97750]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-09-16T00:29:12.020 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:11 smithi112 ceph-mon[97750]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T00:29:12.020 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:11 smithi112 ceph-mon[97750]: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T00:29:12.020 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:11 smithi112 ceph-mon[97750]: from='mgr.14678 ' entity='' 2024-09-16T00:29:12.020 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:11 smithi112 ceph-mon[97750]: mgrmap e33: smithi112.xrkkbc(active, since 94s), standbys: smithi169.mtrobk 2024-09-16T00:29:13.821 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:13 smithi112 ceph-mon[97750]: mgrmap e34: smithi112.xrkkbc(active, since 95s), standbys: smithi169.mtrobk 2024-09-16T00:29:14.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:13 smithi169 ceph-mon[34653]: mgrmap e34: smithi112.xrkkbc(active, since 95s), standbys: smithi169.mtrobk 2024-09-16T00:29:16.698 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:16 smithi169 ceph-mon[34653]: Standby manager daemon smithi169.mtrobk restarted 2024-09-16T00:29:16.698 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:16 smithi169 ceph-mon[34653]: Standby manager daemon smithi169.mtrobk started 2024-09-16T00:29:16.698 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:16 smithi169 ceph-mon[34653]: from='mgr.? 172.21.15.169:0/118714211' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi169.mtrobk/crt"}]: dispatch 2024-09-16T00:29:16.698 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:16 smithi169 ceph-mon[34653]: from='mgr.? 172.21.15.169:0/118714211' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T00:29:16.698 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:16 smithi169 ceph-mon[34653]: from='mgr.? 172.21.15.169:0/118714211' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi169.mtrobk/key"}]: dispatch 2024-09-16T00:29:16.699 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:16 smithi169 ceph-mon[34653]: from='mgr.? 172.21.15.169:0/118714211' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T00:29:16.699 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:16 smithi169 ceph-mon[34653]: Active manager daemon smithi112.xrkkbc restarted 2024-09-16T00:29:16.699 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:16 smithi169 ceph-mon[34653]: Activating manager daemon smithi112.xrkkbc 2024-09-16T00:29:16.769 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:16 smithi112 ceph-mon[97750]: Standby manager daemon smithi169.mtrobk restarted 2024-09-16T00:29:16.769 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:16 smithi112 ceph-mon[97750]: Standby manager daemon smithi169.mtrobk started 2024-09-16T00:29:16.769 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:16 smithi112 ceph-mon[97750]: from='mgr.? 172.21.15.169:0/118714211' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi169.mtrobk/crt"}]: dispatch 2024-09-16T00:29:16.769 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:16 smithi112 ceph-mon[97750]: from='mgr.? 172.21.15.169:0/118714211' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T00:29:16.769 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:16 smithi112 ceph-mon[97750]: from='mgr.? 172.21.15.169:0/118714211' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi169.mtrobk/key"}]: dispatch 2024-09-16T00:29:16.769 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:16 smithi112 ceph-mon[97750]: from='mgr.? 172.21.15.169:0/118714211' entity='mgr.smithi169.mtrobk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T00:29:16.770 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:16 smithi112 ceph-mon[97750]: Active manager daemon smithi112.xrkkbc restarted 2024-09-16T00:29:16.770 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:16 smithi112 ceph-mon[97750]: Activating manager daemon smithi112.xrkkbc 2024-09-16T00:29:17.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:17 smithi169 ceph-mon[34653]: osdmap e47: 6 total, 6 up, 6 in 2024-09-16T00:29:17.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:17 smithi169 ceph-mon[34653]: mgrmap e35: smithi112.xrkkbc(active, starting, since 0.00632035s), standbys: smithi169.mtrobk 2024-09-16T00:29:17.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:17 smithi169 ceph-mon[34653]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mon metadata", "id": "smithi112"}]: dispatch 2024-09-16T00:29:17.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:17 smithi169 ceph-mon[34653]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mon metadata", "id": "smithi169"}]: dispatch 2024-09-16T00:29:17.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:17 smithi169 ceph-mon[34653]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi112.hxfwkh"}]: dispatch 2024-09-16T00:29:17.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:17 smithi169 ceph-mon[34653]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi112.ithwzj"}]: dispatch 2024-09-16T00:29:17.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:17 smithi169 ceph-mon[34653]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi169.gawwgw"}]: dispatch 2024-09-16T00:29:17.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:17 smithi169 ceph-mon[34653]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi169.lgeqrj"}]: dispatch 2024-09-16T00:29:17.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:17 smithi169 ceph-mon[34653]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mgr metadata", "who": "smithi112.xrkkbc", "id": "smithi112.xrkkbc"}]: dispatch 2024-09-16T00:29:17.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:17 smithi169 ceph-mon[34653]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mgr metadata", "who": "smithi169.mtrobk", "id": "smithi169.mtrobk"}]: dispatch 2024-09-16T00:29:17.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:17 smithi169 ceph-mon[34653]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:29:17.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:17 smithi169 ceph-mon[34653]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:29:17.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:17 smithi169 ceph-mon[34653]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:29:17.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:17 smithi169 ceph-mon[34653]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:29:17.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:17 smithi169 ceph-mon[34653]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:29:17.778 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:17 smithi169 ceph-mon[34653]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:29:17.778 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:17 smithi169 ceph-mon[34653]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T00:29:17.778 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:17 smithi169 ceph-mon[34653]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T00:29:17.778 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:17 smithi169 ceph-mon[34653]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T00:29:17.778 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:17 smithi169 ceph-mon[34653]: Manager daemon smithi112.xrkkbc is now available 2024-09-16T00:29:17.778 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:17 smithi169 ceph-mon[34653]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:29:17.778 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:17 smithi169 ceph-mon[34653]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:29:17.778 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:17 smithi169 ceph-mon[34653]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi112.xrkkbc/mirror_snapshot_schedule"}]: dispatch 2024-09-16T00:29:17.778 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:17 smithi169 ceph-mon[34653]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi112.xrkkbc/trash_purge_schedule"}]: dispatch 2024-09-16T00:29:17.832 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:17 smithi112 ceph-mon[97750]: osdmap e47: 6 total, 6 up, 6 in 2024-09-16T00:29:17.832 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:17 smithi112 ceph-mon[97750]: mgrmap e35: smithi112.xrkkbc(active, starting, since 0.00632035s), standbys: smithi169.mtrobk 2024-09-16T00:29:17.833 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:17 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mon metadata", "id": "smithi112"}]: dispatch 2024-09-16T00:29:17.833 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:17 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mon metadata", "id": "smithi169"}]: dispatch 2024-09-16T00:29:17.833 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:17 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi112.hxfwkh"}]: dispatch 2024-09-16T00:29:17.833 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:17 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi112.ithwzj"}]: dispatch 2024-09-16T00:29:17.833 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:17 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi169.gawwgw"}]: dispatch 2024-09-16T00:29:17.833 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:17 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi169.lgeqrj"}]: dispatch 2024-09-16T00:29:17.833 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:17 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mgr metadata", "who": "smithi112.xrkkbc", "id": "smithi112.xrkkbc"}]: dispatch 2024-09-16T00:29:17.833 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:17 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mgr metadata", "who": "smithi169.mtrobk", "id": "smithi169.mtrobk"}]: dispatch 2024-09-16T00:29:17.833 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:17 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:29:17.833 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:17 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:29:17.833 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:17 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:29:17.833 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:17 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:29:17.833 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:17 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:29:17.834 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:17 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:29:17.834 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:17 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T00:29:17.834 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:17 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T00:29:17.834 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:17 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T00:29:17.834 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:17 smithi112 ceph-mon[97750]: Manager daemon smithi112.xrkkbc is now available 2024-09-16T00:29:17.834 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:17 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:29:17.834 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:17 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:29:17.834 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:17 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi112.xrkkbc/mirror_snapshot_schedule"}]: dispatch 2024-09-16T00:29:17.834 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:17 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi112.xrkkbc/trash_purge_schedule"}]: dispatch 2024-09-16T00:29:18.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:18 smithi112 ceph-mon[97750]: mgrmap e36: smithi112.xrkkbc(active, since 1.0925s), standbys: smithi169.mtrobk 2024-09-16T00:29:18.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:18 smithi112 ceph-mon[97750]: pgmap v3: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail 2024-09-16T00:29:18.942 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:18 smithi169 ceph-mon[34653]: mgrmap e36: smithi112.xrkkbc(active, since 1.0925s), standbys: smithi169.mtrobk 2024-09-16T00:29:18.942 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:18 smithi169 ceph-mon[34653]: pgmap v3: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail 2024-09-16T00:29:19.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:19 smithi112 ceph-mon[97750]: [16/Sep/2024:00:29:17] ENGINE Bus STARTING 2024-09-16T00:29:19.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:19 smithi112 ceph-mon[97750]: [16/Sep/2024:00:29:17] ENGINE Serving on http://172.21.15.112:8765 2024-09-16T00:29:19.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:19 smithi112 ceph-mon[97750]: [16/Sep/2024:00:29:17] ENGINE Serving on https://172.21.15.112:7150 2024-09-16T00:29:19.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:19 smithi112 ceph-mon[97750]: [16/Sep/2024:00:29:17] ENGINE Bus STARTED 2024-09-16T00:29:19.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:19 smithi112 ceph-mon[97750]: [16/Sep/2024:00:29:17] ENGINE Error in HTTPServer.serve 2024-09-16T00:29:19.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: Traceback (most recent call last): 2024-09-16T00:29:19.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T00:29:19.841 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: self._connections.run(self.expiration_interval) 2024-09-16T00:29:19.841 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T00:29:19.841 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: self._run(expiration_interval) 2024-09-16T00:29:19.841 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T00:29:19.841 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T00:29:19.841 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T00:29:19.841 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T00:29:19.841 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T00:29:19.841 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: s = self.context.wrap_socket( 2024-09-16T00:29:19.841 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T00:29:19.841 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: return self.sslsocket_class._create( 2024-09-16T00:29:19.841 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T00:29:19.841 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: self.do_handshake() 2024-09-16T00:29:19.842 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T00:29:19.842 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: self._sslobj.do_handshake() 2024-09-16T00:29:19.842 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T00:29:19.842 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:19 smithi112 ceph-mon[97750]: pgmap v4: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail 2024-09-16T00:29:19.842 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:19 smithi112 ceph-mon[97750]: mgrmap e37: smithi112.xrkkbc(active, since 2s), standbys: smithi169.mtrobk 2024-09-16T00:29:20.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:19 smithi169 ceph-mon[34653]: [16/Sep/2024:00:29:17] ENGINE Bus STARTING 2024-09-16T00:29:20.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:19 smithi169 ceph-mon[34653]: [16/Sep/2024:00:29:17] ENGINE Serving on http://172.21.15.112:8765 2024-09-16T00:29:20.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:19 smithi169 ceph-mon[34653]: [16/Sep/2024:00:29:17] ENGINE Serving on https://172.21.15.112:7150 2024-09-16T00:29:20.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:19 smithi169 ceph-mon[34653]: [16/Sep/2024:00:29:17] ENGINE Bus STARTED 2024-09-16T00:29:20.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:19 smithi169 ceph-mon[34653]: [16/Sep/2024:00:29:17] ENGINE Error in HTTPServer.serve 2024-09-16T00:29:20.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Traceback (most recent call last): 2024-09-16T00:29:20.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T00:29:20.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: self._connections.run(self.expiration_interval) 2024-09-16T00:29:20.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T00:29:20.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: self._run(expiration_interval) 2024-09-16T00:29:20.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T00:29:20.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T00:29:20.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T00:29:20.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T00:29:20.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T00:29:20.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: s = self.context.wrap_socket( 2024-09-16T00:29:20.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T00:29:20.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: return self.sslsocket_class._create( 2024-09-16T00:29:20.028 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T00:29:20.028 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: self.do_handshake() 2024-09-16T00:29:20.028 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T00:29:20.028 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: self._sslobj.do_handshake() 2024-09-16T00:29:20.028 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T00:29:20.028 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:19 smithi169 ceph-mon[34653]: pgmap v4: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail 2024-09-16T00:29:20.028 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:19 smithi169 ceph-mon[34653]: mgrmap e37: smithi112.xrkkbc(active, since 2s), standbys: smithi169.mtrobk 2024-09-16T00:29:21.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:20 smithi169 ceph-mon[34653]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:21.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:20 smithi169 ceph-mon[34653]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:21.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:20 smithi169 ceph-mon[34653]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:21.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:20 smithi169 ceph-mon[34653]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:21.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:20 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:21.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:20 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:21.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:20 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:21.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:20 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:22.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:21 smithi169 ceph-mon[34653]: pgmap v5: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail 2024-09-16T00:29:22.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:21 smithi169 ceph-mon[34653]: mgrmap e38: smithi112.xrkkbc(active, since 4s), standbys: smithi169.mtrobk 2024-09-16T00:29:22.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:21 smithi112 ceph-mon[97750]: pgmap v5: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail 2024-09-16T00:29:22.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:21 smithi112 ceph-mon[97750]: mgrmap e38: smithi112.xrkkbc(active, since 4s), standbys: smithi169.mtrobk 2024-09-16T00:29:23.107 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:23 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:23.107 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:23 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:23.107 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:23 smithi112 ceph-mon[97750]: pgmap v6: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail 2024-09-16T00:29:23.107 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:23 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:23.107 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:23 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:23.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:23 smithi169 ceph-mon[34653]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:23.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:23 smithi169 ceph-mon[34653]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:23.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:23 smithi169 ceph-mon[34653]: pgmap v6: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail 2024-09-16T00:29:23.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:23 smithi169 ceph-mon[34653]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:23.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:23 smithi169 ceph-mon[34653]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:25.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:25 smithi112 ceph-mon[97750]: pgmap v7: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 31 KiB/s rd, 0 B/s wr, 19 op/s 2024-09-16T00:29:26.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:25 smithi169 ceph-mon[34653]: pgmap v7: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 31 KiB/s rd, 0 B/s wr, 19 op/s 2024-09-16T00:29:27.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:26 smithi169 ceph-mon[34653]: Detected new or changed devices on smithi112 2024-09-16T00:29:27.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:26 smithi169 ceph-mon[34653]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:27.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:26 smithi169 ceph-mon[34653]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:27.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:26 smithi169 ceph-mon[34653]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:27.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:26 smithi169 ceph-mon[34653]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:29:27.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:26 smithi169 ceph-mon[34653]: Updating smithi112:/etc/ceph/ceph.conf 2024-09-16T00:29:27.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:26 smithi169 ceph-mon[34653]: Updating smithi169:/etc/ceph/ceph.conf 2024-09-16T00:29:27.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:26 smithi169 ceph-mon[34653]: Updating smithi112:/var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:29:27.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:26 smithi169 ceph-mon[34653]: Updating smithi169:/var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:29:27.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:26 smithi169 ceph-mon[34653]: Updating smithi112:/etc/ceph/ceph.client.admin.keyring 2024-09-16T00:29:27.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:26 smithi169 ceph-mon[34653]: Updating smithi169:/etc/ceph/ceph.client.admin.keyring 2024-09-16T00:29:27.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:26 smithi169 ceph-mon[34653]: pgmap v8: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 24 KiB/s rd, 0 B/s wr, 15 op/s 2024-09-16T00:29:27.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:26 smithi169 ceph-mon[34653]: Updating smithi112:/var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T00:29:27.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:26 smithi169 ceph-mon[34653]: Updating smithi169:/var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T00:29:27.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:26 smithi169 ceph-mon[34653]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:27.278 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:26 smithi169 ceph-mon[34653]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:27.278 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:26 smithi169 ceph-mon[34653]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:27.278 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:26 smithi169 ceph-mon[34653]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:27.278 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:26 smithi169 ceph-mon[34653]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:27.278 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:26 smithi169 ceph-mon[34653]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:29:27.278 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:26 smithi169 ceph-mon[34653]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:29:27.278 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:26 smithi169 ceph-mon[34653]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-09-16T00:29:27.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:26 smithi112 ceph-mon[97750]: Detected new or changed devices on smithi112 2024-09-16T00:29:27.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:26 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:27.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:26 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:27.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:26 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:27.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:26 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:29:27.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:26 smithi112 ceph-mon[97750]: Updating smithi112:/etc/ceph/ceph.conf 2024-09-16T00:29:27.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:26 smithi112 ceph-mon[97750]: Updating smithi169:/etc/ceph/ceph.conf 2024-09-16T00:29:27.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:26 smithi112 ceph-mon[97750]: Updating smithi112:/var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:29:27.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:26 smithi112 ceph-mon[97750]: Updating smithi169:/var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:29:27.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:26 smithi112 ceph-mon[97750]: Updating smithi112:/etc/ceph/ceph.client.admin.keyring 2024-09-16T00:29:27.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:26 smithi112 ceph-mon[97750]: Updating smithi169:/etc/ceph/ceph.client.admin.keyring 2024-09-16T00:29:27.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:26 smithi112 ceph-mon[97750]: pgmap v8: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 24 KiB/s rd, 0 B/s wr, 15 op/s 2024-09-16T00:29:27.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:26 smithi112 ceph-mon[97750]: Updating smithi112:/var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T00:29:27.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:26 smithi112 ceph-mon[97750]: Updating smithi169:/var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T00:29:27.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:26 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:27.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:26 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:27.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:26 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:27.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:26 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:27.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:26 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:27.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:26 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:29:27.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:26 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:29:27.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:26 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-09-16T00:29:29.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:29 smithi112 ceph-mon[97750]: Upgrade: Updating mon.smithi169 2024-09-16T00:29:29.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:29 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:29.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:29 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T00:29:29.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:29 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T00:29:29.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:29 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:29.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:29 smithi112 ceph-mon[97750]: Deploying daemon mon.smithi169 on smithi169 2024-09-16T00:29:29.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:29 smithi112 ceph-mon[97750]: pgmap v9: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 21 KiB/s rd, 0 B/s wr, 14 op/s 2024-09-16T00:29:29.596 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:29 smithi169 ceph-mon[34653]: Upgrade: Updating mon.smithi169 2024-09-16T00:29:29.596 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:29 smithi169 ceph-mon[34653]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:29.596 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:29 smithi169 ceph-mon[34653]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T00:29:29.596 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:29 smithi169 ceph-mon[34653]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T00:29:29.596 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:29 smithi169 ceph-mon[34653]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:29.596 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:29 smithi169 ceph-mon[34653]: Deploying daemon mon.smithi169 on smithi169 2024-09-16T00:29:29.596 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:29 smithi169 ceph-mon[34653]: pgmap v9: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 21 KiB/s rd, 0 B/s wr, 14 op/s 2024-09-16T00:29:30.144 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:29 smithi169 systemd[1]: Stopping Ceph mon.smithi169 for d346dd56-73c0-11ef-bceb-c7b262605968... 2024-09-16T00:29:30.452 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:30 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-mon-smithi169[34637]: 2024-09-16T00:29:30.143+0000 7fcb5b6f9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi169 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:29:30.452 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:30 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-mon-smithi169[34637]: 2024-09-16T00:29:30.143+0000 7fcb5b6f9640 -1 mon.smithi169@1(peon) e2 *** Got Signal Terminated *** 2024-09-16T00:29:30.453 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:30 smithi169 podman[77465]: 2024-09-16 00:29:30.20483834 +0000 UTC m=+0.181759869 container died 6c2289bc3e11d143a2e6df08d0ff86f4766325acc0c76bb93177ee99d0dab9a0 (image=quay.ceph.io/ceph-ci/ceph@sha256:83596473fc70117d507bb3d925c5bd11b70f46bbd4dffcc8452f145d93eaedc4, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-mon-smithi169, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-6d025bc, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-16T00:29:30.735 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:30 smithi169 podman[77465]: 2024-09-16 00:29:30.543047696 +0000 UTC m=+0.519969180 container cleanup 6c2289bc3e11d143a2e6df08d0ff86f4766325acc0c76bb93177ee99d0dab9a0 (image=quay.ceph.io/ceph-ci/ceph@sha256:83596473fc70117d507bb3d925c5bd11b70f46bbd4dffcc8452f145d93eaedc4, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-mon-smithi169, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=quincy-6d025bc, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-16T00:29:30.735 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:30 smithi169 bash[77465]: ceph-d346dd56-73c0-11ef-bceb-c7b262605968-mon-smithi169 2024-09-16T00:29:31.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:30 smithi169 podman[77477]: 2024-09-16 00:29:30.735970192 +0000 UTC m=+0.525525095 container remove 6c2289bc3e11d143a2e6df08d0ff86f4766325acc0c76bb93177ee99d0dab9a0 (image=quay.ceph.io/ceph-ci/ceph@sha256:83596473fc70117d507bb3d925c5bd11b70f46bbd4dffcc8452f145d93eaedc4, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-mon-smithi169, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, RELEASE=quincy-6d025bc) 2024-09-16T00:29:31.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:30 smithi169 systemd[1]: ceph-d346dd56-73c0-11ef-bceb-c7b262605968@mon.smithi169.service: Deactivated successfully. 2024-09-16T00:29:31.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:30 smithi169 systemd[1]: Stopped Ceph mon.smithi169 for d346dd56-73c0-11ef-bceb-c7b262605968. 2024-09-16T00:29:31.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:30 smithi169 systemd[1]: ceph-d346dd56-73c0-11ef-bceb-c7b262605968@mon.smithi169.service: Consumed 6.154s CPU time. 2024-09-16T00:29:31.501 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 systemd[1]: Starting Ceph mon.smithi169 for d346dd56-73c0-11ef-bceb-c7b262605968... 2024-09-16T00:29:31.501 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 podman[77607]: 2024-09-16 00:29:31.265653166 +0000 UTC m=+0.016400495 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:29:31.501 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 podman[77607]: 2024-09-16 00:29:31.38245132 +0000 UTC m=+0.133198645 container create 7bc609155f3f4d9e90ff4a79ab736e132c55ea2fa375c91ac74905745e94971c (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-mon-smithi169, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-16T00:29:31.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 podman[77607]: 2024-09-16 00:29:31.618507507 +0000 UTC m=+0.369254829 container init 7bc609155f3f4d9e90ff4a79ab736e132c55ea2fa375c91ac74905745e94971c (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-mon-smithi169, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.schema-version=1.0) 2024-09-16T00:29:31.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 podman[77607]: 2024-09-16 00:29:31.621995235 +0000 UTC m=+0.372742567 container start 7bc609155f3f4d9e90ff4a79ab736e132c55ea2fa375c91ac74905745e94971c (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-mon-smithi169, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731) 2024-09-16T00:29:31.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: set uid:gid to 167:167 (ceph:ceph) 2024-09-16T00:29:31.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable), process ceph-mon, pid 2 2024-09-16T00:29:31.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: pidfile_write: ignore empty --pid-file 2024-09-16T00:29:31.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: load: jerasure load: lrc 2024-09-16T00:29:31.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: RocksDB version: 7.9.2 2024-09-16T00:29:31.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Git sha 0 2024-09-16T00:29:31.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Compile date 2024-09-15 15:04:35 2024-09-16T00:29:31.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: DB SUMMARY 2024-09-16T00:29:31.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: DB Session ID: QL5FUXJWBFEG5FY6A6L5 2024-09-16T00:29:31.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: CURRENT file: CURRENT 2024-09-16T00:29:31.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: IDENTITY file: IDENTITY 2024-09-16T00:29:31.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: MANIFEST file: MANIFEST-000009 size: 1104 Bytes 2024-09-16T00:29:31.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi169/store.db dir, Total Num: 1, files: 000027.sst 2024-09-16T00:29:31.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi169/store.db: 000025.log size: 3294118 ; 2024-09-16T00:29:31.778 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.error_if_exists: 0 2024-09-16T00:29:31.778 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.create_if_missing: 0 2024-09-16T00:29:31.778 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.paranoid_checks: 1 2024-09-16T00:29:31.778 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-16T00:29:31.778 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-16T00:29:31.778 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-16T00:29:31.778 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.env: 0x56473823f940 2024-09-16T00:29:31.778 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.fs: PosixFileSystem 2024-09-16T00:29:31.778 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.info_log: 0x564739b138c0 2024-09-16T00:29:31.778 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.max_file_opening_threads: 16 2024-09-16T00:29:31.780 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.statistics: (nil) 2024-09-16T00:29:31.780 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.use_fsync: 0 2024-09-16T00:29:31.780 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.max_log_file_size: 0 2024-09-16T00:29:31.780 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-16T00:29:31.780 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-16T00:29:31.780 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.keep_log_file_num: 1000 2024-09-16T00:29:31.780 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.recycle_log_file_num: 0 2024-09-16T00:29:31.780 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.allow_fallocate: 1 2024-09-16T00:29:31.780 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.allow_mmap_reads: 0 2024-09-16T00:29:31.780 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.allow_mmap_writes: 0 2024-09-16T00:29:31.780 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.use_direct_reads: 0 2024-09-16T00:29:31.780 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-16T00:29:31.781 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.create_missing_column_families: 0 2024-09-16T00:29:31.781 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.db_log_dir: 2024-09-16T00:29:31.781 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.wal_dir: 2024-09-16T00:29:31.781 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-16T00:29:31.781 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-16T00:29:31.781 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-16T00:29:31.781 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-16T00:29:31.781 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-16T00:29:31.781 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-16T00:29:31.781 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.advise_random_on_open: 1 2024-09-16T00:29:31.781 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.db_write_buffer_size: 0 2024-09-16T00:29:31.781 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.write_buffer_manager: 0x564739b22dc0 2024-09-16T00:29:31.781 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-16T00:29:31.781 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-16T00:29:31.781 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-16T00:29:31.782 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.rate_limiter: (nil) 2024-09-16T00:29:31.782 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-16T00:29:31.782 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.wal_recovery_mode: 2 2024-09-16T00:29:31.782 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.enable_thread_tracking: 0 2024-09-16T00:29:31.782 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.enable_pipelined_write: 0 2024-09-16T00:29:31.782 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.unordered_write: 0 2024-09-16T00:29:31.782 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-16T00:29:31.782 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-16T00:29:31.782 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-16T00:29:31.782 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-16T00:29:31.782 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.row_cache: None 2024-09-16T00:29:31.782 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.wal_filter: None 2024-09-16T00:29:31.782 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-16T00:29:31.782 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.allow_ingest_behind: 0 2024-09-16T00:29:31.783 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.two_write_queues: 0 2024-09-16T00:29:31.783 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.manual_wal_flush: 0 2024-09-16T00:29:31.783 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.wal_compression: 0 2024-09-16T00:29:31.783 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.atomic_flush: 0 2024-09-16T00:29:31.783 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-16T00:29:31.783 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-16T00:29:31.783 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-16T00:29:31.783 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.log_readahead_size: 0 2024-09-16T00:29:31.783 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-16T00:29:31.783 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.best_efforts_recovery: 0 2024-09-16T00:29:31.783 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-16T00:29:31.783 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-16T00:29:31.783 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.allow_data_in_errors: 0 2024-09-16T00:29:31.783 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.db_host_id: __hostname__ 2024-09-16T00:29:31.783 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-16T00:29:31.784 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.max_background_jobs: 2 2024-09-16T00:29:31.784 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.max_background_compactions: -1 2024-09-16T00:29:31.784 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.max_subcompactions: 1 2024-09-16T00:29:31.784 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-16T00:29:31.784 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-16T00:29:31.784 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-16T00:29:31.784 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.max_total_wal_size: 0 2024-09-16T00:29:31.784 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-16T00:29:31.784 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-16T00:29:31.784 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-16T00:29:31.784 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-16T00:29:31.784 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.max_open_files: -1 2024-09-16T00:29:31.784 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.bytes_per_sync: 0 2024-09-16T00:29:31.784 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-16T00:29:31.785 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-16T00:29:31.785 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.compaction_readahead_size: 0 2024-09-16T00:29:31.785 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.max_background_flushes: -1 2024-09-16T00:29:31.785 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Compression algorithms supported: 2024-09-16T00:29:31.785 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: kZSTD supported: 0 2024-09-16T00:29:31.785 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: kXpressCompression supported: 0 2024-09-16T00:29:31.785 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: kBZip2Compression supported: 0 2024-09-16T00:29:31.785 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-16T00:29:31.785 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: kLZ4Compression supported: 1 2024-09-16T00:29:31.785 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: kZlibCompression supported: 1 2024-09-16T00:29:31.785 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: kLZ4HCCompression supported: 1 2024-09-16T00:29:31.785 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: kSnappyCompression supported: 1 2024-09-16T00:29:31.785 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-16T00:29:31.785 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-16T00:29:31.785 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi169/store.db/MANIFEST-000009 2024-09-16T00:29:31.786 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-16T00:29:31.786 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-16T00:29:31.786 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.merge_operator: 2024-09-16T00:29:31.786 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.compaction_filter: None 2024-09-16T00:29:31.786 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.compaction_filter_factory: None 2024-09-16T00:29:31.786 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.sst_partitioner_factory: None 2024-09-16T00:29:31.786 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-16T00:29:31.786 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-16T00:29:31.786 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x564739b13520) 2024-09-16T00:29:31.786 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: cache_index_and_filter_blocks: 1 2024-09-16T00:29:31.786 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-16T00:29:31.786 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-16T00:29:31.786 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: pin_top_level_index_and_filter: 1 2024-09-16T00:29:31.786 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: index_type: 0 2024-09-16T00:29:31.787 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: data_block_index_type: 0 2024-09-16T00:29:31.787 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: index_shortening: 1 2024-09-16T00:29:31.787 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-16T00:29:31.787 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: checksum: 4 2024-09-16T00:29:31.787 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: no_block_cache: 0 2024-09-16T00:29:31.787 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: block_cache: 0x564739b071f0 2024-09-16T00:29:31.787 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: block_cache_name: BinnedLRUCache 2024-09-16T00:29:31.787 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: block_cache_options: 2024-09-16T00:29:31.787 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: capacity : 536870912 2024-09-16T00:29:31.788 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: num_shard_bits : 4 2024-09-16T00:29:31.788 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: strict_capacity_limit : 0 2024-09-16T00:29:31.788 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: high_pri_pool_ratio: 0.000 2024-09-16T00:29:31.788 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: block_cache_compressed: (nil) 2024-09-16T00:29:31.788 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: persistent_cache: (nil) 2024-09-16T00:29:31.788 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: block_size: 4096 2024-09-16T00:29:31.788 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: block_size_deviation: 10 2024-09-16T00:29:31.788 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: block_restart_interval: 16 2024-09-16T00:29:31.788 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: index_block_restart_interval: 1 2024-09-16T00:29:31.788 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: metadata_block_size: 4096 2024-09-16T00:29:31.788 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: partition_filters: 0 2024-09-16T00:29:31.788 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: use_delta_encoding: 1 2024-09-16T00:29:31.788 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: filter_policy: bloomfilter 2024-09-16T00:29:31.789 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: whole_key_filtering: 1 2024-09-16T00:29:31.789 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: verify_compression: 0 2024-09-16T00:29:31.789 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: read_amp_bytes_per_bit: 0 2024-09-16T00:29:31.789 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: format_version: 5 2024-09-16T00:29:31.789 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: enable_index_compression: 1 2024-09-16T00:29:31.789 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: block_align: 0 2024-09-16T00:29:31.789 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: max_auto_readahead_size: 262144 2024-09-16T00:29:31.789 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: prepopulate_block_cache: 0 2024-09-16T00:29:31.789 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: initial_auto_readahead_size: 8192 2024-09-16T00:29:31.789 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: num_file_reads_for_auto_readahead: 2 2024-09-16T00:29:31.789 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.write_buffer_size: 33554432 2024-09-16T00:29:31.789 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.max_write_buffer_number: 2 2024-09-16T00:29:31.789 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.compression: NoCompression 2024-09-16T00:29:31.789 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.bottommost_compression: Disabled 2024-09-16T00:29:31.790 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.prefix_extractor: nullptr 2024-09-16T00:29:31.790 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-16T00:29:31.790 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.num_levels: 7 2024-09-16T00:29:31.790 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-16T00:29:31.790 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-16T00:29:31.790 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-16T00:29:31.790 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-16T00:29:31.790 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-16T00:29:31.790 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-16T00:29:31.790 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-16T00:29:31.790 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-16T00:29:31.790 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-16T00:29:31.790 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-16T00:29:31.790 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-16T00:29:31.790 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-16T00:29:31.791 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-16T00:29:31.791 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.compression_opts.level: 32767 2024-09-16T00:29:31.791 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.compression_opts.strategy: 0 2024-09-16T00:29:31.791 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-16T00:29:31.791 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-16T00:29:31.791 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-16T00:29:31.791 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-16T00:29:31.791 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.compression_opts.enabled: false 2024-09-16T00:29:31.791 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-16T00:29:31.791 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-16T00:29:31.791 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-16T00:29:31.791 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-16T00:29:31.791 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.target_file_size_base: 67108864 2024-09-16T00:29:31.791 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-16T00:29:31.791 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-16T00:29:31.792 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-16T00:29:31.792 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-16T00:29:31.792 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-16T00:29:31.792 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-16T00:29:31.792 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-16T00:29:31.792 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-16T00:29:31.792 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-16T00:29:31.792 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-16T00:29:31.792 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-16T00:29:31.792 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-16T00:29:31.792 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-16T00:29:31.792 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-16T00:29:31.792 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.arena_block_size: 1048576 2024-09-16T00:29:31.793 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-16T00:29:31.793 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-16T00:29:31.793 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.disable_auto_compactions: 0 2024-09-16T00:29:31.793 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-16T00:29:31.793 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-16T00:29:31.793 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-16T00:29:31.793 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-16T00:29:31.793 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-16T00:29:31.793 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-16T00:29:31.793 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-16T00:29:31.793 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-16T00:29:31.793 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-16T00:29:31.793 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-16T00:29:31.793 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-16T00:29:31.793 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.inplace_update_support: 0 2024-09-16T00:29:31.794 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-16T00:29:31.794 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-16T00:29:31.794 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-16T00:29:31.794 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-16T00:29:31.794 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.bloom_locality: 0 2024-09-16T00:29:31.794 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.max_successive_merges: 0 2024-09-16T00:29:31.794 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-16T00:29:31.794 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.paranoid_file_checks: 0 2024-09-16T00:29:31.794 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.force_consistency_checks: 1 2024-09-16T00:29:31.794 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.report_bg_io_stats: 0 2024-09-16T00:29:31.794 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.ttl: 2592000 2024-09-16T00:29:31.794 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-16T00:29:31.794 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-16T00:29:31.794 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-16T00:29:31.795 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.enable_blob_files: false 2024-09-16T00:29:31.795 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.min_blob_size: 0 2024-09-16T00:29:31.795 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.blob_file_size: 268435456 2024-09-16T00:29:31.795 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-16T00:29:31.795 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-16T00:29:31.795 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-16T00:29:31.795 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-16T00:29:31.795 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-16T00:29:31.795 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.blob_file_starting_level: 0 2024-09-16T00:29:31.795 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-16T00:29:31.795 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: [table/block_based/block_based_table_reader.cc:721] At least one SST file opened without unique ID to verify: 27.sst 2024-09-16T00:29:31.795 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-09-16T00:29:31.795 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi169/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 29, last_sequence is 12328, log_number is 25,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-16T00:29:31.795 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 25 2024-09-16T00:29:31.795 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: df03730b-16ce-43ff-a098-3a54a9c13c09 2024-09-16T00:29:31.796 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726446571648837, "job": 1, "event": "recovery_started", "wal_files": [25]} 2024-09-16T00:29:31.796 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #25 mode 2 2024-09-16T00:29:31.796 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726446571656859, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 30, "file_size": 2105431, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12333, "largest_seqno": 13231, "table_properties": {"data_size": 2100830, "index_size": 2534, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 1029, "raw_key_size": 9557, "raw_average_key_size": 24, "raw_value_size": 2092210, "raw_average_value_size": 5256, "num_data_blocks": 115, "num_entries": 398, "num_filter_entries": 398, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726446571, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "df03730b-16ce-43ff-a098-3a54a9c13c09", "db_session_id": "QL5FUXJWBFEG5FY6A6L5", "orig_file_number": 30, "seqno_to_time_mapping": "N/A"}} 2024-09-16T00:29:31.796 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726446571656991, "job": 1, "event": "recovery_finished"} 2024-09-16T00:29:31.796 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: [db/version_set.cc:5047] Creating manifest 32 2024-09-16T00:29:31.796 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-09-16T00:29:31.796 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi169/store.db/000025.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-16T00:29:31.796 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x564739b34e00 2024-09-16T00:29:31.796 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: DB pointer 0x564739c30000 2024-09-16T00:29:31.796 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-16T00:29:31.796 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-16T00:29:31.796 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: ** DB Stats ** 2024-09-16T00:29:31.796 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T00:29:31.797 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-16T00:29:31.797 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T00:29:31.797 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T00:29:31.797 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-16T00:29:31.797 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T00:29:31.797 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T00:29:31.797 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: 2024-09-16T00:29:31.797 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: ** Compaction Stats [default] ** 2024-09-16T00:29:31.797 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T00:29:31.797 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-16T00:29:31.797 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: L0 1/0 2.01 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 594.9 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-09-16T00:29:31.797 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: L6 1/0 6.87 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-09-16T00:29:31.797 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Sum 2/0 8.88 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 594.9 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-09-16T00:29:31.797 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 594.9 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-09-16T00:29:31.797 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: 2024-09-16T00:29:31.798 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: ** Compaction Stats [default] ** 2024-09-16T00:29:31.798 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T00:29:31.798 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-16T00:29:31.798 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 594.9 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-09-16T00:29:31.798 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: 2024-09-16T00:29:31.798 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-16T00:29:31.798 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: 2024-09-16T00:29:31.798 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T00:29:31.798 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Flush(GB): cumulative 0.002, interval 0.002 2024-09-16T00:29:31.798 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-16T00:29:31.798 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-16T00:29:31.798 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-16T00:29:31.798 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-16T00:29:31.798 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Cumulative compaction: 0.00 GB write, 159.84 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T00:29:31.799 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Interval compaction: 0.00 GB write, 159.84 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T00:29:31.799 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-16T00:29:31.799 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Block cache BinnedLRUCache@0x564739b071f0#2 capacity: 512.00 MB usage: 3.73 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2024-09-16T00:29:31.799 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,1.16 KB,0.000220537%) IndexBlock(1,2.58 KB,0.000491738%) Misc(1,0.00 KB,0%) 2024-09-16T00:29:31.799 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: 2024-09-16T00:29:31.799 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-16T00:29:31.799 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: starting mon.smithi169 rank 1 at public addrs [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] at bind addrs [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi169 fsid d346dd56-73c0-11ef-bceb-c7b262605968 2024-09-16T00:29:31.799 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: mon.smithi169@-1(???) e2 preinit fsid d346dd56-73c0-11ef-bceb-c7b262605968 2024-09-16T00:29:31.799 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: mon.smithi169@-1(???).mds e21 new map 2024-09-16T00:29:31.799 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: mon.smithi169@-1(???).mds e21 print_map 2024-09-16T00:29:31.799 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: e21 2024-09-16T00:29:31.799 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: btime 1970-01-01T00:00:00:000000+0000 2024-09-16T00:29:31.799 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T00:29:31.799 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:29:31.800 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: legacy client fscid: 1 2024-09-16T00:29:31.800 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: 2024-09-16T00:29:31.800 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Filesystem 'cephfs' (1) 2024-09-16T00:29:31.800 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: fs_name cephfs 2024-09-16T00:29:31.800 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: epoch 21 2024-09-16T00:29:31.800 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-16T00:29:31.800 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: created 2024-09-16T00:23:22.328199+0000 2024-09-16T00:29:31.800 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: modified 2024-09-16T00:26:38.963153+0000 2024-09-16T00:29:31.800 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: tableserver 0 2024-09-16T00:29:31.800 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: root 0 2024-09-16T00:29:31.801 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: session_timeout 60 2024-09-16T00:29:31.801 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: session_autoclose 300 2024-09-16T00:29:31.801 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: max_file_size 1099511627776 2024-09-16T00:29:31.801 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: max_xattr_size 65536 2024-09-16T00:29:31.801 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: required_client_features {} 2024-09-16T00:29:31.801 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: last_failure 0 2024-09-16T00:29:31.801 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: last_failure_osd_epoch 44 2024-09-16T00:29:31.801 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:29:31.801 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: max_mds 2 2024-09-16T00:29:31.801 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: in 0,1 2024-09-16T00:29:31.801 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: up {0=14480,1=24259} 2024-09-16T00:29:31.801 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: failed 2024-09-16T00:29:31.801 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: damaged 2024-09-16T00:29:31.801 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: stopped 2024-09-16T00:29:31.802 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: data_pools [3] 2024-09-16T00:29:31.802 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: metadata_pool 2 2024-09-16T00:29:31.802 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: inline_data enabled 2024-09-16T00:29:31.802 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: balancer 2024-09-16T00:29:31.802 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: bal_rank_mask -1 2024-09-16T00:29:31.802 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: standby_count_wanted 1 2024-09-16T00:29:31.802 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: qdb_cluster leader: 0 members: 2024-09-16T00:29:31.802 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: [mds.cephfs.smithi112.hxfwkh{0:14480} state up:active seq 6 export targets 1 join_fscid=1 addr [v2:172.21.15.112:6828/486654885,v1:172.21.15.112:6829/486654885] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T00:29:31.802 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: [mds.cephfs.smithi112.ithwzj{0:14492} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.112:6826/381830565,v1:172.21.15.112:6827/381830565] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T00:29:31.802 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: [mds.cephfs.smithi169.gawwgw{1:24259} state up:active seq 4 export targets 0 join_fscid=1 addr [v2:172.21.15.169:6824/2892476953,v1:172.21.15.169:6825/2892476953] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T00:29:31.802 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: [mds.cephfs.smithi169.lgeqrj{1:24269} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.169:6826/861725728,v1:172.21.15.169:6827/861725728] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T00:29:31.802 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: 2024-09-16T00:29:31.802 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: 2024-09-16T00:29:31.802 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: mon.smithi169@-1(???).osd e47 crush map has features 3314933000852226048, adjusting msgr requires 2024-09-16T00:29:31.803 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: mon.smithi169@-1(???).osd e47 crush map has features 288514051259236352, adjusting msgr requires 2024-09-16T00:29:31.803 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: mon.smithi169@-1(???).osd e47 crush map has features 288514051259236352, adjusting msgr requires 2024-09-16T00:29:31.803 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: mon.smithi169@-1(???).osd e47 crush map has features 288514051259236352, adjusting msgr requires 2024-09-16T00:29:31.803 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 ceph-mon[77626]: mon.smithi169@-1(???).paxosservice(auth 1..22) refresh upgraded, format 0 -> 3 2024-09-16T00:29:31.803 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 bash[77607]: 7bc609155f3f4d9e90ff4a79ab736e132c55ea2fa375c91ac74905745e94971c 2024-09-16T00:29:31.803 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:31 smithi169 systemd[1]: Started Ceph mon.smithi169 for d346dd56-73c0-11ef-bceb-c7b262605968. 2024-09-16T00:29:33.218 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:32 smithi169 ceph-mon[77626]: mgrmap e38: smithi112.xrkkbc(active, since 15s), standbys: smithi169.mtrobk 2024-09-16T00:29:33.218 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:32 smithi169 ceph-mon[77626]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-09-16T00:29:33.218 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:32 smithi169 ceph-mon[77626]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T00:29:33.218 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:32 smithi169 ceph-mon[77626]: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T00:29:33.218 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:32 smithi169 ceph-mon[77626]: mon.smithi112 calling monitor election 2024-09-16T00:29:33.218 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:32 smithi169 ceph-mon[77626]: mon.smithi169 calling monitor election 2024-09-16T00:29:33.218 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:32 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mon metadata", "id": "smithi112"}]: dispatch 2024-09-16T00:29:33.218 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:32 smithi169 ceph-mon[77626]: mon.smithi112 is new leader, mons smithi112,smithi169 in quorum (ranks 0,1) 2024-09-16T00:29:33.219 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:32 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mon metadata", "id": "smithi169"}]: dispatch 2024-09-16T00:29:33.219 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:32 smithi169 ceph-mon[77626]: monmap epoch 3 2024-09-16T00:29:33.219 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:32 smithi169 ceph-mon[77626]: fsid d346dd56-73c0-11ef-bceb-c7b262605968 2024-09-16T00:29:33.219 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:32 smithi169 ceph-mon[77626]: last_changed 2024-09-16T00:29:31.951078+0000 2024-09-16T00:29:33.219 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:32 smithi169 ceph-mon[77626]: created 2024-09-16T00:16:40.582056+0000 2024-09-16T00:29:33.219 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:32 smithi169 ceph-mon[77626]: min_mon_release 19 (squid) 2024-09-16T00:29:33.219 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:32 smithi169 ceph-mon[77626]: election_strategy: 1 2024-09-16T00:29:33.219 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:32 smithi169 ceph-mon[77626]: 0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.smithi112 2024-09-16T00:29:33.219 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:32 smithi169 ceph-mon[77626]: 1: [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] mon.smithi169 2024-09-16T00:29:33.219 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:32 smithi169 ceph-mon[77626]: fsmap cephfs:2 {0=cephfs.smithi112.hxfwkh=up:active,1=cephfs.smithi169.gawwgw=up:active} 2 up:standby-replay 2024-09-16T00:29:33.219 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:32 smithi169 ceph-mon[77626]: osdmap e47: 6 total, 6 up, 6 in 2024-09-16T00:29:33.219 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:32 smithi169 ceph-mon[77626]: mgrmap e38: smithi112.xrkkbc(active, since 15s), standbys: smithi169.mtrobk 2024-09-16T00:29:33.219 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:32 smithi169 ceph-mon[77626]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-09-16T00:29:33.219 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:32 smithi169 ceph-mon[77626]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T00:29:33.220 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:32 smithi169 ceph-mon[77626]: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T00:29:33.220 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:32 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:33.220 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:32 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:33.220 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:32 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:29:33.220 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:32 smithi169 ceph-mon[77626]: pgmap v11: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-16T00:29:33.334 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:32 smithi112 ceph-mon[97750]: mgrmap e38: smithi112.xrkkbc(active, since 15s), standbys: smithi169.mtrobk 2024-09-16T00:29:33.334 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:32 smithi112 ceph-mon[97750]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-09-16T00:29:33.334 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:32 smithi112 ceph-mon[97750]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T00:29:33.335 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:32 smithi112 ceph-mon[97750]: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T00:29:33.335 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:32 smithi112 ceph-mon[97750]: mon.smithi112 calling monitor election 2024-09-16T00:29:33.335 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:32 smithi112 ceph-mon[97750]: mon.smithi169 calling monitor election 2024-09-16T00:29:33.335 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:32 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mon metadata", "id": "smithi112"}]: dispatch 2024-09-16T00:29:33.335 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:32 smithi112 ceph-mon[97750]: mon.smithi112 is new leader, mons smithi112,smithi169 in quorum (ranks 0,1) 2024-09-16T00:29:33.336 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:32 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mon metadata", "id": "smithi169"}]: dispatch 2024-09-16T00:29:33.336 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:32 smithi112 ceph-mon[97750]: monmap epoch 3 2024-09-16T00:29:33.336 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:32 smithi112 ceph-mon[97750]: fsid d346dd56-73c0-11ef-bceb-c7b262605968 2024-09-16T00:29:33.336 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:32 smithi112 ceph-mon[97750]: last_changed 2024-09-16T00:29:31.951078+0000 2024-09-16T00:29:33.336 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:32 smithi112 ceph-mon[97750]: created 2024-09-16T00:16:40.582056+0000 2024-09-16T00:29:33.337 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:32 smithi112 ceph-mon[97750]: min_mon_release 19 (squid) 2024-09-16T00:29:33.337 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:32 smithi112 ceph-mon[97750]: election_strategy: 1 2024-09-16T00:29:33.337 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:32 smithi112 ceph-mon[97750]: 0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.smithi112 2024-09-16T00:29:33.337 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:32 smithi112 ceph-mon[97750]: 1: [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] mon.smithi169 2024-09-16T00:29:33.337 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:32 smithi112 ceph-mon[97750]: fsmap cephfs:2 {0=cephfs.smithi112.hxfwkh=up:active,1=cephfs.smithi169.gawwgw=up:active} 2 up:standby-replay 2024-09-16T00:29:33.338 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:32 smithi112 ceph-mon[97750]: osdmap e47: 6 total, 6 up, 6 in 2024-09-16T00:29:33.338 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:32 smithi112 ceph-mon[97750]: mgrmap e38: smithi112.xrkkbc(active, since 15s), standbys: smithi169.mtrobk 2024-09-16T00:29:33.338 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:32 smithi112 ceph-mon[97750]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-09-16T00:29:33.338 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:32 smithi112 ceph-mon[97750]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T00:29:33.338 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:32 smithi112 ceph-mon[97750]: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T00:29:33.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:32 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:33.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:32 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:33.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:32 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:29:33.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:32 smithi112 ceph-mon[97750]: pgmap v11: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-16T00:29:35.926 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:35 smithi169 ceph-mon[77626]: pgmap v12: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 20 KiB/s rd, 0 B/s wr, 14 op/s 2024-09-16T00:29:35.926 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:35 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:35.926 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:35 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:35.926 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:35 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:35.926 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:35 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:36.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:35 smithi112 ceph-mon[97750]: pgmap v12: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 20 KiB/s rd, 0 B/s wr, 14 op/s 2024-09-16T00:29:36.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:35 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:36.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:35 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:36.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:35 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:36.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:35 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:37.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.239+0000 7fb8b2a0b640 1 -- 172.21.15.112:0/3206737539 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8ac102530 msgr2=0x7fb8ac102900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:37.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.239+0000 7fb8b2a0b640 1 --2- 172.21.15.112:0/3206737539 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8ac102530 0x7fb8ac102900 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb89c0048d0 tx=0x7fb89c009da0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.240+0000 7fb8b2a0b640 1 -- 172.21.15.112:0/3206737539 shutdown_connections 2024-09-16T00:29:37.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.240+0000 7fb8b2a0b640 1 --2- 172.21.15.112:0/3206737539 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8ac102ed0 0x7fb8ac0fc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.240+0000 7fb8b2a0b640 1 --2- 172.21.15.112:0/3206737539 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8ac102530 0x7fb8ac102900 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.240+0000 7fb8b2a0b640 1 -- 172.21.15.112:0/3206737539 >> 172.21.15.112:0/3206737539 conn(0x7fb8ac0f8110 msgr2=0x7fb8ac0fa520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:37.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.240+0000 7fb8b2a0b640 1 -- 172.21.15.112:0/3206737539 shutdown_connections 2024-09-16T00:29:37.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.240+0000 7fb8b2a0b640 1 -- 172.21.15.112:0/3206737539 wait complete. 2024-09-16T00:29:37.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.240+0000 7fb8b2a0b640 1 Processor -- start 2024-09-16T00:29:37.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.241+0000 7fb8b2a0b640 1 -- start start 2024-09-16T00:29:37.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.241+0000 7fb8b2a0b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8ac102530 0x7fb8ac06fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:37.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.241+0000 7fb8b2a0b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8ac102ed0 0x7fb8ac06ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:37.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.241+0000 7fb8b2a0b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ac070490 con 0x7fb8ac102ed0 2024-09-16T00:29:37.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.241+0000 7fb8b2a0b640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ac070600 con 0x7fb8ac102530 2024-09-16T00:29:37.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.241+0000 7fb8ab7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8ac102ed0 0x7fb8ac06ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:37.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.241+0000 7fb8ab7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8ac102ed0 0x7fb8ac06ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46084/0 (socket says 172.21.15.112:46084) 2024-09-16T00:29:37.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.241+0000 7fb8ab7fe640 1 -- 172.21.15.112:0/2328273783 learned_addr learned my addr 172.21.15.112:0/2328273783 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:29:37.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.241+0000 7fb8abfff640 1 --2- 172.21.15.112:0/2328273783 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8ac102530 0x7fb8ac06fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:37.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.241+0000 7fb8ab7fe640 1 -- 172.21.15.112:0/2328273783 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8ac102530 msgr2=0x7fb8ac06fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:37.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.241+0000 7fb8ab7fe640 1 --2- 172.21.15.112:0/2328273783 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8ac102530 0x7fb8ac06fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.241+0000 7fb8ab7fe640 1 -- 172.21.15.112:0/2328273783 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb89c005040 con 0x7fb8ac102ed0 2024-09-16T00:29:37.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.241+0000 7fb8ab7fe640 1 --2- 172.21.15.112:0/2328273783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8ac102ed0 0x7fb8ac06ff50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0002870 tx=0x7fb8a0004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:37.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.242+0000 7fb8a8ff9640 1 -- 172.21.15.112:0/2328273783 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8a0007c30 con 0x7fb8ac102ed0 2024-09-16T00:29:37.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.242+0000 7fb8a8ff9640 1 -- 172.21.15.112:0/2328273783 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fb8a000f040 con 0x7fb8ac102ed0 2024-09-16T00:29:37.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.242+0000 7fb8b2a0b640 1 -- 172.21.15.112:0/2328273783 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8ac070890 con 0x7fb8ac102ed0 2024-09-16T00:29:37.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.242+0000 7fb8a8ff9640 1 -- 172.21.15.112:0/2328273783 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8a0009640 con 0x7fb8ac102ed0 2024-09-16T00:29:37.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.242+0000 7fb8b2a0b640 1 -- 172.21.15.112:0/2328273783 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8ac1198c0 con 0x7fb8ac102ed0 2024-09-16T00:29:37.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.243+0000 7fb8b2a0b640 1 -- 172.21.15.112:0/2328273783 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8ac0fdcb0 con 0x7fb8ac102ed0 2024-09-16T00:29:37.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.243+0000 7fb8a8ff9640 1 -- 172.21.15.112:0/2328273783 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 38) v1 ==== 92655+0+0 (secure 0 0 0) 0x7fb8a00097a0 con 0x7fb8ac102ed0 2024-09-16T00:29:37.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.244+0000 7fb8a8ff9640 1 --2- 172.21.15.112:0/2328273783 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fb89406e660 0x7fb894070b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:37.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.244+0000 7fb8a8ff9640 1 -- 172.21.15.112:0/2328273783 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fb8a008e2c0 con 0x7fb8ac102ed0 2024-09-16T00:29:37.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.244+0000 7fb8abfff640 1 --2- 172.21.15.112:0/2328273783 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fb89406e660 0x7fb894070b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:37.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.244+0000 7fb8abfff640 1 --2- 172.21.15.112:0/2328273783 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fb89406e660 0x7fb894070b10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb89c004540 tx=0x7fb89c009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:37.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.248+0000 7fb8a8ff9640 1 -- 172.21.15.112:0/2328273783 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fb8a0057260 con 0x7fb8ac102ed0 2024-09-16T00:29:37.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.432+0000 7fb8b2a0b640 1 -- 172.21.15.112:0/2328273783 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb8ac1095f0 con 0x7fb89406e660 2024-09-16T00:29:37.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.433+0000 7fb8a8ff9640 1 -- 172.21.15.112:0/2328273783 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7fb8ac1095f0 con 0x7fb89406e660 2024-09-16T00:29:37.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.435+0000 7fb8b2a0b640 1 -- 172.21.15.112:0/2328273783 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fb89406e660 msgr2=0x7fb894070b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:37.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.435+0000 7fb8b2a0b640 1 --2- 172.21.15.112:0/2328273783 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fb89406e660 0x7fb894070b10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb89c004540 tx=0x7fb89c009da0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.435+0000 7fb8b2a0b640 1 -- 172.21.15.112:0/2328273783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8ac102ed0 msgr2=0x7fb8ac06ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:37.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.435+0000 7fb8b2a0b640 1 --2- 172.21.15.112:0/2328273783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8ac102ed0 0x7fb8ac06ff50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0002870 tx=0x7fb8a0004600 comp rx=0 tx=0).stop 2024-09-16T00:29:37.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.435+0000 7fb8b2a0b640 1 -- 172.21.15.112:0/2328273783 shutdown_connections 2024-09-16T00:29:37.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.435+0000 7fb8b2a0b640 1 --2- 172.21.15.112:0/2328273783 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fb89406e660 0x7fb894070b10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.435+0000 7fb8b2a0b640 1 --2- 172.21.15.112:0/2328273783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8ac102ed0 0x7fb8ac06ff50 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.435+0000 7fb8b2a0b640 1 --2- 172.21.15.112:0/2328273783 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8ac102530 0x7fb8ac06fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.435+0000 7fb8b2a0b640 1 -- 172.21.15.112:0/2328273783 >> 172.21.15.112:0/2328273783 conn(0x7fb8ac0f8110 msgr2=0x7fb8ac0f9100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:37.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.436+0000 7fb8b2a0b640 1 -- 172.21.15.112:0/2328273783 shutdown_connections 2024-09-16T00:29:37.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.436+0000 7fb8b2a0b640 1 -- 172.21.15.112:0/2328273783 wait complete. 2024-09-16T00:29:37.444 INFO:teuthology.orchestra.run.smithi112.stdout:true 2024-09-16T00:29:37.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.514+0000 7f9314c68640 1 -- 172.21.15.112:0/3758231637 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93100fb810 msgr2=0x7f93100fbbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:37.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.514+0000 7f9314c68640 1 --2- 172.21.15.112:0/3758231637 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93100fb810 0x7f93100fbbe0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f93000048d0 tx=0x7f9300009da0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.514+0000 7f9314c68640 1 -- 172.21.15.112:0/3758231637 shutdown_connections 2024-09-16T00:29:37.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.514+0000 7f9314c68640 1 --2- 172.21.15.112:0/3758231637 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93100fcf80 0x7f93100fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.514+0000 7f9314c68640 1 --2- 172.21.15.112:0/3758231637 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93100fb810 0x7f93100fbbe0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.514+0000 7f9314c68640 1 -- 172.21.15.112:0/3758231637 >> 172.21.15.112:0/3758231637 conn(0x7f93100723b0 msgr2=0x7f93100727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:37.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.514+0000 7f9314c68640 1 -- 172.21.15.112:0/3758231637 shutdown_connections 2024-09-16T00:29:37.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.514+0000 7f9314c68640 1 -- 172.21.15.112:0/3758231637 wait complete. 2024-09-16T00:29:37.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.515+0000 7f9314c68640 1 Processor -- start 2024-09-16T00:29:37.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.515+0000 7f9314c68640 1 -- start start 2024-09-16T00:29:37.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.515+0000 7f9314c68640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93100fb810 0x7f93101153a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:37.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.515+0000 7f9314c68640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93100fcf80 0x7f93101158e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:37.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.515+0000 7f9314c68640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9310115ee0 con 0x7f93100fcf80 2024-09-16T00:29:37.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.515+0000 7f9314c68640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9310116050 con 0x7f93100fb810 2024-09-16T00:29:37.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.515+0000 7f9307fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93100fcf80 0x7f93101158e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:37.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.515+0000 7f9307fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93100fcf80 0x7f93101158e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46108/0 (socket says 172.21.15.112:46108) 2024-09-16T00:29:37.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.515+0000 7f930e59c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93100fb810 0x7f93101153a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:37.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.515+0000 7f9307fff640 1 -- 172.21.15.112:0/3489386755 learned_addr learned my addr 172.21.15.112:0/3489386755 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:29:37.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.515+0000 7f9307fff640 1 -- 172.21.15.112:0/3489386755 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93100fb810 msgr2=0x7f93101153a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:37.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.515+0000 7f9307fff640 1 --2- 172.21.15.112:0/3489386755 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93100fb810 0x7f93101153a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.515+0000 7f9307fff640 1 -- 172.21.15.112:0/3489386755 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9300005040 con 0x7f93100fcf80 2024-09-16T00:29:37.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.516+0000 7f9307fff640 1 --2- 172.21.15.112:0/3489386755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93100fcf80 0x7f93101158e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f92f8002a50 tx=0x7f92f8004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:37.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.516+0000 7f9306ffd640 1 -- 172.21.15.112:0/3489386755 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92f8007c30 con 0x7f93100fcf80 2024-09-16T00:29:37.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.516+0000 7f9306ffd640 1 -- 172.21.15.112:0/3489386755 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f92f800f040 con 0x7f93100fcf80 2024-09-16T00:29:37.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.516+0000 7f9314c68640 1 -- 172.21.15.112:0/3489386755 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f931010f4d0 con 0x7f93100fcf80 2024-09-16T00:29:37.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.516+0000 7f9314c68640 1 -- 172.21.15.112:0/3489386755 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f931010fa70 con 0x7f93100fcf80 2024-09-16T00:29:37.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.516+0000 7f9306ffd640 1 -- 172.21.15.112:0/3489386755 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92f800dad0 con 0x7f93100fcf80 2024-09-16T00:29:37.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.517+0000 7f9314c68640 1 -- 172.21.15.112:0/3489386755 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93100fcbc0 con 0x7f93100fcf80 2024-09-16T00:29:37.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.517+0000 7f9306ffd640 1 -- 172.21.15.112:0/3489386755 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 38) v1 ==== 92655+0+0 (secure 0 0 0) 0x7f92f800dc30 con 0x7f93100fcf80 2024-09-16T00:29:37.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.518+0000 7f9306ffd640 1 --2- 172.21.15.112:0/3489386755 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f92fc06e660 0x7f92fc070b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:37.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.518+0000 7f9306ffd640 1 -- 172.21.15.112:0/3489386755 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f92f808e160 con 0x7f93100fcf80 2024-09-16T00:29:37.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.518+0000 7f930e59c640 1 --2- 172.21.15.112:0/3489386755 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f92fc06e660 0x7f92fc070b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:37.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.519+0000 7f930e59c640 1 --2- 172.21.15.112:0/3489386755 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f92fc06e660 0x7f92fc070b10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9300009ac0 tx=0x7f9300035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:37.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.522+0000 7f9306ffd640 1 -- 172.21.15.112:0/3489386755 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f92f8057100 con 0x7f93100fcf80 2024-09-16T00:29:37.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.709+0000 7f9314c68640 1 -- 172.21.15.112:0/3489386755 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f931004b720 con 0x7f92fc06e660 2024-09-16T00:29:37.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.710+0000 7f9306ffd640 1 -- 172.21.15.112:0/3489386755 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f931004b720 con 0x7f92fc06e660 2024-09-16T00:29:37.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.712+0000 7f9314c68640 1 -- 172.21.15.112:0/3489386755 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f92fc06e660 msgr2=0x7f92fc070b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:37.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.712+0000 7f9314c68640 1 --2- 172.21.15.112:0/3489386755 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f92fc06e660 0x7f92fc070b10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9300009ac0 tx=0x7f9300035040 comp rx=0 tx=0).stop 2024-09-16T00:29:37.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.712+0000 7f9314c68640 1 -- 172.21.15.112:0/3489386755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93100fcf80 msgr2=0x7f93101158e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:37.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.712+0000 7f9314c68640 1 --2- 172.21.15.112:0/3489386755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93100fcf80 0x7f93101158e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f92f8002a50 tx=0x7f92f8004600 comp rx=0 tx=0).stop 2024-09-16T00:29:37.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.713+0000 7f9314c68640 1 -- 172.21.15.112:0/3489386755 shutdown_connections 2024-09-16T00:29:37.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.713+0000 7f9314c68640 1 --2- 172.21.15.112:0/3489386755 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f92fc06e660 0x7f92fc070b10 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.713+0000 7f9314c68640 1 --2- 172.21.15.112:0/3489386755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93100fcf80 0x7f93101158e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.713+0000 7f9314c68640 1 --2- 172.21.15.112:0/3489386755 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93100fb810 0x7f93101153a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.713+0000 7f9314c68640 1 -- 172.21.15.112:0/3489386755 >> 172.21.15.112:0/3489386755 conn(0x7f93100723b0 msgr2=0x7f93100f8470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:37.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.713+0000 7f9314c68640 1 -- 172.21.15.112:0/3489386755 shutdown_connections 2024-09-16T00:29:37.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.713+0000 7f9314c68640 1 -- 172.21.15.112:0/3489386755 wait complete. 2024-09-16T00:29:37.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.790+0000 7f7627ad6640 1 -- 172.21.15.112:0/260727057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76200fe460 msgr2=0x7f76200fe830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:37.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.790+0000 7f7627ad6640 1 --2- 172.21.15.112:0/260727057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76200fe460 0x7f76200fe830 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7610009c50 tx=0x7f7610004d10 comp rx=0 tx=0).stop 2024-09-16T00:29:37.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.791+0000 7f7627ad6640 1 -- 172.21.15.112:0/260727057 shutdown_connections 2024-09-16T00:29:37.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.791+0000 7f7627ad6640 1 --2- 172.21.15.112:0/260727057 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7620064fc0 0x7f7620065430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.791+0000 7f7627ad6640 1 --2- 172.21.15.112:0/260727057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76200fe460 0x7f76200fe830 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.791+0000 7f7627ad6640 1 -- 172.21.15.112:0/260727057 >> 172.21.15.112:0/260727057 conn(0x7f76200723b0 msgr2=0x7f76200727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:37.792 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:37 smithi112 ceph-mon[97750]: pgmap v13: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:29:37.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.791+0000 7f7627ad6640 1 -- 172.21.15.112:0/260727057 shutdown_connections 2024-09-16T00:29:37.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.791+0000 7f7627ad6640 1 -- 172.21.15.112:0/260727057 wait complete. 2024-09-16T00:29:37.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.791+0000 7f7627ad6640 1 Processor -- start 2024-09-16T00:29:37.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.791+0000 7f7627ad6640 1 -- start start 2024-09-16T00:29:37.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.791+0000 7f7627ad6640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7620064fc0 0x7f7620113200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:37.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.791+0000 7f7627ad6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76200fe460 0x7f7620113740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:37.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.791+0000 7f7627ad6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7620113dd0 con 0x7f76200fe460 2024-09-16T00:29:37.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.791+0000 7f7627ad6640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f762010d180 con 0x7f7620064fc0 2024-09-16T00:29:37.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.792+0000 7f7625071640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76200fe460 0x7f7620113740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:37.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.792+0000 7f7625071640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76200fe460 0x7f7620113740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46116/0 (socket says 172.21.15.112:46116) 2024-09-16T00:29:37.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.792+0000 7f7625071640 1 -- 172.21.15.112:0/2858243365 learned_addr learned my addr 172.21.15.112:0/2858243365 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:29:37.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.792+0000 7f7625872640 1 --2- 172.21.15.112:0/2858243365 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7620064fc0 0x7f7620113200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:37.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.792+0000 7f7625071640 1 -- 172.21.15.112:0/2858243365 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7620064fc0 msgr2=0x7f7620113200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:37.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.792+0000 7f7625071640 1 --2- 172.21.15.112:0/2858243365 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7620064fc0 0x7f7620113200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.792+0000 7f7625071640 1 -- 172.21.15.112:0/2858243365 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76100098b0 con 0x7f76200fe460 2024-09-16T00:29:37.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.792+0000 7f7625872640 1 --2- 172.21.15.112:0/2858243365 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7620064fc0 0x7f7620113200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:29:37.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.792+0000 7f7625071640 1 --2- 172.21.15.112:0/2858243365 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76200fe460 0x7f7620113740 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f761400b790 tx=0x7f76140091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:37.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.792+0000 7f761e7fc640 1 -- 172.21.15.112:0/2858243365 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f761400c700 con 0x7f76200fe460 2024-09-16T00:29:37.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.792+0000 7f761e7fc640 1 -- 172.21.15.112:0/2858243365 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f761400cd00 con 0x7f76200fe460 2024-09-16T00:29:37.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.792+0000 7f7627ad6640 1 -- 172.21.15.112:0/2858243365 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f762010d460 con 0x7f76200fe460 2024-09-16T00:29:37.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.792+0000 7f761e7fc640 1 -- 172.21.15.112:0/2858243365 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7614015e70 con 0x7f76200fe460 2024-09-16T00:29:37.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.792+0000 7f7627ad6640 1 -- 172.21.15.112:0/2858243365 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f762010d980 con 0x7f76200fe460 2024-09-16T00:29:37.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.793+0000 7f7627ad6640 1 -- 172.21.15.112:0/2858243365 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76200fdcf0 con 0x7f76200fe460 2024-09-16T00:29:37.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.794+0000 7f761e7fc640 1 -- 172.21.15.112:0/2858243365 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 38) v1 ==== 92655+0+0 (secure 0 0 0) 0x7f7614012070 con 0x7f76200fe460 2024-09-16T00:29:37.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.794+0000 7f761e7fc640 1 --2- 172.21.15.112:0/2858243365 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f760806e660 0x7f7608070b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:37.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.794+0000 7f761e7fc640 1 -- 172.21.15.112:0/2858243365 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f761408dba0 con 0x7f76200fe460 2024-09-16T00:29:37.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.794+0000 7f7625872640 1 --2- 172.21.15.112:0/2858243365 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f760806e660 0x7f7608070b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:37.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.795+0000 7f7625872640 1 --2- 172.21.15.112:0/2858243365 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f760806e660 0x7f7608070b10 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7610004a30 tx=0x7f7610004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:37.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.799+0000 7f761e7fc640 1 -- 172.21.15.112:0/2858243365 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f7614056c50 con 0x7f76200fe460 2024-09-16T00:29:37.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.976+0000 7f7627ad6640 1 -- 172.21.15.112:0/2858243365 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f762010e870 con 0x7f760806e660 2024-09-16T00:29:37.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.985+0000 7f761e7fc640 1 -- 172.21.15.112:0/2858243365 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f762010e870 con 0x7f760806e660 2024-09-16T00:29:37.986 INFO:teuthology.orchestra.run.smithi112.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T00:29:37.986 INFO:teuthology.orchestra.run.smithi112.stdout:alertmanager.smithi112 smithi112 *:9093,9094 running (2m) 15s ago 11m 20.7M - 0.25.0 c8568f914cd2 d85ea0562022 2024-09-16T00:29:37.986 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi112 smithi112 running (11m) 15s ago 11m 6891k - 17.2.7-1633-g6d025bc2 33ed8dba3996 c94220ed2776 2024-09-16T00:29:37.986 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi169 smithi169 running (10m) 3s ago 10m 6899k - 17.2.7-1633-g6d025bc2 33ed8dba3996 d0d3726280a2 2024-09-16T00:29:37.986 INFO:teuthology.orchestra.run.smithi112.stdout:grafana.smithi112 smithi112 *:3000 running (80s) 15s ago 11m 74.2M - 9.4.12 2bacad6d85d8 51f9d2013da2 2024-09-16T00:29:37.986 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.hxfwkh smithi112 running (6m) 15s ago 6m 136M - 17.2.7-1633-g6d025bc2 33ed8dba3996 cf92d9461fc6 2024-09-16T00:29:37.986 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.ithwzj smithi112 running (6m) 15s ago 6m 65.9M - 17.2.7-1633-g6d025bc2 33ed8dba3996 02786c8a83c7 2024-09-16T00:29:37.986 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.gawwgw smithi169 running (6m) 3s ago 6m 114M - 17.2.7-1633-g6d025bc2 33ed8dba3996 dcf3a4dfa879 2024-09-16T00:29:37.986 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.lgeqrj smithi169 running (6m) 3s ago 6m 60.4M - 17.2.7-1633-g6d025bc2 33ed8dba3996 445c14e55831 2024-09-16T00:29:37.986 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi112.xrkkbc smithi112 *:8443,9283,8765 running (2m) 15s ago 12m 472M - 19.1.1-326-g8921c8ef f1455bd0219c a981f7fd2810 2024-09-16T00:29:37.987 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi169.mtrobk smithi169 *:8443,9283,8765 running (101s) 3s ago 10m 438M - 19.1.1-326-g8921c8ef f1455bd0219c 4096e5369c2d 2024-09-16T00:29:37.987 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi112 smithi112 running (28s) 15s ago 12m 27.7M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 5aaec033c042 2024-09-16T00:29:37.987 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi169 smithi169 running (6s) 3s ago 10m 19.8M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 7bc609155f3f 2024-09-16T00:29:37.987 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi112 smithi112 *:9100 running (2m) 15s ago 11m 15.6M - 1.5.0 0da6a335fe13 6404d1d601e2 2024-09-16T00:29:37.987 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi169 smithi169 *:9100 running (2m) 3s ago 10m 15.0M - 1.5.0 0da6a335fe13 5505baf702a6 2024-09-16T00:29:37.987 INFO:teuthology.orchestra.run.smithi112.stdout:osd.0 smithi112 running (9m) 15s ago 9m 191M 1557M 17.2.7-1633-g6d025bc2 33ed8dba3996 f6b2e146ffdf 2024-09-16T00:29:37.987 INFO:teuthology.orchestra.run.smithi112.stdout:osd.1 smithi112 running (9m) 15s ago 9m 201M 1557M 17.2.7-1633-g6d025bc2 33ed8dba3996 e9bee87bba0f 2024-09-16T00:29:37.987 INFO:teuthology.orchestra.run.smithi112.stdout:osd.2 smithi112 running (8m) 15s ago 8m 159M 1557M 17.2.7-1633-g6d025bc2 33ed8dba3996 100ec4e4852a 2024-09-16T00:29:37.987 INFO:teuthology.orchestra.run.smithi112.stdout:osd.3 smithi169 running (8m) 3s ago 8m 231M 2581M 17.2.7-1633-g6d025bc2 33ed8dba3996 f88463e19f23 2024-09-16T00:29:37.987 INFO:teuthology.orchestra.run.smithi112.stdout:osd.4 smithi169 running (8m) 3s ago 8m 211M 2581M 17.2.7-1633-g6d025bc2 33ed8dba3996 d3e4cdb016b9 2024-09-16T00:29:37.987 INFO:teuthology.orchestra.run.smithi112.stdout:osd.5 smithi169 running (7m) 3s ago 7m 198M 2581M 17.2.7-1633-g6d025bc2 33ed8dba3996 8e7d708d74eb 2024-09-16T00:29:37.987 INFO:teuthology.orchestra.run.smithi112.stdout:prometheus.smithi112 smithi112 *:9095 running (105s) 15s ago 11m 53.7M - 2.43.0 a07b618ecd1d 7a1d42e60d2b 2024-09-16T00:29:37.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.988+0000 7f7627ad6640 1 -- 172.21.15.112:0/2858243365 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f760806e660 msgr2=0x7f7608070b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:37.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.988+0000 7f7627ad6640 1 --2- 172.21.15.112:0/2858243365 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f760806e660 0x7f7608070b10 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7610004a30 tx=0x7f7610004d10 comp rx=0 tx=0).stop 2024-09-16T00:29:37.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.988+0000 7f7627ad6640 1 -- 172.21.15.112:0/2858243365 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76200fe460 msgr2=0x7f7620113740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:37.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.988+0000 7f7627ad6640 1 --2- 172.21.15.112:0/2858243365 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76200fe460 0x7f7620113740 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f761400b790 tx=0x7f76140091d0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.988+0000 7f7627ad6640 1 -- 172.21.15.112:0/2858243365 shutdown_connections 2024-09-16T00:29:37.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.988+0000 7f7627ad6640 1 --2- 172.21.15.112:0/2858243365 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f760806e660 0x7f7608070b10 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.988+0000 7f7627ad6640 1 --2- 172.21.15.112:0/2858243365 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76200fe460 0x7f7620113740 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.988+0000 7f7627ad6640 1 --2- 172.21.15.112:0/2858243365 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7620064fc0 0x7f7620113200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.988+0000 7f7627ad6640 1 -- 172.21.15.112:0/2858243365 >> 172.21.15.112:0/2858243365 conn(0x7f76200723b0 msgr2=0x7f7620105410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:37.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.988+0000 7f7627ad6640 1 -- 172.21.15.112:0/2858243365 shutdown_connections 2024-09-16T00:29:37.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:37.988+0000 7f7627ad6640 1 -- 172.21.15.112:0/2858243365 wait complete. 2024-09-16T00:29:38.021 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:37 smithi169 ceph-mon[77626]: pgmap v13: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:29:38.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.065+0000 7f981678c640 1 -- 172.21.15.112:0/3836142958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9810102620 msgr2=0x7f98101029f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:38.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.065+0000 7f981678c640 1 --2- 172.21.15.112:0/3836142958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9810102620 0x7f98101029f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f98000048d0 tx=0x7f9800009da0 comp rx=0 tx=0).stop 2024-09-16T00:29:38.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.065+0000 7f981678c640 1 -- 172.21.15.112:0/3836142958 shutdown_connections 2024-09-16T00:29:38.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.065+0000 7f981678c640 1 --2- 172.21.15.112:0/3836142958 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98100fac80 0x7f98100fb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:38.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.065+0000 7f981678c640 1 --2- 172.21.15.112:0/3836142958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9810102620 0x7f98101029f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:38.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.065+0000 7f981678c640 1 -- 172.21.15.112:0/3836142958 >> 172.21.15.112:0/3836142958 conn(0x7f9810070ce0 msgr2=0x7f98100710e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:38.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.065+0000 7f981678c640 1 -- 172.21.15.112:0/3836142958 shutdown_connections 2024-09-16T00:29:38.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.065+0000 7f981678c640 1 -- 172.21.15.112:0/3836142958 wait complete. 2024-09-16T00:29:38.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.066+0000 7f981678c640 1 Processor -- start 2024-09-16T00:29:38.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.066+0000 7f981678c640 1 -- start start 2024-09-16T00:29:38.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.066+0000 7f981678c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98100fac80 0x7f98101131c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:38.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.066+0000 7f981678c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9810102620 0x7f9810113700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:38.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.066+0000 7f981678c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9810113d00 con 0x7f98100fac80 2024-09-16T00:29:38.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.066+0000 7f981678c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f981010d140 con 0x7f9810102620 2024-09-16T00:29:38.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.066+0000 7f980ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98100fac80 0x7f98101131c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:38.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.066+0000 7f980ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98100fac80 0x7f98101131c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46146/0 (socket says 172.21.15.112:46146) 2024-09-16T00:29:38.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.066+0000 7f980ffff640 1 -- 172.21.15.112:0/2558442208 learned_addr learned my addr 172.21.15.112:0/2558442208 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:29:38.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.066+0000 7f980f7fe640 1 --2- 172.21.15.112:0/2558442208 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9810102620 0x7f9810113700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:38.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.067+0000 7f980ffff640 1 -- 172.21.15.112:0/2558442208 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9810102620 msgr2=0x7f9810113700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:38.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.067+0000 7f980ffff640 1 --2- 172.21.15.112:0/2558442208 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9810102620 0x7f9810113700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:38.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.067+0000 7f980ffff640 1 -- 172.21.15.112:0/2558442208 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9800005040 con 0x7f98100fac80 2024-09-16T00:29:38.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.067+0000 7f980f7fe640 1 --2- 172.21.15.112:0/2558442208 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9810102620 0x7f9810113700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:29:38.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.067+0000 7f980ffff640 1 --2- 172.21.15.112:0/2558442208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98100fac80 0x7f98101131c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9800000c00 tx=0x7f9800009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:38.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.067+0000 7f980cff9640 1 -- 172.21.15.112:0/2558442208 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9800038070 con 0x7f98100fac80 2024-09-16T00:29:38.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.067+0000 7f981678c640 1 -- 172.21.15.112:0/2558442208 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f981010d3c0 con 0x7f98100fac80 2024-09-16T00:29:38.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.067+0000 7f980cff9640 1 -- 172.21.15.112:0/2558442208 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f980002ade0 con 0x7f98100fac80 2024-09-16T00:29:38.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.067+0000 7f980cff9640 1 -- 172.21.15.112:0/2558442208 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f980002c7b0 con 0x7f98100fac80 2024-09-16T00:29:38.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.067+0000 7f981678c640 1 -- 172.21.15.112:0/2558442208 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f981010d900 con 0x7f98100fac80 2024-09-16T00:29:38.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.068+0000 7f981678c640 1 -- 172.21.15.112:0/2558442208 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98100657b0 con 0x7f98100fac80 2024-09-16T00:29:38.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.068+0000 7f980cff9640 1 -- 172.21.15.112:0/2558442208 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 38) v1 ==== 92655+0+0 (secure 0 0 0) 0x7f9800030020 con 0x7f98100fac80 2024-09-16T00:29:38.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.069+0000 7f980cff9640 1 --2- 172.21.15.112:0/2558442208 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f97f806e660 0x7f97f8070b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:38.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.069+0000 7f980cff9640 1 -- 172.21.15.112:0/2558442208 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f98000ae6d0 con 0x7f98100fac80 2024-09-16T00:29:38.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.069+0000 7f980f7fe640 1 --2- 172.21.15.112:0/2558442208 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f97f806e660 0x7f97f8070b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:38.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.070+0000 7f980f7fe640 1 --2- 172.21.15.112:0/2558442208 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f97f806e660 0x7f97f8070b10 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f981010ea80 tx=0x7f980400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:38.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.073+0000 7f980cff9640 1 -- 172.21.15.112:0/2558442208 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f9800077780 con 0x7f98100fac80 2024-09-16T00:29:38.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.307+0000 7f981678c640 1 -- 172.21.15.112:0/2558442208 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f9810074c00 con 0x7f98100fac80 2024-09-16T00:29:38.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.308+0000 7f980cff9640 1 -- 172.21.15.112:0/2558442208 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+745 (secure 0 0 0) 0x7f980007ae40 con 0x7f98100fac80 2024-09-16T00:29:38.308 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:29:38.308 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": { 2024-09-16T00:29:38.308 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:29:38.308 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:29:38.308 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": { 2024-09-16T00:29:38.308 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:29:38.309 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:29:38.309 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": { 2024-09-16T00:29:38.309 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 17.2.7-1633-g6d025bc2 (6d025bc26f3176c910707f8cb4c8d8e076640615) quincy (stable)": 6 2024-09-16T00:29:38.309 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:29:38.309 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": { 2024-09-16T00:29:38.309 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 17.2.7-1633-g6d025bc2 (6d025bc26f3176c910707f8cb4c8d8e076640615) quincy (stable)": 4 2024-09-16T00:29:38.309 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:29:38.309 INFO:teuthology.orchestra.run.smithi112.stdout: "overall": { 2024-09-16T00:29:38.309 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 17.2.7-1633-g6d025bc2 (6d025bc26f3176c910707f8cb4c8d8e076640615) quincy (stable)": 10, 2024-09-16T00:29:38.309 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 4 2024-09-16T00:29:38.309 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-16T00:29:38.309 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:29:38.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.310+0000 7f981678c640 1 -- 172.21.15.112:0/2558442208 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f97f806e660 msgr2=0x7f97f8070b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:38.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.310+0000 7f981678c640 1 --2- 172.21.15.112:0/2558442208 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f97f806e660 0x7f97f8070b10 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f981010ea80 tx=0x7f980400b040 comp rx=0 tx=0).stop 2024-09-16T00:29:38.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.310+0000 7f981678c640 1 -- 172.21.15.112:0/2558442208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98100fac80 msgr2=0x7f98101131c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:38.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.310+0000 7f981678c640 1 --2- 172.21.15.112:0/2558442208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98100fac80 0x7f98101131c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9800000c00 tx=0x7f9800009da0 comp rx=0 tx=0).stop 2024-09-16T00:29:38.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.310+0000 7f981678c640 1 -- 172.21.15.112:0/2558442208 shutdown_connections 2024-09-16T00:29:38.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.310+0000 7f981678c640 1 --2- 172.21.15.112:0/2558442208 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f97f806e660 0x7f97f8070b10 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:38.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.310+0000 7f981678c640 1 --2- 172.21.15.112:0/2558442208 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9810102620 0x7f9810113700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:38.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.310+0000 7f981678c640 1 --2- 172.21.15.112:0/2558442208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98100fac80 0x7f98101131c0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:38.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.310+0000 7f981678c640 1 -- 172.21.15.112:0/2558442208 >> 172.21.15.112:0/2558442208 conn(0x7f9810070ce0 msgr2=0x7f9810105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:38.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.310+0000 7f981678c640 1 -- 172.21.15.112:0/2558442208 shutdown_connections 2024-09-16T00:29:38.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.310+0000 7f981678c640 1 -- 172.21.15.112:0/2558442208 wait complete. 2024-09-16T00:29:38.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.396+0000 7f040797b640 1 -- 172.21.15.112:0/283826410 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04000fe450 msgr2=0x7f04000fe820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:38.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.396+0000 7f040797b640 1 --2- 172.21.15.112:0/283826410 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04000fe450 0x7f04000fe820 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f03fc0048d0 tx=0x7f03fc009da0 comp rx=0 tx=0).stop 2024-09-16T00:29:38.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.396+0000 7f040797b640 1 -- 172.21.15.112:0/283826410 shutdown_connections 2024-09-16T00:29:38.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.396+0000 7f040797b640 1 --2- 172.21.15.112:0/283826410 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0400064fe0 0x7f0400065430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:38.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.396+0000 7f040797b640 1 --2- 172.21.15.112:0/283826410 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04000fe450 0x7f04000fe820 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:38.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.396+0000 7f040797b640 1 -- 172.21.15.112:0/283826410 >> 172.21.15.112:0/283826410 conn(0x7f04000723b0 msgr2=0x7f04000727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:38.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.396+0000 7f040797b640 1 -- 172.21.15.112:0/283826410 shutdown_connections 2024-09-16T00:29:38.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.396+0000 7f040797b640 1 -- 172.21.15.112:0/283826410 wait complete. 2024-09-16T00:29:38.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.396+0000 7f040797b640 1 Processor -- start 2024-09-16T00:29:38.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.396+0000 7f040797b640 1 -- start start 2024-09-16T00:29:38.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.397+0000 7f040797b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0400064fe0 0x7f04001131d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:38.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.397+0000 7f040797b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04000fe450 0x7f0400113710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:38.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.397+0000 7f040797b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0400113d10 con 0x7f0400064fe0 2024-09-16T00:29:38.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.397+0000 7f040797b640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f040010d150 con 0x7f04000fe450 2024-09-16T00:29:38.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.397+0000 7f0405717640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0400064fe0 0x7f04001131d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:38.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.397+0000 7f0405717640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0400064fe0 0x7f04001131d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46162/0 (socket says 172.21.15.112:46162) 2024-09-16T00:29:38.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.397+0000 7f0405717640 1 -- 172.21.15.112:0/472240843 learned_addr learned my addr 172.21.15.112:0/472240843 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:29:38.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.397+0000 7f0404f16640 1 --2- 172.21.15.112:0/472240843 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04000fe450 0x7f0400113710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:38.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.397+0000 7f0405717640 1 -- 172.21.15.112:0/472240843 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04000fe450 msgr2=0x7f0400113710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:38.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.397+0000 7f0405717640 1 --2- 172.21.15.112:0/472240843 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04000fe450 0x7f0400113710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:38.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.397+0000 7f0405717640 1 -- 172.21.15.112:0/472240843 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03fc005040 con 0x7f0400064fe0 2024-09-16T00:29:38.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.397+0000 7f0405717640 1 --2- 172.21.15.112:0/472240843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0400064fe0 0x7f04001131d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f03fc000c00 tx=0x7f03fc009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:38.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.398+0000 7f03f5ffb640 1 -- 172.21.15.112:0/472240843 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03fc038070 con 0x7f0400064fe0 2024-09-16T00:29:38.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.398+0000 7f03f5ffb640 1 -- 172.21.15.112:0/472240843 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f03fc02ade0 con 0x7f0400064fe0 2024-09-16T00:29:38.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.398+0000 7f040797b640 1 -- 172.21.15.112:0/472240843 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f040010d3d0 con 0x7f0400064fe0 2024-09-16T00:29:38.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.398+0000 7f03f5ffb640 1 -- 172.21.15.112:0/472240843 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03fc02c7e0 con 0x7f0400064fe0 2024-09-16T00:29:38.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.398+0000 7f040797b640 1 -- 172.21.15.112:0/472240843 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f040010d910 con 0x7f0400064fe0 2024-09-16T00:29:38.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.399+0000 7f040797b640 1 -- 172.21.15.112:0/472240843 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04000fdce0 con 0x7f0400064fe0 2024-09-16T00:29:38.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.399+0000 7f03f5ffb640 1 -- 172.21.15.112:0/472240843 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 38) v1 ==== 92655+0+0 (secure 0 0 0) 0x7f03fc030020 con 0x7f0400064fe0 2024-09-16T00:29:38.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.400+0000 7f03f5ffb640 1 --2- 172.21.15.112:0/472240843 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f03ec06e660 0x7f03ec070b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:38.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.400+0000 7f03f5ffb640 1 -- 172.21.15.112:0/472240843 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f03fc0ade70 con 0x7f0400064fe0 2024-09-16T00:29:38.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.400+0000 7f0404f16640 1 --2- 172.21.15.112:0/472240843 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f03ec06e660 0x7f03ec070b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:38.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.401+0000 7f0404f16640 1 --2- 172.21.15.112:0/472240843 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f03ec06e660 0x7f03ec070b10 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f040010ea90 tx=0x7f03f000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:38.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.404+0000 7f03f5ffb640 1 -- 172.21.15.112:0/472240843 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f03fc076fd0 con 0x7f0400064fe0 2024-09-16T00:29:38.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.618+0000 7f040797b640 1 -- 172.21.15.112:0/472240843 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f040004b750 con 0x7f0400064fe0 2024-09-16T00:29:38.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.619+0000 7f03f5ffb640 1 -- 172.21.15.112:0/472240843 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1973 (secure 0 0 0) 0x7f03fc07a690 con 0x7f0400064fe0 2024-09-16T00:29:38.620 INFO:teuthology.orchestra.run.smithi112.stdout:e21 2024-09-16T00:29:38.620 INFO:teuthology.orchestra.run.smithi112.stdout:btime 1970-01-01T00:00:00:000000+0000 2024-09-16T00:29:38.620 INFO:teuthology.orchestra.run.smithi112.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T00:29:38.620 INFO:teuthology.orchestra.run.smithi112.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:29:38.620 INFO:teuthology.orchestra.run.smithi112.stdout:legacy client fscid: 1 2024-09-16T00:29:38.620 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:29:38.620 INFO:teuthology.orchestra.run.smithi112.stdout:Filesystem 'cephfs' (1) 2024-09-16T00:29:38.620 INFO:teuthology.orchestra.run.smithi112.stdout:fs_name cephfs 2024-09-16T00:29:38.620 INFO:teuthology.orchestra.run.smithi112.stdout:epoch 21 2024-09-16T00:29:38.620 INFO:teuthology.orchestra.run.smithi112.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-16T00:29:38.620 INFO:teuthology.orchestra.run.smithi112.stdout:created 2024-09-16T00:23:22.328199+0000 2024-09-16T00:29:38.621 INFO:teuthology.orchestra.run.smithi112.stdout:modified 2024-09-16T00:26:38.963153+0000 2024-09-16T00:29:38.621 INFO:teuthology.orchestra.run.smithi112.stdout:tableserver 0 2024-09-16T00:29:38.621 INFO:teuthology.orchestra.run.smithi112.stdout:root 0 2024-09-16T00:29:38.621 INFO:teuthology.orchestra.run.smithi112.stdout:session_timeout 60 2024-09-16T00:29:38.621 INFO:teuthology.orchestra.run.smithi112.stdout:session_autoclose 300 2024-09-16T00:29:38.621 INFO:teuthology.orchestra.run.smithi112.stdout:max_file_size 1099511627776 2024-09-16T00:29:38.621 INFO:teuthology.orchestra.run.smithi112.stdout:max_xattr_size 65536 2024-09-16T00:29:38.621 INFO:teuthology.orchestra.run.smithi112.stdout:required_client_features {} 2024-09-16T00:29:38.621 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure 0 2024-09-16T00:29:38.621 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure_osd_epoch 44 2024-09-16T00:29:38.621 INFO:teuthology.orchestra.run.smithi112.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:29:38.621 INFO:teuthology.orchestra.run.smithi112.stdout:max_mds 2 2024-09-16T00:29:38.621 INFO:teuthology.orchestra.run.smithi112.stdout:in 0,1 2024-09-16T00:29:38.621 INFO:teuthology.orchestra.run.smithi112.stdout:up {0=14480,1=24259} 2024-09-16T00:29:38.621 INFO:teuthology.orchestra.run.smithi112.stdout:failed 2024-09-16T00:29:38.622 INFO:teuthology.orchestra.run.smithi112.stdout:damaged 2024-09-16T00:29:38.622 INFO:teuthology.orchestra.run.smithi112.stdout:stopped 2024-09-16T00:29:38.622 INFO:teuthology.orchestra.run.smithi112.stdout:data_pools [3] 2024-09-16T00:29:38.622 INFO:teuthology.orchestra.run.smithi112.stdout:metadata_pool 2 2024-09-16T00:29:38.622 INFO:teuthology.orchestra.run.smithi112.stdout:inline_data enabled 2024-09-16T00:29:38.622 INFO:teuthology.orchestra.run.smithi112.stdout:balancer 2024-09-16T00:29:38.622 INFO:teuthology.orchestra.run.smithi112.stdout:bal_rank_mask -1 2024-09-16T00:29:38.622 INFO:teuthology.orchestra.run.smithi112.stdout:standby_count_wanted 1 2024-09-16T00:29:38.622 INFO:teuthology.orchestra.run.smithi112.stdout:qdb_cluster leader: 0 members: 2024-09-16T00:29:38.622 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.hxfwkh{0:14480} state up:active seq 6 export targets 1 join_fscid=1 addr [v2:172.21.15.112:6828/486654885,v1:172.21.15.112:6829/486654885] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T00:29:38.622 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.ithwzj{0:14492} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.112:6826/381830565,v1:172.21.15.112:6827/381830565] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T00:29:38.622 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.gawwgw{1:24259} state up:active seq 4 export targets 0 join_fscid=1 addr [v2:172.21.15.169:6824/2892476953,v1:172.21.15.169:6825/2892476953] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T00:29:38.622 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.lgeqrj{1:24269} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.169:6826/861725728,v1:172.21.15.169:6827/861725728] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T00:29:38.622 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:29:38.623 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:29:38.623 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 21 2024-09-16T00:29:38.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.621+0000 7f040797b640 1 -- 172.21.15.112:0/472240843 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f03ec06e660 msgr2=0x7f03ec070b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:38.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.621+0000 7f040797b640 1 --2- 172.21.15.112:0/472240843 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f03ec06e660 0x7f03ec070b10 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f040010ea90 tx=0x7f03f000b040 comp rx=0 tx=0).stop 2024-09-16T00:29:38.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.621+0000 7f040797b640 1 -- 172.21.15.112:0/472240843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0400064fe0 msgr2=0x7f04001131d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:38.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.621+0000 7f040797b640 1 --2- 172.21.15.112:0/472240843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0400064fe0 0x7f04001131d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f03fc000c00 tx=0x7f03fc009da0 comp rx=0 tx=0).stop 2024-09-16T00:29:38.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.621+0000 7f040797b640 1 -- 172.21.15.112:0/472240843 shutdown_connections 2024-09-16T00:29:38.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.621+0000 7f040797b640 1 --2- 172.21.15.112:0/472240843 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f03ec06e660 0x7f03ec070b10 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:38.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.621+0000 7f040797b640 1 --2- 172.21.15.112:0/472240843 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04000fe450 0x7f0400113710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:38.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.621+0000 7f040797b640 1 --2- 172.21.15.112:0/472240843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0400064fe0 0x7f04001131d0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:38.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.621+0000 7f040797b640 1 -- 172.21.15.112:0/472240843 >> 172.21.15.112:0/472240843 conn(0x7f04000723b0 msgr2=0x7f04000fa290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:38.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.622+0000 7f040797b640 1 -- 172.21.15.112:0/472240843 shutdown_connections 2024-09-16T00:29:38.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.622+0000 7f040797b640 1 -- 172.21.15.112:0/472240843 wait complete. 2024-09-16T00:29:38.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.709+0000 7f1cd0941640 1 -- 172.21.15.112:0/1214769289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ccc0fb470 msgr2=0x7f1ccc0fb8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:38.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.709+0000 7f1cd0941640 1 --2- 172.21.15.112:0/1214769289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ccc0fb470 0x7f1ccc0fb8e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1cc00048d0 tx=0x7f1cc0009da0 comp rx=0 tx=0).stop 2024-09-16T00:29:38.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.709+0000 7f1cd0941640 1 -- 172.21.15.112:0/1214769289 shutdown_connections 2024-09-16T00:29:38.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.709+0000 7f1cd0941640 1 --2- 172.21.15.112:0/1214769289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ccc0fb470 0x7f1ccc0fb8e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:38.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.709+0000 7f1cd0941640 1 --2- 172.21.15.112:0/1214769289 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ccc1014b0 0x7f1ccc101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:38.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.709+0000 7f1cd0941640 1 -- 172.21.15.112:0/1214769289 >> 172.21.15.112:0/1214769289 conn(0x7f1ccc0f7030 msgr2=0x7f1ccc0f9440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:38.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.709+0000 7f1cd0941640 1 -- 172.21.15.112:0/1214769289 shutdown_connections 2024-09-16T00:29:38.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.709+0000 7f1cd0941640 1 -- 172.21.15.112:0/1214769289 wait complete. 2024-09-16T00:29:38.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.710+0000 7f1cd0941640 1 Processor -- start 2024-09-16T00:29:38.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.710+0000 7f1cd0941640 1 -- start start 2024-09-16T00:29:38.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.710+0000 7f1cd0941640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ccc0fb470 0x7f1ccc110fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:38.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.710+0000 7f1cd0941640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ccc1014b0 0x7f1ccc1114f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:38.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.710+0000 7f1cd0941640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ccc111af0 con 0x7f1ccc1014b0 2024-09-16T00:29:38.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.710+0000 7f1cd0941640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ccc10af30 con 0x7f1ccc0fb470 2024-09-16T00:29:38.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.711+0000 7f1cc9d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ccc1014b0 0x7f1ccc1114f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:38.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.711+0000 7f1cc9d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ccc1014b0 0x7f1ccc1114f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46178/0 (socket says 172.21.15.112:46178) 2024-09-16T00:29:38.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.711+0000 7f1cc9d9b640 1 -- 172.21.15.112:0/3641741199 learned_addr learned my addr 172.21.15.112:0/3641741199 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:29:38.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.711+0000 7f1cca59c640 1 --2- 172.21.15.112:0/3641741199 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ccc0fb470 0x7f1ccc110fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:38.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.711+0000 7f1cc9d9b640 1 -- 172.21.15.112:0/3641741199 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ccc0fb470 msgr2=0x7f1ccc110fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:38.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.711+0000 7f1cc9d9b640 1 --2- 172.21.15.112:0/3641741199 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ccc0fb470 0x7f1ccc110fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:38.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.711+0000 7f1cc9d9b640 1 -- 172.21.15.112:0/3641741199 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cc0005040 con 0x7f1ccc1014b0 2024-09-16T00:29:38.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.711+0000 7f1cc9d9b640 1 --2- 172.21.15.112:0/3641741199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ccc1014b0 0x7f1ccc1114f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1cc00043d0 tx=0x7f1cc0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:38.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.711+0000 7f1cbaffd640 1 -- 172.21.15.112:0/3641741199 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1cc0038070 con 0x7f1ccc1014b0 2024-09-16T00:29:38.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.711+0000 7f1cd0941640 1 -- 172.21.15.112:0/3641741199 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ccc10b1b0 con 0x7f1ccc1014b0 2024-09-16T00:29:38.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.711+0000 7f1cbaffd640 1 -- 172.21.15.112:0/3641741199 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f1cc00096e0 con 0x7f1ccc1014b0 2024-09-16T00:29:38.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.711+0000 7f1cd0941640 1 -- 172.21.15.112:0/3641741199 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ccc10b750 con 0x7f1ccc1014b0 2024-09-16T00:29:38.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.712+0000 7f1cbaffd640 1 -- 172.21.15.112:0/3641741199 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1cc003d420 con 0x7f1ccc1014b0 2024-09-16T00:29:38.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.712+0000 7f1cd0941640 1 -- 172.21.15.112:0/3641741199 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cac005210 con 0x7f1ccc1014b0 2024-09-16T00:29:38.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.713+0000 7f1cbaffd640 1 -- 172.21.15.112:0/3641741199 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 38) v1 ==== 92655+0+0 (secure 0 0 0) 0x7f1cc0030020 con 0x7f1ccc1014b0 2024-09-16T00:29:38.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.714+0000 7f1cbaffd640 1 --2- 172.21.15.112:0/3641741199 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f1cb406e720 0x7f1cb4070bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:38.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.714+0000 7f1cbaffd640 1 -- 172.21.15.112:0/3641741199 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f1cc00ae5a0 con 0x7f1ccc1014b0 2024-09-16T00:29:38.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.714+0000 7f1cca59c640 1 --2- 172.21.15.112:0/3641741199 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f1cb406e720 0x7f1cb4070bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:38.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.714+0000 7f1cca59c640 1 --2- 172.21.15.112:0/3641741199 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f1cb406e720 0x7f1cb4070bd0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc004ed0 tx=0x7f1cbc00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:38.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.718+0000 7f1cbaffd640 1 -- 172.21.15.112:0/3641741199 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f1cc00775b0 con 0x7f1ccc1014b0 2024-09-16T00:29:38.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.920+0000 7f1cd0941640 1 -- 172.21.15.112:0/3641741199 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1cac000bc0 con 0x7f1cb406e720 2024-09-16T00:29:38.920 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:38 smithi112 ceph-mon[97750]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:29:38.920 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:38 smithi112 ceph-mon[97750]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:29:38.920 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:38 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/2558442208' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:29:38.920 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:38 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:38.921 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:38 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:38.921 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:38 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:38.921 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:38 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:29:38.921 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:38 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:38.921 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:38 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T00:29:38.921 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:38 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T00:29:38.921 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:38 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:38.921 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:38 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/472240843' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:29:38.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.921+0000 7f1cbaffd640 1 -- 172.21.15.112:0/3641741199 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f1cac000bc0 con 0x7f1cb406e720 2024-09-16T00:29:38.922 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:29:38.922 INFO:teuthology.orchestra.run.smithi112.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80", 2024-09-16T00:29:38.922 INFO:teuthology.orchestra.run.smithi112.stdout: "in_progress": true, 2024-09-16T00:29:38.922 INFO:teuthology.orchestra.run.smithi112.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T00:29:38.922 INFO:teuthology.orchestra.run.smithi112.stdout: "services_complete": [ 2024-09-16T00:29:38.922 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr", 2024-09-16T00:29:38.922 INFO:teuthology.orchestra.run.smithi112.stdout: "mon" 2024-09-16T00:29:38.922 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-16T00:29:38.922 INFO:teuthology.orchestra.run.smithi112.stdout: "progress": "4/21 daemons upgraded", 2024-09-16T00:29:38.922 INFO:teuthology.orchestra.run.smithi112.stdout: "message": "Currently upgrading mon daemons", 2024-09-16T00:29:38.922 INFO:teuthology.orchestra.run.smithi112.stdout: "is_paused": false 2024-09-16T00:29:38.922 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:29:38.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.923+0000 7f1cd0941640 1 -- 172.21.15.112:0/3641741199 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f1cb406e720 msgr2=0x7f1cb4070bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:38.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.923+0000 7f1cd0941640 1 --2- 172.21.15.112:0/3641741199 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f1cb406e720 0x7f1cb4070bd0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc004ed0 tx=0x7f1cbc00b040 comp rx=0 tx=0).stop 2024-09-16T00:29:38.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.923+0000 7f1cd0941640 1 -- 172.21.15.112:0/3641741199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ccc1014b0 msgr2=0x7f1ccc1114f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:38.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.923+0000 7f1cd0941640 1 --2- 172.21.15.112:0/3641741199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ccc1014b0 0x7f1ccc1114f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1cc00043d0 tx=0x7f1cc0009da0 comp rx=0 tx=0).stop 2024-09-16T00:29:38.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.923+0000 7f1cd0941640 1 -- 172.21.15.112:0/3641741199 shutdown_connections 2024-09-16T00:29:38.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.924+0000 7f1cd0941640 1 --2- 172.21.15.112:0/3641741199 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f1cb406e720 0x7f1cb4070bd0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:38.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.924+0000 7f1cd0941640 1 --2- 172.21.15.112:0/3641741199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ccc1014b0 0x7f1ccc1114f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:38.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.924+0000 7f1cd0941640 1 --2- 172.21.15.112:0/3641741199 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ccc0fb470 0x7f1ccc110fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:38.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.924+0000 7f1cd0941640 1 -- 172.21.15.112:0/3641741199 >> 172.21.15.112:0/3641741199 conn(0x7f1ccc0f7030 msgr2=0x7f1ccc0f7400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:38.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.924+0000 7f1cd0941640 1 -- 172.21.15.112:0/3641741199 shutdown_connections 2024-09-16T00:29:38.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:38.924+0000 7f1cd0941640 1 -- 172.21.15.112:0/3641741199 wait complete. 2024-09-16T00:29:39.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:39.003+0000 7f3ccdc8c640 1 -- 172.21.15.112:0/2634070623 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cc80fd540 msgr2=0x7f3cc80fd910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:39.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:39.003+0000 7f3ccdc8c640 1 --2- 172.21.15.112:0/2634070623 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cc80fd540 0x7f3cc80fd910 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3cb8004900 tx=0x7f3cb8009da0 comp rx=0 tx=0).stop 2024-09-16T00:29:39.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:39.003+0000 7f3ccdc8c640 1 -- 172.21.15.112:0/2634070623 shutdown_connections 2024-09-16T00:29:39.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:39.003+0000 7f3ccdc8c640 1 --2- 172.21.15.112:0/2634070623 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3cc80fdee0 0x7f3cc81065b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:39.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:39.003+0000 7f3ccdc8c640 1 --2- 172.21.15.112:0/2634070623 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cc80fd540 0x7f3cc80fd910 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:39.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:39.003+0000 7f3ccdc8c640 1 -- 172.21.15.112:0/2634070623 >> 172.21.15.112:0/2634070623 conn(0x7f3cc80f7000 msgr2=0x7f3cc80f9410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:39.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:39.003+0000 7f3ccdc8c640 1 -- 172.21.15.112:0/2634070623 shutdown_connections 2024-09-16T00:29:39.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:39.003+0000 7f3ccdc8c640 1 -- 172.21.15.112:0/2634070623 wait complete. 2024-09-16T00:29:39.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:39.003+0000 7f3ccdc8c640 1 Processor -- start 2024-09-16T00:29:39.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:39.004+0000 7f3ccdc8c640 1 -- start start 2024-09-16T00:29:39.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:39.004+0000 7f3ccdc8c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cc80fd540 0x7f3cc81132d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:39.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:39.004+0000 7f3ccdc8c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3cc80fdee0 0x7f3cc8113810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:39.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:39.004+0000 7f3ccdc8c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc8113ea0 con 0x7f3cc80fd540 2024-09-16T00:29:39.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:39.004+0000 7f3ccdc8c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc810d250 con 0x7f3cc80fdee0 2024-09-16T00:29:39.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:39.004+0000 7f3cc77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cc80fd540 0x7f3cc81132d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:39.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:39.004+0000 7f3cc77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cc80fd540 0x7f3cc81132d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46194/0 (socket says 172.21.15.112:46194) 2024-09-16T00:29:39.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:39.004+0000 7f3cc77fe640 1 -- 172.21.15.112:0/3097330847 learned_addr learned my addr 172.21.15.112:0/3097330847 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:29:39.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:39.004+0000 7f3cc6ffd640 1 --2- 172.21.15.112:0/3097330847 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3cc80fdee0 0x7f3cc8113810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:39.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:39.004+0000 7f3cc77fe640 1 -- 172.21.15.112:0/3097330847 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3cc80fdee0 msgr2=0x7f3cc8113810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:39.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:39.004+0000 7f3cc77fe640 1 --2- 172.21.15.112:0/3097330847 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3cc80fdee0 0x7f3cc8113810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:39.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:39.004+0000 7f3cc77fe640 1 -- 172.21.15.112:0/3097330847 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cb8005040 con 0x7f3cc80fd540 2024-09-16T00:29:39.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:39.005+0000 7f3cc77fe640 1 --2- 172.21.15.112:0/3097330847 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cc80fd540 0x7f3cc81132d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3cb802a560 tx=0x7f3cb8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:39.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:39.005+0000 7f3caffff640 1 -- 172.21.15.112:0/3097330847 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3cb8038070 con 0x7f3cc80fd540 2024-09-16T00:29:39.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:39.005+0000 7f3caffff640 1 -- 172.21.15.112:0/3097330847 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f3cb8032cd0 con 0x7f3cc80fd540 2024-09-16T00:29:39.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:39.005+0000 7f3caffff640 1 -- 172.21.15.112:0/3097330847 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3cb80096c0 con 0x7f3cc80fd540 2024-09-16T00:29:39.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:39.005+0000 7f3ccdc8c640 1 -- 172.21.15.112:0/3097330847 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cc810d4d0 con 0x7f3cc80fd540 2024-09-16T00:29:39.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:39.005+0000 7f3ccdc8c640 1 -- 172.21.15.112:0/3097330847 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cc810d940 con 0x7f3cc80fd540 2024-09-16T00:29:39.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:39.006+0000 7f3ccdc8c640 1 -- 172.21.15.112:0/3097330847 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cc80662d0 con 0x7f3cc80fd540 2024-09-16T00:29:39.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:39.006+0000 7f3caffff640 1 -- 172.21.15.112:0/3097330847 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 38) v1 ==== 92655+0+0 (secure 0 0 0) 0x7f3cb8030020 con 0x7f3cc80fd540 2024-09-16T00:29:39.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:39.007+0000 7f3caffff640 1 --2- 172.21.15.112:0/3097330847 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f3cb006e660 0x7f3cb0070b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:39.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:39.007+0000 7f3caffff640 1 -- 172.21.15.112:0/3097330847 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f3cb80ae830 con 0x7f3cc80fd540 2024-09-16T00:29:39.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:39.007+0000 7f3cc6ffd640 1 --2- 172.21.15.112:0/3097330847 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f3cb006e660 0x7f3cb0070b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:39.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:39.007+0000 7f3cc6ffd640 1 --2- 172.21.15.112:0/3097330847 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f3cb006e660 0x7f3cb0070b10 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc006fd0 tx=0x7f3cbc008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:39.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:39.011+0000 7f3caffff640 1 -- 172.21.15.112:0/3097330847 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f3cb8077860 con 0x7f3cc80fd540 2024-09-16T00:29:39.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:38 smithi169 ceph-mon[77626]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:29:39.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:38 smithi169 ceph-mon[77626]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:29:39.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:38 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/2558442208' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:29:39.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:38 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:39.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:38 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:39.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:38 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:39.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:38 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:29:39.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:38 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:39.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:38 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T00:29:39.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:38 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T00:29:39.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:38 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:39.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:38 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/472240843' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:29:39.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:39.249+0000 7f3ccdc8c640 1 -- 172.21.15.112:0/3097330847 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f3cc804b720 con 0x7f3cc80fd540 2024-09-16T00:29:39.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:39.249+0000 7f3caffff640 1 -- 172.21.15.112:0/3097330847 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f3cb809b020 con 0x7f3cc80fd540 2024-09-16T00:29:39.250 INFO:teuthology.orchestra.run.smithi112.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-09-16T00:29:39.250 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T00:29:39.250 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T00:29:39.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:39.252+0000 7f3ccdc8c640 1 -- 172.21.15.112:0/3097330847 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f3cb006e660 msgr2=0x7f3cb0070b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:39.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:39.252+0000 7f3ccdc8c640 1 --2- 172.21.15.112:0/3097330847 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f3cb006e660 0x7f3cb0070b10 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc006fd0 tx=0x7f3cbc008040 comp rx=0 tx=0).stop 2024-09-16T00:29:39.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:39.252+0000 7f3ccdc8c640 1 -- 172.21.15.112:0/3097330847 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cc80fd540 msgr2=0x7f3cc81132d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:39.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:39.252+0000 7f3ccdc8c640 1 --2- 172.21.15.112:0/3097330847 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cc80fd540 0x7f3cc81132d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3cb802a560 tx=0x7f3cb8009da0 comp rx=0 tx=0).stop 2024-09-16T00:29:39.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:39.252+0000 7f3ccdc8c640 1 -- 172.21.15.112:0/3097330847 shutdown_connections 2024-09-16T00:29:39.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:39.252+0000 7f3ccdc8c640 1 --2- 172.21.15.112:0/3097330847 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f3cb006e660 0x7f3cb0070b10 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:39.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:39.252+0000 7f3ccdc8c640 1 --2- 172.21.15.112:0/3097330847 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3cc80fdee0 0x7f3cc8113810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:39.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:39.252+0000 7f3ccdc8c640 1 --2- 172.21.15.112:0/3097330847 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cc80fd540 0x7f3cc81132d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:39.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:39.252+0000 7f3ccdc8c640 1 -- 172.21.15.112:0/3097330847 >> 172.21.15.112:0/3097330847 conn(0x7f3cc80f7000 msgr2=0x7f3cc80f8580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:39.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:39.252+0000 7f3ccdc8c640 1 -- 172.21.15.112:0/3097330847 shutdown_connections 2024-09-16T00:29:39.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:29:39.252+0000 7f3ccdc8c640 1 -- 172.21.15.112:0/3097330847 wait complete. 2024-09-16T00:29:40.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:39 smithi169 ceph-mon[77626]: from='client.34140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:29:40.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:39 smithi169 ceph-mon[77626]: Detected new or changed devices on smithi169 2024-09-16T00:29:40.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:39 smithi169 ceph-mon[77626]: Reconfiguring mon.smithi112 (monmap changed)... 2024-09-16T00:29:40.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:39 smithi169 ceph-mon[77626]: Reconfiguring daemon mon.smithi112 on smithi112 2024-09-16T00:29:40.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:39 smithi169 ceph-mon[77626]: pgmap v14: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:29:40.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:39 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/3097330847' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:29:40.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:39 smithi112 ceph-mon[97750]: from='client.34140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:29:40.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:39 smithi112 ceph-mon[97750]: Detected new or changed devices on smithi169 2024-09-16T00:29:40.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:39 smithi112 ceph-mon[97750]: Reconfiguring mon.smithi112 (monmap changed)... 2024-09-16T00:29:40.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:39 smithi112 ceph-mon[97750]: Reconfiguring daemon mon.smithi112 on smithi112 2024-09-16T00:29:40.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:39 smithi112 ceph-mon[97750]: pgmap v14: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:29:40.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:39 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/3097330847' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:29:40.830 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:40 smithi112 ceph-mon[97750]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:29:40.830 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:40 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:40.830 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:40 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:40.830 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:40 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi112.xrkkbc", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T00:29:40.830 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:40 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T00:29:40.830 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:40 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:41.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:40 smithi169 ceph-mon[77626]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:29:41.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:40 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:41.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:40 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:41.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:40 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi112.xrkkbc", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T00:29:41.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:40 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T00:29:41.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:40 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:41.819 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:41 smithi112 ceph-mon[97750]: Reconfiguring mgr.smithi112.xrkkbc (monmap changed)... 2024-09-16T00:29:41.819 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:41 smithi112 ceph-mon[97750]: Reconfiguring daemon mgr.smithi112.xrkkbc on smithi112 2024-09-16T00:29:41.819 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:41 smithi112 ceph-mon[97750]: pgmap v15: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:29:41.819 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:41 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:41.819 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:41 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:41.819 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:41 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi112", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-16T00:29:41.819 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:41 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:42.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:41 smithi169 ceph-mon[77626]: Reconfiguring mgr.smithi112.xrkkbc (monmap changed)... 2024-09-16T00:29:42.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:41 smithi169 ceph-mon[77626]: Reconfiguring daemon mgr.smithi112.xrkkbc on smithi112 2024-09-16T00:29:42.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:41 smithi169 ceph-mon[77626]: pgmap v15: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:29:42.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:41 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:42.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:41 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:42.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:41 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi112", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-16T00:29:42.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:41 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:43.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:42 smithi169 ceph-mon[77626]: Reconfiguring crash.smithi112 (monmap changed)... 2024-09-16T00:29:43.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:42 smithi169 ceph-mon[77626]: Reconfiguring daemon crash.smithi112 on smithi112 2024-09-16T00:29:43.079 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:42 smithi112 ceph-mon[97750]: Reconfiguring crash.smithi112 (monmap changed)... 2024-09-16T00:29:43.079 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:42 smithi112 ceph-mon[97750]: Reconfiguring daemon crash.smithi112 on smithi112 2024-09-16T00:29:44.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:43 smithi169 ceph-mon[77626]: pgmap v16: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:29:44.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:43 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:44.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:43 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:44.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:43 smithi169 ceph-mon[77626]: Reconfiguring osd.0 (monmap changed)... 2024-09-16T00:29:44.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:43 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-16T00:29:44.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:43 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:44.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:43 smithi169 ceph-mon[77626]: Reconfiguring daemon osd.0 on smithi112 2024-09-16T00:29:44.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:43 smithi112 ceph-mon[97750]: pgmap v16: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:29:44.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:43 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:44.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:43 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:44.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:43 smithi112 ceph-mon[97750]: Reconfiguring osd.0 (monmap changed)... 2024-09-16T00:29:44.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:43 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-16T00:29:44.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:43 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:44.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:43 smithi112 ceph-mon[97750]: Reconfiguring daemon osd.0 on smithi112 2024-09-16T00:29:45.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:45 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:45.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:45 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:45.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:45 smithi112 ceph-mon[97750]: Reconfiguring osd.1 (monmap changed)... 2024-09-16T00:29:45.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:45 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-16T00:29:45.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:45 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:45.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:45 smithi112 ceph-mon[97750]: Reconfiguring daemon osd.1 on smithi112 2024-09-16T00:29:45.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:45 smithi112 ceph-mon[97750]: pgmap v17: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:29:45.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:45 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:45.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:45 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:45.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:45 smithi169 ceph-mon[77626]: Reconfiguring osd.1 (monmap changed)... 2024-09-16T00:29:45.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:45 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-16T00:29:45.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:45 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:45.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:45 smithi169 ceph-mon[77626]: Reconfiguring daemon osd.1 on smithi112 2024-09-16T00:29:45.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:45 smithi169 ceph-mon[77626]: pgmap v17: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:29:47.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:46 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:47.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:46 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:47.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:46 smithi169 ceph-mon[77626]: Reconfiguring osd.2 (monmap changed)... 2024-09-16T00:29:47.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:46 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-16T00:29:47.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:46 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:47.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:46 smithi169 ceph-mon[77626]: Reconfiguring daemon osd.2 on smithi112 2024-09-16T00:29:47.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:46 smithi169 ceph-mon[77626]: pgmap v18: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:29:47.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:46 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:29:47.277 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:46 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:47.277 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:46 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:47.277 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:46 smithi112 ceph-mon[97750]: Reconfiguring osd.2 (monmap changed)... 2024-09-16T00:29:47.277 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:46 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-16T00:29:47.278 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:46 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:47.278 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:46 smithi112 ceph-mon[97750]: Reconfiguring daemon osd.2 on smithi112 2024-09-16T00:29:47.278 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:46 smithi112 ceph-mon[97750]: pgmap v18: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:29:47.278 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:46 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:29:48.660 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:48 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:48.660 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:48 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:48.660 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:48 smithi112 ceph-mon[97750]: Reconfiguring mds.cephfs.smithi112.ithwzj (monmap changed)... 2024-09-16T00:29:48.660 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:48 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi112.ithwzj", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:29:48.660 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:48 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:48.660 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:48 smithi112 ceph-mon[97750]: Reconfiguring daemon mds.cephfs.smithi112.ithwzj on smithi112 2024-09-16T00:29:48.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:48 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:48.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:48 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:48.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:48 smithi169 ceph-mon[77626]: Reconfiguring mds.cephfs.smithi112.ithwzj (monmap changed)... 2024-09-16T00:29:48.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:48 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi112.ithwzj", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:29:48.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:48 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:48.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:48 smithi169 ceph-mon[77626]: Reconfiguring daemon mds.cephfs.smithi112.ithwzj on smithi112 2024-09-16T00:29:49.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:49 smithi112 ceph-mon[97750]: pgmap v19: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:29:49.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:49 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:49.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:49 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:49.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:49 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi112.hxfwkh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:29:49.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:49 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:49.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:49 smithi169 ceph-mon[77626]: pgmap v19: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:29:49.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:49 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:49.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:49 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:49.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:49 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi112.hxfwkh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:29:49.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:49 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:50.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:50 smithi112 ceph-mon[97750]: Reconfiguring mds.cephfs.smithi112.hxfwkh (monmap changed)... 2024-09-16T00:29:50.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:50 smithi112 ceph-mon[97750]: Reconfiguring daemon mds.cephfs.smithi112.hxfwkh on smithi112 2024-09-16T00:29:50.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:50 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:50.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:50 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:50.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:50 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi169", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-16T00:29:50.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:50 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:50.723 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:50 smithi169 ceph-mon[77626]: Reconfiguring mds.cephfs.smithi112.hxfwkh (monmap changed)... 2024-09-16T00:29:50.723 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:50 smithi169 ceph-mon[77626]: Reconfiguring daemon mds.cephfs.smithi112.hxfwkh on smithi112 2024-09-16T00:29:50.723 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:50 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:50.723 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:50 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:50.723 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:50 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi169", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-16T00:29:50.723 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:50 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:51.662 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:51 smithi169 ceph-mon[77626]: Reconfiguring crash.smithi169 (monmap changed)... 2024-09-16T00:29:51.663 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:51 smithi169 ceph-mon[77626]: Reconfiguring daemon crash.smithi169 on smithi169 2024-09-16T00:29:51.663 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:51 smithi169 ceph-mon[77626]: pgmap v20: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:29:51.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:51 smithi112 ceph-mon[97750]: Reconfiguring crash.smithi169 (monmap changed)... 2024-09-16T00:29:51.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:51 smithi112 ceph-mon[97750]: Reconfiguring daemon crash.smithi169 on smithi169 2024-09-16T00:29:51.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:51 smithi112 ceph-mon[97750]: pgmap v20: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:29:53.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:52 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:53.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:52 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:53.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:52 smithi169 ceph-mon[77626]: Reconfiguring mgr.smithi169.mtrobk (monmap changed)... 2024-09-16T00:29:53.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:52 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi169.mtrobk", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T00:29:53.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:52 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T00:29:53.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:52 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:53.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:52 smithi169 ceph-mon[77626]: Reconfiguring daemon mgr.smithi169.mtrobk on smithi169 2024-09-16T00:29:53.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:52 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:53.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:52 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:53.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:52 smithi112 ceph-mon[97750]: Reconfiguring mgr.smithi169.mtrobk (monmap changed)... 2024-09-16T00:29:53.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:52 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi169.mtrobk", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T00:29:53.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:52 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T00:29:53.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:52 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:53.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:52 smithi112 ceph-mon[97750]: Reconfiguring daemon mgr.smithi169.mtrobk on smithi169 2024-09-16T00:29:53.757 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:53 smithi169 ceph-mon[77626]: pgmap v21: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:29:53.757 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:53 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:53.757 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:53 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:53.757 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:53 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T00:29:53.757 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:53 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T00:29:53.757 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:53 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:53.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:53 smithi112 ceph-mon[97750]: pgmap v21: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:29:53.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:53 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:53.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:53 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:53.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:53 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T00:29:53.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:53 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T00:29:53.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:53 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:55.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:54 smithi169 ceph-mon[77626]: Reconfiguring mon.smithi169 (monmap changed)... 2024-09-16T00:29:55.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:54 smithi169 ceph-mon[77626]: Reconfiguring daemon mon.smithi169 on smithi169 2024-09-16T00:29:55.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:54 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:55.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:54 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:55.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:54 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-16T00:29:55.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:54 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:55.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:54 smithi112 ceph-mon[97750]: Reconfiguring mon.smithi169 (monmap changed)... 2024-09-16T00:29:55.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:54 smithi112 ceph-mon[97750]: Reconfiguring daemon mon.smithi169 on smithi169 2024-09-16T00:29:55.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:54 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:55.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:54 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:55.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:54 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-16T00:29:55.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:54 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:56.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:55 smithi169 ceph-mon[77626]: Reconfiguring osd.3 (monmap changed)... 2024-09-16T00:29:56.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:55 smithi169 ceph-mon[77626]: Reconfiguring daemon osd.3 on smithi169 2024-09-16T00:29:56.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:55 smithi169 ceph-mon[77626]: pgmap v22: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:29:56.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:55 smithi169 ceph-mon[77626]: fsmap cephfs:2 {0=cephfs.smithi112.hxfwkh=up:active,1=cephfs.smithi169.gawwgw=up:active} 2 up:standby-replay 2024-09-16T00:29:56.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:55 smithi112 ceph-mon[97750]: Reconfiguring osd.3 (monmap changed)... 2024-09-16T00:29:56.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:55 smithi112 ceph-mon[97750]: Reconfiguring daemon osd.3 on smithi169 2024-09-16T00:29:56.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:55 smithi112 ceph-mon[97750]: pgmap v22: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:29:56.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:55 smithi112 ceph-mon[97750]: fsmap cephfs:2 {0=cephfs.smithi112.hxfwkh=up:active,1=cephfs.smithi169.gawwgw=up:active} 2 up:standby-replay 2024-09-16T00:29:56.928 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:56 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:56.928 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:56 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:56.928 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:56 smithi169 ceph-mon[77626]: Reconfiguring osd.4 (monmap changed)... 2024-09-16T00:29:57.218 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:56 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-16T00:29:57.218 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:56 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:57.218 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:56 smithi169 ceph-mon[77626]: Reconfiguring daemon osd.4 on smithi169 2024-09-16T00:29:57.218 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:56 smithi169 ceph-mon[77626]: pgmap v23: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:29:57.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:56 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:57.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:56 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:57.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:56 smithi112 ceph-mon[97750]: Reconfiguring osd.4 (monmap changed)... 2024-09-16T00:29:57.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:56 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-16T00:29:57.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:56 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:57.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:56 smithi112 ceph-mon[97750]: Reconfiguring daemon osd.4 on smithi169 2024-09-16T00:29:57.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:56 smithi112 ceph-mon[97750]: pgmap v23: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:29:58.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:58 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:58.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:58 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:58.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:58 smithi112 ceph-mon[97750]: Reconfiguring osd.5 (monmap changed)... 2024-09-16T00:29:58.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:58 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-16T00:29:58.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:58 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:58.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:58 smithi112 ceph-mon[97750]: Reconfiguring daemon osd.5 on smithi169 2024-09-16T00:29:58.675 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:58 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:58.675 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:58 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:58.675 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:58 smithi169 ceph-mon[77626]: Reconfiguring osd.5 (monmap changed)... 2024-09-16T00:29:58.675 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:58 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-16T00:29:58.675 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:58 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:58.675 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:58 smithi169 ceph-mon[77626]: Reconfiguring daemon osd.5 on smithi169 2024-09-16T00:29:59.471 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:59 smithi169 ceph-mon[77626]: pgmap v24: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:29:59.471 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:59 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:59.471 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:59 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:59.471 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:59 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi169.gawwgw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:29:59.471 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:29:59 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:59.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:59 smithi112 ceph-mon[97750]: pgmap v24: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:29:59.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:59 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:59.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:59 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:29:59.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:59 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi169.gawwgw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:29:59.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:29:59 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:30:00.492 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:00 smithi169 ceph-mon[77626]: Reconfiguring mds.cephfs.smithi169.gawwgw (monmap changed)... 2024-09-16T00:30:00.492 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:00 smithi169 ceph-mon[77626]: Reconfiguring daemon mds.cephfs.smithi169.gawwgw on smithi169 2024-09-16T00:30:00.492 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:00 smithi169 ceph-mon[77626]: overall HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-09-16T00:30:00.493 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:00 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:00.493 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:00 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:00.493 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:00 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi169.lgeqrj", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:30:00.493 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:00 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:30:00.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:00 smithi112 ceph-mon[97750]: Reconfiguring mds.cephfs.smithi169.gawwgw (monmap changed)... 2024-09-16T00:30:00.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:00 smithi112 ceph-mon[97750]: Reconfiguring daemon mds.cephfs.smithi169.gawwgw on smithi169 2024-09-16T00:30:00.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:00 smithi112 ceph-mon[97750]: overall HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-09-16T00:30:00.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:00 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:00.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:00 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:00.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:00 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi169.lgeqrj", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:30:00.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:00 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:30:01.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:01 smithi112 ceph-mon[97750]: Reconfiguring mds.cephfs.smithi169.lgeqrj (monmap changed)... 2024-09-16T00:30:01.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:01 smithi112 ceph-mon[97750]: Reconfiguring daemon mds.cephfs.smithi169.lgeqrj on smithi169 2024-09-16T00:30:01.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:01 smithi112 ceph-mon[97750]: pgmap v25: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:30:01.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:01 smithi169 ceph-mon[77626]: Reconfiguring mds.cephfs.smithi169.lgeqrj (monmap changed)... 2024-09-16T00:30:01.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:01 smithi169 ceph-mon[77626]: Reconfiguring daemon mds.cephfs.smithi169.lgeqrj on smithi169 2024-09-16T00:30:01.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:01 smithi169 ceph-mon[77626]: pgmap v25: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:30:02.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:02 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:30:02.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:02 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:02.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:02 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:02.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:02 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:30:02.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:02 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:30:02.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:02 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:30:02.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:02 smithi112 ceph-mon[97750]: Upgrade: Setting container_image for all mon 2024-09-16T00:30:02.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:02 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:02.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:02 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi112"}]: dispatch 2024-09-16T00:30:02.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:02 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi112"}]': finished 2024-09-16T00:30:02.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:02 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi169"}]: dispatch 2024-09-16T00:30:02.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:02 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi169"}]': finished 2024-09-16T00:30:02.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:02 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:30:02.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:02 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:02.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:02 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:02.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:02 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:30:02.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:02 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:30:02.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:02 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:30:02.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:02 smithi169 ceph-mon[77626]: Upgrade: Setting container_image for all mon 2024-09-16T00:30:02.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:02 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:02.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:02 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi112"}]: dispatch 2024-09-16T00:30:02.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:02 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi112"}]': finished 2024-09-16T00:30:02.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:02 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi169"}]: dispatch 2024-09-16T00:30:02.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:02 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi169"}]': finished 2024-09-16T00:30:03.591 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:03 smithi112 ceph-mon[97750]: pgmap v26: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:30:03.591 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:03 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:03.591 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:03 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi112", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-16T00:30:03.591 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:03 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:30:03.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:03 smithi169 ceph-mon[77626]: pgmap v26: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:30:03.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:03 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:03.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:03 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi112", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-16T00:30:03.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:03 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:30:04.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:04 smithi112 ceph-mon[97750]: Upgrade: Updating crash.smithi112 (1/2) 2024-09-16T00:30:04.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:04 smithi112 ceph-mon[97750]: Deploying daemon crash.smithi112 on smithi112 2024-09-16T00:30:04.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:04 smithi169 ceph-mon[77626]: Upgrade: Updating crash.smithi112 (1/2) 2024-09-16T00:30:04.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:04 smithi169 ceph-mon[77626]: Deploying daemon crash.smithi112 on smithi112 2024-09-16T00:30:05.546 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:05 smithi112 ceph-mon[97750]: pgmap v27: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:30:05.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:05 smithi169 ceph-mon[77626]: pgmap v27: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:30:07.697 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:07 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:07.697 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:07 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:07.697 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:07 smithi112 ceph-mon[97750]: pgmap v28: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:30:07.756 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:07 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:07.756 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:07 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:07.756 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:07 smithi169 ceph-mon[77626]: pgmap v28: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:30:09.088 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:08 smithi169 ceph-mon[77626]: Upgrade: Updating crash.smithi169 (2/2) 2024-09-16T00:30:09.089 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:08 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:09.089 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:08 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi169", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-16T00:30:09.089 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:08 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:30:09.089 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:08 smithi169 ceph-mon[77626]: Deploying daemon crash.smithi169 on smithi169 2024-09-16T00:30:09.089 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:08 smithi169 ceph-mon[77626]: pgmap v29: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:30:09.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:08 smithi112 ceph-mon[97750]: Upgrade: Updating crash.smithi169 (2/2) 2024-09-16T00:30:09.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:08 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:09.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:08 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi169", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-16T00:30:09.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:08 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:30:09.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:08 smithi112 ceph-mon[97750]: Deploying daemon crash.smithi169 on smithi169 2024-09-16T00:30:09.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:08 smithi112 ceph-mon[97750]: pgmap v29: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:30:09.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.335+0000 7f70fda1f640 1 -- 172.21.15.112:0/2431298020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70f8064fc0 msgr2=0x7f70f8065430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:09.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.335+0000 7f70fda1f640 1 --2- 172.21.15.112:0/2431298020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70f8064fc0 0x7f70f8065430 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f70ec0048d0 tx=0x7f70ec009da0 comp rx=0 tx=0).stop 2024-09-16T00:30:09.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.335+0000 7f70fda1f640 1 -- 172.21.15.112:0/2431298020 shutdown_connections 2024-09-16T00:30:09.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.335+0000 7f70fda1f640 1 --2- 172.21.15.112:0/2431298020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70f8064fc0 0x7f70f8065430 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:09.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.335+0000 7f70fda1f640 1 --2- 172.21.15.112:0/2431298020 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70f80fe460 0x7f70f80fe830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:09.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.335+0000 7f70fda1f640 1 -- 172.21.15.112:0/2431298020 >> 172.21.15.112:0/2431298020 conn(0x7f70f80723b0 msgr2=0x7f70f80727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:09.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.335+0000 7f70fda1f640 1 -- 172.21.15.112:0/2431298020 shutdown_connections 2024-09-16T00:30:09.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.335+0000 7f70fda1f640 1 -- 172.21.15.112:0/2431298020 wait complete. 2024-09-16T00:30:09.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.336+0000 7f70fda1f640 1 Processor -- start 2024-09-16T00:30:09.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.336+0000 7f70fda1f640 1 -- start start 2024-09-16T00:30:09.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.336+0000 7f70fda1f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70f8064fc0 0x7f70f81131d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:09.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.336+0000 7f70fda1f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70f80fe460 0x7f70f8113710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:09.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.336+0000 7f70fda1f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f8113d10 con 0x7f70f80fe460 2024-09-16T00:30:09.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.336+0000 7f70fda1f640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f810d1a0 con 0x7f70f8064fc0 2024-09-16T00:30:09.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.336+0000 7f70f67fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70f80fe460 0x7f70f8113710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:09.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.336+0000 7f70f6ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70f8064fc0 0x7f70f81131d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:09.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.336+0000 7f70f67fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70f80fe460 0x7f70f8113710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59374/0 (socket says 172.21.15.112:59374) 2024-09-16T00:30:09.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.336+0000 7f70f6ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70f8064fc0 0x7f70f81131d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.112:56632/0 (socket says 172.21.15.112:56632) 2024-09-16T00:30:09.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.336+0000 7f70f67fc640 1 -- 172.21.15.112:0/473765840 learned_addr learned my addr 172.21.15.112:0/473765840 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:30:09.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.336+0000 7f70f67fc640 1 -- 172.21.15.112:0/473765840 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70f8064fc0 msgr2=0x7f70f81131d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:09.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.336+0000 7f70f67fc640 1 --2- 172.21.15.112:0/473765840 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70f8064fc0 0x7f70f81131d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:09.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.336+0000 7f70f67fc640 1 -- 172.21.15.112:0/473765840 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70ec005040 con 0x7f70f80fe460 2024-09-16T00:30:09.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.336+0000 7f70f6ffd640 1 --2- 172.21.15.112:0/473765840 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70f8064fc0 0x7f70f81131d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:30:09.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.336+0000 7f70f67fc640 1 --2- 172.21.15.112:0/473765840 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70f80fe460 0x7f70f8113710 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f70ec000c00 tx=0x7f70ec009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:09.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.337+0000 7f70df7fe640 1 -- 172.21.15.112:0/473765840 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70ec038070 con 0x7f70f80fe460 2024-09-16T00:30:09.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.337+0000 7f70fda1f640 1 -- 172.21.15.112:0/473765840 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70f810d420 con 0x7f70f80fe460 2024-09-16T00:30:09.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.337+0000 7f70df7fe640 1 -- 172.21.15.112:0/473765840 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f70ec02ade0 con 0x7f70f80fe460 2024-09-16T00:30:09.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.337+0000 7f70df7fe640 1 -- 172.21.15.112:0/473765840 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70ec02c7b0 con 0x7f70f80fe460 2024-09-16T00:30:09.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.337+0000 7f70fda1f640 1 -- 172.21.15.112:0/473765840 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70f810d960 con 0x7f70f80fe460 2024-09-16T00:30:09.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.338+0000 7f70fda1f640 1 -- 172.21.15.112:0/473765840 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70f80fdcf0 con 0x7f70f80fe460 2024-09-16T00:30:09.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.339+0000 7f70df7fe640 1 -- 172.21.15.112:0/473765840 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 38) v1 ==== 92655+0+0 (secure 0 0 0) 0x7f70ec030020 con 0x7f70f80fe460 2024-09-16T00:30:09.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.339+0000 7f70df7fe640 1 --2- 172.21.15.112:0/473765840 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f70e006e660 0x7f70e0070b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:09.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.339+0000 7f70df7fe640 1 -- 172.21.15.112:0/473765840 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f70ec0ae760 con 0x7f70f80fe460 2024-09-16T00:30:09.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.339+0000 7f70f6ffd640 1 --2- 172.21.15.112:0/473765840 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f70e006e660 0x7f70e0070b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:09.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.339+0000 7f70f6ffd640 1 --2- 172.21.15.112:0/473765840 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f70e006e660 0x7f70e0070b10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f70e8004fd0 tx=0x7f70e800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:09.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.344+0000 7f70df7fe640 1 -- 172.21.15.112:0/473765840 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f70ec0776f0 con 0x7f70f80fe460 2024-09-16T00:30:09.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.526+0000 7f70fda1f640 1 -- 172.21.15.112:0/473765840 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f70f810e840 con 0x7f70e006e660 2024-09-16T00:30:09.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.527+0000 7f70df7fe640 1 -- 172.21.15.112:0/473765840 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+385 (secure 0 0 0) 0x7f70f810e840 con 0x7f70e006e660 2024-09-16T00:30:09.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.529+0000 7f70fda1f640 1 -- 172.21.15.112:0/473765840 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f70e006e660 msgr2=0x7f70e0070b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:09.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.529+0000 7f70fda1f640 1 --2- 172.21.15.112:0/473765840 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f70e006e660 0x7f70e0070b10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f70e8004fd0 tx=0x7f70e800b040 comp rx=0 tx=0).stop 2024-09-16T00:30:09.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.529+0000 7f70fda1f640 1 -- 172.21.15.112:0/473765840 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70f80fe460 msgr2=0x7f70f8113710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:09.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.529+0000 7f70fda1f640 1 --2- 172.21.15.112:0/473765840 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70f80fe460 0x7f70f8113710 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f70ec000c00 tx=0x7f70ec009da0 comp rx=0 tx=0).stop 2024-09-16T00:30:09.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.529+0000 7f70fda1f640 1 -- 172.21.15.112:0/473765840 shutdown_connections 2024-09-16T00:30:09.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.529+0000 7f70fda1f640 1 --2- 172.21.15.112:0/473765840 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f70e006e660 0x7f70e0070b10 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:09.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.529+0000 7f70fda1f640 1 --2- 172.21.15.112:0/473765840 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70f80fe460 0x7f70f8113710 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:09.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.529+0000 7f70fda1f640 1 --2- 172.21.15.112:0/473765840 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70f8064fc0 0x7f70f81131d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:09.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.529+0000 7f70fda1f640 1 -- 172.21.15.112:0/473765840 >> 172.21.15.112:0/473765840 conn(0x7f70f80723b0 msgr2=0x7f70f81053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:09.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.529+0000 7f70fda1f640 1 -- 172.21.15.112:0/473765840 shutdown_connections 2024-09-16T00:30:09.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.529+0000 7f70fda1f640 1 -- 172.21.15.112:0/473765840 wait complete. 2024-09-16T00:30:09.538 INFO:teuthology.orchestra.run.smithi112.stdout:true 2024-09-16T00:30:09.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.610+0000 7fd17960a640 1 -- 172.21.15.112:0/2098978943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1740fe460 msgr2=0x7fd1740fe830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:09.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.610+0000 7fd17960a640 1 --2- 172.21.15.112:0/2098978943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1740fe460 0x7fd1740fe830 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd164009c50 tx=0x7fd164004d10 comp rx=0 tx=0).stop 2024-09-16T00:30:09.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.611+0000 7fd17960a640 1 -- 172.21.15.112:0/2098978943 shutdown_connections 2024-09-16T00:30:09.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.611+0000 7fd17960a640 1 --2- 172.21.15.112:0/2098978943 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd174064fc0 0x7fd174065430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:09.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.611+0000 7fd17960a640 1 --2- 172.21.15.112:0/2098978943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1740fe460 0x7fd1740fe830 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:09.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.611+0000 7fd17960a640 1 -- 172.21.15.112:0/2098978943 >> 172.21.15.112:0/2098978943 conn(0x7fd1740723b0 msgr2=0x7fd1740727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:09.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.611+0000 7fd17960a640 1 -- 172.21.15.112:0/2098978943 shutdown_connections 2024-09-16T00:30:09.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.611+0000 7fd17960a640 1 -- 172.21.15.112:0/2098978943 wait complete. 2024-09-16T00:30:09.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.611+0000 7fd17960a640 1 Processor -- start 2024-09-16T00:30:09.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.611+0000 7fd17960a640 1 -- start start 2024-09-16T00:30:09.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.612+0000 7fd17960a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd174064fc0 0x7fd1741131b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:09.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.612+0000 7fd17960a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd1740fe460 0x7fd1741136f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:09.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.612+0000 7fd17960a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd174113cf0 con 0x7fd174064fc0 2024-09-16T00:30:09.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.612+0000 7fd17960a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd17410d130 con 0x7fd1740fe460 2024-09-16T00:30:09.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.612+0000 7fd172ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd174064fc0 0x7fd1741131b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:09.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.612+0000 7fd172ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd174064fc0 0x7fd1741131b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59402/0 (socket says 172.21.15.112:59402) 2024-09-16T00:30:09.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.612+0000 7fd172ffd640 1 -- 172.21.15.112:0/1681244622 learned_addr learned my addr 172.21.15.112:0/1681244622 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:30:09.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.612+0000 7fd1727fc640 1 --2- 172.21.15.112:0/1681244622 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd1740fe460 0x7fd1741136f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:09.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.612+0000 7fd172ffd640 1 -- 172.21.15.112:0/1681244622 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd1740fe460 msgr2=0x7fd1741136f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:09.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.612+0000 7fd172ffd640 1 --2- 172.21.15.112:0/1681244622 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd1740fe460 0x7fd1741136f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:09.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.612+0000 7fd172ffd640 1 -- 172.21.15.112:0/1681244622 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1640098b0 con 0x7fd174064fc0 2024-09-16T00:30:09.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.612+0000 7fd172ffd640 1 --2- 172.21.15.112:0/1681244622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd174064fc0 0x7fd1741131b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd164004750 tx=0x7fd164004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:09.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.612+0000 7fd15b7fe640 1 -- 172.21.15.112:0/1681244622 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd164009610 con 0x7fd174064fc0 2024-09-16T00:30:09.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.613+0000 7fd17960a640 1 -- 172.21.15.112:0/1681244622 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd17410d3b0 con 0x7fd174064fc0 2024-09-16T00:30:09.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.613+0000 7fd15b7fe640 1 -- 172.21.15.112:0/1681244622 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fd164032b10 con 0x7fd174064fc0 2024-09-16T00:30:09.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.613+0000 7fd15b7fe640 1 -- 172.21.15.112:0/1681244622 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd16403d990 con 0x7fd174064fc0 2024-09-16T00:30:09.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.613+0000 7fd17960a640 1 -- 172.21.15.112:0/1681244622 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd17410d950 con 0x7fd174064fc0 2024-09-16T00:30:09.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.614+0000 7fd17960a640 1 -- 172.21.15.112:0/1681244622 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1740fdcf0 con 0x7fd174064fc0 2024-09-16T00:30:09.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.614+0000 7fd15b7fe640 1 -- 172.21.15.112:0/1681244622 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 38) v1 ==== 92655+0+0 (secure 0 0 0) 0x7fd16402f030 con 0x7fd174064fc0 2024-09-16T00:30:09.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.614+0000 7fd15b7fe640 1 --2- 172.21.15.112:0/1681244622 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd15c06e660 0x7fd15c070b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:09.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.615+0000 7fd15b7fe640 1 -- 172.21.15.112:0/1681244622 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fd1640af470 con 0x7fd174064fc0 2024-09-16T00:30:09.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.615+0000 7fd1727fc640 1 --2- 172.21.15.112:0/1681244622 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd15c06e660 0x7fd15c070b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:09.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.615+0000 7fd1727fc640 1 --2- 172.21.15.112:0/1681244622 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd15c06e660 0x7fd15c070b10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd17410ea70 tx=0x7fd16800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:09.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.619+0000 7fd15b7fe640 1 -- 172.21.15.112:0/1681244622 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fd164078390 con 0x7fd174064fc0 2024-09-16T00:30:09.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.804+0000 7fd17960a640 1 -- 172.21.15.112:0/1681244622 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd17404b720 con 0x7fd15c06e660 2024-09-16T00:30:09.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.805+0000 7fd15b7fe640 1 -- 172.21.15.112:0/1681244622 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+385 (secure 0 0 0) 0x7fd17404b720 con 0x7fd15c06e660 2024-09-16T00:30:09.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.807+0000 7fd17960a640 1 -- 172.21.15.112:0/1681244622 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd15c06e660 msgr2=0x7fd15c070b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:09.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.807+0000 7fd17960a640 1 --2- 172.21.15.112:0/1681244622 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd15c06e660 0x7fd15c070b10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd17410ea70 tx=0x7fd16800b040 comp rx=0 tx=0).stop 2024-09-16T00:30:09.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.807+0000 7fd17960a640 1 -- 172.21.15.112:0/1681244622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd174064fc0 msgr2=0x7fd1741131b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:09.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.807+0000 7fd17960a640 1 --2- 172.21.15.112:0/1681244622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd174064fc0 0x7fd1741131b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd164004750 tx=0x7fd164004d10 comp rx=0 tx=0).stop 2024-09-16T00:30:09.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.807+0000 7fd17960a640 1 -- 172.21.15.112:0/1681244622 shutdown_connections 2024-09-16T00:30:09.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.807+0000 7fd17960a640 1 --2- 172.21.15.112:0/1681244622 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd15c06e660 0x7fd15c070b10 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:09.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.807+0000 7fd17960a640 1 --2- 172.21.15.112:0/1681244622 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd1740fe460 0x7fd1741136f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:09.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.807+0000 7fd17960a640 1 --2- 172.21.15.112:0/1681244622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd174064fc0 0x7fd1741131b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:09.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.807+0000 7fd17960a640 1 -- 172.21.15.112:0/1681244622 >> 172.21.15.112:0/1681244622 conn(0x7fd1740723b0 msgr2=0x7fd174105410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:09.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.807+0000 7fd17960a640 1 -- 172.21.15.112:0/1681244622 shutdown_connections 2024-09-16T00:30:09.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.807+0000 7fd17960a640 1 -- 172.21.15.112:0/1681244622 wait complete. 2024-09-16T00:30:09.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.885+0000 7fa2bd2b7640 1 -- 172.21.15.112:0/911533184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2b8102530 msgr2=0x7fa2b8102900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:09.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.885+0000 7fa2bd2b7640 1 --2- 172.21.15.112:0/911533184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2b8102530 0x7fa2b8102900 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa2a8009c50 tx=0x7fa2a8004d10 comp rx=0 tx=0).stop 2024-09-16T00:30:09.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.885+0000 7fa2bd2b7640 1 -- 172.21.15.112:0/911533184 shutdown_connections 2024-09-16T00:30:09.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.885+0000 7fa2bd2b7640 1 --2- 172.21.15.112:0/911533184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2b8102ed0 0x7fa2b80fc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:09.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.885+0000 7fa2bd2b7640 1 --2- 172.21.15.112:0/911533184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2b8102530 0x7fa2b8102900 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:09.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.885+0000 7fa2bd2b7640 1 -- 172.21.15.112:0/911533184 >> 172.21.15.112:0/911533184 conn(0x7fa2b80f80f0 msgr2=0x7fa2b80fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:09.886 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:09 smithi112 ceph-mon[97750]: fsmap cephfs:2 {0=cephfs.smithi112.hxfwkh=up:active,1=cephfs.smithi169.gawwgw=up:active} 2 up:standby-replay 2024-09-16T00:30:09.887 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:09 smithi112 ceph-mon[97750]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:30:09.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.885+0000 7fa2bd2b7640 1 -- 172.21.15.112:0/911533184 shutdown_connections 2024-09-16T00:30:09.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.886+0000 7fa2bd2b7640 1 -- 172.21.15.112:0/911533184 wait complete. 2024-09-16T00:30:09.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.886+0000 7fa2bd2b7640 1 Processor -- start 2024-09-16T00:30:09.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.886+0000 7fa2bd2b7640 1 -- start start 2024-09-16T00:30:09.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.887+0000 7fa2bd2b7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2b8102530 0x7fa2b8113240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:09.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.887+0000 7fa2bd2b7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2b8102ed0 0x7fa2b8113780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:09.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.887+0000 7fa2bd2b7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b8113d80 con 0x7fa2b8102530 2024-09-16T00:30:09.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.887+0000 7fa2bd2b7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b810d1c0 con 0x7fa2b8102ed0 2024-09-16T00:30:09.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.887+0000 7fa2b6ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2b8102530 0x7fa2b8113240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:09.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.887+0000 7fa2b6ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2b8102530 0x7fa2b8113240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59414/0 (socket says 172.21.15.112:59414) 2024-09-16T00:30:09.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.887+0000 7fa2b6ffd640 1 -- 172.21.15.112:0/1404781956 learned_addr learned my addr 172.21.15.112:0/1404781956 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:30:09.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.887+0000 7fa2b67fc640 1 --2- 172.21.15.112:0/1404781956 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2b8102ed0 0x7fa2b8113780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:09.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.887+0000 7fa2b6ffd640 1 -- 172.21.15.112:0/1404781956 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2b8102ed0 msgr2=0x7fa2b8113780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:09.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.887+0000 7fa2b6ffd640 1 --2- 172.21.15.112:0/1404781956 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2b8102ed0 0x7fa2b8113780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:09.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.887+0000 7fa2b6ffd640 1 -- 172.21.15.112:0/1404781956 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2a80098b0 con 0x7fa2b8102530 2024-09-16T00:30:09.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.887+0000 7fa2b6ffd640 1 --2- 172.21.15.112:0/1404781956 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2b8102530 0x7fa2b8113240 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa2a8004750 tx=0x7fa2a8004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:09.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.887+0000 7fa29f7fe640 1 -- 172.21.15.112:0/1404781956 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2a8009590 con 0x7fa2b8102530 2024-09-16T00:30:09.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.888+0000 7fa29f7fe640 1 -- 172.21.15.112:0/1404781956 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fa2a80096f0 con 0x7fa2b8102530 2024-09-16T00:30:09.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.888+0000 7fa2bd2b7640 1 -- 172.21.15.112:0/1404781956 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2b810d440 con 0x7fa2b8102530 2024-09-16T00:30:09.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.888+0000 7fa29f7fe640 1 -- 172.21.15.112:0/1404781956 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2a803d960 con 0x7fa2b8102530 2024-09-16T00:30:09.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.888+0000 7fa2bd2b7640 1 -- 172.21.15.112:0/1404781956 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2b810d9e0 con 0x7fa2b8102530 2024-09-16T00:30:09.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.889+0000 7fa2bd2b7640 1 -- 172.21.15.112:0/1404781956 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2b80fdcb0 con 0x7fa2b8102530 2024-09-16T00:30:09.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.889+0000 7fa29f7fe640 1 -- 172.21.15.112:0/1404781956 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 38) v1 ==== 92655+0+0 (secure 0 0 0) 0x7fa2a802f030 con 0x7fa2b8102530 2024-09-16T00:30:09.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.890+0000 7fa29f7fe640 1 --2- 172.21.15.112:0/1404781956 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa2a006e660 0x7fa2a0070b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:09.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.890+0000 7fa29f7fe640 1 -- 172.21.15.112:0/1404781956 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fa2a80af6a0 con 0x7fa2b8102530 2024-09-16T00:30:09.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.890+0000 7fa2b67fc640 1 --2- 172.21.15.112:0/1404781956 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa2a006e660 0x7fa2a0070b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:09.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.890+0000 7fa2b67fc640 1 --2- 172.21.15.112:0/1404781956 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa2a006e660 0x7fa2a0070b10 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa2b810eb00 tx=0x7fa2ac00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:09.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:09.894+0000 7fa29f7fe640 1 -- 172.21.15.112:0/1404781956 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fa2a8078640 con 0x7fa2b8102530 2024-09-16T00:30:09.975 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:09 smithi169 ceph-mon[77626]: fsmap cephfs:2 {0=cephfs.smithi112.hxfwkh=up:active,1=cephfs.smithi169.gawwgw=up:active} 2 up:standby-replay 2024-09-16T00:30:09.975 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:09 smithi169 ceph-mon[77626]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:30:10.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.068+0000 7fa2bd2b7640 1 -- 172.21.15.112:0/1404781956 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa2b804b720 con 0x7fa2a006e660 2024-09-16T00:30:10.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.077+0000 7fa29f7fe640 1 -- 172.21.15.112:0/1404781956 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fa2b804b720 con 0x7fa2a006e660 2024-09-16T00:30:10.078 INFO:teuthology.orchestra.run.smithi112.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T00:30:10.078 INFO:teuthology.orchestra.run.smithi112.stdout:alertmanager.smithi112 smithi112 *:9093,9094 running (3m) 48s ago 12m 20.7M - 0.25.0 c8568f914cd2 d85ea0562022 2024-09-16T00:30:10.078 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi112 smithi112 starting - - - - 2024-09-16T00:30:10.078 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi169 smithi169 running (11m) 35s ago 11m 6899k - 17.2.7-1633-g6d025bc2 33ed8dba3996 d0d3726280a2 2024-09-16T00:30:10.078 INFO:teuthology.orchestra.run.smithi112.stdout:grafana.smithi112 smithi112 *:3000 running (112s) 48s ago 11m 74.2M - 9.4.12 2bacad6d85d8 51f9d2013da2 2024-09-16T00:30:10.078 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.hxfwkh smithi112 running (6m) 48s ago 6m 136M - 17.2.7-1633-g6d025bc2 33ed8dba3996 cf92d9461fc6 2024-09-16T00:30:10.078 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.ithwzj smithi112 running (6m) 48s ago 6m 65.9M - 17.2.7-1633-g6d025bc2 33ed8dba3996 02786c8a83c7 2024-09-16T00:30:10.078 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.gawwgw smithi169 running (6m) 35s ago 6m 114M - 17.2.7-1633-g6d025bc2 33ed8dba3996 dcf3a4dfa879 2024-09-16T00:30:10.078 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.lgeqrj smithi169 running (6m) 35s ago 6m 60.4M - 17.2.7-1633-g6d025bc2 33ed8dba3996 445c14e55831 2024-09-16T00:30:10.078 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi112.xrkkbc smithi112 *:8443,9283,8765 running (2m) 48s ago 13m 472M - 19.1.1-326-g8921c8ef f1455bd0219c a981f7fd2810 2024-09-16T00:30:10.078 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi169.mtrobk smithi169 *:8443,9283,8765 running (2m) 35s ago 11m 438M - 19.1.1-326-g8921c8ef f1455bd0219c 4096e5369c2d 2024-09-16T00:30:10.079 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi112 smithi112 running (60s) 48s ago 13m 27.7M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 5aaec033c042 2024-09-16T00:30:10.079 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi169 smithi169 running (38s) 35s ago 11m 19.8M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 7bc609155f3f 2024-09-16T00:30:10.079 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi112 smithi112 *:9100 running (3m) 48s ago 12m 15.6M - 1.5.0 0da6a335fe13 6404d1d601e2 2024-09-16T00:30:10.079 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi169 smithi169 *:9100 running (2m) 35s ago 11m 15.0M - 1.5.0 0da6a335fe13 5505baf702a6 2024-09-16T00:30:10.079 INFO:teuthology.orchestra.run.smithi112.stdout:osd.0 smithi112 running (10m) 48s ago 10m 191M 1557M 17.2.7-1633-g6d025bc2 33ed8dba3996 f6b2e146ffdf 2024-09-16T00:30:10.079 INFO:teuthology.orchestra.run.smithi112.stdout:osd.1 smithi112 running (9m) 48s ago 9m 201M 1557M 17.2.7-1633-g6d025bc2 33ed8dba3996 e9bee87bba0f 2024-09-16T00:30:10.079 INFO:teuthology.orchestra.run.smithi112.stdout:osd.2 smithi112 running (9m) 48s ago 9m 159M 1557M 17.2.7-1633-g6d025bc2 33ed8dba3996 100ec4e4852a 2024-09-16T00:30:10.079 INFO:teuthology.orchestra.run.smithi112.stdout:osd.3 smithi169 running (9m) 35s ago 9m 231M 2581M 17.2.7-1633-g6d025bc2 33ed8dba3996 f88463e19f23 2024-09-16T00:30:10.079 INFO:teuthology.orchestra.run.smithi112.stdout:osd.4 smithi169 running (8m) 35s ago 8m 211M 2581M 17.2.7-1633-g6d025bc2 33ed8dba3996 d3e4cdb016b9 2024-09-16T00:30:10.079 INFO:teuthology.orchestra.run.smithi112.stdout:osd.5 smithi169 running (8m) 35s ago 8m 198M 2581M 17.2.7-1633-g6d025bc2 33ed8dba3996 8e7d708d74eb 2024-09-16T00:30:10.079 INFO:teuthology.orchestra.run.smithi112.stdout:prometheus.smithi112 smithi112 *:9095 running (2m) 48s ago 11m 53.7M - 2.43.0 a07b618ecd1d 7a1d42e60d2b 2024-09-16T00:30:10.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.079+0000 7fa2bd2b7640 1 -- 172.21.15.112:0/1404781956 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa2a006e660 msgr2=0x7fa2a0070b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:10.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.079+0000 7fa2bd2b7640 1 --2- 172.21.15.112:0/1404781956 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa2a006e660 0x7fa2a0070b10 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa2b810eb00 tx=0x7fa2ac00b040 comp rx=0 tx=0).stop 2024-09-16T00:30:10.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.079+0000 7fa2bd2b7640 1 -- 172.21.15.112:0/1404781956 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2b8102530 msgr2=0x7fa2b8113240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:10.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.079+0000 7fa2bd2b7640 1 --2- 172.21.15.112:0/1404781956 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2b8102530 0x7fa2b8113240 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa2a8004750 tx=0x7fa2a8004d10 comp rx=0 tx=0).stop 2024-09-16T00:30:10.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.079+0000 7fa2bd2b7640 1 -- 172.21.15.112:0/1404781956 shutdown_connections 2024-09-16T00:30:10.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.079+0000 7fa2bd2b7640 1 --2- 172.21.15.112:0/1404781956 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa2a006e660 0x7fa2a0070b10 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:10.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.079+0000 7fa2bd2b7640 1 --2- 172.21.15.112:0/1404781956 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2b8102ed0 0x7fa2b8113780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:10.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.079+0000 7fa2bd2b7640 1 --2- 172.21.15.112:0/1404781956 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2b8102530 0x7fa2b8113240 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:10.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.079+0000 7fa2bd2b7640 1 -- 172.21.15.112:0/1404781956 >> 172.21.15.112:0/1404781956 conn(0x7fa2b80f80f0 msgr2=0x7fa2b80f90f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:10.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.079+0000 7fa2bd2b7640 1 -- 172.21.15.112:0/1404781956 shutdown_connections 2024-09-16T00:30:10.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.080+0000 7fa2bd2b7640 1 -- 172.21.15.112:0/1404781956 wait complete. 2024-09-16T00:30:10.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.156+0000 7f80e68ef640 1 -- 172.21.15.112:0/1693131267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80e0102550 msgr2=0x7f80e0102920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:10.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.156+0000 7f80e68ef640 1 --2- 172.21.15.112:0/1693131267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80e0102550 0x7f80e0102920 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f80d00048d0 tx=0x7f80d0009da0 comp rx=0 tx=0).stop 2024-09-16T00:30:10.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.156+0000 7f80e68ef640 1 -- 172.21.15.112:0/1693131267 shutdown_connections 2024-09-16T00:30:10.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.156+0000 7f80e68ef640 1 --2- 172.21.15.112:0/1693131267 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80e0102ef0 0x7f80e00fc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:10.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.156+0000 7f80e68ef640 1 --2- 172.21.15.112:0/1693131267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80e0102550 0x7f80e0102920 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:10.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.156+0000 7f80e68ef640 1 -- 172.21.15.112:0/1693131267 >> 172.21.15.112:0/1693131267 conn(0x7f80e00f80f0 msgr2=0x7f80e00fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:10.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.157+0000 7f80e68ef640 1 -- 172.21.15.112:0/1693131267 shutdown_connections 2024-09-16T00:30:10.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.157+0000 7f80e68ef640 1 -- 172.21.15.112:0/1693131267 wait complete. 2024-09-16T00:30:10.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.157+0000 7f80e68ef640 1 Processor -- start 2024-09-16T00:30:10.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.158+0000 7f80e68ef640 1 -- start start 2024-09-16T00:30:10.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.158+0000 7f80e68ef640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80e0102550 0x7f80e0113540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:10.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.158+0000 7f80e68ef640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80e0102ef0 0x7f80e0113a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:10.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.158+0000 7f80e68ef640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e01140d0 con 0x7f80e0102550 2024-09-16T00:30:10.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.158+0000 7f80e68ef640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e010d4e0 con 0x7f80e0102ef0 2024-09-16T00:30:10.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.158+0000 7f80dffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80e0102550 0x7f80e0113540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:10.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.158+0000 7f80dffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80e0102550 0x7f80e0113540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59444/0 (socket says 172.21.15.112:59444) 2024-09-16T00:30:10.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.158+0000 7f80dffff640 1 -- 172.21.15.112:0/2546786750 learned_addr learned my addr 172.21.15.112:0/2546786750 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:30:10.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.158+0000 7f80df7fe640 1 --2- 172.21.15.112:0/2546786750 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80e0102ef0 0x7f80e0113a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:10.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.158+0000 7f80dffff640 1 -- 172.21.15.112:0/2546786750 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80e0102ef0 msgr2=0x7f80e0113a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:10.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.158+0000 7f80dffff640 1 --2- 172.21.15.112:0/2546786750 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80e0102ef0 0x7f80e0113a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:10.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.158+0000 7f80dffff640 1 -- 172.21.15.112:0/2546786750 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80d0005040 con 0x7f80e0102550 2024-09-16T00:30:10.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.158+0000 7f80df7fe640 1 --2- 172.21.15.112:0/2546786750 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80e0102ef0 0x7f80e0113a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:30:10.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.159+0000 7f80dffff640 1 --2- 172.21.15.112:0/2546786750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80e0102550 0x7f80e0113540 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f80d0000c00 tx=0x7f80d0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:10.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.159+0000 7f80dcff9640 1 -- 172.21.15.112:0/2546786750 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80d0038070 con 0x7f80e0102550 2024-09-16T00:30:10.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.159+0000 7f80e68ef640 1 -- 172.21.15.112:0/2546786750 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80e010d760 con 0x7f80e0102550 2024-09-16T00:30:10.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.159+0000 7f80dcff9640 1 -- 172.21.15.112:0/2546786750 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f80d002ade0 con 0x7f80e0102550 2024-09-16T00:30:10.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.159+0000 7f80dcff9640 1 -- 172.21.15.112:0/2546786750 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80d002c7b0 con 0x7f80e0102550 2024-09-16T00:30:10.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.159+0000 7f80e68ef640 1 -- 172.21.15.112:0/2546786750 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80e010dca0 con 0x7f80e0102550 2024-09-16T00:30:10.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.160+0000 7f80e68ef640 1 -- 172.21.15.112:0/2546786750 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80e00fdcd0 con 0x7f80e0102550 2024-09-16T00:30:10.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.161+0000 7f80dcff9640 1 -- 172.21.15.112:0/2546786750 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 38) v1 ==== 92655+0+0 (secure 0 0 0) 0x7f80d0030020 con 0x7f80e0102550 2024-09-16T00:30:10.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.161+0000 7f80dcff9640 1 --2- 172.21.15.112:0/2546786750 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f80c806e660 0x7f80c8070b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:10.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.161+0000 7f80dcff9640 1 -- 172.21.15.112:0/2546786750 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f80d00ae960 con 0x7f80e0102550 2024-09-16T00:30:10.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.161+0000 7f80df7fe640 1 --2- 172.21.15.112:0/2546786750 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f80c806e660 0x7f80c8070b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:10.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.162+0000 7f80df7fe640 1 --2- 172.21.15.112:0/2546786750 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f80c806e660 0x7f80c8070b10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f80e010e7c0 tx=0x7f80d400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:10.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.165+0000 7f80dcff9640 1 -- 172.21.15.112:0/2546786750 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f80d0077a10 con 0x7f80e0102550 2024-09-16T00:30:10.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.399+0000 7f80e68ef640 1 -- 172.21.15.112:0/2546786750 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f80e00fbd50 con 0x7f80e0102550 2024-09-16T00:30:10.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.400+0000 7f80dcff9640 1 -- 172.21.15.112:0/2546786750 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+745 (secure 0 0 0) 0x7f80d007b0d0 con 0x7f80e0102550 2024-09-16T00:30:10.400 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:30:10.400 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": { 2024-09-16T00:30:10.400 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:30:10.400 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:30:10.400 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": { 2024-09-16T00:30:10.401 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:30:10.401 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:30:10.401 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": { 2024-09-16T00:30:10.401 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 17.2.7-1633-g6d025bc2 (6d025bc26f3176c910707f8cb4c8d8e076640615) quincy (stable)": 6 2024-09-16T00:30:10.401 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:30:10.401 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": { 2024-09-16T00:30:10.401 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 17.2.7-1633-g6d025bc2 (6d025bc26f3176c910707f8cb4c8d8e076640615) quincy (stable)": 4 2024-09-16T00:30:10.401 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:30:10.401 INFO:teuthology.orchestra.run.smithi112.stdout: "overall": { 2024-09-16T00:30:10.401 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 17.2.7-1633-g6d025bc2 (6d025bc26f3176c910707f8cb4c8d8e076640615) quincy (stable)": 10, 2024-09-16T00:30:10.401 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 4 2024-09-16T00:30:10.401 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-16T00:30:10.401 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:30:10.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.402+0000 7f80e68ef640 1 -- 172.21.15.112:0/2546786750 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f80c806e660 msgr2=0x7f80c8070b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:10.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.402+0000 7f80e68ef640 1 --2- 172.21.15.112:0/2546786750 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f80c806e660 0x7f80c8070b10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f80e010e7c0 tx=0x7f80d400b040 comp rx=0 tx=0).stop 2024-09-16T00:30:10.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.402+0000 7f80e68ef640 1 -- 172.21.15.112:0/2546786750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80e0102550 msgr2=0x7f80e0113540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:10.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.402+0000 7f80e68ef640 1 --2- 172.21.15.112:0/2546786750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80e0102550 0x7f80e0113540 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f80d0000c00 tx=0x7f80d0009da0 comp rx=0 tx=0).stop 2024-09-16T00:30:10.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.402+0000 7f80e68ef640 1 -- 172.21.15.112:0/2546786750 shutdown_connections 2024-09-16T00:30:10.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.402+0000 7f80e68ef640 1 --2- 172.21.15.112:0/2546786750 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f80c806e660 0x7f80c8070b10 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:10.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.402+0000 7f80e68ef640 1 --2- 172.21.15.112:0/2546786750 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80e0102ef0 0x7f80e0113a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:10.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.402+0000 7f80e68ef640 1 --2- 172.21.15.112:0/2546786750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80e0102550 0x7f80e0113540 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:10.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.402+0000 7f80e68ef640 1 -- 172.21.15.112:0/2546786750 >> 172.21.15.112:0/2546786750 conn(0x7f80e00f80f0 msgr2=0x7f80e00f9160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:10.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.402+0000 7f80e68ef640 1 -- 172.21.15.112:0/2546786750 shutdown_connections 2024-09-16T00:30:10.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.402+0000 7f80e68ef640 1 -- 172.21.15.112:0/2546786750 wait complete. 2024-09-16T00:30:10.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.480+0000 7f87f0456640 1 -- 172.21.15.112:0/3492595096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87e8102620 msgr2=0x7f87e81029f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:10.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.480+0000 7f87f0456640 1 --2- 172.21.15.112:0/3492595096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87e8102620 0x7f87e81029f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f87e40048d0 tx=0x7f87e4009da0 comp rx=0 tx=0).stop 2024-09-16T00:30:10.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.480+0000 7f87f0456640 1 -- 172.21.15.112:0/3492595096 shutdown_connections 2024-09-16T00:30:10.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.480+0000 7f87f0456640 1 --2- 172.21.15.112:0/3492595096 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f87e80fac80 0x7f87e80fb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:10.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.480+0000 7f87f0456640 1 --2- 172.21.15.112:0/3492595096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87e8102620 0x7f87e81029f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:10.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.480+0000 7f87f0456640 1 -- 172.21.15.112:0/3492595096 >> 172.21.15.112:0/3492595096 conn(0x7f87e8070ce0 msgr2=0x7f87e80710e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:10.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.480+0000 7f87f0456640 1 -- 172.21.15.112:0/3492595096 shutdown_connections 2024-09-16T00:30:10.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.480+0000 7f87f0456640 1 -- 172.21.15.112:0/3492595096 wait complete. 2024-09-16T00:30:10.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.481+0000 7f87f0456640 1 Processor -- start 2024-09-16T00:30:10.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.481+0000 7f87f0456640 1 -- start start 2024-09-16T00:30:10.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.481+0000 7f87f0456640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f87e80fac80 0x7f87e81131c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:10.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.481+0000 7f87f0456640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87e8102620 0x7f87e8113700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:10.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.481+0000 7f87f0456640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e8113d00 con 0x7f87e8102620 2024-09-16T00:30:10.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.481+0000 7f87f0456640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e810d140 con 0x7f87e80fac80 2024-09-16T00:30:10.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.481+0000 7f87ed9f1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87e8102620 0x7f87e8113700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:10.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.481+0000 7f87ee1f2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f87e80fac80 0x7f87e81131c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:10.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.481+0000 7f87ed9f1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87e8102620 0x7f87e8113700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59474/0 (socket says 172.21.15.112:59474) 2024-09-16T00:30:10.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.481+0000 7f87ed9f1640 1 -- 172.21.15.112:0/641372542 learned_addr learned my addr 172.21.15.112:0/641372542 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:30:10.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.481+0000 7f87ed9f1640 1 -- 172.21.15.112:0/641372542 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f87e80fac80 msgr2=0x7f87e81131c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:10.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.481+0000 7f87ed9f1640 1 --2- 172.21.15.112:0/641372542 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f87e80fac80 0x7f87e81131c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:10.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.481+0000 7f87ed9f1640 1 -- 172.21.15.112:0/641372542 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87e4005040 con 0x7f87e8102620 2024-09-16T00:30:10.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.482+0000 7f87ed9f1640 1 --2- 172.21.15.112:0/641372542 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87e8102620 0x7f87e8113700 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f87d800eb70 tx=0x7f87d800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:10.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.482+0000 7f87deffd640 1 -- 172.21.15.112:0/641372542 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87d8007c30 con 0x7f87e8102620 2024-09-16T00:30:10.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.482+0000 7f87deffd640 1 -- 172.21.15.112:0/641372542 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f87d800f040 con 0x7f87e8102620 2024-09-16T00:30:10.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.482+0000 7f87deffd640 1 -- 172.21.15.112:0/641372542 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87d800ca60 con 0x7f87e8102620 2024-09-16T00:30:10.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.482+0000 7f87f0456640 1 -- 172.21.15.112:0/641372542 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87e810d3c0 con 0x7f87e8102620 2024-09-16T00:30:10.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.482+0000 7f87f0456640 1 -- 172.21.15.112:0/641372542 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87e810d960 con 0x7f87e8102620 2024-09-16T00:30:10.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.483+0000 7f87f0456640 1 -- 172.21.15.112:0/641372542 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87e80657b0 con 0x7f87e8102620 2024-09-16T00:30:10.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.484+0000 7f87deffd640 1 -- 172.21.15.112:0/641372542 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 38) v1 ==== 92655+0+0 (secure 0 0 0) 0x7f87d800cbc0 con 0x7f87e8102620 2024-09-16T00:30:10.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.484+0000 7f87deffd640 1 --2- 172.21.15.112:0/641372542 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f87d406e660 0x7f87d4070b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:10.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.484+0000 7f87deffd640 1 -- 172.21.15.112:0/641372542 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f87d808df00 con 0x7f87e8102620 2024-09-16T00:30:10.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.484+0000 7f87ee1f2640 1 --2- 172.21.15.112:0/641372542 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f87d406e660 0x7f87d4070b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:10.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.485+0000 7f87ee1f2640 1 --2- 172.21.15.112:0/641372542 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f87d406e660 0x7f87d4070b10 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f87e4004540 tx=0x7f87e4035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:10.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.489+0000 7f87deffd640 1 -- 172.21.15.112:0/641372542 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f87d8056f50 con 0x7f87e8102620 2024-09-16T00:30:10.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.688+0000 7f87f0456640 1 -- 172.21.15.112:0/641372542 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f87e8109590 con 0x7f87e8102620 2024-09-16T00:30:10.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.688+0000 7f87deffd640 1 -- 172.21.15.112:0/641372542 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) v1 ==== 76+0+1954 (secure 0 0 0) 0x7f87d80197c0 con 0x7f87e8102620 2024-09-16T00:30:10.689 INFO:teuthology.orchestra.run.smithi112.stdout:e23 2024-09-16T00:30:10.689 INFO:teuthology.orchestra.run.smithi112.stdout:btime 2024-09-16T00:30:08:788506+0000 2024-09-16T00:30:10.689 INFO:teuthology.orchestra.run.smithi112.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T00:30:10.689 INFO:teuthology.orchestra.run.smithi112.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:30:10.689 INFO:teuthology.orchestra.run.smithi112.stdout:legacy client fscid: 1 2024-09-16T00:30:10.689 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:30:10.689 INFO:teuthology.orchestra.run.smithi112.stdout:Filesystem 'cephfs' (1) 2024-09-16T00:30:10.689 INFO:teuthology.orchestra.run.smithi112.stdout:fs_name cephfs 2024-09-16T00:30:10.690 INFO:teuthology.orchestra.run.smithi112.stdout:epoch 23 2024-09-16T00:30:10.690 INFO:teuthology.orchestra.run.smithi112.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-16T00:30:10.690 INFO:teuthology.orchestra.run.smithi112.stdout:created 2024-09-16T00:23:22.328199+0000 2024-09-16T00:30:10.690 INFO:teuthology.orchestra.run.smithi112.stdout:modified 2024-09-16T00:30:08.396661+0000 2024-09-16T00:30:10.690 INFO:teuthology.orchestra.run.smithi112.stdout:tableserver 0 2024-09-16T00:30:10.690 INFO:teuthology.orchestra.run.smithi112.stdout:root 0 2024-09-16T00:30:10.690 INFO:teuthology.orchestra.run.smithi112.stdout:session_timeout 60 2024-09-16T00:30:10.690 INFO:teuthology.orchestra.run.smithi112.stdout:session_autoclose 300 2024-09-16T00:30:10.690 INFO:teuthology.orchestra.run.smithi112.stdout:max_file_size 1099511627776 2024-09-16T00:30:10.690 INFO:teuthology.orchestra.run.smithi112.stdout:max_xattr_size 65536 2024-09-16T00:30:10.690 INFO:teuthology.orchestra.run.smithi112.stdout:required_client_features {} 2024-09-16T00:30:10.690 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure 0 2024-09-16T00:30:10.691 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure_osd_epoch 44 2024-09-16T00:30:10.691 INFO:teuthology.orchestra.run.smithi112.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:30:10.691 INFO:teuthology.orchestra.run.smithi112.stdout:max_mds 2 2024-09-16T00:30:10.691 INFO:teuthology.orchestra.run.smithi112.stdout:in 0,1 2024-09-16T00:30:10.691 INFO:teuthology.orchestra.run.smithi112.stdout:up {0=14480,1=24259} 2024-09-16T00:30:10.691 INFO:teuthology.orchestra.run.smithi112.stdout:failed 2024-09-16T00:30:10.691 INFO:teuthology.orchestra.run.smithi112.stdout:damaged 2024-09-16T00:30:10.691 INFO:teuthology.orchestra.run.smithi112.stdout:stopped 2024-09-16T00:30:10.691 INFO:teuthology.orchestra.run.smithi112.stdout:data_pools [3] 2024-09-16T00:30:10.691 INFO:teuthology.orchestra.run.smithi112.stdout:metadata_pool 2 2024-09-16T00:30:10.691 INFO:teuthology.orchestra.run.smithi112.stdout:inline_data enabled 2024-09-16T00:30:10.691 INFO:teuthology.orchestra.run.smithi112.stdout:balancer 2024-09-16T00:30:10.692 INFO:teuthology.orchestra.run.smithi112.stdout:bal_rank_mask -1 2024-09-16T00:30:10.692 INFO:teuthology.orchestra.run.smithi112.stdout:standby_count_wanted 1 2024-09-16T00:30:10.692 INFO:teuthology.orchestra.run.smithi112.stdout:qdb_cluster leader: 14480 members: 24259,14480 2024-09-16T00:30:10.692 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.hxfwkh{0:14480} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.112:6828/486654885,v1:172.21.15.112:6829/486654885] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T00:30:10.692 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.ithwzj{0:14492} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.112:6826/381830565,v1:172.21.15.112:6827/381830565] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T00:30:10.692 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.gawwgw{1:24259} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.169:6824/2892476953,v1:172.21.15.169:6825/2892476953] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T00:30:10.692 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.lgeqrj{1:24269} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.169:6826/861725728,v1:172.21.15.169:6827/861725728] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T00:30:10.692 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:30:10.692 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:30:10.693 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 23 2024-09-16T00:30:10.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.690+0000 7f87f0456640 1 -- 172.21.15.112:0/641372542 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f87d406e660 msgr2=0x7f87d4070b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:10.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.690+0000 7f87f0456640 1 --2- 172.21.15.112:0/641372542 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f87d406e660 0x7f87d4070b10 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f87e4004540 tx=0x7f87e4035040 comp rx=0 tx=0).stop 2024-09-16T00:30:10.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.690+0000 7f87f0456640 1 -- 172.21.15.112:0/641372542 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87e8102620 msgr2=0x7f87e8113700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:10.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.690+0000 7f87f0456640 1 --2- 172.21.15.112:0/641372542 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87e8102620 0x7f87e8113700 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f87d800eb70 tx=0x7f87d800b040 comp rx=0 tx=0).stop 2024-09-16T00:30:10.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.690+0000 7f87f0456640 1 -- 172.21.15.112:0/641372542 shutdown_connections 2024-09-16T00:30:10.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.690+0000 7f87f0456640 1 --2- 172.21.15.112:0/641372542 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f87d406e660 0x7f87d4070b10 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:10.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.690+0000 7f87f0456640 1 --2- 172.21.15.112:0/641372542 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87e8102620 0x7f87e8113700 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:10.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.690+0000 7f87f0456640 1 --2- 172.21.15.112:0/641372542 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f87e80fac80 0x7f87e81131c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:10.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.690+0000 7f87f0456640 1 -- 172.21.15.112:0/641372542 >> 172.21.15.112:0/641372542 conn(0x7f87e8070ce0 msgr2=0x7f87e8105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:10.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.690+0000 7f87f0456640 1 -- 172.21.15.112:0/641372542 shutdown_connections 2024-09-16T00:30:10.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.690+0000 7f87f0456640 1 -- 172.21.15.112:0/641372542 wait complete. 2024-09-16T00:30:10.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.770+0000 7f34c695d640 1 -- 172.21.15.112:0/327663868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34c00fb810 msgr2=0x7f34c00fbbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:10.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.770+0000 7f34c695d640 1 --2- 172.21.15.112:0/327663868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34c00fb810 0x7f34c00fbbe0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f34b00048d0 tx=0x7f34b0009da0 comp rx=0 tx=0).stop 2024-09-16T00:30:10.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.770+0000 7f34c695d640 1 -- 172.21.15.112:0/327663868 shutdown_connections 2024-09-16T00:30:10.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.770+0000 7f34c695d640 1 --2- 172.21.15.112:0/327663868 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34c00fcf80 0x7f34c00fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:10.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.770+0000 7f34c695d640 1 --2- 172.21.15.112:0/327663868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34c00fb810 0x7f34c00fbbe0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:10.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.770+0000 7f34c695d640 1 -- 172.21.15.112:0/327663868 >> 172.21.15.112:0/327663868 conn(0x7f34c00723b0 msgr2=0x7f34c00727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:10.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.770+0000 7f34c695d640 1 -- 172.21.15.112:0/327663868 shutdown_connections 2024-09-16T00:30:10.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.770+0000 7f34c695d640 1 -- 172.21.15.112:0/327663868 wait complete. 2024-09-16T00:30:10.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.770+0000 7f34c695d640 1 Processor -- start 2024-09-16T00:30:10.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.771+0000 7f34c695d640 1 -- start start 2024-09-16T00:30:10.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.771+0000 7f34c695d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34c00fb810 0x7f34c010e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:10.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.771+0000 7f34c695d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34c00fcf80 0x7f34c010e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:10.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.771+0000 7f34c695d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34c010ed70 con 0x7f34c00fb810 2024-09-16T00:30:10.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.771+0000 7f34c695d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34c010eee0 con 0x7f34c00fcf80 2024-09-16T00:30:10.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.771+0000 7f34bffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34c00fb810 0x7f34c010e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:10.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.771+0000 7f34bffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34c00fb810 0x7f34c010e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59498/0 (socket says 172.21.15.112:59498) 2024-09-16T00:30:10.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.771+0000 7f34bffff640 1 -- 172.21.15.112:0/3857321974 learned_addr learned my addr 172.21.15.112:0/3857321974 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:30:10.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.771+0000 7f34b7fff640 1 --2- 172.21.15.112:0/3857321974 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34c00fcf80 0x7f34c010e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:10.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.771+0000 7f34bffff640 1 -- 172.21.15.112:0/3857321974 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34c00fcf80 msgr2=0x7f34c010e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:10.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.771+0000 7f34bffff640 1 --2- 172.21.15.112:0/3857321974 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34c00fcf80 0x7f34c010e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:10.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.771+0000 7f34bffff640 1 -- 172.21.15.112:0/3857321974 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34b0005040 con 0x7f34c00fb810 2024-09-16T00:30:10.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.771+0000 7f34b7fff640 1 --2- 172.21.15.112:0/3857321974 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34c00fcf80 0x7f34c010e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:30:10.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.771+0000 7f34bffff640 1 --2- 172.21.15.112:0/3857321974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34c00fb810 0x7f34c010e1a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f34b0003080 tx=0x7f34b0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:10.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.772+0000 7f34bd7fa640 1 -- 172.21.15.112:0/3857321974 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34b0038070 con 0x7f34c00fb810 2024-09-16T00:30:10.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.772+0000 7f34c695d640 1 -- 172.21.15.112:0/3857321974 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34c0117210 con 0x7f34c00fb810 2024-09-16T00:30:10.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.772+0000 7f34bd7fa640 1 -- 172.21.15.112:0/3857321974 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f34b00097e0 con 0x7f34c00fb810 2024-09-16T00:30:10.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.772+0000 7f34bd7fa640 1 -- 172.21.15.112:0/3857321974 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34b003d420 con 0x7f34c00fb810 2024-09-16T00:30:10.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.772+0000 7f34c695d640 1 -- 172.21.15.112:0/3857321974 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34c0117680 con 0x7f34c00fb810 2024-09-16T00:30:10.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.773+0000 7f34c695d640 1 -- 172.21.15.112:0/3857321974 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34c00fcbc0 con 0x7f34c00fb810 2024-09-16T00:30:10.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.773+0000 7f34bd7fa640 1 -- 172.21.15.112:0/3857321974 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 38) v1 ==== 92655+0+0 (secure 0 0 0) 0x7f34b0030020 con 0x7f34c00fb810 2024-09-16T00:30:10.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.774+0000 7f34bd7fa640 1 --2- 172.21.15.112:0/3857321974 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f34ac06e660 0x7f34ac070b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:10.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.774+0000 7f34bd7fa640 1 -- 172.21.15.112:0/3857321974 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f34b00ae760 con 0x7f34c00fb810 2024-09-16T00:30:10.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.774+0000 7f34b7fff640 1 --2- 172.21.15.112:0/3857321974 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f34ac06e660 0x7f34ac070b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:10.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.775+0000 7f34b7fff640 1 --2- 172.21.15.112:0/3857321974 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f34ac06e660 0x7f34ac070b10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f34c00659a0 tx=0x7f34a8008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:10.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.778+0000 7f34bd7fa640 1 -- 172.21.15.112:0/3857321974 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f34b0077810 con 0x7f34c00fb810 2024-09-16T00:30:10.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.961+0000 7f34c695d640 1 -- 172.21.15.112:0/3857321974 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f34c0065750 con 0x7f34ac06e660 2024-09-16T00:30:10.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.962+0000 7f34bd7fa640 1 -- 172.21.15.112:0/3857321974 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+385 (secure 0 0 0) 0x7f34c0065750 con 0x7f34ac06e660 2024-09-16T00:30:10.963 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:30:10.963 INFO:teuthology.orchestra.run.smithi112.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80", 2024-09-16T00:30:10.963 INFO:teuthology.orchestra.run.smithi112.stdout: "in_progress": true, 2024-09-16T00:30:10.963 INFO:teuthology.orchestra.run.smithi112.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T00:30:10.963 INFO:teuthology.orchestra.run.smithi112.stdout: "services_complete": [ 2024-09-16T00:30:10.963 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr", 2024-09-16T00:30:10.963 INFO:teuthology.orchestra.run.smithi112.stdout: "mon" 2024-09-16T00:30:10.963 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-16T00:30:10.963 INFO:teuthology.orchestra.run.smithi112.stdout: "progress": "4/21 daemons upgraded", 2024-09-16T00:30:10.963 INFO:teuthology.orchestra.run.smithi112.stdout: "message": "Currently upgrading crash daemons", 2024-09-16T00:30:10.963 INFO:teuthology.orchestra.run.smithi112.stdout: "is_paused": false 2024-09-16T00:30:10.964 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:30:10.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.964+0000 7f34c695d640 1 -- 172.21.15.112:0/3857321974 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f34ac06e660 msgr2=0x7f34ac070b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:10.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.965+0000 7f34c695d640 1 --2- 172.21.15.112:0/3857321974 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f34ac06e660 0x7f34ac070b10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f34c00659a0 tx=0x7f34a8008040 comp rx=0 tx=0).stop 2024-09-16T00:30:10.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.965+0000 7f34c695d640 1 -- 172.21.15.112:0/3857321974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34c00fb810 msgr2=0x7f34c010e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:10.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.965+0000 7f34c695d640 1 --2- 172.21.15.112:0/3857321974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34c00fb810 0x7f34c010e1a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f34b0003080 tx=0x7f34b0009da0 comp rx=0 tx=0).stop 2024-09-16T00:30:10.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.965+0000 7f34c695d640 1 -- 172.21.15.112:0/3857321974 shutdown_connections 2024-09-16T00:30:10.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.965+0000 7f34c695d640 1 --2- 172.21.15.112:0/3857321974 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f34ac06e660 0x7f34ac070b10 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:10.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.965+0000 7f34c695d640 1 --2- 172.21.15.112:0/3857321974 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34c00fcf80 0x7f34c010e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:10.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.965+0000 7f34c695d640 1 --2- 172.21.15.112:0/3857321974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34c00fb810 0x7f34c010e1a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:10.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.965+0000 7f34c695d640 1 -- 172.21.15.112:0/3857321974 >> 172.21.15.112:0/3857321974 conn(0x7f34c00723b0 msgr2=0x7f34c00fa480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:10.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.965+0000 7f34c695d640 1 -- 172.21.15.112:0/3857321974 shutdown_connections 2024-09-16T00:30:10.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:10.965+0000 7f34c695d640 1 -- 172.21.15.112:0/3857321974 wait complete. 2024-09-16T00:30:11.006 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:10 smithi169 ceph-mon[77626]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:30:11.007 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:10 smithi169 ceph-mon[77626]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:30:11.007 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:10 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/2546786750' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:30:11.007 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:10 smithi169 ceph-mon[77626]: pgmap v30: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:30:11.007 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:10 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/641372542' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:30:11.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:11.043+0000 7f8736f19640 1 -- 172.21.15.112:0/2541350683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8730065c10 msgr2=0x7f87300fe0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:11.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:11.043+0000 7f8736f19640 1 --2- 172.21.15.112:0/2541350683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8730065c10 0x7f87300fe0d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f871c0048d0 tx=0x7f871c009da0 comp rx=0 tx=0).stop 2024-09-16T00:30:11.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:11.043+0000 7f8736f19640 1 -- 172.21.15.112:0/2541350683 shutdown_connections 2024-09-16T00:30:11.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:11.043+0000 7f8736f19640 1 --2- 172.21.15.112:0/2541350683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8730065c10 0x7f87300fe0d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:11.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:11.043+0000 7f8736f19640 1 --2- 172.21.15.112:0/2541350683 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8730065300 0x7f87300656d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:11.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:11.043+0000 7f8736f19640 1 -- 172.21.15.112:0/2541350683 >> 172.21.15.112:0/2541350683 conn(0x7f87300723b0 msgr2=0x7f87300727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:11.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:11.043+0000 7f8736f19640 1 -- 172.21.15.112:0/2541350683 shutdown_connections 2024-09-16T00:30:11.044 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:10 smithi112 ceph-mon[97750]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:30:11.045 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:10 smithi112 ceph-mon[97750]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:30:11.045 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:10 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/2546786750' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:30:11.045 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:10 smithi112 ceph-mon[97750]: pgmap v30: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:30:11.045 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:10 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/641372542' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:30:11.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:11.043+0000 7f8736f19640 1 -- 172.21.15.112:0/2541350683 wait complete. 2024-09-16T00:30:11.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:11.044+0000 7f8736f19640 1 Processor -- start 2024-09-16T00:30:11.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:11.044+0000 7f8736f19640 1 -- start start 2024-09-16T00:30:11.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:11.044+0000 7f8736f19640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8730065300 0x7f87301131b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:11.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:11.044+0000 7f8736f19640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8730065c10 0x7f87301136f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:11.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:11.044+0000 7f8736f19640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8730113cf0 con 0x7f8730065300 2024-09-16T00:30:11.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:11.044+0000 7f8736f19640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f873010d130 con 0x7f8730065c10 2024-09-16T00:30:11.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:11.044+0000 7f8734cb5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8730065300 0x7f87301131b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:11.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:11.045+0000 7f872ffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8730065c10 0x7f87301136f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:11.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:11.045+0000 7f8734cb5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8730065300 0x7f87301131b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59512/0 (socket says 172.21.15.112:59512) 2024-09-16T00:30:11.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:11.045+0000 7f8734cb5640 1 -- 172.21.15.112:0/2898126890 learned_addr learned my addr 172.21.15.112:0/2898126890 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:30:11.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:11.045+0000 7f8734cb5640 1 -- 172.21.15.112:0/2898126890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8730065c10 msgr2=0x7f87301136f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:11.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:11.045+0000 7f8734cb5640 1 --2- 172.21.15.112:0/2898126890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8730065c10 0x7f87301136f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:11.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:11.045+0000 7f8734cb5640 1 -- 172.21.15.112:0/2898126890 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f871c005040 con 0x7f8730065300 2024-09-16T00:30:11.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:11.045+0000 7f872ffff640 1 --2- 172.21.15.112:0/2898126890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8730065c10 0x7f87301136f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:30:11.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:11.045+0000 7f8734cb5640 1 --2- 172.21.15.112:0/2898126890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8730065300 0x7f87301131b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f872400eb70 tx=0x7f872400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:11.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:11.045+0000 7f872d7fa640 1 -- 172.21.15.112:0/2898126890 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8724007c30 con 0x7f8730065300 2024-09-16T00:30:11.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:11.045+0000 7f872d7fa640 1 -- 172.21.15.112:0/2898126890 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f872400f040 con 0x7f8730065300 2024-09-16T00:30:11.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:11.045+0000 7f872d7fa640 1 -- 172.21.15.112:0/2898126890 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f872400ca60 con 0x7f8730065300 2024-09-16T00:30:11.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:11.045+0000 7f8736f19640 1 -- 172.21.15.112:0/2898126890 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f873010d3b0 con 0x7f8730065300 2024-09-16T00:30:11.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:11.045+0000 7f8736f19640 1 -- 172.21.15.112:0/2898126890 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f873010d920 con 0x7f8730065300 2024-09-16T00:30:11.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:11.046+0000 7f8736f19640 1 -- 172.21.15.112:0/2898126890 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87300fdce0 con 0x7f8730065300 2024-09-16T00:30:11.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:11.047+0000 7f872d7fa640 1 -- 172.21.15.112:0/2898126890 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 38) v1 ==== 92655+0+0 (secure 0 0 0) 0x7f872400cbc0 con 0x7f8730065300 2024-09-16T00:30:11.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:11.047+0000 7f872d7fa640 1 --2- 172.21.15.112:0/2898126890 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f872006e660 0x7f8720070b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:11.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:11.047+0000 7f872d7fa640 1 -- 172.21.15.112:0/2898126890 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f872408dff0 con 0x7f8730065300 2024-09-16T00:30:11.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:11.047+0000 7f872ffff640 1 --2- 172.21.15.112:0/2898126890 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f872006e660 0x7f8720070b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:11.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:11.048+0000 7f872ffff640 1 --2- 172.21.15.112:0/2898126890 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f872006e660 0x7f8720070b10 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f87300f86a0 tx=0x7f871c035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:11.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:11.053+0000 7f872d7fa640 1 -- 172.21.15.112:0/2898126890 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f8724056fc0 con 0x7f8730065300 2024-09-16T00:30:11.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:11.294+0000 7f8736f19640 1 -- 172.21.15.112:0/2898126890 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f873004b750 con 0x7f8730065300 2024-09-16T00:30:11.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:11.295+0000 7f872d7fa640 1 -- 172.21.15.112:0/2898126890 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f872405a680 con 0x7f8730065300 2024-09-16T00:30:11.295 INFO:teuthology.orchestra.run.smithi112.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-09-16T00:30:11.296 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T00:30:11.296 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T00:30:11.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:11.297+0000 7f8736f19640 1 -- 172.21.15.112:0/2898126890 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f872006e660 msgr2=0x7f8720070b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:11.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:11.297+0000 7f8736f19640 1 --2- 172.21.15.112:0/2898126890 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f872006e660 0x7f8720070b10 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f87300f86a0 tx=0x7f871c035040 comp rx=0 tx=0).stop 2024-09-16T00:30:11.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:11.297+0000 7f8736f19640 1 -- 172.21.15.112:0/2898126890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8730065300 msgr2=0x7f87301131b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:11.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:11.297+0000 7f8736f19640 1 --2- 172.21.15.112:0/2898126890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8730065300 0x7f87301131b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f872400eb70 tx=0x7f872400b040 comp rx=0 tx=0).stop 2024-09-16T00:30:11.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:11.297+0000 7f8736f19640 1 -- 172.21.15.112:0/2898126890 shutdown_connections 2024-09-16T00:30:11.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:11.297+0000 7f8736f19640 1 --2- 172.21.15.112:0/2898126890 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f872006e660 0x7f8720070b10 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:11.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:11.297+0000 7f8736f19640 1 --2- 172.21.15.112:0/2898126890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8730065c10 0x7f87301136f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:11.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:11.297+0000 7f8736f19640 1 --2- 172.21.15.112:0/2898126890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8730065300 0x7f87301131b0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:11.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:11.297+0000 7f8736f19640 1 -- 172.21.15.112:0/2898126890 >> 172.21.15.112:0/2898126890 conn(0x7f87300723b0 msgr2=0x7f87300fa2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:11.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:11.297+0000 7f8736f19640 1 -- 172.21.15.112:0/2898126890 shutdown_connections 2024-09-16T00:30:11.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:11.297+0000 7f8736f19640 1 -- 172.21.15.112:0/2898126890 wait complete. 2024-09-16T00:30:12.317 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:12 smithi112 ceph-mon[97750]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:30:12.317 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:12 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:12.317 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:12 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:12.317 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:12 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:30:12.317 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:12 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/2898126890' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:30:12.371 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:12 smithi169 ceph-mon[77626]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:30:12.372 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:12 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:12.372 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:12 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:12.372 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:12 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:30:12.372 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:12 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/2898126890' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:30:13.307 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:13 smithi169 ceph-mon[77626]: pgmap v31: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:30:13.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:13 smithi112 ceph-mon[97750]: pgmap v31: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:30:15.033 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:14 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:15.034 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:14 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:15.034 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:14 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:15.034 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:14 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:15.034 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:14 smithi169 ceph-mon[77626]: pgmap v32: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:30:15.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:14 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:15.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:14 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:15.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:14 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:15.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:14 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:15.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:14 smithi112 ceph-mon[97750]: pgmap v32: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:30:17.317 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:17 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:17.317 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:17 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:17.317 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:17 smithi112 ceph-mon[97750]: pgmap v33: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:30:17.317 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:17 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:30:17.317 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:17 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:17.318 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:17 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:17.384 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:17 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:17.384 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:17 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:17.384 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:17 smithi169 ceph-mon[77626]: pgmap v33: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:30:17.384 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:17 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:30:17.384 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:17 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:17.384 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:17 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:18.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:18 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:18.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:18 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:18.678 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:18 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:18.678 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:18 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:19.721 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:19 smithi112 ceph-mon[97750]: pgmap v34: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:30:19.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:19 smithi169 ceph-mon[77626]: pgmap v34: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:30:20.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:20 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:20.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:20 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:20.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:20 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:30:20.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:20 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:30:20.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:20 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:20.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:20 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:30:20.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:20 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:30:20.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:20 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:30:20.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:20 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:30:20.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:20 smithi112 ceph-mon[97750]: Upgrade: Setting container_image for all crash 2024-09-16T00:30:20.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:20 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:20.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:20 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi112"}]: dispatch 2024-09-16T00:30:20.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:20 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi112"}]': finished 2024-09-16T00:30:20.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:20 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi169"}]: dispatch 2024-09-16T00:30:20.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:20 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi169"}]': finished 2024-09-16T00:30:20.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:20 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-09-16T00:30:20.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:20 smithi112 ceph-mon[97750]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-09-16T00:30:20.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:20 smithi112 ceph-mon[97750]: Upgrade: osd.0 is safe to restart 2024-09-16T00:30:20.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:20 smithi112 ceph-mon[97750]: pgmap v35: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:30:21.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:20 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:21.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:20 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:21.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:20 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:30:21.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:20 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:30:21.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:20 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:21.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:20 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:30:21.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:20 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:30:21.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:20 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:30:21.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:20 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:30:21.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:20 smithi169 ceph-mon[77626]: Upgrade: Setting container_image for all crash 2024-09-16T00:30:21.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:20 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:21.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:20 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi112"}]: dispatch 2024-09-16T00:30:21.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:20 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi112"}]': finished 2024-09-16T00:30:21.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:20 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi169"}]: dispatch 2024-09-16T00:30:21.028 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:20 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi169"}]': finished 2024-09-16T00:30:21.028 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:20 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-09-16T00:30:21.028 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:20 smithi169 ceph-mon[77626]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-09-16T00:30:21.028 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:20 smithi169 ceph-mon[77626]: Upgrade: osd.0 is safe to restart 2024-09-16T00:30:21.028 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:20 smithi169 ceph-mon[77626]: pgmap v35: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:30:22.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:22 smithi169 ceph-mon[77626]: Upgrade: Updating osd.0 2024-09-16T00:30:22.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:22 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:22.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:22 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-16T00:30:22.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:22 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:30:22.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:22 smithi169 ceph-mon[77626]: Deploying daemon osd.0 on smithi112 2024-09-16T00:30:22.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:22 smithi112 ceph-mon[97750]: Upgrade: Updating osd.0 2024-09-16T00:30:22.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:22 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:22.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:22 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-16T00:30:22.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:22 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:30:22.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:22 smithi112 ceph-mon[97750]: Deploying daemon osd.0 on smithi112 2024-09-16T00:30:23.505 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:23 smithi112 ceph-mon[97750]: pgmap v36: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:30:23.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:23 smithi169 ceph-mon[77626]: pgmap v36: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:30:24.574 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:30:24 smithi112 systemd[1]: Stopping Ceph osd.0 for d346dd56-73c0-11ef-bceb-c7b262605968... 2024-09-16T00:30:24.839 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:30:24 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-0[46028]: 2024-09-16T00:30:24.573+0000 7fc5df058640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:30:24.839 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:30:24 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-0[46028]: 2024-09-16T00:30:24.573+0000 7fc5df058640 -1 osd.0 47 *** Got signal Terminated *** 2024-09-16T00:30:24.839 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:30:24 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-0[46028]: 2024-09-16T00:30:24.573+0000 7fc5df058640 -1 osd.0 47 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:30:25.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:25 smithi112 ceph-mon[97750]: pgmap v37: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:30:25.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:25 smithi112 ceph-mon[97750]: osd.0 marked itself down and dead 2024-09-16T00:30:26.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:25 smithi169 ceph-mon[77626]: pgmap v37: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:30:26.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:25 smithi169 ceph-mon[77626]: osd.0 marked itself down and dead 2024-09-16T00:30:26.410 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:30:26 smithi112 podman[108441]: 2024-09-16 00:30:26.107555215 +0000 UTC m=+1.658105318 container died f6b2e146ffdf2b863b1ecc83bd9744084e487ef11394c26a26607ec01f0614e3 (image=quay.ceph.io/ceph-ci/ceph@sha256:83596473fc70117d507bb3d925c5bd11b70f46bbd4dffcc8452f145d93eaedc4, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=quincy-6d025bc, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:30:26.775 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:26 smithi112 ceph-mon[97750]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-16T00:30:26.775 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:26 smithi112 ceph-mon[97750]: osdmap e48: 6 total, 5 up, 6 in 2024-09-16T00:30:26.776 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:30:26 smithi112 podman[108441]: 2024-09-16 00:30:26.507377007 +0000 UTC m=+2.057927111 container cleanup f6b2e146ffdf2b863b1ecc83bd9744084e487ef11394c26a26607ec01f0614e3 (image=quay.ceph.io/ceph-ci/ceph@sha256:83596473fc70117d507bb3d925c5bd11b70f46bbd4dffcc8452f145d93eaedc4, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-0, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=quincy-6d025bc) 2024-09-16T00:30:26.776 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:30:26 smithi112 bash[108441]: ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-0 2024-09-16T00:30:27.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:26 smithi169 ceph-mon[77626]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-16T00:30:27.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:26 smithi169 ceph-mon[77626]: osdmap e48: 6 total, 5 up, 6 in 2024-09-16T00:30:27.089 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:30:26 smithi112 podman[108456]: 2024-09-16 00:30:26.775531521 +0000 UTC m=+0.670316706 container remove f6b2e146ffdf2b863b1ecc83bd9744084e487ef11394c26a26607ec01f0614e3 (image=quay.ceph.io/ceph-ci/ceph@sha256:83596473fc70117d507bb3d925c5bd11b70f46bbd4dffcc8452f145d93eaedc4, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=quincy-6d025bc, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-16T00:30:27.403 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:30:27 smithi112 podman[108545]: 2024-09-16 00:30:27.030801028 +0000 UTC m=+0.015502465 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:30:27.404 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:30:27 smithi112 podman[108545]: 2024-09-16 00:30:27.193266712 +0000 UTC m=+0.177968141 container create 83368c42b3f6fd39244ec2cb513fccd2b4875942c3a46659cbadfe03329aca12 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:30:27.404 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:30:27 smithi112 podman[108545]: 2024-09-16 00:30:27.404123338 +0000 UTC m=+0.388824769 container init 83368c42b3f6fd39244ec2cb513fccd2b4875942c3a46659cbadfe03329aca12 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-16T00:30:27.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:27 smithi112 ceph-mon[97750]: pgmap v39: 65 pgs: 9 stale+active+clean, 56 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-09-16T00:30:27.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:27 smithi112 ceph-mon[97750]: osdmap e49: 6 total, 5 up, 6 in 2024-09-16T00:30:27.839 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:30:27 smithi112 podman[108545]: 2024-09-16 00:30:27.407556781 +0000 UTC m=+0.392258212 container start 83368c42b3f6fd39244ec2cb513fccd2b4875942c3a46659cbadfe03329aca12 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-16T00:30:27.839 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:30:27 smithi112 podman[108545]: 2024-09-16 00:30:27.495770702 +0000 UTC m=+0.480472143 container attach 83368c42b3f6fd39244ec2cb513fccd2b4875942c3a46659cbadfe03329aca12 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-16T00:30:27.840 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:30:27 smithi112 podman[108545]: 2024-09-16 00:30:27.60170383 +0000 UTC m=+0.586405257 container died 83368c42b3f6fd39244ec2cb513fccd2b4875942c3a46659cbadfe03329aca12 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True) 2024-09-16T00:30:28.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:27 smithi169 ceph-mon[77626]: pgmap v39: 65 pgs: 9 stale+active+clean, 56 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-09-16T00:30:28.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:27 smithi169 ceph-mon[77626]: osdmap e49: 6 total, 5 up, 6 in 2024-09-16T00:30:28.402 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:30:28 smithi112 podman[108568]: 2024-09-16 00:30:28.39995024 +0000 UTC m=+0.791440983 container remove 83368c42b3f6fd39244ec2cb513fccd2b4875942c3a46659cbadfe03329aca12 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-16T00:30:28.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:28 smithi112 ceph-mon[97750]: Health check failed: Degraded data redundancy: 71/372 objects degraded (19.086%), 20 pgs degraded (PG_DEGRADED) 2024-09-16T00:30:28.840 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:30:28 smithi112 systemd[1]: ceph-d346dd56-73c0-11ef-bceb-c7b262605968@osd.0.service: Deactivated successfully. 2024-09-16T00:30:28.840 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:30:28 smithi112 systemd[1]: Stopped Ceph osd.0 for d346dd56-73c0-11ef-bceb-c7b262605968. 2024-09-16T00:30:28.840 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:30:28 smithi112 systemd[1]: ceph-d346dd56-73c0-11ef-bceb-c7b262605968@osd.0.service: Consumed 46.616s CPU time. 2024-09-16T00:30:28.840 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:30:28 smithi112 systemd[1]: Starting Ceph osd.0 for d346dd56-73c0-11ef-bceb-c7b262605968... 2024-09-16T00:30:29.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:28 smithi169 ceph-mon[77626]: Health check failed: Degraded data redundancy: 71/372 objects degraded (19.086%), 20 pgs degraded (PG_DEGRADED) 2024-09-16T00:30:29.169 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:30:28 smithi112 podman[108669]: 2024-09-16 00:30:28.749967713 +0000 UTC m=+0.016059171 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:30:29.169 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:30:28 smithi112 podman[108669]: 2024-09-16 00:30:28.875979795 +0000 UTC m=+0.142071244 container create 64cbad693e48aeb0a80f033e589f90e36c591671156db402b38aae01ffcf5b2f (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-0-activate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux ) 2024-09-16T00:30:29.170 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:30:29 smithi112 podman[108669]: 2024-09-16 00:30:29.079153693 +0000 UTC m=+0.345245154 container init 64cbad693e48aeb0a80f033e589f90e36c591671156db402b38aae01ffcf5b2f (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-0-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-16T00:30:29.170 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:30:29 smithi112 podman[108669]: 2024-09-16 00:30:29.083095343 +0000 UTC m=+0.349186799 container start 64cbad693e48aeb0a80f033e589f90e36c591671156db402b38aae01ffcf5b2f (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-0-activate, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-16T00:30:29.170 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:30:29 smithi112 podman[108669]: 2024-09-16 00:30:29.170070687 +0000 UTC m=+0.436162145 container attach 64cbad693e48aeb0a80f033e589f90e36c591671156db402b38aae01ffcf5b2f (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-0-activate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:30:29.589 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:30:29 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-0-activate[108685]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T00:30:29.589 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:30:29 smithi112 bash[108669]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T00:30:29.589 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:30:29 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-0-activate[108685]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T00:30:29.589 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:30:29 smithi112 bash[108669]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T00:30:29.951 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:29 smithi112 ceph-mon[97750]: pgmap v41: 65 pgs: 14 active+undersized, 20 active+undersized+degraded, 31 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 71/372 objects degraded (19.086%) 2024-09-16T00:30:30.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:29 smithi169 ceph-mon[77626]: pgmap v41: 65 pgs: 14 active+undersized, 20 active+undersized+degraded, 31 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 71/372 objects degraded (19.086%) 2024-09-16T00:30:30.239 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:30:29 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-0-activate[108685]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-16T00:30:30.239 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:30:29 smithi112 bash[108669]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-16T00:30:30.239 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:30:29 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-0-activate[108685]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-09-16T00:30:30.239 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:30:29 smithi112 bash[108669]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-09-16T00:30:30.589 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:30:30 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-0-activate[108685]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-09-16T00:30:30.589 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:30:30 smithi112 bash[108669]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-09-16T00:30:30.589 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:30:30 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-0-activate[108685]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-09-16T00:30:30.589 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:30:30 smithi112 bash[108669]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-09-16T00:30:30.589 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:30:30 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-0-activate[108685]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-09-16T00:30:30.590 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:30:30 smithi112 bash[108669]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-09-16T00:30:30.590 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:30:30 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-0-activate[108685]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-16T00:30:30.590 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:30:30 smithi112 bash[108669]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-16T00:30:30.590 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:30:30 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-0-activate[108685]: --> ceph-volume raw activate successful for osd ID: 0 2024-09-16T00:30:30.590 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:30:30 smithi112 bash[108669]: --> ceph-volume raw activate successful for osd ID: 0 2024-09-16T00:30:30.590 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:30:30 smithi112 podman[108669]: 2024-09-16 00:30:30.267976171 +0000 UTC m=+1.534067619 container died 64cbad693e48aeb0a80f033e589f90e36c591671156db402b38aae01ffcf5b2f (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-0-activate, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731) 2024-09-16T00:30:31.227 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:30:30 smithi112 podman[108820]: 2024-09-16 00:30:30.890899478 +0000 UTC m=+0.617268677 container remove 64cbad693e48aeb0a80f033e589f90e36c591671156db402b38aae01ffcf5b2f (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-0-activate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-16T00:30:31.227 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:30:31 smithi112 podman[108878]: 2024-09-16 00:30:31.000469064 +0000 UTC m=+0.016201506 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:30:31.227 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:30:31 smithi112 podman[108878]: 2024-09-16 00:30:31.11701164 +0000 UTC m=+0.132744107 container create 53efc9397017496cfaf270727f80e9d180674ea6c17b00a0dbb179233d014524 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-16T00:30:31.479 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:30:31 smithi112 podman[108878]: 2024-09-16 00:30:31.386219398 +0000 UTC m=+0.401951839 container init 53efc9397017496cfaf270727f80e9d180674ea6c17b00a0dbb179233d014524 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:30:31.479 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:30:31 smithi112 podman[108878]: 2024-09-16 00:30:31.389502792 +0000 UTC m=+0.405235214 container start 53efc9397017496cfaf270727f80e9d180674ea6c17b00a0dbb179233d014524 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux ) 2024-09-16T00:30:31.479 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:30:31 smithi112 ceph-osd[108897]: -- 172.21.15.112:0/2677705897 <== mon.0 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5639bfa381e0 con 0x5639bf9ce000 2024-09-16T00:30:31.479 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:30:31 smithi112 bash[108878]: 53efc9397017496cfaf270727f80e9d180674ea6c17b00a0dbb179233d014524 2024-09-16T00:30:31.839 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:30:31 smithi112 systemd[1]: Started Ceph osd.0 for d346dd56-73c0-11ef-bceb-c7b262605968. 2024-09-16T00:30:31.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:31 smithi112 ceph-mon[97750]: pgmap v42: 65 pgs: 14 active+undersized, 20 active+undersized+degraded, 31 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 71/372 objects degraded (19.086%) 2024-09-16T00:30:31.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:31 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:31.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:31 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:31.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:31 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:30:31.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:31 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:31.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:31 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:30:32.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:31 smithi169 ceph-mon[77626]: pgmap v42: 65 pgs: 14 active+undersized, 20 active+undersized+degraded, 31 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 71/372 objects degraded (19.086%) 2024-09-16T00:30:32.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:31 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:32.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:31 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:32.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:31 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:30:32.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:31 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:32.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:31 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:30:33.589 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:30:33 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-0[108893]: 2024-09-16T00:30:33.262+0000 7fa8b37d9640 -1 Falling back to public interface 2024-09-16T00:30:34.006 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:33 smithi112 ceph-mon[97750]: pgmap v43: 65 pgs: 14 active+undersized, 20 active+undersized+degraded, 31 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 71/372 objects degraded (19.086%) 2024-09-16T00:30:34.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:33 smithi169 ceph-mon[77626]: pgmap v43: 65 pgs: 14 active+undersized, 20 active+undersized+degraded, 31 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 71/372 objects degraded (19.086%) 2024-09-16T00:30:36.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:35 smithi169 ceph-mon[77626]: pgmap v44: 65 pgs: 14 active+undersized, 20 active+undersized+degraded, 31 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 71/372 objects degraded (19.086%) 2024-09-16T00:30:36.030 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:35 smithi112 ceph-mon[97750]: pgmap v44: 65 pgs: 14 active+undersized, 20 active+undersized+degraded, 31 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 71/372 objects degraded (19.086%) 2024-09-16T00:30:37.920 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:37 smithi112 ceph-mon[97750]: pgmap v45: 65 pgs: 14 active+undersized, 20 active+undersized+degraded, 31 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 71/372 objects degraded (19.086%) 2024-09-16T00:30:37.920 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:37 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:37.921 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:37 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:37.921 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:37 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:37.921 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:37 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:38.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:37 smithi169 ceph-mon[77626]: pgmap v45: 65 pgs: 14 active+undersized, 20 active+undersized+degraded, 31 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 71/372 objects degraded (19.086%) 2024-09-16T00:30:38.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:37 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:38.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:37 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:38.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:37 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:38.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:37 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:38.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:38 smithi112 ceph-mon[97750]: from='osd.0 [v2:172.21.15.112:6802/3473756419,v1:172.21.15.112:6803/3473756419]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-16T00:30:38.840 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:30:38 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-0[108893]: 2024-09-16T00:30:38.497+0000 7fa8b37d9640 -1 osd.0 47 log_to_monitors true 2024-09-16T00:30:39.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:38 smithi169 ceph-mon[77626]: from='osd.0 [v2:172.21.15.112:6802/3473756419,v1:172.21.15.112:6803/3473756419]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-16T00:30:39.839 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:30:39 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-0[108893]: 2024-09-16T00:30:39.482+0000 7fa8a756b640 -1 osd.0 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T00:30:39.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:39 smithi112 ceph-mon[97750]: pgmap v46: 65 pgs: 14 active+undersized, 20 active+undersized+degraded, 31 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 71/372 objects degraded (19.086%) 2024-09-16T00:30:39.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:39 smithi112 ceph-mon[97750]: from='osd.0 [v2:172.21.15.112:6802/3473756419,v1:172.21.15.112:6803/3473756419]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-16T00:30:39.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:39 smithi112 ceph-mon[97750]: osdmap e50: 6 total, 5 up, 6 in 2024-09-16T00:30:39.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:39 smithi112 ceph-mon[97750]: from='osd.0 [v2:172.21.15.112:6802/3473756419,v1:172.21.15.112:6803/3473756419]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-09-16T00:30:39.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:39 smithi112 ceph-mon[97750]: from='osd.0 [v2:172.21.15.112:6802/3473756419,v1:172.21.15.112:6803/3473756419]' entity='osd.0' 2024-09-16T00:30:40.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:39 smithi169 ceph-mon[77626]: pgmap v46: 65 pgs: 14 active+undersized, 20 active+undersized+degraded, 31 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 71/372 objects degraded (19.086%) 2024-09-16T00:30:40.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:39 smithi169 ceph-mon[77626]: from='osd.0 [v2:172.21.15.112:6802/3473756419,v1:172.21.15.112:6803/3473756419]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-16T00:30:40.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:39 smithi169 ceph-mon[77626]: osdmap e50: 6 total, 5 up, 6 in 2024-09-16T00:30:40.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:39 smithi169 ceph-mon[77626]: from='osd.0 [v2:172.21.15.112:6802/3473756419,v1:172.21.15.112:6803/3473756419]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-09-16T00:30:40.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:39 smithi169 ceph-mon[77626]: from='osd.0 [v2:172.21.15.112:6802/3473756419,v1:172.21.15.112:6803/3473756419]' entity='osd.0' 2024-09-16T00:30:41.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:40 smithi169 ceph-mon[77626]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-16T00:30:41.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:40 smithi169 ceph-mon[77626]: osd.0 [v2:172.21.15.112:6802/3473756419,v1:172.21.15.112:6803/3473756419] boot 2024-09-16T00:30:41.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:40 smithi169 ceph-mon[77626]: osdmap e51: 6 total, 6 up, 6 in 2024-09-16T00:30:41.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:40 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:30:41.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:40 smithi112 ceph-mon[97750]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-16T00:30:41.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:40 smithi112 ceph-mon[97750]: osd.0 [v2:172.21.15.112:6802/3473756419,v1:172.21.15.112:6803/3473756419] boot 2024-09-16T00:30:41.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:40 smithi112 ceph-mon[97750]: osdmap e51: 6 total, 6 up, 6 in 2024-09-16T00:30:41.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:40 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:30:41.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.379+0000 7f60d6b09640 1 -- 172.21.15.112:0/2650920802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60d0070340 msgr2=0x7f60d0108910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:41.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.379+0000 7f60d6b09640 1 --2- 172.21.15.112:0/2650920802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60d0070340 0x7f60d0108910 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f60bc0048d0 tx=0x7f60bc009da0 comp rx=0 tx=0).stop 2024-09-16T00:30:41.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.379+0000 7f60d6b09640 1 -- 172.21.15.112:0/2650920802 shutdown_connections 2024-09-16T00:30:41.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.379+0000 7f60d6b09640 1 --2- 172.21.15.112:0/2650920802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60d0070340 0x7f60d0108910 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:41.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.379+0000 7f60d6b09640 1 --2- 172.21.15.112:0/2650920802 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60d006fa30 0x7f60d006fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:41.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.379+0000 7f60d6b09640 1 -- 172.21.15.112:0/2650920802 >> 172.21.15.112:0/2650920802 conn(0x7f60d00f80f0 msgr2=0x7f60d00fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:41.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.379+0000 7f60d6b09640 1 -- 172.21.15.112:0/2650920802 shutdown_connections 2024-09-16T00:30:41.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.379+0000 7f60d6b09640 1 -- 172.21.15.112:0/2650920802 wait complete. 2024-09-16T00:30:41.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.380+0000 7f60d6b09640 1 Processor -- start 2024-09-16T00:30:41.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.380+0000 7f60d6b09640 1 -- start start 2024-09-16T00:30:41.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.380+0000 7f60d6b09640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60d006fa30 0x7f60d01157b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:41.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.380+0000 7f60d6b09640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60d0070340 0x7f60d0115cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:41.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.380+0000 7f60d6b09640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d0116340 con 0x7f60d0070340 2024-09-16T00:30:41.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.380+0000 7f60d6b09640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d010f750 con 0x7f60d006fa30 2024-09-16T00:30:41.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.380+0000 7f60cffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60d0070340 0x7f60d0115cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:41.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.380+0000 7f60cffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60d0070340 0x7f60d0115cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44560/0 (socket says 172.21.15.112:44560) 2024-09-16T00:30:41.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.380+0000 7f60cffff640 1 -- 172.21.15.112:0/852695622 learned_addr learned my addr 172.21.15.112:0/852695622 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:30:41.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.381+0000 7f60d48a5640 1 --2- 172.21.15.112:0/852695622 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60d006fa30 0x7f60d01157b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:41.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.381+0000 7f60cffff640 1 -- 172.21.15.112:0/852695622 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60d006fa30 msgr2=0x7f60d01157b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:41.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.381+0000 7f60cffff640 1 --2- 172.21.15.112:0/852695622 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60d006fa30 0x7f60d01157b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:41.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.381+0000 7f60cffff640 1 -- 172.21.15.112:0/852695622 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60bc005040 con 0x7f60d0070340 2024-09-16T00:30:41.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.381+0000 7f60cffff640 1 --2- 172.21.15.112:0/852695622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60d0070340 0x7f60d0115cf0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f60bc000c00 tx=0x7f60bc009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:41.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.381+0000 7f60cd7fa640 1 -- 172.21.15.112:0/852695622 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60bc038070 con 0x7f60d0070340 2024-09-16T00:30:41.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.381+0000 7f60cd7fa640 1 -- 172.21.15.112:0/852695622 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f60bc02ade0 con 0x7f60d0070340 2024-09-16T00:30:41.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.381+0000 7f60d6b09640 1 -- 172.21.15.112:0/852695622 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60d010f9d0 con 0x7f60d0070340 2024-09-16T00:30:41.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.381+0000 7f60cd7fa640 1 -- 172.21.15.112:0/852695622 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60bc02c7e0 con 0x7f60d0070340 2024-09-16T00:30:41.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.381+0000 7f60d6b09640 1 -- 172.21.15.112:0/852695622 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60d010fe90 con 0x7f60d0070340 2024-09-16T00:30:41.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.382+0000 7f60d6b09640 1 -- 172.21.15.112:0/852695622 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60d0071170 con 0x7f60d0070340 2024-09-16T00:30:41.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.383+0000 7f60cd7fa640 1 -- 172.21.15.112:0/852695622 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 38) v1 ==== 92655+0+0 (secure 0 0 0) 0x7f60bc030020 con 0x7f60d0070340 2024-09-16T00:30:41.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.383+0000 7f60cd7fa640 1 --2- 172.21.15.112:0/852695622 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f60c006e660 0x7f60c0070b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:41.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.383+0000 7f60cd7fa640 1 -- 172.21.15.112:0/852695622 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f60bc0adf10 con 0x7f60d0070340 2024-09-16T00:30:41.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.383+0000 7f60d48a5640 1 --2- 172.21.15.112:0/852695622 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f60c006e660 0x7f60c0070b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:41.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.383+0000 7f60d48a5640 1 --2- 172.21.15.112:0/852695622 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f60c006e660 0x7f60c0070b10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f60c4007fd0 tx=0x7f60c4009e90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:41.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.388+0000 7f60cd7fa640 1 -- 172.21.15.112:0/852695622 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f60bc076ff0 con 0x7f60d0070340 2024-09-16T00:30:41.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.588+0000 7f60d6b09640 1 -- 172.21.15.112:0/852695622 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f60d010b780 con 0x7f60c006e660 2024-09-16T00:30:41.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.589+0000 7f60cd7fa640 1 -- 172.21.15.112:0/852695622 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f60d010b780 con 0x7f60c006e660 2024-09-16T00:30:41.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.591+0000 7f60d6b09640 1 -- 172.21.15.112:0/852695622 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f60c006e660 msgr2=0x7f60c0070b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:41.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.591+0000 7f60d6b09640 1 --2- 172.21.15.112:0/852695622 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f60c006e660 0x7f60c0070b10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f60c4007fd0 tx=0x7f60c4009e90 comp rx=0 tx=0).stop 2024-09-16T00:30:41.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.591+0000 7f60d6b09640 1 -- 172.21.15.112:0/852695622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60d0070340 msgr2=0x7f60d0115cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:41.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.591+0000 7f60d6b09640 1 --2- 172.21.15.112:0/852695622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60d0070340 0x7f60d0115cf0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f60bc000c00 tx=0x7f60bc009da0 comp rx=0 tx=0).stop 2024-09-16T00:30:41.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.592+0000 7f60d6b09640 1 -- 172.21.15.112:0/852695622 shutdown_connections 2024-09-16T00:30:41.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.592+0000 7f60d6b09640 1 --2- 172.21.15.112:0/852695622 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f60c006e660 0x7f60c0070b10 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:41.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.592+0000 7f60d6b09640 1 --2- 172.21.15.112:0/852695622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60d0070340 0x7f60d0115cf0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:41.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.592+0000 7f60d6b09640 1 --2- 172.21.15.112:0/852695622 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60d006fa30 0x7f60d01157b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:41.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.592+0000 7f60d6b09640 1 -- 172.21.15.112:0/852695622 >> 172.21.15.112:0/852695622 conn(0x7f60d00f80f0 msgr2=0x7f60d00f9630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:41.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.592+0000 7f60d6b09640 1 -- 172.21.15.112:0/852695622 shutdown_connections 2024-09-16T00:30:41.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.592+0000 7f60d6b09640 1 -- 172.21.15.112:0/852695622 wait complete. 2024-09-16T00:30:41.601 INFO:teuthology.orchestra.run.smithi112.stdout:true 2024-09-16T00:30:41.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.671+0000 7f6af4236640 1 -- 172.21.15.112:0/293443394 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6aec0fac80 msgr2=0x7f6aec0fb0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:41.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.671+0000 7f6af4236640 1 --2- 172.21.15.112:0/293443394 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6aec0fac80 0x7f6aec0fb0d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6adc0048d0 tx=0x7f6adc009da0 comp rx=0 tx=0).stop 2024-09-16T00:30:41.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.671+0000 7f6af4236640 1 -- 172.21.15.112:0/293443394 shutdown_connections 2024-09-16T00:30:41.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.671+0000 7f6af4236640 1 --2- 172.21.15.112:0/293443394 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6aec0fac80 0x7f6aec0fb0d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:41.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.671+0000 7f6af4236640 1 --2- 172.21.15.112:0/293443394 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6aec100c70 0x7f6aec101040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:41.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.671+0000 7f6af4236640 1 -- 172.21.15.112:0/293443394 >> 172.21.15.112:0/293443394 conn(0x7f6aec070ce0 msgr2=0x7f6aec0710e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:41.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.671+0000 7f6af4236640 1 -- 172.21.15.112:0/293443394 shutdown_connections 2024-09-16T00:30:41.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.671+0000 7f6af4236640 1 -- 172.21.15.112:0/293443394 wait complete. 2024-09-16T00:30:41.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.672+0000 7f6af4236640 1 Processor -- start 2024-09-16T00:30:41.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.672+0000 7f6af4236640 1 -- start start 2024-09-16T00:30:41.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.672+0000 7f6af4236640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6aec0fac80 0x7f6aec1131c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:41.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.672+0000 7f6af4236640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6aec100c70 0x7f6aec113700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:41.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.672+0000 7f6af4236640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec113d00 con 0x7f6aec100c70 2024-09-16T00:30:41.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.672+0000 7f6af4236640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec10d140 con 0x7f6aec0fac80 2024-09-16T00:30:41.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.672+0000 7f6af17d1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6aec100c70 0x7f6aec113700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:41.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.672+0000 7f6af17d1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6aec100c70 0x7f6aec113700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44584/0 (socket says 172.21.15.112:44584) 2024-09-16T00:30:41.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.672+0000 7f6af17d1640 1 -- 172.21.15.112:0/3775627234 learned_addr learned my addr 172.21.15.112:0/3775627234 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:30:41.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.672+0000 7f6af1fd2640 1 --2- 172.21.15.112:0/3775627234 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6aec0fac80 0x7f6aec1131c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:41.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.672+0000 7f6af17d1640 1 -- 172.21.15.112:0/3775627234 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6aec0fac80 msgr2=0x7f6aec1131c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:41.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.672+0000 7f6af17d1640 1 --2- 172.21.15.112:0/3775627234 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6aec0fac80 0x7f6aec1131c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:41.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.673+0000 7f6af17d1640 1 -- 172.21.15.112:0/3775627234 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6adc005040 con 0x7f6aec100c70 2024-09-16T00:30:41.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.673+0000 7f6af1fd2640 1 --2- 172.21.15.112:0/3775627234 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6aec0fac80 0x7f6aec1131c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:30:41.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.673+0000 7f6af17d1640 1 --2- 172.21.15.112:0/3775627234 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6aec100c70 0x7f6aec113700 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6adc000c00 tx=0x7f6adc009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:41.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.673+0000 7f6ae27fc640 1 -- 172.21.15.112:0/3775627234 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6adc038070 con 0x7f6aec100c70 2024-09-16T00:30:41.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.673+0000 7f6ae27fc640 1 -- 172.21.15.112:0/3775627234 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f6adc02ade0 con 0x7f6aec100c70 2024-09-16T00:30:41.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.673+0000 7f6af4236640 1 -- 172.21.15.112:0/3775627234 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aec10d3c0 con 0x7f6aec100c70 2024-09-16T00:30:41.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.673+0000 7f6ae27fc640 1 -- 172.21.15.112:0/3775627234 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6adc02c7e0 con 0x7f6aec100c70 2024-09-16T00:30:41.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.673+0000 7f6af4236640 1 -- 172.21.15.112:0/3775627234 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aec10d900 con 0x7f6aec100c70 2024-09-16T00:30:41.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.674+0000 7f6af4236640 1 -- 172.21.15.112:0/3775627234 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6aec0fc370 con 0x7f6aec100c70 2024-09-16T00:30:41.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.674+0000 7f6ae27fc640 1 -- 172.21.15.112:0/3775627234 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 38) v1 ==== 92655+0+0 (secure 0 0 0) 0x7f6adc030020 con 0x7f6aec100c70 2024-09-16T00:30:41.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.675+0000 7f6ae27fc640 1 --2- 172.21.15.112:0/3775627234 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f6ad806e660 0x7f6ad8070b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:41.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.675+0000 7f6ae27fc640 1 -- 172.21.15.112:0/3775627234 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f6adc0adf10 con 0x7f6aec100c70 2024-09-16T00:30:41.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.675+0000 7f6af1fd2640 1 --2- 172.21.15.112:0/3775627234 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f6ad806e660 0x7f6ad8070b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:41.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.675+0000 7f6af1fd2640 1 --2- 172.21.15.112:0/3775627234 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f6ad806e660 0x7f6ad8070b10 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8007fd0 tx=0x7f6ae8009e90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:41.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.679+0000 7f6ae27fc640 1 -- 172.21.15.112:0/3775627234 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f6adc076ff0 con 0x7f6aec100c70 2024-09-16T00:30:41.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:41 smithi112 ceph-mon[97750]: pgmap v49: 65 pgs: 14 active+undersized, 20 active+undersized+degraded, 31 active+clean; 325 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s; 71/372 objects degraded (19.086%) 2024-09-16T00:30:41.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:41 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:41.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:41 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:41.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:41 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:30:41.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:41 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:30:41.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:41 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:41.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:41 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:30:41.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:41 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:30:41.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:41 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:30:41.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:41 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:30:41.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:41 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-09-16T00:30:41.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:41 smithi112 ceph-mon[97750]: osdmap e52: 6 total, 6 up, 6 in 2024-09-16T00:30:41.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.865+0000 7f6af4236640 1 -- 172.21.15.112:0/3775627234 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6aec108490 con 0x7f6ad806e660 2024-09-16T00:30:41.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.867+0000 7f6ae27fc640 1 -- 172.21.15.112:0/3775627234 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f6aec108490 con 0x7f6ad806e660 2024-09-16T00:30:41.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.869+0000 7f6af4236640 1 -- 172.21.15.112:0/3775627234 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f6ad806e660 msgr2=0x7f6ad8070b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:41.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.869+0000 7f6af4236640 1 --2- 172.21.15.112:0/3775627234 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f6ad806e660 0x7f6ad8070b10 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8007fd0 tx=0x7f6ae8009e90 comp rx=0 tx=0).stop 2024-09-16T00:30:41.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.869+0000 7f6af4236640 1 -- 172.21.15.112:0/3775627234 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6aec100c70 msgr2=0x7f6aec113700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:41.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.869+0000 7f6af4236640 1 --2- 172.21.15.112:0/3775627234 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6aec100c70 0x7f6aec113700 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6adc000c00 tx=0x7f6adc009da0 comp rx=0 tx=0).stop 2024-09-16T00:30:41.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.869+0000 7f6af4236640 1 -- 172.21.15.112:0/3775627234 shutdown_connections 2024-09-16T00:30:41.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.869+0000 7f6af4236640 1 --2- 172.21.15.112:0/3775627234 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f6ad806e660 0x7f6ad8070b10 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:41.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.869+0000 7f6af4236640 1 --2- 172.21.15.112:0/3775627234 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6aec100c70 0x7f6aec113700 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:41.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.869+0000 7f6af4236640 1 --2- 172.21.15.112:0/3775627234 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6aec0fac80 0x7f6aec1131c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:41.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.869+0000 7f6af4236640 1 -- 172.21.15.112:0/3775627234 >> 172.21.15.112:0/3775627234 conn(0x7f6aec070ce0 msgr2=0x7f6aec0fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:41.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.869+0000 7f6af4236640 1 -- 172.21.15.112:0/3775627234 shutdown_connections 2024-09-16T00:30:41.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.869+0000 7f6af4236640 1 -- 172.21.15.112:0/3775627234 wait complete. 2024-09-16T00:30:41.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.947+0000 7f70b9491640 1 -- 172.21.15.112:0/2782855661 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70b41014b0 msgr2=0x7f70b4101880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:41.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.947+0000 7f70b9491640 1 --2- 172.21.15.112:0/2782855661 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70b41014b0 0x7f70b4101880 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f70a4009c50 tx=0x7f70a4004d10 comp rx=0 tx=0).stop 2024-09-16T00:30:41.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.947+0000 7f70b9491640 1 -- 172.21.15.112:0/2782855661 shutdown_connections 2024-09-16T00:30:41.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.947+0000 7f70b9491640 1 --2- 172.21.15.112:0/2782855661 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70b40fb470 0x7f70b40fb8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:41.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.947+0000 7f70b9491640 1 --2- 172.21.15.112:0/2782855661 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70b41014b0 0x7f70b4101880 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:41.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.947+0000 7f70b9491640 1 -- 172.21.15.112:0/2782855661 >> 172.21.15.112:0/2782855661 conn(0x7f70b40f7030 msgr2=0x7f70b40f9440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:41.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.948+0000 7f70b9491640 1 -- 172.21.15.112:0/2782855661 shutdown_connections 2024-09-16T00:30:41.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.948+0000 7f70b9491640 1 -- 172.21.15.112:0/2782855661 wait complete. 2024-09-16T00:30:41.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.948+0000 7f70b9491640 1 Processor -- start 2024-09-16T00:30:41.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.948+0000 7f70b9491640 1 -- start start 2024-09-16T00:30:41.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.948+0000 7f70b9491640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70b40fb470 0x7f70b410e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:41.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.948+0000 7f70b9491640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70b41014b0 0x7f70b410e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:41.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.948+0000 7f70b9491640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b410ecf0 con 0x7f70b40fb470 2024-09-16T00:30:41.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.948+0000 7f70b9491640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b410ee60 con 0x7f70b41014b0 2024-09-16T00:30:41.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.949+0000 7f70b2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70b40fb470 0x7f70b410e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:41.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.949+0000 7f70b2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70b40fb470 0x7f70b410e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44598/0 (socket says 172.21.15.112:44598) 2024-09-16T00:30:41.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.949+0000 7f70b2ffd640 1 -- 172.21.15.112:0/2759330293 learned_addr learned my addr 172.21.15.112:0/2759330293 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:30:41.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.949+0000 7f70b27fc640 1 --2- 172.21.15.112:0/2759330293 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70b41014b0 0x7f70b410e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:41.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.949+0000 7f70b2ffd640 1 -- 172.21.15.112:0/2759330293 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70b41014b0 msgr2=0x7f70b410e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:41.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.949+0000 7f70b2ffd640 1 --2- 172.21.15.112:0/2759330293 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70b41014b0 0x7f70b410e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:41.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.949+0000 7f70b2ffd640 1 -- 172.21.15.112:0/2759330293 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70a40098b0 con 0x7f70b40fb470 2024-09-16T00:30:41.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.949+0000 7f70b2ffd640 1 --2- 172.21.15.112:0/2759330293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70b40fb470 0x7f70b410e160 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f70a4004a30 tx=0x7f70a4004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:41.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.949+0000 7f709b7fe640 1 -- 172.21.15.112:0/2759330293 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70a4009690 con 0x7f70b40fb470 2024-09-16T00:30:41.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.949+0000 7f709b7fe640 1 -- 172.21.15.112:0/2759330293 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f70a4032b10 con 0x7f70b40fb470 2024-09-16T00:30:41.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.949+0000 7f709b7fe640 1 -- 172.21.15.112:0/2759330293 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70a403daa0 con 0x7f70b40fb470 2024-09-16T00:30:41.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.949+0000 7f70b9491640 1 -- 172.21.15.112:0/2759330293 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70b41171d0 con 0x7f70b40fb470 2024-09-16T00:30:41.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.949+0000 7f70b9491640 1 -- 172.21.15.112:0/2759330293 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70b4117690 con 0x7f70b40fb470 2024-09-16T00:30:41.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.950+0000 7f70b9491640 1 -- 172.21.15.112:0/2759330293 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70b40fcbb0 con 0x7f70b40fb470 2024-09-16T00:30:41.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.951+0000 7f709b7fe640 1 -- 172.21.15.112:0/2759330293 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 38) v1 ==== 92655+0+0 (secure 0 0 0) 0x7f70a402f030 con 0x7f70b40fb470 2024-09-16T00:30:41.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.951+0000 7f709b7fe640 1 --2- 172.21.15.112:0/2759330293 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f709c06e660 0x7f709c070b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:41.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.951+0000 7f709b7fe640 1 -- 172.21.15.112:0/2759330293 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f70a40af380 con 0x7f70b40fb470 2024-09-16T00:30:41.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.951+0000 7f70b27fc640 1 --2- 172.21.15.112:0/2759330293 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f709c06e660 0x7f709c070b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:41.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.952+0000 7f70b27fc640 1 --2- 172.21.15.112:0/2759330293 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f709c06e660 0x7f709c070b10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f70b4066210 tx=0x7f70a8009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:41.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:41.956+0000 7f709b7fe640 1 -- 172.21.15.112:0/2759330293 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f70a40783d0 con 0x7f70b40fb470 2024-09-16T00:30:42.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:41 smithi169 ceph-mon[77626]: pgmap v49: 65 pgs: 14 active+undersized, 20 active+undersized+degraded, 31 active+clean; 325 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s; 71/372 objects degraded (19.086%) 2024-09-16T00:30:42.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:41 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:42.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:41 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:42.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:41 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:30:42.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:41 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:30:42.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:41 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:42.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:41 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:30:42.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:41 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:30:42.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:41 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:30:42.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:41 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:30:42.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:41 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-09-16T00:30:42.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:41 smithi169 ceph-mon[77626]: osdmap e52: 6 total, 6 up, 6 in 2024-09-16T00:30:42.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.128+0000 7f70b9491640 1 -- 172.21.15.112:0/2759330293 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f70b4065fc0 con 0x7f709c06e660 2024-09-16T00:30:42.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.137+0000 7f709b7fe640 1 -- 172.21.15.112:0/2759330293 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f70b4065fc0 con 0x7f709c06e660 2024-09-16T00:30:42.138 INFO:teuthology.orchestra.run.smithi112.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T00:30:42.138 INFO:teuthology.orchestra.run.smithi112.stdout:alertmanager.smithi112 smithi112 *:9093,9094 running (3m) 5s ago 12m 26.6M - 0.25.0 c8568f914cd2 d85ea0562022 2024-09-16T00:30:42.138 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi112 smithi112 running (36s) 5s ago 12m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c e4fe58f9702a 2024-09-16T00:30:42.138 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi169 smithi169 running (31s) 28s ago 11m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c 3ba2c81d527f 2024-09-16T00:30:42.138 INFO:teuthology.orchestra.run.smithi112.stdout:grafana.smithi112 smithi112 *:3000 running (2m) 5s ago 12m 76.7M - 9.4.12 2bacad6d85d8 51f9d2013da2 2024-09-16T00:30:42.138 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.hxfwkh smithi112 running (7m) 5s ago 7m 135M - 17.2.7-1633-g6d025bc2 33ed8dba3996 cf92d9461fc6 2024-09-16T00:30:42.139 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.ithwzj smithi112 running (7m) 5s ago 7m 66.1M - 17.2.7-1633-g6d025bc2 33ed8dba3996 02786c8a83c7 2024-09-16T00:30:42.139 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.gawwgw smithi169 running (7m) 28s ago 7m 114M - 17.2.7-1633-g6d025bc2 33ed8dba3996 dcf3a4dfa879 2024-09-16T00:30:42.139 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.lgeqrj smithi169 running (7m) 28s ago 7m 60.4M - 17.2.7-1633-g6d025bc2 33ed8dba3996 445c14e55831 2024-09-16T00:30:42.139 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi112.xrkkbc smithi112 *:8443,9283,8765 running (3m) 5s ago 13m 494M - 19.1.1-326-g8921c8ef f1455bd0219c a981f7fd2810 2024-09-16T00:30:42.139 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi169.mtrobk smithi169 *:8443,9283,8765 running (2m) 28s ago 11m 439M - 19.1.1-326-g8921c8ef f1455bd0219c 4096e5369c2d 2024-09-16T00:30:42.139 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi112 smithi112 running (92s) 5s ago 13m 42.2M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 5aaec033c042 2024-09-16T00:30:42.139 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi169 smithi169 running (70s) 28s ago 11m 31.8M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 7bc609155f3f 2024-09-16T00:30:42.139 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi112 smithi112 *:9100 running (4m) 5s ago 12m 15.7M - 1.5.0 0da6a335fe13 6404d1d601e2 2024-09-16T00:30:42.139 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi169 smithi169 *:9100 running (3m) 28s ago 11m 15.2M - 1.5.0 0da6a335fe13 5505baf702a6 2024-09-16T00:30:42.139 INFO:teuthology.orchestra.run.smithi112.stdout:osd.0 smithi112 running (11s) 5s ago 10m 14.4M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 53efc9397017 2024-09-16T00:30:42.139 INFO:teuthology.orchestra.run.smithi112.stdout:osd.1 smithi112 running (10m) 5s ago 10m 202M 1557M 17.2.7-1633-g6d025bc2 33ed8dba3996 e9bee87bba0f 2024-09-16T00:30:42.139 INFO:teuthology.orchestra.run.smithi112.stdout:osd.2 smithi112 running (9m) 5s ago 9m 160M 1557M 17.2.7-1633-g6d025bc2 33ed8dba3996 100ec4e4852a 2024-09-16T00:30:42.139 INFO:teuthology.orchestra.run.smithi112.stdout:osd.3 smithi169 running (9m) 28s ago 9m 231M 2581M 17.2.7-1633-g6d025bc2 33ed8dba3996 f88463e19f23 2024-09-16T00:30:42.139 INFO:teuthology.orchestra.run.smithi112.stdout:osd.4 smithi169 running (9m) 28s ago 9m 211M 2581M 17.2.7-1633-g6d025bc2 33ed8dba3996 d3e4cdb016b9 2024-09-16T00:30:42.140 INFO:teuthology.orchestra.run.smithi112.stdout:osd.5 smithi169 running (8m) 28s ago 8m 199M 2581M 17.2.7-1633-g6d025bc2 33ed8dba3996 8e7d708d74eb 2024-09-16T00:30:42.140 INFO:teuthology.orchestra.run.smithi112.stdout:prometheus.smithi112 smithi112 *:9095 running (2m) 5s ago 12m 62.4M - 2.43.0 a07b618ecd1d 7a1d42e60d2b 2024-09-16T00:30:42.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.140+0000 7f70b9491640 1 -- 172.21.15.112:0/2759330293 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f709c06e660 msgr2=0x7f709c070b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:42.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.140+0000 7f70b9491640 1 --2- 172.21.15.112:0/2759330293 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f709c06e660 0x7f709c070b10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f70b4066210 tx=0x7f70a8009040 comp rx=0 tx=0).stop 2024-09-16T00:30:42.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.140+0000 7f70b9491640 1 -- 172.21.15.112:0/2759330293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70b40fb470 msgr2=0x7f70b410e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:42.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.140+0000 7f70b9491640 1 --2- 172.21.15.112:0/2759330293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70b40fb470 0x7f70b410e160 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f70a4004a30 tx=0x7f70a4004d10 comp rx=0 tx=0).stop 2024-09-16T00:30:42.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.140+0000 7f70b9491640 1 -- 172.21.15.112:0/2759330293 shutdown_connections 2024-09-16T00:30:42.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.140+0000 7f70b9491640 1 --2- 172.21.15.112:0/2759330293 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f709c06e660 0x7f709c070b10 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:42.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.140+0000 7f70b9491640 1 --2- 172.21.15.112:0/2759330293 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70b41014b0 0x7f70b410e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:42.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.140+0000 7f70b9491640 1 --2- 172.21.15.112:0/2759330293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70b40fb470 0x7f70b410e160 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:42.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.140+0000 7f70b9491640 1 -- 172.21.15.112:0/2759330293 >> 172.21.15.112:0/2759330293 conn(0x7f70b40f7030 msgr2=0x7f70b40f88e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:42.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.140+0000 7f70b9491640 1 -- 172.21.15.112:0/2759330293 shutdown_connections 2024-09-16T00:30:42.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.140+0000 7f70b9491640 1 -- 172.21.15.112:0/2759330293 wait complete. 2024-09-16T00:30:42.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.217+0000 7fc9f335c640 1 -- 172.21.15.112:0/2032644545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9ec1014c0 msgr2=0x7fc9ec101890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:42.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.217+0000 7fc9f335c640 1 --2- 172.21.15.112:0/2032644545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9ec1014c0 0x7fc9ec101890 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc9e80048d0 tx=0x7fc9e8009da0 comp rx=0 tx=0).stop 2024-09-16T00:30:42.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.217+0000 7fc9f335c640 1 -- 172.21.15.112:0/2032644545 shutdown_connections 2024-09-16T00:30:42.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.217+0000 7fc9f335c640 1 --2- 172.21.15.112:0/2032644545 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9ec0fb480 0x7fc9ec0fb8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:42.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.217+0000 7fc9f335c640 1 --2- 172.21.15.112:0/2032644545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9ec1014c0 0x7fc9ec101890 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:42.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.217+0000 7fc9f335c640 1 -- 172.21.15.112:0/2032644545 >> 172.21.15.112:0/2032644545 conn(0x7fc9ec0f7000 msgr2=0x7fc9ec0f9410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:42.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.217+0000 7fc9f335c640 1 -- 172.21.15.112:0/2032644545 shutdown_connections 2024-09-16T00:30:42.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.217+0000 7fc9f335c640 1 -- 172.21.15.112:0/2032644545 wait complete. 2024-09-16T00:30:42.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.218+0000 7fc9f335c640 1 Processor -- start 2024-09-16T00:30:42.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.218+0000 7fc9f335c640 1 -- start start 2024-09-16T00:30:42.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.218+0000 7fc9f335c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9ec0fb480 0x7fc9ec1131e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:42.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.218+0000 7fc9f335c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9ec1014c0 0x7fc9ec113720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:42.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.218+0000 7fc9f335c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ec113d20 con 0x7fc9ec0fb480 2024-09-16T00:30:42.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.218+0000 7fc9f335c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ec10d160 con 0x7fc9ec1014c0 2024-09-16T00:30:42.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.218+0000 7fc9f10f8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9ec0fb480 0x7fc9ec1131e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:42.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.218+0000 7fc9f10f8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9ec0fb480 0x7fc9ec1131e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32768/0 (socket says 172.21.15.112:32768) 2024-09-16T00:30:42.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.218+0000 7fc9f10f8640 1 -- 172.21.15.112:0/3896232268 learned_addr learned my addr 172.21.15.112:0/3896232268 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:30:42.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.218+0000 7fc9f08f7640 1 --2- 172.21.15.112:0/3896232268 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9ec1014c0 0x7fc9ec113720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:42.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.218+0000 7fc9f10f8640 1 -- 172.21.15.112:0/3896232268 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9ec1014c0 msgr2=0x7fc9ec113720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:42.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.218+0000 7fc9f10f8640 1 --2- 172.21.15.112:0/3896232268 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9ec1014c0 0x7fc9ec113720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:42.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.218+0000 7fc9f10f8640 1 -- 172.21.15.112:0/3896232268 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9e8005040 con 0x7fc9ec0fb480 2024-09-16T00:30:42.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.219+0000 7fc9f10f8640 1 --2- 172.21.15.112:0/3896232268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9ec0fb480 0x7fc9ec1131e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc9e8000c00 tx=0x7fc9e8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:42.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.219+0000 7fc9e1ffb640 1 -- 172.21.15.112:0/3896232268 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc9e8038070 con 0x7fc9ec0fb480 2024-09-16T00:30:42.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.219+0000 7fc9e1ffb640 1 -- 172.21.15.112:0/3896232268 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fc9e802ade0 con 0x7fc9ec0fb480 2024-09-16T00:30:42.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.219+0000 7fc9f335c640 1 -- 172.21.15.112:0/3896232268 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9ec10d3e0 con 0x7fc9ec0fb480 2024-09-16T00:30:42.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.219+0000 7fc9e1ffb640 1 -- 172.21.15.112:0/3896232268 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc9e802c7b0 con 0x7fc9ec0fb480 2024-09-16T00:30:42.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.219+0000 7fc9f335c640 1 -- 172.21.15.112:0/3896232268 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9ec10d920 con 0x7fc9ec0fb480 2024-09-16T00:30:42.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.220+0000 7fc9f335c640 1 -- 172.21.15.112:0/3896232268 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9ec0fcbc0 con 0x7fc9ec0fb480 2024-09-16T00:30:42.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.221+0000 7fc9e1ffb640 1 -- 172.21.15.112:0/3896232268 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 38) v1 ==== 92655+0+0 (secure 0 0 0) 0x7fc9e8030020 con 0x7fc9ec0fb480 2024-09-16T00:30:42.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.221+0000 7fc9e1ffb640 1 --2- 172.21.15.112:0/3896232268 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fc9d806e660 0x7fc9d8070b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:42.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.221+0000 7fc9e1ffb640 1 -- 172.21.15.112:0/3896232268 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fc9e80ae790 con 0x7fc9ec0fb480 2024-09-16T00:30:42.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.221+0000 7fc9f08f7640 1 --2- 172.21.15.112:0/3896232268 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fc9d806e660 0x7fc9d8070b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:42.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.222+0000 7fc9f08f7640 1 --2- 172.21.15.112:0/3896232268 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fc9d806e660 0x7fc9d8070b10 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec10eaa0 tx=0x7fc9dc00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:42.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.225+0000 7fc9e1ffb640 1 -- 172.21.15.112:0/3896232268 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fc9e80777c0 con 0x7fc9ec0fb480 2024-09-16T00:30:42.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.456+0000 7fc9f335c640 1 -- 172.21.15.112:0/3896232268 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc9ec0fac40 con 0x7fc9ec0fb480 2024-09-16T00:30:42.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.457+0000 7fc9e1ffb640 1 -- 172.21.15.112:0/3896232268 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+850 (secure 0 0 0) 0x7fc9e807ae80 con 0x7fc9ec0fb480 2024-09-16T00:30:42.457 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:30:42.458 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": { 2024-09-16T00:30:42.458 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:30:42.458 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:30:42.458 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": { 2024-09-16T00:30:42.458 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:30:42.458 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:30:42.458 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": { 2024-09-16T00:30:42.458 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 17.2.7-1633-g6d025bc2 (6d025bc26f3176c910707f8cb4c8d8e076640615) quincy (stable)": 5, 2024-09-16T00:30:42.458 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 1 2024-09-16T00:30:42.458 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:30:42.458 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": { 2024-09-16T00:30:42.458 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 17.2.7-1633-g6d025bc2 (6d025bc26f3176c910707f8cb4c8d8e076640615) quincy (stable)": 4 2024-09-16T00:30:42.458 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:30:42.459 INFO:teuthology.orchestra.run.smithi112.stdout: "overall": { 2024-09-16T00:30:42.459 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 17.2.7-1633-g6d025bc2 (6d025bc26f3176c910707f8cb4c8d8e076640615) quincy (stable)": 9, 2024-09-16T00:30:42.459 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 5 2024-09-16T00:30:42.459 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-16T00:30:42.459 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:30:42.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.459+0000 7fc9f335c640 1 -- 172.21.15.112:0/3896232268 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fc9d806e660 msgr2=0x7fc9d8070b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:42.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.459+0000 7fc9f335c640 1 --2- 172.21.15.112:0/3896232268 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fc9d806e660 0x7fc9d8070b10 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec10eaa0 tx=0x7fc9dc00b040 comp rx=0 tx=0).stop 2024-09-16T00:30:42.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.459+0000 7fc9f335c640 1 -- 172.21.15.112:0/3896232268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9ec0fb480 msgr2=0x7fc9ec1131e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:42.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.459+0000 7fc9f335c640 1 --2- 172.21.15.112:0/3896232268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9ec0fb480 0x7fc9ec1131e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc9e8000c00 tx=0x7fc9e8009da0 comp rx=0 tx=0).stop 2024-09-16T00:30:42.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.459+0000 7fc9f335c640 1 -- 172.21.15.112:0/3896232268 shutdown_connections 2024-09-16T00:30:42.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.459+0000 7fc9f335c640 1 --2- 172.21.15.112:0/3896232268 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fc9d806e660 0x7fc9d8070b10 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:42.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.459+0000 7fc9f335c640 1 --2- 172.21.15.112:0/3896232268 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9ec1014c0 0x7fc9ec113720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:42.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.460+0000 7fc9f335c640 1 --2- 172.21.15.112:0/3896232268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9ec0fb480 0x7fc9ec1131e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:42.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.460+0000 7fc9f335c640 1 -- 172.21.15.112:0/3896232268 >> 172.21.15.112:0/3896232268 conn(0x7fc9ec0f7000 msgr2=0x7fc9ec065a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:42.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.460+0000 7fc9f335c640 1 -- 172.21.15.112:0/3896232268 shutdown_connections 2024-09-16T00:30:42.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.460+0000 7fc9f335c640 1 -- 172.21.15.112:0/3896232268 wait complete. 2024-09-16T00:30:42.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.539+0000 7f29ec78a640 1 -- 172.21.15.112:0/2454451970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29e4102ed0 msgr2=0x7f29e40fc590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:42.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.539+0000 7f29ec78a640 1 --2- 172.21.15.112:0/2454451970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29e4102ed0 0x7f29e40fc590 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f29e00048d0 tx=0x7f29e0009da0 comp rx=0 tx=0).stop 2024-09-16T00:30:42.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.539+0000 7f29ec78a640 1 -- 172.21.15.112:0/2454451970 shutdown_connections 2024-09-16T00:30:42.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.539+0000 7f29ec78a640 1 --2- 172.21.15.112:0/2454451970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29e4102ed0 0x7f29e40fc590 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:42.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.539+0000 7f29ec78a640 1 --2- 172.21.15.112:0/2454451970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29e4102530 0x7f29e4102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:42.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.539+0000 7f29ec78a640 1 -- 172.21.15.112:0/2454451970 >> 172.21.15.112:0/2454451970 conn(0x7f29e40f8110 msgr2=0x7f29e40fa520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:42.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.539+0000 7f29ec78a640 1 -- 172.21.15.112:0/2454451970 shutdown_connections 2024-09-16T00:30:42.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.539+0000 7f29ec78a640 1 -- 172.21.15.112:0/2454451970 wait complete. 2024-09-16T00:30:42.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.539+0000 7f29ec78a640 1 Processor -- start 2024-09-16T00:30:42.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.540+0000 7f29ec78a640 1 -- start start 2024-09-16T00:30:42.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.540+0000 7f29ec78a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29e4102530 0x7f29e406fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:42.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.540+0000 7f29ec78a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29e4102ed0 0x7f29e406ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:42.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.540+0000 7f29ec78a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e4070520 con 0x7f29e4102ed0 2024-09-16T00:30:42.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.540+0000 7f29ec78a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e4070690 con 0x7f29e4102530 2024-09-16T00:30:42.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.540+0000 7f29e9d25640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29e4102ed0 0x7f29e406ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:42.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.540+0000 7f29e9d25640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29e4102ed0 0x7f29e406ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32776/0 (socket says 172.21.15.112:32776) 2024-09-16T00:30:42.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.540+0000 7f29e9d25640 1 -- 172.21.15.112:0/2202479675 learned_addr learned my addr 172.21.15.112:0/2202479675 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:30:42.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.540+0000 7f29ea526640 1 --2- 172.21.15.112:0/2202479675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29e4102530 0x7f29e406fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:42.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.540+0000 7f29e9d25640 1 -- 172.21.15.112:0/2202479675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29e4102530 msgr2=0x7f29e406fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:42.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.540+0000 7f29e9d25640 1 --2- 172.21.15.112:0/2202479675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29e4102530 0x7f29e406fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:42.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.540+0000 7f29e9d25640 1 -- 172.21.15.112:0/2202479675 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29e0005040 con 0x7f29e4102ed0 2024-09-16T00:30:42.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.540+0000 7f29ea526640 1 --2- 172.21.15.112:0/2202479675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29e4102530 0x7f29e406fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:30:42.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.540+0000 7f29e9d25640 1 --2- 172.21.15.112:0/2202479675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29e4102ed0 0x7f29e406ff50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f29e0003080 tx=0x7f29e0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:42.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.541+0000 7f29daffd640 1 -- 172.21.15.112:0/2202479675 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f29e0038070 con 0x7f29e4102ed0 2024-09-16T00:30:42.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.541+0000 7f29daffd640 1 -- 172.21.15.112:0/2202479675 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f29e0009760 con 0x7f29e4102ed0 2024-09-16T00:30:42.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.541+0000 7f29ec78a640 1 -- 172.21.15.112:0/2202479675 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29e4119590 con 0x7f29e4102ed0 2024-09-16T00:30:42.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.541+0000 7f29daffd640 1 -- 172.21.15.112:0/2202479675 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f29e0033420 con 0x7f29e4102ed0 2024-09-16T00:30:42.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.541+0000 7f29ec78a640 1 -- 172.21.15.112:0/2202479675 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29e41198c0 con 0x7f29e4102ed0 2024-09-16T00:30:42.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.542+0000 7f29ec78a640 1 -- 172.21.15.112:0/2202479675 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29e40fdcb0 con 0x7f29e4102ed0 2024-09-16T00:30:42.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.542+0000 7f29daffd640 1 -- 172.21.15.112:0/2202479675 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 38) v1 ==== 92655+0+0 (secure 0 0 0) 0x7f29e0030020 con 0x7f29e4102ed0 2024-09-16T00:30:42.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.543+0000 7f29daffd640 1 --2- 172.21.15.112:0/2202479675 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f29d006e660 0x7f29d0070b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:42.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.543+0000 7f29ea526640 1 --2- 172.21.15.112:0/2202479675 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f29d006e660 0x7f29d0070b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:42.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.543+0000 7f29daffd640 1 -- 172.21.15.112:0/2202479675 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f29e00ae020 con 0x7f29e4102ed0 2024-09-16T00:30:42.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.543+0000 7f29ea526640 1 --2- 172.21.15.112:0/2202479675 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f29d006e660 0x7f29d0070b10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f29d4006fd0 tx=0x7f29d4008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:42.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.547+0000 7f29daffd640 1 -- 172.21.15.112:0/2202479675 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f29e0077050 con 0x7f29e4102ed0 2024-09-16T00:30:42.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.755+0000 7f29ec78a640 1 -- 172.21.15.112:0/2202479675 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f29e4071100 con 0x7f29e4102ed0 2024-09-16T00:30:42.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.756+0000 7f29daffd640 1 -- 172.21.15.112:0/2202479675 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) v1 ==== 76+0+1954 (secure 0 0 0) 0x7f29e007a710 con 0x7f29e4102ed0 2024-09-16T00:30:42.757 INFO:teuthology.orchestra.run.smithi112.stdout:e23 2024-09-16T00:30:42.757 INFO:teuthology.orchestra.run.smithi112.stdout:btime 2024-09-16T00:30:08:788506+0000 2024-09-16T00:30:42.757 INFO:teuthology.orchestra.run.smithi112.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T00:30:42.757 INFO:teuthology.orchestra.run.smithi112.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:30:42.757 INFO:teuthology.orchestra.run.smithi112.stdout:legacy client fscid: 1 2024-09-16T00:30:42.757 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:30:42.757 INFO:teuthology.orchestra.run.smithi112.stdout:Filesystem 'cephfs' (1) 2024-09-16T00:30:42.757 INFO:teuthology.orchestra.run.smithi112.stdout:fs_name cephfs 2024-09-16T00:30:42.757 INFO:teuthology.orchestra.run.smithi112.stdout:epoch 23 2024-09-16T00:30:42.757 INFO:teuthology.orchestra.run.smithi112.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-16T00:30:42.757 INFO:teuthology.orchestra.run.smithi112.stdout:created 2024-09-16T00:23:22.328199+0000 2024-09-16T00:30:42.757 INFO:teuthology.orchestra.run.smithi112.stdout:modified 2024-09-16T00:30:08.396661+0000 2024-09-16T00:30:42.758 INFO:teuthology.orchestra.run.smithi112.stdout:tableserver 0 2024-09-16T00:30:42.758 INFO:teuthology.orchestra.run.smithi112.stdout:root 0 2024-09-16T00:30:42.758 INFO:teuthology.orchestra.run.smithi112.stdout:session_timeout 60 2024-09-16T00:30:42.758 INFO:teuthology.orchestra.run.smithi112.stdout:session_autoclose 300 2024-09-16T00:30:42.758 INFO:teuthology.orchestra.run.smithi112.stdout:max_file_size 1099511627776 2024-09-16T00:30:42.758 INFO:teuthology.orchestra.run.smithi112.stdout:max_xattr_size 65536 2024-09-16T00:30:42.758 INFO:teuthology.orchestra.run.smithi112.stdout:required_client_features {} 2024-09-16T00:30:42.758 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure 0 2024-09-16T00:30:42.758 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure_osd_epoch 44 2024-09-16T00:30:42.758 INFO:teuthology.orchestra.run.smithi112.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:30:42.758 INFO:teuthology.orchestra.run.smithi112.stdout:max_mds 2 2024-09-16T00:30:42.758 INFO:teuthology.orchestra.run.smithi112.stdout:in 0,1 2024-09-16T00:30:42.758 INFO:teuthology.orchestra.run.smithi112.stdout:up {0=14480,1=24259} 2024-09-16T00:30:42.758 INFO:teuthology.orchestra.run.smithi112.stdout:failed 2024-09-16T00:30:42.759 INFO:teuthology.orchestra.run.smithi112.stdout:damaged 2024-09-16T00:30:42.759 INFO:teuthology.orchestra.run.smithi112.stdout:stopped 2024-09-16T00:30:42.759 INFO:teuthology.orchestra.run.smithi112.stdout:data_pools [3] 2024-09-16T00:30:42.759 INFO:teuthology.orchestra.run.smithi112.stdout:metadata_pool 2 2024-09-16T00:30:42.759 INFO:teuthology.orchestra.run.smithi112.stdout:inline_data enabled 2024-09-16T00:30:42.759 INFO:teuthology.orchestra.run.smithi112.stdout:balancer 2024-09-16T00:30:42.759 INFO:teuthology.orchestra.run.smithi112.stdout:bal_rank_mask -1 2024-09-16T00:30:42.759 INFO:teuthology.orchestra.run.smithi112.stdout:standby_count_wanted 1 2024-09-16T00:30:42.759 INFO:teuthology.orchestra.run.smithi112.stdout:qdb_cluster leader: 14480 members: 24259,14480 2024-09-16T00:30:42.759 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.hxfwkh{0:14480} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.112:6828/486654885,v1:172.21.15.112:6829/486654885] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T00:30:42.759 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.ithwzj{0:14492} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.112:6826/381830565,v1:172.21.15.112:6827/381830565] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T00:30:42.759 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.gawwgw{1:24259} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.169:6824/2892476953,v1:172.21.15.169:6825/2892476953] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T00:30:42.759 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.lgeqrj{1:24269} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.169:6826/861725728,v1:172.21.15.169:6827/861725728] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T00:30:42.759 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:30:42.759 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:30:42.760 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 23 2024-09-16T00:30:42.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.758+0000 7f29ec78a640 1 -- 172.21.15.112:0/2202479675 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f29d006e660 msgr2=0x7f29d0070b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:42.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.758+0000 7f29ec78a640 1 --2- 172.21.15.112:0/2202479675 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f29d006e660 0x7f29d0070b10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f29d4006fd0 tx=0x7f29d4008040 comp rx=0 tx=0).stop 2024-09-16T00:30:42.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.758+0000 7f29ec78a640 1 -- 172.21.15.112:0/2202479675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29e4102ed0 msgr2=0x7f29e406ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:42.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.758+0000 7f29ec78a640 1 --2- 172.21.15.112:0/2202479675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29e4102ed0 0x7f29e406ff50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f29e0003080 tx=0x7f29e0009da0 comp rx=0 tx=0).stop 2024-09-16T00:30:42.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.758+0000 7f29ec78a640 1 -- 172.21.15.112:0/2202479675 shutdown_connections 2024-09-16T00:30:42.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.758+0000 7f29ec78a640 1 --2- 172.21.15.112:0/2202479675 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f29d006e660 0x7f29d0070b10 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:42.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.758+0000 7f29ec78a640 1 --2- 172.21.15.112:0/2202479675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29e4102ed0 0x7f29e406ff50 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:42.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.758+0000 7f29ec78a640 1 --2- 172.21.15.112:0/2202479675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29e4102530 0x7f29e406fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:42.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.758+0000 7f29ec78a640 1 -- 172.21.15.112:0/2202479675 >> 172.21.15.112:0/2202479675 conn(0x7f29e40f8110 msgr2=0x7f29e40f91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:42.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.758+0000 7f29ec78a640 1 -- 172.21.15.112:0/2202479675 shutdown_connections 2024-09-16T00:30:42.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.758+0000 7f29ec78a640 1 -- 172.21.15.112:0/2202479675 wait complete. 2024-09-16T00:30:42.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.837+0000 7f3144627640 1 -- 172.21.15.112:0/577696962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f313c0fb810 msgr2=0x7f313c0fbbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:42.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.837+0000 7f3144627640 1 --2- 172.21.15.112:0/577696962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f313c0fb810 0x7f313c0fbbe0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f31380048d0 tx=0x7f3138009da0 comp rx=0 tx=0).stop 2024-09-16T00:30:42.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.837+0000 7f3144627640 1 -- 172.21.15.112:0/577696962 shutdown_connections 2024-09-16T00:30:42.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.837+0000 7f3144627640 1 --2- 172.21.15.112:0/577696962 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f313c0fcf80 0x7f313c0fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:42.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.837+0000 7f3144627640 1 --2- 172.21.15.112:0/577696962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f313c0fb810 0x7f313c0fbbe0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:42.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.837+0000 7f3144627640 1 -- 172.21.15.112:0/577696962 >> 172.21.15.112:0/577696962 conn(0x7f313c0723b0 msgr2=0x7f313c0727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:42.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.837+0000 7f3144627640 1 -- 172.21.15.112:0/577696962 shutdown_connections 2024-09-16T00:30:42.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.837+0000 7f3144627640 1 -- 172.21.15.112:0/577696962 wait complete. 2024-09-16T00:30:42.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.837+0000 7f3144627640 1 Processor -- start 2024-09-16T00:30:42.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.837+0000 7f3144627640 1 -- start start 2024-09-16T00:30:42.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.837+0000 7f3144627640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f313c0fb810 0x7f313c1131e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:42.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.837+0000 7f3144627640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f313c0fcf80 0x7f313c113720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:42.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.837+0000 7f3144627640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f313c113d20 con 0x7f313c0fcf80 2024-09-16T00:30:42.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.837+0000 7f3144627640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f313c10d160 con 0x7f313c0fb810 2024-09-16T00:30:42.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.838+0000 7f3141bc2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f313c0fcf80 0x7f313c113720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:42.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.838+0000 7f3141bc2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f313c0fcf80 0x7f313c113720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32792/0 (socket says 172.21.15.112:32792) 2024-09-16T00:30:42.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.838+0000 7f3141bc2640 1 -- 172.21.15.112:0/2893726797 learned_addr learned my addr 172.21.15.112:0/2893726797 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:30:42.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.838+0000 7f31423c3640 1 --2- 172.21.15.112:0/2893726797 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f313c0fb810 0x7f313c1131e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:42.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.838+0000 7f3141bc2640 1 -- 172.21.15.112:0/2893726797 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f313c0fb810 msgr2=0x7f313c1131e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:42.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.838+0000 7f3141bc2640 1 --2- 172.21.15.112:0/2893726797 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f313c0fb810 0x7f313c1131e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:42.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.838+0000 7f3141bc2640 1 -- 172.21.15.112:0/2893726797 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3138005040 con 0x7f313c0fcf80 2024-09-16T00:30:42.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.838+0000 7f3141bc2640 1 --2- 172.21.15.112:0/2893726797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f313c0fcf80 0x7f313c113720 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f312c00c830 tx=0x7f312c004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:42.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.838+0000 7f3132ffd640 1 -- 172.21.15.112:0/2893726797 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f312c00a7d0 con 0x7f313c0fcf80 2024-09-16T00:30:42.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.838+0000 7f3132ffd640 1 -- 172.21.15.112:0/2893726797 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f312c00add0 con 0x7f313c0fcf80 2024-09-16T00:30:42.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.838+0000 7f3144627640 1 -- 172.21.15.112:0/2893726797 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f313c10d440 con 0x7f313c0fcf80 2024-09-16T00:30:42.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.838+0000 7f3132ffd640 1 -- 172.21.15.112:0/2893726797 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f312c0113a0 con 0x7f313c0fcf80 2024-09-16T00:30:42.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.838+0000 7f3144627640 1 -- 172.21.15.112:0/2893726797 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f313c10d9e0 con 0x7f313c0fcf80 2024-09-16T00:30:42.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.839+0000 7f3144627640 1 -- 172.21.15.112:0/2893726797 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f313c0fcbc0 con 0x7f313c0fcf80 2024-09-16T00:30:42.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.840+0000 7f3132ffd640 1 -- 172.21.15.112:0/2893726797 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 38) v1 ==== 92655+0+0 (secure 0 0 0) 0x7f312c012400 con 0x7f313c0fcf80 2024-09-16T00:30:42.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.841+0000 7f3132ffd640 1 --2- 172.21.15.112:0/2893726797 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f312806e660 0x7f3128070b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:42.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.841+0000 7f3132ffd640 1 -- 172.21.15.112:0/2893726797 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f312c08e2b0 con 0x7f313c0fcf80 2024-09-16T00:30:42.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.841+0000 7f31423c3640 1 --2- 172.21.15.112:0/2893726797 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f312806e660 0x7f3128070b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:42.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.841+0000 7f31423c3640 1 --2- 172.21.15.112:0/2893726797 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f312806e660 0x7f3128070b10 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3138009ac0 tx=0x7f3138035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:42.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:42.845+0000 7f3132ffd640 1 -- 172.21.15.112:0/2893726797 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f312c057400 con 0x7f313c0fcf80 2024-09-16T00:30:43.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:42 smithi169 ceph-mon[77626]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-09-16T00:30:43.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:42 smithi169 ceph-mon[77626]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-09-16T00:30:43.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:42 smithi169 ceph-mon[77626]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:30:43.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:42 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/3896232268' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:30:43.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:43.026+0000 7f3144627640 1 -- 172.21.15.112:0/2893726797 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f313c10e850 con 0x7f312806e660 2024-09-16T00:30:43.027 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:42 smithi112 ceph-mon[97750]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-09-16T00:30:43.027 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:42 smithi112 ceph-mon[97750]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-09-16T00:30:43.027 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:42 smithi112 ceph-mon[97750]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:30:43.027 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:42 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/3896232268' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:30:43.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:43.028+0000 7f3132ffd640 1 -- 172.21.15.112:0/2893726797 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f313c10e850 con 0x7f312806e660 2024-09-16T00:30:43.028 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:30:43.028 INFO:teuthology.orchestra.run.smithi112.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80", 2024-09-16T00:30:43.028 INFO:teuthology.orchestra.run.smithi112.stdout: "in_progress": true, 2024-09-16T00:30:43.028 INFO:teuthology.orchestra.run.smithi112.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T00:30:43.029 INFO:teuthology.orchestra.run.smithi112.stdout: "services_complete": [ 2024-09-16T00:30:43.029 INFO:teuthology.orchestra.run.smithi112.stdout: "crash", 2024-09-16T00:30:43.029 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr", 2024-09-16T00:30:43.029 INFO:teuthology.orchestra.run.smithi112.stdout: "mon" 2024-09-16T00:30:43.029 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-16T00:30:43.029 INFO:teuthology.orchestra.run.smithi112.stdout: "progress": "7/21 daemons upgraded", 2024-09-16T00:30:43.029 INFO:teuthology.orchestra.run.smithi112.stdout: "message": "Currently upgrading osd daemons", 2024-09-16T00:30:43.029 INFO:teuthology.orchestra.run.smithi112.stdout: "is_paused": false 2024-09-16T00:30:43.029 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:30:43.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:43.030+0000 7f3144627640 1 -- 172.21.15.112:0/2893726797 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f312806e660 msgr2=0x7f3128070b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:43.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:43.030+0000 7f3144627640 1 --2- 172.21.15.112:0/2893726797 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f312806e660 0x7f3128070b10 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3138009ac0 tx=0x7f3138035040 comp rx=0 tx=0).stop 2024-09-16T00:30:43.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:43.030+0000 7f3144627640 1 -- 172.21.15.112:0/2893726797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f313c0fcf80 msgr2=0x7f313c113720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:43.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:43.030+0000 7f3144627640 1 --2- 172.21.15.112:0/2893726797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f313c0fcf80 0x7f313c113720 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f312c00c830 tx=0x7f312c004430 comp rx=0 tx=0).stop 2024-09-16T00:30:43.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:43.030+0000 7f3144627640 1 -- 172.21.15.112:0/2893726797 shutdown_connections 2024-09-16T00:30:43.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:43.030+0000 7f3144627640 1 --2- 172.21.15.112:0/2893726797 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f312806e660 0x7f3128070b10 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:43.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:43.030+0000 7f3144627640 1 --2- 172.21.15.112:0/2893726797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f313c0fcf80 0x7f313c113720 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:43.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:43.030+0000 7f3144627640 1 --2- 172.21.15.112:0/2893726797 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f313c0fb810 0x7f313c1131e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:43.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:43.030+0000 7f3144627640 1 -- 172.21.15.112:0/2893726797 >> 172.21.15.112:0/2893726797 conn(0x7f313c0723b0 msgr2=0x7f313c064f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:43.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:43.030+0000 7f3144627640 1 -- 172.21.15.112:0/2893726797 shutdown_connections 2024-09-16T00:30:43.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:43.030+0000 7f3144627640 1 -- 172.21.15.112:0/2893726797 wait complete. 2024-09-16T00:30:43.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:43.107+0000 7f2c7d582640 1 -- 172.21.15.112:0/3485761559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c780fe460 msgr2=0x7f2c780fe830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:43.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:43.107+0000 7f2c7d582640 1 --2- 172.21.15.112:0/3485761559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c780fe460 0x7f2c780fe830 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2c68009c50 tx=0x7f2c68004d10 comp rx=0 tx=0).stop 2024-09-16T00:30:43.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:43.107+0000 7f2c7d582640 1 -- 172.21.15.112:0/3485761559 shutdown_connections 2024-09-16T00:30:43.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:43.107+0000 7f2c7d582640 1 --2- 172.21.15.112:0/3485761559 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c780fabe0 0x7f2c780fb030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:43.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:43.107+0000 7f2c7d582640 1 --2- 172.21.15.112:0/3485761559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c780fe460 0x7f2c780fe830 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:43.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:43.107+0000 7f2c7d582640 1 -- 172.21.15.112:0/3485761559 >> 172.21.15.112:0/3485761559 conn(0x7f2c78071b00 msgr2=0x7f2c78071f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:43.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:43.107+0000 7f2c7d582640 1 -- 172.21.15.112:0/3485761559 shutdown_connections 2024-09-16T00:30:43.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:43.107+0000 7f2c7d582640 1 -- 172.21.15.112:0/3485761559 wait complete. 2024-09-16T00:30:43.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:43.108+0000 7f2c7d582640 1 Processor -- start 2024-09-16T00:30:43.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:43.108+0000 7f2c7d582640 1 -- start start 2024-09-16T00:30:43.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:43.108+0000 7f2c7d582640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c780fabe0 0x7f2c781120c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:43.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:43.108+0000 7f2c7d582640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c780fe460 0x7f2c78112600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:43.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:43.108+0000 7f2c7d582640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c78112c50 con 0x7f2c780fe460 2024-09-16T00:30:43.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:43.108+0000 7f2c7d582640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7810d1f0 con 0x7f2c780fabe0 2024-09-16T00:30:43.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:43.108+0000 7f2c767fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c780fe460 0x7f2c78112600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:43.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:43.108+0000 7f2c767fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c780fe460 0x7f2c78112600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32812/0 (socket says 172.21.15.112:32812) 2024-09-16T00:30:43.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:43.109+0000 7f2c767fc640 1 -- 172.21.15.112:0/3471279244 learned_addr learned my addr 172.21.15.112:0/3471279244 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:30:43.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:43.109+0000 7f2c76ffd640 1 --2- 172.21.15.112:0/3471279244 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c780fabe0 0x7f2c781120c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:43.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:43.109+0000 7f2c767fc640 1 -- 172.21.15.112:0/3471279244 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c780fabe0 msgr2=0x7f2c781120c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:43.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:43.109+0000 7f2c767fc640 1 --2- 172.21.15.112:0/3471279244 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c780fabe0 0x7f2c781120c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:43.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:43.109+0000 7f2c767fc640 1 -- 172.21.15.112:0/3471279244 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c680098b0 con 0x7f2c780fe460 2024-09-16T00:30:43.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:43.109+0000 7f2c76ffd640 1 --2- 172.21.15.112:0/3471279244 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c780fabe0 0x7f2c781120c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:30:43.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:43.109+0000 7f2c767fc640 1 --2- 172.21.15.112:0/3471279244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c780fe460 0x7f2c78112600 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2c6000c830 tx=0x7f2c60004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:43.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:43.109+0000 7f2c6f7fe640 1 -- 172.21.15.112:0/3471279244 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c6000a850 con 0x7f2c780fe460 2024-09-16T00:30:43.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:43.109+0000 7f2c7d582640 1 -- 172.21.15.112:0/3471279244 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c7810d4d0 con 0x7f2c780fe460 2024-09-16T00:30:43.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:43.109+0000 7f2c6f7fe640 1 -- 172.21.15.112:0/3471279244 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f2c6000ae50 con 0x7f2c780fe460 2024-09-16T00:30:43.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:43.109+0000 7f2c6f7fe640 1 -- 172.21.15.112:0/3471279244 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c600113a0 con 0x7f2c780fe460 2024-09-16T00:30:43.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:43.109+0000 7f2c7d582640 1 -- 172.21.15.112:0/3471279244 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c7810da70 con 0x7f2c780fe460 2024-09-16T00:30:43.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:43.110+0000 7f2c7d582640 1 -- 172.21.15.112:0/3471279244 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c78064d50 con 0x7f2c780fe460 2024-09-16T00:30:43.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:43.111+0000 7f2c6f7fe640 1 -- 172.21.15.112:0/3471279244 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 38) v1 ==== 92655+0+0 (secure 0 0 0) 0x7f2c60012400 con 0x7f2c780fe460 2024-09-16T00:30:43.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:43.111+0000 7f2c6f7fe640 1 --2- 172.21.15.112:0/3471279244 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f2c6406e660 0x7f2c64070b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:43.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:43.111+0000 7f2c6f7fe640 1 -- 172.21.15.112:0/3471279244 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f2c6008e4e0 con 0x7f2c780fe460 2024-09-16T00:30:43.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:43.111+0000 7f2c76ffd640 1 --2- 172.21.15.112:0/3471279244 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f2c6406e660 0x7f2c64070b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:43.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:43.112+0000 7f2c76ffd640 1 --2- 172.21.15.112:0/3471279244 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f2c6406e660 0x7f2c64070b10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2c68004a30 tx=0x7f2c68004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:43.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:43.116+0000 7f2c6f7fe640 1 -- 172.21.15.112:0/3471279244 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f2c60057590 con 0x7f2c780fe460 2024-09-16T00:30:43.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:43.345+0000 7f2c7d582640 1 -- 172.21.15.112:0/3471279244 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f2c7804b720 con 0x7f2c780fe460 2024-09-16T00:30:43.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:43.345+0000 7f2c6f7fe640 1 -- 172.21.15.112:0/3471279244 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1484 (secure 0 0 0) 0x7f2c60055e60 con 0x7f2c780fe460 2024-09-16T00:30:43.346 INFO:teuthology.orchestra.run.smithi112.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; Degraded data redundancy: 71/372 objects degraded (19.086%), 20 pgs degraded 2024-09-16T00:30:43.346 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T00:30:43.346 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T00:30:43.346 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 71/372 objects degraded (19.086%), 20 pgs degraded 2024-09-16T00:30:43.346 INFO:teuthology.orchestra.run.smithi112.stdout: pg 1.0 is active+undersized+degraded, acting [3,1] 2024-09-16T00:30:43.346 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.0 is active+undersized+degraded, acting [3,1] 2024-09-16T00:30:43.346 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.1 is active+undersized+degraded, acting [2,1] 2024-09-16T00:30:43.346 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.2 is active+undersized+degraded, acting [5,1] 2024-09-16T00:30:43.346 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.4 is active+undersized+degraded, acting [1,4] 2024-09-16T00:30:43.346 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.5 is active+undersized+degraded, acting [3,4] 2024-09-16T00:30:43.346 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.8 is active+undersized+degraded, acting [3,5] 2024-09-16T00:30:43.346 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.9 is active+undersized+degraded, acting [1,4] 2024-09-16T00:30:43.346 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.c is active+undersized+degraded, acting [2,3] 2024-09-16T00:30:43.347 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.e is active+undersized+degraded, acting [2,3] 2024-09-16T00:30:43.347 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.f is active+undersized+degraded, acting [4,5] 2024-09-16T00:30:43.347 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.10 is active+undersized+degraded, acting [2,1] 2024-09-16T00:30:43.347 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.12 is active+undersized+degraded, acting [3,1] 2024-09-16T00:30:43.347 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.13 is active+undersized+degraded, acting [4,2] 2024-09-16T00:30:43.347 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.15 is active+undersized+degraded, acting [1,3] 2024-09-16T00:30:43.347 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.19 is active+undersized+degraded, acting [4,2] 2024-09-16T00:30:43.347 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.1b is active+undersized+degraded, acting [1,5] 2024-09-16T00:30:43.347 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.1d is active+undersized+degraded, acting [3,5] 2024-09-16T00:30:43.347 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.1e is active+undersized+degraded, acting [2,5] 2024-09-16T00:30:43.347 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.1f is active+undersized+degraded, acting [3,4] 2024-09-16T00:30:43.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:43.347+0000 7f2c7d582640 1 -- 172.21.15.112:0/3471279244 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f2c6406e660 msgr2=0x7f2c64070b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:43.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:43.347+0000 7f2c7d582640 1 --2- 172.21.15.112:0/3471279244 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f2c6406e660 0x7f2c64070b10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2c68004a30 tx=0x7f2c68004d10 comp rx=0 tx=0).stop 2024-09-16T00:30:43.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:43.347+0000 7f2c7d582640 1 -- 172.21.15.112:0/3471279244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c780fe460 msgr2=0x7f2c78112600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:43.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:43.347+0000 7f2c7d582640 1 --2- 172.21.15.112:0/3471279244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c780fe460 0x7f2c78112600 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2c6000c830 tx=0x7f2c60004430 comp rx=0 tx=0).stop 2024-09-16T00:30:43.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:43.347+0000 7f2c7d582640 1 -- 172.21.15.112:0/3471279244 shutdown_connections 2024-09-16T00:30:43.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:43.347+0000 7f2c7d582640 1 --2- 172.21.15.112:0/3471279244 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f2c6406e660 0x7f2c64070b10 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:43.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:43.347+0000 7f2c7d582640 1 --2- 172.21.15.112:0/3471279244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c780fe460 0x7f2c78112600 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:43.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:43.347+0000 7f2c7d582640 1 --2- 172.21.15.112:0/3471279244 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c780fabe0 0x7f2c781120c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:43.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:43.347+0000 7f2c7d582640 1 -- 172.21.15.112:0/3471279244 >> 172.21.15.112:0/3471279244 conn(0x7f2c78071b00 msgr2=0x7f2c780f9a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:43.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:43.347+0000 7f2c7d582640 1 -- 172.21.15.112:0/3471279244 shutdown_connections 2024-09-16T00:30:43.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:30:43.347+0000 7f2c7d582640 1 -- 172.21.15.112:0/3471279244 wait complete. 2024-09-16T00:30:44.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:43 smithi169 ceph-mon[77626]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:30:44.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:43 smithi169 ceph-mon[77626]: from='client.34200 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:30:44.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:43 smithi169 ceph-mon[77626]: pgmap v51: 65 pgs: 14 active+undersized, 20 active+undersized+degraded, 31 active+clean; 325 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 71/372 objects degraded (19.086%) 2024-09-16T00:30:44.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:43 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/2202479675' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:30:44.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:43 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/3471279244' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:30:44.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:43 smithi112 ceph-mon[97750]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:30:44.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:43 smithi112 ceph-mon[97750]: from='client.34200 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:30:44.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:43 smithi112 ceph-mon[97750]: pgmap v51: 65 pgs: 14 active+undersized, 20 active+undersized+degraded, 31 active+clean; 325 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 71/372 objects degraded (19.086%) 2024-09-16T00:30:44.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:43 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/2202479675' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:30:44.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:43 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/3471279244' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:30:45.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:44 smithi169 ceph-mon[77626]: from='client.34212 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:30:45.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:44 smithi112 ceph-mon[97750]: from='client.34212 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:30:46.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:45 smithi169 ceph-mon[77626]: pgmap v52: 65 pgs: 65 active+clean; 325 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-16T00:30:46.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:45 smithi169 ceph-mon[77626]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 71/372 objects degraded (19.086%), 20 pgs degraded) 2024-09-16T00:30:46.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:45 smithi112 ceph-mon[97750]: pgmap v52: 65 pgs: 65 active+clean; 325 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-16T00:30:46.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:45 smithi112 ceph-mon[97750]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 71/372 objects degraded (19.086%), 20 pgs degraded) 2024-09-16T00:30:47.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:47 smithi112 ceph-mon[97750]: pgmap v53: 65 pgs: 65 active+clean; 325 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.3 KiB/s rd, 0 op/s 2024-09-16T00:30:47.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:47 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:47.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:47 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:30:48.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:47 smithi169 ceph-mon[77626]: pgmap v53: 65 pgs: 65 active+clean; 325 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.3 KiB/s rd, 0 op/s 2024-09-16T00:30:48.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:47 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:48.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:47 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:30:49.829 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:49 smithi169 ceph-mon[77626]: pgmap v54: 65 pgs: 65 active+clean; 325 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.6 KiB/s rd, 509 B/s wr, 3 op/s 2024-09-16T00:30:49.829 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:49 smithi169 ceph-mon[77626]: mgrmap e39: smithi112.xrkkbc(active, since 92s), standbys: smithi169.mtrobk 2024-09-16T00:30:49.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:49 smithi112 ceph-mon[97750]: pgmap v54: 65 pgs: 65 active+clean; 325 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.6 KiB/s rd, 509 B/s wr, 3 op/s 2024-09-16T00:30:49.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:49 smithi112 ceph-mon[97750]: mgrmap e39: smithi112.xrkkbc(active, since 92s), standbys: smithi169.mtrobk 2024-09-16T00:30:51.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:51 smithi112 ceph-mon[97750]: pgmap v55: 65 pgs: 65 active+clean; 325 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.2 KiB/s rd, 614 B/s wr, 3 op/s 2024-09-16T00:30:52.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:51 smithi169 ceph-mon[77626]: pgmap v55: 65 pgs: 65 active+clean; 325 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.2 KiB/s rd, 614 B/s wr, 3 op/s 2024-09-16T00:30:52.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:52 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:53.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:52 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:53.822 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:53 smithi112 ceph-mon[97750]: pgmap v56: 65 pgs: 65 active+clean; 325 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.0 KiB/s rd, 557 B/s wr, 3 op/s 2024-09-16T00:30:54.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:53 smithi169 ceph-mon[77626]: pgmap v56: 65 pgs: 65 active+clean; 325 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.0 KiB/s rd, 557 B/s wr, 3 op/s 2024-09-16T00:30:55.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:55 smithi112 ceph-mon[97750]: pgmap v57: 65 pgs: 65 active+clean; 325 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.7 KiB/s rd, 511 B/s wr, 4 op/s 2024-09-16T00:30:56.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:55 smithi169 ceph-mon[77626]: pgmap v57: 65 pgs: 65 active+clean; 325 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.7 KiB/s rd, 511 B/s wr, 4 op/s 2024-09-16T00:30:56.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:56 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-09-16T00:30:57.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:56 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-09-16T00:30:57.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:57 smithi112 ceph-mon[97750]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-09-16T00:30:57.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:57 smithi112 ceph-mon[97750]: Upgrade: osd.1 is safe to restart 2024-09-16T00:30:57.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:57 smithi112 ceph-mon[97750]: pgmap v58: 65 pgs: 65 active+clean; 325 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.8 KiB/s rd, 511 B/s wr, 3 op/s 2024-09-16T00:30:57.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:57 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:57.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:57 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-16T00:30:57.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:57 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:30:58.028 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:57 smithi169 ceph-mon[77626]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-09-16T00:30:58.029 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:57 smithi169 ceph-mon[77626]: Upgrade: osd.1 is safe to restart 2024-09-16T00:30:58.029 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:57 smithi169 ceph-mon[77626]: pgmap v58: 65 pgs: 65 active+clean; 325 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.8 KiB/s rd, 511 B/s wr, 3 op/s 2024-09-16T00:30:58.029 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:57 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:30:58.029 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:57 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-16T00:30:58.029 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:57 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:30:58.838 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:58 smithi112 ceph-mon[97750]: Upgrade: Updating osd.1 2024-09-16T00:30:58.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:58 smithi112 ceph-mon[97750]: Deploying daemon osd.1 on smithi112 2024-09-16T00:30:59.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:58 smithi169 ceph-mon[77626]: Upgrade: Updating osd.1 2024-09-16T00:30:59.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:58 smithi169 ceph-mon[77626]: Deploying daemon osd.1 on smithi112 2024-09-16T00:30:59.731 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:30:59 smithi112 ceph-mon[97750]: pgmap v59: 65 pgs: 65 active+clean; 325 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2024-09-16T00:31:00.136 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:30:59 smithi169 ceph-mon[77626]: pgmap v59: 65 pgs: 65 active+clean; 325 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2024-09-16T00:31:00.339 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:31:00 smithi112 systemd[1]: Stopping Ceph osd.1 for d346dd56-73c0-11ef-bceb-c7b262605968... 2024-09-16T00:31:00.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:00 smithi112 ceph-mon[97750]: osd.1 marked itself down and dead 2024-09-16T00:31:00.839 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:31:00 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-1[53484]: 2024-09-16T00:31:00.352+0000 7fb1d188d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:31:00.839 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:31:00 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-1[53484]: 2024-09-16T00:31:00.352+0000 7fb1d188d640 -1 osd.1 52 *** Got signal Terminated *** 2024-09-16T00:31:00.840 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:31:00 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-1[53484]: 2024-09-16T00:31:00.352+0000 7fb1d188d640 -1 osd.1 52 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:31:01.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:00 smithi169 ceph-mon[77626]: osd.1 marked itself down and dead 2024-09-16T00:31:01.361 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:31:01 smithi112 podman[114240]: 2024-09-16 00:31:01.107245756 +0000 UTC m=+0.881589629 container died e9bee87bba0f30fa44c5796d6a72723ce1c1869d2d1dc8f7bde475b8b864d97b (image=quay.ceph.io/ceph-ci/ceph@sha256:83596473fc70117d507bb3d925c5bd11b70f46bbd4dffcc8452f145d93eaedc4, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-1, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-6d025bc, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-16T00:31:01.787 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:01 smithi112 ceph-mon[97750]: pgmap v60: 65 pgs: 65 active+clean; 325 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 170 B/s wr, 3 op/s 2024-09-16T00:31:01.787 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:01 smithi112 ceph-mon[97750]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-16T00:31:01.787 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:01 smithi112 ceph-mon[97750]: osdmap e53: 6 total, 5 up, 6 in 2024-09-16T00:31:01.787 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:01 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:31:01.787 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:01 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:31:01.788 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:31:01 smithi112 podman[114240]: 2024-09-16 00:31:01.477027524 +0000 UTC m=+1.251371392 container cleanup e9bee87bba0f30fa44c5796d6a72723ce1c1869d2d1dc8f7bde475b8b864d97b (image=quay.ceph.io/ceph-ci/ceph@sha256:83596473fc70117d507bb3d925c5bd11b70f46bbd4dffcc8452f145d93eaedc4, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-1, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=quincy-6d025bc, org.label-schema.vendor=CentOS) 2024-09-16T00:31:01.788 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:31:01 smithi112 bash[114240]: ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-1 2024-09-16T00:31:02.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:01 smithi169 ceph-mon[77626]: pgmap v60: 65 pgs: 65 active+clean; 325 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 170 B/s wr, 3 op/s 2024-09-16T00:31:02.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:01 smithi169 ceph-mon[77626]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-16T00:31:02.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:01 smithi169 ceph-mon[77626]: osdmap e53: 6 total, 5 up, 6 in 2024-09-16T00:31:02.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:01 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:31:02.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:01 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:31:02.089 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:31:01 smithi112 podman[114254]: 2024-09-16 00:31:01.787249104 +0000 UTC m=+0.684329853 container remove e9bee87bba0f30fa44c5796d6a72723ce1c1869d2d1dc8f7bde475b8b864d97b (image=quay.ceph.io/ceph-ci/ceph@sha256:83596473fc70117d507bb3d925c5bd11b70f46bbd4dffcc8452f145d93eaedc4, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-1, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=quincy-6d025bc) 2024-09-16T00:31:02.415 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:31:02 smithi112 podman[114341]: 2024-09-16 00:31:01.997661177 +0000 UTC m=+0.015633238 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:31:02.415 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:31:02 smithi112 podman[114341]: 2024-09-16 00:31:02.122677124 +0000 UTC m=+0.140649176 container create 3dd7e9802b914026ba60f61566c91c070052befe8bbfc61203df1f7fcaf70710 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-16T00:31:02.415 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:31:02 smithi112 podman[114341]: 2024-09-16 00:31:02.340960254 +0000 UTC m=+0.358932319 container init 3dd7e9802b914026ba60f61566c91c070052befe8bbfc61203df1f7fcaf70710 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:31:02.415 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:31:02 smithi112 podman[114341]: 2024-09-16 00:31:02.34450874 +0000 UTC m=+0.362480793 container start 3dd7e9802b914026ba60f61566c91c070052befe8bbfc61203df1f7fcaf70710 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2) 2024-09-16T00:31:02.416 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:31:02 smithi112 podman[114341]: 2024-09-16 00:31:02.415526384 +0000 UTC m=+0.433498441 container attach 3dd7e9802b914026ba60f61566c91c070052befe8bbfc61203df1f7fcaf70710 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-16T00:31:02.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:02 smithi112 ceph-mon[97750]: osdmap e54: 6 total, 5 up, 6 in 2024-09-16T00:31:02.839 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:31:02 smithi112 podman[114341]: 2024-09-16 00:31:02.534608738 +0000 UTC m=+0.552580798 container died 3dd7e9802b914026ba60f61566c91c070052befe8bbfc61203df1f7fcaf70710 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-1-deactivate, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-16T00:31:03.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:02 smithi169 ceph-mon[77626]: osdmap e54: 6 total, 5 up, 6 in 2024-09-16T00:31:03.505 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:31:03 smithi112 podman[114364]: 2024-09-16 00:31:03.178168748 +0000 UTC m=+0.636190475 container remove 3dd7e9802b914026ba60f61566c91c070052befe8bbfc61203df1f7fcaf70710 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-16T00:31:03.506 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:31:03 smithi112 systemd[1]: ceph-d346dd56-73c0-11ef-bceb-c7b262605968@osd.1.service: Deactivated successfully. 2024-09-16T00:31:03.506 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:31:03 smithi112 systemd[1]: Stopped Ceph osd.1 for d346dd56-73c0-11ef-bceb-c7b262605968. 2024-09-16T00:31:03.506 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:31:03 smithi112 systemd[1]: ceph-d346dd56-73c0-11ef-bceb-c7b262605968@osd.1.service: Consumed 51.666s CPU time. 2024-09-16T00:31:03.506 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:31:03 smithi112 systemd[1]: Starting Ceph osd.1 for d346dd56-73c0-11ef-bceb-c7b262605968... 2024-09-16T00:31:03.769 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:03 smithi112 ceph-mon[97750]: pgmap v63: 65 pgs: 12 stale+active+clean, 53 active+clean; 325 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-16T00:31:03.770 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:31:03 smithi112 podman[114464]: 2024-09-16 00:31:03.523657876 +0000 UTC m=+0.015900214 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:31:03.770 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:31:03 smithi112 podman[114464]: 2024-09-16 00:31:03.629223096 +0000 UTC m=+0.121465480 container create 3530779b3c103678283ac273f55860855d58e49ef37989cdf287ab551ab059f4 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-1-activate, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-16T00:31:04.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:03 smithi169 ceph-mon[77626]: pgmap v63: 65 pgs: 12 stale+active+clean, 53 active+clean; 325 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-16T00:31:04.278 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:31:03 smithi112 podman[114464]: 2024-09-16 00:31:03.865321977 +0000 UTC m=+0.357564318 container init 3530779b3c103678283ac273f55860855d58e49ef37989cdf287ab551ab059f4 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-1-activate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-16T00:31:04.278 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:31:03 smithi112 podman[114464]: 2024-09-16 00:31:03.868737977 +0000 UTC m=+0.360980322 container start 3530779b3c103678283ac273f55860855d58e49ef37989cdf287ab551ab059f4 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-1-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:31:04.279 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:31:03 smithi112 podman[114464]: 2024-09-16 00:31:03.940181199 +0000 UTC m=+0.432423544 container attach 3530779b3c103678283ac273f55860855d58e49ef37989cdf287ab551ab059f4 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-1-activate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-16T00:31:04.279 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:31:03 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-1-activate[114479]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T00:31:04.279 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:31:03 smithi112 bash[114464]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T00:31:04.279 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:31:04 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-1-activate[114479]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T00:31:04.279 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:31:04 smithi112 bash[114464]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T00:31:05.018 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:04 smithi112 ceph-mon[97750]: Health check failed: Degraded data redundancy: 57/372 objects degraded (15.323%), 16 pgs degraded (PG_DEGRADED) 2024-09-16T00:31:05.018 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:31:04 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-1-activate[114479]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-16T00:31:05.018 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:31:04 smithi112 bash[114464]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-16T00:31:05.018 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:31:04 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-1-activate[114479]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-09-16T00:31:05.018 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:31:04 smithi112 bash[114464]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-09-16T00:31:05.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:04 smithi169 ceph-mon[77626]: Health check failed: Degraded data redundancy: 57/372 objects degraded (15.323%), 16 pgs degraded (PG_DEGRADED) 2024-09-16T00:31:05.340 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:31:05 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-1-activate[114479]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-09-16T00:31:05.340 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:31:05 smithi112 bash[114464]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-09-16T00:31:05.340 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:31:05 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-1-activate[114479]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-09-16T00:31:05.340 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:31:05 smithi112 bash[114464]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-09-16T00:31:05.340 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:31:05 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-1-activate[114479]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-09-16T00:31:05.340 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:31:05 smithi112 bash[114464]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-09-16T00:31:05.340 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:31:05 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-1-activate[114479]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-16T00:31:05.340 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:31:05 smithi112 bash[114464]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-16T00:31:05.340 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:31:05 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-1-activate[114479]: --> ceph-volume raw activate successful for osd ID: 1 2024-09-16T00:31:05.340 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:31:05 smithi112 bash[114464]: --> ceph-volume raw activate successful for osd ID: 1 2024-09-16T00:31:05.341 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:31:05 smithi112 podman[114464]: 2024-09-16 00:31:05.046261908 +0000 UTC m=+1.538504247 container died 3530779b3c103678283ac273f55860855d58e49ef37989cdf287ab551ab059f4 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-1-activate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=) 2024-09-16T00:31:05.936 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:05 smithi112 ceph-mon[97750]: pgmap v64: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 325 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 57/372 objects degraded (15.323%) 2024-09-16T00:31:05.937 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:31:05 smithi112 podman[114612]: 2024-09-16 00:31:05.68577082 +0000 UTC m=+0.633499340 container remove 3530779b3c103678283ac273f55860855d58e49ef37989cdf287ab551ab059f4 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-1-activate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-16T00:31:05.937 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:31:05 smithi112 podman[114670]: 2024-09-16 00:31:05.796001226 +0000 UTC m=+0.016056742 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:31:05.937 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:31:05 smithi112 podman[114670]: 2024-09-16 00:31:05.937059666 +0000 UTC m=+0.157115180 container create fe6f1a4a2cf45b04eb78e499b9269bcdf555fbe6d14b1e9c8f5fbddb8d70e707 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-1, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-16T00:31:06.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:05 smithi169 ceph-mon[77626]: pgmap v64: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 325 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 57/372 objects degraded (15.323%) 2024-09-16T00:31:06.196 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:31:06 smithi112 podman[114670]: 2024-09-16 00:31:06.173339893 +0000 UTC m=+0.393395412 container init fe6f1a4a2cf45b04eb78e499b9269bcdf555fbe6d14b1e9c8f5fbddb8d70e707 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-1, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-16T00:31:06.197 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:31:06 smithi112 podman[114670]: 2024-09-16 00:31:06.176417917 +0000 UTC m=+0.396473438 container start fe6f1a4a2cf45b04eb78e499b9269bcdf555fbe6d14b1e9c8f5fbddb8d70e707 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-1, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-16T00:31:06.589 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:31:06 smithi112 ceph-osd[114689]: -- 172.21.15.112:0/4142164902 <== mon.0 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5617503e81e0 con 0x56175037e000 2024-09-16T00:31:06.589 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:31:06 smithi112 bash[114670]: fe6f1a4a2cf45b04eb78e499b9269bcdf555fbe6d14b1e9c8f5fbddb8d70e707 2024-09-16T00:31:06.589 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:31:06 smithi112 systemd[1]: Started Ceph osd.1 for d346dd56-73c0-11ef-bceb-c7b262605968. 2024-09-16T00:31:07.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:07 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:31:07.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:07 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:31:07.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:07 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:31:07.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:07 smithi112 ceph-mon[97750]: pgmap v65: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 325 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 57/372 objects degraded (15.323%) 2024-09-16T00:31:07.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:07 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:31:07.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:07 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:31:07.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:07 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:31:07.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:07 smithi169 ceph-mon[77626]: pgmap v65: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 325 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 57/372 objects degraded (15.323%) 2024-09-16T00:31:08.089 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:31:07 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-1[114685]: 2024-09-16T00:31:07.838+0000 7fa0fa231640 -1 Falling back to public interface 2024-09-16T00:31:09.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:09 smithi112 ceph-mon[97750]: pgmap v66: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 325 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 57/372 objects degraded (15.323%) 2024-09-16T00:31:10.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:09 smithi169 ceph-mon[77626]: pgmap v66: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 325 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 57/372 objects degraded (15.323%) 2024-09-16T00:31:11.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:11 smithi112 ceph-mon[97750]: pgmap v67: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 325 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 57/372 objects degraded (15.323%) 2024-09-16T00:31:11.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:11 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:31:11.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:11 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:31:12.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:11 smithi169 ceph-mon[77626]: pgmap v67: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 325 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 57/372 objects degraded (15.323%) 2024-09-16T00:31:12.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:11 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:31:12.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:11 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:31:13.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:13 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:31:13.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:13 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:31:13.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:13 smithi169 ceph-mon[77626]: pgmap v68: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 325 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 57/372 objects degraded (15.323%) 2024-09-16T00:31:13.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:13 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:31:13.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:13 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:31:13.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:13 smithi112 ceph-mon[97750]: pgmap v68: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 325 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 57/372 objects degraded (15.323%) 2024-09-16T00:31:13.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.431+0000 7fb1b8a7c640 1 -- 172.21.15.112:0/2159508565 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1b40fb810 msgr2=0x7fb1b40fbbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:13.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.431+0000 7fb1b8a7c640 1 --2- 172.21.15.112:0/2159508565 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1b40fb810 0x7fb1b40fbbe0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb1a40048d0 tx=0x7fb1a4009da0 comp rx=0 tx=0).stop 2024-09-16T00:31:13.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.431+0000 7fb1b8a7c640 1 -- 172.21.15.112:0/2159508565 shutdown_connections 2024-09-16T00:31:13.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.431+0000 7fb1b8a7c640 1 --2- 172.21.15.112:0/2159508565 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1b40fcf80 0x7fb1b40fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:13.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.431+0000 7fb1b8a7c640 1 --2- 172.21.15.112:0/2159508565 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1b40fb810 0x7fb1b40fbbe0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:13.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.431+0000 7fb1b8a7c640 1 -- 172.21.15.112:0/2159508565 >> 172.21.15.112:0/2159508565 conn(0x7fb1b40723b0 msgr2=0x7fb1b40727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:13.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.431+0000 7fb1b8a7c640 1 -- 172.21.15.112:0/2159508565 shutdown_connections 2024-09-16T00:31:13.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.431+0000 7fb1b8a7c640 1 -- 172.21.15.112:0/2159508565 wait complete. 2024-09-16T00:31:13.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.432+0000 7fb1b8a7c640 1 Processor -- start 2024-09-16T00:31:13.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.432+0000 7fb1b8a7c640 1 -- start start 2024-09-16T00:31:13.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.432+0000 7fb1b8a7c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1b40fb810 0x7fb1b410e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:13.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.432+0000 7fb1b8a7c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1b40fcf80 0x7fb1b410e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:13.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.432+0000 7fb1b8a7c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b410ed00 con 0x7fb1b40fb810 2024-09-16T00:31:13.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.432+0000 7fb1b8a7c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b410ee70 con 0x7fb1b40fcf80 2024-09-16T00:31:13.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.432+0000 7fb1b259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1b40fb810 0x7fb1b410e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:13.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.432+0000 7fb1b259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1b40fb810 0x7fb1b410e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60950/0 (socket says 172.21.15.112:60950) 2024-09-16T00:31:13.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.432+0000 7fb1b259c640 1 -- 172.21.15.112:0/1563583524 learned_addr learned my addr 172.21.15.112:0/1563583524 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:31:13.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.433+0000 7fb1b1d9b640 1 --2- 172.21.15.112:0/1563583524 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1b40fcf80 0x7fb1b410e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:13.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.433+0000 7fb1b259c640 1 -- 172.21.15.112:0/1563583524 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1b40fcf80 msgr2=0x7fb1b410e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:13.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.433+0000 7fb1b259c640 1 --2- 172.21.15.112:0/1563583524 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1b40fcf80 0x7fb1b410e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:13.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.433+0000 7fb1b259c640 1 -- 172.21.15.112:0/1563583524 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1a4005040 con 0x7fb1b40fb810 2024-09-16T00:31:13.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.433+0000 7fb1b1d9b640 1 --2- 172.21.15.112:0/1563583524 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1b40fcf80 0x7fb1b410e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:31:13.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.433+0000 7fb1b259c640 1 --2- 172.21.15.112:0/1563583524 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1b40fb810 0x7fb1b410e170 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb1a4000c00 tx=0x7fb1a4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:13.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.433+0000 7fb1a2ffd640 1 -- 172.21.15.112:0/1563583524 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb1a4038070 con 0x7fb1b40fb810 2024-09-16T00:31:13.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.433+0000 7fb1a2ffd640 1 -- 172.21.15.112:0/1563583524 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fb1a402ade0 con 0x7fb1b40fb810 2024-09-16T00:31:13.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.433+0000 7fb1b8a7c640 1 -- 172.21.15.112:0/1563583524 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1b41171e0 con 0x7fb1b40fb810 2024-09-16T00:31:13.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.433+0000 7fb1a2ffd640 1 -- 172.21.15.112:0/1563583524 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb1a402c7e0 con 0x7fb1b40fb810 2024-09-16T00:31:13.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.433+0000 7fb1b8a7c640 1 -- 172.21.15.112:0/1563583524 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1b41175c0 con 0x7fb1b40fb810 2024-09-16T00:31:13.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.434+0000 7fb1b8a7c640 1 -- 172.21.15.112:0/1563583524 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1b40fcbc0 con 0x7fb1b40fb810 2024-09-16T00:31:13.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.435+0000 7fb1a2ffd640 1 -- 172.21.15.112:0/1563583524 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fb1a4030020 con 0x7fb1b40fb810 2024-09-16T00:31:13.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.436+0000 7fb1a2ffd640 1 --2- 172.21.15.112:0/1563583524 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fb19c06e530 0x7fb19c0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:13.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.436+0000 7fb1a2ffd640 1 -- 172.21.15.112:0/1563583524 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fb1a40adda0 con 0x7fb1b40fb810 2024-09-16T00:31:13.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.436+0000 7fb1b1d9b640 1 --2- 172.21.15.112:0/1563583524 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fb19c06e530 0x7fb19c0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:13.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.436+0000 7fb1b1d9b640 1 --2- 172.21.15.112:0/1563583524 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fb19c06e530 0x7fb19c0709e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb1b4065980 tx=0x7fb1a8009ed0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:13.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.440+0000 7fb1a2ffd640 1 -- 172.21.15.112:0/1563583524 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fb1a4076f90 con 0x7fb1b40fb810 2024-09-16T00:31:13.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.633+0000 7fb1b8a7c640 1 -- 172.21.15.112:0/1563583524 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb1b40657c0 con 0x7fb19c06e530 2024-09-16T00:31:13.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.634+0000 7fb1a2ffd640 1 -- 172.21.15.112:0/1563583524 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fb1b40657c0 con 0x7fb19c06e530 2024-09-16T00:31:13.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.637+0000 7fb1b8a7c640 1 -- 172.21.15.112:0/1563583524 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fb19c06e530 msgr2=0x7fb19c0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:13.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.637+0000 7fb1b8a7c640 1 --2- 172.21.15.112:0/1563583524 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fb19c06e530 0x7fb19c0709e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb1b4065980 tx=0x7fb1a8009ed0 comp rx=0 tx=0).stop 2024-09-16T00:31:13.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.637+0000 7fb1b8a7c640 1 -- 172.21.15.112:0/1563583524 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1b40fb810 msgr2=0x7fb1b410e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:13.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.637+0000 7fb1b8a7c640 1 --2- 172.21.15.112:0/1563583524 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1b40fb810 0x7fb1b410e170 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb1a4000c00 tx=0x7fb1a4009da0 comp rx=0 tx=0).stop 2024-09-16T00:31:13.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.637+0000 7fb1b8a7c640 1 -- 172.21.15.112:0/1563583524 shutdown_connections 2024-09-16T00:31:13.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.637+0000 7fb1b8a7c640 1 --2- 172.21.15.112:0/1563583524 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fb19c06e530 0x7fb19c0709e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:13.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.637+0000 7fb1b8a7c640 1 --2- 172.21.15.112:0/1563583524 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1b40fcf80 0x7fb1b410e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:13.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.637+0000 7fb1b8a7c640 1 --2- 172.21.15.112:0/1563583524 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1b40fb810 0x7fb1b410e170 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:13.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.637+0000 7fb1b8a7c640 1 -- 172.21.15.112:0/1563583524 >> 172.21.15.112:0/1563583524 conn(0x7fb1b40723b0 msgr2=0x7fb1b40fa480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:13.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.637+0000 7fb1b8a7c640 1 -- 172.21.15.112:0/1563583524 shutdown_connections 2024-09-16T00:31:13.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.637+0000 7fb1b8a7c640 1 -- 172.21.15.112:0/1563583524 wait complete. 2024-09-16T00:31:13.647 INFO:teuthology.orchestra.run.smithi112.stdout:true 2024-09-16T00:31:13.687 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:31:13 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-1[114685]: 2024-09-16T00:31:13.426+0000 7fa0fa231640 -1 osd.1 52 log_to_monitors true 2024-09-16T00:31:13.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.719+0000 7f20f4cd8640 1 -- 172.21.15.112:0/1812962503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20f00fb810 msgr2=0x7f20f00fbbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:13.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.719+0000 7f20f4cd8640 1 --2- 172.21.15.112:0/1812962503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20f00fb810 0x7f20f00fbbe0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f20e00048d0 tx=0x7f20e0009da0 comp rx=0 tx=0).stop 2024-09-16T00:31:13.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.719+0000 7f20f4cd8640 1 -- 172.21.15.112:0/1812962503 shutdown_connections 2024-09-16T00:31:13.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.719+0000 7f20f4cd8640 1 --2- 172.21.15.112:0/1812962503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20f00fcf80 0x7f20f00fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:13.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.719+0000 7f20f4cd8640 1 --2- 172.21.15.112:0/1812962503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20f00fb810 0x7f20f00fbbe0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:13.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.719+0000 7f20f4cd8640 1 -- 172.21.15.112:0/1812962503 >> 172.21.15.112:0/1812962503 conn(0x7f20f00723b0 msgr2=0x7f20f00727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:13.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.719+0000 7f20f4cd8640 1 -- 172.21.15.112:0/1812962503 shutdown_connections 2024-09-16T00:31:13.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.719+0000 7f20f4cd8640 1 -- 172.21.15.112:0/1812962503 wait complete. 2024-09-16T00:31:13.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.719+0000 7f20f4cd8640 1 Processor -- start 2024-09-16T00:31:13.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.719+0000 7f20f4cd8640 1 -- start start 2024-09-16T00:31:13.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.719+0000 7f20f4cd8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20f00fb810 0x7f20f01120b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:13.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.719+0000 7f20f4cd8640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20f00fcf80 0x7f20f01125f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:13.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.719+0000 7f20f4cd8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f0112bf0 con 0x7f20f00fb810 2024-09-16T00:31:13.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.719+0000 7f20f4cd8640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f010c030 con 0x7f20f00fcf80 2024-09-16T00:31:13.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.720+0000 7f20ee59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20f00fb810 0x7f20f01120b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:13.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.720+0000 7f20ee59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20f00fb810 0x7f20f01120b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60954/0 (socket says 172.21.15.112:60954) 2024-09-16T00:31:13.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.720+0000 7f20ee59c640 1 -- 172.21.15.112:0/3641312477 learned_addr learned my addr 172.21.15.112:0/3641312477 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:31:13.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.720+0000 7f20e7fff640 1 --2- 172.21.15.112:0/3641312477 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20f00fcf80 0x7f20f01125f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:13.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.720+0000 7f20ee59c640 1 -- 172.21.15.112:0/3641312477 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20f00fcf80 msgr2=0x7f20f01125f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:13.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.720+0000 7f20ee59c640 1 --2- 172.21.15.112:0/3641312477 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20f00fcf80 0x7f20f01125f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:13.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.720+0000 7f20ee59c640 1 -- 172.21.15.112:0/3641312477 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e0005040 con 0x7f20f00fb810 2024-09-16T00:31:13.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.720+0000 7f20ee59c640 1 --2- 172.21.15.112:0/3641312477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20f00fb810 0x7f20f01120b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f20e0000c00 tx=0x7f20e0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:13.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.720+0000 7f20e6ffd640 1 -- 172.21.15.112:0/3641312477 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f20e0038070 con 0x7f20f00fb810 2024-09-16T00:31:13.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.720+0000 7f20f4cd8640 1 -- 172.21.15.112:0/3641312477 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20f010c2b0 con 0x7f20f00fb810 2024-09-16T00:31:13.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.720+0000 7f20e6ffd640 1 -- 172.21.15.112:0/3641312477 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f20e002ade0 con 0x7f20f00fb810 2024-09-16T00:31:13.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.720+0000 7f20e6ffd640 1 -- 172.21.15.112:0/3641312477 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f20e002c7e0 con 0x7f20f00fb810 2024-09-16T00:31:13.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.720+0000 7f20f4cd8640 1 -- 172.21.15.112:0/3641312477 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20f010c7f0 con 0x7f20f00fb810 2024-09-16T00:31:13.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.721+0000 7f20f4cd8640 1 -- 172.21.15.112:0/3641312477 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20f00fcbc0 con 0x7f20f00fb810 2024-09-16T00:31:13.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.722+0000 7f20e6ffd640 1 -- 172.21.15.112:0/3641312477 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f20e0030020 con 0x7f20f00fb810 2024-09-16T00:31:13.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.722+0000 7f20e6ffd640 1 --2- 172.21.15.112:0/3641312477 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f20dc06e530 0x7f20dc0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:13.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.722+0000 7f20e6ffd640 1 -- 172.21.15.112:0/3641312477 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f20e00adda0 con 0x7f20f00fb810 2024-09-16T00:31:13.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.722+0000 7f20e7fff640 1 --2- 172.21.15.112:0/3641312477 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f20dc06e530 0x7f20dc0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:13.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.723+0000 7f20e7fff640 1 --2- 172.21.15.112:0/3641312477 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f20dc06e530 0x7f20dc0709e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f20f010d970 tx=0x7f20d800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:13.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.726+0000 7f20e6ffd640 1 -- 172.21.15.112:0/3641312477 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f20e0076f90 con 0x7f20f00fb810 2024-09-16T00:31:13.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.916+0000 7f20f4cd8640 1 -- 172.21.15.112:0/3641312477 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f20f004b720 con 0x7f20dc06e530 2024-09-16T00:31:13.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.917+0000 7f20e6ffd640 1 -- 172.21.15.112:0/3641312477 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f20f004b720 con 0x7f20dc06e530 2024-09-16T00:31:13.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.919+0000 7f20f4cd8640 1 -- 172.21.15.112:0/3641312477 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f20dc06e530 msgr2=0x7f20dc0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:13.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.919+0000 7f20f4cd8640 1 --2- 172.21.15.112:0/3641312477 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f20dc06e530 0x7f20dc0709e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f20f010d970 tx=0x7f20d800b040 comp rx=0 tx=0).stop 2024-09-16T00:31:13.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.919+0000 7f20f4cd8640 1 -- 172.21.15.112:0/3641312477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20f00fb810 msgr2=0x7f20f01120b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:13.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.919+0000 7f20f4cd8640 1 --2- 172.21.15.112:0/3641312477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20f00fb810 0x7f20f01120b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f20e0000c00 tx=0x7f20e0009da0 comp rx=0 tx=0).stop 2024-09-16T00:31:13.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.919+0000 7f20f4cd8640 1 -- 172.21.15.112:0/3641312477 shutdown_connections 2024-09-16T00:31:13.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.919+0000 7f20f4cd8640 1 --2- 172.21.15.112:0/3641312477 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f20dc06e530 0x7f20dc0709e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:13.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.919+0000 7f20f4cd8640 1 --2- 172.21.15.112:0/3641312477 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20f00fcf80 0x7f20f01125f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:13.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.919+0000 7f20f4cd8640 1 --2- 172.21.15.112:0/3641312477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20f00fb810 0x7f20f01120b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:13.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.919+0000 7f20f4cd8640 1 -- 172.21.15.112:0/3641312477 >> 172.21.15.112:0/3641312477 conn(0x7f20f00723b0 msgr2=0x7f20f00fa480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:13.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.919+0000 7f20f4cd8640 1 -- 172.21.15.112:0/3641312477 shutdown_connections 2024-09-16T00:31:13.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:13.919+0000 7f20f4cd8640 1 -- 172.21.15.112:0/3641312477 wait complete. 2024-09-16T00:31:14.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.001+0000 7fc38fecc640 1 -- 172.21.15.112:0/845445695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3880fb810 msgr2=0x7fc3880fbbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:14.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.001+0000 7fc38fecc640 1 --2- 172.21.15.112:0/845445695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3880fb810 0x7fc3880fbbe0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc3840048d0 tx=0x7fc384009da0 comp rx=0 tx=0).stop 2024-09-16T00:31:14.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.001+0000 7fc38fecc640 1 -- 172.21.15.112:0/845445695 shutdown_connections 2024-09-16T00:31:14.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.001+0000 7fc38fecc640 1 --2- 172.21.15.112:0/845445695 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3880fcf80 0x7fc3880fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:14.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.001+0000 7fc38fecc640 1 --2- 172.21.15.112:0/845445695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3880fb810 0x7fc3880fbbe0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:14.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.001+0000 7fc38fecc640 1 -- 172.21.15.112:0/845445695 >> 172.21.15.112:0/845445695 conn(0x7fc3880723b0 msgr2=0x7fc3880727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:14.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.001+0000 7fc38fecc640 1 -- 172.21.15.112:0/845445695 shutdown_connections 2024-09-16T00:31:14.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.001+0000 7fc38fecc640 1 -- 172.21.15.112:0/845445695 wait complete. 2024-09-16T00:31:14.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.002+0000 7fc38fecc640 1 Processor -- start 2024-09-16T00:31:14.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.002+0000 7fc38fecc640 1 -- start start 2024-09-16T00:31:14.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.002+0000 7fc38fecc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3880fb810 0x7fc3880f8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:14.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.002+0000 7fc38fecc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3880fcf80 0x7fc3880f8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:14.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.002+0000 7fc38fecc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3880f8fc0 con 0x7fc3880fb810 2024-09-16T00:31:14.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.002+0000 7fc38fecc640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3880f9130 con 0x7fc3880fcf80 2024-09-16T00:31:14.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.002+0000 7fc38dc68640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3880fb810 0x7fc3880f8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:14.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.002+0000 7fc38dc68640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3880fb810 0x7fc3880f8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60978/0 (socket says 172.21.15.112:60978) 2024-09-16T00:31:14.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.002+0000 7fc38dc68640 1 -- 172.21.15.112:0/2175988222 learned_addr learned my addr 172.21.15.112:0/2175988222 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:31:14.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.003+0000 7fc38d467640 1 --2- 172.21.15.112:0/2175988222 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3880fcf80 0x7fc3880f8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:14.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.003+0000 7fc38dc68640 1 -- 172.21.15.112:0/2175988222 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3880fcf80 msgr2=0x7fc3880f8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:14.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.003+0000 7fc38dc68640 1 --2- 172.21.15.112:0/2175988222 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3880fcf80 0x7fc3880f8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:14.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.003+0000 7fc38dc68640 1 -- 172.21.15.112:0/2175988222 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc384005040 con 0x7fc3880fb810 2024-09-16T00:31:14.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.003+0000 7fc38dc68640 1 --2- 172.21.15.112:0/2175988222 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3880fb810 0x7fc3880f8540 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc384000c00 tx=0x7fc384009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:14.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.003+0000 7fc37e7fc640 1 -- 172.21.15.112:0/2175988222 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc384038070 con 0x7fc3880fb810 2024-09-16T00:31:14.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.003+0000 7fc38fecc640 1 -- 172.21.15.112:0/2175988222 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3880f9360 con 0x7fc3880fb810 2024-09-16T00:31:14.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.003+0000 7fc37e7fc640 1 -- 172.21.15.112:0/2175988222 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fc38402ade0 con 0x7fc3880fb810 2024-09-16T00:31:14.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.003+0000 7fc37e7fc640 1 -- 172.21.15.112:0/2175988222 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc38402c7b0 con 0x7fc3880fb810 2024-09-16T00:31:14.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.003+0000 7fc38fecc640 1 -- 172.21.15.112:0/2175988222 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc388119830 con 0x7fc3880fb810 2024-09-16T00:31:14.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.004+0000 7fc38fecc640 1 -- 172.21.15.112:0/2175988222 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3880fcbc0 con 0x7fc3880fb810 2024-09-16T00:31:14.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.005+0000 7fc37e7fc640 1 -- 172.21.15.112:0/2175988222 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fc384030020 con 0x7fc3880fb810 2024-09-16T00:31:14.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.006+0000 7fc37e7fc640 1 --2- 172.21.15.112:0/2175988222 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fc37406e530 0x7fc3740709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:14.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.006+0000 7fc37e7fc640 1 -- 172.21.15.112:0/2175988222 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fc3840ae650 con 0x7fc3880fb810 2024-09-16T00:31:14.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.006+0000 7fc38d467640 1 --2- 172.21.15.112:0/2175988222 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fc37406e530 0x7fc3740709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:14.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.006+0000 7fc38d467640 1 --2- 172.21.15.112:0/2175988222 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fc37406e530 0x7fc3740709e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc3880f9e80 tx=0x7fc37800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:14.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.010+0000 7fc37e7fc640 1 -- 172.21.15.112:0/2175988222 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fc384077790 con 0x7fc3880fb810 2024-09-16T00:31:14.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.219+0000 7fc38fecc640 1 -- 172.21.15.112:0/2175988222 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc3880f9c30 con 0x7fc37406e530 2024-09-16T00:31:14.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.232+0000 7fc37e7fc640 1 -- 172.21.15.112:0/2175988222 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fc3880f9c30 con 0x7fc37406e530 2024-09-16T00:31:14.233 INFO:teuthology.orchestra.run.smithi112.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T00:31:14.233 INFO:teuthology.orchestra.run.smithi112.stdout:alertmanager.smithi112 smithi112 *:9093,9094 running (4m) 2s ago 13m 25.2M - 0.25.0 c8568f914cd2 d85ea0562022 2024-09-16T00:31:14.233 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi112 smithi112 running (68s) 2s ago 13m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c e4fe58f9702a 2024-09-16T00:31:14.233 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi169 smithi169 running (63s) 60s ago 12m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c 3ba2c81d527f 2024-09-16T00:31:14.234 INFO:teuthology.orchestra.run.smithi112.stdout:grafana.smithi112 smithi112 *:3000 running (2m) 2s ago 12m 77.5M - 9.4.12 2bacad6d85d8 51f9d2013da2 2024-09-16T00:31:14.234 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.hxfwkh smithi112 running (7m) 2s ago 7m 135M - 17.2.7-1633-g6d025bc2 33ed8dba3996 cf92d9461fc6 2024-09-16T00:31:14.234 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.ithwzj smithi112 running (7m) 2s ago 7m 66.0M - 17.2.7-1633-g6d025bc2 33ed8dba3996 02786c8a83c7 2024-09-16T00:31:14.234 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.gawwgw smithi169 running (7m) 60s ago 7m 114M - 17.2.7-1633-g6d025bc2 33ed8dba3996 dcf3a4dfa879 2024-09-16T00:31:14.234 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.lgeqrj smithi169 running (7m) 60s ago 7m 60.4M - 17.2.7-1633-g6d025bc2 33ed8dba3996 445c14e55831 2024-09-16T00:31:14.234 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi112.xrkkbc smithi112 *:8443,9283,8765 running (3m) 2s ago 14m 496M - 19.1.1-326-g8921c8ef f1455bd0219c a981f7fd2810 2024-09-16T00:31:14.234 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi169.mtrobk smithi169 *:8443,9283,8765 running (3m) 60s ago 12m 439M - 19.1.1-326-g8921c8ef f1455bd0219c 4096e5369c2d 2024-09-16T00:31:14.234 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi112 smithi112 running (2m) 2s ago 14m 44.7M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 5aaec033c042 2024-09-16T00:31:14.234 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi169 smithi169 running (102s) 60s ago 12m 31.8M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 7bc609155f3f 2024-09-16T00:31:14.234 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi112 smithi112 *:9100 running (4m) 2s ago 13m 15.5M - 1.5.0 0da6a335fe13 6404d1d601e2 2024-09-16T00:31:14.234 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi169 smithi169 *:9100 running (3m) 60s ago 12m 15.2M - 1.5.0 0da6a335fe13 5505baf702a6 2024-09-16T00:31:14.234 INFO:teuthology.orchestra.run.smithi112.stdout:osd.0 smithi112 running (43s) 2s ago 11m 87.9M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 53efc9397017 2024-09-16T00:31:14.234 INFO:teuthology.orchestra.run.smithi112.stdout:osd.1 smithi112 running (8s) 2s ago 10m 12.1M 1557M 19.1.1-326-g8921c8ef f1455bd0219c fe6f1a4a2cf4 2024-09-16T00:31:14.234 INFO:teuthology.orchestra.run.smithi112.stdout:osd.2 smithi112 running (10m) 2s ago 10m 159M 1557M 17.2.7-1633-g6d025bc2 33ed8dba3996 100ec4e4852a 2024-09-16T00:31:14.234 INFO:teuthology.orchestra.run.smithi112.stdout:osd.3 smithi169 running (10m) 60s ago 10m 231M 2581M 17.2.7-1633-g6d025bc2 33ed8dba3996 f88463e19f23 2024-09-16T00:31:14.235 INFO:teuthology.orchestra.run.smithi112.stdout:osd.4 smithi169 running (9m) 60s ago 9m 211M 2581M 17.2.7-1633-g6d025bc2 33ed8dba3996 d3e4cdb016b9 2024-09-16T00:31:14.235 INFO:teuthology.orchestra.run.smithi112.stdout:osd.5 smithi169 running (9m) 60s ago 9m 199M 2581M 17.2.7-1633-g6d025bc2 33ed8dba3996 8e7d708d74eb 2024-09-16T00:31:14.235 INFO:teuthology.orchestra.run.smithi112.stdout:prometheus.smithi112 smithi112 *:9095 running (3m) 2s ago 12m 63.2M - 2.43.0 a07b618ecd1d 7a1d42e60d2b 2024-09-16T00:31:14.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.234+0000 7fc38fecc640 1 -- 172.21.15.112:0/2175988222 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fc37406e530 msgr2=0x7fc3740709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:14.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.234+0000 7fc38fecc640 1 --2- 172.21.15.112:0/2175988222 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fc37406e530 0x7fc3740709e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc3880f9e80 tx=0x7fc37800b040 comp rx=0 tx=0).stop 2024-09-16T00:31:14.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.234+0000 7fc38fecc640 1 -- 172.21.15.112:0/2175988222 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3880fb810 msgr2=0x7fc3880f8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:14.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.234+0000 7fc38fecc640 1 --2- 172.21.15.112:0/2175988222 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3880fb810 0x7fc3880f8540 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc384000c00 tx=0x7fc384009da0 comp rx=0 tx=0).stop 2024-09-16T00:31:14.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.234+0000 7fc38fecc640 1 -- 172.21.15.112:0/2175988222 shutdown_connections 2024-09-16T00:31:14.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.234+0000 7fc38fecc640 1 --2- 172.21.15.112:0/2175988222 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fc37406e530 0x7fc3740709e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:14.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.234+0000 7fc38fecc640 1 --2- 172.21.15.112:0/2175988222 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3880fcf80 0x7fc3880f8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:14.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.234+0000 7fc38fecc640 1 --2- 172.21.15.112:0/2175988222 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3880fb810 0x7fc3880f8540 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:14.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.234+0000 7fc38fecc640 1 -- 172.21.15.112:0/2175988222 >> 172.21.15.112:0/2175988222 conn(0x7fc3880723b0 msgr2=0x7fc388064f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:14.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.234+0000 7fc38fecc640 1 -- 172.21.15.112:0/2175988222 shutdown_connections 2024-09-16T00:31:14.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.234+0000 7fc38fecc640 1 -- 172.21.15.112:0/2175988222 wait complete. 2024-09-16T00:31:14.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.324+0000 7f14f33c0640 1 -- 172.21.15.112:0/3298766704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14ec102f10 msgr2=0x7f14ec0fc5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:14.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.324+0000 7f14f33c0640 1 --2- 172.21.15.112:0/3298766704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14ec102f10 0x7f14ec0fc5d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f14dc0048d0 tx=0x7f14dc009da0 comp rx=0 tx=0).stop 2024-09-16T00:31:14.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.324+0000 7f14f33c0640 1 -- 172.21.15.112:0/3298766704 shutdown_connections 2024-09-16T00:31:14.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.324+0000 7f14f33c0640 1 --2- 172.21.15.112:0/3298766704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14ec102f10 0x7f14ec0fc5d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:14.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.324+0000 7f14f33c0640 1 --2- 172.21.15.112:0/3298766704 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14ec102570 0x7f14ec102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:14.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.324+0000 7f14f33c0640 1 -- 172.21.15.112:0/3298766704 >> 172.21.15.112:0/3298766704 conn(0x7f14ec0f8110 msgr2=0x7f14ec0fa520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:14.327 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:14 smithi112 ceph-mon[97750]: from='osd.1 [v2:172.21.15.112:6810/2825714091,v1:172.21.15.112:6811/2825714091]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-16T00:31:14.328 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:14 smithi112 ceph-mon[97750]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:31:14.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.325+0000 7f14f33c0640 1 -- 172.21.15.112:0/3298766704 shutdown_connections 2024-09-16T00:31:14.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.325+0000 7f14f33c0640 1 -- 172.21.15.112:0/3298766704 wait complete. 2024-09-16T00:31:14.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.326+0000 7f14f33c0640 1 Processor -- start 2024-09-16T00:31:14.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.326+0000 7f14f33c0640 1 -- start start 2024-09-16T00:31:14.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.326+0000 7f14f33c0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14ec102570 0x7f14ec1131b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:14.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.326+0000 7f14f33c0640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14ec102f10 0x7f14ec1136f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:14.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.326+0000 7f14f33c0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ec113cf0 con 0x7f14ec102570 2024-09-16T00:31:14.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.326+0000 7f14f33c0640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ec10d130 con 0x7f14ec102f10 2024-09-16T00:31:14.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.326+0000 7f14f115c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14ec102570 0x7f14ec1131b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:14.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.326+0000 7f14f115c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14ec102570 0x7f14ec1131b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60996/0 (socket says 172.21.15.112:60996) 2024-09-16T00:31:14.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.326+0000 7f14f115c640 1 -- 172.21.15.112:0/1955303756 learned_addr learned my addr 172.21.15.112:0/1955303756 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:31:14.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.326+0000 7f14f095b640 1 --2- 172.21.15.112:0/1955303756 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14ec102f10 0x7f14ec1136f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:14.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.327+0000 7f14f115c640 1 -- 172.21.15.112:0/1955303756 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14ec102f10 msgr2=0x7f14ec1136f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:14.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.327+0000 7f14f115c640 1 --2- 172.21.15.112:0/1955303756 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14ec102f10 0x7f14ec1136f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:14.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.327+0000 7f14f115c640 1 -- 172.21.15.112:0/1955303756 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14dc005040 con 0x7f14ec102570 2024-09-16T00:31:14.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.327+0000 7f14f115c640 1 --2- 172.21.15.112:0/1955303756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14ec102570 0x7f14ec1131b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f14e8002a50 tx=0x7f14e8004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:14.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.327+0000 7f14e1ffb640 1 -- 172.21.15.112:0/1955303756 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f14e8007c30 con 0x7f14ec102570 2024-09-16T00:31:14.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.327+0000 7f14f33c0640 1 -- 172.21.15.112:0/1955303756 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14ec10d410 con 0x7f14ec102570 2024-09-16T00:31:14.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.327+0000 7f14e1ffb640 1 -- 172.21.15.112:0/1955303756 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f14e800f040 con 0x7f14ec102570 2024-09-16T00:31:14.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.327+0000 7f14e1ffb640 1 -- 172.21.15.112:0/1955303756 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f14e800db50 con 0x7f14ec102570 2024-09-16T00:31:14.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.328+0000 7f14f33c0640 1 -- 172.21.15.112:0/1955303756 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14ec10d980 con 0x7f14ec102570 2024-09-16T00:31:14.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.329+0000 7f14f33c0640 1 -- 172.21.15.112:0/1955303756 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14ec0fdcf0 con 0x7f14ec102570 2024-09-16T00:31:14.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.329+0000 7f14e1ffb640 1 -- 172.21.15.112:0/1955303756 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f14e800dcb0 con 0x7f14ec102570 2024-09-16T00:31:14.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.330+0000 7f14e1ffb640 1 --2- 172.21.15.112:0/1955303756 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f14d806e530 0x7f14d80709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:14.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.330+0000 7f14e1ffb640 1 -- 172.21.15.112:0/1955303756 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f14e808e110 con 0x7f14ec102570 2024-09-16T00:31:14.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.330+0000 7f14f095b640 1 --2- 172.21.15.112:0/1955303756 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f14d806e530 0x7f14d80709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:14.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.331+0000 7f14f095b640 1 --2- 172.21.15.112:0/1955303756 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f14d806e530 0x7f14d80709e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f14dc000c00 tx=0x7f14dc035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:14.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.336+0000 7f14e1ffb640 1 -- 172.21.15.112:0/1955303756 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f14e8057250 con 0x7f14ec102570 2024-09-16T00:31:14.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:14 smithi169 ceph-mon[77626]: from='osd.1 [v2:172.21.15.112:6810/2825714091,v1:172.21.15.112:6811/2825714091]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-16T00:31:14.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:14 smithi169 ceph-mon[77626]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:31:14.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.700+0000 7f14f33c0640 1 -- 172.21.15.112:0/1955303756 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f14ec04b720 con 0x7f14ec102570 2024-09-16T00:31:14.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.701+0000 7f14e1ffb640 1 -- 172.21.15.112:0/1955303756 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+850 (secure 0 0 0) 0x7f14e805a910 con 0x7f14ec102570 2024-09-16T00:31:14.703 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:31:14.703 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": { 2024-09-16T00:31:14.703 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:31:14.703 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:31:14.703 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": { 2024-09-16T00:31:14.703 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:31:14.703 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:31:14.703 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": { 2024-09-16T00:31:14.703 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 17.2.7-1633-g6d025bc2 (6d025bc26f3176c910707f8cb4c8d8e076640615) quincy (stable)": 4, 2024-09-16T00:31:14.703 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 1 2024-09-16T00:31:14.703 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:31:14.703 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": { 2024-09-16T00:31:14.703 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 17.2.7-1633-g6d025bc2 (6d025bc26f3176c910707f8cb4c8d8e076640615) quincy (stable)": 4 2024-09-16T00:31:14.703 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:31:14.704 INFO:teuthology.orchestra.run.smithi112.stdout: "overall": { 2024-09-16T00:31:14.704 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 17.2.7-1633-g6d025bc2 (6d025bc26f3176c910707f8cb4c8d8e076640615) quincy (stable)": 8, 2024-09-16T00:31:14.704 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 5 2024-09-16T00:31:14.704 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-16T00:31:14.704 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:31:14.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.704+0000 7f14f33c0640 1 -- 172.21.15.112:0/1955303756 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f14d806e530 msgr2=0x7f14d80709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:14.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.704+0000 7f14f33c0640 1 --2- 172.21.15.112:0/1955303756 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f14d806e530 0x7f14d80709e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f14dc000c00 tx=0x7f14dc035040 comp rx=0 tx=0).stop 2024-09-16T00:31:14.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.704+0000 7f14f33c0640 1 -- 172.21.15.112:0/1955303756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14ec102570 msgr2=0x7f14ec1131b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:14.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.704+0000 7f14f33c0640 1 --2- 172.21.15.112:0/1955303756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14ec102570 0x7f14ec1131b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f14e8002a50 tx=0x7f14e8004600 comp rx=0 tx=0).stop 2024-09-16T00:31:14.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.704+0000 7f14f33c0640 1 -- 172.21.15.112:0/1955303756 shutdown_connections 2024-09-16T00:31:14.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.704+0000 7f14f33c0640 1 --2- 172.21.15.112:0/1955303756 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f14d806e530 0x7f14d80709e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:14.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.704+0000 7f14f33c0640 1 --2- 172.21.15.112:0/1955303756 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14ec102f10 0x7f14ec1136f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:14.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.704+0000 7f14f33c0640 1 --2- 172.21.15.112:0/1955303756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14ec102570 0x7f14ec1131b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:14.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.704+0000 7f14f33c0640 1 -- 172.21.15.112:0/1955303756 >> 172.21.15.112:0/1955303756 conn(0x7f14ec0f8110 msgr2=0x7f14ec0f90a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:14.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.704+0000 7f14f33c0640 1 -- 172.21.15.112:0/1955303756 shutdown_connections 2024-09-16T00:31:14.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.704+0000 7f14f33c0640 1 -- 172.21.15.112:0/1955303756 wait complete. 2024-09-16T00:31:14.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.819+0000 7f13e8ab6640 1 -- 172.21.15.112:0/3470947205 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13e40fd340 msgr2=0x7f13e40fd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:14.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.819+0000 7f13e8ab6640 1 --2- 172.21.15.112:0/3470947205 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13e40fd340 0x7f13e40fd710 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f13d40048d0 tx=0x7f13d4009da0 comp rx=0 tx=0).stop 2024-09-16T00:31:14.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.819+0000 7f13e8ab6640 1 -- 172.21.15.112:0/3470947205 shutdown_connections 2024-09-16T00:31:14.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.819+0000 7f13e8ab6640 1 --2- 172.21.15.112:0/3470947205 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13e40fbcf0 0x7f13e40fc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:14.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.819+0000 7f13e8ab6640 1 --2- 172.21.15.112:0/3470947205 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13e40fd340 0x7f13e40fd710 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:14.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.819+0000 7f13e8ab6640 1 -- 172.21.15.112:0/3470947205 >> 172.21.15.112:0/3470947205 conn(0x7f13e4071b00 msgr2=0x7f13e4071f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:14.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.820+0000 7f13e8ab6640 1 -- 172.21.15.112:0/3470947205 shutdown_connections 2024-09-16T00:31:14.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.820+0000 7f13e8ab6640 1 -- 172.21.15.112:0/3470947205 wait complete. 2024-09-16T00:31:14.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.820+0000 7f13e8ab6640 1 Processor -- start 2024-09-16T00:31:14.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.820+0000 7f13e8ab6640 1 -- start start 2024-09-16T00:31:14.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.821+0000 7f13e8ab6640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13e40fbcf0 0x7f13e410df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:14.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.821+0000 7f13e8ab6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13e40fd340 0x7f13e410e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:14.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.821+0000 7f13e8ab6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e410ead0 con 0x7f13e40fd340 2024-09-16T00:31:14.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.821+0000 7f13e8ab6640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e410ec40 con 0x7f13e40fbcf0 2024-09-16T00:31:14.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.821+0000 7f13dbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13e40fd340 0x7f13e410e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:14.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.821+0000 7f13dbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13e40fd340 0x7f13e410e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32786/0 (socket says 172.21.15.112:32786) 2024-09-16T00:31:14.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.821+0000 7f13dbfff640 1 -- 172.21.15.112:0/2289840108 learned_addr learned my addr 172.21.15.112:0/2289840108 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:31:14.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.822+0000 7f13e259c640 1 --2- 172.21.15.112:0/2289840108 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13e40fbcf0 0x7f13e410df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:14.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.822+0000 7f13dbfff640 1 -- 172.21.15.112:0/2289840108 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13e40fbcf0 msgr2=0x7f13e410df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:14.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.822+0000 7f13dbfff640 1 --2- 172.21.15.112:0/2289840108 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13e40fbcf0 0x7f13e410df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:14.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.822+0000 7f13dbfff640 1 -- 172.21.15.112:0/2289840108 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d4005040 con 0x7f13e40fd340 2024-09-16T00:31:14.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.822+0000 7f13dbfff640 1 --2- 172.21.15.112:0/2289840108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13e40fd340 0x7f13e410e480 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f13cc00eb70 tx=0x7f13cc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:14.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.822+0000 7f13daffd640 1 -- 172.21.15.112:0/2289840108 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13cc007c30 con 0x7f13e40fd340 2024-09-16T00:31:14.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.823+0000 7f13daffd640 1 -- 172.21.15.112:0/2289840108 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f13cc00f040 con 0x7f13e40fd340 2024-09-16T00:31:14.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.823+0000 7f13daffd640 1 -- 172.21.15.112:0/2289840108 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13cc009550 con 0x7f13e40fd340 2024-09-16T00:31:14.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.823+0000 7f13e8ab6640 1 -- 172.21.15.112:0/2289840108 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13e4116fb0 con 0x7f13e40fd340 2024-09-16T00:31:14.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.823+0000 7f13e8ab6640 1 -- 172.21.15.112:0/2289840108 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13e41173f0 con 0x7f13e40fd340 2024-09-16T00:31:14.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.825+0000 7f13e8ab6640 1 -- 172.21.15.112:0/2289840108 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13e40fcd00 con 0x7f13e40fd340 2024-09-16T00:31:14.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.826+0000 7f13daffd640 1 -- 172.21.15.112:0/2289840108 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f13cc00cca0 con 0x7f13e40fd340 2024-09-16T00:31:14.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.826+0000 7f13daffd640 1 --2- 172.21.15.112:0/2289840108 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f13d006e530 0x7f13d00709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:14.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.827+0000 7f13daffd640 1 -- 172.21.15.112:0/2289840108 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f13cc08db30 con 0x7f13e40fd340 2024-09-16T00:31:14.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.827+0000 7f13e259c640 1 --2- 172.21.15.112:0/2289840108 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f13d006e530 0x7f13d00709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:14.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.828+0000 7f13e259c640 1 --2- 172.21.15.112:0/2289840108 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f13d006e530 0x7f13d00709e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f13d4000c00 tx=0x7f13d4035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:14.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:14.835+0000 7f13daffd640 1 -- 172.21.15.112:0/2289840108 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f13cc056da0 con 0x7f13e40fd340 2024-09-16T00:31:15.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.104+0000 7f13e8ab6640 1 -- 172.21.15.112:0/2289840108 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f13e404b720 con 0x7f13e40fd340 2024-09-16T00:31:15.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.104+0000 7f13daffd640 1 -- 172.21.15.112:0/2289840108 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) v1 ==== 76+0+1954 (secure 0 0 0) 0x7f13cc05a460 con 0x7f13e40fd340 2024-09-16T00:31:15.106 INFO:teuthology.orchestra.run.smithi112.stdout:e23 2024-09-16T00:31:15.106 INFO:teuthology.orchestra.run.smithi112.stdout:btime 2024-09-16T00:30:08:788506+0000 2024-09-16T00:31:15.106 INFO:teuthology.orchestra.run.smithi112.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T00:31:15.106 INFO:teuthology.orchestra.run.smithi112.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:31:15.106 INFO:teuthology.orchestra.run.smithi112.stdout:legacy client fscid: 1 2024-09-16T00:31:15.106 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:31:15.106 INFO:teuthology.orchestra.run.smithi112.stdout:Filesystem 'cephfs' (1) 2024-09-16T00:31:15.107 INFO:teuthology.orchestra.run.smithi112.stdout:fs_name cephfs 2024-09-16T00:31:15.107 INFO:teuthology.orchestra.run.smithi112.stdout:epoch 23 2024-09-16T00:31:15.107 INFO:teuthology.orchestra.run.smithi112.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-16T00:31:15.107 INFO:teuthology.orchestra.run.smithi112.stdout:created 2024-09-16T00:23:22.328199+0000 2024-09-16T00:31:15.107 INFO:teuthology.orchestra.run.smithi112.stdout:modified 2024-09-16T00:30:08.396661+0000 2024-09-16T00:31:15.107 INFO:teuthology.orchestra.run.smithi112.stdout:tableserver 0 2024-09-16T00:31:15.107 INFO:teuthology.orchestra.run.smithi112.stdout:root 0 2024-09-16T00:31:15.107 INFO:teuthology.orchestra.run.smithi112.stdout:session_timeout 60 2024-09-16T00:31:15.107 INFO:teuthology.orchestra.run.smithi112.stdout:session_autoclose 300 2024-09-16T00:31:15.107 INFO:teuthology.orchestra.run.smithi112.stdout:max_file_size 1099511627776 2024-09-16T00:31:15.107 INFO:teuthology.orchestra.run.smithi112.stdout:max_xattr_size 65536 2024-09-16T00:31:15.107 INFO:teuthology.orchestra.run.smithi112.stdout:required_client_features {} 2024-09-16T00:31:15.107 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure 0 2024-09-16T00:31:15.107 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure_osd_epoch 44 2024-09-16T00:31:15.108 INFO:teuthology.orchestra.run.smithi112.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:31:15.108 INFO:teuthology.orchestra.run.smithi112.stdout:max_mds 2 2024-09-16T00:31:15.108 INFO:teuthology.orchestra.run.smithi112.stdout:in 0,1 2024-09-16T00:31:15.108 INFO:teuthology.orchestra.run.smithi112.stdout:up {0=14480,1=24259} 2024-09-16T00:31:15.108 INFO:teuthology.orchestra.run.smithi112.stdout:failed 2024-09-16T00:31:15.108 INFO:teuthology.orchestra.run.smithi112.stdout:damaged 2024-09-16T00:31:15.108 INFO:teuthology.orchestra.run.smithi112.stdout:stopped 2024-09-16T00:31:15.108 INFO:teuthology.orchestra.run.smithi112.stdout:data_pools [3] 2024-09-16T00:31:15.108 INFO:teuthology.orchestra.run.smithi112.stdout:metadata_pool 2 2024-09-16T00:31:15.108 INFO:teuthology.orchestra.run.smithi112.stdout:inline_data enabled 2024-09-16T00:31:15.108 INFO:teuthology.orchestra.run.smithi112.stdout:balancer 2024-09-16T00:31:15.108 INFO:teuthology.orchestra.run.smithi112.stdout:bal_rank_mask -1 2024-09-16T00:31:15.108 INFO:teuthology.orchestra.run.smithi112.stdout:standby_count_wanted 1 2024-09-16T00:31:15.108 INFO:teuthology.orchestra.run.smithi112.stdout:qdb_cluster leader: 14480 members: 24259,14480 2024-09-16T00:31:15.109 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.hxfwkh{0:14480} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.112:6828/486654885,v1:172.21.15.112:6829/486654885] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T00:31:15.109 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.ithwzj{0:14492} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.112:6826/381830565,v1:172.21.15.112:6827/381830565] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T00:31:15.109 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.gawwgw{1:24259} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.169:6824/2892476953,v1:172.21.15.169:6825/2892476953] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T00:31:15.109 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.lgeqrj{1:24269} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.169:6826/861725728,v1:172.21.15.169:6827/861725728] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T00:31:15.109 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:31:15.109 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:31:15.110 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 23 2024-09-16T00:31:15.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.106+0000 7f13e8ab6640 1 -- 172.21.15.112:0/2289840108 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f13d006e530 msgr2=0x7f13d00709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:15.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.106+0000 7f13e8ab6640 1 --2- 172.21.15.112:0/2289840108 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f13d006e530 0x7f13d00709e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f13d4000c00 tx=0x7f13d4035040 comp rx=0 tx=0).stop 2024-09-16T00:31:15.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.106+0000 7f13e8ab6640 1 -- 172.21.15.112:0/2289840108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13e40fd340 msgr2=0x7f13e410e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:15.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.106+0000 7f13e8ab6640 1 --2- 172.21.15.112:0/2289840108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13e40fd340 0x7f13e410e480 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f13cc00eb70 tx=0x7f13cc00b040 comp rx=0 tx=0).stop 2024-09-16T00:31:15.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.106+0000 7f13e8ab6640 1 -- 172.21.15.112:0/2289840108 shutdown_connections 2024-09-16T00:31:15.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.106+0000 7f13e8ab6640 1 --2- 172.21.15.112:0/2289840108 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f13d006e530 0x7f13d00709e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:15.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.106+0000 7f13e8ab6640 1 --2- 172.21.15.112:0/2289840108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13e40fd340 0x7f13e410e480 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:15.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.106+0000 7f13e8ab6640 1 --2- 172.21.15.112:0/2289840108 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13e40fbcf0 0x7f13e410df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:15.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.106+0000 7f13e8ab6640 1 -- 172.21.15.112:0/2289840108 >> 172.21.15.112:0/2289840108 conn(0x7f13e4071b00 msgr2=0x7f13e40fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:15.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.106+0000 7f13e8ab6640 1 -- 172.21.15.112:0/2289840108 shutdown_connections 2024-09-16T00:31:15.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.106+0000 7f13e8ab6640 1 -- 172.21.15.112:0/2289840108 wait complete. 2024-09-16T00:31:15.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.188+0000 7fe4bea45640 1 -- 172.21.15.112:0/869484283 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4b81023d0 msgr2=0x7fe4b81027a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:15.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.188+0000 7fe4bea45640 1 --2- 172.21.15.112:0/869484283 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4b81023d0 0x7fe4b81027a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe4b40048d0 tx=0x7fe4b4009da0 comp rx=0 tx=0).stop 2024-09-16T00:31:15.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.188+0000 7fe4bea45640 1 -- 172.21.15.112:0/869484283 shutdown_connections 2024-09-16T00:31:15.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.188+0000 7fe4bea45640 1 --2- 172.21.15.112:0/869484283 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4b8102d70 0x7fe4b80fc400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:15.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.188+0000 7fe4bea45640 1 --2- 172.21.15.112:0/869484283 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4b81023d0 0x7fe4b81027a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:15.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.188+0000 7fe4bea45640 1 -- 172.21.15.112:0/869484283 >> 172.21.15.112:0/869484283 conn(0x7fe4b80f8000 msgr2=0x7fe4b80fa410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:15.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.188+0000 7fe4bea45640 1 -- 172.21.15.112:0/869484283 shutdown_connections 2024-09-16T00:31:15.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.188+0000 7fe4bea45640 1 -- 172.21.15.112:0/869484283 wait complete. 2024-09-16T00:31:15.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.189+0000 7fe4bea45640 1 Processor -- start 2024-09-16T00:31:15.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.189+0000 7fe4bea45640 1 -- start start 2024-09-16T00:31:15.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.189+0000 7fe4bea45640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4b81023d0 0x7fe4b8112fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:15.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.189+0000 7fe4bea45640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4b8102d70 0x7fe4b8113500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:15.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.189+0000 7fe4bea45640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b8113b00 con 0x7fe4b81023d0 2024-09-16T00:31:15.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.189+0000 7fe4bea45640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b810cf40 con 0x7fe4b8102d70 2024-09-16T00:31:15.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.189+0000 7fe4bda43640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4b81023d0 0x7fe4b8112fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:15.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.189+0000 7fe4bda43640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4b81023d0 0x7fe4b8112fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32800/0 (socket says 172.21.15.112:32800) 2024-09-16T00:31:15.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.189+0000 7fe4bda43640 1 -- 172.21.15.112:0/999981391 learned_addr learned my addr 172.21.15.112:0/999981391 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:31:15.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.189+0000 7fe4bd242640 1 --2- 172.21.15.112:0/999981391 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4b8102d70 0x7fe4b8113500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:15.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.189+0000 7fe4bda43640 1 -- 172.21.15.112:0/999981391 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4b8102d70 msgr2=0x7fe4b8113500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:15.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.189+0000 7fe4bda43640 1 --2- 172.21.15.112:0/999981391 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4b8102d70 0x7fe4b8113500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:15.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.189+0000 7fe4bda43640 1 -- 172.21.15.112:0/999981391 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4b4005040 con 0x7fe4b81023d0 2024-09-16T00:31:15.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.189+0000 7fe4bda43640 1 --2- 172.21.15.112:0/999981391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4b81023d0 0x7fe4b8112fc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe4b40043d0 tx=0x7fe4b4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:15.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.190+0000 7fe4ae7fc640 1 -- 172.21.15.112:0/999981391 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4b4038070 con 0x7fe4b81023d0 2024-09-16T00:31:15.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.190+0000 7fe4ae7fc640 1 -- 172.21.15.112:0/999981391 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fe4b40096e0 con 0x7fe4b81023d0 2024-09-16T00:31:15.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.190+0000 7fe4bea45640 1 -- 172.21.15.112:0/999981391 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4b810d1c0 con 0x7fe4b81023d0 2024-09-16T00:31:15.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.190+0000 7fe4ae7fc640 1 -- 172.21.15.112:0/999981391 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4b403d420 con 0x7fe4b81023d0 2024-09-16T00:31:15.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.190+0000 7fe4bea45640 1 -- 172.21.15.112:0/999981391 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4b810d760 con 0x7fe4b81023d0 2024-09-16T00:31:15.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.191+0000 7fe4bea45640 1 -- 172.21.15.112:0/999981391 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4b80fdb40 con 0x7fe4b81023d0 2024-09-16T00:31:15.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.191+0000 7fe4ae7fc640 1 -- 172.21.15.112:0/999981391 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fe4b4030020 con 0x7fe4b81023d0 2024-09-16T00:31:15.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.192+0000 7fe4ae7fc640 1 --2- 172.21.15.112:0/999981391 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fe4a406e540 0x7fe4a40709f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:15.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.192+0000 7fe4ae7fc640 1 -- 172.21.15.112:0/999981391 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fe4b40ae5a0 con 0x7fe4b81023d0 2024-09-16T00:31:15.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.192+0000 7fe4bd242640 1 --2- 172.21.15.112:0/999981391 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fe4a406e540 0x7fe4a40709f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:15.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.192+0000 7fe4bd242640 1 --2- 172.21.15.112:0/999981391 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fe4a406e540 0x7fe4a40709f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe4a8009fd0 tx=0x7fe4a8009e50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:15.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.196+0000 7fe4ae7fc640 1 -- 172.21.15.112:0/999981391 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fe4b4077760 con 0x7fe4b81023d0 2024-09-16T00:31:15.306 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:31:14 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-1[114685]: 2024-09-16T00:31:14.962+0000 7fa0edfc3640 -1 osd.1 52 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T00:31:15.306 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:15 smithi112 ceph-mon[97750]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:31:15.306 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:15 smithi112 ceph-mon[97750]: from='osd.1 [v2:172.21.15.112:6810/2825714091,v1:172.21.15.112:6811/2825714091]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-16T00:31:15.307 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:15 smithi112 ceph-mon[97750]: osdmap e55: 6 total, 5 up, 6 in 2024-09-16T00:31:15.307 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:15 smithi112 ceph-mon[97750]: from='osd.1 [v2:172.21.15.112:6810/2825714091,v1:172.21.15.112:6811/2825714091]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-09-16T00:31:15.307 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:15 smithi112 ceph-mon[97750]: from='client.34232 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:31:15.307 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:15 smithi112 ceph-mon[97750]: pgmap v70: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 325 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s; 57/372 objects degraded (15.323%) 2024-09-16T00:31:15.307 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:15 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/1955303756' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:31:15.307 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:15 smithi112 ceph-mon[97750]: from='osd.1 [v2:172.21.15.112:6810/2825714091,v1:172.21.15.112:6811/2825714091]' entity='osd.1' 2024-09-16T00:31:15.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.390+0000 7fe4bea45640 1 -- 172.21.15.112:0/999981391 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe4b804b720 con 0x7fe4a406e540 2024-09-16T00:31:15.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.392+0000 7fe4ae7fc640 1 -- 172.21.15.112:0/999981391 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fe4b804b720 con 0x7fe4a406e540 2024-09-16T00:31:15.393 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:31:15.393 INFO:teuthology.orchestra.run.smithi112.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80", 2024-09-16T00:31:15.394 INFO:teuthology.orchestra.run.smithi112.stdout: "in_progress": true, 2024-09-16T00:31:15.394 INFO:teuthology.orchestra.run.smithi112.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T00:31:15.394 INFO:teuthology.orchestra.run.smithi112.stdout: "services_complete": [ 2024-09-16T00:31:15.394 INFO:teuthology.orchestra.run.smithi112.stdout: "crash", 2024-09-16T00:31:15.394 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr", 2024-09-16T00:31:15.394 INFO:teuthology.orchestra.run.smithi112.stdout: "mon" 2024-09-16T00:31:15.394 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-16T00:31:15.394 INFO:teuthology.orchestra.run.smithi112.stdout: "progress": "8/21 daemons upgraded", 2024-09-16T00:31:15.394 INFO:teuthology.orchestra.run.smithi112.stdout: "message": "Currently upgrading osd daemons", 2024-09-16T00:31:15.394 INFO:teuthology.orchestra.run.smithi112.stdout: "is_paused": false 2024-09-16T00:31:15.394 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:31:15.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.394+0000 7fe4bea45640 1 -- 172.21.15.112:0/999981391 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fe4a406e540 msgr2=0x7fe4a40709f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:15.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.394+0000 7fe4bea45640 1 --2- 172.21.15.112:0/999981391 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fe4a406e540 0x7fe4a40709f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe4a8009fd0 tx=0x7fe4a8009e50 comp rx=0 tx=0).stop 2024-09-16T00:31:15.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.394+0000 7fe4bea45640 1 -- 172.21.15.112:0/999981391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4b81023d0 msgr2=0x7fe4b8112fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:15.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.394+0000 7fe4bea45640 1 --2- 172.21.15.112:0/999981391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4b81023d0 0x7fe4b8112fc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe4b40043d0 tx=0x7fe4b4009da0 comp rx=0 tx=0).stop 2024-09-16T00:31:15.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.394+0000 7fe4bea45640 1 -- 172.21.15.112:0/999981391 shutdown_connections 2024-09-16T00:31:15.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.394+0000 7fe4bea45640 1 --2- 172.21.15.112:0/999981391 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fe4a406e540 0x7fe4a40709f0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:15.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.394+0000 7fe4bea45640 1 --2- 172.21.15.112:0/999981391 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4b8102d70 0x7fe4b8113500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:15.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.394+0000 7fe4bea45640 1 --2- 172.21.15.112:0/999981391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4b81023d0 0x7fe4b8112fc0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:15.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.394+0000 7fe4bea45640 1 -- 172.21.15.112:0/999981391 >> 172.21.15.112:0/999981391 conn(0x7fe4b80f8000 msgr2=0x7fe4b80f9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:15.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.394+0000 7fe4bea45640 1 -- 172.21.15.112:0/999981391 shutdown_connections 2024-09-16T00:31:15.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.394+0000 7fe4bea45640 1 -- 172.21.15.112:0/999981391 wait complete. 2024-09-16T00:31:15.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.475+0000 7f8e181dc640 1 -- 172.21.15.112:0/2451753446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e10102f00 msgr2=0x7f8e100fc5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:15.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.475+0000 7f8e181dc640 1 --2- 172.21.15.112:0/2451753446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e10102f00 0x7f8e100fc5c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8e000048d0 tx=0x7f8e00009da0 comp rx=0 tx=0).stop 2024-09-16T00:31:15.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.475+0000 7f8e181dc640 1 -- 172.21.15.112:0/2451753446 shutdown_connections 2024-09-16T00:31:15.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.475+0000 7f8e181dc640 1 --2- 172.21.15.112:0/2451753446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e10102f00 0x7f8e100fc5c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:15.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.475+0000 7f8e181dc640 1 --2- 172.21.15.112:0/2451753446 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e10102560 0x7f8e10102930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:15.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.475+0000 7f8e181dc640 1 -- 172.21.15.112:0/2451753446 >> 172.21.15.112:0/2451753446 conn(0x7f8e100f8100 msgr2=0x7f8e100fa510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:15.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.475+0000 7f8e181dc640 1 -- 172.21.15.112:0/2451753446 shutdown_connections 2024-09-16T00:31:15.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.475+0000 7f8e181dc640 1 -- 172.21.15.112:0/2451753446 wait complete. 2024-09-16T00:31:15.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.476+0000 7f8e181dc640 1 Processor -- start 2024-09-16T00:31:15.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.476+0000 7f8e181dc640 1 -- start start 2024-09-16T00:31:15.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.476+0000 7f8e181dc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e10102560 0x7f8e10113200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:15.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.476+0000 7f8e181dc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e10102f00 0x7f8e10113740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:15.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.476+0000 7f8e181dc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e10113d40 con 0x7f8e10102560 2024-09-16T00:31:15.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.476+0000 7f8e181dc640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1010d180 con 0x7f8e10102f00 2024-09-16T00:31:15.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.476+0000 7f8e15f78640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e10102560 0x7f8e10113200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:15.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.476+0000 7f8e15f78640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e10102560 0x7f8e10113200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32820/0 (socket says 172.21.15.112:32820) 2024-09-16T00:31:15.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.476+0000 7f8e15f78640 1 -- 172.21.15.112:0/517107698 learned_addr learned my addr 172.21.15.112:0/517107698 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:31:15.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.476+0000 7f8e15f78640 1 -- 172.21.15.112:0/517107698 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e10102f00 msgr2=0x7f8e10113740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:15.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.476+0000 7f8e15f78640 1 --2- 172.21.15.112:0/517107698 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e10102f00 0x7f8e10113740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:15.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.476+0000 7f8e15f78640 1 -- 172.21.15.112:0/517107698 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e00005040 con 0x7f8e10102560 2024-09-16T00:31:15.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.477+0000 7f8e15f78640 1 --2- 172.21.15.112:0/517107698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e10102560 0x7f8e10113200 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c002a50 tx=0x7f8e0c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:15.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.477+0000 7f8e067fc640 1 -- 172.21.15.112:0/517107698 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e0c007c30 con 0x7f8e10102560 2024-09-16T00:31:15.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.477+0000 7f8e181dc640 1 -- 172.21.15.112:0/517107698 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e1010d460 con 0x7f8e10102560 2024-09-16T00:31:15.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.477+0000 7f8e181dc640 1 -- 172.21.15.112:0/517107698 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e1010da00 con 0x7f8e10102560 2024-09-16T00:31:15.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.477+0000 7f8e067fc640 1 -- 172.21.15.112:0/517107698 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f8e0c00f040 con 0x7f8e10102560 2024-09-16T00:31:15.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.478+0000 7f8e067fc640 1 -- 172.21.15.112:0/517107698 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e0c00da90 con 0x7f8e10102560 2024-09-16T00:31:15.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.478+0000 7f8e181dc640 1 -- 172.21.15.112:0/517107698 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8df4005210 con 0x7f8e10102560 2024-09-16T00:31:15.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.479+0000 7f8e067fc640 1 -- 172.21.15.112:0/517107698 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f8e0c00dcf0 con 0x7f8e10102560 2024-09-16T00:31:15.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.479+0000 7f8e067fc640 1 --2- 172.21.15.112:0/517107698 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f8dfc06e320 0x7f8dfc0707d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:15.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.479+0000 7f8e067fc640 1 -- 172.21.15.112:0/517107698 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f8e0c08d980 con 0x7f8e10102560 2024-09-16T00:31:15.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.479+0000 7f8e15777640 1 --2- 172.21.15.112:0/517107698 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f8dfc06e320 0x7f8dfc0707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:15.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.480+0000 7f8e15777640 1 --2- 172.21.15.112:0/517107698 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f8dfc06e320 0x7f8dfc0707d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8e00009ac0 tx=0x7f8e00009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:15.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.484+0000 7f8e067fc640 1 -- 172.21.15.112:0/517107698 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f8e0c056b40 con 0x7f8e10102560 2024-09-16T00:31:15.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:15 smithi169 ceph-mon[77626]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:31:15.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:15 smithi169 ceph-mon[77626]: from='osd.1 [v2:172.21.15.112:6810/2825714091,v1:172.21.15.112:6811/2825714091]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-16T00:31:15.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:15 smithi169 ceph-mon[77626]: osdmap e55: 6 total, 5 up, 6 in 2024-09-16T00:31:15.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:15 smithi169 ceph-mon[77626]: from='osd.1 [v2:172.21.15.112:6810/2825714091,v1:172.21.15.112:6811/2825714091]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-09-16T00:31:15.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:15 smithi169 ceph-mon[77626]: from='client.34232 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:31:15.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:15 smithi169 ceph-mon[77626]: pgmap v70: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 325 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s; 57/372 objects degraded (15.323%) 2024-09-16T00:31:15.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:15 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/1955303756' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:31:15.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:15 smithi169 ceph-mon[77626]: from='osd.1 [v2:172.21.15.112:6810/2825714091,v1:172.21.15.112:6811/2825714091]' entity='osd.1' 2024-09-16T00:31:15.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.745+0000 7f8e181dc640 1 -- 172.21.15.112:0/517107698 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f8df40054c0 con 0x7f8e10102560 2024-09-16T00:31:15.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.745+0000 7f8e067fc640 1 -- 172.21.15.112:0/517107698 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1350 (secure 0 0 0) 0x7f8e0c05a200 con 0x7f8e10102560 2024-09-16T00:31:15.747 INFO:teuthology.orchestra.run.smithi112.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; 1 osds down; Degraded data redundancy: 57/372 objects degraded (15.323%), 16 pgs degraded 2024-09-16T00:31:15.747 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T00:31:15.747 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T00:31:15.747 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] OSD_DOWN: 1 osds down 2024-09-16T00:31:15.747 INFO:teuthology.orchestra.run.smithi112.stdout: osd.1 (root=default,host=smithi112) is down 2024-09-16T00:31:15.747 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 57/372 objects degraded (15.323%), 16 pgs degraded 2024-09-16T00:31:15.747 INFO:teuthology.orchestra.run.smithi112.stdout: pg 1.0 is active+undersized+degraded, acting [3,0] 2024-09-16T00:31:15.747 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.0 is active+undersized+degraded, acting [3,0] 2024-09-16T00:31:15.747 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.1 is active+undersized+degraded, acting [2,0] 2024-09-16T00:31:15.747 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.2 is active+undersized+degraded, acting [5,0] 2024-09-16T00:31:15.747 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.3 is active+undersized+degraded, acting [5,2] 2024-09-16T00:31:15.747 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.4 is active+undersized+degraded, acting [0,4] 2024-09-16T00:31:15.747 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.6 is active+undersized+degraded, acting [3,4] 2024-09-16T00:31:15.747 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.9 is active+undersized+degraded, acting [4,0] 2024-09-16T00:31:15.748 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.a is active+undersized+degraded, acting [4,3] 2024-09-16T00:31:15.748 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.d is active+undersized+degraded, acting [3,2] 2024-09-16T00:31:15.748 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.10 is active+undersized+degraded, acting [2,0] 2024-09-16T00:31:15.748 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.11 is active+undersized+degraded, acting [4,3] 2024-09-16T00:31:15.748 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.12 is active+undersized+degraded, acting [3,0] 2024-09-16T00:31:15.748 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.15 is active+undersized+degraded, acting [3,0] 2024-09-16T00:31:15.748 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.17 is active+undersized+degraded, acting [5,2] 2024-09-16T00:31:15.748 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.1b is active+undersized+degraded, acting [0,5] 2024-09-16T00:31:15.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.747+0000 7f8e181dc640 1 -- 172.21.15.112:0/517107698 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f8dfc06e320 msgr2=0x7f8dfc0707d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:15.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.747+0000 7f8e181dc640 1 --2- 172.21.15.112:0/517107698 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f8dfc06e320 0x7f8dfc0707d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8e00009ac0 tx=0x7f8e00009da0 comp rx=0 tx=0).stop 2024-09-16T00:31:15.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.747+0000 7f8e181dc640 1 -- 172.21.15.112:0/517107698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e10102560 msgr2=0x7f8e10113200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:15.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.747+0000 7f8e181dc640 1 --2- 172.21.15.112:0/517107698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e10102560 0x7f8e10113200 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c002a50 tx=0x7f8e0c004600 comp rx=0 tx=0).stop 2024-09-16T00:31:15.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.748+0000 7f8e181dc640 1 -- 172.21.15.112:0/517107698 shutdown_connections 2024-09-16T00:31:15.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.748+0000 7f8e181dc640 1 --2- 172.21.15.112:0/517107698 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f8dfc06e320 0x7f8dfc0707d0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:15.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.748+0000 7f8e181dc640 1 --2- 172.21.15.112:0/517107698 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e10102f00 0x7f8e10113740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:15.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.748+0000 7f8e181dc640 1 --2- 172.21.15.112:0/517107698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e10102560 0x7f8e10113200 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:15.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.748+0000 7f8e181dc640 1 -- 172.21.15.112:0/517107698 >> 172.21.15.112:0/517107698 conn(0x7f8e100f8100 msgr2=0x7f8e100f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:15.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.748+0000 7f8e181dc640 1 -- 172.21.15.112:0/517107698 shutdown_connections 2024-09-16T00:31:15.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:15.748+0000 7f8e181dc640 1 -- 172.21.15.112:0/517107698 wait complete. 2024-09-16T00:31:16.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:16 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/2289840108' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:31:16.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:16 smithi112 ceph-mon[97750]: from='client.34244 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:31:16.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:16 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:31:16.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:16 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:31:16.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:16 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:31:16.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:16 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:31:16.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:16 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:31:16.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:16 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:31:16.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:16 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:31:16.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:16 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:31:16.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:16 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:31:16.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:16 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-09-16T00:31:16.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:16 smithi112 ceph-mon[97750]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-09-16T00:31:16.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:16 smithi112 ceph-mon[97750]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-09-16T00:31:16.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:16 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/517107698' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:31:16.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:16 smithi112 ceph-mon[97750]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-16T00:31:16.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:16 smithi112 ceph-mon[97750]: osd.1 [v2:172.21.15.112:6810/2825714091,v1:172.21.15.112:6811/2825714091] boot 2024-09-16T00:31:16.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:16 smithi112 ceph-mon[97750]: osdmap e56: 6 total, 6 up, 6 in 2024-09-16T00:31:16.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:16 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:31:16.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:16 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/2289840108' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:31:16.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:16 smithi169 ceph-mon[77626]: from='client.34244 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:31:16.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:16 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:31:16.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:16 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:31:16.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:16 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:31:16.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:16 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:31:16.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:16 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:31:16.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:16 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:31:16.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:16 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:31:16.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:16 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:31:16.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:16 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:31:16.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:16 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-09-16T00:31:16.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:16 smithi169 ceph-mon[77626]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-09-16T00:31:16.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:16 smithi169 ceph-mon[77626]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-09-16T00:31:16.528 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:16 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/517107698' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:31:16.528 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:16 smithi169 ceph-mon[77626]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-16T00:31:16.528 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:16 smithi169 ceph-mon[77626]: osd.1 [v2:172.21.15.112:6810/2825714091,v1:172.21.15.112:6811/2825714091] boot 2024-09-16T00:31:16.528 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:16 smithi169 ceph-mon[77626]: osdmap e56: 6 total, 6 up, 6 in 2024-09-16T00:31:16.528 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:16 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:31:17.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:17 smithi112 ceph-mon[97750]: pgmap v72: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 325 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 3 op/s; 57/372 objects degraded (15.323%) 2024-09-16T00:31:17.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:17 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:31:17.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:17 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:31:17.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:17 smithi112 ceph-mon[97750]: osdmap e57: 6 total, 6 up, 6 in 2024-09-16T00:31:18.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:17 smithi169 ceph-mon[77626]: pgmap v72: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 325 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 3 op/s; 57/372 objects degraded (15.323%) 2024-09-16T00:31:18.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:17 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:31:18.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:17 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:31:18.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:17 smithi169 ceph-mon[77626]: osdmap e57: 6 total, 6 up, 6 in 2024-09-16T00:31:19.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:18 smithi169 ceph-mon[77626]: Health check update: Degraded data redundancy: 24/372 objects degraded (6.452%), 7 pgs degraded (PG_DEGRADED) 2024-09-16T00:31:19.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:18 smithi112 ceph-mon[97750]: Health check update: Degraded data redundancy: 24/372 objects degraded (6.452%), 7 pgs degraded (PG_DEGRADED) 2024-09-16T00:31:19.927 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:19 smithi169 ceph-mon[77626]: pgmap v74: 65 pgs: 7 active+undersized, 7 active+undersized+degraded, 51 active+clean; 325 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 3.0 KiB/s rd, 4 op/s; 24/372 objects degraded (6.452%) 2024-09-16T00:31:20.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:19 smithi112 ceph-mon[97750]: pgmap v74: 65 pgs: 7 active+undersized, 7 active+undersized+degraded, 51 active+clean; 325 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 3.0 KiB/s rd, 4 op/s; 24/372 objects degraded (6.452%) 2024-09-16T00:31:21.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:20 smithi169 ceph-mon[77626]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 24/372 objects degraded (6.452%), 7 pgs degraded) 2024-09-16T00:31:21.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:20 smithi112 ceph-mon[97750]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 24/372 objects degraded (6.452%), 7 pgs degraded) 2024-09-16T00:31:22.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:21 smithi169 ceph-mon[77626]: pgmap v75: 65 pgs: 65 active+clean; 325 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-16T00:31:22.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:21 smithi112 ceph-mon[97750]: pgmap v75: 65 pgs: 65 active+clean; 325 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-16T00:31:23.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:23 smithi112 ceph-mon[97750]: pgmap v76: 65 pgs: 65 active+clean; 325 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1023 B/s rd, 0 op/s 2024-09-16T00:31:24.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:23 smithi169 ceph-mon[77626]: pgmap v76: 65 pgs: 65 active+clean; 325 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1023 B/s rd, 0 op/s 2024-09-16T00:31:26.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:25 smithi169 ceph-mon[77626]: pgmap v77: 65 pgs: 65 active+clean; 325 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.1 KiB/s rd, 2 op/s 2024-09-16T00:31:26.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:25 smithi112 ceph-mon[97750]: pgmap v77: 65 pgs: 65 active+clean; 325 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.1 KiB/s rd, 2 op/s 2024-09-16T00:31:27.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:27 smithi112 ceph-mon[97750]: pgmap v78: 65 pgs: 65 active+clean; 325 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.8 KiB/s rd, 2 op/s 2024-09-16T00:31:27.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:27 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:31:28.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:27 smithi169 ceph-mon[77626]: pgmap v78: 65 pgs: 65 active+clean; 325 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.8 KiB/s rd, 2 op/s 2024-09-16T00:31:28.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:27 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:31:29.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:29 smithi112 ceph-mon[97750]: pgmap v79: 65 pgs: 65 active+clean; 325 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.4 KiB/s rd, 2 op/s 2024-09-16T00:31:30.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:29 smithi169 ceph-mon[77626]: pgmap v79: 65 pgs: 65 active+clean; 325 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.4 KiB/s rd, 2 op/s 2024-09-16T00:31:31.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:31 smithi112 ceph-mon[97750]: pgmap v80: 65 pgs: 65 active+clean; 325 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 2 op/s 2024-09-16T00:31:31.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:31 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-09-16T00:31:31.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:31 smithi112 ceph-mon[97750]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-09-16T00:31:31.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:31 smithi112 ceph-mon[97750]: Upgrade: osd.2 is safe to restart 2024-09-16T00:31:31.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:31 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:31:31.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:31 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:31:32.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:31 smithi169 ceph-mon[77626]: pgmap v80: 65 pgs: 65 active+clean; 325 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 2 op/s 2024-09-16T00:31:32.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:31 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-09-16T00:31:32.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:31 smithi169 ceph-mon[77626]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-09-16T00:31:32.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:31 smithi169 ceph-mon[77626]: Upgrade: osd.2 is safe to restart 2024-09-16T00:31:32.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:31 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:31:32.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:31 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:31:33.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:33 smithi112 ceph-mon[97750]: Upgrade: Updating osd.2 2024-09-16T00:31:33.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:33 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:31:33.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:33 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-16T00:31:33.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:33 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:31:33.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:33 smithi112 ceph-mon[97750]: Deploying daemon osd.2 on smithi112 2024-09-16T00:31:33.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:33 smithi112 ceph-mon[97750]: pgmap v81: 65 pgs: 65 active+clean; 325 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:31:33.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:33 smithi169 ceph-mon[77626]: Upgrade: Updating osd.2 2024-09-16T00:31:33.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:33 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:31:33.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:33 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-16T00:31:33.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:33 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:31:33.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:33 smithi169 ceph-mon[77626]: Deploying daemon osd.2 on smithi112 2024-09-16T00:31:33.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:33 smithi169 ceph-mon[77626]: pgmap v81: 65 pgs: 65 active+clean; 325 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:31:35.579 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:31:35 smithi112 systemd[1]: Stopping Ceph osd.2 for d346dd56-73c0-11ef-bceb-c7b262605968... 2024-09-16T00:31:35.579 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:31:35 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-2[60918]: 2024-09-16T00:31:35.399+0000 7fd4ff88c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:31:35.579 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:31:35 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-2[60918]: 2024-09-16T00:31:35.399+0000 7fd4ff88c640 -1 osd.2 57 *** Got signal Terminated *** 2024-09-16T00:31:35.579 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:31:35 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-2[60918]: 2024-09-16T00:31:35.399+0000 7fd4ff88c640 -1 osd.2 57 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:31:35.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:35 smithi112 ceph-mon[97750]: pgmap v82: 65 pgs: 65 active+clean; 325 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:31:35.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:35 smithi112 ceph-mon[97750]: osd.2 marked itself down and dead 2024-09-16T00:31:36.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:35 smithi169 ceph-mon[77626]: pgmap v82: 65 pgs: 65 active+clean; 325 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:31:36.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:35 smithi169 ceph-mon[77626]: osd.2 marked itself down and dead 2024-09-16T00:31:36.235 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:31:35 smithi112 podman[119568]: 2024-09-16 00:31:35.922249565 +0000 UTC m=+0.621774758 container died 100ec4e4852a46cc3a4d1966c941858088a144bcb94cb5900c5485890af0deef (image=quay.ceph.io/ceph-ci/ceph@sha256:83596473fc70117d507bb3d925c5bd11b70f46bbd4dffcc8452f145d93eaedc4, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=quincy-6d025bc, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True) 2024-09-16T00:31:36.567 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:31:36 smithi112 podman[119568]: 2024-09-16 00:31:36.341738332 +0000 UTC m=+1.041263513 container cleanup 100ec4e4852a46cc3a4d1966c941858088a144bcb94cb5900c5485890af0deef (image=quay.ceph.io/ceph-ci/ceph@sha256:83596473fc70117d507bb3d925c5bd11b70f46bbd4dffcc8452f145d93eaedc4, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=quincy-6d025bc, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-16T00:31:36.568 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:31:36 smithi112 bash[119568]: ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-2 2024-09-16T00:31:36.839 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:31:36 smithi112 podman[119586]: 2024-09-16 00:31:36.568003402 +0000 UTC m=+0.644906792 container remove 100ec4e4852a46cc3a4d1966c941858088a144bcb94cb5900c5485890af0deef (image=quay.ceph.io/ceph-ci/ceph@sha256:83596473fc70117d507bb3d925c5bd11b70f46bbd4dffcc8452f145d93eaedc4, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=quincy-6d025bc, ceph=True, org.label-schema.schema-version=1.0) 2024-09-16T00:31:36.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:36 smithi112 ceph-mon[97750]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-16T00:31:36.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:36 smithi112 ceph-mon[97750]: osdmap e58: 6 total, 5 up, 6 in 2024-09-16T00:31:36.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:36 smithi112 ceph-mon[97750]: osdmap e59: 6 total, 5 up, 6 in 2024-09-16T00:31:37.028 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:36 smithi169 ceph-mon[77626]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-16T00:31:37.028 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:36 smithi169 ceph-mon[77626]: osdmap e58: 6 total, 5 up, 6 in 2024-09-16T00:31:37.028 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:36 smithi169 ceph-mon[77626]: osdmap e59: 6 total, 5 up, 6 in 2024-09-16T00:31:37.288 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:31:37 smithi112 podman[119677]: 2024-09-16 00:31:36.908573029 +0000 UTC m=+0.016029652 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:31:37.288 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:31:37 smithi112 podman[119677]: 2024-09-16 00:31:37.044011041 +0000 UTC m=+0.151467702 container create 25f69335fb7111dedf737f758e346481fb6b66a06de09002044999e3d33add7d (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-2-deactivate, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:31:37.586 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:31:37 smithi112 podman[119677]: 2024-09-16 00:31:37.288576127 +0000 UTC m=+0.396032751 container init 25f69335fb7111dedf737f758e346481fb6b66a06de09002044999e3d33add7d (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2) 2024-09-16T00:31:37.586 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:31:37 smithi112 podman[119677]: 2024-09-16 00:31:37.292503704 +0000 UTC m=+0.399960329 container start 25f69335fb7111dedf737f758e346481fb6b66a06de09002044999e3d33add7d (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-2-deactivate, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, RELEASE=squid-8921c8e) 2024-09-16T00:31:37.586 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:31:37 smithi112 podman[119677]: 2024-09-16 00:31:37.363326045 +0000 UTC m=+0.470782674 container attach 25f69335fb7111dedf737f758e346481fb6b66a06de09002044999e3d33add7d (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:31:37.586 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:31:37 smithi112 podman[119677]: 2024-09-16 00:31:37.477066476 +0000 UTC m=+0.584523093 container died 25f69335fb7111dedf737f758e346481fb6b66a06de09002044999e3d33add7d (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-16T00:31:37.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:37 smithi112 ceph-mon[97750]: pgmap v84: 65 pgs: 9 stale+active+clean, 56 active+clean; 325 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-09-16T00:31:38.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:37 smithi169 ceph-mon[77626]: pgmap v84: 65 pgs: 9 stale+active+clean, 56 active+clean; 325 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-09-16T00:31:38.555 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:31:38 smithi112 podman[119700]: 2024-09-16 00:31:38.125740975 +0000 UTC m=+0.642590890 container remove 25f69335fb7111dedf737f758e346481fb6b66a06de09002044999e3d33add7d (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:31:38.555 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:31:38 smithi112 systemd[1]: ceph-d346dd56-73c0-11ef-bceb-c7b262605968@osd.2.service: Deactivated successfully. 2024-09-16T00:31:38.555 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:31:38 smithi112 systemd[1]: Stopped Ceph osd.2 for d346dd56-73c0-11ef-bceb-c7b262605968. 2024-09-16T00:31:38.555 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:31:38 smithi112 systemd[1]: ceph-d346dd56-73c0-11ef-bceb-c7b262605968@osd.2.service: Consumed 40.578s CPU time. 2024-09-16T00:31:38.555 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:31:38 smithi112 systemd[1]: Starting Ceph osd.2 for d346dd56-73c0-11ef-bceb-c7b262605968... 2024-09-16T00:31:38.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:38 smithi112 ceph-mon[97750]: Health check failed: Degraded data redundancy: 18/372 objects degraded (4.839%), 6 pgs degraded (PG_DEGRADED) 2024-09-16T00:31:38.839 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:31:38 smithi112 podman[119798]: 2024-09-16 00:31:38.457532122 +0000 UTC m=+0.015909253 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:31:38.839 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:31:38 smithi112 podman[119798]: 2024-09-16 00:31:38.5933535 +0000 UTC m=+0.151730622 container create 1df81687542f5821de9ab6d91fff991f532d4ec9a459005340c6032b976ebe1d (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-2-activate, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-16T00:31:39.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:38 smithi169 ceph-mon[77626]: Health check failed: Degraded data redundancy: 18/372 objects degraded (4.839%), 6 pgs degraded (PG_DEGRADED) 2024-09-16T00:31:39.339 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:31:38 smithi112 podman[119798]: 2024-09-16 00:31:38.863255798 +0000 UTC m=+0.421632921 container init 1df81687542f5821de9ab6d91fff991f532d4ec9a459005340c6032b976ebe1d (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-2-activate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True) 2024-09-16T00:31:39.339 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:31:38 smithi112 podman[119798]: 2024-09-16 00:31:38.866592976 +0000 UTC m=+0.424970109 container start 1df81687542f5821de9ab6d91fff991f532d4ec9a459005340c6032b976ebe1d (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-2-activate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:31:39.339 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:31:38 smithi112 podman[119798]: 2024-09-16 00:31:38.954693935 +0000 UTC m=+0.513071062 container attach 1df81687542f5821de9ab6d91fff991f532d4ec9a459005340c6032b976ebe1d (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-2-activate, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:31:39.339 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:31:38 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-2-activate[119813]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T00:31:39.340 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:31:38 smithi112 bash[119798]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T00:31:39.340 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:31:39 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-2-activate[119813]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T00:31:39.340 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:31:39 smithi112 bash[119798]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T00:31:40.016 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:39 smithi112 ceph-mon[97750]: pgmap v86: 65 pgs: 6 active+undersized, 7 stale+active+clean, 6 active+undersized+degraded, 46 active+clean; 325 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 2 op/s; 18/372 objects degraded (4.839%) 2024-09-16T00:31:40.016 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:31:39 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-2-activate[119813]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-16T00:31:40.016 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:31:39 smithi112 bash[119798]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-16T00:31:40.016 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:31:39 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-2-activate[119813]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-09-16T00:31:40.016 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:31:39 smithi112 bash[119798]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-09-16T00:31:40.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:39 smithi169 ceph-mon[77626]: pgmap v86: 65 pgs: 6 active+undersized, 7 stale+active+clean, 6 active+undersized+degraded, 46 active+clean; 325 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 2 op/s; 18/372 objects degraded (4.839%) 2024-09-16T00:31:40.339 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:31:40 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-2-activate[119813]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-09-16T00:31:40.339 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:31:40 smithi112 bash[119798]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-09-16T00:31:40.340 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:31:40 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-2-activate[119813]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-09-16T00:31:40.340 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:31:40 smithi112 bash[119798]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-09-16T00:31:40.340 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:31:40 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-2-activate[119813]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-09-16T00:31:40.340 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:31:40 smithi112 bash[119798]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-09-16T00:31:40.340 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:31:40 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-2-activate[119813]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-16T00:31:40.340 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:31:40 smithi112 bash[119798]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-16T00:31:40.340 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:31:40 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-2-activate[119813]: --> ceph-volume raw activate successful for osd ID: 2 2024-09-16T00:31:40.340 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:31:40 smithi112 bash[119798]: --> ceph-volume raw activate successful for osd ID: 2 2024-09-16T00:31:40.340 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:31:40 smithi112 podman[119798]: 2024-09-16 00:31:40.044397237 +0000 UTC m=+1.602774376 container died 1df81687542f5821de9ab6d91fff991f532d4ec9a459005340c6032b976ebe1d (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-2-activate, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731) 2024-09-16T00:31:41.059 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:31:40 smithi112 podman[119948]: 2024-09-16 00:31:40.733897706 +0000 UTC m=+0.679009702 container remove 1df81687542f5821de9ab6d91fff991f532d4ec9a459005340c6032b976ebe1d (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-2-activate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True) 2024-09-16T00:31:41.059 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:31:40 smithi112 podman[120010]: 2024-09-16 00:31:40.841836293 +0000 UTC m=+0.015500755 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:31:41.059 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:31:40 smithi112 podman[120010]: 2024-09-16 00:31:40.943180968 +0000 UTC m=+0.116845427 container create 5f7863712057784c00db544571a201c6629a07fdd60e2133f923e0af6086cd4c (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-16T00:31:41.589 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:31:41 smithi112 podman[120010]: 2024-09-16 00:31:41.229366505 +0000 UTC m=+0.403030961 container init 5f7863712057784c00db544571a201c6629a07fdd60e2133f923e0af6086cd4c (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-16T00:31:41.589 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:31:41 smithi112 podman[120010]: 2024-09-16 00:31:41.232520146 +0000 UTC m=+0.406184607 container start 5f7863712057784c00db544571a201c6629a07fdd60e2133f923e0af6086cd4c (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-2, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS) 2024-09-16T00:31:41.589 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:31:41 smithi112 ceph-osd[120029]: -- 172.21.15.112:0/619436800 <== mon.0 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5654e57f41e0 con 0x5654e578a000 2024-09-16T00:31:41.589 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:31:41 smithi112 bash[120010]: 5f7863712057784c00db544571a201c6629a07fdd60e2133f923e0af6086cd4c 2024-09-16T00:31:41.589 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:31:41 smithi112 systemd[1]: Started Ceph osd.2 for d346dd56-73c0-11ef-bceb-c7b262605968. 2024-09-16T00:31:41.975 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:41 smithi112 ceph-mon[97750]: pgmap v87: 65 pgs: 14 active+undersized, 13 active+undersized+degraded, 38 active+clean; 325 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 45/372 objects degraded (12.097%) 2024-09-16T00:31:41.975 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:41 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:31:41.976 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:41 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:31:41.976 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:41 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:31:42.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:41 smithi169 ceph-mon[77626]: pgmap v87: 65 pgs: 14 active+undersized, 13 active+undersized+degraded, 38 active+clean; 325 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 45/372 objects degraded (12.097%) 2024-09-16T00:31:42.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:41 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:31:42.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:41 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:31:42.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:41 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:31:43.089 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:31:42 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-2[120025]: 2024-09-16T00:31:42.873+0000 7f749b0d3640 -1 Falling back to public interface 2024-09-16T00:31:43.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:43 smithi112 ceph-mon[97750]: pgmap v88: 65 pgs: 14 active+undersized, 13 active+undersized+degraded, 38 active+clean; 325 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 45/372 objects degraded (12.097%) 2024-09-16T00:31:44.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:43 smithi169 ceph-mon[77626]: pgmap v88: 65 pgs: 14 active+undersized, 13 active+undersized+degraded, 38 active+clean; 325 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 45/372 objects degraded (12.097%) 2024-09-16T00:31:45.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:45.841+0000 7fcc3ac7b640 1 -- 172.21.15.112:0/1921502846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc34102500 msgr2=0x7fcc341028d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:45.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:45.841+0000 7fcc3ac7b640 1 --2- 172.21.15.112:0/1921502846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc34102500 0x7fcc341028d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fcc280048d0 tx=0x7fcc28009da0 comp rx=0 tx=0).stop 2024-09-16T00:31:45.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:45.841+0000 7fcc3ac7b640 1 -- 172.21.15.112:0/1921502846 shutdown_connections 2024-09-16T00:31:45.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:45.841+0000 7fcc3ac7b640 1 --2- 172.21.15.112:0/1921502846 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc34102ea0 0x7fcc34064f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:45.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:45.841+0000 7fcc3ac7b640 1 --2- 172.21.15.112:0/1921502846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc34102500 0x7fcc341028d0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:45.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:45.841+0000 7fcc3ac7b640 1 -- 172.21.15.112:0/1921502846 >> 172.21.15.112:0/1921502846 conn(0x7fcc34071450 msgr2=0x7fcc34071850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:45.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:45.841+0000 7fcc3ac7b640 1 -- 172.21.15.112:0/1921502846 shutdown_connections 2024-09-16T00:31:45.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:45.841+0000 7fcc3ac7b640 1 -- 172.21.15.112:0/1921502846 wait complete. 2024-09-16T00:31:45.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:45.841+0000 7fcc3ac7b640 1 Processor -- start 2024-09-16T00:31:45.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:45.841+0000 7fcc3ac7b640 1 -- start start 2024-09-16T00:31:45.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:45.842+0000 7fcc3ac7b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc34102500 0x7fcc341131a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:45.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:45.842+0000 7fcc3ac7b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc34102ea0 0x7fcc341136e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:45.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:45.842+0000 7fcc3ac7b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc34113d70 con 0x7fcc34102ea0 2024-09-16T00:31:45.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:45.842+0000 7fcc3ac7b640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc3410d120 con 0x7fcc34102500 2024-09-16T00:31:45.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:45.842+0000 7fcc33fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc34102ea0 0x7fcc341136e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:45.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:45.842+0000 7fcc33fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc34102ea0 0x7fcc341136e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41732/0 (socket says 172.21.15.112:41732) 2024-09-16T00:31:45.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:45.842+0000 7fcc33fff640 1 -- 172.21.15.112:0/3685991361 learned_addr learned my addr 172.21.15.112:0/3685991361 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:31:45.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:45.842+0000 7fcc38a17640 1 --2- 172.21.15.112:0/3685991361 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc34102500 0x7fcc341131a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:45.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:45.842+0000 7fcc33fff640 1 -- 172.21.15.112:0/3685991361 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc34102500 msgr2=0x7fcc341131a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:45.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:45.842+0000 7fcc33fff640 1 --2- 172.21.15.112:0/3685991361 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc34102500 0x7fcc341131a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:45.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:45.842+0000 7fcc33fff640 1 -- 172.21.15.112:0/3685991361 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc28005040 con 0x7fcc34102ea0 2024-09-16T00:31:45.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:45.842+0000 7fcc33fff640 1 --2- 172.21.15.112:0/3685991361 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc34102ea0 0x7fcc341136e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcc2000b580 tx=0x7fcc200091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:45.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:45.842+0000 7fcc317fa640 1 -- 172.21.15.112:0/3685991361 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc2000f6f0 con 0x7fcc34102ea0 2024-09-16T00:31:45.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:45.842+0000 7fcc317fa640 1 -- 172.21.15.112:0/3685991361 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fcc2000fcf0 con 0x7fcc34102ea0 2024-09-16T00:31:45.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:45.842+0000 7fcc317fa640 1 -- 172.21.15.112:0/3685991361 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc200103a0 con 0x7fcc34102ea0 2024-09-16T00:31:45.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:45.842+0000 7fcc3ac7b640 1 -- 172.21.15.112:0/3685991361 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc3410d400 con 0x7fcc34102ea0 2024-09-16T00:31:45.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:45.842+0000 7fcc3ac7b640 1 -- 172.21.15.112:0/3685991361 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc3410d920 con 0x7fcc34102ea0 2024-09-16T00:31:45.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:45.844+0000 7fcc3ac7b640 1 -- 172.21.15.112:0/3685991361 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc340fdc80 con 0x7fcc34102ea0 2024-09-16T00:31:45.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:45.844+0000 7fcc317fa640 1 -- 172.21.15.112:0/3685991361 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fcc200116b0 con 0x7fcc34102ea0 2024-09-16T00:31:45.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:45.844+0000 7fcc317fa640 1 --2- 172.21.15.112:0/3685991361 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fcc2406e530 0x7fcc240709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:45.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:45.844+0000 7fcc317fa640 1 -- 172.21.15.112:0/3685991361 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fcc2008df80 con 0x7fcc34102ea0 2024-09-16T00:31:45.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:45.844+0000 7fcc38a17640 1 --2- 172.21.15.112:0/3685991361 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fcc2406e530 0x7fcc240709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:45.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:45.845+0000 7fcc38a17640 1 --2- 172.21.15.112:0/3685991361 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fcc2406e530 0x7fcc240709e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcc28009ac0 tx=0x7fcc28035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:45.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:45.849+0000 7fcc317fa640 1 -- 172.21.15.112:0/3685991361 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fcc2005ac70 con 0x7fcc34102ea0 2024-09-16T00:31:46.015 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:45 smithi112 ceph-mon[97750]: pgmap v89: 65 pgs: 14 active+undersized, 13 active+undersized+degraded, 38 active+clean; 325 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 45/372 objects degraded (12.097%) 2024-09-16T00:31:46.016 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:45 smithi112 ceph-mon[97750]: Health check update: Degraded data redundancy: 45/372 objects degraded (12.097%), 13 pgs degraded (PG_DEGRADED) 2024-09-16T00:31:46.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:45 smithi169 ceph-mon[77626]: pgmap v89: 65 pgs: 14 active+undersized, 13 active+undersized+degraded, 38 active+clean; 325 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 45/372 objects degraded (12.097%) 2024-09-16T00:31:46.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:45 smithi169 ceph-mon[77626]: Health check update: Degraded data redundancy: 45/372 objects degraded (12.097%), 13 pgs degraded (PG_DEGRADED) 2024-09-16T00:31:46.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.034+0000 7fcc3ac7b640 1 -- 172.21.15.112:0/3685991361 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcc341095f0 con 0x7fcc2406e530 2024-09-16T00:31:46.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.035+0000 7fcc317fa640 1 -- 172.21.15.112:0/3685991361 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fcc341095f0 con 0x7fcc2406e530 2024-09-16T00:31:46.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.037+0000 7fcc3ac7b640 1 -- 172.21.15.112:0/3685991361 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fcc2406e530 msgr2=0x7fcc240709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:46.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.037+0000 7fcc3ac7b640 1 --2- 172.21.15.112:0/3685991361 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fcc2406e530 0x7fcc240709e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcc28009ac0 tx=0x7fcc28035040 comp rx=0 tx=0).stop 2024-09-16T00:31:46.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.037+0000 7fcc3ac7b640 1 -- 172.21.15.112:0/3685991361 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc34102ea0 msgr2=0x7fcc341136e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:46.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.037+0000 7fcc3ac7b640 1 --2- 172.21.15.112:0/3685991361 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc34102ea0 0x7fcc341136e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcc2000b580 tx=0x7fcc200091a0 comp rx=0 tx=0).stop 2024-09-16T00:31:46.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.037+0000 7fcc3ac7b640 1 -- 172.21.15.112:0/3685991361 shutdown_connections 2024-09-16T00:31:46.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.037+0000 7fcc3ac7b640 1 --2- 172.21.15.112:0/3685991361 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fcc2406e530 0x7fcc240709e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:46.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.037+0000 7fcc3ac7b640 1 --2- 172.21.15.112:0/3685991361 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc34102ea0 0x7fcc341136e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:46.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.037+0000 7fcc3ac7b640 1 --2- 172.21.15.112:0/3685991361 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc34102500 0x7fcc341131a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:46.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.037+0000 7fcc3ac7b640 1 -- 172.21.15.112:0/3685991361 >> 172.21.15.112:0/3685991361 conn(0x7fcc34071450 msgr2=0x7fcc341052c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:46.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.037+0000 7fcc3ac7b640 1 -- 172.21.15.112:0/3685991361 shutdown_connections 2024-09-16T00:31:46.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.037+0000 7fcc3ac7b640 1 -- 172.21.15.112:0/3685991361 wait complete. 2024-09-16T00:31:46.046 INFO:teuthology.orchestra.run.smithi112.stdout:true 2024-09-16T00:31:46.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.131+0000 7fdd72583640 1 -- 172.21.15.112:0/2147325740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd6c0fe460 msgr2=0x7fdd6c0fe830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:46.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.131+0000 7fdd72583640 1 --2- 172.21.15.112:0/2147325740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd6c0fe460 0x7fdd6c0fe830 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c0048d0 tx=0x7fdd5c009da0 comp rx=0 tx=0).stop 2024-09-16T00:31:46.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.131+0000 7fdd72583640 1 -- 172.21.15.112:0/2147325740 shutdown_connections 2024-09-16T00:31:46.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.131+0000 7fdd72583640 1 --2- 172.21.15.112:0/2147325740 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd6c064fc0 0x7fdd6c065430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:46.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.131+0000 7fdd72583640 1 --2- 172.21.15.112:0/2147325740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd6c0fe460 0x7fdd6c0fe830 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:46.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.131+0000 7fdd72583640 1 -- 172.21.15.112:0/2147325740 >> 172.21.15.112:0/2147325740 conn(0x7fdd6c0723b0 msgr2=0x7fdd6c0727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:46.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.131+0000 7fdd72583640 1 -- 172.21.15.112:0/2147325740 shutdown_connections 2024-09-16T00:31:46.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.131+0000 7fdd72583640 1 -- 172.21.15.112:0/2147325740 wait complete. 2024-09-16T00:31:46.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.131+0000 7fdd72583640 1 Processor -- start 2024-09-16T00:31:46.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.132+0000 7fdd72583640 1 -- start start 2024-09-16T00:31:46.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.132+0000 7fdd72583640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd6c064fc0 0x7fdd6c0fb560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:46.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.132+0000 7fdd72583640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd6c0fe460 0x7fdd6c0fbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:46.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.132+0000 7fdd72583640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6c0fd310 con 0x7fdd6c0fe460 2024-09-16T00:31:46.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.132+0000 7fdd72583640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6c0fd480 con 0x7fdd6c064fc0 2024-09-16T00:31:46.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.132+0000 7fdd6b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd6c0fe460 0x7fdd6c0fbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:46.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.132+0000 7fdd6b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd6c0fe460 0x7fdd6c0fbaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41746/0 (socket says 172.21.15.112:41746) 2024-09-16T00:31:46.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.132+0000 7fdd6b7fe640 1 -- 172.21.15.112:0/2307116701 learned_addr learned my addr 172.21.15.112:0/2307116701 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:31:46.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.132+0000 7fdd6bfff640 1 --2- 172.21.15.112:0/2307116701 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd6c064fc0 0x7fdd6c0fb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:46.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.132+0000 7fdd6b7fe640 1 -- 172.21.15.112:0/2307116701 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd6c064fc0 msgr2=0x7fdd6c0fb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:46.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.133+0000 7fdd6b7fe640 1 --2- 172.21.15.112:0/2307116701 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd6c064fc0 0x7fdd6c0fb560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:46.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.133+0000 7fdd6b7fe640 1 -- 172.21.15.112:0/2307116701 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd5c005040 con 0x7fdd6c0fe460 2024-09-16T00:31:46.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.133+0000 7fdd6bfff640 1 --2- 172.21.15.112:0/2307116701 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd6c064fc0 0x7fdd6c0fb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:31:46.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.133+0000 7fdd6b7fe640 1 --2- 172.21.15.112:0/2307116701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd6c0fe460 0x7fdd6c0fbaa0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fdd60002a50 tx=0x7fdd60004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:46.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.133+0000 7fdd68ff9640 1 -- 172.21.15.112:0/2307116701 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd60007c30 con 0x7fdd6c0fe460 2024-09-16T00:31:46.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.133+0000 7fdd72583640 1 -- 172.21.15.112:0/2307116701 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd6c0fc0a0 con 0x7fdd6c0fe460 2024-09-16T00:31:46.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.133+0000 7fdd68ff9640 1 -- 172.21.15.112:0/2307116701 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fdd6000f040 con 0x7fdd6c0fe460 2024-09-16T00:31:46.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.133+0000 7fdd68ff9640 1 -- 172.21.15.112:0/2307116701 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd6000dad0 con 0x7fdd6c0fe460 2024-09-16T00:31:46.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.133+0000 7fdd72583640 1 -- 172.21.15.112:0/2307116701 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd6c11de40 con 0x7fdd6c0fe460 2024-09-16T00:31:46.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.134+0000 7fdd72583640 1 -- 172.21.15.112:0/2307116701 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd6c0fdcf0 con 0x7fdd6c0fe460 2024-09-16T00:31:46.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.134+0000 7fdd68ff9640 1 -- 172.21.15.112:0/2307116701 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fdd6000dc70 con 0x7fdd6c0fe460 2024-09-16T00:31:46.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.135+0000 7fdd68ff9640 1 --2- 172.21.15.112:0/2307116701 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fdd5406e530 0x7fdd540709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:46.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.135+0000 7fdd68ff9640 1 -- 172.21.15.112:0/2307116701 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fdd6008da20 con 0x7fdd6c0fe460 2024-09-16T00:31:46.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.135+0000 7fdd6bfff640 1 --2- 172.21.15.112:0/2307116701 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fdd5406e530 0x7fdd540709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:46.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.136+0000 7fdd6bfff640 1 --2- 172.21.15.112:0/2307116701 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fdd5406e530 0x7fdd540709e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c007ac0 tx=0x7fdd5c035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:46.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.139+0000 7fdd68ff9640 1 -- 172.21.15.112:0/2307116701 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fdd60056b60 con 0x7fdd6c0fe460 2024-09-16T00:31:46.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.331+0000 7fdd72583640 1 -- 172.21.15.112:0/2307116701 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdd6c0fccc0 con 0x7fdd5406e530 2024-09-16T00:31:46.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.332+0000 7fdd68ff9640 1 -- 172.21.15.112:0/2307116701 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fdd6c0fccc0 con 0x7fdd5406e530 2024-09-16T00:31:46.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.334+0000 7fdd72583640 1 -- 172.21.15.112:0/2307116701 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fdd5406e530 msgr2=0x7fdd540709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:46.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.334+0000 7fdd72583640 1 --2- 172.21.15.112:0/2307116701 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fdd5406e530 0x7fdd540709e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c007ac0 tx=0x7fdd5c035040 comp rx=0 tx=0).stop 2024-09-16T00:31:46.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.334+0000 7fdd72583640 1 -- 172.21.15.112:0/2307116701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd6c0fe460 msgr2=0x7fdd6c0fbaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:46.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.334+0000 7fdd72583640 1 --2- 172.21.15.112:0/2307116701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd6c0fe460 0x7fdd6c0fbaa0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fdd60002a50 tx=0x7fdd60004600 comp rx=0 tx=0).stop 2024-09-16T00:31:46.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.334+0000 7fdd72583640 1 -- 172.21.15.112:0/2307116701 shutdown_connections 2024-09-16T00:31:46.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.334+0000 7fdd72583640 1 --2- 172.21.15.112:0/2307116701 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fdd5406e530 0x7fdd540709e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:46.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.334+0000 7fdd72583640 1 --2- 172.21.15.112:0/2307116701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd6c0fe460 0x7fdd6c0fbaa0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:46.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.334+0000 7fdd72583640 1 --2- 172.21.15.112:0/2307116701 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd6c064fc0 0x7fdd6c0fb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:46.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.334+0000 7fdd72583640 1 -- 172.21.15.112:0/2307116701 >> 172.21.15.112:0/2307116701 conn(0x7fdd6c0723b0 msgr2=0x7fdd6c105410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:46.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.334+0000 7fdd72583640 1 -- 172.21.15.112:0/2307116701 shutdown_connections 2024-09-16T00:31:46.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.334+0000 7fdd72583640 1 -- 172.21.15.112:0/2307116701 wait complete. 2024-09-16T00:31:46.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.416+0000 7f1500a12640 1 -- 172.21.15.112:0/911625764 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14fc0fe460 msgr2=0x7f14fc0fe830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:46.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.416+0000 7f1500a12640 1 --2- 172.21.15.112:0/911625764 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14fc0fe460 0x7f14fc0fe830 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f14ec0048d0 tx=0x7f14ec009da0 comp rx=0 tx=0).stop 2024-09-16T00:31:46.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.416+0000 7f1500a12640 1 -- 172.21.15.112:0/911625764 shutdown_connections 2024-09-16T00:31:46.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.416+0000 7f1500a12640 1 --2- 172.21.15.112:0/911625764 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14fc0fabe0 0x7f14fc0fb030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:46.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.416+0000 7f1500a12640 1 --2- 172.21.15.112:0/911625764 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14fc0fe460 0x7f14fc0fe830 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:46.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.416+0000 7f1500a12640 1 -- 172.21.15.112:0/911625764 >> 172.21.15.112:0/911625764 conn(0x7f14fc071b00 msgr2=0x7f14fc071f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:46.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.416+0000 7f1500a12640 1 -- 172.21.15.112:0/911625764 shutdown_connections 2024-09-16T00:31:46.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.416+0000 7f1500a12640 1 -- 172.21.15.112:0/911625764 wait complete. 2024-09-16T00:31:46.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.417+0000 7f1500a12640 1 Processor -- start 2024-09-16T00:31:46.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.417+0000 7f1500a12640 1 -- start start 2024-09-16T00:31:46.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.417+0000 7f1500a12640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14fc0fabe0 0x7f14fc113190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:46.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.417+0000 7f1500a12640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14fc0fe460 0x7f14fc1136d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:46.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.417+0000 7f1500a12640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14fc113cd0 con 0x7f14fc0fabe0 2024-09-16T00:31:46.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.417+0000 7f1500a12640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14fc10d110 con 0x7f14fc0fe460 2024-09-16T00:31:46.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.417+0000 7f14fa59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14fc0fabe0 0x7f14fc113190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:46.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.417+0000 7f14fa59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14fc0fabe0 0x7f14fc113190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41764/0 (socket says 172.21.15.112:41764) 2024-09-16T00:31:46.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.417+0000 7f14fa59c640 1 -- 172.21.15.112:0/4251155498 learned_addr learned my addr 172.21.15.112:0/4251155498 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:31:46.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.417+0000 7f14f9d9b640 1 --2- 172.21.15.112:0/4251155498 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14fc0fe460 0x7f14fc1136d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:46.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.418+0000 7f14fa59c640 1 -- 172.21.15.112:0/4251155498 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14fc0fe460 msgr2=0x7f14fc1136d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:46.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.418+0000 7f14fa59c640 1 --2- 172.21.15.112:0/4251155498 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14fc0fe460 0x7f14fc1136d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:46.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.418+0000 7f14fa59c640 1 -- 172.21.15.112:0/4251155498 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14ec005040 con 0x7f14fc0fabe0 2024-09-16T00:31:46.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.418+0000 7f14fa59c640 1 --2- 172.21.15.112:0/4251155498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14fc0fabe0 0x7f14fc113190 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f14ec0043d0 tx=0x7f14ec009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:46.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.418+0000 7f14eaffd640 1 -- 172.21.15.112:0/4251155498 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f14ec038070 con 0x7f14fc0fabe0 2024-09-16T00:31:46.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.418+0000 7f14eaffd640 1 -- 172.21.15.112:0/4251155498 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f14ec0096e0 con 0x7f14fc0fabe0 2024-09-16T00:31:46.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.418+0000 7f1500a12640 1 -- 172.21.15.112:0/4251155498 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14fc10d390 con 0x7f14fc0fabe0 2024-09-16T00:31:46.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.418+0000 7f14eaffd640 1 -- 172.21.15.112:0/4251155498 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f14ec03d420 con 0x7f14fc0fabe0 2024-09-16T00:31:46.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.418+0000 7f1500a12640 1 -- 172.21.15.112:0/4251155498 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14fc10d930 con 0x7f14fc0fabe0 2024-09-16T00:31:46.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.419+0000 7f1500a12640 1 -- 172.21.15.112:0/4251155498 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14fc072610 con 0x7f14fc0fabe0 2024-09-16T00:31:46.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.420+0000 7f14eaffd640 1 -- 172.21.15.112:0/4251155498 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f14ec030020 con 0x7f14fc0fabe0 2024-09-16T00:31:46.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.420+0000 7f14eaffd640 1 --2- 172.21.15.112:0/4251155498 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f14e406e530 0x7f14e40709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:46.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.420+0000 7f14eaffd640 1 -- 172.21.15.112:0/4251155498 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f14ec0ae6e0 con 0x7f14fc0fabe0 2024-09-16T00:31:46.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.420+0000 7f14f9d9b640 1 --2- 172.21.15.112:0/4251155498 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f14e406e530 0x7f14e40709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:46.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.421+0000 7f14f9d9b640 1 --2- 172.21.15.112:0/4251155498 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f14e406e530 0x7f14e40709e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f14f0007fd0 tx=0x7f14f0009e90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:46.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.425+0000 7f14eaffd640 1 -- 172.21.15.112:0/4251155498 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f14ec077820 con 0x7f14fc0fabe0 2024-09-16T00:31:46.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.649+0000 7f1500a12640 1 -- 172.21.15.112:0/4251155498 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f14fc064d50 con 0x7f14e406e530 2024-09-16T00:31:46.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.658+0000 7f14eaffd640 1 -- 172.21.15.112:0/4251155498 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f14fc064d50 con 0x7f14e406e530 2024-09-16T00:31:46.659 INFO:teuthology.orchestra.run.smithi112.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T00:31:46.659 INFO:teuthology.orchestra.run.smithi112.stdout:alertmanager.smithi112 smithi112 *:9093,9094 running (4m) 0s ago 13m 25.2M - 0.25.0 c8568f914cd2 d85ea0562022 2024-09-16T00:31:46.659 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi112 smithi112 running (100s) 0s ago 13m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c e4fe58f9702a 2024-09-16T00:31:46.659 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi169 smithi169 running (96s) 92s ago 12m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c 3ba2c81d527f 2024-09-16T00:31:46.659 INFO:teuthology.orchestra.run.smithi112.stdout:grafana.smithi112 smithi112 *:3000 running (3m) 0s ago 13m 82.2M - 9.4.12 2bacad6d85d8 51f9d2013da2 2024-09-16T00:31:46.659 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.hxfwkh smithi112 running (8m) 0s ago 8m 135M - 17.2.7-1633-g6d025bc2 33ed8dba3996 cf92d9461fc6 2024-09-16T00:31:46.659 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.ithwzj smithi112 running (8m) 0s ago 8m 66.1M - 17.2.7-1633-g6d025bc2 33ed8dba3996 02786c8a83c7 2024-09-16T00:31:46.660 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.gawwgw smithi169 running (8m) 92s ago 8m 114M - 17.2.7-1633-g6d025bc2 33ed8dba3996 dcf3a4dfa879 2024-09-16T00:31:46.660 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.lgeqrj smithi169 running (8m) 92s ago 8m 60.4M - 17.2.7-1633-g6d025bc2 33ed8dba3996 445c14e55831 2024-09-16T00:31:46.660 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi112.xrkkbc smithi112 *:8443,9283,8765 running (4m) 0s ago 14m 497M - 19.1.1-326-g8921c8ef f1455bd0219c a981f7fd2810 2024-09-16T00:31:46.660 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi169.mtrobk smithi169 *:8443,9283,8765 running (3m) 92s ago 12m 439M - 19.1.1-326-g8921c8ef f1455bd0219c 4096e5369c2d 2024-09-16T00:31:46.660 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi112 smithi112 running (2m) 0s ago 15m 46.4M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 5aaec033c042 2024-09-16T00:31:46.660 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi169 smithi169 running (2m) 92s ago 12m 31.8M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 7bc609155f3f 2024-09-16T00:31:46.660 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi112 smithi112 *:9100 running (5m) 0s ago 13m 15.5M - 1.5.0 0da6a335fe13 6404d1d601e2 2024-09-16T00:31:46.660 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi169 smithi169 *:9100 running (4m) 92s ago 12m 15.2M - 1.5.0 0da6a335fe13 5505baf702a6 2024-09-16T00:31:46.660 INFO:teuthology.orchestra.run.smithi112.stdout:osd.0 smithi112 running (75s) 0s ago 11m 89.4M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 53efc9397017 2024-09-16T00:31:46.660 INFO:teuthology.orchestra.run.smithi112.stdout:osd.1 smithi112 running (40s) 0s ago 11m 89.1M 1557M 19.1.1-326-g8921c8ef f1455bd0219c fe6f1a4a2cf4 2024-09-16T00:31:46.660 INFO:teuthology.orchestra.run.smithi112.stdout:osd.2 smithi112 running (5s) 0s ago 11m 12.1M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 5f7863712057 2024-09-16T00:31:46.660 INFO:teuthology.orchestra.run.smithi112.stdout:osd.3 smithi169 running (10m) 92s ago 10m 231M 2581M 17.2.7-1633-g6d025bc2 33ed8dba3996 f88463e19f23 2024-09-16T00:31:46.661 INFO:teuthology.orchestra.run.smithi112.stdout:osd.4 smithi169 running (10m) 92s ago 10m 211M 2581M 17.2.7-1633-g6d025bc2 33ed8dba3996 d3e4cdb016b9 2024-09-16T00:31:46.661 INFO:teuthology.orchestra.run.smithi112.stdout:osd.5 smithi169 running (9m) 92s ago 9m 199M 2581M 17.2.7-1633-g6d025bc2 33ed8dba3996 8e7d708d74eb 2024-09-16T00:31:46.661 INFO:teuthology.orchestra.run.smithi112.stdout:prometheus.smithi112 smithi112 *:9095 running (3m) 0s ago 13m 63.2M - 2.43.0 a07b618ecd1d 7a1d42e60d2b 2024-09-16T00:31:46.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.662+0000 7f1500a12640 1 -- 172.21.15.112:0/4251155498 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f14e406e530 msgr2=0x7f14e40709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:46.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.662+0000 7f1500a12640 1 --2- 172.21.15.112:0/4251155498 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f14e406e530 0x7f14e40709e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f14f0007fd0 tx=0x7f14f0009e90 comp rx=0 tx=0).stop 2024-09-16T00:31:46.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.662+0000 7f1500a12640 1 -- 172.21.15.112:0/4251155498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14fc0fabe0 msgr2=0x7f14fc113190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:46.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.662+0000 7f1500a12640 1 --2- 172.21.15.112:0/4251155498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14fc0fabe0 0x7f14fc113190 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f14ec0043d0 tx=0x7f14ec009da0 comp rx=0 tx=0).stop 2024-09-16T00:31:46.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.662+0000 7f1500a12640 1 -- 172.21.15.112:0/4251155498 shutdown_connections 2024-09-16T00:31:46.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.662+0000 7f1500a12640 1 --2- 172.21.15.112:0/4251155498 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f14e406e530 0x7f14e40709e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:46.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.662+0000 7f1500a12640 1 --2- 172.21.15.112:0/4251155498 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14fc0fe460 0x7f14fc1136d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:46.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.662+0000 7f1500a12640 1 --2- 172.21.15.112:0/4251155498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14fc0fabe0 0x7f14fc113190 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:46.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.662+0000 7f1500a12640 1 -- 172.21.15.112:0/4251155498 >> 172.21.15.112:0/4251155498 conn(0x7f14fc071b00 msgr2=0x7f14fc0f99b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:46.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.662+0000 7f1500a12640 1 -- 172.21.15.112:0/4251155498 shutdown_connections 2024-09-16T00:31:46.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.662+0000 7f1500a12640 1 -- 172.21.15.112:0/4251155498 wait complete. 2024-09-16T00:31:46.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.743+0000 7ff7f0971640 1 -- 172.21.15.112:0/3750624832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7ec0fe460 msgr2=0x7ff7ec0fe830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:46.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.743+0000 7ff7f0971640 1 --2- 172.21.15.112:0/3750624832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7ec0fe460 0x7ff7ec0fe830 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc0048d0 tx=0x7ff7dc009da0 comp rx=0 tx=0).stop 2024-09-16T00:31:46.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.743+0000 7ff7f0971640 1 -- 172.21.15.112:0/3750624832 shutdown_connections 2024-09-16T00:31:46.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.743+0000 7ff7f0971640 1 --2- 172.21.15.112:0/3750624832 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7ec0fabe0 0x7ff7ec0fb030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:46.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.743+0000 7ff7f0971640 1 --2- 172.21.15.112:0/3750624832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7ec0fe460 0x7ff7ec0fe830 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:46.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.743+0000 7ff7f0971640 1 -- 172.21.15.112:0/3750624832 >> 172.21.15.112:0/3750624832 conn(0x7ff7ec071b00 msgr2=0x7ff7ec071f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:46.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.743+0000 7ff7f0971640 1 -- 172.21.15.112:0/3750624832 shutdown_connections 2024-09-16T00:31:46.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.743+0000 7ff7f0971640 1 -- 172.21.15.112:0/3750624832 wait complete. 2024-09-16T00:31:46.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.743+0000 7ff7f0971640 1 Processor -- start 2024-09-16T00:31:46.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.744+0000 7ff7f0971640 1 -- start start 2024-09-16T00:31:46.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.744+0000 7ff7f0971640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7ec0fabe0 0x7ff7ec113190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:46.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.744+0000 7ff7f0971640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7ec0fe460 0x7ff7ec1136d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:46.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.744+0000 7ff7f0971640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ec113cd0 con 0x7ff7ec0fe460 2024-09-16T00:31:46.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.744+0000 7ff7f0971640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ec10d110 con 0x7ff7ec0fabe0 2024-09-16T00:31:46.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.744+0000 7ff7e9d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7ec0fe460 0x7ff7ec1136d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:46.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.744+0000 7ff7e9d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7ec0fe460 0x7ff7ec1136d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41784/0 (socket says 172.21.15.112:41784) 2024-09-16T00:31:46.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.744+0000 7ff7e9d9b640 1 -- 172.21.15.112:0/2801164423 learned_addr learned my addr 172.21.15.112:0/2801164423 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:31:46.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.744+0000 7ff7ea59c640 1 --2- 172.21.15.112:0/2801164423 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7ec0fabe0 0x7ff7ec113190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:46.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.744+0000 7ff7e9d9b640 1 -- 172.21.15.112:0/2801164423 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7ec0fabe0 msgr2=0x7ff7ec113190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:46.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.744+0000 7ff7e9d9b640 1 --2- 172.21.15.112:0/2801164423 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7ec0fabe0 0x7ff7ec113190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:46.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.744+0000 7ff7e9d9b640 1 -- 172.21.15.112:0/2801164423 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7dc005040 con 0x7ff7ec0fe460 2024-09-16T00:31:46.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.744+0000 7ff7e9d9b640 1 --2- 172.21.15.112:0/2801164423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7ec0fe460 0x7ff7ec1136d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff7d400eb70 tx=0x7ff7d400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:46.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.745+0000 7ff7e2ffd640 1 -- 172.21.15.112:0/2801164423 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7d4007c30 con 0x7ff7ec0fe460 2024-09-16T00:31:46.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.745+0000 7ff7e2ffd640 1 -- 172.21.15.112:0/2801164423 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7ff7d400f040 con 0x7ff7ec0fe460 2024-09-16T00:31:46.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.745+0000 7ff7e2ffd640 1 -- 172.21.15.112:0/2801164423 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7d400ca60 con 0x7ff7ec0fe460 2024-09-16T00:31:46.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.745+0000 7ff7f0971640 1 -- 172.21.15.112:0/2801164423 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7ec10d390 con 0x7ff7ec0fe460 2024-09-16T00:31:46.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.745+0000 7ff7f0971640 1 -- 172.21.15.112:0/2801164423 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7ec10d930 con 0x7ff7ec0fe460 2024-09-16T00:31:46.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.746+0000 7ff7f0971640 1 -- 172.21.15.112:0/2801164423 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7ec064d50 con 0x7ff7ec0fe460 2024-09-16T00:31:46.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.747+0000 7ff7e2ffd640 1 -- 172.21.15.112:0/2801164423 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7ff7d400cbc0 con 0x7ff7ec0fe460 2024-09-16T00:31:46.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.747+0000 7ff7e2ffd640 1 --2- 172.21.15.112:0/2801164423 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff7d806e530 0x7ff7d80709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:46.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.747+0000 7ff7e2ffd640 1 -- 172.21.15.112:0/2801164423 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6410+0+0 (secure 0 0 0) 0x7ff7d408e1d0 con 0x7ff7ec0fe460 2024-09-16T00:31:46.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.747+0000 7ff7ea59c640 1 --2- 172.21.15.112:0/2801164423 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff7d806e530 0x7ff7d80709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:46.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.748+0000 7ff7ea59c640 1 --2- 172.21.15.112:0/2801164423 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff7d806e530 0x7ff7d80709e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc004540 tx=0x7ff7dc035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:46.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.751+0000 7ff7e2ffd640 1 -- 172.21.15.112:0/2801164423 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7ff7d40573a0 con 0x7ff7ec0fe460 2024-09-16T00:31:46.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.996+0000 7ff7f0971640 1 -- 172.21.15.112:0/2801164423 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff7ec10d520 con 0x7ff7ec0fe460 2024-09-16T00:31:46.997 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:31:46.998 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": { 2024-09-16T00:31:46.998 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:31:46.998 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:31:46.998 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": { 2024-09-16T00:31:46.998 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:31:46.998 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:31:46.998 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": { 2024-09-16T00:31:46.998 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 17.2.7-1633-g6d025bc2 (6d025bc26f3176c910707f8cb4c8d8e076640615) quincy (stable)": 3, 2024-09-16T00:31:46.998 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:31:46.998 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:31:46.998 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": { 2024-09-16T00:31:46.998 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 17.2.7-1633-g6d025bc2 (6d025bc26f3176c910707f8cb4c8d8e076640615) quincy (stable)": 4 2024-09-16T00:31:46.998 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:31:46.998 INFO:teuthology.orchestra.run.smithi112.stdout: "overall": { 2024-09-16T00:31:46.999 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 17.2.7-1633-g6d025bc2 (6d025bc26f3176c910707f8cb4c8d8e076640615) quincy (stable)": 7, 2024-09-16T00:31:46.999 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 6 2024-09-16T00:31:46.999 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-16T00:31:46.999 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:31:46.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:46.997+0000 7ff7e2ffd640 1 -- 172.21.15.112:0/2801164423 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+850 (secure 0 0 0) 0x7ff7ec10d520 con 0x7ff7ec0fe460 2024-09-16T00:31:47.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.000+0000 7ff7f0971640 1 -- 172.21.15.112:0/2801164423 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff7d806e530 msgr2=0x7ff7d80709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:47.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.000+0000 7ff7f0971640 1 --2- 172.21.15.112:0/2801164423 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff7d806e530 0x7ff7d80709e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc004540 tx=0x7ff7dc035040 comp rx=0 tx=0).stop 2024-09-16T00:31:47.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.000+0000 7ff7f0971640 1 -- 172.21.15.112:0/2801164423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7ec0fe460 msgr2=0x7ff7ec1136d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:47.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.000+0000 7ff7f0971640 1 --2- 172.21.15.112:0/2801164423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7ec0fe460 0x7ff7ec1136d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff7d400eb70 tx=0x7ff7d400b040 comp rx=0 tx=0).stop 2024-09-16T00:31:47.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.000+0000 7ff7f0971640 1 -- 172.21.15.112:0/2801164423 shutdown_connections 2024-09-16T00:31:47.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.000+0000 7ff7f0971640 1 --2- 172.21.15.112:0/2801164423 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff7d806e530 0x7ff7d80709e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:47.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.000+0000 7ff7f0971640 1 --2- 172.21.15.112:0/2801164423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7ec0fe460 0x7ff7ec1136d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:47.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.000+0000 7ff7f0971640 1 --2- 172.21.15.112:0/2801164423 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7ec0fabe0 0x7ff7ec113190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:47.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.000+0000 7ff7f0971640 1 -- 172.21.15.112:0/2801164423 >> 172.21.15.112:0/2801164423 conn(0x7ff7ec071b00 msgr2=0x7ff7ec0f99b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:47.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.000+0000 7ff7f0971640 1 -- 172.21.15.112:0/2801164423 shutdown_connections 2024-09-16T00:31:47.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.000+0000 7ff7f0971640 1 -- 172.21.15.112:0/2801164423 wait complete. 2024-09-16T00:31:47.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.089+0000 7f528af0b640 1 -- 172.21.15.112:0/2983451122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52840fe460 msgr2=0x7f52840fe830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:47.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.089+0000 7f528af0b640 1 --2- 172.21.15.112:0/2983451122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52840fe460 0x7f52840fe830 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f52780048d0 tx=0x7f5278009da0 comp rx=0 tx=0).stop 2024-09-16T00:31:47.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.089+0000 7f528af0b640 1 -- 172.21.15.112:0/2983451122 shutdown_connections 2024-09-16T00:31:47.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.089+0000 7f528af0b640 1 --2- 172.21.15.112:0/2983451122 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5284064fc0 0x7f5284065430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:47.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.089+0000 7f528af0b640 1 --2- 172.21.15.112:0/2983451122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52840fe460 0x7f52840fe830 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:47.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.089+0000 7f528af0b640 1 -- 172.21.15.112:0/2983451122 >> 172.21.15.112:0/2983451122 conn(0x7f52840723b0 msgr2=0x7f52840727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:47.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.089+0000 7f528af0b640 1 -- 172.21.15.112:0/2983451122 shutdown_connections 2024-09-16T00:31:47.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.089+0000 7f528af0b640 1 -- 172.21.15.112:0/2983451122 wait complete. 2024-09-16T00:31:47.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.090+0000 7f528af0b640 1 Processor -- start 2024-09-16T00:31:47.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.090+0000 7f528af0b640 1 -- start start 2024-09-16T00:31:47.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.090+0000 7f528af0b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5284064fc0 0x7f5284113200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:47.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.090+0000 7f528af0b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52840fe460 0x7f5284113740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:47.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.090+0000 7f528af0b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5284113dd0 con 0x7f52840fe460 2024-09-16T00:31:47.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.090+0000 7f528af0b640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f528410d180 con 0x7f5284064fc0 2024-09-16T00:31:47.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.090+0000 7f5283fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52840fe460 0x7f5284113740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:47.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.091+0000 7f5283fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52840fe460 0x7f5284113740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41798/0 (socket says 172.21.15.112:41798) 2024-09-16T00:31:47.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.091+0000 7f5283fff640 1 -- 172.21.15.112:0/2105145671 learned_addr learned my addr 172.21.15.112:0/2105145671 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:31:47.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.091+0000 7f5288ca7640 1 --2- 172.21.15.112:0/2105145671 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5284064fc0 0x7f5284113200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:47.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.091+0000 7f5283fff640 1 -- 172.21.15.112:0/2105145671 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5284064fc0 msgr2=0x7f5284113200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:47.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.091+0000 7f5283fff640 1 --2- 172.21.15.112:0/2105145671 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5284064fc0 0x7f5284113200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:47.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.091+0000 7f5283fff640 1 -- 172.21.15.112:0/2105145671 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5278005040 con 0x7f52840fe460 2024-09-16T00:31:47.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.091+0000 7f5288ca7640 1 --2- 172.21.15.112:0/2105145671 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5284064fc0 0x7f5284113200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:31:47.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.091+0000 7f5283fff640 1 --2- 172.21.15.112:0/2105145671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52840fe460 0x7f5284113740 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f527000b790 tx=0x7f52700091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:47.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.091+0000 7f52817fa640 1 -- 172.21.15.112:0/2105145671 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f527000c700 con 0x7f52840fe460 2024-09-16T00:31:47.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.091+0000 7f52817fa640 1 -- 172.21.15.112:0/2105145671 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f527000cd00 con 0x7f52840fe460 2024-09-16T00:31:47.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.091+0000 7f528af0b640 1 -- 172.21.15.112:0/2105145671 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f528410d460 con 0x7f52840fe460 2024-09-16T00:31:47.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.091+0000 7f52817fa640 1 -- 172.21.15.112:0/2105145671 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5270015e70 con 0x7f52840fe460 2024-09-16T00:31:47.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.091+0000 7f528af0b640 1 -- 172.21.15.112:0/2105145671 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f528410d980 con 0x7f52840fe460 2024-09-16T00:31:47.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.092+0000 7f528af0b640 1 -- 172.21.15.112:0/2105145671 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52840fdcf0 con 0x7f52840fe460 2024-09-16T00:31:47.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.093+0000 7f52817fa640 1 -- 172.21.15.112:0/2105145671 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f5270012070 con 0x7f52840fe460 2024-09-16T00:31:47.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.093+0000 7f52817fa640 1 --2- 172.21.15.112:0/2105145671 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f527406e530 0x7f52740709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:47.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.093+0000 7f52817fa640 1 -- 172.21.15.112:0/2105145671 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f527008dce0 con 0x7f52840fe460 2024-09-16T00:31:47.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.093+0000 7f5288ca7640 1 --2- 172.21.15.112:0/2105145671 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f527406e530 0x7f52740709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:47.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.094+0000 7f5288ca7640 1 --2- 172.21.15.112:0/2105145671 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f527406e530 0x7f52740709e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5278007ac0 tx=0x7f5278035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:47.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.098+0000 7f52817fa640 1 -- 172.21.15.112:0/2105145671 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f5270056e20 con 0x7f52840fe460 2024-09-16T00:31:47.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.313+0000 7f528af0b640 1 -- 172.21.15.112:0/2105145671 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f528410e870 con 0x7f52840fe460 2024-09-16T00:31:47.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.314+0000 7f52817fa640 1 -- 172.21.15.112:0/2105145671 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) v1 ==== 76+0+1954 (secure 0 0 0) 0x7f527005a4e0 con 0x7f52840fe460 2024-09-16T00:31:47.314 INFO:teuthology.orchestra.run.smithi112.stdout:e23 2024-09-16T00:31:47.314 INFO:teuthology.orchestra.run.smithi112.stdout:btime 2024-09-16T00:30:08:788506+0000 2024-09-16T00:31:47.315 INFO:teuthology.orchestra.run.smithi112.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T00:31:47.315 INFO:teuthology.orchestra.run.smithi112.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:31:47.315 INFO:teuthology.orchestra.run.smithi112.stdout:legacy client fscid: 1 2024-09-16T00:31:47.315 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:31:47.315 INFO:teuthology.orchestra.run.smithi112.stdout:Filesystem 'cephfs' (1) 2024-09-16T00:31:47.315 INFO:teuthology.orchestra.run.smithi112.stdout:fs_name cephfs 2024-09-16T00:31:47.315 INFO:teuthology.orchestra.run.smithi112.stdout:epoch 23 2024-09-16T00:31:47.315 INFO:teuthology.orchestra.run.smithi112.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-16T00:31:47.315 INFO:teuthology.orchestra.run.smithi112.stdout:created 2024-09-16T00:23:22.328199+0000 2024-09-16T00:31:47.315 INFO:teuthology.orchestra.run.smithi112.stdout:modified 2024-09-16T00:30:08.396661+0000 2024-09-16T00:31:47.315 INFO:teuthology.orchestra.run.smithi112.stdout:tableserver 0 2024-09-16T00:31:47.315 INFO:teuthology.orchestra.run.smithi112.stdout:root 0 2024-09-16T00:31:47.315 INFO:teuthology.orchestra.run.smithi112.stdout:session_timeout 60 2024-09-16T00:31:47.315 INFO:teuthology.orchestra.run.smithi112.stdout:session_autoclose 300 2024-09-16T00:31:47.316 INFO:teuthology.orchestra.run.smithi112.stdout:max_file_size 1099511627776 2024-09-16T00:31:47.316 INFO:teuthology.orchestra.run.smithi112.stdout:max_xattr_size 65536 2024-09-16T00:31:47.316 INFO:teuthology.orchestra.run.smithi112.stdout:required_client_features {} 2024-09-16T00:31:47.316 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure 0 2024-09-16T00:31:47.316 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure_osd_epoch 44 2024-09-16T00:31:47.316 INFO:teuthology.orchestra.run.smithi112.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:31:47.316 INFO:teuthology.orchestra.run.smithi112.stdout:max_mds 2 2024-09-16T00:31:47.316 INFO:teuthology.orchestra.run.smithi112.stdout:in 0,1 2024-09-16T00:31:47.316 INFO:teuthology.orchestra.run.smithi112.stdout:up {0=14480,1=24259} 2024-09-16T00:31:47.316 INFO:teuthology.orchestra.run.smithi112.stdout:failed 2024-09-16T00:31:47.316 INFO:teuthology.orchestra.run.smithi112.stdout:damaged 2024-09-16T00:31:47.316 INFO:teuthology.orchestra.run.smithi112.stdout:stopped 2024-09-16T00:31:47.316 INFO:teuthology.orchestra.run.smithi112.stdout:data_pools [3] 2024-09-16T00:31:47.316 INFO:teuthology.orchestra.run.smithi112.stdout:metadata_pool 2 2024-09-16T00:31:47.317 INFO:teuthology.orchestra.run.smithi112.stdout:inline_data enabled 2024-09-16T00:31:47.317 INFO:teuthology.orchestra.run.smithi112.stdout:balancer 2024-09-16T00:31:47.317 INFO:teuthology.orchestra.run.smithi112.stdout:bal_rank_mask -1 2024-09-16T00:31:47.317 INFO:teuthology.orchestra.run.smithi112.stdout:standby_count_wanted 1 2024-09-16T00:31:47.317 INFO:teuthology.orchestra.run.smithi112.stdout:qdb_cluster leader: 14480 members: 24259,14480 2024-09-16T00:31:47.317 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.hxfwkh{0:14480} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.112:6828/486654885,v1:172.21.15.112:6829/486654885] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T00:31:47.317 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.ithwzj{0:14492} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.112:6826/381830565,v1:172.21.15.112:6827/381830565] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T00:31:47.317 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.gawwgw{1:24259} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.169:6824/2892476953,v1:172.21.15.169:6825/2892476953] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T00:31:47.317 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.lgeqrj{1:24269} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.169:6826/861725728,v1:172.21.15.169:6827/861725728] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T00:31:47.317 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:31:47.317 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:31:47.318 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 23 2024-09-16T00:31:47.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.317+0000 7f528af0b640 1 -- 172.21.15.112:0/2105145671 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f527406e530 msgr2=0x7f52740709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:47.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.317+0000 7f528af0b640 1 --2- 172.21.15.112:0/2105145671 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f527406e530 0x7f52740709e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5278007ac0 tx=0x7f5278035040 comp rx=0 tx=0).stop 2024-09-16T00:31:47.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.317+0000 7f528af0b640 1 -- 172.21.15.112:0/2105145671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52840fe460 msgr2=0x7f5284113740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:47.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.317+0000 7f528af0b640 1 --2- 172.21.15.112:0/2105145671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52840fe460 0x7f5284113740 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f527000b790 tx=0x7f52700091d0 comp rx=0 tx=0).stop 2024-09-16T00:31:47.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.317+0000 7f528af0b640 1 -- 172.21.15.112:0/2105145671 shutdown_connections 2024-09-16T00:31:47.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.317+0000 7f528af0b640 1 --2- 172.21.15.112:0/2105145671 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f527406e530 0x7f52740709e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:47.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.317+0000 7f528af0b640 1 --2- 172.21.15.112:0/2105145671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52840fe460 0x7f5284113740 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:47.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.317+0000 7f528af0b640 1 --2- 172.21.15.112:0/2105145671 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5284064fc0 0x7f5284113200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:47.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.317+0000 7f528af0b640 1 -- 172.21.15.112:0/2105145671 >> 172.21.15.112:0/2105145671 conn(0x7f52840723b0 msgr2=0x7f5284105410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:47.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.317+0000 7f528af0b640 1 -- 172.21.15.112:0/2105145671 shutdown_connections 2024-09-16T00:31:47.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.317+0000 7f528af0b640 1 -- 172.21.15.112:0/2105145671 wait complete. 2024-09-16T00:31:47.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.402+0000 7fd9ea77c640 1 -- 172.21.15.112:0/2808653665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9e40fb810 msgr2=0x7fd9e40fbbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:47.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.402+0000 7fd9ea77c640 1 --2- 172.21.15.112:0/2808653665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9e40fb810 0x7fd9e40fbbe0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd9d40048d0 tx=0x7fd9d4009da0 comp rx=0 tx=0).stop 2024-09-16T00:31:47.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.402+0000 7fd9ea77c640 1 -- 172.21.15.112:0/2808653665 shutdown_connections 2024-09-16T00:31:47.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.402+0000 7fd9ea77c640 1 --2- 172.21.15.112:0/2808653665 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9e40fcf80 0x7fd9e40fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:47.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.402+0000 7fd9ea77c640 1 --2- 172.21.15.112:0/2808653665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9e40fb810 0x7fd9e40fbbe0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:47.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.402+0000 7fd9ea77c640 1 -- 172.21.15.112:0/2808653665 >> 172.21.15.112:0/2808653665 conn(0x7fd9e40723b0 msgr2=0x7fd9e40727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:47.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.402+0000 7fd9ea77c640 1 -- 172.21.15.112:0/2808653665 shutdown_connections 2024-09-16T00:31:47.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.402+0000 7fd9ea77c640 1 -- 172.21.15.112:0/2808653665 wait complete. 2024-09-16T00:31:47.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.403+0000 7fd9ea77c640 1 Processor -- start 2024-09-16T00:31:47.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.403+0000 7fd9ea77c640 1 -- start start 2024-09-16T00:31:47.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.403+0000 7fd9ea77c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9e40fb810 0x7fd9e41131e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:47.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.403+0000 7fd9ea77c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9e40fcf80 0x7fd9e4113720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:47.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.403+0000 7fd9ea77c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e4113d20 con 0x7fd9e40fb810 2024-09-16T00:31:47.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.403+0000 7fd9ea77c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e410d160 con 0x7fd9e40fcf80 2024-09-16T00:31:47.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.403+0000 7fd9e3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9e40fb810 0x7fd9e41131e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:47.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.404+0000 7fd9e3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9e40fb810 0x7fd9e41131e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41814/0 (socket says 172.21.15.112:41814) 2024-09-16T00:31:47.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.404+0000 7fd9e3fff640 1 -- 172.21.15.112:0/2073127624 learned_addr learned my addr 172.21.15.112:0/2073127624 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:31:47.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.404+0000 7fd9e37fe640 1 --2- 172.21.15.112:0/2073127624 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9e40fcf80 0x7fd9e4113720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:47.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.404+0000 7fd9e3fff640 1 -- 172.21.15.112:0/2073127624 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9e40fcf80 msgr2=0x7fd9e4113720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:47.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.404+0000 7fd9e3fff640 1 --2- 172.21.15.112:0/2073127624 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9e40fcf80 0x7fd9e4113720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:47.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.404+0000 7fd9e3fff640 1 -- 172.21.15.112:0/2073127624 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d4005040 con 0x7fd9e40fb810 2024-09-16T00:31:47.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.404+0000 7fd9e3fff640 1 --2- 172.21.15.112:0/2073127624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9e40fb810 0x7fd9e41131e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4003080 tx=0x7fd9d4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:47.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.404+0000 7fd9e0ff9640 1 -- 172.21.15.112:0/2073127624 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd9d4038070 con 0x7fd9e40fb810 2024-09-16T00:31:47.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.404+0000 7fd9e0ff9640 1 -- 172.21.15.112:0/2073127624 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fd9d40097e0 con 0x7fd9e40fb810 2024-09-16T00:31:47.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.404+0000 7fd9ea77c640 1 -- 172.21.15.112:0/2073127624 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9e410d3e0 con 0x7fd9e40fb810 2024-09-16T00:31:47.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.404+0000 7fd9ea77c640 1 -- 172.21.15.112:0/2073127624 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9e410d920 con 0x7fd9e40fb810 2024-09-16T00:31:47.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.404+0000 7fd9e0ff9640 1 -- 172.21.15.112:0/2073127624 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd9d403d420 con 0x7fd9e40fb810 2024-09-16T00:31:47.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.406+0000 7fd9ea77c640 1 -- 172.21.15.112:0/2073127624 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9d0005210 con 0x7fd9e40fb810 2024-09-16T00:31:47.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.406+0000 7fd9e0ff9640 1 -- 172.21.15.112:0/2073127624 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fd9d4030020 con 0x7fd9e40fb810 2024-09-16T00:31:47.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.407+0000 7fd9e0ff9640 1 --2- 172.21.15.112:0/2073127624 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd9cc06e530 0x7fd9cc0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:47.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.407+0000 7fd9e0ff9640 1 -- 172.21.15.112:0/2073127624 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fd9d40ae460 con 0x7fd9e40fb810 2024-09-16T00:31:47.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.407+0000 7fd9e37fe640 1 --2- 172.21.15.112:0/2073127624 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd9cc06e530 0x7fd9cc0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:47.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.408+0000 7fd9e37fe640 1 --2- 172.21.15.112:0/2073127624 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd9cc06e530 0x7fd9cc0709e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd9e410eaa0 tx=0x7fd9d8009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:47.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.411+0000 7fd9e0ff9640 1 -- 172.21.15.112:0/2073127624 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fd9d4077620 con 0x7fd9e40fb810 2024-09-16T00:31:47.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:47 smithi112 ceph-mon[97750]: from='client.34254 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:31:47.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:47 smithi112 ceph-mon[97750]: from='client.34258 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:31:47.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:47 smithi112 ceph-mon[97750]: pgmap v90: 65 pgs: 14 active+undersized, 13 active+undersized+degraded, 38 active+clean; 325 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 45/372 objects degraded (12.097%) 2024-09-16T00:31:47.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:47 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:31:47.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:47 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:31:47.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:47 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:31:47.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:47 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:31:47.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:47 smithi112 ceph-mon[97750]: from='client.34262 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:31:47.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:47 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/2801164423' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:31:47.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:47 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:31:47.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:47 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:31:47.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:47 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/2105145671' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:31:47.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.613+0000 7fd9ea77c640 1 -- 172.21.15.112:0/2073127624 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd9d0000bc0 con 0x7fd9cc06e530 2024-09-16T00:31:47.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.614+0000 7fd9e0ff9640 1 -- 172.21.15.112:0/2073127624 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fd9d0000bc0 con 0x7fd9cc06e530 2024-09-16T00:31:47.615 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:31:47.615 INFO:teuthology.orchestra.run.smithi112.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80", 2024-09-16T00:31:47.615 INFO:teuthology.orchestra.run.smithi112.stdout: "in_progress": true, 2024-09-16T00:31:47.615 INFO:teuthology.orchestra.run.smithi112.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T00:31:47.615 INFO:teuthology.orchestra.run.smithi112.stdout: "services_complete": [ 2024-09-16T00:31:47.615 INFO:teuthology.orchestra.run.smithi112.stdout: "crash", 2024-09-16T00:31:47.615 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr", 2024-09-16T00:31:47.615 INFO:teuthology.orchestra.run.smithi112.stdout: "mon" 2024-09-16T00:31:47.615 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-16T00:31:47.615 INFO:teuthology.orchestra.run.smithi112.stdout: "progress": "9/21 daemons upgraded", 2024-09-16T00:31:47.615 INFO:teuthology.orchestra.run.smithi112.stdout: "message": "Currently upgrading osd daemons", 2024-09-16T00:31:47.615 INFO:teuthology.orchestra.run.smithi112.stdout: "is_paused": false 2024-09-16T00:31:47.615 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:31:47.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.617+0000 7fd9ea77c640 1 -- 172.21.15.112:0/2073127624 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd9cc06e530 msgr2=0x7fd9cc0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:47.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.617+0000 7fd9ea77c640 1 --2- 172.21.15.112:0/2073127624 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd9cc06e530 0x7fd9cc0709e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd9e410eaa0 tx=0x7fd9d8009040 comp rx=0 tx=0).stop 2024-09-16T00:31:47.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.617+0000 7fd9ea77c640 1 -- 172.21.15.112:0/2073127624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9e40fb810 msgr2=0x7fd9e41131e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:47.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.617+0000 7fd9ea77c640 1 --2- 172.21.15.112:0/2073127624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9e40fb810 0x7fd9e41131e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4003080 tx=0x7fd9d4009da0 comp rx=0 tx=0).stop 2024-09-16T00:31:47.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.617+0000 7fd9ea77c640 1 -- 172.21.15.112:0/2073127624 shutdown_connections 2024-09-16T00:31:47.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.617+0000 7fd9ea77c640 1 --2- 172.21.15.112:0/2073127624 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd9cc06e530 0x7fd9cc0709e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:47.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.617+0000 7fd9ea77c640 1 --2- 172.21.15.112:0/2073127624 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9e40fcf80 0x7fd9e4113720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:47.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.617+0000 7fd9ea77c640 1 --2- 172.21.15.112:0/2073127624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9e40fb810 0x7fd9e41131e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:47.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.617+0000 7fd9ea77c640 1 -- 172.21.15.112:0/2073127624 >> 172.21.15.112:0/2073127624 conn(0x7fd9e40723b0 msgr2=0x7fd9e4064f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:47.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.617+0000 7fd9ea77c640 1 -- 172.21.15.112:0/2073127624 shutdown_connections 2024-09-16T00:31:47.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.617+0000 7fd9ea77c640 1 -- 172.21.15.112:0/2073127624 wait complete. 2024-09-16T00:31:47.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.699+0000 7f9df8f96640 1 -- 172.21.15.112:0/2441170671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9df4102ed0 msgr2=0x7f9df40fc590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:47.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.699+0000 7f9df8f96640 1 --2- 172.21.15.112:0/2441170671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9df4102ed0 0x7f9df40fc590 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9de80048d0 tx=0x7f9de8009da0 comp rx=0 tx=0).stop 2024-09-16T00:31:47.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.699+0000 7f9df8f96640 1 -- 172.21.15.112:0/2441170671 shutdown_connections 2024-09-16T00:31:47.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.699+0000 7f9df8f96640 1 --2- 172.21.15.112:0/2441170671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9df4102ed0 0x7f9df40fc590 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:47.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.699+0000 7f9df8f96640 1 --2- 172.21.15.112:0/2441170671 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9df4102530 0x7f9df4102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:47.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.699+0000 7f9df8f96640 1 -- 172.21.15.112:0/2441170671 >> 172.21.15.112:0/2441170671 conn(0x7f9df40f80f0 msgr2=0x7f9df40fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:47.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.699+0000 7f9df8f96640 1 -- 172.21.15.112:0/2441170671 shutdown_connections 2024-09-16T00:31:47.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.699+0000 7f9df8f96640 1 -- 172.21.15.112:0/2441170671 wait complete. 2024-09-16T00:31:47.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.699+0000 7f9df8f96640 1 Processor -- start 2024-09-16T00:31:47.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.700+0000 7f9df8f96640 1 -- start start 2024-09-16T00:31:47.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.700+0000 7f9df8f96640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9df4102530 0x7f9df406fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:47.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.700+0000 7f9df8f96640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9df4102ed0 0x7f9df406ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:47.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.700+0000 7f9df8f96640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df4070490 con 0x7f9df4102530 2024-09-16T00:31:47.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.700+0000 7f9df8f96640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df4070600 con 0x7f9df4102ed0 2024-09-16T00:31:47.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.700+0000 7f9df259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9df4102530 0x7f9df406fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:47.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.700+0000 7f9df259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9df4102530 0x7f9df406fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41826/0 (socket says 172.21.15.112:41826) 2024-09-16T00:31:47.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.700+0000 7f9df259c640 1 -- 172.21.15.112:0/523294566 learned_addr learned my addr 172.21.15.112:0/523294566 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:31:47.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.700+0000 7f9df1d9b640 1 --2- 172.21.15.112:0/523294566 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9df4102ed0 0x7f9df406ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:47.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.700+0000 7f9df259c640 1 -- 172.21.15.112:0/523294566 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9df4102ed0 msgr2=0x7f9df406ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:47.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.700+0000 7f9df259c640 1 --2- 172.21.15.112:0/523294566 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9df4102ed0 0x7f9df406ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:47.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.700+0000 7f9df259c640 1 -- 172.21.15.112:0/523294566 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9de8005040 con 0x7f9df4102530 2024-09-16T00:31:47.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.701+0000 7f9df259c640 1 --2- 172.21.15.112:0/523294566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9df4102530 0x7f9df406fa10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9de4002a50 tx=0x7f9de4004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:47.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.701+0000 7f9de2ffd640 1 -- 172.21.15.112:0/523294566 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9de4007c30 con 0x7f9df4102530 2024-09-16T00:31:47.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.701+0000 7f9de2ffd640 1 -- 172.21.15.112:0/523294566 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f9de400f040 con 0x7f9df4102530 2024-09-16T00:31:47.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.701+0000 7f9de2ffd640 1 -- 172.21.15.112:0/523294566 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9de400da50 con 0x7f9df4102530 2024-09-16T00:31:47.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.701+0000 7f9df8f96640 1 -- 172.21.15.112:0/523294566 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9df4070890 con 0x7f9df4102530 2024-09-16T00:31:47.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.701+0000 7f9df8f96640 1 -- 172.21.15.112:0/523294566 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9df41198a0 con 0x7f9df4102530 2024-09-16T00:31:47.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.702+0000 7f9df8f96640 1 -- 172.21.15.112:0/523294566 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9df40fdcb0 con 0x7f9df4102530 2024-09-16T00:31:47.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.703+0000 7f9de2ffd640 1 -- 172.21.15.112:0/523294566 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f9de400dbf0 con 0x7f9df4102530 2024-09-16T00:31:47.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.703+0000 7f9de2ffd640 1 --2- 172.21.15.112:0/523294566 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f9ddc06e530 0x7f9ddc0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:47.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.703+0000 7f9de2ffd640 1 -- 172.21.15.112:0/523294566 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f9de408d870 con 0x7f9df4102530 2024-09-16T00:31:47.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.703+0000 7f9df1d9b640 1 --2- 172.21.15.112:0/523294566 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f9ddc06e530 0x7f9ddc0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:47.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.704+0000 7f9df1d9b640 1 --2- 172.21.15.112:0/523294566 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f9ddc06e530 0x7f9ddc0709e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9de80048a0 tx=0x7f9de8035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:47.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:47.707+0000 7f9de2ffd640 1 -- 172.21.15.112:0/523294566 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f9de4056a60 con 0x7f9df4102530 2024-09-16T00:31:48.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:48.010+0000 7f9df8f96640 1 -- 172.21.15.112:0/523294566 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f9df4109560 con 0x7f9df4102530 2024-09-16T00:31:48.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:48.010+0000 7f9de2ffd640 1 -- 172.21.15.112:0/523294566 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1186 (secure 0 0 0) 0x7f9de405a120 con 0x7f9df4102530 2024-09-16T00:31:48.011 INFO:teuthology.orchestra.run.smithi112.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; 1 osds down; Degraded data redundancy: 45/372 objects degraded (12.097%), 13 pgs degraded 2024-09-16T00:31:48.012 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T00:31:48.012 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T00:31:48.012 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] OSD_DOWN: 1 osds down 2024-09-16T00:31:48.012 INFO:teuthology.orchestra.run.smithi112.stdout: osd.2 (root=default,host=smithi112) is down 2024-09-16T00:31:48.012 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 45/372 objects degraded (12.097%), 13 pgs degraded 2024-09-16T00:31:48.012 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.1 is active+undersized+degraded, acting [1,0] 2024-09-16T00:31:48.012 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.3 is active+undersized+degraded, acting [5,1] 2024-09-16T00:31:48.012 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.7 is active+undersized+degraded, acting [3,4] 2024-09-16T00:31:48.012 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.c is active+undersized+degraded, acting [3,0] 2024-09-16T00:31:48.012 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.d is active+undersized+degraded, acting [1,3] 2024-09-16T00:31:48.012 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.e is active+undersized+degraded, acting [0,3] 2024-09-16T00:31:48.013 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.10 is active+undersized+degraded, acting [1,0] 2024-09-16T00:31:48.013 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.13 is active+undersized+degraded, acting [0,4] 2024-09-16T00:31:48.013 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.16 is active+undersized+degraded, acting [5,3] 2024-09-16T00:31:48.013 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.17 is active+undersized+degraded, acting [5,1] 2024-09-16T00:31:48.013 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.19 is active+undersized+degraded, acting [0,4] 2024-09-16T00:31:48.013 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.1c is active+undersized+degraded, acting [4,5] 2024-09-16T00:31:48.013 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.1e is active+undersized+degraded, acting [0,5] 2024-09-16T00:31:48.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:48.013+0000 7f9df8f96640 1 -- 172.21.15.112:0/523294566 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f9ddc06e530 msgr2=0x7f9ddc0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:48.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:48.013+0000 7f9df8f96640 1 --2- 172.21.15.112:0/523294566 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f9ddc06e530 0x7f9ddc0709e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9de80048a0 tx=0x7f9de8035040 comp rx=0 tx=0).stop 2024-09-16T00:31:48.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:48.013+0000 7f9df8f96640 1 -- 172.21.15.112:0/523294566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9df4102530 msgr2=0x7f9df406fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:48.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:48.013+0000 7f9df8f96640 1 --2- 172.21.15.112:0/523294566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9df4102530 0x7f9df406fa10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9de4002a50 tx=0x7f9de4004600 comp rx=0 tx=0).stop 2024-09-16T00:31:48.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:48.013+0000 7f9df8f96640 1 -- 172.21.15.112:0/523294566 shutdown_connections 2024-09-16T00:31:48.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:48.013+0000 7f9df8f96640 1 --2- 172.21.15.112:0/523294566 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f9ddc06e530 0x7f9ddc0709e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:48.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:48.013+0000 7f9df8f96640 1 --2- 172.21.15.112:0/523294566 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9df4102ed0 0x7f9df406ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:48.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:48.013+0000 7f9df8f96640 1 --2- 172.21.15.112:0/523294566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9df4102530 0x7f9df406fa10 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:48.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:48.013+0000 7f9df8f96640 1 -- 172.21.15.112:0/523294566 >> 172.21.15.112:0/523294566 conn(0x7f9df40f80f0 msgr2=0x7f9df40f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:48.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:48.014+0000 7f9df8f96640 1 -- 172.21.15.112:0/523294566 shutdown_connections 2024-09-16T00:31:48.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:31:48.014+0000 7f9df8f96640 1 -- 172.21.15.112:0/523294566 wait complete. 2024-09-16T00:31:48.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:47 smithi169 ceph-mon[77626]: from='client.34254 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:31:48.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:47 smithi169 ceph-mon[77626]: from='client.34258 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:31:48.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:47 smithi169 ceph-mon[77626]: pgmap v90: 65 pgs: 14 active+undersized, 13 active+undersized+degraded, 38 active+clean; 325 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 45/372 objects degraded (12.097%) 2024-09-16T00:31:48.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:47 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:31:48.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:47 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:31:48.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:47 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:31:48.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:47 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:31:48.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:47 smithi169 ceph-mon[77626]: from='client.34262 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:31:48.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:47 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/2801164423' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:31:48.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:47 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:31:48.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:47 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:31:48.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:47 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/2105145671' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:31:48.339 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:31:48 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-2[120025]: 2024-09-16T00:31:48.131+0000 7f749b0d3640 -1 osd.2 57 log_to_monitors true 2024-09-16T00:31:48.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:48 smithi112 ceph-mon[97750]: from='client.34274 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:31:48.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:48 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/523294566' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:31:48.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:48 smithi112 ceph-mon[97750]: from='osd.2 [v2:172.21.15.112:6818/2239852213,v1:172.21.15.112:6819/2239852213]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T00:31:49.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:48 smithi169 ceph-mon[77626]: from='client.34274 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:31:49.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:48 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/523294566' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:31:49.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:48 smithi169 ceph-mon[77626]: from='osd.2 [v2:172.21.15.112:6818/2239852213,v1:172.21.15.112:6819/2239852213]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T00:31:49.694 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:31:49 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-2[120025]: 2024-09-16T00:31:49.402+0000 7f748ee65640 -1 osd.2 57 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T00:31:49.695 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:49 smithi112 ceph-mon[97750]: pgmap v91: 65 pgs: 14 active+undersized, 13 active+undersized+degraded, 38 active+clean; 325 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 45/372 objects degraded (12.097%) 2024-09-16T00:31:49.695 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:49 smithi112 ceph-mon[97750]: from='osd.2 [v2:172.21.15.112:6818/2239852213,v1:172.21.15.112:6819/2239852213]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-16T00:31:49.695 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:49 smithi112 ceph-mon[97750]: osdmap e60: 6 total, 5 up, 6 in 2024-09-16T00:31:49.695 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:49 smithi112 ceph-mon[97750]: from='osd.2 [v2:172.21.15.112:6818/2239852213,v1:172.21.15.112:6819/2239852213]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-09-16T00:31:49.695 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:49 smithi112 ceph-mon[97750]: from='osd.2 [v2:172.21.15.112:6818/2239852213,v1:172.21.15.112:6819/2239852213]' entity='osd.2' 2024-09-16T00:31:49.927 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:49 smithi169 ceph-mon[77626]: pgmap v91: 65 pgs: 14 active+undersized, 13 active+undersized+degraded, 38 active+clean; 325 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 45/372 objects degraded (12.097%) 2024-09-16T00:31:49.927 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:49 smithi169 ceph-mon[77626]: from='osd.2 [v2:172.21.15.112:6818/2239852213,v1:172.21.15.112:6819/2239852213]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-16T00:31:49.928 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:49 smithi169 ceph-mon[77626]: osdmap e60: 6 total, 5 up, 6 in 2024-09-16T00:31:49.928 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:49 smithi169 ceph-mon[77626]: from='osd.2 [v2:172.21.15.112:6818/2239852213,v1:172.21.15.112:6819/2239852213]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-09-16T00:31:49.928 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:49 smithi169 ceph-mon[77626]: from='osd.2 [v2:172.21.15.112:6818/2239852213,v1:172.21.15.112:6819/2239852213]' entity='osd.2' 2024-09-16T00:31:50.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:50 smithi112 ceph-mon[97750]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-16T00:31:50.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:50 smithi112 ceph-mon[97750]: osd.2 [v2:172.21.15.112:6818/2239852213,v1:172.21.15.112:6819/2239852213] boot 2024-09-16T00:31:50.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:50 smithi112 ceph-mon[97750]: osdmap e61: 6 total, 6 up, 6 in 2024-09-16T00:31:50.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:50 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:31:50.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:50 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:31:51.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:50 smithi169 ceph-mon[77626]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-16T00:31:51.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:50 smithi169 ceph-mon[77626]: osd.2 [v2:172.21.15.112:6818/2239852213,v1:172.21.15.112:6819/2239852213] boot 2024-09-16T00:31:51.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:50 smithi169 ceph-mon[77626]: osdmap e61: 6 total, 6 up, 6 in 2024-09-16T00:31:51.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:50 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:31:51.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:50 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:31:51.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:51 smithi112 ceph-mon[97750]: pgmap v94: 65 pgs: 14 active+undersized, 13 active+undersized+degraded, 38 active+clean; 325 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s; 45/372 objects degraded (12.097%) 2024-09-16T00:31:51.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:51 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:31:51.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:51 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:31:51.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:51 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:31:51.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:51 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:31:51.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:51 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:31:51.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:51 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:31:51.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:51 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:31:51.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:51 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:31:51.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:51 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-09-16T00:31:51.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:51 smithi112 ceph-mon[97750]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-09-16T00:31:51.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:51 smithi112 ceph-mon[97750]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-09-16T00:31:51.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:51 smithi112 ceph-mon[97750]: osdmap e62: 6 total, 6 up, 6 in 2024-09-16T00:31:52.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:51 smithi169 ceph-mon[77626]: pgmap v94: 65 pgs: 14 active+undersized, 13 active+undersized+degraded, 38 active+clean; 325 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s; 45/372 objects degraded (12.097%) 2024-09-16T00:31:52.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:51 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:31:52.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:51 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:31:52.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:51 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:31:52.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:51 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:31:52.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:51 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:31:52.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:51 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:31:52.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:51 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:31:52.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:51 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:31:52.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:51 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-09-16T00:31:52.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:51 smithi169 ceph-mon[77626]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-09-16T00:31:52.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:51 smithi169 ceph-mon[77626]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-09-16T00:31:52.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:51 smithi169 ceph-mon[77626]: osdmap e62: 6 total, 6 up, 6 in 2024-09-16T00:31:53.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:53 smithi112 ceph-mon[97750]: pgmap v96: 65 pgs: 14 active+undersized, 13 active+undersized+degraded, 38 active+clean; 325 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 45/372 objects degraded (12.097%) 2024-09-16T00:31:54.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:53 smithi169 ceph-mon[77626]: pgmap v96: 65 pgs: 14 active+undersized, 13 active+undersized+degraded, 38 active+clean; 325 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 45/372 objects degraded (12.097%) 2024-09-16T00:31:54.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:54 smithi112 ceph-mon[97750]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 45/372 objects degraded (12.097%), 13 pgs degraded) 2024-09-16T00:31:55.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:54 smithi169 ceph-mon[77626]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 45/372 objects degraded (12.097%), 13 pgs degraded) 2024-09-16T00:31:56.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:55 smithi169 ceph-mon[77626]: pgmap v97: 65 pgs: 65 active+clean; 325 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.3 KiB/s rd, 3 op/s 2024-09-16T00:31:56.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:55 smithi112 ceph-mon[97750]: pgmap v97: 65 pgs: 65 active+clean; 325 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.3 KiB/s rd, 3 op/s 2024-09-16T00:31:58.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:57 smithi169 ceph-mon[77626]: pgmap v98: 65 pgs: 65 active+clean; 325 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.0 KiB/s rd, 3 op/s 2024-09-16T00:31:58.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:57 smithi112 ceph-mon[97750]: pgmap v98: 65 pgs: 65 active+clean; 325 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.0 KiB/s rd, 3 op/s 2024-09-16T00:32:00.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:31:59 smithi169 ceph-mon[77626]: pgmap v99: 65 pgs: 65 active+clean; 325 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.2 KiB/s rd, 3 op/s 2024-09-16T00:32:00.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:31:59 smithi112 ceph-mon[97750]: pgmap v99: 65 pgs: 65 active+clean; 325 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.2 KiB/s rd, 3 op/s 2024-09-16T00:32:02.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:01 smithi169 ceph-mon[77626]: pgmap v100: 65 pgs: 65 active+clean; 325 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-09-16T00:32:02.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:01 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:32:02.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:01 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:32:02.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:01 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:32:02.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:01 smithi112 ceph-mon[97750]: pgmap v100: 65 pgs: 65 active+clean; 325 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-09-16T00:32:02.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:01 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:32:02.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:01 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:32:02.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:01 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:32:03.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:03 smithi112 ceph-mon[97750]: pgmap v101: 65 pgs: 65 active+clean; 325 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-09-16T00:32:04.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:03 smithi169 ceph-mon[77626]: pgmap v101: 65 pgs: 65 active+clean; 325 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-09-16T00:32:06.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:05 smithi169 ceph-mon[77626]: pgmap v102: 65 pgs: 65 active+clean; 325 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.3 KiB/s rd, 4 op/s 2024-09-16T00:32:06.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:05 smithi112 ceph-mon[97750]: pgmap v102: 65 pgs: 65 active+clean; 325 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.3 KiB/s rd, 4 op/s 2024-09-16T00:32:07.028 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:06 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-09-16T00:32:07.030 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:06 smithi169 ceph-mon[77626]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-09-16T00:32:07.030 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:06 smithi169 ceph-mon[77626]: Upgrade: osd.3 is safe to restart 2024-09-16T00:32:07.092 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:06 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-09-16T00:32:07.092 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:06 smithi112 ceph-mon[97750]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-09-16T00:32:07.092 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:06 smithi112 ceph-mon[97750]: Upgrade: osd.3 is safe to restart 2024-09-16T00:32:07.826 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:07 smithi169 ceph-mon[77626]: pgmap v103: 65 pgs: 65 active+clean; 325 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:32:07.826 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:07 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:32:07.827 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:07 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-16T00:32:07.827 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:07 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:32:08.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:07 smithi112 ceph-mon[97750]: pgmap v103: 65 pgs: 65 active+clean; 325 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:32:08.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:07 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:32:08.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:07 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-16T00:32:08.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:07 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:32:08.916 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:08 smithi169 ceph-mon[77626]: Upgrade: Updating osd.3 2024-09-16T00:32:08.916 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:08 smithi169 ceph-mon[77626]: Deploying daemon osd.3 on smithi169 2024-09-16T00:32:09.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:08 smithi112 ceph-mon[97750]: Upgrade: Updating osd.3 2024-09-16T00:32:09.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:08 smithi112 ceph-mon[97750]: Deploying daemon osd.3 on smithi169 2024-09-16T00:32:09.755 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:09 smithi169 ceph-mon[77626]: pgmap v104: 65 pgs: 65 active+clean; 325 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:32:10.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:09 smithi112 ceph-mon[97750]: pgmap v104: 65 pgs: 65 active+clean; 325 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:32:10.393 INFO:journalctl@ceph.osd.3.smithi169.stdout:Sep 16 00:32:10 smithi169 systemd[1]: Stopping Ceph osd.3 for d346dd56-73c0-11ef-bceb-c7b262605968... 2024-09-16T00:32:10.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:10 smithi169 ceph-mon[77626]: osd.3 marked itself down and dead 2024-09-16T00:32:10.776 INFO:journalctl@ceph.osd.3.smithi169.stdout:Sep 16 00:32:10 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-3[40159]: 2024-09-16T00:32:10.392+0000 7fb24eb62640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:32:10.776 INFO:journalctl@ceph.osd.3.smithi169.stdout:Sep 16 00:32:10 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-3[40159]: 2024-09-16T00:32:10.392+0000 7fb24eb62640 -1 osd.3 62 *** Got signal Terminated *** 2024-09-16T00:32:10.777 INFO:journalctl@ceph.osd.3.smithi169.stdout:Sep 16 00:32:10 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-3[40159]: 2024-09-16T00:32:10.392+0000 7fb24eb62640 -1 osd.3 62 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:32:11.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:10 smithi112 ceph-mon[97750]: osd.3 marked itself down and dead 2024-09-16T00:32:11.318 INFO:journalctl@ceph.osd.3.smithi169.stdout:Sep 16 00:32:11 smithi169 podman[85989]: 2024-09-16 00:32:11.038098661 +0000 UTC m=+0.766597846 container died f88463e19f2331a424b7db38539efbe19cfc496f9d566cb8878eb19114519e44 (image=quay.ceph.io/ceph-ci/ceph@sha256:83596473fc70117d507bb3d925c5bd11b70f46bbd4dffcc8452f145d93eaedc4, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-3, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=quincy-6d025bc, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:32:11.638 INFO:journalctl@ceph.osd.3.smithi169.stdout:Sep 16 00:32:11 smithi169 podman[85989]: 2024-09-16 00:32:11.409699225 +0000 UTC m=+1.138198399 container cleanup f88463e19f2331a424b7db38539efbe19cfc496f9d566cb8878eb19114519e44 (image=quay.ceph.io/ceph-ci/ceph@sha256:83596473fc70117d507bb3d925c5bd11b70f46bbd4dffcc8452f145d93eaedc4, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=quincy-6d025bc, org.label-schema.license=GPLv2) 2024-09-16T00:32:11.639 INFO:journalctl@ceph.osd.3.smithi169.stdout:Sep 16 00:32:11 smithi169 bash[85989]: ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-3 2024-09-16T00:32:12.026 INFO:journalctl@ceph.osd.3.smithi169.stdout:Sep 16 00:32:11 smithi169 podman[86008]: 2024-09-16 00:32:11.686736358 +0000 UTC m=+0.642322871 container remove f88463e19f2331a424b7db38539efbe19cfc496f9d566cb8878eb19114519e44 (image=quay.ceph.io/ceph-ci/ceph@sha256:83596473fc70117d507bb3d925c5bd11b70f46bbd4dffcc8452f145d93eaedc4, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-3, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=quincy-6d025bc, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD) 2024-09-16T00:32:12.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:11 smithi169 ceph-mon[77626]: pgmap v105: 65 pgs: 65 active+clean; 325 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-16T00:32:12.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:11 smithi169 ceph-mon[77626]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-16T00:32:12.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:11 smithi169 ceph-mon[77626]: osdmap e63: 6 total, 5 up, 6 in 2024-09-16T00:32:12.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:11 smithi112 ceph-mon[97750]: pgmap v105: 65 pgs: 65 active+clean; 325 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-16T00:32:12.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:11 smithi112 ceph-mon[97750]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-16T00:32:12.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:11 smithi112 ceph-mon[97750]: osdmap e63: 6 total, 5 up, 6 in 2024-09-16T00:32:12.302 INFO:journalctl@ceph.osd.3.smithi169.stdout:Sep 16 00:32:12 smithi169 podman[86095]: 2024-09-16 00:32:11.929910571 +0000 UTC m=+0.015945458 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:32:12.302 INFO:journalctl@ceph.osd.3.smithi169.stdout:Sep 16 00:32:12 smithi169 podman[86095]: 2024-09-16 00:32:12.072181986 +0000 UTC m=+0.158216873 container create c6859d76d9ba20672e988783f0f79c8d3836c9906a99972760e536754a9c32e1 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:32:12.302 INFO:journalctl@ceph.osd.3.smithi169.stdout:Sep 16 00:32:12 smithi169 podman[86095]: 2024-09-16 00:32:12.301046924 +0000 UTC m=+0.387081811 container init c6859d76d9ba20672e988783f0f79c8d3836c9906a99972760e536754a9c32e1 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-16T00:32:12.653 INFO:journalctl@ceph.osd.3.smithi169.stdout:Sep 16 00:32:12 smithi169 podman[86095]: 2024-09-16 00:32:12.304592332 +0000 UTC m=+0.390627226 container start c6859d76d9ba20672e988783f0f79c8d3836c9906a99972760e536754a9c32e1 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-16T00:32:12.653 INFO:journalctl@ceph.osd.3.smithi169.stdout:Sep 16 00:32:12 smithi169 podman[86095]: 2024-09-16 00:32:12.402657762 +0000 UTC m=+0.488692666 container attach c6859d76d9ba20672e988783f0f79c8d3836c9906a99972760e536754a9c32e1 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e) 2024-09-16T00:32:12.653 INFO:journalctl@ceph.osd.3.smithi169.stdout:Sep 16 00:32:12 smithi169 podman[86095]: 2024-09-16 00:32:12.495974687 +0000 UTC m=+0.582009584 container died c6859d76d9ba20672e988783f0f79c8d3836c9906a99972760e536754a9c32e1 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:32:13.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:12 smithi169 ceph-mon[77626]: osdmap e64: 6 total, 5 up, 6 in 2024-09-16T00:32:13.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:12 smithi112 ceph-mon[97750]: osdmap e64: 6 total, 5 up, 6 in 2024-09-16T00:32:13.526 INFO:journalctl@ceph.osd.3.smithi169.stdout:Sep 16 00:32:13 smithi169 podman[86118]: 2024-09-16 00:32:13.126234148 +0000 UTC m=+0.624247889 container remove c6859d76d9ba20672e988783f0f79c8d3836c9906a99972760e536754a9c32e1 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-16T00:32:13.527 INFO:journalctl@ceph.osd.3.smithi169.stdout:Sep 16 00:32:13 smithi169 systemd[1]: ceph-d346dd56-73c0-11ef-bceb-c7b262605968@osd.3.service: Deactivated successfully. 2024-09-16T00:32:13.527 INFO:journalctl@ceph.osd.3.smithi169.stdout:Sep 16 00:32:13 smithi169 systemd[1]: Stopped Ceph osd.3 for d346dd56-73c0-11ef-bceb-c7b262605968. 2024-09-16T00:32:13.527 INFO:journalctl@ceph.osd.3.smithi169.stdout:Sep 16 00:32:13 smithi169 systemd[1]: ceph-d346dd56-73c0-11ef-bceb-c7b262605968@osd.3.service: Consumed 1min 889ms CPU time. 2024-09-16T00:32:13.527 INFO:journalctl@ceph.osd.3.smithi169.stdout:Sep 16 00:32:13 smithi169 systemd[1]: Starting Ceph osd.3 for d346dd56-73c0-11ef-bceb-c7b262605968... 2024-09-16T00:32:13.824 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:13 smithi169 ceph-mon[77626]: pgmap v108: 65 pgs: 16 stale+active+clean, 49 active+clean; 325 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-16T00:32:13.825 INFO:journalctl@ceph.osd.3.smithi169.stdout:Sep 16 00:32:13 smithi169 podman[86219]: 2024-09-16 00:32:13.55538971 +0000 UTC m=+0.106761970 container create 9bf9d824661025194ce8a77448c1ce9dcb63af536429cdb88ab190e01ef8242c (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-3-activate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-16T00:32:13.829 INFO:journalctl@ceph.osd.3.smithi169.stdout:Sep 16 00:32:13 smithi169 podman[86219]: 2024-09-16 00:32:13.465066115 +0000 UTC m=+0.016438345 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:32:13.829 INFO:journalctl@ceph.osd.3.smithi169.stdout:Sep 16 00:32:13 smithi169 podman[86219]: 2024-09-16 00:32:13.750147102 +0000 UTC m=+0.301519317 container init 9bf9d824661025194ce8a77448c1ce9dcb63af536429cdb88ab190e01ef8242c (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-3-activate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:32:13.829 INFO:journalctl@ceph.osd.3.smithi169.stdout:Sep 16 00:32:13 smithi169 podman[86219]: 2024-09-16 00:32:13.753824758 +0000 UTC m=+0.305196980 container start 9bf9d824661025194ce8a77448c1ce9dcb63af536429cdb88ab190e01ef8242c (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-3-activate, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-16T00:32:13.829 INFO:journalctl@ceph.osd.3.smithi169.stdout:Sep 16 00:32:13 smithi169 podman[86219]: 2024-09-16 00:32:13.824921276 +0000 UTC m=+0.376293492 container attach 9bf9d824661025194ce8a77448c1ce9dcb63af536429cdb88ab190e01ef8242c (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-3-activate, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-16T00:32:14.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:13 smithi112 ceph-mon[97750]: pgmap v108: 65 pgs: 16 stale+active+clean, 49 active+clean; 325 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-16T00:32:14.276 INFO:journalctl@ceph.osd.3.smithi169.stdout:Sep 16 00:32:13 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-3-activate[86235]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T00:32:14.276 INFO:journalctl@ceph.osd.3.smithi169.stdout:Sep 16 00:32:13 smithi169 bash[86219]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T00:32:14.276 INFO:journalctl@ceph.osd.3.smithi169.stdout:Sep 16 00:32:13 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-3-activate[86235]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T00:32:14.276 INFO:journalctl@ceph.osd.3.smithi169.stdout:Sep 16 00:32:13 smithi169 bash[86219]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T00:32:14.896 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:14 smithi169 ceph-mon[77626]: Health check failed: Degraded data redundancy: 78/372 objects degraded (20.968%), 20 pgs degraded (PG_DEGRADED) 2024-09-16T00:32:14.896 INFO:journalctl@ceph.osd.3.smithi169.stdout:Sep 16 00:32:14 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-3-activate[86235]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-16T00:32:14.896 INFO:journalctl@ceph.osd.3.smithi169.stdout:Sep 16 00:32:14 smithi169 bash[86219]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-16T00:32:14.896 INFO:journalctl@ceph.osd.3.smithi169.stdout:Sep 16 00:32:14 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-3-activate[86235]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-09-16T00:32:14.896 INFO:journalctl@ceph.osd.3.smithi169.stdout:Sep 16 00:32:14 smithi169 bash[86219]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-09-16T00:32:15.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:14 smithi112 ceph-mon[97750]: Health check failed: Degraded data redundancy: 78/372 objects degraded (20.968%), 20 pgs degraded (PG_DEGRADED) 2024-09-16T00:32:15.203 INFO:journalctl@ceph.osd.3.smithi169.stdout:Sep 16 00:32:14 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-3-activate[86235]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-09-16T00:32:15.203 INFO:journalctl@ceph.osd.3.smithi169.stdout:Sep 16 00:32:14 smithi169 bash[86219]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-09-16T00:32:15.203 INFO:journalctl@ceph.osd.3.smithi169.stdout:Sep 16 00:32:14 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-3-activate[86235]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-09-16T00:32:15.203 INFO:journalctl@ceph.osd.3.smithi169.stdout:Sep 16 00:32:14 smithi169 bash[86219]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-09-16T00:32:15.203 INFO:journalctl@ceph.osd.3.smithi169.stdout:Sep 16 00:32:14 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-3-activate[86235]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-09-16T00:32:15.203 INFO:journalctl@ceph.osd.3.smithi169.stdout:Sep 16 00:32:14 smithi169 bash[86219]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-09-16T00:32:15.203 INFO:journalctl@ceph.osd.3.smithi169.stdout:Sep 16 00:32:14 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-3-activate[86235]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-16T00:32:15.204 INFO:journalctl@ceph.osd.3.smithi169.stdout:Sep 16 00:32:14 smithi169 bash[86219]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-16T00:32:15.204 INFO:journalctl@ceph.osd.3.smithi169.stdout:Sep 16 00:32:14 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-3-activate[86235]: --> ceph-volume raw activate successful for osd ID: 3 2024-09-16T00:32:15.204 INFO:journalctl@ceph.osd.3.smithi169.stdout:Sep 16 00:32:14 smithi169 bash[86219]: --> ceph-volume raw activate successful for osd ID: 3 2024-09-16T00:32:15.204 INFO:journalctl@ceph.osd.3.smithi169.stdout:Sep 16 00:32:14 smithi169 podman[86219]: 2024-09-16 00:32:14.92458484 +0000 UTC m=+1.475957064 container died 9bf9d824661025194ce8a77448c1ce9dcb63af536429cdb88ab190e01ef8242c (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-3-activate, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-8921c8e) 2024-09-16T00:32:15.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:15 smithi169 ceph-mon[77626]: pgmap v109: 65 pgs: 18 active+undersized, 20 active+undersized+degraded, 27 active+clean; 325 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 78/372 objects degraded (20.968%) 2024-09-16T00:32:15.776 INFO:journalctl@ceph.osd.3.smithi169.stdout:Sep 16 00:32:15 smithi169 podman[86365]: 2024-09-16 00:32:15.478942454 +0000 UTC m=+0.548791907 container remove 9bf9d824661025194ce8a77448c1ce9dcb63af536429cdb88ab190e01ef8242c (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-3-activate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-16T00:32:15.776 INFO:journalctl@ceph.osd.3.smithi169.stdout:Sep 16 00:32:15 smithi169 podman[86424]: 2024-09-16 00:32:15.58734457 +0000 UTC m=+0.015809193 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:32:15.777 INFO:journalctl@ceph.osd.3.smithi169.stdout:Sep 16 00:32:15 smithi169 podman[86424]: 2024-09-16 00:32:15.713484005 +0000 UTC m=+0.141948654 container create ee07b557781030b3c0039b44dd7fa8ff42e95f779334231057459f8a4fd594f6 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-16T00:32:16.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:15 smithi112 ceph-mon[97750]: pgmap v109: 65 pgs: 18 active+undersized, 20 active+undersized+degraded, 27 active+clean; 325 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 78/372 objects degraded (20.968%) 2024-09-16T00:32:16.276 INFO:journalctl@ceph.osd.3.smithi169.stdout:Sep 16 00:32:15 smithi169 podman[86424]: 2024-09-16 00:32:15.942299057 +0000 UTC m=+0.370763667 container init ee07b557781030b3c0039b44dd7fa8ff42e95f779334231057459f8a4fd594f6 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:32:16.276 INFO:journalctl@ceph.osd.3.smithi169.stdout:Sep 16 00:32:15 smithi169 podman[86424]: 2024-09-16 00:32:15.945576053 +0000 UTC m=+0.374040673 container start ee07b557781030b3c0039b44dd7fa8ff42e95f779334231057459f8a4fd594f6 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:32:16.276 INFO:journalctl@ceph.osd.3.smithi169.stdout:Sep 16 00:32:15 smithi169 ceph-osd[86444]: -- 172.21.15.169:0/3995846358 <== mon.1 v2:172.21.15.169:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563d183ea1e0 con 0x563d17597c00 2024-09-16T00:32:16.276 INFO:journalctl@ceph.osd.3.smithi169.stdout:Sep 16 00:32:16 smithi169 bash[86424]: ee07b557781030b3c0039b44dd7fa8ff42e95f779334231057459f8a4fd594f6 2024-09-16T00:32:16.277 INFO:journalctl@ceph.osd.3.smithi169.stdout:Sep 16 00:32:16 smithi169 systemd[1]: Started Ceph osd.3 for d346dd56-73c0-11ef-bceb-c7b262605968. 2024-09-16T00:32:17.467 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:17 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:32:17.467 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:17 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:32:17.467 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:17 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:32:17.467 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:17 smithi169 ceph-mon[77626]: pgmap v110: 65 pgs: 18 active+undersized, 20 active+undersized+degraded, 27 active+clean; 325 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 78/372 objects degraded (20.968%) 2024-09-16T00:32:17.467 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:17 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:32:17.467 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:17 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:32:17.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:17 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:32:17.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:17 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:32:17.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:17 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:32:17.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:17 smithi112 ceph-mon[97750]: pgmap v110: 65 pgs: 18 active+undersized, 20 active+undersized+degraded, 27 active+clean; 325 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 78/372 objects degraded (20.968%) 2024-09-16T00:32:17.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:17 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:32:17.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:17 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:32:18.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.105+0000 7fdc5066e640 1 -- 172.21.15.112:0/2278310553 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc480fdf20 msgr2=0x7fdc48106560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:18.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.105+0000 7fdc5066e640 1 --2- 172.21.15.112:0/2278310553 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc480fdf20 0x7fdc48106560 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdc380048d0 tx=0x7fdc38009da0 comp rx=0 tx=0).stop 2024-09-16T00:32:18.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.105+0000 7fdc5066e640 1 -- 172.21.15.112:0/2278310553 shutdown_connections 2024-09-16T00:32:18.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.105+0000 7fdc5066e640 1 --2- 172.21.15.112:0/2278310553 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc480fdf20 0x7fdc48106560 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:18.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.105+0000 7fdc5066e640 1 --2- 172.21.15.112:0/2278310553 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc480fd580 0x7fdc480fd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:18.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.105+0000 7fdc5066e640 1 -- 172.21.15.112:0/2278310553 >> 172.21.15.112:0/2278310553 conn(0x7fdc480f7010 msgr2=0x7fdc480f9420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:18.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.105+0000 7fdc5066e640 1 -- 172.21.15.112:0/2278310553 shutdown_connections 2024-09-16T00:32:18.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.105+0000 7fdc5066e640 1 -- 172.21.15.112:0/2278310553 wait complete. 2024-09-16T00:32:18.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.105+0000 7fdc5066e640 1 Processor -- start 2024-09-16T00:32:18.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.105+0000 7fdc5066e640 1 -- start start 2024-09-16T00:32:18.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.105+0000 7fdc5066e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc480fd580 0x7fdc48113290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:18.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.106+0000 7fdc5066e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc480fdf20 0x7fdc481137d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:18.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.106+0000 7fdc5066e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc48113dd0 con 0x7fdc480fdf20 2024-09-16T00:32:18.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.106+0000 7fdc5066e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4810d210 con 0x7fdc480fd580 2024-09-16T00:32:18.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.106+0000 7fdc4dc09640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc480fdf20 0x7fdc481137d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:18.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.106+0000 7fdc4e40a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc480fd580 0x7fdc48113290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:18.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.106+0000 7fdc4dc09640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc480fdf20 0x7fdc481137d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38734/0 (socket says 172.21.15.112:38734) 2024-09-16T00:32:18.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.106+0000 7fdc4dc09640 1 -- 172.21.15.112:0/2417700439 learned_addr learned my addr 172.21.15.112:0/2417700439 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:32:18.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.106+0000 7fdc4dc09640 1 -- 172.21.15.112:0/2417700439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc480fd580 msgr2=0x7fdc48113290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:18.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.106+0000 7fdc4dc09640 1 --2- 172.21.15.112:0/2417700439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc480fd580 0x7fdc48113290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:18.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.106+0000 7fdc4dc09640 1 -- 172.21.15.112:0/2417700439 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc38005040 con 0x7fdc480fdf20 2024-09-16T00:32:18.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.106+0000 7fdc4dc09640 1 --2- 172.21.15.112:0/2417700439 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc480fdf20 0x7fdc481137d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdc38000c00 tx=0x7fdc38009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:18.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.106+0000 7fdc3effd640 1 -- 172.21.15.112:0/2417700439 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc38038070 con 0x7fdc480fdf20 2024-09-16T00:32:18.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.106+0000 7fdc5066e640 1 -- 172.21.15.112:0/2417700439 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc4810d490 con 0x7fdc480fdf20 2024-09-16T00:32:18.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.106+0000 7fdc3effd640 1 -- 172.21.15.112:0/2417700439 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fdc3802ade0 con 0x7fdc480fdf20 2024-09-16T00:32:18.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.106+0000 7fdc3effd640 1 -- 172.21.15.112:0/2417700439 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc3802c7b0 con 0x7fdc480fdf20 2024-09-16T00:32:18.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.107+0000 7fdc5066e640 1 -- 172.21.15.112:0/2417700439 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc4810d950 con 0x7fdc480fdf20 2024-09-16T00:32:18.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.108+0000 7fdc5066e640 1 -- 172.21.15.112:0/2417700439 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc480663b0 con 0x7fdc480fdf20 2024-09-16T00:32:18.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.108+0000 7fdc3effd640 1 -- 172.21.15.112:0/2417700439 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fdc38030020 con 0x7fdc480fdf20 2024-09-16T00:32:18.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.109+0000 7fdc3effd640 1 --2- 172.21.15.112:0/2417700439 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fdc3406e530 0x7fdc340709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:18.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.109+0000 7fdc3effd640 1 -- 172.21.15.112:0/2417700439 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fdc380ae490 con 0x7fdc480fdf20 2024-09-16T00:32:18.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.109+0000 7fdc4e40a640 1 --2- 172.21.15.112:0/2417700439 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fdc3406e530 0x7fdc340709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:18.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.109+0000 7fdc4e40a640 1 --2- 172.21.15.112:0/2417700439 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fdc3406e530 0x7fdc340709e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fdc44009af0 tx=0x7fdc4400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:18.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.113+0000 7fdc3effd640 1 -- 172.21.15.112:0/2417700439 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fdc38077650 con 0x7fdc480fdf20 2024-09-16T00:32:18.203 INFO:journalctl@ceph.osd.3.smithi169.stdout:Sep 16 00:32:17 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-3[86440]: 2024-09-16T00:32:17.833+0000 7fbd2f541640 -1 Falling back to public interface 2024-09-16T00:32:18.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.296+0000 7fdc5066e640 1 -- 172.21.15.112:0/2417700439 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdc4810e9a0 con 0x7fdc3406e530 2024-09-16T00:32:18.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.297+0000 7fdc3effd640 1 -- 172.21.15.112:0/2417700439 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fdc4810e9a0 con 0x7fdc3406e530 2024-09-16T00:32:18.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.299+0000 7fdc5066e640 1 -- 172.21.15.112:0/2417700439 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fdc3406e530 msgr2=0x7fdc340709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:18.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.299+0000 7fdc5066e640 1 --2- 172.21.15.112:0/2417700439 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fdc3406e530 0x7fdc340709e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fdc44009af0 tx=0x7fdc4400b040 comp rx=0 tx=0).stop 2024-09-16T00:32:18.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.299+0000 7fdc5066e640 1 -- 172.21.15.112:0/2417700439 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc480fdf20 msgr2=0x7fdc481137d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:18.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.299+0000 7fdc5066e640 1 --2- 172.21.15.112:0/2417700439 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc480fdf20 0x7fdc481137d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdc38000c00 tx=0x7fdc38009da0 comp rx=0 tx=0).stop 2024-09-16T00:32:18.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.299+0000 7fdc5066e640 1 -- 172.21.15.112:0/2417700439 shutdown_connections 2024-09-16T00:32:18.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.299+0000 7fdc5066e640 1 --2- 172.21.15.112:0/2417700439 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fdc3406e530 0x7fdc340709e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:18.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.299+0000 7fdc5066e640 1 --2- 172.21.15.112:0/2417700439 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc480fdf20 0x7fdc481137d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:18.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.299+0000 7fdc5066e640 1 --2- 172.21.15.112:0/2417700439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc480fd580 0x7fdc48113290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:18.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.299+0000 7fdc5066e640 1 -- 172.21.15.112:0/2417700439 >> 172.21.15.112:0/2417700439 conn(0x7fdc480f7010 msgr2=0x7fdc480f8540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:18.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.300+0000 7fdc5066e640 1 -- 172.21.15.112:0/2417700439 shutdown_connections 2024-09-16T00:32:18.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.300+0000 7fdc5066e640 1 -- 172.21.15.112:0/2417700439 wait complete. 2024-09-16T00:32:18.309 INFO:teuthology.orchestra.run.smithi112.stdout:true 2024-09-16T00:32:18.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.379+0000 7f5c015e8640 1 -- 172.21.15.112:0/2287158481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bfc1014e0 msgr2=0x7f5bfc1018b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:18.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.379+0000 7f5c015e8640 1 --2- 172.21.15.112:0/2287158481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bfc1014e0 0x7f5bfc1018b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5bec0048d0 tx=0x7f5bec009da0 comp rx=0 tx=0).stop 2024-09-16T00:32:18.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.379+0000 7f5c015e8640 1 -- 172.21.15.112:0/2287158481 shutdown_connections 2024-09-16T00:32:18.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.379+0000 7f5c015e8640 1 --2- 172.21.15.112:0/2287158481 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5bfc0fb4a0 0x7f5bfc0fb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:18.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.379+0000 7f5c015e8640 1 --2- 172.21.15.112:0/2287158481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bfc1014e0 0x7f5bfc1018b0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:18.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.379+0000 7f5c015e8640 1 -- 172.21.15.112:0/2287158481 >> 172.21.15.112:0/2287158481 conn(0x7f5bfc0f7060 msgr2=0x7f5bfc0f9470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:18.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.379+0000 7f5c015e8640 1 -- 172.21.15.112:0/2287158481 shutdown_connections 2024-09-16T00:32:18.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.379+0000 7f5c015e8640 1 -- 172.21.15.112:0/2287158481 wait complete. 2024-09-16T00:32:18.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.380+0000 7f5c015e8640 1 Processor -- start 2024-09-16T00:32:18.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.380+0000 7f5c015e8640 1 -- start start 2024-09-16T00:32:18.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.380+0000 7f5c015e8640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5bfc0fb4a0 0x7f5bfc10e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:18.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.380+0000 7f5c015e8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bfc1014e0 0x7f5bfc10e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:18.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.380+0000 7f5c015e8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bfc10ed20 con 0x7f5bfc1014e0 2024-09-16T00:32:18.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.380+0000 7f5c015e8640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bfc10ee90 con 0x7f5bfc0fb4a0 2024-09-16T00:32:18.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.380+0000 7f5bf27fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bfc1014e0 0x7f5bfc10e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:18.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.380+0000 7f5bf27fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bfc1014e0 0x7f5bfc10e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38748/0 (socket says 172.21.15.112:38748) 2024-09-16T00:32:18.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.380+0000 7f5bf27fc640 1 -- 172.21.15.112:0/3035416282 learned_addr learned my addr 172.21.15.112:0/3035416282 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:32:18.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.380+0000 7f5bfaffd640 1 --2- 172.21.15.112:0/3035416282 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5bfc0fb4a0 0x7f5bfc10e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:18.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.381+0000 7f5bf27fc640 1 -- 172.21.15.112:0/3035416282 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5bfc0fb4a0 msgr2=0x7f5bfc10e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:18.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.381+0000 7f5bf27fc640 1 --2- 172.21.15.112:0/3035416282 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5bfc0fb4a0 0x7f5bfc10e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:18.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.381+0000 7f5bf27fc640 1 -- 172.21.15.112:0/3035416282 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bec005040 con 0x7f5bfc1014e0 2024-09-16T00:32:18.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.381+0000 7f5bf27fc640 1 --2- 172.21.15.112:0/3035416282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bfc1014e0 0x7f5bfc10e7e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5be400eb70 tx=0x7f5be400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:18.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.381+0000 7f5bf3fff640 1 -- 172.21.15.112:0/3035416282 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5be4007c30 con 0x7f5bfc1014e0 2024-09-16T00:32:18.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.381+0000 7f5bf3fff640 1 -- 172.21.15.112:0/3035416282 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f5be400f040 con 0x7f5bfc1014e0 2024-09-16T00:32:18.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.381+0000 7f5c015e8640 1 -- 172.21.15.112:0/3035416282 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bfc117170 con 0x7f5bfc1014e0 2024-09-16T00:32:18.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.381+0000 7f5bf3fff640 1 -- 172.21.15.112:0/3035416282 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5be40094e0 con 0x7f5bfc1014e0 2024-09-16T00:32:18.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.381+0000 7f5c015e8640 1 -- 172.21.15.112:0/3035416282 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bfc117680 con 0x7f5bfc1014e0 2024-09-16T00:32:18.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.382+0000 7f5c015e8640 1 -- 172.21.15.112:0/3035416282 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bfc0fcbe0 con 0x7f5bfc1014e0 2024-09-16T00:32:18.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.383+0000 7f5bf3fff640 1 -- 172.21.15.112:0/3035416282 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f5be4009750 con 0x7f5bfc1014e0 2024-09-16T00:32:18.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.384+0000 7f5bf3fff640 1 --2- 172.21.15.112:0/3035416282 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5be806e530 0x7f5be80709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:18.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.384+0000 7f5bf3fff640 1 -- 172.21.15.112:0/3035416282 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f5be408dd90 con 0x7f5bfc1014e0 2024-09-16T00:32:18.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.384+0000 7f5bfaffd640 1 --2- 172.21.15.112:0/3035416282 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5be806e530 0x7f5be80709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:18.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.384+0000 7f5bfaffd640 1 --2- 172.21.15.112:0/3035416282 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5be806e530 0x7f5be80709e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5bec004540 tx=0x7f5bec035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:18.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.388+0000 7f5bf3fff640 1 -- 172.21.15.112:0/3035416282 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f5be4056f80 con 0x7f5bfc1014e0 2024-09-16T00:32:18.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.572+0000 7f5c015e8640 1 -- 172.21.15.112:0/3035416282 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5bfc108500 con 0x7f5be806e530 2024-09-16T00:32:18.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.573+0000 7f5bf3fff640 1 -- 172.21.15.112:0/3035416282 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f5bec039040 con 0x7f5be806e530 2024-09-16T00:32:18.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.575+0000 7f5c015e8640 1 -- 172.21.15.112:0/3035416282 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5be806e530 msgr2=0x7f5be80709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:18.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.575+0000 7f5c015e8640 1 --2- 172.21.15.112:0/3035416282 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5be806e530 0x7f5be80709e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5bec004540 tx=0x7f5bec035040 comp rx=0 tx=0).stop 2024-09-16T00:32:18.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.575+0000 7f5c015e8640 1 -- 172.21.15.112:0/3035416282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bfc1014e0 msgr2=0x7f5bfc10e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:18.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.575+0000 7f5c015e8640 1 --2- 172.21.15.112:0/3035416282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bfc1014e0 0x7f5bfc10e7e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5be400eb70 tx=0x7f5be400b040 comp rx=0 tx=0).stop 2024-09-16T00:32:18.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.575+0000 7f5c015e8640 1 -- 172.21.15.112:0/3035416282 shutdown_connections 2024-09-16T00:32:18.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.575+0000 7f5c015e8640 1 --2- 172.21.15.112:0/3035416282 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5be806e530 0x7f5be80709e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:18.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.575+0000 7f5c015e8640 1 --2- 172.21.15.112:0/3035416282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bfc1014e0 0x7f5bfc10e7e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:18.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.575+0000 7f5c015e8640 1 --2- 172.21.15.112:0/3035416282 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5bfc0fb4a0 0x7f5bfc10e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:18.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.575+0000 7f5c015e8640 1 -- 172.21.15.112:0/3035416282 >> 172.21.15.112:0/3035416282 conn(0x7f5bfc0f7060 msgr2=0x7f5bfc0f7430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:18.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.575+0000 7f5c015e8640 1 -- 172.21.15.112:0/3035416282 shutdown_connections 2024-09-16T00:32:18.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.575+0000 7f5c015e8640 1 -- 172.21.15.112:0/3035416282 wait complete. 2024-09-16T00:32:18.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.654+0000 7faf8362a640 1 -- 172.21.15.112:0/3024475979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf7c0fbd00 msgr2=0x7faf7c100320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:18.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.654+0000 7faf8362a640 1 --2- 172.21.15.112:0/3024475979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf7c0fbd00 0x7faf7c100320 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7faf6c0048d0 tx=0x7faf6c009da0 comp rx=0 tx=0).stop 2024-09-16T00:32:18.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.654+0000 7faf8362a640 1 -- 172.21.15.112:0/3024475979 shutdown_connections 2024-09-16T00:32:18.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.654+0000 7faf8362a640 1 --2- 172.21.15.112:0/3024475979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf7c0fbd00 0x7faf7c100320 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:18.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.654+0000 7faf8362a640 1 --2- 172.21.15.112:0/3024475979 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf7c0fb360 0x7faf7c0fb730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:18.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.654+0000 7faf8362a640 1 -- 172.21.15.112:0/3024475979 >> 172.21.15.112:0/3024475979 conn(0x7faf7c0f7000 msgr2=0x7faf7c0f9410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:18.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.654+0000 7faf8362a640 1 -- 172.21.15.112:0/3024475979 shutdown_connections 2024-09-16T00:32:18.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.654+0000 7faf8362a640 1 -- 172.21.15.112:0/3024475979 wait complete. 2024-09-16T00:32:18.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.655+0000 7faf8362a640 1 Processor -- start 2024-09-16T00:32:18.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.655+0000 7faf8362a640 1 -- start start 2024-09-16T00:32:18.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.655+0000 7faf8362a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf7c0fb360 0x7faf7c119850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:18.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.655+0000 7faf8362a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf7c0fbd00 0x7faf7c119d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:18.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.655+0000 7faf8362a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf7c11a3e0 con 0x7faf7c0fb360 2024-09-16T00:32:18.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.655+0000 7faf8362a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf7c1137f0 con 0x7faf7c0fbd00 2024-09-16T00:32:18.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.655+0000 7faf813c6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf7c0fb360 0x7faf7c119850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:18.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.655+0000 7faf813c6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf7c0fb360 0x7faf7c119850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38760/0 (socket says 172.21.15.112:38760) 2024-09-16T00:32:18.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.655+0000 7faf813c6640 1 -- 172.21.15.112:0/583632392 learned_addr learned my addr 172.21.15.112:0/583632392 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:32:18.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.655+0000 7faf80bc5640 1 --2- 172.21.15.112:0/583632392 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf7c0fbd00 0x7faf7c119d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:18.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.655+0000 7faf813c6640 1 -- 172.21.15.112:0/583632392 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf7c0fbd00 msgr2=0x7faf7c119d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:18.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.655+0000 7faf813c6640 1 --2- 172.21.15.112:0/583632392 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf7c0fbd00 0x7faf7c119d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:18.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.656+0000 7faf813c6640 1 -- 172.21.15.112:0/583632392 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf6c005040 con 0x7faf7c0fb360 2024-09-16T00:32:18.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.656+0000 7faf813c6640 1 --2- 172.21.15.112:0/583632392 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf7c0fb360 0x7faf7c119850 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7faf78002a50 tx=0x7faf78004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:18.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.656+0000 7faf71ffb640 1 -- 172.21.15.112:0/583632392 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf78007c30 con 0x7faf7c0fb360 2024-09-16T00:32:18.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.656+0000 7faf8362a640 1 -- 172.21.15.112:0/583632392 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf7c113ad0 con 0x7faf7c0fb360 2024-09-16T00:32:18.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.656+0000 7faf71ffb640 1 -- 172.21.15.112:0/583632392 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7faf7800f040 con 0x7faf7c0fb360 2024-09-16T00:32:18.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.656+0000 7faf71ffb640 1 -- 172.21.15.112:0/583632392 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf7800da10 con 0x7faf7c0fb360 2024-09-16T00:32:18.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.656+0000 7faf8362a640 1 -- 172.21.15.112:0/583632392 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf7c113fc0 con 0x7faf7c0fb360 2024-09-16T00:32:18.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.657+0000 7faf8362a640 1 -- 172.21.15.112:0/583632392 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf7c066330 con 0x7faf7c0fb360 2024-09-16T00:32:18.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.658+0000 7faf71ffb640 1 -- 172.21.15.112:0/583632392 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7faf7800dc50 con 0x7faf7c0fb360 2024-09-16T00:32:18.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.658+0000 7faf71ffb640 1 --2- 172.21.15.112:0/583632392 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7faf6806e530 0x7faf680709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:18.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.658+0000 7faf71ffb640 1 -- 172.21.15.112:0/583632392 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6410+0+0 (secure 0 0 0) 0x7faf7808d860 con 0x7faf7c0fb360 2024-09-16T00:32:18.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.658+0000 7faf80bc5640 1 --2- 172.21.15.112:0/583632392 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7faf6806e530 0x7faf680709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:18.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.659+0000 7faf80bc5640 1 --2- 172.21.15.112:0/583632392 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7faf6806e530 0x7faf680709e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7faf6c009ac0 tx=0x7faf6c035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:18.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.663+0000 7faf71ffb640 1 -- 172.21.15.112:0/583632392 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7faf78056ad0 con 0x7faf7c0fb360 2024-09-16T00:32:18.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.843+0000 7faf8362a640 1 -- 172.21.15.112:0/583632392 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7faf7c10fbb0 con 0x7faf6806e530 2024-09-16T00:32:18.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.852+0000 7faf71ffb640 1 -- 172.21.15.112:0/583632392 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7faf7c10fbb0 con 0x7faf6806e530 2024-09-16T00:32:18.853 INFO:teuthology.orchestra.run.smithi112.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T00:32:18.853 INFO:teuthology.orchestra.run.smithi112.stdout:alertmanager.smithi112 smithi112 *:9093,9094 running (5m) 32s ago 14m 25.2M - 0.25.0 c8568f914cd2 d85ea0562022 2024-09-16T00:32:18.853 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi112 smithi112 running (2m) 32s ago 14m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c e4fe58f9702a 2024-09-16T00:32:18.853 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi169 smithi169 running (2m) 0s ago 13m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c 3ba2c81d527f 2024-09-16T00:32:18.853 INFO:teuthology.orchestra.run.smithi112.stdout:grafana.smithi112 smithi112 *:3000 running (4m) 32s ago 13m 82.2M - 9.4.12 2bacad6d85d8 51f9d2013da2 2024-09-16T00:32:18.853 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.hxfwkh smithi112 running (8m) 32s ago 8m 135M - 17.2.7-1633-g6d025bc2 33ed8dba3996 cf92d9461fc6 2024-09-16T00:32:18.853 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.ithwzj smithi112 running (8m) 32s ago 8m 66.1M - 17.2.7-1633-g6d025bc2 33ed8dba3996 02786c8a83c7 2024-09-16T00:32:18.853 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.gawwgw smithi169 running (8m) 0s ago 8m 114M - 17.2.7-1633-g6d025bc2 33ed8dba3996 dcf3a4dfa879 2024-09-16T00:32:18.854 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.lgeqrj smithi169 running (8m) 0s ago 8m 60.4M - 17.2.7-1633-g6d025bc2 33ed8dba3996 445c14e55831 2024-09-16T00:32:18.854 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi112.xrkkbc smithi112 *:8443,9283,8765 running (4m) 32s ago 15m 497M - 19.1.1-326-g8921c8ef f1455bd0219c a981f7fd2810 2024-09-16T00:32:18.854 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi169.mtrobk smithi169 *:8443,9283,8765 running (4m) 0s ago 13m 441M - 19.1.1-326-g8921c8ef f1455bd0219c 4096e5369c2d 2024-09-16T00:32:18.854 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi112 smithi112 running (3m) 32s ago 15m 46.4M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 5aaec033c042 2024-09-16T00:32:18.854 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi169 smithi169 running (2m) 0s ago 13m 37.0M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 7bc609155f3f 2024-09-16T00:32:18.854 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi112 smithi112 *:9100 running (5m) 32s ago 14m 15.5M - 1.5.0 0da6a335fe13 6404d1d601e2 2024-09-16T00:32:18.854 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi169 smithi169 *:9100 running (4m) 0s ago 13m 15.0M - 1.5.0 0da6a335fe13 5505baf702a6 2024-09-16T00:32:18.854 INFO:teuthology.orchestra.run.smithi112.stdout:osd.0 smithi112 running (107s) 32s ago 12m 89.4M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 53efc9397017 2024-09-16T00:32:18.854 INFO:teuthology.orchestra.run.smithi112.stdout:osd.1 smithi112 running (73s) 32s ago 12m 89.1M 1557M 19.1.1-326-g8921c8ef f1455bd0219c fe6f1a4a2cf4 2024-09-16T00:32:18.854 INFO:teuthology.orchestra.run.smithi112.stdout:osd.2 smithi112 running (38s) 32s ago 11m 12.1M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 5f7863712057 2024-09-16T00:32:18.854 INFO:teuthology.orchestra.run.smithi112.stdout:osd.3 smithi169 running (3s) 0s ago 11m 12.1M 2581M 19.1.1-326-g8921c8ef f1455bd0219c ee07b5577810 2024-09-16T00:32:18.854 INFO:teuthology.orchestra.run.smithi112.stdout:osd.4 smithi169 running (10m) 0s ago 10m 213M 2581M 17.2.7-1633-g6d025bc2 33ed8dba3996 d3e4cdb016b9 2024-09-16T00:32:18.855 INFO:teuthology.orchestra.run.smithi112.stdout:osd.5 smithi169 running (10m) 0s ago 10m 198M 2581M 17.2.7-1633-g6d025bc2 33ed8dba3996 8e7d708d74eb 2024-09-16T00:32:18.855 INFO:teuthology.orchestra.run.smithi112.stdout:prometheus.smithi112 smithi112 *:9095 running (4m) 32s ago 13m 63.2M - 2.43.0 a07b618ecd1d 7a1d42e60d2b 2024-09-16T00:32:18.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.854+0000 7faf8362a640 1 -- 172.21.15.112:0/583632392 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7faf6806e530 msgr2=0x7faf680709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:18.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.855+0000 7faf8362a640 1 --2- 172.21.15.112:0/583632392 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7faf6806e530 0x7faf680709e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7faf6c009ac0 tx=0x7faf6c035040 comp rx=0 tx=0).stop 2024-09-16T00:32:18.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.855+0000 7faf8362a640 1 -- 172.21.15.112:0/583632392 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf7c0fb360 msgr2=0x7faf7c119850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:18.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.855+0000 7faf8362a640 1 --2- 172.21.15.112:0/583632392 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf7c0fb360 0x7faf7c119850 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7faf78002a50 tx=0x7faf78004600 comp rx=0 tx=0).stop 2024-09-16T00:32:18.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.855+0000 7faf8362a640 1 -- 172.21.15.112:0/583632392 shutdown_connections 2024-09-16T00:32:18.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.855+0000 7faf8362a640 1 --2- 172.21.15.112:0/583632392 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7faf6806e530 0x7faf680709e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:18.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.855+0000 7faf8362a640 1 --2- 172.21.15.112:0/583632392 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf7c0fbd00 0x7faf7c119d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:18.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.855+0000 7faf8362a640 1 --2- 172.21.15.112:0/583632392 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf7c0fb360 0x7faf7c119850 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:18.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.855+0000 7faf8362a640 1 -- 172.21.15.112:0/583632392 >> 172.21.15.112:0/583632392 conn(0x7faf7c0f7000 msgr2=0x7faf7c0f8690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:18.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.855+0000 7faf8362a640 1 -- 172.21.15.112:0/583632392 shutdown_connections 2024-09-16T00:32:18.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.855+0000 7faf8362a640 1 -- 172.21.15.112:0/583632392 wait complete. 2024-09-16T00:32:18.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.932+0000 7f8c0baf7640 1 -- 172.21.15.112:0/1370496220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c04102ef0 msgr2=0x7f8c040fc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:18.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.932+0000 7f8c0baf7640 1 --2- 172.21.15.112:0/1370496220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c04102ef0 0x7f8c040fc5b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8bf80048d0 tx=0x7f8bf8009da0 comp rx=0 tx=0).stop 2024-09-16T00:32:18.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.932+0000 7f8c0baf7640 1 -- 172.21.15.112:0/1370496220 shutdown_connections 2024-09-16T00:32:18.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.932+0000 7f8c0baf7640 1 --2- 172.21.15.112:0/1370496220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c04102ef0 0x7f8c040fc5b0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:18.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.932+0000 7f8c0baf7640 1 --2- 172.21.15.112:0/1370496220 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c04102550 0x7f8c04102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:18.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.932+0000 7f8c0baf7640 1 -- 172.21.15.112:0/1370496220 >> 172.21.15.112:0/1370496220 conn(0x7f8c040f80f0 msgr2=0x7f8c040fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:18.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.932+0000 7f8c0baf7640 1 -- 172.21.15.112:0/1370496220 shutdown_connections 2024-09-16T00:32:18.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.932+0000 7f8c0baf7640 1 -- 172.21.15.112:0/1370496220 wait complete. 2024-09-16T00:32:18.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.933+0000 7f8c0baf7640 1 Processor -- start 2024-09-16T00:32:18.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.933+0000 7f8c0baf7640 1 -- start start 2024-09-16T00:32:18.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.933+0000 7f8c0baf7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c04102550 0x7f8c04113500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:18.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.933+0000 7f8c0baf7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c04102ef0 0x7f8c04113a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:18.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.933+0000 7f8c0baf7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c04114090 con 0x7f8c04102550 2024-09-16T00:32:18.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.933+0000 7f8c0baf7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0410d4a0 con 0x7f8c04102ef0 2024-09-16T00:32:18.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.933+0000 7f8c09893640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c04102550 0x7f8c04113500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:18.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.933+0000 7f8c09893640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c04102550 0x7f8c04113500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38774/0 (socket says 172.21.15.112:38774) 2024-09-16T00:32:18.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.933+0000 7f8c09893640 1 -- 172.21.15.112:0/3416513098 learned_addr learned my addr 172.21.15.112:0/3416513098 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:32:18.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.933+0000 7f8c09092640 1 --2- 172.21.15.112:0/3416513098 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c04102ef0 0x7f8c04113a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:18.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.933+0000 7f8c09893640 1 -- 172.21.15.112:0/3416513098 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c04102ef0 msgr2=0x7f8c04113a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:18.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.933+0000 7f8c09893640 1 --2- 172.21.15.112:0/3416513098 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c04102ef0 0x7f8c04113a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:18.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.933+0000 7f8c09893640 1 -- 172.21.15.112:0/3416513098 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf8005040 con 0x7f8c04102550 2024-09-16T00:32:18.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.934+0000 7f8c09893640 1 --2- 172.21.15.112:0/3416513098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c04102550 0x7f8c04113500 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc002a50 tx=0x7f8bfc004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:18.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.934+0000 7f8c027fc640 1 -- 172.21.15.112:0/3416513098 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8bfc007c30 con 0x7f8c04102550 2024-09-16T00:32:18.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.934+0000 7f8c027fc640 1 -- 172.21.15.112:0/3416513098 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f8bfc00f040 con 0x7f8c04102550 2024-09-16T00:32:18.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.934+0000 7f8c027fc640 1 -- 172.21.15.112:0/3416513098 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8bfc00dad0 con 0x7f8c04102550 2024-09-16T00:32:18.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.934+0000 7f8c0baf7640 1 -- 172.21.15.112:0/3416513098 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c0410d780 con 0x7f8c04102550 2024-09-16T00:32:18.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.934+0000 7f8c0baf7640 1 -- 172.21.15.112:0/3416513098 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c0410dcf0 con 0x7f8c04102550 2024-09-16T00:32:18.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.935+0000 7f8c0baf7640 1 -- 172.21.15.112:0/3416513098 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c040fdcd0 con 0x7f8c04102550 2024-09-16T00:32:18.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.935+0000 7f8c027fc640 1 -- 172.21.15.112:0/3416513098 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f8bfc00dc30 con 0x7f8c04102550 2024-09-16T00:32:18.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.936+0000 7f8c027fc640 1 --2- 172.21.15.112:0/3416513098 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f8bec06e530 0x7f8bec0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:18.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.936+0000 7f8c027fc640 1 -- 172.21.15.112:0/3416513098 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f8bfc08dee0 con 0x7f8c04102550 2024-09-16T00:32:18.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.936+0000 7f8c09092640 1 --2- 172.21.15.112:0/3416513098 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f8bec06e530 0x7f8bec0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:18.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.937+0000 7f8c09092640 1 --2- 172.21.15.112:0/3416513098 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f8bec06e530 0x7f8bec0709e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8bf80048a0 tx=0x7f8bf8035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:18.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:18.940+0000 7f8c027fc640 1 -- 172.21.15.112:0/3416513098 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f8bfc0570a0 con 0x7f8c04102550 2024-09-16T00:32:19.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.177+0000 7f8c0baf7640 1 -- 172.21.15.112:0/3416513098 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f8c0404b720 con 0x7f8c04102550 2024-09-16T00:32:19.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.177+0000 7f8c027fc640 1 -- 172.21.15.112:0/3416513098 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+850 (secure 0 0 0) 0x7f8bfc05a760 con 0x7f8c04102550 2024-09-16T00:32:19.178 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:32:19.178 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": { 2024-09-16T00:32:19.178 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:32:19.178 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:32:19.178 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": { 2024-09-16T00:32:19.178 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:32:19.178 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:32:19.178 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": { 2024-09-16T00:32:19.178 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 17.2.7-1633-g6d025bc2 (6d025bc26f3176c910707f8cb4c8d8e076640615) quincy (stable)": 2, 2024-09-16T00:32:19.178 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 3 2024-09-16T00:32:19.178 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:32:19.178 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": { 2024-09-16T00:32:19.179 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 17.2.7-1633-g6d025bc2 (6d025bc26f3176c910707f8cb4c8d8e076640615) quincy (stable)": 4 2024-09-16T00:32:19.179 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:32:19.179 INFO:teuthology.orchestra.run.smithi112.stdout: "overall": { 2024-09-16T00:32:19.179 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 17.2.7-1633-g6d025bc2 (6d025bc26f3176c910707f8cb4c8d8e076640615) quincy (stable)": 6, 2024-09-16T00:32:19.179 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 7 2024-09-16T00:32:19.179 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-16T00:32:19.179 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:32:19.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.179+0000 7f8c0baf7640 1 -- 172.21.15.112:0/3416513098 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f8bec06e530 msgr2=0x7f8bec0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:19.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.179+0000 7f8c0baf7640 1 --2- 172.21.15.112:0/3416513098 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f8bec06e530 0x7f8bec0709e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8bf80048a0 tx=0x7f8bf8035040 comp rx=0 tx=0).stop 2024-09-16T00:32:19.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.179+0000 7f8c0baf7640 1 -- 172.21.15.112:0/3416513098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c04102550 msgr2=0x7f8c04113500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:19.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.179+0000 7f8c0baf7640 1 --2- 172.21.15.112:0/3416513098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c04102550 0x7f8c04113500 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc002a50 tx=0x7f8bfc004600 comp rx=0 tx=0).stop 2024-09-16T00:32:19.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.179+0000 7f8c0baf7640 1 -- 172.21.15.112:0/3416513098 shutdown_connections 2024-09-16T00:32:19.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.179+0000 7f8c0baf7640 1 --2- 172.21.15.112:0/3416513098 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f8bec06e530 0x7f8bec0709e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:19.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.179+0000 7f8c0baf7640 1 --2- 172.21.15.112:0/3416513098 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c04102ef0 0x7f8c04113a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:19.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.179+0000 7f8c0baf7640 1 --2- 172.21.15.112:0/3416513098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c04102550 0x7f8c04113500 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:19.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.179+0000 7f8c0baf7640 1 -- 172.21.15.112:0/3416513098 >> 172.21.15.112:0/3416513098 conn(0x7f8c040f80f0 msgr2=0x7f8c040f9150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:19.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.180+0000 7f8c0baf7640 1 -- 172.21.15.112:0/3416513098 shutdown_connections 2024-09-16T00:32:19.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.180+0000 7f8c0baf7640 1 -- 172.21.15.112:0/3416513098 wait complete. 2024-09-16T00:32:19.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.258+0000 7f5c732f2640 1 -- 172.21.15.112:0/2451067677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c6c102f00 msgr2=0x7f5c6c0fc5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:19.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.258+0000 7f5c732f2640 1 --2- 172.21.15.112:0/2451067677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c6c102f00 0x7f5c6c0fc5c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5c680048d0 tx=0x7f5c68009da0 comp rx=0 tx=0).stop 2024-09-16T00:32:19.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.258+0000 7f5c732f2640 1 -- 172.21.15.112:0/2451067677 shutdown_connections 2024-09-16T00:32:19.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.258+0000 7f5c732f2640 1 --2- 172.21.15.112:0/2451067677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c6c102f00 0x7f5c6c0fc5c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:19.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.258+0000 7f5c732f2640 1 --2- 172.21.15.112:0/2451067677 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5c6c102560 0x7f5c6c102930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:19.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.258+0000 7f5c732f2640 1 -- 172.21.15.112:0/2451067677 >> 172.21.15.112:0/2451067677 conn(0x7f5c6c0f8120 msgr2=0x7f5c6c0fa530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:19.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.258+0000 7f5c732f2640 1 -- 172.21.15.112:0/2451067677 shutdown_connections 2024-09-16T00:32:19.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.258+0000 7f5c732f2640 1 -- 172.21.15.112:0/2451067677 wait complete. 2024-09-16T00:32:19.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.258+0000 7f5c732f2640 1 Processor -- start 2024-09-16T00:32:19.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.258+0000 7f5c732f2640 1 -- start start 2024-09-16T00:32:19.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.259+0000 7f5c732f2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c6c102560 0x7f5c6c06fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:19.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.259+0000 7f5c732f2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5c6c102f00 0x7f5c6c06ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:19.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.259+0000 7f5c732f2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6c070490 con 0x7f5c6c102560 2024-09-16T00:32:19.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.259+0000 7f5c732f2640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6c070600 con 0x7f5c6c102f00 2024-09-16T00:32:19.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.259+0000 7f5c7108e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c6c102560 0x7f5c6c06fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:19.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.259+0000 7f5c7108e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c6c102560 0x7f5c6c06fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38782/0 (socket says 172.21.15.112:38782) 2024-09-16T00:32:19.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.259+0000 7f5c7108e640 1 -- 172.21.15.112:0/3295529844 learned_addr learned my addr 172.21.15.112:0/3295529844 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:32:19.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.259+0000 7f5c7088d640 1 --2- 172.21.15.112:0/3295529844 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5c6c102f00 0x7f5c6c06ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:19.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.259+0000 7f5c7108e640 1 -- 172.21.15.112:0/3295529844 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5c6c102f00 msgr2=0x7f5c6c06ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:19.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.259+0000 7f5c7108e640 1 --2- 172.21.15.112:0/3295529844 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5c6c102f00 0x7f5c6c06ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:19.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.259+0000 7f5c7108e640 1 -- 172.21.15.112:0/3295529844 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c68005040 con 0x7f5c6c102560 2024-09-16T00:32:19.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.259+0000 7f5c7108e640 1 --2- 172.21.15.112:0/3295529844 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c6c102560 0x7f5c6c06fa10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5c60002a50 tx=0x7f5c60004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:19.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.260+0000 7f5c5dffb640 1 -- 172.21.15.112:0/3295529844 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c60007c30 con 0x7f5c6c102560 2024-09-16T00:32:19.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.260+0000 7f5c5dffb640 1 -- 172.21.15.112:0/3295529844 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f5c6000f040 con 0x7f5c6c102560 2024-09-16T00:32:19.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.260+0000 7f5c5dffb640 1 -- 172.21.15.112:0/3295529844 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c6000dad0 con 0x7f5c6c102560 2024-09-16T00:32:19.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.260+0000 7f5c732f2640 1 -- 172.21.15.112:0/3295529844 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c6c070890 con 0x7f5c6c102560 2024-09-16T00:32:19.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.260+0000 7f5c732f2640 1 -- 172.21.15.112:0/3295529844 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c6c119900 con 0x7f5c6c102560 2024-09-16T00:32:19.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.261+0000 7f5c732f2640 1 -- 172.21.15.112:0/3295529844 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c6c0fdce0 con 0x7f5c6c102560 2024-09-16T00:32:19.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.261+0000 7f5c5dffb640 1 -- 172.21.15.112:0/3295529844 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f5c6000dc30 con 0x7f5c6c102560 2024-09-16T00:32:19.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.262+0000 7f5c5dffb640 1 --2- 172.21.15.112:0/3295529844 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5c5806e530 0x7f5c580709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:19.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.262+0000 7f5c5dffb640 1 -- 172.21.15.112:0/3295529844 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f5c6008dfc0 con 0x7f5c6c102560 2024-09-16T00:32:19.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.262+0000 7f5c7088d640 1 --2- 172.21.15.112:0/3295529844 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5c5806e530 0x7f5c580709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:19.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.262+0000 7f5c7088d640 1 --2- 172.21.15.112:0/3295529844 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5c5806e530 0x7f5c580709e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5c680048a0 tx=0x7f5c68035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:19.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.266+0000 7f5c5dffb640 1 -- 172.21.15.112:0/3295529844 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f5c60057100 con 0x7f5c6c102560 2024-09-16T00:32:19.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.466+0000 7f5c732f2640 1 -- 172.21.15.112:0/3295529844 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5c6c071100 con 0x7f5c6c102560 2024-09-16T00:32:19.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.467+0000 7f5c5dffb640 1 -- 172.21.15.112:0/3295529844 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) v1 ==== 76+0+1954 (secure 0 0 0) 0x7f5c6005a7c0 con 0x7f5c6c102560 2024-09-16T00:32:19.467 INFO:teuthology.orchestra.run.smithi112.stdout:e23 2024-09-16T00:32:19.467 INFO:teuthology.orchestra.run.smithi112.stdout:btime 2024-09-16T00:30:08:788506+0000 2024-09-16T00:32:19.468 INFO:teuthology.orchestra.run.smithi112.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T00:32:19.468 INFO:teuthology.orchestra.run.smithi112.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:32:19.468 INFO:teuthology.orchestra.run.smithi112.stdout:legacy client fscid: 1 2024-09-16T00:32:19.468 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:32:19.468 INFO:teuthology.orchestra.run.smithi112.stdout:Filesystem 'cephfs' (1) 2024-09-16T00:32:19.468 INFO:teuthology.orchestra.run.smithi112.stdout:fs_name cephfs 2024-09-16T00:32:19.468 INFO:teuthology.orchestra.run.smithi112.stdout:epoch 23 2024-09-16T00:32:19.468 INFO:teuthology.orchestra.run.smithi112.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-16T00:32:19.468 INFO:teuthology.orchestra.run.smithi112.stdout:created 2024-09-16T00:23:22.328199+0000 2024-09-16T00:32:19.468 INFO:teuthology.orchestra.run.smithi112.stdout:modified 2024-09-16T00:30:08.396661+0000 2024-09-16T00:32:19.468 INFO:teuthology.orchestra.run.smithi112.stdout:tableserver 0 2024-09-16T00:32:19.468 INFO:teuthology.orchestra.run.smithi112.stdout:root 0 2024-09-16T00:32:19.468 INFO:teuthology.orchestra.run.smithi112.stdout:session_timeout 60 2024-09-16T00:32:19.469 INFO:teuthology.orchestra.run.smithi112.stdout:session_autoclose 300 2024-09-16T00:32:19.469 INFO:teuthology.orchestra.run.smithi112.stdout:max_file_size 1099511627776 2024-09-16T00:32:19.469 INFO:teuthology.orchestra.run.smithi112.stdout:max_xattr_size 65536 2024-09-16T00:32:19.469 INFO:teuthology.orchestra.run.smithi112.stdout:required_client_features {} 2024-09-16T00:32:19.469 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure 0 2024-09-16T00:32:19.469 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure_osd_epoch 44 2024-09-16T00:32:19.469 INFO:teuthology.orchestra.run.smithi112.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:32:19.469 INFO:teuthology.orchestra.run.smithi112.stdout:max_mds 2 2024-09-16T00:32:19.469 INFO:teuthology.orchestra.run.smithi112.stdout:in 0,1 2024-09-16T00:32:19.469 INFO:teuthology.orchestra.run.smithi112.stdout:up {0=14480,1=24259} 2024-09-16T00:32:19.469 INFO:teuthology.orchestra.run.smithi112.stdout:failed 2024-09-16T00:32:19.469 INFO:teuthology.orchestra.run.smithi112.stdout:damaged 2024-09-16T00:32:19.469 INFO:teuthology.orchestra.run.smithi112.stdout:stopped 2024-09-16T00:32:19.469 INFO:teuthology.orchestra.run.smithi112.stdout:data_pools [3] 2024-09-16T00:32:19.470 INFO:teuthology.orchestra.run.smithi112.stdout:metadata_pool 2 2024-09-16T00:32:19.470 INFO:teuthology.orchestra.run.smithi112.stdout:inline_data enabled 2024-09-16T00:32:19.470 INFO:teuthology.orchestra.run.smithi112.stdout:balancer 2024-09-16T00:32:19.470 INFO:teuthology.orchestra.run.smithi112.stdout:bal_rank_mask -1 2024-09-16T00:32:19.470 INFO:teuthology.orchestra.run.smithi112.stdout:standby_count_wanted 1 2024-09-16T00:32:19.470 INFO:teuthology.orchestra.run.smithi112.stdout:qdb_cluster leader: 14480 members: 24259,14480 2024-09-16T00:32:19.470 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.hxfwkh{0:14480} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.112:6828/486654885,v1:172.21.15.112:6829/486654885] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T00:32:19.470 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.ithwzj{0:14492} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.112:6826/381830565,v1:172.21.15.112:6827/381830565] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T00:32:19.470 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.gawwgw{1:24259} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.169:6824/2892476953,v1:172.21.15.169:6825/2892476953] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T00:32:19.470 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.lgeqrj{1:24269} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.169:6826/861725728,v1:172.21.15.169:6827/861725728] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T00:32:19.470 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:32:19.470 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:32:19.471 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 23 2024-09-16T00:32:19.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.469+0000 7f5c732f2640 1 -- 172.21.15.112:0/3295529844 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5c5806e530 msgr2=0x7f5c580709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:19.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.469+0000 7f5c732f2640 1 --2- 172.21.15.112:0/3295529844 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5c5806e530 0x7f5c580709e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5c680048a0 tx=0x7f5c68035040 comp rx=0 tx=0).stop 2024-09-16T00:32:19.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.469+0000 7f5c732f2640 1 -- 172.21.15.112:0/3295529844 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c6c102560 msgr2=0x7f5c6c06fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:19.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.469+0000 7f5c732f2640 1 --2- 172.21.15.112:0/3295529844 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c6c102560 0x7f5c6c06fa10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5c60002a50 tx=0x7f5c60004600 comp rx=0 tx=0).stop 2024-09-16T00:32:19.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.469+0000 7f5c732f2640 1 -- 172.21.15.112:0/3295529844 shutdown_connections 2024-09-16T00:32:19.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.469+0000 7f5c732f2640 1 --2- 172.21.15.112:0/3295529844 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5c5806e530 0x7f5c580709e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:19.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.469+0000 7f5c732f2640 1 --2- 172.21.15.112:0/3295529844 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5c6c102f00 0x7f5c6c06ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:19.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.469+0000 7f5c732f2640 1 --2- 172.21.15.112:0/3295529844 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c6c102560 0x7f5c6c06fa10 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:19.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.469+0000 7f5c732f2640 1 -- 172.21.15.112:0/3295529844 >> 172.21.15.112:0/3295529844 conn(0x7f5c6c0f8120 msgr2=0x7f5c6c0f90a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:19.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.469+0000 7f5c732f2640 1 -- 172.21.15.112:0/3295529844 shutdown_connections 2024-09-16T00:32:19.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.469+0000 7f5c732f2640 1 -- 172.21.15.112:0/3295529844 wait complete. 2024-09-16T00:32:19.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.548+0000 7fa8b0c8e640 1 -- 172.21.15.112:0/2408185956 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8ac0fb480 msgr2=0x7fa8ac0fb8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:19.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.548+0000 7fa8b0c8e640 1 --2- 172.21.15.112:0/2408185956 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8ac0fb480 0x7fa8ac0fb8f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa8a00048d0 tx=0x7fa8a0009da0 comp rx=0 tx=0).stop 2024-09-16T00:32:19.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.548+0000 7fa8b0c8e640 1 -- 172.21.15.112:0/2408185956 shutdown_connections 2024-09-16T00:32:19.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.548+0000 7fa8b0c8e640 1 --2- 172.21.15.112:0/2408185956 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8ac0fb480 0x7fa8ac0fb8f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:19.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.548+0000 7fa8b0c8e640 1 --2- 172.21.15.112:0/2408185956 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa8ac1014c0 0x7fa8ac101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:19.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.548+0000 7fa8b0c8e640 1 -- 172.21.15.112:0/2408185956 >> 172.21.15.112:0/2408185956 conn(0x7fa8ac0f7000 msgr2=0x7fa8ac0f9410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:19.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.548+0000 7fa8b0c8e640 1 -- 172.21.15.112:0/2408185956 shutdown_connections 2024-09-16T00:32:19.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.548+0000 7fa8b0c8e640 1 -- 172.21.15.112:0/2408185956 wait complete. 2024-09-16T00:32:19.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.549+0000 7fa8b0c8e640 1 Processor -- start 2024-09-16T00:32:19.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.549+0000 7fa8b0c8e640 1 -- start start 2024-09-16T00:32:19.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.549+0000 7fa8b0c8e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa8ac0fb480 0x7fa8ac1131e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:19.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.549+0000 7fa8b0c8e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8ac1014c0 0x7fa8ac113720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:19.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.549+0000 7fa8b0c8e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8ac113d20 con 0x7fa8ac1014c0 2024-09-16T00:32:19.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.549+0000 7fa8b0c8e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8ac10d160 con 0x7fa8ac0fb480 2024-09-16T00:32:19.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.549+0000 7fa8a9d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8ac1014c0 0x7fa8ac113720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:19.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.549+0000 7fa8a9d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8ac1014c0 0x7fa8ac113720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38798/0 (socket says 172.21.15.112:38798) 2024-09-16T00:32:19.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.549+0000 7fa8a9d9b640 1 -- 172.21.15.112:0/1597347857 learned_addr learned my addr 172.21.15.112:0/1597347857 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:32:19.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.549+0000 7fa8aa59c640 1 --2- 172.21.15.112:0/1597347857 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa8ac0fb480 0x7fa8ac1131e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:19.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.550+0000 7fa8a9d9b640 1 -- 172.21.15.112:0/1597347857 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa8ac0fb480 msgr2=0x7fa8ac1131e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:19.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.550+0000 7fa8a9d9b640 1 --2- 172.21.15.112:0/1597347857 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa8ac0fb480 0x7fa8ac1131e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:19.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.550+0000 7fa8a9d9b640 1 -- 172.21.15.112:0/1597347857 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8a0005040 con 0x7fa8ac1014c0 2024-09-16T00:32:19.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.550+0000 7fa8aa59c640 1 --2- 172.21.15.112:0/1597347857 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa8ac0fb480 0x7fa8ac1131e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:32:19.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.550+0000 7fa8a9d9b640 1 --2- 172.21.15.112:0/1597347857 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8ac1014c0 0x7fa8ac113720 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa8a0000c00 tx=0x7fa8a0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:19.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.550+0000 7fa89affd640 1 -- 172.21.15.112:0/1597347857 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa8a0038070 con 0x7fa8ac1014c0 2024-09-16T00:32:19.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.550+0000 7fa89affd640 1 -- 172.21.15.112:0/1597347857 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fa8a002ade0 con 0x7fa8ac1014c0 2024-09-16T00:32:19.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.550+0000 7fa89affd640 1 -- 172.21.15.112:0/1597347857 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa8a002c7b0 con 0x7fa8ac1014c0 2024-09-16T00:32:19.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.550+0000 7fa8b0c8e640 1 -- 172.21.15.112:0/1597347857 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8ac10d3e0 con 0x7fa8ac1014c0 2024-09-16T00:32:19.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.550+0000 7fa8b0c8e640 1 -- 172.21.15.112:0/1597347857 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8ac10d920 con 0x7fa8ac1014c0 2024-09-16T00:32:19.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.551+0000 7fa8b0c8e640 1 -- 172.21.15.112:0/1597347857 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8ac0fcbc0 con 0x7fa8ac1014c0 2024-09-16T00:32:19.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.552+0000 7fa89affd640 1 -- 172.21.15.112:0/1597347857 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fa8a0030020 con 0x7fa8ac1014c0 2024-09-16T00:32:19.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.552+0000 7fa89affd640 1 --2- 172.21.15.112:0/1597347857 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa89406e530 0x7fa8940709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:19.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.552+0000 7fa89affd640 1 -- 172.21.15.112:0/1597347857 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fa8a00ae480 con 0x7fa8ac1014c0 2024-09-16T00:32:19.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.553+0000 7fa8aa59c640 1 --2- 172.21.15.112:0/1597347857 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa89406e530 0x7fa8940709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:19.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.553+0000 7fa8aa59c640 1 --2- 172.21.15.112:0/1597347857 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa89406e530 0x7fa8940709e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa89c004fd0 tx=0x7fa89c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:19.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.556+0000 7fa89affd640 1 -- 172.21.15.112:0/1597347857 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fa8a0077640 con 0x7fa8ac1014c0 2024-09-16T00:32:19.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.742+0000 7fa8b0c8e640 1 -- 172.21.15.112:0/1597347857 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa8ac108460 con 0x7fa89406e530 2024-09-16T00:32:19.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.743+0000 7fa89affd640 1 -- 172.21.15.112:0/1597347857 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fa8ac108460 con 0x7fa89406e530 2024-09-16T00:32:19.744 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:32:19.744 INFO:teuthology.orchestra.run.smithi112.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80", 2024-09-16T00:32:19.744 INFO:teuthology.orchestra.run.smithi112.stdout: "in_progress": true, 2024-09-16T00:32:19.744 INFO:teuthology.orchestra.run.smithi112.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T00:32:19.744 INFO:teuthology.orchestra.run.smithi112.stdout: "services_complete": [ 2024-09-16T00:32:19.744 INFO:teuthology.orchestra.run.smithi112.stdout: "crash", 2024-09-16T00:32:19.744 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr", 2024-09-16T00:32:19.744 INFO:teuthology.orchestra.run.smithi112.stdout: "mon" 2024-09-16T00:32:19.744 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-16T00:32:19.744 INFO:teuthology.orchestra.run.smithi112.stdout: "progress": "10/21 daemons upgraded", 2024-09-16T00:32:19.744 INFO:teuthology.orchestra.run.smithi112.stdout: "message": "Currently upgrading osd daemons", 2024-09-16T00:32:19.745 INFO:teuthology.orchestra.run.smithi112.stdout: "is_paused": false 2024-09-16T00:32:19.745 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:32:19.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.746+0000 7fa8b0c8e640 1 -- 172.21.15.112:0/1597347857 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa89406e530 msgr2=0x7fa8940709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:19.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.746+0000 7fa8b0c8e640 1 --2- 172.21.15.112:0/1597347857 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa89406e530 0x7fa8940709e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa89c004fd0 tx=0x7fa89c00b040 comp rx=0 tx=0).stop 2024-09-16T00:32:19.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.746+0000 7fa8b0c8e640 1 -- 172.21.15.112:0/1597347857 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8ac1014c0 msgr2=0x7fa8ac113720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:19.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.746+0000 7fa8b0c8e640 1 --2- 172.21.15.112:0/1597347857 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8ac1014c0 0x7fa8ac113720 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa8a0000c00 tx=0x7fa8a0009da0 comp rx=0 tx=0).stop 2024-09-16T00:32:19.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.746+0000 7fa8b0c8e640 1 -- 172.21.15.112:0/1597347857 shutdown_connections 2024-09-16T00:32:19.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.746+0000 7fa8b0c8e640 1 --2- 172.21.15.112:0/1597347857 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa89406e530 0x7fa8940709e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:19.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.746+0000 7fa8b0c8e640 1 --2- 172.21.15.112:0/1597347857 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8ac1014c0 0x7fa8ac113720 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:19.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.746+0000 7fa8b0c8e640 1 --2- 172.21.15.112:0/1597347857 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa8ac0fb480 0x7fa8ac1131e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:19.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.746+0000 7fa8b0c8e640 1 -- 172.21.15.112:0/1597347857 >> 172.21.15.112:0/1597347857 conn(0x7fa8ac0f7000 msgr2=0x7fa8ac065a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:19.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.746+0000 7fa8b0c8e640 1 -- 172.21.15.112:0/1597347857 shutdown_connections 2024-09-16T00:32:19.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.746+0000 7fa8b0c8e640 1 -- 172.21.15.112:0/1597347857 wait complete. 2024-09-16T00:32:19.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.824+0000 7fdbd3de1640 1 -- 172.21.15.112:0/507084075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbcc0fd3a0 msgr2=0x7fdbcc0fd770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:19.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.824+0000 7fdbd3de1640 1 --2- 172.21.15.112:0/507084075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbcc0fd3a0 0x7fdbcc0fd770 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fdbc80048d0 tx=0x7fdbc8009da0 comp rx=0 tx=0).stop 2024-09-16T00:32:19.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.824+0000 7fdbd3de1640 1 -- 172.21.15.112:0/507084075 shutdown_connections 2024-09-16T00:32:19.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.824+0000 7fdbd3de1640 1 --2- 172.21.15.112:0/507084075 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdbcc0fac80 0x7fdbcc0fb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:19.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.824+0000 7fdbd3de1640 1 --2- 172.21.15.112:0/507084075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbcc0fd3a0 0x7fdbcc0fd770 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:19.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.824+0000 7fdbd3de1640 1 -- 172.21.15.112:0/507084075 >> 172.21.15.112:0/507084075 conn(0x7fdbcc070ce0 msgr2=0x7fdbcc0710e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:19.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.824+0000 7fdbd3de1640 1 -- 172.21.15.112:0/507084075 shutdown_connections 2024-09-16T00:32:19.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.824+0000 7fdbd3de1640 1 -- 172.21.15.112:0/507084075 wait complete. 2024-09-16T00:32:19.826 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:19 smithi112 ceph-mon[97750]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:32:19.826 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:19 smithi112 ceph-mon[97750]: pgmap v111: 65 pgs: 18 active+undersized, 20 active+undersized+degraded, 27 active+clean; 325 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 78/372 objects degraded (20.968%) 2024-09-16T00:32:19.826 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:19 smithi112 ceph-mon[97750]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:32:19.826 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:19 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:32:19.826 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:19 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:32:19.826 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:19 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/3416513098' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:32:19.826 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:19 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:32:19.826 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:19 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:32:19.826 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:19 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/3295529844' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:32:19.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.825+0000 7fdbd3de1640 1 Processor -- start 2024-09-16T00:32:19.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.825+0000 7fdbd3de1640 1 -- start start 2024-09-16T00:32:19.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.825+0000 7fdbd3de1640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdbcc0fac80 0x7fdbcc1154b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:19.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.825+0000 7fdbd3de1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbcc0fd3a0 0x7fdbcc1159f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:19.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.825+0000 7fdbd3de1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbcc1160d0 con 0x7fdbcc0fd3a0 2024-09-16T00:32:19.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.825+0000 7fdbd3de1640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbcc10f450 con 0x7fdbcc0fac80 2024-09-16T00:32:19.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.825+0000 7fdbd137c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbcc0fd3a0 0x7fdbcc1159f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:19.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.825+0000 7fdbd137c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbcc0fd3a0 0x7fdbcc1159f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38818/0 (socket says 172.21.15.112:38818) 2024-09-16T00:32:19.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.825+0000 7fdbd137c640 1 -- 172.21.15.112:0/101954387 learned_addr learned my addr 172.21.15.112:0/101954387 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:32:19.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.825+0000 7fdbd1b7d640 1 --2- 172.21.15.112:0/101954387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdbcc0fac80 0x7fdbcc1154b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:19.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.826+0000 7fdbd137c640 1 -- 172.21.15.112:0/101954387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdbcc0fac80 msgr2=0x7fdbcc1154b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:19.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.826+0000 7fdbd137c640 1 --2- 172.21.15.112:0/101954387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdbcc0fac80 0x7fdbcc1154b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:19.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.826+0000 7fdbd137c640 1 -- 172.21.15.112:0/101954387 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbc8005040 con 0x7fdbcc0fd3a0 2024-09-16T00:32:19.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.826+0000 7fdbd137c640 1 --2- 172.21.15.112:0/101954387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbcc0fd3a0 0x7fdbcc1159f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc00b790 tx=0x7fdbbc0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:19.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.826+0000 7fdbc27fc640 1 -- 172.21.15.112:0/101954387 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdbbc00c700 con 0x7fdbcc0fd3a0 2024-09-16T00:32:19.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.826+0000 7fdbd3de1640 1 -- 172.21.15.112:0/101954387 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbcc10f6e0 con 0x7fdbcc0fd3a0 2024-09-16T00:32:19.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.826+0000 7fdbc27fc640 1 -- 172.21.15.112:0/101954387 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fdbbc00cd00 con 0x7fdbcc0fd3a0 2024-09-16T00:32:19.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.826+0000 7fdbc27fc640 1 -- 172.21.15.112:0/101954387 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdbbc015e80 con 0x7fdbcc0fd3a0 2024-09-16T00:32:19.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.827+0000 7fdbd3de1640 1 -- 172.21.15.112:0/101954387 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbcc10fc00 con 0x7fdbcc0fd3a0 2024-09-16T00:32:19.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.827+0000 7fdbd3de1640 1 -- 172.21.15.112:0/101954387 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbcc0fbc90 con 0x7fdbcc0fd3a0 2024-09-16T00:32:19.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.828+0000 7fdbc27fc640 1 -- 172.21.15.112:0/101954387 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fdbbc012070 con 0x7fdbcc0fd3a0 2024-09-16T00:32:19.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.828+0000 7fdbc27fc640 1 --2- 172.21.15.112:0/101954387 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fdbb806e530 0x7fdbb80709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:19.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.828+0000 7fdbc27fc640 1 -- 172.21.15.112:0/101954387 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fdbbc08da30 con 0x7fdbcc0fd3a0 2024-09-16T00:32:19.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.829+0000 7fdbd1b7d640 1 --2- 172.21.15.112:0/101954387 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fdbb806e530 0x7fdbb80709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:19.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.829+0000 7fdbd1b7d640 1 --2- 172.21.15.112:0/101954387 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fdbb806e530 0x7fdbb80709e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8009ac0 tx=0x7fdbc8035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:19.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:19.833+0000 7fdbc27fc640 1 -- 172.21.15.112:0/101954387 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fdbbc056b70 con 0x7fdbcc0fd3a0 2024-09-16T00:32:19.840 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:19 smithi169 ceph-mon[77626]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:32:19.840 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:19 smithi169 ceph-mon[77626]: pgmap v111: 65 pgs: 18 active+undersized, 20 active+undersized+degraded, 27 active+clean; 325 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 78/372 objects degraded (20.968%) 2024-09-16T00:32:19.840 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:19 smithi169 ceph-mon[77626]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:32:19.840 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:19 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:32:19.840 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:19 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:32:19.841 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:19 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/3416513098' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:32:19.841 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:19 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:32:19.841 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:19 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:32:19.841 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:19 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/3295529844' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:32:20.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:20.062+0000 7fdbd3de1640 1 -- 172.21.15.112:0/101954387 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fdbcc074c00 con 0x7fdbcc0fd3a0 2024-09-16T00:32:20.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:20.063+0000 7fdbc27fc640 1 -- 172.21.15.112:0/101954387 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1573 (secure 0 0 0) 0x7fdbbc05a230 con 0x7fdbcc0fd3a0 2024-09-16T00:32:20.063 INFO:teuthology.orchestra.run.smithi112.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; 1 osds down; Degraded data redundancy: 78/372 objects degraded (20.968%), 20 pgs degraded 2024-09-16T00:32:20.063 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T00:32:20.063 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T00:32:20.064 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] OSD_DOWN: 1 osds down 2024-09-16T00:32:20.064 INFO:teuthology.orchestra.run.smithi112.stdout: osd.3 (root=default,host=smithi169) is down 2024-09-16T00:32:20.064 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 78/372 objects degraded (20.968%), 20 pgs degraded 2024-09-16T00:32:20.064 INFO:teuthology.orchestra.run.smithi112.stdout: pg 1.0 is active+undersized+degraded, acting [0,1] 2024-09-16T00:32:20.064 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.0 is active+undersized+degraded, acting [1,0] 2024-09-16T00:32:20.064 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.5 is active+undersized+degraded, acting [0,4] 2024-09-16T00:32:20.064 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.6 is active+undersized+degraded, acting [1,4] 2024-09-16T00:32:20.064 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.7 is active+undersized+degraded, acting [4,2] 2024-09-16T00:32:20.064 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.8 is active+undersized+degraded, acting [5,0] 2024-09-16T00:32:20.064 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.a is active+undersized+degraded, acting [1,4] 2024-09-16T00:32:20.064 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.b is active+undersized+degraded, acting [4,5] 2024-09-16T00:32:20.064 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.c is active+undersized+degraded, acting [2,0] 2024-09-16T00:32:20.064 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.d is active+undersized+degraded, acting [1,2] 2024-09-16T00:32:20.064 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.e is active+undersized+degraded, acting [2,0] 2024-09-16T00:32:20.064 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.11 is active+undersized+degraded, acting [4,1] 2024-09-16T00:32:20.065 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.12 is active+undersized+degraded, acting [1,0] 2024-09-16T00:32:20.065 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.14 is active+undersized+degraded, acting [4,5] 2024-09-16T00:32:20.065 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.15 is active+undersized+degraded, acting [1,0] 2024-09-16T00:32:20.065 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.16 is active+undersized+degraded, acting [5,2] 2024-09-16T00:32:20.065 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.18 is active+undersized+degraded, acting [5,4] 2024-09-16T00:32:20.065 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.1a is active+undersized+degraded, acting [4,5] 2024-09-16T00:32:20.065 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.1d is active+undersized+degraded, acting [5,0] 2024-09-16T00:32:20.065 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.1f is active+undersized+degraded, acting [0,4] 2024-09-16T00:32:20.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:20.065+0000 7fdbd3de1640 1 -- 172.21.15.112:0/101954387 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fdbb806e530 msgr2=0x7fdbb80709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:20.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:20.065+0000 7fdbd3de1640 1 --2- 172.21.15.112:0/101954387 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fdbb806e530 0x7fdbb80709e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8009ac0 tx=0x7fdbc8035040 comp rx=0 tx=0).stop 2024-09-16T00:32:20.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:20.065+0000 7fdbd3de1640 1 -- 172.21.15.112:0/101954387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbcc0fd3a0 msgr2=0x7fdbcc1159f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:20.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:20.065+0000 7fdbd3de1640 1 --2- 172.21.15.112:0/101954387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbcc0fd3a0 0x7fdbcc1159f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc00b790 tx=0x7fdbbc0091d0 comp rx=0 tx=0).stop 2024-09-16T00:32:20.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:20.065+0000 7fdbd3de1640 1 -- 172.21.15.112:0/101954387 shutdown_connections 2024-09-16T00:32:20.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:20.065+0000 7fdbd3de1640 1 --2- 172.21.15.112:0/101954387 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fdbb806e530 0x7fdbb80709e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:20.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:20.065+0000 7fdbd3de1640 1 --2- 172.21.15.112:0/101954387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbcc0fd3a0 0x7fdbcc1159f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:20.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:20.065+0000 7fdbd3de1640 1 --2- 172.21.15.112:0/101954387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdbcc0fac80 0x7fdbcc1154b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:20.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:20.065+0000 7fdbd3de1640 1 -- 172.21.15.112:0/101954387 >> 172.21.15.112:0/101954387 conn(0x7fdbcc070ce0 msgr2=0x7fdbcc0f8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:20.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:20.065+0000 7fdbd3de1640 1 -- 172.21.15.112:0/101954387 shutdown_connections 2024-09-16T00:32:20.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:20.065+0000 7fdbd3de1640 1 -- 172.21.15.112:0/101954387 wait complete. 2024-09-16T00:32:20.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:20 smithi112 ceph-mon[97750]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:32:20.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:20 smithi112 ceph-mon[97750]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:32:20.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:20 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/101954387' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:32:20.843 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:20 smithi169 ceph-mon[77626]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:32:20.843 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:20 smithi169 ceph-mon[77626]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:32:20.843 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:20 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/101954387' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:32:21.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:21 smithi112 ceph-mon[97750]: pgmap v112: 65 pgs: 18 active+undersized, 20 active+undersized+degraded, 27 active+clean; 325 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 78/372 objects degraded (20.968%) 2024-09-16T00:32:21.869 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:21 smithi169 ceph-mon[77626]: pgmap v112: 65 pgs: 18 active+undersized, 20 active+undersized+degraded, 27 active+clean; 325 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 78/372 objects degraded (20.968%) 2024-09-16T00:32:24.026 INFO:journalctl@ceph.osd.3.smithi169.stdout:Sep 16 00:32:23 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-3[86440]: 2024-09-16T00:32:23.688+0000 7fbd2f541640 -1 osd.3 62 log_to_monitors true 2024-09-16T00:32:24.026 INFO:journalctl@ceph.osd.3.smithi169.stdout:Sep 16 00:32:23 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-3[86440]: 2024-09-16T00:32:23.748+0000 7fbd272db640 -1 osd.3 62 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T00:32:24.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:23 smithi169 ceph-mon[77626]: pgmap v113: 65 pgs: 18 active+undersized, 20 active+undersized+degraded, 27 active+clean; 325 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 78/372 objects degraded (20.968%) 2024-09-16T00:32:24.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:23 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:32:24.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:23 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:32:24.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:23 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:32:24.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:23 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:32:24.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:23 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:32:24.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:23 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:32:24.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:23 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:32:24.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:23 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:32:24.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:23 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:32:24.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:23 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-09-16T00:32:24.028 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:23 smithi169 ceph-mon[77626]: from='osd.3 [v2:172.21.15.169:6800/3101740036,v1:172.21.15.169:6801/3101740036]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-16T00:32:24.028 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:23 smithi169 ceph-mon[77626]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-16T00:32:24.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:23 smithi112 ceph-mon[97750]: pgmap v113: 65 pgs: 18 active+undersized, 20 active+undersized+degraded, 27 active+clean; 325 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 78/372 objects degraded (20.968%) 2024-09-16T00:32:24.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:23 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:32:24.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:23 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:32:24.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:23 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:32:24.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:23 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:32:24.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:23 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:32:24.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:23 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:32:24.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:23 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:32:24.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:23 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:32:24.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:23 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:32:24.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:23 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-09-16T00:32:24.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:23 smithi112 ceph-mon[97750]: from='osd.3 [v2:172.21.15.169:6800/3101740036,v1:172.21.15.169:6801/3101740036]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-16T00:32:24.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:23 smithi112 ceph-mon[97750]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-16T00:32:25.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:24 smithi169 ceph-mon[77626]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-09-16T00:32:25.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:24 smithi169 ceph-mon[77626]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-09-16T00:32:25.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:24 smithi169 ceph-mon[77626]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-16T00:32:25.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:24 smithi169 ceph-mon[77626]: from='osd.3 [v2:172.21.15.169:6800/3101740036,v1:172.21.15.169:6801/3101740036]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-09-16T00:32:25.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:24 smithi169 ceph-mon[77626]: osdmap e65: 6 total, 5 up, 6 in 2024-09-16T00:32:25.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:24 smithi169 ceph-mon[77626]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-09-16T00:32:25.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:24 smithi112 ceph-mon[97750]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-09-16T00:32:25.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:24 smithi112 ceph-mon[97750]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-09-16T00:32:25.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:24 smithi112 ceph-mon[97750]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-16T00:32:25.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:24 smithi112 ceph-mon[97750]: from='osd.3 [v2:172.21.15.169:6800/3101740036,v1:172.21.15.169:6801/3101740036]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-09-16T00:32:25.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:24 smithi112 ceph-mon[97750]: osdmap e65: 6 total, 5 up, 6 in 2024-09-16T00:32:25.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:24 smithi112 ceph-mon[97750]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-09-16T00:32:26.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:25 smithi169 ceph-mon[77626]: pgmap v115: 65 pgs: 18 active+undersized, 20 active+undersized+degraded, 27 active+clean; 325 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s; 78/372 objects degraded (20.968%) 2024-09-16T00:32:26.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:25 smithi169 ceph-mon[77626]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-16T00:32:26.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:25 smithi169 ceph-mon[77626]: osd.3 [v2:172.21.15.169:6800/3101740036,v1:172.21.15.169:6801/3101740036] boot 2024-09-16T00:32:26.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:25 smithi169 ceph-mon[77626]: osdmap e66: 6 total, 6 up, 6 in 2024-09-16T00:32:26.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:25 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:32:26.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:25 smithi169 ceph-mon[77626]: osdmap e67: 6 total, 6 up, 6 in 2024-09-16T00:32:26.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:25 smithi112 ceph-mon[97750]: pgmap v115: 65 pgs: 18 active+undersized, 20 active+undersized+degraded, 27 active+clean; 325 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s; 78/372 objects degraded (20.968%) 2024-09-16T00:32:26.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:25 smithi112 ceph-mon[97750]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-16T00:32:26.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:25 smithi112 ceph-mon[97750]: osd.3 [v2:172.21.15.169:6800/3101740036,v1:172.21.15.169:6801/3101740036] boot 2024-09-16T00:32:26.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:25 smithi112 ceph-mon[97750]: osdmap e66: 6 total, 6 up, 6 in 2024-09-16T00:32:26.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:25 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:32:26.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:25 smithi112 ceph-mon[97750]: osdmap e67: 6 total, 6 up, 6 in 2024-09-16T00:32:28.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:27 smithi169 ceph-mon[77626]: pgmap v118: 65 pgs: 18 active+undersized, 20 active+undersized+degraded, 27 active+clean; 325 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 78/372 objects degraded (20.968%) 2024-09-16T00:32:28.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:27 smithi112 ceph-mon[97750]: pgmap v118: 65 pgs: 18 active+undersized, 20 active+undersized+degraded, 27 active+clean; 325 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 78/372 objects degraded (20.968%) 2024-09-16T00:32:29.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:28 smithi112 ceph-mon[97750]: pgmap v119: 65 pgs: 13 active+undersized, 17 active+undersized+degraded, 35 active+clean; 325 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s; 67/372 objects degraded (18.011%) 2024-09-16T00:32:29.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:28 smithi112 ceph-mon[97750]: Health check update: Degraded data redundancy: 67/372 objects degraded (18.011%), 17 pgs degraded (PG_DEGRADED) 2024-09-16T00:32:29.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:28 smithi169 ceph-mon[77626]: pgmap v119: 65 pgs: 13 active+undersized, 17 active+undersized+degraded, 35 active+clean; 325 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s; 67/372 objects degraded (18.011%) 2024-09-16T00:32:29.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:28 smithi169 ceph-mon[77626]: Health check update: Degraded data redundancy: 67/372 objects degraded (18.011%), 17 pgs degraded (PG_DEGRADED) 2024-09-16T00:32:31.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:30 smithi169 ceph-mon[77626]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 67/372 objects degraded (18.011%), 17 pgs degraded) 2024-09-16T00:32:31.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:30 smithi112 ceph-mon[97750]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 67/372 objects degraded (18.011%), 17 pgs degraded) 2024-09-16T00:32:32.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:31 smithi169 ceph-mon[77626]: pgmap v120: 65 pgs: 65 active+clean; 325 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 3 op/s 2024-09-16T00:32:32.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:31 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:32:32.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:31 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:32:32.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:31 smithi112 ceph-mon[97750]: pgmap v120: 65 pgs: 65 active+clean; 325 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 3 op/s 2024-09-16T00:32:32.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:31 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:32:32.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:31 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:32:34.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:33 smithi169 ceph-mon[77626]: pgmap v121: 65 pgs: 65 active+clean; 325 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-16T00:32:34.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:33 smithi112 ceph-mon[97750]: pgmap v121: 65 pgs: 65 active+clean; 325 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-16T00:32:36.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:35 smithi169 ceph-mon[77626]: pgmap v122: 65 pgs: 65 active+clean; 325 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-09-16T00:32:36.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:35 smithi112 ceph-mon[97750]: pgmap v122: 65 pgs: 65 active+clean; 325 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-09-16T00:32:38.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:37 smithi169 ceph-mon[77626]: pgmap v123: 65 pgs: 65 active+clean; 325 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-09-16T00:32:38.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:37 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:32:38.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:37 smithi112 ceph-mon[97750]: pgmap v123: 65 pgs: 65 active+clean; 325 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-09-16T00:32:38.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:37 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:32:39.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:38 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-09-16T00:32:39.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:38 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-09-16T00:32:39.864 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:39 smithi169 ceph-mon[77626]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-09-16T00:32:39.864 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:39 smithi169 ceph-mon[77626]: Upgrade: osd.4 is safe to restart 2024-09-16T00:32:39.864 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:39 smithi169 ceph-mon[77626]: pgmap v124: 65 pgs: 65 active+clean; 325 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-16T00:32:39.864 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:39 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:32:39.864 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:39 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-16T00:32:39.864 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:39 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:32:40.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:39 smithi112 ceph-mon[97750]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-09-16T00:32:40.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:39 smithi112 ceph-mon[97750]: Upgrade: osd.4 is safe to restart 2024-09-16T00:32:40.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:39 smithi112 ceph-mon[97750]: pgmap v124: 65 pgs: 65 active+clean; 325 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-16T00:32:40.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:39 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:32:40.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:39 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-16T00:32:40.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:39 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:32:40.875 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:40 smithi169 ceph-mon[77626]: Upgrade: Updating osd.4 2024-09-16T00:32:40.875 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:40 smithi169 ceph-mon[77626]: Deploying daemon osd.4 on smithi169 2024-09-16T00:32:41.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:40 smithi112 ceph-mon[97750]: Upgrade: Updating osd.4 2024-09-16T00:32:41.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:40 smithi112 ceph-mon[97750]: Deploying daemon osd.4 on smithi169 2024-09-16T00:32:41.727 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:41 smithi169 ceph-mon[77626]: pgmap v125: 65 pgs: 65 active+clean; 325 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:32:42.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:41 smithi112 ceph-mon[97750]: pgmap v125: 65 pgs: 65 active+clean; 325 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:32:42.382 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:32:42 smithi169 systemd[1]: Stopping Ceph osd.4 for d346dd56-73c0-11ef-bceb-c7b262605968... 2024-09-16T00:32:42.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:42 smithi169 ceph-mon[77626]: osd.4 marked itself down and dead 2024-09-16T00:32:42.776 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:32:42 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-4[46505]: 2024-09-16T00:32:42.380+0000 7f0c62d46640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:32:42.777 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:32:42 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-4[46505]: 2024-09-16T00:32:42.380+0000 7f0c62d46640 -1 osd.4 67 *** Got signal Terminated *** 2024-09-16T00:32:42.777 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:32:42 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-4[46505]: 2024-09-16T00:32:42.380+0000 7f0c62d46640 -1 osd.4 67 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:32:43.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:42 smithi112 ceph-mon[97750]: osd.4 marked itself down and dead 2024-09-16T00:32:43.365 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:32:43 smithi169 podman[90814]: 2024-09-16 00:32:43.075416167 +0000 UTC m=+0.810674453 container died d3e4cdb016b9f25deedb061235bc072481ed8e08d41ff4c12774e7c86c07f52a (image=quay.ceph.io/ceph-ci/ceph@sha256:83596473fc70117d507bb3d925c5bd11b70f46bbd4dffcc8452f145d93eaedc4, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-4, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=quincy-6d025bc, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-16T00:32:43.699 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:43 smithi169 ceph-mon[77626]: pgmap v126: 65 pgs: 65 active+clean; 325 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:32:43.699 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:43 smithi169 ceph-mon[77626]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-16T00:32:43.700 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:43 smithi169 ceph-mon[77626]: osdmap e68: 6 total, 5 up, 6 in 2024-09-16T00:32:43.700 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:32:43 smithi169 podman[90814]: 2024-09-16 00:32:43.456919526 +0000 UTC m=+1.192177799 container cleanup d3e4cdb016b9f25deedb061235bc072481ed8e08d41ff4c12774e7c86c07f52a (image=quay.ceph.io/ceph-ci/ceph@sha256:83596473fc70117d507bb3d925c5bd11b70f46bbd4dffcc8452f145d93eaedc4, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-4, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=quincy-6d025bc, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:32:43.700 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:32:43 smithi169 bash[90814]: ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-4 2024-09-16T00:32:44.022 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:32:43 smithi169 podman[90831]: 2024-09-16 00:32:43.700065741 +0000 UTC m=+0.622699966 container remove d3e4cdb016b9f25deedb061235bc072481ed8e08d41ff4c12774e7c86c07f52a (image=quay.ceph.io/ceph-ci/ceph@sha256:83596473fc70117d507bb3d925c5bd11b70f46bbd4dffcc8452f145d93eaedc4, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=quincy-6d025bc, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:32:44.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:43 smithi112 ceph-mon[97750]: pgmap v126: 65 pgs: 65 active+clean; 325 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:32:44.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:43 smithi112 ceph-mon[97750]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-16T00:32:44.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:43 smithi112 ceph-mon[97750]: osdmap e68: 6 total, 5 up, 6 in 2024-09-16T00:32:44.276 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:32:44 smithi169 podman[90920]: 2024-09-16 00:32:43.92530703 +0000 UTC m=+0.015934411 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:32:44.276 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:32:44 smithi169 podman[90920]: 2024-09-16 00:32:44.051271411 +0000 UTC m=+0.141898783 container create 02bf017367defd374adbc29d1709e432be715c3a3d9edb12456b4a9a8549250a (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-16T00:32:44.276 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:32:44 smithi169 podman[90920]: 2024-09-16 00:32:44.262299053 +0000 UTC m=+0.352926431 container init 02bf017367defd374adbc29d1709e432be715c3a3d9edb12456b4a9a8549250a (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0) 2024-09-16T00:32:44.276 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:32:44 smithi169 podman[90920]: 2024-09-16 00:32:44.26652262 +0000 UTC m=+0.357149989 container start 02bf017367defd374adbc29d1709e432be715c3a3d9edb12456b4a9a8549250a (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-16T00:32:44.622 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:32:44 smithi169 podman[90920]: 2024-09-16 00:32:44.362225912 +0000 UTC m=+0.452853281 container attach 02bf017367defd374adbc29d1709e432be715c3a3d9edb12456b4a9a8549250a (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:32:44.622 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:32:44 smithi169 podman[90920]: 2024-09-16 00:32:44.458083097 +0000 UTC m=+0.548710468 container died 02bf017367defd374adbc29d1709e432be715c3a3d9edb12456b4a9a8549250a (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-4-deactivate, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-16T00:32:44.973 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:44 smithi169 ceph-mon[77626]: osdmap e69: 6 total, 5 up, 6 in 2024-09-16T00:32:45.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:44 smithi112 ceph-mon[97750]: osdmap e69: 6 total, 5 up, 6 in 2024-09-16T00:32:45.276 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:32:44 smithi169 podman[90943]: 2024-09-16 00:32:44.973942368 +0000 UTC m=+0.510331505 container remove 02bf017367defd374adbc29d1709e432be715c3a3d9edb12456b4a9a8549250a (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-16T00:32:45.276 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:32:44 smithi169 systemd[1]: ceph-d346dd56-73c0-11ef-bceb-c7b262605968@osd.4.service: Deactivated successfully. 2024-09-16T00:32:45.276 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:32:44 smithi169 systemd[1]: Stopped Ceph osd.4 for d346dd56-73c0-11ef-bceb-c7b262605968. 2024-09-16T00:32:45.276 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:32:44 smithi169 systemd[1]: ceph-d346dd56-73c0-11ef-bceb-c7b262605968@osd.4.service: Consumed 51.875s CPU time. 2024-09-16T00:32:45.277 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:32:45 smithi169 systemd[1]: Starting Ceph osd.4 for d346dd56-73c0-11ef-bceb-c7b262605968... 2024-09-16T00:32:45.720 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:45 smithi169 ceph-mon[77626]: pgmap v129: 65 pgs: 13 active+undersized, 2 peering, 1 stale+active+clean, 14 active+undersized+degraded, 35 active+clean; 325 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 58/372 objects degraded (15.591%) 2024-09-16T00:32:45.720 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:45 smithi169 ceph-mon[77626]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-16T00:32:45.720 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:45 smithi169 ceph-mon[77626]: Health check failed: Degraded data redundancy: 58/372 objects degraded (15.591%), 14 pgs degraded (PG_DEGRADED) 2024-09-16T00:32:45.720 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:32:45 smithi169 podman[91042]: 2024-09-16 00:32:45.307153912 +0000 UTC m=+0.016096434 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:32:45.720 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:32:45 smithi169 podman[91042]: 2024-09-16 00:32:45.408773103 +0000 UTC m=+0.117715630 container create a70aeed957941f65fde9d9ee84a7e508514fc89daa542e15010fb0f2a10bb2d0 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-4-activate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:32:45.720 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:32:45 smithi169 podman[91042]: 2024-09-16 00:32:45.644770303 +0000 UTC m=+0.353712830 container init a70aeed957941f65fde9d9ee84a7e508514fc89daa542e15010fb0f2a10bb2d0 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-4-activate, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:32:45.720 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:32:45 smithi169 podman[91042]: 2024-09-16 00:32:45.648494511 +0000 UTC m=+0.357437028 container start a70aeed957941f65fde9d9ee84a7e508514fc89daa542e15010fb0f2a10bb2d0 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-4-activate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-8921c8e, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:32:45.720 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:32:45 smithi169 podman[91042]: 2024-09-16 00:32:45.720345956 +0000 UTC m=+0.429288493 container attach a70aeed957941f65fde9d9ee84a7e508514fc89daa542e15010fb0f2a10bb2d0 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-4-activate, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-16T00:32:46.026 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:32:45 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-4-activate[91057]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T00:32:46.026 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:32:45 smithi169 bash[91042]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T00:32:46.026 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:32:45 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-4-activate[91057]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T00:32:46.026 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:32:45 smithi169 bash[91042]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T00:32:46.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:45 smithi112 ceph-mon[97750]: pgmap v129: 65 pgs: 13 active+undersized, 2 peering, 1 stale+active+clean, 14 active+undersized+degraded, 35 active+clean; 325 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 58/372 objects degraded (15.591%) 2024-09-16T00:32:46.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:45 smithi112 ceph-mon[97750]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-16T00:32:46.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:45 smithi112 ceph-mon[97750]: Health check failed: Degraded data redundancy: 58/372 objects degraded (15.591%), 14 pgs degraded (PG_DEGRADED) 2024-09-16T00:32:46.776 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:32:46 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-4-activate[91057]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-16T00:32:46.777 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:32:46 smithi169 bash[91042]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-16T00:32:46.777 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:32:46 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-4-activate[91057]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-09-16T00:32:46.777 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:32:46 smithi169 bash[91042]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-09-16T00:32:47.106 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:32:46 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-4-activate[91057]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-09-16T00:32:47.106 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:32:46 smithi169 bash[91042]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-09-16T00:32:47.107 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:32:46 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-4-activate[91057]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-09-16T00:32:47.107 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:32:46 smithi169 bash[91042]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-09-16T00:32:47.107 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:32:46 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-4-activate[91057]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-09-16T00:32:47.107 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:32:46 smithi169 bash[91042]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-09-16T00:32:47.107 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:32:46 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-4-activate[91057]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-16T00:32:47.107 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:32:46 smithi169 bash[91042]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-16T00:32:47.107 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:32:46 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-4-activate[91057]: --> ceph-volume raw activate successful for osd ID: 4 2024-09-16T00:32:47.107 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:32:46 smithi169 bash[91042]: --> ceph-volume raw activate successful for osd ID: 4 2024-09-16T00:32:47.107 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:32:46 smithi169 podman[91042]: 2024-09-16 00:32:46.825906953 +0000 UTC m=+1.534849470 container died a70aeed957941f65fde9d9ee84a7e508514fc89daa542e15010fb0f2a10bb2d0 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-4-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=squid-8921c8e) 2024-09-16T00:32:47.526 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:32:47 smithi169 podman[91197]: 2024-09-16 00:32:47.349356169 +0000 UTC m=+0.517398362 container remove a70aeed957941f65fde9d9ee84a7e508514fc89daa542e15010fb0f2a10bb2d0 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-4-activate, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:32:47.819 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:47 smithi169 ceph-mon[77626]: pgmap v130: 65 pgs: 13 active+undersized, 2 peering, 1 stale+active+clean, 14 active+undersized+degraded, 35 active+clean; 325 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 58/372 objects degraded (15.591%) 2024-09-16T00:32:47.820 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:47 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:32:47.820 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:47 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:32:47.820 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:32:47 smithi169 podman[91256]: 2024-09-16 00:32:47.460344606 +0000 UTC m=+0.015690949 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:32:47.820 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:32:47 smithi169 podman[91256]: 2024-09-16 00:32:47.583580927 +0000 UTC m=+0.138927273 container create 769f026a54eb1a19f1cbead116d6087157579022785250a7841626a5d0e8f1d6 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-4, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-16T00:32:47.820 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:32:47 smithi169 podman[91256]: 2024-09-16 00:32:47.819955551 +0000 UTC m=+0.375301894 container init 769f026a54eb1a19f1cbead116d6087157579022785250a7841626a5d0e8f1d6 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-4, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-16T00:32:47.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:47 smithi112 ceph-mon[97750]: pgmap v130: 65 pgs: 13 active+undersized, 2 peering, 1 stale+active+clean, 14 active+undersized+degraded, 35 active+clean; 325 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 58/372 objects degraded (15.591%) 2024-09-16T00:32:47.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:47 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:32:47.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:47 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:32:48.276 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:32:47 smithi169 podman[91256]: 2024-09-16 00:32:47.823402228 +0000 UTC m=+0.378748572 container start 769f026a54eb1a19f1cbead116d6087157579022785250a7841626a5d0e8f1d6 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:32:48.276 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:32:47 smithi169 ceph-osd[91276]: -- 172.21.15.169:0/1200542893 <== mon.1 v2:172.21.15.169:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56447cf0c1e0 con 0x56447cea2000 2024-09-16T00:32:48.276 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:32:47 smithi169 bash[91256]: 769f026a54eb1a19f1cbead116d6087157579022785250a7841626a5d0e8f1d6 2024-09-16T00:32:48.276 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:32:47 smithi169 systemd[1]: Started Ceph osd.4 for d346dd56-73c0-11ef-bceb-c7b262605968. 2024-09-16T00:32:49.234 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:48 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:32:49.234 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:48 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:32:49.234 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:48 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:32:49.234 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:48 smithi169 ceph-mon[77626]: pgmap v131: 65 pgs: 16 active+undersized, 2 peering, 16 active+undersized+degraded, 31 active+clean; 325 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 64/372 objects degraded (17.204%) 2024-09-16T00:32:49.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:48 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:32:49.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:48 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:32:49.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:48 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:32:49.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:48 smithi112 ceph-mon[97750]: pgmap v131: 65 pgs: 16 active+undersized, 2 peering, 16 active+undersized+degraded, 31 active+clean; 325 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 64/372 objects degraded (17.204%) 2024-09-16T00:32:49.492 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:32:49 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-4[91272]: 2024-09-16T00:32:49.454+0000 7f5b73aea640 -1 Falling back to public interface 2024-09-16T00:32:50.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.157+0000 7f4ea8815640 1 -- 172.21.15.112:0/1573347038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ea4102620 msgr2=0x7f4ea41029f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:50.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.157+0000 7f4ea8815640 1 --2- 172.21.15.112:0/1573347038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ea4102620 0x7f4ea41029f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4e940048d0 tx=0x7f4e94009da0 comp rx=0 tx=0).stop 2024-09-16T00:32:50.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.157+0000 7f4ea8815640 1 -- 172.21.15.112:0/1573347038 shutdown_connections 2024-09-16T00:32:50.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.157+0000 7f4ea8815640 1 --2- 172.21.15.112:0/1573347038 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ea40fac80 0x7f4ea40fb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:50.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.157+0000 7f4ea8815640 1 --2- 172.21.15.112:0/1573347038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ea4102620 0x7f4ea41029f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:50.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.157+0000 7f4ea8815640 1 -- 172.21.15.112:0/1573347038 >> 172.21.15.112:0/1573347038 conn(0x7f4ea4070ce0 msgr2=0x7f4ea40710e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:50.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.157+0000 7f4ea8815640 1 -- 172.21.15.112:0/1573347038 shutdown_connections 2024-09-16T00:32:50.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.157+0000 7f4ea8815640 1 -- 172.21.15.112:0/1573347038 wait complete. 2024-09-16T00:32:50.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.157+0000 7f4ea8815640 1 Processor -- start 2024-09-16T00:32:50.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.158+0000 7f4ea8815640 1 -- start start 2024-09-16T00:32:50.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.158+0000 7f4ea8815640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ea40fac80 0x7f4ea41131d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:50.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.158+0000 7f4ea8815640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ea4102620 0x7f4ea4113710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:50.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.158+0000 7f4ea8815640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea4113d10 con 0x7f4ea40fac80 2024-09-16T00:32:50.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.158+0000 7f4ea8815640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea410d150 con 0x7f4ea4102620 2024-09-16T00:32:50.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.158+0000 7f4ea259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ea40fac80 0x7f4ea41131d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:50.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.158+0000 7f4ea259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ea40fac80 0x7f4ea41131d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54516/0 (socket says 172.21.15.112:54516) 2024-09-16T00:32:50.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.158+0000 7f4ea259c640 1 -- 172.21.15.112:0/83664368 learned_addr learned my addr 172.21.15.112:0/83664368 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:32:50.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.158+0000 7f4ea1d9b640 1 --2- 172.21.15.112:0/83664368 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ea4102620 0x7f4ea4113710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:50.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.158+0000 7f4ea259c640 1 -- 172.21.15.112:0/83664368 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ea4102620 msgr2=0x7f4ea4113710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:50.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.158+0000 7f4ea259c640 1 --2- 172.21.15.112:0/83664368 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ea4102620 0x7f4ea4113710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:50.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.158+0000 7f4ea259c640 1 -- 172.21.15.112:0/83664368 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e94005040 con 0x7f4ea40fac80 2024-09-16T00:32:50.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.158+0000 7f4ea259c640 1 --2- 172.21.15.112:0/83664368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ea40fac80 0x7f4ea41131d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4e940043d0 tx=0x7f4e94009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:50.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.159+0000 7f4e92ffd640 1 -- 172.21.15.112:0/83664368 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e94038070 con 0x7f4ea40fac80 2024-09-16T00:32:50.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.159+0000 7f4e92ffd640 1 -- 172.21.15.112:0/83664368 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f4e940096e0 con 0x7f4ea40fac80 2024-09-16T00:32:50.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.159+0000 7f4e92ffd640 1 -- 172.21.15.112:0/83664368 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e9403d420 con 0x7f4ea40fac80 2024-09-16T00:32:50.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.159+0000 7f4ea8815640 1 -- 172.21.15.112:0/83664368 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ea410d3d0 con 0x7f4ea40fac80 2024-09-16T00:32:50.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.159+0000 7f4ea8815640 1 -- 172.21.15.112:0/83664368 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ea410d8f0 con 0x7f4ea40fac80 2024-09-16T00:32:50.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.160+0000 7f4ea8815640 1 -- 172.21.15.112:0/83664368 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ea40657b0 con 0x7f4ea40fac80 2024-09-16T00:32:50.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.160+0000 7f4e92ffd640 1 -- 172.21.15.112:0/83664368 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f4e94030020 con 0x7f4ea40fac80 2024-09-16T00:32:50.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.161+0000 7f4e92ffd640 1 --2- 172.21.15.112:0/83664368 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f4e8c06e530 0x7f4e8c0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:50.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.161+0000 7f4e92ffd640 1 -- 172.21.15.112:0/83664368 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f4e940ae5f0 con 0x7f4ea40fac80 2024-09-16T00:32:50.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.161+0000 7f4ea1d9b640 1 --2- 172.21.15.112:0/83664368 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f4e8c06e530 0x7f4e8c0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:50.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.161+0000 7f4ea1d9b640 1 --2- 172.21.15.112:0/83664368 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f4e8c06e530 0x7f4e8c0709e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4ea410ea90 tx=0x7f4e98009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:50.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.165+0000 7f4e92ffd640 1 -- 172.21.15.112:0/83664368 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f4e94077730 con 0x7f4ea40fac80 2024-09-16T00:32:50.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.352+0000 7f4ea8815640 1 -- 172.21.15.112:0/83664368 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4ea404b720 con 0x7f4e8c06e530 2024-09-16T00:32:50.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.353+0000 7f4e92ffd640 1 -- 172.21.15.112:0/83664368 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f4ea404b720 con 0x7f4e8c06e530 2024-09-16T00:32:50.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.355+0000 7f4ea8815640 1 -- 172.21.15.112:0/83664368 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f4e8c06e530 msgr2=0x7f4e8c0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:50.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.355+0000 7f4ea8815640 1 --2- 172.21.15.112:0/83664368 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f4e8c06e530 0x7f4e8c0709e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4ea410ea90 tx=0x7f4e98009040 comp rx=0 tx=0).stop 2024-09-16T00:32:50.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.355+0000 7f4ea8815640 1 -- 172.21.15.112:0/83664368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ea40fac80 msgr2=0x7f4ea41131d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:50.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.355+0000 7f4ea8815640 1 --2- 172.21.15.112:0/83664368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ea40fac80 0x7f4ea41131d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4e940043d0 tx=0x7f4e94009da0 comp rx=0 tx=0).stop 2024-09-16T00:32:50.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.355+0000 7f4ea8815640 1 -- 172.21.15.112:0/83664368 shutdown_connections 2024-09-16T00:32:50.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.355+0000 7f4ea8815640 1 --2- 172.21.15.112:0/83664368 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f4e8c06e530 0x7f4e8c0709e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:50.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.355+0000 7f4ea8815640 1 --2- 172.21.15.112:0/83664368 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ea4102620 0x7f4ea4113710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:50.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.355+0000 7f4ea8815640 1 --2- 172.21.15.112:0/83664368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ea40fac80 0x7f4ea41131d0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:50.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.355+0000 7f4ea8815640 1 -- 172.21.15.112:0/83664368 >> 172.21.15.112:0/83664368 conn(0x7f4ea4070ce0 msgr2=0x7f4ea4105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:50.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.355+0000 7f4ea8815640 1 -- 172.21.15.112:0/83664368 shutdown_connections 2024-09-16T00:32:50.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.355+0000 7f4ea8815640 1 -- 172.21.15.112:0/83664368 wait complete. 2024-09-16T00:32:50.364 INFO:teuthology.orchestra.run.smithi112.stdout:true 2024-09-16T00:32:50.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.435+0000 7f7631eb7640 1 -- 172.21.15.112:0/3868875689 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f762c0fcf80 msgr2=0x7f762c0fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:50.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.435+0000 7f7631eb7640 1 --2- 172.21.15.112:0/3868875689 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f762c0fcf80 0x7f762c0fd3d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f76140048d0 tx=0x7f7614009da0 comp rx=0 tx=0).stop 2024-09-16T00:32:50.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.435+0000 7f7631eb7640 1 -- 172.21.15.112:0/3868875689 shutdown_connections 2024-09-16T00:32:50.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.435+0000 7f7631eb7640 1 --2- 172.21.15.112:0/3868875689 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f762c0fcf80 0x7f762c0fd3d0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:50.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.435+0000 7f7631eb7640 1 --2- 172.21.15.112:0/3868875689 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f762c0fb810 0x7f762c0fbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:50.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.435+0000 7f7631eb7640 1 -- 172.21.15.112:0/3868875689 >> 172.21.15.112:0/3868875689 conn(0x7f762c0723b0 msgr2=0x7f762c0727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:50.436 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:50 smithi112 ceph-mon[97750]: Health check update: Degraded data redundancy: 64/372 objects degraded (17.204%), 16 pgs degraded (PG_DEGRADED) 2024-09-16T00:32:50.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.435+0000 7f7631eb7640 1 -- 172.21.15.112:0/3868875689 shutdown_connections 2024-09-16T00:32:50.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.435+0000 7f7631eb7640 1 -- 172.21.15.112:0/3868875689 wait complete. 2024-09-16T00:32:50.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.436+0000 7f7631eb7640 1 Processor -- start 2024-09-16T00:32:50.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.436+0000 7f7631eb7640 1 -- start start 2024-09-16T00:32:50.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.436+0000 7f7631eb7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f762c0fb810 0x7f762c0f8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:50.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.436+0000 7f7631eb7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f762c0fcf80 0x7f762c0f8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:50.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.436+0000 7f7631eb7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f762c0f8fc0 con 0x7f762c0fcf80 2024-09-16T00:32:50.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.436+0000 7f7631eb7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f762c0f9130 con 0x7f762c0fb810 2024-09-16T00:32:50.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.436+0000 7f7623fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f762c0fcf80 0x7f762c0f8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:50.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.436+0000 7f7623fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f762c0fcf80 0x7f762c0f8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54528/0 (socket says 172.21.15.112:54528) 2024-09-16T00:32:50.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.436+0000 7f7623fff640 1 -- 172.21.15.112:0/3166536786 learned_addr learned my addr 172.21.15.112:0/3166536786 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:32:50.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.436+0000 7f762b7fe640 1 --2- 172.21.15.112:0/3166536786 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f762c0fb810 0x7f762c0f8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:50.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.437+0000 7f7623fff640 1 -- 172.21.15.112:0/3166536786 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f762c0fb810 msgr2=0x7f762c0f8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:50.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.437+0000 7f7623fff640 1 --2- 172.21.15.112:0/3166536786 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f762c0fb810 0x7f762c0f8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:50.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.437+0000 7f7623fff640 1 -- 172.21.15.112:0/3166536786 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7614005040 con 0x7f762c0fcf80 2024-09-16T00:32:50.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.437+0000 7f7623fff640 1 --2- 172.21.15.112:0/3166536786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f762c0fcf80 0x7f762c0f8a80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7614000c00 tx=0x7f7614009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:50.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.437+0000 7f7628ff9640 1 -- 172.21.15.112:0/3166536786 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7614038070 con 0x7f762c0fcf80 2024-09-16T00:32:50.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.437+0000 7f7628ff9640 1 -- 172.21.15.112:0/3166536786 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f761402ade0 con 0x7f762c0fcf80 2024-09-16T00:32:50.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.437+0000 7f7631eb7640 1 -- 172.21.15.112:0/3166536786 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f762c0f9360 con 0x7f762c0fcf80 2024-09-16T00:32:50.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.437+0000 7f7628ff9640 1 -- 172.21.15.112:0/3166536786 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f761402c7e0 con 0x7f762c0fcf80 2024-09-16T00:32:50.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.437+0000 7f7631eb7640 1 -- 172.21.15.112:0/3166536786 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f762c119830 con 0x7f762c0fcf80 2024-09-16T00:32:50.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.438+0000 7f7631eb7640 1 -- 172.21.15.112:0/3166536786 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f762c0fcbc0 con 0x7f762c0fcf80 2024-09-16T00:32:50.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.439+0000 7f7628ff9640 1 -- 172.21.15.112:0/3166536786 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f7614030020 con 0x7f762c0fcf80 2024-09-16T00:32:50.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.439+0000 7f7628ff9640 1 --2- 172.21.15.112:0/3166536786 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f761806e530 0x7f76180709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:50.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.439+0000 7f7628ff9640 1 -- 172.21.15.112:0/3166536786 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f76140addb0 con 0x7f762c0fcf80 2024-09-16T00:32:50.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.439+0000 7f762b7fe640 1 --2- 172.21.15.112:0/3166536786 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f761806e530 0x7f76180709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:50.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.440+0000 7f762b7fe640 1 --2- 172.21.15.112:0/3166536786 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f761806e530 0x7f76180709e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f761c004ed0 tx=0x7f761c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:50.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.443+0000 7f7628ff9640 1 -- 172.21.15.112:0/3166536786 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f7614076fa0 con 0x7f762c0fcf80 2024-09-16T00:32:50.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:50 smithi169 ceph-mon[77626]: Health check update: Degraded data redundancy: 64/372 objects degraded (17.204%), 16 pgs degraded (PG_DEGRADED) 2024-09-16T00:32:50.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.626+0000 7f7631eb7640 1 -- 172.21.15.112:0/3166536786 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f762c04b720 con 0x7f761806e530 2024-09-16T00:32:50.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.627+0000 7f7628ff9640 1 -- 172.21.15.112:0/3166536786 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f762c04b720 con 0x7f761806e530 2024-09-16T00:32:50.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.630+0000 7f7631eb7640 1 -- 172.21.15.112:0/3166536786 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f761806e530 msgr2=0x7f76180709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:50.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.630+0000 7f7631eb7640 1 --2- 172.21.15.112:0/3166536786 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f761806e530 0x7f76180709e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f761c004ed0 tx=0x7f761c00b040 comp rx=0 tx=0).stop 2024-09-16T00:32:50.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.630+0000 7f7631eb7640 1 -- 172.21.15.112:0/3166536786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f762c0fcf80 msgr2=0x7f762c0f8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:50.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.630+0000 7f7631eb7640 1 --2- 172.21.15.112:0/3166536786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f762c0fcf80 0x7f762c0f8a80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7614000c00 tx=0x7f7614009da0 comp rx=0 tx=0).stop 2024-09-16T00:32:50.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.630+0000 7f7631eb7640 1 -- 172.21.15.112:0/3166536786 shutdown_connections 2024-09-16T00:32:50.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.630+0000 7f7631eb7640 1 --2- 172.21.15.112:0/3166536786 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f761806e530 0x7f76180709e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:50.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.630+0000 7f7631eb7640 1 --2- 172.21.15.112:0/3166536786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f762c0fcf80 0x7f762c0f8a80 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:50.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.630+0000 7f7631eb7640 1 --2- 172.21.15.112:0/3166536786 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f762c0fb810 0x7f762c0f8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:50.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.630+0000 7f7631eb7640 1 -- 172.21.15.112:0/3166536786 >> 172.21.15.112:0/3166536786 conn(0x7f762c0723b0 msgr2=0x7f762c0fa680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:50.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.630+0000 7f7631eb7640 1 -- 172.21.15.112:0/3166536786 shutdown_connections 2024-09-16T00:32:50.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.630+0000 7f7631eb7640 1 -- 172.21.15.112:0/3166536786 wait complete. 2024-09-16T00:32:50.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.711+0000 7fa926ed7640 1 -- 172.21.15.112:0/3728968565 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa920102530 msgr2=0x7fa920102900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:50.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.711+0000 7fa926ed7640 1 --2- 172.21.15.112:0/3728968565 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa920102530 0x7fa920102900 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa9140048d0 tx=0x7fa914009da0 comp rx=0 tx=0).stop 2024-09-16T00:32:50.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.711+0000 7fa926ed7640 1 -- 172.21.15.112:0/3728968565 shutdown_connections 2024-09-16T00:32:50.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.711+0000 7fa926ed7640 1 --2- 172.21.15.112:0/3728968565 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa920102ed0 0x7fa9200fc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:50.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.711+0000 7fa926ed7640 1 --2- 172.21.15.112:0/3728968565 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa920102530 0x7fa920102900 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:50.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.711+0000 7fa926ed7640 1 -- 172.21.15.112:0/3728968565 >> 172.21.15.112:0/3728968565 conn(0x7fa9200f80f0 msgr2=0x7fa9200fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:50.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.711+0000 7fa926ed7640 1 -- 172.21.15.112:0/3728968565 shutdown_connections 2024-09-16T00:32:50.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.711+0000 7fa926ed7640 1 -- 172.21.15.112:0/3728968565 wait complete. 2024-09-16T00:32:50.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.712+0000 7fa926ed7640 1 Processor -- start 2024-09-16T00:32:50.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.712+0000 7fa926ed7640 1 -- start start 2024-09-16T00:32:50.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.712+0000 7fa926ed7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa920102530 0x7fa920113240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:50.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.712+0000 7fa926ed7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa920102ed0 0x7fa920113780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:50.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.712+0000 7fa926ed7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa920113d80 con 0x7fa920102530 2024-09-16T00:32:50.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.712+0000 7fa926ed7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa92010d1c0 con 0x7fa920102ed0 2024-09-16T00:32:50.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.713+0000 7fa924c73640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa920102530 0x7fa920113240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:50.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.713+0000 7fa924c73640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa920102530 0x7fa920113240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54554/0 (socket says 172.21.15.112:54554) 2024-09-16T00:32:50.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.713+0000 7fa924c73640 1 -- 172.21.15.112:0/1503919660 learned_addr learned my addr 172.21.15.112:0/1503919660 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:32:50.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.713+0000 7fa91ffff640 1 --2- 172.21.15.112:0/1503919660 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa920102ed0 0x7fa920113780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:50.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.713+0000 7fa924c73640 1 -- 172.21.15.112:0/1503919660 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa920102ed0 msgr2=0x7fa920113780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:50.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.713+0000 7fa924c73640 1 --2- 172.21.15.112:0/1503919660 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa920102ed0 0x7fa920113780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:50.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.713+0000 7fa924c73640 1 -- 172.21.15.112:0/1503919660 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa914005040 con 0x7fa920102530 2024-09-16T00:32:50.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.713+0000 7fa924c73640 1 --2- 172.21.15.112:0/1503919660 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa920102530 0x7fa920113240 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa914000c00 tx=0x7fa914009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:50.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.713+0000 7fa91d7fa640 1 -- 172.21.15.112:0/1503919660 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa914038070 con 0x7fa920102530 2024-09-16T00:32:50.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.713+0000 7fa91d7fa640 1 -- 172.21.15.112:0/1503919660 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fa91402ade0 con 0x7fa920102530 2024-09-16T00:32:50.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.713+0000 7fa926ed7640 1 -- 172.21.15.112:0/1503919660 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa92010d440 con 0x7fa920102530 2024-09-16T00:32:50.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.713+0000 7fa926ed7640 1 -- 172.21.15.112:0/1503919660 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa92010d980 con 0x7fa920102530 2024-09-16T00:32:50.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.713+0000 7fa91d7fa640 1 -- 172.21.15.112:0/1503919660 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa91402c7b0 con 0x7fa920102530 2024-09-16T00:32:50.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.714+0000 7fa926ed7640 1 -- 172.21.15.112:0/1503919660 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9200fdcb0 con 0x7fa920102530 2024-09-16T00:32:50.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.715+0000 7fa91d7fa640 1 -- 172.21.15.112:0/1503919660 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fa914030020 con 0x7fa920102530 2024-09-16T00:32:50.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.715+0000 7fa91d7fa640 1 --2- 172.21.15.112:0/1503919660 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa91006e530 0x7fa9100709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:50.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.715+0000 7fa91d7fa640 1 -- 172.21.15.112:0/1503919660 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fa9140ae490 con 0x7fa920102530 2024-09-16T00:32:50.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.715+0000 7fa91ffff640 1 --2- 172.21.15.112:0/1503919660 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa91006e530 0x7fa9100709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:50.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.716+0000 7fa91ffff640 1 --2- 172.21.15.112:0/1503919660 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa91006e530 0x7fa9100709e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa92010eb00 tx=0x7fa90c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:50.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.720+0000 7fa91d7fa640 1 -- 172.21.15.112:0/1503919660 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fa914077650 con 0x7fa920102530 2024-09-16T00:32:50.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.893+0000 7fa926ed7640 1 -- 172.21.15.112:0/1503919660 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa92004b720 con 0x7fa91006e530 2024-09-16T00:32:50.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.901+0000 7fa91d7fa640 1 -- 172.21.15.112:0/1503919660 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fa92004b720 con 0x7fa91006e530 2024-09-16T00:32:50.902 INFO:teuthology.orchestra.run.smithi112.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T00:32:50.902 INFO:teuthology.orchestra.run.smithi112.stdout:alertmanager.smithi112 smithi112 *:9093,9094 running (5m) 64s ago 14m 25.2M - 0.25.0 c8568f914cd2 d85ea0562022 2024-09-16T00:32:50.902 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi112 smithi112 running (2m) 64s ago 14m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c e4fe58f9702a 2024-09-16T00:32:50.903 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi169 smithi169 running (2m) 0s ago 14m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c 3ba2c81d527f 2024-09-16T00:32:50.903 INFO:teuthology.orchestra.run.smithi112.stdout:grafana.smithi112 smithi112 *:3000 running (4m) 64s ago 14m 82.2M - 9.4.12 2bacad6d85d8 51f9d2013da2 2024-09-16T00:32:50.903 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.hxfwkh smithi112 running (9m) 64s ago 9m 135M - 17.2.7-1633-g6d025bc2 33ed8dba3996 cf92d9461fc6 2024-09-16T00:32:50.903 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.ithwzj smithi112 running (9m) 64s ago 9m 66.1M - 17.2.7-1633-g6d025bc2 33ed8dba3996 02786c8a83c7 2024-09-16T00:32:50.903 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.gawwgw smithi169 running (9m) 0s ago 9m 114M - 17.2.7-1633-g6d025bc2 33ed8dba3996 dcf3a4dfa879 2024-09-16T00:32:50.904 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.lgeqrj smithi169 running (9m) 0s ago 9m 60.3M - 17.2.7-1633-g6d025bc2 33ed8dba3996 445c14e55831 2024-09-16T00:32:50.904 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi112.xrkkbc smithi112 *:8443,9283,8765 running (5m) 64s ago 15m 497M - 19.1.1-326-g8921c8ef f1455bd0219c a981f7fd2810 2024-09-16T00:32:50.904 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi169.mtrobk smithi169 *:8443,9283,8765 running (4m) 0s ago 13m 441M - 19.1.1-326-g8921c8ef f1455bd0219c 4096e5369c2d 2024-09-16T00:32:50.904 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi112 smithi112 running (3m) 64s ago 16m 46.4M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 5aaec033c042 2024-09-16T00:32:50.904 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi169 smithi169 running (3m) 0s ago 13m 38.1M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 7bc609155f3f 2024-09-16T00:32:50.905 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi112 smithi112 *:9100 running (6m) 64s ago 14m 15.5M - 1.5.0 0da6a335fe13 6404d1d601e2 2024-09-16T00:32:50.905 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi169 smithi169 *:9100 running (5m) 0s ago 13m 15.0M - 1.5.0 0da6a335fe13 5505baf702a6 2024-09-16T00:32:50.905 INFO:teuthology.orchestra.run.smithi112.stdout:osd.0 smithi112 running (2m) 64s ago 13m 89.4M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 53efc9397017 2024-09-16T00:32:50.905 INFO:teuthology.orchestra.run.smithi112.stdout:osd.1 smithi112 running (105s) 64s ago 12m 89.1M 1557M 19.1.1-326-g8921c8ef f1455bd0219c fe6f1a4a2cf4 2024-09-16T00:32:50.905 INFO:teuthology.orchestra.run.smithi112.stdout:osd.2 smithi112 running (70s) 64s ago 12m 12.1M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 5f7863712057 2024-09-16T00:32:50.905 INFO:teuthology.orchestra.run.smithi112.stdout:osd.3 smithi169 running (35s) 0s ago 11m 90.0M 2581M 19.1.1-326-g8921c8ef f1455bd0219c ee07b5577810 2024-09-16T00:32:50.906 INFO:teuthology.orchestra.run.smithi112.stdout:osd.4 smithi169 running (3s) 0s ago 11m 11.3M 2581M 19.1.1-326-g8921c8ef f1455bd0219c 769f026a54eb 2024-09-16T00:32:50.906 INFO:teuthology.orchestra.run.smithi112.stdout:osd.5 smithi169 running (10m) 0s ago 10m 199M 2581M 17.2.7-1633-g6d025bc2 33ed8dba3996 8e7d708d74eb 2024-09-16T00:32:50.906 INFO:teuthology.orchestra.run.smithi112.stdout:prometheus.smithi112 smithi112 *:9095 running (4m) 64s ago 14m 63.2M - 2.43.0 a07b618ecd1d 7a1d42e60d2b 2024-09-16T00:32:50.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.903+0000 7fa926ed7640 1 -- 172.21.15.112:0/1503919660 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa91006e530 msgr2=0x7fa9100709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:50.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.903+0000 7fa926ed7640 1 --2- 172.21.15.112:0/1503919660 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa91006e530 0x7fa9100709e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa92010eb00 tx=0x7fa90c00b040 comp rx=0 tx=0).stop 2024-09-16T00:32:50.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.903+0000 7fa926ed7640 1 -- 172.21.15.112:0/1503919660 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa920102530 msgr2=0x7fa920113240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:50.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.903+0000 7fa926ed7640 1 --2- 172.21.15.112:0/1503919660 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa920102530 0x7fa920113240 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa914000c00 tx=0x7fa914009da0 comp rx=0 tx=0).stop 2024-09-16T00:32:50.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.904+0000 7fa926ed7640 1 -- 172.21.15.112:0/1503919660 shutdown_connections 2024-09-16T00:32:50.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.904+0000 7fa926ed7640 1 --2- 172.21.15.112:0/1503919660 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa91006e530 0x7fa9100709e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:50.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.904+0000 7fa926ed7640 1 --2- 172.21.15.112:0/1503919660 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa920102ed0 0x7fa920113780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:50.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.904+0000 7fa926ed7640 1 --2- 172.21.15.112:0/1503919660 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa920102530 0x7fa920113240 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:50.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.904+0000 7fa926ed7640 1 -- 172.21.15.112:0/1503919660 >> 172.21.15.112:0/1503919660 conn(0x7fa9200f80f0 msgr2=0x7fa9200f90f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:50.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.904+0000 7fa926ed7640 1 -- 172.21.15.112:0/1503919660 shutdown_connections 2024-09-16T00:32:50.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.904+0000 7fa926ed7640 1 -- 172.21.15.112:0/1503919660 wait complete. 2024-09-16T00:32:50.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.981+0000 7f02fe104640 1 -- 172.21.15.112:0/3774474982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02f80fdf30 msgr2=0x7f02f8106570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:50.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.981+0000 7f02fe104640 1 --2- 172.21.15.112:0/3774474982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02f80fdf30 0x7f02f8106570 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f02e00048d0 tx=0x7f02e0009da0 comp rx=0 tx=0).stop 2024-09-16T00:32:50.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.981+0000 7f02fe104640 1 -- 172.21.15.112:0/3774474982 shutdown_connections 2024-09-16T00:32:50.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.981+0000 7f02fe104640 1 --2- 172.21.15.112:0/3774474982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02f80fdf30 0x7f02f8106570 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:50.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.981+0000 7f02fe104640 1 --2- 172.21.15.112:0/3774474982 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02f80fd590 0x7f02f80fd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:50.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.981+0000 7f02fe104640 1 -- 172.21.15.112:0/3774474982 >> 172.21.15.112:0/3774474982 conn(0x7f02f80f7060 msgr2=0x7f02f80f9470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:50.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.982+0000 7f02fe104640 1 -- 172.21.15.112:0/3774474982 shutdown_connections 2024-09-16T00:32:50.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.982+0000 7f02fe104640 1 -- 172.21.15.112:0/3774474982 wait complete. 2024-09-16T00:32:50.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.982+0000 7f02fe104640 1 Processor -- start 2024-09-16T00:32:50.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.982+0000 7f02fe104640 1 -- start start 2024-09-16T00:32:50.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.983+0000 7f02fe104640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02f80fd590 0x7f02f80fb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:50.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.983+0000 7f02fe104640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02f80fdf30 0x7f02f80fb900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:50.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.983+0000 7f02fe104640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02f80fbe40 con 0x7f02f80fd590 2024-09-16T00:32:50.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.983+0000 7f02fe104640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02f80fbfb0 con 0x7f02f80fdf30 2024-09-16T00:32:50.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.983+0000 7f02f77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02f80fd590 0x7f02f80fb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:50.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.983+0000 7f02f77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02f80fd590 0x7f02f80fb3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54576/0 (socket says 172.21.15.112:54576) 2024-09-16T00:32:50.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.983+0000 7f02f77fe640 1 -- 172.21.15.112:0/1244560620 learned_addr learned my addr 172.21.15.112:0/1244560620 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:32:50.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.983+0000 7f02eeffd640 1 --2- 172.21.15.112:0/1244560620 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02f80fdf30 0x7f02f80fb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:50.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.983+0000 7f02f77fe640 1 -- 172.21.15.112:0/1244560620 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02f80fdf30 msgr2=0x7f02f80fb900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:50.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.983+0000 7f02f77fe640 1 --2- 172.21.15.112:0/1244560620 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02f80fdf30 0x7f02f80fb900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:50.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.983+0000 7f02f77fe640 1 -- 172.21.15.112:0/1244560620 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02e0005040 con 0x7f02f80fd590 2024-09-16T00:32:50.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.983+0000 7f02f77fe640 1 --2- 172.21.15.112:0/1244560620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02f80fd590 0x7f02f80fb3c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f02e800eb70 tx=0x7f02e800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:50.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.983+0000 7f02f4ff9640 1 -- 172.21.15.112:0/1244560620 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f02e8007c30 con 0x7f02f80fd590 2024-09-16T00:32:50.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.983+0000 7f02fe104640 1 -- 172.21.15.112:0/1244560620 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02f80fc1e0 con 0x7f02f80fd590 2024-09-16T00:32:50.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.983+0000 7f02f4ff9640 1 -- 172.21.15.112:0/1244560620 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f02e800f040 con 0x7f02f80fd590 2024-09-16T00:32:50.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.983+0000 7f02f4ff9640 1 -- 172.21.15.112:0/1244560620 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f02e800ca60 con 0x7f02f80fd590 2024-09-16T00:32:50.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.984+0000 7f02fe104640 1 -- 172.21.15.112:0/1244560620 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02f81198b0 con 0x7f02f80fd590 2024-09-16T00:32:50.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.985+0000 7f02fe104640 1 -- 172.21.15.112:0/1244560620 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02f8066350 con 0x7f02f80fd590 2024-09-16T00:32:50.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.985+0000 7f02f4ff9640 1 -- 172.21.15.112:0/1244560620 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f02e800cbc0 con 0x7f02f80fd590 2024-09-16T00:32:50.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.986+0000 7f02f4ff9640 1 --2- 172.21.15.112:0/1244560620 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f02e406e530 0x7f02e40709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:50.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.986+0000 7f02f4ff9640 1 -- 172.21.15.112:0/1244560620 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f02e808dd30 con 0x7f02f80fd590 2024-09-16T00:32:50.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.986+0000 7f02eeffd640 1 --2- 172.21.15.112:0/1244560620 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f02e406e530 0x7f02e40709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:50.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.986+0000 7f02eeffd640 1 --2- 172.21.15.112:0/1244560620 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f02e406e530 0x7f02e40709e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f02f80fcd00 tx=0x7f02e0035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:50.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:50.990+0000 7f02f4ff9640 1 -- 172.21.15.112:0/1244560620 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f02e8056f20 con 0x7f02f80fd590 2024-09-16T00:32:51.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.224+0000 7f02fe104640 1 -- 172.21.15.112:0/1244560620 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f02f80fac60 con 0x7f02f80fd590 2024-09-16T00:32:51.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.224+0000 7f02f4ff9640 1 -- 172.21.15.112:0/1244560620 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+850 (secure 0 0 0) 0x7f02e805a5e0 con 0x7f02f80fd590 2024-09-16T00:32:51.225 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:32:51.225 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": { 2024-09-16T00:32:51.225 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:32:51.225 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:32:51.225 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": { 2024-09-16T00:32:51.225 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:32:51.225 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:32:51.225 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": { 2024-09-16T00:32:51.225 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 17.2.7-1633-g6d025bc2 (6d025bc26f3176c910707f8cb4c8d8e076640615) quincy (stable)": 1, 2024-09-16T00:32:51.226 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 4 2024-09-16T00:32:51.226 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:32:51.226 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": { 2024-09-16T00:32:51.226 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 17.2.7-1633-g6d025bc2 (6d025bc26f3176c910707f8cb4c8d8e076640615) quincy (stable)": 4 2024-09-16T00:32:51.226 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:32:51.226 INFO:teuthology.orchestra.run.smithi112.stdout: "overall": { 2024-09-16T00:32:51.226 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 17.2.7-1633-g6d025bc2 (6d025bc26f3176c910707f8cb4c8d8e076640615) quincy (stable)": 5, 2024-09-16T00:32:51.226 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 8 2024-09-16T00:32:51.226 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-16T00:32:51.226 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:32:51.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.227+0000 7f02fe104640 1 -- 172.21.15.112:0/1244560620 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f02e406e530 msgr2=0x7f02e40709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:51.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.227+0000 7f02fe104640 1 --2- 172.21.15.112:0/1244560620 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f02e406e530 0x7f02e40709e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f02f80fcd00 tx=0x7f02e0035040 comp rx=0 tx=0).stop 2024-09-16T00:32:51.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.227+0000 7f02fe104640 1 -- 172.21.15.112:0/1244560620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02f80fd590 msgr2=0x7f02f80fb3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:51.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.227+0000 7f02fe104640 1 --2- 172.21.15.112:0/1244560620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02f80fd590 0x7f02f80fb3c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f02e800eb70 tx=0x7f02e800b040 comp rx=0 tx=0).stop 2024-09-16T00:32:51.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.227+0000 7f02fe104640 1 -- 172.21.15.112:0/1244560620 shutdown_connections 2024-09-16T00:32:51.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.227+0000 7f02fe104640 1 --2- 172.21.15.112:0/1244560620 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f02e406e530 0x7f02e40709e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:51.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.227+0000 7f02fe104640 1 --2- 172.21.15.112:0/1244560620 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02f80fdf30 0x7f02f80fb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:51.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.227+0000 7f02fe104640 1 --2- 172.21.15.112:0/1244560620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02f80fd590 0x7f02f80fb3c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:51.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.227+0000 7f02fe104640 1 -- 172.21.15.112:0/1244560620 >> 172.21.15.112:0/1244560620 conn(0x7f02f80f7060 msgr2=0x7f02f80f8580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:51.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.227+0000 7f02fe104640 1 -- 172.21.15.112:0/1244560620 shutdown_connections 2024-09-16T00:32:51.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.227+0000 7f02fe104640 1 -- 172.21.15.112:0/1244560620 wait complete. 2024-09-16T00:32:51.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.305+0000 7f899adb1640 1 -- 172.21.15.112:0/1996358590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89941014e0 msgr2=0x7f89941018b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:51.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.305+0000 7f899adb1640 1 --2- 172.21.15.112:0/1996358590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89941014e0 0x7f89941018b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f89880048d0 tx=0x7f8988009da0 comp rx=0 tx=0).stop 2024-09-16T00:32:51.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.305+0000 7f899adb1640 1 -- 172.21.15.112:0/1996358590 shutdown_connections 2024-09-16T00:32:51.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.305+0000 7f899adb1640 1 --2- 172.21.15.112:0/1996358590 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89940fb4a0 0x7f89940fb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:51.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.305+0000 7f899adb1640 1 --2- 172.21.15.112:0/1996358590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89941014e0 0x7f89941018b0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:51.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.305+0000 7f899adb1640 1 -- 172.21.15.112:0/1996358590 >> 172.21.15.112:0/1996358590 conn(0x7f89940f7060 msgr2=0x7f89940f9470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:51.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.305+0000 7f899adb1640 1 -- 172.21.15.112:0/1996358590 shutdown_connections 2024-09-16T00:32:51.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.305+0000 7f899adb1640 1 -- 172.21.15.112:0/1996358590 wait complete. 2024-09-16T00:32:51.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.306+0000 7f899adb1640 1 Processor -- start 2024-09-16T00:32:51.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.306+0000 7f899adb1640 1 -- start start 2024-09-16T00:32:51.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.306+0000 7f899adb1640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89940fb4a0 0x7f899410e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:51.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.306+0000 7f899adb1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89941014e0 0x7f899410e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:51.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.306+0000 7f899adb1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f899410ed10 con 0x7f89941014e0 2024-09-16T00:32:51.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.306+0000 7f899adb1640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f899410ee80 con 0x7f89940fb4a0 2024-09-16T00:32:51.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.306+0000 7f8993fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89941014e0 0x7f899410e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:51.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.306+0000 7f8993fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89941014e0 0x7f899410e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54590/0 (socket says 172.21.15.112:54590) 2024-09-16T00:32:51.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.306+0000 7f8993fff640 1 -- 172.21.15.112:0/123950654 learned_addr learned my addr 172.21.15.112:0/123950654 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:32:51.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.306+0000 7f8998b4d640 1 --2- 172.21.15.112:0/123950654 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89940fb4a0 0x7f899410e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:51.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.306+0000 7f8993fff640 1 -- 172.21.15.112:0/123950654 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89940fb4a0 msgr2=0x7f899410e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:51.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.306+0000 7f8993fff640 1 --2- 172.21.15.112:0/123950654 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89940fb4a0 0x7f899410e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:51.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.306+0000 7f8993fff640 1 -- 172.21.15.112:0/123950654 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8988005040 con 0x7f89941014e0 2024-09-16T00:32:51.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.307+0000 7f8993fff640 1 --2- 172.21.15.112:0/123950654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89941014e0 0x7f899410e6c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8980002a50 tx=0x7f8980004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:51.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.307+0000 7f89917fa640 1 -- 172.21.15.112:0/123950654 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8980007c30 con 0x7f89941014e0 2024-09-16T00:32:51.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.307+0000 7f899adb1640 1 -- 172.21.15.112:0/123950654 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89941171f0 con 0x7f89941014e0 2024-09-16T00:32:51.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.307+0000 7f89917fa640 1 -- 172.21.15.112:0/123950654 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f898000f040 con 0x7f89941014e0 2024-09-16T00:32:51.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.307+0000 7f89917fa640 1 -- 172.21.15.112:0/123950654 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f898000da10 con 0x7f89941014e0 2024-09-16T00:32:51.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.307+0000 7f899adb1640 1 -- 172.21.15.112:0/123950654 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8994117700 con 0x7f89941014e0 2024-09-16T00:32:51.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.308+0000 7f899adb1640 1 -- 172.21.15.112:0/123950654 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89940fcbe0 con 0x7f89941014e0 2024-09-16T00:32:51.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.309+0000 7f89917fa640 1 -- 172.21.15.112:0/123950654 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f898000dc80 con 0x7f89941014e0 2024-09-16T00:32:51.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.309+0000 7f89917fa640 1 --2- 172.21.15.112:0/123950654 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f898406e530 0x7f89840709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:51.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.309+0000 7f89917fa640 1 -- 172.21.15.112:0/123950654 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f898008db20 con 0x7f89941014e0 2024-09-16T00:32:51.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.309+0000 7f8998b4d640 1 --2- 172.21.15.112:0/123950654 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f898406e530 0x7f89840709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:51.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.310+0000 7f8998b4d640 1 --2- 172.21.15.112:0/123950654 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f898406e530 0x7f89840709e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8988009ac0 tx=0x7f8988035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:51.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.313+0000 7f89917fa640 1 -- 172.21.15.112:0/123950654 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f8980056cf0 con 0x7f89941014e0 2024-09-16T00:32:51.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.513+0000 7f899adb1640 1 -- 172.21.15.112:0/123950654 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f8994108510 con 0x7f89941014e0 2024-09-16T00:32:51.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.513+0000 7f89917fa640 1 -- 172.21.15.112:0/123950654 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) v1 ==== 76+0+1954 (secure 0 0 0) 0x7f898005a3b0 con 0x7f89941014e0 2024-09-16T00:32:51.514 INFO:teuthology.orchestra.run.smithi112.stdout:e23 2024-09-16T00:32:51.514 INFO:teuthology.orchestra.run.smithi112.stdout:btime 2024-09-16T00:30:08:788506+0000 2024-09-16T00:32:51.514 INFO:teuthology.orchestra.run.smithi112.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T00:32:51.514 INFO:teuthology.orchestra.run.smithi112.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:32:51.515 INFO:teuthology.orchestra.run.smithi112.stdout:legacy client fscid: 1 2024-09-16T00:32:51.515 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:32:51.515 INFO:teuthology.orchestra.run.smithi112.stdout:Filesystem 'cephfs' (1) 2024-09-16T00:32:51.515 INFO:teuthology.orchestra.run.smithi112.stdout:fs_name cephfs 2024-09-16T00:32:51.515 INFO:teuthology.orchestra.run.smithi112.stdout:epoch 23 2024-09-16T00:32:51.515 INFO:teuthology.orchestra.run.smithi112.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-16T00:32:51.515 INFO:teuthology.orchestra.run.smithi112.stdout:created 2024-09-16T00:23:22.328199+0000 2024-09-16T00:32:51.515 INFO:teuthology.orchestra.run.smithi112.stdout:modified 2024-09-16T00:30:08.396661+0000 2024-09-16T00:32:51.515 INFO:teuthology.orchestra.run.smithi112.stdout:tableserver 0 2024-09-16T00:32:51.515 INFO:teuthology.orchestra.run.smithi112.stdout:root 0 2024-09-16T00:32:51.515 INFO:teuthology.orchestra.run.smithi112.stdout:session_timeout 60 2024-09-16T00:32:51.516 INFO:teuthology.orchestra.run.smithi112.stdout:session_autoclose 300 2024-09-16T00:32:51.516 INFO:teuthology.orchestra.run.smithi112.stdout:max_file_size 1099511627776 2024-09-16T00:32:51.516 INFO:teuthology.orchestra.run.smithi112.stdout:max_xattr_size 65536 2024-09-16T00:32:51.516 INFO:teuthology.orchestra.run.smithi112.stdout:required_client_features {} 2024-09-16T00:32:51.516 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure 0 2024-09-16T00:32:51.516 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure_osd_epoch 44 2024-09-16T00:32:51.516 INFO:teuthology.orchestra.run.smithi112.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:32:51.516 INFO:teuthology.orchestra.run.smithi112.stdout:max_mds 2 2024-09-16T00:32:51.516 INFO:teuthology.orchestra.run.smithi112.stdout:in 0,1 2024-09-16T00:32:51.516 INFO:teuthology.orchestra.run.smithi112.stdout:up {0=14480,1=24259} 2024-09-16T00:32:51.516 INFO:teuthology.orchestra.run.smithi112.stdout:failed 2024-09-16T00:32:51.517 INFO:teuthology.orchestra.run.smithi112.stdout:damaged 2024-09-16T00:32:51.517 INFO:teuthology.orchestra.run.smithi112.stdout:stopped 2024-09-16T00:32:51.517 INFO:teuthology.orchestra.run.smithi112.stdout:data_pools [3] 2024-09-16T00:32:51.517 INFO:teuthology.orchestra.run.smithi112.stdout:metadata_pool 2 2024-09-16T00:32:51.517 INFO:teuthology.orchestra.run.smithi112.stdout:inline_data enabled 2024-09-16T00:32:51.517 INFO:teuthology.orchestra.run.smithi112.stdout:balancer 2024-09-16T00:32:51.517 INFO:teuthology.orchestra.run.smithi112.stdout:bal_rank_mask -1 2024-09-16T00:32:51.517 INFO:teuthology.orchestra.run.smithi112.stdout:standby_count_wanted 1 2024-09-16T00:32:51.517 INFO:teuthology.orchestra.run.smithi112.stdout:qdb_cluster leader: 14480 members: 24259,14480 2024-09-16T00:32:51.517 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.hxfwkh{0:14480} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.112:6828/486654885,v1:172.21.15.112:6829/486654885] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T00:32:51.517 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.ithwzj{0:14492} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.112:6826/381830565,v1:172.21.15.112:6827/381830565] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T00:32:51.517 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.gawwgw{1:24259} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.169:6824/2892476953,v1:172.21.15.169:6825/2892476953] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T00:32:51.517 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.lgeqrj{1:24269} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.169:6826/861725728,v1:172.21.15.169:6827/861725728] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T00:32:51.518 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:32:51.518 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:32:51.518 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 23 2024-09-16T00:32:51.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.516+0000 7f899adb1640 1 -- 172.21.15.112:0/123950654 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f898406e530 msgr2=0x7f89840709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:51.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.516+0000 7f899adb1640 1 --2- 172.21.15.112:0/123950654 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f898406e530 0x7f89840709e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8988009ac0 tx=0x7f8988035040 comp rx=0 tx=0).stop 2024-09-16T00:32:51.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.516+0000 7f899adb1640 1 -- 172.21.15.112:0/123950654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89941014e0 msgr2=0x7f899410e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:51.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.516+0000 7f899adb1640 1 --2- 172.21.15.112:0/123950654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89941014e0 0x7f899410e6c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8980002a50 tx=0x7f8980004600 comp rx=0 tx=0).stop 2024-09-16T00:32:51.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.516+0000 7f899adb1640 1 -- 172.21.15.112:0/123950654 shutdown_connections 2024-09-16T00:32:51.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.516+0000 7f899adb1640 1 --2- 172.21.15.112:0/123950654 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f898406e530 0x7f89840709e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:51.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.516+0000 7f899adb1640 1 --2- 172.21.15.112:0/123950654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89941014e0 0x7f899410e6c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:51.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.516+0000 7f899adb1640 1 --2- 172.21.15.112:0/123950654 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89940fb4a0 0x7f899410e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:51.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.516+0000 7f899adb1640 1 -- 172.21.15.112:0/123950654 >> 172.21.15.112:0/123950654 conn(0x7f89940f7060 msgr2=0x7f89940f7430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:51.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.516+0000 7f899adb1640 1 -- 172.21.15.112:0/123950654 shutdown_connections 2024-09-16T00:32:51.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.516+0000 7f899adb1640 1 -- 172.21.15.112:0/123950654 wait complete. 2024-09-16T00:32:51.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.596+0000 7f41fbc77640 1 -- 172.21.15.112:0/2305320785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41f4064fc0 msgr2=0x7f41f4065430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:51.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.596+0000 7f41fbc77640 1 --2- 172.21.15.112:0/2305320785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41f4064fc0 0x7f41f4065430 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f41e40048d0 tx=0x7f41e4009da0 comp rx=0 tx=0).stop 2024-09-16T00:32:51.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.596+0000 7f41fbc77640 1 -- 172.21.15.112:0/2305320785 shutdown_connections 2024-09-16T00:32:51.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.596+0000 7f41fbc77640 1 --2- 172.21.15.112:0/2305320785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41f4064fc0 0x7f41f4065430 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:51.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.596+0000 7f41fbc77640 1 --2- 172.21.15.112:0/2305320785 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f41f40fe460 0x7f41f40fe830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:51.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.596+0000 7f41fbc77640 1 -- 172.21.15.112:0/2305320785 >> 172.21.15.112:0/2305320785 conn(0x7f41f40723b0 msgr2=0x7f41f40727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:51.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.596+0000 7f41fbc77640 1 -- 172.21.15.112:0/2305320785 shutdown_connections 2024-09-16T00:32:51.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.596+0000 7f41fbc77640 1 -- 172.21.15.112:0/2305320785 wait complete. 2024-09-16T00:32:51.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.596+0000 7f41fbc77640 1 Processor -- start 2024-09-16T00:32:51.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.596+0000 7f41fbc77640 1 -- start start 2024-09-16T00:32:51.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.597+0000 7f41fbc77640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41f4064fc0 0x7f41f41131d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:51.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.597+0000 7f41fbc77640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f41f40fe460 0x7f41f4113710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:51.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.597+0000 7f41fbc77640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f4113d10 con 0x7f41f4064fc0 2024-09-16T00:32:51.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.597+0000 7f41fbc77640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f410d1a0 con 0x7f41f40fe460 2024-09-16T00:32:51.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.597+0000 7f41f9a13640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41f4064fc0 0x7f41f41131d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:51.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.597+0000 7f41f9a13640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41f4064fc0 0x7f41f41131d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54608/0 (socket says 172.21.15.112:54608) 2024-09-16T00:32:51.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.597+0000 7f41f9a13640 1 -- 172.21.15.112:0/2579500190 learned_addr learned my addr 172.21.15.112:0/2579500190 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:32:51.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.597+0000 7f41f9212640 1 --2- 172.21.15.112:0/2579500190 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f41f40fe460 0x7f41f4113710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:51.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.597+0000 7f41f9a13640 1 -- 172.21.15.112:0/2579500190 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f41f40fe460 msgr2=0x7f41f4113710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:51.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.597+0000 7f41f9a13640 1 --2- 172.21.15.112:0/2579500190 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f41f40fe460 0x7f41f4113710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:51.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.597+0000 7f41f9a13640 1 -- 172.21.15.112:0/2579500190 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41e4005040 con 0x7f41f4064fc0 2024-09-16T00:32:51.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.597+0000 7f41f9a13640 1 --2- 172.21.15.112:0/2579500190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41f4064fc0 0x7f41f41131d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f41f000eb70 tx=0x7f41f000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:51.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.597+0000 7f41ea7fc640 1 -- 172.21.15.112:0/2579500190 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41f0007c30 con 0x7f41f4064fc0 2024-09-16T00:32:51.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.597+0000 7f41fbc77640 1 -- 172.21.15.112:0/2579500190 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41f410d420 con 0x7f41f4064fc0 2024-09-16T00:32:51.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.597+0000 7f41ea7fc640 1 -- 172.21.15.112:0/2579500190 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f41f000f040 con 0x7f41f4064fc0 2024-09-16T00:32:51.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.597+0000 7f41ea7fc640 1 -- 172.21.15.112:0/2579500190 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41f000ca60 con 0x7f41f4064fc0 2024-09-16T00:32:51.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.597+0000 7f41fbc77640 1 -- 172.21.15.112:0/2579500190 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41f410d990 con 0x7f41f4064fc0 2024-09-16T00:32:51.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.598+0000 7f41fbc77640 1 -- 172.21.15.112:0/2579500190 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41f40fdcf0 con 0x7f41f4064fc0 2024-09-16T00:32:51.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.599+0000 7f41ea7fc640 1 -- 172.21.15.112:0/2579500190 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f41f000cbc0 con 0x7f41f4064fc0 2024-09-16T00:32:51.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.599+0000 7f41ea7fc640 1 --2- 172.21.15.112:0/2579500190 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f41e006e530 0x7f41e00709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:51.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.599+0000 7f41ea7fc640 1 -- 172.21.15.112:0/2579500190 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f41f008dd30 con 0x7f41f4064fc0 2024-09-16T00:32:51.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.600+0000 7f41f9212640 1 --2- 172.21.15.112:0/2579500190 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f41e006e530 0x7f41e00709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:51.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.600+0000 7f41f9212640 1 --2- 172.21.15.112:0/2579500190 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f41e006e530 0x7f41e00709e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f41f4072d60 tx=0x7f41e4035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:51.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.604+0000 7f41ea7fc640 1 -- 172.21.15.112:0/2579500190 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f41f0056f20 con 0x7f41f4064fc0 2024-09-16T00:32:51.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.793+0000 7f41fbc77640 1 -- 172.21.15.112:0/2579500190 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f41f404b750 con 0x7f41e006e530 2024-09-16T00:32:51.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.794+0000 7f41ea7fc640 1 -- 172.21.15.112:0/2579500190 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f41f404b750 con 0x7f41e006e530 2024-09-16T00:32:51.795 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:32:51.795 INFO:teuthology.orchestra.run.smithi112.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80", 2024-09-16T00:32:51.795 INFO:teuthology.orchestra.run.smithi112.stdout: "in_progress": true, 2024-09-16T00:32:51.795 INFO:teuthology.orchestra.run.smithi112.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T00:32:51.795 INFO:teuthology.orchestra.run.smithi112.stdout: "services_complete": [ 2024-09-16T00:32:51.795 INFO:teuthology.orchestra.run.smithi112.stdout: "crash", 2024-09-16T00:32:51.795 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr", 2024-09-16T00:32:51.795 INFO:teuthology.orchestra.run.smithi112.stdout: "mon" 2024-09-16T00:32:51.795 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-16T00:32:51.795 INFO:teuthology.orchestra.run.smithi112.stdout: "progress": "11/21 daemons upgraded", 2024-09-16T00:32:51.795 INFO:teuthology.orchestra.run.smithi112.stdout: "message": "Currently upgrading osd daemons", 2024-09-16T00:32:51.796 INFO:teuthology.orchestra.run.smithi112.stdout: "is_paused": false 2024-09-16T00:32:51.796 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:32:51.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.796+0000 7f41fbc77640 1 -- 172.21.15.112:0/2579500190 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f41e006e530 msgr2=0x7f41e00709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:51.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.796+0000 7f41fbc77640 1 --2- 172.21.15.112:0/2579500190 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f41e006e530 0x7f41e00709e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f41f4072d60 tx=0x7f41e4035040 comp rx=0 tx=0).stop 2024-09-16T00:32:51.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.796+0000 7f41fbc77640 1 -- 172.21.15.112:0/2579500190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41f4064fc0 msgr2=0x7f41f41131d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:51.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.796+0000 7f41fbc77640 1 --2- 172.21.15.112:0/2579500190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41f4064fc0 0x7f41f41131d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f41f000eb70 tx=0x7f41f000b040 comp rx=0 tx=0).stop 2024-09-16T00:32:51.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.796+0000 7f41fbc77640 1 -- 172.21.15.112:0/2579500190 shutdown_connections 2024-09-16T00:32:51.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.796+0000 7f41fbc77640 1 --2- 172.21.15.112:0/2579500190 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f41e006e530 0x7f41e00709e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:51.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.796+0000 7f41fbc77640 1 --2- 172.21.15.112:0/2579500190 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f41f40fe460 0x7f41f4113710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:51.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.796+0000 7f41fbc77640 1 --2- 172.21.15.112:0/2579500190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41f4064fc0 0x7f41f41131d0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:51.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.796+0000 7f41fbc77640 1 -- 172.21.15.112:0/2579500190 >> 172.21.15.112:0/2579500190 conn(0x7f41f40723b0 msgr2=0x7f41f41053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:51.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.797+0000 7f41fbc77640 1 -- 172.21.15.112:0/2579500190 shutdown_connections 2024-09-16T00:32:51.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.797+0000 7f41fbc77640 1 -- 172.21.15.112:0/2579500190 wait complete. 2024-09-16T00:32:51.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.874+0000 7f126272a640 1 -- 172.21.15.112:0/3873843096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f125c1047c0 msgr2=0x7f125c104b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:51.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.874+0000 7f126272a640 1 --2- 172.21.15.112:0/3873843096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f125c1047c0 0x7f125c104b90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f124c0048d0 tx=0x7f124c009da0 comp rx=0 tx=0).stop 2024-09-16T00:32:51.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.874+0000 7f126272a640 1 -- 172.21.15.112:0/3873843096 shutdown_connections 2024-09-16T00:32:51.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.874+0000 7f126272a640 1 --2- 172.21.15.112:0/3873843096 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f125c0fe7a0 0x7f125c0fec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:51.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.874+0000 7f126272a640 1 --2- 172.21.15.112:0/3873843096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f125c1047c0 0x7f125c104b90 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:51.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.874+0000 7f126272a640 1 -- 172.21.15.112:0/3873843096 >> 172.21.15.112:0/3873843096 conn(0x7f125c0fa340 msgr2=0x7f125c0fc750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:51.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.874+0000 7f126272a640 1 -- 172.21.15.112:0/3873843096 shutdown_connections 2024-09-16T00:32:51.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.874+0000 7f126272a640 1 -- 172.21.15.112:0/3873843096 wait complete. 2024-09-16T00:32:51.876 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:51 smithi112 ceph-mon[97750]: from='client.34316 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:32:51.876 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:51 smithi112 ceph-mon[97750]: pgmap v132: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 325 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 64/372 objects degraded (17.204%) 2024-09-16T00:32:51.876 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:51 smithi112 ceph-mon[97750]: from='client.34320 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:32:51.876 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:51 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:32:51.876 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:51 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:32:51.876 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:51 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/1244560620' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:32:51.876 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:51 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:32:51.877 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:51 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:32:51.877 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:51 smithi112 ceph-mon[97750]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-16T00:32:51.877 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:51 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/123950654' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:32:51.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.875+0000 7f126272a640 1 Processor -- start 2024-09-16T00:32:51.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.875+0000 7f126272a640 1 -- start start 2024-09-16T00:32:51.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.876+0000 7f126272a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f125c0fe7a0 0x7f125c1153d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:51.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.876+0000 7f126272a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f125c1047c0 0x7f125c115910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:51.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.876+0000 7f126272a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f125c115fa0 con 0x7f125c1047c0 2024-09-16T00:32:51.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.876+0000 7f126272a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f125c10f350 con 0x7f125c0fe7a0 2024-09-16T00:32:51.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.876+0000 7f125b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f125c1047c0 0x7f125c115910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:51.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.876+0000 7f125b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f125c1047c0 0x7f125c115910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54636/0 (socket says 172.21.15.112:54636) 2024-09-16T00:32:51.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.876+0000 7f125bfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f125c0fe7a0 0x7f125c1153d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:51.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.876+0000 7f125b7fe640 1 -- 172.21.15.112:0/755925424 learned_addr learned my addr 172.21.15.112:0/755925424 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:32:51.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.876+0000 7f125b7fe640 1 -- 172.21.15.112:0/755925424 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f125c0fe7a0 msgr2=0x7f125c1153d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:51.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.876+0000 7f125b7fe640 1 --2- 172.21.15.112:0/755925424 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f125c0fe7a0 0x7f125c1153d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:51.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.876+0000 7f125b7fe640 1 -- 172.21.15.112:0/755925424 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f124c005040 con 0x7f125c1047c0 2024-09-16T00:32:51.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.876+0000 7f125b7fe640 1 --2- 172.21.15.112:0/755925424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f125c1047c0 0x7f125c115910 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f125000b790 tx=0x7f12500091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:51.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.876+0000 7f1258ff9640 1 -- 172.21.15.112:0/755925424 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f125000c700 con 0x7f125c1047c0 2024-09-16T00:32:51.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.876+0000 7f126272a640 1 -- 172.21.15.112:0/755925424 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f125c10f630 con 0x7f125c1047c0 2024-09-16T00:32:51.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.876+0000 7f1258ff9640 1 -- 172.21.15.112:0/755925424 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f125000cd00 con 0x7f125c1047c0 2024-09-16T00:32:51.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.876+0000 7f1258ff9640 1 -- 172.21.15.112:0/755925424 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1250015e80 con 0x7f125c1047c0 2024-09-16T00:32:51.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.876+0000 7f126272a640 1 -- 172.21.15.112:0/755925424 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f125c10fb50 con 0x7f125c1047c0 2024-09-16T00:32:51.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.877+0000 7f126272a640 1 -- 172.21.15.112:0/755925424 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f125c0ffee0 con 0x7f125c1047c0 2024-09-16T00:32:51.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.878+0000 7f1258ff9640 1 -- 172.21.15.112:0/755925424 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f1250012070 con 0x7f125c1047c0 2024-09-16T00:32:51.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.879+0000 7f1258ff9640 1 --2- 172.21.15.112:0/755925424 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f124406e530 0x7f12440709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:51.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.879+0000 7f1258ff9640 1 -- 172.21.15.112:0/755925424 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f125008da40 con 0x7f125c1047c0 2024-09-16T00:32:51.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.879+0000 7f125bfff640 1 --2- 172.21.15.112:0/755925424 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f124406e530 0x7f12440709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:51.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.879+0000 7f125bfff640 1 --2- 172.21.15.112:0/755925424 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f124406e530 0x7f12440709e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f124c009ac0 tx=0x7f124c035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:51.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:51.884+0000 7f1258ff9640 1 -- 172.21.15.112:0/755925424 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f1250056b80 con 0x7f125c1047c0 2024-09-16T00:32:51.984 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:51 smithi169 ceph-mon[77626]: from='client.34316 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:32:51.984 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:51 smithi169 ceph-mon[77626]: pgmap v132: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 325 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 64/372 objects degraded (17.204%) 2024-09-16T00:32:51.984 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:51 smithi169 ceph-mon[77626]: from='client.34320 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:32:51.984 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:51 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:32:51.984 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:51 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:32:51.984 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:51 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/1244560620' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:32:51.984 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:51 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:32:51.985 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:51 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:32:51.985 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:51 smithi169 ceph-mon[77626]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-16T00:32:51.985 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:51 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/123950654' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:32:52.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:52.113+0000 7f126272a640 1 -- 172.21.15.112:0/755925424 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f125c0fdf60 con 0x7f125c1047c0 2024-09-16T00:32:52.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:52.114+0000 7f1258ff9640 1 -- 172.21.15.112:0/755925424 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1352 (secure 0 0 0) 0x7f125005a240 con 0x7f125c1047c0 2024-09-16T00:32:52.114 INFO:teuthology.orchestra.run.smithi112.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; 1 osds down; Degraded data redundancy: 64/372 objects degraded (17.204%), 16 pgs degraded 2024-09-16T00:32:52.115 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T00:32:52.115 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T00:32:52.115 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] OSD_DOWN: 1 osds down 2024-09-16T00:32:52.115 INFO:teuthology.orchestra.run.smithi112.stdout: osd.4 (root=default,host=smithi169) is down 2024-09-16T00:32:52.115 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 64/372 objects degraded (17.204%), 16 pgs degraded 2024-09-16T00:32:52.115 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.4 is active+undersized+degraded, acting [1,0] 2024-09-16T00:32:52.115 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.5 is active+undersized+degraded, acting [3,0] 2024-09-16T00:32:52.115 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.6 is active+undersized+degraded, acting [1,3] 2024-09-16T00:32:52.115 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.7 is active+undersized+degraded, acting [3,2] 2024-09-16T00:32:52.115 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.9 is active+undersized+degraded, acting [1,0] 2024-09-16T00:32:52.115 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.a is active+undersized+degraded, acting [1,3] 2024-09-16T00:32:52.115 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.b is active+undersized+degraded, acting [3,5] 2024-09-16T00:32:52.115 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.f is active+undersized+degraded, acting [0,5] 2024-09-16T00:32:52.115 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.11 is active+undersized+degraded, acting [3,1] 2024-09-16T00:32:52.116 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.13 is active+undersized+degraded, acting [0,2] 2024-09-16T00:32:52.116 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.14 is active+undersized+degraded, acting [3,5] 2024-09-16T00:32:52.116 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.18 is active+undersized+degraded, acting [5,3] 2024-09-16T00:32:52.116 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.19 is active+undersized+degraded, acting [0,2] 2024-09-16T00:32:52.116 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.1a is active+undersized+degraded, acting [3,5] 2024-09-16T00:32:52.116 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.1c is active+undersized+degraded, acting [5,2] 2024-09-16T00:32:52.116 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.1f is active+undersized+degraded, acting [0,3] 2024-09-16T00:32:52.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:52.116+0000 7f126272a640 1 -- 172.21.15.112:0/755925424 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f124406e530 msgr2=0x7f12440709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:52.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:52.116+0000 7f126272a640 1 --2- 172.21.15.112:0/755925424 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f124406e530 0x7f12440709e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f124c009ac0 tx=0x7f124c035040 comp rx=0 tx=0).stop 2024-09-16T00:32:52.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:52.116+0000 7f126272a640 1 -- 172.21.15.112:0/755925424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f125c1047c0 msgr2=0x7f125c115910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:52.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:52.116+0000 7f126272a640 1 --2- 172.21.15.112:0/755925424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f125c1047c0 0x7f125c115910 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f125000b790 tx=0x7f12500091d0 comp rx=0 tx=0).stop 2024-09-16T00:32:52.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:52.116+0000 7f126272a640 1 -- 172.21.15.112:0/755925424 shutdown_connections 2024-09-16T00:32:52.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:52.116+0000 7f126272a640 1 --2- 172.21.15.112:0/755925424 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f124406e530 0x7f12440709e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:52.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:52.116+0000 7f126272a640 1 --2- 172.21.15.112:0/755925424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f125c1047c0 0x7f125c115910 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:52.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:52.116+0000 7f126272a640 1 --2- 172.21.15.112:0/755925424 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f125c0fe7a0 0x7f125c1153d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:52.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:52.116+0000 7f126272a640 1 -- 172.21.15.112:0/755925424 >> 172.21.15.112:0/755925424 conn(0x7f125c0fa340 msgr2=0x7f125c0fb390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:52.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:52.116+0000 7f126272a640 1 -- 172.21.15.112:0/755925424 shutdown_connections 2024-09-16T00:32:52.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:32:52.116+0000 7f126272a640 1 -- 172.21.15.112:0/755925424 wait complete. 2024-09-16T00:32:52.838 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:52 smithi169 ceph-mon[77626]: from='client.34324 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:32:52.838 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:52 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/755925424' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:32:53.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:52 smithi112 ceph-mon[97750]: from='client.34324 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:32:53.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:52 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/755925424' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:32:53.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:53 smithi169 ceph-mon[77626]: from='client.34336 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:32:53.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:53 smithi169 ceph-mon[77626]: pgmap v133: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 325 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 64/372 objects degraded (17.204%) 2024-09-16T00:32:54.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:53 smithi112 ceph-mon[97750]: from='client.34336 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:32:54.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:53 smithi112 ceph-mon[97750]: pgmap v133: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 325 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 64/372 objects degraded (17.204%) 2024-09-16T00:32:55.776 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:32:55 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-4[91272]: 2024-09-16T00:32:55.447+0000 7f5b73aea640 -1 osd.4 67 log_to_monitors true 2024-09-16T00:32:55.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:55 smithi169 ceph-mon[77626]: pgmap v134: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 325 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 64/372 objects degraded (17.204%) 2024-09-16T00:32:55.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:55 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:32:55.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:55 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:32:55.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:55 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:32:55.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:55 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:32:55.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:55 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:32:55.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:55 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:32:55.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:55 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:32:55.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:55 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:32:55.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:55 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:32:55.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:55 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-09-16T00:32:55.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:55 smithi169 ceph-mon[77626]: from='osd.4 [v2:172.21.15.169:6808/3642552521,v1:172.21.15.169:6809/3642552521]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T00:32:55.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:55 smithi169 ceph-mon[77626]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T00:32:56.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:55 smithi112 ceph-mon[97750]: pgmap v134: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 325 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 64/372 objects degraded (17.204%) 2024-09-16T00:32:56.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:55 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:32:56.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:55 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:32:56.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:55 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:32:56.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:55 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:32:56.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:55 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:32:56.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:55 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:32:56.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:55 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:32:56.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:55 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:32:56.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:55 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:32:56.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:55 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-09-16T00:32:56.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:55 smithi112 ceph-mon[97750]: from='osd.4 [v2:172.21.15.169:6808/3642552521,v1:172.21.15.169:6809/3642552521]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T00:32:56.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:55 smithi112 ceph-mon[97750]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T00:32:57.026 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:32:56 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-4[91272]: 2024-09-16T00:32:56.685+0000 7f5b6787c640 -1 osd.4 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T00:32:57.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:56 smithi169 ceph-mon[77626]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-09-16T00:32:57.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:56 smithi169 ceph-mon[77626]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-09-16T00:32:57.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:56 smithi169 ceph-mon[77626]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-16T00:32:57.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:56 smithi169 ceph-mon[77626]: osdmap e70: 6 total, 5 up, 6 in 2024-09-16T00:32:57.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:56 smithi169 ceph-mon[77626]: from='osd.4 [v2:172.21.15.169:6808/3642552521,v1:172.21.15.169:6809/3642552521]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-09-16T00:32:57.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:56 smithi169 ceph-mon[77626]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-09-16T00:32:57.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:56 smithi169 ceph-mon[77626]: from='osd.4 ' entity='osd.4' 2024-09-16T00:32:57.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:56 smithi112 ceph-mon[97750]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-09-16T00:32:57.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:56 smithi112 ceph-mon[97750]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-09-16T00:32:57.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:56 smithi112 ceph-mon[97750]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-16T00:32:57.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:56 smithi112 ceph-mon[97750]: osdmap e70: 6 total, 5 up, 6 in 2024-09-16T00:32:57.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:56 smithi112 ceph-mon[97750]: from='osd.4 [v2:172.21.15.169:6808/3642552521,v1:172.21.15.169:6809/3642552521]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-09-16T00:32:57.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:56 smithi112 ceph-mon[97750]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-09-16T00:32:57.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:56 smithi112 ceph-mon[97750]: from='osd.4 ' entity='osd.4' 2024-09-16T00:32:58.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:57 smithi169 ceph-mon[77626]: pgmap v136: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 325 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 2 op/s; 64/372 objects degraded (17.204%) 2024-09-16T00:32:58.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:57 smithi169 ceph-mon[77626]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-16T00:32:58.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:57 smithi169 ceph-mon[77626]: osd.4 [v2:172.21.15.169:6808/3642552521,v1:172.21.15.169:6809/3642552521] boot 2024-09-16T00:32:58.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:57 smithi169 ceph-mon[77626]: osdmap e71: 6 total, 6 up, 6 in 2024-09-16T00:32:58.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:57 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:32:58.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:57 smithi112 ceph-mon[97750]: pgmap v136: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 325 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 2 op/s; 64/372 objects degraded (17.204%) 2024-09-16T00:32:58.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:57 smithi112 ceph-mon[97750]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-16T00:32:58.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:57 smithi112 ceph-mon[97750]: osd.4 [v2:172.21.15.169:6808/3642552521,v1:172.21.15.169:6809/3642552521] boot 2024-09-16T00:32:58.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:57 smithi112 ceph-mon[97750]: osdmap e71: 6 total, 6 up, 6 in 2024-09-16T00:32:58.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:57 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:33:00.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:59 smithi169 ceph-mon[77626]: pgmap v138: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 325 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s; 64/372 objects degraded (17.204%) 2024-09-16T00:33:00.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:32:59 smithi169 ceph-mon[77626]: osdmap e72: 6 total, 6 up, 6 in 2024-09-16T00:33:00.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:59 smithi112 ceph-mon[97750]: pgmap v138: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 325 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s; 64/372 objects degraded (17.204%) 2024-09-16T00:33:00.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:32:59 smithi112 ceph-mon[97750]: osdmap e72: 6 total, 6 up, 6 in 2024-09-16T00:33:01.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:00 smithi169 ceph-mon[77626]: Health check update: Degraded data redundancy: 11/372 objects degraded (2.957%), 4 pgs degraded (PG_DEGRADED) 2024-09-16T00:33:01.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:00 smithi112 ceph-mon[97750]: Health check update: Degraded data redundancy: 11/372 objects degraded (2.957%), 4 pgs degraded (PG_DEGRADED) 2024-09-16T00:33:02.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:01 smithi169 ceph-mon[77626]: pgmap v140: 65 pgs: 7 peering, 6 active+undersized, 4 active+undersized+degraded, 48 active+clean; 325 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s; 11/372 objects degraded (2.957%) 2024-09-16T00:33:02.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:01 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:33:02.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:01 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:33:02.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:01 smithi112 ceph-mon[97750]: pgmap v140: 65 pgs: 7 peering, 6 active+undersized, 4 active+undersized+degraded, 48 active+clean; 325 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s; 11/372 objects degraded (2.957%) 2024-09-16T00:33:02.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:01 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:33:02.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:01 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:33:04.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:03 smithi169 ceph-mon[77626]: pgmap v141: 65 pgs: 7 peering, 3 active+undersized, 1 active+undersized+degraded, 54 active+clean; 325 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 4/372 objects degraded (1.075%) 2024-09-16T00:33:04.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:03 smithi112 ceph-mon[97750]: pgmap v141: 65 pgs: 7 peering, 3 active+undersized, 1 active+undersized+degraded, 54 active+clean; 325 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 4/372 objects degraded (1.075%) 2024-09-16T00:33:05.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:04 smithi112 ceph-mon[97750]: pgmap v142: 65 pgs: 65 active+clean; 325 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 4 op/s 2024-09-16T00:33:05.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:04 smithi112 ceph-mon[97750]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 4/372 objects degraded (1.075%), 1 pg degraded) 2024-09-16T00:33:05.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:04 smithi169 ceph-mon[77626]: pgmap v142: 65 pgs: 65 active+clean; 325 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 4 op/s 2024-09-16T00:33:05.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:04 smithi169 ceph-mon[77626]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 4/372 objects degraded (1.075%), 1 pg degraded) 2024-09-16T00:33:08.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:07 smithi169 ceph-mon[77626]: pgmap v143: 65 pgs: 65 active+clean; 325 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 3 op/s 2024-09-16T00:33:08.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:07 smithi112 ceph-mon[97750]: pgmap v143: 65 pgs: 65 active+clean; 325 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 3 op/s 2024-09-16T00:33:10.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:09 smithi169 ceph-mon[77626]: pgmap v144: 65 pgs: 65 active+clean; 325 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-09-16T00:33:10.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:09 smithi112 ceph-mon[97750]: pgmap v144: 65 pgs: 65 active+clean; 325 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-09-16T00:33:10.926 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:10 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-09-16T00:33:11.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:10 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-09-16T00:33:11.901 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:11 smithi169 ceph-mon[77626]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-09-16T00:33:11.901 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:11 smithi169 ceph-mon[77626]: Upgrade: osd.5 is safe to restart 2024-09-16T00:33:11.901 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:11 smithi169 ceph-mon[77626]: pgmap v145: 65 pgs: 65 active+clean; 325 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-09-16T00:33:11.901 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:11 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:33:11.901 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:11 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-16T00:33:11.901 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:11 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:33:12.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:11 smithi112 ceph-mon[97750]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-09-16T00:33:12.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:11 smithi112 ceph-mon[97750]: Upgrade: osd.5 is safe to restart 2024-09-16T00:33:12.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:11 smithi112 ceph-mon[97750]: pgmap v145: 65 pgs: 65 active+clean; 325 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-09-16T00:33:12.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:11 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:33:12.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:11 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-16T00:33:12.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:11 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:33:12.936 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:12 smithi169 ceph-mon[77626]: Upgrade: Updating osd.5 2024-09-16T00:33:12.937 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:12 smithi169 ceph-mon[77626]: Deploying daemon osd.5 on smithi169 2024-09-16T00:33:12.937 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:12 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:33:13.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:12 smithi112 ceph-mon[97750]: Upgrade: Updating osd.5 2024-09-16T00:33:13.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:12 smithi112 ceph-mon[97750]: Deploying daemon osd.5 on smithi169 2024-09-16T00:33:13.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:12 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:33:13.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:13 smithi169 ceph-mon[77626]: pgmap v146: 65 pgs: 65 active+clean; 325 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:33:14.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:13 smithi112 ceph-mon[97750]: pgmap v146: 65 pgs: 65 active+clean; 325 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:33:14.439 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:33:14 smithi169 systemd[1]: Stopping Ceph osd.5 for d346dd56-73c0-11ef-bceb-c7b262605968... 2024-09-16T00:33:14.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:14 smithi169 ceph-mon[77626]: osd.5 marked itself down and dead 2024-09-16T00:33:14.776 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:33:14 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5[53200]: 2024-09-16T00:33:14.438+0000 7f72c43e6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:33:14.777 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:33:14 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5[53200]: 2024-09-16T00:33:14.438+0000 7f72c43e6640 -1 osd.5 72 *** Got signal Terminated *** 2024-09-16T00:33:14.777 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:33:14 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5[53200]: 2024-09-16T00:33:14.438+0000 7f72c43e6640 -1 osd.5 72 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:33:15.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:14 smithi112 ceph-mon[97750]: osd.5 marked itself down and dead 2024-09-16T00:33:15.422 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:33:15 smithi169 podman[95934]: 2024-09-16 00:33:15.1314924 +0000 UTC m=+0.809031035 container stop 8e7d708d74ebafbb6284df88e9aee962112763f8058fff6b77d1f25a2466a627 (image=quay.ceph.io/ceph-ci/ceph@sha256:83596473fc70117d507bb3d925c5bd11b70f46bbd4dffcc8452f145d93eaedc4, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=quincy-6d025bc, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2) 2024-09-16T00:33:15.422 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:33:15 smithi169 podman[95934]: 2024-09-16 00:33:15.157184643 +0000 UTC m=+0.834723279 container died 8e7d708d74ebafbb6284df88e9aee962112763f8058fff6b77d1f25a2466a627 (image=quay.ceph.io/ceph-ci/ceph@sha256:83596473fc70117d507bb3d925c5bd11b70f46bbd4dffcc8452f145d93eaedc4, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=quincy-6d025bc, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:33:15.756 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:15 smithi169 ceph-mon[77626]: pgmap v147: 65 pgs: 65 active+clean; 325 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:33:15.757 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:15 smithi169 ceph-mon[77626]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-16T00:33:15.757 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:15 smithi169 ceph-mon[77626]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-09-16T00:33:15.757 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:15 smithi169 ceph-mon[77626]: osdmap e73: 6 total, 5 up, 6 in 2024-09-16T00:33:15.757 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:33:15 smithi169 podman[95934]: 2024-09-16 00:33:15.530120526 +0000 UTC m=+1.207659162 container cleanup 8e7d708d74ebafbb6284df88e9aee962112763f8058fff6b77d1f25a2466a627 (image=quay.ceph.io/ceph-ci/ceph@sha256:83596473fc70117d507bb3d925c5bd11b70f46bbd4dffcc8452f145d93eaedc4, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=quincy-6d025bc) 2024-09-16T00:33:15.757 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:33:15 smithi169 bash[95934]: ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5 2024-09-16T00:33:16.026 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:33:15 smithi169 podman[95950]: 2024-09-16 00:33:15.756959467 +0000 UTC m=+0.619531403 container remove 8e7d708d74ebafbb6284df88e9aee962112763f8058fff6b77d1f25a2466a627 (image=quay.ceph.io/ceph-ci/ceph@sha256:83596473fc70117d507bb3d925c5bd11b70f46bbd4dffcc8452f145d93eaedc4, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=quincy-6d025bc, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-16T00:33:16.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:15 smithi112 ceph-mon[97750]: pgmap v147: 65 pgs: 65 active+clean; 325 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:33:16.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:15 smithi112 ceph-mon[97750]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-16T00:33:16.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:15 smithi112 ceph-mon[97750]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-09-16T00:33:16.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:15 smithi112 ceph-mon[97750]: osdmap e73: 6 total, 5 up, 6 in 2024-09-16T00:33:16.485 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:33:16 smithi169 podman[96040]: 2024-09-16 00:33:16.191313894 +0000 UTC m=+0.110767072 container create 864592e72b4cdd924250a3e2b0995a31ed040ea30240d668ecb015b4aacb70a5 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-16T00:33:16.485 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:33:16 smithi169 podman[96040]: 2024-09-16 00:33:16.096480579 +0000 UTC m=+0.015933734 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:33:16.485 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:33:16 smithi169 podman[96040]: 2024-09-16 00:33:16.402045426 +0000 UTC m=+0.321498572 container init 864592e72b4cdd924250a3e2b0995a31ed040ea30240d668ecb015b4aacb70a5 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5-deactivate, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:33:16.485 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:33:16 smithi169 podman[96040]: 2024-09-16 00:33:16.405415821 +0000 UTC m=+0.324868958 container start 864592e72b4cdd924250a3e2b0995a31ed040ea30240d668ecb015b4aacb70a5 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e) 2024-09-16T00:33:16.486 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:33:16 smithi169 podman[96040]: 2024-09-16 00:33:16.485296322 +0000 UTC m=+0.404749462 container attach 864592e72b4cdd924250a3e2b0995a31ed040ea30240d668ecb015b4aacb70a5 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:33:16.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:16 smithi169 ceph-mon[77626]: osdmap e74: 6 total, 5 up, 6 in 2024-09-16T00:33:16.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:16 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1", "id": [0, 5]}]: dispatch 2024-09-16T00:33:16.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:16 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [3, 5]}]: dispatch 2024-09-16T00:33:16.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:16 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.11", "id": [3, 5]}]: dispatch 2024-09-16T00:33:16.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:16 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [0, 5]}]: dispatch 2024-09-16T00:33:16.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:16 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.15", "id": [3, 5]}]: dispatch 2024-09-16T00:33:16.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:16 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [0, 2]}]: dispatch 2024-09-16T00:33:16.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:16 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 5]}]: dispatch 2024-09-16T00:33:16.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:16 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [1, 5]}]: dispatch 2024-09-16T00:33:16.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:16 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [1, 5]}]: dispatch 2024-09-16T00:33:16.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:16 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.d", "id": [3, 5]}]: dispatch 2024-09-16T00:33:16.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:16 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:33:16.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:16 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:33:16.778 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:33:16 smithi169 podman[96040]: 2024-09-16 00:33:16.604534789 +0000 UTC m=+0.523987927 container died 864592e72b4cdd924250a3e2b0995a31ed040ea30240d668ecb015b4aacb70a5 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux ) 2024-09-16T00:33:17.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:16 smithi112 ceph-mon[97750]: osdmap e74: 6 total, 5 up, 6 in 2024-09-16T00:33:17.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:16 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1", "id": [0, 5]}]: dispatch 2024-09-16T00:33:17.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:16 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [3, 5]}]: dispatch 2024-09-16T00:33:17.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:16 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.11", "id": [3, 5]}]: dispatch 2024-09-16T00:33:17.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:16 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [0, 5]}]: dispatch 2024-09-16T00:33:17.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:16 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.15", "id": [3, 5]}]: dispatch 2024-09-16T00:33:17.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:16 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [0, 2]}]: dispatch 2024-09-16T00:33:17.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:16 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 5]}]: dispatch 2024-09-16T00:33:17.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:16 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [1, 5]}]: dispatch 2024-09-16T00:33:17.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:16 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [1, 5]}]: dispatch 2024-09-16T00:33:17.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:16 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.d", "id": [3, 5]}]: dispatch 2024-09-16T00:33:17.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:16 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:33:17.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:16 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:33:17.526 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:33:17 smithi169 podman[96063]: 2024-09-16 00:33:17.18917202 +0000 UTC m=+0.577931728 container remove 864592e72b4cdd924250a3e2b0995a31ed040ea30240d668ecb015b4aacb70a5 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e) 2024-09-16T00:33:17.526 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:33:17 smithi169 systemd[1]: ceph-d346dd56-73c0-11ef-bceb-c7b262605968@osd.5.service: Deactivated successfully. 2024-09-16T00:33:17.526 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:33:17 smithi169 systemd[1]: Stopped Ceph osd.5 for d346dd56-73c0-11ef-bceb-c7b262605968. 2024-09-16T00:33:17.526 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:33:17 smithi169 systemd[1]: ceph-d346dd56-73c0-11ef-bceb-c7b262605968@osd.5.service: Consumed 50.740s CPU time. 2024-09-16T00:33:17.526 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:33:17 smithi169 systemd[1]: Starting Ceph osd.5 for d346dd56-73c0-11ef-bceb-c7b262605968... 2024-09-16T00:33:17.943 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:17 smithi169 ceph-mon[77626]: pgmap v150: 65 pgs: 4 active+undersized, 9 stale+active+clean, 3 active+undersized+degraded, 49 active+clean; 325 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 3 op/s; 8/372 objects degraded (2.151%) 2024-09-16T00:33:17.943 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:17 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1", "id": [0, 5]}]': finished 2024-09-16T00:33:17.944 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:17 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [3, 5]}]': finished 2024-09-16T00:33:17.944 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:17 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.11", "id": [3, 5]}]': finished 2024-09-16T00:33:17.944 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:17 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [0, 5]}]': finished 2024-09-16T00:33:17.944 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:17 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.15", "id": [3, 5]}]': finished 2024-09-16T00:33:17.944 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:17 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [0, 2]}]': finished 2024-09-16T00:33:17.944 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:17 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 5]}]': finished 2024-09-16T00:33:17.945 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:17 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [1, 5]}]': finished 2024-09-16T00:33:17.945 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:17 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [1, 5]}]': finished 2024-09-16T00:33:17.945 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:17 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.d", "id": [3, 5]}]': finished 2024-09-16T00:33:17.945 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:17 smithi169 ceph-mon[77626]: osdmap e75: 6 total, 5 up, 6 in 2024-09-16T00:33:17.945 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:17 smithi169 ceph-mon[77626]: Health check failed: Degraded data redundancy: 8/372 objects degraded (2.151%), 3 pgs degraded (PG_DEGRADED) 2024-09-16T00:33:17.946 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:33:17 smithi169 podman[96162]: 2024-09-16 00:33:17.529743914 +0000 UTC m=+0.015684117 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:33:17.946 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:33:17 smithi169 podman[96162]: 2024-09-16 00:33:17.642013102 +0000 UTC m=+0.127953301 container create 4bd80864d3b0ec16ad738821bad033f3c9ba82e5a5b825cc02f9b7a981dab4a0 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5-activate, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-16T00:33:17.946 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:33:17 smithi169 podman[96162]: 2024-09-16 00:33:17.868411956 +0000 UTC m=+0.354352149 container init 4bd80864d3b0ec16ad738821bad033f3c9ba82e5a5b825cc02f9b7a981dab4a0 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5-activate, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-16T00:33:17.946 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:33:17 smithi169 podman[96162]: 2024-09-16 00:33:17.87174918 +0000 UTC m=+0.357689375 container start 4bd80864d3b0ec16ad738821bad033f3c9ba82e5a5b825cc02f9b7a981dab4a0 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5-activate, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:33:17.946 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:33:17 smithi169 podman[96162]: 2024-09-16 00:33:17.943303496 +0000 UTC m=+0.429243704 container attach 4bd80864d3b0ec16ad738821bad033f3c9ba82e5a5b825cc02f9b7a981dab4a0 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5-activate, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:33:18.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:17 smithi112 ceph-mon[97750]: pgmap v150: 65 pgs: 4 active+undersized, 9 stale+active+clean, 3 active+undersized+degraded, 49 active+clean; 325 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 3 op/s; 8/372 objects degraded (2.151%) 2024-09-16T00:33:18.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:17 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1", "id": [0, 5]}]': finished 2024-09-16T00:33:18.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:17 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [3, 5]}]': finished 2024-09-16T00:33:18.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:17 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.11", "id": [3, 5]}]': finished 2024-09-16T00:33:18.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:17 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [0, 5]}]': finished 2024-09-16T00:33:18.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:17 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.15", "id": [3, 5]}]': finished 2024-09-16T00:33:18.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:17 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [0, 2]}]': finished 2024-09-16T00:33:18.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:17 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 5]}]': finished 2024-09-16T00:33:18.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:17 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [1, 5]}]': finished 2024-09-16T00:33:18.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:17 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [1, 5]}]': finished 2024-09-16T00:33:18.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:17 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.d", "id": [3, 5]}]': finished 2024-09-16T00:33:18.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:17 smithi112 ceph-mon[97750]: osdmap e75: 6 total, 5 up, 6 in 2024-09-16T00:33:18.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:17 smithi112 ceph-mon[97750]: Health check failed: Degraded data redundancy: 8/372 objects degraded (2.151%), 3 pgs degraded (PG_DEGRADED) 2024-09-16T00:33:18.276 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:33:18 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5-activate[96177]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T00:33:18.276 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:33:18 smithi169 bash[96162]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T00:33:18.276 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:33:18 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5-activate[96177]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T00:33:18.277 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:33:18 smithi169 bash[96162]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T00:33:19.028 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:18 smithi169 ceph-mon[77626]: osdmap e76: 6 total, 5 up, 6 in 2024-09-16T00:33:19.028 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:18 smithi169 ceph-mon[77626]: osdmap e77: 6 total, 5 up, 6 in 2024-09-16T00:33:19.028 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:33:18 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5-activate[96177]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-16T00:33:19.028 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:33:18 smithi169 bash[96162]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-16T00:33:19.028 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:33:18 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5-activate[96177]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-09-16T00:33:19.029 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:33:18 smithi169 bash[96162]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-09-16T00:33:19.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:18 smithi112 ceph-mon[97750]: osdmap e76: 6 total, 5 up, 6 in 2024-09-16T00:33:19.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:18 smithi112 ceph-mon[97750]: osdmap e77: 6 total, 5 up, 6 in 2024-09-16T00:33:19.347 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:33:19 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5-activate[96177]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-09-16T00:33:19.347 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:33:19 smithi169 bash[96162]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-09-16T00:33:19.347 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:33:19 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5-activate[96177]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-09-16T00:33:19.347 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:33:19 smithi169 bash[96162]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-09-16T00:33:19.347 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:33:19 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5-activate[96177]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-09-16T00:33:19.347 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:33:19 smithi169 bash[96162]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-09-16T00:33:19.347 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:33:19 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5-activate[96177]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-16T00:33:19.347 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:33:19 smithi169 bash[96162]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-16T00:33:19.348 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:33:19 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5-activate[96177]: --> ceph-volume raw activate successful for osd ID: 5 2024-09-16T00:33:19.348 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:33:19 smithi169 bash[96162]: --> ceph-volume raw activate successful for osd ID: 5 2024-09-16T00:33:19.348 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:33:19 smithi169 podman[96162]: 2024-09-16 00:33:19.057495349 +0000 UTC m=+1.543435544 container died 4bd80864d3b0ec16ad738821bad033f3c9ba82e5a5b825cc02f9b7a981dab4a0 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5-activate, RELEASE=squid-8921c8e, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731) 2024-09-16T00:33:20.003 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:19 smithi169 ceph-mon[77626]: pgmap v153: 65 pgs: 4 active+undersized, 9 stale+active+clean, 3 active+undersized+degraded, 49 active+clean; 325 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 8/372 objects degraded (2.151%) 2024-09-16T00:33:20.004 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:33:19 smithi169 podman[96317]: 2024-09-16 00:33:19.680774736 +0000 UTC m=+0.616929553 container remove 4bd80864d3b0ec16ad738821bad033f3c9ba82e5a5b825cc02f9b7a981dab4a0 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5-activate, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e) 2024-09-16T00:33:20.004 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:33:19 smithi169 podman[96376]: 2024-09-16 00:33:19.882001114 +0000 UTC m=+0.110157033 container create a6007c13cf78e7f0ea64d10360a3f08e6538f290598f551c004741057c5e116c (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=squid-8921c8e) 2024-09-16T00:33:20.004 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:33:19 smithi169 podman[96376]: 2024-09-16 00:33:19.787472011 +0000 UTC m=+0.015627934 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:33:20.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:19 smithi112 ceph-mon[97750]: pgmap v153: 65 pgs: 4 active+undersized, 9 stale+active+clean, 3 active+undersized+degraded, 49 active+clean; 325 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 8/372 objects degraded (2.151%) 2024-09-16T00:33:20.276 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:33:20 smithi169 podman[96376]: 2024-09-16 00:33:20.101210806 +0000 UTC m=+0.329366729 container init a6007c13cf78e7f0ea64d10360a3f08e6538f290598f551c004741057c5e116c (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731) 2024-09-16T00:33:20.276 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:33:20 smithi169 podman[96376]: 2024-09-16 00:33:20.10440656 +0000 UTC m=+0.332562479 container start a6007c13cf78e7f0ea64d10360a3f08e6538f290598f551c004741057c5e116c (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True) 2024-09-16T00:33:20.276 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:33:20 smithi169 ceph-osd[96395]: -- 172.21.15.169:0/1844911984 <== mon.1 v2:172.21.15.169:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f44803e1e0 con 0x55f447fd4000 2024-09-16T00:33:20.276 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:33:20 smithi169 bash[96376]: a6007c13cf78e7f0ea64d10360a3f08e6538f290598f551c004741057c5e116c 2024-09-16T00:33:20.277 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:33:20 smithi169 systemd[1]: Started Ceph osd.5 for d346dd56-73c0-11ef-bceb-c7b262605968. 2024-09-16T00:33:20.980 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:20 smithi169 ceph-mon[77626]: osdmap e78: 6 total, 5 up, 6 in 2024-09-16T00:33:20.980 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:20 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:33:20.981 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:20 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:33:20.981 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:20 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:33:21.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:20 smithi112 ceph-mon[97750]: osdmap e78: 6 total, 5 up, 6 in 2024-09-16T00:33:21.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:20 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:33:21.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:20 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:33:21.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:20 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:33:21.972 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:21 smithi169 ceph-mon[77626]: pgmap v156: 65 pgs: 2 active+remapped, 1 active+undersized+degraded+remapped, 3 active+clean+remapped, 2 remapped+peering, 14 active+undersized, 13 active+undersized+degraded, 30 active+clean; 325 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 53/372 objects degraded (14.247%); 10/372 objects misplaced (2.688%); 3.0 MiB/s, 0 objects/s recovering 2024-09-16T00:33:21.972 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:21 smithi169 ceph-mon[77626]: osdmap e79: 6 total, 5 up, 6 in 2024-09-16T00:33:22.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:21 smithi112 ceph-mon[97750]: pgmap v156: 65 pgs: 2 active+remapped, 1 active+undersized+degraded+remapped, 3 active+clean+remapped, 2 remapped+peering, 14 active+undersized, 13 active+undersized+degraded, 30 active+clean; 325 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 53/372 objects degraded (14.247%); 10/372 objects misplaced (2.688%); 3.0 MiB/s, 0 objects/s recovering 2024-09-16T00:33:22.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:21 smithi112 ceph-mon[97750]: osdmap e79: 6 total, 5 up, 6 in 2024-09-16T00:33:22.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.206+0000 7f2a2d20c640 1 -- 172.21.15.112:0/3964921709 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a280fac80 msgr2=0x7f2a280fb0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:22.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.206+0000 7f2a2d20c640 1 --2- 172.21.15.112:0/3964921709 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a280fac80 0x7f2a280fb0f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c0048d0 tx=0x7f2a1c009da0 comp rx=0 tx=0).stop 2024-09-16T00:33:22.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.206+0000 7f2a2d20c640 1 -- 172.21.15.112:0/3964921709 shutdown_connections 2024-09-16T00:33:22.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.206+0000 7f2a2d20c640 1 --2- 172.21.15.112:0/3964921709 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a280fac80 0x7f2a280fb0f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:22.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.206+0000 7f2a2d20c640 1 --2- 172.21.15.112:0/3964921709 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a28102620 0x7f2a281029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:22.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.206+0000 7f2a2d20c640 1 -- 172.21.15.112:0/3964921709 >> 172.21.15.112:0/3964921709 conn(0x7f2a28070ce0 msgr2=0x7f2a280710e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:22.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.206+0000 7f2a2d20c640 1 -- 172.21.15.112:0/3964921709 shutdown_connections 2024-09-16T00:33:22.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.206+0000 7f2a2d20c640 1 -- 172.21.15.112:0/3964921709 wait complete. 2024-09-16T00:33:22.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.207+0000 7f2a2d20c640 1 Processor -- start 2024-09-16T00:33:22.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.207+0000 7f2a2d20c640 1 -- start start 2024-09-16T00:33:22.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.207+0000 7f2a2d20c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a280fac80 0x7f2a281131e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:22.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.207+0000 7f2a2d20c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a28102620 0x7f2a28113720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:22.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.207+0000 7f2a2d20c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a28113d20 con 0x7f2a280fac80 2024-09-16T00:33:22.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.207+0000 7f2a2d20c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2810d160 con 0x7f2a28102620 2024-09-16T00:33:22.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.207+0000 7f2a26d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a280fac80 0x7f2a281131e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:22.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.207+0000 7f2a26d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a280fac80 0x7f2a281131e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59078/0 (socket says 172.21.15.112:59078) 2024-09-16T00:33:22.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.207+0000 7f2a26d9d640 1 -- 172.21.15.112:0/825933993 learned_addr learned my addr 172.21.15.112:0/825933993 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:33:22.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.207+0000 7f2a2659c640 1 --2- 172.21.15.112:0/825933993 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a28102620 0x7f2a28113720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:22.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.208+0000 7f2a2659c640 1 -- 172.21.15.112:0/825933993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a280fac80 msgr2=0x7f2a281131e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:22.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.208+0000 7f2a2659c640 1 --2- 172.21.15.112:0/825933993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a280fac80 0x7f2a281131e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:22.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.208+0000 7f2a2659c640 1 -- 172.21.15.112:0/825933993 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a1c005040 con 0x7f2a28102620 2024-09-16T00:33:22.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.208+0000 7f2a26d9d640 1 --2- 172.21.15.112:0/825933993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a280fac80 0x7f2a281131e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:33:22.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.208+0000 7f2a2659c640 1 --2- 172.21.15.112:0/825933993 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a28102620 0x7f2a28113720 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c02a530 tx=0x7f2a1c009da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:22.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.208+0000 7f2a0f7fe640 1 -- 172.21.15.112:0/825933993 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a1c038070 con 0x7f2a28102620 2024-09-16T00:33:22.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.208+0000 7f2a2d20c640 1 -- 172.21.15.112:0/825933993 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a2810d3e0 con 0x7f2a28102620 2024-09-16T00:33:22.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.208+0000 7f2a0f7fe640 1 -- 172.21.15.112:0/825933993 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f2a1c02ade0 con 0x7f2a28102620 2024-09-16T00:33:22.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.208+0000 7f2a0f7fe640 1 -- 172.21.15.112:0/825933993 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a1c02c770 con 0x7f2a28102620 2024-09-16T00:33:22.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.208+0000 7f2a2d20c640 1 -- 172.21.15.112:0/825933993 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a2810d920 con 0x7f2a28102620 2024-09-16T00:33:22.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.209+0000 7f2a2d20c640 1 -- 172.21.15.112:0/825933993 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a280657b0 con 0x7f2a28102620 2024-09-16T00:33:22.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.210+0000 7f2a0f7fe640 1 -- 172.21.15.112:0/825933993 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f2a1c030020 con 0x7f2a28102620 2024-09-16T00:33:22.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.210+0000 7f2a0f7fe640 1 --2- 172.21.15.112:0/825933993 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f2a1006e530 0x7f2a100709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:22.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.210+0000 7f2a0f7fe640 1 -- 172.21.15.112:0/825933993 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7030+0+0 (secure 0 0 0) 0x7f2a1c0ae780 con 0x7f2a28102620 2024-09-16T00:33:22.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.210+0000 7f2a26d9d640 1 --2- 172.21.15.112:0/825933993 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f2a1006e530 0x7f2a100709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:22.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.211+0000 7f2a26d9d640 1 --2- 172.21.15.112:0/825933993 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f2a1006e530 0x7f2a100709e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2a18007e60 tx=0x7f2a1800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:22.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.215+0000 7f2a0f7fe640 1 -- 172.21.15.112:0/825933993 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f2a1c0776b0 con 0x7f2a28102620 2024-09-16T00:33:22.276 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:33:21 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5[96391]: 2024-09-16T00:33:21.971+0000 7f4ed8b7f640 -1 Falling back to public interface 2024-09-16T00:33:22.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.398+0000 7f2a2d20c640 1 -- 172.21.15.112:0/825933993 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2a28109590 con 0x7f2a1006e530 2024-09-16T00:33:22.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.399+0000 7f2a0f7fe640 1 -- 172.21.15.112:0/825933993 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f2a28109590 con 0x7f2a1006e530 2024-09-16T00:33:22.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.401+0000 7f2a2d20c640 1 -- 172.21.15.112:0/825933993 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f2a1006e530 msgr2=0x7f2a100709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:22.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.401+0000 7f2a2d20c640 1 --2- 172.21.15.112:0/825933993 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f2a1006e530 0x7f2a100709e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2a18007e60 tx=0x7f2a1800b040 comp rx=0 tx=0).stop 2024-09-16T00:33:22.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.401+0000 7f2a2d20c640 1 -- 172.21.15.112:0/825933993 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a28102620 msgr2=0x7f2a28113720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:22.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.401+0000 7f2a2d20c640 1 --2- 172.21.15.112:0/825933993 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a28102620 0x7f2a28113720 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c02a530 tx=0x7f2a1c009da0 comp rx=0 tx=0).stop 2024-09-16T00:33:22.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.401+0000 7f2a2d20c640 1 -- 172.21.15.112:0/825933993 shutdown_connections 2024-09-16T00:33:22.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.401+0000 7f2a2d20c640 1 --2- 172.21.15.112:0/825933993 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f2a1006e530 0x7f2a100709e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:22.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.401+0000 7f2a2d20c640 1 --2- 172.21.15.112:0/825933993 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a28102620 0x7f2a28113720 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:22.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.401+0000 7f2a2d20c640 1 --2- 172.21.15.112:0/825933993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a280fac80 0x7f2a281131e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:22.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.401+0000 7f2a2d20c640 1 -- 172.21.15.112:0/825933993 >> 172.21.15.112:0/825933993 conn(0x7f2a28070ce0 msgr2=0x7f2a28105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:22.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.401+0000 7f2a2d20c640 1 -- 172.21.15.112:0/825933993 shutdown_connections 2024-09-16T00:33:22.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.401+0000 7f2a2d20c640 1 -- 172.21.15.112:0/825933993 wait complete. 2024-09-16T00:33:22.410 INFO:teuthology.orchestra.run.smithi112.stdout:true 2024-09-16T00:33:22.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.480+0000 7fd5c46ce640 1 -- 172.21.15.112:0/2578526748 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5bc1025f0 msgr2=0x7fd5bc1029c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:22.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.480+0000 7fd5c46ce640 1 --2- 172.21.15.112:0/2578526748 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5bc1025f0 0x7fd5bc1029c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd5b80048d0 tx=0x7fd5b8009da0 comp rx=0 tx=0).stop 2024-09-16T00:33:22.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.480+0000 7fd5c46ce640 1 -- 172.21.15.112:0/2578526748 shutdown_connections 2024-09-16T00:33:22.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.480+0000 7fd5c46ce640 1 --2- 172.21.15.112:0/2578526748 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5bc0fac60 0x7fd5bc0fb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:22.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.480+0000 7fd5c46ce640 1 --2- 172.21.15.112:0/2578526748 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5bc1025f0 0x7fd5bc1029c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:22.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.480+0000 7fd5c46ce640 1 -- 172.21.15.112:0/2578526748 >> 172.21.15.112:0/2578526748 conn(0x7fd5bc070c80 msgr2=0x7fd5bc071080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:22.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.480+0000 7fd5c46ce640 1 -- 172.21.15.112:0/2578526748 shutdown_connections 2024-09-16T00:33:22.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.480+0000 7fd5c46ce640 1 -- 172.21.15.112:0/2578526748 wait complete. 2024-09-16T00:33:22.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.481+0000 7fd5c46ce640 1 Processor -- start 2024-09-16T00:33:22.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.481+0000 7fd5c46ce640 1 -- start start 2024-09-16T00:33:22.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.481+0000 7fd5c46ce640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5bc0fac60 0x7fd5bc1131d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:22.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.481+0000 7fd5c46ce640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5bc1025f0 0x7fd5bc113710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:22.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.481+0000 7fd5c46ce640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5bc113d10 con 0x7fd5bc0fac60 2024-09-16T00:33:22.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.481+0000 7fd5c46ce640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5bc10d150 con 0x7fd5bc1025f0 2024-09-16T00:33:22.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.481+0000 7fd5c246a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5bc0fac60 0x7fd5bc1131d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:22.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.481+0000 7fd5c246a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5bc0fac60 0x7fd5bc1131d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59088/0 (socket says 172.21.15.112:59088) 2024-09-16T00:33:22.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.481+0000 7fd5c246a640 1 -- 172.21.15.112:0/1931431141 learned_addr learned my addr 172.21.15.112:0/1931431141 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:33:22.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.481+0000 7fd5c1c69640 1 --2- 172.21.15.112:0/1931431141 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5bc1025f0 0x7fd5bc113710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:22.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.482+0000 7fd5c246a640 1 -- 172.21.15.112:0/1931431141 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5bc1025f0 msgr2=0x7fd5bc113710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:22.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.482+0000 7fd5c246a640 1 --2- 172.21.15.112:0/1931431141 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5bc1025f0 0x7fd5bc113710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:22.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.482+0000 7fd5c246a640 1 -- 172.21.15.112:0/1931431141 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5b8005040 con 0x7fd5bc0fac60 2024-09-16T00:33:22.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.482+0000 7fd5c246a640 1 --2- 172.21.15.112:0/1931431141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5bc0fac60 0x7fd5bc1131d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8000c00 tx=0x7fd5b8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:22.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.482+0000 7fd5b2ffd640 1 -- 172.21.15.112:0/1931431141 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd5b8038070 con 0x7fd5bc0fac60 2024-09-16T00:33:22.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.482+0000 7fd5b2ffd640 1 -- 172.21.15.112:0/1931431141 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fd5b802ade0 con 0x7fd5bc0fac60 2024-09-16T00:33:22.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.482+0000 7fd5c46ce640 1 -- 172.21.15.112:0/1931431141 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5bc10d3d0 con 0x7fd5bc0fac60 2024-09-16T00:33:22.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.482+0000 7fd5b2ffd640 1 -- 172.21.15.112:0/1931431141 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd5b802c7e0 con 0x7fd5bc0fac60 2024-09-16T00:33:22.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.482+0000 7fd5c46ce640 1 -- 172.21.15.112:0/1931431141 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5bc10d910 con 0x7fd5bc0fac60 2024-09-16T00:33:22.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.483+0000 7fd5c46ce640 1 -- 172.21.15.112:0/1931431141 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5bc065810 con 0x7fd5bc0fac60 2024-09-16T00:33:22.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.484+0000 7fd5b2ffd640 1 -- 172.21.15.112:0/1931431141 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fd5b8030020 con 0x7fd5bc0fac60 2024-09-16T00:33:22.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.484+0000 7fd5b2ffd640 1 --2- 172.21.15.112:0/1931431141 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd5a806e530 0x7fd5a80709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:22.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.485+0000 7fd5b2ffd640 1 -- 172.21.15.112:0/1931431141 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7030+0+0 (secure 0 0 0) 0x7fd5b80adf60 con 0x7fd5bc0fac60 2024-09-16T00:33:22.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.485+0000 7fd5c1c69640 1 --2- 172.21.15.112:0/1931431141 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd5a806e530 0x7fd5a80709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:22.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.485+0000 7fd5c1c69640 1 --2- 172.21.15.112:0/1931431141 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd5a806e530 0x7fd5a80709e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc10ea90 tx=0x7fd5ac00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:22.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.489+0000 7fd5b2ffd640 1 -- 172.21.15.112:0/1931431141 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fd5b80e31b0 con 0x7fd5bc0fac60 2024-09-16T00:33:22.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.674+0000 7fd5c46ce640 1 -- 172.21.15.112:0/1931431141 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd5bc04b720 con 0x7fd5a806e530 2024-09-16T00:33:22.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.675+0000 7fd5b2ffd640 1 -- 172.21.15.112:0/1931431141 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fd5bc04b720 con 0x7fd5a806e530 2024-09-16T00:33:22.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.677+0000 7fd5c46ce640 1 -- 172.21.15.112:0/1931431141 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd5a806e530 msgr2=0x7fd5a80709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:22.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.677+0000 7fd5c46ce640 1 --2- 172.21.15.112:0/1931431141 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd5a806e530 0x7fd5a80709e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc10ea90 tx=0x7fd5ac00b040 comp rx=0 tx=0).stop 2024-09-16T00:33:22.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.677+0000 7fd5c46ce640 1 -- 172.21.15.112:0/1931431141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5bc0fac60 msgr2=0x7fd5bc1131d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:22.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.677+0000 7fd5c46ce640 1 --2- 172.21.15.112:0/1931431141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5bc0fac60 0x7fd5bc1131d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8000c00 tx=0x7fd5b8009da0 comp rx=0 tx=0).stop 2024-09-16T00:33:22.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.677+0000 7fd5c46ce640 1 -- 172.21.15.112:0/1931431141 shutdown_connections 2024-09-16T00:33:22.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.677+0000 7fd5c46ce640 1 --2- 172.21.15.112:0/1931431141 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd5a806e530 0x7fd5a80709e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:22.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.677+0000 7fd5c46ce640 1 --2- 172.21.15.112:0/1931431141 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5bc1025f0 0x7fd5bc113710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:22.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.677+0000 7fd5c46ce640 1 --2- 172.21.15.112:0/1931431141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5bc0fac60 0x7fd5bc1131d0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:22.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.677+0000 7fd5c46ce640 1 -- 172.21.15.112:0/1931431141 >> 172.21.15.112:0/1931431141 conn(0x7fd5bc070c80 msgr2=0x7fd5bc0f9f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:22.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.678+0000 7fd5c46ce640 1 -- 172.21.15.112:0/1931431141 shutdown_connections 2024-09-16T00:33:22.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.678+0000 7fd5c46ce640 1 -- 172.21.15.112:0/1931431141 wait complete. 2024-09-16T00:33:22.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.757+0000 7fe4f10dd640 1 -- 172.21.15.112:0/2782438486 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4ec102560 msgr2=0x7fe4ec102930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:22.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.757+0000 7fe4f10dd640 1 --2- 172.21.15.112:0/2782438486 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4ec102560 0x7fe4ec102930 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc0048d0 tx=0x7fe4dc009da0 comp rx=0 tx=0).stop 2024-09-16T00:33:22.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.757+0000 7fe4f10dd640 1 -- 172.21.15.112:0/2782438486 shutdown_connections 2024-09-16T00:33:22.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.757+0000 7fe4f10dd640 1 --2- 172.21.15.112:0/2782438486 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4ec102f00 0x7fe4ec0fc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:22.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.757+0000 7fe4f10dd640 1 --2- 172.21.15.112:0/2782438486 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4ec102560 0x7fe4ec102930 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:22.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.757+0000 7fe4f10dd640 1 -- 172.21.15.112:0/2782438486 >> 172.21.15.112:0/2782438486 conn(0x7fe4ec0f8120 msgr2=0x7fe4ec0fa530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:22.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.757+0000 7fe4f10dd640 1 -- 172.21.15.112:0/2782438486 shutdown_connections 2024-09-16T00:33:22.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.757+0000 7fe4f10dd640 1 -- 172.21.15.112:0/2782438486 wait complete. 2024-09-16T00:33:22.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.757+0000 7fe4f10dd640 1 Processor -- start 2024-09-16T00:33:22.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.758+0000 7fe4f10dd640 1 -- start start 2024-09-16T00:33:22.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.758+0000 7fe4f10dd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4ec102560 0x7fe4ec113220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:22.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.758+0000 7fe4f10dd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4ec102f00 0x7fe4ec113760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:22.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.758+0000 7fe4f10dd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ec113d60 con 0x7fe4ec102560 2024-09-16T00:33:22.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.758+0000 7fe4f10dd640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ec10d1a0 con 0x7fe4ec102f00 2024-09-16T00:33:22.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.758+0000 7fe4ead9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4ec102560 0x7fe4ec113220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:22.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.758+0000 7fe4ea59c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4ec102f00 0x7fe4ec113760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:22.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.758+0000 7fe4ead9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4ec102560 0x7fe4ec113220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59110/0 (socket says 172.21.15.112:59110) 2024-09-16T00:33:22.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.758+0000 7fe4ead9d640 1 -- 172.21.15.112:0/886165995 learned_addr learned my addr 172.21.15.112:0/886165995 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:33:22.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.758+0000 7fe4ea59c640 1 -- 172.21.15.112:0/886165995 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4ec102560 msgr2=0x7fe4ec113220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:22.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.758+0000 7fe4ea59c640 1 --2- 172.21.15.112:0/886165995 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4ec102560 0x7fe4ec113220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:22.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.758+0000 7fe4ea59c640 1 -- 172.21.15.112:0/886165995 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4dc005040 con 0x7fe4ec102f00 2024-09-16T00:33:22.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.758+0000 7fe4ead9d640 1 --2- 172.21.15.112:0/886165995 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4ec102560 0x7fe4ec113220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:33:22.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.759+0000 7fe4ea59c640 1 --2- 172.21.15.112:0/886165995 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4ec102f00 0x7fe4ec113760 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec066a60 tx=0x7fe4e000b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:22.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.759+0000 7fe4d37fe640 1 -- 172.21.15.112:0/886165995 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4e0007c30 con 0x7fe4ec102f00 2024-09-16T00:33:22.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.759+0000 7fe4f10dd640 1 -- 172.21.15.112:0/886165995 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4ec10d420 con 0x7fe4ec102f00 2024-09-16T00:33:22.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.759+0000 7fe4d37fe640 1 -- 172.21.15.112:0/886165995 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fe4e000f040 con 0x7fe4ec102f00 2024-09-16T00:33:22.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.759+0000 7fe4d37fe640 1 -- 172.21.15.112:0/886165995 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4e000ca00 con 0x7fe4ec102f00 2024-09-16T00:33:22.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.759+0000 7fe4f10dd640 1 -- 172.21.15.112:0/886165995 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4ec10d9c0 con 0x7fe4ec102f00 2024-09-16T00:33:22.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.760+0000 7fe4f10dd640 1 -- 172.21.15.112:0/886165995 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4ec0fdce0 con 0x7fe4ec102f00 2024-09-16T00:33:22.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.760+0000 7fe4d37fe640 1 -- 172.21.15.112:0/886165995 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fe4e0009670 con 0x7fe4ec102f00 2024-09-16T00:33:22.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.761+0000 7fe4d37fe640 1 --2- 172.21.15.112:0/886165995 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fe4d406e530 0x7fe4d40709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:22.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.761+0000 7fe4d37fe640 1 -- 172.21.15.112:0/886165995 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7030+0+0 (secure 0 0 0) 0x7fe4e008df60 con 0x7fe4ec102f00 2024-09-16T00:33:22.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.761+0000 7fe4ead9d640 1 --2- 172.21.15.112:0/886165995 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fe4d406e530 0x7fe4d40709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:22.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.761+0000 7fe4ead9d640 1 --2- 172.21.15.112:0/886165995 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fe4d406e530 0x7fe4d40709e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc009ac0 tx=0x7fe4dc035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:22.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.765+0000 7fe4d37fe640 1 -- 172.21.15.112:0/886165995 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fe4e005aa90 con 0x7fe4ec102f00 2024-09-16T00:33:22.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.940+0000 7fe4f10dd640 1 -- 172.21.15.112:0/886165995 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe4ec1095d0 con 0x7fe4d406e530 2024-09-16T00:33:22.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.948+0000 7fe4d37fe640 1 -- 172.21.15.112:0/886165995 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fe4ec1095d0 con 0x7fe4d406e530 2024-09-16T00:33:22.949 INFO:teuthology.orchestra.run.smithi112.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T00:33:22.949 INFO:teuthology.orchestra.run.smithi112.stdout:alertmanager.smithi112 smithi112 *:9093,9094 running (6m) 96s ago 15m 25.2M - 0.25.0 c8568f914cd2 d85ea0562022 2024-09-16T00:33:22.949 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi112 smithi112 running (3m) 96s ago 15m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c e4fe58f9702a 2024-09-16T00:33:22.949 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi169 smithi169 running (3m) 32s ago 14m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c 3ba2c81d527f 2024-09-16T00:33:22.949 INFO:teuthology.orchestra.run.smithi112.stdout:grafana.smithi112 smithi112 *:3000 running (5m) 96s ago 15m 82.2M - 9.4.12 2bacad6d85d8 51f9d2013da2 2024-09-16T00:33:22.949 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.hxfwkh smithi112 running (9m) 96s ago 9m 135M - 17.2.7-1633-g6d025bc2 33ed8dba3996 cf92d9461fc6 2024-09-16T00:33:22.949 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.ithwzj smithi112 running (9m) 96s ago 9m 66.1M - 17.2.7-1633-g6d025bc2 33ed8dba3996 02786c8a83c7 2024-09-16T00:33:22.949 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.gawwgw smithi169 running (9m) 32s ago 9m 114M - 17.2.7-1633-g6d025bc2 33ed8dba3996 dcf3a4dfa879 2024-09-16T00:33:22.949 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.lgeqrj smithi169 running (9m) 32s ago 9m 60.3M - 17.2.7-1633-g6d025bc2 33ed8dba3996 445c14e55831 2024-09-16T00:33:22.949 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi112.xrkkbc smithi112 *:8443,9283,8765 running (5m) 96s ago 16m 497M - 19.1.1-326-g8921c8ef f1455bd0219c a981f7fd2810 2024-09-16T00:33:22.949 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi169.mtrobk smithi169 *:8443,9283,8765 running (5m) 32s ago 14m 441M - 19.1.1-326-g8921c8ef f1455bd0219c 4096e5369c2d 2024-09-16T00:33:22.949 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi112 smithi112 running (4m) 96s ago 16m 46.4M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 5aaec033c042 2024-09-16T00:33:22.949 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi169 smithi169 running (3m) 32s ago 14m 38.1M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 7bc609155f3f 2024-09-16T00:33:22.950 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi112 smithi112 *:9100 running (6m) 96s ago 15m 15.5M - 1.5.0 0da6a335fe13 6404d1d601e2 2024-09-16T00:33:22.950 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi169 smithi169 *:9100 running (6m) 32s ago 14m 15.0M - 1.5.0 0da6a335fe13 5505baf702a6 2024-09-16T00:33:22.950 INFO:teuthology.orchestra.run.smithi112.stdout:osd.0 smithi112 running (2m) 96s ago 13m 89.4M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 53efc9397017 2024-09-16T00:33:22.950 INFO:teuthology.orchestra.run.smithi112.stdout:osd.1 smithi112 running (2m) 96s ago 13m 89.1M 1557M 19.1.1-326-g8921c8ef f1455bd0219c fe6f1a4a2cf4 2024-09-16T00:33:22.950 INFO:teuthology.orchestra.run.smithi112.stdout:osd.2 smithi112 running (102s) 96s ago 12m 12.1M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 5f7863712057 2024-09-16T00:33:22.950 INFO:teuthology.orchestra.run.smithi112.stdout:osd.3 smithi169 running (67s) 32s ago 12m 90.0M 2581M 19.1.1-326-g8921c8ef f1455bd0219c ee07b5577810 2024-09-16T00:33:22.950 INFO:teuthology.orchestra.run.smithi112.stdout:osd.4 smithi169 running (35s) 32s ago 11m 11.3M 2581M 19.1.1-326-g8921c8ef f1455bd0219c 769f026a54eb 2024-09-16T00:33:22.950 INFO:teuthology.orchestra.run.smithi112.stdout:osd.5 smithi169 starting - - - 2581M 2024-09-16T00:33:22.950 INFO:teuthology.orchestra.run.smithi112.stdout:prometheus.smithi112 smithi112 *:9095 running (5m) 96s ago 14m 63.2M - 2.43.0 a07b618ecd1d 7a1d42e60d2b 2024-09-16T00:33:22.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.950+0000 7fe4f10dd640 1 -- 172.21.15.112:0/886165995 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fe4d406e530 msgr2=0x7fe4d40709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:22.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.950+0000 7fe4f10dd640 1 --2- 172.21.15.112:0/886165995 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fe4d406e530 0x7fe4d40709e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc009ac0 tx=0x7fe4dc035040 comp rx=0 tx=0).stop 2024-09-16T00:33:22.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.950+0000 7fe4f10dd640 1 -- 172.21.15.112:0/886165995 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4ec102f00 msgr2=0x7fe4ec113760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:22.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.950+0000 7fe4f10dd640 1 --2- 172.21.15.112:0/886165995 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4ec102f00 0x7fe4ec113760 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec066a60 tx=0x7fe4e000b040 comp rx=0 tx=0).stop 2024-09-16T00:33:22.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.950+0000 7fe4f10dd640 1 -- 172.21.15.112:0/886165995 shutdown_connections 2024-09-16T00:33:22.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.950+0000 7fe4f10dd640 1 --2- 172.21.15.112:0/886165995 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fe4d406e530 0x7fe4d40709e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:22.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.951+0000 7fe4f10dd640 1 --2- 172.21.15.112:0/886165995 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4ec102f00 0x7fe4ec113760 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:22.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.951+0000 7fe4f10dd640 1 --2- 172.21.15.112:0/886165995 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4ec102560 0x7fe4ec113220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:22.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.951+0000 7fe4f10dd640 1 -- 172.21.15.112:0/886165995 >> 172.21.15.112:0/886165995 conn(0x7fe4ec0f8120 msgr2=0x7fe4ec0f90e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:22.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.951+0000 7fe4f10dd640 1 -- 172.21.15.112:0/886165995 shutdown_connections 2024-09-16T00:33:22.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:22.951+0000 7fe4f10dd640 1 -- 172.21.15.112:0/886165995 wait complete. 2024-09-16T00:33:23.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.031+0000 7fe624f6c640 1 -- 172.21.15.112:0/2685034099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe620064fc0 msgr2=0x7fe620065430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:23.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.031+0000 7fe624f6c640 1 --2- 172.21.15.112:0/2685034099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe620064fc0 0x7fe620065430 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe6140048d0 tx=0x7fe614009da0 comp rx=0 tx=0).stop 2024-09-16T00:33:23.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.032+0000 7fe624f6c640 1 -- 172.21.15.112:0/2685034099 shutdown_connections 2024-09-16T00:33:23.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.032+0000 7fe624f6c640 1 --2- 172.21.15.112:0/2685034099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe620064fc0 0x7fe620065430 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:23.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.032+0000 7fe624f6c640 1 --2- 172.21.15.112:0/2685034099 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6200fe460 0x7fe6200fe830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:23.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.032+0000 7fe624f6c640 1 -- 172.21.15.112:0/2685034099 >> 172.21.15.112:0/2685034099 conn(0x7fe6200723b0 msgr2=0x7fe6200727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:23.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.032+0000 7fe624f6c640 1 -- 172.21.15.112:0/2685034099 shutdown_connections 2024-09-16T00:33:23.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.032+0000 7fe624f6c640 1 -- 172.21.15.112:0/2685034099 wait complete. 2024-09-16T00:33:23.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.032+0000 7fe624f6c640 1 Processor -- start 2024-09-16T00:33:23.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.032+0000 7fe624f6c640 1 -- start start 2024-09-16T00:33:23.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.032+0000 7fe624f6c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe620064fc0 0x7fe6201153b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:23.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.033+0000 7fe624f6c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6200fe460 0x7fe6201158f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:23.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.033+0000 7fe624f6c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe620115ef0 con 0x7fe6200fe460 2024-09-16T00:33:23.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.033+0000 7fe624f6c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe62010f380 con 0x7fe620064fc0 2024-09-16T00:33:23.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.033+0000 7fe61e59c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe620064fc0 0x7fe6201153b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:23.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.033+0000 7fe61dd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6200fe460 0x7fe6201158f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:23.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.033+0000 7fe61e59c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe620064fc0 0x7fe6201153b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.112:43134/0 (socket says 172.21.15.112:43134) 2024-09-16T00:33:23.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.033+0000 7fe61e59c640 1 -- 172.21.15.112:0/456833723 learned_addr learned my addr 172.21.15.112:0/456833723 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:33:23.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.033+0000 7fe61dd9b640 1 -- 172.21.15.112:0/456833723 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe620064fc0 msgr2=0x7fe6201153b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:23.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.033+0000 7fe61dd9b640 1 --2- 172.21.15.112:0/456833723 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe620064fc0 0x7fe6201153b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:23.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.033+0000 7fe61dd9b640 1 -- 172.21.15.112:0/456833723 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe614005040 con 0x7fe6200fe460 2024-09-16T00:33:23.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.033+0000 7fe61e59c640 1 --2- 172.21.15.112:0/456833723 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe620064fc0 0x7fe6201153b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:33:23.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.033+0000 7fe61dd9b640 1 --2- 172.21.15.112:0/456833723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6200fe460 0x7fe6201158f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe61402a530 tx=0x7fe614009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:23.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.033+0000 7fe60effd640 1 -- 172.21.15.112:0/456833723 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe614038070 con 0x7fe6200fe460 2024-09-16T00:33:23.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.033+0000 7fe624f6c640 1 -- 172.21.15.112:0/456833723 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe62010f600 con 0x7fe6200fe460 2024-09-16T00:33:23.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.033+0000 7fe60effd640 1 -- 172.21.15.112:0/456833723 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fe61402ade0 con 0x7fe6200fe460 2024-09-16T00:33:23.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.034+0000 7fe60effd640 1 -- 172.21.15.112:0/456833723 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe61402c7e0 con 0x7fe6200fe460 2024-09-16T00:33:23.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.034+0000 7fe624f6c640 1 -- 172.21.15.112:0/456833723 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe62010fb40 con 0x7fe6200fe460 2024-09-16T00:33:23.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.035+0000 7fe624f6c640 1 -- 172.21.15.112:0/456833723 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6200fdcf0 con 0x7fe6200fe460 2024-09-16T00:33:23.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.035+0000 7fe60effd640 1 -- 172.21.15.112:0/456833723 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fe614030020 con 0x7fe6200fe460 2024-09-16T00:33:23.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.036+0000 7fe60effd640 1 --2- 172.21.15.112:0/456833723 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fe60806e530 0x7fe6080709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:23.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.036+0000 7fe60effd640 1 -- 172.21.15.112:0/456833723 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7030+0+0 (secure 0 0 0) 0x7fe6140adfc0 con 0x7fe6200fe460 2024-09-16T00:33:23.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.036+0000 7fe61e59c640 1 --2- 172.21.15.112:0/456833723 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fe60806e530 0x7fe6080709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:23.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.036+0000 7fe61e59c640 1 --2- 172.21.15.112:0/456833723 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fe60806e530 0x7fe6080709e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe610004550 tx=0x7fe610009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:23.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.040+0000 7fe60effd640 1 -- 172.21.15.112:0/456833723 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fe614076f40 con 0x7fe6200fe460 2024-09-16T00:33:23.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.272+0000 7fe624f6c640 1 -- 172.21.15.112:0/456833723 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe620110a20 con 0x7fe6200fe460 2024-09-16T00:33:23.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.272+0000 7fe60effd640 1 -- 172.21.15.112:0/456833723 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+742 (secure 0 0 0) 0x7fe61407a600 con 0x7fe6200fe460 2024-09-16T00:33:23.273 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:33:23.273 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": { 2024-09-16T00:33:23.273 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:33:23.273 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:33:23.273 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": { 2024-09-16T00:33:23.273 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:33:23.273 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:33:23.273 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": { 2024-09-16T00:33:23.273 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 5 2024-09-16T00:33:23.273 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:33:23.273 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": { 2024-09-16T00:33:23.273 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 17.2.7-1633-g6d025bc2 (6d025bc26f3176c910707f8cb4c8d8e076640615) quincy (stable)": 4 2024-09-16T00:33:23.274 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:33:23.274 INFO:teuthology.orchestra.run.smithi112.stdout: "overall": { 2024-09-16T00:33:23.274 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 17.2.7-1633-g6d025bc2 (6d025bc26f3176c910707f8cb4c8d8e076640615) quincy (stable)": 4, 2024-09-16T00:33:23.274 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 9 2024-09-16T00:33:23.274 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-16T00:33:23.274 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:33:23.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.275+0000 7fe624f6c640 1 -- 172.21.15.112:0/456833723 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fe60806e530 msgr2=0x7fe6080709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:23.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.275+0000 7fe624f6c640 1 --2- 172.21.15.112:0/456833723 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fe60806e530 0x7fe6080709e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe610004550 tx=0x7fe610009040 comp rx=0 tx=0).stop 2024-09-16T00:33:23.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.275+0000 7fe624f6c640 1 -- 172.21.15.112:0/456833723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6200fe460 msgr2=0x7fe6201158f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:23.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.275+0000 7fe624f6c640 1 --2- 172.21.15.112:0/456833723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6200fe460 0x7fe6201158f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe61402a530 tx=0x7fe614009da0 comp rx=0 tx=0).stop 2024-09-16T00:33:23.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.275+0000 7fe624f6c640 1 -- 172.21.15.112:0/456833723 shutdown_connections 2024-09-16T00:33:23.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.275+0000 7fe624f6c640 1 --2- 172.21.15.112:0/456833723 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fe60806e530 0x7fe6080709e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:23.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.275+0000 7fe624f6c640 1 --2- 172.21.15.112:0/456833723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6200fe460 0x7fe6201158f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:23.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.275+0000 7fe624f6c640 1 --2- 172.21.15.112:0/456833723 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe620064fc0 0x7fe6201153b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:23.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.275+0000 7fe624f6c640 1 -- 172.21.15.112:0/456833723 >> 172.21.15.112:0/456833723 conn(0x7fe6200723b0 msgr2=0x7fe6201033e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:23.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.275+0000 7fe624f6c640 1 -- 172.21.15.112:0/456833723 shutdown_connections 2024-09-16T00:33:23.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.275+0000 7fe624f6c640 1 -- 172.21.15.112:0/456833723 wait complete. 2024-09-16T00:33:23.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.354+0000 7f03064bd640 1 -- 172.21.15.112:0/1497998256 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0300102530 msgr2=0x7f0300102900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:23.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.354+0000 7f03064bd640 1 --2- 172.21.15.112:0/1497998256 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0300102530 0x7f0300102900 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f02f00048d0 tx=0x7f02f0009da0 comp rx=0 tx=0).stop 2024-09-16T00:33:23.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.354+0000 7f03064bd640 1 -- 172.21.15.112:0/1497998256 shutdown_connections 2024-09-16T00:33:23.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.354+0000 7f03064bd640 1 --2- 172.21.15.112:0/1497998256 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0300102ed0 0x7f03000fc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:23.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.354+0000 7f03064bd640 1 --2- 172.21.15.112:0/1497998256 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0300102530 0x7f0300102900 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:23.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.354+0000 7f03064bd640 1 -- 172.21.15.112:0/1497998256 >> 172.21.15.112:0/1497998256 conn(0x7f03000f80f0 msgr2=0x7f03000fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:23.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.354+0000 7f03064bd640 1 -- 172.21.15.112:0/1497998256 shutdown_connections 2024-09-16T00:33:23.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.354+0000 7f03064bd640 1 -- 172.21.15.112:0/1497998256 wait complete. 2024-09-16T00:33:23.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.354+0000 7f03064bd640 1 Processor -- start 2024-09-16T00:33:23.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.355+0000 7f03064bd640 1 -- start start 2024-09-16T00:33:23.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.355+0000 7f03064bd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0300102530 0x7f0300113210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:23.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.355+0000 7f03064bd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0300102ed0 0x7f0300113750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:23.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.355+0000 7f03064bd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0300113d50 con 0x7f0300102ed0 2024-09-16T00:33:23.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.355+0000 7f03064bd640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f030010d190 con 0x7f0300102530 2024-09-16T00:33:23.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.355+0000 7f02ff7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0300102ed0 0x7f0300113750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:23.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.355+0000 7f02ff7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0300102ed0 0x7f0300113750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59134/0 (socket says 172.21.15.112:59134) 2024-09-16T00:33:23.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.355+0000 7f02ff7fe640 1 -- 172.21.15.112:0/3399983324 learned_addr learned my addr 172.21.15.112:0/3399983324 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:33:23.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.355+0000 7f02fffff640 1 --2- 172.21.15.112:0/3399983324 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0300102530 0x7f0300113210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:23.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.355+0000 7f02ff7fe640 1 -- 172.21.15.112:0/3399983324 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0300102530 msgr2=0x7f0300113210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:23.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.355+0000 7f02ff7fe640 1 --2- 172.21.15.112:0/3399983324 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0300102530 0x7f0300113210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:23.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.355+0000 7f02ff7fe640 1 -- 172.21.15.112:0/3399983324 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02f0005040 con 0x7f0300102ed0 2024-09-16T00:33:23.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.356+0000 7f02ff7fe640 1 --2- 172.21.15.112:0/3399983324 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0300102ed0 0x7f0300113750 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f02f4002870 tx=0x7f02f4004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:23.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.356+0000 7f02fcff9640 1 -- 172.21.15.112:0/3399983324 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f02f4007c30 con 0x7f0300102ed0 2024-09-16T00:33:23.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.356+0000 7f03064bd640 1 -- 172.21.15.112:0/3399983324 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f030010d470 con 0x7f0300102ed0 2024-09-16T00:33:23.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.356+0000 7f02fcff9640 1 -- 172.21.15.112:0/3399983324 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f02f400f040 con 0x7f0300102ed0 2024-09-16T00:33:23.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.356+0000 7f02fcff9640 1 -- 172.21.15.112:0/3399983324 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f02f4009640 con 0x7f0300102ed0 2024-09-16T00:33:23.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.356+0000 7f03064bd640 1 -- 172.21.15.112:0/3399983324 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f030010d990 con 0x7f0300102ed0 2024-09-16T00:33:23.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.357+0000 7f03064bd640 1 -- 172.21.15.112:0/3399983324 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03000fdcb0 con 0x7f0300102ed0 2024-09-16T00:33:23.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.358+0000 7f02fcff9640 1 -- 172.21.15.112:0/3399983324 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f02f40097a0 con 0x7f0300102ed0 2024-09-16T00:33:23.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.358+0000 7f02fcff9640 1 --2- 172.21.15.112:0/3399983324 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f02e806e530 0x7f02e80709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:23.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.359+0000 7f02fcff9640 1 -- 172.21.15.112:0/3399983324 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7030+0+0 (secure 0 0 0) 0x7f02f408e310 con 0x7f0300102ed0 2024-09-16T00:33:23.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.359+0000 7f02fffff640 1 --2- 172.21.15.112:0/3399983324 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f02e806e530 0x7f02e80709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:23.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.359+0000 7f02fffff640 1 --2- 172.21.15.112:0/3399983324 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f02e806e530 0x7f02e80709e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f02f0009ac0 tx=0x7f02f0035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:23.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.363+0000 7f02fcff9640 1 -- 172.21.15.112:0/3399983324 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f02f405adc0 con 0x7f0300102ed0 2024-09-16T00:33:23.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.568+0000 7f03064bd640 1 -- 172.21.15.112:0/3399983324 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f03000fbd30 con 0x7f0300102ed0 2024-09-16T00:33:23.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.568+0000 7f02fcff9640 1 -- 172.21.15.112:0/3399983324 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) v1 ==== 76+0+1954 (secure 0 0 0) 0x7f02f405a950 con 0x7f0300102ed0 2024-09-16T00:33:23.569 INFO:teuthology.orchestra.run.smithi112.stdout:e23 2024-09-16T00:33:23.569 INFO:teuthology.orchestra.run.smithi112.stdout:btime 2024-09-16T00:30:08:788506+0000 2024-09-16T00:33:23.569 INFO:teuthology.orchestra.run.smithi112.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T00:33:23.569 INFO:teuthology.orchestra.run.smithi112.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:33:23.569 INFO:teuthology.orchestra.run.smithi112.stdout:legacy client fscid: 1 2024-09-16T00:33:23.569 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:33:23.569 INFO:teuthology.orchestra.run.smithi112.stdout:Filesystem 'cephfs' (1) 2024-09-16T00:33:23.570 INFO:teuthology.orchestra.run.smithi112.stdout:fs_name cephfs 2024-09-16T00:33:23.570 INFO:teuthology.orchestra.run.smithi112.stdout:epoch 23 2024-09-16T00:33:23.570 INFO:teuthology.orchestra.run.smithi112.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-16T00:33:23.570 INFO:teuthology.orchestra.run.smithi112.stdout:created 2024-09-16T00:23:22.328199+0000 2024-09-16T00:33:23.570 INFO:teuthology.orchestra.run.smithi112.stdout:modified 2024-09-16T00:30:08.396661+0000 2024-09-16T00:33:23.570 INFO:teuthology.orchestra.run.smithi112.stdout:tableserver 0 2024-09-16T00:33:23.570 INFO:teuthology.orchestra.run.smithi112.stdout:root 0 2024-09-16T00:33:23.570 INFO:teuthology.orchestra.run.smithi112.stdout:session_timeout 60 2024-09-16T00:33:23.570 INFO:teuthology.orchestra.run.smithi112.stdout:session_autoclose 300 2024-09-16T00:33:23.570 INFO:teuthology.orchestra.run.smithi112.stdout:max_file_size 1099511627776 2024-09-16T00:33:23.570 INFO:teuthology.orchestra.run.smithi112.stdout:max_xattr_size 65536 2024-09-16T00:33:23.570 INFO:teuthology.orchestra.run.smithi112.stdout:required_client_features {} 2024-09-16T00:33:23.570 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure 0 2024-09-16T00:33:23.570 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure_osd_epoch 44 2024-09-16T00:33:23.570 INFO:teuthology.orchestra.run.smithi112.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:33:23.571 INFO:teuthology.orchestra.run.smithi112.stdout:max_mds 2 2024-09-16T00:33:23.571 INFO:teuthology.orchestra.run.smithi112.stdout:in 0,1 2024-09-16T00:33:23.571 INFO:teuthology.orchestra.run.smithi112.stdout:up {0=14480,1=24259} 2024-09-16T00:33:23.571 INFO:teuthology.orchestra.run.smithi112.stdout:failed 2024-09-16T00:33:23.571 INFO:teuthology.orchestra.run.smithi112.stdout:damaged 2024-09-16T00:33:23.571 INFO:teuthology.orchestra.run.smithi112.stdout:stopped 2024-09-16T00:33:23.571 INFO:teuthology.orchestra.run.smithi112.stdout:data_pools [3] 2024-09-16T00:33:23.571 INFO:teuthology.orchestra.run.smithi112.stdout:metadata_pool 2 2024-09-16T00:33:23.571 INFO:teuthology.orchestra.run.smithi112.stdout:inline_data enabled 2024-09-16T00:33:23.571 INFO:teuthology.orchestra.run.smithi112.stdout:balancer 2024-09-16T00:33:23.571 INFO:teuthology.orchestra.run.smithi112.stdout:bal_rank_mask -1 2024-09-16T00:33:23.571 INFO:teuthology.orchestra.run.smithi112.stdout:standby_count_wanted 1 2024-09-16T00:33:23.571 INFO:teuthology.orchestra.run.smithi112.stdout:qdb_cluster leader: 14480 members: 24259,14480 2024-09-16T00:33:23.571 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.hxfwkh{0:14480} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.112:6828/486654885,v1:172.21.15.112:6829/486654885] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T00:33:23.572 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.ithwzj{0:14492} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.112:6826/381830565,v1:172.21.15.112:6827/381830565] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T00:33:23.572 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.gawwgw{1:24259} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.169:6824/2892476953,v1:172.21.15.169:6825/2892476953] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T00:33:23.572 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.lgeqrj{1:24269} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.169:6826/861725728,v1:172.21.15.169:6827/861725728] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T00:33:23.572 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:33:23.572 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:33:23.572 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 23 2024-09-16T00:33:23.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.571+0000 7f03064bd640 1 -- 172.21.15.112:0/3399983324 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f02e806e530 msgr2=0x7f02e80709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:23.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.571+0000 7f03064bd640 1 --2- 172.21.15.112:0/3399983324 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f02e806e530 0x7f02e80709e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f02f0009ac0 tx=0x7f02f0035040 comp rx=0 tx=0).stop 2024-09-16T00:33:23.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.571+0000 7f03064bd640 1 -- 172.21.15.112:0/3399983324 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0300102ed0 msgr2=0x7f0300113750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:23.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.571+0000 7f03064bd640 1 --2- 172.21.15.112:0/3399983324 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0300102ed0 0x7f0300113750 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f02f4002870 tx=0x7f02f4004600 comp rx=0 tx=0).stop 2024-09-16T00:33:23.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.571+0000 7f03064bd640 1 -- 172.21.15.112:0/3399983324 shutdown_connections 2024-09-16T00:33:23.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.571+0000 7f03064bd640 1 --2- 172.21.15.112:0/3399983324 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f02e806e530 0x7f02e80709e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:23.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.571+0000 7f03064bd640 1 --2- 172.21.15.112:0/3399983324 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0300102ed0 0x7f0300113750 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:23.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.571+0000 7f03064bd640 1 --2- 172.21.15.112:0/3399983324 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0300102530 0x7f0300113210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:23.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.571+0000 7f03064bd640 1 -- 172.21.15.112:0/3399983324 >> 172.21.15.112:0/3399983324 conn(0x7f03000f80f0 msgr2=0x7f03000f9100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:23.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.571+0000 7f03064bd640 1 -- 172.21.15.112:0/3399983324 shutdown_connections 2024-09-16T00:33:23.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.571+0000 7f03064bd640 1 -- 172.21.15.112:0/3399983324 wait complete. 2024-09-16T00:33:23.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.660+0000 7f1c12997640 1 -- 172.21.15.112:0/1143857067 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c0c06fa50 msgr2=0x7f1c0c06fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:23.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.660+0000 7f1c12997640 1 --2- 172.21.15.112:0/1143857067 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c0c06fa50 0x7f1c0c06fe20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc004810 tx=0x7f1bfc009da0 comp rx=0 tx=0).stop 2024-09-16T00:33:23.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.660+0000 7f1c12997640 1 -- 172.21.15.112:0/1143857067 shutdown_connections 2024-09-16T00:33:23.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.660+0000 7f1c12997640 1 --2- 172.21.15.112:0/1143857067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c0c070360 0x7f1c0c1088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:23.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.660+0000 7f1c12997640 1 --2- 172.21.15.112:0/1143857067 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c0c06fa50 0x7f1c0c06fe20 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:23.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.660+0000 7f1c12997640 1 -- 172.21.15.112:0/1143857067 >> 172.21.15.112:0/1143857067 conn(0x7f1c0c0f80f0 msgr2=0x7f1c0c0fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:23.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.660+0000 7f1c12997640 1 -- 172.21.15.112:0/1143857067 shutdown_connections 2024-09-16T00:33:23.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.660+0000 7f1c12997640 1 -- 172.21.15.112:0/1143857067 wait complete. 2024-09-16T00:33:23.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.661+0000 7f1c12997640 1 Processor -- start 2024-09-16T00:33:23.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.661+0000 7f1c12997640 1 -- start start 2024-09-16T00:33:23.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.661+0000 7f1c12997640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c0c06fa50 0x7f1c0c1153b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:23.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.661+0000 7f1c12997640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c0c070360 0x7f1c0c1158f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:23.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.661+0000 7f1c12997640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c0c115ef0 con 0x7f1c0c070360 2024-09-16T00:33:23.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.661+0000 7f1c12997640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c0c116060 con 0x7f1c0c06fa50 2024-09-16T00:33:23.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.661+0000 7f1c0b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c0c070360 0x7f1c0c1158f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:23.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.661+0000 7f1c0b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c0c070360 0x7f1c0c1158f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59148/0 (socket says 172.21.15.112:59148) 2024-09-16T00:33:23.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.661+0000 7f1c0b7fe640 1 -- 172.21.15.112:0/1788277337 learned_addr learned my addr 172.21.15.112:0/1788277337 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:33:23.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.661+0000 7f1c0bfff640 1 --2- 172.21.15.112:0/1788277337 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c0c06fa50 0x7f1c0c1153b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:23.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.662+0000 7f1c0b7fe640 1 -- 172.21.15.112:0/1788277337 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c0c06fa50 msgr2=0x7f1c0c1153b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:23.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.662+0000 7f1c0b7fe640 1 --2- 172.21.15.112:0/1788277337 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c0c06fa50 0x7f1c0c1153b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:23.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.662+0000 7f1c0b7fe640 1 -- 172.21.15.112:0/1788277337 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c00005040 con 0x7f1c0c070360 2024-09-16T00:33:23.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.662+0000 7f1c0bfff640 1 --2- 172.21.15.112:0/1788277337 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c0c06fa50 0x7f1c0c1153b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:33:23.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.662+0000 7f1c0b7fe640 1 --2- 172.21.15.112:0/1788277337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c0c070360 0x7f1c0c1158f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1c000094d0 tx=0x7f1c00009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:23.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.662+0000 7f1c08ff9640 1 -- 172.21.15.112:0/1788277337 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c0000e8b0 con 0x7f1c0c070360 2024-09-16T00:33:23.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.662+0000 7f1c08ff9640 1 -- 172.21.15.112:0/1788277337 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f1c0000f040 con 0x7f1c0c070360 2024-09-16T00:33:23.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.662+0000 7f1c12997640 1 -- 172.21.15.112:0/1788277337 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bfc005040 con 0x7f1c0c070360 2024-09-16T00:33:23.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.662+0000 7f1c08ff9640 1 -- 172.21.15.112:0/1788277337 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c00013d90 con 0x7f1c0c070360 2024-09-16T00:33:23.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.662+0000 7f1c12997640 1 -- 172.21.15.112:0/1788277337 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c0c10f870 con 0x7f1c0c070360 2024-09-16T00:33:23.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.663+0000 7f1c12997640 1 -- 172.21.15.112:0/1788277337 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c0c071190 con 0x7f1c0c070360 2024-09-16T00:33:23.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.664+0000 7f1c08ff9640 1 -- 172.21.15.112:0/1788277337 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f1c0000ea10 con 0x7f1c0c070360 2024-09-16T00:33:23.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.664+0000 7f1c08ff9640 1 --2- 172.21.15.112:0/1788277337 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f1bf406e530 0x7f1bf40709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:23.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.664+0000 7f1c08ff9640 1 -- 172.21.15.112:0/1788277337 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7030+0+0 (secure 0 0 0) 0x7f1c0008e1c0 con 0x7f1c0c070360 2024-09-16T00:33:23.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.664+0000 7f1c0bfff640 1 --2- 172.21.15.112:0/1788277337 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f1bf406e530 0x7f1bf40709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:23.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.665+0000 7f1c0bfff640 1 --2- 172.21.15.112:0/1788277337 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f1bf406e530 0x7f1bf40709e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc007ac0 tx=0x7f1bfc035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:23.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.668+0000 7f1c08ff9640 1 -- 172.21.15.112:0/1788277337 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f1c00057090 con 0x7f1c0c070360 2024-09-16T00:33:23.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.854+0000 7f1c12997640 1 -- 172.21.15.112:0/1788277337 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1c0c04b720 con 0x7f1bf406e530 2024-09-16T00:33:23.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.855+0000 7f1c08ff9640 1 -- 172.21.15.112:0/1788277337 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f1c0c04b720 con 0x7f1bf406e530 2024-09-16T00:33:23.856 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:33:23.856 INFO:teuthology.orchestra.run.smithi112.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80", 2024-09-16T00:33:23.856 INFO:teuthology.orchestra.run.smithi112.stdout: "in_progress": true, 2024-09-16T00:33:23.856 INFO:teuthology.orchestra.run.smithi112.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T00:33:23.856 INFO:teuthology.orchestra.run.smithi112.stdout: "services_complete": [ 2024-09-16T00:33:23.856 INFO:teuthology.orchestra.run.smithi112.stdout: "crash", 2024-09-16T00:33:23.856 INFO:teuthology.orchestra.run.smithi112.stdout: "osd", 2024-09-16T00:33:23.856 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr", 2024-09-16T00:33:23.856 INFO:teuthology.orchestra.run.smithi112.stdout: "mon" 2024-09-16T00:33:23.856 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-16T00:33:23.856 INFO:teuthology.orchestra.run.smithi112.stdout: "progress": "12/21 daemons upgraded", 2024-09-16T00:33:23.857 INFO:teuthology.orchestra.run.smithi112.stdout: "message": "Currently upgrading osd daemons", 2024-09-16T00:33:23.857 INFO:teuthology.orchestra.run.smithi112.stdout: "is_paused": false 2024-09-16T00:33:23.857 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:33:23.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.857+0000 7f1c12997640 1 -- 172.21.15.112:0/1788277337 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f1bf406e530 msgr2=0x7f1bf40709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:23.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.857+0000 7f1c12997640 1 --2- 172.21.15.112:0/1788277337 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f1bf406e530 0x7f1bf40709e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc007ac0 tx=0x7f1bfc035040 comp rx=0 tx=0).stop 2024-09-16T00:33:23.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.857+0000 7f1c12997640 1 -- 172.21.15.112:0/1788277337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c0c070360 msgr2=0x7f1c0c1158f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:23.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.857+0000 7f1c12997640 1 --2- 172.21.15.112:0/1788277337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c0c070360 0x7f1c0c1158f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1c000094d0 tx=0x7f1c00009da0 comp rx=0 tx=0).stop 2024-09-16T00:33:23.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.857+0000 7f1c12997640 1 -- 172.21.15.112:0/1788277337 shutdown_connections 2024-09-16T00:33:23.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.857+0000 7f1c12997640 1 --2- 172.21.15.112:0/1788277337 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f1bf406e530 0x7f1bf40709e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:23.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.857+0000 7f1c12997640 1 --2- 172.21.15.112:0/1788277337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c0c070360 0x7f1c0c1158f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:23.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.857+0000 7f1c12997640 1 --2- 172.21.15.112:0/1788277337 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c0c06fa50 0x7f1c0c1153b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:23.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.857+0000 7f1c12997640 1 -- 172.21.15.112:0/1788277337 >> 172.21.15.112:0/1788277337 conn(0x7f1c0c0f80f0 msgr2=0x7f1c0c0f9620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:23.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.858+0000 7f1c12997640 1 -- 172.21.15.112:0/1788277337 shutdown_connections 2024-09-16T00:33:23.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.858+0000 7f1c12997640 1 -- 172.21.15.112:0/1788277337 wait complete. 2024-09-16T00:33:23.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.935+0000 7f7c0bab5640 1 -- 172.21.15.112:0/4060734100 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c04102ef0 msgr2=0x7f7c040fc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:23.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.935+0000 7f7c0bab5640 1 --2- 172.21.15.112:0/4060734100 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c04102ef0 0x7f7c040fc5b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7bf40048d0 tx=0x7f7bf4009da0 comp rx=0 tx=0).stop 2024-09-16T00:33:23.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.935+0000 7f7c0bab5640 1 -- 172.21.15.112:0/4060734100 shutdown_connections 2024-09-16T00:33:23.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.935+0000 7f7c0bab5640 1 --2- 172.21.15.112:0/4060734100 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c04102ef0 0x7f7c040fc5b0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:23.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.935+0000 7f7c0bab5640 1 --2- 172.21.15.112:0/4060734100 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c04102550 0x7f7c04102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:23.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.935+0000 7f7c0bab5640 1 -- 172.21.15.112:0/4060734100 >> 172.21.15.112:0/4060734100 conn(0x7f7c040f80f0 msgr2=0x7f7c040fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:23.937 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:23 smithi112 ceph-mon[97750]: from='client.44275 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:33:23.937 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:23 smithi112 ceph-mon[97750]: pgmap v158: 65 pgs: 2 active+remapped, 1 active+undersized+degraded+remapped, 5 active+clean+remapped, 2 remapped+peering, 14 active+undersized, 13 active+undersized+degraded, 28 active+clean; 325 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 53/372 objects degraded (14.247%); 19/372 objects misplaced (5.108%); 2.4 MiB/s, 0 objects/s recovering 2024-09-16T00:33:23.937 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:23 smithi112 ceph-mon[97750]: from='client.34352 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:33:23.937 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:23 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:33:23.937 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:23 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:33:23.937 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:23 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/456833723' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:33:23.937 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:23 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/3399983324' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:33:23.937 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:23 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:33:23.937 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:23 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:33:23.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.936+0000 7f7c0bab5640 1 -- 172.21.15.112:0/4060734100 shutdown_connections 2024-09-16T00:33:23.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.936+0000 7f7c0bab5640 1 -- 172.21.15.112:0/4060734100 wait complete. 2024-09-16T00:33:23.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.936+0000 7f7c0bab5640 1 Processor -- start 2024-09-16T00:33:23.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.936+0000 7f7c0bab5640 1 -- start start 2024-09-16T00:33:23.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.936+0000 7f7c0bab5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c04102550 0x7f7c04113500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:23.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.936+0000 7f7c0bab5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c04102ef0 0x7f7c04113a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:23.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.936+0000 7f7c0bab5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c04114090 con 0x7f7c04102550 2024-09-16T00:33:23.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.936+0000 7f7c0bab5640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0410d4a0 con 0x7f7c04102ef0 2024-09-16T00:33:23.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.937+0000 7f7c09851640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c04102550 0x7f7c04113500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:23.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.937+0000 7f7c09851640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c04102550 0x7f7c04113500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59162/0 (socket says 172.21.15.112:59162) 2024-09-16T00:33:23.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.937+0000 7f7c09851640 1 -- 172.21.15.112:0/2558345292 learned_addr learned my addr 172.21.15.112:0/2558345292 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:33:23.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.937+0000 7f7c09050640 1 --2- 172.21.15.112:0/2558345292 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c04102ef0 0x7f7c04113a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:23.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.937+0000 7f7c09851640 1 -- 172.21.15.112:0/2558345292 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c04102ef0 msgr2=0x7f7c04113a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:23.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.937+0000 7f7c09851640 1 --2- 172.21.15.112:0/2558345292 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c04102ef0 0x7f7c04113a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:23.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.937+0000 7f7c09851640 1 -- 172.21.15.112:0/2558345292 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bf4005040 con 0x7f7c04102550 2024-09-16T00:33:23.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.937+0000 7f7c09851640 1 --2- 172.21.15.112:0/2558345292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c04102550 0x7f7c04113500 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7c0000eb70 tx=0x7f7c0000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:23.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.937+0000 7f7bfa7fc640 1 -- 172.21.15.112:0/2558345292 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c00007c30 con 0x7f7c04102550 2024-09-16T00:33:23.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.937+0000 7f7c0bab5640 1 -- 172.21.15.112:0/2558345292 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c0410d720 con 0x7f7c04102550 2024-09-16T00:33:23.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.937+0000 7f7bfa7fc640 1 -- 172.21.15.112:0/2558345292 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f7c0000f040 con 0x7f7c04102550 2024-09-16T00:33:23.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.937+0000 7f7c0bab5640 1 -- 172.21.15.112:0/2558345292 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c0410dc90 con 0x7f7c04102550 2024-09-16T00:33:23.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.937+0000 7f7bfa7fc640 1 -- 172.21.15.112:0/2558345292 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c000094e0 con 0x7f7c04102550 2024-09-16T00:33:23.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.938+0000 7f7c0bab5640 1 -- 172.21.15.112:0/2558345292 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c040fdcd0 con 0x7f7c04102550 2024-09-16T00:33:23.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.939+0000 7f7bfa7fc640 1 -- 172.21.15.112:0/2558345292 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f7c00009780 con 0x7f7c04102550 2024-09-16T00:33:23.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.940+0000 7f7bfa7fc640 1 --2- 172.21.15.112:0/2558345292 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f7bf006e530 0x7f7bf00709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:23.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.940+0000 7f7bfa7fc640 1 -- 172.21.15.112:0/2558345292 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7030+0+0 (secure 0 0 0) 0x7f7c0008df70 con 0x7f7c04102550 2024-09-16T00:33:23.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.940+0000 7f7c09050640 1 --2- 172.21.15.112:0/2558345292 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f7bf006e530 0x7f7bf00709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:23.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.941+0000 7f7c09050640 1 --2- 172.21.15.112:0/2558345292 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f7bf006e530 0x7f7bf00709e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7bf4000c00 tx=0x7f7bf4035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:23.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:23.944+0000 7f7bfa7fc640 1 -- 172.21.15.112:0/2558345292 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f7c000c31b0 con 0x7f7c04102550 2024-09-16T00:33:24.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:23 smithi169 ceph-mon[77626]: from='client.44275 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:33:24.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:23 smithi169 ceph-mon[77626]: pgmap v158: 65 pgs: 2 active+remapped, 1 active+undersized+degraded+remapped, 5 active+clean+remapped, 2 remapped+peering, 14 active+undersized, 13 active+undersized+degraded, 28 active+clean; 325 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 53/372 objects degraded (14.247%); 19/372 objects misplaced (5.108%); 2.4 MiB/s, 0 objects/s recovering 2024-09-16T00:33:24.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:23 smithi169 ceph-mon[77626]: from='client.34352 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:33:24.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:23 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:33:24.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:23 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:33:24.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:23 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/456833723' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:33:24.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:23 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/3399983324' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:33:24.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:23 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:33:24.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:23 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:33:24.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:24.174+0000 7f7c0bab5640 1 -- 172.21.15.112:0/2558345292 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f7c0404b7b0 con 0x7f7c04102550 2024-09-16T00:33:24.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:24.175+0000 7f7bfa7fc640 1 -- 172.21.15.112:0/2558345292 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1489 (secure 0 0 0) 0x7f7c00019090 con 0x7f7c04102550 2024-09-16T00:33:24.175 INFO:teuthology.orchestra.run.smithi112.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; 1 osds down; all OSDs are running squid or later but require_osd_release < squid; Degraded data redundancy: 53/372 objects degraded (14.247%), 14 pgs degraded 2024-09-16T00:33:24.176 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T00:33:24.176 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T00:33:24.176 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] OSD_DOWN: 1 osds down 2024-09-16T00:33:24.176 INFO:teuthology.orchestra.run.smithi112.stdout: osd.5 (root=default,host=smithi169) is down 2024-09-16T00:33:24.176 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] OSD_UPGRADE_FINISHED: all OSDs are running squid or later but require_osd_release < squid 2024-09-16T00:33:24.176 INFO:teuthology.orchestra.run.smithi112.stdout: all OSDs are running squid or later but require_osd_release < squid 2024-09-16T00:33:24.176 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 53/372 objects degraded (14.247%), 14 pgs degraded 2024-09-16T00:33:24.176 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.2 is active+undersized+degraded, acting [1,0] 2024-09-16T00:33:24.176 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.3 is active+undersized+degraded, acting [2,1] 2024-09-16T00:33:24.176 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.8 is active+undersized+degraded, acting [3,0] 2024-09-16T00:33:24.176 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.b is active+undersized+degraded, acting [3,4] 2024-09-16T00:33:24.176 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.f is active+undersized+degraded, acting [4,0] 2024-09-16T00:33:24.177 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.14 is active+undersized+degraded, acting [3,4] 2024-09-16T00:33:24.177 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.16 is active+undersized+degraded, acting [3,2] 2024-09-16T00:33:24.177 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.17 is active+undersized+degraded, acting [2,1] 2024-09-16T00:33:24.177 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.18 is active+undersized+degraded, acting [4,3] 2024-09-16T00:33:24.177 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.1a is active+undersized+degraded, acting [3,4] 2024-09-16T00:33:24.177 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.1b is active+undersized+degraded+remapped, acting [1,0] 2024-09-16T00:33:24.177 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.1c is active+undersized+degraded, acting [4,2] 2024-09-16T00:33:24.177 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.1d is active+undersized+degraded, acting [3,0] 2024-09-16T00:33:24.177 INFO:teuthology.orchestra.run.smithi112.stdout: pg 2.1e is active+undersized+degraded, acting [2,0] 2024-09-16T00:33:24.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:24.177+0000 7f7c0bab5640 1 -- 172.21.15.112:0/2558345292 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f7bf006e530 msgr2=0x7f7bf00709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:24.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:24.177+0000 7f7c0bab5640 1 --2- 172.21.15.112:0/2558345292 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f7bf006e530 0x7f7bf00709e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7bf4000c00 tx=0x7f7bf4035040 comp rx=0 tx=0).stop 2024-09-16T00:33:24.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:24.177+0000 7f7c0bab5640 1 -- 172.21.15.112:0/2558345292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c04102550 msgr2=0x7f7c04113500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:24.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:24.177+0000 7f7c0bab5640 1 --2- 172.21.15.112:0/2558345292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c04102550 0x7f7c04113500 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7c0000eb70 tx=0x7f7c0000b040 comp rx=0 tx=0).stop 2024-09-16T00:33:24.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:24.177+0000 7f7c0bab5640 1 -- 172.21.15.112:0/2558345292 shutdown_connections 2024-09-16T00:33:24.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:24.177+0000 7f7c0bab5640 1 --2- 172.21.15.112:0/2558345292 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f7bf006e530 0x7f7bf00709e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:24.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:24.177+0000 7f7c0bab5640 1 --2- 172.21.15.112:0/2558345292 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c04102ef0 0x7f7c04113a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:24.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:24.177+0000 7f7c0bab5640 1 --2- 172.21.15.112:0/2558345292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c04102550 0x7f7c04113500 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:24.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:24.177+0000 7f7c0bab5640 1 -- 172.21.15.112:0/2558345292 >> 172.21.15.112:0/2558345292 conn(0x7f7c040f80f0 msgr2=0x7f7c040f9150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:24.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:24.177+0000 7f7c0bab5640 1 -- 172.21.15.112:0/2558345292 shutdown_connections 2024-09-16T00:33:24.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:24.177+0000 7f7c0bab5640 1 -- 172.21.15.112:0/2558345292 wait complete. 2024-09-16T00:33:24.978 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:24 smithi169 ceph-mon[77626]: from='client.44283 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:33:24.978 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:24 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/2558345292' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:33:25.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:24 smithi112 ceph-mon[97750]: from='client.44283 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:33:25.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:24 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/2558345292' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:33:26.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:25 smithi169 ceph-mon[77626]: from='client.34368 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:33:26.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:25 smithi169 ceph-mon[77626]: pgmap v159: 65 pgs: 10 active+clean+remapped, 14 active+undersized, 13 active+undersized+degraded, 28 active+clean; 325 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 53/372 objects degraded (14.247%); 22/372 objects misplaced (5.914%); 2.0 MiB/s, 0 objects/s recovering 2024-09-16T00:33:26.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:25 smithi169 ceph-mon[77626]: Health check update: Degraded data redundancy: 53/372 objects degraded (14.247%), 13 pgs degraded (PG_DEGRADED) 2024-09-16T00:33:26.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:25 smithi112 ceph-mon[97750]: from='client.34368 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:33:26.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:25 smithi112 ceph-mon[97750]: pgmap v159: 65 pgs: 10 active+clean+remapped, 14 active+undersized, 13 active+undersized+degraded, 28 active+clean; 325 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 53/372 objects degraded (14.247%); 22/372 objects misplaced (5.914%); 2.0 MiB/s, 0 objects/s recovering 2024-09-16T00:33:26.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:25 smithi112 ceph-mon[97750]: Health check update: Degraded data redundancy: 53/372 objects degraded (14.247%), 13 pgs degraded (PG_DEGRADED) 2024-09-16T00:33:27.526 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:33:27 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5[96391]: 2024-09-16T00:33:27.275+0000 7f4ed8b7f640 -1 osd.5 72 log_to_monitors true 2024-09-16T00:33:28.026 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:33:27 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5[96391]: 2024-09-16T00:33:27.882+0000 7f4ed0919640 -1 osd.5 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T00:33:28.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:27 smithi169 ceph-mon[77626]: pgmap v160: 65 pgs: 10 active+clean+remapped, 14 active+undersized, 13 active+undersized+degraded, 28 active+clean; 325 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 53/372 objects degraded (14.247%); 22/372 objects misplaced (5.914%); 1.5 MiB/s, 0 objects/s recovering 2024-09-16T00:33:28.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:27 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:33:28.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:27 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:33:28.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:27 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:33:28.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:27 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:33:28.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:27 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:33:28.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:27 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:33:28.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:27 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:33:28.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:27 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:33:28.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:27 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:33:28.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:27 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:33:28.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:27 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:33:28.028 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:27 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-09-16T00:33:28.028 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:27 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-09-16T00:33:28.028 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:27 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-09-16T00:33:28.028 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:27 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-09-16T00:33:28.028 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:27 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-09-16T00:33:28.028 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:27 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-09-16T00:33:28.028 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:27 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-09-16T00:33:28.028 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:27 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-09-16T00:33:28.028 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:27 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-09-16T00:33:28.028 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:27 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-09-16T00:33:28.028 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:27 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-09-16T00:33:28.028 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:27 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-09-16T00:33:28.028 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:27 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-09-16T00:33:28.029 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:27 smithi169 ceph-mon[77626]: from='osd.5 [v2:172.21.15.169:6816/3774670994,v1:172.21.15.169:6817/3774670994]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T00:33:28.029 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:27 smithi169 ceph-mon[77626]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T00:33:28.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:27 smithi112 ceph-mon[97750]: pgmap v160: 65 pgs: 10 active+clean+remapped, 14 active+undersized, 13 active+undersized+degraded, 28 active+clean; 325 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 53/372 objects degraded (14.247%); 22/372 objects misplaced (5.914%); 1.5 MiB/s, 0 objects/s recovering 2024-09-16T00:33:28.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:27 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:33:28.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:27 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:33:28.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:27 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:33:28.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:27 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:33:28.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:27 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:33:28.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:27 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:33:28.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:27 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:33:28.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:27 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:33:28.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:27 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:33:28.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:27 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:33:28.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:27 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:33:28.091 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:27 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-09-16T00:33:28.091 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:27 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-09-16T00:33:28.091 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:27 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-09-16T00:33:28.091 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:27 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-09-16T00:33:28.091 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:27 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-09-16T00:33:28.091 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:27 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-09-16T00:33:28.091 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:27 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-09-16T00:33:28.091 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:27 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-09-16T00:33:28.091 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:27 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-09-16T00:33:28.091 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:27 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-09-16T00:33:28.091 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:27 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-09-16T00:33:28.091 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:27 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-09-16T00:33:28.092 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:27 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-09-16T00:33:28.092 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:27 smithi112 ceph-mon[97750]: from='osd.5 [v2:172.21.15.169:6816/3774670994,v1:172.21.15.169:6817/3774670994]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T00:33:28.092 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:27 smithi112 ceph-mon[97750]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T00:33:29.012 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:28 smithi112 ceph-mon[97750]: Upgrade: Setting container_image for all osd 2024-09-16T00:33:29.013 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:28 smithi112 ceph-mon[97750]: Upgrade: Setting require_osd_release to 19 squid 2024-09-16T00:33:29.013 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:28 smithi112 ceph-mon[97750]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-09-16T00:33:29.013 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:28 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-09-16T00:33:29.013 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:28 smithi112 ceph-mon[97750]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-16T00:33:29.013 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:28 smithi112 ceph-mon[97750]: from='osd.5 [v2:172.21.15.169:6816/3774670994,v1:172.21.15.169:6817/3774670994]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-09-16T00:33:29.013 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:28 smithi112 ceph-mon[97750]: osdmap e80: 6 total, 5 up, 6 in 2024-09-16T00:33:29.013 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:28 smithi112 ceph-mon[97750]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-09-16T00:33:29.013 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:28 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:33:29.013 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:28 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]: dispatch 2024-09-16T00:33:29.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:28 smithi169 ceph-mon[77626]: Upgrade: Setting container_image for all osd 2024-09-16T00:33:29.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:28 smithi169 ceph-mon[77626]: Upgrade: Setting require_osd_release to 19 squid 2024-09-16T00:33:29.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:28 smithi169 ceph-mon[77626]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-09-16T00:33:29.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:28 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-09-16T00:33:29.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:28 smithi169 ceph-mon[77626]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-16T00:33:29.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:28 smithi169 ceph-mon[77626]: from='osd.5 [v2:172.21.15.169:6816/3774670994,v1:172.21.15.169:6817/3774670994]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-09-16T00:33:29.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:28 smithi169 ceph-mon[77626]: osdmap e80: 6 total, 5 up, 6 in 2024-09-16T00:33:29.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:28 smithi169 ceph-mon[77626]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-09-16T00:33:29.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:28 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:33:29.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:28 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]: dispatch 2024-09-16T00:33:29.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:29 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-mon-smithi112[97746]: 2024-09-16T00:33:29.012+0000 7fccfb907640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T00:33:29.967 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:29 smithi112 ceph-mon[97750]: Upgrade: Disabling standby-replay for filesystem cephfs 2024-09-16T00:33:29.967 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:29 smithi112 ceph-mon[97750]: pgmap v162: 65 pgs: 10 active+clean+remapped, 14 active+undersized, 13 active+undersized+degraded, 28 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 53/372 objects degraded (14.247%); 22/372 objects misplaced (5.914%) 2024-09-16T00:33:29.967 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:29 smithi112 ceph-mon[97750]: osdmap e81: 6 total, 5 up, 6 in 2024-09-16T00:33:29.967 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:29 smithi112 ceph-mon[97750]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-09-16T00:33:29.967 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:29 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]': finished 2024-09-16T00:33:29.967 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:29 smithi112 ceph-mon[97750]: fsmap cephfs:2 {0=cephfs.smithi112.hxfwkh=up:active,1=cephfs.smithi169.gawwgw=up:active} 2024-09-16T00:33:29.967 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:29 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:33:29.968 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:29 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:33:29.968 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:29 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:33:29.968 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:29 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:33:29.968 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:29 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:33:29.968 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:29 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:33:29.968 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:29 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:33:29.968 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:29 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:33:29.968 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:29 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:33:29.968 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:29 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-16T00:33:29.968 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:29 smithi112 ceph-mon[97750]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-16T00:33:29.968 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:29 smithi112 ceph-mon[97750]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-16T00:33:29.968 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:29 smithi112 ceph-mon[97750]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T00:33:29.968 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:29 smithi112 ceph-mon[97750]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-09-16T00:33:29.968 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:29 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-16T00:33:29.969 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:29 smithi112 ceph-mon[97750]: osd.5 [v2:172.21.15.169:6816/3774670994,v1:172.21.15.169:6817/3774670994] boot 2024-09-16T00:33:29.969 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:29 smithi112 ceph-mon[97750]: osdmap e82: 6 total, 6 up, 6 in 2024-09-16T00:33:29.969 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:29 smithi112 ceph-mon[97750]: mds.? [v2:172.21.15.169:6826/3569474985,v1:172.21.15.169:6827/3569474985] up:boot 2024-09-16T00:33:29.969 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:29 smithi112 ceph-mon[97750]: mds.? [v2:172.21.15.112:6826/3994140647,v1:172.21.15.112:6827/3994140647] up:boot 2024-09-16T00:33:29.969 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:29 smithi112 ceph-mon[97750]: fsmap cephfs:0/2 2 up:standby, 2 failed 2024-09-16T00:33:29.969 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:29 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi112.ithwzj"}]: dispatch 2024-09-16T00:33:29.969 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:29 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi169.lgeqrj"}]: dispatch 2024-09-16T00:33:29.969 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:29 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:33:30.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:29 smithi169 ceph-mon[77626]: Upgrade: Disabling standby-replay for filesystem cephfs 2024-09-16T00:33:30.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:29 smithi169 ceph-mon[77626]: pgmap v162: 65 pgs: 10 active+clean+remapped, 14 active+undersized, 13 active+undersized+degraded, 28 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 53/372 objects degraded (14.247%); 22/372 objects misplaced (5.914%) 2024-09-16T00:33:30.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:29 smithi169 ceph-mon[77626]: osdmap e81: 6 total, 5 up, 6 in 2024-09-16T00:33:30.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:29 smithi169 ceph-mon[77626]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-09-16T00:33:30.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:29 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]': finished 2024-09-16T00:33:30.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:29 smithi169 ceph-mon[77626]: fsmap cephfs:2 {0=cephfs.smithi112.hxfwkh=up:active,1=cephfs.smithi169.gawwgw=up:active} 2024-09-16T00:33:30.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:29 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:33:30.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:29 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:33:30.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:29 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:33:30.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:29 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:33:30.028 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:29 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:33:30.028 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:29 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:33:30.028 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:29 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:33:30.028 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:29 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:33:30.028 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:29 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:33:30.028 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:29 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-16T00:33:30.028 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:29 smithi169 ceph-mon[77626]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-16T00:33:30.028 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:29 smithi169 ceph-mon[77626]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-16T00:33:30.028 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:29 smithi169 ceph-mon[77626]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T00:33:30.028 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:29 smithi169 ceph-mon[77626]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-09-16T00:33:30.028 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:29 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-16T00:33:30.028 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:29 smithi169 ceph-mon[77626]: osd.5 [v2:172.21.15.169:6816/3774670994,v1:172.21.15.169:6817/3774670994] boot 2024-09-16T00:33:30.028 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:29 smithi169 ceph-mon[77626]: osdmap e82: 6 total, 6 up, 6 in 2024-09-16T00:33:30.028 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:29 smithi169 ceph-mon[77626]: mds.? [v2:172.21.15.169:6826/3569474985,v1:172.21.15.169:6827/3569474985] up:boot 2024-09-16T00:33:30.029 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:29 smithi169 ceph-mon[77626]: mds.? [v2:172.21.15.112:6826/3994140647,v1:172.21.15.112:6827/3994140647] up:boot 2024-09-16T00:33:30.029 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:29 smithi169 ceph-mon[77626]: fsmap cephfs:0/2 2 up:standby, 2 failed 2024-09-16T00:33:30.029 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:29 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi112.ithwzj"}]: dispatch 2024-09-16T00:33:30.029 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:29 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi169.lgeqrj"}]: dispatch 2024-09-16T00:33:30.029 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:29 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:33:30.979 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:30 smithi112 ceph-mon[97750]: Upgrade: failing fs cephfs for rapid multi-rank mds upgrade 2024-09-16T00:33:30.979 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:30 smithi112 ceph-mon[97750]: osdmap e83: 6 total, 6 up, 6 in 2024-09-16T00:33:30.979 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:30 smithi112 ceph-mon[97750]: mds.? [v2:172.21.15.169:6824/3147876558,v1:172.21.15.169:6825/3147876558] up:boot 2024-09-16T00:33:30.980 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:30 smithi112 ceph-mon[97750]: mds.? [v2:172.21.15.112:6828/1731640001,v1:172.21.15.112:6829/1731640001] up:boot 2024-09-16T00:33:30.980 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:30 smithi112 ceph-mon[97750]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-09-16T00:33:30.980 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:30 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi169.gawwgw"}]: dispatch 2024-09-16T00:33:30.980 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:30 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi112.hxfwkh"}]: dispatch 2024-09-16T00:33:30.980 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:30 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:33:30.980 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:30 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi112.ithwzj", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:33:30.980 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:30 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:33:31.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:30 smithi169 ceph-mon[77626]: Upgrade: failing fs cephfs for rapid multi-rank mds upgrade 2024-09-16T00:33:31.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:30 smithi169 ceph-mon[77626]: osdmap e83: 6 total, 6 up, 6 in 2024-09-16T00:33:31.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:30 smithi169 ceph-mon[77626]: mds.? [v2:172.21.15.169:6824/3147876558,v1:172.21.15.169:6825/3147876558] up:boot 2024-09-16T00:33:31.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:30 smithi169 ceph-mon[77626]: mds.? [v2:172.21.15.112:6828/1731640001,v1:172.21.15.112:6829/1731640001] up:boot 2024-09-16T00:33:31.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:30 smithi169 ceph-mon[77626]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-09-16T00:33:31.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:30 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi169.gawwgw"}]: dispatch 2024-09-16T00:33:31.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:30 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi112.hxfwkh"}]: dispatch 2024-09-16T00:33:31.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:30 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:33:31.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:30 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi112.ithwzj", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:33:31.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:30 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:33:32.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:31 smithi169 ceph-mon[77626]: Upgrade: Updating mds.cephfs.smithi112.ithwzj 2024-09-16T00:33:32.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:31 smithi169 ceph-mon[77626]: Deploying daemon mds.cephfs.smithi112.ithwzj on smithi112 2024-09-16T00:33:32.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:31 smithi169 ceph-mon[77626]: pgmap v166: 65 pgs: 1 peering, 3 remapped+peering, 7 active+clean+remapped, 13 active+undersized, 13 active+undersized+degraded, 28 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 3 op/s; 53/372 objects degraded (14.247%); 12/372 objects misplaced (3.226%) 2024-09-16T00:33:32.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:31 smithi169 ceph-mon[77626]: osdmap e84: 6 total, 6 up, 6 in 2024-09-16T00:33:32.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:31 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:33:32.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:31 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:33:32.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:31 smithi112 ceph-mon[97750]: Upgrade: Updating mds.cephfs.smithi112.ithwzj 2024-09-16T00:33:32.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:31 smithi112 ceph-mon[97750]: Deploying daemon mds.cephfs.smithi112.ithwzj on smithi112 2024-09-16T00:33:32.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:31 smithi112 ceph-mon[97750]: pgmap v166: 65 pgs: 1 peering, 3 remapped+peering, 7 active+clean+remapped, 13 active+undersized, 13 active+undersized+degraded, 28 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 3 op/s; 53/372 objects degraded (14.247%); 12/372 objects misplaced (3.226%) 2024-09-16T00:33:32.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:31 smithi112 ceph-mon[97750]: osdmap e84: 6 total, 6 up, 6 in 2024-09-16T00:33:32.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:31 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:33:32.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:31 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:33:33.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:33 smithi112 ceph-mon[97750]: osdmap e85: 6 total, 6 up, 6 in 2024-09-16T00:33:33.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:33 smithi112 ceph-mon[97750]: pgmap v169: 65 pgs: 1 peering, 5 remapped+peering, 5 active+clean+remapped, 12 active+undersized, 11 active+undersized+degraded, 31 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 3 op/s; 49/372 objects degraded (13.172%); 3/372 objects misplaced (0.806%) 2024-09-16T00:33:33.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:33 smithi169 ceph-mon[77626]: osdmap e85: 6 total, 6 up, 6 in 2024-09-16T00:33:33.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:33 smithi169 ceph-mon[77626]: pgmap v169: 65 pgs: 1 peering, 5 remapped+peering, 5 active+clean+remapped, 12 active+undersized, 11 active+undersized+degraded, 31 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 3 op/s; 49/372 objects degraded (13.172%); 3/372 objects misplaced (0.806%) 2024-09-16T00:33:34.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:34 smithi112 ceph-mon[97750]: Health check update: Degraded data redundancy: 49/372 objects degraded (13.172%), 11 pgs degraded (PG_DEGRADED) 2024-09-16T00:33:34.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:34 smithi112 ceph-mon[97750]: osdmap e86: 6 total, 6 up, 6 in 2024-09-16T00:33:34.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:34 smithi112 ceph-mon[97750]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-09-16T00:33:34.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:34 smithi112 ceph-mon[97750]: osdmap e87: 6 total, 6 up, 6 in 2024-09-16T00:33:34.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:34 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi112.ithwzj"}]: dispatch 2024-09-16T00:33:34.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:34 smithi169 ceph-mon[77626]: Health check update: Degraded data redundancy: 49/372 objects degraded (13.172%), 11 pgs degraded (PG_DEGRADED) 2024-09-16T00:33:34.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:34 smithi169 ceph-mon[77626]: osdmap e86: 6 total, 6 up, 6 in 2024-09-16T00:33:34.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:34 smithi169 ceph-mon[77626]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-09-16T00:33:34.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:34 smithi169 ceph-mon[77626]: osdmap e87: 6 total, 6 up, 6 in 2024-09-16T00:33:34.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:34 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi112.ithwzj"}]: dispatch 2024-09-16T00:33:35.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:35 smithi112 ceph-mon[97750]: pgmap v172: 65 pgs: 4 peering, 1 remapped+peering, 1 active+clean+remapped, 59 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 4.4 MiB/s, 165 objects/s recovering 2024-09-16T00:33:35.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:35 smithi169 ceph-mon[77626]: pgmap v172: 65 pgs: 4 peering, 1 remapped+peering, 1 active+clean+remapped, 59 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 4.4 MiB/s, 165 objects/s recovering 2024-09-16T00:33:36.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:36 smithi112 ceph-mon[97750]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-16T00:33:36.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:36 smithi112 ceph-mon[97750]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 49/372 objects degraded (13.172%), 11 pgs degraded) 2024-09-16T00:33:36.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:36 smithi169 ceph-mon[77626]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-16T00:33:36.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:36 smithi169 ceph-mon[77626]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 49/372 objects degraded (13.172%), 11 pgs degraded) 2024-09-16T00:33:37.306 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:37 smithi112 ceph-mon[97750]: pgmap v173: 65 pgs: 4 peering, 1 active+clean+remapped, 60 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 5.3 MiB/s, 120 objects/s recovering 2024-09-16T00:33:37.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:37 smithi169 ceph-mon[77626]: pgmap v173: 65 pgs: 4 peering, 1 active+clean+remapped, 60 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 5.3 MiB/s, 120 objects/s recovering 2024-09-16T00:33:39.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:39 smithi112 ceph-mon[97750]: mds.? [v2:172.21.15.112:6826/476229221,v1:172.21.15.112:6827/476229221] up:boot 2024-09-16T00:33:39.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:39 smithi112 ceph-mon[97750]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-09-16T00:33:39.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:39 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi112.ithwzj"}]: dispatch 2024-09-16T00:33:39.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:39 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:33:39.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:39 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:33:39.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:39 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:33:39.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:39 smithi112 ceph-mon[97750]: pgmap v174: 65 pgs: 1 peering, 64 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 4.5 MiB/s, 137 objects/s recovering 2024-09-16T00:33:39.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:39 smithi169 ceph-mon[77626]: mds.? [v2:172.21.15.112:6826/476229221,v1:172.21.15.112:6827/476229221] up:boot 2024-09-16T00:33:39.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:39 smithi169 ceph-mon[77626]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-09-16T00:33:39.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:39 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi112.ithwzj"}]: dispatch 2024-09-16T00:33:39.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:39 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:33:39.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:39 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:33:39.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:39 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:33:39.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:39 smithi169 ceph-mon[77626]: pgmap v174: 65 pgs: 1 peering, 64 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 4.5 MiB/s, 137 objects/s recovering 2024-09-16T00:33:40.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:40 smithi112 ceph-mon[97750]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-16T00:33:40.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:40 smithi169 ceph-mon[77626]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-16T00:33:41.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:41 smithi112 ceph-mon[97750]: pgmap v175: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 3.7 MiB/s, 112 objects/s recovering 2024-09-16T00:33:41.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:41 smithi169 ceph-mon[77626]: pgmap v175: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 3.7 MiB/s, 112 objects/s recovering 2024-09-16T00:33:44.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:43 smithi169 ceph-mon[77626]: pgmap v176: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 3.1 MiB/s, 50 objects/s recovering 2024-09-16T00:33:44.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:43 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:33:44.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:43 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:33:44.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:43 smithi112 ceph-mon[97750]: pgmap v176: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 3.1 MiB/s, 50 objects/s recovering 2024-09-16T00:33:44.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:43 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:33:44.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:43 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:33:45.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:45 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:33:45.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:45 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:33:45.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:45 smithi112 ceph-mon[97750]: pgmap v177: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.9 MiB/s, 22 objects/s recovering 2024-09-16T00:33:45.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:45 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:33:45.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:45 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:33:45.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:45 smithi169 ceph-mon[77626]: pgmap v177: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.9 MiB/s, 22 objects/s recovering 2024-09-16T00:33:47.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:47 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:33:47.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:47 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:33:47.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:47 smithi112 ceph-mon[97750]: pgmap v178: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1024 KiB/s, 19 objects/s recovering 2024-09-16T00:33:47.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:47 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:33:47.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:47 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:33:47.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:47 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:33:47.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:47 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:33:47.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:47 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:33:47.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:47 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:33:47.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:47 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:33:47.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:47 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:33:47.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:47 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:33:47.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:47 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:33:47.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:47 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:33:48.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:47 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:33:48.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:47 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:33:48.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:47 smithi169 ceph-mon[77626]: pgmap v178: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1024 KiB/s, 19 objects/s recovering 2024-09-16T00:33:48.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:47 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:33:48.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:47 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:33:48.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:47 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:33:48.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:47 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:33:48.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:47 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:33:48.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:47 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:33:48.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:47 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:33:48.028 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:47 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:33:48.028 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:47 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:33:48.028 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:47 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:33:48.028 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:47 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:33:49.809 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:49 smithi112 ceph-mon[97750]: pgmap v179: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s, 19 objects/s recovering 2024-09-16T00:33:49.810 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:49 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:33:49.810 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:49 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi112.hxfwkh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:33:49.810 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:49 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:33:49.856 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:49 smithi169 ceph-mon[77626]: pgmap v179: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s, 19 objects/s recovering 2024-09-16T00:33:49.856 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:49 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:33:49.857 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:49 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi112.hxfwkh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:33:49.857 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:49 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:33:50.808 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:50 smithi112 ceph-mon[97750]: Upgrade: Updating mds.cephfs.smithi112.hxfwkh 2024-09-16T00:33:50.809 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:50 smithi112 ceph-mon[97750]: Deploying daemon mds.cephfs.smithi112.hxfwkh on smithi112 2024-09-16T00:33:51.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:50 smithi169 ceph-mon[77626]: Upgrade: Updating mds.cephfs.smithi112.hxfwkh 2024-09-16T00:33:51.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:50 smithi169 ceph-mon[77626]: Deploying daemon mds.cephfs.smithi112.hxfwkh on smithi112 2024-09-16T00:33:52.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:51 smithi169 ceph-mon[77626]: pgmap v180: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:33:52.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:51 smithi112 ceph-mon[97750]: pgmap v180: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:33:53.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:52 smithi169 ceph-mon[77626]: osdmap e88: 6 total, 6 up, 6 in 2024-09-16T00:33:53.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:52 smithi169 ceph-mon[77626]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-09-16T00:33:53.074 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:52 smithi112 ceph-mon[97750]: osdmap e88: 6 total, 6 up, 6 in 2024-09-16T00:33:53.074 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:52 smithi112 ceph-mon[97750]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-09-16T00:33:53.979 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:53 smithi112 ceph-mon[97750]: pgmap v182: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:33:54.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:53 smithi169 ceph-mon[77626]: pgmap v182: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:33:54.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.262+0000 7f6c436d0640 1 -- 172.21.15.112:0/713734417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c3c102550 msgr2=0x7f6c3c102920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:54.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.262+0000 7f6c436d0640 1 --2- 172.21.15.112:0/713734417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c3c102550 0x7f6c3c102920 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6c380048d0 tx=0x7f6c38009da0 comp rx=0 tx=0).stop 2024-09-16T00:33:54.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.262+0000 7f6c436d0640 1 -- 172.21.15.112:0/713734417 shutdown_connections 2024-09-16T00:33:54.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.262+0000 7f6c436d0640 1 --2- 172.21.15.112:0/713734417 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c3c102ef0 0x7f6c3c0fc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:54.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.262+0000 7f6c436d0640 1 --2- 172.21.15.112:0/713734417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c3c102550 0x7f6c3c102920 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:54.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.262+0000 7f6c436d0640 1 -- 172.21.15.112:0/713734417 >> 172.21.15.112:0/713734417 conn(0x7f6c3c0f80f0 msgr2=0x7f6c3c0fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:54.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.262+0000 7f6c436d0640 1 -- 172.21.15.112:0/713734417 shutdown_connections 2024-09-16T00:33:54.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.262+0000 7f6c436d0640 1 -- 172.21.15.112:0/713734417 wait complete. 2024-09-16T00:33:54.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.263+0000 7f6c436d0640 1 Processor -- start 2024-09-16T00:33:54.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.263+0000 7f6c436d0640 1 -- start start 2024-09-16T00:33:54.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.263+0000 7f6c436d0640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c3c102550 0x7f6c3c113530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:54.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.263+0000 7f6c436d0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c3c102ef0 0x7f6c3c113a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:54.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.263+0000 7f6c436d0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3c1140c0 con 0x7f6c3c102ef0 2024-09-16T00:33:54.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.263+0000 7f6c436d0640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3c10d4d0 con 0x7f6c3c102550 2024-09-16T00:33:54.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.263+0000 7f6c40c6b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c3c102ef0 0x7f6c3c113a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:54.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.263+0000 7f6c40c6b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c3c102ef0 0x7f6c3c113a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53018/0 (socket says 172.21.15.112:53018) 2024-09-16T00:33:54.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.263+0000 7f6c40c6b640 1 -- 172.21.15.112:0/3308187170 learned_addr learned my addr 172.21.15.112:0/3308187170 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:33:54.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.263+0000 7f6c4146c640 1 --2- 172.21.15.112:0/3308187170 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c3c102550 0x7f6c3c113530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:54.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.263+0000 7f6c40c6b640 1 -- 172.21.15.112:0/3308187170 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c3c102550 msgr2=0x7f6c3c113530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:54.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.263+0000 7f6c40c6b640 1 --2- 172.21.15.112:0/3308187170 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c3c102550 0x7f6c3c113530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:54.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.264+0000 7f6c40c6b640 1 -- 172.21.15.112:0/3308187170 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c38005040 con 0x7f6c3c102ef0 2024-09-16T00:33:54.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.264+0000 7f6c40c6b640 1 --2- 172.21.15.112:0/3308187170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c3c102ef0 0x7f6c3c113a70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c002840 tx=0x7f6c2c004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:54.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.264+0000 7f6c31ffb640 1 -- 172.21.15.112:0/3308187170 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c2c007c30 con 0x7f6c3c102ef0 2024-09-16T00:33:54.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.264+0000 7f6c436d0640 1 -- 172.21.15.112:0/3308187170 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c3c10d750 con 0x7f6c3c102ef0 2024-09-16T00:33:54.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.264+0000 7f6c31ffb640 1 -- 172.21.15.112:0/3308187170 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f6c2c00f040 con 0x7f6c3c102ef0 2024-09-16T00:33:54.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.264+0000 7f6c31ffb640 1 -- 172.21.15.112:0/3308187170 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c2c009410 con 0x7f6c3c102ef0 2024-09-16T00:33:54.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.264+0000 7f6c436d0640 1 -- 172.21.15.112:0/3308187170 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c3c10dcf0 con 0x7f6c3c102ef0 2024-09-16T00:33:54.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.265+0000 7f6c436d0640 1 -- 172.21.15.112:0/3308187170 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c3c0fdcd0 con 0x7f6c3c102ef0 2024-09-16T00:33:54.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.266+0000 7f6c31ffb640 1 -- 172.21.15.112:0/3308187170 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f6c2c009700 con 0x7f6c3c102ef0 2024-09-16T00:33:54.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.266+0000 7f6c31ffb640 1 --2- 172.21.15.112:0/3308187170 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f6c2806e530 0x7f6c280709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:54.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.266+0000 7f6c31ffb640 1 -- 172.21.15.112:0/3308187170 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f6c2c08db40 con 0x7f6c3c102ef0 2024-09-16T00:33:54.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.266+0000 7f6c4146c640 1 --2- 172.21.15.112:0/3308187170 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f6c2806e530 0x7f6c280709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:54.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.267+0000 7f6c4146c640 1 --2- 172.21.15.112:0/3308187170 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f6c2806e530 0x7f6c280709e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6c38009ac0 tx=0x7f6c38035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:54.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.271+0000 7f6c31ffb640 1 -- 172.21.15.112:0/3308187170 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f6c2c056b30 con 0x7f6c3c102ef0 2024-09-16T00:33:54.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.473+0000 7f6c436d0640 1 -- 172.21.15.112:0/3308187170 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6c3c109630 con 0x7f6c2806e530 2024-09-16T00:33:54.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.475+0000 7f6c31ffb640 1 -- 172.21.15.112:0/3308187170 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f6c3c109630 con 0x7f6c2806e530 2024-09-16T00:33:54.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.477+0000 7f6c436d0640 1 -- 172.21.15.112:0/3308187170 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f6c2806e530 msgr2=0x7f6c280709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:54.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.477+0000 7f6c436d0640 1 --2- 172.21.15.112:0/3308187170 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f6c2806e530 0x7f6c280709e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6c38009ac0 tx=0x7f6c38035040 comp rx=0 tx=0).stop 2024-09-16T00:33:54.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.477+0000 7f6c436d0640 1 -- 172.21.15.112:0/3308187170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c3c102ef0 msgr2=0x7f6c3c113a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:54.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.477+0000 7f6c436d0640 1 --2- 172.21.15.112:0/3308187170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c3c102ef0 0x7f6c3c113a70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c002840 tx=0x7f6c2c004570 comp rx=0 tx=0).stop 2024-09-16T00:33:54.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.477+0000 7f6c436d0640 1 -- 172.21.15.112:0/3308187170 shutdown_connections 2024-09-16T00:33:54.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.477+0000 7f6c436d0640 1 --2- 172.21.15.112:0/3308187170 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f6c2806e530 0x7f6c280709e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:54.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.477+0000 7f6c436d0640 1 --2- 172.21.15.112:0/3308187170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c3c102ef0 0x7f6c3c113a70 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:54.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.477+0000 7f6c436d0640 1 --2- 172.21.15.112:0/3308187170 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c3c102550 0x7f6c3c113530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:54.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.477+0000 7f6c436d0640 1 -- 172.21.15.112:0/3308187170 >> 172.21.15.112:0/3308187170 conn(0x7f6c3c0f80f0 msgr2=0x7f6c3c0f90c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:54.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.477+0000 7f6c436d0640 1 -- 172.21.15.112:0/3308187170 shutdown_connections 2024-09-16T00:33:54.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.477+0000 7f6c436d0640 1 -- 172.21.15.112:0/3308187170 wait complete. 2024-09-16T00:33:54.487 INFO:teuthology.orchestra.run.smithi112.stdout:true 2024-09-16T00:33:54.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.558+0000 7f479e722640 1 -- 172.21.15.112:0/3116006773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4798102f00 msgr2=0x7f47980fc5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:54.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.558+0000 7f479e722640 1 --2- 172.21.15.112:0/3116006773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4798102f00 0x7f47980fc5c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f478c0048d0 tx=0x7f478c009da0 comp rx=0 tx=0).stop 2024-09-16T00:33:54.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.558+0000 7f479e722640 1 -- 172.21.15.112:0/3116006773 shutdown_connections 2024-09-16T00:33:54.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.558+0000 7f479e722640 1 --2- 172.21.15.112:0/3116006773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4798102f00 0x7f47980fc5c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:54.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.558+0000 7f479e722640 1 --2- 172.21.15.112:0/3116006773 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4798102560 0x7f4798102930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:54.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.558+0000 7f479e722640 1 -- 172.21.15.112:0/3116006773 >> 172.21.15.112:0/3116006773 conn(0x7f47980f8120 msgr2=0x7f47980fa530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:54.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.558+0000 7f479e722640 1 -- 172.21.15.112:0/3116006773 shutdown_connections 2024-09-16T00:33:54.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.558+0000 7f479e722640 1 -- 172.21.15.112:0/3116006773 wait complete. 2024-09-16T00:33:54.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.558+0000 7f479e722640 1 Processor -- start 2024-09-16T00:33:54.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.559+0000 7f479e722640 1 -- start start 2024-09-16T00:33:54.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.559+0000 7f479e722640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4798102560 0x7f4798113200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:54.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.559+0000 7f479e722640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4798102f00 0x7f4798113740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:54.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.559+0000 7f479e722640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4798113d40 con 0x7f4798102f00 2024-09-16T00:33:54.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.559+0000 7f479e722640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f479810d180 con 0x7f4798102560 2024-09-16T00:33:54.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.559+0000 7f47977fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4798102f00 0x7f4798113740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:54.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.559+0000 7f47977fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4798102f00 0x7f4798113740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53042/0 (socket says 172.21.15.112:53042) 2024-09-16T00:33:54.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.559+0000 7f47977fe640 1 -- 172.21.15.112:0/827497409 learned_addr learned my addr 172.21.15.112:0/827497409 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:33:54.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.559+0000 7f4797fff640 1 --2- 172.21.15.112:0/827497409 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4798102560 0x7f4798113200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:54.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.560+0000 7f47977fe640 1 -- 172.21.15.112:0/827497409 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4798102560 msgr2=0x7f4798113200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:54.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.560+0000 7f47977fe640 1 --2- 172.21.15.112:0/827497409 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4798102560 0x7f4798113200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:54.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.560+0000 7f47977fe640 1 -- 172.21.15.112:0/827497409 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f478c005040 con 0x7f4798102f00 2024-09-16T00:33:54.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.560+0000 7f47977fe640 1 --2- 172.21.15.112:0/827497409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4798102f00 0x7f4798113740 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f478c02a530 tx=0x7f478c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:54.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.560+0000 7f4794ff9640 1 -- 172.21.15.112:0/827497409 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f478c038070 con 0x7f4798102f00 2024-09-16T00:33:54.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.560+0000 7f4794ff9640 1 -- 172.21.15.112:0/827497409 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f478c02ade0 con 0x7f4798102f00 2024-09-16T00:33:54.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.560+0000 7f479e722640 1 -- 172.21.15.112:0/827497409 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f479810d400 con 0x7f4798102f00 2024-09-16T00:33:54.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.560+0000 7f479e722640 1 -- 172.21.15.112:0/827497409 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f479810d940 con 0x7f4798102f00 2024-09-16T00:33:54.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.560+0000 7f4794ff9640 1 -- 172.21.15.112:0/827497409 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f478c02c7a0 con 0x7f4798102f00 2024-09-16T00:33:54.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.561+0000 7f479e722640 1 -- 172.21.15.112:0/827497409 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4784005210 con 0x7f4798102f00 2024-09-16T00:33:54.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.562+0000 7f4794ff9640 1 -- 172.21.15.112:0/827497409 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f478c030020 con 0x7f4798102f00 2024-09-16T00:33:54.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.562+0000 7f4794ff9640 1 --2- 172.21.15.112:0/827497409 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f478006e530 0x7f47800709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:54.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.563+0000 7f4794ff9640 1 -- 172.21.15.112:0/827497409 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f478c0ae5d0 con 0x7f4798102f00 2024-09-16T00:33:54.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.563+0000 7f4797fff640 1 --2- 172.21.15.112:0/827497409 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f478006e530 0x7f47800709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:54.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.563+0000 7f4797fff640 1 --2- 172.21.15.112:0/827497409 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f478006e530 0x7f47800709e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4788009af0 tx=0x7f478800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:54.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.567+0000 7f4794ff9640 1 -- 172.21.15.112:0/827497409 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f478c0775b0 con 0x7f4798102f00 2024-09-16T00:33:54.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.760+0000 7f479e722640 1 -- 172.21.15.112:0/827497409 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4784000bc0 con 0x7f478006e530 2024-09-16T00:33:54.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.762+0000 7f4794ff9640 1 -- 172.21.15.112:0/827497409 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f4784000bc0 con 0x7f478006e530 2024-09-16T00:33:54.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.764+0000 7f479e722640 1 -- 172.21.15.112:0/827497409 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f478006e530 msgr2=0x7f47800709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:54.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.764+0000 7f479e722640 1 --2- 172.21.15.112:0/827497409 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f478006e530 0x7f47800709e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4788009af0 tx=0x7f478800b040 comp rx=0 tx=0).stop 2024-09-16T00:33:54.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.764+0000 7f479e722640 1 -- 172.21.15.112:0/827497409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4798102f00 msgr2=0x7f4798113740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:54.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.764+0000 7f479e722640 1 --2- 172.21.15.112:0/827497409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4798102f00 0x7f4798113740 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f478c02a530 tx=0x7f478c009da0 comp rx=0 tx=0).stop 2024-09-16T00:33:54.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.764+0000 7f479e722640 1 -- 172.21.15.112:0/827497409 shutdown_connections 2024-09-16T00:33:54.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.764+0000 7f479e722640 1 --2- 172.21.15.112:0/827497409 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f478006e530 0x7f47800709e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:54.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.764+0000 7f479e722640 1 --2- 172.21.15.112:0/827497409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4798102f00 0x7f4798113740 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:54.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.764+0000 7f479e722640 1 --2- 172.21.15.112:0/827497409 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4798102560 0x7f4798113200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:54.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.764+0000 7f479e722640 1 -- 172.21.15.112:0/827497409 >> 172.21.15.112:0/827497409 conn(0x7f47980f8120 msgr2=0x7f47980f91a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:54.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.764+0000 7f479e722640 1 -- 172.21.15.112:0/827497409 shutdown_connections 2024-09-16T00:33:54.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.764+0000 7f479e722640 1 -- 172.21.15.112:0/827497409 wait complete. 2024-09-16T00:33:54.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.845+0000 7f8a824eb640 1 -- 172.21.15.112:0/758876870 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a7c102530 msgr2=0x7f8a7c102900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:54.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.845+0000 7f8a824eb640 1 --2- 172.21.15.112:0/758876870 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a7c102530 0x7f8a7c102900 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c0048d0 tx=0x7f8a6c009da0 comp rx=0 tx=0).stop 2024-09-16T00:33:54.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.845+0000 7f8a824eb640 1 -- 172.21.15.112:0/758876870 shutdown_connections 2024-09-16T00:33:54.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.845+0000 7f8a824eb640 1 --2- 172.21.15.112:0/758876870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a7c102ed0 0x7f8a7c0fc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:54.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.845+0000 7f8a824eb640 1 --2- 172.21.15.112:0/758876870 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a7c102530 0x7f8a7c102900 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:54.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.845+0000 7f8a824eb640 1 -- 172.21.15.112:0/758876870 >> 172.21.15.112:0/758876870 conn(0x7f8a7c0f80f0 msgr2=0x7f8a7c0fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:54.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.845+0000 7f8a824eb640 1 -- 172.21.15.112:0/758876870 shutdown_connections 2024-09-16T00:33:54.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.845+0000 7f8a824eb640 1 -- 172.21.15.112:0/758876870 wait complete. 2024-09-16T00:33:54.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.845+0000 7f8a824eb640 1 Processor -- start 2024-09-16T00:33:54.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.845+0000 7f8a824eb640 1 -- start start 2024-09-16T00:33:54.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.846+0000 7f8a824eb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a7c102530 0x7f8a7c1131b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:54.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.846+0000 7f8a824eb640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a7c102ed0 0x7f8a7c1136f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:54.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.846+0000 7f8a824eb640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7c113cf0 con 0x7f8a7c102530 2024-09-16T00:33:54.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.846+0000 7f8a824eb640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7c10d130 con 0x7f8a7c102ed0 2024-09-16T00:33:54.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.846+0000 7f8a7bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a7c102530 0x7f8a7c1131b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:54.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.846+0000 7f8a7bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a7c102530 0x7f8a7c1131b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53072/0 (socket says 172.21.15.112:53072) 2024-09-16T00:33:54.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.846+0000 7f8a7bfff640 1 -- 172.21.15.112:0/3221997289 learned_addr learned my addr 172.21.15.112:0/3221997289 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:33:54.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.846+0000 7f8a7b7fe640 1 --2- 172.21.15.112:0/3221997289 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a7c102ed0 0x7f8a7c1136f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:54.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.846+0000 7f8a7bfff640 1 -- 172.21.15.112:0/3221997289 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a7c102ed0 msgr2=0x7f8a7c1136f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:54.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.846+0000 7f8a7bfff640 1 --2- 172.21.15.112:0/3221997289 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a7c102ed0 0x7f8a7c1136f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:54.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.846+0000 7f8a7bfff640 1 -- 172.21.15.112:0/3221997289 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a70005040 con 0x7f8a7c102530 2024-09-16T00:33:54.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.846+0000 7f8a7b7fe640 1 --2- 172.21.15.112:0/3221997289 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a7c102ed0 0x7f8a7c1136f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:33:54.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.846+0000 7f8a7bfff640 1 --2- 172.21.15.112:0/3221997289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a7c102530 0x7f8a7c1131b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c000c00 tx=0x7f8a6c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:54.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.847+0000 7f8a78ff9640 1 -- 172.21.15.112:0/3221997289 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a6c038070 con 0x7f8a7c102530 2024-09-16T00:33:54.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.847+0000 7f8a78ff9640 1 -- 172.21.15.112:0/3221997289 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f8a6c02ade0 con 0x7f8a7c102530 2024-09-16T00:33:54.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.847+0000 7f8a824eb640 1 -- 172.21.15.112:0/3221997289 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a6c005040 con 0x7f8a7c102530 2024-09-16T00:33:54.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.847+0000 7f8a78ff9640 1 -- 172.21.15.112:0/3221997289 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a6c02c7b0 con 0x7f8a7c102530 2024-09-16T00:33:54.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.847+0000 7f8a824eb640 1 -- 172.21.15.112:0/3221997289 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a7c10d760 con 0x7f8a7c102530 2024-09-16T00:33:54.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.848+0000 7f8a824eb640 1 -- 172.21.15.112:0/3221997289 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a7c0fdcb0 con 0x7f8a7c102530 2024-09-16T00:33:54.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.849+0000 7f8a78ff9640 1 -- 172.21.15.112:0/3221997289 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f8a6c030020 con 0x7f8a7c102530 2024-09-16T00:33:54.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.849+0000 7f8a78ff9640 1 --2- 172.21.15.112:0/3221997289 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f8a6406e530 0x7f8a640709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:54.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.849+0000 7f8a78ff9640 1 -- 172.21.15.112:0/3221997289 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f8a6c0ae9c0 con 0x7f8a7c102530 2024-09-16T00:33:54.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.849+0000 7f8a7b7fe640 1 --2- 172.21.15.112:0/3221997289 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f8a6406e530 0x7f8a640709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:54.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.850+0000 7f8a7b7fe640 1 --2- 172.21.15.112:0/3221997289 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f8a6406e530 0x7f8a640709e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8a70007fd0 tx=0x7f8a70009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:54.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:54.853+0000 7f8a78ff9640 1 -- 172.21.15.112:0/3221997289 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f8a6c0779b0 con 0x7f8a7c102530 2024-09-16T00:33:55.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.050+0000 7f8a824eb640 1 -- 172.21.15.112:0/3221997289 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8a7c04b720 con 0x7f8a6406e530 2024-09-16T00:33:55.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.058+0000 7f8a78ff9640 1 -- 172.21.15.112:0/3221997289 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f8a7c04b720 con 0x7f8a6406e530 2024-09-16T00:33:55.060 INFO:teuthology.orchestra.run.smithi112.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T00:33:55.060 INFO:teuthology.orchestra.run.smithi112.stdout:alertmanager.smithi112 smithi112 *:9093,9094 running (6m) 11s ago 15m 24.7M - 0.25.0 c8568f914cd2 d85ea0562022 2024-09-16T00:33:55.060 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi112 smithi112 running (3m) 11s ago 16m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c e4fe58f9702a 2024-09-16T00:33:55.060 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi169 smithi169 running (3m) 32s ago 15m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c 3ba2c81d527f 2024-09-16T00:33:55.060 INFO:teuthology.orchestra.run.smithi112.stdout:grafana.smithi112 smithi112 *:3000 running (5m) 11s ago 15m 81.8M - 9.4.12 2bacad6d85d8 51f9d2013da2 2024-09-16T00:33:55.060 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.hxfwkh smithi112 starting - - - - 2024-09-16T00:33:55.060 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.ithwzj smithi112 running (17s) 11s ago 10m 12.1M - 19.1.1-326-g8921c8ef f1455bd0219c e7aad5b81db5 2024-09-16T00:33:55.061 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.gawwgw smithi169 running (10m) 32s ago 10m 114M - 17.2.7-1633-g6d025bc2 33ed8dba3996 dcf3a4dfa879 2024-09-16T00:33:55.061 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.lgeqrj smithi169 running (10m) 32s ago 10m 60.3M - 17.2.7-1633-g6d025bc2 33ed8dba3996 445c14e55831 2024-09-16T00:33:55.061 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi112.xrkkbc smithi112 *:8443,9283,8765 running (6m) 11s ago 17m 504M - 19.1.1-326-g8921c8ef f1455bd0219c a981f7fd2810 2024-09-16T00:33:55.061 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi169.mtrobk smithi169 *:8443,9283,8765 running (5m) 32s ago 14m 442M - 19.1.1-326-g8921c8ef f1455bd0219c 4096e5369c2d 2024-09-16T00:33:55.061 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi112 smithi112 running (4m) 11s ago 17m 49.2M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 5aaec033c042 2024-09-16T00:33:55.061 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi169 smithi169 running (4m) 32s ago 14m 35.5M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 7bc609155f3f 2024-09-16T00:33:55.061 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi112 smithi112 *:9100 running (7m) 11s ago 15m 15.2M - 1.5.0 0da6a335fe13 6404d1d601e2 2024-09-16T00:33:55.061 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi169 smithi169 *:9100 running (6m) 32s ago 15m 15.3M - 1.5.0 0da6a335fe13 5505baf702a6 2024-09-16T00:33:55.061 INFO:teuthology.orchestra.run.smithi112.stdout:osd.0 smithi112 running (3m) 11s ago 14m 105M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 53efc9397017 2024-09-16T00:33:55.061 INFO:teuthology.orchestra.run.smithi112.stdout:osd.1 smithi112 running (2m) 11s ago 13m 98.9M 1557M 19.1.1-326-g8921c8ef f1455bd0219c fe6f1a4a2cf4 2024-09-16T00:33:55.061 INFO:teuthology.orchestra.run.smithi112.stdout:osd.2 smithi112 running (2m) 11s ago 13m 102M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 5f7863712057 2024-09-16T00:33:55.061 INFO:teuthology.orchestra.run.smithi112.stdout:osd.3 smithi169 running (99s) 32s ago 12m 90.8M 2581M 19.1.1-326-g8921c8ef f1455bd0219c ee07b5577810 2024-09-16T00:33:55.061 INFO:teuthology.orchestra.run.smithi112.stdout:osd.4 smithi169 running (67s) 32s ago 12m 93.3M 2581M 19.1.1-326-g8921c8ef f1455bd0219c 769f026a54eb 2024-09-16T00:33:55.062 INFO:teuthology.orchestra.run.smithi112.stdout:osd.5 smithi169 running (35s) 32s ago 11m 14.4M 2581M 19.1.1-326-g8921c8ef f1455bd0219c a6007c13cf78 2024-09-16T00:33:55.062 INFO:teuthology.orchestra.run.smithi112.stdout:prometheus.smithi112 smithi112 *:9095 running (6m) 11s ago 15m 64.1M - 2.43.0 a07b618ecd1d 7a1d42e60d2b 2024-09-16T00:33:55.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.061+0000 7f8a824eb640 1 -- 172.21.15.112:0/3221997289 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f8a6406e530 msgr2=0x7f8a640709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:55.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.061+0000 7f8a824eb640 1 --2- 172.21.15.112:0/3221997289 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f8a6406e530 0x7f8a640709e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8a70007fd0 tx=0x7f8a70009da0 comp rx=0 tx=0).stop 2024-09-16T00:33:55.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.061+0000 7f8a824eb640 1 -- 172.21.15.112:0/3221997289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a7c102530 msgr2=0x7f8a7c1131b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:55.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.061+0000 7f8a824eb640 1 --2- 172.21.15.112:0/3221997289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a7c102530 0x7f8a7c1131b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c000c00 tx=0x7f8a6c009da0 comp rx=0 tx=0).stop 2024-09-16T00:33:55.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.061+0000 7f8a824eb640 1 -- 172.21.15.112:0/3221997289 shutdown_connections 2024-09-16T00:33:55.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.061+0000 7f8a824eb640 1 --2- 172.21.15.112:0/3221997289 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f8a6406e530 0x7f8a640709e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:55.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.061+0000 7f8a824eb640 1 --2- 172.21.15.112:0/3221997289 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a7c102ed0 0x7f8a7c1136f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:55.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.061+0000 7f8a824eb640 1 --2- 172.21.15.112:0/3221997289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a7c102530 0x7f8a7c1131b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:55.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.061+0000 7f8a824eb640 1 -- 172.21.15.112:0/3221997289 >> 172.21.15.112:0/3221997289 conn(0x7f8a7c0f80f0 msgr2=0x7f8a7c0f90c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:55.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.061+0000 7f8a824eb640 1 -- 172.21.15.112:0/3221997289 shutdown_connections 2024-09-16T00:33:55.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.061+0000 7f8a824eb640 1 -- 172.21.15.112:0/3221997289 wait complete. 2024-09-16T00:33:55.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.144+0000 7f4eef731640 1 -- 172.21.15.112:0/3469563087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ee8102ef0 msgr2=0x7f4ee80fc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:55.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.144+0000 7f4eef731640 1 --2- 172.21.15.112:0/3469563087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ee8102ef0 0x7f4ee80fc5b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4ed80048d0 tx=0x7f4ed8009da0 comp rx=0 tx=0).stop 2024-09-16T00:33:55.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.144+0000 7f4eef731640 1 -- 172.21.15.112:0/3469563087 shutdown_connections 2024-09-16T00:33:55.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.144+0000 7f4eef731640 1 --2- 172.21.15.112:0/3469563087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ee8102ef0 0x7f4ee80fc5b0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:55.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.144+0000 7f4eef731640 1 --2- 172.21.15.112:0/3469563087 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ee8102550 0x7f4ee8102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:55.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.144+0000 7f4eef731640 1 -- 172.21.15.112:0/3469563087 >> 172.21.15.112:0/3469563087 conn(0x7f4ee80f80d0 msgr2=0x7f4ee80fa4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:55.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.144+0000 7f4eef731640 1 -- 172.21.15.112:0/3469563087 shutdown_connections 2024-09-16T00:33:55.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.144+0000 7f4eef731640 1 -- 172.21.15.112:0/3469563087 wait complete. 2024-09-16T00:33:55.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.145+0000 7f4eef731640 1 Processor -- start 2024-09-16T00:33:55.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.145+0000 7f4eef731640 1 -- start start 2024-09-16T00:33:55.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.145+0000 7f4eef731640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ee8102550 0x7f4ee81131f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:55.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.145+0000 7f4eef731640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ee8102ef0 0x7f4ee8113730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:55.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.145+0000 7f4eef731640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee8113d30 con 0x7f4ee8102ef0 2024-09-16T00:33:55.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.145+0000 7f4eef731640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee810d170 con 0x7f4ee8102550 2024-09-16T00:33:55.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.145+0000 7f4eecccc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ee8102ef0 0x7f4ee8113730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:55.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.145+0000 7f4eecccc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ee8102ef0 0x7f4ee8113730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53088/0 (socket says 172.21.15.112:53088) 2024-09-16T00:33:55.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.145+0000 7f4eecccc640 1 -- 172.21.15.112:0/4116662927 learned_addr learned my addr 172.21.15.112:0/4116662927 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:33:55.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.145+0000 7f4eed4cd640 1 --2- 172.21.15.112:0/4116662927 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ee8102550 0x7f4ee81131f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:55.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.145+0000 7f4eecccc640 1 -- 172.21.15.112:0/4116662927 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ee8102550 msgr2=0x7f4ee81131f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:55.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.145+0000 7f4eecccc640 1 --2- 172.21.15.112:0/4116662927 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ee8102550 0x7f4ee81131f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:55.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.145+0000 7f4eecccc640 1 -- 172.21.15.112:0/4116662927 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ed8005040 con 0x7f4ee8102ef0 2024-09-16T00:33:55.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.145+0000 7f4eed4cd640 1 --2- 172.21.15.112:0/4116662927 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ee8102550 0x7f4ee81131f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:33:55.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.146+0000 7f4eecccc640 1 --2- 172.21.15.112:0/4116662927 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ee8102ef0 0x7f4ee8113730 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4ed8003080 tx=0x7f4ed8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:55.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.146+0000 7f4eddffb640 1 -- 172.21.15.112:0/4116662927 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ed8038070 con 0x7f4ee8102ef0 2024-09-16T00:33:55.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.146+0000 7f4eef731640 1 -- 172.21.15.112:0/4116662927 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ee810d3f0 con 0x7f4ee8102ef0 2024-09-16T00:33:55.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.146+0000 7f4eddffb640 1 -- 172.21.15.112:0/4116662927 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f4ed8009760 con 0x7f4ee8102ef0 2024-09-16T00:33:55.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.146+0000 7f4eddffb640 1 -- 172.21.15.112:0/4116662927 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ed803d420 con 0x7f4ee8102ef0 2024-09-16T00:33:55.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.146+0000 7f4eef731640 1 -- 172.21.15.112:0/4116662927 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ee810d930 con 0x7f4ee8102ef0 2024-09-16T00:33:55.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.147+0000 7f4eef731640 1 -- 172.21.15.112:0/4116662927 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ee80fdcd0 con 0x7f4ee8102ef0 2024-09-16T00:33:55.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.148+0000 7f4eddffb640 1 -- 172.21.15.112:0/4116662927 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f4ed8030020 con 0x7f4ee8102ef0 2024-09-16T00:33:55.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.148+0000 7f4eddffb640 1 --2- 172.21.15.112:0/4116662927 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f4ed406e530 0x7f4ed40709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:55.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.148+0000 7f4eddffb640 1 -- 172.21.15.112:0/4116662927 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f4ed80ae810 con 0x7f4ee8102ef0 2024-09-16T00:33:55.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.148+0000 7f4eed4cd640 1 --2- 172.21.15.112:0/4116662927 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f4ed406e530 0x7f4ed40709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:55.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.149+0000 7f4eed4cd640 1 --2- 172.21.15.112:0/4116662927 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f4ed406e530 0x7f4ed40709e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4004c80 tx=0x7f4ee4009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:55.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.153+0000 7f4eddffb640 1 -- 172.21.15.112:0/4116662927 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f4ed8077780 con 0x7f4ee8102ef0 2024-09-16T00:33:55.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.421+0000 7f4eef731640 1 -- 172.21.15.112:0/4116662927 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4ee8109560 con 0x7f4ee8102ef0 2024-09-16T00:33:55.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.422+0000 7f4eddffb640 1 -- 172.21.15.112:0/4116662927 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+849 (secure 0 0 0) 0x7f4ed807ae40 con 0x7f4ee8102ef0 2024-09-16T00:33:55.423 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:33:55.424 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": { 2024-09-16T00:33:55.424 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:33:55.424 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:33:55.424 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": { 2024-09-16T00:33:55.424 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:33:55.424 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:33:55.424 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": { 2024-09-16T00:33:55.424 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 6 2024-09-16T00:33:55.424 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:33:55.424 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": { 2024-09-16T00:33:55.424 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 17.2.7-1633-g6d025bc2 (6d025bc26f3176c910707f8cb4c8d8e076640615) quincy (stable)": 2, 2024-09-16T00:33:55.424 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:33:55.424 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:33:55.425 INFO:teuthology.orchestra.run.smithi112.stdout: "overall": { 2024-09-16T00:33:55.425 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 17.2.7-1633-g6d025bc2 (6d025bc26f3176c910707f8cb4c8d8e076640615) quincy (stable)": 2, 2024-09-16T00:33:55.425 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 12 2024-09-16T00:33:55.425 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-16T00:33:55.425 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:33:55.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.425+0000 7f4eef731640 1 -- 172.21.15.112:0/4116662927 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f4ed406e530 msgr2=0x7f4ed40709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:55.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.425+0000 7f4eef731640 1 --2- 172.21.15.112:0/4116662927 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f4ed406e530 0x7f4ed40709e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4004c80 tx=0x7f4ee4009040 comp rx=0 tx=0).stop 2024-09-16T00:33:55.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.425+0000 7f4eef731640 1 -- 172.21.15.112:0/4116662927 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ee8102ef0 msgr2=0x7f4ee8113730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:55.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.425+0000 7f4eef731640 1 --2- 172.21.15.112:0/4116662927 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ee8102ef0 0x7f4ee8113730 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4ed8003080 tx=0x7f4ed8009da0 comp rx=0 tx=0).stop 2024-09-16T00:33:55.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.425+0000 7f4eef731640 1 -- 172.21.15.112:0/4116662927 shutdown_connections 2024-09-16T00:33:55.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.425+0000 7f4eef731640 1 --2- 172.21.15.112:0/4116662927 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f4ed406e530 0x7f4ed40709e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:55.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.425+0000 7f4eef731640 1 --2- 172.21.15.112:0/4116662927 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ee8102ef0 0x7f4ee8113730 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:55.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.425+0000 7f4eef731640 1 --2- 172.21.15.112:0/4116662927 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ee8102550 0x7f4ee81131f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:55.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.425+0000 7f4eef731640 1 -- 172.21.15.112:0/4116662927 >> 172.21.15.112:0/4116662927 conn(0x7f4ee80f80d0 msgr2=0x7f4ee80f9190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:55.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.425+0000 7f4eef731640 1 -- 172.21.15.112:0/4116662927 shutdown_connections 2024-09-16T00:33:55.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.425+0000 7f4eef731640 1 -- 172.21.15.112:0/4116662927 wait complete. 2024-09-16T00:33:55.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.504+0000 7fba90b0d640 1 -- 172.21.15.112:0/3896284582 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba8c0fabe0 msgr2=0x7fba8c0fb030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:55.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.504+0000 7fba90b0d640 1 --2- 172.21.15.112:0/3896284582 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba8c0fabe0 0x7fba8c0fb030 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fba800048d0 tx=0x7fba80009da0 comp rx=0 tx=0).stop 2024-09-16T00:33:55.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.504+0000 7fba90b0d640 1 -- 172.21.15.112:0/3896284582 shutdown_connections 2024-09-16T00:33:55.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.504+0000 7fba90b0d640 1 --2- 172.21.15.112:0/3896284582 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba8c0fabe0 0x7fba8c0fb030 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:55.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.504+0000 7fba90b0d640 1 --2- 172.21.15.112:0/3896284582 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba8c0fe460 0x7fba8c0fe830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:55.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.504+0000 7fba90b0d640 1 -- 172.21.15.112:0/3896284582 >> 172.21.15.112:0/3896284582 conn(0x7fba8c071b00 msgr2=0x7fba8c071f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:55.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.504+0000 7fba90b0d640 1 -- 172.21.15.112:0/3896284582 shutdown_connections 2024-09-16T00:33:55.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.504+0000 7fba90b0d640 1 -- 172.21.15.112:0/3896284582 wait complete. 2024-09-16T00:33:55.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.504+0000 7fba90b0d640 1 Processor -- start 2024-09-16T00:33:55.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.504+0000 7fba90b0d640 1 -- start start 2024-09-16T00:33:55.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.505+0000 7fba90b0d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba8c0fabe0 0x7fba8c113250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:55.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.505+0000 7fba90b0d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba8c0fe460 0x7fba8c113790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:55.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.505+0000 7fba90b0d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba8c113d90 con 0x7fba8c0fe460 2024-09-16T00:33:55.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.505+0000 7fba90b0d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba8c10d1d0 con 0x7fba8c0fabe0 2024-09-16T00:33:55.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.505+0000 7fba89d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba8c0fe460 0x7fba8c113790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:55.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.505+0000 7fba89d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba8c0fe460 0x7fba8c113790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53098/0 (socket says 172.21.15.112:53098) 2024-09-16T00:33:55.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.505+0000 7fba89d9b640 1 -- 172.21.15.112:0/2984137141 learned_addr learned my addr 172.21.15.112:0/2984137141 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:33:55.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.505+0000 7fba8a59c640 1 --2- 172.21.15.112:0/2984137141 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba8c0fabe0 0x7fba8c113250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:55.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.505+0000 7fba89d9b640 1 -- 172.21.15.112:0/2984137141 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba8c0fabe0 msgr2=0x7fba8c113250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:55.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.505+0000 7fba89d9b640 1 --2- 172.21.15.112:0/2984137141 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba8c0fabe0 0x7fba8c113250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:55.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.505+0000 7fba89d9b640 1 -- 172.21.15.112:0/2984137141 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba80005040 con 0x7fba8c0fe460 2024-09-16T00:33:55.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.505+0000 7fba8a59c640 1 --2- 172.21.15.112:0/2984137141 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba8c0fabe0 0x7fba8c113250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:33:55.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.505+0000 7fba89d9b640 1 --2- 172.21.15.112:0/2984137141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba8c0fe460 0x7fba8c113790 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fba80000c00 tx=0x7fba80009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:55.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.505+0000 7fba7affd640 1 -- 172.21.15.112:0/2984137141 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba80038070 con 0x7fba8c0fe460 2024-09-16T00:33:55.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.506+0000 7fba7affd640 1 -- 172.21.15.112:0/2984137141 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fba8002ade0 con 0x7fba8c0fe460 2024-09-16T00:33:55.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.506+0000 7fba90b0d640 1 -- 172.21.15.112:0/2984137141 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba8c10d450 con 0x7fba8c0fe460 2024-09-16T00:33:55.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.506+0000 7fba7affd640 1 -- 172.21.15.112:0/2984137141 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba8002c7b0 con 0x7fba8c0fe460 2024-09-16T00:33:55.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.506+0000 7fba90b0d640 1 -- 172.21.15.112:0/2984137141 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba8c10d910 con 0x7fba8c0fe460 2024-09-16T00:33:55.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.507+0000 7fba90b0d640 1 -- 172.21.15.112:0/2984137141 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba8c072610 con 0x7fba8c0fe460 2024-09-16T00:33:55.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.507+0000 7fba7affd640 1 -- 172.21.15.112:0/2984137141 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fba80030020 con 0x7fba8c0fe460 2024-09-16T00:33:55.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.508+0000 7fba7affd640 1 --2- 172.21.15.112:0/2984137141 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fba7406e530 0x7fba740709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:55.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.508+0000 7fba7affd640 1 -- 172.21.15.112:0/2984137141 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6872+0+0 (secure 0 0 0) 0x7fba800ae700 con 0x7fba8c0fe460 2024-09-16T00:33:55.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.508+0000 7fba8a59c640 1 --2- 172.21.15.112:0/2984137141 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fba7406e530 0x7fba740709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:55.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.508+0000 7fba8a59c640 1 --2- 172.21.15.112:0/2984137141 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fba7406e530 0x7fba740709e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fba7c004ab0 tx=0x7fba7c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:55.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.512+0000 7fba7affd640 1 -- 172.21.15.112:0/2984137141 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fba800776f0 con 0x7fba8c0fe460 2024-09-16T00:33:55.711 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:55 smithi112 ceph-mon[97750]: from='client.34396 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:33:55.711 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:55 smithi112 ceph-mon[97750]: pgmap v183: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:33:55.711 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:55 smithi112 ceph-mon[97750]: from='client.34400 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:33:55.711 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:55 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:33:55.711 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:55 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:33:55.711 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:55 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:33:55.711 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:55 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/4116662927' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:33:55.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.722+0000 7fba90b0d640 1 -- 172.21.15.112:0/2984137141 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fba8c04b720 con 0x7fba8c0fe460 2024-09-16T00:33:55.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.723+0000 7fba7affd640 1 -- 172.21.15.112:0/2984137141 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1914 (secure 0 0 0) 0x7fba8007adb0 con 0x7fba8c0fe460 2024-09-16T00:33:55.724 INFO:teuthology.orchestra.run.smithi112.stdout:e30 2024-09-16T00:33:55.725 INFO:teuthology.orchestra.run.smithi112.stdout:btime 2024-09-16T00:33:55:621083+0000 2024-09-16T00:33:55.725 INFO:teuthology.orchestra.run.smithi112.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T00:33:55.725 INFO:teuthology.orchestra.run.smithi112.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:33:55.725 INFO:teuthology.orchestra.run.smithi112.stdout:legacy client fscid: 1 2024-09-16T00:33:55.725 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:33:55.725 INFO:teuthology.orchestra.run.smithi112.stdout:Filesystem 'cephfs' (1) 2024-09-16T00:33:55.725 INFO:teuthology.orchestra.run.smithi112.stdout:fs_name cephfs 2024-09-16T00:33:55.725 INFO:teuthology.orchestra.run.smithi112.stdout:epoch 25 2024-09-16T00:33:55.725 INFO:teuthology.orchestra.run.smithi112.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-16T00:33:55.725 INFO:teuthology.orchestra.run.smithi112.stdout:created 2024-09-16T00:23:22.328199+0000 2024-09-16T00:33:55.725 INFO:teuthology.orchestra.run.smithi112.stdout:modified 2024-09-16T00:33:29.012991+0000 2024-09-16T00:33:55.725 INFO:teuthology.orchestra.run.smithi112.stdout:tableserver 0 2024-09-16T00:33:55.725 INFO:teuthology.orchestra.run.smithi112.stdout:root 0 2024-09-16T00:33:55.726 INFO:teuthology.orchestra.run.smithi112.stdout:session_timeout 60 2024-09-16T00:33:55.726 INFO:teuthology.orchestra.run.smithi112.stdout:session_autoclose 300 2024-09-16T00:33:55.726 INFO:teuthology.orchestra.run.smithi112.stdout:max_file_size 1099511627776 2024-09-16T00:33:55.726 INFO:teuthology.orchestra.run.smithi112.stdout:max_xattr_size 65536 2024-09-16T00:33:55.726 INFO:teuthology.orchestra.run.smithi112.stdout:required_client_features {} 2024-09-16T00:33:55.726 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure 0 2024-09-16T00:33:55.726 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure_osd_epoch 82 2024-09-16T00:33:55.726 INFO:teuthology.orchestra.run.smithi112.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:33:55.726 INFO:teuthology.orchestra.run.smithi112.stdout:max_mds 2 2024-09-16T00:33:55.726 INFO:teuthology.orchestra.run.smithi112.stdout:in 0,1 2024-09-16T00:33:55.726 INFO:teuthology.orchestra.run.smithi112.stdout:up {} 2024-09-16T00:33:55.726 INFO:teuthology.orchestra.run.smithi112.stdout:failed 0,1 2024-09-16T00:33:55.726 INFO:teuthology.orchestra.run.smithi112.stdout:damaged 2024-09-16T00:33:55.726 INFO:teuthology.orchestra.run.smithi112.stdout:stopped 2024-09-16T00:33:55.727 INFO:teuthology.orchestra.run.smithi112.stdout:data_pools [3] 2024-09-16T00:33:55.727 INFO:teuthology.orchestra.run.smithi112.stdout:metadata_pool 2 2024-09-16T00:33:55.727 INFO:teuthology.orchestra.run.smithi112.stdout:inline_data enabled 2024-09-16T00:33:55.727 INFO:teuthology.orchestra.run.smithi112.stdout:balancer 2024-09-16T00:33:55.727 INFO:teuthology.orchestra.run.smithi112.stdout:bal_rank_mask -1 2024-09-16T00:33:55.727 INFO:teuthology.orchestra.run.smithi112.stdout:standby_count_wanted 1 2024-09-16T00:33:55.727 INFO:teuthology.orchestra.run.smithi112.stdout:qdb_cluster leader: 0 members: 2024-09-16T00:33:55.727 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:33:55.727 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:33:55.727 INFO:teuthology.orchestra.run.smithi112.stdout:Standby daemons: 2024-09-16T00:33:55.727 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:33:55.727 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.ithwzj{-1:34392} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.112:6826/476229221,v1:172.21.15.112:6827/476229221] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:33:55.727 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.hxfwkh{-1:34404} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.112:6828/2356899876,v1:172.21.15.112:6829/2356899876] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:33:55.727 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.lgeqrj{-1:44307} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.169:6826/3569474985,v1:172.21.15.169:6827/3569474985] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T00:33:55.727 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.gawwgw{-1:44315} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.169:6824/3147876558,v1:172.21.15.169:6825/3147876558] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T00:33:55.728 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 30 2024-09-16T00:33:55.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.726+0000 7fba90b0d640 1 -- 172.21.15.112:0/2984137141 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fba7406e530 msgr2=0x7fba740709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:55.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.726+0000 7fba90b0d640 1 --2- 172.21.15.112:0/2984137141 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fba7406e530 0x7fba740709e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fba7c004ab0 tx=0x7fba7c00b040 comp rx=0 tx=0).stop 2024-09-16T00:33:55.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.726+0000 7fba90b0d640 1 -- 172.21.15.112:0/2984137141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba8c0fe460 msgr2=0x7fba8c113790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:55.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.726+0000 7fba90b0d640 1 --2- 172.21.15.112:0/2984137141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba8c0fe460 0x7fba8c113790 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fba80000c00 tx=0x7fba80009da0 comp rx=0 tx=0).stop 2024-09-16T00:33:55.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.726+0000 7fba90b0d640 1 -- 172.21.15.112:0/2984137141 shutdown_connections 2024-09-16T00:33:55.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.726+0000 7fba90b0d640 1 --2- 172.21.15.112:0/2984137141 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fba7406e530 0x7fba740709e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:55.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.726+0000 7fba90b0d640 1 --2- 172.21.15.112:0/2984137141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba8c0fe460 0x7fba8c113790 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:55.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.726+0000 7fba90b0d640 1 --2- 172.21.15.112:0/2984137141 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba8c0fabe0 0x7fba8c113250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:55.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.726+0000 7fba90b0d640 1 -- 172.21.15.112:0/2984137141 >> 172.21.15.112:0/2984137141 conn(0x7fba8c071b00 msgr2=0x7fba8c101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:55.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.726+0000 7fba90b0d640 1 -- 172.21.15.112:0/2984137141 shutdown_connections 2024-09-16T00:33:55.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.726+0000 7fba90b0d640 1 -- 172.21.15.112:0/2984137141 wait complete. 2024-09-16T00:33:55.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.807+0000 7fa2b0c46640 1 -- 172.21.15.112:0/2556542521 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2ac0fb480 msgr2=0x7fa2ac0fb8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:55.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.807+0000 7fa2b0c46640 1 --2- 172.21.15.112:0/2556542521 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2ac0fb480 0x7fa2ac0fb8f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa2a00048d0 tx=0x7fa2a0009da0 comp rx=0 tx=0).stop 2024-09-16T00:33:55.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.807+0000 7fa2b0c46640 1 -- 172.21.15.112:0/2556542521 shutdown_connections 2024-09-16T00:33:55.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.807+0000 7fa2b0c46640 1 --2- 172.21.15.112:0/2556542521 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2ac0fb480 0x7fa2ac0fb8f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:55.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.807+0000 7fa2b0c46640 1 --2- 172.21.15.112:0/2556542521 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2ac1014c0 0x7fa2ac101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:55.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.807+0000 7fa2b0c46640 1 -- 172.21.15.112:0/2556542521 >> 172.21.15.112:0/2556542521 conn(0x7fa2ac0f7000 msgr2=0x7fa2ac0f9410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:55.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.807+0000 7fa2b0c46640 1 -- 172.21.15.112:0/2556542521 shutdown_connections 2024-09-16T00:33:55.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.807+0000 7fa2b0c46640 1 -- 172.21.15.112:0/2556542521 wait complete. 2024-09-16T00:33:55.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.807+0000 7fa2b0c46640 1 Processor -- start 2024-09-16T00:33:55.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.808+0000 7fa2b0c46640 1 -- start start 2024-09-16T00:33:55.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.808+0000 7fa2b0c46640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2ac0fb480 0x7fa2ac115430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:55.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.808+0000 7fa2b0c46640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2ac1014c0 0x7fa2ac115970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:55.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.808+0000 7fa2b0c46640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ac115f70 con 0x7fa2ac0fb480 2024-09-16T00:33:55.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.808+0000 7fa2b0c46640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ac1160e0 con 0x7fa2ac1014c0 2024-09-16T00:33:55.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.808+0000 7fa2aa59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2ac0fb480 0x7fa2ac115430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:55.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.808+0000 7fa2aa59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2ac0fb480 0x7fa2ac115430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53120/0 (socket says 172.21.15.112:53120) 2024-09-16T00:33:55.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.808+0000 7fa2aa59c640 1 -- 172.21.15.112:0/470232479 learned_addr learned my addr 172.21.15.112:0/470232479 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:33:55.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.808+0000 7fa2a9d9b640 1 --2- 172.21.15.112:0/470232479 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2ac1014c0 0x7fa2ac115970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:55.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.808+0000 7fa2aa59c640 1 -- 172.21.15.112:0/470232479 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2ac1014c0 msgr2=0x7fa2ac115970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:55.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.808+0000 7fa2aa59c640 1 --2- 172.21.15.112:0/470232479 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2ac1014c0 0x7fa2ac115970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:55.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.808+0000 7fa2aa59c640 1 -- 172.21.15.112:0/470232479 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2a0005040 con 0x7fa2ac0fb480 2024-09-16T00:33:55.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.808+0000 7fa2aa59c640 1 --2- 172.21.15.112:0/470232479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2ac0fb480 0x7fa2ac115430 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa29c00c790 tx=0x7fa29c0043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:55.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.808+0000 7fa29affd640 1 -- 172.21.15.112:0/470232479 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa29c00a7a0 con 0x7fa2ac0fb480 2024-09-16T00:33:55.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.809+0000 7fa29affd640 1 -- 172.21.15.112:0/470232479 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fa29c00ada0 con 0x7fa2ac0fb480 2024-09-16T00:33:55.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.809+0000 7fa29affd640 1 -- 172.21.15.112:0/470232479 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa29c0113a0 con 0x7fa2ac0fb480 2024-09-16T00:33:55.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.809+0000 7fa2b0c46640 1 -- 172.21.15.112:0/470232479 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2ac10f560 con 0x7fa2ac0fb480 2024-09-16T00:33:55.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.809+0000 7fa2b0c46640 1 -- 172.21.15.112:0/470232479 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2ac10fa80 con 0x7fa2ac0fb480 2024-09-16T00:33:55.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.810+0000 7fa2b0c46640 1 -- 172.21.15.112:0/470232479 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2ac0fcbc0 con 0x7fa2ac0fb480 2024-09-16T00:33:55.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.810+0000 7fa29affd640 1 -- 172.21.15.112:0/470232479 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fa29c00a900 con 0x7fa2ac0fb480 2024-09-16T00:33:55.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.810+0000 7fa29affd640 1 --2- 172.21.15.112:0/470232479 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa29406e530 0x7fa2940709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:55.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.811+0000 7fa2a9d9b640 1 --2- 172.21.15.112:0/470232479 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa29406e530 0x7fa2940709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:55.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.811+0000 7fa29affd640 1 -- 172.21.15.112:0/470232479 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6872+0+0 (secure 0 0 0) 0x7fa29c08e3a0 con 0x7fa2ac0fb480 2024-09-16T00:33:55.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.811+0000 7fa2a9d9b640 1 --2- 172.21.15.112:0/470232479 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa29406e530 0x7fa2940709e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0009ac0 tx=0x7fa2a0009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:55.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:55.815+0000 7fa29affd640 1 -- 172.21.15.112:0/470232479 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fa29c057430 con 0x7fa2ac0fb480 2024-09-16T00:33:56.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:56.011+0000 7fa2b0c46640 1 -- 172.21.15.112:0/470232479 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa2ac110ae0 con 0x7fa29406e530 2024-09-16T00:33:56.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:56.013+0000 7fa29affd640 1 -- 172.21.15.112:0/470232479 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fa2ac110ae0 con 0x7fa29406e530 2024-09-16T00:33:56.014 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:33:56.014 INFO:teuthology.orchestra.run.smithi112.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80", 2024-09-16T00:33:56.014 INFO:teuthology.orchestra.run.smithi112.stdout: "in_progress": true, 2024-09-16T00:33:56.014 INFO:teuthology.orchestra.run.smithi112.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T00:33:56.015 INFO:teuthology.orchestra.run.smithi112.stdout: "services_complete": [ 2024-09-16T00:33:56.015 INFO:teuthology.orchestra.run.smithi112.stdout: "crash", 2024-09-16T00:33:56.015 INFO:teuthology.orchestra.run.smithi112.stdout: "osd", 2024-09-16T00:33:56.015 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr", 2024-09-16T00:33:56.015 INFO:teuthology.orchestra.run.smithi112.stdout: "mon" 2024-09-16T00:33:56.015 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-16T00:33:56.016 INFO:teuthology.orchestra.run.smithi112.stdout: "progress": "13/21 daemons upgraded", 2024-09-16T00:33:56.016 INFO:teuthology.orchestra.run.smithi112.stdout: "message": "Currently upgrading mds daemons", 2024-09-16T00:33:56.016 INFO:teuthology.orchestra.run.smithi112.stdout: "is_paused": false 2024-09-16T00:33:56.016 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:33:56.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:56.015+0000 7fa2b0c46640 1 -- 172.21.15.112:0/470232479 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa29406e530 msgr2=0x7fa2940709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:56.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:56.015+0000 7fa2b0c46640 1 --2- 172.21.15.112:0/470232479 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa29406e530 0x7fa2940709e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0009ac0 tx=0x7fa2a0009da0 comp rx=0 tx=0).stop 2024-09-16T00:33:56.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:56.015+0000 7fa2b0c46640 1 -- 172.21.15.112:0/470232479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2ac0fb480 msgr2=0x7fa2ac115430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:56.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:56.015+0000 7fa2b0c46640 1 --2- 172.21.15.112:0/470232479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2ac0fb480 0x7fa2ac115430 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa29c00c790 tx=0x7fa29c0043c0 comp rx=0 tx=0).stop 2024-09-16T00:33:56.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:56.015+0000 7fa2b0c46640 1 -- 172.21.15.112:0/470232479 shutdown_connections 2024-09-16T00:33:56.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:56.015+0000 7fa2b0c46640 1 --2- 172.21.15.112:0/470232479 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa29406e530 0x7fa2940709e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:56.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:56.015+0000 7fa2b0c46640 1 --2- 172.21.15.112:0/470232479 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2ac1014c0 0x7fa2ac115970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:56.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:56.015+0000 7fa2b0c46640 1 --2- 172.21.15.112:0/470232479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2ac0fb480 0x7fa2ac115430 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:56.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:56.015+0000 7fa2b0c46640 1 -- 172.21.15.112:0/470232479 >> 172.21.15.112:0/470232479 conn(0x7fa2ac0f7000 msgr2=0x7fa2ac065a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:56.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:56.015+0000 7fa2b0c46640 1 -- 172.21.15.112:0/470232479 shutdown_connections 2024-09-16T00:33:56.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:56.015+0000 7fa2b0c46640 1 -- 172.21.15.112:0/470232479 wait complete. 2024-09-16T00:33:56.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:55 smithi169 ceph-mon[77626]: from='client.34396 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:33:56.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:55 smithi169 ceph-mon[77626]: pgmap v183: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:33:56.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:55 smithi169 ceph-mon[77626]: from='client.34400 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:33:56.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:55 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:33:56.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:55 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:33:56.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:55 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:33:56.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:55 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/4116662927' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:33:56.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:56.099+0000 7f607faf0640 1 -- 172.21.15.112:0/4286756809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60780fb480 msgr2=0x7f60780fb8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:56.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:56.099+0000 7f607faf0640 1 --2- 172.21.15.112:0/4286756809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60780fb480 0x7f60780fb8f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f606c0048d0 tx=0x7f606c009da0 comp rx=0 tx=0).stop 2024-09-16T00:33:56.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:56.100+0000 7f607faf0640 1 -- 172.21.15.112:0/4286756809 shutdown_connections 2024-09-16T00:33:56.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:56.100+0000 7f607faf0640 1 --2- 172.21.15.112:0/4286756809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60780fb480 0x7f60780fb8f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:56.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:56.100+0000 7f607faf0640 1 --2- 172.21.15.112:0/4286756809 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60781014c0 0x7f6078101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:56.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:56.100+0000 7f607faf0640 1 -- 172.21.15.112:0/4286756809 >> 172.21.15.112:0/4286756809 conn(0x7f60780f7000 msgr2=0x7f60780f9410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:56.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:56.100+0000 7f607faf0640 1 -- 172.21.15.112:0/4286756809 shutdown_connections 2024-09-16T00:33:56.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:56.100+0000 7f607faf0640 1 -- 172.21.15.112:0/4286756809 wait complete. 2024-09-16T00:33:56.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:56.100+0000 7f607faf0640 1 Processor -- start 2024-09-16T00:33:56.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:56.100+0000 7f607faf0640 1 -- start start 2024-09-16T00:33:56.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:56.100+0000 7f607faf0640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60780fb480 0x7f60781131a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:56.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:56.100+0000 7f607faf0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60781014c0 0x7f60781136e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:56.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:56.100+0000 7f607faf0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6078113ce0 con 0x7f60781014c0 2024-09-16T00:33:56.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:56.100+0000 7f607faf0640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f607810d120 con 0x7f60780fb480 2024-09-16T00:33:56.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:56.101+0000 7f607d08b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60781014c0 0x7f60781136e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:56.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:56.101+0000 7f607d08b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60781014c0 0x7f60781136e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53142/0 (socket says 172.21.15.112:53142) 2024-09-16T00:33:56.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:56.101+0000 7f607d88c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60780fb480 0x7f60781131a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:56.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:56.101+0000 7f607d08b640 1 -- 172.21.15.112:0/282450690 learned_addr learned my addr 172.21.15.112:0/282450690 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:33:56.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:56.101+0000 7f607d08b640 1 -- 172.21.15.112:0/282450690 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60780fb480 msgr2=0x7f60781131a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:56.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:56.101+0000 7f607d08b640 1 --2- 172.21.15.112:0/282450690 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60780fb480 0x7f60781131a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:56.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:56.101+0000 7f607d08b640 1 -- 172.21.15.112:0/282450690 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f606c005040 con 0x7f60781014c0 2024-09-16T00:33:56.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:56.101+0000 7f607d08b640 1 --2- 172.21.15.112:0/282450690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60781014c0 0x7f60781136e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f606c000c00 tx=0x7f606c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:56.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:56.101+0000 7f60767fc640 1 -- 172.21.15.112:0/282450690 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f606c038070 con 0x7f60781014c0 2024-09-16T00:33:56.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:56.101+0000 7f60767fc640 1 -- 172.21.15.112:0/282450690 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f606c02ade0 con 0x7f60781014c0 2024-09-16T00:33:56.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:56.101+0000 7f607faf0640 1 -- 172.21.15.112:0/282450690 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f607810d3a0 con 0x7f60781014c0 2024-09-16T00:33:56.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:56.101+0000 7f60767fc640 1 -- 172.21.15.112:0/282450690 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f606c02c7b0 con 0x7f60781014c0 2024-09-16T00:33:56.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:56.101+0000 7f607faf0640 1 -- 172.21.15.112:0/282450690 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f607810d8e0 con 0x7f60781014c0 2024-09-16T00:33:56.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:56.102+0000 7f607faf0640 1 -- 172.21.15.112:0/282450690 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60780fcbc0 con 0x7f60781014c0 2024-09-16T00:33:56.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:56.103+0000 7f60767fc640 1 -- 172.21.15.112:0/282450690 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f606c030020 con 0x7f60781014c0 2024-09-16T00:33:56.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:56.103+0000 7f60767fc640 1 --2- 172.21.15.112:0/282450690 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f606006e530 0x7f60600709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:56.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:56.104+0000 7f60767fc640 1 -- 172.21.15.112:0/282450690 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f606c0ae710 con 0x7f60781014c0 2024-09-16T00:33:56.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:56.104+0000 7f607d88c640 1 --2- 172.21.15.112:0/282450690 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f606006e530 0x7f60600709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:56.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:56.104+0000 7f607d88c640 1 --2- 172.21.15.112:0/282450690 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f606006e530 0x7f60600709e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6070009af0 tx=0x7f607000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:56.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:56.108+0000 7f60767fc640 1 -- 172.21.15.112:0/282450690 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f606c077680 con 0x7f60781014c0 2024-09-16T00:33:56.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:56.346+0000 7f607faf0640 1 -- 172.21.15.112:0/282450690 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f60780fac40 con 0x7f60781014c0 2024-09-16T00:33:56.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:56.346+0000 7f60767fc640 1 -- 172.21.15.112:0/282450690 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f606c07ad40 con 0x7f60781014c0 2024-09-16T00:33:56.348 INFO:teuthology.orchestra.run.smithi112.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-16T00:33:56.348 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-16T00:33:56.348 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs is degraded 2024-09-16T00:33:56.348 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T00:33:56.349 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T00:33:56.349 INFO:teuthology.orchestra.run.smithi112.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-16T00:33:56.349 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-16T00:33:56.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:56.348+0000 7f607faf0640 1 -- 172.21.15.112:0/282450690 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f606006e530 msgr2=0x7f60600709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:56.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:56.348+0000 7f607faf0640 1 --2- 172.21.15.112:0/282450690 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f606006e530 0x7f60600709e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6070009af0 tx=0x7f607000b040 comp rx=0 tx=0).stop 2024-09-16T00:33:56.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:56.348+0000 7f607faf0640 1 -- 172.21.15.112:0/282450690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60781014c0 msgr2=0x7f60781136e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:56.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:56.348+0000 7f607faf0640 1 --2- 172.21.15.112:0/282450690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60781014c0 0x7f60781136e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f606c000c00 tx=0x7f606c009da0 comp rx=0 tx=0).stop 2024-09-16T00:33:56.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:56.348+0000 7f607faf0640 1 -- 172.21.15.112:0/282450690 shutdown_connections 2024-09-16T00:33:56.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:56.348+0000 7f607faf0640 1 --2- 172.21.15.112:0/282450690 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f606006e530 0x7f60600709e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:56.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:56.348+0000 7f607faf0640 1 --2- 172.21.15.112:0/282450690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60781014c0 0x7f60781136e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:56.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:56.348+0000 7f607faf0640 1 --2- 172.21.15.112:0/282450690 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60780fb480 0x7f60781131a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:56.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:56.348+0000 7f607faf0640 1 -- 172.21.15.112:0/282450690 >> 172.21.15.112:0/282450690 conn(0x7f60780f7000 msgr2=0x7f60780659f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:56.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:56.348+0000 7f607faf0640 1 -- 172.21.15.112:0/282450690 shutdown_connections 2024-09-16T00:33:56.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:33:56.348+0000 7f607faf0640 1 -- 172.21.15.112:0/282450690 wait complete. 2024-09-16T00:33:57.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:56 smithi169 ceph-mon[77626]: from='client.34408 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:33:57.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:56 smithi169 ceph-mon[77626]: mds.? [v2:172.21.15.112:6828/2356899876,v1:172.21.15.112:6829/2356899876] up:boot 2024-09-16T00:33:57.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:56 smithi169 ceph-mon[77626]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-09-16T00:33:57.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:56 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi112.hxfwkh"}]: dispatch 2024-09-16T00:33:57.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:56 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/2984137141' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:33:57.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:56 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/282450690' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:33:57.075 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:56 smithi112 ceph-mon[97750]: from='client.34408 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:33:57.076 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:56 smithi112 ceph-mon[97750]: mds.? [v2:172.21.15.112:6828/2356899876,v1:172.21.15.112:6829/2356899876] up:boot 2024-09-16T00:33:57.076 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:56 smithi112 ceph-mon[97750]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-09-16T00:33:57.076 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:56 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi112.hxfwkh"}]: dispatch 2024-09-16T00:33:57.076 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:56 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/2984137141' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:33:57.076 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:56 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/282450690' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:33:58.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:57 smithi169 ceph-mon[77626]: from='client.34420 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:33:58.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:57 smithi169 ceph-mon[77626]: pgmap v184: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:33:58.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:57 smithi112 ceph-mon[97750]: from='client.34420 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:33:58.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:57 smithi112 ceph-mon[97750]: pgmap v184: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:34:00.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:59 smithi169 ceph-mon[77626]: pgmap v185: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:34:00.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:59 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:34:00.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:33:59 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:34:00.062 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:59 smithi112 ceph-mon[97750]: pgmap v185: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:34:00.062 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:59 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:34:00.062 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:33:59 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:34:01.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:00 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:34:01.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:00 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:34:01.084 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:00 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:34:01.084 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:00 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:34:02.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:01 smithi169 ceph-mon[77626]: pgmap v186: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:34:02.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:01 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:34:02.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:01 smithi112 ceph-mon[97750]: pgmap v186: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:34:02.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:01 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:34:03.744 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:03 smithi169 ceph-mon[77626]: pgmap v187: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:34:03.744 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:03 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:34:03.744 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:03 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:34:03.744 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:03 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:34:03.744 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:03 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:34:03.744 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:03 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:34:03.744 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:03 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:34:03.744 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:03 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:34:03.744 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:03 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:34:03.745 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:03 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:34:03.745 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:03 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:34:04.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:03 smithi112 ceph-mon[97750]: pgmap v187: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:34:04.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:03 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:34:04.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:03 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:34:04.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:03 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:34:04.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:03 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:34:04.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:03 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:34:04.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:03 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:34:04.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:03 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:34:04.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:03 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:34:04.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:03 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:34:04.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:03 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:34:05.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:05 smithi112 ceph-mon[97750]: Upgrade: Updating mds.cephfs.smithi169.gawwgw 2024-09-16T00:34:05.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:05 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:34:05.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:05 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi169.gawwgw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:34:05.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:05 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:34:05.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:05 smithi112 ceph-mon[97750]: Deploying daemon mds.cephfs.smithi169.gawwgw on smithi169 2024-09-16T00:34:05.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:05 smithi112 ceph-mon[97750]: pgmap v188: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:34:05.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:05 smithi169 ceph-mon[77626]: Upgrade: Updating mds.cephfs.smithi169.gawwgw 2024-09-16T00:34:05.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:05 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:34:05.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:05 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi169.gawwgw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:34:05.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:05 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:34:05.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:05 smithi169 ceph-mon[77626]: Deploying daemon mds.cephfs.smithi169.gawwgw on smithi169 2024-09-16T00:34:05.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:05 smithi169 ceph-mon[77626]: pgmap v188: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:34:07.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:07 smithi169 ceph-mon[77626]: osdmap e89: 6 total, 6 up, 6 in 2024-09-16T00:34:07.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:07 smithi169 ceph-mon[77626]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-09-16T00:34:07.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:07 smithi169 ceph-mon[77626]: pgmap v190: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:34:07.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:07 smithi112 ceph-mon[97750]: osdmap e89: 6 total, 6 up, 6 in 2024-09-16T00:34:07.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:07 smithi112 ceph-mon[97750]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-09-16T00:34:07.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:07 smithi112 ceph-mon[97750]: pgmap v190: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:34:09.999 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:09 smithi169 ceph-mon[77626]: pgmap v191: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:34:10.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:09 smithi112 ceph-mon[97750]: pgmap v191: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:34:11.955 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:11 smithi169 ceph-mon[77626]: pgmap v192: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:34:11.955 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:11 smithi169 ceph-mon[77626]: mds.? [v2:172.21.15.169:6824/1635329957,v1:172.21.15.169:6825/1635329957] up:boot 2024-09-16T00:34:11.955 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:11 smithi169 ceph-mon[77626]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-09-16T00:34:11.955 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:11 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi169.gawwgw"}]: dispatch 2024-09-16T00:34:11.955 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:11 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:34:11.956 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:11 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:34:11.956 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:11 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:34:12.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:11 smithi112 ceph-mon[97750]: pgmap v192: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:34:12.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:11 smithi112 ceph-mon[97750]: mds.? [v2:172.21.15.169:6824/1635329957,v1:172.21.15.169:6825/1635329957] up:boot 2024-09-16T00:34:12.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:11 smithi112 ceph-mon[97750]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-09-16T00:34:12.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:11 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi169.gawwgw"}]: dispatch 2024-09-16T00:34:12.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:11 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:34:12.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:11 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:34:12.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:11 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:34:14.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:13 smithi169 ceph-mon[77626]: pgmap v193: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:34:14.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:13 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:34:14.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:13 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:34:14.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:13 smithi112 ceph-mon[97750]: pgmap v193: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:34:14.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:13 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:34:14.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:13 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:34:15.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:14 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:34:15.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:14 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:34:15.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:14 smithi169 ceph-mon[77626]: pgmap v194: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:34:15.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:14 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:34:15.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:14 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:34:15.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:14 smithi112 ceph-mon[97750]: pgmap v194: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:34:16.715 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:16 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:34:17.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:16 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:34:17.849 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:17 smithi169 ceph-mon[77626]: pgmap v195: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:34:17.849 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:17 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:34:17.849 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:17 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:34:17.849 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:17 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:34:17.849 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:17 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:34:17.849 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:17 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:34:17.849 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:17 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:34:17.849 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:17 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:34:17.849 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:17 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:34:17.849 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:17 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:34:17.849 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:17 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:34:18.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:17 smithi112 ceph-mon[97750]: pgmap v195: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:34:18.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:17 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:34:18.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:17 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:34:18.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:17 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:34:18.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:17 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:34:18.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:17 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:34:18.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:17 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:34:18.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:17 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:34:18.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:17 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:34:18.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:17 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:34:18.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:17 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:34:19.772 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:19 smithi169 ceph-mon[77626]: Upgrade: Updating mds.cephfs.smithi169.lgeqrj 2024-09-16T00:34:19.773 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:19 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:34:19.773 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:19 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi169.lgeqrj", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:34:19.773 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:19 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:34:19.773 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:19 smithi169 ceph-mon[77626]: Deploying daemon mds.cephfs.smithi169.lgeqrj on smithi169 2024-09-16T00:34:19.773 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:19 smithi169 ceph-mon[77626]: pgmap v196: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:34:19.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:19 smithi112 ceph-mon[97750]: Upgrade: Updating mds.cephfs.smithi169.lgeqrj 2024-09-16T00:34:19.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:19 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:34:19.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:19 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi169.lgeqrj", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:34:19.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:19 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:34:19.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:19 smithi112 ceph-mon[97750]: Deploying daemon mds.cephfs.smithi169.lgeqrj on smithi169 2024-09-16T00:34:19.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:19 smithi112 ceph-mon[97750]: pgmap v196: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:34:21.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:21 smithi169 ceph-mon[77626]: osdmap e90: 6 total, 6 up, 6 in 2024-09-16T00:34:21.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:21 smithi169 ceph-mon[77626]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-09-16T00:34:21.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:21 smithi169 ceph-mon[77626]: pgmap v198: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:34:21.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:21 smithi112 ceph-mon[97750]: osdmap e90: 6 total, 6 up, 6 in 2024-09-16T00:34:21.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:21 smithi112 ceph-mon[97750]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-09-16T00:34:21.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:21 smithi112 ceph-mon[97750]: pgmap v198: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:34:24.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:23 smithi169 ceph-mon[77626]: pgmap v199: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:34:24.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:23 smithi112 ceph-mon[97750]: pgmap v199: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:34:24.919 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:24 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi169.lgeqrj"}]: dispatch 2024-09-16T00:34:25.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:24 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi169.lgeqrj"}]: dispatch 2024-09-16T00:34:26.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:25 smithi169 ceph-mon[77626]: pgmap v200: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:34:26.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:25 smithi112 ceph-mon[97750]: pgmap v200: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:34:26.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.441+0000 7efc364ec640 1 -- 172.21.15.112:0/1565033546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc300fe450 msgr2=0x7efc300fe820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:26.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.441+0000 7efc364ec640 1 --2- 172.21.15.112:0/1565033546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc300fe450 0x7efc300fe820 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7efc200048d0 tx=0x7efc20009da0 comp rx=0 tx=0).stop 2024-09-16T00:34:26.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.441+0000 7efc364ec640 1 -- 172.21.15.112:0/1565033546 shutdown_connections 2024-09-16T00:34:26.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.441+0000 7efc364ec640 1 --2- 172.21.15.112:0/1565033546 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc30064fe0 0x7efc30065430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:26.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.441+0000 7efc364ec640 1 --2- 172.21.15.112:0/1565033546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc300fe450 0x7efc300fe820 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:26.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.441+0000 7efc364ec640 1 -- 172.21.15.112:0/1565033546 >> 172.21.15.112:0/1565033546 conn(0x7efc300723b0 msgr2=0x7efc300727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:26.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.441+0000 7efc364ec640 1 -- 172.21.15.112:0/1565033546 shutdown_connections 2024-09-16T00:34:26.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.441+0000 7efc364ec640 1 -- 172.21.15.112:0/1565033546 wait complete. 2024-09-16T00:34:26.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.442+0000 7efc364ec640 1 Processor -- start 2024-09-16T00:34:26.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.442+0000 7efc364ec640 1 -- start start 2024-09-16T00:34:26.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.442+0000 7efc364ec640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc30064fe0 0x7efc300fb560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:26.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.442+0000 7efc364ec640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc300fe450 0x7efc300fbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:26.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.442+0000 7efc364ec640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc300fbfe0 con 0x7efc30064fe0 2024-09-16T00:34:26.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.442+0000 7efc364ec640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc300fc150 con 0x7efc300fe450 2024-09-16T00:34:26.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.442+0000 7efc2ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc30064fe0 0x7efc300fb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:26.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.442+0000 7efc2ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc30064fe0 0x7efc300fb560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45588/0 (socket says 172.21.15.112:45588) 2024-09-16T00:34:26.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.442+0000 7efc2ffff640 1 -- 172.21.15.112:0/939063514 learned_addr learned my addr 172.21.15.112:0/939063514 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:34:26.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.442+0000 7efc2f7fe640 1 --2- 172.21.15.112:0/939063514 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc300fe450 0x7efc300fbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:26.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.443+0000 7efc2ffff640 1 -- 172.21.15.112:0/939063514 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc300fe450 msgr2=0x7efc300fbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:26.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.443+0000 7efc2ffff640 1 --2- 172.21.15.112:0/939063514 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc300fe450 0x7efc300fbaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:26.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.443+0000 7efc2ffff640 1 -- 172.21.15.112:0/939063514 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc20005040 con 0x7efc30064fe0 2024-09-16T00:34:26.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.443+0000 7efc2ffff640 1 --2- 172.21.15.112:0/939063514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc30064fe0 0x7efc300fb560 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7efc20000c00 tx=0x7efc20009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:26.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.443+0000 7efc2cff9640 1 -- 172.21.15.112:0/939063514 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc20038070 con 0x7efc30064fe0 2024-09-16T00:34:26.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.443+0000 7efc2cff9640 1 -- 172.21.15.112:0/939063514 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7efc2002ade0 con 0x7efc30064fe0 2024-09-16T00:34:26.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.443+0000 7efc2cff9640 1 -- 172.21.15.112:0/939063514 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc2002c7e0 con 0x7efc30064fe0 2024-09-16T00:34:26.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.443+0000 7efc364ec640 1 -- 172.21.15.112:0/939063514 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc300fc380 con 0x7efc30064fe0 2024-09-16T00:34:26.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.443+0000 7efc364ec640 1 -- 172.21.15.112:0/939063514 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc30119810 con 0x7efc30064fe0 2024-09-16T00:34:26.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.444+0000 7efc364ec640 1 -- 172.21.15.112:0/939063514 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc300fdce0 con 0x7efc30064fe0 2024-09-16T00:34:26.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.445+0000 7efc2cff9640 1 -- 172.21.15.112:0/939063514 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7efc20030020 con 0x7efc30064fe0 2024-09-16T00:34:26.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.445+0000 7efc2cff9640 1 --2- 172.21.15.112:0/939063514 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7efc1806e530 0x7efc180709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:26.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.445+0000 7efc2cff9640 1 -- 172.21.15.112:0/939063514 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7efc200aded0 con 0x7efc30064fe0 2024-09-16T00:34:26.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.445+0000 7efc2f7fe640 1 --2- 172.21.15.112:0/939063514 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7efc1806e530 0x7efc180709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:26.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.446+0000 7efc2f7fe640 1 --2- 172.21.15.112:0/939063514 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7efc1806e530 0x7efc180709e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7efc300fcea0 tx=0x7efc2400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:26.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.450+0000 7efc2cff9640 1 -- 172.21.15.112:0/939063514 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7efc20076f70 con 0x7efc30064fe0 2024-09-16T00:34:26.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.638+0000 7efc364ec640 1 -- 172.21.15.112:0/939063514 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7efc300fcc50 con 0x7efc1806e530 2024-09-16T00:34:26.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.639+0000 7efc2cff9640 1 -- 172.21.15.112:0/939063514 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7efc300fcc50 con 0x7efc1806e530 2024-09-16T00:34:26.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.641+0000 7efc364ec640 1 -- 172.21.15.112:0/939063514 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7efc1806e530 msgr2=0x7efc180709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:26.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.641+0000 7efc364ec640 1 --2- 172.21.15.112:0/939063514 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7efc1806e530 0x7efc180709e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7efc300fcea0 tx=0x7efc2400b040 comp rx=0 tx=0).stop 2024-09-16T00:34:26.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.641+0000 7efc364ec640 1 -- 172.21.15.112:0/939063514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc30064fe0 msgr2=0x7efc300fb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:26.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.641+0000 7efc364ec640 1 --2- 172.21.15.112:0/939063514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc30064fe0 0x7efc300fb560 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7efc20000c00 tx=0x7efc20009da0 comp rx=0 tx=0).stop 2024-09-16T00:34:26.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.641+0000 7efc364ec640 1 -- 172.21.15.112:0/939063514 shutdown_connections 2024-09-16T00:34:26.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.641+0000 7efc364ec640 1 --2- 172.21.15.112:0/939063514 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7efc1806e530 0x7efc180709e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:26.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.641+0000 7efc364ec640 1 --2- 172.21.15.112:0/939063514 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc300fe450 0x7efc300fbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:26.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.641+0000 7efc364ec640 1 --2- 172.21.15.112:0/939063514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc30064fe0 0x7efc300fb560 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:26.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.641+0000 7efc364ec640 1 -- 172.21.15.112:0/939063514 >> 172.21.15.112:0/939063514 conn(0x7efc300723b0 msgr2=0x7efc300fa2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:26.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.641+0000 7efc364ec640 1 -- 172.21.15.112:0/939063514 shutdown_connections 2024-09-16T00:34:26.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.641+0000 7efc364ec640 1 -- 172.21.15.112:0/939063514 wait complete. 2024-09-16T00:34:26.650 INFO:teuthology.orchestra.run.smithi112.stdout:true 2024-09-16T00:34:26.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.721+0000 7ff1bdad7640 1 -- 172.21.15.112:0/2878374896 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1b81014c0 msgr2=0x7ff1b8101890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:26.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.721+0000 7ff1bdad7640 1 --2- 172.21.15.112:0/2878374896 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1b81014c0 0x7ff1b8101890 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff1a80048d0 tx=0x7ff1a8009da0 comp rx=0 tx=0).stop 2024-09-16T00:34:26.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.721+0000 7ff1bdad7640 1 -- 172.21.15.112:0/2878374896 shutdown_connections 2024-09-16T00:34:26.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.721+0000 7ff1bdad7640 1 --2- 172.21.15.112:0/2878374896 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1b80fb480 0x7ff1b80fb8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:26.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.721+0000 7ff1bdad7640 1 --2- 172.21.15.112:0/2878374896 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1b81014c0 0x7ff1b8101890 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:26.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.721+0000 7ff1bdad7640 1 -- 172.21.15.112:0/2878374896 >> 172.21.15.112:0/2878374896 conn(0x7ff1b80f7020 msgr2=0x7ff1b80f9430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:26.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.722+0000 7ff1bdad7640 1 -- 172.21.15.112:0/2878374896 shutdown_connections 2024-09-16T00:34:26.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.722+0000 7ff1bdad7640 1 -- 172.21.15.112:0/2878374896 wait complete. 2024-09-16T00:34:26.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.722+0000 7ff1bdad7640 1 Processor -- start 2024-09-16T00:34:26.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.722+0000 7ff1bdad7640 1 -- start start 2024-09-16T00:34:26.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.722+0000 7ff1bdad7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1b80fb480 0x7ff1b810e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:26.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.722+0000 7ff1bdad7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1b81014c0 0x7ff1b810e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:26.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.722+0000 7ff1bdad7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b810ed40 con 0x7ff1b80fb480 2024-09-16T00:34:26.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.722+0000 7ff1bdad7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b810eeb0 con 0x7ff1b81014c0 2024-09-16T00:34:26.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.723+0000 7ff1b77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1b80fb480 0x7ff1b810e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:26.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.723+0000 7ff1b77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1b80fb480 0x7ff1b810e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45600/0 (socket says 172.21.15.112:45600) 2024-09-16T00:34:26.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.723+0000 7ff1b77fe640 1 -- 172.21.15.112:0/812858452 learned_addr learned my addr 172.21.15.112:0/812858452 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:34:26.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.723+0000 7ff1affff640 1 --2- 172.21.15.112:0/812858452 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1b81014c0 0x7ff1b810e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:26.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.723+0000 7ff1b77fe640 1 -- 172.21.15.112:0/812858452 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1b81014c0 msgr2=0x7ff1b810e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:26.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.723+0000 7ff1b77fe640 1 --2- 172.21.15.112:0/812858452 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1b81014c0 0x7ff1b810e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:26.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.723+0000 7ff1b77fe640 1 -- 172.21.15.112:0/812858452 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1a8005040 con 0x7ff1b80fb480 2024-09-16T00:34:26.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.723+0000 7ff1affff640 1 --2- 172.21.15.112:0/812858452 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1b81014c0 0x7ff1b810e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:34:26.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.723+0000 7ff1b77fe640 1 --2- 172.21.15.112:0/812858452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1b80fb480 0x7ff1b810e1b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff1a8000c00 tx=0x7ff1a8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:26.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.723+0000 7ff1b4ff9640 1 -- 172.21.15.112:0/812858452 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff1a8038070 con 0x7ff1b80fb480 2024-09-16T00:34:26.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.723+0000 7ff1b4ff9640 1 -- 172.21.15.112:0/812858452 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7ff1a802ade0 con 0x7ff1b80fb480 2024-09-16T00:34:26.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.723+0000 7ff1bdad7640 1 -- 172.21.15.112:0/812858452 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1b8117220 con 0x7ff1b80fb480 2024-09-16T00:34:26.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.723+0000 7ff1bdad7640 1 -- 172.21.15.112:0/812858452 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1b8117600 con 0x7ff1b80fb480 2024-09-16T00:34:26.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.723+0000 7ff1b4ff9640 1 -- 172.21.15.112:0/812858452 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff1a802c7e0 con 0x7ff1b80fb480 2024-09-16T00:34:26.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.725+0000 7ff1bdad7640 1 -- 172.21.15.112:0/812858452 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1b80fcbc0 con 0x7ff1b80fb480 2024-09-16T00:34:26.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.725+0000 7ff1b4ff9640 1 -- 172.21.15.112:0/812858452 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7ff1a8030020 con 0x7ff1b80fb480 2024-09-16T00:34:26.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.725+0000 7ff1b4ff9640 1 --2- 172.21.15.112:0/812858452 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff1a406e530 0x7ff1a40709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:26.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.725+0000 7ff1b4ff9640 1 -- 172.21.15.112:0/812858452 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7ff1a80adf80 con 0x7ff1b80fb480 2024-09-16T00:34:26.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.725+0000 7ff1affff640 1 --2- 172.21.15.112:0/812858452 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff1a406e530 0x7ff1a40709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:26.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.726+0000 7ff1affff640 1 --2- 172.21.15.112:0/812858452 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff1a406e530 0x7ff1a40709e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff1b8066490 tx=0x7ff1a0009ed0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:26.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.730+0000 7ff1b4ff9640 1 -- 172.21.15.112:0/812858452 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7ff1a8076fa0 con 0x7ff1b80fb480 2024-09-16T00:34:26.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.912+0000 7ff1bdad7640 1 -- 172.21.15.112:0/812858452 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff1b80f7710 con 0x7ff1a406e530 2024-09-16T00:34:26.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.913+0000 7ff1b4ff9640 1 -- 172.21.15.112:0/812858452 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7ff1a0007b30 con 0x7ff1a406e530 2024-09-16T00:34:26.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.915+0000 7ff1bdad7640 1 -- 172.21.15.112:0/812858452 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff1a406e530 msgr2=0x7ff1a40709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:26.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.915+0000 7ff1bdad7640 1 --2- 172.21.15.112:0/812858452 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff1a406e530 0x7ff1a40709e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff1b8066490 tx=0x7ff1a0009ed0 comp rx=0 tx=0).stop 2024-09-16T00:34:26.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.915+0000 7ff1bdad7640 1 -- 172.21.15.112:0/812858452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1b80fb480 msgr2=0x7ff1b810e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:26.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.915+0000 7ff1bdad7640 1 --2- 172.21.15.112:0/812858452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1b80fb480 0x7ff1b810e1b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff1a8000c00 tx=0x7ff1a8009da0 comp rx=0 tx=0).stop 2024-09-16T00:34:26.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.915+0000 7ff1bdad7640 1 -- 172.21.15.112:0/812858452 shutdown_connections 2024-09-16T00:34:26.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.915+0000 7ff1bdad7640 1 --2- 172.21.15.112:0/812858452 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff1a406e530 0x7ff1a40709e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:26.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.915+0000 7ff1bdad7640 1 --2- 172.21.15.112:0/812858452 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1b81014c0 0x7ff1b810e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:26.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.915+0000 7ff1bdad7640 1 --2- 172.21.15.112:0/812858452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1b80fb480 0x7ff1b810e1b0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:26.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.915+0000 7ff1bdad7640 1 -- 172.21.15.112:0/812858452 >> 172.21.15.112:0/812858452 conn(0x7ff1b80f7020 msgr2=0x7ff1b80f8920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:26.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.915+0000 7ff1bdad7640 1 -- 172.21.15.112:0/812858452 shutdown_connections 2024-09-16T00:34:26.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.915+0000 7ff1bdad7640 1 -- 172.21.15.112:0/812858452 wait complete. 2024-09-16T00:34:26.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.994+0000 7f34699eb640 1 -- 172.21.15.112:0/256680568 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34640fcf80 msgr2=0x7f34640fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:26.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.994+0000 7f34699eb640 1 --2- 172.21.15.112:0/256680568 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34640fcf80 0x7f34640fd3d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f344c0048d0 tx=0x7f344c009da0 comp rx=0 tx=0).stop 2024-09-16T00:34:26.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.994+0000 7f34699eb640 1 -- 172.21.15.112:0/256680568 shutdown_connections 2024-09-16T00:34:26.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.994+0000 7f34699eb640 1 --2- 172.21.15.112:0/256680568 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34640fcf80 0x7f34640fd3d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:26.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.994+0000 7f34699eb640 1 --2- 172.21.15.112:0/256680568 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34640fb810 0x7f34640fbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:26.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.994+0000 7f34699eb640 1 -- 172.21.15.112:0/256680568 >> 172.21.15.112:0/256680568 conn(0x7f34640723b0 msgr2=0x7f34640727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:26.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.994+0000 7f34699eb640 1 -- 172.21.15.112:0/256680568 shutdown_connections 2024-09-16T00:34:26.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.994+0000 7f34699eb640 1 -- 172.21.15.112:0/256680568 wait complete. 2024-09-16T00:34:26.995 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:26 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:34:26.995 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:26 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:34:26.995 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:26 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:34:26.995 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:26 smithi112 ceph-mon[97750]: pgmap v201: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:34:26.995 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:26 smithi112 ceph-mon[97750]: mds.? [v2:172.21.15.169:6826/184810031,v1:172.21.15.169:6827/184810031] up:boot 2024-09-16T00:34:26.996 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:26 smithi112 ceph-mon[97750]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-09-16T00:34:26.996 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:26 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi169.lgeqrj"}]: dispatch 2024-09-16T00:34:26.996 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:26 smithi112 ceph-mon[97750]: from='client.34436 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:34:26.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.994+0000 7f34699eb640 1 Processor -- start 2024-09-16T00:34:26.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.994+0000 7f34699eb640 1 -- start start 2024-09-16T00:34:26.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.994+0000 7f34699eb640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34640fb810 0x7f3464113180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:26.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.994+0000 7f34699eb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34640fcf80 0x7f34641136c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:26.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.994+0000 7f34699eb640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3464113cc0 con 0x7f34640fcf80 2024-09-16T00:34:26.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.994+0000 7f34699eb640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f346410d100 con 0x7f34640fb810 2024-09-16T00:34:26.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.995+0000 7f345bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34640fcf80 0x7f34641136c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:26.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.995+0000 7f345bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34640fcf80 0x7f34641136c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45616/0 (socket says 172.21.15.112:45616) 2024-09-16T00:34:26.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.995+0000 7f3462ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34640fb810 0x7f3464113180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:26.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.995+0000 7f345bfff640 1 -- 172.21.15.112:0/3760699570 learned_addr learned my addr 172.21.15.112:0/3760699570 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:34:26.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.995+0000 7f345bfff640 1 -- 172.21.15.112:0/3760699570 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34640fb810 msgr2=0x7f3464113180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:26.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.995+0000 7f345bfff640 1 --2- 172.21.15.112:0/3760699570 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34640fb810 0x7f3464113180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:26.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.995+0000 7f345bfff640 1 -- 172.21.15.112:0/3760699570 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f344c005040 con 0x7f34640fcf80 2024-09-16T00:34:26.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.995+0000 7f345bfff640 1 --2- 172.21.15.112:0/3760699570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34640fcf80 0x7f34641136c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f344c003080 tx=0x7f344c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:26.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.995+0000 7f345b7fe640 1 -- 172.21.15.112:0/3760699570 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f344c038070 con 0x7f34640fcf80 2024-09-16T00:34:26.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.995+0000 7f345b7fe640 1 -- 172.21.15.112:0/3760699570 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f344c0097e0 con 0x7f34640fcf80 2024-09-16T00:34:26.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.995+0000 7f345b7fe640 1 -- 172.21.15.112:0/3760699570 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f344c03d420 con 0x7f34640fcf80 2024-09-16T00:34:26.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.995+0000 7f34699eb640 1 -- 172.21.15.112:0/3760699570 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f346410d380 con 0x7f34640fcf80 2024-09-16T00:34:26.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.996+0000 7f34699eb640 1 -- 172.21.15.112:0/3760699570 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f346410d8c0 con 0x7f34640fcf80 2024-09-16T00:34:26.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.997+0000 7f34699eb640 1 -- 172.21.15.112:0/3760699570 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34640fcbc0 con 0x7f34640fcf80 2024-09-16T00:34:26.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.997+0000 7f345b7fe640 1 -- 172.21.15.112:0/3760699570 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f344c030020 con 0x7f34640fcf80 2024-09-16T00:34:26.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.997+0000 7f345b7fe640 1 --2- 172.21.15.112:0/3760699570 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f345006e530 0x7f34500709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:26.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.998+0000 7f345b7fe640 1 -- 172.21.15.112:0/3760699570 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f344c0ae760 con 0x7f34640fcf80 2024-09-16T00:34:26.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.998+0000 7f3462ffd640 1 --2- 172.21.15.112:0/3760699570 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f345006e530 0x7f34500709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:26.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:26.998+0000 7f3462ffd640 1 --2- 172.21.15.112:0/3760699570 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f345006e530 0x7f34500709e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f34540041d0 tx=0x7f3454004330 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:27.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.002+0000 7f345b7fe640 1 -- 172.21.15.112:0/3760699570 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f344c077750 con 0x7f34640fcf80 2024-09-16T00:34:27.137 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:26 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:34:27.137 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:26 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:34:27.137 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:26 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:34:27.137 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:26 smithi169 ceph-mon[77626]: pgmap v201: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:34:27.137 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:26 smithi169 ceph-mon[77626]: mds.? [v2:172.21.15.169:6826/184810031,v1:172.21.15.169:6827/184810031] up:boot 2024-09-16T00:34:27.138 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:26 smithi169 ceph-mon[77626]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-09-16T00:34:27.138 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:26 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi169.lgeqrj"}]: dispatch 2024-09-16T00:34:27.138 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:26 smithi169 ceph-mon[77626]: from='client.34436 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:34:27.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.176+0000 7f34699eb640 1 -- 172.21.15.112:0/3760699570 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f346410e7f0 con 0x7f345006e530 2024-09-16T00:34:27.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.184+0000 7f345b7fe640 1 -- 172.21.15.112:0/3760699570 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7f346410e7f0 con 0x7f345006e530 2024-09-16T00:34:27.185 INFO:teuthology.orchestra.run.smithi112.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T00:34:27.185 INFO:teuthology.orchestra.run.smithi112.stdout:alertmanager.smithi112 smithi112 *:9093,9094 running (7m) 28s ago 16m 24.7M - 0.25.0 c8568f914cd2 d85ea0562022 2024-09-16T00:34:27.185 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi112 smithi112 running (4m) 28s ago 16m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c e4fe58f9702a 2024-09-16T00:34:27.185 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi169 smithi169 running (4m) 13s ago 15m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c 3ba2c81d527f 2024-09-16T00:34:27.185 INFO:teuthology.orchestra.run.smithi112.stdout:grafana.smithi112 smithi112 *:3000 running (6m) 28s ago 16m 82.1M - 9.4.12 2bacad6d85d8 51f9d2013da2 2024-09-16T00:34:27.185 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.hxfwkh smithi112 running (32s) 28s ago 10m 14.2M - 19.1.1-326-g8921c8ef f1455bd0219c 1723069902d6 2024-09-16T00:34:27.185 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.ithwzj smithi112 running (49s) 28s ago 11m 12.2M - 19.1.1-326-g8921c8ef f1455bd0219c e7aad5b81db5 2024-09-16T00:34:27.185 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.gawwgw smithi169 running (17s) 13s ago 10m 15.6M - 19.1.1-326-g8921c8ef f1455bd0219c 469be459ee12 2024-09-16T00:34:27.186 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.lgeqrj smithi169 starting - - - - 2024-09-16T00:34:27.186 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi112.xrkkbc smithi112 *:8443,9283,8765 running (7m) 28s ago 17m 504M - 19.1.1-326-g8921c8ef f1455bd0219c a981f7fd2810 2024-09-16T00:34:27.186 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi169.mtrobk smithi169 *:8443,9283,8765 running (6m) 13s ago 15m 442M - 19.1.1-326-g8921c8ef f1455bd0219c 4096e5369c2d 2024-09-16T00:34:27.186 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi112 smithi112 running (5m) 28s ago 17m 49.8M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 5aaec033c042 2024-09-16T00:34:27.186 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi169 smithi169 running (4m) 13s ago 15m 38.4M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 7bc609155f3f 2024-09-16T00:34:27.186 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi112 smithi112 *:9100 running (7m) 28s ago 16m 15.5M - 1.5.0 0da6a335fe13 6404d1d601e2 2024-09-16T00:34:27.186 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi169 smithi169 *:9100 running (7m) 13s ago 15m 14.9M - 1.5.0 0da6a335fe13 5505baf702a6 2024-09-16T00:34:27.186 INFO:teuthology.orchestra.run.smithi112.stdout:osd.0 smithi112 running (3m) 28s ago 14m 105M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 53efc9397017 2024-09-16T00:34:27.186 INFO:teuthology.orchestra.run.smithi112.stdout:osd.1 smithi112 running (3m) 28s ago 14m 98.9M 1557M 19.1.1-326-g8921c8ef f1455bd0219c fe6f1a4a2cf4 2024-09-16T00:34:27.186 INFO:teuthology.orchestra.run.smithi112.stdout:osd.2 smithi112 running (2m) 28s ago 13m 102M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 5f7863712057 2024-09-16T00:34:27.186 INFO:teuthology.orchestra.run.smithi112.stdout:osd.3 smithi169 running (2m) 13s ago 13m 92.0M 2581M 19.1.1-326-g8921c8ef f1455bd0219c ee07b5577810 2024-09-16T00:34:27.186 INFO:teuthology.orchestra.run.smithi112.stdout:osd.4 smithi169 running (99s) 13s ago 12m 103M 2581M 19.1.1-326-g8921c8ef f1455bd0219c 769f026a54eb 2024-09-16T00:34:27.186 INFO:teuthology.orchestra.run.smithi112.stdout:osd.5 smithi169 running (67s) 13s ago 12m 125M 2581M 19.1.1-326-g8921c8ef f1455bd0219c a6007c13cf78 2024-09-16T00:34:27.186 INFO:teuthology.orchestra.run.smithi112.stdout:prometheus.smithi112 smithi112 *:9095 running (6m) 28s ago 15m 64.1M - 2.43.0 a07b618ecd1d 7a1d42e60d2b 2024-09-16T00:34:27.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.186+0000 7f34699eb640 1 -- 172.21.15.112:0/3760699570 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f345006e530 msgr2=0x7f34500709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:27.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.186+0000 7f34699eb640 1 --2- 172.21.15.112:0/3760699570 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f345006e530 0x7f34500709e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f34540041d0 tx=0x7f3454004330 comp rx=0 tx=0).stop 2024-09-16T00:34:27.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.186+0000 7f34699eb640 1 -- 172.21.15.112:0/3760699570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34640fcf80 msgr2=0x7f34641136c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:27.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.186+0000 7f34699eb640 1 --2- 172.21.15.112:0/3760699570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34640fcf80 0x7f34641136c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f344c003080 tx=0x7f344c009da0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.187+0000 7f34699eb640 1 -- 172.21.15.112:0/3760699570 shutdown_connections 2024-09-16T00:34:27.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.187+0000 7f34699eb640 1 --2- 172.21.15.112:0/3760699570 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f345006e530 0x7f34500709e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.187+0000 7f34699eb640 1 --2- 172.21.15.112:0/3760699570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34640fcf80 0x7f34641136c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.187+0000 7f34699eb640 1 --2- 172.21.15.112:0/3760699570 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34640fb810 0x7f3464113180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.187+0000 7f34699eb640 1 -- 172.21.15.112:0/3760699570 >> 172.21.15.112:0/3760699570 conn(0x7f34640723b0 msgr2=0x7f3464065810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:27.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.187+0000 7f34699eb640 1 -- 172.21.15.112:0/3760699570 shutdown_connections 2024-09-16T00:34:27.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.187+0000 7f34699eb640 1 -- 172.21.15.112:0/3760699570 wait complete. 2024-09-16T00:34:27.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.264+0000 7ff0d0383640 1 -- 172.21.15.112:0/4039332311 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0c8102680 msgr2=0x7ff0c8102a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:27.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.264+0000 7ff0d0383640 1 --2- 172.21.15.112:0/4039332311 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0c8102680 0x7ff0c8102a50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff0c40048d0 tx=0x7ff0c4009da0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.264+0000 7ff0d0383640 1 -- 172.21.15.112:0/4039332311 shutdown_connections 2024-09-16T00:34:27.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.264+0000 7ff0d0383640 1 --2- 172.21.15.112:0/4039332311 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff0c80fabd0 0x7ff0c80fb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.264+0000 7ff0d0383640 1 --2- 172.21.15.112:0/4039332311 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0c8102680 0x7ff0c8102a50 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.264+0000 7ff0d0383640 1 -- 172.21.15.112:0/4039332311 >> 172.21.15.112:0/4039332311 conn(0x7ff0c8071ac0 msgr2=0x7ff0c8071ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:27.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.264+0000 7ff0d0383640 1 -- 172.21.15.112:0/4039332311 shutdown_connections 2024-09-16T00:34:27.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.264+0000 7ff0d0383640 1 -- 172.21.15.112:0/4039332311 wait complete. 2024-09-16T00:34:27.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.265+0000 7ff0d0383640 1 Processor -- start 2024-09-16T00:34:27.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.265+0000 7ff0d0383640 1 -- start start 2024-09-16T00:34:27.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.265+0000 7ff0d0383640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff0c80fabd0 0x7ff0c81132b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:27.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.265+0000 7ff0d0383640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0c8102680 0x7ff0c81137f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:27.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.265+0000 7ff0d0383640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c8113df0 con 0x7ff0c8102680 2024-09-16T00:34:27.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.265+0000 7ff0d0383640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c810d230 con 0x7ff0c80fabd0 2024-09-16T00:34:27.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.265+0000 7ff0cd91e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0c8102680 0x7ff0c81137f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:27.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.266+0000 7ff0ce11f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff0c80fabd0 0x7ff0c81132b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:27.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.266+0000 7ff0cd91e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0c8102680 0x7ff0c81137f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45626/0 (socket says 172.21.15.112:45626) 2024-09-16T00:34:27.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.266+0000 7ff0cd91e640 1 -- 172.21.15.112:0/4140297602 learned_addr learned my addr 172.21.15.112:0/4140297602 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:34:27.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.266+0000 7ff0cd91e640 1 -- 172.21.15.112:0/4140297602 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff0c80fabd0 msgr2=0x7ff0c81132b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:27.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.266+0000 7ff0cd91e640 1 --2- 172.21.15.112:0/4140297602 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff0c80fabd0 0x7ff0c81132b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.266+0000 7ff0cd91e640 1 -- 172.21.15.112:0/4140297602 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c4005040 con 0x7ff0c8102680 2024-09-16T00:34:27.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.266+0000 7ff0cd91e640 1 --2- 172.21.15.112:0/4140297602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0c8102680 0x7ff0c81137f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff0b800eb70 tx=0x7ff0b800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:27.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.266+0000 7ff0beffd640 1 -- 172.21.15.112:0/4140297602 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff0b8007c30 con 0x7ff0c8102680 2024-09-16T00:34:27.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.266+0000 7ff0beffd640 1 -- 172.21.15.112:0/4140297602 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7ff0b800f040 con 0x7ff0c8102680 2024-09-16T00:34:27.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.266+0000 7ff0d0383640 1 -- 172.21.15.112:0/4140297602 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0c810d4b0 con 0x7ff0c8102680 2024-09-16T00:34:27.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.266+0000 7ff0beffd640 1 -- 172.21.15.112:0/4140297602 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff0b800ca60 con 0x7ff0c8102680 2024-09-16T00:34:27.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.266+0000 7ff0d0383640 1 -- 172.21.15.112:0/4140297602 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0c810da50 con 0x7ff0c8102680 2024-09-16T00:34:27.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.268+0000 7ff0beffd640 1 -- 172.21.15.112:0/4140297602 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7ff0b800cbc0 con 0x7ff0c8102680 2024-09-16T00:34:27.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.268+0000 7ff0d0383640 1 -- 172.21.15.112:0/4140297602 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0c80725d0 con 0x7ff0c8102680 2024-09-16T00:34:27.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.268+0000 7ff0beffd640 1 --2- 172.21.15.112:0/4140297602 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff0b406e530 0x7ff0b40709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:27.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.268+0000 7ff0beffd640 1 -- 172.21.15.112:0/4140297602 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7ff0b808e040 con 0x7ff0c8102680 2024-09-16T00:34:27.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.268+0000 7ff0ce11f640 1 --2- 172.21.15.112:0/4140297602 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff0b406e530 0x7ff0b40709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:27.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.269+0000 7ff0ce11f640 1 --2- 172.21.15.112:0/4140297602 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff0b406e530 0x7ff0b40709e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4004540 tx=0x7ff0c4035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:27.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.273+0000 7ff0beffd640 1 -- 172.21.15.112:0/4140297602 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7ff0b8057f10 con 0x7ff0c8102680 2024-09-16T00:34:27.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.503+0000 7ff0d0383640 1 -- 172.21.15.112:0/4140297602 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff0c8109630 con 0x7ff0c8102680 2024-09-16T00:34:27.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.504+0000 7ff0beffd640 1 -- 172.21.15.112:0/4140297602 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7ff0b805b5d0 con 0x7ff0c8102680 2024-09-16T00:34:27.504 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:34:27.505 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": { 2024-09-16T00:34:27.505 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:34:27.505 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:34:27.505 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": { 2024-09-16T00:34:27.505 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:34:27.505 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:34:27.505 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": { 2024-09-16T00:34:27.505 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 6 2024-09-16T00:34:27.505 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:34:27.505 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": { 2024-09-16T00:34:27.505 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 4 2024-09-16T00:34:27.505 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:34:27.505 INFO:teuthology.orchestra.run.smithi112.stdout: "overall": { 2024-09-16T00:34:27.506 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 14 2024-09-16T00:34:27.506 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-16T00:34:27.506 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:34:27.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.506+0000 7ff0d0383640 1 -- 172.21.15.112:0/4140297602 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff0b406e530 msgr2=0x7ff0b40709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:27.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.506+0000 7ff0d0383640 1 --2- 172.21.15.112:0/4140297602 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff0b406e530 0x7ff0b40709e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4004540 tx=0x7ff0c4035040 comp rx=0 tx=0).stop 2024-09-16T00:34:27.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.506+0000 7ff0d0383640 1 -- 172.21.15.112:0/4140297602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0c8102680 msgr2=0x7ff0c81137f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:27.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.506+0000 7ff0d0383640 1 --2- 172.21.15.112:0/4140297602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0c8102680 0x7ff0c81137f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff0b800eb70 tx=0x7ff0b800b040 comp rx=0 tx=0).stop 2024-09-16T00:34:27.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.506+0000 7ff0d0383640 1 -- 172.21.15.112:0/4140297602 shutdown_connections 2024-09-16T00:34:27.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.506+0000 7ff0d0383640 1 --2- 172.21.15.112:0/4140297602 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff0b406e530 0x7ff0b40709e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.506+0000 7ff0d0383640 1 --2- 172.21.15.112:0/4140297602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0c8102680 0x7ff0c81137f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.506+0000 7ff0d0383640 1 --2- 172.21.15.112:0/4140297602 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff0c80fabd0 0x7ff0c81132b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.506+0000 7ff0d0383640 1 -- 172.21.15.112:0/4140297602 >> 172.21.15.112:0/4140297602 conn(0x7ff0c8071ac0 msgr2=0x7ff0c8105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:27.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.506+0000 7ff0d0383640 1 -- 172.21.15.112:0/4140297602 shutdown_connections 2024-09-16T00:34:27.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.506+0000 7ff0d0383640 1 -- 172.21.15.112:0/4140297602 wait complete. 2024-09-16T00:34:27.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.585+0000 7f4095cc9640 1 -- 172.21.15.112:0/505395122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40900fcf80 msgr2=0x7f40900fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:27.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.585+0000 7f4095cc9640 1 --2- 172.21.15.112:0/505395122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40900fcf80 0x7f40900fd3d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f40780048d0 tx=0x7f4078009da0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.585+0000 7f4095cc9640 1 -- 172.21.15.112:0/505395122 shutdown_connections 2024-09-16T00:34:27.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.585+0000 7f4095cc9640 1 --2- 172.21.15.112:0/505395122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40900fcf80 0x7f40900fd3d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.585+0000 7f4095cc9640 1 --2- 172.21.15.112:0/505395122 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40900fb810 0x7f40900fbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.585+0000 7f4095cc9640 1 -- 172.21.15.112:0/505395122 >> 172.21.15.112:0/505395122 conn(0x7f40900723b0 msgr2=0x7f40900727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:27.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.585+0000 7f4095cc9640 1 -- 172.21.15.112:0/505395122 shutdown_connections 2024-09-16T00:34:27.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.585+0000 7f4095cc9640 1 -- 172.21.15.112:0/505395122 wait complete. 2024-09-16T00:34:27.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.585+0000 7f4095cc9640 1 Processor -- start 2024-09-16T00:34:27.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.585+0000 7f4095cc9640 1 -- start start 2024-09-16T00:34:27.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.586+0000 7f4095cc9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40900fb810 0x7f4090113150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:27.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.586+0000 7f4095cc9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40900fcf80 0x7f4090113690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:27.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.586+0000 7f4095cc9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4090113c90 con 0x7f40900fcf80 2024-09-16T00:34:27.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.586+0000 7f4095cc9640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f409010d0d0 con 0x7f40900fb810 2024-09-16T00:34:27.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.586+0000 7f4087fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40900fcf80 0x7f4090113690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:27.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.586+0000 7f408f7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40900fb810 0x7f4090113150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:27.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.586+0000 7f4087fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40900fcf80 0x7f4090113690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45656/0 (socket says 172.21.15.112:45656) 2024-09-16T00:34:27.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.586+0000 7f4087fff640 1 -- 172.21.15.112:0/2516290208 learned_addr learned my addr 172.21.15.112:0/2516290208 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:34:27.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.586+0000 7f4087fff640 1 -- 172.21.15.112:0/2516290208 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40900fb810 msgr2=0x7f4090113150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:27.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.586+0000 7f4087fff640 1 --2- 172.21.15.112:0/2516290208 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40900fb810 0x7f4090113150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.586+0000 7f4087fff640 1 -- 172.21.15.112:0/2516290208 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4078005040 con 0x7f40900fcf80 2024-09-16T00:34:27.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.586+0000 7f4087fff640 1 --2- 172.21.15.112:0/2516290208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40900fcf80 0x7f4090113690 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f40780043d0 tx=0x7f4078009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:27.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.586+0000 7f408cff9640 1 -- 172.21.15.112:0/2516290208 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4078038070 con 0x7f40900fcf80 2024-09-16T00:34:27.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.586+0000 7f408cff9640 1 -- 172.21.15.112:0/2516290208 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f40780096e0 con 0x7f40900fcf80 2024-09-16T00:34:27.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.586+0000 7f4095cc9640 1 -- 172.21.15.112:0/2516290208 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f409010d350 con 0x7f40900fcf80 2024-09-16T00:34:27.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.587+0000 7f408cff9640 1 -- 172.21.15.112:0/2516290208 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f407803d420 con 0x7f40900fcf80 2024-09-16T00:34:27.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.587+0000 7f4095cc9640 1 -- 172.21.15.112:0/2516290208 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f409010d870 con 0x7f40900fcf80 2024-09-16T00:34:27.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.588+0000 7f4095cc9640 1 -- 172.21.15.112:0/2516290208 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40900fcbc0 con 0x7f40900fcf80 2024-09-16T00:34:27.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.588+0000 7f408cff9640 1 -- 172.21.15.112:0/2516290208 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f4078030020 con 0x7f40900fcf80 2024-09-16T00:34:27.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.589+0000 7f408cff9640 1 --2- 172.21.15.112:0/2516290208 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f407c06e530 0x7f407c0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:27.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.589+0000 7f408cff9640 1 -- 172.21.15.112:0/2516290208 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f40780ae6e0 con 0x7f40900fcf80 2024-09-16T00:34:27.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.589+0000 7f408f7fe640 1 --2- 172.21.15.112:0/2516290208 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f407c06e530 0x7f407c0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:27.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.589+0000 7f408f7fe640 1 --2- 172.21.15.112:0/2516290208 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f407c06e530 0x7f407c0709e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4080009fd0 tx=0x7f408000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:27.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.593+0000 7f408cff9640 1 -- 172.21.15.112:0/2516290208 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f40780776d0 con 0x7f40900fcf80 2024-09-16T00:34:27.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.793+0000 7f4095cc9640 1 -- 172.21.15.112:0/2516290208 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f409010e7c0 con 0x7f40900fcf80 2024-09-16T00:34:27.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.793+0000 7f408cff9640 1 -- 172.21.15.112:0/2516290208 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1914 (secure 0 0 0) 0x7f407807ad90 con 0x7f40900fcf80 2024-09-16T00:34:27.794 INFO:teuthology.orchestra.run.smithi112.stdout:e34 2024-09-16T00:34:27.794 INFO:teuthology.orchestra.run.smithi112.stdout:btime 2024-09-16T00:34:26:619254+0000 2024-09-16T00:34:27.794 INFO:teuthology.orchestra.run.smithi112.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T00:34:27.794 INFO:teuthology.orchestra.run.smithi112.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:34:27.794 INFO:teuthology.orchestra.run.smithi112.stdout:legacy client fscid: 1 2024-09-16T00:34:27.794 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:34:27.794 INFO:teuthology.orchestra.run.smithi112.stdout:Filesystem 'cephfs' (1) 2024-09-16T00:34:27.794 INFO:teuthology.orchestra.run.smithi112.stdout:fs_name cephfs 2024-09-16T00:34:27.794 INFO:teuthology.orchestra.run.smithi112.stdout:epoch 25 2024-09-16T00:34:27.795 INFO:teuthology.orchestra.run.smithi112.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-16T00:34:27.795 INFO:teuthology.orchestra.run.smithi112.stdout:created 2024-09-16T00:23:22.328199+0000 2024-09-16T00:34:27.795 INFO:teuthology.orchestra.run.smithi112.stdout:modified 2024-09-16T00:33:29.012991+0000 2024-09-16T00:34:27.795 INFO:teuthology.orchestra.run.smithi112.stdout:tableserver 0 2024-09-16T00:34:27.795 INFO:teuthology.orchestra.run.smithi112.stdout:root 0 2024-09-16T00:34:27.795 INFO:teuthology.orchestra.run.smithi112.stdout:session_timeout 60 2024-09-16T00:34:27.795 INFO:teuthology.orchestra.run.smithi112.stdout:session_autoclose 300 2024-09-16T00:34:27.795 INFO:teuthology.orchestra.run.smithi112.stdout:max_file_size 1099511627776 2024-09-16T00:34:27.795 INFO:teuthology.orchestra.run.smithi112.stdout:max_xattr_size 65536 2024-09-16T00:34:27.795 INFO:teuthology.orchestra.run.smithi112.stdout:required_client_features {} 2024-09-16T00:34:27.795 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure 0 2024-09-16T00:34:27.795 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure_osd_epoch 82 2024-09-16T00:34:27.795 INFO:teuthology.orchestra.run.smithi112.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:34:27.795 INFO:teuthology.orchestra.run.smithi112.stdout:max_mds 2 2024-09-16T00:34:27.796 INFO:teuthology.orchestra.run.smithi112.stdout:in 0,1 2024-09-16T00:34:27.796 INFO:teuthology.orchestra.run.smithi112.stdout:up {} 2024-09-16T00:34:27.796 INFO:teuthology.orchestra.run.smithi112.stdout:failed 0,1 2024-09-16T00:34:27.796 INFO:teuthology.orchestra.run.smithi112.stdout:damaged 2024-09-16T00:34:27.796 INFO:teuthology.orchestra.run.smithi112.stdout:stopped 2024-09-16T00:34:27.796 INFO:teuthology.orchestra.run.smithi112.stdout:data_pools [3] 2024-09-16T00:34:27.796 INFO:teuthology.orchestra.run.smithi112.stdout:metadata_pool 2 2024-09-16T00:34:27.796 INFO:teuthology.orchestra.run.smithi112.stdout:inline_data enabled 2024-09-16T00:34:27.796 INFO:teuthology.orchestra.run.smithi112.stdout:balancer 2024-09-16T00:34:27.796 INFO:teuthology.orchestra.run.smithi112.stdout:bal_rank_mask -1 2024-09-16T00:34:27.796 INFO:teuthology.orchestra.run.smithi112.stdout:standby_count_wanted 1 2024-09-16T00:34:27.796 INFO:teuthology.orchestra.run.smithi112.stdout:qdb_cluster leader: 0 members: 2024-09-16T00:34:27.796 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:34:27.797 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:34:27.797 INFO:teuthology.orchestra.run.smithi112.stdout:Standby daemons: 2024-09-16T00:34:27.797 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:34:27.797 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.ithwzj{-1:34392} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.112:6826/476229221,v1:172.21.15.112:6827/476229221] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:34:27.797 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.hxfwkh{-1:34404} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.112:6828/2356899876,v1:172.21.15.112:6829/2356899876] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:34:27.797 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.gawwgw{-1:44343} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.169:6824/1635329957,v1:172.21.15.169:6825/1635329957] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:34:27.797 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.lgeqrj{-1:44347} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.169:6826/184810031,v1:172.21.15.169:6827/184810031] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:34:27.799 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 34 2024-09-16T00:34:27.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.795+0000 7f4095cc9640 1 -- 172.21.15.112:0/2516290208 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f407c06e530 msgr2=0x7f407c0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:27.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.795+0000 7f4095cc9640 1 --2- 172.21.15.112:0/2516290208 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f407c06e530 0x7f407c0709e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4080009fd0 tx=0x7f408000b040 comp rx=0 tx=0).stop 2024-09-16T00:34:27.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.796+0000 7f4095cc9640 1 -- 172.21.15.112:0/2516290208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40900fcf80 msgr2=0x7f4090113690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:27.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.796+0000 7f4095cc9640 1 --2- 172.21.15.112:0/2516290208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40900fcf80 0x7f4090113690 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f40780043d0 tx=0x7f4078009da0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.796+0000 7f4095cc9640 1 -- 172.21.15.112:0/2516290208 shutdown_connections 2024-09-16T00:34:27.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.796+0000 7f4095cc9640 1 --2- 172.21.15.112:0/2516290208 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f407c06e530 0x7f407c0709e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.796+0000 7f4095cc9640 1 --2- 172.21.15.112:0/2516290208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40900fcf80 0x7f4090113690 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.796+0000 7f4095cc9640 1 --2- 172.21.15.112:0/2516290208 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40900fb810 0x7f4090113150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.796+0000 7f4095cc9640 1 -- 172.21.15.112:0/2516290208 >> 172.21.15.112:0/2516290208 conn(0x7f40900723b0 msgr2=0x7f4090065810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:27.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.796+0000 7f4095cc9640 1 -- 172.21.15.112:0/2516290208 shutdown_connections 2024-09-16T00:34:27.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.796+0000 7f4095cc9640 1 -- 172.21.15.112:0/2516290208 wait complete. 2024-09-16T00:34:27.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.877+0000 7f4059ec0640 1 -- 172.21.15.112:0/1547013537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40540fb810 msgr2=0x7f40540fbbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:27.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.877+0000 7f4059ec0640 1 --2- 172.21.15.112:0/1547013537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40540fb810 0x7f40540fbbe0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4044009c50 tx=0x7f4044004d10 comp rx=0 tx=0).stop 2024-09-16T00:34:27.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.877+0000 7f4059ec0640 1 -- 172.21.15.112:0/1547013537 shutdown_connections 2024-09-16T00:34:27.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.877+0000 7f4059ec0640 1 --2- 172.21.15.112:0/1547013537 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40540fcf80 0x7f40540fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.877+0000 7f4059ec0640 1 --2- 172.21.15.112:0/1547013537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40540fb810 0x7f40540fbbe0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.877+0000 7f4059ec0640 1 -- 172.21.15.112:0/1547013537 >> 172.21.15.112:0/1547013537 conn(0x7f40540723b0 msgr2=0x7f40540727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:27.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.877+0000 7f4059ec0640 1 -- 172.21.15.112:0/1547013537 shutdown_connections 2024-09-16T00:34:27.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.877+0000 7f4059ec0640 1 -- 172.21.15.112:0/1547013537 wait complete. 2024-09-16T00:34:27.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.878+0000 7f4059ec0640 1 Processor -- start 2024-09-16T00:34:27.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.878+0000 7f4059ec0640 1 -- start start 2024-09-16T00:34:27.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.878+0000 7f4059ec0640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40540fb810 0x7f405410bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:27.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.878+0000 7f4059ec0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40540fcf80 0x7f405410c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:27.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.878+0000 7f4059ec0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f405410ca20 con 0x7f40540fcf80 2024-09-16T00:34:27.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.878+0000 7f4059ec0640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f405410cb60 con 0x7f40540fb810 2024-09-16T00:34:27.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.878+0000 7f4052ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40540fcf80 0x7f405410c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:27.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.878+0000 7f4052ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40540fcf80 0x7f405410c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45670/0 (socket says 172.21.15.112:45670) 2024-09-16T00:34:27.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.878+0000 7f4052ffd640 1 -- 172.21.15.112:0/1092875338 learned_addr learned my addr 172.21.15.112:0/1092875338 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:34:27.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.878+0000 7f40537fe640 1 --2- 172.21.15.112:0/1092875338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40540fb810 0x7f405410bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:27.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.879+0000 7f4052ffd640 1 -- 172.21.15.112:0/1092875338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40540fb810 msgr2=0x7f405410bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:27.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.879+0000 7f4052ffd640 1 --2- 172.21.15.112:0/1092875338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40540fb810 0x7f405410bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.879+0000 7f4052ffd640 1 -- 172.21.15.112:0/1092875338 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40440098b0 con 0x7f40540fcf80 2024-09-16T00:34:27.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.879+0000 7f4052ffd640 1 --2- 172.21.15.112:0/1092875338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40540fcf80 0x7f405410c4e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f403c00eb70 tx=0x7f403c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:27.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.879+0000 7f404bfff640 1 -- 172.21.15.112:0/1092875338 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f403c007c30 con 0x7f40540fcf80 2024-09-16T00:34:27.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.879+0000 7f404bfff640 1 -- 172.21.15.112:0/1092875338 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f403c00f040 con 0x7f40540fcf80 2024-09-16T00:34:27.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.879+0000 7f4059ec0640 1 -- 172.21.15.112:0/1092875338 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f405410cde0 con 0x7f40540fcf80 2024-09-16T00:34:27.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.879+0000 7f404bfff640 1 -- 172.21.15.112:0/1092875338 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f403c0094e0 con 0x7f40540fcf80 2024-09-16T00:34:27.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.879+0000 7f4059ec0640 1 -- 172.21.15.112:0/1092875338 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4054119940 con 0x7f40540fcf80 2024-09-16T00:34:27.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.880+0000 7f4059ec0640 1 -- 172.21.15.112:0/1092875338 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40540fcbc0 con 0x7f40540fcf80 2024-09-16T00:34:27.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.881+0000 7f404bfff640 1 -- 172.21.15.112:0/1092875338 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f403c009660 con 0x7f40540fcf80 2024-09-16T00:34:27.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.881+0000 7f404bfff640 1 --2- 172.21.15.112:0/1092875338 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f404006e530 0x7f40400709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:27.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.881+0000 7f404bfff640 1 -- 172.21.15.112:0/1092875338 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f403c08df60 con 0x7f40540fcf80 2024-09-16T00:34:27.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.881+0000 7f40537fe640 1 --2- 172.21.15.112:0/1092875338 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f404006e530 0x7f40400709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:27.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.882+0000 7f40537fe640 1 --2- 172.21.15.112:0/1092875338 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f404006e530 0x7f40400709e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4044000f80 tx=0x7f4044004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:27.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:27.885+0000 7f404bfff640 1 -- 172.21.15.112:0/1092875338 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f403c0c31b0 con 0x7f40540fcf80 2024-09-16T00:34:28.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:28.068+0000 7f4059ec0640 1 -- 172.21.15.112:0/1092875338 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f405404b720 con 0x7f404006e530 2024-09-16T00:34:28.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:28.069+0000 7f404bfff640 1 -- 172.21.15.112:0/1092875338 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f405404b720 con 0x7f404006e530 2024-09-16T00:34:28.070 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:34:28.070 INFO:teuthology.orchestra.run.smithi112.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80", 2024-09-16T00:34:28.070 INFO:teuthology.orchestra.run.smithi112.stdout: "in_progress": true, 2024-09-16T00:34:28.070 INFO:teuthology.orchestra.run.smithi112.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T00:34:28.071 INFO:teuthology.orchestra.run.smithi112.stdout: "services_complete": [ 2024-09-16T00:34:28.071 INFO:teuthology.orchestra.run.smithi112.stdout: "mds", 2024-09-16T00:34:28.071 INFO:teuthology.orchestra.run.smithi112.stdout: "osd", 2024-09-16T00:34:28.071 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr", 2024-09-16T00:34:28.071 INFO:teuthology.orchestra.run.smithi112.stdout: "mon", 2024-09-16T00:34:28.071 INFO:teuthology.orchestra.run.smithi112.stdout: "crash" 2024-09-16T00:34:28.072 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-16T00:34:28.072 INFO:teuthology.orchestra.run.smithi112.stdout: "progress": "16/21 daemons upgraded", 2024-09-16T00:34:28.072 INFO:teuthology.orchestra.run.smithi112.stdout: "message": "Currently upgrading mds daemons", 2024-09-16T00:34:28.072 INFO:teuthology.orchestra.run.smithi112.stdout: "is_paused": false 2024-09-16T00:34:28.072 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:34:28.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:28.071+0000 7f4059ec0640 1 -- 172.21.15.112:0/1092875338 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f404006e530 msgr2=0x7f40400709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:28.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:28.071+0000 7f4059ec0640 1 --2- 172.21.15.112:0/1092875338 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f404006e530 0x7f40400709e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4044000f80 tx=0x7f4044004d10 comp rx=0 tx=0).stop 2024-09-16T00:34:28.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:28.071+0000 7f4059ec0640 1 -- 172.21.15.112:0/1092875338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40540fcf80 msgr2=0x7f405410c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:28.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:28.071+0000 7f4059ec0640 1 --2- 172.21.15.112:0/1092875338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40540fcf80 0x7f405410c4e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f403c00eb70 tx=0x7f403c00b040 comp rx=0 tx=0).stop 2024-09-16T00:34:28.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:28.071+0000 7f4059ec0640 1 -- 172.21.15.112:0/1092875338 shutdown_connections 2024-09-16T00:34:28.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:28.071+0000 7f4059ec0640 1 --2- 172.21.15.112:0/1092875338 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f404006e530 0x7f40400709e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:28.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:28.071+0000 7f4059ec0640 1 --2- 172.21.15.112:0/1092875338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40540fcf80 0x7f405410c4e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:28.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:28.071+0000 7f4059ec0640 1 --2- 172.21.15.112:0/1092875338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40540fb810 0x7f405410bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:28.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:28.071+0000 7f4059ec0640 1 -- 172.21.15.112:0/1092875338 >> 172.21.15.112:0/1092875338 conn(0x7f40540723b0 msgr2=0x7f40540fa480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:28.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:28.072+0000 7f4059ec0640 1 -- 172.21.15.112:0/1092875338 shutdown_connections 2024-09-16T00:34:28.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:28.072+0000 7f4059ec0640 1 -- 172.21.15.112:0/1092875338 wait complete. 2024-09-16T00:34:28.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:28.149+0000 7f5a8f31c640 1 -- 172.21.15.112:0/3383317603 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a88102550 msgr2=0x7f5a88102920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:28.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:28.149+0000 7f5a8f31c640 1 --2- 172.21.15.112:0/3383317603 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a88102550 0x7f5a88102920 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5a840048d0 tx=0x7f5a84009da0 comp rx=0 tx=0).stop 2024-09-16T00:34:28.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:28.149+0000 7f5a8f31c640 1 -- 172.21.15.112:0/3383317603 shutdown_connections 2024-09-16T00:34:28.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:28.149+0000 7f5a8f31c640 1 --2- 172.21.15.112:0/3383317603 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a88102ef0 0x7f5a880fc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:28.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:28.149+0000 7f5a8f31c640 1 --2- 172.21.15.112:0/3383317603 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a88102550 0x7f5a88102920 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:28.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:28.149+0000 7f5a8f31c640 1 -- 172.21.15.112:0/3383317603 >> 172.21.15.112:0/3383317603 conn(0x7f5a880f80f0 msgr2=0x7f5a880fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:28.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:28.150+0000 7f5a8f31c640 1 -- 172.21.15.112:0/3383317603 shutdown_connections 2024-09-16T00:34:28.152 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:27 smithi112 ceph-mon[97750]: from='client.34440 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:34:28.152 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:27 smithi112 ceph-mon[97750]: from='client.34444 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:34:28.152 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:27 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/4140297602' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:34:28.152 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:27 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/2516290208' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:34:28.152 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:27 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:34:28.152 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:27 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:34:28.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:28.150+0000 7f5a8f31c640 1 -- 172.21.15.112:0/3383317603 wait complete. 2024-09-16T00:34:28.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:28.150+0000 7f5a8f31c640 1 Processor -- start 2024-09-16T00:34:28.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:28.150+0000 7f5a8f31c640 1 -- start start 2024-09-16T00:34:28.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:28.151+0000 7f5a8f31c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a88102550 0x7f5a88113570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:28.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:28.151+0000 7f5a8f31c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a88102ef0 0x7f5a88113ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:28.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:28.151+0000 7f5a8f31c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a88114100 con 0x7f5a88102550 2024-09-16T00:34:28.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:28.151+0000 7f5a8f31c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a8810d510 con 0x7f5a88102ef0 2024-09-16T00:34:28.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:28.151+0000 7f5a8d0b8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a88102550 0x7f5a88113570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:28.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:28.151+0000 7f5a8d0b8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a88102550 0x7f5a88113570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45692/0 (socket says 172.21.15.112:45692) 2024-09-16T00:34:28.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:28.151+0000 7f5a8d0b8640 1 -- 172.21.15.112:0/3537677279 learned_addr learned my addr 172.21.15.112:0/3537677279 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:34:28.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:28.151+0000 7f5a8c8b7640 1 --2- 172.21.15.112:0/3537677279 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a88102ef0 0x7f5a88113ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:28.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:28.151+0000 7f5a8d0b8640 1 -- 172.21.15.112:0/3537677279 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a88102ef0 msgr2=0x7f5a88113ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:28.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:28.151+0000 7f5a8d0b8640 1 --2- 172.21.15.112:0/3537677279 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a88102ef0 0x7f5a88113ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:28.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:28.151+0000 7f5a8d0b8640 1 -- 172.21.15.112:0/3537677279 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a84005040 con 0x7f5a88102550 2024-09-16T00:34:28.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:28.151+0000 7f5a8d0b8640 1 --2- 172.21.15.112:0/3537677279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a88102550 0x7f5a88113570 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5a84000c00 tx=0x7f5a84009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:28.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:28.151+0000 7f5a7dffb640 1 -- 172.21.15.112:0/3537677279 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a84038070 con 0x7f5a88102550 2024-09-16T00:34:28.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:28.151+0000 7f5a7dffb640 1 -- 172.21.15.112:0/3537677279 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f5a8402ade0 con 0x7f5a88102550 2024-09-16T00:34:28.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:28.151+0000 7f5a8f31c640 1 -- 172.21.15.112:0/3537677279 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a8810d790 con 0x7f5a88102550 2024-09-16T00:34:28.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:28.152+0000 7f5a7dffb640 1 -- 172.21.15.112:0/3537677279 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a8402c7e0 con 0x7f5a88102550 2024-09-16T00:34:28.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:28.152+0000 7f5a8f31c640 1 -- 172.21.15.112:0/3537677279 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a8810dcd0 con 0x7f5a88102550 2024-09-16T00:34:28.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:28.153+0000 7f5a8f31c640 1 -- 172.21.15.112:0/3537677279 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a880fdcd0 con 0x7f5a88102550 2024-09-16T00:34:28.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:28.153+0000 7f5a7dffb640 1 -- 172.21.15.112:0/3537677279 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f5a84030020 con 0x7f5a88102550 2024-09-16T00:34:28.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:28.154+0000 7f5a7dffb640 1 --2- 172.21.15.112:0/3537677279 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5a7406e530 0x7f5a740709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:28.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:28.154+0000 7f5a7dffb640 1 -- 172.21.15.112:0/3537677279 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f5a840aded0 con 0x7f5a88102550 2024-09-16T00:34:28.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:28.154+0000 7f5a8c8b7640 1 --2- 172.21.15.112:0/3537677279 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5a7406e530 0x7f5a740709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:28.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:28.154+0000 7f5a8c8b7640 1 --2- 172.21.15.112:0/3537677279 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5a7406e530 0x7f5a740709e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5a8810e7f0 tx=0x7f5a7800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:28.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:28.158+0000 7f5a7dffb640 1 -- 172.21.15.112:0/3537677279 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f5a84076f70 con 0x7f5a88102550 2024-09-16T00:34:28.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:27 smithi169 ceph-mon[77626]: from='client.34440 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:34:28.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:27 smithi169 ceph-mon[77626]: from='client.34444 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:34:28.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:27 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/4140297602' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:34:28.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:27 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/2516290208' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:34:28.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:27 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:34:28.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:27 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:34:28.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:28.394+0000 7f5a8f31c640 1 -- 172.21.15.112:0/3537677279 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f5a8804b720 con 0x7f5a88102550 2024-09-16T00:34:28.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:28.394+0000 7f5a7dffb640 1 -- 172.21.15.112:0/3537677279 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f5a8407a630 con 0x7f5a88102550 2024-09-16T00:34:28.395 INFO:teuthology.orchestra.run.smithi112.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-16T00:34:28.395 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-16T00:34:28.395 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs is degraded 2024-09-16T00:34:28.395 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T00:34:28.395 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T00:34:28.395 INFO:teuthology.orchestra.run.smithi112.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-16T00:34:28.395 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-16T00:34:28.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:28.396+0000 7f5a8f31c640 1 -- 172.21.15.112:0/3537677279 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5a7406e530 msgr2=0x7f5a740709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:28.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:28.396+0000 7f5a8f31c640 1 --2- 172.21.15.112:0/3537677279 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5a7406e530 0x7f5a740709e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5a8810e7f0 tx=0x7f5a7800b040 comp rx=0 tx=0).stop 2024-09-16T00:34:28.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:28.396+0000 7f5a8f31c640 1 -- 172.21.15.112:0/3537677279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a88102550 msgr2=0x7f5a88113570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:28.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:28.396+0000 7f5a8f31c640 1 --2- 172.21.15.112:0/3537677279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a88102550 0x7f5a88113570 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5a84000c00 tx=0x7f5a84009da0 comp rx=0 tx=0).stop 2024-09-16T00:34:28.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:28.396+0000 7f5a8f31c640 1 -- 172.21.15.112:0/3537677279 shutdown_connections 2024-09-16T00:34:28.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:28.396+0000 7f5a8f31c640 1 --2- 172.21.15.112:0/3537677279 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5a7406e530 0x7f5a740709e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:28.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:28.396+0000 7f5a8f31c640 1 --2- 172.21.15.112:0/3537677279 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a88102ef0 0x7f5a88113ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:28.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:28.396+0000 7f5a8f31c640 1 --2- 172.21.15.112:0/3537677279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a88102550 0x7f5a88113570 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:28.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:28.396+0000 7f5a8f31c640 1 -- 172.21.15.112:0/3537677279 >> 172.21.15.112:0/3537677279 conn(0x7f5a880f80f0 msgr2=0x7f5a880f90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:28.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:28.396+0000 7f5a8f31c640 1 -- 172.21.15.112:0/3537677279 shutdown_connections 2024-09-16T00:34:28.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:28.396+0000 7f5a8f31c640 1 -- 172.21.15.112:0/3537677279 wait complete. 2024-09-16T00:34:29.141 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:28 smithi169 ceph-mon[77626]: from='client.34456 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:34:29.142 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:28 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/3537677279' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:34:29.142 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:28 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:34:29.142 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:28 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:34:29.142 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:28 smithi169 ceph-mon[77626]: pgmap v202: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:34:29.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:28 smithi112 ceph-mon[97750]: from='client.34456 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:34:29.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:28 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/3537677279' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:34:29.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:28 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:34:29.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:28 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:34:29.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:28 smithi112 ceph-mon[97750]: pgmap v202: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:34:32.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:31 smithi169 ceph-mon[77626]: pgmap v203: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:34:32.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:31 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:34:32.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:31 smithi112 ceph-mon[97750]: pgmap v203: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:34:32.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:31 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:34:33.892 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:33 smithi112 ceph-mon[97750]: pgmap v204: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:34:34.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:33 smithi169 ceph-mon[77626]: pgmap v204: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:34:36.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:35 smithi169 ceph-mon[77626]: pgmap v205: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:34:36.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:35 smithi112 ceph-mon[97750]: pgmap v205: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:34:38.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:37 smithi169 ceph-mon[77626]: pgmap v206: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:34:38.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:37 smithi112 ceph-mon[97750]: pgmap v206: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:34:40.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:39 smithi169 ceph-mon[77626]: pgmap v207: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:34:40.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:39 smithi112 ceph-mon[97750]: pgmap v207: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:34:42.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:41 smithi169 ceph-mon[77626]: pgmap v208: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:34:42.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:41 smithi112 ceph-mon[97750]: pgmap v208: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:34:44.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:43 smithi169 ceph-mon[77626]: pgmap v209: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:34:44.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:43 smithi112 ceph-mon[97750]: pgmap v209: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:34:46.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:45 smithi169 ceph-mon[77626]: pgmap v210: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:34:46.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:45 smithi112 ceph-mon[97750]: pgmap v210: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:34:47.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:46 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:34:47.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:46 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:34:48.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:47 smithi169 ceph-mon[77626]: pgmap v211: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:34:48.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:47 smithi112 ceph-mon[97750]: pgmap v211: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:34:50.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:49 smithi169 ceph-mon[77626]: pgmap v212: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:34:50.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:49 smithi112 ceph-mon[97750]: pgmap v212: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:34:52.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:51 smithi169 ceph-mon[77626]: pgmap v213: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:34:52.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:51 smithi112 ceph-mon[97750]: pgmap v213: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:34:54.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:53 smithi169 ceph-mon[77626]: pgmap v214: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:34:54.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:53 smithi112 ceph-mon[97750]: pgmap v214: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:34:56.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:55 smithi169 ceph-mon[77626]: pgmap v215: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:34:56.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:55 smithi112 ceph-mon[97750]: pgmap v215: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:34:58.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:57 smithi169 ceph-mon[77626]: pgmap v216: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:34:58.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:57 smithi112 ceph-mon[97750]: pgmap v216: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:34:58.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.487+0000 7f7e843bd640 1 -- 172.21.15.112:0/1366138567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e7c064fc0 msgr2=0x7f7e7c065430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:58.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.487+0000 7f7e843bd640 1 --2- 172.21.15.112:0/1366138567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e7c064fc0 0x7f7e7c065430 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7e700048d0 tx=0x7f7e70009da0 comp rx=0 tx=0).stop 2024-09-16T00:34:58.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.487+0000 7f7e843bd640 1 -- 172.21.15.112:0/1366138567 shutdown_connections 2024-09-16T00:34:58.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.487+0000 7f7e843bd640 1 --2- 172.21.15.112:0/1366138567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e7c064fc0 0x7f7e7c065430 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:58.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.487+0000 7f7e843bd640 1 --2- 172.21.15.112:0/1366138567 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e7c0fe460 0x7f7e7c0fe830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:58.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.487+0000 7f7e843bd640 1 -- 172.21.15.112:0/1366138567 >> 172.21.15.112:0/1366138567 conn(0x7f7e7c0723b0 msgr2=0x7f7e7c0727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:58.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.487+0000 7f7e843bd640 1 -- 172.21.15.112:0/1366138567 shutdown_connections 2024-09-16T00:34:58.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.487+0000 7f7e843bd640 1 -- 172.21.15.112:0/1366138567 wait complete. 2024-09-16T00:34:58.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.488+0000 7f7e843bd640 1 Processor -- start 2024-09-16T00:34:58.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.488+0000 7f7e843bd640 1 -- start start 2024-09-16T00:34:58.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.488+0000 7f7e843bd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e7c064fc0 0x7f7e7c0fb560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:58.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.488+0000 7f7e843bd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e7c0fe460 0x7f7e7c0fbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:58.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.488+0000 7f7e843bd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7c0fbfe0 con 0x7f7e7c064fc0 2024-09-16T00:34:58.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.488+0000 7f7e843bd640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7c0fc150 con 0x7f7e7c0fe460 2024-09-16T00:34:58.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.488+0000 7f7e82159640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e7c064fc0 0x7f7e7c0fb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:58.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.488+0000 7f7e82159640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e7c064fc0 0x7f7e7c0fb560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35656/0 (socket says 172.21.15.112:35656) 2024-09-16T00:34:58.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.488+0000 7f7e82159640 1 -- 172.21.15.112:0/1748653036 learned_addr learned my addr 172.21.15.112:0/1748653036 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:34:58.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.488+0000 7f7e81958640 1 --2- 172.21.15.112:0/1748653036 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e7c0fe460 0x7f7e7c0fbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:58.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.489+0000 7f7e82159640 1 -- 172.21.15.112:0/1748653036 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e7c0fe460 msgr2=0x7f7e7c0fbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:58.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.489+0000 7f7e82159640 1 --2- 172.21.15.112:0/1748653036 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e7c0fe460 0x7f7e7c0fbaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:58.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.489+0000 7f7e82159640 1 -- 172.21.15.112:0/1748653036 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e70005040 con 0x7f7e7c064fc0 2024-09-16T00:34:58.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.489+0000 7f7e81958640 1 --2- 172.21.15.112:0/1748653036 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e7c0fe460 0x7f7e7c0fbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:34:58.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.489+0000 7f7e82159640 1 --2- 172.21.15.112:0/1748653036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e7c064fc0 0x7f7e7c0fb560 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c00db00 tx=0x7f7e6c004e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:58.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.489+0000 7f7e7affd640 1 -- 172.21.15.112:0/1748653036 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e6c017070 con 0x7f7e7c064fc0 2024-09-16T00:34:58.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.489+0000 7f7e7affd640 1 -- 172.21.15.112:0/1748653036 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f7e6c00f040 con 0x7f7e7c064fc0 2024-09-16T00:34:58.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.489+0000 7f7e843bd640 1 -- 172.21.15.112:0/1748653036 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e7c119540 con 0x7f7e7c064fc0 2024-09-16T00:34:58.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.489+0000 7f7e7affd640 1 -- 172.21.15.112:0/1748653036 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e6c003940 con 0x7f7e7c064fc0 2024-09-16T00:34:58.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.489+0000 7f7e843bd640 1 -- 172.21.15.112:0/1748653036 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e7c119940 con 0x7f7e7c064fc0 2024-09-16T00:34:58.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.490+0000 7f7e843bd640 1 -- 172.21.15.112:0/1748653036 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e7c0fdcf0 con 0x7f7e7c064fc0 2024-09-16T00:34:58.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.491+0000 7f7e7affd640 1 -- 172.21.15.112:0/1748653036 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f7e6c003b70 con 0x7f7e7c064fc0 2024-09-16T00:34:58.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.491+0000 7f7e7affd640 1 --2- 172.21.15.112:0/1748653036 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f7e6406e530 0x7f7e640709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:58.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.491+0000 7f7e7affd640 1 -- 172.21.15.112:0/1748653036 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f7e6c08e420 con 0x7f7e7c064fc0 2024-09-16T00:34:58.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.491+0000 7f7e81958640 1 --2- 172.21.15.112:0/1748653036 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f7e6406e530 0x7f7e640709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:58.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.492+0000 7f7e81958640 1 --2- 172.21.15.112:0/1748653036 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f7e6406e530 0x7f7e640709e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c0fce50 tx=0x7f7e70035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:58.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.496+0000 7f7e7affd640 1 -- 172.21.15.112:0/1748653036 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f7e6c057390 con 0x7f7e7c064fc0 2024-09-16T00:34:58.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.680+0000 7f7e843bd640 1 -- 172.21.15.112:0/1748653036 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7e7c0fcc00 con 0x7f7e6406e530 2024-09-16T00:34:58.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.681+0000 7f7e7affd640 1 -- 172.21.15.112:0/1748653036 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f7e7c0fcc00 con 0x7f7e6406e530 2024-09-16T00:34:58.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.684+0000 7f7e843bd640 1 -- 172.21.15.112:0/1748653036 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f7e6406e530 msgr2=0x7f7e640709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:58.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.684+0000 7f7e843bd640 1 --2- 172.21.15.112:0/1748653036 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f7e6406e530 0x7f7e640709e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c0fce50 tx=0x7f7e70035040 comp rx=0 tx=0).stop 2024-09-16T00:34:58.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.684+0000 7f7e843bd640 1 -- 172.21.15.112:0/1748653036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e7c064fc0 msgr2=0x7f7e7c0fb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:58.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.684+0000 7f7e843bd640 1 --2- 172.21.15.112:0/1748653036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e7c064fc0 0x7f7e7c0fb560 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c00db00 tx=0x7f7e6c004e60 comp rx=0 tx=0).stop 2024-09-16T00:34:58.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.684+0000 7f7e843bd640 1 -- 172.21.15.112:0/1748653036 shutdown_connections 2024-09-16T00:34:58.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.684+0000 7f7e843bd640 1 --2- 172.21.15.112:0/1748653036 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f7e6406e530 0x7f7e640709e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:58.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.684+0000 7f7e843bd640 1 --2- 172.21.15.112:0/1748653036 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e7c0fe460 0x7f7e7c0fbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:58.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.684+0000 7f7e843bd640 1 --2- 172.21.15.112:0/1748653036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e7c064fc0 0x7f7e7c0fb560 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:58.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.684+0000 7f7e843bd640 1 -- 172.21.15.112:0/1748653036 >> 172.21.15.112:0/1748653036 conn(0x7f7e7c0723b0 msgr2=0x7f7e7c1053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:58.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.684+0000 7f7e843bd640 1 -- 172.21.15.112:0/1748653036 shutdown_connections 2024-09-16T00:34:58.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.684+0000 7f7e843bd640 1 -- 172.21.15.112:0/1748653036 wait complete. 2024-09-16T00:34:58.693 INFO:teuthology.orchestra.run.smithi112.stdout:true 2024-09-16T00:34:58.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.765+0000 7f6df89de640 1 -- 172.21.15.112:0/979601150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6df4102550 msgr2=0x7f6df4102920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:58.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.765+0000 7f6df89de640 1 --2- 172.21.15.112:0/979601150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6df4102550 0x7f6df4102920 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6de40048d0 tx=0x7f6de4009da0 comp rx=0 tx=0).stop 2024-09-16T00:34:58.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.765+0000 7f6df89de640 1 -- 172.21.15.112:0/979601150 shutdown_connections 2024-09-16T00:34:58.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.765+0000 7f6df89de640 1 --2- 172.21.15.112:0/979601150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6df4102ef0 0x7f6df40fc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:58.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.765+0000 7f6df89de640 1 --2- 172.21.15.112:0/979601150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6df4102550 0x7f6df4102920 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:58.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.765+0000 7f6df89de640 1 -- 172.21.15.112:0/979601150 >> 172.21.15.112:0/979601150 conn(0x7f6df40f80f0 msgr2=0x7f6df40fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:58.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.765+0000 7f6df89de640 1 -- 172.21.15.112:0/979601150 shutdown_connections 2024-09-16T00:34:58.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.765+0000 7f6df89de640 1 -- 172.21.15.112:0/979601150 wait complete. 2024-09-16T00:34:58.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.766+0000 7f6df89de640 1 Processor -- start 2024-09-16T00:34:58.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.766+0000 7f6df89de640 1 -- start start 2024-09-16T00:34:58.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.766+0000 7f6df89de640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6df4102550 0x7f6df4113540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:58.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.766+0000 7f6df89de640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6df4102ef0 0x7f6df4113a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:58.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.766+0000 7f6df89de640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df41140d0 con 0x7f6df4102550 2024-09-16T00:34:58.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.766+0000 7f6df89de640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df410d4e0 con 0x7f6df4102ef0 2024-09-16T00:34:58.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.766+0000 7f6df259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6df4102550 0x7f6df4113540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:58.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.766+0000 7f6df259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6df4102550 0x7f6df4113540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35672/0 (socket says 172.21.15.112:35672) 2024-09-16T00:34:58.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.766+0000 7f6df259c640 1 -- 172.21.15.112:0/3275578094 learned_addr learned my addr 172.21.15.112:0/3275578094 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:34:58.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.766+0000 7f6df259c640 1 -- 172.21.15.112:0/3275578094 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6df4102ef0 msgr2=0x7f6df4113a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:58.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.766+0000 7f6df259c640 1 --2- 172.21.15.112:0/3275578094 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6df4102ef0 0x7f6df4113a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:58.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.766+0000 7f6df259c640 1 -- 172.21.15.112:0/3275578094 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6de4005040 con 0x7f6df4102550 2024-09-16T00:34:58.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.767+0000 7f6df259c640 1 --2- 172.21.15.112:0/3275578094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6df4102550 0x7f6df4113540 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6de4003080 tx=0x7f6de4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:58.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.767+0000 7f6deaffd640 1 -- 172.21.15.112:0/3275578094 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6de4038070 con 0x7f6df4102550 2024-09-16T00:34:58.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.767+0000 7f6deaffd640 1 -- 172.21.15.112:0/3275578094 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f6de40097e0 con 0x7f6df4102550 2024-09-16T00:34:58.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.767+0000 7f6deaffd640 1 -- 172.21.15.112:0/3275578094 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6de403d420 con 0x7f6df4102550 2024-09-16T00:34:58.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.767+0000 7f6df89de640 1 -- 172.21.15.112:0/3275578094 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6df410d760 con 0x7f6df4102550 2024-09-16T00:34:58.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.767+0000 7f6df89de640 1 -- 172.21.15.112:0/3275578094 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6df410dca0 con 0x7f6df4102550 2024-09-16T00:34:58.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.768+0000 7f6df89de640 1 -- 172.21.15.112:0/3275578094 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6df40fdcd0 con 0x7f6df4102550 2024-09-16T00:34:58.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.769+0000 7f6deaffd640 1 -- 172.21.15.112:0/3275578094 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f6de4030020 con 0x7f6df4102550 2024-09-16T00:34:58.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.770+0000 7f6deaffd640 1 --2- 172.21.15.112:0/3275578094 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f6de006e530 0x7f6de00709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:58.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.770+0000 7f6deaffd640 1 -- 172.21.15.112:0/3275578094 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f6de40ae8a0 con 0x7f6df4102550 2024-09-16T00:34:58.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.770+0000 7f6debfff640 1 --2- 172.21.15.112:0/3275578094 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f6de006e530 0x7f6de00709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:58.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.771+0000 7f6debfff640 1 --2- 172.21.15.112:0/3275578094 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f6de006e530 0x7f6de00709e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6df410e7c0 tx=0x7f6ddc007cb0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:58.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.774+0000 7f6deaffd640 1 -- 172.21.15.112:0/3275578094 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f6de4077810 con 0x7f6df4102550 2024-09-16T00:34:58.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.959+0000 7f6df89de640 1 -- 172.21.15.112:0/3275578094 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6df404b720 con 0x7f6de006e530 2024-09-16T00:34:58.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.961+0000 7f6deaffd640 1 -- 172.21.15.112:0/3275578094 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f6df404b720 con 0x7f6de006e530 2024-09-16T00:34:58.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.963+0000 7f6df89de640 1 -- 172.21.15.112:0/3275578094 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f6de006e530 msgr2=0x7f6de00709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:58.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.963+0000 7f6df89de640 1 --2- 172.21.15.112:0/3275578094 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f6de006e530 0x7f6de00709e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6df410e7c0 tx=0x7f6ddc007cb0 comp rx=0 tx=0).stop 2024-09-16T00:34:58.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.963+0000 7f6df89de640 1 -- 172.21.15.112:0/3275578094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6df4102550 msgr2=0x7f6df4113540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:58.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.963+0000 7f6df89de640 1 --2- 172.21.15.112:0/3275578094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6df4102550 0x7f6df4113540 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6de4003080 tx=0x7f6de4009da0 comp rx=0 tx=0).stop 2024-09-16T00:34:58.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.963+0000 7f6df89de640 1 -- 172.21.15.112:0/3275578094 shutdown_connections 2024-09-16T00:34:58.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.963+0000 7f6df89de640 1 --2- 172.21.15.112:0/3275578094 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f6de006e530 0x7f6de00709e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:58.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.963+0000 7f6df89de640 1 --2- 172.21.15.112:0/3275578094 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6df4102ef0 0x7f6df4113a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:58.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.963+0000 7f6df89de640 1 --2- 172.21.15.112:0/3275578094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6df4102550 0x7f6df4113540 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:58.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.963+0000 7f6df89de640 1 -- 172.21.15.112:0/3275578094 >> 172.21.15.112:0/3275578094 conn(0x7f6df40f80f0 msgr2=0x7f6df40f9160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:58.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.963+0000 7f6df89de640 1 -- 172.21.15.112:0/3275578094 shutdown_connections 2024-09-16T00:34:58.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:58.963+0000 7f6df89de640 1 -- 172.21.15.112:0/3275578094 wait complete. 2024-09-16T00:34:59.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.040+0000 7fb207e13640 1 -- 172.21.15.112:0/2968058619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2000fcf80 msgr2=0x7fb2000fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:59.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.040+0000 7fb207e13640 1 --2- 172.21.15.112:0/2968058619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2000fcf80 0x7fb2000fd3d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb1f00048d0 tx=0x7fb1f0009da0 comp rx=0 tx=0).stop 2024-09-16T00:34:59.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.040+0000 7fb207e13640 1 -- 172.21.15.112:0/2968058619 shutdown_connections 2024-09-16T00:34:59.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.040+0000 7fb207e13640 1 --2- 172.21.15.112:0/2968058619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2000fcf80 0x7fb2000fd3d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:59.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.040+0000 7fb207e13640 1 --2- 172.21.15.112:0/2968058619 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2000fb810 0x7fb2000fbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:59.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.040+0000 7fb207e13640 1 -- 172.21.15.112:0/2968058619 >> 172.21.15.112:0/2968058619 conn(0x7fb2000723b0 msgr2=0x7fb2000727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:59.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.040+0000 7fb207e13640 1 -- 172.21.15.112:0/2968058619 shutdown_connections 2024-09-16T00:34:59.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.040+0000 7fb207e13640 1 -- 172.21.15.112:0/2968058619 wait complete. 2024-09-16T00:34:59.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.041+0000 7fb207e13640 1 Processor -- start 2024-09-16T00:34:59.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.041+0000 7fb207e13640 1 -- start start 2024-09-16T00:34:59.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.041+0000 7fb207e13640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2000fb810 0x7fb20010e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:59.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.041+0000 7fb207e13640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2000fcf80 0x7fb20010e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:59.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.041+0000 7fb207e13640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb20010eda0 con 0x7fb2000fb810 2024-09-16T00:34:59.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.041+0000 7fb207e13640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb20010ef10 con 0x7fb2000fcf80 2024-09-16T00:34:59.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.041+0000 7fb205baf640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2000fb810 0x7fb20010e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:59.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.041+0000 7fb205baf640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2000fb810 0x7fb20010e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35680/0 (socket says 172.21.15.112:35680) 2024-09-16T00:34:59.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.041+0000 7fb205baf640 1 -- 172.21.15.112:0/3705278352 learned_addr learned my addr 172.21.15.112:0/3705278352 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:34:59.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.042+0000 7fb2053ae640 1 --2- 172.21.15.112:0/3705278352 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2000fcf80 0x7fb20010e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:59.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.042+0000 7fb205baf640 1 -- 172.21.15.112:0/3705278352 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2000fcf80 msgr2=0x7fb20010e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:59.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.042+0000 7fb205baf640 1 --2- 172.21.15.112:0/3705278352 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2000fcf80 0x7fb20010e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:59.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.042+0000 7fb205baf640 1 -- 172.21.15.112:0/3705278352 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1f0005040 con 0x7fb2000fb810 2024-09-16T00:34:59.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.042+0000 7fb205baf640 1 --2- 172.21.15.112:0/3705278352 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2000fb810 0x7fb20010e1d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc00b560 tx=0x7fb1fc0090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:59.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.042+0000 7fb1f67fc640 1 -- 172.21.15.112:0/3705278352 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb1fc00f6f0 con 0x7fb2000fb810 2024-09-16T00:34:59.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.042+0000 7fb1f67fc640 1 -- 172.21.15.112:0/3705278352 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fb1fc00fcf0 con 0x7fb2000fb810 2024-09-16T00:34:59.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.042+0000 7fb207e13640 1 -- 172.21.15.112:0/3705278352 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2001172a0 con 0x7fb2000fb810 2024-09-16T00:34:59.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.042+0000 7fb1f67fc640 1 -- 172.21.15.112:0/3705278352 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb1fc011e70 con 0x7fb2000fb810 2024-09-16T00:34:59.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.043+0000 7fb207e13640 1 -- 172.21.15.112:0/3705278352 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2001177c0 con 0x7fb2000fb810 2024-09-16T00:34:59.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.043+0000 7fb207e13640 1 -- 172.21.15.112:0/3705278352 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2000fcbc0 con 0x7fb2000fb810 2024-09-16T00:34:59.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.044+0000 7fb1f67fc640 1 -- 172.21.15.112:0/3705278352 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fb1fc011620 con 0x7fb2000fb810 2024-09-16T00:34:59.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.044+0000 7fb1f67fc640 1 --2- 172.21.15.112:0/3705278352 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fb1ec06e530 0x7fb1ec0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:59.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.044+0000 7fb1f67fc640 1 -- 172.21.15.112:0/3705278352 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7fb1fc08dff0 con 0x7fb2000fb810 2024-09-16T00:34:59.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.044+0000 7fb2053ae640 1 --2- 172.21.15.112:0/3705278352 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fb1ec06e530 0x7fb1ec0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:59.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.045+0000 7fb2053ae640 1 --2- 172.21.15.112:0/3705278352 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fb1ec06e530 0x7fb1ec0709e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb1f00048a0 tx=0x7fb1f0035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:59.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.048+0000 7fb1f67fc640 1 -- 172.21.15.112:0/3705278352 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fb1fc056fe0 con 0x7fb2000fb810 2024-09-16T00:34:59.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.221+0000 7fb207e13640 1 -- 172.21.15.112:0/3705278352 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb200065750 con 0x7fb1ec06e530 2024-09-16T00:34:59.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.230+0000 7fb1f67fc640 1 -- 172.21.15.112:0/3705278352 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7fb200065750 con 0x7fb1ec06e530 2024-09-16T00:34:59.230 INFO:teuthology.orchestra.run.smithi112.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T00:34:59.230 INFO:teuthology.orchestra.run.smithi112.stdout:alertmanager.smithi112 smithi112 *:9093,9094 running (7m) 60s ago 16m 24.7M - 0.25.0 c8568f914cd2 d85ea0562022 2024-09-16T00:34:59.230 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi112 smithi112 running (4m) 60s ago 17m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c e4fe58f9702a 2024-09-16T00:34:59.231 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi169 smithi169 running (4m) 31s ago 16m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c 3ba2c81d527f 2024-09-16T00:34:59.231 INFO:teuthology.orchestra.run.smithi112.stdout:grafana.smithi112 smithi112 *:3000 running (6m) 60s ago 16m 82.1M - 9.4.12 2bacad6d85d8 51f9d2013da2 2024-09-16T00:34:59.231 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.hxfwkh smithi112 running (64s) 60s ago 11m 14.2M - 19.1.1-326-g8921c8ef f1455bd0219c 1723069902d6 2024-09-16T00:34:59.231 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.ithwzj smithi112 running (81s) 60s ago 11m 12.2M - 19.1.1-326-g8921c8ef f1455bd0219c e7aad5b81db5 2024-09-16T00:34:59.231 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.gawwgw smithi169 running (49s) 31s ago 11m 15.7M - 19.1.1-326-g8921c8ef f1455bd0219c 469be459ee12 2024-09-16T00:34:59.231 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.lgeqrj smithi169 running (33s) 31s ago 11m 13.3M - 19.1.1-326-g8921c8ef f1455bd0219c 31f0fbd2395d 2024-09-16T00:34:59.231 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi112.xrkkbc smithi112 *:8443,9283,8765 running (7m) 60s ago 18m 504M - 19.1.1-326-g8921c8ef f1455bd0219c a981f7fd2810 2024-09-16T00:34:59.231 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi169.mtrobk smithi169 *:8443,9283,8765 running (7m) 31s ago 16m 442M - 19.1.1-326-g8921c8ef f1455bd0219c 4096e5369c2d 2024-09-16T00:34:59.231 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi112 smithi112 running (5m) 60s ago 18m 49.8M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 5aaec033c042 2024-09-16T00:34:59.231 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi169 smithi169 running (5m) 31s ago 15m 38.9M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 7bc609155f3f 2024-09-16T00:34:59.231 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi112 smithi112 *:9100 running (8m) 60s ago 17m 15.5M - 1.5.0 0da6a335fe13 6404d1d601e2 2024-09-16T00:34:59.231 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi169 smithi169 *:9100 running (7m) 31s ago 16m 15.4M - 1.5.0 0da6a335fe13 5505baf702a6 2024-09-16T00:34:59.231 INFO:teuthology.orchestra.run.smithi112.stdout:osd.0 smithi112 running (4m) 60s ago 15m 105M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 53efc9397017 2024-09-16T00:34:59.232 INFO:teuthology.orchestra.run.smithi112.stdout:osd.1 smithi112 running (3m) 60s ago 14m 98.9M 1557M 19.1.1-326-g8921c8ef f1455bd0219c fe6f1a4a2cf4 2024-09-16T00:34:59.232 INFO:teuthology.orchestra.run.smithi112.stdout:osd.2 smithi112 running (3m) 60s ago 14m 102M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 5f7863712057 2024-09-16T00:34:59.232 INFO:teuthology.orchestra.run.smithi112.stdout:osd.3 smithi169 running (2m) 31s ago 13m 92.1M 2581M 19.1.1-326-g8921c8ef f1455bd0219c ee07b5577810 2024-09-16T00:34:59.232 INFO:teuthology.orchestra.run.smithi112.stdout:osd.4 smithi169 running (2m) 31s ago 13m 103M 2581M 19.1.1-326-g8921c8ef f1455bd0219c 769f026a54eb 2024-09-16T00:34:59.232 INFO:teuthology.orchestra.run.smithi112.stdout:osd.5 smithi169 running (99s) 31s ago 13m 125M 2581M 19.1.1-326-g8921c8ef f1455bd0219c a6007c13cf78 2024-09-16T00:34:59.232 INFO:teuthology.orchestra.run.smithi112.stdout:prometheus.smithi112 smithi112 *:9095 running (7m) 60s ago 16m 64.1M - 2.43.0 a07b618ecd1d 7a1d42e60d2b 2024-09-16T00:34:59.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.232+0000 7fb207e13640 1 -- 172.21.15.112:0/3705278352 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fb1ec06e530 msgr2=0x7fb1ec0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:59.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.232+0000 7fb207e13640 1 --2- 172.21.15.112:0/3705278352 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fb1ec06e530 0x7fb1ec0709e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb1f00048a0 tx=0x7fb1f0035040 comp rx=0 tx=0).stop 2024-09-16T00:34:59.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.232+0000 7fb207e13640 1 -- 172.21.15.112:0/3705278352 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2000fb810 msgr2=0x7fb20010e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:59.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.232+0000 7fb207e13640 1 --2- 172.21.15.112:0/3705278352 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2000fb810 0x7fb20010e1d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc00b560 tx=0x7fb1fc0090f0 comp rx=0 tx=0).stop 2024-09-16T00:34:59.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.232+0000 7fb207e13640 1 -- 172.21.15.112:0/3705278352 shutdown_connections 2024-09-16T00:34:59.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.232+0000 7fb207e13640 1 --2- 172.21.15.112:0/3705278352 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fb1ec06e530 0x7fb1ec0709e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:59.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.232+0000 7fb207e13640 1 --2- 172.21.15.112:0/3705278352 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2000fcf80 0x7fb20010e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:59.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.232+0000 7fb207e13640 1 --2- 172.21.15.112:0/3705278352 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2000fb810 0x7fb20010e1d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:59.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.232+0000 7fb207e13640 1 -- 172.21.15.112:0/3705278352 >> 172.21.15.112:0/3705278352 conn(0x7fb2000723b0 msgr2=0x7fb2000fa480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:59.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.232+0000 7fb207e13640 1 -- 172.21.15.112:0/3705278352 shutdown_connections 2024-09-16T00:34:59.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.232+0000 7fb207e13640 1 -- 172.21.15.112:0/3705278352 wait complete. 2024-09-16T00:34:59.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.309+0000 7fde02619640 1 -- 172.21.15.112:0/3441929752 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fddfc102620 msgr2=0x7fddfc1029f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:59.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.309+0000 7fde02619640 1 --2- 172.21.15.112:0/3441929752 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fddfc102620 0x7fddfc1029f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdde40048d0 tx=0x7fdde4009da0 comp rx=0 tx=0).stop 2024-09-16T00:34:59.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.309+0000 7fde02619640 1 -- 172.21.15.112:0/3441929752 shutdown_connections 2024-09-16T00:34:59.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.309+0000 7fde02619640 1 --2- 172.21.15.112:0/3441929752 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fddfc0fac80 0x7fddfc0fb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:59.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.309+0000 7fde02619640 1 --2- 172.21.15.112:0/3441929752 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fddfc102620 0x7fddfc1029f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:59.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.309+0000 7fde02619640 1 -- 172.21.15.112:0/3441929752 >> 172.21.15.112:0/3441929752 conn(0x7fddfc070ce0 msgr2=0x7fddfc0710e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:59.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.310+0000 7fde02619640 1 -- 172.21.15.112:0/3441929752 shutdown_connections 2024-09-16T00:34:59.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.310+0000 7fde02619640 1 -- 172.21.15.112:0/3441929752 wait complete. 2024-09-16T00:34:59.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.310+0000 7fde02619640 1 Processor -- start 2024-09-16T00:34:59.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.310+0000 7fde02619640 1 -- start start 2024-09-16T00:34:59.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.311+0000 7fde02619640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fddfc0fac80 0x7fddfc1131a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:59.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.311+0000 7fde02619640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fddfc102620 0x7fddfc1136e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:59.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.311+0000 7fde02619640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddfc113ce0 con 0x7fddfc0fac80 2024-09-16T00:34:59.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.311+0000 7fde02619640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddfc10d120 con 0x7fddfc102620 2024-09-16T00:34:59.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.311+0000 7fddfbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fddfc0fac80 0x7fddfc1131a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:59.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.311+0000 7fddfbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fddfc0fac80 0x7fddfc1131a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35698/0 (socket says 172.21.15.112:35698) 2024-09-16T00:34:59.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.311+0000 7fddfbfff640 1 -- 172.21.15.112:0/2074071595 learned_addr learned my addr 172.21.15.112:0/2074071595 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:34:59.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.311+0000 7fddfb7fe640 1 --2- 172.21.15.112:0/2074071595 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fddfc102620 0x7fddfc1136e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:59.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.311+0000 7fddfbfff640 1 -- 172.21.15.112:0/2074071595 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fddfc102620 msgr2=0x7fddfc1136e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:59.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.311+0000 7fddfbfff640 1 --2- 172.21.15.112:0/2074071595 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fddfc102620 0x7fddfc1136e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:59.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.311+0000 7fddfbfff640 1 -- 172.21.15.112:0/2074071595 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdde4005040 con 0x7fddfc0fac80 2024-09-16T00:34:59.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.311+0000 7fddfbfff640 1 --2- 172.21.15.112:0/2074071595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fddfc0fac80 0x7fddfc1131a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdde40043d0 tx=0x7fdde4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:59.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.311+0000 7fddf8ff9640 1 -- 172.21.15.112:0/2074071595 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdde4038070 con 0x7fddfc0fac80 2024-09-16T00:34:59.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.311+0000 7fddf8ff9640 1 -- 172.21.15.112:0/2074071595 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fdde40096e0 con 0x7fddfc0fac80 2024-09-16T00:34:59.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.311+0000 7fde02619640 1 -- 172.21.15.112:0/2074071595 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddfc10d3a0 con 0x7fddfc0fac80 2024-09-16T00:34:59.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.311+0000 7fde02619640 1 -- 172.21.15.112:0/2074071595 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddfc10d940 con 0x7fddfc0fac80 2024-09-16T00:34:59.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.311+0000 7fddf8ff9640 1 -- 172.21.15.112:0/2074071595 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdde403d420 con 0x7fddfc0fac80 2024-09-16T00:34:59.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.313+0000 7fde02619640 1 -- 172.21.15.112:0/2074071595 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddfc0657b0 con 0x7fddfc0fac80 2024-09-16T00:34:59.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.313+0000 7fddf8ff9640 1 -- 172.21.15.112:0/2074071595 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fdde4030020 con 0x7fddfc0fac80 2024-09-16T00:34:59.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.314+0000 7fddf8ff9640 1 --2- 172.21.15.112:0/2074071595 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fdde806e530 0x7fdde80709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:59.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.314+0000 7fddf8ff9640 1 -- 172.21.15.112:0/2074071595 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7fdde40ae7c0 con 0x7fddfc0fac80 2024-09-16T00:34:59.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.314+0000 7fddfb7fe640 1 --2- 172.21.15.112:0/2074071595 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fdde806e530 0x7fdde80709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:59.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.314+0000 7fddfb7fe640 1 --2- 172.21.15.112:0/2074071595 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fdde806e530 0x7fdde80709e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fddfc10ea60 tx=0x7fddec00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:59.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.318+0000 7fddf8ff9640 1 -- 172.21.15.112:0/2074071595 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fdde4077730 con 0x7fddfc0fac80 2024-09-16T00:34:59.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.550+0000 7fde02619640 1 -- 172.21.15.112:0/2074071595 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fddfc04b720 con 0x7fddfc0fac80 2024-09-16T00:34:59.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.551+0000 7fddf8ff9640 1 -- 172.21.15.112:0/2074071595 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7fdde407adf0 con 0x7fddfc0fac80 2024-09-16T00:34:59.551 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:34:59.552 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": { 2024-09-16T00:34:59.552 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:34:59.552 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:34:59.552 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": { 2024-09-16T00:34:59.552 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:34:59.552 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:34:59.552 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": { 2024-09-16T00:34:59.552 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 6 2024-09-16T00:34:59.552 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:34:59.552 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": { 2024-09-16T00:34:59.552 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 4 2024-09-16T00:34:59.552 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:34:59.552 INFO:teuthology.orchestra.run.smithi112.stdout: "overall": { 2024-09-16T00:34:59.553 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 14 2024-09-16T00:34:59.553 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-16T00:34:59.553 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:34:59.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.553+0000 7fde02619640 1 -- 172.21.15.112:0/2074071595 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fdde806e530 msgr2=0x7fdde80709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:59.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.553+0000 7fde02619640 1 --2- 172.21.15.112:0/2074071595 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fdde806e530 0x7fdde80709e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fddfc10ea60 tx=0x7fddec00b040 comp rx=0 tx=0).stop 2024-09-16T00:34:59.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.553+0000 7fde02619640 1 -- 172.21.15.112:0/2074071595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fddfc0fac80 msgr2=0x7fddfc1131a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:59.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.553+0000 7fde02619640 1 --2- 172.21.15.112:0/2074071595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fddfc0fac80 0x7fddfc1131a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdde40043d0 tx=0x7fdde4009da0 comp rx=0 tx=0).stop 2024-09-16T00:34:59.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.553+0000 7fde02619640 1 -- 172.21.15.112:0/2074071595 shutdown_connections 2024-09-16T00:34:59.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.553+0000 7fde02619640 1 --2- 172.21.15.112:0/2074071595 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fdde806e530 0x7fdde80709e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:59.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.553+0000 7fde02619640 1 --2- 172.21.15.112:0/2074071595 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fddfc102620 0x7fddfc1136e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:59.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.553+0000 7fde02619640 1 --2- 172.21.15.112:0/2074071595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fddfc0fac80 0x7fddfc1131a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:59.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.553+0000 7fde02619640 1 -- 172.21.15.112:0/2074071595 >> 172.21.15.112:0/2074071595 conn(0x7fddfc070ce0 msgr2=0x7fddfc105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:59.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.554+0000 7fde02619640 1 -- 172.21.15.112:0/2074071595 shutdown_connections 2024-09-16T00:34:59.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.554+0000 7fde02619640 1 -- 172.21.15.112:0/2074071595 wait complete. 2024-09-16T00:34:59.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.632+0000 7f9cda582640 1 -- 172.21.15.112:0/4234759496 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cd4102550 msgr2=0x7f9cd4102920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:59.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.632+0000 7f9cda582640 1 --2- 172.21.15.112:0/4234759496 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cd4102550 0x7f9cd4102920 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9cc40048d0 tx=0x7f9cc4009da0 comp rx=0 tx=0).stop 2024-09-16T00:34:59.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.632+0000 7f9cda582640 1 -- 172.21.15.112:0/4234759496 shutdown_connections 2024-09-16T00:34:59.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.632+0000 7f9cda582640 1 --2- 172.21.15.112:0/4234759496 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9cd4102ef0 0x7f9cd40fc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:59.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.632+0000 7f9cda582640 1 --2- 172.21.15.112:0/4234759496 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cd4102550 0x7f9cd4102920 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:59.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.632+0000 7f9cda582640 1 -- 172.21.15.112:0/4234759496 >> 172.21.15.112:0/4234759496 conn(0x7f9cd40f80f0 msgr2=0x7f9cd40fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:59.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.632+0000 7f9cda582640 1 -- 172.21.15.112:0/4234759496 shutdown_connections 2024-09-16T00:34:59.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.633+0000 7f9cda582640 1 -- 172.21.15.112:0/4234759496 wait complete. 2024-09-16T00:34:59.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.633+0000 7f9cda582640 1 Processor -- start 2024-09-16T00:34:59.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.633+0000 7f9cda582640 1 -- start start 2024-09-16T00:34:59.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.633+0000 7f9cda582640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9cd4102550 0x7f9cd4113570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:59.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.633+0000 7f9cda582640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cd4102ef0 0x7f9cd4113ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:59.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.633+0000 7f9cda582640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd4114100 con 0x7f9cd4102ef0 2024-09-16T00:34:59.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.633+0000 7f9cda582640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd410d510 con 0x7f9cd4102550 2024-09-16T00:34:59.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.634+0000 7f9cd37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cd4102ef0 0x7f9cd4113ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:59.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.634+0000 7f9cd37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cd4102ef0 0x7f9cd4113ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35722/0 (socket says 172.21.15.112:35722) 2024-09-16T00:34:59.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.634+0000 7f9cd37fe640 1 -- 172.21.15.112:0/2944490500 learned_addr learned my addr 172.21.15.112:0/2944490500 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:34:59.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.634+0000 7f9cd3fff640 1 --2- 172.21.15.112:0/2944490500 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9cd4102550 0x7f9cd4113570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:59.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.634+0000 7f9cd37fe640 1 -- 172.21.15.112:0/2944490500 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9cd4102550 msgr2=0x7f9cd4113570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:59.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.634+0000 7f9cd37fe640 1 --2- 172.21.15.112:0/2944490500 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9cd4102550 0x7f9cd4113570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:59.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.634+0000 7f9cd37fe640 1 -- 172.21.15.112:0/2944490500 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cc4005040 con 0x7f9cd4102ef0 2024-09-16T00:34:59.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.634+0000 7f9cd37fe640 1 --2- 172.21.15.112:0/2944490500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cd4102ef0 0x7f9cd4113ab0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9cc8002870 tx=0x7f9cc8004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:59.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.634+0000 7f9cd0ff9640 1 -- 172.21.15.112:0/2944490500 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9cc8007c30 con 0x7f9cd4102ef0 2024-09-16T00:34:59.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.634+0000 7f9cd0ff9640 1 -- 172.21.15.112:0/2944490500 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f9cc800f040 con 0x7f9cd4102ef0 2024-09-16T00:34:59.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.634+0000 7f9cd0ff9640 1 -- 172.21.15.112:0/2944490500 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9cc800daf0 con 0x7f9cd4102ef0 2024-09-16T00:34:59.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.634+0000 7f9cda582640 1 -- 172.21.15.112:0/2944490500 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cd410d7f0 con 0x7f9cd4102ef0 2024-09-16T00:34:59.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.634+0000 7f9cda582640 1 -- 172.21.15.112:0/2944490500 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cd410dc50 con 0x7f9cd4102ef0 2024-09-16T00:34:59.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.635+0000 7f9cda582640 1 -- 172.21.15.112:0/2944490500 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cd40fdcd0 con 0x7f9cd4102ef0 2024-09-16T00:34:59.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.636+0000 7f9cd0ff9640 1 -- 172.21.15.112:0/2944490500 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f9cc800dc50 con 0x7f9cd4102ef0 2024-09-16T00:34:59.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.636+0000 7f9cd0ff9640 1 --2- 172.21.15.112:0/2944490500 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f9cbc06e530 0x7f9cbc0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:59.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.636+0000 7f9cd3fff640 1 --2- 172.21.15.112:0/2944490500 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f9cbc06e530 0x7f9cbc0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:59.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.636+0000 7f9cd0ff9640 1 -- 172.21.15.112:0/2944490500 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f9cc808e190 con 0x7f9cd4102ef0 2024-09-16T00:34:59.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.637+0000 7f9cd3fff640 1 --2- 172.21.15.112:0/2944490500 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f9cbc06e530 0x7f9cbc0709e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9cc4007ac0 tx=0x7f9cc4035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:59.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.641+0000 7f9cd0ff9640 1 -- 172.21.15.112:0/2944490500 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f9cc8057180 con 0x7f9cd4102ef0 2024-09-16T00:34:59.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.839+0000 7f9cda582640 1 -- 172.21.15.112:0/2944490500 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f9cd40fbd50 con 0x7f9cd4102ef0 2024-09-16T00:34:59.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.839+0000 7f9cd0ff9640 1 -- 172.21.15.112:0/2944490500 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1914 (secure 0 0 0) 0x7f9cc805a840 con 0x7f9cd4102ef0 2024-09-16T00:34:59.840 INFO:teuthology.orchestra.run.smithi112.stdout:e34 2024-09-16T00:34:59.840 INFO:teuthology.orchestra.run.smithi112.stdout:btime 2024-09-16T00:34:26:619254+0000 2024-09-16T00:34:59.840 INFO:teuthology.orchestra.run.smithi112.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T00:34:59.840 INFO:teuthology.orchestra.run.smithi112.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:34:59.840 INFO:teuthology.orchestra.run.smithi112.stdout:legacy client fscid: 1 2024-09-16T00:34:59.840 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:34:59.840 INFO:teuthology.orchestra.run.smithi112.stdout:Filesystem 'cephfs' (1) 2024-09-16T00:34:59.841 INFO:teuthology.orchestra.run.smithi112.stdout:fs_name cephfs 2024-09-16T00:34:59.841 INFO:teuthology.orchestra.run.smithi112.stdout:epoch 25 2024-09-16T00:34:59.841 INFO:teuthology.orchestra.run.smithi112.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-16T00:34:59.841 INFO:teuthology.orchestra.run.smithi112.stdout:created 2024-09-16T00:23:22.328199+0000 2024-09-16T00:34:59.841 INFO:teuthology.orchestra.run.smithi112.stdout:modified 2024-09-16T00:33:29.012991+0000 2024-09-16T00:34:59.841 INFO:teuthology.orchestra.run.smithi112.stdout:tableserver 0 2024-09-16T00:34:59.841 INFO:teuthology.orchestra.run.smithi112.stdout:root 0 2024-09-16T00:34:59.841 INFO:teuthology.orchestra.run.smithi112.stdout:session_timeout 60 2024-09-16T00:34:59.841 INFO:teuthology.orchestra.run.smithi112.stdout:session_autoclose 300 2024-09-16T00:34:59.841 INFO:teuthology.orchestra.run.smithi112.stdout:max_file_size 1099511627776 2024-09-16T00:34:59.841 INFO:teuthology.orchestra.run.smithi112.stdout:max_xattr_size 65536 2024-09-16T00:34:59.841 INFO:teuthology.orchestra.run.smithi112.stdout:required_client_features {} 2024-09-16T00:34:59.841 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure 0 2024-09-16T00:34:59.841 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure_osd_epoch 82 2024-09-16T00:34:59.842 INFO:teuthology.orchestra.run.smithi112.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:34:59.842 INFO:teuthology.orchestra.run.smithi112.stdout:max_mds 2 2024-09-16T00:34:59.842 INFO:teuthology.orchestra.run.smithi112.stdout:in 0,1 2024-09-16T00:34:59.842 INFO:teuthology.orchestra.run.smithi112.stdout:up {} 2024-09-16T00:34:59.842 INFO:teuthology.orchestra.run.smithi112.stdout:failed 0,1 2024-09-16T00:34:59.842 INFO:teuthology.orchestra.run.smithi112.stdout:damaged 2024-09-16T00:34:59.842 INFO:teuthology.orchestra.run.smithi112.stdout:stopped 2024-09-16T00:34:59.842 INFO:teuthology.orchestra.run.smithi112.stdout:data_pools [3] 2024-09-16T00:34:59.842 INFO:teuthology.orchestra.run.smithi112.stdout:metadata_pool 2 2024-09-16T00:34:59.842 INFO:teuthology.orchestra.run.smithi112.stdout:inline_data enabled 2024-09-16T00:34:59.842 INFO:teuthology.orchestra.run.smithi112.stdout:balancer 2024-09-16T00:34:59.842 INFO:teuthology.orchestra.run.smithi112.stdout:bal_rank_mask -1 2024-09-16T00:34:59.842 INFO:teuthology.orchestra.run.smithi112.stdout:standby_count_wanted 1 2024-09-16T00:34:59.842 INFO:teuthology.orchestra.run.smithi112.stdout:qdb_cluster leader: 0 members: 2024-09-16T00:34:59.842 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:34:59.843 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:34:59.843 INFO:teuthology.orchestra.run.smithi112.stdout:Standby daemons: 2024-09-16T00:34:59.843 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:34:59.843 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.ithwzj{-1:34392} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.112:6826/476229221,v1:172.21.15.112:6827/476229221] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:34:59.843 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.hxfwkh{-1:34404} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.112:6828/2356899876,v1:172.21.15.112:6829/2356899876] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:34:59.843 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.gawwgw{-1:44343} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.169:6824/1635329957,v1:172.21.15.169:6825/1635329957] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:34:59.843 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.lgeqrj{-1:44347} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.169:6826/184810031,v1:172.21.15.169:6827/184810031] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:34:59.844 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 34 2024-09-16T00:34:59.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.842+0000 7f9cda582640 1 -- 172.21.15.112:0/2944490500 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f9cbc06e530 msgr2=0x7f9cbc0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:59.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.842+0000 7f9cda582640 1 --2- 172.21.15.112:0/2944490500 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f9cbc06e530 0x7f9cbc0709e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9cc4007ac0 tx=0x7f9cc4035040 comp rx=0 tx=0).stop 2024-09-16T00:34:59.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.842+0000 7f9cda582640 1 -- 172.21.15.112:0/2944490500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cd4102ef0 msgr2=0x7f9cd4113ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:59.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.842+0000 7f9cda582640 1 --2- 172.21.15.112:0/2944490500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cd4102ef0 0x7f9cd4113ab0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9cc8002870 tx=0x7f9cc8004600 comp rx=0 tx=0).stop 2024-09-16T00:34:59.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.842+0000 7f9cda582640 1 -- 172.21.15.112:0/2944490500 shutdown_connections 2024-09-16T00:34:59.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.842+0000 7f9cda582640 1 --2- 172.21.15.112:0/2944490500 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f9cbc06e530 0x7f9cbc0709e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:59.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.842+0000 7f9cda582640 1 --2- 172.21.15.112:0/2944490500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cd4102ef0 0x7f9cd4113ab0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:59.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.842+0000 7f9cda582640 1 --2- 172.21.15.112:0/2944490500 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9cd4102550 0x7f9cd4113570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:59.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.842+0000 7f9cda582640 1 -- 172.21.15.112:0/2944490500 >> 172.21.15.112:0/2944490500 conn(0x7f9cd40f80f0 msgr2=0x7f9cd40f90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:59.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.843+0000 7f9cda582640 1 -- 172.21.15.112:0/2944490500 shutdown_connections 2024-09-16T00:34:59.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.843+0000 7f9cda582640 1 -- 172.21.15.112:0/2944490500 wait complete. 2024-09-16T00:34:59.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.921+0000 7f4fe903e640 1 -- 172.21.15.112:0/2405724625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fe40fd5c0 msgr2=0x7f4fe40fd990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:59.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.921+0000 7f4fe903e640 1 --2- 172.21.15.112:0/2405724625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fe40fd5c0 0x7f4fe40fd990 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4fd40048d0 tx=0x7f4fd4009da0 comp rx=0 tx=0).stop 2024-09-16T00:34:59.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.921+0000 7f4fe903e640 1 -- 172.21.15.112:0/2405724625 shutdown_connections 2024-09-16T00:34:59.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.921+0000 7f4fe903e640 1 --2- 172.21.15.112:0/2405724625 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4fe40fdf60 0x7f4fe4106630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:59.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.921+0000 7f4fe903e640 1 --2- 172.21.15.112:0/2405724625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fe40fd5c0 0x7f4fe40fd990 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:59.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.921+0000 7f4fe903e640 1 -- 172.21.15.112:0/2405724625 >> 172.21.15.112:0/2405724625 conn(0x7f4fe40f7060 msgr2=0x7f4fe40f9470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:59.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.922+0000 7f4fe903e640 1 -- 172.21.15.112:0/2405724625 shutdown_connections 2024-09-16T00:34:59.923 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:59 smithi112 ceph-mon[97750]: pgmap v217: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:34:59.923 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:59 smithi112 ceph-mon[97750]: from='client.34464 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:34:59.923 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:34:59 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/2074071595' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:34:59.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.922+0000 7f4fe903e640 1 -- 172.21.15.112:0/2405724625 wait complete. 2024-09-16T00:34:59.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.922+0000 7f4fe903e640 1 Processor -- start 2024-09-16T00:34:59.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.922+0000 7f4fe903e640 1 -- start start 2024-09-16T00:34:59.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.922+0000 7f4fe903e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fe40fd5c0 0x7f4fe40fb480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:59.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.922+0000 7f4fe903e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4fe40fdf60 0x7f4fe40fb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:59.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.922+0000 7f4fe903e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe40fbf00 con 0x7f4fe40fd5c0 2024-09-16T00:34:59.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.922+0000 7f4fe903e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe40fc070 con 0x7f4fe40fdf60 2024-09-16T00:34:59.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.922+0000 7f4fe2d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fe40fd5c0 0x7f4fe40fb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:59.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.922+0000 7f4fe2d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fe40fd5c0 0x7f4fe40fb480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35734/0 (socket says 172.21.15.112:35734) 2024-09-16T00:34:59.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.922+0000 7f4fe2d9d640 1 -- 172.21.15.112:0/3977179570 learned_addr learned my addr 172.21.15.112:0/3977179570 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:34:59.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.923+0000 7f4fdbfff640 1 --2- 172.21.15.112:0/3977179570 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4fe40fdf60 0x7f4fe40fb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:59.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.923+0000 7f4fe2d9d640 1 -- 172.21.15.112:0/3977179570 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4fe40fdf60 msgr2=0x7f4fe40fb9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:59.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.923+0000 7f4fe2d9d640 1 --2- 172.21.15.112:0/3977179570 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4fe40fdf60 0x7f4fe40fb9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:59.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.923+0000 7f4fe2d9d640 1 -- 172.21.15.112:0/3977179570 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fd4005040 con 0x7f4fe40fd5c0 2024-09-16T00:34:59.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.923+0000 7f4fdbfff640 1 --2- 172.21.15.112:0/3977179570 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4fe40fdf60 0x7f4fe40fb9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:34:59.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.923+0000 7f4fe2d9d640 1 --2- 172.21.15.112:0/3977179570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fe40fd5c0 0x7f4fe40fb480 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4fd40048a0 tx=0x7f4fd4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:59.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.923+0000 7f4fdb7fe640 1 -- 172.21.15.112:0/3977179570 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4fd4038070 con 0x7f4fe40fd5c0 2024-09-16T00:34:59.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.923+0000 7f4fdb7fe640 1 -- 172.21.15.112:0/3977179570 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f4fd4008eb0 con 0x7f4fe40fd5c0 2024-09-16T00:34:59.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.923+0000 7f4fdb7fe640 1 -- 172.21.15.112:0/3977179570 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4fd4009590 con 0x7f4fe40fd5c0 2024-09-16T00:34:59.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.923+0000 7f4fe903e640 1 -- 172.21.15.112:0/3977179570 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fe40fc2a0 con 0x7f4fe40fd5c0 2024-09-16T00:34:59.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.923+0000 7f4fe903e640 1 -- 172.21.15.112:0/3977179570 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fe4119820 con 0x7f4fe40fd5c0 2024-09-16T00:34:59.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.924+0000 7f4fe903e640 1 -- 172.21.15.112:0/3977179570 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fe4066290 con 0x7f4fe40fd5c0 2024-09-16T00:34:59.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.925+0000 7f4fdb7fe640 1 -- 172.21.15.112:0/3977179570 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f4fd4030020 con 0x7f4fe40fd5c0 2024-09-16T00:34:59.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.926+0000 7f4fdb7fe640 1 --2- 172.21.15.112:0/3977179570 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f4fd006e530 0x7f4fd00709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:59.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.926+0000 7f4fdb7fe640 1 -- 172.21.15.112:0/3977179570 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f4fd40ae280 con 0x7f4fe40fd5c0 2024-09-16T00:34:59.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.926+0000 7f4fdbfff640 1 --2- 172.21.15.112:0/3977179570 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f4fd006e530 0x7f4fd00709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:59.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.926+0000 7f4fdbfff640 1 --2- 172.21.15.112:0/3977179570 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f4fd006e530 0x7f4fd00709e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc007fd0 tx=0x7f4fcc009e90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:59.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:34:59.930+0000 7f4fdb7fe640 1 -- 172.21.15.112:0/3977179570 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f4fd4077310 con 0x7f4fe40fd5c0 2024-09-16T00:35:00.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:59 smithi169 ceph-mon[77626]: pgmap v217: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:35:00.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:59 smithi169 ceph-mon[77626]: from='client.34464 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:35:00.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:34:59 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/2074071595' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:35:00.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:00.113+0000 7f4fe903e640 1 -- 172.21.15.112:0/3977179570 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4fe40fcb70 con 0x7f4fd006e530 2024-09-16T00:35:00.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:00.114+0000 7f4fdb7fe640 1 -- 172.21.15.112:0/3977179570 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f4fe40fcb70 con 0x7f4fd006e530 2024-09-16T00:35:00.115 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:35:00.115 INFO:teuthology.orchestra.run.smithi112.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80", 2024-09-16T00:35:00.115 INFO:teuthology.orchestra.run.smithi112.stdout: "in_progress": true, 2024-09-16T00:35:00.115 INFO:teuthology.orchestra.run.smithi112.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T00:35:00.115 INFO:teuthology.orchestra.run.smithi112.stdout: "services_complete": [ 2024-09-16T00:35:00.115 INFO:teuthology.orchestra.run.smithi112.stdout: "mds", 2024-09-16T00:35:00.115 INFO:teuthology.orchestra.run.smithi112.stdout: "osd", 2024-09-16T00:35:00.115 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr", 2024-09-16T00:35:00.115 INFO:teuthology.orchestra.run.smithi112.stdout: "mon", 2024-09-16T00:35:00.115 INFO:teuthology.orchestra.run.smithi112.stdout: "crash" 2024-09-16T00:35:00.115 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-16T00:35:00.115 INFO:teuthology.orchestra.run.smithi112.stdout: "progress": "16/21 daemons upgraded", 2024-09-16T00:35:00.115 INFO:teuthology.orchestra.run.smithi112.stdout: "message": "Currently upgrading mds daemons", 2024-09-16T00:35:00.116 INFO:teuthology.orchestra.run.smithi112.stdout: "is_paused": false 2024-09-16T00:35:00.116 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:35:00.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:00.116+0000 7f4fe903e640 1 -- 172.21.15.112:0/3977179570 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f4fd006e530 msgr2=0x7f4fd00709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:00.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:00.116+0000 7f4fe903e640 1 --2- 172.21.15.112:0/3977179570 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f4fd006e530 0x7f4fd00709e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc007fd0 tx=0x7f4fcc009e90 comp rx=0 tx=0).stop 2024-09-16T00:35:00.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:00.116+0000 7f4fe903e640 1 -- 172.21.15.112:0/3977179570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fe40fd5c0 msgr2=0x7f4fe40fb480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:00.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:00.116+0000 7f4fe903e640 1 --2- 172.21.15.112:0/3977179570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fe40fd5c0 0x7f4fe40fb480 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4fd40048a0 tx=0x7f4fd4009da0 comp rx=0 tx=0).stop 2024-09-16T00:35:00.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:00.116+0000 7f4fe903e640 1 -- 172.21.15.112:0/3977179570 shutdown_connections 2024-09-16T00:35:00.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:00.116+0000 7f4fe903e640 1 --2- 172.21.15.112:0/3977179570 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f4fd006e530 0x7f4fd00709e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:00.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:00.116+0000 7f4fe903e640 1 --2- 172.21.15.112:0/3977179570 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4fe40fdf60 0x7f4fe40fb9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:00.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:00.116+0000 7f4fe903e640 1 --2- 172.21.15.112:0/3977179570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fe40fd5c0 0x7f4fe40fb480 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:00.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:00.116+0000 7f4fe903e640 1 -- 172.21.15.112:0/3977179570 >> 172.21.15.112:0/3977179570 conn(0x7f4fe40f7060 msgr2=0x7f4fe40f86e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:00.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:00.117+0000 7f4fe903e640 1 -- 172.21.15.112:0/3977179570 shutdown_connections 2024-09-16T00:35:00.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:00.117+0000 7f4fe903e640 1 -- 172.21.15.112:0/3977179570 wait complete. 2024-09-16T00:35:00.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:00.194+0000 7faeb2998640 1 -- 172.21.15.112:0/1192543289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faeac101510 msgr2=0x7faeac1018e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:00.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:00.194+0000 7faeb2998640 1 --2- 172.21.15.112:0/1192543289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faeac101510 0x7faeac1018e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fae9c0048d0 tx=0x7fae9c009da0 comp rx=0 tx=0).stop 2024-09-16T00:35:00.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:00.195+0000 7faeb2998640 1 -- 172.21.15.112:0/1192543289 shutdown_connections 2024-09-16T00:35:00.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:00.195+0000 7faeb2998640 1 --2- 172.21.15.112:0/1192543289 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faeac0fb4d0 0x7faeac0fb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:00.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:00.195+0000 7faeb2998640 1 --2- 172.21.15.112:0/1192543289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faeac101510 0x7faeac1018e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:00.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:00.195+0000 7faeb2998640 1 -- 172.21.15.112:0/1192543289 >> 172.21.15.112:0/1192543289 conn(0x7faeac0f7050 msgr2=0x7faeac0f9460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:00.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:00.195+0000 7faeb2998640 1 -- 172.21.15.112:0/1192543289 shutdown_connections 2024-09-16T00:35:00.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:00.195+0000 7faeb2998640 1 -- 172.21.15.112:0/1192543289 wait complete. 2024-09-16T00:35:00.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:00.195+0000 7faeb2998640 1 Processor -- start 2024-09-16T00:35:00.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:00.195+0000 7faeb2998640 1 -- start start 2024-09-16T00:35:00.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:00.196+0000 7faeb2998640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faeac0fb4d0 0x7faeac10bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:00.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:00.196+0000 7faeb2998640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faeac101510 0x7faeac10c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:00.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:00.196+0000 7faeb2998640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeac10c9a0 con 0x7faeac101510 2024-09-16T00:35:00.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:00.196+0000 7faeb2998640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeac10cae0 con 0x7faeac0fb4d0 2024-09-16T00:35:00.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:00.196+0000 7faeab7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faeac101510 0x7faeac10c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:00.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:00.196+0000 7faeab7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faeac101510 0x7faeac10c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35750/0 (socket says 172.21.15.112:35750) 2024-09-16T00:35:00.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:00.196+0000 7faeab7fe640 1 -- 172.21.15.112:0/3302604619 learned_addr learned my addr 172.21.15.112:0/3302604619 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:35:00.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:00.196+0000 7faeabfff640 1 --2- 172.21.15.112:0/3302604619 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faeac0fb4d0 0x7faeac10bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:00.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:00.196+0000 7faeab7fe640 1 -- 172.21.15.112:0/3302604619 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faeac0fb4d0 msgr2=0x7faeac10bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:00.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:00.196+0000 7faeab7fe640 1 --2- 172.21.15.112:0/3302604619 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faeac0fb4d0 0x7faeac10bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:00.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:00.196+0000 7faeab7fe640 1 -- 172.21.15.112:0/3302604619 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae9c005040 con 0x7faeac101510 2024-09-16T00:35:00.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:00.196+0000 7faeab7fe640 1 --2- 172.21.15.112:0/3302604619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faeac101510 0x7faeac10c460 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7faea0002a80 tx=0x7faea0004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:00.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:00.197+0000 7faea8ff9640 1 -- 172.21.15.112:0/3302604619 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faea0007c30 con 0x7faeac101510 2024-09-16T00:35:00.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:00.197+0000 7faea8ff9640 1 -- 172.21.15.112:0/3302604619 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7faea000f040 con 0x7faeac101510 2024-09-16T00:35:00.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:00.197+0000 7faeb2998640 1 -- 172.21.15.112:0/3302604619 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faeac10cd90 con 0x7faeac101510 2024-09-16T00:35:00.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:00.197+0000 7faea8ff9640 1 -- 172.21.15.112:0/3302604619 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faea0009530 con 0x7faeac101510 2024-09-16T00:35:00.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:00.197+0000 7faeb2998640 1 -- 172.21.15.112:0/3302604619 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faeac1198c0 con 0x7faeac101510 2024-09-16T00:35:00.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:00.198+0000 7faeb2998640 1 -- 172.21.15.112:0/3302604619 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faeac0fcc10 con 0x7faeac101510 2024-09-16T00:35:00.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:00.199+0000 7faea8ff9640 1 -- 172.21.15.112:0/3302604619 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7faea00097a0 con 0x7faeac101510 2024-09-16T00:35:00.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:00.199+0000 7faea8ff9640 1 --2- 172.21.15.112:0/3302604619 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fae9406e530 0x7fae940709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:00.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:00.199+0000 7faea8ff9640 1 -- 172.21.15.112:0/3302604619 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7faea008dad0 con 0x7faeac101510 2024-09-16T00:35:00.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:00.199+0000 7faeabfff640 1 --2- 172.21.15.112:0/3302604619 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fae9406e530 0x7fae940709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:00.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:00.200+0000 7faeabfff640 1 --2- 172.21.15.112:0/3302604619 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fae9406e530 0x7fae940709e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fae9c009ac0 tx=0x7fae9c035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:00.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:00.203+0000 7faea8ff9640 1 -- 172.21.15.112:0/3302604619 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7faea0056a40 con 0x7faeac101510 2024-09-16T00:35:00.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:00.434+0000 7faeb2998640 1 -- 172.21.15.112:0/3302604619 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7faeac0fac90 con 0x7faeac101510 2024-09-16T00:35:00.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:00.434+0000 7faea8ff9640 1 -- 172.21.15.112:0/3302604619 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7faea005a100 con 0x7faeac101510 2024-09-16T00:35:00.435 INFO:teuthology.orchestra.run.smithi112.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-16T00:35:00.435 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-16T00:35:00.435 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs is degraded 2024-09-16T00:35:00.435 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T00:35:00.435 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T00:35:00.435 INFO:teuthology.orchestra.run.smithi112.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-16T00:35:00.435 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-16T00:35:00.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:00.436+0000 7faeb2998640 1 -- 172.21.15.112:0/3302604619 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fae9406e530 msgr2=0x7fae940709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:00.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:00.436+0000 7faeb2998640 1 --2- 172.21.15.112:0/3302604619 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fae9406e530 0x7fae940709e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fae9c009ac0 tx=0x7fae9c035040 comp rx=0 tx=0).stop 2024-09-16T00:35:00.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:00.436+0000 7faeb2998640 1 -- 172.21.15.112:0/3302604619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faeac101510 msgr2=0x7faeac10c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:00.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:00.436+0000 7faeb2998640 1 --2- 172.21.15.112:0/3302604619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faeac101510 0x7faeac10c460 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7faea0002a80 tx=0x7faea0004600 comp rx=0 tx=0).stop 2024-09-16T00:35:00.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:00.436+0000 7faeb2998640 1 -- 172.21.15.112:0/3302604619 shutdown_connections 2024-09-16T00:35:00.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:00.436+0000 7faeb2998640 1 --2- 172.21.15.112:0/3302604619 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fae9406e530 0x7fae940709e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:00.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:00.436+0000 7faeb2998640 1 --2- 172.21.15.112:0/3302604619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faeac101510 0x7faeac10c460 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:00.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:00.436+0000 7faeb2998640 1 --2- 172.21.15.112:0/3302604619 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faeac0fb4d0 0x7faeac10bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:00.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:00.436+0000 7faeb2998640 1 -- 172.21.15.112:0/3302604619 >> 172.21.15.112:0/3302604619 conn(0x7faeac0f7050 msgr2=0x7faeac0f7420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:00.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:00.436+0000 7faeb2998640 1 -- 172.21.15.112:0/3302604619 shutdown_connections 2024-09-16T00:35:00.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:00.436+0000 7faeb2998640 1 -- 172.21.15.112:0/3302604619 wait complete. 2024-09-16T00:35:01.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:35:00 smithi169 ceph-mon[77626]: from='client.34468 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:35:01.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:35:00 smithi169 ceph-mon[77626]: from='client.34472 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:35:01.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:35:00 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/2944490500' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:35:01.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:35:00 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/3302604619' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:35:01.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:35:00 smithi112 ceph-mon[97750]: from='client.34468 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:35:01.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:35:00 smithi112 ceph-mon[97750]: from='client.34472 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:35:01.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:35:00 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/2944490500' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:35:01.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:35:00 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/3302604619' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:35:02.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:35:01 smithi169 ceph-mon[77626]: from='client.34484 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:35:02.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:35:01 smithi169 ceph-mon[77626]: pgmap v218: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:35:02.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:35:01 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:35:02.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:35:01 smithi112 ceph-mon[97750]: from='client.34484 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:35:02.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:35:01 smithi112 ceph-mon[97750]: pgmap v218: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:35:02.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:35:01 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:35:04.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:35:03 smithi169 ceph-mon[77626]: pgmap v219: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:35:04.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:35:03 smithi112 ceph-mon[97750]: pgmap v219: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:35:06.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:35:05 smithi169 ceph-mon[77626]: pgmap v220: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:35:06.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:35:05 smithi112 ceph-mon[97750]: pgmap v220: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:35:08.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:35:07 smithi169 ceph-mon[77626]: pgmap v221: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:35:08.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:35:07 smithi112 ceph-mon[97750]: pgmap v221: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:35:10.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:35:09 smithi169 ceph-mon[77626]: pgmap v222: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:35:10.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:35:09 smithi112 ceph-mon[97750]: pgmap v222: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:35:12.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:35:11 smithi169 ceph-mon[77626]: pgmap v223: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:35:12.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:35:11 smithi112 ceph-mon[97750]: pgmap v223: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:35:14.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:35:13 smithi169 ceph-mon[77626]: pgmap v224: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:35:14.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:35:13 smithi112 ceph-mon[97750]: pgmap v224: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:35:16.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:35:15 smithi169 ceph-mon[77626]: pgmap v225: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:35:16.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:35:15 smithi112 ceph-mon[97750]: pgmap v225: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:35:17.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:35:16 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:35:17.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:35:16 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:35:18.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:35:17 smithi169 ceph-mon[77626]: pgmap v226: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:35:18.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:35:17 smithi112 ceph-mon[97750]: pgmap v226: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:35:20.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:35:19 smithi169 ceph-mon[77626]: pgmap v227: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:35:20.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:35:19 smithi112 ceph-mon[97750]: pgmap v227: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:35:22.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:35:21 smithi169 ceph-mon[77626]: pgmap v228: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:35:22.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:35:21 smithi112 ceph-mon[97750]: pgmap v228: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:35:24.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:35:23 smithi169 ceph-mon[77626]: pgmap v229: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:35:24.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:35:23 smithi112 ceph-mon[97750]: pgmap v229: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:35:26.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:35:25 smithi169 ceph-mon[77626]: pgmap v230: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:35:26.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:35:25 smithi112 ceph-mon[97750]: pgmap v230: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:35:28.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:35:27 smithi169 ceph-mon[77626]: pgmap v231: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:35:28.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:35:27 smithi112 ceph-mon[97750]: pgmap v231: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:35:30.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:35:29 smithi169 ceph-mon[77626]: pgmap v232: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:35:30.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:35:29 smithi112 ceph-mon[97750]: pgmap v232: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:35:30.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.527+0000 7f8b62995640 1 -- 172.21.15.112:0/443781477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b5c102550 msgr2=0x7f8b5c102920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:30.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.527+0000 7f8b62995640 1 --2- 172.21.15.112:0/443781477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b5c102550 0x7f8b5c102920 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c004810 tx=0x7f8b4c009da0 comp rx=0 tx=0).stop 2024-09-16T00:35:30.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.528+0000 7f8b62995640 1 -- 172.21.15.112:0/443781477 shutdown_connections 2024-09-16T00:35:30.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.528+0000 7f8b62995640 1 --2- 172.21.15.112:0/443781477 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b5c102ef0 0x7f8b5c0fc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:30.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.528+0000 7f8b62995640 1 --2- 172.21.15.112:0/443781477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b5c102550 0x7f8b5c102920 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:30.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.528+0000 7f8b62995640 1 -- 172.21.15.112:0/443781477 >> 172.21.15.112:0/443781477 conn(0x7f8b5c0f80f0 msgr2=0x7f8b5c0fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:30.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.528+0000 7f8b62995640 1 -- 172.21.15.112:0/443781477 shutdown_connections 2024-09-16T00:35:30.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.528+0000 7f8b62995640 1 -- 172.21.15.112:0/443781477 wait complete. 2024-09-16T00:35:30.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.528+0000 7f8b62995640 1 Processor -- start 2024-09-16T00:35:30.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.528+0000 7f8b62995640 1 -- start start 2024-09-16T00:35:30.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.529+0000 7f8b62995640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b5c102550 0x7f8b5c113570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:30.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.529+0000 7f8b62995640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b5c102ef0 0x7f8b5c113ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:30.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.529+0000 7f8b62995640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5c114100 con 0x7f8b5c102ef0 2024-09-16T00:35:30.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.529+0000 7f8b62995640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5c10d510 con 0x7f8b5c102550 2024-09-16T00:35:30.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.529+0000 7f8b5b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b5c102ef0 0x7f8b5c113ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:30.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.529+0000 7f8b5b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b5c102ef0 0x7f8b5c113ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56098/0 (socket says 172.21.15.112:56098) 2024-09-16T00:35:30.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.529+0000 7f8b5b7fe640 1 -- 172.21.15.112:0/2436848732 learned_addr learned my addr 172.21.15.112:0/2436848732 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:35:30.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.529+0000 7f8b5bfff640 1 --2- 172.21.15.112:0/2436848732 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b5c102550 0x7f8b5c113570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:30.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.529+0000 7f8b5b7fe640 1 -- 172.21.15.112:0/2436848732 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b5c102550 msgr2=0x7f8b5c113570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:30.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.529+0000 7f8b5b7fe640 1 --2- 172.21.15.112:0/2436848732 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b5c102550 0x7f8b5c113570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:30.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.529+0000 7f8b5b7fe640 1 -- 172.21.15.112:0/2436848732 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b4c005040 con 0x7f8b5c102ef0 2024-09-16T00:35:30.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.529+0000 7f8b5b7fe640 1 --2- 172.21.15.112:0/2436848732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b5c102ef0 0x7f8b5c113ab0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8b5000eb40 tx=0x7f8b5000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:30.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.530+0000 7f8b58ff9640 1 -- 172.21.15.112:0/2436848732 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b50007c30 con 0x7f8b5c102ef0 2024-09-16T00:35:30.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.530+0000 7f8b58ff9640 1 -- 172.21.15.112:0/2436848732 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f8b5000f040 con 0x7f8b5c102ef0 2024-09-16T00:35:30.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.530+0000 7f8b62995640 1 -- 172.21.15.112:0/2436848732 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b5c10d790 con 0x7f8b5c102ef0 2024-09-16T00:35:30.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.530+0000 7f8b58ff9640 1 -- 172.21.15.112:0/2436848732 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b50009570 con 0x7f8b5c102ef0 2024-09-16T00:35:30.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.530+0000 7f8b62995640 1 -- 172.21.15.112:0/2436848732 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b5c10dd30 con 0x7f8b5c102ef0 2024-09-16T00:35:30.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.531+0000 7f8b62995640 1 -- 172.21.15.112:0/2436848732 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b5c0fdcd0 con 0x7f8b5c102ef0 2024-09-16T00:35:30.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.531+0000 7f8b58ff9640 1 -- 172.21.15.112:0/2436848732 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f8b500096d0 con 0x7f8b5c102ef0 2024-09-16T00:35:30.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.532+0000 7f8b58ff9640 1 --2- 172.21.15.112:0/2436848732 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f8b4406e530 0x7f8b440709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:30.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.532+0000 7f8b5bfff640 1 --2- 172.21.15.112:0/2436848732 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f8b4406e530 0x7f8b440709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:30.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.532+0000 7f8b58ff9640 1 -- 172.21.15.112:0/2436848732 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f8b5008df30 con 0x7f8b5c102ef0 2024-09-16T00:35:30.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.533+0000 7f8b5bfff640 1 --2- 172.21.15.112:0/2436848732 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f8b4406e530 0x7f8b440709e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c007ac0 tx=0x7f8b4c035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:30.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.537+0000 7f8b58ff9640 1 -- 172.21.15.112:0/2436848732 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f8b500c31b0 con 0x7f8b5c102ef0 2024-09-16T00:35:30.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.724+0000 7f8b62995640 1 -- 172.21.15.112:0/2436848732 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8b5c113260 con 0x7f8b4406e530 2024-09-16T00:35:30.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.726+0000 7f8b58ff9640 1 -- 172.21.15.112:0/2436848732 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f8b5c113260 con 0x7f8b4406e530 2024-09-16T00:35:30.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.728+0000 7f8b62995640 1 -- 172.21.15.112:0/2436848732 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f8b4406e530 msgr2=0x7f8b440709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:30.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.728+0000 7f8b62995640 1 --2- 172.21.15.112:0/2436848732 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f8b4406e530 0x7f8b440709e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c007ac0 tx=0x7f8b4c035040 comp rx=0 tx=0).stop 2024-09-16T00:35:30.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.728+0000 7f8b62995640 1 -- 172.21.15.112:0/2436848732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b5c102ef0 msgr2=0x7f8b5c113ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:30.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.728+0000 7f8b62995640 1 --2- 172.21.15.112:0/2436848732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b5c102ef0 0x7f8b5c113ab0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8b5000eb40 tx=0x7f8b5000b040 comp rx=0 tx=0).stop 2024-09-16T00:35:30.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.728+0000 7f8b62995640 1 -- 172.21.15.112:0/2436848732 shutdown_connections 2024-09-16T00:35:30.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.728+0000 7f8b62995640 1 --2- 172.21.15.112:0/2436848732 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f8b4406e530 0x7f8b440709e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:30.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.728+0000 7f8b62995640 1 --2- 172.21.15.112:0/2436848732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b5c102ef0 0x7f8b5c113ab0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:30.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.728+0000 7f8b62995640 1 --2- 172.21.15.112:0/2436848732 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b5c102550 0x7f8b5c113570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:30.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.728+0000 7f8b62995640 1 -- 172.21.15.112:0/2436848732 >> 172.21.15.112:0/2436848732 conn(0x7f8b5c0f80f0 msgr2=0x7f8b5c0f90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:30.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.728+0000 7f8b62995640 1 -- 172.21.15.112:0/2436848732 shutdown_connections 2024-09-16T00:35:30.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.728+0000 7f8b62995640 1 -- 172.21.15.112:0/2436848732 wait complete. 2024-09-16T00:35:30.737 INFO:teuthology.orchestra.run.smithi112.stdout:true 2024-09-16T00:35:30.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.807+0000 7f3d87f1f640 1 -- 172.21.15.112:0/2692704222 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d801014e0 msgr2=0x7f3d801018b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:30.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.807+0000 7f3d87f1f640 1 --2- 172.21.15.112:0/2692704222 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d801014e0 0x7f3d801018b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3d78009c50 tx=0x7f3d78004d10 comp rx=0 tx=0).stop 2024-09-16T00:35:30.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.807+0000 7f3d87f1f640 1 -- 172.21.15.112:0/2692704222 shutdown_connections 2024-09-16T00:35:30.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.807+0000 7f3d87f1f640 1 --2- 172.21.15.112:0/2692704222 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d800fb4a0 0x7f3d800fb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:30.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.807+0000 7f3d87f1f640 1 --2- 172.21.15.112:0/2692704222 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d801014e0 0x7f3d801018b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:30.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.807+0000 7f3d87f1f640 1 -- 172.21.15.112:0/2692704222 >> 172.21.15.112:0/2692704222 conn(0x7f3d800f7060 msgr2=0x7f3d800f9470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:30.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.807+0000 7f3d87f1f640 1 -- 172.21.15.112:0/2692704222 shutdown_connections 2024-09-16T00:35:30.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.807+0000 7f3d87f1f640 1 -- 172.21.15.112:0/2692704222 wait complete. 2024-09-16T00:35:30.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.808+0000 7f3d87f1f640 1 Processor -- start 2024-09-16T00:35:30.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.808+0000 7f3d87f1f640 1 -- start start 2024-09-16T00:35:30.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.808+0000 7f3d87f1f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d800fb4a0 0x7f3d8010bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:30.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.808+0000 7f3d87f1f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d801014e0 0x7f3d8010c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:30.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.808+0000 7f3d87f1f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8010ca40 con 0x7f3d800fb4a0 2024-09-16T00:35:30.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.808+0000 7f3d87f1f640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8010cb80 con 0x7f3d801014e0 2024-09-16T00:35:30.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.808+0000 7f3d85cbb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d800fb4a0 0x7f3d8010bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:30.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.808+0000 7f3d85cbb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d800fb4a0 0x7f3d8010bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56116/0 (socket says 172.21.15.112:56116) 2024-09-16T00:35:30.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.808+0000 7f3d85cbb640 1 -- 172.21.15.112:0/1421122577 learned_addr learned my addr 172.21.15.112:0/1421122577 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:35:30.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.808+0000 7f3d854ba640 1 --2- 172.21.15.112:0/1421122577 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d801014e0 0x7f3d8010c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:30.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.809+0000 7f3d85cbb640 1 -- 172.21.15.112:0/1421122577 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d801014e0 msgr2=0x7f3d8010c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:30.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.809+0000 7f3d85cbb640 1 --2- 172.21.15.112:0/1421122577 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d801014e0 0x7f3d8010c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:30.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.809+0000 7f3d85cbb640 1 -- 172.21.15.112:0/1421122577 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d780098b0 con 0x7f3d800fb4a0 2024-09-16T00:35:30.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.809+0000 7f3d85cbb640 1 --2- 172.21.15.112:0/1421122577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d800fb4a0 0x7f3d8010bfc0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3d78004a30 tx=0x7f3d78004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:30.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.809+0000 7f3d7e7fc640 1 -- 172.21.15.112:0/1421122577 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d78009690 con 0x7f3d800fb4a0 2024-09-16T00:35:30.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.809+0000 7f3d7e7fc640 1 -- 172.21.15.112:0/1421122577 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f3d78032b10 con 0x7f3d800fb4a0 2024-09-16T00:35:30.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.809+0000 7f3d87f1f640 1 -- 172.21.15.112:0/1421122577 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d8010ce00 con 0x7f3d800fb4a0 2024-09-16T00:35:30.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.809+0000 7f3d7e7fc640 1 -- 172.21.15.112:0/1421122577 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d7803daa0 con 0x7f3d800fb4a0 2024-09-16T00:35:30.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.809+0000 7f3d87f1f640 1 -- 172.21.15.112:0/1421122577 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d80119930 con 0x7f3d800fb4a0 2024-09-16T00:35:30.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.810+0000 7f3d87f1f640 1 -- 172.21.15.112:0/1421122577 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d800fcbe0 con 0x7f3d800fb4a0 2024-09-16T00:35:30.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.811+0000 7f3d7e7fc640 1 -- 172.21.15.112:0/1421122577 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f3d7802f030 con 0x7f3d800fb4a0 2024-09-16T00:35:30.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.811+0000 7f3d7e7fc640 1 --2- 172.21.15.112:0/1421122577 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f3d6806e530 0x7f3d680709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:30.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.811+0000 7f3d7e7fc640 1 -- 172.21.15.112:0/1421122577 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f3d780af3b0 con 0x7f3d800fb4a0 2024-09-16T00:35:30.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.812+0000 7f3d854ba640 1 --2- 172.21.15.112:0/1421122577 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f3d6806e530 0x7f3d680709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:30.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.812+0000 7f3d854ba640 1 --2- 172.21.15.112:0/1421122577 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f3d6806e530 0x7f3d680709e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3d74006fd0 tx=0x7f3d74008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:30.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:30.816+0000 7f3d7e7fc640 1 -- 172.21.15.112:0/1421122577 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f3d78078320 con 0x7f3d800fb4a0 2024-09-16T00:35:31.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.000+0000 7f3d87f1f640 1 -- 172.21.15.112:0/1421122577 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3d80066220 con 0x7f3d6806e530 2024-09-16T00:35:31.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.001+0000 7f3d7e7fc640 1 -- 172.21.15.112:0/1421122577 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f3d80066220 con 0x7f3d6806e530 2024-09-16T00:35:31.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.004+0000 7f3d87f1f640 1 -- 172.21.15.112:0/1421122577 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f3d6806e530 msgr2=0x7f3d680709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:31.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.004+0000 7f3d87f1f640 1 --2- 172.21.15.112:0/1421122577 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f3d6806e530 0x7f3d680709e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3d74006fd0 tx=0x7f3d74008040 comp rx=0 tx=0).stop 2024-09-16T00:35:31.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.004+0000 7f3d87f1f640 1 -- 172.21.15.112:0/1421122577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d800fb4a0 msgr2=0x7f3d8010bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:31.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.004+0000 7f3d87f1f640 1 --2- 172.21.15.112:0/1421122577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d800fb4a0 0x7f3d8010bfc0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3d78004a30 tx=0x7f3d78004d10 comp rx=0 tx=0).stop 2024-09-16T00:35:31.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.004+0000 7f3d87f1f640 1 -- 172.21.15.112:0/1421122577 shutdown_connections 2024-09-16T00:35:31.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.004+0000 7f3d87f1f640 1 --2- 172.21.15.112:0/1421122577 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f3d6806e530 0x7f3d680709e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:31.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.004+0000 7f3d87f1f640 1 --2- 172.21.15.112:0/1421122577 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d801014e0 0x7f3d8010c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:31.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.004+0000 7f3d87f1f640 1 --2- 172.21.15.112:0/1421122577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d800fb4a0 0x7f3d8010bfc0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:31.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.004+0000 7f3d87f1f640 1 -- 172.21.15.112:0/1421122577 >> 172.21.15.112:0/1421122577 conn(0x7f3d800f7060 msgr2=0x7f3d800f8910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:31.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.004+0000 7f3d87f1f640 1 -- 172.21.15.112:0/1421122577 shutdown_connections 2024-09-16T00:35:31.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.004+0000 7f3d87f1f640 1 -- 172.21.15.112:0/1421122577 wait complete. 2024-09-16T00:35:31.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.082+0000 7f331a509640 1 -- 172.21.15.112:0/1436272948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3314102ed0 msgr2=0x7f33140fc590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:31.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.082+0000 7f331a509640 1 --2- 172.21.15.112:0/1436272948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3314102ed0 0x7f33140fc590 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f33080048d0 tx=0x7f3308009da0 comp rx=0 tx=0).stop 2024-09-16T00:35:31.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.082+0000 7f331a509640 1 -- 172.21.15.112:0/1436272948 shutdown_connections 2024-09-16T00:35:31.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.082+0000 7f331a509640 1 --2- 172.21.15.112:0/1436272948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3314102ed0 0x7f33140fc590 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:31.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.082+0000 7f331a509640 1 --2- 172.21.15.112:0/1436272948 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3314102530 0x7f3314102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:31.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.082+0000 7f331a509640 1 -- 172.21.15.112:0/1436272948 >> 172.21.15.112:0/1436272948 conn(0x7f33140f80f0 msgr2=0x7f33140fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:31.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.083+0000 7f331a509640 1 -- 172.21.15.112:0/1436272948 shutdown_connections 2024-09-16T00:35:31.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.083+0000 7f331a509640 1 -- 172.21.15.112:0/1436272948 wait complete. 2024-09-16T00:35:31.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.083+0000 7f331a509640 1 Processor -- start 2024-09-16T00:35:31.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.083+0000 7f331a509640 1 -- start start 2024-09-16T00:35:31.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.084+0000 7f331a509640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3314102530 0x7f33141131f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:31.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.084+0000 7f331a509640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3314102ed0 0x7f3314113730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:31.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.084+0000 7f331a509640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3314113d30 con 0x7f3314102530 2024-09-16T00:35:31.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.084+0000 7f331a509640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f331410d170 con 0x7f3314102ed0 2024-09-16T00:35:31.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.084+0000 7f3313fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3314102530 0x7f33141131f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:31.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.084+0000 7f3313fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3314102530 0x7f33141131f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56128/0 (socket says 172.21.15.112:56128) 2024-09-16T00:35:31.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.084+0000 7f3313fff640 1 -- 172.21.15.112:0/2613596817 learned_addr learned my addr 172.21.15.112:0/2613596817 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:35:31.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.084+0000 7f33137fe640 1 --2- 172.21.15.112:0/2613596817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3314102ed0 0x7f3314113730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:31.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.084+0000 7f3313fff640 1 -- 172.21.15.112:0/2613596817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3314102ed0 msgr2=0x7f3314113730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:31.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.084+0000 7f3313fff640 1 --2- 172.21.15.112:0/2613596817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3314102ed0 0x7f3314113730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:31.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.084+0000 7f3313fff640 1 -- 172.21.15.112:0/2613596817 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3308005040 con 0x7f3314102530 2024-09-16T00:35:31.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.084+0000 7f3313fff640 1 --2- 172.21.15.112:0/2613596817 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3314102530 0x7f33141131f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3304002a50 tx=0x7f3304004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:31.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.084+0000 7f3310ff9640 1 -- 172.21.15.112:0/2613596817 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3304007c30 con 0x7f3314102530 2024-09-16T00:35:31.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.084+0000 7f3310ff9640 1 -- 172.21.15.112:0/2613596817 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f330400f040 con 0x7f3314102530 2024-09-16T00:35:31.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.084+0000 7f3310ff9640 1 -- 172.21.15.112:0/2613596817 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f330400da10 con 0x7f3314102530 2024-09-16T00:35:31.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.084+0000 7f331a509640 1 -- 172.21.15.112:0/2613596817 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f331410d450 con 0x7f3314102530 2024-09-16T00:35:31.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.085+0000 7f331a509640 1 -- 172.21.15.112:0/2613596817 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f331410d900 con 0x7f3314102530 2024-09-16T00:35:31.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.086+0000 7f331a509640 1 -- 172.21.15.112:0/2613596817 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33140fdcb0 con 0x7f3314102530 2024-09-16T00:35:31.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.086+0000 7f3310ff9640 1 -- 172.21.15.112:0/2613596817 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f330400dbb0 con 0x7f3314102530 2024-09-16T00:35:31.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.086+0000 7f3310ff9640 1 --2- 172.21.15.112:0/2613596817 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f32fc06e530 0x7f32fc0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:31.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.087+0000 7f3310ff9640 1 -- 172.21.15.112:0/2613596817 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f330408da40 con 0x7f3314102530 2024-09-16T00:35:31.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.087+0000 7f33137fe640 1 --2- 172.21.15.112:0/2613596817 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f32fc06e530 0x7f32fc0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:31.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.087+0000 7f33137fe640 1 --2- 172.21.15.112:0/2613596817 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f32fc06e530 0x7f32fc0709e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f33080048a0 tx=0x7f3308035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:31.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.091+0000 7f3310ff9640 1 -- 172.21.15.112:0/2613596817 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f3304056ae0 con 0x7f3314102530 2024-09-16T00:35:31.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.264+0000 7f331a509640 1 -- 172.21.15.112:0/2613596817 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f331404b720 con 0x7f32fc06e530 2024-09-16T00:35:31.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.273+0000 7f3310ff9640 1 -- 172.21.15.112:0/2613596817 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f331404b720 con 0x7f32fc06e530 2024-09-16T00:35:31.273 INFO:teuthology.orchestra.run.smithi112.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T00:35:31.273 INFO:teuthology.orchestra.run.smithi112.stdout:alertmanager.smithi112 smithi112 *:9093,9094 running (8m) 92s ago 17m 24.7M - 0.25.0 c8568f914cd2 d85ea0562022 2024-09-16T00:35:31.273 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi112 smithi112 running (5m) 92s ago 17m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c e4fe58f9702a 2024-09-16T00:35:31.274 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi169 smithi169 running (5m) 63s ago 16m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c 3ba2c81d527f 2024-09-16T00:35:31.274 INFO:teuthology.orchestra.run.smithi112.stdout:grafana.smithi112 smithi112 *:3000 running (7m) 92s ago 17m 82.1M - 9.4.12 2bacad6d85d8 51f9d2013da2 2024-09-16T00:35:31.274 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.hxfwkh smithi112 running (96s) 92s ago 11m 14.2M - 19.1.1-326-g8921c8ef f1455bd0219c 1723069902d6 2024-09-16T00:35:31.274 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.ithwzj smithi112 running (113s) 92s ago 12m 12.2M - 19.1.1-326-g8921c8ef f1455bd0219c e7aad5b81db5 2024-09-16T00:35:31.274 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.gawwgw smithi169 running (81s) 63s ago 12m 15.7M - 19.1.1-326-g8921c8ef f1455bd0219c 469be459ee12 2024-09-16T00:35:31.274 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.lgeqrj smithi169 running (65s) 63s ago 11m 13.3M - 19.1.1-326-g8921c8ef f1455bd0219c 31f0fbd2395d 2024-09-16T00:35:31.274 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi112.xrkkbc smithi112 *:8443,9283,8765 running (8m) 92s ago 18m 504M - 19.1.1-326-g8921c8ef f1455bd0219c a981f7fd2810 2024-09-16T00:35:31.274 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi169.mtrobk smithi169 *:8443,9283,8765 running (7m) 63s ago 16m 442M - 19.1.1-326-g8921c8ef f1455bd0219c 4096e5369c2d 2024-09-16T00:35:31.274 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi112 smithi112 running (6m) 92s ago 18m 49.8M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 5aaec033c042 2024-09-16T00:35:31.274 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi169 smithi169 running (6m) 63s ago 16m 38.9M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 7bc609155f3f 2024-09-16T00:35:31.275 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi112 smithi112 *:9100 running (8m) 92s ago 17m 15.5M - 1.5.0 0da6a335fe13 6404d1d601e2 2024-09-16T00:35:31.275 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi169 smithi169 *:9100 running (8m) 63s ago 16m 15.4M - 1.5.0 0da6a335fe13 5505baf702a6 2024-09-16T00:35:31.275 INFO:teuthology.orchestra.run.smithi112.stdout:osd.0 smithi112 running (5m) 92s ago 15m 105M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 53efc9397017 2024-09-16T00:35:31.275 INFO:teuthology.orchestra.run.smithi112.stdout:osd.1 smithi112 running (4m) 92s ago 15m 98.9M 1557M 19.1.1-326-g8921c8ef f1455bd0219c fe6f1a4a2cf4 2024-09-16T00:35:31.275 INFO:teuthology.orchestra.run.smithi112.stdout:osd.2 smithi112 running (3m) 92s ago 14m 102M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 5f7863712057 2024-09-16T00:35:31.275 INFO:teuthology.orchestra.run.smithi112.stdout:osd.3 smithi169 running (3m) 63s ago 14m 92.1M 2581M 19.1.1-326-g8921c8ef f1455bd0219c ee07b5577810 2024-09-16T00:35:31.275 INFO:teuthology.orchestra.run.smithi112.stdout:osd.4 smithi169 running (2m) 63s ago 14m 103M 2581M 19.1.1-326-g8921c8ef f1455bd0219c 769f026a54eb 2024-09-16T00:35:31.275 INFO:teuthology.orchestra.run.smithi112.stdout:osd.5 smithi169 running (2m) 63s ago 13m 125M 2581M 19.1.1-326-g8921c8ef f1455bd0219c a6007c13cf78 2024-09-16T00:35:31.275 INFO:teuthology.orchestra.run.smithi112.stdout:prometheus.smithi112 smithi112 *:9095 running (7m) 92s ago 17m 64.1M - 2.43.0 a07b618ecd1d 7a1d42e60d2b 2024-09-16T00:35:31.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.275+0000 7f331a509640 1 -- 172.21.15.112:0/2613596817 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f32fc06e530 msgr2=0x7f32fc0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:31.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.275+0000 7f331a509640 1 --2- 172.21.15.112:0/2613596817 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f32fc06e530 0x7f32fc0709e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f33080048a0 tx=0x7f3308035040 comp rx=0 tx=0).stop 2024-09-16T00:35:31.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.275+0000 7f331a509640 1 -- 172.21.15.112:0/2613596817 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3314102530 msgr2=0x7f33141131f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:31.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.275+0000 7f331a509640 1 --2- 172.21.15.112:0/2613596817 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3314102530 0x7f33141131f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3304002a50 tx=0x7f3304004600 comp rx=0 tx=0).stop 2024-09-16T00:35:31.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.275+0000 7f331a509640 1 -- 172.21.15.112:0/2613596817 shutdown_connections 2024-09-16T00:35:31.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.275+0000 7f331a509640 1 --2- 172.21.15.112:0/2613596817 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f32fc06e530 0x7f32fc0709e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:31.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.275+0000 7f331a509640 1 --2- 172.21.15.112:0/2613596817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3314102ed0 0x7f3314113730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:31.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.275+0000 7f331a509640 1 --2- 172.21.15.112:0/2613596817 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3314102530 0x7f33141131f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:31.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.275+0000 7f331a509640 1 -- 172.21.15.112:0/2613596817 >> 172.21.15.112:0/2613596817 conn(0x7f33140f80f0 msgr2=0x7f33140f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:31.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.275+0000 7f331a509640 1 -- 172.21.15.112:0/2613596817 shutdown_connections 2024-09-16T00:35:31.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.275+0000 7f331a509640 1 -- 172.21.15.112:0/2613596817 wait complete. 2024-09-16T00:35:31.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.352+0000 7ffb4638c640 1 -- 172.21.15.112:0/812860314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb400facf0 msgr2=0x7ffb400fb160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:31.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.352+0000 7ffb4638c640 1 --2- 172.21.15.112:0/812860314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb400facf0 0x7ffb400fb160 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ffb340048d0 tx=0x7ffb34009da0 comp rx=0 tx=0).stop 2024-09-16T00:35:31.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.352+0000 7ffb4638c640 1 -- 172.21.15.112:0/812860314 shutdown_connections 2024-09-16T00:35:31.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.352+0000 7ffb4638c640 1 --2- 172.21.15.112:0/812860314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb400facf0 0x7ffb400fb160 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:31.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.352+0000 7ffb4638c640 1 --2- 172.21.15.112:0/812860314 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb40102680 0x7ffb40102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:31.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.352+0000 7ffb4638c640 1 -- 172.21.15.112:0/812860314 >> 172.21.15.112:0/812860314 conn(0x7ffb40070d50 msgr2=0x7ffb40071150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:31.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.352+0000 7ffb4638c640 1 -- 172.21.15.112:0/812860314 shutdown_connections 2024-09-16T00:35:31.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.352+0000 7ffb4638c640 1 -- 172.21.15.112:0/812860314 wait complete. 2024-09-16T00:35:31.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.353+0000 7ffb4638c640 1 Processor -- start 2024-09-16T00:35:31.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.353+0000 7ffb4638c640 1 -- start start 2024-09-16T00:35:31.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.353+0000 7ffb4638c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb400facf0 0x7ffb401132d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:31.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.353+0000 7ffb4638c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb40102680 0x7ffb40113810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:31.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.353+0000 7ffb4638c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb40113e10 con 0x7ffb40102680 2024-09-16T00:35:31.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.353+0000 7ffb4638c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4010d250 con 0x7ffb400facf0 2024-09-16T00:35:31.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.353+0000 7ffb3f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb40102680 0x7ffb40113810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:31.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.353+0000 7ffb3f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb40102680 0x7ffb40113810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56144/0 (socket says 172.21.15.112:56144) 2024-09-16T00:35:31.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.353+0000 7ffb3f7fe640 1 -- 172.21.15.112:0/1415359184 learned_addr learned my addr 172.21.15.112:0/1415359184 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:35:31.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.353+0000 7ffb3ffff640 1 --2- 172.21.15.112:0/1415359184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb400facf0 0x7ffb401132d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:31.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.354+0000 7ffb3f7fe640 1 -- 172.21.15.112:0/1415359184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb400facf0 msgr2=0x7ffb401132d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:31.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.354+0000 7ffb3f7fe640 1 --2- 172.21.15.112:0/1415359184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb400facf0 0x7ffb401132d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:31.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.354+0000 7ffb3f7fe640 1 -- 172.21.15.112:0/1415359184 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb34005040 con 0x7ffb40102680 2024-09-16T00:35:31.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.354+0000 7ffb3f7fe640 1 --2- 172.21.15.112:0/1415359184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb40102680 0x7ffb40113810 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ffb34000c00 tx=0x7ffb34009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:31.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.354+0000 7ffb3cff9640 1 -- 172.21.15.112:0/1415359184 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb34038070 con 0x7ffb40102680 2024-09-16T00:35:31.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.354+0000 7ffb4638c640 1 -- 172.21.15.112:0/1415359184 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb4010d4d0 con 0x7ffb40102680 2024-09-16T00:35:31.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.354+0000 7ffb3cff9640 1 -- 172.21.15.112:0/1415359184 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7ffb3402ade0 con 0x7ffb40102680 2024-09-16T00:35:31.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.354+0000 7ffb3cff9640 1 -- 172.21.15.112:0/1415359184 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb3402c7b0 con 0x7ffb40102680 2024-09-16T00:35:31.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.354+0000 7ffb4638c640 1 -- 172.21.15.112:0/1415359184 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb4010d990 con 0x7ffb40102680 2024-09-16T00:35:31.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.355+0000 7ffb4638c640 1 -- 172.21.15.112:0/1415359184 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb400657f0 con 0x7ffb40102680 2024-09-16T00:35:31.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.356+0000 7ffb3cff9640 1 -- 172.21.15.112:0/1415359184 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7ffb34030020 con 0x7ffb40102680 2024-09-16T00:35:31.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.356+0000 7ffb3cff9640 1 --2- 172.21.15.112:0/1415359184 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ffb2806e530 0x7ffb280709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:31.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.356+0000 7ffb3cff9640 1 -- 172.21.15.112:0/1415359184 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7ffb340ae720 con 0x7ffb40102680 2024-09-16T00:35:31.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.356+0000 7ffb3ffff640 1 --2- 172.21.15.112:0/1415359184 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ffb2806e530 0x7ffb280709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:31.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.357+0000 7ffb3ffff640 1 --2- 172.21.15.112:0/1415359184 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ffb2806e530 0x7ffb280709e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ffb30009af0 tx=0x7ffb3000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:31.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.360+0000 7ffb3cff9640 1 -- 172.21.15.112:0/1415359184 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7ffb34077690 con 0x7ffb40102680 2024-09-16T00:35:31.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.595+0000 7ffb4638c640 1 -- 172.21.15.112:0/1415359184 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ffb40074c70 con 0x7ffb40102680 2024-09-16T00:35:31.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.595+0000 7ffb3cff9640 1 -- 172.21.15.112:0/1415359184 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7ffb3407ad50 con 0x7ffb40102680 2024-09-16T00:35:31.596 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:35:31.596 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": { 2024-09-16T00:35:31.596 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:35:31.596 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:35:31.596 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": { 2024-09-16T00:35:31.596 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:35:31.596 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:35:31.596 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": { 2024-09-16T00:35:31.596 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 6 2024-09-16T00:35:31.597 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:35:31.597 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": { 2024-09-16T00:35:31.597 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 4 2024-09-16T00:35:31.597 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:35:31.597 INFO:teuthology.orchestra.run.smithi112.stdout: "overall": { 2024-09-16T00:35:31.597 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 14 2024-09-16T00:35:31.597 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-16T00:35:31.597 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:35:31.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.597+0000 7ffb4638c640 1 -- 172.21.15.112:0/1415359184 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ffb2806e530 msgr2=0x7ffb280709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:31.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.597+0000 7ffb4638c640 1 --2- 172.21.15.112:0/1415359184 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ffb2806e530 0x7ffb280709e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ffb30009af0 tx=0x7ffb3000b040 comp rx=0 tx=0).stop 2024-09-16T00:35:31.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.597+0000 7ffb4638c640 1 -- 172.21.15.112:0/1415359184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb40102680 msgr2=0x7ffb40113810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:31.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.597+0000 7ffb4638c640 1 --2- 172.21.15.112:0/1415359184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb40102680 0x7ffb40113810 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ffb34000c00 tx=0x7ffb34009da0 comp rx=0 tx=0).stop 2024-09-16T00:35:31.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.597+0000 7ffb4638c640 1 -- 172.21.15.112:0/1415359184 shutdown_connections 2024-09-16T00:35:31.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.597+0000 7ffb4638c640 1 --2- 172.21.15.112:0/1415359184 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ffb2806e530 0x7ffb280709e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:31.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.597+0000 7ffb4638c640 1 --2- 172.21.15.112:0/1415359184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb40102680 0x7ffb40113810 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:31.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.597+0000 7ffb4638c640 1 --2- 172.21.15.112:0/1415359184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb400facf0 0x7ffb401132d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:31.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.597+0000 7ffb4638c640 1 -- 172.21.15.112:0/1415359184 >> 172.21.15.112:0/1415359184 conn(0x7ffb40070d50 msgr2=0x7ffb400fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:31.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.598+0000 7ffb4638c640 1 -- 172.21.15.112:0/1415359184 shutdown_connections 2024-09-16T00:35:31.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.598+0000 7ffb4638c640 1 -- 172.21.15.112:0/1415359184 wait complete. 2024-09-16T00:35:31.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.676+0000 7f0658200640 1 -- 172.21.15.112:0/3812932708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06500fb4a0 msgr2=0x7f06500fb910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:31.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.676+0000 7f0658200640 1 --2- 172.21.15.112:0/3812932708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06500fb4a0 0x7f06500fb910 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f064c0048d0 tx=0x7f064c009da0 comp rx=0 tx=0).stop 2024-09-16T00:35:31.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.676+0000 7f0658200640 1 -- 172.21.15.112:0/3812932708 shutdown_connections 2024-09-16T00:35:31.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.676+0000 7f0658200640 1 --2- 172.21.15.112:0/3812932708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06500fb4a0 0x7f06500fb910 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:31.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.676+0000 7f0658200640 1 --2- 172.21.15.112:0/3812932708 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f06501014e0 0x7f06501018b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:31.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.676+0000 7f0658200640 1 -- 172.21.15.112:0/3812932708 >> 172.21.15.112:0/3812932708 conn(0x7f06500f7060 msgr2=0x7f06500f9470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:31.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.676+0000 7f0658200640 1 -- 172.21.15.112:0/3812932708 shutdown_connections 2024-09-16T00:35:31.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.676+0000 7f0658200640 1 -- 172.21.15.112:0/3812932708 wait complete. 2024-09-16T00:35:31.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.676+0000 7f0658200640 1 Processor -- start 2024-09-16T00:35:31.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.676+0000 7f0658200640 1 -- start start 2024-09-16T00:35:31.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.677+0000 7f0658200640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06500fb4a0 0x7f06501120d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:31.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.677+0000 7f0658200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f06501014e0 0x7f0650112610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:31.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.677+0000 7f0658200640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0650112c10 con 0x7f06500fb4a0 2024-09-16T00:35:31.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.677+0000 7f0658200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f065010c050 con 0x7f06501014e0 2024-09-16T00:35:31.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.677+0000 7f0655f9c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06500fb4a0 0x7f06501120d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:31.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.677+0000 7f0655f9c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06500fb4a0 0x7f06501120d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56158/0 (socket says 172.21.15.112:56158) 2024-09-16T00:35:31.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.677+0000 7f0655f9c640 1 -- 172.21.15.112:0/2296743607 learned_addr learned my addr 172.21.15.112:0/2296743607 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:35:31.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.677+0000 7f065579b640 1 --2- 172.21.15.112:0/2296743607 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f06501014e0 0x7f0650112610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:31.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.677+0000 7f0655f9c640 1 -- 172.21.15.112:0/2296743607 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f06501014e0 msgr2=0x7f0650112610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:31.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.677+0000 7f0655f9c640 1 --2- 172.21.15.112:0/2296743607 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f06501014e0 0x7f0650112610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:31.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.677+0000 7f0655f9c640 1 -- 172.21.15.112:0/2296743607 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f064c005040 con 0x7f06500fb4a0 2024-09-16T00:35:31.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.677+0000 7f065579b640 1 --2- 172.21.15.112:0/2296743607 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f06501014e0 0x7f0650112610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:35:31.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.677+0000 7f0655f9c640 1 --2- 172.21.15.112:0/2296743607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06500fb4a0 0x7f06501120d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0644002a50 tx=0x7f0644004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:31.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.678+0000 7f06427fc640 1 -- 172.21.15.112:0/2296743607 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0644007c30 con 0x7f06500fb4a0 2024-09-16T00:35:31.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.678+0000 7f06427fc640 1 -- 172.21.15.112:0/2296743607 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f064400f040 con 0x7f06500fb4a0 2024-09-16T00:35:31.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.678+0000 7f0658200640 1 -- 172.21.15.112:0/2296743607 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f065010c330 con 0x7f06500fb4a0 2024-09-16T00:35:31.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.678+0000 7f06427fc640 1 -- 172.21.15.112:0/2296743607 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f064400da90 con 0x7f06500fb4a0 2024-09-16T00:35:31.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.678+0000 7f0658200640 1 -- 172.21.15.112:0/2296743607 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f065010c8d0 con 0x7f06500fb4a0 2024-09-16T00:35:31.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.679+0000 7f0658200640 1 -- 172.21.15.112:0/2296743607 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06500fcbe0 con 0x7f06500fb4a0 2024-09-16T00:35:31.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.679+0000 7f06427fc640 1 -- 172.21.15.112:0/2296743607 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f064400dc30 con 0x7f06500fb4a0 2024-09-16T00:35:31.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.680+0000 7f06427fc640 1 --2- 172.21.15.112:0/2296743607 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f063c06e530 0x7f063c0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:31.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.680+0000 7f06427fc640 1 -- 172.21.15.112:0/2296743607 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f064408dbe0 con 0x7f06500fb4a0 2024-09-16T00:35:31.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.680+0000 7f065579b640 1 --2- 172.21.15.112:0/2296743607 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f063c06e530 0x7f063c0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:31.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.680+0000 7f065579b640 1 --2- 172.21.15.112:0/2296743607 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f063c06e530 0x7f063c0709e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f064c00b020 tx=0x7f064c009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:31.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.684+0000 7f06427fc640 1 -- 172.21.15.112:0/2296743607 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f0644056b30 con 0x7f06500fb4a0 2024-09-16T00:35:31.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.889+0000 7f0658200640 1 -- 172.21.15.112:0/2296743607 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f065004b720 con 0x7f06500fb4a0 2024-09-16T00:35:31.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.889+0000 7f06427fc640 1 -- 172.21.15.112:0/2296743607 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1914 (secure 0 0 0) 0x7f064405a1f0 con 0x7f06500fb4a0 2024-09-16T00:35:31.890 INFO:teuthology.orchestra.run.smithi112.stdout:e34 2024-09-16T00:35:31.890 INFO:teuthology.orchestra.run.smithi112.stdout:btime 2024-09-16T00:34:26:619254+0000 2024-09-16T00:35:31.890 INFO:teuthology.orchestra.run.smithi112.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T00:35:31.890 INFO:teuthology.orchestra.run.smithi112.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:35:31.890 INFO:teuthology.orchestra.run.smithi112.stdout:legacy client fscid: 1 2024-09-16T00:35:31.890 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:35:31.890 INFO:teuthology.orchestra.run.smithi112.stdout:Filesystem 'cephfs' (1) 2024-09-16T00:35:31.890 INFO:teuthology.orchestra.run.smithi112.stdout:fs_name cephfs 2024-09-16T00:35:31.891 INFO:teuthology.orchestra.run.smithi112.stdout:epoch 25 2024-09-16T00:35:31.891 INFO:teuthology.orchestra.run.smithi112.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-16T00:35:31.891 INFO:teuthology.orchestra.run.smithi112.stdout:created 2024-09-16T00:23:22.328199+0000 2024-09-16T00:35:31.891 INFO:teuthology.orchestra.run.smithi112.stdout:modified 2024-09-16T00:33:29.012991+0000 2024-09-16T00:35:31.891 INFO:teuthology.orchestra.run.smithi112.stdout:tableserver 0 2024-09-16T00:35:31.891 INFO:teuthology.orchestra.run.smithi112.stdout:root 0 2024-09-16T00:35:31.891 INFO:teuthology.orchestra.run.smithi112.stdout:session_timeout 60 2024-09-16T00:35:31.891 INFO:teuthology.orchestra.run.smithi112.stdout:session_autoclose 300 2024-09-16T00:35:31.891 INFO:teuthology.orchestra.run.smithi112.stdout:max_file_size 1099511627776 2024-09-16T00:35:31.891 INFO:teuthology.orchestra.run.smithi112.stdout:max_xattr_size 65536 2024-09-16T00:35:31.892 INFO:teuthology.orchestra.run.smithi112.stdout:required_client_features {} 2024-09-16T00:35:31.892 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure 0 2024-09-16T00:35:31.892 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure_osd_epoch 82 2024-09-16T00:35:31.892 INFO:teuthology.orchestra.run.smithi112.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:35:31.892 INFO:teuthology.orchestra.run.smithi112.stdout:max_mds 2 2024-09-16T00:35:31.892 INFO:teuthology.orchestra.run.smithi112.stdout:in 0,1 2024-09-16T00:35:31.892 INFO:teuthology.orchestra.run.smithi112.stdout:up {} 2024-09-16T00:35:31.892 INFO:teuthology.orchestra.run.smithi112.stdout:failed 0,1 2024-09-16T00:35:31.892 INFO:teuthology.orchestra.run.smithi112.stdout:damaged 2024-09-16T00:35:31.892 INFO:teuthology.orchestra.run.smithi112.stdout:stopped 2024-09-16T00:35:31.892 INFO:teuthology.orchestra.run.smithi112.stdout:data_pools [3] 2024-09-16T00:35:31.893 INFO:teuthology.orchestra.run.smithi112.stdout:metadata_pool 2 2024-09-16T00:35:31.893 INFO:teuthology.orchestra.run.smithi112.stdout:inline_data enabled 2024-09-16T00:35:31.893 INFO:teuthology.orchestra.run.smithi112.stdout:balancer 2024-09-16T00:35:31.893 INFO:teuthology.orchestra.run.smithi112.stdout:bal_rank_mask -1 2024-09-16T00:35:31.893 INFO:teuthology.orchestra.run.smithi112.stdout:standby_count_wanted 1 2024-09-16T00:35:31.893 INFO:teuthology.orchestra.run.smithi112.stdout:qdb_cluster leader: 0 members: 2024-09-16T00:35:31.893 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:35:31.893 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:35:31.894 INFO:teuthology.orchestra.run.smithi112.stdout:Standby daemons: 2024-09-16T00:35:31.894 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:35:31.894 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.ithwzj{-1:34392} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.112:6826/476229221,v1:172.21.15.112:6827/476229221] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:35:31.894 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.hxfwkh{-1:34404} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.112:6828/2356899876,v1:172.21.15.112:6829/2356899876] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:35:31.894 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.gawwgw{-1:44343} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.169:6824/1635329957,v1:172.21.15.169:6825/1635329957] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:35:31.894 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.lgeqrj{-1:44347} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.169:6826/184810031,v1:172.21.15.169:6827/184810031] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:35:31.895 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 34 2024-09-16T00:35:31.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.891+0000 7f0658200640 1 -- 172.21.15.112:0/2296743607 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f063c06e530 msgr2=0x7f063c0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:31.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.891+0000 7f0658200640 1 --2- 172.21.15.112:0/2296743607 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f063c06e530 0x7f063c0709e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f064c00b020 tx=0x7f064c009da0 comp rx=0 tx=0).stop 2024-09-16T00:35:31.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.891+0000 7f0658200640 1 -- 172.21.15.112:0/2296743607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06500fb4a0 msgr2=0x7f06501120d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:31.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.891+0000 7f0658200640 1 --2- 172.21.15.112:0/2296743607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06500fb4a0 0x7f06501120d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0644002a50 tx=0x7f0644004600 comp rx=0 tx=0).stop 2024-09-16T00:35:31.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.891+0000 7f0658200640 1 -- 172.21.15.112:0/2296743607 shutdown_connections 2024-09-16T00:35:31.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.891+0000 7f0658200640 1 --2- 172.21.15.112:0/2296743607 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f063c06e530 0x7f063c0709e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:31.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.891+0000 7f0658200640 1 --2- 172.21.15.112:0/2296743607 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f06501014e0 0x7f0650112610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:31.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.891+0000 7f0658200640 1 --2- 172.21.15.112:0/2296743607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06500fb4a0 0x7f06501120d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:31.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.891+0000 7f0658200640 1 -- 172.21.15.112:0/2296743607 >> 172.21.15.112:0/2296743607 conn(0x7f06500f7060 msgr2=0x7f06500f8910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:31.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.892+0000 7f0658200640 1 -- 172.21.15.112:0/2296743607 shutdown_connections 2024-09-16T00:35:31.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.892+0000 7f0658200640 1 -- 172.21.15.112:0/2296743607 wait complete. 2024-09-16T00:35:31.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.970+0000 7fce5cc90640 1 -- 172.21.15.112:0/1917697678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce58101590 msgr2=0x7fce580fac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:31.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.970+0000 7fce5cc90640 1 --2- 172.21.15.112:0/1917697678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce58101590 0x7fce580fac30 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fce400049c0 tx=0x7fce4000aeb0 comp rx=0 tx=0).stop 2024-09-16T00:35:31.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.971+0000 7fce5cc90640 1 -- 172.21.15.112:0/1917697678 shutdown_connections 2024-09-16T00:35:31.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.971+0000 7fce5cc90640 1 --2- 172.21.15.112:0/1917697678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce58101590 0x7fce580fac30 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:31.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.971+0000 7fce5cc90640 1 --2- 172.21.15.112:0/1917697678 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce58100bf0 0x7fce58100fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:31.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.971+0000 7fce5cc90640 1 -- 172.21.15.112:0/1917697678 >> 172.21.15.112:0/1917697678 conn(0x7fce58071ac0 msgr2=0x7fce58071ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:31.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.971+0000 7fce5cc90640 1 -- 172.21.15.112:0/1917697678 shutdown_connections 2024-09-16T00:35:31.973 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:35:31 smithi112 ceph-mon[97750]: pgmap v233: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:35:31.973 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:35:31 smithi112 ceph-mon[97750]: from='client.34492 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:35:31.973 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:35:31 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:35:31.973 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:35:31 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/1415359184' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:35:31.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.971+0000 7fce5cc90640 1 -- 172.21.15.112:0/1917697678 wait complete. 2024-09-16T00:35:31.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.971+0000 7fce5cc90640 1 Processor -- start 2024-09-16T00:35:31.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.971+0000 7fce5cc90640 1 -- start start 2024-09-16T00:35:31.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.972+0000 7fce5cc90640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce58100bf0 0x7fce58115430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:31.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.972+0000 7fce5cc90640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce58101590 0x7fce58115970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:31.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.972+0000 7fce5cc90640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce58115f70 con 0x7fce58101590 2024-09-16T00:35:31.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.972+0000 7fce5cc90640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce581160e0 con 0x7fce58100bf0 2024-09-16T00:35:31.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.972+0000 7fce55d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce58101590 0x7fce58115970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:31.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.972+0000 7fce55d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce58101590 0x7fce58115970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56178/0 (socket says 172.21.15.112:56178) 2024-09-16T00:35:31.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.972+0000 7fce5659c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce58100bf0 0x7fce58115430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:31.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.972+0000 7fce55d9b640 1 -- 172.21.15.112:0/648118869 learned_addr learned my addr 172.21.15.112:0/648118869 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:35:31.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.972+0000 7fce55d9b640 1 -- 172.21.15.112:0/648118869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce58100bf0 msgr2=0x7fce58115430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:31.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.972+0000 7fce55d9b640 1 --2- 172.21.15.112:0/648118869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce58100bf0 0x7fce58115430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:31.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.972+0000 7fce55d9b640 1 -- 172.21.15.112:0/648118869 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce40005040 con 0x7fce58101590 2024-09-16T00:35:31.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.972+0000 7fce55d9b640 1 --2- 172.21.15.112:0/648118869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce58101590 0x7fce58115970 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fce40000c00 tx=0x7fce4000aeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:31.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.973+0000 7fce4effd640 1 -- 172.21.15.112:0/648118869 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce40033920 con 0x7fce58101590 2024-09-16T00:35:31.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.973+0000 7fce4effd640 1 -- 172.21.15.112:0/648118869 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fce40034420 con 0x7fce58101590 2024-09-16T00:35:31.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.973+0000 7fce5cc90640 1 -- 172.21.15.112:0/648118869 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce5810f500 con 0x7fce58101590 2024-09-16T00:35:31.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.973+0000 7fce4effd640 1 -- 172.21.15.112:0/648118869 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce4003c600 con 0x7fce58101590 2024-09-16T00:35:31.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.973+0000 7fce5cc90640 1 -- 172.21.15.112:0/648118869 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce5810faa0 con 0x7fce58101590 2024-09-16T00:35:31.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.974+0000 7fce5cc90640 1 -- 172.21.15.112:0/648118869 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce580fc360 con 0x7fce58101590 2024-09-16T00:35:31.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.975+0000 7fce4effd640 1 -- 172.21.15.112:0/648118869 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fce40031020 con 0x7fce58101590 2024-09-16T00:35:31.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.975+0000 7fce4effd640 1 --2- 172.21.15.112:0/648118869 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fce4406e530 0x7fce440709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:31.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.975+0000 7fce4effd640 1 -- 172.21.15.112:0/648118869 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7fce400aea80 con 0x7fce58101590 2024-09-16T00:35:31.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.975+0000 7fce5659c640 1 --2- 172.21.15.112:0/648118869 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fce4406e530 0x7fce440709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:31.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.976+0000 7fce5659c640 1 --2- 172.21.15.112:0/648118869 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fce4406e530 0x7fce440709e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fce48007fd0 tx=0x7fce48009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:31.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:31.981+0000 7fce4effd640 1 -- 172.21.15.112:0/648118869 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fce40077b10 con 0x7fce58101590 2024-09-16T00:35:32.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:35:31 smithi169 ceph-mon[77626]: pgmap v233: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:35:32.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:35:31 smithi169 ceph-mon[77626]: from='client.34492 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:35:32.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:35:31 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:35:32.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:35:31 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/1415359184' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:35:32.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:32.161+0000 7fce5cc90640 1 -- 172.21.15.112:0/648118869 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fce58107380 con 0x7fce4406e530 2024-09-16T00:35:32.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:32.162+0000 7fce4effd640 1 -- 172.21.15.112:0/648118869 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fce58107380 con 0x7fce4406e530 2024-09-16T00:35:32.162 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:35:32.162 INFO:teuthology.orchestra.run.smithi112.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80", 2024-09-16T00:35:32.162 INFO:teuthology.orchestra.run.smithi112.stdout: "in_progress": true, 2024-09-16T00:35:32.162 INFO:teuthology.orchestra.run.smithi112.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T00:35:32.162 INFO:teuthology.orchestra.run.smithi112.stdout: "services_complete": [ 2024-09-16T00:35:32.162 INFO:teuthology.orchestra.run.smithi112.stdout: "mds", 2024-09-16T00:35:32.163 INFO:teuthology.orchestra.run.smithi112.stdout: "osd", 2024-09-16T00:35:32.163 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr", 2024-09-16T00:35:32.163 INFO:teuthology.orchestra.run.smithi112.stdout: "mon", 2024-09-16T00:35:32.163 INFO:teuthology.orchestra.run.smithi112.stdout: "crash" 2024-09-16T00:35:32.163 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-16T00:35:32.163 INFO:teuthology.orchestra.run.smithi112.stdout: "progress": "16/21 daemons upgraded", 2024-09-16T00:35:32.163 INFO:teuthology.orchestra.run.smithi112.stdout: "message": "Currently upgrading mds daemons", 2024-09-16T00:35:32.163 INFO:teuthology.orchestra.run.smithi112.stdout: "is_paused": false 2024-09-16T00:35:32.163 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:35:32.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:32.164+0000 7fce5cc90640 1 -- 172.21.15.112:0/648118869 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fce4406e530 msgr2=0x7fce440709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:32.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:32.164+0000 7fce5cc90640 1 --2- 172.21.15.112:0/648118869 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fce4406e530 0x7fce440709e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fce48007fd0 tx=0x7fce48009040 comp rx=0 tx=0).stop 2024-09-16T00:35:32.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:32.164+0000 7fce5cc90640 1 -- 172.21.15.112:0/648118869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce58101590 msgr2=0x7fce58115970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:32.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:32.164+0000 7fce5cc90640 1 --2- 172.21.15.112:0/648118869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce58101590 0x7fce58115970 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fce40000c00 tx=0x7fce4000aeb0 comp rx=0 tx=0).stop 2024-09-16T00:35:32.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:32.164+0000 7fce5cc90640 1 -- 172.21.15.112:0/648118869 shutdown_connections 2024-09-16T00:35:32.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:32.164+0000 7fce5cc90640 1 --2- 172.21.15.112:0/648118869 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fce4406e530 0x7fce440709e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:32.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:32.164+0000 7fce5cc90640 1 --2- 172.21.15.112:0/648118869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce58101590 0x7fce58115970 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:32.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:32.164+0000 7fce5cc90640 1 --2- 172.21.15.112:0/648118869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce58100bf0 0x7fce58115430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:32.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:32.164+0000 7fce5cc90640 1 -- 172.21.15.112:0/648118869 >> 172.21.15.112:0/648118869 conn(0x7fce58071ac0 msgr2=0x7fce580fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:32.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:32.164+0000 7fce5cc90640 1 -- 172.21.15.112:0/648118869 shutdown_connections 2024-09-16T00:35:32.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:32.164+0000 7fce5cc90640 1 -- 172.21.15.112:0/648118869 wait complete. 2024-09-16T00:35:32.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:32.242+0000 7f28f7be3640 1 -- 172.21.15.112:0/1495758154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28f00fb810 msgr2=0x7f28f00fbbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:32.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:32.242+0000 7f28f7be3640 1 --2- 172.21.15.112:0/1495758154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28f00fb810 0x7f28f00fbbe0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f28e00048d0 tx=0x7f28e0009da0 comp rx=0 tx=0).stop 2024-09-16T00:35:32.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:32.242+0000 7f28f7be3640 1 -- 172.21.15.112:0/1495758154 shutdown_connections 2024-09-16T00:35:32.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:32.242+0000 7f28f7be3640 1 --2- 172.21.15.112:0/1495758154 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28f00fcf80 0x7f28f00fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:32.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:32.242+0000 7f28f7be3640 1 --2- 172.21.15.112:0/1495758154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28f00fb810 0x7f28f00fbbe0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:32.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:32.242+0000 7f28f7be3640 1 -- 172.21.15.112:0/1495758154 >> 172.21.15.112:0/1495758154 conn(0x7f28f00723b0 msgr2=0x7f28f00727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:32.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:32.242+0000 7f28f7be3640 1 -- 172.21.15.112:0/1495758154 shutdown_connections 2024-09-16T00:35:32.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:32.242+0000 7f28f7be3640 1 -- 172.21.15.112:0/1495758154 wait complete. 2024-09-16T00:35:32.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:32.242+0000 7f28f7be3640 1 Processor -- start 2024-09-16T00:35:32.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:32.242+0000 7f28f7be3640 1 -- start start 2024-09-16T00:35:32.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:32.242+0000 7f28f7be3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28f00fb810 0x7f28f01120f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:32.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:32.242+0000 7f28f7be3640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28f00fcf80 0x7f28f0112630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:32.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:32.242+0000 7f28f7be3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f0112c30 con 0x7f28f00fb810 2024-09-16T00:35:32.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:32.242+0000 7f28f7be3640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f010c070 con 0x7f28f00fcf80 2024-09-16T00:35:32.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:32.243+0000 7f28f597f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28f00fb810 0x7f28f01120f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:32.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:32.243+0000 7f28f597f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28f00fb810 0x7f28f01120f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58970/0 (socket says 172.21.15.112:58970) 2024-09-16T00:35:32.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:32.243+0000 7f28f597f640 1 -- 172.21.15.112:0/789860567 learned_addr learned my addr 172.21.15.112:0/789860567 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:35:32.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:32.243+0000 7f28f517e640 1 --2- 172.21.15.112:0/789860567 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28f00fcf80 0x7f28f0112630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:32.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:32.243+0000 7f28f597f640 1 -- 172.21.15.112:0/789860567 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28f00fcf80 msgr2=0x7f28f0112630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:32.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:32.243+0000 7f28f597f640 1 --2- 172.21.15.112:0/789860567 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28f00fcf80 0x7f28f0112630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:32.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:32.243+0000 7f28f597f640 1 -- 172.21.15.112:0/789860567 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28e0005040 con 0x7f28f00fb810 2024-09-16T00:35:32.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:32.243+0000 7f28f597f640 1 --2- 172.21.15.112:0/789860567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28f00fb810 0x7f28f01120f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f28e0000c00 tx=0x7f28e0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:32.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:32.243+0000 7f28e67fc640 1 -- 172.21.15.112:0/789860567 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28e0038070 con 0x7f28f00fb810 2024-09-16T00:35:32.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:32.243+0000 7f28e67fc640 1 -- 172.21.15.112:0/789860567 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f28e002ade0 con 0x7f28f00fb810 2024-09-16T00:35:32.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:32.243+0000 7f28f7be3640 1 -- 172.21.15.112:0/789860567 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28f010c2f0 con 0x7f28f00fb810 2024-09-16T00:35:32.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:32.243+0000 7f28e67fc640 1 -- 172.21.15.112:0/789860567 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28e002c7e0 con 0x7f28f00fb810 2024-09-16T00:35:32.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:32.243+0000 7f28f7be3640 1 -- 172.21.15.112:0/789860567 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28f010c830 con 0x7f28f00fb810 2024-09-16T00:35:32.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:32.245+0000 7f28f7be3640 1 -- 172.21.15.112:0/789860567 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28f00fcbc0 con 0x7f28f00fb810 2024-09-16T00:35:32.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:32.245+0000 7f28e67fc640 1 -- 172.21.15.112:0/789860567 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f28e0030020 con 0x7f28f00fb810 2024-09-16T00:35:32.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:32.246+0000 7f28e67fc640 1 --2- 172.21.15.112:0/789860567 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f28dc06e530 0x7f28dc0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:32.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:32.246+0000 7f28e67fc640 1 -- 172.21.15.112:0/789860567 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f28e00adf70 con 0x7f28f00fb810 2024-09-16T00:35:32.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:32.246+0000 7f28f517e640 1 --2- 172.21.15.112:0/789860567 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f28dc06e530 0x7f28dc0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:32.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:32.246+0000 7f28f517e640 1 --2- 172.21.15.112:0/789860567 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f28dc06e530 0x7f28dc0709e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f28f010d9b0 tx=0x7f28ec002040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:32.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:32.250+0000 7f28e67fc640 1 -- 172.21.15.112:0/789860567 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f28e00e31b0 con 0x7f28f00fb810 2024-09-16T00:35:32.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:32.481+0000 7f28f7be3640 1 -- 172.21.15.112:0/789860567 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f28f004b780 con 0x7f28f00fb810 2024-09-16T00:35:32.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:32.481+0000 7f28e67fc640 1 -- 172.21.15.112:0/789860567 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f28e0076f90 con 0x7f28f00fb810 2024-09-16T00:35:32.482 INFO:teuthology.orchestra.run.smithi112.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-16T00:35:32.482 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-16T00:35:32.482 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs is degraded 2024-09-16T00:35:32.482 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T00:35:32.482 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T00:35:32.482 INFO:teuthology.orchestra.run.smithi112.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-16T00:35:32.482 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-16T00:35:32.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:32.483+0000 7f28f7be3640 1 -- 172.21.15.112:0/789860567 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f28dc06e530 msgr2=0x7f28dc0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:32.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:32.483+0000 7f28f7be3640 1 --2- 172.21.15.112:0/789860567 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f28dc06e530 0x7f28dc0709e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f28f010d9b0 tx=0x7f28ec002040 comp rx=0 tx=0).stop 2024-09-16T00:35:32.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:32.483+0000 7f28f7be3640 1 -- 172.21.15.112:0/789860567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28f00fb810 msgr2=0x7f28f01120f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:32.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:32.483+0000 7f28f7be3640 1 --2- 172.21.15.112:0/789860567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28f00fb810 0x7f28f01120f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f28e0000c00 tx=0x7f28e0009da0 comp rx=0 tx=0).stop 2024-09-16T00:35:32.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:32.483+0000 7f28f7be3640 1 -- 172.21.15.112:0/789860567 shutdown_connections 2024-09-16T00:35:32.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:32.483+0000 7f28f7be3640 1 --2- 172.21.15.112:0/789860567 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f28dc06e530 0x7f28dc0709e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:32.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:32.483+0000 7f28f7be3640 1 --2- 172.21.15.112:0/789860567 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28f00fcf80 0x7f28f0112630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:32.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:32.483+0000 7f28f7be3640 1 --2- 172.21.15.112:0/789860567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28f00fb810 0x7f28f01120f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:32.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:32.483+0000 7f28f7be3640 1 -- 172.21.15.112:0/789860567 >> 172.21.15.112:0/789860567 conn(0x7f28f00723b0 msgr2=0x7f28f00fa480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:32.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:32.483+0000 7f28f7be3640 1 -- 172.21.15.112:0/789860567 shutdown_connections 2024-09-16T00:35:32.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:35:32.483+0000 7f28f7be3640 1 -- 172.21.15.112:0/789860567 wait complete. 2024-09-16T00:35:33.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:35:32 smithi169 ceph-mon[77626]: from='client.34496 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:35:33.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:35:32 smithi169 ceph-mon[77626]: from='client.34500 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:35:33.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:35:32 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/2296743607' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:35:33.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:35:32 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/789860567' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:35:33.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:35:32 smithi112 ceph-mon[97750]: from='client.34496 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:35:33.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:35:32 smithi112 ceph-mon[97750]: from='client.34500 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:35:33.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:35:32 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/2296743607' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:35:33.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:35:32 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/789860567' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:35:34.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:35:33 smithi169 ceph-mon[77626]: from='client.34512 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:35:34.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:35:33 smithi169 ceph-mon[77626]: pgmap v234: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:35:34.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:35:33 smithi112 ceph-mon[97750]: from='client.34512 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:35:34.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:35:33 smithi112 ceph-mon[97750]: pgmap v234: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:35:36.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:35:35 smithi169 ceph-mon[77626]: pgmap v235: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:35:36.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:35:35 smithi112 ceph-mon[97750]: pgmap v235: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:35:38.028 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:35:37 smithi169 ceph-mon[77626]: pgmap v236: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:35:38.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:35:37 smithi112 ceph-mon[97750]: pgmap v236: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:35:40.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:35:39 smithi169 ceph-mon[77626]: pgmap v237: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:35:40.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:35:39 smithi112 ceph-mon[97750]: pgmap v237: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:35:42.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:35:41 smithi169 ceph-mon[77626]: pgmap v238: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:35:42.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:35:41 smithi112 ceph-mon[97750]: pgmap v238: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:35:44.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:35:43 smithi169 ceph-mon[77626]: pgmap v239: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:35:44.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:35:43 smithi112 ceph-mon[97750]: pgmap v239: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:35:46.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:35:45 smithi169 ceph-mon[77626]: pgmap v240: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:35:46.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:35:45 smithi112 ceph-mon[97750]: pgmap v240: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:35:47.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:35:46 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:35:47.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:35:46 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:35:48.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:35:47 smithi169 ceph-mon[77626]: pgmap v241: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:35:48.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:35:47 smithi112 ceph-mon[97750]: pgmap v241: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:35:50.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:35:49 smithi169 ceph-mon[77626]: pgmap v242: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:35:50.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:35:49 smithi112 ceph-mon[97750]: pgmap v242: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:35:52.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:35:51 smithi169 ceph-mon[77626]: pgmap v243: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:35:52.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:35:51 smithi112 ceph-mon[97750]: pgmap v243: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:35:54.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:35:53 smithi169 ceph-mon[77626]: pgmap v244: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:35:54.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:35:53 smithi112 ceph-mon[97750]: pgmap v244: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:35:56.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:35:55 smithi169 ceph-mon[77626]: pgmap v245: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:35:56.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:35:55 smithi112 ceph-mon[97750]: pgmap v245: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:35:58.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:35:57 smithi169 ceph-mon[77626]: pgmap v246: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:35:58.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:35:57 smithi112 ceph-mon[97750]: pgmap v246: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:36:00.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:35:59 smithi169 ceph-mon[77626]: pgmap v247: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:36:00.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:35:59 smithi112 ceph-mon[97750]: pgmap v247: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:36:02.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:36:01 smithi169 ceph-mon[77626]: pgmap v248: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:36:02.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:36:01 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:36:02.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:36:01 smithi112 ceph-mon[97750]: pgmap v248: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:36:02.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:36:01 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:36:02.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.575+0000 7fd6fde59640 1 -- 172.21.15.112:0/3772604259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6f8102550 msgr2=0x7fd6f8102920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:02.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.575+0000 7fd6fde59640 1 --2- 172.21.15.112:0/3772604259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6f8102550 0x7fd6f8102920 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd6e80048d0 tx=0x7fd6e8009da0 comp rx=0 tx=0).stop 2024-09-16T00:36:02.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.575+0000 7fd6fde59640 1 -- 172.21.15.112:0/3772604259 shutdown_connections 2024-09-16T00:36:02.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.575+0000 7fd6fde59640 1 --2- 172.21.15.112:0/3772604259 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6f8102ef0 0x7fd6f80fc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:02.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.575+0000 7fd6fde59640 1 --2- 172.21.15.112:0/3772604259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6f8102550 0x7fd6f8102920 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:02.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.575+0000 7fd6fde59640 1 -- 172.21.15.112:0/3772604259 >> 172.21.15.112:0/3772604259 conn(0x7fd6f80f80f0 msgr2=0x7fd6f80fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:02.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.575+0000 7fd6fde59640 1 -- 172.21.15.112:0/3772604259 shutdown_connections 2024-09-16T00:36:02.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.575+0000 7fd6fde59640 1 -- 172.21.15.112:0/3772604259 wait complete. 2024-09-16T00:36:02.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.575+0000 7fd6fde59640 1 Processor -- start 2024-09-16T00:36:02.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.576+0000 7fd6fde59640 1 -- start start 2024-09-16T00:36:02.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.576+0000 7fd6fde59640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6f8102550 0x7fd6f8113540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:02.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.576+0000 7fd6fde59640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6f8102ef0 0x7fd6f8113a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:02.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.576+0000 7fd6fde59640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f81140d0 con 0x7fd6f8102550 2024-09-16T00:36:02.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.576+0000 7fd6fde59640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f810d4e0 con 0x7fd6f8102ef0 2024-09-16T00:36:02.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.576+0000 7fd6f77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6f8102550 0x7fd6f8113540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:02.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.576+0000 7fd6f77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6f8102550 0x7fd6f8113540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53010/0 (socket says 172.21.15.112:53010) 2024-09-16T00:36:02.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.576+0000 7fd6f77fe640 1 -- 172.21.15.112:0/3465240382 learned_addr learned my addr 172.21.15.112:0/3465240382 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:36:02.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.576+0000 7fd6f6ffd640 1 --2- 172.21.15.112:0/3465240382 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6f8102ef0 0x7fd6f8113a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:02.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.576+0000 7fd6f77fe640 1 -- 172.21.15.112:0/3465240382 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6f8102ef0 msgr2=0x7fd6f8113a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:02.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.576+0000 7fd6f77fe640 1 --2- 172.21.15.112:0/3465240382 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6f8102ef0 0x7fd6f8113a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:02.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.576+0000 7fd6f77fe640 1 -- 172.21.15.112:0/3465240382 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e8005040 con 0x7fd6f8102550 2024-09-16T00:36:02.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.577+0000 7fd6f77fe640 1 --2- 172.21.15.112:0/3465240382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6f8102550 0x7fd6f8113540 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd6e8000c00 tx=0x7fd6e8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:02.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.577+0000 7fd6dffff640 1 -- 172.21.15.112:0/3465240382 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6e8038070 con 0x7fd6f8102550 2024-09-16T00:36:02.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.577+0000 7fd6dffff640 1 -- 172.21.15.112:0/3465240382 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fd6e802ade0 con 0x7fd6f8102550 2024-09-16T00:36:02.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.577+0000 7fd6fde59640 1 -- 172.21.15.112:0/3465240382 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6f810d760 con 0x7fd6f8102550 2024-09-16T00:36:02.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.577+0000 7fd6dffff640 1 -- 172.21.15.112:0/3465240382 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6e802c7b0 con 0x7fd6f8102550 2024-09-16T00:36:02.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.577+0000 7fd6fde59640 1 -- 172.21.15.112:0/3465240382 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6f810dca0 con 0x7fd6f8102550 2024-09-16T00:36:02.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.578+0000 7fd6fde59640 1 -- 172.21.15.112:0/3465240382 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6f80fdcd0 con 0x7fd6f8102550 2024-09-16T00:36:02.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.578+0000 7fd6dffff640 1 -- 172.21.15.112:0/3465240382 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fd6e8030020 con 0x7fd6f8102550 2024-09-16T00:36:02.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.579+0000 7fd6dffff640 1 --2- 172.21.15.112:0/3465240382 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd6e006e530 0x7fd6e00709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:02.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.579+0000 7fd6dffff640 1 -- 172.21.15.112:0/3465240382 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7fd6e80ae640 con 0x7fd6f8102550 2024-09-16T00:36:02.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.579+0000 7fd6f6ffd640 1 --2- 172.21.15.112:0/3465240382 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd6e006e530 0x7fd6e00709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:02.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.579+0000 7fd6f6ffd640 1 --2- 172.21.15.112:0/3465240382 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd6e006e530 0x7fd6e00709e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd6f810e7c0 tx=0x7fd6ec00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:02.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.583+0000 7fd6dffff640 1 -- 172.21.15.112:0/3465240382 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fd6e8077630 con 0x7fd6f8102550 2024-09-16T00:36:02.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.768+0000 7fd6fde59640 1 -- 172.21.15.112:0/3465240382 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd6f804b720 con 0x7fd6e006e530 2024-09-16T00:36:02.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.770+0000 7fd6dffff640 1 -- 172.21.15.112:0/3465240382 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fd6f804b720 con 0x7fd6e006e530 2024-09-16T00:36:02.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.771+0000 7fd6fde59640 1 -- 172.21.15.112:0/3465240382 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd6e006e530 msgr2=0x7fd6e00709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:02.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.771+0000 7fd6fde59640 1 --2- 172.21.15.112:0/3465240382 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd6e006e530 0x7fd6e00709e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd6f810e7c0 tx=0x7fd6ec00b040 comp rx=0 tx=0).stop 2024-09-16T00:36:02.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.771+0000 7fd6fde59640 1 -- 172.21.15.112:0/3465240382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6f8102550 msgr2=0x7fd6f8113540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:02.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.771+0000 7fd6fde59640 1 --2- 172.21.15.112:0/3465240382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6f8102550 0x7fd6f8113540 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd6e8000c00 tx=0x7fd6e8009da0 comp rx=0 tx=0).stop 2024-09-16T00:36:02.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.772+0000 7fd6fde59640 1 -- 172.21.15.112:0/3465240382 shutdown_connections 2024-09-16T00:36:02.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.772+0000 7fd6fde59640 1 --2- 172.21.15.112:0/3465240382 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd6e006e530 0x7fd6e00709e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:02.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.772+0000 7fd6fde59640 1 --2- 172.21.15.112:0/3465240382 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6f8102ef0 0x7fd6f8113a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:02.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.772+0000 7fd6fde59640 1 --2- 172.21.15.112:0/3465240382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6f8102550 0x7fd6f8113540 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:02.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.772+0000 7fd6fde59640 1 -- 172.21.15.112:0/3465240382 >> 172.21.15.112:0/3465240382 conn(0x7fd6f80f80f0 msgr2=0x7fd6f80f9160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:02.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.772+0000 7fd6fde59640 1 -- 172.21.15.112:0/3465240382 shutdown_connections 2024-09-16T00:36:02.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.772+0000 7fd6fde59640 1 -- 172.21.15.112:0/3465240382 wait complete. 2024-09-16T00:36:02.781 INFO:teuthology.orchestra.run.smithi112.stdout:true 2024-09-16T00:36:02.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.851+0000 7fe762a87640 1 -- 172.21.15.112:0/1437876876 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe75c0fb810 msgr2=0x7fe75c0fbbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:02.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.851+0000 7fe762a87640 1 --2- 172.21.15.112:0/1437876876 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe75c0fb810 0x7fe75c0fbbe0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe7500048d0 tx=0x7fe750009da0 comp rx=0 tx=0).stop 2024-09-16T00:36:02.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.852+0000 7fe762a87640 1 -- 172.21.15.112:0/1437876876 shutdown_connections 2024-09-16T00:36:02.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.852+0000 7fe762a87640 1 --2- 172.21.15.112:0/1437876876 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe75c0fcf80 0x7fe75c0fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:02.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.852+0000 7fe762a87640 1 --2- 172.21.15.112:0/1437876876 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe75c0fb810 0x7fe75c0fbbe0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:02.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.852+0000 7fe762a87640 1 -- 172.21.15.112:0/1437876876 >> 172.21.15.112:0/1437876876 conn(0x7fe75c0723b0 msgr2=0x7fe75c0727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:02.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.852+0000 7fe762a87640 1 -- 172.21.15.112:0/1437876876 shutdown_connections 2024-09-16T00:36:02.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.852+0000 7fe762a87640 1 -- 172.21.15.112:0/1437876876 wait complete. 2024-09-16T00:36:02.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.852+0000 7fe762a87640 1 Processor -- start 2024-09-16T00:36:02.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.852+0000 7fe762a87640 1 -- start start 2024-09-16T00:36:02.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.853+0000 7fe762a87640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe75c0fb810 0x7fe75c10e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:02.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.853+0000 7fe762a87640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe75c0fcf80 0x7fe75c10e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:02.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.853+0000 7fe762a87640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe75c10ed40 con 0x7fe75c0fb810 2024-09-16T00:36:02.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.853+0000 7fe762a87640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe75c10eeb0 con 0x7fe75c0fcf80 2024-09-16T00:36:02.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.853+0000 7fe760823640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe75c0fb810 0x7fe75c10e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:02.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.853+0000 7fe760823640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe75c0fb810 0x7fe75c10e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53018/0 (socket says 172.21.15.112:53018) 2024-09-16T00:36:02.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.853+0000 7fe760823640 1 -- 172.21.15.112:0/1788183891 learned_addr learned my addr 172.21.15.112:0/1788183891 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:36:02.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.853+0000 7fe75bfff640 1 --2- 172.21.15.112:0/1788183891 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe75c0fcf80 0x7fe75c10e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:02.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.853+0000 7fe760823640 1 -- 172.21.15.112:0/1788183891 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe75c0fcf80 msgr2=0x7fe75c10e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:02.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.853+0000 7fe760823640 1 --2- 172.21.15.112:0/1788183891 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe75c0fcf80 0x7fe75c10e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:02.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.853+0000 7fe760823640 1 -- 172.21.15.112:0/1788183891 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe750005040 con 0x7fe75c0fb810 2024-09-16T00:36:02.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.853+0000 7fe760823640 1 --2- 172.21.15.112:0/1788183891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe75c0fb810 0x7fe75c10e1b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe750000c00 tx=0x7fe750009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:02.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.853+0000 7fe7597fa640 1 -- 172.21.15.112:0/1788183891 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe750038070 con 0x7fe75c0fb810 2024-09-16T00:36:02.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.853+0000 7fe7597fa640 1 -- 172.21.15.112:0/1788183891 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fe75002ade0 con 0x7fe75c0fb810 2024-09-16T00:36:02.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.853+0000 7fe762a87640 1 -- 172.21.15.112:0/1788183891 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe75c117220 con 0x7fe75c0fb810 2024-09-16T00:36:02.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.853+0000 7fe7597fa640 1 -- 172.21.15.112:0/1788183891 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe75002c7e0 con 0x7fe75c0fb810 2024-09-16T00:36:02.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.854+0000 7fe762a87640 1 -- 172.21.15.112:0/1788183891 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe75c117680 con 0x7fe75c0fb810 2024-09-16T00:36:02.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.855+0000 7fe762a87640 1 -- 172.21.15.112:0/1788183891 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe75c0fcbc0 con 0x7fe75c0fb810 2024-09-16T00:36:02.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.855+0000 7fe7597fa640 1 -- 172.21.15.112:0/1788183891 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fe750030020 con 0x7fe75c0fb810 2024-09-16T00:36:02.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.855+0000 7fe7597fa640 1 --2- 172.21.15.112:0/1788183891 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fe74c06e530 0x7fe74c0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:02.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.856+0000 7fe7597fa640 1 -- 172.21.15.112:0/1788183891 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7fe7500adf70 con 0x7fe75c0fb810 2024-09-16T00:36:02.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.856+0000 7fe75bfff640 1 --2- 172.21.15.112:0/1788183891 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fe74c06e530 0x7fe74c0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:02.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.856+0000 7fe75bfff640 1 --2- 172.21.15.112:0/1788183891 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fe74c06e530 0x7fe74c0709e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe75c065980 tx=0x7fe74800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:02.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:02.860+0000 7fe7597fa640 1 -- 172.21.15.112:0/1788183891 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fe7500e31b0 con 0x7fe75c0fb810 2024-09-16T00:36:03.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.044+0000 7fe762a87640 1 -- 172.21.15.112:0/1788183891 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe75c065730 con 0x7fe74c06e530 2024-09-16T00:36:03.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.045+0000 7fe7597fa640 1 -- 172.21.15.112:0/1788183891 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fe75c065730 con 0x7fe74c06e530 2024-09-16T00:36:03.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.047+0000 7fe762a87640 1 -- 172.21.15.112:0/1788183891 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fe74c06e530 msgr2=0x7fe74c0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:03.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.047+0000 7fe762a87640 1 --2- 172.21.15.112:0/1788183891 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fe74c06e530 0x7fe74c0709e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe75c065980 tx=0x7fe74800b040 comp rx=0 tx=0).stop 2024-09-16T00:36:03.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.047+0000 7fe762a87640 1 -- 172.21.15.112:0/1788183891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe75c0fb810 msgr2=0x7fe75c10e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:03.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.047+0000 7fe762a87640 1 --2- 172.21.15.112:0/1788183891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe75c0fb810 0x7fe75c10e1b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe750000c00 tx=0x7fe750009da0 comp rx=0 tx=0).stop 2024-09-16T00:36:03.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.047+0000 7fe762a87640 1 -- 172.21.15.112:0/1788183891 shutdown_connections 2024-09-16T00:36:03.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.047+0000 7fe762a87640 1 --2- 172.21.15.112:0/1788183891 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fe74c06e530 0x7fe74c0709e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:03.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.047+0000 7fe762a87640 1 --2- 172.21.15.112:0/1788183891 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe75c0fcf80 0x7fe75c10e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:03.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.047+0000 7fe762a87640 1 --2- 172.21.15.112:0/1788183891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe75c0fb810 0x7fe75c10e1b0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:03.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.047+0000 7fe762a87640 1 -- 172.21.15.112:0/1788183891 >> 172.21.15.112:0/1788183891 conn(0x7fe75c0723b0 msgr2=0x7fe75c0fa480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:03.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.047+0000 7fe762a87640 1 -- 172.21.15.112:0/1788183891 shutdown_connections 2024-09-16T00:36:03.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.047+0000 7fe762a87640 1 -- 172.21.15.112:0/1788183891 wait complete. 2024-09-16T00:36:03.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.125+0000 7fda44a53640 1 -- 172.21.15.112:0/1174428354 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda40102ef0 msgr2=0x7fda400fc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:03.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.125+0000 7fda44a53640 1 --2- 172.21.15.112:0/1174428354 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda40102ef0 0x7fda400fc5b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fda340048d0 tx=0x7fda34009da0 comp rx=0 tx=0).stop 2024-09-16T00:36:03.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.125+0000 7fda44a53640 1 -- 172.21.15.112:0/1174428354 shutdown_connections 2024-09-16T00:36:03.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.125+0000 7fda44a53640 1 --2- 172.21.15.112:0/1174428354 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda40102ef0 0x7fda400fc5b0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:03.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.125+0000 7fda44a53640 1 --2- 172.21.15.112:0/1174428354 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda40102550 0x7fda40102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:03.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.125+0000 7fda44a53640 1 -- 172.21.15.112:0/1174428354 >> 172.21.15.112:0/1174428354 conn(0x7fda400f80f0 msgr2=0x7fda400fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:03.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.125+0000 7fda44a53640 1 -- 172.21.15.112:0/1174428354 shutdown_connections 2024-09-16T00:36:03.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.125+0000 7fda44a53640 1 -- 172.21.15.112:0/1174428354 wait complete. 2024-09-16T00:36:03.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.126+0000 7fda44a53640 1 Processor -- start 2024-09-16T00:36:03.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.126+0000 7fda44a53640 1 -- start start 2024-09-16T00:36:03.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.126+0000 7fda44a53640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda40102550 0x7fda40113520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:03.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.126+0000 7fda44a53640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda40102ef0 0x7fda40113a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:03.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.126+0000 7fda44a53640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda401140b0 con 0x7fda40102ef0 2024-09-16T00:36:03.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.126+0000 7fda44a53640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda4010d4c0 con 0x7fda40102550 2024-09-16T00:36:03.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.126+0000 7fda3dd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda40102ef0 0x7fda40113a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:03.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.126+0000 7fda3dd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda40102ef0 0x7fda40113a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53026/0 (socket says 172.21.15.112:53026) 2024-09-16T00:36:03.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.126+0000 7fda3dd9b640 1 -- 172.21.15.112:0/2283950269 learned_addr learned my addr 172.21.15.112:0/2283950269 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:36:03.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.126+0000 7fda3e59c640 1 --2- 172.21.15.112:0/2283950269 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda40102550 0x7fda40113520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:03.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.127+0000 7fda3dd9b640 1 -- 172.21.15.112:0/2283950269 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda40102550 msgr2=0x7fda40113520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:03.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.127+0000 7fda3dd9b640 1 --2- 172.21.15.112:0/2283950269 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda40102550 0x7fda40113520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:03.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.127+0000 7fda3dd9b640 1 -- 172.21.15.112:0/2283950269 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda34005040 con 0x7fda40102ef0 2024-09-16T00:36:03.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.127+0000 7fda3dd9b640 1 --2- 172.21.15.112:0/2283950269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda40102ef0 0x7fda40113a60 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fda34003080 tx=0x7fda34009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:03.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.127+0000 7fda2effd640 1 -- 172.21.15.112:0/2283950269 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda34038070 con 0x7fda40102ef0 2024-09-16T00:36:03.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.127+0000 7fda44a53640 1 -- 172.21.15.112:0/2283950269 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda4010d740 con 0x7fda40102ef0 2024-09-16T00:36:03.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.127+0000 7fda2effd640 1 -- 172.21.15.112:0/2283950269 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fda34009760 con 0x7fda40102ef0 2024-09-16T00:36:03.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.127+0000 7fda2effd640 1 -- 172.21.15.112:0/2283950269 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda34033420 con 0x7fda40102ef0 2024-09-16T00:36:03.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.127+0000 7fda44a53640 1 -- 172.21.15.112:0/2283950269 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda4010dc20 con 0x7fda40102ef0 2024-09-16T00:36:03.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.128+0000 7fda44a53640 1 -- 172.21.15.112:0/2283950269 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda400fdcd0 con 0x7fda40102ef0 2024-09-16T00:36:03.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.128+0000 7fda2effd640 1 -- 172.21.15.112:0/2283950269 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fda34030020 con 0x7fda40102ef0 2024-09-16T00:36:03.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.129+0000 7fda2effd640 1 --2- 172.21.15.112:0/2283950269 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fda2806e530 0x7fda280709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:03.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.129+0000 7fda2effd640 1 -- 172.21.15.112:0/2283950269 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7fda340adfb0 con 0x7fda40102ef0 2024-09-16T00:36:03.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.129+0000 7fda3e59c640 1 --2- 172.21.15.112:0/2283950269 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fda2806e530 0x7fda280709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:03.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.129+0000 7fda3e59c640 1 --2- 172.21.15.112:0/2283950269 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fda2806e530 0x7fda280709e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fda300041d0 tx=0x7fda30004330 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:03.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.133+0000 7fda2effd640 1 -- 172.21.15.112:0/2283950269 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fda34076f20 con 0x7fda40102ef0 2024-09-16T00:36:03.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.309+0000 7fda44a53640 1 -- 172.21.15.112:0/2283950269 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fda401095f0 con 0x7fda2806e530 2024-09-16T00:36:03.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.318+0000 7fda2effd640 1 -- 172.21.15.112:0/2283950269 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fda401095f0 con 0x7fda2806e530 2024-09-16T00:36:03.318 INFO:teuthology.orchestra.run.smithi112.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T00:36:03.318 INFO:teuthology.orchestra.run.smithi112.stdout:alertmanager.smithi112 smithi112 *:9093,9094 running (9m) 2m ago 17m 24.7M - 0.25.0 c8568f914cd2 d85ea0562022 2024-09-16T00:36:03.318 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi112 smithi112 running (5m) 2m ago 18m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c e4fe58f9702a 2024-09-16T00:36:03.319 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi169 smithi169 running (5m) 95s ago 17m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c 3ba2c81d527f 2024-09-16T00:36:03.319 INFO:teuthology.orchestra.run.smithi112.stdout:grafana.smithi112 smithi112 *:3000 running (7m) 2m ago 17m 82.1M - 9.4.12 2bacad6d85d8 51f9d2013da2 2024-09-16T00:36:03.319 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.hxfwkh smithi112 running (2m) 2m ago 12m 14.2M - 19.1.1-326-g8921c8ef f1455bd0219c 1723069902d6 2024-09-16T00:36:03.319 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.ithwzj smithi112 running (2m) 2m ago 12m 12.2M - 19.1.1-326-g8921c8ef f1455bd0219c e7aad5b81db5 2024-09-16T00:36:03.319 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.gawwgw smithi169 running (113s) 95s ago 12m 15.7M - 19.1.1-326-g8921c8ef f1455bd0219c 469be459ee12 2024-09-16T00:36:03.319 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.lgeqrj smithi169 running (97s) 95s ago 12m 13.3M - 19.1.1-326-g8921c8ef f1455bd0219c 31f0fbd2395d 2024-09-16T00:36:03.319 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi112.xrkkbc smithi112 *:8443,9283,8765 running (8m) 2m ago 19m 504M - 19.1.1-326-g8921c8ef f1455bd0219c a981f7fd2810 2024-09-16T00:36:03.319 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi169.mtrobk smithi169 *:8443,9283,8765 running (8m) 95s ago 17m 442M - 19.1.1-326-g8921c8ef f1455bd0219c 4096e5369c2d 2024-09-16T00:36:03.319 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi112 smithi112 running (6m) 2m ago 19m 49.8M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 5aaec033c042 2024-09-16T00:36:03.319 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi169 smithi169 running (6m) 95s ago 17m 38.9M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 7bc609155f3f 2024-09-16T00:36:03.319 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi112 smithi112 *:9100 running (9m) 2m ago 18m 15.5M - 1.5.0 0da6a335fe13 6404d1d601e2 2024-09-16T00:36:03.319 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi169 smithi169 *:9100 running (8m) 95s ago 17m 15.4M - 1.5.0 0da6a335fe13 5505baf702a6 2024-09-16T00:36:03.319 INFO:teuthology.orchestra.run.smithi112.stdout:osd.0 smithi112 running (5m) 2m ago 16m 105M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 53efc9397017 2024-09-16T00:36:03.320 INFO:teuthology.orchestra.run.smithi112.stdout:osd.1 smithi112 running (4m) 2m ago 15m 98.9M 1557M 19.1.1-326-g8921c8ef f1455bd0219c fe6f1a4a2cf4 2024-09-16T00:36:03.320 INFO:teuthology.orchestra.run.smithi112.stdout:osd.2 smithi112 running (4m) 2m ago 15m 102M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 5f7863712057 2024-09-16T00:36:03.320 INFO:teuthology.orchestra.run.smithi112.stdout:osd.3 smithi169 running (3m) 95s ago 14m 92.1M 2581M 19.1.1-326-g8921c8ef f1455bd0219c ee07b5577810 2024-09-16T00:36:03.320 INFO:teuthology.orchestra.run.smithi112.stdout:osd.4 smithi169 running (3m) 95s ago 14m 103M 2581M 19.1.1-326-g8921c8ef f1455bd0219c 769f026a54eb 2024-09-16T00:36:03.320 INFO:teuthology.orchestra.run.smithi112.stdout:osd.5 smithi169 running (2m) 95s ago 14m 125M 2581M 19.1.1-326-g8921c8ef f1455bd0219c a6007c13cf78 2024-09-16T00:36:03.320 INFO:teuthology.orchestra.run.smithi112.stdout:prometheus.smithi112 smithi112 *:9095 running (8m) 2m ago 17m 64.1M - 2.43.0 a07b618ecd1d 7a1d42e60d2b 2024-09-16T00:36:03.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.320+0000 7fda44a53640 1 -- 172.21.15.112:0/2283950269 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fda2806e530 msgr2=0x7fda280709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:03.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.320+0000 7fda44a53640 1 --2- 172.21.15.112:0/2283950269 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fda2806e530 0x7fda280709e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fda300041d0 tx=0x7fda30004330 comp rx=0 tx=0).stop 2024-09-16T00:36:03.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.320+0000 7fda44a53640 1 -- 172.21.15.112:0/2283950269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda40102ef0 msgr2=0x7fda40113a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:03.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.320+0000 7fda44a53640 1 --2- 172.21.15.112:0/2283950269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda40102ef0 0x7fda40113a60 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fda34003080 tx=0x7fda34009da0 comp rx=0 tx=0).stop 2024-09-16T00:36:03.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.320+0000 7fda44a53640 1 -- 172.21.15.112:0/2283950269 shutdown_connections 2024-09-16T00:36:03.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.320+0000 7fda44a53640 1 --2- 172.21.15.112:0/2283950269 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fda2806e530 0x7fda280709e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:03.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.320+0000 7fda44a53640 1 --2- 172.21.15.112:0/2283950269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda40102ef0 0x7fda40113a60 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:03.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.321+0000 7fda44a53640 1 --2- 172.21.15.112:0/2283950269 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda40102550 0x7fda40113520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:03.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.321+0000 7fda44a53640 1 -- 172.21.15.112:0/2283950269 >> 172.21.15.112:0/2283950269 conn(0x7fda400f80f0 msgr2=0x7fda400f9150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:03.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.321+0000 7fda44a53640 1 -- 172.21.15.112:0/2283950269 shutdown_connections 2024-09-16T00:36:03.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.321+0000 7fda44a53640 1 -- 172.21.15.112:0/2283950269 wait complete. 2024-09-16T00:36:03.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.397+0000 7faa6ffae640 1 -- 172.21.15.112:0/881413173 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa680fe450 msgr2=0x7faa680fe820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:03.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.397+0000 7faa6ffae640 1 --2- 172.21.15.112:0/881413173 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa680fe450 0x7faa680fe820 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7faa58009c50 tx=0x7faa58004d10 comp rx=0 tx=0).stop 2024-09-16T00:36:03.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.398+0000 7faa6ffae640 1 -- 172.21.15.112:0/881413173 shutdown_connections 2024-09-16T00:36:03.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.398+0000 7faa6ffae640 1 --2- 172.21.15.112:0/881413173 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa68064fe0 0x7faa68065430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:03.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.398+0000 7faa6ffae640 1 --2- 172.21.15.112:0/881413173 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa680fe450 0x7faa680fe820 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:03.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.398+0000 7faa6ffae640 1 -- 172.21.15.112:0/881413173 >> 172.21.15.112:0/881413173 conn(0x7faa680723b0 msgr2=0x7faa680727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:03.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.398+0000 7faa6ffae640 1 -- 172.21.15.112:0/881413173 shutdown_connections 2024-09-16T00:36:03.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.398+0000 7faa6ffae640 1 -- 172.21.15.112:0/881413173 wait complete. 2024-09-16T00:36:03.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.398+0000 7faa6ffae640 1 Processor -- start 2024-09-16T00:36:03.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.398+0000 7faa6ffae640 1 -- start start 2024-09-16T00:36:03.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.398+0000 7faa6ffae640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa68064fe0 0x7faa681131d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:03.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.398+0000 7faa6ffae640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa680fe450 0x7faa68113710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:03.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.398+0000 7faa6ffae640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa68113d10 con 0x7faa68064fe0 2024-09-16T00:36:03.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.398+0000 7faa6ffae640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa6810d150 con 0x7faa680fe450 2024-09-16T00:36:03.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.399+0000 7faa6dd4a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa68064fe0 0x7faa681131d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:03.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.399+0000 7faa6dd4a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa68064fe0 0x7faa681131d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53058/0 (socket says 172.21.15.112:53058) 2024-09-16T00:36:03.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.399+0000 7faa6dd4a640 1 -- 172.21.15.112:0/3803545012 learned_addr learned my addr 172.21.15.112:0/3803545012 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:36:03.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.399+0000 7faa6d549640 1 --2- 172.21.15.112:0/3803545012 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa680fe450 0x7faa68113710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:03.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.399+0000 7faa6dd4a640 1 -- 172.21.15.112:0/3803545012 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa680fe450 msgr2=0x7faa68113710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:03.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.399+0000 7faa6dd4a640 1 --2- 172.21.15.112:0/3803545012 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa680fe450 0x7faa68113710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:03.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.399+0000 7faa6dd4a640 1 -- 172.21.15.112:0/3803545012 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa580098b0 con 0x7faa68064fe0 2024-09-16T00:36:03.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.399+0000 7faa6dd4a640 1 --2- 172.21.15.112:0/3803545012 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa68064fe0 0x7faa681131d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7faa58004a30 tx=0x7faa58004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:03.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.399+0000 7faa667fc640 1 -- 172.21.15.112:0/3803545012 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa58009690 con 0x7faa68064fe0 2024-09-16T00:36:03.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.399+0000 7faa6ffae640 1 -- 172.21.15.112:0/3803545012 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa6810d3d0 con 0x7faa68064fe0 2024-09-16T00:36:03.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.399+0000 7faa667fc640 1 -- 172.21.15.112:0/3803545012 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7faa58032b10 con 0x7faa68064fe0 2024-09-16T00:36:03.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.399+0000 7faa667fc640 1 -- 172.21.15.112:0/3803545012 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa5803daa0 con 0x7faa68064fe0 2024-09-16T00:36:03.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.399+0000 7faa6ffae640 1 -- 172.21.15.112:0/3803545012 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa6810d970 con 0x7faa68064fe0 2024-09-16T00:36:03.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.400+0000 7faa6ffae640 1 -- 172.21.15.112:0/3803545012 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa680fdce0 con 0x7faa68064fe0 2024-09-16T00:36:03.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.401+0000 7faa667fc640 1 -- 172.21.15.112:0/3803545012 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7faa5802f030 con 0x7faa68064fe0 2024-09-16T00:36:03.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.401+0000 7faa667fc640 1 --2- 172.21.15.112:0/3803545012 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7faa5006e530 0x7faa500709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:03.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.401+0000 7faa667fc640 1 -- 172.21.15.112:0/3803545012 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7faa580af2b0 con 0x7faa68064fe0 2024-09-16T00:36:03.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.401+0000 7faa6d549640 1 --2- 172.21.15.112:0/3803545012 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7faa5006e530 0x7faa500709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:03.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.402+0000 7faa6d549640 1 --2- 172.21.15.112:0/3803545012 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7faa5006e530 0x7faa500709e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7faa6810ea90 tx=0x7faa5c009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:03.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.406+0000 7faa667fc640 1 -- 172.21.15.112:0/3803545012 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7faa580782a0 con 0x7faa68064fe0 2024-09-16T00:36:03.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.640+0000 7faa6ffae640 1 -- 172.21.15.112:0/3803545012 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7faa6804b720 con 0x7faa68064fe0 2024-09-16T00:36:03.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.640+0000 7faa667fc640 1 -- 172.21.15.112:0/3803545012 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7faa5807b960 con 0x7faa68064fe0 2024-09-16T00:36:03.641 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:36:03.641 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": { 2024-09-16T00:36:03.641 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:36:03.641 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:36:03.641 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": { 2024-09-16T00:36:03.641 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:36:03.641 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:36:03.641 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": { 2024-09-16T00:36:03.641 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 6 2024-09-16T00:36:03.641 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:36:03.641 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": { 2024-09-16T00:36:03.642 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 4 2024-09-16T00:36:03.642 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:36:03.642 INFO:teuthology.orchestra.run.smithi112.stdout: "overall": { 2024-09-16T00:36:03.642 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 14 2024-09-16T00:36:03.642 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-16T00:36:03.642 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:36:03.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.642+0000 7faa6ffae640 1 -- 172.21.15.112:0/3803545012 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7faa5006e530 msgr2=0x7faa500709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:03.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.642+0000 7faa6ffae640 1 --2- 172.21.15.112:0/3803545012 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7faa5006e530 0x7faa500709e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7faa6810ea90 tx=0x7faa5c009040 comp rx=0 tx=0).stop 2024-09-16T00:36:03.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.642+0000 7faa6ffae640 1 -- 172.21.15.112:0/3803545012 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa68064fe0 msgr2=0x7faa681131d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:03.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.642+0000 7faa6ffae640 1 --2- 172.21.15.112:0/3803545012 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa68064fe0 0x7faa681131d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7faa58004a30 tx=0x7faa58004d10 comp rx=0 tx=0).stop 2024-09-16T00:36:03.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.642+0000 7faa6ffae640 1 -- 172.21.15.112:0/3803545012 shutdown_connections 2024-09-16T00:36:03.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.642+0000 7faa6ffae640 1 --2- 172.21.15.112:0/3803545012 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7faa5006e530 0x7faa500709e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:03.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.642+0000 7faa6ffae640 1 --2- 172.21.15.112:0/3803545012 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa680fe450 0x7faa68113710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:03.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.642+0000 7faa6ffae640 1 --2- 172.21.15.112:0/3803545012 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa68064fe0 0x7faa681131d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:03.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.642+0000 7faa6ffae640 1 -- 172.21.15.112:0/3803545012 >> 172.21.15.112:0/3803545012 conn(0x7faa680723b0 msgr2=0x7faa680fa290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:03.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.642+0000 7faa6ffae640 1 -- 172.21.15.112:0/3803545012 shutdown_connections 2024-09-16T00:36:03.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.642+0000 7faa6ffae640 1 -- 172.21.15.112:0/3803545012 wait complete. 2024-09-16T00:36:03.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.725+0000 7ffa0e328640 1 -- 172.21.15.112:0/1418263325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa08102550 msgr2=0x7ffa08102920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:03.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.725+0000 7ffa0e328640 1 --2- 172.21.15.112:0/1418263325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa08102550 0x7ffa08102920 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff9f80048d0 tx=0x7ff9f8009da0 comp rx=0 tx=0).stop 2024-09-16T00:36:03.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.725+0000 7ffa0e328640 1 -- 172.21.15.112:0/1418263325 shutdown_connections 2024-09-16T00:36:03.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.725+0000 7ffa0e328640 1 --2- 172.21.15.112:0/1418263325 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa08102ef0 0x7ffa080fc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:03.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.725+0000 7ffa0e328640 1 --2- 172.21.15.112:0/1418263325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa08102550 0x7ffa08102920 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:03.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.725+0000 7ffa0e328640 1 -- 172.21.15.112:0/1418263325 >> 172.21.15.112:0/1418263325 conn(0x7ffa080f80f0 msgr2=0x7ffa080fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:03.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.725+0000 7ffa0e328640 1 -- 172.21.15.112:0/1418263325 shutdown_connections 2024-09-16T00:36:03.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.725+0000 7ffa0e328640 1 -- 172.21.15.112:0/1418263325 wait complete. 2024-09-16T00:36:03.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.726+0000 7ffa0e328640 1 Processor -- start 2024-09-16T00:36:03.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.726+0000 7ffa0e328640 1 -- start start 2024-09-16T00:36:03.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.726+0000 7ffa0e328640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa08102550 0x7ffa0806fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:03.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.726+0000 7ffa0e328640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa08102ef0 0x7ffa0806ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:03.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.726+0000 7ffa0e328640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa08070490 con 0x7ffa08102550 2024-09-16T00:36:03.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.726+0000 7ffa0e328640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa08070600 con 0x7ffa08102ef0 2024-09-16T00:36:03.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.726+0000 7ffa07fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa08102550 0x7ffa0806fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:03.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.726+0000 7ffa07fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa08102550 0x7ffa0806fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53080/0 (socket says 172.21.15.112:53080) 2024-09-16T00:36:03.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.726+0000 7ffa07fff640 1 -- 172.21.15.112:0/140766727 learned_addr learned my addr 172.21.15.112:0/140766727 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:36:03.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.727+0000 7ffa077fe640 1 --2- 172.21.15.112:0/140766727 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa08102ef0 0x7ffa0806ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:03.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.727+0000 7ffa07fff640 1 -- 172.21.15.112:0/140766727 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa08102ef0 msgr2=0x7ffa0806ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:03.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.727+0000 7ffa07fff640 1 --2- 172.21.15.112:0/140766727 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa08102ef0 0x7ffa0806ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:03.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.727+0000 7ffa07fff640 1 -- 172.21.15.112:0/140766727 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9f8005040 con 0x7ffa08102550 2024-09-16T00:36:03.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.727+0000 7ffa077fe640 1 --2- 172.21.15.112:0/140766727 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa08102ef0 0x7ffa0806ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:36:03.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.727+0000 7ffa07fff640 1 --2- 172.21.15.112:0/140766727 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa08102550 0x7ffa0806fa10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff9f80043d0 tx=0x7ff9f8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:03.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.727+0000 7ffa04ff9640 1 -- 172.21.15.112:0/140766727 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9f8038070 con 0x7ffa08102550 2024-09-16T00:36:03.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.727+0000 7ffa04ff9640 1 -- 172.21.15.112:0/140766727 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7ff9f80096e0 con 0x7ffa08102550 2024-09-16T00:36:03.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.727+0000 7ffa0e328640 1 -- 172.21.15.112:0/140766727 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa08070830 con 0x7ffa08102550 2024-09-16T00:36:03.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.727+0000 7ffa04ff9640 1 -- 172.21.15.112:0/140766727 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9f803d420 con 0x7ffa08102550 2024-09-16T00:36:03.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.727+0000 7ffa0e328640 1 -- 172.21.15.112:0/140766727 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa08119b70 con 0x7ffa08102550 2024-09-16T00:36:03.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.729+0000 7ffa0e328640 1 -- 172.21.15.112:0/140766727 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa080fdcd0 con 0x7ffa08102550 2024-09-16T00:36:03.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.729+0000 7ffa04ff9640 1 -- 172.21.15.112:0/140766727 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7ff9f8030020 con 0x7ffa08102550 2024-09-16T00:36:03.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.730+0000 7ffa04ff9640 1 --2- 172.21.15.112:0/140766727 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff9f006e530 0x7ff9f00709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:03.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.730+0000 7ffa04ff9640 1 -- 172.21.15.112:0/140766727 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7ff9f80ae7c0 con 0x7ffa08102550 2024-09-16T00:36:03.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.730+0000 7ffa077fe640 1 --2- 172.21.15.112:0/140766727 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff9f006e530 0x7ff9f00709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:03.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.730+0000 7ffa077fe640 1 --2- 172.21.15.112:0/140766727 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff9f006e530 0x7ff9f00709e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ffa08071350 tx=0x7ff9fc00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:03.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.734+0000 7ffa04ff9640 1 -- 172.21.15.112:0/140766727 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7ff9f8077730 con 0x7ffa08102550 2024-09-16T00:36:03.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.935+0000 7ffa0e328640 1 -- 172.21.15.112:0/140766727 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ffa080fbd50 con 0x7ffa08102550 2024-09-16T00:36:03.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.936+0000 7ffa04ff9640 1 -- 172.21.15.112:0/140766727 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1914 (secure 0 0 0) 0x7ff9f807adf0 con 0x7ffa08102550 2024-09-16T00:36:03.936 INFO:teuthology.orchestra.run.smithi112.stdout:e34 2024-09-16T00:36:03.936 INFO:teuthology.orchestra.run.smithi112.stdout:btime 2024-09-16T00:34:26:619254+0000 2024-09-16T00:36:03.937 INFO:teuthology.orchestra.run.smithi112.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T00:36:03.937 INFO:teuthology.orchestra.run.smithi112.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:36:03.937 INFO:teuthology.orchestra.run.smithi112.stdout:legacy client fscid: 1 2024-09-16T00:36:03.937 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:36:03.937 INFO:teuthology.orchestra.run.smithi112.stdout:Filesystem 'cephfs' (1) 2024-09-16T00:36:03.937 INFO:teuthology.orchestra.run.smithi112.stdout:fs_name cephfs 2024-09-16T00:36:03.937 INFO:teuthology.orchestra.run.smithi112.stdout:epoch 25 2024-09-16T00:36:03.937 INFO:teuthology.orchestra.run.smithi112.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-16T00:36:03.937 INFO:teuthology.orchestra.run.smithi112.stdout:created 2024-09-16T00:23:22.328199+0000 2024-09-16T00:36:03.937 INFO:teuthology.orchestra.run.smithi112.stdout:modified 2024-09-16T00:33:29.012991+0000 2024-09-16T00:36:03.937 INFO:teuthology.orchestra.run.smithi112.stdout:tableserver 0 2024-09-16T00:36:03.937 INFO:teuthology.orchestra.run.smithi112.stdout:root 0 2024-09-16T00:36:03.937 INFO:teuthology.orchestra.run.smithi112.stdout:session_timeout 60 2024-09-16T00:36:03.937 INFO:teuthology.orchestra.run.smithi112.stdout:session_autoclose 300 2024-09-16T00:36:03.938 INFO:teuthology.orchestra.run.smithi112.stdout:max_file_size 1099511627776 2024-09-16T00:36:03.938 INFO:teuthology.orchestra.run.smithi112.stdout:max_xattr_size 65536 2024-09-16T00:36:03.938 INFO:teuthology.orchestra.run.smithi112.stdout:required_client_features {} 2024-09-16T00:36:03.938 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure 0 2024-09-16T00:36:03.938 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure_osd_epoch 82 2024-09-16T00:36:03.938 INFO:teuthology.orchestra.run.smithi112.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:36:03.938 INFO:teuthology.orchestra.run.smithi112.stdout:max_mds 2 2024-09-16T00:36:03.938 INFO:teuthology.orchestra.run.smithi112.stdout:in 0,1 2024-09-16T00:36:03.938 INFO:teuthology.orchestra.run.smithi112.stdout:up {} 2024-09-16T00:36:03.938 INFO:teuthology.orchestra.run.smithi112.stdout:failed 0,1 2024-09-16T00:36:03.938 INFO:teuthology.orchestra.run.smithi112.stdout:damaged 2024-09-16T00:36:03.938 INFO:teuthology.orchestra.run.smithi112.stdout:stopped 2024-09-16T00:36:03.938 INFO:teuthology.orchestra.run.smithi112.stdout:data_pools [3] 2024-09-16T00:36:03.939 INFO:teuthology.orchestra.run.smithi112.stdout:metadata_pool 2 2024-09-16T00:36:03.939 INFO:teuthology.orchestra.run.smithi112.stdout:inline_data enabled 2024-09-16T00:36:03.939 INFO:teuthology.orchestra.run.smithi112.stdout:balancer 2024-09-16T00:36:03.939 INFO:teuthology.orchestra.run.smithi112.stdout:bal_rank_mask -1 2024-09-16T00:36:03.939 INFO:teuthology.orchestra.run.smithi112.stdout:standby_count_wanted 1 2024-09-16T00:36:03.939 INFO:teuthology.orchestra.run.smithi112.stdout:qdb_cluster leader: 0 members: 2024-09-16T00:36:03.939 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:36:03.939 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:36:03.939 INFO:teuthology.orchestra.run.smithi112.stdout:Standby daemons: 2024-09-16T00:36:03.939 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:36:03.939 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.ithwzj{-1:34392} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.112:6826/476229221,v1:172.21.15.112:6827/476229221] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:36:03.939 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.hxfwkh{-1:34404} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.112:6828/2356899876,v1:172.21.15.112:6829/2356899876] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:36:03.939 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.gawwgw{-1:44343} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.169:6824/1635329957,v1:172.21.15.169:6825/1635329957] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:36:03.939 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.lgeqrj{-1:44347} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.169:6826/184810031,v1:172.21.15.169:6827/184810031] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:36:03.940 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 34 2024-09-16T00:36:03.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.938+0000 7ffa0e328640 1 -- 172.21.15.112:0/140766727 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff9f006e530 msgr2=0x7ff9f00709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:03.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.938+0000 7ffa0e328640 1 --2- 172.21.15.112:0/140766727 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff9f006e530 0x7ff9f00709e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ffa08071350 tx=0x7ff9fc00b040 comp rx=0 tx=0).stop 2024-09-16T00:36:03.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.938+0000 7ffa0e328640 1 -- 172.21.15.112:0/140766727 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa08102550 msgr2=0x7ffa0806fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:03.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.938+0000 7ffa0e328640 1 --2- 172.21.15.112:0/140766727 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa08102550 0x7ffa0806fa10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff9f80043d0 tx=0x7ff9f8009da0 comp rx=0 tx=0).stop 2024-09-16T00:36:03.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.938+0000 7ffa0e328640 1 -- 172.21.15.112:0/140766727 shutdown_connections 2024-09-16T00:36:03.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.938+0000 7ffa0e328640 1 --2- 172.21.15.112:0/140766727 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff9f006e530 0x7ff9f00709e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:03.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.938+0000 7ffa0e328640 1 --2- 172.21.15.112:0/140766727 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa08102ef0 0x7ffa0806ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:03.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.938+0000 7ffa0e328640 1 --2- 172.21.15.112:0/140766727 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa08102550 0x7ffa0806fa10 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:03.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.938+0000 7ffa0e328640 1 -- 172.21.15.112:0/140766727 >> 172.21.15.112:0/140766727 conn(0x7ffa080f80f0 msgr2=0x7ffa080f90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:03.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.938+0000 7ffa0e328640 1 -- 172.21.15.112:0/140766727 shutdown_connections 2024-09-16T00:36:03.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:03.938+0000 7ffa0e328640 1 -- 172.21.15.112:0/140766727 wait complete. 2024-09-16T00:36:04.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.017+0000 7f2b782ce640 1 -- 172.21.15.112:0/3305633116 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b700fd580 msgr2=0x7f2b700fd950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:04.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.017+0000 7f2b782ce640 1 --2- 172.21.15.112:0/3305633116 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b700fd580 0x7f2b700fd950 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c0048d0 tx=0x7f2b6c009da0 comp rx=0 tx=0).stop 2024-09-16T00:36:04.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.018+0000 7f2b782ce640 1 -- 172.21.15.112:0/3305633116 shutdown_connections 2024-09-16T00:36:04.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.018+0000 7f2b782ce640 1 --2- 172.21.15.112:0/3305633116 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b700fdf20 0x7f2b70106560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:04.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.018+0000 7f2b782ce640 1 --2- 172.21.15.112:0/3305633116 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b700fd580 0x7f2b700fd950 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:04.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.018+0000 7f2b782ce640 1 -- 172.21.15.112:0/3305633116 >> 172.21.15.112:0/3305633116 conn(0x7f2b700f7010 msgr2=0x7f2b700f9420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:04.019 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:36:03 smithi112 ceph-mon[97750]: pgmap v249: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:36:04.019 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:36:03 smithi112 ceph-mon[97750]: from='client.34520 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:36:04.019 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:36:03 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/3803545012' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:36:04.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.018+0000 7f2b782ce640 1 -- 172.21.15.112:0/3305633116 shutdown_connections 2024-09-16T00:36:04.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.018+0000 7f2b782ce640 1 -- 172.21.15.112:0/3305633116 wait complete. 2024-09-16T00:36:04.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.018+0000 7f2b782ce640 1 Processor -- start 2024-09-16T00:36:04.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.018+0000 7f2b782ce640 1 -- start start 2024-09-16T00:36:04.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.018+0000 7f2b782ce640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b700fd580 0x7f2b70113230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:04.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.018+0000 7f2b782ce640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b700fdf20 0x7f2b70113770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:04.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.018+0000 7f2b782ce640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b70113d70 con 0x7f2b700fdf20 2024-09-16T00:36:04.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.018+0000 7f2b782ce640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7010d1b0 con 0x7f2b700fd580 2024-09-16T00:36:04.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.019+0000 7f2b75869640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b700fdf20 0x7f2b70113770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:04.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.019+0000 7f2b75869640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b700fdf20 0x7f2b70113770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53096/0 (socket says 172.21.15.112:53096) 2024-09-16T00:36:04.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.019+0000 7f2b75869640 1 -- 172.21.15.112:0/3245275153 learned_addr learned my addr 172.21.15.112:0/3245275153 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:36:04.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.019+0000 7f2b7606a640 1 --2- 172.21.15.112:0/3245275153 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b700fd580 0x7f2b70113230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:04.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.019+0000 7f2b75869640 1 -- 172.21.15.112:0/3245275153 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b700fd580 msgr2=0x7f2b70113230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:04.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.019+0000 7f2b75869640 1 --2- 172.21.15.112:0/3245275153 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b700fd580 0x7f2b70113230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:04.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.019+0000 7f2b75869640 1 -- 172.21.15.112:0/3245275153 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b6c005040 con 0x7f2b700fdf20 2024-09-16T00:36:04.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.019+0000 7f2b75869640 1 --2- 172.21.15.112:0/3245275153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b700fdf20 0x7f2b70113770 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2b6000c790 tx=0x7f2b600043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:04.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.019+0000 7f2b66ffd640 1 -- 172.21.15.112:0/3245275153 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b6000a7a0 con 0x7f2b700fdf20 2024-09-16T00:36:04.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.020+0000 7f2b782ce640 1 -- 172.21.15.112:0/3245275153 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b7010d490 con 0x7f2b700fdf20 2024-09-16T00:36:04.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.020+0000 7f2b66ffd640 1 -- 172.21.15.112:0/3245275153 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f2b6000ada0 con 0x7f2b700fdf20 2024-09-16T00:36:04.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.020+0000 7f2b66ffd640 1 -- 172.21.15.112:0/3245275153 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b600113a0 con 0x7f2b700fdf20 2024-09-16T00:36:04.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.020+0000 7f2b782ce640 1 -- 172.21.15.112:0/3245275153 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b7010d9b0 con 0x7f2b700fdf20 2024-09-16T00:36:04.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.021+0000 7f2b782ce640 1 -- 172.21.15.112:0/3245275153 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b700663b0 con 0x7f2b700fdf20 2024-09-16T00:36:04.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.021+0000 7f2b66ffd640 1 -- 172.21.15.112:0/3245275153 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f2b6000a900 con 0x7f2b700fdf20 2024-09-16T00:36:04.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.022+0000 7f2b66ffd640 1 --2- 172.21.15.112:0/3245275153 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f2b5c06e530 0x7f2b5c0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:04.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.022+0000 7f2b66ffd640 1 -- 172.21.15.112:0/3245275153 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f2b6008e2c0 con 0x7f2b700fdf20 2024-09-16T00:36:04.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.022+0000 7f2b7606a640 1 --2- 172.21.15.112:0/3245275153 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f2b5c06e530 0x7f2b5c0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:04.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.022+0000 7f2b7606a640 1 --2- 172.21.15.112:0/3245275153 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f2b5c06e530 0x7f2b5c0709e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c009ac0 tx=0x7f2b6c035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:04.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.026+0000 7f2b66ffd640 1 -- 172.21.15.112:0/3245275153 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f2b6005ae80 con 0x7f2b700fdf20 2024-09-16T00:36:04.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.207+0000 7f2b782ce640 1 -- 172.21.15.112:0/3245275153 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2b70109580 con 0x7f2b5c06e530 2024-09-16T00:36:04.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.208+0000 7f2b66ffd640 1 -- 172.21.15.112:0/3245275153 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f2b70109580 con 0x7f2b5c06e530 2024-09-16T00:36:04.209 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:36:04.209 INFO:teuthology.orchestra.run.smithi112.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80", 2024-09-16T00:36:04.209 INFO:teuthology.orchestra.run.smithi112.stdout: "in_progress": true, 2024-09-16T00:36:04.209 INFO:teuthology.orchestra.run.smithi112.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T00:36:04.209 INFO:teuthology.orchestra.run.smithi112.stdout: "services_complete": [ 2024-09-16T00:36:04.209 INFO:teuthology.orchestra.run.smithi112.stdout: "mds", 2024-09-16T00:36:04.209 INFO:teuthology.orchestra.run.smithi112.stdout: "osd", 2024-09-16T00:36:04.209 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr", 2024-09-16T00:36:04.209 INFO:teuthology.orchestra.run.smithi112.stdout: "mon", 2024-09-16T00:36:04.210 INFO:teuthology.orchestra.run.smithi112.stdout: "crash" 2024-09-16T00:36:04.210 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-16T00:36:04.210 INFO:teuthology.orchestra.run.smithi112.stdout: "progress": "16/21 daemons upgraded", 2024-09-16T00:36:04.210 INFO:teuthology.orchestra.run.smithi112.stdout: "message": "Currently upgrading mds daemons", 2024-09-16T00:36:04.210 INFO:teuthology.orchestra.run.smithi112.stdout: "is_paused": false 2024-09-16T00:36:04.210 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:36:04.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.210+0000 7f2b782ce640 1 -- 172.21.15.112:0/3245275153 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f2b5c06e530 msgr2=0x7f2b5c0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:04.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.210+0000 7f2b782ce640 1 --2- 172.21.15.112:0/3245275153 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f2b5c06e530 0x7f2b5c0709e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c009ac0 tx=0x7f2b6c035040 comp rx=0 tx=0).stop 2024-09-16T00:36:04.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.210+0000 7f2b782ce640 1 -- 172.21.15.112:0/3245275153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b700fdf20 msgr2=0x7f2b70113770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:04.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.210+0000 7f2b782ce640 1 --2- 172.21.15.112:0/3245275153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b700fdf20 0x7f2b70113770 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2b6000c790 tx=0x7f2b600043c0 comp rx=0 tx=0).stop 2024-09-16T00:36:04.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.210+0000 7f2b782ce640 1 -- 172.21.15.112:0/3245275153 shutdown_connections 2024-09-16T00:36:04.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.210+0000 7f2b782ce640 1 --2- 172.21.15.112:0/3245275153 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f2b5c06e530 0x7f2b5c0709e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:04.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.210+0000 7f2b782ce640 1 --2- 172.21.15.112:0/3245275153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b700fdf20 0x7f2b70113770 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:04.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.210+0000 7f2b782ce640 1 --2- 172.21.15.112:0/3245275153 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b700fd580 0x7f2b70113230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:04.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.210+0000 7f2b782ce640 1 -- 172.21.15.112:0/3245275153 >> 172.21.15.112:0/3245275153 conn(0x7f2b700f7010 msgr2=0x7f2b700f85d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:04.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.210+0000 7f2b782ce640 1 -- 172.21.15.112:0/3245275153 shutdown_connections 2024-09-16T00:36:04.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.210+0000 7f2b782ce640 1 -- 172.21.15.112:0/3245275153 wait complete. 2024-09-16T00:36:04.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:36:03 smithi169 ceph-mon[77626]: pgmap v249: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:36:04.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:36:03 smithi169 ceph-mon[77626]: from='client.34520 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:36:04.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:36:03 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/3803545012' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:36:04.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.288+0000 7fcc03fb9640 1 -- 172.21.15.112:0/2475103850 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcbfc102580 msgr2=0x7fcbfc102950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:04.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.288+0000 7fcc03fb9640 1 --2- 172.21.15.112:0/2475103850 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcbfc102580 0x7fcbfc102950 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fcbf80048d0 tx=0x7fcbf8009da0 comp rx=0 tx=0).stop 2024-09-16T00:36:04.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.288+0000 7fcc03fb9640 1 -- 172.21.15.112:0/2475103850 shutdown_connections 2024-09-16T00:36:04.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.288+0000 7fcc03fb9640 1 --2- 172.21.15.112:0/2475103850 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbfc102f20 0x7fcbfc0fc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:04.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.288+0000 7fcc03fb9640 1 --2- 172.21.15.112:0/2475103850 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcbfc102580 0x7fcbfc102950 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:04.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.288+0000 7fcc03fb9640 1 -- 172.21.15.112:0/2475103850 >> 172.21.15.112:0/2475103850 conn(0x7fcbfc0f81a0 msgr2=0x7fcbfc0fa5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:04.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.288+0000 7fcc03fb9640 1 -- 172.21.15.112:0/2475103850 shutdown_connections 2024-09-16T00:36:04.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.288+0000 7fcc03fb9640 1 -- 172.21.15.112:0/2475103850 wait complete. 2024-09-16T00:36:04.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.289+0000 7fcc03fb9640 1 Processor -- start 2024-09-16T00:36:04.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.289+0000 7fcc03fb9640 1 -- start start 2024-09-16T00:36:04.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.289+0000 7fcc03fb9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcbfc102580 0x7fcbfc113220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:04.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.289+0000 7fcc03fb9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbfc102f20 0x7fcbfc113760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:04.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.289+0000 7fcc03fb9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbfc113d60 con 0x7fcbfc102580 2024-09-16T00:36:04.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.289+0000 7fcc03fb9640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbfc10d1a0 con 0x7fcbfc102f20 2024-09-16T00:36:04.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.289+0000 7fcc01d55640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcbfc102580 0x7fcbfc113220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:04.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.289+0000 7fcc01d55640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcbfc102580 0x7fcbfc113220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53108/0 (socket says 172.21.15.112:53108) 2024-09-16T00:36:04.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.289+0000 7fcc01d55640 1 -- 172.21.15.112:0/1672033403 learned_addr learned my addr 172.21.15.112:0/1672033403 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:36:04.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.290+0000 7fcc01554640 1 --2- 172.21.15.112:0/1672033403 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbfc102f20 0x7fcbfc113760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:04.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.290+0000 7fcc01d55640 1 -- 172.21.15.112:0/1672033403 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbfc102f20 msgr2=0x7fcbfc113760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:04.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.290+0000 7fcc01d55640 1 --2- 172.21.15.112:0/1672033403 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbfc102f20 0x7fcbfc113760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:04.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.290+0000 7fcc01d55640 1 -- 172.21.15.112:0/1672033403 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbf8005040 con 0x7fcbfc102580 2024-09-16T00:36:04.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.290+0000 7fcc01d55640 1 --2- 172.21.15.112:0/1672033403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcbfc102580 0x7fcbfc113220 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fcbf8003080 tx=0x7fcbf8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:04.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.290+0000 7fcbf27fc640 1 -- 172.21.15.112:0/1672033403 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcbf8038070 con 0x7fcbfc102580 2024-09-16T00:36:04.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.290+0000 7fcbf27fc640 1 -- 172.21.15.112:0/1672033403 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fcbf80097e0 con 0x7fcbfc102580 2024-09-16T00:36:04.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.290+0000 7fcc03fb9640 1 -- 172.21.15.112:0/1672033403 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbfc10d420 con 0x7fcbfc102580 2024-09-16T00:36:04.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.290+0000 7fcbf27fc640 1 -- 172.21.15.112:0/1672033403 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcbf803d420 con 0x7fcbfc102580 2024-09-16T00:36:04.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.290+0000 7fcc03fb9640 1 -- 172.21.15.112:0/1672033403 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbfc10d960 con 0x7fcbfc102580 2024-09-16T00:36:04.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.291+0000 7fcc03fb9640 1 -- 172.21.15.112:0/1672033403 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbfc0fdd00 con 0x7fcbfc102580 2024-09-16T00:36:04.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.292+0000 7fcbf27fc640 1 -- 172.21.15.112:0/1672033403 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fcbf8030020 con 0x7fcbfc102580 2024-09-16T00:36:04.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.292+0000 7fcbf27fc640 1 --2- 172.21.15.112:0/1672033403 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fcbe806e530 0x7fcbe80709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:04.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.292+0000 7fcbf27fc640 1 -- 172.21.15.112:0/1672033403 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7fcbf80ae7c0 con 0x7fcbfc102580 2024-09-16T00:36:04.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.292+0000 7fcc01554640 1 --2- 172.21.15.112:0/1672033403 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fcbe806e530 0x7fcbe80709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:04.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.293+0000 7fcc01554640 1 --2- 172.21.15.112:0/1672033403 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fcbe806e530 0x7fcbe80709e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc10eae0 tx=0x7fcbec009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:04.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.296+0000 7fcbf27fc640 1 -- 172.21.15.112:0/1672033403 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fcbf80777b0 con 0x7fcbfc102580 2024-09-16T00:36:04.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.529+0000 7fcc03fb9640 1 -- 172.21.15.112:0/1672033403 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fcbfc0fbda0 con 0x7fcbfc102580 2024-09-16T00:36:04.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.529+0000 7fcbf27fc640 1 -- 172.21.15.112:0/1672033403 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7fcbf807ae70 con 0x7fcbfc102580 2024-09-16T00:36:04.530 INFO:teuthology.orchestra.run.smithi112.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-16T00:36:04.530 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-16T00:36:04.530 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs is degraded 2024-09-16T00:36:04.530 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T00:36:04.530 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T00:36:04.530 INFO:teuthology.orchestra.run.smithi112.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-16T00:36:04.530 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-16T00:36:04.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.531+0000 7fcc03fb9640 1 -- 172.21.15.112:0/1672033403 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fcbe806e530 msgr2=0x7fcbe80709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:04.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.531+0000 7fcc03fb9640 1 --2- 172.21.15.112:0/1672033403 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fcbe806e530 0x7fcbe80709e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc10eae0 tx=0x7fcbec009040 comp rx=0 tx=0).stop 2024-09-16T00:36:04.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.531+0000 7fcc03fb9640 1 -- 172.21.15.112:0/1672033403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcbfc102580 msgr2=0x7fcbfc113220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:04.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.531+0000 7fcc03fb9640 1 --2- 172.21.15.112:0/1672033403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcbfc102580 0x7fcbfc113220 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fcbf8003080 tx=0x7fcbf8009da0 comp rx=0 tx=0).stop 2024-09-16T00:36:04.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.531+0000 7fcc03fb9640 1 -- 172.21.15.112:0/1672033403 shutdown_connections 2024-09-16T00:36:04.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.531+0000 7fcc03fb9640 1 --2- 172.21.15.112:0/1672033403 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fcbe806e530 0x7fcbe80709e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:04.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.531+0000 7fcc03fb9640 1 --2- 172.21.15.112:0/1672033403 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbfc102f20 0x7fcbfc113760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:04.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.531+0000 7fcc03fb9640 1 --2- 172.21.15.112:0/1672033403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcbfc102580 0x7fcbfc113220 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:04.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.531+0000 7fcc03fb9640 1 -- 172.21.15.112:0/1672033403 >> 172.21.15.112:0/1672033403 conn(0x7fcbfc0f81a0 msgr2=0x7fcbfc0f91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:04.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.532+0000 7fcc03fb9640 1 -- 172.21.15.112:0/1672033403 shutdown_connections 2024-09-16T00:36:04.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:04.532+0000 7fcc03fb9640 1 -- 172.21.15.112:0/1672033403 wait complete. 2024-09-16T00:36:05.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:36:04 smithi112 ceph-mon[97750]: from='client.34524 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:36:05.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:36:04 smithi112 ceph-mon[97750]: from='client.34528 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:36:05.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:36:04 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/140766727' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:36:05.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:36:04 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/1672033403' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:36:05.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:36:04 smithi169 ceph-mon[77626]: from='client.34524 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:36:05.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:36:04 smithi169 ceph-mon[77626]: from='client.34528 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:36:05.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:36:04 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/140766727' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:36:05.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:36:04 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/1672033403' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:36:06.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:36:05 smithi112 ceph-mon[97750]: from='client.34540 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:36:06.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:36:05 smithi112 ceph-mon[97750]: pgmap v250: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:36:06.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:36:05 smithi169 ceph-mon[77626]: from='client.34540 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:36:06.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:36:05 smithi169 ceph-mon[77626]: pgmap v250: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:36:08.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:36:07 smithi112 ceph-mon[97750]: pgmap v251: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:36:08.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:36:07 smithi169 ceph-mon[77626]: pgmap v251: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:36:10.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:36:09 smithi112 ceph-mon[97750]: pgmap v252: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:36:10.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:36:09 smithi169 ceph-mon[77626]: pgmap v252: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:36:12.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:36:11 smithi112 ceph-mon[97750]: pgmap v253: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:36:12.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:36:11 smithi169 ceph-mon[77626]: pgmap v253: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:36:14.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:36:13 smithi112 ceph-mon[97750]: pgmap v254: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:36:14.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:36:13 smithi169 ceph-mon[77626]: pgmap v254: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:36:16.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:36:15 smithi112 ceph-mon[97750]: pgmap v255: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:36:16.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:36:15 smithi169 ceph-mon[77626]: pgmap v255: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:36:17.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:36:16 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:36:17.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:36:16 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:36:18.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:36:17 smithi112 ceph-mon[97750]: pgmap v256: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:36:18.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:36:17 smithi169 ceph-mon[77626]: pgmap v256: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:36:19.177 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:36:18 smithi169 ceph-mon[77626]: pgmap v257: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:36:19.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:36:18 smithi112 ceph-mon[97750]: pgmap v257: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:36:22.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:36:21 smithi169 ceph-mon[77626]: pgmap v258: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:36:22.042 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:36:21 smithi112 ceph-mon[97750]: pgmap v258: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:36:24.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:36:23 smithi169 ceph-mon[77626]: pgmap v259: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:36:24.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:36:23 smithi112 ceph-mon[97750]: pgmap v259: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:36:26.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:36:25 smithi169 ceph-mon[77626]: pgmap v260: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:36:26.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:36:25 smithi112 ceph-mon[97750]: pgmap v260: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:36:28.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:36:27 smithi169 ceph-mon[77626]: pgmap v261: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:36:28.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:36:27 smithi112 ceph-mon[97750]: pgmap v261: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:36:30.028 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:36:29 smithi169 ceph-mon[77626]: pgmap v262: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:36:30.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:36:29 smithi112 ceph-mon[97750]: pgmap v262: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:36:32.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:36:31 smithi169 ceph-mon[77626]: pgmap v263: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:36:32.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:36:31 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:36:32.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:36:31 smithi112 ceph-mon[97750]: pgmap v263: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:36:32.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:36:31 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:36:34.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:36:33 smithi169 ceph-mon[77626]: pgmap v264: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:36:34.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:36:33 smithi112 ceph-mon[97750]: pgmap v264: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:36:34.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.622+0000 7f848935b640 1 -- 172.21.15.112:0/1899118349 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8484065c10 msgr2=0x7f84840fe0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:34.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.622+0000 7f848935b640 1 --2- 172.21.15.112:0/1899118349 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8484065c10 0x7f84840fe0d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f846c0048d0 tx=0x7f846c009da0 comp rx=0 tx=0).stop 2024-09-16T00:36:34.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.622+0000 7f848935b640 1 -- 172.21.15.112:0/1899118349 shutdown_connections 2024-09-16T00:36:34.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.622+0000 7f848935b640 1 --2- 172.21.15.112:0/1899118349 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8484065c10 0x7f84840fe0d0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:34.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.622+0000 7f848935b640 1 --2- 172.21.15.112:0/1899118349 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8484065300 0x7f84840656d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:34.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.622+0000 7f848935b640 1 -- 172.21.15.112:0/1899118349 >> 172.21.15.112:0/1899118349 conn(0x7f84840723b0 msgr2=0x7f84840727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:34.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.622+0000 7f848935b640 1 -- 172.21.15.112:0/1899118349 shutdown_connections 2024-09-16T00:36:34.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.622+0000 7f848935b640 1 -- 172.21.15.112:0/1899118349 wait complete. 2024-09-16T00:36:34.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.623+0000 7f848935b640 1 Processor -- start 2024-09-16T00:36:34.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.623+0000 7f848935b640 1 -- start start 2024-09-16T00:36:34.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.623+0000 7f848935b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8484065300 0x7f84841131b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:34.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.623+0000 7f848935b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8484065c10 0x7f84841136f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:34.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.623+0000 7f848935b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8484113cf0 con 0x7f8484065300 2024-09-16T00:36:34.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.623+0000 7f848935b640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f848410d130 con 0x7f8484065c10 2024-09-16T00:36:34.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.623+0000 7f8482ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8484065300 0x7f84841131b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:34.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.623+0000 7f8482ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8484065300 0x7f84841131b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58802/0 (socket says 172.21.15.112:58802) 2024-09-16T00:36:34.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.623+0000 7f8482ffd640 1 -- 172.21.15.112:0/3684713849 learned_addr learned my addr 172.21.15.112:0/3684713849 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:36:34.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.623+0000 7f84827fc640 1 --2- 172.21.15.112:0/3684713849 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8484065c10 0x7f84841136f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:34.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.623+0000 7f8482ffd640 1 -- 172.21.15.112:0/3684713849 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8484065c10 msgr2=0x7f84841136f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:34.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.623+0000 7f8482ffd640 1 --2- 172.21.15.112:0/3684713849 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8484065c10 0x7f84841136f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:34.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.623+0000 7f8482ffd640 1 -- 172.21.15.112:0/3684713849 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f846c005040 con 0x7f8484065300 2024-09-16T00:36:34.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.623+0000 7f84827fc640 1 --2- 172.21.15.112:0/3684713849 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8484065c10 0x7f84841136f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:36:34.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.624+0000 7f8482ffd640 1 --2- 172.21.15.112:0/3684713849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8484065300 0x7f84841131b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f847400cb10 tx=0x7f8474007ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:34.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.624+0000 7f847b7fe640 1 -- 172.21.15.112:0/3684713849 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8474004200 con 0x7f8484065300 2024-09-16T00:36:34.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.624+0000 7f847b7fe640 1 -- 172.21.15.112:0/3684713849 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f847400acc0 con 0x7f8484065300 2024-09-16T00:36:34.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.624+0000 7f848935b640 1 -- 172.21.15.112:0/3684713849 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f848410d410 con 0x7f8484065300 2024-09-16T00:36:34.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.624+0000 7f847b7fe640 1 -- 172.21.15.112:0/3684713849 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8474003cd0 con 0x7f8484065300 2024-09-16T00:36:34.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.624+0000 7f848935b640 1 -- 172.21.15.112:0/3684713849 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f848410d980 con 0x7f8484065300 2024-09-16T00:36:34.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.625+0000 7f848935b640 1 -- 172.21.15.112:0/3684713849 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84840fdce0 con 0x7f8484065300 2024-09-16T00:36:34.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.625+0000 7f847b7fe640 1 -- 172.21.15.112:0/3684713849 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f847400a820 con 0x7f8484065300 2024-09-16T00:36:34.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.626+0000 7f847b7fe640 1 --2- 172.21.15.112:0/3684713849 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f847006e530 0x7f84700709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:34.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.626+0000 7f847b7fe640 1 -- 172.21.15.112:0/3684713849 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f847408ea40 con 0x7f8484065300 2024-09-16T00:36:34.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.626+0000 7f84827fc640 1 --2- 172.21.15.112:0/3684713849 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f847006e530 0x7f84700709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:34.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.626+0000 7f84827fc640 1 --2- 172.21.15.112:0/3684713849 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f847006e530 0x7f84700709e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f84840f86a0 tx=0x7f846c035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:34.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.631+0000 7f847b7fe640 1 -- 172.21.15.112:0/3684713849 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f84740579b0 con 0x7f8484065300 2024-09-16T00:36:34.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.816+0000 7f848935b640 1 -- 172.21.15.112:0/3684713849 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f848404b750 con 0x7f847006e530 2024-09-16T00:36:34.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.817+0000 7f847b7fe640 1 -- 172.21.15.112:0/3684713849 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f848404b750 con 0x7f847006e530 2024-09-16T00:36:34.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.819+0000 7f848935b640 1 -- 172.21.15.112:0/3684713849 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f847006e530 msgr2=0x7f84700709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:34.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.819+0000 7f848935b640 1 --2- 172.21.15.112:0/3684713849 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f847006e530 0x7f84700709e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f84840f86a0 tx=0x7f846c035040 comp rx=0 tx=0).stop 2024-09-16T00:36:34.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.819+0000 7f848935b640 1 -- 172.21.15.112:0/3684713849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8484065300 msgr2=0x7f84841131b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:34.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.819+0000 7f848935b640 1 --2- 172.21.15.112:0/3684713849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8484065300 0x7f84841131b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f847400cb10 tx=0x7f8474007ee0 comp rx=0 tx=0).stop 2024-09-16T00:36:34.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.819+0000 7f848935b640 1 -- 172.21.15.112:0/3684713849 shutdown_connections 2024-09-16T00:36:34.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.819+0000 7f848935b640 1 --2- 172.21.15.112:0/3684713849 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f847006e530 0x7f84700709e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:34.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.819+0000 7f848935b640 1 --2- 172.21.15.112:0/3684713849 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8484065c10 0x7f84841136f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:34.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.819+0000 7f848935b640 1 --2- 172.21.15.112:0/3684713849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8484065300 0x7f84841131b0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:34.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.819+0000 7f848935b640 1 -- 172.21.15.112:0/3684713849 >> 172.21.15.112:0/3684713849 conn(0x7f84840723b0 msgr2=0x7f84840fa2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:34.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.819+0000 7f848935b640 1 -- 172.21.15.112:0/3684713849 shutdown_connections 2024-09-16T00:36:34.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.819+0000 7f848935b640 1 -- 172.21.15.112:0/3684713849 wait complete. 2024-09-16T00:36:34.830 INFO:teuthology.orchestra.run.smithi112.stdout:true 2024-09-16T00:36:34.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.900+0000 7f846681b640 1 -- 172.21.15.112:0/2933355879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8460102530 msgr2=0x7f8460102900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:34.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.900+0000 7f846681b640 1 --2- 172.21.15.112:0/2933355879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8460102530 0x7f8460102900 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f84500048d0 tx=0x7f8450009da0 comp rx=0 tx=0).stop 2024-09-16T00:36:34.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.900+0000 7f846681b640 1 -- 172.21.15.112:0/2933355879 shutdown_connections 2024-09-16T00:36:34.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.900+0000 7f846681b640 1 --2- 172.21.15.112:0/2933355879 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8460102ed0 0x7f84600fc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:34.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.900+0000 7f846681b640 1 --2- 172.21.15.112:0/2933355879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8460102530 0x7f8460102900 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:34.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.900+0000 7f846681b640 1 -- 172.21.15.112:0/2933355879 >> 172.21.15.112:0/2933355879 conn(0x7f84600f80f0 msgr2=0x7f84600fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:34.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.900+0000 7f846681b640 1 -- 172.21.15.112:0/2933355879 shutdown_connections 2024-09-16T00:36:34.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.900+0000 7f846681b640 1 -- 172.21.15.112:0/2933355879 wait complete. 2024-09-16T00:36:34.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.900+0000 7f846681b640 1 Processor -- start 2024-09-16T00:36:34.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.900+0000 7f846681b640 1 -- start start 2024-09-16T00:36:34.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.901+0000 7f846681b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8460102530 0x7f846006fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:34.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.901+0000 7f846681b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8460102ed0 0x7f846006ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:34.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.901+0000 7f846681b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8460070520 con 0x7f8460102530 2024-09-16T00:36:34.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.901+0000 7f846681b640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8460070690 con 0x7f8460102ed0 2024-09-16T00:36:34.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.901+0000 7f845ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8460102530 0x7f846006fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:34.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.901+0000 7f845ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8460102530 0x7f846006fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58814/0 (socket says 172.21.15.112:58814) 2024-09-16T00:36:34.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.901+0000 7f845ffff640 1 -- 172.21.15.112:0/21618229 learned_addr learned my addr 172.21.15.112:0/21618229 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:36:34.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.901+0000 7f8457fff640 1 --2- 172.21.15.112:0/21618229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8460102ed0 0x7f846006ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:34.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.901+0000 7f845ffff640 1 -- 172.21.15.112:0/21618229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8460102ed0 msgr2=0x7f846006ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:34.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.901+0000 7f845ffff640 1 --2- 172.21.15.112:0/21618229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8460102ed0 0x7f846006ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:34.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.901+0000 7f845ffff640 1 -- 172.21.15.112:0/21618229 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8450005040 con 0x7f8460102530 2024-09-16T00:36:34.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.901+0000 7f845ffff640 1 --2- 172.21.15.112:0/21618229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8460102530 0x7f846006fa10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8450000c00 tx=0x7f8450009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:34.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.902+0000 7f845d7fa640 1 -- 172.21.15.112:0/21618229 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8450038070 con 0x7f8460102530 2024-09-16T00:36:34.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.902+0000 7f845d7fa640 1 -- 172.21.15.112:0/21618229 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f845003daf0 con 0x7f8460102530 2024-09-16T00:36:34.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.902+0000 7f846681b640 1 -- 172.21.15.112:0/21618229 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8460119570 con 0x7f8460102530 2024-09-16T00:36:34.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.902+0000 7f845d7fa640 1 -- 172.21.15.112:0/21618229 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8450033b20 con 0x7f8460102530 2024-09-16T00:36:34.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.902+0000 7f846681b640 1 -- 172.21.15.112:0/21618229 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8460119950 con 0x7f8460102530 2024-09-16T00:36:34.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.903+0000 7f846681b640 1 -- 172.21.15.112:0/21618229 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84600fdcb0 con 0x7f8460102530 2024-09-16T00:36:34.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.903+0000 7f845d7fa640 1 -- 172.21.15.112:0/21618229 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f8450030020 con 0x7f8460102530 2024-09-16T00:36:34.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.904+0000 7f845d7fa640 1 --2- 172.21.15.112:0/21618229 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f844c06e530 0x7f844c0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:34.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.904+0000 7f8457fff640 1 --2- 172.21.15.112:0/21618229 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f844c06e530 0x7f844c0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:34.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.904+0000 7f845d7fa640 1 -- 172.21.15.112:0/21618229 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f84500ae480 con 0x7f8460102530 2024-09-16T00:36:34.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.904+0000 7f8457fff640 1 --2- 172.21.15.112:0/21618229 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f844c06e530 0x7f844c0709e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8460071350 tx=0x7f8448009120 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:34.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:34.908+0000 7f845d7fa640 1 -- 172.21.15.112:0/21618229 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f8450077390 con 0x7f8460102530 2024-09-16T00:36:35.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.090+0000 7f846681b640 1 -- 172.21.15.112:0/21618229 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8460070800 con 0x7f844c06e530 2024-09-16T00:36:35.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.091+0000 7f845d7fa640 1 -- 172.21.15.112:0/21618229 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f8460070800 con 0x7f844c06e530 2024-09-16T00:36:35.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.093+0000 7f846681b640 1 -- 172.21.15.112:0/21618229 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f844c06e530 msgr2=0x7f844c0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:35.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.093+0000 7f846681b640 1 --2- 172.21.15.112:0/21618229 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f844c06e530 0x7f844c0709e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8460071350 tx=0x7f8448009120 comp rx=0 tx=0).stop 2024-09-16T00:36:35.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.093+0000 7f846681b640 1 -- 172.21.15.112:0/21618229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8460102530 msgr2=0x7f846006fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:35.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.093+0000 7f846681b640 1 --2- 172.21.15.112:0/21618229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8460102530 0x7f846006fa10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8450000c00 tx=0x7f8450009da0 comp rx=0 tx=0).stop 2024-09-16T00:36:35.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.093+0000 7f846681b640 1 -- 172.21.15.112:0/21618229 shutdown_connections 2024-09-16T00:36:35.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.093+0000 7f846681b640 1 --2- 172.21.15.112:0/21618229 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f844c06e530 0x7f844c0709e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:35.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.093+0000 7f846681b640 1 --2- 172.21.15.112:0/21618229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8460102ed0 0x7f846006ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:35.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.093+0000 7f846681b640 1 --2- 172.21.15.112:0/21618229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8460102530 0x7f846006fa10 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:35.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.093+0000 7f846681b640 1 -- 172.21.15.112:0/21618229 >> 172.21.15.112:0/21618229 conn(0x7f84600f80f0 msgr2=0x7f84600f9140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:35.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.094+0000 7f846681b640 1 -- 172.21.15.112:0/21618229 shutdown_connections 2024-09-16T00:36:35.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.094+0000 7f846681b640 1 -- 172.21.15.112:0/21618229 wait complete. 2024-09-16T00:36:35.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.173+0000 7f577db34640 1 -- 172.21.15.112:0/4260198252 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5778102ef0 msgr2=0x7f57780fc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:35.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.173+0000 7f577db34640 1 --2- 172.21.15.112:0/4260198252 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5778102ef0 0x7f57780fc5b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f576c0048d0 tx=0x7f576c009da0 comp rx=0 tx=0).stop 2024-09-16T00:36:35.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.173+0000 7f577db34640 1 -- 172.21.15.112:0/4260198252 shutdown_connections 2024-09-16T00:36:35.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.173+0000 7f577db34640 1 --2- 172.21.15.112:0/4260198252 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5778102ef0 0x7f57780fc5b0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:35.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.173+0000 7f577db34640 1 --2- 172.21.15.112:0/4260198252 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5778102550 0x7f5778102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:35.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.173+0000 7f577db34640 1 -- 172.21.15.112:0/4260198252 >> 172.21.15.112:0/4260198252 conn(0x7f57780f80f0 msgr2=0x7f57780fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:35.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.173+0000 7f577db34640 1 -- 172.21.15.112:0/4260198252 shutdown_connections 2024-09-16T00:36:35.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.173+0000 7f577db34640 1 -- 172.21.15.112:0/4260198252 wait complete. 2024-09-16T00:36:35.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.174+0000 7f577db34640 1 Processor -- start 2024-09-16T00:36:35.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.174+0000 7f577db34640 1 -- start start 2024-09-16T00:36:35.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.174+0000 7f577db34640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5778102550 0x7f577806fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:35.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.174+0000 7f577db34640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5778102ef0 0x7f577806ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:35.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.174+0000 7f577db34640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5778070490 con 0x7f5778102550 2024-09-16T00:36:35.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.174+0000 7f577db34640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5778070600 con 0x7f5778102ef0 2024-09-16T00:36:35.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.174+0000 7f57777fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5778102550 0x7f577806fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:35.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.174+0000 7f57777fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5778102550 0x7f577806fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58840/0 (socket says 172.21.15.112:58840) 2024-09-16T00:36:35.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.174+0000 7f57777fe640 1 -- 172.21.15.112:0/3806059968 learned_addr learned my addr 172.21.15.112:0/3806059968 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:36:35.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.174+0000 7f5776ffd640 1 --2- 172.21.15.112:0/3806059968 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5778102ef0 0x7f577806ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:35.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.174+0000 7f57777fe640 1 -- 172.21.15.112:0/3806059968 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5778102ef0 msgr2=0x7f577806ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:35.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.174+0000 7f57777fe640 1 --2- 172.21.15.112:0/3806059968 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5778102ef0 0x7f577806ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:35.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.174+0000 7f57777fe640 1 -- 172.21.15.112:0/3806059968 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f576c005040 con 0x7f5778102550 2024-09-16T00:36:35.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.174+0000 7f5776ffd640 1 --2- 172.21.15.112:0/3806059968 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5778102ef0 0x7f577806ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:36:35.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.175+0000 7f57777fe640 1 --2- 172.21.15.112:0/3806059968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5778102550 0x7f577806fa10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f576800c790 tx=0x7f57680043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:35.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.175+0000 7f575ffff640 1 -- 172.21.15.112:0/3806059968 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f576800a820 con 0x7f5778102550 2024-09-16T00:36:35.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.175+0000 7f577db34640 1 -- 172.21.15.112:0/3806059968 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5778070890 con 0x7f5778102550 2024-09-16T00:36:35.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.175+0000 7f575ffff640 1 -- 172.21.15.112:0/3806059968 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f576800ae20 con 0x7f5778102550 2024-09-16T00:36:35.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.175+0000 7f575ffff640 1 -- 172.21.15.112:0/3806059968 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f57680113a0 con 0x7f5778102550 2024-09-16T00:36:35.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.175+0000 7f577db34640 1 -- 172.21.15.112:0/3806059968 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5778119c50 con 0x7f5778102550 2024-09-16T00:36:35.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.176+0000 7f577db34640 1 -- 172.21.15.112:0/3806059968 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57780fdcd0 con 0x7f5778102550 2024-09-16T00:36:35.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.177+0000 7f575ffff640 1 -- 172.21.15.112:0/3806059968 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f5768012400 con 0x7f5778102550 2024-09-16T00:36:35.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.177+0000 7f575ffff640 1 --2- 172.21.15.112:0/3806059968 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f576006e530 0x7f57600709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:35.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.177+0000 7f5776ffd640 1 --2- 172.21.15.112:0/3806059968 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f576006e530 0x7f57600709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:35.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.177+0000 7f575ffff640 1 -- 172.21.15.112:0/3806059968 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f576808e320 con 0x7f5778102550 2024-09-16T00:36:35.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.178+0000 7f5776ffd640 1 --2- 172.21.15.112:0/3806059968 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f576006e530 0x7f57600709e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f576c009ac0 tx=0x7f576c009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:35.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.182+0000 7f575ffff640 1 -- 172.21.15.112:0/3806059968 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f57680573b0 con 0x7f5778102550 2024-09-16T00:36:35.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.353+0000 7f577db34640 1 -- 172.21.15.112:0/3806059968 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5778071100 con 0x7f576006e530 2024-09-16T00:36:35.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.362+0000 7f575ffff640 1 -- 172.21.15.112:0/3806059968 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3454 (secure 0 0 0) 0x7f5778071100 con 0x7f576006e530 2024-09-16T00:36:35.363 INFO:teuthology.orchestra.run.smithi112.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T00:36:35.363 INFO:teuthology.orchestra.run.smithi112.stdout:alertmanager.smithi112 smithi112 *:9093,9094 running (9m) 2m ago 18m 24.7M - 0.25.0 c8568f914cd2 d85ea0562022 2024-09-16T00:36:35.363 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi112 smithi112 running (6m) 2m ago 18m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c e4fe58f9702a 2024-09-16T00:36:35.364 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi169 smithi169 running (6m) 2m ago 17m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c 3ba2c81d527f 2024-09-16T00:36:35.364 INFO:teuthology.orchestra.run.smithi112.stdout:grafana.smithi112 smithi112 *:3000 running (8m) 2m ago 18m 82.1M - 9.4.12 2bacad6d85d8 51f9d2013da2 2024-09-16T00:36:35.364 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.hxfwkh smithi112 running (2m) 2m ago 13m 14.2M - 19.1.1-326-g8921c8ef f1455bd0219c 1723069902d6 2024-09-16T00:36:35.364 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.ithwzj smithi112 running (2m) 2m ago 13m 12.2M - 19.1.1-326-g8921c8ef f1455bd0219c e7aad5b81db5 2024-09-16T00:36:35.364 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.gawwgw smithi169 running (2m) 2m ago 13m 15.7M - 19.1.1-326-g8921c8ef f1455bd0219c 469be459ee12 2024-09-16T00:36:35.364 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.lgeqrj smithi169 running (2m) 2m ago 12m 13.3M - 19.1.1-326-g8921c8ef f1455bd0219c 31f0fbd2395d 2024-09-16T00:36:35.364 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi112.xrkkbc smithi112 *:8443,9283,8765 running (9m) 2m ago 19m 504M - 19.1.1-326-g8921c8ef f1455bd0219c a981f7fd2810 2024-09-16T00:36:35.364 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi169.mtrobk smithi169 *:8443,9283,8765 running (8m) 2m ago 17m 442M - 19.1.1-326-g8921c8ef f1455bd0219c 4096e5369c2d 2024-09-16T00:36:35.364 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi112 smithi112 running (7m) 2m ago 19m 49.8M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 5aaec033c042 2024-09-16T00:36:35.364 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi169 smithi169 running (7m) 2m ago 17m 38.9M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 7bc609155f3f 2024-09-16T00:36:35.364 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi112 smithi112 *:9100 running (9m) 2m ago 18m 15.5M - 1.5.0 0da6a335fe13 6404d1d601e2 2024-09-16T00:36:35.364 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi169 smithi169 *:9100 running (9m) 2m ago 17m 15.4M - 1.5.0 0da6a335fe13 5505baf702a6 2024-09-16T00:36:35.364 INFO:teuthology.orchestra.run.smithi112.stdout:osd.0 smithi112 running (6m) 2m ago 16m 105M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 53efc9397017 2024-09-16T00:36:35.365 INFO:teuthology.orchestra.run.smithi112.stdout:osd.1 smithi112 running (5m) 2m ago 16m 98.9M 1557M 19.1.1-326-g8921c8ef f1455bd0219c fe6f1a4a2cf4 2024-09-16T00:36:35.365 INFO:teuthology.orchestra.run.smithi112.stdout:osd.2 smithi112 running (4m) 2m ago 15m 102M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 5f7863712057 2024-09-16T00:36:35.365 INFO:teuthology.orchestra.run.smithi112.stdout:osd.3 smithi169 running (4m) 2m ago 15m 92.1M 2581M 19.1.1-326-g8921c8ef f1455bd0219c ee07b5577810 2024-09-16T00:36:35.365 INFO:teuthology.orchestra.run.smithi112.stdout:osd.4 smithi169 running (3m) 2m ago 15m 103M 2581M 19.1.1-326-g8921c8ef f1455bd0219c 769f026a54eb 2024-09-16T00:36:35.365 INFO:teuthology.orchestra.run.smithi112.stdout:osd.5 smithi169 running (3m) 2m ago 14m 125M 2581M 19.1.1-326-g8921c8ef f1455bd0219c a6007c13cf78 2024-09-16T00:36:35.365 INFO:teuthology.orchestra.run.smithi112.stdout:prometheus.smithi112 smithi112 *:9095 running (8m) 2m ago 18m 64.1M - 2.43.0 a07b618ecd1d 7a1d42e60d2b 2024-09-16T00:36:35.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.364+0000 7f577db34640 1 -- 172.21.15.112:0/3806059968 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f576006e530 msgr2=0x7f57600709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:35.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.364+0000 7f577db34640 1 --2- 172.21.15.112:0/3806059968 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f576006e530 0x7f57600709e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f576c009ac0 tx=0x7f576c009da0 comp rx=0 tx=0).stop 2024-09-16T00:36:35.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.364+0000 7f577db34640 1 -- 172.21.15.112:0/3806059968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5778102550 msgr2=0x7f577806fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:35.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.364+0000 7f577db34640 1 --2- 172.21.15.112:0/3806059968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5778102550 0x7f577806fa10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f576800c790 tx=0x7f57680043c0 comp rx=0 tx=0).stop 2024-09-16T00:36:35.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.365+0000 7f577db34640 1 -- 172.21.15.112:0/3806059968 shutdown_connections 2024-09-16T00:36:35.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.365+0000 7f577db34640 1 --2- 172.21.15.112:0/3806059968 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f576006e530 0x7f57600709e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:35.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.365+0000 7f577db34640 1 --2- 172.21.15.112:0/3806059968 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5778102ef0 0x7f577806ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:35.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.365+0000 7f577db34640 1 --2- 172.21.15.112:0/3806059968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5778102550 0x7f577806fa10 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:35.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.365+0000 7f577db34640 1 -- 172.21.15.112:0/3806059968 >> 172.21.15.112:0/3806059968 conn(0x7f57780f80f0 msgr2=0x7f57780f9150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:35.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.365+0000 7f577db34640 1 -- 172.21.15.112:0/3806059968 shutdown_connections 2024-09-16T00:36:35.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.365+0000 7f577db34640 1 -- 172.21.15.112:0/3806059968 wait complete. 2024-09-16T00:36:35.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.443+0000 7f2f931f4640 1 -- 172.21.15.112:0/2316605444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f8c0fd580 msgr2=0x7f2f8c0fd950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:35.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.443+0000 7f2f931f4640 1 --2- 172.21.15.112:0/2316605444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f8c0fd580 0x7f2f8c0fd950 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2f80004900 tx=0x7f2f80009da0 comp rx=0 tx=0).stop 2024-09-16T00:36:35.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.443+0000 7f2f931f4640 1 -- 172.21.15.112:0/2316605444 shutdown_connections 2024-09-16T00:36:35.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.443+0000 7f2f931f4640 1 --2- 172.21.15.112:0/2316605444 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f8c0fdf20 0x7f2f8c1065f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:35.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.443+0000 7f2f931f4640 1 --2- 172.21.15.112:0/2316605444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f8c0fd580 0x7f2f8c0fd950 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:35.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.443+0000 7f2f931f4640 1 -- 172.21.15.112:0/2316605444 >> 172.21.15.112:0/2316605444 conn(0x7f2f8c0f7060 msgr2=0x7f2f8c0f9470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:35.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.443+0000 7f2f931f4640 1 -- 172.21.15.112:0/2316605444 shutdown_connections 2024-09-16T00:36:35.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.443+0000 7f2f931f4640 1 -- 172.21.15.112:0/2316605444 wait complete. 2024-09-16T00:36:35.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.443+0000 7f2f931f4640 1 Processor -- start 2024-09-16T00:36:35.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.443+0000 7f2f931f4640 1 -- start start 2024-09-16T00:36:35.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.444+0000 7f2f931f4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f8c0fd580 0x7f2f8c0fb440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:35.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.444+0000 7f2f931f4640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f8c0fdf20 0x7f2f8c0fb980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:35.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.444+0000 7f2f931f4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8c0fbec0 con 0x7f2f8c0fd580 2024-09-16T00:36:35.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.444+0000 7f2f931f4640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8c0fc030 con 0x7f2f8c0fdf20 2024-09-16T00:36:35.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.444+0000 7f2f90f90640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f8c0fd580 0x7f2f8c0fb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:35.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.444+0000 7f2f90f90640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f8c0fd580 0x7f2f8c0fb440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58856/0 (socket says 172.21.15.112:58856) 2024-09-16T00:36:35.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.444+0000 7f2f90f90640 1 -- 172.21.15.112:0/1386355394 learned_addr learned my addr 172.21.15.112:0/1386355394 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:36:35.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.444+0000 7f2f8bfff640 1 --2- 172.21.15.112:0/1386355394 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f8c0fdf20 0x7f2f8c0fb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:35.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.444+0000 7f2f90f90640 1 -- 172.21.15.112:0/1386355394 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f8c0fdf20 msgr2=0x7f2f8c0fb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:35.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.444+0000 7f2f90f90640 1 --2- 172.21.15.112:0/1386355394 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f8c0fdf20 0x7f2f8c0fb980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:35.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.444+0000 7f2f90f90640 1 -- 172.21.15.112:0/1386355394 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f80005040 con 0x7f2f8c0fd580 2024-09-16T00:36:35.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.444+0000 7f2f90f90640 1 --2- 172.21.15.112:0/1386355394 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f8c0fd580 0x7f2f8c0fb440 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2f80004960 tx=0x7f2f80009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:35.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.445+0000 7f2f897fa640 1 -- 172.21.15.112:0/1386355394 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f80038070 con 0x7f2f8c0fd580 2024-09-16T00:36:35.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.445+0000 7f2f931f4640 1 -- 172.21.15.112:0/1386355394 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f8c0fc260 con 0x7f2f8c0fd580 2024-09-16T00:36:35.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.445+0000 7f2f897fa640 1 -- 172.21.15.112:0/1386355394 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f2f80008ed0 con 0x7f2f8c0fd580 2024-09-16T00:36:35.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.445+0000 7f2f931f4640 1 -- 172.21.15.112:0/1386355394 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f8c1197e0 con 0x7f2f8c0fd580 2024-09-16T00:36:35.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.445+0000 7f2f897fa640 1 -- 172.21.15.112:0/1386355394 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f800095a0 con 0x7f2f8c0fd580 2024-09-16T00:36:35.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.446+0000 7f2f931f4640 1 -- 172.21.15.112:0/1386355394 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f8c0662d0 con 0x7f2f8c0fd580 2024-09-16T00:36:35.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.446+0000 7f2f897fa640 1 -- 172.21.15.112:0/1386355394 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f2f80030020 con 0x7f2f8c0fd580 2024-09-16T00:36:35.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.447+0000 7f2f897fa640 1 --2- 172.21.15.112:0/1386355394 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f2f7c06e530 0x7f2f7c0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:35.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.447+0000 7f2f897fa640 1 -- 172.21.15.112:0/1386355394 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f2f800ae290 con 0x7f2f8c0fd580 2024-09-16T00:36:35.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.447+0000 7f2f8bfff640 1 --2- 172.21.15.112:0/1386355394 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f2f7c06e530 0x7f2f7c0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:35.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.447+0000 7f2f8bfff640 1 --2- 172.21.15.112:0/1386355394 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f2f7c06e530 0x7f2f7c0709e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c0fcd80 tx=0x7f2f78009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:35.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.451+0000 7f2f897fa640 1 -- 172.21.15.112:0/1386355394 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f2f800772c0 con 0x7f2f8c0fd580 2024-09-16T00:36:35.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.683+0000 7f2f931f4640 1 -- 172.21.15.112:0/1386355394 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f2f8c0fac60 con 0x7f2f8c0fd580 2024-09-16T00:36:35.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.683+0000 7f2f897fa640 1 -- 172.21.15.112:0/1386355394 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f2f8007a980 con 0x7f2f8c0fd580 2024-09-16T00:36:35.685 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:36:35.685 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": { 2024-09-16T00:36:35.685 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:36:35.685 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:36:35.685 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": { 2024-09-16T00:36:35.685 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:36:35.685 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:36:35.685 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": { 2024-09-16T00:36:35.685 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 6 2024-09-16T00:36:35.685 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:36:35.686 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": { 2024-09-16T00:36:35.686 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 4 2024-09-16T00:36:35.686 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:36:35.686 INFO:teuthology.orchestra.run.smithi112.stdout: "overall": { 2024-09-16T00:36:35.686 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 14 2024-09-16T00:36:35.686 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-16T00:36:35.686 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:36:35.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.686+0000 7f2f931f4640 1 -- 172.21.15.112:0/1386355394 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f2f7c06e530 msgr2=0x7f2f7c0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:35.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.686+0000 7f2f931f4640 1 --2- 172.21.15.112:0/1386355394 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f2f7c06e530 0x7f2f7c0709e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c0fcd80 tx=0x7f2f78009040 comp rx=0 tx=0).stop 2024-09-16T00:36:35.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.686+0000 7f2f931f4640 1 -- 172.21.15.112:0/1386355394 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f8c0fd580 msgr2=0x7f2f8c0fb440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:35.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.686+0000 7f2f931f4640 1 --2- 172.21.15.112:0/1386355394 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f8c0fd580 0x7f2f8c0fb440 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2f80004960 tx=0x7f2f80009da0 comp rx=0 tx=0).stop 2024-09-16T00:36:35.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.686+0000 7f2f931f4640 1 -- 172.21.15.112:0/1386355394 shutdown_connections 2024-09-16T00:36:35.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.686+0000 7f2f931f4640 1 --2- 172.21.15.112:0/1386355394 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f2f7c06e530 0x7f2f7c0709e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:35.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.686+0000 7f2f931f4640 1 --2- 172.21.15.112:0/1386355394 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f8c0fdf20 0x7f2f8c0fb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:35.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.686+0000 7f2f931f4640 1 --2- 172.21.15.112:0/1386355394 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f8c0fd580 0x7f2f8c0fb440 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:35.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.686+0000 7f2f931f4640 1 -- 172.21.15.112:0/1386355394 >> 172.21.15.112:0/1386355394 conn(0x7f2f8c0f7060 msgr2=0x7f2f8c0f85d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:35.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.686+0000 7f2f931f4640 1 -- 172.21.15.112:0/1386355394 shutdown_connections 2024-09-16T00:36:35.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.686+0000 7f2f931f4640 1 -- 172.21.15.112:0/1386355394 wait complete. 2024-09-16T00:36:35.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.764+0000 7f7eceb4d640 1 -- 172.21.15.112:0/1650046533 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ec80fac80 msgr2=0x7f7ec80fb0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:35.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.764+0000 7f7eceb4d640 1 --2- 172.21.15.112:0/1650046533 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ec80fac80 0x7f7ec80fb0f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7eb80048d0 tx=0x7f7eb8009da0 comp rx=0 tx=0).stop 2024-09-16T00:36:35.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.764+0000 7f7eceb4d640 1 -- 172.21.15.112:0/1650046533 shutdown_connections 2024-09-16T00:36:35.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.764+0000 7f7eceb4d640 1 --2- 172.21.15.112:0/1650046533 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ec80fac80 0x7f7ec80fb0f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:35.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.764+0000 7f7eceb4d640 1 --2- 172.21.15.112:0/1650046533 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ec8102620 0x7f7ec81029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:35.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.764+0000 7f7eceb4d640 1 -- 172.21.15.112:0/1650046533 >> 172.21.15.112:0/1650046533 conn(0x7f7ec8070ce0 msgr2=0x7f7ec80710e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:35.767 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:36:35 smithi112 ceph-mon[97750]: pgmap v265: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:36:35.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.765+0000 7f7eceb4d640 1 -- 172.21.15.112:0/1650046533 shutdown_connections 2024-09-16T00:36:35.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.765+0000 7f7eceb4d640 1 -- 172.21.15.112:0/1650046533 wait complete. 2024-09-16T00:36:35.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.765+0000 7f7eceb4d640 1 Processor -- start 2024-09-16T00:36:35.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.765+0000 7f7eceb4d640 1 -- start start 2024-09-16T00:36:35.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.766+0000 7f7eceb4d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ec80fac80 0x7f7ec81131e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:35.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.766+0000 7f7eceb4d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ec8102620 0x7f7ec8113720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:35.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.766+0000 7f7eceb4d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec8113d20 con 0x7f7ec80fac80 2024-09-16T00:36:35.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.766+0000 7f7eceb4d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec810d160 con 0x7f7ec8102620 2024-09-16T00:36:35.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.766+0000 7f7ecc8e9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ec80fac80 0x7f7ec81131e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:35.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.766+0000 7f7ecc8e9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ec80fac80 0x7f7ec81131e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58866/0 (socket says 172.21.15.112:58866) 2024-09-16T00:36:35.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.766+0000 7f7ecc8e9640 1 -- 172.21.15.112:0/3635801242 learned_addr learned my addr 172.21.15.112:0/3635801242 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:36:35.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.766+0000 7f7ebffff640 1 --2- 172.21.15.112:0/3635801242 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ec8102620 0x7f7ec8113720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:35.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.766+0000 7f7ecc8e9640 1 -- 172.21.15.112:0/3635801242 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ec8102620 msgr2=0x7f7ec8113720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:35.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.766+0000 7f7ecc8e9640 1 --2- 172.21.15.112:0/3635801242 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ec8102620 0x7f7ec8113720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:35.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.766+0000 7f7ecc8e9640 1 -- 172.21.15.112:0/3635801242 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eb8005040 con 0x7f7ec80fac80 2024-09-16T00:36:35.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.766+0000 7f7ecc8e9640 1 --2- 172.21.15.112:0/3635801242 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ec80fac80 0x7f7ec81131e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7ec400db00 tx=0x7f7ec4004e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:35.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.766+0000 7f7ebd7fa640 1 -- 172.21.15.112:0/3635801242 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ec4017070 con 0x7f7ec80fac80 2024-09-16T00:36:35.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.766+0000 7f7ebd7fa640 1 -- 172.21.15.112:0/3635801242 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f7ec400f040 con 0x7f7ec80fac80 2024-09-16T00:36:35.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.766+0000 7f7ebd7fa640 1 -- 172.21.15.112:0/3635801242 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ec4003940 con 0x7f7ec80fac80 2024-09-16T00:36:35.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.767+0000 7f7eceb4d640 1 -- 172.21.15.112:0/3635801242 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ec810d440 con 0x7f7ec80fac80 2024-09-16T00:36:35.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.767+0000 7f7eceb4d640 1 -- 172.21.15.112:0/3635801242 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ec810d960 con 0x7f7ec80fac80 2024-09-16T00:36:35.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.768+0000 7f7eceb4d640 1 -- 172.21.15.112:0/3635801242 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ec80657b0 con 0x7f7ec80fac80 2024-09-16T00:36:35.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.768+0000 7f7ebd7fa640 1 -- 172.21.15.112:0/3635801242 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f7ec4003b70 con 0x7f7ec80fac80 2024-09-16T00:36:35.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.768+0000 7f7ebd7fa640 1 --2- 172.21.15.112:0/3635801242 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f7eb006e530 0x7f7eb00709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:35.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.769+0000 7f7ebd7fa640 1 -- 172.21.15.112:0/3635801242 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f7ec408e450 con 0x7f7ec80fac80 2024-09-16T00:36:35.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.769+0000 7f7ebffff640 1 --2- 172.21.15.112:0/3635801242 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f7eb006e530 0x7f7eb00709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:35.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.769+0000 7f7ebffff640 1 --2- 172.21.15.112:0/3635801242 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f7eb006e530 0x7f7eb00709e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7ec80f8f90 tx=0x7f7eb8035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:35.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.773+0000 7f7ebd7fa640 1 -- 172.21.15.112:0/3635801242 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f7ec405aef0 con 0x7f7ec80fac80 2024-09-16T00:36:35.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.970+0000 7f7eceb4d640 1 -- 172.21.15.112:0/3635801242 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f7ec804b720 con 0x7f7ec80fac80 2024-09-16T00:36:35.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.971+0000 7f7ebd7fa640 1 -- 172.21.15.112:0/3635801242 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1914 (secure 0 0 0) 0x7f7ec4003e20 con 0x7f7ec80fac80 2024-09-16T00:36:35.972 INFO:teuthology.orchestra.run.smithi112.stdout:e34 2024-09-16T00:36:35.972 INFO:teuthology.orchestra.run.smithi112.stdout:btime 2024-09-16T00:34:26:619254+0000 2024-09-16T00:36:35.972 INFO:teuthology.orchestra.run.smithi112.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T00:36:35.973 INFO:teuthology.orchestra.run.smithi112.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:36:35.973 INFO:teuthology.orchestra.run.smithi112.stdout:legacy client fscid: 1 2024-09-16T00:36:35.973 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:36:35.973 INFO:teuthology.orchestra.run.smithi112.stdout:Filesystem 'cephfs' (1) 2024-09-16T00:36:35.973 INFO:teuthology.orchestra.run.smithi112.stdout:fs_name cephfs 2024-09-16T00:36:35.973 INFO:teuthology.orchestra.run.smithi112.stdout:epoch 25 2024-09-16T00:36:35.973 INFO:teuthology.orchestra.run.smithi112.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-16T00:36:35.973 INFO:teuthology.orchestra.run.smithi112.stdout:created 2024-09-16T00:23:22.328199+0000 2024-09-16T00:36:35.973 INFO:teuthology.orchestra.run.smithi112.stdout:modified 2024-09-16T00:33:29.012991+0000 2024-09-16T00:36:35.973 INFO:teuthology.orchestra.run.smithi112.stdout:tableserver 0 2024-09-16T00:36:35.973 INFO:teuthology.orchestra.run.smithi112.stdout:root 0 2024-09-16T00:36:35.973 INFO:teuthology.orchestra.run.smithi112.stdout:session_timeout 60 2024-09-16T00:36:35.973 INFO:teuthology.orchestra.run.smithi112.stdout:session_autoclose 300 2024-09-16T00:36:35.974 INFO:teuthology.orchestra.run.smithi112.stdout:max_file_size 1099511627776 2024-09-16T00:36:35.974 INFO:teuthology.orchestra.run.smithi112.stdout:max_xattr_size 65536 2024-09-16T00:36:35.974 INFO:teuthology.orchestra.run.smithi112.stdout:required_client_features {} 2024-09-16T00:36:35.974 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure 0 2024-09-16T00:36:35.974 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure_osd_epoch 82 2024-09-16T00:36:35.974 INFO:teuthology.orchestra.run.smithi112.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:36:35.974 INFO:teuthology.orchestra.run.smithi112.stdout:max_mds 2 2024-09-16T00:36:35.974 INFO:teuthology.orchestra.run.smithi112.stdout:in 0,1 2024-09-16T00:36:35.974 INFO:teuthology.orchestra.run.smithi112.stdout:up {} 2024-09-16T00:36:35.974 INFO:teuthology.orchestra.run.smithi112.stdout:failed 0,1 2024-09-16T00:36:35.974 INFO:teuthology.orchestra.run.smithi112.stdout:damaged 2024-09-16T00:36:35.974 INFO:teuthology.orchestra.run.smithi112.stdout:stopped 2024-09-16T00:36:35.974 INFO:teuthology.orchestra.run.smithi112.stdout:data_pools [3] 2024-09-16T00:36:35.975 INFO:teuthology.orchestra.run.smithi112.stdout:metadata_pool 2 2024-09-16T00:36:35.975 INFO:teuthology.orchestra.run.smithi112.stdout:inline_data enabled 2024-09-16T00:36:35.975 INFO:teuthology.orchestra.run.smithi112.stdout:balancer 2024-09-16T00:36:35.975 INFO:teuthology.orchestra.run.smithi112.stdout:bal_rank_mask -1 2024-09-16T00:36:35.975 INFO:teuthology.orchestra.run.smithi112.stdout:standby_count_wanted 1 2024-09-16T00:36:35.975 INFO:teuthology.orchestra.run.smithi112.stdout:qdb_cluster leader: 0 members: 2024-09-16T00:36:35.975 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:36:35.975 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:36:35.975 INFO:teuthology.orchestra.run.smithi112.stdout:Standby daemons: 2024-09-16T00:36:35.975 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:36:35.975 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.ithwzj{-1:34392} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.112:6826/476229221,v1:172.21.15.112:6827/476229221] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:36:35.975 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.hxfwkh{-1:34404} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.112:6828/2356899876,v1:172.21.15.112:6829/2356899876] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:36:35.975 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.gawwgw{-1:44343} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.169:6824/1635329957,v1:172.21.15.169:6825/1635329957] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:36:35.975 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.lgeqrj{-1:44347} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.169:6826/184810031,v1:172.21.15.169:6827/184810031] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:36:35.976 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 34 2024-09-16T00:36:35.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.973+0000 7f7eceb4d640 1 -- 172.21.15.112:0/3635801242 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f7eb006e530 msgr2=0x7f7eb00709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:35.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.973+0000 7f7eceb4d640 1 --2- 172.21.15.112:0/3635801242 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f7eb006e530 0x7f7eb00709e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7ec80f8f90 tx=0x7f7eb8035040 comp rx=0 tx=0).stop 2024-09-16T00:36:35.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.973+0000 7f7eceb4d640 1 -- 172.21.15.112:0/3635801242 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ec80fac80 msgr2=0x7f7ec81131e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:35.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.973+0000 7f7eceb4d640 1 --2- 172.21.15.112:0/3635801242 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ec80fac80 0x7f7ec81131e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7ec400db00 tx=0x7f7ec4004e60 comp rx=0 tx=0).stop 2024-09-16T00:36:35.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.973+0000 7f7eceb4d640 1 -- 172.21.15.112:0/3635801242 shutdown_connections 2024-09-16T00:36:35.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.973+0000 7f7eceb4d640 1 --2- 172.21.15.112:0/3635801242 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f7eb006e530 0x7f7eb00709e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:35.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.973+0000 7f7eceb4d640 1 --2- 172.21.15.112:0/3635801242 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ec8102620 0x7f7ec8113720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:35.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.973+0000 7f7eceb4d640 1 --2- 172.21.15.112:0/3635801242 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ec80fac80 0x7f7ec81131e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:35.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.973+0000 7f7eceb4d640 1 -- 172.21.15.112:0/3635801242 >> 172.21.15.112:0/3635801242 conn(0x7f7ec8070ce0 msgr2=0x7f7ec8105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:35.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.973+0000 7f7eceb4d640 1 -- 172.21.15.112:0/3635801242 shutdown_connections 2024-09-16T00:36:35.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:35.973+0000 7f7eceb4d640 1 -- 172.21.15.112:0/3635801242 wait complete. 2024-09-16T00:36:36.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:36:35 smithi169 ceph-mon[77626]: pgmap v265: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:36:36.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.052+0000 7f3bca63c640 1 -- 172.21.15.112:0/4267662467 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3bc4102620 msgr2=0x7f3bc41029f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:36.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.052+0000 7f3bca63c640 1 --2- 172.21.15.112:0/4267662467 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3bc4102620 0x7f3bc41029f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3bb40048d0 tx=0x7f3bb4009da0 comp rx=0 tx=0).stop 2024-09-16T00:36:36.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.053+0000 7f3bca63c640 1 -- 172.21.15.112:0/4267662467 shutdown_connections 2024-09-16T00:36:36.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.053+0000 7f3bca63c640 1 --2- 172.21.15.112:0/4267662467 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3bc40fac80 0x7f3bc40fb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:36.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.053+0000 7f3bca63c640 1 --2- 172.21.15.112:0/4267662467 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3bc4102620 0x7f3bc41029f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:36.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.053+0000 7f3bca63c640 1 -- 172.21.15.112:0/4267662467 >> 172.21.15.112:0/4267662467 conn(0x7f3bc4070ce0 msgr2=0x7f3bc40710e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:36.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.053+0000 7f3bca63c640 1 -- 172.21.15.112:0/4267662467 shutdown_connections 2024-09-16T00:36:36.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.053+0000 7f3bca63c640 1 -- 172.21.15.112:0/4267662467 wait complete. 2024-09-16T00:36:36.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.054+0000 7f3bca63c640 1 Processor -- start 2024-09-16T00:36:36.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.054+0000 7f3bca63c640 1 -- start start 2024-09-16T00:36:36.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.054+0000 7f3bca63c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3bc40fac80 0x7f3bc41153b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:36.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.054+0000 7f3bca63c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3bc4102620 0x7f3bc41158f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:36.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.054+0000 7f3bca63c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc4115ef0 con 0x7f3bc4102620 2024-09-16T00:36:36.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.054+0000 7f3bca63c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc410f330 con 0x7f3bc40fac80 2024-09-16T00:36:36.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.054+0000 7f3bc37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3bc4102620 0x7f3bc41158f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:36.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.054+0000 7f3bc37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3bc4102620 0x7f3bc41158f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58888/0 (socket says 172.21.15.112:58888) 2024-09-16T00:36:36.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.054+0000 7f3bc37fe640 1 -- 172.21.15.112:0/804799718 learned_addr learned my addr 172.21.15.112:0/804799718 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:36:36.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.054+0000 7f3bc3fff640 1 --2- 172.21.15.112:0/804799718 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3bc40fac80 0x7f3bc41153b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:36.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.055+0000 7f3bc37fe640 1 -- 172.21.15.112:0/804799718 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3bc40fac80 msgr2=0x7f3bc41153b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:36.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.055+0000 7f3bc37fe640 1 --2- 172.21.15.112:0/804799718 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3bc40fac80 0x7f3bc41153b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:36.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.055+0000 7f3bc37fe640 1 -- 172.21.15.112:0/804799718 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bb4005040 con 0x7f3bc4102620 2024-09-16T00:36:36.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.055+0000 7f3bc3fff640 1 --2- 172.21.15.112:0/804799718 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3bc40fac80 0x7f3bc41153b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:36:36.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.055+0000 7f3bc37fe640 1 --2- 172.21.15.112:0/804799718 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3bc4102620 0x7f3bc41158f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3bac00c810 tx=0x7f3bac0043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:36.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.055+0000 7f3bc0ff9640 1 -- 172.21.15.112:0/804799718 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3bac00a820 con 0x7f3bc4102620 2024-09-16T00:36:36.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.055+0000 7f3bc0ff9640 1 -- 172.21.15.112:0/804799718 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f3bac00ae20 con 0x7f3bc4102620 2024-09-16T00:36:36.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.055+0000 7f3bca63c640 1 -- 172.21.15.112:0/804799718 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bc410f610 con 0x7f3bc4102620 2024-09-16T00:36:36.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.055+0000 7f3bc0ff9640 1 -- 172.21.15.112:0/804799718 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3bac0115c0 con 0x7f3bc4102620 2024-09-16T00:36:36.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.055+0000 7f3bca63c640 1 -- 172.21.15.112:0/804799718 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bc410fbb0 con 0x7f3bc4102620 2024-09-16T00:36:36.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.056+0000 7f3bca63c640 1 -- 172.21.15.112:0/804799718 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bc4065790 con 0x7f3bc4102620 2024-09-16T00:36:36.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.057+0000 7f3bc0ff9640 1 -- 172.21.15.112:0/804799718 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f3bac012840 con 0x7f3bc4102620 2024-09-16T00:36:36.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.057+0000 7f3bc0ff9640 1 --2- 172.21.15.112:0/804799718 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f3bb006e530 0x7f3bb00709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:36.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.057+0000 7f3bc0ff9640 1 -- 172.21.15.112:0/804799718 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f3bac08e800 con 0x7f3bc4102620 2024-09-16T00:36:36.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.057+0000 7f3bc3fff640 1 --2- 172.21.15.112:0/804799718 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f3bb006e530 0x7f3bb00709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:36.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.058+0000 7f3bc3fff640 1 --2- 172.21.15.112:0/804799718 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f3bb006e530 0x7f3bb00709e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4009ac0 tx=0x7f3bb4035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:36.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.062+0000 7f3bc0ff9640 1 -- 172.21.15.112:0/804799718 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f3bac0577f0 con 0x7f3bc4102620 2024-09-16T00:36:36.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.241+0000 7f3bca63c640 1 -- 172.21.15.112:0/804799718 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3bc4109620 con 0x7f3bb006e530 2024-09-16T00:36:36.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.243+0000 7f3bc0ff9640 1 -- 172.21.15.112:0/804799718 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f3bb402ab20 con 0x7f3bb006e530 2024-09-16T00:36:36.244 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:36:36.244 INFO:teuthology.orchestra.run.smithi112.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80", 2024-09-16T00:36:36.244 INFO:teuthology.orchestra.run.smithi112.stdout: "in_progress": true, 2024-09-16T00:36:36.244 INFO:teuthology.orchestra.run.smithi112.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T00:36:36.244 INFO:teuthology.orchestra.run.smithi112.stdout: "services_complete": [ 2024-09-16T00:36:36.244 INFO:teuthology.orchestra.run.smithi112.stdout: "mds", 2024-09-16T00:36:36.244 INFO:teuthology.orchestra.run.smithi112.stdout: "osd", 2024-09-16T00:36:36.245 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr", 2024-09-16T00:36:36.245 INFO:teuthology.orchestra.run.smithi112.stdout: "mon", 2024-09-16T00:36:36.245 INFO:teuthology.orchestra.run.smithi112.stdout: "crash" 2024-09-16T00:36:36.245 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-16T00:36:36.245 INFO:teuthology.orchestra.run.smithi112.stdout: "progress": "16/21 daemons upgraded", 2024-09-16T00:36:36.245 INFO:teuthology.orchestra.run.smithi112.stdout: "message": "Currently upgrading mds daemons", 2024-09-16T00:36:36.245 INFO:teuthology.orchestra.run.smithi112.stdout: "is_paused": false 2024-09-16T00:36:36.245 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:36:36.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.245+0000 7f3bca63c640 1 -- 172.21.15.112:0/804799718 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f3bb006e530 msgr2=0x7f3bb00709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:36.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.245+0000 7f3bca63c640 1 --2- 172.21.15.112:0/804799718 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f3bb006e530 0x7f3bb00709e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4009ac0 tx=0x7f3bb4035040 comp rx=0 tx=0).stop 2024-09-16T00:36:36.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.245+0000 7f3bca63c640 1 -- 172.21.15.112:0/804799718 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3bc4102620 msgr2=0x7f3bc41158f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:36.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.245+0000 7f3bca63c640 1 --2- 172.21.15.112:0/804799718 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3bc4102620 0x7f3bc41158f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3bac00c810 tx=0x7f3bac0043c0 comp rx=0 tx=0).stop 2024-09-16T00:36:36.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.245+0000 7f3bca63c640 1 -- 172.21.15.112:0/804799718 shutdown_connections 2024-09-16T00:36:36.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.245+0000 7f3bca63c640 1 --2- 172.21.15.112:0/804799718 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f3bb006e530 0x7f3bb00709e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:36.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.245+0000 7f3bca63c640 1 --2- 172.21.15.112:0/804799718 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3bc4102620 0x7f3bc41158f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:36.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.245+0000 7f3bca63c640 1 --2- 172.21.15.112:0/804799718 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3bc40fac80 0x7f3bc41153b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:36.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.245+0000 7f3bca63c640 1 -- 172.21.15.112:0/804799718 >> 172.21.15.112:0/804799718 conn(0x7f3bc4070ce0 msgr2=0x7f3bc40f8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:36.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.245+0000 7f3bca63c640 1 -- 172.21.15.112:0/804799718 shutdown_connections 2024-09-16T00:36:36.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.245+0000 7f3bca63c640 1 -- 172.21.15.112:0/804799718 wait complete. 2024-09-16T00:36:36.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.322+0000 7fc327989640 1 -- 172.21.15.112:0/1551740104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc320102ed0 msgr2=0x7fc3200fc590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:36.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.322+0000 7fc327989640 1 --2- 172.21.15.112:0/1551740104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc320102ed0 0x7fc3200fc590 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc3100048d0 tx=0x7fc310009da0 comp rx=0 tx=0).stop 2024-09-16T00:36:36.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.322+0000 7fc327989640 1 -- 172.21.15.112:0/1551740104 shutdown_connections 2024-09-16T00:36:36.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.322+0000 7fc327989640 1 --2- 172.21.15.112:0/1551740104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc320102ed0 0x7fc3200fc590 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:36.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.322+0000 7fc327989640 1 --2- 172.21.15.112:0/1551740104 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc320102530 0x7fc320102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:36.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.322+0000 7fc327989640 1 -- 172.21.15.112:0/1551740104 >> 172.21.15.112:0/1551740104 conn(0x7fc3200f80f0 msgr2=0x7fc3200fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:36.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.323+0000 7fc327989640 1 -- 172.21.15.112:0/1551740104 shutdown_connections 2024-09-16T00:36:36.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.323+0000 7fc327989640 1 -- 172.21.15.112:0/1551740104 wait complete. 2024-09-16T00:36:36.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.323+0000 7fc327989640 1 Processor -- start 2024-09-16T00:36:36.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.323+0000 7fc327989640 1 -- start start 2024-09-16T00:36:36.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.324+0000 7fc327989640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc320102530 0x7fc3201131d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:36.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.324+0000 7fc327989640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc320102ed0 0x7fc320113710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:36.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.324+0000 7fc327989640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc320113d10 con 0x7fc320102ed0 2024-09-16T00:36:36.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.324+0000 7fc327989640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc32010d150 con 0x7fc320102530 2024-09-16T00:36:36.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.324+0000 7fc324f24640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc320102ed0 0x7fc320113710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:36.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.324+0000 7fc324f24640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc320102ed0 0x7fc320113710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58900/0 (socket says 172.21.15.112:58900) 2024-09-16T00:36:36.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.324+0000 7fc324f24640 1 -- 172.21.15.112:0/2054362570 learned_addr learned my addr 172.21.15.112:0/2054362570 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:36:36.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.324+0000 7fc325725640 1 --2- 172.21.15.112:0/2054362570 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc320102530 0x7fc3201131d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:36.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.324+0000 7fc324f24640 1 -- 172.21.15.112:0/2054362570 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc320102530 msgr2=0x7fc3201131d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:36.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.324+0000 7fc324f24640 1 --2- 172.21.15.112:0/2054362570 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc320102530 0x7fc3201131d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:36.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.324+0000 7fc324f24640 1 -- 172.21.15.112:0/2054362570 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc310005040 con 0x7fc320102ed0 2024-09-16T00:36:36.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.324+0000 7fc325725640 1 --2- 172.21.15.112:0/2054362570 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc320102530 0x7fc3201131d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:36:36.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.324+0000 7fc324f24640 1 --2- 172.21.15.112:0/2054362570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc320102ed0 0x7fc320113710 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc3100043d0 tx=0x7fc310009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:36.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.324+0000 7fc315ffb640 1 -- 172.21.15.112:0/2054362570 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc310038070 con 0x7fc320102ed0 2024-09-16T00:36:36.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.324+0000 7fc327989640 1 -- 172.21.15.112:0/2054362570 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc32010d3d0 con 0x7fc320102ed0 2024-09-16T00:36:36.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.324+0000 7fc315ffb640 1 -- 172.21.15.112:0/2054362570 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fc3100096e0 con 0x7fc320102ed0 2024-09-16T00:36:36.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.324+0000 7fc315ffb640 1 -- 172.21.15.112:0/2054362570 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc31002c8a0 con 0x7fc320102ed0 2024-09-16T00:36:36.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.325+0000 7fc327989640 1 -- 172.21.15.112:0/2054362570 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc32010d890 con 0x7fc320102ed0 2024-09-16T00:36:36.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.326+0000 7fc327989640 1 -- 172.21.15.112:0/2054362570 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3200fdcb0 con 0x7fc320102ed0 2024-09-16T00:36:36.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.326+0000 7fc315ffb640 1 -- 172.21.15.112:0/2054362570 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fc310030020 con 0x7fc320102ed0 2024-09-16T00:36:36.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.327+0000 7fc315ffb640 1 --2- 172.21.15.112:0/2054362570 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fc30c06e530 0x7fc30c0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:36:36.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.327+0000 7fc315ffb640 1 -- 172.21.15.112:0/2054362570 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7fc3100ae0b0 con 0x7fc320102ed0 2024-09-16T00:36:36.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.327+0000 7fc325725640 1 --2- 172.21.15.112:0/2054362570 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fc30c06e530 0x7fc30c0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:36:36.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.327+0000 7fc325725640 1 --2- 172.21.15.112:0/2054362570 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fc30c06e530 0x7fc30c0709e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc31c009af0 tx=0x7fc31c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:36:36.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.331+0000 7fc315ffb640 1 -- 172.21.15.112:0/2054362570 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fc310077020 con 0x7fc320102ed0 2024-09-16T00:36:36.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.561+0000 7fc327989640 1 -- 172.21.15.112:0/2054362570 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fc320109540 con 0x7fc320102ed0 2024-09-16T00:36:36.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.561+0000 7fc315ffb640 1 -- 172.21.15.112:0/2054362570 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7fc31007a6e0 con 0x7fc320102ed0 2024-09-16T00:36:36.563 INFO:teuthology.orchestra.run.smithi112.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-16T00:36:36.563 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-16T00:36:36.563 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs is degraded 2024-09-16T00:36:36.563 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T00:36:36.563 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T00:36:36.563 INFO:teuthology.orchestra.run.smithi112.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-16T00:36:36.563 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-16T00:36:36.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.564+0000 7fc327989640 1 -- 172.21.15.112:0/2054362570 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fc30c06e530 msgr2=0x7fc30c0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:36.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.564+0000 7fc327989640 1 --2- 172.21.15.112:0/2054362570 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fc30c06e530 0x7fc30c0709e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc31c009af0 tx=0x7fc31c00b040 comp rx=0 tx=0).stop 2024-09-16T00:36:36.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.564+0000 7fc327989640 1 -- 172.21.15.112:0/2054362570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc320102ed0 msgr2=0x7fc320113710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:36:36.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.564+0000 7fc327989640 1 --2- 172.21.15.112:0/2054362570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc320102ed0 0x7fc320113710 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc3100043d0 tx=0x7fc310009da0 comp rx=0 tx=0).stop 2024-09-16T00:36:36.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.564+0000 7fc327989640 1 -- 172.21.15.112:0/2054362570 shutdown_connections 2024-09-16T00:36:36.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.564+0000 7fc327989640 1 --2- 172.21.15.112:0/2054362570 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fc30c06e530 0x7fc30c0709e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:36.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.564+0000 7fc327989640 1 --2- 172.21.15.112:0/2054362570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc320102ed0 0x7fc320113710 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:36.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.564+0000 7fc327989640 1 --2- 172.21.15.112:0/2054362570 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc320102530 0x7fc3201131d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:36:36.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.564+0000 7fc327989640 1 -- 172.21.15.112:0/2054362570 >> 172.21.15.112:0/2054362570 conn(0x7fc3200f80f0 msgr2=0x7fc3200f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:36:36.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.564+0000 7fc327989640 1 -- 172.21.15.112:0/2054362570 shutdown_connections 2024-09-16T00:36:36.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:36:36.564+0000 7fc327989640 1 -- 172.21.15.112:0/2054362570 wait complete. 2024-09-16T00:36:37.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:36:36 smithi169 ceph-mon[77626]: from='client.34548 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:36:37.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:36:36 smithi169 ceph-mon[77626]: from='client.34552 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:36:37.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:36:36 smithi169 ceph-mon[77626]: from='client.34556 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:36:37.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:36:36 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/1386355394' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:36:37.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:36:36 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/3635801242' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:36:37.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:36:36 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/2054362570' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:36:37.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:36:36 smithi112 ceph-mon[97750]: from='client.34548 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:36:37.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:36:36 smithi112 ceph-mon[97750]: from='client.34552 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:36:37.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:36:36 smithi112 ceph-mon[97750]: from='client.34556 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:36:37.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:36:36 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/1386355394' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:36:37.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:36:36 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/3635801242' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:36:37.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:36:36 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/2054362570' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:36:38.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:36:37 smithi169 ceph-mon[77626]: from='client.34568 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:36:38.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:36:37 smithi169 ceph-mon[77626]: pgmap v266: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:36:38.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:36:37 smithi112 ceph-mon[97750]: from='client.34568 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:36:38.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:36:37 smithi112 ceph-mon[97750]: pgmap v266: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:36:40.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:36:39 smithi169 ceph-mon[77626]: pgmap v267: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:36:40.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:36:39 smithi112 ceph-mon[97750]: pgmap v267: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:36:42.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:36:41 smithi169 ceph-mon[77626]: pgmap v268: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:36:42.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:36:41 smithi112 ceph-mon[97750]: pgmap v268: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:36:44.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:36:43 smithi169 ceph-mon[77626]: pgmap v269: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:36:44.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:36:43 smithi112 ceph-mon[97750]: pgmap v269: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:36:46.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:36:45 smithi169 ceph-mon[77626]: pgmap v270: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:36:46.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:36:45 smithi112 ceph-mon[97750]: pgmap v270: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:36:47.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:36:46 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:36:47.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:36:46 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:36:48.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:36:47 smithi169 ceph-mon[77626]: pgmap v271: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:36:48.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:36:47 smithi112 ceph-mon[97750]: pgmap v271: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:36:50.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:36:49 smithi169 ceph-mon[77626]: pgmap v272: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:36:50.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:36:49 smithi112 ceph-mon[97750]: pgmap v272: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:36:52.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:36:51 smithi169 ceph-mon[77626]: pgmap v273: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:36:52.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:36:51 smithi112 ceph-mon[97750]: pgmap v273: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:36:54.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:36:53 smithi169 ceph-mon[77626]: pgmap v274: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:36:54.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:36:53 smithi112 ceph-mon[97750]: pgmap v274: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:36:56.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:36:55 smithi169 ceph-mon[77626]: pgmap v275: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:36:56.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:36:55 smithi112 ceph-mon[97750]: pgmap v275: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:36:58.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:36:57 smithi169 ceph-mon[77626]: pgmap v276: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:36:58.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:36:57 smithi112 ceph-mon[97750]: pgmap v276: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:37:00.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:36:59 smithi169 ceph-mon[77626]: pgmap v277: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:37:00.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:36:59 smithi112 ceph-mon[97750]: pgmap v277: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:37:02.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:37:01 smithi169 ceph-mon[77626]: pgmap v278: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:37:02.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:37:01 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:37:02.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:37:01 smithi112 ceph-mon[97750]: pgmap v278: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:37:02.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:37:01 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:37:04.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:37:03 smithi169 ceph-mon[77626]: pgmap v279: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:37:04.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:37:03 smithi112 ceph-mon[97750]: pgmap v279: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:37:06.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:37:05 smithi169 ceph-mon[77626]: pgmap v280: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:37:06.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:37:05 smithi112 ceph-mon[97750]: pgmap v280: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:37:06.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.656+0000 7f114cd53640 1 -- 172.21.15.112:0/321426860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1148101480 msgr2=0x7f1148101850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:06.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.656+0000 7f114cd53640 1 --2- 172.21.15.112:0/321426860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1148101480 0x7f1148101850 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1138009c50 tx=0x7f1138004d10 comp rx=0 tx=0).stop 2024-09-16T00:37:06.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.656+0000 7f114cd53640 1 -- 172.21.15.112:0/321426860 shutdown_connections 2024-09-16T00:37:06.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.656+0000 7f114cd53640 1 --2- 172.21.15.112:0/321426860 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1148101e20 0x7f11480fbd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:06.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.656+0000 7f114cd53640 1 --2- 172.21.15.112:0/321426860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1148101480 0x7f1148101850 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:06.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.656+0000 7f114cd53640 1 -- 172.21.15.112:0/321426860 >> 172.21.15.112:0/321426860 conn(0x7f1148070c80 msgr2=0x7f1148071080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:06.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.656+0000 7f114cd53640 1 -- 172.21.15.112:0/321426860 shutdown_connections 2024-09-16T00:37:06.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.656+0000 7f114cd53640 1 -- 172.21.15.112:0/321426860 wait complete. 2024-09-16T00:37:06.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.656+0000 7f114cd53640 1 Processor -- start 2024-09-16T00:37:06.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.656+0000 7f114cd53640 1 -- start start 2024-09-16T00:37:06.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.657+0000 7f114cd53640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1148101480 0x7f114810e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:06.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.657+0000 7f114cd53640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1148101e20 0x7f114810e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:06.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.657+0000 7f114cd53640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f114810ed00 con 0x7f1148101480 2024-09-16T00:37:06.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.657+0000 7f114cd53640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f114810ee70 con 0x7f1148101e20 2024-09-16T00:37:06.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.657+0000 7f114659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1148101480 0x7f114810e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:06.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.657+0000 7f114659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1148101480 0x7f114810e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50474/0 (socket says 172.21.15.112:50474) 2024-09-16T00:37:06.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.657+0000 7f114659c640 1 -- 172.21.15.112:0/621843949 learned_addr learned my addr 172.21.15.112:0/621843949 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:37:06.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.657+0000 7f1145d9b640 1 --2- 172.21.15.112:0/621843949 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1148101e20 0x7f114810e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:06.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.657+0000 7f114659c640 1 -- 172.21.15.112:0/621843949 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1148101e20 msgr2=0x7f114810e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:06.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.657+0000 7f114659c640 1 --2- 172.21.15.112:0/621843949 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1148101e20 0x7f114810e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:06.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.657+0000 7f114659c640 1 -- 172.21.15.112:0/621843949 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11380098b0 con 0x7f1148101480 2024-09-16T00:37:06.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.657+0000 7f114659c640 1 --2- 172.21.15.112:0/621843949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1148101480 0x7f114810e1c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1138004a30 tx=0x7f1138004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:06.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.657+0000 7f1136ffd640 1 -- 172.21.15.112:0/621843949 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1138009690 con 0x7f1148101480 2024-09-16T00:37:06.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.657+0000 7f1136ffd640 1 -- 172.21.15.112:0/621843949 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f1138032b10 con 0x7f1148101480 2024-09-16T00:37:06.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.657+0000 7f1136ffd640 1 -- 172.21.15.112:0/621843949 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f113803daa0 con 0x7f1148101480 2024-09-16T00:37:06.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.657+0000 7f114cd53640 1 -- 172.21.15.112:0/621843949 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1148117230 con 0x7f1148101480 2024-09-16T00:37:06.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.658+0000 7f114cd53640 1 -- 172.21.15.112:0/621843949 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11481176f0 con 0x7f1148101480 2024-09-16T00:37:06.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.659+0000 7f114cd53640 1 -- 172.21.15.112:0/621843949 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11480fcd80 con 0x7f1148101480 2024-09-16T00:37:06.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.659+0000 7f1136ffd640 1 -- 172.21.15.112:0/621843949 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f113802f030 con 0x7f1148101480 2024-09-16T00:37:06.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.660+0000 7f1136ffd640 1 --2- 172.21.15.112:0/621843949 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f113006e530 0x7f11300709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:06.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.660+0000 7f1136ffd640 1 -- 172.21.15.112:0/621843949 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f11380af450 con 0x7f1148101480 2024-09-16T00:37:06.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.660+0000 7f1145d9b640 1 --2- 172.21.15.112:0/621843949 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f113006e530 0x7f11300709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:06.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.660+0000 7f1145d9b640 1 --2- 172.21.15.112:0/621843949 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f113006e530 0x7f11300709e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f11480fb680 tx=0x7f113c009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:06.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.664+0000 7f1136ffd640 1 -- 172.21.15.112:0/621843949 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f11380783c0 con 0x7f1148101480 2024-09-16T00:37:06.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.850+0000 7f114cd53640 1 -- 172.21.15.112:0/621843949 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f11480fb430 con 0x7f113006e530 2024-09-16T00:37:06.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.851+0000 7f1136ffd640 1 -- 172.21.15.112:0/621843949 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f11480fb430 con 0x7f113006e530 2024-09-16T00:37:06.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.853+0000 7f114cd53640 1 -- 172.21.15.112:0/621843949 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f113006e530 msgr2=0x7f11300709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:06.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.853+0000 7f114cd53640 1 --2- 172.21.15.112:0/621843949 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f113006e530 0x7f11300709e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f11480fb680 tx=0x7f113c009040 comp rx=0 tx=0).stop 2024-09-16T00:37:06.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.853+0000 7f114cd53640 1 -- 172.21.15.112:0/621843949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1148101480 msgr2=0x7f114810e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:06.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.853+0000 7f114cd53640 1 --2- 172.21.15.112:0/621843949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1148101480 0x7f114810e1c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1138004a30 tx=0x7f1138004d10 comp rx=0 tx=0).stop 2024-09-16T00:37:06.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.853+0000 7f114cd53640 1 -- 172.21.15.112:0/621843949 shutdown_connections 2024-09-16T00:37:06.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.853+0000 7f114cd53640 1 --2- 172.21.15.112:0/621843949 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f113006e530 0x7f11300709e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:06.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.853+0000 7f114cd53640 1 --2- 172.21.15.112:0/621843949 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1148101e20 0x7f114810e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:06.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.853+0000 7f114cd53640 1 --2- 172.21.15.112:0/621843949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1148101480 0x7f114810e1c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:06.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.853+0000 7f114cd53640 1 -- 172.21.15.112:0/621843949 >> 172.21.15.112:0/621843949 conn(0x7f1148070c80 msgr2=0x7f1148104220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:06.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.853+0000 7f114cd53640 1 -- 172.21.15.112:0/621843949 shutdown_connections 2024-09-16T00:37:06.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.853+0000 7f114cd53640 1 -- 172.21.15.112:0/621843949 wait complete. 2024-09-16T00:37:06.862 INFO:teuthology.orchestra.run.smithi112.stdout:true 2024-09-16T00:37:06.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.934+0000 7f90d3be9640 1 -- 172.21.15.112:0/2165211113 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90cc0fac80 msgr2=0x7f90cc0fb0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:06.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.934+0000 7f90d3be9640 1 --2- 172.21.15.112:0/2165211113 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90cc0fac80 0x7f90cc0fb0f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f90bc0048d0 tx=0x7f90bc009da0 comp rx=0 tx=0).stop 2024-09-16T00:37:06.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.934+0000 7f90d3be9640 1 -- 172.21.15.112:0/2165211113 shutdown_connections 2024-09-16T00:37:06.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.934+0000 7f90d3be9640 1 --2- 172.21.15.112:0/2165211113 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90cc0fac80 0x7f90cc0fb0f0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:06.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.934+0000 7f90d3be9640 1 --2- 172.21.15.112:0/2165211113 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90cc102620 0x7f90cc1029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:06.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.934+0000 7f90d3be9640 1 -- 172.21.15.112:0/2165211113 >> 172.21.15.112:0/2165211113 conn(0x7f90cc070ce0 msgr2=0x7f90cc0710e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:06.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.934+0000 7f90d3be9640 1 -- 172.21.15.112:0/2165211113 shutdown_connections 2024-09-16T00:37:06.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.934+0000 7f90d3be9640 1 -- 172.21.15.112:0/2165211113 wait complete. 2024-09-16T00:37:06.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.934+0000 7f90d3be9640 1 Processor -- start 2024-09-16T00:37:06.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.934+0000 7f90d3be9640 1 -- start start 2024-09-16T00:37:06.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.935+0000 7f90d3be9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90cc0fac80 0x7f90cc113240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:06.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.935+0000 7f90d3be9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90cc102620 0x7f90cc113780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:06.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.935+0000 7f90d3be9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90cc113e10 con 0x7f90cc102620 2024-09-16T00:37:06.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.935+0000 7f90d3be9640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90cc10d1c0 con 0x7f90cc0fac80 2024-09-16T00:37:06.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.935+0000 7f90d1184640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90cc102620 0x7f90cc113780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:06.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.935+0000 7f90d1184640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90cc102620 0x7f90cc113780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50498/0 (socket says 172.21.15.112:50498) 2024-09-16T00:37:06.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.935+0000 7f90d1184640 1 -- 172.21.15.112:0/2319619470 learned_addr learned my addr 172.21.15.112:0/2319619470 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:37:06.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.935+0000 7f90d1985640 1 --2- 172.21.15.112:0/2319619470 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90cc0fac80 0x7f90cc113240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:06.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.935+0000 7f90d1184640 1 -- 172.21.15.112:0/2319619470 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90cc0fac80 msgr2=0x7f90cc113240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:06.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.935+0000 7f90d1184640 1 --2- 172.21.15.112:0/2319619470 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90cc0fac80 0x7f90cc113240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:06.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.935+0000 7f90d1184640 1 -- 172.21.15.112:0/2319619470 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90bc005040 con 0x7f90cc102620 2024-09-16T00:37:06.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.935+0000 7f90d1985640 1 --2- 172.21.15.112:0/2319619470 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90cc0fac80 0x7f90cc113240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:37:06.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.935+0000 7f90d1184640 1 --2- 172.21.15.112:0/2319619470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90cc102620 0x7f90cc113780 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f90bc003080 tx=0x7f90bc009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:06.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.935+0000 7f90c27fc640 1 -- 172.21.15.112:0/2319619470 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f90bc038070 con 0x7f90cc102620 2024-09-16T00:37:06.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.935+0000 7f90c27fc640 1 -- 172.21.15.112:0/2319619470 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f90bc02ade0 con 0x7f90cc102620 2024-09-16T00:37:06.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.935+0000 7f90c27fc640 1 -- 172.21.15.112:0/2319619470 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f90bc02c880 con 0x7f90cc102620 2024-09-16T00:37:06.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.935+0000 7f90d3be9640 1 -- 172.21.15.112:0/2319619470 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90cc10d440 con 0x7f90cc102620 2024-09-16T00:37:06.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.936+0000 7f90d3be9640 1 -- 172.21.15.112:0/2319619470 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90cc10d850 con 0x7f90cc102620 2024-09-16T00:37:06.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.937+0000 7f90d3be9640 1 -- 172.21.15.112:0/2319619470 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90cc0657b0 con 0x7f90cc102620 2024-09-16T00:37:06.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.937+0000 7f90c27fc640 1 -- 172.21.15.112:0/2319619470 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f90bc030020 con 0x7f90cc102620 2024-09-16T00:37:06.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.937+0000 7f90c27fc640 1 --2- 172.21.15.112:0/2319619470 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f90b806e530 0x7f90b80709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:06.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.937+0000 7f90c27fc640 1 -- 172.21.15.112:0/2319619470 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f90bc0ae3a0 con 0x7f90cc102620 2024-09-16T00:37:06.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.937+0000 7f90d1985640 1 --2- 172.21.15.112:0/2319619470 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f90b806e530 0x7f90b80709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:06.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.938+0000 7f90d1985640 1 --2- 172.21.15.112:0/2319619470 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f90b806e530 0x7f90b80709e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f90c8006fd0 tx=0x7f90c8008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:06.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:06.942+0000 7f90c27fc640 1 -- 172.21.15.112:0/2319619470 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f90bc077310 con 0x7f90cc102620 2024-09-16T00:37:07.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.122+0000 7f90d3be9640 1 -- 172.21.15.112:0/2319619470 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f90cc04b720 con 0x7f90b806e530 2024-09-16T00:37:07.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.123+0000 7f90c27fc640 1 -- 172.21.15.112:0/2319619470 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f90cc04b720 con 0x7f90b806e530 2024-09-16T00:37:07.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.125+0000 7f90d3be9640 1 -- 172.21.15.112:0/2319619470 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f90b806e530 msgr2=0x7f90b80709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:07.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.125+0000 7f90d3be9640 1 --2- 172.21.15.112:0/2319619470 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f90b806e530 0x7f90b80709e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f90c8006fd0 tx=0x7f90c8008040 comp rx=0 tx=0).stop 2024-09-16T00:37:07.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.125+0000 7f90d3be9640 1 -- 172.21.15.112:0/2319619470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90cc102620 msgr2=0x7f90cc113780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:07.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.125+0000 7f90d3be9640 1 --2- 172.21.15.112:0/2319619470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90cc102620 0x7f90cc113780 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f90bc003080 tx=0x7f90bc009da0 comp rx=0 tx=0).stop 2024-09-16T00:37:07.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.125+0000 7f90d3be9640 1 -- 172.21.15.112:0/2319619470 shutdown_connections 2024-09-16T00:37:07.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.125+0000 7f90d3be9640 1 --2- 172.21.15.112:0/2319619470 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f90b806e530 0x7f90b80709e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:07.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.125+0000 7f90d3be9640 1 --2- 172.21.15.112:0/2319619470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90cc102620 0x7f90cc113780 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:07.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.125+0000 7f90d3be9640 1 --2- 172.21.15.112:0/2319619470 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90cc0fac80 0x7f90cc113240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:07.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.125+0000 7f90d3be9640 1 -- 172.21.15.112:0/2319619470 >> 172.21.15.112:0/2319619470 conn(0x7f90cc070ce0 msgr2=0x7f90cc105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:07.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.125+0000 7f90d3be9640 1 -- 172.21.15.112:0/2319619470 shutdown_connections 2024-09-16T00:37:07.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.125+0000 7f90d3be9640 1 -- 172.21.15.112:0/2319619470 wait complete. 2024-09-16T00:37:07.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.204+0000 7fad7f145640 1 -- 172.21.15.112:0/3994326481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad78102ef0 msgr2=0x7fad780fc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:07.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.204+0000 7fad7f145640 1 --2- 172.21.15.112:0/3994326481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad78102ef0 0x7fad780fc5b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fad640048d0 tx=0x7fad64009da0 comp rx=0 tx=0).stop 2024-09-16T00:37:07.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.204+0000 7fad7f145640 1 -- 172.21.15.112:0/3994326481 shutdown_connections 2024-09-16T00:37:07.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.204+0000 7fad7f145640 1 --2- 172.21.15.112:0/3994326481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad78102ef0 0x7fad780fc5b0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:07.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.204+0000 7fad7f145640 1 --2- 172.21.15.112:0/3994326481 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad78102550 0x7fad78102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:07.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.204+0000 7fad7f145640 1 -- 172.21.15.112:0/3994326481 >> 172.21.15.112:0/3994326481 conn(0x7fad780f80f0 msgr2=0x7fad780fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:07.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.204+0000 7fad7f145640 1 -- 172.21.15.112:0/3994326481 shutdown_connections 2024-09-16T00:37:07.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.204+0000 7fad7f145640 1 -- 172.21.15.112:0/3994326481 wait complete. 2024-09-16T00:37:07.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.204+0000 7fad7f145640 1 Processor -- start 2024-09-16T00:37:07.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.205+0000 7fad7f145640 1 -- start start 2024-09-16T00:37:07.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.205+0000 7fad7f145640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad78102550 0x7fad7806fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:07.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.205+0000 7fad7f145640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad78102ef0 0x7fad7806ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:07.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.205+0000 7fad7f145640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad78070490 con 0x7fad78102550 2024-09-16T00:37:07.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.205+0000 7fad7f145640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad78070600 con 0x7fad78102ef0 2024-09-16T00:37:07.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.205+0000 7fad7cee1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad78102550 0x7fad7806fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:07.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.205+0000 7fad7cee1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad78102550 0x7fad7806fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50524/0 (socket says 172.21.15.112:50524) 2024-09-16T00:37:07.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.205+0000 7fad7cee1640 1 -- 172.21.15.112:0/679677988 learned_addr learned my addr 172.21.15.112:0/679677988 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:37:07.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.205+0000 7fad77fff640 1 --2- 172.21.15.112:0/679677988 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad78102ef0 0x7fad7806ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:07.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.205+0000 7fad7cee1640 1 -- 172.21.15.112:0/679677988 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad78102ef0 msgr2=0x7fad7806ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:07.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.205+0000 7fad7cee1640 1 --2- 172.21.15.112:0/679677988 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad78102ef0 0x7fad7806ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:07.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.205+0000 7fad7cee1640 1 -- 172.21.15.112:0/679677988 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad64005040 con 0x7fad78102550 2024-09-16T00:37:07.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.206+0000 7fad7cee1640 1 --2- 172.21.15.112:0/679677988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad78102550 0x7fad7806fa10 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fad6c002a50 tx=0x7fad6c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:07.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.206+0000 7fad757fa640 1 -- 172.21.15.112:0/679677988 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad6c007c30 con 0x7fad78102550 2024-09-16T00:37:07.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.206+0000 7fad757fa640 1 -- 172.21.15.112:0/679677988 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fad6c00f040 con 0x7fad78102550 2024-09-16T00:37:07.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.206+0000 7fad757fa640 1 -- 172.21.15.112:0/679677988 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad6c00da10 con 0x7fad78102550 2024-09-16T00:37:07.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.206+0000 7fad7f145640 1 -- 172.21.15.112:0/679677988 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad78070890 con 0x7fad78102550 2024-09-16T00:37:07.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.206+0000 7fad7f145640 1 -- 172.21.15.112:0/679677988 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad78119bd0 con 0x7fad78102550 2024-09-16T00:37:07.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.207+0000 7fad7f145640 1 -- 172.21.15.112:0/679677988 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad780fdcd0 con 0x7fad78102550 2024-09-16T00:37:07.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.207+0000 7fad757fa640 1 -- 172.21.15.112:0/679677988 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fad6c00dc80 con 0x7fad78102550 2024-09-16T00:37:07.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.208+0000 7fad757fa640 1 --2- 172.21.15.112:0/679677988 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fad6806e530 0x7fad680709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:07.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.208+0000 7fad757fa640 1 -- 172.21.15.112:0/679677988 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7fad6c08dab0 con 0x7fad78102550 2024-09-16T00:37:07.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.208+0000 7fad77fff640 1 --2- 172.21.15.112:0/679677988 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fad6806e530 0x7fad680709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:07.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.208+0000 7fad77fff640 1 --2- 172.21.15.112:0/679677988 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fad6806e530 0x7fad680709e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fad640048a0 tx=0x7fad64035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:07.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.212+0000 7fad757fa640 1 -- 172.21.15.112:0/679677988 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fad6c056ad0 con 0x7fad78102550 2024-09-16T00:37:07.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.384+0000 7fad7f145640 1 -- 172.21.15.112:0/679677988 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fad78071100 con 0x7fad6806e530 2024-09-16T00:37:07.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.393+0000 7fad757fa640 1 -- 172.21.15.112:0/679677988 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7fad78071100 con 0x7fad6806e530 2024-09-16T00:37:07.393 INFO:teuthology.orchestra.run.smithi112.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T00:37:07.393 INFO:teuthology.orchestra.run.smithi112.stdout:alertmanager.smithi112 smithi112 *:9093,9094 running (10m) 3m ago 18m 24.7M - 0.25.0 c8568f914cd2 d85ea0562022 2024-09-16T00:37:07.393 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi112 smithi112 running (7m) 3m ago 19m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c e4fe58f9702a 2024-09-16T00:37:07.393 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi169 smithi169 running (6m) 2m ago 18m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c 3ba2c81d527f 2024-09-16T00:37:07.393 INFO:teuthology.orchestra.run.smithi112.stdout:grafana.smithi112 smithi112 *:3000 running (8m) 3m ago 18m 82.1M - 9.4.12 2bacad6d85d8 51f9d2013da2 2024-09-16T00:37:07.393 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.hxfwkh smithi112 running (3m) 3m ago 13m 14.2M - 19.1.1-326-g8921c8ef f1455bd0219c 1723069902d6 2024-09-16T00:37:07.393 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.ithwzj smithi112 running (3m) 3m ago 13m 12.2M - 19.1.1-326-g8921c8ef f1455bd0219c e7aad5b81db5 2024-09-16T00:37:07.394 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.gawwgw smithi169 running (2m) 2m ago 13m 15.7M - 19.1.1-326-g8921c8ef f1455bd0219c 469be459ee12 2024-09-16T00:37:07.394 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.lgeqrj smithi169 running (2m) 2m ago 13m 13.3M - 19.1.1-326-g8921c8ef f1455bd0219c 31f0fbd2395d 2024-09-16T00:37:07.394 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi112.xrkkbc smithi112 *:8443,9283,8765 running (9m) 3m ago 20m 504M - 19.1.1-326-g8921c8ef f1455bd0219c a981f7fd2810 2024-09-16T00:37:07.394 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi169.mtrobk smithi169 *:8443,9283,8765 running (9m) 2m ago 18m 442M - 19.1.1-326-g8921c8ef f1455bd0219c 4096e5369c2d 2024-09-16T00:37:07.394 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi112 smithi112 running (7m) 3m ago 20m 49.8M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 5aaec033c042 2024-09-16T00:37:07.394 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi169 smithi169 running (7m) 2m ago 18m 38.9M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 7bc609155f3f 2024-09-16T00:37:07.394 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi112 smithi112 *:9100 running (10m) 3m ago 19m 15.5M - 1.5.0 0da6a335fe13 6404d1d601e2 2024-09-16T00:37:07.394 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi169 smithi169 *:9100 running (9m) 2m ago 18m 15.4M - 1.5.0 0da6a335fe13 5505baf702a6 2024-09-16T00:37:07.394 INFO:teuthology.orchestra.run.smithi112.stdout:osd.0 smithi112 running (6m) 3m ago 17m 105M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 53efc9397017 2024-09-16T00:37:07.394 INFO:teuthology.orchestra.run.smithi112.stdout:osd.1 smithi112 running (6m) 3m ago 16m 98.9M 1557M 19.1.1-326-g8921c8ef f1455bd0219c fe6f1a4a2cf4 2024-09-16T00:37:07.394 INFO:teuthology.orchestra.run.smithi112.stdout:osd.2 smithi112 running (5m) 3m ago 16m 102M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 5f7863712057 2024-09-16T00:37:07.394 INFO:teuthology.orchestra.run.smithi112.stdout:osd.3 smithi169 running (4m) 2m ago 16m 92.1M 2581M 19.1.1-326-g8921c8ef f1455bd0219c ee07b5577810 2024-09-16T00:37:07.394 INFO:teuthology.orchestra.run.smithi112.stdout:osd.4 smithi169 running (4m) 2m ago 15m 103M 2581M 19.1.1-326-g8921c8ef f1455bd0219c 769f026a54eb 2024-09-16T00:37:07.394 INFO:teuthology.orchestra.run.smithi112.stdout:osd.5 smithi169 running (3m) 2m ago 15m 125M 2581M 19.1.1-326-g8921c8ef f1455bd0219c a6007c13cf78 2024-09-16T00:37:07.395 INFO:teuthology.orchestra.run.smithi112.stdout:prometheus.smithi112 smithi112 *:9095 running (9m) 3m ago 18m 64.1M - 2.43.0 a07b618ecd1d 7a1d42e60d2b 2024-09-16T00:37:07.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.395+0000 7fad7f145640 1 -- 172.21.15.112:0/679677988 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fad6806e530 msgr2=0x7fad680709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:07.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.395+0000 7fad7f145640 1 --2- 172.21.15.112:0/679677988 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fad6806e530 0x7fad680709e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fad640048a0 tx=0x7fad64035040 comp rx=0 tx=0).stop 2024-09-16T00:37:07.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.395+0000 7fad7f145640 1 -- 172.21.15.112:0/679677988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad78102550 msgr2=0x7fad7806fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:07.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.395+0000 7fad7f145640 1 --2- 172.21.15.112:0/679677988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad78102550 0x7fad7806fa10 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fad6c002a50 tx=0x7fad6c004600 comp rx=0 tx=0).stop 2024-09-16T00:37:07.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.395+0000 7fad7f145640 1 -- 172.21.15.112:0/679677988 shutdown_connections 2024-09-16T00:37:07.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.395+0000 7fad7f145640 1 --2- 172.21.15.112:0/679677988 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fad6806e530 0x7fad680709e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:07.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.395+0000 7fad7f145640 1 --2- 172.21.15.112:0/679677988 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad78102ef0 0x7fad7806ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:07.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.395+0000 7fad7f145640 1 --2- 172.21.15.112:0/679677988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad78102550 0x7fad7806fa10 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:07.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.395+0000 7fad7f145640 1 -- 172.21.15.112:0/679677988 >> 172.21.15.112:0/679677988 conn(0x7fad780f80f0 msgr2=0x7fad780f9150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:07.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.395+0000 7fad7f145640 1 -- 172.21.15.112:0/679677988 shutdown_connections 2024-09-16T00:37:07.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.395+0000 7fad7f145640 1 -- 172.21.15.112:0/679677988 wait complete. 2024-09-16T00:37:07.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.472+0000 7f0d581b2640 1 -- 172.21.15.112:0/3994101490 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d500facf0 msgr2=0x7f0d500fb160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:07.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.472+0000 7f0d581b2640 1 --2- 172.21.15.112:0/3994101490 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d500facf0 0x7f0d500fb160 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0d44009c50 tx=0x7f0d44004d10 comp rx=0 tx=0).stop 2024-09-16T00:37:07.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.472+0000 7f0d581b2640 1 -- 172.21.15.112:0/3994101490 shutdown_connections 2024-09-16T00:37:07.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.472+0000 7f0d581b2640 1 --2- 172.21.15.112:0/3994101490 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d500facf0 0x7f0d500fb160 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:07.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.472+0000 7f0d581b2640 1 --2- 172.21.15.112:0/3994101490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d50102670 0x7f0d50102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:07.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.472+0000 7f0d581b2640 1 -- 172.21.15.112:0/3994101490 >> 172.21.15.112:0/3994101490 conn(0x7f0d50070d50 msgr2=0x7f0d50071150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:07.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.472+0000 7f0d581b2640 1 -- 172.21.15.112:0/3994101490 shutdown_connections 2024-09-16T00:37:07.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.472+0000 7f0d581b2640 1 -- 172.21.15.112:0/3994101490 wait complete. 2024-09-16T00:37:07.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.473+0000 7f0d581b2640 1 Processor -- start 2024-09-16T00:37:07.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.473+0000 7f0d581b2640 1 -- start start 2024-09-16T00:37:07.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.473+0000 7f0d581b2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d500facf0 0x7f0d501131d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:07.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.473+0000 7f0d581b2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d50102670 0x7f0d50113710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:07.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.473+0000 7f0d581b2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d50113d10 con 0x7f0d50102670 2024-09-16T00:37:07.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.473+0000 7f0d581b2640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d5010d150 con 0x7f0d500facf0 2024-09-16T00:37:07.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.473+0000 7f0d5574d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d50102670 0x7f0d50113710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:07.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.473+0000 7f0d5574d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d50102670 0x7f0d50113710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50548/0 (socket says 172.21.15.112:50548) 2024-09-16T00:37:07.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.473+0000 7f0d5574d640 1 -- 172.21.15.112:0/2499345445 learned_addr learned my addr 172.21.15.112:0/2499345445 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:37:07.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.473+0000 7f0d55f4e640 1 --2- 172.21.15.112:0/2499345445 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d500facf0 0x7f0d501131d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:07.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.474+0000 7f0d5574d640 1 -- 172.21.15.112:0/2499345445 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d500facf0 msgr2=0x7f0d501131d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:07.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.474+0000 7f0d5574d640 1 --2- 172.21.15.112:0/2499345445 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d500facf0 0x7f0d501131d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:07.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.474+0000 7f0d5574d640 1 -- 172.21.15.112:0/2499345445 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d440098b0 con 0x7f0d50102670 2024-09-16T00:37:07.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.474+0000 7f0d5574d640 1 --2- 172.21.15.112:0/2499345445 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d50102670 0x7f0d50113710 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0d44004a30 tx=0x7f0d44004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:07.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.474+0000 7f0d4e7fc640 1 -- 172.21.15.112:0/2499345445 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d44009690 con 0x7f0d50102670 2024-09-16T00:37:07.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.474+0000 7f0d581b2640 1 -- 172.21.15.112:0/2499345445 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d5010d3d0 con 0x7f0d50102670 2024-09-16T00:37:07.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.474+0000 7f0d4e7fc640 1 -- 172.21.15.112:0/2499345445 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f0d44032b10 con 0x7f0d50102670 2024-09-16T00:37:07.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.475+0000 7f0d4e7fc640 1 -- 172.21.15.112:0/2499345445 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d4403daa0 con 0x7f0d50102670 2024-09-16T00:37:07.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.475+0000 7f0d581b2640 1 -- 172.21.15.112:0/2499345445 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d5010d970 con 0x7f0d50102670 2024-09-16T00:37:07.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.476+0000 7f0d581b2640 1 -- 172.21.15.112:0/2499345445 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d500657b0 con 0x7f0d50102670 2024-09-16T00:37:07.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.476+0000 7f0d4e7fc640 1 -- 172.21.15.112:0/2499345445 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f0d4402f030 con 0x7f0d50102670 2024-09-16T00:37:07.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.477+0000 7f0d4e7fc640 1 --2- 172.21.15.112:0/2499345445 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f0d3806e530 0x7f0d380709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:07.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.477+0000 7f0d4e7fc640 1 -- 172.21.15.112:0/2499345445 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f0d440af420 con 0x7f0d50102670 2024-09-16T00:37:07.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.477+0000 7f0d55f4e640 1 --2- 172.21.15.112:0/2499345445 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f0d3806e530 0x7f0d380709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:07.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.477+0000 7f0d55f4e640 1 --2- 172.21.15.112:0/2499345445 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f0d3806e530 0x7f0d380709e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0d480041d0 tx=0x7f0d48009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:07.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.481+0000 7f0d4e7fc640 1 -- 172.21.15.112:0/2499345445 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f0d44078390 con 0x7f0d50102670 2024-09-16T00:37:07.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.710+0000 7f0d581b2640 1 -- 172.21.15.112:0/2499345445 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f0d50109620 con 0x7f0d50102670 2024-09-16T00:37:07.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.711+0000 7f0d4e7fc640 1 -- 172.21.15.112:0/2499345445 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f0d4407ba50 con 0x7f0d50102670 2024-09-16T00:37:07.711 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:37:07.711 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": { 2024-09-16T00:37:07.711 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:37:07.711 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:37:07.712 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": { 2024-09-16T00:37:07.712 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:37:07.712 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:37:07.712 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": { 2024-09-16T00:37:07.712 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 6 2024-09-16T00:37:07.712 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:37:07.712 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": { 2024-09-16T00:37:07.712 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 4 2024-09-16T00:37:07.712 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:37:07.712 INFO:teuthology.orchestra.run.smithi112.stdout: "overall": { 2024-09-16T00:37:07.712 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 14 2024-09-16T00:37:07.712 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-16T00:37:07.712 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:37:07.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.713+0000 7f0d581b2640 1 -- 172.21.15.112:0/2499345445 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f0d3806e530 msgr2=0x7f0d380709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:07.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.713+0000 7f0d581b2640 1 --2- 172.21.15.112:0/2499345445 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f0d3806e530 0x7f0d380709e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0d480041d0 tx=0x7f0d48009040 comp rx=0 tx=0).stop 2024-09-16T00:37:07.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.713+0000 7f0d581b2640 1 -- 172.21.15.112:0/2499345445 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d50102670 msgr2=0x7f0d50113710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:07.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.713+0000 7f0d581b2640 1 --2- 172.21.15.112:0/2499345445 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d50102670 0x7f0d50113710 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0d44004a30 tx=0x7f0d44004d10 comp rx=0 tx=0).stop 2024-09-16T00:37:07.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.713+0000 7f0d581b2640 1 -- 172.21.15.112:0/2499345445 shutdown_connections 2024-09-16T00:37:07.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.713+0000 7f0d581b2640 1 --2- 172.21.15.112:0/2499345445 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f0d3806e530 0x7f0d380709e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:07.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.713+0000 7f0d581b2640 1 --2- 172.21.15.112:0/2499345445 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d50102670 0x7f0d50113710 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:07.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.713+0000 7f0d581b2640 1 --2- 172.21.15.112:0/2499345445 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d500facf0 0x7f0d501131d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:07.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.713+0000 7f0d581b2640 1 -- 172.21.15.112:0/2499345445 >> 172.21.15.112:0/2499345445 conn(0x7f0d50070d50 msgr2=0x7f0d500fa040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:07.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.713+0000 7f0d581b2640 1 -- 172.21.15.112:0/2499345445 shutdown_connections 2024-09-16T00:37:07.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.713+0000 7f0d581b2640 1 -- 172.21.15.112:0/2499345445 wait complete. 2024-09-16T00:37:07.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.791+0000 7f38a1902640 1 -- 172.21.15.112:0/1358766276 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f389c102680 msgr2=0x7f389c102a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:07.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.791+0000 7f38a1902640 1 --2- 172.21.15.112:0/1358766276 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f389c102680 0x7f389c102a50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f388c0048d0 tx=0x7f388c009da0 comp rx=0 tx=0).stop 2024-09-16T00:37:07.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.791+0000 7f38a1902640 1 -- 172.21.15.112:0/1358766276 shutdown_connections 2024-09-16T00:37:07.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.791+0000 7f38a1902640 1 --2- 172.21.15.112:0/1358766276 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f389c0facf0 0x7f389c0fb160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:07.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.791+0000 7f38a1902640 1 --2- 172.21.15.112:0/1358766276 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f389c102680 0x7f389c102a50 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:07.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.791+0000 7f38a1902640 1 -- 172.21.15.112:0/1358766276 >> 172.21.15.112:0/1358766276 conn(0x7f389c070d50 msgr2=0x7f389c071150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:07.793 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:37:07 smithi112 ceph-mon[97750]: pgmap v281: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:37:07.793 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:37:07 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/2499345445' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:37:07.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.792+0000 7f38a1902640 1 -- 172.21.15.112:0/1358766276 shutdown_connections 2024-09-16T00:37:07.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.792+0000 7f38a1902640 1 -- 172.21.15.112:0/1358766276 wait complete. 2024-09-16T00:37:07.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.792+0000 7f38a1902640 1 Processor -- start 2024-09-16T00:37:07.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.792+0000 7f38a1902640 1 -- start start 2024-09-16T00:37:07.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.793+0000 7f38a1902640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f389c0facf0 0x7f389c1132f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:07.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.793+0000 7f38a1902640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f389c102680 0x7f389c113830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:07.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.793+0000 7f38a1902640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f389c113e30 con 0x7f389c0facf0 2024-09-16T00:37:07.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.793+0000 7f38a1902640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f389c10d270 con 0x7f389c102680 2024-09-16T00:37:07.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.793+0000 7f389affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f389c0facf0 0x7f389c1132f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:07.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.793+0000 7f389affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f389c0facf0 0x7f389c1132f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50566/0 (socket says 172.21.15.112:50566) 2024-09-16T00:37:07.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.793+0000 7f389affd640 1 -- 172.21.15.112:0/3216736114 learned_addr learned my addr 172.21.15.112:0/3216736114 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:37:07.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.793+0000 7f38927fc640 1 --2- 172.21.15.112:0/3216736114 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f389c102680 0x7f389c113830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:07.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.793+0000 7f389affd640 1 -- 172.21.15.112:0/3216736114 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f389c102680 msgr2=0x7f389c113830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:07.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.793+0000 7f389affd640 1 --2- 172.21.15.112:0/3216736114 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f389c102680 0x7f389c113830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:07.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.793+0000 7f389affd640 1 -- 172.21.15.112:0/3216736114 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f388c005040 con 0x7f389c0facf0 2024-09-16T00:37:07.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.793+0000 7f389affd640 1 --2- 172.21.15.112:0/3216736114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f389c0facf0 0x7f389c1132f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f388c003080 tx=0x7f388c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:07.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.793+0000 7f3893fff640 1 -- 172.21.15.112:0/3216736114 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f388c038070 con 0x7f389c0facf0 2024-09-16T00:37:07.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.793+0000 7f3893fff640 1 -- 172.21.15.112:0/3216736114 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f388c0097e0 con 0x7f389c0facf0 2024-09-16T00:37:07.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.794+0000 7f38a1902640 1 -- 172.21.15.112:0/3216736114 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f389c10d4f0 con 0x7f389c0facf0 2024-09-16T00:37:07.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.794+0000 7f3893fff640 1 -- 172.21.15.112:0/3216736114 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f388c03d420 con 0x7f389c0facf0 2024-09-16T00:37:07.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.794+0000 7f38a1902640 1 -- 172.21.15.112:0/3216736114 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f389c10d9b0 con 0x7f389c0facf0 2024-09-16T00:37:07.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.795+0000 7f38a1902640 1 -- 172.21.15.112:0/3216736114 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f389c0657f0 con 0x7f389c0facf0 2024-09-16T00:37:07.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.795+0000 7f3893fff640 1 -- 172.21.15.112:0/3216736114 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f388c030020 con 0x7f389c0facf0 2024-09-16T00:37:07.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.796+0000 7f3893fff640 1 --2- 172.21.15.112:0/3216736114 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f388806e530 0x7f38880709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:07.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.796+0000 7f38927fc640 1 --2- 172.21.15.112:0/3216736114 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f388806e530 0x7f38880709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:07.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.796+0000 7f3893fff640 1 -- 172.21.15.112:0/3216736114 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f388c0ae8d0 con 0x7f389c0facf0 2024-09-16T00:37:07.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.796+0000 7f38927fc640 1 --2- 172.21.15.112:0/3216736114 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f388806e530 0x7f38880709e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f389c10ebb0 tx=0x7f3884009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:07.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:07.800+0000 7f3893fff640 1 -- 172.21.15.112:0/3216736114 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f388c077840 con 0x7f389c0facf0 2024-09-16T00:37:08.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.000+0000 7f38a1902640 1 -- 172.21.15.112:0/3216736114 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f389c04b720 con 0x7f389c0facf0 2024-09-16T00:37:08.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.001+0000 7f3893fff640 1 -- 172.21.15.112:0/3216736114 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1914 (secure 0 0 0) 0x7f388c07af00 con 0x7f389c0facf0 2024-09-16T00:37:08.002 INFO:teuthology.orchestra.run.smithi112.stdout:e34 2024-09-16T00:37:08.002 INFO:teuthology.orchestra.run.smithi112.stdout:btime 2024-09-16T00:34:26:619254+0000 2024-09-16T00:37:08.002 INFO:teuthology.orchestra.run.smithi112.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T00:37:08.002 INFO:teuthology.orchestra.run.smithi112.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:37:08.002 INFO:teuthology.orchestra.run.smithi112.stdout:legacy client fscid: 1 2024-09-16T00:37:08.002 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:37:08.002 INFO:teuthology.orchestra.run.smithi112.stdout:Filesystem 'cephfs' (1) 2024-09-16T00:37:08.002 INFO:teuthology.orchestra.run.smithi112.stdout:fs_name cephfs 2024-09-16T00:37:08.002 INFO:teuthology.orchestra.run.smithi112.stdout:epoch 25 2024-09-16T00:37:08.002 INFO:teuthology.orchestra.run.smithi112.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-16T00:37:08.002 INFO:teuthology.orchestra.run.smithi112.stdout:created 2024-09-16T00:23:22.328199+0000 2024-09-16T00:37:08.002 INFO:teuthology.orchestra.run.smithi112.stdout:modified 2024-09-16T00:33:29.012991+0000 2024-09-16T00:37:08.002 INFO:teuthology.orchestra.run.smithi112.stdout:tableserver 0 2024-09-16T00:37:08.002 INFO:teuthology.orchestra.run.smithi112.stdout:root 0 2024-09-16T00:37:08.003 INFO:teuthology.orchestra.run.smithi112.stdout:session_timeout 60 2024-09-16T00:37:08.003 INFO:teuthology.orchestra.run.smithi112.stdout:session_autoclose 300 2024-09-16T00:37:08.003 INFO:teuthology.orchestra.run.smithi112.stdout:max_file_size 1099511627776 2024-09-16T00:37:08.003 INFO:teuthology.orchestra.run.smithi112.stdout:max_xattr_size 65536 2024-09-16T00:37:08.003 INFO:teuthology.orchestra.run.smithi112.stdout:required_client_features {} 2024-09-16T00:37:08.003 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure 0 2024-09-16T00:37:08.003 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure_osd_epoch 82 2024-09-16T00:37:08.003 INFO:teuthology.orchestra.run.smithi112.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:37:08.003 INFO:teuthology.orchestra.run.smithi112.stdout:max_mds 2 2024-09-16T00:37:08.003 INFO:teuthology.orchestra.run.smithi112.stdout:in 0,1 2024-09-16T00:37:08.003 INFO:teuthology.orchestra.run.smithi112.stdout:up {} 2024-09-16T00:37:08.003 INFO:teuthology.orchestra.run.smithi112.stdout:failed 0,1 2024-09-16T00:37:08.003 INFO:teuthology.orchestra.run.smithi112.stdout:damaged 2024-09-16T00:37:08.003 INFO:teuthology.orchestra.run.smithi112.stdout:stopped 2024-09-16T00:37:08.004 INFO:teuthology.orchestra.run.smithi112.stdout:data_pools [3] 2024-09-16T00:37:08.004 INFO:teuthology.orchestra.run.smithi112.stdout:metadata_pool 2 2024-09-16T00:37:08.004 INFO:teuthology.orchestra.run.smithi112.stdout:inline_data enabled 2024-09-16T00:37:08.004 INFO:teuthology.orchestra.run.smithi112.stdout:balancer 2024-09-16T00:37:08.004 INFO:teuthology.orchestra.run.smithi112.stdout:bal_rank_mask -1 2024-09-16T00:37:08.004 INFO:teuthology.orchestra.run.smithi112.stdout:standby_count_wanted 1 2024-09-16T00:37:08.004 INFO:teuthology.orchestra.run.smithi112.stdout:qdb_cluster leader: 0 members: 2024-09-16T00:37:08.004 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:37:08.004 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:37:08.004 INFO:teuthology.orchestra.run.smithi112.stdout:Standby daemons: 2024-09-16T00:37:08.004 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:37:08.004 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.ithwzj{-1:34392} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.112:6826/476229221,v1:172.21.15.112:6827/476229221] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:37:08.004 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.hxfwkh{-1:34404} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.112:6828/2356899876,v1:172.21.15.112:6829/2356899876] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:37:08.004 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.gawwgw{-1:44343} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.169:6824/1635329957,v1:172.21.15.169:6825/1635329957] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:37:08.005 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.lgeqrj{-1:44347} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.169:6826/184810031,v1:172.21.15.169:6827/184810031] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:37:08.005 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 34 2024-09-16T00:37:08.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.004+0000 7f38a1902640 1 -- 172.21.15.112:0/3216736114 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f388806e530 msgr2=0x7f38880709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:08.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.004+0000 7f38a1902640 1 --2- 172.21.15.112:0/3216736114 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f388806e530 0x7f38880709e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f389c10ebb0 tx=0x7f3884009040 comp rx=0 tx=0).stop 2024-09-16T00:37:08.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.004+0000 7f38a1902640 1 -- 172.21.15.112:0/3216736114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f389c0facf0 msgr2=0x7f389c1132f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:08.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.004+0000 7f38a1902640 1 --2- 172.21.15.112:0/3216736114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f389c0facf0 0x7f389c1132f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f388c003080 tx=0x7f388c009da0 comp rx=0 tx=0).stop 2024-09-16T00:37:08.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.004+0000 7f38a1902640 1 -- 172.21.15.112:0/3216736114 shutdown_connections 2024-09-16T00:37:08.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.004+0000 7f38a1902640 1 --2- 172.21.15.112:0/3216736114 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f388806e530 0x7f38880709e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:08.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.004+0000 7f38a1902640 1 --2- 172.21.15.112:0/3216736114 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f389c102680 0x7f389c113830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:08.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.004+0000 7f38a1902640 1 --2- 172.21.15.112:0/3216736114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f389c0facf0 0x7f389c1132f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:08.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.004+0000 7f38a1902640 1 -- 172.21.15.112:0/3216736114 >> 172.21.15.112:0/3216736114 conn(0x7f389c070d50 msgr2=0x7f389c0fa0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:08.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.004+0000 7f38a1902640 1 -- 172.21.15.112:0/3216736114 shutdown_connections 2024-09-16T00:37:08.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.004+0000 7f38a1902640 1 -- 172.21.15.112:0/3216736114 wait complete. 2024-09-16T00:37:08.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:37:07 smithi169 ceph-mon[77626]: pgmap v281: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:37:08.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:37:07 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/2499345445' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:37:08.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.085+0000 7f22e9030640 1 -- 172.21.15.112:0/2986557321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22e4102550 msgr2=0x7f22e4102920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:08.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.085+0000 7f22e9030640 1 --2- 172.21.15.112:0/2986557321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22e4102550 0x7f22e4102920 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f22d40048d0 tx=0x7f22d4009da0 comp rx=0 tx=0).stop 2024-09-16T00:37:08.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.085+0000 7f22e9030640 1 -- 172.21.15.112:0/2986557321 shutdown_connections 2024-09-16T00:37:08.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.085+0000 7f22e9030640 1 --2- 172.21.15.112:0/2986557321 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22e4102ef0 0x7f22e40fc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:08.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.085+0000 7f22e9030640 1 --2- 172.21.15.112:0/2986557321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22e4102550 0x7f22e4102920 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:08.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.085+0000 7f22e9030640 1 -- 172.21.15.112:0/2986557321 >> 172.21.15.112:0/2986557321 conn(0x7f22e40f80f0 msgr2=0x7f22e40fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:08.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.085+0000 7f22e9030640 1 -- 172.21.15.112:0/2986557321 shutdown_connections 2024-09-16T00:37:08.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.085+0000 7f22e9030640 1 -- 172.21.15.112:0/2986557321 wait complete. 2024-09-16T00:37:08.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.085+0000 7f22e9030640 1 Processor -- start 2024-09-16T00:37:08.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.085+0000 7f22e9030640 1 -- start start 2024-09-16T00:37:08.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.086+0000 7f22e9030640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22e4102550 0x7f22e4113540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:08.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.086+0000 7f22e9030640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22e4102ef0 0x7f22e4113a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:08.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.086+0000 7f22e9030640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e41140d0 con 0x7f22e4102ef0 2024-09-16T00:37:08.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.086+0000 7f22e9030640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e410d4e0 con 0x7f22e4102550 2024-09-16T00:37:08.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.086+0000 7f22e259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22e4102ef0 0x7f22e4113a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:08.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.086+0000 7f22e259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22e4102ef0 0x7f22e4113a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50596/0 (socket says 172.21.15.112:50596) 2024-09-16T00:37:08.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.086+0000 7f22e259c640 1 -- 172.21.15.112:0/546552503 learned_addr learned my addr 172.21.15.112:0/546552503 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:37:08.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.086+0000 7f22e2d9d640 1 --2- 172.21.15.112:0/546552503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22e4102550 0x7f22e4113540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:08.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.086+0000 7f22e259c640 1 -- 172.21.15.112:0/546552503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22e4102550 msgr2=0x7f22e4113540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:08.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.086+0000 7f22e259c640 1 --2- 172.21.15.112:0/546552503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22e4102550 0x7f22e4113540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:08.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.086+0000 7f22e259c640 1 -- 172.21.15.112:0/546552503 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d4005040 con 0x7f22e4102ef0 2024-09-16T00:37:08.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.086+0000 7f22e259c640 1 --2- 172.21.15.112:0/546552503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22e4102ef0 0x7f22e4113a80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f22cc00c790 tx=0x7f22cc0043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:08.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.087+0000 7f22db7fe640 1 -- 172.21.15.112:0/546552503 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22cc00a7a0 con 0x7f22e4102ef0 2024-09-16T00:37:08.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.087+0000 7f22db7fe640 1 -- 172.21.15.112:0/546552503 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f22cc00ada0 con 0x7f22e4102ef0 2024-09-16T00:37:08.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.087+0000 7f22e9030640 1 -- 172.21.15.112:0/546552503 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22e410d7c0 con 0x7f22e4102ef0 2024-09-16T00:37:08.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.087+0000 7f22db7fe640 1 -- 172.21.15.112:0/546552503 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22cc011380 con 0x7f22e4102ef0 2024-09-16T00:37:08.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.087+0000 7f22e9030640 1 -- 172.21.15.112:0/546552503 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22e410dd60 con 0x7f22e4102ef0 2024-09-16T00:37:08.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.088+0000 7f22e9030640 1 -- 172.21.15.112:0/546552503 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22e40fdcd0 con 0x7f22e4102ef0 2024-09-16T00:37:08.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.089+0000 7f22db7fe640 1 -- 172.21.15.112:0/546552503 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f22cc012420 con 0x7f22e4102ef0 2024-09-16T00:37:08.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.089+0000 7f22db7fe640 1 --2- 172.21.15.112:0/546552503 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f22d006e530 0x7f22d00709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:08.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.090+0000 7f22db7fe640 1 -- 172.21.15.112:0/546552503 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f22cc08e6f0 con 0x7f22e4102ef0 2024-09-16T00:37:08.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.090+0000 7f22e2d9d640 1 --2- 172.21.15.112:0/546552503 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f22d006e530 0x7f22d00709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:08.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.090+0000 7f22e2d9d640 1 --2- 172.21.15.112:0/546552503 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f22d006e530 0x7f22d00709e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f22d4009ac0 tx=0x7f22d4035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:08.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.094+0000 7f22db7fe640 1 -- 172.21.15.112:0/546552503 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f22cc0576e0 con 0x7f22e4102ef0 2024-09-16T00:37:08.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.276+0000 7f22e9030640 1 -- 172.21.15.112:0/546552503 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f22e4113230 con 0x7f22d006e530 2024-09-16T00:37:08.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.278+0000 7f22db7fe640 1 -- 172.21.15.112:0/546552503 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f22e4113230 con 0x7f22d006e530 2024-09-16T00:37:08.278 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:37:08.278 INFO:teuthology.orchestra.run.smithi112.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80", 2024-09-16T00:37:08.278 INFO:teuthology.orchestra.run.smithi112.stdout: "in_progress": true, 2024-09-16T00:37:08.278 INFO:teuthology.orchestra.run.smithi112.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T00:37:08.278 INFO:teuthology.orchestra.run.smithi112.stdout: "services_complete": [ 2024-09-16T00:37:08.278 INFO:teuthology.orchestra.run.smithi112.stdout: "mds", 2024-09-16T00:37:08.278 INFO:teuthology.orchestra.run.smithi112.stdout: "osd", 2024-09-16T00:37:08.278 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr", 2024-09-16T00:37:08.278 INFO:teuthology.orchestra.run.smithi112.stdout: "mon", 2024-09-16T00:37:08.278 INFO:teuthology.orchestra.run.smithi112.stdout: "crash" 2024-09-16T00:37:08.279 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-16T00:37:08.279 INFO:teuthology.orchestra.run.smithi112.stdout: "progress": "16/21 daemons upgraded", 2024-09-16T00:37:08.279 INFO:teuthology.orchestra.run.smithi112.stdout: "message": "Currently upgrading mds daemons", 2024-09-16T00:37:08.279 INFO:teuthology.orchestra.run.smithi112.stdout: "is_paused": false 2024-09-16T00:37:08.279 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:37:08.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.280+0000 7f22e9030640 1 -- 172.21.15.112:0/546552503 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f22d006e530 msgr2=0x7f22d00709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:08.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.280+0000 7f22e9030640 1 --2- 172.21.15.112:0/546552503 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f22d006e530 0x7f22d00709e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f22d4009ac0 tx=0x7f22d4035040 comp rx=0 tx=0).stop 2024-09-16T00:37:08.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.280+0000 7f22e9030640 1 -- 172.21.15.112:0/546552503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22e4102ef0 msgr2=0x7f22e4113a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:08.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.280+0000 7f22e9030640 1 --2- 172.21.15.112:0/546552503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22e4102ef0 0x7f22e4113a80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f22cc00c790 tx=0x7f22cc0043c0 comp rx=0 tx=0).stop 2024-09-16T00:37:08.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.280+0000 7f22e9030640 1 -- 172.21.15.112:0/546552503 shutdown_connections 2024-09-16T00:37:08.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.280+0000 7f22e9030640 1 --2- 172.21.15.112:0/546552503 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f22d006e530 0x7f22d00709e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:08.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.280+0000 7f22e9030640 1 --2- 172.21.15.112:0/546552503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22e4102ef0 0x7f22e4113a80 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:08.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.280+0000 7f22e9030640 1 --2- 172.21.15.112:0/546552503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22e4102550 0x7f22e4113540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:08.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.280+0000 7f22e9030640 1 -- 172.21.15.112:0/546552503 >> 172.21.15.112:0/546552503 conn(0x7f22e40f80f0 msgr2=0x7f22e40f9160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:08.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.280+0000 7f22e9030640 1 -- 172.21.15.112:0/546552503 shutdown_connections 2024-09-16T00:37:08.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.280+0000 7f22e9030640 1 -- 172.21.15.112:0/546552503 wait complete. 2024-09-16T00:37:08.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.359+0000 7f5de4bdb640 1 -- 172.21.15.112:0/2248367931 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5de00fdf50 msgr2=0x7f5de0106620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:08.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.359+0000 7f5de4bdb640 1 --2- 172.21.15.112:0/2248367931 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5de00fdf50 0x7f5de0106620 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f5dc80048d0 tx=0x7f5dc8009da0 comp rx=0 tx=0).stop 2024-09-16T00:37:08.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.359+0000 7f5de4bdb640 1 -- 172.21.15.112:0/2248367931 shutdown_connections 2024-09-16T00:37:08.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.359+0000 7f5de4bdb640 1 --2- 172.21.15.112:0/2248367931 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5de00fdf50 0x7f5de0106620 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:08.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.359+0000 7f5de4bdb640 1 --2- 172.21.15.112:0/2248367931 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5de00fd5b0 0x7f5de00fd980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:08.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.359+0000 7f5de4bdb640 1 -- 172.21.15.112:0/2248367931 >> 172.21.15.112:0/2248367931 conn(0x7f5de00f7010 msgr2=0x7f5de00f9420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:08.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.359+0000 7f5de4bdb640 1 -- 172.21.15.112:0/2248367931 shutdown_connections 2024-09-16T00:37:08.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.359+0000 7f5de4bdb640 1 -- 172.21.15.112:0/2248367931 wait complete. 2024-09-16T00:37:08.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.359+0000 7f5de4bdb640 1 Processor -- start 2024-09-16T00:37:08.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.359+0000 7f5de4bdb640 1 -- start start 2024-09-16T00:37:08.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.360+0000 7f5de4bdb640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5de00fd5b0 0x7f5de0113250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:08.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.360+0000 7f5de4bdb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5de00fdf50 0x7f5de0113790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:08.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.360+0000 7f5de4bdb640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de0113d90 con 0x7f5de00fdf50 2024-09-16T00:37:08.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.360+0000 7f5de4bdb640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de010d1d0 con 0x7f5de00fd5b0 2024-09-16T00:37:08.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.360+0000 7f5dddd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5de00fdf50 0x7f5de0113790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:08.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.360+0000 7f5dddd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5de00fdf50 0x7f5de0113790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50610/0 (socket says 172.21.15.112:50610) 2024-09-16T00:37:08.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.360+0000 7f5dddd9b640 1 -- 172.21.15.112:0/2969117996 learned_addr learned my addr 172.21.15.112:0/2969117996 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:37:08.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.360+0000 7f5dde59c640 1 --2- 172.21.15.112:0/2969117996 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5de00fd5b0 0x7f5de0113250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:08.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.360+0000 7f5dddd9b640 1 -- 172.21.15.112:0/2969117996 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5de00fd5b0 msgr2=0x7f5de0113250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:08.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.360+0000 7f5dddd9b640 1 --2- 172.21.15.112:0/2969117996 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5de00fd5b0 0x7f5de0113250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:08.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.360+0000 7f5dddd9b640 1 -- 172.21.15.112:0/2969117996 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dc8005040 con 0x7f5de00fdf50 2024-09-16T00:37:08.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.360+0000 7f5dddd9b640 1 --2- 172.21.15.112:0/2969117996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5de00fdf50 0x7f5de0113790 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5dc80048a0 tx=0x7f5dc8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:08.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.360+0000 7f5dd6ffd640 1 -- 172.21.15.112:0/2969117996 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5dc8038070 con 0x7f5de00fdf50 2024-09-16T00:37:08.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.361+0000 7f5dd6ffd640 1 -- 172.21.15.112:0/2969117996 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f5dc8032cb0 con 0x7f5de00fdf50 2024-09-16T00:37:08.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.361+0000 7f5de4bdb640 1 -- 172.21.15.112:0/2969117996 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5de010d450 con 0x7f5de00fdf50 2024-09-16T00:37:08.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.361+0000 7f5dd6ffd640 1 -- 172.21.15.112:0/2969117996 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5dc80097b0 con 0x7f5de00fdf50 2024-09-16T00:37:08.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.361+0000 7f5de4bdb640 1 -- 172.21.15.112:0/2969117996 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5de010d910 con 0x7f5de00fdf50 2024-09-16T00:37:08.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.361+0000 7f5de4bdb640 1 -- 172.21.15.112:0/2969117996 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5de00662f0 con 0x7f5de00fdf50 2024-09-16T00:37:08.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.362+0000 7f5dd6ffd640 1 -- 172.21.15.112:0/2969117996 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f5dc8030020 con 0x7f5de00fdf50 2024-09-16T00:37:08.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.363+0000 7f5dd6ffd640 1 --2- 172.21.15.112:0/2969117996 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5dcc06e530 0x7f5dcc0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:08.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.363+0000 7f5dd6ffd640 1 -- 172.21.15.112:0/2969117996 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f5dc80ae770 con 0x7f5de00fdf50 2024-09-16T00:37:08.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.363+0000 7f5dde59c640 1 --2- 172.21.15.112:0/2969117996 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5dcc06e530 0x7f5dcc0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:08.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.363+0000 7f5dde59c640 1 --2- 172.21.15.112:0/2969117996 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5dcc06e530 0x7f5dcc0709e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8004e40 tx=0x7f5dd8009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:08.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.367+0000 7f5dd6ffd640 1 -- 172.21.15.112:0/2969117996 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f5dc8077760 con 0x7f5de00fdf50 2024-09-16T00:37:08.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.598+0000 7f5de4bdb640 1 -- 172.21.15.112:0/2969117996 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f5de00fac50 con 0x7f5de00fdf50 2024-09-16T00:37:08.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.599+0000 7f5dd6ffd640 1 -- 172.21.15.112:0/2969117996 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f5dc807ae20 con 0x7f5de00fdf50 2024-09-16T00:37:08.599 INFO:teuthology.orchestra.run.smithi112.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-16T00:37:08.599 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-16T00:37:08.599 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs is degraded 2024-09-16T00:37:08.600 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T00:37:08.600 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T00:37:08.600 INFO:teuthology.orchestra.run.smithi112.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-16T00:37:08.600 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-16T00:37:08.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.601+0000 7f5de4bdb640 1 -- 172.21.15.112:0/2969117996 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5dcc06e530 msgr2=0x7f5dcc0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:08.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.601+0000 7f5de4bdb640 1 --2- 172.21.15.112:0/2969117996 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5dcc06e530 0x7f5dcc0709e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8004e40 tx=0x7f5dd8009040 comp rx=0 tx=0).stop 2024-09-16T00:37:08.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.601+0000 7f5de4bdb640 1 -- 172.21.15.112:0/2969117996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5de00fdf50 msgr2=0x7f5de0113790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:08.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.601+0000 7f5de4bdb640 1 --2- 172.21.15.112:0/2969117996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5de00fdf50 0x7f5de0113790 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5dc80048a0 tx=0x7f5dc8009da0 comp rx=0 tx=0).stop 2024-09-16T00:37:08.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.601+0000 7f5de4bdb640 1 -- 172.21.15.112:0/2969117996 shutdown_connections 2024-09-16T00:37:08.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.601+0000 7f5de4bdb640 1 --2- 172.21.15.112:0/2969117996 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5dcc06e530 0x7f5dcc0709e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:08.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.601+0000 7f5de4bdb640 1 --2- 172.21.15.112:0/2969117996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5de00fdf50 0x7f5de0113790 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:08.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.601+0000 7f5de4bdb640 1 --2- 172.21.15.112:0/2969117996 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5de00fd5b0 0x7f5de0113250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:08.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.601+0000 7f5de4bdb640 1 -- 172.21.15.112:0/2969117996 >> 172.21.15.112:0/2969117996 conn(0x7f5de00f7010 msgr2=0x7f5de00f8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:08.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.601+0000 7f5de4bdb640 1 -- 172.21.15.112:0/2969117996 shutdown_connections 2024-09-16T00:37:08.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:08.601+0000 7f5de4bdb640 1 -- 172.21.15.112:0/2969117996 wait complete. 2024-09-16T00:37:09.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:37:08 smithi169 ceph-mon[77626]: from='client.34576 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:37:09.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:37:08 smithi169 ceph-mon[77626]: from='client.34580 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:37:09.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:37:08 smithi169 ceph-mon[77626]: from='client.34584 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:37:09.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:37:08 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/3216736114' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:37:09.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:37:08 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/2969117996' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:37:09.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:37:08 smithi112 ceph-mon[97750]: from='client.34576 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:37:09.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:37:08 smithi112 ceph-mon[97750]: from='client.34580 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:37:09.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:37:08 smithi112 ceph-mon[97750]: from='client.34584 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:37:09.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:37:08 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/3216736114' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:37:09.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:37:08 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/2969117996' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:37:10.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:37:09 smithi169 ceph-mon[77626]: from='client.34596 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:37:10.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:37:09 smithi169 ceph-mon[77626]: pgmap v282: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:37:10.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:37:09 smithi112 ceph-mon[97750]: from='client.34596 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:37:10.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:37:09 smithi112 ceph-mon[97750]: pgmap v282: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:37:12.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:37:11 smithi169 ceph-mon[77626]: pgmap v283: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:37:12.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:37:11 smithi112 ceph-mon[97750]: pgmap v283: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:37:14.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:37:13 smithi169 ceph-mon[77626]: pgmap v284: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:37:14.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:37:13 smithi112 ceph-mon[97750]: pgmap v284: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:37:16.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:37:15 smithi169 ceph-mon[77626]: pgmap v285: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:37:16.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:37:15 smithi112 ceph-mon[97750]: pgmap v285: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:37:17.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:37:16 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:37:17.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:37:16 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:37:18.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:37:17 smithi169 ceph-mon[77626]: pgmap v286: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:37:18.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:37:17 smithi112 ceph-mon[97750]: pgmap v286: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:37:20.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:37:19 smithi169 ceph-mon[77626]: pgmap v287: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:37:20.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:37:19 smithi112 ceph-mon[97750]: pgmap v287: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:37:22.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:37:21 smithi169 ceph-mon[77626]: pgmap v288: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:37:22.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:37:21 smithi112 ceph-mon[97750]: pgmap v288: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:37:24.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:37:23 smithi169 ceph-mon[77626]: pgmap v289: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:37:24.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:37:23 smithi112 ceph-mon[97750]: pgmap v289: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:37:26.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:37:25 smithi169 ceph-mon[77626]: pgmap v290: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:37:26.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:37:25 smithi112 ceph-mon[97750]: pgmap v290: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:37:28.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:37:27 smithi169 ceph-mon[77626]: pgmap v291: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:37:28.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:37:27 smithi112 ceph-mon[97750]: pgmap v291: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:37:30.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:37:29 smithi169 ceph-mon[77626]: pgmap v292: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:37:30.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:37:29 smithi112 ceph-mon[97750]: pgmap v292: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:37:32.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:37:31 smithi169 ceph-mon[77626]: pgmap v293: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:37:32.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:37:31 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:37:32.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:37:31 smithi112 ceph-mon[97750]: pgmap v293: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:37:32.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:37:31 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:37:34.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:37:33 smithi169 ceph-mon[77626]: pgmap v294: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:37:34.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:37:33 smithi112 ceph-mon[97750]: pgmap v294: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:37:36.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:37:35 smithi169 ceph-mon[77626]: pgmap v295: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:37:36.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:37:35 smithi112 ceph-mon[97750]: pgmap v295: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:37:38.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:37:37 smithi169 ceph-mon[77626]: pgmap v296: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:37:38.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:37:37 smithi112 ceph-mon[97750]: pgmap v296: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:37:38.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.691+0000 7fd42299d640 1 -- 172.21.15.112:0/2192440144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd41c102580 msgr2=0x7fd41c102950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:38.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.691+0000 7fd42299d640 1 --2- 172.21.15.112:0/2192440144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd41c102580 0x7fd41c102950 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd40c0048d0 tx=0x7fd40c009da0 comp rx=0 tx=0).stop 2024-09-16T00:37:38.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.691+0000 7fd42299d640 1 -- 172.21.15.112:0/2192440144 shutdown_connections 2024-09-16T00:37:38.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.691+0000 7fd42299d640 1 --2- 172.21.15.112:0/2192440144 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd41c102f20 0x7fd41c0fc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:38.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.691+0000 7fd42299d640 1 --2- 172.21.15.112:0/2192440144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd41c102580 0x7fd41c102950 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:38.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.691+0000 7fd42299d640 1 -- 172.21.15.112:0/2192440144 >> 172.21.15.112:0/2192440144 conn(0x7fd41c0f81a0 msgr2=0x7fd41c0fa5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:38.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.691+0000 7fd42299d640 1 -- 172.21.15.112:0/2192440144 shutdown_connections 2024-09-16T00:37:38.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.691+0000 7fd42299d640 1 -- 172.21.15.112:0/2192440144 wait complete. 2024-09-16T00:37:38.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.691+0000 7fd42299d640 1 Processor -- start 2024-09-16T00:37:38.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.692+0000 7fd42299d640 1 -- start start 2024-09-16T00:37:38.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.692+0000 7fd42299d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd41c102580 0x7fd41c1131d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:38.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.692+0000 7fd42299d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd41c102f20 0x7fd41c113710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:38.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.692+0000 7fd42299d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd41c113d10 con 0x7fd41c102580 2024-09-16T00:37:38.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.692+0000 7fd42299d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd41c10d150 con 0x7fd41c102f20 2024-09-16T00:37:38.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.692+0000 7fd41bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd41c102580 0x7fd41c1131d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:38.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.692+0000 7fd41bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd41c102580 0x7fd41c1131d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47576/0 (socket says 172.21.15.112:47576) 2024-09-16T00:37:38.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.692+0000 7fd41bfff640 1 -- 172.21.15.112:0/142518216 learned_addr learned my addr 172.21.15.112:0/142518216 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:37:38.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.692+0000 7fd41b7fe640 1 --2- 172.21.15.112:0/142518216 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd41c102f20 0x7fd41c113710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:38.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.692+0000 7fd41bfff640 1 -- 172.21.15.112:0/142518216 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd41c102f20 msgr2=0x7fd41c113710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:38.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.692+0000 7fd41bfff640 1 --2- 172.21.15.112:0/142518216 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd41c102f20 0x7fd41c113710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:38.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.692+0000 7fd41bfff640 1 -- 172.21.15.112:0/142518216 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd40c005040 con 0x7fd41c102580 2024-09-16T00:37:38.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.693+0000 7fd41bfff640 1 --2- 172.21.15.112:0/142518216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd41c102580 0x7fd41c1131d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd40c003080 tx=0x7fd40c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:38.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.693+0000 7fd418ff9640 1 -- 172.21.15.112:0/142518216 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd40c038070 con 0x7fd41c102580 2024-09-16T00:37:38.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.693+0000 7fd42299d640 1 -- 172.21.15.112:0/142518216 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd41c10d3d0 con 0x7fd41c102580 2024-09-16T00:37:38.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.693+0000 7fd418ff9640 1 -- 172.21.15.112:0/142518216 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fd40c0097e0 con 0x7fd41c102580 2024-09-16T00:37:38.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.693+0000 7fd418ff9640 1 -- 172.21.15.112:0/142518216 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd40c03d420 con 0x7fd41c102580 2024-09-16T00:37:38.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.693+0000 7fd42299d640 1 -- 172.21.15.112:0/142518216 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd41c10d910 con 0x7fd41c102580 2024-09-16T00:37:38.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.694+0000 7fd42299d640 1 -- 172.21.15.112:0/142518216 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd41c0fdd00 con 0x7fd41c102580 2024-09-16T00:37:38.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.694+0000 7fd418ff9640 1 -- 172.21.15.112:0/142518216 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fd40c030020 con 0x7fd41c102580 2024-09-16T00:37:38.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.695+0000 7fd418ff9640 1 --2- 172.21.15.112:0/142518216 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd40406e530 0x7fd4040709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:38.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.695+0000 7fd418ff9640 1 -- 172.21.15.112:0/142518216 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7fd40c0ae7f0 con 0x7fd41c102580 2024-09-16T00:37:38.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.695+0000 7fd41b7fe640 1 --2- 172.21.15.112:0/142518216 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd40406e530 0x7fd4040709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:38.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.695+0000 7fd41b7fe640 1 --2- 172.21.15.112:0/142518216 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd40406e530 0x7fd4040709e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd41c10ea90 tx=0x7fd410009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:38.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.699+0000 7fd418ff9640 1 -- 172.21.15.112:0/142518216 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fd40c077760 con 0x7fd41c102580 2024-09-16T00:37:38.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.897+0000 7fd42299d640 1 -- 172.21.15.112:0/142518216 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd41c04b720 con 0x7fd40406e530 2024-09-16T00:37:38.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.898+0000 7fd418ff9640 1 -- 172.21.15.112:0/142518216 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fd41c04b720 con 0x7fd40406e530 2024-09-16T00:37:38.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.900+0000 7fd42299d640 1 -- 172.21.15.112:0/142518216 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd40406e530 msgr2=0x7fd4040709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:38.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.900+0000 7fd42299d640 1 --2- 172.21.15.112:0/142518216 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd40406e530 0x7fd4040709e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd41c10ea90 tx=0x7fd410009040 comp rx=0 tx=0).stop 2024-09-16T00:37:38.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.900+0000 7fd42299d640 1 -- 172.21.15.112:0/142518216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd41c102580 msgr2=0x7fd41c1131d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:38.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.900+0000 7fd42299d640 1 --2- 172.21.15.112:0/142518216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd41c102580 0x7fd41c1131d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd40c003080 tx=0x7fd40c009da0 comp rx=0 tx=0).stop 2024-09-16T00:37:38.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.900+0000 7fd42299d640 1 -- 172.21.15.112:0/142518216 shutdown_connections 2024-09-16T00:37:38.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.900+0000 7fd42299d640 1 --2- 172.21.15.112:0/142518216 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd40406e530 0x7fd4040709e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:38.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.900+0000 7fd42299d640 1 --2- 172.21.15.112:0/142518216 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd41c102f20 0x7fd41c113710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:38.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.900+0000 7fd42299d640 1 --2- 172.21.15.112:0/142518216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd41c102580 0x7fd41c1131d0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:38.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.900+0000 7fd42299d640 1 -- 172.21.15.112:0/142518216 >> 172.21.15.112:0/142518216 conn(0x7fd41c0f81a0 msgr2=0x7fd41c0f91a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:38.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.901+0000 7fd42299d640 1 -- 172.21.15.112:0/142518216 shutdown_connections 2024-09-16T00:37:38.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.901+0000 7fd42299d640 1 -- 172.21.15.112:0/142518216 wait complete. 2024-09-16T00:37:38.910 INFO:teuthology.orchestra.run.smithi112.stdout:true 2024-09-16T00:37:38.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.980+0000 7fa4024c6640 1 -- 172.21.15.112:0/3313363214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3fc102680 msgr2=0x7fa3fc102a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:38.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.980+0000 7fa4024c6640 1 --2- 172.21.15.112:0/3313363214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3fc102680 0x7fa3fc102a50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec0048d0 tx=0x7fa3ec009da0 comp rx=0 tx=0).stop 2024-09-16T00:37:38.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.980+0000 7fa4024c6640 1 -- 172.21.15.112:0/3313363214 shutdown_connections 2024-09-16T00:37:38.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.980+0000 7fa4024c6640 1 --2- 172.21.15.112:0/3313363214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3fc0fabd0 0x7fa3fc0fb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:38.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.980+0000 7fa4024c6640 1 --2- 172.21.15.112:0/3313363214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3fc102680 0x7fa3fc102a50 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:38.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.980+0000 7fa4024c6640 1 -- 172.21.15.112:0/3313363214 >> 172.21.15.112:0/3313363214 conn(0x7fa3fc071ac0 msgr2=0x7fa3fc071ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:38.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.980+0000 7fa4024c6640 1 -- 172.21.15.112:0/3313363214 shutdown_connections 2024-09-16T00:37:38.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.980+0000 7fa4024c6640 1 -- 172.21.15.112:0/3313363214 wait complete. 2024-09-16T00:37:38.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.981+0000 7fa4024c6640 1 Processor -- start 2024-09-16T00:37:38.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.981+0000 7fa4024c6640 1 -- start start 2024-09-16T00:37:38.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.981+0000 7fa4024c6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3fc0fabd0 0x7fa3fc1132e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:38.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.981+0000 7fa4024c6640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3fc102680 0x7fa3fc113820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:38.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.981+0000 7fa4024c6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3fc113eb0 con 0x7fa3fc0fabd0 2024-09-16T00:37:38.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.981+0000 7fa4024c6640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3fc10d260 con 0x7fa3fc102680 2024-09-16T00:37:38.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.981+0000 7fa3fbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3fc0fabd0 0x7fa3fc1132e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:38.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.981+0000 7fa3fbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3fc0fabd0 0x7fa3fc1132e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47604/0 (socket says 172.21.15.112:47604) 2024-09-16T00:37:38.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.981+0000 7fa3fbfff640 1 -- 172.21.15.112:0/1137223581 learned_addr learned my addr 172.21.15.112:0/1137223581 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:37:38.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.981+0000 7fa3f3fff640 1 --2- 172.21.15.112:0/1137223581 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3fc102680 0x7fa3fc113820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:38.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.981+0000 7fa3fbfff640 1 -- 172.21.15.112:0/1137223581 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3fc102680 msgr2=0x7fa3fc113820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:38.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.981+0000 7fa3fbfff640 1 --2- 172.21.15.112:0/1137223581 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3fc102680 0x7fa3fc113820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:38.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.981+0000 7fa3fbfff640 1 -- 172.21.15.112:0/1137223581 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3ec005040 con 0x7fa3fc0fabd0 2024-09-16T00:37:38.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.982+0000 7fa3fbfff640 1 --2- 172.21.15.112:0/1137223581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3fc0fabd0 0x7fa3fc1132e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec003080 tx=0x7fa3ec009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:38.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.982+0000 7fa3f97fa640 1 -- 172.21.15.112:0/1137223581 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3ec038070 con 0x7fa3fc0fabd0 2024-09-16T00:37:38.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.982+0000 7fa3f97fa640 1 -- 172.21.15.112:0/1137223581 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fa3ec0097e0 con 0x7fa3fc0fabd0 2024-09-16T00:37:38.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.982+0000 7fa3f97fa640 1 -- 172.21.15.112:0/1137223581 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3ec03d420 con 0x7fa3fc0fabd0 2024-09-16T00:37:38.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.982+0000 7fa4024c6640 1 -- 172.21.15.112:0/1137223581 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3fc10d4e0 con 0x7fa3fc0fabd0 2024-09-16T00:37:38.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.982+0000 7fa4024c6640 1 -- 172.21.15.112:0/1137223581 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3fc10d9a0 con 0x7fa3fc0fabd0 2024-09-16T00:37:38.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.983+0000 7fa4024c6640 1 -- 172.21.15.112:0/1137223581 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3fc0725d0 con 0x7fa3fc0fabd0 2024-09-16T00:37:38.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.984+0000 7fa3f97fa640 1 -- 172.21.15.112:0/1137223581 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fa3ec030020 con 0x7fa3fc0fabd0 2024-09-16T00:37:38.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.985+0000 7fa3f97fa640 1 --2- 172.21.15.112:0/1137223581 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa3e806e530 0x7fa3e80709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:38.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.985+0000 7fa3f97fa640 1 -- 172.21.15.112:0/1137223581 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7fa3ec0ae7f0 con 0x7fa3fc0fabd0 2024-09-16T00:37:38.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.985+0000 7fa3f3fff640 1 --2- 172.21.15.112:0/1137223581 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa3e806e530 0x7fa3e80709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:38.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.985+0000 7fa3f3fff640 1 --2- 172.21.15.112:0/1137223581 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa3e806e530 0x7fa3e80709e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc10eba0 tx=0x7fa3e4009120 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:38.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:38.990+0000 7fa3f97fa640 1 -- 172.21.15.112:0/1137223581 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fa3ec077760 con 0x7fa3fc0fabd0 2024-09-16T00:37:39.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.173+0000 7fa4024c6640 1 -- 172.21.15.112:0/1137223581 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa3fc04b720 con 0x7fa3e806e530 2024-09-16T00:37:39.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.174+0000 7fa3f97fa640 1 -- 172.21.15.112:0/1137223581 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fa3fc04b720 con 0x7fa3e806e530 2024-09-16T00:37:39.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.176+0000 7fa4024c6640 1 -- 172.21.15.112:0/1137223581 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa3e806e530 msgr2=0x7fa3e80709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:39.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.176+0000 7fa4024c6640 1 --2- 172.21.15.112:0/1137223581 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa3e806e530 0x7fa3e80709e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc10eba0 tx=0x7fa3e4009120 comp rx=0 tx=0).stop 2024-09-16T00:37:39.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.176+0000 7fa4024c6640 1 -- 172.21.15.112:0/1137223581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3fc0fabd0 msgr2=0x7fa3fc1132e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:39.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.176+0000 7fa4024c6640 1 --2- 172.21.15.112:0/1137223581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3fc0fabd0 0x7fa3fc1132e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec003080 tx=0x7fa3ec009da0 comp rx=0 tx=0).stop 2024-09-16T00:37:39.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.176+0000 7fa4024c6640 1 -- 172.21.15.112:0/1137223581 shutdown_connections 2024-09-16T00:37:39.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.176+0000 7fa4024c6640 1 --2- 172.21.15.112:0/1137223581 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa3e806e530 0x7fa3e80709e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:39.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.176+0000 7fa4024c6640 1 --2- 172.21.15.112:0/1137223581 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3fc102680 0x7fa3fc113820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:39.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.176+0000 7fa4024c6640 1 --2- 172.21.15.112:0/1137223581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3fc0fabd0 0x7fa3fc1132e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:39.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.176+0000 7fa4024c6640 1 -- 172.21.15.112:0/1137223581 >> 172.21.15.112:0/1137223581 conn(0x7fa3fc071ac0 msgr2=0x7fa3fc105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:39.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.176+0000 7fa4024c6640 1 -- 172.21.15.112:0/1137223581 shutdown_connections 2024-09-16T00:37:39.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.176+0000 7fa4024c6640 1 -- 172.21.15.112:0/1137223581 wait complete. 2024-09-16T00:37:39.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.254+0000 7f5e0bf9e640 1 -- 172.21.15.112:0/2557139704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e04102550 msgr2=0x7f5e04102920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:39.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.254+0000 7f5e0bf9e640 1 --2- 172.21.15.112:0/2557139704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e04102550 0x7f5e04102920 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5e000048d0 tx=0x7f5e00009da0 comp rx=0 tx=0).stop 2024-09-16T00:37:39.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.254+0000 7f5e0bf9e640 1 -- 172.21.15.112:0/2557139704 shutdown_connections 2024-09-16T00:37:39.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.254+0000 7f5e0bf9e640 1 --2- 172.21.15.112:0/2557139704 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5e04102ef0 0x7f5e040fc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:39.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.254+0000 7f5e0bf9e640 1 --2- 172.21.15.112:0/2557139704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e04102550 0x7f5e04102920 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:39.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.254+0000 7f5e0bf9e640 1 -- 172.21.15.112:0/2557139704 >> 172.21.15.112:0/2557139704 conn(0x7f5e040f80f0 msgr2=0x7f5e040fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:39.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.254+0000 7f5e0bf9e640 1 -- 172.21.15.112:0/2557139704 shutdown_connections 2024-09-16T00:37:39.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.254+0000 7f5e0bf9e640 1 -- 172.21.15.112:0/2557139704 wait complete. 2024-09-16T00:37:39.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.255+0000 7f5e0bf9e640 1 Processor -- start 2024-09-16T00:37:39.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.255+0000 7f5e0bf9e640 1 -- start start 2024-09-16T00:37:39.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.255+0000 7f5e0bf9e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e04102550 0x7f5e04113540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:39.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.255+0000 7f5e0bf9e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5e04102ef0 0x7f5e04113a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:39.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.255+0000 7f5e0bf9e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e041140d0 con 0x7f5e04102550 2024-09-16T00:37:39.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.255+0000 7f5e0bf9e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e0410d4e0 con 0x7f5e04102ef0 2024-09-16T00:37:39.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.255+0000 7f5e09d3a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e04102550 0x7f5e04113540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:39.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.255+0000 7f5e09d3a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e04102550 0x7f5e04113540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47620/0 (socket says 172.21.15.112:47620) 2024-09-16T00:37:39.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.255+0000 7f5e09d3a640 1 -- 172.21.15.112:0/821581458 learned_addr learned my addr 172.21.15.112:0/821581458 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:37:39.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.255+0000 7f5e09539640 1 --2- 172.21.15.112:0/821581458 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5e04102ef0 0x7f5e04113a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:39.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.256+0000 7f5e09d3a640 1 -- 172.21.15.112:0/821581458 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5e04102ef0 msgr2=0x7f5e04113a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:39.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.256+0000 7f5e09d3a640 1 --2- 172.21.15.112:0/821581458 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5e04102ef0 0x7f5e04113a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:39.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.256+0000 7f5e09d3a640 1 -- 172.21.15.112:0/821581458 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e00005040 con 0x7f5e04102550 2024-09-16T00:37:39.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.256+0000 7f5e09d3a640 1 --2- 172.21.15.112:0/821581458 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e04102550 0x7f5e04113540 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5e00003080 tx=0x7f5e00009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:39.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.256+0000 7f5dfa7fc640 1 -- 172.21.15.112:0/821581458 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e00038070 con 0x7f5e04102550 2024-09-16T00:37:39.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.256+0000 7f5dfa7fc640 1 -- 172.21.15.112:0/821581458 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f5e000097e0 con 0x7f5e04102550 2024-09-16T00:37:39.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.256+0000 7f5dfa7fc640 1 -- 172.21.15.112:0/821581458 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e0003d420 con 0x7f5e04102550 2024-09-16T00:37:39.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.256+0000 7f5e0bf9e640 1 -- 172.21.15.112:0/821581458 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e0410d760 con 0x7f5e04102550 2024-09-16T00:37:39.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.256+0000 7f5e0bf9e640 1 -- 172.21.15.112:0/821581458 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e0410dc50 con 0x7f5e04102550 2024-09-16T00:37:39.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.257+0000 7f5e0bf9e640 1 -- 172.21.15.112:0/821581458 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e040fdcd0 con 0x7f5e04102550 2024-09-16T00:37:39.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.257+0000 7f5dfa7fc640 1 -- 172.21.15.112:0/821581458 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f5e00030020 con 0x7f5e04102550 2024-09-16T00:37:39.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.258+0000 7f5dfa7fc640 1 --2- 172.21.15.112:0/821581458 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5df006e530 0x7f5df00709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:39.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.258+0000 7f5dfa7fc640 1 -- 172.21.15.112:0/821581458 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f5e000ae8a0 con 0x7f5e04102550 2024-09-16T00:37:39.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.258+0000 7f5e09539640 1 --2- 172.21.15.112:0/821581458 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5df006e530 0x7f5df00709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:39.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.258+0000 7f5e09539640 1 --2- 172.21.15.112:0/821581458 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5df006e530 0x7f5df00709e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5e0410e7c0 tx=0x7f5df4009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:39.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.262+0000 7f5dfa7fc640 1 -- 172.21.15.112:0/821581458 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f5e00077810 con 0x7f5e04102550 2024-09-16T00:37:39.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.434+0000 7f5e0bf9e640 1 -- 172.21.15.112:0/821581458 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5e0404b720 con 0x7f5df006e530 2024-09-16T00:37:39.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.443+0000 7f5dfa7fc640 1 -- 172.21.15.112:0/821581458 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7f5e0404b720 con 0x7f5df006e530 2024-09-16T00:37:39.443 INFO:teuthology.orchestra.run.smithi112.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T00:37:39.443 INFO:teuthology.orchestra.run.smithi112.stdout:alertmanager.smithi112 smithi112 *:9093,9094 running (10m) 3m ago 19m 24.7M - 0.25.0 c8568f914cd2 d85ea0562022 2024-09-16T00:37:39.443 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi112 smithi112 running (7m) 3m ago 19m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c e4fe58f9702a 2024-09-16T00:37:39.443 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi169 smithi169 running (7m) 3m ago 18m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c 3ba2c81d527f 2024-09-16T00:37:39.444 INFO:teuthology.orchestra.run.smithi112.stdout:grafana.smithi112 smithi112 *:3000 running (9m) 3m ago 19m 82.1M - 9.4.12 2bacad6d85d8 51f9d2013da2 2024-09-16T00:37:39.444 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.hxfwkh smithi112 running (3m) 3m ago 14m 14.2M - 19.1.1-326-g8921c8ef f1455bd0219c 1723069902d6 2024-09-16T00:37:39.444 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.ithwzj smithi112 running (4m) 3m ago 14m 12.2M - 19.1.1-326-g8921c8ef f1455bd0219c e7aad5b81db5 2024-09-16T00:37:39.444 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.gawwgw smithi169 running (3m) 3m ago 14m 15.7M - 19.1.1-326-g8921c8ef f1455bd0219c 469be459ee12 2024-09-16T00:37:39.444 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.lgeqrj smithi169 running (3m) 3m ago 14m 13.3M - 19.1.1-326-g8921c8ef f1455bd0219c 31f0fbd2395d 2024-09-16T00:37:39.444 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi112.xrkkbc smithi112 *:8443,9283,8765 running (10m) 3m ago 20m 504M - 19.1.1-326-g8921c8ef f1455bd0219c a981f7fd2810 2024-09-16T00:37:39.444 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi169.mtrobk smithi169 *:8443,9283,8765 running (9m) 3m ago 18m 442M - 19.1.1-326-g8921c8ef f1455bd0219c 4096e5369c2d 2024-09-16T00:37:39.444 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi112 smithi112 running (8m) 3m ago 20m 49.8M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 5aaec033c042 2024-09-16T00:37:39.444 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi169 smithi169 running (8m) 3m ago 18m 38.9M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 7bc609155f3f 2024-09-16T00:37:39.444 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi112 smithi112 *:9100 running (10m) 3m ago 19m 15.5M - 1.5.0 0da6a335fe13 6404d1d601e2 2024-09-16T00:37:39.444 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi169 smithi169 *:9100 running (10m) 3m ago 18m 15.4M - 1.5.0 0da6a335fe13 5505baf702a6 2024-09-16T00:37:39.444 INFO:teuthology.orchestra.run.smithi112.stdout:osd.0 smithi112 running (7m) 3m ago 17m 105M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 53efc9397017 2024-09-16T00:37:39.444 INFO:teuthology.orchestra.run.smithi112.stdout:osd.1 smithi112 running (6m) 3m ago 17m 98.9M 1557M 19.1.1-326-g8921c8ef f1455bd0219c fe6f1a4a2cf4 2024-09-16T00:37:39.444 INFO:teuthology.orchestra.run.smithi112.stdout:osd.2 smithi112 running (5m) 3m ago 16m 102M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 5f7863712057 2024-09-16T00:37:39.445 INFO:teuthology.orchestra.run.smithi112.stdout:osd.3 smithi169 running (5m) 3m ago 16m 92.1M 2581M 19.1.1-326-g8921c8ef f1455bd0219c ee07b5577810 2024-09-16T00:37:39.445 INFO:teuthology.orchestra.run.smithi112.stdout:osd.4 smithi169 running (4m) 3m ago 16m 103M 2581M 19.1.1-326-g8921c8ef f1455bd0219c 769f026a54eb 2024-09-16T00:37:39.445 INFO:teuthology.orchestra.run.smithi112.stdout:osd.5 smithi169 running (4m) 3m ago 15m 125M 2581M 19.1.1-326-g8921c8ef f1455bd0219c a6007c13cf78 2024-09-16T00:37:39.445 INFO:teuthology.orchestra.run.smithi112.stdout:prometheus.smithi112 smithi112 *:9095 running (9m) 3m ago 19m 64.1M - 2.43.0 a07b618ecd1d 7a1d42e60d2b 2024-09-16T00:37:39.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.445+0000 7f5e0bf9e640 1 -- 172.21.15.112:0/821581458 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5df006e530 msgr2=0x7f5df00709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:39.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.445+0000 7f5e0bf9e640 1 --2- 172.21.15.112:0/821581458 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5df006e530 0x7f5df00709e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5e0410e7c0 tx=0x7f5df4009040 comp rx=0 tx=0).stop 2024-09-16T00:37:39.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.445+0000 7f5e0bf9e640 1 -- 172.21.15.112:0/821581458 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e04102550 msgr2=0x7f5e04113540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:39.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.445+0000 7f5e0bf9e640 1 --2- 172.21.15.112:0/821581458 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e04102550 0x7f5e04113540 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5e00003080 tx=0x7f5e00009da0 comp rx=0 tx=0).stop 2024-09-16T00:37:39.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.445+0000 7f5e0bf9e640 1 -- 172.21.15.112:0/821581458 shutdown_connections 2024-09-16T00:37:39.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.445+0000 7f5e0bf9e640 1 --2- 172.21.15.112:0/821581458 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5df006e530 0x7f5df00709e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:39.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.445+0000 7f5e0bf9e640 1 --2- 172.21.15.112:0/821581458 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5e04102ef0 0x7f5e04113a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:39.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.445+0000 7f5e0bf9e640 1 --2- 172.21.15.112:0/821581458 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e04102550 0x7f5e04113540 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:39.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.445+0000 7f5e0bf9e640 1 -- 172.21.15.112:0/821581458 >> 172.21.15.112:0/821581458 conn(0x7f5e040f80f0 msgr2=0x7f5e040f9160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:39.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.445+0000 7f5e0bf9e640 1 -- 172.21.15.112:0/821581458 shutdown_connections 2024-09-16T00:37:39.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.445+0000 7f5e0bf9e640 1 -- 172.21.15.112:0/821581458 wait complete. 2024-09-16T00:37:39.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.522+0000 7fcdb7cea640 1 -- 172.21.15.112:0/259514790 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcdb00fac80 msgr2=0x7fcdb00fb0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:39.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.522+0000 7fcdb7cea640 1 --2- 172.21.15.112:0/259514790 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcdb00fac80 0x7fcdb00fb0d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fcda00048d0 tx=0x7fcda0009da0 comp rx=0 tx=0).stop 2024-09-16T00:37:39.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.522+0000 7fcdb7cea640 1 -- 172.21.15.112:0/259514790 shutdown_connections 2024-09-16T00:37:39.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.522+0000 7fcdb7cea640 1 --2- 172.21.15.112:0/259514790 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcdb00fac80 0x7fcdb00fb0d0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:39.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.522+0000 7fcdb7cea640 1 --2- 172.21.15.112:0/259514790 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdb00fd3a0 0x7fcdb00fd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:39.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.522+0000 7fcdb7cea640 1 -- 172.21.15.112:0/259514790 >> 172.21.15.112:0/259514790 conn(0x7fcdb0070ce0 msgr2=0x7fcdb00710e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:39.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.522+0000 7fcdb7cea640 1 -- 172.21.15.112:0/259514790 shutdown_connections 2024-09-16T00:37:39.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.522+0000 7fcdb7cea640 1 -- 172.21.15.112:0/259514790 wait complete. 2024-09-16T00:37:39.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.523+0000 7fcdb7cea640 1 Processor -- start 2024-09-16T00:37:39.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.523+0000 7fcdb7cea640 1 -- start start 2024-09-16T00:37:39.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.523+0000 7fcdb7cea640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdb00fac80 0x7fcdb0115480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:39.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.523+0000 7fcdb7cea640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcdb00fd3a0 0x7fcdb01159c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:39.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.523+0000 7fcdb7cea640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb0116010 con 0x7fcdb00fd3a0 2024-09-16T00:37:39.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.523+0000 7fcdb7cea640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb010f420 con 0x7fcdb00fac80 2024-09-16T00:37:39.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.523+0000 7fcdb5285640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcdb00fd3a0 0x7fcdb01159c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:39.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.523+0000 7fcdb5285640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcdb00fd3a0 0x7fcdb01159c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47650/0 (socket says 172.21.15.112:47650) 2024-09-16T00:37:39.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.523+0000 7fcdb5285640 1 -- 172.21.15.112:0/2005431639 learned_addr learned my addr 172.21.15.112:0/2005431639 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:37:39.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.523+0000 7fcdb5a86640 1 --2- 172.21.15.112:0/2005431639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdb00fac80 0x7fcdb0115480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:39.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.524+0000 7fcdb5285640 1 -- 172.21.15.112:0/2005431639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdb00fac80 msgr2=0x7fcdb0115480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:39.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.524+0000 7fcdb5285640 1 --2- 172.21.15.112:0/2005431639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdb00fac80 0x7fcdb0115480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:39.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.524+0000 7fcdb5285640 1 -- 172.21.15.112:0/2005431639 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcda0005040 con 0x7fcdb00fd3a0 2024-09-16T00:37:39.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.524+0000 7fcdb5285640 1 --2- 172.21.15.112:0/2005431639 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcdb00fd3a0 0x7fcdb01159c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fcda0003080 tx=0x7fcda0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:39.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.524+0000 7fcda67fc640 1 -- 172.21.15.112:0/2005431639 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcda0038070 con 0x7fcdb00fd3a0 2024-09-16T00:37:39.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.524+0000 7fcdb7cea640 1 -- 172.21.15.112:0/2005431639 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdb010f6a0 con 0x7fcdb00fd3a0 2024-09-16T00:37:39.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.524+0000 7fcda67fc640 1 -- 172.21.15.112:0/2005431639 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fcda00097e0 con 0x7fcdb00fd3a0 2024-09-16T00:37:39.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.524+0000 7fcda67fc640 1 -- 172.21.15.112:0/2005431639 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcda003d420 con 0x7fcdb00fd3a0 2024-09-16T00:37:39.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.524+0000 7fcdb7cea640 1 -- 172.21.15.112:0/2005431639 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdb010fb60 con 0x7fcdb00fd3a0 2024-09-16T00:37:39.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.525+0000 7fcdb7cea640 1 -- 172.21.15.112:0/2005431639 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdb00fbc90 con 0x7fcdb00fd3a0 2024-09-16T00:37:39.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.526+0000 7fcda67fc640 1 -- 172.21.15.112:0/2005431639 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fcda0030020 con 0x7fcdb00fd3a0 2024-09-16T00:37:39.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.526+0000 7fcda67fc640 1 --2- 172.21.15.112:0/2005431639 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fcd9c06e530 0x7fcd9c0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:39.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.526+0000 7fcda67fc640 1 -- 172.21.15.112:0/2005431639 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7fcda00ae7f0 con 0x7fcdb00fd3a0 2024-09-16T00:37:39.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.526+0000 7fcdb5a86640 1 --2- 172.21.15.112:0/2005431639 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fcd9c06e530 0x7fcd9c0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:39.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.527+0000 7fcdb5a86640 1 --2- 172.21.15.112:0/2005431639 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fcd9c06e530 0x7fcd9c0709e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcdac0041d0 tx=0x7fcdac004330 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:39.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.531+0000 7fcda67fc640 1 -- 172.21.15.112:0/2005431639 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fcda0077760 con 0x7fcdb00fd3a0 2024-09-16T00:37:39.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.764+0000 7fcdb7cea640 1 -- 172.21.15.112:0/2005431639 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fcdb004b720 con 0x7fcdb00fd3a0 2024-09-16T00:37:39.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.765+0000 7fcda67fc640 1 -- 172.21.15.112:0/2005431639 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7fcda007ae20 con 0x7fcdb00fd3a0 2024-09-16T00:37:39.765 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:37:39.765 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": { 2024-09-16T00:37:39.765 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:37:39.765 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:37:39.766 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": { 2024-09-16T00:37:39.766 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:37:39.766 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:37:39.766 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": { 2024-09-16T00:37:39.766 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 6 2024-09-16T00:37:39.766 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:37:39.766 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": { 2024-09-16T00:37:39.766 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 4 2024-09-16T00:37:39.766 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:37:39.766 INFO:teuthology.orchestra.run.smithi112.stdout: "overall": { 2024-09-16T00:37:39.766 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 14 2024-09-16T00:37:39.766 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-16T00:37:39.766 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:37:39.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.767+0000 7fcdb7cea640 1 -- 172.21.15.112:0/2005431639 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fcd9c06e530 msgr2=0x7fcd9c0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:39.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.767+0000 7fcdb7cea640 1 --2- 172.21.15.112:0/2005431639 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fcd9c06e530 0x7fcd9c0709e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcdac0041d0 tx=0x7fcdac004330 comp rx=0 tx=0).stop 2024-09-16T00:37:39.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.767+0000 7fcdb7cea640 1 -- 172.21.15.112:0/2005431639 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcdb00fd3a0 msgr2=0x7fcdb01159c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:39.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.767+0000 7fcdb7cea640 1 --2- 172.21.15.112:0/2005431639 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcdb00fd3a0 0x7fcdb01159c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fcda0003080 tx=0x7fcda0009da0 comp rx=0 tx=0).stop 2024-09-16T00:37:39.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.767+0000 7fcdb7cea640 1 -- 172.21.15.112:0/2005431639 shutdown_connections 2024-09-16T00:37:39.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.767+0000 7fcdb7cea640 1 --2- 172.21.15.112:0/2005431639 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fcd9c06e530 0x7fcd9c0709e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:39.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.767+0000 7fcdb7cea640 1 --2- 172.21.15.112:0/2005431639 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcdb00fd3a0 0x7fcdb01159c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:39.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.767+0000 7fcdb7cea640 1 --2- 172.21.15.112:0/2005431639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdb00fac80 0x7fcdb0115480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:39.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.767+0000 7fcdb7cea640 1 -- 172.21.15.112:0/2005431639 >> 172.21.15.112:0/2005431639 conn(0x7fcdb0070ce0 msgr2=0x7fcdb00f8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:39.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.767+0000 7fcdb7cea640 1 -- 172.21.15.112:0/2005431639 shutdown_connections 2024-09-16T00:37:39.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.767+0000 7fcdb7cea640 1 -- 172.21.15.112:0/2005431639 wait complete. 2024-09-16T00:37:39.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.845+0000 7f87164cf640 1 -- 172.21.15.112:0/1875527085 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87100fcf80 msgr2=0x7f87100fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:39.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.846+0000 7f87164cf640 1 --2- 172.21.15.112:0/1875527085 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87100fcf80 0x7f87100fd3d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f86f80048d0 tx=0x7f86f8009da0 comp rx=0 tx=0).stop 2024-09-16T00:37:39.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.846+0000 7f87164cf640 1 -- 172.21.15.112:0/1875527085 shutdown_connections 2024-09-16T00:37:39.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.846+0000 7f87164cf640 1 --2- 172.21.15.112:0/1875527085 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87100fcf80 0x7f87100fd3d0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:39.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.846+0000 7f87164cf640 1 --2- 172.21.15.112:0/1875527085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f87100fb810 0x7f87100fbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:39.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.846+0000 7f87164cf640 1 -- 172.21.15.112:0/1875527085 >> 172.21.15.112:0/1875527085 conn(0x7f87100723b0 msgr2=0x7f87100727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:39.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.846+0000 7f87164cf640 1 -- 172.21.15.112:0/1875527085 shutdown_connections 2024-09-16T00:37:39.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.846+0000 7f87164cf640 1 -- 172.21.15.112:0/1875527085 wait complete. 2024-09-16T00:37:39.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.846+0000 7f87164cf640 1 Processor -- start 2024-09-16T00:37:39.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.846+0000 7f87164cf640 1 -- start start 2024-09-16T00:37:39.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.847+0000 7f87164cf640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f87100fb810 0x7f8710112140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:39.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.847+0000 7f87164cf640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87100fcf80 0x7f8710112680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:39.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.847+0000 7f87164cf640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8710112d10 con 0x7f87100fcf80 2024-09-16T00:37:39.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.847+0000 7f87164cf640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f871010c100 con 0x7f87100fb810 2024-09-16T00:37:39.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.847+0000 7f870f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87100fcf80 0x7f8710112680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:39.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.847+0000 7f870f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87100fcf80 0x7f8710112680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47672/0 (socket says 172.21.15.112:47672) 2024-09-16T00:37:39.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.847+0000 7f870f7fe640 1 -- 172.21.15.112:0/172057481 learned_addr learned my addr 172.21.15.112:0/172057481 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:37:39.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.847+0000 7f870ffff640 1 --2- 172.21.15.112:0/172057481 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f87100fb810 0x7f8710112140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:39.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.847+0000 7f870f7fe640 1 -- 172.21.15.112:0/172057481 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f87100fb810 msgr2=0x7f8710112140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:39.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.847+0000 7f870f7fe640 1 --2- 172.21.15.112:0/172057481 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f87100fb810 0x7f8710112140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:39.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.847+0000 7f870f7fe640 1 -- 172.21.15.112:0/172057481 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f8005040 con 0x7f87100fcf80 2024-09-16T00:37:39.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.847+0000 7f870ffff640 1 --2- 172.21.15.112:0/172057481 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f87100fb810 0x7f8710112140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:37:39.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.847+0000 7f870f7fe640 1 --2- 172.21.15.112:0/172057481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87100fcf80 0x7f8710112680 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f86f8003080 tx=0x7f86f8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:39.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.847+0000 7f870cff9640 1 -- 172.21.15.112:0/172057481 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f86f8038070 con 0x7f87100fcf80 2024-09-16T00:37:39.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.847+0000 7f87164cf640 1 -- 172.21.15.112:0/172057481 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f871010c380 con 0x7f87100fcf80 2024-09-16T00:37:39.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.847+0000 7f870cff9640 1 -- 172.21.15.112:0/172057481 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f86f80097e0 con 0x7f87100fcf80 2024-09-16T00:37:39.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.847+0000 7f870cff9640 1 -- 172.21.15.112:0/172057481 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f86f803d420 con 0x7f87100fcf80 2024-09-16T00:37:39.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.847+0000 7f87164cf640 1 -- 172.21.15.112:0/172057481 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f871010c840 con 0x7f87100fcf80 2024-09-16T00:37:39.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.848+0000 7f87164cf640 1 -- 172.21.15.112:0/172057481 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87100fcbc0 con 0x7f87100fcf80 2024-09-16T00:37:39.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.849+0000 7f870cff9640 1 -- 172.21.15.112:0/172057481 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f86f8030020 con 0x7f87100fcf80 2024-09-16T00:37:39.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.849+0000 7f870cff9640 1 --2- 172.21.15.112:0/172057481 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f86fc06e530 0x7f86fc0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:39.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.850+0000 7f870ffff640 1 --2- 172.21.15.112:0/172057481 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f86fc06e530 0x7f86fc0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:39.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.850+0000 7f870cff9640 1 -- 172.21.15.112:0/172057481 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f86f80ae830 con 0x7f87100fcf80 2024-09-16T00:37:39.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.850+0000 7f870ffff640 1 --2- 172.21.15.112:0/172057481 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f86fc06e530 0x7f86fc0709e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8700006fd0 tx=0x7f8700008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:39.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:39.854+0000 7f870cff9640 1 -- 172.21.15.112:0/172057481 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f86f80777a0 con 0x7f87100fcf80 2024-09-16T00:37:40.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:37:39 smithi169 ceph-mon[77626]: pgmap v297: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:37:40.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:37:39 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/2005431639' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:37:40.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.052+0000 7f87164cf640 1 -- 172.21.15.112:0/172057481 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f8710072ec0 con 0x7f87100fcf80 2024-09-16T00:37:40.053 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:37:39 smithi112 ceph-mon[97750]: pgmap v297: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:37:40.053 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:37:39 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/2005431639' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:37:40.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.053+0000 7f870cff9640 1 -- 172.21.15.112:0/172057481 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1914 (secure 0 0 0) 0x7f86f807ae60 con 0x7f87100fcf80 2024-09-16T00:37:40.054 INFO:teuthology.orchestra.run.smithi112.stdout:e34 2024-09-16T00:37:40.054 INFO:teuthology.orchestra.run.smithi112.stdout:btime 2024-09-16T00:34:26:619254+0000 2024-09-16T00:37:40.054 INFO:teuthology.orchestra.run.smithi112.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T00:37:40.054 INFO:teuthology.orchestra.run.smithi112.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:37:40.054 INFO:teuthology.orchestra.run.smithi112.stdout:legacy client fscid: 1 2024-09-16T00:37:40.054 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:37:40.054 INFO:teuthology.orchestra.run.smithi112.stdout:Filesystem 'cephfs' (1) 2024-09-16T00:37:40.054 INFO:teuthology.orchestra.run.smithi112.stdout:fs_name cephfs 2024-09-16T00:37:40.054 INFO:teuthology.orchestra.run.smithi112.stdout:epoch 25 2024-09-16T00:37:40.054 INFO:teuthology.orchestra.run.smithi112.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-16T00:37:40.055 INFO:teuthology.orchestra.run.smithi112.stdout:created 2024-09-16T00:23:22.328199+0000 2024-09-16T00:37:40.055 INFO:teuthology.orchestra.run.smithi112.stdout:modified 2024-09-16T00:33:29.012991+0000 2024-09-16T00:37:40.055 INFO:teuthology.orchestra.run.smithi112.stdout:tableserver 0 2024-09-16T00:37:40.055 INFO:teuthology.orchestra.run.smithi112.stdout:root 0 2024-09-16T00:37:40.055 INFO:teuthology.orchestra.run.smithi112.stdout:session_timeout 60 2024-09-16T00:37:40.055 INFO:teuthology.orchestra.run.smithi112.stdout:session_autoclose 300 2024-09-16T00:37:40.055 INFO:teuthology.orchestra.run.smithi112.stdout:max_file_size 1099511627776 2024-09-16T00:37:40.055 INFO:teuthology.orchestra.run.smithi112.stdout:max_xattr_size 65536 2024-09-16T00:37:40.055 INFO:teuthology.orchestra.run.smithi112.stdout:required_client_features {} 2024-09-16T00:37:40.055 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure 0 2024-09-16T00:37:40.055 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure_osd_epoch 82 2024-09-16T00:37:40.055 INFO:teuthology.orchestra.run.smithi112.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:37:40.055 INFO:teuthology.orchestra.run.smithi112.stdout:max_mds 2 2024-09-16T00:37:40.055 INFO:teuthology.orchestra.run.smithi112.stdout:in 0,1 2024-09-16T00:37:40.056 INFO:teuthology.orchestra.run.smithi112.stdout:up {} 2024-09-16T00:37:40.056 INFO:teuthology.orchestra.run.smithi112.stdout:failed 0,1 2024-09-16T00:37:40.056 INFO:teuthology.orchestra.run.smithi112.stdout:damaged 2024-09-16T00:37:40.056 INFO:teuthology.orchestra.run.smithi112.stdout:stopped 2024-09-16T00:37:40.056 INFO:teuthology.orchestra.run.smithi112.stdout:data_pools [3] 2024-09-16T00:37:40.056 INFO:teuthology.orchestra.run.smithi112.stdout:metadata_pool 2 2024-09-16T00:37:40.056 INFO:teuthology.orchestra.run.smithi112.stdout:inline_data enabled 2024-09-16T00:37:40.056 INFO:teuthology.orchestra.run.smithi112.stdout:balancer 2024-09-16T00:37:40.056 INFO:teuthology.orchestra.run.smithi112.stdout:bal_rank_mask -1 2024-09-16T00:37:40.056 INFO:teuthology.orchestra.run.smithi112.stdout:standby_count_wanted 1 2024-09-16T00:37:40.056 INFO:teuthology.orchestra.run.smithi112.stdout:qdb_cluster leader: 0 members: 2024-09-16T00:37:40.056 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:37:40.056 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:37:40.057 INFO:teuthology.orchestra.run.smithi112.stdout:Standby daemons: 2024-09-16T00:37:40.057 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:37:40.057 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.ithwzj{-1:34392} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.112:6826/476229221,v1:172.21.15.112:6827/476229221] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:37:40.057 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.hxfwkh{-1:34404} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.112:6828/2356899876,v1:172.21.15.112:6829/2356899876] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:37:40.057 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.gawwgw{-1:44343} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.169:6824/1635329957,v1:172.21.15.169:6825/1635329957] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:37:40.057 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.lgeqrj{-1:44347} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.169:6826/184810031,v1:172.21.15.169:6827/184810031] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:37:40.058 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 34 2024-09-16T00:37:40.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.055+0000 7f87164cf640 1 -- 172.21.15.112:0/172057481 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f86fc06e530 msgr2=0x7f86fc0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:40.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.055+0000 7f87164cf640 1 --2- 172.21.15.112:0/172057481 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f86fc06e530 0x7f86fc0709e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8700006fd0 tx=0x7f8700008040 comp rx=0 tx=0).stop 2024-09-16T00:37:40.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.055+0000 7f87164cf640 1 -- 172.21.15.112:0/172057481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87100fcf80 msgr2=0x7f8710112680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:40.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.055+0000 7f87164cf640 1 --2- 172.21.15.112:0/172057481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87100fcf80 0x7f8710112680 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f86f8003080 tx=0x7f86f8009da0 comp rx=0 tx=0).stop 2024-09-16T00:37:40.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.055+0000 7f87164cf640 1 -- 172.21.15.112:0/172057481 shutdown_connections 2024-09-16T00:37:40.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.055+0000 7f87164cf640 1 --2- 172.21.15.112:0/172057481 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f86fc06e530 0x7f86fc0709e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:40.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.055+0000 7f87164cf640 1 --2- 172.21.15.112:0/172057481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87100fcf80 0x7f8710112680 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:40.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.055+0000 7f87164cf640 1 --2- 172.21.15.112:0/172057481 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f87100fb810 0x7f8710112140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:40.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.055+0000 7f87164cf640 1 -- 172.21.15.112:0/172057481 >> 172.21.15.112:0/172057481 conn(0x7f87100723b0 msgr2=0x7f87100fa480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:40.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.055+0000 7f87164cf640 1 -- 172.21.15.112:0/172057481 shutdown_connections 2024-09-16T00:37:40.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.055+0000 7f87164cf640 1 -- 172.21.15.112:0/172057481 wait complete. 2024-09-16T00:37:40.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.134+0000 7fd65794c640 1 -- 172.21.15.112:0/352870613 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd650102ef0 msgr2=0x7fd6500fc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:40.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.134+0000 7fd65794c640 1 --2- 172.21.15.112:0/352870613 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd650102ef0 0x7fd6500fc5b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd64c0048d0 tx=0x7fd64c009da0 comp rx=0 tx=0).stop 2024-09-16T00:37:40.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.134+0000 7fd65794c640 1 -- 172.21.15.112:0/352870613 shutdown_connections 2024-09-16T00:37:40.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.134+0000 7fd65794c640 1 --2- 172.21.15.112:0/352870613 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd650102ef0 0x7fd6500fc5b0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:40.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.134+0000 7fd65794c640 1 --2- 172.21.15.112:0/352870613 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd650102550 0x7fd650102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:40.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.134+0000 7fd65794c640 1 -- 172.21.15.112:0/352870613 >> 172.21.15.112:0/352870613 conn(0x7fd6500f80f0 msgr2=0x7fd6500fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:40.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.134+0000 7fd65794c640 1 -- 172.21.15.112:0/352870613 shutdown_connections 2024-09-16T00:37:40.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.134+0000 7fd65794c640 1 -- 172.21.15.112:0/352870613 wait complete. 2024-09-16T00:37:40.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.135+0000 7fd65794c640 1 Processor -- start 2024-09-16T00:37:40.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.135+0000 7fd65794c640 1 -- start start 2024-09-16T00:37:40.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.135+0000 7fd65794c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd650102550 0x7fd650113500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:40.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.135+0000 7fd65794c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd650102ef0 0x7fd650113a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:40.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.135+0000 7fd65794c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd650114090 con 0x7fd650102550 2024-09-16T00:37:40.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.135+0000 7fd65794c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd65010d4a0 con 0x7fd650102ef0 2024-09-16T00:37:40.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.135+0000 7fd6556e8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd650102550 0x7fd650113500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:40.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.135+0000 7fd6556e8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd650102550 0x7fd650113500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47688/0 (socket says 172.21.15.112:47688) 2024-09-16T00:37:40.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.135+0000 7fd6556e8640 1 -- 172.21.15.112:0/1445193487 learned_addr learned my addr 172.21.15.112:0/1445193487 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:37:40.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.135+0000 7fd654ee7640 1 --2- 172.21.15.112:0/1445193487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd650102ef0 0x7fd650113a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:40.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.135+0000 7fd6556e8640 1 -- 172.21.15.112:0/1445193487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd650102ef0 msgr2=0x7fd650113a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:40.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.135+0000 7fd6556e8640 1 --2- 172.21.15.112:0/1445193487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd650102ef0 0x7fd650113a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:40.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.135+0000 7fd6556e8640 1 -- 172.21.15.112:0/1445193487 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd64c005040 con 0x7fd650102550 2024-09-16T00:37:40.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.135+0000 7fd654ee7640 1 --2- 172.21.15.112:0/1445193487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd650102ef0 0x7fd650113a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:37:40.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.136+0000 7fd6556e8640 1 --2- 172.21.15.112:0/1445193487 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd650102550 0x7fd650113500 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd640002a50 tx=0x7fd640004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:40.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.136+0000 7fd645ffb640 1 -- 172.21.15.112:0/1445193487 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd640007c30 con 0x7fd650102550 2024-09-16T00:37:40.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.136+0000 7fd645ffb640 1 -- 172.21.15.112:0/1445193487 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fd64000f040 con 0x7fd650102550 2024-09-16T00:37:40.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.136+0000 7fd645ffb640 1 -- 172.21.15.112:0/1445193487 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd64000db50 con 0x7fd650102550 2024-09-16T00:37:40.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.136+0000 7fd65794c640 1 -- 172.21.15.112:0/1445193487 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd65010d780 con 0x7fd650102550 2024-09-16T00:37:40.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.136+0000 7fd65794c640 1 -- 172.21.15.112:0/1445193487 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd65010dcf0 con 0x7fd650102550 2024-09-16T00:37:40.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.137+0000 7fd65794c640 1 -- 172.21.15.112:0/1445193487 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6500fdcd0 con 0x7fd650102550 2024-09-16T00:37:40.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.137+0000 7fd645ffb640 1 -- 172.21.15.112:0/1445193487 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fd64000dcb0 con 0x7fd650102550 2024-09-16T00:37:40.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.138+0000 7fd645ffb640 1 --2- 172.21.15.112:0/1445193487 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd63c06e530 0x7fd63c0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:40.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.138+0000 7fd645ffb640 1 -- 172.21.15.112:0/1445193487 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7fd64008e2e0 con 0x7fd650102550 2024-09-16T00:37:40.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.138+0000 7fd654ee7640 1 --2- 172.21.15.112:0/1445193487 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd63c06e530 0x7fd63c0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:40.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.139+0000 7fd654ee7640 1 --2- 172.21.15.112:0/1445193487 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd63c06e530 0x7fd63c0709e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd64c000c00 tx=0x7fd64c035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:40.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.142+0000 7fd645ffb640 1 -- 172.21.15.112:0/1445193487 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fd640057250 con 0x7fd650102550 2024-09-16T00:37:40.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.325+0000 7fd65794c640 1 -- 172.21.15.112:0/1445193487 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd65004b750 con 0x7fd63c06e530 2024-09-16T00:37:40.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.326+0000 7fd645ffb640 1 -- 172.21.15.112:0/1445193487 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fd65004b750 con 0x7fd63c06e530 2024-09-16T00:37:40.326 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:37:40.326 INFO:teuthology.orchestra.run.smithi112.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80", 2024-09-16T00:37:40.326 INFO:teuthology.orchestra.run.smithi112.stdout: "in_progress": true, 2024-09-16T00:37:40.326 INFO:teuthology.orchestra.run.smithi112.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T00:37:40.326 INFO:teuthology.orchestra.run.smithi112.stdout: "services_complete": [ 2024-09-16T00:37:40.327 INFO:teuthology.orchestra.run.smithi112.stdout: "mds", 2024-09-16T00:37:40.327 INFO:teuthology.orchestra.run.smithi112.stdout: "osd", 2024-09-16T00:37:40.327 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr", 2024-09-16T00:37:40.327 INFO:teuthology.orchestra.run.smithi112.stdout: "mon", 2024-09-16T00:37:40.327 INFO:teuthology.orchestra.run.smithi112.stdout: "crash" 2024-09-16T00:37:40.327 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-16T00:37:40.327 INFO:teuthology.orchestra.run.smithi112.stdout: "progress": "16/21 daemons upgraded", 2024-09-16T00:37:40.327 INFO:teuthology.orchestra.run.smithi112.stdout: "message": "Currently upgrading mds daemons", 2024-09-16T00:37:40.327 INFO:teuthology.orchestra.run.smithi112.stdout: "is_paused": false 2024-09-16T00:37:40.327 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:37:40.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.328+0000 7fd65794c640 1 -- 172.21.15.112:0/1445193487 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd63c06e530 msgr2=0x7fd63c0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:40.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.328+0000 7fd65794c640 1 --2- 172.21.15.112:0/1445193487 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd63c06e530 0x7fd63c0709e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd64c000c00 tx=0x7fd64c035040 comp rx=0 tx=0).stop 2024-09-16T00:37:40.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.328+0000 7fd65794c640 1 -- 172.21.15.112:0/1445193487 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd650102550 msgr2=0x7fd650113500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:40.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.328+0000 7fd65794c640 1 --2- 172.21.15.112:0/1445193487 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd650102550 0x7fd650113500 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd640002a50 tx=0x7fd640004600 comp rx=0 tx=0).stop 2024-09-16T00:37:40.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.328+0000 7fd65794c640 1 -- 172.21.15.112:0/1445193487 shutdown_connections 2024-09-16T00:37:40.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.328+0000 7fd65794c640 1 --2- 172.21.15.112:0/1445193487 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd63c06e530 0x7fd63c0709e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:40.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.328+0000 7fd65794c640 1 --2- 172.21.15.112:0/1445193487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd650102ef0 0x7fd650113a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:40.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.328+0000 7fd65794c640 1 --2- 172.21.15.112:0/1445193487 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd650102550 0x7fd650113500 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:40.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.328+0000 7fd65794c640 1 -- 172.21.15.112:0/1445193487 >> 172.21.15.112:0/1445193487 conn(0x7fd6500f80f0 msgr2=0x7fd6500f9150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:40.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.329+0000 7fd65794c640 1 -- 172.21.15.112:0/1445193487 shutdown_connections 2024-09-16T00:37:40.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.329+0000 7fd65794c640 1 -- 172.21.15.112:0/1445193487 wait complete. 2024-09-16T00:37:40.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.406+0000 7ff8e6ee8640 1 -- 172.21.15.112:0/3862121020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8e00facf0 msgr2=0x7ff8e00fb160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:40.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.406+0000 7ff8e6ee8640 1 --2- 172.21.15.112:0/3862121020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8e00facf0 0x7ff8e00fb160 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff8d00048d0 tx=0x7ff8d0009da0 comp rx=0 tx=0).stop 2024-09-16T00:37:40.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.406+0000 7ff8e6ee8640 1 -- 172.21.15.112:0/3862121020 shutdown_connections 2024-09-16T00:37:40.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.406+0000 7ff8e6ee8640 1 --2- 172.21.15.112:0/3862121020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8e00facf0 0x7ff8e00fb160 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:40.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.406+0000 7ff8e6ee8640 1 --2- 172.21.15.112:0/3862121020 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8e0102680 0x7ff8e0102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:40.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.406+0000 7ff8e6ee8640 1 -- 172.21.15.112:0/3862121020 >> 172.21.15.112:0/3862121020 conn(0x7ff8e0070d50 msgr2=0x7ff8e0071150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:40.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.406+0000 7ff8e6ee8640 1 -- 172.21.15.112:0/3862121020 shutdown_connections 2024-09-16T00:37:40.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.406+0000 7ff8e6ee8640 1 -- 172.21.15.112:0/3862121020 wait complete. 2024-09-16T00:37:40.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.407+0000 7ff8e6ee8640 1 Processor -- start 2024-09-16T00:37:40.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.407+0000 7ff8e6ee8640 1 -- start start 2024-09-16T00:37:40.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.407+0000 7ff8e6ee8640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8e00facf0 0x7ff8e01132d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:40.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.407+0000 7ff8e6ee8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8e0102680 0x7ff8e0113810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:40.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.407+0000 7ff8e6ee8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e0113e10 con 0x7ff8e0102680 2024-09-16T00:37:40.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.407+0000 7ff8e6ee8640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e010d250 con 0x7ff8e00facf0 2024-09-16T00:37:40.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.407+0000 7ff8d7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8e0102680 0x7ff8e0113810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:40.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.408+0000 7ff8d7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8e0102680 0x7ff8e0113810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47720/0 (socket says 172.21.15.112:47720) 2024-09-16T00:37:40.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.408+0000 7ff8d7fff640 1 -- 172.21.15.112:0/3020423730 learned_addr learned my addr 172.21.15.112:0/3020423730 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:37:40.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.408+0000 7ff8e4c84640 1 --2- 172.21.15.112:0/3020423730 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8e00facf0 0x7ff8e01132d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:40.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.408+0000 7ff8d7fff640 1 -- 172.21.15.112:0/3020423730 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8e00facf0 msgr2=0x7ff8e01132d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:40.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.408+0000 7ff8d7fff640 1 --2- 172.21.15.112:0/3020423730 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8e00facf0 0x7ff8e01132d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:40.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.408+0000 7ff8d7fff640 1 -- 172.21.15.112:0/3020423730 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8d0005040 con 0x7ff8e0102680 2024-09-16T00:37:40.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.408+0000 7ff8d7fff640 1 --2- 172.21.15.112:0/3020423730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8e0102680 0x7ff8e0113810 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0000c00 tx=0x7ff8d0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:40.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.408+0000 7ff8d57fa640 1 -- 172.21.15.112:0/3020423730 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8d0038070 con 0x7ff8e0102680 2024-09-16T00:37:40.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.408+0000 7ff8d57fa640 1 -- 172.21.15.112:0/3020423730 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7ff8d002ade0 con 0x7ff8e0102680 2024-09-16T00:37:40.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.408+0000 7ff8e6ee8640 1 -- 172.21.15.112:0/3020423730 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8e010d4d0 con 0x7ff8e0102680 2024-09-16T00:37:40.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.408+0000 7ff8d57fa640 1 -- 172.21.15.112:0/3020423730 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8d002c7e0 con 0x7ff8e0102680 2024-09-16T00:37:40.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.408+0000 7ff8e6ee8640 1 -- 172.21.15.112:0/3020423730 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8e010d990 con 0x7ff8e0102680 2024-09-16T00:37:40.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.409+0000 7ff8e6ee8640 1 -- 172.21.15.112:0/3020423730 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8e00657f0 con 0x7ff8e0102680 2024-09-16T00:37:40.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.410+0000 7ff8d57fa640 1 -- 172.21.15.112:0/3020423730 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7ff8d0030020 con 0x7ff8e0102680 2024-09-16T00:37:40.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.410+0000 7ff8d57fa640 1 --2- 172.21.15.112:0/3020423730 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff8c806e530 0x7ff8c80709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:37:40.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.410+0000 7ff8d57fa640 1 -- 172.21.15.112:0/3020423730 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7ff8d00adf70 con 0x7ff8e0102680 2024-09-16T00:37:40.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.410+0000 7ff8e4c84640 1 --2- 172.21.15.112:0/3020423730 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff8c806e530 0x7ff8c80709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:37:40.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.411+0000 7ff8e4c84640 1 --2- 172.21.15.112:0/3020423730 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff8c806e530 0x7ff8c80709e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc004ed0 tx=0x7ff8dc00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:37:40.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.415+0000 7ff8d57fa640 1 -- 172.21.15.112:0/3020423730 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7ff8d00e31b0 con 0x7ff8e0102680 2024-09-16T00:37:40.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.645+0000 7ff8e6ee8640 1 -- 172.21.15.112:0/3020423730 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ff8e01096a0 con 0x7ff8e0102680 2024-09-16T00:37:40.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.646+0000 7ff8d57fa640 1 -- 172.21.15.112:0/3020423730 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7ff8d0076f90 con 0x7ff8e0102680 2024-09-16T00:37:40.646 INFO:teuthology.orchestra.run.smithi112.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-16T00:37:40.646 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-16T00:37:40.646 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs is degraded 2024-09-16T00:37:40.646 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T00:37:40.646 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T00:37:40.647 INFO:teuthology.orchestra.run.smithi112.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-16T00:37:40.647 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-16T00:37:40.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.648+0000 7ff8e6ee8640 1 -- 172.21.15.112:0/3020423730 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff8c806e530 msgr2=0x7ff8c80709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:40.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.648+0000 7ff8e6ee8640 1 --2- 172.21.15.112:0/3020423730 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff8c806e530 0x7ff8c80709e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc004ed0 tx=0x7ff8dc00b040 comp rx=0 tx=0).stop 2024-09-16T00:37:40.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.648+0000 7ff8e6ee8640 1 -- 172.21.15.112:0/3020423730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8e0102680 msgr2=0x7ff8e0113810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:37:40.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.648+0000 7ff8e6ee8640 1 --2- 172.21.15.112:0/3020423730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8e0102680 0x7ff8e0113810 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0000c00 tx=0x7ff8d0009da0 comp rx=0 tx=0).stop 2024-09-16T00:37:40.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.648+0000 7ff8e6ee8640 1 -- 172.21.15.112:0/3020423730 shutdown_connections 2024-09-16T00:37:40.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.648+0000 7ff8e6ee8640 1 --2- 172.21.15.112:0/3020423730 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff8c806e530 0x7ff8c80709e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:40.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.648+0000 7ff8e6ee8640 1 --2- 172.21.15.112:0/3020423730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8e0102680 0x7ff8e0113810 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:40.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.648+0000 7ff8e6ee8640 1 --2- 172.21.15.112:0/3020423730 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8e00facf0 0x7ff8e01132d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:37:40.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.648+0000 7ff8e6ee8640 1 -- 172.21.15.112:0/3020423730 >> 172.21.15.112:0/3020423730 conn(0x7ff8e0070d50 msgr2=0x7ff8e00fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:37:40.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.648+0000 7ff8e6ee8640 1 -- 172.21.15.112:0/3020423730 shutdown_connections 2024-09-16T00:37:40.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:37:40.648+0000 7ff8e6ee8640 1 -- 172.21.15.112:0/3020423730 wait complete. 2024-09-16T00:37:41.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:37:40 smithi112 ceph-mon[97750]: from='client.34604 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:37:41.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:37:40 smithi112 ceph-mon[97750]: from='client.34608 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:37:41.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:37:40 smithi112 ceph-mon[97750]: from='client.34612 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:37:41.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:37:40 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/172057481' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:37:41.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:37:40 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/3020423730' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:37:41.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:37:40 smithi169 ceph-mon[77626]: from='client.34604 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:37:41.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:37:40 smithi169 ceph-mon[77626]: from='client.34608 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:37:41.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:37:40 smithi169 ceph-mon[77626]: from='client.34612 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:37:41.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:37:40 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/172057481' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:37:41.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:37:40 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/3020423730' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:37:42.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:37:41 smithi112 ceph-mon[97750]: from='client.34624 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:37:42.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:37:41 smithi112 ceph-mon[97750]: pgmap v298: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:37:42.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:37:41 smithi169 ceph-mon[77626]: from='client.34624 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:37:42.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:37:41 smithi169 ceph-mon[77626]: pgmap v298: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:37:44.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:37:43 smithi112 ceph-mon[97750]: pgmap v299: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:37:44.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:37:43 smithi169 ceph-mon[77626]: pgmap v299: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:37:46.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:37:45 smithi112 ceph-mon[97750]: pgmap v300: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:37:46.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:37:45 smithi169 ceph-mon[77626]: pgmap v300: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:37:47.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:37:46 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:37:47.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:37:46 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:37:48.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:37:47 smithi112 ceph-mon[97750]: pgmap v301: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:37:48.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:37:47 smithi169 ceph-mon[77626]: pgmap v301: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:37:50.078 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:37:49 smithi169 ceph-mon[77626]: pgmap v302: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:37:50.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:37:49 smithi112 ceph-mon[97750]: pgmap v302: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:37:52.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:37:51 smithi112 ceph-mon[97750]: pgmap v303: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:37:52.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:37:51 smithi169 ceph-mon[77626]: pgmap v303: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:37:54.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:37:53 smithi112 ceph-mon[97750]: pgmap v304: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:37:54.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:37:53 smithi169 ceph-mon[77626]: pgmap v304: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:37:56.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:37:55 smithi112 ceph-mon[97750]: pgmap v305: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:37:56.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:37:55 smithi169 ceph-mon[77626]: pgmap v305: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:37:58.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:37:57 smithi112 ceph-mon[97750]: pgmap v306: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:37:58.278 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:37:57 smithi169 ceph-mon[77626]: pgmap v306: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:38:00.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:37:59 smithi112 ceph-mon[97750]: pgmap v307: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:38:00.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:37:59 smithi169 ceph-mon[77626]: pgmap v307: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:38:02.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:38:01 smithi112 ceph-mon[97750]: pgmap v308: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:38:02.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:38:01 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:38:02.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:38:01 smithi169 ceph-mon[77626]: pgmap v308: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:38:02.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:38:01 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:38:04.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:38:03 smithi112 ceph-mon[97750]: pgmap v309: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:38:04.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:38:03 smithi169 ceph-mon[77626]: pgmap v309: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:38:06.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:38:05 smithi112 ceph-mon[97750]: pgmap v310: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:38:06.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:38:05 smithi169 ceph-mon[77626]: pgmap v310: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:38:08.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:38:07 smithi112 ceph-mon[97750]: pgmap v311: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:38:08.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:38:07 smithi169 ceph-mon[77626]: pgmap v311: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:38:09.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:38:08 smithi169 ceph-mon[77626]: pgmap v312: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:38:09.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:38:08 smithi112 ceph-mon[97750]: pgmap v312: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:38:10.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:10.740+0000 7f2b50f9b640 1 -- 172.21.15.112:0/1907282079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b4c0fac80 msgr2=0x7f2b4c0fb0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:10.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:10.740+0000 7f2b50f9b640 1 --2- 172.21.15.112:0/1907282079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b4c0fac80 0x7f2b4c0fb0f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c0048d0 tx=0x7f2b3c009da0 comp rx=0 tx=0).stop 2024-09-16T00:38:10.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:10.740+0000 7f2b50f9b640 1 -- 172.21.15.112:0/1907282079 shutdown_connections 2024-09-16T00:38:10.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:10.740+0000 7f2b50f9b640 1 --2- 172.21.15.112:0/1907282079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b4c0fac80 0x7f2b4c0fb0f0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:10.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:10.740+0000 7f2b50f9b640 1 --2- 172.21.15.112:0/1907282079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b4c102620 0x7f2b4c1029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:10.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:10.740+0000 7f2b50f9b640 1 -- 172.21.15.112:0/1907282079 >> 172.21.15.112:0/1907282079 conn(0x7f2b4c070ce0 msgr2=0x7f2b4c0710e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:10.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:10.740+0000 7f2b50f9b640 1 -- 172.21.15.112:0/1907282079 shutdown_connections 2024-09-16T00:38:10.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:10.740+0000 7f2b50f9b640 1 -- 172.21.15.112:0/1907282079 wait complete. 2024-09-16T00:38:10.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:10.741+0000 7f2b50f9b640 1 Processor -- start 2024-09-16T00:38:10.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:10.741+0000 7f2b50f9b640 1 -- start start 2024-09-16T00:38:10.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:10.741+0000 7f2b50f9b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b4c0fac80 0x7f2b4c1131e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:10.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:10.741+0000 7f2b50f9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b4c102620 0x7f2b4c113720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:10.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:10.741+0000 7f2b50f9b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b4c113d20 con 0x7f2b4c102620 2024-09-16T00:38:10.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:10.741+0000 7f2b50f9b640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b4c10d160 con 0x7f2b4c0fac80 2024-09-16T00:38:10.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:10.741+0000 7f2b49d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b4c102620 0x7f2b4c113720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:10.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:10.741+0000 7f2b49d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b4c102620 0x7f2b4c113720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50732/0 (socket says 172.21.15.112:50732) 2024-09-16T00:38:10.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:10.741+0000 7f2b49d9b640 1 -- 172.21.15.112:0/2713462833 learned_addr learned my addr 172.21.15.112:0/2713462833 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:38:10.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:10.741+0000 7f2b4a59c640 1 --2- 172.21.15.112:0/2713462833 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b4c0fac80 0x7f2b4c1131e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:10.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:10.741+0000 7f2b49d9b640 1 -- 172.21.15.112:0/2713462833 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b4c0fac80 msgr2=0x7f2b4c1131e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:10.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:10.741+0000 7f2b49d9b640 1 --2- 172.21.15.112:0/2713462833 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b4c0fac80 0x7f2b4c1131e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:10.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:10.741+0000 7f2b49d9b640 1 -- 172.21.15.112:0/2713462833 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b3c005040 con 0x7f2b4c102620 2024-09-16T00:38:10.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:10.741+0000 7f2b49d9b640 1 --2- 172.21.15.112:0/2713462833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b4c102620 0x7f2b4c113720 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c0043d0 tx=0x7f2b3c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:10.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:10.742+0000 7f2b42ffd640 1 -- 172.21.15.112:0/2713462833 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b3c038070 con 0x7f2b4c102620 2024-09-16T00:38:10.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:10.742+0000 7f2b42ffd640 1 -- 172.21.15.112:0/2713462833 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f2b3c0096e0 con 0x7f2b4c102620 2024-09-16T00:38:10.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:10.742+0000 7f2b50f9b640 1 -- 172.21.15.112:0/2713462833 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b4c10d3e0 con 0x7f2b4c102620 2024-09-16T00:38:10.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:10.742+0000 7f2b42ffd640 1 -- 172.21.15.112:0/2713462833 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b3c03d420 con 0x7f2b4c102620 2024-09-16T00:38:10.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:10.742+0000 7f2b50f9b640 1 -- 172.21.15.112:0/2713462833 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b4c10d980 con 0x7f2b4c102620 2024-09-16T00:38:10.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:10.743+0000 7f2b50f9b640 1 -- 172.21.15.112:0/2713462833 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b4c0657b0 con 0x7f2b4c102620 2024-09-16T00:38:10.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:10.743+0000 7f2b42ffd640 1 -- 172.21.15.112:0/2713462833 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f2b3c030020 con 0x7f2b4c102620 2024-09-16T00:38:10.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:10.744+0000 7f2b42ffd640 1 --2- 172.21.15.112:0/2713462833 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f2b3806e530 0x7f2b380709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:10.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:10.744+0000 7f2b42ffd640 1 -- 172.21.15.112:0/2713462833 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f2b3c0ae770 con 0x7f2b4c102620 2024-09-16T00:38:10.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:10.744+0000 7f2b4a59c640 1 --2- 172.21.15.112:0/2713462833 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f2b3806e530 0x7f2b380709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:10.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:10.744+0000 7f2b4a59c640 1 --2- 172.21.15.112:0/2713462833 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f2b3806e530 0x7f2b380709e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2b34007fd0 tx=0x7f2b34009e90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:10.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:10.748+0000 7f2b42ffd640 1 -- 172.21.15.112:0/2713462833 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f2b3c077760 con 0x7f2b4c102620 2024-09-16T00:38:10.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:10.931+0000 7f2b50f9b640 1 -- 172.21.15.112:0/2713462833 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2b4c109590 con 0x7f2b3806e530 2024-09-16T00:38:10.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:10.932+0000 7f2b42ffd640 1 -- 172.21.15.112:0/2713462833 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f2b4c109590 con 0x7f2b3806e530 2024-09-16T00:38:10.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:10.934+0000 7f2b50f9b640 1 -- 172.21.15.112:0/2713462833 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f2b3806e530 msgr2=0x7f2b380709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:10.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:10.934+0000 7f2b50f9b640 1 --2- 172.21.15.112:0/2713462833 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f2b3806e530 0x7f2b380709e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2b34007fd0 tx=0x7f2b34009e90 comp rx=0 tx=0).stop 2024-09-16T00:38:10.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:10.934+0000 7f2b50f9b640 1 -- 172.21.15.112:0/2713462833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b4c102620 msgr2=0x7f2b4c113720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:10.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:10.934+0000 7f2b50f9b640 1 --2- 172.21.15.112:0/2713462833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b4c102620 0x7f2b4c113720 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c0043d0 tx=0x7f2b3c009da0 comp rx=0 tx=0).stop 2024-09-16T00:38:10.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:10.934+0000 7f2b50f9b640 1 -- 172.21.15.112:0/2713462833 shutdown_connections 2024-09-16T00:38:10.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:10.934+0000 7f2b50f9b640 1 --2- 172.21.15.112:0/2713462833 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f2b3806e530 0x7f2b380709e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:10.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:10.934+0000 7f2b50f9b640 1 --2- 172.21.15.112:0/2713462833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b4c102620 0x7f2b4c113720 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:10.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:10.934+0000 7f2b50f9b640 1 --2- 172.21.15.112:0/2713462833 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b4c0fac80 0x7f2b4c1131e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:10.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:10.934+0000 7f2b50f9b640 1 -- 172.21.15.112:0/2713462833 >> 172.21.15.112:0/2713462833 conn(0x7f2b4c070ce0 msgr2=0x7f2b4c105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:10.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:10.935+0000 7f2b50f9b640 1 -- 172.21.15.112:0/2713462833 shutdown_connections 2024-09-16T00:38:10.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:10.935+0000 7f2b50f9b640 1 -- 172.21.15.112:0/2713462833 wait complete. 2024-09-16T00:38:10.943 INFO:teuthology.orchestra.run.smithi112.stdout:true 2024-09-16T00:38:11.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.013+0000 7f1a34711640 1 -- 172.21.15.112:0/1223732791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a2c102550 msgr2=0x7f1a2c102920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:11.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.013+0000 7f1a34711640 1 --2- 172.21.15.112:0/1223732791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a2c102550 0x7f1a2c102920 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1a200048d0 tx=0x7f1a20009da0 comp rx=0 tx=0).stop 2024-09-16T00:38:11.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.013+0000 7f1a34711640 1 -- 172.21.15.112:0/1223732791 shutdown_connections 2024-09-16T00:38:11.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.013+0000 7f1a34711640 1 --2- 172.21.15.112:0/1223732791 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a2c102ef0 0x7f1a2c0fc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:11.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.013+0000 7f1a34711640 1 --2- 172.21.15.112:0/1223732791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a2c102550 0x7f1a2c102920 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:11.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.013+0000 7f1a34711640 1 -- 172.21.15.112:0/1223732791 >> 172.21.15.112:0/1223732791 conn(0x7f1a2c0f80f0 msgr2=0x7f1a2c0fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:11.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.013+0000 7f1a34711640 1 -- 172.21.15.112:0/1223732791 shutdown_connections 2024-09-16T00:38:11.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.013+0000 7f1a34711640 1 -- 172.21.15.112:0/1223732791 wait complete. 2024-09-16T00:38:11.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.014+0000 7f1a34711640 1 Processor -- start 2024-09-16T00:38:11.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.014+0000 7f1a34711640 1 -- start start 2024-09-16T00:38:11.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.014+0000 7f1a34711640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a2c102550 0x7f1a2c113570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:11.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.014+0000 7f1a34711640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a2c102ef0 0x7f1a2c113ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:11.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.014+0000 7f1a34711640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2c114100 con 0x7f1a2c102ef0 2024-09-16T00:38:11.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.014+0000 7f1a34711640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2c10d510 con 0x7f1a2c102550 2024-09-16T00:38:11.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.014+0000 7f1a31cac640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a2c102ef0 0x7f1a2c113ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:11.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.014+0000 7f1a31cac640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a2c102ef0 0x7f1a2c113ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50742/0 (socket says 172.21.15.112:50742) 2024-09-16T00:38:11.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.014+0000 7f1a31cac640 1 -- 172.21.15.112:0/1148785654 learned_addr learned my addr 172.21.15.112:0/1148785654 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:38:11.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.014+0000 7f1a324ad640 1 --2- 172.21.15.112:0/1148785654 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a2c102550 0x7f1a2c113570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:11.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.015+0000 7f1a31cac640 1 -- 172.21.15.112:0/1148785654 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a2c102550 msgr2=0x7f1a2c113570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:11.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.015+0000 7f1a31cac640 1 --2- 172.21.15.112:0/1148785654 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a2c102550 0x7f1a2c113570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:11.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.015+0000 7f1a31cac640 1 -- 172.21.15.112:0/1148785654 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a20005040 con 0x7f1a2c102ef0 2024-09-16T00:38:11.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.015+0000 7f1a324ad640 1 --2- 172.21.15.112:0/1148785654 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a2c102550 0x7f1a2c113570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:38:11.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.015+0000 7f1a31cac640 1 --2- 172.21.15.112:0/1148785654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a2c102ef0 0x7f1a2c113ab0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1a28009af0 tx=0x7f1a28004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:11.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.015+0000 7f1a1effd640 1 -- 172.21.15.112:0/1148785654 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a28007c30 con 0x7f1a2c102ef0 2024-09-16T00:38:11.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.015+0000 7f1a1effd640 1 -- 172.21.15.112:0/1148785654 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f1a2800f040 con 0x7f1a2c102ef0 2024-09-16T00:38:11.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.015+0000 7f1a34711640 1 -- 172.21.15.112:0/1148785654 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a2c10d7f0 con 0x7f1a2c102ef0 2024-09-16T00:38:11.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.015+0000 7f1a1effd640 1 -- 172.21.15.112:0/1148785654 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a280094b0 con 0x7f1a2c102ef0 2024-09-16T00:38:11.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.015+0000 7f1a34711640 1 -- 172.21.15.112:0/1148785654 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a2c10dd90 con 0x7f1a2c102ef0 2024-09-16T00:38:11.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.016+0000 7f1a34711640 1 -- 172.21.15.112:0/1148785654 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a2c0fdcd0 con 0x7f1a2c102ef0 2024-09-16T00:38:11.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.017+0000 7f1a1effd640 1 -- 172.21.15.112:0/1148785654 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f1a280096e0 con 0x7f1a2c102ef0 2024-09-16T00:38:11.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.017+0000 7f1a1effd640 1 --2- 172.21.15.112:0/1148785654 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f1a1806e530 0x7f1a180709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:11.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.017+0000 7f1a1effd640 1 -- 172.21.15.112:0/1148785654 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f1a2808e1f0 con 0x7f1a2c102ef0 2024-09-16T00:38:11.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.017+0000 7f1a324ad640 1 --2- 172.21.15.112:0/1148785654 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f1a1806e530 0x7f1a180709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:11.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.018+0000 7f1a324ad640 1 --2- 172.21.15.112:0/1148785654 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f1a1806e530 0x7f1a180709e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1a20007ac0 tx=0x7f1a20035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:11.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.022+0000 7f1a1effd640 1 -- 172.21.15.112:0/1148785654 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f1a280571e0 con 0x7f1a2c102ef0 2024-09-16T00:38:11.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.204+0000 7f1a34711640 1 -- 172.21.15.112:0/1148785654 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1a2c1095a0 con 0x7f1a1806e530 2024-09-16T00:38:11.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.205+0000 7f1a1effd640 1 -- 172.21.15.112:0/1148785654 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f1a2c1095a0 con 0x7f1a1806e530 2024-09-16T00:38:11.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.207+0000 7f1a34711640 1 -- 172.21.15.112:0/1148785654 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f1a1806e530 msgr2=0x7f1a180709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:11.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.207+0000 7f1a34711640 1 --2- 172.21.15.112:0/1148785654 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f1a1806e530 0x7f1a180709e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1a20007ac0 tx=0x7f1a20035040 comp rx=0 tx=0).stop 2024-09-16T00:38:11.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.207+0000 7f1a34711640 1 -- 172.21.15.112:0/1148785654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a2c102ef0 msgr2=0x7f1a2c113ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:11.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.207+0000 7f1a34711640 1 --2- 172.21.15.112:0/1148785654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a2c102ef0 0x7f1a2c113ab0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1a28009af0 tx=0x7f1a28004570 comp rx=0 tx=0).stop 2024-09-16T00:38:11.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.208+0000 7f1a34711640 1 -- 172.21.15.112:0/1148785654 shutdown_connections 2024-09-16T00:38:11.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.208+0000 7f1a34711640 1 --2- 172.21.15.112:0/1148785654 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f1a1806e530 0x7f1a180709e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:11.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.208+0000 7f1a34711640 1 --2- 172.21.15.112:0/1148785654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a2c102ef0 0x7f1a2c113ab0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:11.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.208+0000 7f1a34711640 1 --2- 172.21.15.112:0/1148785654 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a2c102550 0x7f1a2c113570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:11.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.208+0000 7f1a34711640 1 -- 172.21.15.112:0/1148785654 >> 172.21.15.112:0/1148785654 conn(0x7f1a2c0f80f0 msgr2=0x7f1a2c0f90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:11.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.208+0000 7f1a34711640 1 -- 172.21.15.112:0/1148785654 shutdown_connections 2024-09-16T00:38:11.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.208+0000 7f1a34711640 1 -- 172.21.15.112:0/1148785654 wait complete. 2024-09-16T00:38:11.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.286+0000 7f04764bc640 1 -- 172.21.15.112:0/3841369649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0470102670 msgr2=0x7f0470102a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:11.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.286+0000 7f04764bc640 1 --2- 172.21.15.112:0/3841369649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0470102670 0x7f0470102a40 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f04600048d0 tx=0x7f0460009da0 comp rx=0 tx=0).stop 2024-09-16T00:38:11.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.286+0000 7f04764bc640 1 -- 172.21.15.112:0/3841369649 shutdown_connections 2024-09-16T00:38:11.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.286+0000 7f04764bc640 1 --2- 172.21.15.112:0/3841369649 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04700facf0 0x7f04700fb160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:11.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.286+0000 7f04764bc640 1 --2- 172.21.15.112:0/3841369649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0470102670 0x7f0470102a40 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:11.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.286+0000 7f04764bc640 1 -- 172.21.15.112:0/3841369649 >> 172.21.15.112:0/3841369649 conn(0x7f0470070d50 msgr2=0x7f0470071150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:11.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.286+0000 7f04764bc640 1 -- 172.21.15.112:0/3841369649 shutdown_connections 2024-09-16T00:38:11.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.286+0000 7f04764bc640 1 -- 172.21.15.112:0/3841369649 wait complete. 2024-09-16T00:38:11.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.287+0000 7f04764bc640 1 Processor -- start 2024-09-16T00:38:11.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.287+0000 7f04764bc640 1 -- start start 2024-09-16T00:38:11.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.287+0000 7f04764bc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04700facf0 0x7f04701131b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:11.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.287+0000 7f04764bc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0470102670 0x7f04701136f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:11.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.287+0000 7f04764bc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0470113cf0 con 0x7f0470102670 2024-09-16T00:38:11.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.287+0000 7f04764bc640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f047010d130 con 0x7f04700facf0 2024-09-16T00:38:11.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.287+0000 7f046f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0470102670 0x7f04701136f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:11.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.287+0000 7f046f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0470102670 0x7f04701136f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50774/0 (socket says 172.21.15.112:50774) 2024-09-16T00:38:11.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.287+0000 7f046f7fe640 1 -- 172.21.15.112:0/3674665942 learned_addr learned my addr 172.21.15.112:0/3674665942 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:38:11.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.287+0000 7f046ffff640 1 --2- 172.21.15.112:0/3674665942 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04700facf0 0x7f04701131b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:11.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.288+0000 7f046f7fe640 1 -- 172.21.15.112:0/3674665942 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04700facf0 msgr2=0x7f04701131b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:11.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.288+0000 7f046f7fe640 1 --2- 172.21.15.112:0/3674665942 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04700facf0 0x7f04701131b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:11.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.288+0000 7f046f7fe640 1 -- 172.21.15.112:0/3674665942 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0460005040 con 0x7f0470102670 2024-09-16T00:38:11.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.288+0000 7f046ffff640 1 --2- 172.21.15.112:0/3674665942 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04700facf0 0x7f04701131b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:38:11.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.288+0000 7f046f7fe640 1 --2- 172.21.15.112:0/3674665942 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0470102670 0x7f04701136f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f046400c830 tx=0x7f0464004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:11.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.288+0000 7f046cff9640 1 -- 172.21.15.112:0/3674665942 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f046400a850 con 0x7f0470102670 2024-09-16T00:38:11.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.288+0000 7f04764bc640 1 -- 172.21.15.112:0/3674665942 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f047010d410 con 0x7f0470102670 2024-09-16T00:38:11.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.288+0000 7f046cff9640 1 -- 172.21.15.112:0/3674665942 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f046400ae50 con 0x7f0470102670 2024-09-16T00:38:11.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.288+0000 7f046cff9640 1 -- 172.21.15.112:0/3674665942 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f04640113a0 con 0x7f0470102670 2024-09-16T00:38:11.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.288+0000 7f04764bc640 1 -- 172.21.15.112:0/3674665942 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f047010d9b0 con 0x7f0470102670 2024-09-16T00:38:11.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.289+0000 7f04764bc640 1 -- 172.21.15.112:0/3674665942 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04700657b0 con 0x7f0470102670 2024-09-16T00:38:11.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.290+0000 7f046cff9640 1 -- 172.21.15.112:0/3674665942 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f0464012400 con 0x7f0470102670 2024-09-16T00:38:11.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.290+0000 7f046cff9640 1 --2- 172.21.15.112:0/3674665942 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f045806e530 0x7f04580709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:11.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.290+0000 7f046cff9640 1 -- 172.21.15.112:0/3674665942 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f046408e460 con 0x7f0470102670 2024-09-16T00:38:11.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.290+0000 7f046ffff640 1 --2- 172.21.15.112:0/3674665942 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f045806e530 0x7f04580709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:11.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.291+0000 7f046ffff640 1 --2- 172.21.15.112:0/3674665942 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f045806e530 0x7f04580709e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0460009ac0 tx=0x7f0460035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:11.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.294+0000 7f046cff9640 1 -- 172.21.15.112:0/3674665942 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f04640574f0 con 0x7f0470102670 2024-09-16T00:38:11.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.467+0000 7f04764bc640 1 -- 172.21.15.112:0/3674665942 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0470109600 con 0x7f045806e530 2024-09-16T00:38:11.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.476+0000 7f046cff9640 1 -- 172.21.15.112:0/3674665942 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7f0470109600 con 0x7f045806e530 2024-09-16T00:38:11.476 INFO:teuthology.orchestra.run.smithi112.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T00:38:11.476 INFO:teuthology.orchestra.run.smithi112.stdout:alertmanager.smithi112 smithi112 *:9093,9094 running (11m) 4m ago 20m 24.7M - 0.25.0 c8568f914cd2 d85ea0562022 2024-09-16T00:38:11.476 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi112 smithi112 running (8m) 4m ago 20m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c e4fe58f9702a 2024-09-16T00:38:11.477 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi169 smithi169 running (8m) 3m ago 19m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c 3ba2c81d527f 2024-09-16T00:38:11.477 INFO:teuthology.orchestra.run.smithi112.stdout:grafana.smithi112 smithi112 *:3000 running (9m) 4m ago 19m 82.1M - 9.4.12 2bacad6d85d8 51f9d2013da2 2024-09-16T00:38:11.477 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.hxfwkh smithi112 running (4m) 4m ago 14m 14.2M - 19.1.1-326-g8921c8ef f1455bd0219c 1723069902d6 2024-09-16T00:38:11.477 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.ithwzj smithi112 running (4m) 4m ago 14m 12.2M - 19.1.1-326-g8921c8ef f1455bd0219c e7aad5b81db5 2024-09-16T00:38:11.477 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.gawwgw smithi169 running (4m) 3m ago 14m 15.7M - 19.1.1-326-g8921c8ef f1455bd0219c 469be459ee12 2024-09-16T00:38:11.477 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.lgeqrj smithi169 running (3m) 3m ago 14m 13.3M - 19.1.1-326-g8921c8ef f1455bd0219c 31f0fbd2395d 2024-09-16T00:38:11.477 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi112.xrkkbc smithi112 *:8443,9283,8765 running (10m) 4m ago 21m 504M - 19.1.1-326-g8921c8ef f1455bd0219c a981f7fd2810 2024-09-16T00:38:11.477 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi169.mtrobk smithi169 *:8443,9283,8765 running (10m) 3m ago 19m 442M - 19.1.1-326-g8921c8ef f1455bd0219c 4096e5369c2d 2024-09-16T00:38:11.477 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi112 smithi112 running (9m) 4m ago 21m 49.8M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 5aaec033c042 2024-09-16T00:38:11.477 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi169 smithi169 running (8m) 3m ago 19m 38.9M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 7bc609155f3f 2024-09-16T00:38:11.477 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi112 smithi112 *:9100 running (11m) 4m ago 20m 15.5M - 1.5.0 0da6a335fe13 6404d1d601e2 2024-09-16T00:38:11.477 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi169 smithi169 *:9100 running (10m) 3m ago 19m 15.4M - 1.5.0 0da6a335fe13 5505baf702a6 2024-09-16T00:38:11.477 INFO:teuthology.orchestra.run.smithi112.stdout:osd.0 smithi112 running (7m) 4m ago 18m 105M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 53efc9397017 2024-09-16T00:38:11.478 INFO:teuthology.orchestra.run.smithi112.stdout:osd.1 smithi112 running (7m) 4m ago 17m 98.9M 1557M 19.1.1-326-g8921c8ef f1455bd0219c fe6f1a4a2cf4 2024-09-16T00:38:11.478 INFO:teuthology.orchestra.run.smithi112.stdout:osd.2 smithi112 running (6m) 4m ago 17m 102M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 5f7863712057 2024-09-16T00:38:11.478 INFO:teuthology.orchestra.run.smithi112.stdout:osd.3 smithi169 running (5m) 3m ago 17m 92.1M 2581M 19.1.1-326-g8921c8ef f1455bd0219c ee07b5577810 2024-09-16T00:38:11.478 INFO:teuthology.orchestra.run.smithi112.stdout:osd.4 smithi169 running (5m) 3m ago 16m 103M 2581M 19.1.1-326-g8921c8ef f1455bd0219c 769f026a54eb 2024-09-16T00:38:11.478 INFO:teuthology.orchestra.run.smithi112.stdout:osd.5 smithi169 running (4m) 3m ago 16m 125M 2581M 19.1.1-326-g8921c8ef f1455bd0219c a6007c13cf78 2024-09-16T00:38:11.478 INFO:teuthology.orchestra.run.smithi112.stdout:prometheus.smithi112 smithi112 *:9095 running (10m) 4m ago 19m 64.1M - 2.43.0 a07b618ecd1d 7a1d42e60d2b 2024-09-16T00:38:11.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.478+0000 7f04764bc640 1 -- 172.21.15.112:0/3674665942 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f045806e530 msgr2=0x7f04580709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:11.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.478+0000 7f04764bc640 1 --2- 172.21.15.112:0/3674665942 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f045806e530 0x7f04580709e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0460009ac0 tx=0x7f0460035040 comp rx=0 tx=0).stop 2024-09-16T00:38:11.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.478+0000 7f04764bc640 1 -- 172.21.15.112:0/3674665942 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0470102670 msgr2=0x7f04701136f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:11.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.478+0000 7f04764bc640 1 --2- 172.21.15.112:0/3674665942 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0470102670 0x7f04701136f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f046400c830 tx=0x7f0464004430 comp rx=0 tx=0).stop 2024-09-16T00:38:11.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.478+0000 7f04764bc640 1 -- 172.21.15.112:0/3674665942 shutdown_connections 2024-09-16T00:38:11.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.478+0000 7f04764bc640 1 --2- 172.21.15.112:0/3674665942 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f045806e530 0x7f04580709e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:11.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.478+0000 7f04764bc640 1 --2- 172.21.15.112:0/3674665942 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0470102670 0x7f04701136f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:11.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.478+0000 7f04764bc640 1 --2- 172.21.15.112:0/3674665942 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04700facf0 0x7f04701131b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:11.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.478+0000 7f04764bc640 1 -- 172.21.15.112:0/3674665942 >> 172.21.15.112:0/3674665942 conn(0x7f0470070d50 msgr2=0x7f04700fa020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:11.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.478+0000 7f04764bc640 1 -- 172.21.15.112:0/3674665942 shutdown_connections 2024-09-16T00:38:11.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.478+0000 7f04764bc640 1 -- 172.21.15.112:0/3674665942 wait complete. 2024-09-16T00:38:11.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.555+0000 7f5857c08640 1 -- 172.21.15.112:0/2104946529 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5850102ed0 msgr2=0x7f58500fc590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:11.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.555+0000 7f5857c08640 1 --2- 172.21.15.112:0/2104946529 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5850102ed0 0x7f58500fc590 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f58440048d0 tx=0x7f5844009da0 comp rx=0 tx=0).stop 2024-09-16T00:38:11.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.555+0000 7f5857c08640 1 -- 172.21.15.112:0/2104946529 shutdown_connections 2024-09-16T00:38:11.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.555+0000 7f5857c08640 1 --2- 172.21.15.112:0/2104946529 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5850102ed0 0x7f58500fc590 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:11.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.555+0000 7f5857c08640 1 --2- 172.21.15.112:0/2104946529 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5850102530 0x7f5850102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:11.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.555+0000 7f5857c08640 1 -- 172.21.15.112:0/2104946529 >> 172.21.15.112:0/2104946529 conn(0x7f58500f80f0 msgr2=0x7f58500fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:11.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.556+0000 7f5857c08640 1 -- 172.21.15.112:0/2104946529 shutdown_connections 2024-09-16T00:38:11.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.556+0000 7f5857c08640 1 -- 172.21.15.112:0/2104946529 wait complete. 2024-09-16T00:38:11.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.556+0000 7f5857c08640 1 Processor -- start 2024-09-16T00:38:11.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.556+0000 7f5857c08640 1 -- start start 2024-09-16T00:38:11.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.556+0000 7f5857c08640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5850102530 0x7f58501131d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:11.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.556+0000 7f5857c08640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5850102ed0 0x7f5850113710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:11.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.556+0000 7f5857c08640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5850113d10 con 0x7f5850102530 2024-09-16T00:38:11.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.556+0000 7f5857c08640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f585010d150 con 0x7f5850102ed0 2024-09-16T00:38:11.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.556+0000 7f58559a4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5850102530 0x7f58501131d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:11.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.557+0000 7f58559a4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5850102530 0x7f58501131d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50798/0 (socket says 172.21.15.112:50798) 2024-09-16T00:38:11.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.557+0000 7f58559a4640 1 -- 172.21.15.112:0/4110889337 learned_addr learned my addr 172.21.15.112:0/4110889337 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:38:11.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.557+0000 7f58551a3640 1 --2- 172.21.15.112:0/4110889337 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5850102ed0 0x7f5850113710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:11.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.557+0000 7f58559a4640 1 -- 172.21.15.112:0/4110889337 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5850102ed0 msgr2=0x7f5850113710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:11.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.557+0000 7f58559a4640 1 --2- 172.21.15.112:0/4110889337 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5850102ed0 0x7f5850113710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:11.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.557+0000 7f58559a4640 1 -- 172.21.15.112:0/4110889337 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5844005040 con 0x7f5850102530 2024-09-16T00:38:11.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.557+0000 7f58551a3640 1 --2- 172.21.15.112:0/4110889337 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5850102ed0 0x7f5850113710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:38:11.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.557+0000 7f58559a4640 1 --2- 172.21.15.112:0/4110889337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5850102530 0x7f58501131d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f584000cae0 tx=0x7f5840007e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:11.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.557+0000 7f584e7fc640 1 -- 172.21.15.112:0/4110889337 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f584000ab00 con 0x7f5850102530 2024-09-16T00:38:11.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.557+0000 7f584e7fc640 1 -- 172.21.15.112:0/4110889337 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f584000ac60 con 0x7f5850102530 2024-09-16T00:38:11.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.557+0000 7f5857c08640 1 -- 172.21.15.112:0/4110889337 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f585010d430 con 0x7f5850102530 2024-09-16T00:38:11.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.557+0000 7f5857c08640 1 -- 172.21.15.112:0/4110889337 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f585010d9d0 con 0x7f5850102530 2024-09-16T00:38:11.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.557+0000 7f584e7fc640 1 -- 172.21.15.112:0/4110889337 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58400115e0 con 0x7f5850102530 2024-09-16T00:38:11.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.558+0000 7f5857c08640 1 -- 172.21.15.112:0/4110889337 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f583c005210 con 0x7f5850102530 2024-09-16T00:38:11.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.559+0000 7f584e7fc640 1 -- 172.21.15.112:0/4110889337 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f58400129e0 con 0x7f5850102530 2024-09-16T00:38:11.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.559+0000 7f584e7fc640 1 --2- 172.21.15.112:0/4110889337 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f583806e530 0x7f58380709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:11.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.560+0000 7f584e7fc640 1 -- 172.21.15.112:0/4110889337 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f584008ea80 con 0x7f5850102530 2024-09-16T00:38:11.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.560+0000 7f58551a3640 1 --2- 172.21.15.112:0/4110889337 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f583806e530 0x7f58380709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:11.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.560+0000 7f58551a3640 1 --2- 172.21.15.112:0/4110889337 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f583806e530 0x7f58380709e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f585010ea90 tx=0x7f5844009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:11.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.565+0000 7f584e7fc640 1 -- 172.21.15.112:0/4110889337 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f5840057a70 con 0x7f5850102530 2024-09-16T00:38:11.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.798+0000 7f5857c08640 1 -- 172.21.15.112:0/4110889337 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f583c005cd0 con 0x7f5850102530 2024-09-16T00:38:11.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.799+0000 7f584e7fc640 1 -- 172.21.15.112:0/4110889337 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f584005b130 con 0x7f5850102530 2024-09-16T00:38:11.799 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:38:11.799 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": { 2024-09-16T00:38:11.799 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:38:11.799 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:38:11.799 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": { 2024-09-16T00:38:11.799 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:38:11.800 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:38:11.800 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": { 2024-09-16T00:38:11.800 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 6 2024-09-16T00:38:11.800 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:38:11.800 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": { 2024-09-16T00:38:11.800 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 4 2024-09-16T00:38:11.800 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:38:11.800 INFO:teuthology.orchestra.run.smithi112.stdout: "overall": { 2024-09-16T00:38:11.800 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 14 2024-09-16T00:38:11.800 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-16T00:38:11.800 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:38:11.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.801+0000 7f5857c08640 1 -- 172.21.15.112:0/4110889337 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f583806e530 msgr2=0x7f58380709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:11.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.801+0000 7f5857c08640 1 --2- 172.21.15.112:0/4110889337 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f583806e530 0x7f58380709e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f585010ea90 tx=0x7f5844009da0 comp rx=0 tx=0).stop 2024-09-16T00:38:11.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.801+0000 7f5857c08640 1 -- 172.21.15.112:0/4110889337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5850102530 msgr2=0x7f58501131d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:11.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.801+0000 7f5857c08640 1 --2- 172.21.15.112:0/4110889337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5850102530 0x7f58501131d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f584000cae0 tx=0x7f5840007e50 comp rx=0 tx=0).stop 2024-09-16T00:38:11.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.801+0000 7f5857c08640 1 -- 172.21.15.112:0/4110889337 shutdown_connections 2024-09-16T00:38:11.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.801+0000 7f5857c08640 1 --2- 172.21.15.112:0/4110889337 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f583806e530 0x7f58380709e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:11.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.801+0000 7f5857c08640 1 --2- 172.21.15.112:0/4110889337 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5850102ed0 0x7f5850113710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:11.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.801+0000 7f5857c08640 1 --2- 172.21.15.112:0/4110889337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5850102530 0x7f58501131d0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:11.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.801+0000 7f5857c08640 1 -- 172.21.15.112:0/4110889337 >> 172.21.15.112:0/4110889337 conn(0x7f58500f80f0 msgr2=0x7f58500f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:11.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.801+0000 7f5857c08640 1 -- 172.21.15.112:0/4110889337 shutdown_connections 2024-09-16T00:38:11.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.801+0000 7f5857c08640 1 -- 172.21.15.112:0/4110889337 wait complete. 2024-09-16T00:38:11.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:38:11 smithi112 ceph-mon[97750]: pgmap v313: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:38:11.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.879+0000 7f3955666640 1 -- 172.21.15.112:0/3986657354 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3950102ed0 msgr2=0x7f39500fc590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:11.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.879+0000 7f3955666640 1 --2- 172.21.15.112:0/3986657354 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3950102ed0 0x7f39500fc590 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f39380048d0 tx=0x7f3938009da0 comp rx=0 tx=0).stop 2024-09-16T00:38:11.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.879+0000 7f3955666640 1 -- 172.21.15.112:0/3986657354 shutdown_connections 2024-09-16T00:38:11.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.879+0000 7f3955666640 1 --2- 172.21.15.112:0/3986657354 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3950102ed0 0x7f39500fc590 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:11.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.879+0000 7f3955666640 1 --2- 172.21.15.112:0/3986657354 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3950102530 0x7f3950102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:11.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.879+0000 7f3955666640 1 -- 172.21.15.112:0/3986657354 >> 172.21.15.112:0/3986657354 conn(0x7f39500f80f0 msgr2=0x7f39500fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:11.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.879+0000 7f3955666640 1 -- 172.21.15.112:0/3986657354 shutdown_connections 2024-09-16T00:38:11.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.879+0000 7f3955666640 1 -- 172.21.15.112:0/3986657354 wait complete. 2024-09-16T00:38:11.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.880+0000 7f3955666640 1 Processor -- start 2024-09-16T00:38:11.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.880+0000 7f3955666640 1 -- start start 2024-09-16T00:38:11.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.881+0000 7f3955666640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3950102530 0x7f39501131f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:11.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.881+0000 7f3955666640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3950102ed0 0x7f3950113730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:11.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.881+0000 7f3955666640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3950113d30 con 0x7f3950102ed0 2024-09-16T00:38:11.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.881+0000 7f3955666640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f395010d170 con 0x7f3950102530 2024-09-16T00:38:11.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.881+0000 7f394e7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3950102ed0 0x7f3950113730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:11.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.881+0000 7f394e7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3950102ed0 0x7f3950113730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50814/0 (socket says 172.21.15.112:50814) 2024-09-16T00:38:11.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.881+0000 7f394e7fc640 1 -- 172.21.15.112:0/2016826434 learned_addr learned my addr 172.21.15.112:0/2016826434 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:38:11.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.881+0000 7f394e7fc640 1 -- 172.21.15.112:0/2016826434 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3950102530 msgr2=0x7f39501131f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T00:38:11.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.881+0000 7f394e7fc640 1 --2- 172.21.15.112:0/2016826434 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3950102530 0x7f39501131f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:11.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.881+0000 7f394e7fc640 1 -- 172.21.15.112:0/2016826434 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3938005040 con 0x7f3950102ed0 2024-09-16T00:38:11.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.881+0000 7f394e7fc640 1 --2- 172.21.15.112:0/2016826434 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3950102ed0 0x7f3950113730 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f393802a530 tx=0x7f3938009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:11.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.881+0000 7f39477fe640 1 -- 172.21.15.112:0/2016826434 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3938038070 con 0x7f3950102ed0 2024-09-16T00:38:11.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.881+0000 7f3955666640 1 -- 172.21.15.112:0/2016826434 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f395010d3f0 con 0x7f3950102ed0 2024-09-16T00:38:11.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.881+0000 7f39477fe640 1 -- 172.21.15.112:0/2016826434 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f393802ade0 con 0x7f3950102ed0 2024-09-16T00:38:11.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.881+0000 7f39477fe640 1 -- 172.21.15.112:0/2016826434 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f393802c7a0 con 0x7f3950102ed0 2024-09-16T00:38:11.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.881+0000 7f3955666640 1 -- 172.21.15.112:0/2016826434 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f395010d930 con 0x7f3950102ed0 2024-09-16T00:38:11.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.882+0000 7f3955666640 1 -- 172.21.15.112:0/2016826434 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39500fdcb0 con 0x7f3950102ed0 2024-09-16T00:38:11.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.883+0000 7f39477fe640 1 -- 172.21.15.112:0/2016826434 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f3938030020 con 0x7f3950102ed0 2024-09-16T00:38:11.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.883+0000 7f39477fe640 1 --2- 172.21.15.112:0/2016826434 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f393c06e530 0x7f393c0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:11.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.883+0000 7f39477fe640 1 -- 172.21.15.112:0/2016826434 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f39380ae7b0 con 0x7f3950102ed0 2024-09-16T00:38:11.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.883+0000 7f394effd640 1 --2- 172.21.15.112:0/2016826434 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f393c06e530 0x7f393c0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:11.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.884+0000 7f394effd640 1 --2- 172.21.15.112:0/2016826434 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f393c06e530 0x7f393c0709e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3940004fd0 tx=0x7f394000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:11.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:11.888+0000 7f39477fe640 1 -- 172.21.15.112:0/2016826434 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f3938077720 con 0x7f3950102ed0 2024-09-16T00:38:11.979 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:38:11 smithi169 ceph-mon[77626]: pgmap v313: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:38:12.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.089+0000 7f3955666640 1 -- 172.21.15.112:0/2016826434 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f3950109560 con 0x7f3950102ed0 2024-09-16T00:38:12.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.089+0000 7f39477fe640 1 -- 172.21.15.112:0/2016826434 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1914 (secure 0 0 0) 0x7f393807ade0 con 0x7f3950102ed0 2024-09-16T00:38:12.090 INFO:teuthology.orchestra.run.smithi112.stdout:e34 2024-09-16T00:38:12.090 INFO:teuthology.orchestra.run.smithi112.stdout:btime 2024-09-16T00:34:26:619254+0000 2024-09-16T00:38:12.090 INFO:teuthology.orchestra.run.smithi112.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T00:38:12.090 INFO:teuthology.orchestra.run.smithi112.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:38:12.090 INFO:teuthology.orchestra.run.smithi112.stdout:legacy client fscid: 1 2024-09-16T00:38:12.090 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:38:12.090 INFO:teuthology.orchestra.run.smithi112.stdout:Filesystem 'cephfs' (1) 2024-09-16T00:38:12.090 INFO:teuthology.orchestra.run.smithi112.stdout:fs_name cephfs 2024-09-16T00:38:12.090 INFO:teuthology.orchestra.run.smithi112.stdout:epoch 25 2024-09-16T00:38:12.090 INFO:teuthology.orchestra.run.smithi112.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-16T00:38:12.091 INFO:teuthology.orchestra.run.smithi112.stdout:created 2024-09-16T00:23:22.328199+0000 2024-09-16T00:38:12.091 INFO:teuthology.orchestra.run.smithi112.stdout:modified 2024-09-16T00:33:29.012991+0000 2024-09-16T00:38:12.091 INFO:teuthology.orchestra.run.smithi112.stdout:tableserver 0 2024-09-16T00:38:12.091 INFO:teuthology.orchestra.run.smithi112.stdout:root 0 2024-09-16T00:38:12.091 INFO:teuthology.orchestra.run.smithi112.stdout:session_timeout 60 2024-09-16T00:38:12.091 INFO:teuthology.orchestra.run.smithi112.stdout:session_autoclose 300 2024-09-16T00:38:12.091 INFO:teuthology.orchestra.run.smithi112.stdout:max_file_size 1099511627776 2024-09-16T00:38:12.091 INFO:teuthology.orchestra.run.smithi112.stdout:max_xattr_size 65536 2024-09-16T00:38:12.091 INFO:teuthology.orchestra.run.smithi112.stdout:required_client_features {} 2024-09-16T00:38:12.091 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure 0 2024-09-16T00:38:12.091 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure_osd_epoch 82 2024-09-16T00:38:12.091 INFO:teuthology.orchestra.run.smithi112.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:38:12.091 INFO:teuthology.orchestra.run.smithi112.stdout:max_mds 2 2024-09-16T00:38:12.092 INFO:teuthology.orchestra.run.smithi112.stdout:in 0,1 2024-09-16T00:38:12.092 INFO:teuthology.orchestra.run.smithi112.stdout:up {} 2024-09-16T00:38:12.092 INFO:teuthology.orchestra.run.smithi112.stdout:failed 0,1 2024-09-16T00:38:12.092 INFO:teuthology.orchestra.run.smithi112.stdout:damaged 2024-09-16T00:38:12.092 INFO:teuthology.orchestra.run.smithi112.stdout:stopped 2024-09-16T00:38:12.092 INFO:teuthology.orchestra.run.smithi112.stdout:data_pools [3] 2024-09-16T00:38:12.092 INFO:teuthology.orchestra.run.smithi112.stdout:metadata_pool 2 2024-09-16T00:38:12.092 INFO:teuthology.orchestra.run.smithi112.stdout:inline_data enabled 2024-09-16T00:38:12.092 INFO:teuthology.orchestra.run.smithi112.stdout:balancer 2024-09-16T00:38:12.092 INFO:teuthology.orchestra.run.smithi112.stdout:bal_rank_mask -1 2024-09-16T00:38:12.092 INFO:teuthology.orchestra.run.smithi112.stdout:standby_count_wanted 1 2024-09-16T00:38:12.092 INFO:teuthology.orchestra.run.smithi112.stdout:qdb_cluster leader: 0 members: 2024-09-16T00:38:12.093 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:38:12.093 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:38:12.093 INFO:teuthology.orchestra.run.smithi112.stdout:Standby daemons: 2024-09-16T00:38:12.093 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:38:12.093 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.ithwzj{-1:34392} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.112:6826/476229221,v1:172.21.15.112:6827/476229221] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:38:12.093 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.hxfwkh{-1:34404} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.112:6828/2356899876,v1:172.21.15.112:6829/2356899876] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:38:12.093 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.gawwgw{-1:44343} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.169:6824/1635329957,v1:172.21.15.169:6825/1635329957] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:38:12.093 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.lgeqrj{-1:44347} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.169:6826/184810031,v1:172.21.15.169:6827/184810031] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:38:12.094 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 34 2024-09-16T00:38:12.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.091+0000 7f3955666640 1 -- 172.21.15.112:0/2016826434 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f393c06e530 msgr2=0x7f393c0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:12.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.091+0000 7f3955666640 1 --2- 172.21.15.112:0/2016826434 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f393c06e530 0x7f393c0709e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3940004fd0 tx=0x7f394000b040 comp rx=0 tx=0).stop 2024-09-16T00:38:12.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.091+0000 7f3955666640 1 -- 172.21.15.112:0/2016826434 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3950102ed0 msgr2=0x7f3950113730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:12.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.091+0000 7f3955666640 1 --2- 172.21.15.112:0/2016826434 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3950102ed0 0x7f3950113730 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f393802a530 tx=0x7f3938009da0 comp rx=0 tx=0).stop 2024-09-16T00:38:12.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.091+0000 7f3955666640 1 -- 172.21.15.112:0/2016826434 shutdown_connections 2024-09-16T00:38:12.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.091+0000 7f3955666640 1 --2- 172.21.15.112:0/2016826434 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f393c06e530 0x7f393c0709e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:12.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.091+0000 7f3955666640 1 --2- 172.21.15.112:0/2016826434 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3950102ed0 0x7f3950113730 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:12.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.091+0000 7f3955666640 1 --2- 172.21.15.112:0/2016826434 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3950102530 0x7f39501131f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:12.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.091+0000 7f3955666640 1 -- 172.21.15.112:0/2016826434 >> 172.21.15.112:0/2016826434 conn(0x7f39500f80f0 msgr2=0x7f39500f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:12.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.091+0000 7f3955666640 1 -- 172.21.15.112:0/2016826434 shutdown_connections 2024-09-16T00:38:12.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.091+0000 7f3955666640 1 -- 172.21.15.112:0/2016826434 wait complete. 2024-09-16T00:38:12.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.170+0000 7f4fa2679640 1 -- 172.21.15.112:0/204157673 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f9c102550 msgr2=0x7f4f9c102920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:12.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.170+0000 7f4fa2679640 1 --2- 172.21.15.112:0/204157673 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f9c102550 0x7f4f9c102920 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c0048d0 tx=0x7f4f8c009da0 comp rx=0 tx=0).stop 2024-09-16T00:38:12.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.170+0000 7f4fa2679640 1 -- 172.21.15.112:0/204157673 shutdown_connections 2024-09-16T00:38:12.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.170+0000 7f4fa2679640 1 --2- 172.21.15.112:0/204157673 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f9c102ef0 0x7f4f9c0fc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:12.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.170+0000 7f4fa2679640 1 --2- 172.21.15.112:0/204157673 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f9c102550 0x7f4f9c102920 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:12.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.170+0000 7f4fa2679640 1 -- 172.21.15.112:0/204157673 >> 172.21.15.112:0/204157673 conn(0x7f4f9c0f80f0 msgr2=0x7f4f9c0fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:12.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.170+0000 7f4fa2679640 1 -- 172.21.15.112:0/204157673 shutdown_connections 2024-09-16T00:38:12.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.170+0000 7f4fa2679640 1 -- 172.21.15.112:0/204157673 wait complete. 2024-09-16T00:38:12.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.171+0000 7f4fa2679640 1 Processor -- start 2024-09-16T00:38:12.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.171+0000 7f4fa2679640 1 -- start start 2024-09-16T00:38:12.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.171+0000 7f4fa2679640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f9c102550 0x7f4f9c06fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:12.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.171+0000 7f4fa2679640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f9c102ef0 0x7f4f9c06ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:12.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.171+0000 7f4fa2679640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9c070490 con 0x7f4f9c102ef0 2024-09-16T00:38:12.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.171+0000 7f4fa2679640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9c070600 con 0x7f4f9c102550 2024-09-16T00:38:12.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.171+0000 7f4f93fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f9c102ef0 0x7f4f9c06ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:12.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.171+0000 7f4f93fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f9c102ef0 0x7f4f9c06ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44504/0 (socket says 172.21.15.112:44504) 2024-09-16T00:38:12.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.171+0000 7f4f93fff640 1 -- 172.21.15.112:0/3141519576 learned_addr learned my addr 172.21.15.112:0/3141519576 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:38:12.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.171+0000 7f4f9bfff640 1 --2- 172.21.15.112:0/3141519576 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f9c102550 0x7f4f9c06fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:12.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.172+0000 7f4f93fff640 1 -- 172.21.15.112:0/3141519576 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f9c102550 msgr2=0x7f4f9c06fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:12.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.172+0000 7f4f93fff640 1 --2- 172.21.15.112:0/3141519576 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f9c102550 0x7f4f9c06fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:12.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.172+0000 7f4f93fff640 1 -- 172.21.15.112:0/3141519576 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f8c005040 con 0x7f4f9c102ef0 2024-09-16T00:38:12.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.172+0000 7f4f93fff640 1 --2- 172.21.15.112:0/3141519576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f9c102ef0 0x7f4f9c06ff50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4f8400eb40 tx=0x7f4f8400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:12.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.172+0000 7f4f997fa640 1 -- 172.21.15.112:0/3141519576 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f84007c30 con 0x7f4f9c102ef0 2024-09-16T00:38:12.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.172+0000 7f4f997fa640 1 -- 172.21.15.112:0/3141519576 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f4f8400f040 con 0x7f4f9c102ef0 2024-09-16T00:38:12.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.172+0000 7f4fa2679640 1 -- 172.21.15.112:0/3141519576 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f9c070830 con 0x7f4f9c102ef0 2024-09-16T00:38:12.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.172+0000 7f4f997fa640 1 -- 172.21.15.112:0/3141519576 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f8400ca90 con 0x7f4f9c102ef0 2024-09-16T00:38:12.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.172+0000 7f4fa2679640 1 -- 172.21.15.112:0/3141519576 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f9c119c10 con 0x7f4f9c102ef0 2024-09-16T00:38:12.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.173+0000 7f4fa2679640 1 -- 172.21.15.112:0/3141519576 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f9c0fdcd0 con 0x7f4f9c102ef0 2024-09-16T00:38:12.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.174+0000 7f4f997fa640 1 -- 172.21.15.112:0/3141519576 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f4f8400cbf0 con 0x7f4f9c102ef0 2024-09-16T00:38:12.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.175+0000 7f4f997fa640 1 --2- 172.21.15.112:0/3141519576 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f4f8806e530 0x7f4f880709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:12.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.175+0000 7f4f997fa640 1 -- 172.21.15.112:0/3141519576 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f4f8408de30 con 0x7f4f9c102ef0 2024-09-16T00:38:12.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.175+0000 7f4f9bfff640 1 --2- 172.21.15.112:0/3141519576 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f4f8806e530 0x7f4f880709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:12.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.175+0000 7f4f9bfff640 1 --2- 172.21.15.112:0/3141519576 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f4f8806e530 0x7f4f880709e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c004540 tx=0x7f4f8c035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:12.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.179+0000 7f4f997fa640 1 -- 172.21.15.112:0/3141519576 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f4f84056ed0 con 0x7f4f9c102ef0 2024-09-16T00:38:12.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.358+0000 7f4fa2679640 1 -- 172.21.15.112:0/3141519576 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4f9c04b720 con 0x7f4f8806e530 2024-09-16T00:38:12.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.360+0000 7f4f997fa640 1 -- 172.21.15.112:0/3141519576 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f4f9c04b720 con 0x7f4f8806e530 2024-09-16T00:38:12.360 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:38:12.360 INFO:teuthology.orchestra.run.smithi112.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80", 2024-09-16T00:38:12.360 INFO:teuthology.orchestra.run.smithi112.stdout: "in_progress": true, 2024-09-16T00:38:12.360 INFO:teuthology.orchestra.run.smithi112.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T00:38:12.360 INFO:teuthology.orchestra.run.smithi112.stdout: "services_complete": [ 2024-09-16T00:38:12.360 INFO:teuthology.orchestra.run.smithi112.stdout: "mds", 2024-09-16T00:38:12.360 INFO:teuthology.orchestra.run.smithi112.stdout: "osd", 2024-09-16T00:38:12.360 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr", 2024-09-16T00:38:12.361 INFO:teuthology.orchestra.run.smithi112.stdout: "mon", 2024-09-16T00:38:12.361 INFO:teuthology.orchestra.run.smithi112.stdout: "crash" 2024-09-16T00:38:12.361 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-16T00:38:12.361 INFO:teuthology.orchestra.run.smithi112.stdout: "progress": "16/21 daemons upgraded", 2024-09-16T00:38:12.361 INFO:teuthology.orchestra.run.smithi112.stdout: "message": "Currently upgrading mds daemons", 2024-09-16T00:38:12.361 INFO:teuthology.orchestra.run.smithi112.stdout: "is_paused": false 2024-09-16T00:38:12.361 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:38:12.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.362+0000 7f4fa2679640 1 -- 172.21.15.112:0/3141519576 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f4f8806e530 msgr2=0x7f4f880709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:12.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.362+0000 7f4fa2679640 1 --2- 172.21.15.112:0/3141519576 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f4f8806e530 0x7f4f880709e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c004540 tx=0x7f4f8c035040 comp rx=0 tx=0).stop 2024-09-16T00:38:12.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.362+0000 7f4fa2679640 1 -- 172.21.15.112:0/3141519576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f9c102ef0 msgr2=0x7f4f9c06ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:12.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.362+0000 7f4fa2679640 1 --2- 172.21.15.112:0/3141519576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f9c102ef0 0x7f4f9c06ff50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4f8400eb40 tx=0x7f4f8400b040 comp rx=0 tx=0).stop 2024-09-16T00:38:12.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.362+0000 7f4fa2679640 1 -- 172.21.15.112:0/3141519576 shutdown_connections 2024-09-16T00:38:12.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.362+0000 7f4fa2679640 1 --2- 172.21.15.112:0/3141519576 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f4f8806e530 0x7f4f880709e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:12.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.362+0000 7f4fa2679640 1 --2- 172.21.15.112:0/3141519576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f9c102ef0 0x7f4f9c06ff50 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:12.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.362+0000 7f4fa2679640 1 --2- 172.21.15.112:0/3141519576 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f9c102550 0x7f4f9c06fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:12.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.362+0000 7f4fa2679640 1 -- 172.21.15.112:0/3141519576 >> 172.21.15.112:0/3141519576 conn(0x7f4f9c0f80f0 msgr2=0x7f4f9c0f9160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:12.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.362+0000 7f4fa2679640 1 -- 172.21.15.112:0/3141519576 shutdown_connections 2024-09-16T00:38:12.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.362+0000 7f4fa2679640 1 -- 172.21.15.112:0/3141519576 wait complete. 2024-09-16T00:38:12.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.440+0000 7fac0587a640 1 -- 172.21.15.112:0/2894500847 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac00102ef0 msgr2=0x7fac000fc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:12.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.440+0000 7fac0587a640 1 --2- 172.21.15.112:0/2894500847 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac00102ef0 0x7fac000fc5b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fabf40048d0 tx=0x7fabf4009da0 comp rx=0 tx=0).stop 2024-09-16T00:38:12.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.440+0000 7fac0587a640 1 -- 172.21.15.112:0/2894500847 shutdown_connections 2024-09-16T00:38:12.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.440+0000 7fac0587a640 1 --2- 172.21.15.112:0/2894500847 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac00102ef0 0x7fac000fc5b0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:12.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.440+0000 7fac0587a640 1 --2- 172.21.15.112:0/2894500847 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac00102550 0x7fac00102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:12.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.440+0000 7fac0587a640 1 -- 172.21.15.112:0/2894500847 >> 172.21.15.112:0/2894500847 conn(0x7fac000f80f0 msgr2=0x7fac000fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:12.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.441+0000 7fac0587a640 1 -- 172.21.15.112:0/2894500847 shutdown_connections 2024-09-16T00:38:12.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.441+0000 7fac0587a640 1 -- 172.21.15.112:0/2894500847 wait complete. 2024-09-16T00:38:12.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.441+0000 7fac0587a640 1 Processor -- start 2024-09-16T00:38:12.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.441+0000 7fac0587a640 1 -- start start 2024-09-16T00:38:12.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.441+0000 7fac0587a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac00102550 0x7fac0006fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:12.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.441+0000 7fac0587a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac00102ef0 0x7fac0006ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:12.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.441+0000 7fac0587a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac00070490 con 0x7fac00102550 2024-09-16T00:38:12.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.441+0000 7fac0587a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac00070600 con 0x7fac00102ef0 2024-09-16T00:38:12.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.442+0000 7fabfeffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac00102550 0x7fac0006fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:12.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.442+0000 7fabfeffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac00102550 0x7fac0006fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44520/0 (socket says 172.21.15.112:44520) 2024-09-16T00:38:12.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.442+0000 7fabfeffd640 1 -- 172.21.15.112:0/3328958388 learned_addr learned my addr 172.21.15.112:0/3328958388 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:38:12.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.442+0000 7fabfe7fc640 1 --2- 172.21.15.112:0/3328958388 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac00102ef0 0x7fac0006ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:12.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.442+0000 7fabfeffd640 1 -- 172.21.15.112:0/3328958388 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac00102ef0 msgr2=0x7fac0006ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:12.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.442+0000 7fabfeffd640 1 --2- 172.21.15.112:0/3328958388 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac00102ef0 0x7fac0006ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:12.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.442+0000 7fabfeffd640 1 -- 172.21.15.112:0/3328958388 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabf4005040 con 0x7fac00102550 2024-09-16T00:38:12.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.442+0000 7fabfeffd640 1 --2- 172.21.15.112:0/3328958388 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac00102550 0x7fac0006fa10 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fabf000c830 tx=0x7fabf0004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:12.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.442+0000 7fabe77fe640 1 -- 172.21.15.112:0/3328958388 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fabf000a850 con 0x7fac00102550 2024-09-16T00:38:12.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.442+0000 7fac0587a640 1 -- 172.21.15.112:0/3328958388 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac00070890 con 0x7fac00102550 2024-09-16T00:38:12.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.442+0000 7fabe77fe640 1 -- 172.21.15.112:0/3328958388 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fabf000ae50 con 0x7fac00102550 2024-09-16T00:38:12.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.442+0000 7fabe77fe640 1 -- 172.21.15.112:0/3328958388 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fabf00113a0 con 0x7fac00102550 2024-09-16T00:38:12.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.442+0000 7fac0587a640 1 -- 172.21.15.112:0/3328958388 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac00119c00 con 0x7fac00102550 2024-09-16T00:38:12.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.444+0000 7fabe77fe640 1 -- 172.21.15.112:0/3328958388 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fabf0012400 con 0x7fac00102550 2024-09-16T00:38:12.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.444+0000 7fac0587a640 1 -- 172.21.15.112:0/3328958388 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac000fdcd0 con 0x7fac00102550 2024-09-16T00:38:12.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.444+0000 7fabe77fe640 1 --2- 172.21.15.112:0/3328958388 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fabe806e530 0x7fabe80709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:12.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.444+0000 7fabe77fe640 1 -- 172.21.15.112:0/3328958388 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7fabf008e450 con 0x7fac00102550 2024-09-16T00:38:12.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.445+0000 7fabfe7fc640 1 --2- 172.21.15.112:0/3328958388 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fabe806e530 0x7fabe80709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:12.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.445+0000 7fabfe7fc640 1 --2- 172.21.15.112:0/3328958388 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fabe806e530 0x7fabe80709e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fabf4000c00 tx=0x7fabf4035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:12.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.449+0000 7fabe77fe640 1 -- 172.21.15.112:0/3328958388 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fabf00574e0 con 0x7fac00102550 2024-09-16T00:38:12.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.678+0000 7fac0587a640 1 -- 172.21.15.112:0/3328958388 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fac0004b7b0 con 0x7fac00102550 2024-09-16T00:38:12.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.678+0000 7fabe77fe640 1 -- 172.21.15.112:0/3328958388 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7fabf005aba0 con 0x7fac00102550 2024-09-16T00:38:12.678 INFO:teuthology.orchestra.run.smithi112.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-16T00:38:12.678 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-16T00:38:12.678 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs is degraded 2024-09-16T00:38:12.679 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T00:38:12.679 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T00:38:12.679 INFO:teuthology.orchestra.run.smithi112.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-16T00:38:12.679 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-16T00:38:12.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.680+0000 7fac0587a640 1 -- 172.21.15.112:0/3328958388 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fabe806e530 msgr2=0x7fabe80709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:12.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.680+0000 7fac0587a640 1 --2- 172.21.15.112:0/3328958388 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fabe806e530 0x7fabe80709e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fabf4000c00 tx=0x7fabf4035040 comp rx=0 tx=0).stop 2024-09-16T00:38:12.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.680+0000 7fac0587a640 1 -- 172.21.15.112:0/3328958388 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac00102550 msgr2=0x7fac0006fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:12.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.680+0000 7fac0587a640 1 --2- 172.21.15.112:0/3328958388 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac00102550 0x7fac0006fa10 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fabf000c830 tx=0x7fabf0004430 comp rx=0 tx=0).stop 2024-09-16T00:38:12.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.680+0000 7fac0587a640 1 -- 172.21.15.112:0/3328958388 shutdown_connections 2024-09-16T00:38:12.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.680+0000 7fac0587a640 1 --2- 172.21.15.112:0/3328958388 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fabe806e530 0x7fabe80709e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:12.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.680+0000 7fac0587a640 1 --2- 172.21.15.112:0/3328958388 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac00102ef0 0x7fac0006ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:12.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.680+0000 7fac0587a640 1 --2- 172.21.15.112:0/3328958388 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac00102550 0x7fac0006fa10 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:12.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.680+0000 7fac0587a640 1 -- 172.21.15.112:0/3328958388 >> 172.21.15.112:0/3328958388 conn(0x7fac000f80f0 msgr2=0x7fac000f9150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:12.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.680+0000 7fac0587a640 1 -- 172.21.15.112:0/3328958388 shutdown_connections 2024-09-16T00:38:12.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:12.680+0000 7fac0587a640 1 -- 172.21.15.112:0/3328958388 wait complete. 2024-09-16T00:38:12.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:38:12 smithi112 ceph-mon[97750]: from='client.34632 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:38:12.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:38:12 smithi112 ceph-mon[97750]: from='client.34636 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:38:12.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:38:12 smithi112 ceph-mon[97750]: from='client.34640 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:38:12.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:38:12 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/4110889337' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:38:12.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:38:12 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/2016826434' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:38:13.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:38:12 smithi169 ceph-mon[77626]: from='client.34632 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:38:13.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:38:12 smithi169 ceph-mon[77626]: from='client.34636 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:38:13.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:38:12 smithi169 ceph-mon[77626]: from='client.34640 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:38:13.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:38:12 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/4110889337' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:38:13.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:38:12 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/2016826434' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:38:14.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:38:13 smithi169 ceph-mon[77626]: from='client.34652 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:38:14.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:38:13 smithi169 ceph-mon[77626]: pgmap v314: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:38:14.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:38:13 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/3328958388' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:38:14.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:38:13 smithi112 ceph-mon[97750]: from='client.34652 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:38:14.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:38:13 smithi112 ceph-mon[97750]: pgmap v314: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:38:14.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:38:13 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/3328958388' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:38:16.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:38:15 smithi169 ceph-mon[77626]: pgmap v315: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:38:16.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:38:15 smithi112 ceph-mon[97750]: pgmap v315: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:38:17.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:38:16 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:38:17.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:38:16 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:38:18.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:38:17 smithi169 ceph-mon[77626]: pgmap v316: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:38:18.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:38:17 smithi112 ceph-mon[97750]: pgmap v316: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:38:20.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:38:19 smithi169 ceph-mon[77626]: pgmap v317: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:38:20.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:38:19 smithi112 ceph-mon[97750]: pgmap v317: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:38:22.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:38:21 smithi169 ceph-mon[77626]: pgmap v318: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:38:22.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:38:21 smithi112 ceph-mon[97750]: pgmap v318: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:38:24.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:38:23 smithi169 ceph-mon[77626]: pgmap v319: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:38:24.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:38:23 smithi112 ceph-mon[97750]: pgmap v319: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:38:26.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:38:25 smithi169 ceph-mon[77626]: pgmap v320: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:38:26.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:38:25 smithi112 ceph-mon[97750]: pgmap v320: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:38:28.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:38:27 smithi169 ceph-mon[77626]: pgmap v321: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:38:28.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:38:27 smithi112 ceph-mon[97750]: pgmap v321: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:38:30.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:38:29 smithi169 ceph-mon[77626]: pgmap v322: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:38:30.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:38:29 smithi112 ceph-mon[97750]: pgmap v322: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:38:32.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:38:31 smithi169 ceph-mon[77626]: pgmap v323: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:38:32.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:38:31 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:38:32.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:38:31 smithi112 ceph-mon[97750]: pgmap v323: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:38:32.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:38:31 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:38:34.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:38:33 smithi169 ceph-mon[77626]: pgmap v324: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:38:34.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:38:33 smithi112 ceph-mon[97750]: pgmap v324: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:38:36.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:38:35 smithi169 ceph-mon[77626]: pgmap v325: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:38:36.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:38:35 smithi112 ceph-mon[97750]: pgmap v325: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:38:38.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:38:37 smithi169 ceph-mon[77626]: pgmap v326: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:38:38.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:38:37 smithi112 ceph-mon[97750]: pgmap v326: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:38:40.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:38:39 smithi169 ceph-mon[77626]: pgmap v327: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:38:40.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:38:39 smithi112 ceph-mon[97750]: pgmap v327: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:38:42.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:38:41 smithi169 ceph-mon[77626]: pgmap v328: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:38:42.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:38:41 smithi112 ceph-mon[97750]: pgmap v328: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:38:42.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:42.773+0000 7f70103a5640 1 -- 172.21.15.112:0/2829136784 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7008102550 msgr2=0x7f7008102920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:42.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:42.773+0000 7f70103a5640 1 --2- 172.21.15.112:0/2829136784 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7008102550 0x7f7008102920 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6ff80048d0 tx=0x7f6ff8009da0 comp rx=0 tx=0).stop 2024-09-16T00:38:42.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:42.773+0000 7f70103a5640 1 -- 172.21.15.112:0/2829136784 shutdown_connections 2024-09-16T00:38:42.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:42.773+0000 7f70103a5640 1 --2- 172.21.15.112:0/2829136784 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7008102ef0 0x7f70080fc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:42.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:42.773+0000 7f70103a5640 1 --2- 172.21.15.112:0/2829136784 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7008102550 0x7f7008102920 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:42.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:42.773+0000 7f70103a5640 1 -- 172.21.15.112:0/2829136784 >> 172.21.15.112:0/2829136784 conn(0x7f70080f80f0 msgr2=0x7f70080fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:42.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:42.773+0000 7f70103a5640 1 -- 172.21.15.112:0/2829136784 shutdown_connections 2024-09-16T00:38:42.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:42.773+0000 7f70103a5640 1 -- 172.21.15.112:0/2829136784 wait complete. 2024-09-16T00:38:42.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:42.773+0000 7f70103a5640 1 Processor -- start 2024-09-16T00:38:42.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:42.773+0000 7f70103a5640 1 -- start start 2024-09-16T00:38:42.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:42.774+0000 7f70103a5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7008102550 0x7f700806fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:42.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:42.774+0000 7f70103a5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7008102ef0 0x7f700806ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:42.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:42.774+0000 7f70103a5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7008070490 con 0x7f7008102550 2024-09-16T00:38:42.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:42.774+0000 7f70103a5640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7008070600 con 0x7f7008102ef0 2024-09-16T00:38:42.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:42.774+0000 7f700e141640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7008102550 0x7f700806fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:42.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:42.774+0000 7f700e141640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7008102550 0x7f700806fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52888/0 (socket says 172.21.15.112:52888) 2024-09-16T00:38:42.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:42.774+0000 7f700e141640 1 -- 172.21.15.112:0/4173544824 learned_addr learned my addr 172.21.15.112:0/4173544824 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:38:42.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:42.774+0000 7f700d940640 1 --2- 172.21.15.112:0/4173544824 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7008102ef0 0x7f700806ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:42.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:42.774+0000 7f700e141640 1 -- 172.21.15.112:0/4173544824 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7008102ef0 msgr2=0x7f700806ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:42.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:42.774+0000 7f700e141640 1 --2- 172.21.15.112:0/4173544824 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7008102ef0 0x7f700806ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:42.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:42.774+0000 7f700e141640 1 -- 172.21.15.112:0/4173544824 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ff8005040 con 0x7f7008102550 2024-09-16T00:38:42.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:42.774+0000 7f700d940640 1 --2- 172.21.15.112:0/4173544824 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7008102ef0 0x7f700806ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:38:42.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:42.774+0000 7f700e141640 1 --2- 172.21.15.112:0/4173544824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7008102550 0x7f700806fa10 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6ff80043d0 tx=0x7f6ff8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:42.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:42.775+0000 7f7006ffd640 1 -- 172.21.15.112:0/4173544824 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ff8038070 con 0x7f7008102550 2024-09-16T00:38:42.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:42.775+0000 7f70103a5640 1 -- 172.21.15.112:0/4173544824 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7008070830 con 0x7f7008102550 2024-09-16T00:38:42.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:42.775+0000 7f7006ffd640 1 -- 172.21.15.112:0/4173544824 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f6ff80096e0 con 0x7f7008102550 2024-09-16T00:38:42.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:42.775+0000 7f7006ffd640 1 -- 172.21.15.112:0/4173544824 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ff803d420 con 0x7f7008102550 2024-09-16T00:38:42.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:42.775+0000 7f70103a5640 1 -- 172.21.15.112:0/4173544824 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7008119c10 con 0x7f7008102550 2024-09-16T00:38:42.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:42.776+0000 7f70103a5640 1 -- 172.21.15.112:0/4173544824 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70080fdcd0 con 0x7f7008102550 2024-09-16T00:38:42.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:42.776+0000 7f7006ffd640 1 -- 172.21.15.112:0/4173544824 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f6ff8030020 con 0x7f7008102550 2024-09-16T00:38:42.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:42.777+0000 7f7006ffd640 1 --2- 172.21.15.112:0/4173544824 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f6ff006e530 0x7f6ff00709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:42.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:42.777+0000 7f700d940640 1 --2- 172.21.15.112:0/4173544824 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f6ff006e530 0x7f6ff00709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:42.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:42.777+0000 7f7006ffd640 1 -- 172.21.15.112:0/4173544824 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f6ff80ae8b0 con 0x7f7008102550 2024-09-16T00:38:42.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:42.777+0000 7f700d940640 1 --2- 172.21.15.112:0/4173544824 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f6ff006e530 0x7f6ff00709e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7008071350 tx=0x7f6ffc00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:42.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:42.781+0000 7f7006ffd640 1 -- 172.21.15.112:0/4173544824 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f6ff8077820 con 0x7f7008102550 2024-09-16T00:38:42.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:42.965+0000 7f70103a5640 1 -- 172.21.15.112:0/4173544824 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7008071100 con 0x7f6ff006e530 2024-09-16T00:38:42.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:42.966+0000 7f7006ffd640 1 -- 172.21.15.112:0/4173544824 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f7008071100 con 0x7f6ff006e530 2024-09-16T00:38:42.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:42.968+0000 7f70103a5640 1 -- 172.21.15.112:0/4173544824 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f6ff006e530 msgr2=0x7f6ff00709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:42.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:42.968+0000 7f70103a5640 1 --2- 172.21.15.112:0/4173544824 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f6ff006e530 0x7f6ff00709e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7008071350 tx=0x7f6ffc00b040 comp rx=0 tx=0).stop 2024-09-16T00:38:42.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:42.968+0000 7f70103a5640 1 -- 172.21.15.112:0/4173544824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7008102550 msgr2=0x7f700806fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:42.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:42.968+0000 7f70103a5640 1 --2- 172.21.15.112:0/4173544824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7008102550 0x7f700806fa10 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6ff80043d0 tx=0x7f6ff8009da0 comp rx=0 tx=0).stop 2024-09-16T00:38:42.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:42.968+0000 7f70103a5640 1 -- 172.21.15.112:0/4173544824 shutdown_connections 2024-09-16T00:38:42.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:42.968+0000 7f70103a5640 1 --2- 172.21.15.112:0/4173544824 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f6ff006e530 0x7f6ff00709e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:42.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:42.968+0000 7f70103a5640 1 --2- 172.21.15.112:0/4173544824 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7008102ef0 0x7f700806ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:42.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:42.968+0000 7f70103a5640 1 --2- 172.21.15.112:0/4173544824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7008102550 0x7f700806fa10 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:42.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:42.968+0000 7f70103a5640 1 -- 172.21.15.112:0/4173544824 >> 172.21.15.112:0/4173544824 conn(0x7f70080f80f0 msgr2=0x7f70080f9160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:42.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:42.968+0000 7f70103a5640 1 -- 172.21.15.112:0/4173544824 shutdown_connections 2024-09-16T00:38:42.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:42.968+0000 7f70103a5640 1 -- 172.21.15.112:0/4173544824 wait complete. 2024-09-16T00:38:42.977 INFO:teuthology.orchestra.run.smithi112.stdout:true 2024-09-16T00:38:43.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.047+0000 7ffac9488640 1 -- 172.21.15.112:0/3640683280 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffac40fcf80 msgr2=0x7ffac40fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:43.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.047+0000 7ffac9488640 1 --2- 172.21.15.112:0/3640683280 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffac40fcf80 0x7ffac40fd3d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ffaac009c50 tx=0x7ffaac004d10 comp rx=0 tx=0).stop 2024-09-16T00:38:43.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.047+0000 7ffac9488640 1 -- 172.21.15.112:0/3640683280 shutdown_connections 2024-09-16T00:38:43.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.047+0000 7ffac9488640 1 --2- 172.21.15.112:0/3640683280 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffac40fcf80 0x7ffac40fd3d0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:43.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.047+0000 7ffac9488640 1 --2- 172.21.15.112:0/3640683280 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffac40fb810 0x7ffac40fbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:43.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.047+0000 7ffac9488640 1 -- 172.21.15.112:0/3640683280 >> 172.21.15.112:0/3640683280 conn(0x7ffac40723b0 msgr2=0x7ffac40727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:43.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.047+0000 7ffac9488640 1 -- 172.21.15.112:0/3640683280 shutdown_connections 2024-09-16T00:38:43.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.047+0000 7ffac9488640 1 -- 172.21.15.112:0/3640683280 wait complete. 2024-09-16T00:38:43.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.048+0000 7ffac9488640 1 Processor -- start 2024-09-16T00:38:43.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.048+0000 7ffac9488640 1 -- start start 2024-09-16T00:38:43.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.048+0000 7ffac9488640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffac40fb810 0x7ffac4113210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:43.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.048+0000 7ffac9488640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffac40fcf80 0x7ffac4113750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:43.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.048+0000 7ffac9488640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac4113de0 con 0x7ffac40fb810 2024-09-16T00:38:43.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.048+0000 7ffac9488640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac410d190 con 0x7ffac40fcf80 2024-09-16T00:38:43.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.048+0000 7ffac2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffac40fb810 0x7ffac4113210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:43.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.049+0000 7ffac2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffac40fb810 0x7ffac4113210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52918/0 (socket says 172.21.15.112:52918) 2024-09-16T00:38:43.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.049+0000 7ffac2ffd640 1 -- 172.21.15.112:0/1133129212 learned_addr learned my addr 172.21.15.112:0/1133129212 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:38:43.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.049+0000 7ffaba7fc640 1 --2- 172.21.15.112:0/1133129212 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffac40fcf80 0x7ffac4113750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:43.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.049+0000 7ffac2ffd640 1 -- 172.21.15.112:0/1133129212 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffac40fcf80 msgr2=0x7ffac4113750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:43.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.049+0000 7ffac2ffd640 1 --2- 172.21.15.112:0/1133129212 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffac40fcf80 0x7ffac4113750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:43.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.049+0000 7ffac2ffd640 1 -- 172.21.15.112:0/1133129212 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaac0098b0 con 0x7ffac40fb810 2024-09-16T00:38:43.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.049+0000 7ffac2ffd640 1 --2- 172.21.15.112:0/1133129212 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffac40fb810 0x7ffac4113210 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ffab400b790 tx=0x7ffab40091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:43.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.049+0000 7ffabbfff640 1 -- 172.21.15.112:0/1133129212 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffab400c700 con 0x7ffac40fb810 2024-09-16T00:38:43.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.049+0000 7ffac9488640 1 -- 172.21.15.112:0/1133129212 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffac410d470 con 0x7ffac40fb810 2024-09-16T00:38:43.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.049+0000 7ffabbfff640 1 -- 172.21.15.112:0/1133129212 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7ffab400cd00 con 0x7ffac40fb810 2024-09-16T00:38:43.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.049+0000 7ffabbfff640 1 -- 172.21.15.112:0/1133129212 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffab4015e70 con 0x7ffac40fb810 2024-09-16T00:38:43.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.049+0000 7ffac9488640 1 -- 172.21.15.112:0/1133129212 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffac410d960 con 0x7ffac40fb810 2024-09-16T00:38:43.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.050+0000 7ffac9488640 1 -- 172.21.15.112:0/1133129212 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffac40fcbc0 con 0x7ffac40fb810 2024-09-16T00:38:43.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.051+0000 7ffabbfff640 1 -- 172.21.15.112:0/1133129212 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7ffab4012070 con 0x7ffac40fb810 2024-09-16T00:38:43.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.051+0000 7ffabbfff640 1 --2- 172.21.15.112:0/1133129212 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ffab006e530 0x7ffab00709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:43.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.051+0000 7ffabbfff640 1 -- 172.21.15.112:0/1133129212 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7ffab408dd60 con 0x7ffac40fb810 2024-09-16T00:38:43.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.051+0000 7ffaba7fc640 1 --2- 172.21.15.112:0/1133129212 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ffab006e530 0x7ffab00709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:43.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.052+0000 7ffaba7fc640 1 --2- 172.21.15.112:0/1133129212 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ffab006e530 0x7ffab00709e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ffaac009c20 tx=0x7ffaac004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:43.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.056+0000 7ffabbfff640 1 -- 172.21.15.112:0/1133129212 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7ffab4056d50 con 0x7ffac40fb810 2024-09-16T00:38:43.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.236+0000 7ffac9488640 1 -- 172.21.15.112:0/1133129212 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ffac404b7b0 con 0x7ffab006e530 2024-09-16T00:38:43.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.237+0000 7ffabbfff640 1 -- 172.21.15.112:0/1133129212 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7ffac404b7b0 con 0x7ffab006e530 2024-09-16T00:38:43.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.239+0000 7ffac9488640 1 -- 172.21.15.112:0/1133129212 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ffab006e530 msgr2=0x7ffab00709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:43.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.239+0000 7ffac9488640 1 --2- 172.21.15.112:0/1133129212 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ffab006e530 0x7ffab00709e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ffaac009c20 tx=0x7ffaac004d10 comp rx=0 tx=0).stop 2024-09-16T00:38:43.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.239+0000 7ffac9488640 1 -- 172.21.15.112:0/1133129212 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffac40fb810 msgr2=0x7ffac4113210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:43.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.239+0000 7ffac9488640 1 --2- 172.21.15.112:0/1133129212 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffac40fb810 0x7ffac4113210 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ffab400b790 tx=0x7ffab40091d0 comp rx=0 tx=0).stop 2024-09-16T00:38:43.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.239+0000 7ffac9488640 1 -- 172.21.15.112:0/1133129212 shutdown_connections 2024-09-16T00:38:43.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.239+0000 7ffac9488640 1 --2- 172.21.15.112:0/1133129212 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ffab006e530 0x7ffab00709e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:43.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.239+0000 7ffac9488640 1 --2- 172.21.15.112:0/1133129212 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffac40fcf80 0x7ffac4113750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:43.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.239+0000 7ffac9488640 1 --2- 172.21.15.112:0/1133129212 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffac40fb810 0x7ffac4113210 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:43.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.239+0000 7ffac9488640 1 -- 172.21.15.112:0/1133129212 >> 172.21.15.112:0/1133129212 conn(0x7ffac40723b0 msgr2=0x7ffac40fa860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:43.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.240+0000 7ffac9488640 1 -- 172.21.15.112:0/1133129212 shutdown_connections 2024-09-16T00:38:43.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.240+0000 7ffac9488640 1 -- 172.21.15.112:0/1133129212 wait complete. 2024-09-16T00:38:43.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.319+0000 7ff929ac3640 1 -- 172.21.15.112:0/4208411121 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff924102530 msgr2=0x7ff924102900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:43.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.319+0000 7ff929ac3640 1 --2- 172.21.15.112:0/4208411121 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff924102530 0x7ff924102900 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff9140048d0 tx=0x7ff914009da0 comp rx=0 tx=0).stop 2024-09-16T00:38:43.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.319+0000 7ff929ac3640 1 -- 172.21.15.112:0/4208411121 shutdown_connections 2024-09-16T00:38:43.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.319+0000 7ff929ac3640 1 --2- 172.21.15.112:0/4208411121 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff924102ed0 0x7ff9240fc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:43.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.319+0000 7ff929ac3640 1 --2- 172.21.15.112:0/4208411121 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff924102530 0x7ff924102900 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:43.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.319+0000 7ff929ac3640 1 -- 172.21.15.112:0/4208411121 >> 172.21.15.112:0/4208411121 conn(0x7ff9240f80f0 msgr2=0x7ff9240fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:43.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.319+0000 7ff929ac3640 1 -- 172.21.15.112:0/4208411121 shutdown_connections 2024-09-16T00:38:43.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.319+0000 7ff929ac3640 1 -- 172.21.15.112:0/4208411121 wait complete. 2024-09-16T00:38:43.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.319+0000 7ff929ac3640 1 Processor -- start 2024-09-16T00:38:43.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.319+0000 7ff929ac3640 1 -- start start 2024-09-16T00:38:43.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.320+0000 7ff929ac3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff924102530 0x7ff924113240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:43.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.320+0000 7ff929ac3640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff924102ed0 0x7ff924113780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:43.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.320+0000 7ff929ac3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff924113d80 con 0x7ff924102530 2024-09-16T00:38:43.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.320+0000 7ff929ac3640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92410d1c0 con 0x7ff924102ed0 2024-09-16T00:38:43.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.320+0000 7ff9237fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff924102530 0x7ff924113240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:43.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.320+0000 7ff9237fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff924102530 0x7ff924113240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52934/0 (socket says 172.21.15.112:52934) 2024-09-16T00:38:43.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.320+0000 7ff9237fe640 1 -- 172.21.15.112:0/1500556904 learned_addr learned my addr 172.21.15.112:0/1500556904 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:38:43.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.320+0000 7ff91bfff640 1 --2- 172.21.15.112:0/1500556904 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff924102ed0 0x7ff924113780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:43.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.320+0000 7ff9237fe640 1 -- 172.21.15.112:0/1500556904 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff924102ed0 msgr2=0x7ff924113780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:43.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.320+0000 7ff9237fe640 1 --2- 172.21.15.112:0/1500556904 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff924102ed0 0x7ff924113780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:43.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.320+0000 7ff9237fe640 1 -- 172.21.15.112:0/1500556904 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff914005040 con 0x7ff924102530 2024-09-16T00:38:43.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.320+0000 7ff9237fe640 1 --2- 172.21.15.112:0/1500556904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff924102530 0x7ff924113240 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff9140043d0 tx=0x7ff914009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:43.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.321+0000 7ff920ff9640 1 -- 172.21.15.112:0/1500556904 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff914038070 con 0x7ff924102530 2024-09-16T00:38:43.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.321+0000 7ff920ff9640 1 -- 172.21.15.112:0/1500556904 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7ff9140096e0 con 0x7ff924102530 2024-09-16T00:38:43.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.321+0000 7ff929ac3640 1 -- 172.21.15.112:0/1500556904 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff92410d440 con 0x7ff924102530 2024-09-16T00:38:43.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.321+0000 7ff929ac3640 1 -- 172.21.15.112:0/1500556904 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff92410d9e0 con 0x7ff924102530 2024-09-16T00:38:43.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.321+0000 7ff920ff9640 1 -- 172.21.15.112:0/1500556904 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff91403d420 con 0x7ff924102530 2024-09-16T00:38:43.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.322+0000 7ff929ac3640 1 -- 172.21.15.112:0/1500556904 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9240fdcb0 con 0x7ff924102530 2024-09-16T00:38:43.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.322+0000 7ff920ff9640 1 -- 172.21.15.112:0/1500556904 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7ff914030020 con 0x7ff924102530 2024-09-16T00:38:43.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.323+0000 7ff920ff9640 1 --2- 172.21.15.112:0/1500556904 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff91006e530 0x7ff9100709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:43.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.323+0000 7ff920ff9640 1 -- 172.21.15.112:0/1500556904 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7ff9140ae7c0 con 0x7ff924102530 2024-09-16T00:38:43.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.323+0000 7ff91bfff640 1 --2- 172.21.15.112:0/1500556904 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff91006e530 0x7ff9100709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:43.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.323+0000 7ff91bfff640 1 --2- 172.21.15.112:0/1500556904 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff91006e530 0x7ff9100709e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff92410eb00 tx=0x7ff90c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:43.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.327+0000 7ff920ff9640 1 -- 172.21.15.112:0/1500556904 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7ff914077730 con 0x7ff924102530 2024-09-16T00:38:43.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.501+0000 7ff929ac3640 1 -- 172.21.15.112:0/1500556904 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff92404b720 con 0x7ff91006e530 2024-09-16T00:38:43.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.510+0000 7ff920ff9640 1 -- 172.21.15.112:0/1500556904 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7ff92404b720 con 0x7ff91006e530 2024-09-16T00:38:43.510 INFO:teuthology.orchestra.run.smithi112.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T00:38:43.510 INFO:teuthology.orchestra.run.smithi112.stdout:alertmanager.smithi112 smithi112 *:9093,9094 running (11m) 4m ago 20m 24.7M - 0.25.0 c8568f914cd2 d85ea0562022 2024-09-16T00:38:43.510 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi112 smithi112 running (8m) 4m ago 20m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c e4fe58f9702a 2024-09-16T00:38:43.510 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi169 smithi169 running (8m) 4m ago 19m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c 3ba2c81d527f 2024-09-16T00:38:43.511 INFO:teuthology.orchestra.run.smithi112.stdout:grafana.smithi112 smithi112 *:3000 running (10m) 4m ago 20m 82.1M - 9.4.12 2bacad6d85d8 51f9d2013da2 2024-09-16T00:38:43.511 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.hxfwkh smithi112 running (4m) 4m ago 15m 14.2M - 19.1.1-326-g8921c8ef f1455bd0219c 1723069902d6 2024-09-16T00:38:43.511 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.ithwzj smithi112 running (5m) 4m ago 15m 12.2M - 19.1.1-326-g8921c8ef f1455bd0219c e7aad5b81db5 2024-09-16T00:38:43.511 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.gawwgw smithi169 running (4m) 4m ago 15m 15.7M - 19.1.1-326-g8921c8ef f1455bd0219c 469be459ee12 2024-09-16T00:38:43.511 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.lgeqrj smithi169 running (4m) 4m ago 15m 13.3M - 19.1.1-326-g8921c8ef f1455bd0219c 31f0fbd2395d 2024-09-16T00:38:43.511 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi112.xrkkbc smithi112 *:8443,9283,8765 running (11m) 4m ago 21m 504M - 19.1.1-326-g8921c8ef f1455bd0219c a981f7fd2810 2024-09-16T00:38:43.511 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi169.mtrobk smithi169 *:8443,9283,8765 running (10m) 4m ago 19m 442M - 19.1.1-326-g8921c8ef f1455bd0219c 4096e5369c2d 2024-09-16T00:38:43.511 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi112 smithi112 running (9m) 4m ago 21m 49.8M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 5aaec033c042 2024-09-16T00:38:43.511 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi169 smithi169 running (9m) 4m ago 19m 38.9M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 7bc609155f3f 2024-09-16T00:38:43.511 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi112 smithi112 *:9100 running (12m) 4m ago 20m 15.5M - 1.5.0 0da6a335fe13 6404d1d601e2 2024-09-16T00:38:43.511 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi169 smithi169 *:9100 running (11m) 4m ago 19m 15.4M - 1.5.0 0da6a335fe13 5505baf702a6 2024-09-16T00:38:43.511 INFO:teuthology.orchestra.run.smithi112.stdout:osd.0 smithi112 running (8m) 4m ago 18m 105M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 53efc9397017 2024-09-16T00:38:43.512 INFO:teuthology.orchestra.run.smithi112.stdout:osd.1 smithi112 running (7m) 4m ago 18m 98.9M 1557M 19.1.1-326-g8921c8ef f1455bd0219c fe6f1a4a2cf4 2024-09-16T00:38:43.512 INFO:teuthology.orchestra.run.smithi112.stdout:osd.2 smithi112 running (7m) 4m ago 18m 102M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 5f7863712057 2024-09-16T00:38:43.512 INFO:teuthology.orchestra.run.smithi112.stdout:osd.3 smithi169 running (6m) 4m ago 17m 92.1M 2581M 19.1.1-326-g8921c8ef f1455bd0219c ee07b5577810 2024-09-16T00:38:43.512 INFO:teuthology.orchestra.run.smithi112.stdout:osd.4 smithi169 running (5m) 4m ago 17m 103M 2581M 19.1.1-326-g8921c8ef f1455bd0219c 769f026a54eb 2024-09-16T00:38:43.512 INFO:teuthology.orchestra.run.smithi112.stdout:osd.5 smithi169 running (5m) 4m ago 16m 125M 2581M 19.1.1-326-g8921c8ef f1455bd0219c a6007c13cf78 2024-09-16T00:38:43.512 INFO:teuthology.orchestra.run.smithi112.stdout:prometheus.smithi112 smithi112 *:9095 running (10m) 4m ago 20m 64.1M - 2.43.0 a07b618ecd1d 7a1d42e60d2b 2024-09-16T00:38:43.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.512+0000 7ff929ac3640 1 -- 172.21.15.112:0/1500556904 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff91006e530 msgr2=0x7ff9100709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:43.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.512+0000 7ff929ac3640 1 --2- 172.21.15.112:0/1500556904 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff91006e530 0x7ff9100709e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff92410eb00 tx=0x7ff90c00b040 comp rx=0 tx=0).stop 2024-09-16T00:38:43.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.512+0000 7ff929ac3640 1 -- 172.21.15.112:0/1500556904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff924102530 msgr2=0x7ff924113240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:43.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.512+0000 7ff929ac3640 1 --2- 172.21.15.112:0/1500556904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff924102530 0x7ff924113240 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff9140043d0 tx=0x7ff914009da0 comp rx=0 tx=0).stop 2024-09-16T00:38:43.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.512+0000 7ff929ac3640 1 -- 172.21.15.112:0/1500556904 shutdown_connections 2024-09-16T00:38:43.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.512+0000 7ff929ac3640 1 --2- 172.21.15.112:0/1500556904 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff91006e530 0x7ff9100709e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:43.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.512+0000 7ff929ac3640 1 --2- 172.21.15.112:0/1500556904 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff924102ed0 0x7ff924113780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:43.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.512+0000 7ff929ac3640 1 --2- 172.21.15.112:0/1500556904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff924102530 0x7ff924113240 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:43.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.512+0000 7ff929ac3640 1 -- 172.21.15.112:0/1500556904 >> 172.21.15.112:0/1500556904 conn(0x7ff9240f80f0 msgr2=0x7ff9240f90f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:43.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.512+0000 7ff929ac3640 1 -- 172.21.15.112:0/1500556904 shutdown_connections 2024-09-16T00:38:43.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.512+0000 7ff929ac3640 1 -- 172.21.15.112:0/1500556904 wait complete. 2024-09-16T00:38:43.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.590+0000 7f9922521640 1 -- 172.21.15.112:0/1696444349 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f991c0fb810 msgr2=0x7f991c0fbbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:43.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.590+0000 7f9922521640 1 --2- 172.21.15.112:0/1696444349 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f991c0fb810 0x7f991c0fbbe0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f990c0048d0 tx=0x7f990c009da0 comp rx=0 tx=0).stop 2024-09-16T00:38:43.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.590+0000 7f9922521640 1 -- 172.21.15.112:0/1696444349 shutdown_connections 2024-09-16T00:38:43.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.590+0000 7f9922521640 1 --2- 172.21.15.112:0/1696444349 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f991c0fcf80 0x7f991c0fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:43.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.590+0000 7f9922521640 1 --2- 172.21.15.112:0/1696444349 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f991c0fb810 0x7f991c0fbbe0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:43.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.590+0000 7f9922521640 1 -- 172.21.15.112:0/1696444349 >> 172.21.15.112:0/1696444349 conn(0x7f991c0723b0 msgr2=0x7f991c0727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:43.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.590+0000 7f9922521640 1 -- 172.21.15.112:0/1696444349 shutdown_connections 2024-09-16T00:38:43.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.590+0000 7f9922521640 1 -- 172.21.15.112:0/1696444349 wait complete. 2024-09-16T00:38:43.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.590+0000 7f9922521640 1 Processor -- start 2024-09-16T00:38:43.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.591+0000 7f9922521640 1 -- start start 2024-09-16T00:38:43.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.591+0000 7f9922521640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f991c0fb810 0x7f991c10e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:43.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.591+0000 7f9922521640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f991c0fcf80 0x7f991c10e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:43.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.591+0000 7f9922521640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991c10ed40 con 0x7f991c0fb810 2024-09-16T00:38:43.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.591+0000 7f9922521640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991c10eeb0 con 0x7f991c0fcf80 2024-09-16T00:38:43.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.591+0000 7f991bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f991c0fb810 0x7f991c10e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:43.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.591+0000 7f991bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f991c0fb810 0x7f991c10e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52962/0 (socket says 172.21.15.112:52962) 2024-09-16T00:38:43.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.591+0000 7f991bfff640 1 -- 172.21.15.112:0/2912063364 learned_addr learned my addr 172.21.15.112:0/2912063364 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:38:43.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.591+0000 7f991bfff640 1 -- 172.21.15.112:0/2912063364 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f991c0fcf80 msgr2=0x7f991c10e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:43.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.591+0000 7f991bfff640 1 --2- 172.21.15.112:0/2912063364 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f991c0fcf80 0x7f991c10e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:43.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.591+0000 7f991bfff640 1 -- 172.21.15.112:0/2912063364 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f990c005040 con 0x7f991c0fb810 2024-09-16T00:38:43.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.591+0000 7f991bfff640 1 --2- 172.21.15.112:0/2912063364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f991c0fb810 0x7f991c10e1b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f990c000c00 tx=0x7f990c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:43.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.592+0000 7f9918ff9640 1 -- 172.21.15.112:0/2912063364 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f990c038070 con 0x7f991c0fb810 2024-09-16T00:38:43.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.592+0000 7f9918ff9640 1 -- 172.21.15.112:0/2912063364 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f990c02ade0 con 0x7f991c0fb810 2024-09-16T00:38:43.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.592+0000 7f9922521640 1 -- 172.21.15.112:0/2912063364 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f991c117220 con 0x7f991c0fb810 2024-09-16T00:38:43.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.592+0000 7f9918ff9640 1 -- 172.21.15.112:0/2912063364 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f990c02c7b0 con 0x7f991c0fb810 2024-09-16T00:38:43.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.592+0000 7f9922521640 1 -- 172.21.15.112:0/2912063364 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f991c117680 con 0x7f991c0fb810 2024-09-16T00:38:43.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.593+0000 7f9922521640 1 -- 172.21.15.112:0/2912063364 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f991c0fcbc0 con 0x7f991c0fb810 2024-09-16T00:38:43.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.594+0000 7f9918ff9640 1 -- 172.21.15.112:0/2912063364 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f990c030020 con 0x7f991c0fb810 2024-09-16T00:38:43.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.594+0000 7f9918ff9640 1 --2- 172.21.15.112:0/2912063364 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f990806e530 0x7f99080709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:43.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.594+0000 7f9918ff9640 1 -- 172.21.15.112:0/2912063364 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f990c0ae9c0 con 0x7f991c0fb810 2024-09-16T00:38:43.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.594+0000 7f991b7fe640 1 --2- 172.21.15.112:0/2912063364 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f990806e530 0x7f99080709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:43.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.595+0000 7f991b7fe640 1 --2- 172.21.15.112:0/2912063364 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f990806e530 0x7f99080709e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f991c065980 tx=0x7f990400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:43.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.598+0000 7f9918ff9640 1 -- 172.21.15.112:0/2912063364 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f990c0779b0 con 0x7f991c0fb810 2024-09-16T00:38:43.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.832+0000 7f9922521640 1 -- 172.21.15.112:0/2912063364 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f991c0657c0 con 0x7f991c0fb810 2024-09-16T00:38:43.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.832+0000 7f9918ff9640 1 -- 172.21.15.112:0/2912063364 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f990c07b070 con 0x7f991c0fb810 2024-09-16T00:38:43.833 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:38:43.833 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": { 2024-09-16T00:38:43.833 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:38:43.833 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:38:43.833 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": { 2024-09-16T00:38:43.833 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:38:43.833 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:38:43.833 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": { 2024-09-16T00:38:43.833 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 6 2024-09-16T00:38:43.834 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:38:43.834 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": { 2024-09-16T00:38:43.834 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 4 2024-09-16T00:38:43.834 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:38:43.834 INFO:teuthology.orchestra.run.smithi112.stdout: "overall": { 2024-09-16T00:38:43.834 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 14 2024-09-16T00:38:43.834 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-16T00:38:43.834 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:38:43.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.835+0000 7f9922521640 1 -- 172.21.15.112:0/2912063364 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f990806e530 msgr2=0x7f99080709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:43.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.835+0000 7f9922521640 1 --2- 172.21.15.112:0/2912063364 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f990806e530 0x7f99080709e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f991c065980 tx=0x7f990400b040 comp rx=0 tx=0).stop 2024-09-16T00:38:43.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.835+0000 7f9922521640 1 -- 172.21.15.112:0/2912063364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f991c0fb810 msgr2=0x7f991c10e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:43.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.835+0000 7f9922521640 1 --2- 172.21.15.112:0/2912063364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f991c0fb810 0x7f991c10e1b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f990c000c00 tx=0x7f990c009da0 comp rx=0 tx=0).stop 2024-09-16T00:38:43.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.835+0000 7f9922521640 1 -- 172.21.15.112:0/2912063364 shutdown_connections 2024-09-16T00:38:43.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.835+0000 7f9922521640 1 --2- 172.21.15.112:0/2912063364 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f990806e530 0x7f99080709e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:43.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.835+0000 7f9922521640 1 --2- 172.21.15.112:0/2912063364 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f991c0fcf80 0x7f991c10e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:43.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.835+0000 7f9922521640 1 --2- 172.21.15.112:0/2912063364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f991c0fb810 0x7f991c10e1b0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:43.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.835+0000 7f9922521640 1 -- 172.21.15.112:0/2912063364 >> 172.21.15.112:0/2912063364 conn(0x7f991c0723b0 msgr2=0x7f991c0fa480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:43.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.835+0000 7f9922521640 1 -- 172.21.15.112:0/2912063364 shutdown_connections 2024-09-16T00:38:43.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.835+0000 7f9922521640 1 -- 172.21.15.112:0/2912063364 wait complete. 2024-09-16T00:38:43.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.913+0000 7f41c1be5640 1 -- 172.21.15.112:0/1164641467 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41bc0fb810 msgr2=0x7f41bc0fbbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:43.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.913+0000 7f41c1be5640 1 --2- 172.21.15.112:0/1164641467 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41bc0fb810 0x7f41bc0fbbe0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f41ac0048d0 tx=0x7f41ac009da0 comp rx=0 tx=0).stop 2024-09-16T00:38:43.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.913+0000 7f41c1be5640 1 -- 172.21.15.112:0/1164641467 shutdown_connections 2024-09-16T00:38:43.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.913+0000 7f41c1be5640 1 --2- 172.21.15.112:0/1164641467 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f41bc0fcf80 0x7f41bc0fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:43.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.913+0000 7f41c1be5640 1 --2- 172.21.15.112:0/1164641467 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41bc0fb810 0x7f41bc0fbbe0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:43.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.913+0000 7f41c1be5640 1 -- 172.21.15.112:0/1164641467 >> 172.21.15.112:0/1164641467 conn(0x7f41bc0723b0 msgr2=0x7f41bc0727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:43.914 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:38:43 smithi112 ceph-mon[97750]: pgmap v329: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:38:43.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.913+0000 7f41c1be5640 1 -- 172.21.15.112:0/1164641467 shutdown_connections 2024-09-16T00:38:43.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.913+0000 7f41c1be5640 1 -- 172.21.15.112:0/1164641467 wait complete. 2024-09-16T00:38:43.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.914+0000 7f41c1be5640 1 Processor -- start 2024-09-16T00:38:43.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.914+0000 7f41c1be5640 1 -- start start 2024-09-16T00:38:43.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.914+0000 7f41c1be5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41bc0fb810 0x7f41bc10e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:43.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.914+0000 7f41c1be5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f41bc0fcf80 0x7f41bc10e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:43.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.914+0000 7f41c1be5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41bc10ede0 con 0x7f41bc0fb810 2024-09-16T00:38:43.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.914+0000 7f41c1be5640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41bc10ef50 con 0x7f41bc0fcf80 2024-09-16T00:38:43.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.914+0000 7f41bb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41bc0fb810 0x7f41bc10e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:43.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.914+0000 7f41bb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41bc0fb810 0x7f41bc10e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52980/0 (socket says 172.21.15.112:52980) 2024-09-16T00:38:43.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.914+0000 7f41bb7fe640 1 -- 172.21.15.112:0/2868515572 learned_addr learned my addr 172.21.15.112:0/2868515572 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:38:43.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.914+0000 7f41b3fff640 1 --2- 172.21.15.112:0/2868515572 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f41bc0fcf80 0x7f41bc10e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:43.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.915+0000 7f41bb7fe640 1 -- 172.21.15.112:0/2868515572 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f41bc0fcf80 msgr2=0x7f41bc10e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:43.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.915+0000 7f41bb7fe640 1 --2- 172.21.15.112:0/2868515572 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f41bc0fcf80 0x7f41bc10e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:43.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.915+0000 7f41bb7fe640 1 -- 172.21.15.112:0/2868515572 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41ac005040 con 0x7f41bc0fb810 2024-09-16T00:38:43.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.915+0000 7f41bb7fe640 1 --2- 172.21.15.112:0/2868515572 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41bc0fb810 0x7f41bc10e1c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f41ac003080 tx=0x7f41ac009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:43.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.915+0000 7f41b8ff9640 1 -- 172.21.15.112:0/2868515572 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41ac038070 con 0x7f41bc0fb810 2024-09-16T00:38:43.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.915+0000 7f41c1be5640 1 -- 172.21.15.112:0/2868515572 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41bc117230 con 0x7f41bc0fb810 2024-09-16T00:38:43.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.915+0000 7f41b8ff9640 1 -- 172.21.15.112:0/2868515572 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f41ac0097e0 con 0x7f41bc0fb810 2024-09-16T00:38:43.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.915+0000 7f41b8ff9640 1 -- 172.21.15.112:0/2868515572 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41ac03d420 con 0x7f41bc0fb810 2024-09-16T00:38:43.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.915+0000 7f41c1be5640 1 -- 172.21.15.112:0/2868515572 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41bc1176a0 con 0x7f41bc0fb810 2024-09-16T00:38:43.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.916+0000 7f41c1be5640 1 -- 172.21.15.112:0/2868515572 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41bc0fcbc0 con 0x7f41bc0fb810 2024-09-16T00:38:43.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.917+0000 7f41b8ff9640 1 -- 172.21.15.112:0/2868515572 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f41ac030020 con 0x7f41bc0fb810 2024-09-16T00:38:43.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.917+0000 7f41b8ff9640 1 --2- 172.21.15.112:0/2868515572 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f41a806e530 0x7f41a80709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:43.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.917+0000 7f41b8ff9640 1 -- 172.21.15.112:0/2868515572 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f41ac0ae6c0 con 0x7f41bc0fb810 2024-09-16T00:38:43.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.917+0000 7f41b3fff640 1 --2- 172.21.15.112:0/2868515572 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f41a806e530 0x7f41a80709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:43.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.918+0000 7f41b3fff640 1 --2- 172.21.15.112:0/2868515572 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f41a806e530 0x7f41a80709e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f41bc065980 tx=0x7f41a4009120 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:43.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:43.921+0000 7f41b8ff9640 1 -- 172.21.15.112:0/2868515572 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f41ac077630 con 0x7f41bc0fb810 2024-09-16T00:38:44.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:38:43 smithi169 ceph-mon[77626]: pgmap v329: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:38:44.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.121+0000 7f41c1be5640 1 -- 172.21.15.112:0/2868515572 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f41bc04b750 con 0x7f41bc0fb810 2024-09-16T00:38:44.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.121+0000 7f41b8ff9640 1 -- 172.21.15.112:0/2868515572 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1914 (secure 0 0 0) 0x7f41ac07acf0 con 0x7f41bc0fb810 2024-09-16T00:38:44.122 INFO:teuthology.orchestra.run.smithi112.stdout:e34 2024-09-16T00:38:44.122 INFO:teuthology.orchestra.run.smithi112.stdout:btime 2024-09-16T00:34:26:619254+0000 2024-09-16T00:38:44.122 INFO:teuthology.orchestra.run.smithi112.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T00:38:44.122 INFO:teuthology.orchestra.run.smithi112.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:38:44.122 INFO:teuthology.orchestra.run.smithi112.stdout:legacy client fscid: 1 2024-09-16T00:38:44.122 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:38:44.122 INFO:teuthology.orchestra.run.smithi112.stdout:Filesystem 'cephfs' (1) 2024-09-16T00:38:44.122 INFO:teuthology.orchestra.run.smithi112.stdout:fs_name cephfs 2024-09-16T00:38:44.123 INFO:teuthology.orchestra.run.smithi112.stdout:epoch 25 2024-09-16T00:38:44.123 INFO:teuthology.orchestra.run.smithi112.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-16T00:38:44.123 INFO:teuthology.orchestra.run.smithi112.stdout:created 2024-09-16T00:23:22.328199+0000 2024-09-16T00:38:44.123 INFO:teuthology.orchestra.run.smithi112.stdout:modified 2024-09-16T00:33:29.012991+0000 2024-09-16T00:38:44.123 INFO:teuthology.orchestra.run.smithi112.stdout:tableserver 0 2024-09-16T00:38:44.123 INFO:teuthology.orchestra.run.smithi112.stdout:root 0 2024-09-16T00:38:44.123 INFO:teuthology.orchestra.run.smithi112.stdout:session_timeout 60 2024-09-16T00:38:44.123 INFO:teuthology.orchestra.run.smithi112.stdout:session_autoclose 300 2024-09-16T00:38:44.123 INFO:teuthology.orchestra.run.smithi112.stdout:max_file_size 1099511627776 2024-09-16T00:38:44.123 INFO:teuthology.orchestra.run.smithi112.stdout:max_xattr_size 65536 2024-09-16T00:38:44.123 INFO:teuthology.orchestra.run.smithi112.stdout:required_client_features {} 2024-09-16T00:38:44.123 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure 0 2024-09-16T00:38:44.123 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure_osd_epoch 82 2024-09-16T00:38:44.123 INFO:teuthology.orchestra.run.smithi112.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:38:44.124 INFO:teuthology.orchestra.run.smithi112.stdout:max_mds 2 2024-09-16T00:38:44.124 INFO:teuthology.orchestra.run.smithi112.stdout:in 0,1 2024-09-16T00:38:44.124 INFO:teuthology.orchestra.run.smithi112.stdout:up {} 2024-09-16T00:38:44.124 INFO:teuthology.orchestra.run.smithi112.stdout:failed 0,1 2024-09-16T00:38:44.124 INFO:teuthology.orchestra.run.smithi112.stdout:damaged 2024-09-16T00:38:44.124 INFO:teuthology.orchestra.run.smithi112.stdout:stopped 2024-09-16T00:38:44.124 INFO:teuthology.orchestra.run.smithi112.stdout:data_pools [3] 2024-09-16T00:38:44.124 INFO:teuthology.orchestra.run.smithi112.stdout:metadata_pool 2 2024-09-16T00:38:44.124 INFO:teuthology.orchestra.run.smithi112.stdout:inline_data enabled 2024-09-16T00:38:44.124 INFO:teuthology.orchestra.run.smithi112.stdout:balancer 2024-09-16T00:38:44.124 INFO:teuthology.orchestra.run.smithi112.stdout:bal_rank_mask -1 2024-09-16T00:38:44.124 INFO:teuthology.orchestra.run.smithi112.stdout:standby_count_wanted 1 2024-09-16T00:38:44.125 INFO:teuthology.orchestra.run.smithi112.stdout:qdb_cluster leader: 0 members: 2024-09-16T00:38:44.125 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:38:44.125 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:38:44.125 INFO:teuthology.orchestra.run.smithi112.stdout:Standby daemons: 2024-09-16T00:38:44.125 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:38:44.125 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.ithwzj{-1:34392} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.112:6826/476229221,v1:172.21.15.112:6827/476229221] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:38:44.125 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.hxfwkh{-1:34404} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.112:6828/2356899876,v1:172.21.15.112:6829/2356899876] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:38:44.125 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.gawwgw{-1:44343} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.169:6824/1635329957,v1:172.21.15.169:6825/1635329957] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:38:44.125 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.lgeqrj{-1:44347} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.169:6826/184810031,v1:172.21.15.169:6827/184810031] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:38:44.126 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 34 2024-09-16T00:38:44.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.124+0000 7f41c1be5640 1 -- 172.21.15.112:0/2868515572 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f41a806e530 msgr2=0x7f41a80709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:44.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.124+0000 7f41c1be5640 1 --2- 172.21.15.112:0/2868515572 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f41a806e530 0x7f41a80709e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f41bc065980 tx=0x7f41a4009120 comp rx=0 tx=0).stop 2024-09-16T00:38:44.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.124+0000 7f41c1be5640 1 -- 172.21.15.112:0/2868515572 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41bc0fb810 msgr2=0x7f41bc10e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:44.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.124+0000 7f41c1be5640 1 --2- 172.21.15.112:0/2868515572 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41bc0fb810 0x7f41bc10e1c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f41ac003080 tx=0x7f41ac009da0 comp rx=0 tx=0).stop 2024-09-16T00:38:44.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.124+0000 7f41c1be5640 1 -- 172.21.15.112:0/2868515572 shutdown_connections 2024-09-16T00:38:44.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.124+0000 7f41c1be5640 1 --2- 172.21.15.112:0/2868515572 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f41a806e530 0x7f41a80709e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:44.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.124+0000 7f41c1be5640 1 --2- 172.21.15.112:0/2868515572 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f41bc0fcf80 0x7f41bc10e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:44.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.124+0000 7f41c1be5640 1 --2- 172.21.15.112:0/2868515572 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41bc0fb810 0x7f41bc10e1c0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:44.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.124+0000 7f41c1be5640 1 -- 172.21.15.112:0/2868515572 >> 172.21.15.112:0/2868515572 conn(0x7f41bc0723b0 msgr2=0x7f41bc0fa480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:44.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.125+0000 7f41c1be5640 1 -- 172.21.15.112:0/2868515572 shutdown_connections 2024-09-16T00:38:44.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.125+0000 7f41c1be5640 1 -- 172.21.15.112:0/2868515572 wait complete. 2024-09-16T00:38:44.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.204+0000 7f55067c4640 1 -- 172.21.15.112:0/4264763569 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5500102ef0 msgr2=0x7f55000fc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:44.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.204+0000 7f55067c4640 1 --2- 172.21.15.112:0/4264763569 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5500102ef0 0x7f55000fc5b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f54f40048d0 tx=0x7f54f4009da0 comp rx=0 tx=0).stop 2024-09-16T00:38:44.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.204+0000 7f55067c4640 1 -- 172.21.15.112:0/4264763569 shutdown_connections 2024-09-16T00:38:44.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.204+0000 7f55067c4640 1 --2- 172.21.15.112:0/4264763569 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5500102ef0 0x7f55000fc5b0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:44.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.204+0000 7f55067c4640 1 --2- 172.21.15.112:0/4264763569 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5500102550 0x7f5500102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:44.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.204+0000 7f55067c4640 1 -- 172.21.15.112:0/4264763569 >> 172.21.15.112:0/4264763569 conn(0x7f55000f80f0 msgr2=0x7f55000fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:44.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.204+0000 7f55067c4640 1 -- 172.21.15.112:0/4264763569 shutdown_connections 2024-09-16T00:38:44.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.204+0000 7f55067c4640 1 -- 172.21.15.112:0/4264763569 wait complete. 2024-09-16T00:38:44.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.205+0000 7f55067c4640 1 Processor -- start 2024-09-16T00:38:44.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.205+0000 7f55067c4640 1 -- start start 2024-09-16T00:38:44.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.205+0000 7f55067c4640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5500102550 0x7f5500113520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:44.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.205+0000 7f55067c4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5500102ef0 0x7f5500113a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:44.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.205+0000 7f55067c4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55001140b0 con 0x7f5500102ef0 2024-09-16T00:38:44.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.205+0000 7f55067c4640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f550010d4c0 con 0x7f5500102550 2024-09-16T00:38:44.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.205+0000 7f54ff7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5500102ef0 0x7f5500113a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:44.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.205+0000 7f54ff7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5500102ef0 0x7f5500113a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52998/0 (socket says 172.21.15.112:52998) 2024-09-16T00:38:44.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.205+0000 7f54ff7fe640 1 -- 172.21.15.112:0/1181544789 learned_addr learned my addr 172.21.15.112:0/1181544789 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:38:44.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.205+0000 7f54fffff640 1 --2- 172.21.15.112:0/1181544789 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5500102550 0x7f5500113520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:44.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.206+0000 7f54ff7fe640 1 -- 172.21.15.112:0/1181544789 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5500102550 msgr2=0x7f5500113520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:44.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.206+0000 7f54ff7fe640 1 --2- 172.21.15.112:0/1181544789 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5500102550 0x7f5500113520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:44.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.206+0000 7f54ff7fe640 1 -- 172.21.15.112:0/1181544789 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54f4005040 con 0x7f5500102ef0 2024-09-16T00:38:44.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.206+0000 7f54ff7fe640 1 --2- 172.21.15.112:0/1181544789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5500102ef0 0x7f5500113a60 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f54f40043d0 tx=0x7f54f4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:44.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.206+0000 7f54fcff9640 1 -- 172.21.15.112:0/1181544789 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f54f4038070 con 0x7f5500102ef0 2024-09-16T00:38:44.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.206+0000 7f54fcff9640 1 -- 172.21.15.112:0/1181544789 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f54f40096e0 con 0x7f5500102ef0 2024-09-16T00:38:44.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.206+0000 7f54fcff9640 1 -- 172.21.15.112:0/1181544789 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f54f402c8a0 con 0x7f5500102ef0 2024-09-16T00:38:44.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.206+0000 7f55067c4640 1 -- 172.21.15.112:0/1181544789 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f550010d740 con 0x7f5500102ef0 2024-09-16T00:38:44.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.206+0000 7f55067c4640 1 -- 172.21.15.112:0/1181544789 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f550010dc80 con 0x7f5500102ef0 2024-09-16T00:38:44.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.207+0000 7f55067c4640 1 -- 172.21.15.112:0/1181544789 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55000fdcd0 con 0x7f5500102ef0 2024-09-16T00:38:44.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.208+0000 7f54fcff9640 1 -- 172.21.15.112:0/1181544789 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f54f4030020 con 0x7f5500102ef0 2024-09-16T00:38:44.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.208+0000 7f54fcff9640 1 --2- 172.21.15.112:0/1181544789 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f54e806e530 0x7f54e80709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:44.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.209+0000 7f54fcff9640 1 -- 172.21.15.112:0/1181544789 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f54f40ae070 con 0x7f5500102ef0 2024-09-16T00:38:44.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.209+0000 7f54fffff640 1 --2- 172.21.15.112:0/1181544789 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f54e806e530 0x7f54e80709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:44.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.209+0000 7f54fffff640 1 --2- 172.21.15.112:0/1181544789 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f54e806e530 0x7f54e80709e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f54f0004ed0 tx=0x7f54f000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:44.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.213+0000 7f54fcff9640 1 -- 172.21.15.112:0/1181544789 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f54f4077070 con 0x7f5500102ef0 2024-09-16T00:38:44.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.393+0000 7f55067c4640 1 -- 172.21.15.112:0/1181544789 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5500109580 con 0x7f54e806e530 2024-09-16T00:38:44.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.394+0000 7f54fcff9640 1 -- 172.21.15.112:0/1181544789 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f5500109580 con 0x7f54e806e530 2024-09-16T00:38:44.395 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:38:44.395 INFO:teuthology.orchestra.run.smithi112.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80", 2024-09-16T00:38:44.395 INFO:teuthology.orchestra.run.smithi112.stdout: "in_progress": true, 2024-09-16T00:38:44.395 INFO:teuthology.orchestra.run.smithi112.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T00:38:44.395 INFO:teuthology.orchestra.run.smithi112.stdout: "services_complete": [ 2024-09-16T00:38:44.395 INFO:teuthology.orchestra.run.smithi112.stdout: "mds", 2024-09-16T00:38:44.395 INFO:teuthology.orchestra.run.smithi112.stdout: "osd", 2024-09-16T00:38:44.395 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr", 2024-09-16T00:38:44.395 INFO:teuthology.orchestra.run.smithi112.stdout: "mon", 2024-09-16T00:38:44.395 INFO:teuthology.orchestra.run.smithi112.stdout: "crash" 2024-09-16T00:38:44.395 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-16T00:38:44.395 INFO:teuthology.orchestra.run.smithi112.stdout: "progress": "16/21 daemons upgraded", 2024-09-16T00:38:44.395 INFO:teuthology.orchestra.run.smithi112.stdout: "message": "Currently upgrading mds daemons", 2024-09-16T00:38:44.396 INFO:teuthology.orchestra.run.smithi112.stdout: "is_paused": false 2024-09-16T00:38:44.396 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:38:44.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.397+0000 7f55067c4640 1 -- 172.21.15.112:0/1181544789 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f54e806e530 msgr2=0x7f54e80709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:44.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.397+0000 7f55067c4640 1 --2- 172.21.15.112:0/1181544789 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f54e806e530 0x7f54e80709e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f54f0004ed0 tx=0x7f54f000b040 comp rx=0 tx=0).stop 2024-09-16T00:38:44.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.397+0000 7f55067c4640 1 -- 172.21.15.112:0/1181544789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5500102ef0 msgr2=0x7f5500113a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:44.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.397+0000 7f55067c4640 1 --2- 172.21.15.112:0/1181544789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5500102ef0 0x7f5500113a60 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f54f40043d0 tx=0x7f54f4009da0 comp rx=0 tx=0).stop 2024-09-16T00:38:44.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.397+0000 7f55067c4640 1 -- 172.21.15.112:0/1181544789 shutdown_connections 2024-09-16T00:38:44.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.397+0000 7f55067c4640 1 --2- 172.21.15.112:0/1181544789 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f54e806e530 0x7f54e80709e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:44.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.397+0000 7f55067c4640 1 --2- 172.21.15.112:0/1181544789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5500102ef0 0x7f5500113a60 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:44.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.397+0000 7f55067c4640 1 --2- 172.21.15.112:0/1181544789 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5500102550 0x7f5500113520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:44.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.397+0000 7f55067c4640 1 -- 172.21.15.112:0/1181544789 >> 172.21.15.112:0/1181544789 conn(0x7f55000f80f0 msgr2=0x7f55000f9150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:44.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.397+0000 7f55067c4640 1 -- 172.21.15.112:0/1181544789 shutdown_connections 2024-09-16T00:38:44.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.397+0000 7f55067c4640 1 -- 172.21.15.112:0/1181544789 wait complete. 2024-09-16T00:38:44.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.474+0000 7fe840e55640 1 -- 172.21.15.112:0/3391011440 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe83c102ef0 msgr2=0x7fe83c0fc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:44.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.474+0000 7fe840e55640 1 --2- 172.21.15.112:0/3391011440 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe83c102ef0 0x7fe83c0fc5b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe8300048d0 tx=0x7fe830009da0 comp rx=0 tx=0).stop 2024-09-16T00:38:44.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.475+0000 7fe840e55640 1 -- 172.21.15.112:0/3391011440 shutdown_connections 2024-09-16T00:38:44.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.475+0000 7fe840e55640 1 --2- 172.21.15.112:0/3391011440 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe83c102ef0 0x7fe83c0fc5b0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:44.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.475+0000 7fe840e55640 1 --2- 172.21.15.112:0/3391011440 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe83c102550 0x7fe83c102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:44.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.475+0000 7fe840e55640 1 -- 172.21.15.112:0/3391011440 >> 172.21.15.112:0/3391011440 conn(0x7fe83c0f80f0 msgr2=0x7fe83c0fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:44.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.475+0000 7fe840e55640 1 -- 172.21.15.112:0/3391011440 shutdown_connections 2024-09-16T00:38:44.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.475+0000 7fe840e55640 1 -- 172.21.15.112:0/3391011440 wait complete. 2024-09-16T00:38:44.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.475+0000 7fe840e55640 1 Processor -- start 2024-09-16T00:38:44.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.475+0000 7fe840e55640 1 -- start start 2024-09-16T00:38:44.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.476+0000 7fe840e55640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe83c102550 0x7fe83c113520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:44.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.476+0000 7fe840e55640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe83c102ef0 0x7fe83c113a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:44.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.476+0000 7fe840e55640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe83c1140b0 con 0x7fe83c102ef0 2024-09-16T00:38:44.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.476+0000 7fe840e55640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe83c10d4c0 con 0x7fe83c102550 2024-09-16T00:38:44.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.476+0000 7fe839d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe83c102ef0 0x7fe83c113a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:44.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.476+0000 7fe839d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe83c102ef0 0x7fe83c113a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53026/0 (socket says 172.21.15.112:53026) 2024-09-16T00:38:44.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.476+0000 7fe839d9b640 1 -- 172.21.15.112:0/332026003 learned_addr learned my addr 172.21.15.112:0/332026003 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:38:44.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.476+0000 7fe83a59c640 1 --2- 172.21.15.112:0/332026003 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe83c102550 0x7fe83c113520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:44.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.476+0000 7fe839d9b640 1 -- 172.21.15.112:0/332026003 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe83c102550 msgr2=0x7fe83c113520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:44.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.476+0000 7fe839d9b640 1 --2- 172.21.15.112:0/332026003 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe83c102550 0x7fe83c113520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:44.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.476+0000 7fe839d9b640 1 -- 172.21.15.112:0/332026003 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe830005040 con 0x7fe83c102ef0 2024-09-16T00:38:44.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.476+0000 7fe839d9b640 1 --2- 172.21.15.112:0/332026003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe83c102ef0 0x7fe83c113a60 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe830003080 tx=0x7fe830009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:44.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.477+0000 7fe82affd640 1 -- 172.21.15.112:0/332026003 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe830038070 con 0x7fe83c102ef0 2024-09-16T00:38:44.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.477+0000 7fe82affd640 1 -- 172.21.15.112:0/332026003 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fe830009760 con 0x7fe83c102ef0 2024-09-16T00:38:44.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.477+0000 7fe840e55640 1 -- 172.21.15.112:0/332026003 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe83c10d740 con 0x7fe83c102ef0 2024-09-16T00:38:44.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.477+0000 7fe840e55640 1 -- 172.21.15.112:0/332026003 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe83c10dc20 con 0x7fe83c102ef0 2024-09-16T00:38:44.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.477+0000 7fe82affd640 1 -- 172.21.15.112:0/332026003 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe830033420 con 0x7fe83c102ef0 2024-09-16T00:38:44.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.478+0000 7fe840e55640 1 -- 172.21.15.112:0/332026003 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe83c0fdcd0 con 0x7fe83c102ef0 2024-09-16T00:38:44.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.478+0000 7fe82affd640 1 -- 172.21.15.112:0/332026003 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fe830030020 con 0x7fe83c102ef0 2024-09-16T00:38:44.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.479+0000 7fe82affd640 1 --2- 172.21.15.112:0/332026003 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fe82406e530 0x7fe8240709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:38:44.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.479+0000 7fe82affd640 1 -- 172.21.15.112:0/332026003 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7fe8300adee0 con 0x7fe83c102ef0 2024-09-16T00:38:44.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.479+0000 7fe83a59c640 1 --2- 172.21.15.112:0/332026003 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fe82406e530 0x7fe8240709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:38:44.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.479+0000 7fe83a59c640 1 --2- 172.21.15.112:0/332026003 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fe82406e530 0x7fe8240709e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe82c0041d0 tx=0x7fe82c004330 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:38:44.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.483+0000 7fe82affd640 1 -- 172.21.15.112:0/332026003 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fe830076ed0 con 0x7fe83c102ef0 2024-09-16T00:38:44.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.714+0000 7fe840e55640 1 -- 172.21.15.112:0/332026003 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fe83c0fbd50 con 0x7fe83c102ef0 2024-09-16T00:38:44.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.715+0000 7fe82affd640 1 -- 172.21.15.112:0/332026003 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7fe830081020 con 0x7fe83c102ef0 2024-09-16T00:38:44.715 INFO:teuthology.orchestra.run.smithi112.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-16T00:38:44.715 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-16T00:38:44.715 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs is degraded 2024-09-16T00:38:44.716 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T00:38:44.716 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T00:38:44.716 INFO:teuthology.orchestra.run.smithi112.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-16T00:38:44.716 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-16T00:38:44.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.717+0000 7fe840e55640 1 -- 172.21.15.112:0/332026003 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fe82406e530 msgr2=0x7fe8240709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:44.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.717+0000 7fe840e55640 1 --2- 172.21.15.112:0/332026003 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fe82406e530 0x7fe8240709e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe82c0041d0 tx=0x7fe82c004330 comp rx=0 tx=0).stop 2024-09-16T00:38:44.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.717+0000 7fe840e55640 1 -- 172.21.15.112:0/332026003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe83c102ef0 msgr2=0x7fe83c113a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:38:44.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.717+0000 7fe840e55640 1 --2- 172.21.15.112:0/332026003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe83c102ef0 0x7fe83c113a60 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe830003080 tx=0x7fe830009da0 comp rx=0 tx=0).stop 2024-09-16T00:38:44.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.717+0000 7fe840e55640 1 -- 172.21.15.112:0/332026003 shutdown_connections 2024-09-16T00:38:44.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.717+0000 7fe840e55640 1 --2- 172.21.15.112:0/332026003 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fe82406e530 0x7fe8240709e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:44.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.717+0000 7fe840e55640 1 --2- 172.21.15.112:0/332026003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe83c102ef0 0x7fe83c113a60 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:44.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.717+0000 7fe840e55640 1 --2- 172.21.15.112:0/332026003 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe83c102550 0x7fe83c113520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:38:44.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.717+0000 7fe840e55640 1 -- 172.21.15.112:0/332026003 >> 172.21.15.112:0/332026003 conn(0x7fe83c0f80f0 msgr2=0x7fe83c0f9150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:38:44.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.717+0000 7fe840e55640 1 -- 172.21.15.112:0/332026003 shutdown_connections 2024-09-16T00:38:44.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:38:44.717+0000 7fe840e55640 1 -- 172.21.15.112:0/332026003 wait complete. 2024-09-16T00:38:45.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:38:44 smithi169 ceph-mon[77626]: from='client.34660 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:38:45.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:38:44 smithi169 ceph-mon[77626]: from='client.34664 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:38:45.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:38:44 smithi169 ceph-mon[77626]: from='client.34668 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:38:45.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:38:44 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/2912063364' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:38:45.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:38:44 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/2868515572' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:38:45.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:38:44 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/332026003' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:38:45.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:38:44 smithi112 ceph-mon[97750]: from='client.34660 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:38:45.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:38:44 smithi112 ceph-mon[97750]: from='client.34664 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:38:45.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:38:44 smithi112 ceph-mon[97750]: from='client.34668 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:38:45.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:38:44 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/2912063364' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:38:45.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:38:44 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/2868515572' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:38:45.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:38:44 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/332026003' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:38:46.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:38:45 smithi169 ceph-mon[77626]: from='client.34680 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:38:46.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:38:45 smithi169 ceph-mon[77626]: pgmap v330: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:38:46.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:38:45 smithi112 ceph-mon[97750]: from='client.34680 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:38:46.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:38:45 smithi112 ceph-mon[97750]: pgmap v330: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:38:47.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:38:46 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:38:47.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:38:46 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:38:48.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:38:47 smithi169 ceph-mon[77626]: pgmap v331: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:38:48.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:38:47 smithi112 ceph-mon[97750]: pgmap v331: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:38:50.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:38:49 smithi169 ceph-mon[77626]: pgmap v332: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:38:50.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:38:49 smithi112 ceph-mon[97750]: pgmap v332: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:38:52.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:38:51 smithi169 ceph-mon[77626]: pgmap v333: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:38:52.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:38:51 smithi112 ceph-mon[97750]: pgmap v333: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:38:54.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:38:53 smithi169 ceph-mon[77626]: pgmap v334: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:38:54.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:38:53 smithi112 ceph-mon[97750]: pgmap v334: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:38:56.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:38:55 smithi169 ceph-mon[77626]: pgmap v335: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:38:56.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:38:55 smithi112 ceph-mon[97750]: pgmap v335: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:38:58.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:38:57 smithi169 ceph-mon[77626]: pgmap v336: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:38:58.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:38:57 smithi112 ceph-mon[97750]: pgmap v336: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:39:00.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:38:59 smithi169 ceph-mon[77626]: pgmap v337: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:39:00.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:38:59 smithi112 ceph-mon[97750]: pgmap v337: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:39:02.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:39:01 smithi169 ceph-mon[77626]: pgmap v338: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:39:02.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:39:01 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:39:02.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:39:01 smithi112 ceph-mon[97750]: pgmap v338: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:39:02.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:39:01 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:39:04.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:39:03 smithi169 ceph-mon[77626]: pgmap v339: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:39:04.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:39:03 smithi112 ceph-mon[97750]: pgmap v339: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:39:06.028 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:39:05 smithi169 ceph-mon[77626]: pgmap v340: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:39:06.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:39:05 smithi112 ceph-mon[97750]: pgmap v340: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:39:08.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:39:07 smithi112 ceph-mon[97750]: pgmap v341: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:39:08.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:39:07 smithi169 ceph-mon[77626]: pgmap v341: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:39:10.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:39:09 smithi112 ceph-mon[97750]: pgmap v342: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:39:10.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:39:09 smithi169 ceph-mon[77626]: pgmap v342: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:39:12.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:39:11 smithi112 ceph-mon[97750]: pgmap v343: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:39:12.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:39:11 smithi169 ceph-mon[77626]: pgmap v343: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:39:14.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:39:13 smithi112 ceph-mon[97750]: pgmap v344: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:39:14.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:39:13 smithi169 ceph-mon[77626]: pgmap v344: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:39:14.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:14.807+0000 7f5f316cd640 1 -- 172.21.15.112:0/2772764190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f2c0fb810 msgr2=0x7f5f2c0fbbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:14.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:14.807+0000 7f5f316cd640 1 --2- 172.21.15.112:0/2772764190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f2c0fb810 0x7f5f2c0fbbe0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c0048d0 tx=0x7f5f1c009da0 comp rx=0 tx=0).stop 2024-09-16T00:39:14.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:14.807+0000 7f5f316cd640 1 -- 172.21.15.112:0/2772764190 shutdown_connections 2024-09-16T00:39:14.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:14.807+0000 7f5f316cd640 1 --2- 172.21.15.112:0/2772764190 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f2c0fcf80 0x7f5f2c0fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:14.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:14.807+0000 7f5f316cd640 1 --2- 172.21.15.112:0/2772764190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f2c0fb810 0x7f5f2c0fbbe0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:14.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:14.807+0000 7f5f316cd640 1 -- 172.21.15.112:0/2772764190 >> 172.21.15.112:0/2772764190 conn(0x7f5f2c0723b0 msgr2=0x7f5f2c0727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:14.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:14.807+0000 7f5f316cd640 1 -- 172.21.15.112:0/2772764190 shutdown_connections 2024-09-16T00:39:14.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:14.807+0000 7f5f316cd640 1 -- 172.21.15.112:0/2772764190 wait complete. 2024-09-16T00:39:14.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:14.807+0000 7f5f316cd640 1 Processor -- start 2024-09-16T00:39:14.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:14.807+0000 7f5f316cd640 1 -- start start 2024-09-16T00:39:14.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:14.808+0000 7f5f316cd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f2c0fb810 0x7f5f2c1120f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:14.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:14.808+0000 7f5f316cd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f2c0fcf80 0x7f5f2c112630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:14.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:14.808+0000 7f5f316cd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c112c30 con 0x7f5f2c0fcf80 2024-09-16T00:39:14.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:14.808+0000 7f5f316cd640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c10c070 con 0x7f5f2c0fb810 2024-09-16T00:39:14.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:14.808+0000 7f5f2a7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f2c0fcf80 0x7f5f2c112630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:14.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:14.808+0000 7f5f2a7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f2c0fcf80 0x7f5f2c112630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44990/0 (socket says 172.21.15.112:44990) 2024-09-16T00:39:14.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:14.808+0000 7f5f2a7fc640 1 -- 172.21.15.112:0/4008413689 learned_addr learned my addr 172.21.15.112:0/4008413689 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:39:14.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:14.808+0000 7f5f2affd640 1 --2- 172.21.15.112:0/4008413689 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f2c0fb810 0x7f5f2c1120f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:14.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:14.808+0000 7f5f2a7fc640 1 -- 172.21.15.112:0/4008413689 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f2c0fb810 msgr2=0x7f5f2c1120f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:14.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:14.808+0000 7f5f2a7fc640 1 --2- 172.21.15.112:0/4008413689 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f2c0fb810 0x7f5f2c1120f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:14.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:14.808+0000 7f5f2a7fc640 1 -- 172.21.15.112:0/4008413689 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f1c005040 con 0x7f5f2c0fcf80 2024-09-16T00:39:14.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:14.808+0000 7f5f2a7fc640 1 --2- 172.21.15.112:0/4008413689 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f2c0fcf80 0x7f5f2c112630 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5f2000fce0 tx=0x7f5f20004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:14.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:14.809+0000 7f5f137fe640 1 -- 172.21.15.112:0/4008413689 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f2000de30 con 0x7f5f2c0fcf80 2024-09-16T00:39:14.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:14.809+0000 7f5f316cd640 1 -- 172.21.15.112:0/4008413689 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f2c10c2f0 con 0x7f5f2c0fcf80 2024-09-16T00:39:14.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:14.809+0000 7f5f137fe640 1 -- 172.21.15.112:0/4008413689 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f5f200090d0 con 0x7f5f2c0fcf80 2024-09-16T00:39:14.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:14.809+0000 7f5f137fe640 1 -- 172.21.15.112:0/4008413689 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f200126a0 con 0x7f5f2c0fcf80 2024-09-16T00:39:14.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:14.809+0000 7f5f316cd640 1 -- 172.21.15.112:0/4008413689 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f2c10c890 con 0x7f5f2c0fcf80 2024-09-16T00:39:14.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:14.810+0000 7f5f316cd640 1 -- 172.21.15.112:0/4008413689 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f2c0fcbc0 con 0x7f5f2c0fcf80 2024-09-16T00:39:14.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:14.810+0000 7f5f137fe640 1 -- 172.21.15.112:0/4008413689 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f5f20009240 con 0x7f5f2c0fcf80 2024-09-16T00:39:14.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:14.811+0000 7f5f137fe640 1 --2- 172.21.15.112:0/4008413689 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5f1406e530 0x7f5f140709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:14.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:14.811+0000 7f5f137fe640 1 -- 172.21.15.112:0/4008413689 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f5f20090820 con 0x7f5f2c0fcf80 2024-09-16T00:39:14.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:14.811+0000 7f5f2affd640 1 --2- 172.21.15.112:0/4008413689 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5f1406e530 0x7f5f140709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:14.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:14.811+0000 7f5f2affd640 1 --2- 172.21.15.112:0/4008413689 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5f1406e530 0x7f5f140709e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c009ac0 tx=0x7f5f1c035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:14.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:14.816+0000 7f5f137fe640 1 -- 172.21.15.112:0/4008413689 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f5f20059840 con 0x7f5f2c0fcf80 2024-09-16T00:39:15.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.001+0000 7f5f316cd640 1 -- 172.21.15.112:0/4008413689 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5f2c10d7f0 con 0x7f5f1406e530 2024-09-16T00:39:15.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.003+0000 7f5f137fe640 1 -- 172.21.15.112:0/4008413689 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f5f1c039040 con 0x7f5f1406e530 2024-09-16T00:39:15.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.005+0000 7f5f316cd640 1 -- 172.21.15.112:0/4008413689 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5f1406e530 msgr2=0x7f5f140709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:15.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.005+0000 7f5f316cd640 1 --2- 172.21.15.112:0/4008413689 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5f1406e530 0x7f5f140709e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c009ac0 tx=0x7f5f1c035040 comp rx=0 tx=0).stop 2024-09-16T00:39:15.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.005+0000 7f5f316cd640 1 -- 172.21.15.112:0/4008413689 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f2c0fcf80 msgr2=0x7f5f2c112630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:15.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.005+0000 7f5f316cd640 1 --2- 172.21.15.112:0/4008413689 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f2c0fcf80 0x7f5f2c112630 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5f2000fce0 tx=0x7f5f20004600 comp rx=0 tx=0).stop 2024-09-16T00:39:15.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.005+0000 7f5f316cd640 1 -- 172.21.15.112:0/4008413689 shutdown_connections 2024-09-16T00:39:15.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.005+0000 7f5f316cd640 1 --2- 172.21.15.112:0/4008413689 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5f1406e530 0x7f5f140709e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:15.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.005+0000 7f5f316cd640 1 --2- 172.21.15.112:0/4008413689 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f2c0fcf80 0x7f5f2c112630 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:15.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.005+0000 7f5f316cd640 1 --2- 172.21.15.112:0/4008413689 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f2c0fb810 0x7f5f2c1120f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:15.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.005+0000 7f5f316cd640 1 -- 172.21.15.112:0/4008413689 >> 172.21.15.112:0/4008413689 conn(0x7f5f2c0723b0 msgr2=0x7f5f2c0fa480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:15.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.005+0000 7f5f316cd640 1 -- 172.21.15.112:0/4008413689 shutdown_connections 2024-09-16T00:39:15.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.005+0000 7f5f316cd640 1 -- 172.21.15.112:0/4008413689 wait complete. 2024-09-16T00:39:15.015 INFO:teuthology.orchestra.run.smithi112.stdout:true 2024-09-16T00:39:15.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.085+0000 7ff83a9f6640 1 -- 172.21.15.112:0/2759503096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff834102ed0 msgr2=0x7ff8340fc590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:15.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.085+0000 7ff83a9f6640 1 --2- 172.21.15.112:0/2759503096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff834102ed0 0x7ff8340fc590 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff81c0048d0 tx=0x7ff81c009da0 comp rx=0 tx=0).stop 2024-09-16T00:39:15.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.085+0000 7ff83a9f6640 1 -- 172.21.15.112:0/2759503096 shutdown_connections 2024-09-16T00:39:15.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.085+0000 7ff83a9f6640 1 --2- 172.21.15.112:0/2759503096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff834102ed0 0x7ff8340fc590 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:15.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.085+0000 7ff83a9f6640 1 --2- 172.21.15.112:0/2759503096 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff834102530 0x7ff834102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:15.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.085+0000 7ff83a9f6640 1 -- 172.21.15.112:0/2759503096 >> 172.21.15.112:0/2759503096 conn(0x7ff8340f8110 msgr2=0x7ff8340fa520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:15.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.085+0000 7ff83a9f6640 1 -- 172.21.15.112:0/2759503096 shutdown_connections 2024-09-16T00:39:15.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.085+0000 7ff83a9f6640 1 -- 172.21.15.112:0/2759503096 wait complete. 2024-09-16T00:39:15.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.086+0000 7ff83a9f6640 1 Processor -- start 2024-09-16T00:39:15.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.086+0000 7ff83a9f6640 1 -- start start 2024-09-16T00:39:15.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.086+0000 7ff83a9f6640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff834102530 0x7ff83406fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:15.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.086+0000 7ff83a9f6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff834102ed0 0x7ff83406ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:15.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.086+0000 7ff83a9f6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff834070490 con 0x7ff834102ed0 2024-09-16T00:39:15.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.086+0000 7ff83a9f6640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff834070600 con 0x7ff834102530 2024-09-16T00:39:15.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.086+0000 7ff82bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff834102ed0 0x7ff83406ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:15.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.086+0000 7ff82bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff834102ed0 0x7ff83406ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45002/0 (socket says 172.21.15.112:45002) 2024-09-16T00:39:15.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.086+0000 7ff82bfff640 1 -- 172.21.15.112:0/3617341166 learned_addr learned my addr 172.21.15.112:0/3617341166 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:39:15.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.086+0000 7ff833fff640 1 --2- 172.21.15.112:0/3617341166 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff834102530 0x7ff83406fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:15.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.086+0000 7ff82bfff640 1 -- 172.21.15.112:0/3617341166 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff834102530 msgr2=0x7ff83406fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:15.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.086+0000 7ff82bfff640 1 --2- 172.21.15.112:0/3617341166 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff834102530 0x7ff83406fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:15.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.086+0000 7ff82bfff640 1 -- 172.21.15.112:0/3617341166 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff81c005040 con 0x7ff834102ed0 2024-09-16T00:39:15.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.087+0000 7ff82bfff640 1 --2- 172.21.15.112:0/3617341166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff834102ed0 0x7ff83406ff50 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff81c02a530 tx=0x7ff81c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:15.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.087+0000 7ff8317fa640 1 -- 172.21.15.112:0/3617341166 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff81c038070 con 0x7ff834102ed0 2024-09-16T00:39:15.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.087+0000 7ff8317fa640 1 -- 172.21.15.112:0/3617341166 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7ff81c02ade0 con 0x7ff834102ed0 2024-09-16T00:39:15.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.087+0000 7ff83a9f6640 1 -- 172.21.15.112:0/3617341166 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff834070830 con 0x7ff834102ed0 2024-09-16T00:39:15.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.087+0000 7ff8317fa640 1 -- 172.21.15.112:0/3617341166 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff81c02c770 con 0x7ff834102ed0 2024-09-16T00:39:15.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.087+0000 7ff83a9f6640 1 -- 172.21.15.112:0/3617341166 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff834119890 con 0x7ff834102ed0 2024-09-16T00:39:15.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.088+0000 7ff83a9f6640 1 -- 172.21.15.112:0/3617341166 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8340fdcb0 con 0x7ff834102ed0 2024-09-16T00:39:15.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.088+0000 7ff8317fa640 1 -- 172.21.15.112:0/3617341166 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7ff81c030020 con 0x7ff834102ed0 2024-09-16T00:39:15.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.089+0000 7ff8317fa640 1 --2- 172.21.15.112:0/3617341166 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff82006e530 0x7ff8200709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:15.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.089+0000 7ff8317fa640 1 -- 172.21.15.112:0/3617341166 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7ff81c0ae670 con 0x7ff834102ed0 2024-09-16T00:39:15.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.089+0000 7ff833fff640 1 --2- 172.21.15.112:0/3617341166 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff82006e530 0x7ff8200709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:15.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.089+0000 7ff833fff640 1 --2- 172.21.15.112:0/3617341166 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff82006e530 0x7ff8200709e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff824009af0 tx=0x7ff82400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:15.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.093+0000 7ff8317fa640 1 -- 172.21.15.112:0/3617341166 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7ff81c0775e0 con 0x7ff834102ed0 2024-09-16T00:39:15.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.275+0000 7ff83a9f6640 1 -- 172.21.15.112:0/3617341166 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff834109560 con 0x7ff82006e530 2024-09-16T00:39:15.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.277+0000 7ff8317fa640 1 -- 172.21.15.112:0/3617341166 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7ff834109560 con 0x7ff82006e530 2024-09-16T00:39:15.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.279+0000 7ff83a9f6640 1 -- 172.21.15.112:0/3617341166 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff82006e530 msgr2=0x7ff8200709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:15.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.279+0000 7ff83a9f6640 1 --2- 172.21.15.112:0/3617341166 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff82006e530 0x7ff8200709e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff824009af0 tx=0x7ff82400b040 comp rx=0 tx=0).stop 2024-09-16T00:39:15.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.279+0000 7ff83a9f6640 1 -- 172.21.15.112:0/3617341166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff834102ed0 msgr2=0x7ff83406ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:15.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.279+0000 7ff83a9f6640 1 --2- 172.21.15.112:0/3617341166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff834102ed0 0x7ff83406ff50 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff81c02a530 tx=0x7ff81c009da0 comp rx=0 tx=0).stop 2024-09-16T00:39:15.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.279+0000 7ff83a9f6640 1 -- 172.21.15.112:0/3617341166 shutdown_connections 2024-09-16T00:39:15.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.279+0000 7ff83a9f6640 1 --2- 172.21.15.112:0/3617341166 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff82006e530 0x7ff8200709e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:15.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.279+0000 7ff83a9f6640 1 --2- 172.21.15.112:0/3617341166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff834102ed0 0x7ff83406ff50 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:15.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.279+0000 7ff83a9f6640 1 --2- 172.21.15.112:0/3617341166 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff834102530 0x7ff83406fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:15.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.279+0000 7ff83a9f6640 1 -- 172.21.15.112:0/3617341166 >> 172.21.15.112:0/3617341166 conn(0x7ff8340f8110 msgr2=0x7ff8340f91a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:15.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.279+0000 7ff83a9f6640 1 -- 172.21.15.112:0/3617341166 shutdown_connections 2024-09-16T00:39:15.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.279+0000 7ff83a9f6640 1 -- 172.21.15.112:0/3617341166 wait complete. 2024-09-16T00:39:15.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.358+0000 7fbe69284640 1 -- 172.21.15.112:0/964839347 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe640fb360 msgr2=0x7fbe640fb730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:15.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.358+0000 7fbe69284640 1 --2- 172.21.15.112:0/964839347 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe640fb360 0x7fbe640fb730 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fbe540048d0 tx=0x7fbe54009da0 comp rx=0 tx=0).stop 2024-09-16T00:39:15.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.358+0000 7fbe69284640 1 -- 172.21.15.112:0/964839347 shutdown_connections 2024-09-16T00:39:15.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.358+0000 7fbe69284640 1 --2- 172.21.15.112:0/964839347 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe640fbd00 0x7fbe64100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:15.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.358+0000 7fbe69284640 1 --2- 172.21.15.112:0/964839347 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe640fb360 0x7fbe640fb730 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:15.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.358+0000 7fbe69284640 1 -- 172.21.15.112:0/964839347 >> 172.21.15.112:0/964839347 conn(0x7fbe640f7000 msgr2=0x7fbe640f9410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:15.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.358+0000 7fbe69284640 1 -- 172.21.15.112:0/964839347 shutdown_connections 2024-09-16T00:39:15.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.358+0000 7fbe69284640 1 -- 172.21.15.112:0/964839347 wait complete. 2024-09-16T00:39:15.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.359+0000 7fbe69284640 1 Processor -- start 2024-09-16T00:39:15.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.359+0000 7fbe69284640 1 -- start start 2024-09-16T00:39:15.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.359+0000 7fbe69284640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe640fb360 0x7fbe641197e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:15.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.359+0000 7fbe69284640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe640fbd00 0x7fbe64119d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:15.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.359+0000 7fbe69284640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6411a370 con 0x7fbe640fbd00 2024-09-16T00:39:15.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.359+0000 7fbe69284640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe641137d0 con 0x7fbe640fb360 2024-09-16T00:39:15.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.359+0000 7fbe5bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe640fbd00 0x7fbe64119d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:15.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.359+0000 7fbe5bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe640fbd00 0x7fbe64119d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45024/0 (socket says 172.21.15.112:45024) 2024-09-16T00:39:15.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.359+0000 7fbe5bfff640 1 -- 172.21.15.112:0/2839619001 learned_addr learned my addr 172.21.15.112:0/2839619001 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:39:15.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.360+0000 7fbe62ffd640 1 --2- 172.21.15.112:0/2839619001 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe640fb360 0x7fbe641197e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:15.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.360+0000 7fbe5bfff640 1 -- 172.21.15.112:0/2839619001 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe640fb360 msgr2=0x7fbe641197e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:15.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.360+0000 7fbe5bfff640 1 --2- 172.21.15.112:0/2839619001 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe640fb360 0x7fbe641197e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:15.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.360+0000 7fbe5bfff640 1 -- 172.21.15.112:0/2839619001 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe54005040 con 0x7fbe640fbd00 2024-09-16T00:39:15.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.360+0000 7fbe62ffd640 1 --2- 172.21.15.112:0/2839619001 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe640fb360 0x7fbe641197e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:39:15.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.360+0000 7fbe5bfff640 1 --2- 172.21.15.112:0/2839619001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe640fbd00 0x7fbe64119d20 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c002a80 tx=0x7fbe4c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:15.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.360+0000 7fbe5b7fe640 1 -- 172.21.15.112:0/2839619001 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe4c007c30 con 0x7fbe640fbd00 2024-09-16T00:39:15.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.360+0000 7fbe5b7fe640 1 -- 172.21.15.112:0/2839619001 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fbe4c00f040 con 0x7fbe640fbd00 2024-09-16T00:39:15.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.360+0000 7fbe69284640 1 -- 172.21.15.112:0/2839619001 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe64113ab0 con 0x7fbe640fbd00 2024-09-16T00:39:15.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.360+0000 7fbe5b7fe640 1 -- 172.21.15.112:0/2839619001 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe4c009530 con 0x7fbe640fbd00 2024-09-16T00:39:15.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.360+0000 7fbe69284640 1 -- 172.21.15.112:0/2839619001 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe64113fd0 con 0x7fbe640fbd00 2024-09-16T00:39:15.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.361+0000 7fbe69284640 1 -- 172.21.15.112:0/2839619001 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe640662d0 con 0x7fbe640fbd00 2024-09-16T00:39:15.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.362+0000 7fbe5b7fe640 1 -- 172.21.15.112:0/2839619001 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fbe4c0097a0 con 0x7fbe640fbd00 2024-09-16T00:39:15.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.362+0000 7fbe5b7fe640 1 --2- 172.21.15.112:0/2839619001 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fbe5006e530 0x7fbe500709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:15.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.362+0000 7fbe62ffd640 1 --2- 172.21.15.112:0/2839619001 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fbe5006e530 0x7fbe500709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:15.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.362+0000 7fbe5b7fe640 1 -- 172.21.15.112:0/2839619001 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7fbe4c08dd00 con 0x7fbe640fbd00 2024-09-16T00:39:15.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.363+0000 7fbe62ffd640 1 --2- 172.21.15.112:0/2839619001 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fbe5006e530 0x7fbe500709e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbe54009ac0 tx=0x7fbe54035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:15.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.367+0000 7fbe5b7fe640 1 -- 172.21.15.112:0/2839619001 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fbe4c091030 con 0x7fbe640fbd00 2024-09-16T00:39:15.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.542+0000 7fbe69284640 1 -- 172.21.15.112:0/2839619001 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fbe64114ef0 con 0x7fbe5006e530 2024-09-16T00:39:15.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.551+0000 7fbe5b7fe640 1 -- 172.21.15.112:0/2839619001 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7fbe64114ef0 con 0x7fbe5006e530 2024-09-16T00:39:15.552 INFO:teuthology.orchestra.run.smithi112.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T00:39:15.552 INFO:teuthology.orchestra.run.smithi112.stdout:alertmanager.smithi112 smithi112 *:9093,9094 running (12m) 5m ago 21m 24.7M - 0.25.0 c8568f914cd2 d85ea0562022 2024-09-16T00:39:15.552 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi112 smithi112 running (9m) 5m ago 21m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c e4fe58f9702a 2024-09-16T00:39:15.553 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi169 smithi169 running (9m) 4m ago 20m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c 3ba2c81d527f 2024-09-16T00:39:15.553 INFO:teuthology.orchestra.run.smithi112.stdout:grafana.smithi112 smithi112 *:3000 running (10m) 5m ago 20m 82.1M - 9.4.12 2bacad6d85d8 51f9d2013da2 2024-09-16T00:39:15.553 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.hxfwkh smithi112 running (5m) 5m ago 15m 14.2M - 19.1.1-326-g8921c8ef f1455bd0219c 1723069902d6 2024-09-16T00:39:15.553 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.ithwzj smithi112 running (5m) 5m ago 15m 12.2M - 19.1.1-326-g8921c8ef f1455bd0219c e7aad5b81db5 2024-09-16T00:39:15.553 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.gawwgw smithi169 running (5m) 4m ago 15m 15.7M - 19.1.1-326-g8921c8ef f1455bd0219c 469be459ee12 2024-09-16T00:39:15.553 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.lgeqrj smithi169 running (4m) 4m ago 15m 13.3M - 19.1.1-326-g8921c8ef f1455bd0219c 31f0fbd2395d 2024-09-16T00:39:15.553 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi112.xrkkbc smithi112 *:8443,9283,8765 running (11m) 5m ago 22m 504M - 19.1.1-326-g8921c8ef f1455bd0219c a981f7fd2810 2024-09-16T00:39:15.553 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi169.mtrobk smithi169 *:8443,9283,8765 running (11m) 4m ago 20m 442M - 19.1.1-326-g8921c8ef f1455bd0219c 4096e5369c2d 2024-09-16T00:39:15.553 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi112 smithi112 running (10m) 5m ago 22m 49.8M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 5aaec033c042 2024-09-16T00:39:15.553 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi169 smithi169 running (9m) 4m ago 20m 38.9M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 7bc609155f3f 2024-09-16T00:39:15.553 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi112 smithi112 *:9100 running (12m) 5m ago 21m 15.5M - 1.5.0 0da6a335fe13 6404d1d601e2 2024-09-16T00:39:15.553 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi169 smithi169 *:9100 running (11m) 4m ago 20m 15.4M - 1.5.0 0da6a335fe13 5505baf702a6 2024-09-16T00:39:15.553 INFO:teuthology.orchestra.run.smithi112.stdout:osd.0 smithi112 running (8m) 5m ago 19m 105M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 53efc9397017 2024-09-16T00:39:15.553 INFO:teuthology.orchestra.run.smithi112.stdout:osd.1 smithi112 running (8m) 5m ago 19m 98.9M 1557M 19.1.1-326-g8921c8ef f1455bd0219c fe6f1a4a2cf4 2024-09-16T00:39:15.554 INFO:teuthology.orchestra.run.smithi112.stdout:osd.2 smithi112 running (7m) 5m ago 18m 102M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 5f7863712057 2024-09-16T00:39:15.554 INFO:teuthology.orchestra.run.smithi112.stdout:osd.3 smithi169 running (6m) 4m ago 18m 92.1M 2581M 19.1.1-326-g8921c8ef f1455bd0219c ee07b5577810 2024-09-16T00:39:15.554 INFO:teuthology.orchestra.run.smithi112.stdout:osd.4 smithi169 running (6m) 4m ago 17m 103M 2581M 19.1.1-326-g8921c8ef f1455bd0219c 769f026a54eb 2024-09-16T00:39:15.554 INFO:teuthology.orchestra.run.smithi112.stdout:osd.5 smithi169 running (5m) 4m ago 17m 125M 2581M 19.1.1-326-g8921c8ef f1455bd0219c a6007c13cf78 2024-09-16T00:39:15.554 INFO:teuthology.orchestra.run.smithi112.stdout:prometheus.smithi112 smithi112 *:9095 running (11m) 5m ago 20m 64.1M - 2.43.0 a07b618ecd1d 7a1d42e60d2b 2024-09-16T00:39:15.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.553+0000 7fbe69284640 1 -- 172.21.15.112:0/2839619001 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fbe5006e530 msgr2=0x7fbe500709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:15.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.553+0000 7fbe69284640 1 --2- 172.21.15.112:0/2839619001 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fbe5006e530 0x7fbe500709e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbe54009ac0 tx=0x7fbe54035040 comp rx=0 tx=0).stop 2024-09-16T00:39:15.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.553+0000 7fbe69284640 1 -- 172.21.15.112:0/2839619001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe640fbd00 msgr2=0x7fbe64119d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:15.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.553+0000 7fbe69284640 1 --2- 172.21.15.112:0/2839619001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe640fbd00 0x7fbe64119d20 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c002a80 tx=0x7fbe4c004600 comp rx=0 tx=0).stop 2024-09-16T00:39:15.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.553+0000 7fbe69284640 1 -- 172.21.15.112:0/2839619001 shutdown_connections 2024-09-16T00:39:15.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.553+0000 7fbe69284640 1 --2- 172.21.15.112:0/2839619001 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fbe5006e530 0x7fbe500709e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:15.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.553+0000 7fbe69284640 1 --2- 172.21.15.112:0/2839619001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe640fbd00 0x7fbe64119d20 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:15.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.553+0000 7fbe69284640 1 --2- 172.21.15.112:0/2839619001 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe640fb360 0x7fbe641197e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:15.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.553+0000 7fbe69284640 1 -- 172.21.15.112:0/2839619001 >> 172.21.15.112:0/2839619001 conn(0x7fbe640f7000 msgr2=0x7fbe640f86f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:15.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.553+0000 7fbe69284640 1 -- 172.21.15.112:0/2839619001 shutdown_connections 2024-09-16T00:39:15.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.553+0000 7fbe69284640 1 -- 172.21.15.112:0/2839619001 wait complete. 2024-09-16T00:39:15.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.631+0000 7f3177c4d640 1 -- 172.21.15.112:0/722183030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31701014e0 msgr2=0x7f31701018b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:15.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.631+0000 7f3177c4d640 1 --2- 172.21.15.112:0/722183030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31701014e0 0x7f31701018b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f316c0048d0 tx=0x7f316c009da0 comp rx=0 tx=0).stop 2024-09-16T00:39:15.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.631+0000 7f3177c4d640 1 -- 172.21.15.112:0/722183030 shutdown_connections 2024-09-16T00:39:15.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.631+0000 7f3177c4d640 1 --2- 172.21.15.112:0/722183030 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31700fb4a0 0x7f31700fb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:15.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.631+0000 7f3177c4d640 1 --2- 172.21.15.112:0/722183030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31701014e0 0x7f31701018b0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:15.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.631+0000 7f3177c4d640 1 -- 172.21.15.112:0/722183030 >> 172.21.15.112:0/722183030 conn(0x7f31700f7060 msgr2=0x7f31700f9470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:15.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.631+0000 7f3177c4d640 1 -- 172.21.15.112:0/722183030 shutdown_connections 2024-09-16T00:39:15.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.631+0000 7f3177c4d640 1 -- 172.21.15.112:0/722183030 wait complete. 2024-09-16T00:39:15.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.632+0000 7f3177c4d640 1 Processor -- start 2024-09-16T00:39:15.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.632+0000 7f3177c4d640 1 -- start start 2024-09-16T00:39:15.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.632+0000 7f3177c4d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31700fb4a0 0x7f31700fd070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:15.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.632+0000 7f3177c4d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31701014e0 0x7f31700fd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:15.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.632+0000 7f3177c4d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31700fdaf0 con 0x7f31701014e0 2024-09-16T00:39:15.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.632+0000 7f3177c4d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31700fdc60 con 0x7f31700fb4a0 2024-09-16T00:39:15.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.632+0000 7f31751e8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31701014e0 0x7f31700fd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:15.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.632+0000 7f31751e8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31701014e0 0x7f31700fd5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45044/0 (socket says 172.21.15.112:45044) 2024-09-16T00:39:15.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.632+0000 7f31751e8640 1 -- 172.21.15.112:0/3159758806 learned_addr learned my addr 172.21.15.112:0/3159758806 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:39:15.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.632+0000 7f31759e9640 1 --2- 172.21.15.112:0/3159758806 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31700fb4a0 0x7f31700fd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:15.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.633+0000 7f31751e8640 1 -- 172.21.15.112:0/3159758806 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31700fb4a0 msgr2=0x7f31700fd070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:15.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.633+0000 7f31751e8640 1 --2- 172.21.15.112:0/3159758806 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31700fb4a0 0x7f31700fd070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:15.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.633+0000 7f31751e8640 1 -- 172.21.15.112:0/3159758806 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f316c005040 con 0x7f31701014e0 2024-09-16T00:39:15.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.633+0000 7f31751e8640 1 --2- 172.21.15.112:0/3159758806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31701014e0 0x7f31700fd5b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3160002a50 tx=0x7f3160004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:15.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.633+0000 7f31667fc640 1 -- 172.21.15.112:0/3159758806 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3160007c30 con 0x7f31701014e0 2024-09-16T00:39:15.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.633+0000 7f31667fc640 1 -- 172.21.15.112:0/3159758806 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f316000f040 con 0x7f31701014e0 2024-09-16T00:39:15.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.633+0000 7f3177c4d640 1 -- 172.21.15.112:0/3159758806 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31700fdef0 con 0x7f31701014e0 2024-09-16T00:39:15.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.633+0000 7f31667fc640 1 -- 172.21.15.112:0/3159758806 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f316000da90 con 0x7f31701014e0 2024-09-16T00:39:15.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.633+0000 7f3177c4d640 1 -- 172.21.15.112:0/3159758806 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31701198e0 con 0x7f31701014e0 2024-09-16T00:39:15.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.634+0000 7f3177c4d640 1 -- 172.21.15.112:0/3159758806 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3154005210 con 0x7f31701014e0 2024-09-16T00:39:15.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.635+0000 7f31667fc640 1 -- 172.21.15.112:0/3159758806 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f316000dc30 con 0x7f31701014e0 2024-09-16T00:39:15.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.635+0000 7f31667fc640 1 --2- 172.21.15.112:0/3159758806 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f315c06e530 0x7f315c0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:15.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.635+0000 7f31759e9640 1 --2- 172.21.15.112:0/3159758806 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f315c06e530 0x7f315c0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:15.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.635+0000 7f31667fc640 1 -- 172.21.15.112:0/3159758806 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f316008dad0 con 0x7f31701014e0 2024-09-16T00:39:15.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.636+0000 7f31759e9640 1 --2- 172.21.15.112:0/3159758806 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f315c06e530 0x7f315c0709e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f316c009ac0 tx=0x7f316c035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:15.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.640+0000 7f31667fc640 1 -- 172.21.15.112:0/3159758806 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f3160056ac0 con 0x7f31701014e0 2024-09-16T00:39:15.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.885+0000 7f3177c4d640 1 -- 172.21.15.112:0/3159758806 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3154005cd0 con 0x7f31701014e0 2024-09-16T00:39:15.886 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:39:15 smithi112 ceph-mon[97750]: pgmap v345: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:39:15.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.886+0000 7f31667fc640 1 -- 172.21.15.112:0/3159758806 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f316005a180 con 0x7f31701014e0 2024-09-16T00:39:15.887 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:39:15.887 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": { 2024-09-16T00:39:15.887 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:39:15.887 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:39:15.887 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": { 2024-09-16T00:39:15.887 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:39:15.887 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:39:15.887 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": { 2024-09-16T00:39:15.887 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 6 2024-09-16T00:39:15.888 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:39:15.888 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": { 2024-09-16T00:39:15.888 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 4 2024-09-16T00:39:15.888 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:39:15.888 INFO:teuthology.orchestra.run.smithi112.stdout: "overall": { 2024-09-16T00:39:15.888 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 14 2024-09-16T00:39:15.888 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-16T00:39:15.888 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:39:15.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.888+0000 7f3177c4d640 1 -- 172.21.15.112:0/3159758806 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f315c06e530 msgr2=0x7f315c0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:15.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.888+0000 7f3177c4d640 1 --2- 172.21.15.112:0/3159758806 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f315c06e530 0x7f315c0709e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f316c009ac0 tx=0x7f316c035040 comp rx=0 tx=0).stop 2024-09-16T00:39:15.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.888+0000 7f3177c4d640 1 -- 172.21.15.112:0/3159758806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31701014e0 msgr2=0x7f31700fd5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:15.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.888+0000 7f3177c4d640 1 --2- 172.21.15.112:0/3159758806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31701014e0 0x7f31700fd5b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3160002a50 tx=0x7f3160004600 comp rx=0 tx=0).stop 2024-09-16T00:39:15.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.888+0000 7f3177c4d640 1 -- 172.21.15.112:0/3159758806 shutdown_connections 2024-09-16T00:39:15.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.888+0000 7f3177c4d640 1 --2- 172.21.15.112:0/3159758806 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f315c06e530 0x7f315c0709e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:15.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.888+0000 7f3177c4d640 1 --2- 172.21.15.112:0/3159758806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31701014e0 0x7f31700fd5b0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:15.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.888+0000 7f3177c4d640 1 --2- 172.21.15.112:0/3159758806 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31700fb4a0 0x7f31700fd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:15.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.888+0000 7f3177c4d640 1 -- 172.21.15.112:0/3159758806 >> 172.21.15.112:0/3159758806 conn(0x7f31700f7060 msgr2=0x7f3170065a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:15.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.888+0000 7f3177c4d640 1 -- 172.21.15.112:0/3159758806 shutdown_connections 2024-09-16T00:39:15.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.888+0000 7f3177c4d640 1 -- 172.21.15.112:0/3159758806 wait complete. 2024-09-16T00:39:15.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.966+0000 7f8bd511d640 1 -- 172.21.15.112:0/736689528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bd0102ef0 msgr2=0x7f8bd00fc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:15.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.966+0000 7f8bd511d640 1 --2- 172.21.15.112:0/736689528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bd0102ef0 0x7f8bd00fc5b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8bb80048d0 tx=0x7f8bb8009da0 comp rx=0 tx=0).stop 2024-09-16T00:39:15.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.966+0000 7f8bd511d640 1 -- 172.21.15.112:0/736689528 shutdown_connections 2024-09-16T00:39:15.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.966+0000 7f8bd511d640 1 --2- 172.21.15.112:0/736689528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bd0102ef0 0x7f8bd00fc5b0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:15.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.966+0000 7f8bd511d640 1 --2- 172.21.15.112:0/736689528 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8bd0102550 0x7f8bd0102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:15.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.966+0000 7f8bd511d640 1 -- 172.21.15.112:0/736689528 >> 172.21.15.112:0/736689528 conn(0x7f8bd00f80f0 msgr2=0x7f8bd00fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:15.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.966+0000 7f8bd511d640 1 -- 172.21.15.112:0/736689528 shutdown_connections 2024-09-16T00:39:15.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.966+0000 7f8bd511d640 1 -- 172.21.15.112:0/736689528 wait complete. 2024-09-16T00:39:15.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.967+0000 7f8bd511d640 1 Processor -- start 2024-09-16T00:39:15.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.967+0000 7f8bd511d640 1 -- start start 2024-09-16T00:39:15.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.967+0000 7f8bd511d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bd0102550 0x7f8bd0113500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:15.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.967+0000 7f8bd511d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8bd0102ef0 0x7f8bd0113a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:15.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.967+0000 7f8bd511d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd0114090 con 0x7f8bd0102550 2024-09-16T00:39:15.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.967+0000 7f8bd511d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd010d4a0 con 0x7f8bd0102ef0 2024-09-16T00:39:15.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.967+0000 7f8bced9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bd0102550 0x7f8bd0113500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:15.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.967+0000 7f8bced9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bd0102550 0x7f8bd0113500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45056/0 (socket says 172.21.15.112:45056) 2024-09-16T00:39:15.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.967+0000 7f8bced9d640 1 -- 172.21.15.112:0/3430606017 learned_addr learned my addr 172.21.15.112:0/3430606017 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:39:15.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.967+0000 7f8bce59c640 1 --2- 172.21.15.112:0/3430606017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8bd0102ef0 0x7f8bd0113a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:15.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.967+0000 7f8bced9d640 1 -- 172.21.15.112:0/3430606017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8bd0102ef0 msgr2=0x7f8bd0113a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:15.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.967+0000 7f8bced9d640 1 --2- 172.21.15.112:0/3430606017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8bd0102ef0 0x7f8bd0113a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:15.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.967+0000 7f8bced9d640 1 -- 172.21.15.112:0/3430606017 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb8005040 con 0x7f8bd0102550 2024-09-16T00:39:15.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.968+0000 7f8bced9d640 1 --2- 172.21.15.112:0/3430606017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bd0102550 0x7f8bd0113500 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0002a80 tx=0x7f8bc0004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:15.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.968+0000 7f8bc77fe640 1 -- 172.21.15.112:0/3430606017 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8bc0007c30 con 0x7f8bd0102550 2024-09-16T00:39:15.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.968+0000 7f8bc77fe640 1 -- 172.21.15.112:0/3430606017 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f8bc000f040 con 0x7f8bd0102550 2024-09-16T00:39:15.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.968+0000 7f8bd511d640 1 -- 172.21.15.112:0/3430606017 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bd010d780 con 0x7f8bd0102550 2024-09-16T00:39:15.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.968+0000 7f8bc77fe640 1 -- 172.21.15.112:0/3430606017 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8bc0009530 con 0x7f8bd0102550 2024-09-16T00:39:15.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.968+0000 7f8bd511d640 1 -- 172.21.15.112:0/3430606017 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bd010dca0 con 0x7f8bd0102550 2024-09-16T00:39:15.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.969+0000 7f8bd511d640 1 -- 172.21.15.112:0/3430606017 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bd00fdcd0 con 0x7f8bd0102550 2024-09-16T00:39:15.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.969+0000 7f8bc77fe640 1 -- 172.21.15.112:0/3430606017 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f8bc00097a0 con 0x7f8bd0102550 2024-09-16T00:39:15.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.970+0000 7f8bc77fe640 1 --2- 172.21.15.112:0/3430606017 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f8bbc06e530 0x7f8bbc0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:15.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.970+0000 7f8bc77fe640 1 -- 172.21.15.112:0/3430606017 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f8bc008dd00 con 0x7f8bd0102550 2024-09-16T00:39:15.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.970+0000 7f8bce59c640 1 --2- 172.21.15.112:0/3430606017 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f8bbc06e530 0x7f8bbc0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:15.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.970+0000 7f8bce59c640 1 --2- 172.21.15.112:0/3430606017 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f8bbc06e530 0x7f8bbc0709e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8bd010e780 tx=0x7f8bb8009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:15.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:15.974+0000 7f8bc77fe640 1 -- 172.21.15.112:0/3430606017 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f8bc0091030 con 0x7f8bd0102550 2024-09-16T00:39:16.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.175+0000 7f8bd511d640 1 -- 172.21.15.112:0/3430606017 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f8bd004b7b0 con 0x7f8bd0102550 2024-09-16T00:39:16.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.175+0000 7f8bc77fe640 1 -- 172.21.15.112:0/3430606017 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1914 (secure 0 0 0) 0x7f8bc0056c70 con 0x7f8bd0102550 2024-09-16T00:39:16.177 INFO:teuthology.orchestra.run.smithi112.stdout:e34 2024-09-16T00:39:16.177 INFO:teuthology.orchestra.run.smithi112.stdout:btime 2024-09-16T00:34:26:619254+0000 2024-09-16T00:39:16.177 INFO:teuthology.orchestra.run.smithi112.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T00:39:16.177 INFO:teuthology.orchestra.run.smithi112.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:39:16.177 INFO:teuthology.orchestra.run.smithi112.stdout:legacy client fscid: 1 2024-09-16T00:39:16.177 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:39:16.177 INFO:teuthology.orchestra.run.smithi112.stdout:Filesystem 'cephfs' (1) 2024-09-16T00:39:16.177 INFO:teuthology.orchestra.run.smithi112.stdout:fs_name cephfs 2024-09-16T00:39:16.178 INFO:teuthology.orchestra.run.smithi112.stdout:epoch 25 2024-09-16T00:39:16.178 INFO:teuthology.orchestra.run.smithi112.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-16T00:39:16.178 INFO:teuthology.orchestra.run.smithi112.stdout:created 2024-09-16T00:23:22.328199+0000 2024-09-16T00:39:16.178 INFO:teuthology.orchestra.run.smithi112.stdout:modified 2024-09-16T00:33:29.012991+0000 2024-09-16T00:39:16.178 INFO:teuthology.orchestra.run.smithi112.stdout:tableserver 0 2024-09-16T00:39:16.178 INFO:teuthology.orchestra.run.smithi112.stdout:root 0 2024-09-16T00:39:16.178 INFO:teuthology.orchestra.run.smithi112.stdout:session_timeout 60 2024-09-16T00:39:16.178 INFO:teuthology.orchestra.run.smithi112.stdout:session_autoclose 300 2024-09-16T00:39:16.178 INFO:teuthology.orchestra.run.smithi112.stdout:max_file_size 1099511627776 2024-09-16T00:39:16.178 INFO:teuthology.orchestra.run.smithi112.stdout:max_xattr_size 65536 2024-09-16T00:39:16.179 INFO:teuthology.orchestra.run.smithi112.stdout:required_client_features {} 2024-09-16T00:39:16.179 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure 0 2024-09-16T00:39:16.179 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure_osd_epoch 82 2024-09-16T00:39:16.179 INFO:teuthology.orchestra.run.smithi112.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:39:16.179 INFO:teuthology.orchestra.run.smithi112.stdout:max_mds 2 2024-09-16T00:39:16.179 INFO:teuthology.orchestra.run.smithi112.stdout:in 0,1 2024-09-16T00:39:16.179 INFO:teuthology.orchestra.run.smithi112.stdout:up {} 2024-09-16T00:39:16.179 INFO:teuthology.orchestra.run.smithi112.stdout:failed 0,1 2024-09-16T00:39:16.180 INFO:teuthology.orchestra.run.smithi112.stdout:damaged 2024-09-16T00:39:16.180 INFO:teuthology.orchestra.run.smithi112.stdout:stopped 2024-09-16T00:39:16.180 INFO:teuthology.orchestra.run.smithi112.stdout:data_pools [3] 2024-09-16T00:39:16.180 INFO:teuthology.orchestra.run.smithi112.stdout:metadata_pool 2 2024-09-16T00:39:16.180 INFO:teuthology.orchestra.run.smithi112.stdout:inline_data enabled 2024-09-16T00:39:16.180 INFO:teuthology.orchestra.run.smithi112.stdout:balancer 2024-09-16T00:39:16.180 INFO:teuthology.orchestra.run.smithi112.stdout:bal_rank_mask -1 2024-09-16T00:39:16.180 INFO:teuthology.orchestra.run.smithi112.stdout:standby_count_wanted 1 2024-09-16T00:39:16.180 INFO:teuthology.orchestra.run.smithi112.stdout:qdb_cluster leader: 0 members: 2024-09-16T00:39:16.180 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:39:16.180 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:39:16.180 INFO:teuthology.orchestra.run.smithi112.stdout:Standby daemons: 2024-09-16T00:39:16.180 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:39:16.180 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.ithwzj{-1:34392} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.112:6826/476229221,v1:172.21.15.112:6827/476229221] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:39:16.181 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.hxfwkh{-1:34404} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.112:6828/2356899876,v1:172.21.15.112:6829/2356899876] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:39:16.181 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.gawwgw{-1:44343} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.169:6824/1635329957,v1:172.21.15.169:6825/1635329957] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:39:16.181 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.lgeqrj{-1:44347} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.169:6826/184810031,v1:172.21.15.169:6827/184810031] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:39:16.181 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 34 2024-09-16T00:39:16.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.177+0000 7f8bd511d640 1 -- 172.21.15.112:0/3430606017 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f8bbc06e530 msgr2=0x7f8bbc0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:16.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.177+0000 7f8bd511d640 1 --2- 172.21.15.112:0/3430606017 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f8bbc06e530 0x7f8bbc0709e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8bd010e780 tx=0x7f8bb8009da0 comp rx=0 tx=0).stop 2024-09-16T00:39:16.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.177+0000 7f8bd511d640 1 -- 172.21.15.112:0/3430606017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bd0102550 msgr2=0x7f8bd0113500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:16.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.177+0000 7f8bd511d640 1 --2- 172.21.15.112:0/3430606017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bd0102550 0x7f8bd0113500 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0002a80 tx=0x7f8bc0004600 comp rx=0 tx=0).stop 2024-09-16T00:39:16.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.177+0000 7f8bd511d640 1 -- 172.21.15.112:0/3430606017 shutdown_connections 2024-09-16T00:39:16.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.177+0000 7f8bd511d640 1 --2- 172.21.15.112:0/3430606017 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f8bbc06e530 0x7f8bbc0709e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:16.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.177+0000 7f8bd511d640 1 --2- 172.21.15.112:0/3430606017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8bd0102ef0 0x7f8bd0113a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:16.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.177+0000 7f8bd511d640 1 --2- 172.21.15.112:0/3430606017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bd0102550 0x7f8bd0113500 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:16.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.177+0000 7f8bd511d640 1 -- 172.21.15.112:0/3430606017 >> 172.21.15.112:0/3430606017 conn(0x7f8bd00f80f0 msgr2=0x7f8bd00f9150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:16.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.178+0000 7f8bd511d640 1 -- 172.21.15.112:0/3430606017 shutdown_connections 2024-09-16T00:39:16.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.178+0000 7f8bd511d640 1 -- 172.21.15.112:0/3430606017 wait complete. 2024-09-16T00:39:16.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.256+0000 7f8e61ad5640 1 -- 172.21.15.112:0/1148306517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e5c102550 msgr2=0x7f8e5c102920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:16.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.256+0000 7f8e61ad5640 1 --2- 172.21.15.112:0/1148306517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e5c102550 0x7f8e5c102920 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c0048d0 tx=0x7f8e4c009da0 comp rx=0 tx=0).stop 2024-09-16T00:39:16.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.257+0000 7f8e61ad5640 1 -- 172.21.15.112:0/1148306517 shutdown_connections 2024-09-16T00:39:16.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.257+0000 7f8e61ad5640 1 --2- 172.21.15.112:0/1148306517 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e5c102ef0 0x7f8e5c0fc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:16.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.257+0000 7f8e61ad5640 1 --2- 172.21.15.112:0/1148306517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e5c102550 0x7f8e5c102920 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:16.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.257+0000 7f8e61ad5640 1 -- 172.21.15.112:0/1148306517 >> 172.21.15.112:0/1148306517 conn(0x7f8e5c0f80f0 msgr2=0x7f8e5c0fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:16.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.257+0000 7f8e61ad5640 1 -- 172.21.15.112:0/1148306517 shutdown_connections 2024-09-16T00:39:16.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.257+0000 7f8e61ad5640 1 -- 172.21.15.112:0/1148306517 wait complete. 2024-09-16T00:39:16.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.257+0000 7f8e61ad5640 1 Processor -- start 2024-09-16T00:39:16.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.257+0000 7f8e61ad5640 1 -- start start 2024-09-16T00:39:16.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.257+0000 7f8e61ad5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e5c102550 0x7f8e5c1135b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:16.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.257+0000 7f8e61ad5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e5c102ef0 0x7f8e5c113af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:16.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.257+0000 7f8e61ad5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5c1141d0 con 0x7f8e5c102ef0 2024-09-16T00:39:16.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.257+0000 7f8e61ad5640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5c10d550 con 0x7f8e5c102550 2024-09-16T00:39:16.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.257+0000 7f8e53fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e5c102ef0 0x7f8e5c113af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:16.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.257+0000 7f8e53fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e5c102ef0 0x7f8e5c113af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45080/0 (socket says 172.21.15.112:45080) 2024-09-16T00:39:16.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.257+0000 7f8e53fff640 1 -- 172.21.15.112:0/2149279120 learned_addr learned my addr 172.21.15.112:0/2149279120 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:39:16.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.258+0000 7f8e5b7fe640 1 --2- 172.21.15.112:0/2149279120 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e5c102550 0x7f8e5c1135b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:16.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.258+0000 7f8e53fff640 1 -- 172.21.15.112:0/2149279120 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e5c102550 msgr2=0x7f8e5c1135b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:16.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.258+0000 7f8e53fff640 1 --2- 172.21.15.112:0/2149279120 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e5c102550 0x7f8e5c1135b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:16.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.258+0000 7f8e53fff640 1 -- 172.21.15.112:0/2149279120 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e4c005040 con 0x7f8e5c102ef0 2024-09-16T00:39:16.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.258+0000 7f8e53fff640 1 --2- 172.21.15.112:0/2149279120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e5c102ef0 0x7f8e5c113af0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8e4400b540 tx=0x7f8e440091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:16.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.258+0000 7f8e58ff9640 1 -- 172.21.15.112:0/2149279120 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e4400f6f0 con 0x7f8e5c102ef0 2024-09-16T00:39:16.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.258+0000 7f8e61ad5640 1 -- 172.21.15.112:0/2149279120 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e5c10d7e0 con 0x7f8e5c102ef0 2024-09-16T00:39:16.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.258+0000 7f8e58ff9640 1 -- 172.21.15.112:0/2149279120 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f8e4400fcf0 con 0x7f8e5c102ef0 2024-09-16T00:39:16.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.258+0000 7f8e58ff9640 1 -- 172.21.15.112:0/2149279120 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e440103a0 con 0x7f8e5c102ef0 2024-09-16T00:39:16.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.258+0000 7f8e61ad5640 1 -- 172.21.15.112:0/2149279120 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e5c10dd00 con 0x7f8e5c102ef0 2024-09-16T00:39:16.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.259+0000 7f8e61ad5640 1 -- 172.21.15.112:0/2149279120 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e5c0fdcd0 con 0x7f8e5c102ef0 2024-09-16T00:39:16.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.260+0000 7f8e58ff9640 1 -- 172.21.15.112:0/2149279120 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f8e4400f850 con 0x7f8e5c102ef0 2024-09-16T00:39:16.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.260+0000 7f8e58ff9640 1 --2- 172.21.15.112:0/2149279120 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f8e4806e530 0x7f8e480709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:16.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.260+0000 7f8e58ff9640 1 -- 172.21.15.112:0/2149279120 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f8e4408e290 con 0x7f8e5c102ef0 2024-09-16T00:39:16.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.260+0000 7f8e5b7fe640 1 --2- 172.21.15.112:0/2149279120 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f8e4806e530 0x7f8e480709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:16.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.261+0000 7f8e5b7fe640 1 --2- 172.21.15.112:0/2149279120 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f8e4806e530 0x7f8e480709e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c000c00 tx=0x7f8e4c035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:16.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.265+0000 7f8e58ff9640 1 -- 172.21.15.112:0/2149279120 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f8e44057280 con 0x7f8e5c102ef0 2024-09-16T00:39:16.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:39:15 smithi169 ceph-mon[77626]: pgmap v345: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:39:16.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.448+0000 7f8e61ad5640 1 -- 172.21.15.112:0/2149279120 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8e5c109670 con 0x7f8e4806e530 2024-09-16T00:39:16.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.449+0000 7f8e58ff9640 1 -- 172.21.15.112:0/2149279120 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f8e5c109670 con 0x7f8e4806e530 2024-09-16T00:39:16.450 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:39:16.450 INFO:teuthology.orchestra.run.smithi112.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80", 2024-09-16T00:39:16.450 INFO:teuthology.orchestra.run.smithi112.stdout: "in_progress": true, 2024-09-16T00:39:16.451 INFO:teuthology.orchestra.run.smithi112.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T00:39:16.451 INFO:teuthology.orchestra.run.smithi112.stdout: "services_complete": [ 2024-09-16T00:39:16.451 INFO:teuthology.orchestra.run.smithi112.stdout: "mds", 2024-09-16T00:39:16.451 INFO:teuthology.orchestra.run.smithi112.stdout: "osd", 2024-09-16T00:39:16.451 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr", 2024-09-16T00:39:16.451 INFO:teuthology.orchestra.run.smithi112.stdout: "mon", 2024-09-16T00:39:16.451 INFO:teuthology.orchestra.run.smithi112.stdout: "crash" 2024-09-16T00:39:16.451 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-16T00:39:16.451 INFO:teuthology.orchestra.run.smithi112.stdout: "progress": "16/21 daemons upgraded", 2024-09-16T00:39:16.451 INFO:teuthology.orchestra.run.smithi112.stdout: "message": "Currently upgrading mds daemons", 2024-09-16T00:39:16.451 INFO:teuthology.orchestra.run.smithi112.stdout: "is_paused": false 2024-09-16T00:39:16.451 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:39:16.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.451+0000 7f8e61ad5640 1 -- 172.21.15.112:0/2149279120 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f8e4806e530 msgr2=0x7f8e480709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:16.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.451+0000 7f8e61ad5640 1 --2- 172.21.15.112:0/2149279120 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f8e4806e530 0x7f8e480709e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c000c00 tx=0x7f8e4c035040 comp rx=0 tx=0).stop 2024-09-16T00:39:16.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.451+0000 7f8e61ad5640 1 -- 172.21.15.112:0/2149279120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e5c102ef0 msgr2=0x7f8e5c113af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:16.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.451+0000 7f8e61ad5640 1 --2- 172.21.15.112:0/2149279120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e5c102ef0 0x7f8e5c113af0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8e4400b540 tx=0x7f8e440091a0 comp rx=0 tx=0).stop 2024-09-16T00:39:16.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.451+0000 7f8e61ad5640 1 -- 172.21.15.112:0/2149279120 shutdown_connections 2024-09-16T00:39:16.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.451+0000 7f8e61ad5640 1 --2- 172.21.15.112:0/2149279120 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f8e4806e530 0x7f8e480709e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:16.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.451+0000 7f8e61ad5640 1 --2- 172.21.15.112:0/2149279120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e5c102ef0 0x7f8e5c113af0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:16.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.451+0000 7f8e61ad5640 1 --2- 172.21.15.112:0/2149279120 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e5c102550 0x7f8e5c1135b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:16.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.451+0000 7f8e61ad5640 1 -- 172.21.15.112:0/2149279120 >> 172.21.15.112:0/2149279120 conn(0x7f8e5c0f80f0 msgr2=0x7f8e5c0f9180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:16.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.451+0000 7f8e61ad5640 1 -- 172.21.15.112:0/2149279120 shutdown_connections 2024-09-16T00:39:16.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.451+0000 7f8e61ad5640 1 -- 172.21.15.112:0/2149279120 wait complete. 2024-09-16T00:39:16.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.530+0000 7f131bfb0640 1 -- 172.21.15.112:0/1242399005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1314102550 msgr2=0x7f1314102920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:16.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.530+0000 7f131bfb0640 1 --2- 172.21.15.112:0/1242399005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1314102550 0x7f1314102920 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f13100048d0 tx=0x7f1310009da0 comp rx=0 tx=0).stop 2024-09-16T00:39:16.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.530+0000 7f131bfb0640 1 -- 172.21.15.112:0/1242399005 shutdown_connections 2024-09-16T00:39:16.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.530+0000 7f131bfb0640 1 --2- 172.21.15.112:0/1242399005 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1314102ef0 0x7f13140fc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:16.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.530+0000 7f131bfb0640 1 --2- 172.21.15.112:0/1242399005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1314102550 0x7f1314102920 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:16.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.530+0000 7f131bfb0640 1 -- 172.21.15.112:0/1242399005 >> 172.21.15.112:0/1242399005 conn(0x7f13140f80f0 msgr2=0x7f13140fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:16.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.530+0000 7f131bfb0640 1 -- 172.21.15.112:0/1242399005 shutdown_connections 2024-09-16T00:39:16.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.530+0000 7f131bfb0640 1 -- 172.21.15.112:0/1242399005 wait complete. 2024-09-16T00:39:16.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.530+0000 7f131bfb0640 1 Processor -- start 2024-09-16T00:39:16.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.531+0000 7f131bfb0640 1 -- start start 2024-09-16T00:39:16.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.531+0000 7f131bfb0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1314102550 0x7f1314113540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:16.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.531+0000 7f131bfb0640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1314102ef0 0x7f1314113a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:16.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.531+0000 7f131bfb0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13141140d0 con 0x7f1314102550 2024-09-16T00:39:16.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.531+0000 7f131bfb0640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f131410d4e0 con 0x7f1314102ef0 2024-09-16T00:39:16.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.531+0000 7f1319d4c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1314102550 0x7f1314113540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:16.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.531+0000 7f1319d4c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1314102550 0x7f1314113540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45096/0 (socket says 172.21.15.112:45096) 2024-09-16T00:39:16.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.531+0000 7f1319d4c640 1 -- 172.21.15.112:0/4153623712 learned_addr learned my addr 172.21.15.112:0/4153623712 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:39:16.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.531+0000 7f131954b640 1 --2- 172.21.15.112:0/4153623712 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1314102ef0 0x7f1314113a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:16.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.531+0000 7f1319d4c640 1 -- 172.21.15.112:0/4153623712 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1314102ef0 msgr2=0x7f1314113a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:16.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.531+0000 7f1319d4c640 1 --2- 172.21.15.112:0/4153623712 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1314102ef0 0x7f1314113a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:16.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.531+0000 7f1319d4c640 1 -- 172.21.15.112:0/4153623712 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1310005040 con 0x7f1314102550 2024-09-16T00:39:16.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.531+0000 7f1319d4c640 1 --2- 172.21.15.112:0/4153623712 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1314102550 0x7f1314113540 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1310003080 tx=0x7f1310009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:16.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.532+0000 7f13067fc640 1 -- 172.21.15.112:0/4153623712 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1310038070 con 0x7f1314102550 2024-09-16T00:39:16.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.532+0000 7f131bfb0640 1 -- 172.21.15.112:0/4153623712 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f131410d760 con 0x7f1314102550 2024-09-16T00:39:16.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.532+0000 7f13067fc640 1 -- 172.21.15.112:0/4153623712 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f13100097e0 con 0x7f1314102550 2024-09-16T00:39:16.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.532+0000 7f13067fc640 1 -- 172.21.15.112:0/4153623712 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f131003d420 con 0x7f1314102550 2024-09-16T00:39:16.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.532+0000 7f131bfb0640 1 -- 172.21.15.112:0/4153623712 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f131410dca0 con 0x7f1314102550 2024-09-16T00:39:16.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.533+0000 7f131bfb0640 1 -- 172.21.15.112:0/4153623712 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13140fdcd0 con 0x7f1314102550 2024-09-16T00:39:16.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.533+0000 7f13067fc640 1 -- 172.21.15.112:0/4153623712 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f1310030020 con 0x7f1314102550 2024-09-16T00:39:16.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.533+0000 7f13067fc640 1 --2- 172.21.15.112:0/4153623712 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f130006e530 0x7f13000709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:16.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.534+0000 7f13067fc640 1 -- 172.21.15.112:0/4153623712 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f13100ae8a0 con 0x7f1314102550 2024-09-16T00:39:16.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.534+0000 7f131954b640 1 --2- 172.21.15.112:0/4153623712 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f130006e530 0x7f13000709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:16.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.534+0000 7f131954b640 1 --2- 172.21.15.112:0/4153623712 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f130006e530 0x7f13000709e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f131410e7c0 tx=0x7f1308009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:16.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.538+0000 7f13067fc640 1 -- 172.21.15.112:0/4153623712 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f1310077810 con 0x7f1314102550 2024-09-16T00:39:16.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.775+0000 7f131bfb0640 1 -- 172.21.15.112:0/4153623712 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f131404b720 con 0x7f1314102550 2024-09-16T00:39:16.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.775+0000 7f13067fc640 1 -- 172.21.15.112:0/4153623712 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f131007aed0 con 0x7f1314102550 2024-09-16T00:39:16.777 INFO:teuthology.orchestra.run.smithi112.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-16T00:39:16.777 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-16T00:39:16.777 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs is degraded 2024-09-16T00:39:16.777 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T00:39:16.777 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T00:39:16.777 INFO:teuthology.orchestra.run.smithi112.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-16T00:39:16.777 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-16T00:39:16.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.777+0000 7f131bfb0640 1 -- 172.21.15.112:0/4153623712 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f130006e530 msgr2=0x7f13000709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:16.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.777+0000 7f131bfb0640 1 --2- 172.21.15.112:0/4153623712 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f130006e530 0x7f13000709e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f131410e7c0 tx=0x7f1308009040 comp rx=0 tx=0).stop 2024-09-16T00:39:16.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.778+0000 7f131bfb0640 1 -- 172.21.15.112:0/4153623712 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1314102550 msgr2=0x7f1314113540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:16.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.778+0000 7f131bfb0640 1 --2- 172.21.15.112:0/4153623712 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1314102550 0x7f1314113540 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1310003080 tx=0x7f1310009da0 comp rx=0 tx=0).stop 2024-09-16T00:39:16.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.778+0000 7f131bfb0640 1 -- 172.21.15.112:0/4153623712 shutdown_connections 2024-09-16T00:39:16.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.778+0000 7f131bfb0640 1 --2- 172.21.15.112:0/4153623712 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f130006e530 0x7f13000709e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:16.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.778+0000 7f131bfb0640 1 --2- 172.21.15.112:0/4153623712 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1314102ef0 0x7f1314113a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:16.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.778+0000 7f131bfb0640 1 --2- 172.21.15.112:0/4153623712 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1314102550 0x7f1314113540 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:16.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.778+0000 7f131bfb0640 1 -- 172.21.15.112:0/4153623712 >> 172.21.15.112:0/4153623712 conn(0x7f13140f80f0 msgr2=0x7f13140f9160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:16.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.778+0000 7f131bfb0640 1 -- 172.21.15.112:0/4153623712 shutdown_connections 2024-09-16T00:39:16.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:16.778+0000 7f131bfb0640 1 -- 172.21.15.112:0/4153623712 wait complete. 2024-09-16T00:39:17.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:39:16 smithi112 ceph-mon[97750]: from='client.34688 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:39:17.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:39:16 smithi112 ceph-mon[97750]: from='client.34692 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:39:17.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:39:16 smithi112 ceph-mon[97750]: from='client.34696 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:39:17.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:39:16 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/3159758806' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:39:17.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:39:16 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/3430606017' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:39:17.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:39:16 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:39:17.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:39:16 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/4153623712' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:39:17.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:39:16 smithi169 ceph-mon[77626]: from='client.34688 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:39:17.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:39:16 smithi169 ceph-mon[77626]: from='client.34692 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:39:17.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:39:16 smithi169 ceph-mon[77626]: from='client.34696 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:39:17.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:39:16 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/3159758806' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:39:17.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:39:16 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/3430606017' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:39:17.278 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:39:16 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:39:17.278 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:39:16 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/4153623712' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:39:18.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:39:17 smithi112 ceph-mon[97750]: from='client.34708 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:39:18.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:39:17 smithi112 ceph-mon[97750]: pgmap v346: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:39:18.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:39:17 smithi169 ceph-mon[77626]: from='client.34708 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:39:18.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:39:17 smithi169 ceph-mon[77626]: pgmap v346: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:39:20.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:39:19 smithi112 ceph-mon[97750]: pgmap v347: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:39:20.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:39:19 smithi169 ceph-mon[77626]: pgmap v347: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:39:22.078 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:39:21 smithi169 ceph-mon[77626]: pgmap v348: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:39:22.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:39:21 smithi112 ceph-mon[97750]: pgmap v348: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:39:24.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:39:23 smithi112 ceph-mon[97750]: pgmap v349: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:39:24.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:39:23 smithi169 ceph-mon[77626]: pgmap v349: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:39:26.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:39:25 smithi112 ceph-mon[97750]: pgmap v350: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:39:26.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:39:25 smithi169 ceph-mon[77626]: pgmap v350: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:39:28.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:39:27 smithi112 ceph-mon[97750]: pgmap v351: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:39:28.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:39:27 smithi169 ceph-mon[77626]: pgmap v351: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:39:30.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:39:29 smithi112 ceph-mon[97750]: pgmap v352: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:39:30.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:39:29 smithi169 ceph-mon[77626]: pgmap v352: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:39:32.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:39:31 smithi112 ceph-mon[97750]: pgmap v353: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:39:32.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:39:31 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:39:32.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:39:31 smithi169 ceph-mon[77626]: pgmap v353: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:39:32.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:39:31 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:39:34.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:39:33 smithi112 ceph-mon[97750]: pgmap v354: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:39:34.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:39:33 smithi169 ceph-mon[77626]: pgmap v354: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:39:36.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:39:35 smithi112 ceph-mon[97750]: pgmap v355: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:39:36.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:39:35 smithi169 ceph-mon[77626]: pgmap v355: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:39:38.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:39:37 smithi112 ceph-mon[97750]: pgmap v356: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:39:38.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:39:37 smithi169 ceph-mon[77626]: pgmap v356: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:39:40.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:39:39 smithi112 ceph-mon[97750]: pgmap v357: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:39:40.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:39:39 smithi169 ceph-mon[77626]: pgmap v357: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:39:42.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:39:41 smithi112 ceph-mon[97750]: pgmap v358: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:39:42.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:39:41 smithi169 ceph-mon[77626]: pgmap v358: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:39:44.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:39:43 smithi112 ceph-mon[97750]: pgmap v359: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:39:44.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:39:43 smithi169 ceph-mon[77626]: pgmap v359: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:39:45.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:39:44 smithi169 ceph-mon[77626]: pgmap v360: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:39:45.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:39:44 smithi112 ceph-mon[97750]: pgmap v360: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:39:46.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:46.870+0000 7fa6ec45d640 1 -- 172.21.15.112:0/2013251756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6e4102550 msgr2=0x7fa6e4102920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:46.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:46.870+0000 7fa6ec45d640 1 --2- 172.21.15.112:0/2013251756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6e4102550 0x7fa6e4102920 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa6e00048d0 tx=0x7fa6e0009da0 comp rx=0 tx=0).stop 2024-09-16T00:39:46.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:46.870+0000 7fa6ec45d640 1 -- 172.21.15.112:0/2013251756 shutdown_connections 2024-09-16T00:39:46.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:46.870+0000 7fa6ec45d640 1 --2- 172.21.15.112:0/2013251756 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6e4102ef0 0x7fa6e40fc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:46.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:46.870+0000 7fa6ec45d640 1 --2- 172.21.15.112:0/2013251756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6e4102550 0x7fa6e4102920 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:46.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:46.870+0000 7fa6ec45d640 1 -- 172.21.15.112:0/2013251756 >> 172.21.15.112:0/2013251756 conn(0x7fa6e40f80f0 msgr2=0x7fa6e40fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:46.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:46.870+0000 7fa6ec45d640 1 -- 172.21.15.112:0/2013251756 shutdown_connections 2024-09-16T00:39:46.872 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:39:46 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:39:46.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:46.871+0000 7fa6ec45d640 1 -- 172.21.15.112:0/2013251756 wait complete. 2024-09-16T00:39:46.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:46.871+0000 7fa6ec45d640 1 Processor -- start 2024-09-16T00:39:46.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:46.871+0000 7fa6ec45d640 1 -- start start 2024-09-16T00:39:46.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:46.871+0000 7fa6ec45d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6e4102550 0x7fa6e4113520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:46.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:46.871+0000 7fa6ec45d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6e4102ef0 0x7fa6e4113a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:46.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:46.871+0000 7fa6ec45d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e41140b0 con 0x7fa6e4102ef0 2024-09-16T00:39:46.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:46.871+0000 7fa6ec45d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e410d4c0 con 0x7fa6e4102550 2024-09-16T00:39:46.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:46.872+0000 7fa6e99f8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6e4102ef0 0x7fa6e4113a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:46.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:46.872+0000 7fa6e99f8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6e4102ef0 0x7fa6e4113a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33488/0 (socket says 172.21.15.112:33488) 2024-09-16T00:39:46.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:46.872+0000 7fa6e99f8640 1 -- 172.21.15.112:0/420999570 learned_addr learned my addr 172.21.15.112:0/420999570 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:39:46.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:46.872+0000 7fa6ea1f9640 1 --2- 172.21.15.112:0/420999570 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6e4102550 0x7fa6e4113520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:46.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:46.872+0000 7fa6e99f8640 1 -- 172.21.15.112:0/420999570 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6e4102550 msgr2=0x7fa6e4113520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:46.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:46.872+0000 7fa6e99f8640 1 --2- 172.21.15.112:0/420999570 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6e4102550 0x7fa6e4113520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:46.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:46.872+0000 7fa6e99f8640 1 -- 172.21.15.112:0/420999570 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6e0005040 con 0x7fa6e4102ef0 2024-09-16T00:39:46.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:46.872+0000 7fa6e99f8640 1 --2- 172.21.15.112:0/420999570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6e4102ef0 0x7fa6e4113a60 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa6d4009af0 tx=0x7fa6d4004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:46.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:46.872+0000 7fa6daffd640 1 -- 172.21.15.112:0/420999570 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa6d4007c30 con 0x7fa6e4102ef0 2024-09-16T00:39:46.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:46.872+0000 7fa6daffd640 1 -- 172.21.15.112:0/420999570 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fa6d400f040 con 0x7fa6e4102ef0 2024-09-16T00:39:46.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:46.872+0000 7fa6ec45d640 1 -- 172.21.15.112:0/420999570 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6e410d7a0 con 0x7fa6e4102ef0 2024-09-16T00:39:46.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:46.872+0000 7fa6daffd640 1 -- 172.21.15.112:0/420999570 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa6d400da10 con 0x7fa6e4102ef0 2024-09-16T00:39:46.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:46.873+0000 7fa6ec45d640 1 -- 172.21.15.112:0/420999570 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6e410dd40 con 0x7fa6e4102ef0 2024-09-16T00:39:46.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:46.873+0000 7fa6ec45d640 1 -- 172.21.15.112:0/420999570 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6e40fdcd0 con 0x7fa6e4102ef0 2024-09-16T00:39:46.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:46.874+0000 7fa6daffd640 1 -- 172.21.15.112:0/420999570 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fa6d4009670 con 0x7fa6e4102ef0 2024-09-16T00:39:46.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:46.875+0000 7fa6daffd640 1 --2- 172.21.15.112:0/420999570 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa6d006e530 0x7fa6d00709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:46.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:46.875+0000 7fa6daffd640 1 -- 172.21.15.112:0/420999570 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7fa6d408e110 con 0x7fa6e4102ef0 2024-09-16T00:39:46.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:46.875+0000 7fa6ea1f9640 1 --2- 172.21.15.112:0/420999570 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa6d006e530 0x7fa6d00709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:46.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:46.875+0000 7fa6ea1f9640 1 --2- 172.21.15.112:0/420999570 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa6d006e530 0x7fa6d00709e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0009ac0 tx=0x7fa6e0035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:46.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:46.880+0000 7fa6daffd640 1 -- 172.21.15.112:0/420999570 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fa6d4057080 con 0x7fa6e4102ef0 2024-09-16T00:39:47.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:39:46 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:39:47.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.062+0000 7fa6ec45d640 1 -- 172.21.15.112:0/420999570 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa6e4113210 con 0x7fa6d006e530 2024-09-16T00:39:47.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.063+0000 7fa6daffd640 1 -- 172.21.15.112:0/420999570 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fa6e4113210 con 0x7fa6d006e530 2024-09-16T00:39:47.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.065+0000 7fa6ec45d640 1 -- 172.21.15.112:0/420999570 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa6d006e530 msgr2=0x7fa6d00709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:47.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.065+0000 7fa6ec45d640 1 --2- 172.21.15.112:0/420999570 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa6d006e530 0x7fa6d00709e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0009ac0 tx=0x7fa6e0035040 comp rx=0 tx=0).stop 2024-09-16T00:39:47.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.065+0000 7fa6ec45d640 1 -- 172.21.15.112:0/420999570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6e4102ef0 msgr2=0x7fa6e4113a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:47.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.065+0000 7fa6ec45d640 1 --2- 172.21.15.112:0/420999570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6e4102ef0 0x7fa6e4113a60 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa6d4009af0 tx=0x7fa6d4004570 comp rx=0 tx=0).stop 2024-09-16T00:39:47.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.066+0000 7fa6ec45d640 1 -- 172.21.15.112:0/420999570 shutdown_connections 2024-09-16T00:39:47.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.066+0000 7fa6ec45d640 1 --2- 172.21.15.112:0/420999570 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa6d006e530 0x7fa6d00709e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:47.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.066+0000 7fa6ec45d640 1 --2- 172.21.15.112:0/420999570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6e4102ef0 0x7fa6e4113a60 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:47.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.066+0000 7fa6ec45d640 1 --2- 172.21.15.112:0/420999570 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6e4102550 0x7fa6e4113520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:47.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.066+0000 7fa6ec45d640 1 -- 172.21.15.112:0/420999570 >> 172.21.15.112:0/420999570 conn(0x7fa6e40f80f0 msgr2=0x7fa6e40f9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:47.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.066+0000 7fa6ec45d640 1 -- 172.21.15.112:0/420999570 shutdown_connections 2024-09-16T00:39:47.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.066+0000 7fa6ec45d640 1 -- 172.21.15.112:0/420999570 wait complete. 2024-09-16T00:39:47.075 INFO:teuthology.orchestra.run.smithi112.stdout:true 2024-09-16T00:39:47.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.146+0000 7f4162112640 1 -- 172.21.15.112:0/3264264308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f415c102ed0 msgr2=0x7f415c0fc590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:47.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.146+0000 7f4162112640 1 --2- 172.21.15.112:0/3264264308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f415c102ed0 0x7f415c0fc590 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f41500048d0 tx=0x7f4150009da0 comp rx=0 tx=0).stop 2024-09-16T00:39:47.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.146+0000 7f4162112640 1 -- 172.21.15.112:0/3264264308 shutdown_connections 2024-09-16T00:39:47.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.146+0000 7f4162112640 1 --2- 172.21.15.112:0/3264264308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f415c102ed0 0x7f415c0fc590 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:47.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.146+0000 7f4162112640 1 --2- 172.21.15.112:0/3264264308 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f415c102530 0x7f415c102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:47.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.146+0000 7f4162112640 1 -- 172.21.15.112:0/3264264308 >> 172.21.15.112:0/3264264308 conn(0x7f415c0f8110 msgr2=0x7f415c0fa520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:47.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.146+0000 7f4162112640 1 -- 172.21.15.112:0/3264264308 shutdown_connections 2024-09-16T00:39:47.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.146+0000 7f4162112640 1 -- 172.21.15.112:0/3264264308 wait complete. 2024-09-16T00:39:47.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.147+0000 7f4162112640 1 Processor -- start 2024-09-16T00:39:47.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.147+0000 7f4162112640 1 -- start start 2024-09-16T00:39:47.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.147+0000 7f4162112640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f415c102530 0x7f415c113210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:47.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.147+0000 7f4162112640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f415c102ed0 0x7f415c113750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:47.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.147+0000 7f4162112640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f415c113d50 con 0x7f415c102530 2024-09-16T00:39:47.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.147+0000 7f4162112640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f415c10d190 con 0x7f415c102ed0 2024-09-16T00:39:47.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.147+0000 7f415b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f415c102530 0x7f415c113210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:47.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.147+0000 7f415b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f415c102530 0x7f415c113210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33502/0 (socket says 172.21.15.112:33502) 2024-09-16T00:39:47.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.147+0000 7f415b7fe640 1 -- 172.21.15.112:0/1379662142 learned_addr learned my addr 172.21.15.112:0/1379662142 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:39:47.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.147+0000 7f415affd640 1 --2- 172.21.15.112:0/1379662142 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f415c102ed0 0x7f415c113750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:47.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.147+0000 7f415b7fe640 1 -- 172.21.15.112:0/1379662142 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f415c102ed0 msgr2=0x7f415c113750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:47.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.147+0000 7f415b7fe640 1 --2- 172.21.15.112:0/1379662142 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f415c102ed0 0x7f415c113750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:47.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.148+0000 7f415b7fe640 1 -- 172.21.15.112:0/1379662142 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4150005040 con 0x7f415c102530 2024-09-16T00:39:47.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.148+0000 7f415b7fe640 1 --2- 172.21.15.112:0/1379662142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f415c102530 0x7f415c113210 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f414c002a50 tx=0x7f414c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:47.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.148+0000 7f4143fff640 1 -- 172.21.15.112:0/1379662142 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f414c007c30 con 0x7f415c102530 2024-09-16T00:39:47.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.148+0000 7f4162112640 1 -- 172.21.15.112:0/1379662142 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f415c10d470 con 0x7f415c102530 2024-09-16T00:39:47.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.148+0000 7f4143fff640 1 -- 172.21.15.112:0/1379662142 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f414c00f040 con 0x7f415c102530 2024-09-16T00:39:47.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.148+0000 7f4143fff640 1 -- 172.21.15.112:0/1379662142 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f414c00da10 con 0x7f415c102530 2024-09-16T00:39:47.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.148+0000 7f4162112640 1 -- 172.21.15.112:0/1379662142 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f415c10d9e0 con 0x7f415c102530 2024-09-16T00:39:47.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.149+0000 7f4162112640 1 -- 172.21.15.112:0/1379662142 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f415c0fdcb0 con 0x7f415c102530 2024-09-16T00:39:47.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.150+0000 7f4143fff640 1 -- 172.21.15.112:0/1379662142 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f414c00dc80 con 0x7f415c102530 2024-09-16T00:39:47.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.150+0000 7f4143fff640 1 --2- 172.21.15.112:0/1379662142 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f414406e530 0x7f41440709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:47.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.150+0000 7f4143fff640 1 -- 172.21.15.112:0/1379662142 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f414c08daf0 con 0x7f415c102530 2024-09-16T00:39:47.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.150+0000 7f415affd640 1 --2- 172.21.15.112:0/1379662142 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f414406e530 0x7f41440709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:47.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.151+0000 7f415affd640 1 --2- 172.21.15.112:0/1379662142 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f414406e530 0x7f41440709e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f41500048a0 tx=0x7f4150035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:47.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.155+0000 7f4143fff640 1 -- 172.21.15.112:0/1379662142 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f414c056b10 con 0x7f415c102530 2024-09-16T00:39:47.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.337+0000 7f4162112640 1 -- 172.21.15.112:0/1379662142 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f415c04b720 con 0x7f414406e530 2024-09-16T00:39:47.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.338+0000 7f4143fff640 1 -- 172.21.15.112:0/1379662142 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f415c04b720 con 0x7f414406e530 2024-09-16T00:39:47.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.340+0000 7f4162112640 1 -- 172.21.15.112:0/1379662142 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f414406e530 msgr2=0x7f41440709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:47.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.340+0000 7f4162112640 1 --2- 172.21.15.112:0/1379662142 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f414406e530 0x7f41440709e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f41500048a0 tx=0x7f4150035040 comp rx=0 tx=0).stop 2024-09-16T00:39:47.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.340+0000 7f4162112640 1 -- 172.21.15.112:0/1379662142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f415c102530 msgr2=0x7f415c113210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:47.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.340+0000 7f4162112640 1 --2- 172.21.15.112:0/1379662142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f415c102530 0x7f415c113210 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f414c002a50 tx=0x7f414c004600 comp rx=0 tx=0).stop 2024-09-16T00:39:47.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.341+0000 7f4162112640 1 -- 172.21.15.112:0/1379662142 shutdown_connections 2024-09-16T00:39:47.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.341+0000 7f4162112640 1 --2- 172.21.15.112:0/1379662142 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f414406e530 0x7f41440709e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:47.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.341+0000 7f4162112640 1 --2- 172.21.15.112:0/1379662142 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f415c102ed0 0x7f415c113750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:47.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.341+0000 7f4162112640 1 --2- 172.21.15.112:0/1379662142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f415c102530 0x7f415c113210 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:47.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.341+0000 7f4162112640 1 -- 172.21.15.112:0/1379662142 >> 172.21.15.112:0/1379662142 conn(0x7f415c0f8110 msgr2=0x7f415c0f91a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:47.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.341+0000 7f4162112640 1 -- 172.21.15.112:0/1379662142 shutdown_connections 2024-09-16T00:39:47.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.341+0000 7f4162112640 1 -- 172.21.15.112:0/1379662142 wait complete. 2024-09-16T00:39:47.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.418+0000 7f376cfef640 1 -- 172.21.15.112:0/3452147434 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37680fac80 msgr2=0x7f37680fb0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:47.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.418+0000 7f376cfef640 1 --2- 172.21.15.112:0/3452147434 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37680fac80 0x7f37680fb0f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f375c0048d0 tx=0x7f375c009da0 comp rx=0 tx=0).stop 2024-09-16T00:39:47.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.418+0000 7f376cfef640 1 -- 172.21.15.112:0/3452147434 shutdown_connections 2024-09-16T00:39:47.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.418+0000 7f376cfef640 1 --2- 172.21.15.112:0/3452147434 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37680fac80 0x7f37680fb0f0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:47.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.418+0000 7f376cfef640 1 --2- 172.21.15.112:0/3452147434 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3768102620 0x7f37681029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:47.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.418+0000 7f376cfef640 1 -- 172.21.15.112:0/3452147434 >> 172.21.15.112:0/3452147434 conn(0x7f3768070ce0 msgr2=0x7f37680710e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:47.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.418+0000 7f376cfef640 1 -- 172.21.15.112:0/3452147434 shutdown_connections 2024-09-16T00:39:47.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.418+0000 7f376cfef640 1 -- 172.21.15.112:0/3452147434 wait complete. 2024-09-16T00:39:47.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.419+0000 7f376cfef640 1 Processor -- start 2024-09-16T00:39:47.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.419+0000 7f376cfef640 1 -- start start 2024-09-16T00:39:47.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.419+0000 7f376cfef640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37680fac80 0x7f37680fbe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:47.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.419+0000 7f376cfef640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3768102620 0x7f37680fc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:47.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.419+0000 7f376cfef640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37680fc8e0 con 0x7f37680fac80 2024-09-16T00:39:47.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.419+0000 7f376cfef640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37680fca50 con 0x7f3768102620 2024-09-16T00:39:47.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.419+0000 7f376659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37680fac80 0x7f37680fbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:47.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.419+0000 7f376659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37680fac80 0x7f37680fbe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33518/0 (socket says 172.21.15.112:33518) 2024-09-16T00:39:47.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.419+0000 7f376659c640 1 -- 172.21.15.112:0/4006154884 learned_addr learned my addr 172.21.15.112:0/4006154884 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:39:47.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.419+0000 7f3765d9b640 1 --2- 172.21.15.112:0/4006154884 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3768102620 0x7f37680fc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:47.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.419+0000 7f376659c640 1 -- 172.21.15.112:0/4006154884 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3768102620 msgr2=0x7f37680fc3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:47.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.419+0000 7f376659c640 1 --2- 172.21.15.112:0/4006154884 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3768102620 0x7f37680fc3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:47.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.420+0000 7f376659c640 1 -- 172.21.15.112:0/4006154884 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f375c005040 con 0x7f37680fac80 2024-09-16T00:39:47.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.420+0000 7f376659c640 1 --2- 172.21.15.112:0/4006154884 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37680fac80 0x7f37680fbe60 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f375800c790 tx=0x7f37580043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:47.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.420+0000 7f3756ffd640 1 -- 172.21.15.112:0/4006154884 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f375800a7a0 con 0x7f37680fac80 2024-09-16T00:39:47.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.420+0000 7f3756ffd640 1 -- 172.21.15.112:0/4006154884 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f375800ada0 con 0x7f37680fac80 2024-09-16T00:39:47.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.420+0000 7f376cfef640 1 -- 172.21.15.112:0/4006154884 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37680fcce0 con 0x7f37680fac80 2024-09-16T00:39:47.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.420+0000 7f3756ffd640 1 -- 172.21.15.112:0/4006154884 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f37580113a0 con 0x7f37680fac80 2024-09-16T00:39:47.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.420+0000 7f376cfef640 1 -- 172.21.15.112:0/4006154884 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37681198e0 con 0x7f37680fac80 2024-09-16T00:39:47.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.421+0000 7f376cfef640 1 -- 172.21.15.112:0/4006154884 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37680657b0 con 0x7f37680fac80 2024-09-16T00:39:47.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.422+0000 7f3756ffd640 1 -- 172.21.15.112:0/4006154884 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f375800a900 con 0x7f37680fac80 2024-09-16T00:39:47.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.422+0000 7f3756ffd640 1 --2- 172.21.15.112:0/4006154884 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f375006e530 0x7f37500709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:47.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.422+0000 7f3756ffd640 1 -- 172.21.15.112:0/4006154884 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f375808e400 con 0x7f37680fac80 2024-09-16T00:39:47.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.422+0000 7f3765d9b640 1 --2- 172.21.15.112:0/4006154884 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f375006e530 0x7f37500709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:47.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.423+0000 7f3765d9b640 1 --2- 172.21.15.112:0/4006154884 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f375006e530 0x7f37500709e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f375c009ac0 tx=0x7f375c035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:47.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.426+0000 7f3756ffd640 1 -- 172.21.15.112:0/4006154884 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f37580573f0 con 0x7f37680fac80 2024-09-16T00:39:47.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.603+0000 7f376cfef640 1 -- 172.21.15.112:0/4006154884 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f37680fd550 con 0x7f375006e530 2024-09-16T00:39:47.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.612+0000 7f3756ffd640 1 -- 172.21.15.112:0/4006154884 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7f37680fd550 con 0x7f375006e530 2024-09-16T00:39:47.612 INFO:teuthology.orchestra.run.smithi112.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T00:39:47.612 INFO:teuthology.orchestra.run.smithi112.stdout:alertmanager.smithi112 smithi112 *:9093,9094 running (12m) 5m ago 21m 24.7M - 0.25.0 c8568f914cd2 d85ea0562022 2024-09-16T00:39:47.613 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi112 smithi112 running (9m) 5m ago 21m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c e4fe58f9702a 2024-09-16T00:39:47.613 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi169 smithi169 running (9m) 5m ago 20m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c 3ba2c81d527f 2024-09-16T00:39:47.613 INFO:teuthology.orchestra.run.smithi112.stdout:grafana.smithi112 smithi112 *:3000 running (11m) 5m ago 21m 82.1M - 9.4.12 2bacad6d85d8 51f9d2013da2 2024-09-16T00:39:47.613 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.hxfwkh smithi112 running (5m) 5m ago 16m 14.2M - 19.1.1-326-g8921c8ef f1455bd0219c 1723069902d6 2024-09-16T00:39:47.613 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.ithwzj smithi112 running (6m) 5m ago 16m 12.2M - 19.1.1-326-g8921c8ef f1455bd0219c e7aad5b81db5 2024-09-16T00:39:47.613 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.gawwgw smithi169 running (5m) 5m ago 16m 15.7M - 19.1.1-326-g8921c8ef f1455bd0219c 469be459ee12 2024-09-16T00:39:47.613 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.lgeqrj smithi169 running (5m) 5m ago 16m 13.3M - 19.1.1-326-g8921c8ef f1455bd0219c 31f0fbd2395d 2024-09-16T00:39:47.613 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi112.xrkkbc smithi112 *:8443,9283,8765 running (12m) 5m ago 22m 504M - 19.1.1-326-g8921c8ef f1455bd0219c a981f7fd2810 2024-09-16T00:39:47.613 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi169.mtrobk smithi169 *:8443,9283,8765 running (11m) 5m ago 20m 442M - 19.1.1-326-g8921c8ef f1455bd0219c 4096e5369c2d 2024-09-16T00:39:47.613 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi112 smithi112 running (10m) 5m ago 23m 49.8M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 5aaec033c042 2024-09-16T00:39:47.613 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi169 smithi169 running (10m) 5m ago 20m 38.9M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 7bc609155f3f 2024-09-16T00:39:47.613 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi112 smithi112 *:9100 running (13m) 5m ago 21m 15.5M - 1.5.0 0da6a335fe13 6404d1d601e2 2024-09-16T00:39:47.614 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi169 smithi169 *:9100 running (12m) 5m ago 20m 15.4M - 1.5.0 0da6a335fe13 5505baf702a6 2024-09-16T00:39:47.614 INFO:teuthology.orchestra.run.smithi112.stdout:osd.0 smithi112 running (9m) 5m ago 20m 105M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 53efc9397017 2024-09-16T00:39:47.614 INFO:teuthology.orchestra.run.smithi112.stdout:osd.1 smithi112 running (8m) 5m ago 19m 98.9M 1557M 19.1.1-326-g8921c8ef f1455bd0219c fe6f1a4a2cf4 2024-09-16T00:39:47.614 INFO:teuthology.orchestra.run.smithi112.stdout:osd.2 smithi112 running (8m) 5m ago 19m 102M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 5f7863712057 2024-09-16T00:39:47.614 INFO:teuthology.orchestra.run.smithi112.stdout:osd.3 smithi169 running (7m) 5m ago 18m 92.1M 2581M 19.1.1-326-g8921c8ef f1455bd0219c ee07b5577810 2024-09-16T00:39:47.614 INFO:teuthology.orchestra.run.smithi112.stdout:osd.4 smithi169 running (7m) 5m ago 18m 103M 2581M 19.1.1-326-g8921c8ef f1455bd0219c 769f026a54eb 2024-09-16T00:39:47.614 INFO:teuthology.orchestra.run.smithi112.stdout:osd.5 smithi169 running (6m) 5m ago 17m 125M 2581M 19.1.1-326-g8921c8ef f1455bd0219c a6007c13cf78 2024-09-16T00:39:47.614 INFO:teuthology.orchestra.run.smithi112.stdout:prometheus.smithi112 smithi112 *:9095 running (11m) 5m ago 21m 64.1M - 2.43.0 a07b618ecd1d 7a1d42e60d2b 2024-09-16T00:39:47.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.614+0000 7f376cfef640 1 -- 172.21.15.112:0/4006154884 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f375006e530 msgr2=0x7f37500709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:47.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.614+0000 7f376cfef640 1 --2- 172.21.15.112:0/4006154884 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f375006e530 0x7f37500709e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f375c009ac0 tx=0x7f375c035040 comp rx=0 tx=0).stop 2024-09-16T00:39:47.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.614+0000 7f376cfef640 1 -- 172.21.15.112:0/4006154884 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37680fac80 msgr2=0x7f37680fbe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:47.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.614+0000 7f376cfef640 1 --2- 172.21.15.112:0/4006154884 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37680fac80 0x7f37680fbe60 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f375800c790 tx=0x7f37580043c0 comp rx=0 tx=0).stop 2024-09-16T00:39:47.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.614+0000 7f376cfef640 1 -- 172.21.15.112:0/4006154884 shutdown_connections 2024-09-16T00:39:47.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.614+0000 7f376cfef640 1 --2- 172.21.15.112:0/4006154884 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f375006e530 0x7f37500709e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:47.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.614+0000 7f376cfef640 1 --2- 172.21.15.112:0/4006154884 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3768102620 0x7f37680fc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:47.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.614+0000 7f376cfef640 1 --2- 172.21.15.112:0/4006154884 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37680fac80 0x7f37680fbe60 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:47.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.614+0000 7f376cfef640 1 -- 172.21.15.112:0/4006154884 >> 172.21.15.112:0/4006154884 conn(0x7f3768070ce0 msgr2=0x7f3768105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:47.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.615+0000 7f376cfef640 1 -- 172.21.15.112:0/4006154884 shutdown_connections 2024-09-16T00:39:47.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.615+0000 7f376cfef640 1 -- 172.21.15.112:0/4006154884 wait complete. 2024-09-16T00:39:47.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.691+0000 7f95dd379640 1 -- 172.21.15.112:0/661760456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95d81025f0 msgr2=0x7f95d81029c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:47.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.691+0000 7f95dd379640 1 --2- 172.21.15.112:0/661760456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95d81025f0 0x7f95d81029c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f95c80048d0 tx=0x7f95c8009da0 comp rx=0 tx=0).stop 2024-09-16T00:39:47.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.691+0000 7f95dd379640 1 -- 172.21.15.112:0/661760456 shutdown_connections 2024-09-16T00:39:47.692 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:39:47 smithi112 ceph-mon[97750]: pgmap v361: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:39:47.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.691+0000 7f95dd379640 1 --2- 172.21.15.112:0/661760456 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95d80fac60 0x7f95d80fb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:47.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.691+0000 7f95dd379640 1 --2- 172.21.15.112:0/661760456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95d81025f0 0x7f95d81029c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:47.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.691+0000 7f95dd379640 1 -- 172.21.15.112:0/661760456 >> 172.21.15.112:0/661760456 conn(0x7f95d8070c80 msgr2=0x7f95d8071080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:47.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.692+0000 7f95dd379640 1 -- 172.21.15.112:0/661760456 shutdown_connections 2024-09-16T00:39:47.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.692+0000 7f95dd379640 1 -- 172.21.15.112:0/661760456 wait complete. 2024-09-16T00:39:47.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.692+0000 7f95dd379640 1 Processor -- start 2024-09-16T00:39:47.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.692+0000 7f95dd379640 1 -- start start 2024-09-16T00:39:47.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.692+0000 7f95dd379640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95d80fac60 0x7f95d80fbe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:47.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.692+0000 7f95dd379640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95d81025f0 0x7f95d80fc380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:47.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.692+0000 7f95dd379640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d80fc8c0 con 0x7f95d80fac60 2024-09-16T00:39:47.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.692+0000 7f95dd379640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d80fca30 con 0x7f95d81025f0 2024-09-16T00:39:47.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.693+0000 7f95d6ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95d80fac60 0x7f95d80fbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:47.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.693+0000 7f95d6ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95d80fac60 0x7f95d80fbe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33526/0 (socket says 172.21.15.112:33526) 2024-09-16T00:39:47.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.693+0000 7f95d6ffd640 1 -- 172.21.15.112:0/654974243 learned_addr learned my addr 172.21.15.112:0/654974243 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:39:47.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.693+0000 7f95d67fc640 1 --2- 172.21.15.112:0/654974243 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95d81025f0 0x7f95d80fc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:47.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.693+0000 7f95d6ffd640 1 -- 172.21.15.112:0/654974243 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95d81025f0 msgr2=0x7f95d80fc380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:47.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.693+0000 7f95d6ffd640 1 --2- 172.21.15.112:0/654974243 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95d81025f0 0x7f95d80fc380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:47.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.693+0000 7f95d6ffd640 1 -- 172.21.15.112:0/654974243 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c8005040 con 0x7f95d80fac60 2024-09-16T00:39:47.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.693+0000 7f95d6ffd640 1 --2- 172.21.15.112:0/654974243 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95d80fac60 0x7f95d80fbe40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f95c8000c00 tx=0x7f95c8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:47.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.693+0000 7f95bf7fe640 1 -- 172.21.15.112:0/654974243 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95c8038070 con 0x7f95d80fac60 2024-09-16T00:39:47.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.693+0000 7f95bf7fe640 1 -- 172.21.15.112:0/654974243 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f95c802ade0 con 0x7f95d80fac60 2024-09-16T00:39:47.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.693+0000 7f95dd379640 1 -- 172.21.15.112:0/654974243 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95d80fcc60 con 0x7f95d80fac60 2024-09-16T00:39:47.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.693+0000 7f95bf7fe640 1 -- 172.21.15.112:0/654974243 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95c802c7e0 con 0x7f95d80fac60 2024-09-16T00:39:47.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.693+0000 7f95dd379640 1 -- 172.21.15.112:0/654974243 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95d8119870 con 0x7f95d80fac60 2024-09-16T00:39:47.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.695+0000 7f95dd379640 1 -- 172.21.15.112:0/654974243 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95d8065810 con 0x7f95d80fac60 2024-09-16T00:39:47.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.695+0000 7f95bf7fe640 1 -- 172.21.15.112:0/654974243 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f95c8030020 con 0x7f95d80fac60 2024-09-16T00:39:47.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.695+0000 7f95bf7fe640 1 --2- 172.21.15.112:0/654974243 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f95c006e530 0x7f95c00709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:47.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.696+0000 7f95bf7fe640 1 -- 172.21.15.112:0/654974243 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f95c80adf70 con 0x7f95d80fac60 2024-09-16T00:39:47.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.696+0000 7f95d67fc640 1 --2- 172.21.15.112:0/654974243 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f95c006e530 0x7f95c00709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:47.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.696+0000 7f95d67fc640 1 --2- 172.21.15.112:0/654974243 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f95c006e530 0x7f95c00709e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f95d80fd780 tx=0x7f95cc00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:47.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.700+0000 7f95bf7fe640 1 -- 172.21.15.112:0/654974243 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f95c80e31b0 con 0x7f95d80fac60 2024-09-16T00:39:47.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.931+0000 7f95dd379640 1 -- 172.21.15.112:0/654974243 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f95d80fd530 con 0x7f95d80fac60 2024-09-16T00:39:47.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.931+0000 7f95bf7fe640 1 -- 172.21.15.112:0/654974243 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f95c8076f90 con 0x7f95d80fac60 2024-09-16T00:39:47.931 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:39:47.932 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": { 2024-09-16T00:39:47.932 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:39:47.932 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:39:47.932 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": { 2024-09-16T00:39:47.932 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:39:47.932 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:39:47.932 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": { 2024-09-16T00:39:47.932 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 6 2024-09-16T00:39:47.932 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:39:47.932 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": { 2024-09-16T00:39:47.932 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 4 2024-09-16T00:39:47.932 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:39:47.932 INFO:teuthology.orchestra.run.smithi112.stdout: "overall": { 2024-09-16T00:39:47.933 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 14 2024-09-16T00:39:47.933 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-16T00:39:47.933 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:39:47.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.933+0000 7f95dd379640 1 -- 172.21.15.112:0/654974243 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f95c006e530 msgr2=0x7f95c00709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:47.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.933+0000 7f95dd379640 1 --2- 172.21.15.112:0/654974243 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f95c006e530 0x7f95c00709e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f95d80fd780 tx=0x7f95cc00b040 comp rx=0 tx=0).stop 2024-09-16T00:39:47.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.933+0000 7f95dd379640 1 -- 172.21.15.112:0/654974243 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95d80fac60 msgr2=0x7f95d80fbe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:47.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.933+0000 7f95dd379640 1 --2- 172.21.15.112:0/654974243 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95d80fac60 0x7f95d80fbe40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f95c8000c00 tx=0x7f95c8009da0 comp rx=0 tx=0).stop 2024-09-16T00:39:47.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.933+0000 7f95dd379640 1 -- 172.21.15.112:0/654974243 shutdown_connections 2024-09-16T00:39:47.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.933+0000 7f95dd379640 1 --2- 172.21.15.112:0/654974243 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f95c006e530 0x7f95c00709e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:47.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.933+0000 7f95dd379640 1 --2- 172.21.15.112:0/654974243 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95d81025f0 0x7f95d80fc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:47.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.933+0000 7f95dd379640 1 --2- 172.21.15.112:0/654974243 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95d80fac60 0x7f95d80fbe40 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:47.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.933+0000 7f95dd379640 1 -- 172.21.15.112:0/654974243 >> 172.21.15.112:0/654974243 conn(0x7f95d8070c80 msgr2=0x7f95d80f9ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:47.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.933+0000 7f95dd379640 1 -- 172.21.15.112:0/654974243 shutdown_connections 2024-09-16T00:39:47.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:47.933+0000 7f95dd379640 1 -- 172.21.15.112:0/654974243 wait complete. 2024-09-16T00:39:48.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.012+0000 7ff3d3546640 1 -- 172.21.15.112:0/3735978623 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3cc1014c0 msgr2=0x7ff3cc101890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:48.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.012+0000 7ff3d3546640 1 --2- 172.21.15.112:0/3735978623 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3cc1014c0 0x7ff3cc101890 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc0048d0 tx=0x7ff3bc009da0 comp rx=0 tx=0).stop 2024-09-16T00:39:48.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.012+0000 7ff3d3546640 1 -- 172.21.15.112:0/3735978623 shutdown_connections 2024-09-16T00:39:48.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.012+0000 7ff3d3546640 1 --2- 172.21.15.112:0/3735978623 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3cc0fb480 0x7ff3cc0fb8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:48.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.012+0000 7ff3d3546640 1 --2- 172.21.15.112:0/3735978623 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3cc1014c0 0x7ff3cc101890 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:48.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.012+0000 7ff3d3546640 1 -- 172.21.15.112:0/3735978623 >> 172.21.15.112:0/3735978623 conn(0x7ff3cc0f7000 msgr2=0x7ff3cc0f9410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:48.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.012+0000 7ff3d3546640 1 -- 172.21.15.112:0/3735978623 shutdown_connections 2024-09-16T00:39:48.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.012+0000 7ff3d3546640 1 -- 172.21.15.112:0/3735978623 wait complete. 2024-09-16T00:39:48.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.013+0000 7ff3d3546640 1 Processor -- start 2024-09-16T00:39:48.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.013+0000 7ff3d3546640 1 -- start start 2024-09-16T00:39:48.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.013+0000 7ff3d3546640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3cc0fb480 0x7ff3cc1131c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:48.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.013+0000 7ff3d3546640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3cc1014c0 0x7ff3cc113700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:48.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.013+0000 7ff3d3546640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3cc113d00 con 0x7ff3cc1014c0 2024-09-16T00:39:48.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.013+0000 7ff3d3546640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3cc10d140 con 0x7ff3cc0fb480 2024-09-16T00:39:48.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.013+0000 7ff3d0ae1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3cc1014c0 0x7ff3cc113700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:48.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.013+0000 7ff3d0ae1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3cc1014c0 0x7ff3cc113700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33558/0 (socket says 172.21.15.112:33558) 2024-09-16T00:39:48.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.013+0000 7ff3d0ae1640 1 -- 172.21.15.112:0/3890363645 learned_addr learned my addr 172.21.15.112:0/3890363645 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:39:48.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.013+0000 7ff3d12e2640 1 --2- 172.21.15.112:0/3890363645 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3cc0fb480 0x7ff3cc1131c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:48.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.013+0000 7ff3d0ae1640 1 -- 172.21.15.112:0/3890363645 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3cc0fb480 msgr2=0x7ff3cc1131c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:48.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.013+0000 7ff3d0ae1640 1 --2- 172.21.15.112:0/3890363645 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3cc0fb480 0x7ff3cc1131c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:48.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.013+0000 7ff3d0ae1640 1 -- 172.21.15.112:0/3890363645 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3bc005040 con 0x7ff3cc1014c0 2024-09-16T00:39:48.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.013+0000 7ff3d12e2640 1 --2- 172.21.15.112:0/3890363645 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3cc0fb480 0x7ff3cc1131c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:39:48.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.013+0000 7ff3d0ae1640 1 --2- 172.21.15.112:0/3890363645 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3cc1014c0 0x7ff3cc113700 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff3c800c830 tx=0x7ff3c8004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:48.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.014+0000 7ff3c1ffb640 1 -- 172.21.15.112:0/3890363645 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff3c800a850 con 0x7ff3cc1014c0 2024-09-16T00:39:48.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.014+0000 7ff3d3546640 1 -- 172.21.15.112:0/3890363645 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3cc10d420 con 0x7ff3cc1014c0 2024-09-16T00:39:48.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.014+0000 7ff3c1ffb640 1 -- 172.21.15.112:0/3890363645 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7ff3c800ae50 con 0x7ff3cc1014c0 2024-09-16T00:39:48.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.014+0000 7ff3c1ffb640 1 -- 172.21.15.112:0/3890363645 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff3c80113a0 con 0x7ff3cc1014c0 2024-09-16T00:39:48.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.014+0000 7ff3d3546640 1 -- 172.21.15.112:0/3890363645 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3cc10d9c0 con 0x7ff3cc1014c0 2024-09-16T00:39:48.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.015+0000 7ff3d3546640 1 -- 172.21.15.112:0/3890363645 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3cc0fcbc0 con 0x7ff3cc1014c0 2024-09-16T00:39:48.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.016+0000 7ff3c1ffb640 1 -- 172.21.15.112:0/3890363645 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7ff3c8012400 con 0x7ff3cc1014c0 2024-09-16T00:39:48.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.016+0000 7ff3c1ffb640 1 --2- 172.21.15.112:0/3890363645 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff3b806e530 0x7ff3b80709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:48.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.016+0000 7ff3c1ffb640 1 -- 172.21.15.112:0/3890363645 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7ff3c808e480 con 0x7ff3cc1014c0 2024-09-16T00:39:48.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.016+0000 7ff3d12e2640 1 --2- 172.21.15.112:0/3890363645 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff3b806e530 0x7ff3b80709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:48.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.017+0000 7ff3d12e2640 1 --2- 172.21.15.112:0/3890363645 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff3b806e530 0x7ff3b80709e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc00b020 tx=0x7ff3bc009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:48.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.021+0000 7ff3c1ffb640 1 -- 172.21.15.112:0/3890363645 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7ff3c8057510 con 0x7ff3cc1014c0 2024-09-16T00:39:48.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:39:47 smithi169 ceph-mon[77626]: pgmap v361: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:39:48.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.221+0000 7ff3d3546640 1 -- 172.21.15.112:0/3890363645 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff3cc0fac40 con 0x7ff3cc1014c0 2024-09-16T00:39:48.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.221+0000 7ff3c1ffb640 1 -- 172.21.15.112:0/3890363645 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1914 (secure 0 0 0) 0x7ff3c805abd0 con 0x7ff3cc1014c0 2024-09-16T00:39:48.222 INFO:teuthology.orchestra.run.smithi112.stdout:e34 2024-09-16T00:39:48.222 INFO:teuthology.orchestra.run.smithi112.stdout:btime 2024-09-16T00:34:26:619254+0000 2024-09-16T00:39:48.222 INFO:teuthology.orchestra.run.smithi112.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T00:39:48.222 INFO:teuthology.orchestra.run.smithi112.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:39:48.222 INFO:teuthology.orchestra.run.smithi112.stdout:legacy client fscid: 1 2024-09-16T00:39:48.222 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:39:48.222 INFO:teuthology.orchestra.run.smithi112.stdout:Filesystem 'cephfs' (1) 2024-09-16T00:39:48.222 INFO:teuthology.orchestra.run.smithi112.stdout:fs_name cephfs 2024-09-16T00:39:48.222 INFO:teuthology.orchestra.run.smithi112.stdout:epoch 25 2024-09-16T00:39:48.223 INFO:teuthology.orchestra.run.smithi112.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-16T00:39:48.223 INFO:teuthology.orchestra.run.smithi112.stdout:created 2024-09-16T00:23:22.328199+0000 2024-09-16T00:39:48.223 INFO:teuthology.orchestra.run.smithi112.stdout:modified 2024-09-16T00:33:29.012991+0000 2024-09-16T00:39:48.223 INFO:teuthology.orchestra.run.smithi112.stdout:tableserver 0 2024-09-16T00:39:48.223 INFO:teuthology.orchestra.run.smithi112.stdout:root 0 2024-09-16T00:39:48.223 INFO:teuthology.orchestra.run.smithi112.stdout:session_timeout 60 2024-09-16T00:39:48.223 INFO:teuthology.orchestra.run.smithi112.stdout:session_autoclose 300 2024-09-16T00:39:48.223 INFO:teuthology.orchestra.run.smithi112.stdout:max_file_size 1099511627776 2024-09-16T00:39:48.223 INFO:teuthology.orchestra.run.smithi112.stdout:max_xattr_size 65536 2024-09-16T00:39:48.224 INFO:teuthology.orchestra.run.smithi112.stdout:required_client_features {} 2024-09-16T00:39:48.224 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure 0 2024-09-16T00:39:48.224 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure_osd_epoch 82 2024-09-16T00:39:48.224 INFO:teuthology.orchestra.run.smithi112.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:39:48.224 INFO:teuthology.orchestra.run.smithi112.stdout:max_mds 2 2024-09-16T00:39:48.224 INFO:teuthology.orchestra.run.smithi112.stdout:in 0,1 2024-09-16T00:39:48.224 INFO:teuthology.orchestra.run.smithi112.stdout:up {} 2024-09-16T00:39:48.224 INFO:teuthology.orchestra.run.smithi112.stdout:failed 0,1 2024-09-16T00:39:48.224 INFO:teuthology.orchestra.run.smithi112.stdout:damaged 2024-09-16T00:39:48.224 INFO:teuthology.orchestra.run.smithi112.stdout:stopped 2024-09-16T00:39:48.224 INFO:teuthology.orchestra.run.smithi112.stdout:data_pools [3] 2024-09-16T00:39:48.225 INFO:teuthology.orchestra.run.smithi112.stdout:metadata_pool 2 2024-09-16T00:39:48.225 INFO:teuthology.orchestra.run.smithi112.stdout:inline_data enabled 2024-09-16T00:39:48.225 INFO:teuthology.orchestra.run.smithi112.stdout:balancer 2024-09-16T00:39:48.225 INFO:teuthology.orchestra.run.smithi112.stdout:bal_rank_mask -1 2024-09-16T00:39:48.225 INFO:teuthology.orchestra.run.smithi112.stdout:standby_count_wanted 1 2024-09-16T00:39:48.225 INFO:teuthology.orchestra.run.smithi112.stdout:qdb_cluster leader: 0 members: 2024-09-16T00:39:48.225 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:39:48.225 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:39:48.225 INFO:teuthology.orchestra.run.smithi112.stdout:Standby daemons: 2024-09-16T00:39:48.225 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:39:48.225 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.ithwzj{-1:34392} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.112:6826/476229221,v1:172.21.15.112:6827/476229221] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:39:48.225 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.hxfwkh{-1:34404} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.112:6828/2356899876,v1:172.21.15.112:6829/2356899876] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:39:48.225 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.gawwgw{-1:44343} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.169:6824/1635329957,v1:172.21.15.169:6825/1635329957] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:39:48.225 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.lgeqrj{-1:44347} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.169:6826/184810031,v1:172.21.15.169:6827/184810031] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:39:48.226 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 34 2024-09-16T00:39:48.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.223+0000 7ff3d3546640 1 -- 172.21.15.112:0/3890363645 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff3b806e530 msgr2=0x7ff3b80709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:48.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.223+0000 7ff3d3546640 1 --2- 172.21.15.112:0/3890363645 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff3b806e530 0x7ff3b80709e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc00b020 tx=0x7ff3bc009da0 comp rx=0 tx=0).stop 2024-09-16T00:39:48.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.223+0000 7ff3d3546640 1 -- 172.21.15.112:0/3890363645 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3cc1014c0 msgr2=0x7ff3cc113700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:48.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.223+0000 7ff3d3546640 1 --2- 172.21.15.112:0/3890363645 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3cc1014c0 0x7ff3cc113700 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff3c800c830 tx=0x7ff3c8004430 comp rx=0 tx=0).stop 2024-09-16T00:39:48.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.223+0000 7ff3d3546640 1 -- 172.21.15.112:0/3890363645 shutdown_connections 2024-09-16T00:39:48.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.223+0000 7ff3d3546640 1 --2- 172.21.15.112:0/3890363645 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff3b806e530 0x7ff3b80709e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:48.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.223+0000 7ff3d3546640 1 --2- 172.21.15.112:0/3890363645 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3cc1014c0 0x7ff3cc113700 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:48.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.223+0000 7ff3d3546640 1 --2- 172.21.15.112:0/3890363645 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3cc0fb480 0x7ff3cc1131c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:48.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.223+0000 7ff3d3546640 1 -- 172.21.15.112:0/3890363645 >> 172.21.15.112:0/3890363645 conn(0x7ff3cc0f7000 msgr2=0x7ff3cc065a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:48.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.223+0000 7ff3d3546640 1 -- 172.21.15.112:0/3890363645 shutdown_connections 2024-09-16T00:39:48.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.223+0000 7ff3d3546640 1 -- 172.21.15.112:0/3890363645 wait complete. 2024-09-16T00:39:48.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.302+0000 7f0f29902640 1 -- 172.21.15.112:0/3497449511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f240fb480 msgr2=0x7f0f240fb8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:48.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.302+0000 7f0f29902640 1 --2- 172.21.15.112:0/3497449511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f240fb480 0x7f0f240fb8f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c0048d0 tx=0x7f0f0c009da0 comp rx=0 tx=0).stop 2024-09-16T00:39:48.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.302+0000 7f0f29902640 1 -- 172.21.15.112:0/3497449511 shutdown_connections 2024-09-16T00:39:48.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.302+0000 7f0f29902640 1 --2- 172.21.15.112:0/3497449511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f240fb480 0x7f0f240fb8f0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:48.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.302+0000 7f0f29902640 1 --2- 172.21.15.112:0/3497449511 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f241014c0 0x7f0f24101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:48.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.302+0000 7f0f29902640 1 -- 172.21.15.112:0/3497449511 >> 172.21.15.112:0/3497449511 conn(0x7f0f240f7000 msgr2=0x7f0f240f9410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:48.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.302+0000 7f0f29902640 1 -- 172.21.15.112:0/3497449511 shutdown_connections 2024-09-16T00:39:48.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.302+0000 7f0f29902640 1 -- 172.21.15.112:0/3497449511 wait complete. 2024-09-16T00:39:48.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.303+0000 7f0f29902640 1 Processor -- start 2024-09-16T00:39:48.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.303+0000 7f0f29902640 1 -- start start 2024-09-16T00:39:48.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.303+0000 7f0f29902640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f240fb480 0x7f0f2410e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:48.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.303+0000 7f0f29902640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f241014c0 0x7f0f2410e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:48.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.303+0000 7f0f29902640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2410ed20 con 0x7f0f240fb480 2024-09-16T00:39:48.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.303+0000 7f0f29902640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2410ee90 con 0x7f0f241014c0 2024-09-16T00:39:48.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.303+0000 7f0f22ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f240fb480 0x7f0f2410e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:48.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.303+0000 7f0f22ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f240fb480 0x7f0f2410e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33578/0 (socket says 172.21.15.112:33578) 2024-09-16T00:39:48.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.303+0000 7f0f22ffd640 1 -- 172.21.15.112:0/3432065939 learned_addr learned my addr 172.21.15.112:0/3432065939 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:39:48.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.303+0000 7f0f1bfff640 1 --2- 172.21.15.112:0/3432065939 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f241014c0 0x7f0f2410e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:48.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.303+0000 7f0f22ffd640 1 -- 172.21.15.112:0/3432065939 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f241014c0 msgr2=0x7f0f2410e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:48.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.303+0000 7f0f22ffd640 1 --2- 172.21.15.112:0/3432065939 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f241014c0 0x7f0f2410e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:48.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.303+0000 7f0f22ffd640 1 -- 172.21.15.112:0/3432065939 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f0c005040 con 0x7f0f240fb480 2024-09-16T00:39:48.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.304+0000 7f0f1bfff640 1 --2- 172.21.15.112:0/3432065939 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f241014c0 0x7f0f2410e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:39:48.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.304+0000 7f0f22ffd640 1 --2- 172.21.15.112:0/3432065939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f240fb480 0x7f0f2410e2a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0f14002a80 tx=0x7f0f14004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:48.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.304+0000 7f0f1b7fe640 1 -- 172.21.15.112:0/3432065939 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f14007c30 con 0x7f0f240fb480 2024-09-16T00:39:48.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.304+0000 7f0f29902640 1 -- 172.21.15.112:0/3432065939 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f24117170 con 0x7f0f240fb480 2024-09-16T00:39:48.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.304+0000 7f0f1b7fe640 1 -- 172.21.15.112:0/3432065939 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f0f1400f040 con 0x7f0f240fb480 2024-09-16T00:39:48.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.304+0000 7f0f1b7fe640 1 -- 172.21.15.112:0/3432065939 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f14009530 con 0x7f0f240fb480 2024-09-16T00:39:48.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.304+0000 7f0f29902640 1 -- 172.21.15.112:0/3432065939 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f24117660 con 0x7f0f240fb480 2024-09-16T00:39:48.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.305+0000 7f0f29902640 1 -- 172.21.15.112:0/3432065939 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f240fcbc0 con 0x7f0f240fb480 2024-09-16T00:39:48.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.306+0000 7f0f1b7fe640 1 -- 172.21.15.112:0/3432065939 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f0f140097a0 con 0x7f0f240fb480 2024-09-16T00:39:48.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.306+0000 7f0f1b7fe640 1 --2- 172.21.15.112:0/3432065939 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f0f1006e530 0x7f0f100709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:48.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.306+0000 7f0f1b7fe640 1 -- 172.21.15.112:0/3432065939 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f0f1408dd00 con 0x7f0f240fb480 2024-09-16T00:39:48.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.306+0000 7f0f1bfff640 1 --2- 172.21.15.112:0/3432065939 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f0f1006e530 0x7f0f100709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:48.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.307+0000 7f0f1bfff640 1 --2- 172.21.15.112:0/3432065939 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f0f1006e530 0x7f0f100709e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0f24066450 tx=0x7f0f0c035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:48.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.311+0000 7f0f1b7fe640 1 -- 172.21.15.112:0/3432065939 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f0f14091030 con 0x7f0f240fb480 2024-09-16T00:39:48.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.491+0000 7f0f29902640 1 -- 172.21.15.112:0/3432065939 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0f24066200 con 0x7f0f1006e530 2024-09-16T00:39:48.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.492+0000 7f0f1b7fe640 1 -- 172.21.15.112:0/3432065939 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f0f24066200 con 0x7f0f1006e530 2024-09-16T00:39:48.492 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:39:48.492 INFO:teuthology.orchestra.run.smithi112.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80", 2024-09-16T00:39:48.492 INFO:teuthology.orchestra.run.smithi112.stdout: "in_progress": true, 2024-09-16T00:39:48.493 INFO:teuthology.orchestra.run.smithi112.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T00:39:48.493 INFO:teuthology.orchestra.run.smithi112.stdout: "services_complete": [ 2024-09-16T00:39:48.493 INFO:teuthology.orchestra.run.smithi112.stdout: "mds", 2024-09-16T00:39:48.493 INFO:teuthology.orchestra.run.smithi112.stdout: "osd", 2024-09-16T00:39:48.493 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr", 2024-09-16T00:39:48.493 INFO:teuthology.orchestra.run.smithi112.stdout: "mon", 2024-09-16T00:39:48.493 INFO:teuthology.orchestra.run.smithi112.stdout: "crash" 2024-09-16T00:39:48.493 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-16T00:39:48.493 INFO:teuthology.orchestra.run.smithi112.stdout: "progress": "16/21 daemons upgraded", 2024-09-16T00:39:48.493 INFO:teuthology.orchestra.run.smithi112.stdout: "message": "Currently upgrading mds daemons", 2024-09-16T00:39:48.493 INFO:teuthology.orchestra.run.smithi112.stdout: "is_paused": false 2024-09-16T00:39:48.493 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:39:48.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.494+0000 7f0f29902640 1 -- 172.21.15.112:0/3432065939 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f0f1006e530 msgr2=0x7f0f100709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:48.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.494+0000 7f0f29902640 1 --2- 172.21.15.112:0/3432065939 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f0f1006e530 0x7f0f100709e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0f24066450 tx=0x7f0f0c035040 comp rx=0 tx=0).stop 2024-09-16T00:39:48.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.494+0000 7f0f29902640 1 -- 172.21.15.112:0/3432065939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f240fb480 msgr2=0x7f0f2410e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:48.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.494+0000 7f0f29902640 1 --2- 172.21.15.112:0/3432065939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f240fb480 0x7f0f2410e2a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0f14002a80 tx=0x7f0f14004600 comp rx=0 tx=0).stop 2024-09-16T00:39:48.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.494+0000 7f0f29902640 1 -- 172.21.15.112:0/3432065939 shutdown_connections 2024-09-16T00:39:48.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.494+0000 7f0f29902640 1 --2- 172.21.15.112:0/3432065939 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f0f1006e530 0x7f0f100709e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:48.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.494+0000 7f0f29902640 1 --2- 172.21.15.112:0/3432065939 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f241014c0 0x7f0f2410e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:48.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.494+0000 7f0f29902640 1 --2- 172.21.15.112:0/3432065939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f240fb480 0x7f0f2410e2a0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:48.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.494+0000 7f0f29902640 1 -- 172.21.15.112:0/3432065939 >> 172.21.15.112:0/3432065939 conn(0x7f0f240f7000 msgr2=0x7f0f240f88d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:48.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.494+0000 7f0f29902640 1 -- 172.21.15.112:0/3432065939 shutdown_connections 2024-09-16T00:39:48.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.494+0000 7f0f29902640 1 -- 172.21.15.112:0/3432065939 wait complete. 2024-09-16T00:39:48.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.573+0000 7fcf2f4c4640 1 -- 172.21.15.112:0/2651038342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf280fb490 msgr2=0x7fcf280fb900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:48.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.573+0000 7fcf2f4c4640 1 --2- 172.21.15.112:0/2651038342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf280fb490 0x7fcf280fb900 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fcf240048d0 tx=0x7fcf24009da0 comp rx=0 tx=0).stop 2024-09-16T00:39:48.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.573+0000 7fcf2f4c4640 1 -- 172.21.15.112:0/2651038342 shutdown_connections 2024-09-16T00:39:48.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.573+0000 7fcf2f4c4640 1 --2- 172.21.15.112:0/2651038342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf280fb490 0x7fcf280fb900 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:48.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.573+0000 7fcf2f4c4640 1 --2- 172.21.15.112:0/2651038342 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf281014d0 0x7fcf281018a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:48.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.573+0000 7fcf2f4c4640 1 -- 172.21.15.112:0/2651038342 >> 172.21.15.112:0/2651038342 conn(0x7fcf280f7010 msgr2=0x7fcf280f9420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:48.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.573+0000 7fcf2f4c4640 1 -- 172.21.15.112:0/2651038342 shutdown_connections 2024-09-16T00:39:48.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.573+0000 7fcf2f4c4640 1 -- 172.21.15.112:0/2651038342 wait complete. 2024-09-16T00:39:48.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.573+0000 7fcf2f4c4640 1 Processor -- start 2024-09-16T00:39:48.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.573+0000 7fcf2f4c4640 1 -- start start 2024-09-16T00:39:48.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.574+0000 7fcf2f4c4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf280fb490 0x7fcf2810e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:48.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.574+0000 7fcf2f4c4640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf281014d0 0x7fcf2810e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:48.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.574+0000 7fcf2f4c4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2810ed30 con 0x7fcf280fb490 2024-09-16T00:39:48.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.574+0000 7fcf2f4c4640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2810eea0 con 0x7fcf281014d0 2024-09-16T00:39:48.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.574+0000 7fcf2d260640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf280fb490 0x7fcf2810e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:48.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.574+0000 7fcf2d260640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf280fb490 0x7fcf2810e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33600/0 (socket says 172.21.15.112:33600) 2024-09-16T00:39:48.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.574+0000 7fcf2d260640 1 -- 172.21.15.112:0/2248600160 learned_addr learned my addr 172.21.15.112:0/2248600160 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:39:48.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.574+0000 7fcf2ca5f640 1 --2- 172.21.15.112:0/2248600160 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf281014d0 0x7fcf2810e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:48.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.574+0000 7fcf2d260640 1 -- 172.21.15.112:0/2248600160 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf281014d0 msgr2=0x7fcf2810e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:48.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.574+0000 7fcf2d260640 1 --2- 172.21.15.112:0/2248600160 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf281014d0 0x7fcf2810e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:48.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.574+0000 7fcf2d260640 1 -- 172.21.15.112:0/2248600160 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf24005040 con 0x7fcf280fb490 2024-09-16T00:39:48.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.574+0000 7fcf2d260640 1 --2- 172.21.15.112:0/2248600160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf280fb490 0x7fcf2810e1a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fcf18002a50 tx=0x7fcf18004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:48.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.574+0000 7fcf1dffb640 1 -- 172.21.15.112:0/2248600160 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf18007c30 con 0x7fcf280fb490 2024-09-16T00:39:48.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.575+0000 7fcf1dffb640 1 -- 172.21.15.112:0/2248600160 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fcf1800f040 con 0x7fcf280fb490 2024-09-16T00:39:48.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.575+0000 7fcf2f4c4640 1 -- 172.21.15.112:0/2248600160 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf28117210 con 0x7fcf280fb490 2024-09-16T00:39:48.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.575+0000 7fcf1dffb640 1 -- 172.21.15.112:0/2248600160 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf1800da10 con 0x7fcf280fb490 2024-09-16T00:39:48.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.575+0000 7fcf2f4c4640 1 -- 172.21.15.112:0/2248600160 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf28117720 con 0x7fcf280fb490 2024-09-16T00:39:48.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.576+0000 7fcf2f4c4640 1 -- 172.21.15.112:0/2248600160 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf280fcbd0 con 0x7fcf280fb490 2024-09-16T00:39:48.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.576+0000 7fcf1dffb640 1 -- 172.21.15.112:0/2248600160 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fcf1800dc50 con 0x7fcf280fb490 2024-09-16T00:39:48.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.577+0000 7fcf1dffb640 1 --2- 172.21.15.112:0/2248600160 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fcf1406e530 0x7fcf140709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:39:48.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.577+0000 7fcf1dffb640 1 -- 172.21.15.112:0/2248600160 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7fcf1808d9b0 con 0x7fcf280fb490 2024-09-16T00:39:48.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.577+0000 7fcf2ca5f640 1 --2- 172.21.15.112:0/2248600160 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fcf1406e530 0x7fcf140709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:39:48.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.578+0000 7fcf2ca5f640 1 --2- 172.21.15.112:0/2248600160 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fcf1406e530 0x7fcf140709e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fcf24009ac0 tx=0x7fcf24035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:39:48.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.581+0000 7fcf1dffb640 1 -- 172.21.15.112:0/2248600160 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fcf18056a50 con 0x7fcf280fb490 2024-09-16T00:39:48.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.828+0000 7fcf2f4c4640 1 -- 172.21.15.112:0/2248600160 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fcf28066280 con 0x7fcf280fb490 2024-09-16T00:39:48.829 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:39:48 smithi112 ceph-mon[97750]: from='client.34716 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:39:48.829 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:39:48 smithi112 ceph-mon[97750]: from='client.34720 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:39:48.829 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:39:48 smithi112 ceph-mon[97750]: from='client.34724 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:39:48.829 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:39:48 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/654974243' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:39:48.829 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:39:48 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/3890363645' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:39:48.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.829+0000 7fcf1dffb640 1 -- 172.21.15.112:0/2248600160 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7fcf1808be60 con 0x7fcf280fb490 2024-09-16T00:39:48.830 INFO:teuthology.orchestra.run.smithi112.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-16T00:39:48.830 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-16T00:39:48.830 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs is degraded 2024-09-16T00:39:48.830 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T00:39:48.830 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T00:39:48.830 INFO:teuthology.orchestra.run.smithi112.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-16T00:39:48.831 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-16T00:39:48.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.831+0000 7fcf2f4c4640 1 -- 172.21.15.112:0/2248600160 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fcf1406e530 msgr2=0x7fcf140709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:48.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.831+0000 7fcf2f4c4640 1 --2- 172.21.15.112:0/2248600160 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fcf1406e530 0x7fcf140709e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fcf24009ac0 tx=0x7fcf24035040 comp rx=0 tx=0).stop 2024-09-16T00:39:48.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.831+0000 7fcf2f4c4640 1 -- 172.21.15.112:0/2248600160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf280fb490 msgr2=0x7fcf2810e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:39:48.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.831+0000 7fcf2f4c4640 1 --2- 172.21.15.112:0/2248600160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf280fb490 0x7fcf2810e1a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fcf18002a50 tx=0x7fcf18004600 comp rx=0 tx=0).stop 2024-09-16T00:39:48.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.831+0000 7fcf2f4c4640 1 -- 172.21.15.112:0/2248600160 shutdown_connections 2024-09-16T00:39:48.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.831+0000 7fcf2f4c4640 1 --2- 172.21.15.112:0/2248600160 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fcf1406e530 0x7fcf140709e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:48.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.831+0000 7fcf2f4c4640 1 --2- 172.21.15.112:0/2248600160 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf281014d0 0x7fcf2810e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:48.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.831+0000 7fcf2f4c4640 1 --2- 172.21.15.112:0/2248600160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf280fb490 0x7fcf2810e1a0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:39:48.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.831+0000 7fcf2f4c4640 1 -- 172.21.15.112:0/2248600160 >> 172.21.15.112:0/2248600160 conn(0x7fcf280f7010 msgr2=0x7fcf280f8930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:39:48.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.831+0000 7fcf2f4c4640 1 -- 172.21.15.112:0/2248600160 shutdown_connections 2024-09-16T00:39:48.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:39:48.831+0000 7fcf2f4c4640 1 -- 172.21.15.112:0/2248600160 wait complete. 2024-09-16T00:39:49.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:39:48 smithi169 ceph-mon[77626]: from='client.34716 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:39:49.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:39:48 smithi169 ceph-mon[77626]: from='client.34720 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:39:49.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:39:48 smithi169 ceph-mon[77626]: from='client.34724 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:39:49.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:39:48 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/654974243' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:39:49.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:39:48 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/3890363645' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:39:50.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:39:49 smithi169 ceph-mon[77626]: from='client.34736 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:39:50.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:39:49 smithi169 ceph-mon[77626]: pgmap v362: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:39:50.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:39:49 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/2248600160' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:39:50.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:39:49 smithi112 ceph-mon[97750]: from='client.34736 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:39:50.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:39:49 smithi112 ceph-mon[97750]: pgmap v362: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:39:50.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:39:49 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/2248600160' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:39:52.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:39:51 smithi169 ceph-mon[77626]: pgmap v363: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:39:52.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:39:51 smithi112 ceph-mon[97750]: pgmap v363: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:39:54.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:39:53 smithi169 ceph-mon[77626]: pgmap v364: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:39:54.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:39:53 smithi112 ceph-mon[97750]: pgmap v364: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:39:56.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:39:55 smithi169 ceph-mon[77626]: pgmap v365: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:39:56.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:39:55 smithi112 ceph-mon[97750]: pgmap v365: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:39:58.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:39:57 smithi169 ceph-mon[77626]: pgmap v366: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:39:58.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:39:57 smithi112 ceph-mon[97750]: pgmap v366: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:39:59.999 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:39:59 smithi112 ceph-mon[97750]: pgmap v367: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:40:00.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:39:59 smithi169 ceph-mon[77626]: pgmap v367: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:40:00.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:40:00 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-mon-smithi112[97746]: 2024-09-16T00:39:59.999+0000 7fccfe10c640 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-16T00:40:00.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:40:00 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-mon-smithi112[97746]: 2024-09-16T00:39:59.999+0000 7fccfe10c640 -1 log_channel(cluster) log [ERR] : [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-16T00:40:00.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:40:00 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-mon-smithi112[97746]: 2024-09-16T00:39:59.999+0000 7fccfe10c640 -1 log_channel(cluster) log [ERR] : fs cephfs is degraded 2024-09-16T00:40:00.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:40:00 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-mon-smithi112[97746]: 2024-09-16T00:39:59.999+0000 7fccfe10c640 -1 log_channel(cluster) log [ERR] : [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T00:40:00.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:40:00 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-mon-smithi112[97746]: 2024-09-16T00:39:59.999+0000 7fccfe10c640 -1 log_channel(cluster) log [ERR] : fs cephfs has deprecated feature inline_data enabled. 2024-09-16T00:40:00.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:40:00 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-mon-smithi112[97746]: 2024-09-16T00:39:59.999+0000 7fccfe10c640 -1 log_channel(cluster) log [ERR] : [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-16T00:40:00.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:40:00 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-mon-smithi112[97746]: 2024-09-16T00:39:59.999+0000 7fccfe10c640 -1 log_channel(cluster) log [ERR] : fs cephfs is offline because no MDS is active for it. 2024-09-16T00:40:01.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:40:00 smithi169 ceph-mon[77626]: Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-16T00:40:01.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:40:00 smithi169 ceph-mon[77626]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-16T00:40:01.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:40:00 smithi169 ceph-mon[77626]: fs cephfs is degraded 2024-09-16T00:40:01.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:40:00 smithi169 ceph-mon[77626]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T00:40:01.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:40:00 smithi169 ceph-mon[77626]: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T00:40:01.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:40:00 smithi169 ceph-mon[77626]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-16T00:40:01.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:40:00 smithi169 ceph-mon[77626]: fs cephfs is offline because no MDS is active for it. 2024-09-16T00:40:01.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:40:00 smithi112 ceph-mon[97750]: Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-16T00:40:01.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:40:00 smithi112 ceph-mon[97750]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-16T00:40:01.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:40:00 smithi112 ceph-mon[97750]: fs cephfs is degraded 2024-09-16T00:40:01.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:40:00 smithi112 ceph-mon[97750]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T00:40:01.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:40:00 smithi112 ceph-mon[97750]: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T00:40:01.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:40:00 smithi112 ceph-mon[97750]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-16T00:40:01.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:40:00 smithi112 ceph-mon[97750]: fs cephfs is offline because no MDS is active for it. 2024-09-16T00:40:02.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:40:01 smithi169 ceph-mon[77626]: pgmap v368: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:40:02.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:40:01 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:40:02.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:40:01 smithi112 ceph-mon[97750]: pgmap v368: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:40:02.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:40:01 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:40:04.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:40:03 smithi169 ceph-mon[77626]: pgmap v369: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:40:04.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:40:03 smithi112 ceph-mon[97750]: pgmap v369: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:40:06.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:40:05 smithi169 ceph-mon[77626]: pgmap v370: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:40:06.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:40:05 smithi112 ceph-mon[97750]: pgmap v370: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:40:08.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:40:07 smithi169 ceph-mon[77626]: pgmap v371: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:40:08.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:40:07 smithi112 ceph-mon[97750]: pgmap v371: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:40:10.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:40:09 smithi169 ceph-mon[77626]: pgmap v372: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:40:10.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:40:09 smithi112 ceph-mon[97750]: pgmap v372: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:40:12.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:40:11 smithi169 ceph-mon[77626]: pgmap v373: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:40:12.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:40:11 smithi112 ceph-mon[97750]: pgmap v373: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:40:14.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:40:13 smithi169 ceph-mon[77626]: pgmap v374: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:40:14.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:40:13 smithi112 ceph-mon[97750]: pgmap v374: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:40:16.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:40:15 smithi169 ceph-mon[77626]: pgmap v375: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:40:16.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:40:15 smithi112 ceph-mon[97750]: pgmap v375: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:40:17.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:40:16 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:40:17.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:40:16 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:40:18.033 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:40:17 smithi169 ceph-mon[77626]: pgmap v376: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:40:18.690 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:40:17 smithi112 ceph-mon[97750]: pgmap v376: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:40:18.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:18.917+0000 7fbf91f65640 1 -- 172.21.15.112:0/2776516445 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf8c0fdf30 msgr2=0x7fbf8c106570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:18.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:18.917+0000 7fbf91f65640 1 --2- 172.21.15.112:0/2776516445 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf8c0fdf30 0x7fbf8c106570 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fbf800048d0 tx=0x7fbf80009da0 comp rx=0 tx=0).stop 2024-09-16T00:40:18.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:18.917+0000 7fbf91f65640 1 -- 172.21.15.112:0/2776516445 shutdown_connections 2024-09-16T00:40:18.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:18.917+0000 7fbf91f65640 1 --2- 172.21.15.112:0/2776516445 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf8c0fdf30 0x7fbf8c106570 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:18.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:18.917+0000 7fbf91f65640 1 --2- 172.21.15.112:0/2776516445 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf8c0fd590 0x7fbf8c0fd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:18.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:18.917+0000 7fbf91f65640 1 -- 172.21.15.112:0/2776516445 >> 172.21.15.112:0/2776516445 conn(0x7fbf8c0f7060 msgr2=0x7fbf8c0f9470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:18.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:18.917+0000 7fbf91f65640 1 -- 172.21.15.112:0/2776516445 shutdown_connections 2024-09-16T00:40:18.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:18.917+0000 7fbf91f65640 1 -- 172.21.15.112:0/2776516445 wait complete. 2024-09-16T00:40:18.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:18.918+0000 7fbf91f65640 1 Processor -- start 2024-09-16T00:40:18.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:18.918+0000 7fbf91f65640 1 -- start start 2024-09-16T00:40:18.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:18.918+0000 7fbf91f65640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf8c0fd590 0x7fbf8c113260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:18.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:18.918+0000 7fbf91f65640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf8c0fdf30 0x7fbf8c1137a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:18.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:18.918+0000 7fbf91f65640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8c113da0 con 0x7fbf8c0fd590 2024-09-16T00:40:18.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:18.918+0000 7fbf91f65640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8c10d1e0 con 0x7fbf8c0fdf30 2024-09-16T00:40:18.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:18.918+0000 7fbf8b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf8c0fd590 0x7fbf8c113260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:18.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:18.918+0000 7fbf8b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf8c0fd590 0x7fbf8c113260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49050/0 (socket says 172.21.15.112:49050) 2024-09-16T00:40:18.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:18.918+0000 7fbf8b7fe640 1 -- 172.21.15.112:0/962029793 learned_addr learned my addr 172.21.15.112:0/962029793 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:40:18.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:18.919+0000 7fbf8affd640 1 --2- 172.21.15.112:0/962029793 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf8c0fdf30 0x7fbf8c1137a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:18.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:18.919+0000 7fbf8b7fe640 1 -- 172.21.15.112:0/962029793 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf8c0fdf30 msgr2=0x7fbf8c1137a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:18.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:18.919+0000 7fbf8b7fe640 1 --2- 172.21.15.112:0/962029793 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf8c0fdf30 0x7fbf8c1137a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:18.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:18.919+0000 7fbf8b7fe640 1 -- 172.21.15.112:0/962029793 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf80005040 con 0x7fbf8c0fd590 2024-09-16T00:40:18.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:18.919+0000 7fbf8b7fe640 1 --2- 172.21.15.112:0/962029793 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf8c0fd590 0x7fbf8c113260 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c00c790 tx=0x7fbf7c0043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:18.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:18.919+0000 7fbf73fff640 1 -- 172.21.15.112:0/962029793 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf7c00a7a0 con 0x7fbf8c0fd590 2024-09-16T00:40:18.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:18.919+0000 7fbf73fff640 1 -- 172.21.15.112:0/962029793 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fbf7c00ada0 con 0x7fbf8c0fd590 2024-09-16T00:40:18.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:18.919+0000 7fbf91f65640 1 -- 172.21.15.112:0/962029793 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf8c10d4c0 con 0x7fbf8c0fd590 2024-09-16T00:40:18.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:18.919+0000 7fbf73fff640 1 -- 172.21.15.112:0/962029793 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf7c0113a0 con 0x7fbf8c0fd590 2024-09-16T00:40:18.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:18.919+0000 7fbf91f65640 1 -- 172.21.15.112:0/962029793 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf8c10d9e0 con 0x7fbf8c0fd590 2024-09-16T00:40:18.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:18.920+0000 7fbf91f65640 1 -- 172.21.15.112:0/962029793 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf8c066350 con 0x7fbf8c0fd590 2024-09-16T00:40:18.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:18.921+0000 7fbf73fff640 1 -- 172.21.15.112:0/962029793 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fbf7c00a900 con 0x7fbf8c0fd590 2024-09-16T00:40:18.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:18.921+0000 7fbf73fff640 1 --2- 172.21.15.112:0/962029793 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fbf7406e530 0x7fbf740709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:18.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:18.921+0000 7fbf73fff640 1 -- 172.21.15.112:0/962029793 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7fbf7c08e470 con 0x7fbf8c0fd590 2024-09-16T00:40:18.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:18.921+0000 7fbf8affd640 1 --2- 172.21.15.112:0/962029793 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fbf7406e530 0x7fbf740709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:18.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:18.922+0000 7fbf8affd640 1 --2- 172.21.15.112:0/962029793 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fbf7406e530 0x7fbf740709e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbf80007ac0 tx=0x7fbf80009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:18.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:18.926+0000 7fbf73fff640 1 -- 172.21.15.112:0/962029793 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fbf7c057460 con 0x7fbf8c0fd590 2024-09-16T00:40:19.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.109+0000 7fbf91f65640 1 -- 172.21.15.112:0/962029793 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbf8c04b720 con 0x7fbf7406e530 2024-09-16T00:40:19.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.110+0000 7fbf73fff640 1 -- 172.21.15.112:0/962029793 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fbf8c04b720 con 0x7fbf7406e530 2024-09-16T00:40:19.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.112+0000 7fbf91f65640 1 -- 172.21.15.112:0/962029793 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fbf7406e530 msgr2=0x7fbf740709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:19.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.112+0000 7fbf91f65640 1 --2- 172.21.15.112:0/962029793 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fbf7406e530 0x7fbf740709e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbf80007ac0 tx=0x7fbf80009da0 comp rx=0 tx=0).stop 2024-09-16T00:40:19.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.112+0000 7fbf91f65640 1 -- 172.21.15.112:0/962029793 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf8c0fd590 msgr2=0x7fbf8c113260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:19.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.112+0000 7fbf91f65640 1 --2- 172.21.15.112:0/962029793 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf8c0fd590 0x7fbf8c113260 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c00c790 tx=0x7fbf7c0043c0 comp rx=0 tx=0).stop 2024-09-16T00:40:19.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.113+0000 7fbf91f65640 1 -- 172.21.15.112:0/962029793 shutdown_connections 2024-09-16T00:40:19.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.113+0000 7fbf91f65640 1 --2- 172.21.15.112:0/962029793 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fbf7406e530 0x7fbf740709e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:19.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.113+0000 7fbf91f65640 1 --2- 172.21.15.112:0/962029793 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf8c0fdf30 0x7fbf8c1137a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:19.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.113+0000 7fbf91f65640 1 --2- 172.21.15.112:0/962029793 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf8c0fd590 0x7fbf8c113260 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:19.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.113+0000 7fbf91f65640 1 -- 172.21.15.112:0/962029793 >> 172.21.15.112:0/962029793 conn(0x7fbf8c0f7060 msgr2=0x7fbf8c0f8580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:19.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.113+0000 7fbf91f65640 1 -- 172.21.15.112:0/962029793 shutdown_connections 2024-09-16T00:40:19.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.113+0000 7fbf91f65640 1 -- 172.21.15.112:0/962029793 wait complete. 2024-09-16T00:40:19.122 INFO:teuthology.orchestra.run.smithi112.stdout:true 2024-09-16T00:40:19.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.192+0000 7eff36a55640 1 -- 172.21.15.112:0/1539424544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff300fb490 msgr2=0x7eff300fb900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:19.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.192+0000 7eff36a55640 1 --2- 172.21.15.112:0/1539424544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff300fb490 0x7eff300fb900 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7eff240048d0 tx=0x7eff24009da0 comp rx=0 tx=0).stop 2024-09-16T00:40:19.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.192+0000 7eff36a55640 1 -- 172.21.15.112:0/1539424544 shutdown_connections 2024-09-16T00:40:19.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.192+0000 7eff36a55640 1 --2- 172.21.15.112:0/1539424544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff300fb490 0x7eff300fb900 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:19.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.192+0000 7eff36a55640 1 --2- 172.21.15.112:0/1539424544 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff301014d0 0x7eff301018a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:19.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.192+0000 7eff36a55640 1 -- 172.21.15.112:0/1539424544 >> 172.21.15.112:0/1539424544 conn(0x7eff300f7010 msgr2=0x7eff300f9420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:19.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.192+0000 7eff36a55640 1 -- 172.21.15.112:0/1539424544 shutdown_connections 2024-09-16T00:40:19.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.192+0000 7eff36a55640 1 -- 172.21.15.112:0/1539424544 wait complete. 2024-09-16T00:40:19.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.193+0000 7eff36a55640 1 Processor -- start 2024-09-16T00:40:19.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.193+0000 7eff36a55640 1 -- start start 2024-09-16T00:40:19.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.193+0000 7eff36a55640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff300fb490 0x7eff3010e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:19.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.193+0000 7eff36a55640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff301014d0 0x7eff3010e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:19.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.193+0000 7eff36a55640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff3010ed30 con 0x7eff301014d0 2024-09-16T00:40:19.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.193+0000 7eff36a55640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff3010eea0 con 0x7eff300fb490 2024-09-16T00:40:19.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.194+0000 7eff2f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff301014d0 0x7eff3010e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:19.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.194+0000 7eff2f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff301014d0 0x7eff3010e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49070/0 (socket says 172.21.15.112:49070) 2024-09-16T00:40:19.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.194+0000 7eff2f7fe640 1 -- 172.21.15.112:0/542032520 learned_addr learned my addr 172.21.15.112:0/542032520 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:40:19.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.194+0000 7eff2ffff640 1 --2- 172.21.15.112:0/542032520 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff300fb490 0x7eff3010e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:19.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.194+0000 7eff2f7fe640 1 -- 172.21.15.112:0/542032520 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff300fb490 msgr2=0x7eff3010e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:19.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.194+0000 7eff2f7fe640 1 --2- 172.21.15.112:0/542032520 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff300fb490 0x7eff3010e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:19.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.194+0000 7eff2f7fe640 1 -- 172.21.15.112:0/542032520 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff24005040 con 0x7eff301014d0 2024-09-16T00:40:19.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.194+0000 7eff2ffff640 1 --2- 172.21.15.112:0/542032520 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff300fb490 0x7eff3010e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:40:19.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.194+0000 7eff2f7fe640 1 --2- 172.21.15.112:0/542032520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff301014d0 0x7eff3010e6e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7eff24000c00 tx=0x7eff24009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:19.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.194+0000 7eff2cff9640 1 -- 172.21.15.112:0/542032520 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff24038070 con 0x7eff301014d0 2024-09-16T00:40:19.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.194+0000 7eff36a55640 1 -- 172.21.15.112:0/542032520 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff30117210 con 0x7eff301014d0 2024-09-16T00:40:19.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.194+0000 7eff2cff9640 1 -- 172.21.15.112:0/542032520 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7eff2402ade0 con 0x7eff301014d0 2024-09-16T00:40:19.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.194+0000 7eff2cff9640 1 -- 172.21.15.112:0/542032520 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff2402c7b0 con 0x7eff301014d0 2024-09-16T00:40:19.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.194+0000 7eff36a55640 1 -- 172.21.15.112:0/542032520 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff30117670 con 0x7eff301014d0 2024-09-16T00:40:19.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.195+0000 7eff36a55640 1 -- 172.21.15.112:0/542032520 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff300fcbd0 con 0x7eff301014d0 2024-09-16T00:40:19.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.196+0000 7eff2cff9640 1 -- 172.21.15.112:0/542032520 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7eff24030020 con 0x7eff301014d0 2024-09-16T00:40:19.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.196+0000 7eff2cff9640 1 --2- 172.21.15.112:0/542032520 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7eff1806e530 0x7eff180709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:19.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.196+0000 7eff2cff9640 1 -- 172.21.15.112:0/542032520 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7eff240ae650 con 0x7eff301014d0 2024-09-16T00:40:19.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.197+0000 7eff2ffff640 1 --2- 172.21.15.112:0/542032520 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7eff1806e530 0x7eff180709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:19.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.197+0000 7eff2ffff640 1 --2- 172.21.15.112:0/542032520 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7eff1806e530 0x7eff180709e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7eff20004fd0 tx=0x7eff2000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:19.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.201+0000 7eff2cff9640 1 -- 172.21.15.112:0/542032520 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7eff24077640 con 0x7eff301014d0 2024-09-16T00:40:19.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.384+0000 7eff36a55640 1 -- 172.21.15.112:0/542032520 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7eff3004b720 con 0x7eff1806e530 2024-09-16T00:40:19.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.385+0000 7eff2cff9640 1 -- 172.21.15.112:0/542032520 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7eff3004b720 con 0x7eff1806e530 2024-09-16T00:40:19.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.387+0000 7eff36a55640 1 -- 172.21.15.112:0/542032520 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7eff1806e530 msgr2=0x7eff180709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:19.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.387+0000 7eff36a55640 1 --2- 172.21.15.112:0/542032520 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7eff1806e530 0x7eff180709e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7eff20004fd0 tx=0x7eff2000b040 comp rx=0 tx=0).stop 2024-09-16T00:40:19.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.387+0000 7eff36a55640 1 -- 172.21.15.112:0/542032520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff301014d0 msgr2=0x7eff3010e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:19.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.387+0000 7eff36a55640 1 --2- 172.21.15.112:0/542032520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff301014d0 0x7eff3010e6e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7eff24000c00 tx=0x7eff24009da0 comp rx=0 tx=0).stop 2024-09-16T00:40:19.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.387+0000 7eff36a55640 1 -- 172.21.15.112:0/542032520 shutdown_connections 2024-09-16T00:40:19.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.387+0000 7eff36a55640 1 --2- 172.21.15.112:0/542032520 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7eff1806e530 0x7eff180709e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:19.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.387+0000 7eff36a55640 1 --2- 172.21.15.112:0/542032520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff301014d0 0x7eff3010e6e0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:19.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.387+0000 7eff36a55640 1 --2- 172.21.15.112:0/542032520 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff300fb490 0x7eff3010e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:19.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.387+0000 7eff36a55640 1 -- 172.21.15.112:0/542032520 >> 172.21.15.112:0/542032520 conn(0x7eff300f7010 msgr2=0x7eff300f8930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:19.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.387+0000 7eff36a55640 1 -- 172.21.15.112:0/542032520 shutdown_connections 2024-09-16T00:40:19.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.387+0000 7eff36a55640 1 -- 172.21.15.112:0/542032520 wait complete. 2024-09-16T00:40:19.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.465+0000 7f7f4a90d640 1 -- 172.21.15.112:0/2372121627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f44065c10 msgr2=0x7f7f440fe0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:19.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.465+0000 7f7f4a90d640 1 --2- 172.21.15.112:0/2372121627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f44065c10 0x7f7f440fe0d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7f380048d0 tx=0x7f7f38009da0 comp rx=0 tx=0).stop 2024-09-16T00:40:19.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.465+0000 7f7f4a90d640 1 -- 172.21.15.112:0/2372121627 shutdown_connections 2024-09-16T00:40:19.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.465+0000 7f7f4a90d640 1 --2- 172.21.15.112:0/2372121627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f44065c10 0x7f7f440fe0d0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:19.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.465+0000 7f7f4a90d640 1 --2- 172.21.15.112:0/2372121627 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f44065300 0x7f7f440656d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:19.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.465+0000 7f7f4a90d640 1 -- 172.21.15.112:0/2372121627 >> 172.21.15.112:0/2372121627 conn(0x7f7f440723b0 msgr2=0x7f7f440727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:19.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.466+0000 7f7f4a90d640 1 -- 172.21.15.112:0/2372121627 shutdown_connections 2024-09-16T00:40:19.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.466+0000 7f7f4a90d640 1 -- 172.21.15.112:0/2372121627 wait complete. 2024-09-16T00:40:19.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.466+0000 7f7f4a90d640 1 Processor -- start 2024-09-16T00:40:19.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.466+0000 7f7f4a90d640 1 -- start start 2024-09-16T00:40:19.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.467+0000 7f7f4a90d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f44065300 0x7f7f441131b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:19.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.467+0000 7f7f4a90d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f44065c10 0x7f7f441136f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:19.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.467+0000 7f7f4a90d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f44113cf0 con 0x7f7f44065c10 2024-09-16T00:40:19.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.467+0000 7f7f4a90d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f4410d130 con 0x7f7f44065300 2024-09-16T00:40:19.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.467+0000 7f7f437fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f44065c10 0x7f7f441136f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:19.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.467+0000 7f7f43fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f44065300 0x7f7f441131b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:19.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.467+0000 7f7f43fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f44065300 0x7f7f441131b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.112:50428/0 (socket says 172.21.15.112:50428) 2024-09-16T00:40:19.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.467+0000 7f7f43fff640 1 -- 172.21.15.112:0/4269896433 learned_addr learned my addr 172.21.15.112:0/4269896433 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:40:19.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.467+0000 7f7f437fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f44065c10 0x7f7f441136f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49074/0 (socket says 172.21.15.112:49074) 2024-09-16T00:40:19.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.467+0000 7f7f437fe640 1 -- 172.21.15.112:0/4269896433 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f44065300 msgr2=0x7f7f441131b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:19.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.467+0000 7f7f437fe640 1 --2- 172.21.15.112:0/4269896433 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f44065300 0x7f7f441131b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:19.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.467+0000 7f7f437fe640 1 -- 172.21.15.112:0/4269896433 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f38005040 con 0x7f7f44065c10 2024-09-16T00:40:19.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.467+0000 7f7f437fe640 1 --2- 172.21.15.112:0/4269896433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f44065c10 0x7f7f441136f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7f380043d0 tx=0x7f7f38009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:19.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.467+0000 7f7f40ff9640 1 -- 172.21.15.112:0/4269896433 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f38038070 con 0x7f7f44065c10 2024-09-16T00:40:19.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.468+0000 7f7f40ff9640 1 -- 172.21.15.112:0/4269896433 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f7f380096e0 con 0x7f7f44065c10 2024-09-16T00:40:19.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.468+0000 7f7f4a90d640 1 -- 172.21.15.112:0/4269896433 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f4410d3b0 con 0x7f7f44065c10 2024-09-16T00:40:19.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.468+0000 7f7f40ff9640 1 -- 172.21.15.112:0/4269896433 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f3802c870 con 0x7f7f44065c10 2024-09-16T00:40:19.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.468+0000 7f7f4a90d640 1 -- 172.21.15.112:0/4269896433 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f4410d8f0 con 0x7f7f44065c10 2024-09-16T00:40:19.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.469+0000 7f7f4a90d640 1 -- 172.21.15.112:0/4269896433 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f440fdce0 con 0x7f7f44065c10 2024-09-16T00:40:19.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.470+0000 7f7f40ff9640 1 -- 172.21.15.112:0/4269896433 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f7f38030020 con 0x7f7f44065c10 2024-09-16T00:40:19.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.470+0000 7f7f40ff9640 1 --2- 172.21.15.112:0/4269896433 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f7f2c06e530 0x7f7f2c0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:19.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.470+0000 7f7f40ff9640 1 -- 172.21.15.112:0/4269896433 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f7f380adf90 con 0x7f7f44065c10 2024-09-16T00:40:19.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.470+0000 7f7f43fff640 1 --2- 172.21.15.112:0/4269896433 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f7f2c06e530 0x7f7f2c0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:19.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.471+0000 7f7f43fff640 1 --2- 172.21.15.112:0/4269896433 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f7f2c06e530 0x7f7f2c0709e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7f340041d0 tx=0x7f7f3400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:19.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.475+0000 7f7f40ff9640 1 -- 172.21.15.112:0/4269896433 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f7f38076f90 con 0x7f7f44065c10 2024-09-16T00:40:19.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.649+0000 7f7f4a90d640 1 -- 172.21.15.112:0/4269896433 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7f44113e30 con 0x7f7f2c06e530 2024-09-16T00:40:19.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.657+0000 7f7f40ff9640 1 -- 172.21.15.112:0/4269896433 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7f7f3400f040 con 0x7f7f2c06e530 2024-09-16T00:40:19.658 INFO:teuthology.orchestra.run.smithi112.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T00:40:19.658 INFO:teuthology.orchestra.run.smithi112.stdout:alertmanager.smithi112 smithi112 *:9093,9094 running (13m) 6m ago 22m 24.7M - 0.25.0 c8568f914cd2 d85ea0562022 2024-09-16T00:40:19.658 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi112 smithi112 running (10m) 6m ago 22m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c e4fe58f9702a 2024-09-16T00:40:19.658 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi169 smithi169 running (10m) 5m ago 21m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c 3ba2c81d527f 2024-09-16T00:40:19.658 INFO:teuthology.orchestra.run.smithi112.stdout:grafana.smithi112 smithi112 *:3000 running (12m) 6m ago 21m 82.1M - 9.4.12 2bacad6d85d8 51f9d2013da2 2024-09-16T00:40:19.658 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.hxfwkh smithi112 running (6m) 6m ago 16m 14.2M - 19.1.1-326-g8921c8ef f1455bd0219c 1723069902d6 2024-09-16T00:40:19.658 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.ithwzj smithi112 running (6m) 6m ago 16m 12.2M - 19.1.1-326-g8921c8ef f1455bd0219c e7aad5b81db5 2024-09-16T00:40:19.658 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.gawwgw smithi169 running (6m) 5m ago 16m 15.7M - 19.1.1-326-g8921c8ef f1455bd0219c 469be459ee12 2024-09-16T00:40:19.659 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.lgeqrj smithi169 running (5m) 5m ago 16m 13.3M - 19.1.1-326-g8921c8ef f1455bd0219c 31f0fbd2395d 2024-09-16T00:40:19.659 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi112.xrkkbc smithi112 *:8443,9283,8765 running (12m) 6m ago 23m 504M - 19.1.1-326-g8921c8ef f1455bd0219c a981f7fd2810 2024-09-16T00:40:19.659 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi169.mtrobk smithi169 *:8443,9283,8765 running (12m) 5m ago 21m 442M - 19.1.1-326-g8921c8ef f1455bd0219c 4096e5369c2d 2024-09-16T00:40:19.659 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi112 smithi112 running (11m) 6m ago 23m 49.8M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 5aaec033c042 2024-09-16T00:40:19.659 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi169 smithi169 running (10m) 5m ago 21m 38.9M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 7bc609155f3f 2024-09-16T00:40:19.659 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi112 smithi112 *:9100 running (13m) 6m ago 22m 15.5M - 1.5.0 0da6a335fe13 6404d1d601e2 2024-09-16T00:40:19.659 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi169 smithi169 *:9100 running (12m) 5m ago 21m 15.4M - 1.5.0 0da6a335fe13 5505baf702a6 2024-09-16T00:40:19.659 INFO:teuthology.orchestra.run.smithi112.stdout:osd.0 smithi112 running (9m) 6m ago 20m 105M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 53efc9397017 2024-09-16T00:40:19.659 INFO:teuthology.orchestra.run.smithi112.stdout:osd.1 smithi112 running (9m) 6m ago 20m 98.9M 1557M 19.1.1-326-g8921c8ef f1455bd0219c fe6f1a4a2cf4 2024-09-16T00:40:19.659 INFO:teuthology.orchestra.run.smithi112.stdout:osd.2 smithi112 running (8m) 6m ago 19m 102M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 5f7863712057 2024-09-16T00:40:19.659 INFO:teuthology.orchestra.run.smithi112.stdout:osd.3 smithi169 running (8m) 5m ago 19m 92.1M 2581M 19.1.1-326-g8921c8ef f1455bd0219c ee07b5577810 2024-09-16T00:40:19.659 INFO:teuthology.orchestra.run.smithi112.stdout:osd.4 smithi169 running (7m) 5m ago 18m 103M 2581M 19.1.1-326-g8921c8ef f1455bd0219c 769f026a54eb 2024-09-16T00:40:19.659 INFO:teuthology.orchestra.run.smithi112.stdout:osd.5 smithi169 running (6m) 5m ago 18m 125M 2581M 19.1.1-326-g8921c8ef f1455bd0219c a6007c13cf78 2024-09-16T00:40:19.660 INFO:teuthology.orchestra.run.smithi112.stdout:prometheus.smithi112 smithi112 *:9095 running (12m) 6m ago 21m 64.1M - 2.43.0 a07b618ecd1d 7a1d42e60d2b 2024-09-16T00:40:19.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.660+0000 7f7f4a90d640 1 -- 172.21.15.112:0/4269896433 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f7f2c06e530 msgr2=0x7f7f2c0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:19.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.660+0000 7f7f4a90d640 1 --2- 172.21.15.112:0/4269896433 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f7f2c06e530 0x7f7f2c0709e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7f340041d0 tx=0x7f7f3400b040 comp rx=0 tx=0).stop 2024-09-16T00:40:19.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.660+0000 7f7f4a90d640 1 -- 172.21.15.112:0/4269896433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f44065c10 msgr2=0x7f7f441136f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:19.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.660+0000 7f7f4a90d640 1 --2- 172.21.15.112:0/4269896433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f44065c10 0x7f7f441136f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7f380043d0 tx=0x7f7f38009da0 comp rx=0 tx=0).stop 2024-09-16T00:40:19.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.660+0000 7f7f4a90d640 1 -- 172.21.15.112:0/4269896433 shutdown_connections 2024-09-16T00:40:19.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.660+0000 7f7f4a90d640 1 --2- 172.21.15.112:0/4269896433 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f7f2c06e530 0x7f7f2c0709e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:19.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.660+0000 7f7f4a90d640 1 --2- 172.21.15.112:0/4269896433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f44065c10 0x7f7f441136f0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:19.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.660+0000 7f7f4a90d640 1 --2- 172.21.15.112:0/4269896433 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f44065300 0x7f7f441131b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:19.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.660+0000 7f7f4a90d640 1 -- 172.21.15.112:0/4269896433 >> 172.21.15.112:0/4269896433 conn(0x7f7f440723b0 msgr2=0x7f7f440fa2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:19.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.660+0000 7f7f4a90d640 1 -- 172.21.15.112:0/4269896433 shutdown_connections 2024-09-16T00:40:19.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.660+0000 7f7f4a90d640 1 -- 172.21.15.112:0/4269896433 wait complete. 2024-09-16T00:40:19.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.737+0000 7f04f2c67640 1 -- 172.21.15.112:0/1649654937 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04ec0fb480 msgr2=0x7f04ec0fb8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:19.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.737+0000 7f04f2c67640 1 --2- 172.21.15.112:0/1649654937 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04ec0fb480 0x7f04ec0fb8f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f04d40048d0 tx=0x7f04d4009da0 comp rx=0 tx=0).stop 2024-09-16T00:40:19.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.737+0000 7f04f2c67640 1 -- 172.21.15.112:0/1649654937 shutdown_connections 2024-09-16T00:40:19.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.737+0000 7f04f2c67640 1 --2- 172.21.15.112:0/1649654937 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04ec0fb480 0x7f04ec0fb8f0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:19.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.737+0000 7f04f2c67640 1 --2- 172.21.15.112:0/1649654937 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04ec1014c0 0x7f04ec101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:19.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.737+0000 7f04f2c67640 1 -- 172.21.15.112:0/1649654937 >> 172.21.15.112:0/1649654937 conn(0x7f04ec0f7000 msgr2=0x7f04ec0f9410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:19.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.737+0000 7f04f2c67640 1 -- 172.21.15.112:0/1649654937 shutdown_connections 2024-09-16T00:40:19.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.737+0000 7f04f2c67640 1 -- 172.21.15.112:0/1649654937 wait complete. 2024-09-16T00:40:19.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.737+0000 7f04f2c67640 1 Processor -- start 2024-09-16T00:40:19.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.737+0000 7f04f2c67640 1 -- start start 2024-09-16T00:40:19.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.738+0000 7f04f2c67640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04ec0fb480 0x7f04ec1153b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:19.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.738+0000 7f04f2c67640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04ec1014c0 0x7f04ec1158f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:19.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.738+0000 7f04f2c67640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04ec115ef0 con 0x7f04ec1014c0 2024-09-16T00:40:19.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.738+0000 7f04f2c67640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04ec116060 con 0x7f04ec0fb480 2024-09-16T00:40:19.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.738+0000 7f04ebfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04ec1014c0 0x7f04ec1158f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:19.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.738+0000 7f04ebfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04ec1014c0 0x7f04ec1158f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49098/0 (socket says 172.21.15.112:49098) 2024-09-16T00:40:19.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.738+0000 7f04ebfff640 1 -- 172.21.15.112:0/3492794150 learned_addr learned my addr 172.21.15.112:0/3492794150 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:40:19.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.738+0000 7f04f0a03640 1 --2- 172.21.15.112:0/3492794150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04ec0fb480 0x7f04ec1153b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:19.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.738+0000 7f04ebfff640 1 -- 172.21.15.112:0/3492794150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04ec0fb480 msgr2=0x7f04ec1153b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:19.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.738+0000 7f04ebfff640 1 --2- 172.21.15.112:0/3492794150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04ec0fb480 0x7f04ec1153b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:19.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.738+0000 7f04ebfff640 1 -- 172.21.15.112:0/3492794150 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04d4005040 con 0x7f04ec1014c0 2024-09-16T00:40:19.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.738+0000 7f04ebfff640 1 --2- 172.21.15.112:0/3492794150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04ec1014c0 0x7f04ec1158f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f04d4003080 tx=0x7f04d4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:19.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.739+0000 7f04e97fa640 1 -- 172.21.15.112:0/3492794150 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f04d4038070 con 0x7f04ec1014c0 2024-09-16T00:40:19.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.739+0000 7f04f2c67640 1 -- 172.21.15.112:0/3492794150 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04ec10f480 con 0x7f04ec1014c0 2024-09-16T00:40:19.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.739+0000 7f04e97fa640 1 -- 172.21.15.112:0/3492794150 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f04d40097e0 con 0x7f04ec1014c0 2024-09-16T00:40:19.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.739+0000 7f04e97fa640 1 -- 172.21.15.112:0/3492794150 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f04d403d420 con 0x7f04ec1014c0 2024-09-16T00:40:19.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.739+0000 7f04f2c67640 1 -- 172.21.15.112:0/3492794150 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04ec10f9c0 con 0x7f04ec1014c0 2024-09-16T00:40:19.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.740+0000 7f04f2c67640 1 -- 172.21.15.112:0/3492794150 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04ec0fcbc0 con 0x7f04ec1014c0 2024-09-16T00:40:19.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.740+0000 7f04e97fa640 1 -- 172.21.15.112:0/3492794150 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f04d4030020 con 0x7f04ec1014c0 2024-09-16T00:40:19.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.741+0000 7f04e97fa640 1 --2- 172.21.15.112:0/3492794150 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f04d806e530 0x7f04d80709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:19.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.741+0000 7f04e97fa640 1 -- 172.21.15.112:0/3492794150 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f04d40ae760 con 0x7f04ec1014c0 2024-09-16T00:40:19.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.741+0000 7f04f0a03640 1 --2- 172.21.15.112:0/3492794150 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f04d806e530 0x7f04d80709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:19.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.741+0000 7f04f0a03640 1 --2- 172.21.15.112:0/3492794150 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f04d806e530 0x7f04d80709e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f04dc007fd0 tx=0x7f04dc009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:19.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.746+0000 7f04e97fa640 1 -- 172.21.15.112:0/3492794150 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f04d4077750 con 0x7f04ec1014c0 2024-09-16T00:40:19.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.979+0000 7f04f2c67640 1 -- 172.21.15.112:0/3492794150 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f04ec1084a0 con 0x7f04ec1014c0 2024-09-16T00:40:19.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.979+0000 7f04e97fa640 1 -- 172.21.15.112:0/3492794150 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f04d407ae10 con 0x7f04ec1014c0 2024-09-16T00:40:19.980 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:40:19.980 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": { 2024-09-16T00:40:19.980 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:40:19.980 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:40:19.980 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": { 2024-09-16T00:40:19.980 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:40:19.980 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:40:19.980 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": { 2024-09-16T00:40:19.980 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 6 2024-09-16T00:40:19.980 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:40:19.981 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": { 2024-09-16T00:40:19.981 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 4 2024-09-16T00:40:19.981 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:40:19.981 INFO:teuthology.orchestra.run.smithi112.stdout: "overall": { 2024-09-16T00:40:19.981 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 14 2024-09-16T00:40:19.981 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-16T00:40:19.981 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:40:19.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.982+0000 7f04f2c67640 1 -- 172.21.15.112:0/3492794150 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f04d806e530 msgr2=0x7f04d80709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:19.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.982+0000 7f04f2c67640 1 --2- 172.21.15.112:0/3492794150 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f04d806e530 0x7f04d80709e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f04dc007fd0 tx=0x7f04dc009040 comp rx=0 tx=0).stop 2024-09-16T00:40:19.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.982+0000 7f04f2c67640 1 -- 172.21.15.112:0/3492794150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04ec1014c0 msgr2=0x7f04ec1158f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:19.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.982+0000 7f04f2c67640 1 --2- 172.21.15.112:0/3492794150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04ec1014c0 0x7f04ec1158f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f04d4003080 tx=0x7f04d4009da0 comp rx=0 tx=0).stop 2024-09-16T00:40:19.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.982+0000 7f04f2c67640 1 -- 172.21.15.112:0/3492794150 shutdown_connections 2024-09-16T00:40:19.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.982+0000 7f04f2c67640 1 --2- 172.21.15.112:0/3492794150 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f04d806e530 0x7f04d80709e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:19.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.982+0000 7f04f2c67640 1 --2- 172.21.15.112:0/3492794150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04ec1014c0 0x7f04ec1158f0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:19.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.982+0000 7f04f2c67640 1 --2- 172.21.15.112:0/3492794150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04ec0fb480 0x7f04ec1153b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:19.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.982+0000 7f04f2c67640 1 -- 172.21.15.112:0/3492794150 >> 172.21.15.112:0/3492794150 conn(0x7f04ec0f7000 msgr2=0x7f04ec0f73d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:19.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.982+0000 7f04f2c67640 1 -- 172.21.15.112:0/3492794150 shutdown_connections 2024-09-16T00:40:19.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:19.982+0000 7f04f2c67640 1 -- 172.21.15.112:0/3492794150 wait complete. 2024-09-16T00:40:20.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:40:19 smithi169 ceph-mon[77626]: pgmap v377: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:40:20.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.061+0000 7f470765d640 1 -- 172.21.15.112:0/1212800210 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4700102550 msgr2=0x7f4700102920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:20.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.061+0000 7f470765d640 1 --2- 172.21.15.112:0/1212800210 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4700102550 0x7f4700102920 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f46fc0048d0 tx=0x7f46fc009da0 comp rx=0 tx=0).stop 2024-09-16T00:40:20.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.061+0000 7f470765d640 1 -- 172.21.15.112:0/1212800210 shutdown_connections 2024-09-16T00:40:20.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.061+0000 7f470765d640 1 --2- 172.21.15.112:0/1212800210 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4700102ef0 0x7f47000fc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:20.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.061+0000 7f470765d640 1 --2- 172.21.15.112:0/1212800210 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4700102550 0x7f4700102920 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:20.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.061+0000 7f470765d640 1 -- 172.21.15.112:0/1212800210 >> 172.21.15.112:0/1212800210 conn(0x7f47000f80f0 msgr2=0x7f47000fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:20.063 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:40:19 smithi112 ceph-mon[97750]: pgmap v377: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:40:20.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.062+0000 7f470765d640 1 -- 172.21.15.112:0/1212800210 shutdown_connections 2024-09-16T00:40:20.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.062+0000 7f470765d640 1 -- 172.21.15.112:0/1212800210 wait complete. 2024-09-16T00:40:20.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.062+0000 7f470765d640 1 Processor -- start 2024-09-16T00:40:20.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.062+0000 7f470765d640 1 -- start start 2024-09-16T00:40:20.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.062+0000 7f470765d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4700102550 0x7f4700113520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:20.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.062+0000 7f470765d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4700102ef0 0x7f4700113a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:20.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.062+0000 7f470765d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47001140b0 con 0x7f4700102ef0 2024-09-16T00:40:20.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.062+0000 7f470765d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f470010d4c0 con 0x7f4700102550 2024-09-16T00:40:20.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.063+0000 7f4704bf8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4700102ef0 0x7f4700113a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:20.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.063+0000 7f4704bf8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4700102ef0 0x7f4700113a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49120/0 (socket says 172.21.15.112:49120) 2024-09-16T00:40:20.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.063+0000 7f4704bf8640 1 -- 172.21.15.112:0/1984367921 learned_addr learned my addr 172.21.15.112:0/1984367921 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:40:20.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.063+0000 7f47053f9640 1 --2- 172.21.15.112:0/1984367921 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4700102550 0x7f4700113520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:20.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.063+0000 7f4704bf8640 1 -- 172.21.15.112:0/1984367921 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4700102550 msgr2=0x7f4700113520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:20.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.063+0000 7f4704bf8640 1 --2- 172.21.15.112:0/1984367921 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4700102550 0x7f4700113520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:20.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.063+0000 7f4704bf8640 1 -- 172.21.15.112:0/1984367921 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46fc005040 con 0x7f4700102ef0 2024-09-16T00:40:20.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.063+0000 7f4704bf8640 1 --2- 172.21.15.112:0/1984367921 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4700102ef0 0x7f4700113a60 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f46f000c740 tx=0x7f46f00043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:20.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.063+0000 7f46f5ffb640 1 -- 172.21.15.112:0/1984367921 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46f0012810 con 0x7f4700102ef0 2024-09-16T00:40:20.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.063+0000 7f46f5ffb640 1 -- 172.21.15.112:0/1984367921 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f46f0012e10 con 0x7f4700102ef0 2024-09-16T00:40:20.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.063+0000 7f470765d640 1 -- 172.21.15.112:0/1984367921 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f470010d7a0 con 0x7f4700102ef0 2024-09-16T00:40:20.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.064+0000 7f470765d640 1 -- 172.21.15.112:0/1984367921 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f470010dd40 con 0x7f4700102ef0 2024-09-16T00:40:20.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.064+0000 7f46f5ffb640 1 -- 172.21.15.112:0/1984367921 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46f000a2f0 con 0x7f4700102ef0 2024-09-16T00:40:20.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.065+0000 7f46f5ffb640 1 -- 172.21.15.112:0/1984367921 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f46f0011380 con 0x7f4700102ef0 2024-09-16T00:40:20.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.065+0000 7f470765d640 1 -- 172.21.15.112:0/1984367921 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47000fdcd0 con 0x7f4700102ef0 2024-09-16T00:40:20.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.065+0000 7f46f5ffb640 1 --2- 172.21.15.112:0/1984367921 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f46ec06e530 0x7f46ec0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:20.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.065+0000 7f46f5ffb640 1 -- 172.21.15.112:0/1984367921 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f46f008e720 con 0x7f4700102ef0 2024-09-16T00:40:20.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.065+0000 7f47053f9640 1 --2- 172.21.15.112:0/1984367921 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f46ec06e530 0x7f46ec0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:20.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.066+0000 7f47053f9640 1 --2- 172.21.15.112:0/1984367921 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f46ec06e530 0x7f46ec0709e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f46fc009ac0 tx=0x7f46fc035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:20.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.070+0000 7f46f5ffb640 1 -- 172.21.15.112:0/1984367921 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f46f0057710 con 0x7f4700102ef0 2024-09-16T00:40:20.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.271+0000 7f470765d640 1 -- 172.21.15.112:0/1984367921 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f47000fbd50 con 0x7f4700102ef0 2024-09-16T00:40:20.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.272+0000 7f46f5ffb640 1 -- 172.21.15.112:0/1984367921 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1914 (secure 0 0 0) 0x7f46f005add0 con 0x7f4700102ef0 2024-09-16T00:40:20.272 INFO:teuthology.orchestra.run.smithi112.stdout:e34 2024-09-16T00:40:20.273 INFO:teuthology.orchestra.run.smithi112.stdout:btime 2024-09-16T00:34:26:619254+0000 2024-09-16T00:40:20.273 INFO:teuthology.orchestra.run.smithi112.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T00:40:20.273 INFO:teuthology.orchestra.run.smithi112.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:40:20.273 INFO:teuthology.orchestra.run.smithi112.stdout:legacy client fscid: 1 2024-09-16T00:40:20.273 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:40:20.273 INFO:teuthology.orchestra.run.smithi112.stdout:Filesystem 'cephfs' (1) 2024-09-16T00:40:20.273 INFO:teuthology.orchestra.run.smithi112.stdout:fs_name cephfs 2024-09-16T00:40:20.273 INFO:teuthology.orchestra.run.smithi112.stdout:epoch 25 2024-09-16T00:40:20.273 INFO:teuthology.orchestra.run.smithi112.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-16T00:40:20.273 INFO:teuthology.orchestra.run.smithi112.stdout:created 2024-09-16T00:23:22.328199+0000 2024-09-16T00:40:20.273 INFO:teuthology.orchestra.run.smithi112.stdout:modified 2024-09-16T00:33:29.012991+0000 2024-09-16T00:40:20.274 INFO:teuthology.orchestra.run.smithi112.stdout:tableserver 0 2024-09-16T00:40:20.274 INFO:teuthology.orchestra.run.smithi112.stdout:root 0 2024-09-16T00:40:20.274 INFO:teuthology.orchestra.run.smithi112.stdout:session_timeout 60 2024-09-16T00:40:20.274 INFO:teuthology.orchestra.run.smithi112.stdout:session_autoclose 300 2024-09-16T00:40:20.274 INFO:teuthology.orchestra.run.smithi112.stdout:max_file_size 1099511627776 2024-09-16T00:40:20.274 INFO:teuthology.orchestra.run.smithi112.stdout:max_xattr_size 65536 2024-09-16T00:40:20.274 INFO:teuthology.orchestra.run.smithi112.stdout:required_client_features {} 2024-09-16T00:40:20.274 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure 0 2024-09-16T00:40:20.274 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure_osd_epoch 82 2024-09-16T00:40:20.274 INFO:teuthology.orchestra.run.smithi112.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:40:20.274 INFO:teuthology.orchestra.run.smithi112.stdout:max_mds 2 2024-09-16T00:40:20.274 INFO:teuthology.orchestra.run.smithi112.stdout:in 0,1 2024-09-16T00:40:20.275 INFO:teuthology.orchestra.run.smithi112.stdout:up {} 2024-09-16T00:40:20.275 INFO:teuthology.orchestra.run.smithi112.stdout:failed 0,1 2024-09-16T00:40:20.275 INFO:teuthology.orchestra.run.smithi112.stdout:damaged 2024-09-16T00:40:20.275 INFO:teuthology.orchestra.run.smithi112.stdout:stopped 2024-09-16T00:40:20.275 INFO:teuthology.orchestra.run.smithi112.stdout:data_pools [3] 2024-09-16T00:40:20.275 INFO:teuthology.orchestra.run.smithi112.stdout:metadata_pool 2 2024-09-16T00:40:20.275 INFO:teuthology.orchestra.run.smithi112.stdout:inline_data enabled 2024-09-16T00:40:20.275 INFO:teuthology.orchestra.run.smithi112.stdout:balancer 2024-09-16T00:40:20.275 INFO:teuthology.orchestra.run.smithi112.stdout:bal_rank_mask -1 2024-09-16T00:40:20.275 INFO:teuthology.orchestra.run.smithi112.stdout:standby_count_wanted 1 2024-09-16T00:40:20.275 INFO:teuthology.orchestra.run.smithi112.stdout:qdb_cluster leader: 0 members: 2024-09-16T00:40:20.275 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:40:20.275 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:40:20.275 INFO:teuthology.orchestra.run.smithi112.stdout:Standby daemons: 2024-09-16T00:40:20.276 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:40:20.276 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.ithwzj{-1:34392} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.112:6826/476229221,v1:172.21.15.112:6827/476229221] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:40:20.276 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.hxfwkh{-1:34404} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.112:6828/2356899876,v1:172.21.15.112:6829/2356899876] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:40:20.276 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.gawwgw{-1:44343} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.169:6824/1635329957,v1:172.21.15.169:6825/1635329957] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:40:20.276 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.lgeqrj{-1:44347} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.169:6826/184810031,v1:172.21.15.169:6827/184810031] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:40:20.277 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 34 2024-09-16T00:40:20.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.274+0000 7f470765d640 1 -- 172.21.15.112:0/1984367921 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f46ec06e530 msgr2=0x7f46ec0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:20.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.274+0000 7f470765d640 1 --2- 172.21.15.112:0/1984367921 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f46ec06e530 0x7f46ec0709e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f46fc009ac0 tx=0x7f46fc035040 comp rx=0 tx=0).stop 2024-09-16T00:40:20.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.274+0000 7f470765d640 1 -- 172.21.15.112:0/1984367921 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4700102ef0 msgr2=0x7f4700113a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:20.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.274+0000 7f470765d640 1 --2- 172.21.15.112:0/1984367921 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4700102ef0 0x7f4700113a60 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f46f000c740 tx=0x7f46f00043c0 comp rx=0 tx=0).stop 2024-09-16T00:40:20.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.274+0000 7f470765d640 1 -- 172.21.15.112:0/1984367921 shutdown_connections 2024-09-16T00:40:20.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.274+0000 7f470765d640 1 --2- 172.21.15.112:0/1984367921 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f46ec06e530 0x7f46ec0709e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:20.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.274+0000 7f470765d640 1 --2- 172.21.15.112:0/1984367921 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4700102ef0 0x7f4700113a60 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:20.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.274+0000 7f470765d640 1 --2- 172.21.15.112:0/1984367921 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4700102550 0x7f4700113520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:20.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.274+0000 7f470765d640 1 -- 172.21.15.112:0/1984367921 >> 172.21.15.112:0/1984367921 conn(0x7f47000f80f0 msgr2=0x7f47000f9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:20.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.275+0000 7f470765d640 1 -- 172.21.15.112:0/1984367921 shutdown_connections 2024-09-16T00:40:20.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.275+0000 7f470765d640 1 -- 172.21.15.112:0/1984367921 wait complete. 2024-09-16T00:40:20.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.355+0000 7f15515a1640 1 -- 172.21.15.112:0/1180546951 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f154c102380 msgr2=0x7f154c102750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:20.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.355+0000 7f15515a1640 1 --2- 172.21.15.112:0/1180546951 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f154c102380 0x7f154c102750 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f153c0048d0 tx=0x7f153c009da0 comp rx=0 tx=0).stop 2024-09-16T00:40:20.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.355+0000 7f15515a1640 1 -- 172.21.15.112:0/1180546951 shutdown_connections 2024-09-16T00:40:20.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.355+0000 7f15515a1640 1 --2- 172.21.15.112:0/1180546951 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f154c102d20 0x7f154c0fc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:20.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.355+0000 7f15515a1640 1 --2- 172.21.15.112:0/1180546951 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f154c102380 0x7f154c102750 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:20.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.355+0000 7f15515a1640 1 -- 172.21.15.112:0/1180546951 >> 172.21.15.112:0/1180546951 conn(0x7f154c0f7f30 msgr2=0x7f154c0fa340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:20.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.355+0000 7f15515a1640 1 -- 172.21.15.112:0/1180546951 shutdown_connections 2024-09-16T00:40:20.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.355+0000 7f15515a1640 1 -- 172.21.15.112:0/1180546951 wait complete. 2024-09-16T00:40:20.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.356+0000 7f15515a1640 1 Processor -- start 2024-09-16T00:40:20.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.356+0000 7f15515a1640 1 -- start start 2024-09-16T00:40:20.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.356+0000 7f15515a1640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f154c102380 0x7f154c115230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:20.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.356+0000 7f15515a1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f154c102d20 0x7f154c115770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:20.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.356+0000 7f15515a1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f154c115d70 con 0x7f154c102d20 2024-09-16T00:40:20.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.356+0000 7f15515a1640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f154c10f1b0 con 0x7f154c102380 2024-09-16T00:40:20.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.356+0000 7f154b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f154c102d20 0x7f154c115770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:20.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.356+0000 7f154b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f154c102d20 0x7f154c115770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49142/0 (socket says 172.21.15.112:49142) 2024-09-16T00:40:20.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.356+0000 7f154b7fe640 1 -- 172.21.15.112:0/3646952507 learned_addr learned my addr 172.21.15.112:0/3646952507 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:40:20.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.356+0000 7f154bfff640 1 --2- 172.21.15.112:0/3646952507 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f154c102380 0x7f154c115230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:20.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.356+0000 7f154b7fe640 1 -- 172.21.15.112:0/3646952507 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f154c102380 msgr2=0x7f154c115230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:20.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.356+0000 7f154b7fe640 1 --2- 172.21.15.112:0/3646952507 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f154c102380 0x7f154c115230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:20.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.356+0000 7f154b7fe640 1 -- 172.21.15.112:0/3646952507 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f153c005040 con 0x7f154c102d20 2024-09-16T00:40:20.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.357+0000 7f154b7fe640 1 --2- 172.21.15.112:0/3646952507 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f154c102d20 0x7f154c115770 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f154000eb70 tx=0x7f154000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:20.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.357+0000 7f1548ff9640 1 -- 172.21.15.112:0/3646952507 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1540007c30 con 0x7f154c102d20 2024-09-16T00:40:20.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.357+0000 7f1548ff9640 1 -- 172.21.15.112:0/3646952507 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f154000f040 con 0x7f154c102d20 2024-09-16T00:40:20.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.357+0000 7f15515a1640 1 -- 172.21.15.112:0/3646952507 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f154c10f430 con 0x7f154c102d20 2024-09-16T00:40:20.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.357+0000 7f1548ff9640 1 -- 172.21.15.112:0/3646952507 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f154000ca60 con 0x7f154c102d20 2024-09-16T00:40:20.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.357+0000 7f15515a1640 1 -- 172.21.15.112:0/3646952507 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f154c10f9d0 con 0x7f154c102d20 2024-09-16T00:40:20.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.358+0000 7f15515a1640 1 -- 172.21.15.112:0/3646952507 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f154c0fdaf0 con 0x7f154c102d20 2024-09-16T00:40:20.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.358+0000 7f1548ff9640 1 -- 172.21.15.112:0/3646952507 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f154000cbc0 con 0x7f154c102d20 2024-09-16T00:40:20.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.359+0000 7f1548ff9640 1 --2- 172.21.15.112:0/3646952507 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f153406e530 0x7f15340709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:20.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.359+0000 7f154bfff640 1 --2- 172.21.15.112:0/3646952507 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f153406e530 0x7f15340709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:20.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.359+0000 7f1548ff9640 1 -- 172.21.15.112:0/3646952507 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f154008df30 con 0x7f154c102d20 2024-09-16T00:40:20.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.360+0000 7f154bfff640 1 --2- 172.21.15.112:0/3646952507 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f153406e530 0x7f15340709e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f153c007ac0 tx=0x7f153c035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:20.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.364+0000 7f1548ff9640 1 -- 172.21.15.112:0/3646952507 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f15400c31b0 con 0x7f154c102d20 2024-09-16T00:40:20.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.543+0000 7f15515a1640 1 -- 172.21.15.112:0/3646952507 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f154c109390 con 0x7f153406e530 2024-09-16T00:40:20.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.545+0000 7f1548ff9640 1 -- 172.21.15.112:0/3646952507 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f154c109390 con 0x7f153406e530 2024-09-16T00:40:20.545 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:40:20.545 INFO:teuthology.orchestra.run.smithi112.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80", 2024-09-16T00:40:20.545 INFO:teuthology.orchestra.run.smithi112.stdout: "in_progress": true, 2024-09-16T00:40:20.545 INFO:teuthology.orchestra.run.smithi112.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T00:40:20.545 INFO:teuthology.orchestra.run.smithi112.stdout: "services_complete": [ 2024-09-16T00:40:20.545 INFO:teuthology.orchestra.run.smithi112.stdout: "mds", 2024-09-16T00:40:20.545 INFO:teuthology.orchestra.run.smithi112.stdout: "osd", 2024-09-16T00:40:20.545 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr", 2024-09-16T00:40:20.545 INFO:teuthology.orchestra.run.smithi112.stdout: "mon", 2024-09-16T00:40:20.545 INFO:teuthology.orchestra.run.smithi112.stdout: "crash" 2024-09-16T00:40:20.545 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-16T00:40:20.546 INFO:teuthology.orchestra.run.smithi112.stdout: "progress": "16/21 daemons upgraded", 2024-09-16T00:40:20.546 INFO:teuthology.orchestra.run.smithi112.stdout: "message": "Currently upgrading mds daemons", 2024-09-16T00:40:20.546 INFO:teuthology.orchestra.run.smithi112.stdout: "is_paused": false 2024-09-16T00:40:20.546 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:40:20.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.547+0000 7f15515a1640 1 -- 172.21.15.112:0/3646952507 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f153406e530 msgr2=0x7f15340709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:20.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.547+0000 7f15515a1640 1 --2- 172.21.15.112:0/3646952507 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f153406e530 0x7f15340709e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f153c007ac0 tx=0x7f153c035040 comp rx=0 tx=0).stop 2024-09-16T00:40:20.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.547+0000 7f15515a1640 1 -- 172.21.15.112:0/3646952507 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f154c102d20 msgr2=0x7f154c115770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:20.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.547+0000 7f15515a1640 1 --2- 172.21.15.112:0/3646952507 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f154c102d20 0x7f154c115770 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f154000eb70 tx=0x7f154000b040 comp rx=0 tx=0).stop 2024-09-16T00:40:20.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.547+0000 7f15515a1640 1 -- 172.21.15.112:0/3646952507 shutdown_connections 2024-09-16T00:40:20.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.547+0000 7f15515a1640 1 --2- 172.21.15.112:0/3646952507 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f153406e530 0x7f15340709e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:20.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.547+0000 7f15515a1640 1 --2- 172.21.15.112:0/3646952507 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f154c102d20 0x7f154c115770 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:20.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.547+0000 7f15515a1640 1 --2- 172.21.15.112:0/3646952507 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f154c102380 0x7f154c115230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:20.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.547+0000 7f15515a1640 1 -- 172.21.15.112:0/3646952507 >> 172.21.15.112:0/3646952507 conn(0x7f154c0f7f30 msgr2=0x7f154c0f8ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:20.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.547+0000 7f15515a1640 1 -- 172.21.15.112:0/3646952507 shutdown_connections 2024-09-16T00:40:20.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.547+0000 7f15515a1640 1 -- 172.21.15.112:0/3646952507 wait complete. 2024-09-16T00:40:20.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.624+0000 7fe0e427a640 1 -- 172.21.15.112:0/3812235373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0dc0fb810 msgr2=0x7fe0dc0fbbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:20.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.624+0000 7fe0e427a640 1 --2- 172.21.15.112:0/3812235373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0dc0fb810 0x7fe0dc0fbbe0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc0048d0 tx=0x7fe0cc009da0 comp rx=0 tx=0).stop 2024-09-16T00:40:20.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.625+0000 7fe0e427a640 1 -- 172.21.15.112:0/3812235373 shutdown_connections 2024-09-16T00:40:20.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.625+0000 7fe0e427a640 1 --2- 172.21.15.112:0/3812235373 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0dc0fcf80 0x7fe0dc0fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:20.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.625+0000 7fe0e427a640 1 --2- 172.21.15.112:0/3812235373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0dc0fb810 0x7fe0dc0fbbe0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:20.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.625+0000 7fe0e427a640 1 -- 172.21.15.112:0/3812235373 >> 172.21.15.112:0/3812235373 conn(0x7fe0dc0723b0 msgr2=0x7fe0dc0727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:20.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.625+0000 7fe0e427a640 1 -- 172.21.15.112:0/3812235373 shutdown_connections 2024-09-16T00:40:20.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.625+0000 7fe0e427a640 1 -- 172.21.15.112:0/3812235373 wait complete. 2024-09-16T00:40:20.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.625+0000 7fe0e427a640 1 Processor -- start 2024-09-16T00:40:20.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.625+0000 7fe0e427a640 1 -- start start 2024-09-16T00:40:20.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.625+0000 7fe0e427a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0dc0fb810 0x7fe0dc113220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:20.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.625+0000 7fe0e427a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0dc0fcf80 0x7fe0dc113760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:20.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.625+0000 7fe0e427a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0dc113df0 con 0x7fe0dc0fcf80 2024-09-16T00:40:20.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.625+0000 7fe0e427a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0dc10d1a0 con 0x7fe0dc0fb810 2024-09-16T00:40:20.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.626+0000 7fe0e1815640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0dc0fcf80 0x7fe0dc113760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:20.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.626+0000 7fe0e1815640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0dc0fcf80 0x7fe0dc113760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49160/0 (socket says 172.21.15.112:49160) 2024-09-16T00:40:20.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.626+0000 7fe0e1815640 1 -- 172.21.15.112:0/2741362496 learned_addr learned my addr 172.21.15.112:0/2741362496 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:40:20.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.626+0000 7fe0e2016640 1 --2- 172.21.15.112:0/2741362496 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0dc0fb810 0x7fe0dc113220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:20.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.626+0000 7fe0e1815640 1 -- 172.21.15.112:0/2741362496 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0dc0fb810 msgr2=0x7fe0dc113220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:20.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.626+0000 7fe0e1815640 1 --2- 172.21.15.112:0/2741362496 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0dc0fb810 0x7fe0dc113220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:20.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.626+0000 7fe0e1815640 1 -- 172.21.15.112:0/2741362496 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0cc005040 con 0x7fe0dc0fcf80 2024-09-16T00:40:20.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.626+0000 7fe0e1815640 1 --2- 172.21.15.112:0/2741362496 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0dc0fcf80 0x7fe0dc113760 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe0d800b790 tx=0x7fe0d80091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:20.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.626+0000 7fe0d2ffd640 1 -- 172.21.15.112:0/2741362496 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe0d800c700 con 0x7fe0dc0fcf80 2024-09-16T00:40:20.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.626+0000 7fe0d2ffd640 1 -- 172.21.15.112:0/2741362496 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fe0d800cd00 con 0x7fe0dc0fcf80 2024-09-16T00:40:20.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.626+0000 7fe0e427a640 1 -- 172.21.15.112:0/2741362496 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0dc10d480 con 0x7fe0dc0fcf80 2024-09-16T00:40:20.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.626+0000 7fe0d2ffd640 1 -- 172.21.15.112:0/2741362496 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe0d8015e70 con 0x7fe0dc0fcf80 2024-09-16T00:40:20.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.626+0000 7fe0e427a640 1 -- 172.21.15.112:0/2741362496 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0dc10d9a0 con 0x7fe0dc0fcf80 2024-09-16T00:40:20.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.627+0000 7fe0e427a640 1 -- 172.21.15.112:0/2741362496 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0dc0fcbc0 con 0x7fe0dc0fcf80 2024-09-16T00:40:20.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.628+0000 7fe0d2ffd640 1 -- 172.21.15.112:0/2741362496 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fe0d8012070 con 0x7fe0dc0fcf80 2024-09-16T00:40:20.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.628+0000 7fe0d2ffd640 1 --2- 172.21.15.112:0/2741362496 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fe0c806e530 0x7fe0c80709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:20.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.629+0000 7fe0e2016640 1 --2- 172.21.15.112:0/2741362496 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fe0c806e530 0x7fe0c80709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:20.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.629+0000 7fe0d2ffd640 1 -- 172.21.15.112:0/2741362496 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7fe0d808df60 con 0x7fe0dc0fcf80 2024-09-16T00:40:20.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.629+0000 7fe0e2016640 1 --2- 172.21.15.112:0/2741362496 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fe0c806e530 0x7fe0c80709e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc004540 tx=0x7fe0cc009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:20.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.633+0000 7fe0d2ffd640 1 -- 172.21.15.112:0/2741362496 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fe0d8056e70 con 0x7fe0dc0fcf80 2024-09-16T00:40:20.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.862+0000 7fe0e427a640 1 -- 172.21.15.112:0/2741362496 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fe0dc072ec0 con 0x7fe0dc0fcf80 2024-09-16T00:40:20.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.863+0000 7fe0d2ffd640 1 -- 172.21.15.112:0/2741362496 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7fe0d805a530 con 0x7fe0dc0fcf80 2024-09-16T00:40:20.863 INFO:teuthology.orchestra.run.smithi112.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-16T00:40:20.863 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-16T00:40:20.863 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs is degraded 2024-09-16T00:40:20.863 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T00:40:20.863 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T00:40:20.863 INFO:teuthology.orchestra.run.smithi112.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-16T00:40:20.863 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-16T00:40:20.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.865+0000 7fe0e427a640 1 -- 172.21.15.112:0/2741362496 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fe0c806e530 msgr2=0x7fe0c80709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:20.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.865+0000 7fe0e427a640 1 --2- 172.21.15.112:0/2741362496 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fe0c806e530 0x7fe0c80709e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc004540 tx=0x7fe0cc009da0 comp rx=0 tx=0).stop 2024-09-16T00:40:20.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.865+0000 7fe0e427a640 1 -- 172.21.15.112:0/2741362496 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0dc0fcf80 msgr2=0x7fe0dc113760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:20.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.865+0000 7fe0e427a640 1 --2- 172.21.15.112:0/2741362496 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0dc0fcf80 0x7fe0dc113760 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe0d800b790 tx=0x7fe0d80091d0 comp rx=0 tx=0).stop 2024-09-16T00:40:20.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.865+0000 7fe0e427a640 1 -- 172.21.15.112:0/2741362496 shutdown_connections 2024-09-16T00:40:20.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.865+0000 7fe0e427a640 1 --2- 172.21.15.112:0/2741362496 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fe0c806e530 0x7fe0c80709e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:20.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.865+0000 7fe0e427a640 1 --2- 172.21.15.112:0/2741362496 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0dc0fcf80 0x7fe0dc113760 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:20.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.865+0000 7fe0e427a640 1 --2- 172.21.15.112:0/2741362496 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0dc0fb810 0x7fe0dc113220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:20.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.865+0000 7fe0e427a640 1 -- 172.21.15.112:0/2741362496 >> 172.21.15.112:0/2741362496 conn(0x7fe0dc0723b0 msgr2=0x7fe0dc064f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:20.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.865+0000 7fe0e427a640 1 -- 172.21.15.112:0/2741362496 shutdown_connections 2024-09-16T00:40:20.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:20.865+0000 7fe0e427a640 1 -- 172.21.15.112:0/2741362496 wait complete. 2024-09-16T00:40:21.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:40:20 smithi169 ceph-mon[77626]: from='client.34744 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:40:21.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:40:20 smithi169 ceph-mon[77626]: from='client.34748 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:40:21.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:40:20 smithi169 ceph-mon[77626]: from='client.34752 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:40:21.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:40:20 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/3492794150' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:40:21.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:40:20 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/1984367921' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:40:21.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:40:20 smithi112 ceph-mon[97750]: from='client.34744 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:40:21.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:40:20 smithi112 ceph-mon[97750]: from='client.34748 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:40:21.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:40:20 smithi112 ceph-mon[97750]: from='client.34752 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:40:21.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:40:20 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/3492794150' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:40:21.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:40:20 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/1984367921' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:40:22.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:40:21 smithi169 ceph-mon[77626]: from='client.34764 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:40:22.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:40:21 smithi169 ceph-mon[77626]: pgmap v378: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:40:22.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:40:21 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/2741362496' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:40:22.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:40:21 smithi112 ceph-mon[97750]: from='client.34764 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:40:22.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:40:21 smithi112 ceph-mon[97750]: pgmap v378: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:40:22.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:40:21 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/2741362496' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:40:24.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:40:23 smithi169 ceph-mon[77626]: pgmap v379: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:40:24.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:40:23 smithi112 ceph-mon[97750]: pgmap v379: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:40:26.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:40:25 smithi169 ceph-mon[77626]: pgmap v380: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:40:26.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:40:25 smithi112 ceph-mon[97750]: pgmap v380: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:40:28.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:40:27 smithi169 ceph-mon[77626]: pgmap v381: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:40:28.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:40:27 smithi112 ceph-mon[97750]: pgmap v381: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:40:30.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:40:29 smithi169 ceph-mon[77626]: pgmap v382: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:40:30.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:40:29 smithi112 ceph-mon[97750]: pgmap v382: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:40:32.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:40:31 smithi169 ceph-mon[77626]: pgmap v383: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:40:32.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:40:31 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:40:32.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:40:31 smithi112 ceph-mon[97750]: pgmap v383: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:40:32.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:40:31 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:40:34.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:40:33 smithi169 ceph-mon[77626]: pgmap v384: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:40:34.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:40:33 smithi112 ceph-mon[97750]: pgmap v384: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:40:36.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:40:35 smithi112 ceph-mon[97750]: pgmap v385: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:40:36.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:40:35 smithi169 ceph-mon[77626]: pgmap v385: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:40:38.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:40:37 smithi112 ceph-mon[97750]: pgmap v386: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:40:38.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:40:37 smithi169 ceph-mon[77626]: pgmap v386: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:40:40.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:40:39 smithi112 ceph-mon[97750]: pgmap v387: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:40:40.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:40:39 smithi169 ceph-mon[77626]: pgmap v387: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:40:42.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:40:41 smithi112 ceph-mon[97750]: pgmap v388: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:40:42.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:40:41 smithi169 ceph-mon[77626]: pgmap v388: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:40:44.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:40:43 smithi112 ceph-mon[97750]: pgmap v389: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:40:44.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:40:43 smithi169 ceph-mon[77626]: pgmap v389: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:40:46.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:40:45 smithi112 ceph-mon[97750]: pgmap v390: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:40:46.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:40:45 smithi169 ceph-mon[77626]: pgmap v390: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:40:47.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:40:46 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:40:47.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:40:46 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:40:48.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:40:47 smithi112 ceph-mon[97750]: pgmap v391: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:40:48.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:40:47 smithi169 ceph-mon[77626]: pgmap v391: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:40:50.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:40:49 smithi112 ceph-mon[97750]: pgmap v392: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:40:50.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:40:49 smithi169 ceph-mon[77626]: pgmap v392: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:40:50.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:50.955+0000 7fde98f56640 1 -- 172.21.15.112:0/3026907412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde94102620 msgr2=0x7fde941029f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:50.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:50.955+0000 7fde98f56640 1 --2- 172.21.15.112:0/3026907412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde94102620 0x7fde941029f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fde840048d0 tx=0x7fde84009da0 comp rx=0 tx=0).stop 2024-09-16T00:40:50.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:50.955+0000 7fde98f56640 1 -- 172.21.15.112:0/3026907412 shutdown_connections 2024-09-16T00:40:50.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:50.955+0000 7fde98f56640 1 --2- 172.21.15.112:0/3026907412 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fde940fac80 0x7fde940fb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:50.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:50.955+0000 7fde98f56640 1 --2- 172.21.15.112:0/3026907412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde94102620 0x7fde941029f0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:50.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:50.955+0000 7fde98f56640 1 -- 172.21.15.112:0/3026907412 >> 172.21.15.112:0/3026907412 conn(0x7fde94070ce0 msgr2=0x7fde940710e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:50.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:50.955+0000 7fde98f56640 1 -- 172.21.15.112:0/3026907412 shutdown_connections 2024-09-16T00:40:50.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:50.955+0000 7fde98f56640 1 -- 172.21.15.112:0/3026907412 wait complete. 2024-09-16T00:40:50.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:50.956+0000 7fde98f56640 1 Processor -- start 2024-09-16T00:40:50.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:50.956+0000 7fde98f56640 1 -- start start 2024-09-16T00:40:50.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:50.956+0000 7fde98f56640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde940fac80 0x7fde941131d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:50.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:50.956+0000 7fde98f56640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fde94102620 0x7fde94113710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:50.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:50.956+0000 7fde98f56640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde94113d10 con 0x7fde940fac80 2024-09-16T00:40:50.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:50.956+0000 7fde98f56640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde9410d150 con 0x7fde94102620 2024-09-16T00:40:50.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:50.956+0000 7fde9259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde940fac80 0x7fde941131d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:50.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:50.956+0000 7fde9259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde940fac80 0x7fde941131d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43440/0 (socket says 172.21.15.112:43440) 2024-09-16T00:40:50.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:50.956+0000 7fde9259c640 1 -- 172.21.15.112:0/3694358379 learned_addr learned my addr 172.21.15.112:0/3694358379 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:40:50.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:50.956+0000 7fde91d9b640 1 --2- 172.21.15.112:0/3694358379 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fde94102620 0x7fde94113710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:50.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:50.956+0000 7fde9259c640 1 -- 172.21.15.112:0/3694358379 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fde94102620 msgr2=0x7fde94113710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:50.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:50.956+0000 7fde9259c640 1 --2- 172.21.15.112:0/3694358379 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fde94102620 0x7fde94113710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:50.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:50.956+0000 7fde9259c640 1 -- 172.21.15.112:0/3694358379 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde84005040 con 0x7fde940fac80 2024-09-16T00:40:50.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:50.957+0000 7fde9259c640 1 --2- 172.21.15.112:0/3694358379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde940fac80 0x7fde941131d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fde84000c00 tx=0x7fde84009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:50.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:50.957+0000 7fde82ffd640 1 -- 172.21.15.112:0/3694358379 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde84038070 con 0x7fde940fac80 2024-09-16T00:40:50.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:50.957+0000 7fde82ffd640 1 -- 172.21.15.112:0/3694358379 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fde8402ade0 con 0x7fde940fac80 2024-09-16T00:40:50.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:50.957+0000 7fde82ffd640 1 -- 172.21.15.112:0/3694358379 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde8402c7e0 con 0x7fde940fac80 2024-09-16T00:40:50.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:50.957+0000 7fde98f56640 1 -- 172.21.15.112:0/3694358379 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde9410d3d0 con 0x7fde940fac80 2024-09-16T00:40:50.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:50.957+0000 7fde98f56640 1 -- 172.21.15.112:0/3694358379 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde9410d890 con 0x7fde940fac80 2024-09-16T00:40:50.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:50.958+0000 7fde98f56640 1 -- 172.21.15.112:0/3694358379 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde940657b0 con 0x7fde940fac80 2024-09-16T00:40:50.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:50.958+0000 7fde82ffd640 1 -- 172.21.15.112:0/3694358379 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fde84030020 con 0x7fde940fac80 2024-09-16T00:40:50.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:50.959+0000 7fde82ffd640 1 --2- 172.21.15.112:0/3694358379 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fde7c06e530 0x7fde7c0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:50.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:50.959+0000 7fde82ffd640 1 -- 172.21.15.112:0/3694358379 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7fde840aded0 con 0x7fde940fac80 2024-09-16T00:40:50.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:50.959+0000 7fde91d9b640 1 --2- 172.21.15.112:0/3694358379 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fde7c06e530 0x7fde7c0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:50.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:50.959+0000 7fde91d9b640 1 --2- 172.21.15.112:0/3694358379 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fde7c06e530 0x7fde7c0709e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fde9410ea90 tx=0x7fde8800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:50.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:50.963+0000 7fde82ffd640 1 -- 172.21.15.112:0/3694358379 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fde84076f70 con 0x7fde940fac80 2024-09-16T00:40:51.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.149+0000 7fde98f56640 1 -- 172.21.15.112:0/3694358379 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fde9404b720 con 0x7fde7c06e530 2024-09-16T00:40:51.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.150+0000 7fde82ffd640 1 -- 172.21.15.112:0/3694358379 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fde9404b720 con 0x7fde7c06e530 2024-09-16T00:40:51.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.152+0000 7fde98f56640 1 -- 172.21.15.112:0/3694358379 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fde7c06e530 msgr2=0x7fde7c0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:51.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.152+0000 7fde98f56640 1 --2- 172.21.15.112:0/3694358379 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fde7c06e530 0x7fde7c0709e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fde9410ea90 tx=0x7fde8800b040 comp rx=0 tx=0).stop 2024-09-16T00:40:51.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.152+0000 7fde98f56640 1 -- 172.21.15.112:0/3694358379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde940fac80 msgr2=0x7fde941131d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:51.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.152+0000 7fde98f56640 1 --2- 172.21.15.112:0/3694358379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde940fac80 0x7fde941131d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fde84000c00 tx=0x7fde84009da0 comp rx=0 tx=0).stop 2024-09-16T00:40:51.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.152+0000 7fde98f56640 1 -- 172.21.15.112:0/3694358379 shutdown_connections 2024-09-16T00:40:51.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.152+0000 7fde98f56640 1 --2- 172.21.15.112:0/3694358379 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fde7c06e530 0x7fde7c0709e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:51.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.152+0000 7fde98f56640 1 --2- 172.21.15.112:0/3694358379 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fde94102620 0x7fde94113710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:51.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.152+0000 7fde98f56640 1 --2- 172.21.15.112:0/3694358379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde940fac80 0x7fde941131d0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:51.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.152+0000 7fde98f56640 1 -- 172.21.15.112:0/3694358379 >> 172.21.15.112:0/3694358379 conn(0x7fde94070ce0 msgr2=0x7fde94105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:51.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.152+0000 7fde98f56640 1 -- 172.21.15.112:0/3694358379 shutdown_connections 2024-09-16T00:40:51.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.152+0000 7fde98f56640 1 -- 172.21.15.112:0/3694358379 wait complete. 2024-09-16T00:40:51.161 INFO:teuthology.orchestra.run.smithi112.stdout:true 2024-09-16T00:40:51.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.232+0000 7f5dd30e5640 1 -- 172.21.15.112:0/2020216188 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dcc0fb810 msgr2=0x7f5dcc0fbbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:51.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.232+0000 7f5dd30e5640 1 --2- 172.21.15.112:0/2020216188 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dcc0fb810 0x7f5dcc0fbbe0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc0048d0 tx=0x7f5dbc009da0 comp rx=0 tx=0).stop 2024-09-16T00:40:51.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.232+0000 7f5dd30e5640 1 -- 172.21.15.112:0/2020216188 shutdown_connections 2024-09-16T00:40:51.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.232+0000 7f5dd30e5640 1 --2- 172.21.15.112:0/2020216188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5dcc0fcf80 0x7f5dcc0fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:51.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.232+0000 7f5dd30e5640 1 --2- 172.21.15.112:0/2020216188 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dcc0fb810 0x7f5dcc0fbbe0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:51.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.232+0000 7f5dd30e5640 1 -- 172.21.15.112:0/2020216188 >> 172.21.15.112:0/2020216188 conn(0x7f5dcc0723b0 msgr2=0x7f5dcc0727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:51.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.232+0000 7f5dd30e5640 1 -- 172.21.15.112:0/2020216188 shutdown_connections 2024-09-16T00:40:51.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.232+0000 7f5dd30e5640 1 -- 172.21.15.112:0/2020216188 wait complete. 2024-09-16T00:40:51.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.232+0000 7f5dd30e5640 1 Processor -- start 2024-09-16T00:40:51.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.232+0000 7f5dd30e5640 1 -- start start 2024-09-16T00:40:51.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.233+0000 7f5dd30e5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dcc0fb810 0x7f5dcc0f8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:51.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.233+0000 7f5dd30e5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5dcc0fcf80 0x7f5dcc0f8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:51.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.233+0000 7f5dd30e5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dcc0f8fc0 con 0x7f5dcc0fb810 2024-09-16T00:40:51.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.233+0000 7f5dd30e5640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dcc0f9130 con 0x7f5dcc0fcf80 2024-09-16T00:40:51.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.233+0000 7f5dd0e81640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dcc0fb810 0x7f5dcc0f8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:51.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.233+0000 7f5dd0e81640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dcc0fb810 0x7f5dcc0f8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43456/0 (socket says 172.21.15.112:43456) 2024-09-16T00:40:51.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.233+0000 7f5dd0e81640 1 -- 172.21.15.112:0/2943224811 learned_addr learned my addr 172.21.15.112:0/2943224811 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:40:51.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.233+0000 7f5dcbfff640 1 --2- 172.21.15.112:0/2943224811 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5dcc0fcf80 0x7f5dcc0f8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:51.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.233+0000 7f5dd0e81640 1 -- 172.21.15.112:0/2943224811 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5dcc0fcf80 msgr2=0x7f5dcc0f8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:51.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.233+0000 7f5dd0e81640 1 --2- 172.21.15.112:0/2943224811 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5dcc0fcf80 0x7f5dcc0f8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:51.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.233+0000 7f5dd0e81640 1 -- 172.21.15.112:0/2943224811 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dbc005040 con 0x7f5dcc0fb810 2024-09-16T00:40:51.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.233+0000 7f5dd0e81640 1 --2- 172.21.15.112:0/2943224811 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dcc0fb810 0x7f5dcc0f8540 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc000c00 tx=0x7f5dbc009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:51.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.234+0000 7f5dc97fa640 1 -- 172.21.15.112:0/2943224811 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5dbc038070 con 0x7f5dcc0fb810 2024-09-16T00:40:51.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.234+0000 7f5dd30e5640 1 -- 172.21.15.112:0/2943224811 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dcc0f9360 con 0x7f5dcc0fb810 2024-09-16T00:40:51.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.234+0000 7f5dc97fa640 1 -- 172.21.15.112:0/2943224811 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f5dbc02ade0 con 0x7f5dcc0fb810 2024-09-16T00:40:51.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.234+0000 7f5dc97fa640 1 -- 172.21.15.112:0/2943224811 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5dbc02c7b0 con 0x7f5dcc0fb810 2024-09-16T00:40:51.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.234+0000 7f5dd30e5640 1 -- 172.21.15.112:0/2943224811 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dcc119890 con 0x7f5dcc0fb810 2024-09-16T00:40:51.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.235+0000 7f5dd30e5640 1 -- 172.21.15.112:0/2943224811 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dcc0fcbc0 con 0x7f5dcc0fb810 2024-09-16T00:40:51.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.235+0000 7f5dc97fa640 1 -- 172.21.15.112:0/2943224811 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f5dbc030020 con 0x7f5dcc0fb810 2024-09-16T00:40:51.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.236+0000 7f5dc97fa640 1 --2- 172.21.15.112:0/2943224811 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5db806e530 0x7f5db80709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:51.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.236+0000 7f5dc97fa640 1 -- 172.21.15.112:0/2943224811 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f5dbc0ae660 con 0x7f5dcc0fb810 2024-09-16T00:40:51.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.236+0000 7f5dcbfff640 1 --2- 172.21.15.112:0/2943224811 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5db806e530 0x7f5db80709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:51.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.236+0000 7f5dcbfff640 1 --2- 172.21.15.112:0/2943224811 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5db806e530 0x7f5db80709e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc0f9e80 tx=0x7f5db400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:51.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.240+0000 7f5dc97fa640 1 -- 172.21.15.112:0/2943224811 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f5dbc077650 con 0x7f5dcc0fb810 2024-09-16T00:40:51.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.422+0000 7f5dd30e5640 1 -- 172.21.15.112:0/2943224811 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5dcc0f9c30 con 0x7f5db806e530 2024-09-16T00:40:51.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.423+0000 7f5dc97fa640 1 -- 172.21.15.112:0/2943224811 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f5dcc0f9c30 con 0x7f5db806e530 2024-09-16T00:40:51.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.425+0000 7f5dd30e5640 1 -- 172.21.15.112:0/2943224811 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5db806e530 msgr2=0x7f5db80709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:51.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.425+0000 7f5dd30e5640 1 --2- 172.21.15.112:0/2943224811 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5db806e530 0x7f5db80709e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc0f9e80 tx=0x7f5db400b040 comp rx=0 tx=0).stop 2024-09-16T00:40:51.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.425+0000 7f5dd30e5640 1 -- 172.21.15.112:0/2943224811 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dcc0fb810 msgr2=0x7f5dcc0f8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:51.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.425+0000 7f5dd30e5640 1 --2- 172.21.15.112:0/2943224811 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dcc0fb810 0x7f5dcc0f8540 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc000c00 tx=0x7f5dbc009da0 comp rx=0 tx=0).stop 2024-09-16T00:40:51.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.425+0000 7f5dd30e5640 1 -- 172.21.15.112:0/2943224811 shutdown_connections 2024-09-16T00:40:51.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.425+0000 7f5dd30e5640 1 --2- 172.21.15.112:0/2943224811 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5db806e530 0x7f5db80709e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:51.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.425+0000 7f5dd30e5640 1 --2- 172.21.15.112:0/2943224811 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5dcc0fcf80 0x7f5dcc0f8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:51.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.425+0000 7f5dd30e5640 1 --2- 172.21.15.112:0/2943224811 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dcc0fb810 0x7f5dcc0f8540 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:51.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.425+0000 7f5dd30e5640 1 -- 172.21.15.112:0/2943224811 >> 172.21.15.112:0/2943224811 conn(0x7f5dcc0723b0 msgr2=0x7f5dcc064f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:51.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.426+0000 7f5dd30e5640 1 -- 172.21.15.112:0/2943224811 shutdown_connections 2024-09-16T00:40:51.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.426+0000 7f5dd30e5640 1 -- 172.21.15.112:0/2943224811 wait complete. 2024-09-16T00:40:51.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.504+0000 7f232bdeb640 1 -- 172.21.15.112:0/3328314216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2324102ed0 msgr2=0x7f23240fc590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:51.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.504+0000 7f232bdeb640 1 --2- 172.21.15.112:0/3328314216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2324102ed0 0x7f23240fc590 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f23180048d0 tx=0x7f2318009da0 comp rx=0 tx=0).stop 2024-09-16T00:40:51.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.504+0000 7f232bdeb640 1 -- 172.21.15.112:0/3328314216 shutdown_connections 2024-09-16T00:40:51.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.504+0000 7f232bdeb640 1 --2- 172.21.15.112:0/3328314216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2324102ed0 0x7f23240fc590 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:51.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.504+0000 7f232bdeb640 1 --2- 172.21.15.112:0/3328314216 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2324102530 0x7f2324102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:51.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.504+0000 7f232bdeb640 1 -- 172.21.15.112:0/3328314216 >> 172.21.15.112:0/3328314216 conn(0x7f23240f8110 msgr2=0x7f23240fa520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:51.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.504+0000 7f232bdeb640 1 -- 172.21.15.112:0/3328314216 shutdown_connections 2024-09-16T00:40:51.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.504+0000 7f232bdeb640 1 -- 172.21.15.112:0/3328314216 wait complete. 2024-09-16T00:40:51.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.505+0000 7f232bdeb640 1 Processor -- start 2024-09-16T00:40:51.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.505+0000 7f232bdeb640 1 -- start start 2024-09-16T00:40:51.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.505+0000 7f232bdeb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2324102530 0x7f232406fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:51.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.505+0000 7f232bdeb640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2324102ed0 0x7f232406ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:51.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.505+0000 7f232bdeb640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2324070490 con 0x7f2324102530 2024-09-16T00:40:51.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.505+0000 7f232bdeb640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2324070600 con 0x7f2324102ed0 2024-09-16T00:40:51.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.505+0000 7f2329b87640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2324102530 0x7f232406fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:51.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.505+0000 7f2329b87640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2324102530 0x7f232406fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43474/0 (socket says 172.21.15.112:43474) 2024-09-16T00:40:51.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.505+0000 7f2329b87640 1 -- 172.21.15.112:0/3129537078 learned_addr learned my addr 172.21.15.112:0/3129537078 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:40:51.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.505+0000 7f2329b87640 1 -- 172.21.15.112:0/3129537078 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2324102ed0 msgr2=0x7f232406ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:51.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.505+0000 7f2329386640 1 --2- 172.21.15.112:0/3129537078 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2324102ed0 0x7f232406ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:51.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.505+0000 7f2329b87640 1 --2- 172.21.15.112:0/3129537078 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2324102ed0 0x7f232406ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:51.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.505+0000 7f2329b87640 1 -- 172.21.15.112:0/3129537078 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2318005040 con 0x7f2324102530 2024-09-16T00:40:51.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.506+0000 7f2329b87640 1 --2- 172.21.15.112:0/3129537078 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2324102530 0x7f232406fa10 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f232000c790 tx=0x7f23200043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:51.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.506+0000 7f23167fc640 1 -- 172.21.15.112:0/3129537078 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f232000a7a0 con 0x7f2324102530 2024-09-16T00:40:51.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.506+0000 7f23167fc640 1 -- 172.21.15.112:0/3129537078 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f232000ada0 con 0x7f2324102530 2024-09-16T00:40:51.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.506+0000 7f232bdeb640 1 -- 172.21.15.112:0/3129537078 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2324070890 con 0x7f2324102530 2024-09-16T00:40:51.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.506+0000 7f23167fc640 1 -- 172.21.15.112:0/3129537078 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23200113a0 con 0x7f2324102530 2024-09-16T00:40:51.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.506+0000 7f232bdeb640 1 -- 172.21.15.112:0/3129537078 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2324119940 con 0x7f2324102530 2024-09-16T00:40:51.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.507+0000 7f232bdeb640 1 -- 172.21.15.112:0/3129537078 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23240fdcb0 con 0x7f2324102530 2024-09-16T00:40:51.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.507+0000 7f23167fc640 1 -- 172.21.15.112:0/3129537078 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f232000a900 con 0x7f2324102530 2024-09-16T00:40:51.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.508+0000 7f23167fc640 1 --2- 172.21.15.112:0/3129537078 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f231006e530 0x7f23100709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:51.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.508+0000 7f2329386640 1 --2- 172.21.15.112:0/3129537078 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f231006e530 0x7f23100709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:51.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.508+0000 7f23167fc640 1 -- 172.21.15.112:0/3129537078 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f232008e3a0 con 0x7f2324102530 2024-09-16T00:40:51.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.509+0000 7f2329386640 1 --2- 172.21.15.112:0/3129537078 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f231006e530 0x7f23100709e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2318000c00 tx=0x7f2318009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:51.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.512+0000 7f23167fc640 1 -- 172.21.15.112:0/3129537078 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f2320057430 con 0x7f2324102530 2024-09-16T00:40:51.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.687+0000 7f232bdeb640 1 -- 172.21.15.112:0/3129537078 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2324071100 con 0x7f231006e530 2024-09-16T00:40:51.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.695+0000 7f23167fc640 1 -- 172.21.15.112:0/3129537078 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7f2324071100 con 0x7f231006e530 2024-09-16T00:40:51.695 INFO:teuthology.orchestra.run.smithi112.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T00:40:51.696 INFO:teuthology.orchestra.run.smithi112.stdout:alertmanager.smithi112 smithi112 *:9093,9094 running (13m) 6m ago 22m 24.7M - 0.25.0 c8568f914cd2 d85ea0562022 2024-09-16T00:40:51.696 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi112 smithi112 running (10m) 6m ago 22m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c e4fe58f9702a 2024-09-16T00:40:51.696 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi169 smithi169 running (10m) 6m ago 22m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c 3ba2c81d527f 2024-09-16T00:40:51.696 INFO:teuthology.orchestra.run.smithi112.stdout:grafana.smithi112 smithi112 *:3000 running (12m) 6m ago 22m 82.1M - 9.4.12 2bacad6d85d8 51f9d2013da2 2024-09-16T00:40:51.696 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.hxfwkh smithi112 running (6m) 6m ago 17m 14.2M - 19.1.1-326-g8921c8ef f1455bd0219c 1723069902d6 2024-09-16T00:40:51.696 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.ithwzj smithi112 running (7m) 6m ago 17m 12.2M - 19.1.1-326-g8921c8ef f1455bd0219c e7aad5b81db5 2024-09-16T00:40:51.696 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.gawwgw smithi169 running (6m) 6m ago 17m 15.7M - 19.1.1-326-g8921c8ef f1455bd0219c 469be459ee12 2024-09-16T00:40:51.696 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.lgeqrj smithi169 running (6m) 6m ago 17m 13.3M - 19.1.1-326-g8921c8ef f1455bd0219c 31f0fbd2395d 2024-09-16T00:40:51.696 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi112.xrkkbc smithi112 *:8443,9283,8765 running (13m) 6m ago 23m 504M - 19.1.1-326-g8921c8ef f1455bd0219c a981f7fd2810 2024-09-16T00:40:51.696 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi169.mtrobk smithi169 *:8443,9283,8765 running (12m) 6m ago 21m 442M - 19.1.1-326-g8921c8ef f1455bd0219c 4096e5369c2d 2024-09-16T00:40:51.696 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi112 smithi112 running (11m) 6m ago 24m 49.8M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 5aaec033c042 2024-09-16T00:40:51.696 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi169 smithi169 running (11m) 6m ago 21m 38.9M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 7bc609155f3f 2024-09-16T00:40:51.696 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi112 smithi112 *:9100 running (14m) 6m ago 22m 15.5M - 1.5.0 0da6a335fe13 6404d1d601e2 2024-09-16T00:40:51.696 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi169 smithi169 *:9100 running (13m) 6m ago 21m 15.4M - 1.5.0 0da6a335fe13 5505baf702a6 2024-09-16T00:40:51.697 INFO:teuthology.orchestra.run.smithi112.stdout:osd.0 smithi112 running (10m) 6m ago 21m 105M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 53efc9397017 2024-09-16T00:40:51.697 INFO:teuthology.orchestra.run.smithi112.stdout:osd.1 smithi112 running (9m) 6m ago 20m 98.9M 1557M 19.1.1-326-g8921c8ef f1455bd0219c fe6f1a4a2cf4 2024-09-16T00:40:51.697 INFO:teuthology.orchestra.run.smithi112.stdout:osd.2 smithi112 running (9m) 6m ago 20m 102M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 5f7863712057 2024-09-16T00:40:51.697 INFO:teuthology.orchestra.run.smithi112.stdout:osd.3 smithi169 running (8m) 6m ago 19m 92.1M 2581M 19.1.1-326-g8921c8ef f1455bd0219c ee07b5577810 2024-09-16T00:40:51.697 INFO:teuthology.orchestra.run.smithi112.stdout:osd.4 smithi169 running (8m) 6m ago 19m 103M 2581M 19.1.1-326-g8921c8ef f1455bd0219c 769f026a54eb 2024-09-16T00:40:51.697 INFO:teuthology.orchestra.run.smithi112.stdout:osd.5 smithi169 running (7m) 6m ago 18m 125M 2581M 19.1.1-326-g8921c8ef f1455bd0219c a6007c13cf78 2024-09-16T00:40:51.697 INFO:teuthology.orchestra.run.smithi112.stdout:prometheus.smithi112 smithi112 *:9095 running (12m) 6m ago 22m 64.1M - 2.43.0 a07b618ecd1d 7a1d42e60d2b 2024-09-16T00:40:51.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.697+0000 7f232bdeb640 1 -- 172.21.15.112:0/3129537078 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f231006e530 msgr2=0x7f23100709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:51.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.697+0000 7f232bdeb640 1 --2- 172.21.15.112:0/3129537078 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f231006e530 0x7f23100709e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2318000c00 tx=0x7f2318009da0 comp rx=0 tx=0).stop 2024-09-16T00:40:51.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.697+0000 7f232bdeb640 1 -- 172.21.15.112:0/3129537078 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2324102530 msgr2=0x7f232406fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:51.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.697+0000 7f232bdeb640 1 --2- 172.21.15.112:0/3129537078 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2324102530 0x7f232406fa10 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f232000c790 tx=0x7f23200043c0 comp rx=0 tx=0).stop 2024-09-16T00:40:51.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.697+0000 7f232bdeb640 1 -- 172.21.15.112:0/3129537078 shutdown_connections 2024-09-16T00:40:51.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.697+0000 7f232bdeb640 1 --2- 172.21.15.112:0/3129537078 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f231006e530 0x7f23100709e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:51.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.697+0000 7f232bdeb640 1 --2- 172.21.15.112:0/3129537078 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2324102ed0 0x7f232406ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:51.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.697+0000 7f232bdeb640 1 --2- 172.21.15.112:0/3129537078 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2324102530 0x7f232406fa10 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:51.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.697+0000 7f232bdeb640 1 -- 172.21.15.112:0/3129537078 >> 172.21.15.112:0/3129537078 conn(0x7f23240f8110 msgr2=0x7f23240f91a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:51.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.697+0000 7f232bdeb640 1 -- 172.21.15.112:0/3129537078 shutdown_connections 2024-09-16T00:40:51.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.697+0000 7f232bdeb640 1 -- 172.21.15.112:0/3129537078 wait complete. 2024-09-16T00:40:51.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.774+0000 7f611543f640 1 -- 172.21.15.112:0/2758858752 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61100fb4a0 msgr2=0x7f61100fb910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:51.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.774+0000 7f611543f640 1 --2- 172.21.15.112:0/2758858752 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61100fb4a0 0x7f61100fb910 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f60f80048d0 tx=0x7f60f8009da0 comp rx=0 tx=0).stop 2024-09-16T00:40:51.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.774+0000 7f611543f640 1 -- 172.21.15.112:0/2758858752 shutdown_connections 2024-09-16T00:40:51.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.774+0000 7f611543f640 1 --2- 172.21.15.112:0/2758858752 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61100fb4a0 0x7f61100fb910 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:51.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.774+0000 7f611543f640 1 --2- 172.21.15.112:0/2758858752 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61101014e0 0x7f61101018b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:51.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.774+0000 7f611543f640 1 -- 172.21.15.112:0/2758858752 >> 172.21.15.112:0/2758858752 conn(0x7f61100f7060 msgr2=0x7f61100f9470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:51.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.774+0000 7f611543f640 1 -- 172.21.15.112:0/2758858752 shutdown_connections 2024-09-16T00:40:51.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.774+0000 7f611543f640 1 -- 172.21.15.112:0/2758858752 wait complete. 2024-09-16T00:40:51.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.775+0000 7f611543f640 1 Processor -- start 2024-09-16T00:40:51.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.775+0000 7f611543f640 1 -- start start 2024-09-16T00:40:51.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.775+0000 7f611543f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61100fb4a0 0x7f611010e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:51.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.775+0000 7f611543f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61101014e0 0x7f611010e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:51.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.775+0000 7f611543f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f611010ed20 con 0x7f61100fb4a0 2024-09-16T00:40:51.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.775+0000 7f611543f640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f611010ee90 con 0x7f61101014e0 2024-09-16T00:40:51.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.776+0000 7f610effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61100fb4a0 0x7f611010e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:51.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.776+0000 7f610effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61100fb4a0 0x7f611010e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43490/0 (socket says 172.21.15.112:43490) 2024-09-16T00:40:51.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.776+0000 7f610effd640 1 -- 172.21.15.112:0/275486513 learned_addr learned my addr 172.21.15.112:0/275486513 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:40:51.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.776+0000 7f61067fc640 1 --2- 172.21.15.112:0/275486513 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61101014e0 0x7f611010e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:51.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.776+0000 7f610effd640 1 -- 172.21.15.112:0/275486513 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61101014e0 msgr2=0x7f611010e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:51.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.776+0000 7f610effd640 1 --2- 172.21.15.112:0/275486513 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61101014e0 0x7f611010e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:51.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.776+0000 7f610effd640 1 -- 172.21.15.112:0/275486513 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60f8005040 con 0x7f61100fb4a0 2024-09-16T00:40:51.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.776+0000 7f610effd640 1 --2- 172.21.15.112:0/275486513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61100fb4a0 0x7f611010e190 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f610000c790 tx=0x7f61000043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:51.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.776+0000 7f6107fff640 1 -- 172.21.15.112:0/275486513 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f610000a7a0 con 0x7f61100fb4a0 2024-09-16T00:40:51.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.776+0000 7f611543f640 1 -- 172.21.15.112:0/275486513 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6110117200 con 0x7f61100fb4a0 2024-09-16T00:40:51.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.776+0000 7f6107fff640 1 -- 172.21.15.112:0/275486513 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f610000ada0 con 0x7f61100fb4a0 2024-09-16T00:40:51.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.776+0000 7f611543f640 1 -- 172.21.15.112:0/275486513 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6110117710 con 0x7f61100fb4a0 2024-09-16T00:40:51.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.776+0000 7f6107fff640 1 -- 172.21.15.112:0/275486513 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f61000113a0 con 0x7f61100fb4a0 2024-09-16T00:40:51.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.778+0000 7f611543f640 1 -- 172.21.15.112:0/275486513 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61100fcbe0 con 0x7f61100fb4a0 2024-09-16T00:40:51.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.778+0000 7f6107fff640 1 -- 172.21.15.112:0/275486513 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f6100011610 con 0x7f61100fb4a0 2024-09-16T00:40:51.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.779+0000 7f6107fff640 1 --2- 172.21.15.112:0/275486513 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f60fc06e530 0x7f60fc0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:51.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.779+0000 7f6107fff640 1 -- 172.21.15.112:0/275486513 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f610008e390 con 0x7f61100fb4a0 2024-09-16T00:40:51.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.779+0000 7f61067fc640 1 --2- 172.21.15.112:0/275486513 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f60fc06e530 0x7f60fc0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:51.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.779+0000 7f61067fc640 1 --2- 172.21.15.112:0/275486513 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f60fc06e530 0x7f60fc0709e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f60f8009ac0 tx=0x7f60f8035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:51.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:51.783+0000 7f6107fff640 1 -- 172.21.15.112:0/275486513 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f6100057460 con 0x7f61100fb4a0 2024-09-16T00:40:52.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.017+0000 7f611543f640 1 -- 172.21.15.112:0/275486513 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f61100fac60 con 0x7f61100fb4a0 2024-09-16T00:40:52.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.017+0000 7f6107fff640 1 -- 172.21.15.112:0/275486513 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f610005ab20 con 0x7f61100fb4a0 2024-09-16T00:40:52.018 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:40:52.018 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": { 2024-09-16T00:40:52.018 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:40:52.018 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:40:52.018 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": { 2024-09-16T00:40:52.019 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:40:52.019 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:40:52.019 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": { 2024-09-16T00:40:52.019 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 6 2024-09-16T00:40:52.019 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:40:52.019 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": { 2024-09-16T00:40:52.019 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 4 2024-09-16T00:40:52.019 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:40:52.019 INFO:teuthology.orchestra.run.smithi112.stdout: "overall": { 2024-09-16T00:40:52.019 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 14 2024-09-16T00:40:52.019 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-16T00:40:52.019 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:40:52.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.019+0000 7f611543f640 1 -- 172.21.15.112:0/275486513 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f60fc06e530 msgr2=0x7f60fc0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:52.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.019+0000 7f611543f640 1 --2- 172.21.15.112:0/275486513 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f60fc06e530 0x7f60fc0709e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f60f8009ac0 tx=0x7f60f8035040 comp rx=0 tx=0).stop 2024-09-16T00:40:52.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.019+0000 7f611543f640 1 -- 172.21.15.112:0/275486513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61100fb4a0 msgr2=0x7f611010e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:52.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.019+0000 7f611543f640 1 --2- 172.21.15.112:0/275486513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61100fb4a0 0x7f611010e190 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f610000c790 tx=0x7f61000043c0 comp rx=0 tx=0).stop 2024-09-16T00:40:52.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.020+0000 7f611543f640 1 -- 172.21.15.112:0/275486513 shutdown_connections 2024-09-16T00:40:52.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.020+0000 7f611543f640 1 --2- 172.21.15.112:0/275486513 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f60fc06e530 0x7f60fc0709e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:52.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.020+0000 7f611543f640 1 --2- 172.21.15.112:0/275486513 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61101014e0 0x7f611010e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:52.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.020+0000 7f611543f640 1 --2- 172.21.15.112:0/275486513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61100fb4a0 0x7f611010e190 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:52.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.020+0000 7f611543f640 1 -- 172.21.15.112:0/275486513 >> 172.21.15.112:0/275486513 conn(0x7f61100f7060 msgr2=0x7f61100f8910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:52.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.020+0000 7f611543f640 1 -- 172.21.15.112:0/275486513 shutdown_connections 2024-09-16T00:40:52.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.020+0000 7f611543f640 1 -- 172.21.15.112:0/275486513 wait complete. 2024-09-16T00:40:52.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:40:51 smithi112 ceph-mon[97750]: pgmap v393: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:40:52.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.097+0000 7f6aa6099640 1 -- 172.21.15.112:0/3260487931 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6aa0102ed0 msgr2=0x7f6aa00fc590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:52.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.097+0000 7f6aa6099640 1 --2- 172.21.15.112:0/3260487931 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6aa0102ed0 0x7f6aa00fc590 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6a940048d0 tx=0x7f6a94009da0 comp rx=0 tx=0).stop 2024-09-16T00:40:52.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.097+0000 7f6aa6099640 1 -- 172.21.15.112:0/3260487931 shutdown_connections 2024-09-16T00:40:52.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.098+0000 7f6aa6099640 1 --2- 172.21.15.112:0/3260487931 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6aa0102ed0 0x7f6aa00fc590 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:52.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.098+0000 7f6aa6099640 1 --2- 172.21.15.112:0/3260487931 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6aa0102530 0x7f6aa0102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:52.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.098+0000 7f6aa6099640 1 -- 172.21.15.112:0/3260487931 >> 172.21.15.112:0/3260487931 conn(0x7f6aa00f80f0 msgr2=0x7f6aa00fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:52.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.098+0000 7f6aa6099640 1 -- 172.21.15.112:0/3260487931 shutdown_connections 2024-09-16T00:40:52.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.098+0000 7f6aa6099640 1 -- 172.21.15.112:0/3260487931 wait complete. 2024-09-16T00:40:52.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.098+0000 7f6aa6099640 1 Processor -- start 2024-09-16T00:40:52.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.098+0000 7f6aa6099640 1 -- start start 2024-09-16T00:40:52.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.098+0000 7f6aa6099640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6aa0102530 0x7f6aa01153a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:52.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.098+0000 7f6aa6099640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6aa0102ed0 0x7f6aa01158e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:52.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.098+0000 7f6aa6099640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa0115ee0 con 0x7f6aa0102ed0 2024-09-16T00:40:52.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.098+0000 7f6aa6099640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa0116050 con 0x7f6aa0102530 2024-09-16T00:40:52.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.099+0000 7f6a9effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6aa0102ed0 0x7f6aa01158e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:52.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.099+0000 7f6a9effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6aa0102ed0 0x7f6aa01158e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43518/0 (socket says 172.21.15.112:43518) 2024-09-16T00:40:52.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.099+0000 7f6a9effd640 1 -- 172.21.15.112:0/2753487292 learned_addr learned my addr 172.21.15.112:0/2753487292 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:40:52.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.099+0000 7f6a9f7fe640 1 --2- 172.21.15.112:0/2753487292 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6aa0102530 0x7f6aa01153a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:52.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.099+0000 7f6a9effd640 1 -- 172.21.15.112:0/2753487292 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6aa0102530 msgr2=0x7f6aa01153a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:52.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.099+0000 7f6a9effd640 1 --2- 172.21.15.112:0/2753487292 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6aa0102530 0x7f6aa01153a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:52.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.099+0000 7f6a9effd640 1 -- 172.21.15.112:0/2753487292 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a94005040 con 0x7f6aa0102ed0 2024-09-16T00:40:52.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.099+0000 7f6a9f7fe640 1 --2- 172.21.15.112:0/2753487292 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6aa0102530 0x7f6aa01153a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:40:52.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.099+0000 7f6a9effd640 1 --2- 172.21.15.112:0/2753487292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6aa0102ed0 0x7f6aa01158e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f6a940043d0 tx=0x7f6a94009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:52.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.099+0000 7f6a87fff640 1 -- 172.21.15.112:0/2753487292 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a94038070 con 0x7f6aa0102ed0 2024-09-16T00:40:52.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.099+0000 7f6a87fff640 1 -- 172.21.15.112:0/2753487292 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f6a940096e0 con 0x7f6aa0102ed0 2024-09-16T00:40:52.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.099+0000 7f6aa6099640 1 -- 172.21.15.112:0/2753487292 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aa010f470 con 0x7f6aa0102ed0 2024-09-16T00:40:52.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.099+0000 7f6a87fff640 1 -- 172.21.15.112:0/2753487292 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a9402c8a0 con 0x7f6aa0102ed0 2024-09-16T00:40:52.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.099+0000 7f6aa6099640 1 -- 172.21.15.112:0/2753487292 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aa010f9b0 con 0x7f6aa0102ed0 2024-09-16T00:40:52.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.100+0000 7f6aa6099640 1 -- 172.21.15.112:0/2753487292 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6aa00fdcb0 con 0x7f6aa0102ed0 2024-09-16T00:40:52.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.101+0000 7f6a87fff640 1 -- 172.21.15.112:0/2753487292 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f6a94030020 con 0x7f6aa0102ed0 2024-09-16T00:40:52.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.101+0000 7f6a87fff640 1 --2- 172.21.15.112:0/2753487292 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f6a8806e530 0x7f6a880709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:52.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.101+0000 7f6a87fff640 1 -- 172.21.15.112:0/2753487292 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f6a940ae090 con 0x7f6aa0102ed0 2024-09-16T00:40:52.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.101+0000 7f6a9f7fe640 1 --2- 172.21.15.112:0/2753487292 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f6a8806e530 0x7f6a880709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:52.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.102+0000 7f6a9f7fe640 1 --2- 172.21.15.112:0/2753487292 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f6a8806e530 0x7f6a880709e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6a90004d80 tx=0x7f6a9000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:52.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.106+0000 7f6a87fff640 1 -- 172.21.15.112:0/2753487292 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f6a94077000 con 0x7f6aa0102ed0 2024-09-16T00:40:52.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:40:51 smithi169 ceph-mon[77626]: pgmap v393: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:40:52.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.305+0000 7f6aa6099640 1 -- 172.21.15.112:0/2753487292 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f6aa00fbd30 con 0x7f6aa0102ed0 2024-09-16T00:40:52.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.305+0000 7f6a87fff640 1 -- 172.21.15.112:0/2753487292 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1914 (secure 0 0 0) 0x7f6a9407a6c0 con 0x7f6aa0102ed0 2024-09-16T00:40:52.305 INFO:teuthology.orchestra.run.smithi112.stdout:e34 2024-09-16T00:40:52.306 INFO:teuthology.orchestra.run.smithi112.stdout:btime 2024-09-16T00:34:26:619254+0000 2024-09-16T00:40:52.306 INFO:teuthology.orchestra.run.smithi112.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T00:40:52.306 INFO:teuthology.orchestra.run.smithi112.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:40:52.306 INFO:teuthology.orchestra.run.smithi112.stdout:legacy client fscid: 1 2024-09-16T00:40:52.306 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:40:52.306 INFO:teuthology.orchestra.run.smithi112.stdout:Filesystem 'cephfs' (1) 2024-09-16T00:40:52.306 INFO:teuthology.orchestra.run.smithi112.stdout:fs_name cephfs 2024-09-16T00:40:52.306 INFO:teuthology.orchestra.run.smithi112.stdout:epoch 25 2024-09-16T00:40:52.306 INFO:teuthology.orchestra.run.smithi112.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-16T00:40:52.306 INFO:teuthology.orchestra.run.smithi112.stdout:created 2024-09-16T00:23:22.328199+0000 2024-09-16T00:40:52.306 INFO:teuthology.orchestra.run.smithi112.stdout:modified 2024-09-16T00:33:29.012991+0000 2024-09-16T00:40:52.306 INFO:teuthology.orchestra.run.smithi112.stdout:tableserver 0 2024-09-16T00:40:52.307 INFO:teuthology.orchestra.run.smithi112.stdout:root 0 2024-09-16T00:40:52.307 INFO:teuthology.orchestra.run.smithi112.stdout:session_timeout 60 2024-09-16T00:40:52.307 INFO:teuthology.orchestra.run.smithi112.stdout:session_autoclose 300 2024-09-16T00:40:52.307 INFO:teuthology.orchestra.run.smithi112.stdout:max_file_size 1099511627776 2024-09-16T00:40:52.307 INFO:teuthology.orchestra.run.smithi112.stdout:max_xattr_size 65536 2024-09-16T00:40:52.307 INFO:teuthology.orchestra.run.smithi112.stdout:required_client_features {} 2024-09-16T00:40:52.307 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure 0 2024-09-16T00:40:52.307 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure_osd_epoch 82 2024-09-16T00:40:52.307 INFO:teuthology.orchestra.run.smithi112.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:40:52.307 INFO:teuthology.orchestra.run.smithi112.stdout:max_mds 2 2024-09-16T00:40:52.307 INFO:teuthology.orchestra.run.smithi112.stdout:in 0,1 2024-09-16T00:40:52.307 INFO:teuthology.orchestra.run.smithi112.stdout:up {} 2024-09-16T00:40:52.307 INFO:teuthology.orchestra.run.smithi112.stdout:failed 0,1 2024-09-16T00:40:52.307 INFO:teuthology.orchestra.run.smithi112.stdout:damaged 2024-09-16T00:40:52.308 INFO:teuthology.orchestra.run.smithi112.stdout:stopped 2024-09-16T00:40:52.308 INFO:teuthology.orchestra.run.smithi112.stdout:data_pools [3] 2024-09-16T00:40:52.308 INFO:teuthology.orchestra.run.smithi112.stdout:metadata_pool 2 2024-09-16T00:40:52.308 INFO:teuthology.orchestra.run.smithi112.stdout:inline_data enabled 2024-09-16T00:40:52.308 INFO:teuthology.orchestra.run.smithi112.stdout:balancer 2024-09-16T00:40:52.308 INFO:teuthology.orchestra.run.smithi112.stdout:bal_rank_mask -1 2024-09-16T00:40:52.308 INFO:teuthology.orchestra.run.smithi112.stdout:standby_count_wanted 1 2024-09-16T00:40:52.308 INFO:teuthology.orchestra.run.smithi112.stdout:qdb_cluster leader: 0 members: 2024-09-16T00:40:52.308 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:40:52.308 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:40:52.308 INFO:teuthology.orchestra.run.smithi112.stdout:Standby daemons: 2024-09-16T00:40:52.308 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:40:52.308 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.ithwzj{-1:34392} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.112:6826/476229221,v1:172.21.15.112:6827/476229221] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:40:52.308 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.hxfwkh{-1:34404} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.112:6828/2356899876,v1:172.21.15.112:6829/2356899876] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:40:52.309 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.gawwgw{-1:44343} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.169:6824/1635329957,v1:172.21.15.169:6825/1635329957] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:40:52.309 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.lgeqrj{-1:44347} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.169:6826/184810031,v1:172.21.15.169:6827/184810031] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:40:52.309 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 34 2024-09-16T00:40:52.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.307+0000 7f6aa6099640 1 -- 172.21.15.112:0/2753487292 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f6a8806e530 msgr2=0x7f6a880709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:52.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.307+0000 7f6aa6099640 1 --2- 172.21.15.112:0/2753487292 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f6a8806e530 0x7f6a880709e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6a90004d80 tx=0x7f6a9000b040 comp rx=0 tx=0).stop 2024-09-16T00:40:52.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.307+0000 7f6aa6099640 1 -- 172.21.15.112:0/2753487292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6aa0102ed0 msgr2=0x7f6aa01158e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:52.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.307+0000 7f6aa6099640 1 --2- 172.21.15.112:0/2753487292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6aa0102ed0 0x7f6aa01158e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f6a940043d0 tx=0x7f6a94009da0 comp rx=0 tx=0).stop 2024-09-16T00:40:52.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.307+0000 7f6aa6099640 1 -- 172.21.15.112:0/2753487292 shutdown_connections 2024-09-16T00:40:52.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.307+0000 7f6aa6099640 1 --2- 172.21.15.112:0/2753487292 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f6a8806e530 0x7f6a880709e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:52.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.307+0000 7f6aa6099640 1 --2- 172.21.15.112:0/2753487292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6aa0102ed0 0x7f6aa01158e0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:52.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.307+0000 7f6aa6099640 1 --2- 172.21.15.112:0/2753487292 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6aa0102530 0x7f6aa01153a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:52.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.307+0000 7f6aa6099640 1 -- 172.21.15.112:0/2753487292 >> 172.21.15.112:0/2753487292 conn(0x7f6aa00f80f0 msgr2=0x7f6aa00f9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:52.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.307+0000 7f6aa6099640 1 -- 172.21.15.112:0/2753487292 shutdown_connections 2024-09-16T00:40:52.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.307+0000 7f6aa6099640 1 -- 172.21.15.112:0/2753487292 wait complete. 2024-09-16T00:40:52.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.386+0000 7ff8aab6c640 1 -- 172.21.15.112:0/3649927037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8a40fdf00 msgr2=0x7ff8a41065d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:52.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.386+0000 7ff8aab6c640 1 --2- 172.21.15.112:0/3649927037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8a40fdf00 0x7ff8a41065d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff898004900 tx=0x7ff898009da0 comp rx=0 tx=0).stop 2024-09-16T00:40:52.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.386+0000 7ff8aab6c640 1 -- 172.21.15.112:0/3649927037 shutdown_connections 2024-09-16T00:40:52.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.386+0000 7ff8aab6c640 1 --2- 172.21.15.112:0/3649927037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8a40fdf00 0x7ff8a41065d0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:52.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.386+0000 7ff8aab6c640 1 --2- 172.21.15.112:0/3649927037 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8a40fd560 0x7ff8a40fd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:52.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.386+0000 7ff8aab6c640 1 -- 172.21.15.112:0/3649927037 >> 172.21.15.112:0/3649927037 conn(0x7ff8a40f7000 msgr2=0x7ff8a40f9410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:52.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.386+0000 7ff8aab6c640 1 -- 172.21.15.112:0/3649927037 shutdown_connections 2024-09-16T00:40:52.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.387+0000 7ff8aab6c640 1 -- 172.21.15.112:0/3649927037 wait complete. 2024-09-16T00:40:52.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.387+0000 7ff8aab6c640 1 Processor -- start 2024-09-16T00:40:52.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.387+0000 7ff8aab6c640 1 -- start start 2024-09-16T00:40:52.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.387+0000 7ff8aab6c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8a40fd560 0x7ff8a41132d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:52.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.387+0000 7ff8aab6c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8a40fdf00 0x7ff8a4113810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:52.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.387+0000 7ff8aab6c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a4113e10 con 0x7ff8a40fd560 2024-09-16T00:40:52.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.387+0000 7ff8aab6c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a4113f80 con 0x7ff8a40fdf00 2024-09-16T00:40:52.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.388+0000 7ff8a8908640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8a40fd560 0x7ff8a41132d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:52.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.388+0000 7ff8a8908640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8a40fd560 0x7ff8a41132d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52618/0 (socket says 172.21.15.112:52618) 2024-09-16T00:40:52.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.388+0000 7ff8a8908640 1 -- 172.21.15.112:0/959212515 learned_addr learned my addr 172.21.15.112:0/959212515 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:40:52.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.388+0000 7ff8a3fff640 1 --2- 172.21.15.112:0/959212515 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8a40fdf00 0x7ff8a4113810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:52.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.388+0000 7ff8a8908640 1 -- 172.21.15.112:0/959212515 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8a40fdf00 msgr2=0x7ff8a4113810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:52.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.388+0000 7ff8a8908640 1 --2- 172.21.15.112:0/959212515 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8a40fdf00 0x7ff8a4113810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:52.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.388+0000 7ff8a8908640 1 -- 172.21.15.112:0/959212515 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff898005040 con 0x7ff8a40fd560 2024-09-16T00:40:52.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.388+0000 7ff8a3fff640 1 --2- 172.21.15.112:0/959212515 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8a40fdf00 0x7ff8a4113810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:40:52.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.388+0000 7ff8a8908640 1 --2- 172.21.15.112:0/959212515 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8a40fd560 0x7ff8a41132d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff89c002a50 tx=0x7ff89c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:52.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.388+0000 7ff8a17fa640 1 -- 172.21.15.112:0/959212515 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff89c007c30 con 0x7ff8a40fd560 2024-09-16T00:40:52.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.388+0000 7ff8a17fa640 1 -- 172.21.15.112:0/959212515 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7ff89c00f040 con 0x7ff8a40fd560 2024-09-16T00:40:52.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.388+0000 7ff8a17fa640 1 -- 172.21.15.112:0/959212515 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff89c00da90 con 0x7ff8a40fd560 2024-09-16T00:40:52.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.388+0000 7ff8aab6c640 1 -- 172.21.15.112:0/959212515 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8a410d400 con 0x7ff8a40fd560 2024-09-16T00:40:52.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.388+0000 7ff8aab6c640 1 -- 172.21.15.112:0/959212515 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8a410d920 con 0x7ff8a40fd560 2024-09-16T00:40:52.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.389+0000 7ff8aab6c640 1 -- 172.21.15.112:0/959212515 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8a40662b0 con 0x7ff8a40fd560 2024-09-16T00:40:52.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.390+0000 7ff8a17fa640 1 -- 172.21.15.112:0/959212515 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7ff89c00dc30 con 0x7ff8a40fd560 2024-09-16T00:40:52.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.390+0000 7ff8a17fa640 1 --2- 172.21.15.112:0/959212515 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff88c06e530 0x7ff88c0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:52.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.391+0000 7ff8a17fa640 1 -- 172.21.15.112:0/959212515 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7ff89c08db40 con 0x7ff8a40fd560 2024-09-16T00:40:52.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.391+0000 7ff8a3fff640 1 --2- 172.21.15.112:0/959212515 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff88c06e530 0x7ff88c0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:52.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.391+0000 7ff8a3fff640 1 --2- 172.21.15.112:0/959212515 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff88c06e530 0x7ff88c0709e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff898004540 tx=0x7ff898035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:52.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.395+0000 7ff8a17fa640 1 -- 172.21.15.112:0/959212515 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7ff89c056b30 con 0x7ff8a40fd560 2024-09-16T00:40:52.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.578+0000 7ff8aab6c640 1 -- 172.21.15.112:0/959212515 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff8a410e980 con 0x7ff88c06e530 2024-09-16T00:40:52.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.579+0000 7ff8a17fa640 1 -- 172.21.15.112:0/959212515 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7ff8a410e980 con 0x7ff88c06e530 2024-09-16T00:40:52.579 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:40:52.579 INFO:teuthology.orchestra.run.smithi112.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80", 2024-09-16T00:40:52.580 INFO:teuthology.orchestra.run.smithi112.stdout: "in_progress": true, 2024-09-16T00:40:52.580 INFO:teuthology.orchestra.run.smithi112.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T00:40:52.580 INFO:teuthology.orchestra.run.smithi112.stdout: "services_complete": [ 2024-09-16T00:40:52.580 INFO:teuthology.orchestra.run.smithi112.stdout: "mds", 2024-09-16T00:40:52.580 INFO:teuthology.orchestra.run.smithi112.stdout: "osd", 2024-09-16T00:40:52.580 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr", 2024-09-16T00:40:52.580 INFO:teuthology.orchestra.run.smithi112.stdout: "mon", 2024-09-16T00:40:52.580 INFO:teuthology.orchestra.run.smithi112.stdout: "crash" 2024-09-16T00:40:52.580 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-16T00:40:52.580 INFO:teuthology.orchestra.run.smithi112.stdout: "progress": "16/21 daemons upgraded", 2024-09-16T00:40:52.580 INFO:teuthology.orchestra.run.smithi112.stdout: "message": "Currently upgrading mds daemons", 2024-09-16T00:40:52.580 INFO:teuthology.orchestra.run.smithi112.stdout: "is_paused": false 2024-09-16T00:40:52.580 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:40:52.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.581+0000 7ff8aab6c640 1 -- 172.21.15.112:0/959212515 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff88c06e530 msgr2=0x7ff88c0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:52.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.581+0000 7ff8aab6c640 1 --2- 172.21.15.112:0/959212515 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff88c06e530 0x7ff88c0709e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff898004540 tx=0x7ff898035040 comp rx=0 tx=0).stop 2024-09-16T00:40:52.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.581+0000 7ff8aab6c640 1 -- 172.21.15.112:0/959212515 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8a40fd560 msgr2=0x7ff8a41132d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:52.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.581+0000 7ff8aab6c640 1 --2- 172.21.15.112:0/959212515 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8a40fd560 0x7ff8a41132d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff89c002a50 tx=0x7ff89c004600 comp rx=0 tx=0).stop 2024-09-16T00:40:52.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.582+0000 7ff8aab6c640 1 -- 172.21.15.112:0/959212515 shutdown_connections 2024-09-16T00:40:52.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.582+0000 7ff8aab6c640 1 --2- 172.21.15.112:0/959212515 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff88c06e530 0x7ff88c0709e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:52.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.582+0000 7ff8aab6c640 1 --2- 172.21.15.112:0/959212515 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8a40fdf00 0x7ff8a4113810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:52.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.582+0000 7ff8aab6c640 1 --2- 172.21.15.112:0/959212515 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8a40fd560 0x7ff8a41132d0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:52.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.582+0000 7ff8aab6c640 1 -- 172.21.15.112:0/959212515 >> 172.21.15.112:0/959212515 conn(0x7ff8a40f7000 msgr2=0x7ff8a40f8510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:52.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.582+0000 7ff8aab6c640 1 -- 172.21.15.112:0/959212515 shutdown_connections 2024-09-16T00:40:52.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.582+0000 7ff8aab6c640 1 -- 172.21.15.112:0/959212515 wait complete. 2024-09-16T00:40:52.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.659+0000 7fd11c72e640 1 -- 172.21.15.112:0/2460878707 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd114102530 msgr2=0x7fd114102900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:52.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.659+0000 7fd11c72e640 1 --2- 172.21.15.112:0/2460878707 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd114102530 0x7fd114102900 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd1100048d0 tx=0x7fd110009da0 comp rx=0 tx=0).stop 2024-09-16T00:40:52.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.660+0000 7fd11c72e640 1 -- 172.21.15.112:0/2460878707 shutdown_connections 2024-09-16T00:40:52.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.660+0000 7fd11c72e640 1 --2- 172.21.15.112:0/2460878707 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd114102ed0 0x7fd1140fc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:52.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.660+0000 7fd11c72e640 1 --2- 172.21.15.112:0/2460878707 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd114102530 0x7fd114102900 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:52.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.660+0000 7fd11c72e640 1 -- 172.21.15.112:0/2460878707 >> 172.21.15.112:0/2460878707 conn(0x7fd1140f80f0 msgr2=0x7fd1140fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:52.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.660+0000 7fd11c72e640 1 -- 172.21.15.112:0/2460878707 shutdown_connections 2024-09-16T00:40:52.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.660+0000 7fd11c72e640 1 -- 172.21.15.112:0/2460878707 wait complete. 2024-09-16T00:40:52.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.660+0000 7fd11c72e640 1 Processor -- start 2024-09-16T00:40:52.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.660+0000 7fd11c72e640 1 -- start start 2024-09-16T00:40:52.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.661+0000 7fd11c72e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd114102530 0x7fd11406fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:52.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.661+0000 7fd11c72e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd114102ed0 0x7fd11406ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:52.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.661+0000 7fd11c72e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd114070490 con 0x7fd114102530 2024-09-16T00:40:52.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.661+0000 7fd11c72e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd114070600 con 0x7fd114102ed0 2024-09-16T00:40:52.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.661+0000 7fd11a4ca640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd114102530 0x7fd11406fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:52.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.661+0000 7fd11a4ca640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd114102530 0x7fd11406fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52630/0 (socket says 172.21.15.112:52630) 2024-09-16T00:40:52.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.661+0000 7fd11a4ca640 1 -- 172.21.15.112:0/2091591590 learned_addr learned my addr 172.21.15.112:0/2091591590 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:40:52.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.661+0000 7fd119cc9640 1 --2- 172.21.15.112:0/2091591590 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd114102ed0 0x7fd11406ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:52.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.661+0000 7fd11a4ca640 1 -- 172.21.15.112:0/2091591590 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd114102ed0 msgr2=0x7fd11406ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:52.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.661+0000 7fd11a4ca640 1 --2- 172.21.15.112:0/2091591590 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd114102ed0 0x7fd11406ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:52.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.661+0000 7fd11a4ca640 1 -- 172.21.15.112:0/2091591590 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd110005040 con 0x7fd114102530 2024-09-16T00:40:52.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.661+0000 7fd11a4ca640 1 --2- 172.21.15.112:0/2091591590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd114102530 0x7fd11406fa10 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fd110003080 tx=0x7fd110009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:52.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.661+0000 7fd10affd640 1 -- 172.21.15.112:0/2091591590 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd110038070 con 0x7fd114102530 2024-09-16T00:40:52.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.661+0000 7fd10affd640 1 -- 172.21.15.112:0/2091591590 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fd1100097e0 con 0x7fd114102530 2024-09-16T00:40:52.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.661+0000 7fd11c72e640 1 -- 172.21.15.112:0/2091591590 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd114070830 con 0x7fd114102530 2024-09-16T00:40:52.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.661+0000 7fd10affd640 1 -- 172.21.15.112:0/2091591590 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd11003d420 con 0x7fd114102530 2024-09-16T00:40:52.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.662+0000 7fd11c72e640 1 -- 172.21.15.112:0/2091591590 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd114119890 con 0x7fd114102530 2024-09-16T00:40:52.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.663+0000 7fd11c72e640 1 -- 172.21.15.112:0/2091591590 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1140fdcb0 con 0x7fd114102530 2024-09-16T00:40:52.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.663+0000 7fd10affd640 1 -- 172.21.15.112:0/2091591590 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fd110030020 con 0x7fd114102530 2024-09-16T00:40:52.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.663+0000 7fd10affd640 1 --2- 172.21.15.112:0/2091591590 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd10006e530 0x7fd1000709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:40:52.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.664+0000 7fd10affd640 1 -- 172.21.15.112:0/2091591590 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7fd1100ae880 con 0x7fd114102530 2024-09-16T00:40:52.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.664+0000 7fd119cc9640 1 --2- 172.21.15.112:0/2091591590 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd10006e530 0x7fd1000709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:40:52.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.664+0000 7fd119cc9640 1 --2- 172.21.15.112:0/2091591590 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd10006e530 0x7fd1000709e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd114071350 tx=0x7fd104009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:40:52.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.668+0000 7fd10affd640 1 -- 172.21.15.112:0/2091591590 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fd1100777f0 con 0x7fd114102530 2024-09-16T00:40:52.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.901+0000 7fd11c72e640 1 -- 172.21.15.112:0/2091591590 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd1140fbd30 con 0x7fd114102530 2024-09-16T00:40:52.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.902+0000 7fd10affd640 1 -- 172.21.15.112:0/2091591590 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7fd11007aeb0 con 0x7fd114102530 2024-09-16T00:40:52.902 INFO:teuthology.orchestra.run.smithi112.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-16T00:40:52.902 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-16T00:40:52.902 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs is degraded 2024-09-16T00:40:52.902 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T00:40:52.902 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T00:40:52.902 INFO:teuthology.orchestra.run.smithi112.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-16T00:40:52.903 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-16T00:40:52.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.904+0000 7fd11c72e640 1 -- 172.21.15.112:0/2091591590 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd10006e530 msgr2=0x7fd1000709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:52.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.904+0000 7fd11c72e640 1 --2- 172.21.15.112:0/2091591590 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd10006e530 0x7fd1000709e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd114071350 tx=0x7fd104009040 comp rx=0 tx=0).stop 2024-09-16T00:40:52.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.904+0000 7fd11c72e640 1 -- 172.21.15.112:0/2091591590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd114102530 msgr2=0x7fd11406fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:40:52.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.904+0000 7fd11c72e640 1 --2- 172.21.15.112:0/2091591590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd114102530 0x7fd11406fa10 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fd110003080 tx=0x7fd110009da0 comp rx=0 tx=0).stop 2024-09-16T00:40:52.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.904+0000 7fd11c72e640 1 -- 172.21.15.112:0/2091591590 shutdown_connections 2024-09-16T00:40:52.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.904+0000 7fd11c72e640 1 --2- 172.21.15.112:0/2091591590 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd10006e530 0x7fd1000709e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:52.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.904+0000 7fd11c72e640 1 --2- 172.21.15.112:0/2091591590 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd114102ed0 0x7fd11406ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:52.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.904+0000 7fd11c72e640 1 --2- 172.21.15.112:0/2091591590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd114102530 0x7fd11406fa10 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:40:52.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.904+0000 7fd11c72e640 1 -- 172.21.15.112:0/2091591590 >> 172.21.15.112:0/2091591590 conn(0x7fd1140f80f0 msgr2=0x7fd1140f9100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:40:52.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.904+0000 7fd11c72e640 1 -- 172.21.15.112:0/2091591590 shutdown_connections 2024-09-16T00:40:52.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:40:52.904+0000 7fd11c72e640 1 -- 172.21.15.112:0/2091591590 wait complete. 2024-09-16T00:40:53.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:40:52 smithi112 ceph-mon[97750]: from='client.34772 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:40:53.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:40:52 smithi112 ceph-mon[97750]: from='client.34776 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:40:53.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:40:52 smithi112 ceph-mon[97750]: from='client.34780 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:40:53.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:40:52 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/275486513' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:40:53.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:40:52 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/2753487292' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:40:53.178 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:40:52 smithi169 ceph-mon[77626]: from='client.34772 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:40:53.178 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:40:52 smithi169 ceph-mon[77626]: from='client.34776 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:40:53.178 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:40:52 smithi169 ceph-mon[77626]: from='client.34780 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:40:53.178 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:40:52 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/275486513' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:40:53.178 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:40:52 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/2753487292' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:40:54.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:40:53 smithi112 ceph-mon[97750]: from='client.34792 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:40:54.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:40:53 smithi112 ceph-mon[97750]: pgmap v394: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:40:54.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:40:53 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/2091591590' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:40:54.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:40:53 smithi169 ceph-mon[77626]: from='client.34792 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:40:54.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:40:53 smithi169 ceph-mon[77626]: pgmap v394: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:40:54.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:40:53 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/2091591590' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:40:56.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:40:55 smithi112 ceph-mon[97750]: pgmap v395: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:40:56.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:40:55 smithi169 ceph-mon[77626]: pgmap v395: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:40:58.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:40:57 smithi112 ceph-mon[97750]: pgmap v396: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:40:58.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:40:57 smithi169 ceph-mon[77626]: pgmap v396: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:41:00.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:40:59 smithi112 ceph-mon[97750]: pgmap v397: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:41:00.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:40:59 smithi169 ceph-mon[77626]: pgmap v397: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:41:02.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:41:01 smithi112 ceph-mon[97750]: pgmap v398: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:41:02.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:41:01 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:41:02.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:41:01 smithi169 ceph-mon[77626]: pgmap v398: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:41:02.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:41:01 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:41:04.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:41:03 smithi112 ceph-mon[97750]: pgmap v399: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:41:04.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:41:03 smithi169 ceph-mon[77626]: pgmap v399: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:41:06.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:41:05 smithi112 ceph-mon[97750]: pgmap v400: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:41:06.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:41:05 smithi169 ceph-mon[77626]: pgmap v400: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:41:08.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:41:07 smithi112 ceph-mon[97750]: pgmap v401: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:41:08.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:41:07 smithi169 ceph-mon[77626]: pgmap v401: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:41:10.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:41:09 smithi112 ceph-mon[97750]: pgmap v402: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:41:10.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:41:09 smithi169 ceph-mon[77626]: pgmap v402: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:41:12.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:41:11 smithi112 ceph-mon[97750]: pgmap v403: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:41:12.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:41:11 smithi169 ceph-mon[77626]: pgmap v403: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:41:14.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:41:13 smithi112 ceph-mon[97750]: pgmap v404: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:41:14.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:41:13 smithi169 ceph-mon[77626]: pgmap v404: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:41:16.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:41:15 smithi169 ceph-mon[77626]: pgmap v405: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:41:16.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:41:15 smithi112 ceph-mon[97750]: pgmap v405: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:41:17.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:41:16 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:41:17.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:41:16 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:41:18.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:41:17 smithi169 ceph-mon[77626]: pgmap v406: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:41:18.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:41:17 smithi112 ceph-mon[97750]: pgmap v406: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:41:20.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:41:19 smithi169 ceph-mon[77626]: pgmap v407: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:41:20.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:41:19 smithi112 ceph-mon[97750]: pgmap v407: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:41:22.141 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:41:21 smithi112 ceph-mon[97750]: pgmap v408: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:41:22.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:41:21 smithi169 ceph-mon[77626]: pgmap v408: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:41:22.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:22.993+0000 7f5bbba9a640 1 -- 172.21.15.112:0/2030785458 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bb40fb4a0 msgr2=0x7f5bb40fb910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:22.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:22.993+0000 7f5bbba9a640 1 --2- 172.21.15.112:0/2030785458 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bb40fb4a0 0x7f5bb40fb910 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f5ba40048d0 tx=0x7f5ba4009da0 comp rx=0 tx=0).stop 2024-09-16T00:41:22.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:22.993+0000 7f5bbba9a640 1 -- 172.21.15.112:0/2030785458 shutdown_connections 2024-09-16T00:41:22.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:22.993+0000 7f5bbba9a640 1 --2- 172.21.15.112:0/2030785458 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bb40fb4a0 0x7f5bb40fb910 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:22.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:22.993+0000 7f5bbba9a640 1 --2- 172.21.15.112:0/2030785458 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5bb41014e0 0x7f5bb41018b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:22.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:22.993+0000 7f5bbba9a640 1 -- 172.21.15.112:0/2030785458 >> 172.21.15.112:0/2030785458 conn(0x7f5bb40f7060 msgr2=0x7f5bb40f9470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:22.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:22.993+0000 7f5bbba9a640 1 -- 172.21.15.112:0/2030785458 shutdown_connections 2024-09-16T00:41:22.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:22.993+0000 7f5bbba9a640 1 -- 172.21.15.112:0/2030785458 wait complete. 2024-09-16T00:41:22.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:22.994+0000 7f5bbba9a640 1 Processor -- start 2024-09-16T00:41:22.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:22.994+0000 7f5bbba9a640 1 -- start start 2024-09-16T00:41:22.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:22.994+0000 7f5bbba9a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5bb40fb4a0 0x7f5bb41120b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:22.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:22.994+0000 7f5bbba9a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bb41014e0 0x7f5bb41125f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:22.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:22.994+0000 7f5bbba9a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb4112bf0 con 0x7f5bb41014e0 2024-09-16T00:41:22.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:22.994+0000 7f5bbba9a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb410c030 con 0x7f5bb40fb4a0 2024-09-16T00:41:22.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:22.994+0000 7f5bb9035640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bb41014e0 0x7f5bb41125f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:22.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:22.994+0000 7f5bb9035640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bb41014e0 0x7f5bb41125f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40372/0 (socket says 172.21.15.112:40372) 2024-09-16T00:41:22.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:22.994+0000 7f5bb9035640 1 -- 172.21.15.112:0/3843711132 learned_addr learned my addr 172.21.15.112:0/3843711132 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:41:22.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:22.994+0000 7f5bb9836640 1 --2- 172.21.15.112:0/3843711132 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5bb40fb4a0 0x7f5bb41120b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:22.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:22.995+0000 7f5bb9035640 1 -- 172.21.15.112:0/3843711132 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5bb40fb4a0 msgr2=0x7f5bb41120b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:22.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:22.995+0000 7f5bb9035640 1 --2- 172.21.15.112:0/3843711132 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5bb40fb4a0 0x7f5bb41120b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:22.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:22.995+0000 7f5bb9035640 1 -- 172.21.15.112:0/3843711132 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ba4005040 con 0x7f5bb41014e0 2024-09-16T00:41:22.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:22.995+0000 7f5bb9035640 1 --2- 172.21.15.112:0/3843711132 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bb41014e0 0x7f5bb41125f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f5ba4000c00 tx=0x7f5ba4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:22.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:22.995+0000 7f5baa7fc640 1 -- 172.21.15.112:0/3843711132 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ba403a070 con 0x7f5bb41014e0 2024-09-16T00:41:22.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:22.995+0000 7f5baa7fc640 1 -- 172.21.15.112:0/3843711132 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f5ba4009880 con 0x7f5bb41014e0 2024-09-16T00:41:22.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:22.995+0000 7f5bbba9a640 1 -- 172.21.15.112:0/3843711132 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bb410c2b0 con 0x7f5bb41014e0 2024-09-16T00:41:22.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:22.995+0000 7f5baa7fc640 1 -- 172.21.15.112:0/3843711132 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ba40338f0 con 0x7f5bb41014e0 2024-09-16T00:41:22.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:22.995+0000 7f5bbba9a640 1 -- 172.21.15.112:0/3843711132 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bb410c7f0 con 0x7f5bb41014e0 2024-09-16T00:41:22.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:22.996+0000 7f5bbba9a640 1 -- 172.21.15.112:0/3843711132 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bb40fcbe0 con 0x7f5bb41014e0 2024-09-16T00:41:22.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:22.997+0000 7f5baa7fc640 1 -- 172.21.15.112:0/3843711132 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f5ba4030020 con 0x7f5bb41014e0 2024-09-16T00:41:22.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:22.997+0000 7f5baa7fc640 1 --2- 172.21.15.112:0/3843711132 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5ba006e530 0x7f5ba00709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:22.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:22.997+0000 7f5baa7fc640 1 -- 172.21.15.112:0/3843711132 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f5ba40b0ba0 con 0x7f5bb41014e0 2024-09-16T00:41:22.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:22.997+0000 7f5bb9836640 1 --2- 172.21.15.112:0/3843711132 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5ba006e530 0x7f5ba00709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:22.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:22.998+0000 7f5bb9836640 1 --2- 172.21.15.112:0/3843711132 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5ba006e530 0x7f5ba00709e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5bb00041d0 tx=0x7f5bb0009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:23.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.002+0000 7f5baa7fc640 1 -- 172.21.15.112:0/3843711132 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f5ba4079b10 con 0x7f5bb41014e0 2024-09-16T00:41:23.177 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:41:22 smithi169 ceph-mon[77626]: pgmap v409: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:41:23.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.184+0000 7f5bbba9a640 1 -- 172.21.15.112:0/3843711132 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5bb4108460 con 0x7f5ba006e530 2024-09-16T00:41:23.185 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:41:22 smithi112 ceph-mon[97750]: pgmap v409: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:41:23.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.186+0000 7f5baa7fc640 1 -- 172.21.15.112:0/3843711132 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f5bb4108460 con 0x7f5ba006e530 2024-09-16T00:41:23.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.188+0000 7f5bbba9a640 1 -- 172.21.15.112:0/3843711132 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5ba006e530 msgr2=0x7f5ba00709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:23.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.188+0000 7f5bbba9a640 1 --2- 172.21.15.112:0/3843711132 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5ba006e530 0x7f5ba00709e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5bb00041d0 tx=0x7f5bb0009040 comp rx=0 tx=0).stop 2024-09-16T00:41:23.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.188+0000 7f5bbba9a640 1 -- 172.21.15.112:0/3843711132 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bb41014e0 msgr2=0x7f5bb41125f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:23.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.188+0000 7f5bbba9a640 1 --2- 172.21.15.112:0/3843711132 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bb41014e0 0x7f5bb41125f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f5ba4000c00 tx=0x7f5ba4009da0 comp rx=0 tx=0).stop 2024-09-16T00:41:23.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.189+0000 7f5bbba9a640 1 -- 172.21.15.112:0/3843711132 shutdown_connections 2024-09-16T00:41:23.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.189+0000 7f5bbba9a640 1 --2- 172.21.15.112:0/3843711132 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5ba006e530 0x7f5ba00709e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:23.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.189+0000 7f5bbba9a640 1 --2- 172.21.15.112:0/3843711132 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bb41014e0 0x7f5bb41125f0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:23.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.189+0000 7f5bbba9a640 1 --2- 172.21.15.112:0/3843711132 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5bb40fb4a0 0x7f5bb41120b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:23.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.189+0000 7f5bbba9a640 1 -- 172.21.15.112:0/3843711132 >> 172.21.15.112:0/3843711132 conn(0x7f5bb40f7060 msgr2=0x7f5bb40f8910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:23.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.189+0000 7f5bbba9a640 1 -- 172.21.15.112:0/3843711132 shutdown_connections 2024-09-16T00:41:23.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.189+0000 7f5bbba9a640 1 -- 172.21.15.112:0/3843711132 wait complete. 2024-09-16T00:41:23.197 INFO:teuthology.orchestra.run.smithi112.stdout:true 2024-09-16T00:41:23.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.267+0000 7feea7132640 1 -- 172.21.15.112:0/704392943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feea0102ef0 msgr2=0x7feea00fc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:23.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.267+0000 7feea7132640 1 --2- 172.21.15.112:0/704392943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feea0102ef0 0x7feea00fc5b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fee940048d0 tx=0x7fee94009da0 comp rx=0 tx=0).stop 2024-09-16T00:41:23.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.267+0000 7feea7132640 1 -- 172.21.15.112:0/704392943 shutdown_connections 2024-09-16T00:41:23.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.267+0000 7feea7132640 1 --2- 172.21.15.112:0/704392943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feea0102ef0 0x7feea00fc5b0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:23.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.267+0000 7feea7132640 1 --2- 172.21.15.112:0/704392943 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feea0102550 0x7feea0102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:23.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.267+0000 7feea7132640 1 -- 172.21.15.112:0/704392943 >> 172.21.15.112:0/704392943 conn(0x7feea00f80f0 msgr2=0x7feea00fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:23.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.268+0000 7feea7132640 1 -- 172.21.15.112:0/704392943 shutdown_connections 2024-09-16T00:41:23.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.268+0000 7feea7132640 1 -- 172.21.15.112:0/704392943 wait complete. 2024-09-16T00:41:23.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.268+0000 7feea7132640 1 Processor -- start 2024-09-16T00:41:23.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.268+0000 7feea7132640 1 -- start start 2024-09-16T00:41:23.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.268+0000 7feea7132640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feea0102550 0x7feea0113520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:23.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.268+0000 7feea7132640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feea0102ef0 0x7feea0113a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:23.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.268+0000 7feea7132640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feea01140b0 con 0x7feea0102ef0 2024-09-16T00:41:23.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.268+0000 7feea7132640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feea010d4c0 con 0x7feea0102550 2024-09-16T00:41:23.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.269+0000 7fee9ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feea0102ef0 0x7feea0113a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:23.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.269+0000 7fee9ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feea0102ef0 0x7feea0113a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40382/0 (socket says 172.21.15.112:40382) 2024-09-16T00:41:23.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.269+0000 7fee9ffff640 1 -- 172.21.15.112:0/1867013841 learned_addr learned my addr 172.21.15.112:0/1867013841 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:41:23.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.269+0000 7feea4ece640 1 --2- 172.21.15.112:0/1867013841 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feea0102550 0x7feea0113520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:23.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.269+0000 7fee9ffff640 1 -- 172.21.15.112:0/1867013841 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feea0102550 msgr2=0x7feea0113520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:23.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.269+0000 7fee9ffff640 1 --2- 172.21.15.112:0/1867013841 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feea0102550 0x7feea0113520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:23.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.269+0000 7fee9ffff640 1 -- 172.21.15.112:0/1867013841 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee94005040 con 0x7feea0102ef0 2024-09-16T00:41:23.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.269+0000 7fee9ffff640 1 --2- 172.21.15.112:0/1867013841 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feea0102ef0 0x7feea0113a60 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fee940043d0 tx=0x7fee94009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:23.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.269+0000 7fee9d7fa640 1 -- 172.21.15.112:0/1867013841 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee94038070 con 0x7feea0102ef0 2024-09-16T00:41:23.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.269+0000 7fee9d7fa640 1 -- 172.21.15.112:0/1867013841 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fee940096e0 con 0x7feea0102ef0 2024-09-16T00:41:23.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.269+0000 7fee9d7fa640 1 -- 172.21.15.112:0/1867013841 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee9402c8a0 con 0x7feea0102ef0 2024-09-16T00:41:23.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.269+0000 7feea7132640 1 -- 172.21.15.112:0/1867013841 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feea010d740 con 0x7feea0102ef0 2024-09-16T00:41:23.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.269+0000 7feea7132640 1 -- 172.21.15.112:0/1867013841 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feea010dc30 con 0x7feea0102ef0 2024-09-16T00:41:23.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.270+0000 7feea7132640 1 -- 172.21.15.112:0/1867013841 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feea00fdcd0 con 0x7feea0102ef0 2024-09-16T00:41:23.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.271+0000 7fee9d7fa640 1 -- 172.21.15.112:0/1867013841 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fee94030020 con 0x7feea0102ef0 2024-09-16T00:41:23.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.271+0000 7fee9d7fa640 1 --2- 172.21.15.112:0/1867013841 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fee8806e530 0x7fee880709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:23.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.271+0000 7fee9d7fa640 1 -- 172.21.15.112:0/1867013841 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7fee940ae010 con 0x7feea0102ef0 2024-09-16T00:41:23.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.271+0000 7feea4ece640 1 --2- 172.21.15.112:0/1867013841 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fee8806e530 0x7fee880709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:23.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.272+0000 7feea4ece640 1 --2- 172.21.15.112:0/1867013841 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fee8806e530 0x7fee880709e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fee90007fd0 tx=0x7fee90009e90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:23.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.275+0000 7fee9d7fa640 1 -- 172.21.15.112:0/1867013841 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fee94077000 con 0x7feea0102ef0 2024-09-16T00:41:23.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.455+0000 7feea7132640 1 -- 172.21.15.112:0/1867013841 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7feea0109580 con 0x7fee8806e530 2024-09-16T00:41:23.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.457+0000 7fee9d7fa640 1 -- 172.21.15.112:0/1867013841 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7feea0109580 con 0x7fee8806e530 2024-09-16T00:41:23.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.460+0000 7feea7132640 1 -- 172.21.15.112:0/1867013841 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fee8806e530 msgr2=0x7fee880709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:23.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.460+0000 7feea7132640 1 --2- 172.21.15.112:0/1867013841 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fee8806e530 0x7fee880709e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fee90007fd0 tx=0x7fee90009e90 comp rx=0 tx=0).stop 2024-09-16T00:41:23.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.460+0000 7feea7132640 1 -- 172.21.15.112:0/1867013841 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feea0102ef0 msgr2=0x7feea0113a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:23.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.460+0000 7feea7132640 1 --2- 172.21.15.112:0/1867013841 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feea0102ef0 0x7feea0113a60 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fee940043d0 tx=0x7fee94009da0 comp rx=0 tx=0).stop 2024-09-16T00:41:23.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.460+0000 7feea7132640 1 -- 172.21.15.112:0/1867013841 shutdown_connections 2024-09-16T00:41:23.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.460+0000 7feea7132640 1 --2- 172.21.15.112:0/1867013841 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fee8806e530 0x7fee880709e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:23.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.460+0000 7feea7132640 1 --2- 172.21.15.112:0/1867013841 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feea0102ef0 0x7feea0113a60 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:23.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.460+0000 7feea7132640 1 --2- 172.21.15.112:0/1867013841 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feea0102550 0x7feea0113520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:23.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.460+0000 7feea7132640 1 -- 172.21.15.112:0/1867013841 >> 172.21.15.112:0/1867013841 conn(0x7feea00f80f0 msgr2=0x7feea00f9150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:23.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.460+0000 7feea7132640 1 -- 172.21.15.112:0/1867013841 shutdown_connections 2024-09-16T00:41:23.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.460+0000 7feea7132640 1 -- 172.21.15.112:0/1867013841 wait complete. 2024-09-16T00:41:23.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.539+0000 7f4973e04640 1 -- 172.21.15.112:0/2402392818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f496c1014d0 msgr2=0x7f496c1018a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:23.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.539+0000 7f4973e04640 1 --2- 172.21.15.112:0/2402392818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f496c1014d0 0x7f496c1018a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f49680048d0 tx=0x7f4968009da0 comp rx=0 tx=0).stop 2024-09-16T00:41:23.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.539+0000 7f4973e04640 1 -- 172.21.15.112:0/2402392818 shutdown_connections 2024-09-16T00:41:23.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.539+0000 7f4973e04640 1 --2- 172.21.15.112:0/2402392818 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f496c0fb490 0x7f496c0fb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:23.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.539+0000 7f4973e04640 1 --2- 172.21.15.112:0/2402392818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f496c1014d0 0x7f496c1018a0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:23.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.539+0000 7f4973e04640 1 -- 172.21.15.112:0/2402392818 >> 172.21.15.112:0/2402392818 conn(0x7f496c0f7010 msgr2=0x7f496c0f9420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:23.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.539+0000 7f4973e04640 1 -- 172.21.15.112:0/2402392818 shutdown_connections 2024-09-16T00:41:23.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.539+0000 7f4973e04640 1 -- 172.21.15.112:0/2402392818 wait complete. 2024-09-16T00:41:23.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.539+0000 7f4973e04640 1 Processor -- start 2024-09-16T00:41:23.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.540+0000 7f4973e04640 1 -- start start 2024-09-16T00:41:23.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.540+0000 7f4973e04640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f496c0fb490 0x7f496c0f74b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:23.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.540+0000 7f4973e04640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f496c1014d0 0x7f496c0f79f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:23.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.540+0000 7f4973e04640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f496c0f7f30 con 0x7f496c1014d0 2024-09-16T00:41:23.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.540+0000 7f4973e04640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f496c0f80a0 con 0x7f496c0fb490 2024-09-16T00:41:23.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.540+0000 7f497139f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f496c1014d0 0x7f496c0f79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:23.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.540+0000 7f497139f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f496c1014d0 0x7f496c0f79f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40394/0 (socket says 172.21.15.112:40394) 2024-09-16T00:41:23.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.540+0000 7f497139f640 1 -- 172.21.15.112:0/4187258081 learned_addr learned my addr 172.21.15.112:0/4187258081 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:41:23.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.540+0000 7f4971ba0640 1 --2- 172.21.15.112:0/4187258081 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f496c0fb490 0x7f496c0f74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:23.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.540+0000 7f497139f640 1 -- 172.21.15.112:0/4187258081 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f496c0fb490 msgr2=0x7f496c0f74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:23.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.540+0000 7f497139f640 1 --2- 172.21.15.112:0/4187258081 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f496c0fb490 0x7f496c0f74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:23.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.540+0000 7f497139f640 1 -- 172.21.15.112:0/4187258081 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4968005040 con 0x7f496c1014d0 2024-09-16T00:41:23.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.540+0000 7f497139f640 1 --2- 172.21.15.112:0/4187258081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f496c1014d0 0x7f496c0f79f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f495c00eb70 tx=0x7f495c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:23.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.541+0000 7f49627fc640 1 -- 172.21.15.112:0/4187258081 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f495c007c30 con 0x7f496c1014d0 2024-09-16T00:41:23.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.541+0000 7f49627fc640 1 -- 172.21.15.112:0/4187258081 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f495c00f040 con 0x7f496c1014d0 2024-09-16T00:41:23.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.541+0000 7f4973e04640 1 -- 172.21.15.112:0/4187258081 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f496c0f82d0 con 0x7f496c1014d0 2024-09-16T00:41:23.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.541+0000 7f49627fc640 1 -- 172.21.15.112:0/4187258081 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f495c00ca60 con 0x7f496c1014d0 2024-09-16T00:41:23.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.541+0000 7f4973e04640 1 -- 172.21.15.112:0/4187258081 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f496c119890 con 0x7f496c1014d0 2024-09-16T00:41:23.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.541+0000 7f4973e04640 1 -- 172.21.15.112:0/4187258081 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f496c0fcbd0 con 0x7f496c1014d0 2024-09-16T00:41:23.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.542+0000 7f49627fc640 1 -- 172.21.15.112:0/4187258081 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f495c00cbc0 con 0x7f496c1014d0 2024-09-16T00:41:23.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.543+0000 7f49627fc640 1 --2- 172.21.15.112:0/4187258081 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f495806e530 0x7f49580709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:23.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.543+0000 7f49627fc640 1 -- 172.21.15.112:0/4187258081 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f495c08dee0 con 0x7f496c1014d0 2024-09-16T00:41:23.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.543+0000 7f4971ba0640 1 --2- 172.21.15.112:0/4187258081 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f495806e530 0x7f49580709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:23.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.543+0000 7f4971ba0640 1 --2- 172.21.15.112:0/4187258081 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f495806e530 0x7f49580709e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4968004540 tx=0x7f4968035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:23.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.547+0000 7f49627fc640 1 -- 172.21.15.112:0/4187258081 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f495c056f20 con 0x7f496c1014d0 2024-09-16T00:41:23.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.724+0000 7f4973e04640 1 -- 172.21.15.112:0/4187258081 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f496c1084e0 con 0x7f495806e530 2024-09-16T00:41:23.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.733+0000 7f49627fc640 1 -- 172.21.15.112:0/4187258081 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7f496c1084e0 con 0x7f495806e530 2024-09-16T00:41:23.733 INFO:teuthology.orchestra.run.smithi112.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T00:41:23.733 INFO:teuthology.orchestra.run.smithi112.stdout:alertmanager.smithi112 smithi112 *:9093,9094 running (14m) 7m ago 23m 24.7M - 0.25.0 c8568f914cd2 d85ea0562022 2024-09-16T00:41:23.736 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi112 smithi112 running (11m) 7m ago 23m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c e4fe58f9702a 2024-09-16T00:41:23.736 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi169 smithi169 running (11m) 6m ago 22m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c 3ba2c81d527f 2024-09-16T00:41:23.736 INFO:teuthology.orchestra.run.smithi112.stdout:grafana.smithi112 smithi112 *:3000 running (13m) 7m ago 23m 82.1M - 9.4.12 2bacad6d85d8 51f9d2013da2 2024-09-16T00:41:23.736 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.hxfwkh smithi112 running (7m) 7m ago 17m 14.2M - 19.1.1-326-g8921c8ef f1455bd0219c 1723069902d6 2024-09-16T00:41:23.736 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.ithwzj smithi112 running (7m) 7m ago 17m 12.2M - 19.1.1-326-g8921c8ef f1455bd0219c e7aad5b81db5 2024-09-16T00:41:23.736 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.gawwgw smithi169 running (7m) 6m ago 17m 15.7M - 19.1.1-326-g8921c8ef f1455bd0219c 469be459ee12 2024-09-16T00:41:23.736 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.lgeqrj smithi169 running (6m) 6m ago 17m 13.3M - 19.1.1-326-g8921c8ef f1455bd0219c 31f0fbd2395d 2024-09-16T00:41:23.736 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi112.xrkkbc smithi112 *:8443,9283,8765 running (13m) 7m ago 24m 504M - 19.1.1-326-g8921c8ef f1455bd0219c a981f7fd2810 2024-09-16T00:41:23.737 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi169.mtrobk smithi169 *:8443,9283,8765 running (13m) 6m ago 22m 442M - 19.1.1-326-g8921c8ef f1455bd0219c 4096e5369c2d 2024-09-16T00:41:23.737 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi112 smithi112 running (12m) 7m ago 24m 49.8M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 5aaec033c042 2024-09-16T00:41:23.737 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi169 smithi169 running (11m) 6m ago 22m 38.9M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 7bc609155f3f 2024-09-16T00:41:23.737 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi112 smithi112 *:9100 running (14m) 7m ago 23m 15.5M - 1.5.0 0da6a335fe13 6404d1d601e2 2024-09-16T00:41:23.737 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi169 smithi169 *:9100 running (14m) 6m ago 22m 15.4M - 1.5.0 0da6a335fe13 5505baf702a6 2024-09-16T00:41:23.737 INFO:teuthology.orchestra.run.smithi112.stdout:osd.0 smithi112 running (10m) 7m ago 21m 105M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 53efc9397017 2024-09-16T00:41:23.737 INFO:teuthology.orchestra.run.smithi112.stdout:osd.1 smithi112 running (10m) 7m ago 21m 98.9M 1557M 19.1.1-326-g8921c8ef f1455bd0219c fe6f1a4a2cf4 2024-09-16T00:41:23.737 INFO:teuthology.orchestra.run.smithi112.stdout:osd.2 smithi112 running (9m) 7m ago 20m 102M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 5f7863712057 2024-09-16T00:41:23.737 INFO:teuthology.orchestra.run.smithi112.stdout:osd.3 smithi169 running (9m) 6m ago 20m 92.1M 2581M 19.1.1-326-g8921c8ef f1455bd0219c ee07b5577810 2024-09-16T00:41:23.737 INFO:teuthology.orchestra.run.smithi112.stdout:osd.4 smithi169 running (8m) 6m ago 19m 103M 2581M 19.1.1-326-g8921c8ef f1455bd0219c 769f026a54eb 2024-09-16T00:41:23.737 INFO:teuthology.orchestra.run.smithi112.stdout:osd.5 smithi169 running (8m) 6m ago 19m 125M 2581M 19.1.1-326-g8921c8ef f1455bd0219c a6007c13cf78 2024-09-16T00:41:23.737 INFO:teuthology.orchestra.run.smithi112.stdout:prometheus.smithi112 smithi112 *:9095 running (13m) 7m ago 22m 64.1M - 2.43.0 a07b618ecd1d 7a1d42e60d2b 2024-09-16T00:41:23.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.735+0000 7f4973e04640 1 -- 172.21.15.112:0/4187258081 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f495806e530 msgr2=0x7f49580709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:23.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.735+0000 7f4973e04640 1 --2- 172.21.15.112:0/4187258081 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f495806e530 0x7f49580709e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4968004540 tx=0x7f4968035040 comp rx=0 tx=0).stop 2024-09-16T00:41:23.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.735+0000 7f4973e04640 1 -- 172.21.15.112:0/4187258081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f496c1014d0 msgr2=0x7f496c0f79f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:23.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.735+0000 7f4973e04640 1 --2- 172.21.15.112:0/4187258081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f496c1014d0 0x7f496c0f79f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f495c00eb70 tx=0x7f495c00b040 comp rx=0 tx=0).stop 2024-09-16T00:41:23.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.735+0000 7f4973e04640 1 -- 172.21.15.112:0/4187258081 shutdown_connections 2024-09-16T00:41:23.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.735+0000 7f4973e04640 1 --2- 172.21.15.112:0/4187258081 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f495806e530 0x7f49580709e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:23.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.735+0000 7f4973e04640 1 --2- 172.21.15.112:0/4187258081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f496c1014d0 0x7f496c0f79f0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:23.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.735+0000 7f4973e04640 1 --2- 172.21.15.112:0/4187258081 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f496c0fb490 0x7f496c0f74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:23.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.735+0000 7f4973e04640 1 -- 172.21.15.112:0/4187258081 >> 172.21.15.112:0/4187258081 conn(0x7f496c0f7010 msgr2=0x7f496c1052c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:23.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.735+0000 7f4973e04640 1 -- 172.21.15.112:0/4187258081 shutdown_connections 2024-09-16T00:41:23.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.735+0000 7f4973e04640 1 -- 172.21.15.112:0/4187258081 wait complete. 2024-09-16T00:41:23.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.812+0000 7fcf2f4e8640 1 -- 172.21.15.112:0/3236829221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf280fcf80 msgr2=0x7fcf280fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:23.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.812+0000 7fcf2f4e8640 1 --2- 172.21.15.112:0/3236829221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf280fcf80 0x7fcf280fd3d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fcf240048d0 tx=0x7fcf24009da0 comp rx=0 tx=0).stop 2024-09-16T00:41:23.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.813+0000 7fcf2f4e8640 1 -- 172.21.15.112:0/3236829221 shutdown_connections 2024-09-16T00:41:23.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.813+0000 7fcf2f4e8640 1 --2- 172.21.15.112:0/3236829221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf280fcf80 0x7fcf280fd3d0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:23.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.813+0000 7fcf2f4e8640 1 --2- 172.21.15.112:0/3236829221 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf280fb810 0x7fcf280fbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:23.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.813+0000 7fcf2f4e8640 1 -- 172.21.15.112:0/3236829221 >> 172.21.15.112:0/3236829221 conn(0x7fcf280723b0 msgr2=0x7fcf280727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:23.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.813+0000 7fcf2f4e8640 1 -- 172.21.15.112:0/3236829221 shutdown_connections 2024-09-16T00:41:23.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.813+0000 7fcf2f4e8640 1 -- 172.21.15.112:0/3236829221 wait complete. 2024-09-16T00:41:23.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.813+0000 7fcf2f4e8640 1 Processor -- start 2024-09-16T00:41:23.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.813+0000 7fcf2f4e8640 1 -- start start 2024-09-16T00:41:23.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.813+0000 7fcf2f4e8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf280fb810 0x7fcf28113150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:23.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.813+0000 7fcf2f4e8640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf280fcf80 0x7fcf28113690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:23.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.813+0000 7fcf2f4e8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf28113c90 con 0x7fcf280fb810 2024-09-16T00:41:23.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.813+0000 7fcf2f4e8640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2810d0d0 con 0x7fcf280fcf80 2024-09-16T00:41:23.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.813+0000 7fcf2d284640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf280fb810 0x7fcf28113150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:23.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.813+0000 7fcf2d284640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf280fb810 0x7fcf28113150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40412/0 (socket says 172.21.15.112:40412) 2024-09-16T00:41:23.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.813+0000 7fcf2d284640 1 -- 172.21.15.112:0/1960207546 learned_addr learned my addr 172.21.15.112:0/1960207546 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:41:23.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.814+0000 7fcf2ca83640 1 --2- 172.21.15.112:0/1960207546 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf280fcf80 0x7fcf28113690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:23.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.814+0000 7fcf2d284640 1 -- 172.21.15.112:0/1960207546 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf280fcf80 msgr2=0x7fcf28113690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:23.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.814+0000 7fcf2d284640 1 --2- 172.21.15.112:0/1960207546 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf280fcf80 0x7fcf28113690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:23.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.814+0000 7fcf2d284640 1 -- 172.21.15.112:0/1960207546 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf24005040 con 0x7fcf280fb810 2024-09-16T00:41:23.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.814+0000 7fcf2d284640 1 --2- 172.21.15.112:0/1960207546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf280fb810 0x7fcf28113150 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fcf1800c790 tx=0x7fcf180043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:23.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.814+0000 7fcf1dffb640 1 -- 172.21.15.112:0/1960207546 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf1800a7a0 con 0x7fcf280fb810 2024-09-16T00:41:23.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.814+0000 7fcf1dffb640 1 -- 172.21.15.112:0/1960207546 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fcf1800ada0 con 0x7fcf280fb810 2024-09-16T00:41:23.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.814+0000 7fcf2f4e8640 1 -- 172.21.15.112:0/1960207546 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf2810d3b0 con 0x7fcf280fb810 2024-09-16T00:41:23.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.814+0000 7fcf1dffb640 1 -- 172.21.15.112:0/1960207546 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf180113a0 con 0x7fcf280fb810 2024-09-16T00:41:23.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.814+0000 7fcf2f4e8640 1 -- 172.21.15.112:0/1960207546 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf2810d920 con 0x7fcf280fb810 2024-09-16T00:41:23.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.815+0000 7fcf2f4e8640 1 -- 172.21.15.112:0/1960207546 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf280fcbc0 con 0x7fcf280fb810 2024-09-16T00:41:23.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.816+0000 7fcf1dffb640 1 -- 172.21.15.112:0/1960207546 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fcf1800a900 con 0x7fcf280fb810 2024-09-16T00:41:23.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.816+0000 7fcf1dffb640 1 --2- 172.21.15.112:0/1960207546 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fcf1406e530 0x7fcf140709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:23.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.816+0000 7fcf1dffb640 1 -- 172.21.15.112:0/1960207546 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7fcf1808e3a0 con 0x7fcf280fb810 2024-09-16T00:41:23.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.816+0000 7fcf2ca83640 1 --2- 172.21.15.112:0/1960207546 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fcf1406e530 0x7fcf140709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:23.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.817+0000 7fcf2ca83640 1 --2- 172.21.15.112:0/1960207546 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fcf1406e530 0x7fcf140709e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcf28072890 tx=0x7fcf24035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:23.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:23.821+0000 7fcf1dffb640 1 -- 172.21.15.112:0/1960207546 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fcf18057430 con 0x7fcf280fb810 2024-09-16T00:41:24.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.055+0000 7fcf2f4e8640 1 -- 172.21.15.112:0/1960207546 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fcf2804b750 con 0x7fcf280fb810 2024-09-16T00:41:24.056 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:41:23 smithi112 ceph-mon[97750]: from='client.34800 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:41:24.056 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:41:23 smithi112 ceph-mon[97750]: from='client.34804 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:41:24.056 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:41:23 smithi112 ceph-mon[97750]: from='client.34808 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:41:24.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.056+0000 7fcf1dffb640 1 -- 172.21.15.112:0/1960207546 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7fcf1805aaf0 con 0x7fcf280fb810 2024-09-16T00:41:24.056 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:41:24.057 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": { 2024-09-16T00:41:24.057 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:41:24.057 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:41:24.057 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": { 2024-09-16T00:41:24.057 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:41:24.057 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:41:24.057 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": { 2024-09-16T00:41:24.057 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 6 2024-09-16T00:41:24.057 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:41:24.057 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": { 2024-09-16T00:41:24.057 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 4 2024-09-16T00:41:24.057 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:41:24.057 INFO:teuthology.orchestra.run.smithi112.stdout: "overall": { 2024-09-16T00:41:24.057 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 14 2024-09-16T00:41:24.058 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-16T00:41:24.058 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:41:24.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.058+0000 7fcf2f4e8640 1 -- 172.21.15.112:0/1960207546 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fcf1406e530 msgr2=0x7fcf140709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:24.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.058+0000 7fcf2f4e8640 1 --2- 172.21.15.112:0/1960207546 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fcf1406e530 0x7fcf140709e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcf28072890 tx=0x7fcf24035040 comp rx=0 tx=0).stop 2024-09-16T00:41:24.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.058+0000 7fcf2f4e8640 1 -- 172.21.15.112:0/1960207546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf280fb810 msgr2=0x7fcf28113150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:24.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.058+0000 7fcf2f4e8640 1 --2- 172.21.15.112:0/1960207546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf280fb810 0x7fcf28113150 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fcf1800c790 tx=0x7fcf180043c0 comp rx=0 tx=0).stop 2024-09-16T00:41:24.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.058+0000 7fcf2f4e8640 1 -- 172.21.15.112:0/1960207546 shutdown_connections 2024-09-16T00:41:24.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.058+0000 7fcf2f4e8640 1 --2- 172.21.15.112:0/1960207546 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fcf1406e530 0x7fcf140709e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:24.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.058+0000 7fcf2f4e8640 1 --2- 172.21.15.112:0/1960207546 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf280fcf80 0x7fcf28113690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:24.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.058+0000 7fcf2f4e8640 1 --2- 172.21.15.112:0/1960207546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf280fb810 0x7fcf28113150 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:24.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.058+0000 7fcf2f4e8640 1 -- 172.21.15.112:0/1960207546 >> 172.21.15.112:0/1960207546 conn(0x7fcf280723b0 msgr2=0x7fcf28101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:24.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.058+0000 7fcf2f4e8640 1 -- 172.21.15.112:0/1960207546 shutdown_connections 2024-09-16T00:41:24.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.058+0000 7fcf2f4e8640 1 -- 172.21.15.112:0/1960207546 wait complete. 2024-09-16T00:41:24.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.136+0000 7f53935a4640 1 -- 172.21.15.112:0/430190648 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f538c0fb490 msgr2=0x7f538c0fb900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:24.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.136+0000 7f53935a4640 1 --2- 172.21.15.112:0/430190648 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f538c0fb490 0x7f538c0fb900 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f537c0048d0 tx=0x7f537c009da0 comp rx=0 tx=0).stop 2024-09-16T00:41:24.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.137+0000 7f53935a4640 1 -- 172.21.15.112:0/430190648 shutdown_connections 2024-09-16T00:41:24.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.137+0000 7f53935a4640 1 --2- 172.21.15.112:0/430190648 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f538c0fb490 0x7f538c0fb900 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:24.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.137+0000 7f53935a4640 1 --2- 172.21.15.112:0/430190648 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f538c1014d0 0x7f538c1018a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:24.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.137+0000 7f53935a4640 1 -- 172.21.15.112:0/430190648 >> 172.21.15.112:0/430190648 conn(0x7f538c0f7010 msgr2=0x7f538c0f9420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:24.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.137+0000 7f53935a4640 1 -- 172.21.15.112:0/430190648 shutdown_connections 2024-09-16T00:41:24.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.137+0000 7f53935a4640 1 -- 172.21.15.112:0/430190648 wait complete. 2024-09-16T00:41:24.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.137+0000 7f53935a4640 1 Processor -- start 2024-09-16T00:41:24.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.137+0000 7f53935a4640 1 -- start start 2024-09-16T00:41:24.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.137+0000 7f53935a4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f538c0fb490 0x7f538c10e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:24.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.137+0000 7f53935a4640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f538c1014d0 0x7f538c10e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:24.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.137+0000 7f53935a4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f538c10ec80 con 0x7f538c0fb490 2024-09-16T00:41:24.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.137+0000 7f53935a4640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f538c10edf0 con 0x7f538c1014d0 2024-09-16T00:41:24.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.138+0000 7f5391340640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f538c0fb490 0x7f538c10e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:24.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.138+0000 7f5391340640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f538c0fb490 0x7f538c10e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40440/0 (socket says 172.21.15.112:40440) 2024-09-16T00:41:24.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.138+0000 7f5391340640 1 -- 172.21.15.112:0/1403806186 learned_addr learned my addr 172.21.15.112:0/1403806186 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:41:24.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.138+0000 7f5390b3f640 1 --2- 172.21.15.112:0/1403806186 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f538c1014d0 0x7f538c10e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:24.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.138+0000 7f5391340640 1 -- 172.21.15.112:0/1403806186 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f538c1014d0 msgr2=0x7f538c10e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:24.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.138+0000 7f5391340640 1 --2- 172.21.15.112:0/1403806186 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f538c1014d0 0x7f538c10e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:24.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.138+0000 7f5391340640 1 -- 172.21.15.112:0/1403806186 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f537c005040 con 0x7f538c0fb490 2024-09-16T00:41:24.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.138+0000 7f5390b3f640 1 --2- 172.21.15.112:0/1403806186 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f538c1014d0 0x7f538c10e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:41:24.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.138+0000 7f5391340640 1 --2- 172.21.15.112:0/1403806186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f538c0fb490 0x7f538c10e140 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f538800c830 tx=0x7f5388004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:24.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.138+0000 7f5381ffb640 1 -- 172.21.15.112:0/1403806186 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f538800a850 con 0x7f538c0fb490 2024-09-16T00:41:24.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.138+0000 7f5381ffb640 1 -- 172.21.15.112:0/1403806186 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f538800ae50 con 0x7f538c0fb490 2024-09-16T00:41:24.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.138+0000 7f53935a4640 1 -- 172.21.15.112:0/1403806186 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f538c1171b0 con 0x7f538c0fb490 2024-09-16T00:41:24.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.138+0000 7f5381ffb640 1 -- 172.21.15.112:0/1403806186 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f53880113a0 con 0x7f538c0fb490 2024-09-16T00:41:24.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.139+0000 7f53935a4640 1 -- 172.21.15.112:0/1403806186 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f538c1176f0 con 0x7f538c0fb490 2024-09-16T00:41:24.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.139+0000 7f53935a4640 1 -- 172.21.15.112:0/1403806186 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f538c0fcbd0 con 0x7f538c0fb490 2024-09-16T00:41:24.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.140+0000 7f5381ffb640 1 -- 172.21.15.112:0/1403806186 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f5388012400 con 0x7f538c0fb490 2024-09-16T00:41:24.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.140+0000 7f5381ffb640 1 --2- 172.21.15.112:0/1403806186 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f537806e530 0x7f53780709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:24.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.140+0000 7f5381ffb640 1 -- 172.21.15.112:0/1403806186 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f538808e530 con 0x7f538c0fb490 2024-09-16T00:41:24.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.140+0000 7f5390b3f640 1 --2- 172.21.15.112:0/1403806186 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f537806e530 0x7f53780709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:24.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.141+0000 7f5390b3f640 1 --2- 172.21.15.112:0/1403806186 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f537806e530 0x7f53780709e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f538c0664f0 tx=0x7f537c035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:24.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.145+0000 7f5381ffb640 1 -- 172.21.15.112:0/1403806186 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f5388057520 con 0x7f538c0fb490 2024-09-16T00:41:24.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:41:23 smithi169 ceph-mon[77626]: from='client.34800 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:41:24.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:41:23 smithi169 ceph-mon[77626]: from='client.34804 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:41:24.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:41:23 smithi169 ceph-mon[77626]: from='client.34808 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:41:24.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.344+0000 7f53935a4640 1 -- 172.21.15.112:0/1403806186 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f538c0662a0 con 0x7f538c0fb490 2024-09-16T00:41:24.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.344+0000 7f5381ffb640 1 -- 172.21.15.112:0/1403806186 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1914 (secure 0 0 0) 0x7f538805abe0 con 0x7f538c0fb490 2024-09-16T00:41:24.345 INFO:teuthology.orchestra.run.smithi112.stdout:e34 2024-09-16T00:41:24.345 INFO:teuthology.orchestra.run.smithi112.stdout:btime 2024-09-16T00:34:26:619254+0000 2024-09-16T00:41:24.345 INFO:teuthology.orchestra.run.smithi112.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T00:41:24.345 INFO:teuthology.orchestra.run.smithi112.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:41:24.345 INFO:teuthology.orchestra.run.smithi112.stdout:legacy client fscid: 1 2024-09-16T00:41:24.345 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:41:24.345 INFO:teuthology.orchestra.run.smithi112.stdout:Filesystem 'cephfs' (1) 2024-09-16T00:41:24.346 INFO:teuthology.orchestra.run.smithi112.stdout:fs_name cephfs 2024-09-16T00:41:24.346 INFO:teuthology.orchestra.run.smithi112.stdout:epoch 25 2024-09-16T00:41:24.346 INFO:teuthology.orchestra.run.smithi112.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-16T00:41:24.346 INFO:teuthology.orchestra.run.smithi112.stdout:created 2024-09-16T00:23:22.328199+0000 2024-09-16T00:41:24.346 INFO:teuthology.orchestra.run.smithi112.stdout:modified 2024-09-16T00:33:29.012991+0000 2024-09-16T00:41:24.346 INFO:teuthology.orchestra.run.smithi112.stdout:tableserver 0 2024-09-16T00:41:24.346 INFO:teuthology.orchestra.run.smithi112.stdout:root 0 2024-09-16T00:41:24.346 INFO:teuthology.orchestra.run.smithi112.stdout:session_timeout 60 2024-09-16T00:41:24.346 INFO:teuthology.orchestra.run.smithi112.stdout:session_autoclose 300 2024-09-16T00:41:24.355 INFO:teuthology.orchestra.run.smithi112.stdout:max_file_size 1099511627776 2024-09-16T00:41:24.355 INFO:teuthology.orchestra.run.smithi112.stdout:max_xattr_size 65536 2024-09-16T00:41:24.355 INFO:teuthology.orchestra.run.smithi112.stdout:required_client_features {} 2024-09-16T00:41:24.355 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure 0 2024-09-16T00:41:24.355 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure_osd_epoch 82 2024-09-16T00:41:24.355 INFO:teuthology.orchestra.run.smithi112.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:41:24.355 INFO:teuthology.orchestra.run.smithi112.stdout:max_mds 2 2024-09-16T00:41:24.356 INFO:teuthology.orchestra.run.smithi112.stdout:in 0,1 2024-09-16T00:41:24.356 INFO:teuthology.orchestra.run.smithi112.stdout:up {} 2024-09-16T00:41:24.356 INFO:teuthology.orchestra.run.smithi112.stdout:failed 0,1 2024-09-16T00:41:24.356 INFO:teuthology.orchestra.run.smithi112.stdout:damaged 2024-09-16T00:41:24.356 INFO:teuthology.orchestra.run.smithi112.stdout:stopped 2024-09-16T00:41:24.356 INFO:teuthology.orchestra.run.smithi112.stdout:data_pools [3] 2024-09-16T00:41:24.356 INFO:teuthology.orchestra.run.smithi112.stdout:metadata_pool 2 2024-09-16T00:41:24.356 INFO:teuthology.orchestra.run.smithi112.stdout:inline_data enabled 2024-09-16T00:41:24.356 INFO:teuthology.orchestra.run.smithi112.stdout:balancer 2024-09-16T00:41:24.356 INFO:teuthology.orchestra.run.smithi112.stdout:bal_rank_mask -1 2024-09-16T00:41:24.356 INFO:teuthology.orchestra.run.smithi112.stdout:standby_count_wanted 1 2024-09-16T00:41:24.356 INFO:teuthology.orchestra.run.smithi112.stdout:qdb_cluster leader: 0 members: 2024-09-16T00:41:24.356 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:41:24.357 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:41:24.357 INFO:teuthology.orchestra.run.smithi112.stdout:Standby daemons: 2024-09-16T00:41:24.357 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:41:24.357 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.ithwzj{-1:34392} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.112:6826/476229221,v1:172.21.15.112:6827/476229221] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:41:24.357 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.hxfwkh{-1:34404} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.112:6828/2356899876,v1:172.21.15.112:6829/2356899876] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:41:24.357 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.gawwgw{-1:44343} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.169:6824/1635329957,v1:172.21.15.169:6825/1635329957] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:41:24.357 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.lgeqrj{-1:44347} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.169:6826/184810031,v1:172.21.15.169:6827/184810031] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:41:24.358 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 34 2024-09-16T00:41:24.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.347+0000 7f53935a4640 1 -- 172.21.15.112:0/1403806186 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f537806e530 msgr2=0x7f53780709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:24.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.347+0000 7f53935a4640 1 --2- 172.21.15.112:0/1403806186 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f537806e530 0x7f53780709e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f538c0664f0 tx=0x7f537c035040 comp rx=0 tx=0).stop 2024-09-16T00:41:24.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.347+0000 7f53935a4640 1 -- 172.21.15.112:0/1403806186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f538c0fb490 msgr2=0x7f538c10e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:24.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.347+0000 7f53935a4640 1 --2- 172.21.15.112:0/1403806186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f538c0fb490 0x7f538c10e140 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f538800c830 tx=0x7f5388004430 comp rx=0 tx=0).stop 2024-09-16T00:41:24.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.347+0000 7f53935a4640 1 -- 172.21.15.112:0/1403806186 shutdown_connections 2024-09-16T00:41:24.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.347+0000 7f53935a4640 1 --2- 172.21.15.112:0/1403806186 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f537806e530 0x7f53780709e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:24.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.347+0000 7f53935a4640 1 --2- 172.21.15.112:0/1403806186 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f538c1014d0 0x7f538c10e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:24.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.347+0000 7f53935a4640 1 --2- 172.21.15.112:0/1403806186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f538c0fb490 0x7f538c10e140 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:24.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.347+0000 7f53935a4640 1 -- 172.21.15.112:0/1403806186 >> 172.21.15.112:0/1403806186 conn(0x7f538c0f7010 msgr2=0x7f538c0f8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:24.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.347+0000 7f53935a4640 1 -- 172.21.15.112:0/1403806186 shutdown_connections 2024-09-16T00:41:24.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.347+0000 7f53935a4640 1 -- 172.21.15.112:0/1403806186 wait complete. 2024-09-16T00:41:24.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.427+0000 7f31b61a1640 1 -- 172.21.15.112:0/544201797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31b0101c20 msgr2=0x7f31b00fb2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:24.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.427+0000 7f31b61a1640 1 --2- 172.21.15.112:0/544201797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31b0101c20 0x7f31b00fb2b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f31ac0048d0 tx=0x7f31ac009da0 comp rx=0 tx=0).stop 2024-09-16T00:41:24.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.427+0000 7f31b61a1640 1 -- 172.21.15.112:0/544201797 shutdown_connections 2024-09-16T00:41:24.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.427+0000 7f31b61a1640 1 --2- 172.21.15.112:0/544201797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31b0101c20 0x7f31b00fb2b0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:24.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.427+0000 7f31b61a1640 1 --2- 172.21.15.112:0/544201797 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31b0101280 0x7f31b0101650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:24.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.427+0000 7f31b61a1640 1 -- 172.21.15.112:0/544201797 >> 172.21.15.112:0/544201797 conn(0x7f31b00f6e30 msgr2=0x7f31b00f9240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:24.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.427+0000 7f31b61a1640 1 -- 172.21.15.112:0/544201797 shutdown_connections 2024-09-16T00:41:24.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.427+0000 7f31b61a1640 1 -- 172.21.15.112:0/544201797 wait complete. 2024-09-16T00:41:24.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.428+0000 7f31b61a1640 1 Processor -- start 2024-09-16T00:41:24.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.428+0000 7f31b61a1640 1 -- start start 2024-09-16T00:41:24.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.428+0000 7f31b61a1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31b0101280 0x7f31b0111f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:24.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.428+0000 7f31b61a1640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31b0101c20 0x7f31b0112440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:24.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.428+0000 7f31b61a1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b0112a40 con 0x7f31b0101280 2024-09-16T00:41:24.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.428+0000 7f31b61a1640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b010be80 con 0x7f31b0101c20 2024-09-16T00:41:24.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.428+0000 7f31b519f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31b0101280 0x7f31b0111f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:24.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.428+0000 7f31b519f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31b0101280 0x7f31b0111f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40460/0 (socket says 172.21.15.112:40460) 2024-09-16T00:41:24.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.428+0000 7f31b519f640 1 -- 172.21.15.112:0/1299759851 learned_addr learned my addr 172.21.15.112:0/1299759851 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:41:24.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.428+0000 7f31b499e640 1 --2- 172.21.15.112:0/1299759851 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31b0101c20 0x7f31b0112440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:24.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.428+0000 7f31b519f640 1 -- 172.21.15.112:0/1299759851 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31b0101c20 msgr2=0x7f31b0112440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:24.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.428+0000 7f31b519f640 1 --2- 172.21.15.112:0/1299759851 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31b0101c20 0x7f31b0112440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:24.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.428+0000 7f31b519f640 1 -- 172.21.15.112:0/1299759851 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31ac005040 con 0x7f31b0101280 2024-09-16T00:41:24.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.428+0000 7f31b519f640 1 --2- 172.21.15.112:0/1299759851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31b0101280 0x7f31b0111f00 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f31a400fce0 tx=0x7f31a4004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:24.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.429+0000 7f31a1ffb640 1 -- 172.21.15.112:0/1299759851 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f31a400de30 con 0x7f31b0101280 2024-09-16T00:41:24.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.429+0000 7f31a1ffb640 1 -- 172.21.15.112:0/1299759851 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f31a40090d0 con 0x7f31b0101280 2024-09-16T00:41:24.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.429+0000 7f31b61a1640 1 -- 172.21.15.112:0/1299759851 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31b010c100 con 0x7f31b0101280 2024-09-16T00:41:24.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.429+0000 7f31a1ffb640 1 -- 172.21.15.112:0/1299759851 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f31a40126a0 con 0x7f31b0101280 2024-09-16T00:41:24.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.429+0000 7f31b61a1640 1 -- 172.21.15.112:0/1299759851 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31b010c670 con 0x7f31b0101280 2024-09-16T00:41:24.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.430+0000 7f31b61a1640 1 -- 172.21.15.112:0/1299759851 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31b00fc9f0 con 0x7f31b0101280 2024-09-16T00:41:24.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.430+0000 7f31a1ffb640 1 -- 172.21.15.112:0/1299759851 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f31a4009240 con 0x7f31b0101280 2024-09-16T00:41:24.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.431+0000 7f31a1ffb640 1 --2- 172.21.15.112:0/1299759851 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f319c06e530 0x7f319c0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:24.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.431+0000 7f31a1ffb640 1 -- 172.21.15.112:0/1299759851 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f31a4090760 con 0x7f31b0101280 2024-09-16T00:41:24.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.431+0000 7f31b499e640 1 --2- 172.21.15.112:0/1299759851 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f319c06e530 0x7f319c0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:24.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.431+0000 7f31b499e640 1 --2- 172.21.15.112:0/1299759851 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f319c06e530 0x7f319c0709e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f31ac009ac0 tx=0x7f31ac035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:24.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.435+0000 7f31a1ffb640 1 -- 172.21.15.112:0/1299759851 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f31a4059800 con 0x7f31b0101280 2024-09-16T00:41:24.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.615+0000 7f31b61a1640 1 -- 172.21.15.112:0/1299759851 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f31b004b720 con 0x7f319c06e530 2024-09-16T00:41:24.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.617+0000 7f31a1ffb640 1 -- 172.21.15.112:0/1299759851 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f31b004b720 con 0x7f319c06e530 2024-09-16T00:41:24.617 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:41:24.617 INFO:teuthology.orchestra.run.smithi112.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80", 2024-09-16T00:41:24.617 INFO:teuthology.orchestra.run.smithi112.stdout: "in_progress": true, 2024-09-16T00:41:24.617 INFO:teuthology.orchestra.run.smithi112.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T00:41:24.617 INFO:teuthology.orchestra.run.smithi112.stdout: "services_complete": [ 2024-09-16T00:41:24.617 INFO:teuthology.orchestra.run.smithi112.stdout: "mds", 2024-09-16T00:41:24.617 INFO:teuthology.orchestra.run.smithi112.stdout: "osd", 2024-09-16T00:41:24.617 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr", 2024-09-16T00:41:24.617 INFO:teuthology.orchestra.run.smithi112.stdout: "mon", 2024-09-16T00:41:24.618 INFO:teuthology.orchestra.run.smithi112.stdout: "crash" 2024-09-16T00:41:24.618 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-16T00:41:24.618 INFO:teuthology.orchestra.run.smithi112.stdout: "progress": "16/21 daemons upgraded", 2024-09-16T00:41:24.618 INFO:teuthology.orchestra.run.smithi112.stdout: "message": "Currently upgrading mds daemons", 2024-09-16T00:41:24.618 INFO:teuthology.orchestra.run.smithi112.stdout: "is_paused": false 2024-09-16T00:41:24.618 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:41:24.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.619+0000 7f31b61a1640 1 -- 172.21.15.112:0/1299759851 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f319c06e530 msgr2=0x7f319c0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:24.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.619+0000 7f31b61a1640 1 --2- 172.21.15.112:0/1299759851 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f319c06e530 0x7f319c0709e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f31ac009ac0 tx=0x7f31ac035040 comp rx=0 tx=0).stop 2024-09-16T00:41:24.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.619+0000 7f31b61a1640 1 -- 172.21.15.112:0/1299759851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31b0101280 msgr2=0x7f31b0111f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:24.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.619+0000 7f31b61a1640 1 --2- 172.21.15.112:0/1299759851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31b0101280 0x7f31b0111f00 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f31a400fce0 tx=0x7f31a4004600 comp rx=0 tx=0).stop 2024-09-16T00:41:24.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.619+0000 7f31b61a1640 1 -- 172.21.15.112:0/1299759851 shutdown_connections 2024-09-16T00:41:24.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.619+0000 7f31b61a1640 1 --2- 172.21.15.112:0/1299759851 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f319c06e530 0x7f319c0709e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:24.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.619+0000 7f31b61a1640 1 --2- 172.21.15.112:0/1299759851 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31b0101c20 0x7f31b0112440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:24.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.619+0000 7f31b61a1640 1 --2- 172.21.15.112:0/1299759851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31b0101280 0x7f31b0111f00 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:24.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.619+0000 7f31b61a1640 1 -- 172.21.15.112:0/1299759851 >> 172.21.15.112:0/1299759851 conn(0x7f31b00f6e30 msgr2=0x7f31b00f7ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:24.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.619+0000 7f31b61a1640 1 -- 172.21.15.112:0/1299759851 shutdown_connections 2024-09-16T00:41:24.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.619+0000 7f31b61a1640 1 -- 172.21.15.112:0/1299759851 wait complete. 2024-09-16T00:41:24.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.698+0000 7f5f22027640 1 -- 172.21.15.112:0/3056354988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f1c1014e0 msgr2=0x7f5f1c1018b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:24.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.698+0000 7f5f22027640 1 --2- 172.21.15.112:0/3056354988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f1c1014e0 0x7f5f1c1018b0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c0048d0 tx=0x7f5f0c009da0 comp rx=0 tx=0).stop 2024-09-16T00:41:24.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.698+0000 7f5f22027640 1 -- 172.21.15.112:0/3056354988 shutdown_connections 2024-09-16T00:41:24.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.698+0000 7f5f22027640 1 --2- 172.21.15.112:0/3056354988 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f1c0fb4a0 0x7f5f1c0fb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:24.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.698+0000 7f5f22027640 1 --2- 172.21.15.112:0/3056354988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f1c1014e0 0x7f5f1c1018b0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:24.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.698+0000 7f5f22027640 1 -- 172.21.15.112:0/3056354988 >> 172.21.15.112:0/3056354988 conn(0x7f5f1c0f7060 msgr2=0x7f5f1c0f9470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:24.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.698+0000 7f5f22027640 1 -- 172.21.15.112:0/3056354988 shutdown_connections 2024-09-16T00:41:24.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.698+0000 7f5f22027640 1 -- 172.21.15.112:0/3056354988 wait complete. 2024-09-16T00:41:24.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.699+0000 7f5f22027640 1 Processor -- start 2024-09-16T00:41:24.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.699+0000 7f5f22027640 1 -- start start 2024-09-16T00:41:24.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.699+0000 7f5f22027640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f1c0fb4a0 0x7f5f1c112110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:24.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.699+0000 7f5f22027640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f1c1014e0 0x7f5f1c112650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:24.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.699+0000 7f5f22027640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c112ce0 con 0x7f5f1c0fb4a0 2024-09-16T00:41:24.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.699+0000 7f5f22027640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c10c090 con 0x7f5f1c1014e0 2024-09-16T00:41:24.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.699+0000 7f5f1b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f1c0fb4a0 0x7f5f1c112110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:24.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.699+0000 7f5f1b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f1c0fb4a0 0x7f5f1c112110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40486/0 (socket says 172.21.15.112:40486) 2024-09-16T00:41:24.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.699+0000 7f5f1b7fe640 1 -- 172.21.15.112:0/4198854933 learned_addr learned my addr 172.21.15.112:0/4198854933 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:41:24.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.699+0000 7f5f1affd640 1 --2- 172.21.15.112:0/4198854933 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f1c1014e0 0x7f5f1c112650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:24.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.700+0000 7f5f1b7fe640 1 -- 172.21.15.112:0/4198854933 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f1c1014e0 msgr2=0x7f5f1c112650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:24.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.700+0000 7f5f1b7fe640 1 --2- 172.21.15.112:0/4198854933 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f1c1014e0 0x7f5f1c112650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:24.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.700+0000 7f5f1b7fe640 1 -- 172.21.15.112:0/4198854933 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f0c005040 con 0x7f5f1c0fb4a0 2024-09-16T00:41:24.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.700+0000 7f5f1affd640 1 --2- 172.21.15.112:0/4198854933 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f1c1014e0 0x7f5f1c112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:41:24.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.700+0000 7f5f1b7fe640 1 --2- 172.21.15.112:0/4198854933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f1c0fb4a0 0x7f5f1c112110 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c003080 tx=0x7f5f0c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:24.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.700+0000 7f5f13fff640 1 -- 172.21.15.112:0/4198854933 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f0c038070 con 0x7f5f1c0fb4a0 2024-09-16T00:41:24.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.700+0000 7f5f13fff640 1 -- 172.21.15.112:0/4198854933 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f5f0c0097e0 con 0x7f5f1c0fb4a0 2024-09-16T00:41:24.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.700+0000 7f5f22027640 1 -- 172.21.15.112:0/4198854933 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f1c10c310 con 0x7f5f1c0fb4a0 2024-09-16T00:41:24.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.700+0000 7f5f13fff640 1 -- 172.21.15.112:0/4198854933 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f0c03d420 con 0x7f5f1c0fb4a0 2024-09-16T00:41:24.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.700+0000 7f5f22027640 1 -- 172.21.15.112:0/4198854933 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f1c10c7d0 con 0x7f5f1c0fb4a0 2024-09-16T00:41:24.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.701+0000 7f5f22027640 1 -- 172.21.15.112:0/4198854933 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f1c0fcbe0 con 0x7f5f1c0fb4a0 2024-09-16T00:41:24.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.702+0000 7f5f13fff640 1 -- 172.21.15.112:0/4198854933 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f5f0c030020 con 0x7f5f1c0fb4a0 2024-09-16T00:41:24.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.702+0000 7f5f13fff640 1 --2- 172.21.15.112:0/4198854933 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5f0806e530 0x7f5f080709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:24.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.702+0000 7f5f13fff640 1 -- 172.21.15.112:0/4198854933 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f5f0c0ae8e0 con 0x7f5f1c0fb4a0 2024-09-16T00:41:24.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.702+0000 7f5f1affd640 1 --2- 172.21.15.112:0/4198854933 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5f0806e530 0x7f5f080709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:24.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.703+0000 7f5f1affd640 1 --2- 172.21.15.112:0/4198854933 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5f0806e530 0x7f5f080709e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c10d9d0 tx=0x7f5f04008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:24.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.707+0000 7f5f13fff640 1 -- 172.21.15.112:0/4198854933 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f5f0c077850 con 0x7f5f1c0fb4a0 2024-09-16T00:41:24.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.941+0000 7f5f22027640 1 -- 172.21.15.112:0/4198854933 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f5f1c04b780 con 0x7f5f1c0fb4a0 2024-09-16T00:41:24.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.941+0000 7f5f13fff640 1 -- 172.21.15.112:0/4198854933 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f5f0c07af10 con 0x7f5f1c0fb4a0 2024-09-16T00:41:24.942 INFO:teuthology.orchestra.run.smithi112.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-16T00:41:24.942 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-16T00:41:24.942 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs is degraded 2024-09-16T00:41:24.942 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T00:41:24.942 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T00:41:24.942 INFO:teuthology.orchestra.run.smithi112.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-16T00:41:24.942 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-16T00:41:24.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.943+0000 7f5f22027640 1 -- 172.21.15.112:0/4198854933 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5f0806e530 msgr2=0x7f5f080709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:24.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.943+0000 7f5f22027640 1 --2- 172.21.15.112:0/4198854933 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5f0806e530 0x7f5f080709e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c10d9d0 tx=0x7f5f04008040 comp rx=0 tx=0).stop 2024-09-16T00:41:24.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.943+0000 7f5f22027640 1 -- 172.21.15.112:0/4198854933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f1c0fb4a0 msgr2=0x7f5f1c112110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:24.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.943+0000 7f5f22027640 1 --2- 172.21.15.112:0/4198854933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f1c0fb4a0 0x7f5f1c112110 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c003080 tx=0x7f5f0c009da0 comp rx=0 tx=0).stop 2024-09-16T00:41:24.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.943+0000 7f5f22027640 1 -- 172.21.15.112:0/4198854933 shutdown_connections 2024-09-16T00:41:24.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.943+0000 7f5f22027640 1 --2- 172.21.15.112:0/4198854933 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5f0806e530 0x7f5f080709e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:24.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.943+0000 7f5f22027640 1 --2- 172.21.15.112:0/4198854933 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f1c1014e0 0x7f5f1c112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:24.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.943+0000 7f5f22027640 1 --2- 172.21.15.112:0/4198854933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f1c0fb4a0 0x7f5f1c112110 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:24.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.943+0000 7f5f22027640 1 -- 172.21.15.112:0/4198854933 >> 172.21.15.112:0/4198854933 conn(0x7f5f1c0f7060 msgr2=0x7f5f1c0f8910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:24.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.944+0000 7f5f22027640 1 -- 172.21.15.112:0/4198854933 shutdown_connections 2024-09-16T00:41:24.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:24.944+0000 7f5f22027640 1 -- 172.21.15.112:0/4198854933 wait complete. 2024-09-16T00:41:25.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:41:24 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/1960207546' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:41:25.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:41:24 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/1403806186' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:41:25.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:41:24 smithi169 ceph-mon[77626]: from='client.34820 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:41:25.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:41:24 smithi169 ceph-mon[77626]: pgmap v410: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:41:25.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:41:24 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/1960207546' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:41:25.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:41:24 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/1403806186' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:41:25.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:41:24 smithi112 ceph-mon[97750]: from='client.34820 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:41:25.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:41:24 smithi112 ceph-mon[97750]: pgmap v410: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:41:26.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:41:25 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/4198854933' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:41:26.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:41:25 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/4198854933' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:41:27.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:41:26 smithi169 ceph-mon[77626]: pgmap v411: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:41:27.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:41:26 smithi112 ceph-mon[97750]: pgmap v411: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:41:30.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:41:29 smithi169 ceph-mon[77626]: pgmap v412: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:41:30.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:41:29 smithi112 ceph-mon[97750]: pgmap v412: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:41:32.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:41:31 smithi169 ceph-mon[77626]: pgmap v413: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:41:32.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:41:31 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:41:32.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:41:31 smithi112 ceph-mon[97750]: pgmap v413: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:41:32.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:41:31 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:41:34.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:41:33 smithi169 ceph-mon[77626]: pgmap v414: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:41:34.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:41:33 smithi112 ceph-mon[97750]: pgmap v414: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:41:36.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:41:35 smithi169 ceph-mon[77626]: pgmap v415: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:41:36.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:41:35 smithi112 ceph-mon[97750]: pgmap v415: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:41:38.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:41:37 smithi169 ceph-mon[77626]: pgmap v416: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:41:38.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:41:37 smithi112 ceph-mon[97750]: pgmap v416: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:41:40.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:41:39 smithi169 ceph-mon[77626]: pgmap v417: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:41:40.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:41:39 smithi112 ceph-mon[97750]: pgmap v417: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:41:42.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:41:41 smithi169 ceph-mon[77626]: pgmap v418: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:41:42.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:41:41 smithi112 ceph-mon[97750]: pgmap v418: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:41:44.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:41:43 smithi169 ceph-mon[77626]: pgmap v419: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:41:44.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:41:43 smithi112 ceph-mon[97750]: pgmap v419: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:41:46.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:41:45 smithi169 ceph-mon[77626]: pgmap v420: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:41:46.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:41:45 smithi112 ceph-mon[97750]: pgmap v420: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:41:47.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:41:46 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:41:47.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:41:46 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:41:48.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:41:47 smithi169 ceph-mon[77626]: pgmap v421: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:41:48.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:41:47 smithi112 ceph-mon[97750]: pgmap v421: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:41:50.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:41:49 smithi169 ceph-mon[77626]: pgmap v422: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:41:50.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:41:49 smithi112 ceph-mon[97750]: pgmap v422: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:41:52.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:41:51 smithi169 ceph-mon[77626]: pgmap v423: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:41:52.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:41:51 smithi112 ceph-mon[97750]: pgmap v423: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:41:54.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:41:53 smithi169 ceph-mon[77626]: pgmap v424: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:41:54.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:41:53 smithi112 ceph-mon[97750]: pgmap v424: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:41:55.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.033+0000 7f5b2508c640 1 -- 172.21.15.112:0/3272501895 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b200fb810 msgr2=0x7f5b200fbbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:55.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.033+0000 7f5b2508c640 1 --2- 172.21.15.112:0/3272501895 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b200fb810 0x7f5b200fbbe0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f5b080048d0 tx=0x7f5b08009da0 comp rx=0 tx=0).stop 2024-09-16T00:41:55.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.033+0000 7f5b2508c640 1 -- 172.21.15.112:0/3272501895 shutdown_connections 2024-09-16T00:41:55.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.033+0000 7f5b2508c640 1 --2- 172.21.15.112:0/3272501895 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b200fcf80 0x7f5b200fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:55.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.033+0000 7f5b2508c640 1 --2- 172.21.15.112:0/3272501895 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b200fb810 0x7f5b200fbbe0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:55.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.033+0000 7f5b2508c640 1 -- 172.21.15.112:0/3272501895 >> 172.21.15.112:0/3272501895 conn(0x7f5b200723b0 msgr2=0x7f5b200727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:55.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.033+0000 7f5b2508c640 1 -- 172.21.15.112:0/3272501895 shutdown_connections 2024-09-16T00:41:55.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.033+0000 7f5b2508c640 1 -- 172.21.15.112:0/3272501895 wait complete. 2024-09-16T00:41:55.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.034+0000 7f5b2508c640 1 Processor -- start 2024-09-16T00:41:55.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.034+0000 7f5b2508c640 1 -- start start 2024-09-16T00:41:55.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.034+0000 7f5b2508c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b200fb810 0x7f5b201131e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:55.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.034+0000 7f5b2508c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b200fcf80 0x7f5b20113720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:55.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.034+0000 7f5b2508c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b20113d20 con 0x7f5b200fcf80 2024-09-16T00:41:55.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.034+0000 7f5b2508c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2010d1b0 con 0x7f5b200fb810 2024-09-16T00:41:55.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.034+0000 7f5b1e59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b200fcf80 0x7f5b20113720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:55.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.034+0000 7f5b1e59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b200fcf80 0x7f5b20113720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58052/0 (socket says 172.21.15.112:58052) 2024-09-16T00:41:55.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.034+0000 7f5b1e59c640 1 -- 172.21.15.112:0/1573913338 learned_addr learned my addr 172.21.15.112:0/1573913338 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:41:55.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.034+0000 7f5b1ed9d640 1 --2- 172.21.15.112:0/1573913338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b200fb810 0x7f5b201131e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:55.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.034+0000 7f5b1e59c640 1 -- 172.21.15.112:0/1573913338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b200fb810 msgr2=0x7f5b201131e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:55.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.034+0000 7f5b1e59c640 1 --2- 172.21.15.112:0/1573913338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b200fb810 0x7f5b201131e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:55.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.034+0000 7f5b1e59c640 1 -- 172.21.15.112:0/1573913338 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b08005040 con 0x7f5b200fcf80 2024-09-16T00:41:55.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.035+0000 7f5b1e59c640 1 --2- 172.21.15.112:0/1573913338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b200fcf80 0x7f5b20113720 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f5b1000c790 tx=0x7f5b100043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:55.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.035+0000 7f5b177fe640 1 -- 172.21.15.112:0/1573913338 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b1000a7a0 con 0x7f5b200fcf80 2024-09-16T00:41:55.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.035+0000 7f5b177fe640 1 -- 172.21.15.112:0/1573913338 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f5b1000ada0 con 0x7f5b200fcf80 2024-09-16T00:41:55.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.035+0000 7f5b177fe640 1 -- 172.21.15.112:0/1573913338 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b100113a0 con 0x7f5b200fcf80 2024-09-16T00:41:55.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.035+0000 7f5b2508c640 1 -- 172.21.15.112:0/1573913338 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b2010d490 con 0x7f5b200fcf80 2024-09-16T00:41:55.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.035+0000 7f5b2508c640 1 -- 172.21.15.112:0/1573913338 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b2010d9e0 con 0x7f5b200fcf80 2024-09-16T00:41:55.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.036+0000 7f5b2508c640 1 -- 172.21.15.112:0/1573913338 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b200fcbc0 con 0x7f5b200fcf80 2024-09-16T00:41:55.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.037+0000 7f5b177fe640 1 -- 172.21.15.112:0/1573913338 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f5b1000a900 con 0x7f5b200fcf80 2024-09-16T00:41:55.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.037+0000 7f5b177fe640 1 --2- 172.21.15.112:0/1573913338 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5b0c06e530 0x7f5b0c0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:55.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.037+0000 7f5b1ed9d640 1 --2- 172.21.15.112:0/1573913338 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5b0c06e530 0x7f5b0c0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:55.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.037+0000 7f5b177fe640 1 -- 172.21.15.112:0/1573913338 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f5b1008e4a0 con 0x7f5b200fcf80 2024-09-16T00:41:55.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.038+0000 7f5b1ed9d640 1 --2- 172.21.15.112:0/1573913338 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5b0c06e530 0x7f5b0c0709e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5b08009ac0 tx=0x7f5b08035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:55.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.042+0000 7f5b177fe640 1 -- 172.21.15.112:0/1573913338 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f5b100573f0 con 0x7f5b200fcf80 2024-09-16T00:41:55.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.225+0000 7f5b2508c640 1 -- 172.21.15.112:0/1573913338 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5b2010e8e0 con 0x7f5b0c06e530 2024-09-16T00:41:55.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.226+0000 7f5b177fe640 1 -- 172.21.15.112:0/1573913338 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f5b08039040 con 0x7f5b0c06e530 2024-09-16T00:41:55.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.228+0000 7f5b2508c640 1 -- 172.21.15.112:0/1573913338 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5b0c06e530 msgr2=0x7f5b0c0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:55.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.228+0000 7f5b2508c640 1 --2- 172.21.15.112:0/1573913338 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5b0c06e530 0x7f5b0c0709e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5b08009ac0 tx=0x7f5b08035040 comp rx=0 tx=0).stop 2024-09-16T00:41:55.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.228+0000 7f5b2508c640 1 -- 172.21.15.112:0/1573913338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b200fcf80 msgr2=0x7f5b20113720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:55.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.228+0000 7f5b2508c640 1 --2- 172.21.15.112:0/1573913338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b200fcf80 0x7f5b20113720 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f5b1000c790 tx=0x7f5b100043c0 comp rx=0 tx=0).stop 2024-09-16T00:41:55.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.228+0000 7f5b2508c640 1 -- 172.21.15.112:0/1573913338 shutdown_connections 2024-09-16T00:41:55.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.228+0000 7f5b2508c640 1 --2- 172.21.15.112:0/1573913338 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5b0c06e530 0x7f5b0c0709e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:55.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.228+0000 7f5b2508c640 1 --2- 172.21.15.112:0/1573913338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b200fcf80 0x7f5b20113720 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:55.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.228+0000 7f5b2508c640 1 --2- 172.21.15.112:0/1573913338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b200fb810 0x7f5b201131e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:55.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.228+0000 7f5b2508c640 1 -- 172.21.15.112:0/1573913338 >> 172.21.15.112:0/1573913338 conn(0x7f5b200723b0 msgr2=0x7f5b200fa900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:55.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.228+0000 7f5b2508c640 1 -- 172.21.15.112:0/1573913338 shutdown_connections 2024-09-16T00:41:55.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.228+0000 7f5b2508c640 1 -- 172.21.15.112:0/1573913338 wait complete. 2024-09-16T00:41:55.238 INFO:teuthology.orchestra.run.smithi112.stdout:true 2024-09-16T00:41:55.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.308+0000 7ff8cb72d640 1 -- 172.21.15.112:0/2667954042 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8c4102680 msgr2=0x7ff8c4102a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:55.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.308+0000 7ff8cb72d640 1 --2- 172.21.15.112:0/2667954042 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8c4102680 0x7ff8c4102a50 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4009c50 tx=0x7ff8b4004d10 comp rx=0 tx=0).stop 2024-09-16T00:41:55.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.308+0000 7ff8cb72d640 1 -- 172.21.15.112:0/2667954042 shutdown_connections 2024-09-16T00:41:55.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.308+0000 7ff8cb72d640 1 --2- 172.21.15.112:0/2667954042 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8c40facf0 0x7ff8c40fb160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:55.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.308+0000 7ff8cb72d640 1 --2- 172.21.15.112:0/2667954042 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8c4102680 0x7ff8c4102a50 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:55.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.308+0000 7ff8cb72d640 1 -- 172.21.15.112:0/2667954042 >> 172.21.15.112:0/2667954042 conn(0x7ff8c4070d50 msgr2=0x7ff8c4071150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:55.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.308+0000 7ff8cb72d640 1 -- 172.21.15.112:0/2667954042 shutdown_connections 2024-09-16T00:41:55.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.308+0000 7ff8cb72d640 1 -- 172.21.15.112:0/2667954042 wait complete. 2024-09-16T00:41:55.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.308+0000 7ff8cb72d640 1 Processor -- start 2024-09-16T00:41:55.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.308+0000 7ff8cb72d640 1 -- start start 2024-09-16T00:41:55.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.309+0000 7ff8cb72d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8c40facf0 0x7ff8c41154c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:55.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.309+0000 7ff8cb72d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8c4102680 0x7ff8c4115a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:55.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.309+0000 7ff8cb72d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c4116050 con 0x7ff8c40facf0 2024-09-16T00:41:55.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.309+0000 7ff8cb72d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c410f460 con 0x7ff8c4102680 2024-09-16T00:41:55.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.309+0000 7ff8c94c9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8c40facf0 0x7ff8c41154c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:55.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.309+0000 7ff8c94c9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8c40facf0 0x7ff8c41154c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58072/0 (socket says 172.21.15.112:58072) 2024-09-16T00:41:55.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.309+0000 7ff8c94c9640 1 -- 172.21.15.112:0/4141372193 learned_addr learned my addr 172.21.15.112:0/4141372193 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:41:55.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.309+0000 7ff8c8cc8640 1 --2- 172.21.15.112:0/4141372193 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8c4102680 0x7ff8c4115a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:55.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.309+0000 7ff8c94c9640 1 -- 172.21.15.112:0/4141372193 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8c4102680 msgr2=0x7ff8c4115a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:55.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.309+0000 7ff8c94c9640 1 --2- 172.21.15.112:0/4141372193 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8c4102680 0x7ff8c4115a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:55.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.309+0000 7ff8c94c9640 1 -- 172.21.15.112:0/4141372193 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8b40098b0 con 0x7ff8c40facf0 2024-09-16T00:41:55.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.309+0000 7ff8c8cc8640 1 --2- 172.21.15.112:0/4141372193 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8c4102680 0x7ff8c4115a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:41:55.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.309+0000 7ff8c94c9640 1 --2- 172.21.15.112:0/4141372193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8c40facf0 0x7ff8c41154c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4004750 tx=0x7ff8b4004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:55.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.309+0000 7ff8c1ffb640 1 -- 172.21.15.112:0/4141372193 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8b4009590 con 0x7ff8c40facf0 2024-09-16T00:41:55.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.309+0000 7ff8c1ffb640 1 -- 172.21.15.112:0/4141372193 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7ff8b40096f0 con 0x7ff8c40facf0 2024-09-16T00:41:55.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.310+0000 7ff8cb72d640 1 -- 172.21.15.112:0/4141372193 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8c410f6e0 con 0x7ff8c40facf0 2024-09-16T00:41:55.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.310+0000 7ff8c1ffb640 1 -- 172.21.15.112:0/4141372193 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8b403d960 con 0x7ff8c40facf0 2024-09-16T00:41:55.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.310+0000 7ff8cb72d640 1 -- 172.21.15.112:0/4141372193 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8c410fc00 con 0x7ff8c40facf0 2024-09-16T00:41:55.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.311+0000 7ff8cb72d640 1 -- 172.21.15.112:0/4141372193 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8c40657f0 con 0x7ff8c40facf0 2024-09-16T00:41:55.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.312+0000 7ff8c1ffb640 1 -- 172.21.15.112:0/4141372193 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7ff8b402f030 con 0x7ff8c40facf0 2024-09-16T00:41:55.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.312+0000 7ff8c1ffb640 1 --2- 172.21.15.112:0/4141372193 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff8ac06e530 0x7ff8ac0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:55.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.312+0000 7ff8c1ffb640 1 -- 172.21.15.112:0/4141372193 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7ff8b40af7a0 con 0x7ff8c40facf0 2024-09-16T00:41:55.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.312+0000 7ff8c8cc8640 1 --2- 172.21.15.112:0/4141372193 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff8ac06e530 0x7ff8ac0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:55.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.313+0000 7ff8c8cc8640 1 --2- 172.21.15.112:0/4141372193 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff8ac06e530 0x7ff8ac0709e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff8c4110740 tx=0x7ff8b800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:55.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.317+0000 7ff8c1ffb640 1 -- 172.21.15.112:0/4141372193 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7ff8b40787d0 con 0x7ff8c40facf0 2024-09-16T00:41:55.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.501+0000 7ff8cb72d640 1 -- 172.21.15.112:0/4141372193 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff8c4109750 con 0x7ff8ac06e530 2024-09-16T00:41:55.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.503+0000 7ff8c1ffb640 1 -- 172.21.15.112:0/4141372193 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7ff8c4109750 con 0x7ff8ac06e530 2024-09-16T00:41:55.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.505+0000 7ff8cb72d640 1 -- 172.21.15.112:0/4141372193 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff8ac06e530 msgr2=0x7ff8ac0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:55.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.505+0000 7ff8cb72d640 1 --2- 172.21.15.112:0/4141372193 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff8ac06e530 0x7ff8ac0709e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff8c4110740 tx=0x7ff8b800b040 comp rx=0 tx=0).stop 2024-09-16T00:41:55.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.505+0000 7ff8cb72d640 1 -- 172.21.15.112:0/4141372193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8c40facf0 msgr2=0x7ff8c41154c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:55.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.505+0000 7ff8cb72d640 1 --2- 172.21.15.112:0/4141372193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8c40facf0 0x7ff8c41154c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4004750 tx=0x7ff8b4004d10 comp rx=0 tx=0).stop 2024-09-16T00:41:55.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.505+0000 7ff8cb72d640 1 -- 172.21.15.112:0/4141372193 shutdown_connections 2024-09-16T00:41:55.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.505+0000 7ff8cb72d640 1 --2- 172.21.15.112:0/4141372193 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff8ac06e530 0x7ff8ac0709e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:55.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.505+0000 7ff8cb72d640 1 --2- 172.21.15.112:0/4141372193 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8c4102680 0x7ff8c4115a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:55.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.505+0000 7ff8cb72d640 1 --2- 172.21.15.112:0/4141372193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8c40facf0 0x7ff8c41154c0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:55.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.505+0000 7ff8cb72d640 1 -- 172.21.15.112:0/4141372193 >> 172.21.15.112:0/4141372193 conn(0x7ff8c4070d50 msgr2=0x7ff8c40fc790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:55.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.505+0000 7ff8cb72d640 1 -- 172.21.15.112:0/4141372193 shutdown_connections 2024-09-16T00:41:55.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.505+0000 7ff8cb72d640 1 -- 172.21.15.112:0/4141372193 wait complete. 2024-09-16T00:41:55.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.583+0000 7f12661ab640 1 -- 172.21.15.112:0/1251020971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1260102550 msgr2=0x7f1260102920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:55.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.583+0000 7f12661ab640 1 --2- 172.21.15.112:0/1251020971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1260102550 0x7f1260102920 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f12500048d0 tx=0x7f1250009da0 comp rx=0 tx=0).stop 2024-09-16T00:41:55.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.583+0000 7f12661ab640 1 -- 172.21.15.112:0/1251020971 shutdown_connections 2024-09-16T00:41:55.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.583+0000 7f12661ab640 1 --2- 172.21.15.112:0/1251020971 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1260102ef0 0x7f12600fc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:55.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.583+0000 7f12661ab640 1 --2- 172.21.15.112:0/1251020971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1260102550 0x7f1260102920 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:55.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.583+0000 7f12661ab640 1 -- 172.21.15.112:0/1251020971 >> 172.21.15.112:0/1251020971 conn(0x7f12600f80f0 msgr2=0x7f12600fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:55.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.583+0000 7f12661ab640 1 -- 172.21.15.112:0/1251020971 shutdown_connections 2024-09-16T00:41:55.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.583+0000 7f12661ab640 1 -- 172.21.15.112:0/1251020971 wait complete. 2024-09-16T00:41:55.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.584+0000 7f12661ab640 1 Processor -- start 2024-09-16T00:41:55.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.584+0000 7f12661ab640 1 -- start start 2024-09-16T00:41:55.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.584+0000 7f12661ab640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1260102550 0x7f1260113570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:55.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.584+0000 7f12661ab640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1260102ef0 0x7f1260113ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:55.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.584+0000 7f12661ab640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1260114100 con 0x7f1260102550 2024-09-16T00:41:55.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.584+0000 7f12661ab640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f126010d510 con 0x7f1260102ef0 2024-09-16T00:41:55.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.584+0000 7f125f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1260102550 0x7f1260113570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:55.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.584+0000 7f125f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1260102550 0x7f1260113570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58086/0 (socket says 172.21.15.112:58086) 2024-09-16T00:41:55.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.584+0000 7f125f7fe640 1 -- 172.21.15.112:0/3733154177 learned_addr learned my addr 172.21.15.112:0/3733154177 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:41:55.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.584+0000 7f1257fff640 1 --2- 172.21.15.112:0/3733154177 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1260102ef0 0x7f1260113ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:55.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.584+0000 7f125f7fe640 1 -- 172.21.15.112:0/3733154177 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1260102ef0 msgr2=0x7f1260113ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:55.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.584+0000 7f125f7fe640 1 --2- 172.21.15.112:0/3733154177 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1260102ef0 0x7f1260113ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:55.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.584+0000 7f125f7fe640 1 -- 172.21.15.112:0/3733154177 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1250005040 con 0x7f1260102550 2024-09-16T00:41:55.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.585+0000 7f125f7fe640 1 --2- 172.21.15.112:0/3733154177 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1260102550 0x7f1260113570 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1250000c00 tx=0x7f1250009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:55.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.585+0000 7f125cff9640 1 -- 172.21.15.112:0/3733154177 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1250038070 con 0x7f1260102550 2024-09-16T00:41:55.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.585+0000 7f125cff9640 1 -- 172.21.15.112:0/3733154177 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f125002ade0 con 0x7f1260102550 2024-09-16T00:41:55.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.585+0000 7f12661ab640 1 -- 172.21.15.112:0/3733154177 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f126010d790 con 0x7f1260102550 2024-09-16T00:41:55.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.585+0000 7f125cff9640 1 -- 172.21.15.112:0/3733154177 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f125002c7e0 con 0x7f1260102550 2024-09-16T00:41:55.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.585+0000 7f12661ab640 1 -- 172.21.15.112:0/3733154177 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f126010dc50 con 0x7f1260102550 2024-09-16T00:41:55.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.586+0000 7f12661ab640 1 -- 172.21.15.112:0/3733154177 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12600fdcd0 con 0x7f1260102550 2024-09-16T00:41:55.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.586+0000 7f125cff9640 1 -- 172.21.15.112:0/3733154177 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f1250030020 con 0x7f1260102550 2024-09-16T00:41:55.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.587+0000 7f125cff9640 1 --2- 172.21.15.112:0/3733154177 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f124c06e530 0x7f124c0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:55.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.587+0000 7f125cff9640 1 -- 172.21.15.112:0/3733154177 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f12500adf70 con 0x7f1260102550 2024-09-16T00:41:55.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.587+0000 7f1257fff640 1 --2- 172.21.15.112:0/3733154177 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f124c06e530 0x7f124c0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:55.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.588+0000 7f1257fff640 1 --2- 172.21.15.112:0/3733154177 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f124c06e530 0x7f124c0709e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f126010e7f0 tx=0x7f1248009ed0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:55.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.592+0000 7f125cff9640 1 -- 172.21.15.112:0/3733154177 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f12500e31b0 con 0x7f1260102550 2024-09-16T00:41:55.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.764+0000 7f12661ab640 1 -- 172.21.15.112:0/3733154177 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f126004b720 con 0x7f124c06e530 2024-09-16T00:41:55.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.773+0000 7f125cff9640 1 -- 172.21.15.112:0/3733154177 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7f126004b720 con 0x7f124c06e530 2024-09-16T00:41:55.774 INFO:teuthology.orchestra.run.smithi112.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T00:41:55.774 INFO:teuthology.orchestra.run.smithi112.stdout:alertmanager.smithi112 smithi112 *:9093,9094 running (14m) 7m ago 23m 24.7M - 0.25.0 c8568f914cd2 d85ea0562022 2024-09-16T00:41:55.774 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi112 smithi112 running (11m) 7m ago 24m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c e4fe58f9702a 2024-09-16T00:41:55.774 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi169 smithi169 running (11m) 7m ago 23m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c 3ba2c81d527f 2024-09-16T00:41:55.774 INFO:teuthology.orchestra.run.smithi112.stdout:grafana.smithi112 smithi112 *:3000 running (13m) 7m ago 23m 82.1M - 9.4.12 2bacad6d85d8 51f9d2013da2 2024-09-16T00:41:55.774 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.hxfwkh smithi112 running (8m) 7m ago 18m 14.2M - 19.1.1-326-g8921c8ef f1455bd0219c 1723069902d6 2024-09-16T00:41:55.775 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.ithwzj smithi112 running (8m) 7m ago 18m 12.2M - 19.1.1-326-g8921c8ef f1455bd0219c e7aad5b81db5 2024-09-16T00:41:55.775 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.gawwgw smithi169 running (7m) 7m ago 18m 15.7M - 19.1.1-326-g8921c8ef f1455bd0219c 469be459ee12 2024-09-16T00:41:55.775 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.lgeqrj smithi169 running (7m) 7m ago 18m 13.3M - 19.1.1-326-g8921c8ef f1455bd0219c 31f0fbd2395d 2024-09-16T00:41:55.775 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi112.xrkkbc smithi112 *:8443,9283,8765 running (14m) 7m ago 25m 504M - 19.1.1-326-g8921c8ef f1455bd0219c a981f7fd2810 2024-09-16T00:41:55.775 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi169.mtrobk smithi169 *:8443,9283,8765 running (13m) 7m ago 22m 442M - 19.1.1-326-g8921c8ef f1455bd0219c 4096e5369c2d 2024-09-16T00:41:55.775 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi112 smithi112 running (12m) 7m ago 25m 49.8M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 5aaec033c042 2024-09-16T00:41:55.775 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi169 smithi169 running (12m) 7m ago 22m 38.9M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 7bc609155f3f 2024-09-16T00:41:55.775 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi112 smithi112 *:9100 running (15m) 7m ago 23m 15.5M - 1.5.0 0da6a335fe13 6404d1d601e2 2024-09-16T00:41:55.775 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi169 smithi169 *:9100 running (14m) 7m ago 23m 15.4M - 1.5.0 0da6a335fe13 5505baf702a6 2024-09-16T00:41:55.775 INFO:teuthology.orchestra.run.smithi112.stdout:osd.0 smithi112 running (11m) 7m ago 22m 105M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 53efc9397017 2024-09-16T00:41:55.775 INFO:teuthology.orchestra.run.smithi112.stdout:osd.1 smithi112 running (10m) 7m ago 21m 98.9M 1557M 19.1.1-326-g8921c8ef f1455bd0219c fe6f1a4a2cf4 2024-09-16T00:41:55.775 INFO:teuthology.orchestra.run.smithi112.stdout:osd.2 smithi112 running (10m) 7m ago 21m 102M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 5f7863712057 2024-09-16T00:41:55.775 INFO:teuthology.orchestra.run.smithi112.stdout:osd.3 smithi169 running (9m) 7m ago 20m 92.1M 2581M 19.1.1-326-g8921c8ef f1455bd0219c ee07b5577810 2024-09-16T00:41:55.775 INFO:teuthology.orchestra.run.smithi112.stdout:osd.4 smithi169 running (9m) 7m ago 20m 103M 2581M 19.1.1-326-g8921c8ef f1455bd0219c 769f026a54eb 2024-09-16T00:41:55.776 INFO:teuthology.orchestra.run.smithi112.stdout:osd.5 smithi169 running (8m) 7m ago 19m 125M 2581M 19.1.1-326-g8921c8ef f1455bd0219c a6007c13cf78 2024-09-16T00:41:55.776 INFO:teuthology.orchestra.run.smithi112.stdout:prometheus.smithi112 smithi112 *:9095 running (14m) 7m ago 23m 64.1M - 2.43.0 a07b618ecd1d 7a1d42e60d2b 2024-09-16T00:41:55.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.775+0000 7f12661ab640 1 -- 172.21.15.112:0/3733154177 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f124c06e530 msgr2=0x7f124c0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:55.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.775+0000 7f12661ab640 1 --2- 172.21.15.112:0/3733154177 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f124c06e530 0x7f124c0709e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f126010e7f0 tx=0x7f1248009ed0 comp rx=0 tx=0).stop 2024-09-16T00:41:55.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.775+0000 7f12661ab640 1 -- 172.21.15.112:0/3733154177 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1260102550 msgr2=0x7f1260113570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:55.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.775+0000 7f12661ab640 1 --2- 172.21.15.112:0/3733154177 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1260102550 0x7f1260113570 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1250000c00 tx=0x7f1250009da0 comp rx=0 tx=0).stop 2024-09-16T00:41:55.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.775+0000 7f12661ab640 1 -- 172.21.15.112:0/3733154177 shutdown_connections 2024-09-16T00:41:55.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.775+0000 7f12661ab640 1 --2- 172.21.15.112:0/3733154177 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f124c06e530 0x7f124c0709e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:55.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.775+0000 7f12661ab640 1 --2- 172.21.15.112:0/3733154177 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1260102ef0 0x7f1260113ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:55.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.775+0000 7f12661ab640 1 --2- 172.21.15.112:0/3733154177 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1260102550 0x7f1260113570 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:55.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.775+0000 7f12661ab640 1 -- 172.21.15.112:0/3733154177 >> 172.21.15.112:0/3733154177 conn(0x7f12600f80f0 msgr2=0x7f12600f90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:55.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.775+0000 7f12661ab640 1 -- 172.21.15.112:0/3733154177 shutdown_connections 2024-09-16T00:41:55.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.775+0000 7f12661ab640 1 -- 172.21.15.112:0/3733154177 wait complete. 2024-09-16T00:41:55.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:41:55 smithi112 ceph-mon[97750]: pgmap v425: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:41:55.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.853+0000 7f8366fa5640 1 -- 172.21.15.112:0/1093984344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83600fcf80 msgr2=0x7f83600fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:55.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.853+0000 7f8366fa5640 1 --2- 172.21.15.112:0/1093984344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83600fcf80 0x7f83600fd3d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8350009c50 tx=0x7f8350004d10 comp rx=0 tx=0).stop 2024-09-16T00:41:55.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.853+0000 7f8366fa5640 1 -- 172.21.15.112:0/1093984344 shutdown_connections 2024-09-16T00:41:55.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.853+0000 7f8366fa5640 1 --2- 172.21.15.112:0/1093984344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83600fcf80 0x7f83600fd3d0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:55.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.853+0000 7f8366fa5640 1 --2- 172.21.15.112:0/1093984344 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83600fb810 0x7f83600fbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:55.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.853+0000 7f8366fa5640 1 -- 172.21.15.112:0/1093984344 >> 172.21.15.112:0/1093984344 conn(0x7f83600723b0 msgr2=0x7f83600727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:55.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.853+0000 7f8366fa5640 1 -- 172.21.15.112:0/1093984344 shutdown_connections 2024-09-16T00:41:55.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.853+0000 7f8366fa5640 1 -- 172.21.15.112:0/1093984344 wait complete. 2024-09-16T00:41:55.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.853+0000 7f8366fa5640 1 Processor -- start 2024-09-16T00:41:55.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.853+0000 7f8366fa5640 1 -- start start 2024-09-16T00:41:55.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.854+0000 7f8366fa5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83600fb810 0x7f836010bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:55.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.854+0000 7f8366fa5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83600fcf80 0x7f836010c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:55.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.854+0000 7f8366fa5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f836010ca00 con 0x7f83600fcf80 2024-09-16T00:41:55.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.854+0000 7f8366fa5640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f836010cb40 con 0x7f83600fb810 2024-09-16T00:41:55.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.854+0000 7f835ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83600fcf80 0x7f836010c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:55.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.854+0000 7f835ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83600fcf80 0x7f836010c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58102/0 (socket says 172.21.15.112:58102) 2024-09-16T00:41:55.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.854+0000 7f835ffff640 1 -- 172.21.15.112:0/1984262374 learned_addr learned my addr 172.21.15.112:0/1984262374 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:41:55.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.854+0000 7f8364d41640 1 --2- 172.21.15.112:0/1984262374 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83600fb810 0x7f836010bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:55.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.854+0000 7f835ffff640 1 -- 172.21.15.112:0/1984262374 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83600fb810 msgr2=0x7f836010bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:55.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.854+0000 7f835ffff640 1 --2- 172.21.15.112:0/1984262374 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83600fb810 0x7f836010bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:55.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.854+0000 7f835ffff640 1 -- 172.21.15.112:0/1984262374 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83500098b0 con 0x7f83600fcf80 2024-09-16T00:41:55.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.854+0000 7f835ffff640 1 --2- 172.21.15.112:0/1984262374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83600fcf80 0x7f836010c4c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f8350004a30 tx=0x7f8350004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:55.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.854+0000 7f835d7fa640 1 -- 172.21.15.112:0/1984262374 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8350009690 con 0x7f83600fcf80 2024-09-16T00:41:55.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.854+0000 7f835d7fa640 1 -- 172.21.15.112:0/1984262374 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f8350032b10 con 0x7f83600fcf80 2024-09-16T00:41:55.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.854+0000 7f8366fa5640 1 -- 172.21.15.112:0/1984262374 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f836010cdc0 con 0x7f83600fcf80 2024-09-16T00:41:55.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.854+0000 7f835d7fa640 1 -- 172.21.15.112:0/1984262374 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f835003daa0 con 0x7f83600fcf80 2024-09-16T00:41:55.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.855+0000 7f8366fa5640 1 -- 172.21.15.112:0/1984262374 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8360119920 con 0x7f83600fcf80 2024-09-16T00:41:55.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.855+0000 7f8366fa5640 1 -- 172.21.15.112:0/1984262374 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83600fcbc0 con 0x7f83600fcf80 2024-09-16T00:41:55.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.856+0000 7f835d7fa640 1 -- 172.21.15.112:0/1984262374 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f835002f030 con 0x7f83600fcf80 2024-09-16T00:41:55.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.856+0000 7f835d7fa640 1 --2- 172.21.15.112:0/1984262374 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f834c06e530 0x7f834c0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:55.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.857+0000 7f835d7fa640 1 -- 172.21.15.112:0/1984262374 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f83500af2c0 con 0x7f83600fcf80 2024-09-16T00:41:55.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.857+0000 7f8364d41640 1 --2- 172.21.15.112:0/1984262374 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f834c06e530 0x7f834c0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:55.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.857+0000 7f8364d41640 1 --2- 172.21.15.112:0/1984262374 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f834c06e530 0x7f834c0709e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8358007fd0 tx=0x7f8358009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:55.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:55.861+0000 7f835d7fa640 1 -- 172.21.15.112:0/1984262374 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f8350078360 con 0x7f83600fcf80 2024-09-16T00:41:56.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:41:55 smithi169 ceph-mon[77626]: pgmap v425: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:41:56.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.093+0000 7f8366fa5640 1 -- 172.21.15.112:0/1984262374 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f8360072ec0 con 0x7f83600fcf80 2024-09-16T00:41:56.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.093+0000 7f835d7fa640 1 -- 172.21.15.112:0/1984262374 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f835007ba20 con 0x7f83600fcf80 2024-09-16T00:41:56.094 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:41:56.094 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": { 2024-09-16T00:41:56.095 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:41:56.095 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:41:56.095 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": { 2024-09-16T00:41:56.095 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:41:56.095 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:41:56.095 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": { 2024-09-16T00:41:56.095 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 6 2024-09-16T00:41:56.095 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:41:56.095 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": { 2024-09-16T00:41:56.095 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 4 2024-09-16T00:41:56.095 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:41:56.095 INFO:teuthology.orchestra.run.smithi112.stdout: "overall": { 2024-09-16T00:41:56.095 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 14 2024-09-16T00:41:56.095 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-16T00:41:56.096 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:41:56.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.095+0000 7f8366fa5640 1 -- 172.21.15.112:0/1984262374 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f834c06e530 msgr2=0x7f834c0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:56.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.095+0000 7f8366fa5640 1 --2- 172.21.15.112:0/1984262374 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f834c06e530 0x7f834c0709e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8358007fd0 tx=0x7f8358009040 comp rx=0 tx=0).stop 2024-09-16T00:41:56.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.095+0000 7f8366fa5640 1 -- 172.21.15.112:0/1984262374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83600fcf80 msgr2=0x7f836010c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:56.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.095+0000 7f8366fa5640 1 --2- 172.21.15.112:0/1984262374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83600fcf80 0x7f836010c4c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f8350004a30 tx=0x7f8350004d10 comp rx=0 tx=0).stop 2024-09-16T00:41:56.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.095+0000 7f8366fa5640 1 -- 172.21.15.112:0/1984262374 shutdown_connections 2024-09-16T00:41:56.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.095+0000 7f8366fa5640 1 --2- 172.21.15.112:0/1984262374 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f834c06e530 0x7f834c0709e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:56.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.095+0000 7f8366fa5640 1 --2- 172.21.15.112:0/1984262374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83600fcf80 0x7f836010c4c0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:56.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.095+0000 7f8366fa5640 1 --2- 172.21.15.112:0/1984262374 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83600fb810 0x7f836010bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:56.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.095+0000 7f8366fa5640 1 -- 172.21.15.112:0/1984262374 >> 172.21.15.112:0/1984262374 conn(0x7f83600723b0 msgr2=0x7f83600fa480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:56.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.095+0000 7f8366fa5640 1 -- 172.21.15.112:0/1984262374 shutdown_connections 2024-09-16T00:41:56.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.095+0000 7f8366fa5640 1 -- 172.21.15.112:0/1984262374 wait complete. 2024-09-16T00:41:56.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.173+0000 7f377bc42640 1 -- 172.21.15.112:0/1548861285 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3774102ef0 msgr2=0x7f37740fc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:56.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.173+0000 7f377bc42640 1 --2- 172.21.15.112:0/1548861285 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3774102ef0 0x7f37740fc5b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f37640048d0 tx=0x7f3764009da0 comp rx=0 tx=0).stop 2024-09-16T00:41:56.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.173+0000 7f377bc42640 1 -- 172.21.15.112:0/1548861285 shutdown_connections 2024-09-16T00:41:56.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.173+0000 7f377bc42640 1 --2- 172.21.15.112:0/1548861285 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3774102ef0 0x7f37740fc5b0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:56.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.173+0000 7f377bc42640 1 --2- 172.21.15.112:0/1548861285 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3774102550 0x7f3774102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:56.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.173+0000 7f377bc42640 1 -- 172.21.15.112:0/1548861285 >> 172.21.15.112:0/1548861285 conn(0x7f37740f80f0 msgr2=0x7f37740fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:56.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.173+0000 7f377bc42640 1 -- 172.21.15.112:0/1548861285 shutdown_connections 2024-09-16T00:41:56.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.173+0000 7f377bc42640 1 -- 172.21.15.112:0/1548861285 wait complete. 2024-09-16T00:41:56.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.174+0000 7f377bc42640 1 Processor -- start 2024-09-16T00:41:56.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.174+0000 7f377bc42640 1 -- start start 2024-09-16T00:41:56.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.174+0000 7f377bc42640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3774102550 0x7f3774113520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:56.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.174+0000 7f377bc42640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3774102ef0 0x7f3774113a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:56.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.174+0000 7f377bc42640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37741140b0 con 0x7f3774102ef0 2024-09-16T00:41:56.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.174+0000 7f377bc42640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f377410d4c0 con 0x7f3774102550 2024-09-16T00:41:56.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.174+0000 7f37791dd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3774102ef0 0x7f3774113a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:56.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.174+0000 7f37791dd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3774102ef0 0x7f3774113a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58124/0 (socket says 172.21.15.112:58124) 2024-09-16T00:41:56.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.174+0000 7f37791dd640 1 -- 172.21.15.112:0/3670830541 learned_addr learned my addr 172.21.15.112:0/3670830541 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:41:56.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.174+0000 7f37799de640 1 --2- 172.21.15.112:0/3670830541 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3774102550 0x7f3774113520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:56.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.175+0000 7f37791dd640 1 -- 172.21.15.112:0/3670830541 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3774102550 msgr2=0x7f3774113520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:56.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.175+0000 7f37791dd640 1 --2- 172.21.15.112:0/3670830541 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3774102550 0x7f3774113520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:56.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.175+0000 7f37791dd640 1 -- 172.21.15.112:0/3670830541 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3764005040 con 0x7f3774102ef0 2024-09-16T00:41:56.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.175+0000 7f37791dd640 1 --2- 172.21.15.112:0/3670830541 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3774102ef0 0x7f3774113a60 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f37640043d0 tx=0x7f3764009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:56.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.175+0000 7f376a7fc640 1 -- 172.21.15.112:0/3670830541 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3764038070 con 0x7f3774102ef0 2024-09-16T00:41:56.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.175+0000 7f376a7fc640 1 -- 172.21.15.112:0/3670830541 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f37640096e0 con 0x7f3774102ef0 2024-09-16T00:41:56.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.175+0000 7f377bc42640 1 -- 172.21.15.112:0/3670830541 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f377410d740 con 0x7f3774102ef0 2024-09-16T00:41:56.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.175+0000 7f376a7fc640 1 -- 172.21.15.112:0/3670830541 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f376402c8a0 con 0x7f3774102ef0 2024-09-16T00:41:56.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.175+0000 7f377bc42640 1 -- 172.21.15.112:0/3670830541 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f377410dc80 con 0x7f3774102ef0 2024-09-16T00:41:56.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.176+0000 7f377bc42640 1 -- 172.21.15.112:0/3670830541 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37740fdcd0 con 0x7f3774102ef0 2024-09-16T00:41:56.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.177+0000 7f376a7fc640 1 -- 172.21.15.112:0/3670830541 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f3764030020 con 0x7f3774102ef0 2024-09-16T00:41:56.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.177+0000 7f376a7fc640 1 --2- 172.21.15.112:0/3670830541 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f376006e530 0x7f37600709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:56.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.178+0000 7f376a7fc640 1 -- 172.21.15.112:0/3670830541 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f37640ae450 con 0x7f3774102ef0 2024-09-16T00:41:56.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.178+0000 7f37799de640 1 --2- 172.21.15.112:0/3670830541 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f376006e530 0x7f37600709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:56.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.178+0000 7f37799de640 1 --2- 172.21.15.112:0/3670830541 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f376006e530 0x7f37600709e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3770004ed0 tx=0x7f377000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:56.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.181+0000 7f376a7fc640 1 -- 172.21.15.112:0/3670830541 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f3764077510 con 0x7f3774102ef0 2024-09-16T00:41:56.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.381+0000 7f377bc42640 1 -- 172.21.15.112:0/3670830541 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f3774109580 con 0x7f3774102ef0 2024-09-16T00:41:56.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.381+0000 7f376a7fc640 1 -- 172.21.15.112:0/3670830541 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1914 (secure 0 0 0) 0x7f376407abd0 con 0x7f3774102ef0 2024-09-16T00:41:56.436 INFO:teuthology.orchestra.run.smithi112.stdout:e34 2024-09-16T00:41:56.436 INFO:teuthology.orchestra.run.smithi112.stdout:btime 2024-09-16T00:34:26:619254+0000 2024-09-16T00:41:56.436 INFO:teuthology.orchestra.run.smithi112.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T00:41:56.436 INFO:teuthology.orchestra.run.smithi112.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:41:56.436 INFO:teuthology.orchestra.run.smithi112.stdout:legacy client fscid: 1 2024-09-16T00:41:56.436 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:41:56.436 INFO:teuthology.orchestra.run.smithi112.stdout:Filesystem 'cephfs' (1) 2024-09-16T00:41:56.436 INFO:teuthology.orchestra.run.smithi112.stdout:fs_name cephfs 2024-09-16T00:41:56.437 INFO:teuthology.orchestra.run.smithi112.stdout:epoch 25 2024-09-16T00:41:56.437 INFO:teuthology.orchestra.run.smithi112.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-16T00:41:56.437 INFO:teuthology.orchestra.run.smithi112.stdout:created 2024-09-16T00:23:22.328199+0000 2024-09-16T00:41:56.437 INFO:teuthology.orchestra.run.smithi112.stdout:modified 2024-09-16T00:33:29.012991+0000 2024-09-16T00:41:56.437 INFO:teuthology.orchestra.run.smithi112.stdout:tableserver 0 2024-09-16T00:41:56.437 INFO:teuthology.orchestra.run.smithi112.stdout:root 0 2024-09-16T00:41:56.437 INFO:teuthology.orchestra.run.smithi112.stdout:session_timeout 60 2024-09-16T00:41:56.437 INFO:teuthology.orchestra.run.smithi112.stdout:session_autoclose 300 2024-09-16T00:41:56.437 INFO:teuthology.orchestra.run.smithi112.stdout:max_file_size 1099511627776 2024-09-16T00:41:56.437 INFO:teuthology.orchestra.run.smithi112.stdout:max_xattr_size 65536 2024-09-16T00:41:56.437 INFO:teuthology.orchestra.run.smithi112.stdout:required_client_features {} 2024-09-16T00:41:56.437 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure 0 2024-09-16T00:41:56.437 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure_osd_epoch 82 2024-09-16T00:41:56.437 INFO:teuthology.orchestra.run.smithi112.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:41:56.438 INFO:teuthology.orchestra.run.smithi112.stdout:max_mds 2 2024-09-16T00:41:56.438 INFO:teuthology.orchestra.run.smithi112.stdout:in 0,1 2024-09-16T00:41:56.438 INFO:teuthology.orchestra.run.smithi112.stdout:up {} 2024-09-16T00:41:56.438 INFO:teuthology.orchestra.run.smithi112.stdout:failed 0,1 2024-09-16T00:41:56.438 INFO:teuthology.orchestra.run.smithi112.stdout:damaged 2024-09-16T00:41:56.438 INFO:teuthology.orchestra.run.smithi112.stdout:stopped 2024-09-16T00:41:56.438 INFO:teuthology.orchestra.run.smithi112.stdout:data_pools [3] 2024-09-16T00:41:56.438 INFO:teuthology.orchestra.run.smithi112.stdout:metadata_pool 2 2024-09-16T00:41:56.438 INFO:teuthology.orchestra.run.smithi112.stdout:inline_data enabled 2024-09-16T00:41:56.438 INFO:teuthology.orchestra.run.smithi112.stdout:balancer 2024-09-16T00:41:56.438 INFO:teuthology.orchestra.run.smithi112.stdout:bal_rank_mask -1 2024-09-16T00:41:56.438 INFO:teuthology.orchestra.run.smithi112.stdout:standby_count_wanted 1 2024-09-16T00:41:56.438 INFO:teuthology.orchestra.run.smithi112.stdout:qdb_cluster leader: 0 members: 2024-09-16T00:41:56.438 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:41:56.439 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:41:56.439 INFO:teuthology.orchestra.run.smithi112.stdout:Standby daemons: 2024-09-16T00:41:56.439 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:41:56.439 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.ithwzj{-1:34392} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.112:6826/476229221,v1:172.21.15.112:6827/476229221] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:41:56.439 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.hxfwkh{-1:34404} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.112:6828/2356899876,v1:172.21.15.112:6829/2356899876] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:41:56.439 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.gawwgw{-1:44343} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.169:6824/1635329957,v1:172.21.15.169:6825/1635329957] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:41:56.439 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.lgeqrj{-1:44347} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.169:6826/184810031,v1:172.21.15.169:6827/184810031] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:41:56.440 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 34 2024-09-16T00:41:56.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.383+0000 7f377bc42640 1 -- 172.21.15.112:0/3670830541 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f376006e530 msgr2=0x7f37600709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:56.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.383+0000 7f377bc42640 1 --2- 172.21.15.112:0/3670830541 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f376006e530 0x7f37600709e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3770004ed0 tx=0x7f377000b040 comp rx=0 tx=0).stop 2024-09-16T00:41:56.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.383+0000 7f377bc42640 1 -- 172.21.15.112:0/3670830541 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3774102ef0 msgr2=0x7f3774113a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:56.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.383+0000 7f377bc42640 1 --2- 172.21.15.112:0/3670830541 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3774102ef0 0x7f3774113a60 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f37640043d0 tx=0x7f3764009da0 comp rx=0 tx=0).stop 2024-09-16T00:41:56.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.383+0000 7f377bc42640 1 -- 172.21.15.112:0/3670830541 shutdown_connections 2024-09-16T00:41:56.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.383+0000 7f377bc42640 1 --2- 172.21.15.112:0/3670830541 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f376006e530 0x7f37600709e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:56.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.383+0000 7f377bc42640 1 --2- 172.21.15.112:0/3670830541 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3774102ef0 0x7f3774113a60 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:56.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.383+0000 7f377bc42640 1 --2- 172.21.15.112:0/3670830541 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3774102550 0x7f3774113520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:56.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.383+0000 7f377bc42640 1 -- 172.21.15.112:0/3670830541 >> 172.21.15.112:0/3670830541 conn(0x7f37740f80f0 msgr2=0x7f37740f9150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:56.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.383+0000 7f377bc42640 1 -- 172.21.15.112:0/3670830541 shutdown_connections 2024-09-16T00:41:56.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.383+0000 7f377bc42640 1 -- 172.21.15.112:0/3670830541 wait complete. 2024-09-16T00:41:56.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.462+0000 7f33d06f5640 1 -- 172.21.15.112:0/3277870899 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33c80fe460 msgr2=0x7f33c80fe830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:56.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.462+0000 7f33d06f5640 1 --2- 172.21.15.112:0/3277870899 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33c80fe460 0x7f33c80fe830 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f33c40048d0 tx=0x7f33c4009da0 comp rx=0 tx=0).stop 2024-09-16T00:41:56.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.463+0000 7f33d06f5640 1 -- 172.21.15.112:0/3277870899 shutdown_connections 2024-09-16T00:41:56.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.463+0000 7f33d06f5640 1 --2- 172.21.15.112:0/3277870899 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33c8064fc0 0x7f33c8065430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:56.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.463+0000 7f33d06f5640 1 --2- 172.21.15.112:0/3277870899 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33c80fe460 0x7f33c80fe830 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:56.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.463+0000 7f33d06f5640 1 -- 172.21.15.112:0/3277870899 >> 172.21.15.112:0/3277870899 conn(0x7f33c80723b0 msgr2=0x7f33c80727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:56.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.463+0000 7f33d06f5640 1 -- 172.21.15.112:0/3277870899 shutdown_connections 2024-09-16T00:41:56.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.463+0000 7f33d06f5640 1 -- 172.21.15.112:0/3277870899 wait complete. 2024-09-16T00:41:56.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.463+0000 7f33d06f5640 1 Processor -- start 2024-09-16T00:41:56.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.463+0000 7f33d06f5640 1 -- start start 2024-09-16T00:41:56.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.463+0000 7f33d06f5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33c8064fc0 0x7f33c8113190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:56.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.464+0000 7f33d06f5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33c80fe460 0x7f33c81136d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:56.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.464+0000 7f33d06f5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c8113cd0 con 0x7f33c8064fc0 2024-09-16T00:41:56.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.464+0000 7f33d06f5640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c810d110 con 0x7f33c80fe460 2024-09-16T00:41:56.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.464+0000 7f33ce491640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33c8064fc0 0x7f33c8113190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:56.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.464+0000 7f33ce491640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33c8064fc0 0x7f33c8113190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58142/0 (socket says 172.21.15.112:58142) 2024-09-16T00:41:56.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.464+0000 7f33ce491640 1 -- 172.21.15.112:0/3715569619 learned_addr learned my addr 172.21.15.112:0/3715569619 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:41:56.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.464+0000 7f33cdc90640 1 --2- 172.21.15.112:0/3715569619 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33c80fe460 0x7f33c81136d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:56.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.464+0000 7f33ce491640 1 -- 172.21.15.112:0/3715569619 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33c80fe460 msgr2=0x7f33c81136d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:56.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.464+0000 7f33ce491640 1 --2- 172.21.15.112:0/3715569619 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33c80fe460 0x7f33c81136d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:56.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.464+0000 7f33ce491640 1 -- 172.21.15.112:0/3715569619 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33c4005040 con 0x7f33c8064fc0 2024-09-16T00:41:56.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.464+0000 7f33ce491640 1 --2- 172.21.15.112:0/3715569619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33c8064fc0 0x7f33c8113190 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f33c4003080 tx=0x7f33c4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:56.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.464+0000 7f33beffd640 1 -- 172.21.15.112:0/3715569619 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f33c4038070 con 0x7f33c8064fc0 2024-09-16T00:41:56.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.464+0000 7f33beffd640 1 -- 172.21.15.112:0/3715569619 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f33c40097e0 con 0x7f33c8064fc0 2024-09-16T00:41:56.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.464+0000 7f33beffd640 1 -- 172.21.15.112:0/3715569619 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f33c403d420 con 0x7f33c8064fc0 2024-09-16T00:41:56.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.465+0000 7f33d06f5640 1 -- 172.21.15.112:0/3715569619 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33c810d390 con 0x7f33c8064fc0 2024-09-16T00:41:56.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.465+0000 7f33d06f5640 1 -- 172.21.15.112:0/3715569619 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33c810d8d0 con 0x7f33c8064fc0 2024-09-16T00:41:56.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.466+0000 7f33d06f5640 1 -- 172.21.15.112:0/3715569619 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33c80fdcf0 con 0x7f33c8064fc0 2024-09-16T00:41:56.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.466+0000 7f33beffd640 1 -- 172.21.15.112:0/3715569619 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f33c4030020 con 0x7f33c8064fc0 2024-09-16T00:41:56.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.467+0000 7f33beffd640 1 --2- 172.21.15.112:0/3715569619 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f33b406e530 0x7f33b40709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:56.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.467+0000 7f33beffd640 1 -- 172.21.15.112:0/3715569619 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f33c40ae760 con 0x7f33c8064fc0 2024-09-16T00:41:56.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.467+0000 7f33cdc90640 1 --2- 172.21.15.112:0/3715569619 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f33b406e530 0x7f33b40709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:56.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.467+0000 7f33cdc90640 1 --2- 172.21.15.112:0/3715569619 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f33b406e530 0x7f33b40709e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f33c810ea50 tx=0x7f33b8009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:56.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.471+0000 7f33beffd640 1 -- 172.21.15.112:0/3715569619 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f33c4077750 con 0x7f33c8064fc0 2024-09-16T00:41:56.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.653+0000 7f33d06f5640 1 -- 172.21.15.112:0/3715569619 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f33c804b720 con 0x7f33b406e530 2024-09-16T00:41:56.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.654+0000 7f33beffd640 1 -- 172.21.15.112:0/3715569619 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f33c804b720 con 0x7f33b406e530 2024-09-16T00:41:56.655 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:41:56.655 INFO:teuthology.orchestra.run.smithi112.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80", 2024-09-16T00:41:56.655 INFO:teuthology.orchestra.run.smithi112.stdout: "in_progress": true, 2024-09-16T00:41:56.655 INFO:teuthology.orchestra.run.smithi112.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T00:41:56.655 INFO:teuthology.orchestra.run.smithi112.stdout: "services_complete": [ 2024-09-16T00:41:56.655 INFO:teuthology.orchestra.run.smithi112.stdout: "mds", 2024-09-16T00:41:56.655 INFO:teuthology.orchestra.run.smithi112.stdout: "osd", 2024-09-16T00:41:56.655 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr", 2024-09-16T00:41:56.656 INFO:teuthology.orchestra.run.smithi112.stdout: "mon", 2024-09-16T00:41:56.656 INFO:teuthology.orchestra.run.smithi112.stdout: "crash" 2024-09-16T00:41:56.656 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-16T00:41:56.656 INFO:teuthology.orchestra.run.smithi112.stdout: "progress": "16/21 daemons upgraded", 2024-09-16T00:41:56.656 INFO:teuthology.orchestra.run.smithi112.stdout: "message": "Currently upgrading mds daemons", 2024-09-16T00:41:56.656 INFO:teuthology.orchestra.run.smithi112.stdout: "is_paused": false 2024-09-16T00:41:56.656 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:41:56.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.656+0000 7f33d06f5640 1 -- 172.21.15.112:0/3715569619 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f33b406e530 msgr2=0x7f33b40709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:56.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.656+0000 7f33d06f5640 1 --2- 172.21.15.112:0/3715569619 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f33b406e530 0x7f33b40709e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f33c810ea50 tx=0x7f33b8009040 comp rx=0 tx=0).stop 2024-09-16T00:41:56.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.656+0000 7f33d06f5640 1 -- 172.21.15.112:0/3715569619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33c8064fc0 msgr2=0x7f33c8113190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:56.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.656+0000 7f33d06f5640 1 --2- 172.21.15.112:0/3715569619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33c8064fc0 0x7f33c8113190 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f33c4003080 tx=0x7f33c4009da0 comp rx=0 tx=0).stop 2024-09-16T00:41:56.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.656+0000 7f33d06f5640 1 -- 172.21.15.112:0/3715569619 shutdown_connections 2024-09-16T00:41:56.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.656+0000 7f33d06f5640 1 --2- 172.21.15.112:0/3715569619 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f33b406e530 0x7f33b40709e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:56.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.656+0000 7f33d06f5640 1 --2- 172.21.15.112:0/3715569619 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33c80fe460 0x7f33c81136d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:56.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.656+0000 7f33d06f5640 1 --2- 172.21.15.112:0/3715569619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33c8064fc0 0x7f33c8113190 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:56.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.656+0000 7f33d06f5640 1 -- 172.21.15.112:0/3715569619 >> 172.21.15.112:0/3715569619 conn(0x7f33c80723b0 msgr2=0x7f33c8105410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:56.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.656+0000 7f33d06f5640 1 -- 172.21.15.112:0/3715569619 shutdown_connections 2024-09-16T00:41:56.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.656+0000 7f33d06f5640 1 -- 172.21.15.112:0/3715569619 wait complete. 2024-09-16T00:41:56.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.735+0000 7fa2a4a4e640 1 -- 172.21.15.112:0/1266194720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2a0101f40 msgr2=0x7fa2a00fb4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:56.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.735+0000 7fa2a4a4e640 1 --2- 172.21.15.112:0/1266194720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2a0101f40 0x7fa2a00fb4f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa2880048d0 tx=0x7fa288009da0 comp rx=0 tx=0).stop 2024-09-16T00:41:56.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.735+0000 7fa2a4a4e640 1 -- 172.21.15.112:0/1266194720 shutdown_connections 2024-09-16T00:41:56.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.735+0000 7fa2a4a4e640 1 --2- 172.21.15.112:0/1266194720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2a0101f40 0x7fa2a00fb4f0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:56.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.735+0000 7fa2a4a4e640 1 --2- 172.21.15.112:0/1266194720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2a01015a0 0x7fa2a0101970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:56.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.735+0000 7fa2a4a4e640 1 -- 172.21.15.112:0/1266194720 >> 172.21.15.112:0/1266194720 conn(0x7fa2a00f7030 msgr2=0x7fa2a00f9440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:56.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.735+0000 7fa2a4a4e640 1 -- 172.21.15.112:0/1266194720 shutdown_connections 2024-09-16T00:41:56.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.735+0000 7fa2a4a4e640 1 -- 172.21.15.112:0/1266194720 wait complete. 2024-09-16T00:41:56.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.736+0000 7fa2a4a4e640 1 Processor -- start 2024-09-16T00:41:56.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.736+0000 7fa2a4a4e640 1 -- start start 2024-09-16T00:41:56.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.736+0000 7fa2a4a4e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2a01015a0 0x7fa2a010c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:56.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.736+0000 7fa2a4a4e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2a0101f40 0x7fa2a010c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:56.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.736+0000 7fa2a4a4e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a010cb90 con 0x7fa2a01015a0 2024-09-16T00:41:56.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.736+0000 7fa2a4a4e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a010ccd0 con 0x7fa2a0101f40 2024-09-16T00:41:56.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.736+0000 7fa29e59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2a01015a0 0x7fa2a010c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:56.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.736+0000 7fa29e59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2a01015a0 0x7fa2a010c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58164/0 (socket says 172.21.15.112:58164) 2024-09-16T00:41:56.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.736+0000 7fa29e59c640 1 -- 172.21.15.112:0/3985813916 learned_addr learned my addr 172.21.15.112:0/3985813916 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:41:56.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.736+0000 7fa29dd9b640 1 --2- 172.21.15.112:0/3985813916 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2a0101f40 0x7fa2a010c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:56.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.736+0000 7fa29e59c640 1 -- 172.21.15.112:0/3985813916 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2a0101f40 msgr2=0x7fa2a010c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:56.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.736+0000 7fa29e59c640 1 --2- 172.21.15.112:0/3985813916 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2a0101f40 0x7fa2a010c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:56.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.736+0000 7fa29e59c640 1 -- 172.21.15.112:0/3985813916 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa288005040 con 0x7fa2a01015a0 2024-09-16T00:41:56.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.736+0000 7fa29dd9b640 1 --2- 172.21.15.112:0/3985813916 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2a0101f40 0x7fa2a010c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:41:56.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.737+0000 7fa29e59c640 1 --2- 172.21.15.112:0/3985813916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2a01015a0 0x7fa2a010c110 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fa29000c790 tx=0x7fa2900043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:56.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.737+0000 7fa296ffd640 1 -- 172.21.15.112:0/3985813916 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa29000a820 con 0x7fa2a01015a0 2024-09-16T00:41:56.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.737+0000 7fa296ffd640 1 -- 172.21.15.112:0/3985813916 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fa29000ae20 con 0x7fa2a01015a0 2024-09-16T00:41:56.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.737+0000 7fa2a4a4e640 1 -- 172.21.15.112:0/3985813916 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2a0119780 con 0x7fa2a01015a0 2024-09-16T00:41:56.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.737+0000 7fa296ffd640 1 -- 172.21.15.112:0/3985813916 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2900113a0 con 0x7fa2a01015a0 2024-09-16T00:41:56.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.737+0000 7fa2a4a4e640 1 -- 172.21.15.112:0/3985813916 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2a0119b30 con 0x7fa2a01015a0 2024-09-16T00:41:56.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.738+0000 7fa2a4a4e640 1 -- 172.21.15.112:0/3985813916 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2a00fac90 con 0x7fa2a01015a0 2024-09-16T00:41:56.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.739+0000 7fa296ffd640 1 -- 172.21.15.112:0/3985813916 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fa290012400 con 0x7fa2a01015a0 2024-09-16T00:41:56.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.739+0000 7fa296ffd640 1 --2- 172.21.15.112:0/3985813916 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa28c06e530 0x7fa28c0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:41:56.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.740+0000 7fa29dd9b640 1 --2- 172.21.15.112:0/3985813916 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa28c06e530 0x7fa28c0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:41:56.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.740+0000 7fa296ffd640 1 -- 172.21.15.112:0/3985813916 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7fa29008e380 con 0x7fa2a01015a0 2024-09-16T00:41:56.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.740+0000 7fa29dd9b640 1 --2- 172.21.15.112:0/3985813916 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa28c06e530 0x7fa28c0709e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa288000c00 tx=0x7fa288035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:41:56.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.744+0000 7fa296ffd640 1 -- 172.21.15.112:0/3985813916 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fa290057410 con 0x7fa2a01015a0 2024-09-16T00:41:56.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.977+0000 7fa2a4a4e640 1 -- 172.21.15.112:0/3985813916 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa2a01085d0 con 0x7fa2a01015a0 2024-09-16T00:41:56.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.977+0000 7fa296ffd640 1 -- 172.21.15.112:0/3985813916 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7fa29005aad0 con 0x7fa2a01015a0 2024-09-16T00:41:56.979 INFO:teuthology.orchestra.run.smithi112.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-16T00:41:56.979 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-16T00:41:56.979 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs is degraded 2024-09-16T00:41:56.979 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T00:41:56.979 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T00:41:56.979 INFO:teuthology.orchestra.run.smithi112.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-16T00:41:56.979 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-16T00:41:56.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.979+0000 7fa2a4a4e640 1 -- 172.21.15.112:0/3985813916 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa28c06e530 msgr2=0x7fa28c0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:56.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.979+0000 7fa2a4a4e640 1 --2- 172.21.15.112:0/3985813916 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa28c06e530 0x7fa28c0709e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa288000c00 tx=0x7fa288035040 comp rx=0 tx=0).stop 2024-09-16T00:41:56.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.979+0000 7fa2a4a4e640 1 -- 172.21.15.112:0/3985813916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2a01015a0 msgr2=0x7fa2a010c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:41:56.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.979+0000 7fa2a4a4e640 1 --2- 172.21.15.112:0/3985813916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2a01015a0 0x7fa2a010c110 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fa29000c790 tx=0x7fa2900043c0 comp rx=0 tx=0).stop 2024-09-16T00:41:56.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.980+0000 7fa2a4a4e640 1 -- 172.21.15.112:0/3985813916 shutdown_connections 2024-09-16T00:41:56.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.980+0000 7fa2a4a4e640 1 --2- 172.21.15.112:0/3985813916 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa28c06e530 0x7fa28c0709e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:56.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.980+0000 7fa2a4a4e640 1 --2- 172.21.15.112:0/3985813916 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2a0101f40 0x7fa2a010c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:56.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.980+0000 7fa2a4a4e640 1 --2- 172.21.15.112:0/3985813916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2a01015a0 0x7fa2a010c110 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:41:56.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.980+0000 7fa2a4a4e640 1 -- 172.21.15.112:0/3985813916 >> 172.21.15.112:0/3985813916 conn(0x7fa2a00f7030 msgr2=0x7fa2a00f80a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:41:56.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.980+0000 7fa2a4a4e640 1 -- 172.21.15.112:0/3985813916 shutdown_connections 2024-09-16T00:41:56.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:41:56.980+0000 7fa2a4a4e640 1 -- 172.21.15.112:0/3985813916 wait complete. 2024-09-16T00:41:57.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:41:56 smithi169 ceph-mon[77626]: from='client.34828 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:41:57.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:41:56 smithi169 ceph-mon[77626]: from='client.34832 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:41:57.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:41:56 smithi169 ceph-mon[77626]: from='client.34836 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:41:57.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:41:56 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/1984262374' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:41:57.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:41:56 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/3670830541' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:41:57.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:41:56 smithi112 ceph-mon[97750]: from='client.34828 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:41:57.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:41:56 smithi112 ceph-mon[97750]: from='client.34832 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:41:57.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:41:56 smithi112 ceph-mon[97750]: from='client.34836 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:41:57.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:41:56 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/1984262374' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:41:57.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:41:56 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/3670830541' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:41:58.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:41:57 smithi169 ceph-mon[77626]: from='client.34848 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:41:58.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:41:57 smithi169 ceph-mon[77626]: pgmap v426: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:41:58.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:41:57 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/3985813916' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:41:58.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:41:57 smithi112 ceph-mon[97750]: from='client.34848 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:41:58.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:41:57 smithi112 ceph-mon[97750]: pgmap v426: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:41:58.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:41:57 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/3985813916' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:42:00.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:41:59 smithi169 ceph-mon[77626]: pgmap v427: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:42:00.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:41:59 smithi112 ceph-mon[97750]: pgmap v427: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:42:02.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:42:01 smithi169 ceph-mon[77626]: pgmap v428: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:42:02.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:42:01 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:42:02.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:42:01 smithi112 ceph-mon[97750]: pgmap v428: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:42:02.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:42:01 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:42:04.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:42:03 smithi112 ceph-mon[97750]: pgmap v429: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:42:04.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:42:03 smithi169 ceph-mon[77626]: pgmap v429: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:42:06.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:42:05 smithi112 ceph-mon[97750]: pgmap v430: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:42:06.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:42:05 smithi169 ceph-mon[77626]: pgmap v430: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:42:08.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:42:07 smithi112 ceph-mon[97750]: pgmap v431: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:42:08.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:42:07 smithi169 ceph-mon[77626]: pgmap v431: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:42:10.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:42:09 smithi112 ceph-mon[97750]: pgmap v432: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:42:10.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:42:09 smithi169 ceph-mon[77626]: pgmap v432: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:42:12.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:42:11 smithi112 ceph-mon[97750]: pgmap v433: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:42:12.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:42:11 smithi169 ceph-mon[77626]: pgmap v433: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:42:14.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:42:13 smithi112 ceph-mon[97750]: pgmap v434: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:42:14.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:42:13 smithi169 ceph-mon[77626]: pgmap v434: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:42:16.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:42:15 smithi112 ceph-mon[97750]: pgmap v435: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:42:16.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:42:15 smithi169 ceph-mon[77626]: pgmap v435: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:42:17.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:42:16 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:42:17.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:42:16 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:42:18.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:42:17 smithi169 ceph-mon[77626]: pgmap v436: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:42:18.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:42:17 smithi112 ceph-mon[97750]: pgmap v436: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:42:20.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:42:19 smithi112 ceph-mon[97750]: pgmap v437: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:42:20.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:42:19 smithi169 ceph-mon[77626]: pgmap v437: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:42:21.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:42:21 smithi169 ceph-mon[77626]: pgmap v438: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:42:21.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:42:21 smithi112 ceph-mon[97750]: pgmap v438: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:42:24.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:42:23 smithi169 ceph-mon[77626]: pgmap v439: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:42:24.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:42:23 smithi112 ceph-mon[97750]: pgmap v439: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:42:26.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:42:25 smithi169 ceph-mon[77626]: pgmap v440: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:42:26.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:42:25 smithi112 ceph-mon[97750]: pgmap v440: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:42:27.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.071+0000 7ff0790ee640 1 -- 172.21.15.112:0/1588746076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff074102ed0 msgr2=0x7ff0740fc590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:27.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.071+0000 7ff0790ee640 1 --2- 172.21.15.112:0/1588746076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff074102ed0 0x7ff0740fc590 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff05c0048d0 tx=0x7ff05c009da0 comp rx=0 tx=0).stop 2024-09-16T00:42:27.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.071+0000 7ff0790ee640 1 -- 172.21.15.112:0/1588746076 shutdown_connections 2024-09-16T00:42:27.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.071+0000 7ff0790ee640 1 --2- 172.21.15.112:0/1588746076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff074102ed0 0x7ff0740fc590 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:27.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.071+0000 7ff0790ee640 1 --2- 172.21.15.112:0/1588746076 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff074102530 0x7ff074102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:27.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.071+0000 7ff0790ee640 1 -- 172.21.15.112:0/1588746076 >> 172.21.15.112:0/1588746076 conn(0x7ff0740f80f0 msgr2=0x7ff0740fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:27.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.071+0000 7ff0790ee640 1 -- 172.21.15.112:0/1588746076 shutdown_connections 2024-09-16T00:42:27.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.071+0000 7ff0790ee640 1 -- 172.21.15.112:0/1588746076 wait complete. 2024-09-16T00:42:27.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.072+0000 7ff0790ee640 1 Processor -- start 2024-09-16T00:42:27.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.072+0000 7ff0790ee640 1 -- start start 2024-09-16T00:42:27.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.072+0000 7ff0790ee640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff074102530 0x7ff0741132b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:27.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.072+0000 7ff0790ee640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff074102ed0 0x7ff0741137f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:27.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.072+0000 7ff0790ee640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff074113e80 con 0x7ff074102ed0 2024-09-16T00:42:27.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.072+0000 7ff0790ee640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff07410d230 con 0x7ff074102530 2024-09-16T00:42:27.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.072+0000 7ff06a59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff074102ed0 0x7ff0741137f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:27.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.072+0000 7ff06a59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff074102ed0 0x7ff0741137f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47278/0 (socket says 172.21.15.112:47278) 2024-09-16T00:42:27.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.072+0000 7ff06a59c640 1 -- 172.21.15.112:0/709563400 learned_addr learned my addr 172.21.15.112:0/709563400 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:42:27.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.073+0000 7ff072d9d640 1 --2- 172.21.15.112:0/709563400 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff074102530 0x7ff0741132b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:27.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.073+0000 7ff06a59c640 1 -- 172.21.15.112:0/709563400 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff074102530 msgr2=0x7ff0741132b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:27.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.073+0000 7ff06a59c640 1 --2- 172.21.15.112:0/709563400 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff074102530 0x7ff0741132b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:27.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.073+0000 7ff06a59c640 1 -- 172.21.15.112:0/709563400 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff05c005040 con 0x7ff074102ed0 2024-09-16T00:42:27.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.073+0000 7ff06a59c640 1 --2- 172.21.15.112:0/709563400 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff074102ed0 0x7ff0741137f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff05c003080 tx=0x7ff05c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:27.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.073+0000 7ff06bfff640 1 -- 172.21.15.112:0/709563400 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff05c038070 con 0x7ff074102ed0 2024-09-16T00:42:27.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.073+0000 7ff06bfff640 1 -- 172.21.15.112:0/709563400 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7ff05c009760 con 0x7ff074102ed0 2024-09-16T00:42:27.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.073+0000 7ff0790ee640 1 -- 172.21.15.112:0/709563400 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff07410d4b0 con 0x7ff074102ed0 2024-09-16T00:42:27.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.073+0000 7ff06bfff640 1 -- 172.21.15.112:0/709563400 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff05c033420 con 0x7ff074102ed0 2024-09-16T00:42:27.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.073+0000 7ff0790ee640 1 -- 172.21.15.112:0/709563400 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff07410d910 con 0x7ff074102ed0 2024-09-16T00:42:27.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.074+0000 7ff0790ee640 1 -- 172.21.15.112:0/709563400 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0740fdcb0 con 0x7ff074102ed0 2024-09-16T00:42:27.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.075+0000 7ff06bfff640 1 -- 172.21.15.112:0/709563400 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7ff05c030020 con 0x7ff074102ed0 2024-09-16T00:42:27.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.075+0000 7ff06bfff640 1 --2- 172.21.15.112:0/709563400 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff06006e530 0x7ff0600709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:27.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.075+0000 7ff06bfff640 1 -- 172.21.15.112:0/709563400 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7ff05c0adf20 con 0x7ff074102ed0 2024-09-16T00:42:27.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.076+0000 7ff072d9d640 1 --2- 172.21.15.112:0/709563400 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff06006e530 0x7ff0600709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:27.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.076+0000 7ff072d9d640 1 --2- 172.21.15.112:0/709563400 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff06006e530 0x7ff0600709e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff064006fd0 tx=0x7ff064008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:27.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.080+0000 7ff06bfff640 1 -- 172.21.15.112:0/709563400 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7ff05c0e31b0 con 0x7ff074102ed0 2024-09-16T00:42:27.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.262+0000 7ff0790ee640 1 -- 172.21.15.112:0/709563400 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff0741095f0 con 0x7ff06006e530 2024-09-16T00:42:27.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.263+0000 7ff06bfff640 1 -- 172.21.15.112:0/709563400 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7ff0741095f0 con 0x7ff06006e530 2024-09-16T00:42:27.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.265+0000 7ff0790ee640 1 -- 172.21.15.112:0/709563400 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff06006e530 msgr2=0x7ff0600709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:27.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.265+0000 7ff0790ee640 1 --2- 172.21.15.112:0/709563400 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff06006e530 0x7ff0600709e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff064006fd0 tx=0x7ff064008040 comp rx=0 tx=0).stop 2024-09-16T00:42:27.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.265+0000 7ff0790ee640 1 -- 172.21.15.112:0/709563400 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff074102ed0 msgr2=0x7ff0741137f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:27.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.265+0000 7ff0790ee640 1 --2- 172.21.15.112:0/709563400 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff074102ed0 0x7ff0741137f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff05c003080 tx=0x7ff05c009da0 comp rx=0 tx=0).stop 2024-09-16T00:42:27.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.265+0000 7ff0790ee640 1 -- 172.21.15.112:0/709563400 shutdown_connections 2024-09-16T00:42:27.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.265+0000 7ff0790ee640 1 --2- 172.21.15.112:0/709563400 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff06006e530 0x7ff0600709e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:27.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.265+0000 7ff0790ee640 1 --2- 172.21.15.112:0/709563400 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff074102ed0 0x7ff0741137f0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:27.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.265+0000 7ff0790ee640 1 --2- 172.21.15.112:0/709563400 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff074102530 0x7ff0741132b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:27.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.265+0000 7ff0790ee640 1 -- 172.21.15.112:0/709563400 >> 172.21.15.112:0/709563400 conn(0x7ff0740f80f0 msgr2=0x7ff0740f9130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:27.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.265+0000 7ff0790ee640 1 -- 172.21.15.112:0/709563400 shutdown_connections 2024-09-16T00:42:27.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.265+0000 7ff0790ee640 1 -- 172.21.15.112:0/709563400 wait complete. 2024-09-16T00:42:27.488 INFO:teuthology.orchestra.run.smithi112.stdout:true 2024-09-16T00:42:27.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.346+0000 7f5953d82640 1 -- 172.21.15.112:0/538099912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f594c1014e0 msgr2=0x7f594c1018b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:27.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.346+0000 7f5953d82640 1 --2- 172.21.15.112:0/538099912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f594c1014e0 0x7f594c1018b0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f59480048d0 tx=0x7f5948009da0 comp rx=0 tx=0).stop 2024-09-16T00:42:27.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.346+0000 7f5953d82640 1 -- 172.21.15.112:0/538099912 shutdown_connections 2024-09-16T00:42:27.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.346+0000 7f5953d82640 1 --2- 172.21.15.112:0/538099912 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f594c0fb4a0 0x7f594c0fb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:27.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.346+0000 7f5953d82640 1 --2- 172.21.15.112:0/538099912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f594c1014e0 0x7f594c1018b0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:27.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.346+0000 7f5953d82640 1 -- 172.21.15.112:0/538099912 >> 172.21.15.112:0/538099912 conn(0x7f594c0f7060 msgr2=0x7f594c0f9470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:27.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.346+0000 7f5953d82640 1 -- 172.21.15.112:0/538099912 shutdown_connections 2024-09-16T00:42:27.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.346+0000 7f5953d82640 1 -- 172.21.15.112:0/538099912 wait complete. 2024-09-16T00:42:27.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.346+0000 7f5953d82640 1 Processor -- start 2024-09-16T00:42:27.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.346+0000 7f5953d82640 1 -- start start 2024-09-16T00:42:27.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.347+0000 7f5953d82640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f594c0fb4a0 0x7f594c10e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:27.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.347+0000 7f5953d82640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f594c1014e0 0x7f594c10e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:27.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.347+0000 7f5953d82640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f594c10ecf0 con 0x7f594c1014e0 2024-09-16T00:42:27.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.347+0000 7f5953d82640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f594c10ee60 con 0x7f594c0fb4a0 2024-09-16T00:42:27.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.347+0000 7f595131d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f594c1014e0 0x7f594c10e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:27.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.347+0000 7f595131d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f594c1014e0 0x7f594c10e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47302/0 (socket says 172.21.15.112:47302) 2024-09-16T00:42:27.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.347+0000 7f595131d640 1 -- 172.21.15.112:0/1721003094 learned_addr learned my addr 172.21.15.112:0/1721003094 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:42:27.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.347+0000 7f5951b1e640 1 --2- 172.21.15.112:0/1721003094 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f594c0fb4a0 0x7f594c10e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:27.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.347+0000 7f595131d640 1 -- 172.21.15.112:0/1721003094 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f594c0fb4a0 msgr2=0x7f594c10e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:27.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.347+0000 7f595131d640 1 --2- 172.21.15.112:0/1721003094 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f594c0fb4a0 0x7f594c10e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:27.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.347+0000 7f595131d640 1 -- 172.21.15.112:0/1721003094 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5948005040 con 0x7f594c1014e0 2024-09-16T00:42:27.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.347+0000 7f595131d640 1 --2- 172.21.15.112:0/1721003094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f594c1014e0 0x7f594c10e6a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f594000c810 tx=0x7f59400043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:27.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.347+0000 7f593e7fc640 1 -- 172.21.15.112:0/1721003094 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f594000a7a0 con 0x7f594c1014e0 2024-09-16T00:42:27.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.347+0000 7f593e7fc640 1 -- 172.21.15.112:0/1721003094 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f594000ada0 con 0x7f594c1014e0 2024-09-16T00:42:27.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.347+0000 7f5953d82640 1 -- 172.21.15.112:0/1721003094 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f594c1171d0 con 0x7f594c1014e0 2024-09-16T00:42:27.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.347+0000 7f593e7fc640 1 -- 172.21.15.112:0/1721003094 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5940011380 con 0x7f594c1014e0 2024-09-16T00:42:27.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.348+0000 7f5953d82640 1 -- 172.21.15.112:0/1721003094 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f594c1176e0 con 0x7f594c1014e0 2024-09-16T00:42:27.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.349+0000 7f5953d82640 1 -- 172.21.15.112:0/1721003094 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f594c0fcbe0 con 0x7f594c1014e0 2024-09-16T00:42:27.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.349+0000 7f593e7fc640 1 -- 172.21.15.112:0/1721003094 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f59400127d0 con 0x7f594c1014e0 2024-09-16T00:42:27.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.349+0000 7f593e7fc640 1 --2- 172.21.15.112:0/1721003094 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f593806e530 0x7f59380709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:27.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.350+0000 7f5951b1e640 1 --2- 172.21.15.112:0/1721003094 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f593806e530 0x7f59380709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:27.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.350+0000 7f593e7fc640 1 -- 172.21.15.112:0/1721003094 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f594008e7e0 con 0x7f594c1014e0 2024-09-16T00:42:27.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.350+0000 7f5951b1e640 1 --2- 172.21.15.112:0/1721003094 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f593806e530 0x7f59380709e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5948009ac0 tx=0x7f5948035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:27.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.354+0000 7f593e7fc640 1 -- 172.21.15.112:0/1721003094 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f5940057750 con 0x7f594c1014e0 2024-09-16T00:42:27.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.535+0000 7f5953d82640 1 -- 172.21.15.112:0/1721003094 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f594c04b720 con 0x7f593806e530 2024-09-16T00:42:27.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.537+0000 7f593e7fc640 1 -- 172.21.15.112:0/1721003094 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f594c04b720 con 0x7f593806e530 2024-09-16T00:42:27.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.538+0000 7f5953d82640 1 -- 172.21.15.112:0/1721003094 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f593806e530 msgr2=0x7f59380709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:27.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.538+0000 7f5953d82640 1 --2- 172.21.15.112:0/1721003094 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f593806e530 0x7f59380709e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5948009ac0 tx=0x7f5948035040 comp rx=0 tx=0).stop 2024-09-16T00:42:27.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.538+0000 7f5953d82640 1 -- 172.21.15.112:0/1721003094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f594c1014e0 msgr2=0x7f594c10e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:27.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.538+0000 7f5953d82640 1 --2- 172.21.15.112:0/1721003094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f594c1014e0 0x7f594c10e6a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f594000c810 tx=0x7f59400043c0 comp rx=0 tx=0).stop 2024-09-16T00:42:27.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.539+0000 7f5953d82640 1 -- 172.21.15.112:0/1721003094 shutdown_connections 2024-09-16T00:42:27.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.539+0000 7f5953d82640 1 --2- 172.21.15.112:0/1721003094 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f593806e530 0x7f59380709e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:27.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.539+0000 7f5953d82640 1 --2- 172.21.15.112:0/1721003094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f594c1014e0 0x7f594c10e6a0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:27.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.539+0000 7f5953d82640 1 --2- 172.21.15.112:0/1721003094 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f594c0fb4a0 0x7f594c10e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:27.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.539+0000 7f5953d82640 1 -- 172.21.15.112:0/1721003094 >> 172.21.15.112:0/1721003094 conn(0x7f594c0f7060 msgr2=0x7f594c0f7430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:27.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.539+0000 7f5953d82640 1 -- 172.21.15.112:0/1721003094 shutdown_connections 2024-09-16T00:42:27.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.539+0000 7f5953d82640 1 -- 172.21.15.112:0/1721003094 wait complete. 2024-09-16T00:42:27.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.616+0000 7f5244eed640 1 -- 172.21.15.112:0/880811905 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5240101500 msgr2=0x7f52401018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:27.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.616+0000 7f5244eed640 1 --2- 172.21.15.112:0/880811905 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5240101500 0x7f52401018d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f52300048d0 tx=0x7f5230009da0 comp rx=0 tx=0).stop 2024-09-16T00:42:27.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.616+0000 7f5244eed640 1 -- 172.21.15.112:0/880811905 shutdown_connections 2024-09-16T00:42:27.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.616+0000 7f5244eed640 1 --2- 172.21.15.112:0/880811905 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52400fb4c0 0x7f52400fb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:27.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.616+0000 7f5244eed640 1 --2- 172.21.15.112:0/880811905 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5240101500 0x7f52401018d0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:27.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.616+0000 7f5244eed640 1 -- 172.21.15.112:0/880811905 >> 172.21.15.112:0/880811905 conn(0x7f52400f70c0 msgr2=0x7f52400f94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:27.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.616+0000 7f5244eed640 1 -- 172.21.15.112:0/880811905 shutdown_connections 2024-09-16T00:42:27.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.616+0000 7f5244eed640 1 -- 172.21.15.112:0/880811905 wait complete. 2024-09-16T00:42:27.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.617+0000 7f5244eed640 1 Processor -- start 2024-09-16T00:42:27.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.617+0000 7f5244eed640 1 -- start start 2024-09-16T00:42:27.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.617+0000 7f5244eed640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52400fb4c0 0x7f5240113170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:27.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.617+0000 7f5244eed640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5240101500 0x7f52401136b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:27.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.617+0000 7f5244eed640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5240113cb0 con 0x7f5240101500 2024-09-16T00:42:27.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.617+0000 7f5244eed640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f524010d0f0 con 0x7f52400fb4c0 2024-09-16T00:42:27.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.617+0000 7f523dd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5240101500 0x7f52401136b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:27.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.617+0000 7f523dd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5240101500 0x7f52401136b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47316/0 (socket says 172.21.15.112:47316) 2024-09-16T00:42:27.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.617+0000 7f523dd9b640 1 -- 172.21.15.112:0/2964871669 learned_addr learned my addr 172.21.15.112:0/2964871669 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:42:27.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.618+0000 7f523e59c640 1 --2- 172.21.15.112:0/2964871669 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52400fb4c0 0x7f5240113170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:27.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.618+0000 7f523dd9b640 1 -- 172.21.15.112:0/2964871669 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52400fb4c0 msgr2=0x7f5240113170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:27.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.618+0000 7f523dd9b640 1 --2- 172.21.15.112:0/2964871669 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52400fb4c0 0x7f5240113170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:27.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.618+0000 7f523dd9b640 1 -- 172.21.15.112:0/2964871669 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5230005040 con 0x7f5240101500 2024-09-16T00:42:27.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.618+0000 7f523dd9b640 1 --2- 172.21.15.112:0/2964871669 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5240101500 0x7f52401136b0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f523400c790 tx=0x7f52340043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:27.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.618+0000 7f522effd640 1 -- 172.21.15.112:0/2964871669 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f523400a7a0 con 0x7f5240101500 2024-09-16T00:42:27.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.618+0000 7f522effd640 1 -- 172.21.15.112:0/2964871669 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f523400ada0 con 0x7f5240101500 2024-09-16T00:42:27.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.618+0000 7f5244eed640 1 -- 172.21.15.112:0/2964871669 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f524010d3d0 con 0x7f5240101500 2024-09-16T00:42:27.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.618+0000 7f522effd640 1 -- 172.21.15.112:0/2964871669 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f52340113a0 con 0x7f5240101500 2024-09-16T00:42:27.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.618+0000 7f5244eed640 1 -- 172.21.15.112:0/2964871669 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f524010d970 con 0x7f5240101500 2024-09-16T00:42:27.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.620+0000 7f5244eed640 1 -- 172.21.15.112:0/2964871669 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52400fcc00 con 0x7f5240101500 2024-09-16T00:42:27.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.620+0000 7f522effd640 1 -- 172.21.15.112:0/2964871669 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f5234011610 con 0x7f5240101500 2024-09-16T00:42:27.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.620+0000 7f522effd640 1 --2- 172.21.15.112:0/2964871669 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f522806e530 0x7f52280709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:27.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.620+0000 7f522effd640 1 -- 172.21.15.112:0/2964871669 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f523408e510 con 0x7f5240101500 2024-09-16T00:42:27.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.620+0000 7f523e59c640 1 --2- 172.21.15.112:0/2964871669 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f522806e530 0x7f52280709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:27.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.621+0000 7f523e59c640 1 --2- 172.21.15.112:0/2964871669 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f522806e530 0x7f52280709e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5230009ac0 tx=0x7f5230035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:27.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.625+0000 7f522effd640 1 -- 172.21.15.112:0/2964871669 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f5234058350 con 0x7f5240101500 2024-09-16T00:42:27.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.807+0000 7f5244eed640 1 -- 172.21.15.112:0/2964871669 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f52401095a0 con 0x7f522806e530 2024-09-16T00:42:27.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.816+0000 7f522effd640 1 -- 172.21.15.112:0/2964871669 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7f52401095a0 con 0x7f522806e530 2024-09-16T00:42:27.816 INFO:teuthology.orchestra.run.smithi112.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T00:42:27.817 INFO:teuthology.orchestra.run.smithi112.stdout:alertmanager.smithi112 smithi112 *:9093,9094 running (15m) 8m ago 24m 24.7M - 0.25.0 c8568f914cd2 d85ea0562022 2024-09-16T00:42:27.817 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi112 smithi112 running (12m) 8m ago 24m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c e4fe58f9702a 2024-09-16T00:42:27.817 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi169 smithi169 running (12m) 7m ago 23m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c 3ba2c81d527f 2024-09-16T00:42:27.817 INFO:teuthology.orchestra.run.smithi112.stdout:grafana.smithi112 smithi112 *:3000 running (14m) 8m ago 24m 82.1M - 9.4.12 2bacad6d85d8 51f9d2013da2 2024-09-16T00:42:27.817 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.hxfwkh smithi112 running (8m) 8m ago 18m 14.2M - 19.1.1-326-g8921c8ef f1455bd0219c 1723069902d6 2024-09-16T00:42:27.817 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.ithwzj smithi112 running (8m) 8m ago 19m 12.2M - 19.1.1-326-g8921c8ef f1455bd0219c e7aad5b81db5 2024-09-16T00:42:27.817 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.gawwgw smithi169 running (8m) 7m ago 18m 15.7M - 19.1.1-326-g8921c8ef f1455bd0219c 469be459ee12 2024-09-16T00:42:27.817 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.lgeqrj smithi169 running (8m) 7m ago 18m 13.3M - 19.1.1-326-g8921c8ef f1455bd0219c 31f0fbd2395d 2024-09-16T00:42:27.817 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi112.xrkkbc smithi112 *:8443,9283,8765 running (15m) 8m ago 25m 504M - 19.1.1-326-g8921c8ef f1455bd0219c a981f7fd2810 2024-09-16T00:42:27.817 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi169.mtrobk smithi169 *:8443,9283,8765 running (14m) 7m ago 23m 442M - 19.1.1-326-g8921c8ef f1455bd0219c 4096e5369c2d 2024-09-16T00:42:27.817 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi112 smithi112 running (13m) 8m ago 25m 49.8M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 5aaec033c042 2024-09-16T00:42:27.817 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi169 smithi169 running (12m) 7m ago 23m 38.9M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 7bc609155f3f 2024-09-16T00:42:27.817 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi112 smithi112 *:9100 running (15m) 8m ago 24m 15.5M - 1.5.0 0da6a335fe13 6404d1d601e2 2024-09-16T00:42:27.818 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi169 smithi169 *:9100 running (15m) 7m ago 23m 15.4M - 1.5.0 0da6a335fe13 5505baf702a6 2024-09-16T00:42:27.818 INFO:teuthology.orchestra.run.smithi112.stdout:osd.0 smithi112 running (11m) 8m ago 22m 105M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 53efc9397017 2024-09-16T00:42:27.818 INFO:teuthology.orchestra.run.smithi112.stdout:osd.1 smithi112 running (11m) 8m ago 22m 98.9M 1557M 19.1.1-326-g8921c8ef f1455bd0219c fe6f1a4a2cf4 2024-09-16T00:42:27.818 INFO:teuthology.orchestra.run.smithi112.stdout:osd.2 smithi112 running (10m) 8m ago 21m 102M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 5f7863712057 2024-09-16T00:42:27.818 INFO:teuthology.orchestra.run.smithi112.stdout:osd.3 smithi169 running (10m) 7m ago 21m 92.1M 2581M 19.1.1-326-g8921c8ef f1455bd0219c ee07b5577810 2024-09-16T00:42:27.818 INFO:teuthology.orchestra.run.smithi112.stdout:osd.4 smithi169 running (9m) 7m ago 20m 103M 2581M 19.1.1-326-g8921c8ef f1455bd0219c 769f026a54eb 2024-09-16T00:42:27.818 INFO:teuthology.orchestra.run.smithi112.stdout:osd.5 smithi169 running (9m) 7m ago 20m 125M 2581M 19.1.1-326-g8921c8ef f1455bd0219c a6007c13cf78 2024-09-16T00:42:27.818 INFO:teuthology.orchestra.run.smithi112.stdout:prometheus.smithi112 smithi112 *:9095 running (14m) 8m ago 23m 64.1M - 2.43.0 a07b618ecd1d 7a1d42e60d2b 2024-09-16T00:42:27.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.818+0000 7f5244eed640 1 -- 172.21.15.112:0/2964871669 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f522806e530 msgr2=0x7f52280709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:27.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.819+0000 7f5244eed640 1 --2- 172.21.15.112:0/2964871669 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f522806e530 0x7f52280709e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5230009ac0 tx=0x7f5230035040 comp rx=0 tx=0).stop 2024-09-16T00:42:27.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.819+0000 7f5244eed640 1 -- 172.21.15.112:0/2964871669 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5240101500 msgr2=0x7f52401136b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:27.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.819+0000 7f5244eed640 1 --2- 172.21.15.112:0/2964871669 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5240101500 0x7f52401136b0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f523400c790 tx=0x7f52340043c0 comp rx=0 tx=0).stop 2024-09-16T00:42:27.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.819+0000 7f5244eed640 1 -- 172.21.15.112:0/2964871669 shutdown_connections 2024-09-16T00:42:27.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.819+0000 7f5244eed640 1 --2- 172.21.15.112:0/2964871669 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f522806e530 0x7f52280709e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:27.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.819+0000 7f5244eed640 1 --2- 172.21.15.112:0/2964871669 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5240101500 0x7f52401136b0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:27.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.819+0000 7f5244eed640 1 --2- 172.21.15.112:0/2964871669 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52400fb4c0 0x7f5240113170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:27.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.819+0000 7f5244eed640 1 -- 172.21.15.112:0/2964871669 >> 172.21.15.112:0/2964871669 conn(0x7f52400f70c0 msgr2=0x7f52400f7490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:27.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.819+0000 7f5244eed640 1 -- 172.21.15.112:0/2964871669 shutdown_connections 2024-09-16T00:42:27.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.819+0000 7f5244eed640 1 -- 172.21.15.112:0/2964871669 wait complete. 2024-09-16T00:42:27.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.895+0000 7fd0c2f7f640 1 -- 172.21.15.112:0/1222309367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0bc064fe0 msgr2=0x7fd0bc065430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:27.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.895+0000 7fd0c2f7f640 1 --2- 172.21.15.112:0/1222309367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0bc064fe0 0x7fd0bc065430 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0009c50 tx=0x7fd0b0004d10 comp rx=0 tx=0).stop 2024-09-16T00:42:27.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.895+0000 7fd0c2f7f640 1 -- 172.21.15.112:0/1222309367 shutdown_connections 2024-09-16T00:42:27.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.895+0000 7fd0c2f7f640 1 --2- 172.21.15.112:0/1222309367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0bc064fe0 0x7fd0bc065430 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:27.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.895+0000 7fd0c2f7f640 1 --2- 172.21.15.112:0/1222309367 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd0bc0fe450 0x7fd0bc0fe820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:27.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.895+0000 7fd0c2f7f640 1 -- 172.21.15.112:0/1222309367 >> 172.21.15.112:0/1222309367 conn(0x7fd0bc0723b0 msgr2=0x7fd0bc0727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:27.897 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:42:27 smithi112 ceph-mon[97750]: pgmap v441: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:42:27.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.896+0000 7fd0c2f7f640 1 -- 172.21.15.112:0/1222309367 shutdown_connections 2024-09-16T00:42:27.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.896+0000 7fd0c2f7f640 1 -- 172.21.15.112:0/1222309367 wait complete. 2024-09-16T00:42:27.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.896+0000 7fd0c2f7f640 1 Processor -- start 2024-09-16T00:42:27.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.896+0000 7fd0c2f7f640 1 -- start start 2024-09-16T00:42:27.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.897+0000 7fd0c2f7f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0bc064fe0 0x7fd0bc0fb560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:27.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.897+0000 7fd0c2f7f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd0bc0fe450 0x7fd0bc0fbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:27.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.897+0000 7fd0c2f7f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0bc0fbfe0 con 0x7fd0bc064fe0 2024-09-16T00:42:27.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.897+0000 7fd0c2f7f640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0bc0fc150 con 0x7fd0bc0fe450 2024-09-16T00:42:27.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.897+0000 7fd0c0d1b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0bc064fe0 0x7fd0bc0fb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:27.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.897+0000 7fd0c0d1b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0bc064fe0 0x7fd0bc0fb560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47326/0 (socket says 172.21.15.112:47326) 2024-09-16T00:42:27.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.897+0000 7fd0c0d1b640 1 -- 172.21.15.112:0/1112372678 learned_addr learned my addr 172.21.15.112:0/1112372678 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:42:27.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.897+0000 7fd0bbfff640 1 --2- 172.21.15.112:0/1112372678 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd0bc0fe450 0x7fd0bc0fbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:27.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.897+0000 7fd0c0d1b640 1 -- 172.21.15.112:0/1112372678 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd0bc0fe450 msgr2=0x7fd0bc0fbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:27.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.897+0000 7fd0c0d1b640 1 --2- 172.21.15.112:0/1112372678 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd0bc0fe450 0x7fd0bc0fbaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:27.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.897+0000 7fd0c0d1b640 1 -- 172.21.15.112:0/1112372678 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0b00098b0 con 0x7fd0bc064fe0 2024-09-16T00:42:27.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.898+0000 7fd0c0d1b640 1 --2- 172.21.15.112:0/1112372678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0bc064fe0 0x7fd0bc0fb560 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac002a50 tx=0x7fd0ac004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:27.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.898+0000 7fd0b97fa640 1 -- 172.21.15.112:0/1112372678 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd0ac007c30 con 0x7fd0bc064fe0 2024-09-16T00:42:27.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.898+0000 7fd0b97fa640 1 -- 172.21.15.112:0/1112372678 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fd0ac00f040 con 0x7fd0bc064fe0 2024-09-16T00:42:27.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.898+0000 7fd0b97fa640 1 -- 172.21.15.112:0/1112372678 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd0ac00da90 con 0x7fd0bc064fe0 2024-09-16T00:42:27.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.898+0000 7fd0c2f7f640 1 -- 172.21.15.112:0/1112372678 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0bc0fc3e0 con 0x7fd0bc064fe0 2024-09-16T00:42:27.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.898+0000 7fd0c2f7f640 1 -- 172.21.15.112:0/1112372678 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0bc1198a0 con 0x7fd0bc064fe0 2024-09-16T00:42:27.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.899+0000 7fd0c2f7f640 1 -- 172.21.15.112:0/1112372678 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0bc0fdce0 con 0x7fd0bc064fe0 2024-09-16T00:42:27.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.899+0000 7fd0b97fa640 1 -- 172.21.15.112:0/1112372678 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fd0ac00dc30 con 0x7fd0bc064fe0 2024-09-16T00:42:27.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.900+0000 7fd0b97fa640 1 --2- 172.21.15.112:0/1112372678 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd0a406e530 0x7fd0a40709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:27.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.900+0000 7fd0bbfff640 1 --2- 172.21.15.112:0/1112372678 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd0a406e530 0x7fd0a40709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:27.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.900+0000 7fd0b97fa640 1 -- 172.21.15.112:0/1112372678 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7fd0ac08da90 con 0x7fd0bc064fe0 2024-09-16T00:42:27.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.900+0000 7fd0bbfff640 1 --2- 172.21.15.112:0/1112372678 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd0a406e530 0x7fd0a40709e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0009c20 tx=0x7fd0b0004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:27.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:27.905+0000 7fd0b97fa640 1 -- 172.21.15.112:0/1112372678 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fd0ac056b30 con 0x7fd0bc064fe0 2024-09-16T00:42:28.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:42:27 smithi169 ceph-mon[77626]: pgmap v441: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:42:28.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.135+0000 7fd0c2f7f640 1 -- 172.21.15.112:0/1112372678 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd0bc0fcc50 con 0x7fd0bc064fe0 2024-09-16T00:42:28.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.135+0000 7fd0b97fa640 1 -- 172.21.15.112:0/1112372678 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7fd0ac05a1f0 con 0x7fd0bc064fe0 2024-09-16T00:42:28.135 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:42:28.135 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": { 2024-09-16T00:42:28.135 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:42:28.136 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:42:28.136 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": { 2024-09-16T00:42:28.136 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:42:28.136 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:42:28.136 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": { 2024-09-16T00:42:28.136 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 6 2024-09-16T00:42:28.136 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:42:28.136 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": { 2024-09-16T00:42:28.136 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 4 2024-09-16T00:42:28.136 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:42:28.136 INFO:teuthology.orchestra.run.smithi112.stdout: "overall": { 2024-09-16T00:42:28.136 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 14 2024-09-16T00:42:28.136 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-16T00:42:28.136 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:42:28.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.137+0000 7fd0c2f7f640 1 -- 172.21.15.112:0/1112372678 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd0a406e530 msgr2=0x7fd0a40709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:28.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.137+0000 7fd0c2f7f640 1 --2- 172.21.15.112:0/1112372678 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd0a406e530 0x7fd0a40709e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0009c20 tx=0x7fd0b0004d10 comp rx=0 tx=0).stop 2024-09-16T00:42:28.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.137+0000 7fd0c2f7f640 1 -- 172.21.15.112:0/1112372678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0bc064fe0 msgr2=0x7fd0bc0fb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:28.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.137+0000 7fd0c2f7f640 1 --2- 172.21.15.112:0/1112372678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0bc064fe0 0x7fd0bc0fb560 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac002a50 tx=0x7fd0ac004600 comp rx=0 tx=0).stop 2024-09-16T00:42:28.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.137+0000 7fd0c2f7f640 1 -- 172.21.15.112:0/1112372678 shutdown_connections 2024-09-16T00:42:28.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.137+0000 7fd0c2f7f640 1 --2- 172.21.15.112:0/1112372678 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd0a406e530 0x7fd0a40709e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:28.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.137+0000 7fd0c2f7f640 1 --2- 172.21.15.112:0/1112372678 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd0bc0fe450 0x7fd0bc0fbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:28.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.137+0000 7fd0c2f7f640 1 --2- 172.21.15.112:0/1112372678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0bc064fe0 0x7fd0bc0fb560 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:28.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.137+0000 7fd0c2f7f640 1 -- 172.21.15.112:0/1112372678 >> 172.21.15.112:0/1112372678 conn(0x7fd0bc0723b0 msgr2=0x7fd0bc105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:28.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.137+0000 7fd0c2f7f640 1 -- 172.21.15.112:0/1112372678 shutdown_connections 2024-09-16T00:42:28.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.137+0000 7fd0c2f7f640 1 -- 172.21.15.112:0/1112372678 wait complete. 2024-09-16T00:42:28.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.215+0000 7fd69b136640 1 -- 172.21.15.112:0/1952579331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6940fb810 msgr2=0x7fd6940fbbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:28.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.215+0000 7fd69b136640 1 --2- 172.21.15.112:0/1952579331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6940fb810 0x7fd6940fbbe0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fd6880048d0 tx=0x7fd688009da0 comp rx=0 tx=0).stop 2024-09-16T00:42:28.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.216+0000 7fd69b136640 1 -- 172.21.15.112:0/1952579331 shutdown_connections 2024-09-16T00:42:28.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.216+0000 7fd69b136640 1 --2- 172.21.15.112:0/1952579331 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6940fcf80 0x7fd6940fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:28.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.216+0000 7fd69b136640 1 --2- 172.21.15.112:0/1952579331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6940fb810 0x7fd6940fbbe0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:28.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.216+0000 7fd69b136640 1 -- 172.21.15.112:0/1952579331 >> 172.21.15.112:0/1952579331 conn(0x7fd6940723b0 msgr2=0x7fd6940727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:28.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.216+0000 7fd69b136640 1 -- 172.21.15.112:0/1952579331 shutdown_connections 2024-09-16T00:42:28.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.216+0000 7fd69b136640 1 -- 172.21.15.112:0/1952579331 wait complete. 2024-09-16T00:42:28.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.216+0000 7fd69b136640 1 Processor -- start 2024-09-16T00:42:28.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.216+0000 7fd69b136640 1 -- start start 2024-09-16T00:42:28.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.216+0000 7fd69b136640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6940fb810 0x7fd69410bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:28.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.216+0000 7fd69b136640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6940fcf80 0x7fd69410c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:28.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.217+0000 7fd69b136640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd69410c9e0 con 0x7fd6940fcf80 2024-09-16T00:42:28.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.217+0000 7fd69b136640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd69410cb20 con 0x7fd6940fb810 2024-09-16T00:42:28.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.217+0000 7fd693fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6940fcf80 0x7fd69410c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:28.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.217+0000 7fd693fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6940fcf80 0x7fd69410c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47332/0 (socket says 172.21.15.112:47332) 2024-09-16T00:42:28.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.217+0000 7fd693fff640 1 -- 172.21.15.112:0/4247691513 learned_addr learned my addr 172.21.15.112:0/4247691513 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:42:28.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.217+0000 7fd698ed2640 1 --2- 172.21.15.112:0/4247691513 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6940fb810 0x7fd69410bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:28.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.217+0000 7fd693fff640 1 -- 172.21.15.112:0/4247691513 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6940fb810 msgr2=0x7fd69410bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:28.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.217+0000 7fd693fff640 1 --2- 172.21.15.112:0/4247691513 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6940fb810 0x7fd69410bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:28.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.217+0000 7fd693fff640 1 -- 172.21.15.112:0/4247691513 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd688005040 con 0x7fd6940fcf80 2024-09-16T00:42:28.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.217+0000 7fd693fff640 1 --2- 172.21.15.112:0/4247691513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6940fcf80 0x7fd69410c4a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd68000c810 tx=0x7fd6800043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:28.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.217+0000 7fd6917fa640 1 -- 172.21.15.112:0/4247691513 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd68000a7a0 con 0x7fd6940fcf80 2024-09-16T00:42:28.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.217+0000 7fd6917fa640 1 -- 172.21.15.112:0/4247691513 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fd68000ada0 con 0x7fd6940fcf80 2024-09-16T00:42:28.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.217+0000 7fd69b136640 1 -- 172.21.15.112:0/4247691513 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6941195d0 con 0x7fd6940fcf80 2024-09-16T00:42:28.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.217+0000 7fd6917fa640 1 -- 172.21.15.112:0/4247691513 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd680011380 con 0x7fd6940fcf80 2024-09-16T00:42:28.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.217+0000 7fd69b136640 1 -- 172.21.15.112:0/4247691513 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd694119980 con 0x7fd6940fcf80 2024-09-16T00:42:28.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.219+0000 7fd69b136640 1 -- 172.21.15.112:0/4247691513 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6940fcbc0 con 0x7fd6940fcf80 2024-09-16T00:42:28.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.219+0000 7fd6917fa640 1 -- 172.21.15.112:0/4247691513 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fd6800127d0 con 0x7fd6940fcf80 2024-09-16T00:42:28.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.220+0000 7fd6917fa640 1 --2- 172.21.15.112:0/4247691513 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd68406e530 0x7fd6840709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:28.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.220+0000 7fd6917fa640 1 -- 172.21.15.112:0/4247691513 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7fd68008e6e0 con 0x7fd6940fcf80 2024-09-16T00:42:28.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.220+0000 7fd698ed2640 1 --2- 172.21.15.112:0/4247691513 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd68406e530 0x7fd6840709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:28.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.220+0000 7fd698ed2640 1 --2- 172.21.15.112:0/4247691513 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd68406e530 0x7fd6840709e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd688009ac0 tx=0x7fd688035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:28.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.224+0000 7fd6917fa640 1 -- 172.21.15.112:0/4247691513 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fd68005b200 con 0x7fd6940fcf80 2024-09-16T00:42:28.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.423+0000 7fd69b136640 1 -- 172.21.15.112:0/4247691513 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd69404b720 con 0x7fd6940fcf80 2024-09-16T00:42:28.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.424+0000 7fd6917fa640 1 -- 172.21.15.112:0/4247691513 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1914 (secure 0 0 0) 0x7fd68005ad90 con 0x7fd6940fcf80 2024-09-16T00:42:28.424 INFO:teuthology.orchestra.run.smithi112.stdout:e34 2024-09-16T00:42:28.424 INFO:teuthology.orchestra.run.smithi112.stdout:btime 2024-09-16T00:34:26:619254+0000 2024-09-16T00:42:28.424 INFO:teuthology.orchestra.run.smithi112.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T00:42:28.424 INFO:teuthology.orchestra.run.smithi112.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:42:28.425 INFO:teuthology.orchestra.run.smithi112.stdout:legacy client fscid: 1 2024-09-16T00:42:28.425 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:42:28.425 INFO:teuthology.orchestra.run.smithi112.stdout:Filesystem 'cephfs' (1) 2024-09-16T00:42:28.425 INFO:teuthology.orchestra.run.smithi112.stdout:fs_name cephfs 2024-09-16T00:42:28.425 INFO:teuthology.orchestra.run.smithi112.stdout:epoch 25 2024-09-16T00:42:28.425 INFO:teuthology.orchestra.run.smithi112.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-16T00:42:28.425 INFO:teuthology.orchestra.run.smithi112.stdout:created 2024-09-16T00:23:22.328199+0000 2024-09-16T00:42:28.425 INFO:teuthology.orchestra.run.smithi112.stdout:modified 2024-09-16T00:33:29.012991+0000 2024-09-16T00:42:28.425 INFO:teuthology.orchestra.run.smithi112.stdout:tableserver 0 2024-09-16T00:42:28.425 INFO:teuthology.orchestra.run.smithi112.stdout:root 0 2024-09-16T00:42:28.425 INFO:teuthology.orchestra.run.smithi112.stdout:session_timeout 60 2024-09-16T00:42:28.425 INFO:teuthology.orchestra.run.smithi112.stdout:session_autoclose 300 2024-09-16T00:42:28.426 INFO:teuthology.orchestra.run.smithi112.stdout:max_file_size 1099511627776 2024-09-16T00:42:28.426 INFO:teuthology.orchestra.run.smithi112.stdout:max_xattr_size 65536 2024-09-16T00:42:28.426 INFO:teuthology.orchestra.run.smithi112.stdout:required_client_features {} 2024-09-16T00:42:28.426 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure 0 2024-09-16T00:42:28.426 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure_osd_epoch 82 2024-09-16T00:42:28.426 INFO:teuthology.orchestra.run.smithi112.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:42:28.426 INFO:teuthology.orchestra.run.smithi112.stdout:max_mds 2 2024-09-16T00:42:28.426 INFO:teuthology.orchestra.run.smithi112.stdout:in 0,1 2024-09-16T00:42:28.426 INFO:teuthology.orchestra.run.smithi112.stdout:up {} 2024-09-16T00:42:28.426 INFO:teuthology.orchestra.run.smithi112.stdout:failed 0,1 2024-09-16T00:42:28.426 INFO:teuthology.orchestra.run.smithi112.stdout:damaged 2024-09-16T00:42:28.427 INFO:teuthology.orchestra.run.smithi112.stdout:stopped 2024-09-16T00:42:28.427 INFO:teuthology.orchestra.run.smithi112.stdout:data_pools [3] 2024-09-16T00:42:28.427 INFO:teuthology.orchestra.run.smithi112.stdout:metadata_pool 2 2024-09-16T00:42:28.427 INFO:teuthology.orchestra.run.smithi112.stdout:inline_data enabled 2024-09-16T00:42:28.427 INFO:teuthology.orchestra.run.smithi112.stdout:balancer 2024-09-16T00:42:28.427 INFO:teuthology.orchestra.run.smithi112.stdout:bal_rank_mask -1 2024-09-16T00:42:28.427 INFO:teuthology.orchestra.run.smithi112.stdout:standby_count_wanted 1 2024-09-16T00:42:28.427 INFO:teuthology.orchestra.run.smithi112.stdout:qdb_cluster leader: 0 members: 2024-09-16T00:42:28.427 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:42:28.427 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:42:28.427 INFO:teuthology.orchestra.run.smithi112.stdout:Standby daemons: 2024-09-16T00:42:28.427 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:42:28.428 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.ithwzj{-1:34392} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.112:6826/476229221,v1:172.21.15.112:6827/476229221] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:42:28.428 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.hxfwkh{-1:34404} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.112:6828/2356899876,v1:172.21.15.112:6829/2356899876] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:42:28.428 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.gawwgw{-1:44343} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.169:6824/1635329957,v1:172.21.15.169:6825/1635329957] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:42:28.428 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.lgeqrj{-1:44347} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.169:6826/184810031,v1:172.21.15.169:6827/184810031] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:42:28.429 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 34 2024-09-16T00:42:28.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.426+0000 7fd69b136640 1 -- 172.21.15.112:0/4247691513 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd68406e530 msgr2=0x7fd6840709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:28.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.426+0000 7fd69b136640 1 --2- 172.21.15.112:0/4247691513 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd68406e530 0x7fd6840709e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd688009ac0 tx=0x7fd688035040 comp rx=0 tx=0).stop 2024-09-16T00:42:28.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.426+0000 7fd69b136640 1 -- 172.21.15.112:0/4247691513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6940fcf80 msgr2=0x7fd69410c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:28.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.426+0000 7fd69b136640 1 --2- 172.21.15.112:0/4247691513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6940fcf80 0x7fd69410c4a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd68000c810 tx=0x7fd6800043c0 comp rx=0 tx=0).stop 2024-09-16T00:42:28.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.426+0000 7fd69b136640 1 -- 172.21.15.112:0/4247691513 shutdown_connections 2024-09-16T00:42:28.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.426+0000 7fd69b136640 1 --2- 172.21.15.112:0/4247691513 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd68406e530 0x7fd6840709e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:28.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.426+0000 7fd69b136640 1 --2- 172.21.15.112:0/4247691513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6940fcf80 0x7fd69410c4a0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:28.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.426+0000 7fd69b136640 1 --2- 172.21.15.112:0/4247691513 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6940fb810 0x7fd69410bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:28.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.426+0000 7fd69b136640 1 -- 172.21.15.112:0/4247691513 >> 172.21.15.112:0/4247691513 conn(0x7fd6940723b0 msgr2=0x7fd6940fa480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:28.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.426+0000 7fd69b136640 1 -- 172.21.15.112:0/4247691513 shutdown_connections 2024-09-16T00:42:28.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.426+0000 7fd69b136640 1 -- 172.21.15.112:0/4247691513 wait complete. 2024-09-16T00:42:28.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.506+0000 7f55b6be3640 1 -- 172.21.15.112:0/260447508 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55b00fb480 msgr2=0x7f55b00fb8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:28.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.506+0000 7f55b6be3640 1 --2- 172.21.15.112:0/260447508 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55b00fb480 0x7f55b00fb8f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f559c0048d0 tx=0x7f559c009da0 comp rx=0 tx=0).stop 2024-09-16T00:42:28.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.506+0000 7f55b6be3640 1 -- 172.21.15.112:0/260447508 shutdown_connections 2024-09-16T00:42:28.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.506+0000 7f55b6be3640 1 --2- 172.21.15.112:0/260447508 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55b00fb480 0x7f55b00fb8f0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:28.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.506+0000 7f55b6be3640 1 --2- 172.21.15.112:0/260447508 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55b01014c0 0x7f55b0101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:28.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.506+0000 7f55b6be3640 1 -- 172.21.15.112:0/260447508 >> 172.21.15.112:0/260447508 conn(0x7f55b00f7000 msgr2=0x7f55b00f9410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:28.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.506+0000 7f55b6be3640 1 -- 172.21.15.112:0/260447508 shutdown_connections 2024-09-16T00:42:28.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.506+0000 7f55b6be3640 1 -- 172.21.15.112:0/260447508 wait complete. 2024-09-16T00:42:28.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.507+0000 7f55b6be3640 1 Processor -- start 2024-09-16T00:42:28.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.507+0000 7f55b6be3640 1 -- start start 2024-09-16T00:42:28.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.507+0000 7f55b6be3640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55b00fb480 0x7f55b01131b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:28.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.507+0000 7f55b6be3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55b01014c0 0x7f55b01136f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:28.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.507+0000 7f55b6be3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55b0113cf0 con 0x7f55b01014c0 2024-09-16T00:42:28.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.507+0000 7f55b6be3640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55b010d130 con 0x7f55b00fb480 2024-09-16T00:42:28.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.507+0000 7f55affff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55b01014c0 0x7f55b01136f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:28.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.507+0000 7f55affff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55b01014c0 0x7f55b01136f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47346/0 (socket says 172.21.15.112:47346) 2024-09-16T00:42:28.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.507+0000 7f55affff640 1 -- 172.21.15.112:0/3620706505 learned_addr learned my addr 172.21.15.112:0/3620706505 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:42:28.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.508+0000 7f55b497f640 1 --2- 172.21.15.112:0/3620706505 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55b00fb480 0x7f55b01131b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:28.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.508+0000 7f55affff640 1 -- 172.21.15.112:0/3620706505 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55b00fb480 msgr2=0x7f55b01131b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:28.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.508+0000 7f55affff640 1 --2- 172.21.15.112:0/3620706505 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55b00fb480 0x7f55b01131b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:28.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.508+0000 7f55affff640 1 -- 172.21.15.112:0/3620706505 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f559c005040 con 0x7f55b01014c0 2024-09-16T00:42:28.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.508+0000 7f55b497f640 1 --2- 172.21.15.112:0/3620706505 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55b00fb480 0x7f55b01131b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:42:28.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.508+0000 7f55affff640 1 --2- 172.21.15.112:0/3620706505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55b01014c0 0x7f55b01136f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f559c003080 tx=0x7f559c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:28.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.508+0000 7f55ad7fa640 1 -- 172.21.15.112:0/3620706505 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f559c038070 con 0x7f55b01014c0 2024-09-16T00:42:28.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.508+0000 7f55ad7fa640 1 -- 172.21.15.112:0/3620706505 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f559c02ade0 con 0x7f55b01014c0 2024-09-16T00:42:28.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.508+0000 7f55b6be3640 1 -- 172.21.15.112:0/3620706505 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55b010d3b0 con 0x7f55b01014c0 2024-09-16T00:42:28.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.508+0000 7f55ad7fa640 1 -- 172.21.15.112:0/3620706505 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f559c02c880 con 0x7f55b01014c0 2024-09-16T00:42:28.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.508+0000 7f55b6be3640 1 -- 172.21.15.112:0/3620706505 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55b010d890 con 0x7f55b01014c0 2024-09-16T00:42:28.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.509+0000 7f55b6be3640 1 -- 172.21.15.112:0/3620706505 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55b00fcbc0 con 0x7f55b01014c0 2024-09-16T00:42:28.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.510+0000 7f55ad7fa640 1 -- 172.21.15.112:0/3620706505 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f559c030020 con 0x7f55b01014c0 2024-09-16T00:42:28.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.510+0000 7f55ad7fa640 1 --2- 172.21.15.112:0/3620706505 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f55a006e530 0x7f55a00709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:28.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.510+0000 7f55ad7fa640 1 -- 172.21.15.112:0/3620706505 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f559c0ae380 con 0x7f55b01014c0 2024-09-16T00:42:28.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.511+0000 7f55b497f640 1 --2- 172.21.15.112:0/3620706505 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f55a006e530 0x7f55a00709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:28.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.511+0000 7f55b497f640 1 --2- 172.21.15.112:0/3620706505 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f55a006e530 0x7f55a00709e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f55a4004c80 tx=0x7f55a4009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:28.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.515+0000 7f55ad7fa640 1 -- 172.21.15.112:0/3620706505 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f559c0772f0 con 0x7f55b01014c0 2024-09-16T00:42:28.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.697+0000 7f55b6be3640 1 -- 172.21.15.112:0/3620706505 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f55b0108470 con 0x7f55a006e530 2024-09-16T00:42:28.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.698+0000 7f55ad7fa640 1 -- 172.21.15.112:0/3620706505 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f55b0108470 con 0x7f55a006e530 2024-09-16T00:42:28.698 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:42:28.698 INFO:teuthology.orchestra.run.smithi112.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80", 2024-09-16T00:42:28.698 INFO:teuthology.orchestra.run.smithi112.stdout: "in_progress": true, 2024-09-16T00:42:28.698 INFO:teuthology.orchestra.run.smithi112.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T00:42:28.698 INFO:teuthology.orchestra.run.smithi112.stdout: "services_complete": [ 2024-09-16T00:42:28.698 INFO:teuthology.orchestra.run.smithi112.stdout: "mds", 2024-09-16T00:42:28.698 INFO:teuthology.orchestra.run.smithi112.stdout: "osd", 2024-09-16T00:42:28.699 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr", 2024-09-16T00:42:28.699 INFO:teuthology.orchestra.run.smithi112.stdout: "mon", 2024-09-16T00:42:28.699 INFO:teuthology.orchestra.run.smithi112.stdout: "crash" 2024-09-16T00:42:28.699 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-16T00:42:28.699 INFO:teuthology.orchestra.run.smithi112.stdout: "progress": "16/21 daemons upgraded", 2024-09-16T00:42:28.699 INFO:teuthology.orchestra.run.smithi112.stdout: "message": "Currently upgrading mds daemons", 2024-09-16T00:42:28.699 INFO:teuthology.orchestra.run.smithi112.stdout: "is_paused": false 2024-09-16T00:42:28.699 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:42:28.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.700+0000 7f55b6be3640 1 -- 172.21.15.112:0/3620706505 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f55a006e530 msgr2=0x7f55a00709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:28.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.700+0000 7f55b6be3640 1 --2- 172.21.15.112:0/3620706505 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f55a006e530 0x7f55a00709e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f55a4004c80 tx=0x7f55a4009040 comp rx=0 tx=0).stop 2024-09-16T00:42:28.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.700+0000 7f55b6be3640 1 -- 172.21.15.112:0/3620706505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55b01014c0 msgr2=0x7f55b01136f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:28.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.700+0000 7f55b6be3640 1 --2- 172.21.15.112:0/3620706505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55b01014c0 0x7f55b01136f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f559c003080 tx=0x7f559c009da0 comp rx=0 tx=0).stop 2024-09-16T00:42:28.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.700+0000 7f55b6be3640 1 -- 172.21.15.112:0/3620706505 shutdown_connections 2024-09-16T00:42:28.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.700+0000 7f55b6be3640 1 --2- 172.21.15.112:0/3620706505 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f55a006e530 0x7f55a00709e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:28.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.700+0000 7f55b6be3640 1 --2- 172.21.15.112:0/3620706505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55b01014c0 0x7f55b01136f0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:28.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.700+0000 7f55b6be3640 1 --2- 172.21.15.112:0/3620706505 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55b00fb480 0x7f55b01131b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:28.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.700+0000 7f55b6be3640 1 -- 172.21.15.112:0/3620706505 >> 172.21.15.112:0/3620706505 conn(0x7f55b00f7000 msgr2=0x7f55b0065a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:28.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.700+0000 7f55b6be3640 1 -- 172.21.15.112:0/3620706505 shutdown_connections 2024-09-16T00:42:28.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.700+0000 7f55b6be3640 1 -- 172.21.15.112:0/3620706505 wait complete. 2024-09-16T00:42:28.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.780+0000 7f0ce8687640 1 -- 172.21.15.112:0/1353185290 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ce00fb480 msgr2=0x7f0ce00fb8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:28.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.780+0000 7f0ce8687640 1 --2- 172.21.15.112:0/1353185290 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ce00fb480 0x7f0ce00fb8f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f0cd00048d0 tx=0x7f0cd0009da0 comp rx=0 tx=0).stop 2024-09-16T00:42:28.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.780+0000 7f0ce8687640 1 -- 172.21.15.112:0/1353185290 shutdown_connections 2024-09-16T00:42:28.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.780+0000 7f0ce8687640 1 --2- 172.21.15.112:0/1353185290 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ce00fb480 0x7f0ce00fb8f0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:28.781 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:42:28 smithi112 ceph-mon[97750]: from='client.34856 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:42:28.781 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:42:28 smithi112 ceph-mon[97750]: from='client.34860 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:42:28.781 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:42:28 smithi112 ceph-mon[97750]: from='client.34864 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:42:28.781 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:42:28 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/1112372678' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:42:28.781 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:42:28 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/4247691513' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:42:28.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.780+0000 7f0ce8687640 1 --2- 172.21.15.112:0/1353185290 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ce01014c0 0x7f0ce0101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:28.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.780+0000 7f0ce8687640 1 -- 172.21.15.112:0/1353185290 >> 172.21.15.112:0/1353185290 conn(0x7f0ce00f7000 msgr2=0x7f0ce00f9410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:28.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.780+0000 7f0ce8687640 1 -- 172.21.15.112:0/1353185290 shutdown_connections 2024-09-16T00:42:28.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.780+0000 7f0ce8687640 1 -- 172.21.15.112:0/1353185290 wait complete. 2024-09-16T00:42:28.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.781+0000 7f0ce8687640 1 Processor -- start 2024-09-16T00:42:28.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.781+0000 7f0ce8687640 1 -- start start 2024-09-16T00:42:28.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.781+0000 7f0ce8687640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ce00fb480 0x7f0ce010e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:28.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.781+0000 7f0ce8687640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ce01014c0 0x7f0ce010e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:28.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.781+0000 7f0ce8687640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce010ecf0 con 0x7f0ce00fb480 2024-09-16T00:42:28.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.781+0000 7f0ce8687640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce010ee60 con 0x7f0ce01014c0 2024-09-16T00:42:28.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.781+0000 7f0ce6423640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ce00fb480 0x7f0ce010e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:28.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.781+0000 7f0ce6423640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ce00fb480 0x7f0ce010e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47370/0 (socket says 172.21.15.112:47370) 2024-09-16T00:42:28.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.781+0000 7f0ce6423640 1 -- 172.21.15.112:0/1276787665 learned_addr learned my addr 172.21.15.112:0/1276787665 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:42:28.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.782+0000 7f0ce5c22640 1 --2- 172.21.15.112:0/1276787665 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ce01014c0 0x7f0ce010e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:28.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.782+0000 7f0ce6423640 1 -- 172.21.15.112:0/1276787665 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ce01014c0 msgr2=0x7f0ce010e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:28.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.782+0000 7f0ce6423640 1 --2- 172.21.15.112:0/1276787665 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ce01014c0 0x7f0ce010e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:28.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.782+0000 7f0ce6423640 1 -- 172.21.15.112:0/1276787665 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cd0005040 con 0x7f0ce00fb480 2024-09-16T00:42:28.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.782+0000 7f0ce6423640 1 --2- 172.21.15.112:0/1276787665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ce00fb480 0x7f0ce010e160 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc00c810 tx=0x7f0cdc0043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:28.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.782+0000 7f0cd6ffd640 1 -- 172.21.15.112:0/1276787665 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0cdc00a7a0 con 0x7f0ce00fb480 2024-09-16T00:42:28.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.782+0000 7f0cd6ffd640 1 -- 172.21.15.112:0/1276787665 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f0cdc00ada0 con 0x7f0ce00fb480 2024-09-16T00:42:28.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.782+0000 7f0cd6ffd640 1 -- 172.21.15.112:0/1276787665 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0cdc011380 con 0x7f0ce00fb480 2024-09-16T00:42:28.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.782+0000 7f0ce8687640 1 -- 172.21.15.112:0/1276787665 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ce01171d0 con 0x7f0ce00fb480 2024-09-16T00:42:28.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.782+0000 7f0ce8687640 1 -- 172.21.15.112:0/1276787665 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ce0117690 con 0x7f0ce00fb480 2024-09-16T00:42:28.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.784+0000 7f0ce8687640 1 -- 172.21.15.112:0/1276787665 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ce00fcbc0 con 0x7f0ce00fb480 2024-09-16T00:42:28.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.785+0000 7f0cd6ffd640 1 -- 172.21.15.112:0/1276787665 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f0cdc0115f0 con 0x7f0ce00fb480 2024-09-16T00:42:28.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.785+0000 7f0cd6ffd640 1 --2- 172.21.15.112:0/1276787665 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f0ccc06e530 0x7f0ccc0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:28.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.785+0000 7f0cd6ffd640 1 -- 172.21.15.112:0/1276787665 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f0cdc08e750 con 0x7f0ce00fb480 2024-09-16T00:42:28.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.785+0000 7f0ce5c22640 1 --2- 172.21.15.112:0/1276787665 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f0ccc06e530 0x7f0ccc0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:28.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.786+0000 7f0ce5c22640 1 --2- 172.21.15.112:0/1276787665 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f0ccc06e530 0x7f0ccc0709e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0009ac0 tx=0x7f0cd0035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:28.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:28.789+0000 7f0cd6ffd640 1 -- 172.21.15.112:0/1276787665 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f0cdc057810 con 0x7f0ce00fb480 2024-09-16T00:42:29.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:42:28 smithi169 ceph-mon[77626]: from='client.34856 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:42:29.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:42:28 smithi169 ceph-mon[77626]: from='client.34860 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:42:29.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:42:28 smithi169 ceph-mon[77626]: from='client.34864 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:42:29.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:42:28 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/1112372678' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:42:29.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:42:28 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/4247691513' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:42:29.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:29.028+0000 7f0ce8687640 1 -- 172.21.15.112:0/1276787665 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f0ce00fac40 con 0x7f0ce00fb480 2024-09-16T00:42:29.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:29.028+0000 7f0cd6ffd640 1 -- 172.21.15.112:0/1276787665 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f0cdc05aed0 con 0x7f0ce00fb480 2024-09-16T00:42:29.028 INFO:teuthology.orchestra.run.smithi112.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-16T00:42:29.028 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-16T00:42:29.028 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs is degraded 2024-09-16T00:42:29.028 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T00:42:29.028 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T00:42:29.028 INFO:teuthology.orchestra.run.smithi112.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-16T00:42:29.029 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-16T00:42:29.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:29.030+0000 7f0ce8687640 1 -- 172.21.15.112:0/1276787665 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f0ccc06e530 msgr2=0x7f0ccc0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:29.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:29.030+0000 7f0ce8687640 1 --2- 172.21.15.112:0/1276787665 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f0ccc06e530 0x7f0ccc0709e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0009ac0 tx=0x7f0cd0035040 comp rx=0 tx=0).stop 2024-09-16T00:42:29.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:29.030+0000 7f0ce8687640 1 -- 172.21.15.112:0/1276787665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ce00fb480 msgr2=0x7f0ce010e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:29.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:29.030+0000 7f0ce8687640 1 --2- 172.21.15.112:0/1276787665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ce00fb480 0x7f0ce010e160 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc00c810 tx=0x7f0cdc0043c0 comp rx=0 tx=0).stop 2024-09-16T00:42:29.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:29.030+0000 7f0ce8687640 1 -- 172.21.15.112:0/1276787665 shutdown_connections 2024-09-16T00:42:29.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:29.030+0000 7f0ce8687640 1 --2- 172.21.15.112:0/1276787665 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f0ccc06e530 0x7f0ccc0709e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:29.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:29.030+0000 7f0ce8687640 1 --2- 172.21.15.112:0/1276787665 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ce01014c0 0x7f0ce010e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:29.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:29.030+0000 7f0ce8687640 1 --2- 172.21.15.112:0/1276787665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ce00fb480 0x7f0ce010e160 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:29.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:29.030+0000 7f0ce8687640 1 -- 172.21.15.112:0/1276787665 >> 172.21.15.112:0/1276787665 conn(0x7f0ce00f7000 msgr2=0x7f0ce00f88d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:29.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:29.030+0000 7f0ce8687640 1 -- 172.21.15.112:0/1276787665 shutdown_connections 2024-09-16T00:42:29.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:29.030+0000 7f0ce8687640 1 -- 172.21.15.112:0/1276787665 wait complete. 2024-09-16T00:42:30.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:42:29 smithi169 ceph-mon[77626]: pgmap v442: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:42:30.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:42:29 smithi169 ceph-mon[77626]: from='client.34876 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:42:30.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:42:29 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/1276787665' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:42:30.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:42:29 smithi112 ceph-mon[97750]: pgmap v442: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:42:30.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:42:29 smithi112 ceph-mon[97750]: from='client.34876 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:42:30.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:42:29 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/1276787665' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:42:32.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:42:31 smithi169 ceph-mon[77626]: pgmap v443: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:42:32.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:42:31 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:42:32.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:42:31 smithi112 ceph-mon[97750]: pgmap v443: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:42:32.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:42:31 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:42:34.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:42:33 smithi169 ceph-mon[77626]: pgmap v444: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:42:34.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:42:33 smithi112 ceph-mon[97750]: pgmap v444: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:42:36.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:42:35 smithi169 ceph-mon[77626]: pgmap v445: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:42:36.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:42:35 smithi112 ceph-mon[97750]: pgmap v445: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:42:38.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:42:37 smithi169 ceph-mon[77626]: pgmap v446: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:42:38.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:42:37 smithi112 ceph-mon[97750]: pgmap v446: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:42:40.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:42:39 smithi169 ceph-mon[77626]: pgmap v447: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:42:40.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:42:39 smithi112 ceph-mon[97750]: pgmap v447: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:42:42.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:42:41 smithi169 ceph-mon[77626]: pgmap v448: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:42:42.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:42:41 smithi112 ceph-mon[97750]: pgmap v448: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:42:44.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:42:43 smithi169 ceph-mon[77626]: pgmap v449: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:42:44.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:42:43 smithi112 ceph-mon[97750]: pgmap v449: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:42:46.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:42:45 smithi169 ceph-mon[77626]: pgmap v450: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:42:46.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:42:45 smithi112 ceph-mon[97750]: pgmap v450: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:42:47.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:42:46 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:42:47.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:42:46 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:42:48.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:42:47 smithi112 ceph-mon[97750]: pgmap v451: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:42:48.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:42:47 smithi169 ceph-mon[77626]: pgmap v451: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:42:50.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:42:49 smithi112 ceph-mon[97750]: pgmap v452: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:42:50.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:42:49 smithi169 ceph-mon[77626]: pgmap v452: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:42:52.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:42:51 smithi112 ceph-mon[97750]: pgmap v453: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:42:52.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:42:51 smithi169 ceph-mon[77626]: pgmap v453: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:42:54.078 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:42:53 smithi169 ceph-mon[77626]: pgmap v454: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:42:54.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:42:53 smithi112 ceph-mon[97750]: pgmap v454: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:42:56.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:42:55 smithi112 ceph-mon[97750]: pgmap v455: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:42:56.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:42:55 smithi169 ceph-mon[77626]: pgmap v455: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:42:58.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:42:57 smithi112 ceph-mon[97750]: pgmap v456: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:42:58.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:42:57 smithi169 ceph-mon[77626]: pgmap v456: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:42:59.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.121+0000 7f55ba0eb640 1 -- 172.21.15.112:0/583436679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55b4102530 msgr2=0x7f55b4102900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:59.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.121+0000 7f55ba0eb640 1 --2- 172.21.15.112:0/583436679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55b4102530 0x7f55b4102900 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f55a40048d0 tx=0x7f55a4009da0 comp rx=0 tx=0).stop 2024-09-16T00:42:59.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.121+0000 7f55ba0eb640 1 -- 172.21.15.112:0/583436679 shutdown_connections 2024-09-16T00:42:59.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.121+0000 7f55ba0eb640 1 --2- 172.21.15.112:0/583436679 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55b4102ed0 0x7f55b40fc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:59.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.121+0000 7f55ba0eb640 1 --2- 172.21.15.112:0/583436679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55b4102530 0x7f55b4102900 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:59.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.121+0000 7f55ba0eb640 1 -- 172.21.15.112:0/583436679 >> 172.21.15.112:0/583436679 conn(0x7f55b40f8110 msgr2=0x7f55b40fa520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:59.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.121+0000 7f55ba0eb640 1 -- 172.21.15.112:0/583436679 shutdown_connections 2024-09-16T00:42:59.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.121+0000 7f55ba0eb640 1 -- 172.21.15.112:0/583436679 wait complete. 2024-09-16T00:42:59.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.122+0000 7f55ba0eb640 1 Processor -- start 2024-09-16T00:42:59.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.122+0000 7f55ba0eb640 1 -- start start 2024-09-16T00:42:59.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.122+0000 7f55ba0eb640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55b4102530 0x7f55b406fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:59.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.122+0000 7f55ba0eb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55b4102ed0 0x7f55b406ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:59.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.122+0000 7f55ba0eb640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55b4070520 con 0x7f55b4102ed0 2024-09-16T00:42:59.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.122+0000 7f55ba0eb640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55b4070690 con 0x7f55b4102530 2024-09-16T00:42:59.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.122+0000 7f55aaffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55b4102ed0 0x7f55b406ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:59.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.122+0000 7f55aaffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55b4102ed0 0x7f55b406ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53980/0 (socket says 172.21.15.112:53980) 2024-09-16T00:42:59.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.122+0000 7f55aaffd640 1 -- 172.21.15.112:0/2657984895 learned_addr learned my addr 172.21.15.112:0/2657984895 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:42:59.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.123+0000 7f55b37fe640 1 --2- 172.21.15.112:0/2657984895 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55b4102530 0x7f55b406fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:59.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.123+0000 7f55aaffd640 1 -- 172.21.15.112:0/2657984895 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55b4102530 msgr2=0x7f55b406fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:59.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.123+0000 7f55aaffd640 1 --2- 172.21.15.112:0/2657984895 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55b4102530 0x7f55b406fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:59.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.123+0000 7f55aaffd640 1 -- 172.21.15.112:0/2657984895 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55a4005040 con 0x7f55b4102ed0 2024-09-16T00:42:59.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.123+0000 7f55aaffd640 1 --2- 172.21.15.112:0/2657984895 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55b4102ed0 0x7f55b406ff50 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f559c00b540 tx=0x7f559c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:59.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.123+0000 7f55b0ff9640 1 -- 172.21.15.112:0/2657984895 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f559c00f6f0 con 0x7f55b4102ed0 2024-09-16T00:42:59.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.123+0000 7f55b0ff9640 1 -- 172.21.15.112:0/2657984895 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f559c00fcf0 con 0x7f55b4102ed0 2024-09-16T00:42:59.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.123+0000 7f55ba0eb640 1 -- 172.21.15.112:0/2657984895 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55b4119590 con 0x7f55b4102ed0 2024-09-16T00:42:59.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.123+0000 7f55b0ff9640 1 -- 172.21.15.112:0/2657984895 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f559c011e80 con 0x7f55b4102ed0 2024-09-16T00:42:59.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.123+0000 7f55ba0eb640 1 -- 172.21.15.112:0/2657984895 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55b41199d0 con 0x7f55b4102ed0 2024-09-16T00:42:59.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.124+0000 7f55ba0eb640 1 -- 172.21.15.112:0/2657984895 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55b40fdcb0 con 0x7f55b4102ed0 2024-09-16T00:42:59.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.125+0000 7f55b0ff9640 1 -- 172.21.15.112:0/2657984895 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f559c0116b0 con 0x7f55b4102ed0 2024-09-16T00:42:59.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.125+0000 7f55b0ff9640 1 --2- 172.21.15.112:0/2657984895 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f55a006e530 0x7f55a00709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:59.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.125+0000 7f55b0ff9640 1 -- 172.21.15.112:0/2657984895 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f559c08e200 con 0x7f55b4102ed0 2024-09-16T00:42:59.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.125+0000 7f55b37fe640 1 --2- 172.21.15.112:0/2657984895 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f55a006e530 0x7f55a00709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:59.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.126+0000 7f55b37fe640 1 --2- 172.21.15.112:0/2657984895 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f55a006e530 0x7f55a00709e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f55a4000c00 tx=0x7f55a4035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:59.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.130+0000 7f55b0ff9640 1 -- 172.21.15.112:0/2657984895 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f559c0571f0 con 0x7f55b4102ed0 2024-09-16T00:42:59.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.312+0000 7f55ba0eb640 1 -- 172.21.15.112:0/2657984895 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f55b404b720 con 0x7f55a006e530 2024-09-16T00:42:59.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.313+0000 7f55b0ff9640 1 -- 172.21.15.112:0/2657984895 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f55b404b720 con 0x7f55a006e530 2024-09-16T00:42:59.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.315+0000 7f55ba0eb640 1 -- 172.21.15.112:0/2657984895 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f55a006e530 msgr2=0x7f55a00709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:59.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.315+0000 7f55ba0eb640 1 --2- 172.21.15.112:0/2657984895 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f55a006e530 0x7f55a00709e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f55a4000c00 tx=0x7f55a4035040 comp rx=0 tx=0).stop 2024-09-16T00:42:59.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.315+0000 7f55ba0eb640 1 -- 172.21.15.112:0/2657984895 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55b4102ed0 msgr2=0x7f55b406ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:59.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.315+0000 7f55ba0eb640 1 --2- 172.21.15.112:0/2657984895 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55b4102ed0 0x7f55b406ff50 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f559c00b540 tx=0x7f559c0091a0 comp rx=0 tx=0).stop 2024-09-16T00:42:59.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.315+0000 7f55ba0eb640 1 -- 172.21.15.112:0/2657984895 shutdown_connections 2024-09-16T00:42:59.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.315+0000 7f55ba0eb640 1 --2- 172.21.15.112:0/2657984895 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f55a006e530 0x7f55a00709e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:59.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.315+0000 7f55ba0eb640 1 --2- 172.21.15.112:0/2657984895 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55b4102ed0 0x7f55b406ff50 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:59.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.315+0000 7f55ba0eb640 1 --2- 172.21.15.112:0/2657984895 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55b4102530 0x7f55b406fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:59.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.315+0000 7f55ba0eb640 1 -- 172.21.15.112:0/2657984895 >> 172.21.15.112:0/2657984895 conn(0x7f55b40f8110 msgr2=0x7f55b40f91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:59.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.315+0000 7f55ba0eb640 1 -- 172.21.15.112:0/2657984895 shutdown_connections 2024-09-16T00:42:59.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.315+0000 7f55ba0eb640 1 -- 172.21.15.112:0/2657984895 wait complete. 2024-09-16T00:42:59.324 INFO:teuthology.orchestra.run.smithi112.stdout:true 2024-09-16T00:42:59.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.394+0000 7f2dae1dc640 1 -- 172.21.15.112:0/3586115826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2da80fb810 msgr2=0x7f2da80fbbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:59.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.394+0000 7f2dae1dc640 1 --2- 172.21.15.112:0/3586115826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2da80fb810 0x7f2da80fbbe0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f2d980048d0 tx=0x7f2d98009da0 comp rx=0 tx=0).stop 2024-09-16T00:42:59.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.394+0000 7f2dae1dc640 1 -- 172.21.15.112:0/3586115826 shutdown_connections 2024-09-16T00:42:59.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.394+0000 7f2dae1dc640 1 --2- 172.21.15.112:0/3586115826 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2da80fcf80 0x7f2da80fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:59.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.394+0000 7f2dae1dc640 1 --2- 172.21.15.112:0/3586115826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2da80fb810 0x7f2da80fbbe0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:59.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.394+0000 7f2dae1dc640 1 -- 172.21.15.112:0/3586115826 >> 172.21.15.112:0/3586115826 conn(0x7f2da80723b0 msgr2=0x7f2da80727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:59.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.394+0000 7f2dae1dc640 1 -- 172.21.15.112:0/3586115826 shutdown_connections 2024-09-16T00:42:59.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.395+0000 7f2dae1dc640 1 -- 172.21.15.112:0/3586115826 wait complete. 2024-09-16T00:42:59.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.395+0000 7f2dae1dc640 1 Processor -- start 2024-09-16T00:42:59.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.395+0000 7f2dae1dc640 1 -- start start 2024-09-16T00:42:59.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.396+0000 7f2dae1dc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2da80fb810 0x7f2da810e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:59.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.396+0000 7f2dae1dc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2da80fcf80 0x7f2da810e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:59.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.396+0000 7f2dae1dc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da810ed00 con 0x7f2da80fcf80 2024-09-16T00:42:59.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.396+0000 7f2dae1dc640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da810ee70 con 0x7f2da80fb810 2024-09-16T00:42:59.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.396+0000 7f2da6ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2da80fcf80 0x7f2da810e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:59.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.396+0000 7f2da6ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2da80fcf80 0x7f2da810e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53998/0 (socket says 172.21.15.112:53998) 2024-09-16T00:42:59.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.396+0000 7f2da6ffd640 1 -- 172.21.15.112:0/428075107 learned_addr learned my addr 172.21.15.112:0/428075107 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:42:59.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.396+0000 7f2da77fe640 1 --2- 172.21.15.112:0/428075107 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2da80fb810 0x7f2da810e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:59.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.396+0000 7f2da6ffd640 1 -- 172.21.15.112:0/428075107 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2da80fb810 msgr2=0x7f2da810e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:59.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.396+0000 7f2da6ffd640 1 --2- 172.21.15.112:0/428075107 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2da80fb810 0x7f2da810e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:59.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.396+0000 7f2da6ffd640 1 -- 172.21.15.112:0/428075107 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d98005040 con 0x7f2da80fcf80 2024-09-16T00:42:59.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.396+0000 7f2da77fe640 1 --2- 172.21.15.112:0/428075107 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2da80fb810 0x7f2da810e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:42:59.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.396+0000 7f2da6ffd640 1 --2- 172.21.15.112:0/428075107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2da80fcf80 0x7f2da810e6b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c00c830 tx=0x7f2d9c004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:59.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.397+0000 7f2d8ffff640 1 -- 172.21.15.112:0/428075107 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d9c00a850 con 0x7f2da80fcf80 2024-09-16T00:42:59.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.397+0000 7f2d8ffff640 1 -- 172.21.15.112:0/428075107 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f2d9c00ae50 con 0x7f2da80fcf80 2024-09-16T00:42:59.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.397+0000 7f2dae1dc640 1 -- 172.21.15.112:0/428075107 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2da81171e0 con 0x7f2da80fcf80 2024-09-16T00:42:59.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.397+0000 7f2d8ffff640 1 -- 172.21.15.112:0/428075107 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d9c0113a0 con 0x7f2da80fcf80 2024-09-16T00:42:59.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.397+0000 7f2dae1dc640 1 -- 172.21.15.112:0/428075107 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2da81176f0 con 0x7f2da80fcf80 2024-09-16T00:42:59.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.398+0000 7f2dae1dc640 1 -- 172.21.15.112:0/428075107 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2da80fcbc0 con 0x7f2da80fcf80 2024-09-16T00:42:59.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.398+0000 7f2d8ffff640 1 -- 172.21.15.112:0/428075107 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f2d9c00a9b0 con 0x7f2da80fcf80 2024-09-16T00:42:59.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.399+0000 7f2d8ffff640 1 --2- 172.21.15.112:0/428075107 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f2d9006e530 0x7f2d900709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:59.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.399+0000 7f2d8ffff640 1 -- 172.21.15.112:0/428075107 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f2d9c08e3e0 con 0x7f2da80fcf80 2024-09-16T00:42:59.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.399+0000 7f2da77fe640 1 --2- 172.21.15.112:0/428075107 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f2d9006e530 0x7f2d900709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:59.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.399+0000 7f2da77fe640 1 --2- 172.21.15.112:0/428075107 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f2d9006e530 0x7f2d900709e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2d98007ac0 tx=0x7f2d98035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:59.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.405+0000 7f2d8ffff640 1 -- 172.21.15.112:0/428075107 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f2d9c057470 con 0x7f2da80fcf80 2024-09-16T00:42:59.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.584+0000 7f2dae1dc640 1 -- 172.21.15.112:0/428075107 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2da80657d0 con 0x7f2d9006e530 2024-09-16T00:42:59.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.585+0000 7f2d8ffff640 1 -- 172.21.15.112:0/428075107 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f2da80657d0 con 0x7f2d9006e530 2024-09-16T00:42:59.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.587+0000 7f2dae1dc640 1 -- 172.21.15.112:0/428075107 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f2d9006e530 msgr2=0x7f2d900709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:59.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.587+0000 7f2dae1dc640 1 --2- 172.21.15.112:0/428075107 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f2d9006e530 0x7f2d900709e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2d98007ac0 tx=0x7f2d98035040 comp rx=0 tx=0).stop 2024-09-16T00:42:59.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.587+0000 7f2dae1dc640 1 -- 172.21.15.112:0/428075107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2da80fcf80 msgr2=0x7f2da810e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:59.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.587+0000 7f2dae1dc640 1 --2- 172.21.15.112:0/428075107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2da80fcf80 0x7f2da810e6b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c00c830 tx=0x7f2d9c004430 comp rx=0 tx=0).stop 2024-09-16T00:42:59.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.587+0000 7f2dae1dc640 1 -- 172.21.15.112:0/428075107 shutdown_connections 2024-09-16T00:42:59.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.587+0000 7f2dae1dc640 1 --2- 172.21.15.112:0/428075107 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f2d9006e530 0x7f2d900709e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:59.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.587+0000 7f2dae1dc640 1 --2- 172.21.15.112:0/428075107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2da80fcf80 0x7f2da810e6b0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:59.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.587+0000 7f2dae1dc640 1 --2- 172.21.15.112:0/428075107 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2da80fb810 0x7f2da810e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:59.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.587+0000 7f2dae1dc640 1 -- 172.21.15.112:0/428075107 >> 172.21.15.112:0/428075107 conn(0x7f2da80723b0 msgr2=0x7f2da80fa480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:59.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.587+0000 7f2dae1dc640 1 -- 172.21.15.112:0/428075107 shutdown_connections 2024-09-16T00:42:59.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.587+0000 7f2dae1dc640 1 -- 172.21.15.112:0/428075107 wait complete. 2024-09-16T00:42:59.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.666+0000 7f1b6594f640 1 -- 172.21.15.112:0/1653655629 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b600fd580 msgr2=0x7f1b600fd950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:59.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.666+0000 7f1b6594f640 1 --2- 172.21.15.112:0/1653655629 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b600fd580 0x7f1b600fd950 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f1b50004900 tx=0x7f1b50009da0 comp rx=0 tx=0).stop 2024-09-16T00:42:59.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.666+0000 7f1b6594f640 1 -- 172.21.15.112:0/1653655629 shutdown_connections 2024-09-16T00:42:59.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.666+0000 7f1b6594f640 1 --2- 172.21.15.112:0/1653655629 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b600fdf20 0x7f1b601065f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:59.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.666+0000 7f1b6594f640 1 --2- 172.21.15.112:0/1653655629 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b600fd580 0x7f1b600fd950 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:59.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.666+0000 7f1b6594f640 1 -- 172.21.15.112:0/1653655629 >> 172.21.15.112:0/1653655629 conn(0x7f1b600f7060 msgr2=0x7f1b600f9470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:59.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.666+0000 7f1b6594f640 1 -- 172.21.15.112:0/1653655629 shutdown_connections 2024-09-16T00:42:59.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.666+0000 7f1b6594f640 1 -- 172.21.15.112:0/1653655629 wait complete. 2024-09-16T00:42:59.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.667+0000 7f1b6594f640 1 Processor -- start 2024-09-16T00:42:59.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.667+0000 7f1b6594f640 1 -- start start 2024-09-16T00:42:59.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.667+0000 7f1b6594f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b600fd580 0x7f1b60113240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:59.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.667+0000 7f1b6594f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b600fdf20 0x7f1b60113780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:59.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.667+0000 7f1b6594f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b60113d80 con 0x7f1b600fd580 2024-09-16T00:42:59.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.667+0000 7f1b6594f640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6010d1c0 con 0x7f1b600fdf20 2024-09-16T00:42:59.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.667+0000 7f1b5effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b600fd580 0x7f1b60113240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:59.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.667+0000 7f1b5effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b600fd580 0x7f1b60113240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54022/0 (socket says 172.21.15.112:54022) 2024-09-16T00:42:59.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.667+0000 7f1b5effd640 1 -- 172.21.15.112:0/3279864982 learned_addr learned my addr 172.21.15.112:0/3279864982 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:42:59.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.667+0000 7f1b57fff640 1 --2- 172.21.15.112:0/3279864982 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b600fdf20 0x7f1b60113780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:59.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.667+0000 7f1b5effd640 1 -- 172.21.15.112:0/3279864982 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b600fdf20 msgr2=0x7f1b60113780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:59.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.667+0000 7f1b5effd640 1 --2- 172.21.15.112:0/3279864982 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b600fdf20 0x7f1b60113780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:59.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.668+0000 7f1b5effd640 1 -- 172.21.15.112:0/3279864982 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b50005040 con 0x7f1b600fd580 2024-09-16T00:42:59.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.668+0000 7f1b5effd640 1 --2- 172.21.15.112:0/3279864982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b600fd580 0x7f1b60113240 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f1b5002a560 tx=0x7f1b50009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:59.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.668+0000 7f1b577fe640 1 -- 172.21.15.112:0/3279864982 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b50038070 con 0x7f1b600fd580 2024-09-16T00:42:59.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.668+0000 7f1b577fe640 1 -- 172.21.15.112:0/3279864982 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f1b50032cd0 con 0x7f1b600fd580 2024-09-16T00:42:59.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.668+0000 7f1b6594f640 1 -- 172.21.15.112:0/3279864982 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b6010d440 con 0x7f1b600fd580 2024-09-16T00:42:59.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.668+0000 7f1b577fe640 1 -- 172.21.15.112:0/3279864982 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b500096c0 con 0x7f1b600fd580 2024-09-16T00:42:59.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.668+0000 7f1b6594f640 1 -- 172.21.15.112:0/3279864982 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b6010d900 con 0x7f1b600fd580 2024-09-16T00:42:59.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.669+0000 7f1b6594f640 1 -- 172.21.15.112:0/3279864982 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b600662d0 con 0x7f1b600fd580 2024-09-16T00:42:59.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.670+0000 7f1b577fe640 1 -- 172.21.15.112:0/3279864982 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f1b50030020 con 0x7f1b600fd580 2024-09-16T00:42:59.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.670+0000 7f1b577fe640 1 --2- 172.21.15.112:0/3279864982 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f1b4c06e530 0x7f1b4c0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:59.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.670+0000 7f1b577fe640 1 -- 172.21.15.112:0/3279864982 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f1b500ae920 con 0x7f1b600fd580 2024-09-16T00:42:59.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.670+0000 7f1b57fff640 1 --2- 172.21.15.112:0/3279864982 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f1b4c06e530 0x7f1b4c0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:59.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.671+0000 7f1b57fff640 1 --2- 172.21.15.112:0/3279864982 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f1b4c06e530 0x7f1b4c0709e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1b6010eb00 tx=0x7f1b48009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:59.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.674+0000 7f1b577fe640 1 -- 172.21.15.112:0/3279864982 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f1b50077890 con 0x7f1b600fd580 2024-09-16T00:42:59.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.848+0000 7f1b6594f640 1 -- 172.21.15.112:0/3279864982 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1b6004b720 con 0x7f1b4c06e530 2024-09-16T00:42:59.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.856+0000 7f1b577fe640 1 -- 172.21.15.112:0/3279864982 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7f1b6004b720 con 0x7f1b4c06e530 2024-09-16T00:42:59.856 INFO:teuthology.orchestra.run.smithi112.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T00:42:59.857 INFO:teuthology.orchestra.run.smithi112.stdout:alertmanager.smithi112 smithi112 *:9093,9094 running (15m) 9m ago 24m 24.7M - 0.25.0 c8568f914cd2 d85ea0562022 2024-09-16T00:42:59.857 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi112 smithi112 running (12m) 9m ago 25m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c e4fe58f9702a 2024-09-16T00:42:59.857 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi169 smithi169 running (12m) 8m ago 24m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c 3ba2c81d527f 2024-09-16T00:42:59.857 INFO:teuthology.orchestra.run.smithi112.stdout:grafana.smithi112 smithi112 *:3000 running (14m) 9m ago 24m 82.1M - 9.4.12 2bacad6d85d8 51f9d2013da2 2024-09-16T00:42:59.857 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.hxfwkh smithi112 running (9m) 9m ago 19m 14.2M - 19.1.1-326-g8921c8ef f1455bd0219c 1723069902d6 2024-09-16T00:42:59.857 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.ithwzj smithi112 running (9m) 9m ago 19m 12.2M - 19.1.1-326-g8921c8ef f1455bd0219c e7aad5b81db5 2024-09-16T00:42:59.857 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.gawwgw smithi169 running (8m) 8m ago 19m 15.7M - 19.1.1-326-g8921c8ef f1455bd0219c 469be459ee12 2024-09-16T00:42:59.857 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.lgeqrj smithi169 running (8m) 8m ago 19m 13.3M - 19.1.1-326-g8921c8ef f1455bd0219c 31f0fbd2395d 2024-09-16T00:42:59.857 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi112.xrkkbc smithi112 *:8443,9283,8765 running (15m) 9m ago 26m 504M - 19.1.1-326-g8921c8ef f1455bd0219c a981f7fd2810 2024-09-16T00:42:59.857 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi169.mtrobk smithi169 *:8443,9283,8765 running (15m) 8m ago 24m 442M - 19.1.1-326-g8921c8ef f1455bd0219c 4096e5369c2d 2024-09-16T00:42:59.857 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi112 smithi112 running (13m) 9m ago 26m 49.8M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 5aaec033c042 2024-09-16T00:42:59.857 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi169 smithi169 running (13m) 8m ago 23m 38.9M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 7bc609155f3f 2024-09-16T00:42:59.857 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi112 smithi112 *:9100 running (16m) 9m ago 25m 15.5M - 1.5.0 0da6a335fe13 6404d1d601e2 2024-09-16T00:42:59.858 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi169 smithi169 *:9100 running (15m) 8m ago 24m 15.4M - 1.5.0 0da6a335fe13 5505baf702a6 2024-09-16T00:42:59.858 INFO:teuthology.orchestra.run.smithi112.stdout:osd.0 smithi112 running (12m) 9m ago 23m 105M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 53efc9397017 2024-09-16T00:42:59.858 INFO:teuthology.orchestra.run.smithi112.stdout:osd.1 smithi112 running (11m) 9m ago 22m 98.9M 1557M 19.1.1-326-g8921c8ef f1455bd0219c fe6f1a4a2cf4 2024-09-16T00:42:59.858 INFO:teuthology.orchestra.run.smithi112.stdout:osd.2 smithi112 running (11m) 9m ago 22m 102M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 5f7863712057 2024-09-16T00:42:59.858 INFO:teuthology.orchestra.run.smithi112.stdout:osd.3 smithi169 running (10m) 8m ago 21m 92.1M 2581M 19.1.1-326-g8921c8ef f1455bd0219c ee07b5577810 2024-09-16T00:42:59.858 INFO:teuthology.orchestra.run.smithi112.stdout:osd.4 smithi169 running (10m) 8m ago 21m 103M 2581M 19.1.1-326-g8921c8ef f1455bd0219c 769f026a54eb 2024-09-16T00:42:59.858 INFO:teuthology.orchestra.run.smithi112.stdout:osd.5 smithi169 running (9m) 8m ago 21m 125M 2581M 19.1.1-326-g8921c8ef f1455bd0219c a6007c13cf78 2024-09-16T00:42:59.858 INFO:teuthology.orchestra.run.smithi112.stdout:prometheus.smithi112 smithi112 *:9095 running (15m) 9m ago 24m 64.1M - 2.43.0 a07b618ecd1d 7a1d42e60d2b 2024-09-16T00:42:59.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.858+0000 7f1b6594f640 1 -- 172.21.15.112:0/3279864982 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f1b4c06e530 msgr2=0x7f1b4c0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:59.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.858+0000 7f1b6594f640 1 --2- 172.21.15.112:0/3279864982 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f1b4c06e530 0x7f1b4c0709e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1b6010eb00 tx=0x7f1b48009040 comp rx=0 tx=0).stop 2024-09-16T00:42:59.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.858+0000 7f1b6594f640 1 -- 172.21.15.112:0/3279864982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b600fd580 msgr2=0x7f1b60113240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:59.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.858+0000 7f1b6594f640 1 --2- 172.21.15.112:0/3279864982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b600fd580 0x7f1b60113240 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f1b5002a560 tx=0x7f1b50009da0 comp rx=0 tx=0).stop 2024-09-16T00:42:59.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.859+0000 7f1b6594f640 1 -- 172.21.15.112:0/3279864982 shutdown_connections 2024-09-16T00:42:59.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.859+0000 7f1b6594f640 1 --2- 172.21.15.112:0/3279864982 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f1b4c06e530 0x7f1b4c0709e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:59.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.859+0000 7f1b6594f640 1 --2- 172.21.15.112:0/3279864982 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b600fdf20 0x7f1b60113780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:59.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.859+0000 7f1b6594f640 1 --2- 172.21.15.112:0/3279864982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b600fd580 0x7f1b60113240 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:59.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.859+0000 7f1b6594f640 1 -- 172.21.15.112:0/3279864982 >> 172.21.15.112:0/3279864982 conn(0x7f1b600f7060 msgr2=0x7f1b600f85d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:59.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.859+0000 7f1b6594f640 1 -- 172.21.15.112:0/3279864982 shutdown_connections 2024-09-16T00:42:59.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.859+0000 7f1b6594f640 1 -- 172.21.15.112:0/3279864982 wait complete. 2024-09-16T00:42:59.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.937+0000 7f29bc3a8640 1 -- 172.21.15.112:0/847656268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29b41014f0 msgr2=0x7f29b41018c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:59.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.937+0000 7f29bc3a8640 1 --2- 172.21.15.112:0/847656268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29b41014f0 0x7f29b41018c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f29b00048d0 tx=0x7f29b0009da0 comp rx=0 tx=0).stop 2024-09-16T00:42:59.937 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:42:59 smithi112 ceph-mon[97750]: pgmap v457: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:42:59.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.937+0000 7f29bc3a8640 1 -- 172.21.15.112:0/847656268 shutdown_connections 2024-09-16T00:42:59.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.937+0000 7f29bc3a8640 1 --2- 172.21.15.112:0/847656268 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29b40fb4b0 0x7f29b40fb920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:59.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.937+0000 7f29bc3a8640 1 --2- 172.21.15.112:0/847656268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29b41014f0 0x7f29b41018c0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:59.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.937+0000 7f29bc3a8640 1 -- 172.21.15.112:0/847656268 >> 172.21.15.112:0/847656268 conn(0x7f29b40f7030 msgr2=0x7f29b40f9440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:42:59.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.937+0000 7f29bc3a8640 1 -- 172.21.15.112:0/847656268 shutdown_connections 2024-09-16T00:42:59.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.937+0000 7f29bc3a8640 1 -- 172.21.15.112:0/847656268 wait complete. 2024-09-16T00:42:59.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.938+0000 7f29bc3a8640 1 Processor -- start 2024-09-16T00:42:59.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.938+0000 7f29bc3a8640 1 -- start start 2024-09-16T00:42:59.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.938+0000 7f29bc3a8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29b40fb4b0 0x7f29b41120b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:59.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.938+0000 7f29bc3a8640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29b41014f0 0x7f29b41125f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:59.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.938+0000 7f29bc3a8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b4112bf0 con 0x7f29b40fb4b0 2024-09-16T00:42:59.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.938+0000 7f29bc3a8640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b410c030 con 0x7f29b41014f0 2024-09-16T00:42:59.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.938+0000 7f29ba144640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29b40fb4b0 0x7f29b41120b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:59.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.938+0000 7f29ba144640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29b40fb4b0 0x7f29b41120b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54054/0 (socket says 172.21.15.112:54054) 2024-09-16T00:42:59.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.938+0000 7f29ba144640 1 -- 172.21.15.112:0/4255386578 learned_addr learned my addr 172.21.15.112:0/4255386578 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:42:59.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.939+0000 7f29b9943640 1 --2- 172.21.15.112:0/4255386578 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29b41014f0 0x7f29b41125f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:59.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.939+0000 7f29ba144640 1 -- 172.21.15.112:0/4255386578 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29b41014f0 msgr2=0x7f29b41125f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:42:59.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.939+0000 7f29ba144640 1 --2- 172.21.15.112:0/4255386578 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29b41014f0 0x7f29b41125f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:42:59.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.939+0000 7f29ba144640 1 -- 172.21.15.112:0/4255386578 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29b0005040 con 0x7f29b40fb4b0 2024-09-16T00:42:59.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.939+0000 7f29b9943640 1 --2- 172.21.15.112:0/4255386578 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29b41014f0 0x7f29b41125f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:42:59.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.939+0000 7f29ba144640 1 --2- 172.21.15.112:0/4255386578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29b40fb4b0 0x7f29b41120b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f29b0000c00 tx=0x7f29b0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:59.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.939+0000 7f29aaffd640 1 -- 172.21.15.112:0/4255386578 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f29b003a070 con 0x7f29b40fb4b0 2024-09-16T00:42:59.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.939+0000 7f29bc3a8640 1 -- 172.21.15.112:0/4255386578 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29b410c2b0 con 0x7f29b40fb4b0 2024-09-16T00:42:59.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.939+0000 7f29aaffd640 1 -- 172.21.15.112:0/4255386578 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f29b0009880 con 0x7f29b40fb4b0 2024-09-16T00:42:59.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.939+0000 7f29aaffd640 1 -- 172.21.15.112:0/4255386578 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f29b00338f0 con 0x7f29b40fb4b0 2024-09-16T00:42:59.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.939+0000 7f29bc3a8640 1 -- 172.21.15.112:0/4255386578 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29b410c7f0 con 0x7f29b40fb4b0 2024-09-16T00:42:59.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.940+0000 7f29bc3a8640 1 -- 172.21.15.112:0/4255386578 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29b40fcbf0 con 0x7f29b40fb4b0 2024-09-16T00:42:59.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.941+0000 7f29aaffd640 1 -- 172.21.15.112:0/4255386578 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f29b0030020 con 0x7f29b40fb4b0 2024-09-16T00:42:59.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.941+0000 7f29aaffd640 1 --2- 172.21.15.112:0/4255386578 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f29a006e530 0x7f29a00709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:42:59.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.941+0000 7f29aaffd640 1 -- 172.21.15.112:0/4255386578 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f29b00b0ba0 con 0x7f29b40fb4b0 2024-09-16T00:42:59.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.941+0000 7f29b9943640 1 --2- 172.21.15.112:0/4255386578 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f29a006e530 0x7f29a00709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:42:59.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.942+0000 7f29b9943640 1 --2- 172.21.15.112:0/4255386578 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f29a006e530 0x7f29a00709e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f29b410d970 tx=0x7f29a4009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:42:59.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:42:59.946+0000 7f29aaffd640 1 -- 172.21.15.112:0/4255386578 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f29b0079b10 con 0x7f29b40fb4b0 2024-09-16T00:43:00.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.177+0000 7f29bc3a8640 1 -- 172.21.15.112:0/4255386578 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f29b404b720 con 0x7f29b40fb4b0 2024-09-16T00:43:00.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.178+0000 7f29aaffd640 1 -- 172.21.15.112:0/4255386578 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f29b007d1d0 con 0x7f29b40fb4b0 2024-09-16T00:43:00.178 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:43:00.178 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": { 2024-09-16T00:43:00.179 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:43:00.179 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:43:00.179 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": { 2024-09-16T00:43:00.179 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:43:00.179 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:43:00.179 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": { 2024-09-16T00:43:00.179 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 6 2024-09-16T00:43:00.179 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:43:00.179 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": { 2024-09-16T00:43:00.179 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 4 2024-09-16T00:43:00.179 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:43:00.179 INFO:teuthology.orchestra.run.smithi112.stdout: "overall": { 2024-09-16T00:43:00.179 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 14 2024-09-16T00:43:00.180 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-16T00:43:00.180 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:43:00.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.180+0000 7f29bc3a8640 1 -- 172.21.15.112:0/4255386578 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f29a006e530 msgr2=0x7f29a00709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:00.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.180+0000 7f29bc3a8640 1 --2- 172.21.15.112:0/4255386578 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f29a006e530 0x7f29a00709e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f29b410d970 tx=0x7f29a4009040 comp rx=0 tx=0).stop 2024-09-16T00:43:00.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.180+0000 7f29bc3a8640 1 -- 172.21.15.112:0/4255386578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29b40fb4b0 msgr2=0x7f29b41120b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:00.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.180+0000 7f29bc3a8640 1 --2- 172.21.15.112:0/4255386578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29b40fb4b0 0x7f29b41120b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f29b0000c00 tx=0x7f29b0009da0 comp rx=0 tx=0).stop 2024-09-16T00:43:00.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.180+0000 7f29bc3a8640 1 -- 172.21.15.112:0/4255386578 shutdown_connections 2024-09-16T00:43:00.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.180+0000 7f29bc3a8640 1 --2- 172.21.15.112:0/4255386578 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f29a006e530 0x7f29a00709e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:00.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.180+0000 7f29bc3a8640 1 --2- 172.21.15.112:0/4255386578 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29b41014f0 0x7f29b41125f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:00.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.180+0000 7f29bc3a8640 1 --2- 172.21.15.112:0/4255386578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29b40fb4b0 0x7f29b41120b0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:00.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.180+0000 7f29bc3a8640 1 -- 172.21.15.112:0/4255386578 >> 172.21.15.112:0/4255386578 conn(0x7f29b40f7030 msgr2=0x7f29b40f8930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:00.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.180+0000 7f29bc3a8640 1 -- 172.21.15.112:0/4255386578 shutdown_connections 2024-09-16T00:43:00.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.181+0000 7f29bc3a8640 1 -- 172.21.15.112:0/4255386578 wait complete. 2024-09-16T00:43:00.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.258+0000 7f47cb8e6640 1 -- 172.21.15.112:0/1585472490 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47c4064fc0 msgr2=0x7f47c4065430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:00.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.258+0000 7f47cb8e6640 1 --2- 172.21.15.112:0/1585472490 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47c4064fc0 0x7f47c4065430 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f47b40048d0 tx=0x7f47b4009da0 comp rx=0 tx=0).stop 2024-09-16T00:43:00.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.258+0000 7f47cb8e6640 1 -- 172.21.15.112:0/1585472490 shutdown_connections 2024-09-16T00:43:00.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.258+0000 7f47cb8e6640 1 --2- 172.21.15.112:0/1585472490 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47c4064fc0 0x7f47c4065430 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:00.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.258+0000 7f47cb8e6640 1 --2- 172.21.15.112:0/1585472490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47c40fe460 0x7f47c40fe830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:00.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.258+0000 7f47cb8e6640 1 -- 172.21.15.112:0/1585472490 >> 172.21.15.112:0/1585472490 conn(0x7f47c40723b0 msgr2=0x7f47c40727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:00.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.258+0000 7f47cb8e6640 1 -- 172.21.15.112:0/1585472490 shutdown_connections 2024-09-16T00:43:00.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.258+0000 7f47cb8e6640 1 -- 172.21.15.112:0/1585472490 wait complete. 2024-09-16T00:43:00.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.259+0000 7f47cb8e6640 1 Processor -- start 2024-09-16T00:43:00.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.259+0000 7f47cb8e6640 1 -- start start 2024-09-16T00:43:00.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.259+0000 7f47cb8e6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47c4064fc0 0x7f47c41131d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:00.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.259+0000 7f47cb8e6640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47c40fe460 0x7f47c4113710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:00.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.259+0000 7f47cb8e6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c4113d10 con 0x7f47c4064fc0 2024-09-16T00:43:00.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.259+0000 7f47cb8e6640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c410d1a0 con 0x7f47c40fe460 2024-09-16T00:43:00.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.259+0000 7f47c9682640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47c4064fc0 0x7f47c41131d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:00.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.259+0000 7f47c9682640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47c4064fc0 0x7f47c41131d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54072/0 (socket says 172.21.15.112:54072) 2024-09-16T00:43:00.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.259+0000 7f47c9682640 1 -- 172.21.15.112:0/2323058429 learned_addr learned my addr 172.21.15.112:0/2323058429 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:43:00.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.260+0000 7f47c8e81640 1 --2- 172.21.15.112:0/2323058429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47c40fe460 0x7f47c4113710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:00.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.260+0000 7f47c9682640 1 -- 172.21.15.112:0/2323058429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47c40fe460 msgr2=0x7f47c4113710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:00.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.260+0000 7f47c9682640 1 --2- 172.21.15.112:0/2323058429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47c40fe460 0x7f47c4113710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:00.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.260+0000 7f47c9682640 1 -- 172.21.15.112:0/2323058429 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47b4005040 con 0x7f47c4064fc0 2024-09-16T00:43:00.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.260+0000 7f47c9682640 1 --2- 172.21.15.112:0/2323058429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47c4064fc0 0x7f47c41131d0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f47c0002a50 tx=0x7f47c0004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:00.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.260+0000 7f47b9ffb640 1 -- 172.21.15.112:0/2323058429 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47c0007c30 con 0x7f47c4064fc0 2024-09-16T00:43:00.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.260+0000 7f47b9ffb640 1 -- 172.21.15.112:0/2323058429 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f47c000f040 con 0x7f47c4064fc0 2024-09-16T00:43:00.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.260+0000 7f47b9ffb640 1 -- 172.21.15.112:0/2323058429 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47c000da10 con 0x7f47c4064fc0 2024-09-16T00:43:00.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.260+0000 7f47cb8e6640 1 -- 172.21.15.112:0/2323058429 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47c410d480 con 0x7f47c4064fc0 2024-09-16T00:43:00.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.260+0000 7f47cb8e6640 1 -- 172.21.15.112:0/2323058429 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47c410d9f0 con 0x7f47c4064fc0 2024-09-16T00:43:00.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.262+0000 7f47cb8e6640 1 -- 172.21.15.112:0/2323058429 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47c40fdcf0 con 0x7f47c4064fc0 2024-09-16T00:43:00.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.262+0000 7f47b9ffb640 1 -- 172.21.15.112:0/2323058429 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f47c000dbb0 con 0x7f47c4064fc0 2024-09-16T00:43:00.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.262+0000 7f47b9ffb640 1 --2- 172.21.15.112:0/2323058429 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f47b006e530 0x7f47b00709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:00.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.262+0000 7f47b9ffb640 1 -- 172.21.15.112:0/2323058429 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f47c008da40 con 0x7f47c4064fc0 2024-09-16T00:43:00.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.262+0000 7f47c8e81640 1 --2- 172.21.15.112:0/2323058429 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f47b006e530 0x7f47b00709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:00.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.263+0000 7f47c8e81640 1 --2- 172.21.15.112:0/2323058429 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f47b006e530 0x7f47b00709e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f47b4009ac0 tx=0x7f47b4035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:00.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.267+0000 7f47b9ffb640 1 -- 172.21.15.112:0/2323058429 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f47c0056ae0 con 0x7f47c4064fc0 2024-09-16T00:43:00.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:42:59 smithi169 ceph-mon[77626]: pgmap v457: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:43:00.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.465+0000 7f47cb8e6640 1 -- 172.21.15.112:0/2323058429 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f47c404b750 con 0x7f47c4064fc0 2024-09-16T00:43:00.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.466+0000 7f47b9ffb640 1 -- 172.21.15.112:0/2323058429 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1914 (secure 0 0 0) 0x7f47c000de60 con 0x7f47c4064fc0 2024-09-16T00:43:00.466 INFO:teuthology.orchestra.run.smithi112.stdout:e34 2024-09-16T00:43:00.466 INFO:teuthology.orchestra.run.smithi112.stdout:btime 2024-09-16T00:34:26:619254+0000 2024-09-16T00:43:00.467 INFO:teuthology.orchestra.run.smithi112.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T00:43:00.467 INFO:teuthology.orchestra.run.smithi112.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:43:00.467 INFO:teuthology.orchestra.run.smithi112.stdout:legacy client fscid: 1 2024-09-16T00:43:00.467 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:43:00.467 INFO:teuthology.orchestra.run.smithi112.stdout:Filesystem 'cephfs' (1) 2024-09-16T00:43:00.467 INFO:teuthology.orchestra.run.smithi112.stdout:fs_name cephfs 2024-09-16T00:43:00.467 INFO:teuthology.orchestra.run.smithi112.stdout:epoch 25 2024-09-16T00:43:00.467 INFO:teuthology.orchestra.run.smithi112.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-16T00:43:00.467 INFO:teuthology.orchestra.run.smithi112.stdout:created 2024-09-16T00:23:22.328199+0000 2024-09-16T00:43:00.467 INFO:teuthology.orchestra.run.smithi112.stdout:modified 2024-09-16T00:33:29.012991+0000 2024-09-16T00:43:00.467 INFO:teuthology.orchestra.run.smithi112.stdout:tableserver 0 2024-09-16T00:43:00.467 INFO:teuthology.orchestra.run.smithi112.stdout:root 0 2024-09-16T00:43:00.467 INFO:teuthology.orchestra.run.smithi112.stdout:session_timeout 60 2024-09-16T00:43:00.468 INFO:teuthology.orchestra.run.smithi112.stdout:session_autoclose 300 2024-09-16T00:43:00.468 INFO:teuthology.orchestra.run.smithi112.stdout:max_file_size 1099511627776 2024-09-16T00:43:00.468 INFO:teuthology.orchestra.run.smithi112.stdout:max_xattr_size 65536 2024-09-16T00:43:00.468 INFO:teuthology.orchestra.run.smithi112.stdout:required_client_features {} 2024-09-16T00:43:00.468 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure 0 2024-09-16T00:43:00.468 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure_osd_epoch 82 2024-09-16T00:43:00.468 INFO:teuthology.orchestra.run.smithi112.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:43:00.468 INFO:teuthology.orchestra.run.smithi112.stdout:max_mds 2 2024-09-16T00:43:00.468 INFO:teuthology.orchestra.run.smithi112.stdout:in 0,1 2024-09-16T00:43:00.468 INFO:teuthology.orchestra.run.smithi112.stdout:up {} 2024-09-16T00:43:00.468 INFO:teuthology.orchestra.run.smithi112.stdout:failed 0,1 2024-09-16T00:43:00.468 INFO:teuthology.orchestra.run.smithi112.stdout:damaged 2024-09-16T00:43:00.468 INFO:teuthology.orchestra.run.smithi112.stdout:stopped 2024-09-16T00:43:00.469 INFO:teuthology.orchestra.run.smithi112.stdout:data_pools [3] 2024-09-16T00:43:00.469 INFO:teuthology.orchestra.run.smithi112.stdout:metadata_pool 2 2024-09-16T00:43:00.469 INFO:teuthology.orchestra.run.smithi112.stdout:inline_data enabled 2024-09-16T00:43:00.469 INFO:teuthology.orchestra.run.smithi112.stdout:balancer 2024-09-16T00:43:00.469 INFO:teuthology.orchestra.run.smithi112.stdout:bal_rank_mask -1 2024-09-16T00:43:00.469 INFO:teuthology.orchestra.run.smithi112.stdout:standby_count_wanted 1 2024-09-16T00:43:00.469 INFO:teuthology.orchestra.run.smithi112.stdout:qdb_cluster leader: 0 members: 2024-09-16T00:43:00.469 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:43:00.469 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:43:00.469 INFO:teuthology.orchestra.run.smithi112.stdout:Standby daemons: 2024-09-16T00:43:00.469 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:43:00.469 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.ithwzj{-1:34392} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.112:6826/476229221,v1:172.21.15.112:6827/476229221] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:43:00.469 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.hxfwkh{-1:34404} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.112:6828/2356899876,v1:172.21.15.112:6829/2356899876] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:43:00.469 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.gawwgw{-1:44343} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.169:6824/1635329957,v1:172.21.15.169:6825/1635329957] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:43:00.470 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.lgeqrj{-1:44347} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.169:6826/184810031,v1:172.21.15.169:6827/184810031] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:43:00.470 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 34 2024-09-16T00:43:00.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.468+0000 7f47cb8e6640 1 -- 172.21.15.112:0/2323058429 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f47b006e530 msgr2=0x7f47b00709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:00.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.468+0000 7f47cb8e6640 1 --2- 172.21.15.112:0/2323058429 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f47b006e530 0x7f47b00709e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f47b4009ac0 tx=0x7f47b4035040 comp rx=0 tx=0).stop 2024-09-16T00:43:00.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.468+0000 7f47cb8e6640 1 -- 172.21.15.112:0/2323058429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47c4064fc0 msgr2=0x7f47c41131d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:00.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.468+0000 7f47cb8e6640 1 --2- 172.21.15.112:0/2323058429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47c4064fc0 0x7f47c41131d0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f47c0002a50 tx=0x7f47c0004600 comp rx=0 tx=0).stop 2024-09-16T00:43:00.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.468+0000 7f47cb8e6640 1 -- 172.21.15.112:0/2323058429 shutdown_connections 2024-09-16T00:43:00.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.468+0000 7f47cb8e6640 1 --2- 172.21.15.112:0/2323058429 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f47b006e530 0x7f47b00709e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:00.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.468+0000 7f47cb8e6640 1 --2- 172.21.15.112:0/2323058429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47c40fe460 0x7f47c4113710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:00.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.468+0000 7f47cb8e6640 1 --2- 172.21.15.112:0/2323058429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47c4064fc0 0x7f47c41131d0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:00.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.468+0000 7f47cb8e6640 1 -- 172.21.15.112:0/2323058429 >> 172.21.15.112:0/2323058429 conn(0x7f47c40723b0 msgr2=0x7f47c41053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:00.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.468+0000 7f47cb8e6640 1 -- 172.21.15.112:0/2323058429 shutdown_connections 2024-09-16T00:43:00.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.468+0000 7f47cb8e6640 1 -- 172.21.15.112:0/2323058429 wait complete. 2024-09-16T00:43:00.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.547+0000 7f936dcbe640 1 -- 172.21.15.112:0/848993146 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93680fb4a0 msgr2=0x7f93680fb910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:00.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.547+0000 7f936dcbe640 1 --2- 172.21.15.112:0/848993146 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93680fb4a0 0x7f93680fb910 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f935c0048d0 tx=0x7f935c009da0 comp rx=0 tx=0).stop 2024-09-16T00:43:00.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.547+0000 7f936dcbe640 1 -- 172.21.15.112:0/848993146 shutdown_connections 2024-09-16T00:43:00.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.547+0000 7f936dcbe640 1 --2- 172.21.15.112:0/848993146 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93680fb4a0 0x7f93680fb910 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:00.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.547+0000 7f936dcbe640 1 --2- 172.21.15.112:0/848993146 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93681014e0 0x7f93681018b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:00.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.547+0000 7f936dcbe640 1 -- 172.21.15.112:0/848993146 >> 172.21.15.112:0/848993146 conn(0x7f93680f7060 msgr2=0x7f93680f9470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:00.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.547+0000 7f936dcbe640 1 -- 172.21.15.112:0/848993146 shutdown_connections 2024-09-16T00:43:00.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.548+0000 7f936dcbe640 1 -- 172.21.15.112:0/848993146 wait complete. 2024-09-16T00:43:00.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.548+0000 7f936dcbe640 1 Processor -- start 2024-09-16T00:43:00.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.548+0000 7f936dcbe640 1 -- start start 2024-09-16T00:43:00.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.548+0000 7f936dcbe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93680fb4a0 0x7f9368115420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:00.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.548+0000 7f936dcbe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93681014e0 0x7f9368115960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:00.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.548+0000 7f936dcbe640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9368115f60 con 0x7f93681014e0 2024-09-16T00:43:00.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.548+0000 7f936dcbe640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93681160d0 con 0x7f93680fb4a0 2024-09-16T00:43:00.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.549+0000 7f9366ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93681014e0 0x7f9368115960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:00.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.549+0000 7f9366ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93681014e0 0x7f9368115960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54086/0 (socket says 172.21.15.112:54086) 2024-09-16T00:43:00.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.549+0000 7f9366ffd640 1 -- 172.21.15.112:0/1770078412 learned_addr learned my addr 172.21.15.112:0/1770078412 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:43:00.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.549+0000 7f93677fe640 1 --2- 172.21.15.112:0/1770078412 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93680fb4a0 0x7f9368115420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:00.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.549+0000 7f9366ffd640 1 -- 172.21.15.112:0/1770078412 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93680fb4a0 msgr2=0x7f9368115420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:00.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.549+0000 7f9366ffd640 1 --2- 172.21.15.112:0/1770078412 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93680fb4a0 0x7f9368115420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:00.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.549+0000 7f9366ffd640 1 -- 172.21.15.112:0/1770078412 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f935c005040 con 0x7f93681014e0 2024-09-16T00:43:00.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.549+0000 7f9366ffd640 1 --2- 172.21.15.112:0/1770078412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93681014e0 0x7f9368115960 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f935c0043d0 tx=0x7f935c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:00.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.549+0000 7f934ffff640 1 -- 172.21.15.112:0/1770078412 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f935c038070 con 0x7f93681014e0 2024-09-16T00:43:00.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.549+0000 7f934ffff640 1 -- 172.21.15.112:0/1770078412 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f935c0096e0 con 0x7f93681014e0 2024-09-16T00:43:00.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.549+0000 7f936dcbe640 1 -- 172.21.15.112:0/1770078412 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f936810f4f0 con 0x7f93681014e0 2024-09-16T00:43:00.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.549+0000 7f934ffff640 1 -- 172.21.15.112:0/1770078412 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f935c03d420 con 0x7f93681014e0 2024-09-16T00:43:00.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.549+0000 7f936dcbe640 1 -- 172.21.15.112:0/1770078412 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f936810fa10 con 0x7f93681014e0 2024-09-16T00:43:00.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.551+0000 7f936dcbe640 1 -- 172.21.15.112:0/1770078412 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93680fcbe0 con 0x7f93681014e0 2024-09-16T00:43:00.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.551+0000 7f934ffff640 1 -- 172.21.15.112:0/1770078412 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f935c030020 con 0x7f93681014e0 2024-09-16T00:43:00.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.552+0000 7f934ffff640 1 --2- 172.21.15.112:0/1770078412 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f935006e530 0x7f93500709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:00.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.552+0000 7f934ffff640 1 -- 172.21.15.112:0/1770078412 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f935c0ae7c0 con 0x7f93681014e0 2024-09-16T00:43:00.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.552+0000 7f93677fe640 1 --2- 172.21.15.112:0/1770078412 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f935006e530 0x7f93500709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:00.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.552+0000 7f93677fe640 1 --2- 172.21.15.112:0/1770078412 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f935006e530 0x7f93500709e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9358004ed0 tx=0x7f935800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:00.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.556+0000 7f934ffff640 1 -- 172.21.15.112:0/1770078412 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f935c077730 con 0x7f93681014e0 2024-09-16T00:43:00.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.741+0000 7f936dcbe640 1 -- 172.21.15.112:0/1770078412 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9368108480 con 0x7f935006e530 2024-09-16T00:43:00.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.742+0000 7f934ffff640 1 -- 172.21.15.112:0/1770078412 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f9368108480 con 0x7f935006e530 2024-09-16T00:43:00.743 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:43:00.743 INFO:teuthology.orchestra.run.smithi112.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80", 2024-09-16T00:43:00.743 INFO:teuthology.orchestra.run.smithi112.stdout: "in_progress": true, 2024-09-16T00:43:00.743 INFO:teuthology.orchestra.run.smithi112.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T00:43:00.743 INFO:teuthology.orchestra.run.smithi112.stdout: "services_complete": [ 2024-09-16T00:43:00.743 INFO:teuthology.orchestra.run.smithi112.stdout: "mds", 2024-09-16T00:43:00.743 INFO:teuthology.orchestra.run.smithi112.stdout: "osd", 2024-09-16T00:43:00.743 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr", 2024-09-16T00:43:00.743 INFO:teuthology.orchestra.run.smithi112.stdout: "mon", 2024-09-16T00:43:00.743 INFO:teuthology.orchestra.run.smithi112.stdout: "crash" 2024-09-16T00:43:00.743 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-16T00:43:00.743 INFO:teuthology.orchestra.run.smithi112.stdout: "progress": "16/21 daemons upgraded", 2024-09-16T00:43:00.744 INFO:teuthology.orchestra.run.smithi112.stdout: "message": "Currently upgrading mds daemons", 2024-09-16T00:43:00.744 INFO:teuthology.orchestra.run.smithi112.stdout: "is_paused": false 2024-09-16T00:43:00.744 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:43:00.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.745+0000 7f936dcbe640 1 -- 172.21.15.112:0/1770078412 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f935006e530 msgr2=0x7f93500709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:00.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.745+0000 7f936dcbe640 1 --2- 172.21.15.112:0/1770078412 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f935006e530 0x7f93500709e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9358004ed0 tx=0x7f935800b040 comp rx=0 tx=0).stop 2024-09-16T00:43:00.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.745+0000 7f936dcbe640 1 -- 172.21.15.112:0/1770078412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93681014e0 msgr2=0x7f9368115960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:00.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.745+0000 7f936dcbe640 1 --2- 172.21.15.112:0/1770078412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93681014e0 0x7f9368115960 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f935c0043d0 tx=0x7f935c009da0 comp rx=0 tx=0).stop 2024-09-16T00:43:00.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.745+0000 7f936dcbe640 1 -- 172.21.15.112:0/1770078412 shutdown_connections 2024-09-16T00:43:00.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.745+0000 7f936dcbe640 1 --2- 172.21.15.112:0/1770078412 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f935006e530 0x7f93500709e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:00.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.745+0000 7f936dcbe640 1 --2- 172.21.15.112:0/1770078412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93681014e0 0x7f9368115960 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:00.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.745+0000 7f936dcbe640 1 --2- 172.21.15.112:0/1770078412 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93680fb4a0 0x7f9368115420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:00.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.745+0000 7f936dcbe640 1 -- 172.21.15.112:0/1770078412 >> 172.21.15.112:0/1770078412 conn(0x7f93680f7060 msgr2=0x7f93680f7ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:00.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.745+0000 7f936dcbe640 1 -- 172.21.15.112:0/1770078412 shutdown_connections 2024-09-16T00:43:00.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.745+0000 7f936dcbe640 1 -- 172.21.15.112:0/1770078412 wait complete. 2024-09-16T00:43:00.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.824+0000 7f4ce6e31640 1 -- 172.21.15.112:0/3044969355 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ce00fdf00 msgr2=0x7f4ce01065d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:00.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.824+0000 7f4ce6e31640 1 --2- 172.21.15.112:0/3044969355 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ce00fdf00 0x7f4ce01065d0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0004900 tx=0x7f4cd0009da0 comp rx=0 tx=0).stop 2024-09-16T00:43:00.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.824+0000 7f4ce6e31640 1 -- 172.21.15.112:0/3044969355 shutdown_connections 2024-09-16T00:43:00.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.824+0000 7f4ce6e31640 1 --2- 172.21.15.112:0/3044969355 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ce00fdf00 0x7f4ce01065d0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:00.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.824+0000 7f4ce6e31640 1 --2- 172.21.15.112:0/3044969355 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ce00fd560 0x7f4ce00fd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:00.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.824+0000 7f4ce6e31640 1 -- 172.21.15.112:0/3044969355 >> 172.21.15.112:0/3044969355 conn(0x7f4ce00f7060 msgr2=0x7f4ce00f9470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:00.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.824+0000 7f4ce6e31640 1 -- 172.21.15.112:0/3044969355 shutdown_connections 2024-09-16T00:43:00.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.824+0000 7f4ce6e31640 1 -- 172.21.15.112:0/3044969355 wait complete. 2024-09-16T00:43:00.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.825+0000 7f4ce6e31640 1 Processor -- start 2024-09-16T00:43:00.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.825+0000 7f4ce6e31640 1 -- start start 2024-09-16T00:43:00.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.825+0000 7f4ce6e31640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ce00fd560 0x7f4ce00fb420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:00.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.825+0000 7f4ce6e31640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ce00fdf00 0x7f4ce00fb960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:00.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.825+0000 7f4ce6e31640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce00fbea0 con 0x7f4ce00fdf00 2024-09-16T00:43:00.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.825+0000 7f4ce6e31640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce00fc010 con 0x7f4ce00fd560 2024-09-16T00:43:00.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.826+0000 7f4cd7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ce00fdf00 0x7f4ce00fb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:00.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.826+0000 7f4cd7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ce00fdf00 0x7f4ce00fb960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54102/0 (socket says 172.21.15.112:54102) 2024-09-16T00:43:00.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.826+0000 7f4cd7fff640 1 -- 172.21.15.112:0/578833303 learned_addr learned my addr 172.21.15.112:0/578833303 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:43:00.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.826+0000 7f4ce4bcd640 1 --2- 172.21.15.112:0/578833303 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ce00fd560 0x7f4ce00fb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:00.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.826+0000 7f4cd7fff640 1 -- 172.21.15.112:0/578833303 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ce00fd560 msgr2=0x7f4ce00fb420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:00.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.826+0000 7f4cd7fff640 1 --2- 172.21.15.112:0/578833303 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ce00fd560 0x7f4ce00fb420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:00.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.826+0000 7f4cd7fff640 1 -- 172.21.15.112:0/578833303 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cd0005040 con 0x7f4ce00fdf00 2024-09-16T00:43:00.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.826+0000 7f4cd7fff640 1 --2- 172.21.15.112:0/578833303 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ce00fdf00 0x7f4ce00fb960 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0004960 tx=0x7f4cd0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:00.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.826+0000 7f4cd57fa640 1 -- 172.21.15.112:0/578833303 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4cd0038070 con 0x7f4ce00fdf00 2024-09-16T00:43:00.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.826+0000 7f4cd57fa640 1 -- 172.21.15.112:0/578833303 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f4cd0008ed0 con 0x7f4ce00fdf00 2024-09-16T00:43:00.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.826+0000 7f4ce6e31640 1 -- 172.21.15.112:0/578833303 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ce00fc240 con 0x7f4ce00fdf00 2024-09-16T00:43:00.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.826+0000 7f4cd57fa640 1 -- 172.21.15.112:0/578833303 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4cd00095a0 con 0x7f4ce00fdf00 2024-09-16T00:43:00.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.826+0000 7f4ce6e31640 1 -- 172.21.15.112:0/578833303 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ce01198b0 con 0x7f4ce00fdf00 2024-09-16T00:43:00.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.827+0000 7f4ce6e31640 1 -- 172.21.15.112:0/578833303 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ce00662f0 con 0x7f4ce00fdf00 2024-09-16T00:43:00.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.828+0000 7f4cd57fa640 1 -- 172.21.15.112:0/578833303 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f4cd0030020 con 0x7f4ce00fdf00 2024-09-16T00:43:00.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.828+0000 7f4cd57fa640 1 --2- 172.21.15.112:0/578833303 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f4cc806e530 0x7f4cc80709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:00.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.828+0000 7f4cd57fa640 1 -- 172.21.15.112:0/578833303 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f4cd00ae1a0 con 0x7f4ce00fdf00 2024-09-16T00:43:00.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.828+0000 7f4ce4bcd640 1 --2- 172.21.15.112:0/578833303 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f4cc806e530 0x7f4cc80709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:00.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.829+0000 7f4ce4bcd640 1 --2- 172.21.15.112:0/578833303 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f4cc806e530 0x7f4cc80709e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc0041d0 tx=0x7f4cdc009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:00.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:00.833+0000 7f4cd57fa640 1 -- 172.21.15.112:0/578833303 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f4cd0077190 con 0x7f4ce00fdf00 2024-09-16T00:43:01.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:01.062+0000 7f4ce6e31640 1 -- 172.21.15.112:0/578833303 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f4ce004b720 con 0x7f4ce00fdf00 2024-09-16T00:43:01.063 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:43:00 smithi112 ceph-mon[97750]: from='client.34884 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:43:01.063 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:43:00 smithi112 ceph-mon[97750]: from='client.34888 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:43:01.063 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:43:00 smithi112 ceph-mon[97750]: from='client.34892 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:43:01.063 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:43:00 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/4255386578' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:43:01.063 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:43:00 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/2323058429' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:43:01.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:01.062+0000 7f4cd57fa640 1 -- 172.21.15.112:0/578833303 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f4cd007a850 con 0x7f4ce00fdf00 2024-09-16T00:43:01.063 INFO:teuthology.orchestra.run.smithi112.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-16T00:43:01.063 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-16T00:43:01.064 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs is degraded 2024-09-16T00:43:01.064 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T00:43:01.064 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T00:43:01.064 INFO:teuthology.orchestra.run.smithi112.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-16T00:43:01.064 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-16T00:43:01.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:01.064+0000 7f4ce6e31640 1 -- 172.21.15.112:0/578833303 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f4cc806e530 msgr2=0x7f4cc80709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:01.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:01.064+0000 7f4ce6e31640 1 --2- 172.21.15.112:0/578833303 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f4cc806e530 0x7f4cc80709e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc0041d0 tx=0x7f4cdc009040 comp rx=0 tx=0).stop 2024-09-16T00:43:01.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:01.064+0000 7f4ce6e31640 1 -- 172.21.15.112:0/578833303 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ce00fdf00 msgr2=0x7f4ce00fb960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:01.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:01.064+0000 7f4ce6e31640 1 --2- 172.21.15.112:0/578833303 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ce00fdf00 0x7f4ce00fb960 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0004960 tx=0x7f4cd0009da0 comp rx=0 tx=0).stop 2024-09-16T00:43:01.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:01.065+0000 7f4ce6e31640 1 -- 172.21.15.112:0/578833303 shutdown_connections 2024-09-16T00:43:01.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:01.065+0000 7f4ce6e31640 1 --2- 172.21.15.112:0/578833303 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f4cc806e530 0x7f4cc80709e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:01.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:01.065+0000 7f4ce6e31640 1 --2- 172.21.15.112:0/578833303 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ce00fdf00 0x7f4ce00fb960 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:01.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:01.065+0000 7f4ce6e31640 1 --2- 172.21.15.112:0/578833303 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ce00fd560 0x7f4ce00fb420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:01.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:01.065+0000 7f4ce6e31640 1 -- 172.21.15.112:0/578833303 >> 172.21.15.112:0/578833303 conn(0x7f4ce00f7060 msgr2=0x7f4ce00f85a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:01.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:01.065+0000 7f4ce6e31640 1 -- 172.21.15.112:0/578833303 shutdown_connections 2024-09-16T00:43:01.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:01.065+0000 7f4ce6e31640 1 -- 172.21.15.112:0/578833303 wait complete. 2024-09-16T00:43:01.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:43:00 smithi169 ceph-mon[77626]: from='client.34884 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:43:01.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:43:00 smithi169 ceph-mon[77626]: from='client.34888 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:43:01.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:43:00 smithi169 ceph-mon[77626]: from='client.34892 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:43:01.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:43:00 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/4255386578' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:43:01.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:43:00 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/2323058429' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:43:02.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:43:01 smithi112 ceph-mon[97750]: pgmap v458: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:43:02.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:43:01 smithi112 ceph-mon[97750]: from='client.34904 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:43:02.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:43:01 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/578833303' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:43:02.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:43:01 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:43:02.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:43:01 smithi169 ceph-mon[77626]: pgmap v458: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:43:02.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:43:01 smithi169 ceph-mon[77626]: from='client.34904 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:43:02.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:43:01 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/578833303' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:43:02.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:43:01 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:43:04.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:43:03 smithi112 ceph-mon[97750]: pgmap v459: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:43:04.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:43:03 smithi169 ceph-mon[77626]: pgmap v459: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:43:06.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:43:05 smithi112 ceph-mon[97750]: pgmap v460: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:43:06.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:43:05 smithi169 ceph-mon[77626]: pgmap v460: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:43:08.088 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:43:07 smithi112 ceph-mon[97750]: pgmap v461: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:43:08.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:43:07 smithi169 ceph-mon[77626]: pgmap v461: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:43:10.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:43:09 smithi112 ceph-mon[97750]: pgmap v462: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:43:10.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:43:09 smithi169 ceph-mon[77626]: pgmap v462: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:43:12.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:43:11 smithi112 ceph-mon[97750]: pgmap v463: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:43:12.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:43:11 smithi169 ceph-mon[77626]: pgmap v463: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:43:14.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:43:13 smithi112 ceph-mon[97750]: pgmap v464: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:43:14.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:43:13 smithi169 ceph-mon[77626]: pgmap v464: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:43:16.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:43:15 smithi112 ceph-mon[97750]: pgmap v465: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:43:16.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:43:15 smithi169 ceph-mon[77626]: pgmap v465: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:43:17.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:43:16 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:43:17.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:43:16 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:43:18.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:43:17 smithi112 ceph-mon[97750]: pgmap v466: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:43:18.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:43:17 smithi169 ceph-mon[77626]: pgmap v466: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:43:20.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:43:19 smithi112 ceph-mon[97750]: pgmap v467: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:43:20.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:43:19 smithi169 ceph-mon[77626]: pgmap v467: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:43:22.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:43:21 smithi112 ceph-mon[97750]: pgmap v468: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:43:22.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:43:21 smithi169 ceph-mon[77626]: pgmap v468: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:43:24.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:43:23 smithi112 ceph-mon[97750]: pgmap v469: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:43:24.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:43:23 smithi169 ceph-mon[77626]: pgmap v469: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:43:26.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:43:25 smithi169 ceph-mon[77626]: pgmap v470: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:43:26.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:43:25 smithi112 ceph-mon[97750]: pgmap v470: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:43:28.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:43:27 smithi169 ceph-mon[77626]: pgmap v471: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:43:28.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:43:27 smithi112 ceph-mon[97750]: pgmap v471: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:43:30.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:43:29 smithi169 ceph-mon[77626]: pgmap v472: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:43:30.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:43:29 smithi112 ceph-mon[97750]: pgmap v472: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:43:31.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.155+0000 7f731e7da640 1 -- 172.21.15.112:0/1351509579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7318102550 msgr2=0x7f7318102920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:31.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.155+0000 7f731e7da640 1 --2- 172.21.15.112:0/1351509579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7318102550 0x7f7318102920 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f73080048d0 tx=0x7f7308009da0 comp rx=0 tx=0).stop 2024-09-16T00:43:31.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.155+0000 7f731e7da640 1 -- 172.21.15.112:0/1351509579 shutdown_connections 2024-09-16T00:43:31.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.155+0000 7f731e7da640 1 --2- 172.21.15.112:0/1351509579 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7318102ef0 0x7f73180fc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:31.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.155+0000 7f731e7da640 1 --2- 172.21.15.112:0/1351509579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7318102550 0x7f7318102920 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:31.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.155+0000 7f731e7da640 1 -- 172.21.15.112:0/1351509579 >> 172.21.15.112:0/1351509579 conn(0x7f73180f80f0 msgr2=0x7f73180fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:31.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.155+0000 7f731e7da640 1 -- 172.21.15.112:0/1351509579 shutdown_connections 2024-09-16T00:43:31.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.155+0000 7f731e7da640 1 -- 172.21.15.112:0/1351509579 wait complete. 2024-09-16T00:43:31.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.156+0000 7f731e7da640 1 Processor -- start 2024-09-16T00:43:31.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.156+0000 7f731e7da640 1 -- start start 2024-09-16T00:43:31.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.156+0000 7f731e7da640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7318102550 0x7f7318113570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:31.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.156+0000 7f731e7da640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7318102ef0 0x7f7318113ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:31.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.156+0000 7f731e7da640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7318114100 con 0x7f7318102ef0 2024-09-16T00:43:31.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.156+0000 7f731e7da640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f731810d510 con 0x7f7318102550 2024-09-16T00:43:31.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.156+0000 7f73177fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7318102ef0 0x7f7318113ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:31.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.156+0000 7f73177fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7318102ef0 0x7f7318113ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42802/0 (socket says 172.21.15.112:42802) 2024-09-16T00:43:31.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.156+0000 7f73177fe640 1 -- 172.21.15.112:0/2912711068 learned_addr learned my addr 172.21.15.112:0/2912711068 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:43:31.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.157+0000 7f7317fff640 1 --2- 172.21.15.112:0/2912711068 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7318102550 0x7f7318113570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:31.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.157+0000 7f73177fe640 1 -- 172.21.15.112:0/2912711068 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7318102550 msgr2=0x7f7318113570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:31.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.157+0000 7f73177fe640 1 --2- 172.21.15.112:0/2912711068 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7318102550 0x7f7318113570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:31.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.157+0000 7f73177fe640 1 -- 172.21.15.112:0/2912711068 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7308005040 con 0x7f7318102ef0 2024-09-16T00:43:31.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.157+0000 7f7317fff640 1 --2- 172.21.15.112:0/2912711068 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7318102550 0x7f7318113570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:43:31.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.157+0000 7f73177fe640 1 --2- 172.21.15.112:0/2912711068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7318102ef0 0x7f7318113ab0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f730c00c760 tx=0x7f730c00cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:31.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.157+0000 7f7314ff9640 1 -- 172.21.15.112:0/2912711068 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f730c00a820 con 0x7f7318102ef0 2024-09-16T00:43:31.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.157+0000 7f7314ff9640 1 -- 172.21.15.112:0/2912711068 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f730c00ae20 con 0x7f7318102ef0 2024-09-16T00:43:31.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.157+0000 7f7314ff9640 1 -- 172.21.15.112:0/2912711068 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f730c0113a0 con 0x7f7318102ef0 2024-09-16T00:43:31.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.157+0000 7f731e7da640 1 -- 172.21.15.112:0/2912711068 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f731810d7f0 con 0x7f7318102ef0 2024-09-16T00:43:31.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.157+0000 7f731e7da640 1 -- 172.21.15.112:0/2912711068 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f731810dd90 con 0x7f7318102ef0 2024-09-16T00:43:31.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.158+0000 7f731e7da640 1 -- 172.21.15.112:0/2912711068 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73180fdcd0 con 0x7f7318102ef0 2024-09-16T00:43:31.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.159+0000 7f7314ff9640 1 -- 172.21.15.112:0/2912711068 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f730c012790 con 0x7f7318102ef0 2024-09-16T00:43:31.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.159+0000 7f7314ff9640 1 --2- 172.21.15.112:0/2912711068 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f730006e530 0x7f73000709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:31.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.160+0000 7f7314ff9640 1 -- 172.21.15.112:0/2912711068 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f730c08e3e0 con 0x7f7318102ef0 2024-09-16T00:43:31.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.160+0000 7f7317fff640 1 --2- 172.21.15.112:0/2912711068 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f730006e530 0x7f73000709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:31.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.160+0000 7f7317fff640 1 --2- 172.21.15.112:0/2912711068 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f730006e530 0x7f73000709e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7308007ac0 tx=0x7f7308035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:31.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.164+0000 7f7314ff9640 1 -- 172.21.15.112:0/2912711068 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f730c057470 con 0x7f7318102ef0 2024-09-16T00:43:31.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.346+0000 7f731e7da640 1 -- 172.21.15.112:0/2912711068 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f73181095a0 con 0x7f730006e530 2024-09-16T00:43:31.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.348+0000 7f7314ff9640 1 -- 172.21.15.112:0/2912711068 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f73181095a0 con 0x7f730006e530 2024-09-16T00:43:31.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.350+0000 7f731e7da640 1 -- 172.21.15.112:0/2912711068 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f730006e530 msgr2=0x7f73000709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:31.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.350+0000 7f731e7da640 1 --2- 172.21.15.112:0/2912711068 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f730006e530 0x7f73000709e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7308007ac0 tx=0x7f7308035040 comp rx=0 tx=0).stop 2024-09-16T00:43:31.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.350+0000 7f731e7da640 1 -- 172.21.15.112:0/2912711068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7318102ef0 msgr2=0x7f7318113ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:31.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.350+0000 7f731e7da640 1 --2- 172.21.15.112:0/2912711068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7318102ef0 0x7f7318113ab0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f730c00c760 tx=0x7f730c00cb60 comp rx=0 tx=0).stop 2024-09-16T00:43:31.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.350+0000 7f731e7da640 1 -- 172.21.15.112:0/2912711068 shutdown_connections 2024-09-16T00:43:31.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.350+0000 7f731e7da640 1 --2- 172.21.15.112:0/2912711068 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f730006e530 0x7f73000709e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:31.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.350+0000 7f731e7da640 1 --2- 172.21.15.112:0/2912711068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7318102ef0 0x7f7318113ab0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:31.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.350+0000 7f731e7da640 1 --2- 172.21.15.112:0/2912711068 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7318102550 0x7f7318113570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:31.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.350+0000 7f731e7da640 1 -- 172.21.15.112:0/2912711068 >> 172.21.15.112:0/2912711068 conn(0x7f73180f80f0 msgr2=0x7f73180f90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:31.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.350+0000 7f731e7da640 1 -- 172.21.15.112:0/2912711068 shutdown_connections 2024-09-16T00:43:31.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.350+0000 7f731e7da640 1 -- 172.21.15.112:0/2912711068 wait complete. 2024-09-16T00:43:31.359 INFO:teuthology.orchestra.run.smithi112.stdout:true 2024-09-16T00:43:31.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.430+0000 7fd886313640 1 -- 172.21.15.112:0/2328830347 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd880102550 msgr2=0x7fd880102920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:31.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.430+0000 7fd886313640 1 --2- 172.21.15.112:0/2328830347 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd880102550 0x7fd880102920 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd8700048d0 tx=0x7fd870009da0 comp rx=0 tx=0).stop 2024-09-16T00:43:31.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.430+0000 7fd886313640 1 -- 172.21.15.112:0/2328830347 shutdown_connections 2024-09-16T00:43:31.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.430+0000 7fd886313640 1 --2- 172.21.15.112:0/2328830347 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd880102ef0 0x7fd8800fc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:31.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.430+0000 7fd886313640 1 --2- 172.21.15.112:0/2328830347 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd880102550 0x7fd880102920 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:31.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.430+0000 7fd886313640 1 -- 172.21.15.112:0/2328830347 >> 172.21.15.112:0/2328830347 conn(0x7fd8800f80f0 msgr2=0x7fd8800fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:31.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.430+0000 7fd886313640 1 -- 172.21.15.112:0/2328830347 shutdown_connections 2024-09-16T00:43:31.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.430+0000 7fd886313640 1 -- 172.21.15.112:0/2328830347 wait complete. 2024-09-16T00:43:31.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.431+0000 7fd886313640 1 Processor -- start 2024-09-16T00:43:31.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.431+0000 7fd886313640 1 -- start start 2024-09-16T00:43:31.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.431+0000 7fd886313640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd880102550 0x7fd880113570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:31.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.431+0000 7fd886313640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd880102ef0 0x7fd880113ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:31.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.431+0000 7fd886313640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd880114100 con 0x7fd880102ef0 2024-09-16T00:43:31.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.431+0000 7fd886313640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88010d510 con 0x7fd880102550 2024-09-16T00:43:31.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.431+0000 7fd877fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd880102ef0 0x7fd880113ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:31.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.431+0000 7fd877fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd880102ef0 0x7fd880113ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42812/0 (socket says 172.21.15.112:42812) 2024-09-16T00:43:31.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.431+0000 7fd877fff640 1 -- 172.21.15.112:0/2966329620 learned_addr learned my addr 172.21.15.112:0/2966329620 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:43:31.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.431+0000 7fd87ffff640 1 --2- 172.21.15.112:0/2966329620 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd880102550 0x7fd880113570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:31.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.432+0000 7fd877fff640 1 -- 172.21.15.112:0/2966329620 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd880102550 msgr2=0x7fd880113570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:31.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.432+0000 7fd877fff640 1 --2- 172.21.15.112:0/2966329620 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd880102550 0x7fd880113570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:31.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.432+0000 7fd877fff640 1 -- 172.21.15.112:0/2966329620 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd870005040 con 0x7fd880102ef0 2024-09-16T00:43:31.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.432+0000 7fd87ffff640 1 --2- 172.21.15.112:0/2966329620 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd880102550 0x7fd880113570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:43:31.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.432+0000 7fd877fff640 1 --2- 172.21.15.112:0/2966329620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd880102ef0 0x7fd880113ab0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd868009af0 tx=0x7fd868004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:31.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.432+0000 7fd87d7fa640 1 -- 172.21.15.112:0/2966329620 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd868007c30 con 0x7fd880102ef0 2024-09-16T00:43:31.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.432+0000 7fd87d7fa640 1 -- 172.21.15.112:0/2966329620 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fd86800f040 con 0x7fd880102ef0 2024-09-16T00:43:31.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.432+0000 7fd886313640 1 -- 172.21.15.112:0/2966329620 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd88010d7f0 con 0x7fd880102ef0 2024-09-16T00:43:31.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.432+0000 7fd87d7fa640 1 -- 172.21.15.112:0/2966329620 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd8680094b0 con 0x7fd880102ef0 2024-09-16T00:43:31.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.432+0000 7fd886313640 1 -- 172.21.15.112:0/2966329620 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd88010dd90 con 0x7fd880102ef0 2024-09-16T00:43:31.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.433+0000 7fd886313640 1 -- 172.21.15.112:0/2966329620 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8800fdcd0 con 0x7fd880102ef0 2024-09-16T00:43:31.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.434+0000 7fd87d7fa640 1 -- 172.21.15.112:0/2966329620 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fd868009730 con 0x7fd880102ef0 2024-09-16T00:43:31.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.434+0000 7fd87d7fa640 1 --2- 172.21.15.112:0/2966329620 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd86c06e530 0x7fd86c0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:31.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.434+0000 7fd87d7fa640 1 -- 172.21.15.112:0/2966329620 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7fd86808e2f0 con 0x7fd880102ef0 2024-09-16T00:43:31.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.434+0000 7fd87ffff640 1 --2- 172.21.15.112:0/2966329620 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd86c06e530 0x7fd86c0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:31.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.435+0000 7fd87ffff640 1 --2- 172.21.15.112:0/2966329620 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd86c06e530 0x7fd86c0709e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd870007ac0 tx=0x7fd870035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:31.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.438+0000 7fd87d7fa640 1 -- 172.21.15.112:0/2966329620 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fd868057260 con 0x7fd880102ef0 2024-09-16T00:43:31.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.622+0000 7fd886313640 1 -- 172.21.15.112:0/2966329620 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd880113260 con 0x7fd86c06e530 2024-09-16T00:43:31.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.623+0000 7fd87d7fa640 1 -- 172.21.15.112:0/2966329620 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fd880113260 con 0x7fd86c06e530 2024-09-16T00:43:31.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.625+0000 7fd886313640 1 -- 172.21.15.112:0/2966329620 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd86c06e530 msgr2=0x7fd86c0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:31.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.625+0000 7fd886313640 1 --2- 172.21.15.112:0/2966329620 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd86c06e530 0x7fd86c0709e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd870007ac0 tx=0x7fd870035040 comp rx=0 tx=0).stop 2024-09-16T00:43:31.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.625+0000 7fd886313640 1 -- 172.21.15.112:0/2966329620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd880102ef0 msgr2=0x7fd880113ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:31.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.625+0000 7fd886313640 1 --2- 172.21.15.112:0/2966329620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd880102ef0 0x7fd880113ab0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd868009af0 tx=0x7fd868004570 comp rx=0 tx=0).stop 2024-09-16T00:43:31.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.625+0000 7fd886313640 1 -- 172.21.15.112:0/2966329620 shutdown_connections 2024-09-16T00:43:31.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.625+0000 7fd886313640 1 --2- 172.21.15.112:0/2966329620 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd86c06e530 0x7fd86c0709e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:31.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.625+0000 7fd886313640 1 --2- 172.21.15.112:0/2966329620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd880102ef0 0x7fd880113ab0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:31.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.625+0000 7fd886313640 1 --2- 172.21.15.112:0/2966329620 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd880102550 0x7fd880113570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:31.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.625+0000 7fd886313640 1 -- 172.21.15.112:0/2966329620 >> 172.21.15.112:0/2966329620 conn(0x7fd8800f80f0 msgr2=0x7fd8800f90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:31.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.626+0000 7fd886313640 1 -- 172.21.15.112:0/2966329620 shutdown_connections 2024-09-16T00:43:31.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.626+0000 7fd886313640 1 -- 172.21.15.112:0/2966329620 wait complete. 2024-09-16T00:43:31.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.703+0000 7f54ecbf3640 1 -- 172.21.15.112:0/50678761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54e80fcf80 msgr2=0x7f54e80fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:31.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.703+0000 7f54ecbf3640 1 --2- 172.21.15.112:0/50678761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54e80fcf80 0x7f54e80fd3d0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f54dc0048d0 tx=0x7f54dc009da0 comp rx=0 tx=0).stop 2024-09-16T00:43:31.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.704+0000 7f54ecbf3640 1 -- 172.21.15.112:0/50678761 shutdown_connections 2024-09-16T00:43:31.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.704+0000 7f54ecbf3640 1 --2- 172.21.15.112:0/50678761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54e80fcf80 0x7f54e80fd3d0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:31.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.704+0000 7f54ecbf3640 1 --2- 172.21.15.112:0/50678761 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54e80fb810 0x7f54e80fbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:31.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.704+0000 7f54ecbf3640 1 -- 172.21.15.112:0/50678761 >> 172.21.15.112:0/50678761 conn(0x7f54e80723b0 msgr2=0x7f54e80727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:31.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.704+0000 7f54ecbf3640 1 -- 172.21.15.112:0/50678761 shutdown_connections 2024-09-16T00:43:31.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.704+0000 7f54ecbf3640 1 -- 172.21.15.112:0/50678761 wait complete. 2024-09-16T00:43:31.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.704+0000 7f54ecbf3640 1 Processor -- start 2024-09-16T00:43:31.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.704+0000 7f54ecbf3640 1 -- start start 2024-09-16T00:43:31.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.704+0000 7f54ecbf3640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54e80fb810 0x7f54e80f8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:31.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.704+0000 7f54ecbf3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54e80fcf80 0x7f54e80f8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:31.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.704+0000 7f54ecbf3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e80f8fc0 con 0x7f54e80fcf80 2024-09-16T00:43:31.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.704+0000 7f54ecbf3640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e80f9130 con 0x7f54e80fb810 2024-09-16T00:43:31.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.705+0000 7f54e5d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54e80fcf80 0x7f54e80f8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:31.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.705+0000 7f54e5d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54e80fcf80 0x7f54e80f8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42832/0 (socket says 172.21.15.112:42832) 2024-09-16T00:43:31.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.705+0000 7f54e5d9b640 1 -- 172.21.15.112:0/74518244 learned_addr learned my addr 172.21.15.112:0/74518244 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:43:31.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.705+0000 7f54e659c640 1 --2- 172.21.15.112:0/74518244 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54e80fb810 0x7f54e80f8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:31.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.705+0000 7f54e5d9b640 1 -- 172.21.15.112:0/74518244 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54e80fb810 msgr2=0x7f54e80f8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:31.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.705+0000 7f54e5d9b640 1 --2- 172.21.15.112:0/74518244 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54e80fb810 0x7f54e80f8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:31.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.705+0000 7f54e5d9b640 1 -- 172.21.15.112:0/74518244 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54dc005040 con 0x7f54e80fcf80 2024-09-16T00:43:31.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.705+0000 7f54e659c640 1 --2- 172.21.15.112:0/74518244 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54e80fb810 0x7f54e80f8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:43:31.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.705+0000 7f54e5d9b640 1 --2- 172.21.15.112:0/74518244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54e80fcf80 0x7f54e80f8a80 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f54dc003080 tx=0x7f54dc009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:31.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.705+0000 7f54d6ffd640 1 -- 172.21.15.112:0/74518244 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f54dc038070 con 0x7f54e80fcf80 2024-09-16T00:43:31.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.705+0000 7f54d6ffd640 1 -- 172.21.15.112:0/74518244 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f54dc02ade0 con 0x7f54e80fcf80 2024-09-16T00:43:31.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.705+0000 7f54d6ffd640 1 -- 172.21.15.112:0/74518244 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f54dc02c880 con 0x7f54e80fcf80 2024-09-16T00:43:31.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.705+0000 7f54ecbf3640 1 -- 172.21.15.112:0/74518244 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54e80f9360 con 0x7f54e80fcf80 2024-09-16T00:43:31.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.705+0000 7f54ecbf3640 1 -- 172.21.15.112:0/74518244 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54e8119770 con 0x7f54e80fcf80 2024-09-16T00:43:31.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.707+0000 7f54ecbf3640 1 -- 172.21.15.112:0/74518244 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54e80fcbc0 con 0x7f54e80fcf80 2024-09-16T00:43:31.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.707+0000 7f54d6ffd640 1 -- 172.21.15.112:0/74518244 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f54dc030020 con 0x7f54e80fcf80 2024-09-16T00:43:31.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.707+0000 7f54d6ffd640 1 --2- 172.21.15.112:0/74518244 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f54d006e530 0x7f54d00709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:31.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.708+0000 7f54d6ffd640 1 -- 172.21.15.112:0/74518244 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f54dc0ae410 con 0x7f54e80fcf80 2024-09-16T00:43:31.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.708+0000 7f54e659c640 1 --2- 172.21.15.112:0/74518244 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f54d006e530 0x7f54d00709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:31.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.708+0000 7f54e659c640 1 --2- 172.21.15.112:0/74518244 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f54d006e530 0x7f54d00709e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f54d80041d0 tx=0x7f54d8009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:31.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.712+0000 7f54d6ffd640 1 -- 172.21.15.112:0/74518244 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f54dc0772e0 con 0x7f54e80fcf80 2024-09-16T00:43:31.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.886+0000 7f54ecbf3640 1 -- 172.21.15.112:0/74518244 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f54e804b720 con 0x7f54d006e530 2024-09-16T00:43:31.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.895+0000 7f54d6ffd640 1 -- 172.21.15.112:0/74518244 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7f54e804b720 con 0x7f54d006e530 2024-09-16T00:43:31.895 INFO:teuthology.orchestra.run.smithi112.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T00:43:31.895 INFO:teuthology.orchestra.run.smithi112.stdout:alertmanager.smithi112 smithi112 *:9093,9094 running (16m) 9m ago 25m 24.7M - 0.25.0 c8568f914cd2 d85ea0562022 2024-09-16T00:43:31.895 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi112 smithi112 running (13m) 9m ago 25m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c e4fe58f9702a 2024-09-16T00:43:31.895 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi169 smithi169 running (13m) 9m ago 24m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c 3ba2c81d527f 2024-09-16T00:43:31.895 INFO:teuthology.orchestra.run.smithi112.stdout:grafana.smithi112 smithi112 *:3000 running (15m) 9m ago 25m 82.1M - 9.4.12 2bacad6d85d8 51f9d2013da2 2024-09-16T00:43:31.895 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.hxfwkh smithi112 running (9m) 9m ago 19m 14.2M - 19.1.1-326-g8921c8ef f1455bd0219c 1723069902d6 2024-09-16T00:43:31.896 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.ithwzj smithi112 running (9m) 9m ago 20m 12.2M - 19.1.1-326-g8921c8ef f1455bd0219c e7aad5b81db5 2024-09-16T00:43:31.896 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.gawwgw smithi169 running (9m) 9m ago 20m 15.7M - 19.1.1-326-g8921c8ef f1455bd0219c 469be459ee12 2024-09-16T00:43:31.896 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.lgeqrj smithi169 running (9m) 9m ago 19m 13.3M - 19.1.1-326-g8921c8ef f1455bd0219c 31f0fbd2395d 2024-09-16T00:43:31.896 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi112.xrkkbc smithi112 *:8443,9283,8765 running (16m) 9m ago 26m 504M - 19.1.1-326-g8921c8ef f1455bd0219c a981f7fd2810 2024-09-16T00:43:31.896 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi169.mtrobk smithi169 *:8443,9283,8765 running (15m) 9m ago 24m 442M - 19.1.1-326-g8921c8ef f1455bd0219c 4096e5369c2d 2024-09-16T00:43:31.896 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi112 smithi112 running (14m) 9m ago 26m 49.8M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 5aaec033c042 2024-09-16T00:43:31.896 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi169 smithi169 running (14m) 9m ago 24m 38.9M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 7bc609155f3f 2024-09-16T00:43:31.896 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi112 smithi112 *:9100 running (16m) 9m ago 25m 15.5M - 1.5.0 0da6a335fe13 6404d1d601e2 2024-09-16T00:43:31.896 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi169 smithi169 *:9100 running (16m) 9m ago 24m 15.4M - 1.5.0 0da6a335fe13 5505baf702a6 2024-09-16T00:43:31.896 INFO:teuthology.orchestra.run.smithi112.stdout:osd.0 smithi112 running (13m) 9m ago 23m 105M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 53efc9397017 2024-09-16T00:43:31.896 INFO:teuthology.orchestra.run.smithi112.stdout:osd.1 smithi112 running (12m) 9m ago 23m 98.9M 1557M 19.1.1-326-g8921c8ef f1455bd0219c fe6f1a4a2cf4 2024-09-16T00:43:31.896 INFO:teuthology.orchestra.run.smithi112.stdout:osd.2 smithi112 running (11m) 9m ago 22m 102M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 5f7863712057 2024-09-16T00:43:31.896 INFO:teuthology.orchestra.run.smithi112.stdout:osd.3 smithi169 running (11m) 9m ago 22m 92.1M 2581M 19.1.1-326-g8921c8ef f1455bd0219c ee07b5577810 2024-09-16T00:43:31.896 INFO:teuthology.orchestra.run.smithi112.stdout:osd.4 smithi169 running (10m) 9m ago 22m 103M 2581M 19.1.1-326-g8921c8ef f1455bd0219c 769f026a54eb 2024-09-16T00:43:31.897 INFO:teuthology.orchestra.run.smithi112.stdout:osd.5 smithi169 running (10m) 9m ago 21m 125M 2581M 19.1.1-326-g8921c8ef f1455bd0219c a6007c13cf78 2024-09-16T00:43:31.897 INFO:teuthology.orchestra.run.smithi112.stdout:prometheus.smithi112 smithi112 *:9095 running (15m) 9m ago 25m 64.1M - 2.43.0 a07b618ecd1d 7a1d42e60d2b 2024-09-16T00:43:31.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.897+0000 7f54ecbf3640 1 -- 172.21.15.112:0/74518244 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f54d006e530 msgr2=0x7f54d00709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:31.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.897+0000 7f54ecbf3640 1 --2- 172.21.15.112:0/74518244 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f54d006e530 0x7f54d00709e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f54d80041d0 tx=0x7f54d8009040 comp rx=0 tx=0).stop 2024-09-16T00:43:31.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.897+0000 7f54ecbf3640 1 -- 172.21.15.112:0/74518244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54e80fcf80 msgr2=0x7f54e80f8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:31.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.897+0000 7f54ecbf3640 1 --2- 172.21.15.112:0/74518244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54e80fcf80 0x7f54e80f8a80 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f54dc003080 tx=0x7f54dc009da0 comp rx=0 tx=0).stop 2024-09-16T00:43:31.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.897+0000 7f54ecbf3640 1 -- 172.21.15.112:0/74518244 shutdown_connections 2024-09-16T00:43:31.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.897+0000 7f54ecbf3640 1 --2- 172.21.15.112:0/74518244 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f54d006e530 0x7f54d00709e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:31.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.897+0000 7f54ecbf3640 1 --2- 172.21.15.112:0/74518244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54e80fcf80 0x7f54e80f8a80 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:31.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.897+0000 7f54ecbf3640 1 --2- 172.21.15.112:0/74518244 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54e80fb810 0x7f54e80f8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:31.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.897+0000 7f54ecbf3640 1 -- 172.21.15.112:0/74518244 >> 172.21.15.112:0/74518244 conn(0x7f54e80723b0 msgr2=0x7f54e8065810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:31.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.897+0000 7f54ecbf3640 1 -- 172.21.15.112:0/74518244 shutdown_connections 2024-09-16T00:43:31.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.897+0000 7f54ecbf3640 1 -- 172.21.15.112:0/74518244 wait complete. 2024-09-16T00:43:31.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.974+0000 7f8654bdc640 1 -- 172.21.15.112:0/4163028801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8650102550 msgr2=0x7f8650102920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:31.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.974+0000 7f8654bdc640 1 --2- 172.21.15.112:0/4163028801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8650102550 0x7f8650102920 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f86400048d0 tx=0x7f8640009da0 comp rx=0 tx=0).stop 2024-09-16T00:43:31.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.974+0000 7f8654bdc640 1 -- 172.21.15.112:0/4163028801 shutdown_connections 2024-09-16T00:43:31.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.974+0000 7f8654bdc640 1 --2- 172.21.15.112:0/4163028801 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8650102ef0 0x7f86500fc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:31.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.974+0000 7f8654bdc640 1 --2- 172.21.15.112:0/4163028801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8650102550 0x7f8650102920 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:31.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.974+0000 7f8654bdc640 1 -- 172.21.15.112:0/4163028801 >> 172.21.15.112:0/4163028801 conn(0x7f86500f80f0 msgr2=0x7f86500fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:31.975 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:43:31 smithi112 ceph-mon[97750]: pgmap v473: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:43:31.975 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:43:31 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:43:31.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.974+0000 7f8654bdc640 1 -- 172.21.15.112:0/4163028801 shutdown_connections 2024-09-16T00:43:31.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.974+0000 7f8654bdc640 1 -- 172.21.15.112:0/4163028801 wait complete. 2024-09-16T00:43:31.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.975+0000 7f8654bdc640 1 Processor -- start 2024-09-16T00:43:31.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.975+0000 7f8654bdc640 1 -- start start 2024-09-16T00:43:31.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.975+0000 7f8654bdc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8650102550 0x7f865006fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:31.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.976+0000 7f8654bdc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8650102ef0 0x7f865006ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:31.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.976+0000 7f8654bdc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8650070490 con 0x7f8650102ef0 2024-09-16T00:43:31.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.976+0000 7f8654bdc640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8650070600 con 0x7f8650102550 2024-09-16T00:43:31.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.976+0000 7f8645d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8650102ef0 0x7f865006ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:31.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.976+0000 7f8645d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8650102ef0 0x7f865006ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42856/0 (socket says 172.21.15.112:42856) 2024-09-16T00:43:31.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.976+0000 7f8645d9b640 1 -- 172.21.15.112:0/376130301 learned_addr learned my addr 172.21.15.112:0/376130301 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:43:31.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.976+0000 7f864e59c640 1 --2- 172.21.15.112:0/376130301 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8650102550 0x7f865006fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:31.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.976+0000 7f8645d9b640 1 -- 172.21.15.112:0/376130301 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8650102550 msgr2=0x7f865006fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:31.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.976+0000 7f8645d9b640 1 --2- 172.21.15.112:0/376130301 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8650102550 0x7f865006fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:31.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.976+0000 7f8645d9b640 1 -- 172.21.15.112:0/376130301 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8640005040 con 0x7f8650102ef0 2024-09-16T00:43:31.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.976+0000 7f8645d9b640 1 --2- 172.21.15.112:0/376130301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8650102ef0 0x7f865006ff50 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f863800c790 tx=0x7f86380043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:31.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.976+0000 7f86477fe640 1 -- 172.21.15.112:0/376130301 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f863800a7a0 con 0x7f8650102ef0 2024-09-16T00:43:31.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.976+0000 7f8654bdc640 1 -- 172.21.15.112:0/376130301 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8650070890 con 0x7f8650102ef0 2024-09-16T00:43:31.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.977+0000 7f86477fe640 1 -- 172.21.15.112:0/376130301 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f863800ada0 con 0x7f8650102ef0 2024-09-16T00:43:31.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.977+0000 7f86477fe640 1 -- 172.21.15.112:0/376130301 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8638011380 con 0x7f8650102ef0 2024-09-16T00:43:31.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.977+0000 7f8654bdc640 1 -- 172.21.15.112:0/376130301 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8650119c70 con 0x7f8650102ef0 2024-09-16T00:43:31.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.978+0000 7f8654bdc640 1 -- 172.21.15.112:0/376130301 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86500fdcd0 con 0x7f8650102ef0 2024-09-16T00:43:31.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.978+0000 7f86477fe640 1 -- 172.21.15.112:0/376130301 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f863800a900 con 0x7f8650102ef0 2024-09-16T00:43:31.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.979+0000 7f86477fe640 1 --2- 172.21.15.112:0/376130301 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f863c06e530 0x7f863c0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:31.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.979+0000 7f86477fe640 1 -- 172.21.15.112:0/376130301 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f863808e700 con 0x7f8650102ef0 2024-09-16T00:43:31.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.979+0000 7f864e59c640 1 --2- 172.21.15.112:0/376130301 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f863c06e530 0x7f863c0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:31.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.979+0000 7f864e59c640 1 --2- 172.21.15.112:0/376130301 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f863c06e530 0x7f863c0709e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8640009ac0 tx=0x7f8640035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:31.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:31.983+0000 7f86477fe640 1 -- 172.21.15.112:0/376130301 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f86380576f0 con 0x7f8650102ef0 2024-09-16T00:43:32.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.215+0000 7f8654bdc640 1 -- 172.21.15.112:0/376130301 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f8650109620 con 0x7f8650102ef0 2024-09-16T00:43:32.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.216+0000 7f86477fe640 1 -- 172.21.15.112:0/376130301 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f863805adb0 con 0x7f8650102ef0 2024-09-16T00:43:32.216 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:43:32.216 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": { 2024-09-16T00:43:32.217 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:43:32.217 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:43:32.217 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": { 2024-09-16T00:43:32.217 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:43:32.217 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:43:32.217 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": { 2024-09-16T00:43:32.217 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 6 2024-09-16T00:43:32.217 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:43:32.217 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": { 2024-09-16T00:43:32.217 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 4 2024-09-16T00:43:32.217 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:43:32.217 INFO:teuthology.orchestra.run.smithi112.stdout: "overall": { 2024-09-16T00:43:32.217 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 14 2024-09-16T00:43:32.217 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-16T00:43:32.218 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:43:32.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.218+0000 7f8654bdc640 1 -- 172.21.15.112:0/376130301 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f863c06e530 msgr2=0x7f863c0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:32.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.218+0000 7f8654bdc640 1 --2- 172.21.15.112:0/376130301 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f863c06e530 0x7f863c0709e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8640009ac0 tx=0x7f8640035040 comp rx=0 tx=0).stop 2024-09-16T00:43:32.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.218+0000 7f8654bdc640 1 -- 172.21.15.112:0/376130301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8650102ef0 msgr2=0x7f865006ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:32.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.218+0000 7f8654bdc640 1 --2- 172.21.15.112:0/376130301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8650102ef0 0x7f865006ff50 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f863800c790 tx=0x7f86380043c0 comp rx=0 tx=0).stop 2024-09-16T00:43:32.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.218+0000 7f8654bdc640 1 -- 172.21.15.112:0/376130301 shutdown_connections 2024-09-16T00:43:32.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.218+0000 7f8654bdc640 1 --2- 172.21.15.112:0/376130301 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f863c06e530 0x7f863c0709e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:32.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.218+0000 7f8654bdc640 1 --2- 172.21.15.112:0/376130301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8650102ef0 0x7f865006ff50 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:32.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.218+0000 7f8654bdc640 1 --2- 172.21.15.112:0/376130301 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8650102550 0x7f865006fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:32.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.218+0000 7f8654bdc640 1 -- 172.21.15.112:0/376130301 >> 172.21.15.112:0/376130301 conn(0x7f86500f80f0 msgr2=0x7f86500f9160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:32.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.218+0000 7f8654bdc640 1 -- 172.21.15.112:0/376130301 shutdown_connections 2024-09-16T00:43:32.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.218+0000 7f8654bdc640 1 -- 172.21.15.112:0/376130301 wait complete. 2024-09-16T00:43:32.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:43:31 smithi169 ceph-mon[77626]: pgmap v473: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:43:32.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:43:31 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:43:32.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.297+0000 7f25b02af640 1 -- 172.21.15.112:0/2866597606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25a80fb810 msgr2=0x7f25a80fbbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:32.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.297+0000 7f25b02af640 1 --2- 172.21.15.112:0/2866597606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25a80fb810 0x7f25a80fbbe0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f25a40048d0 tx=0x7f25a4009da0 comp rx=0 tx=0).stop 2024-09-16T00:43:32.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.297+0000 7f25b02af640 1 -- 172.21.15.112:0/2866597606 shutdown_connections 2024-09-16T00:43:32.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.297+0000 7f25b02af640 1 --2- 172.21.15.112:0/2866597606 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25a80fcf80 0x7f25a80fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:32.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.297+0000 7f25b02af640 1 --2- 172.21.15.112:0/2866597606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25a80fb810 0x7f25a80fbbe0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:32.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.297+0000 7f25b02af640 1 -- 172.21.15.112:0/2866597606 >> 172.21.15.112:0/2866597606 conn(0x7f25a80723b0 msgr2=0x7f25a80727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:32.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.297+0000 7f25b02af640 1 -- 172.21.15.112:0/2866597606 shutdown_connections 2024-09-16T00:43:32.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.297+0000 7f25b02af640 1 -- 172.21.15.112:0/2866597606 wait complete. 2024-09-16T00:43:32.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.297+0000 7f25b02af640 1 Processor -- start 2024-09-16T00:43:32.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.297+0000 7f25b02af640 1 -- start start 2024-09-16T00:43:32.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.297+0000 7f25b02af640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25a80fb810 0x7f25a81154a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:32.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.298+0000 7f25b02af640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25a80fcf80 0x7f25a81159e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:32.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.298+0000 7f25b02af640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a81160c0 con 0x7f25a80fb810 2024-09-16T00:43:32.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.298+0000 7f25b02af640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a810f440 con 0x7f25a80fcf80 2024-09-16T00:43:32.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.298+0000 7f25ae04b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25a80fb810 0x7f25a81154a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:32.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.298+0000 7f25ae04b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25a80fb810 0x7f25a81154a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57356/0 (socket says 172.21.15.112:57356) 2024-09-16T00:43:32.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.298+0000 7f25ae04b640 1 -- 172.21.15.112:0/2901381584 learned_addr learned my addr 172.21.15.112:0/2901381584 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:43:32.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.298+0000 7f25ad84a640 1 --2- 172.21.15.112:0/2901381584 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25a80fcf80 0x7f25a81159e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:32.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.298+0000 7f25ae04b640 1 -- 172.21.15.112:0/2901381584 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25a80fcf80 msgr2=0x7f25a81159e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:32.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.298+0000 7f25ae04b640 1 --2- 172.21.15.112:0/2901381584 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25a80fcf80 0x7f25a81159e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:32.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.298+0000 7f25ae04b640 1 -- 172.21.15.112:0/2901381584 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a4005040 con 0x7f25a80fb810 2024-09-16T00:43:32.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.298+0000 7f25ae04b640 1 --2- 172.21.15.112:0/2901381584 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25a80fb810 0x7f25a81154a0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f25a4003080 tx=0x7f25a4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:32.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.298+0000 7f259effd640 1 -- 172.21.15.112:0/2901381584 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25a4038070 con 0x7f25a80fb810 2024-09-16T00:43:32.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.298+0000 7f259effd640 1 -- 172.21.15.112:0/2901381584 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f25a40097e0 con 0x7f25a80fb810 2024-09-16T00:43:32.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.298+0000 7f25b02af640 1 -- 172.21.15.112:0/2901381584 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25a810f670 con 0x7f25a80fb810 2024-09-16T00:43:32.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.298+0000 7f259effd640 1 -- 172.21.15.112:0/2901381584 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25a403d420 con 0x7f25a80fb810 2024-09-16T00:43:32.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.299+0000 7f25b02af640 1 -- 172.21.15.112:0/2901381584 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25a810fb30 con 0x7f25a80fb810 2024-09-16T00:43:32.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.299+0000 7f25b02af640 1 -- 172.21.15.112:0/2901381584 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25a80fcbc0 con 0x7f25a80fb810 2024-09-16T00:43:32.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.300+0000 7f259effd640 1 -- 172.21.15.112:0/2901381584 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f25a4030020 con 0x7f25a80fb810 2024-09-16T00:43:32.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.301+0000 7f259effd640 1 --2- 172.21.15.112:0/2901381584 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f259406e530 0x7f25940709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:32.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.301+0000 7f259effd640 1 -- 172.21.15.112:0/2901381584 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f25a40ae830 con 0x7f25a80fb810 2024-09-16T00:43:32.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.301+0000 7f25ad84a640 1 --2- 172.21.15.112:0/2901381584 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f259406e530 0x7f25940709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:32.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.301+0000 7f25ad84a640 1 --2- 172.21.15.112:0/2901381584 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f259406e530 0x7f25940709e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2598006fd0 tx=0x7f2598008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:32.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.305+0000 7f259effd640 1 -- 172.21.15.112:0/2901381584 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f25a40777a0 con 0x7f25a80fb810 2024-09-16T00:43:32.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.506+0000 7f25b02af640 1 -- 172.21.15.112:0/2901381584 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f25a8110b10 con 0x7f25a80fb810 2024-09-16T00:43:32.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.507+0000 7f259effd640 1 -- 172.21.15.112:0/2901381584 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1914 (secure 0 0 0) 0x7f25a407ae60 con 0x7f25a80fb810 2024-09-16T00:43:32.507 INFO:teuthology.orchestra.run.smithi112.stdout:e34 2024-09-16T00:43:32.507 INFO:teuthology.orchestra.run.smithi112.stdout:btime 2024-09-16T00:34:26:619254+0000 2024-09-16T00:43:32.507 INFO:teuthology.orchestra.run.smithi112.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T00:43:32.507 INFO:teuthology.orchestra.run.smithi112.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:43:32.508 INFO:teuthology.orchestra.run.smithi112.stdout:legacy client fscid: 1 2024-09-16T00:43:32.508 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:43:32.508 INFO:teuthology.orchestra.run.smithi112.stdout:Filesystem 'cephfs' (1) 2024-09-16T00:43:32.508 INFO:teuthology.orchestra.run.smithi112.stdout:fs_name cephfs 2024-09-16T00:43:32.508 INFO:teuthology.orchestra.run.smithi112.stdout:epoch 25 2024-09-16T00:43:32.508 INFO:teuthology.orchestra.run.smithi112.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-16T00:43:32.508 INFO:teuthology.orchestra.run.smithi112.stdout:created 2024-09-16T00:23:22.328199+0000 2024-09-16T00:43:32.508 INFO:teuthology.orchestra.run.smithi112.stdout:modified 2024-09-16T00:33:29.012991+0000 2024-09-16T00:43:32.508 INFO:teuthology.orchestra.run.smithi112.stdout:tableserver 0 2024-09-16T00:43:32.508 INFO:teuthology.orchestra.run.smithi112.stdout:root 0 2024-09-16T00:43:32.508 INFO:teuthology.orchestra.run.smithi112.stdout:session_timeout 60 2024-09-16T00:43:32.508 INFO:teuthology.orchestra.run.smithi112.stdout:session_autoclose 300 2024-09-16T00:43:32.508 INFO:teuthology.orchestra.run.smithi112.stdout:max_file_size 1099511627776 2024-09-16T00:43:32.508 INFO:teuthology.orchestra.run.smithi112.stdout:max_xattr_size 65536 2024-09-16T00:43:32.509 INFO:teuthology.orchestra.run.smithi112.stdout:required_client_features {} 2024-09-16T00:43:32.509 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure 0 2024-09-16T00:43:32.509 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure_osd_epoch 82 2024-09-16T00:43:32.509 INFO:teuthology.orchestra.run.smithi112.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:43:32.509 INFO:teuthology.orchestra.run.smithi112.stdout:max_mds 2 2024-09-16T00:43:32.509 INFO:teuthology.orchestra.run.smithi112.stdout:in 0,1 2024-09-16T00:43:32.509 INFO:teuthology.orchestra.run.smithi112.stdout:up {} 2024-09-16T00:43:32.509 INFO:teuthology.orchestra.run.smithi112.stdout:failed 0,1 2024-09-16T00:43:32.509 INFO:teuthology.orchestra.run.smithi112.stdout:damaged 2024-09-16T00:43:32.509 INFO:teuthology.orchestra.run.smithi112.stdout:stopped 2024-09-16T00:43:32.509 INFO:teuthology.orchestra.run.smithi112.stdout:data_pools [3] 2024-09-16T00:43:32.509 INFO:teuthology.orchestra.run.smithi112.stdout:metadata_pool 2 2024-09-16T00:43:32.509 INFO:teuthology.orchestra.run.smithi112.stdout:inline_data enabled 2024-09-16T00:43:32.509 INFO:teuthology.orchestra.run.smithi112.stdout:balancer 2024-09-16T00:43:32.510 INFO:teuthology.orchestra.run.smithi112.stdout:bal_rank_mask -1 2024-09-16T00:43:32.510 INFO:teuthology.orchestra.run.smithi112.stdout:standby_count_wanted 1 2024-09-16T00:43:32.510 INFO:teuthology.orchestra.run.smithi112.stdout:qdb_cluster leader: 0 members: 2024-09-16T00:43:32.510 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:43:32.510 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:43:32.510 INFO:teuthology.orchestra.run.smithi112.stdout:Standby daemons: 2024-09-16T00:43:32.510 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:43:32.510 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.ithwzj{-1:34392} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.112:6826/476229221,v1:172.21.15.112:6827/476229221] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:43:32.510 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.hxfwkh{-1:34404} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.112:6828/2356899876,v1:172.21.15.112:6829/2356899876] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:43:32.510 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.gawwgw{-1:44343} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.169:6824/1635329957,v1:172.21.15.169:6825/1635329957] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:43:32.510 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.lgeqrj{-1:44347} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.169:6826/184810031,v1:172.21.15.169:6827/184810031] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:43:32.511 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 34 2024-09-16T00:43:32.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.509+0000 7f25b02af640 1 -- 172.21.15.112:0/2901381584 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f259406e530 msgr2=0x7f25940709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:32.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.509+0000 7f25b02af640 1 --2- 172.21.15.112:0/2901381584 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f259406e530 0x7f25940709e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2598006fd0 tx=0x7f2598008040 comp rx=0 tx=0).stop 2024-09-16T00:43:32.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.509+0000 7f25b02af640 1 -- 172.21.15.112:0/2901381584 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25a80fb810 msgr2=0x7f25a81154a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:32.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.509+0000 7f25b02af640 1 --2- 172.21.15.112:0/2901381584 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25a80fb810 0x7f25a81154a0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f25a4003080 tx=0x7f25a4009da0 comp rx=0 tx=0).stop 2024-09-16T00:43:32.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.509+0000 7f25b02af640 1 -- 172.21.15.112:0/2901381584 shutdown_connections 2024-09-16T00:43:32.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.509+0000 7f25b02af640 1 --2- 172.21.15.112:0/2901381584 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f259406e530 0x7f25940709e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:32.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.509+0000 7f25b02af640 1 --2- 172.21.15.112:0/2901381584 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25a80fcf80 0x7f25a81159e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:32.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.509+0000 7f25b02af640 1 --2- 172.21.15.112:0/2901381584 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25a80fb810 0x7f25a81154a0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:32.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.509+0000 7f25b02af640 1 -- 172.21.15.112:0/2901381584 >> 172.21.15.112:0/2901381584 conn(0x7f25a80723b0 msgr2=0x7f25a80f8470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:32.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.509+0000 7f25b02af640 1 -- 172.21.15.112:0/2901381584 shutdown_connections 2024-09-16T00:43:32.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.509+0000 7f25b02af640 1 -- 172.21.15.112:0/2901381584 wait complete. 2024-09-16T00:43:32.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.588+0000 7fb524334640 1 -- 172.21.15.112:0/3330944812 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb51c1014c0 msgr2=0x7fb51c101890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:32.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.588+0000 7fb524334640 1 --2- 172.21.15.112:0/3330944812 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb51c1014c0 0x7fb51c101890 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb5180048d0 tx=0x7fb518009da0 comp rx=0 tx=0).stop 2024-09-16T00:43:32.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.588+0000 7fb524334640 1 -- 172.21.15.112:0/3330944812 shutdown_connections 2024-09-16T00:43:32.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.588+0000 7fb524334640 1 --2- 172.21.15.112:0/3330944812 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb51c0fb480 0x7fb51c0fb8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:32.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.588+0000 7fb524334640 1 --2- 172.21.15.112:0/3330944812 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb51c1014c0 0x7fb51c101890 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:32.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.588+0000 7fb524334640 1 -- 172.21.15.112:0/3330944812 >> 172.21.15.112:0/3330944812 conn(0x7fb51c0f7000 msgr2=0x7fb51c0f9410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:32.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.588+0000 7fb524334640 1 -- 172.21.15.112:0/3330944812 shutdown_connections 2024-09-16T00:43:32.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.588+0000 7fb524334640 1 -- 172.21.15.112:0/3330944812 wait complete. 2024-09-16T00:43:32.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.589+0000 7fb524334640 1 Processor -- start 2024-09-16T00:43:32.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.589+0000 7fb524334640 1 -- start start 2024-09-16T00:43:32.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.589+0000 7fb524334640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb51c0fb480 0x7fb51c10bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:32.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.589+0000 7fb524334640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb51c1014c0 0x7fb51c10c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:32.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.589+0000 7fb524334640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51c10ca00 con 0x7fb51c0fb480 2024-09-16T00:43:32.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.589+0000 7fb524334640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51c10cb40 con 0x7fb51c1014c0 2024-09-16T00:43:32.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.589+0000 7fb5220d0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb51c0fb480 0x7fb51c10bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:32.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.589+0000 7fb5220d0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb51c0fb480 0x7fb51c10bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57382/0 (socket says 172.21.15.112:57382) 2024-09-16T00:43:32.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.589+0000 7fb5220d0640 1 -- 172.21.15.112:0/1572530785 learned_addr learned my addr 172.21.15.112:0/1572530785 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:43:32.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.589+0000 7fb5218cf640 1 --2- 172.21.15.112:0/1572530785 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb51c1014c0 0x7fb51c10c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:32.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.590+0000 7fb5220d0640 1 -- 172.21.15.112:0/1572530785 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb51c1014c0 msgr2=0x7fb51c10c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:32.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.590+0000 7fb5220d0640 1 --2- 172.21.15.112:0/1572530785 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb51c1014c0 0x7fb51c10c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:32.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.590+0000 7fb5220d0640 1 -- 172.21.15.112:0/1572530785 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb518005040 con 0x7fb51c0fb480 2024-09-16T00:43:32.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.590+0000 7fb5220d0640 1 --2- 172.21.15.112:0/1572530785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb51c0fb480 0x7fb51c10bf80 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb5180043d0 tx=0x7fb518009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:32.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.590+0000 7fb512ffd640 1 -- 172.21.15.112:0/1572530785 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb518038070 con 0x7fb51c0fb480 2024-09-16T00:43:32.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.590+0000 7fb524334640 1 -- 172.21.15.112:0/1572530785 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb51c10cdc0 con 0x7fb51c0fb480 2024-09-16T00:43:32.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.590+0000 7fb512ffd640 1 -- 172.21.15.112:0/1572530785 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fb5180096e0 con 0x7fb51c0fb480 2024-09-16T00:43:32.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.590+0000 7fb512ffd640 1 -- 172.21.15.112:0/1572530785 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb51803d420 con 0x7fb51c0fb480 2024-09-16T00:43:32.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.590+0000 7fb524334640 1 -- 172.21.15.112:0/1572530785 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb51c119920 con 0x7fb51c0fb480 2024-09-16T00:43:32.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.591+0000 7fb524334640 1 -- 172.21.15.112:0/1572530785 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb51c0fcbc0 con 0x7fb51c0fb480 2024-09-16T00:43:32.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.592+0000 7fb512ffd640 1 -- 172.21.15.112:0/1572530785 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fb518030020 con 0x7fb51c0fb480 2024-09-16T00:43:32.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.592+0000 7fb512ffd640 1 --2- 172.21.15.112:0/1572530785 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fb50806e530 0x7fb5080709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:32.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.592+0000 7fb512ffd640 1 -- 172.21.15.112:0/1572530785 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7fb5180aea20 con 0x7fb51c0fb480 2024-09-16T00:43:32.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.592+0000 7fb5218cf640 1 --2- 172.21.15.112:0/1572530785 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fb50806e530 0x7fb5080709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:32.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.593+0000 7fb5218cf640 1 --2- 172.21.15.112:0/1572530785 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fb50806e530 0x7fb5080709e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb51c066500 tx=0x7fb50c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:32.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.596+0000 7fb512ffd640 1 -- 172.21.15.112:0/1572530785 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fb5180779b0 con 0x7fb51c0fb480 2024-09-16T00:43:32.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.779+0000 7fb524334640 1 -- 172.21.15.112:0/1572530785 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb51c066200 con 0x7fb50806e530 2024-09-16T00:43:32.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.780+0000 7fb512ffd640 1 -- 172.21.15.112:0/1572530785 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fb51c066200 con 0x7fb50806e530 2024-09-16T00:43:32.780 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:43:32.780 INFO:teuthology.orchestra.run.smithi112.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80", 2024-09-16T00:43:32.780 INFO:teuthology.orchestra.run.smithi112.stdout: "in_progress": true, 2024-09-16T00:43:32.780 INFO:teuthology.orchestra.run.smithi112.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T00:43:32.780 INFO:teuthology.orchestra.run.smithi112.stdout: "services_complete": [ 2024-09-16T00:43:32.781 INFO:teuthology.orchestra.run.smithi112.stdout: "mds", 2024-09-16T00:43:32.781 INFO:teuthology.orchestra.run.smithi112.stdout: "osd", 2024-09-16T00:43:32.781 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr", 2024-09-16T00:43:32.781 INFO:teuthology.orchestra.run.smithi112.stdout: "mon", 2024-09-16T00:43:32.781 INFO:teuthology.orchestra.run.smithi112.stdout: "crash" 2024-09-16T00:43:32.781 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-16T00:43:32.781 INFO:teuthology.orchestra.run.smithi112.stdout: "progress": "16/21 daemons upgraded", 2024-09-16T00:43:32.781 INFO:teuthology.orchestra.run.smithi112.stdout: "message": "Currently upgrading mds daemons", 2024-09-16T00:43:32.781 INFO:teuthology.orchestra.run.smithi112.stdout: "is_paused": false 2024-09-16T00:43:32.781 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:43:32.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.782+0000 7fb524334640 1 -- 172.21.15.112:0/1572530785 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fb50806e530 msgr2=0x7fb5080709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:32.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.782+0000 7fb524334640 1 --2- 172.21.15.112:0/1572530785 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fb50806e530 0x7fb5080709e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb51c066500 tx=0x7fb50c00b040 comp rx=0 tx=0).stop 2024-09-16T00:43:32.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.782+0000 7fb524334640 1 -- 172.21.15.112:0/1572530785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb51c0fb480 msgr2=0x7fb51c10bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:32.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.782+0000 7fb524334640 1 --2- 172.21.15.112:0/1572530785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb51c0fb480 0x7fb51c10bf80 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb5180043d0 tx=0x7fb518009da0 comp rx=0 tx=0).stop 2024-09-16T00:43:32.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.782+0000 7fb524334640 1 -- 172.21.15.112:0/1572530785 shutdown_connections 2024-09-16T00:43:32.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.782+0000 7fb524334640 1 --2- 172.21.15.112:0/1572530785 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fb50806e530 0x7fb5080709e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:32.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.782+0000 7fb524334640 1 --2- 172.21.15.112:0/1572530785 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb51c1014c0 0x7fb51c10c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:32.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.782+0000 7fb524334640 1 --2- 172.21.15.112:0/1572530785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb51c0fb480 0x7fb51c10bf80 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:32.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.782+0000 7fb524334640 1 -- 172.21.15.112:0/1572530785 >> 172.21.15.112:0/1572530785 conn(0x7fb51c0f7000 msgr2=0x7fb51c0f88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:32.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.783+0000 7fb524334640 1 -- 172.21.15.112:0/1572530785 shutdown_connections 2024-09-16T00:43:32.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.783+0000 7fb524334640 1 -- 172.21.15.112:0/1572530785 wait complete. 2024-09-16T00:43:32.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.862+0000 7f2461d23640 1 -- 172.21.15.112:0/2648563535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f245c102530 msgr2=0x7f245c102900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:32.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.862+0000 7f2461d23640 1 --2- 172.21.15.112:0/2648563535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f245c102530 0x7f245c102900 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f244c0048d0 tx=0x7f244c009da0 comp rx=0 tx=0).stop 2024-09-16T00:43:32.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.862+0000 7f2461d23640 1 -- 172.21.15.112:0/2648563535 shutdown_connections 2024-09-16T00:43:32.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.862+0000 7f2461d23640 1 --2- 172.21.15.112:0/2648563535 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f245c102ed0 0x7f245c0fc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:32.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.862+0000 7f2461d23640 1 --2- 172.21.15.112:0/2648563535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f245c102530 0x7f245c102900 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:32.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.862+0000 7f2461d23640 1 -- 172.21.15.112:0/2648563535 >> 172.21.15.112:0/2648563535 conn(0x7f245c0f80f0 msgr2=0x7f245c0fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:32.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.862+0000 7f2461d23640 1 -- 172.21.15.112:0/2648563535 shutdown_connections 2024-09-16T00:43:32.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.862+0000 7f2461d23640 1 -- 172.21.15.112:0/2648563535 wait complete. 2024-09-16T00:43:32.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.862+0000 7f2461d23640 1 Processor -- start 2024-09-16T00:43:32.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.862+0000 7f2461d23640 1 -- start start 2024-09-16T00:43:32.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.863+0000 7f2461d23640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f245c102530 0x7f245c113240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:32.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.863+0000 7f2461d23640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f245c102ed0 0x7f245c113780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:32.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.863+0000 7f2461d23640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f245c113d80 con 0x7f245c102ed0 2024-09-16T00:43:32.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.863+0000 7f2461d23640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f245c10d1c0 con 0x7f245c102530 2024-09-16T00:43:32.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.863+0000 7f245affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f245c102ed0 0x7f245c113780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:32.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.863+0000 7f245affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f245c102ed0 0x7f245c113780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57390/0 (socket says 172.21.15.112:57390) 2024-09-16T00:43:32.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.863+0000 7f245affd640 1 -- 172.21.15.112:0/1420856671 learned_addr learned my addr 172.21.15.112:0/1420856671 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:43:32.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.863+0000 7f245b7fe640 1 --2- 172.21.15.112:0/1420856671 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f245c102530 0x7f245c113240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:32.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.863+0000 7f245affd640 1 -- 172.21.15.112:0/1420856671 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f245c102530 msgr2=0x7f245c113240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:32.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.863+0000 7f245affd640 1 --2- 172.21.15.112:0/1420856671 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f245c102530 0x7f245c113240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:32.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.863+0000 7f245affd640 1 -- 172.21.15.112:0/1420856671 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f244c005040 con 0x7f245c102ed0 2024-09-16T00:43:32.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.863+0000 7f245affd640 1 --2- 172.21.15.112:0/1420856671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f245c102ed0 0x7f245c113780 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f245000eb40 tx=0x7f245000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:32.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.863+0000 7f2443fff640 1 -- 172.21.15.112:0/1420856671 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2450007c30 con 0x7f245c102ed0 2024-09-16T00:43:32.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.863+0000 7f2461d23640 1 -- 172.21.15.112:0/1420856671 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f245c10d440 con 0x7f245c102ed0 2024-09-16T00:43:32.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.863+0000 7f2443fff640 1 -- 172.21.15.112:0/1420856671 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f245000f040 con 0x7f245c102ed0 2024-09-16T00:43:32.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.863+0000 7f2443fff640 1 -- 172.21.15.112:0/1420856671 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f245000ca90 con 0x7f245c102ed0 2024-09-16T00:43:32.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.863+0000 7f2461d23640 1 -- 172.21.15.112:0/1420856671 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f245c10d9e0 con 0x7f245c102ed0 2024-09-16T00:43:32.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.864+0000 7f2461d23640 1 -- 172.21.15.112:0/1420856671 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f245c0fdcb0 con 0x7f245c102ed0 2024-09-16T00:43:32.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.865+0000 7f2443fff640 1 -- 172.21.15.112:0/1420856671 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f245000cc10 con 0x7f245c102ed0 2024-09-16T00:43:32.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.865+0000 7f2443fff640 1 --2- 172.21.15.112:0/1420856671 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f244406e530 0x7f24440709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:43:32.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.865+0000 7f2443fff640 1 -- 172.21.15.112:0/1420856671 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f245008df60 con 0x7f245c102ed0 2024-09-16T00:43:32.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.865+0000 7f245b7fe640 1 --2- 172.21.15.112:0/1420856671 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f244406e530 0x7f24440709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:43:32.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.866+0000 7f245b7fe640 1 --2- 172.21.15.112:0/1420856671 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f244406e530 0x7f24440709e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f244c003080 tx=0x7f244c035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:43:32.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:32.870+0000 7f2443fff640 1 -- 172.21.15.112:0/1420856671 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f2450056ee0 con 0x7f245c102ed0 2024-09-16T00:43:33.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:33.102+0000 7f2461d23640 1 -- 172.21.15.112:0/1420856671 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f245c1095a0 con 0x7f245c102ed0 2024-09-16T00:43:33.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:33.102+0000 7f2443fff640 1 -- 172.21.15.112:0/1420856671 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f245005a5a0 con 0x7f245c102ed0 2024-09-16T00:43:33.102 INFO:teuthology.orchestra.run.smithi112.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-16T00:43:33.102 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-16T00:43:33.103 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs is degraded 2024-09-16T00:43:33.103 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T00:43:33.103 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T00:43:33.103 INFO:teuthology.orchestra.run.smithi112.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-16T00:43:33.103 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-16T00:43:33.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:33.104+0000 7f2461d23640 1 -- 172.21.15.112:0/1420856671 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f244406e530 msgr2=0x7f24440709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:33.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:33.104+0000 7f2461d23640 1 --2- 172.21.15.112:0/1420856671 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f244406e530 0x7f24440709e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f244c003080 tx=0x7f244c035040 comp rx=0 tx=0).stop 2024-09-16T00:43:33.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:33.104+0000 7f2461d23640 1 -- 172.21.15.112:0/1420856671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f245c102ed0 msgr2=0x7f245c113780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:43:33.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:33.104+0000 7f2461d23640 1 --2- 172.21.15.112:0/1420856671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f245c102ed0 0x7f245c113780 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f245000eb40 tx=0x7f245000b040 comp rx=0 tx=0).stop 2024-09-16T00:43:33.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:33.104+0000 7f2461d23640 1 -- 172.21.15.112:0/1420856671 shutdown_connections 2024-09-16T00:43:33.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:33.104+0000 7f2461d23640 1 --2- 172.21.15.112:0/1420856671 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f244406e530 0x7f24440709e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:33.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:33.104+0000 7f2461d23640 1 --2- 172.21.15.112:0/1420856671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f245c102ed0 0x7f245c113780 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:33.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:33.104+0000 7f2461d23640 1 --2- 172.21.15.112:0/1420856671 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f245c102530 0x7f245c113240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:43:33.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:33.104+0000 7f2461d23640 1 -- 172.21.15.112:0/1420856671 >> 172.21.15.112:0/1420856671 conn(0x7f245c0f80f0 msgr2=0x7f245c0f90f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:43:33.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:33.104+0000 7f2461d23640 1 -- 172.21.15.112:0/1420856671 shutdown_connections 2024-09-16T00:43:33.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:43:33.104+0000 7f2461d23640 1 -- 172.21.15.112:0/1420856671 wait complete. 2024-09-16T00:43:33.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:43:32 smithi169 ceph-mon[77626]: from='client.34912 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:43:33.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:43:32 smithi169 ceph-mon[77626]: from='client.34916 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:43:33.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:43:32 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/376130301' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:43:33.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:43:32 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/2901381584' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:43:33.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:43:32 smithi112 ceph-mon[97750]: from='client.34912 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:43:33.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:43:32 smithi112 ceph-mon[97750]: from='client.34916 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:43:33.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:43:32 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/376130301' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:43:33.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:43:32 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/2901381584' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:43:34.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:43:33 smithi112 ceph-mon[97750]: from='client.34920 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:43:34.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:43:33 smithi112 ceph-mon[97750]: pgmap v474: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:43:34.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:43:33 smithi112 ceph-mon[97750]: from='client.34932 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:43:34.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:43:33 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/1420856671' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:43:34.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:43:33 smithi169 ceph-mon[77626]: from='client.34920 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:43:34.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:43:33 smithi169 ceph-mon[77626]: pgmap v474: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:43:34.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:43:33 smithi169 ceph-mon[77626]: from='client.34932 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:43:34.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:43:33 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/1420856671' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:43:36.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:43:35 smithi169 ceph-mon[77626]: pgmap v475: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:43:36.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:43:35 smithi112 ceph-mon[97750]: pgmap v475: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:43:38.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:43:37 smithi169 ceph-mon[77626]: pgmap v476: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:43:38.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:43:37 smithi112 ceph-mon[97750]: pgmap v476: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:43:40.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:43:39 smithi169 ceph-mon[77626]: pgmap v477: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:43:40.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:43:39 smithi112 ceph-mon[97750]: pgmap v477: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:43:42.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:43:41 smithi169 ceph-mon[77626]: pgmap v478: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:43:42.314 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:43:41 smithi112 ceph-mon[97750]: pgmap v478: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:43:44.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:43:43 smithi112 ceph-mon[97750]: pgmap v479: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:43:44.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:43:43 smithi169 ceph-mon[77626]: pgmap v479: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:43:46.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:43:45 smithi169 ceph-mon[77626]: pgmap v480: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:43:46.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:43:45 smithi112 ceph-mon[97750]: pgmap v480: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:43:47.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:43:46 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:43:47.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:43:46 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:43:48.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:43:47 smithi169 ceph-mon[77626]: pgmap v481: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:43:48.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:43:47 smithi112 ceph-mon[97750]: pgmap v481: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:43:49.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:43:48 smithi169 ceph-mon[77626]: pgmap v482: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:43:49.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:43:48 smithi112 ceph-mon[97750]: pgmap v482: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:43:52.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:43:51 smithi169 ceph-mon[77626]: pgmap v483: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:43:52.042 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:43:51 smithi112 ceph-mon[97750]: pgmap v483: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:43:54.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:43:53 smithi169 ceph-mon[77626]: pgmap v484: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:43:54.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:43:53 smithi112 ceph-mon[97750]: pgmap v484: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:43:56.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:43:55 smithi169 ceph-mon[77626]: pgmap v485: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:43:56.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:43:55 smithi112 ceph-mon[97750]: pgmap v485: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:43:58.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:43:57 smithi169 ceph-mon[77626]: pgmap v486: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:43:58.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:43:57 smithi112 ceph-mon[97750]: pgmap v486: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:44:00.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:43:59 smithi169 ceph-mon[77626]: pgmap v487: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:44:00.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:43:59 smithi112 ceph-mon[97750]: pgmap v487: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:44:02.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:44:01 smithi169 ceph-mon[77626]: pgmap v488: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:44:02.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:44:01 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:44:02.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:44:01 smithi112 ceph-mon[97750]: pgmap v488: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:44:02.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:44:01 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:44:03.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.194+0000 7f123c9a0640 1 -- 172.21.15.112:0/2050374723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12380fac80 msgr2=0x7f12380fb0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:03.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.194+0000 7f123c9a0640 1 --2- 172.21.15.112:0/2050374723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12380fac80 0x7f12380fb0d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f12280048d0 tx=0x7f1228009da0 comp rx=0 tx=0).stop 2024-09-16T00:44:03.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.194+0000 7f123c9a0640 1 -- 172.21.15.112:0/2050374723 shutdown_connections 2024-09-16T00:44:03.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.194+0000 7f123c9a0640 1 --2- 172.21.15.112:0/2050374723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12380fac80 0x7f12380fb0d0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:03.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.194+0000 7f123c9a0640 1 --2- 172.21.15.112:0/2050374723 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1238100c70 0x7f1238101040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:03.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.194+0000 7f123c9a0640 1 -- 172.21.15.112:0/2050374723 >> 172.21.15.112:0/2050374723 conn(0x7f1238070ce0 msgr2=0x7f12380710e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:03.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.194+0000 7f123c9a0640 1 -- 172.21.15.112:0/2050374723 shutdown_connections 2024-09-16T00:44:03.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.194+0000 7f123c9a0640 1 -- 172.21.15.112:0/2050374723 wait complete. 2024-09-16T00:44:03.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.195+0000 7f123c9a0640 1 Processor -- start 2024-09-16T00:44:03.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.195+0000 7f123c9a0640 1 -- start start 2024-09-16T00:44:03.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.195+0000 7f123c9a0640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12380fac80 0x7f12381120e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:03.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.195+0000 7f123c9a0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1238100c70 0x7f1238112620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:03.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.195+0000 7f123c9a0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1238112c20 con 0x7f1238100c70 2024-09-16T00:44:03.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.195+0000 7f123c9a0640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f123810c060 con 0x7f12380fac80 2024-09-16T00:44:03.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.195+0000 7f1235d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1238100c70 0x7f1238112620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:03.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.196+0000 7f1235d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1238100c70 0x7f1238112620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50854/0 (socket says 172.21.15.112:50854) 2024-09-16T00:44:03.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.196+0000 7f1235d9b640 1 -- 172.21.15.112:0/502833285 learned_addr learned my addr 172.21.15.112:0/502833285 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:44:03.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.196+0000 7f123659c640 1 --2- 172.21.15.112:0/502833285 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12380fac80 0x7f12381120e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:03.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.196+0000 7f1235d9b640 1 -- 172.21.15.112:0/502833285 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12380fac80 msgr2=0x7f12381120e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:03.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.196+0000 7f1235d9b640 1 --2- 172.21.15.112:0/502833285 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12380fac80 0x7f12381120e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:03.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.196+0000 7f1235d9b640 1 -- 172.21.15.112:0/502833285 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1228005040 con 0x7f1238100c70 2024-09-16T00:44:03.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.196+0000 7f1235d9b640 1 --2- 172.21.15.112:0/502833285 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1238100c70 0x7f1238112620 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f1228003080 tx=0x7f1228009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:03.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.196+0000 7f122effd640 1 -- 172.21.15.112:0/502833285 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1228038070 con 0x7f1238100c70 2024-09-16T00:44:03.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.196+0000 7f122effd640 1 -- 172.21.15.112:0/502833285 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f12280097e0 con 0x7f1238100c70 2024-09-16T00:44:03.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.196+0000 7f123c9a0640 1 -- 172.21.15.112:0/502833285 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f123810c2e0 con 0x7f1238100c70 2024-09-16T00:44:03.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.196+0000 7f122effd640 1 -- 172.21.15.112:0/502833285 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f122803d420 con 0x7f1238100c70 2024-09-16T00:44:03.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.196+0000 7f123c9a0640 1 -- 172.21.15.112:0/502833285 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f123810c820 con 0x7f1238100c70 2024-09-16T00:44:03.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.198+0000 7f123c9a0640 1 -- 172.21.15.112:0/502833285 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12380fc370 con 0x7f1238100c70 2024-09-16T00:44:03.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.198+0000 7f122effd640 1 -- 172.21.15.112:0/502833285 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f1228030020 con 0x7f1238100c70 2024-09-16T00:44:03.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.198+0000 7f122effd640 1 --2- 172.21.15.112:0/502833285 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f122406e530 0x7f12240709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:03.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.199+0000 7f123659c640 1 --2- 172.21.15.112:0/502833285 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f122406e530 0x7f12240709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:03.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.199+0000 7f122effd640 1 -- 172.21.15.112:0/502833285 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f12280ae760 con 0x7f1238100c70 2024-09-16T00:44:03.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.199+0000 7f123659c640 1 --2- 172.21.15.112:0/502833285 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f122406e530 0x7f12240709e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f12200041d0 tx=0x7f1220004330 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:03.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.203+0000 7f122effd640 1 -- 172.21.15.112:0/502833285 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f1228077750 con 0x7f1238100c70 2024-09-16T00:44:03.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.388+0000 7f123c9a0640 1 -- 172.21.15.112:0/502833285 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1238107c10 con 0x7f122406e530 2024-09-16T00:44:03.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.389+0000 7f122effd640 1 -- 172.21.15.112:0/502833285 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f1238107c10 con 0x7f122406e530 2024-09-16T00:44:03.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.391+0000 7f123c9a0640 1 -- 172.21.15.112:0/502833285 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f122406e530 msgr2=0x7f12240709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:03.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.391+0000 7f123c9a0640 1 --2- 172.21.15.112:0/502833285 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f122406e530 0x7f12240709e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f12200041d0 tx=0x7f1220004330 comp rx=0 tx=0).stop 2024-09-16T00:44:03.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.391+0000 7f123c9a0640 1 -- 172.21.15.112:0/502833285 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1238100c70 msgr2=0x7f1238112620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:03.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.391+0000 7f123c9a0640 1 --2- 172.21.15.112:0/502833285 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1238100c70 0x7f1238112620 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f1228003080 tx=0x7f1228009da0 comp rx=0 tx=0).stop 2024-09-16T00:44:03.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.391+0000 7f123c9a0640 1 -- 172.21.15.112:0/502833285 shutdown_connections 2024-09-16T00:44:03.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.391+0000 7f123c9a0640 1 --2- 172.21.15.112:0/502833285 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f122406e530 0x7f12240709e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:03.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.391+0000 7f123c9a0640 1 --2- 172.21.15.112:0/502833285 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1238100c70 0x7f1238112620 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:03.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.391+0000 7f123c9a0640 1 --2- 172.21.15.112:0/502833285 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12380fac80 0x7f12381120e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:03.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.391+0000 7f123c9a0640 1 -- 172.21.15.112:0/502833285 >> 172.21.15.112:0/502833285 conn(0x7f1238070ce0 msgr2=0x7f12380febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:03.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.391+0000 7f123c9a0640 1 -- 172.21.15.112:0/502833285 shutdown_connections 2024-09-16T00:44:03.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.391+0000 7f123c9a0640 1 -- 172.21.15.112:0/502833285 wait complete. 2024-09-16T00:44:03.400 INFO:teuthology.orchestra.run.smithi112.stdout:true 2024-09-16T00:44:03.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.471+0000 7f1acc21c640 1 -- 172.21.15.112:0/1393872965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ac40fb810 msgr2=0x7f1ac40fbbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:03.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.471+0000 7f1acc21c640 1 --2- 172.21.15.112:0/1393872965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ac40fb810 0x7f1ac40fbbe0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f1ab40048d0 tx=0x7f1ab4009da0 comp rx=0 tx=0).stop 2024-09-16T00:44:03.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.471+0000 7f1acc21c640 1 -- 172.21.15.112:0/1393872965 shutdown_connections 2024-09-16T00:44:03.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.471+0000 7f1acc21c640 1 --2- 172.21.15.112:0/1393872965 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ac40fcf80 0x7f1ac40fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:03.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.471+0000 7f1acc21c640 1 --2- 172.21.15.112:0/1393872965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ac40fb810 0x7f1ac40fbbe0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:03.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.471+0000 7f1acc21c640 1 -- 172.21.15.112:0/1393872965 >> 172.21.15.112:0/1393872965 conn(0x7f1ac40723b0 msgr2=0x7f1ac40727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:03.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.472+0000 7f1acc21c640 1 -- 172.21.15.112:0/1393872965 shutdown_connections 2024-09-16T00:44:03.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.472+0000 7f1acc21c640 1 -- 172.21.15.112:0/1393872965 wait complete. 2024-09-16T00:44:03.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.472+0000 7f1acc21c640 1 Processor -- start 2024-09-16T00:44:03.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.472+0000 7f1acc21c640 1 -- start start 2024-09-16T00:44:03.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.472+0000 7f1acc21c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ac40fb810 0x7f1ac4113220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:03.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.472+0000 7f1acc21c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ac40fcf80 0x7f1ac4113760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:03.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.472+0000 7f1acc21c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac4113d60 con 0x7f1ac40fb810 2024-09-16T00:44:03.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.472+0000 7f1acc21c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac410d1a0 con 0x7f1ac40fcf80 2024-09-16T00:44:03.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.473+0000 7f1ac9fb8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ac40fb810 0x7f1ac4113220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:03.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.473+0000 7f1ac9fb8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ac40fb810 0x7f1ac4113220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50870/0 (socket says 172.21.15.112:50870) 2024-09-16T00:44:03.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.473+0000 7f1ac9fb8640 1 -- 172.21.15.112:0/2619343136 learned_addr learned my addr 172.21.15.112:0/2619343136 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:44:03.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.473+0000 7f1ac97b7640 1 --2- 172.21.15.112:0/2619343136 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ac40fcf80 0x7f1ac4113760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:03.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.473+0000 7f1ac9fb8640 1 -- 172.21.15.112:0/2619343136 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ac40fcf80 msgr2=0x7f1ac4113760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:03.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.473+0000 7f1ac9fb8640 1 --2- 172.21.15.112:0/2619343136 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ac40fcf80 0x7f1ac4113760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:03.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.473+0000 7f1ac9fb8640 1 -- 172.21.15.112:0/2619343136 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ab4005040 con 0x7f1ac40fb810 2024-09-16T00:44:03.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.473+0000 7f1ac9fb8640 1 --2- 172.21.15.112:0/2619343136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ac40fb810 0x7f1ac4113220 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4003080 tx=0x7f1ab4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:03.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.473+0000 7f1aba7fc640 1 -- 172.21.15.112:0/2619343136 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ab4038070 con 0x7f1ac40fb810 2024-09-16T00:44:03.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.473+0000 7f1aba7fc640 1 -- 172.21.15.112:0/2619343136 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f1ab40097e0 con 0x7f1ac40fb810 2024-09-16T00:44:03.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.473+0000 7f1acc21c640 1 -- 172.21.15.112:0/2619343136 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ac410d420 con 0x7f1ac40fb810 2024-09-16T00:44:03.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.473+0000 7f1aba7fc640 1 -- 172.21.15.112:0/2619343136 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ab403d420 con 0x7f1ac40fb810 2024-09-16T00:44:03.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.473+0000 7f1acc21c640 1 -- 172.21.15.112:0/2619343136 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ac410d960 con 0x7f1ac40fb810 2024-09-16T00:44:03.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.474+0000 7f1acc21c640 1 -- 172.21.15.112:0/2619343136 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ac40fcbc0 con 0x7f1ac40fb810 2024-09-16T00:44:03.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.475+0000 7f1aba7fc640 1 -- 172.21.15.112:0/2619343136 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f1ab4030020 con 0x7f1ac40fb810 2024-09-16T00:44:03.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.475+0000 7f1aba7fc640 1 --2- 172.21.15.112:0/2619343136 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f1ab006e530 0x7f1ab00709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:03.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.476+0000 7f1aba7fc640 1 -- 172.21.15.112:0/2619343136 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f1ab40ae8a0 con 0x7f1ac40fb810 2024-09-16T00:44:03.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.476+0000 7f1ac97b7640 1 --2- 172.21.15.112:0/2619343136 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f1ab006e530 0x7f1ab00709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:03.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.476+0000 7f1ac97b7640 1 --2- 172.21.15.112:0/2619343136 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f1ab006e530 0x7f1ab00709e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1ac410eae0 tx=0x7f1ac0009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:03.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.481+0000 7f1aba7fc640 1 -- 172.21.15.112:0/2619343136 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f1ab4077810 con 0x7f1ac40fb810 2024-09-16T00:44:03.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.665+0000 7f1acc21c640 1 -- 172.21.15.112:0/2619343136 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1ac404b720 con 0x7f1ab006e530 2024-09-16T00:44:03.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.666+0000 7f1aba7fc640 1 -- 172.21.15.112:0/2619343136 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f1ac404b720 con 0x7f1ab006e530 2024-09-16T00:44:03.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.668+0000 7f1acc21c640 1 -- 172.21.15.112:0/2619343136 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f1ab006e530 msgr2=0x7f1ab00709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:03.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.668+0000 7f1acc21c640 1 --2- 172.21.15.112:0/2619343136 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f1ab006e530 0x7f1ab00709e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1ac410eae0 tx=0x7f1ac0009040 comp rx=0 tx=0).stop 2024-09-16T00:44:03.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.668+0000 7f1acc21c640 1 -- 172.21.15.112:0/2619343136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ac40fb810 msgr2=0x7f1ac4113220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:03.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.668+0000 7f1acc21c640 1 --2- 172.21.15.112:0/2619343136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ac40fb810 0x7f1ac4113220 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4003080 tx=0x7f1ab4009da0 comp rx=0 tx=0).stop 2024-09-16T00:44:03.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.668+0000 7f1acc21c640 1 -- 172.21.15.112:0/2619343136 shutdown_connections 2024-09-16T00:44:03.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.668+0000 7f1acc21c640 1 --2- 172.21.15.112:0/2619343136 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f1ab006e530 0x7f1ab00709e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:03.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.668+0000 7f1acc21c640 1 --2- 172.21.15.112:0/2619343136 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ac40fcf80 0x7f1ac4113760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:03.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.668+0000 7f1acc21c640 1 --2- 172.21.15.112:0/2619343136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ac40fb810 0x7f1ac4113220 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:03.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.668+0000 7f1acc21c640 1 -- 172.21.15.112:0/2619343136 >> 172.21.15.112:0/2619343136 conn(0x7f1ac40723b0 msgr2=0x7f1ac4064f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:03.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.668+0000 7f1acc21c640 1 -- 172.21.15.112:0/2619343136 shutdown_connections 2024-09-16T00:44:03.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.668+0000 7f1acc21c640 1 -- 172.21.15.112:0/2619343136 wait complete. 2024-09-16T00:44:03.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.747+0000 7f82aa5c2640 1 -- 172.21.15.112:0/2112435287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82a40fb490 msgr2=0x7f82a40fb900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:03.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.747+0000 7f82aa5c2640 1 --2- 172.21.15.112:0/2112435287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82a40fb490 0x7f82a40fb900 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f828c0048d0 tx=0x7f828c009da0 comp rx=0 tx=0).stop 2024-09-16T00:44:03.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.748+0000 7f82aa5c2640 1 -- 172.21.15.112:0/2112435287 shutdown_connections 2024-09-16T00:44:03.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.748+0000 7f82aa5c2640 1 --2- 172.21.15.112:0/2112435287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82a40fb490 0x7f82a40fb900 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:03.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.748+0000 7f82aa5c2640 1 --2- 172.21.15.112:0/2112435287 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82a41014d0 0x7f82a41018a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:03.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.748+0000 7f82aa5c2640 1 -- 172.21.15.112:0/2112435287 >> 172.21.15.112:0/2112435287 conn(0x7f82a40f7010 msgr2=0x7f82a40f9420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:03.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.748+0000 7f82aa5c2640 1 -- 172.21.15.112:0/2112435287 shutdown_connections 2024-09-16T00:44:03.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.748+0000 7f82aa5c2640 1 -- 172.21.15.112:0/2112435287 wait complete. 2024-09-16T00:44:03.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.749+0000 7f82aa5c2640 1 Processor -- start 2024-09-16T00:44:03.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.749+0000 7f82aa5c2640 1 -- start start 2024-09-16T00:44:03.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.749+0000 7f82aa5c2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82a40fb490 0x7f82a4112160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:03.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.749+0000 7f82aa5c2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82a41014d0 0x7f82a41126a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:03.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.749+0000 7f82aa5c2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a4112d30 con 0x7f82a41014d0 2024-09-16T00:44:03.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.749+0000 7f82aa5c2640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a410c0e0 con 0x7f82a40fb490 2024-09-16T00:44:03.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.749+0000 7f829bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82a41014d0 0x7f82a41126a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:03.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.749+0000 7f829bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82a41014d0 0x7f82a41126a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50886/0 (socket says 172.21.15.112:50886) 2024-09-16T00:44:03.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.749+0000 7f829bfff640 1 -- 172.21.15.112:0/726137115 learned_addr learned my addr 172.21.15.112:0/726137115 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:44:03.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.749+0000 7f82a3fff640 1 --2- 172.21.15.112:0/726137115 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82a40fb490 0x7f82a4112160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:03.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.749+0000 7f829bfff640 1 -- 172.21.15.112:0/726137115 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82a40fb490 msgr2=0x7f82a4112160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:03.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.749+0000 7f829bfff640 1 --2- 172.21.15.112:0/726137115 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82a40fb490 0x7f82a4112160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:03.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.749+0000 7f829bfff640 1 -- 172.21.15.112:0/726137115 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f828c005040 con 0x7f82a41014d0 2024-09-16T00:44:03.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.749+0000 7f829bfff640 1 --2- 172.21.15.112:0/726137115 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82a41014d0 0x7f82a41126a0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f828c000c00 tx=0x7f828c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:03.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.750+0000 7f82a17fa640 1 -- 172.21.15.112:0/726137115 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f828c038070 con 0x7f82a41014d0 2024-09-16T00:44:03.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.750+0000 7f82a17fa640 1 -- 172.21.15.112:0/726137115 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f828c03daf0 con 0x7f82a41014d0 2024-09-16T00:44:03.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.750+0000 7f82aa5c2640 1 -- 172.21.15.112:0/726137115 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82a410c360 con 0x7f82a41014d0 2024-09-16T00:44:03.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.750+0000 7f82a17fa640 1 -- 172.21.15.112:0/726137115 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f828c033b20 con 0x7f82a41014d0 2024-09-16T00:44:03.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.750+0000 7f82aa5c2640 1 -- 172.21.15.112:0/726137115 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82a410c900 con 0x7f82a41014d0 2024-09-16T00:44:03.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.751+0000 7f82aa5c2640 1 -- 172.21.15.112:0/726137115 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82a40fcbd0 con 0x7f82a41014d0 2024-09-16T00:44:03.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.751+0000 7f82a17fa640 1 -- 172.21.15.112:0/726137115 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f828c030020 con 0x7f82a41014d0 2024-09-16T00:44:03.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.752+0000 7f82a17fa640 1 --2- 172.21.15.112:0/726137115 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f829006e530 0x7f82900709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:03.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.752+0000 7f82a17fa640 1 -- 172.21.15.112:0/726137115 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f828c0ae470 con 0x7f82a41014d0 2024-09-16T00:44:03.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.752+0000 7f82a3fff640 1 --2- 172.21.15.112:0/726137115 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f829006e530 0x7f82900709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:03.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.752+0000 7f82a3fff640 1 --2- 172.21.15.112:0/726137115 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f829006e530 0x7f82900709e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f829400a8a0 tx=0x7f8294008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:03.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.756+0000 7f82a17fa640 1 -- 172.21.15.112:0/726137115 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f828c0773e0 con 0x7f82a41014d0 2024-09-16T00:44:03.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.931+0000 7f82aa5c2640 1 -- 172.21.15.112:0/726137115 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f82a4108580 con 0x7f829006e530 2024-09-16T00:44:03.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.939+0000 7f82a17fa640 1 -- 172.21.15.112:0/726137115 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7f82a4108580 con 0x7f829006e530 2024-09-16T00:44:03.939 INFO:teuthology.orchestra.run.smithi112.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T00:44:03.939 INFO:teuthology.orchestra.run.smithi112.stdout:alertmanager.smithi112 smithi112 *:9093,9094 running (17m) 10m ago 25m 24.7M - 0.25.0 c8568f914cd2 d85ea0562022 2024-09-16T00:44:03.939 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi112 smithi112 running (13m) 10m ago 26m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c e4fe58f9702a 2024-09-16T00:44:03.940 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi169 smithi169 running (13m) 9m ago 25m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c 3ba2c81d527f 2024-09-16T00:44:03.940 INFO:teuthology.orchestra.run.smithi112.stdout:grafana.smithi112 smithi112 *:3000 running (15m) 10m ago 25m 82.1M - 9.4.12 2bacad6d85d8 51f9d2013da2 2024-09-16T00:44:03.940 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.hxfwkh smithi112 running (10m) 10m ago 20m 14.2M - 19.1.1-326-g8921c8ef f1455bd0219c 1723069902d6 2024-09-16T00:44:03.940 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.ithwzj smithi112 running (10m) 10m ago 20m 12.2M - 19.1.1-326-g8921c8ef f1455bd0219c e7aad5b81db5 2024-09-16T00:44:03.940 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.gawwgw smithi169 running (9m) 9m ago 20m 15.7M - 19.1.1-326-g8921c8ef f1455bd0219c 469be459ee12 2024-09-16T00:44:03.940 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.lgeqrj smithi169 running (9m) 9m ago 20m 13.3M - 19.1.1-326-g8921c8ef f1455bd0219c 31f0fbd2395d 2024-09-16T00:44:03.940 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi112.xrkkbc smithi112 *:8443,9283,8765 running (16m) 10m ago 27m 504M - 19.1.1-326-g8921c8ef f1455bd0219c a981f7fd2810 2024-09-16T00:44:03.940 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi169.mtrobk smithi169 *:8443,9283,8765 running (16m) 9m ago 25m 442M - 19.1.1-326-g8921c8ef f1455bd0219c 4096e5369c2d 2024-09-16T00:44:03.940 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi112 smithi112 running (14m) 10m ago 27m 49.8M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 5aaec033c042 2024-09-16T00:44:03.940 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi169 smithi169 running (14m) 9m ago 25m 38.9M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 7bc609155f3f 2024-09-16T00:44:03.940 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi112 smithi112 *:9100 running (17m) 10m ago 26m 15.5M - 1.5.0 0da6a335fe13 6404d1d601e2 2024-09-16T00:44:03.940 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi169 smithi169 *:9100 running (16m) 9m ago 25m 15.4M - 1.5.0 0da6a335fe13 5505baf702a6 2024-09-16T00:44:03.941 INFO:teuthology.orchestra.run.smithi112.stdout:osd.0 smithi112 running (13m) 10m ago 24m 105M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 53efc9397017 2024-09-16T00:44:03.941 INFO:teuthology.orchestra.run.smithi112.stdout:osd.1 smithi112 running (12m) 10m ago 23m 98.9M 1557M 19.1.1-326-g8921c8ef f1455bd0219c fe6f1a4a2cf4 2024-09-16T00:44:03.941 INFO:teuthology.orchestra.run.smithi112.stdout:osd.2 smithi112 running (12m) 10m ago 23m 102M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 5f7863712057 2024-09-16T00:44:03.941 INFO:teuthology.orchestra.run.smithi112.stdout:osd.3 smithi169 running (11m) 9m ago 22m 92.1M 2581M 19.1.1-326-g8921c8ef f1455bd0219c ee07b5577810 2024-09-16T00:44:03.941 INFO:teuthology.orchestra.run.smithi112.stdout:osd.4 smithi169 running (11m) 9m ago 22m 103M 2581M 19.1.1-326-g8921c8ef f1455bd0219c 769f026a54eb 2024-09-16T00:44:03.941 INFO:teuthology.orchestra.run.smithi112.stdout:osd.5 smithi169 running (10m) 9m ago 22m 125M 2581M 19.1.1-326-g8921c8ef f1455bd0219c a6007c13cf78 2024-09-16T00:44:03.941 INFO:teuthology.orchestra.run.smithi112.stdout:prometheus.smithi112 smithi112 *:9095 running (16m) 10m ago 25m 64.1M - 2.43.0 a07b618ecd1d 7a1d42e60d2b 2024-09-16T00:44:03.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.941+0000 7f82aa5c2640 1 -- 172.21.15.112:0/726137115 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f829006e530 msgr2=0x7f82900709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:03.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.941+0000 7f82aa5c2640 1 --2- 172.21.15.112:0/726137115 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f829006e530 0x7f82900709e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f829400a8a0 tx=0x7f8294008040 comp rx=0 tx=0).stop 2024-09-16T00:44:03.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.941+0000 7f82aa5c2640 1 -- 172.21.15.112:0/726137115 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82a41014d0 msgr2=0x7f82a41126a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:03.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.941+0000 7f82aa5c2640 1 --2- 172.21.15.112:0/726137115 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82a41014d0 0x7f82a41126a0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f828c000c00 tx=0x7f828c009da0 comp rx=0 tx=0).stop 2024-09-16T00:44:03.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.941+0000 7f82aa5c2640 1 -- 172.21.15.112:0/726137115 shutdown_connections 2024-09-16T00:44:03.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.941+0000 7f82aa5c2640 1 --2- 172.21.15.112:0/726137115 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f829006e530 0x7f82900709e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:03.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.941+0000 7f82aa5c2640 1 --2- 172.21.15.112:0/726137115 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82a41014d0 0x7f82a41126a0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:03.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.941+0000 7f82aa5c2640 1 --2- 172.21.15.112:0/726137115 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82a40fb490 0x7f82a4112160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:03.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.941+0000 7f82aa5c2640 1 -- 172.21.15.112:0/726137115 >> 172.21.15.112:0/726137115 conn(0x7f82a40f7010 msgr2=0x7f82a40f8910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:03.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.942+0000 7f82aa5c2640 1 -- 172.21.15.112:0/726137115 shutdown_connections 2024-09-16T00:44:03.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:03.942+0000 7f82aa5c2640 1 -- 172.21.15.112:0/726137115 wait complete. 2024-09-16T00:44:04.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.018+0000 7f5a34fdd640 1 -- 172.21.15.112:0/4235634477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a30102550 msgr2=0x7f5a30102920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:04.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.018+0000 7f5a34fdd640 1 --2- 172.21.15.112:0/4235634477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a30102550 0x7f5a30102920 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f5a280048d0 tx=0x7f5a28009da0 comp rx=0 tx=0).stop 2024-09-16T00:44:04.019 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:44:03 smithi112 ceph-mon[97750]: pgmap v489: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:44:04.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.019+0000 7f5a34fdd640 1 -- 172.21.15.112:0/4235634477 shutdown_connections 2024-09-16T00:44:04.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.019+0000 7f5a34fdd640 1 --2- 172.21.15.112:0/4235634477 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a30102ef0 0x7f5a300fc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:04.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.019+0000 7f5a34fdd640 1 --2- 172.21.15.112:0/4235634477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a30102550 0x7f5a30102920 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:04.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.019+0000 7f5a34fdd640 1 -- 172.21.15.112:0/4235634477 >> 172.21.15.112:0/4235634477 conn(0x7f5a300f80f0 msgr2=0x7f5a300fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:04.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.019+0000 7f5a34fdd640 1 -- 172.21.15.112:0/4235634477 shutdown_connections 2024-09-16T00:44:04.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.019+0000 7f5a34fdd640 1 -- 172.21.15.112:0/4235634477 wait complete. 2024-09-16T00:44:04.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.019+0000 7f5a34fdd640 1 Processor -- start 2024-09-16T00:44:04.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.019+0000 7f5a34fdd640 1 -- start start 2024-09-16T00:44:04.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.020+0000 7f5a34fdd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a30102550 0x7f5a30113540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:04.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.020+0000 7f5a34fdd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a30102ef0 0x7f5a30113a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:04.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.020+0000 7f5a34fdd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a30114160 con 0x7f5a30102550 2024-09-16T00:44:04.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.020+0000 7f5a34fdd640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3010d550 con 0x7f5a30102ef0 2024-09-16T00:44:04.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.020+0000 7f5a2e59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a30102550 0x7f5a30113540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:04.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.020+0000 7f5a2e59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a30102550 0x7f5a30113540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50906/0 (socket says 172.21.15.112:50906) 2024-09-16T00:44:04.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.020+0000 7f5a2e59c640 1 -- 172.21.15.112:0/2920354913 learned_addr learned my addr 172.21.15.112:0/2920354913 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:44:04.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.020+0000 7f5a2dd9b640 1 --2- 172.21.15.112:0/2920354913 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a30102ef0 0x7f5a30113a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:04.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.020+0000 7f5a2e59c640 1 -- 172.21.15.112:0/2920354913 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a30102ef0 msgr2=0x7f5a30113a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:04.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.020+0000 7f5a2e59c640 1 --2- 172.21.15.112:0/2920354913 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a30102ef0 0x7f5a30113a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:04.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.020+0000 7f5a2e59c640 1 -- 172.21.15.112:0/2920354913 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a28005040 con 0x7f5a30102550 2024-09-16T00:44:04.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.020+0000 7f5a2e59c640 1 --2- 172.21.15.112:0/2920354913 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a30102550 0x7f5a30113540 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f5a28000c00 tx=0x7f5a28009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:04.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.021+0000 7f5a26ffd640 1 -- 172.21.15.112:0/2920354913 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a28038070 con 0x7f5a30102550 2024-09-16T00:44:04.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.021+0000 7f5a26ffd640 1 -- 172.21.15.112:0/2920354913 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f5a28033ad0 con 0x7f5a30102550 2024-09-16T00:44:04.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.021+0000 7f5a26ffd640 1 -- 172.21.15.112:0/2920354913 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a2803db70 con 0x7f5a30102550 2024-09-16T00:44:04.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.021+0000 7f5a34fdd640 1 -- 172.21.15.112:0/2920354913 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a3010d780 con 0x7f5a30102550 2024-09-16T00:44:04.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.021+0000 7f5a34fdd640 1 -- 172.21.15.112:0/2920354913 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a3010dca0 con 0x7f5a30102550 2024-09-16T00:44:04.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.022+0000 7f5a34fdd640 1 -- 172.21.15.112:0/2920354913 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a300fdcd0 con 0x7f5a30102550 2024-09-16T00:44:04.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.022+0000 7f5a26ffd640 1 -- 172.21.15.112:0/2920354913 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f5a28030020 con 0x7f5a30102550 2024-09-16T00:44:04.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.023+0000 7f5a26ffd640 1 --2- 172.21.15.112:0/2920354913 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5a1c06e530 0x7f5a1c0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:04.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.023+0000 7f5a26ffd640 1 -- 172.21.15.112:0/2920354913 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f5a280ae410 con 0x7f5a30102550 2024-09-16T00:44:04.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.023+0000 7f5a2dd9b640 1 --2- 172.21.15.112:0/2920354913 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5a1c06e530 0x7f5a1c0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:04.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.024+0000 7f5a2dd9b640 1 --2- 172.21.15.112:0/2920354913 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5a1c06e530 0x7f5a1c0709e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5a3010ef00 tx=0x7f5a18009120 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:04.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:44:03 smithi169 ceph-mon[77626]: pgmap v489: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:44:04.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.027+0000 7f5a26ffd640 1 -- 172.21.15.112:0/2920354913 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f5a28077400 con 0x7f5a30102550 2024-09-16T00:44:04.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.263+0000 7f5a34fdd640 1 -- 172.21.15.112:0/2920354913 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f5a3004b720 con 0x7f5a30102550 2024-09-16T00:44:04.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.264+0000 7f5a26ffd640 1 -- 172.21.15.112:0/2920354913 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f5a2807aac0 con 0x7f5a30102550 2024-09-16T00:44:04.264 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:44:04.264 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": { 2024-09-16T00:44:04.264 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:44:04.264 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:44:04.264 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": { 2024-09-16T00:44:04.264 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:44:04.264 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:44:04.264 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": { 2024-09-16T00:44:04.264 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 6 2024-09-16T00:44:04.264 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:44:04.264 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": { 2024-09-16T00:44:04.265 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 4 2024-09-16T00:44:04.265 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:44:04.265 INFO:teuthology.orchestra.run.smithi112.stdout: "overall": { 2024-09-16T00:44:04.265 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 14 2024-09-16T00:44:04.265 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-16T00:44:04.265 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:44:04.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.266+0000 7f5a34fdd640 1 -- 172.21.15.112:0/2920354913 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5a1c06e530 msgr2=0x7f5a1c0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:04.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.266+0000 7f5a34fdd640 1 --2- 172.21.15.112:0/2920354913 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5a1c06e530 0x7f5a1c0709e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5a3010ef00 tx=0x7f5a18009120 comp rx=0 tx=0).stop 2024-09-16T00:44:04.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.266+0000 7f5a34fdd640 1 -- 172.21.15.112:0/2920354913 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a30102550 msgr2=0x7f5a30113540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:04.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.266+0000 7f5a34fdd640 1 --2- 172.21.15.112:0/2920354913 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a30102550 0x7f5a30113540 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f5a28000c00 tx=0x7f5a28009da0 comp rx=0 tx=0).stop 2024-09-16T00:44:04.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.266+0000 7f5a34fdd640 1 -- 172.21.15.112:0/2920354913 shutdown_connections 2024-09-16T00:44:04.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.266+0000 7f5a34fdd640 1 --2- 172.21.15.112:0/2920354913 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5a1c06e530 0x7f5a1c0709e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:04.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.266+0000 7f5a34fdd640 1 --2- 172.21.15.112:0/2920354913 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a30102ef0 0x7f5a30113a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:04.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.266+0000 7f5a34fdd640 1 --2- 172.21.15.112:0/2920354913 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a30102550 0x7f5a30113540 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:04.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.266+0000 7f5a34fdd640 1 -- 172.21.15.112:0/2920354913 >> 172.21.15.112:0/2920354913 conn(0x7f5a300f80f0 msgr2=0x7f5a300f9180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:04.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.266+0000 7f5a34fdd640 1 -- 172.21.15.112:0/2920354913 shutdown_connections 2024-09-16T00:44:04.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.266+0000 7f5a34fdd640 1 -- 172.21.15.112:0/2920354913 wait complete. 2024-09-16T00:44:04.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.344+0000 7f2e3b3ab640 1 -- 172.21.15.112:0/1892811485 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e34070360 msgr2=0x7f2e341088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:04.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.344+0000 7f2e3b3ab640 1 --2- 172.21.15.112:0/1892811485 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e34070360 0x7f2e341088d0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f2e240048d0 tx=0x7f2e24009da0 comp rx=0 tx=0).stop 2024-09-16T00:44:04.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.344+0000 7f2e3b3ab640 1 -- 172.21.15.112:0/1892811485 shutdown_connections 2024-09-16T00:44:04.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.344+0000 7f2e3b3ab640 1 --2- 172.21.15.112:0/1892811485 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e34070360 0x7f2e341088d0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:04.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.344+0000 7f2e3b3ab640 1 --2- 172.21.15.112:0/1892811485 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e3406fa50 0x7f2e3406fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:04.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.344+0000 7f2e3b3ab640 1 -- 172.21.15.112:0/1892811485 >> 172.21.15.112:0/1892811485 conn(0x7f2e340f80f0 msgr2=0x7f2e340fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:04.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.344+0000 7f2e3b3ab640 1 -- 172.21.15.112:0/1892811485 shutdown_connections 2024-09-16T00:44:04.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.344+0000 7f2e3b3ab640 1 -- 172.21.15.112:0/1892811485 wait complete. 2024-09-16T00:44:04.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.345+0000 7f2e3b3ab640 1 Processor -- start 2024-09-16T00:44:04.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.345+0000 7f2e3b3ab640 1 -- start start 2024-09-16T00:44:04.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.345+0000 7f2e3b3ab640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e3406fa50 0x7f2e341153e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:04.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.345+0000 7f2e3b3ab640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e34070360 0x7f2e34115920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:04.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.345+0000 7f2e3b3ab640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e34115f20 con 0x7f2e34070360 2024-09-16T00:44:04.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.345+0000 7f2e3b3ab640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e34116090 con 0x7f2e3406fa50 2024-09-16T00:44:04.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.345+0000 7f2e38946640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e34070360 0x7f2e34115920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:04.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.345+0000 7f2e38946640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e34070360 0x7f2e34115920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50918/0 (socket says 172.21.15.112:50918) 2024-09-16T00:44:04.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.345+0000 7f2e38946640 1 -- 172.21.15.112:0/1527007629 learned_addr learned my addr 172.21.15.112:0/1527007629 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:44:04.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.345+0000 7f2e39147640 1 --2- 172.21.15.112:0/1527007629 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e3406fa50 0x7f2e341153e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:04.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.345+0000 7f2e38946640 1 -- 172.21.15.112:0/1527007629 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e3406fa50 msgr2=0x7f2e341153e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:04.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.345+0000 7f2e38946640 1 --2- 172.21.15.112:0/1527007629 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e3406fa50 0x7f2e341153e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:04.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.345+0000 7f2e38946640 1 -- 172.21.15.112:0/1527007629 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e24005040 con 0x7f2e34070360 2024-09-16T00:44:04.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.346+0000 7f2e38946640 1 --2- 172.21.15.112:0/1527007629 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e34070360 0x7f2e34115920 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f2e24000c00 tx=0x7f2e24009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:04.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.346+0000 7f2e29ffb640 1 -- 172.21.15.112:0/1527007629 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e24038070 con 0x7f2e34070360 2024-09-16T00:44:04.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.346+0000 7f2e3b3ab640 1 -- 172.21.15.112:0/1527007629 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e3410f4b0 con 0x7f2e34070360 2024-09-16T00:44:04.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.346+0000 7f2e29ffb640 1 -- 172.21.15.112:0/1527007629 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f2e2402ade0 con 0x7f2e34070360 2024-09-16T00:44:04.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.346+0000 7f2e29ffb640 1 -- 172.21.15.112:0/1527007629 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e2402c7e0 con 0x7f2e34070360 2024-09-16T00:44:04.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.346+0000 7f2e3b3ab640 1 -- 172.21.15.112:0/1527007629 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e3410f970 con 0x7f2e34070360 2024-09-16T00:44:04.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.347+0000 7f2e3b3ab640 1 -- 172.21.15.112:0/1527007629 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e34071190 con 0x7f2e34070360 2024-09-16T00:44:04.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.347+0000 7f2e29ffb640 1 -- 172.21.15.112:0/1527007629 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f2e24030020 con 0x7f2e34070360 2024-09-16T00:44:04.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.348+0000 7f2e29ffb640 1 --2- 172.21.15.112:0/1527007629 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f2e2006e530 0x7f2e200709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:04.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.348+0000 7f2e29ffb640 1 -- 172.21.15.112:0/1527007629 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f2e240aded0 con 0x7f2e34070360 2024-09-16T00:44:04.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.348+0000 7f2e39147640 1 --2- 172.21.15.112:0/1527007629 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f2e2006e530 0x7f2e200709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:04.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.348+0000 7f2e39147640 1 --2- 172.21.15.112:0/1527007629 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f2e2006e530 0x7f2e200709e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2e30007fd0 tx=0x7f2e30009e90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:04.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.352+0000 7f2e29ffb640 1 -- 172.21.15.112:0/1527007629 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f2e24076f70 con 0x7f2e34070360 2024-09-16T00:44:04.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.552+0000 7f2e3b3ab640 1 -- 172.21.15.112:0/1527007629 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f2e340fbd30 con 0x7f2e34070360 2024-09-16T00:44:04.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.552+0000 7f2e29ffb640 1 -- 172.21.15.112:0/1527007629 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1914 (secure 0 0 0) 0x7f2e2407a630 con 0x7f2e34070360 2024-09-16T00:44:04.553 INFO:teuthology.orchestra.run.smithi112.stdout:e34 2024-09-16T00:44:04.553 INFO:teuthology.orchestra.run.smithi112.stdout:btime 2024-09-16T00:34:26:619254+0000 2024-09-16T00:44:04.553 INFO:teuthology.orchestra.run.smithi112.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T00:44:04.553 INFO:teuthology.orchestra.run.smithi112.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:44:04.553 INFO:teuthology.orchestra.run.smithi112.stdout:legacy client fscid: 1 2024-09-16T00:44:04.553 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:44:04.553 INFO:teuthology.orchestra.run.smithi112.stdout:Filesystem 'cephfs' (1) 2024-09-16T00:44:04.553 INFO:teuthology.orchestra.run.smithi112.stdout:fs_name cephfs 2024-09-16T00:44:04.553 INFO:teuthology.orchestra.run.smithi112.stdout:epoch 25 2024-09-16T00:44:04.554 INFO:teuthology.orchestra.run.smithi112.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-16T00:44:04.554 INFO:teuthology.orchestra.run.smithi112.stdout:created 2024-09-16T00:23:22.328199+0000 2024-09-16T00:44:04.554 INFO:teuthology.orchestra.run.smithi112.stdout:modified 2024-09-16T00:33:29.012991+0000 2024-09-16T00:44:04.554 INFO:teuthology.orchestra.run.smithi112.stdout:tableserver 0 2024-09-16T00:44:04.554 INFO:teuthology.orchestra.run.smithi112.stdout:root 0 2024-09-16T00:44:04.554 INFO:teuthology.orchestra.run.smithi112.stdout:session_timeout 60 2024-09-16T00:44:04.554 INFO:teuthology.orchestra.run.smithi112.stdout:session_autoclose 300 2024-09-16T00:44:04.554 INFO:teuthology.orchestra.run.smithi112.stdout:max_file_size 1099511627776 2024-09-16T00:44:04.554 INFO:teuthology.orchestra.run.smithi112.stdout:max_xattr_size 65536 2024-09-16T00:44:04.554 INFO:teuthology.orchestra.run.smithi112.stdout:required_client_features {} 2024-09-16T00:44:04.554 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure 0 2024-09-16T00:44:04.554 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure_osd_epoch 82 2024-09-16T00:44:04.554 INFO:teuthology.orchestra.run.smithi112.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:44:04.554 INFO:teuthology.orchestra.run.smithi112.stdout:max_mds 2 2024-09-16T00:44:04.555 INFO:teuthology.orchestra.run.smithi112.stdout:in 0,1 2024-09-16T00:44:04.555 INFO:teuthology.orchestra.run.smithi112.stdout:up {} 2024-09-16T00:44:04.555 INFO:teuthology.orchestra.run.smithi112.stdout:failed 0,1 2024-09-16T00:44:04.555 INFO:teuthology.orchestra.run.smithi112.stdout:damaged 2024-09-16T00:44:04.555 INFO:teuthology.orchestra.run.smithi112.stdout:stopped 2024-09-16T00:44:04.555 INFO:teuthology.orchestra.run.smithi112.stdout:data_pools [3] 2024-09-16T00:44:04.555 INFO:teuthology.orchestra.run.smithi112.stdout:metadata_pool 2 2024-09-16T00:44:04.555 INFO:teuthology.orchestra.run.smithi112.stdout:inline_data enabled 2024-09-16T00:44:04.555 INFO:teuthology.orchestra.run.smithi112.stdout:balancer 2024-09-16T00:44:04.555 INFO:teuthology.orchestra.run.smithi112.stdout:bal_rank_mask -1 2024-09-16T00:44:04.555 INFO:teuthology.orchestra.run.smithi112.stdout:standby_count_wanted 1 2024-09-16T00:44:04.555 INFO:teuthology.orchestra.run.smithi112.stdout:qdb_cluster leader: 0 members: 2024-09-16T00:44:04.555 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:44:04.555 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:44:04.556 INFO:teuthology.orchestra.run.smithi112.stdout:Standby daemons: 2024-09-16T00:44:04.556 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:44:04.556 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.ithwzj{-1:34392} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.112:6826/476229221,v1:172.21.15.112:6827/476229221] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:44:04.556 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.hxfwkh{-1:34404} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.112:6828/2356899876,v1:172.21.15.112:6829/2356899876] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:44:04.556 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.gawwgw{-1:44343} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.169:6824/1635329957,v1:172.21.15.169:6825/1635329957] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:44:04.556 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.lgeqrj{-1:44347} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.169:6826/184810031,v1:172.21.15.169:6827/184810031] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:44:04.557 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 34 2024-09-16T00:44:04.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.554+0000 7f2e3b3ab640 1 -- 172.21.15.112:0/1527007629 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f2e2006e530 msgr2=0x7f2e200709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:04.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.554+0000 7f2e3b3ab640 1 --2- 172.21.15.112:0/1527007629 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f2e2006e530 0x7f2e200709e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2e30007fd0 tx=0x7f2e30009e90 comp rx=0 tx=0).stop 2024-09-16T00:44:04.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.554+0000 7f2e3b3ab640 1 -- 172.21.15.112:0/1527007629 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e34070360 msgr2=0x7f2e34115920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:04.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.554+0000 7f2e3b3ab640 1 --2- 172.21.15.112:0/1527007629 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e34070360 0x7f2e34115920 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f2e24000c00 tx=0x7f2e24009da0 comp rx=0 tx=0).stop 2024-09-16T00:44:04.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.554+0000 7f2e3b3ab640 1 -- 172.21.15.112:0/1527007629 shutdown_connections 2024-09-16T00:44:04.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.554+0000 7f2e3b3ab640 1 --2- 172.21.15.112:0/1527007629 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f2e2006e530 0x7f2e200709e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:04.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.554+0000 7f2e3b3ab640 1 --2- 172.21.15.112:0/1527007629 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e34070360 0x7f2e34115920 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:04.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.554+0000 7f2e3b3ab640 1 --2- 172.21.15.112:0/1527007629 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e3406fa50 0x7f2e341153e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:04.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.554+0000 7f2e3b3ab640 1 -- 172.21.15.112:0/1527007629 >> 172.21.15.112:0/1527007629 conn(0x7f2e340f80f0 msgr2=0x7f2e340f9640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:04.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.555+0000 7f2e3b3ab640 1 -- 172.21.15.112:0/1527007629 shutdown_connections 2024-09-16T00:44:04.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.555+0000 7f2e3b3ab640 1 -- 172.21.15.112:0/1527007629 wait complete. 2024-09-16T00:44:04.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.633+0000 7f8318dc6640 1 -- 172.21.15.112:0/3062172095 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83140fac80 msgr2=0x7f83140fb0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:04.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.633+0000 7f8318dc6640 1 --2- 172.21.15.112:0/3062172095 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83140fac80 0x7f83140fb0d0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f83080048d0 tx=0x7f8308009da0 comp rx=0 tx=0).stop 2024-09-16T00:44:04.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.633+0000 7f8318dc6640 1 -- 172.21.15.112:0/3062172095 shutdown_connections 2024-09-16T00:44:04.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.633+0000 7f8318dc6640 1 --2- 172.21.15.112:0/3062172095 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83140fac80 0x7f83140fb0d0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:04.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.633+0000 7f8318dc6640 1 --2- 172.21.15.112:0/3062172095 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8314102660 0x7f8314102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:04.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.633+0000 7f8318dc6640 1 -- 172.21.15.112:0/3062172095 >> 172.21.15.112:0/3062172095 conn(0x7f8314070ce0 msgr2=0x7f83140710e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:04.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.633+0000 7f8318dc6640 1 -- 172.21.15.112:0/3062172095 shutdown_connections 2024-09-16T00:44:04.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.633+0000 7f8318dc6640 1 -- 172.21.15.112:0/3062172095 wait complete. 2024-09-16T00:44:04.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.634+0000 7f8318dc6640 1 Processor -- start 2024-09-16T00:44:04.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.634+0000 7f8318dc6640 1 -- start start 2024-09-16T00:44:04.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.634+0000 7f8318dc6640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83140fac80 0x7f83140fbe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:04.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.634+0000 7f8318dc6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8314102660 0x7f83140fc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:04.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.634+0000 7f8318dc6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83140fc8e0 con 0x7f8314102660 2024-09-16T00:44:04.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.634+0000 7f8318dc6640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83140fca50 con 0x7f83140fac80 2024-09-16T00:44:04.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.634+0000 7f8311d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8314102660 0x7f83140fc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:04.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.634+0000 7f8311d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8314102660 0x7f83140fc3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50928/0 (socket says 172.21.15.112:50928) 2024-09-16T00:44:04.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.634+0000 7f8311d9b640 1 -- 172.21.15.112:0/364809469 learned_addr learned my addr 172.21.15.112:0/364809469 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:44:04.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.634+0000 7f831259c640 1 --2- 172.21.15.112:0/364809469 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83140fac80 0x7f83140fbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:04.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.635+0000 7f8311d9b640 1 -- 172.21.15.112:0/364809469 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83140fac80 msgr2=0x7f83140fbe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:04.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.635+0000 7f8311d9b640 1 --2- 172.21.15.112:0/364809469 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83140fac80 0x7f83140fbe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:04.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.635+0000 7f8311d9b640 1 -- 172.21.15.112:0/364809469 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8308005040 con 0x7f8314102660 2024-09-16T00:44:04.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.635+0000 7f8311d9b640 1 --2- 172.21.15.112:0/364809469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8314102660 0x7f83140fc3a0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f8308000c00 tx=0x7f8308009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:04.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.635+0000 7f8302ffd640 1 -- 172.21.15.112:0/364809469 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8308038070 con 0x7f8314102660 2024-09-16T00:44:04.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.635+0000 7f8302ffd640 1 -- 172.21.15.112:0/364809469 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f830802ade0 con 0x7f8314102660 2024-09-16T00:44:04.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.635+0000 7f8318dc6640 1 -- 172.21.15.112:0/364809469 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83140fcc80 con 0x7f8314102660 2024-09-16T00:44:04.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.635+0000 7f8302ffd640 1 -- 172.21.15.112:0/364809469 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f830802c7e0 con 0x7f8314102660 2024-09-16T00:44:04.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.635+0000 7f8318dc6640 1 -- 172.21.15.112:0/364809469 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8314119870 con 0x7f8314102660 2024-09-16T00:44:04.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.636+0000 7f8318dc6640 1 -- 172.21.15.112:0/364809469 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83140657d0 con 0x7f8314102660 2024-09-16T00:44:04.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.637+0000 7f8302ffd640 1 -- 172.21.15.112:0/364809469 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f8308030020 con 0x7f8314102660 2024-09-16T00:44:04.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.637+0000 7f8302ffd640 1 --2- 172.21.15.112:0/364809469 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f82fc06e530 0x7f82fc0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:04.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.637+0000 7f8302ffd640 1 -- 172.21.15.112:0/364809469 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f83080aded0 con 0x7f8314102660 2024-09-16T00:44:04.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.637+0000 7f831259c640 1 --2- 172.21.15.112:0/364809469 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f82fc06e530 0x7f82fc0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:04.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.638+0000 7f831259c640 1 --2- 172.21.15.112:0/364809469 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f82fc06e530 0x7f82fc0709e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8304004ed0 tx=0x7f830400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:04.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.642+0000 7f8302ffd640 1 -- 172.21.15.112:0/364809469 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f8308076f70 con 0x7f8314102660 2024-09-16T00:44:04.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.828+0000 7f8318dc6640 1 -- 172.21.15.112:0/364809469 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f831404b720 con 0x7f82fc06e530 2024-09-16T00:44:04.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.830+0000 7f8302ffd640 1 -- 172.21.15.112:0/364809469 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f831404b720 con 0x7f82fc06e530 2024-09-16T00:44:04.830 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:44:04.830 INFO:teuthology.orchestra.run.smithi112.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80", 2024-09-16T00:44:04.830 INFO:teuthology.orchestra.run.smithi112.stdout: "in_progress": true, 2024-09-16T00:44:04.830 INFO:teuthology.orchestra.run.smithi112.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T00:44:04.830 INFO:teuthology.orchestra.run.smithi112.stdout: "services_complete": [ 2024-09-16T00:44:04.830 INFO:teuthology.orchestra.run.smithi112.stdout: "mds", 2024-09-16T00:44:04.830 INFO:teuthology.orchestra.run.smithi112.stdout: "osd", 2024-09-16T00:44:04.830 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr", 2024-09-16T00:44:04.830 INFO:teuthology.orchestra.run.smithi112.stdout: "mon", 2024-09-16T00:44:04.831 INFO:teuthology.orchestra.run.smithi112.stdout: "crash" 2024-09-16T00:44:04.831 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-16T00:44:04.831 INFO:teuthology.orchestra.run.smithi112.stdout: "progress": "16/21 daemons upgraded", 2024-09-16T00:44:04.831 INFO:teuthology.orchestra.run.smithi112.stdout: "message": "Currently upgrading mds daemons", 2024-09-16T00:44:04.831 INFO:teuthology.orchestra.run.smithi112.stdout: "is_paused": false 2024-09-16T00:44:04.831 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:44:04.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.832+0000 7f8318dc6640 1 -- 172.21.15.112:0/364809469 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f82fc06e530 msgr2=0x7f82fc0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:04.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.832+0000 7f8318dc6640 1 --2- 172.21.15.112:0/364809469 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f82fc06e530 0x7f82fc0709e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8304004ed0 tx=0x7f830400b040 comp rx=0 tx=0).stop 2024-09-16T00:44:04.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.832+0000 7f8318dc6640 1 -- 172.21.15.112:0/364809469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8314102660 msgr2=0x7f83140fc3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:04.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.832+0000 7f8318dc6640 1 --2- 172.21.15.112:0/364809469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8314102660 0x7f83140fc3a0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f8308000c00 tx=0x7f8308009da0 comp rx=0 tx=0).stop 2024-09-16T00:44:04.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.832+0000 7f8318dc6640 1 -- 172.21.15.112:0/364809469 shutdown_connections 2024-09-16T00:44:04.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.832+0000 7f8318dc6640 1 --2- 172.21.15.112:0/364809469 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f82fc06e530 0x7f82fc0709e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:04.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.832+0000 7f8318dc6640 1 --2- 172.21.15.112:0/364809469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8314102660 0x7f83140fc3a0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:04.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.832+0000 7f8318dc6640 1 --2- 172.21.15.112:0/364809469 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83140fac80 0x7f83140fbe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:04.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.832+0000 7f8318dc6640 1 -- 172.21.15.112:0/364809469 >> 172.21.15.112:0/364809469 conn(0x7f8314070ce0 msgr2=0x7f83140fa0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:04.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.832+0000 7f8318dc6640 1 -- 172.21.15.112:0/364809469 shutdown_connections 2024-09-16T00:44:04.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.832+0000 7f8318dc6640 1 -- 172.21.15.112:0/364809469 wait complete. 2024-09-16T00:44:04.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.911+0000 7f75e6661640 1 -- 172.21.15.112:0/4115589778 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75e0102550 msgr2=0x7f75e0102920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:04.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.911+0000 7f75e6661640 1 --2- 172.21.15.112:0/4115589778 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75e0102550 0x7f75e0102920 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f75d00048d0 tx=0x7f75d0009da0 comp rx=0 tx=0).stop 2024-09-16T00:44:04.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.911+0000 7f75e6661640 1 -- 172.21.15.112:0/4115589778 shutdown_connections 2024-09-16T00:44:04.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.911+0000 7f75e6661640 1 --2- 172.21.15.112:0/4115589778 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75e0102ef0 0x7f75e00fc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:04.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.911+0000 7f75e6661640 1 --2- 172.21.15.112:0/4115589778 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75e0102550 0x7f75e0102920 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:04.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.911+0000 7f75e6661640 1 -- 172.21.15.112:0/4115589778 >> 172.21.15.112:0/4115589778 conn(0x7f75e00f80f0 msgr2=0x7f75e00fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:04.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.911+0000 7f75e6661640 1 -- 172.21.15.112:0/4115589778 shutdown_connections 2024-09-16T00:44:04.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.912+0000 7f75e6661640 1 -- 172.21.15.112:0/4115589778 wait complete. 2024-09-16T00:44:04.913 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:44:04 smithi112 ceph-mon[97750]: from='client.34940 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:44:04.913 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:44:04 smithi112 ceph-mon[97750]: from='client.34944 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:44:04.914 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:44:04 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/2920354913' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:44:04.914 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:44:04 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/1527007629' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:44:04.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.912+0000 7f75e6661640 1 Processor -- start 2024-09-16T00:44:04.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.912+0000 7f75e6661640 1 -- start start 2024-09-16T00:44:04.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.913+0000 7f75e6661640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75e0102550 0x7f75e01135b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:04.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.913+0000 7f75e6661640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75e0102ef0 0x7f75e0113af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:04.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.913+0000 7f75e6661640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e01141d0 con 0x7f75e0102550 2024-09-16T00:44:04.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.913+0000 7f75e6661640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e010d550 con 0x7f75e0102ef0 2024-09-16T00:44:04.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.913+0000 7f75dffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75e0102550 0x7f75e01135b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:04.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.913+0000 7f75dffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75e0102550 0x7f75e01135b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50948/0 (socket says 172.21.15.112:50948) 2024-09-16T00:44:04.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.913+0000 7f75dffff640 1 -- 172.21.15.112:0/2541402366 learned_addr learned my addr 172.21.15.112:0/2541402366 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:44:04.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.913+0000 7f75df7fe640 1 --2- 172.21.15.112:0/2541402366 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75e0102ef0 0x7f75e0113af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:04.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.913+0000 7f75dffff640 1 -- 172.21.15.112:0/2541402366 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75e0102ef0 msgr2=0x7f75e0113af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:04.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.913+0000 7f75dffff640 1 --2- 172.21.15.112:0/2541402366 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75e0102ef0 0x7f75e0113af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:04.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.913+0000 7f75dffff640 1 -- 172.21.15.112:0/2541402366 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75d0005040 con 0x7f75e0102550 2024-09-16T00:44:04.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.913+0000 7f75dffff640 1 --2- 172.21.15.112:0/2541402366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75e0102550 0x7f75e01135b0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f75d0000c00 tx=0x7f75d0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:04.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.914+0000 7f75dcff9640 1 -- 172.21.15.112:0/2541402366 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f75d0038070 con 0x7f75e0102550 2024-09-16T00:44:04.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.914+0000 7f75dcff9640 1 -- 172.21.15.112:0/2541402366 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f75d003daf0 con 0x7f75e0102550 2024-09-16T00:44:04.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.914+0000 7f75e6661640 1 -- 172.21.15.112:0/2541402366 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75e010d780 con 0x7f75e0102550 2024-09-16T00:44:04.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.914+0000 7f75dcff9640 1 -- 172.21.15.112:0/2541402366 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f75d0033b20 con 0x7f75e0102550 2024-09-16T00:44:04.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.915+0000 7f75e6661640 1 -- 172.21.15.112:0/2541402366 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75e010dca0 con 0x7f75e0102550 2024-09-16T00:44:04.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.915+0000 7f75dcff9640 1 -- 172.21.15.112:0/2541402366 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f75d0030020 con 0x7f75e0102550 2024-09-16T00:44:04.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.916+0000 7f75e6661640 1 -- 172.21.15.112:0/2541402366 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75e00fdcd0 con 0x7f75e0102550 2024-09-16T00:44:04.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.916+0000 7f75dcff9640 1 --2- 172.21.15.112:0/2541402366 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f75c806e530 0x7f75c80709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:04.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.916+0000 7f75dcff9640 1 -- 172.21.15.112:0/2541402366 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f75d00adc00 con 0x7f75e0102550 2024-09-16T00:44:04.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.916+0000 7f75df7fe640 1 --2- 172.21.15.112:0/2541402366 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f75c806e530 0x7f75c80709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:04.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.917+0000 7f75df7fe640 1 --2- 172.21.15.112:0/2541402366 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f75c806e530 0x7f75c80709e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f75e010e830 tx=0x7f75d4009120 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:04.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:04.921+0000 7f75dcff9640 1 -- 172.21.15.112:0/2541402366 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f75d0076ae0 con 0x7f75e0102550 2024-09-16T00:44:05.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:05.152+0000 7f75e6661640 1 -- 172.21.15.112:0/2541402366 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f75e010ec20 con 0x7f75e0102550 2024-09-16T00:44:05.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:05.152+0000 7f75dcff9640 1 -- 172.21.15.112:0/2541402366 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f75d007a1a0 con 0x7f75e0102550 2024-09-16T00:44:05.152 INFO:teuthology.orchestra.run.smithi112.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-16T00:44:05.152 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-16T00:44:05.153 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs is degraded 2024-09-16T00:44:05.153 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T00:44:05.153 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T00:44:05.153 INFO:teuthology.orchestra.run.smithi112.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-16T00:44:05.153 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-16T00:44:05.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:05.154+0000 7f75e6661640 1 -- 172.21.15.112:0/2541402366 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f75c806e530 msgr2=0x7f75c80709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:05.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:05.154+0000 7f75e6661640 1 --2- 172.21.15.112:0/2541402366 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f75c806e530 0x7f75c80709e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f75e010e830 tx=0x7f75d4009120 comp rx=0 tx=0).stop 2024-09-16T00:44:05.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:05.154+0000 7f75e6661640 1 -- 172.21.15.112:0/2541402366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75e0102550 msgr2=0x7f75e01135b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:05.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:05.154+0000 7f75e6661640 1 --2- 172.21.15.112:0/2541402366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75e0102550 0x7f75e01135b0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f75d0000c00 tx=0x7f75d0009da0 comp rx=0 tx=0).stop 2024-09-16T00:44:05.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:05.154+0000 7f75e6661640 1 -- 172.21.15.112:0/2541402366 shutdown_connections 2024-09-16T00:44:05.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:05.154+0000 7f75e6661640 1 --2- 172.21.15.112:0/2541402366 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f75c806e530 0x7f75c80709e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:05.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:05.154+0000 7f75e6661640 1 --2- 172.21.15.112:0/2541402366 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75e0102ef0 0x7f75e0113af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:05.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:05.154+0000 7f75e6661640 1 --2- 172.21.15.112:0/2541402366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75e0102550 0x7f75e01135b0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:05.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:05.154+0000 7f75e6661640 1 -- 172.21.15.112:0/2541402366 >> 172.21.15.112:0/2541402366 conn(0x7f75e00f80f0 msgr2=0x7f75e00f9180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:05.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:05.154+0000 7f75e6661640 1 -- 172.21.15.112:0/2541402366 shutdown_connections 2024-09-16T00:44:05.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:05.154+0000 7f75e6661640 1 -- 172.21.15.112:0/2541402366 wait complete. 2024-09-16T00:44:05.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:44:04 smithi169 ceph-mon[77626]: from='client.34940 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:44:05.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:44:04 smithi169 ceph-mon[77626]: from='client.34944 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:44:05.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:44:04 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/2920354913' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:44:05.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:44:04 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/1527007629' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:44:06.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:44:05 smithi112 ceph-mon[97750]: from='client.34948 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:44:06.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:44:05 smithi112 ceph-mon[97750]: pgmap v490: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:44:06.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:44:05 smithi112 ceph-mon[97750]: from='client.34960 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:44:06.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:44:05 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/2541402366' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:44:06.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:44:05 smithi169 ceph-mon[77626]: from='client.34948 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:44:06.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:44:05 smithi169 ceph-mon[77626]: pgmap v490: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:44:06.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:44:05 smithi169 ceph-mon[77626]: from='client.34960 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:44:06.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:44:05 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/2541402366' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:44:08.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:44:07 smithi112 ceph-mon[97750]: pgmap v491: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:44:08.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:44:07 smithi169 ceph-mon[77626]: pgmap v491: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:44:10.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:44:09 smithi112 ceph-mon[97750]: pgmap v492: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:44:10.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:44:09 smithi169 ceph-mon[77626]: pgmap v492: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:44:12.088 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:44:11 smithi112 ceph-mon[97750]: pgmap v493: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:44:12.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:44:11 smithi169 ceph-mon[77626]: pgmap v493: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:44:14.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:44:13 smithi112 ceph-mon[97750]: pgmap v494: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:44:14.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:44:13 smithi169 ceph-mon[77626]: pgmap v494: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:44:16.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:44:15 smithi112 ceph-mon[97750]: pgmap v495: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:44:16.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:44:15 smithi169 ceph-mon[77626]: pgmap v495: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:44:17.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:44:16 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:44:17.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:44:16 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:44:18.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:44:17 smithi112 ceph-mon[97750]: pgmap v496: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:44:18.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:44:17 smithi169 ceph-mon[77626]: pgmap v496: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:44:20.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:44:19 smithi112 ceph-mon[97750]: pgmap v497: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:44:20.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:44:19 smithi169 ceph-mon[77626]: pgmap v497: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:44:22.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:44:21 smithi112 ceph-mon[97750]: pgmap v498: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:44:22.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:44:21 smithi169 ceph-mon[77626]: pgmap v498: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:44:24.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:44:23 smithi112 ceph-mon[97750]: pgmap v499: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:44:24.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:44:23 smithi169 ceph-mon[77626]: pgmap v499: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:44:26.078 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:44:25 smithi169 ceph-mon[77626]: pgmap v500: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:44:26.088 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:44:25 smithi112 ceph-mon[97750]: pgmap v500: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:44:28.092 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:44:27 smithi112 ceph-mon[97750]: pgmap v501: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:44:28.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:44:27 smithi169 ceph-mon[77626]: pgmap v501: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:44:30.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:44:29 smithi112 ceph-mon[97750]: pgmap v502: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:44:30.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:44:29 smithi169 ceph-mon[77626]: pgmap v502: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:44:32.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:44:31 smithi112 ceph-mon[97750]: pgmap v503: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:44:32.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:44:31 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:44:32.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:44:31 smithi169 ceph-mon[77626]: pgmap v503: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:44:32.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:44:31 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:44:34.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:44:33 smithi112 ceph-mon[97750]: pgmap v504: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:44:34.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:44:33 smithi169 ceph-mon[77626]: pgmap v504: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:44:35.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.244+0000 7f23b17f7640 1 -- 172.21.15.112:0/851029626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23ac0fb480 msgr2=0x7f23ac0fb8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:35.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.244+0000 7f23b17f7640 1 --2- 172.21.15.112:0/851029626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23ac0fb480 0x7f23ac0fb8f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f23940048d0 tx=0x7f2394009da0 comp rx=0 tx=0).stop 2024-09-16T00:44:35.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.244+0000 7f23b17f7640 1 -- 172.21.15.112:0/851029626 shutdown_connections 2024-09-16T00:44:35.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.244+0000 7f23b17f7640 1 --2- 172.21.15.112:0/851029626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23ac0fb480 0x7f23ac0fb8f0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:35.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.244+0000 7f23b17f7640 1 --2- 172.21.15.112:0/851029626 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23ac1014c0 0x7f23ac101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:35.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.244+0000 7f23b17f7640 1 -- 172.21.15.112:0/851029626 >> 172.21.15.112:0/851029626 conn(0x7f23ac0f7000 msgr2=0x7f23ac0f9410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:35.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.244+0000 7f23b17f7640 1 -- 172.21.15.112:0/851029626 shutdown_connections 2024-09-16T00:44:35.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.244+0000 7f23b17f7640 1 -- 172.21.15.112:0/851029626 wait complete. 2024-09-16T00:44:35.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.245+0000 7f23b17f7640 1 Processor -- start 2024-09-16T00:44:35.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.245+0000 7f23b17f7640 1 -- start start 2024-09-16T00:44:35.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.245+0000 7f23b17f7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23ac0fb480 0x7f23ac10e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:35.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.245+0000 7f23b17f7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23ac1014c0 0x7f23ac10e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:35.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.245+0000 7f23b17f7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23ac10ed40 con 0x7f23ac1014c0 2024-09-16T00:44:35.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.245+0000 7f23b17f7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23ac10eeb0 con 0x7f23ac0fb480 2024-09-16T00:44:35.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.245+0000 7f23aa7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23ac1014c0 0x7f23ac10e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:35.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.245+0000 7f23aa7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23ac1014c0 0x7f23ac10e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33310/0 (socket says 172.21.15.112:33310) 2024-09-16T00:44:35.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.245+0000 7f23aa7fc640 1 -- 172.21.15.112:0/3091496162 learned_addr learned my addr 172.21.15.112:0/3091496162 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:44:35.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.245+0000 7f23aaffd640 1 --2- 172.21.15.112:0/3091496162 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23ac0fb480 0x7f23ac10e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:35.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.245+0000 7f23aa7fc640 1 -- 172.21.15.112:0/3091496162 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23ac0fb480 msgr2=0x7f23ac10e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:35.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.245+0000 7f23aa7fc640 1 --2- 172.21.15.112:0/3091496162 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23ac0fb480 0x7f23ac10e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:35.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.245+0000 7f23aa7fc640 1 -- 172.21.15.112:0/3091496162 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2394005040 con 0x7f23ac1014c0 2024-09-16T00:44:35.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.245+0000 7f23aaffd640 1 --2- 172.21.15.112:0/3091496162 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23ac0fb480 0x7f23ac10e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:44:35.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.246+0000 7f23aa7fc640 1 --2- 172.21.15.112:0/3091496162 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23ac1014c0 0x7f23ac10e6f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f2394000c00 tx=0x7f2394009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:35.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.246+0000 7f23a37fe640 1 -- 172.21.15.112:0/3091496162 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2394038070 con 0x7f23ac1014c0 2024-09-16T00:44:35.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.246+0000 7f23b17f7640 1 -- 172.21.15.112:0/3091496162 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23ac117220 con 0x7f23ac1014c0 2024-09-16T00:44:35.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.246+0000 7f23a37fe640 1 -- 172.21.15.112:0/3091496162 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f239402ade0 con 0x7f23ac1014c0 2024-09-16T00:44:35.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.246+0000 7f23a37fe640 1 -- 172.21.15.112:0/3091496162 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f239402c7e0 con 0x7f23ac1014c0 2024-09-16T00:44:35.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.246+0000 7f23b17f7640 1 -- 172.21.15.112:0/3091496162 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23ac117680 con 0x7f23ac1014c0 2024-09-16T00:44:35.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.247+0000 7f23b17f7640 1 -- 172.21.15.112:0/3091496162 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23ac0fcbc0 con 0x7f23ac1014c0 2024-09-16T00:44:35.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.248+0000 7f23a37fe640 1 -- 172.21.15.112:0/3091496162 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f2394030020 con 0x7f23ac1014c0 2024-09-16T00:44:35.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.248+0000 7f23a37fe640 1 --2- 172.21.15.112:0/3091496162 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f239806e530 0x7f23980709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:35.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.248+0000 7f23aaffd640 1 --2- 172.21.15.112:0/3091496162 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f239806e530 0x7f23980709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:35.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.248+0000 7f23a37fe640 1 -- 172.21.15.112:0/3091496162 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f23940adf70 con 0x7f23ac1014c0 2024-09-16T00:44:35.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.249+0000 7f23aaffd640 1 --2- 172.21.15.112:0/3091496162 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f239806e530 0x7f23980709e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f239c007ee0 tx=0x7f239c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:35.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.253+0000 7f23a37fe640 1 -- 172.21.15.112:0/3091496162 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f23940e31b0 con 0x7f23ac1014c0 2024-09-16T00:44:35.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.439+0000 7f23b17f7640 1 -- 172.21.15.112:0/3091496162 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f23ac04b790 con 0x7f239806e530 2024-09-16T00:44:35.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.440+0000 7f23a37fe640 1 -- 172.21.15.112:0/3091496162 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f23ac04b790 con 0x7f239806e530 2024-09-16T00:44:35.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.442+0000 7f23b17f7640 1 -- 172.21.15.112:0/3091496162 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f239806e530 msgr2=0x7f23980709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:35.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.442+0000 7f23b17f7640 1 --2- 172.21.15.112:0/3091496162 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f239806e530 0x7f23980709e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f239c007ee0 tx=0x7f239c00b040 comp rx=0 tx=0).stop 2024-09-16T00:44:35.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.442+0000 7f23b17f7640 1 -- 172.21.15.112:0/3091496162 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23ac1014c0 msgr2=0x7f23ac10e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:35.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.442+0000 7f23b17f7640 1 --2- 172.21.15.112:0/3091496162 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23ac1014c0 0x7f23ac10e6f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f2394000c00 tx=0x7f2394009da0 comp rx=0 tx=0).stop 2024-09-16T00:44:35.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.442+0000 7f23b17f7640 1 -- 172.21.15.112:0/3091496162 shutdown_connections 2024-09-16T00:44:35.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.442+0000 7f23b17f7640 1 --2- 172.21.15.112:0/3091496162 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f239806e530 0x7f23980709e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:35.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.442+0000 7f23b17f7640 1 --2- 172.21.15.112:0/3091496162 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23ac1014c0 0x7f23ac10e6f0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:35.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.442+0000 7f23b17f7640 1 --2- 172.21.15.112:0/3091496162 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23ac0fb480 0x7f23ac10e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:35.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.442+0000 7f23b17f7640 1 -- 172.21.15.112:0/3091496162 >> 172.21.15.112:0/3091496162 conn(0x7f23ac0f7000 msgr2=0x7f23ac0f8900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:35.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.442+0000 7f23b17f7640 1 -- 172.21.15.112:0/3091496162 shutdown_connections 2024-09-16T00:44:35.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.442+0000 7f23b17f7640 1 -- 172.21.15.112:0/3091496162 wait complete. 2024-09-16T00:44:35.452 INFO:teuthology.orchestra.run.smithi112.stdout:true 2024-09-16T00:44:35.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.522+0000 7fec4f13f640 1 -- 172.21.15.112:0/449924024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec480fb810 msgr2=0x7fec480fbbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:35.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.522+0000 7fec4f13f640 1 --2- 172.21.15.112:0/449924024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec480fb810 0x7fec480fbbe0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fec380048d0 tx=0x7fec38009da0 comp rx=0 tx=0).stop 2024-09-16T00:44:35.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.523+0000 7fec4f13f640 1 -- 172.21.15.112:0/449924024 shutdown_connections 2024-09-16T00:44:35.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.523+0000 7fec4f13f640 1 --2- 172.21.15.112:0/449924024 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec480fcf80 0x7fec480fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:35.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.523+0000 7fec4f13f640 1 --2- 172.21.15.112:0/449924024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec480fb810 0x7fec480fbbe0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:35.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.523+0000 7fec4f13f640 1 -- 172.21.15.112:0/449924024 >> 172.21.15.112:0/449924024 conn(0x7fec480723b0 msgr2=0x7fec480727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:35.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.523+0000 7fec4f13f640 1 -- 172.21.15.112:0/449924024 shutdown_connections 2024-09-16T00:44:35.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.523+0000 7fec4f13f640 1 -- 172.21.15.112:0/449924024 wait complete. 2024-09-16T00:44:35.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.523+0000 7fec4f13f640 1 Processor -- start 2024-09-16T00:44:35.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.524+0000 7fec4f13f640 1 -- start start 2024-09-16T00:44:35.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.524+0000 7fec4f13f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec480fb810 0x7fec48113200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:35.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.524+0000 7fec4f13f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec480fcf80 0x7fec48113740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:35.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.524+0000 7fec4f13f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec48113d40 con 0x7fec480fb810 2024-09-16T00:44:35.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.524+0000 7fec4f13f640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec4810d180 con 0x7fec480fcf80 2024-09-16T00:44:35.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.524+0000 7fec4cedb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec480fb810 0x7fec48113200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:35.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.524+0000 7fec4cedb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec480fb810 0x7fec48113200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33332/0 (socket says 172.21.15.112:33332) 2024-09-16T00:44:35.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.524+0000 7fec4cedb640 1 -- 172.21.15.112:0/528951104 learned_addr learned my addr 172.21.15.112:0/528951104 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:44:35.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.524+0000 7fec47fff640 1 --2- 172.21.15.112:0/528951104 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec480fcf80 0x7fec48113740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:35.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.524+0000 7fec4cedb640 1 -- 172.21.15.112:0/528951104 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec480fcf80 msgr2=0x7fec48113740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:35.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.524+0000 7fec4cedb640 1 --2- 172.21.15.112:0/528951104 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec480fcf80 0x7fec48113740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:35.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.524+0000 7fec4cedb640 1 -- 172.21.15.112:0/528951104 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec38005040 con 0x7fec480fb810 2024-09-16T00:44:35.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.525+0000 7fec4cedb640 1 --2- 172.21.15.112:0/528951104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec480fb810 0x7fec48113200 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fec38003080 tx=0x7fec38009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:35.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.525+0000 7fec457fa640 1 -- 172.21.15.112:0/528951104 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec38038070 con 0x7fec480fb810 2024-09-16T00:44:35.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.525+0000 7fec457fa640 1 -- 172.21.15.112:0/528951104 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fec380097e0 con 0x7fec480fb810 2024-09-16T00:44:35.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.525+0000 7fec4f13f640 1 -- 172.21.15.112:0/528951104 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec4810d400 con 0x7fec480fb810 2024-09-16T00:44:35.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.525+0000 7fec457fa640 1 -- 172.21.15.112:0/528951104 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec3803d420 con 0x7fec480fb810 2024-09-16T00:44:35.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.525+0000 7fec4f13f640 1 -- 172.21.15.112:0/528951104 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec4810d940 con 0x7fec480fb810 2024-09-16T00:44:35.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.526+0000 7fec4f13f640 1 -- 172.21.15.112:0/528951104 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec480fcbc0 con 0x7fec480fb810 2024-09-16T00:44:35.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.526+0000 7fec457fa640 1 -- 172.21.15.112:0/528951104 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fec38030020 con 0x7fec480fb810 2024-09-16T00:44:35.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.527+0000 7fec457fa640 1 --2- 172.21.15.112:0/528951104 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fec3006e530 0x7fec300709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:35.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.527+0000 7fec457fa640 1 -- 172.21.15.112:0/528951104 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7fec380ae7c0 con 0x7fec480fb810 2024-09-16T00:44:35.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.527+0000 7fec47fff640 1 --2- 172.21.15.112:0/528951104 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fec3006e530 0x7fec300709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:35.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.527+0000 7fec47fff640 1 --2- 172.21.15.112:0/528951104 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fec3006e530 0x7fec300709e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fec4810eac0 tx=0x7fec3c009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:35.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.531+0000 7fec457fa640 1 -- 172.21.15.112:0/528951104 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fec380777b0 con 0x7fec480fb810 2024-09-16T00:44:35.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.712+0000 7fec4f13f640 1 -- 172.21.15.112:0/528951104 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fec4804b720 con 0x7fec3006e530 2024-09-16T00:44:35.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.713+0000 7fec457fa640 1 -- 172.21.15.112:0/528951104 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fec4804b720 con 0x7fec3006e530 2024-09-16T00:44:35.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.715+0000 7fec4f13f640 1 -- 172.21.15.112:0/528951104 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fec3006e530 msgr2=0x7fec300709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:35.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.715+0000 7fec4f13f640 1 --2- 172.21.15.112:0/528951104 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fec3006e530 0x7fec300709e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fec4810eac0 tx=0x7fec3c009040 comp rx=0 tx=0).stop 2024-09-16T00:44:35.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.715+0000 7fec4f13f640 1 -- 172.21.15.112:0/528951104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec480fb810 msgr2=0x7fec48113200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:35.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.715+0000 7fec4f13f640 1 --2- 172.21.15.112:0/528951104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec480fb810 0x7fec48113200 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fec38003080 tx=0x7fec38009da0 comp rx=0 tx=0).stop 2024-09-16T00:44:35.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.715+0000 7fec4f13f640 1 -- 172.21.15.112:0/528951104 shutdown_connections 2024-09-16T00:44:35.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.715+0000 7fec4f13f640 1 --2- 172.21.15.112:0/528951104 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fec3006e530 0x7fec300709e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:35.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.715+0000 7fec4f13f640 1 --2- 172.21.15.112:0/528951104 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec480fcf80 0x7fec48113740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:35.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.715+0000 7fec4f13f640 1 --2- 172.21.15.112:0/528951104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec480fb810 0x7fec48113200 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:35.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.715+0000 7fec4f13f640 1 -- 172.21.15.112:0/528951104 >> 172.21.15.112:0/528951104 conn(0x7fec480723b0 msgr2=0x7fec48064f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:35.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.715+0000 7fec4f13f640 1 -- 172.21.15.112:0/528951104 shutdown_connections 2024-09-16T00:44:35.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.715+0000 7fec4f13f640 1 -- 172.21.15.112:0/528951104 wait complete. 2024-09-16T00:44:35.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.792+0000 7fcb85ef1640 1 -- 172.21.15.112:0/1758429560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb80102510 msgr2=0x7fcb801028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:35.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.792+0000 7fcb85ef1640 1 --2- 172.21.15.112:0/1758429560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb80102510 0x7fcb801028e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fcb700048d0 tx=0x7fcb70009da0 comp rx=0 tx=0).stop 2024-09-16T00:44:35.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.792+0000 7fcb85ef1640 1 -- 172.21.15.112:0/1758429560 shutdown_connections 2024-09-16T00:44:35.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.792+0000 7fcb85ef1640 1 --2- 172.21.15.112:0/1758429560 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb80102eb0 0x7fcb800fc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:35.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.792+0000 7fcb85ef1640 1 --2- 172.21.15.112:0/1758429560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb80102510 0x7fcb801028e0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:35.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.792+0000 7fcb85ef1640 1 -- 172.21.15.112:0/1758429560 >> 172.21.15.112:0/1758429560 conn(0x7fcb800f8130 msgr2=0x7fcb800fa540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:35.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.792+0000 7fcb85ef1640 1 -- 172.21.15.112:0/1758429560 shutdown_connections 2024-09-16T00:44:35.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.793+0000 7fcb85ef1640 1 -- 172.21.15.112:0/1758429560 wait complete. 2024-09-16T00:44:35.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.793+0000 7fcb85ef1640 1 Processor -- start 2024-09-16T00:44:35.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.793+0000 7fcb85ef1640 1 -- start start 2024-09-16T00:44:35.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.793+0000 7fcb85ef1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb80102510 0x7fcb801131d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:35.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.793+0000 7fcb85ef1640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb80102eb0 0x7fcb80113710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:35.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.793+0000 7fcb85ef1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb80113d10 con 0x7fcb80102510 2024-09-16T00:44:35.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.793+0000 7fcb85ef1640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8010d150 con 0x7fcb80102eb0 2024-09-16T00:44:35.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.794+0000 7fcb7f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb80102510 0x7fcb801131d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:35.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.794+0000 7fcb7f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb80102510 0x7fcb801131d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33360/0 (socket says 172.21.15.112:33360) 2024-09-16T00:44:35.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.794+0000 7fcb7f7fe640 1 -- 172.21.15.112:0/2092195714 learned_addr learned my addr 172.21.15.112:0/2092195714 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:44:35.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.794+0000 7fcb7effd640 1 --2- 172.21.15.112:0/2092195714 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb80102eb0 0x7fcb80113710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:35.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.794+0000 7fcb7f7fe640 1 -- 172.21.15.112:0/2092195714 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb80102eb0 msgr2=0x7fcb80113710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:35.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.794+0000 7fcb7f7fe640 1 --2- 172.21.15.112:0/2092195714 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb80102eb0 0x7fcb80113710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:35.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.794+0000 7fcb7f7fe640 1 -- 172.21.15.112:0/2092195714 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb70005040 con 0x7fcb80102510 2024-09-16T00:44:35.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.794+0000 7fcb7f7fe640 1 --2- 172.21.15.112:0/2092195714 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb80102510 0x7fcb801131d0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fcb70000c00 tx=0x7fcb70009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:35.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.794+0000 7fcb67fff640 1 -- 172.21.15.112:0/2092195714 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb70038070 con 0x7fcb80102510 2024-09-16T00:44:35.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.794+0000 7fcb67fff640 1 -- 172.21.15.112:0/2092195714 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fcb7002ade0 con 0x7fcb80102510 2024-09-16T00:44:35.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.794+0000 7fcb85ef1640 1 -- 172.21.15.112:0/2092195714 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb8010d3d0 con 0x7fcb80102510 2024-09-16T00:44:35.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.794+0000 7fcb67fff640 1 -- 172.21.15.112:0/2092195714 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb7002c7e0 con 0x7fcb80102510 2024-09-16T00:44:35.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.794+0000 7fcb85ef1640 1 -- 172.21.15.112:0/2092195714 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb8010d910 con 0x7fcb80102510 2024-09-16T00:44:35.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.795+0000 7fcb85ef1640 1 -- 172.21.15.112:0/2092195714 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb800fdc90 con 0x7fcb80102510 2024-09-16T00:44:35.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.796+0000 7fcb67fff640 1 -- 172.21.15.112:0/2092195714 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fcb70030020 con 0x7fcb80102510 2024-09-16T00:44:35.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.796+0000 7fcb67fff640 1 --2- 172.21.15.112:0/2092195714 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fcb6806e530 0x7fcb680709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:35.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.797+0000 7fcb67fff640 1 -- 172.21.15.112:0/2092195714 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7fcb700adf70 con 0x7fcb80102510 2024-09-16T00:44:35.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.797+0000 7fcb7effd640 1 --2- 172.21.15.112:0/2092195714 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fcb6806e530 0x7fcb680709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:35.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.797+0000 7fcb7effd640 1 --2- 172.21.15.112:0/2092195714 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fcb6806e530 0x7fcb680709e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fcb8010ea90 tx=0x7fcb7400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:35.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.801+0000 7fcb67fff640 1 -- 172.21.15.112:0/2092195714 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fcb700e31b0 con 0x7fcb80102510 2024-09-16T00:44:35.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.977+0000 7fcb85ef1640 1 -- 172.21.15.112:0/2092195714 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fcb8004b720 con 0x7fcb6806e530 2024-09-16T00:44:35.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.985+0000 7fcb67fff640 1 -- 172.21.15.112:0/2092195714 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7fcb8004b720 con 0x7fcb6806e530 2024-09-16T00:44:35.987 INFO:teuthology.orchestra.run.smithi112.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T00:44:35.987 INFO:teuthology.orchestra.run.smithi112.stdout:alertmanager.smithi112 smithi112 *:9093,9094 running (17m) 10m ago 26m 24.7M - 0.25.0 c8568f914cd2 d85ea0562022 2024-09-16T00:44:35.987 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi112 smithi112 running (14m) 10m ago 26m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c e4fe58f9702a 2024-09-16T00:44:35.987 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi169 smithi169 running (14m) 10m ago 25m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c 3ba2c81d527f 2024-09-16T00:44:35.987 INFO:teuthology.orchestra.run.smithi112.stdout:grafana.smithi112 smithi112 *:3000 running (16m) 10m ago 26m 82.1M - 9.4.12 2bacad6d85d8 51f9d2013da2 2024-09-16T00:44:35.987 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.hxfwkh smithi112 running (10m) 10m ago 21m 14.2M - 19.1.1-326-g8921c8ef f1455bd0219c 1723069902d6 2024-09-16T00:44:35.987 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.ithwzj smithi112 running (10m) 10m ago 21m 12.2M - 19.1.1-326-g8921c8ef f1455bd0219c e7aad5b81db5 2024-09-16T00:44:35.987 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.gawwgw smithi169 running (10m) 10m ago 21m 15.7M - 19.1.1-326-g8921c8ef f1455bd0219c 469be459ee12 2024-09-16T00:44:35.987 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.lgeqrj smithi169 running (10m) 10m ago 20m 13.3M - 19.1.1-326-g8921c8ef f1455bd0219c 31f0fbd2395d 2024-09-16T00:44:35.987 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi112.xrkkbc smithi112 *:8443,9283,8765 running (17m) 10m ago 27m 504M - 19.1.1-326-g8921c8ef f1455bd0219c a981f7fd2810 2024-09-16T00:44:35.987 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi169.mtrobk smithi169 *:8443,9283,8765 running (16m) 10m ago 25m 442M - 19.1.1-326-g8921c8ef f1455bd0219c 4096e5369c2d 2024-09-16T00:44:35.987 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi112 smithi112 running (15m) 10m ago 27m 49.8M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 5aaec033c042 2024-09-16T00:44:35.987 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi169 smithi169 running (15m) 10m ago 25m 38.9M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 7bc609155f3f 2024-09-16T00:44:35.987 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi112 smithi112 *:9100 running (17m) 10m ago 26m 15.5M - 1.5.0 0da6a335fe13 6404d1d601e2 2024-09-16T00:44:35.988 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi169 smithi169 *:9100 running (17m) 10m ago 25m 15.4M - 1.5.0 0da6a335fe13 5505baf702a6 2024-09-16T00:44:35.988 INFO:teuthology.orchestra.run.smithi112.stdout:osd.0 smithi112 running (14m) 10m ago 24m 105M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 53efc9397017 2024-09-16T00:44:35.988 INFO:teuthology.orchestra.run.smithi112.stdout:osd.1 smithi112 running (13m) 10m ago 24m 98.9M 1557M 19.1.1-326-g8921c8ef f1455bd0219c fe6f1a4a2cf4 2024-09-16T00:44:35.988 INFO:teuthology.orchestra.run.smithi112.stdout:osd.2 smithi112 running (12m) 10m ago 23m 102M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 5f7863712057 2024-09-16T00:44:35.988 INFO:teuthology.orchestra.run.smithi112.stdout:osd.3 smithi169 running (12m) 10m ago 23m 92.1M 2581M 19.1.1-326-g8921c8ef f1455bd0219c ee07b5577810 2024-09-16T00:44:35.988 INFO:teuthology.orchestra.run.smithi112.stdout:osd.4 smithi169 running (11m) 10m ago 23m 103M 2581M 19.1.1-326-g8921c8ef f1455bd0219c 769f026a54eb 2024-09-16T00:44:35.988 INFO:teuthology.orchestra.run.smithi112.stdout:osd.5 smithi169 running (11m) 10m ago 22m 125M 2581M 19.1.1-326-g8921c8ef f1455bd0219c a6007c13cf78 2024-09-16T00:44:35.988 INFO:teuthology.orchestra.run.smithi112.stdout:prometheus.smithi112 smithi112 *:9095 running (16m) 10m ago 26m 64.1M - 2.43.0 a07b618ecd1d 7a1d42e60d2b 2024-09-16T00:44:35.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.987+0000 7fcb85ef1640 1 -- 172.21.15.112:0/2092195714 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fcb6806e530 msgr2=0x7fcb680709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:35.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.987+0000 7fcb85ef1640 1 --2- 172.21.15.112:0/2092195714 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fcb6806e530 0x7fcb680709e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fcb8010ea90 tx=0x7fcb7400b040 comp rx=0 tx=0).stop 2024-09-16T00:44:35.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.987+0000 7fcb85ef1640 1 -- 172.21.15.112:0/2092195714 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb80102510 msgr2=0x7fcb801131d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:35.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.987+0000 7fcb85ef1640 1 --2- 172.21.15.112:0/2092195714 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb80102510 0x7fcb801131d0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fcb70000c00 tx=0x7fcb70009da0 comp rx=0 tx=0).stop 2024-09-16T00:44:35.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.987+0000 7fcb85ef1640 1 -- 172.21.15.112:0/2092195714 shutdown_connections 2024-09-16T00:44:35.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.987+0000 7fcb85ef1640 1 --2- 172.21.15.112:0/2092195714 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fcb6806e530 0x7fcb680709e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:35.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.987+0000 7fcb85ef1640 1 --2- 172.21.15.112:0/2092195714 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb80102eb0 0x7fcb80113710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:35.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.987+0000 7fcb85ef1640 1 --2- 172.21.15.112:0/2092195714 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb80102510 0x7fcb801131d0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:35.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.987+0000 7fcb85ef1640 1 -- 172.21.15.112:0/2092195714 >> 172.21.15.112:0/2092195714 conn(0x7fcb800f8130 msgr2=0x7fcb800f9100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:35.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.988+0000 7fcb85ef1640 1 -- 172.21.15.112:0/2092195714 shutdown_connections 2024-09-16T00:44:35.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:35.988+0000 7fcb85ef1640 1 -- 172.21.15.112:0/2092195714 wait complete. 2024-09-16T00:44:36.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.064+0000 7f6c4abfc640 1 -- 172.21.15.112:0/1740230274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c44102550 msgr2=0x7f6c44102920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:36.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.064+0000 7f6c4abfc640 1 --2- 172.21.15.112:0/1740230274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c44102550 0x7f6c44102920 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c009c50 tx=0x7f6c3c004d10 comp rx=0 tx=0).stop 2024-09-16T00:44:36.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.064+0000 7f6c4abfc640 1 -- 172.21.15.112:0/1740230274 shutdown_connections 2024-09-16T00:44:36.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.064+0000 7f6c4abfc640 1 --2- 172.21.15.112:0/1740230274 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c44102ef0 0x7f6c440fc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:36.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.064+0000 7f6c4abfc640 1 --2- 172.21.15.112:0/1740230274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c44102550 0x7f6c44102920 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:36.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.064+0000 7f6c4abfc640 1 -- 172.21.15.112:0/1740230274 >> 172.21.15.112:0/1740230274 conn(0x7f6c440f80f0 msgr2=0x7f6c440fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:36.066 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:44:35 smithi112 ceph-mon[97750]: pgmap v505: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:44:36.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.064+0000 7f6c4abfc640 1 -- 172.21.15.112:0/1740230274 shutdown_connections 2024-09-16T00:44:36.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.064+0000 7f6c4abfc640 1 -- 172.21.15.112:0/1740230274 wait complete. 2024-09-16T00:44:36.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.064+0000 7f6c4abfc640 1 Processor -- start 2024-09-16T00:44:36.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.065+0000 7f6c4abfc640 1 -- start start 2024-09-16T00:44:36.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.065+0000 7f6c4abfc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c44102550 0x7f6c44113570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:36.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.065+0000 7f6c4abfc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c44102ef0 0x7f6c44113ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:36.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.065+0000 7f6c4abfc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c44114100 con 0x7f6c44102ef0 2024-09-16T00:44:36.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.065+0000 7f6c4abfc640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4410d510 con 0x7f6c44102550 2024-09-16T00:44:36.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.065+0000 7f6c43fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c44102ef0 0x7f6c44113ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:36.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.065+0000 7f6c43fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c44102ef0 0x7f6c44113ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33374/0 (socket says 172.21.15.112:33374) 2024-09-16T00:44:36.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.065+0000 7f6c43fff640 1 -- 172.21.15.112:0/790510876 learned_addr learned my addr 172.21.15.112:0/790510876 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:44:36.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.065+0000 7f6c48998640 1 --2- 172.21.15.112:0/790510876 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c44102550 0x7f6c44113570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:36.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.065+0000 7f6c43fff640 1 -- 172.21.15.112:0/790510876 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c44102550 msgr2=0x7f6c44113570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:36.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.065+0000 7f6c43fff640 1 --2- 172.21.15.112:0/790510876 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c44102550 0x7f6c44113570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:36.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.065+0000 7f6c43fff640 1 -- 172.21.15.112:0/790510876 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c3c0098b0 con 0x7f6c44102ef0 2024-09-16T00:44:36.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.066+0000 7f6c43fff640 1 --2- 172.21.15.112:0/790510876 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c44102ef0 0x7f6c44113ab0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f6c34009af0 tx=0x7f6c34004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:36.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.066+0000 7f6c417fa640 1 -- 172.21.15.112:0/790510876 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c34007c30 con 0x7f6c44102ef0 2024-09-16T00:44:36.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.066+0000 7f6c417fa640 1 -- 172.21.15.112:0/790510876 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f6c3400f040 con 0x7f6c44102ef0 2024-09-16T00:44:36.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.066+0000 7f6c4abfc640 1 -- 172.21.15.112:0/790510876 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c4410d7f0 con 0x7f6c44102ef0 2024-09-16T00:44:36.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.066+0000 7f6c417fa640 1 -- 172.21.15.112:0/790510876 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c340094b0 con 0x7f6c44102ef0 2024-09-16T00:44:36.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.066+0000 7f6c4abfc640 1 -- 172.21.15.112:0/790510876 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c4410dd90 con 0x7f6c44102ef0 2024-09-16T00:44:36.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.067+0000 7f6c4abfc640 1 -- 172.21.15.112:0/790510876 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c440fdcd0 con 0x7f6c44102ef0 2024-09-16T00:44:36.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.067+0000 7f6c417fa640 1 -- 172.21.15.112:0/790510876 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f6c34009730 con 0x7f6c44102ef0 2024-09-16T00:44:36.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.068+0000 7f6c417fa640 1 --2- 172.21.15.112:0/790510876 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f6c3006e530 0x7f6c300709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:36.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.068+0000 7f6c417fa640 1 -- 172.21.15.112:0/790510876 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f6c3408e2e0 con 0x7f6c44102ef0 2024-09-16T00:44:36.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.068+0000 7f6c48998640 1 --2- 172.21.15.112:0/790510876 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f6c3006e530 0x7f6c300709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:36.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.069+0000 7f6c48998640 1 --2- 172.21.15.112:0/790510876 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f6c3006e530 0x7f6c300709e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c004a30 tx=0x7f6c3c004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:36.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.072+0000 7f6c417fa640 1 -- 172.21.15.112:0/790510876 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f6c34057250 con 0x7f6c44102ef0 2024-09-16T00:44:36.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:44:35 smithi169 ceph-mon[77626]: pgmap v505: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:44:36.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.303+0000 7f6c4abfc640 1 -- 172.21.15.112:0/790510876 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6c4410ec10 con 0x7f6c44102ef0 2024-09-16T00:44:36.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.304+0000 7f6c417fa640 1 -- 172.21.15.112:0/790510876 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f6c3400dc50 con 0x7f6c44102ef0 2024-09-16T00:44:36.305 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:44:36.305 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": { 2024-09-16T00:44:36.305 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:44:36.305 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:44:36.306 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": { 2024-09-16T00:44:36.306 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:44:36.306 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:44:36.306 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": { 2024-09-16T00:44:36.306 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 6 2024-09-16T00:44:36.306 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:44:36.306 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": { 2024-09-16T00:44:36.306 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 4 2024-09-16T00:44:36.306 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:44:36.306 INFO:teuthology.orchestra.run.smithi112.stdout: "overall": { 2024-09-16T00:44:36.306 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 14 2024-09-16T00:44:36.306 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-16T00:44:36.306 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:44:36.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.306+0000 7f6c4abfc640 1 -- 172.21.15.112:0/790510876 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f6c3006e530 msgr2=0x7f6c300709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:36.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.306+0000 7f6c4abfc640 1 --2- 172.21.15.112:0/790510876 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f6c3006e530 0x7f6c300709e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c004a30 tx=0x7f6c3c004d10 comp rx=0 tx=0).stop 2024-09-16T00:44:36.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.306+0000 7f6c4abfc640 1 -- 172.21.15.112:0/790510876 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c44102ef0 msgr2=0x7f6c44113ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:36.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.306+0000 7f6c4abfc640 1 --2- 172.21.15.112:0/790510876 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c44102ef0 0x7f6c44113ab0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f6c34009af0 tx=0x7f6c34004570 comp rx=0 tx=0).stop 2024-09-16T00:44:36.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.306+0000 7f6c4abfc640 1 -- 172.21.15.112:0/790510876 shutdown_connections 2024-09-16T00:44:36.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.306+0000 7f6c4abfc640 1 --2- 172.21.15.112:0/790510876 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f6c3006e530 0x7f6c300709e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:36.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.306+0000 7f6c4abfc640 1 --2- 172.21.15.112:0/790510876 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c44102ef0 0x7f6c44113ab0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:36.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.306+0000 7f6c4abfc640 1 --2- 172.21.15.112:0/790510876 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c44102550 0x7f6c44113570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:36.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.306+0000 7f6c4abfc640 1 -- 172.21.15.112:0/790510876 >> 172.21.15.112:0/790510876 conn(0x7f6c440f80f0 msgr2=0x7f6c440f90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:36.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.306+0000 7f6c4abfc640 1 -- 172.21.15.112:0/790510876 shutdown_connections 2024-09-16T00:44:36.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.306+0000 7f6c4abfc640 1 -- 172.21.15.112:0/790510876 wait complete. 2024-09-16T00:44:36.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.386+0000 7f1746a43640 1 -- 172.21.15.112:0/2609475771 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17400fb4a0 msgr2=0x7f17400fb910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:36.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.386+0000 7f1746a43640 1 --2- 172.21.15.112:0/2609475771 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17400fb4a0 0x7f17400fb910 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f17340048d0 tx=0x7f1734009da0 comp rx=0 tx=0).stop 2024-09-16T00:44:36.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.386+0000 7f1746a43640 1 -- 172.21.15.112:0/2609475771 shutdown_connections 2024-09-16T00:44:36.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.386+0000 7f1746a43640 1 --2- 172.21.15.112:0/2609475771 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17400fb4a0 0x7f17400fb910 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:36.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.386+0000 7f1746a43640 1 --2- 172.21.15.112:0/2609475771 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17401014e0 0x7f17401018b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:36.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.386+0000 7f1746a43640 1 -- 172.21.15.112:0/2609475771 >> 172.21.15.112:0/2609475771 conn(0x7f17400f7060 msgr2=0x7f17400f9470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:36.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.386+0000 7f1746a43640 1 -- 172.21.15.112:0/2609475771 shutdown_connections 2024-09-16T00:44:36.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.386+0000 7f1746a43640 1 -- 172.21.15.112:0/2609475771 wait complete. 2024-09-16T00:44:36.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.386+0000 7f1746a43640 1 Processor -- start 2024-09-16T00:44:36.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.387+0000 7f1746a43640 1 -- start start 2024-09-16T00:44:36.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.387+0000 7f1746a43640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17400fb4a0 0x7f17401131c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:36.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.387+0000 7f1746a43640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17401014e0 0x7f1740113700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:36.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.387+0000 7f1746a43640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1740113d00 con 0x7f17401014e0 2024-09-16T00:44:36.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.387+0000 7f1746a43640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f174010d140 con 0x7f17400fb4a0 2024-09-16T00:44:36.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.387+0000 7f173f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17401014e0 0x7f1740113700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:36.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.387+0000 7f173f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17401014e0 0x7f1740113700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33392/0 (socket says 172.21.15.112:33392) 2024-09-16T00:44:36.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.387+0000 7f173f7fe640 1 -- 172.21.15.112:0/1168501364 learned_addr learned my addr 172.21.15.112:0/1168501364 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:44:36.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.387+0000 7f173ffff640 1 --2- 172.21.15.112:0/1168501364 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17400fb4a0 0x7f17401131c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:36.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.387+0000 7f173f7fe640 1 -- 172.21.15.112:0/1168501364 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17400fb4a0 msgr2=0x7f17401131c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:36.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.387+0000 7f173f7fe640 1 --2- 172.21.15.112:0/1168501364 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17400fb4a0 0x7f17401131c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:36.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.387+0000 7f173f7fe640 1 -- 172.21.15.112:0/1168501364 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1734005040 con 0x7f17401014e0 2024-09-16T00:44:36.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.387+0000 7f173f7fe640 1 --2- 172.21.15.112:0/1168501364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17401014e0 0x7f1740113700 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f1734000c00 tx=0x7f1734009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:36.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.388+0000 7f173cff9640 1 -- 172.21.15.112:0/1168501364 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1734038070 con 0x7f17401014e0 2024-09-16T00:44:36.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.388+0000 7f173cff9640 1 -- 172.21.15.112:0/1168501364 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f173402ade0 con 0x7f17401014e0 2024-09-16T00:44:36.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.388+0000 7f1746a43640 1 -- 172.21.15.112:0/1168501364 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f174010d3c0 con 0x7f17401014e0 2024-09-16T00:44:36.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.388+0000 7f173cff9640 1 -- 172.21.15.112:0/1168501364 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f173402c7e0 con 0x7f17401014e0 2024-09-16T00:44:36.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.388+0000 7f1746a43640 1 -- 172.21.15.112:0/1168501364 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f174010d900 con 0x7f17401014e0 2024-09-16T00:44:36.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.389+0000 7f1746a43640 1 -- 172.21.15.112:0/1168501364 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17400fcbe0 con 0x7f17401014e0 2024-09-16T00:44:36.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.389+0000 7f173cff9640 1 -- 172.21.15.112:0/1168501364 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f1734030020 con 0x7f17401014e0 2024-09-16T00:44:36.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.390+0000 7f173cff9640 1 --2- 172.21.15.112:0/1168501364 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f172806e530 0x7f17280709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:36.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.390+0000 7f173ffff640 1 --2- 172.21.15.112:0/1168501364 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f172806e530 0x7f17280709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:36.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.390+0000 7f173cff9640 1 -- 172.21.15.112:0/1168501364 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f17340adf70 con 0x7f17401014e0 2024-09-16T00:44:36.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.390+0000 7f173ffff640 1 --2- 172.21.15.112:0/1168501364 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f172806e530 0x7f17280709e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1730004ed0 tx=0x7f173000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:36.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.394+0000 7f173cff9640 1 -- 172.21.15.112:0/1168501364 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f17340e31b0 con 0x7f17401014e0 2024-09-16T00:44:36.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.594+0000 7f1746a43640 1 -- 172.21.15.112:0/1168501364 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f1740066360 con 0x7f17401014e0 2024-09-16T00:44:36.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.594+0000 7f173cff9640 1 -- 172.21.15.112:0/1168501364 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1914 (secure 0 0 0) 0x7f1734076f90 con 0x7f17401014e0 2024-09-16T00:44:36.619 INFO:teuthology.orchestra.run.smithi112.stdout:e34 2024-09-16T00:44:36.619 INFO:teuthology.orchestra.run.smithi112.stdout:btime 2024-09-16T00:34:26:619254+0000 2024-09-16T00:44:36.619 INFO:teuthology.orchestra.run.smithi112.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T00:44:36.619 INFO:teuthology.orchestra.run.smithi112.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:44:36.619 INFO:teuthology.orchestra.run.smithi112.stdout:legacy client fscid: 1 2024-09-16T00:44:36.619 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:44:36.620 INFO:teuthology.orchestra.run.smithi112.stdout:Filesystem 'cephfs' (1) 2024-09-16T00:44:36.620 INFO:teuthology.orchestra.run.smithi112.stdout:fs_name cephfs 2024-09-16T00:44:36.620 INFO:teuthology.orchestra.run.smithi112.stdout:epoch 25 2024-09-16T00:44:36.620 INFO:teuthology.orchestra.run.smithi112.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-16T00:44:36.620 INFO:teuthology.orchestra.run.smithi112.stdout:created 2024-09-16T00:23:22.328199+0000 2024-09-16T00:44:36.620 INFO:teuthology.orchestra.run.smithi112.stdout:modified 2024-09-16T00:33:29.012991+0000 2024-09-16T00:44:36.620 INFO:teuthology.orchestra.run.smithi112.stdout:tableserver 0 2024-09-16T00:44:36.620 INFO:teuthology.orchestra.run.smithi112.stdout:root 0 2024-09-16T00:44:36.620 INFO:teuthology.orchestra.run.smithi112.stdout:session_timeout 60 2024-09-16T00:44:36.620 INFO:teuthology.orchestra.run.smithi112.stdout:session_autoclose 300 2024-09-16T00:44:36.620 INFO:teuthology.orchestra.run.smithi112.stdout:max_file_size 1099511627776 2024-09-16T00:44:36.620 INFO:teuthology.orchestra.run.smithi112.stdout:max_xattr_size 65536 2024-09-16T00:44:36.620 INFO:teuthology.orchestra.run.smithi112.stdout:required_client_features {} 2024-09-16T00:44:36.621 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure 0 2024-09-16T00:44:36.621 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure_osd_epoch 82 2024-09-16T00:44:36.621 INFO:teuthology.orchestra.run.smithi112.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:44:36.621 INFO:teuthology.orchestra.run.smithi112.stdout:max_mds 2 2024-09-16T00:44:36.621 INFO:teuthology.orchestra.run.smithi112.stdout:in 0,1 2024-09-16T00:44:36.621 INFO:teuthology.orchestra.run.smithi112.stdout:up {} 2024-09-16T00:44:36.621 INFO:teuthology.orchestra.run.smithi112.stdout:failed 0,1 2024-09-16T00:44:36.621 INFO:teuthology.orchestra.run.smithi112.stdout:damaged 2024-09-16T00:44:36.621 INFO:teuthology.orchestra.run.smithi112.stdout:stopped 2024-09-16T00:44:36.621 INFO:teuthology.orchestra.run.smithi112.stdout:data_pools [3] 2024-09-16T00:44:36.621 INFO:teuthology.orchestra.run.smithi112.stdout:metadata_pool 2 2024-09-16T00:44:36.621 INFO:teuthology.orchestra.run.smithi112.stdout:inline_data enabled 2024-09-16T00:44:36.621 INFO:teuthology.orchestra.run.smithi112.stdout:balancer 2024-09-16T00:44:36.621 INFO:teuthology.orchestra.run.smithi112.stdout:bal_rank_mask -1 2024-09-16T00:44:36.622 INFO:teuthology.orchestra.run.smithi112.stdout:standby_count_wanted 1 2024-09-16T00:44:36.622 INFO:teuthology.orchestra.run.smithi112.stdout:qdb_cluster leader: 0 members: 2024-09-16T00:44:36.622 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:44:36.622 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:44:36.622 INFO:teuthology.orchestra.run.smithi112.stdout:Standby daemons: 2024-09-16T00:44:36.622 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:44:36.622 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.ithwzj{-1:34392} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.112:6826/476229221,v1:172.21.15.112:6827/476229221] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:44:36.622 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.hxfwkh{-1:34404} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.112:6828/2356899876,v1:172.21.15.112:6829/2356899876] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:44:36.622 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.gawwgw{-1:44343} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.169:6824/1635329957,v1:172.21.15.169:6825/1635329957] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:44:36.622 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.lgeqrj{-1:44347} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.169:6826/184810031,v1:172.21.15.169:6827/184810031] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:44:36.623 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 34 2024-09-16T00:44:36.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.596+0000 7f1746a43640 1 -- 172.21.15.112:0/1168501364 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f172806e530 msgr2=0x7f17280709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:36.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.596+0000 7f1746a43640 1 --2- 172.21.15.112:0/1168501364 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f172806e530 0x7f17280709e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1730004ed0 tx=0x7f173000b040 comp rx=0 tx=0).stop 2024-09-16T00:44:36.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.596+0000 7f1746a43640 1 -- 172.21.15.112:0/1168501364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17401014e0 msgr2=0x7f1740113700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:36.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.596+0000 7f1746a43640 1 --2- 172.21.15.112:0/1168501364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17401014e0 0x7f1740113700 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f1734000c00 tx=0x7f1734009da0 comp rx=0 tx=0).stop 2024-09-16T00:44:36.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.596+0000 7f1746a43640 1 -- 172.21.15.112:0/1168501364 shutdown_connections 2024-09-16T00:44:36.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.596+0000 7f1746a43640 1 --2- 172.21.15.112:0/1168501364 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f172806e530 0x7f17280709e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:36.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.596+0000 7f1746a43640 1 --2- 172.21.15.112:0/1168501364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17401014e0 0x7f1740113700 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:36.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.596+0000 7f1746a43640 1 --2- 172.21.15.112:0/1168501364 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17400fb4a0 0x7f17401131c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:36.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.596+0000 7f1746a43640 1 -- 172.21.15.112:0/1168501364 >> 172.21.15.112:0/1168501364 conn(0x7f17400f7060 msgr2=0x7f17400f8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:36.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.597+0000 7f1746a43640 1 -- 172.21.15.112:0/1168501364 shutdown_connections 2024-09-16T00:44:36.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.597+0000 7f1746a43640 1 -- 172.21.15.112:0/1168501364 wait complete. 2024-09-16T00:44:36.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.675+0000 7f131fd91640 1 -- 172.21.15.112:0/2236175099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13181025f0 msgr2=0x7f13181029c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:36.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.675+0000 7f131fd91640 1 --2- 172.21.15.112:0/2236175099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13181025f0 0x7f13181029c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f13140048d0 tx=0x7f1314009da0 comp rx=0 tx=0).stop 2024-09-16T00:44:36.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.675+0000 7f131fd91640 1 -- 172.21.15.112:0/2236175099 shutdown_connections 2024-09-16T00:44:36.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.675+0000 7f131fd91640 1 --2- 172.21.15.112:0/2236175099 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13180fac60 0x7f13180fb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:36.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.675+0000 7f131fd91640 1 --2- 172.21.15.112:0/2236175099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13181025f0 0x7f13181029c0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:36.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.675+0000 7f131fd91640 1 -- 172.21.15.112:0/2236175099 >> 172.21.15.112:0/2236175099 conn(0x7f1318070c80 msgr2=0x7f1318071080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:36.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.676+0000 7f131fd91640 1 -- 172.21.15.112:0/2236175099 shutdown_connections 2024-09-16T00:44:36.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.676+0000 7f131fd91640 1 -- 172.21.15.112:0/2236175099 wait complete. 2024-09-16T00:44:36.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.676+0000 7f131fd91640 1 Processor -- start 2024-09-16T00:44:36.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.676+0000 7f131fd91640 1 -- start start 2024-09-16T00:44:36.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.676+0000 7f131fd91640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13180fac60 0x7f13180fbe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:36.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.676+0000 7f131fd91640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13181025f0 0x7f13180fc380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:36.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.676+0000 7f131fd91640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13180fc950 con 0x7f13180fac60 2024-09-16T00:44:36.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.676+0000 7f131fd91640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13180fcac0 con 0x7f13181025f0 2024-09-16T00:44:36.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.676+0000 7f131db2d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13180fac60 0x7f13180fbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:36.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.677+0000 7f131db2d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13180fac60 0x7f13180fbe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33418/0 (socket says 172.21.15.112:33418) 2024-09-16T00:44:36.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.677+0000 7f131db2d640 1 -- 172.21.15.112:0/289359139 learned_addr learned my addr 172.21.15.112:0/289359139 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:44:36.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.677+0000 7f131d32c640 1 --2- 172.21.15.112:0/289359139 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13181025f0 0x7f13180fc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:36.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.677+0000 7f131db2d640 1 -- 172.21.15.112:0/289359139 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13181025f0 msgr2=0x7f13180fc380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:36.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.677+0000 7f131db2d640 1 --2- 172.21.15.112:0/289359139 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13181025f0 0x7f13180fc380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:36.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.677+0000 7f131db2d640 1 -- 172.21.15.112:0/289359139 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1314005040 con 0x7f13180fac60 2024-09-16T00:44:36.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.677+0000 7f131db2d640 1 --2- 172.21.15.112:0/289359139 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13180fac60 0x7f13180fbe40 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f1314003080 tx=0x7f1314009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:36.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.677+0000 7f130e7fc640 1 -- 172.21.15.112:0/289359139 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1314038070 con 0x7f13180fac60 2024-09-16T00:44:36.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.677+0000 7f130e7fc640 1 -- 172.21.15.112:0/289359139 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f13140097e0 con 0x7f13180fac60 2024-09-16T00:44:36.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.677+0000 7f130e7fc640 1 -- 172.21.15.112:0/289359139 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f131403d420 con 0x7f13180fac60 2024-09-16T00:44:36.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.677+0000 7f131fd91640 1 -- 172.21.15.112:0/289359139 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1318119570 con 0x7f13180fac60 2024-09-16T00:44:36.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.677+0000 7f131fd91640 1 -- 172.21.15.112:0/289359139 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13181198f0 con 0x7f13180fac60 2024-09-16T00:44:36.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.678+0000 7f131fd91640 1 -- 172.21.15.112:0/289359139 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1318065810 con 0x7f13180fac60 2024-09-16T00:44:36.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.679+0000 7f130e7fc640 1 -- 172.21.15.112:0/289359139 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f1314030020 con 0x7f13180fac60 2024-09-16T00:44:36.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.680+0000 7f130e7fc640 1 --2- 172.21.15.112:0/289359139 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f130406e530 0x7f13040709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:36.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.680+0000 7f130e7fc640 1 -- 172.21.15.112:0/289359139 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f13140ae7f0 con 0x7f13180fac60 2024-09-16T00:44:36.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.680+0000 7f131d32c640 1 --2- 172.21.15.112:0/289359139 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f130406e530 0x7f13040709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:36.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.680+0000 7f131d32c640 1 --2- 172.21.15.112:0/289359139 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f130406e530 0x7f13040709e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f13180fd780 tx=0x7f1308009120 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:36.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.688+0000 7f130e7fc640 1 -- 172.21.15.112:0/289359139 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f1314077760 con 0x7f13180fac60 2024-09-16T00:44:36.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.872+0000 7f131fd91640 1 -- 172.21.15.112:0/289359139 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f131804b790 con 0x7f130406e530 2024-09-16T00:44:36.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.873+0000 7f130e7fc640 1 -- 172.21.15.112:0/289359139 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f131804b790 con 0x7f130406e530 2024-09-16T00:44:36.874 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:44:36.874 INFO:teuthology.orchestra.run.smithi112.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80", 2024-09-16T00:44:36.874 INFO:teuthology.orchestra.run.smithi112.stdout: "in_progress": true, 2024-09-16T00:44:36.874 INFO:teuthology.orchestra.run.smithi112.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T00:44:36.874 INFO:teuthology.orchestra.run.smithi112.stdout: "services_complete": [ 2024-09-16T00:44:36.875 INFO:teuthology.orchestra.run.smithi112.stdout: "mds", 2024-09-16T00:44:36.875 INFO:teuthology.orchestra.run.smithi112.stdout: "osd", 2024-09-16T00:44:36.875 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr", 2024-09-16T00:44:36.875 INFO:teuthology.orchestra.run.smithi112.stdout: "mon", 2024-09-16T00:44:36.875 INFO:teuthology.orchestra.run.smithi112.stdout: "crash" 2024-09-16T00:44:36.875 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-16T00:44:36.875 INFO:teuthology.orchestra.run.smithi112.stdout: "progress": "16/21 daemons upgraded", 2024-09-16T00:44:36.875 INFO:teuthology.orchestra.run.smithi112.stdout: "message": "Currently upgrading mds daemons", 2024-09-16T00:44:36.875 INFO:teuthology.orchestra.run.smithi112.stdout: "is_paused": false 2024-09-16T00:44:36.875 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:44:36.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.876+0000 7f131fd91640 1 -- 172.21.15.112:0/289359139 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f130406e530 msgr2=0x7f13040709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:36.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.876+0000 7f131fd91640 1 --2- 172.21.15.112:0/289359139 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f130406e530 0x7f13040709e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f13180fd780 tx=0x7f1308009120 comp rx=0 tx=0).stop 2024-09-16T00:44:36.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.876+0000 7f131fd91640 1 -- 172.21.15.112:0/289359139 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13180fac60 msgr2=0x7f13180fbe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:36.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.876+0000 7f131fd91640 1 --2- 172.21.15.112:0/289359139 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13180fac60 0x7f13180fbe40 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f1314003080 tx=0x7f1314009da0 comp rx=0 tx=0).stop 2024-09-16T00:44:36.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.876+0000 7f131fd91640 1 -- 172.21.15.112:0/289359139 shutdown_connections 2024-09-16T00:44:36.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.876+0000 7f131fd91640 1 --2- 172.21.15.112:0/289359139 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f130406e530 0x7f13040709e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:36.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.876+0000 7f131fd91640 1 --2- 172.21.15.112:0/289359139 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13181025f0 0x7f13180fc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:36.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.876+0000 7f131fd91640 1 --2- 172.21.15.112:0/289359139 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13180fac60 0x7f13180fbe40 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:36.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.876+0000 7f131fd91640 1 -- 172.21.15.112:0/289359139 >> 172.21.15.112:0/289359139 conn(0x7f1318070c80 msgr2=0x7f13180f9f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:36.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.876+0000 7f131fd91640 1 -- 172.21.15.112:0/289359139 shutdown_connections 2024-09-16T00:44:36.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.876+0000 7f131fd91640 1 -- 172.21.15.112:0/289359139 wait complete. 2024-09-16T00:44:36.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.953+0000 7f576960c640 1 -- 172.21.15.112:0/4148882207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5764064fc0 msgr2=0x7f5764065430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:36.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.953+0000 7f576960c640 1 --2- 172.21.15.112:0/4148882207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5764064fc0 0x7f5764065430 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f574c0048d0 tx=0x7f574c009da0 comp rx=0 tx=0).stop 2024-09-16T00:44:36.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.953+0000 7f576960c640 1 -- 172.21.15.112:0/4148882207 shutdown_connections 2024-09-16T00:44:36.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.953+0000 7f576960c640 1 --2- 172.21.15.112:0/4148882207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5764064fc0 0x7f5764065430 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:36.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.953+0000 7f576960c640 1 --2- 172.21.15.112:0/4148882207 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f57640fe460 0x7f57640fe830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:36.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.953+0000 7f576960c640 1 -- 172.21.15.112:0/4148882207 >> 172.21.15.112:0/4148882207 conn(0x7f57640723b0 msgr2=0x7f57640727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:36.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.954+0000 7f576960c640 1 -- 172.21.15.112:0/4148882207 shutdown_connections 2024-09-16T00:44:36.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.954+0000 7f576960c640 1 -- 172.21.15.112:0/4148882207 wait complete. 2024-09-16T00:44:36.956 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:44:36 smithi112 ceph-mon[97750]: from='client.34968 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:44:36.956 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:44:36 smithi112 ceph-mon[97750]: from='client.34972 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:44:36.956 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:44:36 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/790510876' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:44:36.956 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:44:36 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/1168501364' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:44:36.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.954+0000 7f576960c640 1 Processor -- start 2024-09-16T00:44:36.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.954+0000 7f576960c640 1 -- start start 2024-09-16T00:44:36.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.955+0000 7f576960c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5764064fc0 0x7f57641131d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:36.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.955+0000 7f576960c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57640fe460 0x7f5764113710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:36.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.955+0000 7f576960c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5764113d10 con 0x7f57640fe460 2024-09-16T00:44:36.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.955+0000 7f576960c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f576410d1a0 con 0x7f5764064fc0 2024-09-16T00:44:36.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.955+0000 7f575a7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57640fe460 0x7f5764113710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:36.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.955+0000 7f575a7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57640fe460 0x7f5764113710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33436/0 (socket says 172.21.15.112:33436) 2024-09-16T00:44:36.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.955+0000 7f575a7fc640 1 -- 172.21.15.112:0/3703621091 learned_addr learned my addr 172.21.15.112:0/3703621091 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:44:36.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.955+0000 7f5762ffd640 1 --2- 172.21.15.112:0/3703621091 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5764064fc0 0x7f57641131d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:36.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.955+0000 7f575a7fc640 1 -- 172.21.15.112:0/3703621091 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5764064fc0 msgr2=0x7f57641131d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:36.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.955+0000 7f575a7fc640 1 --2- 172.21.15.112:0/3703621091 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5764064fc0 0x7f57641131d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:36.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.955+0000 7f575a7fc640 1 -- 172.21.15.112:0/3703621091 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f574c005040 con 0x7f57640fe460 2024-09-16T00:44:36.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.955+0000 7f575a7fc640 1 --2- 172.21.15.112:0/3703621091 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57640fe460 0x7f5764113710 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f574c000c00 tx=0x7f574c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:36.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.955+0000 7f575bfff640 1 -- 172.21.15.112:0/3703621091 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f574c038070 con 0x7f57640fe460 2024-09-16T00:44:36.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.955+0000 7f576960c640 1 -- 172.21.15.112:0/3703621091 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f576410d420 con 0x7f57640fe460 2024-09-16T00:44:36.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.955+0000 7f575bfff640 1 -- 172.21.15.112:0/3703621091 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f574c02ade0 con 0x7f57640fe460 2024-09-16T00:44:36.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.956+0000 7f575bfff640 1 -- 172.21.15.112:0/3703621091 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f574c02c7e0 con 0x7f57640fe460 2024-09-16T00:44:36.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.956+0000 7f576960c640 1 -- 172.21.15.112:0/3703621091 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f576410d8e0 con 0x7f57640fe460 2024-09-16T00:44:36.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.957+0000 7f575bfff640 1 -- 172.21.15.112:0/3703621091 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f574c030020 con 0x7f57640fe460 2024-09-16T00:44:36.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.957+0000 7f576960c640 1 -- 172.21.15.112:0/3703621091 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57640fdcf0 con 0x7f57640fe460 2024-09-16T00:44:36.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.957+0000 7f575bfff640 1 --2- 172.21.15.112:0/3703621091 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f575006e530 0x7f57500709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:44:36.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.958+0000 7f575bfff640 1 -- 172.21.15.112:0/3703621091 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f574c0adf70 con 0x7f57640fe460 2024-09-16T00:44:36.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.958+0000 7f5762ffd640 1 --2- 172.21.15.112:0/3703621091 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f575006e530 0x7f57500709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:44:36.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.958+0000 7f5762ffd640 1 --2- 172.21.15.112:0/3703621091 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f575006e530 0x7f57500709e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5754004ed0 tx=0x7f575400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:44:36.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:36.962+0000 7f575bfff640 1 -- 172.21.15.112:0/3703621091 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f574c0e31b0 con 0x7f57640fe460 2024-09-16T00:44:37.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:37.195+0000 7f576960c640 1 -- 172.21.15.112:0/3703621091 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f5764072ec0 con 0x7f57640fe460 2024-09-16T00:44:37.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:37.195+0000 7f575bfff640 1 -- 172.21.15.112:0/3703621091 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f574c076f90 con 0x7f57640fe460 2024-09-16T00:44:37.197 INFO:teuthology.orchestra.run.smithi112.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-16T00:44:37.197 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-16T00:44:37.197 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs is degraded 2024-09-16T00:44:37.197 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T00:44:37.197 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T00:44:37.197 INFO:teuthology.orchestra.run.smithi112.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-16T00:44:37.197 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-16T00:44:37.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:37.198+0000 7f576960c640 1 -- 172.21.15.112:0/3703621091 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f575006e530 msgr2=0x7f57500709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:37.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:37.198+0000 7f576960c640 1 --2- 172.21.15.112:0/3703621091 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f575006e530 0x7f57500709e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5754004ed0 tx=0x7f575400b040 comp rx=0 tx=0).stop 2024-09-16T00:44:37.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:37.198+0000 7f576960c640 1 -- 172.21.15.112:0/3703621091 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57640fe460 msgr2=0x7f5764113710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:44:37.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:37.198+0000 7f576960c640 1 --2- 172.21.15.112:0/3703621091 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57640fe460 0x7f5764113710 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f574c000c00 tx=0x7f574c009da0 comp rx=0 tx=0).stop 2024-09-16T00:44:37.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:37.198+0000 7f576960c640 1 -- 172.21.15.112:0/3703621091 shutdown_connections 2024-09-16T00:44:37.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:37.198+0000 7f576960c640 1 --2- 172.21.15.112:0/3703621091 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f575006e530 0x7f57500709e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:37.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:37.198+0000 7f576960c640 1 --2- 172.21.15.112:0/3703621091 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57640fe460 0x7f5764113710 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:37.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:37.198+0000 7f576960c640 1 --2- 172.21.15.112:0/3703621091 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5764064fc0 0x7f57641131d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:44:37.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:37.198+0000 7f576960c640 1 -- 172.21.15.112:0/3703621091 >> 172.21.15.112:0/3703621091 conn(0x7f57640723b0 msgr2=0x7f57641053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:44:37.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:37.198+0000 7f576960c640 1 -- 172.21.15.112:0/3703621091 shutdown_connections 2024-09-16T00:44:37.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:44:37.198+0000 7f576960c640 1 -- 172.21.15.112:0/3703621091 wait complete. 2024-09-16T00:44:37.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:44:36 smithi169 ceph-mon[77626]: from='client.34968 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:44:37.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:44:36 smithi169 ceph-mon[77626]: from='client.34972 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:44:37.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:44:36 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/790510876' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:44:37.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:44:36 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/1168501364' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:44:38.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:44:37 smithi112 ceph-mon[97750]: from='client.34976 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:44:38.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:44:37 smithi112 ceph-mon[97750]: pgmap v506: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:44:38.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:44:37 smithi112 ceph-mon[97750]: from='client.34988 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:44:38.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:44:37 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/3703621091' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:44:38.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:44:37 smithi169 ceph-mon[77626]: from='client.34976 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:44:38.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:44:37 smithi169 ceph-mon[77626]: pgmap v506: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:44:38.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:44:37 smithi169 ceph-mon[77626]: from='client.34988 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:44:38.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:44:37 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/3703621091' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:44:40.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:44:39 smithi169 ceph-mon[77626]: pgmap v507: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:44:40.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:44:39 smithi112 ceph-mon[97750]: pgmap v507: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:44:42.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:44:41 smithi169 ceph-mon[77626]: pgmap v508: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:44:42.314 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:44:41 smithi112 ceph-mon[97750]: pgmap v508: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:44:44.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:44:43 smithi112 ceph-mon[97750]: pgmap v509: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:44:44.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:44:43 smithi169 ceph-mon[77626]: pgmap v509: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:44:46.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:44:45 smithi169 ceph-mon[77626]: pgmap v510: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:44:46.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:44:45 smithi112 ceph-mon[97750]: pgmap v510: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:44:47.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:44:46 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:44:47.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:44:46 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:44:48.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:44:47 smithi169 ceph-mon[77626]: pgmap v511: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:44:48.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:44:47 smithi112 ceph-mon[97750]: pgmap v511: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:44:50.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:44:49 smithi169 ceph-mon[77626]: pgmap v512: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:44:50.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:44:49 smithi112 ceph-mon[97750]: pgmap v512: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:44:52.141 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:44:51 smithi112 ceph-mon[97750]: pgmap v513: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:44:52.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:44:51 smithi169 ceph-mon[77626]: pgmap v513: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:44:54.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:44:53 smithi112 ceph-mon[97750]: pgmap v514: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:44:54.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:44:53 smithi169 ceph-mon[77626]: pgmap v514: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:44:56.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:44:55 smithi169 ceph-mon[77626]: pgmap v515: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:44:56.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:44:55 smithi112 ceph-mon[97750]: pgmap v515: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:44:58.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:44:57 smithi169 ceph-mon[77626]: pgmap v516: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:44:58.338 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:44:57 smithi112 ceph-mon[97750]: pgmap v516: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:45:00.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:44:59 smithi169 ceph-mon[77626]: pgmap v517: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:45:00.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:44:59 smithi112 ceph-mon[97750]: pgmap v517: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:45:02.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:45:01 smithi169 ceph-mon[77626]: pgmap v518: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:45:02.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:45:01 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:45:02.314 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:45:01 smithi112 ceph-mon[97750]: pgmap v518: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:45:02.314 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:45:01 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:45:04.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:45:03 smithi112 ceph-mon[97750]: pgmap v519: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:45:04.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:45:03 smithi169 ceph-mon[77626]: pgmap v519: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:45:06.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:45:05 smithi169 ceph-mon[77626]: pgmap v520: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:45:06.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:45:05 smithi112 ceph-mon[97750]: pgmap v520: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:45:07.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.290+0000 7f45a07f3640 1 -- 172.21.15.112:0/304051216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45980fb810 msgr2=0x7f45980fbbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:07.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.290+0000 7f45a07f3640 1 --2- 172.21.15.112:0/304051216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45980fb810 0x7f45980fbbe0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f45940048d0 tx=0x7f4594009da0 comp rx=0 tx=0).stop 2024-09-16T00:45:07.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.290+0000 7f45a07f3640 1 -- 172.21.15.112:0/304051216 shutdown_connections 2024-09-16T00:45:07.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.290+0000 7f45a07f3640 1 --2- 172.21.15.112:0/304051216 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45980fcf80 0x7f45980fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:07.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.290+0000 7f45a07f3640 1 --2- 172.21.15.112:0/304051216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45980fb810 0x7f45980fbbe0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:07.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.290+0000 7f45a07f3640 1 -- 172.21.15.112:0/304051216 >> 172.21.15.112:0/304051216 conn(0x7f45980723b0 msgr2=0x7f45980727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:07.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.290+0000 7f45a07f3640 1 -- 172.21.15.112:0/304051216 shutdown_connections 2024-09-16T00:45:07.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.290+0000 7f45a07f3640 1 -- 172.21.15.112:0/304051216 wait complete. 2024-09-16T00:45:07.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.291+0000 7f45a07f3640 1 Processor -- start 2024-09-16T00:45:07.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.291+0000 7f45a07f3640 1 -- start start 2024-09-16T00:45:07.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.291+0000 7f45a07f3640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45980fb810 0x7f45981131b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:07.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.291+0000 7f45a07f3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45980fcf80 0x7f45981136f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:07.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.291+0000 7f45a07f3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4598113cf0 con 0x7f45980fcf80 2024-09-16T00:45:07.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.291+0000 7f45a07f3640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f459810d130 con 0x7f45980fb810 2024-09-16T00:45:07.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.291+0000 7f459dd8e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45980fcf80 0x7f45981136f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:07.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.291+0000 7f459dd8e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45980fcf80 0x7f45981136f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49700/0 (socket says 172.21.15.112:49700) 2024-09-16T00:45:07.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.291+0000 7f459dd8e640 1 -- 172.21.15.112:0/1064807672 learned_addr learned my addr 172.21.15.112:0/1064807672 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:45:07.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.291+0000 7f459e58f640 1 --2- 172.21.15.112:0/1064807672 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45980fb810 0x7f45981131b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:07.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.292+0000 7f459dd8e640 1 -- 172.21.15.112:0/1064807672 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45980fb810 msgr2=0x7f45981131b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:07.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.292+0000 7f459dd8e640 1 --2- 172.21.15.112:0/1064807672 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45980fb810 0x7f45981131b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:07.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.292+0000 7f459dd8e640 1 -- 172.21.15.112:0/1064807672 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4594005040 con 0x7f45980fcf80 2024-09-16T00:45:07.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.292+0000 7f459dd8e640 1 --2- 172.21.15.112:0/1064807672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45980fcf80 0x7f45981136f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f4588002a50 tx=0x7f4588004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:07.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.292+0000 7f458effd640 1 -- 172.21.15.112:0/1064807672 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4588007c30 con 0x7f45980fcf80 2024-09-16T00:45:07.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.292+0000 7f45a07f3640 1 -- 172.21.15.112:0/1064807672 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f459810d410 con 0x7f45980fcf80 2024-09-16T00:45:07.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.292+0000 7f458effd640 1 -- 172.21.15.112:0/1064807672 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f458800f040 con 0x7f45980fcf80 2024-09-16T00:45:07.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.292+0000 7f45a07f3640 1 -- 172.21.15.112:0/1064807672 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f459810d9b0 con 0x7f45980fcf80 2024-09-16T00:45:07.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.292+0000 7f458effd640 1 -- 172.21.15.112:0/1064807672 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f458800da10 con 0x7f45980fcf80 2024-09-16T00:45:07.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.293+0000 7f45a07f3640 1 -- 172.21.15.112:0/1064807672 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45980fcbc0 con 0x7f45980fcf80 2024-09-16T00:45:07.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.294+0000 7f458effd640 1 -- 172.21.15.112:0/1064807672 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f458800dc50 con 0x7f45980fcf80 2024-09-16T00:45:07.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.294+0000 7f458effd640 1 --2- 172.21.15.112:0/1064807672 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f458406e530 0x7f45840709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:07.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.294+0000 7f458effd640 1 -- 172.21.15.112:0/1064807672 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f458808d9e0 con 0x7f45980fcf80 2024-09-16T00:45:07.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.294+0000 7f459e58f640 1 --2- 172.21.15.112:0/1064807672 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f458406e530 0x7f45840709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:07.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.295+0000 7f459e58f640 1 --2- 172.21.15.112:0/1064807672 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f458406e530 0x7f45840709e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4594009ac0 tx=0x7f4594035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:07.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.299+0000 7f458effd640 1 -- 172.21.15.112:0/1064807672 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f4588056a80 con 0x7f45980fcf80 2024-09-16T00:45:07.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.481+0000 7f45a07f3640 1 -- 172.21.15.112:0/1064807672 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f459810e820 con 0x7f458406e530 2024-09-16T00:45:07.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.482+0000 7f458effd640 1 -- 172.21.15.112:0/1064807672 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f459810e820 con 0x7f458406e530 2024-09-16T00:45:07.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.484+0000 7f45a07f3640 1 -- 172.21.15.112:0/1064807672 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f458406e530 msgr2=0x7f45840709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:07.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.484+0000 7f45a07f3640 1 --2- 172.21.15.112:0/1064807672 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f458406e530 0x7f45840709e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4594009ac0 tx=0x7f4594035040 comp rx=0 tx=0).stop 2024-09-16T00:45:07.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.484+0000 7f45a07f3640 1 -- 172.21.15.112:0/1064807672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45980fcf80 msgr2=0x7f45981136f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:07.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.484+0000 7f45a07f3640 1 --2- 172.21.15.112:0/1064807672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45980fcf80 0x7f45981136f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f4588002a50 tx=0x7f4588004600 comp rx=0 tx=0).stop 2024-09-16T00:45:07.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.484+0000 7f45a07f3640 1 -- 172.21.15.112:0/1064807672 shutdown_connections 2024-09-16T00:45:07.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.484+0000 7f45a07f3640 1 --2- 172.21.15.112:0/1064807672 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f458406e530 0x7f45840709e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:07.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.484+0000 7f45a07f3640 1 --2- 172.21.15.112:0/1064807672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45980fcf80 0x7f45981136f0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:07.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.484+0000 7f45a07f3640 1 --2- 172.21.15.112:0/1064807672 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45980fb810 0x7f45981131b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:07.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.484+0000 7f45a07f3640 1 -- 172.21.15.112:0/1064807672 >> 172.21.15.112:0/1064807672 conn(0x7f45980723b0 msgr2=0x7f45980fa480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:07.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.484+0000 7f45a07f3640 1 -- 172.21.15.112:0/1064807672 shutdown_connections 2024-09-16T00:45:07.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.484+0000 7f45a07f3640 1 -- 172.21.15.112:0/1064807672 wait complete. 2024-09-16T00:45:07.493 INFO:teuthology.orchestra.run.smithi112.stdout:true 2024-09-16T00:45:07.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.563+0000 7f00c4fa2640 1 -- 172.21.15.112:0/1299182357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00c00fdf30 msgr2=0x7f00c0106570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:07.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.563+0000 7f00c4fa2640 1 --2- 172.21.15.112:0/1299182357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00c00fdf30 0x7f00c0106570 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f00a80048d0 tx=0x7f00a8009da0 comp rx=0 tx=0).stop 2024-09-16T00:45:07.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.563+0000 7f00c4fa2640 1 -- 172.21.15.112:0/1299182357 shutdown_connections 2024-09-16T00:45:07.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.563+0000 7f00c4fa2640 1 --2- 172.21.15.112:0/1299182357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00c00fdf30 0x7f00c0106570 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:07.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.563+0000 7f00c4fa2640 1 --2- 172.21.15.112:0/1299182357 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00c00fd590 0x7f00c00fd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:07.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.563+0000 7f00c4fa2640 1 -- 172.21.15.112:0/1299182357 >> 172.21.15.112:0/1299182357 conn(0x7f00c00f7060 msgr2=0x7f00c00f9470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:07.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.563+0000 7f00c4fa2640 1 -- 172.21.15.112:0/1299182357 shutdown_connections 2024-09-16T00:45:07.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.563+0000 7f00c4fa2640 1 -- 172.21.15.112:0/1299182357 wait complete. 2024-09-16T00:45:07.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.564+0000 7f00c4fa2640 1 Processor -- start 2024-09-16T00:45:07.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.564+0000 7f00c4fa2640 1 -- start start 2024-09-16T00:45:07.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.564+0000 7f00c4fa2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00c00fd590 0x7f00c0112110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:07.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.564+0000 7f00c4fa2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00c00fdf30 0x7f00c0112650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:07.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.564+0000 7f00c4fa2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c0112ca0 con 0x7f00c00fdf30 2024-09-16T00:45:07.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.564+0000 7f00c4fa2640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c010afa0 con 0x7f00c00fd590 2024-09-16T00:45:07.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.564+0000 7f00b5d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00c00fdf30 0x7f00c0112650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:07.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.564+0000 7f00b5d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00c00fdf30 0x7f00c0112650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49722/0 (socket says 172.21.15.112:49722) 2024-09-16T00:45:07.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.564+0000 7f00b5d9b640 1 -- 172.21.15.112:0/1922071873 learned_addr learned my addr 172.21.15.112:0/1922071873 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:45:07.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.565+0000 7f00be59c640 1 --2- 172.21.15.112:0/1922071873 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00c00fd590 0x7f00c0112110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:07.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.565+0000 7f00b5d9b640 1 -- 172.21.15.112:0/1922071873 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00c00fd590 msgr2=0x7f00c0112110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:07.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.565+0000 7f00b5d9b640 1 --2- 172.21.15.112:0/1922071873 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00c00fd590 0x7f00c0112110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:07.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.565+0000 7f00b5d9b640 1 -- 172.21.15.112:0/1922071873 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00a8005040 con 0x7f00c00fdf30 2024-09-16T00:45:07.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.565+0000 7f00b5d9b640 1 --2- 172.21.15.112:0/1922071873 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00c00fdf30 0x7f00c0112650 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f00a8000c00 tx=0x7f00a8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:07.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.565+0000 7f00b77fe640 1 -- 172.21.15.112:0/1922071873 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f00a8038070 con 0x7f00c00fdf30 2024-09-16T00:45:07.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.565+0000 7f00c4fa2640 1 -- 172.21.15.112:0/1922071873 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00c010b1c0 con 0x7f00c00fdf30 2024-09-16T00:45:07.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.565+0000 7f00b77fe640 1 -- 172.21.15.112:0/1922071873 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f00a802ade0 con 0x7f00c00fdf30 2024-09-16T00:45:07.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.565+0000 7f00b77fe640 1 -- 172.21.15.112:0/1922071873 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f00a802c7e0 con 0x7f00c00fdf30 2024-09-16T00:45:07.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.565+0000 7f00c4fa2640 1 -- 172.21.15.112:0/1922071873 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00c010b680 con 0x7f00c00fdf30 2024-09-16T00:45:07.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.566+0000 7f00c4fa2640 1 -- 172.21.15.112:0/1922071873 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00c0066350 con 0x7f00c00fdf30 2024-09-16T00:45:07.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.567+0000 7f00b77fe640 1 -- 172.21.15.112:0/1922071873 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f00a8030020 con 0x7f00c00fdf30 2024-09-16T00:45:07.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.567+0000 7f00b77fe640 1 --2- 172.21.15.112:0/1922071873 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f00ac06e530 0x7f00ac0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:07.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.568+0000 7f00b77fe640 1 -- 172.21.15.112:0/1922071873 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f00a80adf70 con 0x7f00c00fdf30 2024-09-16T00:45:07.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.568+0000 7f00be59c640 1 --2- 172.21.15.112:0/1922071873 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f00ac06e530 0x7f00ac0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:07.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.568+0000 7f00be59c640 1 --2- 172.21.15.112:0/1922071873 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f00ac06e530 0x7f00ac0709e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f00b0007fd0 tx=0x7f00b0009e90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:07.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.572+0000 7f00b77fe640 1 -- 172.21.15.112:0/1922071873 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f00a80e31b0 con 0x7f00c00fdf30 2024-09-16T00:45:07.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.753+0000 7f00c4fa2640 1 -- 172.21.15.112:0/1922071873 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f00c004b720 con 0x7f00ac06e530 2024-09-16T00:45:07.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.755+0000 7f00b77fe640 1 -- 172.21.15.112:0/1922071873 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f00c004b720 con 0x7f00ac06e530 2024-09-16T00:45:07.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.757+0000 7f00c4fa2640 1 -- 172.21.15.112:0/1922071873 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f00ac06e530 msgr2=0x7f00ac0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:07.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.757+0000 7f00c4fa2640 1 --2- 172.21.15.112:0/1922071873 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f00ac06e530 0x7f00ac0709e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f00b0007fd0 tx=0x7f00b0009e90 comp rx=0 tx=0).stop 2024-09-16T00:45:07.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.757+0000 7f00c4fa2640 1 -- 172.21.15.112:0/1922071873 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00c00fdf30 msgr2=0x7f00c0112650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:07.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.757+0000 7f00c4fa2640 1 --2- 172.21.15.112:0/1922071873 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00c00fdf30 0x7f00c0112650 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f00a8000c00 tx=0x7f00a8009da0 comp rx=0 tx=0).stop 2024-09-16T00:45:07.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.757+0000 7f00c4fa2640 1 -- 172.21.15.112:0/1922071873 shutdown_connections 2024-09-16T00:45:07.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.757+0000 7f00c4fa2640 1 --2- 172.21.15.112:0/1922071873 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f00ac06e530 0x7f00ac0709e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:07.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.757+0000 7f00c4fa2640 1 --2- 172.21.15.112:0/1922071873 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00c00fdf30 0x7f00c0112650 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:07.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.757+0000 7f00c4fa2640 1 --2- 172.21.15.112:0/1922071873 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00c00fd590 0x7f00c0112110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:07.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.757+0000 7f00c4fa2640 1 -- 172.21.15.112:0/1922071873 >> 172.21.15.112:0/1922071873 conn(0x7f00c00f7060 msgr2=0x7f00c00fb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:07.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.757+0000 7f00c4fa2640 1 -- 172.21.15.112:0/1922071873 shutdown_connections 2024-09-16T00:45:07.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.757+0000 7f00c4fa2640 1 -- 172.21.15.112:0/1922071873 wait complete. 2024-09-16T00:45:07.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.835+0000 7fa8527fe640 1 -- 172.21.15.112:0/4233969471 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa84c0fbce0 msgr2=0x7fa84c0fc150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:07.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.835+0000 7fa8527fe640 1 --2- 172.21.15.112:0/4233969471 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa84c0fbce0 0x7fa84c0fc150 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa8340048d0 tx=0x7fa834009da0 comp rx=0 tx=0).stop 2024-09-16T00:45:07.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.835+0000 7fa8527fe640 1 -- 172.21.15.112:0/4233969471 shutdown_connections 2024-09-16T00:45:07.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.835+0000 7fa8527fe640 1 --2- 172.21.15.112:0/4233969471 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa84c0fbce0 0x7fa84c0fc150 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:07.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.835+0000 7fa8527fe640 1 --2- 172.21.15.112:0/4233969471 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa84c101500 0x7fa84c1018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:07.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.835+0000 7fa8527fe640 1 -- 172.21.15.112:0/4233969471 >> 172.21.15.112:0/4233969471 conn(0x7fa84c071ac0 msgr2=0x7fa84c071ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:07.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.835+0000 7fa8527fe640 1 -- 172.21.15.112:0/4233969471 shutdown_connections 2024-09-16T00:45:07.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.835+0000 7fa8527fe640 1 -- 172.21.15.112:0/4233969471 wait complete. 2024-09-16T00:45:07.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.835+0000 7fa8527fe640 1 Processor -- start 2024-09-16T00:45:07.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.836+0000 7fa8527fe640 1 -- start start 2024-09-16T00:45:07.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.836+0000 7fa8527fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa84c0fbce0 0x7fa84c1022d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:07.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.836+0000 7fa8527fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa84c101500 0x7fa84c102810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:07.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.836+0000 7fa8527fe640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa84c102d50 con 0x7fa84c101500 2024-09-16T00:45:07.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.836+0000 7fa8527fe640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa84c102ec0 con 0x7fa84c0fbce0 2024-09-16T00:45:07.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.836+0000 7fa84b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa84c101500 0x7fa84c102810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:07.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.836+0000 7fa84b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa84c101500 0x7fa84c102810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49740/0 (socket says 172.21.15.112:49740) 2024-09-16T00:45:07.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.836+0000 7fa84b7fe640 1 -- 172.21.15.112:0/2120062104 learned_addr learned my addr 172.21.15.112:0/2120062104 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:45:07.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.836+0000 7fa84bfff640 1 --2- 172.21.15.112:0/2120062104 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa84c0fbce0 0x7fa84c1022d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:07.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.836+0000 7fa84b7fe640 1 -- 172.21.15.112:0/2120062104 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa84c0fbce0 msgr2=0x7fa84c1022d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:07.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.836+0000 7fa84b7fe640 1 --2- 172.21.15.112:0/2120062104 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa84c0fbce0 0x7fa84c1022d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:07.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.836+0000 7fa84b7fe640 1 -- 172.21.15.112:0/2120062104 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa834005040 con 0x7fa84c101500 2024-09-16T00:45:07.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.836+0000 7fa84b7fe640 1 --2- 172.21.15.112:0/2120062104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa84c101500 0x7fa84c102810 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa834003080 tx=0x7fa834009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:07.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.837+0000 7fa848ff9640 1 -- 172.21.15.112:0/2120062104 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa834038070 con 0x7fa84c101500 2024-09-16T00:45:07.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.837+0000 7fa848ff9640 1 -- 172.21.15.112:0/2120062104 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fa83402ade0 con 0x7fa84c101500 2024-09-16T00:45:07.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.837+0000 7fa8527fe640 1 -- 172.21.15.112:0/2120062104 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa84c1030f0 con 0x7fa84c101500 2024-09-16T00:45:07.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.837+0000 7fa848ff9640 1 -- 172.21.15.112:0/2120062104 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa83402c880 con 0x7fa84c101500 2024-09-16T00:45:07.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.837+0000 7fa8527fe640 1 -- 172.21.15.112:0/2120062104 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa84c1197d0 con 0x7fa84c101500 2024-09-16T00:45:07.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.838+0000 7fa8527fe640 1 -- 172.21.15.112:0/2120062104 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa84c0fccf0 con 0x7fa84c101500 2024-09-16T00:45:07.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.838+0000 7fa848ff9640 1 -- 172.21.15.112:0/2120062104 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fa834030020 con 0x7fa84c101500 2024-09-16T00:45:07.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.839+0000 7fa848ff9640 1 --2- 172.21.15.112:0/2120062104 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa83806e530 0x7fa8380709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:07.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.839+0000 7fa848ff9640 1 -- 172.21.15.112:0/2120062104 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7fa8340ae260 con 0x7fa84c101500 2024-09-16T00:45:07.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.839+0000 7fa84bfff640 1 --2- 172.21.15.112:0/2120062104 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa83806e530 0x7fa8380709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:07.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.839+0000 7fa84bfff640 1 --2- 172.21.15.112:0/2120062104 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa83806e530 0x7fa8380709e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa83c0041d0 tx=0x7fa83c004330 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:07.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:07.843+0000 7fa848ff9640 1 -- 172.21.15.112:0/2120062104 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fa834077250 con 0x7fa84c101500 2024-09-16T00:45:08.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.019+0000 7fa8527fe640 1 -- 172.21.15.112:0/2120062104 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa84c04b720 con 0x7fa83806e530 2024-09-16T00:45:08.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.028+0000 7fa848ff9640 1 -- 172.21.15.112:0/2120062104 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7fa84c04b720 con 0x7fa83806e530 2024-09-16T00:45:08.028 INFO:teuthology.orchestra.run.smithi112.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T00:45:08.029 INFO:teuthology.orchestra.run.smithi112.stdout:alertmanager.smithi112 smithi112 *:9093,9094 running (18m) 11m ago 26m 24.7M - 0.25.0 c8568f914cd2 d85ea0562022 2024-09-16T00:45:08.029 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi112 smithi112 running (15m) 11m ago 27m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c e4fe58f9702a 2024-09-16T00:45:08.029 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi169 smithi169 running (14m) 10m ago 26m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c 3ba2c81d527f 2024-09-16T00:45:08.029 INFO:teuthology.orchestra.run.smithi112.stdout:grafana.smithi112 smithi112 *:3000 running (16m) 11m ago 26m 82.1M - 9.4.12 2bacad6d85d8 51f9d2013da2 2024-09-16T00:45:08.029 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.hxfwkh smithi112 running (11m) 11m ago 21m 14.2M - 19.1.1-326-g8921c8ef f1455bd0219c 1723069902d6 2024-09-16T00:45:08.029 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.ithwzj smithi112 running (11m) 11m ago 21m 12.2M - 19.1.1-326-g8921c8ef f1455bd0219c e7aad5b81db5 2024-09-16T00:45:08.029 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.gawwgw smithi169 running (10m) 10m ago 21m 15.7M - 19.1.1-326-g8921c8ef f1455bd0219c 469be459ee12 2024-09-16T00:45:08.029 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.lgeqrj smithi169 running (10m) 10m ago 21m 13.3M - 19.1.1-326-g8921c8ef f1455bd0219c 31f0fbd2395d 2024-09-16T00:45:08.029 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi112.xrkkbc smithi112 *:8443,9283,8765 running (17m) 11m ago 28m 504M - 19.1.1-326-g8921c8ef f1455bd0219c a981f7fd2810 2024-09-16T00:45:08.029 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi169.mtrobk smithi169 *:8443,9283,8765 running (17m) 10m ago 26m 442M - 19.1.1-326-g8921c8ef f1455bd0219c 4096e5369c2d 2024-09-16T00:45:08.029 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi112 smithi112 running (15m) 11m ago 28m 49.8M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 5aaec033c042 2024-09-16T00:45:08.029 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi169 smithi169 running (15m) 10m ago 26m 38.9M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 7bc609155f3f 2024-09-16T00:45:08.029 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi112 smithi112 *:9100 running (18m) 11m ago 27m 15.5M - 1.5.0 0da6a335fe13 6404d1d601e2 2024-09-16T00:45:08.030 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi169 smithi169 *:9100 running (17m) 10m ago 26m 15.4M - 1.5.0 0da6a335fe13 5505baf702a6 2024-09-16T00:45:08.030 INFO:teuthology.orchestra.run.smithi112.stdout:osd.0 smithi112 running (14m) 11m ago 25m 105M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 53efc9397017 2024-09-16T00:45:08.030 INFO:teuthology.orchestra.run.smithi112.stdout:osd.1 smithi112 running (14m) 11m ago 24m 98.9M 1557M 19.1.1-326-g8921c8ef f1455bd0219c fe6f1a4a2cf4 2024-09-16T00:45:08.030 INFO:teuthology.orchestra.run.smithi112.stdout:osd.2 smithi112 running (13m) 11m ago 24m 102M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 5f7863712057 2024-09-16T00:45:08.030 INFO:teuthology.orchestra.run.smithi112.stdout:osd.3 smithi169 running (12m) 10m ago 24m 92.1M 2581M 19.1.1-326-g8921c8ef f1455bd0219c ee07b5577810 2024-09-16T00:45:08.030 INFO:teuthology.orchestra.run.smithi112.stdout:osd.4 smithi169 running (12m) 10m ago 23m 103M 2581M 19.1.1-326-g8921c8ef f1455bd0219c 769f026a54eb 2024-09-16T00:45:08.030 INFO:teuthology.orchestra.run.smithi112.stdout:osd.5 smithi169 running (11m) 10m ago 23m 125M 2581M 19.1.1-326-g8921c8ef f1455bd0219c a6007c13cf78 2024-09-16T00:45:08.030 INFO:teuthology.orchestra.run.smithi112.stdout:prometheus.smithi112 smithi112 *:9095 running (17m) 11m ago 26m 64.1M - 2.43.0 a07b618ecd1d 7a1d42e60d2b 2024-09-16T00:45:08.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.030+0000 7fa8527fe640 1 -- 172.21.15.112:0/2120062104 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa83806e530 msgr2=0x7fa8380709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:08.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.030+0000 7fa8527fe640 1 --2- 172.21.15.112:0/2120062104 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa83806e530 0x7fa8380709e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa83c0041d0 tx=0x7fa83c004330 comp rx=0 tx=0).stop 2024-09-16T00:45:08.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.030+0000 7fa8527fe640 1 -- 172.21.15.112:0/2120062104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa84c101500 msgr2=0x7fa84c102810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:08.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.030+0000 7fa8527fe640 1 --2- 172.21.15.112:0/2120062104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa84c101500 0x7fa84c102810 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa834003080 tx=0x7fa834009da0 comp rx=0 tx=0).stop 2024-09-16T00:45:08.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.030+0000 7fa8527fe640 1 -- 172.21.15.112:0/2120062104 shutdown_connections 2024-09-16T00:45:08.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.030+0000 7fa8527fe640 1 --2- 172.21.15.112:0/2120062104 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa83806e530 0x7fa8380709e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:08.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.030+0000 7fa8527fe640 1 --2- 172.21.15.112:0/2120062104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa84c101500 0x7fa84c102810 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:08.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.030+0000 7fa8527fe640 1 --2- 172.21.15.112:0/2120062104 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa84c0fbce0 0x7fa84c1022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:08.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.030+0000 7fa8527fe640 1 -- 172.21.15.112:0/2120062104 >> 172.21.15.112:0/2120062104 conn(0x7fa84c071ac0 msgr2=0x7fa84c104ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:08.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.031+0000 7fa8527fe640 1 -- 172.21.15.112:0/2120062104 shutdown_connections 2024-09-16T00:45:08.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.031+0000 7fa8527fe640 1 -- 172.21.15.112:0/2120062104 wait complete. 2024-09-16T00:45:08.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:45:07 smithi112 ceph-mon[97750]: pgmap v521: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:45:08.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.108+0000 7f0dbfe76640 1 -- 172.21.15.112:0/1872590743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0db80fac60 msgr2=0x7f0db80fb0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:08.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.108+0000 7f0dbfe76640 1 --2- 172.21.15.112:0/1872590743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0db80fac60 0x7f0db80fb0b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0db40048d0 tx=0x7f0db4009da0 comp rx=0 tx=0).stop 2024-09-16T00:45:08.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.108+0000 7f0dbfe76640 1 -- 172.21.15.112:0/1872590743 shutdown_connections 2024-09-16T00:45:08.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.108+0000 7f0dbfe76640 1 --2- 172.21.15.112:0/1872590743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0db80fac60 0x7f0db80fb0b0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:08.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.108+0000 7f0dbfe76640 1 --2- 172.21.15.112:0/1872590743 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0db81025f0 0x7f0db81029c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:08.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.108+0000 7f0dbfe76640 1 -- 172.21.15.112:0/1872590743 >> 172.21.15.112:0/1872590743 conn(0x7f0db8070c80 msgr2=0x7f0db8071080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:08.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.108+0000 7f0dbfe76640 1 -- 172.21.15.112:0/1872590743 shutdown_connections 2024-09-16T00:45:08.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.108+0000 7f0dbfe76640 1 -- 172.21.15.112:0/1872590743 wait complete. 2024-09-16T00:45:08.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.108+0000 7f0dbfe76640 1 Processor -- start 2024-09-16T00:45:08.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.109+0000 7f0dbfe76640 1 -- start start 2024-09-16T00:45:08.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.109+0000 7f0dbfe76640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0db80fac60 0x7f0db81131d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:08.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.109+0000 7f0dbfe76640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0db81025f0 0x7f0db8113710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:08.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.109+0000 7f0dbfe76640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0db8113d10 con 0x7f0db80fac60 2024-09-16T00:45:08.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.109+0000 7f0dbfe76640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0db810d150 con 0x7f0db81025f0 2024-09-16T00:45:08.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.109+0000 7f0dbdc12640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0db80fac60 0x7f0db81131d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:08.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.109+0000 7f0dbdc12640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0db80fac60 0x7f0db81131d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49764/0 (socket says 172.21.15.112:49764) 2024-09-16T00:45:08.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.109+0000 7f0dbdc12640 1 -- 172.21.15.112:0/2561391833 learned_addr learned my addr 172.21.15.112:0/2561391833 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:45:08.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.109+0000 7f0dbd411640 1 --2- 172.21.15.112:0/2561391833 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0db81025f0 0x7f0db8113710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:08.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.109+0000 7f0dbdc12640 1 -- 172.21.15.112:0/2561391833 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0db81025f0 msgr2=0x7f0db8113710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:08.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.109+0000 7f0dbdc12640 1 --2- 172.21.15.112:0/2561391833 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0db81025f0 0x7f0db8113710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:08.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.109+0000 7f0dbdc12640 1 -- 172.21.15.112:0/2561391833 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0db4005040 con 0x7f0db80fac60 2024-09-16T00:45:08.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.109+0000 7f0dbdc12640 1 --2- 172.21.15.112:0/2561391833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0db80fac60 0x7f0db81131d0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f0dac00c790 tx=0x7f0dac0043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:08.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.109+0000 7f0daa7fc640 1 -- 172.21.15.112:0/2561391833 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0dac00a7a0 con 0x7f0db80fac60 2024-09-16T00:45:08.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.110+0000 7f0dbfe76640 1 -- 172.21.15.112:0/2561391833 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0db810d430 con 0x7f0db80fac60 2024-09-16T00:45:08.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.110+0000 7f0daa7fc640 1 -- 172.21.15.112:0/2561391833 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f0dac00ada0 con 0x7f0db80fac60 2024-09-16T00:45:08.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.110+0000 7f0daa7fc640 1 -- 172.21.15.112:0/2561391833 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0dac0113a0 con 0x7f0db80fac60 2024-09-16T00:45:08.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.110+0000 7f0dbfe76640 1 -- 172.21.15.112:0/2561391833 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0db810d9a0 con 0x7f0db80fac60 2024-09-16T00:45:08.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.111+0000 7f0dbfe76640 1 -- 172.21.15.112:0/2561391833 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0db8065810 con 0x7f0db80fac60 2024-09-16T00:45:08.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.111+0000 7f0daa7fc640 1 -- 172.21.15.112:0/2561391833 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f0dac00a900 con 0x7f0db80fac60 2024-09-16T00:45:08.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.111+0000 7f0daa7fc640 1 --2- 172.21.15.112:0/2561391833 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f0da406e530 0x7f0da40709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:08.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.112+0000 7f0daa7fc640 1 -- 172.21.15.112:0/2561391833 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f0dac08e400 con 0x7f0db80fac60 2024-09-16T00:45:08.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.112+0000 7f0dbd411640 1 --2- 172.21.15.112:0/2561391833 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f0da406e530 0x7f0da40709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:08.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.112+0000 7f0dbd411640 1 --2- 172.21.15.112:0/2561391833 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f0da406e530 0x7f0da40709e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0db4009ac0 tx=0x7f0db4035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:08.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.116+0000 7f0daa7fc640 1 -- 172.21.15.112:0/2561391833 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f0dac0573f0 con 0x7f0db80fac60 2024-09-16T00:45:08.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:45:07 smithi169 ceph-mon[77626]: pgmap v521: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:45:08.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.347+0000 7f0dbfe76640 1 -- 172.21.15.112:0/2561391833 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f0db804b720 con 0x7f0db80fac60 2024-09-16T00:45:08.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.348+0000 7f0daa7fc640 1 -- 172.21.15.112:0/2561391833 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f0dac05aab0 con 0x7f0db80fac60 2024-09-16T00:45:08.348 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:45:08.348 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": { 2024-09-16T00:45:08.348 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:45:08.348 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:45:08.348 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": { 2024-09-16T00:45:08.348 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:45:08.348 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:45:08.348 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": { 2024-09-16T00:45:08.349 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 6 2024-09-16T00:45:08.349 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:45:08.349 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": { 2024-09-16T00:45:08.349 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 4 2024-09-16T00:45:08.349 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:45:08.349 INFO:teuthology.orchestra.run.smithi112.stdout: "overall": { 2024-09-16T00:45:08.349 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 14 2024-09-16T00:45:08.349 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-16T00:45:08.349 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:45:08.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.350+0000 7f0dbfe76640 1 -- 172.21.15.112:0/2561391833 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f0da406e530 msgr2=0x7f0da40709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:08.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.350+0000 7f0dbfe76640 1 --2- 172.21.15.112:0/2561391833 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f0da406e530 0x7f0da40709e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0db4009ac0 tx=0x7f0db4035040 comp rx=0 tx=0).stop 2024-09-16T00:45:08.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.350+0000 7f0dbfe76640 1 -- 172.21.15.112:0/2561391833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0db80fac60 msgr2=0x7f0db81131d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:08.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.350+0000 7f0dbfe76640 1 --2- 172.21.15.112:0/2561391833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0db80fac60 0x7f0db81131d0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f0dac00c790 tx=0x7f0dac0043c0 comp rx=0 tx=0).stop 2024-09-16T00:45:08.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.350+0000 7f0dbfe76640 1 -- 172.21.15.112:0/2561391833 shutdown_connections 2024-09-16T00:45:08.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.350+0000 7f0dbfe76640 1 --2- 172.21.15.112:0/2561391833 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f0da406e530 0x7f0da40709e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:08.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.350+0000 7f0dbfe76640 1 --2- 172.21.15.112:0/2561391833 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0db81025f0 0x7f0db8113710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:08.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.350+0000 7f0dbfe76640 1 --2- 172.21.15.112:0/2561391833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0db80fac60 0x7f0db81131d0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:08.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.350+0000 7f0dbfe76640 1 -- 172.21.15.112:0/2561391833 >> 172.21.15.112:0/2561391833 conn(0x7f0db8070c80 msgr2=0x7f0db80f9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:08.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.350+0000 7f0dbfe76640 1 -- 172.21.15.112:0/2561391833 shutdown_connections 2024-09-16T00:45:08.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.350+0000 7f0dbfe76640 1 -- 172.21.15.112:0/2561391833 wait complete. 2024-09-16T00:45:08.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.428+0000 7fbfd9cd6640 1 -- 172.21.15.112:0/2757756114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfd40fdf40 msgr2=0x7fbfd4106610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:08.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.428+0000 7fbfd9cd6640 1 --2- 172.21.15.112:0/2757756114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfd40fdf40 0x7fbfd4106610 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fbfc80048d0 tx=0x7fbfc8009da0 comp rx=0 tx=0).stop 2024-09-16T00:45:08.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.428+0000 7fbfd9cd6640 1 -- 172.21.15.112:0/2757756114 shutdown_connections 2024-09-16T00:45:08.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.428+0000 7fbfd9cd6640 1 --2- 172.21.15.112:0/2757756114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfd40fdf40 0x7fbfd4106610 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:08.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.428+0000 7fbfd9cd6640 1 --2- 172.21.15.112:0/2757756114 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfd40fd5a0 0x7fbfd40fd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:08.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.428+0000 7fbfd9cd6640 1 -- 172.21.15.112:0/2757756114 >> 172.21.15.112:0/2757756114 conn(0x7fbfd40f7060 msgr2=0x7fbfd40f9470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:08.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.428+0000 7fbfd9cd6640 1 -- 172.21.15.112:0/2757756114 shutdown_connections 2024-09-16T00:45:08.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.428+0000 7fbfd9cd6640 1 -- 172.21.15.112:0/2757756114 wait complete. 2024-09-16T00:45:08.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.429+0000 7fbfd9cd6640 1 Processor -- start 2024-09-16T00:45:08.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.429+0000 7fbfd9cd6640 1 -- start start 2024-09-16T00:45:08.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.429+0000 7fbfd9cd6640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfd40fd5a0 0x7fbfd4113250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:08.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.429+0000 7fbfd9cd6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfd40fdf40 0x7fbfd4113790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:08.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.429+0000 7fbfd9cd6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd4113d90 con 0x7fbfd40fdf40 2024-09-16T00:45:08.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.429+0000 7fbfd9cd6640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd410d1d0 con 0x7fbfd40fd5a0 2024-09-16T00:45:08.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.430+0000 7fbfd2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfd40fdf40 0x7fbfd4113790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:08.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.430+0000 7fbfd2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfd40fdf40 0x7fbfd4113790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49792/0 (socket says 172.21.15.112:49792) 2024-09-16T00:45:08.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.430+0000 7fbfd2ffd640 1 -- 172.21.15.112:0/1251829525 learned_addr learned my addr 172.21.15.112:0/1251829525 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:45:08.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.430+0000 7fbfd37fe640 1 --2- 172.21.15.112:0/1251829525 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfd40fd5a0 0x7fbfd4113250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:08.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.430+0000 7fbfd2ffd640 1 -- 172.21.15.112:0/1251829525 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfd40fd5a0 msgr2=0x7fbfd4113250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:08.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.430+0000 7fbfd2ffd640 1 --2- 172.21.15.112:0/1251829525 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfd40fd5a0 0x7fbfd4113250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:08.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.430+0000 7fbfd2ffd640 1 -- 172.21.15.112:0/1251829525 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc8005040 con 0x7fbfd40fdf40 2024-09-16T00:45:08.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.430+0000 7fbfd2ffd640 1 --2- 172.21.15.112:0/1251829525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfd40fdf40 0x7fbfd4113790 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8000c00 tx=0x7fbfc8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:08.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.430+0000 7fbfbbfff640 1 -- 172.21.15.112:0/1251829525 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbfc8038070 con 0x7fbfd40fdf40 2024-09-16T00:45:08.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.430+0000 7fbfd9cd6640 1 -- 172.21.15.112:0/1251829525 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfd410d450 con 0x7fbfd40fdf40 2024-09-16T00:45:08.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.430+0000 7fbfbbfff640 1 -- 172.21.15.112:0/1251829525 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fbfc802ade0 con 0x7fbfd40fdf40 2024-09-16T00:45:08.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.430+0000 7fbfbbfff640 1 -- 172.21.15.112:0/1251829525 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbfc802c770 con 0x7fbfd40fdf40 2024-09-16T00:45:08.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.430+0000 7fbfd9cd6640 1 -- 172.21.15.112:0/1251829525 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfd410d910 con 0x7fbfd40fdf40 2024-09-16T00:45:08.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.431+0000 7fbfd9cd6640 1 -- 172.21.15.112:0/1251829525 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfd40662b0 con 0x7fbfd40fdf40 2024-09-16T00:45:08.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.433+0000 7fbfbbfff640 1 -- 172.21.15.112:0/1251829525 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fbfc8030020 con 0x7fbfd40fdf40 2024-09-16T00:45:08.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.433+0000 7fbfbbfff640 1 --2- 172.21.15.112:0/1251829525 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fbfbc06e530 0x7fbfbc0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:08.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.433+0000 7fbfbbfff640 1 -- 172.21.15.112:0/1251829525 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7fbfc80ae250 con 0x7fbfd40fdf40 2024-09-16T00:45:08.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.433+0000 7fbfd37fe640 1 --2- 172.21.15.112:0/1251829525 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fbfbc06e530 0x7fbfbc0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:08.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.434+0000 7fbfd37fe640 1 --2- 172.21.15.112:0/1251829525 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fbfbc06e530 0x7fbfbc0709e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4007fa0 tx=0x7fbfc4004c70 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:08.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.437+0000 7fbfbbfff640 1 -- 172.21.15.112:0/1251829525 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fbfc80772f0 con 0x7fbfd40fdf40 2024-09-16T00:45:08.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.639+0000 7fbfd9cd6640 1 -- 172.21.15.112:0/1251829525 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fbfd40fac60 con 0x7fbfd40fdf40 2024-09-16T00:45:08.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.639+0000 7fbfbbfff640 1 -- 172.21.15.112:0/1251829525 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1914 (secure 0 0 0) 0x7fbfc804b790 con 0x7fbfd40fdf40 2024-09-16T00:45:08.640 INFO:teuthology.orchestra.run.smithi112.stdout:e34 2024-09-16T00:45:08.640 INFO:teuthology.orchestra.run.smithi112.stdout:btime 2024-09-16T00:34:26:619254+0000 2024-09-16T00:45:08.640 INFO:teuthology.orchestra.run.smithi112.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T00:45:08.640 INFO:teuthology.orchestra.run.smithi112.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:45:08.640 INFO:teuthology.orchestra.run.smithi112.stdout:legacy client fscid: 1 2024-09-16T00:45:08.640 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:45:08.640 INFO:teuthology.orchestra.run.smithi112.stdout:Filesystem 'cephfs' (1) 2024-09-16T00:45:08.640 INFO:teuthology.orchestra.run.smithi112.stdout:fs_name cephfs 2024-09-16T00:45:08.640 INFO:teuthology.orchestra.run.smithi112.stdout:epoch 25 2024-09-16T00:45:08.640 INFO:teuthology.orchestra.run.smithi112.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-16T00:45:08.640 INFO:teuthology.orchestra.run.smithi112.stdout:created 2024-09-16T00:23:22.328199+0000 2024-09-16T00:45:08.641 INFO:teuthology.orchestra.run.smithi112.stdout:modified 2024-09-16T00:33:29.012991+0000 2024-09-16T00:45:08.641 INFO:teuthology.orchestra.run.smithi112.stdout:tableserver 0 2024-09-16T00:45:08.641 INFO:teuthology.orchestra.run.smithi112.stdout:root 0 2024-09-16T00:45:08.641 INFO:teuthology.orchestra.run.smithi112.stdout:session_timeout 60 2024-09-16T00:45:08.641 INFO:teuthology.orchestra.run.smithi112.stdout:session_autoclose 300 2024-09-16T00:45:08.641 INFO:teuthology.orchestra.run.smithi112.stdout:max_file_size 1099511627776 2024-09-16T00:45:08.641 INFO:teuthology.orchestra.run.smithi112.stdout:max_xattr_size 65536 2024-09-16T00:45:08.641 INFO:teuthology.orchestra.run.smithi112.stdout:required_client_features {} 2024-09-16T00:45:08.641 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure 0 2024-09-16T00:45:08.641 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure_osd_epoch 82 2024-09-16T00:45:08.641 INFO:teuthology.orchestra.run.smithi112.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:45:08.642 INFO:teuthology.orchestra.run.smithi112.stdout:max_mds 2 2024-09-16T00:45:08.642 INFO:teuthology.orchestra.run.smithi112.stdout:in 0,1 2024-09-16T00:45:08.642 INFO:teuthology.orchestra.run.smithi112.stdout:up {} 2024-09-16T00:45:08.642 INFO:teuthology.orchestra.run.smithi112.stdout:failed 0,1 2024-09-16T00:45:08.642 INFO:teuthology.orchestra.run.smithi112.stdout:damaged 2024-09-16T00:45:08.642 INFO:teuthology.orchestra.run.smithi112.stdout:stopped 2024-09-16T00:45:08.642 INFO:teuthology.orchestra.run.smithi112.stdout:data_pools [3] 2024-09-16T00:45:08.642 INFO:teuthology.orchestra.run.smithi112.stdout:metadata_pool 2 2024-09-16T00:45:08.642 INFO:teuthology.orchestra.run.smithi112.stdout:inline_data enabled 2024-09-16T00:45:08.642 INFO:teuthology.orchestra.run.smithi112.stdout:balancer 2024-09-16T00:45:08.642 INFO:teuthology.orchestra.run.smithi112.stdout:bal_rank_mask -1 2024-09-16T00:45:08.642 INFO:teuthology.orchestra.run.smithi112.stdout:standby_count_wanted 1 2024-09-16T00:45:08.642 INFO:teuthology.orchestra.run.smithi112.stdout:qdb_cluster leader: 0 members: 2024-09-16T00:45:08.643 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:45:08.643 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:45:08.643 INFO:teuthology.orchestra.run.smithi112.stdout:Standby daemons: 2024-09-16T00:45:08.643 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:45:08.643 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.ithwzj{-1:34392} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.112:6826/476229221,v1:172.21.15.112:6827/476229221] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:45:08.643 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.hxfwkh{-1:34404} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.112:6828/2356899876,v1:172.21.15.112:6829/2356899876] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:45:08.643 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.gawwgw{-1:44343} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.169:6824/1635329957,v1:172.21.15.169:6825/1635329957] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:45:08.643 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.lgeqrj{-1:44347} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.169:6826/184810031,v1:172.21.15.169:6827/184810031] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:45:08.644 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 34 2024-09-16T00:45:08.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.641+0000 7fbfd9cd6640 1 -- 172.21.15.112:0/1251829525 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fbfbc06e530 msgr2=0x7fbfbc0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:08.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.641+0000 7fbfd9cd6640 1 --2- 172.21.15.112:0/1251829525 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fbfbc06e530 0x7fbfbc0709e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4007fa0 tx=0x7fbfc4004c70 comp rx=0 tx=0).stop 2024-09-16T00:45:08.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.641+0000 7fbfd9cd6640 1 -- 172.21.15.112:0/1251829525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfd40fdf40 msgr2=0x7fbfd4113790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:08.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.641+0000 7fbfd9cd6640 1 --2- 172.21.15.112:0/1251829525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfd40fdf40 0x7fbfd4113790 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8000c00 tx=0x7fbfc8009da0 comp rx=0 tx=0).stop 2024-09-16T00:45:08.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.641+0000 7fbfd9cd6640 1 -- 172.21.15.112:0/1251829525 shutdown_connections 2024-09-16T00:45:08.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.641+0000 7fbfd9cd6640 1 --2- 172.21.15.112:0/1251829525 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fbfbc06e530 0x7fbfbc0709e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:08.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.641+0000 7fbfd9cd6640 1 --2- 172.21.15.112:0/1251829525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfd40fdf40 0x7fbfd4113790 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:08.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.641+0000 7fbfd9cd6640 1 --2- 172.21.15.112:0/1251829525 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfd40fd5a0 0x7fbfd4113250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:08.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.641+0000 7fbfd9cd6640 1 -- 172.21.15.112:0/1251829525 >> 172.21.15.112:0/1251829525 conn(0x7fbfd40f7060 msgr2=0x7fbfd40f85a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:08.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.642+0000 7fbfd9cd6640 1 -- 172.21.15.112:0/1251829525 shutdown_connections 2024-09-16T00:45:08.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.642+0000 7fbfd9cd6640 1 -- 172.21.15.112:0/1251829525 wait complete. 2024-09-16T00:45:08.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.720+0000 7f365340c640 1 -- 172.21.15.112:0/2766161737 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f364c0fe460 msgr2=0x7f364c0fe830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:08.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.720+0000 7f365340c640 1 --2- 172.21.15.112:0/2766161737 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f364c0fe460 0x7f364c0fe830 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f363c009c50 tx=0x7f363c004d10 comp rx=0 tx=0).stop 2024-09-16T00:45:08.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.721+0000 7f365340c640 1 -- 172.21.15.112:0/2766161737 shutdown_connections 2024-09-16T00:45:08.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.721+0000 7f365340c640 1 --2- 172.21.15.112:0/2766161737 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f364c064fc0 0x7f364c065430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:08.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.721+0000 7f365340c640 1 --2- 172.21.15.112:0/2766161737 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f364c0fe460 0x7f364c0fe830 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:08.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.721+0000 7f365340c640 1 -- 172.21.15.112:0/2766161737 >> 172.21.15.112:0/2766161737 conn(0x7f364c0723b0 msgr2=0x7f364c0727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:08.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.721+0000 7f365340c640 1 -- 172.21.15.112:0/2766161737 shutdown_connections 2024-09-16T00:45:08.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.721+0000 7f365340c640 1 -- 172.21.15.112:0/2766161737 wait complete. 2024-09-16T00:45:08.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.721+0000 7f365340c640 1 Processor -- start 2024-09-16T00:45:08.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.721+0000 7f365340c640 1 -- start start 2024-09-16T00:45:08.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.722+0000 7f365340c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f364c064fc0 0x7f364c113190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:08.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.722+0000 7f365340c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f364c0fe460 0x7f364c1136d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:08.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.722+0000 7f365340c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364c113cd0 con 0x7f364c0fe460 2024-09-16T00:45:08.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.722+0000 7f365340c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364c10d110 con 0x7f364c064fc0 2024-09-16T00:45:08.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.722+0000 7f36509a7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f364c0fe460 0x7f364c1136d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:08.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.722+0000 7f36509a7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f364c0fe460 0x7f364c1136d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49816/0 (socket says 172.21.15.112:49816) 2024-09-16T00:45:08.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.722+0000 7f36509a7640 1 -- 172.21.15.112:0/2663353268 learned_addr learned my addr 172.21.15.112:0/2663353268 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:45:08.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.722+0000 7f36511a8640 1 --2- 172.21.15.112:0/2663353268 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f364c064fc0 0x7f364c113190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:08.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.722+0000 7f36509a7640 1 -- 172.21.15.112:0/2663353268 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f364c064fc0 msgr2=0x7f364c113190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:08.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.722+0000 7f36509a7640 1 --2- 172.21.15.112:0/2663353268 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f364c064fc0 0x7f364c113190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:08.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.722+0000 7f36509a7640 1 -- 172.21.15.112:0/2663353268 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f363c0098b0 con 0x7f364c0fe460 2024-09-16T00:45:08.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.722+0000 7f36511a8640 1 --2- 172.21.15.112:0/2663353268 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f364c064fc0 0x7f364c113190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:45:08.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.722+0000 7f36509a7640 1 --2- 172.21.15.112:0/2663353268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f364c0fe460 0x7f364c1136d0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f3640002a80 tx=0x7f3640004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:08.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.722+0000 7f3649ffb640 1 -- 172.21.15.112:0/2663353268 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3640007c30 con 0x7f364c0fe460 2024-09-16T00:45:08.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.723+0000 7f365340c640 1 -- 172.21.15.112:0/2663353268 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f364c10d3f0 con 0x7f364c0fe460 2024-09-16T00:45:08.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.723+0000 7f3649ffb640 1 -- 172.21.15.112:0/2663353268 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f364000f040 con 0x7f364c0fe460 2024-09-16T00:45:08.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.723+0000 7f3649ffb640 1 -- 172.21.15.112:0/2663353268 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3640009530 con 0x7f364c0fe460 2024-09-16T00:45:08.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.723+0000 7f365340c640 1 -- 172.21.15.112:0/2663353268 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f364c10d910 con 0x7f364c0fe460 2024-09-16T00:45:08.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.724+0000 7f365340c640 1 -- 172.21.15.112:0/2663353268 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f364c0fdcf0 con 0x7f364c0fe460 2024-09-16T00:45:08.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.724+0000 7f3649ffb640 1 -- 172.21.15.112:0/2663353268 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f3640009690 con 0x7f364c0fe460 2024-09-16T00:45:08.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.725+0000 7f3649ffb640 1 --2- 172.21.15.112:0/2663353268 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f363406e530 0x7f36340709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:08.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.725+0000 7f3649ffb640 1 -- 172.21.15.112:0/2663353268 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f364008dc30 con 0x7f364c0fe460 2024-09-16T00:45:08.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.725+0000 7f36511a8640 1 --2- 172.21.15.112:0/2663353268 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f363406e530 0x7f36340709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:08.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.726+0000 7f36511a8640 1 --2- 172.21.15.112:0/2663353268 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f363406e530 0x7f36340709e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f363c004a30 tx=0x7f363c004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:08.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.731+0000 7f3649ffb640 1 -- 172.21.15.112:0/2663353268 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f3640056c20 con 0x7f364c0fe460 2024-09-16T00:45:08.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.918+0000 7f365340c640 1 -- 172.21.15.112:0/2663353268 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f364c10e830 con 0x7f363406e530 2024-09-16T00:45:08.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.919+0000 7f3649ffb640 1 -- 172.21.15.112:0/2663353268 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f364c10e830 con 0x7f363406e530 2024-09-16T00:45:08.919 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:45:08.919 INFO:teuthology.orchestra.run.smithi112.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80", 2024-09-16T00:45:08.919 INFO:teuthology.orchestra.run.smithi112.stdout: "in_progress": true, 2024-09-16T00:45:08.919 INFO:teuthology.orchestra.run.smithi112.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T00:45:08.919 INFO:teuthology.orchestra.run.smithi112.stdout: "services_complete": [ 2024-09-16T00:45:08.920 INFO:teuthology.orchestra.run.smithi112.stdout: "mds", 2024-09-16T00:45:08.920 INFO:teuthology.orchestra.run.smithi112.stdout: "osd", 2024-09-16T00:45:08.920 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr", 2024-09-16T00:45:08.920 INFO:teuthology.orchestra.run.smithi112.stdout: "mon", 2024-09-16T00:45:08.920 INFO:teuthology.orchestra.run.smithi112.stdout: "crash" 2024-09-16T00:45:08.920 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-16T00:45:08.920 INFO:teuthology.orchestra.run.smithi112.stdout: "progress": "16/21 daemons upgraded", 2024-09-16T00:45:08.920 INFO:teuthology.orchestra.run.smithi112.stdout: "message": "Currently upgrading mds daemons", 2024-09-16T00:45:08.920 INFO:teuthology.orchestra.run.smithi112.stdout: "is_paused": false 2024-09-16T00:45:08.920 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:45:08.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.921+0000 7f365340c640 1 -- 172.21.15.112:0/2663353268 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f363406e530 msgr2=0x7f36340709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:08.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.921+0000 7f365340c640 1 --2- 172.21.15.112:0/2663353268 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f363406e530 0x7f36340709e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f363c004a30 tx=0x7f363c004d10 comp rx=0 tx=0).stop 2024-09-16T00:45:08.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.921+0000 7f365340c640 1 -- 172.21.15.112:0/2663353268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f364c0fe460 msgr2=0x7f364c1136d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:08.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.921+0000 7f365340c640 1 --2- 172.21.15.112:0/2663353268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f364c0fe460 0x7f364c1136d0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f3640002a80 tx=0x7f3640004600 comp rx=0 tx=0).stop 2024-09-16T00:45:08.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.921+0000 7f365340c640 1 -- 172.21.15.112:0/2663353268 shutdown_connections 2024-09-16T00:45:08.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.921+0000 7f365340c640 1 --2- 172.21.15.112:0/2663353268 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f363406e530 0x7f36340709e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:08.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.921+0000 7f365340c640 1 --2- 172.21.15.112:0/2663353268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f364c0fe460 0x7f364c1136d0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:08.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.921+0000 7f365340c640 1 --2- 172.21.15.112:0/2663353268 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f364c064fc0 0x7f364c113190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:08.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.921+0000 7f365340c640 1 -- 172.21.15.112:0/2663353268 >> 172.21.15.112:0/2663353268 conn(0x7f364c0723b0 msgr2=0x7f364c105410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:08.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.921+0000 7f365340c640 1 -- 172.21.15.112:0/2663353268 shutdown_connections 2024-09-16T00:45:08.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.921+0000 7f365340c640 1 -- 172.21.15.112:0/2663353268 wait complete. 2024-09-16T00:45:09.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.999+0000 7fb605996640 1 -- 172.21.15.112:0/3762787562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6000fe460 msgr2=0x7fb6000fe830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:09.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.999+0000 7fb605996640 1 --2- 172.21.15.112:0/3762787562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6000fe460 0x7fb6000fe830 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fb5f0009c50 tx=0x7fb5f0004d10 comp rx=0 tx=0).stop 2024-09-16T00:45:09.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.999+0000 7fb605996640 1 -- 172.21.15.112:0/3762787562 shutdown_connections 2024-09-16T00:45:09.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.999+0000 7fb605996640 1 --2- 172.21.15.112:0/3762787562 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb600064fc0 0x7fb600065430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:09.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.999+0000 7fb605996640 1 --2- 172.21.15.112:0/3762787562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6000fe460 0x7fb6000fe830 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:09.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.999+0000 7fb605996640 1 -- 172.21.15.112:0/3762787562 >> 172.21.15.112:0/3762787562 conn(0x7fb6000723b0 msgr2=0x7fb6000727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:09.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:08.999+0000 7fb605996640 1 -- 172.21.15.112:0/3762787562 shutdown_connections 2024-09-16T00:45:09.001 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:45:08 smithi112 ceph-mon[97750]: from='client.34996 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:45:09.001 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:45:08 smithi112 ceph-mon[97750]: from='client.35000 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:45:09.001 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:45:08 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/2561391833' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:45:09.001 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:45:08 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/1251829525' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:45:09.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:09.000+0000 7fb605996640 1 -- 172.21.15.112:0/3762787562 wait complete. 2024-09-16T00:45:09.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:09.000+0000 7fb605996640 1 Processor -- start 2024-09-16T00:45:09.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:09.000+0000 7fb605996640 1 -- start start 2024-09-16T00:45:09.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:09.000+0000 7fb605996640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb600064fc0 0x7fb600113190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:09.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:09.000+0000 7fb605996640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6000fe460 0x7fb6001136d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:09.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:09.000+0000 7fb605996640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb600113cd0 con 0x7fb600064fc0 2024-09-16T00:45:09.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:09.000+0000 7fb605996640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb60010d110 con 0x7fb6000fe460 2024-09-16T00:45:09.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:09.001+0000 7fb5feffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb600064fc0 0x7fb600113190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:09.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:09.001+0000 7fb5feffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb600064fc0 0x7fb600113190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49842/0 (socket says 172.21.15.112:49842) 2024-09-16T00:45:09.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:09.001+0000 7fb5feffd640 1 -- 172.21.15.112:0/1819426131 learned_addr learned my addr 172.21.15.112:0/1819426131 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:45:09.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:09.001+0000 7fb5fe7fc640 1 --2- 172.21.15.112:0/1819426131 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6000fe460 0x7fb6001136d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:09.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:09.001+0000 7fb5feffd640 1 -- 172.21.15.112:0/1819426131 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6000fe460 msgr2=0x7fb6001136d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:09.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:09.001+0000 7fb5feffd640 1 --2- 172.21.15.112:0/1819426131 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6000fe460 0x7fb6001136d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:09.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:09.001+0000 7fb5feffd640 1 -- 172.21.15.112:0/1819426131 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f00098b0 con 0x7fb600064fc0 2024-09-16T00:45:09.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:09.001+0000 7fb5feffd640 1 --2- 172.21.15.112:0/1819426131 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb600064fc0 0x7fb600113190 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fb5f0004a30 tx=0x7fb5f0004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:09.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:09.001+0000 7fb5e77fe640 1 -- 172.21.15.112:0/1819426131 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5f0009690 con 0x7fb600064fc0 2024-09-16T00:45:09.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:09.001+0000 7fb5e77fe640 1 -- 172.21.15.112:0/1819426131 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fb5f0032b10 con 0x7fb600064fc0 2024-09-16T00:45:09.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:09.001+0000 7fb605996640 1 -- 172.21.15.112:0/1819426131 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb60010d390 con 0x7fb600064fc0 2024-09-16T00:45:09.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:09.001+0000 7fb5e77fe640 1 -- 172.21.15.112:0/1819426131 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5f003daa0 con 0x7fb600064fc0 2024-09-16T00:45:09.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:09.001+0000 7fb605996640 1 -- 172.21.15.112:0/1819426131 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb60010d930 con 0x7fb600064fc0 2024-09-16T00:45:09.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:09.002+0000 7fb605996640 1 -- 172.21.15.112:0/1819426131 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6000fdcf0 con 0x7fb600064fc0 2024-09-16T00:45:09.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:09.003+0000 7fb5e77fe640 1 -- 172.21.15.112:0/1819426131 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fb5f002f030 con 0x7fb600064fc0 2024-09-16T00:45:09.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:09.004+0000 7fb5e77fe640 1 --2- 172.21.15.112:0/1819426131 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fb5e806e530 0x7fb5e80709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:09.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:09.004+0000 7fb5e77fe640 1 -- 172.21.15.112:0/1819426131 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7fb5f00af3c0 con 0x7fb600064fc0 2024-09-16T00:45:09.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:09.004+0000 7fb5fe7fc640 1 --2- 172.21.15.112:0/1819426131 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fb5e806e530 0x7fb5e80709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:09.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:09.004+0000 7fb5fe7fc640 1 --2- 172.21.15.112:0/1819426131 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fb5e806e530 0x7fb5e80709e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb60010ea50 tx=0x7fb5f4004330 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:09.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:09.008+0000 7fb5e77fe640 1 -- 172.21.15.112:0/1819426131 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fb5f0078330 con 0x7fb600064fc0 2024-09-16T00:45:09.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:09.242+0000 7fb605996640 1 -- 172.21.15.112:0/1819426131 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fb60004b7b0 con 0x7fb600064fc0 2024-09-16T00:45:09.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:09.242+0000 7fb5e77fe640 1 -- 172.21.15.112:0/1819426131 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7fb5f007b9f0 con 0x7fb600064fc0 2024-09-16T00:45:09.242 INFO:teuthology.orchestra.run.smithi112.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-16T00:45:09.243 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-16T00:45:09.243 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs is degraded 2024-09-16T00:45:09.243 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T00:45:09.243 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T00:45:09.243 INFO:teuthology.orchestra.run.smithi112.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-16T00:45:09.243 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-16T00:45:09.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:09.244+0000 7fb605996640 1 -- 172.21.15.112:0/1819426131 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fb5e806e530 msgr2=0x7fb5e80709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:09.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:09.244+0000 7fb605996640 1 --2- 172.21.15.112:0/1819426131 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fb5e806e530 0x7fb5e80709e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb60010ea50 tx=0x7fb5f4004330 comp rx=0 tx=0).stop 2024-09-16T00:45:09.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:09.244+0000 7fb605996640 1 -- 172.21.15.112:0/1819426131 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb600064fc0 msgr2=0x7fb600113190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:09.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:09.244+0000 7fb605996640 1 --2- 172.21.15.112:0/1819426131 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb600064fc0 0x7fb600113190 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fb5f0004a30 tx=0x7fb5f0004d10 comp rx=0 tx=0).stop 2024-09-16T00:45:09.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:09.244+0000 7fb605996640 1 -- 172.21.15.112:0/1819426131 shutdown_connections 2024-09-16T00:45:09.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:09.244+0000 7fb605996640 1 --2- 172.21.15.112:0/1819426131 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fb5e806e530 0x7fb5e80709e0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:09.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:09.244+0000 7fb605996640 1 --2- 172.21.15.112:0/1819426131 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6000fe460 0x7fb6001136d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:09.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:09.244+0000 7fb605996640 1 --2- 172.21.15.112:0/1819426131 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb600064fc0 0x7fb600113190 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:09.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:09.244+0000 7fb605996640 1 -- 172.21.15.112:0/1819426131 >> 172.21.15.112:0/1819426131 conn(0x7fb6000723b0 msgr2=0x7fb600105410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:09.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:09.245+0000 7fb605996640 1 -- 172.21.15.112:0/1819426131 shutdown_connections 2024-09-16T00:45:09.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:09.245+0000 7fb605996640 1 -- 172.21.15.112:0/1819426131 wait complete. 2024-09-16T00:45:09.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:45:08 smithi169 ceph-mon[77626]: from='client.34996 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:45:09.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:45:08 smithi169 ceph-mon[77626]: from='client.35000 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:45:09.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:45:08 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/2561391833' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:45:09.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:45:08 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/1251829525' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:45:10.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:45:09 smithi169 ceph-mon[77626]: from='client.35004 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:45:10.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:45:09 smithi169 ceph-mon[77626]: pgmap v522: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:45:10.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:45:09 smithi169 ceph-mon[77626]: from='client.35016 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:45:10.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:45:09 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/1819426131' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:45:10.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:45:09 smithi112 ceph-mon[97750]: from='client.35004 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:45:10.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:45:09 smithi112 ceph-mon[97750]: pgmap v522: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:45:10.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:45:09 smithi112 ceph-mon[97750]: from='client.35016 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:45:10.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:45:09 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/1819426131' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:45:11.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:45:10 smithi169 ceph-mon[77626]: pgmap v523: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:45:11.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:45:10 smithi112 ceph-mon[97750]: pgmap v523: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:45:14.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:45:13 smithi112 ceph-mon[97750]: pgmap v524: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:45:14.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:45:13 smithi169 ceph-mon[77626]: pgmap v524: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:45:16.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:45:15 smithi112 ceph-mon[97750]: pgmap v525: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:45:16.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:45:15 smithi169 ceph-mon[77626]: pgmap v525: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:45:17.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:45:16 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:45:17.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:45:16 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:45:18.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:45:17 smithi112 ceph-mon[97750]: pgmap v526: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:45:18.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:45:17 smithi169 ceph-mon[77626]: pgmap v526: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:45:20.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:45:19 smithi112 ceph-mon[97750]: pgmap v527: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:45:20.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:45:19 smithi169 ceph-mon[77626]: pgmap v527: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:45:22.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:45:21 smithi112 ceph-mon[97750]: pgmap v528: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:45:22.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:45:21 smithi169 ceph-mon[77626]: pgmap v528: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:45:24.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:45:23 smithi112 ceph-mon[97750]: pgmap v529: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:45:24.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:45:23 smithi169 ceph-mon[77626]: pgmap v529: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:45:26.078 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:45:25 smithi169 ceph-mon[77626]: pgmap v530: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:45:26.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:45:25 smithi112 ceph-mon[97750]: pgmap v530: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:45:28.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:45:27 smithi112 ceph-mon[97750]: pgmap v531: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:45:28.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:45:27 smithi169 ceph-mon[77626]: pgmap v531: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:45:30.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:45:29 smithi112 ceph-mon[97750]: pgmap v532: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:45:30.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:45:29 smithi169 ceph-mon[77626]: pgmap v532: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:45:32.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:45:31 smithi112 ceph-mon[97750]: pgmap v533: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:45:32.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:45:31 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:45:32.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:45:31 smithi169 ceph-mon[77626]: pgmap v533: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:45:32.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:45:31 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:45:34.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:45:33 smithi112 ceph-mon[97750]: pgmap v534: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:45:34.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:45:33 smithi169 ceph-mon[77626]: pgmap v534: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:45:36.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:45:35 smithi112 ceph-mon[97750]: pgmap v535: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:45:36.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:45:35 smithi169 ceph-mon[77626]: pgmap v535: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:45:38.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:45:37 smithi112 ceph-mon[97750]: pgmap v536: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:45:38.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:45:37 smithi169 ceph-mon[77626]: pgmap v536: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:45:39.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.335+0000 7f35f9d34640 1 -- 172.21.15.112:0/3062316483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35f4102620 msgr2=0x7f35f41029f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:39.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.335+0000 7f35f9d34640 1 --2- 172.21.15.112:0/3062316483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35f4102620 0x7f35f41029f0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f35e40048d0 tx=0x7f35e4009da0 comp rx=0 tx=0).stop 2024-09-16T00:45:39.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.335+0000 7f35f9d34640 1 -- 172.21.15.112:0/3062316483 shutdown_connections 2024-09-16T00:45:39.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.335+0000 7f35f9d34640 1 --2- 172.21.15.112:0/3062316483 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35f40fac80 0x7f35f40fb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:39.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.335+0000 7f35f9d34640 1 --2- 172.21.15.112:0/3062316483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35f4102620 0x7f35f41029f0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:39.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.335+0000 7f35f9d34640 1 -- 172.21.15.112:0/3062316483 >> 172.21.15.112:0/3062316483 conn(0x7f35f4070ce0 msgr2=0x7f35f40710e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:39.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.335+0000 7f35f9d34640 1 -- 172.21.15.112:0/3062316483 shutdown_connections 2024-09-16T00:45:39.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.335+0000 7f35f9d34640 1 -- 172.21.15.112:0/3062316483 wait complete. 2024-09-16T00:45:39.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.336+0000 7f35f9d34640 1 Processor -- start 2024-09-16T00:45:39.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.336+0000 7f35f9d34640 1 -- start start 2024-09-16T00:45:39.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.336+0000 7f35f9d34640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35f40fac80 0x7f35f41131d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:39.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.336+0000 7f35f9d34640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35f4102620 0x7f35f4113710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:39.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.336+0000 7f35f9d34640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f4113d10 con 0x7f35f40fac80 2024-09-16T00:45:39.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.336+0000 7f35f9d34640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f410d150 con 0x7f35f4102620 2024-09-16T00:45:39.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.336+0000 7f35f37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35f40fac80 0x7f35f41131d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:39.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.336+0000 7f35f37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35f40fac80 0x7f35f41131d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38814/0 (socket says 172.21.15.112:38814) 2024-09-16T00:45:39.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.336+0000 7f35f37fe640 1 -- 172.21.15.112:0/1526196961 learned_addr learned my addr 172.21.15.112:0/1526196961 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:45:39.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.337+0000 7f35f2ffd640 1 --2- 172.21.15.112:0/1526196961 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35f4102620 0x7f35f4113710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:39.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.337+0000 7f35f37fe640 1 -- 172.21.15.112:0/1526196961 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35f4102620 msgr2=0x7f35f4113710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:39.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.337+0000 7f35f37fe640 1 --2- 172.21.15.112:0/1526196961 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35f4102620 0x7f35f4113710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:39.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.337+0000 7f35f37fe640 1 -- 172.21.15.112:0/1526196961 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35e4005040 con 0x7f35f40fac80 2024-09-16T00:45:39.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.337+0000 7f35f37fe640 1 --2- 172.21.15.112:0/1526196961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35f40fac80 0x7f35f41131d0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f35e40043d0 tx=0x7f35e4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:39.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.337+0000 7f35dbfff640 1 -- 172.21.15.112:0/1526196961 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f35e4038070 con 0x7f35f40fac80 2024-09-16T00:45:39.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.337+0000 7f35dbfff640 1 -- 172.21.15.112:0/1526196961 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f35e40096e0 con 0x7f35f40fac80 2024-09-16T00:45:39.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.337+0000 7f35f9d34640 1 -- 172.21.15.112:0/1526196961 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35f410d3d0 con 0x7f35f40fac80 2024-09-16T00:45:39.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.337+0000 7f35dbfff640 1 -- 172.21.15.112:0/1526196961 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f35e403d420 con 0x7f35f40fac80 2024-09-16T00:45:39.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.337+0000 7f35f9d34640 1 -- 172.21.15.112:0/1526196961 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35f410d970 con 0x7f35f40fac80 2024-09-16T00:45:39.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.338+0000 7f35f9d34640 1 -- 172.21.15.112:0/1526196961 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35f40657b0 con 0x7f35f40fac80 2024-09-16T00:45:39.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.339+0000 7f35dbfff640 1 -- 172.21.15.112:0/1526196961 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f35e4030020 con 0x7f35f40fac80 2024-09-16T00:45:39.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.339+0000 7f35dbfff640 1 --2- 172.21.15.112:0/1526196961 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f35dc06e530 0x7f35dc0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:39.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.339+0000 7f35dbfff640 1 -- 172.21.15.112:0/1526196961 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f35e40ae8b0 con 0x7f35f40fac80 2024-09-16T00:45:39.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.339+0000 7f35f2ffd640 1 --2- 172.21.15.112:0/1526196961 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f35dc06e530 0x7f35dc0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:39.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.340+0000 7f35f2ffd640 1 --2- 172.21.15.112:0/1526196961 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f35dc06e530 0x7f35dc0709e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f35e8009fd0 tx=0x7f35e8009e50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:39.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.344+0000 7f35dbfff640 1 -- 172.21.15.112:0/1526196961 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f35e4077820 con 0x7f35f40fac80 2024-09-16T00:45:39.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.525+0000 7f35f9d34640 1 -- 172.21.15.112:0/1526196961 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f35f404b720 con 0x7f35dc06e530 2024-09-16T00:45:39.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.526+0000 7f35dbfff640 1 -- 172.21.15.112:0/1526196961 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f35f404b720 con 0x7f35dc06e530 2024-09-16T00:45:39.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.528+0000 7f35f9d34640 1 -- 172.21.15.112:0/1526196961 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f35dc06e530 msgr2=0x7f35dc0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:39.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.528+0000 7f35f9d34640 1 --2- 172.21.15.112:0/1526196961 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f35dc06e530 0x7f35dc0709e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f35e8009fd0 tx=0x7f35e8009e50 comp rx=0 tx=0).stop 2024-09-16T00:45:39.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.528+0000 7f35f9d34640 1 -- 172.21.15.112:0/1526196961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35f40fac80 msgr2=0x7f35f41131d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:39.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.528+0000 7f35f9d34640 1 --2- 172.21.15.112:0/1526196961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35f40fac80 0x7f35f41131d0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f35e40043d0 tx=0x7f35e4009da0 comp rx=0 tx=0).stop 2024-09-16T00:45:39.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.528+0000 7f35f9d34640 1 -- 172.21.15.112:0/1526196961 shutdown_connections 2024-09-16T00:45:39.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.528+0000 7f35f9d34640 1 --2- 172.21.15.112:0/1526196961 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f35dc06e530 0x7f35dc0709e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:39.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.528+0000 7f35f9d34640 1 --2- 172.21.15.112:0/1526196961 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35f4102620 0x7f35f4113710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:39.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.528+0000 7f35f9d34640 1 --2- 172.21.15.112:0/1526196961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35f40fac80 0x7f35f41131d0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:39.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.528+0000 7f35f9d34640 1 -- 172.21.15.112:0/1526196961 >> 172.21.15.112:0/1526196961 conn(0x7f35f4070ce0 msgr2=0x7f35f4105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:39.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.528+0000 7f35f9d34640 1 -- 172.21.15.112:0/1526196961 shutdown_connections 2024-09-16T00:45:39.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.528+0000 7f35f9d34640 1 -- 172.21.15.112:0/1526196961 wait complete. 2024-09-16T00:45:39.537 INFO:teuthology.orchestra.run.smithi112.stdout:true 2024-09-16T00:45:39.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.607+0000 7f376c910640 1 -- 172.21.15.112:0/3761181772 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37680fdf30 msgr2=0x7f3768106570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:39.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.607+0000 7f376c910640 1 --2- 172.21.15.112:0/3761181772 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37680fdf30 0x7f3768106570 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f375c0048d0 tx=0x7f375c009da0 comp rx=0 tx=0).stop 2024-09-16T00:45:39.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.607+0000 7f376c910640 1 -- 172.21.15.112:0/3761181772 shutdown_connections 2024-09-16T00:45:39.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.607+0000 7f376c910640 1 --2- 172.21.15.112:0/3761181772 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37680fdf30 0x7f3768106570 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:39.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.607+0000 7f376c910640 1 --2- 172.21.15.112:0/3761181772 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37680fd590 0x7f37680fd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:39.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.608+0000 7f376c910640 1 -- 172.21.15.112:0/3761181772 >> 172.21.15.112:0/3761181772 conn(0x7f37680f7060 msgr2=0x7f37680f9470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:39.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.608+0000 7f376c910640 1 -- 172.21.15.112:0/3761181772 shutdown_connections 2024-09-16T00:45:39.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.608+0000 7f376c910640 1 -- 172.21.15.112:0/3761181772 wait complete. 2024-09-16T00:45:39.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.608+0000 7f376c910640 1 Processor -- start 2024-09-16T00:45:39.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.608+0000 7f376c910640 1 -- start start 2024-09-16T00:45:39.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.608+0000 7f376c910640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37680fd590 0x7f3768113260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:39.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.608+0000 7f376c910640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37680fdf30 0x7f37681137a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:39.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.608+0000 7f376c910640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3768113da0 con 0x7f37680fd590 2024-09-16T00:45:39.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.608+0000 7f376c910640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f376810d1e0 con 0x7f37680fdf30 2024-09-16T00:45:39.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.609+0000 7f376659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37680fd590 0x7f3768113260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:39.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.609+0000 7f376659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37680fd590 0x7f3768113260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38830/0 (socket says 172.21.15.112:38830) 2024-09-16T00:45:39.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.609+0000 7f376659c640 1 -- 172.21.15.112:0/857554733 learned_addr learned my addr 172.21.15.112:0/857554733 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:45:39.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.609+0000 7f3765d9b640 1 --2- 172.21.15.112:0/857554733 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37680fdf30 0x7f37681137a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:39.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.609+0000 7f376659c640 1 -- 172.21.15.112:0/857554733 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37680fdf30 msgr2=0x7f37681137a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:39.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.609+0000 7f376659c640 1 --2- 172.21.15.112:0/857554733 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37680fdf30 0x7f37681137a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:39.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.609+0000 7f376659c640 1 -- 172.21.15.112:0/857554733 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f375c005040 con 0x7f37680fd590 2024-09-16T00:45:39.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.609+0000 7f376659c640 1 --2- 172.21.15.112:0/857554733 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37680fd590 0x7f3768113260 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f375800db00 tx=0x7f3758004e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:39.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.609+0000 7f3756ffd640 1 -- 172.21.15.112:0/857554733 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3758017070 con 0x7f37680fd590 2024-09-16T00:45:39.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.609+0000 7f3756ffd640 1 -- 172.21.15.112:0/857554733 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f375800f040 con 0x7f37680fd590 2024-09-16T00:45:39.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.609+0000 7f376c910640 1 -- 172.21.15.112:0/857554733 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f376810d4c0 con 0x7f37680fd590 2024-09-16T00:45:39.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.609+0000 7f3756ffd640 1 -- 172.21.15.112:0/857554733 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3758003940 con 0x7f37680fd590 2024-09-16T00:45:39.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.609+0000 7f376c910640 1 -- 172.21.15.112:0/857554733 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f376810d9e0 con 0x7f37680fd590 2024-09-16T00:45:39.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.610+0000 7f376c910640 1 -- 172.21.15.112:0/857554733 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3768066350 con 0x7f37680fd590 2024-09-16T00:45:39.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.611+0000 7f3756ffd640 1 -- 172.21.15.112:0/857554733 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f3758003b40 con 0x7f37680fd590 2024-09-16T00:45:39.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.611+0000 7f3756ffd640 1 --2- 172.21.15.112:0/857554733 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f375006e530 0x7f37500709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:39.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.611+0000 7f3756ffd640 1 -- 172.21.15.112:0/857554733 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f375808e370 con 0x7f37680fd590 2024-09-16T00:45:39.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.611+0000 7f3765d9b640 1 --2- 172.21.15.112:0/857554733 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f375006e530 0x7f37500709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:39.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.612+0000 7f3765d9b640 1 --2- 172.21.15.112:0/857554733 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f375006e530 0x7f37500709e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f375c009ac0 tx=0x7f375c009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:39.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.615+0000 7f3756ffd640 1 -- 172.21.15.112:0/857554733 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f3758057360 con 0x7f37680fd590 2024-09-16T00:45:39.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.803+0000 7f376c910640 1 -- 172.21.15.112:0/857554733 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f376804b720 con 0x7f375006e530 2024-09-16T00:45:39.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.805+0000 7f3756ffd640 1 -- 172.21.15.112:0/857554733 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f376804b720 con 0x7f375006e530 2024-09-16T00:45:39.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.807+0000 7f376c910640 1 -- 172.21.15.112:0/857554733 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f375006e530 msgr2=0x7f37500709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:39.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.807+0000 7f376c910640 1 --2- 172.21.15.112:0/857554733 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f375006e530 0x7f37500709e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f375c009ac0 tx=0x7f375c009da0 comp rx=0 tx=0).stop 2024-09-16T00:45:39.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.807+0000 7f376c910640 1 -- 172.21.15.112:0/857554733 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37680fd590 msgr2=0x7f3768113260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:39.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.807+0000 7f376c910640 1 --2- 172.21.15.112:0/857554733 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37680fd590 0x7f3768113260 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f375800db00 tx=0x7f3758004e60 comp rx=0 tx=0).stop 2024-09-16T00:45:39.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.807+0000 7f376c910640 1 -- 172.21.15.112:0/857554733 shutdown_connections 2024-09-16T00:45:39.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.807+0000 7f376c910640 1 --2- 172.21.15.112:0/857554733 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f375006e530 0x7f37500709e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:39.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.807+0000 7f376c910640 1 --2- 172.21.15.112:0/857554733 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37680fdf30 0x7f37681137a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:39.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.807+0000 7f376c910640 1 --2- 172.21.15.112:0/857554733 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37680fd590 0x7f3768113260 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:39.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.807+0000 7f376c910640 1 -- 172.21.15.112:0/857554733 >> 172.21.15.112:0/857554733 conn(0x7f37680f7060 msgr2=0x7f37680f8580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:39.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.807+0000 7f376c910640 1 -- 172.21.15.112:0/857554733 shutdown_connections 2024-09-16T00:45:39.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.807+0000 7f376c910640 1 -- 172.21.15.112:0/857554733 wait complete. 2024-09-16T00:45:39.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.886+0000 7fef1ccd8640 1 -- 172.21.15.112:0/2191858231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef18102530 msgr2=0x7fef18102900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:39.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.886+0000 7fef1ccd8640 1 --2- 172.21.15.112:0/2191858231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef18102530 0x7fef18102900 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fef080048d0 tx=0x7fef08009da0 comp rx=0 tx=0).stop 2024-09-16T00:45:39.886 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:45:39 smithi112 ceph-mon[97750]: pgmap v537: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:45:39.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.886+0000 7fef1ccd8640 1 -- 172.21.15.112:0/2191858231 shutdown_connections 2024-09-16T00:45:39.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.886+0000 7fef1ccd8640 1 --2- 172.21.15.112:0/2191858231 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef18102ed0 0x7fef180fc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:39.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.886+0000 7fef1ccd8640 1 --2- 172.21.15.112:0/2191858231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef18102530 0x7fef18102900 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:39.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.886+0000 7fef1ccd8640 1 -- 172.21.15.112:0/2191858231 >> 172.21.15.112:0/2191858231 conn(0x7fef180f80f0 msgr2=0x7fef180fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:39.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.886+0000 7fef1ccd8640 1 -- 172.21.15.112:0/2191858231 shutdown_connections 2024-09-16T00:45:39.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.886+0000 7fef1ccd8640 1 -- 172.21.15.112:0/2191858231 wait complete. 2024-09-16T00:45:39.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.886+0000 7fef1ccd8640 1 Processor -- start 2024-09-16T00:45:39.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.886+0000 7fef1ccd8640 1 -- start start 2024-09-16T00:45:39.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.887+0000 7fef1ccd8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef18102530 0x7fef1806fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:39.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.887+0000 7fef1ccd8640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef18102ed0 0x7fef1806ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:39.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.887+0000 7fef1ccd8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef18070490 con 0x7fef18102530 2024-09-16T00:45:39.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.887+0000 7fef1ccd8640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef18070600 con 0x7fef18102ed0 2024-09-16T00:45:39.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.887+0000 7fef1659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef18102530 0x7fef1806fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:39.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.887+0000 7fef1659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef18102530 0x7fef1806fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38854/0 (socket says 172.21.15.112:38854) 2024-09-16T00:45:39.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.887+0000 7fef1659c640 1 -- 172.21.15.112:0/562889948 learned_addr learned my addr 172.21.15.112:0/562889948 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:45:39.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.887+0000 7fef15d9b640 1 --2- 172.21.15.112:0/562889948 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef18102ed0 0x7fef1806ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:39.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.887+0000 7fef1659c640 1 -- 172.21.15.112:0/562889948 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef18102ed0 msgr2=0x7fef1806ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:39.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.887+0000 7fef1659c640 1 --2- 172.21.15.112:0/562889948 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef18102ed0 0x7fef1806ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:39.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.887+0000 7fef1659c640 1 -- 172.21.15.112:0/562889948 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef08005040 con 0x7fef18102530 2024-09-16T00:45:39.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.887+0000 7fef1659c640 1 --2- 172.21.15.112:0/562889948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef18102530 0x7fef1806fa10 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fef08000c00 tx=0x7fef08009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:39.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.888+0000 7fef06ffd640 1 -- 172.21.15.112:0/562889948 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef08038070 con 0x7fef18102530 2024-09-16T00:45:39.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.888+0000 7fef1ccd8640 1 -- 172.21.15.112:0/562889948 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef18070830 con 0x7fef18102530 2024-09-16T00:45:39.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.888+0000 7fef06ffd640 1 -- 172.21.15.112:0/562889948 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fef0802ade0 con 0x7fef18102530 2024-09-16T00:45:39.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.888+0000 7fef06ffd640 1 -- 172.21.15.112:0/562889948 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef0802c7e0 con 0x7fef18102530 2024-09-16T00:45:39.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.888+0000 7fef1ccd8640 1 -- 172.21.15.112:0/562889948 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef18119810 con 0x7fef18102530 2024-09-16T00:45:39.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.889+0000 7fef1ccd8640 1 -- 172.21.15.112:0/562889948 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef180fdcb0 con 0x7fef18102530 2024-09-16T00:45:39.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.889+0000 7fef06ffd640 1 -- 172.21.15.112:0/562889948 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fef08030020 con 0x7fef18102530 2024-09-16T00:45:39.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.890+0000 7fef06ffd640 1 --2- 172.21.15.112:0/562889948 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fef0006e530 0x7fef000709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:39.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.890+0000 7fef06ffd640 1 -- 172.21.15.112:0/562889948 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7fef080aded0 con 0x7fef18102530 2024-09-16T00:45:39.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.890+0000 7fef15d9b640 1 --2- 172.21.15.112:0/562889948 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fef0006e530 0x7fef000709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:39.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.890+0000 7fef15d9b640 1 --2- 172.21.15.112:0/562889948 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fef0006e530 0x7fef000709e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fef18071350 tx=0x7fef0c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:39.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:39.894+0000 7fef06ffd640 1 -- 172.21.15.112:0/562889948 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fef08076f70 con 0x7fef18102530 2024-09-16T00:45:40.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.067+0000 7fef1ccd8640 1 -- 172.21.15.112:0/562889948 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fef18071100 con 0x7fef0006e530 2024-09-16T00:45:40.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.076+0000 7fef06ffd640 1 -- 172.21.15.112:0/562889948 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7fef18071100 con 0x7fef0006e530 2024-09-16T00:45:40.076 INFO:teuthology.orchestra.run.smithi112.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T00:45:40.076 INFO:teuthology.orchestra.run.smithi112.stdout:alertmanager.smithi112 smithi112 *:9093,9094 running (18m) 11m ago 27m 24.7M - 0.25.0 c8568f914cd2 d85ea0562022 2024-09-16T00:45:40.076 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi112 smithi112 running (15m) 11m ago 27m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c e4fe58f9702a 2024-09-16T00:45:40.076 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi169 smithi169 running (15m) 11m ago 26m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c 3ba2c81d527f 2024-09-16T00:45:40.076 INFO:teuthology.orchestra.run.smithi112.stdout:grafana.smithi112 smithi112 *:3000 running (17m) 11m ago 27m 82.1M - 9.4.12 2bacad6d85d8 51f9d2013da2 2024-09-16T00:45:40.076 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.hxfwkh smithi112 running (11m) 11m ago 22m 14.2M - 19.1.1-326-g8921c8ef f1455bd0219c 1723069902d6 2024-09-16T00:45:40.077 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.ithwzj smithi112 running (12m) 11m ago 22m 12.2M - 19.1.1-326-g8921c8ef f1455bd0219c e7aad5b81db5 2024-09-16T00:45:40.077 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.gawwgw smithi169 running (11m) 11m ago 22m 15.7M - 19.1.1-326-g8921c8ef f1455bd0219c 469be459ee12 2024-09-16T00:45:40.077 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.lgeqrj smithi169 running (11m) 11m ago 22m 13.3M - 19.1.1-326-g8921c8ef f1455bd0219c 31f0fbd2395d 2024-09-16T00:45:40.077 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi112.xrkkbc smithi112 *:8443,9283,8765 running (18m) 11m ago 28m 504M - 19.1.1-326-g8921c8ef f1455bd0219c a981f7fd2810 2024-09-16T00:45:40.077 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi169.mtrobk smithi169 *:8443,9283,8765 running (17m) 11m ago 26m 442M - 19.1.1-326-g8921c8ef f1455bd0219c 4096e5369c2d 2024-09-16T00:45:40.077 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi112 smithi112 running (16m) 11m ago 28m 49.8M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 5aaec033c042 2024-09-16T00:45:40.077 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi169 smithi169 running (16m) 11m ago 26m 38.9M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 7bc609155f3f 2024-09-16T00:45:40.077 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi112 smithi112 *:9100 running (18m) 11m ago 27m 15.5M - 1.5.0 0da6a335fe13 6404d1d601e2 2024-09-16T00:45:40.077 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi169 smithi169 *:9100 running (18m) 11m ago 26m 15.4M - 1.5.0 0da6a335fe13 5505baf702a6 2024-09-16T00:45:40.077 INFO:teuthology.orchestra.run.smithi112.stdout:osd.0 smithi112 running (15m) 11m ago 25m 105M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 53efc9397017 2024-09-16T00:45:40.077 INFO:teuthology.orchestra.run.smithi112.stdout:osd.1 smithi112 running (14m) 11m ago 25m 98.9M 1557M 19.1.1-326-g8921c8ef f1455bd0219c fe6f1a4a2cf4 2024-09-16T00:45:40.077 INFO:teuthology.orchestra.run.smithi112.stdout:osd.2 smithi112 running (13m) 11m ago 24m 102M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 5f7863712057 2024-09-16T00:45:40.077 INFO:teuthology.orchestra.run.smithi112.stdout:osd.3 smithi169 running (13m) 11m ago 24m 92.1M 2581M 19.1.1-326-g8921c8ef f1455bd0219c ee07b5577810 2024-09-16T00:45:40.078 INFO:teuthology.orchestra.run.smithi112.stdout:osd.4 smithi169 running (12m) 11m ago 24m 103M 2581M 19.1.1-326-g8921c8ef f1455bd0219c 769f026a54eb 2024-09-16T00:45:40.078 INFO:teuthology.orchestra.run.smithi112.stdout:osd.5 smithi169 running (12m) 11m ago 23m 125M 2581M 19.1.1-326-g8921c8ef f1455bd0219c a6007c13cf78 2024-09-16T00:45:40.078 INFO:teuthology.orchestra.run.smithi112.stdout:prometheus.smithi112 smithi112 *:9095 running (17m) 11m ago 27m 64.1M - 2.43.0 a07b618ecd1d 7a1d42e60d2b 2024-09-16T00:45:40.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.078+0000 7fef1ccd8640 1 -- 172.21.15.112:0/562889948 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fef0006e530 msgr2=0x7fef000709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:40.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.078+0000 7fef1ccd8640 1 --2- 172.21.15.112:0/562889948 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fef0006e530 0x7fef000709e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fef18071350 tx=0x7fef0c00b040 comp rx=0 tx=0).stop 2024-09-16T00:45:40.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.078+0000 7fef1ccd8640 1 -- 172.21.15.112:0/562889948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef18102530 msgr2=0x7fef1806fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:40.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.078+0000 7fef1ccd8640 1 --2- 172.21.15.112:0/562889948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef18102530 0x7fef1806fa10 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fef08000c00 tx=0x7fef08009da0 comp rx=0 tx=0).stop 2024-09-16T00:45:40.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.078+0000 7fef1ccd8640 1 -- 172.21.15.112:0/562889948 shutdown_connections 2024-09-16T00:45:40.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.078+0000 7fef1ccd8640 1 --2- 172.21.15.112:0/562889948 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fef0006e530 0x7fef000709e0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:40.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.078+0000 7fef1ccd8640 1 --2- 172.21.15.112:0/562889948 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef18102ed0 0x7fef1806ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:40.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.078+0000 7fef1ccd8640 1 --2- 172.21.15.112:0/562889948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef18102530 0x7fef1806fa10 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:40.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.078+0000 7fef1ccd8640 1 -- 172.21.15.112:0/562889948 >> 172.21.15.112:0/562889948 conn(0x7fef180f80f0 msgr2=0x7fef180f9100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:40.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.078+0000 7fef1ccd8640 1 -- 172.21.15.112:0/562889948 shutdown_connections 2024-09-16T00:45:40.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.078+0000 7fef1ccd8640 1 -- 172.21.15.112:0/562889948 wait complete. 2024-09-16T00:45:40.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.156+0000 7fd9d0cae640 1 -- 172.21.15.112:0/628861079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9cc102ef0 msgr2=0x7fd9cc0fc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:40.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.156+0000 7fd9d0cae640 1 --2- 172.21.15.112:0/628861079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9cc102ef0 0x7fd9cc0fc5b0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fd9b40048d0 tx=0x7fd9b4009da0 comp rx=0 tx=0).stop 2024-09-16T00:45:40.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.156+0000 7fd9d0cae640 1 -- 172.21.15.112:0/628861079 shutdown_connections 2024-09-16T00:45:40.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.156+0000 7fd9d0cae640 1 --2- 172.21.15.112:0/628861079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9cc102ef0 0x7fd9cc0fc5b0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:40.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.156+0000 7fd9d0cae640 1 --2- 172.21.15.112:0/628861079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9cc102550 0x7fd9cc102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:40.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.156+0000 7fd9d0cae640 1 -- 172.21.15.112:0/628861079 >> 172.21.15.112:0/628861079 conn(0x7fd9cc0f80f0 msgr2=0x7fd9cc0fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:40.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.156+0000 7fd9d0cae640 1 -- 172.21.15.112:0/628861079 shutdown_connections 2024-09-16T00:45:40.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.156+0000 7fd9d0cae640 1 -- 172.21.15.112:0/628861079 wait complete. 2024-09-16T00:45:40.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.156+0000 7fd9d0cae640 1 Processor -- start 2024-09-16T00:45:40.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.157+0000 7fd9d0cae640 1 -- start start 2024-09-16T00:45:40.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.157+0000 7fd9d0cae640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9cc102550 0x7fd9cc113500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:40.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.157+0000 7fd9d0cae640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9cc102ef0 0x7fd9cc113a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:40.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.157+0000 7fd9d0cae640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9cc114090 con 0x7fd9cc102550 2024-09-16T00:45:40.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.157+0000 7fd9d0cae640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9cc10d4a0 con 0x7fd9cc102ef0 2024-09-16T00:45:40.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.157+0000 7fd9ca59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9cc102550 0x7fd9cc113500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:40.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.157+0000 7fd9ca59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9cc102550 0x7fd9cc113500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38864/0 (socket says 172.21.15.112:38864) 2024-09-16T00:45:40.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.157+0000 7fd9ca59c640 1 -- 172.21.15.112:0/1508608551 learned_addr learned my addr 172.21.15.112:0/1508608551 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:45:40.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.157+0000 7fd9c9d9b640 1 --2- 172.21.15.112:0/1508608551 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9cc102ef0 0x7fd9cc113a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:40.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.157+0000 7fd9ca59c640 1 -- 172.21.15.112:0/1508608551 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9cc102ef0 msgr2=0x7fd9cc113a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:40.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.157+0000 7fd9ca59c640 1 --2- 172.21.15.112:0/1508608551 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9cc102ef0 0x7fd9cc113a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:40.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.157+0000 7fd9ca59c640 1 -- 172.21.15.112:0/1508608551 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b4005040 con 0x7fd9cc102550 2024-09-16T00:45:40.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.157+0000 7fd9ca59c640 1 --2- 172.21.15.112:0/1508608551 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9cc102550 0x7fd9cc113500 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fd9c400cb10 tx=0x7fd9c4007f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:40.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.158+0000 7fd9c2ffd640 1 -- 172.21.15.112:0/1508608551 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd9c4004200 con 0x7fd9cc102550 2024-09-16T00:45:40.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.158+0000 7fd9d0cae640 1 -- 172.21.15.112:0/1508608551 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9cc10d780 con 0x7fd9cc102550 2024-09-16T00:45:40.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.158+0000 7fd9c2ffd640 1 -- 172.21.15.112:0/1508608551 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fd9c400ac40 con 0x7fd9cc102550 2024-09-16T00:45:40.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.158+0000 7fd9c2ffd640 1 -- 172.21.15.112:0/1508608551 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd9c4003d30 con 0x7fd9cc102550 2024-09-16T00:45:40.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.158+0000 7fd9d0cae640 1 -- 172.21.15.112:0/1508608551 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9cc10dcf0 con 0x7fd9cc102550 2024-09-16T00:45:40.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.159+0000 7fd9c2ffd640 1 -- 172.21.15.112:0/1508608551 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fd9c400a7a0 con 0x7fd9cc102550 2024-09-16T00:45:40.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.159+0000 7fd9d0cae640 1 -- 172.21.15.112:0/1508608551 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9cc0fdcd0 con 0x7fd9cc102550 2024-09-16T00:45:40.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.160+0000 7fd9c2ffd640 1 --2- 172.21.15.112:0/1508608551 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd9b806e530 0x7fd9b80709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:40.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.160+0000 7fd9c2ffd640 1 -- 172.21.15.112:0/1508608551 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7fd9c408e8a0 con 0x7fd9cc102550 2024-09-16T00:45:40.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.160+0000 7fd9c9d9b640 1 --2- 172.21.15.112:0/1508608551 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd9b806e530 0x7fd9b80709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:40.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.160+0000 7fd9c9d9b640 1 --2- 172.21.15.112:0/1508608551 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd9b806e530 0x7fd9b80709e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd9b40048a0 tx=0x7fd9b4035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:40.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.165+0000 7fd9c2ffd640 1 -- 172.21.15.112:0/1508608551 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fd9c4057810 con 0x7fd9cc102550 2024-09-16T00:45:40.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:45:39 smithi169 ceph-mon[77626]: pgmap v537: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:45:40.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.395+0000 7fd9d0cae640 1 -- 172.21.15.112:0/1508608551 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd9cc04b720 con 0x7fd9cc102550 2024-09-16T00:45:40.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.395+0000 7fd9c2ffd640 1 -- 172.21.15.112:0/1508608551 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7fd9c405aed0 con 0x7fd9cc102550 2024-09-16T00:45:40.395 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:45:40.395 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": { 2024-09-16T00:45:40.396 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:45:40.396 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:45:40.396 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": { 2024-09-16T00:45:40.396 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:45:40.396 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:45:40.396 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": { 2024-09-16T00:45:40.396 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 6 2024-09-16T00:45:40.396 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:45:40.396 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": { 2024-09-16T00:45:40.396 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 4 2024-09-16T00:45:40.396 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:45:40.396 INFO:teuthology.orchestra.run.smithi112.stdout: "overall": { 2024-09-16T00:45:40.396 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 14 2024-09-16T00:45:40.397 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-16T00:45:40.397 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:45:40.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.397+0000 7fd9d0cae640 1 -- 172.21.15.112:0/1508608551 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd9b806e530 msgr2=0x7fd9b80709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:40.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.397+0000 7fd9d0cae640 1 --2- 172.21.15.112:0/1508608551 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd9b806e530 0x7fd9b80709e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd9b40048a0 tx=0x7fd9b4035040 comp rx=0 tx=0).stop 2024-09-16T00:45:40.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.397+0000 7fd9d0cae640 1 -- 172.21.15.112:0/1508608551 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9cc102550 msgr2=0x7fd9cc113500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:40.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.397+0000 7fd9d0cae640 1 --2- 172.21.15.112:0/1508608551 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9cc102550 0x7fd9cc113500 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fd9c400cb10 tx=0x7fd9c4007f00 comp rx=0 tx=0).stop 2024-09-16T00:45:40.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.397+0000 7fd9d0cae640 1 -- 172.21.15.112:0/1508608551 shutdown_connections 2024-09-16T00:45:40.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.397+0000 7fd9d0cae640 1 --2- 172.21.15.112:0/1508608551 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd9b806e530 0x7fd9b80709e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:40.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.397+0000 7fd9d0cae640 1 --2- 172.21.15.112:0/1508608551 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9cc102ef0 0x7fd9cc113a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:40.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.397+0000 7fd9d0cae640 1 --2- 172.21.15.112:0/1508608551 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9cc102550 0x7fd9cc113500 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:40.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.397+0000 7fd9d0cae640 1 -- 172.21.15.112:0/1508608551 >> 172.21.15.112:0/1508608551 conn(0x7fd9cc0f80f0 msgr2=0x7fd9cc0f9150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:40.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.397+0000 7fd9d0cae640 1 -- 172.21.15.112:0/1508608551 shutdown_connections 2024-09-16T00:45:40.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.397+0000 7fd9d0cae640 1 -- 172.21.15.112:0/1508608551 wait complete. 2024-09-16T00:45:40.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.475+0000 7f5915114640 1 -- 172.21.15.112:0/1286426950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59100716e0 msgr2=0x7f591006fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:40.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.475+0000 7f5915114640 1 --2- 172.21.15.112:0/1286426950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59100716e0 0x7f591006fab0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f59000048d0 tx=0x7f5900009da0 comp rx=0 tx=0).stop 2024-09-16T00:45:40.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.475+0000 7f5915114640 1 -- 172.21.15.112:0/1286426950 shutdown_connections 2024-09-16T00:45:40.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.475+0000 7f5915114640 1 --2- 172.21.15.112:0/1286426950 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f591006fff0 0x7f5910070460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:40.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.475+0000 7f5915114640 1 --2- 172.21.15.112:0/1286426950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59100716e0 0x7f591006fab0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:40.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.475+0000 7f5915114640 1 -- 172.21.15.112:0/1286426950 >> 172.21.15.112:0/1286426950 conn(0x7f59100f81a0 msgr2=0x7f59100fa5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:40.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.475+0000 7f5915114640 1 -- 172.21.15.112:0/1286426950 shutdown_connections 2024-09-16T00:45:40.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.475+0000 7f5915114640 1 -- 172.21.15.112:0/1286426950 wait complete. 2024-09-16T00:45:40.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.476+0000 7f5915114640 1 Processor -- start 2024-09-16T00:45:40.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.476+0000 7f5915114640 1 -- start start 2024-09-16T00:45:40.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.476+0000 7f5915114640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f591006fff0 0x7f59101198a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:40.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.476+0000 7f5915114640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59100716e0 0x7f5910119de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:40.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.476+0000 7f5915114640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f591011a430 con 0x7f59100716e0 2024-09-16T00:45:40.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.476+0000 7f5915114640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5910113840 con 0x7f591006fff0 2024-09-16T00:45:40.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.476+0000 7f590e59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59100716e0 0x7f5910119de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:40.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.476+0000 7f590e59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59100716e0 0x7f5910119de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38884/0 (socket says 172.21.15.112:38884) 2024-09-16T00:45:40.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.476+0000 7f590e59c640 1 -- 172.21.15.112:0/503962879 learned_addr learned my addr 172.21.15.112:0/503962879 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:45:40.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.476+0000 7f590ed9d640 1 --2- 172.21.15.112:0/503962879 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f591006fff0 0x7f59101198a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:40.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.477+0000 7f590e59c640 1 -- 172.21.15.112:0/503962879 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f591006fff0 msgr2=0x7f59101198a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:40.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.477+0000 7f590e59c640 1 --2- 172.21.15.112:0/503962879 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f591006fff0 0x7f59101198a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:40.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.477+0000 7f590e59c640 1 -- 172.21.15.112:0/503962879 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5900005040 con 0x7f59100716e0 2024-09-16T00:45:40.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.477+0000 7f590ed9d640 1 --2- 172.21.15.112:0/503962879 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f591006fff0 0x7f59101198a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:45:40.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.477+0000 7f590e59c640 1 --2- 172.21.15.112:0/503962879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59100716e0 0x7f5910119de0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f590400eb70 tx=0x7f590400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:40.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.477+0000 7f58f77fe640 1 -- 172.21.15.112:0/503962879 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5904007c30 con 0x7f59100716e0 2024-09-16T00:45:40.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.477+0000 7f5915114640 1 -- 172.21.15.112:0/503962879 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5910113ac0 con 0x7f59100716e0 2024-09-16T00:45:40.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.477+0000 7f58f77fe640 1 -- 172.21.15.112:0/503962879 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f590400f040 con 0x7f59100716e0 2024-09-16T00:45:40.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.477+0000 7f58f77fe640 1 -- 172.21.15.112:0/503962879 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5904009420 con 0x7f59100716e0 2024-09-16T00:45:40.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.477+0000 7f5915114640 1 -- 172.21.15.112:0/503962879 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5910113fe0 con 0x7f59100716e0 2024-09-16T00:45:40.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.478+0000 7f5915114640 1 -- 172.21.15.112:0/503962879 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59100711f0 con 0x7f59100716e0 2024-09-16T00:45:40.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.479+0000 7f58f77fe640 1 -- 172.21.15.112:0/503962879 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f5904009690 con 0x7f59100716e0 2024-09-16T00:45:40.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.480+0000 7f58f77fe640 1 --2- 172.21.15.112:0/503962879 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f58f806e530 0x7f58f80709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:40.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.480+0000 7f58f77fe640 1 -- 172.21.15.112:0/503962879 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f590408dfa0 con 0x7f59100716e0 2024-09-16T00:45:40.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.480+0000 7f590ed9d640 1 --2- 172.21.15.112:0/503962879 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f58f806e530 0x7f58f80709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:40.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.480+0000 7f590ed9d640 1 --2- 172.21.15.112:0/503962879 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f58f806e530 0x7f58f80709e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f59000048a0 tx=0x7f5900035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:40.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.484+0000 7f58f77fe640 1 -- 172.21.15.112:0/503962879 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f5904056fc0 con 0x7f59100716e0 2024-09-16T00:45:40.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.684+0000 7f5915114640 1 -- 172.21.15.112:0/503962879 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f59100fbda0 con 0x7f59100716e0 2024-09-16T00:45:40.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.685+0000 7f58f77fe640 1 -- 172.21.15.112:0/503962879 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1914 (secure 0 0 0) 0x7f590405a680 con 0x7f59100716e0 2024-09-16T00:45:40.685 INFO:teuthology.orchestra.run.smithi112.stdout:e34 2024-09-16T00:45:40.685 INFO:teuthology.orchestra.run.smithi112.stdout:btime 2024-09-16T00:34:26:619254+0000 2024-09-16T00:45:40.685 INFO:teuthology.orchestra.run.smithi112.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T00:45:40.685 INFO:teuthology.orchestra.run.smithi112.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:45:40.686 INFO:teuthology.orchestra.run.smithi112.stdout:legacy client fscid: 1 2024-09-16T00:45:40.686 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:45:40.686 INFO:teuthology.orchestra.run.smithi112.stdout:Filesystem 'cephfs' (1) 2024-09-16T00:45:40.686 INFO:teuthology.orchestra.run.smithi112.stdout:fs_name cephfs 2024-09-16T00:45:40.686 INFO:teuthology.orchestra.run.smithi112.stdout:epoch 25 2024-09-16T00:45:40.686 INFO:teuthology.orchestra.run.smithi112.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-16T00:45:40.686 INFO:teuthology.orchestra.run.smithi112.stdout:created 2024-09-16T00:23:22.328199+0000 2024-09-16T00:45:40.686 INFO:teuthology.orchestra.run.smithi112.stdout:modified 2024-09-16T00:33:29.012991+0000 2024-09-16T00:45:40.686 INFO:teuthology.orchestra.run.smithi112.stdout:tableserver 0 2024-09-16T00:45:40.686 INFO:teuthology.orchestra.run.smithi112.stdout:root 0 2024-09-16T00:45:40.686 INFO:teuthology.orchestra.run.smithi112.stdout:session_timeout 60 2024-09-16T00:45:40.686 INFO:teuthology.orchestra.run.smithi112.stdout:session_autoclose 300 2024-09-16T00:45:40.686 INFO:teuthology.orchestra.run.smithi112.stdout:max_file_size 1099511627776 2024-09-16T00:45:40.686 INFO:teuthology.orchestra.run.smithi112.stdout:max_xattr_size 65536 2024-09-16T00:45:40.687 INFO:teuthology.orchestra.run.smithi112.stdout:required_client_features {} 2024-09-16T00:45:40.687 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure 0 2024-09-16T00:45:40.687 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure_osd_epoch 82 2024-09-16T00:45:40.687 INFO:teuthology.orchestra.run.smithi112.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:45:40.687 INFO:teuthology.orchestra.run.smithi112.stdout:max_mds 2 2024-09-16T00:45:40.687 INFO:teuthology.orchestra.run.smithi112.stdout:in 0,1 2024-09-16T00:45:40.687 INFO:teuthology.orchestra.run.smithi112.stdout:up {} 2024-09-16T00:45:40.687 INFO:teuthology.orchestra.run.smithi112.stdout:failed 0,1 2024-09-16T00:45:40.687 INFO:teuthology.orchestra.run.smithi112.stdout:damaged 2024-09-16T00:45:40.687 INFO:teuthology.orchestra.run.smithi112.stdout:stopped 2024-09-16T00:45:40.687 INFO:teuthology.orchestra.run.smithi112.stdout:data_pools [3] 2024-09-16T00:45:40.687 INFO:teuthology.orchestra.run.smithi112.stdout:metadata_pool 2 2024-09-16T00:45:40.687 INFO:teuthology.orchestra.run.smithi112.stdout:inline_data enabled 2024-09-16T00:45:40.687 INFO:teuthology.orchestra.run.smithi112.stdout:balancer 2024-09-16T00:45:40.688 INFO:teuthology.orchestra.run.smithi112.stdout:bal_rank_mask -1 2024-09-16T00:45:40.688 INFO:teuthology.orchestra.run.smithi112.stdout:standby_count_wanted 1 2024-09-16T00:45:40.688 INFO:teuthology.orchestra.run.smithi112.stdout:qdb_cluster leader: 0 members: 2024-09-16T00:45:40.688 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:45:40.688 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:45:40.688 INFO:teuthology.orchestra.run.smithi112.stdout:Standby daemons: 2024-09-16T00:45:40.688 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:45:40.688 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.ithwzj{-1:34392} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.112:6826/476229221,v1:172.21.15.112:6827/476229221] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:45:40.688 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.hxfwkh{-1:34404} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.112:6828/2356899876,v1:172.21.15.112:6829/2356899876] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:45:40.688 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.gawwgw{-1:44343} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.169:6824/1635329957,v1:172.21.15.169:6825/1635329957] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:45:40.688 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.lgeqrj{-1:44347} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.169:6826/184810031,v1:172.21.15.169:6827/184810031] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:45:40.689 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 34 2024-09-16T00:45:40.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.687+0000 7f5915114640 1 -- 172.21.15.112:0/503962879 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f58f806e530 msgr2=0x7f58f80709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:40.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.687+0000 7f5915114640 1 --2- 172.21.15.112:0/503962879 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f58f806e530 0x7f58f80709e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f59000048a0 tx=0x7f5900035040 comp rx=0 tx=0).stop 2024-09-16T00:45:40.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.687+0000 7f5915114640 1 -- 172.21.15.112:0/503962879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59100716e0 msgr2=0x7f5910119de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:40.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.687+0000 7f5915114640 1 --2- 172.21.15.112:0/503962879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59100716e0 0x7f5910119de0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f590400eb70 tx=0x7f590400b040 comp rx=0 tx=0).stop 2024-09-16T00:45:40.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.687+0000 7f5915114640 1 -- 172.21.15.112:0/503962879 shutdown_connections 2024-09-16T00:45:40.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.687+0000 7f5915114640 1 --2- 172.21.15.112:0/503962879 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f58f806e530 0x7f58f80709e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:40.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.687+0000 7f5915114640 1 --2- 172.21.15.112:0/503962879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59100716e0 0x7f5910119de0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:40.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.687+0000 7f5915114640 1 --2- 172.21.15.112:0/503962879 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f591006fff0 0x7f59101198a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:40.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.687+0000 7f5915114640 1 -- 172.21.15.112:0/503962879 >> 172.21.15.112:0/503962879 conn(0x7f59100f81a0 msgr2=0x7f59100f9420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:40.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.687+0000 7f5915114640 1 -- 172.21.15.112:0/503962879 shutdown_connections 2024-09-16T00:45:40.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.687+0000 7f5915114640 1 -- 172.21.15.112:0/503962879 wait complete. 2024-09-16T00:45:40.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.766+0000 7fa1bf4ca640 1 -- 172.21.15.112:0/2253003384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1b8102620 msgr2=0x7fa1b81029f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:40.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.766+0000 7fa1bf4ca640 1 --2- 172.21.15.112:0/2253003384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1b8102620 0x7fa1b81029f0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fa1b40048d0 tx=0x7fa1b4009da0 comp rx=0 tx=0).stop 2024-09-16T00:45:40.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.766+0000 7fa1bf4ca640 1 -- 172.21.15.112:0/2253003384 shutdown_connections 2024-09-16T00:45:40.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.766+0000 7fa1bf4ca640 1 --2- 172.21.15.112:0/2253003384 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa1b80fac80 0x7fa1b80fb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:40.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.766+0000 7fa1bf4ca640 1 --2- 172.21.15.112:0/2253003384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1b8102620 0x7fa1b81029f0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:40.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.766+0000 7fa1bf4ca640 1 -- 172.21.15.112:0/2253003384 >> 172.21.15.112:0/2253003384 conn(0x7fa1b8070ce0 msgr2=0x7fa1b80710e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:40.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.766+0000 7fa1bf4ca640 1 -- 172.21.15.112:0/2253003384 shutdown_connections 2024-09-16T00:45:40.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.766+0000 7fa1bf4ca640 1 -- 172.21.15.112:0/2253003384 wait complete. 2024-09-16T00:45:40.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.767+0000 7fa1bf4ca640 1 Processor -- start 2024-09-16T00:45:40.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.767+0000 7fa1bf4ca640 1 -- start start 2024-09-16T00:45:40.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.767+0000 7fa1bf4ca640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1b80fac80 0x7fa1b81131f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:40.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.767+0000 7fa1bf4ca640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa1b8102620 0x7fa1b8113730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:40.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.767+0000 7fa1bf4ca640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b8113d30 con 0x7fa1b80fac80 2024-09-16T00:45:40.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.767+0000 7fa1bf4ca640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b810d170 con 0x7fa1b8102620 2024-09-16T00:45:40.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.767+0000 7fa1bd266640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1b80fac80 0x7fa1b81131f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:40.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.767+0000 7fa1bd266640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1b80fac80 0x7fa1b81131f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38896/0 (socket says 172.21.15.112:38896) 2024-09-16T00:45:40.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.767+0000 7fa1bd266640 1 -- 172.21.15.112:0/3402865909 learned_addr learned my addr 172.21.15.112:0/3402865909 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:45:40.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.767+0000 7fa1bca65640 1 --2- 172.21.15.112:0/3402865909 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa1b8102620 0x7fa1b8113730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:40.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.768+0000 7fa1bd266640 1 -- 172.21.15.112:0/3402865909 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa1b8102620 msgr2=0x7fa1b8113730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:40.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.768+0000 7fa1bd266640 1 --2- 172.21.15.112:0/3402865909 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa1b8102620 0x7fa1b8113730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:40.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.768+0000 7fa1bd266640 1 -- 172.21.15.112:0/3402865909 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1b4005040 con 0x7fa1b80fac80 2024-09-16T00:45:40.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.768+0000 7fa1bca65640 1 --2- 172.21.15.112:0/3402865909 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa1b8102620 0x7fa1b8113730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:45:40.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.768+0000 7fa1bd266640 1 --2- 172.21.15.112:0/3402865909 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1b80fac80 0x7fa1b81131f0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fa1b4003080 tx=0x7fa1b4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:40.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.768+0000 7fa1adffb640 1 -- 172.21.15.112:0/3402865909 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa1b4038070 con 0x7fa1b80fac80 2024-09-16T00:45:40.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.768+0000 7fa1adffb640 1 -- 172.21.15.112:0/3402865909 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fa1b402ade0 con 0x7fa1b80fac80 2024-09-16T00:45:40.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.768+0000 7fa1bf4ca640 1 -- 172.21.15.112:0/3402865909 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1b810d3f0 con 0x7fa1b80fac80 2024-09-16T00:45:40.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.768+0000 7fa1adffb640 1 -- 172.21.15.112:0/3402865909 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa1b402c880 con 0x7fa1b80fac80 2024-09-16T00:45:40.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.768+0000 7fa1bf4ca640 1 -- 172.21.15.112:0/3402865909 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1b810d8d0 con 0x7fa1b80fac80 2024-09-16T00:45:40.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.769+0000 7fa1bf4ca640 1 -- 172.21.15.112:0/3402865909 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1b80657b0 con 0x7fa1b80fac80 2024-09-16T00:45:40.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.770+0000 7fa1adffb640 1 -- 172.21.15.112:0/3402865909 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fa1b4030020 con 0x7fa1b80fac80 2024-09-16T00:45:40.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.770+0000 7fa1adffb640 1 --2- 172.21.15.112:0/3402865909 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa1a406e530 0x7fa1a40709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:40.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.771+0000 7fa1adffb640 1 -- 172.21.15.112:0/3402865909 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7fa1b40ae2e0 con 0x7fa1b80fac80 2024-09-16T00:45:40.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.771+0000 7fa1bca65640 1 --2- 172.21.15.112:0/3402865909 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa1a406e530 0x7fa1a40709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:40.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.771+0000 7fa1bca65640 1 --2- 172.21.15.112:0/3402865909 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa1a406e530 0x7fa1a40709e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa1b810eab0 tx=0x7fa1a8009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:40.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.775+0000 7fa1adffb640 1 -- 172.21.15.112:0/3402865909 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fa1b40772d0 con 0x7fa1b80fac80 2024-09-16T00:45:40.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.958+0000 7fa1bf4ca640 1 -- 172.21.15.112:0/3402865909 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa1b804b7b0 con 0x7fa1a406e530 2024-09-16T00:45:40.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.959+0000 7fa1adffb640 1 -- 172.21.15.112:0/3402865909 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fa1b804b7b0 con 0x7fa1a406e530 2024-09-16T00:45:40.959 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:45:40.959 INFO:teuthology.orchestra.run.smithi112.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80", 2024-09-16T00:45:40.959 INFO:teuthology.orchestra.run.smithi112.stdout: "in_progress": true, 2024-09-16T00:45:40.959 INFO:teuthology.orchestra.run.smithi112.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T00:45:40.959 INFO:teuthology.orchestra.run.smithi112.stdout: "services_complete": [ 2024-09-16T00:45:40.959 INFO:teuthology.orchestra.run.smithi112.stdout: "mds", 2024-09-16T00:45:40.959 INFO:teuthology.orchestra.run.smithi112.stdout: "osd", 2024-09-16T00:45:40.960 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr", 2024-09-16T00:45:40.960 INFO:teuthology.orchestra.run.smithi112.stdout: "mon", 2024-09-16T00:45:40.960 INFO:teuthology.orchestra.run.smithi112.stdout: "crash" 2024-09-16T00:45:40.960 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-16T00:45:40.960 INFO:teuthology.orchestra.run.smithi112.stdout: "progress": "16/21 daemons upgraded", 2024-09-16T00:45:40.960 INFO:teuthology.orchestra.run.smithi112.stdout: "message": "Currently upgrading mds daemons", 2024-09-16T00:45:40.960 INFO:teuthology.orchestra.run.smithi112.stdout: "is_paused": false 2024-09-16T00:45:40.960 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:45:40.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.961+0000 7fa1bf4ca640 1 -- 172.21.15.112:0/3402865909 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa1a406e530 msgr2=0x7fa1a40709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:40.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.961+0000 7fa1bf4ca640 1 --2- 172.21.15.112:0/3402865909 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa1a406e530 0x7fa1a40709e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa1b810eab0 tx=0x7fa1a8009040 comp rx=0 tx=0).stop 2024-09-16T00:45:40.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.961+0000 7fa1bf4ca640 1 -- 172.21.15.112:0/3402865909 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1b80fac80 msgr2=0x7fa1b81131f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:40.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.961+0000 7fa1bf4ca640 1 --2- 172.21.15.112:0/3402865909 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1b80fac80 0x7fa1b81131f0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fa1b4003080 tx=0x7fa1b4009da0 comp rx=0 tx=0).stop 2024-09-16T00:45:40.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.961+0000 7fa1bf4ca640 1 -- 172.21.15.112:0/3402865909 shutdown_connections 2024-09-16T00:45:40.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.961+0000 7fa1bf4ca640 1 --2- 172.21.15.112:0/3402865909 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa1a406e530 0x7fa1a40709e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:40.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.961+0000 7fa1bf4ca640 1 --2- 172.21.15.112:0/3402865909 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa1b8102620 0x7fa1b8113730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:40.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.961+0000 7fa1bf4ca640 1 --2- 172.21.15.112:0/3402865909 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1b80fac80 0x7fa1b81131f0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:40.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.961+0000 7fa1bf4ca640 1 -- 172.21.15.112:0/3402865909 >> 172.21.15.112:0/3402865909 conn(0x7fa1b8070ce0 msgr2=0x7fa1b8105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:40.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.961+0000 7fa1bf4ca640 1 -- 172.21.15.112:0/3402865909 shutdown_connections 2024-09-16T00:45:40.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:40.961+0000 7fa1bf4ca640 1 -- 172.21.15.112:0/3402865909 wait complete. 2024-09-16T00:45:41.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:41.040+0000 7fe4d3bb3640 1 -- 172.21.15.112:0/3560859813 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4cc102530 msgr2=0x7fe4cc102900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:41.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:41.040+0000 7fe4d3bb3640 1 --2- 172.21.15.112:0/3560859813 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4cc102530 0x7fe4cc102900 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fe4c80048d0 tx=0x7fe4c8009da0 comp rx=0 tx=0).stop 2024-09-16T00:45:41.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:41.040+0000 7fe4d3bb3640 1 -- 172.21.15.112:0/3560859813 shutdown_connections 2024-09-16T00:45:41.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:41.040+0000 7fe4d3bb3640 1 --2- 172.21.15.112:0/3560859813 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4cc102ed0 0x7fe4cc0fc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:41.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:41.040+0000 7fe4d3bb3640 1 --2- 172.21.15.112:0/3560859813 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4cc102530 0x7fe4cc102900 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:41.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:41.040+0000 7fe4d3bb3640 1 -- 172.21.15.112:0/3560859813 >> 172.21.15.112:0/3560859813 conn(0x7fe4cc0f8110 msgr2=0x7fe4cc0fa520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:41.041 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:45:40 smithi112 ceph-mon[97750]: from='client.35024 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:45:41.041 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:45:40 smithi112 ceph-mon[97750]: from='client.35028 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:45:41.041 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:45:40 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/1508608551' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:45:41.041 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:45:40 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/503962879' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:45:41.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:41.040+0000 7fe4d3bb3640 1 -- 172.21.15.112:0/3560859813 shutdown_connections 2024-09-16T00:45:41.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:41.040+0000 7fe4d3bb3640 1 -- 172.21.15.112:0/3560859813 wait complete. 2024-09-16T00:45:41.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:41.041+0000 7fe4d3bb3640 1 Processor -- start 2024-09-16T00:45:41.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:41.041+0000 7fe4d3bb3640 1 -- start start 2024-09-16T00:45:41.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:41.041+0000 7fe4d3bb3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4cc102530 0x7fe4cc1131f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:41.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:41.041+0000 7fe4d3bb3640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4cc102ed0 0x7fe4cc113730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:41.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:41.041+0000 7fe4d3bb3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4cc113d30 con 0x7fe4cc102530 2024-09-16T00:45:41.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:41.041+0000 7fe4d3bb3640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4cc10d170 con 0x7fe4cc102ed0 2024-09-16T00:45:41.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:41.041+0000 7fe4d194f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4cc102530 0x7fe4cc1131f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:41.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:41.041+0000 7fe4d194f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4cc102530 0x7fe4cc1131f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38900/0 (socket says 172.21.15.112:38900) 2024-09-16T00:45:41.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:41.041+0000 7fe4d194f640 1 -- 172.21.15.112:0/2584551585 learned_addr learned my addr 172.21.15.112:0/2584551585 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:45:41.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:41.041+0000 7fe4d114e640 1 --2- 172.21.15.112:0/2584551585 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4cc102ed0 0x7fe4cc113730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:41.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:41.041+0000 7fe4d194f640 1 -- 172.21.15.112:0/2584551585 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4cc102ed0 msgr2=0x7fe4cc113730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:41.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:41.041+0000 7fe4d194f640 1 --2- 172.21.15.112:0/2584551585 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4cc102ed0 0x7fe4cc113730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:41.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:41.041+0000 7fe4d194f640 1 -- 172.21.15.112:0/2584551585 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4c8005040 con 0x7fe4cc102530 2024-09-16T00:45:41.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:41.042+0000 7fe4d194f640 1 --2- 172.21.15.112:0/2584551585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4cc102530 0x7fe4cc1131f0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8000c00 tx=0x7fe4c8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:41.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:41.042+0000 7fe4be7fc640 1 -- 172.21.15.112:0/2584551585 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4c8038070 con 0x7fe4cc102530 2024-09-16T00:45:41.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:41.042+0000 7fe4d3bb3640 1 -- 172.21.15.112:0/2584551585 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4cc10d3f0 con 0x7fe4cc102530 2024-09-16T00:45:41.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:41.042+0000 7fe4be7fc640 1 -- 172.21.15.112:0/2584551585 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fe4c802ade0 con 0x7fe4cc102530 2024-09-16T00:45:41.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:41.042+0000 7fe4be7fc640 1 -- 172.21.15.112:0/2584551585 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4c802c7e0 con 0x7fe4cc102530 2024-09-16T00:45:41.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:41.042+0000 7fe4d3bb3640 1 -- 172.21.15.112:0/2584551585 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4cc10d930 con 0x7fe4cc102530 2024-09-16T00:45:41.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:41.043+0000 7fe4d3bb3640 1 -- 172.21.15.112:0/2584551585 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4cc0fdcb0 con 0x7fe4cc102530 2024-09-16T00:45:41.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:41.043+0000 7fe4be7fc640 1 -- 172.21.15.112:0/2584551585 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fe4c8030020 con 0x7fe4cc102530 2024-09-16T00:45:41.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:41.044+0000 7fe4be7fc640 1 --2- 172.21.15.112:0/2584551585 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fe4b806e530 0x7fe4b80709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:45:41.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:41.044+0000 7fe4be7fc640 1 -- 172.21.15.112:0/2584551585 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7fe4c80adf70 con 0x7fe4cc102530 2024-09-16T00:45:41.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:41.044+0000 7fe4d114e640 1 --2- 172.21.15.112:0/2584551585 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fe4b806e530 0x7fe4b80709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:45:41.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:41.044+0000 7fe4d114e640 1 --2- 172.21.15.112:0/2584551585 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fe4b806e530 0x7fe4b80709e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc10eab0 tx=0x7fe4c000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:45:41.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:41.048+0000 7fe4be7fc640 1 -- 172.21.15.112:0/2584551585 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fe4c80e31b0 con 0x7fe4cc102530 2024-09-16T00:45:41.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:45:40 smithi169 ceph-mon[77626]: from='client.35024 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:45:41.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:45:40 smithi169 ceph-mon[77626]: from='client.35028 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:45:41.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:45:40 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/1508608551' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:45:41.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:45:40 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/503962879' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:45:41.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:41.281+0000 7fe4d3bb3640 1 -- 172.21.15.112:0/2584551585 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fe4cc04b720 con 0x7fe4cc102530 2024-09-16T00:45:41.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:41.281+0000 7fe4be7fc640 1 -- 172.21.15.112:0/2584551585 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7fe4c8076f90 con 0x7fe4cc102530 2024-09-16T00:45:41.281 INFO:teuthology.orchestra.run.smithi112.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-16T00:45:41.281 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-16T00:45:41.281 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs is degraded 2024-09-16T00:45:41.281 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T00:45:41.281 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T00:45:41.281 INFO:teuthology.orchestra.run.smithi112.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-16T00:45:41.281 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-16T00:45:41.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:41.283+0000 7fe4d3bb3640 1 -- 172.21.15.112:0/2584551585 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fe4b806e530 msgr2=0x7fe4b80709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:41.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:41.283+0000 7fe4d3bb3640 1 --2- 172.21.15.112:0/2584551585 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fe4b806e530 0x7fe4b80709e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc10eab0 tx=0x7fe4c000b040 comp rx=0 tx=0).stop 2024-09-16T00:45:41.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:41.283+0000 7fe4d3bb3640 1 -- 172.21.15.112:0/2584551585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4cc102530 msgr2=0x7fe4cc1131f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:45:41.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:41.283+0000 7fe4d3bb3640 1 --2- 172.21.15.112:0/2584551585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4cc102530 0x7fe4cc1131f0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8000c00 tx=0x7fe4c8009da0 comp rx=0 tx=0).stop 2024-09-16T00:45:41.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:41.283+0000 7fe4d3bb3640 1 -- 172.21.15.112:0/2584551585 shutdown_connections 2024-09-16T00:45:41.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:41.283+0000 7fe4d3bb3640 1 --2- 172.21.15.112:0/2584551585 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fe4b806e530 0x7fe4b80709e0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:41.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:41.283+0000 7fe4d3bb3640 1 --2- 172.21.15.112:0/2584551585 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4cc102ed0 0x7fe4cc113730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:41.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:41.283+0000 7fe4d3bb3640 1 --2- 172.21.15.112:0/2584551585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4cc102530 0x7fe4cc1131f0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:45:41.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:41.283+0000 7fe4d3bb3640 1 -- 172.21.15.112:0/2584551585 >> 172.21.15.112:0/2584551585 conn(0x7fe4cc0f8110 msgr2=0x7fe4cc0f9190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:45:41.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:41.283+0000 7fe4d3bb3640 1 -- 172.21.15.112:0/2584551585 shutdown_connections 2024-09-16T00:45:41.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:45:41.283+0000 7fe4d3bb3640 1 -- 172.21.15.112:0/2584551585 wait complete. 2024-09-16T00:45:42.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:45:41 smithi112 ceph-mon[97750]: from='client.35032 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:45:42.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:45:41 smithi112 ceph-mon[97750]: pgmap v538: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:45:42.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:45:41 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/2584551585' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:45:42.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:45:41 smithi169 ceph-mon[77626]: from='client.35032 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:45:42.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:45:41 smithi169 ceph-mon[77626]: pgmap v538: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:45:42.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:45:41 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/2584551585' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:45:43.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:45:42 smithi112 ceph-mon[97750]: from='client.35044 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:45:43.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:45:42 smithi169 ceph-mon[77626]: from='client.35044 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:45:44.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:45:43 smithi112 ceph-mon[97750]: pgmap v539: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:45:44.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:45:43 smithi169 ceph-mon[77626]: pgmap v539: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:45:46.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:45:45 smithi112 ceph-mon[97750]: pgmap v540: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:45:46.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:45:45 smithi169 ceph-mon[77626]: pgmap v540: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:45:47.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:45:46 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:45:47.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:45:46 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:45:48.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:45:47 smithi169 ceph-mon[77626]: pgmap v541: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:45:48.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:45:47 smithi112 ceph-mon[97750]: pgmap v541: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:45:50.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:45:49 smithi169 ceph-mon[77626]: pgmap v542: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:45:50.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:45:49 smithi112 ceph-mon[97750]: pgmap v542: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:45:52.141 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:45:51 smithi112 ceph-mon[97750]: pgmap v543: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:45:52.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:45:51 smithi169 ceph-mon[77626]: pgmap v543: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:45:54.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:45:53 smithi112 ceph-mon[97750]: pgmap v544: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:45:54.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:45:53 smithi169 ceph-mon[77626]: pgmap v544: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:45:56.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:45:55 smithi169 ceph-mon[77626]: pgmap v545: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:45:56.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:45:55 smithi112 ceph-mon[97750]: pgmap v545: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:45:58.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:45:57 smithi169 ceph-mon[77626]: pgmap v546: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:45:58.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:45:57 smithi112 ceph-mon[97750]: pgmap v546: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:46:00.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:45:59 smithi169 ceph-mon[77626]: pgmap v547: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:46:00.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:45:59 smithi112 ceph-mon[97750]: pgmap v547: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:46:02.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:46:01 smithi169 ceph-mon[77626]: pgmap v548: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:46:02.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:46:01 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:46:02.314 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:46:01 smithi112 ceph-mon[97750]: pgmap v548: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:46:02.314 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:46:01 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:46:04.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:46:03 smithi112 ceph-mon[97750]: pgmap v549: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:46:04.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:46:03 smithi169 ceph-mon[77626]: pgmap v549: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:46:06.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:46:05 smithi169 ceph-mon[77626]: pgmap v550: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:46:06.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:46:05 smithi112 ceph-mon[97750]: pgmap v550: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:46:08.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:46:07 smithi169 ceph-mon[77626]: pgmap v551: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:46:08.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:46:07 smithi112 ceph-mon[97750]: pgmap v551: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:46:10.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:46:09 smithi169 ceph-mon[77626]: pgmap v552: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:46:10.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:46:09 smithi112 ceph-mon[97750]: pgmap v552: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:46:11.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.373+0000 7faab9a75640 1 -- 172.21.15.112:0/2320000846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faab4102ef0 msgr2=0x7faab40fc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:11.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.373+0000 7faab9a75640 1 --2- 172.21.15.112:0/2320000846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faab4102ef0 0x7faab40fc5b0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7faaa80048d0 tx=0x7faaa8009da0 comp rx=0 tx=0).stop 2024-09-16T00:46:11.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.373+0000 7faab9a75640 1 -- 172.21.15.112:0/2320000846 shutdown_connections 2024-09-16T00:46:11.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.373+0000 7faab9a75640 1 --2- 172.21.15.112:0/2320000846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faab4102ef0 0x7faab40fc5b0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:11.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.373+0000 7faab9a75640 1 --2- 172.21.15.112:0/2320000846 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faab4102550 0x7faab4102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:11.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.373+0000 7faab9a75640 1 -- 172.21.15.112:0/2320000846 >> 172.21.15.112:0/2320000846 conn(0x7faab40f80f0 msgr2=0x7faab40fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:11.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.373+0000 7faab9a75640 1 -- 172.21.15.112:0/2320000846 shutdown_connections 2024-09-16T00:46:11.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.373+0000 7faab9a75640 1 -- 172.21.15.112:0/2320000846 wait complete. 2024-09-16T00:46:11.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.374+0000 7faab9a75640 1 Processor -- start 2024-09-16T00:46:11.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.374+0000 7faab9a75640 1 -- start start 2024-09-16T00:46:11.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.374+0000 7faab9a75640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faab4102550 0x7faab4113500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:11.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.374+0000 7faab9a75640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faab4102ef0 0x7faab4113a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:11.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.374+0000 7faab9a75640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faab4114090 con 0x7faab4102ef0 2024-09-16T00:46:11.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.374+0000 7faab9a75640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faab410d4a0 con 0x7faab4102550 2024-09-16T00:46:11.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.375+0000 7faab2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faab4102ef0 0x7faab4113a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:11.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.375+0000 7faab2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faab4102ef0 0x7faab4113a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60478/0 (socket says 172.21.15.112:60478) 2024-09-16T00:46:11.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.375+0000 7faab2ffd640 1 -- 172.21.15.112:0/1410630119 learned_addr learned my addr 172.21.15.112:0/1410630119 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:46:11.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.375+0000 7faab37fe640 1 --2- 172.21.15.112:0/1410630119 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faab4102550 0x7faab4113500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:11.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.375+0000 7faab2ffd640 1 -- 172.21.15.112:0/1410630119 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faab4102550 msgr2=0x7faab4113500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:11.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.375+0000 7faab2ffd640 1 --2- 172.21.15.112:0/1410630119 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faab4102550 0x7faab4113500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:11.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.375+0000 7faab2ffd640 1 -- 172.21.15.112:0/1410630119 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaa8005040 con 0x7faab4102ef0 2024-09-16T00:46:11.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.375+0000 7faab37fe640 1 --2- 172.21.15.112:0/1410630119 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faab4102550 0x7faab4113500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:46:11.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.375+0000 7faab2ffd640 1 --2- 172.21.15.112:0/1410630119 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faab4102ef0 0x7faab4113a40 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7faaa80043d0 tx=0x7faaa8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:11.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.375+0000 7faa9bfff640 1 -- 172.21.15.112:0/1410630119 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faaa8038070 con 0x7faab4102ef0 2024-09-16T00:46:11.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.375+0000 7faab9a75640 1 -- 172.21.15.112:0/1410630119 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faab410d720 con 0x7faab4102ef0 2024-09-16T00:46:11.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.375+0000 7faa9bfff640 1 -- 172.21.15.112:0/1410630119 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7faaa80096e0 con 0x7faab4102ef0 2024-09-16T00:46:11.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.375+0000 7faa9bfff640 1 -- 172.21.15.112:0/1410630119 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faaa802c8a0 con 0x7faab4102ef0 2024-09-16T00:46:11.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.375+0000 7faab9a75640 1 -- 172.21.15.112:0/1410630119 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faab410dbe0 con 0x7faab4102ef0 2024-09-16T00:46:11.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.377+0000 7faab9a75640 1 -- 172.21.15.112:0/1410630119 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faab40fdcd0 con 0x7faab4102ef0 2024-09-16T00:46:11.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.377+0000 7faa9bfff640 1 -- 172.21.15.112:0/1410630119 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7faaa8030020 con 0x7faab4102ef0 2024-09-16T00:46:11.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.378+0000 7faa9bfff640 1 --2- 172.21.15.112:0/1410630119 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7faa9c06e530 0x7faa9c0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:11.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.378+0000 7faa9bfff640 1 -- 172.21.15.112:0/1410630119 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7faaa80ae010 con 0x7faab4102ef0 2024-09-16T00:46:11.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.378+0000 7faab37fe640 1 --2- 172.21.15.112:0/1410630119 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7faa9c06e530 0x7faa9c0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:11.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.378+0000 7faab37fe640 1 --2- 172.21.15.112:0/1410630119 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7faa9c06e530 0x7faa9c0709e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7faaa40093f0 tx=0x7faaa4009ed0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:11.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.382+0000 7faa9bfff640 1 -- 172.21.15.112:0/1410630119 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7faaa8077000 con 0x7faab4102ef0 2024-09-16T00:46:11.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.565+0000 7faab9a75640 1 -- 172.21.15.112:0/1410630119 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7faab4109560 con 0x7faa9c06e530 2024-09-16T00:46:11.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.566+0000 7faa9bfff640 1 -- 172.21.15.112:0/1410630119 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7faab4109560 con 0x7faa9c06e530 2024-09-16T00:46:11.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.568+0000 7faab9a75640 1 -- 172.21.15.112:0/1410630119 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7faa9c06e530 msgr2=0x7faa9c0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:11.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.568+0000 7faab9a75640 1 --2- 172.21.15.112:0/1410630119 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7faa9c06e530 0x7faa9c0709e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7faaa40093f0 tx=0x7faaa4009ed0 comp rx=0 tx=0).stop 2024-09-16T00:46:11.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.568+0000 7faab9a75640 1 -- 172.21.15.112:0/1410630119 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faab4102ef0 msgr2=0x7faab4113a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:11.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.568+0000 7faab9a75640 1 --2- 172.21.15.112:0/1410630119 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faab4102ef0 0x7faab4113a40 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7faaa80043d0 tx=0x7faaa8009da0 comp rx=0 tx=0).stop 2024-09-16T00:46:11.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.569+0000 7faab9a75640 1 -- 172.21.15.112:0/1410630119 shutdown_connections 2024-09-16T00:46:11.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.569+0000 7faab9a75640 1 --2- 172.21.15.112:0/1410630119 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7faa9c06e530 0x7faa9c0709e0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:11.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.569+0000 7faab9a75640 1 --2- 172.21.15.112:0/1410630119 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faab4102ef0 0x7faab4113a40 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:11.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.569+0000 7faab9a75640 1 --2- 172.21.15.112:0/1410630119 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faab4102550 0x7faab4113500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:11.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.569+0000 7faab9a75640 1 -- 172.21.15.112:0/1410630119 >> 172.21.15.112:0/1410630119 conn(0x7faab40f80f0 msgr2=0x7faab40f9150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:11.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.569+0000 7faab9a75640 1 -- 172.21.15.112:0/1410630119 shutdown_connections 2024-09-16T00:46:11.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.569+0000 7faab9a75640 1 -- 172.21.15.112:0/1410630119 wait complete. 2024-09-16T00:46:11.577 INFO:teuthology.orchestra.run.smithi112.stdout:true 2024-09-16T00:46:11.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.647+0000 7f6cef83c640 1 -- 172.21.15.112:0/360318535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ce80fac80 msgr2=0x7f6ce80fb0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:11.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.647+0000 7f6cef83c640 1 --2- 172.21.15.112:0/360318535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ce80fac80 0x7f6ce80fb0f0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f6cd80048d0 tx=0x7f6cd8009da0 comp rx=0 tx=0).stop 2024-09-16T00:46:11.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.647+0000 7f6cef83c640 1 -- 172.21.15.112:0/360318535 shutdown_connections 2024-09-16T00:46:11.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.647+0000 7f6cef83c640 1 --2- 172.21.15.112:0/360318535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ce80fac80 0x7f6ce80fb0f0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:11.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.647+0000 7f6cef83c640 1 --2- 172.21.15.112:0/360318535 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ce8102620 0x7f6ce81029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:11.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.647+0000 7f6cef83c640 1 -- 172.21.15.112:0/360318535 >> 172.21.15.112:0/360318535 conn(0x7f6ce8070ce0 msgr2=0x7f6ce80710e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:11.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.648+0000 7f6cef83c640 1 -- 172.21.15.112:0/360318535 shutdown_connections 2024-09-16T00:46:11.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.648+0000 7f6cef83c640 1 -- 172.21.15.112:0/360318535 wait complete. 2024-09-16T00:46:11.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.648+0000 7f6cef83c640 1 Processor -- start 2024-09-16T00:46:11.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.648+0000 7f6cef83c640 1 -- start start 2024-09-16T00:46:11.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.649+0000 7f6cef83c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ce80fac80 0x7f6ce81153d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:11.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.649+0000 7f6cef83c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ce8102620 0x7f6ce8115910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:11.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.649+0000 7f6cef83c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce8115f10 con 0x7f6ce80fac80 2024-09-16T00:46:11.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.649+0000 7f6cef83c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce8116080 con 0x7f6ce8102620 2024-09-16T00:46:11.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.649+0000 7f6ced5d8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ce80fac80 0x7f6ce81153d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:11.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.649+0000 7f6ced5d8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ce80fac80 0x7f6ce81153d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60500/0 (socket says 172.21.15.112:60500) 2024-09-16T00:46:11.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.649+0000 7f6ced5d8640 1 -- 172.21.15.112:0/609341142 learned_addr learned my addr 172.21.15.112:0/609341142 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:46:11.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.649+0000 7f6cecdd7640 1 --2- 172.21.15.112:0/609341142 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ce8102620 0x7f6ce8115910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:11.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.649+0000 7f6ced5d8640 1 -- 172.21.15.112:0/609341142 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ce8102620 msgr2=0x7f6ce8115910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:11.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.649+0000 7f6ced5d8640 1 --2- 172.21.15.112:0/609341142 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ce8102620 0x7f6ce8115910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:11.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.649+0000 7f6ced5d8640 1 -- 172.21.15.112:0/609341142 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cd8005040 con 0x7f6ce80fac80 2024-09-16T00:46:11.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.649+0000 7f6cecdd7640 1 --2- 172.21.15.112:0/609341142 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ce8102620 0x7f6ce8115910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:46:11.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.649+0000 7f6ced5d8640 1 --2- 172.21.15.112:0/609341142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ce80fac80 0x7f6ce81153d0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4002a50 tx=0x7f6ce4004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:11.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.650+0000 7f6cddffb640 1 -- 172.21.15.112:0/609341142 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ce4007c30 con 0x7f6ce80fac80 2024-09-16T00:46:11.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.650+0000 7f6cef83c640 1 -- 172.21.15.112:0/609341142 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ce810f550 con 0x7f6ce80fac80 2024-09-16T00:46:11.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.650+0000 7f6cddffb640 1 -- 172.21.15.112:0/609341142 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f6ce400f040 con 0x7f6ce80fac80 2024-09-16T00:46:11.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.650+0000 7f6cddffb640 1 -- 172.21.15.112:0/609341142 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ce400da90 con 0x7f6ce80fac80 2024-09-16T00:46:11.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.650+0000 7f6cef83c640 1 -- 172.21.15.112:0/609341142 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ce810faf0 con 0x7f6ce80fac80 2024-09-16T00:46:11.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.651+0000 7f6cef83c640 1 -- 172.21.15.112:0/609341142 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ce80657b0 con 0x7f6ce80fac80 2024-09-16T00:46:11.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.651+0000 7f6cddffb640 1 -- 172.21.15.112:0/609341142 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f6ce400dd00 con 0x7f6ce80fac80 2024-09-16T00:46:11.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.652+0000 7f6cddffb640 1 --2- 172.21.15.112:0/609341142 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f6cd406e530 0x7f6cd40709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:11.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.652+0000 7f6cddffb640 1 -- 172.21.15.112:0/609341142 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f6ce408dbf0 con 0x7f6ce80fac80 2024-09-16T00:46:11.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.652+0000 7f6cecdd7640 1 --2- 172.21.15.112:0/609341142 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f6cd406e530 0x7f6cd40709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:11.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.653+0000 7f6cecdd7640 1 --2- 172.21.15.112:0/609341142 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f6cd406e530 0x7f6cd40709e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6ce8110cd0 tx=0x7f6cd8035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:11.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.656+0000 7f6cddffb640 1 -- 172.21.15.112:0/609341142 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f6ce4056b60 con 0x7f6ce80fac80 2024-09-16T00:46:11.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.837+0000 7f6cef83c640 1 -- 172.21.15.112:0/609341142 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6ce8110a80 con 0x7f6cd406e530 2024-09-16T00:46:11.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.839+0000 7f6cddffb640 1 -- 172.21.15.112:0/609341142 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f6ce8110a80 con 0x7f6cd406e530 2024-09-16T00:46:11.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.841+0000 7f6cef83c640 1 -- 172.21.15.112:0/609341142 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f6cd406e530 msgr2=0x7f6cd40709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:11.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.841+0000 7f6cef83c640 1 --2- 172.21.15.112:0/609341142 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f6cd406e530 0x7f6cd40709e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6ce8110cd0 tx=0x7f6cd8035040 comp rx=0 tx=0).stop 2024-09-16T00:46:11.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.841+0000 7f6cef83c640 1 -- 172.21.15.112:0/609341142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ce80fac80 msgr2=0x7f6ce81153d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:11.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.841+0000 7f6cef83c640 1 --2- 172.21.15.112:0/609341142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ce80fac80 0x7f6ce81153d0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4002a50 tx=0x7f6ce4004600 comp rx=0 tx=0).stop 2024-09-16T00:46:11.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.841+0000 7f6cef83c640 1 -- 172.21.15.112:0/609341142 shutdown_connections 2024-09-16T00:46:11.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.841+0000 7f6cef83c640 1 --2- 172.21.15.112:0/609341142 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f6cd406e530 0x7f6cd40709e0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:11.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.841+0000 7f6cef83c640 1 --2- 172.21.15.112:0/609341142 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ce8102620 0x7f6ce8115910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:11.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.841+0000 7f6cef83c640 1 --2- 172.21.15.112:0/609341142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ce80fac80 0x7f6ce81153d0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:11.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.841+0000 7f6cef83c640 1 -- 172.21.15.112:0/609341142 >> 172.21.15.112:0/609341142 conn(0x7f6ce8070ce0 msgr2=0x7f6ce8103360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:11.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.841+0000 7f6cef83c640 1 -- 172.21.15.112:0/609341142 shutdown_connections 2024-09-16T00:46:11.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.841+0000 7f6cef83c640 1 -- 172.21.15.112:0/609341142 wait complete. 2024-09-16T00:46:11.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.919+0000 7f5bfc061640 1 -- 172.21.15.112:0/1131927353 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bf40716e0 msgr2=0x7f5bf406fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:11.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.919+0000 7f5bfc061640 1 --2- 172.21.15.112:0/1131927353 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bf40716e0 0x7f5bf406fab0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f5bf00048d0 tx=0x7f5bf0009da0 comp rx=0 tx=0).stop 2024-09-16T00:46:11.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.919+0000 7f5bfc061640 1 -- 172.21.15.112:0/1131927353 shutdown_connections 2024-09-16T00:46:11.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.919+0000 7f5bfc061640 1 --2- 172.21.15.112:0/1131927353 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5bf406fff0 0x7f5bf4070460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:11.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.919+0000 7f5bfc061640 1 --2- 172.21.15.112:0/1131927353 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bf40716e0 0x7f5bf406fab0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:11.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.919+0000 7f5bfc061640 1 -- 172.21.15.112:0/1131927353 >> 172.21.15.112:0/1131927353 conn(0x7f5bf40f8120 msgr2=0x7f5bf40fa530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:11.920 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:46:11 smithi112 ceph-mon[97750]: pgmap v553: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:46:11.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.920+0000 7f5bfc061640 1 -- 172.21.15.112:0/1131927353 shutdown_connections 2024-09-16T00:46:11.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.920+0000 7f5bfc061640 1 -- 172.21.15.112:0/1131927353 wait complete. 2024-09-16T00:46:11.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.920+0000 7f5bfc061640 1 Processor -- start 2024-09-16T00:46:11.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.920+0000 7f5bfc061640 1 -- start start 2024-09-16T00:46:11.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.920+0000 7f5bfc061640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5bf406fff0 0x7f5bf4119880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:11.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.920+0000 7f5bfc061640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bf40716e0 0x7f5bf4119dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:11.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.920+0000 7f5bfc061640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf411a410 con 0x7f5bf40716e0 2024-09-16T00:46:11.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.920+0000 7f5bfc061640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf4113820 con 0x7f5bf406fff0 2024-09-16T00:46:11.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.921+0000 7f5bf95fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bf40716e0 0x7f5bf4119dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:11.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.921+0000 7f5bf95fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bf40716e0 0x7f5bf4119dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60508/0 (socket says 172.21.15.112:60508) 2024-09-16T00:46:11.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.921+0000 7f5bf95fc640 1 -- 172.21.15.112:0/3621648723 learned_addr learned my addr 172.21.15.112:0/3621648723 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:46:11.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.921+0000 7f5bf9dfd640 1 --2- 172.21.15.112:0/3621648723 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5bf406fff0 0x7f5bf4119880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:11.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.921+0000 7f5bf95fc640 1 -- 172.21.15.112:0/3621648723 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5bf406fff0 msgr2=0x7f5bf4119880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:11.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.921+0000 7f5bf95fc640 1 --2- 172.21.15.112:0/3621648723 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5bf406fff0 0x7f5bf4119880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:11.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.921+0000 7f5bf95fc640 1 -- 172.21.15.112:0/3621648723 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bf0005040 con 0x7f5bf40716e0 2024-09-16T00:46:11.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.921+0000 7f5bf95fc640 1 --2- 172.21.15.112:0/3621648723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bf40716e0 0x7f5bf4119dc0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f5be4002a50 tx=0x7f5be4004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:11.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.921+0000 7f5bea7fc640 1 -- 172.21.15.112:0/3621648723 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5be4007c30 con 0x7f5bf40716e0 2024-09-16T00:46:11.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.921+0000 7f5bea7fc640 1 -- 172.21.15.112:0/3621648723 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f5be400f040 con 0x7f5bf40716e0 2024-09-16T00:46:11.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.921+0000 7f5bea7fc640 1 -- 172.21.15.112:0/3621648723 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5be400da90 con 0x7f5bf40716e0 2024-09-16T00:46:11.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.921+0000 7f5bfc061640 1 -- 172.21.15.112:0/3621648723 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bf4113b00 con 0x7f5bf40716e0 2024-09-16T00:46:11.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.921+0000 7f5bfc061640 1 -- 172.21.15.112:0/3621648723 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bf4114020 con 0x7f5bf40716e0 2024-09-16T00:46:11.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.923+0000 7f5bfc061640 1 -- 172.21.15.112:0/3621648723 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bf40711f0 con 0x7f5bf40716e0 2024-09-16T00:46:11.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.923+0000 7f5bea7fc640 1 -- 172.21.15.112:0/3621648723 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f5be400dc30 con 0x7f5bf40716e0 2024-09-16T00:46:11.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.923+0000 7f5bea7fc640 1 --2- 172.21.15.112:0/3621648723 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5be006e530 0x7f5be00709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:11.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.924+0000 7f5bea7fc640 1 -- 172.21.15.112:0/3621648723 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f5be408db40 con 0x7f5bf40716e0 2024-09-16T00:46:11.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.924+0000 7f5bf9dfd640 1 --2- 172.21.15.112:0/3621648723 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5be006e530 0x7f5be00709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:11.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.924+0000 7f5bf9dfd640 1 --2- 172.21.15.112:0/3621648723 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5be006e530 0x7f5be00709e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5bf00048a0 tx=0x7f5bf0009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:11.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:11.928+0000 7f5bea7fc640 1 -- 172.21.15.112:0/3621648723 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f5be4056b30 con 0x7f5bf40716e0 2024-09-16T00:46:12.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.101+0000 7f5bfc061640 1 -- 172.21.15.112:0/3621648723 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5bf410fb50 con 0x7f5be006e530 2024-09-16T00:46:12.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.110+0000 7f5bea7fc640 1 -- 172.21.15.112:0/3621648723 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7f5bf410fb50 con 0x7f5be006e530 2024-09-16T00:46:12.110 INFO:teuthology.orchestra.run.smithi112.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T00:46:12.110 INFO:teuthology.orchestra.run.smithi112.stdout:alertmanager.smithi112 smithi112 *:9093,9094 running (19m) 12m ago 28m 24.7M - 0.25.0 c8568f914cd2 d85ea0562022 2024-09-16T00:46:12.110 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi112 smithi112 running (16m) 12m ago 28m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c e4fe58f9702a 2024-09-16T00:46:12.110 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi169 smithi169 running (16m) 11m ago 27m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c 3ba2c81d527f 2024-09-16T00:46:12.110 INFO:teuthology.orchestra.run.smithi112.stdout:grafana.smithi112 smithi112 *:3000 running (17m) 12m ago 27m 82.1M - 9.4.12 2bacad6d85d8 51f9d2013da2 2024-09-16T00:46:12.110 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.hxfwkh smithi112 running (12m) 12m ago 22m 14.2M - 19.1.1-326-g8921c8ef f1455bd0219c 1723069902d6 2024-09-16T00:46:12.110 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.ithwzj smithi112 running (12m) 12m ago 22m 12.2M - 19.1.1-326-g8921c8ef f1455bd0219c e7aad5b81db5 2024-09-16T00:46:12.110 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.gawwgw smithi169 running (12m) 11m ago 22m 15.7M - 19.1.1-326-g8921c8ef f1455bd0219c 469be459ee12 2024-09-16T00:46:12.110 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.lgeqrj smithi169 running (11m) 11m ago 22m 13.3M - 19.1.1-326-g8921c8ef f1455bd0219c 31f0fbd2395d 2024-09-16T00:46:12.110 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi112.xrkkbc smithi112 *:8443,9283,8765 running (18m) 12m ago 29m 504M - 19.1.1-326-g8921c8ef f1455bd0219c a981f7fd2810 2024-09-16T00:46:12.110 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi169.mtrobk smithi169 *:8443,9283,8765 running (18m) 11m ago 27m 442M - 19.1.1-326-g8921c8ef f1455bd0219c 4096e5369c2d 2024-09-16T00:46:12.111 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi112 smithi112 running (17m) 12m ago 29m 49.8M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 5aaec033c042 2024-09-16T00:46:12.111 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi169 smithi169 running (16m) 11m ago 27m 38.9M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 7bc609155f3f 2024-09-16T00:46:12.111 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi112 smithi112 *:9100 running (19m) 12m ago 28m 15.5M - 1.5.0 0da6a335fe13 6404d1d601e2 2024-09-16T00:46:12.111 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi169 smithi169 *:9100 running (18m) 11m ago 27m 15.4M - 1.5.0 0da6a335fe13 5505baf702a6 2024-09-16T00:46:12.111 INFO:teuthology.orchestra.run.smithi112.stdout:osd.0 smithi112 running (15m) 12m ago 26m 105M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 53efc9397017 2024-09-16T00:46:12.111 INFO:teuthology.orchestra.run.smithi112.stdout:osd.1 smithi112 running (15m) 12m ago 25m 98.9M 1557M 19.1.1-326-g8921c8ef f1455bd0219c fe6f1a4a2cf4 2024-09-16T00:46:12.111 INFO:teuthology.orchestra.run.smithi112.stdout:osd.2 smithi112 running (14m) 12m ago 25m 102M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 5f7863712057 2024-09-16T00:46:12.111 INFO:teuthology.orchestra.run.smithi112.stdout:osd.3 smithi169 running (13m) 11m ago 25m 92.1M 2581M 19.1.1-326-g8921c8ef f1455bd0219c ee07b5577810 2024-09-16T00:46:12.111 INFO:teuthology.orchestra.run.smithi112.stdout:osd.4 smithi169 running (13m) 11m ago 24m 103M 2581M 19.1.1-326-g8921c8ef f1455bd0219c 769f026a54eb 2024-09-16T00:46:12.111 INFO:teuthology.orchestra.run.smithi112.stdout:osd.5 smithi169 running (12m) 11m ago 24m 125M 2581M 19.1.1-326-g8921c8ef f1455bd0219c a6007c13cf78 2024-09-16T00:46:12.111 INFO:teuthology.orchestra.run.smithi112.stdout:prometheus.smithi112 smithi112 *:9095 running (18m) 12m ago 27m 64.1M - 2.43.0 a07b618ecd1d 7a1d42e60d2b 2024-09-16T00:46:12.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.112+0000 7f5bfc061640 1 -- 172.21.15.112:0/3621648723 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5be006e530 msgr2=0x7f5be00709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:12.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.112+0000 7f5bfc061640 1 --2- 172.21.15.112:0/3621648723 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5be006e530 0x7f5be00709e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5bf00048a0 tx=0x7f5bf0009da0 comp rx=0 tx=0).stop 2024-09-16T00:46:12.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.112+0000 7f5bfc061640 1 -- 172.21.15.112:0/3621648723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bf40716e0 msgr2=0x7f5bf4119dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:12.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.112+0000 7f5bfc061640 1 --2- 172.21.15.112:0/3621648723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bf40716e0 0x7f5bf4119dc0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f5be4002a50 tx=0x7f5be4004600 comp rx=0 tx=0).stop 2024-09-16T00:46:12.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.112+0000 7f5bfc061640 1 -- 172.21.15.112:0/3621648723 shutdown_connections 2024-09-16T00:46:12.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.112+0000 7f5bfc061640 1 --2- 172.21.15.112:0/3621648723 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f5be006e530 0x7f5be00709e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:12.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.112+0000 7f5bfc061640 1 --2- 172.21.15.112:0/3621648723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bf40716e0 0x7f5bf4119dc0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:12.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.112+0000 7f5bfc061640 1 --2- 172.21.15.112:0/3621648723 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5bf406fff0 0x7f5bf4119880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:12.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.112+0000 7f5bfc061640 1 -- 172.21.15.112:0/3621648723 >> 172.21.15.112:0/3621648723 conn(0x7f5bf40f8120 msgr2=0x7f5bf40f93f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:12.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.112+0000 7f5bfc061640 1 -- 172.21.15.112:0/3621648723 shutdown_connections 2024-09-16T00:46:12.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.112+0000 7f5bfc061640 1 -- 172.21.15.112:0/3621648723 wait complete. 2024-09-16T00:46:12.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.189+0000 7fd180156640 1 -- 172.21.15.112:0/1656746981 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1781014c0 msgr2=0x7fd178101890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:12.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.189+0000 7fd180156640 1 --2- 172.21.15.112:0/1656746981 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1781014c0 0x7fd178101890 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd1740048d0 tx=0x7fd174009da0 comp rx=0 tx=0).stop 2024-09-16T00:46:12.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.189+0000 7fd180156640 1 -- 172.21.15.112:0/1656746981 shutdown_connections 2024-09-16T00:46:12.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.189+0000 7fd180156640 1 --2- 172.21.15.112:0/1656746981 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd1780fb480 0x7fd1780fb8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:12.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.189+0000 7fd180156640 1 --2- 172.21.15.112:0/1656746981 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1781014c0 0x7fd178101890 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:12.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.189+0000 7fd180156640 1 -- 172.21.15.112:0/1656746981 >> 172.21.15.112:0/1656746981 conn(0x7fd1780f7000 msgr2=0x7fd1780f9410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:12.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.189+0000 7fd180156640 1 -- 172.21.15.112:0/1656746981 shutdown_connections 2024-09-16T00:46:12.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.189+0000 7fd180156640 1 -- 172.21.15.112:0/1656746981 wait complete. 2024-09-16T00:46:12.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.190+0000 7fd180156640 1 Processor -- start 2024-09-16T00:46:12.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.190+0000 7fd180156640 1 -- start start 2024-09-16T00:46:12.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.190+0000 7fd180156640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd1780fb480 0x7fd17810e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:12.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.190+0000 7fd180156640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1781014c0 0x7fd17810e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:12.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.190+0000 7fd180156640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd17810ed20 con 0x7fd1781014c0 2024-09-16T00:46:12.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.190+0000 7fd180156640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd17810ee90 con 0x7fd1780fb480 2024-09-16T00:46:12.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.190+0000 7fd17d6f1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1781014c0 0x7fd17810e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:12.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.190+0000 7fd17d6f1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1781014c0 0x7fd17810e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49480/0 (socket says 172.21.15.112:49480) 2024-09-16T00:46:12.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.190+0000 7fd17d6f1640 1 -- 172.21.15.112:0/230505797 learned_addr learned my addr 172.21.15.112:0/230505797 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:46:12.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.190+0000 7fd17def2640 1 --2- 172.21.15.112:0/230505797 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd1780fb480 0x7fd17810e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:12.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.190+0000 7fd17d6f1640 1 -- 172.21.15.112:0/230505797 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd1780fb480 msgr2=0x7fd17810e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:12.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.190+0000 7fd17d6f1640 1 --2- 172.21.15.112:0/230505797 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd1780fb480 0x7fd17810e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:12.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.190+0000 7fd17d6f1640 1 -- 172.21.15.112:0/230505797 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd174005040 con 0x7fd1781014c0 2024-09-16T00:46:12.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.191+0000 7fd17d6f1640 1 --2- 172.21.15.112:0/230505797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1781014c0 0x7fd17810e6d0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd16800eb70 tx=0x7fd16800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:12.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.191+0000 7fd16e7fc640 1 -- 172.21.15.112:0/230505797 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd168007c30 con 0x7fd1781014c0 2024-09-16T00:46:12.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.191+0000 7fd16e7fc640 1 -- 172.21.15.112:0/230505797 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fd16800f040 con 0x7fd1781014c0 2024-09-16T00:46:12.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.191+0000 7fd180156640 1 -- 172.21.15.112:0/230505797 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd178117200 con 0x7fd1781014c0 2024-09-16T00:46:12.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.191+0000 7fd16e7fc640 1 -- 172.21.15.112:0/230505797 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd16800ca60 con 0x7fd1781014c0 2024-09-16T00:46:12.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.191+0000 7fd180156640 1 -- 172.21.15.112:0/230505797 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd178117660 con 0x7fd1781014c0 2024-09-16T00:46:12.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.192+0000 7fd180156640 1 -- 172.21.15.112:0/230505797 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1780fcbc0 con 0x7fd1781014c0 2024-09-16T00:46:12.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.193+0000 7fd16e7fc640 1 -- 172.21.15.112:0/230505797 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fd16800cbc0 con 0x7fd1781014c0 2024-09-16T00:46:12.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.193+0000 7fd16e7fc640 1 --2- 172.21.15.112:0/230505797 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd16406e530 0x7fd1640709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:12.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.193+0000 7fd16e7fc640 1 -- 172.21.15.112:0/230505797 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7fd16808de90 con 0x7fd1781014c0 2024-09-16T00:46:12.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.193+0000 7fd17def2640 1 --2- 172.21.15.112:0/230505797 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd16406e530 0x7fd1640709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:12.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.194+0000 7fd17def2640 1 --2- 172.21.15.112:0/230505797 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd16406e530 0x7fd1640709e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd174004540 tx=0x7fd174035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:12.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.198+0000 7fd16e7fc640 1 -- 172.21.15.112:0/230505797 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fd168056eb0 con 0x7fd1781014c0 2024-09-16T00:46:12.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:46:11 smithi169 ceph-mon[77626]: pgmap v553: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:46:12.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.426+0000 7fd180156640 1 -- 172.21.15.112:0/230505797 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd1781084f0 con 0x7fd1781014c0 2024-09-16T00:46:12.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.427+0000 7fd16e7fc640 1 -- 172.21.15.112:0/230505797 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7fd16805a570 con 0x7fd1781014c0 2024-09-16T00:46:12.427 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:46:12.427 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": { 2024-09-16T00:46:12.427 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:46:12.427 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:46:12.427 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": { 2024-09-16T00:46:12.428 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:46:12.428 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:46:12.428 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": { 2024-09-16T00:46:12.428 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 6 2024-09-16T00:46:12.428 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:46:12.428 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": { 2024-09-16T00:46:12.428 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 4 2024-09-16T00:46:12.428 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:46:12.428 INFO:teuthology.orchestra.run.smithi112.stdout: "overall": { 2024-09-16T00:46:12.428 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 14 2024-09-16T00:46:12.428 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-16T00:46:12.428 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:46:12.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.429+0000 7fd180156640 1 -- 172.21.15.112:0/230505797 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd16406e530 msgr2=0x7fd1640709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:12.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.429+0000 7fd180156640 1 --2- 172.21.15.112:0/230505797 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd16406e530 0x7fd1640709e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd174004540 tx=0x7fd174035040 comp rx=0 tx=0).stop 2024-09-16T00:46:12.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.429+0000 7fd180156640 1 -- 172.21.15.112:0/230505797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1781014c0 msgr2=0x7fd17810e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:12.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.429+0000 7fd180156640 1 --2- 172.21.15.112:0/230505797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1781014c0 0x7fd17810e6d0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd16800eb70 tx=0x7fd16800b040 comp rx=0 tx=0).stop 2024-09-16T00:46:12.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.429+0000 7fd180156640 1 -- 172.21.15.112:0/230505797 shutdown_connections 2024-09-16T00:46:12.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.429+0000 7fd180156640 1 --2- 172.21.15.112:0/230505797 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd16406e530 0x7fd1640709e0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:12.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.429+0000 7fd180156640 1 --2- 172.21.15.112:0/230505797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1781014c0 0x7fd17810e6d0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:12.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.429+0000 7fd180156640 1 --2- 172.21.15.112:0/230505797 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd1780fb480 0x7fd17810e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:12.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.429+0000 7fd180156640 1 -- 172.21.15.112:0/230505797 >> 172.21.15.112:0/230505797 conn(0x7fd1780f7000 msgr2=0x7fd1780f88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:12.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.429+0000 7fd180156640 1 -- 172.21.15.112:0/230505797 shutdown_connections 2024-09-16T00:46:12.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.429+0000 7fd180156640 1 -- 172.21.15.112:0/230505797 wait complete. 2024-09-16T00:46:12.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.507+0000 7f46265bb640 1 -- 172.21.15.112:0/1413969184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4620102ef0 msgr2=0x7f46200fc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:12.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.507+0000 7f46265bb640 1 --2- 172.21.15.112:0/1413969184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4620102ef0 0x7f46200fc5b0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f46140048d0 tx=0x7f4614009da0 comp rx=0 tx=0).stop 2024-09-16T00:46:12.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.507+0000 7f46265bb640 1 -- 172.21.15.112:0/1413969184 shutdown_connections 2024-09-16T00:46:12.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.507+0000 7f46265bb640 1 --2- 172.21.15.112:0/1413969184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4620102ef0 0x7f46200fc5b0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:12.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.507+0000 7f46265bb640 1 --2- 172.21.15.112:0/1413969184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4620102550 0x7f4620102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:12.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.507+0000 7f46265bb640 1 -- 172.21.15.112:0/1413969184 >> 172.21.15.112:0/1413969184 conn(0x7f46200f80f0 msgr2=0x7f46200fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:12.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.507+0000 7f46265bb640 1 -- 172.21.15.112:0/1413969184 shutdown_connections 2024-09-16T00:46:12.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.507+0000 7f46265bb640 1 -- 172.21.15.112:0/1413969184 wait complete. 2024-09-16T00:46:12.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.507+0000 7f46265bb640 1 Processor -- start 2024-09-16T00:46:12.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.507+0000 7f46265bb640 1 -- start start 2024-09-16T00:46:12.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.508+0000 7f46265bb640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4620102550 0x7f462006fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:12.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.508+0000 7f46265bb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4620102ef0 0x7f462006ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:12.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.508+0000 7f46265bb640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4620070490 con 0x7f4620102ef0 2024-09-16T00:46:12.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.508+0000 7f46265bb640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4620070600 con 0x7f4620102550 2024-09-16T00:46:12.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.508+0000 7f461f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4620102ef0 0x7f462006ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:12.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.508+0000 7f461f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4620102ef0 0x7f462006ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49498/0 (socket says 172.21.15.112:49498) 2024-09-16T00:46:12.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.508+0000 7f461f7fe640 1 -- 172.21.15.112:0/1204590087 learned_addr learned my addr 172.21.15.112:0/1204590087 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:46:12.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.508+0000 7f461ffff640 1 --2- 172.21.15.112:0/1204590087 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4620102550 0x7f462006fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:12.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.508+0000 7f461f7fe640 1 -- 172.21.15.112:0/1204590087 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4620102550 msgr2=0x7f462006fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:12.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.508+0000 7f461f7fe640 1 --2- 172.21.15.112:0/1204590087 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4620102550 0x7f462006fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:12.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.508+0000 7f461f7fe640 1 -- 172.21.15.112:0/1204590087 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4614005040 con 0x7f4620102ef0 2024-09-16T00:46:12.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.508+0000 7f461f7fe640 1 --2- 172.21.15.112:0/1204590087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4620102ef0 0x7f462006ff50 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f461402a530 tx=0x7f4614009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:12.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.508+0000 7f461cff9640 1 -- 172.21.15.112:0/1204590087 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4614038070 con 0x7f4620102ef0 2024-09-16T00:46:12.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.509+0000 7f461cff9640 1 -- 172.21.15.112:0/1204590087 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f461402ade0 con 0x7f4620102ef0 2024-09-16T00:46:12.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.509+0000 7f46265bb640 1 -- 172.21.15.112:0/1204590087 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4620070830 con 0x7f4620102ef0 2024-09-16T00:46:12.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.509+0000 7f461cff9640 1 -- 172.21.15.112:0/1204590087 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f461402c770 con 0x7f4620102ef0 2024-09-16T00:46:12.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.509+0000 7f46265bb640 1 -- 172.21.15.112:0/1204590087 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4620119b80 con 0x7f4620102ef0 2024-09-16T00:46:12.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.510+0000 7f46265bb640 1 -- 172.21.15.112:0/1204590087 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46200fdcd0 con 0x7f4620102ef0 2024-09-16T00:46:12.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.510+0000 7f461cff9640 1 -- 172.21.15.112:0/1204590087 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f4614030020 con 0x7f4620102ef0 2024-09-16T00:46:12.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.511+0000 7f461cff9640 1 --2- 172.21.15.112:0/1204590087 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f460806e530 0x7f46080709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:12.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.511+0000 7f461cff9640 1 -- 172.21.15.112:0/1204590087 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f46140ae720 con 0x7f4620102ef0 2024-09-16T00:46:12.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.511+0000 7f461ffff640 1 --2- 172.21.15.112:0/1204590087 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f460806e530 0x7f46080709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:12.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.511+0000 7f461ffff640 1 --2- 172.21.15.112:0/1204590087 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f460806e530 0x7f46080709e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4610004a70 tx=0x7f461000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:12.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.515+0000 7f461cff9640 1 -- 172.21.15.112:0/1204590087 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f4614077690 con 0x7f4620102ef0 2024-09-16T00:46:12.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.716+0000 7f46265bb640 1 -- 172.21.15.112:0/1204590087 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f462004b720 con 0x7f4620102ef0 2024-09-16T00:46:12.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.717+0000 7f461cff9640 1 -- 172.21.15.112:0/1204590087 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1914 (secure 0 0 0) 0x7f461407ad50 con 0x7f4620102ef0 2024-09-16T00:46:12.717 INFO:teuthology.orchestra.run.smithi112.stdout:e34 2024-09-16T00:46:12.717 INFO:teuthology.orchestra.run.smithi112.stdout:btime 2024-09-16T00:34:26:619254+0000 2024-09-16T00:46:12.717 INFO:teuthology.orchestra.run.smithi112.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T00:46:12.717 INFO:teuthology.orchestra.run.smithi112.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:46:12.717 INFO:teuthology.orchestra.run.smithi112.stdout:legacy client fscid: 1 2024-09-16T00:46:12.717 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:46:12.717 INFO:teuthology.orchestra.run.smithi112.stdout:Filesystem 'cephfs' (1) 2024-09-16T00:46:12.717 INFO:teuthology.orchestra.run.smithi112.stdout:fs_name cephfs 2024-09-16T00:46:12.717 INFO:teuthology.orchestra.run.smithi112.stdout:epoch 25 2024-09-16T00:46:12.718 INFO:teuthology.orchestra.run.smithi112.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-16T00:46:12.718 INFO:teuthology.orchestra.run.smithi112.stdout:created 2024-09-16T00:23:22.328199+0000 2024-09-16T00:46:12.718 INFO:teuthology.orchestra.run.smithi112.stdout:modified 2024-09-16T00:33:29.012991+0000 2024-09-16T00:46:12.718 INFO:teuthology.orchestra.run.smithi112.stdout:tableserver 0 2024-09-16T00:46:12.718 INFO:teuthology.orchestra.run.smithi112.stdout:root 0 2024-09-16T00:46:12.718 INFO:teuthology.orchestra.run.smithi112.stdout:session_timeout 60 2024-09-16T00:46:12.718 INFO:teuthology.orchestra.run.smithi112.stdout:session_autoclose 300 2024-09-16T00:46:12.718 INFO:teuthology.orchestra.run.smithi112.stdout:max_file_size 1099511627776 2024-09-16T00:46:12.718 INFO:teuthology.orchestra.run.smithi112.stdout:max_xattr_size 65536 2024-09-16T00:46:12.718 INFO:teuthology.orchestra.run.smithi112.stdout:required_client_features {} 2024-09-16T00:46:12.718 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure 0 2024-09-16T00:46:12.718 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure_osd_epoch 82 2024-09-16T00:46:12.718 INFO:teuthology.orchestra.run.smithi112.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:46:12.718 INFO:teuthology.orchestra.run.smithi112.stdout:max_mds 2 2024-09-16T00:46:12.719 INFO:teuthology.orchestra.run.smithi112.stdout:in 0,1 2024-09-16T00:46:12.719 INFO:teuthology.orchestra.run.smithi112.stdout:up {} 2024-09-16T00:46:12.719 INFO:teuthology.orchestra.run.smithi112.stdout:failed 0,1 2024-09-16T00:46:12.719 INFO:teuthology.orchestra.run.smithi112.stdout:damaged 2024-09-16T00:46:12.719 INFO:teuthology.orchestra.run.smithi112.stdout:stopped 2024-09-16T00:46:12.719 INFO:teuthology.orchestra.run.smithi112.stdout:data_pools [3] 2024-09-16T00:46:12.719 INFO:teuthology.orchestra.run.smithi112.stdout:metadata_pool 2 2024-09-16T00:46:12.719 INFO:teuthology.orchestra.run.smithi112.stdout:inline_data enabled 2024-09-16T00:46:12.719 INFO:teuthology.orchestra.run.smithi112.stdout:balancer 2024-09-16T00:46:12.719 INFO:teuthology.orchestra.run.smithi112.stdout:bal_rank_mask -1 2024-09-16T00:46:12.719 INFO:teuthology.orchestra.run.smithi112.stdout:standby_count_wanted 1 2024-09-16T00:46:12.719 INFO:teuthology.orchestra.run.smithi112.stdout:qdb_cluster leader: 0 members: 2024-09-16T00:46:12.719 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:46:12.719 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:46:12.720 INFO:teuthology.orchestra.run.smithi112.stdout:Standby daemons: 2024-09-16T00:46:12.720 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:46:12.720 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.ithwzj{-1:34392} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.112:6826/476229221,v1:172.21.15.112:6827/476229221] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:46:12.720 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.hxfwkh{-1:34404} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.112:6828/2356899876,v1:172.21.15.112:6829/2356899876] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:46:12.720 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.gawwgw{-1:44343} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.169:6824/1635329957,v1:172.21.15.169:6825/1635329957] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:46:12.720 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.lgeqrj{-1:44347} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.169:6826/184810031,v1:172.21.15.169:6827/184810031] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:46:12.720 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 34 2024-09-16T00:46:12.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.723+0000 7f46265bb640 1 -- 172.21.15.112:0/1204590087 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f460806e530 msgr2=0x7f46080709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:12.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.723+0000 7f46265bb640 1 --2- 172.21.15.112:0/1204590087 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f460806e530 0x7f46080709e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4610004a70 tx=0x7f461000b040 comp rx=0 tx=0).stop 2024-09-16T00:46:12.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.723+0000 7f46265bb640 1 -- 172.21.15.112:0/1204590087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4620102ef0 msgr2=0x7f462006ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:12.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.723+0000 7f46265bb640 1 --2- 172.21.15.112:0/1204590087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4620102ef0 0x7f462006ff50 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f461402a530 tx=0x7f4614009da0 comp rx=0 tx=0).stop 2024-09-16T00:46:12.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.723+0000 7f46265bb640 1 -- 172.21.15.112:0/1204590087 shutdown_connections 2024-09-16T00:46:12.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.723+0000 7f46265bb640 1 --2- 172.21.15.112:0/1204590087 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f460806e530 0x7f46080709e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:12.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.723+0000 7f46265bb640 1 --2- 172.21.15.112:0/1204590087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4620102ef0 0x7f462006ff50 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:12.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.723+0000 7f46265bb640 1 --2- 172.21.15.112:0/1204590087 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4620102550 0x7f462006fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:12.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.723+0000 7f46265bb640 1 -- 172.21.15.112:0/1204590087 >> 172.21.15.112:0/1204590087 conn(0x7f46200f80f0 msgr2=0x7f46200f9150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:12.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.723+0000 7f46265bb640 1 -- 172.21.15.112:0/1204590087 shutdown_connections 2024-09-16T00:46:12.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.723+0000 7f46265bb640 1 -- 172.21.15.112:0/1204590087 wait complete. 2024-09-16T00:46:12.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.803+0000 7f0951b15640 1 -- 172.21.15.112:0/3839709382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f094c0fb810 msgr2=0x7f094c0fbbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:12.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.803+0000 7f0951b15640 1 --2- 172.21.15.112:0/3839709382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f094c0fb810 0x7f094c0fbbe0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f093c0048d0 tx=0x7f093c009da0 comp rx=0 tx=0).stop 2024-09-16T00:46:12.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.803+0000 7f0951b15640 1 -- 172.21.15.112:0/3839709382 shutdown_connections 2024-09-16T00:46:12.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.803+0000 7f0951b15640 1 --2- 172.21.15.112:0/3839709382 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f094c0fcf80 0x7f094c0fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:12.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.803+0000 7f0951b15640 1 --2- 172.21.15.112:0/3839709382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f094c0fb810 0x7f094c0fbbe0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:12.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.803+0000 7f0951b15640 1 -- 172.21.15.112:0/3839709382 >> 172.21.15.112:0/3839709382 conn(0x7f094c0723b0 msgr2=0x7f094c0727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:12.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.803+0000 7f0951b15640 1 -- 172.21.15.112:0/3839709382 shutdown_connections 2024-09-16T00:46:12.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.803+0000 7f0951b15640 1 -- 172.21.15.112:0/3839709382 wait complete. 2024-09-16T00:46:12.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.804+0000 7f0951b15640 1 Processor -- start 2024-09-16T00:46:12.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.804+0000 7f0951b15640 1 -- start start 2024-09-16T00:46:12.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.804+0000 7f0951b15640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f094c0fb810 0x7f094c1120b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:12.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.804+0000 7f0951b15640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f094c0fcf80 0x7f094c1125f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:12.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.804+0000 7f0951b15640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f094c112bf0 con 0x7f094c0fb810 2024-09-16T00:46:12.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.804+0000 7f0951b15640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f094c10c030 con 0x7f094c0fcf80 2024-09-16T00:46:12.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.804+0000 7f094b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f094c0fb810 0x7f094c1120b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:12.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.805+0000 7f094b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f094c0fb810 0x7f094c1120b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49516/0 (socket says 172.21.15.112:49516) 2024-09-16T00:46:12.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.805+0000 7f094b7fe640 1 -- 172.21.15.112:0/976215906 learned_addr learned my addr 172.21.15.112:0/976215906 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:46:12.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.805+0000 7f094affd640 1 --2- 172.21.15.112:0/976215906 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f094c0fcf80 0x7f094c1125f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:12.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.805+0000 7f094b7fe640 1 -- 172.21.15.112:0/976215906 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f094c0fcf80 msgr2=0x7f094c1125f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:12.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.805+0000 7f094b7fe640 1 --2- 172.21.15.112:0/976215906 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f094c0fcf80 0x7f094c1125f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:12.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.805+0000 7f094b7fe640 1 -- 172.21.15.112:0/976215906 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f093c005040 con 0x7f094c0fb810 2024-09-16T00:46:12.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.805+0000 7f094b7fe640 1 --2- 172.21.15.112:0/976215906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f094c0fb810 0x7f094c1120b0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f093c003080 tx=0x7f093c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:12.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.805+0000 7f0933fff640 1 -- 172.21.15.112:0/976215906 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f093c038070 con 0x7f094c0fb810 2024-09-16T00:46:12.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.805+0000 7f0933fff640 1 -- 172.21.15.112:0/976215906 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f093c02ade0 con 0x7f094c0fb810 2024-09-16T00:46:12.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.805+0000 7f0933fff640 1 -- 172.21.15.112:0/976215906 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f093c02c880 con 0x7f094c0fb810 2024-09-16T00:46:12.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.805+0000 7f0951b15640 1 -- 172.21.15.112:0/976215906 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f094c10c2b0 con 0x7f094c0fb810 2024-09-16T00:46:12.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.805+0000 7f0951b15640 1 -- 172.21.15.112:0/976215906 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f094c10c790 con 0x7f094c0fb810 2024-09-16T00:46:12.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.807+0000 7f0951b15640 1 -- 172.21.15.112:0/976215906 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f094c0fcbc0 con 0x7f094c0fb810 2024-09-16T00:46:12.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.807+0000 7f0933fff640 1 -- 172.21.15.112:0/976215906 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f093c030020 con 0x7f094c0fb810 2024-09-16T00:46:12.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.807+0000 7f0933fff640 1 --2- 172.21.15.112:0/976215906 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f093406e530 0x7f09340709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:12.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.808+0000 7f0933fff640 1 -- 172.21.15.112:0/976215906 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f093c0ae2a0 con 0x7f094c0fb810 2024-09-16T00:46:12.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.808+0000 7f094affd640 1 --2- 172.21.15.112:0/976215906 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f093406e530 0x7f09340709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:12.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.808+0000 7f094affd640 1 --2- 172.21.15.112:0/976215906 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f093406e530 0x7f09340709e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f094c10d970 tx=0x7f0940009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:12.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.812+0000 7f0933fff640 1 -- 172.21.15.112:0/976215906 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f093c077290 con 0x7f094c0fb810 2024-09-16T00:46:12.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.996+0000 7f0951b15640 1 -- 172.21.15.112:0/976215906 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f094c04b780 con 0x7f093406e530 2024-09-16T00:46:12.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.998+0000 7f0933fff640 1 -- 172.21.15.112:0/976215906 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f094c04b780 con 0x7f093406e530 2024-09-16T00:46:12.998 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:46:12.998 INFO:teuthology.orchestra.run.smithi112.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80", 2024-09-16T00:46:12.998 INFO:teuthology.orchestra.run.smithi112.stdout: "in_progress": true, 2024-09-16T00:46:12.998 INFO:teuthology.orchestra.run.smithi112.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T00:46:12.998 INFO:teuthology.orchestra.run.smithi112.stdout: "services_complete": [ 2024-09-16T00:46:12.998 INFO:teuthology.orchestra.run.smithi112.stdout: "mds", 2024-09-16T00:46:12.998 INFO:teuthology.orchestra.run.smithi112.stdout: "osd", 2024-09-16T00:46:12.998 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr", 2024-09-16T00:46:12.998 INFO:teuthology.orchestra.run.smithi112.stdout: "mon", 2024-09-16T00:46:12.998 INFO:teuthology.orchestra.run.smithi112.stdout: "crash" 2024-09-16T00:46:12.998 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-16T00:46:12.998 INFO:teuthology.orchestra.run.smithi112.stdout: "progress": "16/21 daemons upgraded", 2024-09-16T00:46:12.999 INFO:teuthology.orchestra.run.smithi112.stdout: "message": "Currently upgrading mds daemons", 2024-09-16T00:46:12.999 INFO:teuthology.orchestra.run.smithi112.stdout: "is_paused": false 2024-09-16T00:46:12.999 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:46:13.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.999+0000 7f0951b15640 1 -- 172.21.15.112:0/976215906 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f093406e530 msgr2=0x7f09340709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:13.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.999+0000 7f0951b15640 1 --2- 172.21.15.112:0/976215906 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f093406e530 0x7f09340709e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f094c10d970 tx=0x7f0940009040 comp rx=0 tx=0).stop 2024-09-16T00:46:13.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.999+0000 7f0951b15640 1 -- 172.21.15.112:0/976215906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f094c0fb810 msgr2=0x7f094c1120b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:13.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:12.999+0000 7f0951b15640 1 --2- 172.21.15.112:0/976215906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f094c0fb810 0x7f094c1120b0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f093c003080 tx=0x7f093c009da0 comp rx=0 tx=0).stop 2024-09-16T00:46:13.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:13.000+0000 7f0951b15640 1 -- 172.21.15.112:0/976215906 shutdown_connections 2024-09-16T00:46:13.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:13.000+0000 7f0951b15640 1 --2- 172.21.15.112:0/976215906 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f093406e530 0x7f09340709e0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:13.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:13.000+0000 7f0951b15640 1 --2- 172.21.15.112:0/976215906 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f094c0fcf80 0x7f094c1125f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:13.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:13.000+0000 7f0951b15640 1 --2- 172.21.15.112:0/976215906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f094c0fb810 0x7f094c1120b0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:13.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:13.000+0000 7f0951b15640 1 -- 172.21.15.112:0/976215906 >> 172.21.15.112:0/976215906 conn(0x7f094c0723b0 msgr2=0x7f094c0fa480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:13.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:13.000+0000 7f0951b15640 1 -- 172.21.15.112:0/976215906 shutdown_connections 2024-09-16T00:46:13.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:13.000+0000 7f0951b15640 1 -- 172.21.15.112:0/976215906 wait complete. 2024-09-16T00:46:13.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:13.078+0000 7fd5f4e26640 1 -- 172.21.15.112:0/487774832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5f00fdf20 msgr2=0x7fd5f01065f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:13.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:13.078+0000 7fd5f4e26640 1 --2- 172.21.15.112:0/487774832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5f00fdf20 0x7fd5f01065f0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4004900 tx=0x7fd5e4009da0 comp rx=0 tx=0).stop 2024-09-16T00:46:13.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:13.078+0000 7fd5f4e26640 1 -- 172.21.15.112:0/487774832 shutdown_connections 2024-09-16T00:46:13.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:13.078+0000 7fd5f4e26640 1 --2- 172.21.15.112:0/487774832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5f00fdf20 0x7fd5f01065f0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:13.079 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:46:12 smithi112 ceph-mon[97750]: from='client.35052 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:46:13.079 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:46:12 smithi112 ceph-mon[97750]: from='client.35056 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:46:13.079 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:46:12 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/230505797' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:46:13.079 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:46:12 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/1204590087' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:46:13.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:13.078+0000 7fd5f4e26640 1 --2- 172.21.15.112:0/487774832 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5f00fd580 0x7fd5f00fd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:13.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:13.078+0000 7fd5f4e26640 1 -- 172.21.15.112:0/487774832 >> 172.21.15.112:0/487774832 conn(0x7fd5f00f7020 msgr2=0x7fd5f00f9430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:13.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:13.078+0000 7fd5f4e26640 1 -- 172.21.15.112:0/487774832 shutdown_connections 2024-09-16T00:46:13.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:13.078+0000 7fd5f4e26640 1 -- 172.21.15.112:0/487774832 wait complete. 2024-09-16T00:46:13.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:13.079+0000 7fd5f4e26640 1 Processor -- start 2024-09-16T00:46:13.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:13.079+0000 7fd5f4e26640 1 -- start start 2024-09-16T00:46:13.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:13.079+0000 7fd5f4e26640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5f00fd580 0x7fd5f00fb440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:13.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:13.079+0000 7fd5f4e26640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5f00fdf20 0x7fd5f00fb980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:13.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:13.079+0000 7fd5f4e26640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f00fbf50 con 0x7fd5f00fdf20 2024-09-16T00:46:13.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:13.079+0000 7fd5f4e26640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f00fc0c0 con 0x7fd5f00fd580 2024-09-16T00:46:13.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:13.079+0000 7fd5edd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5f00fdf20 0x7fd5f00fb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:13.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:13.079+0000 7fd5edd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5f00fdf20 0x7fd5f00fb980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49542/0 (socket says 172.21.15.112:49542) 2024-09-16T00:46:13.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:13.079+0000 7fd5edd9b640 1 -- 172.21.15.112:0/1150911623 learned_addr learned my addr 172.21.15.112:0/1150911623 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:46:13.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:13.079+0000 7fd5ee59c640 1 --2- 172.21.15.112:0/1150911623 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5f00fd580 0x7fd5f00fb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:13.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:13.080+0000 7fd5edd9b640 1 -- 172.21.15.112:0/1150911623 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5f00fd580 msgr2=0x7fd5f00fb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:13.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:13.080+0000 7fd5edd9b640 1 --2- 172.21.15.112:0/1150911623 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5f00fd580 0x7fd5f00fb440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:13.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:13.080+0000 7fd5edd9b640 1 -- 172.21.15.112:0/1150911623 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5e4005040 con 0x7fd5f00fdf20 2024-09-16T00:46:13.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:13.080+0000 7fd5edd9b640 1 --2- 172.21.15.112:0/1150911623 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5f00fdf20 0x7fd5f00fb980 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4004960 tx=0x7fd5e4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:13.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:13.080+0000 7fd5deffd640 1 -- 172.21.15.112:0/1150911623 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd5e4038070 con 0x7fd5f00fdf20 2024-09-16T00:46:13.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:13.080+0000 7fd5f4e26640 1 -- 172.21.15.112:0/1150911623 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5f0119650 con 0x7fd5f00fdf20 2024-09-16T00:46:13.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:13.080+0000 7fd5deffd640 1 -- 172.21.15.112:0/1150911623 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fd5e4008ed0 con 0x7fd5f00fdf20 2024-09-16T00:46:13.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:13.080+0000 7fd5deffd640 1 -- 172.21.15.112:0/1150911623 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd5e40095a0 con 0x7fd5f00fdf20 2024-09-16T00:46:13.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:13.080+0000 7fd5f4e26640 1 -- 172.21.15.112:0/1150911623 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5f01199d0 con 0x7fd5f00fdf20 2024-09-16T00:46:13.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:13.081+0000 7fd5f4e26640 1 -- 172.21.15.112:0/1150911623 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5f00662d0 con 0x7fd5f00fdf20 2024-09-16T00:46:13.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:13.082+0000 7fd5deffd640 1 -- 172.21.15.112:0/1150911623 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fd5e4030020 con 0x7fd5f00fdf20 2024-09-16T00:46:13.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:13.082+0000 7fd5deffd640 1 --2- 172.21.15.112:0/1150911623 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd5d806e530 0x7fd5d80709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:13.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:13.082+0000 7fd5deffd640 1 -- 172.21.15.112:0/1150911623 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7fd5e40ae2a0 con 0x7fd5f00fdf20 2024-09-16T00:46:13.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:13.082+0000 7fd5ee59c640 1 --2- 172.21.15.112:0/1150911623 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd5d806e530 0x7fd5d80709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:13.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:13.083+0000 7fd5ee59c640 1 --2- 172.21.15.112:0/1150911623 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd5d806e530 0x7fd5d80709e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd5e0006fd0 tx=0x7fd5e0008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:13.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:13.087+0000 7fd5deffd640 1 -- 172.21.15.112:0/1150911623 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fd5e4077270 con 0x7fd5f00fdf20 2024-09-16T00:46:13.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:46:12 smithi169 ceph-mon[77626]: from='client.35052 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:46:13.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:46:12 smithi169 ceph-mon[77626]: from='client.35056 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:46:13.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:46:12 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/230505797' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:46:13.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:46:12 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/1204590087' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:46:13.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:13.316+0000 7fd5f4e26640 1 -- 172.21.15.112:0/1150911623 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd5f004b720 con 0x7fd5f00fdf20 2024-09-16T00:46:13.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:13.316+0000 7fd5deffd640 1 -- 172.21.15.112:0/1150911623 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7fd5e407a930 con 0x7fd5f00fdf20 2024-09-16T00:46:13.316 INFO:teuthology.orchestra.run.smithi112.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-16T00:46:13.317 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-16T00:46:13.317 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs is degraded 2024-09-16T00:46:13.317 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T00:46:13.317 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T00:46:13.317 INFO:teuthology.orchestra.run.smithi112.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-16T00:46:13.317 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-16T00:46:13.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:13.318+0000 7fd5f4e26640 1 -- 172.21.15.112:0/1150911623 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd5d806e530 msgr2=0x7fd5d80709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:13.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:13.318+0000 7fd5f4e26640 1 --2- 172.21.15.112:0/1150911623 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd5d806e530 0x7fd5d80709e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd5e0006fd0 tx=0x7fd5e0008040 comp rx=0 tx=0).stop 2024-09-16T00:46:13.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:13.318+0000 7fd5f4e26640 1 -- 172.21.15.112:0/1150911623 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5f00fdf20 msgr2=0x7fd5f00fb980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:13.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:13.318+0000 7fd5f4e26640 1 --2- 172.21.15.112:0/1150911623 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5f00fdf20 0x7fd5f00fb980 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4004960 tx=0x7fd5e4009da0 comp rx=0 tx=0).stop 2024-09-16T00:46:13.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:13.318+0000 7fd5f4e26640 1 -- 172.21.15.112:0/1150911623 shutdown_connections 2024-09-16T00:46:13.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:13.318+0000 7fd5f4e26640 1 --2- 172.21.15.112:0/1150911623 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd5d806e530 0x7fd5d80709e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:13.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:13.318+0000 7fd5f4e26640 1 --2- 172.21.15.112:0/1150911623 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5f00fdf20 0x7fd5f00fb980 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:13.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:13.318+0000 7fd5f4e26640 1 --2- 172.21.15.112:0/1150911623 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5f00fd580 0x7fd5f00fb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:13.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:13.318+0000 7fd5f4e26640 1 -- 172.21.15.112:0/1150911623 >> 172.21.15.112:0/1150911623 conn(0x7fd5f00f7020 msgr2=0x7fd5f00f8540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:13.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:13.319+0000 7fd5f4e26640 1 -- 172.21.15.112:0/1150911623 shutdown_connections 2024-09-16T00:46:13.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:13.319+0000 7fd5f4e26640 1 -- 172.21.15.112:0/1150911623 wait complete. 2024-09-16T00:46:14.088 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:46:13 smithi112 ceph-mon[97750]: from='client.35060 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:46:14.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:46:13 smithi112 ceph-mon[97750]: pgmap v554: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:46:14.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:46:13 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/1150911623' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:46:14.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:46:13 smithi169 ceph-mon[77626]: from='client.35060 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:46:14.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:46:13 smithi169 ceph-mon[77626]: pgmap v554: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:46:14.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:46:13 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/1150911623' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:46:15.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:46:14 smithi169 ceph-mon[77626]: from='client.35072 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:46:15.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:46:14 smithi112 ceph-mon[97750]: from='client.35072 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:46:16.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:46:15 smithi169 ceph-mon[77626]: pgmap v555: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:46:16.338 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:46:15 smithi112 ceph-mon[97750]: pgmap v555: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:46:17.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:46:16 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:46:17.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:46:16 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:46:18.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:46:17 smithi169 ceph-mon[77626]: pgmap v556: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:46:18.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:46:17 smithi112 ceph-mon[97750]: pgmap v556: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:46:19.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:46:18 smithi169 ceph-mon[77626]: pgmap v557: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:46:19.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:46:18 smithi112 ceph-mon[97750]: pgmap v557: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:46:22.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:46:21 smithi112 ceph-mon[97750]: pgmap v558: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:46:22.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:46:21 smithi169 ceph-mon[77626]: pgmap v558: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:46:24.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:46:23 smithi112 ceph-mon[97750]: pgmap v559: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:46:24.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:46:23 smithi169 ceph-mon[77626]: pgmap v559: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:46:26.078 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:46:25 smithi169 ceph-mon[77626]: pgmap v560: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:46:26.088 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:46:25 smithi112 ceph-mon[97750]: pgmap v560: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:46:28.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:46:27 smithi112 ceph-mon[97750]: pgmap v561: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:46:28.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:46:27 smithi169 ceph-mon[77626]: pgmap v561: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:46:30.088 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:46:29 smithi112 ceph-mon[97750]: pgmap v562: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:46:30.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:46:29 smithi169 ceph-mon[77626]: pgmap v562: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:46:32.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:46:31 smithi112 ceph-mon[97750]: pgmap v563: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:46:32.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:46:31 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:46:32.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:46:31 smithi169 ceph-mon[77626]: pgmap v563: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:46:32.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:46:31 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:46:34.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:46:33 smithi112 ceph-mon[97750]: pgmap v564: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:46:34.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:46:33 smithi169 ceph-mon[77626]: pgmap v564: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:46:36.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:46:35 smithi112 ceph-mon[97750]: pgmap v565: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:46:36.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:46:35 smithi169 ceph-mon[77626]: pgmap v565: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:46:38.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:46:37 smithi112 ceph-mon[97750]: pgmap v566: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:46:38.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:46:37 smithi169 ceph-mon[77626]: pgmap v566: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:46:40.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:46:39 smithi112 ceph-mon[97750]: pgmap v567: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:46:40.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:46:39 smithi169 ceph-mon[77626]: pgmap v567: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:46:42.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:46:41 smithi112 ceph-mon[97750]: pgmap v568: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:46:42.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:46:41 smithi169 ceph-mon[77626]: pgmap v568: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:46:43.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.408+0000 7f96543a9640 1 -- 172.21.15.112:0/1823421111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f964c0fcf80 msgr2=0x7f964c0fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:43.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.408+0000 7f96543a9640 1 --2- 172.21.15.112:0/1823421111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f964c0fcf80 0x7f964c0fd3d0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f96400048d0 tx=0x7f9640009da0 comp rx=0 tx=0).stop 2024-09-16T00:46:43.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.408+0000 7f96543a9640 1 -- 172.21.15.112:0/1823421111 shutdown_connections 2024-09-16T00:46:43.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.408+0000 7f96543a9640 1 --2- 172.21.15.112:0/1823421111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f964c0fcf80 0x7f964c0fd3d0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:43.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.408+0000 7f96543a9640 1 --2- 172.21.15.112:0/1823421111 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f964c0fb810 0x7f964c0fbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:43.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.408+0000 7f96543a9640 1 -- 172.21.15.112:0/1823421111 >> 172.21.15.112:0/1823421111 conn(0x7f964c0723b0 msgr2=0x7f964c0727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:43.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.408+0000 7f96543a9640 1 -- 172.21.15.112:0/1823421111 shutdown_connections 2024-09-16T00:46:43.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.408+0000 7f96543a9640 1 -- 172.21.15.112:0/1823421111 wait complete. 2024-09-16T00:46:43.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.409+0000 7f96543a9640 1 Processor -- start 2024-09-16T00:46:43.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.409+0000 7f96543a9640 1 -- start start 2024-09-16T00:46:43.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.409+0000 7f96543a9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f964c0fb810 0x7f964c1120d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:43.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.409+0000 7f96543a9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f964c0fcf80 0x7f964c112610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:43.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.409+0000 7f96543a9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f964c112c10 con 0x7f964c0fcf80 2024-09-16T00:46:43.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.409+0000 7f96543a9640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f964c10c050 con 0x7f964c0fb810 2024-09-16T00:46:43.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.409+0000 7f9651944640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f964c0fcf80 0x7f964c112610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:43.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.409+0000 7f9651944640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f964c0fcf80 0x7f964c112610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59568/0 (socket says 172.21.15.112:59568) 2024-09-16T00:46:43.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.409+0000 7f9651944640 1 -- 172.21.15.112:0/3308732250 learned_addr learned my addr 172.21.15.112:0/3308732250 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:46:43.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.409+0000 7f9652145640 1 --2- 172.21.15.112:0/3308732250 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f964c0fb810 0x7f964c1120d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:43.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.410+0000 7f9651944640 1 -- 172.21.15.112:0/3308732250 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f964c0fb810 msgr2=0x7f964c1120d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:43.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.410+0000 7f9651944640 1 --2- 172.21.15.112:0/3308732250 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f964c0fb810 0x7f964c1120d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:43.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.410+0000 7f9651944640 1 -- 172.21.15.112:0/3308732250 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9640005040 con 0x7f964c0fcf80 2024-09-16T00:46:43.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.410+0000 7f9652145640 1 --2- 172.21.15.112:0/3308732250 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f964c0fb810 0x7f964c1120d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:46:43.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.410+0000 7f9651944640 1 --2- 172.21.15.112:0/3308732250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f964c0fcf80 0x7f964c112610 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f9640000c00 tx=0x7f9640009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:43.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.410+0000 7f964affd640 1 -- 172.21.15.112:0/3308732250 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9640038070 con 0x7f964c0fcf80 2024-09-16T00:46:43.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.410+0000 7f964affd640 1 -- 172.21.15.112:0/3308732250 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f964002ade0 con 0x7f964c0fcf80 2024-09-16T00:46:43.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.410+0000 7f964affd640 1 -- 172.21.15.112:0/3308732250 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f964002c7e0 con 0x7f964c0fcf80 2024-09-16T00:46:43.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.410+0000 7f96543a9640 1 -- 172.21.15.112:0/3308732250 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f964c10c2d0 con 0x7f964c0fcf80 2024-09-16T00:46:43.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.410+0000 7f96543a9640 1 -- 172.21.15.112:0/3308732250 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f964c10c7c0 con 0x7f964c0fcf80 2024-09-16T00:46:43.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.411+0000 7f96543a9640 1 -- 172.21.15.112:0/3308732250 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f964c0fcbc0 con 0x7f964c0fcf80 2024-09-16T00:46:43.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.412+0000 7f964affd640 1 -- 172.21.15.112:0/3308732250 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f9640030020 con 0x7f964c0fcf80 2024-09-16T00:46:43.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.412+0000 7f964affd640 1 --2- 172.21.15.112:0/3308732250 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f963406e530 0x7f96340709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:43.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.412+0000 7f964affd640 1 -- 172.21.15.112:0/3308732250 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f96400adf70 con 0x7f964c0fcf80 2024-09-16T00:46:43.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.412+0000 7f9652145640 1 --2- 172.21.15.112:0/3308732250 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f963406e530 0x7f96340709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:43.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.413+0000 7f9652145640 1 --2- 172.21.15.112:0/3308732250 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f963406e530 0x7f96340709e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f963c007ee0 tx=0x7f963c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:43.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.417+0000 7f964affd640 1 -- 172.21.15.112:0/3308732250 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f96400e31b0 con 0x7f964c0fcf80 2024-09-16T00:46:43.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.600+0000 7f96543a9640 1 -- 172.21.15.112:0/3308732250 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f964c10d740 con 0x7f963406e530 2024-09-16T00:46:43.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.601+0000 7f964affd640 1 -- 172.21.15.112:0/3308732250 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f964c10d740 con 0x7f963406e530 2024-09-16T00:46:43.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.603+0000 7f96543a9640 1 -- 172.21.15.112:0/3308732250 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f963406e530 msgr2=0x7f96340709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:43.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.603+0000 7f96543a9640 1 --2- 172.21.15.112:0/3308732250 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f963406e530 0x7f96340709e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f963c007ee0 tx=0x7f963c00b040 comp rx=0 tx=0).stop 2024-09-16T00:46:43.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.603+0000 7f96543a9640 1 -- 172.21.15.112:0/3308732250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f964c0fcf80 msgr2=0x7f964c112610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:43.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.603+0000 7f96543a9640 1 --2- 172.21.15.112:0/3308732250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f964c0fcf80 0x7f964c112610 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f9640000c00 tx=0x7f9640009da0 comp rx=0 tx=0).stop 2024-09-16T00:46:43.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.603+0000 7f96543a9640 1 -- 172.21.15.112:0/3308732250 shutdown_connections 2024-09-16T00:46:43.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.603+0000 7f96543a9640 1 --2- 172.21.15.112:0/3308732250 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f963406e530 0x7f96340709e0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:43.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.603+0000 7f96543a9640 1 --2- 172.21.15.112:0/3308732250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f964c0fcf80 0x7f964c112610 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:43.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.603+0000 7f96543a9640 1 --2- 172.21.15.112:0/3308732250 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f964c0fb810 0x7f964c1120d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:43.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.603+0000 7f96543a9640 1 -- 172.21.15.112:0/3308732250 >> 172.21.15.112:0/3308732250 conn(0x7f964c0723b0 msgr2=0x7f964c0fa480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:43.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.604+0000 7f96543a9640 1 -- 172.21.15.112:0/3308732250 shutdown_connections 2024-09-16T00:46:43.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.604+0000 7f96543a9640 1 -- 172.21.15.112:0/3308732250 wait complete. 2024-09-16T00:46:43.612 INFO:teuthology.orchestra.run.smithi112.stdout:true 2024-09-16T00:46:43.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.684+0000 7efcde51a640 1 -- 172.21.15.112:0/1821842739 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcd80fb810 msgr2=0x7efcd80fbbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:43.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.684+0000 7efcde51a640 1 --2- 172.21.15.112:0/1821842739 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcd80fb810 0x7efcd80fbbe0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7efcc80048d0 tx=0x7efcc8009da0 comp rx=0 tx=0).stop 2024-09-16T00:46:43.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.684+0000 7efcde51a640 1 -- 172.21.15.112:0/1821842739 shutdown_connections 2024-09-16T00:46:43.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.684+0000 7efcde51a640 1 --2- 172.21.15.112:0/1821842739 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efcd80fcf80 0x7efcd80fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:43.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.684+0000 7efcde51a640 1 --2- 172.21.15.112:0/1821842739 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcd80fb810 0x7efcd80fbbe0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:43.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.684+0000 7efcde51a640 1 -- 172.21.15.112:0/1821842739 >> 172.21.15.112:0/1821842739 conn(0x7efcd80723b0 msgr2=0x7efcd80727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:43.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.684+0000 7efcde51a640 1 -- 172.21.15.112:0/1821842739 shutdown_connections 2024-09-16T00:46:43.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.684+0000 7efcde51a640 1 -- 172.21.15.112:0/1821842739 wait complete. 2024-09-16T00:46:43.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.685+0000 7efcde51a640 1 Processor -- start 2024-09-16T00:46:43.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.685+0000 7efcde51a640 1 -- start start 2024-09-16T00:46:43.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.685+0000 7efcde51a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcd80fb810 0x7efcd810e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:43.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.685+0000 7efcde51a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efcd80fcf80 0x7efcd810e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:43.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.685+0000 7efcde51a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd810ed70 con 0x7efcd80fb810 2024-09-16T00:46:43.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.685+0000 7efcde51a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd810eee0 con 0x7efcd80fcf80 2024-09-16T00:46:43.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.685+0000 7efcd7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcd80fb810 0x7efcd810e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:43.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.685+0000 7efcd7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcd80fb810 0x7efcd810e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59582/0 (socket says 172.21.15.112:59582) 2024-09-16T00:46:43.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.685+0000 7efcd7fff640 1 -- 172.21.15.112:0/3176382338 learned_addr learned my addr 172.21.15.112:0/3176382338 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:46:43.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.685+0000 7efccffff640 1 --2- 172.21.15.112:0/3176382338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efcd80fcf80 0x7efcd810e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:43.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.685+0000 7efcd7fff640 1 -- 172.21.15.112:0/3176382338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efcd80fcf80 msgr2=0x7efcd810e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:43.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.685+0000 7efcd7fff640 1 --2- 172.21.15.112:0/3176382338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efcd80fcf80 0x7efcd810e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:43.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.685+0000 7efcd7fff640 1 -- 172.21.15.112:0/3176382338 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcc8005040 con 0x7efcd80fb810 2024-09-16T00:46:43.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.686+0000 7efcd7fff640 1 --2- 172.21.15.112:0/3176382338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcd80fb810 0x7efcd810e2f0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7efcc8000c00 tx=0x7efcc8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:43.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.686+0000 7efcd57fa640 1 -- 172.21.15.112:0/3176382338 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efcc8038070 con 0x7efcd80fb810 2024-09-16T00:46:43.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.686+0000 7efcd57fa640 1 -- 172.21.15.112:0/3176382338 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7efcc802ade0 con 0x7efcd80fb810 2024-09-16T00:46:43.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.686+0000 7efcde51a640 1 -- 172.21.15.112:0/3176382338 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcd81171c0 con 0x7efcd80fb810 2024-09-16T00:46:43.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.686+0000 7efcde51a640 1 -- 172.21.15.112:0/3176382338 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcd8117680 con 0x7efcd80fb810 2024-09-16T00:46:43.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.686+0000 7efcd57fa640 1 -- 172.21.15.112:0/3176382338 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efcc802c7b0 con 0x7efcd80fb810 2024-09-16T00:46:43.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.687+0000 7efcde51a640 1 -- 172.21.15.112:0/3176382338 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcd80fcbc0 con 0x7efcd80fb810 2024-09-16T00:46:43.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.687+0000 7efcd57fa640 1 -- 172.21.15.112:0/3176382338 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7efcc8030020 con 0x7efcd80fb810 2024-09-16T00:46:43.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.688+0000 7efcd57fa640 1 --2- 172.21.15.112:0/3176382338 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7efcc406e530 0x7efcc40709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:43.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.688+0000 7efcd57fa640 1 -- 172.21.15.112:0/3176382338 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7efcc80ae6f0 con 0x7efcd80fb810 2024-09-16T00:46:43.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.688+0000 7efccffff640 1 --2- 172.21.15.112:0/3176382338 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7efcc406e530 0x7efcc40709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:43.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.688+0000 7efccffff640 1 --2- 172.21.15.112:0/3176382338 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7efcc406e530 0x7efcc40709e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7efcd80659e0 tx=0x7efcc000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:43.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.692+0000 7efcd57fa640 1 -- 172.21.15.112:0/3176382338 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7efcc80776e0 con 0x7efcd80fb810 2024-09-16T00:46:43.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.875+0000 7efcde51a640 1 -- 172.21.15.112:0/3176382338 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7efcd804b750 con 0x7efcc406e530 2024-09-16T00:46:43.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.876+0000 7efcd57fa640 1 -- 172.21.15.112:0/3176382338 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7efcd804b750 con 0x7efcc406e530 2024-09-16T00:46:43.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.878+0000 7efcde51a640 1 -- 172.21.15.112:0/3176382338 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7efcc406e530 msgr2=0x7efcc40709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:43.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.878+0000 7efcde51a640 1 --2- 172.21.15.112:0/3176382338 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7efcc406e530 0x7efcc40709e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7efcd80659e0 tx=0x7efcc000b040 comp rx=0 tx=0).stop 2024-09-16T00:46:43.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.878+0000 7efcde51a640 1 -- 172.21.15.112:0/3176382338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcd80fb810 msgr2=0x7efcd810e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:43.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.878+0000 7efcde51a640 1 --2- 172.21.15.112:0/3176382338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcd80fb810 0x7efcd810e2f0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7efcc8000c00 tx=0x7efcc8009da0 comp rx=0 tx=0).stop 2024-09-16T00:46:43.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.878+0000 7efcde51a640 1 -- 172.21.15.112:0/3176382338 shutdown_connections 2024-09-16T00:46:43.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.878+0000 7efcde51a640 1 --2- 172.21.15.112:0/3176382338 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7efcc406e530 0x7efcc40709e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:43.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.878+0000 7efcde51a640 1 --2- 172.21.15.112:0/3176382338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efcd80fcf80 0x7efcd810e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:43.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.878+0000 7efcde51a640 1 --2- 172.21.15.112:0/3176382338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcd80fb810 0x7efcd810e2f0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:43.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.878+0000 7efcde51a640 1 -- 172.21.15.112:0/3176382338 >> 172.21.15.112:0/3176382338 conn(0x7efcd80723b0 msgr2=0x7efcd80fa480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:43.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.878+0000 7efcde51a640 1 -- 172.21.15.112:0/3176382338 shutdown_connections 2024-09-16T00:46:43.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.878+0000 7efcde51a640 1 -- 172.21.15.112:0/3176382338 wait complete. 2024-09-16T00:46:43.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.956+0000 7fd70a6f0640 1 -- 172.21.15.112:0/775082559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7041014c0 msgr2=0x7fd704101890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:43.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.956+0000 7fd70a6f0640 1 --2- 172.21.15.112:0/775082559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7041014c0 0x7fd704101890 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fd6f40048d0 tx=0x7fd6f4009da0 comp rx=0 tx=0).stop 2024-09-16T00:46:43.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.956+0000 7fd70a6f0640 1 -- 172.21.15.112:0/775082559 shutdown_connections 2024-09-16T00:46:43.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.956+0000 7fd70a6f0640 1 --2- 172.21.15.112:0/775082559 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7040fb480 0x7fd7040fb8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:43.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.956+0000 7fd70a6f0640 1 --2- 172.21.15.112:0/775082559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7041014c0 0x7fd704101890 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:43.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.956+0000 7fd70a6f0640 1 -- 172.21.15.112:0/775082559 >> 172.21.15.112:0/775082559 conn(0x7fd7040f7000 msgr2=0x7fd7040f9410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:43.957 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:46:43 smithi112 ceph-mon[97750]: pgmap v569: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:46:43.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.956+0000 7fd70a6f0640 1 -- 172.21.15.112:0/775082559 shutdown_connections 2024-09-16T00:46:43.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.957+0000 7fd70a6f0640 1 -- 172.21.15.112:0/775082559 wait complete. 2024-09-16T00:46:43.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.957+0000 7fd70a6f0640 1 Processor -- start 2024-09-16T00:46:43.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.957+0000 7fd70a6f0640 1 -- start start 2024-09-16T00:46:43.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.958+0000 7fd70a6f0640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7040fb480 0x7fd70410e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:43.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.958+0000 7fd70a6f0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7041014c0 0x7fd70410e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:43.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.958+0000 7fd70a6f0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd70410ed20 con 0x7fd7041014c0 2024-09-16T00:46:43.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.958+0000 7fd70a6f0640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd70410ee90 con 0x7fd7040fb480 2024-09-16T00:46:43.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.958+0000 7fd7037fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7041014c0 0x7fd70410e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:43.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.958+0000 7fd7037fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7041014c0 0x7fd70410e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59594/0 (socket says 172.21.15.112:59594) 2024-09-16T00:46:43.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.958+0000 7fd7037fe640 1 -- 172.21.15.112:0/2073053005 learned_addr learned my addr 172.21.15.112:0/2073053005 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:46:43.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.958+0000 7fd703fff640 1 --2- 172.21.15.112:0/2073053005 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7040fb480 0x7fd70410e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:43.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.958+0000 7fd7037fe640 1 -- 172.21.15.112:0/2073053005 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7040fb480 msgr2=0x7fd70410e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:43.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.958+0000 7fd7037fe640 1 --2- 172.21.15.112:0/2073053005 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7040fb480 0x7fd70410e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:43.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.958+0000 7fd7037fe640 1 -- 172.21.15.112:0/2073053005 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f4005040 con 0x7fd7041014c0 2024-09-16T00:46:43.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.958+0000 7fd7037fe640 1 --2- 172.21.15.112:0/2073053005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7041014c0 0x7fd70410e6d0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fd6f8002a50 tx=0x7fd6f8004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:43.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.958+0000 7fd700ff9640 1 -- 172.21.15.112:0/2073053005 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6f8007c30 con 0x7fd7041014c0 2024-09-16T00:46:43.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.958+0000 7fd70a6f0640 1 -- 172.21.15.112:0/2073053005 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd704117200 con 0x7fd7041014c0 2024-09-16T00:46:43.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.958+0000 7fd700ff9640 1 -- 172.21.15.112:0/2073053005 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fd6f800f040 con 0x7fd7041014c0 2024-09-16T00:46:43.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.959+0000 7fd70a6f0640 1 -- 172.21.15.112:0/2073053005 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd704117710 con 0x7fd7041014c0 2024-09-16T00:46:43.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.959+0000 7fd700ff9640 1 -- 172.21.15.112:0/2073053005 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6f800da10 con 0x7fd7041014c0 2024-09-16T00:46:43.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.960+0000 7fd70a6f0640 1 -- 172.21.15.112:0/2073053005 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6f0005210 con 0x7fd7041014c0 2024-09-16T00:46:43.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.960+0000 7fd700ff9640 1 -- 172.21.15.112:0/2073053005 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fd6f800dcc0 con 0x7fd7041014c0 2024-09-16T00:46:43.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.961+0000 7fd700ff9640 1 --2- 172.21.15.112:0/2073053005 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd6ec06e530 0x7fd6ec0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:43.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.961+0000 7fd700ff9640 1 -- 172.21.15.112:0/2073053005 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7fd6f808d9b0 con 0x7fd7041014c0 2024-09-16T00:46:43.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.961+0000 7fd703fff640 1 --2- 172.21.15.112:0/2073053005 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd6ec06e530 0x7fd6ec0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:43.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.961+0000 7fd703fff640 1 --2- 172.21.15.112:0/2073053005 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd6ec06e530 0x7fd6ec0709e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd6f4009ac0 tx=0x7fd6f4035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:43.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:43.965+0000 7fd700ff9640 1 -- 172.21.15.112:0/2073053005 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fd6f80569d0 con 0x7fd7041014c0 2024-09-16T00:46:44.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.138+0000 7fd70a6f0640 1 -- 172.21.15.112:0/2073053005 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd6f0000bc0 con 0x7fd6ec06e530 2024-09-16T00:46:44.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.147+0000 7fd700ff9640 1 -- 172.21.15.112:0/2073053005 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7fd6f0000bc0 con 0x7fd6ec06e530 2024-09-16T00:46:44.147 INFO:teuthology.orchestra.run.smithi112.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T00:46:44.147 INFO:teuthology.orchestra.run.smithi112.stdout:alertmanager.smithi112 smithi112 *:9093,9094 running (19m) 12m ago 28m 24.7M - 0.25.0 c8568f914cd2 d85ea0562022 2024-09-16T00:46:44.147 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi112 smithi112 running (16m) 12m ago 28m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c e4fe58f9702a 2024-09-16T00:46:44.147 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi169 smithi169 running (16m) 12m ago 27m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c 3ba2c81d527f 2024-09-16T00:46:44.147 INFO:teuthology.orchestra.run.smithi112.stdout:grafana.smithi112 smithi112 *:3000 running (18m) 12m ago 28m 82.1M - 9.4.12 2bacad6d85d8 51f9d2013da2 2024-09-16T00:46:44.147 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.hxfwkh smithi112 running (12m) 12m ago 23m 14.2M - 19.1.1-326-g8921c8ef f1455bd0219c 1723069902d6 2024-09-16T00:46:44.147 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.ithwzj smithi112 running (13m) 12m ago 23m 12.2M - 19.1.1-326-g8921c8ef f1455bd0219c e7aad5b81db5 2024-09-16T00:46:44.147 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.gawwgw smithi169 running (12m) 12m ago 23m 15.7M - 19.1.1-326-g8921c8ef f1455bd0219c 469be459ee12 2024-09-16T00:46:44.148 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.lgeqrj smithi169 running (12m) 12m ago 23m 13.3M - 19.1.1-326-g8921c8ef f1455bd0219c 31f0fbd2395d 2024-09-16T00:46:44.148 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi112.xrkkbc smithi112 *:8443,9283,8765 running (19m) 12m ago 29m 504M - 19.1.1-326-g8921c8ef f1455bd0219c a981f7fd2810 2024-09-16T00:46:44.148 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi169.mtrobk smithi169 *:8443,9283,8765 running (18m) 12m ago 27m 442M - 19.1.1-326-g8921c8ef f1455bd0219c 4096e5369c2d 2024-09-16T00:46:44.148 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi112 smithi112 running (17m) 12m ago 30m 49.8M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 5aaec033c042 2024-09-16T00:46:44.148 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi169 smithi169 running (17m) 12m ago 27m 38.9M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 7bc609155f3f 2024-09-16T00:46:44.148 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi112 smithi112 *:9100 running (20m) 12m ago 28m 15.5M - 1.5.0 0da6a335fe13 6404d1d601e2 2024-09-16T00:46:44.148 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi169 smithi169 *:9100 running (19m) 12m ago 27m 15.4M - 1.5.0 0da6a335fe13 5505baf702a6 2024-09-16T00:46:44.148 INFO:teuthology.orchestra.run.smithi112.stdout:osd.0 smithi112 running (16m) 12m ago 26m 105M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 53efc9397017 2024-09-16T00:46:44.148 INFO:teuthology.orchestra.run.smithi112.stdout:osd.1 smithi112 running (15m) 12m ago 26m 98.9M 1557M 19.1.1-326-g8921c8ef f1455bd0219c fe6f1a4a2cf4 2024-09-16T00:46:44.148 INFO:teuthology.orchestra.run.smithi112.stdout:osd.2 smithi112 running (15m) 12m ago 26m 102M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 5f7863712057 2024-09-16T00:46:44.148 INFO:teuthology.orchestra.run.smithi112.stdout:osd.3 smithi169 running (14m) 12m ago 25m 92.1M 2581M 19.1.1-326-g8921c8ef f1455bd0219c ee07b5577810 2024-09-16T00:46:44.148 INFO:teuthology.orchestra.run.smithi112.stdout:osd.4 smithi169 running (13m) 12m ago 25m 103M 2581M 19.1.1-326-g8921c8ef f1455bd0219c 769f026a54eb 2024-09-16T00:46:44.148 INFO:teuthology.orchestra.run.smithi112.stdout:osd.5 smithi169 running (13m) 12m ago 24m 125M 2581M 19.1.1-326-g8921c8ef f1455bd0219c a6007c13cf78 2024-09-16T00:46:44.148 INFO:teuthology.orchestra.run.smithi112.stdout:prometheus.smithi112 smithi112 *:9095 running (18m) 12m ago 28m 64.1M - 2.43.0 a07b618ecd1d 7a1d42e60d2b 2024-09-16T00:46:44.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.149+0000 7fd70a6f0640 1 -- 172.21.15.112:0/2073053005 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd6ec06e530 msgr2=0x7fd6ec0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:44.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.149+0000 7fd70a6f0640 1 --2- 172.21.15.112:0/2073053005 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd6ec06e530 0x7fd6ec0709e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd6f4009ac0 tx=0x7fd6f4035040 comp rx=0 tx=0).stop 2024-09-16T00:46:44.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.149+0000 7fd70a6f0640 1 -- 172.21.15.112:0/2073053005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7041014c0 msgr2=0x7fd70410e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:44.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.149+0000 7fd70a6f0640 1 --2- 172.21.15.112:0/2073053005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7041014c0 0x7fd70410e6d0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fd6f8002a50 tx=0x7fd6f8004600 comp rx=0 tx=0).stop 2024-09-16T00:46:44.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.149+0000 7fd70a6f0640 1 -- 172.21.15.112:0/2073053005 shutdown_connections 2024-09-16T00:46:44.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.149+0000 7fd70a6f0640 1 --2- 172.21.15.112:0/2073053005 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd6ec06e530 0x7fd6ec0709e0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:44.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.149+0000 7fd70a6f0640 1 --2- 172.21.15.112:0/2073053005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7041014c0 0x7fd70410e6d0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:44.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.149+0000 7fd70a6f0640 1 --2- 172.21.15.112:0/2073053005 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7040fb480 0x7fd70410e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:44.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.149+0000 7fd70a6f0640 1 -- 172.21.15.112:0/2073053005 >> 172.21.15.112:0/2073053005 conn(0x7fd7040f7000 msgr2=0x7fd7040f88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:44.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.149+0000 7fd70a6f0640 1 -- 172.21.15.112:0/2073053005 shutdown_connections 2024-09-16T00:46:44.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.149+0000 7fd70a6f0640 1 -- 172.21.15.112:0/2073053005 wait complete. 2024-09-16T00:46:44.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.226+0000 7f7d40715640 1 -- 172.21.15.112:0/2737902253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d38102550 msgr2=0x7f7d38102920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:44.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.226+0000 7f7d40715640 1 --2- 172.21.15.112:0/2737902253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d38102550 0x7f7d38102920 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f7d340048d0 tx=0x7f7d34009da0 comp rx=0 tx=0).stop 2024-09-16T00:46:44.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.226+0000 7f7d40715640 1 -- 172.21.15.112:0/2737902253 shutdown_connections 2024-09-16T00:46:44.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.226+0000 7f7d40715640 1 --2- 172.21.15.112:0/2737902253 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d38102ef0 0x7f7d380fc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:44.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.226+0000 7f7d40715640 1 --2- 172.21.15.112:0/2737902253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d38102550 0x7f7d38102920 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:44.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.226+0000 7f7d40715640 1 -- 172.21.15.112:0/2737902253 >> 172.21.15.112:0/2737902253 conn(0x7f7d380f80f0 msgr2=0x7f7d380fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:44.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.226+0000 7f7d40715640 1 -- 172.21.15.112:0/2737902253 shutdown_connections 2024-09-16T00:46:44.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.226+0000 7f7d40715640 1 -- 172.21.15.112:0/2737902253 wait complete. 2024-09-16T00:46:44.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.227+0000 7f7d40715640 1 Processor -- start 2024-09-16T00:46:44.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.227+0000 7f7d40715640 1 -- start start 2024-09-16T00:46:44.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.227+0000 7f7d40715640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d38102550 0x7f7d38113540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:44.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.227+0000 7f7d40715640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d38102ef0 0x7f7d38113a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:44.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.227+0000 7f7d40715640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d381140d0 con 0x7f7d38102550 2024-09-16T00:46:44.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.227+0000 7f7d40715640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3810d4e0 con 0x7f7d38102ef0 2024-09-16T00:46:44.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.227+0000 7f7d3e4b1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d38102550 0x7f7d38113540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:44.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.227+0000 7f7d3e4b1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d38102550 0x7f7d38113540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59618/0 (socket says 172.21.15.112:59618) 2024-09-16T00:46:44.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.227+0000 7f7d3e4b1640 1 -- 172.21.15.112:0/940808019 learned_addr learned my addr 172.21.15.112:0/940808019 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:46:44.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.227+0000 7f7d3dcb0640 1 --2- 172.21.15.112:0/940808019 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d38102ef0 0x7f7d38113a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:44.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.227+0000 7f7d3e4b1640 1 -- 172.21.15.112:0/940808019 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d38102ef0 msgr2=0x7f7d38113a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:44.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.227+0000 7f7d3e4b1640 1 --2- 172.21.15.112:0/940808019 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d38102ef0 0x7f7d38113a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:44.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.227+0000 7f7d3e4b1640 1 -- 172.21.15.112:0/940808019 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d34005040 con 0x7f7d38102550 2024-09-16T00:46:44.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.228+0000 7f7d3e4b1640 1 --2- 172.21.15.112:0/940808019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d38102550 0x7f7d38113540 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f7d34000c00 tx=0x7f7d34009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:44.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.228+0000 7f7d2affd640 1 -- 172.21.15.112:0/940808019 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d34038070 con 0x7f7d38102550 2024-09-16T00:46:44.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.228+0000 7f7d2affd640 1 -- 172.21.15.112:0/940808019 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f7d3402ade0 con 0x7f7d38102550 2024-09-16T00:46:44.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.228+0000 7f7d2affd640 1 -- 172.21.15.112:0/940808019 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d3402c7e0 con 0x7f7d38102550 2024-09-16T00:46:44.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.228+0000 7f7d40715640 1 -- 172.21.15.112:0/940808019 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d3810d760 con 0x7f7d38102550 2024-09-16T00:46:44.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.228+0000 7f7d40715640 1 -- 172.21.15.112:0/940808019 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d3810dc50 con 0x7f7d38102550 2024-09-16T00:46:44.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.229+0000 7f7d40715640 1 -- 172.21.15.112:0/940808019 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d380fdcd0 con 0x7f7d38102550 2024-09-16T00:46:44.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.230+0000 7f7d2affd640 1 -- 172.21.15.112:0/940808019 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f7d34030020 con 0x7f7d38102550 2024-09-16T00:46:44.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.230+0000 7f7d2affd640 1 --2- 172.21.15.112:0/940808019 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f7d2406e530 0x7f7d240709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:44.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.230+0000 7f7d2affd640 1 -- 172.21.15.112:0/940808019 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f7d340adf70 con 0x7f7d38102550 2024-09-16T00:46:44.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.230+0000 7f7d3dcb0640 1 --2- 172.21.15.112:0/940808019 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f7d2406e530 0x7f7d240709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:44.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.231+0000 7f7d3dcb0640 1 --2- 172.21.15.112:0/940808019 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f7d2406e530 0x7f7d240709e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7d3810e7c0 tx=0x7f7d2c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:44.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.235+0000 7f7d2affd640 1 -- 172.21.15.112:0/940808019 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f7d340e31b0 con 0x7f7d38102550 2024-09-16T00:46:44.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:46:43 smithi169 ceph-mon[77626]: pgmap v569: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:46:44.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.468+0000 7f7d40715640 1 -- 172.21.15.112:0/940808019 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7d3804b720 con 0x7f7d38102550 2024-09-16T00:46:44.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.468+0000 7f7d2affd640 1 -- 172.21.15.112:0/940808019 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f7d34076f90 con 0x7f7d38102550 2024-09-16T00:46:44.468 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:46:44.469 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": { 2024-09-16T00:46:44.469 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:46:44.469 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:46:44.469 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": { 2024-09-16T00:46:44.469 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:46:44.469 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:46:44.469 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": { 2024-09-16T00:46:44.469 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 6 2024-09-16T00:46:44.469 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:46:44.470 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": { 2024-09-16T00:46:44.470 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 4 2024-09-16T00:46:44.470 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:46:44.470 INFO:teuthology.orchestra.run.smithi112.stdout: "overall": { 2024-09-16T00:46:44.470 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 14 2024-09-16T00:46:44.470 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-16T00:46:44.470 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:46:44.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.471+0000 7f7d40715640 1 -- 172.21.15.112:0/940808019 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f7d2406e530 msgr2=0x7f7d240709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:44.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.471+0000 7f7d40715640 1 --2- 172.21.15.112:0/940808019 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f7d2406e530 0x7f7d240709e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7d3810e7c0 tx=0x7f7d2c00b040 comp rx=0 tx=0).stop 2024-09-16T00:46:44.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.471+0000 7f7d40715640 1 -- 172.21.15.112:0/940808019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d38102550 msgr2=0x7f7d38113540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:44.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.471+0000 7f7d40715640 1 --2- 172.21.15.112:0/940808019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d38102550 0x7f7d38113540 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f7d34000c00 tx=0x7f7d34009da0 comp rx=0 tx=0).stop 2024-09-16T00:46:44.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.471+0000 7f7d40715640 1 -- 172.21.15.112:0/940808019 shutdown_connections 2024-09-16T00:46:44.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.471+0000 7f7d40715640 1 --2- 172.21.15.112:0/940808019 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f7d2406e530 0x7f7d240709e0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:44.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.471+0000 7f7d40715640 1 --2- 172.21.15.112:0/940808019 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d38102ef0 0x7f7d38113a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:44.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.471+0000 7f7d40715640 1 --2- 172.21.15.112:0/940808019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d38102550 0x7f7d38113540 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:44.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.471+0000 7f7d40715640 1 -- 172.21.15.112:0/940808019 >> 172.21.15.112:0/940808019 conn(0x7f7d380f80f0 msgr2=0x7f7d380f9160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:44.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.471+0000 7f7d40715640 1 -- 172.21.15.112:0/940808019 shutdown_connections 2024-09-16T00:46:44.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.471+0000 7f7d40715640 1 -- 172.21.15.112:0/940808019 wait complete. 2024-09-16T00:46:44.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.548+0000 7fb0de11e640 1 -- 172.21.15.112:0/300130747 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0d80fb480 msgr2=0x7fb0d80fb8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:44.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.548+0000 7fb0de11e640 1 --2- 172.21.15.112:0/300130747 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0d80fb480 0x7fb0d80fb8f0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fb0c00048d0 tx=0x7fb0c0009da0 comp rx=0 tx=0).stop 2024-09-16T00:46:44.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.548+0000 7fb0de11e640 1 -- 172.21.15.112:0/300130747 shutdown_connections 2024-09-16T00:46:44.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.548+0000 7fb0de11e640 1 --2- 172.21.15.112:0/300130747 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0d80fb480 0x7fb0d80fb8f0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:44.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.548+0000 7fb0de11e640 1 --2- 172.21.15.112:0/300130747 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0d81014c0 0x7fb0d8101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:44.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.548+0000 7fb0de11e640 1 -- 172.21.15.112:0/300130747 >> 172.21.15.112:0/300130747 conn(0x7fb0d80f7000 msgr2=0x7fb0d80f9410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:44.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.548+0000 7fb0de11e640 1 -- 172.21.15.112:0/300130747 shutdown_connections 2024-09-16T00:46:44.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.548+0000 7fb0de11e640 1 -- 172.21.15.112:0/300130747 wait complete. 2024-09-16T00:46:44.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.548+0000 7fb0de11e640 1 Processor -- start 2024-09-16T00:46:44.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.549+0000 7fb0de11e640 1 -- start start 2024-09-16T00:46:44.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.549+0000 7fb0de11e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0d80fb480 0x7fb0d8113240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:44.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.549+0000 7fb0de11e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0d81014c0 0x7fb0d8113780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:44.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.549+0000 7fb0de11e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d8113e10 con 0x7fb0d80fb480 2024-09-16T00:46:44.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.549+0000 7fb0de11e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d810d1c0 con 0x7fb0d81014c0 2024-09-16T00:46:44.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.549+0000 7fb0d77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0d80fb480 0x7fb0d8113240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:44.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.549+0000 7fb0d77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0d80fb480 0x7fb0d8113240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59626/0 (socket says 172.21.15.112:59626) 2024-09-16T00:46:44.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.549+0000 7fb0d77fe640 1 -- 172.21.15.112:0/593479363 learned_addr learned my addr 172.21.15.112:0/593479363 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:46:44.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.549+0000 7fb0d6ffd640 1 --2- 172.21.15.112:0/593479363 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0d81014c0 0x7fb0d8113780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:44.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.549+0000 7fb0d77fe640 1 -- 172.21.15.112:0/593479363 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0d81014c0 msgr2=0x7fb0d8113780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:44.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.549+0000 7fb0d77fe640 1 --2- 172.21.15.112:0/593479363 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0d81014c0 0x7fb0d8113780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:44.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.549+0000 7fb0d77fe640 1 -- 172.21.15.112:0/593479363 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0c0005040 con 0x7fb0d80fb480 2024-09-16T00:46:44.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.550+0000 7fb0d77fe640 1 --2- 172.21.15.112:0/593479363 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0d80fb480 0x7fb0d8113240 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fb0c800b790 tx=0x7fb0c80091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:44.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.550+0000 7fb0cffff640 1 -- 172.21.15.112:0/593479363 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb0c800c700 con 0x7fb0d80fb480 2024-09-16T00:46:44.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.550+0000 7fb0cffff640 1 -- 172.21.15.112:0/593479363 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fb0c800cd00 con 0x7fb0d80fb480 2024-09-16T00:46:44.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.550+0000 7fb0cffff640 1 -- 172.21.15.112:0/593479363 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb0c8015e80 con 0x7fb0d80fb480 2024-09-16T00:46:44.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.550+0000 7fb0de11e640 1 -- 172.21.15.112:0/593479363 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0d810d4a0 con 0x7fb0d80fb480 2024-09-16T00:46:44.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.550+0000 7fb0de11e640 1 -- 172.21.15.112:0/593479363 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0d810d940 con 0x7fb0d80fb480 2024-09-16T00:46:44.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.551+0000 7fb0de11e640 1 -- 172.21.15.112:0/593479363 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0d80fcbc0 con 0x7fb0d80fb480 2024-09-16T00:46:44.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.552+0000 7fb0cffff640 1 -- 172.21.15.112:0/593479363 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fb0c8012070 con 0x7fb0d80fb480 2024-09-16T00:46:44.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.552+0000 7fb0cffff640 1 --2- 172.21.15.112:0/593479363 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fb0c406e530 0x7fb0c40709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:44.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.552+0000 7fb0d6ffd640 1 --2- 172.21.15.112:0/593479363 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fb0c406e530 0x7fb0c40709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:44.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.552+0000 7fb0cffff640 1 -- 172.21.15.112:0/593479363 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7fb0c808dc10 con 0x7fb0d80fb480 2024-09-16T00:46:44.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.553+0000 7fb0d6ffd640 1 --2- 172.21.15.112:0/593479363 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fb0c406e530 0x7fb0c40709e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0009ac0 tx=0x7fb0c0035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:44.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.557+0000 7fb0cffff640 1 -- 172.21.15.112:0/593479363 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fb0c8056b80 con 0x7fb0d80fb480 2024-09-16T00:46:44.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.756+0000 7fb0de11e640 1 -- 172.21.15.112:0/593479363 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb0d804b720 con 0x7fb0d80fb480 2024-09-16T00:46:44.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.757+0000 7fb0cffff640 1 -- 172.21.15.112:0/593479363 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1914 (secure 0 0 0) 0x7fb0c805a240 con 0x7fb0d80fb480 2024-09-16T00:46:44.757 INFO:teuthology.orchestra.run.smithi112.stdout:e34 2024-09-16T00:46:44.757 INFO:teuthology.orchestra.run.smithi112.stdout:btime 2024-09-16T00:34:26:619254+0000 2024-09-16T00:46:44.757 INFO:teuthology.orchestra.run.smithi112.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T00:46:44.757 INFO:teuthology.orchestra.run.smithi112.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:46:44.757 INFO:teuthology.orchestra.run.smithi112.stdout:legacy client fscid: 1 2024-09-16T00:46:44.757 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:46:44.758 INFO:teuthology.orchestra.run.smithi112.stdout:Filesystem 'cephfs' (1) 2024-09-16T00:46:44.758 INFO:teuthology.orchestra.run.smithi112.stdout:fs_name cephfs 2024-09-16T00:46:44.758 INFO:teuthology.orchestra.run.smithi112.stdout:epoch 25 2024-09-16T00:46:44.758 INFO:teuthology.orchestra.run.smithi112.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-16T00:46:44.758 INFO:teuthology.orchestra.run.smithi112.stdout:created 2024-09-16T00:23:22.328199+0000 2024-09-16T00:46:44.758 INFO:teuthology.orchestra.run.smithi112.stdout:modified 2024-09-16T00:33:29.012991+0000 2024-09-16T00:46:44.758 INFO:teuthology.orchestra.run.smithi112.stdout:tableserver 0 2024-09-16T00:46:44.758 INFO:teuthology.orchestra.run.smithi112.stdout:root 0 2024-09-16T00:46:44.758 INFO:teuthology.orchestra.run.smithi112.stdout:session_timeout 60 2024-09-16T00:46:44.758 INFO:teuthology.orchestra.run.smithi112.stdout:session_autoclose 300 2024-09-16T00:46:44.758 INFO:teuthology.orchestra.run.smithi112.stdout:max_file_size 1099511627776 2024-09-16T00:46:44.758 INFO:teuthology.orchestra.run.smithi112.stdout:max_xattr_size 65536 2024-09-16T00:46:44.758 INFO:teuthology.orchestra.run.smithi112.stdout:required_client_features {} 2024-09-16T00:46:44.759 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure 0 2024-09-16T00:46:44.759 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure_osd_epoch 82 2024-09-16T00:46:44.759 INFO:teuthology.orchestra.run.smithi112.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:46:44.759 INFO:teuthology.orchestra.run.smithi112.stdout:max_mds 2 2024-09-16T00:46:44.759 INFO:teuthology.orchestra.run.smithi112.stdout:in 0,1 2024-09-16T00:46:44.759 INFO:teuthology.orchestra.run.smithi112.stdout:up {} 2024-09-16T00:46:44.759 INFO:teuthology.orchestra.run.smithi112.stdout:failed 0,1 2024-09-16T00:46:44.759 INFO:teuthology.orchestra.run.smithi112.stdout:damaged 2024-09-16T00:46:44.759 INFO:teuthology.orchestra.run.smithi112.stdout:stopped 2024-09-16T00:46:44.759 INFO:teuthology.orchestra.run.smithi112.stdout:data_pools [3] 2024-09-16T00:46:44.759 INFO:teuthology.orchestra.run.smithi112.stdout:metadata_pool 2 2024-09-16T00:46:44.759 INFO:teuthology.orchestra.run.smithi112.stdout:inline_data enabled 2024-09-16T00:46:44.759 INFO:teuthology.orchestra.run.smithi112.stdout:balancer 2024-09-16T00:46:44.759 INFO:teuthology.orchestra.run.smithi112.stdout:bal_rank_mask -1 2024-09-16T00:46:44.760 INFO:teuthology.orchestra.run.smithi112.stdout:standby_count_wanted 1 2024-09-16T00:46:44.760 INFO:teuthology.orchestra.run.smithi112.stdout:qdb_cluster leader: 0 members: 2024-09-16T00:46:44.760 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:46:44.760 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:46:44.760 INFO:teuthology.orchestra.run.smithi112.stdout:Standby daemons: 2024-09-16T00:46:44.760 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:46:44.760 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.ithwzj{-1:34392} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.112:6826/476229221,v1:172.21.15.112:6827/476229221] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:46:44.760 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.hxfwkh{-1:34404} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.112:6828/2356899876,v1:172.21.15.112:6829/2356899876] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:46:44.760 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.gawwgw{-1:44343} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.169:6824/1635329957,v1:172.21.15.169:6825/1635329957] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:46:44.760 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.lgeqrj{-1:44347} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.169:6826/184810031,v1:172.21.15.169:6827/184810031] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:46:44.761 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 34 2024-09-16T00:46:44.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.759+0000 7fb0de11e640 1 -- 172.21.15.112:0/593479363 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fb0c406e530 msgr2=0x7fb0c40709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:44.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.759+0000 7fb0de11e640 1 --2- 172.21.15.112:0/593479363 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fb0c406e530 0x7fb0c40709e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0009ac0 tx=0x7fb0c0035040 comp rx=0 tx=0).stop 2024-09-16T00:46:44.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.759+0000 7fb0de11e640 1 -- 172.21.15.112:0/593479363 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0d80fb480 msgr2=0x7fb0d8113240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:44.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.759+0000 7fb0de11e640 1 --2- 172.21.15.112:0/593479363 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0d80fb480 0x7fb0d8113240 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fb0c800b790 tx=0x7fb0c80091d0 comp rx=0 tx=0).stop 2024-09-16T00:46:44.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.759+0000 7fb0de11e640 1 -- 172.21.15.112:0/593479363 shutdown_connections 2024-09-16T00:46:44.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.759+0000 7fb0de11e640 1 --2- 172.21.15.112:0/593479363 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fb0c406e530 0x7fb0c40709e0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:44.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.759+0000 7fb0de11e640 1 --2- 172.21.15.112:0/593479363 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0d81014c0 0x7fb0d8113780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:44.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.759+0000 7fb0de11e640 1 --2- 172.21.15.112:0/593479363 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0d80fb480 0x7fb0d8113240 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:44.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.759+0000 7fb0de11e640 1 -- 172.21.15.112:0/593479363 >> 172.21.15.112:0/593479363 conn(0x7fb0d80f7000 msgr2=0x7fb0d80f8950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:44.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.759+0000 7fb0de11e640 1 -- 172.21.15.112:0/593479363 shutdown_connections 2024-09-16T00:46:44.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.759+0000 7fb0de11e640 1 -- 172.21.15.112:0/593479363 wait complete. 2024-09-16T00:46:44.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.839+0000 7f7208ba4640 1 -- 172.21.15.112:0/3067839821 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72040fb480 msgr2=0x7f72040fb8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:44.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.839+0000 7f7208ba4640 1 --2- 172.21.15.112:0/3067839821 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72040fb480 0x7f72040fb8f0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f71ec0048d0 tx=0x7f71ec009da0 comp rx=0 tx=0).stop 2024-09-16T00:46:44.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.839+0000 7f7208ba4640 1 -- 172.21.15.112:0/3067839821 shutdown_connections 2024-09-16T00:46:44.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.839+0000 7f7208ba4640 1 --2- 172.21.15.112:0/3067839821 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72040fb480 0x7f72040fb8f0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:44.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.839+0000 7f7208ba4640 1 --2- 172.21.15.112:0/3067839821 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72041014c0 0x7f7204101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:44.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.839+0000 7f7208ba4640 1 -- 172.21.15.112:0/3067839821 >> 172.21.15.112:0/3067839821 conn(0x7f72040f7000 msgr2=0x7f72040f9410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:44.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.839+0000 7f7208ba4640 1 -- 172.21.15.112:0/3067839821 shutdown_connections 2024-09-16T00:46:44.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.839+0000 7f7208ba4640 1 -- 172.21.15.112:0/3067839821 wait complete. 2024-09-16T00:46:44.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.840+0000 7f7208ba4640 1 Processor -- start 2024-09-16T00:46:44.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.840+0000 7f7208ba4640 1 -- start start 2024-09-16T00:46:44.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.840+0000 7f7208ba4640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72040fb480 0x7f72041131c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:44.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.840+0000 7f7208ba4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72041014c0 0x7f7204113700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:44.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.840+0000 7f7208ba4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7204113d00 con 0x7f72041014c0 2024-09-16T00:46:44.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.840+0000 7f7208ba4640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f720410d140 con 0x7f72040fb480 2024-09-16T00:46:44.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.840+0000 7f71fbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72041014c0 0x7f7204113700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:44.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.840+0000 7f71fbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72041014c0 0x7f7204113700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59642/0 (socket says 172.21.15.112:59642) 2024-09-16T00:46:44.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.840+0000 7f71fbfff640 1 -- 172.21.15.112:0/167073195 learned_addr learned my addr 172.21.15.112:0/167073195 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:46:44.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.840+0000 7f720259c640 1 --2- 172.21.15.112:0/167073195 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72040fb480 0x7f72041131c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:44.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.840+0000 7f71fbfff640 1 -- 172.21.15.112:0/167073195 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72040fb480 msgr2=0x7f72041131c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:44.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.840+0000 7f71fbfff640 1 --2- 172.21.15.112:0/167073195 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72040fb480 0x7f72041131c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:44.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.840+0000 7f71fbfff640 1 -- 172.21.15.112:0/167073195 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71ec005040 con 0x7f72041014c0 2024-09-16T00:46:44.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.840+0000 7f71fbfff640 1 --2- 172.21.15.112:0/167073195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72041014c0 0x7f7204113700 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f71ec0043d0 tx=0x7f71ec009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:44.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.841+0000 7f71faffd640 1 -- 172.21.15.112:0/167073195 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f71ec038070 con 0x7f72041014c0 2024-09-16T00:46:44.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.841+0000 7f7208ba4640 1 -- 172.21.15.112:0/167073195 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f720410d3c0 con 0x7f72041014c0 2024-09-16T00:46:44.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.841+0000 7f71faffd640 1 -- 172.21.15.112:0/167073195 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f71ec0096e0 con 0x7f72041014c0 2024-09-16T00:46:44.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.841+0000 7f71faffd640 1 -- 172.21.15.112:0/167073195 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f71ec03d420 con 0x7f72041014c0 2024-09-16T00:46:44.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.841+0000 7f7208ba4640 1 -- 172.21.15.112:0/167073195 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f720410d8e0 con 0x7f72041014c0 2024-09-16T00:46:44.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.842+0000 7f7208ba4640 1 -- 172.21.15.112:0/167073195 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72040fcbc0 con 0x7f72041014c0 2024-09-16T00:46:44.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.842+0000 7f71faffd640 1 -- 172.21.15.112:0/167073195 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f71ec030020 con 0x7f72041014c0 2024-09-16T00:46:44.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.843+0000 7f71faffd640 1 --2- 172.21.15.112:0/167073195 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f71f006e530 0x7f71f00709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:44.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.843+0000 7f720259c640 1 --2- 172.21.15.112:0/167073195 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f71f006e530 0x7f71f00709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:44.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.843+0000 7f71faffd640 1 -- 172.21.15.112:0/167073195 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f71ec0ae7c0 con 0x7f72041014c0 2024-09-16T00:46:44.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.843+0000 7f720259c640 1 --2- 172.21.15.112:0/167073195 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f71f006e530 0x7f71f00709e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f71f4009fd0 tx=0x7f71f400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:44.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:44.847+0000 7f71faffd640 1 -- 172.21.15.112:0/167073195 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f71ec077730 con 0x7f72041014c0 2024-09-16T00:46:45.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:45.029+0000 7f7208ba4640 1 -- 172.21.15.112:0/167073195 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7204108440 con 0x7f71f006e530 2024-09-16T00:46:45.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:45.030+0000 7f71faffd640 1 -- 172.21.15.112:0/167073195 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f7204108440 con 0x7f71f006e530 2024-09-16T00:46:45.030 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:46:45.030 INFO:teuthology.orchestra.run.smithi112.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80", 2024-09-16T00:46:45.030 INFO:teuthology.orchestra.run.smithi112.stdout: "in_progress": true, 2024-09-16T00:46:45.031 INFO:teuthology.orchestra.run.smithi112.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T00:46:45.031 INFO:teuthology.orchestra.run.smithi112.stdout: "services_complete": [ 2024-09-16T00:46:45.031 INFO:teuthology.orchestra.run.smithi112.stdout: "mds", 2024-09-16T00:46:45.031 INFO:teuthology.orchestra.run.smithi112.stdout: "osd", 2024-09-16T00:46:45.031 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr", 2024-09-16T00:46:45.031 INFO:teuthology.orchestra.run.smithi112.stdout: "mon", 2024-09-16T00:46:45.031 INFO:teuthology.orchestra.run.smithi112.stdout: "crash" 2024-09-16T00:46:45.031 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-16T00:46:45.031 INFO:teuthology.orchestra.run.smithi112.stdout: "progress": "16/21 daemons upgraded", 2024-09-16T00:46:45.031 INFO:teuthology.orchestra.run.smithi112.stdout: "message": "Currently upgrading mds daemons", 2024-09-16T00:46:45.031 INFO:teuthology.orchestra.run.smithi112.stdout: "is_paused": false 2024-09-16T00:46:45.031 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:46:45.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:45.032+0000 7f7208ba4640 1 -- 172.21.15.112:0/167073195 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f71f006e530 msgr2=0x7f71f00709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:45.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:45.032+0000 7f7208ba4640 1 --2- 172.21.15.112:0/167073195 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f71f006e530 0x7f71f00709e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f71f4009fd0 tx=0x7f71f400b040 comp rx=0 tx=0).stop 2024-09-16T00:46:45.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:45.032+0000 7f7208ba4640 1 -- 172.21.15.112:0/167073195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72041014c0 msgr2=0x7f7204113700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:45.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:45.032+0000 7f7208ba4640 1 --2- 172.21.15.112:0/167073195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72041014c0 0x7f7204113700 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f71ec0043d0 tx=0x7f71ec009da0 comp rx=0 tx=0).stop 2024-09-16T00:46:45.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:45.032+0000 7f7208ba4640 1 -- 172.21.15.112:0/167073195 shutdown_connections 2024-09-16T00:46:45.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:45.032+0000 7f7208ba4640 1 --2- 172.21.15.112:0/167073195 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f71f006e530 0x7f71f00709e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:45.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:45.032+0000 7f7208ba4640 1 --2- 172.21.15.112:0/167073195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72041014c0 0x7f7204113700 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:45.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:45.032+0000 7f7208ba4640 1 --2- 172.21.15.112:0/167073195 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72040fb480 0x7f72041131c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:45.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:45.032+0000 7f7208ba4640 1 -- 172.21.15.112:0/167073195 >> 172.21.15.112:0/167073195 conn(0x7f72040f7000 msgr2=0x7f72040f73d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:45.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:45.033+0000 7f7208ba4640 1 -- 172.21.15.112:0/167073195 shutdown_connections 2024-09-16T00:46:45.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:45.033+0000 7f7208ba4640 1 -- 172.21.15.112:0/167073195 wait complete. 2024-09-16T00:46:45.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:46:44 smithi112 ceph-mon[97750]: from='client.35080 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:46:45.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:46:44 smithi112 ceph-mon[97750]: from='client.35084 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:46:45.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:46:44 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/940808019' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:46:45.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:46:44 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/593479363' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:46:45.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:45.111+0000 7f958bfd9640 1 -- 172.21.15.112:0/1182405872 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9584102530 msgr2=0x7f9584102900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:45.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:45.111+0000 7f958bfd9640 1 --2- 172.21.15.112:0/1182405872 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9584102530 0x7f9584102900 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f95780048d0 tx=0x7f9578009da0 comp rx=0 tx=0).stop 2024-09-16T00:46:45.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:45.112+0000 7f958bfd9640 1 -- 172.21.15.112:0/1182405872 shutdown_connections 2024-09-16T00:46:45.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:45.112+0000 7f958bfd9640 1 --2- 172.21.15.112:0/1182405872 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9584102ed0 0x7f95840fc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:45.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:45.112+0000 7f958bfd9640 1 --2- 172.21.15.112:0/1182405872 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9584102530 0x7f9584102900 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:45.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:45.112+0000 7f958bfd9640 1 -- 172.21.15.112:0/1182405872 >> 172.21.15.112:0/1182405872 conn(0x7f95840f80f0 msgr2=0x7f95840fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:45.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:45.112+0000 7f958bfd9640 1 -- 172.21.15.112:0/1182405872 shutdown_connections 2024-09-16T00:46:45.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:45.112+0000 7f958bfd9640 1 -- 172.21.15.112:0/1182405872 wait complete. 2024-09-16T00:46:45.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:45.112+0000 7f958bfd9640 1 Processor -- start 2024-09-16T00:46:45.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:45.112+0000 7f958bfd9640 1 -- start start 2024-09-16T00:46:45.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:45.112+0000 7f958bfd9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9584102530 0x7f958406fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:45.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:45.112+0000 7f958bfd9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9584102ed0 0x7f958406ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:45.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:45.112+0000 7f958bfd9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9584070490 con 0x7f9584102530 2024-09-16T00:46:45.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:45.112+0000 7f958bfd9640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9584070600 con 0x7f9584102ed0 2024-09-16T00:46:45.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:45.113+0000 7f9589d75640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9584102530 0x7f958406fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:45.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:45.113+0000 7f9589d75640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9584102530 0x7f958406fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59646/0 (socket says 172.21.15.112:59646) 2024-09-16T00:46:45.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:45.113+0000 7f9589d75640 1 -- 172.21.15.112:0/1167757337 learned_addr learned my addr 172.21.15.112:0/1167757337 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:46:45.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:45.113+0000 7f9589574640 1 --2- 172.21.15.112:0/1167757337 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9584102ed0 0x7f958406ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:45.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:45.113+0000 7f9589d75640 1 -- 172.21.15.112:0/1167757337 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9584102ed0 msgr2=0x7f958406ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:45.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:45.113+0000 7f9589d75640 1 --2- 172.21.15.112:0/1167757337 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9584102ed0 0x7f958406ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:45.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:45.113+0000 7f9589d75640 1 -- 172.21.15.112:0/1167757337 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9578005040 con 0x7f9584102530 2024-09-16T00:46:45.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:45.113+0000 7f9589d75640 1 --2- 172.21.15.112:0/1167757337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9584102530 0x7f958406fa10 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f95780043d0 tx=0x7f9578009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:45.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:45.113+0000 7f95767fc640 1 -- 172.21.15.112:0/1167757337 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9578038070 con 0x7f9584102530 2024-09-16T00:46:45.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:45.113+0000 7f95767fc640 1 -- 172.21.15.112:0/1167757337 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f95780096e0 con 0x7f9584102530 2024-09-16T00:46:45.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:45.113+0000 7f95767fc640 1 -- 172.21.15.112:0/1167757337 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f957803d420 con 0x7f9584102530 2024-09-16T00:46:45.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:45.113+0000 7f958bfd9640 1 -- 172.21.15.112:0/1167757337 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9584070830 con 0x7f9584102530 2024-09-16T00:46:45.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:45.113+0000 7f958bfd9640 1 -- 172.21.15.112:0/1167757337 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9584119910 con 0x7f9584102530 2024-09-16T00:46:45.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:45.114+0000 7f958bfd9640 1 -- 172.21.15.112:0/1167757337 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95840fdcb0 con 0x7f9584102530 2024-09-16T00:46:45.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:45.115+0000 7f95767fc640 1 -- 172.21.15.112:0/1167757337 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f9578030020 con 0x7f9584102530 2024-09-16T00:46:45.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:45.115+0000 7f95767fc640 1 --2- 172.21.15.112:0/1167757337 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f957006e530 0x7f95700709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:46:45.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:45.115+0000 7f95767fc640 1 -- 172.21.15.112:0/1167757337 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f95780ae6e0 con 0x7f9584102530 2024-09-16T00:46:45.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:45.115+0000 7f9589574640 1 --2- 172.21.15.112:0/1167757337 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f957006e530 0x7f95700709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:46:45.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:45.116+0000 7f9589574640 1 --2- 172.21.15.112:0/1167757337 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f957006e530 0x7f95700709e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9584071350 tx=0x7f958000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:46:45.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:45.120+0000 7f95767fc640 1 -- 172.21.15.112:0/1167757337 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f95780776d0 con 0x7f9584102530 2024-09-16T00:46:45.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:46:44 smithi169 ceph-mon[77626]: from='client.35080 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:46:45.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:46:44 smithi169 ceph-mon[77626]: from='client.35084 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:46:45.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:46:44 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/940808019' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:46:45.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:46:44 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/593479363' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:46:45.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:45.349+0000 7f958bfd9640 1 -- 172.21.15.112:0/1167757337 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f95840fbd30 con 0x7f9584102530 2024-09-16T00:46:45.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:45.349+0000 7f95767fc640 1 -- 172.21.15.112:0/1167757337 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f957807ad90 con 0x7f9584102530 2024-09-16T00:46:45.349 INFO:teuthology.orchestra.run.smithi112.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-16T00:46:45.349 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-16T00:46:45.350 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs is degraded 2024-09-16T00:46:45.350 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T00:46:45.350 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T00:46:45.350 INFO:teuthology.orchestra.run.smithi112.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-16T00:46:45.350 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-16T00:46:45.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:45.351+0000 7f958bfd9640 1 -- 172.21.15.112:0/1167757337 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f957006e530 msgr2=0x7f95700709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:45.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:45.351+0000 7f958bfd9640 1 --2- 172.21.15.112:0/1167757337 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f957006e530 0x7f95700709e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9584071350 tx=0x7f958000b040 comp rx=0 tx=0).stop 2024-09-16T00:46:45.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:45.351+0000 7f958bfd9640 1 -- 172.21.15.112:0/1167757337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9584102530 msgr2=0x7f958406fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:46:45.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:45.351+0000 7f958bfd9640 1 --2- 172.21.15.112:0/1167757337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9584102530 0x7f958406fa10 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f95780043d0 tx=0x7f9578009da0 comp rx=0 tx=0).stop 2024-09-16T00:46:45.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:45.351+0000 7f958bfd9640 1 -- 172.21.15.112:0/1167757337 shutdown_connections 2024-09-16T00:46:45.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:45.351+0000 7f958bfd9640 1 --2- 172.21.15.112:0/1167757337 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f957006e530 0x7f95700709e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:45.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:45.351+0000 7f958bfd9640 1 --2- 172.21.15.112:0/1167757337 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9584102ed0 0x7f958406ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:45.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:45.351+0000 7f958bfd9640 1 --2- 172.21.15.112:0/1167757337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9584102530 0x7f958406fa10 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:46:45.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:45.351+0000 7f958bfd9640 1 -- 172.21.15.112:0/1167757337 >> 172.21.15.112:0/1167757337 conn(0x7f95840f80f0 msgr2=0x7f95840f90f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:46:45.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:45.351+0000 7f958bfd9640 1 -- 172.21.15.112:0/1167757337 shutdown_connections 2024-09-16T00:46:45.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:46:45.351+0000 7f958bfd9640 1 -- 172.21.15.112:0/1167757337 wait complete. 2024-09-16T00:46:46.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:46:45 smithi112 ceph-mon[97750]: from='client.35088 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:46:46.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:46:45 smithi112 ceph-mon[97750]: pgmap v570: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:46:46.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:46:45 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/1167757337' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:46:46.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:46:45 smithi169 ceph-mon[77626]: from='client.35088 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:46:46.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:46:45 smithi169 ceph-mon[77626]: pgmap v570: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:46:46.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:46:45 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/1167757337' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:46:47.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:46:46 smithi169 ceph-mon[77626]: from='client.35100 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:46:47.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:46:46 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:46:47.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:46:46 smithi112 ceph-mon[97750]: from='client.35100 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:46:47.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:46:46 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:46:48.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:46:47 smithi169 ceph-mon[77626]: pgmap v571: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:46:48.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:46:47 smithi112 ceph-mon[97750]: pgmap v571: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:46:50.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:46:49 smithi169 ceph-mon[77626]: pgmap v572: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:46:50.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:46:49 smithi112 ceph-mon[97750]: pgmap v572: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:46:52.141 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:46:51 smithi112 ceph-mon[97750]: pgmap v573: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:46:52.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:46:51 smithi169 ceph-mon[77626]: pgmap v573: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:46:54.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:46:53 smithi112 ceph-mon[97750]: pgmap v574: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:46:54.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:46:53 smithi169 ceph-mon[77626]: pgmap v574: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:46:56.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:46:55 smithi169 ceph-mon[77626]: pgmap v575: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:46:56.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:46:55 smithi112 ceph-mon[97750]: pgmap v575: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:46:58.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:46:57 smithi169 ceph-mon[77626]: pgmap v576: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:46:58.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:46:57 smithi112 ceph-mon[97750]: pgmap v576: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:47:00.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:46:59 smithi169 ceph-mon[77626]: pgmap v577: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:47:00.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:46:59 smithi112 ceph-mon[97750]: pgmap v577: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:47:02.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:47:01 smithi169 ceph-mon[77626]: pgmap v578: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:47:02.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:47:01 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:47:02.314 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:47:01 smithi112 ceph-mon[97750]: pgmap v578: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:47:02.314 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:47:01 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:47:04.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:47:03 smithi112 ceph-mon[97750]: pgmap v579: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:47:04.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:47:03 smithi169 ceph-mon[77626]: pgmap v579: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:47:06.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:47:05 smithi169 ceph-mon[77626]: pgmap v580: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:47:06.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:47:05 smithi112 ceph-mon[97750]: pgmap v580: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:47:08.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:47:07 smithi169 ceph-mon[77626]: pgmap v581: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:47:08.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:47:07 smithi112 ceph-mon[97750]: pgmap v581: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:47:10.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:47:09 smithi169 ceph-mon[77626]: pgmap v582: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:47:10.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:47:09 smithi112 ceph-mon[97750]: pgmap v582: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:47:12.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:47:11 smithi169 ceph-mon[77626]: pgmap v583: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:47:12.314 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:47:11 smithi112 ceph-mon[97750]: pgmap v583: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:47:14.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:47:13 smithi112 ceph-mon[97750]: pgmap v584: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:47:14.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:47:13 smithi169 ceph-mon[77626]: pgmap v584: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:47:15.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.442+0000 7f81889d3640 1 -- 172.21.15.112:0/3579896803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8184102660 msgr2=0x7f8184102a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:15.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.442+0000 7f81889d3640 1 --2- 172.21.15.112:0/3579896803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8184102660 0x7f8184102a30 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f81740048d0 tx=0x7f8174009da0 comp rx=0 tx=0).stop 2024-09-16T00:47:15.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.442+0000 7f81889d3640 1 -- 172.21.15.112:0/3579896803 shutdown_connections 2024-09-16T00:47:15.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.442+0000 7f81889d3640 1 --2- 172.21.15.112:0/3579896803 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81840fac80 0x7f81840fb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:15.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.442+0000 7f81889d3640 1 --2- 172.21.15.112:0/3579896803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8184102660 0x7f8184102a30 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:15.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.442+0000 7f81889d3640 1 -- 172.21.15.112:0/3579896803 >> 172.21.15.112:0/3579896803 conn(0x7f8184070ce0 msgr2=0x7f81840710e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:15.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.443+0000 7f81889d3640 1 -- 172.21.15.112:0/3579896803 shutdown_connections 2024-09-16T00:47:15.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.443+0000 7f81889d3640 1 -- 172.21.15.112:0/3579896803 wait complete. 2024-09-16T00:47:15.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.443+0000 7f81889d3640 1 Processor -- start 2024-09-16T00:47:15.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.443+0000 7f81889d3640 1 -- start start 2024-09-16T00:47:15.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.443+0000 7f81889d3640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81840fac80 0x7f8184113170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:15.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.443+0000 7f81889d3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8184102660 0x7f81841136b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:15.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.443+0000 7f81889d3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8184113cb0 con 0x7f8184102660 2024-09-16T00:47:15.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.443+0000 7f81889d3640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8184113e20 con 0x7f81840fac80 2024-09-16T00:47:15.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.443+0000 7f8181d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8184102660 0x7f81841136b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:15.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.444+0000 7f8181d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8184102660 0x7f81841136b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53920/0 (socket says 172.21.15.112:53920) 2024-09-16T00:47:15.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.444+0000 7f8181d9b640 1 -- 172.21.15.112:0/3601995550 learned_addr learned my addr 172.21.15.112:0/3601995550 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:47:15.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.444+0000 7f818259c640 1 --2- 172.21.15.112:0/3601995550 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81840fac80 0x7f8184113170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:15.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.444+0000 7f8181d9b640 1 -- 172.21.15.112:0/3601995550 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81840fac80 msgr2=0x7f8184113170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:15.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.444+0000 7f8181d9b640 1 --2- 172.21.15.112:0/3601995550 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81840fac80 0x7f8184113170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:15.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.444+0000 7f8181d9b640 1 -- 172.21.15.112:0/3601995550 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8174005040 con 0x7f8184102660 2024-09-16T00:47:15.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.444+0000 7f818259c640 1 --2- 172.21.15.112:0/3601995550 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81840fac80 0x7f8184113170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:47:15.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.444+0000 7f8181d9b640 1 --2- 172.21.15.112:0/3601995550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8184102660 0x7f81841136b0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f817800c790 tx=0x7f81780043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:15.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.444+0000 7f8172ffd640 1 -- 172.21.15.112:0/3601995550 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f817800a820 con 0x7f8184102660 2024-09-16T00:47:15.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.444+0000 7f8172ffd640 1 -- 172.21.15.112:0/3601995550 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f817800ae20 con 0x7f8184102660 2024-09-16T00:47:15.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.444+0000 7f81889d3640 1 -- 172.21.15.112:0/3601995550 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f818410d2a0 con 0x7f8184102660 2024-09-16T00:47:15.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.444+0000 7f8172ffd640 1 -- 172.21.15.112:0/3601995550 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f81780113a0 con 0x7f8184102660 2024-09-16T00:47:15.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.444+0000 7f81889d3640 1 -- 172.21.15.112:0/3601995550 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f818410d840 con 0x7f8184102660 2024-09-16T00:47:15.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.445+0000 7f81889d3640 1 -- 172.21.15.112:0/3601995550 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81840657d0 con 0x7f8184102660 2024-09-16T00:47:15.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.446+0000 7f8172ffd640 1 -- 172.21.15.112:0/3601995550 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f8178012400 con 0x7f8184102660 2024-09-16T00:47:15.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.446+0000 7f8172ffd640 1 --2- 172.21.15.112:0/3601995550 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f816c06e530 0x7f816c0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:15.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.447+0000 7f8172ffd640 1 -- 172.21.15.112:0/3601995550 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f817808e480 con 0x7f8184102660 2024-09-16T00:47:15.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.447+0000 7f818259c640 1 --2- 172.21.15.112:0/3601995550 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f816c06e530 0x7f816c0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:15.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.447+0000 7f818259c640 1 --2- 172.21.15.112:0/3601995550 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f816c06e530 0x7f816c0709e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8174007ac0 tx=0x7f8174035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:15.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.451+0000 7f8172ffd640 1 -- 172.21.15.112:0/3601995550 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f8178057470 con 0x7f8184102660 2024-09-16T00:47:15.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.633+0000 7f81889d3640 1 -- 172.21.15.112:0/3601995550 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f818404b720 con 0x7f816c06e530 2024-09-16T00:47:15.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.634+0000 7f8172ffd640 1 -- 172.21.15.112:0/3601995550 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f818404b720 con 0x7f816c06e530 2024-09-16T00:47:15.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.636+0000 7f81889d3640 1 -- 172.21.15.112:0/3601995550 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f816c06e530 msgr2=0x7f816c0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:15.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.636+0000 7f81889d3640 1 --2- 172.21.15.112:0/3601995550 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f816c06e530 0x7f816c0709e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8174007ac0 tx=0x7f8174035040 comp rx=0 tx=0).stop 2024-09-16T00:47:15.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.636+0000 7f81889d3640 1 -- 172.21.15.112:0/3601995550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8184102660 msgr2=0x7f81841136b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:15.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.636+0000 7f81889d3640 1 --2- 172.21.15.112:0/3601995550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8184102660 0x7f81841136b0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f817800c790 tx=0x7f81780043c0 comp rx=0 tx=0).stop 2024-09-16T00:47:15.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.636+0000 7f81889d3640 1 -- 172.21.15.112:0/3601995550 shutdown_connections 2024-09-16T00:47:15.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.636+0000 7f81889d3640 1 --2- 172.21.15.112:0/3601995550 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f816c06e530 0x7f816c0709e0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:15.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.636+0000 7f81889d3640 1 --2- 172.21.15.112:0/3601995550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8184102660 0x7f81841136b0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:15.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.636+0000 7f81889d3640 1 --2- 172.21.15.112:0/3601995550 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81840fac80 0x7f8184113170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:15.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.636+0000 7f81889d3640 1 -- 172.21.15.112:0/3601995550 >> 172.21.15.112:0/3601995550 conn(0x7f8184070ce0 msgr2=0x7f81840fa050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:15.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.636+0000 7f81889d3640 1 -- 172.21.15.112:0/3601995550 shutdown_connections 2024-09-16T00:47:15.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.636+0000 7f81889d3640 1 -- 172.21.15.112:0/3601995550 wait complete. 2024-09-16T00:47:15.646 INFO:teuthology.orchestra.run.smithi112.stdout:true 2024-09-16T00:47:15.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.715+0000 7f0b3b9bf640 1 -- 172.21.15.112:0/3195610687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b34102ed0 msgr2=0x7f0b340fc590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:15.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.715+0000 7f0b3b9bf640 1 --2- 172.21.15.112:0/3195610687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b34102ed0 0x7f0b340fc590 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f0b240048d0 tx=0x7f0b24009da0 comp rx=0 tx=0).stop 2024-09-16T00:47:15.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.715+0000 7f0b3b9bf640 1 -- 172.21.15.112:0/3195610687 shutdown_connections 2024-09-16T00:47:15.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.715+0000 7f0b3b9bf640 1 --2- 172.21.15.112:0/3195610687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b34102ed0 0x7f0b340fc590 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:15.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.715+0000 7f0b3b9bf640 1 --2- 172.21.15.112:0/3195610687 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b34102530 0x7f0b34102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:15.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.715+0000 7f0b3b9bf640 1 -- 172.21.15.112:0/3195610687 >> 172.21.15.112:0/3195610687 conn(0x7f0b340f80f0 msgr2=0x7f0b340fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:15.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.716+0000 7f0b3b9bf640 1 -- 172.21.15.112:0/3195610687 shutdown_connections 2024-09-16T00:47:15.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.716+0000 7f0b3b9bf640 1 -- 172.21.15.112:0/3195610687 wait complete. 2024-09-16T00:47:15.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.716+0000 7f0b3b9bf640 1 Processor -- start 2024-09-16T00:47:15.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.716+0000 7f0b3b9bf640 1 -- start start 2024-09-16T00:47:15.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.716+0000 7f0b3b9bf640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b34102530 0x7f0b341131f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:15.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.717+0000 7f0b3b9bf640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b34102ed0 0x7f0b34113730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:15.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.717+0000 7f0b3b9bf640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b34113d30 con 0x7f0b34102ed0 2024-09-16T00:47:15.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.717+0000 7f0b3b9bf640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3410d170 con 0x7f0b34102530 2024-09-16T00:47:15.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.717+0000 7f0b38f5a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b34102ed0 0x7f0b34113730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:15.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.717+0000 7f0b38f5a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b34102ed0 0x7f0b34113730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53932/0 (socket says 172.21.15.112:53932) 2024-09-16T00:47:15.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.717+0000 7f0b38f5a640 1 -- 172.21.15.112:0/3636253781 learned_addr learned my addr 172.21.15.112:0/3636253781 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:47:15.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.717+0000 7f0b3975b640 1 --2- 172.21.15.112:0/3636253781 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b34102530 0x7f0b341131f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:15.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.717+0000 7f0b38f5a640 1 -- 172.21.15.112:0/3636253781 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b34102530 msgr2=0x7f0b341131f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:15.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.717+0000 7f0b38f5a640 1 --2- 172.21.15.112:0/3636253781 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b34102530 0x7f0b341131f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:15.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.717+0000 7f0b38f5a640 1 -- 172.21.15.112:0/3636253781 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b24005040 con 0x7f0b34102ed0 2024-09-16T00:47:15.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.717+0000 7f0b38f5a640 1 --2- 172.21.15.112:0/3636253781 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b34102ed0 0x7f0b34113730 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f0b240043d0 tx=0x7f0b24009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:15.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.717+0000 7f0b29ffb640 1 -- 172.21.15.112:0/3636253781 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b24038070 con 0x7f0b34102ed0 2024-09-16T00:47:15.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.717+0000 7f0b29ffb640 1 -- 172.21.15.112:0/3636253781 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f0b240096e0 con 0x7f0b34102ed0 2024-09-16T00:47:15.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.717+0000 7f0b3b9bf640 1 -- 172.21.15.112:0/3636253781 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b3410d3f0 con 0x7f0b34102ed0 2024-09-16T00:47:15.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.718+0000 7f0b29ffb640 1 -- 172.21.15.112:0/3636253781 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b2402c8a0 con 0x7f0b34102ed0 2024-09-16T00:47:15.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.718+0000 7f0b3b9bf640 1 -- 172.21.15.112:0/3636253781 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b3410d930 con 0x7f0b34102ed0 2024-09-16T00:47:15.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.719+0000 7f0b3b9bf640 1 -- 172.21.15.112:0/3636253781 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b340fdcb0 con 0x7f0b34102ed0 2024-09-16T00:47:15.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.719+0000 7f0b29ffb640 1 -- 172.21.15.112:0/3636253781 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f0b24030020 con 0x7f0b34102ed0 2024-09-16T00:47:15.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.720+0000 7f0b29ffb640 1 --2- 172.21.15.112:0/3636253781 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f0b2006e530 0x7f0b200709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:15.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.720+0000 7f0b29ffb640 1 -- 172.21.15.112:0/3636253781 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f0b240adff0 con 0x7f0b34102ed0 2024-09-16T00:47:15.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.720+0000 7f0b3975b640 1 --2- 172.21.15.112:0/3636253781 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f0b2006e530 0x7f0b200709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:15.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.721+0000 7f0b3975b640 1 --2- 172.21.15.112:0/3636253781 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f0b2006e530 0x7f0b200709e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0b30004ed0 tx=0x7f0b3000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:15.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.724+0000 7f0b29ffb640 1 -- 172.21.15.112:0/3636253781 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f0b24076fd0 con 0x7f0b34102ed0 2024-09-16T00:47:15.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.907+0000 7f0b3b9bf640 1 -- 172.21.15.112:0/3636253781 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0b34109590 con 0x7f0b2006e530 2024-09-16T00:47:15.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.908+0000 7f0b29ffb640 1 -- 172.21.15.112:0/3636253781 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f0b34109590 con 0x7f0b2006e530 2024-09-16T00:47:15.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.910+0000 7f0b3b9bf640 1 -- 172.21.15.112:0/3636253781 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f0b2006e530 msgr2=0x7f0b200709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:15.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.910+0000 7f0b3b9bf640 1 --2- 172.21.15.112:0/3636253781 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f0b2006e530 0x7f0b200709e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0b30004ed0 tx=0x7f0b3000b040 comp rx=0 tx=0).stop 2024-09-16T00:47:15.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.910+0000 7f0b3b9bf640 1 -- 172.21.15.112:0/3636253781 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b34102ed0 msgr2=0x7f0b34113730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:15.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.910+0000 7f0b3b9bf640 1 --2- 172.21.15.112:0/3636253781 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b34102ed0 0x7f0b34113730 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f0b240043d0 tx=0x7f0b24009da0 comp rx=0 tx=0).stop 2024-09-16T00:47:15.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.910+0000 7f0b3b9bf640 1 -- 172.21.15.112:0/3636253781 shutdown_connections 2024-09-16T00:47:15.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.910+0000 7f0b3b9bf640 1 --2- 172.21.15.112:0/3636253781 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f0b2006e530 0x7f0b200709e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:15.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.910+0000 7f0b3b9bf640 1 --2- 172.21.15.112:0/3636253781 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b34102ed0 0x7f0b34113730 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:15.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.910+0000 7f0b3b9bf640 1 --2- 172.21.15.112:0/3636253781 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b34102530 0x7f0b341131f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:15.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.910+0000 7f0b3b9bf640 1 -- 172.21.15.112:0/3636253781 >> 172.21.15.112:0/3636253781 conn(0x7f0b340f80f0 msgr2=0x7f0b340f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:15.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.910+0000 7f0b3b9bf640 1 -- 172.21.15.112:0/3636253781 shutdown_connections 2024-09-16T00:47:15.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.910+0000 7f0b3b9bf640 1 -- 172.21.15.112:0/3636253781 wait complete. 2024-09-16T00:47:15.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.988+0000 7f2859d13640 1 -- 172.21.15.112:0/929833635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2854102ef0 msgr2=0x7f28540fc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:15.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.988+0000 7f2859d13640 1 --2- 172.21.15.112:0/929833635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2854102ef0 0x7f28540fc5b0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f28440048d0 tx=0x7f2844009da0 comp rx=0 tx=0).stop 2024-09-16T00:47:15.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.988+0000 7f2859d13640 1 -- 172.21.15.112:0/929833635 shutdown_connections 2024-09-16T00:47:15.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.988+0000 7f2859d13640 1 --2- 172.21.15.112:0/929833635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2854102ef0 0x7f28540fc5b0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:15.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.988+0000 7f2859d13640 1 --2- 172.21.15.112:0/929833635 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2854102550 0x7f2854102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:15.990 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:47:15 smithi112 ceph-mon[97750]: pgmap v585: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:47:15.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.988+0000 7f2859d13640 1 -- 172.21.15.112:0/929833635 >> 172.21.15.112:0/929833635 conn(0x7f28540f80f0 msgr2=0x7f28540fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:15.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.988+0000 7f2859d13640 1 -- 172.21.15.112:0/929833635 shutdown_connections 2024-09-16T00:47:15.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.988+0000 7f2859d13640 1 -- 172.21.15.112:0/929833635 wait complete. 2024-09-16T00:47:15.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.989+0000 7f2859d13640 1 Processor -- start 2024-09-16T00:47:15.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.989+0000 7f2859d13640 1 -- start start 2024-09-16T00:47:15.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.989+0000 7f2859d13640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2854102550 0x7f285406fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:15.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.989+0000 7f2859d13640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2854102ef0 0x7f285406ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:15.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.989+0000 7f2859d13640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2854070490 con 0x7f2854102550 2024-09-16T00:47:15.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.989+0000 7f2859d13640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2854070600 con 0x7f2854102ef0 2024-09-16T00:47:15.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.990+0000 7f28537fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2854102550 0x7f285406fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:15.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.990+0000 7f28537fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2854102550 0x7f285406fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53958/0 (socket says 172.21.15.112:53958) 2024-09-16T00:47:15.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.990+0000 7f28537fe640 1 -- 172.21.15.112:0/1730658583 learned_addr learned my addr 172.21.15.112:0/1730658583 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:47:15.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.990+0000 7f2852ffd640 1 --2- 172.21.15.112:0/1730658583 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2854102ef0 0x7f285406ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:15.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.990+0000 7f28537fe640 1 -- 172.21.15.112:0/1730658583 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2854102ef0 msgr2=0x7f285406ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:15.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.990+0000 7f28537fe640 1 --2- 172.21.15.112:0/1730658583 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2854102ef0 0x7f285406ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:15.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.990+0000 7f28537fe640 1 -- 172.21.15.112:0/1730658583 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2844005040 con 0x7f2854102550 2024-09-16T00:47:15.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.990+0000 7f28537fe640 1 --2- 172.21.15.112:0/1730658583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2854102550 0x7f285406fa10 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f283c00c830 tx=0x7f283c004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:15.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.990+0000 7f284bfff640 1 -- 172.21.15.112:0/1730658583 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f283c00a7d0 con 0x7f2854102550 2024-09-16T00:47:15.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.990+0000 7f284bfff640 1 -- 172.21.15.112:0/1730658583 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f283c00add0 con 0x7f2854102550 2024-09-16T00:47:15.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.990+0000 7f2859d13640 1 -- 172.21.15.112:0/1730658583 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2854070890 con 0x7f2854102550 2024-09-16T00:47:15.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.990+0000 7f284bfff640 1 -- 172.21.15.112:0/1730658583 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f283c0113a0 con 0x7f2854102550 2024-09-16T00:47:15.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.990+0000 7f2859d13640 1 -- 172.21.15.112:0/1730658583 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2854119c00 con 0x7f2854102550 2024-09-16T00:47:15.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.991+0000 7f2859d13640 1 -- 172.21.15.112:0/1730658583 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28540fdcd0 con 0x7f2854102550 2024-09-16T00:47:15.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.993+0000 7f284bfff640 1 -- 172.21.15.112:0/1730658583 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f283c012400 con 0x7f2854102550 2024-09-16T00:47:15.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.993+0000 7f284bfff640 1 --2- 172.21.15.112:0/1730658583 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f284006e530 0x7f28400709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:15.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.993+0000 7f284bfff640 1 -- 172.21.15.112:0/1730658583 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f283c08ee50 con 0x7f2854102550 2024-09-16T00:47:15.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.993+0000 7f2852ffd640 1 --2- 172.21.15.112:0/1730658583 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f284006e530 0x7f28400709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:15.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.994+0000 7f2852ffd640 1 --2- 172.21.15.112:0/1730658583 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f284006e530 0x7f28400709e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f28440048a0 tx=0x7f2844035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:15.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:15.996+0000 7f284bfff640 1 -- 172.21.15.112:0/1730658583 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f283c057ee0 con 0x7f2854102550 2024-09-16T00:47:16.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.170+0000 7f2859d13640 1 -- 172.21.15.112:0/1730658583 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2854071100 con 0x7f284006e530 2024-09-16T00:47:16.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.178+0000 7f284bfff640 1 -- 172.21.15.112:0/1730658583 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7f2854071100 con 0x7f284006e530 2024-09-16T00:47:16.179 INFO:teuthology.orchestra.run.smithi112.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T00:47:16.179 INFO:teuthology.orchestra.run.smithi112.stdout:alertmanager.smithi112 smithi112 *:9093,9094 running (20m) 13m ago 29m 24.7M - 0.25.0 c8568f914cd2 d85ea0562022 2024-09-16T00:47:16.180 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi112 smithi112 running (17m) 13m ago 29m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c e4fe58f9702a 2024-09-16T00:47:16.180 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi169 smithi169 running (17m) 12m ago 28m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c 3ba2c81d527f 2024-09-16T00:47:16.180 INFO:teuthology.orchestra.run.smithi112.stdout:grafana.smithi112 smithi112 *:3000 running (18m) 13m ago 28m 82.1M - 9.4.12 2bacad6d85d8 51f9d2013da2 2024-09-16T00:47:16.180 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.hxfwkh smithi112 running (13m) 13m ago 23m 14.2M - 19.1.1-326-g8921c8ef f1455bd0219c 1723069902d6 2024-09-16T00:47:16.180 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.ithwzj smithi112 running (13m) 13m ago 23m 12.2M - 19.1.1-326-g8921c8ef f1455bd0219c e7aad5b81db5 2024-09-16T00:47:16.180 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.gawwgw smithi169 running (13m) 12m ago 23m 15.7M - 19.1.1-326-g8921c8ef f1455bd0219c 469be459ee12 2024-09-16T00:47:16.180 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.lgeqrj smithi169 running (12m) 12m ago 23m 13.3M - 19.1.1-326-g8921c8ef f1455bd0219c 31f0fbd2395d 2024-09-16T00:47:16.180 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi112.xrkkbc smithi112 *:8443,9283,8765 running (19m) 13m ago 30m 504M - 19.1.1-326-g8921c8ef f1455bd0219c a981f7fd2810 2024-09-16T00:47:16.180 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi169.mtrobk smithi169 *:8443,9283,8765 running (19m) 12m ago 28m 442M - 19.1.1-326-g8921c8ef f1455bd0219c 4096e5369c2d 2024-09-16T00:47:16.180 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi112 smithi112 running (18m) 13m ago 30m 49.8M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 5aaec033c042 2024-09-16T00:47:16.181 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi169 smithi169 running (17m) 12m ago 28m 38.9M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 7bc609155f3f 2024-09-16T00:47:16.181 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi112 smithi112 *:9100 running (20m) 13m ago 29m 15.5M - 1.5.0 0da6a335fe13 6404d1d601e2 2024-09-16T00:47:16.181 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi169 smithi169 *:9100 running (19m) 12m ago 28m 15.4M - 1.5.0 0da6a335fe13 5505baf702a6 2024-09-16T00:47:16.181 INFO:teuthology.orchestra.run.smithi112.stdout:osd.0 smithi112 running (16m) 13m ago 27m 105M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 53efc9397017 2024-09-16T00:47:16.181 INFO:teuthology.orchestra.run.smithi112.stdout:osd.1 smithi112 running (16m) 13m ago 27m 98.9M 1557M 19.1.1-326-g8921c8ef f1455bd0219c fe6f1a4a2cf4 2024-09-16T00:47:16.181 INFO:teuthology.orchestra.run.smithi112.stdout:osd.2 smithi112 running (15m) 13m ago 26m 102M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 5f7863712057 2024-09-16T00:47:16.181 INFO:teuthology.orchestra.run.smithi112.stdout:osd.3 smithi169 running (15m) 12m ago 26m 92.1M 2581M 19.1.1-326-g8921c8ef f1455bd0219c ee07b5577810 2024-09-16T00:47:16.181 INFO:teuthology.orchestra.run.smithi112.stdout:osd.4 smithi169 running (14m) 12m ago 25m 103M 2581M 19.1.1-326-g8921c8ef f1455bd0219c 769f026a54eb 2024-09-16T00:47:16.181 INFO:teuthology.orchestra.run.smithi112.stdout:osd.5 smithi169 running (13m) 12m ago 25m 125M 2581M 19.1.1-326-g8921c8ef f1455bd0219c a6007c13cf78 2024-09-16T00:47:16.181 INFO:teuthology.orchestra.run.smithi112.stdout:prometheus.smithi112 smithi112 *:9095 running (19m) 13m ago 28m 64.1M - 2.43.0 a07b618ecd1d 7a1d42e60d2b 2024-09-16T00:47:16.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.181+0000 7f2859d13640 1 -- 172.21.15.112:0/1730658583 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f284006e530 msgr2=0x7f28400709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:16.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.181+0000 7f2859d13640 1 --2- 172.21.15.112:0/1730658583 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f284006e530 0x7f28400709e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f28440048a0 tx=0x7f2844035040 comp rx=0 tx=0).stop 2024-09-16T00:47:16.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.181+0000 7f2859d13640 1 -- 172.21.15.112:0/1730658583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2854102550 msgr2=0x7f285406fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:16.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.181+0000 7f2859d13640 1 --2- 172.21.15.112:0/1730658583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2854102550 0x7f285406fa10 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f283c00c830 tx=0x7f283c004430 comp rx=0 tx=0).stop 2024-09-16T00:47:16.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.181+0000 7f2859d13640 1 -- 172.21.15.112:0/1730658583 shutdown_connections 2024-09-16T00:47:16.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.181+0000 7f2859d13640 1 --2- 172.21.15.112:0/1730658583 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f284006e530 0x7f28400709e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:16.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.181+0000 7f2859d13640 1 --2- 172.21.15.112:0/1730658583 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2854102ef0 0x7f285406ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:16.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.181+0000 7f2859d13640 1 --2- 172.21.15.112:0/1730658583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2854102550 0x7f285406fa10 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:16.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.181+0000 7f2859d13640 1 -- 172.21.15.112:0/1730658583 >> 172.21.15.112:0/1730658583 conn(0x7f28540f80f0 msgr2=0x7f28540f9150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:16.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.181+0000 7f2859d13640 1 -- 172.21.15.112:0/1730658583 shutdown_connections 2024-09-16T00:47:16.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.181+0000 7f2859d13640 1 -- 172.21.15.112:0/1730658583 wait complete. 2024-09-16T00:47:16.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.257+0000 7f3bf5256640 1 -- 172.21.15.112:0/3683432359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3bf0102550 msgr2=0x7f3bf0102920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:16.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.257+0000 7f3bf5256640 1 --2- 172.21.15.112:0/3683432359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3bf0102550 0x7f3bf0102920 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f3be00048d0 tx=0x7f3be0009da0 comp rx=0 tx=0).stop 2024-09-16T00:47:16.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.257+0000 7f3bf5256640 1 -- 172.21.15.112:0/3683432359 shutdown_connections 2024-09-16T00:47:16.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.257+0000 7f3bf5256640 1 --2- 172.21.15.112:0/3683432359 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3bf0102ef0 0x7f3bf00fc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:16.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.257+0000 7f3bf5256640 1 --2- 172.21.15.112:0/3683432359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3bf0102550 0x7f3bf0102920 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:16.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.257+0000 7f3bf5256640 1 -- 172.21.15.112:0/3683432359 >> 172.21.15.112:0/3683432359 conn(0x7f3bf00f80f0 msgr2=0x7f3bf00fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:16.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.257+0000 7f3bf5256640 1 -- 172.21.15.112:0/3683432359 shutdown_connections 2024-09-16T00:47:16.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.258+0000 7f3bf5256640 1 -- 172.21.15.112:0/3683432359 wait complete. 2024-09-16T00:47:16.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.258+0000 7f3bf5256640 1 Processor -- start 2024-09-16T00:47:16.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.258+0000 7f3bf5256640 1 -- start start 2024-09-16T00:47:16.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.258+0000 7f3bf5256640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3bf0102550 0x7f3bf0113570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:16.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.258+0000 7f3bf5256640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3bf0102ef0 0x7f3bf0113ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:16.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.258+0000 7f3bf5256640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf0114100 con 0x7f3bf0102ef0 2024-09-16T00:47:16.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.258+0000 7f3bf5256640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf010d510 con 0x7f3bf0102550 2024-09-16T00:47:16.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.259+0000 7f3be659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3bf0102ef0 0x7f3bf0113ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:16.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.259+0000 7f3be659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3bf0102ef0 0x7f3bf0113ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53976/0 (socket says 172.21.15.112:53976) 2024-09-16T00:47:16.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.259+0000 7f3be659c640 1 -- 172.21.15.112:0/2922647337 learned_addr learned my addr 172.21.15.112:0/2922647337 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:47:16.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.259+0000 7f3beed9d640 1 --2- 172.21.15.112:0/2922647337 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3bf0102550 0x7f3bf0113570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:16.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.259+0000 7f3be659c640 1 -- 172.21.15.112:0/2922647337 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3bf0102550 msgr2=0x7f3bf0113570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:16.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.259+0000 7f3be659c640 1 --2- 172.21.15.112:0/2922647337 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3bf0102550 0x7f3bf0113570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:16.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.259+0000 7f3be659c640 1 -- 172.21.15.112:0/2922647337 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3be0005040 con 0x7f3bf0102ef0 2024-09-16T00:47:16.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.259+0000 7f3be659c640 1 --2- 172.21.15.112:0/2922647337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3bf0102ef0 0x7f3bf0113ab0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f3bd8002870 tx=0x7f3bd8004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:16.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.259+0000 7f3be7fff640 1 -- 172.21.15.112:0/2922647337 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3bd8007c30 con 0x7f3bf0102ef0 2024-09-16T00:47:16.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.259+0000 7f3be7fff640 1 -- 172.21.15.112:0/2922647337 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f3bd800f040 con 0x7f3bf0102ef0 2024-09-16T00:47:16.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.259+0000 7f3bf5256640 1 -- 172.21.15.112:0/2922647337 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bf010d7f0 con 0x7f3bf0102ef0 2024-09-16T00:47:16.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.259+0000 7f3be7fff640 1 -- 172.21.15.112:0/2922647337 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3bd8009640 con 0x7f3bf0102ef0 2024-09-16T00:47:16.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.259+0000 7f3bf5256640 1 -- 172.21.15.112:0/2922647337 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bf010dd10 con 0x7f3bf0102ef0 2024-09-16T00:47:16.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.260+0000 7f3bf5256640 1 -- 172.21.15.112:0/2922647337 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bf00fdcd0 con 0x7f3bf0102ef0 2024-09-16T00:47:16.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.261+0000 7f3be7fff640 1 -- 172.21.15.112:0/2922647337 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f3bd80097a0 con 0x7f3bf0102ef0 2024-09-16T00:47:16.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.261+0000 7f3be7fff640 1 --2- 172.21.15.112:0/2922647337 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f3bdc06e530 0x7f3bdc0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:16.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.262+0000 7f3be7fff640 1 -- 172.21.15.112:0/2922647337 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f3bd808e230 con 0x7f3bf0102ef0 2024-09-16T00:47:16.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.262+0000 7f3beed9d640 1 --2- 172.21.15.112:0/2922647337 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f3bdc06e530 0x7f3bdc0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:16.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.262+0000 7f3beed9d640 1 --2- 172.21.15.112:0/2922647337 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f3bdc06e530 0x7f3bdc0709e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3be0009ac0 tx=0x7f3be0035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:16.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.266+0000 7f3be7fff640 1 -- 172.21.15.112:0/2922647337 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f3bd805acd0 con 0x7f3bf0102ef0 2024-09-16T00:47:16.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:47:15 smithi169 ceph-mon[77626]: pgmap v585: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:47:16.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.495+0000 7f3bf5256640 1 -- 172.21.15.112:0/2922647337 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3bf010ec10 con 0x7f3bf0102ef0 2024-09-16T00:47:16.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.496+0000 7f3be7fff640 1 -- 172.21.15.112:0/2922647337 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f3bd805a860 con 0x7f3bf0102ef0 2024-09-16T00:47:16.497 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:47:16.497 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": { 2024-09-16T00:47:16.497 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:47:16.498 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:47:16.498 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": { 2024-09-16T00:47:16.498 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:47:16.498 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:47:16.498 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": { 2024-09-16T00:47:16.498 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 6 2024-09-16T00:47:16.498 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:47:16.498 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": { 2024-09-16T00:47:16.498 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 4 2024-09-16T00:47:16.498 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:47:16.498 INFO:teuthology.orchestra.run.smithi112.stdout: "overall": { 2024-09-16T00:47:16.498 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 14 2024-09-16T00:47:16.499 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-16T00:47:16.499 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:47:16.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.498+0000 7f3bf5256640 1 -- 172.21.15.112:0/2922647337 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f3bdc06e530 msgr2=0x7f3bdc0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:16.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.498+0000 7f3bf5256640 1 --2- 172.21.15.112:0/2922647337 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f3bdc06e530 0x7f3bdc0709e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3be0009ac0 tx=0x7f3be0035040 comp rx=0 tx=0).stop 2024-09-16T00:47:16.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.498+0000 7f3bf5256640 1 -- 172.21.15.112:0/2922647337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3bf0102ef0 msgr2=0x7f3bf0113ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:16.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.498+0000 7f3bf5256640 1 --2- 172.21.15.112:0/2922647337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3bf0102ef0 0x7f3bf0113ab0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f3bd8002870 tx=0x7f3bd8004600 comp rx=0 tx=0).stop 2024-09-16T00:47:16.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.498+0000 7f3bf5256640 1 -- 172.21.15.112:0/2922647337 shutdown_connections 2024-09-16T00:47:16.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.498+0000 7f3bf5256640 1 --2- 172.21.15.112:0/2922647337 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f3bdc06e530 0x7f3bdc0709e0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:16.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.498+0000 7f3bf5256640 1 --2- 172.21.15.112:0/2922647337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3bf0102ef0 0x7f3bf0113ab0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:16.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.498+0000 7f3bf5256640 1 --2- 172.21.15.112:0/2922647337 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3bf0102550 0x7f3bf0113570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:16.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.498+0000 7f3bf5256640 1 -- 172.21.15.112:0/2922647337 >> 172.21.15.112:0/2922647337 conn(0x7f3bf00f80f0 msgr2=0x7f3bf00f90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:16.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.498+0000 7f3bf5256640 1 -- 172.21.15.112:0/2922647337 shutdown_connections 2024-09-16T00:47:16.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.498+0000 7f3bf5256640 1 -- 172.21.15.112:0/2922647337 wait complete. 2024-09-16T00:47:16.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.577+0000 7fc9654dc640 1 -- 172.21.15.112:0/2627197019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc960102ef0 msgr2=0x7fc9600fc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:16.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.577+0000 7fc9654dc640 1 --2- 172.21.15.112:0/2627197019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc960102ef0 0x7fc9600fc5b0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fc9480048d0 tx=0x7fc948009da0 comp rx=0 tx=0).stop 2024-09-16T00:47:16.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.577+0000 7fc9654dc640 1 -- 172.21.15.112:0/2627197019 shutdown_connections 2024-09-16T00:47:16.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.577+0000 7fc9654dc640 1 --2- 172.21.15.112:0/2627197019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc960102ef0 0x7fc9600fc5b0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:16.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.577+0000 7fc9654dc640 1 --2- 172.21.15.112:0/2627197019 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc960102550 0x7fc960102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:16.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.577+0000 7fc9654dc640 1 -- 172.21.15.112:0/2627197019 >> 172.21.15.112:0/2627197019 conn(0x7fc9600f80f0 msgr2=0x7fc9600fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:16.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.577+0000 7fc9654dc640 1 -- 172.21.15.112:0/2627197019 shutdown_connections 2024-09-16T00:47:16.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.577+0000 7fc9654dc640 1 -- 172.21.15.112:0/2627197019 wait complete. 2024-09-16T00:47:16.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.578+0000 7fc9654dc640 1 Processor -- start 2024-09-16T00:47:16.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.578+0000 7fc9654dc640 1 -- start start 2024-09-16T00:47:16.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.578+0000 7fc9654dc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc960102550 0x7fc960113500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:16.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.578+0000 7fc9654dc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc960102ef0 0x7fc960113a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:16.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.578+0000 7fc9654dc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc960114090 con 0x7fc960102550 2024-09-16T00:47:16.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.578+0000 7fc9654dc640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc96010d4a0 con 0x7fc960102ef0 2024-09-16T00:47:16.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.578+0000 7fc95effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc960102550 0x7fc960113500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:16.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.578+0000 7fc95effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc960102550 0x7fc960113500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53986/0 (socket says 172.21.15.112:53986) 2024-09-16T00:47:16.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.578+0000 7fc95effd640 1 -- 172.21.15.112:0/755515974 learned_addr learned my addr 172.21.15.112:0/755515974 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:47:16.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.579+0000 7fc95e7fc640 1 --2- 172.21.15.112:0/755515974 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc960102ef0 0x7fc960113a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:16.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.579+0000 7fc95effd640 1 -- 172.21.15.112:0/755515974 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc960102ef0 msgr2=0x7fc960113a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:16.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.579+0000 7fc95effd640 1 --2- 172.21.15.112:0/755515974 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc960102ef0 0x7fc960113a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:16.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.579+0000 7fc95effd640 1 -- 172.21.15.112:0/755515974 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc948005040 con 0x7fc960102550 2024-09-16T00:47:16.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.579+0000 7fc95effd640 1 --2- 172.21.15.112:0/755515974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc960102550 0x7fc960113500 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fc95000cb10 tx=0x7fc950007f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:16.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.579+0000 7fc9577fe640 1 -- 172.21.15.112:0/755515974 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc950004200 con 0x7fc960102550 2024-09-16T00:47:16.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.579+0000 7fc9654dc640 1 -- 172.21.15.112:0/755515974 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc96010d780 con 0x7fc960102550 2024-09-16T00:47:16.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.579+0000 7fc9577fe640 1 -- 172.21.15.112:0/755515974 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fc95000acc0 con 0x7fc960102550 2024-09-16T00:47:16.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.579+0000 7fc9577fe640 1 -- 172.21.15.112:0/755515974 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc950003dc0 con 0x7fc960102550 2024-09-16T00:47:16.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.579+0000 7fc9654dc640 1 -- 172.21.15.112:0/755515974 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc96010dd20 con 0x7fc960102550 2024-09-16T00:47:16.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.580+0000 7fc9654dc640 1 -- 172.21.15.112:0/755515974 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9600fdcd0 con 0x7fc960102550 2024-09-16T00:47:16.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.581+0000 7fc9577fe640 1 -- 172.21.15.112:0/755515974 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fc95000a820 con 0x7fc960102550 2024-09-16T00:47:16.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.581+0000 7fc9577fe640 1 --2- 172.21.15.112:0/755515974 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fc94c06e530 0x7fc94c0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:16.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.581+0000 7fc9577fe640 1 -- 172.21.15.112:0/755515974 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7fc95008ea50 con 0x7fc960102550 2024-09-16T00:47:16.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.581+0000 7fc95e7fc640 1 --2- 172.21.15.112:0/755515974 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fc94c06e530 0x7fc94c0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:16.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.582+0000 7fc95e7fc640 1 --2- 172.21.15.112:0/755515974 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fc94c06e530 0x7fc94c0709e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc96010e780 tx=0x7fc948009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:16.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.586+0000 7fc9577fe640 1 -- 172.21.15.112:0/755515974 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fc9500579c0 con 0x7fc960102550 2024-09-16T00:47:16.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.792+0000 7fc9654dc640 1 -- 172.21.15.112:0/755515974 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc9600fbd50 con 0x7fc960102550 2024-09-16T00:47:16.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.792+0000 7fc9577fe640 1 -- 172.21.15.112:0/755515974 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1914 (secure 0 0 0) 0x7fc950004360 con 0x7fc960102550 2024-09-16T00:47:16.793 INFO:teuthology.orchestra.run.smithi112.stdout:e34 2024-09-16T00:47:16.794 INFO:teuthology.orchestra.run.smithi112.stdout:btime 2024-09-16T00:34:26:619254+0000 2024-09-16T00:47:16.794 INFO:teuthology.orchestra.run.smithi112.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T00:47:16.794 INFO:teuthology.orchestra.run.smithi112.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:47:16.794 INFO:teuthology.orchestra.run.smithi112.stdout:legacy client fscid: 1 2024-09-16T00:47:16.794 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:47:16.794 INFO:teuthology.orchestra.run.smithi112.stdout:Filesystem 'cephfs' (1) 2024-09-16T00:47:16.794 INFO:teuthology.orchestra.run.smithi112.stdout:fs_name cephfs 2024-09-16T00:47:16.794 INFO:teuthology.orchestra.run.smithi112.stdout:epoch 25 2024-09-16T00:47:16.794 INFO:teuthology.orchestra.run.smithi112.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-16T00:47:16.794 INFO:teuthology.orchestra.run.smithi112.stdout:created 2024-09-16T00:23:22.328199+0000 2024-09-16T00:47:16.794 INFO:teuthology.orchestra.run.smithi112.stdout:modified 2024-09-16T00:33:29.012991+0000 2024-09-16T00:47:16.794 INFO:teuthology.orchestra.run.smithi112.stdout:tableserver 0 2024-09-16T00:47:16.794 INFO:teuthology.orchestra.run.smithi112.stdout:root 0 2024-09-16T00:47:16.795 INFO:teuthology.orchestra.run.smithi112.stdout:session_timeout 60 2024-09-16T00:47:16.795 INFO:teuthology.orchestra.run.smithi112.stdout:session_autoclose 300 2024-09-16T00:47:16.795 INFO:teuthology.orchestra.run.smithi112.stdout:max_file_size 1099511627776 2024-09-16T00:47:16.795 INFO:teuthology.orchestra.run.smithi112.stdout:max_xattr_size 65536 2024-09-16T00:47:16.795 INFO:teuthology.orchestra.run.smithi112.stdout:required_client_features {} 2024-09-16T00:47:16.795 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure 0 2024-09-16T00:47:16.795 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure_osd_epoch 82 2024-09-16T00:47:16.795 INFO:teuthology.orchestra.run.smithi112.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:47:16.795 INFO:teuthology.orchestra.run.smithi112.stdout:max_mds 2 2024-09-16T00:47:16.795 INFO:teuthology.orchestra.run.smithi112.stdout:in 0,1 2024-09-16T00:47:16.795 INFO:teuthology.orchestra.run.smithi112.stdout:up {} 2024-09-16T00:47:16.795 INFO:teuthology.orchestra.run.smithi112.stdout:failed 0,1 2024-09-16T00:47:16.795 INFO:teuthology.orchestra.run.smithi112.stdout:damaged 2024-09-16T00:47:16.795 INFO:teuthology.orchestra.run.smithi112.stdout:stopped 2024-09-16T00:47:16.796 INFO:teuthology.orchestra.run.smithi112.stdout:data_pools [3] 2024-09-16T00:47:16.796 INFO:teuthology.orchestra.run.smithi112.stdout:metadata_pool 2 2024-09-16T00:47:16.796 INFO:teuthology.orchestra.run.smithi112.stdout:inline_data enabled 2024-09-16T00:47:16.796 INFO:teuthology.orchestra.run.smithi112.stdout:balancer 2024-09-16T00:47:16.796 INFO:teuthology.orchestra.run.smithi112.stdout:bal_rank_mask -1 2024-09-16T00:47:16.796 INFO:teuthology.orchestra.run.smithi112.stdout:standby_count_wanted 1 2024-09-16T00:47:16.796 INFO:teuthology.orchestra.run.smithi112.stdout:qdb_cluster leader: 0 members: 2024-09-16T00:47:16.796 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:47:16.796 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:47:16.796 INFO:teuthology.orchestra.run.smithi112.stdout:Standby daemons: 2024-09-16T00:47:16.796 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:47:16.796 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.ithwzj{-1:34392} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.112:6826/476229221,v1:172.21.15.112:6827/476229221] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:47:16.796 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.hxfwkh{-1:34404} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.112:6828/2356899876,v1:172.21.15.112:6829/2356899876] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:47:16.796 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.gawwgw{-1:44343} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.169:6824/1635329957,v1:172.21.15.169:6825/1635329957] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:47:16.797 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.lgeqrj{-1:44347} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.169:6826/184810031,v1:172.21.15.169:6827/184810031] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:47:16.797 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 34 2024-09-16T00:47:16.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.794+0000 7fc9654dc640 1 -- 172.21.15.112:0/755515974 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fc94c06e530 msgr2=0x7fc94c0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:16.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.794+0000 7fc9654dc640 1 --2- 172.21.15.112:0/755515974 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fc94c06e530 0x7fc94c0709e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc96010e780 tx=0x7fc948009da0 comp rx=0 tx=0).stop 2024-09-16T00:47:16.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.794+0000 7fc9654dc640 1 -- 172.21.15.112:0/755515974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc960102550 msgr2=0x7fc960113500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:16.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.794+0000 7fc9654dc640 1 --2- 172.21.15.112:0/755515974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc960102550 0x7fc960113500 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fc95000cb10 tx=0x7fc950007f00 comp rx=0 tx=0).stop 2024-09-16T00:47:16.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.795+0000 7fc9654dc640 1 -- 172.21.15.112:0/755515974 shutdown_connections 2024-09-16T00:47:16.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.795+0000 7fc9654dc640 1 --2- 172.21.15.112:0/755515974 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fc94c06e530 0x7fc94c0709e0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:16.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.795+0000 7fc9654dc640 1 --2- 172.21.15.112:0/755515974 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc960102ef0 0x7fc960113a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:16.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.795+0000 7fc9654dc640 1 --2- 172.21.15.112:0/755515974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc960102550 0x7fc960113500 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:16.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.795+0000 7fc9654dc640 1 -- 172.21.15.112:0/755515974 >> 172.21.15.112:0/755515974 conn(0x7fc9600f80f0 msgr2=0x7fc9600f9150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:16.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.795+0000 7fc9654dc640 1 -- 172.21.15.112:0/755515974 shutdown_connections 2024-09-16T00:47:16.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.795+0000 7fc9654dc640 1 -- 172.21.15.112:0/755515974 wait complete. 2024-09-16T00:47:16.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.875+0000 7f14ca68e640 1 -- 172.21.15.112:0/1265499289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14c4102530 msgr2=0x7f14c4102900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:16.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.875+0000 7f14ca68e640 1 --2- 172.21.15.112:0/1265499289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14c4102530 0x7f14c4102900 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f14b40048d0 tx=0x7f14b4009da0 comp rx=0 tx=0).stop 2024-09-16T00:47:16.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.875+0000 7f14ca68e640 1 -- 172.21.15.112:0/1265499289 shutdown_connections 2024-09-16T00:47:16.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.875+0000 7f14ca68e640 1 --2- 172.21.15.112:0/1265499289 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14c4102ed0 0x7f14c40fc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:16.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.875+0000 7f14ca68e640 1 --2- 172.21.15.112:0/1265499289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14c4102530 0x7f14c4102900 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:16.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.875+0000 7f14ca68e640 1 -- 172.21.15.112:0/1265499289 >> 172.21.15.112:0/1265499289 conn(0x7f14c40f80f0 msgr2=0x7f14c40fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:16.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.875+0000 7f14ca68e640 1 -- 172.21.15.112:0/1265499289 shutdown_connections 2024-09-16T00:47:16.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.875+0000 7f14ca68e640 1 -- 172.21.15.112:0/1265499289 wait complete. 2024-09-16T00:47:16.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.875+0000 7f14ca68e640 1 Processor -- start 2024-09-16T00:47:16.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.875+0000 7f14ca68e640 1 -- start start 2024-09-16T00:47:16.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.875+0000 7f14ca68e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14c4102530 0x7f14c406fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:16.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.875+0000 7f14ca68e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14c4102ed0 0x7f14c406ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:16.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.875+0000 7f14ca68e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14c4070490 con 0x7f14c4102530 2024-09-16T00:47:16.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.875+0000 7f14ca68e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14c4070600 con 0x7f14c4102ed0 2024-09-16T00:47:16.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.876+0000 7f14c3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14c4102530 0x7f14c406fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:16.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.876+0000 7f14c3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14c4102530 0x7f14c406fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54004/0 (socket says 172.21.15.112:54004) 2024-09-16T00:47:16.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.876+0000 7f14c3fff640 1 -- 172.21.15.112:0/1531570767 learned_addr learned my addr 172.21.15.112:0/1531570767 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:47:16.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.876+0000 7f14bbfff640 1 --2- 172.21.15.112:0/1531570767 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14c4102ed0 0x7f14c406ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:16.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.876+0000 7f14c3fff640 1 -- 172.21.15.112:0/1531570767 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14c4102ed0 msgr2=0x7f14c406ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:16.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.876+0000 7f14c3fff640 1 --2- 172.21.15.112:0/1531570767 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14c4102ed0 0x7f14c406ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:16.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.876+0000 7f14c3fff640 1 -- 172.21.15.112:0/1531570767 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14b4005040 con 0x7f14c4102530 2024-09-16T00:47:16.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.876+0000 7f14c3fff640 1 --2- 172.21.15.112:0/1531570767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14c4102530 0x7f14c406fa10 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f14b40043d0 tx=0x7f14b4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:16.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.876+0000 7f14c17fa640 1 -- 172.21.15.112:0/1531570767 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f14b4038070 con 0x7f14c4102530 2024-09-16T00:47:16.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.876+0000 7f14c17fa640 1 -- 172.21.15.112:0/1531570767 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f14b40096e0 con 0x7f14c4102530 2024-09-16T00:47:16.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.876+0000 7f14ca68e640 1 -- 172.21.15.112:0/1531570767 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14c4070830 con 0x7f14c4102530 2024-09-16T00:47:16.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.876+0000 7f14c17fa640 1 -- 172.21.15.112:0/1531570767 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f14b403d420 con 0x7f14c4102530 2024-09-16T00:47:16.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.876+0000 7f14ca68e640 1 -- 172.21.15.112:0/1531570767 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14c4119910 con 0x7f14c4102530 2024-09-16T00:47:16.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.877+0000 7f14ca68e640 1 -- 172.21.15.112:0/1531570767 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14c40fdcb0 con 0x7f14c4102530 2024-09-16T00:47:16.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.878+0000 7f14c17fa640 1 -- 172.21.15.112:0/1531570767 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f14b4030020 con 0x7f14c4102530 2024-09-16T00:47:16.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.878+0000 7f14c17fa640 1 --2- 172.21.15.112:0/1531570767 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f14b006e530 0x7f14b00709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:16.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.878+0000 7f14c17fa640 1 -- 172.21.15.112:0/1531570767 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f14b40ae7c0 con 0x7f14c4102530 2024-09-16T00:47:16.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.878+0000 7f14bbfff640 1 --2- 172.21.15.112:0/1531570767 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f14b006e530 0x7f14b00709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:16.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.879+0000 7f14bbfff640 1 --2- 172.21.15.112:0/1531570767 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f14b006e530 0x7f14b00709e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f14c4071350 tx=0x7f14ac00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:16.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:16.882+0000 7f14c17fa640 1 -- 172.21.15.112:0/1531570767 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f14b4077730 con 0x7f14c4102530 2024-09-16T00:47:17.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:17.071+0000 7f14ca68e640 1 -- 172.21.15.112:0/1531570767 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f14c4071100 con 0x7f14b006e530 2024-09-16T00:47:17.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:17.072+0000 7f14c17fa640 1 -- 172.21.15.112:0/1531570767 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f14c4071100 con 0x7f14b006e530 2024-09-16T00:47:17.072 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:47:17.072 INFO:teuthology.orchestra.run.smithi112.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80", 2024-09-16T00:47:17.072 INFO:teuthology.orchestra.run.smithi112.stdout: "in_progress": true, 2024-09-16T00:47:17.072 INFO:teuthology.orchestra.run.smithi112.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T00:47:17.072 INFO:teuthology.orchestra.run.smithi112.stdout: "services_complete": [ 2024-09-16T00:47:17.073 INFO:teuthology.orchestra.run.smithi112.stdout: "mds", 2024-09-16T00:47:17.073 INFO:teuthology.orchestra.run.smithi112.stdout: "osd", 2024-09-16T00:47:17.073 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr", 2024-09-16T00:47:17.073 INFO:teuthology.orchestra.run.smithi112.stdout: "mon", 2024-09-16T00:47:17.073 INFO:teuthology.orchestra.run.smithi112.stdout: "crash" 2024-09-16T00:47:17.073 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-16T00:47:17.073 INFO:teuthology.orchestra.run.smithi112.stdout: "progress": "16/21 daemons upgraded", 2024-09-16T00:47:17.073 INFO:teuthology.orchestra.run.smithi112.stdout: "message": "Currently upgrading mds daemons", 2024-09-16T00:47:17.073 INFO:teuthology.orchestra.run.smithi112.stdout: "is_paused": false 2024-09-16T00:47:17.073 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:47:17.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:17.074+0000 7f14ca68e640 1 -- 172.21.15.112:0/1531570767 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f14b006e530 msgr2=0x7f14b00709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:17.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:17.074+0000 7f14ca68e640 1 --2- 172.21.15.112:0/1531570767 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f14b006e530 0x7f14b00709e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f14c4071350 tx=0x7f14ac00b040 comp rx=0 tx=0).stop 2024-09-16T00:47:17.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:17.074+0000 7f14ca68e640 1 -- 172.21.15.112:0/1531570767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14c4102530 msgr2=0x7f14c406fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:17.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:17.074+0000 7f14ca68e640 1 --2- 172.21.15.112:0/1531570767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14c4102530 0x7f14c406fa10 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f14b40043d0 tx=0x7f14b4009da0 comp rx=0 tx=0).stop 2024-09-16T00:47:17.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:17.074+0000 7f14ca68e640 1 -- 172.21.15.112:0/1531570767 shutdown_connections 2024-09-16T00:47:17.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:17.074+0000 7f14ca68e640 1 --2- 172.21.15.112:0/1531570767 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f14b006e530 0x7f14b00709e0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:17.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:17.074+0000 7f14ca68e640 1 --2- 172.21.15.112:0/1531570767 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14c4102ed0 0x7f14c406ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:17.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:17.074+0000 7f14ca68e640 1 --2- 172.21.15.112:0/1531570767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14c4102530 0x7f14c406fa10 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:17.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:17.074+0000 7f14ca68e640 1 -- 172.21.15.112:0/1531570767 >> 172.21.15.112:0/1531570767 conn(0x7f14c40f80f0 msgr2=0x7f14c40f90f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:17.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:17.075+0000 7f14ca68e640 1 -- 172.21.15.112:0/1531570767 shutdown_connections 2024-09-16T00:47:17.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:17.075+0000 7f14ca68e640 1 -- 172.21.15.112:0/1531570767 wait complete. 2024-09-16T00:47:17.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:17.152+0000 7f9fc26e3640 1 -- 172.21.15.112:0/1083965166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fbc1014c0 msgr2=0x7f9fbc101890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:17.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:17.152+0000 7f9fc26e3640 1 --2- 172.21.15.112:0/1083965166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fbc1014c0 0x7f9fbc101890 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f9fac0048d0 tx=0x7f9fac009da0 comp rx=0 tx=0).stop 2024-09-16T00:47:17.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:17.152+0000 7f9fc26e3640 1 -- 172.21.15.112:0/1083965166 shutdown_connections 2024-09-16T00:47:17.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:17.152+0000 7f9fc26e3640 1 --2- 172.21.15.112:0/1083965166 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9fbc0fb480 0x7f9fbc0fb8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:17.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:17.152+0000 7f9fc26e3640 1 --2- 172.21.15.112:0/1083965166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fbc1014c0 0x7f9fbc101890 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:17.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:17.152+0000 7f9fc26e3640 1 -- 172.21.15.112:0/1083965166 >> 172.21.15.112:0/1083965166 conn(0x7f9fbc0f7000 msgr2=0x7f9fbc0f9410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:17.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:17.152+0000 7f9fc26e3640 1 -- 172.21.15.112:0/1083965166 shutdown_connections 2024-09-16T00:47:17.154 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:47:16 smithi112 ceph-mon[97750]: from='client.35108 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:47:17.154 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:47:16 smithi112 ceph-mon[97750]: from='client.35112 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:47:17.154 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:47:16 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/2922647337' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:47:17.154 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:47:16 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:47:17.154 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:47:16 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/755515974' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:47:17.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:17.153+0000 7f9fc26e3640 1 -- 172.21.15.112:0/1083965166 wait complete. 2024-09-16T00:47:17.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:17.153+0000 7f9fc26e3640 1 Processor -- start 2024-09-16T00:47:17.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:17.153+0000 7f9fc26e3640 1 -- start start 2024-09-16T00:47:17.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:17.154+0000 7f9fc26e3640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9fbc0fb480 0x7f9fbc10e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:17.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:17.154+0000 7f9fc26e3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fbc1014c0 0x7f9fbc10e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:17.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:17.154+0000 7f9fc26e3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fbc10ee00 con 0x7f9fbc1014c0 2024-09-16T00:47:17.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:17.154+0000 7f9fc26e3640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fbc10ef70 con 0x7f9fbc0fb480 2024-09-16T00:47:17.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:17.154+0000 7f9fbb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fbc1014c0 0x7f9fbc10e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:17.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:17.154+0000 7f9fbb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fbc1014c0 0x7f9fbc10e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54022/0 (socket says 172.21.15.112:54022) 2024-09-16T00:47:17.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:17.154+0000 7f9fbb7fe640 1 -- 172.21.15.112:0/2034889919 learned_addr learned my addr 172.21.15.112:0/2034889919 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:47:17.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:17.154+0000 7f9fbbfff640 1 --2- 172.21.15.112:0/2034889919 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9fbc0fb480 0x7f9fbc10e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:17.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:17.154+0000 7f9fbb7fe640 1 -- 172.21.15.112:0/2034889919 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9fbc0fb480 msgr2=0x7f9fbc10e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:17.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:17.154+0000 7f9fbb7fe640 1 --2- 172.21.15.112:0/2034889919 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9fbc0fb480 0x7f9fbc10e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:17.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:17.154+0000 7f9fbb7fe640 1 -- 172.21.15.112:0/2034889919 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fac005040 con 0x7f9fbc1014c0 2024-09-16T00:47:17.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:17.154+0000 7f9fbb7fe640 1 --2- 172.21.15.112:0/2034889919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fbc1014c0 0x7f9fbc10e720 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f9fb000b790 tx=0x7f9fb00091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:17.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:17.154+0000 7f9fb8ff9640 1 -- 172.21.15.112:0/2034889919 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9fb000c700 con 0x7f9fbc1014c0 2024-09-16T00:47:17.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:17.154+0000 7f9fb8ff9640 1 -- 172.21.15.112:0/2034889919 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f9fb000cd00 con 0x7f9fbc1014c0 2024-09-16T00:47:17.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:17.154+0000 7f9fc26e3640 1 -- 172.21.15.112:0/2034889919 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fbc117250 con 0x7f9fbc1014c0 2024-09-16T00:47:17.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:17.155+0000 7f9fb8ff9640 1 -- 172.21.15.112:0/2034889919 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9fb0015e80 con 0x7f9fbc1014c0 2024-09-16T00:47:17.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:17.155+0000 7f9fc26e3640 1 -- 172.21.15.112:0/2034889919 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fbc117720 con 0x7f9fbc1014c0 2024-09-16T00:47:17.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:17.156+0000 7f9fc26e3640 1 -- 172.21.15.112:0/2034889919 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fbc0fcbc0 con 0x7f9fbc1014c0 2024-09-16T00:47:17.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:17.157+0000 7f9fb8ff9640 1 -- 172.21.15.112:0/2034889919 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f9fb0012070 con 0x7f9fbc1014c0 2024-09-16T00:47:17.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:17.157+0000 7f9fb8ff9640 1 --2- 172.21.15.112:0/2034889919 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f9fa406e530 0x7f9fa40709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:17.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:17.157+0000 7f9fb8ff9640 1 -- 172.21.15.112:0/2034889919 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f9fb008dbe0 con 0x7f9fbc1014c0 2024-09-16T00:47:17.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:17.157+0000 7f9fbbfff640 1 --2- 172.21.15.112:0/2034889919 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f9fa406e530 0x7f9fa40709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:17.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:17.158+0000 7f9fbbfff640 1 --2- 172.21.15.112:0/2034889919 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f9fa406e530 0x7f9fa40709e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f9fac009ac0 tx=0x7f9fac035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:17.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:17.163+0000 7f9fb8ff9640 1 -- 172.21.15.112:0/2034889919 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f9fb0056b50 con 0x7f9fbc1014c0 2024-09-16T00:47:17.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:47:16 smithi169 ceph-mon[77626]: from='client.35108 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:47:17.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:47:16 smithi169 ceph-mon[77626]: from='client.35112 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:47:17.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:47:16 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/2922647337' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:47:17.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:47:16 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:47:17.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:47:16 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/755515974' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:47:17.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:17.391+0000 7f9fc26e3640 1 -- 172.21.15.112:0/2034889919 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f9fbc04b720 con 0x7f9fbc1014c0 2024-09-16T00:47:17.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:17.391+0000 7f9fb8ff9640 1 -- 172.21.15.112:0/2034889919 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f9fb005a210 con 0x7f9fbc1014c0 2024-09-16T00:47:17.391 INFO:teuthology.orchestra.run.smithi112.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-16T00:47:17.391 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-16T00:47:17.391 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs is degraded 2024-09-16T00:47:17.392 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T00:47:17.392 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T00:47:17.392 INFO:teuthology.orchestra.run.smithi112.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-16T00:47:17.392 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-16T00:47:17.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:17.393+0000 7f9fc26e3640 1 -- 172.21.15.112:0/2034889919 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f9fa406e530 msgr2=0x7f9fa40709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:17.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:17.393+0000 7f9fc26e3640 1 --2- 172.21.15.112:0/2034889919 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f9fa406e530 0x7f9fa40709e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f9fac009ac0 tx=0x7f9fac035040 comp rx=0 tx=0).stop 2024-09-16T00:47:17.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:17.393+0000 7f9fc26e3640 1 -- 172.21.15.112:0/2034889919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fbc1014c0 msgr2=0x7f9fbc10e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:17.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:17.393+0000 7f9fc26e3640 1 --2- 172.21.15.112:0/2034889919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fbc1014c0 0x7f9fbc10e720 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f9fb000b790 tx=0x7f9fb00091d0 comp rx=0 tx=0).stop 2024-09-16T00:47:17.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:17.393+0000 7f9fc26e3640 1 -- 172.21.15.112:0/2034889919 shutdown_connections 2024-09-16T00:47:17.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:17.393+0000 7f9fc26e3640 1 --2- 172.21.15.112:0/2034889919 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f9fa406e530 0x7f9fa40709e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:17.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:17.393+0000 7f9fc26e3640 1 --2- 172.21.15.112:0/2034889919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fbc1014c0 0x7f9fbc10e720 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:17.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:17.393+0000 7f9fc26e3640 1 --2- 172.21.15.112:0/2034889919 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9fbc0fb480 0x7f9fbc10e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:17.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:17.393+0000 7f9fc26e3640 1 -- 172.21.15.112:0/2034889919 >> 172.21.15.112:0/2034889919 conn(0x7f9fbc0f7000 msgr2=0x7f9fbc0f88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:17.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:17.393+0000 7f9fc26e3640 1 -- 172.21.15.112:0/2034889919 shutdown_connections 2024-09-16T00:47:17.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:17.394+0000 7f9fc26e3640 1 -- 172.21.15.112:0/2034889919 wait complete. 2024-09-16T00:47:18.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:47:17 smithi169 ceph-mon[77626]: from='client.35116 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:47:18.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:47:17 smithi169 ceph-mon[77626]: pgmap v586: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:47:18.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:47:17 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/2034889919' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:47:18.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:47:17 smithi112 ceph-mon[97750]: from='client.35116 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:47:18.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:47:17 smithi112 ceph-mon[97750]: pgmap v586: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:47:18.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:47:17 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/2034889919' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:47:19.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:47:18 smithi169 ceph-mon[77626]: from='client.35128 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:47:19.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:47:18 smithi112 ceph-mon[97750]: from='client.35128 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:47:20.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:47:19 smithi169 ceph-mon[77626]: pgmap v587: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:47:20.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:47:19 smithi112 ceph-mon[97750]: pgmap v587: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:47:22.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:47:21 smithi169 ceph-mon[77626]: pgmap v588: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:47:22.314 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:47:21 smithi112 ceph-mon[97750]: pgmap v588: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:47:24.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:47:23 smithi169 ceph-mon[77626]: pgmap v589: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:47:24.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:47:23 smithi112 ceph-mon[97750]: pgmap v589: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:47:26.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:47:25 smithi169 ceph-mon[77626]: pgmap v590: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:47:26.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:47:25 smithi112 ceph-mon[97750]: pgmap v590: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:47:28.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:47:27 smithi169 ceph-mon[77626]: pgmap v591: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:47:28.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:47:27 smithi112 ceph-mon[97750]: pgmap v591: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:47:30.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:47:29 smithi169 ceph-mon[77626]: pgmap v592: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:47:30.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:47:29 smithi112 ceph-mon[97750]: pgmap v592: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:47:31.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:47:30 smithi169 ceph-mon[77626]: pgmap v593: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:47:31.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:47:30 smithi112 ceph-mon[97750]: pgmap v593: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:47:32.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:47:31 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:47:32.314 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:47:31 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:47:33.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:47:32 smithi169 ceph-mon[77626]: pgmap v594: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:47:33.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:47:32 smithi112 ceph-mon[97750]: pgmap v594: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:47:36.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:47:35 smithi112 ceph-mon[97750]: pgmap v595: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:47:36.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:47:35 smithi169 ceph-mon[77626]: pgmap v595: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:47:38.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:47:37 smithi112 ceph-mon[97750]: pgmap v596: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:47:38.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:47:37 smithi169 ceph-mon[77626]: pgmap v596: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:47:40.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:47:39 smithi112 ceph-mon[97750]: pgmap v597: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:47:40.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:47:39 smithi169 ceph-mon[77626]: pgmap v597: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:47:42.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:47:41 smithi112 ceph-mon[97750]: pgmap v598: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:47:42.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:47:41 smithi169 ceph-mon[77626]: pgmap v598: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:47:44.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:47:43 smithi112 ceph-mon[97750]: pgmap v599: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:47:44.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:47:43 smithi169 ceph-mon[77626]: pgmap v599: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:47:46.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:47:45 smithi112 ceph-mon[97750]: pgmap v600: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:47:46.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:47:45 smithi169 ceph-mon[77626]: pgmap v600: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:47:47.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:47:46 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:47:47.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:47:46 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:47:47.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.485+0000 7fa90e743640 1 -- 172.21.15.112:0/1490798026 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa908102560 msgr2=0x7fa908102930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:47.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.485+0000 7fa90e743640 1 --2- 172.21.15.112:0/1490798026 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa908102560 0x7fa908102930 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa8f80048d0 tx=0x7fa8f8009da0 comp rx=0 tx=0).stop 2024-09-16T00:47:47.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.485+0000 7fa90e743640 1 -- 172.21.15.112:0/1490798026 shutdown_connections 2024-09-16T00:47:47.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.485+0000 7fa90e743640 1 --2- 172.21.15.112:0/1490798026 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa908102f00 0x7fa9080fc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:47.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.485+0000 7fa90e743640 1 --2- 172.21.15.112:0/1490798026 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa908102560 0x7fa908102930 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:47.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.485+0000 7fa90e743640 1 -- 172.21.15.112:0/1490798026 >> 172.21.15.112:0/1490798026 conn(0x7fa9080f8120 msgr2=0x7fa9080fa530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:47.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.485+0000 7fa90e743640 1 -- 172.21.15.112:0/1490798026 shutdown_connections 2024-09-16T00:47:47.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.485+0000 7fa90e743640 1 -- 172.21.15.112:0/1490798026 wait complete. 2024-09-16T00:47:47.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.485+0000 7fa90e743640 1 Processor -- start 2024-09-16T00:47:47.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.486+0000 7fa90e743640 1 -- start start 2024-09-16T00:47:47.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.486+0000 7fa90e743640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa908102560 0x7fa90806fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:47.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.486+0000 7fa90e743640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa908102f00 0x7fa90806ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:47.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.486+0000 7fa90e743640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa908070490 con 0x7fa908102f00 2024-09-16T00:47:47.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.486+0000 7fa90e743640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa908070600 con 0x7fa908102560 2024-09-16T00:47:47.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.486+0000 7fa9077fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa908102f00 0x7fa90806ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:47.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.486+0000 7fa9077fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa908102f00 0x7fa90806ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51182/0 (socket says 172.21.15.112:51182) 2024-09-16T00:47:47.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.486+0000 7fa9077fe640 1 -- 172.21.15.112:0/3272470561 learned_addr learned my addr 172.21.15.112:0/3272470561 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:47:47.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.486+0000 7fa907fff640 1 --2- 172.21.15.112:0/3272470561 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa908102560 0x7fa90806fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:47.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.487+0000 7fa9077fe640 1 -- 172.21.15.112:0/3272470561 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa908102560 msgr2=0x7fa90806fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:47.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.487+0000 7fa9077fe640 1 --2- 172.21.15.112:0/3272470561 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa908102560 0x7fa90806fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:47.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.487+0000 7fa9077fe640 1 -- 172.21.15.112:0/3272470561 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8f8005040 con 0x7fa908102f00 2024-09-16T00:47:47.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.487+0000 7fa9077fe640 1 --2- 172.21.15.112:0/3272470561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa908102f00 0x7fa90806ff50 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc00c740 tx=0x7fa8fc0043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:47.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.487+0000 7fa904ff9640 1 -- 172.21.15.112:0/3272470561 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa8fc012810 con 0x7fa908102f00 2024-09-16T00:47:47.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.487+0000 7fa90e743640 1 -- 172.21.15.112:0/3272470561 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa908070890 con 0x7fa908102f00 2024-09-16T00:47:47.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.487+0000 7fa904ff9640 1 -- 172.21.15.112:0/3272470561 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fa8fc012e10 con 0x7fa908102f00 2024-09-16T00:47:47.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.487+0000 7fa904ff9640 1 -- 172.21.15.112:0/3272470561 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa8fc00a2f0 con 0x7fa908102f00 2024-09-16T00:47:47.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.487+0000 7fa90e743640 1 -- 172.21.15.112:0/3272470561 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa908119970 con 0x7fa908102f00 2024-09-16T00:47:47.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.488+0000 7fa90e743640 1 -- 172.21.15.112:0/3272470561 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9080fdce0 con 0x7fa908102f00 2024-09-16T00:47:47.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.489+0000 7fa904ff9640 1 -- 172.21.15.112:0/3272470561 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fa8fc012970 con 0x7fa908102f00 2024-09-16T00:47:47.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.489+0000 7fa904ff9640 1 --2- 172.21.15.112:0/3272470561 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa8f006e530 0x7fa8f00709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:47.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.489+0000 7fa907fff640 1 --2- 172.21.15.112:0/3272470561 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa8f006e530 0x7fa8f00709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:47.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.489+0000 7fa904ff9640 1 -- 172.21.15.112:0/3272470561 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7fa8fc08e860 con 0x7fa908102f00 2024-09-16T00:47:47.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.490+0000 7fa907fff640 1 --2- 172.21.15.112:0/3272470561 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa8f006e530 0x7fa8f00709e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa8f8007ac0 tx=0x7fa8f8035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:47.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.494+0000 7fa904ff9640 1 -- 172.21.15.112:0/3272470561 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fa8fc0577d0 con 0x7fa908102f00 2024-09-16T00:47:47.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.678+0000 7fa90e743640 1 -- 172.21.15.112:0/3272470561 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa90804b720 con 0x7fa8f006e530 2024-09-16T00:47:47.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.679+0000 7fa904ff9640 1 -- 172.21.15.112:0/3272470561 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fa90804b720 con 0x7fa8f006e530 2024-09-16T00:47:47.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.681+0000 7fa90e743640 1 -- 172.21.15.112:0/3272470561 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa8f006e530 msgr2=0x7fa8f00709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:47.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.681+0000 7fa90e743640 1 --2- 172.21.15.112:0/3272470561 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa8f006e530 0x7fa8f00709e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa8f8007ac0 tx=0x7fa8f8035040 comp rx=0 tx=0).stop 2024-09-16T00:47:47.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.681+0000 7fa90e743640 1 -- 172.21.15.112:0/3272470561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa908102f00 msgr2=0x7fa90806ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:47.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.681+0000 7fa90e743640 1 --2- 172.21.15.112:0/3272470561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa908102f00 0x7fa90806ff50 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc00c740 tx=0x7fa8fc0043c0 comp rx=0 tx=0).stop 2024-09-16T00:47:47.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.681+0000 7fa90e743640 1 -- 172.21.15.112:0/3272470561 shutdown_connections 2024-09-16T00:47:47.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.681+0000 7fa90e743640 1 --2- 172.21.15.112:0/3272470561 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa8f006e530 0x7fa8f00709e0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:47.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.681+0000 7fa90e743640 1 --2- 172.21.15.112:0/3272470561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa908102f00 0x7fa90806ff50 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:47.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.681+0000 7fa90e743640 1 --2- 172.21.15.112:0/3272470561 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa908102560 0x7fa90806fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:47.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.681+0000 7fa90e743640 1 -- 172.21.15.112:0/3272470561 >> 172.21.15.112:0/3272470561 conn(0x7fa9080f8120 msgr2=0x7fa9080f90a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:47.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.681+0000 7fa90e743640 1 -- 172.21.15.112:0/3272470561 shutdown_connections 2024-09-16T00:47:47.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.681+0000 7fa90e743640 1 -- 172.21.15.112:0/3272470561 wait complete. 2024-09-16T00:47:47.690 INFO:teuthology.orchestra.run.smithi112.stdout:true 2024-09-16T00:47:47.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.761+0000 7f7113f14640 1 -- 172.21.15.112:0/4272620394 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f710c102550 msgr2=0x7f710c102920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:47.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.761+0000 7f7113f14640 1 --2- 172.21.15.112:0/4272620394 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f710c102550 0x7f710c102920 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f71080048d0 tx=0x7f7108009da0 comp rx=0 tx=0).stop 2024-09-16T00:47:47.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.761+0000 7f7113f14640 1 -- 172.21.15.112:0/4272620394 shutdown_connections 2024-09-16T00:47:47.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.761+0000 7f7113f14640 1 --2- 172.21.15.112:0/4272620394 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f710c102ef0 0x7f710c0fc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:47.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.761+0000 7f7113f14640 1 --2- 172.21.15.112:0/4272620394 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f710c102550 0x7f710c102920 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:47.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.761+0000 7f7113f14640 1 -- 172.21.15.112:0/4272620394 >> 172.21.15.112:0/4272620394 conn(0x7f710c0f80f0 msgr2=0x7f710c0fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:47.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.761+0000 7f7113f14640 1 -- 172.21.15.112:0/4272620394 shutdown_connections 2024-09-16T00:47:47.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.761+0000 7f7113f14640 1 -- 172.21.15.112:0/4272620394 wait complete. 2024-09-16T00:47:47.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.761+0000 7f7113f14640 1 Processor -- start 2024-09-16T00:47:47.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.761+0000 7f7113f14640 1 -- start start 2024-09-16T00:47:47.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.762+0000 7f7113f14640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f710c102550 0x7f710c113540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:47.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.762+0000 7f7113f14640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f710c102ef0 0x7f710c113a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:47.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.762+0000 7f7113f14640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f710c1140d0 con 0x7f710c102ef0 2024-09-16T00:47:47.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.762+0000 7f7113f14640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f710c10d4e0 con 0x7f710c102550 2024-09-16T00:47:47.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.762+0000 7f71114af640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f710c102ef0 0x7f710c113a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:47.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.762+0000 7f71114af640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f710c102ef0 0x7f710c113a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51210/0 (socket says 172.21.15.112:51210) 2024-09-16T00:47:47.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.762+0000 7f71114af640 1 -- 172.21.15.112:0/2868511297 learned_addr learned my addr 172.21.15.112:0/2868511297 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:47:47.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.762+0000 7f7111cb0640 1 --2- 172.21.15.112:0/2868511297 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f710c102550 0x7f710c113540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:47.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.762+0000 7f71114af640 1 -- 172.21.15.112:0/2868511297 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f710c102550 msgr2=0x7f710c113540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:47.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.762+0000 7f71114af640 1 --2- 172.21.15.112:0/2868511297 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f710c102550 0x7f710c113540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:47.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.762+0000 7f71114af640 1 -- 172.21.15.112:0/2868511297 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7108005040 con 0x7f710c102ef0 2024-09-16T00:47:47.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.762+0000 7f71114af640 1 --2- 172.21.15.112:0/2868511297 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f710c102ef0 0x7f710c113a80 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f7100002840 tx=0x7f7100004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:47.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.762+0000 7f70fe7fc640 1 -- 172.21.15.112:0/2868511297 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7100007c30 con 0x7f710c102ef0 2024-09-16T00:47:47.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.763+0000 7f7113f14640 1 -- 172.21.15.112:0/2868511297 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f710c10d7c0 con 0x7f710c102ef0 2024-09-16T00:47:47.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.763+0000 7f70fe7fc640 1 -- 172.21.15.112:0/2868511297 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f710000f040 con 0x7f710c102ef0 2024-09-16T00:47:47.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.763+0000 7f7113f14640 1 -- 172.21.15.112:0/2868511297 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f710c10dd60 con 0x7f710c102ef0 2024-09-16T00:47:47.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.763+0000 7f70fe7fc640 1 -- 172.21.15.112:0/2868511297 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f710000d970 con 0x7f710c102ef0 2024-09-16T00:47:47.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.764+0000 7f7113f14640 1 -- 172.21.15.112:0/2868511297 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f710c0fdcd0 con 0x7f710c102ef0 2024-09-16T00:47:47.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.764+0000 7f70fe7fc640 1 -- 172.21.15.112:0/2868511297 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f7100009600 con 0x7f710c102ef0 2024-09-16T00:47:47.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.765+0000 7f70fe7fc640 1 --2- 172.21.15.112:0/2868511297 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f70f806e530 0x7f70f80709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:47.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.765+0000 7f70fe7fc640 1 -- 172.21.15.112:0/2868511297 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f710008da40 con 0x7f710c102ef0 2024-09-16T00:47:47.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.765+0000 7f7111cb0640 1 --2- 172.21.15.112:0/2868511297 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f70f806e530 0x7f70f80709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:47.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.765+0000 7f7111cb0640 1 --2- 172.21.15.112:0/2868511297 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f70f806e530 0x7f70f80709e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7108009ac0 tx=0x7f7108035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:47.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.769+0000 7f70fe7fc640 1 -- 172.21.15.112:0/2868511297 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f7100056a60 con 0x7f710c102ef0 2024-09-16T00:47:47.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.952+0000 7f7113f14640 1 -- 172.21.15.112:0/2868511297 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f710c10ec20 con 0x7f70f806e530 2024-09-16T00:47:47.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.953+0000 7f70fe7fc640 1 -- 172.21.15.112:0/2868511297 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f710c10ec20 con 0x7f70f806e530 2024-09-16T00:47:47.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.955+0000 7f7113f14640 1 -- 172.21.15.112:0/2868511297 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f70f806e530 msgr2=0x7f70f80709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:47.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.955+0000 7f7113f14640 1 --2- 172.21.15.112:0/2868511297 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f70f806e530 0x7f70f80709e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7108009ac0 tx=0x7f7108035040 comp rx=0 tx=0).stop 2024-09-16T00:47:47.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.955+0000 7f7113f14640 1 -- 172.21.15.112:0/2868511297 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f710c102ef0 msgr2=0x7f710c113a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:47.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.955+0000 7f7113f14640 1 --2- 172.21.15.112:0/2868511297 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f710c102ef0 0x7f710c113a80 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f7100002840 tx=0x7f7100004570 comp rx=0 tx=0).stop 2024-09-16T00:47:47.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.955+0000 7f7113f14640 1 -- 172.21.15.112:0/2868511297 shutdown_connections 2024-09-16T00:47:47.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.955+0000 7f7113f14640 1 --2- 172.21.15.112:0/2868511297 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f70f806e530 0x7f70f80709e0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:47.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.955+0000 7f7113f14640 1 --2- 172.21.15.112:0/2868511297 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f710c102ef0 0x7f710c113a80 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:47.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.955+0000 7f7113f14640 1 --2- 172.21.15.112:0/2868511297 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f710c102550 0x7f710c113540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:47.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.955+0000 7f7113f14640 1 -- 172.21.15.112:0/2868511297 >> 172.21.15.112:0/2868511297 conn(0x7f710c0f80f0 msgr2=0x7f710c0f9160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:47.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.955+0000 7f7113f14640 1 -- 172.21.15.112:0/2868511297 shutdown_connections 2024-09-16T00:47:47.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:47.955+0000 7f7113f14640 1 -- 172.21.15.112:0/2868511297 wait complete. 2024-09-16T00:47:48.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.033+0000 7fa64a7e2640 1 -- 172.21.15.112:0/2036792587 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6440fb4a0 msgr2=0x7fa6440fb910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:48.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.033+0000 7fa64a7e2640 1 --2- 172.21.15.112:0/2036792587 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6440fb4a0 0x7fa6440fb910 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fa6380048d0 tx=0x7fa638009da0 comp rx=0 tx=0).stop 2024-09-16T00:47:48.033 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:47:47 smithi112 ceph-mon[97750]: pgmap v601: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:47:48.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.033+0000 7fa64a7e2640 1 -- 172.21.15.112:0/2036792587 shutdown_connections 2024-09-16T00:47:48.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.033+0000 7fa64a7e2640 1 --2- 172.21.15.112:0/2036792587 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6440fb4a0 0x7fa6440fb910 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:48.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.033+0000 7fa64a7e2640 1 --2- 172.21.15.112:0/2036792587 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6441014e0 0x7fa6441018b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:48.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.033+0000 7fa64a7e2640 1 -- 172.21.15.112:0/2036792587 >> 172.21.15.112:0/2036792587 conn(0x7fa6440f7060 msgr2=0x7fa6440f9470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:48.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.033+0000 7fa64a7e2640 1 -- 172.21.15.112:0/2036792587 shutdown_connections 2024-09-16T00:47:48.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.033+0000 7fa64a7e2640 1 -- 172.21.15.112:0/2036792587 wait complete. 2024-09-16T00:47:48.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.034+0000 7fa64a7e2640 1 Processor -- start 2024-09-16T00:47:48.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.034+0000 7fa64a7e2640 1 -- start start 2024-09-16T00:47:48.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.034+0000 7fa64a7e2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6440fb4a0 0x7fa644112130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:48.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.034+0000 7fa64a7e2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6441014e0 0x7fa644112670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:48.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.034+0000 7fa64a7e2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa644112d00 con 0x7fa6441014e0 2024-09-16T00:47:48.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.034+0000 7fa64a7e2640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa64410c0b0 con 0x7fa6440fb4a0 2024-09-16T00:47:48.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.035+0000 7fa6437fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6441014e0 0x7fa644112670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:48.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.035+0000 7fa6437fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6441014e0 0x7fa644112670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51214/0 (socket says 172.21.15.112:51214) 2024-09-16T00:47:48.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.035+0000 7fa6437fe640 1 -- 172.21.15.112:0/1311061133 learned_addr learned my addr 172.21.15.112:0/1311061133 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:47:48.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.035+0000 7fa643fff640 1 --2- 172.21.15.112:0/1311061133 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6440fb4a0 0x7fa644112130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:48.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.035+0000 7fa6437fe640 1 -- 172.21.15.112:0/1311061133 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6440fb4a0 msgr2=0x7fa644112130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:48.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.035+0000 7fa6437fe640 1 --2- 172.21.15.112:0/1311061133 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6440fb4a0 0x7fa644112130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:48.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.035+0000 7fa6437fe640 1 -- 172.21.15.112:0/1311061133 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa638005040 con 0x7fa6441014e0 2024-09-16T00:47:48.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.035+0000 7fa6437fe640 1 --2- 172.21.15.112:0/1311061133 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6441014e0 0x7fa644112670 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fa638000c00 tx=0x7fa638009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:48.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.035+0000 7fa640ff9640 1 -- 172.21.15.112:0/1311061133 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa63803a070 con 0x7fa6441014e0 2024-09-16T00:47:48.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.035+0000 7fa640ff9640 1 -- 172.21.15.112:0/1311061133 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fa6380338c0 con 0x7fa6441014e0 2024-09-16T00:47:48.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.035+0000 7fa64a7e2640 1 -- 172.21.15.112:0/1311061133 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa64410c330 con 0x7fa6441014e0 2024-09-16T00:47:48.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.035+0000 7fa640ff9640 1 -- 172.21.15.112:0/1311061133 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa63803e8f0 con 0x7fa6441014e0 2024-09-16T00:47:48.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.035+0000 7fa64a7e2640 1 -- 172.21.15.112:0/1311061133 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa64410c790 con 0x7fa6441014e0 2024-09-16T00:47:48.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.037+0000 7fa64a7e2640 1 -- 172.21.15.112:0/1311061133 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6440fcbe0 con 0x7fa6441014e0 2024-09-16T00:47:48.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.037+0000 7fa640ff9640 1 -- 172.21.15.112:0/1311061133 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fa638030020 con 0x7fa6441014e0 2024-09-16T00:47:48.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.037+0000 7fa640ff9640 1 --2- 172.21.15.112:0/1311061133 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa62c06e530 0x7fa62c0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:48.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.037+0000 7fa640ff9640 1 -- 172.21.15.112:0/1311061133 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7fa6380b0380 con 0x7fa6441014e0 2024-09-16T00:47:48.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.037+0000 7fa643fff640 1 --2- 172.21.15.112:0/1311061133 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa62c06e530 0x7fa62c0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:48.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.038+0000 7fa643fff640 1 --2- 172.21.15.112:0/1311061133 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa62c06e530 0x7fa62c0709e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa634009b50 tx=0x7fa634009120 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:48.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.042+0000 7fa640ff9640 1 -- 172.21.15.112:0/1311061133 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fa638079420 con 0x7fa6441014e0 2024-09-16T00:47:48.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.215+0000 7fa64a7e2640 1 -- 172.21.15.112:0/1311061133 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa644108510 con 0x7fa62c06e530 2024-09-16T00:47:48.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.224+0000 7fa640ff9640 1 -- 172.21.15.112:0/1311061133 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7fa644108510 con 0x7fa62c06e530 2024-09-16T00:47:48.225 INFO:teuthology.orchestra.run.smithi112.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T00:47:48.225 INFO:teuthology.orchestra.run.smithi112.stdout:alertmanager.smithi112 smithi112 *:9093,9094 running (20m) 13m ago 29m 24.7M - 0.25.0 c8568f914cd2 d85ea0562022 2024-09-16T00:47:48.225 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi112 smithi112 running (17m) 13m ago 29m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c e4fe58f9702a 2024-09-16T00:47:48.225 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi169 smithi169 running (17m) 13m ago 28m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c 3ba2c81d527f 2024-09-16T00:47:48.225 INFO:teuthology.orchestra.run.smithi112.stdout:grafana.smithi112 smithi112 *:3000 running (19m) 13m ago 29m 82.1M - 9.4.12 2bacad6d85d8 51f9d2013da2 2024-09-16T00:47:48.225 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.hxfwkh smithi112 running (13m) 13m ago 24m 14.2M - 19.1.1-326-g8921c8ef f1455bd0219c 1723069902d6 2024-09-16T00:47:48.225 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.ithwzj smithi112 running (14m) 13m ago 24m 12.2M - 19.1.1-326-g8921c8ef f1455bd0219c e7aad5b81db5 2024-09-16T00:47:48.225 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.gawwgw smithi169 running (13m) 13m ago 24m 15.7M - 19.1.1-326-g8921c8ef f1455bd0219c 469be459ee12 2024-09-16T00:47:48.226 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.lgeqrj smithi169 running (13m) 13m ago 24m 13.3M - 19.1.1-326-g8921c8ef f1455bd0219c 31f0fbd2395d 2024-09-16T00:47:48.226 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi112.xrkkbc smithi112 *:8443,9283,8765 running (20m) 13m ago 30m 504M - 19.1.1-326-g8921c8ef f1455bd0219c a981f7fd2810 2024-09-16T00:47:48.226 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi169.mtrobk smithi169 *:8443,9283,8765 running (19m) 13m ago 28m 442M - 19.1.1-326-g8921c8ef f1455bd0219c 4096e5369c2d 2024-09-16T00:47:48.226 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi112 smithi112 running (18m) 13m ago 31m 49.8M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 5aaec033c042 2024-09-16T00:47:48.226 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi169 smithi169 running (18m) 13m ago 28m 38.9M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 7bc609155f3f 2024-09-16T00:47:48.226 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi112 smithi112 *:9100 running (21m) 13m ago 29m 15.5M - 1.5.0 0da6a335fe13 6404d1d601e2 2024-09-16T00:47:48.226 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi169 smithi169 *:9100 running (20m) 13m ago 28m 15.4M - 1.5.0 0da6a335fe13 5505baf702a6 2024-09-16T00:47:48.226 INFO:teuthology.orchestra.run.smithi112.stdout:osd.0 smithi112 running (17m) 13m ago 28m 105M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 53efc9397017 2024-09-16T00:47:48.226 INFO:teuthology.orchestra.run.smithi112.stdout:osd.1 smithi112 running (16m) 13m ago 27m 98.9M 1557M 19.1.1-326-g8921c8ef f1455bd0219c fe6f1a4a2cf4 2024-09-16T00:47:48.226 INFO:teuthology.orchestra.run.smithi112.stdout:osd.2 smithi112 running (16m) 13m ago 27m 102M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 5f7863712057 2024-09-16T00:47:48.226 INFO:teuthology.orchestra.run.smithi112.stdout:osd.3 smithi169 running (15m) 13m ago 26m 92.1M 2581M 19.1.1-326-g8921c8ef f1455bd0219c ee07b5577810 2024-09-16T00:47:48.226 INFO:teuthology.orchestra.run.smithi112.stdout:osd.4 smithi169 running (15m) 13m ago 26m 103M 2581M 19.1.1-326-g8921c8ef f1455bd0219c 769f026a54eb 2024-09-16T00:47:48.226 INFO:teuthology.orchestra.run.smithi112.stdout:osd.5 smithi169 running (14m) 13m ago 25m 125M 2581M 19.1.1-326-g8921c8ef f1455bd0219c a6007c13cf78 2024-09-16T00:47:48.226 INFO:teuthology.orchestra.run.smithi112.stdout:prometheus.smithi112 smithi112 *:9095 running (19m) 13m ago 29m 64.1M - 2.43.0 a07b618ecd1d 7a1d42e60d2b 2024-09-16T00:47:48.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.226+0000 7fa64a7e2640 1 -- 172.21.15.112:0/1311061133 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa62c06e530 msgr2=0x7fa62c0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:48.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.226+0000 7fa64a7e2640 1 --2- 172.21.15.112:0/1311061133 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa62c06e530 0x7fa62c0709e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa634009b50 tx=0x7fa634009120 comp rx=0 tx=0).stop 2024-09-16T00:47:48.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.226+0000 7fa64a7e2640 1 -- 172.21.15.112:0/1311061133 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6441014e0 msgr2=0x7fa644112670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:48.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.226+0000 7fa64a7e2640 1 --2- 172.21.15.112:0/1311061133 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6441014e0 0x7fa644112670 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fa638000c00 tx=0x7fa638009da0 comp rx=0 tx=0).stop 2024-09-16T00:47:48.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.226+0000 7fa64a7e2640 1 -- 172.21.15.112:0/1311061133 shutdown_connections 2024-09-16T00:47:48.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.226+0000 7fa64a7e2640 1 --2- 172.21.15.112:0/1311061133 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa62c06e530 0x7fa62c0709e0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:48.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.226+0000 7fa64a7e2640 1 --2- 172.21.15.112:0/1311061133 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6441014e0 0x7fa644112670 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:48.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.226+0000 7fa64a7e2640 1 --2- 172.21.15.112:0/1311061133 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6440fb4a0 0x7fa644112130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:48.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.226+0000 7fa64a7e2640 1 -- 172.21.15.112:0/1311061133 >> 172.21.15.112:0/1311061133 conn(0x7fa6440f7060 msgr2=0x7fa6440f8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:48.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.226+0000 7fa64a7e2640 1 -- 172.21.15.112:0/1311061133 shutdown_connections 2024-09-16T00:47:48.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.226+0000 7fa64a7e2640 1 -- 172.21.15.112:0/1311061133 wait complete. 2024-09-16T00:47:48.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:47:47 smithi169 ceph-mon[77626]: pgmap v601: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:47:48.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.304+0000 7f4106e91640 1 -- 172.21.15.112:0/3735825161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4100102ed0 msgr2=0x7f41000fc590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:48.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.304+0000 7f4106e91640 1 --2- 172.21.15.112:0/3735825161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4100102ed0 0x7f41000fc590 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f40f00048d0 tx=0x7f40f0009da0 comp rx=0 tx=0).stop 2024-09-16T00:47:48.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.304+0000 7f4106e91640 1 -- 172.21.15.112:0/3735825161 shutdown_connections 2024-09-16T00:47:48.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.304+0000 7f4106e91640 1 --2- 172.21.15.112:0/3735825161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4100102ed0 0x7f41000fc590 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:48.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.304+0000 7f4106e91640 1 --2- 172.21.15.112:0/3735825161 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4100102530 0x7f4100102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:48.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.304+0000 7f4106e91640 1 -- 172.21.15.112:0/3735825161 >> 172.21.15.112:0/3735825161 conn(0x7f41000f80f0 msgr2=0x7f41000fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:48.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.304+0000 7f4106e91640 1 -- 172.21.15.112:0/3735825161 shutdown_connections 2024-09-16T00:47:48.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.304+0000 7f4106e91640 1 -- 172.21.15.112:0/3735825161 wait complete. 2024-09-16T00:47:48.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.304+0000 7f4106e91640 1 Processor -- start 2024-09-16T00:47:48.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.304+0000 7f4106e91640 1 -- start start 2024-09-16T00:47:48.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.305+0000 7f4106e91640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4100102530 0x7f41001131f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:48.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.305+0000 7f4106e91640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4100102ed0 0x7f4100113730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:48.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.305+0000 7f4106e91640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4100113d30 con 0x7f4100102ed0 2024-09-16T00:47:48.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.305+0000 7f4106e91640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f410010d170 con 0x7f4100102530 2024-09-16T00:47:48.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.305+0000 7f40fffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4100102ed0 0x7f4100113730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:48.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.305+0000 7f40fffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4100102ed0 0x7f4100113730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51240/0 (socket says 172.21.15.112:51240) 2024-09-16T00:47:48.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.305+0000 7f40fffff640 1 -- 172.21.15.112:0/2523080505 learned_addr learned my addr 172.21.15.112:0/2523080505 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:47:48.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.305+0000 7f4104c2d640 1 --2- 172.21.15.112:0/2523080505 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4100102530 0x7f41001131f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:48.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.305+0000 7f40fffff640 1 -- 172.21.15.112:0/2523080505 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4100102530 msgr2=0x7f41001131f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:48.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.305+0000 7f40fffff640 1 --2- 172.21.15.112:0/2523080505 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4100102530 0x7f41001131f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:48.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.305+0000 7f40fffff640 1 -- 172.21.15.112:0/2523080505 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40f0005040 con 0x7f4100102ed0 2024-09-16T00:47:48.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.306+0000 7f40fffff640 1 --2- 172.21.15.112:0/2523080505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4100102ed0 0x7f4100113730 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f40f00043d0 tx=0x7f40f0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:48.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.306+0000 7f40fd7fa640 1 -- 172.21.15.112:0/2523080505 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f40f0038070 con 0x7f4100102ed0 2024-09-16T00:47:48.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.306+0000 7f40fd7fa640 1 -- 172.21.15.112:0/2523080505 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f40f00096e0 con 0x7f4100102ed0 2024-09-16T00:47:48.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.306+0000 7f4106e91640 1 -- 172.21.15.112:0/2523080505 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f410010d3f0 con 0x7f4100102ed0 2024-09-16T00:47:48.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.306+0000 7f40fd7fa640 1 -- 172.21.15.112:0/2523080505 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f40f002c8a0 con 0x7f4100102ed0 2024-09-16T00:47:48.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.306+0000 7f4106e91640 1 -- 172.21.15.112:0/2523080505 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f410010d8e0 con 0x7f4100102ed0 2024-09-16T00:47:48.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.307+0000 7f4106e91640 1 -- 172.21.15.112:0/2523080505 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41000fdcb0 con 0x7f4100102ed0 2024-09-16T00:47:48.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.307+0000 7f40fd7fa640 1 -- 172.21.15.112:0/2523080505 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f40f0030020 con 0x7f4100102ed0 2024-09-16T00:47:48.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.308+0000 7f40fd7fa640 1 --2- 172.21.15.112:0/2523080505 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f40ec06e530 0x7f40ec0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:48.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.308+0000 7f40fd7fa640 1 -- 172.21.15.112:0/2523080505 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f40f00adfe0 con 0x7f4100102ed0 2024-09-16T00:47:48.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.308+0000 7f4104c2d640 1 --2- 172.21.15.112:0/2523080505 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f40ec06e530 0x7f40ec0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:48.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.309+0000 7f4104c2d640 1 --2- 172.21.15.112:0/2523080505 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f40ec06e530 0x7f40ec0709e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f40e8004ed0 tx=0x7f40e800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:48.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.312+0000 7f40fd7fa640 1 -- 172.21.15.112:0/2523080505 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f40f0077000 con 0x7f4100102ed0 2024-09-16T00:47:48.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.544+0000 7f4106e91640 1 -- 172.21.15.112:0/2523080505 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4100109560 con 0x7f4100102ed0 2024-09-16T00:47:48.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.545+0000 7f40fd7fa640 1 -- 172.21.15.112:0/2523080505 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f40f007a6c0 con 0x7f4100102ed0 2024-09-16T00:47:48.545 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:47:48.545 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": { 2024-09-16T00:47:48.545 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:47:48.546 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:47:48.546 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": { 2024-09-16T00:47:48.546 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:47:48.546 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:47:48.546 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": { 2024-09-16T00:47:48.546 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 6 2024-09-16T00:47:48.546 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:47:48.546 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": { 2024-09-16T00:47:48.546 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 4 2024-09-16T00:47:48.546 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:47:48.546 INFO:teuthology.orchestra.run.smithi112.stdout: "overall": { 2024-09-16T00:47:48.546 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 14 2024-09-16T00:47:48.546 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-16T00:47:48.546 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:47:48.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.547+0000 7f4106e91640 1 -- 172.21.15.112:0/2523080505 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f40ec06e530 msgr2=0x7f40ec0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:48.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.547+0000 7f4106e91640 1 --2- 172.21.15.112:0/2523080505 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f40ec06e530 0x7f40ec0709e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f40e8004ed0 tx=0x7f40e800b040 comp rx=0 tx=0).stop 2024-09-16T00:47:48.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.547+0000 7f4106e91640 1 -- 172.21.15.112:0/2523080505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4100102ed0 msgr2=0x7f4100113730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:48.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.547+0000 7f4106e91640 1 --2- 172.21.15.112:0/2523080505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4100102ed0 0x7f4100113730 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f40f00043d0 tx=0x7f40f0009da0 comp rx=0 tx=0).stop 2024-09-16T00:47:48.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.547+0000 7f4106e91640 1 -- 172.21.15.112:0/2523080505 shutdown_connections 2024-09-16T00:47:48.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.547+0000 7f4106e91640 1 --2- 172.21.15.112:0/2523080505 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f40ec06e530 0x7f40ec0709e0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:48.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.547+0000 7f4106e91640 1 --2- 172.21.15.112:0/2523080505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4100102ed0 0x7f4100113730 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:48.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.547+0000 7f4106e91640 1 --2- 172.21.15.112:0/2523080505 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4100102530 0x7f41001131f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:48.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.547+0000 7f4106e91640 1 -- 172.21.15.112:0/2523080505 >> 172.21.15.112:0/2523080505 conn(0x7f41000f80f0 msgr2=0x7f41000f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:48.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.547+0000 7f4106e91640 1 -- 172.21.15.112:0/2523080505 shutdown_connections 2024-09-16T00:47:48.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.547+0000 7f4106e91640 1 -- 172.21.15.112:0/2523080505 wait complete. 2024-09-16T00:47:48.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.626+0000 7fb74d57e640 1 -- 172.21.15.112:0/324460961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb74806fab0 msgr2=0x7fb74806fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:48.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.626+0000 7fb74d57e640 1 --2- 172.21.15.112:0/324460961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb74806fab0 0x7fb74806fe80 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fb738009c50 tx=0x7fb738004d10 comp rx=0 tx=0).stop 2024-09-16T00:47:48.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.626+0000 7fb74d57e640 1 -- 172.21.15.112:0/324460961 shutdown_connections 2024-09-16T00:47:48.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.626+0000 7fb74d57e640 1 --2- 172.21.15.112:0/324460961 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb7480703c0 0x7fb748108870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:48.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.626+0000 7fb74d57e640 1 --2- 172.21.15.112:0/324460961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb74806fab0 0x7fb74806fe80 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:48.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.626+0000 7fb74d57e640 1 -- 172.21.15.112:0/324460961 >> 172.21.15.112:0/324460961 conn(0x7fb7480f80f0 msgr2=0x7fb7480fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:48.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.626+0000 7fb74d57e640 1 -- 172.21.15.112:0/324460961 shutdown_connections 2024-09-16T00:47:48.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.626+0000 7fb74d57e640 1 -- 172.21.15.112:0/324460961 wait complete. 2024-09-16T00:47:48.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.626+0000 7fb74d57e640 1 Processor -- start 2024-09-16T00:47:48.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.626+0000 7fb74d57e640 1 -- start start 2024-09-16T00:47:48.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.627+0000 7fb74d57e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb74806fab0 0x7fb7481153e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:48.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.627+0000 7fb74d57e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb7480703c0 0x7fb748115920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:48.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.627+0000 7fb74d57e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb748115f20 con 0x7fb74806fab0 2024-09-16T00:47:48.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.627+0000 7fb74d57e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb748116090 con 0x7fb7480703c0 2024-09-16T00:47:48.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.627+0000 7fb746ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb74806fab0 0x7fb7481153e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:48.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.627+0000 7fb746ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb74806fab0 0x7fb7481153e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51260/0 (socket says 172.21.15.112:51260) 2024-09-16T00:47:48.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.627+0000 7fb746ffd640 1 -- 172.21.15.112:0/3097709050 learned_addr learned my addr 172.21.15.112:0/3097709050 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:47:48.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.627+0000 7fb7467fc640 1 --2- 172.21.15.112:0/3097709050 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb7480703c0 0x7fb748115920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:48.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.627+0000 7fb746ffd640 1 -- 172.21.15.112:0/3097709050 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb7480703c0 msgr2=0x7fb748115920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:48.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.627+0000 7fb746ffd640 1 --2- 172.21.15.112:0/3097709050 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb7480703c0 0x7fb748115920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:48.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.627+0000 7fb746ffd640 1 -- 172.21.15.112:0/3097709050 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7380098b0 con 0x7fb74806fab0 2024-09-16T00:47:48.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.627+0000 7fb746ffd640 1 --2- 172.21.15.112:0/3097709050 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb74806fab0 0x7fb7481153e0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fb738004a30 tx=0x7fb738004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:48.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.627+0000 7fb73f7fe640 1 -- 172.21.15.112:0/3097709050 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb738009710 con 0x7fb74806fab0 2024-09-16T00:47:48.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.627+0000 7fb73f7fe640 1 -- 172.21.15.112:0/3097709050 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fb738032a80 con 0x7fb74806fab0 2024-09-16T00:47:48.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.627+0000 7fb74d57e640 1 -- 172.21.15.112:0/3097709050 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb74810f4b0 con 0x7fb74806fab0 2024-09-16T00:47:48.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.627+0000 7fb73f7fe640 1 -- 172.21.15.112:0/3097709050 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb73803db30 con 0x7fb74806fab0 2024-09-16T00:47:48.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.627+0000 7fb74d57e640 1 -- 172.21.15.112:0/3097709050 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb74810f9d0 con 0x7fb74806fab0 2024-09-16T00:47:48.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.628+0000 7fb74d57e640 1 -- 172.21.15.112:0/3097709050 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7480711f0 con 0x7fb74806fab0 2024-09-16T00:47:48.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.629+0000 7fb73f7fe640 1 -- 172.21.15.112:0/3097709050 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fb73802f030 con 0x7fb74806fab0 2024-09-16T00:47:48.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.629+0000 7fb73f7fe640 1 --2- 172.21.15.112:0/3097709050 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fb73406e530 0x7fb7340709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:48.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.629+0000 7fb73f7fe640 1 -- 172.21.15.112:0/3097709050 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7fb7380af900 con 0x7fb74806fab0 2024-09-16T00:47:48.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.629+0000 7fb7467fc640 1 --2- 172.21.15.112:0/3097709050 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fb73406e530 0x7fb7340709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:48.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.630+0000 7fb7467fc640 1 --2- 172.21.15.112:0/3097709050 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fb73406e530 0x7fb7340709e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb748110ce0 tx=0x7fb730009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:48.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.634+0000 7fb73f7fe640 1 -- 172.21.15.112:0/3097709050 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fb738078870 con 0x7fb74806fab0 2024-09-16T00:47:48.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.837+0000 7fb74d57e640 1 -- 172.21.15.112:0/3097709050 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb74804b7b0 con 0x7fb74806fab0 2024-09-16T00:47:48.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.837+0000 7fb73f7fe640 1 -- 172.21.15.112:0/3097709050 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1914 (secure 0 0 0) 0x7fb73807bf30 con 0x7fb74806fab0 2024-09-16T00:47:48.839 INFO:teuthology.orchestra.run.smithi112.stdout:e34 2024-09-16T00:47:48.839 INFO:teuthology.orchestra.run.smithi112.stdout:btime 2024-09-16T00:34:26:619254+0000 2024-09-16T00:47:48.839 INFO:teuthology.orchestra.run.smithi112.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T00:47:48.839 INFO:teuthology.orchestra.run.smithi112.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:47:48.839 INFO:teuthology.orchestra.run.smithi112.stdout:legacy client fscid: 1 2024-09-16T00:47:48.839 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:47:48.839 INFO:teuthology.orchestra.run.smithi112.stdout:Filesystem 'cephfs' (1) 2024-09-16T00:47:48.839 INFO:teuthology.orchestra.run.smithi112.stdout:fs_name cephfs 2024-09-16T00:47:48.839 INFO:teuthology.orchestra.run.smithi112.stdout:epoch 25 2024-09-16T00:47:48.839 INFO:teuthology.orchestra.run.smithi112.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-16T00:47:48.839 INFO:teuthology.orchestra.run.smithi112.stdout:created 2024-09-16T00:23:22.328199+0000 2024-09-16T00:47:48.839 INFO:teuthology.orchestra.run.smithi112.stdout:modified 2024-09-16T00:33:29.012991+0000 2024-09-16T00:47:48.839 INFO:teuthology.orchestra.run.smithi112.stdout:tableserver 0 2024-09-16T00:47:48.839 INFO:teuthology.orchestra.run.smithi112.stdout:root 0 2024-09-16T00:47:48.840 INFO:teuthology.orchestra.run.smithi112.stdout:session_timeout 60 2024-09-16T00:47:48.840 INFO:teuthology.orchestra.run.smithi112.stdout:session_autoclose 300 2024-09-16T00:47:48.840 INFO:teuthology.orchestra.run.smithi112.stdout:max_file_size 1099511627776 2024-09-16T00:47:48.840 INFO:teuthology.orchestra.run.smithi112.stdout:max_xattr_size 65536 2024-09-16T00:47:48.840 INFO:teuthology.orchestra.run.smithi112.stdout:required_client_features {} 2024-09-16T00:47:48.840 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure 0 2024-09-16T00:47:48.840 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure_osd_epoch 82 2024-09-16T00:47:48.840 INFO:teuthology.orchestra.run.smithi112.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:47:48.840 INFO:teuthology.orchestra.run.smithi112.stdout:max_mds 2 2024-09-16T00:47:48.840 INFO:teuthology.orchestra.run.smithi112.stdout:in 0,1 2024-09-16T00:47:48.840 INFO:teuthology.orchestra.run.smithi112.stdout:up {} 2024-09-16T00:47:48.840 INFO:teuthology.orchestra.run.smithi112.stdout:failed 0,1 2024-09-16T00:47:48.840 INFO:teuthology.orchestra.run.smithi112.stdout:damaged 2024-09-16T00:47:48.841 INFO:teuthology.orchestra.run.smithi112.stdout:stopped 2024-09-16T00:47:48.841 INFO:teuthology.orchestra.run.smithi112.stdout:data_pools [3] 2024-09-16T00:47:48.841 INFO:teuthology.orchestra.run.smithi112.stdout:metadata_pool 2 2024-09-16T00:47:48.841 INFO:teuthology.orchestra.run.smithi112.stdout:inline_data enabled 2024-09-16T00:47:48.841 INFO:teuthology.orchestra.run.smithi112.stdout:balancer 2024-09-16T00:47:48.841 INFO:teuthology.orchestra.run.smithi112.stdout:bal_rank_mask -1 2024-09-16T00:47:48.841 INFO:teuthology.orchestra.run.smithi112.stdout:standby_count_wanted 1 2024-09-16T00:47:48.841 INFO:teuthology.orchestra.run.smithi112.stdout:qdb_cluster leader: 0 members: 2024-09-16T00:47:48.841 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:47:48.841 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:47:48.841 INFO:teuthology.orchestra.run.smithi112.stdout:Standby daemons: 2024-09-16T00:47:48.841 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:47:48.841 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.ithwzj{-1:34392} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.112:6826/476229221,v1:172.21.15.112:6827/476229221] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:47:48.841 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.hxfwkh{-1:34404} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.112:6828/2356899876,v1:172.21.15.112:6829/2356899876] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:47:48.842 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.gawwgw{-1:44343} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.169:6824/1635329957,v1:172.21.15.169:6825/1635329957] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:47:48.842 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.lgeqrj{-1:44347} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.169:6826/184810031,v1:172.21.15.169:6827/184810031] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:47:48.842 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 34 2024-09-16T00:47:48.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.839+0000 7fb74d57e640 1 -- 172.21.15.112:0/3097709050 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fb73406e530 msgr2=0x7fb7340709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:48.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.839+0000 7fb74d57e640 1 --2- 172.21.15.112:0/3097709050 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fb73406e530 0x7fb7340709e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb748110ce0 tx=0x7fb730009040 comp rx=0 tx=0).stop 2024-09-16T00:47:48.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.839+0000 7fb74d57e640 1 -- 172.21.15.112:0/3097709050 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb74806fab0 msgr2=0x7fb7481153e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:48.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.839+0000 7fb74d57e640 1 --2- 172.21.15.112:0/3097709050 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb74806fab0 0x7fb7481153e0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fb738004a30 tx=0x7fb738004d10 comp rx=0 tx=0).stop 2024-09-16T00:47:48.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.839+0000 7fb74d57e640 1 -- 172.21.15.112:0/3097709050 shutdown_connections 2024-09-16T00:47:48.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.839+0000 7fb74d57e640 1 --2- 172.21.15.112:0/3097709050 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fb73406e530 0x7fb7340709e0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:48.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.839+0000 7fb74d57e640 1 --2- 172.21.15.112:0/3097709050 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb7480703c0 0x7fb748115920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:48.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.839+0000 7fb74d57e640 1 --2- 172.21.15.112:0/3097709050 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb74806fab0 0x7fb7481153e0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:48.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.839+0000 7fb74d57e640 1 -- 172.21.15.112:0/3097709050 >> 172.21.15.112:0/3097709050 conn(0x7fb7480f80f0 msgr2=0x7fb7480f9700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:48.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.839+0000 7fb74d57e640 1 -- 172.21.15.112:0/3097709050 shutdown_connections 2024-09-16T00:47:48.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.839+0000 7fb74d57e640 1 -- 172.21.15.112:0/3097709050 wait complete. 2024-09-16T00:47:48.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.919+0000 7fdf07c92640 1 -- 172.21.15.112:0/2255257111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf000fdf40 msgr2=0x7fdf00106610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:48.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.919+0000 7fdf07c92640 1 --2- 172.21.15.112:0/2255257111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf000fdf40 0x7fdf00106610 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fdef40048d0 tx=0x7fdef4009da0 comp rx=0 tx=0).stop 2024-09-16T00:47:48.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.919+0000 7fdf07c92640 1 -- 172.21.15.112:0/2255257111 shutdown_connections 2024-09-16T00:47:48.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.919+0000 7fdf07c92640 1 --2- 172.21.15.112:0/2255257111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf000fdf40 0x7fdf00106610 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:48.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.919+0000 7fdf07c92640 1 --2- 172.21.15.112:0/2255257111 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf000fd5a0 0x7fdf000fd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:48.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.919+0000 7fdf07c92640 1 -- 172.21.15.112:0/2255257111 >> 172.21.15.112:0/2255257111 conn(0x7fdf000f7020 msgr2=0x7fdf000f9430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:48.921 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:47:48 smithi112 ceph-mon[97750]: from='client.35136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:47:48.921 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:47:48 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/2523080505' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:47:48.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.919+0000 7fdf07c92640 1 -- 172.21.15.112:0/2255257111 shutdown_connections 2024-09-16T00:47:48.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.919+0000 7fdf07c92640 1 -- 172.21.15.112:0/2255257111 wait complete. 2024-09-16T00:47:48.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.920+0000 7fdf07c92640 1 Processor -- start 2024-09-16T00:47:48.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.920+0000 7fdf07c92640 1 -- start start 2024-09-16T00:47:48.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.920+0000 7fdf07c92640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf000fd5a0 0x7fdf00113260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:48.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.920+0000 7fdf07c92640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf000fdf40 0x7fdf001137a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:48.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.920+0000 7fdf07c92640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf00113da0 con 0x7fdf000fd5a0 2024-09-16T00:47:48.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.920+0000 7fdf07c92640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0010d1e0 con 0x7fdf000fdf40 2024-09-16T00:47:48.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.920+0000 7fdf05a2e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf000fd5a0 0x7fdf00113260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:48.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.920+0000 7fdf05a2e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf000fd5a0 0x7fdf00113260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51282/0 (socket says 172.21.15.112:51282) 2024-09-16T00:47:48.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.920+0000 7fdf05a2e640 1 -- 172.21.15.112:0/3700068902 learned_addr learned my addr 172.21.15.112:0/3700068902 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:47:48.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.920+0000 7fdf0522d640 1 --2- 172.21.15.112:0/3700068902 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf000fdf40 0x7fdf001137a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:48.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.920+0000 7fdf05a2e640 1 -- 172.21.15.112:0/3700068902 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf000fdf40 msgr2=0x7fdf001137a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:48.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.920+0000 7fdf05a2e640 1 --2- 172.21.15.112:0/3700068902 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf000fdf40 0x7fdf001137a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:48.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.920+0000 7fdf05a2e640 1 -- 172.21.15.112:0/3700068902 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef4005040 con 0x7fdf000fd5a0 2024-09-16T00:47:48.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.920+0000 7fdf0522d640 1 --2- 172.21.15.112:0/3700068902 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf000fdf40 0x7fdf001137a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:47:48.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.920+0000 7fdf05a2e640 1 --2- 172.21.15.112:0/3700068902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf000fd5a0 0x7fdf00113260 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fdefc002a50 tx=0x7fdefc004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:48.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.921+0000 7fdef27fc640 1 -- 172.21.15.112:0/3700068902 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdefc007c30 con 0x7fdf000fd5a0 2024-09-16T00:47:48.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.921+0000 7fdef27fc640 1 -- 172.21.15.112:0/3700068902 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fdefc00f040 con 0x7fdf000fd5a0 2024-09-16T00:47:48.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.921+0000 7fdf07c92640 1 -- 172.21.15.112:0/3700068902 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf0010d4c0 con 0x7fdf000fd5a0 2024-09-16T00:47:48.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.921+0000 7fdef27fc640 1 -- 172.21.15.112:0/3700068902 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdefc00da90 con 0x7fdf000fd5a0 2024-09-16T00:47:48.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.921+0000 7fdf07c92640 1 -- 172.21.15.112:0/3700068902 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf0010d9e0 con 0x7fdf000fd5a0 2024-09-16T00:47:48.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.922+0000 7fdf07c92640 1 -- 172.21.15.112:0/3700068902 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf000662b0 con 0x7fdf000fd5a0 2024-09-16T00:47:48.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.922+0000 7fdef27fc640 1 -- 172.21.15.112:0/3700068902 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fdefc00dd00 con 0x7fdf000fd5a0 2024-09-16T00:47:48.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.923+0000 7fdef27fc640 1 --2- 172.21.15.112:0/3700068902 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fdeec06e530 0x7fdeec0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:48.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.923+0000 7fdef27fc640 1 -- 172.21.15.112:0/3700068902 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7fdefc08dc10 con 0x7fdf000fd5a0 2024-09-16T00:47:48.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.923+0000 7fdf0522d640 1 --2- 172.21.15.112:0/3700068902 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fdeec06e530 0x7fdeec0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:48.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.923+0000 7fdf0522d640 1 --2- 172.21.15.112:0/3700068902 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fdeec06e530 0x7fdeec0709e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fdef400b020 tx=0x7fdef4009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:48.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:48.927+0000 7fdef27fc640 1 -- 172.21.15.112:0/3700068902 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fdefc056b80 con 0x7fdf000fd5a0 2024-09-16T00:47:49.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:49.108+0000 7fdf07c92640 1 -- 172.21.15.112:0/3700068902 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdf0004b720 con 0x7fdeec06e530 2024-09-16T00:47:49.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:49.110+0000 7fdef27fc640 1 -- 172.21.15.112:0/3700068902 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fdf0004b720 con 0x7fdeec06e530 2024-09-16T00:47:49.111 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:47:49.111 INFO:teuthology.orchestra.run.smithi112.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80", 2024-09-16T00:47:49.111 INFO:teuthology.orchestra.run.smithi112.stdout: "in_progress": true, 2024-09-16T00:47:49.111 INFO:teuthology.orchestra.run.smithi112.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T00:47:49.111 INFO:teuthology.orchestra.run.smithi112.stdout: "services_complete": [ 2024-09-16T00:47:49.111 INFO:teuthology.orchestra.run.smithi112.stdout: "mds", 2024-09-16T00:47:49.111 INFO:teuthology.orchestra.run.smithi112.stdout: "osd", 2024-09-16T00:47:49.111 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr", 2024-09-16T00:47:49.111 INFO:teuthology.orchestra.run.smithi112.stdout: "mon", 2024-09-16T00:47:49.111 INFO:teuthology.orchestra.run.smithi112.stdout: "crash" 2024-09-16T00:47:49.111 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-16T00:47:49.112 INFO:teuthology.orchestra.run.smithi112.stdout: "progress": "16/21 daemons upgraded", 2024-09-16T00:47:49.112 INFO:teuthology.orchestra.run.smithi112.stdout: "message": "Currently upgrading mds daemons", 2024-09-16T00:47:49.112 INFO:teuthology.orchestra.run.smithi112.stdout: "is_paused": false 2024-09-16T00:47:49.112 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:47:49.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:49.112+0000 7fdf07c92640 1 -- 172.21.15.112:0/3700068902 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fdeec06e530 msgr2=0x7fdeec0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:49.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:49.112+0000 7fdf07c92640 1 --2- 172.21.15.112:0/3700068902 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fdeec06e530 0x7fdeec0709e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fdef400b020 tx=0x7fdef4009da0 comp rx=0 tx=0).stop 2024-09-16T00:47:49.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:49.112+0000 7fdf07c92640 1 -- 172.21.15.112:0/3700068902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf000fd5a0 msgr2=0x7fdf00113260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:49.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:49.112+0000 7fdf07c92640 1 --2- 172.21.15.112:0/3700068902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf000fd5a0 0x7fdf00113260 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fdefc002a50 tx=0x7fdefc004600 comp rx=0 tx=0).stop 2024-09-16T00:47:49.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:49.112+0000 7fdf07c92640 1 -- 172.21.15.112:0/3700068902 shutdown_connections 2024-09-16T00:47:49.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:49.112+0000 7fdf07c92640 1 --2- 172.21.15.112:0/3700068902 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fdeec06e530 0x7fdeec0709e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:49.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:49.112+0000 7fdf07c92640 1 --2- 172.21.15.112:0/3700068902 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf000fdf40 0x7fdf001137a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:49.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:49.112+0000 7fdf07c92640 1 --2- 172.21.15.112:0/3700068902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf000fd5a0 0x7fdf00113260 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:49.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:49.112+0000 7fdf07c92640 1 -- 172.21.15.112:0/3700068902 >> 172.21.15.112:0/3700068902 conn(0x7fdf000f7020 msgr2=0x7fdf000f8570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:49.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:49.112+0000 7fdf07c92640 1 -- 172.21.15.112:0/3700068902 shutdown_connections 2024-09-16T00:47:49.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:49.112+0000 7fdf07c92640 1 -- 172.21.15.112:0/3700068902 wait complete. 2024-09-16T00:47:49.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:49.190+0000 7f5120c68640 1 -- 172.21.15.112:0/3171184210 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f511c102ef0 msgr2=0x7f511c0fc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:49.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:49.190+0000 7f5120c68640 1 --2- 172.21.15.112:0/3171184210 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f511c102ef0 0x7f511c0fc5b0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f51100048d0 tx=0x7f5110009da0 comp rx=0 tx=0).stop 2024-09-16T00:47:49.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:49.190+0000 7f5120c68640 1 -- 172.21.15.112:0/3171184210 shutdown_connections 2024-09-16T00:47:49.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:49.190+0000 7f5120c68640 1 --2- 172.21.15.112:0/3171184210 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f511c102ef0 0x7f511c0fc5b0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:49.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:49.190+0000 7f5120c68640 1 --2- 172.21.15.112:0/3171184210 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f511c102550 0x7f511c102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:49.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:49.190+0000 7f5120c68640 1 -- 172.21.15.112:0/3171184210 >> 172.21.15.112:0/3171184210 conn(0x7f511c0f80f0 msgr2=0x7f511c0fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:49.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:49.190+0000 7f5120c68640 1 -- 172.21.15.112:0/3171184210 shutdown_connections 2024-09-16T00:47:49.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:49.190+0000 7f5120c68640 1 -- 172.21.15.112:0/3171184210 wait complete. 2024-09-16T00:47:49.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:49.191+0000 7f5120c68640 1 Processor -- start 2024-09-16T00:47:49.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:49.191+0000 7f5120c68640 1 -- start start 2024-09-16T00:47:49.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:49.191+0000 7f5120c68640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f511c102550 0x7f511c113520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:49.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:49.191+0000 7f5120c68640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f511c102ef0 0x7f511c113a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:49.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:49.191+0000 7f5120c68640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f511c1140b0 con 0x7f511c102ef0 2024-09-16T00:47:49.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:49.191+0000 7f5120c68640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f511c10d4c0 con 0x7f511c102550 2024-09-16T00:47:49.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:49.191+0000 7f5119d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f511c102ef0 0x7f511c113a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:49.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:49.191+0000 7f5119d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f511c102ef0 0x7f511c113a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51288/0 (socket says 172.21.15.112:51288) 2024-09-16T00:47:49.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:49.191+0000 7f5119d9b640 1 -- 172.21.15.112:0/961245166 learned_addr learned my addr 172.21.15.112:0/961245166 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:47:49.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:49.191+0000 7f511a59c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f511c102550 0x7f511c113520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:49.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:49.191+0000 7f5119d9b640 1 -- 172.21.15.112:0/961245166 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f511c102550 msgr2=0x7f511c113520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:49.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:49.191+0000 7f5119d9b640 1 --2- 172.21.15.112:0/961245166 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f511c102550 0x7f511c113520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:49.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:49.191+0000 7f5119d9b640 1 -- 172.21.15.112:0/961245166 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5110005040 con 0x7f511c102ef0 2024-09-16T00:47:49.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:49.192+0000 7f5119d9b640 1 --2- 172.21.15.112:0/961245166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f511c102ef0 0x7f511c113a60 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f51100043d0 tx=0x7f5110009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:49.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:49.192+0000 7f510affd640 1 -- 172.21.15.112:0/961245166 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5110038070 con 0x7f511c102ef0 2024-09-16T00:47:49.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:49.192+0000 7f510affd640 1 -- 172.21.15.112:0/961245166 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f51100096e0 con 0x7f511c102ef0 2024-09-16T00:47:49.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:49.192+0000 7f5120c68640 1 -- 172.21.15.112:0/961245166 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f511c10d740 con 0x7f511c102ef0 2024-09-16T00:47:49.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:49.192+0000 7f510affd640 1 -- 172.21.15.112:0/961245166 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f511002c8a0 con 0x7f511c102ef0 2024-09-16T00:47:49.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:49.192+0000 7f5120c68640 1 -- 172.21.15.112:0/961245166 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f511c10dc80 con 0x7f511c102ef0 2024-09-16T00:47:49.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:49.193+0000 7f510affd640 1 -- 172.21.15.112:0/961245166 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f5110030020 con 0x7f511c102ef0 2024-09-16T00:47:49.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:49.193+0000 7f5120c68640 1 -- 172.21.15.112:0/961245166 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f511c0fdcd0 con 0x7f511c102ef0 2024-09-16T00:47:49.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:49.194+0000 7f510affd640 1 --2- 172.21.15.112:0/961245166 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f510406e530 0x7f51040709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:47:49.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:49.194+0000 7f510affd640 1 -- 172.21.15.112:0/961245166 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f51100addb0 con 0x7f511c102ef0 2024-09-16T00:47:49.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:49.194+0000 7f511a59c640 1 --2- 172.21.15.112:0/961245166 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f510406e530 0x7f51040709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:47:49.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:49.194+0000 7f511a59c640 1 --2- 172.21.15.112:0/961245166 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f510406e530 0x7f51040709e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f510c004ed0 tx=0x7f510c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:47:49.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:49.198+0000 7f510affd640 1 -- 172.21.15.112:0/961245166 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f5110076e50 con 0x7f511c102ef0 2024-09-16T00:47:49.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:47:48 smithi169 ceph-mon[77626]: from='client.35136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:47:49.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:47:48 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/2523080505' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:47:49.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:49.429+0000 7f5120c68640 1 -- 172.21.15.112:0/961245166 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f511c0fbd50 con 0x7f511c102ef0 2024-09-16T00:47:49.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:49.429+0000 7f510affd640 1 -- 172.21.15.112:0/961245166 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f511007a510 con 0x7f511c102ef0 2024-09-16T00:47:49.431 INFO:teuthology.orchestra.run.smithi112.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-16T00:47:49.431 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-16T00:47:49.431 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs is degraded 2024-09-16T00:47:49.431 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T00:47:49.431 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T00:47:49.431 INFO:teuthology.orchestra.run.smithi112.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-16T00:47:49.431 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-16T00:47:49.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:49.432+0000 7f5120c68640 1 -- 172.21.15.112:0/961245166 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f510406e530 msgr2=0x7f51040709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:49.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:49.432+0000 7f5120c68640 1 --2- 172.21.15.112:0/961245166 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f510406e530 0x7f51040709e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f510c004ed0 tx=0x7f510c00b040 comp rx=0 tx=0).stop 2024-09-16T00:47:49.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:49.432+0000 7f5120c68640 1 -- 172.21.15.112:0/961245166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f511c102ef0 msgr2=0x7f511c113a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:47:49.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:49.432+0000 7f5120c68640 1 --2- 172.21.15.112:0/961245166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f511c102ef0 0x7f511c113a60 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f51100043d0 tx=0x7f5110009da0 comp rx=0 tx=0).stop 2024-09-16T00:47:49.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:49.432+0000 7f5120c68640 1 -- 172.21.15.112:0/961245166 shutdown_connections 2024-09-16T00:47:49.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:49.432+0000 7f5120c68640 1 --2- 172.21.15.112:0/961245166 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f510406e530 0x7f51040709e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:49.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:49.432+0000 7f5120c68640 1 --2- 172.21.15.112:0/961245166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f511c102ef0 0x7f511c113a60 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:49.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:49.432+0000 7f5120c68640 1 --2- 172.21.15.112:0/961245166 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f511c102550 0x7f511c113520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:47:49.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:49.432+0000 7f5120c68640 1 -- 172.21.15.112:0/961245166 >> 172.21.15.112:0/961245166 conn(0x7f511c0f80f0 msgr2=0x7f511c0f9150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:47:49.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:49.432+0000 7f5120c68640 1 -- 172.21.15.112:0/961245166 shutdown_connections 2024-09-16T00:47:49.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:47:49.432+0000 7f5120c68640 1 -- 172.21.15.112:0/961245166 wait complete. 2024-09-16T00:47:50.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:47:49 smithi169 ceph-mon[77626]: from='client.35140 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:47:50.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:47:49 smithi169 ceph-mon[77626]: from='client.35144 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:47:50.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:47:49 smithi169 ceph-mon[77626]: pgmap v602: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:47:50.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:47:49 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/3097709050' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:47:50.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:47:49 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/961245166' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:47:50.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:47:49 smithi112 ceph-mon[97750]: from='client.35140 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:47:50.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:47:49 smithi112 ceph-mon[97750]: from='client.35144 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:47:50.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:47:49 smithi112 ceph-mon[97750]: pgmap v602: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:47:50.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:47:49 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/3097709050' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:47:50.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:47:49 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/961245166' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:47:51.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:47:50 smithi169 ceph-mon[77626]: from='client.35156 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:47:51.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:47:50 smithi112 ceph-mon[97750]: from='client.35156 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:47:52.240 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:47:51 smithi112 ceph-mon[97750]: pgmap v603: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:47:52.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:47:51 smithi169 ceph-mon[77626]: pgmap v603: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:47:54.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:47:53 smithi112 ceph-mon[97750]: pgmap v604: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:47:54.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:47:53 smithi169 ceph-mon[77626]: pgmap v604: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:47:56.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:47:55 smithi169 ceph-mon[77626]: pgmap v605: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:47:56.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:47:55 smithi112 ceph-mon[97750]: pgmap v605: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:47:58.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:47:57 smithi169 ceph-mon[77626]: pgmap v606: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:47:58.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:47:57 smithi112 ceph-mon[97750]: pgmap v606: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:48:00.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:47:59 smithi169 ceph-mon[77626]: pgmap v607: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:48:00.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:47:59 smithi112 ceph-mon[97750]: pgmap v607: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:48:02.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:48:01 smithi169 ceph-mon[77626]: pgmap v608: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:48:02.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:48:01 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:48:02.314 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:48:01 smithi112 ceph-mon[97750]: pgmap v608: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:48:02.315 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:48:01 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:48:04.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:48:03 smithi112 ceph-mon[97750]: pgmap v609: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:48:04.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:48:03 smithi169 ceph-mon[77626]: pgmap v609: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:48:06.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:48:05 smithi169 ceph-mon[77626]: pgmap v610: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:48:06.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:48:05 smithi112 ceph-mon[97750]: pgmap v610: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:48:08.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:48:07 smithi169 ceph-mon[77626]: pgmap v611: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:48:08.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:48:07 smithi112 ceph-mon[97750]: pgmap v611: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:48:10.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:48:09 smithi169 ceph-mon[77626]: pgmap v612: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:48:10.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:48:09 smithi112 ceph-mon[97750]: pgmap v612: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:48:12.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:48:11 smithi169 ceph-mon[77626]: pgmap v613: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:48:12.314 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:48:11 smithi112 ceph-mon[97750]: pgmap v613: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:48:14.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:48:13 smithi112 ceph-mon[97750]: pgmap v614: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:48:14.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:48:13 smithi169 ceph-mon[77626]: pgmap v614: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:48:16.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:48:15 smithi169 ceph-mon[77626]: pgmap v615: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:48:16.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:48:15 smithi112 ceph-mon[97750]: pgmap v615: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:48:17.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:48:16 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:48:17.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:48:16 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:48:18.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:48:17 smithi169 ceph-mon[77626]: pgmap v616: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:48:18.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:48:17 smithi112 ceph-mon[97750]: pgmap v616: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:48:19.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.522+0000 7ff5830b0640 1 -- 172.21.15.112:0/3334500854 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff57c102ef0 msgr2=0x7ff57c0fc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:19.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.522+0000 7ff5830b0640 1 --2- 172.21.15.112:0/3334500854 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff57c102ef0 0x7ff57c0fc5b0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7ff5640048d0 tx=0x7ff564009da0 comp rx=0 tx=0).stop 2024-09-16T00:48:19.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.522+0000 7ff5830b0640 1 -- 172.21.15.112:0/3334500854 shutdown_connections 2024-09-16T00:48:19.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.522+0000 7ff5830b0640 1 --2- 172.21.15.112:0/3334500854 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff57c102ef0 0x7ff57c0fc5b0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:19.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.522+0000 7ff5830b0640 1 --2- 172.21.15.112:0/3334500854 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff57c102550 0x7ff57c102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:19.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.522+0000 7ff5830b0640 1 -- 172.21.15.112:0/3334500854 >> 172.21.15.112:0/3334500854 conn(0x7ff57c0f80f0 msgr2=0x7ff57c0fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:48:19.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.522+0000 7ff5830b0640 1 -- 172.21.15.112:0/3334500854 shutdown_connections 2024-09-16T00:48:19.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.522+0000 7ff5830b0640 1 -- 172.21.15.112:0/3334500854 wait complete. 2024-09-16T00:48:19.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.523+0000 7ff5830b0640 1 Processor -- start 2024-09-16T00:48:19.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.523+0000 7ff5830b0640 1 -- start start 2024-09-16T00:48:19.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.523+0000 7ff5830b0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff57c102550 0x7ff57c115760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:19.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.523+0000 7ff5830b0640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff57c102ef0 0x7ff57c115ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:19.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.523+0000 7ff5830b0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff57c1162f0 con 0x7ff57c102550 2024-09-16T00:48:19.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.523+0000 7ff5830b0640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff57c10f700 con 0x7ff57c102ef0 2024-09-16T00:48:19.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.523+0000 7ff580e4c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff57c102550 0x7ff57c115760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:19.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.523+0000 7ff580e4c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff57c102550 0x7ff57c115760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46618/0 (socket says 172.21.15.112:46618) 2024-09-16T00:48:19.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.523+0000 7ff580e4c640 1 -- 172.21.15.112:0/2413475625 learned_addr learned my addr 172.21.15.112:0/2413475625 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:48:19.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.523+0000 7ff57bfff640 1 --2- 172.21.15.112:0/2413475625 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff57c102ef0 0x7ff57c115ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:19.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.524+0000 7ff580e4c640 1 -- 172.21.15.112:0/2413475625 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff57c102ef0 msgr2=0x7ff57c115ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:19.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.524+0000 7ff580e4c640 1 --2- 172.21.15.112:0/2413475625 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff57c102ef0 0x7ff57c115ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:19.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.524+0000 7ff580e4c640 1 -- 172.21.15.112:0/2413475625 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff564005040 con 0x7ff57c102550 2024-09-16T00:48:19.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.524+0000 7ff580e4c640 1 --2- 172.21.15.112:0/2413475625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff57c102550 0x7ff57c115760 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7ff56c002a50 tx=0x7ff56c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:48:19.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.524+0000 7ff5797fa640 1 -- 172.21.15.112:0/2413475625 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff56c007c30 con 0x7ff57c102550 2024-09-16T00:48:19.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.524+0000 7ff5797fa640 1 -- 172.21.15.112:0/2413475625 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7ff56c00f040 con 0x7ff57c102550 2024-09-16T00:48:19.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.524+0000 7ff5830b0640 1 -- 172.21.15.112:0/2413475625 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff57c10f9e0 con 0x7ff57c102550 2024-09-16T00:48:19.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.524+0000 7ff5797fa640 1 -- 172.21.15.112:0/2413475625 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff56c00da10 con 0x7ff57c102550 2024-09-16T00:48:19.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.524+0000 7ff5830b0640 1 -- 172.21.15.112:0/2413475625 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff57c10ff50 con 0x7ff57c102550 2024-09-16T00:48:19.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.525+0000 7ff5830b0640 1 -- 172.21.15.112:0/2413475625 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff57c0fdcd0 con 0x7ff57c102550 2024-09-16T00:48:19.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.526+0000 7ff5797fa640 1 -- 172.21.15.112:0/2413475625 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7ff56c00dc80 con 0x7ff57c102550 2024-09-16T00:48:19.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.526+0000 7ff5797fa640 1 --2- 172.21.15.112:0/2413475625 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff56806e530 0x7ff5680709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:19.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.527+0000 7ff5797fa640 1 -- 172.21.15.112:0/2413475625 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7ff56c08daf0 con 0x7ff57c102550 2024-09-16T00:48:19.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.527+0000 7ff57bfff640 1 --2- 172.21.15.112:0/2413475625 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff56806e530 0x7ff5680709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:19.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.527+0000 7ff57bfff640 1 --2- 172.21.15.112:0/2413475625 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff56806e530 0x7ff5680709e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff5640048a0 tx=0x7ff564035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:48:19.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.531+0000 7ff5797fa640 1 -- 172.21.15.112:0/2413475625 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7ff56c056b10 con 0x7ff57c102550 2024-09-16T00:48:19.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.713+0000 7ff5830b0640 1 -- 172.21.15.112:0/2413475625 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff57c109580 con 0x7ff56806e530 2024-09-16T00:48:19.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.714+0000 7ff5797fa640 1 -- 172.21.15.112:0/2413475625 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7ff57c109580 con 0x7ff56806e530 2024-09-16T00:48:19.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.717+0000 7ff5830b0640 1 -- 172.21.15.112:0/2413475625 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff56806e530 msgr2=0x7ff5680709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:19.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.717+0000 7ff5830b0640 1 --2- 172.21.15.112:0/2413475625 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff56806e530 0x7ff5680709e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff5640048a0 tx=0x7ff564035040 comp rx=0 tx=0).stop 2024-09-16T00:48:19.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.717+0000 7ff5830b0640 1 -- 172.21.15.112:0/2413475625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff57c102550 msgr2=0x7ff57c115760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:19.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.717+0000 7ff5830b0640 1 --2- 172.21.15.112:0/2413475625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff57c102550 0x7ff57c115760 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7ff56c002a50 tx=0x7ff56c004600 comp rx=0 tx=0).stop 2024-09-16T00:48:19.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.717+0000 7ff5830b0640 1 -- 172.21.15.112:0/2413475625 shutdown_connections 2024-09-16T00:48:19.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.717+0000 7ff5830b0640 1 --2- 172.21.15.112:0/2413475625 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff56806e530 0x7ff5680709e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:19.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.717+0000 7ff5830b0640 1 --2- 172.21.15.112:0/2413475625 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff57c102ef0 0x7ff57c115ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:19.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.717+0000 7ff5830b0640 1 --2- 172.21.15.112:0/2413475625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff57c102550 0x7ff57c115760 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:19.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.717+0000 7ff5830b0640 1 -- 172.21.15.112:0/2413475625 >> 172.21.15.112:0/2413475625 conn(0x7ff57c0f80f0 msgr2=0x7ff57c0f9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:48:19.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.717+0000 7ff5830b0640 1 -- 172.21.15.112:0/2413475625 shutdown_connections 2024-09-16T00:48:19.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.717+0000 7ff5830b0640 1 -- 172.21.15.112:0/2413475625 wait complete. 2024-09-16T00:48:19.725 INFO:teuthology.orchestra.run.smithi112.stdout:true 2024-09-16T00:48:19.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.796+0000 7efe557a5640 1 -- 172.21.15.112:0/4261683877 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe500fb480 msgr2=0x7efe500fb8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:19.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.796+0000 7efe557a5640 1 --2- 172.21.15.112:0/4261683877 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe500fb480 0x7efe500fb8f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7efe380048d0 tx=0x7efe38009da0 comp rx=0 tx=0).stop 2024-09-16T00:48:19.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.796+0000 7efe557a5640 1 -- 172.21.15.112:0/4261683877 shutdown_connections 2024-09-16T00:48:19.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.796+0000 7efe557a5640 1 --2- 172.21.15.112:0/4261683877 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe500fb480 0x7efe500fb8f0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:19.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.796+0000 7efe557a5640 1 --2- 172.21.15.112:0/4261683877 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe501014c0 0x7efe50101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:19.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.796+0000 7efe557a5640 1 -- 172.21.15.112:0/4261683877 >> 172.21.15.112:0/4261683877 conn(0x7efe500f7000 msgr2=0x7efe500f9410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:48:19.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.796+0000 7efe557a5640 1 -- 172.21.15.112:0/4261683877 shutdown_connections 2024-09-16T00:48:19.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.796+0000 7efe557a5640 1 -- 172.21.15.112:0/4261683877 wait complete. 2024-09-16T00:48:19.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.797+0000 7efe557a5640 1 Processor -- start 2024-09-16T00:48:19.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.797+0000 7efe557a5640 1 -- start start 2024-09-16T00:48:19.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.797+0000 7efe557a5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe500fb480 0x7efe501131b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:19.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.797+0000 7efe557a5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe501014c0 0x7efe501136f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:19.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.797+0000 7efe557a5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe50113cf0 con 0x7efe500fb480 2024-09-16T00:48:19.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.797+0000 7efe557a5640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe5010d130 con 0x7efe501014c0 2024-09-16T00:48:19.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.797+0000 7efe4effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe500fb480 0x7efe501131b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:19.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.797+0000 7efe4effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe500fb480 0x7efe501131b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46626/0 (socket says 172.21.15.112:46626) 2024-09-16T00:48:19.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.797+0000 7efe4effd640 1 -- 172.21.15.112:0/1995271687 learned_addr learned my addr 172.21.15.112:0/1995271687 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:48:19.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.797+0000 7efe47fff640 1 --2- 172.21.15.112:0/1995271687 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe501014c0 0x7efe501136f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:19.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.798+0000 7efe4effd640 1 -- 172.21.15.112:0/1995271687 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe501014c0 msgr2=0x7efe501136f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:19.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.798+0000 7efe4effd640 1 --2- 172.21.15.112:0/1995271687 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe501014c0 0x7efe501136f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:19.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.798+0000 7efe4effd640 1 -- 172.21.15.112:0/1995271687 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe38005040 con 0x7efe500fb480 2024-09-16T00:48:19.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.798+0000 7efe4effd640 1 --2- 172.21.15.112:0/1995271687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe500fb480 0x7efe501131b0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7efe40002a80 tx=0x7efe40004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:48:19.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.798+0000 7efe477fe640 1 -- 172.21.15.112:0/1995271687 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe40007c30 con 0x7efe500fb480 2024-09-16T00:48:19.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.798+0000 7efe557a5640 1 -- 172.21.15.112:0/1995271687 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe5010d410 con 0x7efe500fb480 2024-09-16T00:48:19.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.798+0000 7efe557a5640 1 -- 172.21.15.112:0/1995271687 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe5010d900 con 0x7efe500fb480 2024-09-16T00:48:19.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.798+0000 7efe477fe640 1 -- 172.21.15.112:0/1995271687 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7efe4000f040 con 0x7efe500fb480 2024-09-16T00:48:19.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.798+0000 7efe477fe640 1 -- 172.21.15.112:0/1995271687 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe4000dab0 con 0x7efe500fb480 2024-09-16T00:48:19.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.799+0000 7efe557a5640 1 -- 172.21.15.112:0/1995271687 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe500fcbc0 con 0x7efe500fb480 2024-09-16T00:48:19.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.800+0000 7efe477fe640 1 -- 172.21.15.112:0/1995271687 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7efe4000dc30 con 0x7efe500fb480 2024-09-16T00:48:19.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.800+0000 7efe477fe640 1 --2- 172.21.15.112:0/1995271687 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7efe3c06e530 0x7efe3c0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:19.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.800+0000 7efe477fe640 1 -- 172.21.15.112:0/1995271687 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7efe4008db60 con 0x7efe500fb480 2024-09-16T00:48:19.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.800+0000 7efe47fff640 1 --2- 172.21.15.112:0/1995271687 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7efe3c06e530 0x7efe3c0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:19.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.801+0000 7efe47fff640 1 --2- 172.21.15.112:0/1995271687 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7efe3c06e530 0x7efe3c0709e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7efe38009ac0 tx=0x7efe38035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:48:19.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.804+0000 7efe477fe640 1 -- 172.21.15.112:0/1995271687 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7efe40056b80 con 0x7efe500fb480 2024-09-16T00:48:19.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.988+0000 7efe557a5640 1 -- 172.21.15.112:0/1995271687 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7efe5004b720 con 0x7efe3c06e530 2024-09-16T00:48:19.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.989+0000 7efe477fe640 1 -- 172.21.15.112:0/1995271687 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7efe5004b720 con 0x7efe3c06e530 2024-09-16T00:48:19.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.991+0000 7efe557a5640 1 -- 172.21.15.112:0/1995271687 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7efe3c06e530 msgr2=0x7efe3c0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:19.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.991+0000 7efe557a5640 1 --2- 172.21.15.112:0/1995271687 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7efe3c06e530 0x7efe3c0709e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7efe38009ac0 tx=0x7efe38035040 comp rx=0 tx=0).stop 2024-09-16T00:48:19.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.991+0000 7efe557a5640 1 -- 172.21.15.112:0/1995271687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe500fb480 msgr2=0x7efe501131b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:19.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.991+0000 7efe557a5640 1 --2- 172.21.15.112:0/1995271687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe500fb480 0x7efe501131b0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7efe40002a80 tx=0x7efe40004600 comp rx=0 tx=0).stop 2024-09-16T00:48:19.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.991+0000 7efe557a5640 1 -- 172.21.15.112:0/1995271687 shutdown_connections 2024-09-16T00:48:19.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.991+0000 7efe557a5640 1 --2- 172.21.15.112:0/1995271687 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7efe3c06e530 0x7efe3c0709e0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:19.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.991+0000 7efe557a5640 1 --2- 172.21.15.112:0/1995271687 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe501014c0 0x7efe501136f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:19.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.991+0000 7efe557a5640 1 --2- 172.21.15.112:0/1995271687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe500fb480 0x7efe501131b0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:19.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.991+0000 7efe557a5640 1 -- 172.21.15.112:0/1995271687 >> 172.21.15.112:0/1995271687 conn(0x7efe500f7000 msgr2=0x7efe50065a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:48:19.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.992+0000 7efe557a5640 1 -- 172.21.15.112:0/1995271687 shutdown_connections 2024-09-16T00:48:19.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:19.992+0000 7efe557a5640 1 -- 172.21.15.112:0/1995271687 wait complete. 2024-09-16T00:48:20.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.069+0000 7f1cd4f52640 1 -- 172.21.15.112:0/3244173443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1cd0102530 msgr2=0x7f1cd0102900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:20.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.069+0000 7f1cd4f52640 1 --2- 172.21.15.112:0/3244173443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1cd0102530 0x7f1cd0102900 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f1cc00048d0 tx=0x7f1cc0009da0 comp rx=0 tx=0).stop 2024-09-16T00:48:20.070 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:48:19 smithi112 ceph-mon[97750]: pgmap v617: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:48:20.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.070+0000 7f1cd4f52640 1 -- 172.21.15.112:0/3244173443 shutdown_connections 2024-09-16T00:48:20.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.070+0000 7f1cd4f52640 1 --2- 172.21.15.112:0/3244173443 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1cd0102ed0 0x7f1cd00fc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:20.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.070+0000 7f1cd4f52640 1 --2- 172.21.15.112:0/3244173443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1cd0102530 0x7f1cd0102900 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:20.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.070+0000 7f1cd4f52640 1 -- 172.21.15.112:0/3244173443 >> 172.21.15.112:0/3244173443 conn(0x7f1cd00f80f0 msgr2=0x7f1cd00fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:48:20.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.070+0000 7f1cd4f52640 1 -- 172.21.15.112:0/3244173443 shutdown_connections 2024-09-16T00:48:20.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.070+0000 7f1cd4f52640 1 -- 172.21.15.112:0/3244173443 wait complete. 2024-09-16T00:48:20.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.070+0000 7f1cd4f52640 1 Processor -- start 2024-09-16T00:48:20.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.070+0000 7f1cd4f52640 1 -- start start 2024-09-16T00:48:20.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.070+0000 7f1cd4f52640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1cd0102530 0x7f1cd0113240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:20.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.070+0000 7f1cd4f52640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1cd0102ed0 0x7f1cd0113780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:20.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.070+0000 7f1cd4f52640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd0113d80 con 0x7f1cd0102ed0 2024-09-16T00:48:20.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.070+0000 7f1cd4f52640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd010d1c0 con 0x7f1cd0102530 2024-09-16T00:48:20.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.071+0000 7f1ccdd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1cd0102ed0 0x7f1cd0113780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:20.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.071+0000 7f1ccdd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1cd0102ed0 0x7f1cd0113780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46642/0 (socket says 172.21.15.112:46642) 2024-09-16T00:48:20.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.071+0000 7f1ccdd9b640 1 -- 172.21.15.112:0/2932062519 learned_addr learned my addr 172.21.15.112:0/2932062519 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:48:20.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.071+0000 7f1cce59c640 1 --2- 172.21.15.112:0/2932062519 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1cd0102530 0x7f1cd0113240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:20.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.071+0000 7f1ccdd9b640 1 -- 172.21.15.112:0/2932062519 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1cd0102530 msgr2=0x7f1cd0113240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:20.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.071+0000 7f1ccdd9b640 1 --2- 172.21.15.112:0/2932062519 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1cd0102530 0x7f1cd0113240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:20.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.071+0000 7f1ccdd9b640 1 -- 172.21.15.112:0/2932062519 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cc0005040 con 0x7f1cd0102ed0 2024-09-16T00:48:20.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.071+0000 7f1cce59c640 1 --2- 172.21.15.112:0/2932062519 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1cd0102530 0x7f1cd0113240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:48:20.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.071+0000 7f1ccdd9b640 1 --2- 172.21.15.112:0/2932062519 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1cd0102ed0 0x7f1cd0113780 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f1cc4002840 tx=0x7f1cc4004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:48:20.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.071+0000 7f1cbeffd640 1 -- 172.21.15.112:0/2932062519 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1cc4007c30 con 0x7f1cd0102ed0 2024-09-16T00:48:20.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.071+0000 7f1cd4f52640 1 -- 172.21.15.112:0/2932062519 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cd010d4a0 con 0x7f1cd0102ed0 2024-09-16T00:48:20.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.071+0000 7f1cbeffd640 1 -- 172.21.15.112:0/2932062519 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f1cc400f040 con 0x7f1cd0102ed0 2024-09-16T00:48:20.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.071+0000 7f1cbeffd640 1 -- 172.21.15.112:0/2932062519 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1cc4009510 con 0x7f1cd0102ed0 2024-09-16T00:48:20.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.071+0000 7f1cd4f52640 1 -- 172.21.15.112:0/2932062519 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cd010da40 con 0x7f1cd0102ed0 2024-09-16T00:48:20.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.072+0000 7f1cd4f52640 1 -- 172.21.15.112:0/2932062519 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cd00fdcb0 con 0x7f1cd0102ed0 2024-09-16T00:48:20.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.073+0000 7f1cbeffd640 1 -- 172.21.15.112:0/2932062519 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f1cc4009780 con 0x7f1cd0102ed0 2024-09-16T00:48:20.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.073+0000 7f1cbeffd640 1 --2- 172.21.15.112:0/2932062519 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f1cb806e530 0x7f1cb80709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:20.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.073+0000 7f1cbeffd640 1 -- 172.21.15.112:0/2932062519 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f1cc408db80 con 0x7f1cd0102ed0 2024-09-16T00:48:20.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.073+0000 7f1cce59c640 1 --2- 172.21.15.112:0/2932062519 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f1cb806e530 0x7f1cb80709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:20.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.074+0000 7f1cce59c640 1 --2- 172.21.15.112:0/2932062519 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f1cb806e530 0x7f1cb80709e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0007ac0 tx=0x7f1cc0035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:48:20.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.077+0000 7f1cbeffd640 1 -- 172.21.15.112:0/2932062519 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f1cc4056af0 con 0x7f1cd0102ed0 2024-09-16T00:48:20.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.252+0000 7f1cd4f52640 1 -- 172.21.15.112:0/2932062519 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1cd01095a0 con 0x7f1cb806e530 2024-09-16T00:48:20.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.261+0000 7f1cbeffd640 1 -- 172.21.15.112:0/2932062519 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7f1cd01095a0 con 0x7f1cb806e530 2024-09-16T00:48:20.262 INFO:teuthology.orchestra.run.smithi112.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T00:48:20.262 INFO:teuthology.orchestra.run.smithi112.stdout:alertmanager.smithi112 smithi112 *:9093,9094 running (21m) 14m ago 30m 24.7M - 0.25.0 c8568f914cd2 d85ea0562022 2024-09-16T00:48:20.262 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi112 smithi112 running (18m) 14m ago 30m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c e4fe58f9702a 2024-09-16T00:48:20.262 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi169 smithi169 running (18m) 13m ago 29m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c 3ba2c81d527f 2024-09-16T00:48:20.263 INFO:teuthology.orchestra.run.smithi112.stdout:grafana.smithi112 smithi112 *:3000 running (20m) 14m ago 29m 82.1M - 9.4.12 2bacad6d85d8 51f9d2013da2 2024-09-16T00:48:20.263 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.hxfwkh smithi112 running (14m) 14m ago 24m 14.2M - 19.1.1-326-g8921c8ef f1455bd0219c 1723069902d6 2024-09-16T00:48:20.263 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.ithwzj smithi112 running (14m) 14m ago 24m 12.2M - 19.1.1-326-g8921c8ef f1455bd0219c e7aad5b81db5 2024-09-16T00:48:20.263 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.gawwgw smithi169 running (14m) 13m ago 24m 15.7M - 19.1.1-326-g8921c8ef f1455bd0219c 469be459ee12 2024-09-16T00:48:20.263 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.lgeqrj smithi169 running (13m) 13m ago 24m 13.3M - 19.1.1-326-g8921c8ef f1455bd0219c 31f0fbd2395d 2024-09-16T00:48:20.263 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi112.xrkkbc smithi112 *:8443,9283,8765 running (20m) 14m ago 31m 504M - 19.1.1-326-g8921c8ef f1455bd0219c a981f7fd2810 2024-09-16T00:48:20.264 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi169.mtrobk smithi169 *:8443,9283,8765 running (20m) 13m ago 29m 442M - 19.1.1-326-g8921c8ef f1455bd0219c 4096e5369c2d 2024-09-16T00:48:20.264 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi112 smithi112 running (19m) 14m ago 31m 49.8M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 5aaec033c042 2024-09-16T00:48:20.264 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi169 smithi169 running (18m) 13m ago 29m 38.9M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 7bc609155f3f 2024-09-16T00:48:20.264 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi112 smithi112 *:9100 running (21m) 14m ago 30m 15.5M - 1.5.0 0da6a335fe13 6404d1d601e2 2024-09-16T00:48:20.264 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi169 smithi169 *:9100 running (20m) 13m ago 29m 15.4M - 1.5.0 0da6a335fe13 5505baf702a6 2024-09-16T00:48:20.264 INFO:teuthology.orchestra.run.smithi112.stdout:osd.0 smithi112 running (17m) 14m ago 28m 105M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 53efc9397017 2024-09-16T00:48:20.264 INFO:teuthology.orchestra.run.smithi112.stdout:osd.1 smithi112 running (17m) 14m ago 28m 98.9M 1557M 19.1.1-326-g8921c8ef f1455bd0219c fe6f1a4a2cf4 2024-09-16T00:48:20.265 INFO:teuthology.orchestra.run.smithi112.stdout:osd.2 smithi112 running (16m) 14m ago 27m 102M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 5f7863712057 2024-09-16T00:48:20.265 INFO:teuthology.orchestra.run.smithi112.stdout:osd.3 smithi169 running (16m) 13m ago 27m 92.1M 2581M 19.1.1-326-g8921c8ef f1455bd0219c ee07b5577810 2024-09-16T00:48:20.265 INFO:teuthology.orchestra.run.smithi112.stdout:osd.4 smithi169 running (15m) 13m ago 26m 103M 2581M 19.1.1-326-g8921c8ef f1455bd0219c 769f026a54eb 2024-09-16T00:48:20.265 INFO:teuthology.orchestra.run.smithi112.stdout:osd.5 smithi169 running (15m) 13m ago 26m 125M 2581M 19.1.1-326-g8921c8ef f1455bd0219c a6007c13cf78 2024-09-16T00:48:20.265 INFO:teuthology.orchestra.run.smithi112.stdout:prometheus.smithi112 smithi112 *:9095 running (20m) 14m ago 29m 64.1M - 2.43.0 a07b618ecd1d 7a1d42e60d2b 2024-09-16T00:48:20.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.263+0000 7f1cd4f52640 1 -- 172.21.15.112:0/2932062519 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f1cb806e530 msgr2=0x7f1cb80709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:20.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.263+0000 7f1cd4f52640 1 --2- 172.21.15.112:0/2932062519 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f1cb806e530 0x7f1cb80709e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0007ac0 tx=0x7f1cc0035040 comp rx=0 tx=0).stop 2024-09-16T00:48:20.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.263+0000 7f1cd4f52640 1 -- 172.21.15.112:0/2932062519 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1cd0102ed0 msgr2=0x7f1cd0113780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:20.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.263+0000 7f1cd4f52640 1 --2- 172.21.15.112:0/2932062519 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1cd0102ed0 0x7f1cd0113780 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f1cc4002840 tx=0x7f1cc4004570 comp rx=0 tx=0).stop 2024-09-16T00:48:20.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.263+0000 7f1cd4f52640 1 -- 172.21.15.112:0/2932062519 shutdown_connections 2024-09-16T00:48:20.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.263+0000 7f1cd4f52640 1 --2- 172.21.15.112:0/2932062519 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f1cb806e530 0x7f1cb80709e0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:20.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.263+0000 7f1cd4f52640 1 --2- 172.21.15.112:0/2932062519 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1cd0102ed0 0x7f1cd0113780 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:20.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.263+0000 7f1cd4f52640 1 --2- 172.21.15.112:0/2932062519 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1cd0102530 0x7f1cd0113240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:20.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.263+0000 7f1cd4f52640 1 -- 172.21.15.112:0/2932062519 >> 172.21.15.112:0/2932062519 conn(0x7f1cd00f80f0 msgr2=0x7f1cd00f90f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:48:20.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.263+0000 7f1cd4f52640 1 -- 172.21.15.112:0/2932062519 shutdown_connections 2024-09-16T00:48:20.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.264+0000 7f1cd4f52640 1 -- 172.21.15.112:0/2932062519 wait complete. 2024-09-16T00:48:20.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:48:19 smithi169 ceph-mon[77626]: pgmap v617: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:48:20.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.340+0000 7f74e2c28640 1 -- 172.21.15.112:0/2737927259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74dc0fdf70 msgr2=0x7f74dc1065b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:20.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.340+0000 7f74e2c28640 1 --2- 172.21.15.112:0/2737927259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74dc0fdf70 0x7f74dc1065b0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f74c8004900 tx=0x7f74c8009da0 comp rx=0 tx=0).stop 2024-09-16T00:48:20.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.340+0000 7f74e2c28640 1 -- 172.21.15.112:0/2737927259 shutdown_connections 2024-09-16T00:48:20.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.340+0000 7f74e2c28640 1 --2- 172.21.15.112:0/2737927259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74dc0fdf70 0x7f74dc1065b0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:20.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.340+0000 7f74e2c28640 1 --2- 172.21.15.112:0/2737927259 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f74dc0fd5d0 0x7f74dc0fd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:20.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.340+0000 7f74e2c28640 1 -- 172.21.15.112:0/2737927259 >> 172.21.15.112:0/2737927259 conn(0x7f74dc0f7060 msgr2=0x7f74dc0f9470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:48:20.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.340+0000 7f74e2c28640 1 -- 172.21.15.112:0/2737927259 shutdown_connections 2024-09-16T00:48:20.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.340+0000 7f74e2c28640 1 -- 172.21.15.112:0/2737927259 wait complete. 2024-09-16T00:48:20.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.341+0000 7f74e2c28640 1 Processor -- start 2024-09-16T00:48:20.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.341+0000 7f74e2c28640 1 -- start start 2024-09-16T00:48:20.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.341+0000 7f74e2c28640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74dc0fd5d0 0x7f74dc1131f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:20.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.341+0000 7f74e2c28640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f74dc0fdf70 0x7f74dc113730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:20.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.341+0000 7f74e2c28640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74dc113d30 con 0x7f74dc0fd5d0 2024-09-16T00:48:20.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.341+0000 7f74e2c28640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74dc10d170 con 0x7f74dc0fdf70 2024-09-16T00:48:20.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.341+0000 7f74e09c4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74dc0fd5d0 0x7f74dc1131f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:20.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.342+0000 7f74e09c4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74dc0fd5d0 0x7f74dc1131f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46650/0 (socket says 172.21.15.112:46650) 2024-09-16T00:48:20.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.342+0000 7f74e09c4640 1 -- 172.21.15.112:0/2945489962 learned_addr learned my addr 172.21.15.112:0/2945489962 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:48:20.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.342+0000 7f74dbfff640 1 --2- 172.21.15.112:0/2945489962 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f74dc0fdf70 0x7f74dc113730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:20.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.342+0000 7f74e09c4640 1 -- 172.21.15.112:0/2945489962 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f74dc0fdf70 msgr2=0x7f74dc113730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:20.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.342+0000 7f74e09c4640 1 --2- 172.21.15.112:0/2945489962 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f74dc0fdf70 0x7f74dc113730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:20.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.342+0000 7f74e09c4640 1 -- 172.21.15.112:0/2945489962 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c8005040 con 0x7f74dc0fd5d0 2024-09-16T00:48:20.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.342+0000 7f74e09c4640 1 --2- 172.21.15.112:0/2945489962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74dc0fd5d0 0x7f74dc1131f0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f74d000c830 tx=0x7f74d0004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:48:20.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.342+0000 7f74d97fa640 1 -- 172.21.15.112:0/2945489962 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f74d000a7d0 con 0x7f74dc0fd5d0 2024-09-16T00:48:20.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.342+0000 7f74d97fa640 1 -- 172.21.15.112:0/2945489962 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f74d000add0 con 0x7f74dc0fd5d0 2024-09-16T00:48:20.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.342+0000 7f74d97fa640 1 -- 172.21.15.112:0/2945489962 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f74d00113a0 con 0x7f74dc0fd5d0 2024-09-16T00:48:20.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.342+0000 7f74e2c28640 1 -- 172.21.15.112:0/2945489962 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74dc10d450 con 0x7f74dc0fd5d0 2024-09-16T00:48:20.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.342+0000 7f74e2c28640 1 -- 172.21.15.112:0/2945489962 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74dc10d970 con 0x7f74dc0fd5d0 2024-09-16T00:48:20.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.343+0000 7f74e2c28640 1 -- 172.21.15.112:0/2945489962 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74dc066310 con 0x7f74dc0fd5d0 2024-09-16T00:48:20.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.344+0000 7f74d97fa640 1 -- 172.21.15.112:0/2945489962 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f74d0012400 con 0x7f74dc0fd5d0 2024-09-16T00:48:20.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.345+0000 7f74d97fa640 1 --2- 172.21.15.112:0/2945489962 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f74cc06e530 0x7f74cc0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:20.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.345+0000 7f74d97fa640 1 -- 172.21.15.112:0/2945489962 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f74d008e400 con 0x7f74dc0fd5d0 2024-09-16T00:48:20.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.345+0000 7f74dbfff640 1 --2- 172.21.15.112:0/2945489962 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f74cc06e530 0x7f74cc0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:20.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.345+0000 7f74dbfff640 1 --2- 172.21.15.112:0/2945489962 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f74cc06e530 0x7f74cc0709e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f74c802a630 tx=0x7f74c8035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:48:20.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.349+0000 7f74d97fa640 1 -- 172.21.15.112:0/2945489962 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f74d00573f0 con 0x7f74dc0fd5d0 2024-09-16T00:48:20.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.577+0000 7f74e2c28640 1 -- 172.21.15.112:0/2945489962 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f74dc04b720 con 0x7f74dc0fd5d0 2024-09-16T00:48:20.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.578+0000 7f74d97fa640 1 -- 172.21.15.112:0/2945489962 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f74d005aab0 con 0x7f74dc0fd5d0 2024-09-16T00:48:20.578 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:48:20.578 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": { 2024-09-16T00:48:20.579 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:48:20.579 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:48:20.579 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": { 2024-09-16T00:48:20.579 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:48:20.579 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:48:20.579 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": { 2024-09-16T00:48:20.579 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 6 2024-09-16T00:48:20.579 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:48:20.579 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": { 2024-09-16T00:48:20.579 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 4 2024-09-16T00:48:20.579 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:48:20.579 INFO:teuthology.orchestra.run.smithi112.stdout: "overall": { 2024-09-16T00:48:20.580 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 14 2024-09-16T00:48:20.580 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-16T00:48:20.580 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:48:20.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.580+0000 7f74e2c28640 1 -- 172.21.15.112:0/2945489962 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f74cc06e530 msgr2=0x7f74cc0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:20.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.580+0000 7f74e2c28640 1 --2- 172.21.15.112:0/2945489962 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f74cc06e530 0x7f74cc0709e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f74c802a630 tx=0x7f74c8035040 comp rx=0 tx=0).stop 2024-09-16T00:48:20.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.580+0000 7f74e2c28640 1 -- 172.21.15.112:0/2945489962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74dc0fd5d0 msgr2=0x7f74dc1131f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:20.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.580+0000 7f74e2c28640 1 --2- 172.21.15.112:0/2945489962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74dc0fd5d0 0x7f74dc1131f0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f74d000c830 tx=0x7f74d0004430 comp rx=0 tx=0).stop 2024-09-16T00:48:20.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.580+0000 7f74e2c28640 1 -- 172.21.15.112:0/2945489962 shutdown_connections 2024-09-16T00:48:20.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.580+0000 7f74e2c28640 1 --2- 172.21.15.112:0/2945489962 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f74cc06e530 0x7f74cc0709e0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:20.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.580+0000 7f74e2c28640 1 --2- 172.21.15.112:0/2945489962 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f74dc0fdf70 0x7f74dc113730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:20.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.580+0000 7f74e2c28640 1 --2- 172.21.15.112:0/2945489962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74dc0fd5d0 0x7f74dc1131f0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:20.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.580+0000 7f74e2c28640 1 -- 172.21.15.112:0/2945489962 >> 172.21.15.112:0/2945489962 conn(0x7f74dc0f7060 msgr2=0x7f74dc100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:48:20.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.580+0000 7f74e2c28640 1 -- 172.21.15.112:0/2945489962 shutdown_connections 2024-09-16T00:48:20.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.580+0000 7f74e2c28640 1 -- 172.21.15.112:0/2945489962 wait complete. 2024-09-16T00:48:20.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.658+0000 7f676d19d640 1 -- 172.21.15.112:0/2228239239 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67681023b0 msgr2=0x7f6768102780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:20.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.658+0000 7f676d19d640 1 --2- 172.21.15.112:0/2228239239 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67681023b0 0x7f6768102780 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f67580048d0 tx=0x7f6758009da0 comp rx=0 tx=0).stop 2024-09-16T00:48:20.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.658+0000 7f676d19d640 1 -- 172.21.15.112:0/2228239239 shutdown_connections 2024-09-16T00:48:20.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.658+0000 7f676d19d640 1 --2- 172.21.15.112:0/2228239239 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6768102d50 0x7f67680fc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:20.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.658+0000 7f676d19d640 1 --2- 172.21.15.112:0/2228239239 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67681023b0 0x7f6768102780 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:20.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.658+0000 7f676d19d640 1 -- 172.21.15.112:0/2228239239 >> 172.21.15.112:0/2228239239 conn(0x7f67680f7f60 msgr2=0x7f67680fa370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:48:20.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.658+0000 7f676d19d640 1 -- 172.21.15.112:0/2228239239 shutdown_connections 2024-09-16T00:48:20.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.658+0000 7f676d19d640 1 -- 172.21.15.112:0/2228239239 wait complete. 2024-09-16T00:48:20.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.659+0000 7f676d19d640 1 Processor -- start 2024-09-16T00:48:20.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.659+0000 7f676d19d640 1 -- start start 2024-09-16T00:48:20.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.659+0000 7f676d19d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f67681023b0 0x7f6768112fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:20.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.659+0000 7f676d19d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6768102d50 0x7f6768113520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:20.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.659+0000 7f676d19d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6768113b20 con 0x7f6768102d50 2024-09-16T00:48:20.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.659+0000 7f676d19d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f676810cf60 con 0x7f67681023b0 2024-09-16T00:48:20.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.659+0000 7f67677fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6768102d50 0x7f6768113520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:20.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.659+0000 7f67677fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6768102d50 0x7f6768113520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46666/0 (socket says 172.21.15.112:46666) 2024-09-16T00:48:20.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.659+0000 7f67677fe640 1 -- 172.21.15.112:0/985310279 learned_addr learned my addr 172.21.15.112:0/985310279 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:48:20.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.659+0000 7f6767fff640 1 --2- 172.21.15.112:0/985310279 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f67681023b0 0x7f6768112fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:20.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.660+0000 7f67677fe640 1 -- 172.21.15.112:0/985310279 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f67681023b0 msgr2=0x7f6768112fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:20.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.660+0000 7f67677fe640 1 --2- 172.21.15.112:0/985310279 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f67681023b0 0x7f6768112fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:20.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.660+0000 7f67677fe640 1 -- 172.21.15.112:0/985310279 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6758005040 con 0x7f6768102d50 2024-09-16T00:48:20.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.660+0000 7f67677fe640 1 --2- 172.21.15.112:0/985310279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6768102d50 0x7f6768113520 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f675c002a50 tx=0x7f675c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:48:20.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.660+0000 7f6764ff9640 1 -- 172.21.15.112:0/985310279 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f675c007c30 con 0x7f6768102d50 2024-09-16T00:48:20.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.660+0000 7f6764ff9640 1 -- 172.21.15.112:0/985310279 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f675c00f040 con 0x7f6768102d50 2024-09-16T00:48:20.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.660+0000 7f676d19d640 1 -- 172.21.15.112:0/985310279 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f676810d240 con 0x7f6768102d50 2024-09-16T00:48:20.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.660+0000 7f6764ff9640 1 -- 172.21.15.112:0/985310279 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f675c00db50 con 0x7f6768102d50 2024-09-16T00:48:20.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.660+0000 7f676d19d640 1 -- 172.21.15.112:0/985310279 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f676810d7e0 con 0x7f6768102d50 2024-09-16T00:48:20.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.661+0000 7f676d19d640 1 -- 172.21.15.112:0/985310279 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67680fdb20 con 0x7f6768102d50 2024-09-16T00:48:20.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.662+0000 7f6764ff9640 1 -- 172.21.15.112:0/985310279 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f675c00dcb0 con 0x7f6768102d50 2024-09-16T00:48:20.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.662+0000 7f6764ff9640 1 --2- 172.21.15.112:0/985310279 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f675006e530 0x7f67500709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:20.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.662+0000 7f6764ff9640 1 -- 172.21.15.112:0/985310279 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f675c08e210 con 0x7f6768102d50 2024-09-16T00:48:20.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.662+0000 7f6767fff640 1 --2- 172.21.15.112:0/985310279 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f675006e530 0x7f67500709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:20.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.663+0000 7f6767fff640 1 --2- 172.21.15.112:0/985310279 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f675006e530 0x7f67500709e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6758009ac0 tx=0x7f6758035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:48:20.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.667+0000 7f6764ff9640 1 -- 172.21.15.112:0/985310279 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f675c057270 con 0x7f6768102d50 2024-09-16T00:48:20.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.867+0000 7f676d19d640 1 -- 172.21.15.112:0/985310279 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f67681093a0 con 0x7f6768102d50 2024-09-16T00:48:20.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.867+0000 7f6764ff9640 1 -- 172.21.15.112:0/985310279 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1914 (secure 0 0 0) 0x7f675c05a930 con 0x7f6768102d50 2024-09-16T00:48:20.868 INFO:teuthology.orchestra.run.smithi112.stdout:e34 2024-09-16T00:48:20.868 INFO:teuthology.orchestra.run.smithi112.stdout:btime 2024-09-16T00:34:26:619254+0000 2024-09-16T00:48:20.868 INFO:teuthology.orchestra.run.smithi112.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T00:48:20.869 INFO:teuthology.orchestra.run.smithi112.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:48:20.869 INFO:teuthology.orchestra.run.smithi112.stdout:legacy client fscid: 1 2024-09-16T00:48:20.869 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:48:20.869 INFO:teuthology.orchestra.run.smithi112.stdout:Filesystem 'cephfs' (1) 2024-09-16T00:48:20.869 INFO:teuthology.orchestra.run.smithi112.stdout:fs_name cephfs 2024-09-16T00:48:20.870 INFO:teuthology.orchestra.run.smithi112.stdout:epoch 25 2024-09-16T00:48:20.870 INFO:teuthology.orchestra.run.smithi112.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-16T00:48:20.870 INFO:teuthology.orchestra.run.smithi112.stdout:created 2024-09-16T00:23:22.328199+0000 2024-09-16T00:48:20.870 INFO:teuthology.orchestra.run.smithi112.stdout:modified 2024-09-16T00:33:29.012991+0000 2024-09-16T00:48:20.870 INFO:teuthology.orchestra.run.smithi112.stdout:tableserver 0 2024-09-16T00:48:20.870 INFO:teuthology.orchestra.run.smithi112.stdout:root 0 2024-09-16T00:48:20.870 INFO:teuthology.orchestra.run.smithi112.stdout:session_timeout 60 2024-09-16T00:48:20.871 INFO:teuthology.orchestra.run.smithi112.stdout:session_autoclose 300 2024-09-16T00:48:20.871 INFO:teuthology.orchestra.run.smithi112.stdout:max_file_size 1099511627776 2024-09-16T00:48:20.871 INFO:teuthology.orchestra.run.smithi112.stdout:max_xattr_size 65536 2024-09-16T00:48:20.871 INFO:teuthology.orchestra.run.smithi112.stdout:required_client_features {} 2024-09-16T00:48:20.871 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure 0 2024-09-16T00:48:20.871 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure_osd_epoch 82 2024-09-16T00:48:20.872 INFO:teuthology.orchestra.run.smithi112.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:48:20.872 INFO:teuthology.orchestra.run.smithi112.stdout:max_mds 2 2024-09-16T00:48:20.872 INFO:teuthology.orchestra.run.smithi112.stdout:in 0,1 2024-09-16T00:48:20.872 INFO:teuthology.orchestra.run.smithi112.stdout:up {} 2024-09-16T00:48:20.872 INFO:teuthology.orchestra.run.smithi112.stdout:failed 0,1 2024-09-16T00:48:20.873 INFO:teuthology.orchestra.run.smithi112.stdout:damaged 2024-09-16T00:48:20.873 INFO:teuthology.orchestra.run.smithi112.stdout:stopped 2024-09-16T00:48:20.873 INFO:teuthology.orchestra.run.smithi112.stdout:data_pools [3] 2024-09-16T00:48:20.873 INFO:teuthology.orchestra.run.smithi112.stdout:metadata_pool 2 2024-09-16T00:48:20.873 INFO:teuthology.orchestra.run.smithi112.stdout:inline_data enabled 2024-09-16T00:48:20.873 INFO:teuthology.orchestra.run.smithi112.stdout:balancer 2024-09-16T00:48:20.874 INFO:teuthology.orchestra.run.smithi112.stdout:bal_rank_mask -1 2024-09-16T00:48:20.874 INFO:teuthology.orchestra.run.smithi112.stdout:standby_count_wanted 1 2024-09-16T00:48:20.874 INFO:teuthology.orchestra.run.smithi112.stdout:qdb_cluster leader: 0 members: 2024-09-16T00:48:20.874 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:48:20.874 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:48:20.874 INFO:teuthology.orchestra.run.smithi112.stdout:Standby daemons: 2024-09-16T00:48:20.874 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:48:20.875 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.ithwzj{-1:34392} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.112:6826/476229221,v1:172.21.15.112:6827/476229221] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:48:20.875 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.hxfwkh{-1:34404} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.112:6828/2356899876,v1:172.21.15.112:6829/2356899876] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:48:20.875 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.gawwgw{-1:44343} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.169:6824/1635329957,v1:172.21.15.169:6825/1635329957] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:48:20.875 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.lgeqrj{-1:44347} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.169:6826/184810031,v1:172.21.15.169:6827/184810031] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:48:20.877 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 34 2024-09-16T00:48:20.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.869+0000 7f676d19d640 1 -- 172.21.15.112:0/985310279 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f675006e530 msgr2=0x7f67500709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:20.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.869+0000 7f676d19d640 1 --2- 172.21.15.112:0/985310279 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f675006e530 0x7f67500709e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6758009ac0 tx=0x7f6758035040 comp rx=0 tx=0).stop 2024-09-16T00:48:20.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.869+0000 7f676d19d640 1 -- 172.21.15.112:0/985310279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6768102d50 msgr2=0x7f6768113520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:20.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.869+0000 7f676d19d640 1 --2- 172.21.15.112:0/985310279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6768102d50 0x7f6768113520 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f675c002a50 tx=0x7f675c004600 comp rx=0 tx=0).stop 2024-09-16T00:48:20.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.869+0000 7f676d19d640 1 -- 172.21.15.112:0/985310279 shutdown_connections 2024-09-16T00:48:20.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.869+0000 7f676d19d640 1 --2- 172.21.15.112:0/985310279 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f675006e530 0x7f67500709e0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:20.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.869+0000 7f676d19d640 1 --2- 172.21.15.112:0/985310279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6768102d50 0x7f6768113520 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:20.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.869+0000 7f676d19d640 1 --2- 172.21.15.112:0/985310279 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f67681023b0 0x7f6768112fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:20.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.869+0000 7f676d19d640 1 -- 172.21.15.112:0/985310279 >> 172.21.15.112:0/985310279 conn(0x7f67680f7f60 msgr2=0x7f67680f8f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:48:20.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.869+0000 7f676d19d640 1 -- 172.21.15.112:0/985310279 shutdown_connections 2024-09-16T00:48:20.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.869+0000 7f676d19d640 1 -- 172.21.15.112:0/985310279 wait complete. 2024-09-16T00:48:20.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.948+0000 7fcd53164640 1 -- 172.21.15.112:0/1611361146 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd4c102ed0 msgr2=0x7fcd4c0fc590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:20.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.948+0000 7fcd53164640 1 --2- 172.21.15.112:0/1611361146 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd4c102ed0 0x7fcd4c0fc590 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fcd400048d0 tx=0x7fcd40009da0 comp rx=0 tx=0).stop 2024-09-16T00:48:20.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.948+0000 7fcd53164640 1 -- 172.21.15.112:0/1611361146 shutdown_connections 2024-09-16T00:48:20.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.948+0000 7fcd53164640 1 --2- 172.21.15.112:0/1611361146 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd4c102ed0 0x7fcd4c0fc590 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:20.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.948+0000 7fcd53164640 1 --2- 172.21.15.112:0/1611361146 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd4c102530 0x7fcd4c102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:20.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.948+0000 7fcd53164640 1 -- 172.21.15.112:0/1611361146 >> 172.21.15.112:0/1611361146 conn(0x7fcd4c0f80f0 msgr2=0x7fcd4c0fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:48:20.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.948+0000 7fcd53164640 1 -- 172.21.15.112:0/1611361146 shutdown_connections 2024-09-16T00:48:20.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.948+0000 7fcd53164640 1 -- 172.21.15.112:0/1611361146 wait complete. 2024-09-16T00:48:20.950 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:48:20 smithi112 ceph-mon[97750]: from='client.35164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:48:20.951 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:48:20 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/2945489962' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:48:20.951 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:48:20 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/985310279' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:48:20.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.949+0000 7fcd53164640 1 Processor -- start 2024-09-16T00:48:20.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.949+0000 7fcd53164640 1 -- start start 2024-09-16T00:48:20.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.949+0000 7fcd53164640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd4c102530 0x7fcd4c06fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:20.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.949+0000 7fcd53164640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd4c102ed0 0x7fcd4c06ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:20.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.949+0000 7fcd53164640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4c070490 con 0x7fcd4c102ed0 2024-09-16T00:48:20.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.949+0000 7fcd53164640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4c070600 con 0x7fcd4c102530 2024-09-16T00:48:20.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.950+0000 7fcd4bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd4c102ed0 0x7fcd4c06ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:20.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.950+0000 7fcd4bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd4c102ed0 0x7fcd4c06ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46674/0 (socket says 172.21.15.112:46674) 2024-09-16T00:48:20.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.950+0000 7fcd4bfff640 1 -- 172.21.15.112:0/729789581 learned_addr learned my addr 172.21.15.112:0/729789581 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:48:20.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.950+0000 7fcd50f00640 1 --2- 172.21.15.112:0/729789581 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd4c102530 0x7fcd4c06fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:20.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.950+0000 7fcd4bfff640 1 -- 172.21.15.112:0/729789581 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd4c102530 msgr2=0x7fcd4c06fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:20.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.950+0000 7fcd4bfff640 1 --2- 172.21.15.112:0/729789581 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd4c102530 0x7fcd4c06fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:20.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.950+0000 7fcd4bfff640 1 -- 172.21.15.112:0/729789581 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd40005040 con 0x7fcd4c102ed0 2024-09-16T00:48:20.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.950+0000 7fcd4bfff640 1 --2- 172.21.15.112:0/729789581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd4c102ed0 0x7fcd4c06ff50 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fcd400043d0 tx=0x7fcd40009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:48:20.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.950+0000 7fcd497fa640 1 -- 172.21.15.112:0/729789581 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd40038070 con 0x7fcd4c102ed0 2024-09-16T00:48:20.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.950+0000 7fcd497fa640 1 -- 172.21.15.112:0/729789581 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fcd400096e0 con 0x7fcd4c102ed0 2024-09-16T00:48:20.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.950+0000 7fcd53164640 1 -- 172.21.15.112:0/729789581 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd4c070830 con 0x7fcd4c102ed0 2024-09-16T00:48:20.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.950+0000 7fcd497fa640 1 -- 172.21.15.112:0/729789581 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd4002c8a0 con 0x7fcd4c102ed0 2024-09-16T00:48:20.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.950+0000 7fcd53164640 1 -- 172.21.15.112:0/729789581 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd4c119870 con 0x7fcd4c102ed0 2024-09-16T00:48:20.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.951+0000 7fcd53164640 1 -- 172.21.15.112:0/729789581 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd4c0fdcb0 con 0x7fcd4c102ed0 2024-09-16T00:48:20.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.952+0000 7fcd497fa640 1 -- 172.21.15.112:0/729789581 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fcd40030020 con 0x7fcd4c102ed0 2024-09-16T00:48:20.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.952+0000 7fcd497fa640 1 --2- 172.21.15.112:0/729789581 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fcd3406e530 0x7fcd340709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:20.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.952+0000 7fcd497fa640 1 -- 172.21.15.112:0/729789581 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7fcd400adff0 con 0x7fcd4c102ed0 2024-09-16T00:48:20.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.952+0000 7fcd50f00640 1 --2- 172.21.15.112:0/729789581 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fcd3406e530 0x7fcd340709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:20.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.953+0000 7fcd50f00640 1 --2- 172.21.15.112:0/729789581 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fcd3406e530 0x7fcd340709e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c007fa0 tx=0x7fcd3c004c70 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:48:20.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:20.958+0000 7fcd497fa640 1 -- 172.21.15.112:0/729789581 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fcd40076fe0 con 0x7fcd4c102ed0 2024-09-16T00:48:21.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:21.140+0000 7fcd53164640 1 -- 172.21.15.112:0/729789581 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcd4c1095d0 con 0x7fcd3406e530 2024-09-16T00:48:21.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:21.141+0000 7fcd497fa640 1 -- 172.21.15.112:0/729789581 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fcd4c1095d0 con 0x7fcd3406e530 2024-09-16T00:48:21.142 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:48:21.142 INFO:teuthology.orchestra.run.smithi112.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80", 2024-09-16T00:48:21.142 INFO:teuthology.orchestra.run.smithi112.stdout: "in_progress": true, 2024-09-16T00:48:21.142 INFO:teuthology.orchestra.run.smithi112.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T00:48:21.142 INFO:teuthology.orchestra.run.smithi112.stdout: "services_complete": [ 2024-09-16T00:48:21.143 INFO:teuthology.orchestra.run.smithi112.stdout: "mds", 2024-09-16T00:48:21.143 INFO:teuthology.orchestra.run.smithi112.stdout: "osd", 2024-09-16T00:48:21.143 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr", 2024-09-16T00:48:21.143 INFO:teuthology.orchestra.run.smithi112.stdout: "mon", 2024-09-16T00:48:21.143 INFO:teuthology.orchestra.run.smithi112.stdout: "crash" 2024-09-16T00:48:21.143 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-16T00:48:21.144 INFO:teuthology.orchestra.run.smithi112.stdout: "progress": "16/21 daemons upgraded", 2024-09-16T00:48:21.144 INFO:teuthology.orchestra.run.smithi112.stdout: "message": "Currently upgrading mds daemons", 2024-09-16T00:48:21.144 INFO:teuthology.orchestra.run.smithi112.stdout: "is_paused": false 2024-09-16T00:48:21.144 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:48:21.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:21.143+0000 7fcd53164640 1 -- 172.21.15.112:0/729789581 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fcd3406e530 msgr2=0x7fcd340709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:21.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:21.143+0000 7fcd53164640 1 --2- 172.21.15.112:0/729789581 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fcd3406e530 0x7fcd340709e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c007fa0 tx=0x7fcd3c004c70 comp rx=0 tx=0).stop 2024-09-16T00:48:21.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:21.143+0000 7fcd53164640 1 -- 172.21.15.112:0/729789581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd4c102ed0 msgr2=0x7fcd4c06ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:21.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:21.143+0000 7fcd53164640 1 --2- 172.21.15.112:0/729789581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd4c102ed0 0x7fcd4c06ff50 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fcd400043d0 tx=0x7fcd40009da0 comp rx=0 tx=0).stop 2024-09-16T00:48:21.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:21.143+0000 7fcd53164640 1 -- 172.21.15.112:0/729789581 shutdown_connections 2024-09-16T00:48:21.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:21.143+0000 7fcd53164640 1 --2- 172.21.15.112:0/729789581 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fcd3406e530 0x7fcd340709e0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:21.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:21.143+0000 7fcd53164640 1 --2- 172.21.15.112:0/729789581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd4c102ed0 0x7fcd4c06ff50 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:21.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:21.143+0000 7fcd53164640 1 --2- 172.21.15.112:0/729789581 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd4c102530 0x7fcd4c06fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:21.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:21.143+0000 7fcd53164640 1 -- 172.21.15.112:0/729789581 >> 172.21.15.112:0/729789581 conn(0x7fcd4c0f80f0 msgr2=0x7fcd4c0f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:48:21.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:21.144+0000 7fcd53164640 1 -- 172.21.15.112:0/729789581 shutdown_connections 2024-09-16T00:48:21.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:21.144+0000 7fcd53164640 1 -- 172.21.15.112:0/729789581 wait complete. 2024-09-16T00:48:21.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:21.221+0000 7f3fa6116640 1 -- 172.21.15.112:0/372257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fa0102ef0 msgr2=0x7f3fa00fc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:21.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:21.221+0000 7f3fa6116640 1 --2- 172.21.15.112:0/372257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fa0102ef0 0x7f3fa00fc5b0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f3f940048d0 tx=0x7f3f94009da0 comp rx=0 tx=0).stop 2024-09-16T00:48:21.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:21.221+0000 7f3fa6116640 1 -- 172.21.15.112:0/372257 shutdown_connections 2024-09-16T00:48:21.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:21.221+0000 7f3fa6116640 1 --2- 172.21.15.112:0/372257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fa0102ef0 0x7f3fa00fc5b0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:21.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:21.221+0000 7f3fa6116640 1 --2- 172.21.15.112:0/372257 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3fa0102550 0x7f3fa0102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:21.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:21.221+0000 7f3fa6116640 1 -- 172.21.15.112:0/372257 >> 172.21.15.112:0/372257 conn(0x7f3fa00f80f0 msgr2=0x7f3fa00fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:48:21.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:21.221+0000 7f3fa6116640 1 -- 172.21.15.112:0/372257 shutdown_connections 2024-09-16T00:48:21.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:21.221+0000 7f3fa6116640 1 -- 172.21.15.112:0/372257 wait complete. 2024-09-16T00:48:21.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:21.222+0000 7f3fa6116640 1 Processor -- start 2024-09-16T00:48:21.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:21.222+0000 7f3fa6116640 1 -- start start 2024-09-16T00:48:21.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:21.222+0000 7f3fa6116640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3fa0102550 0x7f3fa0113520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:21.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:21.222+0000 7f3fa6116640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fa0102ef0 0x7f3fa0113a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:21.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:21.222+0000 7f3fa6116640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa01140b0 con 0x7f3fa0102ef0 2024-09-16T00:48:21.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:21.222+0000 7f3fa6116640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa010d4c0 con 0x7f3fa0102550 2024-09-16T00:48:21.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:21.222+0000 7f3f9effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fa0102ef0 0x7f3fa0113a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:21.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:21.222+0000 7f3f9effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fa0102ef0 0x7f3fa0113a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46692/0 (socket says 172.21.15.112:46692) 2024-09-16T00:48:21.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:21.222+0000 7f3f9f7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3fa0102550 0x7f3fa0113520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:21.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:21.222+0000 7f3f9effd640 1 -- 172.21.15.112:0/1700699357 learned_addr learned my addr 172.21.15.112:0/1700699357 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:48:21.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:21.223+0000 7f3f9effd640 1 -- 172.21.15.112:0/1700699357 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3fa0102550 msgr2=0x7f3fa0113520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:21.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:21.223+0000 7f3f9effd640 1 --2- 172.21.15.112:0/1700699357 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3fa0102550 0x7f3fa0113520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:21.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:21.223+0000 7f3f9effd640 1 -- 172.21.15.112:0/1700699357 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f94005040 con 0x7f3fa0102ef0 2024-09-16T00:48:21.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:21.223+0000 7f3f9effd640 1 --2- 172.21.15.112:0/1700699357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fa0102ef0 0x7f3fa0113a60 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f3f940043d0 tx=0x7f3f94009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:48:21.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:21.223+0000 7f3f87fff640 1 -- 172.21.15.112:0/1700699357 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f94038070 con 0x7f3fa0102ef0 2024-09-16T00:48:21.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:21.223+0000 7f3fa6116640 1 -- 172.21.15.112:0/1700699357 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fa010d740 con 0x7f3fa0102ef0 2024-09-16T00:48:21.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:21.223+0000 7f3f87fff640 1 -- 172.21.15.112:0/1700699357 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f3f940096e0 con 0x7f3fa0102ef0 2024-09-16T00:48:21.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:21.223+0000 7f3f87fff640 1 -- 172.21.15.112:0/1700699357 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f9402c8a0 con 0x7f3fa0102ef0 2024-09-16T00:48:21.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:21.223+0000 7f3fa6116640 1 -- 172.21.15.112:0/1700699357 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fa010dc80 con 0x7f3fa0102ef0 2024-09-16T00:48:21.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:21.224+0000 7f3fa6116640 1 -- 172.21.15.112:0/1700699357 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fa00fdcd0 con 0x7f3fa0102ef0 2024-09-16T00:48:21.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:21.225+0000 7f3f87fff640 1 -- 172.21.15.112:0/1700699357 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f3f94030020 con 0x7f3fa0102ef0 2024-09-16T00:48:21.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:21.225+0000 7f3f87fff640 1 --2- 172.21.15.112:0/1700699357 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f3f8806e530 0x7f3f880709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:21.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:21.225+0000 7f3f87fff640 1 -- 172.21.15.112:0/1700699357 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f3f940ae000 con 0x7f3fa0102ef0 2024-09-16T00:48:21.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:21.225+0000 7f3f9f7fe640 1 --2- 172.21.15.112:0/1700699357 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f3f8806e530 0x7f3f880709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:21.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:21.226+0000 7f3f9f7fe640 1 --2- 172.21.15.112:0/1700699357 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f3f8806e530 0x7f3f880709e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3f90004fd0 tx=0x7f3f90009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:48:21.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:21.230+0000 7f3f87fff640 1 -- 172.21.15.112:0/1700699357 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f3f94077020 con 0x7f3fa0102ef0 2024-09-16T00:48:21.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:48:20 smithi169 ceph-mon[77626]: from='client.35164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:48:21.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:48:20 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/2945489962' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:48:21.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:48:20 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/985310279' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:48:21.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:21.460+0000 7f3fa6116640 1 -- 172.21.15.112:0/1700699357 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f3fa010ec00 con 0x7f3fa0102ef0 2024-09-16T00:48:21.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:21.460+0000 7f3f87fff640 1 -- 172.21.15.112:0/1700699357 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f3fa010ec00 con 0x7f3fa0102ef0 2024-09-16T00:48:21.460 INFO:teuthology.orchestra.run.smithi112.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-16T00:48:21.460 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-16T00:48:21.461 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs is degraded 2024-09-16T00:48:21.461 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T00:48:21.461 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T00:48:21.461 INFO:teuthology.orchestra.run.smithi112.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-16T00:48:21.461 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-16T00:48:21.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:21.462+0000 7f3fa6116640 1 -- 172.21.15.112:0/1700699357 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f3f8806e530 msgr2=0x7f3f880709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:21.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:21.462+0000 7f3fa6116640 1 --2- 172.21.15.112:0/1700699357 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f3f8806e530 0x7f3f880709e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3f90004fd0 tx=0x7f3f90009040 comp rx=0 tx=0).stop 2024-09-16T00:48:21.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:21.462+0000 7f3fa6116640 1 -- 172.21.15.112:0/1700699357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fa0102ef0 msgr2=0x7f3fa0113a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:21.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:21.462+0000 7f3fa6116640 1 --2- 172.21.15.112:0/1700699357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fa0102ef0 0x7f3fa0113a60 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f3f940043d0 tx=0x7f3f94009da0 comp rx=0 tx=0).stop 2024-09-16T00:48:21.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:21.462+0000 7f3fa6116640 1 -- 172.21.15.112:0/1700699357 shutdown_connections 2024-09-16T00:48:21.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:21.462+0000 7f3fa6116640 1 --2- 172.21.15.112:0/1700699357 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f3f8806e530 0x7f3f880709e0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:21.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:21.462+0000 7f3fa6116640 1 --2- 172.21.15.112:0/1700699357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fa0102ef0 0x7f3fa0113a60 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:21.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:21.462+0000 7f3fa6116640 1 --2- 172.21.15.112:0/1700699357 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3fa0102550 0x7f3fa0113520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:21.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:21.462+0000 7f3fa6116640 1 -- 172.21.15.112:0/1700699357 >> 172.21.15.112:0/1700699357 conn(0x7f3fa00f80f0 msgr2=0x7f3fa00f9150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:48:21.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:21.462+0000 7f3fa6116640 1 -- 172.21.15.112:0/1700699357 shutdown_connections 2024-09-16T00:48:21.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:21.462+0000 7f3fa6116640 1 -- 172.21.15.112:0/1700699357 wait complete. 2024-09-16T00:48:22.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:48:21 smithi169 ceph-mon[77626]: from='client.35168 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:48:22.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:48:21 smithi169 ceph-mon[77626]: from='client.35172 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:48:22.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:48:21 smithi169 ceph-mon[77626]: pgmap v618: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:48:22.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:48:21 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/1700699357' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:48:22.314 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:48:21 smithi112 ceph-mon[97750]: from='client.35168 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:48:22.314 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:48:21 smithi112 ceph-mon[97750]: from='client.35172 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:48:22.314 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:48:21 smithi112 ceph-mon[97750]: pgmap v618: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:48:22.314 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:48:21 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/1700699357' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:48:23.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:48:22 smithi169 ceph-mon[77626]: from='client.35184 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:48:23.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:48:22 smithi112 ceph-mon[97750]: from='client.35184 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:48:24.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:48:23 smithi169 ceph-mon[77626]: pgmap v619: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:48:24.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:48:23 smithi112 ceph-mon[97750]: pgmap v619: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:48:26.270 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:48:25 smithi169 ceph-mon[77626]: pgmap v620: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:48:26.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:48:25 smithi112 ceph-mon[97750]: pgmap v620: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:48:28.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:48:27 smithi169 ceph-mon[77626]: pgmap v621: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:48:28.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:48:27 smithi112 ceph-mon[97750]: pgmap v621: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:48:30.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:48:29 smithi169 ceph-mon[77626]: pgmap v622: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:48:30.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:48:29 smithi112 ceph-mon[97750]: pgmap v622: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:48:32.278 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:48:31 smithi169 ceph-mon[77626]: pgmap v623: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:48:32.278 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:48:31 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:48:32.314 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:48:31 smithi112 ceph-mon[97750]: pgmap v623: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:48:32.315 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:48:31 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:48:34.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:48:33 smithi169 ceph-mon[77626]: pgmap v624: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:48:34.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:48:33 smithi112 ceph-mon[97750]: pgmap v624: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:48:36.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:48:35 smithi169 ceph-mon[77626]: pgmap v625: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:48:36.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:48:35 smithi112 ceph-mon[97750]: pgmap v625: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:48:37.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:48:36 smithi169 ceph-mon[77626]: pgmap v626: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:48:37.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:48:36 smithi112 ceph-mon[97750]: pgmap v626: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:48:40.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:48:39 smithi112 ceph-mon[97750]: pgmap v627: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:48:40.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:48:39 smithi169 ceph-mon[77626]: pgmap v627: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:48:42.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:48:41 smithi112 ceph-mon[97750]: pgmap v628: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:48:42.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:48:41 smithi169 ceph-mon[77626]: pgmap v628: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:48:44.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:48:43 smithi112 ceph-mon[97750]: pgmap v629: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:48:44.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:48:43 smithi169 ceph-mon[77626]: pgmap v629: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:48:46.088 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:48:45 smithi112 ceph-mon[97750]: pgmap v630: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:48:46.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:48:45 smithi169 ceph-mon[77626]: pgmap v630: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:48:47.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:48:46 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:48:47.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:48:46 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:48:48.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:48:47 smithi169 ceph-mon[77626]: pgmap v631: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:48:48.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:48:47 smithi112 ceph-mon[97750]: pgmap v631: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:48:50.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:48:49 smithi169 ceph-mon[77626]: pgmap v632: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:48:50.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:48:49 smithi112 ceph-mon[97750]: pgmap v632: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:48:51.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.553+0000 7f51d8c0f640 1 -- 172.21.15.112:0/2093627221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51d40fcf80 msgr2=0x7f51d40fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:51.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.553+0000 7f51d8c0f640 1 --2- 172.21.15.112:0/2093627221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51d40fcf80 0x7f51d40fd3d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f51c80048d0 tx=0x7f51c8009da0 comp rx=0 tx=0).stop 2024-09-16T00:48:51.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.553+0000 7f51d8c0f640 1 -- 172.21.15.112:0/2093627221 shutdown_connections 2024-09-16T00:48:51.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.553+0000 7f51d8c0f640 1 --2- 172.21.15.112:0/2093627221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51d40fcf80 0x7f51d40fd3d0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:51.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.553+0000 7f51d8c0f640 1 --2- 172.21.15.112:0/2093627221 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51d40fb810 0x7f51d40fbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:51.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.553+0000 7f51d8c0f640 1 -- 172.21.15.112:0/2093627221 >> 172.21.15.112:0/2093627221 conn(0x7f51d40723b0 msgr2=0x7f51d40727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:48:51.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.553+0000 7f51d8c0f640 1 -- 172.21.15.112:0/2093627221 shutdown_connections 2024-09-16T00:48:51.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.553+0000 7f51d8c0f640 1 -- 172.21.15.112:0/2093627221 wait complete. 2024-09-16T00:48:51.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.553+0000 7f51d8c0f640 1 Processor -- start 2024-09-16T00:48:51.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.554+0000 7f51d8c0f640 1 -- start start 2024-09-16T00:48:51.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.554+0000 7f51d8c0f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51d40fb810 0x7f51d410e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:51.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.554+0000 7f51d8c0f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51d40fcf80 0x7f51d410e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:51.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.554+0000 7f51d8c0f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d410ed00 con 0x7f51d40fb810 2024-09-16T00:48:51.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.554+0000 7f51d8c0f640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d410ee70 con 0x7f51d40fcf80 2024-09-16T00:48:51.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.554+0000 7f51d259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51d40fb810 0x7f51d410e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:51.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.554+0000 7f51d259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51d40fb810 0x7f51d410e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51152/0 (socket says 172.21.15.112:51152) 2024-09-16T00:48:51.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.554+0000 7f51d259c640 1 -- 172.21.15.112:0/1220497625 learned_addr learned my addr 172.21.15.112:0/1220497625 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:48:51.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.554+0000 7f51d1d9b640 1 --2- 172.21.15.112:0/1220497625 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51d40fcf80 0x7f51d410e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:51.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.554+0000 7f51d259c640 1 -- 172.21.15.112:0/1220497625 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51d40fcf80 msgr2=0x7f51d410e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:51.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.554+0000 7f51d259c640 1 --2- 172.21.15.112:0/1220497625 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51d40fcf80 0x7f51d410e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:51.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.554+0000 7f51d259c640 1 -- 172.21.15.112:0/1220497625 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c8005040 con 0x7f51d40fb810 2024-09-16T00:48:51.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.554+0000 7f51d259c640 1 --2- 172.21.15.112:0/1220497625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51d40fb810 0x7f51d410e170 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f51c400cae0 tx=0x7f51c4007e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:48:51.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.555+0000 7f51c2ffd640 1 -- 172.21.15.112:0/1220497625 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51c400ab00 con 0x7f51d40fb810 2024-09-16T00:48:51.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.555+0000 7f51c2ffd640 1 -- 172.21.15.112:0/1220497625 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f51c400ac60 con 0x7f51d40fb810 2024-09-16T00:48:51.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.555+0000 7f51d8c0f640 1 -- 172.21.15.112:0/1220497625 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51d41171e0 con 0x7f51d40fb810 2024-09-16T00:48:51.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.555+0000 7f51c2ffd640 1 -- 172.21.15.112:0/1220497625 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51c40115e0 con 0x7f51d40fb810 2024-09-16T00:48:51.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.555+0000 7f51d8c0f640 1 -- 172.21.15.112:0/1220497625 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51d41176f0 con 0x7f51d40fb810 2024-09-16T00:48:51.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.556+0000 7f51d8c0f640 1 -- 172.21.15.112:0/1220497625 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51d40fcbc0 con 0x7f51d40fb810 2024-09-16T00:48:51.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.556+0000 7f51c2ffd640 1 -- 172.21.15.112:0/1220497625 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f51c40129e0 con 0x7f51d40fb810 2024-09-16T00:48:51.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.557+0000 7f51c2ffd640 1 --2- 172.21.15.112:0/1220497625 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f51bc06e530 0x7f51bc0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:51.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.557+0000 7f51c2ffd640 1 -- 172.21.15.112:0/1220497625 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f51c408eb80 con 0x7f51d40fb810 2024-09-16T00:48:51.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.557+0000 7f51d1d9b640 1 --2- 172.21.15.112:0/1220497625 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f51bc06e530 0x7f51bc0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:51.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.557+0000 7f51d1d9b640 1 --2- 172.21.15.112:0/1220497625 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f51bc06e530 0x7f51bc0709e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f51d4065980 tx=0x7f51c8035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:48:51.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.561+0000 7f51c2ffd640 1 -- 172.21.15.112:0/1220497625 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f51c4057af0 con 0x7f51d40fb810 2024-09-16T00:48:51.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.748+0000 7f51d8c0f640 1 -- 172.21.15.112:0/1220497625 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f51d4065730 con 0x7f51bc06e530 2024-09-16T00:48:51.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.750+0000 7f51c2ffd640 1 -- 172.21.15.112:0/1220497625 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f51d4065730 con 0x7f51bc06e530 2024-09-16T00:48:51.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.752+0000 7f51d8c0f640 1 -- 172.21.15.112:0/1220497625 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f51bc06e530 msgr2=0x7f51bc0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:51.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.752+0000 7f51d8c0f640 1 --2- 172.21.15.112:0/1220497625 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f51bc06e530 0x7f51bc0709e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f51d4065980 tx=0x7f51c8035040 comp rx=0 tx=0).stop 2024-09-16T00:48:51.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.752+0000 7f51d8c0f640 1 -- 172.21.15.112:0/1220497625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51d40fb810 msgr2=0x7f51d410e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:51.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.752+0000 7f51d8c0f640 1 --2- 172.21.15.112:0/1220497625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51d40fb810 0x7f51d410e170 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f51c400cae0 tx=0x7f51c4007e50 comp rx=0 tx=0).stop 2024-09-16T00:48:51.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.752+0000 7f51d8c0f640 1 -- 172.21.15.112:0/1220497625 shutdown_connections 2024-09-16T00:48:51.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.752+0000 7f51d8c0f640 1 --2- 172.21.15.112:0/1220497625 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f51bc06e530 0x7f51bc0709e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:51.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.752+0000 7f51d8c0f640 1 --2- 172.21.15.112:0/1220497625 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51d40fcf80 0x7f51d410e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:51.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.752+0000 7f51d8c0f640 1 --2- 172.21.15.112:0/1220497625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51d40fb810 0x7f51d410e170 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:51.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.752+0000 7f51d8c0f640 1 -- 172.21.15.112:0/1220497625 >> 172.21.15.112:0/1220497625 conn(0x7f51d40723b0 msgr2=0x7f51d40fa480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:48:51.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.752+0000 7f51d8c0f640 1 -- 172.21.15.112:0/1220497625 shutdown_connections 2024-09-16T00:48:51.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.752+0000 7f51d8c0f640 1 -- 172.21.15.112:0/1220497625 wait complete. 2024-09-16T00:48:51.761 INFO:teuthology.orchestra.run.smithi112.stdout:true 2024-09-16T00:48:51.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.831+0000 7f7d175dd640 1 -- 172.21.15.112:0/3184246037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d10101480 msgr2=0x7f7d10101850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:51.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.831+0000 7f7d175dd640 1 --2- 172.21.15.112:0/3184246037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d10101480 0x7f7d10101850 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f7d04009c50 tx=0x7f7d04004d10 comp rx=0 tx=0).stop 2024-09-16T00:48:51.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.831+0000 7f7d175dd640 1 -- 172.21.15.112:0/3184246037 shutdown_connections 2024-09-16T00:48:51.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.831+0000 7f7d175dd640 1 --2- 172.21.15.112:0/3184246037 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d10101e20 0x7f7d100fbd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:51.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.831+0000 7f7d175dd640 1 --2- 172.21.15.112:0/3184246037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d10101480 0x7f7d10101850 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:51.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.831+0000 7f7d175dd640 1 -- 172.21.15.112:0/3184246037 >> 172.21.15.112:0/3184246037 conn(0x7f7d10070c80 msgr2=0x7f7d10071080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:48:51.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.831+0000 7f7d175dd640 1 -- 172.21.15.112:0/3184246037 shutdown_connections 2024-09-16T00:48:51.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.831+0000 7f7d175dd640 1 -- 172.21.15.112:0/3184246037 wait complete. 2024-09-16T00:48:51.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.832+0000 7f7d175dd640 1 Processor -- start 2024-09-16T00:48:51.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.832+0000 7f7d175dd640 1 -- start start 2024-09-16T00:48:51.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.832+0000 7f7d175dd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d10101480 0x7f7d101022e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:51.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.832+0000 7f7d175dd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d10101e20 0x7f7d10102820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:51.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.832+0000 7f7d175dd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d10102df0 con 0x7f7d10101e20 2024-09-16T00:48:51.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.832+0000 7f7d175dd640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d10102f60 con 0x7f7d10101480 2024-09-16T00:48:51.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.832+0000 7f7d14b78640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d10101e20 0x7f7d10102820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:51.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.832+0000 7f7d14b78640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d10101e20 0x7f7d10102820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51174/0 (socket says 172.21.15.112:51174) 2024-09-16T00:48:51.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.832+0000 7f7d14b78640 1 -- 172.21.15.112:0/337516108 learned_addr learned my addr 172.21.15.112:0/337516108 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:48:51.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.832+0000 7f7d15379640 1 --2- 172.21.15.112:0/337516108 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d10101480 0x7f7d101022e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:51.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.832+0000 7f7d14b78640 1 -- 172.21.15.112:0/337516108 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d10101480 msgr2=0x7f7d101022e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:51.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.832+0000 7f7d14b78640 1 --2- 172.21.15.112:0/337516108 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d10101480 0x7f7d101022e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:51.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.832+0000 7f7d14b78640 1 -- 172.21.15.112:0/337516108 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d040098b0 con 0x7f7d10101e20 2024-09-16T00:48:51.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.833+0000 7f7d14b78640 1 --2- 172.21.15.112:0/337516108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d10101e20 0x7f7d10102820 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c00b540 tx=0x7f7d0c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:48:51.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.833+0000 7f7d01ffb640 1 -- 172.21.15.112:0/337516108 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d0c00f6f0 con 0x7f7d10101e20 2024-09-16T00:48:51.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.833+0000 7f7d01ffb640 1 -- 172.21.15.112:0/337516108 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f7d0c00fcf0 con 0x7f7d10101e20 2024-09-16T00:48:51.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.833+0000 7f7d175dd640 1 -- 172.21.15.112:0/337516108 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d10119640 con 0x7f7d10101e20 2024-09-16T00:48:51.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.833+0000 7f7d01ffb640 1 -- 172.21.15.112:0/337516108 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d0c0103a0 con 0x7f7d10101e20 2024-09-16T00:48:51.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.833+0000 7f7d175dd640 1 -- 172.21.15.112:0/337516108 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d10119a80 con 0x7f7d10101e20 2024-09-16T00:48:51.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.834+0000 7f7d01ffb640 1 -- 172.21.15.112:0/337516108 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f7d0c00f850 con 0x7f7d10101e20 2024-09-16T00:48:51.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.834+0000 7f7d175dd640 1 -- 172.21.15.112:0/337516108 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d100fcd80 con 0x7f7d10101e20 2024-09-16T00:48:51.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.835+0000 7f7d01ffb640 1 --2- 172.21.15.112:0/337516108 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f7cfc06e530 0x7f7cfc0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:51.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.835+0000 7f7d01ffb640 1 -- 172.21.15.112:0/337516108 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f7d0c08e260 con 0x7f7d10101e20 2024-09-16T00:48:51.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.835+0000 7f7d15379640 1 --2- 172.21.15.112:0/337516108 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f7cfc06e530 0x7f7cfc0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:51.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.835+0000 7f7d15379640 1 --2- 172.21.15.112:0/337516108 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f7cfc06e530 0x7f7cfc0709e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7d04004a30 tx=0x7f7d04004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:48:51.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:51.839+0000 7f7d01ffb640 1 -- 172.21.15.112:0/337516108 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f7d0c057230 con 0x7f7d10101e20 2024-09-16T00:48:52.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.022+0000 7f7d175dd640 1 -- 172.21.15.112:0/337516108 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7d1004b720 con 0x7f7cfc06e530 2024-09-16T00:48:52.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.023+0000 7f7d01ffb640 1 -- 172.21.15.112:0/337516108 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f7d1004b720 con 0x7f7cfc06e530 2024-09-16T00:48:52.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.026+0000 7f7d175dd640 1 -- 172.21.15.112:0/337516108 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f7cfc06e530 msgr2=0x7f7cfc0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:52.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.026+0000 7f7d175dd640 1 --2- 172.21.15.112:0/337516108 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f7cfc06e530 0x7f7cfc0709e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7d04004a30 tx=0x7f7d04004d10 comp rx=0 tx=0).stop 2024-09-16T00:48:52.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.026+0000 7f7d175dd640 1 -- 172.21.15.112:0/337516108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d10101e20 msgr2=0x7f7d10102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:52.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.026+0000 7f7d175dd640 1 --2- 172.21.15.112:0/337516108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d10101e20 0x7f7d10102820 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c00b540 tx=0x7f7d0c0091a0 comp rx=0 tx=0).stop 2024-09-16T00:48:52.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.026+0000 7f7d175dd640 1 -- 172.21.15.112:0/337516108 shutdown_connections 2024-09-16T00:48:52.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.026+0000 7f7d175dd640 1 --2- 172.21.15.112:0/337516108 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f7cfc06e530 0x7f7cfc0709e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:52.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.026+0000 7f7d175dd640 1 --2- 172.21.15.112:0/337516108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d10101e20 0x7f7d10102820 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:52.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.026+0000 7f7d175dd640 1 --2- 172.21.15.112:0/337516108 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d10101480 0x7f7d101022e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:52.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.026+0000 7f7d175dd640 1 -- 172.21.15.112:0/337516108 >> 172.21.15.112:0/337516108 conn(0x7f7d10070c80 msgr2=0x7f7d10072040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:48:52.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.026+0000 7f7d175dd640 1 -- 172.21.15.112:0/337516108 shutdown_connections 2024-09-16T00:48:52.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.026+0000 7f7d175dd640 1 -- 172.21.15.112:0/337516108 wait complete. 2024-09-16T00:48:52.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:48:51 smithi112 ceph-mon[97750]: pgmap v633: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:48:52.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.104+0000 7efd1ff73640 1 -- 172.21.15.112:0/1424261677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd18101e40 msgr2=0x7efd180fbe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:52.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.104+0000 7efd1ff73640 1 --2- 172.21.15.112:0/1424261677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd18101e40 0x7efd180fbe00 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7efd080048d0 tx=0x7efd08009da0 comp rx=0 tx=0).stop 2024-09-16T00:48:52.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.104+0000 7efd1ff73640 1 -- 172.21.15.112:0/1424261677 shutdown_connections 2024-09-16T00:48:52.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.104+0000 7efd1ff73640 1 --2- 172.21.15.112:0/1424261677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd18101e40 0x7efd180fbe00 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:52.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.104+0000 7efd1ff73640 1 --2- 172.21.15.112:0/1424261677 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd18101530 0x7efd18101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:52.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.104+0000 7efd1ff73640 1 -- 172.21.15.112:0/1424261677 >> 172.21.15.112:0/1424261677 conn(0x7efd18070d50 msgr2=0x7efd18071150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:48:52.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.104+0000 7efd1ff73640 1 -- 172.21.15.112:0/1424261677 shutdown_connections 2024-09-16T00:48:52.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.104+0000 7efd1ff73640 1 -- 172.21.15.112:0/1424261677 wait complete. 2024-09-16T00:48:52.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.105+0000 7efd1ff73640 1 Processor -- start 2024-09-16T00:48:52.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.105+0000 7efd1ff73640 1 -- start start 2024-09-16T00:48:52.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.105+0000 7efd1ff73640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd18101530 0x7efd18112110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:52.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.105+0000 7efd1ff73640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd18101e40 0x7efd18112650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:52.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.105+0000 7efd1ff73640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd18112c50 con 0x7efd18101530 2024-09-16T00:48:52.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.105+0000 7efd1ff73640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd1810c090 con 0x7efd18101e40 2024-09-16T00:48:52.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.105+0000 7efd1dd0f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd18101530 0x7efd18112110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:52.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.105+0000 7efd1dd0f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd18101530 0x7efd18112110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51186/0 (socket says 172.21.15.112:51186) 2024-09-16T00:48:52.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.105+0000 7efd1dd0f640 1 -- 172.21.15.112:0/2255686953 learned_addr learned my addr 172.21.15.112:0/2255686953 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:48:52.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.105+0000 7efd1dd0f640 1 -- 172.21.15.112:0/2255686953 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd18101e40 msgr2=0x7efd18112650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:52.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.105+0000 7efd1dd0f640 1 --2- 172.21.15.112:0/2255686953 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd18101e40 0x7efd18112650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:52.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.105+0000 7efd1dd0f640 1 -- 172.21.15.112:0/2255686953 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd08005040 con 0x7efd18101530 2024-09-16T00:48:52.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.105+0000 7efd1dd0f640 1 --2- 172.21.15.112:0/2255686953 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd18101530 0x7efd18112110 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7efd1400c810 tx=0x7efd140043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:48:52.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.106+0000 7efd0e7fc640 1 -- 172.21.15.112:0/2255686953 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd1400a820 con 0x7efd18101530 2024-09-16T00:48:52.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.106+0000 7efd0e7fc640 1 -- 172.21.15.112:0/2255686953 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7efd1400ae20 con 0x7efd18101530 2024-09-16T00:48:52.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.106+0000 7efd1ff73640 1 -- 172.21.15.112:0/2255686953 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd1810c370 con 0x7efd18101530 2024-09-16T00:48:52.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.106+0000 7efd0e7fc640 1 -- 172.21.15.112:0/2255686953 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd140115c0 con 0x7efd18101530 2024-09-16T00:48:52.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.106+0000 7efd1ff73640 1 -- 172.21.15.112:0/2255686953 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd1810c890 con 0x7efd18101530 2024-09-16T00:48:52.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.107+0000 7efd1ff73640 1 -- 172.21.15.112:0/2255686953 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd180fce10 con 0x7efd18101530 2024-09-16T00:48:52.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.107+0000 7efd0e7fc640 1 -- 172.21.15.112:0/2255686953 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7efd14012840 con 0x7efd18101530 2024-09-16T00:48:52.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.108+0000 7efd0e7fc640 1 --2- 172.21.15.112:0/2255686953 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7efd0406e530 0x7efd040709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:52.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.108+0000 7efd0e7fc640 1 -- 172.21.15.112:0/2255686953 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7efd1408e8c0 con 0x7efd18101530 2024-09-16T00:48:52.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.108+0000 7efd1d50e640 1 --2- 172.21.15.112:0/2255686953 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7efd0406e530 0x7efd040709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:52.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.108+0000 7efd1d50e640 1 --2- 172.21.15.112:0/2255686953 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7efd0406e530 0x7efd040709e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7efd08009ac0 tx=0x7efd08009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:48:52.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.112+0000 7efd0e7fc640 1 -- 172.21.15.112:0/2255686953 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7efd14057830 con 0x7efd18101530 2024-09-16T00:48:52.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:48:51 smithi169 ceph-mon[77626]: pgmap v633: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:48:52.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.285+0000 7efd1ff73640 1 -- 172.21.15.112:0/2255686953 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7efd1804b720 con 0x7efd0406e530 2024-09-16T00:48:52.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.294+0000 7efd0e7fc640 1 -- 172.21.15.112:0/2255686953 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7efd1804b720 con 0x7efd0406e530 2024-09-16T00:48:52.294 INFO:teuthology.orchestra.run.smithi112.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T00:48:52.295 INFO:teuthology.orchestra.run.smithi112.stdout:alertmanager.smithi112 smithi112 *:9093,9094 running (21m) 14m ago 30m 24.7M - 0.25.0 c8568f914cd2 d85ea0562022 2024-09-16T00:48:52.295 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi112 smithi112 running (18m) 14m ago 30m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c e4fe58f9702a 2024-09-16T00:48:52.295 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi169 smithi169 running (18m) 14m ago 30m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c 3ba2c81d527f 2024-09-16T00:48:52.295 INFO:teuthology.orchestra.run.smithi112.stdout:grafana.smithi112 smithi112 *:3000 running (20m) 14m ago 30m 82.1M - 9.4.12 2bacad6d85d8 51f9d2013da2 2024-09-16T00:48:52.295 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.hxfwkh smithi112 running (14m) 14m ago 25m 14.2M - 19.1.1-326-g8921c8ef f1455bd0219c 1723069902d6 2024-09-16T00:48:52.295 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.ithwzj smithi112 running (15m) 14m ago 25m 12.2M - 19.1.1-326-g8921c8ef f1455bd0219c e7aad5b81db5 2024-09-16T00:48:52.295 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.gawwgw smithi169 running (14m) 14m ago 25m 15.7M - 19.1.1-326-g8921c8ef f1455bd0219c 469be459ee12 2024-09-16T00:48:52.295 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.lgeqrj smithi169 running (14m) 14m ago 25m 13.3M - 19.1.1-326-g8921c8ef f1455bd0219c 31f0fbd2395d 2024-09-16T00:48:52.295 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi112.xrkkbc smithi112 *:8443,9283,8765 running (21m) 14m ago 32m 504M - 19.1.1-326-g8921c8ef f1455bd0219c a981f7fd2810 2024-09-16T00:48:52.295 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi169.mtrobk smithi169 *:8443,9283,8765 running (20m) 14m ago 29m 442M - 19.1.1-326-g8921c8ef f1455bd0219c 4096e5369c2d 2024-09-16T00:48:52.295 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi112 smithi112 running (19m) 14m ago 32m 49.8M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 5aaec033c042 2024-09-16T00:48:52.295 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi169 smithi169 running (19m) 14m ago 29m 38.9M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 7bc609155f3f 2024-09-16T00:48:52.296 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi112 smithi112 *:9100 running (22m) 14m ago 30m 15.5M - 1.5.0 0da6a335fe13 6404d1d601e2 2024-09-16T00:48:52.296 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi169 smithi169 *:9100 running (21m) 14m ago 29m 15.4M - 1.5.0 0da6a335fe13 5505baf702a6 2024-09-16T00:48:52.296 INFO:teuthology.orchestra.run.smithi112.stdout:osd.0 smithi112 running (18m) 14m ago 29m 105M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 53efc9397017 2024-09-16T00:48:52.296 INFO:teuthology.orchestra.run.smithi112.stdout:osd.1 smithi112 running (17m) 14m ago 28m 98.9M 1557M 19.1.1-326-g8921c8ef f1455bd0219c fe6f1a4a2cf4 2024-09-16T00:48:52.296 INFO:teuthology.orchestra.run.smithi112.stdout:osd.2 smithi112 running (17m) 14m ago 28m 102M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 5f7863712057 2024-09-16T00:48:52.296 INFO:teuthology.orchestra.run.smithi112.stdout:osd.3 smithi169 running (16m) 14m ago 27m 92.1M 2581M 19.1.1-326-g8921c8ef f1455bd0219c ee07b5577810 2024-09-16T00:48:52.296 INFO:teuthology.orchestra.run.smithi112.stdout:osd.4 smithi169 running (16m) 14m ago 27m 103M 2581M 19.1.1-326-g8921c8ef f1455bd0219c 769f026a54eb 2024-09-16T00:48:52.296 INFO:teuthology.orchestra.run.smithi112.stdout:osd.5 smithi169 running (15m) 14m ago 26m 125M 2581M 19.1.1-326-g8921c8ef f1455bd0219c a6007c13cf78 2024-09-16T00:48:52.296 INFO:teuthology.orchestra.run.smithi112.stdout:prometheus.smithi112 smithi112 *:9095 running (21m) 14m ago 30m 64.1M - 2.43.0 a07b618ecd1d 7a1d42e60d2b 2024-09-16T00:48:52.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.296+0000 7efd1ff73640 1 -- 172.21.15.112:0/2255686953 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7efd0406e530 msgr2=0x7efd040709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:52.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.296+0000 7efd1ff73640 1 --2- 172.21.15.112:0/2255686953 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7efd0406e530 0x7efd040709e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7efd08009ac0 tx=0x7efd08009da0 comp rx=0 tx=0).stop 2024-09-16T00:48:52.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.296+0000 7efd1ff73640 1 -- 172.21.15.112:0/2255686953 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd18101530 msgr2=0x7efd18112110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:52.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.296+0000 7efd1ff73640 1 --2- 172.21.15.112:0/2255686953 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd18101530 0x7efd18112110 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7efd1400c810 tx=0x7efd140043c0 comp rx=0 tx=0).stop 2024-09-16T00:48:52.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.296+0000 7efd1ff73640 1 -- 172.21.15.112:0/2255686953 shutdown_connections 2024-09-16T00:48:52.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.296+0000 7efd1ff73640 1 --2- 172.21.15.112:0/2255686953 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7efd0406e530 0x7efd040709e0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:52.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.296+0000 7efd1ff73640 1 --2- 172.21.15.112:0/2255686953 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd18101e40 0x7efd18112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:52.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.296+0000 7efd1ff73640 1 --2- 172.21.15.112:0/2255686953 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd18101530 0x7efd18112110 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:52.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.296+0000 7efd1ff73640 1 -- 172.21.15.112:0/2255686953 >> 172.21.15.112:0/2255686953 conn(0x7efd18070d50 msgr2=0x7efd180f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:48:52.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.296+0000 7efd1ff73640 1 -- 172.21.15.112:0/2255686953 shutdown_connections 2024-09-16T00:48:52.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.296+0000 7efd1ff73640 1 -- 172.21.15.112:0/2255686953 wait complete. 2024-09-16T00:48:52.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.374+0000 7f73db410640 1 -- 172.21.15.112:0/229747636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73d40fb4a0 msgr2=0x7f73d40fb910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:52.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.374+0000 7f73db410640 1 --2- 172.21.15.112:0/229747636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73d40fb4a0 0x7f73d40fb910 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f73c80048d0 tx=0x7f73c8009da0 comp rx=0 tx=0).stop 2024-09-16T00:48:52.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.374+0000 7f73db410640 1 -- 172.21.15.112:0/229747636 shutdown_connections 2024-09-16T00:48:52.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.374+0000 7f73db410640 1 --2- 172.21.15.112:0/229747636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73d40fb4a0 0x7f73d40fb910 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:52.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.374+0000 7f73db410640 1 --2- 172.21.15.112:0/229747636 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73d41014e0 0x7f73d41018b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:52.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.374+0000 7f73db410640 1 -- 172.21.15.112:0/229747636 >> 172.21.15.112:0/229747636 conn(0x7f73d40f7060 msgr2=0x7f73d40f9470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:48:52.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.374+0000 7f73db410640 1 -- 172.21.15.112:0/229747636 shutdown_connections 2024-09-16T00:48:52.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.374+0000 7f73db410640 1 -- 172.21.15.112:0/229747636 wait complete. 2024-09-16T00:48:52.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.375+0000 7f73db410640 1 Processor -- start 2024-09-16T00:48:52.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.375+0000 7f73db410640 1 -- start start 2024-09-16T00:48:52.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.375+0000 7f73db410640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73d40fb4a0 0x7f73d41131a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:52.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.375+0000 7f73db410640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73d41014e0 0x7f73d41136e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:52.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.375+0000 7f73db410640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d4113ce0 con 0x7f73d40fb4a0 2024-09-16T00:48:52.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.375+0000 7f73db410640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d4113e50 con 0x7f73d41014e0 2024-09-16T00:48:52.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.375+0000 7f73d91ac640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73d40fb4a0 0x7f73d41131a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:52.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.375+0000 7f73d91ac640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73d40fb4a0 0x7f73d41131a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50782/0 (socket says 172.21.15.112:50782) 2024-09-16T00:48:52.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.375+0000 7f73d91ac640 1 -- 172.21.15.112:0/1504222167 learned_addr learned my addr 172.21.15.112:0/1504222167 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:48:52.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.375+0000 7f73d89ab640 1 --2- 172.21.15.112:0/1504222167 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73d41014e0 0x7f73d41136e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:52.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.375+0000 7f73d91ac640 1 -- 172.21.15.112:0/1504222167 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73d41014e0 msgr2=0x7f73d41136e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:52.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.375+0000 7f73d91ac640 1 --2- 172.21.15.112:0/1504222167 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73d41014e0 0x7f73d41136e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:52.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.375+0000 7f73d91ac640 1 -- 172.21.15.112:0/1504222167 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73c8005040 con 0x7f73d40fb4a0 2024-09-16T00:48:52.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.376+0000 7f73d89ab640 1 --2- 172.21.15.112:0/1504222167 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73d41014e0 0x7f73d41136e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:48:52.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.376+0000 7f73d91ac640 1 --2- 172.21.15.112:0/1504222167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73d40fb4a0 0x7f73d41131a0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f73cc00db80 tx=0x7f73cc004f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:48:52.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.376+0000 7f73d1ffb640 1 -- 172.21.15.112:0/1504222167 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f73cc00b950 con 0x7f73d40fb4a0 2024-09-16T00:48:52.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.376+0000 7f73d1ffb640 1 -- 172.21.15.112:0/1504222167 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f73cc00f040 con 0x7f73d40fb4a0 2024-09-16T00:48:52.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.376+0000 7f73db410640 1 -- 172.21.15.112:0/1504222167 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73d410d2d0 con 0x7f73d40fb4a0 2024-09-16T00:48:52.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.376+0000 7f73d1ffb640 1 -- 172.21.15.112:0/1504222167 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f73cc012390 con 0x7f73d40fb4a0 2024-09-16T00:48:52.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.376+0000 7f73db410640 1 -- 172.21.15.112:0/1504222167 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73d410d7f0 con 0x7f73d40fb4a0 2024-09-16T00:48:52.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.377+0000 7f73db410640 1 -- 172.21.15.112:0/1504222167 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73d40fcbe0 con 0x7f73d40fb4a0 2024-09-16T00:48:52.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.378+0000 7f73d1ffb640 1 -- 172.21.15.112:0/1504222167 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f73cc0136e0 con 0x7f73d40fb4a0 2024-09-16T00:48:52.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.378+0000 7f73d1ffb640 1 --2- 172.21.15.112:0/1504222167 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f73bc06e530 0x7f73bc0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:52.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.378+0000 7f73d1ffb640 1 -- 172.21.15.112:0/1504222167 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f73cc08e2c0 con 0x7f73d40fb4a0 2024-09-16T00:48:52.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.378+0000 7f73d89ab640 1 --2- 172.21.15.112:0/1504222167 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f73bc06e530 0x7f73bc0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:52.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.379+0000 7f73d89ab640 1 --2- 172.21.15.112:0/1504222167 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f73bc06e530 0x7f73bc0709e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f73c800b020 tx=0x7f73c8009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:48:52.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.383+0000 7f73d1ffb640 1 -- 172.21.15.112:0/1504222167 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f73cc05ad40 con 0x7f73d40fb4a0 2024-09-16T00:48:52.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.616+0000 7f73db410640 1 -- 172.21.15.112:0/1504222167 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f73d410e850 con 0x7f73d40fb4a0 2024-09-16T00:48:52.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.617+0000 7f73d1ffb640 1 -- 172.21.15.112:0/1504222167 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f73cc05a8d0 con 0x7f73d40fb4a0 2024-09-16T00:48:52.617 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:48:52.617 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": { 2024-09-16T00:48:52.618 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:48:52.618 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:48:52.618 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": { 2024-09-16T00:48:52.618 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:48:52.618 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:48:52.618 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": { 2024-09-16T00:48:52.618 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 6 2024-09-16T00:48:52.618 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:48:52.618 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": { 2024-09-16T00:48:52.618 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 4 2024-09-16T00:48:52.618 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:48:52.618 INFO:teuthology.orchestra.run.smithi112.stdout: "overall": { 2024-09-16T00:48:52.618 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 14 2024-09-16T00:48:52.618 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-16T00:48:52.619 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:48:52.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.619+0000 7f73db410640 1 -- 172.21.15.112:0/1504222167 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f73bc06e530 msgr2=0x7f73bc0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:52.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.619+0000 7f73db410640 1 --2- 172.21.15.112:0/1504222167 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f73bc06e530 0x7f73bc0709e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f73c800b020 tx=0x7f73c8009da0 comp rx=0 tx=0).stop 2024-09-16T00:48:52.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.619+0000 7f73db410640 1 -- 172.21.15.112:0/1504222167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73d40fb4a0 msgr2=0x7f73d41131a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:52.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.619+0000 7f73db410640 1 --2- 172.21.15.112:0/1504222167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73d40fb4a0 0x7f73d41131a0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f73cc00db80 tx=0x7f73cc004f00 comp rx=0 tx=0).stop 2024-09-16T00:48:52.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.619+0000 7f73db410640 1 -- 172.21.15.112:0/1504222167 shutdown_connections 2024-09-16T00:48:52.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.619+0000 7f73db410640 1 --2- 172.21.15.112:0/1504222167 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f73bc06e530 0x7f73bc0709e0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:52.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.619+0000 7f73db410640 1 --2- 172.21.15.112:0/1504222167 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73d41014e0 0x7f73d41136e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:52.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.619+0000 7f73db410640 1 --2- 172.21.15.112:0/1504222167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73d40fb4a0 0x7f73d41131a0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:52.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.619+0000 7f73db410640 1 -- 172.21.15.112:0/1504222167 >> 172.21.15.112:0/1504222167 conn(0x7f73d40f7060 msgr2=0x7f73d4065a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:48:52.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.619+0000 7f73db410640 1 -- 172.21.15.112:0/1504222167 shutdown_connections 2024-09-16T00:48:52.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.619+0000 7f73db410640 1 -- 172.21.15.112:0/1504222167 wait complete. 2024-09-16T00:48:52.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.697+0000 7f1faf537640 1 -- 172.21.15.112:0/987477356 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fa81014e0 msgr2=0x7f1fa81018b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:52.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.697+0000 7f1faf537640 1 --2- 172.21.15.112:0/987477356 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fa81014e0 0x7f1fa81018b0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f1fa40048d0 tx=0x7f1fa4009da0 comp rx=0 tx=0).stop 2024-09-16T00:48:52.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.697+0000 7f1faf537640 1 -- 172.21.15.112:0/987477356 shutdown_connections 2024-09-16T00:48:52.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.697+0000 7f1faf537640 1 --2- 172.21.15.112:0/987477356 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1fa80fb4a0 0x7f1fa80fb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:52.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.697+0000 7f1faf537640 1 --2- 172.21.15.112:0/987477356 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fa81014e0 0x7f1fa81018b0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:52.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.697+0000 7f1faf537640 1 -- 172.21.15.112:0/987477356 >> 172.21.15.112:0/987477356 conn(0x7f1fa80f7060 msgr2=0x7f1fa80f9470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:48:52.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.698+0000 7f1faf537640 1 -- 172.21.15.112:0/987477356 shutdown_connections 2024-09-16T00:48:52.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.698+0000 7f1faf537640 1 -- 172.21.15.112:0/987477356 wait complete. 2024-09-16T00:48:52.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.698+0000 7f1faf537640 1 Processor -- start 2024-09-16T00:48:52.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.698+0000 7f1faf537640 1 -- start start 2024-09-16T00:48:52.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.698+0000 7f1faf537640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1fa80fb4a0 0x7f1fa81131b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:52.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.698+0000 7f1faf537640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fa81014e0 0x7f1fa81136f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:52.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.698+0000 7f1faf537640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa8113cf0 con 0x7f1fa81014e0 2024-09-16T00:48:52.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.698+0000 7f1faf537640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa810d130 con 0x7f1fa80fb4a0 2024-09-16T00:48:52.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.699+0000 7f1facad2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fa81014e0 0x7f1fa81136f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:52.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.699+0000 7f1facad2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fa81014e0 0x7f1fa81136f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50814/0 (socket says 172.21.15.112:50814) 2024-09-16T00:48:52.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.699+0000 7f1facad2640 1 -- 172.21.15.112:0/873166197 learned_addr learned my addr 172.21.15.112:0/873166197 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:48:52.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.699+0000 7f1fad2d3640 1 --2- 172.21.15.112:0/873166197 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1fa80fb4a0 0x7f1fa81131b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:52.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.699+0000 7f1facad2640 1 -- 172.21.15.112:0/873166197 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1fa80fb4a0 msgr2=0x7f1fa81131b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:52.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.699+0000 7f1facad2640 1 --2- 172.21.15.112:0/873166197 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1fa80fb4a0 0x7f1fa81131b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:52.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.699+0000 7f1facad2640 1 -- 172.21.15.112:0/873166197 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fa4005040 con 0x7f1fa81014e0 2024-09-16T00:48:52.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.699+0000 7f1fad2d3640 1 --2- 172.21.15.112:0/873166197 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1fa80fb4a0 0x7f1fa81131b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:48:52.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.699+0000 7f1facad2640 1 --2- 172.21.15.112:0/873166197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fa81014e0 0x7f1fa81136f0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f1f9800c790 tx=0x7f1f980043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:48:52.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.699+0000 7f1f9dffb640 1 -- 172.21.15.112:0/873166197 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f9800a820 con 0x7f1fa81014e0 2024-09-16T00:48:52.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.699+0000 7f1f9dffb640 1 -- 172.21.15.112:0/873166197 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f1f9800ae20 con 0x7f1fa81014e0 2024-09-16T00:48:52.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.699+0000 7f1faf537640 1 -- 172.21.15.112:0/873166197 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fa810d410 con 0x7f1fa81014e0 2024-09-16T00:48:52.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.699+0000 7f1f9dffb640 1 -- 172.21.15.112:0/873166197 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f980113a0 con 0x7f1fa81014e0 2024-09-16T00:48:52.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.700+0000 7f1faf537640 1 -- 172.21.15.112:0/873166197 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fa810d9b0 con 0x7f1fa81014e0 2024-09-16T00:48:52.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.701+0000 7f1faf537640 1 -- 172.21.15.112:0/873166197 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fa80fcbe0 con 0x7f1fa81014e0 2024-09-16T00:48:52.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.701+0000 7f1f9dffb640 1 -- 172.21.15.112:0/873166197 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f1f98012400 con 0x7f1fa81014e0 2024-09-16T00:48:52.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.701+0000 7f1f9dffb640 1 --2- 172.21.15.112:0/873166197 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f1f9406e530 0x7f1f940709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:52.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.701+0000 7f1f9dffb640 1 -- 172.21.15.112:0/873166197 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f1f9808e3e0 con 0x7f1fa81014e0 2024-09-16T00:48:52.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.702+0000 7f1fad2d3640 1 --2- 172.21.15.112:0/873166197 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f1f9406e530 0x7f1f940709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:52.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.702+0000 7f1fad2d3640 1 --2- 172.21.15.112:0/873166197 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f1f9406e530 0x7f1f940709e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4007ac0 tx=0x7f1fa4035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:48:52.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.706+0000 7f1f9dffb640 1 -- 172.21.15.112:0/873166197 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f1f98057470 con 0x7f1fa81014e0 2024-09-16T00:48:52.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.908+0000 7f1faf537640 1 -- 172.21.15.112:0/873166197 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f1fa8108480 con 0x7f1fa81014e0 2024-09-16T00:48:52.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.908+0000 7f1f9dffb640 1 -- 172.21.15.112:0/873166197 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1914 (secure 0 0 0) 0x7f1f9805ab30 con 0x7f1fa81014e0 2024-09-16T00:48:52.909 INFO:teuthology.orchestra.run.smithi112.stdout:e34 2024-09-16T00:48:52.909 INFO:teuthology.orchestra.run.smithi112.stdout:btime 2024-09-16T00:34:26:619254+0000 2024-09-16T00:48:52.909 INFO:teuthology.orchestra.run.smithi112.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T00:48:52.909 INFO:teuthology.orchestra.run.smithi112.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:48:52.909 INFO:teuthology.orchestra.run.smithi112.stdout:legacy client fscid: 1 2024-09-16T00:48:52.910 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:48:52.910 INFO:teuthology.orchestra.run.smithi112.stdout:Filesystem 'cephfs' (1) 2024-09-16T00:48:52.910 INFO:teuthology.orchestra.run.smithi112.stdout:fs_name cephfs 2024-09-16T00:48:52.910 INFO:teuthology.orchestra.run.smithi112.stdout:epoch 25 2024-09-16T00:48:52.910 INFO:teuthology.orchestra.run.smithi112.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-16T00:48:52.910 INFO:teuthology.orchestra.run.smithi112.stdout:created 2024-09-16T00:23:22.328199+0000 2024-09-16T00:48:52.910 INFO:teuthology.orchestra.run.smithi112.stdout:modified 2024-09-16T00:33:29.012991+0000 2024-09-16T00:48:52.910 INFO:teuthology.orchestra.run.smithi112.stdout:tableserver 0 2024-09-16T00:48:52.910 INFO:teuthology.orchestra.run.smithi112.stdout:root 0 2024-09-16T00:48:52.910 INFO:teuthology.orchestra.run.smithi112.stdout:session_timeout 60 2024-09-16T00:48:52.910 INFO:teuthology.orchestra.run.smithi112.stdout:session_autoclose 300 2024-09-16T00:48:52.910 INFO:teuthology.orchestra.run.smithi112.stdout:max_file_size 1099511627776 2024-09-16T00:48:52.910 INFO:teuthology.orchestra.run.smithi112.stdout:max_xattr_size 65536 2024-09-16T00:48:52.911 INFO:teuthology.orchestra.run.smithi112.stdout:required_client_features {} 2024-09-16T00:48:52.911 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure 0 2024-09-16T00:48:52.911 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure_osd_epoch 82 2024-09-16T00:48:52.911 INFO:teuthology.orchestra.run.smithi112.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:48:52.911 INFO:teuthology.orchestra.run.smithi112.stdout:max_mds 2 2024-09-16T00:48:52.911 INFO:teuthology.orchestra.run.smithi112.stdout:in 0,1 2024-09-16T00:48:52.911 INFO:teuthology.orchestra.run.smithi112.stdout:up {} 2024-09-16T00:48:52.911 INFO:teuthology.orchestra.run.smithi112.stdout:failed 0,1 2024-09-16T00:48:52.911 INFO:teuthology.orchestra.run.smithi112.stdout:damaged 2024-09-16T00:48:52.912 INFO:teuthology.orchestra.run.smithi112.stdout:stopped 2024-09-16T00:48:52.912 INFO:teuthology.orchestra.run.smithi112.stdout:data_pools [3] 2024-09-16T00:48:52.912 INFO:teuthology.orchestra.run.smithi112.stdout:metadata_pool 2 2024-09-16T00:48:52.912 INFO:teuthology.orchestra.run.smithi112.stdout:inline_data enabled 2024-09-16T00:48:52.912 INFO:teuthology.orchestra.run.smithi112.stdout:balancer 2024-09-16T00:48:52.912 INFO:teuthology.orchestra.run.smithi112.stdout:bal_rank_mask -1 2024-09-16T00:48:52.912 INFO:teuthology.orchestra.run.smithi112.stdout:standby_count_wanted 1 2024-09-16T00:48:52.912 INFO:teuthology.orchestra.run.smithi112.stdout:qdb_cluster leader: 0 members: 2024-09-16T00:48:52.912 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:48:52.912 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:48:52.913 INFO:teuthology.orchestra.run.smithi112.stdout:Standby daemons: 2024-09-16T00:48:52.913 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:48:52.913 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.ithwzj{-1:34392} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.112:6826/476229221,v1:172.21.15.112:6827/476229221] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:48:52.913 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.hxfwkh{-1:34404} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.112:6828/2356899876,v1:172.21.15.112:6829/2356899876] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:48:52.913 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.gawwgw{-1:44343} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.169:6824/1635329957,v1:172.21.15.169:6825/1635329957] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:48:52.913 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.lgeqrj{-1:44347} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.169:6826/184810031,v1:172.21.15.169:6827/184810031] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:48:52.914 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 34 2024-09-16T00:48:52.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.910+0000 7f1faf537640 1 -- 172.21.15.112:0/873166197 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f1f9406e530 msgr2=0x7f1f940709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:52.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.910+0000 7f1faf537640 1 --2- 172.21.15.112:0/873166197 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f1f9406e530 0x7f1f940709e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4007ac0 tx=0x7f1fa4035040 comp rx=0 tx=0).stop 2024-09-16T00:48:52.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.910+0000 7f1faf537640 1 -- 172.21.15.112:0/873166197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fa81014e0 msgr2=0x7f1fa81136f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:52.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.910+0000 7f1faf537640 1 --2- 172.21.15.112:0/873166197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fa81014e0 0x7f1fa81136f0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f1f9800c790 tx=0x7f1f980043c0 comp rx=0 tx=0).stop 2024-09-16T00:48:52.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.910+0000 7f1faf537640 1 -- 172.21.15.112:0/873166197 shutdown_connections 2024-09-16T00:48:52.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.910+0000 7f1faf537640 1 --2- 172.21.15.112:0/873166197 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f1f9406e530 0x7f1f940709e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:52.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.910+0000 7f1faf537640 1 --2- 172.21.15.112:0/873166197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fa81014e0 0x7f1fa81136f0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:52.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.910+0000 7f1faf537640 1 --2- 172.21.15.112:0/873166197 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1fa80fb4a0 0x7f1fa81131b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:52.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.910+0000 7f1faf537640 1 -- 172.21.15.112:0/873166197 >> 172.21.15.112:0/873166197 conn(0x7f1fa80f7060 msgr2=0x7f1fa8065a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:48:52.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.910+0000 7f1faf537640 1 -- 172.21.15.112:0/873166197 shutdown_connections 2024-09-16T00:48:52.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.910+0000 7f1faf537640 1 -- 172.21.15.112:0/873166197 wait complete. 2024-09-16T00:48:52.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.989+0000 7fc0fa22f640 1 -- 172.21.15.112:0/3280007849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0f4102ef0 msgr2=0x7fc0f40fc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:52.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.989+0000 7fc0fa22f640 1 --2- 172.21.15.112:0/3280007849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0f4102ef0 0x7fc0f40fc5b0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fc0e80048d0 tx=0x7fc0e8009da0 comp rx=0 tx=0).stop 2024-09-16T00:48:52.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.989+0000 7fc0fa22f640 1 -- 172.21.15.112:0/3280007849 shutdown_connections 2024-09-16T00:48:52.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.989+0000 7fc0fa22f640 1 --2- 172.21.15.112:0/3280007849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0f4102ef0 0x7fc0f40fc5b0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:52.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.989+0000 7fc0fa22f640 1 --2- 172.21.15.112:0/3280007849 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0f4102550 0x7fc0f4102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:52.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.989+0000 7fc0fa22f640 1 -- 172.21.15.112:0/3280007849 >> 172.21.15.112:0/3280007849 conn(0x7fc0f40f80f0 msgr2=0x7fc0f40fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:48:52.991 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:48:52 smithi112 ceph-mon[97750]: from='client.35192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:48:52.991 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:48:52 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/1504222167' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:48:52.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.990+0000 7fc0fa22f640 1 -- 172.21.15.112:0/3280007849 shutdown_connections 2024-09-16T00:48:52.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.990+0000 7fc0fa22f640 1 -- 172.21.15.112:0/3280007849 wait complete. 2024-09-16T00:48:52.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.990+0000 7fc0fa22f640 1 Processor -- start 2024-09-16T00:48:52.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.990+0000 7fc0fa22f640 1 -- start start 2024-09-16T00:48:52.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.991+0000 7fc0fa22f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0f4102550 0x7fc0f4113520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:52.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.991+0000 7fc0fa22f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0f4102ef0 0x7fc0f4113a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:52.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.991+0000 7fc0fa22f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f41140b0 con 0x7fc0f4102ef0 2024-09-16T00:48:52.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.991+0000 7fc0fa22f640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f410d4c0 con 0x7fc0f4102550 2024-09-16T00:48:52.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.991+0000 7fc0f2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0f4102ef0 0x7fc0f4113a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:52.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.991+0000 7fc0f2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0f4102ef0 0x7fc0f4113a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50830/0 (socket says 172.21.15.112:50830) 2024-09-16T00:48:52.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.991+0000 7fc0f2ffd640 1 -- 172.21.15.112:0/2746775317 learned_addr learned my addr 172.21.15.112:0/2746775317 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:48:52.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.991+0000 7fc0f37fe640 1 --2- 172.21.15.112:0/2746775317 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0f4102550 0x7fc0f4113520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:52.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.991+0000 7fc0f2ffd640 1 -- 172.21.15.112:0/2746775317 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0f4102550 msgr2=0x7fc0f4113520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:52.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.991+0000 7fc0f2ffd640 1 --2- 172.21.15.112:0/2746775317 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0f4102550 0x7fc0f4113520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:52.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.991+0000 7fc0f2ffd640 1 -- 172.21.15.112:0/2746775317 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0e8005040 con 0x7fc0f4102ef0 2024-09-16T00:48:52.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.991+0000 7fc0f37fe640 1 --2- 172.21.15.112:0/2746775317 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0f4102550 0x7fc0f4113520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:48:52.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.992+0000 7fc0f2ffd640 1 --2- 172.21.15.112:0/2746775317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0f4102ef0 0x7fc0f4113a60 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fc0e8003080 tx=0x7fc0e8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:48:52.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.992+0000 7fc0dbfff640 1 -- 172.21.15.112:0/2746775317 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0e8038070 con 0x7fc0f4102ef0 2024-09-16T00:48:52.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.992+0000 7fc0dbfff640 1 -- 172.21.15.112:0/2746775317 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fc0e8009760 con 0x7fc0f4102ef0 2024-09-16T00:48:52.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.992+0000 7fc0fa22f640 1 -- 172.21.15.112:0/2746775317 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0f410d740 con 0x7fc0f4102ef0 2024-09-16T00:48:52.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.992+0000 7fc0dbfff640 1 -- 172.21.15.112:0/2746775317 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0e8033420 con 0x7fc0f4102ef0 2024-09-16T00:48:52.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.992+0000 7fc0fa22f640 1 -- 172.21.15.112:0/2746775317 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0f410dc20 con 0x7fc0f4102ef0 2024-09-16T00:48:52.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.993+0000 7fc0fa22f640 1 -- 172.21.15.112:0/2746775317 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0f40fdcd0 con 0x7fc0f4102ef0 2024-09-16T00:48:52.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.994+0000 7fc0dbfff640 1 -- 172.21.15.112:0/2746775317 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fc0e8030020 con 0x7fc0f4102ef0 2024-09-16T00:48:52.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.994+0000 7fc0dbfff640 1 --2- 172.21.15.112:0/2746775317 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fc0dc06e530 0x7fc0dc0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:52.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.994+0000 7fc0dbfff640 1 -- 172.21.15.112:0/2746775317 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7fc0e80ae080 con 0x7fc0f4102ef0 2024-09-16T00:48:52.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.994+0000 7fc0f37fe640 1 --2- 172.21.15.112:0/2746775317 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fc0dc06e530 0x7fc0dc0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:52.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.995+0000 7fc0f37fe640 1 --2- 172.21.15.112:0/2746775317 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fc0dc06e530 0x7fc0dc0709e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4004550 tx=0x7fc0e4009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:48:52.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:52.998+0000 7fc0dbfff640 1 -- 172.21.15.112:0/2746775317 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fc0e8076ff0 con 0x7fc0f4102ef0 2024-09-16T00:48:53.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:53.180+0000 7fc0fa22f640 1 -- 172.21.15.112:0/2746775317 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc0f4109580 con 0x7fc0dc06e530 2024-09-16T00:48:53.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:53.182+0000 7fc0dbfff640 1 -- 172.21.15.112:0/2746775317 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fc0f4109580 con 0x7fc0dc06e530 2024-09-16T00:48:53.182 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:48:53.182 INFO:teuthology.orchestra.run.smithi112.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80", 2024-09-16T00:48:53.182 INFO:teuthology.orchestra.run.smithi112.stdout: "in_progress": true, 2024-09-16T00:48:53.182 INFO:teuthology.orchestra.run.smithi112.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T00:48:53.182 INFO:teuthology.orchestra.run.smithi112.stdout: "services_complete": [ 2024-09-16T00:48:53.182 INFO:teuthology.orchestra.run.smithi112.stdout: "mds", 2024-09-16T00:48:53.182 INFO:teuthology.orchestra.run.smithi112.stdout: "osd", 2024-09-16T00:48:53.182 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr", 2024-09-16T00:48:53.182 INFO:teuthology.orchestra.run.smithi112.stdout: "mon", 2024-09-16T00:48:53.183 INFO:teuthology.orchestra.run.smithi112.stdout: "crash" 2024-09-16T00:48:53.183 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-16T00:48:53.183 INFO:teuthology.orchestra.run.smithi112.stdout: "progress": "16/21 daemons upgraded", 2024-09-16T00:48:53.183 INFO:teuthology.orchestra.run.smithi112.stdout: "message": "Currently upgrading mds daemons", 2024-09-16T00:48:53.183 INFO:teuthology.orchestra.run.smithi112.stdout: "is_paused": false 2024-09-16T00:48:53.183 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:48:53.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:53.183+0000 7fc0fa22f640 1 -- 172.21.15.112:0/2746775317 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fc0dc06e530 msgr2=0x7fc0dc0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:53.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:53.183+0000 7fc0fa22f640 1 --2- 172.21.15.112:0/2746775317 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fc0dc06e530 0x7fc0dc0709e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4004550 tx=0x7fc0e4009040 comp rx=0 tx=0).stop 2024-09-16T00:48:53.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:53.183+0000 7fc0fa22f640 1 -- 172.21.15.112:0/2746775317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0f4102ef0 msgr2=0x7fc0f4113a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:53.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:53.184+0000 7fc0fa22f640 1 --2- 172.21.15.112:0/2746775317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0f4102ef0 0x7fc0f4113a60 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fc0e8003080 tx=0x7fc0e8009da0 comp rx=0 tx=0).stop 2024-09-16T00:48:53.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:53.184+0000 7fc0fa22f640 1 -- 172.21.15.112:0/2746775317 shutdown_connections 2024-09-16T00:48:53.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:53.184+0000 7fc0fa22f640 1 --2- 172.21.15.112:0/2746775317 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fc0dc06e530 0x7fc0dc0709e0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:53.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:53.184+0000 7fc0fa22f640 1 --2- 172.21.15.112:0/2746775317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0f4102ef0 0x7fc0f4113a60 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:53.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:53.184+0000 7fc0fa22f640 1 --2- 172.21.15.112:0/2746775317 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0f4102550 0x7fc0f4113520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:53.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:53.184+0000 7fc0fa22f640 1 -- 172.21.15.112:0/2746775317 >> 172.21.15.112:0/2746775317 conn(0x7fc0f40f80f0 msgr2=0x7fc0f40f9150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:48:53.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:53.184+0000 7fc0fa22f640 1 -- 172.21.15.112:0/2746775317 shutdown_connections 2024-09-16T00:48:53.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:53.184+0000 7fc0fa22f640 1 -- 172.21.15.112:0/2746775317 wait complete. 2024-09-16T00:48:53.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:53.263+0000 7f4443b46640 1 -- 172.21.15.112:0/691533066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f443c0fb810 msgr2=0x7f443c0fbbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:53.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:53.263+0000 7f4443b46640 1 --2- 172.21.15.112:0/691533066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f443c0fb810 0x7f443c0fbbe0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f442c0048d0 tx=0x7f442c009da0 comp rx=0 tx=0).stop 2024-09-16T00:48:53.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:53.263+0000 7f4443b46640 1 -- 172.21.15.112:0/691533066 shutdown_connections 2024-09-16T00:48:53.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:53.263+0000 7f4443b46640 1 --2- 172.21.15.112:0/691533066 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f443c0fcf80 0x7f443c0fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:53.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:53.263+0000 7f4443b46640 1 --2- 172.21.15.112:0/691533066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f443c0fb810 0x7f443c0fbbe0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:53.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:53.263+0000 7f4443b46640 1 -- 172.21.15.112:0/691533066 >> 172.21.15.112:0/691533066 conn(0x7f443c0723b0 msgr2=0x7f443c0727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:48:53.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:53.263+0000 7f4443b46640 1 -- 172.21.15.112:0/691533066 shutdown_connections 2024-09-16T00:48:53.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:53.263+0000 7f4443b46640 1 -- 172.21.15.112:0/691533066 wait complete. 2024-09-16T00:48:53.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:53.263+0000 7f4443b46640 1 Processor -- start 2024-09-16T00:48:53.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:53.263+0000 7f4443b46640 1 -- start start 2024-09-16T00:48:53.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:53.264+0000 7f4443b46640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f443c0fb810 0x7f443c0f8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:53.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:53.264+0000 7f4443b46640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f443c0fcf80 0x7f443c0f8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:53.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:53.264+0000 7f4443b46640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f443c0f8fc0 con 0x7f443c0fcf80 2024-09-16T00:48:53.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:53.264+0000 7f4443b46640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f443c0f9130 con 0x7f443c0fb810 2024-09-16T00:48:53.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:53.264+0000 7f44410e1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f443c0fcf80 0x7f443c0f8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:53.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:53.264+0000 7f44410e1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f443c0fcf80 0x7f443c0f8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50850/0 (socket says 172.21.15.112:50850) 2024-09-16T00:48:53.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:53.264+0000 7f44410e1640 1 -- 172.21.15.112:0/1309572571 learned_addr learned my addr 172.21.15.112:0/1309572571 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:48:53.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:53.264+0000 7f44418e2640 1 --2- 172.21.15.112:0/1309572571 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f443c0fb810 0x7f443c0f8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:53.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:53.264+0000 7f44410e1640 1 -- 172.21.15.112:0/1309572571 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f443c0fb810 msgr2=0x7f443c0f8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:53.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:53.264+0000 7f44410e1640 1 --2- 172.21.15.112:0/1309572571 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f443c0fb810 0x7f443c0f8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:53.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:53.264+0000 7f44410e1640 1 -- 172.21.15.112:0/1309572571 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f442c005040 con 0x7f443c0fcf80 2024-09-16T00:48:53.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:53.264+0000 7f44418e2640 1 --2- 172.21.15.112:0/1309572571 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f443c0fb810 0x7f443c0f8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:48:53.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:53.264+0000 7f44410e1640 1 --2- 172.21.15.112:0/1309572571 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f443c0fcf80 0x7f443c0f8a80 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f443800c830 tx=0x7f4438004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:48:53.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:53.264+0000 7f44327fc640 1 -- 172.21.15.112:0/1309572571 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f443800a850 con 0x7f443c0fcf80 2024-09-16T00:48:53.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:53.264+0000 7f44327fc640 1 -- 172.21.15.112:0/1309572571 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f443800ae50 con 0x7f443c0fcf80 2024-09-16T00:48:53.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:53.265+0000 7f4443b46640 1 -- 172.21.15.112:0/1309572571 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f443c0f93c0 con 0x7f443c0fcf80 2024-09-16T00:48:53.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:53.265+0000 7f4443b46640 1 -- 172.21.15.112:0/1309572571 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f443c119940 con 0x7f443c0fcf80 2024-09-16T00:48:53.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:53.265+0000 7f44327fc640 1 -- 172.21.15.112:0/1309572571 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f44380113a0 con 0x7f443c0fcf80 2024-09-16T00:48:53.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:53.265+0000 7f4443b46640 1 -- 172.21.15.112:0/1309572571 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f443c0fcbc0 con 0x7f443c0fcf80 2024-09-16T00:48:53.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:53.266+0000 7f44327fc640 1 -- 172.21.15.112:0/1309572571 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f4438012400 con 0x7f443c0fcf80 2024-09-16T00:48:53.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:53.267+0000 7f44327fc640 1 --2- 172.21.15.112:0/1309572571 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f442806e530 0x7f44280709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:48:53.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:53.267+0000 7f44418e2640 1 --2- 172.21.15.112:0/1309572571 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f442806e530 0x7f44280709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:48:53.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:53.267+0000 7f44327fc640 1 -- 172.21.15.112:0/1309572571 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f443808e540 con 0x7f443c0fcf80 2024-09-16T00:48:53.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:53.268+0000 7f44418e2640 1 --2- 172.21.15.112:0/1309572571 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f442806e530 0x7f44280709e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f442c007ac0 tx=0x7f442c035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:48:53.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:53.271+0000 7f44327fc640 1 -- 172.21.15.112:0/1309572571 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f4438057530 con 0x7f443c0fcf80 2024-09-16T00:48:53.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:48:52 smithi169 ceph-mon[77626]: from='client.35192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:48:53.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:48:52 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/1504222167' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:48:53.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:53.499+0000 7f4443b46640 1 -- 172.21.15.112:0/1309572571 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f443c072ec0 con 0x7f443c0fcf80 2024-09-16T00:48:53.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:53.499+0000 7f44327fc640 1 -- 172.21.15.112:0/1309572571 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f443805abf0 con 0x7f443c0fcf80 2024-09-16T00:48:53.500 INFO:teuthology.orchestra.run.smithi112.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-16T00:48:53.500 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-16T00:48:53.500 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs is degraded 2024-09-16T00:48:53.500 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T00:48:53.500 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T00:48:53.500 INFO:teuthology.orchestra.run.smithi112.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-16T00:48:53.500 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-16T00:48:53.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:53.501+0000 7f4443b46640 1 -- 172.21.15.112:0/1309572571 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f442806e530 msgr2=0x7f44280709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:53.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:53.501+0000 7f4443b46640 1 --2- 172.21.15.112:0/1309572571 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f442806e530 0x7f44280709e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f442c007ac0 tx=0x7f442c035040 comp rx=0 tx=0).stop 2024-09-16T00:48:53.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:53.501+0000 7f4443b46640 1 -- 172.21.15.112:0/1309572571 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f443c0fcf80 msgr2=0x7f443c0f8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:48:53.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:53.501+0000 7f4443b46640 1 --2- 172.21.15.112:0/1309572571 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f443c0fcf80 0x7f443c0f8a80 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f443800c830 tx=0x7f4438004430 comp rx=0 tx=0).stop 2024-09-16T00:48:53.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:53.501+0000 7f4443b46640 1 -- 172.21.15.112:0/1309572571 shutdown_connections 2024-09-16T00:48:53.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:53.501+0000 7f4443b46640 1 --2- 172.21.15.112:0/1309572571 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f442806e530 0x7f44280709e0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:53.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:53.501+0000 7f4443b46640 1 --2- 172.21.15.112:0/1309572571 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f443c0fcf80 0x7f443c0f8a80 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:53.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:53.501+0000 7f4443b46640 1 --2- 172.21.15.112:0/1309572571 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f443c0fb810 0x7f443c0f8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:48:53.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:53.501+0000 7f4443b46640 1 -- 172.21.15.112:0/1309572571 >> 172.21.15.112:0/1309572571 conn(0x7f443c0723b0 msgr2=0x7f443c064f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:48:53.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:53.501+0000 7f4443b46640 1 -- 172.21.15.112:0/1309572571 shutdown_connections 2024-09-16T00:48:53.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:48:53.501+0000 7f4443b46640 1 -- 172.21.15.112:0/1309572571 wait complete. 2024-09-16T00:48:54.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:48:53 smithi112 ceph-mon[97750]: from='client.35196 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:48:54.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:48:53 smithi112 ceph-mon[97750]: from='client.35200 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:48:54.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:48:53 smithi112 ceph-mon[97750]: pgmap v634: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:48:54.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:48:53 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/873166197' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:48:54.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:48:53 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/1309572571' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:48:54.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:48:53 smithi169 ceph-mon[77626]: from='client.35196 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:48:54.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:48:53 smithi169 ceph-mon[77626]: from='client.35200 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:48:54.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:48:53 smithi169 ceph-mon[77626]: pgmap v634: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:48:54.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:48:53 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/873166197' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:48:54.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:48:53 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/1309572571' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:48:55.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:48:54 smithi169 ceph-mon[77626]: from='client.35212 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:48:55.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:48:54 smithi112 ceph-mon[97750]: from='client.35212 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:48:56.178 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:48:55 smithi169 ceph-mon[77626]: pgmap v635: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:48:56.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:48:55 smithi112 ceph-mon[97750]: pgmap v635: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:48:58.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:48:57 smithi169 ceph-mon[77626]: pgmap v636: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:48:58.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:48:57 smithi112 ceph-mon[97750]: pgmap v636: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:49:00.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:48:59 smithi169 ceph-mon[77626]: pgmap v637: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:49:00.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:48:59 smithi112 ceph-mon[97750]: pgmap v637: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:49:02.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:01 smithi169 ceph-mon[77626]: pgmap v638: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:49:02.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:01 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:49:02.314 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:01 smithi112 ceph-mon[97750]: pgmap v638: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:49:02.314 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:01 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:49:04.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:03 smithi112 ceph-mon[97750]: pgmap v639: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:49:04.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:03 smithi169 ceph-mon[77626]: pgmap v639: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:49:06.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:05 smithi169 ceph-mon[77626]: pgmap v640: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:49:06.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:05 smithi112 ceph-mon[97750]: pgmap v640: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:49:08.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:07 smithi169 ceph-mon[77626]: pgmap v641: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:49:08.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:07 smithi112 ceph-mon[97750]: pgmap v641: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:49:10.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:09 smithi169 ceph-mon[77626]: pgmap v642: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:49:10.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:09 smithi112 ceph-mon[97750]: pgmap v642: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:49:12.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:11 smithi169 ceph-mon[77626]: pgmap v643: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:49:12.314 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:11 smithi112 ceph-mon[97750]: pgmap v643: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:49:14.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:13 smithi112 ceph-mon[97750]: pgmap v644: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:49:14.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:13 smithi169 ceph-mon[77626]: pgmap v644: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:49:16.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:15 smithi169 ceph-mon[77626]: pgmap v645: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:49:16.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:15 smithi112 ceph-mon[97750]: pgmap v645: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:49:17.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:16 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:49:17.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:16 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:49:18.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:17 smithi169 ceph-mon[77626]: pgmap v646: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:49:18.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:17 smithi112 ceph-mon[97750]: pgmap v646: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:49:20.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:19 smithi169 ceph-mon[77626]: pgmap v647: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:49:20.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:19 smithi112 ceph-mon[97750]: pgmap v647: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:49:22.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:21 smithi169 ceph-mon[77626]: pgmap v648: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:49:22.314 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:21 smithi112 ceph-mon[97750]: pgmap v648: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:49:23.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.590+0000 7fa98f631640 1 -- 172.21.15.112:0/3271711514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9880fac80 msgr2=0x7fa9880fb0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:23.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.590+0000 7fa98f631640 1 --2- 172.21.15.112:0/3271711514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9880fac80 0x7fa9880fb0f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fa9780048d0 tx=0x7fa978009da0 comp rx=0 tx=0).stop 2024-09-16T00:49:23.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.590+0000 7fa98f631640 1 -- 172.21.15.112:0/3271711514 shutdown_connections 2024-09-16T00:49:23.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.590+0000 7fa98f631640 1 --2- 172.21.15.112:0/3271711514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9880fac80 0x7fa9880fb0f0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:23.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.590+0000 7fa98f631640 1 --2- 172.21.15.112:0/3271711514 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa988102620 0x7fa9881029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:23.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.590+0000 7fa98f631640 1 -- 172.21.15.112:0/3271711514 >> 172.21.15.112:0/3271711514 conn(0x7fa988070ce0 msgr2=0x7fa9880710e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:49:23.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.590+0000 7fa98f631640 1 -- 172.21.15.112:0/3271711514 shutdown_connections 2024-09-16T00:49:23.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.590+0000 7fa98f631640 1 -- 172.21.15.112:0/3271711514 wait complete. 2024-09-16T00:49:23.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.591+0000 7fa98f631640 1 Processor -- start 2024-09-16T00:49:23.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.591+0000 7fa98f631640 1 -- start start 2024-09-16T00:49:23.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.591+0000 7fa98f631640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9880fac80 0x7fa9880fbe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:23.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.591+0000 7fa98f631640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa988102620 0x7fa9880fc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:23.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.591+0000 7fa98f631640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9880fc8e0 con 0x7fa988102620 2024-09-16T00:49:23.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.591+0000 7fa98f631640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9880fca50 con 0x7fa9880fac80 2024-09-16T00:49:23.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.591+0000 7fa98cbcc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa988102620 0x7fa9880fc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:23.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.591+0000 7fa98cbcc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa988102620 0x7fa9880fc3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49132/0 (socket says 172.21.15.112:49132) 2024-09-16T00:49:23.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.591+0000 7fa98cbcc640 1 -- 172.21.15.112:0/3383327863 learned_addr learned my addr 172.21.15.112:0/3383327863 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:49:23.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.591+0000 7fa98d3cd640 1 --2- 172.21.15.112:0/3383327863 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9880fac80 0x7fa9880fbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:23.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.592+0000 7fa98cbcc640 1 -- 172.21.15.112:0/3383327863 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9880fac80 msgr2=0x7fa9880fbe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:23.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.592+0000 7fa98cbcc640 1 --2- 172.21.15.112:0/3383327863 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9880fac80 0x7fa9880fbe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:23.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.592+0000 7fa98cbcc640 1 -- 172.21.15.112:0/3383327863 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa978005040 con 0x7fa988102620 2024-09-16T00:49:23.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.592+0000 7fa98cbcc640 1 --2- 172.21.15.112:0/3383327863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa988102620 0x7fa9880fc3a0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fa978000c00 tx=0x7fa978009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:49:23.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.592+0000 7fa97dffb640 1 -- 172.21.15.112:0/3383327863 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa978038070 con 0x7fa988102620 2024-09-16T00:49:23.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.592+0000 7fa97dffb640 1 -- 172.21.15.112:0/3383327863 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fa97802ade0 con 0x7fa988102620 2024-09-16T00:49:23.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.592+0000 7fa97dffb640 1 -- 172.21.15.112:0/3383327863 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa97802c7e0 con 0x7fa988102620 2024-09-16T00:49:23.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.592+0000 7fa98f631640 1 -- 172.21.15.112:0/3383327863 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9880fcc80 con 0x7fa988102620 2024-09-16T00:49:23.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.592+0000 7fa98f631640 1 -- 172.21.15.112:0/3383327863 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9881197d0 con 0x7fa988102620 2024-09-16T00:49:23.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.594+0000 7fa98f631640 1 -- 172.21.15.112:0/3383327863 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9880657b0 con 0x7fa988102620 2024-09-16T00:49:23.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.594+0000 7fa97dffb640 1 -- 172.21.15.112:0/3383327863 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fa978030020 con 0x7fa988102620 2024-09-16T00:49:23.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.594+0000 7fa97dffb640 1 --2- 172.21.15.112:0/3383327863 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa97406e530 0x7fa9740709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:23.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.595+0000 7fa97dffb640 1 -- 172.21.15.112:0/3383327863 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7fa9780aded0 con 0x7fa988102620 2024-09-16T00:49:23.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.595+0000 7fa98d3cd640 1 --2- 172.21.15.112:0/3383327863 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa97406e530 0x7fa9740709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:23.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.595+0000 7fa98d3cd640 1 --2- 172.21.15.112:0/3383327863 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa97406e530 0x7fa9740709e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa984009470 tx=0x7fa984009ed0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:49:23.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.599+0000 7fa97dffb640 1 -- 172.21.15.112:0/3383327863 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fa978076f70 con 0x7fa988102620 2024-09-16T00:49:23.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.783+0000 7fa98f631640 1 -- 172.21.15.112:0/3383327863 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa98804b720 con 0x7fa97406e530 2024-09-16T00:49:23.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.784+0000 7fa97dffb640 1 -- 172.21.15.112:0/3383327863 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fa98804b720 con 0x7fa97406e530 2024-09-16T00:49:23.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.787+0000 7fa98f631640 1 -- 172.21.15.112:0/3383327863 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa97406e530 msgr2=0x7fa9740709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:23.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.787+0000 7fa98f631640 1 --2- 172.21.15.112:0/3383327863 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa97406e530 0x7fa9740709e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa984009470 tx=0x7fa984009ed0 comp rx=0 tx=0).stop 2024-09-16T00:49:23.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.787+0000 7fa98f631640 1 -- 172.21.15.112:0/3383327863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa988102620 msgr2=0x7fa9880fc3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:23.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.787+0000 7fa98f631640 1 --2- 172.21.15.112:0/3383327863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa988102620 0x7fa9880fc3a0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fa978000c00 tx=0x7fa978009da0 comp rx=0 tx=0).stop 2024-09-16T00:49:23.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.787+0000 7fa98f631640 1 -- 172.21.15.112:0/3383327863 shutdown_connections 2024-09-16T00:49:23.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.787+0000 7fa98f631640 1 --2- 172.21.15.112:0/3383327863 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fa97406e530 0x7fa9740709e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:23.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.787+0000 7fa98f631640 1 --2- 172.21.15.112:0/3383327863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa988102620 0x7fa9880fc3a0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:23.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.787+0000 7fa98f631640 1 --2- 172.21.15.112:0/3383327863 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9880fac80 0x7fa9880fbe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:23.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.787+0000 7fa98f631640 1 -- 172.21.15.112:0/3383327863 >> 172.21.15.112:0/3383327863 conn(0x7fa988070ce0 msgr2=0x7fa988100130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:49:23.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.787+0000 7fa98f631640 1 -- 172.21.15.112:0/3383327863 shutdown_connections 2024-09-16T00:49:23.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.787+0000 7fa98f631640 1 -- 172.21.15.112:0/3383327863 wait complete. 2024-09-16T00:49:23.796 INFO:teuthology.orchestra.run.smithi112.stdout:true 2024-09-16T00:49:23.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.867+0000 7f2fee5a5640 1 -- 172.21.15.112:0/176892304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fe8102ed0 msgr2=0x7f2fe80fc590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:23.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.867+0000 7f2fee5a5640 1 --2- 172.21.15.112:0/176892304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fe8102ed0 0x7f2fe80fc590 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc0048d0 tx=0x7f2fdc009da0 comp rx=0 tx=0).stop 2024-09-16T00:49:23.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.867+0000 7f2fee5a5640 1 -- 172.21.15.112:0/176892304 shutdown_connections 2024-09-16T00:49:23.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.867+0000 7f2fee5a5640 1 --2- 172.21.15.112:0/176892304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fe8102ed0 0x7f2fe80fc590 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:23.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.867+0000 7f2fee5a5640 1 --2- 172.21.15.112:0/176892304 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2fe8102530 0x7f2fe8102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:23.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.867+0000 7f2fee5a5640 1 -- 172.21.15.112:0/176892304 >> 172.21.15.112:0/176892304 conn(0x7f2fe80f80f0 msgr2=0x7f2fe80fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:49:23.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.867+0000 7f2fee5a5640 1 -- 172.21.15.112:0/176892304 shutdown_connections 2024-09-16T00:49:23.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.867+0000 7f2fee5a5640 1 -- 172.21.15.112:0/176892304 wait complete. 2024-09-16T00:49:23.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.867+0000 7f2fee5a5640 1 Processor -- start 2024-09-16T00:49:23.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.867+0000 7f2fee5a5640 1 -- start start 2024-09-16T00:49:23.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.868+0000 7f2fee5a5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fe8102530 0x7f2fe81131d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:23.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.868+0000 7f2fee5a5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2fe8102ed0 0x7f2fe8113710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:23.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.868+0000 7f2fee5a5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe8113d10 con 0x7f2fe8102530 2024-09-16T00:49:23.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.868+0000 7f2fee5a5640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe810d150 con 0x7f2fe8102ed0 2024-09-16T00:49:23.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.868+0000 7f2fe7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fe8102530 0x7f2fe81131d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:23.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.868+0000 7f2fe7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fe8102530 0x7f2fe81131d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49144/0 (socket says 172.21.15.112:49144) 2024-09-16T00:49:23.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.868+0000 7f2fe7fff640 1 -- 172.21.15.112:0/2324715584 learned_addr learned my addr 172.21.15.112:0/2324715584 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:49:23.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.868+0000 7f2fe77fe640 1 --2- 172.21.15.112:0/2324715584 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2fe8102ed0 0x7f2fe8113710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:23.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.868+0000 7f2fe7fff640 1 -- 172.21.15.112:0/2324715584 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2fe8102ed0 msgr2=0x7f2fe8113710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:23.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.868+0000 7f2fe7fff640 1 --2- 172.21.15.112:0/2324715584 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2fe8102ed0 0x7f2fe8113710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:23.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.868+0000 7f2fe7fff640 1 -- 172.21.15.112:0/2324715584 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fdc005040 con 0x7f2fe8102530 2024-09-16T00:49:23.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.868+0000 7f2fe7fff640 1 --2- 172.21.15.112:0/2324715584 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fe8102530 0x7f2fe81131d0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f2fd800c830 tx=0x7f2fd8004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:49:23.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.868+0000 7f2fe4ff9640 1 -- 172.21.15.112:0/2324715584 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2fd800a7d0 con 0x7f2fe8102530 2024-09-16T00:49:23.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.868+0000 7f2fe4ff9640 1 -- 172.21.15.112:0/2324715584 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f2fd800add0 con 0x7f2fe8102530 2024-09-16T00:49:23.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.868+0000 7f2fee5a5640 1 -- 172.21.15.112:0/2324715584 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fe810d430 con 0x7f2fe8102530 2024-09-16T00:49:23.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.869+0000 7f2fe4ff9640 1 -- 172.21.15.112:0/2324715584 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2fd80113a0 con 0x7f2fe8102530 2024-09-16T00:49:23.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.869+0000 7f2fee5a5640 1 -- 172.21.15.112:0/2324715584 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fe810d9a0 con 0x7f2fe8102530 2024-09-16T00:49:23.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.870+0000 7f2fee5a5640 1 -- 172.21.15.112:0/2324715584 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fe80fdcb0 con 0x7f2fe8102530 2024-09-16T00:49:23.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.870+0000 7f2fe4ff9640 1 -- 172.21.15.112:0/2324715584 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f2fd8012400 con 0x7f2fe8102530 2024-09-16T00:49:23.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.871+0000 7f2fe4ff9640 1 --2- 172.21.15.112:0/2324715584 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f2fd006e530 0x7f2fd00709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:23.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.871+0000 7f2fe4ff9640 1 -- 172.21.15.112:0/2324715584 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f2fd808e320 con 0x7f2fe8102530 2024-09-16T00:49:23.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.871+0000 7f2fe77fe640 1 --2- 172.21.15.112:0/2324715584 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f2fd006e530 0x7f2fd00709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:23.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.871+0000 7f2fe77fe640 1 --2- 172.21.15.112:0/2324715584 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f2fd006e530 0x7f2fd00709e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc0048a0 tx=0x7f2fdc035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:49:23.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:23.875+0000 7f2fe4ff9640 1 -- 172.21.15.112:0/2324715584 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f2fd80573b0 con 0x7f2fe8102530 2024-09-16T00:49:24.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.057+0000 7f2fee5a5640 1 -- 172.21.15.112:0/2324715584 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2fe804b720 con 0x7f2fd006e530 2024-09-16T00:49:24.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.059+0000 7f2fe4ff9640 1 -- 172.21.15.112:0/2324715584 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f2fe804b720 con 0x7f2fd006e530 2024-09-16T00:49:24.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.060+0000 7f2fee5a5640 1 -- 172.21.15.112:0/2324715584 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f2fd006e530 msgr2=0x7f2fd00709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:24.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.060+0000 7f2fee5a5640 1 --2- 172.21.15.112:0/2324715584 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f2fd006e530 0x7f2fd00709e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc0048a0 tx=0x7f2fdc035040 comp rx=0 tx=0).stop 2024-09-16T00:49:24.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.061+0000 7f2fee5a5640 1 -- 172.21.15.112:0/2324715584 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fe8102530 msgr2=0x7f2fe81131d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:24.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.061+0000 7f2fee5a5640 1 --2- 172.21.15.112:0/2324715584 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fe8102530 0x7f2fe81131d0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f2fd800c830 tx=0x7f2fd8004430 comp rx=0 tx=0).stop 2024-09-16T00:49:24.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.061+0000 7f2fee5a5640 1 -- 172.21.15.112:0/2324715584 shutdown_connections 2024-09-16T00:49:24.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.061+0000 7f2fee5a5640 1 --2- 172.21.15.112:0/2324715584 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f2fd006e530 0x7f2fd00709e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:24.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.061+0000 7f2fee5a5640 1 --2- 172.21.15.112:0/2324715584 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2fe8102ed0 0x7f2fe8113710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:24.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.061+0000 7f2fee5a5640 1 --2- 172.21.15.112:0/2324715584 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fe8102530 0x7f2fe81131d0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:24.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.061+0000 7f2fee5a5640 1 -- 172.21.15.112:0/2324715584 >> 172.21.15.112:0/2324715584 conn(0x7f2fe80f80f0 msgr2=0x7f2fe80f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:49:24.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.061+0000 7f2fee5a5640 1 -- 172.21.15.112:0/2324715584 shutdown_connections 2024-09-16T00:49:24.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.061+0000 7f2fee5a5640 1 -- 172.21.15.112:0/2324715584 wait complete. 2024-09-16T00:49:24.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.138+0000 7f1021ef7640 1 -- 172.21.15.112:0/117759154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f101c102570 msgr2=0x7f101c102940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:24.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.138+0000 7f1021ef7640 1 --2- 172.21.15.112:0/117759154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f101c102570 0x7f101c102940 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f100c0048d0 tx=0x7f100c009da0 comp rx=0 tx=0).stop 2024-09-16T00:49:24.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.139+0000 7f1021ef7640 1 -- 172.21.15.112:0/117759154 shutdown_connections 2024-09-16T00:49:24.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.139+0000 7f1021ef7640 1 --2- 172.21.15.112:0/117759154 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f101c102f10 0x7f101c0fc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:24.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.139+0000 7f1021ef7640 1 --2- 172.21.15.112:0/117759154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f101c102570 0x7f101c102940 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:24.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.139+0000 7f1021ef7640 1 -- 172.21.15.112:0/117759154 >> 172.21.15.112:0/117759154 conn(0x7f101c0f8110 msgr2=0x7f101c0fa520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:49:24.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.139+0000 7f1021ef7640 1 -- 172.21.15.112:0/117759154 shutdown_connections 2024-09-16T00:49:24.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.139+0000 7f1021ef7640 1 -- 172.21.15.112:0/117759154 wait complete. 2024-09-16T00:49:24.140 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:23 smithi112 ceph-mon[97750]: pgmap v649: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:49:24.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.139+0000 7f1021ef7640 1 Processor -- start 2024-09-16T00:49:24.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.139+0000 7f1021ef7640 1 -- start start 2024-09-16T00:49:24.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.140+0000 7f1021ef7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f101c102570 0x7f101c1131f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:24.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.140+0000 7f1021ef7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f101c102f10 0x7f101c113730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:24.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.140+0000 7f1021ef7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f101c113d30 con 0x7f101c102570 2024-09-16T00:49:24.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.140+0000 7f1021ef7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f101c10d170 con 0x7f101c102f10 2024-09-16T00:49:24.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.140+0000 7f101b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f101c102570 0x7f101c1131f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:24.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.140+0000 7f101b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f101c102570 0x7f101c1131f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49174/0 (socket says 172.21.15.112:49174) 2024-09-16T00:49:24.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.140+0000 7f101b7fe640 1 -- 172.21.15.112:0/2342989972 learned_addr learned my addr 172.21.15.112:0/2342989972 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:49:24.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.140+0000 7f101affd640 1 --2- 172.21.15.112:0/2342989972 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f101c102f10 0x7f101c113730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:24.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.140+0000 7f101b7fe640 1 -- 172.21.15.112:0/2342989972 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f101c102f10 msgr2=0x7f101c113730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:24.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.140+0000 7f101b7fe640 1 --2- 172.21.15.112:0/2342989972 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f101c102f10 0x7f101c113730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:24.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.140+0000 7f101b7fe640 1 -- 172.21.15.112:0/2342989972 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f100c005040 con 0x7f101c102570 2024-09-16T00:49:24.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.140+0000 7f101b7fe640 1 --2- 172.21.15.112:0/2342989972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f101c102570 0x7f101c1131f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f100c000c00 tx=0x7f100c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:49:24.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.141+0000 7f1003fff640 1 -- 172.21.15.112:0/2342989972 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f100c038070 con 0x7f101c102570 2024-09-16T00:49:24.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.141+0000 7f1003fff640 1 -- 172.21.15.112:0/2342989972 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f100c02ade0 con 0x7f101c102570 2024-09-16T00:49:24.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.141+0000 7f1021ef7640 1 -- 172.21.15.112:0/2342989972 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f101c10d3f0 con 0x7f101c102570 2024-09-16T00:49:24.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.141+0000 7f1003fff640 1 -- 172.21.15.112:0/2342989972 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f100c02c7e0 con 0x7f101c102570 2024-09-16T00:49:24.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.141+0000 7f1021ef7640 1 -- 172.21.15.112:0/2342989972 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f101c10d930 con 0x7f101c102570 2024-09-16T00:49:24.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.142+0000 7f1021ef7640 1 -- 172.21.15.112:0/2342989972 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f101c0fdcf0 con 0x7f101c102570 2024-09-16T00:49:24.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.142+0000 7f1003fff640 1 -- 172.21.15.112:0/2342989972 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f100c030020 con 0x7f101c102570 2024-09-16T00:49:24.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.143+0000 7f1003fff640 1 --2- 172.21.15.112:0/2342989972 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f100406e530 0x7f10040709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:24.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.143+0000 7f1003fff640 1 -- 172.21.15.112:0/2342989972 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f100c0adf70 con 0x7f101c102570 2024-09-16T00:49:24.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.143+0000 7f101affd640 1 --2- 172.21.15.112:0/2342989972 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f100406e530 0x7f10040709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:24.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.143+0000 7f101affd640 1 --2- 172.21.15.112:0/2342989972 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f100406e530 0x7f10040709e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f101c10eab0 tx=0x7f101000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:49:24.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.147+0000 7f1003fff640 1 -- 172.21.15.112:0/2342989972 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f100c0e31b0 con 0x7f101c102570 2024-09-16T00:49:24.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:23 smithi169 ceph-mon[77626]: pgmap v649: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:49:24.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.320+0000 7f1021ef7640 1 -- 172.21.15.112:0/2342989972 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f101c04b720 con 0x7f100406e530 2024-09-16T00:49:24.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.329+0000 7f1003fff640 1 -- 172.21.15.112:0/2342989972 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7f101c04b720 con 0x7f100406e530 2024-09-16T00:49:24.330 INFO:teuthology.orchestra.run.smithi112.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T00:49:24.330 INFO:teuthology.orchestra.run.smithi112.stdout:alertmanager.smithi112 smithi112 *:9093,9094 running (22m) 15m ago 31m 24.7M - 0.25.0 c8568f914cd2 d85ea0562022 2024-09-16T00:49:24.330 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi112 smithi112 running (19m) 15m ago 31m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c e4fe58f9702a 2024-09-16T00:49:24.330 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi169 smithi169 running (19m) 14m ago 30m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c 3ba2c81d527f 2024-09-16T00:49:24.330 INFO:teuthology.orchestra.run.smithi112.stdout:grafana.smithi112 smithi112 *:3000 running (21m) 15m ago 31m 82.1M - 9.4.12 2bacad6d85d8 51f9d2013da2 2024-09-16T00:49:24.330 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.hxfwkh smithi112 running (15m) 15m ago 25m 14.2M - 19.1.1-326-g8921c8ef f1455bd0219c 1723069902d6 2024-09-16T00:49:24.330 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.ithwzj smithi112 running (15m) 15m ago 25m 12.2M - 19.1.1-326-g8921c8ef f1455bd0219c e7aad5b81db5 2024-09-16T00:49:24.330 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.gawwgw smithi169 running (15m) 14m ago 25m 15.7M - 19.1.1-326-g8921c8ef f1455bd0219c 469be459ee12 2024-09-16T00:49:24.331 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.lgeqrj smithi169 running (14m) 14m ago 25m 13.3M - 19.1.1-326-g8921c8ef f1455bd0219c 31f0fbd2395d 2024-09-16T00:49:24.331 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi112.xrkkbc smithi112 *:8443,9283,8765 running (21m) 15m ago 32m 504M - 19.1.1-326-g8921c8ef f1455bd0219c a981f7fd2810 2024-09-16T00:49:24.331 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi169.mtrobk smithi169 *:8443,9283,8765 running (21m) 14m ago 30m 442M - 19.1.1-326-g8921c8ef f1455bd0219c 4096e5369c2d 2024-09-16T00:49:24.331 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi112 smithi112 running (20m) 15m ago 32m 49.8M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 5aaec033c042 2024-09-16T00:49:24.331 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi169 smithi169 running (19m) 14m ago 30m 38.9M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 7bc609155f3f 2024-09-16T00:49:24.331 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi112 smithi112 *:9100 running (22m) 15m ago 31m 15.5M - 1.5.0 0da6a335fe13 6404d1d601e2 2024-09-16T00:49:24.331 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi169 smithi169 *:9100 running (22m) 14m ago 30m 15.4M - 1.5.0 0da6a335fe13 5505baf702a6 2024-09-16T00:49:24.331 INFO:teuthology.orchestra.run.smithi112.stdout:osd.0 smithi112 running (18m) 15m ago 29m 105M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 53efc9397017 2024-09-16T00:49:24.331 INFO:teuthology.orchestra.run.smithi112.stdout:osd.1 smithi112 running (18m) 15m ago 29m 98.9M 1557M 19.1.1-326-g8921c8ef f1455bd0219c fe6f1a4a2cf4 2024-09-16T00:49:24.331 INFO:teuthology.orchestra.run.smithi112.stdout:osd.2 smithi112 running (17m) 15m ago 28m 102M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 5f7863712057 2024-09-16T00:49:24.331 INFO:teuthology.orchestra.run.smithi112.stdout:osd.3 smithi169 running (17m) 14m ago 28m 92.1M 2581M 19.1.1-326-g8921c8ef f1455bd0219c ee07b5577810 2024-09-16T00:49:24.331 INFO:teuthology.orchestra.run.smithi112.stdout:osd.4 smithi169 running (16m) 14m ago 27m 103M 2581M 19.1.1-326-g8921c8ef f1455bd0219c 769f026a54eb 2024-09-16T00:49:24.332 INFO:teuthology.orchestra.run.smithi112.stdout:osd.5 smithi169 running (16m) 14m ago 27m 125M 2581M 19.1.1-326-g8921c8ef f1455bd0219c a6007c13cf78 2024-09-16T00:49:24.332 INFO:teuthology.orchestra.run.smithi112.stdout:prometheus.smithi112 smithi112 *:9095 running (21m) 15m ago 30m 64.1M - 2.43.0 a07b618ecd1d 7a1d42e60d2b 2024-09-16T00:49:24.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.332+0000 7f1021ef7640 1 -- 172.21.15.112:0/2342989972 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f100406e530 msgr2=0x7f10040709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:24.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.332+0000 7f1021ef7640 1 --2- 172.21.15.112:0/2342989972 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f100406e530 0x7f10040709e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f101c10eab0 tx=0x7f101000b040 comp rx=0 tx=0).stop 2024-09-16T00:49:24.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.332+0000 7f1021ef7640 1 -- 172.21.15.112:0/2342989972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f101c102570 msgr2=0x7f101c1131f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:24.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.332+0000 7f1021ef7640 1 --2- 172.21.15.112:0/2342989972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f101c102570 0x7f101c1131f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f100c000c00 tx=0x7f100c009da0 comp rx=0 tx=0).stop 2024-09-16T00:49:24.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.332+0000 7f1021ef7640 1 -- 172.21.15.112:0/2342989972 shutdown_connections 2024-09-16T00:49:24.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.332+0000 7f1021ef7640 1 --2- 172.21.15.112:0/2342989972 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f100406e530 0x7f10040709e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:24.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.332+0000 7f1021ef7640 1 --2- 172.21.15.112:0/2342989972 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f101c102f10 0x7f101c113730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:24.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.332+0000 7f1021ef7640 1 --2- 172.21.15.112:0/2342989972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f101c102570 0x7f101c1131f0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:24.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.332+0000 7f1021ef7640 1 -- 172.21.15.112:0/2342989972 >> 172.21.15.112:0/2342989972 conn(0x7f101c0f8110 msgr2=0x7f101c0f90a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:49:24.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.332+0000 7f1021ef7640 1 -- 172.21.15.112:0/2342989972 shutdown_connections 2024-09-16T00:49:24.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.332+0000 7f1021ef7640 1 -- 172.21.15.112:0/2342989972 wait complete. 2024-09-16T00:49:24.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.409+0000 7f5536367640 1 -- 172.21.15.112:0/1640315646 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55300fb810 msgr2=0x7f55300fbbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:24.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.409+0000 7f5536367640 1 --2- 172.21.15.112:0/1640315646 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55300fb810 0x7f55300fbbe0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f55200048d0 tx=0x7f5520009da0 comp rx=0 tx=0).stop 2024-09-16T00:49:24.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.410+0000 7f5536367640 1 -- 172.21.15.112:0/1640315646 shutdown_connections 2024-09-16T00:49:24.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.410+0000 7f5536367640 1 --2- 172.21.15.112:0/1640315646 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55300fcf80 0x7f55300fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:24.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.410+0000 7f5536367640 1 --2- 172.21.15.112:0/1640315646 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55300fb810 0x7f55300fbbe0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:24.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.410+0000 7f5536367640 1 -- 172.21.15.112:0/1640315646 >> 172.21.15.112:0/1640315646 conn(0x7f55300723b0 msgr2=0x7f55300727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:49:24.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.410+0000 7f5536367640 1 -- 172.21.15.112:0/1640315646 shutdown_connections 2024-09-16T00:49:24.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.410+0000 7f5536367640 1 -- 172.21.15.112:0/1640315646 wait complete. 2024-09-16T00:49:24.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.410+0000 7f5536367640 1 Processor -- start 2024-09-16T00:49:24.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.410+0000 7f5536367640 1 -- start start 2024-09-16T00:49:24.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.410+0000 7f5536367640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55300fb810 0x7f5530113200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:24.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.411+0000 7f5536367640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55300fcf80 0x7f5530113740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:24.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.411+0000 7f5536367640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5530113d40 con 0x7f55300fb810 2024-09-16T00:49:24.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.411+0000 7f5536367640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f553010d180 con 0x7f55300fcf80 2024-09-16T00:49:24.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.411+0000 7f552ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55300fb810 0x7f5530113200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:24.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.411+0000 7f552ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55300fb810 0x7f5530113200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49188/0 (socket says 172.21.15.112:49188) 2024-09-16T00:49:24.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.411+0000 7f552ffff640 1 -- 172.21.15.112:0/2883937645 learned_addr learned my addr 172.21.15.112:0/2883937645 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:49:24.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.411+0000 7f552f7fe640 1 --2- 172.21.15.112:0/2883937645 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55300fcf80 0x7f5530113740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:24.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.411+0000 7f552ffff640 1 -- 172.21.15.112:0/2883937645 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55300fcf80 msgr2=0x7f5530113740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:24.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.411+0000 7f552ffff640 1 --2- 172.21.15.112:0/2883937645 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55300fcf80 0x7f5530113740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:24.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.411+0000 7f552ffff640 1 -- 172.21.15.112:0/2883937645 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5520005040 con 0x7f55300fb810 2024-09-16T00:49:24.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.411+0000 7f552f7fe640 1 --2- 172.21.15.112:0/2883937645 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55300fcf80 0x7f5530113740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:49:24.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.411+0000 7f552ffff640 1 --2- 172.21.15.112:0/2883937645 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55300fb810 0x7f5530113200 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f5520000c00 tx=0x7f5520009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:49:24.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.411+0000 7f552cff9640 1 -- 172.21.15.112:0/2883937645 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5520038070 con 0x7f55300fb810 2024-09-16T00:49:24.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.411+0000 7f5536367640 1 -- 172.21.15.112:0/2883937645 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f553010d400 con 0x7f55300fb810 2024-09-16T00:49:24.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.412+0000 7f552cff9640 1 -- 172.21.15.112:0/2883937645 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f552002ade0 con 0x7f55300fb810 2024-09-16T00:49:24.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.412+0000 7f5536367640 1 -- 172.21.15.112:0/2883937645 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f553010d940 con 0x7f55300fb810 2024-09-16T00:49:24.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.412+0000 7f552cff9640 1 -- 172.21.15.112:0/2883937645 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f552002c7e0 con 0x7f55300fb810 2024-09-16T00:49:24.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.413+0000 7f5536367640 1 -- 172.21.15.112:0/2883937645 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55300fcbc0 con 0x7f55300fb810 2024-09-16T00:49:24.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.413+0000 7f552cff9640 1 -- 172.21.15.112:0/2883937645 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f5520030020 con 0x7f55300fb810 2024-09-16T00:49:24.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.414+0000 7f552cff9640 1 --2- 172.21.15.112:0/2883937645 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f551c06e530 0x7f551c0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:24.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.414+0000 7f552cff9640 1 -- 172.21.15.112:0/2883937645 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f55200adec0 con 0x7f55300fb810 2024-09-16T00:49:24.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.414+0000 7f552f7fe640 1 --2- 172.21.15.112:0/2883937645 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f551c06e530 0x7f551c0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:24.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.414+0000 7f552f7fe640 1 --2- 172.21.15.112:0/2883937645 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f551c06e530 0x7f551c0709e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5518007fd0 tx=0x7f5518009e90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:49:24.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.418+0000 7f552cff9640 1 -- 172.21.15.112:0/2883937645 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f5520076f60 con 0x7f55300fb810 2024-09-16T00:49:24.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.648+0000 7f5536367640 1 -- 172.21.15.112:0/2883937645 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f553004b750 con 0x7f55300fb810 2024-09-16T00:49:24.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.648+0000 7f552cff9640 1 -- 172.21.15.112:0/2883937645 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f552007a620 con 0x7f55300fb810 2024-09-16T00:49:24.649 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:49:24.649 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": { 2024-09-16T00:49:24.649 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:49:24.649 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:49:24.649 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": { 2024-09-16T00:49:24.649 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:49:24.649 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:49:24.650 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": { 2024-09-16T00:49:24.650 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 6 2024-09-16T00:49:24.650 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:49:24.650 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": { 2024-09-16T00:49:24.650 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 4 2024-09-16T00:49:24.650 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:49:24.650 INFO:teuthology.orchestra.run.smithi112.stdout: "overall": { 2024-09-16T00:49:24.650 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 14 2024-09-16T00:49:24.650 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-16T00:49:24.650 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:49:24.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.651+0000 7f5536367640 1 -- 172.21.15.112:0/2883937645 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f551c06e530 msgr2=0x7f551c0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:24.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.651+0000 7f5536367640 1 --2- 172.21.15.112:0/2883937645 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f551c06e530 0x7f551c0709e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5518007fd0 tx=0x7f5518009e90 comp rx=0 tx=0).stop 2024-09-16T00:49:24.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.651+0000 7f5536367640 1 -- 172.21.15.112:0/2883937645 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55300fb810 msgr2=0x7f5530113200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:24.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.651+0000 7f5536367640 1 --2- 172.21.15.112:0/2883937645 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55300fb810 0x7f5530113200 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f5520000c00 tx=0x7f5520009da0 comp rx=0 tx=0).stop 2024-09-16T00:49:24.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.651+0000 7f5536367640 1 -- 172.21.15.112:0/2883937645 shutdown_connections 2024-09-16T00:49:24.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.651+0000 7f5536367640 1 --2- 172.21.15.112:0/2883937645 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f551c06e530 0x7f551c0709e0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:24.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.651+0000 7f5536367640 1 --2- 172.21.15.112:0/2883937645 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55300fcf80 0x7f5530113740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:24.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.651+0000 7f5536367640 1 --2- 172.21.15.112:0/2883937645 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55300fb810 0x7f5530113200 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:24.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.651+0000 7f5536367640 1 -- 172.21.15.112:0/2883937645 >> 172.21.15.112:0/2883937645 conn(0x7f55300723b0 msgr2=0x7f5530064f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:49:24.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.651+0000 7f5536367640 1 -- 172.21.15.112:0/2883937645 shutdown_connections 2024-09-16T00:49:24.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.651+0000 7f5536367640 1 -- 172.21.15.112:0/2883937645 wait complete. 2024-09-16T00:49:24.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.732+0000 7f4128ac9640 1 -- 172.21.15.112:0/3231486765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41240facf0 msgr2=0x7f41240fb160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:24.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.732+0000 7f4128ac9640 1 --2- 172.21.15.112:0/3231486765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41240facf0 0x7f41240fb160 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f410c0048d0 tx=0x7f410c009da0 comp rx=0 tx=0).stop 2024-09-16T00:49:24.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.732+0000 7f4128ac9640 1 -- 172.21.15.112:0/3231486765 shutdown_connections 2024-09-16T00:49:24.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.732+0000 7f4128ac9640 1 --2- 172.21.15.112:0/3231486765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41240facf0 0x7f41240fb160 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:24.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.732+0000 7f4128ac9640 1 --2- 172.21.15.112:0/3231486765 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4124102670 0x7f4124102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:24.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.732+0000 7f4128ac9640 1 -- 172.21.15.112:0/3231486765 >> 172.21.15.112:0/3231486765 conn(0x7f4124070d50 msgr2=0x7f4124071150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:49:24.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.732+0000 7f4128ac9640 1 -- 172.21.15.112:0/3231486765 shutdown_connections 2024-09-16T00:49:24.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.732+0000 7f4128ac9640 1 -- 172.21.15.112:0/3231486765 wait complete. 2024-09-16T00:49:24.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.733+0000 7f4128ac9640 1 Processor -- start 2024-09-16T00:49:24.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.733+0000 7f4128ac9640 1 -- start start 2024-09-16T00:49:24.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.733+0000 7f4128ac9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41240facf0 0x7f41241131d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:24.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.733+0000 7f4128ac9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4124102670 0x7f4124113710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:24.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.733+0000 7f4128ac9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4124113d10 con 0x7f41240facf0 2024-09-16T00:49:24.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.733+0000 7f4128ac9640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f412410d150 con 0x7f4124102670 2024-09-16T00:49:24.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.733+0000 7f412259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41240facf0 0x7f41241131d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:24.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.733+0000 7f412259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41240facf0 0x7f41241131d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49206/0 (socket says 172.21.15.112:49206) 2024-09-16T00:49:24.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.733+0000 7f412259c640 1 -- 172.21.15.112:0/1461915395 learned_addr learned my addr 172.21.15.112:0/1461915395 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:49:24.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.733+0000 7f411bfff640 1 --2- 172.21.15.112:0/1461915395 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4124102670 0x7f4124113710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:24.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.733+0000 7f412259c640 1 -- 172.21.15.112:0/1461915395 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4124102670 msgr2=0x7f4124113710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:24.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.733+0000 7f412259c640 1 --2- 172.21.15.112:0/1461915395 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4124102670 0x7f4124113710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:24.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.733+0000 7f412259c640 1 -- 172.21.15.112:0/1461915395 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f410c005040 con 0x7f41240facf0 2024-09-16T00:49:24.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.734+0000 7f412259c640 1 --2- 172.21.15.112:0/1461915395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41240facf0 0x7f41241131d0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f411400c7e0 tx=0x7f41140043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:49:24.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.734+0000 7f411affd640 1 -- 172.21.15.112:0/1461915395 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f411400a820 con 0x7f41240facf0 2024-09-16T00:49:24.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.734+0000 7f411affd640 1 -- 172.21.15.112:0/1461915395 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f411400ae20 con 0x7f41240facf0 2024-09-16T00:49:24.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.734+0000 7f4128ac9640 1 -- 172.21.15.112:0/1461915395 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f412410d430 con 0x7f41240facf0 2024-09-16T00:49:24.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.734+0000 7f411affd640 1 -- 172.21.15.112:0/1461915395 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41140113a0 con 0x7f41240facf0 2024-09-16T00:49:24.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.734+0000 7f4128ac9640 1 -- 172.21.15.112:0/1461915395 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f412410d9a0 con 0x7f41240facf0 2024-09-16T00:49:24.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.735+0000 7f4128ac9640 1 -- 172.21.15.112:0/1461915395 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41240657b0 con 0x7f41240facf0 2024-09-16T00:49:24.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.736+0000 7f411affd640 1 -- 172.21.15.112:0/1461915395 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f4114012400 con 0x7f41240facf0 2024-09-16T00:49:24.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.736+0000 7f411affd640 1 --2- 172.21.15.112:0/1461915395 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f411006e530 0x7f41100709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:24.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.736+0000 7f411affd640 1 -- 172.21.15.112:0/1461915395 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f411408e450 con 0x7f41240facf0 2024-09-16T00:49:24.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.736+0000 7f411bfff640 1 --2- 172.21.15.112:0/1461915395 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f411006e530 0x7f41100709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:24.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.737+0000 7f411bfff640 1 --2- 172.21.15.112:0/1461915395 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f411006e530 0x7f41100709e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4124106dc0 tx=0x7f410c035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:49:24.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.740+0000 7f411affd640 1 -- 172.21.15.112:0/1461915395 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f4114057420 con 0x7f41240facf0 2024-09-16T00:49:24.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.944+0000 7f4128ac9640 1 -- 172.21.15.112:0/1461915395 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f41240ff8d0 con 0x7f41240facf0 2024-09-16T00:49:24.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.944+0000 7f411affd640 1 -- 172.21.15.112:0/1461915395 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1914 (secure 0 0 0) 0x7f411405aae0 con 0x7f41240facf0 2024-09-16T00:49:24.944 INFO:teuthology.orchestra.run.smithi112.stdout:e34 2024-09-16T00:49:24.945 INFO:teuthology.orchestra.run.smithi112.stdout:btime 2024-09-16T00:34:26:619254+0000 2024-09-16T00:49:24.945 INFO:teuthology.orchestra.run.smithi112.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T00:49:24.945 INFO:teuthology.orchestra.run.smithi112.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:49:24.945 INFO:teuthology.orchestra.run.smithi112.stdout:legacy client fscid: 1 2024-09-16T00:49:24.945 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:49:24.945 INFO:teuthology.orchestra.run.smithi112.stdout:Filesystem 'cephfs' (1) 2024-09-16T00:49:24.945 INFO:teuthology.orchestra.run.smithi112.stdout:fs_name cephfs 2024-09-16T00:49:24.945 INFO:teuthology.orchestra.run.smithi112.stdout:epoch 25 2024-09-16T00:49:24.945 INFO:teuthology.orchestra.run.smithi112.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-16T00:49:24.945 INFO:teuthology.orchestra.run.smithi112.stdout:created 2024-09-16T00:23:22.328199+0000 2024-09-16T00:49:24.945 INFO:teuthology.orchestra.run.smithi112.stdout:modified 2024-09-16T00:33:29.012991+0000 2024-09-16T00:49:24.945 INFO:teuthology.orchestra.run.smithi112.stdout:tableserver 0 2024-09-16T00:49:24.945 INFO:teuthology.orchestra.run.smithi112.stdout:root 0 2024-09-16T00:49:24.946 INFO:teuthology.orchestra.run.smithi112.stdout:session_timeout 60 2024-09-16T00:49:24.946 INFO:teuthology.orchestra.run.smithi112.stdout:session_autoclose 300 2024-09-16T00:49:24.946 INFO:teuthology.orchestra.run.smithi112.stdout:max_file_size 1099511627776 2024-09-16T00:49:24.946 INFO:teuthology.orchestra.run.smithi112.stdout:max_xattr_size 65536 2024-09-16T00:49:24.946 INFO:teuthology.orchestra.run.smithi112.stdout:required_client_features {} 2024-09-16T00:49:24.946 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure 0 2024-09-16T00:49:24.946 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure_osd_epoch 82 2024-09-16T00:49:24.946 INFO:teuthology.orchestra.run.smithi112.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:49:24.946 INFO:teuthology.orchestra.run.smithi112.stdout:max_mds 2 2024-09-16T00:49:24.946 INFO:teuthology.orchestra.run.smithi112.stdout:in 0,1 2024-09-16T00:49:24.946 INFO:teuthology.orchestra.run.smithi112.stdout:up {} 2024-09-16T00:49:24.946 INFO:teuthology.orchestra.run.smithi112.stdout:failed 0,1 2024-09-16T00:49:24.946 INFO:teuthology.orchestra.run.smithi112.stdout:damaged 2024-09-16T00:49:24.946 INFO:teuthology.orchestra.run.smithi112.stdout:stopped 2024-09-16T00:49:24.947 INFO:teuthology.orchestra.run.smithi112.stdout:data_pools [3] 2024-09-16T00:49:24.947 INFO:teuthology.orchestra.run.smithi112.stdout:metadata_pool 2 2024-09-16T00:49:24.947 INFO:teuthology.orchestra.run.smithi112.stdout:inline_data enabled 2024-09-16T00:49:24.947 INFO:teuthology.orchestra.run.smithi112.stdout:balancer 2024-09-16T00:49:24.947 INFO:teuthology.orchestra.run.smithi112.stdout:bal_rank_mask -1 2024-09-16T00:49:24.947 INFO:teuthology.orchestra.run.smithi112.stdout:standby_count_wanted 1 2024-09-16T00:49:24.947 INFO:teuthology.orchestra.run.smithi112.stdout:qdb_cluster leader: 0 members: 2024-09-16T00:49:24.947 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:49:24.947 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:49:24.947 INFO:teuthology.orchestra.run.smithi112.stdout:Standby daemons: 2024-09-16T00:49:24.947 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:49:24.947 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.ithwzj{-1:34392} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.112:6826/476229221,v1:172.21.15.112:6827/476229221] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:49:24.947 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.hxfwkh{-1:34404} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.112:6828/2356899876,v1:172.21.15.112:6829/2356899876] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:49:24.947 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.gawwgw{-1:44343} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.169:6824/1635329957,v1:172.21.15.169:6825/1635329957] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:49:24.948 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.lgeqrj{-1:44347} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.169:6826/184810031,v1:172.21.15.169:6827/184810031] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:49:24.948 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 34 2024-09-16T00:49:24.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.946+0000 7f4128ac9640 1 -- 172.21.15.112:0/1461915395 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f411006e530 msgr2=0x7f41100709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:24.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.946+0000 7f4128ac9640 1 --2- 172.21.15.112:0/1461915395 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f411006e530 0x7f41100709e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4124106dc0 tx=0x7f410c035040 comp rx=0 tx=0).stop 2024-09-16T00:49:24.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.946+0000 7f4128ac9640 1 -- 172.21.15.112:0/1461915395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41240facf0 msgr2=0x7f41241131d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:24.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.946+0000 7f4128ac9640 1 --2- 172.21.15.112:0/1461915395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41240facf0 0x7f41241131d0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f411400c7e0 tx=0x7f41140043c0 comp rx=0 tx=0).stop 2024-09-16T00:49:24.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.946+0000 7f4128ac9640 1 -- 172.21.15.112:0/1461915395 shutdown_connections 2024-09-16T00:49:24.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.946+0000 7f4128ac9640 1 --2- 172.21.15.112:0/1461915395 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f411006e530 0x7f41100709e0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:24.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.946+0000 7f4128ac9640 1 --2- 172.21.15.112:0/1461915395 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4124102670 0x7f4124113710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:24.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.946+0000 7f4128ac9640 1 --2- 172.21.15.112:0/1461915395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41240facf0 0x7f41241131d0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:24.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.946+0000 7f4128ac9640 1 -- 172.21.15.112:0/1461915395 >> 172.21.15.112:0/1461915395 conn(0x7f4124070d50 msgr2=0x7f41240fa040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:49:24.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.946+0000 7f4128ac9640 1 -- 172.21.15.112:0/1461915395 shutdown_connections 2024-09-16T00:49:24.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:24.946+0000 7f4128ac9640 1 -- 172.21.15.112:0/1461915395 wait complete. 2024-09-16T00:49:25.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.025+0000 7ff19c131640 1 -- 172.21.15.112:0/1774610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1941014b0 msgr2=0x7ff194101880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:25.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.025+0000 7ff19c131640 1 --2- 172.21.15.112:0/1774610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1941014b0 0x7ff194101880 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7ff1900048d0 tx=0x7ff190009da0 comp rx=0 tx=0).stop 2024-09-16T00:49:25.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.025+0000 7ff19c131640 1 -- 172.21.15.112:0/1774610 shutdown_connections 2024-09-16T00:49:25.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.025+0000 7ff19c131640 1 --2- 172.21.15.112:0/1774610 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1940fb470 0x7ff1940fb8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:25.029 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:24 smithi112 ceph-mon[97750]: from='client.35220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:49:25.029 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:24 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/2883937645' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:49:25.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.025+0000 7ff19c131640 1 --2- 172.21.15.112:0/1774610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1941014b0 0x7ff194101880 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:25.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.025+0000 7ff19c131640 1 -- 172.21.15.112:0/1774610 >> 172.21.15.112:0/1774610 conn(0x7ff1940f7030 msgr2=0x7ff1940f9440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:49:25.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.025+0000 7ff19c131640 1 -- 172.21.15.112:0/1774610 shutdown_connections 2024-09-16T00:49:25.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.025+0000 7ff19c131640 1 -- 172.21.15.112:0/1774610 wait complete. 2024-09-16T00:49:25.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.025+0000 7ff19c131640 1 Processor -- start 2024-09-16T00:49:25.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.026+0000 7ff19c131640 1 -- start start 2024-09-16T00:49:25.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.026+0000 7ff19c131640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1940fb470 0x7ff19410e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:25.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.026+0000 7ff19c131640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1941014b0 0x7ff19410e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:25.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.026+0000 7ff19c131640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff19410ed10 con 0x7ff1940fb470 2024-09-16T00:49:25.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.026+0000 7ff19c131640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff19410ee80 con 0x7ff1941014b0 2024-09-16T00:49:25.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.026+0000 7ff199ecd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1940fb470 0x7ff19410e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:25.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.026+0000 7ff199ecd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1940fb470 0x7ff19410e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49226/0 (socket says 172.21.15.112:49226) 2024-09-16T00:49:25.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.026+0000 7ff199ecd640 1 -- 172.21.15.112:0/584714428 learned_addr learned my addr 172.21.15.112:0/584714428 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:49:25.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.026+0000 7ff1996cc640 1 --2- 172.21.15.112:0/584714428 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1941014b0 0x7ff19410e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:25.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.026+0000 7ff199ecd640 1 -- 172.21.15.112:0/584714428 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1941014b0 msgr2=0x7ff19410e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:25.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.026+0000 7ff199ecd640 1 --2- 172.21.15.112:0/584714428 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1941014b0 0x7ff19410e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:25.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.026+0000 7ff199ecd640 1 -- 172.21.15.112:0/584714428 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff190005040 con 0x7ff1940fb470 2024-09-16T00:49:25.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.027+0000 7ff199ecd640 1 --2- 172.21.15.112:0/584714428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1940fb470 0x7ff19410e180 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7ff190000c00 tx=0x7ff190009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:49:25.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.027+0000 7ff18a7fc640 1 -- 172.21.15.112:0/584714428 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff190038070 con 0x7ff1940fb470 2024-09-16T00:49:25.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.027+0000 7ff18a7fc640 1 -- 172.21.15.112:0/584714428 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7ff19002ade0 con 0x7ff1940fb470 2024-09-16T00:49:25.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.027+0000 7ff18a7fc640 1 -- 172.21.15.112:0/584714428 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff19002c7e0 con 0x7ff1940fb470 2024-09-16T00:49:25.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.027+0000 7ff19c131640 1 -- 172.21.15.112:0/584714428 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1941171f0 con 0x7ff1940fb470 2024-09-16T00:49:25.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.027+0000 7ff19c131640 1 -- 172.21.15.112:0/584714428 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff194117650 con 0x7ff1940fb470 2024-09-16T00:49:25.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.028+0000 7ff19c131640 1 -- 172.21.15.112:0/584714428 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1940fcbb0 con 0x7ff1940fb470 2024-09-16T00:49:25.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.029+0000 7ff18a7fc640 1 -- 172.21.15.112:0/584714428 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7ff190030020 con 0x7ff1940fb470 2024-09-16T00:49:25.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.029+0000 7ff18a7fc640 1 --2- 172.21.15.112:0/584714428 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff18006e530 0x7ff1800709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:25.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.030+0000 7ff18a7fc640 1 -- 172.21.15.112:0/584714428 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7ff1900aded0 con 0x7ff1940fb470 2024-09-16T00:49:25.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.030+0000 7ff1996cc640 1 --2- 172.21.15.112:0/584714428 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff18006e530 0x7ff1800709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:25.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.030+0000 7ff1996cc640 1 --2- 172.21.15.112:0/584714428 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff18006e530 0x7ff1800709e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff194066210 tx=0x7ff18400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:49:25.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.034+0000 7ff18a7fc640 1 -- 172.21.15.112:0/584714428 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7ff190076f70 con 0x7ff1940fb470 2024-09-16T00:49:25.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.222+0000 7ff19c131640 1 -- 172.21.15.112:0/584714428 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff194065fc0 con 0x7ff18006e530 2024-09-16T00:49:25.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.223+0000 7ff18a7fc640 1 -- 172.21.15.112:0/584714428 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7ff194065fc0 con 0x7ff18006e530 2024-09-16T00:49:25.223 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:49:25.223 INFO:teuthology.orchestra.run.smithi112.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80", 2024-09-16T00:49:25.223 INFO:teuthology.orchestra.run.smithi112.stdout: "in_progress": true, 2024-09-16T00:49:25.224 INFO:teuthology.orchestra.run.smithi112.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T00:49:25.224 INFO:teuthology.orchestra.run.smithi112.stdout: "services_complete": [ 2024-09-16T00:49:25.224 INFO:teuthology.orchestra.run.smithi112.stdout: "mds", 2024-09-16T00:49:25.224 INFO:teuthology.orchestra.run.smithi112.stdout: "osd", 2024-09-16T00:49:25.224 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr", 2024-09-16T00:49:25.224 INFO:teuthology.orchestra.run.smithi112.stdout: "mon", 2024-09-16T00:49:25.224 INFO:teuthology.orchestra.run.smithi112.stdout: "crash" 2024-09-16T00:49:25.224 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-16T00:49:25.224 INFO:teuthology.orchestra.run.smithi112.stdout: "progress": "16/21 daemons upgraded", 2024-09-16T00:49:25.224 INFO:teuthology.orchestra.run.smithi112.stdout: "message": "Currently upgrading mds daemons", 2024-09-16T00:49:25.224 INFO:teuthology.orchestra.run.smithi112.stdout: "is_paused": false 2024-09-16T00:49:25.224 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:49:25.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.225+0000 7ff19c131640 1 -- 172.21.15.112:0/584714428 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff18006e530 msgr2=0x7ff1800709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:25.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.225+0000 7ff19c131640 1 --2- 172.21.15.112:0/584714428 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff18006e530 0x7ff1800709e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff194066210 tx=0x7ff18400b040 comp rx=0 tx=0).stop 2024-09-16T00:49:25.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.225+0000 7ff19c131640 1 -- 172.21.15.112:0/584714428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1940fb470 msgr2=0x7ff19410e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:25.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.225+0000 7ff19c131640 1 --2- 172.21.15.112:0/584714428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1940fb470 0x7ff19410e180 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7ff190000c00 tx=0x7ff190009da0 comp rx=0 tx=0).stop 2024-09-16T00:49:25.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.225+0000 7ff19c131640 1 -- 172.21.15.112:0/584714428 shutdown_connections 2024-09-16T00:49:25.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.225+0000 7ff19c131640 1 --2- 172.21.15.112:0/584714428 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff18006e530 0x7ff1800709e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:25.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.225+0000 7ff19c131640 1 --2- 172.21.15.112:0/584714428 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1941014b0 0x7ff19410e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:25.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.225+0000 7ff19c131640 1 --2- 172.21.15.112:0/584714428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1940fb470 0x7ff19410e180 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:25.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.225+0000 7ff19c131640 1 -- 172.21.15.112:0/584714428 >> 172.21.15.112:0/584714428 conn(0x7ff1940f7030 msgr2=0x7ff1940f88e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:49:25.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.225+0000 7ff19c131640 1 -- 172.21.15.112:0/584714428 shutdown_connections 2024-09-16T00:49:25.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.225+0000 7ff19c131640 1 -- 172.21.15.112:0/584714428 wait complete. 2024-09-16T00:49:25.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:24 smithi169 ceph-mon[77626]: from='client.35220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:49:25.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:24 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/2883937645' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:49:25.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.305+0000 7f19694c1640 1 -- 172.21.15.112:0/171220474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19640fd580 msgr2=0x7f19640fd950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:25.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.305+0000 7f19694c1640 1 --2- 172.21.15.112:0/171220474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19640fd580 0x7f19640fd950 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f195c009c80 tx=0x7f195c004d10 comp rx=0 tx=0).stop 2024-09-16T00:49:25.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.305+0000 7f19694c1640 1 -- 172.21.15.112:0/171220474 shutdown_connections 2024-09-16T00:49:25.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.305+0000 7f19694c1640 1 --2- 172.21.15.112:0/171220474 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19640fdf20 0x7f19641065f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:25.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.305+0000 7f19694c1640 1 --2- 172.21.15.112:0/171220474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19640fd580 0x7f19640fd950 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:25.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.305+0000 7f19694c1640 1 -- 172.21.15.112:0/171220474 >> 172.21.15.112:0/171220474 conn(0x7f19640f7060 msgr2=0x7f19640f9470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:49:25.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.306+0000 7f19694c1640 1 -- 172.21.15.112:0/171220474 shutdown_connections 2024-09-16T00:49:25.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.306+0000 7f19694c1640 1 -- 172.21.15.112:0/171220474 wait complete. 2024-09-16T00:49:25.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.306+0000 7f19694c1640 1 Processor -- start 2024-09-16T00:49:25.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.306+0000 7f19694c1640 1 -- start start 2024-09-16T00:49:25.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.307+0000 7f19694c1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19640fd580 0x7f1964113240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:25.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.307+0000 7f19694c1640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19640fdf20 0x7f1964113780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:25.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.307+0000 7f19694c1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1964113d80 con 0x7f19640fd580 2024-09-16T00:49:25.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.307+0000 7f19694c1640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196410d1c0 con 0x7f19640fdf20 2024-09-16T00:49:25.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.307+0000 7f1962ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19640fd580 0x7f1964113240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:25.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.307+0000 7f1962ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19640fd580 0x7f1964113240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49244/0 (socket says 172.21.15.112:49244) 2024-09-16T00:49:25.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.307+0000 7f1962ffd640 1 -- 172.21.15.112:0/3581038479 learned_addr learned my addr 172.21.15.112:0/3581038479 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:49:25.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.307+0000 7f19627fc640 1 --2- 172.21.15.112:0/3581038479 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19640fdf20 0x7f1964113780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:25.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.307+0000 7f1962ffd640 1 -- 172.21.15.112:0/3581038479 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19640fdf20 msgr2=0x7f1964113780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:25.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.307+0000 7f1962ffd640 1 --2- 172.21.15.112:0/3581038479 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19640fdf20 0x7f1964113780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:25.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.307+0000 7f1962ffd640 1 -- 172.21.15.112:0/3581038479 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f195c0098b0 con 0x7f19640fd580 2024-09-16T00:49:25.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.307+0000 7f1962ffd640 1 --2- 172.21.15.112:0/3581038479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19640fd580 0x7f1964113240 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f195c004750 tx=0x7f195c004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:49:25.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.308+0000 7f194b7fe640 1 -- 172.21.15.112:0/3581038479 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f195c008ea0 con 0x7f19640fd580 2024-09-16T00:49:25.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.308+0000 7f194b7fe640 1 -- 172.21.15.112:0/3581038479 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f195c032d20 con 0x7f19640fd580 2024-09-16T00:49:25.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.308+0000 7f19694c1640 1 -- 172.21.15.112:0/3581038479 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f196410d440 con 0x7f19640fd580 2024-09-16T00:49:25.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.308+0000 7f194b7fe640 1 -- 172.21.15.112:0/3581038479 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f195c03dc20 con 0x7f19640fd580 2024-09-16T00:49:25.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.308+0000 7f19694c1640 1 -- 172.21.15.112:0/3581038479 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f196410d960 con 0x7f19640fd580 2024-09-16T00:49:25.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.309+0000 7f19694c1640 1 -- 172.21.15.112:0/3581038479 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19640662d0 con 0x7f19640fd580 2024-09-16T00:49:25.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.309+0000 7f194b7fe640 1 -- 172.21.15.112:0/3581038479 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f195c02f030 con 0x7f19640fd580 2024-09-16T00:49:25.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.310+0000 7f194b7fe640 1 --2- 172.21.15.112:0/3581038479 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f194c06e530 0x7f194c0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:25.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.310+0000 7f194b7fe640 1 -- 172.21.15.112:0/3581038479 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f195c0af540 con 0x7f19640fd580 2024-09-16T00:49:25.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.310+0000 7f19627fc640 1 --2- 172.21.15.112:0/3581038479 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f194c06e530 0x7f194c0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:25.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.310+0000 7f19627fc640 1 --2- 172.21.15.112:0/3581038479 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f194c06e530 0x7f194c0709e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f196410eb00 tx=0x7f195800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:49:25.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.314+0000 7f194b7fe640 1 -- 172.21.15.112:0/3581038479 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f195c078530 con 0x7f19640fd580 2024-09-16T00:49:25.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.593+0000 7f19694c1640 1 -- 172.21.15.112:0/3581038479 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f196404b7b0 con 0x7f19640fd580 2024-09-16T00:49:25.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.593+0000 7f194b7fe640 1 -- 172.21.15.112:0/3581038479 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+297 (secure 0 0 0) 0x7f195c07bbf0 con 0x7f19640fd580 2024-09-16T00:49:25.594 INFO:teuthology.orchestra.run.smithi112.stdout:HEALTH_WARN 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data 2024-09-16T00:49:25.594 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-16T00:49:25.594 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs is degraded 2024-09-16T00:49:25.594 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T00:49:25.594 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T00:49:25.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.595+0000 7f19694c1640 1 -- 172.21.15.112:0/3581038479 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f194c06e530 msgr2=0x7f194c0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:25.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.595+0000 7f19694c1640 1 --2- 172.21.15.112:0/3581038479 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f194c06e530 0x7f194c0709e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f196410eb00 tx=0x7f195800b040 comp rx=0 tx=0).stop 2024-09-16T00:49:25.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.595+0000 7f19694c1640 1 -- 172.21.15.112:0/3581038479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19640fd580 msgr2=0x7f1964113240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:25.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.595+0000 7f19694c1640 1 --2- 172.21.15.112:0/3581038479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19640fd580 0x7f1964113240 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f195c004750 tx=0x7f195c004d10 comp rx=0 tx=0).stop 2024-09-16T00:49:25.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.595+0000 7f19694c1640 1 -- 172.21.15.112:0/3581038479 shutdown_connections 2024-09-16T00:49:25.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.595+0000 7f19694c1640 1 --2- 172.21.15.112:0/3581038479 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f194c06e530 0x7f194c0709e0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:25.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.596+0000 7f19694c1640 1 --2- 172.21.15.112:0/3581038479 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19640fdf20 0x7f1964113780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:25.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.596+0000 7f19694c1640 1 --2- 172.21.15.112:0/3581038479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19640fd580 0x7f1964113240 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:25.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.596+0000 7f19694c1640 1 -- 172.21.15.112:0/3581038479 >> 172.21.15.112:0/3581038479 conn(0x7f19640f7060 msgr2=0x7f19640f85d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:49:25.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.596+0000 7f19694c1640 1 -- 172.21.15.112:0/3581038479 shutdown_connections 2024-09-16T00:49:25.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:25.596+0000 7f19694c1640 1 -- 172.21.15.112:0/3581038479 wait complete. 2024-09-16T00:49:26.178 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:25 smithi169 ceph-mon[77626]: from='client.35224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:49:26.178 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:25 smithi169 ceph-mon[77626]: from='client.35228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:49:26.179 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:25 smithi169 ceph-mon[77626]: pgmap v650: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:49:26.179 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:25 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/1461915395' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:49:26.179 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:25 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:49:26.179 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:25 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:49:26.179 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:25 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:49:26.179 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:25 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:49:26.179 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:25 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:49:26.179 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:25 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:49:26.179 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:25 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:49:26.179 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:25 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:49:26.179 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:25 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:49:26.180 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:25 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:49:26.180 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:25 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi112.hxfwkh"}]: dispatch 2024-09-16T00:49:26.180 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:25 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi112.hxfwkh"}]': finished 2024-09-16T00:49:26.180 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:25 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi112.ithwzj"}]: dispatch 2024-09-16T00:49:26.180 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:25 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi112.ithwzj"}]': finished 2024-09-16T00:49:26.180 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:25 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi169.gawwgw"}]: dispatch 2024-09-16T00:49:26.180 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:25 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi169.gawwgw"}]': finished 2024-09-16T00:49:26.181 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:25 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi169.lgeqrj"}]: dispatch 2024-09-16T00:49:26.181 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:25 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi169.lgeqrj"}]': finished 2024-09-16T00:49:26.181 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:25 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-09-16T00:49:26.181 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:25 smithi169 ceph-mon[77626]: Standby daemon mds.cephfs.smithi112.ithwzj assigned to filesystem cephfs as rank 0 2024-09-16T00:49:26.181 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:25 smithi169 ceph-mon[77626]: Standby daemon mds.cephfs.smithi112.hxfwkh assigned to filesystem cephfs as rank 1 2024-09-16T00:49:26.181 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:25 smithi169 ceph-mon[77626]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-16T00:49:26.181 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:25 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-09-16T00:49:26.182 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:25 smithi169 ceph-mon[77626]: fsmap cephfs:2/2 {0=cephfs.smithi112.ithwzj=up:replay,1=cephfs.smithi112.hxfwkh=up:replay} 2 up:standby 2024-09-16T00:49:26.182 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:25 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]: dispatch 2024-09-16T00:49:26.182 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:25 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/3581038479' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:49:26.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:25 smithi112 ceph-mon[97750]: from='client.35224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:49:26.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:25 smithi112 ceph-mon[97750]: from='client.35228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:49:26.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:25 smithi112 ceph-mon[97750]: pgmap v650: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:49:26.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:25 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/1461915395' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:49:26.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:25 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:49:26.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:25 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:49:26.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:25 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:49:26.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:25 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:49:26.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:25 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:49:26.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:25 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:49:26.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:25 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:49:26.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:25 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:49:26.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:25 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:49:26.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:25 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:49:26.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:25 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi112.hxfwkh"}]: dispatch 2024-09-16T00:49:26.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:25 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi112.hxfwkh"}]': finished 2024-09-16T00:49:26.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:25 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi112.ithwzj"}]: dispatch 2024-09-16T00:49:26.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:25 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi112.ithwzj"}]': finished 2024-09-16T00:49:26.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:25 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi169.gawwgw"}]: dispatch 2024-09-16T00:49:26.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:25 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi169.gawwgw"}]': finished 2024-09-16T00:49:26.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:25 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi169.lgeqrj"}]: dispatch 2024-09-16T00:49:26.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:25 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi169.lgeqrj"}]': finished 2024-09-16T00:49:26.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:25 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-09-16T00:49:26.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:25 smithi112 ceph-mon[97750]: Standby daemon mds.cephfs.smithi112.ithwzj assigned to filesystem cephfs as rank 0 2024-09-16T00:49:26.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:25 smithi112 ceph-mon[97750]: Standby daemon mds.cephfs.smithi112.hxfwkh assigned to filesystem cephfs as rank 1 2024-09-16T00:49:26.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:25 smithi112 ceph-mon[97750]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-16T00:49:26.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:25 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-09-16T00:49:26.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:25 smithi112 ceph-mon[97750]: fsmap cephfs:2/2 {0=cephfs.smithi112.ithwzj=up:replay,1=cephfs.smithi112.hxfwkh=up:replay} 2 up:standby 2024-09-16T00:49:26.342 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:25 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]: dispatch 2024-09-16T00:49:26.342 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:25 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/3581038479' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:49:27.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:26 smithi169 ceph-mon[77626]: pgmap v651: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:49:27.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:26 smithi169 ceph-mon[77626]: from='client.35240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:49:27.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:26 smithi169 ceph-mon[77626]: Upgrade: Setting container_image for all mds 2024-09-16T00:49:27.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:26 smithi169 ceph-mon[77626]: Upgrade: Setting filesystem cephfs Joinable 2024-09-16T00:49:27.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:26 smithi169 ceph-mon[77626]: Upgrade: Enabling allow_standby_replay on filesystem cephfs 2024-09-16T00:49:27.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:26 smithi169 ceph-mon[77626]: Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED) 2024-09-16T00:49:27.278 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:26 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]': finished 2024-09-16T00:49:27.278 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:26 smithi169 ceph-mon[77626]: fsmap cephfs:2/2 {0=cephfs.smithi112.ithwzj=up:replay,1=cephfs.smithi112.hxfwkh=up:replay} 2 up:standby 2024-09-16T00:49:27.278 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:26 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:49:27.278 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:26 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:49:27.278 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:26 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:49:27.278 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:26 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:49:27.278 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:26 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:49:27.278 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:26 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:49:27.278 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:26 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:49:27.278 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:26 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:49:27.279 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:26 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:49:27.279 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:26 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:49:27.279 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:26 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:49:27.279 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:26 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:49:27.279 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:26 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:49:27.279 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:26 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:49:27.279 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:26 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:49:27.279 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:26 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:49:27.279 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:26 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:49:27.279 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:26 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:49:27.279 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:26 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:49:27.279 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:26 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:49:27.279 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:26 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-09-16T00:49:27.279 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:26 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-09-16T00:49:27.280 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:26 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-16T00:49:27.280 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:26 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-09-16T00:49:27.280 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:26 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-09-16T00:49:27.280 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:26 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-09-16T00:49:27.280 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:26 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-09-16T00:49:27.280 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:26 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-09-16T00:49:27.280 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:26 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-09-16T00:49:27.281 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:26 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-09-16T00:49:27.281 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:26 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-09-16T00:49:27.281 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:26 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-09-16T00:49:27.281 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:26 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-09-16T00:49:27.281 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:26 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-09-16T00:49:27.281 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:26 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-16T00:49:27.281 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:26 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-09-16T00:49:27.281 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:26 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-09-16T00:49:27.281 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:26 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-09-16T00:49:27.281 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:26 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-09-16T00:49:27.281 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:26 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-09-16T00:49:27.281 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:26 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-09-16T00:49:27.281 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:26 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-09-16T00:49:27.281 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:26 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-09-16T00:49:27.282 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:26 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-16T00:49:27.282 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:26 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-16T00:49:27.282 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:26 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-16T00:49:27.282 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:26 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-16T00:49:27.282 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:26 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-16T00:49:27.282 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:26 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-16T00:49:27.282 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:26 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-09-16T00:49:27.282 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:26 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-09-16T00:49:27.282 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:26 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:49:27.282 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:26 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:49:27.282 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:26 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:49:27.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:26 smithi112 ceph-mon[97750]: pgmap v651: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T00:49:27.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:26 smithi112 ceph-mon[97750]: from='client.35240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:49:27.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:26 smithi112 ceph-mon[97750]: Upgrade: Setting container_image for all mds 2024-09-16T00:49:27.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:26 smithi112 ceph-mon[97750]: Upgrade: Setting filesystem cephfs Joinable 2024-09-16T00:49:27.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:26 smithi112 ceph-mon[97750]: Upgrade: Enabling allow_standby_replay on filesystem cephfs 2024-09-16T00:49:27.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:26 smithi112 ceph-mon[97750]: Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED) 2024-09-16T00:49:27.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:26 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]': finished 2024-09-16T00:49:27.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:26 smithi112 ceph-mon[97750]: fsmap cephfs:2/2 {0=cephfs.smithi112.ithwzj=up:replay,1=cephfs.smithi112.hxfwkh=up:replay} 2 up:standby 2024-09-16T00:49:27.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:26 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:49:27.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:26 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:49:27.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:26 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:49:27.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:26 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:49:27.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:26 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:49:27.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:26 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:49:27.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:26 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:49:27.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:26 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:49:27.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:26 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:49:27.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:26 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:49:27.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:26 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:49:27.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:26 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:49:27.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:26 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:49:27.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:26 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:49:27.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:26 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:49:27.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:26 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:49:27.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:26 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:49:27.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:26 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:49:27.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:26 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:49:27.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:26 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:49:27.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:26 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-09-16T00:49:27.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:26 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-09-16T00:49:27.342 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:26 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-16T00:49:27.342 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:26 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-09-16T00:49:27.342 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:26 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-09-16T00:49:27.342 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:26 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-09-16T00:49:27.342 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:26 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-09-16T00:49:27.342 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:26 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-09-16T00:49:27.342 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:26 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-09-16T00:49:27.342 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:26 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-09-16T00:49:27.342 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:26 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-09-16T00:49:27.342 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:26 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-09-16T00:49:27.342 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:26 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-09-16T00:49:27.342 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:26 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-09-16T00:49:27.342 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:26 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-16T00:49:27.342 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:26 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-09-16T00:49:27.343 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:26 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-09-16T00:49:27.343 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:26 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-09-16T00:49:27.343 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:26 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-09-16T00:49:27.343 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:26 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-09-16T00:49:27.343 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:26 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-09-16T00:49:27.343 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:26 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-09-16T00:49:27.343 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:26 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-09-16T00:49:27.343 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:26 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-16T00:49:27.343 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:26 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-16T00:49:27.343 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:26 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-16T00:49:27.343 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:26 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-16T00:49:27.343 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:26 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-16T00:49:27.343 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:26 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-16T00:49:27.343 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:26 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-09-16T00:49:27.343 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:26 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-09-16T00:49:27.344 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:26 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:49:27.344 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:26 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:49:27.344 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:26 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:49:28.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:27 smithi112 ceph-mon[97750]: Upgrade: Setting container_image for all rgw 2024-09-16T00:49:28.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:27 smithi112 ceph-mon[97750]: Upgrade: Setting container_image for all rbd-mirror 2024-09-16T00:49:28.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:27 smithi112 ceph-mon[97750]: Upgrade: Setting container_image for all ceph-exporter 2024-09-16T00:49:28.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:27 smithi112 ceph-mon[97750]: Upgrade: Setting container_image for all iscsi 2024-09-16T00:49:28.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:27 smithi112 ceph-mon[97750]: Upgrade: Setting container_image for all nfs 2024-09-16T00:49:28.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:27 smithi112 ceph-mon[97750]: Upgrade: Setting container_image for all nvmeof 2024-09-16T00:49:28.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:27 smithi112 ceph-mon[97750]: Upgrade: Finalizing container_image settings 2024-09-16T00:49:28.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:27 smithi112 ceph-mon[97750]: Upgrade: Complete! 2024-09-16T00:49:28.091 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:27 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:49:28.091 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:27 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:49:28.091 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:27 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:49:28.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:27 smithi169 ceph-mon[77626]: Upgrade: Setting container_image for all rgw 2024-09-16T00:49:28.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:27 smithi169 ceph-mon[77626]: Upgrade: Setting container_image for all rbd-mirror 2024-09-16T00:49:28.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:27 smithi169 ceph-mon[77626]: Upgrade: Setting container_image for all ceph-exporter 2024-09-16T00:49:28.281 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:27 smithi169 ceph-mon[77626]: Upgrade: Setting container_image for all iscsi 2024-09-16T00:49:28.281 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:27 smithi169 ceph-mon[77626]: Upgrade: Setting container_image for all nfs 2024-09-16T00:49:28.281 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:27 smithi169 ceph-mon[77626]: Upgrade: Setting container_image for all nvmeof 2024-09-16T00:49:28.281 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:27 smithi169 ceph-mon[77626]: Upgrade: Finalizing container_image settings 2024-09-16T00:49:28.281 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:27 smithi169 ceph-mon[77626]: Upgrade: Complete! 2024-09-16T00:49:28.281 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:27 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:49:28.282 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:27 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:49:28.282 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:27 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:49:29.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:28 smithi169 ceph-mon[77626]: pgmap v652: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 793 KiB/s rd, 0 op/s 2024-09-16T00:49:29.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:28 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:49:29.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:28 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:49:29.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:28 smithi112 ceph-mon[97750]: pgmap v652: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 793 KiB/s rd, 0 op/s 2024-09-16T00:49:29.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:28 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:49:29.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:28 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:49:31.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:30 smithi169 ceph-mon[77626]: pgmap v653: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 6 op/s 2024-09-16T00:49:31.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:30 smithi112 ceph-mon[97750]: pgmap v653: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 6 op/s 2024-09-16T00:49:32.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:31 smithi169 ceph-mon[77626]: mds.? [v2:172.21.15.112:6828/2356899876,v1:172.21.15.112:6829/2356899876] up:resolve 2024-09-16T00:49:32.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:31 smithi169 ceph-mon[77626]: fsmap cephfs:2/2 {0=cephfs.smithi112.ithwzj=up:replay,1=cephfs.smithi112.hxfwkh=up:resolve} 2 up:standby 2024-09-16T00:49:32.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:31 smithi169 ceph-mon[77626]: pgmap v654: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 27 MiB/s rd, 9 op/s 2024-09-16T00:49:32.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:31 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:49:32.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:31 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:49:32.316 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:31 smithi112 ceph-mon[97750]: mds.? [v2:172.21.15.112:6828/2356899876,v1:172.21.15.112:6829/2356899876] up:resolve 2024-09-16T00:49:32.316 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:31 smithi112 ceph-mon[97750]: fsmap cephfs:2/2 {0=cephfs.smithi112.ithwzj=up:replay,1=cephfs.smithi112.hxfwkh=up:resolve} 2 up:standby 2024-09-16T00:49:32.316 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:31 smithi112 ceph-mon[97750]: pgmap v654: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 27 MiB/s rd, 9 op/s 2024-09-16T00:49:32.316 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:31 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:49:32.316 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:31 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:49:33.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:33 smithi112 ceph-mon[97750]: mds.? [v2:172.21.15.112:6826/476229221,v1:172.21.15.112:6827/476229221] up:resolve 2024-09-16T00:49:33.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:33 smithi112 ceph-mon[97750]: fsmap cephfs:2/2 {0=cephfs.smithi112.ithwzj=up:resolve,1=cephfs.smithi112.hxfwkh=up:resolve} 2 up:standby 2024-09-16T00:49:34.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:33 smithi169 ceph-mon[77626]: mds.? [v2:172.21.15.112:6826/476229221,v1:172.21.15.112:6827/476229221] up:resolve 2024-09-16T00:49:34.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:33 smithi169 ceph-mon[77626]: fsmap cephfs:2/2 {0=cephfs.smithi112.ithwzj=up:resolve,1=cephfs.smithi112.hxfwkh=up:resolve} 2 up:standby 2024-09-16T00:49:34.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:34 smithi112 ceph-mon[97750]: pgmap v655: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 29 MiB/s rd, 9 op/s 2024-09-16T00:49:34.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:34 smithi112 ceph-mon[97750]: reconnect by client.14516 192.168.0.1:0/4053007363 after 0.000999974 2024-09-16T00:49:34.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:34 smithi112 ceph-mon[97750]: reconnect by client.14516 192.168.0.1:0/4053007363 after 0.000999974 2024-09-16T00:49:34.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:34 smithi112 ceph-mon[97750]: reconnect by client.24291 192.168.0.1:0/26155201 after 0.00199995 2024-09-16T00:49:34.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:34 smithi112 ceph-mon[97750]: reconnect by client.24291 192.168.0.1:0/26155201 after 0.00199995 2024-09-16T00:49:34.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:34 smithi112 ceph-mon[97750]: mds.? [v2:172.21.15.112:6826/476229221,v1:172.21.15.112:6827/476229221] up:reconnect 2024-09-16T00:49:34.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:34 smithi112 ceph-mon[97750]: mds.? [v2:172.21.15.112:6828/2356899876,v1:172.21.15.112:6829/2356899876] up:reconnect 2024-09-16T00:49:34.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:34 smithi112 ceph-mon[97750]: fsmap cephfs:2/2 {0=cephfs.smithi112.ithwzj=up:reconnect,1=cephfs.smithi112.hxfwkh=up:reconnect} 2 up:standby 2024-09-16T00:49:35.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:34 smithi169 ceph-mon[77626]: pgmap v655: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 29 MiB/s rd, 9 op/s 2024-09-16T00:49:35.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:34 smithi169 ceph-mon[77626]: reconnect by client.14516 192.168.0.1:0/4053007363 after 0.000999974 2024-09-16T00:49:35.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:34 smithi169 ceph-mon[77626]: reconnect by client.14516 192.168.0.1:0/4053007363 after 0.000999974 2024-09-16T00:49:35.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:34 smithi169 ceph-mon[77626]: reconnect by client.24291 192.168.0.1:0/26155201 after 0.00199995 2024-09-16T00:49:35.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:34 smithi169 ceph-mon[77626]: reconnect by client.24291 192.168.0.1:0/26155201 after 0.00199995 2024-09-16T00:49:35.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:34 smithi169 ceph-mon[77626]: mds.? [v2:172.21.15.112:6826/476229221,v1:172.21.15.112:6827/476229221] up:reconnect 2024-09-16T00:49:35.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:34 smithi169 ceph-mon[77626]: mds.? [v2:172.21.15.112:6828/2356899876,v1:172.21.15.112:6829/2356899876] up:reconnect 2024-09-16T00:49:35.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:34 smithi169 ceph-mon[77626]: fsmap cephfs:2/2 {0=cephfs.smithi112.ithwzj=up:reconnect,1=cephfs.smithi112.hxfwkh=up:reconnect} 2 up:standby 2024-09-16T00:49:35.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:35 smithi169 ceph-mon[77626]: mds.? [v2:172.21.15.112:6826/476229221,v1:172.21.15.112:6827/476229221] up:rejoin 2024-09-16T00:49:35.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:35 smithi169 ceph-mon[77626]: mds.? [v2:172.21.15.112:6828/2356899876,v1:172.21.15.112:6829/2356899876] up:rejoin 2024-09-16T00:49:35.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:35 smithi169 ceph-mon[77626]: fsmap cephfs:2/2 {0=cephfs.smithi112.ithwzj=up:rejoin,1=cephfs.smithi112.hxfwkh=up:rejoin} 2 up:standby 2024-09-16T00:49:35.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:35 smithi169 ceph-mon[77626]: daemon mds.cephfs.smithi112.hxfwkh is now active in filesystem cephfs as rank 1 2024-09-16T00:49:35.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:35 smithi169 ceph-mon[77626]: daemon mds.cephfs.smithi112.ithwzj is now active in filesystem cephfs as rank 0 2024-09-16T00:49:35.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:35 smithi169 ceph-mon[77626]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-16T00:49:35.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:35 smithi169 ceph-mon[77626]: mds.? [v2:172.21.15.112:6828/2356899876,v1:172.21.15.112:6829/2356899876] up:active 2024-09-16T00:49:35.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:35 smithi169 ceph-mon[77626]: mds.? [v2:172.21.15.112:6826/476229221,v1:172.21.15.112:6827/476229221] up:active 2024-09-16T00:49:35.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:35 smithi169 ceph-mon[77626]: fsmap cephfs:2 {0=cephfs.smithi112.ithwzj=up:active,1=cephfs.smithi112.hxfwkh=up:active} 2 up:standby-replay 2024-09-16T00:49:35.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:35 smithi112 ceph-mon[97750]: mds.? [v2:172.21.15.112:6826/476229221,v1:172.21.15.112:6827/476229221] up:rejoin 2024-09-16T00:49:35.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:35 smithi112 ceph-mon[97750]: mds.? [v2:172.21.15.112:6828/2356899876,v1:172.21.15.112:6829/2356899876] up:rejoin 2024-09-16T00:49:35.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:35 smithi112 ceph-mon[97750]: fsmap cephfs:2/2 {0=cephfs.smithi112.ithwzj=up:rejoin,1=cephfs.smithi112.hxfwkh=up:rejoin} 2 up:standby 2024-09-16T00:49:35.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:35 smithi112 ceph-mon[97750]: daemon mds.cephfs.smithi112.hxfwkh is now active in filesystem cephfs as rank 1 2024-09-16T00:49:35.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:35 smithi112 ceph-mon[97750]: daemon mds.cephfs.smithi112.ithwzj is now active in filesystem cephfs as rank 0 2024-09-16T00:49:35.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:35 smithi112 ceph-mon[97750]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-16T00:49:35.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:35 smithi112 ceph-mon[97750]: mds.? [v2:172.21.15.112:6828/2356899876,v1:172.21.15.112:6829/2356899876] up:active 2024-09-16T00:49:35.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:35 smithi112 ceph-mon[97750]: mds.? [v2:172.21.15.112:6826/476229221,v1:172.21.15.112:6827/476229221] up:active 2024-09-16T00:49:35.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:35 smithi112 ceph-mon[97750]: fsmap cephfs:2 {0=cephfs.smithi112.ithwzj=up:active,1=cephfs.smithi112.hxfwkh=up:active} 2 up:standby-replay 2024-09-16T00:49:36.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:36 smithi112 ceph-mon[97750]: pgmap v656: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 31 MiB/s rd, 10 op/s 2024-09-16T00:49:37.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:36 smithi169 ceph-mon[77626]: pgmap v656: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 31 MiB/s rd, 10 op/s 2024-09-16T00:49:38.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:38 smithi112 ceph-mon[97750]: pgmap v657: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 27 MiB/s rd, 9 op/s 2024-09-16T00:49:38.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:38 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:49:38.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:38 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:49:38.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:38 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:49:38.340 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:38 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:49:38.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:38 smithi112 ceph-mon[97750]: pgmap v658: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 30 MiB/s rd, 10 op/s 2024-09-16T00:49:38.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:38 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:49:38.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:38 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:49:38.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:38 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:49:38.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:38 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:49:38.341 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:38 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:49:38.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:38 smithi169 ceph-mon[77626]: pgmap v657: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 27 MiB/s rd, 9 op/s 2024-09-16T00:49:38.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:38 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:49:38.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:38 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:49:38.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:38 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:49:38.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:38 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:49:38.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:38 smithi169 ceph-mon[77626]: pgmap v658: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 30 MiB/s rd, 10 op/s 2024-09-16T00:49:38.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:38 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:49:38.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:38 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:49:38.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:38 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:49:38.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:38 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:49:38.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:38 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:49:39.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:39 smithi112 ceph-mon[97750]: Health check cleared: CEPHADM_REFRESH_FAILED (was: failed to probe daemons or devices) 2024-09-16T00:49:39.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:39 smithi169 ceph-mon[77626]: Health check cleared: CEPHADM_REFRESH_FAILED (was: failed to probe daemons or devices) 2024-09-16T00:49:40.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:40 smithi112 ceph-mon[97750]: pgmap v659: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 36 MiB/s rd, 5.0 KiB/s wr, 20 op/s 2024-09-16T00:49:40.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:40 smithi169 ceph-mon[77626]: pgmap v659: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 36 MiB/s rd, 5.0 KiB/s wr, 20 op/s 2024-09-16T00:49:43.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:42 smithi169 ceph-mon[77626]: pgmap v660: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 31 MiB/s rd, 5.0 KiB/s wr, 18 op/s 2024-09-16T00:49:43.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:42 smithi112 ceph-mon[97750]: pgmap v660: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 31 MiB/s rd, 5.0 KiB/s wr, 18 op/s 2024-09-16T00:49:45.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:44 smithi169 ceph-mon[77626]: pgmap v661: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 32 MiB/s rd, 9.6 KiB/s wr, 21 op/s 2024-09-16T00:49:45.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:44 smithi112 ceph-mon[97750]: pgmap v661: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 32 MiB/s rd, 9.6 KiB/s wr, 21 op/s 2024-09-16T00:49:46.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:46 smithi169 ceph-mon[77626]: pgmap v662: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 30 MiB/s rd, 9.6 KiB/s wr, 21 op/s 2024-09-16T00:49:46.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:46 smithi112 ceph-mon[97750]: pgmap v662: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 30 MiB/s rd, 9.6 KiB/s wr, 21 op/s 2024-09-16T00:49:47.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:47 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:49:47.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:47 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:49:48.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:47 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:49:48.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:47 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:49:48.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:48 smithi112 ceph-mon[97750]: pgmap v663: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 30 MiB/s rd, 9.6 KiB/s wr, 21 op/s 2024-09-16T00:49:49.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:48 smithi169 ceph-mon[77626]: pgmap v663: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 30 MiB/s rd, 9.6 KiB/s wr, 21 op/s 2024-09-16T00:49:51.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:50 smithi169 ceph-mon[77626]: pgmap v664: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 26 MiB/s rd, 8.4 KiB/s wr, 20 op/s 2024-09-16T00:49:51.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:50 smithi112 ceph-mon[97750]: pgmap v664: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 26 MiB/s rd, 8.4 KiB/s wr, 20 op/s 2024-09-16T00:49:53.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:52 smithi169 ceph-mon[77626]: pgmap v665: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 3.8 MiB/s rd, 4.0 KiB/s wr, 5 op/s 2024-09-16T00:49:53.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:52 smithi112 ceph-mon[97750]: pgmap v665: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 3.8 MiB/s rd, 4.0 KiB/s wr, 5 op/s 2024-09-16T00:49:55.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:54 smithi169 ceph-mon[77626]: pgmap v666: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.8 MiB/s rd, 4.0 KiB/s wr, 6 op/s 2024-09-16T00:49:55.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:54 smithi112 ceph-mon[97750]: pgmap v666: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.8 MiB/s rd, 4.0 KiB/s wr, 6 op/s 2024-09-16T00:49:55.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:55.682+0000 7f2180301640 1 -- 172.21.15.112:0/3111433987 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2178102ef0 msgr2=0x7f21780fc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:55.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:55.682+0000 7f2180301640 1 --2- 172.21.15.112:0/3111433987 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2178102ef0 0x7f21780fc5b0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f21680048d0 tx=0x7f2168009da0 comp rx=0 tx=0).stop 2024-09-16T00:49:55.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:55.683+0000 7f2180301640 1 -- 172.21.15.112:0/3111433987 shutdown_connections 2024-09-16T00:49:55.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:55.683+0000 7f2180301640 1 --2- 172.21.15.112:0/3111433987 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2178102ef0 0x7f21780fc5b0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:55.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:55.683+0000 7f2180301640 1 --2- 172.21.15.112:0/3111433987 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2178102550 0x7f2178102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:55.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:55.683+0000 7f2180301640 1 -- 172.21.15.112:0/3111433987 >> 172.21.15.112:0/3111433987 conn(0x7f21780f80f0 msgr2=0x7f21780fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:49:55.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:55.683+0000 7f2180301640 1 -- 172.21.15.112:0/3111433987 shutdown_connections 2024-09-16T00:49:55.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:55.683+0000 7f2180301640 1 -- 172.21.15.112:0/3111433987 wait complete. 2024-09-16T00:49:55.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:55.683+0000 7f2180301640 1 Processor -- start 2024-09-16T00:49:55.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:55.683+0000 7f2180301640 1 -- start start 2024-09-16T00:49:55.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:55.683+0000 7f2180301640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2178102550 0x7f2178113520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:55.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:55.684+0000 7f2180301640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2178102ef0 0x7f2178113a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:55.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:55.684+0000 7f2180301640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21781140b0 con 0x7f2178102550 2024-09-16T00:49:55.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:55.684+0000 7f2180301640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217810d4c0 con 0x7f2178102ef0 2024-09-16T00:49:55.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:55.684+0000 7f217e09d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2178102550 0x7f2178113520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:55.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:55.684+0000 7f217e09d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2178102550 0x7f2178113520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33368/0 (socket says 172.21.15.112:33368) 2024-09-16T00:49:55.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:55.684+0000 7f217e09d640 1 -- 172.21.15.112:0/3655188738 learned_addr learned my addr 172.21.15.112:0/3655188738 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:49:55.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:55.684+0000 7f217d89c640 1 --2- 172.21.15.112:0/3655188738 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2178102ef0 0x7f2178113a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:55.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:55.684+0000 7f217e09d640 1 -- 172.21.15.112:0/3655188738 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2178102ef0 msgr2=0x7f2178113a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:55.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:55.684+0000 7f217e09d640 1 --2- 172.21.15.112:0/3655188738 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2178102ef0 0x7f2178113a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:55.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:55.684+0000 7f217e09d640 1 -- 172.21.15.112:0/3655188738 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2168005040 con 0x7f2178102550 2024-09-16T00:49:55.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:55.684+0000 7f217d89c640 1 --2- 172.21.15.112:0/3655188738 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2178102ef0 0x7f2178113a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:49:55.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:55.684+0000 7f217e09d640 1 --2- 172.21.15.112:0/3655188738 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2178102550 0x7f2178113520 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f2174002a50 tx=0x7f2174004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:49:55.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:55.685+0000 7f216effd640 1 -- 172.21.15.112:0/3655188738 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2174007c30 con 0x7f2178102550 2024-09-16T00:49:55.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:55.685+0000 7f216effd640 1 -- 172.21.15.112:0/3655188738 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f217400f040 con 0x7f2178102550 2024-09-16T00:49:55.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:55.685+0000 7f2180301640 1 -- 172.21.15.112:0/3655188738 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f217810d7a0 con 0x7f2178102550 2024-09-16T00:49:55.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:55.685+0000 7f216effd640 1 -- 172.21.15.112:0/3655188738 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f217400da90 con 0x7f2178102550 2024-09-16T00:49:55.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:55.685+0000 7f2180301640 1 -- 172.21.15.112:0/3655188738 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f217810dd10 con 0x7f2178102550 2024-09-16T00:49:55.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:55.686+0000 7f2180301640 1 -- 172.21.15.112:0/3655188738 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21780fdcd0 con 0x7f2178102550 2024-09-16T00:49:55.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:55.686+0000 7f216effd640 1 -- 172.21.15.112:0/3655188738 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f217400dd00 con 0x7f2178102550 2024-09-16T00:49:55.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:55.687+0000 7f216effd640 1 --2- 172.21.15.112:0/3655188738 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f216406e530 0x7f21640709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:55.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:55.687+0000 7f216effd640 1 -- 172.21.15.112:0/3655188738 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f217408dbf0 con 0x7f2178102550 2024-09-16T00:49:55.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:55.687+0000 7f217d89c640 1 --2- 172.21.15.112:0/3655188738 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f216406e530 0x7f21640709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:55.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:55.687+0000 7f217d89c640 1 --2- 172.21.15.112:0/3655188738 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f216406e530 0x7f21640709e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2168000c00 tx=0x7f2168035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:49:55.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:55.691+0000 7f216effd640 1 -- 172.21.15.112:0/3655188738 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f2174056b60 con 0x7f2178102550 2024-09-16T00:49:55.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:55.871+0000 7f2180301640 1 -- 172.21.15.112:0/3655188738 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f217804b750 con 0x7f216406e530 2024-09-16T00:49:55.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:55.872+0000 7f216effd640 1 -- 172.21.15.112:0/3655188738 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f217804b750 con 0x7f216406e530 2024-09-16T00:49:55.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:55.874+0000 7f2180301640 1 -- 172.21.15.112:0/3655188738 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f216406e530 msgr2=0x7f21640709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:55.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:55.874+0000 7f2180301640 1 --2- 172.21.15.112:0/3655188738 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f216406e530 0x7f21640709e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2168000c00 tx=0x7f2168035040 comp rx=0 tx=0).stop 2024-09-16T00:49:55.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:55.874+0000 7f2180301640 1 -- 172.21.15.112:0/3655188738 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2178102550 msgr2=0x7f2178113520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:55.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:55.874+0000 7f2180301640 1 --2- 172.21.15.112:0/3655188738 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2178102550 0x7f2178113520 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f2174002a50 tx=0x7f2174004600 comp rx=0 tx=0).stop 2024-09-16T00:49:55.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:55.874+0000 7f2180301640 1 -- 172.21.15.112:0/3655188738 shutdown_connections 2024-09-16T00:49:55.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:55.874+0000 7f2180301640 1 --2- 172.21.15.112:0/3655188738 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f216406e530 0x7f21640709e0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:55.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:55.874+0000 7f2180301640 1 --2- 172.21.15.112:0/3655188738 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2178102ef0 0x7f2178113a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:55.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:55.874+0000 7f2180301640 1 --2- 172.21.15.112:0/3655188738 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2178102550 0x7f2178113520 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:55.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:55.874+0000 7f2180301640 1 -- 172.21.15.112:0/3655188738 >> 172.21.15.112:0/3655188738 conn(0x7f21780f80f0 msgr2=0x7f21780f9150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:49:55.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:55.874+0000 7f2180301640 1 -- 172.21.15.112:0/3655188738 shutdown_connections 2024-09-16T00:49:55.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:55.874+0000 7f2180301640 1 -- 172.21.15.112:0/3655188738 wait complete. 2024-09-16T00:49:56.667 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -e sha1=8921c8ef947807b717db041d85f0761b7914d5fc -- bash -c 'ceph orch ps' 2024-09-16T00:49:56.961 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:49:57.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:56 smithi169 ceph-mon[77626]: pgmap v667: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 4 op/s 2024-09-16T00:49:57.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:56 smithi169 ceph-mon[77626]: from='client.35248 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:49:57.082 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:56 smithi112 ceph-mon[97750]: pgmap v667: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 4 op/s 2024-09-16T00:49:57.082 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:56 smithi112 ceph-mon[97750]: from='client.35248 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:49:58.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:58.569+0000 7f68383f8640 1 -- 172.21.15.112:0/4211455906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6830102ed0 msgr2=0x7f68300fc590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:58.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:58.569+0000 7f68383f8640 1 --2- 172.21.15.112:0/4211455906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6830102ed0 0x7f68300fc590 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f68200048d0 tx=0x7f6820009da0 comp rx=0 tx=0).stop 2024-09-16T00:49:58.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:58.569+0000 7f68383f8640 1 -- 172.21.15.112:0/4211455906 shutdown_connections 2024-09-16T00:49:58.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:58.569+0000 7f68383f8640 1 --2- 172.21.15.112:0/4211455906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6830102ed0 0x7f68300fc590 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:58.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:58.569+0000 7f68383f8640 1 --2- 172.21.15.112:0/4211455906 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6830102530 0x7f6830102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:58.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:58.569+0000 7f68383f8640 1 -- 172.21.15.112:0/4211455906 >> 172.21.15.112:0/4211455906 conn(0x7f68300f8110 msgr2=0x7f68300fa520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:49:58.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:58.569+0000 7f68383f8640 1 -- 172.21.15.112:0/4211455906 shutdown_connections 2024-09-16T00:49:58.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:58.569+0000 7f68383f8640 1 -- 172.21.15.112:0/4211455906 wait complete. 2024-09-16T00:49:58.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:58.570+0000 7f68383f8640 1 Processor -- start 2024-09-16T00:49:58.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:58.570+0000 7f68383f8640 1 -- start start 2024-09-16T00:49:58.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:58.570+0000 7f68383f8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6830102530 0x7f6830113210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:58.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:58.570+0000 7f68383f8640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6830102ed0 0x7f6830113750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:58.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:58.570+0000 7f68383f8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6830113d50 con 0x7f6830102530 2024-09-16T00:49:58.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:58.570+0000 7f68383f8640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f683010d190 con 0x7f6830102ed0 2024-09-16T00:49:58.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:58.570+0000 7f6836194640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6830102530 0x7f6830113210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:58.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:58.570+0000 7f6836194640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6830102530 0x7f6830113210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33374/0 (socket says 172.21.15.112:33374) 2024-09-16T00:49:58.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:58.570+0000 7f6836194640 1 -- 172.21.15.112:0/921755251 learned_addr learned my addr 172.21.15.112:0/921755251 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:49:58.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:58.570+0000 7f6835993640 1 --2- 172.21.15.112:0/921755251 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6830102ed0 0x7f6830113750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:58.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:58.571+0000 7f6836194640 1 -- 172.21.15.112:0/921755251 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6830102ed0 msgr2=0x7f6830113750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:58.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:58.571+0000 7f6836194640 1 --2- 172.21.15.112:0/921755251 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6830102ed0 0x7f6830113750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:58.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:58.571+0000 7f6836194640 1 -- 172.21.15.112:0/921755251 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6820005040 con 0x7f6830102530 2024-09-16T00:49:58.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:58.571+0000 7f6836194640 1 --2- 172.21.15.112:0/921755251 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6830102530 0x7f6830113210 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f682800ece0 tx=0x7f682800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:49:58.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:58.571+0000 7f6826ffd640 1 -- 172.21.15.112:0/921755251 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f682800ee40 con 0x7f6830102530 2024-09-16T00:49:58.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:58.571+0000 7f6826ffd640 1 -- 172.21.15.112:0/921755251 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f682800f040 con 0x7f6830102530 2024-09-16T00:49:58.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:58.571+0000 7f68383f8640 1 -- 172.21.15.112:0/921755251 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f683010d470 con 0x7f6830102530 2024-09-16T00:49:58.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:58.571+0000 7f6826ffd640 1 -- 172.21.15.112:0/921755251 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6828009580 con 0x7f6830102530 2024-09-16T00:49:58.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:58.571+0000 7f68383f8640 1 -- 172.21.15.112:0/921755251 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f683010d9e0 con 0x7f6830102530 2024-09-16T00:49:58.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:58.572+0000 7f68383f8640 1 -- 172.21.15.112:0/921755251 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68300fdcb0 con 0x7f6830102530 2024-09-16T00:49:58.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:58.572+0000 7f6826ffd640 1 -- 172.21.15.112:0/921755251 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f682800ccc0 con 0x7f6830102530 2024-09-16T00:49:58.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:58.573+0000 7f6826ffd640 1 --2- 172.21.15.112:0/921755251 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f681c06e530 0x7f681c0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:49:58.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:58.573+0000 7f6835993640 1 --2- 172.21.15.112:0/921755251 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f681c06e530 0x7f681c0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:49:58.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:58.573+0000 7f6826ffd640 1 -- 172.21.15.112:0/921755251 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f682808e870 con 0x7f6830102530 2024-09-16T00:49:58.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:58.573+0000 7f6835993640 1 --2- 172.21.15.112:0/921755251 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f681c06e530 0x7f681c0709e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6820000c00 tx=0x7f6820035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:49:58.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:58.577+0000 7f6826ffd640 1 -- 172.21.15.112:0/921755251 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f6828057860 con 0x7f6830102530 2024-09-16T00:49:58.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:58.755+0000 7f68383f8640 1 -- 172.21.15.112:0/921755251 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f683004b720 con 0x7f681c06e530 2024-09-16T00:49:58.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:58.764+0000 7f6826ffd640 1 -- 172.21.15.112:0/921755251 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7f683004b720 con 0x7f681c06e530 2024-09-16T00:49:58.764 INFO:teuthology.orchestra.run.smithi112.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T00:49:58.764 INFO:teuthology.orchestra.run.smithi112.stdout:alertmanager.smithi112 smithi112 *:9093,9094 running (22m) 21s ago 31m 24.1M - 0.25.0 c8568f914cd2 d85ea0562022 2024-09-16T00:49:58.764 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi112 smithi112 running (19m) 21s ago 32m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c e4fe58f9702a 2024-09-16T00:49:58.764 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi169 smithi169 running (19m) 29s ago 31m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c 3ba2c81d527f 2024-09-16T00:49:58.764 INFO:teuthology.orchestra.run.smithi112.stdout:grafana.smithi112 smithi112 *:3000 running (21m) 21s ago 31m 82.7M - 9.4.12 2bacad6d85d8 51f9d2013da2 2024-09-16T00:49:58.764 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.hxfwkh smithi112 running (16m) 21s ago 26m 194M - 19.1.1-326-g8921c8ef f1455bd0219c 1723069902d6 2024-09-16T00:49:58.765 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.ithwzj smithi112 running (16m) 21s ago 26m 141M - 19.1.1-326-g8921c8ef f1455bd0219c e7aad5b81db5 2024-09-16T00:49:58.765 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.gawwgw smithi169 running (15m) 29s ago 26m 21.0M - 19.1.1-326-g8921c8ef f1455bd0219c 469be459ee12 2024-09-16T00:49:58.765 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.lgeqrj smithi169 running (15m) 29s ago 26m 19.8M - 19.1.1-326-g8921c8ef f1455bd0219c 31f0fbd2395d 2024-09-16T00:49:58.765 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi112.xrkkbc smithi112 *:8443,9283,8765 running (22m) 21s ago 33m 507M - 19.1.1-326-g8921c8ef f1455bd0219c a981f7fd2810 2024-09-16T00:49:58.765 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi169.mtrobk smithi169 *:8443,9283,8765 running (22m) 29s ago 31m 448M - 19.1.1-326-g8921c8ef f1455bd0219c 4096e5369c2d 2024-09-16T00:49:58.765 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi112 smithi112 running (20m) 21s ago 33m 67.5M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 5aaec033c042 2024-09-16T00:49:58.765 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi169 smithi169 running (20m) 29s ago 30m 49.0M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 7bc609155f3f 2024-09-16T00:49:58.765 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi112 smithi112 *:9100 running (23m) 21s ago 32m 15.2M - 1.5.0 0da6a335fe13 6404d1d601e2 2024-09-16T00:49:58.765 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi169 smithi169 *:9100 running (22m) 29s ago 31m 14.9M - 1.5.0 0da6a335fe13 5505baf702a6 2024-09-16T00:49:58.765 INFO:teuthology.orchestra.run.smithi112.stdout:osd.0 smithi112 running (19m) 21s ago 30m 112M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 53efc9397017 2024-09-16T00:49:58.765 INFO:teuthology.orchestra.run.smithi112.stdout:osd.1 smithi112 running (18m) 21s ago 29m 108M 1557M 19.1.1-326-g8921c8ef f1455bd0219c fe6f1a4a2cf4 2024-09-16T00:49:58.765 INFO:teuthology.orchestra.run.smithi112.stdout:osd.2 smithi112 running (18m) 21s ago 29m 96.5M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 5f7863712057 2024-09-16T00:49:58.765 INFO:teuthology.orchestra.run.smithi112.stdout:osd.3 smithi169 running (17m) 29s ago 28m 104M 2581M 19.1.1-326-g8921c8ef f1455bd0219c ee07b5577810 2024-09-16T00:49:58.766 INFO:teuthology.orchestra.run.smithi112.stdout:osd.4 smithi169 running (17m) 29s ago 28m 109M 2581M 19.1.1-326-g8921c8ef f1455bd0219c 769f026a54eb 2024-09-16T00:49:58.766 INFO:teuthology.orchestra.run.smithi112.stdout:osd.5 smithi169 running (16m) 29s ago 28m 108M 2581M 19.1.1-326-g8921c8ef f1455bd0219c a6007c13cf78 2024-09-16T00:49:58.766 INFO:teuthology.orchestra.run.smithi112.stdout:prometheus.smithi112 smithi112 *:9095 running (22m) 21s ago 31m 66.9M - 2.43.0 a07b618ecd1d 7a1d42e60d2b 2024-09-16T00:49:58.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:58.766+0000 7f68383f8640 1 -- 172.21.15.112:0/921755251 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f681c06e530 msgr2=0x7f681c0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:58.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:58.766+0000 7f68383f8640 1 --2- 172.21.15.112:0/921755251 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f681c06e530 0x7f681c0709e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6820000c00 tx=0x7f6820035040 comp rx=0 tx=0).stop 2024-09-16T00:49:58.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:58.766+0000 7f68383f8640 1 -- 172.21.15.112:0/921755251 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6830102530 msgr2=0x7f6830113210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:49:58.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:58.766+0000 7f68383f8640 1 --2- 172.21.15.112:0/921755251 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6830102530 0x7f6830113210 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f682800ece0 tx=0x7f682800b040 comp rx=0 tx=0).stop 2024-09-16T00:49:58.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:58.766+0000 7f68383f8640 1 -- 172.21.15.112:0/921755251 shutdown_connections 2024-09-16T00:49:58.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:58.767+0000 7f68383f8640 1 --2- 172.21.15.112:0/921755251 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f681c06e530 0x7f681c0709e0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:58.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:58.767+0000 7f68383f8640 1 --2- 172.21.15.112:0/921755251 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6830102ed0 0x7f6830113750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:58.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:58.767+0000 7f68383f8640 1 --2- 172.21.15.112:0/921755251 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6830102530 0x7f6830113210 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:49:58.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:58.767+0000 7f68383f8640 1 -- 172.21.15.112:0/921755251 >> 172.21.15.112:0/921755251 conn(0x7f68300f8110 msgr2=0x7f68300f91a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:49:58.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:58.767+0000 7f68383f8640 1 -- 172.21.15.112:0/921755251 shutdown_connections 2024-09-16T00:49:58.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:49:58.767+0000 7f68383f8640 1 -- 172.21.15.112:0/921755251 wait complete. 2024-09-16T00:49:59.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:58 smithi169 ceph-mon[77626]: pgmap v668: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:49:59.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:58 smithi112 ceph-mon[97750]: pgmap v668: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:49:59.378 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -e sha1=8921c8ef947807b717db041d85f0761b7914d5fc -- bash -c 'ceph orch upgrade status' 2024-09-16T00:49:59.625 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:50:00.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:49:59 smithi169 ceph-mon[77626]: from='client.35252 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:50:00.050 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:49:59 smithi112 ceph-mon[97750]: from='client.35252 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:50:00.989 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:50:00 smithi112 ceph-mon[97750]: pgmap v669: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:50:00.989 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:50:00 smithi112 ceph-mon[97750]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-09-16T00:50:00.989 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:50:00 smithi112 ceph-mon[97750]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T00:50:00.989 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:50:00 smithi112 ceph-mon[97750]: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T00:50:01.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:50:00 smithi169 ceph-mon[77626]: pgmap v669: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:50:01.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:50:00 smithi169 ceph-mon[77626]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-09-16T00:50:01.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:50:00 smithi169 ceph-mon[77626]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T00:50:01.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:50:00 smithi169 ceph-mon[77626]: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T00:50:01.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:01.195+0000 7f4532d3e640 1 -- 172.21.15.112:0/3869462233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f452c0fd550 msgr2=0x7f452c0fd920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:50:01.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:01.195+0000 7f4532d3e640 1 --2- 172.21.15.112:0/3869462233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f452c0fd550 0x7f452c0fd920 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f451c004900 tx=0x7f451c009da0 comp rx=0 tx=0).stop 2024-09-16T00:50:01.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:01.195+0000 7f4532d3e640 1 -- 172.21.15.112:0/3869462233 shutdown_connections 2024-09-16T00:50:01.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:01.195+0000 7f4532d3e640 1 --2- 172.21.15.112:0/3869462233 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f452c0fdef0 0x7f452c1065c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:50:01.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:01.195+0000 7f4532d3e640 1 --2- 172.21.15.112:0/3869462233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f452c0fd550 0x7f452c0fd920 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:50:01.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:01.195+0000 7f4532d3e640 1 -- 172.21.15.112:0/3869462233 >> 172.21.15.112:0/3869462233 conn(0x7f452c0f7030 msgr2=0x7f452c0f9440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:50:01.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:01.195+0000 7f4532d3e640 1 -- 172.21.15.112:0/3869462233 shutdown_connections 2024-09-16T00:50:01.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:01.195+0000 7f4532d3e640 1 -- 172.21.15.112:0/3869462233 wait complete. 2024-09-16T00:50:01.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:01.195+0000 7f4532d3e640 1 Processor -- start 2024-09-16T00:50:01.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:01.195+0000 7f4532d3e640 1 -- start start 2024-09-16T00:50:01.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:01.195+0000 7f4532d3e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f452c0fd550 0x7f452c1132b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:50:01.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:01.196+0000 7f4532d3e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f452c0fdef0 0x7f452c1137f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:50:01.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:01.196+0000 7f4532d3e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f452c113e80 con 0x7f452c0fd550 2024-09-16T00:50:01.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:01.196+0000 7f4532d3e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f452c10d230 con 0x7f452c0fdef0 2024-09-16T00:50:01.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:01.196+0000 7f4530ada640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f452c0fd550 0x7f452c1132b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:50:01.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:01.196+0000 7f4530ada640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f452c0fd550 0x7f452c1132b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33388/0 (socket says 172.21.15.112:33388) 2024-09-16T00:50:01.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:01.196+0000 7f4530ada640 1 -- 172.21.15.112:0/2143705116 learned_addr learned my addr 172.21.15.112:0/2143705116 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:50:01.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:01.196+0000 7f4530ada640 1 -- 172.21.15.112:0/2143705116 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f452c0fdef0 msgr2=0x7f452c1137f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:50:01.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:01.196+0000 7f4530ada640 1 --2- 172.21.15.112:0/2143705116 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f452c0fdef0 0x7f452c1137f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:50:01.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:01.196+0000 7f4530ada640 1 -- 172.21.15.112:0/2143705116 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f451c005040 con 0x7f452c0fd550 2024-09-16T00:50:01.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:01.196+0000 7f4530ada640 1 --2- 172.21.15.112:0/2143705116 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f452c0fd550 0x7f452c1132b0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f451c02a560 tx=0x7f451c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:50:01.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:01.196+0000 7f45297fa640 1 -- 172.21.15.112:0/2143705116 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f451c038070 con 0x7f452c0fd550 2024-09-16T00:50:01.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:01.196+0000 7f45297fa640 1 -- 172.21.15.112:0/2143705116 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f451c032cd0 con 0x7f452c0fd550 2024-09-16T00:50:01.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:01.196+0000 7f4532d3e640 1 -- 172.21.15.112:0/2143705116 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f452c10d4b0 con 0x7f452c0fd550 2024-09-16T00:50:01.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:01.196+0000 7f45297fa640 1 -- 172.21.15.112:0/2143705116 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f451c0096c0 con 0x7f452c0fd550 2024-09-16T00:50:01.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:01.196+0000 7f4532d3e640 1 -- 172.21.15.112:0/2143705116 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f452c10d970 con 0x7f452c0fd550 2024-09-16T00:50:01.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:01.197+0000 7f4532d3e640 1 -- 172.21.15.112:0/2143705116 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f452c066070 con 0x7f452c0fd550 2024-09-16T00:50:01.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:01.198+0000 7f45297fa640 1 -- 172.21.15.112:0/2143705116 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f451c030020 con 0x7f452c0fd550 2024-09-16T00:50:01.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:01.198+0000 7f45297fa640 1 --2- 172.21.15.112:0/2143705116 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f451406e530 0x7f45140709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:50:01.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:01.198+0000 7f45297fa640 1 -- 172.21.15.112:0/2143705116 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f451c0ae7d0 con 0x7f452c0fd550 2024-09-16T00:50:01.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:01.198+0000 7f452bfff640 1 --2- 172.21.15.112:0/2143705116 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f451406e530 0x7f45140709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:50:01.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:01.199+0000 7f452bfff640 1 --2- 172.21.15.112:0/2143705116 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f451406e530 0x7f45140709e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f452c10eb70 tx=0x7f4520006d20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:50:01.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:01.203+0000 7f45297fa640 1 -- 172.21.15.112:0/2143705116 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f451c0777c0 con 0x7f452c0fd550 2024-09-16T00:50:01.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:01.387+0000 7f4532d3e640 1 -- 172.21.15.112:0/2143705116 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f452c109620 con 0x7f451406e530 2024-09-16T00:50:01.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:01.389+0000 7f45297fa640 1 -- 172.21.15.112:0/2143705116 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f452c109620 con 0x7f451406e530 2024-09-16T00:50:01.389 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:50:01.389 INFO:teuthology.orchestra.run.smithi112.stdout: "target_image": null, 2024-09-16T00:50:01.389 INFO:teuthology.orchestra.run.smithi112.stdout: "in_progress": false, 2024-09-16T00:50:01.389 INFO:teuthology.orchestra.run.smithi112.stdout: "which": "", 2024-09-16T00:50:01.389 INFO:teuthology.orchestra.run.smithi112.stdout: "services_complete": [], 2024-09-16T00:50:01.389 INFO:teuthology.orchestra.run.smithi112.stdout: "progress": null, 2024-09-16T00:50:01.389 INFO:teuthology.orchestra.run.smithi112.stdout: "message": "", 2024-09-16T00:50:01.389 INFO:teuthology.orchestra.run.smithi112.stdout: "is_paused": false 2024-09-16T00:50:01.389 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:50:01.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:01.390+0000 7f4532d3e640 1 -- 172.21.15.112:0/2143705116 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f451406e530 msgr2=0x7f45140709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:50:01.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:01.390+0000 7f4532d3e640 1 --2- 172.21.15.112:0/2143705116 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f451406e530 0x7f45140709e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f452c10eb70 tx=0x7f4520006d20 comp rx=0 tx=0).stop 2024-09-16T00:50:01.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:01.391+0000 7f4532d3e640 1 -- 172.21.15.112:0/2143705116 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f452c0fd550 msgr2=0x7f452c1132b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:50:01.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:01.391+0000 7f4532d3e640 1 --2- 172.21.15.112:0/2143705116 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f452c0fd550 0x7f452c1132b0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f451c02a560 tx=0x7f451c009da0 comp rx=0 tx=0).stop 2024-09-16T00:50:01.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:01.391+0000 7f4532d3e640 1 -- 172.21.15.112:0/2143705116 shutdown_connections 2024-09-16T00:50:01.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:01.391+0000 7f4532d3e640 1 --2- 172.21.15.112:0/2143705116 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f451406e530 0x7f45140709e0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:50:01.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:01.391+0000 7f4532d3e640 1 --2- 172.21.15.112:0/2143705116 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f452c0fdef0 0x7f452c1137f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:50:01.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:01.391+0000 7f4532d3e640 1 --2- 172.21.15.112:0/2143705116 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f452c0fd550 0x7f452c1132b0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:50:01.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:01.391+0000 7f4532d3e640 1 -- 172.21.15.112:0/2143705116 >> 172.21.15.112:0/2143705116 conn(0x7f452c0f7030 msgr2=0x7f452c102930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:50:01.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:01.391+0000 7f4532d3e640 1 -- 172.21.15.112:0/2143705116 shutdown_connections 2024-09-16T00:50:01.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:01.391+0000 7f4532d3e640 1 -- 172.21.15.112:0/2143705116 wait complete. 2024-09-16T00:50:01.979 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:50:01 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:50:02.003 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -e sha1=8921c8ef947807b717db041d85f0761b7914d5fc -- bash -c 'ceph health detail' 2024-09-16T00:50:02.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:50:01 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:50:02.251 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:50:03.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:50:02 smithi169 ceph-mon[77626]: from='client.35256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:50:03.027 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:50:02 smithi169 ceph-mon[77626]: pgmap v670: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:50:03.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:50:02 smithi112 ceph-mon[97750]: from='client.35256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:50:03.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:50:02 smithi112 ceph-mon[97750]: pgmap v670: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:50:04.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:03.993+0000 7fb8d6fb7640 1 -- 172.21.15.112:0/1817702612 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8d00fb4a0 msgr2=0x7fb8d00fb910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:50:04.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:03.993+0000 7fb8d6fb7640 1 --2- 172.21.15.112:0/1817702612 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8d00fb4a0 0x7fb8d00fb910 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fb8b80048d0 tx=0x7fb8b8009da0 comp rx=0 tx=0).stop 2024-09-16T00:50:04.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:03.993+0000 7fb8d6fb7640 1 -- 172.21.15.112:0/1817702612 shutdown_connections 2024-09-16T00:50:04.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:03.993+0000 7fb8d6fb7640 1 --2- 172.21.15.112:0/1817702612 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8d00fb4a0 0x7fb8d00fb910 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:50:04.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:03.993+0000 7fb8d6fb7640 1 --2- 172.21.15.112:0/1817702612 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8d01014e0 0x7fb8d01018b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:50:04.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:03.993+0000 7fb8d6fb7640 1 -- 172.21.15.112:0/1817702612 >> 172.21.15.112:0/1817702612 conn(0x7fb8d00f7060 msgr2=0x7fb8d00f9470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:50:04.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:03.993+0000 7fb8d6fb7640 1 -- 172.21.15.112:0/1817702612 shutdown_connections 2024-09-16T00:50:04.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:03.993+0000 7fb8d6fb7640 1 -- 172.21.15.112:0/1817702612 wait complete. 2024-09-16T00:50:04.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:03.993+0000 7fb8d6fb7640 1 Processor -- start 2024-09-16T00:50:04.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:03.993+0000 7fb8d6fb7640 1 -- start start 2024-09-16T00:50:04.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:03.994+0000 7fb8d6fb7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8d00fb4a0 0x7fb8d01120d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:50:04.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:03.994+0000 7fb8d6fb7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8d01014e0 0x7fb8d0112610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:50:04.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:03.994+0000 7fb8d6fb7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d0112c10 con 0x7fb8d01014e0 2024-09-16T00:50:04.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:03.994+0000 7fb8d6fb7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d010c050 con 0x7fb8d00fb4a0 2024-09-16T00:50:04.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:03.994+0000 7fb8cffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8d01014e0 0x7fb8d0112610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:50:04.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:03.994+0000 7fb8cffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8d01014e0 0x7fb8d0112610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60792/0 (socket says 172.21.15.112:60792) 2024-09-16T00:50:04.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:03.994+0000 7fb8cffff640 1 -- 172.21.15.112:0/2708796239 learned_addr learned my addr 172.21.15.112:0/2708796239 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:50:04.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:03.994+0000 7fb8d4d53640 1 --2- 172.21.15.112:0/2708796239 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8d00fb4a0 0x7fb8d01120d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:50:04.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:03.994+0000 7fb8cffff640 1 -- 172.21.15.112:0/2708796239 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8d00fb4a0 msgr2=0x7fb8d01120d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:50:04.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:03.994+0000 7fb8cffff640 1 --2- 172.21.15.112:0/2708796239 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8d00fb4a0 0x7fb8d01120d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:50:04.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:03.994+0000 7fb8cffff640 1 -- 172.21.15.112:0/2708796239 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8b8005040 con 0x7fb8d01014e0 2024-09-16T00:50:04.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:03.994+0000 7fb8cffff640 1 --2- 172.21.15.112:0/2708796239 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8d01014e0 0x7fb8d0112610 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8000c00 tx=0x7fb8b8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:50:04.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:03.995+0000 7fb8cd7fa640 1 -- 172.21.15.112:0/2708796239 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8b8038070 con 0x7fb8d01014e0 2024-09-16T00:50:04.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:03.995+0000 7fb8d6fb7640 1 -- 172.21.15.112:0/2708796239 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8d010c2d0 con 0x7fb8d01014e0 2024-09-16T00:50:04.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:03.995+0000 7fb8cd7fa640 1 -- 172.21.15.112:0/2708796239 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fb8b802ade0 con 0x7fb8d01014e0 2024-09-16T00:50:04.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:03.995+0000 7fb8cd7fa640 1 -- 172.21.15.112:0/2708796239 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8b802c7e0 con 0x7fb8d01014e0 2024-09-16T00:50:04.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:03.995+0000 7fb8d6fb7640 1 -- 172.21.15.112:0/2708796239 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8d010c810 con 0x7fb8d01014e0 2024-09-16T00:50:04.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:03.996+0000 7fb8d6fb7640 1 -- 172.21.15.112:0/2708796239 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8d00fcbe0 con 0x7fb8d01014e0 2024-09-16T00:50:04.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:03.996+0000 7fb8cd7fa640 1 -- 172.21.15.112:0/2708796239 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fb8b8030020 con 0x7fb8d01014e0 2024-09-16T00:50:04.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:03.997+0000 7fb8cd7fa640 1 --2- 172.21.15.112:0/2708796239 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fb8bc06e530 0x7fb8bc0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:50:04.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:03.997+0000 7fb8cd7fa640 1 -- 172.21.15.112:0/2708796239 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7fb8b80adf70 con 0x7fb8d01014e0 2024-09-16T00:50:04.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:03.997+0000 7fb8d4d53640 1 --2- 172.21.15.112:0/2708796239 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fb8bc06e530 0x7fb8bc0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:50:04.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:03.997+0000 7fb8d4d53640 1 --2- 172.21.15.112:0/2708796239 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fb8bc06e530 0x7fb8bc0709e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb8c0004ed0 tx=0x7fb8c000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:50:04.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:04.001+0000 7fb8cd7fa640 1 -- 172.21.15.112:0/2708796239 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fb8b80e31b0 con 0x7fb8d01014e0 2024-09-16T00:50:04.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:04.232+0000 7fb8d6fb7640 1 -- 172.21.15.112:0/2708796239 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fb8d0108480 con 0x7fb8d01014e0 2024-09-16T00:50:04.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:04.232+0000 7fb8cd7fa640 1 -- 172.21.15.112:0/2708796239 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fb8b8076f90 con 0x7fb8d01014e0 2024-09-16T00:50:04.233 INFO:teuthology.orchestra.run.smithi112.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-09-16T00:50:04.233 INFO:teuthology.orchestra.run.smithi112.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T00:50:04.233 INFO:teuthology.orchestra.run.smithi112.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T00:50:04.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:04.235+0000 7fb8d6fb7640 1 -- 172.21.15.112:0/2708796239 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fb8bc06e530 msgr2=0x7fb8bc0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:50:04.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:04.235+0000 7fb8d6fb7640 1 --2- 172.21.15.112:0/2708796239 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fb8bc06e530 0x7fb8bc0709e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb8c0004ed0 tx=0x7fb8c000b040 comp rx=0 tx=0).stop 2024-09-16T00:50:04.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:04.235+0000 7fb8d6fb7640 1 -- 172.21.15.112:0/2708796239 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8d01014e0 msgr2=0x7fb8d0112610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:50:04.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:04.235+0000 7fb8d6fb7640 1 --2- 172.21.15.112:0/2708796239 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8d01014e0 0x7fb8d0112610 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8000c00 tx=0x7fb8b8009da0 comp rx=0 tx=0).stop 2024-09-16T00:50:04.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:04.235+0000 7fb8d6fb7640 1 -- 172.21.15.112:0/2708796239 shutdown_connections 2024-09-16T00:50:04.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:04.235+0000 7fb8d6fb7640 1 --2- 172.21.15.112:0/2708796239 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fb8bc06e530 0x7fb8bc0709e0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:50:04.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:04.235+0000 7fb8d6fb7640 1 --2- 172.21.15.112:0/2708796239 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8d01014e0 0x7fb8d0112610 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:50:04.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:04.235+0000 7fb8d6fb7640 1 --2- 172.21.15.112:0/2708796239 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8d00fb4a0 0x7fb8d01120d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:50:04.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:04.235+0000 7fb8d6fb7640 1 -- 172.21.15.112:0/2708796239 >> 172.21.15.112:0/2708796239 conn(0x7fb8d00f7060 msgr2=0x7fb8d00f8910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:50:04.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:04.235+0000 7fb8d6fb7640 1 -- 172.21.15.112:0/2708796239 shutdown_connections 2024-09-16T00:50:04.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:04.235+0000 7fb8d6fb7640 1 -- 172.21.15.112:0/2708796239 wait complete. 2024-09-16T00:50:04.811 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:50:04 smithi112 ceph-mon[97750]: pgmap v671: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:50:04.811 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:50:04 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/2708796239' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:50:04.837 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -e sha1=8921c8ef947807b717db041d85f0761b7914d5fc -- bash -c 'ceph versions' 2024-09-16T00:50:05.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:50:04 smithi169 ceph-mon[77626]: pgmap v671: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:50:05.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:50:04 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/2708796239' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T00:50:05.083 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:50:06.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:06.648+0000 7f7420831640 1 -- 172.21.15.112:0/785167382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f741c0fd560 msgr2=0x7f741c0fd930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:50:06.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:06.648+0000 7f7420831640 1 --2- 172.21.15.112:0/785167382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f741c0fd560 0x7f741c0fd930 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f740c0048d0 tx=0x7f740c009da0 comp rx=0 tx=0).stop 2024-09-16T00:50:06.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:06.648+0000 7f7420831640 1 -- 172.21.15.112:0/785167382 shutdown_connections 2024-09-16T00:50:06.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:06.648+0000 7f7420831640 1 --2- 172.21.15.112:0/785167382 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f741c0fdf00 0x7f741c106540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:50:06.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:06.649+0000 7f7420831640 1 --2- 172.21.15.112:0/785167382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f741c0fd560 0x7f741c0fd930 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:50:06.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:06.649+0000 7f7420831640 1 -- 172.21.15.112:0/785167382 >> 172.21.15.112:0/785167382 conn(0x7f741c0f7030 msgr2=0x7f741c0f9440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:50:06.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:06.649+0000 7f7420831640 1 -- 172.21.15.112:0/785167382 shutdown_connections 2024-09-16T00:50:06.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:06.649+0000 7f7420831640 1 -- 172.21.15.112:0/785167382 wait complete. 2024-09-16T00:50:06.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:06.649+0000 7f7420831640 1 Processor -- start 2024-09-16T00:50:06.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:06.649+0000 7f7420831640 1 -- start start 2024-09-16T00:50:06.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:06.649+0000 7f7420831640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f741c0fd560 0x7f741c113260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:50:06.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:06.649+0000 7f7420831640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f741c0fdf00 0x7f741c1137a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:50:06.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:06.649+0000 7f7420831640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f741c113e30 con 0x7f741c0fd560 2024-09-16T00:50:06.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:06.649+0000 7f7420831640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f741c10d1e0 con 0x7f741c0fdf00 2024-09-16T00:50:06.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:06.649+0000 7f741a59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f741c0fd560 0x7f741c113260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:50:06.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:06.649+0000 7f741a59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f741c0fd560 0x7f741c113260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60808/0 (socket says 172.21.15.112:60808) 2024-09-16T00:50:06.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:06.649+0000 7f741a59c640 1 -- 172.21.15.112:0/208493855 learned_addr learned my addr 172.21.15.112:0/208493855 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:50:06.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:06.650+0000 7f741a59c640 1 -- 172.21.15.112:0/208493855 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f741c0fdf00 msgr2=0x7f741c1137a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:50:06.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:06.650+0000 7f7413fff640 1 --2- 172.21.15.112:0/208493855 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f741c0fdf00 0x7f741c1137a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:50:06.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:06.650+0000 7f741a59c640 1 --2- 172.21.15.112:0/208493855 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f741c0fdf00 0x7f741c1137a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:50:06.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:06.650+0000 7f741a59c640 1 -- 172.21.15.112:0/208493855 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f740c005040 con 0x7f741c0fd560 2024-09-16T00:50:06.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:06.650+0000 7f7413fff640 1 --2- 172.21.15.112:0/208493855 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f741c0fdf00 0x7f741c1137a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:50:06.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:06.650+0000 7f741a59c640 1 --2- 172.21.15.112:0/208493855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f741c0fd560 0x7f741c113260 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f740c003080 tx=0x7f740c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:50:06.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:06.650+0000 7f7412ffd640 1 -- 172.21.15.112:0/208493855 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f740c038070 con 0x7f741c0fd560 2024-09-16T00:50:06.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:06.650+0000 7f7412ffd640 1 -- 172.21.15.112:0/208493855 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f740c0097e0 con 0x7f741c0fd560 2024-09-16T00:50:06.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:06.650+0000 7f7412ffd640 1 -- 172.21.15.112:0/208493855 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f740c03d420 con 0x7f741c0fd560 2024-09-16T00:50:06.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:06.650+0000 7f7420831640 1 -- 172.21.15.112:0/208493855 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f741c10d460 con 0x7f741c0fd560 2024-09-16T00:50:06.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:06.650+0000 7f7420831640 1 -- 172.21.15.112:0/208493855 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f741c10d920 con 0x7f741c0fd560 2024-09-16T00:50:06.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:06.651+0000 7f7420831640 1 -- 172.21.15.112:0/208493855 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f741c0660f0 con 0x7f741c0fd560 2024-09-16T00:50:06.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:06.652+0000 7f7412ffd640 1 -- 172.21.15.112:0/208493855 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f740c030020 con 0x7f741c0fd560 2024-09-16T00:50:06.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:06.652+0000 7f7412ffd640 1 --2- 172.21.15.112:0/208493855 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f740806e530 0x7f74080709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:50:06.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:06.652+0000 7f7412ffd640 1 -- 172.21.15.112:0/208493855 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f740c0ae770 con 0x7f741c0fd560 2024-09-16T00:50:06.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:06.652+0000 7f7413fff640 1 --2- 172.21.15.112:0/208493855 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f740806e530 0x7f74080709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:50:06.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:06.653+0000 7f7413fff640 1 --2- 172.21.15.112:0/208493855 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f740806e530 0x7f74080709e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7404006fd0 tx=0x7f7404008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:50:06.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:06.656+0000 7f7412ffd640 1 -- 172.21.15.112:0/208493855 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f740c077760 con 0x7f741c0fd560 2024-09-16T00:50:06.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:06.896+0000 7f7420831640 1 -- 172.21.15.112:0/208493855 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f741c10e8d0 con 0x7f741c0fd560 2024-09-16T00:50:06.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:06.897+0000 7f7412ffd640 1 -- 172.21.15.112:0/208493855 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f740c07ae20 con 0x7f741c0fd560 2024-09-16T00:50:06.897 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:50:06.897 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": { 2024-09-16T00:50:06.897 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:50:06.897 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:50:06.898 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": { 2024-09-16T00:50:06.898 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:50:06.898 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:50:06.898 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": { 2024-09-16T00:50:06.898 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 6 2024-09-16T00:50:06.898 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:50:06.898 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": { 2024-09-16T00:50:06.898 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 4 2024-09-16T00:50:06.898 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:50:06.898 INFO:teuthology.orchestra.run.smithi112.stdout: "overall": { 2024-09-16T00:50:06.898 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 14 2024-09-16T00:50:06.898 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-16T00:50:06.899 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:50:06.899 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:50:06 smithi112 ceph-mon[97750]: pgmap v672: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:50:06.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:06.899+0000 7f7420831640 1 -- 172.21.15.112:0/208493855 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f740806e530 msgr2=0x7f74080709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:50:06.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:06.899+0000 7f7420831640 1 --2- 172.21.15.112:0/208493855 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f740806e530 0x7f74080709e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7404006fd0 tx=0x7f7404008040 comp rx=0 tx=0).stop 2024-09-16T00:50:06.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:06.899+0000 7f7420831640 1 -- 172.21.15.112:0/208493855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f741c0fd560 msgr2=0x7f741c113260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:50:06.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:06.899+0000 7f7420831640 1 --2- 172.21.15.112:0/208493855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f741c0fd560 0x7f741c113260 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f740c003080 tx=0x7f740c009da0 comp rx=0 tx=0).stop 2024-09-16T00:50:06.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:06.899+0000 7f7420831640 1 -- 172.21.15.112:0/208493855 shutdown_connections 2024-09-16T00:50:06.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:06.899+0000 7f7420831640 1 --2- 172.21.15.112:0/208493855 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f740806e530 0x7f74080709e0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:50:06.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:06.899+0000 7f7420831640 1 --2- 172.21.15.112:0/208493855 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f741c0fdf00 0x7f741c1137a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:50:06.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:06.899+0000 7f7420831640 1 --2- 172.21.15.112:0/208493855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f741c0fd560 0x7f741c113260 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:50:06.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:06.899+0000 7f7420831640 1 -- 172.21.15.112:0/208493855 >> 172.21.15.112:0/208493855 conn(0x7f741c0f7030 msgr2=0x7f741c0f8580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:50:06.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:06.899+0000 7f7420831640 1 -- 172.21.15.112:0/208493855 shutdown_connections 2024-09-16T00:50:06.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:50:06.899+0000 7f7420831640 1 -- 172.21.15.112:0/208493855 wait complete. 2024-09-16T00:50:07.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:50:06 smithi169 ceph-mon[77626]: pgmap v672: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:50:07.478 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -e sha1=8921c8ef947807b717db041d85f0761b7914d5fc -- bash -c 'echo "wait for servicemap items w/ changing names to refresh"' 2024-09-16T00:50:07.726 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:50:08.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:50:07 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/208493855' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:50:08.079 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:50:07 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/208493855' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:50:09.017 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:50:08 smithi112 ceph-mon[97750]: pgmap v673: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:50:09.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:50:08 smithi169 ceph-mon[77626]: pgmap v673: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:50:09.337 INFO:teuthology.orchestra.run.smithi112.stdout:wait for servicemap items w/ changing names to refresh 2024-09-16T00:50:10.012 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -e sha1=8921c8ef947807b717db041d85f0761b7914d5fc -- bash -c 'sleep 60' 2024-09-16T00:50:10.261 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:50:10.932 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:50:10 smithi112 ceph-mon[97750]: pgmap v674: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:50:11.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:50:10 smithi169 ceph-mon[77626]: pgmap v674: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:50:13.092 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:50:12 smithi112 ceph-mon[97750]: pgmap v675: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:50:13.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:50:12 smithi169 ceph-mon[77626]: pgmap v675: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:50:15.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:50:14 smithi112 ceph-mon[97750]: pgmap v676: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:50:15.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:50:14 smithi169 ceph-mon[77626]: pgmap v676: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:50:17.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:50:16 smithi112 ceph-mon[97750]: pgmap v677: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:50:17.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:50:16 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:50:17.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:50:16 smithi169 ceph-mon[77626]: pgmap v677: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:50:17.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:50:16 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:50:19.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:50:18 smithi112 ceph-mon[97750]: pgmap v678: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:50:19.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:50:18 smithi169 ceph-mon[77626]: pgmap v678: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:50:21.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:50:20 smithi112 ceph-mon[97750]: pgmap v679: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:50:21.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:50:20 smithi169 ceph-mon[77626]: pgmap v679: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:50:22.852 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:50:22 smithi112 ceph-mon[97750]: pgmap v680: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:50:23.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:50:22 smithi169 ceph-mon[77626]: pgmap v680: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:50:25.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:50:24 smithi112 ceph-mon[97750]: pgmap v681: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:50:25.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:50:24 smithi169 ceph-mon[77626]: pgmap v681: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:50:27.079 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:50:26 smithi169 ceph-mon[77626]: pgmap v682: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:50:27.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:50:26 smithi112 ceph-mon[97750]: pgmap v682: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:50:29.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:50:28 smithi112 ceph-mon[97750]: pgmap v683: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:50:29.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:50:28 smithi169 ceph-mon[77626]: pgmap v683: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:50:31.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:50:30 smithi112 ceph-mon[97750]: pgmap v684: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:50:31.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:50:30 smithi169 ceph-mon[77626]: pgmap v684: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:50:32.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:50:31 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:50:32.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:50:31 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:50:33.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:50:32 smithi112 ceph-mon[97750]: pgmap v685: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:50:33.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:50:32 smithi169 ceph-mon[77626]: pgmap v685: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:50:35.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:50:34 smithi112 ceph-mon[97750]: pgmap v686: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:50:35.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:50:34 smithi169 ceph-mon[77626]: pgmap v686: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:50:37.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:50:36 smithi112 ceph-mon[97750]: pgmap v687: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:50:37.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:50:36 smithi169 ceph-mon[77626]: pgmap v687: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:50:39.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:50:38 smithi112 ceph-mon[97750]: pgmap v688: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:50:39.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:50:38 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:50:39.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:50:38 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:50:39.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:50:38 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:50:39.090 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:50:38 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:50:39.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:50:38 smithi169 ceph-mon[77626]: pgmap v688: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:50:39.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:50:38 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:50:39.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:50:38 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:50:39.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:50:38 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:50:39.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:50:38 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:50:41.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:50:40 smithi112 ceph-mon[97750]: pgmap v689: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:50:41.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:50:40 smithi169 ceph-mon[77626]: pgmap v689: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:50:43.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:50:42 smithi112 ceph-mon[97750]: pgmap v690: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:50:43.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:50:42 smithi169 ceph-mon[77626]: pgmap v690: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:50:45.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:50:44 smithi112 ceph-mon[97750]: pgmap v691: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:50:45.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:50:44 smithi169 ceph-mon[77626]: pgmap v691: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:50:47.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:50:46 smithi112 ceph-mon[97750]: pgmap v692: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:50:47.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:50:46 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:50:47.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:50:46 smithi169 ceph-mon[77626]: pgmap v692: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:50:47.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:50:46 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:50:49.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:50:48 smithi169 ceph-mon[77626]: pgmap v693: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:50:49.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:50:48 smithi112 ceph-mon[97750]: pgmap v693: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:50:51.283 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:50:50 smithi169 ceph-mon[77626]: pgmap v694: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:50:51.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:50:50 smithi112 ceph-mon[97750]: pgmap v694: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:50:53.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:50:52 smithi169 ceph-mon[77626]: pgmap v695: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:50:53.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:50:52 smithi112 ceph-mon[97750]: pgmap v695: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:50:55.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:50:54 smithi169 ceph-mon[77626]: pgmap v696: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:50:55.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:50:54 smithi112 ceph-mon[97750]: pgmap v696: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:50:57.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:50:56 smithi169 ceph-mon[77626]: pgmap v697: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:50:57.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:50:56 smithi112 ceph-mon[97750]: pgmap v697: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:50:59.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:50:58 smithi169 ceph-mon[77626]: pgmap v698: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:50:59.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:50:58 smithi112 ceph-mon[97750]: pgmap v698: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:51:01.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:51:00 smithi169 ceph-mon[77626]: pgmap v699: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:51:01.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:51:00 smithi112 ceph-mon[97750]: pgmap v699: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:51:02.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:51:01 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:51:02.315 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:51:01 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:51:03.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:51:02 smithi169 ceph-mon[77626]: pgmap v700: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:51:03.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:51:02 smithi112 ceph-mon[97750]: pgmap v700: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:51:05.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:51:04 smithi169 ceph-mon[77626]: pgmap v701: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:51:05.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:51:04 smithi112 ceph-mon[97750]: pgmap v701: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:51:07.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:51:06 smithi169 ceph-mon[77626]: pgmap v702: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:51:07.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:51:06 smithi112 ceph-mon[97750]: pgmap v702: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:51:09.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:51:08 smithi169 ceph-mon[77626]: pgmap v703: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:51:09.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:51:08 smithi112 ceph-mon[97750]: pgmap v703: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:51:11.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:51:10 smithi169 ceph-mon[77626]: pgmap v704: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:51:11.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:51:10 smithi112 ceph-mon[97750]: pgmap v704: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:51:12.286 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -e sha1=8921c8ef947807b717db041d85f0761b7914d5fc -- bash -c 'ceph orch ps' 2024-09-16T00:51:12.540 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:51:12.912 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:51:12 smithi112 ceph-mon[97750]: pgmap v705: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:51:13.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:51:12 smithi169 ceph-mon[77626]: pgmap v705: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:51:14.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:14.177+0000 7fd37ce0b640 1 -- 172.21.15.112:0/4051251960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3780fe450 msgr2=0x7fd3780fe820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:51:14.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:14.177+0000 7fd37ce0b640 1 --2- 172.21.15.112:0/4051251960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3780fe450 0x7fd3780fe820 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fd3680048d0 tx=0x7fd368009da0 comp rx=0 tx=0).stop 2024-09-16T00:51:14.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:14.177+0000 7fd37ce0b640 1 -- 172.21.15.112:0/4051251960 shutdown_connections 2024-09-16T00:51:14.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:14.177+0000 7fd37ce0b640 1 --2- 172.21.15.112:0/4051251960 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd378064fe0 0x7fd378065430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:14.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:14.177+0000 7fd37ce0b640 1 --2- 172.21.15.112:0/4051251960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3780fe450 0x7fd3780fe820 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:14.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:14.177+0000 7fd37ce0b640 1 -- 172.21.15.112:0/4051251960 >> 172.21.15.112:0/4051251960 conn(0x7fd3780723b0 msgr2=0x7fd3780727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:51:14.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:14.177+0000 7fd37ce0b640 1 -- 172.21.15.112:0/4051251960 shutdown_connections 2024-09-16T00:51:14.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:14.178+0000 7fd37ce0b640 1 -- 172.21.15.112:0/4051251960 wait complete. 2024-09-16T00:51:14.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:14.178+0000 7fd37ce0b640 1 Processor -- start 2024-09-16T00:51:14.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:14.178+0000 7fd37ce0b640 1 -- start start 2024-09-16T00:51:14.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:14.178+0000 7fd37ce0b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd378064fe0 0x7fd378113210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:51:14.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:14.178+0000 7fd37ce0b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3780fe450 0x7fd378113750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:51:14.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:14.178+0000 7fd37ce0b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd378113d50 con 0x7fd3780fe450 2024-09-16T00:51:14.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:14.178+0000 7fd37ce0b640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd37810d190 con 0x7fd378064fe0 2024-09-16T00:51:14.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:14.179+0000 7fd375d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3780fe450 0x7fd378113750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:51:14.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:14.179+0000 7fd375d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3780fe450 0x7fd378113750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37816/0 (socket says 172.21.15.112:37816) 2024-09-16T00:51:14.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:14.179+0000 7fd375d9b640 1 -- 172.21.15.112:0/3091033651 learned_addr learned my addr 172.21.15.112:0/3091033651 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:51:14.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:14.179+0000 7fd37659c640 1 --2- 172.21.15.112:0/3091033651 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd378064fe0 0x7fd378113210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:51:14.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:14.179+0000 7fd375d9b640 1 -- 172.21.15.112:0/3091033651 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd378064fe0 msgr2=0x7fd378113210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:51:14.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:14.179+0000 7fd375d9b640 1 --2- 172.21.15.112:0/3091033651 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd378064fe0 0x7fd378113210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:14.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:14.179+0000 7fd375d9b640 1 -- 172.21.15.112:0/3091033651 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd368005040 con 0x7fd3780fe450 2024-09-16T00:51:14.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:14.179+0000 7fd375d9b640 1 --2- 172.21.15.112:0/3091033651 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3780fe450 0x7fd378113750 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fd36c002a50 tx=0x7fd36c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:51:14.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:14.179+0000 7fd366ffd640 1 -- 172.21.15.112:0/3091033651 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd36c007c30 con 0x7fd3780fe450 2024-09-16T00:51:14.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:14.179+0000 7fd366ffd640 1 -- 172.21.15.112:0/3091033651 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fd36c00f040 con 0x7fd3780fe450 2024-09-16T00:51:14.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:14.179+0000 7fd37ce0b640 1 -- 172.21.15.112:0/3091033651 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd37810d470 con 0x7fd3780fe450 2024-09-16T00:51:14.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:14.179+0000 7fd366ffd640 1 -- 172.21.15.112:0/3091033651 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd36c00da10 con 0x7fd3780fe450 2024-09-16T00:51:14.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:14.179+0000 7fd37ce0b640 1 -- 172.21.15.112:0/3091033651 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd37810d990 con 0x7fd3780fe450 2024-09-16T00:51:14.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:14.180+0000 7fd37ce0b640 1 -- 172.21.15.112:0/3091033651 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3780fdce0 con 0x7fd3780fe450 2024-09-16T00:51:14.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:14.181+0000 7fd366ffd640 1 -- 172.21.15.112:0/3091033651 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fd36c00dc80 con 0x7fd3780fe450 2024-09-16T00:51:14.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:14.181+0000 7fd366ffd640 1 --2- 172.21.15.112:0/3091033651 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd36006e530 0x7fd3600709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:51:14.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:14.181+0000 7fd366ffd640 1 -- 172.21.15.112:0/3091033651 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7fd36c08dab0 con 0x7fd3780fe450 2024-09-16T00:51:14.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:14.181+0000 7fd37659c640 1 --2- 172.21.15.112:0/3091033651 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd36006e530 0x7fd3600709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:51:14.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:14.182+0000 7fd37659c640 1 --2- 172.21.15.112:0/3091033651 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd36006e530 0x7fd3600709e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd368009ac0 tx=0x7fd368035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:51:14.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:14.186+0000 7fd366ffd640 1 -- 172.21.15.112:0/3091033651 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fd36c056ad0 con 0x7fd3780fe450 2024-09-16T00:51:14.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:14.360+0000 7fd37ce0b640 1 -- 172.21.15.112:0/3091033651 --> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd37810e8b0 con 0x7fd36006e530 2024-09-16T00:51:14.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:14.369+0000 7fd366ffd640 1 -- 172.21.15.112:0/3091033651 <== mgr.34104 v2:172.21.15.112:6800/1498212142 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7fd37810e8b0 con 0x7fd36006e530 2024-09-16T00:51:14.370 INFO:teuthology.orchestra.run.smithi112.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T00:51:14.370 INFO:teuthology.orchestra.run.smithi112.stdout:alertmanager.smithi112 smithi112 *:9093,9094 running (24m) 97s ago 33m 24.1M - 0.25.0 c8568f914cd2 d85ea0562022 2024-09-16T00:51:14.370 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi112 smithi112 running (21m) 97s ago 33m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c e4fe58f9702a 2024-09-16T00:51:14.370 INFO:teuthology.orchestra.run.smithi112.stdout:crash.smithi169 smithi169 running (21m) 105s ago 32m 6895k - 19.1.1-326-g8921c8ef f1455bd0219c 3ba2c81d527f 2024-09-16T00:51:14.370 INFO:teuthology.orchestra.run.smithi112.stdout:grafana.smithi112 smithi112 *:3000 running (22m) 97s ago 32m 82.7M - 9.4.12 2bacad6d85d8 51f9d2013da2 2024-09-16T00:51:14.370 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.hxfwkh smithi112 running (17m) 97s ago 27m 194M - 19.1.1-326-g8921c8ef f1455bd0219c 1723069902d6 2024-09-16T00:51:14.370 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi112.ithwzj smithi112 running (17m) 97s ago 27m 141M - 19.1.1-326-g8921c8ef f1455bd0219c e7aad5b81db5 2024-09-16T00:51:14.370 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.gawwgw smithi169 running (17m) 105s ago 27m 21.0M - 19.1.1-326-g8921c8ef f1455bd0219c 469be459ee12 2024-09-16T00:51:14.371 INFO:teuthology.orchestra.run.smithi112.stdout:mds.cephfs.smithi169.lgeqrj smithi169 running (16m) 105s ago 27m 19.8M - 19.1.1-326-g8921c8ef f1455bd0219c 31f0fbd2395d 2024-09-16T00:51:14.371 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi112.xrkkbc smithi112 *:8443,9283,8765 running (23m) 97s ago 34m 507M - 19.1.1-326-g8921c8ef f1455bd0219c a981f7fd2810 2024-09-16T00:51:14.371 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.smithi169.mtrobk smithi169 *:8443,9283,8765 running (23m) 105s ago 32m 448M - 19.1.1-326-g8921c8ef f1455bd0219c 4096e5369c2d 2024-09-16T00:51:14.371 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi112 smithi112 running (22m) 97s ago 34m 67.5M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 5aaec033c042 2024-09-16T00:51:14.371 INFO:teuthology.orchestra.run.smithi112.stdout:mon.smithi169 smithi169 running (21m) 105s ago 32m 49.0M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 7bc609155f3f 2024-09-16T00:51:14.371 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi112 smithi112 *:9100 running (24m) 97s ago 33m 15.2M - 1.5.0 0da6a335fe13 6404d1d601e2 2024-09-16T00:51:14.371 INFO:teuthology.orchestra.run.smithi112.stdout:node-exporter.smithi169 smithi169 *:9100 running (23m) 105s ago 32m 14.9M - 1.5.0 0da6a335fe13 5505baf702a6 2024-09-16T00:51:14.371 INFO:teuthology.orchestra.run.smithi112.stdout:osd.0 smithi112 running (20m) 97s ago 31m 112M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 53efc9397017 2024-09-16T00:51:14.371 INFO:teuthology.orchestra.run.smithi112.stdout:osd.1 smithi112 running (20m) 97s ago 30m 108M 1557M 19.1.1-326-g8921c8ef f1455bd0219c fe6f1a4a2cf4 2024-09-16T00:51:14.371 INFO:teuthology.orchestra.run.smithi112.stdout:osd.2 smithi112 running (19m) 97s ago 30m 96.5M 1557M 19.1.1-326-g8921c8ef f1455bd0219c 5f7863712057 2024-09-16T00:51:14.371 INFO:teuthology.orchestra.run.smithi112.stdout:osd.3 smithi169 running (18m) 105s ago 30m 104M 2581M 19.1.1-326-g8921c8ef f1455bd0219c ee07b5577810 2024-09-16T00:51:14.371 INFO:teuthology.orchestra.run.smithi112.stdout:osd.4 smithi169 running (18m) 105s ago 29m 109M 2581M 19.1.1-326-g8921c8ef f1455bd0219c 769f026a54eb 2024-09-16T00:51:14.371 INFO:teuthology.orchestra.run.smithi112.stdout:osd.5 smithi169 running (17m) 105s ago 29m 108M 2581M 19.1.1-326-g8921c8ef f1455bd0219c a6007c13cf78 2024-09-16T00:51:14.371 INFO:teuthology.orchestra.run.smithi112.stdout:prometheus.smithi112 smithi112 *:9095 running (23m) 97s ago 32m 66.9M - 2.43.0 a07b618ecd1d 7a1d42e60d2b 2024-09-16T00:51:14.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:14.372+0000 7fd37ce0b640 1 -- 172.21.15.112:0/3091033651 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd36006e530 msgr2=0x7fd3600709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:51:14.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:14.372+0000 7fd37ce0b640 1 --2- 172.21.15.112:0/3091033651 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd36006e530 0x7fd3600709e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd368009ac0 tx=0x7fd368035040 comp rx=0 tx=0).stop 2024-09-16T00:51:14.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:14.372+0000 7fd37ce0b640 1 -- 172.21.15.112:0/3091033651 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3780fe450 msgr2=0x7fd378113750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:51:14.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:14.372+0000 7fd37ce0b640 1 --2- 172.21.15.112:0/3091033651 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3780fe450 0x7fd378113750 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fd36c002a50 tx=0x7fd36c004600 comp rx=0 tx=0).stop 2024-09-16T00:51:14.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:14.372+0000 7fd37ce0b640 1 -- 172.21.15.112:0/3091033651 shutdown_connections 2024-09-16T00:51:14.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:14.372+0000 7fd37ce0b640 1 --2- 172.21.15.112:0/3091033651 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd36006e530 0x7fd3600709e0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:14.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:14.372+0000 7fd37ce0b640 1 --2- 172.21.15.112:0/3091033651 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3780fe450 0x7fd378113750 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:14.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:14.372+0000 7fd37ce0b640 1 --2- 172.21.15.112:0/3091033651 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd378064fe0 0x7fd378113210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:14.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:14.372+0000 7fd37ce0b640 1 -- 172.21.15.112:0/3091033651 >> 172.21.15.112:0/3091033651 conn(0x7fd3780723b0 msgr2=0x7fd3780fa2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:51:14.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:14.372+0000 7fd37ce0b640 1 -- 172.21.15.112:0/3091033651 shutdown_connections 2024-09-16T00:51:14.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:14.372+0000 7fd37ce0b640 1 -- 172.21.15.112:0/3091033651 wait complete. 2024-09-16T00:51:15.013 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -e sha1=8921c8ef947807b717db041d85f0761b7914d5fc -- bash -c 'ceph versions' 2024-09-16T00:51:15.267 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:51:15.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:51:14 smithi169 ceph-mon[77626]: pgmap v706: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:51:15.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:51:14 smithi112 ceph-mon[97750]: pgmap v706: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:51:16.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:51:15 smithi112 ceph-mon[97750]: from='client.35268 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:51:16.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:51:15 smithi169 ceph-mon[77626]: from='client.35268 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:51:16.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:16.791+0000 7f8f91b98640 1 -- 172.21.15.112:0/662852156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f8c0fabd0 msgr2=0x7f8f8c0fb040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:51:16.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:16.791+0000 7f8f91b98640 1 --2- 172.21.15.112:0/662852156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f8c0fabd0 0x7f8f8c0fb040 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f8f800048d0 tx=0x7f8f80009da0 comp rx=0 tx=0).stop 2024-09-16T00:51:16.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:16.791+0000 7f8f91b98640 1 -- 172.21.15.112:0/662852156 shutdown_connections 2024-09-16T00:51:16.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:16.791+0000 7f8f91b98640 1 --2- 172.21.15.112:0/662852156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f8c0fabd0 0x7f8f8c0fb040 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:16.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:16.791+0000 7f8f91b98640 1 --2- 172.21.15.112:0/662852156 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f8c102680 0x7f8f8c102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:16.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:16.791+0000 7f8f91b98640 1 -- 172.21.15.112:0/662852156 >> 172.21.15.112:0/662852156 conn(0x7f8f8c071ac0 msgr2=0x7f8f8c071ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:51:16.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:16.791+0000 7f8f91b98640 1 -- 172.21.15.112:0/662852156 shutdown_connections 2024-09-16T00:51:16.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:16.791+0000 7f8f91b98640 1 -- 172.21.15.112:0/662852156 wait complete. 2024-09-16T00:51:16.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:16.792+0000 7f8f91b98640 1 Processor -- start 2024-09-16T00:51:16.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:16.792+0000 7f8f91b98640 1 -- start start 2024-09-16T00:51:16.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:16.792+0000 7f8f91b98640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f8c0fabd0 0x7f8f8c113260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:51:16.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:16.792+0000 7f8f91b98640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f8c102680 0x7f8f8c1137a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:51:16.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:16.792+0000 7f8f91b98640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8c113da0 con 0x7f8f8c0fabd0 2024-09-16T00:51:16.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:16.792+0000 7f8f91b98640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8c10d1e0 con 0x7f8f8c102680 2024-09-16T00:51:16.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:16.792+0000 7f8f8b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f8c0fabd0 0x7f8f8c113260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:51:16.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:16.793+0000 7f8f8b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f8c0fabd0 0x7f8f8c113260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37840/0 (socket says 172.21.15.112:37840) 2024-09-16T00:51:16.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:16.793+0000 7f8f8b7fe640 1 -- 172.21.15.112:0/2890511285 learned_addr learned my addr 172.21.15.112:0/2890511285 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:51:16.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:16.793+0000 7f8f8b7fe640 1 -- 172.21.15.112:0/2890511285 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f8c102680 msgr2=0x7f8f8c1137a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:51:16.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:16.793+0000 7f8f8affd640 1 --2- 172.21.15.112:0/2890511285 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f8c102680 0x7f8f8c1137a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:51:16.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:16.793+0000 7f8f8b7fe640 1 --2- 172.21.15.112:0/2890511285 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f8c102680 0x7f8f8c1137a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:16.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:16.793+0000 7f8f8b7fe640 1 -- 172.21.15.112:0/2890511285 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f80005040 con 0x7f8f8c0fabd0 2024-09-16T00:51:16.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:16.793+0000 7f8f8b7fe640 1 --2- 172.21.15.112:0/2890511285 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f8c0fabd0 0x7f8f8c113260 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c002a50 tx=0x7f8f7c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:51:16.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:16.793+0000 7f8f73fff640 1 -- 172.21.15.112:0/2890511285 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f7c007c30 con 0x7f8f8c0fabd0 2024-09-16T00:51:16.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:16.793+0000 7f8f91b98640 1 -- 172.21.15.112:0/2890511285 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f8c10d4c0 con 0x7f8f8c0fabd0 2024-09-16T00:51:16.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:16.793+0000 7f8f73fff640 1 -- 172.21.15.112:0/2890511285 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f8f7c00f040 con 0x7f8f8c0fabd0 2024-09-16T00:51:16.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:16.793+0000 7f8f73fff640 1 -- 172.21.15.112:0/2890511285 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f7c00da90 con 0x7f8f8c0fabd0 2024-09-16T00:51:16.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:16.793+0000 7f8f91b98640 1 -- 172.21.15.112:0/2890511285 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f8c10da60 con 0x7f8f8c0fabd0 2024-09-16T00:51:16.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:16.794+0000 7f8f91b98640 1 -- 172.21.15.112:0/2890511285 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f8c0725d0 con 0x7f8f8c0fabd0 2024-09-16T00:51:16.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:16.795+0000 7f8f73fff640 1 -- 172.21.15.112:0/2890511285 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f8f7c00dd00 con 0x7f8f8c0fabd0 2024-09-16T00:51:16.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:16.795+0000 7f8f73fff640 1 --2- 172.21.15.112:0/2890511285 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f8f7406e530 0x7f8f740709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:51:16.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:16.795+0000 7f8f73fff640 1 -- 172.21.15.112:0/2890511285 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f8f7c08dbf0 con 0x7f8f8c0fabd0 2024-09-16T00:51:16.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:16.795+0000 7f8f8affd640 1 --2- 172.21.15.112:0/2890511285 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f8f7406e530 0x7f8f740709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:51:16.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:16.796+0000 7f8f8affd640 1 --2- 172.21.15.112:0/2890511285 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f8f7406e530 0x7f8f740709e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8f80009ac0 tx=0x7f8f80009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:51:16.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:16.800+0000 7f8f73fff640 1 -- 172.21.15.112:0/2890511285 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f8f7c056b60 con 0x7f8f8c0fabd0 2024-09-16T00:51:17.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:17.033+0000 7f8f91b98640 1 -- 172.21.15.112:0/2890511285 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f8f8c04b720 con 0x7f8f8c0fabd0 2024-09-16T00:51:17.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:17.033+0000 7f8f73fff640 1 -- 172.21.15.112:0/2890511285 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f8f7c05a220 con 0x7f8f8c0fabd0 2024-09-16T00:51:17.034 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-16T00:51:17.034 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": { 2024-09-16T00:51:17.034 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:51:17.034 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:51:17.034 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": { 2024-09-16T00:51:17.034 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-16T00:51:17.034 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:51:17.034 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": { 2024-09-16T00:51:17.035 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 6 2024-09-16T00:51:17.035 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:51:17.035 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": { 2024-09-16T00:51:17.035 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 4 2024-09-16T00:51:17.035 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-16T00:51:17.035 INFO:teuthology.orchestra.run.smithi112.stdout: "overall": { 2024-09-16T00:51:17.035 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 14 2024-09-16T00:51:17.035 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-16T00:51:17.035 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-16T00:51:17.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:17.036+0000 7f8f91b98640 1 -- 172.21.15.112:0/2890511285 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f8f7406e530 msgr2=0x7f8f740709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:51:17.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:17.036+0000 7f8f91b98640 1 --2- 172.21.15.112:0/2890511285 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f8f7406e530 0x7f8f740709e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8f80009ac0 tx=0x7f8f80009da0 comp rx=0 tx=0).stop 2024-09-16T00:51:17.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:17.036+0000 7f8f91b98640 1 -- 172.21.15.112:0/2890511285 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f8c0fabd0 msgr2=0x7f8f8c113260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:51:17.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:17.036+0000 7f8f91b98640 1 --2- 172.21.15.112:0/2890511285 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f8c0fabd0 0x7f8f8c113260 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c002a50 tx=0x7f8f7c004600 comp rx=0 tx=0).stop 2024-09-16T00:51:17.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:17.036+0000 7f8f91b98640 1 -- 172.21.15.112:0/2890511285 shutdown_connections 2024-09-16T00:51:17.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:17.036+0000 7f8f91b98640 1 --2- 172.21.15.112:0/2890511285 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f8f7406e530 0x7f8f740709e0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:17.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:17.036+0000 7f8f91b98640 1 --2- 172.21.15.112:0/2890511285 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f8c102680 0x7f8f8c1137a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:17.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:17.036+0000 7f8f91b98640 1 --2- 172.21.15.112:0/2890511285 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f8c0fabd0 0x7f8f8c113260 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:17.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:17.036+0000 7f8f91b98640 1 -- 172.21.15.112:0/2890511285 >> 172.21.15.112:0/2890511285 conn(0x7f8f8c071ac0 msgr2=0x7f8f8c105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:51:17.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:17.036+0000 7f8f91b98640 1 -- 172.21.15.112:0/2890511285 shutdown_connections 2024-09-16T00:51:17.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:17.036+0000 7f8f91b98640 1 -- 172.21.15.112:0/2890511285 wait complete. 2024-09-16T00:51:17.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:51:16 smithi169 ceph-mon[77626]: pgmap v707: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:51:17.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:51:16 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:51:17.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:51:16 smithi112 ceph-mon[97750]: pgmap v707: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:51:17.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:51:16 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:51:17.648 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -e sha1=8921c8ef947807b717db041d85f0761b7914d5fc -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 1'"'"'' 2024-09-16T00:51:17.895 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:51:18.016 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:51:17 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/2890511285' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:51:18.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:51:17 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/2890511285' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:51:19.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:51:18 smithi112 ceph-mon[97750]: pgmap v708: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:51:19.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:51:18 smithi169 ceph-mon[77626]: pgmap v708: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:51:19.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:19.625+0000 7fe78d673640 1 -- 172.21.15.112:0/3777934698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7880fcf80 msgr2=0x7fe7880fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:51:19.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:19.625+0000 7fe78d673640 1 --2- 172.21.15.112:0/3777934698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7880fcf80 0x7fe7880fd3d0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fe77c0048d0 tx=0x7fe77c009da0 comp rx=0 tx=0).stop 2024-09-16T00:51:19.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:19.625+0000 7fe78d673640 1 -- 172.21.15.112:0/3777934698 shutdown_connections 2024-09-16T00:51:19.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:19.625+0000 7fe78d673640 1 --2- 172.21.15.112:0/3777934698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7880fcf80 0x7fe7880fd3d0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:19.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:19.625+0000 7fe78d673640 1 --2- 172.21.15.112:0/3777934698 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7880fb810 0x7fe7880fbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:19.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:19.625+0000 7fe78d673640 1 -- 172.21.15.112:0/3777934698 >> 172.21.15.112:0/3777934698 conn(0x7fe7880723b0 msgr2=0x7fe7880727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:51:19.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:19.625+0000 7fe78d673640 1 -- 172.21.15.112:0/3777934698 shutdown_connections 2024-09-16T00:51:19.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:19.625+0000 7fe78d673640 1 -- 172.21.15.112:0/3777934698 wait complete. 2024-09-16T00:51:19.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:19.626+0000 7fe78d673640 1 Processor -- start 2024-09-16T00:51:19.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:19.626+0000 7fe78d673640 1 -- start start 2024-09-16T00:51:19.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:19.626+0000 7fe78d673640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7880fb810 0x7fe788112230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:51:19.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:19.626+0000 7fe78d673640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe788112770 0x7fe78810c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:51:19.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:19.626+0000 7fe78d673640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe788112cf0 con 0x7fe7880fb810 2024-09-16T00:51:19.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:19.626+0000 7fe78d673640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe788112e30 con 0x7fe788112770 2024-09-16T00:51:19.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:19.626+0000 7fe786ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7880fb810 0x7fe788112230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:51:19.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:19.626+0000 7fe786ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7880fb810 0x7fe788112230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37864/0 (socket says 172.21.15.112:37864) 2024-09-16T00:51:19.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:19.626+0000 7fe786ffd640 1 -- 172.21.15.112:0/3059838419 learned_addr learned my addr 172.21.15.112:0/3059838419 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:51:19.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:19.626+0000 7fe7867fc640 1 --2- 172.21.15.112:0/3059838419 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe788112770 0x7fe78810c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:51:19.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:19.626+0000 7fe786ffd640 1 -- 172.21.15.112:0/3059838419 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe788112770 msgr2=0x7fe78810c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:51:19.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:19.626+0000 7fe786ffd640 1 --2- 172.21.15.112:0/3059838419 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe788112770 0x7fe78810c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:19.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:19.626+0000 7fe786ffd640 1 -- 172.21.15.112:0/3059838419 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe77c005040 con 0x7fe7880fb810 2024-09-16T00:51:19.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:19.626+0000 7fe7867fc640 1 --2- 172.21.15.112:0/3059838419 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe788112770 0x7fe78810c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:51:19.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:19.627+0000 7fe786ffd640 1 --2- 172.21.15.112:0/3059838419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7880fb810 0x7fe788112230 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fe77800c790 tx=0x7fe7780043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:51:19.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:19.627+0000 7fe76f7fe640 1 -- 172.21.15.112:0/3059838419 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe77800a820 con 0x7fe7880fb810 2024-09-16T00:51:19.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:19.627+0000 7fe76f7fe640 1 -- 172.21.15.112:0/3059838419 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fe77800ae20 con 0x7fe7880fb810 2024-09-16T00:51:19.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:19.627+0000 7fe78d673640 1 -- 172.21.15.112:0/3059838419 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe78810c750 con 0x7fe7880fb810 2024-09-16T00:51:19.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:19.627+0000 7fe76f7fe640 1 -- 172.21.15.112:0/3059838419 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe7780113a0 con 0x7fe7880fb810 2024-09-16T00:51:19.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:19.627+0000 7fe78d673640 1 -- 172.21.15.112:0/3059838419 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe78810ccf0 con 0x7fe7880fb810 2024-09-16T00:51:19.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:19.628+0000 7fe78d673640 1 -- 172.21.15.112:0/3059838419 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe78810d020 con 0x7fe7880fb810 2024-09-16T00:51:19.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:19.628+0000 7fe76f7fe640 1 -- 172.21.15.112:0/3059838419 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fe778011540 con 0x7fe7880fb810 2024-09-16T00:51:19.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:19.629+0000 7fe76f7fe640 1 --2- 172.21.15.112:0/3059838419 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fe77006e530 0x7fe7700709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:51:19.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:19.629+0000 7fe76f7fe640 1 -- 172.21.15.112:0/3059838419 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7fe77808e440 con 0x7fe7880fb810 2024-09-16T00:51:19.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:19.629+0000 7fe7867fc640 1 --2- 172.21.15.112:0/3059838419 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fe77006e530 0x7fe7700709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:51:19.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:19.629+0000 7fe7867fc640 1 --2- 172.21.15.112:0/3059838419 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fe77006e530 0x7fe7700709e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe77c0048a0 tx=0x7fe77c035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:51:19.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:19.633+0000 7fe76f7fe640 1 -- 172.21.15.112:0/3059838419 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fe78810d020 con 0x7fe7880fb810 2024-09-16T00:51:19.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:19.865+0000 7fe78d673640 1 -- 172.21.15.112:0/3059838419 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe78810d020 con 0x7fe7880fb810 2024-09-16T00:51:19.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:19.865+0000 7fe76f7fe640 1 -- 172.21.15.112:0/3059838419 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7fe78810d020 con 0x7fe7880fb810 2024-09-16T00:51:19.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:19.867+0000 7fe78d673640 1 -- 172.21.15.112:0/3059838419 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fe77006e530 msgr2=0x7fe7700709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:51:19.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:19.867+0000 7fe78d673640 1 --2- 172.21.15.112:0/3059838419 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fe77006e530 0x7fe7700709e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe77c0048a0 tx=0x7fe77c035040 comp rx=0 tx=0).stop 2024-09-16T00:51:19.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:19.867+0000 7fe78d673640 1 -- 172.21.15.112:0/3059838419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7880fb810 msgr2=0x7fe788112230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:51:19.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:19.867+0000 7fe78d673640 1 --2- 172.21.15.112:0/3059838419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7880fb810 0x7fe788112230 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fe77800c790 tx=0x7fe7780043c0 comp rx=0 tx=0).stop 2024-09-16T00:51:19.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:19.867+0000 7fe78d673640 1 -- 172.21.15.112:0/3059838419 shutdown_connections 2024-09-16T00:51:19.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:19.867+0000 7fe78d673640 1 --2- 172.21.15.112:0/3059838419 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fe77006e530 0x7fe7700709e0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:19.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:19.867+0000 7fe78d673640 1 --2- 172.21.15.112:0/3059838419 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe788112770 0x7fe78810c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:19.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:19.867+0000 7fe78d673640 1 --2- 172.21.15.112:0/3059838419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7880fb810 0x7fe788112230 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:19.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:19.867+0000 7fe78d673640 1 -- 172.21.15.112:0/3059838419 >> 172.21.15.112:0/3059838419 conn(0x7fe7880723b0 msgr2=0x7fe7880fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:51:19.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:19.868+0000 7fe78d673640 1 -- 172.21.15.112:0/3059838419 shutdown_connections 2024-09-16T00:51:19.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:19.868+0000 7fe78d673640 1 -- 172.21.15.112:0/3059838419 wait complete. 2024-09-16T00:51:19.876 INFO:teuthology.orchestra.run.smithi112.stdout:true 2024-09-16T00:51:20.160 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:51:19 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/3059838419' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:51:20.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:51:19 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/3059838419' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:51:20.460 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -e sha1=8921c8ef947807b717db041d85f0761b7914d5fc -- bash -c 'ceph versions | jq -e '"'"'.overall | keys'"'"' | grep $sha1' 2024-09-16T00:51:20.706 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:51:21.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:51:20 smithi112 ceph-mon[97750]: pgmap v709: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:51:21.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:51:20 smithi169 ceph-mon[77626]: pgmap v709: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:51:22.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:22.314+0000 7f30840bc640 1 -- 172.21.15.112:0/2008630525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f307c064fc0 msgr2=0x7f307c065430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:51:22.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:22.314+0000 7f30840bc640 1 --2- 172.21.15.112:0/2008630525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f307c064fc0 0x7f307c065430 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f30780048d0 tx=0x7f3078009da0 comp rx=0 tx=0).stop 2024-09-16T00:51:22.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:22.314+0000 7f30840bc640 1 -- 172.21.15.112:0/2008630525 shutdown_connections 2024-09-16T00:51:22.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:22.314+0000 7f30840bc640 1 --2- 172.21.15.112:0/2008630525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f307c064fc0 0x7f307c065430 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:22.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:22.314+0000 7f30840bc640 1 --2- 172.21.15.112:0/2008630525 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f307c0fe460 0x7f307c0fe830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:22.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:22.314+0000 7f30840bc640 1 -- 172.21.15.112:0/2008630525 >> 172.21.15.112:0/2008630525 conn(0x7f307c0723b0 msgr2=0x7f307c0727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:51:22.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:22.315+0000 7f30840bc640 1 -- 172.21.15.112:0/2008630525 shutdown_connections 2024-09-16T00:51:22.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:22.315+0000 7f30840bc640 1 -- 172.21.15.112:0/2008630525 wait complete. 2024-09-16T00:51:22.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:22.315+0000 7f30840bc640 1 Processor -- start 2024-09-16T00:51:22.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:22.315+0000 7f30840bc640 1 -- start start 2024-09-16T00:51:22.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:22.315+0000 7f30840bc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f307c064fc0 0x7f307c1131b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:51:22.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:22.315+0000 7f30840bc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f307c0fe460 0x7f307c1136f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:51:22.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:22.315+0000 7f30840bc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f307c113cf0 con 0x7f307c0fe460 2024-09-16T00:51:22.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:22.315+0000 7f30840bc640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f307c10d180 con 0x7f307c064fc0 2024-09-16T00:51:22.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:22.315+0000 7f3081657640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f307c0fe460 0x7f307c1136f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:51:22.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:22.315+0000 7f3081657640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f307c0fe460 0x7f307c1136f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50712/0 (socket says 172.21.15.112:50712) 2024-09-16T00:51:22.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:22.315+0000 7f3081657640 1 -- 172.21.15.112:0/1603285044 learned_addr learned my addr 172.21.15.112:0/1603285044 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:51:22.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:22.316+0000 7f3081e58640 1 --2- 172.21.15.112:0/1603285044 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f307c064fc0 0x7f307c1131b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:51:22.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:22.316+0000 7f3081657640 1 -- 172.21.15.112:0/1603285044 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f307c064fc0 msgr2=0x7f307c1131b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:51:22.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:22.316+0000 7f3081657640 1 --2- 172.21.15.112:0/1603285044 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f307c064fc0 0x7f307c1131b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:22.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:22.316+0000 7f3081657640 1 -- 172.21.15.112:0/1603285044 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3078005040 con 0x7f307c0fe460 2024-09-16T00:51:22.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:22.316+0000 7f3081e58640 1 --2- 172.21.15.112:0/1603285044 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f307c064fc0 0x7f307c1131b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:51:22.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:22.316+0000 7f3081657640 1 --2- 172.21.15.112:0/1603285044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f307c0fe460 0x7f307c1136f0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f3078003080 tx=0x7f3078009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:51:22.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:22.316+0000 7f30727fc640 1 -- 172.21.15.112:0/1603285044 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3078038070 con 0x7f307c0fe460 2024-09-16T00:51:22.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:22.316+0000 7f30727fc640 1 -- 172.21.15.112:0/1603285044 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f307802ade0 con 0x7f307c0fe460 2024-09-16T00:51:22.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:22.316+0000 7f30727fc640 1 -- 172.21.15.112:0/1603285044 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f307802c880 con 0x7f307c0fe460 2024-09-16T00:51:22.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:22.316+0000 7f30840bc640 1 -- 172.21.15.112:0/1603285044 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f307c10d400 con 0x7f307c0fe460 2024-09-16T00:51:22.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:22.316+0000 7f30840bc640 1 -- 172.21.15.112:0/1603285044 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f307c10d8e0 con 0x7f307c0fe460 2024-09-16T00:51:22.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:22.317+0000 7f30840bc640 1 -- 172.21.15.112:0/1603285044 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f307c0fdcf0 con 0x7f307c0fe460 2024-09-16T00:51:22.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:22.318+0000 7f30727fc640 1 -- 172.21.15.112:0/1603285044 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f3078030020 con 0x7f307c0fe460 2024-09-16T00:51:22.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:22.318+0000 7f30727fc640 1 --2- 172.21.15.112:0/1603285044 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f306806e530 0x7f30680709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:51:22.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:22.318+0000 7f30727fc640 1 -- 172.21.15.112:0/1603285044 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f30780ae2f0 con 0x7f307c0fe460 2024-09-16T00:51:22.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:22.318+0000 7f3081e58640 1 --2- 172.21.15.112:0/1603285044 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f306806e530 0x7f30680709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:51:22.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:22.319+0000 7f3081e58640 1 --2- 172.21.15.112:0/1603285044 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f306806e530 0x7f30680709e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f306c004c80 tx=0x7f306c009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:51:22.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:22.323+0000 7f30727fc640 1 -- 172.21.15.112:0/1603285044 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f30780772e0 con 0x7f307c0fe460 2024-09-16T00:51:22.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:22.552+0000 7f30840bc640 1 -- 172.21.15.112:0/1603285044 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f307c10e880 con 0x7f307c0fe460 2024-09-16T00:51:22.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:22.553+0000 7f30727fc640 1 -- 172.21.15.112:0/1603285044 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f307807a9a0 con 0x7f307c0fe460 2024-09-16T00:51:22.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:22.555+0000 7f30840bc640 1 -- 172.21.15.112:0/1603285044 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f306806e530 msgr2=0x7f30680709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:51:22.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:22.555+0000 7f30840bc640 1 --2- 172.21.15.112:0/1603285044 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f306806e530 0x7f30680709e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f306c004c80 tx=0x7f306c009040 comp rx=0 tx=0).stop 2024-09-16T00:51:22.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:22.555+0000 7f30840bc640 1 -- 172.21.15.112:0/1603285044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f307c0fe460 msgr2=0x7f307c1136f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:51:22.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:22.555+0000 7f30840bc640 1 --2- 172.21.15.112:0/1603285044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f307c0fe460 0x7f307c1136f0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f3078003080 tx=0x7f3078009da0 comp rx=0 tx=0).stop 2024-09-16T00:51:22.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:22.555+0000 7f30840bc640 1 -- 172.21.15.112:0/1603285044 shutdown_connections 2024-09-16T00:51:22.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:22.555+0000 7f30840bc640 1 --2- 172.21.15.112:0/1603285044 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f306806e530 0x7f30680709e0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:22.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:22.555+0000 7f30840bc640 1 --2- 172.21.15.112:0/1603285044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f307c0fe460 0x7f307c1136f0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:22.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:22.555+0000 7f30840bc640 1 --2- 172.21.15.112:0/1603285044 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f307c064fc0 0x7f307c1131b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:22.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:22.555+0000 7f30840bc640 1 -- 172.21.15.112:0/1603285044 >> 172.21.15.112:0/1603285044 conn(0x7f307c0723b0 msgr2=0x7f307c1053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:51:22.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:22.555+0000 7f30840bc640 1 -- 172.21.15.112:0/1603285044 shutdown_connections 2024-09-16T00:51:22.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:22.555+0000 7f30840bc640 1 -- 172.21.15.112:0/1603285044 wait complete. 2024-09-16T00:51:22.564 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)" 2024-09-16T00:51:23.144 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:51:22 smithi112 ceph-mon[97750]: pgmap v710: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:51:23.144 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:51:22 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/1603285044' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:51:23.168 DEBUG:teuthology.parallel:result is None 2024-09-16T00:51:23.168 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-16T00:51:23.643 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi112.front.sepia.ceph.com 2024-09-16T00:51:23.643 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-16T00:51:23.644 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:51:22 smithi169 ceph-mon[77626]: pgmap v710: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:51:23.644 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:51:22 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/1603285044' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T00:51:23.892 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:51:25.159 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:51:24 smithi112 ceph-mon[97750]: pgmap v711: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:51:25.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:51:24 smithi169 ceph-mon[77626]: pgmap v711: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:51:25.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:25.455+0000 7fce8bc41640 1 -- 172.21.15.112:0/2646833776 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce841025f0 msgr2=0x7fce841029c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:51:25.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:25.455+0000 7fce8bc41640 1 --2- 172.21.15.112:0/2646833776 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce841025f0 0x7fce841029c0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fce800048d0 tx=0x7fce80009da0 comp rx=0 tx=0).stop 2024-09-16T00:51:25.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:25.455+0000 7fce8bc41640 1 -- 172.21.15.112:0/2646833776 shutdown_connections 2024-09-16T00:51:25.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:25.455+0000 7fce8bc41640 1 --2- 172.21.15.112:0/2646833776 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce840fac60 0x7fce840fb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:25.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:25.455+0000 7fce8bc41640 1 --2- 172.21.15.112:0/2646833776 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce841025f0 0x7fce841029c0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:25.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:25.455+0000 7fce8bc41640 1 -- 172.21.15.112:0/2646833776 >> 172.21.15.112:0/2646833776 conn(0x7fce84070c80 msgr2=0x7fce84071080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:51:25.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:25.455+0000 7fce8bc41640 1 -- 172.21.15.112:0/2646833776 shutdown_connections 2024-09-16T00:51:25.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:25.455+0000 7fce8bc41640 1 -- 172.21.15.112:0/2646833776 wait complete. 2024-09-16T00:51:25.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:25.456+0000 7fce8bc41640 1 Processor -- start 2024-09-16T00:51:25.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:25.456+0000 7fce8bc41640 1 -- start start 2024-09-16T00:51:25.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:25.456+0000 7fce8bc41640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce840fac60 0x7fce841131f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:51:25.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:25.456+0000 7fce8bc41640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce841025f0 0x7fce84113730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:51:25.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:25.456+0000 7fce8bc41640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce84113d30 con 0x7fce840fac60 2024-09-16T00:51:25.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:25.456+0000 7fce8bc41640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce8410d170 con 0x7fce841025f0 2024-09-16T00:51:25.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:25.456+0000 7fce899dd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce840fac60 0x7fce841131f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:51:25.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:25.456+0000 7fce899dd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce840fac60 0x7fce841131f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50726/0 (socket says 172.21.15.112:50726) 2024-09-16T00:51:25.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:25.457+0000 7fce899dd640 1 -- 172.21.15.112:0/1288416675 learned_addr learned my addr 172.21.15.112:0/1288416675 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:51:25.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:25.457+0000 7fce891dc640 1 --2- 172.21.15.112:0/1288416675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce841025f0 0x7fce84113730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:51:25.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:25.457+0000 7fce899dd640 1 -- 172.21.15.112:0/1288416675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce841025f0 msgr2=0x7fce84113730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:51:25.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:25.457+0000 7fce899dd640 1 --2- 172.21.15.112:0/1288416675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce841025f0 0x7fce84113730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:25.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:25.457+0000 7fce899dd640 1 -- 172.21.15.112:0/1288416675 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce80005040 con 0x7fce840fac60 2024-09-16T00:51:25.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:25.457+0000 7fce891dc640 1 --2- 172.21.15.112:0/1288416675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce841025f0 0x7fce84113730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:51:25.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:25.457+0000 7fce899dd640 1 --2- 172.21.15.112:0/1288416675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce840fac60 0x7fce841131f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fce80000c00 tx=0x7fce80009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:51:25.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:25.457+0000 7fce7a7fc640 1 -- 172.21.15.112:0/1288416675 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce80038070 con 0x7fce840fac60 2024-09-16T00:51:25.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:25.457+0000 7fce8bc41640 1 -- 172.21.15.112:0/1288416675 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce8410d3f0 con 0x7fce840fac60 2024-09-16T00:51:25.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:25.457+0000 7fce7a7fc640 1 -- 172.21.15.112:0/1288416675 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fce8002ade0 con 0x7fce840fac60 2024-09-16T00:51:25.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:25.457+0000 7fce7a7fc640 1 -- 172.21.15.112:0/1288416675 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce8002c7e0 con 0x7fce840fac60 2024-09-16T00:51:25.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:25.457+0000 7fce8bc41640 1 -- 172.21.15.112:0/1288416675 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce8410d930 con 0x7fce840fac60 2024-09-16T00:51:25.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:25.458+0000 7fce8bc41640 1 -- 172.21.15.112:0/1288416675 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce84065810 con 0x7fce840fac60 2024-09-16T00:51:25.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:25.459+0000 7fce7a7fc640 1 -- 172.21.15.112:0/1288416675 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fce80030020 con 0x7fce840fac60 2024-09-16T00:51:25.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:25.459+0000 7fce7a7fc640 1 --2- 172.21.15.112:0/1288416675 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fce7006e530 0x7fce700709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:51:25.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:25.459+0000 7fce7a7fc640 1 -- 172.21.15.112:0/1288416675 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7fce800adf80 con 0x7fce840fac60 2024-09-16T00:51:25.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:25.459+0000 7fce891dc640 1 --2- 172.21.15.112:0/1288416675 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fce7006e530 0x7fce700709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:51:25.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:25.460+0000 7fce891dc640 1 --2- 172.21.15.112:0/1288416675 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fce7006e530 0x7fce700709e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fce8410eab0 tx=0x7fce7400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:51:25.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:25.464+0000 7fce7a7fc640 1 -- 172.21.15.112:0/1288416675 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fce80076fa0 con 0x7fce840fac60 2024-09-16T00:51:25.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:25.664+0000 7fce8bc41640 1 -- 172.21.15.112:0/1288416675 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fce8404b720 con 0x7fce840fac60 2024-09-16T00:51:25.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:25.665+0000 7fce7a7fc640 1 -- 172.21.15.112:0/1288416675 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 41 v41) v1 ==== 76+0+2008 (secure 0 0 0) 0x7fce8007a660 con 0x7fce840fac60 2024-09-16T00:51:25.666 INFO:teuthology.orchestra.run.smithi112.stdout:e41 2024-09-16T00:51:25.666 INFO:teuthology.orchestra.run.smithi112.stdout:btime 2024-09-16T00:49:35:467470+0000 2024-09-16T00:51:25.666 INFO:teuthology.orchestra.run.smithi112.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T00:51:25.666 INFO:teuthology.orchestra.run.smithi112.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T00:51:25.666 INFO:teuthology.orchestra.run.smithi112.stdout:legacy client fscid: 1 2024-09-16T00:51:25.666 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:51:25.666 INFO:teuthology.orchestra.run.smithi112.stdout:Filesystem 'cephfs' (1) 2024-09-16T00:51:25.666 INFO:teuthology.orchestra.run.smithi112.stdout:fs_name cephfs 2024-09-16T00:51:25.666 INFO:teuthology.orchestra.run.smithi112.stdout:epoch 41 2024-09-16T00:51:25.666 INFO:teuthology.orchestra.run.smithi112.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-16T00:51:25.666 INFO:teuthology.orchestra.run.smithi112.stdout:created 2024-09-16T00:23:22.328199+0000 2024-09-16T00:51:25.667 INFO:teuthology.orchestra.run.smithi112.stdout:modified 2024-09-16T00:49:35.467466+0000 2024-09-16T00:51:25.667 INFO:teuthology.orchestra.run.smithi112.stdout:tableserver 0 2024-09-16T00:51:25.667 INFO:teuthology.orchestra.run.smithi112.stdout:root 0 2024-09-16T00:51:25.667 INFO:teuthology.orchestra.run.smithi112.stdout:session_timeout 60 2024-09-16T00:51:25.667 INFO:teuthology.orchestra.run.smithi112.stdout:session_autoclose 300 2024-09-16T00:51:25.667 INFO:teuthology.orchestra.run.smithi112.stdout:max_file_size 1099511627776 2024-09-16T00:51:25.667 INFO:teuthology.orchestra.run.smithi112.stdout:max_xattr_size 65536 2024-09-16T00:51:25.667 INFO:teuthology.orchestra.run.smithi112.stdout:required_client_features {} 2024-09-16T00:51:25.667 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure 0 2024-09-16T00:51:25.667 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure_osd_epoch 82 2024-09-16T00:51:25.667 INFO:teuthology.orchestra.run.smithi112.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-16T00:51:25.667 INFO:teuthology.orchestra.run.smithi112.stdout:max_mds 2 2024-09-16T00:51:25.667 INFO:teuthology.orchestra.run.smithi112.stdout:in 0,1 2024-09-16T00:51:25.668 INFO:teuthology.orchestra.run.smithi112.stdout:up {0=34392,1=34404} 2024-09-16T00:51:25.668 INFO:teuthology.orchestra.run.smithi112.stdout:failed 2024-09-16T00:51:25.668 INFO:teuthology.orchestra.run.smithi112.stdout:damaged 2024-09-16T00:51:25.668 INFO:teuthology.orchestra.run.smithi112.stdout:stopped 2024-09-16T00:51:25.668 INFO:teuthology.orchestra.run.smithi112.stdout:data_pools [3] 2024-09-16T00:51:25.668 INFO:teuthology.orchestra.run.smithi112.stdout:metadata_pool 2 2024-09-16T00:51:25.668 INFO:teuthology.orchestra.run.smithi112.stdout:inline_data enabled 2024-09-16T00:51:25.668 INFO:teuthology.orchestra.run.smithi112.stdout:balancer 2024-09-16T00:51:25.668 INFO:teuthology.orchestra.run.smithi112.stdout:bal_rank_mask -1 2024-09-16T00:51:25.668 INFO:teuthology.orchestra.run.smithi112.stdout:standby_count_wanted 1 2024-09-16T00:51:25.668 INFO:teuthology.orchestra.run.smithi112.stdout:qdb_cluster leader: 34392 members: 34392,34404 2024-09-16T00:51:25.668 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.ithwzj{0:34392} state up:active seq 243 join_fscid=1 addr [v2:172.21.15.112:6826/476229221,v1:172.21.15.112:6827/476229221] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:51:25.668 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.gawwgw{0:44343} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.169:6824/1635329957,v1:172.21.15.169:6825/1635329957] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:51:25.669 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi112.hxfwkh{1:34404} state up:active seq 238 join_fscid=1 addr [v2:172.21.15.112:6828/2356899876,v1:172.21.15.112:6829/2356899876] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:51:25.669 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.cephfs.smithi169.lgeqrj{1:44347} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.169:6826/184810031,v1:172.21.15.169:6827/184810031] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:51:25.669 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:51:25.669 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:51:25.670 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 41 2024-09-16T00:51:25.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:25.667+0000 7fce8bc41640 1 -- 172.21.15.112:0/1288416675 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fce7006e530 msgr2=0x7fce700709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:51:25.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:25.667+0000 7fce8bc41640 1 --2- 172.21.15.112:0/1288416675 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fce7006e530 0x7fce700709e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fce8410eab0 tx=0x7fce7400b040 comp rx=0 tx=0).stop 2024-09-16T00:51:25.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:25.667+0000 7fce8bc41640 1 -- 172.21.15.112:0/1288416675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce840fac60 msgr2=0x7fce841131f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:51:25.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:25.667+0000 7fce8bc41640 1 --2- 172.21.15.112:0/1288416675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce840fac60 0x7fce841131f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fce80000c00 tx=0x7fce80009da0 comp rx=0 tx=0).stop 2024-09-16T00:51:25.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:25.667+0000 7fce8bc41640 1 -- 172.21.15.112:0/1288416675 shutdown_connections 2024-09-16T00:51:25.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:25.667+0000 7fce8bc41640 1 --2- 172.21.15.112:0/1288416675 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fce7006e530 0x7fce700709e0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:25.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:25.667+0000 7fce8bc41640 1 --2- 172.21.15.112:0/1288416675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce841025f0 0x7fce84113730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:25.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:25.667+0000 7fce8bc41640 1 --2- 172.21.15.112:0/1288416675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce840fac60 0x7fce841131f0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:25.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:25.667+0000 7fce8bc41640 1 -- 172.21.15.112:0/1288416675 >> 172.21.15.112:0/1288416675 conn(0x7fce84070c80 msgr2=0x7fce840f9ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:51:25.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:25.667+0000 7fce8bc41640 1 -- 172.21.15.112:0/1288416675 shutdown_connections 2024-09-16T00:51:25.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:25.667+0000 7fce8bc41640 1 -- 172.21.15.112:0/1288416675 wait complete. 2024-09-16T00:51:26.266 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:51:25 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/1288416675' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:51:26.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:51:25 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/1288416675' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:51:26.290 INFO:teuthology.run_tasks:Running task fs.post_upgrade_checks... 2024-09-16T00:51:26.301 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:51:26.547 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:51:26.995 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:51:26 smithi112 ceph-mon[97750]: pgmap v712: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:51:27.177 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:51:26 smithi169 ceph-mon[77626]: pgmap v712: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:51:28.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:28.212+0000 7fd2d6a79640 1 -- 172.21.15.112:0/3170074186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2d0102550 msgr2=0x7fd2d0102920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:51:28.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:28.212+0000 7fd2d6a79640 1 --2- 172.21.15.112:0/3170074186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2d0102550 0x7fd2d0102920 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fd2c00048d0 tx=0x7fd2c0009da0 comp rx=0 tx=0).stop 2024-09-16T00:51:28.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:28.212+0000 7fd2d6a79640 1 -- 172.21.15.112:0/3170074186 shutdown_connections 2024-09-16T00:51:28.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:28.212+0000 7fd2d6a79640 1 --2- 172.21.15.112:0/3170074186 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2d0102ef0 0x7fd2d00fc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:28.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:28.212+0000 7fd2d6a79640 1 --2- 172.21.15.112:0/3170074186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2d0102550 0x7fd2d0102920 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:28.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:28.212+0000 7fd2d6a79640 1 -- 172.21.15.112:0/3170074186 >> 172.21.15.112:0/3170074186 conn(0x7fd2d00f80f0 msgr2=0x7fd2d00fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:51:28.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:28.212+0000 7fd2d6a79640 1 -- 172.21.15.112:0/3170074186 shutdown_connections 2024-09-16T00:51:28.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:28.212+0000 7fd2d6a79640 1 -- 172.21.15.112:0/3170074186 wait complete. 2024-09-16T00:51:28.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:28.212+0000 7fd2d6a79640 1 Processor -- start 2024-09-16T00:51:28.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:28.212+0000 7fd2d6a79640 1 -- start start 2024-09-16T00:51:28.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:28.213+0000 7fd2d6a79640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2d0102550 0x7fd2d0113510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:51:28.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:28.213+0000 7fd2d6a79640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2d0102ef0 0x7fd2d0113a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:51:28.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:28.213+0000 7fd2d6a79640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d01140a0 con 0x7fd2d0102550 2024-09-16T00:51:28.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:28.213+0000 7fd2d6a79640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d010d4b0 con 0x7fd2d0102ef0 2024-09-16T00:51:28.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:28.213+0000 7fd2d4815640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2d0102550 0x7fd2d0113510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:51:28.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:28.213+0000 7fd2d4815640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2d0102550 0x7fd2d0113510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50756/0 (socket says 172.21.15.112:50756) 2024-09-16T00:51:28.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:28.213+0000 7fd2d4815640 1 -- 172.21.15.112:0/2397049558 learned_addr learned my addr 172.21.15.112:0/2397049558 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:51:28.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:28.213+0000 7fd2cffff640 1 --2- 172.21.15.112:0/2397049558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2d0102ef0 0x7fd2d0113a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:51:28.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:28.213+0000 7fd2d4815640 1 -- 172.21.15.112:0/2397049558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2d0102ef0 msgr2=0x7fd2d0113a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:51:28.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:28.213+0000 7fd2d4815640 1 --2- 172.21.15.112:0/2397049558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2d0102ef0 0x7fd2d0113a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:28.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:28.213+0000 7fd2d4815640 1 -- 172.21.15.112:0/2397049558 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c0005040 con 0x7fd2d0102550 2024-09-16T00:51:28.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:28.213+0000 7fd2cffff640 1 --2- 172.21.15.112:0/2397049558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2d0102ef0 0x7fd2d0113a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:51:28.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:28.213+0000 7fd2d4815640 1 --2- 172.21.15.112:0/2397049558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2d0102550 0x7fd2d0113510 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fd2c0003080 tx=0x7fd2c0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:51:28.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:28.213+0000 7fd2cd7fa640 1 -- 172.21.15.112:0/2397049558 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd2c0038070 con 0x7fd2d0102550 2024-09-16T00:51:28.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:28.214+0000 7fd2d6a79640 1 -- 172.21.15.112:0/2397049558 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2d010d730 con 0x7fd2d0102550 2024-09-16T00:51:28.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:28.214+0000 7fd2cd7fa640 1 -- 172.21.15.112:0/2397049558 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fd2c00097e0 con 0x7fd2d0102550 2024-09-16T00:51:28.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:28.214+0000 7fd2cd7fa640 1 -- 172.21.15.112:0/2397049558 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd2c003d420 con 0x7fd2d0102550 2024-09-16T00:51:28.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:28.214+0000 7fd2d6a79640 1 -- 172.21.15.112:0/2397049558 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2d010dc70 con 0x7fd2d0102550 2024-09-16T00:51:28.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:28.215+0000 7fd2d6a79640 1 -- 172.21.15.112:0/2397049558 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2d00fdcd0 con 0x7fd2d0102550 2024-09-16T00:51:28.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:28.215+0000 7fd2cd7fa640 1 -- 172.21.15.112:0/2397049558 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fd2c0030020 con 0x7fd2d0102550 2024-09-16T00:51:28.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:28.216+0000 7fd2cd7fa640 1 --2- 172.21.15.112:0/2397049558 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd2bc06e530 0x7fd2bc0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:51:28.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:28.216+0000 7fd2cd7fa640 1 -- 172.21.15.112:0/2397049558 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7fd2c00ae8a0 con 0x7fd2d0102550 2024-09-16T00:51:28.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:28.216+0000 7fd2cffff640 1 --2- 172.21.15.112:0/2397049558 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd2bc06e530 0x7fd2bc0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:51:28.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:28.216+0000 7fd2cffff640 1 --2- 172.21.15.112:0/2397049558 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd2bc06e530 0x7fd2bc0709e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd2d010e790 tx=0x7fd2b8009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:51:28.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:28.220+0000 7fd2cd7fa640 1 -- 172.21.15.112:0/2397049558 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fd2c0077810 con 0x7fd2d0102550 2024-09-16T00:51:28.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:28.420+0000 7fd2d6a79640 1 -- 172.21.15.112:0/2397049558 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd2d004b720 con 0x7fd2d0102550 2024-09-16T00:51:28.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:28.421+0000 7fd2cd7fa640 1 -- 172.21.15.112:0/2397049558 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+5296 (secure 0 0 0) 0x7fd2c007aed0 con 0x7fd2d0102550 2024-09-16T00:51:28.422 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:51:28.422 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":41,"btime":"2024-09-16T00:49:35:467470+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":41,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:23:22.328199+0000","modified":"2024-09-16T00:49:35.467466+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34392,"mds_1":34404},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34392":{"gid":34392,"name":"cephfs.smithi112.ithwzj","rank":0,"incarnation":35,"state":"up:active","state_seq":243,"addr":"172.21.15.112:6827/476229221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":476229221},{"type":"v1","addr":"172.21.15.112:6827","nonce":476229221}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34404":{"gid":34404,"name":"cephfs.smithi112.hxfwkh","rank":1,"incarnation":35,"state":"up:active","state_seq":238,"addr":"172.21.15.112:6829/2356899876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2356899876},{"type":"v1","addr":"172.21.15.112:6829","nonce":2356899876}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_44343":{"gid":44343,"name":"cephfs.smithi169.gawwgw","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6825/1635329957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":1635329957},{"type":"v1","addr":"172.21.15.169:6825","nonce":1635329957}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_44347":{"gid":44347,"name":"cephfs.smithi169.lgeqrj","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6827/184810031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":184810031},{"type":"v1","addr":"172.21.15.169:6827","nonce":184810031}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34392,"qdb_cluster":[34392,34404]},"id":1}]} 2024-09-16T00:51:28.422 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 41 2024-09-16T00:51:28.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:28.423+0000 7fd2d6a79640 1 -- 172.21.15.112:0/2397049558 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd2bc06e530 msgr2=0x7fd2bc0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:51:28.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:28.423+0000 7fd2d6a79640 1 --2- 172.21.15.112:0/2397049558 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd2bc06e530 0x7fd2bc0709e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd2d010e790 tx=0x7fd2b8009040 comp rx=0 tx=0).stop 2024-09-16T00:51:28.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:28.423+0000 7fd2d6a79640 1 -- 172.21.15.112:0/2397049558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2d0102550 msgr2=0x7fd2d0113510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:51:28.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:28.423+0000 7fd2d6a79640 1 --2- 172.21.15.112:0/2397049558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2d0102550 0x7fd2d0113510 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fd2c0003080 tx=0x7fd2c0009da0 comp rx=0 tx=0).stop 2024-09-16T00:51:28.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:28.423+0000 7fd2d6a79640 1 -- 172.21.15.112:0/2397049558 shutdown_connections 2024-09-16T00:51:28.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:28.423+0000 7fd2d6a79640 1 --2- 172.21.15.112:0/2397049558 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd2bc06e530 0x7fd2bc0709e0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:28.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:28.423+0000 7fd2d6a79640 1 --2- 172.21.15.112:0/2397049558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2d0102ef0 0x7fd2d0113a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:28.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:28.423+0000 7fd2d6a79640 1 --2- 172.21.15.112:0/2397049558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2d0102550 0x7fd2d0113510 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:28.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:28.423+0000 7fd2d6a79640 1 -- 172.21.15.112:0/2397049558 >> 172.21.15.112:0/2397049558 conn(0x7fd2d00f80f0 msgr2=0x7fd2d00f90c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:51:28.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:28.423+0000 7fd2d6a79640 1 -- 172.21.15.112:0/2397049558 shutdown_connections 2024-09-16T00:51:28.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:28.423+0000 7fd2d6a79640 1 -- 172.21.15.112:0/2397049558 wait complete. 2024-09-16T00:51:29.050 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:51:28 smithi112 ceph-mon[97750]: pgmap v713: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:51:29.051 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:51:28 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/2397049558' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:51:29.075 DEBUG:tasks.fs:checking fs fscid=1,name=cephfs state = {'epoch': 19, 'max_mds': 2, 'flags': 50} 2024-09-16T00:51:29.075 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 20 2024-09-16T00:51:29.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:51:28 smithi169 ceph-mon[77626]: pgmap v713: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:51:29.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:51:28 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/2397049558' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:51:29.323 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:51:30.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:30.955+0000 7f74640d1640 1 -- 172.21.15.112:0/2040172079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f745c1014b0 msgr2=0x7f745c101880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:51:30.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:30.955+0000 7f74640d1640 1 --2- 172.21.15.112:0/2040172079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f745c1014b0 0x7f745c101880 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f744c0048d0 tx=0x7f744c009da0 comp rx=0 tx=0).stop 2024-09-16T00:51:30.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:30.955+0000 7f74640d1640 1 -- 172.21.15.112:0/2040172079 shutdown_connections 2024-09-16T00:51:30.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:30.955+0000 7f74640d1640 1 --2- 172.21.15.112:0/2040172079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f745c0fb470 0x7f745c0fb8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:30.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:30.955+0000 7f74640d1640 1 --2- 172.21.15.112:0/2040172079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f745c1014b0 0x7f745c101880 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:30.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:30.955+0000 7f74640d1640 1 -- 172.21.15.112:0/2040172079 >> 172.21.15.112:0/2040172079 conn(0x7f745c0f6ff0 msgr2=0x7f745c0f9400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:51:30.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:30.955+0000 7f74640d1640 1 -- 172.21.15.112:0/2040172079 shutdown_connections 2024-09-16T00:51:30.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:30.955+0000 7f74640d1640 1 -- 172.21.15.112:0/2040172079 wait complete. 2024-09-16T00:51:30.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:30.956+0000 7f74640d1640 1 Processor -- start 2024-09-16T00:51:30.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:30.956+0000 7f74640d1640 1 -- start start 2024-09-16T00:51:30.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:30.956+0000 7f74640d1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f745c0fb470 0x7f745c0f7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:51:30.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:30.956+0000 7f74640d1640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f745c1014b0 0x7f745c0f79d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:51:30.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:30.956+0000 7f74640d1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f745c0f7fa0 con 0x7f745c0fb470 2024-09-16T00:51:30.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:30.956+0000 7f74640d1640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f745c0f8110 con 0x7f745c1014b0 2024-09-16T00:51:30.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:30.956+0000 7f7461e6d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f745c0fb470 0x7f745c0f7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:51:30.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:30.956+0000 7f7461e6d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f745c0fb470 0x7f745c0f7490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50776/0 (socket says 172.21.15.112:50776) 2024-09-16T00:51:30.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:30.956+0000 7f7461e6d640 1 -- 172.21.15.112:0/3810596430 learned_addr learned my addr 172.21.15.112:0/3810596430 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:51:30.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:30.956+0000 7f7461e6d640 1 -- 172.21.15.112:0/3810596430 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f745c1014b0 msgr2=0x7f745c0f79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:51:30.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:30.956+0000 7f7461e6d640 1 --2- 172.21.15.112:0/3810596430 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f745c1014b0 0x7f745c0f79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:30.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:30.956+0000 7f7461e6d640 1 -- 172.21.15.112:0/3810596430 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f744c005040 con 0x7f745c0fb470 2024-09-16T00:51:30.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:30.957+0000 7f7461e6d640 1 --2- 172.21.15.112:0/3810596430 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f745c0fb470 0x7f745c0f7490 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f744c003080 tx=0x7f744c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:51:30.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:30.957+0000 7f74527fc640 1 -- 172.21.15.112:0/3810596430 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f744c038070 con 0x7f745c0fb470 2024-09-16T00:51:30.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:30.957+0000 7f74527fc640 1 -- 172.21.15.112:0/3810596430 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f744c0097e0 con 0x7f745c0fb470 2024-09-16T00:51:30.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:30.957+0000 7f74640d1640 1 -- 172.21.15.112:0/3810596430 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f745c119550 con 0x7f745c0fb470 2024-09-16T00:51:30.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:30.957+0000 7f74527fc640 1 -- 172.21.15.112:0/3810596430 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f744c03d420 con 0x7f745c0fb470 2024-09-16T00:51:30.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:30.957+0000 7f74640d1640 1 -- 172.21.15.112:0/3810596430 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f745c1198d0 con 0x7f745c0fb470 2024-09-16T00:51:30.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:30.958+0000 7f74640d1640 1 -- 172.21.15.112:0/3810596430 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f745c0fcbb0 con 0x7f745c0fb470 2024-09-16T00:51:30.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:30.958+0000 7f74527fc640 1 -- 172.21.15.112:0/3810596430 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f744c030020 con 0x7f745c0fb470 2024-09-16T00:51:30.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:30.959+0000 7f74527fc640 1 --2- 172.21.15.112:0/3810596430 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f744806e530 0x7f74480709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:51:30.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:30.959+0000 7f74527fc640 1 -- 172.21.15.112:0/3810596430 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f744c0ae8a0 con 0x7f745c0fb470 2024-09-16T00:51:30.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:30.959+0000 7f746166c640 1 --2- 172.21.15.112:0/3810596430 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f744806e530 0x7f74480709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:51:30.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:30.959+0000 7f746166c640 1 --2- 172.21.15.112:0/3810596430 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f744806e530 0x7f74480709e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f745c0f8dd0 tx=0x7f7458006d20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:51:30.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:30.963+0000 7f74527fc640 1 -- 172.21.15.112:0/3810596430 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f744c077810 con 0x7f745c0fb470 2024-09-16T00:51:31.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:51:30 smithi112 ceph-mon[97750]: pgmap v714: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:51:31.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:31.163+0000 7f74640d1640 1 -- 172.21.15.112:0/3810596430 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 20, "format": "json"} v 0) v1 -- 0x7f745c04b720 con 0x7f745c0fb470 2024-09-16T00:51:31.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:31.163+0000 7f74527fc640 1 -- 172.21.15.112:0/3810596430 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 20, "format": "json"}]=0 dumped fsmap epoch 20 v41) v1 ==== 107+0+4934 (secure 0 0 0) 0x7f744c07aed0 con 0x7f745c0fb470 2024-09-16T00:51:31.164 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:51:31.164 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":20,"btime":"1970-01-01T00:00:00:000000+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:23:22.328199+0000","modified":"2024-09-16T00:25:13.766466+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14480,"mds_1":24259},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14480":{"gid":14480,"name":"cephfs.smithi112.hxfwkh","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.112:6829/486654885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":486654885},{"type":"v1","addr":"172.21.15.112:6829","nonce":486654885}]},"join_fscid":1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14492":{"gid":14492,"name":"cephfs.smithi112.ithwzj","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6827/381830565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":381830565},{"type":"v1","addr":"172.21.15.112:6827","nonce":381830565}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24259":{"gid":24259,"name":"cephfs.smithi169.gawwgw","rank":1,"incarnation":7,"state":"up:active","state_seq":4,"addr":"172.21.15.169:6825/2892476953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2892476953},{"type":"v1","addr":"172.21.15.169:6825","nonce":2892476953}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24269":{"gid":24269,"name":"cephfs.smithi169.lgeqrj","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.169:6827/861725728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":861725728},{"type":"v1","addr":"172.21.15.169:6827","nonce":861725728}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-16T00:51:31.164 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 20 2024-09-16T00:51:31.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:31.165+0000 7f74640d1640 1 -- 172.21.15.112:0/3810596430 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f744806e530 msgr2=0x7f74480709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:51:31.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:31.165+0000 7f74640d1640 1 --2- 172.21.15.112:0/3810596430 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f744806e530 0x7f74480709e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f745c0f8dd0 tx=0x7f7458006d20 comp rx=0 tx=0).stop 2024-09-16T00:51:31.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:31.165+0000 7f74640d1640 1 -- 172.21.15.112:0/3810596430 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f745c0fb470 msgr2=0x7f745c0f7490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:51:31.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:31.165+0000 7f74640d1640 1 --2- 172.21.15.112:0/3810596430 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f745c0fb470 0x7f745c0f7490 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f744c003080 tx=0x7f744c009da0 comp rx=0 tx=0).stop 2024-09-16T00:51:31.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:31.165+0000 7f74640d1640 1 -- 172.21.15.112:0/3810596430 shutdown_connections 2024-09-16T00:51:31.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:31.165+0000 7f74640d1640 1 --2- 172.21.15.112:0/3810596430 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f744806e530 0x7f74480709e0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:31.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:31.165+0000 7f74640d1640 1 --2- 172.21.15.112:0/3810596430 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f745c1014b0 0x7f745c0f79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:31.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:31.165+0000 7f74640d1640 1 --2- 172.21.15.112:0/3810596430 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f745c0fb470 0x7f745c0f7490 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:31.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:31.165+0000 7f74640d1640 1 -- 172.21.15.112:0/3810596430 >> 172.21.15.112:0/3810596430 conn(0x7f745c0f6ff0 msgr2=0x7f745c065ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:51:31.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:31.165+0000 7f74640d1640 1 -- 172.21.15.112:0/3810596430 shutdown_connections 2024-09-16T00:51:31.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:31.165+0000 7f74640d1640 1 -- 172.21.15.112:0/3810596430 wait complete. 2024-09-16T00:51:31.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:51:30 smithi169 ceph-mon[77626]: pgmap v714: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:51:31.881 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 21 2024-09-16T00:51:32.127 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:51:32.248 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:51:31 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/3810596430' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-09-16T00:51:32.248 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:51:31 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:51:32.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:51:31 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/3810596430' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-09-16T00:51:32.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:51:31 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:51:33.189 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:51:32 smithi112 ceph-mon[97750]: pgmap v715: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:51:33.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:51:32 smithi169 ceph-mon[77626]: pgmap v715: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:51:33.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:33.670+0000 7f79b15ba640 1 -- 172.21.15.112:0/2160927022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79ac102620 msgr2=0x7f79ac1029f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:51:33.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:33.670+0000 7f79b15ba640 1 --2- 172.21.15.112:0/2160927022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79ac102620 0x7f79ac1029f0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f799c0048d0 tx=0x7f799c009da0 comp rx=0 tx=0).stop 2024-09-16T00:51:33.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:33.670+0000 7f79b15ba640 1 -- 172.21.15.112:0/2160927022 shutdown_connections 2024-09-16T00:51:33.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:33.670+0000 7f79b15ba640 1 --2- 172.21.15.112:0/2160927022 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79ac0fac80 0x7f79ac0fb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:33.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:33.670+0000 7f79b15ba640 1 --2- 172.21.15.112:0/2160927022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79ac102620 0x7f79ac1029f0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:33.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:33.670+0000 7f79b15ba640 1 -- 172.21.15.112:0/2160927022 >> 172.21.15.112:0/2160927022 conn(0x7f79ac070ce0 msgr2=0x7f79ac0710e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:51:33.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:33.670+0000 7f79b15ba640 1 -- 172.21.15.112:0/2160927022 shutdown_connections 2024-09-16T00:51:33.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:33.670+0000 7f79b15ba640 1 -- 172.21.15.112:0/2160927022 wait complete. 2024-09-16T00:51:33.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:33.671+0000 7f79b15ba640 1 Processor -- start 2024-09-16T00:51:33.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:33.671+0000 7f79b15ba640 1 -- start start 2024-09-16T00:51:33.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:33.671+0000 7f79b15ba640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79ac0fac80 0x7f79ac1131d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:51:33.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:33.671+0000 7f79b15ba640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79ac102620 0x7f79ac113710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:51:33.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:33.671+0000 7f79b15ba640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ac113da0 con 0x7f79ac102620 2024-09-16T00:51:33.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:33.671+0000 7f79b15ba640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ac10d150 con 0x7f79ac0fac80 2024-09-16T00:51:33.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:33.671+0000 7f79aa7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79ac102620 0x7f79ac113710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:51:33.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:33.671+0000 7f79aa7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79ac102620 0x7f79ac113710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51520/0 (socket says 172.21.15.112:51520) 2024-09-16T00:51:33.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:33.671+0000 7f79aa7fc640 1 -- 172.21.15.112:0/401693286 learned_addr learned my addr 172.21.15.112:0/401693286 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:51:33.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:33.671+0000 7f79aaffd640 1 --2- 172.21.15.112:0/401693286 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79ac0fac80 0x7f79ac1131d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:51:33.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:33.672+0000 7f79aa7fc640 1 -- 172.21.15.112:0/401693286 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79ac0fac80 msgr2=0x7f79ac1131d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:51:33.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:33.672+0000 7f79aa7fc640 1 --2- 172.21.15.112:0/401693286 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79ac0fac80 0x7f79ac1131d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:33.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:33.672+0000 7f79aa7fc640 1 -- 172.21.15.112:0/401693286 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f799c005040 con 0x7f79ac102620 2024-09-16T00:51:33.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:33.672+0000 7f79aa7fc640 1 --2- 172.21.15.112:0/401693286 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79ac102620 0x7f79ac113710 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f79a000b790 tx=0x7f79a00091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:51:33.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:33.672+0000 7f79937fe640 1 -- 172.21.15.112:0/401693286 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f79a000c700 con 0x7f79ac102620 2024-09-16T00:51:33.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:33.672+0000 7f79b15ba640 1 -- 172.21.15.112:0/401693286 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79ac10d430 con 0x7f79ac102620 2024-09-16T00:51:33.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:33.672+0000 7f79937fe640 1 -- 172.21.15.112:0/401693286 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f79a000cd00 con 0x7f79ac102620 2024-09-16T00:51:33.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:33.672+0000 7f79937fe640 1 -- 172.21.15.112:0/401693286 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f79a0015e80 con 0x7f79ac102620 2024-09-16T00:51:33.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:33.672+0000 7f79b15ba640 1 -- 172.21.15.112:0/401693286 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79ac10d950 con 0x7f79ac102620 2024-09-16T00:51:33.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:33.673+0000 7f79b15ba640 1 -- 172.21.15.112:0/401693286 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79ac0657b0 con 0x7f79ac102620 2024-09-16T00:51:33.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:33.674+0000 7f79937fe640 1 -- 172.21.15.112:0/401693286 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f79a0012070 con 0x7f79ac102620 2024-09-16T00:51:33.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:33.674+0000 7f79937fe640 1 --2- 172.21.15.112:0/401693286 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f799406e530 0x7f79940709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:51:33.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:33.674+0000 7f79937fe640 1 -- 172.21.15.112:0/401693286 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f79a008dbe0 con 0x7f79ac102620 2024-09-16T00:51:33.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:33.674+0000 7f79aaffd640 1 --2- 172.21.15.112:0/401693286 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f799406e530 0x7f79940709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:51:33.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:33.675+0000 7f79aaffd640 1 --2- 172.21.15.112:0/401693286 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f799406e530 0x7f79940709e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f799c009ac0 tx=0x7f799c035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:51:33.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:33.678+0000 7f79937fe640 1 -- 172.21.15.112:0/401693286 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f79a0056b50 con 0x7f79ac102620 2024-09-16T00:51:33.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:33.880+0000 7f79b15ba640 1 -- 172.21.15.112:0/401693286 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 21, "format": "json"} v 0) v1 -- 0x7f79ac1095f0 con 0x7f79ac102620 2024-09-16T00:51:33.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:33.881+0000 7f79937fe640 1 -- 172.21.15.112:0/401693286 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 21, "format": "json"}]=0 dumped fsmap epoch 21 v41) v1 ==== 107+0+4935 (secure 0 0 0) 0x7f79a005a210 con 0x7f79ac102620 2024-09-16T00:51:33.881 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:51:33.881 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":21,"btime":"1970-01-01T00:00:00:000000+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:23:22.328199+0000","modified":"2024-09-16T00:26:38.963153+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14480,"mds_1":24259},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14480":{"gid":14480,"name":"cephfs.smithi112.hxfwkh","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.112:6829/486654885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":486654885},{"type":"v1","addr":"172.21.15.112:6829","nonce":486654885}]},"join_fscid":1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14492":{"gid":14492,"name":"cephfs.smithi112.ithwzj","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6827/381830565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":381830565},{"type":"v1","addr":"172.21.15.112:6827","nonce":381830565}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24259":{"gid":24259,"name":"cephfs.smithi169.gawwgw","rank":1,"incarnation":7,"state":"up:active","state_seq":4,"addr":"172.21.15.169:6825/2892476953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2892476953},{"type":"v1","addr":"172.21.15.169:6825","nonce":2892476953}]},"join_fscid":1,"export_targets":[0],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24269":{"gid":24269,"name":"cephfs.smithi169.lgeqrj","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.169:6827/861725728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":861725728},{"type":"v1","addr":"172.21.15.169:6827","nonce":861725728}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-16T00:51:33.881 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 21 2024-09-16T00:51:33.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:33.883+0000 7f79b15ba640 1 -- 172.21.15.112:0/401693286 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f799406e530 msgr2=0x7f79940709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:51:33.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:33.883+0000 7f79b15ba640 1 --2- 172.21.15.112:0/401693286 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f799406e530 0x7f79940709e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f799c009ac0 tx=0x7f799c035040 comp rx=0 tx=0).stop 2024-09-16T00:51:33.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:33.883+0000 7f79b15ba640 1 -- 172.21.15.112:0/401693286 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79ac102620 msgr2=0x7f79ac113710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:51:33.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:33.883+0000 7f79b15ba640 1 --2- 172.21.15.112:0/401693286 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79ac102620 0x7f79ac113710 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f79a000b790 tx=0x7f79a00091d0 comp rx=0 tx=0).stop 2024-09-16T00:51:33.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:33.883+0000 7f79b15ba640 1 -- 172.21.15.112:0/401693286 shutdown_connections 2024-09-16T00:51:33.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:33.883+0000 7f79b15ba640 1 --2- 172.21.15.112:0/401693286 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f799406e530 0x7f79940709e0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:33.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:33.883+0000 7f79b15ba640 1 --2- 172.21.15.112:0/401693286 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79ac102620 0x7f79ac113710 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:33.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:33.883+0000 7f79b15ba640 1 --2- 172.21.15.112:0/401693286 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79ac0fac80 0x7f79ac1131d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:33.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:33.883+0000 7f79b15ba640 1 -- 172.21.15.112:0/401693286 >> 172.21.15.112:0/401693286 conn(0x7f79ac070ce0 msgr2=0x7f79ac105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:51:33.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:33.883+0000 7f79b15ba640 1 -- 172.21.15.112:0/401693286 shutdown_connections 2024-09-16T00:51:33.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:33.883+0000 7f79b15ba640 1 -- 172.21.15.112:0/401693286 wait complete. 2024-09-16T00:51:34.180 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:51:33 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/401693286' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-09-16T00:51:34.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:51:33 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/401693286' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-09-16T00:51:34.430 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 22 2024-09-16T00:51:34.678 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:51:35.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:51:34 smithi112 ceph-mon[97750]: pgmap v716: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:51:35.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:51:34 smithi169 ceph-mon[77626]: pgmap v716: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:51:36.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:36.226+0000 7fe1c5336640 1 -- 172.21.15.112:0/2987508847 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1c0065c10 msgr2=0x7fe1c00fe0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:51:36.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:36.226+0000 7fe1c5336640 1 --2- 172.21.15.112:0/2987508847 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1c0065c10 0x7fe1c00fe0d0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fe1b40048d0 tx=0x7fe1b4009da0 comp rx=0 tx=0).stop 2024-09-16T00:51:36.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:36.226+0000 7fe1c5336640 1 -- 172.21.15.112:0/2987508847 shutdown_connections 2024-09-16T00:51:36.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:36.226+0000 7fe1c5336640 1 --2- 172.21.15.112:0/2987508847 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1c0065c10 0x7fe1c00fe0d0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:36.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:36.226+0000 7fe1c5336640 1 --2- 172.21.15.112:0/2987508847 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1c0065300 0x7fe1c00656d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:36.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:36.226+0000 7fe1c5336640 1 -- 172.21.15.112:0/2987508847 >> 172.21.15.112:0/2987508847 conn(0x7fe1c00723b0 msgr2=0x7fe1c00727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:51:36.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:36.226+0000 7fe1c5336640 1 -- 172.21.15.112:0/2987508847 shutdown_connections 2024-09-16T00:51:36.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:36.226+0000 7fe1c5336640 1 -- 172.21.15.112:0/2987508847 wait complete. 2024-09-16T00:51:36.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:36.227+0000 7fe1c5336640 1 Processor -- start 2024-09-16T00:51:36.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:36.227+0000 7fe1c5336640 1 -- start start 2024-09-16T00:51:36.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:36.227+0000 7fe1c5336640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1c0065300 0x7fe1c01131b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:51:36.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:36.227+0000 7fe1c5336640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1c0065c10 0x7fe1c01136f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:51:36.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:36.227+0000 7fe1c5336640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c0113cf0 con 0x7fe1c0065c10 2024-09-16T00:51:36.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:36.227+0000 7fe1c5336640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c010d130 con 0x7fe1c0065300 2024-09-16T00:51:36.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:36.227+0000 7fe1be7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1c0065c10 0x7fe1c01136f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:51:36.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:36.227+0000 7fe1be7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1c0065c10 0x7fe1c01136f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51544/0 (socket says 172.21.15.112:51544) 2024-09-16T00:51:36.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:36.227+0000 7fe1be7fc640 1 -- 172.21.15.112:0/1355344068 learned_addr learned my addr 172.21.15.112:0/1355344068 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:51:36.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:36.227+0000 7fe1beffd640 1 --2- 172.21.15.112:0/1355344068 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1c0065300 0x7fe1c01131b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:51:36.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:36.227+0000 7fe1be7fc640 1 -- 172.21.15.112:0/1355344068 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1c0065300 msgr2=0x7fe1c01131b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:51:36.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:36.227+0000 7fe1be7fc640 1 --2- 172.21.15.112:0/1355344068 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1c0065300 0x7fe1c01131b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:36.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:36.227+0000 7fe1be7fc640 1 -- 172.21.15.112:0/1355344068 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1b4005040 con 0x7fe1c0065c10 2024-09-16T00:51:36.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:36.228+0000 7fe1be7fc640 1 --2- 172.21.15.112:0/1355344068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1c0065c10 0x7fe1c01136f0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fe1b4000c00 tx=0x7fe1b4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:51:36.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:36.228+0000 7fe1a77fe640 1 -- 172.21.15.112:0/1355344068 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1b4038070 con 0x7fe1c0065c10 2024-09-16T00:51:36.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:36.228+0000 7fe1c5336640 1 -- 172.21.15.112:0/1355344068 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1c010d3b0 con 0x7fe1c0065c10 2024-09-16T00:51:36.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:36.228+0000 7fe1a77fe640 1 -- 172.21.15.112:0/1355344068 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fe1b402ade0 con 0x7fe1c0065c10 2024-09-16T00:51:36.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:36.228+0000 7fe1a77fe640 1 -- 172.21.15.112:0/1355344068 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1b402c7e0 con 0x7fe1c0065c10 2024-09-16T00:51:36.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:36.228+0000 7fe1c5336640 1 -- 172.21.15.112:0/1355344068 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1c010d8f0 con 0x7fe1c0065c10 2024-09-16T00:51:36.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:36.229+0000 7fe1c5336640 1 -- 172.21.15.112:0/1355344068 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1c00fdce0 con 0x7fe1c0065c10 2024-09-16T00:51:36.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:36.229+0000 7fe1a77fe640 1 -- 172.21.15.112:0/1355344068 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fe1b4030020 con 0x7fe1c0065c10 2024-09-16T00:51:36.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:36.230+0000 7fe1a77fe640 1 --2- 172.21.15.112:0/1355344068 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fe1a806e530 0x7fe1a80709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:51:36.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:36.230+0000 7fe1a77fe640 1 -- 172.21.15.112:0/1355344068 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7fe1b40adff0 con 0x7fe1c0065c10 2024-09-16T00:51:36.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:36.230+0000 7fe1beffd640 1 --2- 172.21.15.112:0/1355344068 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fe1a806e530 0x7fe1a80709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:51:36.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:36.230+0000 7fe1beffd640 1 --2- 172.21.15.112:0/1355344068 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fe1a806e530 0x7fe1a80709e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0004fd0 tx=0x7fe1b0009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:51:36.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:36.234+0000 7fe1a77fe640 1 -- 172.21.15.112:0/1355344068 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fe1b4076f90 con 0x7fe1c0065c10 2024-09-16T00:51:36.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:36.434+0000 7fe1c5336640 1 -- 172.21.15.112:0/1355344068 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 22, "format": "json"} v 0) v1 -- 0x7fe1c010e820 con 0x7fe1c0065c10 2024-09-16T00:51:36.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:36.435+0000 7fe1a77fe640 1 -- 172.21.15.112:0/1355344068 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 22, "format": "json"}]=0 dumped fsmap epoch 22 v41) v1 ==== 107+0+4949 (secure 0 0 0) 0x7fe1b407a650 con 0x7fe1c0065c10 2024-09-16T00:51:36.435 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:51:36.436 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":22,"btime":"2024-09-16T00:29:54:709333+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:23:22.328199+0000","modified":"2024-09-16T00:29:54.709330+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14480,"mds_1":24259},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14480":{"gid":14480,"name":"cephfs.smithi112.hxfwkh","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.112:6829/486654885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":486654885},{"type":"v1","addr":"172.21.15.112:6829","nonce":486654885}]},"join_fscid":1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14492":{"gid":14492,"name":"cephfs.smithi112.ithwzj","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6827/381830565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":381830565},{"type":"v1","addr":"172.21.15.112:6827","nonce":381830565}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24259":{"gid":24259,"name":"cephfs.smithi169.gawwgw","rank":1,"incarnation":7,"state":"up:active","state_seq":4,"addr":"172.21.15.169:6825/2892476953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2892476953},{"type":"v1","addr":"172.21.15.169:6825","nonce":2892476953}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24269":{"gid":24269,"name":"cephfs.smithi169.lgeqrj","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.169:6827/861725728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":861725728},{"type":"v1","addr":"172.21.15.169:6827","nonce":861725728}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14480,"qdb_cluster":[14480,24259]},"id":1}]} 2024-09-16T00:51:36.436 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 22 2024-09-16T00:51:36.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:36.437+0000 7fe1c5336640 1 -- 172.21.15.112:0/1355344068 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fe1a806e530 msgr2=0x7fe1a80709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:51:36.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:36.437+0000 7fe1c5336640 1 --2- 172.21.15.112:0/1355344068 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fe1a806e530 0x7fe1a80709e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0004fd0 tx=0x7fe1b0009040 comp rx=0 tx=0).stop 2024-09-16T00:51:36.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:36.437+0000 7fe1c5336640 1 -- 172.21.15.112:0/1355344068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1c0065c10 msgr2=0x7fe1c01136f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:51:36.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:36.437+0000 7fe1c5336640 1 --2- 172.21.15.112:0/1355344068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1c0065c10 0x7fe1c01136f0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fe1b4000c00 tx=0x7fe1b4009da0 comp rx=0 tx=0).stop 2024-09-16T00:51:36.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:36.437+0000 7fe1c5336640 1 -- 172.21.15.112:0/1355344068 shutdown_connections 2024-09-16T00:51:36.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:36.437+0000 7fe1c5336640 1 --2- 172.21.15.112:0/1355344068 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fe1a806e530 0x7fe1a80709e0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:36.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:36.437+0000 7fe1c5336640 1 --2- 172.21.15.112:0/1355344068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1c0065c10 0x7fe1c01136f0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:36.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:36.437+0000 7fe1c5336640 1 --2- 172.21.15.112:0/1355344068 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1c0065300 0x7fe1c01131b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:36.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:36.437+0000 7fe1c5336640 1 -- 172.21.15.112:0/1355344068 >> 172.21.15.112:0/1355344068 conn(0x7fe1c00723b0 msgr2=0x7fe1c00fa2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:51:36.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:36.437+0000 7fe1c5336640 1 -- 172.21.15.112:0/1355344068 shutdown_connections 2024-09-16T00:51:36.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:36.437+0000 7fe1c5336640 1 -- 172.21.15.112:0/1355344068 wait complete. 2024-09-16T00:51:37.051 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 23 2024-09-16T00:51:37.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:51:36 smithi169 ceph-mon[77626]: pgmap v717: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:51:37.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:51:36 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/1355344068' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-09-16T00:51:37.294 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:51:37.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:51:36 smithi112 ceph-mon[97750]: pgmap v717: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:51:37.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:51:36 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/1355344068' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-09-16T00:51:39.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:38.984+0000 7f32f4f82640 1 -- 172.21.15.112:0/2374283941 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32f0102f20 msgr2=0x7f32f00fc5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:51:39.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:38.984+0000 7f32f4f82640 1 --2- 172.21.15.112:0/2374283941 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32f0102f20 0x7f32f00fc5e0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f32d80048d0 tx=0x7f32d8009da0 comp rx=0 tx=0).stop 2024-09-16T00:51:39.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:38.985+0000 7f32f4f82640 1 -- 172.21.15.112:0/2374283941 shutdown_connections 2024-09-16T00:51:39.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:38.985+0000 7f32f4f82640 1 --2- 172.21.15.112:0/2374283941 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32f0102f20 0x7f32f00fc5e0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:39.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:38.985+0000 7f32f4f82640 1 --2- 172.21.15.112:0/2374283941 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32f0102580 0x7f32f0102950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:39.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:38.985+0000 7f32f4f82640 1 -- 172.21.15.112:0/2374283941 >> 172.21.15.112:0/2374283941 conn(0x7f32f00f81a0 msgr2=0x7f32f00fa5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:51:39.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:38.985+0000 7f32f4f82640 1 -- 172.21.15.112:0/2374283941 shutdown_connections 2024-09-16T00:51:39.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:38.985+0000 7f32f4f82640 1 -- 172.21.15.112:0/2374283941 wait complete. 2024-09-16T00:51:39.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:38.985+0000 7f32f4f82640 1 Processor -- start 2024-09-16T00:51:39.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:38.985+0000 7f32f4f82640 1 -- start start 2024-09-16T00:51:39.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:38.985+0000 7f32f4f82640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32f0102580 0x7f32f006fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:51:39.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:38.985+0000 7f32f4f82640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32f0102f20 0x7f32f006ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:51:39.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:38.985+0000 7f32f4f82640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f0070490 con 0x7f32f0102f20 2024-09-16T00:51:39.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:38.985+0000 7f32f4f82640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f0070600 con 0x7f32f0102580 2024-09-16T00:51:39.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:38.986+0000 7f32edd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32f0102f20 0x7f32f006ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:51:39.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:38.986+0000 7f32edd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32f0102f20 0x7f32f006ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51568/0 (socket says 172.21.15.112:51568) 2024-09-16T00:51:39.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:38.986+0000 7f32edd9b640 1 -- 172.21.15.112:0/2729606001 learned_addr learned my addr 172.21.15.112:0/2729606001 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:51:39.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:38.986+0000 7f32ee59c640 1 --2- 172.21.15.112:0/2729606001 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32f0102580 0x7f32f006fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:51:39.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:38.986+0000 7f32edd9b640 1 -- 172.21.15.112:0/2729606001 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32f0102580 msgr2=0x7f32f006fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:51:39.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:38.986+0000 7f32edd9b640 1 --2- 172.21.15.112:0/2729606001 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32f0102580 0x7f32f006fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:39.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:38.986+0000 7f32edd9b640 1 -- 172.21.15.112:0/2729606001 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d8005040 con 0x7f32f0102f20 2024-09-16T00:51:39.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:38.986+0000 7f32edd9b640 1 --2- 172.21.15.112:0/2729606001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32f0102f20 0x7f32f006ff50 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f32d802a530 tx=0x7f32d8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:51:39.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:38.986+0000 7f32e6ffd640 1 -- 172.21.15.112:0/2729606001 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32d8038070 con 0x7f32f0102f20 2024-09-16T00:51:39.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:38.986+0000 7f32e6ffd640 1 -- 172.21.15.112:0/2729606001 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f32d802ade0 con 0x7f32f0102f20 2024-09-16T00:51:39.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:38.986+0000 7f32e6ffd640 1 -- 172.21.15.112:0/2729606001 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32d802c7a0 con 0x7f32f0102f20 2024-09-16T00:51:39.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:38.986+0000 7f32f4f82640 1 -- 172.21.15.112:0/2729606001 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32f0070830 con 0x7f32f0102f20 2024-09-16T00:51:39.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:38.986+0000 7f32f4f82640 1 -- 172.21.15.112:0/2729606001 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32f01198a0 con 0x7f32f0102f20 2024-09-16T00:51:39.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:38.987+0000 7f32f4f82640 1 -- 172.21.15.112:0/2729606001 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32f00fdd00 con 0x7f32f0102f20 2024-09-16T00:51:39.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:38.988+0000 7f32e6ffd640 1 -- 172.21.15.112:0/2729606001 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f32d8030020 con 0x7f32f0102f20 2024-09-16T00:51:39.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:38.989+0000 7f32e6ffd640 1 --2- 172.21.15.112:0/2729606001 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f32dc06e530 0x7f32dc0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:51:39.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:38.989+0000 7f32e6ffd640 1 -- 172.21.15.112:0/2729606001 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f32d80ae7d0 con 0x7f32f0102f20 2024-09-16T00:51:39.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:38.989+0000 7f32ee59c640 1 --2- 172.21.15.112:0/2729606001 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f32dc06e530 0x7f32dc0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:51:39.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:38.989+0000 7f32ee59c640 1 --2- 172.21.15.112:0/2729606001 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f32dc06e530 0x7f32dc0709e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f32e0009af0 tx=0x7f32e000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:51:39.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:38.993+0000 7f32e6ffd640 1 -- 172.21.15.112:0/2729606001 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f32d8077860 con 0x7f32f0102f20 2024-09-16T00:51:39.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:51:38 smithi112 ceph-mon[97750]: pgmap v718: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:51:39.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:51:38 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:51:39.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:51:38 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:51:39.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:51:38 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:51:39.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:51:38 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:51:39.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:39.196+0000 7f32f4f82640 1 -- 172.21.15.112:0/2729606001 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 23, "format": "json"} v 0) v1 -- 0x7f32f00fbda0 con 0x7f32f0102f20 2024-09-16T00:51:39.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:39.197+0000 7f32e6ffd640 1 -- 172.21.15.112:0/2729606001 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 23, "format": "json"}]=0 dumped fsmap epoch 23 v41) v1 ==== 107+0+4948 (secure 0 0 0) 0x7f32d807af20 con 0x7f32f0102f20 2024-09-16T00:51:39.197 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:51:39.197 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":23,"btime":"2024-09-16T00:30:08:788506+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:23:22.328199+0000","modified":"2024-09-16T00:30:08.396661+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14480,"mds_1":24259},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14480":{"gid":14480,"name":"cephfs.smithi112.hxfwkh","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.112:6829/486654885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":486654885},{"type":"v1","addr":"172.21.15.112:6829","nonce":486654885}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14492":{"gid":14492,"name":"cephfs.smithi112.ithwzj","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6827/381830565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":381830565},{"type":"v1","addr":"172.21.15.112:6827","nonce":381830565}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24259":{"gid":24259,"name":"cephfs.smithi169.gawwgw","rank":1,"incarnation":7,"state":"up:active","state_seq":4,"addr":"172.21.15.169:6825/2892476953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2892476953},{"type":"v1","addr":"172.21.15.169:6825","nonce":2892476953}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24269":{"gid":24269,"name":"cephfs.smithi169.lgeqrj","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.169:6827/861725728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":861725728},{"type":"v1","addr":"172.21.15.169:6827","nonce":861725728}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14480,"qdb_cluster":[24259,14480]},"id":1}]} 2024-09-16T00:51:39.198 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 23 2024-09-16T00:51:39.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:39.199+0000 7f32f4f82640 1 -- 172.21.15.112:0/2729606001 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f32dc06e530 msgr2=0x7f32dc0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:51:39.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:39.199+0000 7f32f4f82640 1 --2- 172.21.15.112:0/2729606001 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f32dc06e530 0x7f32dc0709e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f32e0009af0 tx=0x7f32e000b040 comp rx=0 tx=0).stop 2024-09-16T00:51:39.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:39.199+0000 7f32f4f82640 1 -- 172.21.15.112:0/2729606001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32f0102f20 msgr2=0x7f32f006ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:51:39.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:39.199+0000 7f32f4f82640 1 --2- 172.21.15.112:0/2729606001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32f0102f20 0x7f32f006ff50 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f32d802a530 tx=0x7f32d8009da0 comp rx=0 tx=0).stop 2024-09-16T00:51:39.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:39.199+0000 7f32f4f82640 1 -- 172.21.15.112:0/2729606001 shutdown_connections 2024-09-16T00:51:39.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:39.199+0000 7f32f4f82640 1 --2- 172.21.15.112:0/2729606001 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f32dc06e530 0x7f32dc0709e0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:39.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:39.199+0000 7f32f4f82640 1 --2- 172.21.15.112:0/2729606001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32f0102f20 0x7f32f006ff50 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:39.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:39.199+0000 7f32f4f82640 1 --2- 172.21.15.112:0/2729606001 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32f0102580 0x7f32f006fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:39.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:39.199+0000 7f32f4f82640 1 -- 172.21.15.112:0/2729606001 >> 172.21.15.112:0/2729606001 conn(0x7f32f00f81a0 msgr2=0x7f32f00f91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:51:39.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:39.199+0000 7f32f4f82640 1 -- 172.21.15.112:0/2729606001 shutdown_connections 2024-09-16T00:51:39.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:39.199+0000 7f32f4f82640 1 -- 172.21.15.112:0/2729606001 wait complete. 2024-09-16T00:51:39.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:51:38 smithi169 ceph-mon[77626]: pgmap v718: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:51:39.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:51:38 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:51:39.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:51:38 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:51:39.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:51:38 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:51:39.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:51:38 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:51:39.794 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 24 2024-09-16T00:51:40.041 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:51:40.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:51:39 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/2729606001' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-09-16T00:51:40.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:51:39 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/2729606001' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-09-16T00:51:41.125 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:51:40 smithi112 ceph-mon[97750]: pgmap v719: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:51:41.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:51:40 smithi169 ceph-mon[77626]: pgmap v719: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:51:41.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:41.639+0000 7fc144075640 1 -- 172.21.15.112:0/240579304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc13c0fe460 msgr2=0x7fc13c0fe830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:51:41.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:41.639+0000 7fc144075640 1 --2- 172.21.15.112:0/240579304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc13c0fe460 0x7fc13c0fe830 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fc12c009c50 tx=0x7fc12c004d10 comp rx=0 tx=0).stop 2024-09-16T00:51:41.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:41.640+0000 7fc144075640 1 -- 172.21.15.112:0/240579304 shutdown_connections 2024-09-16T00:51:41.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:41.640+0000 7fc144075640 1 --2- 172.21.15.112:0/240579304 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc13c064fc0 0x7fc13c065430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:41.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:41.640+0000 7fc144075640 1 --2- 172.21.15.112:0/240579304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc13c0fe460 0x7fc13c0fe830 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:41.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:41.640+0000 7fc144075640 1 -- 172.21.15.112:0/240579304 >> 172.21.15.112:0/240579304 conn(0x7fc13c0723b0 msgr2=0x7fc13c0727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:51:41.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:41.640+0000 7fc144075640 1 -- 172.21.15.112:0/240579304 shutdown_connections 2024-09-16T00:51:41.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:41.640+0000 7fc144075640 1 -- 172.21.15.112:0/240579304 wait complete. 2024-09-16T00:51:41.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:41.640+0000 7fc144075640 1 Processor -- start 2024-09-16T00:51:41.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:41.640+0000 7fc144075640 1 -- start start 2024-09-16T00:51:41.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:41.640+0000 7fc144075640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc13c064fc0 0x7fc13c1131b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:51:41.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:41.640+0000 7fc144075640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc13c0fe460 0x7fc13c1136f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:51:41.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:41.640+0000 7fc144075640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc13c113cf0 con 0x7fc13c0fe460 2024-09-16T00:51:41.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:41.640+0000 7fc144075640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc13c10d130 con 0x7fc13c064fc0 2024-09-16T00:51:41.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:41.640+0000 7fc141610640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc13c0fe460 0x7fc13c1136f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:51:41.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:41.640+0000 7fc141610640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc13c0fe460 0x7fc13c1136f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51588/0 (socket says 172.21.15.112:51588) 2024-09-16T00:51:41.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:41.640+0000 7fc141610640 1 -- 172.21.15.112:0/2005539999 learned_addr learned my addr 172.21.15.112:0/2005539999 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:51:41.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:41.641+0000 7fc141610640 1 -- 172.21.15.112:0/2005539999 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc13c064fc0 msgr2=0x7fc13c1131b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:51:41.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:41.641+0000 7fc141610640 1 --2- 172.21.15.112:0/2005539999 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc13c064fc0 0x7fc13c1131b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:41.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:41.641+0000 7fc141610640 1 -- 172.21.15.112:0/2005539999 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc12c0098b0 con 0x7fc13c0fe460 2024-09-16T00:51:41.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:41.641+0000 7fc141610640 1 --2- 172.21.15.112:0/2005539999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc13c0fe460 0x7fc13c1136f0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fc130002a50 tx=0x7fc130004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:51:41.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:41.641+0000 7fc13a7fc640 1 -- 172.21.15.112:0/2005539999 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc130007c30 con 0x7fc13c0fe460 2024-09-16T00:51:41.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:41.641+0000 7fc13a7fc640 1 -- 172.21.15.112:0/2005539999 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fc13000f040 con 0x7fc13c0fe460 2024-09-16T00:51:41.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:41.641+0000 7fc13a7fc640 1 -- 172.21.15.112:0/2005539999 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc13000da90 con 0x7fc13c0fe460 2024-09-16T00:51:41.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:41.641+0000 7fc144075640 1 -- 172.21.15.112:0/2005539999 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc13c10d410 con 0x7fc13c0fe460 2024-09-16T00:51:41.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:41.641+0000 7fc144075640 1 -- 172.21.15.112:0/2005539999 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc13c10d9b0 con 0x7fc13c0fe460 2024-09-16T00:51:41.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:41.642+0000 7fc144075640 1 -- 172.21.15.112:0/2005539999 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc13c0fdcf0 con 0x7fc13c0fe460 2024-09-16T00:51:41.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:41.642+0000 7fc13a7fc640 1 -- 172.21.15.112:0/2005539999 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fc13000dd00 con 0x7fc13c0fe460 2024-09-16T00:51:41.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:41.643+0000 7fc13a7fc640 1 --2- 172.21.15.112:0/2005539999 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fc12406e530 0x7fc1240709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:51:41.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:41.643+0000 7fc13a7fc640 1 -- 172.21.15.112:0/2005539999 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7fc13008dc50 con 0x7fc13c0fe460 2024-09-16T00:51:41.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:41.643+0000 7fc141e11640 1 --2- 172.21.15.112:0/2005539999 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fc12406e530 0x7fc1240709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:51:41.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:41.643+0000 7fc141e11640 1 --2- 172.21.15.112:0/2005539999 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fc12406e530 0x7fc1240709e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc12c004f50 tx=0x7fc12c004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:51:41.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:41.647+0000 7fc13a7fc640 1 -- 172.21.15.112:0/2005539999 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fc130056b60 con 0x7fc13c0fe460 2024-09-16T00:51:41.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:41.849+0000 7fc144075640 1 -- 172.21.15.112:0/2005539999 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 24, "format": "json"} v 0) v1 -- 0x7fc13c10e850 con 0x7fc13c0fe460 2024-09-16T00:51:41.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:41.850+0000 7fc13a7fc640 1 -- 172.21.15.112:0/2005539999 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 24, "format": "json"}]=0 dumped fsmap epoch 24 v41) v1 ==== 107+0+3376 (secure 0 0 0) 0x7fc130017d40 con 0x7fc13c0fe460 2024-09-16T00:51:41.851 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:51:41.851 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":24,"btime":"2024-09-16T00:33:28:871479+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:23:22.328199+0000","modified":"2024-09-16T00:33:27.874201+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14480,"mds_1":24259},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14480":{"gid":14480,"name":"cephfs.smithi112.hxfwkh","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.112:6829/486654885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":486654885},{"type":"v1","addr":"172.21.15.112:6829","nonce":486654885}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24259":{"gid":24259,"name":"cephfs.smithi169.gawwgw","rank":1,"incarnation":7,"state":"up:active","state_seq":4,"addr":"172.21.15.169:6825/2892476953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2892476953},{"type":"v1","addr":"172.21.15.169:6825","nonce":2892476953}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14480,"qdb_cluster":[14480,24259]},"id":1}]} 2024-09-16T00:51:41.851 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 24 2024-09-16T00:51:41.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:41.852+0000 7fc144075640 1 -- 172.21.15.112:0/2005539999 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fc12406e530 msgr2=0x7fc1240709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:51:41.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:41.852+0000 7fc144075640 1 --2- 172.21.15.112:0/2005539999 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fc12406e530 0x7fc1240709e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc12c004f50 tx=0x7fc12c004d10 comp rx=0 tx=0).stop 2024-09-16T00:51:41.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:41.852+0000 7fc144075640 1 -- 172.21.15.112:0/2005539999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc13c0fe460 msgr2=0x7fc13c1136f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:51:41.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:41.852+0000 7fc144075640 1 --2- 172.21.15.112:0/2005539999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc13c0fe460 0x7fc13c1136f0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fc130002a50 tx=0x7fc130004600 comp rx=0 tx=0).stop 2024-09-16T00:51:41.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:41.852+0000 7fc144075640 1 -- 172.21.15.112:0/2005539999 shutdown_connections 2024-09-16T00:51:41.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:41.852+0000 7fc144075640 1 --2- 172.21.15.112:0/2005539999 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fc12406e530 0x7fc1240709e0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:41.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:41.852+0000 7fc144075640 1 --2- 172.21.15.112:0/2005539999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc13c0fe460 0x7fc13c1136f0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:41.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:41.852+0000 7fc144075640 1 --2- 172.21.15.112:0/2005539999 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc13c064fc0 0x7fc13c1131b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:41.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:41.852+0000 7fc144075640 1 -- 172.21.15.112:0/2005539999 >> 172.21.15.112:0/2005539999 conn(0x7fc13c0723b0 msgr2=0x7fc13c105410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:51:41.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:41.852+0000 7fc144075640 1 -- 172.21.15.112:0/2005539999 shutdown_connections 2024-09-16T00:51:41.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:41.852+0000 7fc144075640 1 -- 172.21.15.112:0/2005539999 wait complete. 2024-09-16T00:51:42.165 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:51:41 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/2005539999' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-09-16T00:51:42.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:51:41 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/2005539999' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-09-16T00:51:42.490 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 24 2024-09-16T00:51:42.490 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 25 2024-09-16T00:51:42.737 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:51:43.110 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:51:42 smithi112 ceph-mon[97750]: pgmap v720: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:51:43.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:51:42 smithi169 ceph-mon[77626]: pgmap v720: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:51:44.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:51:44 smithi169 ceph-mon[77626]: pgmap v721: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-09-16T00:51:44.301 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:51:44 smithi112 ceph-mon[97750]: pgmap v721: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-09-16T00:51:44.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:44.379+0000 7f7e281f9640 1 -- 172.21.15.112:0/1949594545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e20100c70 msgr2=0x7f7e20101040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:51:44.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:44.379+0000 7f7e281f9640 1 --2- 172.21.15.112:0/1949594545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e20100c70 0x7f7e20101040 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f7e140048d0 tx=0x7f7e14009da0 comp rx=0 tx=0).stop 2024-09-16T00:51:44.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:44.379+0000 7f7e281f9640 1 -- 172.21.15.112:0/1949594545 shutdown_connections 2024-09-16T00:51:44.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:44.379+0000 7f7e281f9640 1 --2- 172.21.15.112:0/1949594545 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e200fac80 0x7f7e200fb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:44.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:44.379+0000 7f7e281f9640 1 --2- 172.21.15.112:0/1949594545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e20100c70 0x7f7e20101040 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:44.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:44.379+0000 7f7e281f9640 1 -- 172.21.15.112:0/1949594545 >> 172.21.15.112:0/1949594545 conn(0x7f7e20070ce0 msgr2=0x7f7e200710e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:51:44.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:44.379+0000 7f7e281f9640 1 -- 172.21.15.112:0/1949594545 shutdown_connections 2024-09-16T00:51:44.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:44.379+0000 7f7e281f9640 1 -- 172.21.15.112:0/1949594545 wait complete. 2024-09-16T00:51:44.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:44.380+0000 7f7e281f9640 1 Processor -- start 2024-09-16T00:51:44.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:44.380+0000 7f7e281f9640 1 -- start start 2024-09-16T00:51:44.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:44.380+0000 7f7e281f9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e200fac80 0x7f7e2010e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:51:44.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:44.380+0000 7f7e281f9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e20100c70 0x7f7e2010e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:51:44.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:44.380+0000 7f7e281f9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2010ed20 con 0x7f7e200fac80 2024-09-16T00:51:44.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:44.380+0000 7f7e281f9640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2010ee60 con 0x7f7e20100c70 2024-09-16T00:51:44.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:44.380+0000 7f7e25f95640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e200fac80 0x7f7e2010e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:51:44.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:44.380+0000 7f7e25f95640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e200fac80 0x7f7e2010e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45088/0 (socket says 172.21.15.112:45088) 2024-09-16T00:51:44.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:44.380+0000 7f7e25f95640 1 -- 172.21.15.112:0/2174854758 learned_addr learned my addr 172.21.15.112:0/2174854758 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:51:44.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:44.380+0000 7f7e25f95640 1 -- 172.21.15.112:0/2174854758 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e20100c70 msgr2=0x7f7e2010e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:51:44.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:44.380+0000 7f7e25f95640 1 --2- 172.21.15.112:0/2174854758 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e20100c70 0x7f7e2010e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:44.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:44.380+0000 7f7e25f95640 1 -- 172.21.15.112:0/2174854758 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e14005040 con 0x7f7e200fac80 2024-09-16T00:51:44.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:44.381+0000 7f7e25f95640 1 --2- 172.21.15.112:0/2174854758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e200fac80 0x7f7e2010e190 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f7e14003080 tx=0x7f7e14009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:51:44.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:44.381+0000 7f7e127fc640 1 -- 172.21.15.112:0/2174854758 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e14038070 con 0x7f7e200fac80 2024-09-16T00:51:44.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:44.381+0000 7f7e281f9640 1 -- 172.21.15.112:0/2174854758 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e20117200 con 0x7f7e200fac80 2024-09-16T00:51:44.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:44.381+0000 7f7e127fc640 1 -- 172.21.15.112:0/2174854758 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f7e140097e0 con 0x7f7e200fac80 2024-09-16T00:51:44.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:44.381+0000 7f7e127fc640 1 -- 172.21.15.112:0/2174854758 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e1403d420 con 0x7f7e200fac80 2024-09-16T00:51:44.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:44.381+0000 7f7e281f9640 1 -- 172.21.15.112:0/2174854758 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e20117660 con 0x7f7e200fac80 2024-09-16T00:51:44.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:44.382+0000 7f7e281f9640 1 -- 172.21.15.112:0/2174854758 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e200fc370 con 0x7f7e200fac80 2024-09-16T00:51:44.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:44.382+0000 7f7e127fc640 1 -- 172.21.15.112:0/2174854758 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f7e14030020 con 0x7f7e200fac80 2024-09-16T00:51:44.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:44.383+0000 7f7e127fc640 1 --2- 172.21.15.112:0/2174854758 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f7e0c06e530 0x7f7e0c0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:51:44.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:44.383+0000 7f7e127fc640 1 -- 172.21.15.112:0/2174854758 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f7e140ae8a0 con 0x7f7e200fac80 2024-09-16T00:51:44.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:44.383+0000 7f7e25794640 1 --2- 172.21.15.112:0/2174854758 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f7e0c06e530 0x7f7e0c0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:51:44.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:44.383+0000 7f7e25794640 1 --2- 172.21.15.112:0/2174854758 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f7e0c06e530 0x7f7e0c0709e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c007fd0 tx=0x7f7e1c007b20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:51:44.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:44.387+0000 7f7e127fc640 1 -- 172.21.15.112:0/2174854758 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f7e14077810 con 0x7f7e200fac80 2024-09-16T00:51:44.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:44.588+0000 7f7e281f9640 1 -- 172.21.15.112:0/2174854758 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 25, "format": "json"} v 0) v1 -- 0x7f7e20074c00 con 0x7f7e200fac80 2024-09-16T00:51:44.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:44.589+0000 7f7e127fc640 1 -- 172.21.15.112:0/2174854758 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 25, "format": "json"}]=0 dumped fsmap epoch 25 v41) v1 ==== 107+0+3343 (secure 0 0 0) 0x7f7e1407aed0 con 0x7f7e200fac80 2024-09-16T00:51:44.589 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:51:44.590 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":25,"btime":"2024-09-16T00:33:29:012993+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34380,"name":"cephfs.smithi112.ithwzj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6827/3994140647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3994140647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3994140647}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":44307,"name":"cephfs.smithi169.lgeqrj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6827/3569474985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3569474985},{"type":"v1","addr":"172.21.15.169:6827","nonce":3569474985}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":25,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:23:22.328199+0000","modified":"2024-09-16T00:33:29.012991+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-16T00:51:44.590 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 25 2024-09-16T00:51:44.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:44.591+0000 7f7e281f9640 1 -- 172.21.15.112:0/2174854758 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f7e0c06e530 msgr2=0x7f7e0c0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:51:44.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:44.591+0000 7f7e281f9640 1 --2- 172.21.15.112:0/2174854758 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f7e0c06e530 0x7f7e0c0709e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c007fd0 tx=0x7f7e1c007b20 comp rx=0 tx=0).stop 2024-09-16T00:51:44.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:44.591+0000 7f7e281f9640 1 -- 172.21.15.112:0/2174854758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e200fac80 msgr2=0x7f7e2010e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:51:44.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:44.591+0000 7f7e281f9640 1 --2- 172.21.15.112:0/2174854758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e200fac80 0x7f7e2010e190 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f7e14003080 tx=0x7f7e14009da0 comp rx=0 tx=0).stop 2024-09-16T00:51:44.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:44.591+0000 7f7e281f9640 1 -- 172.21.15.112:0/2174854758 shutdown_connections 2024-09-16T00:51:44.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:44.591+0000 7f7e281f9640 1 --2- 172.21.15.112:0/2174854758 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f7e0c06e530 0x7f7e0c0709e0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:44.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:44.591+0000 7f7e281f9640 1 --2- 172.21.15.112:0/2174854758 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e20100c70 0x7f7e2010e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:44.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:44.591+0000 7f7e281f9640 1 --2- 172.21.15.112:0/2174854758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e200fac80 0x7f7e2010e190 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:44.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:44.591+0000 7f7e281f9640 1 -- 172.21.15.112:0/2174854758 >> 172.21.15.112:0/2174854758 conn(0x7f7e20070ce0 msgr2=0x7f7e201039c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:51:44.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:44.591+0000 7f7e281f9640 1 -- 172.21.15.112:0/2174854758 shutdown_connections 2024-09-16T00:51:44.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:44.591+0000 7f7e281f9640 1 -- 172.21.15.112:0/2174854758 wait complete. 2024-09-16T00:51:45.230 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:51:45 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/2174854758' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-09-16T00:51:45.255 DEBUG:tasks.fs:max_mds not reduced in epoch 25 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-09-16T00:51:45.256 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 25 2024-09-16T00:51:45.256 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 26 2024-09-16T00:51:45.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:51:45 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/2174854758' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-09-16T00:51:45.513 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:51:46.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:51:46 smithi112 ceph-mon[97750]: pgmap v722: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-09-16T00:51:46.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:51:46 smithi169 ceph-mon[77626]: pgmap v722: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-09-16T00:51:47.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:47.059+0000 7f8830b7b640 1 -- 172.21.15.112:0/120979300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f882c0fb4d0 msgr2=0x7f882c0fb940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:51:47.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:47.059+0000 7f8830b7b640 1 --2- 172.21.15.112:0/120979300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f882c0fb4d0 0x7f882c0fb940 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f88200048d0 tx=0x7f8820009da0 comp rx=0 tx=0).stop 2024-09-16T00:51:47.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:47.060+0000 7f8830b7b640 1 -- 172.21.15.112:0/120979300 shutdown_connections 2024-09-16T00:51:47.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:47.060+0000 7f8830b7b640 1 --2- 172.21.15.112:0/120979300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f882c0fb4d0 0x7f882c0fb940 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:47.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:47.060+0000 7f8830b7b640 1 --2- 172.21.15.112:0/120979300 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f882c101510 0x7f882c1018e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:47.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:47.060+0000 7f8830b7b640 1 -- 172.21.15.112:0/120979300 >> 172.21.15.112:0/120979300 conn(0x7f882c0f7090 msgr2=0x7f882c0f94a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:51:47.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:47.060+0000 7f8830b7b640 1 -- 172.21.15.112:0/120979300 shutdown_connections 2024-09-16T00:51:47.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:47.060+0000 7f8830b7b640 1 -- 172.21.15.112:0/120979300 wait complete. 2024-09-16T00:51:47.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:47.060+0000 7f8830b7b640 1 Processor -- start 2024-09-16T00:51:47.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:47.060+0000 7f8830b7b640 1 -- start start 2024-09-16T00:51:47.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:47.060+0000 7f8830b7b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f882c0fb4d0 0x7f882c1120f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:51:47.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:47.060+0000 7f8830b7b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f882c101510 0x7f882c112630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:51:47.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:47.060+0000 7f8830b7b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f882c112c30 con 0x7f882c101510 2024-09-16T00:51:47.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:47.060+0000 7f8830b7b640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f882c10c070 con 0x7f882c0fb4d0 2024-09-16T00:51:47.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:47.060+0000 7f8829d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f882c101510 0x7f882c112630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:51:47.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:47.060+0000 7f8829d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f882c101510 0x7f882c112630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45112/0 (socket says 172.21.15.112:45112) 2024-09-16T00:51:47.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:47.060+0000 7f8829d9b640 1 -- 172.21.15.112:0/429119338 learned_addr learned my addr 172.21.15.112:0/429119338 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:51:47.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:47.061+0000 7f882a59c640 1 --2- 172.21.15.112:0/429119338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f882c0fb4d0 0x7f882c1120f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:51:47.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:47.061+0000 7f8829d9b640 1 -- 172.21.15.112:0/429119338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f882c0fb4d0 msgr2=0x7f882c1120f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:51:47.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:47.061+0000 7f8829d9b640 1 --2- 172.21.15.112:0/429119338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f882c0fb4d0 0x7f882c1120f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:47.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:47.061+0000 7f8829d9b640 1 -- 172.21.15.112:0/429119338 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8820005040 con 0x7f882c101510 2024-09-16T00:51:47.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:47.061+0000 7f8829d9b640 1 --2- 172.21.15.112:0/429119338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f882c101510 0x7f882c112630 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f8820000c00 tx=0x7f8820009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:51:47.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:47.061+0000 7f881affd640 1 -- 172.21.15.112:0/429119338 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8820038070 con 0x7f882c101510 2024-09-16T00:51:47.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:47.061+0000 7f881affd640 1 -- 172.21.15.112:0/429119338 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f882002ade0 con 0x7f882c101510 2024-09-16T00:51:47.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:47.061+0000 7f8830b7b640 1 -- 172.21.15.112:0/429119338 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f882c10c2f0 con 0x7f882c101510 2024-09-16T00:51:47.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:47.061+0000 7f881affd640 1 -- 172.21.15.112:0/429119338 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f882002c7e0 con 0x7f882c101510 2024-09-16T00:51:47.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:47.061+0000 7f8830b7b640 1 -- 172.21.15.112:0/429119338 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f882c10c830 con 0x7f882c101510 2024-09-16T00:51:47.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:47.062+0000 7f8830b7b640 1 -- 172.21.15.112:0/429119338 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f882c0fcc10 con 0x7f882c101510 2024-09-16T00:51:47.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:47.063+0000 7f881affd640 1 -- 172.21.15.112:0/429119338 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f8820030020 con 0x7f882c101510 2024-09-16T00:51:47.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:47.063+0000 7f881affd640 1 --2- 172.21.15.112:0/429119338 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f881406e530 0x7f88140709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:51:47.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:47.063+0000 7f881affd640 1 -- 172.21.15.112:0/429119338 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f88200adf70 con 0x7f882c101510 2024-09-16T00:51:47.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:47.063+0000 7f882a59c640 1 --2- 172.21.15.112:0/429119338 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f881406e530 0x7f88140709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:51:47.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:47.064+0000 7f882a59c640 1 --2- 172.21.15.112:0/429119338 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f881406e530 0x7f88140709e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f881c004ed0 tx=0x7f881c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:51:47.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:47.068+0000 7f881affd640 1 -- 172.21.15.112:0/429119338 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f88200e31b0 con 0x7f882c101510 2024-09-16T00:51:47.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:47.267+0000 7f8830b7b640 1 -- 172.21.15.112:0/429119338 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 26, "format": "json"} v 0) v1 -- 0x7f882c1084b0 con 0x7f882c101510 2024-09-16T00:51:47.269 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:51:47 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:51:47.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:47.268+0000 7f881affd640 1 -- 172.21.15.112:0/429119338 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 26, "format": "json"}]=0 dumped fsmap epoch 26 v41) v1 ==== 107+0+4907 (secure 0 0 0) 0x7f8820076f90 con 0x7f882c101510 2024-09-16T00:51:47.270 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:51:47.270 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":26,"btime":"2024-09-16T00:33:30:014980+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34380,"name":"cephfs.smithi112.ithwzj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6827/3994140647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3994140647},{"type":"v1","addr":"172.21.15.112:6827","nonce":3994140647}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":44307,"name":"cephfs.smithi169.lgeqrj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6827/3569474985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3569474985},{"type":"v1","addr":"172.21.15.169:6827","nonce":3569474985}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":44315,"name":"cephfs.smithi169.gawwgw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6825/3147876558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3147876558},{"type":"v1","addr":"172.21.15.169:6825","nonce":3147876558}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":44317,"name":"cephfs.smithi112.hxfwkh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6829/1731640001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":1731640001},{"type":"v1","addr":"172.21.15.112:6829","nonce":1731640001}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":25,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:23:22.328199+0000","modified":"2024-09-16T00:33:29.012991+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-16T00:51:47.270 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 26 2024-09-16T00:51:47.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:47.270+0000 7f8830b7b640 1 -- 172.21.15.112:0/429119338 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f881406e530 msgr2=0x7f88140709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:51:47.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:47.270+0000 7f8830b7b640 1 --2- 172.21.15.112:0/429119338 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f881406e530 0x7f88140709e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f881c004ed0 tx=0x7f881c00b040 comp rx=0 tx=0).stop 2024-09-16T00:51:47.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:47.270+0000 7f8830b7b640 1 -- 172.21.15.112:0/429119338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f882c101510 msgr2=0x7f882c112630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:51:47.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:47.270+0000 7f8830b7b640 1 --2- 172.21.15.112:0/429119338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f882c101510 0x7f882c112630 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f8820000c00 tx=0x7f8820009da0 comp rx=0 tx=0).stop 2024-09-16T00:51:47.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:47.271+0000 7f8830b7b640 1 -- 172.21.15.112:0/429119338 shutdown_connections 2024-09-16T00:51:47.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:47.271+0000 7f8830b7b640 1 --2- 172.21.15.112:0/429119338 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f881406e530 0x7f88140709e0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:47.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:47.271+0000 7f8830b7b640 1 --2- 172.21.15.112:0/429119338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f882c101510 0x7f882c112630 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:47.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:47.271+0000 7f8830b7b640 1 --2- 172.21.15.112:0/429119338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f882c0fb4d0 0x7f882c1120f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:47.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:47.271+0000 7f8830b7b640 1 -- 172.21.15.112:0/429119338 >> 172.21.15.112:0/429119338 conn(0x7f882c0f7090 msgr2=0x7f882c0f8940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:51:47.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:47.271+0000 7f8830b7b640 1 -- 172.21.15.112:0/429119338 shutdown_connections 2024-09-16T00:51:47.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:47.271+0000 7f8830b7b640 1 -- 172.21.15.112:0/429119338 wait complete. 2024-09-16T00:51:47.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:51:47 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:51:47.894 DEBUG:tasks.fs:max_mds not reduced in epoch 26 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-09-16T00:51:47.894 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 26 2024-09-16T00:51:47.894 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 27 2024-09-16T00:51:48.144 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:51:48.264 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:51:48 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/429119338' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-09-16T00:51:48.264 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:51:48 smithi112 ceph-mon[97750]: pgmap v723: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-09-16T00:51:48.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:51:48 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/429119338' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-09-16T00:51:48.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:51:48 smithi169 ceph-mon[77626]: pgmap v723: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-09-16T00:51:49.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:49.949+0000 7f1c12ac6640 1 -- 172.21.15.112:0/993858685 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c0c102ed0 msgr2=0x7f1c0c0fc590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:51:49.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:49.949+0000 7f1c12ac6640 1 --2- 172.21.15.112:0/993858685 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c0c102ed0 0x7f1c0c0fc590 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f1c000048d0 tx=0x7f1c00009da0 comp rx=0 tx=0).stop 2024-09-16T00:51:49.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:49.950+0000 7f1c12ac6640 1 -- 172.21.15.112:0/993858685 shutdown_connections 2024-09-16T00:51:49.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:49.950+0000 7f1c12ac6640 1 --2- 172.21.15.112:0/993858685 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c0c102ed0 0x7f1c0c0fc590 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:49.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:49.950+0000 7f1c12ac6640 1 --2- 172.21.15.112:0/993858685 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c0c102530 0x7f1c0c102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:49.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:49.950+0000 7f1c12ac6640 1 -- 172.21.15.112:0/993858685 >> 172.21.15.112:0/993858685 conn(0x7f1c0c0f80f0 msgr2=0x7f1c0c0fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:51:49.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:49.950+0000 7f1c12ac6640 1 -- 172.21.15.112:0/993858685 shutdown_connections 2024-09-16T00:51:49.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:49.950+0000 7f1c12ac6640 1 -- 172.21.15.112:0/993858685 wait complete. 2024-09-16T00:51:49.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:49.950+0000 7f1c12ac6640 1 Processor -- start 2024-09-16T00:51:49.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:49.950+0000 7f1c12ac6640 1 -- start start 2024-09-16T00:51:49.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:49.950+0000 7f1c12ac6640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c0c102530 0x7f1c0c1131f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:51:49.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:49.950+0000 7f1c12ac6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c0c102ed0 0x7f1c0c113730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:51:49.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:49.950+0000 7f1c12ac6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c0c113d30 con 0x7f1c0c102ed0 2024-09-16T00:51:49.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:49.950+0000 7f1c12ac6640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c0c10d170 con 0x7f1c0c102530 2024-09-16T00:51:49.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:49.951+0000 7f1c0bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c0c102ed0 0x7f1c0c113730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:51:49.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:49.951+0000 7f1c0bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c0c102ed0 0x7f1c0c113730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45122/0 (socket says 172.21.15.112:45122) 2024-09-16T00:51:49.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:49.951+0000 7f1c0bfff640 1 -- 172.21.15.112:0/1206192617 learned_addr learned my addr 172.21.15.112:0/1206192617 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:51:49.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:49.951+0000 7f1c10862640 1 --2- 172.21.15.112:0/1206192617 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c0c102530 0x7f1c0c1131f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:51:49.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:49.951+0000 7f1c0bfff640 1 -- 172.21.15.112:0/1206192617 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c0c102530 msgr2=0x7f1c0c1131f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:51:49.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:49.951+0000 7f1c0bfff640 1 --2- 172.21.15.112:0/1206192617 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c0c102530 0x7f1c0c1131f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:49.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:49.951+0000 7f1c0bfff640 1 -- 172.21.15.112:0/1206192617 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c00005040 con 0x7f1c0c102ed0 2024-09-16T00:51:49.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:49.951+0000 7f1c10862640 1 --2- 172.21.15.112:0/1206192617 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c0c102530 0x7f1c0c1131f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:51:49.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:49.951+0000 7f1c0bfff640 1 --2- 172.21.15.112:0/1206192617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c0c102ed0 0x7f1c0c113730 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f1c00003080 tx=0x7f1c00009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:51:49.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:49.951+0000 7f1c097fa640 1 -- 172.21.15.112:0/1206192617 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c00038070 con 0x7f1c0c102ed0 2024-09-16T00:51:49.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:49.951+0000 7f1c12ac6640 1 -- 172.21.15.112:0/1206192617 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c0c10d3f0 con 0x7f1c0c102ed0 2024-09-16T00:51:49.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:49.951+0000 7f1c097fa640 1 -- 172.21.15.112:0/1206192617 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f1c00009760 con 0x7f1c0c102ed0 2024-09-16T00:51:49.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:49.951+0000 7f1c097fa640 1 -- 172.21.15.112:0/1206192617 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c00033420 con 0x7f1c0c102ed0 2024-09-16T00:51:49.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:49.951+0000 7f1c12ac6640 1 -- 172.21.15.112:0/1206192617 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c0c10d8d0 con 0x7f1c0c102ed0 2024-09-16T00:51:49.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:49.952+0000 7f1c12ac6640 1 -- 172.21.15.112:0/1206192617 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c0c0fdcb0 con 0x7f1c0c102ed0 2024-09-16T00:51:49.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:49.953+0000 7f1c097fa640 1 -- 172.21.15.112:0/1206192617 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f1c00030020 con 0x7f1c0c102ed0 2024-09-16T00:51:49.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:49.953+0000 7f1c097fa640 1 --2- 172.21.15.112:0/1206192617 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f1bf406e530 0x7f1bf40709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:51:49.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:49.953+0000 7f1c10862640 1 --2- 172.21.15.112:0/1206192617 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f1bf406e530 0x7f1bf40709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:51:49.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:49.953+0000 7f1c097fa640 1 -- 172.21.15.112:0/1206192617 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f1c000ae0b0 con 0x7f1c0c102ed0 2024-09-16T00:51:49.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:49.954+0000 7f1c10862640 1 --2- 172.21.15.112:0/1206192617 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f1bf406e530 0x7f1bf40709e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc0041d0 tx=0x7f1bfc009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:51:49.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:49.958+0000 7f1c097fa640 1 -- 172.21.15.112:0/1206192617 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f1c00077020 con 0x7f1c0c102ed0 2024-09-16T00:51:50.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:50.161+0000 7f1c12ac6640 1 -- 172.21.15.112:0/1206192617 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 27, "format": "json"} v 0) v1 -- 0x7f1c0c1095f0 con 0x7f1c0c102ed0 2024-09-16T00:51:50.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:50.162+0000 7f1c097fa640 1 -- 172.21.15.112:0/1206192617 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 27, "format": "json"}]=0 dumped fsmap epoch 27 v41) v1 ==== 107+0+4125 (secure 0 0 0) 0x7f1c0007a6e0 con 0x7f1c0c102ed0 2024-09-16T00:51:50.163 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:51:50.163 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":27,"btime":"2024-09-16T00:33:33:021372+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44307,"name":"cephfs.smithi169.lgeqrj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6827/3569474985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3569474985},{"type":"v1","addr":"172.21.15.169:6827","nonce":3569474985}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":44315,"name":"cephfs.smithi169.gawwgw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6825/3147876558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3147876558},{"type":"v1","addr":"172.21.15.169:6825","nonce":3147876558}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":44317,"name":"cephfs.smithi112.hxfwkh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6829/1731640001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":1731640001},{"type":"v1","addr":"172.21.15.112:6829","nonce":1731640001}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":25,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:23:22.328199+0000","modified":"2024-09-16T00:33:29.012991+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-16T00:51:50.164 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 27 2024-09-16T00:51:50.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:50.164+0000 7f1c12ac6640 1 -- 172.21.15.112:0/1206192617 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f1bf406e530 msgr2=0x7f1bf40709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:51:50.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:50.164+0000 7f1c12ac6640 1 --2- 172.21.15.112:0/1206192617 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f1bf406e530 0x7f1bf40709e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc0041d0 tx=0x7f1bfc009040 comp rx=0 tx=0).stop 2024-09-16T00:51:50.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:50.164+0000 7f1c12ac6640 1 -- 172.21.15.112:0/1206192617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c0c102ed0 msgr2=0x7f1c0c113730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:51:50.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:50.164+0000 7f1c12ac6640 1 --2- 172.21.15.112:0/1206192617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c0c102ed0 0x7f1c0c113730 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f1c00003080 tx=0x7f1c00009da0 comp rx=0 tx=0).stop 2024-09-16T00:51:50.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:50.164+0000 7f1c12ac6640 1 -- 172.21.15.112:0/1206192617 shutdown_connections 2024-09-16T00:51:50.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:50.164+0000 7f1c12ac6640 1 --2- 172.21.15.112:0/1206192617 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f1bf406e530 0x7f1bf40709e0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:50.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:50.164+0000 7f1c12ac6640 1 --2- 172.21.15.112:0/1206192617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c0c102ed0 0x7f1c0c113730 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:50.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:50.164+0000 7f1c12ac6640 1 --2- 172.21.15.112:0/1206192617 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c0c102530 0x7f1c0c1131f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:50.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:50.164+0000 7f1c12ac6640 1 -- 172.21.15.112:0/1206192617 >> 172.21.15.112:0/1206192617 conn(0x7f1c0c0f80f0 msgr2=0x7f1c0c0f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:51:50.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:50.164+0000 7f1c12ac6640 1 -- 172.21.15.112:0/1206192617 shutdown_connections 2024-09-16T00:51:50.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:50.164+0000 7f1c12ac6640 1 -- 172.21.15.112:0/1206192617 wait complete. 2024-09-16T00:51:50.818 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:51:50 smithi112 ceph-mon[97750]: pgmap v724: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-16T00:51:50.818 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:51:50 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/1206192617' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-09-16T00:51:50.843 DEBUG:tasks.fs:max_mds not reduced in epoch 27 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-09-16T00:51:50.843 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 27 2024-09-16T00:51:50.843 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 28 2024-09-16T00:51:51.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:51:50 smithi169 ceph-mon[77626]: pgmap v724: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-16T00:51:51.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:51:50 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/1206192617' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-09-16T00:51:51.091 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:51:52.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:52.605+0000 7f40dca72640 1 -- 172.21.15.112:0/2084990997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40d81014e0 msgr2=0x7f40d81018b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:51:52.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:52.605+0000 7f40dca72640 1 --2- 172.21.15.112:0/2084990997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40d81014e0 0x7f40d81018b0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f40c80048d0 tx=0x7f40c8009da0 comp rx=0 tx=0).stop 2024-09-16T00:51:52.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:52.605+0000 7f40dca72640 1 -- 172.21.15.112:0/2084990997 shutdown_connections 2024-09-16T00:51:52.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:52.605+0000 7f40dca72640 1 --2- 172.21.15.112:0/2084990997 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40d80fb4a0 0x7f40d80fb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:52.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:52.605+0000 7f40dca72640 1 --2- 172.21.15.112:0/2084990997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40d81014e0 0x7f40d81018b0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:52.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:52.605+0000 7f40dca72640 1 -- 172.21.15.112:0/2084990997 >> 172.21.15.112:0/2084990997 conn(0x7f40d80f7060 msgr2=0x7f40d80f9470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:51:52.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:52.605+0000 7f40dca72640 1 -- 172.21.15.112:0/2084990997 shutdown_connections 2024-09-16T00:51:52.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:52.605+0000 7f40dca72640 1 -- 172.21.15.112:0/2084990997 wait complete. 2024-09-16T00:51:52.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:52.606+0000 7f40dca72640 1 Processor -- start 2024-09-16T00:51:52.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:52.606+0000 7f40dca72640 1 -- start start 2024-09-16T00:51:52.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:52.606+0000 7f40dca72640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40d80fb4a0 0x7f40d81131c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:51:52.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:52.606+0000 7f40dca72640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40d81014e0 0x7f40d8113700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:51:52.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:52.606+0000 7f40dca72640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d8113d00 con 0x7f40d80fb4a0 2024-09-16T00:51:52.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:52.606+0000 7f40dca72640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d810d140 con 0x7f40d81014e0 2024-09-16T00:51:52.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:52.606+0000 7f40d659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40d80fb4a0 0x7f40d81131c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:51:52.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:52.606+0000 7f40d659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40d80fb4a0 0x7f40d81131c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47538/0 (socket says 172.21.15.112:47538) 2024-09-16T00:51:52.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:52.606+0000 7f40d659c640 1 -- 172.21.15.112:0/3330141362 learned_addr learned my addr 172.21.15.112:0/3330141362 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:51:52.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:52.607+0000 7f40d659c640 1 -- 172.21.15.112:0/3330141362 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40d81014e0 msgr2=0x7f40d8113700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:51:52.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:52.607+0000 7f40cffff640 1 --2- 172.21.15.112:0/3330141362 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40d81014e0 0x7f40d8113700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:51:52.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:52.607+0000 7f40d659c640 1 --2- 172.21.15.112:0/3330141362 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40d81014e0 0x7f40d8113700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:52.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:52.607+0000 7f40d659c640 1 -- 172.21.15.112:0/3330141362 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c8005040 con 0x7f40d80fb4a0 2024-09-16T00:51:52.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:52.607+0000 7f40d659c640 1 --2- 172.21.15.112:0/3330141362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40d80fb4a0 0x7f40d81131c0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f40c8000c00 tx=0x7f40c8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:51:52.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:52.607+0000 7f40ceffd640 1 -- 172.21.15.112:0/3330141362 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f40c8038070 con 0x7f40d80fb4a0 2024-09-16T00:51:52.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:52.607+0000 7f40ceffd640 1 -- 172.21.15.112:0/3330141362 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f40c802ade0 con 0x7f40d80fb4a0 2024-09-16T00:51:52.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:52.607+0000 7f40dca72640 1 -- 172.21.15.112:0/3330141362 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40d810d3c0 con 0x7f40d80fb4a0 2024-09-16T00:51:52.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:52.607+0000 7f40ceffd640 1 -- 172.21.15.112:0/3330141362 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f40c802c7e0 con 0x7f40d80fb4a0 2024-09-16T00:51:52.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:52.607+0000 7f40dca72640 1 -- 172.21.15.112:0/3330141362 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40d810d900 con 0x7f40d80fb4a0 2024-09-16T00:51:52.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:52.608+0000 7f40dca72640 1 -- 172.21.15.112:0/3330141362 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40d80fcbe0 con 0x7f40d80fb4a0 2024-09-16T00:51:52.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:52.609+0000 7f40ceffd640 1 -- 172.21.15.112:0/3330141362 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f40c8030020 con 0x7f40d80fb4a0 2024-09-16T00:51:52.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:52.609+0000 7f40ceffd640 1 --2- 172.21.15.112:0/3330141362 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f40c406e530 0x7f40c40709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:51:52.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:52.609+0000 7f40ceffd640 1 -- 172.21.15.112:0/3330141362 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f40c80aded0 con 0x7f40d80fb4a0 2024-09-16T00:51:52.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:52.609+0000 7f40cffff640 1 --2- 172.21.15.112:0/3330141362 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f40c406e530 0x7f40c40709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:51:52.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:52.610+0000 7f40cffff640 1 --2- 172.21.15.112:0/3330141362 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f40c406e530 0x7f40c40709e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f40d810ea80 tx=0x7f40c000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:51:52.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:52.613+0000 7f40ceffd640 1 -- 172.21.15.112:0/3330141362 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f40c8076f70 con 0x7f40d80fb4a0 2024-09-16T00:51:52.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:52.818+0000 7f40dca72640 1 -- 172.21.15.112:0/3330141362 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 28, "format": "json"} v 0) v1 -- 0x7f40d80fac60 con 0x7f40d80fb4a0 2024-09-16T00:51:52.819 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:51:52 smithi112 ceph-mon[97750]: pgmap v725: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-09-16T00:51:52.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:52.819+0000 7f40ceffd640 1 -- 172.21.15.112:0/3330141362 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 28, "format": "json"}]=0 dumped fsmap epoch 28 v41) v1 ==== 107+0+4972 (secure 0 0 0) 0x7f40c807a630 con 0x7f40d80fb4a0 2024-09-16T00:51:52.820 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:51:52.820 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":28,"btime":"2024-09-16T00:33:38:043009+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34392,"name":"cephfs.smithi112.ithwzj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6827/476229221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":476229221},{"type":"v1","addr":"172.21.15.112:6827","nonce":476229221}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":44307,"name":"cephfs.smithi169.lgeqrj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6827/3569474985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3569474985},{"type":"v1","addr":"172.21.15.169:6827","nonce":3569474985}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":44315,"name":"cephfs.smithi169.gawwgw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6825/3147876558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3147876558},{"type":"v1","addr":"172.21.15.169:6825","nonce":3147876558}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":44317,"name":"cephfs.smithi112.hxfwkh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6829/1731640001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":1731640001},{"type":"v1","addr":"172.21.15.112:6829","nonce":1731640001}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":25,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:23:22.328199+0000","modified":"2024-09-16T00:33:29.012991+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-16T00:51:52.820 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 28 2024-09-16T00:51:52.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:52.821+0000 7f40dca72640 1 -- 172.21.15.112:0/3330141362 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f40c406e530 msgr2=0x7f40c40709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:51:52.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:52.821+0000 7f40dca72640 1 --2- 172.21.15.112:0/3330141362 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f40c406e530 0x7f40c40709e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f40d810ea80 tx=0x7f40c000b040 comp rx=0 tx=0).stop 2024-09-16T00:51:52.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:52.821+0000 7f40dca72640 1 -- 172.21.15.112:0/3330141362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40d80fb4a0 msgr2=0x7f40d81131c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:51:52.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:52.821+0000 7f40dca72640 1 --2- 172.21.15.112:0/3330141362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40d80fb4a0 0x7f40d81131c0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f40c8000c00 tx=0x7f40c8009da0 comp rx=0 tx=0).stop 2024-09-16T00:51:52.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:52.821+0000 7f40dca72640 1 -- 172.21.15.112:0/3330141362 shutdown_connections 2024-09-16T00:51:52.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:52.821+0000 7f40dca72640 1 --2- 172.21.15.112:0/3330141362 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f40c406e530 0x7f40c40709e0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:52.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:52.821+0000 7f40dca72640 1 --2- 172.21.15.112:0/3330141362 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40d81014e0 0x7f40d8113700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:52.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:52.821+0000 7f40dca72640 1 --2- 172.21.15.112:0/3330141362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40d80fb4a0 0x7f40d81131c0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:52.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:52.821+0000 7f40dca72640 1 -- 172.21.15.112:0/3330141362 >> 172.21.15.112:0/3330141362 conn(0x7f40d80f7060 msgr2=0x7f40d8065a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:51:52.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:52.821+0000 7f40dca72640 1 -- 172.21.15.112:0/3330141362 shutdown_connections 2024-09-16T00:51:52.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:52.821+0000 7f40dca72640 1 -- 172.21.15.112:0/3330141362 wait complete. 2024-09-16T00:51:53.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:51:52 smithi169 ceph-mon[77626]: pgmap v725: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-09-16T00:51:53.443 DEBUG:tasks.fs:max_mds not reduced in epoch 28 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-09-16T00:51:53.443 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 28 2024-09-16T00:51:53.443 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 29 2024-09-16T00:51:53.692 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:51:54.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:51:53 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/3330141362' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-09-16T00:51:54.069 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:51:53 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/3330141362' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-09-16T00:51:55.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:51:54 smithi169 ceph-mon[77626]: pgmap v726: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-09-16T00:51:55.087 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:51:54 smithi112 ceph-mon[97750]: pgmap v726: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-09-16T00:51:55.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:55.270+0000 7f06558c1640 1 -- 172.21.15.112:0/3872561454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0650102550 msgr2=0x7f0650102920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:51:55.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:55.270+0000 7f06558c1640 1 --2- 172.21.15.112:0/3872561454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0650102550 0x7f0650102920 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f06400048d0 tx=0x7f0640009da0 comp rx=0 tx=0).stop 2024-09-16T00:51:55.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:55.270+0000 7f06558c1640 1 -- 172.21.15.112:0/3872561454 shutdown_connections 2024-09-16T00:51:55.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:55.270+0000 7f06558c1640 1 --2- 172.21.15.112:0/3872561454 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0650102ef0 0x7f06500fc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:55.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:55.270+0000 7f06558c1640 1 --2- 172.21.15.112:0/3872561454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0650102550 0x7f0650102920 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:55.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:55.270+0000 7f06558c1640 1 -- 172.21.15.112:0/3872561454 >> 172.21.15.112:0/3872561454 conn(0x7f06500f80f0 msgr2=0x7f06500fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:51:55.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:55.270+0000 7f06558c1640 1 -- 172.21.15.112:0/3872561454 shutdown_connections 2024-09-16T00:51:55.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:55.270+0000 7f06558c1640 1 -- 172.21.15.112:0/3872561454 wait complete. 2024-09-16T00:51:55.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:55.271+0000 7f06558c1640 1 Processor -- start 2024-09-16T00:51:55.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:55.271+0000 7f06558c1640 1 -- start start 2024-09-16T00:51:55.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:55.271+0000 7f06558c1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0650102550 0x7f065006fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:51:55.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:55.271+0000 7f06558c1640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0650102ef0 0x7f065006ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:51:55.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:55.271+0000 7f06558c1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0650070490 con 0x7f0650102550 2024-09-16T00:51:55.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:55.271+0000 7f06558c1640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0650070600 con 0x7f0650102ef0 2024-09-16T00:51:55.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:55.271+0000 7f064effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0650102550 0x7f065006fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:51:55.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:55.271+0000 7f064effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0650102550 0x7f065006fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47560/0 (socket says 172.21.15.112:47560) 2024-09-16T00:51:55.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:55.271+0000 7f064effd640 1 -- 172.21.15.112:0/1939006760 learned_addr learned my addr 172.21.15.112:0/1939006760 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:51:55.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:55.271+0000 7f06467fc640 1 --2- 172.21.15.112:0/1939006760 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0650102ef0 0x7f065006ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:51:55.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:55.272+0000 7f064effd640 1 -- 172.21.15.112:0/1939006760 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0650102ef0 msgr2=0x7f065006ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:51:55.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:55.272+0000 7f064effd640 1 --2- 172.21.15.112:0/1939006760 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0650102ef0 0x7f065006ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:55.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:55.272+0000 7f064effd640 1 -- 172.21.15.112:0/1939006760 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0640005040 con 0x7f0650102550 2024-09-16T00:51:55.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:55.272+0000 7f06467fc640 1 --2- 172.21.15.112:0/1939006760 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0650102ef0 0x7f065006ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:51:55.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:55.272+0000 7f064effd640 1 --2- 172.21.15.112:0/1939006760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0650102550 0x7f065006fa10 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f0640003080 tx=0x7f0640009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:51:55.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:55.272+0000 7f0647fff640 1 -- 172.21.15.112:0/1939006760 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0640038070 con 0x7f0650102550 2024-09-16T00:51:55.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:55.272+0000 7f06558c1640 1 -- 172.21.15.112:0/1939006760 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0650070830 con 0x7f0650102550 2024-09-16T00:51:55.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:55.272+0000 7f0647fff640 1 -- 172.21.15.112:0/1939006760 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f06400097e0 con 0x7f0650102550 2024-09-16T00:51:55.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:55.272+0000 7f0647fff640 1 -- 172.21.15.112:0/1939006760 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f064003d420 con 0x7f0650102550 2024-09-16T00:51:55.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:55.272+0000 7f06558c1640 1 -- 172.21.15.112:0/1939006760 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0650119bf0 con 0x7f0650102550 2024-09-16T00:51:55.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:55.273+0000 7f06558c1640 1 -- 172.21.15.112:0/1939006760 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06500fdcd0 con 0x7f0650102550 2024-09-16T00:51:55.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:55.273+0000 7f0647fff640 1 -- 172.21.15.112:0/1939006760 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f0640030020 con 0x7f0650102550 2024-09-16T00:51:55.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:55.274+0000 7f0647fff640 1 --2- 172.21.15.112:0/1939006760 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f063c06e530 0x7f063c0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:51:55.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:55.274+0000 7f0647fff640 1 -- 172.21.15.112:0/1939006760 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f06400ae880 con 0x7f0650102550 2024-09-16T00:51:55.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:55.274+0000 7f06467fc640 1 --2- 172.21.15.112:0/1939006760 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f063c06e530 0x7f063c0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:51:55.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:55.274+0000 7f06467fc640 1 --2- 172.21.15.112:0/1939006760 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f063c06e530 0x7f063c0709e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f0650071350 tx=0x7f0638009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:51:55.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:55.278+0000 7f0647fff640 1 -- 172.21.15.112:0/1939006760 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f06400777f0 con 0x7f0650102550 2024-09-16T00:51:55.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:55.481+0000 7f06558c1640 1 -- 172.21.15.112:0/1939006760 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 29, "format": "json"} v 0) v1 -- 0x7f06500fbd50 con 0x7f0650102550 2024-09-16T00:51:55.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:55.481+0000 7f0647fff640 1 -- 172.21.15.112:0/1939006760 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 29, "format": "json"}]=0 dumped fsmap epoch 29 v41) v1 ==== 107+0+4190 (secure 0 0 0) 0x7f064007aeb0 con 0x7f0650102550 2024-09-16T00:51:55.482 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:51:55.482 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":29,"btime":"2024-09-16T00:33:51:608496+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34392,"name":"cephfs.smithi112.ithwzj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6827/476229221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":476229221},{"type":"v1","addr":"172.21.15.112:6827","nonce":476229221}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":44307,"name":"cephfs.smithi169.lgeqrj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6827/3569474985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3569474985},{"type":"v1","addr":"172.21.15.169:6827","nonce":3569474985}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":44315,"name":"cephfs.smithi169.gawwgw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6825/3147876558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3147876558},{"type":"v1","addr":"172.21.15.169:6825","nonce":3147876558}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":25,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:23:22.328199+0000","modified":"2024-09-16T00:33:29.012991+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-16T00:51:55.482 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 29 2024-09-16T00:51:55.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:55.484+0000 7f06558c1640 1 -- 172.21.15.112:0/1939006760 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f063c06e530 msgr2=0x7f063c0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:51:55.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:55.484+0000 7f06558c1640 1 --2- 172.21.15.112:0/1939006760 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f063c06e530 0x7f063c0709e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f0650071350 tx=0x7f0638009040 comp rx=0 tx=0).stop 2024-09-16T00:51:55.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:55.484+0000 7f06558c1640 1 -- 172.21.15.112:0/1939006760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0650102550 msgr2=0x7f065006fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:51:55.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:55.484+0000 7f06558c1640 1 --2- 172.21.15.112:0/1939006760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0650102550 0x7f065006fa10 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f0640003080 tx=0x7f0640009da0 comp rx=0 tx=0).stop 2024-09-16T00:51:55.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:55.484+0000 7f06558c1640 1 -- 172.21.15.112:0/1939006760 shutdown_connections 2024-09-16T00:51:55.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:55.484+0000 7f06558c1640 1 --2- 172.21.15.112:0/1939006760 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f063c06e530 0x7f063c0709e0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:55.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:55.484+0000 7f06558c1640 1 --2- 172.21.15.112:0/1939006760 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0650102ef0 0x7f065006ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:55.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:55.484+0000 7f06558c1640 1 --2- 172.21.15.112:0/1939006760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0650102550 0x7f065006fa10 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:55.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:55.484+0000 7f06558c1640 1 -- 172.21.15.112:0/1939006760 >> 172.21.15.112:0/1939006760 conn(0x7f06500f80f0 msgr2=0x7f06500f90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:51:55.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:55.484+0000 7f06558c1640 1 -- 172.21.15.112:0/1939006760 shutdown_connections 2024-09-16T00:51:55.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:55.484+0000 7f06558c1640 1 -- 172.21.15.112:0/1939006760 wait complete. 2024-09-16T00:51:56.046 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:51:55 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/1939006760' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-09-16T00:51:56.073 DEBUG:tasks.fs:max_mds not reduced in epoch 29 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-09-16T00:51:56.073 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 29 2024-09-16T00:51:56.073 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 30 2024-09-16T00:51:56.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:51:55 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/1939006760' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-09-16T00:51:56.321 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:51:57.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:51:56 smithi112 ceph-mon[97750]: pgmap v727: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:51:57.177 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:51:56 smithi169 ceph-mon[77626]: pgmap v727: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:51:57.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:57.859+0000 7f1d2594c640 1 -- 172.21.15.112:0/637155551 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d20065300 msgr2=0x7f1d200656d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:51:57.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:57.859+0000 7f1d2594c640 1 --2- 172.21.15.112:0/637155551 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d20065300 0x7f1d200656d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f1d080048d0 tx=0x7f1d08009da0 comp rx=0 tx=0).stop 2024-09-16T00:51:57.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:57.859+0000 7f1d2594c640 1 -- 172.21.15.112:0/637155551 shutdown_connections 2024-09-16T00:51:57.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:57.859+0000 7f1d2594c640 1 --2- 172.21.15.112:0/637155551 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d20065c10 0x7f1d200fe0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:57.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:57.859+0000 7f1d2594c640 1 --2- 172.21.15.112:0/637155551 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d20065300 0x7f1d200656d0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:57.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:57.859+0000 7f1d2594c640 1 -- 172.21.15.112:0/637155551 >> 172.21.15.112:0/637155551 conn(0x7f1d200723b0 msgr2=0x7f1d200727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:51:57.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:57.859+0000 7f1d2594c640 1 -- 172.21.15.112:0/637155551 shutdown_connections 2024-09-16T00:51:57.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:57.859+0000 7f1d2594c640 1 -- 172.21.15.112:0/637155551 wait complete. 2024-09-16T00:51:57.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:57.859+0000 7f1d2594c640 1 Processor -- start 2024-09-16T00:51:57.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:57.859+0000 7f1d2594c640 1 -- start start 2024-09-16T00:51:57.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:57.860+0000 7f1d2594c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d20065300 0x7f1d20113230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:51:57.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:57.860+0000 7f1d2594c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d20065c10 0x7f1d20113770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:51:57.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:57.860+0000 7f1d2594c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d20113e00 con 0x7f1d20065c10 2024-09-16T00:51:57.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:57.860+0000 7f1d2594c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d2010d1b0 con 0x7f1d20065300 2024-09-16T00:51:57.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:57.860+0000 7f1d1e7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d20065c10 0x7f1d20113770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:51:57.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:57.860+0000 7f1d1e7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d20065c10 0x7f1d20113770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47576/0 (socket says 172.21.15.112:47576) 2024-09-16T00:51:57.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:57.860+0000 7f1d1e7fc640 1 -- 172.21.15.112:0/1864181160 learned_addr learned my addr 172.21.15.112:0/1864181160 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:51:57.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:57.860+0000 7f1d1effd640 1 --2- 172.21.15.112:0/1864181160 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d20065300 0x7f1d20113230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:51:57.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:57.860+0000 7f1d1e7fc640 1 -- 172.21.15.112:0/1864181160 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d20065300 msgr2=0x7f1d20113230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:51:57.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:57.860+0000 7f1d1e7fc640 1 --2- 172.21.15.112:0/1864181160 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d20065300 0x7f1d20113230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:57.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:57.860+0000 7f1d1e7fc640 1 -- 172.21.15.112:0/1864181160 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d08005040 con 0x7f1d20065c10 2024-09-16T00:51:57.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:57.860+0000 7f1d1e7fc640 1 --2- 172.21.15.112:0/1864181160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d20065c10 0x7f1d20113770 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f1d1000b790 tx=0x7f1d100091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:51:57.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:57.862+0000 7f1d177fe640 1 -- 172.21.15.112:0/1864181160 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d1000c700 con 0x7f1d20065c10 2024-09-16T00:51:57.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:57.862+0000 7f1d177fe640 1 -- 172.21.15.112:0/1864181160 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f1d1000cd00 con 0x7f1d20065c10 2024-09-16T00:51:57.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:57.862+0000 7f1d2594c640 1 -- 172.21.15.112:0/1864181160 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d2010d490 con 0x7f1d20065c10 2024-09-16T00:51:57.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:57.862+0000 7f1d177fe640 1 -- 172.21.15.112:0/1864181160 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d10015e80 con 0x7f1d20065c10 2024-09-16T00:51:57.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:57.862+0000 7f1d2594c640 1 -- 172.21.15.112:0/1864181160 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d2010d9b0 con 0x7f1d20065c10 2024-09-16T00:51:57.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:57.863+0000 7f1d2594c640 1 -- 172.21.15.112:0/1864181160 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d200fdce0 con 0x7f1d20065c10 2024-09-16T00:51:57.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:57.863+0000 7f1d177fe640 1 -- 172.21.15.112:0/1864181160 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f1d10012070 con 0x7f1d20065c10 2024-09-16T00:51:57.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:57.863+0000 7f1d177fe640 1 --2- 172.21.15.112:0/1864181160 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f1d0c06e530 0x7f1d0c0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:51:57.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:57.864+0000 7f1d177fe640 1 -- 172.21.15.112:0/1864181160 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f1d1008dbb0 con 0x7f1d20065c10 2024-09-16T00:51:57.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:57.864+0000 7f1d1effd640 1 --2- 172.21.15.112:0/1864181160 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f1d0c06e530 0x7f1d0c0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:51:57.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:57.864+0000 7f1d1effd640 1 --2- 172.21.15.112:0/1864181160 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f1d0c06e530 0x7f1d0c0709e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1d08009ac0 tx=0x7f1d08035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:51:57.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:57.868+0000 7f1d177fe640 1 -- 172.21.15.112:0/1864181160 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f1d10056b00 con 0x7f1d20065c10 2024-09-16T00:51:58.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:58.070+0000 7f1d2594c640 1 -- 172.21.15.112:0/1864181160 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 30, "format": "json"} v 0) v1 -- 0x7f1d20072ec0 con 0x7f1d20065c10 2024-09-16T00:51:58.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:58.070+0000 7f1d177fe640 1 -- 172.21.15.112:0/1864181160 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 30, "format": "json"}]=0 dumped fsmap epoch 30 v41) v1 ==== 107+0+5040 (secure 0 0 0) 0x7f1d1005a1c0 con 0x7f1d20065c10 2024-09-16T00:51:58.071 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:51:58.071 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":30,"btime":"2024-09-16T00:33:55:621083+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34392,"name":"cephfs.smithi112.ithwzj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6827/476229221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":476229221},{"type":"v1","addr":"172.21.15.112:6827","nonce":476229221}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":34404,"name":"cephfs.smithi112.hxfwkh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6829/2356899876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2356899876},{"type":"v1","addr":"172.21.15.112:6829","nonce":2356899876}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":44307,"name":"cephfs.smithi169.lgeqrj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6827/3569474985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3569474985},{"type":"v1","addr":"172.21.15.169:6827","nonce":3569474985}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":44315,"name":"cephfs.smithi169.gawwgw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6825/3147876558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3147876558},{"type":"v1","addr":"172.21.15.169:6825","nonce":3147876558}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":25,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:23:22.328199+0000","modified":"2024-09-16T00:33:29.012991+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-16T00:51:58.071 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 30 2024-09-16T00:51:58.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:58.073+0000 7f1d2594c640 1 -- 172.21.15.112:0/1864181160 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f1d0c06e530 msgr2=0x7f1d0c0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:51:58.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:58.073+0000 7f1d2594c640 1 --2- 172.21.15.112:0/1864181160 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f1d0c06e530 0x7f1d0c0709e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1d08009ac0 tx=0x7f1d08035040 comp rx=0 tx=0).stop 2024-09-16T00:51:58.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:58.073+0000 7f1d2594c640 1 -- 172.21.15.112:0/1864181160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d20065c10 msgr2=0x7f1d20113770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:51:58.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:58.073+0000 7f1d2594c640 1 --2- 172.21.15.112:0/1864181160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d20065c10 0x7f1d20113770 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f1d1000b790 tx=0x7f1d100091d0 comp rx=0 tx=0).stop 2024-09-16T00:51:58.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:58.073+0000 7f1d2594c640 1 -- 172.21.15.112:0/1864181160 shutdown_connections 2024-09-16T00:51:58.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:58.073+0000 7f1d2594c640 1 --2- 172.21.15.112:0/1864181160 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f1d0c06e530 0x7f1d0c0709e0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:58.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:58.073+0000 7f1d2594c640 1 --2- 172.21.15.112:0/1864181160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d20065c10 0x7f1d20113770 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:58.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:58.073+0000 7f1d2594c640 1 --2- 172.21.15.112:0/1864181160 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d20065300 0x7f1d20113230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:51:58.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:58.073+0000 7f1d2594c640 1 -- 172.21.15.112:0/1864181160 >> 172.21.15.112:0/1864181160 conn(0x7f1d200723b0 msgr2=0x7f1d20105420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:51:58.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:58.073+0000 7f1d2594c640 1 -- 172.21.15.112:0/1864181160 shutdown_connections 2024-09-16T00:51:58.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:51:58.073+0000 7f1d2594c640 1 -- 172.21.15.112:0/1864181160 wait complete. 2024-09-16T00:51:58.753 DEBUG:tasks.fs:max_mds not reduced in epoch 30 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-09-16T00:51:58.753 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 30 2024-09-16T00:51:58.753 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 31 2024-09-16T00:51:59.001 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:51:59.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:51:58 smithi112 ceph-mon[97750]: pgmap v728: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:51:59.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:51:58 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/1864181160' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-09-16T00:51:59.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:51:58 smithi169 ceph-mon[77626]: pgmap v728: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:51:59.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:51:58 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/1864181160' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-09-16T00:52:00.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:00.573+0000 7f906b96b640 1 -- 172.21.15.112:0/2996874924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90640703c0 msgr2=0x7f90641088a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:52:00.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:00.573+0000 7f906b96b640 1 --2- 172.21.15.112:0/2996874924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90640703c0 0x7f90641088a0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f90600048d0 tx=0x7f9060009da0 comp rx=0 tx=0).stop 2024-09-16T00:52:00.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:00.573+0000 7f906b96b640 1 -- 172.21.15.112:0/2996874924 shutdown_connections 2024-09-16T00:52:00.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:00.573+0000 7f906b96b640 1 --2- 172.21.15.112:0/2996874924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90640703c0 0x7f90641088a0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:00.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:00.573+0000 7f906b96b640 1 --2- 172.21.15.112:0/2996874924 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f906406fab0 0x7f906406fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:00.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:00.573+0000 7f906b96b640 1 -- 172.21.15.112:0/2996874924 >> 172.21.15.112:0/2996874924 conn(0x7f90640f8120 msgr2=0x7f90640fa530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:52:00.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:00.573+0000 7f906b96b640 1 -- 172.21.15.112:0/2996874924 shutdown_connections 2024-09-16T00:52:00.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:00.573+0000 7f906b96b640 1 -- 172.21.15.112:0/2996874924 wait complete. 2024-09-16T00:52:00.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:00.574+0000 7f906b96b640 1 Processor -- start 2024-09-16T00:52:00.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:00.574+0000 7f906b96b640 1 -- start start 2024-09-16T00:52:00.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:00.574+0000 7f906b96b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f906406fab0 0x7f9064115490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:52:00.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:00.574+0000 7f906b96b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90640703c0 0x7f90641159d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:52:00.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:00.574+0000 7f906b96b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90641160b0 con 0x7f90640703c0 2024-09-16T00:52:00.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:00.574+0000 7f906b96b640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f906410f430 con 0x7f906406fab0 2024-09-16T00:52:00.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:00.574+0000 7f9068f06640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90640703c0 0x7f90641159d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:52:00.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:00.574+0000 7f9068f06640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90640703c0 0x7f90641159d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47600/0 (socket says 172.21.15.112:47600) 2024-09-16T00:52:00.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:00.574+0000 7f9068f06640 1 -- 172.21.15.112:0/1035931389 learned_addr learned my addr 172.21.15.112:0/1035931389 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:52:00.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:00.575+0000 7f9068f06640 1 -- 172.21.15.112:0/1035931389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f906406fab0 msgr2=0x7f9064115490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:52:00.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:00.575+0000 7f9069707640 1 --2- 172.21.15.112:0/1035931389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f906406fab0 0x7f9064115490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:52:00.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:00.575+0000 7f9068f06640 1 --2- 172.21.15.112:0/1035931389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f906406fab0 0x7f9064115490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:00.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:00.575+0000 7f9068f06640 1 -- 172.21.15.112:0/1035931389 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9060005040 con 0x7f90640703c0 2024-09-16T00:52:00.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:00.575+0000 7f9069707640 1 --2- 172.21.15.112:0/1035931389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f906406fab0 0x7f9064115490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:52:00.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:00.575+0000 7f9068f06640 1 --2- 172.21.15.112:0/1035931389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90640703c0 0x7f90641159d0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f9060003080 tx=0x7f9060009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:52:00.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:00.575+0000 7f9059ffb640 1 -- 172.21.15.112:0/1035931389 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9060038070 con 0x7f90640703c0 2024-09-16T00:52:00.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:00.575+0000 7f9059ffb640 1 -- 172.21.15.112:0/1035931389 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f90600097e0 con 0x7f90640703c0 2024-09-16T00:52:00.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:00.575+0000 7f906b96b640 1 -- 172.21.15.112:0/1035931389 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f906410f660 con 0x7f90640703c0 2024-09-16T00:52:00.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:00.575+0000 7f9059ffb640 1 -- 172.21.15.112:0/1035931389 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f906003d420 con 0x7f90640703c0 2024-09-16T00:52:00.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:00.575+0000 7f906b96b640 1 -- 172.21.15.112:0/1035931389 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f906410fb20 con 0x7f90640703c0 2024-09-16T00:52:00.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:00.576+0000 7f906b96b640 1 -- 172.21.15.112:0/1035931389 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90640711f0 con 0x7f90640703c0 2024-09-16T00:52:00.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:00.577+0000 7f9059ffb640 1 -- 172.21.15.112:0/1035931389 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f9060030020 con 0x7f90640703c0 2024-09-16T00:52:00.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:00.577+0000 7f9059ffb640 1 --2- 172.21.15.112:0/1035931389 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f905006e530 0x7f90500709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:52:00.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:00.577+0000 7f9059ffb640 1 -- 172.21.15.112:0/1035931389 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f90600ae7b0 con 0x7f90640703c0 2024-09-16T00:52:00.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:00.577+0000 7f9069707640 1 --2- 172.21.15.112:0/1035931389 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f905006e530 0x7f90500709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:52:00.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:00.578+0000 7f9069707640 1 --2- 172.21.15.112:0/1035931389 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f905006e530 0x7f90500709e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9054006fd0 tx=0x7f9054008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:52:00.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:00.581+0000 7f9059ffb640 1 -- 172.21.15.112:0/1035931389 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f90600777a0 con 0x7f90640703c0 2024-09-16T00:52:00.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:00.786+0000 7f906b96b640 1 -- 172.21.15.112:0/1035931389 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 31, "format": "json"} v 0) v1 -- 0x7f90640fbd60 con 0x7f90640703c0 2024-09-16T00:52:00.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:00.786+0000 7f9059ffb640 1 -- 172.21.15.112:0/1035931389 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 31, "format": "json"}]=0 dumped fsmap epoch 31 v41) v1 ==== 107+0+4258 (secure 0 0 0) 0x7f906007ae60 con 0x7f90640703c0 2024-09-16T00:52:00.787 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:52:00.787 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":31,"btime":"2024-09-16T00:34:06:396991+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34392,"name":"cephfs.smithi112.ithwzj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6827/476229221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":476229221},{"type":"v1","addr":"172.21.15.112:6827","nonce":476229221}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":34404,"name":"cephfs.smithi112.hxfwkh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6829/2356899876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2356899876},{"type":"v1","addr":"172.21.15.112:6829","nonce":2356899876}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":44307,"name":"cephfs.smithi169.lgeqrj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6827/3569474985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3569474985},{"type":"v1","addr":"172.21.15.169:6827","nonce":3569474985}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":25,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:23:22.328199+0000","modified":"2024-09-16T00:33:29.012991+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-16T00:52:00.787 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 31 2024-09-16T00:52:00.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:00.788+0000 7f906b96b640 1 -- 172.21.15.112:0/1035931389 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f905006e530 msgr2=0x7f90500709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:52:00.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:00.788+0000 7f906b96b640 1 --2- 172.21.15.112:0/1035931389 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f905006e530 0x7f90500709e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9054006fd0 tx=0x7f9054008040 comp rx=0 tx=0).stop 2024-09-16T00:52:00.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:00.788+0000 7f906b96b640 1 -- 172.21.15.112:0/1035931389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90640703c0 msgr2=0x7f90641159d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:52:00.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:00.788+0000 7f906b96b640 1 --2- 172.21.15.112:0/1035931389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90640703c0 0x7f90641159d0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f9060003080 tx=0x7f9060009da0 comp rx=0 tx=0).stop 2024-09-16T00:52:00.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:00.788+0000 7f906b96b640 1 -- 172.21.15.112:0/1035931389 shutdown_connections 2024-09-16T00:52:00.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:00.788+0000 7f906b96b640 1 --2- 172.21.15.112:0/1035931389 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f905006e530 0x7f90500709e0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:00.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:00.788+0000 7f906b96b640 1 --2- 172.21.15.112:0/1035931389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90640703c0 0x7f90641159d0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:00.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:00.788+0000 7f906b96b640 1 --2- 172.21.15.112:0/1035931389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f906406fab0 0x7f9064115490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:00.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:00.788+0000 7f906b96b640 1 -- 172.21.15.112:0/1035931389 >> 172.21.15.112:0/1035931389 conn(0x7f90640f8120 msgr2=0x7f90640f95f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:52:00.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:00.788+0000 7f906b96b640 1 -- 172.21.15.112:0/1035931389 shutdown_connections 2024-09-16T00:52:00.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:00.788+0000 7f906b96b640 1 -- 172.21.15.112:0/1035931389 wait complete. 2024-09-16T00:52:01.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:52:00 smithi112 ceph-mon[97750]: pgmap v729: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:52:01.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:52:00 smithi169 ceph-mon[77626]: pgmap v729: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:52:01.518 DEBUG:tasks.fs:max_mds not reduced in epoch 31 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-09-16T00:52:01.518 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 31 2024-09-16T00:52:01.518 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 32 2024-09-16T00:52:01.765 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:52:02.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:52:01 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/1035931389' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-09-16T00:52:02.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:52:01 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:52:02.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:52:01 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/1035931389' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-09-16T00:52:02.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:52:01 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:52:02.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:52:02 smithi112 ceph-mon[97750]: pgmap v730: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:52:03.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:52:02 smithi169 ceph-mon[77626]: pgmap v730: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:52:03.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:03.347+0000 7f13d56ea640 1 -- 172.21.15.112:0/2684211944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13d01014e0 msgr2=0x7f13d01018b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:52:03.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:03.347+0000 7f13d56ea640 1 --2- 172.21.15.112:0/2684211944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13d01014e0 0x7f13d01018b0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f13c00048d0 tx=0x7f13c0009da0 comp rx=0 tx=0).stop 2024-09-16T00:52:03.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:03.347+0000 7f13d56ea640 1 -- 172.21.15.112:0/2684211944 shutdown_connections 2024-09-16T00:52:03.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:03.347+0000 7f13d56ea640 1 --2- 172.21.15.112:0/2684211944 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13d00fb4a0 0x7f13d00fb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:03.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:03.347+0000 7f13d56ea640 1 --2- 172.21.15.112:0/2684211944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13d01014e0 0x7f13d01018b0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:03.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:03.347+0000 7f13d56ea640 1 -- 172.21.15.112:0/2684211944 >> 172.21.15.112:0/2684211944 conn(0x7f13d00f7060 msgr2=0x7f13d00f9470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:52:03.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:03.347+0000 7f13d56ea640 1 -- 172.21.15.112:0/2684211944 shutdown_connections 2024-09-16T00:52:03.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:03.347+0000 7f13d56ea640 1 -- 172.21.15.112:0/2684211944 wait complete. 2024-09-16T00:52:03.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:03.348+0000 7f13d56ea640 1 Processor -- start 2024-09-16T00:52:03.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:03.348+0000 7f13d56ea640 1 -- start start 2024-09-16T00:52:03.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:03.348+0000 7f13d56ea640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13d00fb4a0 0x7f13d010e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:52:03.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:03.348+0000 7f13d56ea640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13d01014e0 0x7f13d010e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:52:03.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:03.348+0000 7f13d56ea640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d010eca0 con 0x7f13d00fb4a0 2024-09-16T00:52:03.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:03.348+0000 7f13d56ea640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d010ee10 con 0x7f13d01014e0 2024-09-16T00:52:03.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:03.348+0000 7f13ceffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13d00fb4a0 0x7f13d010e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:52:03.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:03.348+0000 7f13ceffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13d00fb4a0 0x7f13d010e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59450/0 (socket says 172.21.15.112:59450) 2024-09-16T00:52:03.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:03.348+0000 7f13ceffd640 1 -- 172.21.15.112:0/1066524635 learned_addr learned my addr 172.21.15.112:0/1066524635 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:52:03.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:03.348+0000 7f13c67fc640 1 --2- 172.21.15.112:0/1066524635 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13d01014e0 0x7f13d010e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:52:03.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:03.348+0000 7f13ceffd640 1 -- 172.21.15.112:0/1066524635 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13d01014e0 msgr2=0x7f13d010e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:52:03.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:03.348+0000 7f13ceffd640 1 --2- 172.21.15.112:0/1066524635 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13d01014e0 0x7f13d010e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:03.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:03.348+0000 7f13ceffd640 1 -- 172.21.15.112:0/1066524635 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13c0005040 con 0x7f13d00fb4a0 2024-09-16T00:52:03.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:03.349+0000 7f13ceffd640 1 --2- 172.21.15.112:0/1066524635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13d00fb4a0 0x7f13d010e160 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f13c00043d0 tx=0x7f13c0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:52:03.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:03.349+0000 7f13c7fff640 1 -- 172.21.15.112:0/1066524635 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13c0038070 con 0x7f13d00fb4a0 2024-09-16T00:52:03.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:03.349+0000 7f13d56ea640 1 -- 172.21.15.112:0/1066524635 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13d01171d0 con 0x7f13d00fb4a0 2024-09-16T00:52:03.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:03.349+0000 7f13c7fff640 1 -- 172.21.15.112:0/1066524635 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f13c00096e0 con 0x7f13d00fb4a0 2024-09-16T00:52:03.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:03.349+0000 7f13c7fff640 1 -- 172.21.15.112:0/1066524635 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13c003d420 con 0x7f13d00fb4a0 2024-09-16T00:52:03.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:03.349+0000 7f13d56ea640 1 -- 172.21.15.112:0/1066524635 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13d0117690 con 0x7f13d00fb4a0 2024-09-16T00:52:03.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:03.350+0000 7f13d56ea640 1 -- 172.21.15.112:0/1066524635 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13d00fcbe0 con 0x7f13d00fb4a0 2024-09-16T00:52:03.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:03.350+0000 7f13c7fff640 1 -- 172.21.15.112:0/1066524635 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f13c0030020 con 0x7f13d00fb4a0 2024-09-16T00:52:03.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:03.351+0000 7f13c7fff640 1 --2- 172.21.15.112:0/1066524635 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f13bc06e530 0x7f13bc0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:52:03.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:03.351+0000 7f13c7fff640 1 -- 172.21.15.112:0/1066524635 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f13c00ae8b0 con 0x7f13d00fb4a0 2024-09-16T00:52:03.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:03.351+0000 7f13c67fc640 1 --2- 172.21.15.112:0/1066524635 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f13bc06e530 0x7f13bc0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:52:03.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:03.351+0000 7f13c67fc640 1 --2- 172.21.15.112:0/1066524635 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f13bc06e530 0x7f13bc0709e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f13b8007fd0 tx=0x7f13b8009e90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:52:03.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:03.355+0000 7f13c7fff640 1 -- 172.21.15.112:0/1066524635 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f13c0077820 con 0x7f13d00fb4a0 2024-09-16T00:52:03.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:03.563+0000 7f13d56ea640 1 -- 172.21.15.112:0/1066524635 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 32, "format": "json"} v 0) v1 -- 0x7f13d004b770 con 0x7f13d00fb4a0 2024-09-16T00:52:03.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:03.563+0000 7f13c7fff640 1 -- 172.21.15.112:0/1066524635 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 32, "format": "json"}]=0 dumped fsmap epoch 32 v41) v1 ==== 107+0+5108 (secure 0 0 0) 0x7f13c007aee0 con 0x7f13d00fb4a0 2024-09-16T00:52:03.564 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:52:03.564 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":32,"btime":"2024-09-16T00:34:10:610023+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34392,"name":"cephfs.smithi112.ithwzj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6827/476229221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":476229221},{"type":"v1","addr":"172.21.15.112:6827","nonce":476229221}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":34404,"name":"cephfs.smithi112.hxfwkh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6829/2356899876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2356899876},{"type":"v1","addr":"172.21.15.112:6829","nonce":2356899876}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":44307,"name":"cephfs.smithi169.lgeqrj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6827/3569474985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3569474985},{"type":"v1","addr":"172.21.15.169:6827","nonce":3569474985}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":44343,"name":"cephfs.smithi169.gawwgw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6825/1635329957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":1635329957},{"type":"v1","addr":"172.21.15.169:6825","nonce":1635329957}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":25,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:23:22.328199+0000","modified":"2024-09-16T00:33:29.012991+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-16T00:52:03.564 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 32 2024-09-16T00:52:03.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:03.566+0000 7f13d56ea640 1 -- 172.21.15.112:0/1066524635 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f13bc06e530 msgr2=0x7f13bc0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:52:03.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:03.566+0000 7f13d56ea640 1 --2- 172.21.15.112:0/1066524635 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f13bc06e530 0x7f13bc0709e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f13b8007fd0 tx=0x7f13b8009e90 comp rx=0 tx=0).stop 2024-09-16T00:52:03.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:03.566+0000 7f13d56ea640 1 -- 172.21.15.112:0/1066524635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13d00fb4a0 msgr2=0x7f13d010e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:52:03.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:03.566+0000 7f13d56ea640 1 --2- 172.21.15.112:0/1066524635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13d00fb4a0 0x7f13d010e160 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f13c00043d0 tx=0x7f13c0009da0 comp rx=0 tx=0).stop 2024-09-16T00:52:03.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:03.566+0000 7f13d56ea640 1 -- 172.21.15.112:0/1066524635 shutdown_connections 2024-09-16T00:52:03.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:03.566+0000 7f13d56ea640 1 --2- 172.21.15.112:0/1066524635 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f13bc06e530 0x7f13bc0709e0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:03.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:03.566+0000 7f13d56ea640 1 --2- 172.21.15.112:0/1066524635 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13d01014e0 0x7f13d010e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:03.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:03.566+0000 7f13d56ea640 1 --2- 172.21.15.112:0/1066524635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13d00fb4a0 0x7f13d010e160 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:03.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:03.566+0000 7f13d56ea640 1 -- 172.21.15.112:0/1066524635 >> 172.21.15.112:0/1066524635 conn(0x7f13d00f7060 msgr2=0x7f13d00f7430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:52:03.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:03.566+0000 7f13d56ea640 1 -- 172.21.15.112:0/1066524635 shutdown_connections 2024-09-16T00:52:03.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:03.566+0000 7f13d56ea640 1 -- 172.21.15.112:0/1066524635 wait complete. 2024-09-16T00:52:04.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:52:03 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/1066524635' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-09-16T00:52:04.206 DEBUG:tasks.fs:max_mds not reduced in epoch 32 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-09-16T00:52:04.207 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 32 2024-09-16T00:52:04.207 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 33 2024-09-16T00:52:04.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:52:03 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/1066524635' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-09-16T00:52:04.451 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:52:04.840 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:52:04 smithi112 ceph-mon[97750]: pgmap v731: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:52:05.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:52:04 smithi169 ceph-mon[77626]: pgmap v731: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:52:06.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:06.028+0000 7f3dc62fb640 1 -- 172.21.15.112:0/2486910167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3dc0102ed0 msgr2=0x7f3dc00fc590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:52:06.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:06.028+0000 7f3dc62fb640 1 --2- 172.21.15.112:0/2486910167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3dc0102ed0 0x7f3dc00fc590 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f3da80048d0 tx=0x7f3da8009da0 comp rx=0 tx=0).stop 2024-09-16T00:52:06.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:06.028+0000 7f3dc62fb640 1 -- 172.21.15.112:0/2486910167 shutdown_connections 2024-09-16T00:52:06.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:06.028+0000 7f3dc62fb640 1 --2- 172.21.15.112:0/2486910167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3dc0102ed0 0x7f3dc00fc590 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:06.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:06.028+0000 7f3dc62fb640 1 --2- 172.21.15.112:0/2486910167 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3dc0102530 0x7f3dc0102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:06.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:06.028+0000 7f3dc62fb640 1 -- 172.21.15.112:0/2486910167 >> 172.21.15.112:0/2486910167 conn(0x7f3dc00f8110 msgr2=0x7f3dc00fa520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:52:06.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:06.028+0000 7f3dc62fb640 1 -- 172.21.15.112:0/2486910167 shutdown_connections 2024-09-16T00:52:06.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:06.028+0000 7f3dc62fb640 1 -- 172.21.15.112:0/2486910167 wait complete. 2024-09-16T00:52:06.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:06.029+0000 7f3dc62fb640 1 Processor -- start 2024-09-16T00:52:06.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:06.029+0000 7f3dc62fb640 1 -- start start 2024-09-16T00:52:06.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:06.029+0000 7f3dc62fb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3dc0102530 0x7f3dc006fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:52:06.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:06.029+0000 7f3dc62fb640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3dc0102ed0 0x7f3dc006ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:52:06.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:06.029+0000 7f3dc62fb640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc0070490 con 0x7f3dc0102530 2024-09-16T00:52:06.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:06.029+0000 7f3dc62fb640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc0070600 con 0x7f3dc0102ed0 2024-09-16T00:52:06.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:06.029+0000 7f3dbffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3dc0102530 0x7f3dc006fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:52:06.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:06.029+0000 7f3dbffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3dc0102530 0x7f3dc006fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59478/0 (socket says 172.21.15.112:59478) 2024-09-16T00:52:06.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:06.029+0000 7f3dbffff640 1 -- 172.21.15.112:0/1823058628 learned_addr learned my addr 172.21.15.112:0/1823058628 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:52:06.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:06.029+0000 7f3db7fff640 1 --2- 172.21.15.112:0/1823058628 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3dc0102ed0 0x7f3dc006ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:52:06.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:06.029+0000 7f3dbffff640 1 -- 172.21.15.112:0/1823058628 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3dc0102ed0 msgr2=0x7f3dc006ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:52:06.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:06.029+0000 7f3dbffff640 1 --2- 172.21.15.112:0/1823058628 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3dc0102ed0 0x7f3dc006ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:06.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:06.029+0000 7f3dbffff640 1 -- 172.21.15.112:0/1823058628 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3da8005040 con 0x7f3dc0102530 2024-09-16T00:52:06.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:06.029+0000 7f3dbffff640 1 --2- 172.21.15.112:0/1823058628 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3dc0102530 0x7f3dc006fa10 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f3db0002a50 tx=0x7f3db0004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:52:06.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:06.030+0000 7f3dbd7fa640 1 -- 172.21.15.112:0/1823058628 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3db0007c30 con 0x7f3dc0102530 2024-09-16T00:52:06.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:06.030+0000 7f3dbd7fa640 1 -- 172.21.15.112:0/1823058628 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f3db000f040 con 0x7f3dc0102530 2024-09-16T00:52:06.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:06.030+0000 7f3dc62fb640 1 -- 172.21.15.112:0/1823058628 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dc0070890 con 0x7f3dc0102530 2024-09-16T00:52:06.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:06.030+0000 7f3dbd7fa640 1 -- 172.21.15.112:0/1823058628 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3db000da10 con 0x7f3dc0102530 2024-09-16T00:52:06.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:06.030+0000 7f3dc62fb640 1 -- 172.21.15.112:0/1823058628 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dc0119910 con 0x7f3dc0102530 2024-09-16T00:52:06.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:06.031+0000 7f3dc62fb640 1 -- 172.21.15.112:0/1823058628 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dc00fdcb0 con 0x7f3dc0102530 2024-09-16T00:52:06.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:06.031+0000 7f3dbd7fa640 1 -- 172.21.15.112:0/1823058628 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f3db000dc80 con 0x7f3dc0102530 2024-09-16T00:52:06.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:06.032+0000 7f3dbd7fa640 1 --2- 172.21.15.112:0/1823058628 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f3dac06e530 0x7f3dac0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:52:06.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:06.032+0000 7f3dbd7fa640 1 -- 172.21.15.112:0/1823058628 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f3db008dab0 con 0x7f3dc0102530 2024-09-16T00:52:06.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:06.032+0000 7f3db7fff640 1 --2- 172.21.15.112:0/1823058628 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f3dac06e530 0x7f3dac0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:52:06.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:06.032+0000 7f3db7fff640 1 --2- 172.21.15.112:0/1823058628 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f3dac06e530 0x7f3dac0709e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3da80048a0 tx=0x7f3da8035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:52:06.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:06.036+0000 7f3dbd7fa640 1 -- 172.21.15.112:0/1823058628 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f3db0056ad0 con 0x7f3dc0102530 2024-09-16T00:52:06.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:06.240+0000 7f3dc62fb640 1 -- 172.21.15.112:0/1823058628 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 33, "format": "json"} v 0) v1 -- 0x7f3dc00fbd30 con 0x7f3dc0102530 2024-09-16T00:52:06.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:06.240+0000 7f3dbd7fa640 1 -- 172.21.15.112:0/1823058628 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 33, "format": "json"}]=0 dumped fsmap epoch 33 v41) v1 ==== 107+0+4326 (secure 0 0 0) 0x7f3db005a190 con 0x7f3dc0102530 2024-09-16T00:52:06.241 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:52:06.241 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":33,"btime":"2024-09-16T00:34:20:462755+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34392,"name":"cephfs.smithi112.ithwzj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6827/476229221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":476229221},{"type":"v1","addr":"172.21.15.112:6827","nonce":476229221}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":34404,"name":"cephfs.smithi112.hxfwkh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6829/2356899876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2356899876},{"type":"v1","addr":"172.21.15.112:6829","nonce":2356899876}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":44343,"name":"cephfs.smithi169.gawwgw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6825/1635329957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":1635329957},{"type":"v1","addr":"172.21.15.169:6825","nonce":1635329957}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":25,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:23:22.328199+0000","modified":"2024-09-16T00:33:29.012991+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-16T00:52:06.241 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 33 2024-09-16T00:52:06.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:06.242+0000 7f3dc62fb640 1 -- 172.21.15.112:0/1823058628 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f3dac06e530 msgr2=0x7f3dac0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:52:06.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:06.242+0000 7f3dc62fb640 1 --2- 172.21.15.112:0/1823058628 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f3dac06e530 0x7f3dac0709e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3da80048a0 tx=0x7f3da8035040 comp rx=0 tx=0).stop 2024-09-16T00:52:06.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:06.242+0000 7f3dc62fb640 1 -- 172.21.15.112:0/1823058628 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3dc0102530 msgr2=0x7f3dc006fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:52:06.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:06.242+0000 7f3dc62fb640 1 --2- 172.21.15.112:0/1823058628 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3dc0102530 0x7f3dc006fa10 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f3db0002a50 tx=0x7f3db0004600 comp rx=0 tx=0).stop 2024-09-16T00:52:06.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:06.242+0000 7f3dc62fb640 1 -- 172.21.15.112:0/1823058628 shutdown_connections 2024-09-16T00:52:06.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:06.243+0000 7f3dc62fb640 1 --2- 172.21.15.112:0/1823058628 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f3dac06e530 0x7f3dac0709e0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:06.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:06.243+0000 7f3dc62fb640 1 --2- 172.21.15.112:0/1823058628 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3dc0102ed0 0x7f3dc006ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:06.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:06.243+0000 7f3dc62fb640 1 --2- 172.21.15.112:0/1823058628 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3dc0102530 0x7f3dc006fa10 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:06.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:06.243+0000 7f3dc62fb640 1 -- 172.21.15.112:0/1823058628 >> 172.21.15.112:0/1823058628 conn(0x7f3dc00f8110 msgr2=0x7f3dc00f91a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:52:06.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:06.243+0000 7f3dc62fb640 1 -- 172.21.15.112:0/1823058628 shutdown_connections 2024-09-16T00:52:06.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:06.243+0000 7f3dc62fb640 1 -- 172.21.15.112:0/1823058628 wait complete. 2024-09-16T00:52:06.922 DEBUG:tasks.fs:max_mds not reduced in epoch 33 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-09-16T00:52:06.922 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 33 2024-09-16T00:52:06.922 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 34 2024-09-16T00:52:06.945 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:52:06 smithi112 ceph-mon[97750]: pgmap v732: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:52:06.945 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:52:06 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/1823058628' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-09-16T00:52:07.168 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:52:07.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:52:06 smithi169 ceph-mon[77626]: pgmap v732: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:52:07.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:52:06 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/1823058628' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-09-16T00:52:08.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:08.794+0000 7f3b00c3e640 1 -- 172.21.15.112:0/2377133500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3afc0fcf80 msgr2=0x7f3afc0fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:52:08.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:08.794+0000 7f3b00c3e640 1 --2- 172.21.15.112:0/2377133500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3afc0fcf80 0x7f3afc0fd3d0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f3af0009c50 tx=0x7f3af0004d10 comp rx=0 tx=0).stop 2024-09-16T00:52:08.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:08.794+0000 7f3b00c3e640 1 -- 172.21.15.112:0/2377133500 shutdown_connections 2024-09-16T00:52:08.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:08.794+0000 7f3b00c3e640 1 --2- 172.21.15.112:0/2377133500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3afc0fcf80 0x7f3afc0fd3d0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:08.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:08.794+0000 7f3b00c3e640 1 --2- 172.21.15.112:0/2377133500 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3afc0fb810 0x7f3afc0fbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:08.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:08.794+0000 7f3b00c3e640 1 -- 172.21.15.112:0/2377133500 >> 172.21.15.112:0/2377133500 conn(0x7f3afc0723b0 msgr2=0x7f3afc0727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:52:08.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:08.794+0000 7f3b00c3e640 1 -- 172.21.15.112:0/2377133500 shutdown_connections 2024-09-16T00:52:08.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:08.794+0000 7f3b00c3e640 1 -- 172.21.15.112:0/2377133500 wait complete. 2024-09-16T00:52:08.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:08.794+0000 7f3b00c3e640 1 Processor -- start 2024-09-16T00:52:08.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:08.794+0000 7f3b00c3e640 1 -- start start 2024-09-16T00:52:08.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:08.794+0000 7f3b00c3e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3afc0fb810 0x7f3afc111030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:52:08.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:08.794+0000 7f3b00c3e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3afc0fcf80 0x7f3afc111570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:52:08.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:08.794+0000 7f3b00c3e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3afc111c00 con 0x7f3afc0fcf80 2024-09-16T00:52:08.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:08.794+0000 7f3b00c3e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3afc10afb0 con 0x7f3afc0fb810 2024-09-16T00:52:08.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:08.795+0000 7f3af9d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3afc0fcf80 0x7f3afc111570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:52:08.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:08.795+0000 7f3af9d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3afc0fcf80 0x7f3afc111570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59486/0 (socket says 172.21.15.112:59486) 2024-09-16T00:52:08.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:08.795+0000 7f3af9d9b640 1 -- 172.21.15.112:0/1791059747 learned_addr learned my addr 172.21.15.112:0/1791059747 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:52:08.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:08.795+0000 7f3afa59c640 1 --2- 172.21.15.112:0/1791059747 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3afc0fb810 0x7f3afc111030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:52:08.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:08.795+0000 7f3af9d9b640 1 -- 172.21.15.112:0/1791059747 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3afc0fb810 msgr2=0x7f3afc111030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:52:08.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:08.795+0000 7f3af9d9b640 1 --2- 172.21.15.112:0/1791059747 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3afc0fb810 0x7f3afc111030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:08.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:08.795+0000 7f3af9d9b640 1 -- 172.21.15.112:0/1791059747 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3af00098b0 con 0x7f3afc0fcf80 2024-09-16T00:52:08.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:08.795+0000 7f3af9d9b640 1 --2- 172.21.15.112:0/1791059747 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3afc0fcf80 0x7f3afc111570 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f3af0004a30 tx=0x7f3af0004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:52:08.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:08.795+0000 7f3aeaffd640 1 -- 172.21.15.112:0/1791059747 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3af0009690 con 0x7f3afc0fcf80 2024-09-16T00:52:08.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:08.795+0000 7f3aeaffd640 1 -- 172.21.15.112:0/1791059747 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f3af0032b10 con 0x7f3afc0fcf80 2024-09-16T00:52:08.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:08.795+0000 7f3b00c3e640 1 -- 172.21.15.112:0/1791059747 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3afc10b230 con 0x7f3afc0fcf80 2024-09-16T00:52:08.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:08.795+0000 7f3aeaffd640 1 -- 172.21.15.112:0/1791059747 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3af003daa0 con 0x7f3afc0fcf80 2024-09-16T00:52:08.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:08.795+0000 7f3b00c3e640 1 -- 172.21.15.112:0/1791059747 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3afc10b750 con 0x7f3afc0fcf80 2024-09-16T00:52:08.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:08.796+0000 7f3b00c3e640 1 -- 172.21.15.112:0/1791059747 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3afc0fcbc0 con 0x7f3afc0fcf80 2024-09-16T00:52:08.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:08.797+0000 7f3aeaffd640 1 -- 172.21.15.112:0/1791059747 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f3af002f030 con 0x7f3afc0fcf80 2024-09-16T00:52:08.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:08.797+0000 7f3aeaffd640 1 --2- 172.21.15.112:0/1791059747 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f3ae4070740 0x7f3ae4072bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:52:08.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:08.797+0000 7f3aeaffd640 1 -- 172.21.15.112:0/1791059747 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f3af00af420 con 0x7f3afc0fcf80 2024-09-16T00:52:08.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:08.797+0000 7f3afa59c640 1 --2- 172.21.15.112:0/1791059747 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f3ae4070740 0x7f3ae4072bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:52:08.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:08.798+0000 7f3afa59c640 1 --2- 172.21.15.112:0/1791059747 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f3ae4070740 0x7f3ae4072bf0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f3af4006fd0 tx=0x7f3af4008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:52:08.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:08.802+0000 7f3aeaffd640 1 -- 172.21.15.112:0/1791059747 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f3af0078390 con 0x7f3afc0fcf80 2024-09-16T00:52:09.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:09.003+0000 7f3b00c3e640 1 -- 172.21.15.112:0/1791059747 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 34, "format": "json"} v 0) v1 -- 0x7f3afc10c6a0 con 0x7f3afc0fcf80 2024-09-16T00:52:09.004 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:52:08 smithi112 ceph-mon[97750]: pgmap v733: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:52:09.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:09.004+0000 7f3aeaffd640 1 -- 172.21.15.112:0/1791059747 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 34, "format": "json"}]=0 dumped fsmap epoch 34 v41) v1 ==== 107+0+5173 (secure 0 0 0) 0x7f3af007ba50 con 0x7f3afc0fcf80 2024-09-16T00:52:09.005 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:52:09.005 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":34,"btime":"2024-09-16T00:34:26:619254+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34392,"name":"cephfs.smithi112.ithwzj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6827/476229221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":476229221},{"type":"v1","addr":"172.21.15.112:6827","nonce":476229221}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":34404,"name":"cephfs.smithi112.hxfwkh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6829/2356899876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2356899876},{"type":"v1","addr":"172.21.15.112:6829","nonce":2356899876}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":44343,"name":"cephfs.smithi169.gawwgw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6825/1635329957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":1635329957},{"type":"v1","addr":"172.21.15.169:6825","nonce":1635329957}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":44347,"name":"cephfs.smithi169.lgeqrj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6827/184810031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":184810031},{"type":"v1","addr":"172.21.15.169:6827","nonce":184810031}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":25,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:23:22.328199+0000","modified":"2024-09-16T00:33:29.012991+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-16T00:52:09.005 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 34 2024-09-16T00:52:09.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:09.006+0000 7f3b00c3e640 1 -- 172.21.15.112:0/1791059747 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f3ae4070740 msgr2=0x7f3ae4072bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:52:09.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:09.006+0000 7f3b00c3e640 1 --2- 172.21.15.112:0/1791059747 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f3ae4070740 0x7f3ae4072bf0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f3af4006fd0 tx=0x7f3af4008040 comp rx=0 tx=0).stop 2024-09-16T00:52:09.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:09.006+0000 7f3b00c3e640 1 -- 172.21.15.112:0/1791059747 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3afc0fcf80 msgr2=0x7f3afc111570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:52:09.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:09.006+0000 7f3b00c3e640 1 --2- 172.21.15.112:0/1791059747 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3afc0fcf80 0x7f3afc111570 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f3af0004a30 tx=0x7f3af0004d10 comp rx=0 tx=0).stop 2024-09-16T00:52:09.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:09.006+0000 7f3b00c3e640 1 -- 172.21.15.112:0/1791059747 shutdown_connections 2024-09-16T00:52:09.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:09.006+0000 7f3b00c3e640 1 --2- 172.21.15.112:0/1791059747 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f3ae4070740 0x7f3ae4072bf0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:09.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:09.006+0000 7f3b00c3e640 1 --2- 172.21.15.112:0/1791059747 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3afc0fcf80 0x7f3afc111570 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:09.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:09.006+0000 7f3b00c3e640 1 --2- 172.21.15.112:0/1791059747 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3afc0fb810 0x7f3afc111030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:09.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:09.006+0000 7f3b00c3e640 1 -- 172.21.15.112:0/1791059747 >> 172.21.15.112:0/1791059747 conn(0x7f3afc0723b0 msgr2=0x7f3afc0fa480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:52:09.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:09.006+0000 7f3b00c3e640 1 -- 172.21.15.112:0/1791059747 shutdown_connections 2024-09-16T00:52:09.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:09.006+0000 7f3b00c3e640 1 -- 172.21.15.112:0/1791059747 wait complete. 2024-09-16T00:52:09.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:52:08 smithi169 ceph-mon[77626]: pgmap v733: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:52:09.611 DEBUG:tasks.fs:max_mds not reduced in epoch 34 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-09-16T00:52:09.611 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 34 2024-09-16T00:52:09.611 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 35 2024-09-16T00:52:09.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:52:09 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/1791059747' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-09-16T00:52:09.898 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:52:10.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:52:09 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/1791059747' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-09-16T00:52:11.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:52:10 smithi112 ceph-mon[97750]: pgmap v734: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:52:11.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:52:10 smithi169 ceph-mon[77626]: pgmap v734: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:52:11.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:11.539+0000 7f4943082640 1 -- 172.21.15.112:0/2254433177 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f493c102f10 msgr2=0x7f493c0fac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:52:11.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:11.539+0000 7f4943082640 1 --2- 172.21.15.112:0/2254433177 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f493c102f10 0x7f493c0fac60 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f492c009c50 tx=0x7f492c004d10 comp rx=0 tx=0).stop 2024-09-16T00:52:11.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:11.539+0000 7f4943082640 1 -- 172.21.15.112:0/2254433177 shutdown_connections 2024-09-16T00:52:11.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:11.539+0000 7f4943082640 1 --2- 172.21.15.112:0/2254433177 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f493c102f10 0x7f493c0fac60 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:11.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:11.539+0000 7f4943082640 1 --2- 172.21.15.112:0/2254433177 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f493c102570 0x7f493c102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:11.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:11.540+0000 7f4943082640 1 -- 172.21.15.112:0/2254433177 >> 172.21.15.112:0/2254433177 conn(0x7f493c070c80 msgr2=0x7f493c071080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:52:11.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:11.540+0000 7f4943082640 1 -- 172.21.15.112:0/2254433177 shutdown_connections 2024-09-16T00:52:11.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:11.540+0000 7f4943082640 1 -- 172.21.15.112:0/2254433177 wait complete. 2024-09-16T00:52:11.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:11.540+0000 7f4943082640 1 Processor -- start 2024-09-16T00:52:11.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:11.540+0000 7f4943082640 1 -- start start 2024-09-16T00:52:11.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:11.541+0000 7f4943082640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f493c102570 0x7f493c113290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:52:11.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:11.541+0000 7f4943082640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f493c102f10 0x7f493c1137d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:52:11.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:11.541+0000 7f4943082640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f493c113e60 con 0x7f493c102f10 2024-09-16T00:52:11.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:11.541+0000 7f4943082640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f493c10d210 con 0x7f493c102570 2024-09-16T00:52:11.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:11.541+0000 7f493bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f493c102f10 0x7f493c1137d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:52:11.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:11.541+0000 7f493bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f493c102f10 0x7f493c1137d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59512/0 (socket says 172.21.15.112:59512) 2024-09-16T00:52:11.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:11.541+0000 7f493bfff640 1 -- 172.21.15.112:0/3785152170 learned_addr learned my addr 172.21.15.112:0/3785152170 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:52:11.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:11.541+0000 7f4940e1e640 1 --2- 172.21.15.112:0/3785152170 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f493c102570 0x7f493c113290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:52:11.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:11.541+0000 7f493bfff640 1 -- 172.21.15.112:0/3785152170 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f493c102570 msgr2=0x7f493c113290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:52:11.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:11.541+0000 7f493bfff640 1 --2- 172.21.15.112:0/3785152170 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f493c102570 0x7f493c113290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:11.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:11.541+0000 7f493bfff640 1 -- 172.21.15.112:0/3785152170 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f492c0098b0 con 0x7f493c102f10 2024-09-16T00:52:11.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:11.541+0000 7f493bfff640 1 --2- 172.21.15.112:0/3785152170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f493c102f10 0x7f493c1137d0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f492c003400 tx=0x7f492c004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:52:11.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:11.541+0000 7f49397fa640 1 -- 172.21.15.112:0/3785152170 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f492c009610 con 0x7f493c102f10 2024-09-16T00:52:11.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:11.541+0000 7f49397fa640 1 -- 172.21.15.112:0/3785152170 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f492c032b10 con 0x7f493c102f10 2024-09-16T00:52:11.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:11.542+0000 7f49397fa640 1 -- 172.21.15.112:0/3785152170 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f492c03d9c0 con 0x7f493c102f10 2024-09-16T00:52:11.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:11.542+0000 7f4943082640 1 -- 172.21.15.112:0/3785152170 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f493c10d490 con 0x7f493c102f10 2024-09-16T00:52:11.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:11.542+0000 7f4943082640 1 -- 172.21.15.112:0/3785152170 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f493c10d9b0 con 0x7f493c102f10 2024-09-16T00:52:11.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:11.543+0000 7f4943082640 1 -- 172.21.15.112:0/3785152170 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f493c065800 con 0x7f493c102f10 2024-09-16T00:52:11.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:11.543+0000 7f49397fa640 1 -- 172.21.15.112:0/3785152170 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f492c02f030 con 0x7f493c102f10 2024-09-16T00:52:11.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:11.543+0000 7f49397fa640 1 --2- 172.21.15.112:0/3785152170 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f492806e530 0x7f49280709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:52:11.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:11.544+0000 7f49397fa640 1 -- 172.21.15.112:0/3785152170 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f492c0af3a0 con 0x7f493c102f10 2024-09-16T00:52:11.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:11.544+0000 7f4940e1e640 1 --2- 172.21.15.112:0/3785152170 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f492806e530 0x7f49280709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:52:11.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:11.544+0000 7f4940e1e640 1 --2- 172.21.15.112:0/3785152170 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f492806e530 0x7f49280709e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f4924009b50 tx=0x7f4924009120 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:52:11.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:11.548+0000 7f49397fa640 1 -- 172.21.15.112:0/3785152170 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f492c078310 con 0x7f493c102f10 2024-09-16T00:52:11.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:11.749+0000 7f4943082640 1 -- 172.21.15.112:0/3785152170 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 35, "format": "json"} v 0) v1 -- 0x7f493c109630 con 0x7f493c102f10 2024-09-16T00:52:11.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:11.749+0000 7f49397fa640 1 -- 172.21.15.112:0/3785152170 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 35, "format": "json"}]=0 dumped fsmap epoch 35 v41) v1 ==== 107+0+5263 (secure 0 0 0) 0x7f492c07b9d0 con 0x7f493c102f10 2024-09-16T00:52:11.750 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:52:11.750 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":35,"btime":"2024-09-16T00:49:25:462044+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44343,"name":"cephfs.smithi169.gawwgw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6825/1635329957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":1635329957},{"type":"v1","addr":"172.21.15.169:6825","nonce":1635329957}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":44347,"name":"cephfs.smithi169.lgeqrj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6827/184810031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":184810031},{"type":"v1","addr":"172.21.15.169:6827","nonce":184810031}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:23:22.328199+0000","modified":"2024-09-16T00:49:25.462025+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34392,"mds_1":34404},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34392":{"gid":34392,"name":"cephfs.smithi112.ithwzj","rank":0,"incarnation":35,"state":"up:replay","state_seq":1,"addr":"172.21.15.112:6827/476229221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":476229221},{"type":"v1","addr":"172.21.15.112:6827","nonce":476229221}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34404":{"gid":34404,"name":"cephfs.smithi112.hxfwkh","rank":1,"incarnation":35,"state":"up:replay","state_seq":1,"addr":"172.21.15.112:6829/2356899876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2356899876},{"type":"v1","addr":"172.21.15.112:6829","nonce":2356899876}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-16T00:52:11.750 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 35 2024-09-16T00:52:11.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:11.752+0000 7f4943082640 1 -- 172.21.15.112:0/3785152170 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f492806e530 msgr2=0x7f49280709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:52:11.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:11.752+0000 7f4943082640 1 --2- 172.21.15.112:0/3785152170 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f492806e530 0x7f49280709e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f4924009b50 tx=0x7f4924009120 comp rx=0 tx=0).stop 2024-09-16T00:52:11.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:11.752+0000 7f4943082640 1 -- 172.21.15.112:0/3785152170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f493c102f10 msgr2=0x7f493c1137d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:52:11.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:11.752+0000 7f4943082640 1 --2- 172.21.15.112:0/3785152170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f493c102f10 0x7f493c1137d0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f492c003400 tx=0x7f492c004d10 comp rx=0 tx=0).stop 2024-09-16T00:52:11.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:11.752+0000 7f4943082640 1 -- 172.21.15.112:0/3785152170 shutdown_connections 2024-09-16T00:52:11.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:11.752+0000 7f4943082640 1 --2- 172.21.15.112:0/3785152170 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f492806e530 0x7f49280709e0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:11.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:11.752+0000 7f4943082640 1 --2- 172.21.15.112:0/3785152170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f493c102f10 0x7f493c1137d0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:11.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:11.752+0000 7f4943082640 1 --2- 172.21.15.112:0/3785152170 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f493c102570 0x7f493c113290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:11.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:11.752+0000 7f4943082640 1 -- 172.21.15.112:0/3785152170 >> 172.21.15.112:0/3785152170 conn(0x7f493c070c80 msgr2=0x7f493c105330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:52:11.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:11.752+0000 7f4943082640 1 -- 172.21.15.112:0/3785152170 shutdown_connections 2024-09-16T00:52:11.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:11.752+0000 7f4943082640 1 -- 172.21.15.112:0/3785152170 wait complete. 2024-09-16T00:52:12.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:52:11 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/3785152170' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-09-16T00:52:12.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:52:11 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/3785152170' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-09-16T00:52:12.418 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 35 2024-09-16T00:52:12.418 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 36 2024-09-16T00:52:12.667 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:52:13.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:52:12 smithi112 ceph-mon[97750]: pgmap v735: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:52:13.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:52:12 smithi169 ceph-mon[77626]: pgmap v735: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:52:14.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:14.239+0000 7fd0205d0640 1 -- 172.21.15.112:0/3048804267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd018102560 msgr2=0x7fd018102930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:52:14.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:14.239+0000 7fd0205d0640 1 --2- 172.21.15.112:0/3048804267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd018102560 0x7fd018102930 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fd0140048d0 tx=0x7fd014009da0 comp rx=0 tx=0).stop 2024-09-16T00:52:14.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:14.239+0000 7fd0205d0640 1 -- 172.21.15.112:0/3048804267 shutdown_connections 2024-09-16T00:52:14.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:14.239+0000 7fd0205d0640 1 --2- 172.21.15.112:0/3048804267 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd018102f00 0x7fd0180fc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:14.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:14.239+0000 7fd0205d0640 1 --2- 172.21.15.112:0/3048804267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd018102560 0x7fd018102930 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:14.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:14.239+0000 7fd0205d0640 1 -- 172.21.15.112:0/3048804267 >> 172.21.15.112:0/3048804267 conn(0x7fd0180f8120 msgr2=0x7fd0180fa530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:52:14.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:14.239+0000 7fd0205d0640 1 -- 172.21.15.112:0/3048804267 shutdown_connections 2024-09-16T00:52:14.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:14.239+0000 7fd0205d0640 1 -- 172.21.15.112:0/3048804267 wait complete. 2024-09-16T00:52:14.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:14.240+0000 7fd0205d0640 1 Processor -- start 2024-09-16T00:52:14.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:14.240+0000 7fd0205d0640 1 -- start start 2024-09-16T00:52:14.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:14.240+0000 7fd0205d0640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd018102560 0x7fd018113230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:52:14.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:14.240+0000 7fd0205d0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd018102f00 0x7fd018113770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:52:14.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:14.240+0000 7fd0205d0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd018113d70 con 0x7fd018102f00 2024-09-16T00:52:14.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:14.240+0000 7fd0205d0640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd01810d1b0 con 0x7fd018102560 2024-09-16T00:52:14.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:14.240+0000 7fd01db6b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd018102f00 0x7fd018113770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:52:14.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:14.240+0000 7fd01db6b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd018102f00 0x7fd018113770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36378/0 (socket says 172.21.15.112:36378) 2024-09-16T00:52:14.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:14.240+0000 7fd01db6b640 1 -- 172.21.15.112:0/3249075855 learned_addr learned my addr 172.21.15.112:0/3249075855 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:52:14.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:14.240+0000 7fd01e36c640 1 --2- 172.21.15.112:0/3249075855 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd018102560 0x7fd018113230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:52:14.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:14.240+0000 7fd01db6b640 1 -- 172.21.15.112:0/3249075855 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd018102560 msgr2=0x7fd018113230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:52:14.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:14.240+0000 7fd01db6b640 1 --2- 172.21.15.112:0/3249075855 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd018102560 0x7fd018113230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:14.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:14.240+0000 7fd01db6b640 1 -- 172.21.15.112:0/3249075855 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd014005040 con 0x7fd018102f00 2024-09-16T00:52:14.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:14.240+0000 7fd01db6b640 1 --2- 172.21.15.112:0/3249075855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd018102f00 0x7fd018113770 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fd008002840 tx=0x7fd008004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:52:14.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:14.240+0000 7fd00effd640 1 -- 172.21.15.112:0/3249075855 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd008007c30 con 0x7fd018102f00 2024-09-16T00:52:14.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:14.241+0000 7fd00effd640 1 -- 172.21.15.112:0/3249075855 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fd00800f040 con 0x7fd018102f00 2024-09-16T00:52:14.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:14.241+0000 7fd0205d0640 1 -- 172.21.15.112:0/3249075855 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd01810d490 con 0x7fd018102f00 2024-09-16T00:52:14.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:14.241+0000 7fd00effd640 1 -- 172.21.15.112:0/3249075855 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd008009410 con 0x7fd018102f00 2024-09-16T00:52:14.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:14.241+0000 7fd0205d0640 1 -- 172.21.15.112:0/3249075855 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd01810da30 con 0x7fd018102f00 2024-09-16T00:52:14.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:14.242+0000 7fd0205d0640 1 -- 172.21.15.112:0/3249075855 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0180fdce0 con 0x7fd018102f00 2024-09-16T00:52:14.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:14.242+0000 7fd00effd640 1 -- 172.21.15.112:0/3249075855 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fd0080096d0 con 0x7fd018102f00 2024-09-16T00:52:14.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:14.243+0000 7fd00effd640 1 --2- 172.21.15.112:0/3249075855 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd00406e530 0x7fd0040709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:52:14.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:14.243+0000 7fd00effd640 1 -- 172.21.15.112:0/3249075855 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7fd00808dbe0 con 0x7fd018102f00 2024-09-16T00:52:14.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:14.243+0000 7fd01e36c640 1 --2- 172.21.15.112:0/3249075855 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd00406e530 0x7fd0040709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:52:14.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:14.243+0000 7fd01e36c640 1 --2- 172.21.15.112:0/3249075855 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd00406e530 0x7fd0040709e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fd014009ac0 tx=0x7fd014035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:52:14.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:14.247+0000 7fd00effd640 1 -- 172.21.15.112:0/3249075855 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fd008056b50 con 0x7fd018102f00 2024-09-16T00:52:14.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:14.450+0000 7fd0205d0640 1 -- 172.21.15.112:0/3249075855 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 36, "format": "json"} v 0) v1 -- 0x7fd0181095d0 con 0x7fd018102f00 2024-09-16T00:52:14.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:14.451+0000 7fd00effd640 1 -- 172.21.15.112:0/3249075855 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 36, "format": "json"}]=0 dumped fsmap epoch 36 v41) v1 ==== 107+0+5262 (secure 0 0 0) 0x7fd00805a210 con 0x7fd018102f00 2024-09-16T00:52:14.451 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:52:14.452 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":36,"btime":"2024-09-16T00:49:26:465459+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44343,"name":"cephfs.smithi169.gawwgw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6825/1635329957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":1635329957},{"type":"v1","addr":"172.21.15.169:6825","nonce":1635329957}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":44347,"name":"cephfs.smithi169.lgeqrj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6827/184810031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":184810031},{"type":"v1","addr":"172.21.15.169:6827","nonce":184810031}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:23:22.328199+0000","modified":"2024-09-16T00:49:25.469589+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34392,"mds_1":34404},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34392":{"gid":34392,"name":"cephfs.smithi112.ithwzj","rank":0,"incarnation":35,"state":"up:replay","state_seq":1,"addr":"172.21.15.112:6827/476229221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":476229221},{"type":"v1","addr":"172.21.15.112:6827","nonce":476229221}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34404":{"gid":34404,"name":"cephfs.smithi112.hxfwkh","rank":1,"incarnation":35,"state":"up:replay","state_seq":1,"addr":"172.21.15.112:6829/2356899876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2356899876},{"type":"v1","addr":"172.21.15.112:6829","nonce":2356899876}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-16T00:52:14.452 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 36 2024-09-16T00:52:14.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:14.453+0000 7fd0205d0640 1 -- 172.21.15.112:0/3249075855 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd00406e530 msgr2=0x7fd0040709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:52:14.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:14.453+0000 7fd0205d0640 1 --2- 172.21.15.112:0/3249075855 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd00406e530 0x7fd0040709e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fd014009ac0 tx=0x7fd014035040 comp rx=0 tx=0).stop 2024-09-16T00:52:14.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:14.453+0000 7fd0205d0640 1 -- 172.21.15.112:0/3249075855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd018102f00 msgr2=0x7fd018113770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:52:14.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:14.453+0000 7fd0205d0640 1 --2- 172.21.15.112:0/3249075855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd018102f00 0x7fd018113770 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fd008002840 tx=0x7fd008004570 comp rx=0 tx=0).stop 2024-09-16T00:52:14.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:14.453+0000 7fd0205d0640 1 -- 172.21.15.112:0/3249075855 shutdown_connections 2024-09-16T00:52:14.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:14.453+0000 7fd0205d0640 1 --2- 172.21.15.112:0/3249075855 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fd00406e530 0x7fd0040709e0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:14.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:14.453+0000 7fd0205d0640 1 --2- 172.21.15.112:0/3249075855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd018102f00 0x7fd018113770 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:14.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:14.453+0000 7fd0205d0640 1 --2- 172.21.15.112:0/3249075855 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd018102560 0x7fd018113230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:14.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:14.453+0000 7fd0205d0640 1 -- 172.21.15.112:0/3249075855 >> 172.21.15.112:0/3249075855 conn(0x7fd0180f8120 msgr2=0x7fd0180f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:52:14.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:14.453+0000 7fd0205d0640 1 -- 172.21.15.112:0/3249075855 shutdown_connections 2024-09-16T00:52:14.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:14.453+0000 7fd0205d0640 1 -- 172.21.15.112:0/3249075855 wait complete. 2024-09-16T00:52:15.034 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:52:14 smithi112 ceph-mon[97750]: pgmap v736: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:52:15.034 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:52:14 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/3249075855' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 36, "format": "json"}]: dispatch 2024-09-16T00:52:15.059 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 37 2024-09-16T00:52:15.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:52:14 smithi169 ceph-mon[77626]: pgmap v736: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:52:15.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:52:14 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/3249075855' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 36, "format": "json"}]: dispatch 2024-09-16T00:52:15.318 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:52:16.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:52:16 smithi112 ceph-mon[97750]: pgmap v737: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:52:16.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:52:16 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:52:16.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:16.939+0000 7ff6bb61c640 1 -- 172.21.15.112:0/106618162 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6b4102600 msgr2=0x7ff6b41029d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:52:16.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:16.939+0000 7ff6bb61c640 1 --2- 172.21.15.112:0/106618162 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6b4102600 0x7ff6b41029d0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7ff6b00048d0 tx=0x7ff6b0009da0 comp rx=0 tx=0).stop 2024-09-16T00:52:16.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:16.939+0000 7ff6bb61c640 1 -- 172.21.15.112:0/106618162 shutdown_connections 2024-09-16T00:52:16.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:16.939+0000 7ff6bb61c640 1 --2- 172.21.15.112:0/106618162 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6b40fac00 0x7ff6b40fb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:16.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:16.939+0000 7ff6bb61c640 1 --2- 172.21.15.112:0/106618162 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6b4102600 0x7ff6b41029d0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:16.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:16.939+0000 7ff6bb61c640 1 -- 172.21.15.112:0/106618162 >> 172.21.15.112:0/106618162 conn(0x7ff6b4070c80 msgr2=0x7ff6b4071080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:52:16.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:16.939+0000 7ff6bb61c640 1 -- 172.21.15.112:0/106618162 shutdown_connections 2024-09-16T00:52:16.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:16.939+0000 7ff6bb61c640 1 -- 172.21.15.112:0/106618162 wait complete. 2024-09-16T00:52:16.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:16.940+0000 7ff6bb61c640 1 Processor -- start 2024-09-16T00:52:16.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:16.940+0000 7ff6bb61c640 1 -- start start 2024-09-16T00:52:16.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:16.940+0000 7ff6bb61c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6b40fac00 0x7ff6b41131d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:52:16.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:16.940+0000 7ff6bb61c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6b4102600 0x7ff6b4113710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:52:16.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:16.940+0000 7ff6bb61c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b4113d10 con 0x7ff6b40fac00 2024-09-16T00:52:16.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:16.940+0000 7ff6bb61c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b410d150 con 0x7ff6b4102600 2024-09-16T00:52:16.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:16.940+0000 7ff6b93b8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6b40fac00 0x7ff6b41131d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:52:16.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:16.940+0000 7ff6b93b8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6b40fac00 0x7ff6b41131d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36392/0 (socket says 172.21.15.112:36392) 2024-09-16T00:52:16.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:16.940+0000 7ff6b93b8640 1 -- 172.21.15.112:0/3923416251 learned_addr learned my addr 172.21.15.112:0/3923416251 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:52:16.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:16.940+0000 7ff6b8bb7640 1 --2- 172.21.15.112:0/3923416251 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6b4102600 0x7ff6b4113710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:52:16.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:16.940+0000 7ff6b93b8640 1 -- 172.21.15.112:0/3923416251 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6b4102600 msgr2=0x7ff6b4113710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:52:16.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:16.940+0000 7ff6b93b8640 1 --2- 172.21.15.112:0/3923416251 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6b4102600 0x7ff6b4113710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:16.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:16.940+0000 7ff6b93b8640 1 -- 172.21.15.112:0/3923416251 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6b0005040 con 0x7ff6b40fac00 2024-09-16T00:52:16.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:16.940+0000 7ff6b8bb7640 1 --2- 172.21.15.112:0/3923416251 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6b4102600 0x7ff6b4113710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:52:16.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:16.941+0000 7ff6b93b8640 1 --2- 172.21.15.112:0/3923416251 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6b40fac00 0x7ff6b41131d0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7ff6b00043d0 tx=0x7ff6b0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:52:16.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:16.941+0000 7ff6a9ffb640 1 -- 172.21.15.112:0/3923416251 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff6b0038070 con 0x7ff6b40fac00 2024-09-16T00:52:16.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:16.941+0000 7ff6a9ffb640 1 -- 172.21.15.112:0/3923416251 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7ff6b00096e0 con 0x7ff6b40fac00 2024-09-16T00:52:16.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:16.941+0000 7ff6bb61c640 1 -- 172.21.15.112:0/3923416251 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6b410d3d0 con 0x7ff6b40fac00 2024-09-16T00:52:16.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:16.941+0000 7ff6a9ffb640 1 -- 172.21.15.112:0/3923416251 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff6b002c830 con 0x7ff6b40fac00 2024-09-16T00:52:16.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:16.941+0000 7ff6bb61c640 1 -- 172.21.15.112:0/3923416251 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6b410d910 con 0x7ff6b40fac00 2024-09-16T00:52:16.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:16.942+0000 7ff6bb61c640 1 -- 172.21.15.112:0/3923416251 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6b4065890 con 0x7ff6b40fac00 2024-09-16T00:52:16.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:16.943+0000 7ff6a9ffb640 1 -- 172.21.15.112:0/3923416251 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7ff6b0030020 con 0x7ff6b40fac00 2024-09-16T00:52:16.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:16.943+0000 7ff6a9ffb640 1 --2- 172.21.15.112:0/3923416251 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff6a006e530 0x7ff6a00709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:52:16.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:16.943+0000 7ff6a9ffb640 1 -- 172.21.15.112:0/3923416251 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7ff6b00ae0f0 con 0x7ff6b40fac00 2024-09-16T00:52:16.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:16.943+0000 7ff6b8bb7640 1 --2- 172.21.15.112:0/3923416251 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff6a006e530 0x7ff6a00709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:52:16.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:16.944+0000 7ff6b8bb7640 1 --2- 172.21.15.112:0/3923416251 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff6a006e530 0x7ff6a00709e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff6b410ea90 tx=0x7ff6ac00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:52:16.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:16.948+0000 7ff6a9ffb640 1 -- 172.21.15.112:0/3923416251 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7ff6b0077060 con 0x7ff6b40fac00 2024-09-16T00:52:17.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:17.147+0000 7ff6bb61c640 1 -- 172.21.15.112:0/3923416251 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 37, "format": "json"} v 0) v1 -- 0x7ff6b404b750 con 0x7ff6b40fac00 2024-09-16T00:52:17.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:17.148+0000 7ff6a9ffb640 1 -- 172.21.15.112:0/3923416251 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 37, "format": "json"}]=0 dumped fsmap epoch 37 v41) v1 ==== 107+0+5265 (secure 0 0 0) 0x7ff6b003e080 con 0x7ff6b40fac00 2024-09-16T00:52:17.148 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:52:17.148 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":37,"btime":"2024-09-16T00:49:30:931094+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44343,"name":"cephfs.smithi169.gawwgw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6825/1635329957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":1635329957},{"type":"v1","addr":"172.21.15.169:6825","nonce":1635329957}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":44347,"name":"cephfs.smithi169.lgeqrj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6827/184810031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":184810031},{"type":"v1","addr":"172.21.15.169:6827","nonce":184810031}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":37,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:23:22.328199+0000","modified":"2024-09-16T00:49:30.362758+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34392,"mds_1":34404},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34392":{"gid":34392,"name":"cephfs.smithi112.ithwzj","rank":0,"incarnation":35,"state":"up:replay","state_seq":1,"addr":"172.21.15.112:6827/476229221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":476229221},{"type":"v1","addr":"172.21.15.112:6827","nonce":476229221}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34404":{"gid":34404,"name":"cephfs.smithi112.hxfwkh","rank":1,"incarnation":35,"state":"up:resolve","state_seq":235,"addr":"172.21.15.112:6829/2356899876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2356899876},{"type":"v1","addr":"172.21.15.112:6829","nonce":2356899876}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-16T00:52:17.148 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 37 2024-09-16T00:52:17.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:17.150+0000 7ff6bb61c640 1 -- 172.21.15.112:0/3923416251 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff6a006e530 msgr2=0x7ff6a00709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:52:17.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:17.150+0000 7ff6bb61c640 1 --2- 172.21.15.112:0/3923416251 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff6a006e530 0x7ff6a00709e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff6b410ea90 tx=0x7ff6ac00b040 comp rx=0 tx=0).stop 2024-09-16T00:52:17.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:17.150+0000 7ff6bb61c640 1 -- 172.21.15.112:0/3923416251 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6b40fac00 msgr2=0x7ff6b41131d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:52:17.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:17.150+0000 7ff6bb61c640 1 --2- 172.21.15.112:0/3923416251 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6b40fac00 0x7ff6b41131d0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7ff6b00043d0 tx=0x7ff6b0009da0 comp rx=0 tx=0).stop 2024-09-16T00:52:17.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:17.150+0000 7ff6bb61c640 1 -- 172.21.15.112:0/3923416251 shutdown_connections 2024-09-16T00:52:17.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:17.150+0000 7ff6bb61c640 1 --2- 172.21.15.112:0/3923416251 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff6a006e530 0x7ff6a00709e0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:17.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:17.150+0000 7ff6bb61c640 1 --2- 172.21.15.112:0/3923416251 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6b4102600 0x7ff6b4113710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:17.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:17.150+0000 7ff6bb61c640 1 --2- 172.21.15.112:0/3923416251 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6b40fac00 0x7ff6b41131d0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:17.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:17.150+0000 7ff6bb61c640 1 -- 172.21.15.112:0/3923416251 >> 172.21.15.112:0/3923416251 conn(0x7ff6b4070c80 msgr2=0x7ff6b4105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:52:17.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:17.150+0000 7ff6bb61c640 1 -- 172.21.15.112:0/3923416251 shutdown_connections 2024-09-16T00:52:17.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:17.150+0000 7ff6bb61c640 1 -- 172.21.15.112:0/3923416251 wait complete. 2024-09-16T00:52:17.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:52:16 smithi169 ceph-mon[77626]: pgmap v737: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:52:17.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:52:16 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:52:17.843 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 38 2024-09-16T00:52:18.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:52:17 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/3923416251' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 37, "format": "json"}]: dispatch 2024-09-16T00:52:18.093 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:52:18.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:52:17 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/3923416251' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 37, "format": "json"}]: dispatch 2024-09-16T00:52:19.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:52:18 smithi112 ceph-mon[97750]: pgmap v738: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:52:19.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:52:18 smithi169 ceph-mon[77626]: pgmap v738: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:52:19.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:19.753+0000 7f8f64972640 1 -- 172.21.15.112:0/1478263147 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f600fb4a0 msgr2=0x7f8f600fb910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:52:19.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:19.753+0000 7f8f64972640 1 --2- 172.21.15.112:0/1478263147 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f600fb4a0 0x7f8f600fb910 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f8f480048d0 tx=0x7f8f48009da0 comp rx=0 tx=0).stop 2024-09-16T00:52:19.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:19.753+0000 7f8f64972640 1 -- 172.21.15.112:0/1478263147 shutdown_connections 2024-09-16T00:52:19.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:19.753+0000 7f8f64972640 1 --2- 172.21.15.112:0/1478263147 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f600fb4a0 0x7f8f600fb910 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:19.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:19.753+0000 7f8f64972640 1 --2- 172.21.15.112:0/1478263147 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f601014e0 0x7f8f601018b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:19.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:19.753+0000 7f8f64972640 1 -- 172.21.15.112:0/1478263147 >> 172.21.15.112:0/1478263147 conn(0x7f8f600f7060 msgr2=0x7f8f600f9470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:52:19.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:19.753+0000 7f8f64972640 1 -- 172.21.15.112:0/1478263147 shutdown_connections 2024-09-16T00:52:19.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:19.753+0000 7f8f64972640 1 -- 172.21.15.112:0/1478263147 wait complete. 2024-09-16T00:52:19.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:19.754+0000 7f8f64972640 1 Processor -- start 2024-09-16T00:52:19.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:19.754+0000 7f8f64972640 1 -- start start 2024-09-16T00:52:19.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:19.754+0000 7f8f64972640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f600fb4a0 0x7f8f601120d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:52:19.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:19.754+0000 7f8f64972640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f601014e0 0x7f8f60112610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:52:19.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:19.754+0000 7f8f64972640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f60112c10 con 0x7f8f601014e0 2024-09-16T00:52:19.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:19.754+0000 7f8f64972640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6010c050 con 0x7f8f600fb4a0 2024-09-16T00:52:19.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:19.754+0000 7f8f57fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f601014e0 0x7f8f60112610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:52:19.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:19.754+0000 7f8f57fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f601014e0 0x7f8f60112610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36418/0 (socket says 172.21.15.112:36418) 2024-09-16T00:52:19.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:19.754+0000 7f8f57fff640 1 -- 172.21.15.112:0/309020608 learned_addr learned my addr 172.21.15.112:0/309020608 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:52:19.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:19.754+0000 7f8f5e59c640 1 --2- 172.21.15.112:0/309020608 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f600fb4a0 0x7f8f601120d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:52:19.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:19.755+0000 7f8f57fff640 1 -- 172.21.15.112:0/309020608 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f600fb4a0 msgr2=0x7f8f601120d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:52:19.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:19.755+0000 7f8f57fff640 1 --2- 172.21.15.112:0/309020608 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f600fb4a0 0x7f8f601120d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:19.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:19.755+0000 7f8f57fff640 1 -- 172.21.15.112:0/309020608 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f48005040 con 0x7f8f601014e0 2024-09-16T00:52:19.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:19.755+0000 7f8f57fff640 1 --2- 172.21.15.112:0/309020608 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f601014e0 0x7f8f60112610 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f8f48000c00 tx=0x7f8f48009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:52:19.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:19.755+0000 7f8f56ffd640 1 -- 172.21.15.112:0/309020608 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f48038070 con 0x7f8f601014e0 2024-09-16T00:52:19.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:19.755+0000 7f8f56ffd640 1 -- 172.21.15.112:0/309020608 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f8f4802ade0 con 0x7f8f601014e0 2024-09-16T00:52:19.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:19.755+0000 7f8f64972640 1 -- 172.21.15.112:0/309020608 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f6010c2d0 con 0x7f8f601014e0 2024-09-16T00:52:19.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:19.755+0000 7f8f56ffd640 1 -- 172.21.15.112:0/309020608 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f4802c7e0 con 0x7f8f601014e0 2024-09-16T00:52:19.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:19.755+0000 7f8f64972640 1 -- 172.21.15.112:0/309020608 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f6010c810 con 0x7f8f601014e0 2024-09-16T00:52:19.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:19.756+0000 7f8f64972640 1 -- 172.21.15.112:0/309020608 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f600fcbe0 con 0x7f8f601014e0 2024-09-16T00:52:19.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:19.756+0000 7f8f56ffd640 1 -- 172.21.15.112:0/309020608 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f8f48030020 con 0x7f8f601014e0 2024-09-16T00:52:19.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:19.757+0000 7f8f56ffd640 1 --2- 172.21.15.112:0/309020608 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f8f4c06e530 0x7f8f4c0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:52:19.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:19.757+0000 7f8f56ffd640 1 -- 172.21.15.112:0/309020608 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f8f480adf70 con 0x7f8f601014e0 2024-09-16T00:52:19.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:19.757+0000 7f8f5e59c640 1 --2- 172.21.15.112:0/309020608 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f8f4c06e530 0x7f8f4c0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:52:19.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:19.757+0000 7f8f5e59c640 1 --2- 172.21.15.112:0/309020608 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f8f4c06e530 0x7f8f4c0709e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8f50007fd0 tx=0x7f8f50009e90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:52:19.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:19.761+0000 7f8f56ffd640 1 -- 172.21.15.112:0/309020608 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f8f480e31b0 con 0x7f8f601014e0 2024-09-16T00:52:19.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:19.961+0000 7f8f64972640 1 -- 172.21.15.112:0/309020608 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 38, "format": "json"} v 0) v1 -- 0x7f8f60108480 con 0x7f8f601014e0 2024-09-16T00:52:19.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:19.961+0000 7f8f56ffd640 1 -- 172.21.15.112:0/309020608 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 38, "format": "json"}]=0 dumped fsmap epoch 38 v41) v1 ==== 107+0+5268 (secure 0 0 0) 0x7f8f48076f90 con 0x7f8f601014e0 2024-09-16T00:52:19.962 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:52:19.962 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":38,"btime":"2024-09-16T00:49:32:558730+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44343,"name":"cephfs.smithi169.gawwgw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6825/1635329957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":1635329957},{"type":"v1","addr":"172.21.15.169:6825","nonce":1635329957}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":44347,"name":"cephfs.smithi169.lgeqrj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6827/184810031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":184810031},{"type":"v1","addr":"172.21.15.169:6827","nonce":184810031}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":38,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:23:22.328199+0000","modified":"2024-09-16T00:49:31.590216+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34392,"mds_1":34404},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34392":{"gid":34392,"name":"cephfs.smithi112.ithwzj","rank":0,"incarnation":35,"state":"up:resolve","state_seq":240,"addr":"172.21.15.112:6827/476229221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":476229221},{"type":"v1","addr":"172.21.15.112:6827","nonce":476229221}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34404":{"gid":34404,"name":"cephfs.smithi112.hxfwkh","rank":1,"incarnation":35,"state":"up:resolve","state_seq":235,"addr":"172.21.15.112:6829/2356899876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2356899876},{"type":"v1","addr":"172.21.15.112:6829","nonce":2356899876}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-16T00:52:19.962 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 38 2024-09-16T00:52:19.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:19.964+0000 7f8f64972640 1 -- 172.21.15.112:0/309020608 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f8f4c06e530 msgr2=0x7f8f4c0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:52:19.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:19.964+0000 7f8f64972640 1 --2- 172.21.15.112:0/309020608 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f8f4c06e530 0x7f8f4c0709e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8f50007fd0 tx=0x7f8f50009e90 comp rx=0 tx=0).stop 2024-09-16T00:52:19.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:19.964+0000 7f8f64972640 1 -- 172.21.15.112:0/309020608 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f601014e0 msgr2=0x7f8f60112610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:52:19.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:19.964+0000 7f8f64972640 1 --2- 172.21.15.112:0/309020608 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f601014e0 0x7f8f60112610 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f8f48000c00 tx=0x7f8f48009da0 comp rx=0 tx=0).stop 2024-09-16T00:52:19.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:19.964+0000 7f8f64972640 1 -- 172.21.15.112:0/309020608 shutdown_connections 2024-09-16T00:52:19.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:19.964+0000 7f8f64972640 1 --2- 172.21.15.112:0/309020608 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f8f4c06e530 0x7f8f4c0709e0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:19.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:19.964+0000 7f8f64972640 1 --2- 172.21.15.112:0/309020608 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f601014e0 0x7f8f60112610 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:19.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:19.964+0000 7f8f64972640 1 --2- 172.21.15.112:0/309020608 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f600fb4a0 0x7f8f601120d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:19.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:19.964+0000 7f8f64972640 1 -- 172.21.15.112:0/309020608 >> 172.21.15.112:0/309020608 conn(0x7f8f600f7060 msgr2=0x7f8f600f8910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:52:19.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:19.964+0000 7f8f64972640 1 -- 172.21.15.112:0/309020608 shutdown_connections 2024-09-16T00:52:19.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:19.964+0000 7f8f64972640 1 -- 172.21.15.112:0/309020608 wait complete. 2024-09-16T00:52:20.562 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 39 2024-09-16T00:52:20.809 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:52:20.836 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:52:20 smithi112 ceph-mon[97750]: pgmap v739: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:52:21.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:52:20 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/309020608' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 38, "format": "json"}]: dispatch 2024-09-16T00:52:21.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:52:20 smithi169 ceph-mon[77626]: pgmap v739: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:52:21.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:52:20 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/309020608' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 38, "format": "json"}]: dispatch 2024-09-16T00:52:22.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:22.492+0000 7fe1311e9640 1 -- 172.21.15.112:0/2371382000 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe12c1014e0 msgr2=0x7fe12c1018b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:52:22.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:22.492+0000 7fe1311e9640 1 --2- 172.21.15.112:0/2371382000 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe12c1014e0 0x7fe12c1018b0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fe11c0048d0 tx=0x7fe11c009da0 comp rx=0 tx=0).stop 2024-09-16T00:52:22.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:22.492+0000 7fe1311e9640 1 -- 172.21.15.112:0/2371382000 shutdown_connections 2024-09-16T00:52:22.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:22.492+0000 7fe1311e9640 1 --2- 172.21.15.112:0/2371382000 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe12c0fb4a0 0x7fe12c0fb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:22.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:22.492+0000 7fe1311e9640 1 --2- 172.21.15.112:0/2371382000 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe12c1014e0 0x7fe12c1018b0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:22.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:22.492+0000 7fe1311e9640 1 -- 172.21.15.112:0/2371382000 >> 172.21.15.112:0/2371382000 conn(0x7fe12c0f7020 msgr2=0x7fe12c0f9430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:52:22.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:22.492+0000 7fe1311e9640 1 -- 172.21.15.112:0/2371382000 shutdown_connections 2024-09-16T00:52:22.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:22.492+0000 7fe1311e9640 1 -- 172.21.15.112:0/2371382000 wait complete. 2024-09-16T00:52:22.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:22.493+0000 7fe1311e9640 1 Processor -- start 2024-09-16T00:52:22.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:22.493+0000 7fe1311e9640 1 -- start start 2024-09-16T00:52:22.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:22.493+0000 7fe1311e9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe12c0fb4a0 0x7fe12c10e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:52:22.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:22.493+0000 7fe1311e9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe12c1014e0 0x7fe12c10e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:52:22.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:22.493+0000 7fe1311e9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe12c10ee00 con 0x7fe12c1014e0 2024-09-16T00:52:22.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:22.493+0000 7fe1311e9640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe12c10ef70 con 0x7fe12c0fb4a0 2024-09-16T00:52:22.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:22.493+0000 7fe123fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe12c1014e0 0x7fe12c10e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:52:22.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:22.493+0000 7fe123fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe12c1014e0 0x7fe12c10e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58612/0 (socket says 172.21.15.112:58612) 2024-09-16T00:52:22.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:22.493+0000 7fe123fff640 1 -- 172.21.15.112:0/1339751392 learned_addr learned my addr 172.21.15.112:0/1339751392 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:52:22.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:22.493+0000 7fe12ad9d640 1 --2- 172.21.15.112:0/1339751392 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe12c0fb4a0 0x7fe12c10e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:52:22.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:22.493+0000 7fe123fff640 1 -- 172.21.15.112:0/1339751392 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe12c0fb4a0 msgr2=0x7fe12c10e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:52:22.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:22.493+0000 7fe123fff640 1 --2- 172.21.15.112:0/1339751392 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe12c0fb4a0 0x7fe12c10e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:22.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:22.493+0000 7fe123fff640 1 -- 172.21.15.112:0/1339751392 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe11c005040 con 0x7fe12c1014e0 2024-09-16T00:52:22.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:22.493+0000 7fe12ad9d640 1 --2- 172.21.15.112:0/1339751392 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe12c0fb4a0 0x7fe12c10e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:52:22.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:22.494+0000 7fe123fff640 1 --2- 172.21.15.112:0/1339751392 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe12c1014e0 0x7fe12c10e720 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fe11400b790 tx=0x7fe1140091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:52:22.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:22.494+0000 7fe1237fe640 1 -- 172.21.15.112:0/1339751392 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe11400c700 con 0x7fe12c1014e0 2024-09-16T00:52:22.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:22.494+0000 7fe1237fe640 1 -- 172.21.15.112:0/1339751392 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7fe11400cd00 con 0x7fe12c1014e0 2024-09-16T00:52:22.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:22.494+0000 7fe1311e9640 1 -- 172.21.15.112:0/1339751392 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe12c117250 con 0x7fe12c1014e0 2024-09-16T00:52:22.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:22.494+0000 7fe1237fe640 1 -- 172.21.15.112:0/1339751392 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe114015e70 con 0x7fe12c1014e0 2024-09-16T00:52:22.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:22.494+0000 7fe1311e9640 1 -- 172.21.15.112:0/1339751392 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe12c117770 con 0x7fe12c1014e0 2024-09-16T00:52:22.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:22.495+0000 7fe1311e9640 1 -- 172.21.15.112:0/1339751392 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe12c0fcbe0 con 0x7fe12c1014e0 2024-09-16T00:52:22.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:22.495+0000 7fe1237fe640 1 -- 172.21.15.112:0/1339751392 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7fe114012070 con 0x7fe12c1014e0 2024-09-16T00:52:22.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:22.496+0000 7fe1237fe640 1 --2- 172.21.15.112:0/1339751392 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fe11806e530 0x7fe1180709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:52:22.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:22.496+0000 7fe1237fe640 1 -- 172.21.15.112:0/1339751392 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7fe11408dea0 con 0x7fe12c1014e0 2024-09-16T00:52:22.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:22.496+0000 7fe12ad9d640 1 --2- 172.21.15.112:0/1339751392 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fe11806e530 0x7fe1180709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:52:22.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:22.496+0000 7fe12ad9d640 1 --2- 172.21.15.112:0/1339751392 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fe11806e530 0x7fe1180709e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe11c007ac0 tx=0x7fe11c035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:52:22.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:22.500+0000 7fe1237fe640 1 -- 172.21.15.112:0/1339751392 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fe114056df0 con 0x7fe12c1014e0 2024-09-16T00:52:22.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:22.704+0000 7fe1311e9640 1 -- 172.21.15.112:0/1339751392 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 39, "format": "json"} v 0) v1 -- 0x7fe12c108510 con 0x7fe12c1014e0 2024-09-16T00:52:22.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:22.704+0000 7fe1237fe640 1 -- 172.21.15.112:0/1339751392 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 39, "format": "json"}]=0 dumped fsmap epoch 39 v41) v1 ==== 107+0+5272 (secure 0 0 0) 0x7fe11405a4b0 con 0x7fe12c1014e0 2024-09-16T00:52:22.705 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:52:22.705 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":39,"btime":"2024-09-16T00:49:33:560940+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44343,"name":"cephfs.smithi169.gawwgw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6825/1635329957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":1635329957},{"type":"v1","addr":"172.21.15.169:6825","nonce":1635329957}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":44347,"name":"cephfs.smithi169.lgeqrj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6827/184810031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":184810031},{"type":"v1","addr":"172.21.15.169:6827","nonce":184810031}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:23:22.328199+0000","modified":"2024-09-16T00:49:32.598117+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34392,"mds_1":34404},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34392":{"gid":34392,"name":"cephfs.smithi112.ithwzj","rank":0,"incarnation":35,"state":"up:reconnect","state_seq":241,"addr":"172.21.15.112:6827/476229221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":476229221},{"type":"v1","addr":"172.21.15.112:6827","nonce":476229221}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34404":{"gid":34404,"name":"cephfs.smithi112.hxfwkh","rank":1,"incarnation":35,"state":"up:reconnect","state_seq":236,"addr":"172.21.15.112:6829/2356899876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2356899876},{"type":"v1","addr":"172.21.15.112:6829","nonce":2356899876}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-16T00:52:22.705 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 39 2024-09-16T00:52:22.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:22.707+0000 7fe1311e9640 1 -- 172.21.15.112:0/1339751392 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fe11806e530 msgr2=0x7fe1180709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:52:22.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:22.707+0000 7fe1311e9640 1 --2- 172.21.15.112:0/1339751392 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fe11806e530 0x7fe1180709e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe11c007ac0 tx=0x7fe11c035040 comp rx=0 tx=0).stop 2024-09-16T00:52:22.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:22.707+0000 7fe1311e9640 1 -- 172.21.15.112:0/1339751392 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe12c1014e0 msgr2=0x7fe12c10e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:52:22.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:22.707+0000 7fe1311e9640 1 --2- 172.21.15.112:0/1339751392 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe12c1014e0 0x7fe12c10e720 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fe11400b790 tx=0x7fe1140091d0 comp rx=0 tx=0).stop 2024-09-16T00:52:22.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:22.707+0000 7fe1311e9640 1 -- 172.21.15.112:0/1339751392 shutdown_connections 2024-09-16T00:52:22.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:22.707+0000 7fe1311e9640 1 --2- 172.21.15.112:0/1339751392 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7fe11806e530 0x7fe1180709e0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:22.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:22.707+0000 7fe1311e9640 1 --2- 172.21.15.112:0/1339751392 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe12c1014e0 0x7fe12c10e720 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:22.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:22.707+0000 7fe1311e9640 1 --2- 172.21.15.112:0/1339751392 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe12c0fb4a0 0x7fe12c10e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:22.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:22.707+0000 7fe1311e9640 1 -- 172.21.15.112:0/1339751392 >> 172.21.15.112:0/1339751392 conn(0x7fe12c0f7020 msgr2=0x7fe12c0f8850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:52:22.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:22.707+0000 7fe1311e9640 1 -- 172.21.15.112:0/1339751392 shutdown_connections 2024-09-16T00:52:22.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:22.707+0000 7fe1311e9640 1 -- 172.21.15.112:0/1339751392 wait complete. 2024-09-16T00:52:23.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:52:22 smithi112 ceph-mon[97750]: pgmap v740: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:52:23.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:52:22 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/1339751392' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 39, "format": "json"}]: dispatch 2024-09-16T00:52:23.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:52:22 smithi169 ceph-mon[77626]: pgmap v740: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:52:23.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:52:22 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/1339751392' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 39, "format": "json"}]: dispatch 2024-09-16T00:52:23.319 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 40 2024-09-16T00:52:23.566 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:52:25.008 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:52:24 smithi112 ceph-mon[97750]: pgmap v741: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:52:25.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:25.189+0000 7ff13b6f2640 1 -- 172.21.15.112:0/3162371356 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1340fb810 msgr2=0x7ff1340fbbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:52:25.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:25.189+0000 7ff13b6f2640 1 --2- 172.21.15.112:0/3162371356 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1340fb810 0x7ff1340fbbe0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7ff1300048d0 tx=0x7ff130009da0 comp rx=0 tx=0).stop 2024-09-16T00:52:25.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:25.189+0000 7ff13b6f2640 1 -- 172.21.15.112:0/3162371356 shutdown_connections 2024-09-16T00:52:25.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:25.189+0000 7ff13b6f2640 1 --2- 172.21.15.112:0/3162371356 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1340fcf80 0x7ff1340fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:25.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:25.189+0000 7ff13b6f2640 1 --2- 172.21.15.112:0/3162371356 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1340fb810 0x7ff1340fbbe0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:25.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:25.189+0000 7ff13b6f2640 1 -- 172.21.15.112:0/3162371356 >> 172.21.15.112:0/3162371356 conn(0x7ff1340723b0 msgr2=0x7ff1340727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:52:25.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:25.190+0000 7ff13b6f2640 1 -- 172.21.15.112:0/3162371356 shutdown_connections 2024-09-16T00:52:25.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:25.190+0000 7ff13b6f2640 1 -- 172.21.15.112:0/3162371356 wait complete. 2024-09-16T00:52:25.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:25.190+0000 7ff13b6f2640 1 Processor -- start 2024-09-16T00:52:25.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:25.190+0000 7ff13b6f2640 1 -- start start 2024-09-16T00:52:25.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:25.190+0000 7ff13b6f2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1340fb810 0x7ff13410e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:52:25.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:25.190+0000 7ff13b6f2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1340fcf80 0x7ff13410e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:52:25.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:25.190+0000 7ff13b6f2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff13410ed40 con 0x7ff1340fb810 2024-09-16T00:52:25.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:25.190+0000 7ff13b6f2640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff13410eeb0 con 0x7ff1340fcf80 2024-09-16T00:52:25.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:25.190+0000 7ff13948e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1340fb810 0x7ff13410e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:52:25.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:25.190+0000 7ff13948e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1340fb810 0x7ff13410e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58626/0 (socket says 172.21.15.112:58626) 2024-09-16T00:52:25.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:25.190+0000 7ff13948e640 1 -- 172.21.15.112:0/2756209919 learned_addr learned my addr 172.21.15.112:0/2756209919 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:52:25.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:25.191+0000 7ff138c8d640 1 --2- 172.21.15.112:0/2756209919 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1340fcf80 0x7ff13410e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:52:25.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:25.191+0000 7ff13948e640 1 -- 172.21.15.112:0/2756209919 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1340fcf80 msgr2=0x7ff13410e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:52:25.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:25.191+0000 7ff13948e640 1 --2- 172.21.15.112:0/2756209919 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1340fcf80 0x7ff13410e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:25.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:25.191+0000 7ff13948e640 1 -- 172.21.15.112:0/2756209919 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff130005040 con 0x7ff1340fb810 2024-09-16T00:52:25.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:25.191+0000 7ff138c8d640 1 --2- 172.21.15.112:0/2756209919 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1340fcf80 0x7ff13410e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:52:25.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:25.191+0000 7ff13948e640 1 --2- 172.21.15.112:0/2756209919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1340fb810 0x7ff13410e1b0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7ff130000c00 tx=0x7ff130009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:52:25.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:25.191+0000 7ff129ffb640 1 -- 172.21.15.112:0/2756209919 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff130038070 con 0x7ff1340fb810 2024-09-16T00:52:25.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:25.191+0000 7ff129ffb640 1 -- 172.21.15.112:0/2756209919 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7ff13002ade0 con 0x7ff1340fb810 2024-09-16T00:52:25.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:25.191+0000 7ff129ffb640 1 -- 172.21.15.112:0/2756209919 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff13002c7e0 con 0x7ff1340fb810 2024-09-16T00:52:25.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:25.191+0000 7ff13b6f2640 1 -- 172.21.15.112:0/2756209919 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff134117220 con 0x7ff1340fb810 2024-09-16T00:52:25.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:25.191+0000 7ff13b6f2640 1 -- 172.21.15.112:0/2756209919 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff134117680 con 0x7ff1340fb810 2024-09-16T00:52:25.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:25.192+0000 7ff13b6f2640 1 -- 172.21.15.112:0/2756209919 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1340fcbc0 con 0x7ff1340fb810 2024-09-16T00:52:25.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:25.193+0000 7ff129ffb640 1 -- 172.21.15.112:0/2756209919 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7ff130030020 con 0x7ff1340fb810 2024-09-16T00:52:25.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:25.193+0000 7ff129ffb640 1 --2- 172.21.15.112:0/2756209919 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff12006e530 0x7ff1200709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:52:25.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:25.193+0000 7ff129ffb640 1 -- 172.21.15.112:0/2756209919 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7ff1300aded0 con 0x7ff1340fb810 2024-09-16T00:52:25.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:25.193+0000 7ff138c8d640 1 --2- 172.21.15.112:0/2756209919 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff12006e530 0x7ff1200709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:52:25.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:25.194+0000 7ff138c8d640 1 --2- 172.21.15.112:0/2756209919 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff12006e530 0x7ff1200709e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff134065980 tx=0x7ff12400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:52:25.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:25.198+0000 7ff129ffb640 1 -- 172.21.15.112:0/2756209919 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7ff130076f70 con 0x7ff1340fb810 2024-09-16T00:52:25.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:52:24 smithi169 ceph-mon[77626]: pgmap v741: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:52:25.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:25.399+0000 7ff13b6f2640 1 -- 172.21.15.112:0/2756209919 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 40, "format": "json"} v 0) v1 -- 0x7ff134065730 con 0x7ff1340fb810 2024-09-16T00:52:25.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:25.400+0000 7ff129ffb640 1 -- 172.21.15.112:0/2756209919 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 40, "format": "json"}]=0 dumped fsmap epoch 40 v41) v1 ==== 107+0+5266 (secure 0 0 0) 0x7ff13007a630 con 0x7ff1340fb810 2024-09-16T00:52:25.401 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-16T00:52:25.401 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":40,"btime":"2024-09-16T00:49:34:566444+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44343,"name":"cephfs.smithi169.gawwgw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6825/1635329957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":1635329957},{"type":"v1","addr":"172.21.15.169:6825","nonce":1635329957}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":44347,"name":"cephfs.smithi169.lgeqrj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6827/184810031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":184810031},{"type":"v1","addr":"172.21.15.169:6827","nonce":184810031}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":40,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:23:22.328199+0000","modified":"2024-09-16T00:49:33.570222+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34392,"mds_1":34404},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34392":{"gid":34392,"name":"cephfs.smithi112.ithwzj","rank":0,"incarnation":35,"state":"up:rejoin","state_seq":242,"addr":"172.21.15.112:6827/476229221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":476229221},{"type":"v1","addr":"172.21.15.112:6827","nonce":476229221}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34404":{"gid":34404,"name":"cephfs.smithi112.hxfwkh","rank":1,"incarnation":35,"state":"up:rejoin","state_seq":237,"addr":"172.21.15.112:6829/2356899876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2356899876},{"type":"v1","addr":"172.21.15.112:6829","nonce":2356899876}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-16T00:52:25.402 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 40 2024-09-16T00:52:25.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:25.402+0000 7ff13b6f2640 1 -- 172.21.15.112:0/2756209919 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff12006e530 msgr2=0x7ff1200709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:52:25.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:25.402+0000 7ff13b6f2640 1 --2- 172.21.15.112:0/2756209919 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff12006e530 0x7ff1200709e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff134065980 tx=0x7ff12400b040 comp rx=0 tx=0).stop 2024-09-16T00:52:25.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:25.402+0000 7ff13b6f2640 1 -- 172.21.15.112:0/2756209919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1340fb810 msgr2=0x7ff13410e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:52:25.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:25.402+0000 7ff13b6f2640 1 --2- 172.21.15.112:0/2756209919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1340fb810 0x7ff13410e1b0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7ff130000c00 tx=0x7ff130009da0 comp rx=0 tx=0).stop 2024-09-16T00:52:25.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:25.402+0000 7ff13b6f2640 1 -- 172.21.15.112:0/2756209919 shutdown_connections 2024-09-16T00:52:25.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:25.402+0000 7ff13b6f2640 1 --2- 172.21.15.112:0/2756209919 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7ff12006e530 0x7ff1200709e0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:25.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:25.402+0000 7ff13b6f2640 1 --2- 172.21.15.112:0/2756209919 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1340fcf80 0x7ff13410e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:25.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:25.402+0000 7ff13b6f2640 1 --2- 172.21.15.112:0/2756209919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1340fb810 0x7ff13410e1b0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:25.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:25.402+0000 7ff13b6f2640 1 -- 172.21.15.112:0/2756209919 >> 172.21.15.112:0/2756209919 conn(0x7ff1340723b0 msgr2=0x7ff1340fa480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:52:25.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:25.402+0000 7ff13b6f2640 1 -- 172.21.15.112:0/2756209919 shutdown_connections 2024-09-16T00:52:25.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:25.402+0000 7ff13b6f2640 1 -- 172.21.15.112:0/2756209919 wait complete. 2024-09-16T00:52:26.070 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:52:25 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/2756209919' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 40, "format": "json"}]: dispatch 2024-09-16T00:52:26.095 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-09-16T00:52:26.105 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-09-16T00:52:26.105 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-16T00:52:26.105 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T00:52:26.123 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-16T00:52:26.123 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T00:52:26.180 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph osd blocklist ls 2024-09-16T00:52:26.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:52:25 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/2756209919' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 40, "format": "json"}]: dispatch 2024-09-16T00:52:26.468 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:52:27.177 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:52:26 smithi169 ceph-mon[77626]: pgmap v742: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:52:27.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:52:26 smithi112 ceph-mon[97750]: pgmap v742: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:52:28.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:28.021+0000 7f7803389640 1 -- 172.21.15.112:0/770164066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77fc0fb480 msgr2=0x7f77fc0fb8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:52:28.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:28.021+0000 7f7803389640 1 --2- 172.21.15.112:0/770164066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77fc0fb480 0x7f77fc0fb8f0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f77ec0048d0 tx=0x7f77ec009da0 comp rx=0 tx=0).stop 2024-09-16T00:52:28.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:28.021+0000 7f7803389640 1 -- 172.21.15.112:0/770164066 shutdown_connections 2024-09-16T00:52:28.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:28.021+0000 7f7803389640 1 --2- 172.21.15.112:0/770164066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77fc0fb480 0x7f77fc0fb8f0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:28.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:28.021+0000 7f7803389640 1 --2- 172.21.15.112:0/770164066 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77fc1014c0 0x7f77fc101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:28.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:28.021+0000 7f7803389640 1 -- 172.21.15.112:0/770164066 >> 172.21.15.112:0/770164066 conn(0x7f77fc0f7000 msgr2=0x7f77fc0f9410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:52:28.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:28.021+0000 7f7803389640 1 -- 172.21.15.112:0/770164066 shutdown_connections 2024-09-16T00:52:28.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:28.021+0000 7f7803389640 1 -- 172.21.15.112:0/770164066 wait complete. 2024-09-16T00:52:28.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:28.022+0000 7f7803389640 1 Processor -- start 2024-09-16T00:52:28.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:28.022+0000 7f7803389640 1 -- start start 2024-09-16T00:52:28.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:28.022+0000 7f7803389640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77fc0fb480 0x7f77fc10e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:52:28.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:28.022+0000 7f7803389640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77fc1014c0 0x7f77fc10e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:52:28.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:28.022+0000 7f7803389640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77fc10ecb0 con 0x7f77fc0fb480 2024-09-16T00:52:28.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:28.022+0000 7f7803389640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77fc10ee20 con 0x7f77fc1014c0 2024-09-16T00:52:28.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:28.022+0000 7f7801125640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77fc0fb480 0x7f77fc10e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:52:28.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:28.022+0000 7f7801125640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77fc0fb480 0x7f77fc10e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58646/0 (socket says 172.21.15.112:58646) 2024-09-16T00:52:28.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:28.022+0000 7f7801125640 1 -- 172.21.15.112:0/3552995226 learned_addr learned my addr 172.21.15.112:0/3552995226 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:52:28.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:28.022+0000 7f7801125640 1 -- 172.21.15.112:0/3552995226 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77fc1014c0 msgr2=0x7f77fc10e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:52:28.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:28.022+0000 7f7801125640 1 --2- 172.21.15.112:0/3552995226 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77fc1014c0 0x7f77fc10e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:28.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:28.022+0000 7f7801125640 1 -- 172.21.15.112:0/3552995226 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77ec005040 con 0x7f77fc0fb480 2024-09-16T00:52:28.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:28.023+0000 7f7801125640 1 --2- 172.21.15.112:0/3552995226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77fc0fb480 0x7f77fc10e170 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f77f800c830 tx=0x7f77f8004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:52:28.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:28.023+0000 7f77f1ffb640 1 -- 172.21.15.112:0/3552995226 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f77f800a850 con 0x7f77fc0fb480 2024-09-16T00:52:28.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:28.023+0000 7f77f1ffb640 1 -- 172.21.15.112:0/3552995226 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f77f800ae50 con 0x7f77fc0fb480 2024-09-16T00:52:28.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:28.023+0000 7f7803389640 1 -- 172.21.15.112:0/3552995226 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77fc1171e0 con 0x7f77fc0fb480 2024-09-16T00:52:28.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:28.023+0000 7f77f1ffb640 1 -- 172.21.15.112:0/3552995226 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f77f80113a0 con 0x7f77fc0fb480 2024-09-16T00:52:28.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:28.023+0000 7f7803389640 1 -- 172.21.15.112:0/3552995226 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77fc117720 con 0x7f77fc0fb480 2024-09-16T00:52:28.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:28.024+0000 7f7803389640 1 -- 172.21.15.112:0/3552995226 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77fc0fcbc0 con 0x7f77fc0fb480 2024-09-16T00:52:28.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:28.024+0000 7f77f1ffb640 1 -- 172.21.15.112:0/3552995226 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f77f8012400 con 0x7f77fc0fb480 2024-09-16T00:52:28.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:28.025+0000 7f77f1ffb640 1 --2- 172.21.15.112:0/3552995226 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f77e806e530 0x7f77e80709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:52:28.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:28.025+0000 7f77f1ffb640 1 -- 172.21.15.112:0/3552995226 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f77f808e5d0 con 0x7f77fc0fb480 2024-09-16T00:52:28.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:28.025+0000 7f7800924640 1 --2- 172.21.15.112:0/3552995226 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f77e806e530 0x7f77e80709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:52:28.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:28.025+0000 7f7800924640 1 --2- 172.21.15.112:0/3552995226 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f77e806e530 0x7f77e80709e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f77ec0043d0 tx=0x7f77ec009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:52:28.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:28.029+0000 7f77f1ffb640 1 -- 172.21.15.112:0/3552995226 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f77f8057520 con 0x7f77fc0fb480 2024-09-16T00:52:28.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:28.211+0000 7f7803389640 1 -- 172.21.15.112:0/3552995226 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f77fc0fac40 con 0x7f77fc0fb480 2024-09-16T00:52:28.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:28.211+0000 7f77f1ffb640 1 -- 172.21.15.112:0/3552995226 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 41 entries v90) v1 ==== 81+0+2462 (secure 0 0 0) 0x7f77f805abe0 con 0x7f77fc0fb480 2024-09-16T00:52:28.213 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6828/486654885 2024-09-17T00:33:29.012500+0000 2024-09-16T00:52:28.213 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.169:6825/2892476953 2024-09-17T00:33:29.012512+0000 2024-09-16T00:52:28.213 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6801/1965658616 2024-09-17T00:29:16.474829+0000 2024-09-16T00:52:28.213 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/1343134839 2024-09-17T00:29:16.474829+0000 2024-09-16T00:52:28.213 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/2146682732 2024-09-17T00:29:16.474829+0000 2024-09-16T00:52:28.213 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.169:6824/2892476953 2024-09-17T00:33:29.012512+0000 2024-09-16T00:52:28.213 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6800/663349889 2024-09-17T00:17:05.974495+0000 2024-09-16T00:52:28.213 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6826/777980670 2024-09-17T00:23:37.740225+0000 2024-09-16T00:52:28.213 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6800/2021251488 2024-09-17T00:18:36.660338+0000 2024-09-16T00:52:28.213 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/1686004085 2024-09-17T00:17:35.477448+0000 2024-09-16T00:52:28.213 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/4017814212 2024-09-17T00:26:21.037829+0000 2024-09-16T00:52:28.214 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/4168804950 2024-09-17T00:17:35.477448+0000 2024-09-16T00:52:28.214 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/882448326 2024-09-17T00:29:16.474829+0000 2024-09-16T00:52:28.214 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.169:6829/1220507355 2024-09-17T00:27:36.939526+0000 2024-09-16T00:52:28.214 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.169:0/2496523892 2024-09-17T00:27:36.939526+0000 2024-09-16T00:52:28.214 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/2126830902 2024-09-17T00:18:36.660338+0000 2024-09-16T00:52:28.214 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6801/2480414866 2024-09-17T00:26:21.037829+0000 2024-09-16T00:52:28.214 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.169:0/2593080060 2024-09-17T00:27:36.939526+0000 2024-09-16T00:52:28.214 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6801/663349889 2024-09-17T00:17:05.974495+0000 2024-09-16T00:52:28.214 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/3430050847 2024-09-17T00:18:36.660338+0000 2024-09-16T00:52:28.214 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6827/777980670 2024-09-17T00:23:37.740225+0000 2024-09-16T00:52:28.214 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/3031370105 2024-09-17T00:26:21.037829+0000 2024-09-16T00:52:28.214 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/2350812481 2024-09-17T00:29:16.474829+0000 2024-09-16T00:52:28.214 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/2081985644 2024-09-17T00:17:05.974495+0000 2024-09-16T00:52:28.215 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.169:0/3537145635 2024-09-17T00:27:36.939526+0000 2024-09-16T00:52:28.215 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/2455516344 2024-09-17T00:26:21.037829+0000 2024-09-16T00:52:28.215 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/3758162203 2024-09-17T00:26:21.037829+0000 2024-09-16T00:52:28.215 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/3817770007 2024-09-17T00:18:36.660338+0000 2024-09-16T00:52:28.215 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/648744088 2024-09-17T00:17:35.477448+0000 2024-09-16T00:52:28.215 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.169:6828/1220507355 2024-09-17T00:27:36.939526+0000 2024-09-16T00:52:28.215 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.169:0/3788010385 2024-09-17T00:27:36.939526+0000 2024-09-16T00:52:28.215 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6800/2480414866 2024-09-17T00:26:21.037829+0000 2024-09-16T00:52:28.215 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.169:0/3513433251 2024-09-17T00:27:36.939526+0000 2024-09-16T00:52:28.215 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/2155316648 2024-09-17T00:17:05.974495+0000 2024-09-16T00:52:28.215 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6800/122493397 2024-09-17T00:17:35.477448+0000 2024-09-16T00:52:28.215 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6829/486654885 2024-09-17T00:33:29.012500+0000 2024-09-16T00:52:28.215 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.169:0/1621361774 2024-09-17T00:27:36.939526+0000 2024-09-16T00:52:28.215 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6801/2021251488 2024-09-17T00:18:36.660338+0000 2024-09-16T00:52:28.216 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6801/122493397 2024-09-17T00:17:35.477448+0000 2024-09-16T00:52:28.216 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6800/1965658616 2024-09-17T00:29:16.474829+0000 2024-09-16T00:52:28.216 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/214318070 2024-09-17T00:17:05.974495+0000 2024-09-16T00:52:28.216 INFO:teuthology.orchestra.run.smithi112.stderr:listed 41 entries 2024-09-16T00:52:28.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:28.213+0000 7f7803389640 1 -- 172.21.15.112:0/3552995226 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f77e806e530 msgr2=0x7f77e80709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:52:28.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:28.213+0000 7f7803389640 1 --2- 172.21.15.112:0/3552995226 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f77e806e530 0x7f77e80709e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f77ec0043d0 tx=0x7f77ec009da0 comp rx=0 tx=0).stop 2024-09-16T00:52:28.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:28.214+0000 7f7803389640 1 -- 172.21.15.112:0/3552995226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77fc0fb480 msgr2=0x7f77fc10e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:52:28.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:28.214+0000 7f7803389640 1 --2- 172.21.15.112:0/3552995226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77fc0fb480 0x7f77fc10e170 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f77f800c830 tx=0x7f77f8004430 comp rx=0 tx=0).stop 2024-09-16T00:52:28.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:28.214+0000 7f7803389640 1 -- 172.21.15.112:0/3552995226 shutdown_connections 2024-09-16T00:52:28.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:28.214+0000 7f7803389640 1 --2- 172.21.15.112:0/3552995226 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f77e806e530 0x7f77e80709e0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:28.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:28.214+0000 7f7803389640 1 --2- 172.21.15.112:0/3552995226 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77fc1014c0 0x7f77fc10e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:28.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:28.214+0000 7f7803389640 1 --2- 172.21.15.112:0/3552995226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77fc0fb480 0x7f77fc10e170 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:28.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:28.214+0000 7f7803389640 1 -- 172.21.15.112:0/3552995226 >> 172.21.15.112:0/3552995226 conn(0x7f77fc0f7000 msgr2=0x7f77fc0f73d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:52:28.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:28.214+0000 7f7803389640 1 -- 172.21.15.112:0/3552995226 shutdown_connections 2024-09-16T00:52:28.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:28.214+0000 7f7803389640 1 -- 172.21.15.112:0/3552995226 wait complete. 2024-09-16T00:52:28.931 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:52:28 smithi112 ceph-mon[97750]: pgmap v743: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:52:28.932 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:52:28 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/3552995226' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T00:52:28.956 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-16T00:52:28.956 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T00:52:28.974 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid d346dd56-73c0-11ef-bceb-c7b262605968 -- ceph osd blocklist ls 2024-09-16T00:52:29.260 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/mon.smithi112/config 2024-09-16T00:52:29.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:52:28 smithi169 ceph-mon[77626]: pgmap v743: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:52:29.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:52:28 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/3552995226' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T00:52:30.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:30.800+0000 7f9004be0640 1 -- 172.21.15.112:0/696697219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90000fe460 msgr2=0x7f90000fe830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:52:30.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:30.800+0000 7f9004be0640 1 --2- 172.21.15.112:0/696697219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90000fe460 0x7f90000fe830 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f8ff0009c50 tx=0x7f8ff0004d10 comp rx=0 tx=0).stop 2024-09-16T00:52:30.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:30.800+0000 7f9004be0640 1 -- 172.21.15.112:0/696697219 shutdown_connections 2024-09-16T00:52:30.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:30.800+0000 7f9004be0640 1 --2- 172.21.15.112:0/696697219 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90000fabe0 0x7f90000fb030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:30.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:30.800+0000 7f9004be0640 1 --2- 172.21.15.112:0/696697219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90000fe460 0x7f90000fe830 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:30.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:30.800+0000 7f9004be0640 1 -- 172.21.15.112:0/696697219 >> 172.21.15.112:0/696697219 conn(0x7f9000071b00 msgr2=0x7f9000071f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:52:30.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:30.800+0000 7f9004be0640 1 -- 172.21.15.112:0/696697219 shutdown_connections 2024-09-16T00:52:30.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:30.800+0000 7f9004be0640 1 -- 172.21.15.112:0/696697219 wait complete. 2024-09-16T00:52:30.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:30.801+0000 7f9004be0640 1 Processor -- start 2024-09-16T00:52:30.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:30.801+0000 7f9004be0640 1 -- start start 2024-09-16T00:52:30.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:30.801+0000 7f9004be0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90000fabe0 0x7f90001120c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:52:30.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:30.801+0000 7f9004be0640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90000fe460 0x7f9000112600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:52:30.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:30.801+0000 7f9004be0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9000112c50 con 0x7f90000fabe0 2024-09-16T00:52:30.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:30.801+0000 7f9004be0640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f900010d1f0 con 0x7f90000fe460 2024-09-16T00:52:30.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:30.801+0000 7f8ffe59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90000fabe0 0x7f90001120c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:52:30.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:30.801+0000 7f8ffe59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90000fabe0 0x7f90001120c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58664/0 (socket says 172.21.15.112:58664) 2024-09-16T00:52:30.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:30.801+0000 7f8ffe59c640 1 -- 172.21.15.112:0/3987559349 learned_addr learned my addr 172.21.15.112:0/3987559349 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-16T00:52:30.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:30.801+0000 7f8ffdd9b640 1 --2- 172.21.15.112:0/3987559349 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90000fe460 0x7f9000112600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:52:30.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:30.801+0000 7f8ffe59c640 1 -- 172.21.15.112:0/3987559349 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90000fe460 msgr2=0x7f9000112600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:52:30.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:30.801+0000 7f8ffe59c640 1 --2- 172.21.15.112:0/3987559349 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90000fe460 0x7f9000112600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:30.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:30.801+0000 7f8ffe59c640 1 -- 172.21.15.112:0/3987559349 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff00098b0 con 0x7f90000fabe0 2024-09-16T00:52:30.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:30.802+0000 7f8ffe59c640 1 --2- 172.21.15.112:0/3987559349 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90000fabe0 0x7f90001120c0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f8ff0004a30 tx=0x7f8ff0004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:52:30.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:30.802+0000 7f8ff6ffd640 1 -- 172.21.15.112:0/3987559349 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ff0009690 con 0x7f90000fabe0 2024-09-16T00:52:30.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:30.802+0000 7f8ff6ffd640 1 -- 172.21.15.112:0/3987559349 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) v1 ==== 1070+0+0 (secure 0 0 0) 0x7f8ff0032b10 con 0x7f90000fabe0 2024-09-16T00:52:30.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:30.802+0000 7f9004be0640 1 -- 172.21.15.112:0/3987559349 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f900010d470 con 0x7f90000fabe0 2024-09-16T00:52:30.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:30.802+0000 7f8ff6ffd640 1 -- 172.21.15.112:0/3987559349 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ff003daa0 con 0x7f90000fabe0 2024-09-16T00:52:30.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:30.802+0000 7f9004be0640 1 -- 172.21.15.112:0/3987559349 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f900010da10 con 0x7f90000fabe0 2024-09-16T00:52:30.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:30.803+0000 7f9004be0640 1 -- 172.21.15.112:0/3987559349 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9000064d50 con 0x7f90000fabe0 2024-09-16T00:52:30.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:30.803+0000 7f8ff6ffd640 1 -- 172.21.15.112:0/3987559349 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 39) v1 ==== 92553+0+0 (secure 0 0 0) 0x7f8ff002f030 con 0x7f90000fabe0 2024-09-16T00:52:30.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:30.804+0000 7f8ff6ffd640 1 --2- 172.21.15.112:0/3987559349 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f8fec06e530 0x7f8fec0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:52:30.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:30.804+0000 7f8ff6ffd640 1 -- 172.21.15.112:0/3987559349 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6872+0+0 (secure 0 0 0) 0x7f8ff00af350 con 0x7f90000fabe0 2024-09-16T00:52:30.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:30.804+0000 7f8ffdd9b640 1 --2- 172.21.15.112:0/3987559349 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f8fec06e530 0x7f8fec0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:52:30.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:30.804+0000 7f8ffdd9b640 1 --2- 172.21.15.112:0/3987559349 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f8fec06e530 0x7f8fec0709e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f90000f7e00 tx=0x7f8fe8009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:52:30.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:30.808+0000 7f8ff6ffd640 1 -- 172.21.15.112:0/3987559349 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f8ff0078340 con 0x7f90000fabe0 2024-09-16T00:52:30.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:30.987+0000 7f9004be0640 1 -- 172.21.15.112:0/3987559349 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f900004b720 con 0x7f90000fabe0 2024-09-16T00:52:30.989 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:52:30 smithi112 ceph-mon[97750]: pgmap v744: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:52:30.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:30.988+0000 7f8ff6ffd640 1 -- 172.21.15.112:0/3987559349 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 41 entries v90) v1 ==== 81+0+2462 (secure 0 0 0) 0x7f8ff007ba00 con 0x7f90000fabe0 2024-09-16T00:52:30.990 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6828/486654885 2024-09-17T00:33:29.012500+0000 2024-09-16T00:52:30.990 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.169:6825/2892476953 2024-09-17T00:33:29.012512+0000 2024-09-16T00:52:30.990 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6801/1965658616 2024-09-17T00:29:16.474829+0000 2024-09-16T00:52:30.990 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/1343134839 2024-09-17T00:29:16.474829+0000 2024-09-16T00:52:30.990 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/2146682732 2024-09-17T00:29:16.474829+0000 2024-09-16T00:52:30.991 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.169:6824/2892476953 2024-09-17T00:33:29.012512+0000 2024-09-16T00:52:30.991 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6800/663349889 2024-09-17T00:17:05.974495+0000 2024-09-16T00:52:30.991 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6826/777980670 2024-09-17T00:23:37.740225+0000 2024-09-16T00:52:30.991 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6800/2021251488 2024-09-17T00:18:36.660338+0000 2024-09-16T00:52:30.991 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/1686004085 2024-09-17T00:17:35.477448+0000 2024-09-16T00:52:30.991 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/4017814212 2024-09-17T00:26:21.037829+0000 2024-09-16T00:52:30.991 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/4168804950 2024-09-17T00:17:35.477448+0000 2024-09-16T00:52:30.991 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/882448326 2024-09-17T00:29:16.474829+0000 2024-09-16T00:52:30.991 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.169:6829/1220507355 2024-09-17T00:27:36.939526+0000 2024-09-16T00:52:30.991 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.169:0/2496523892 2024-09-17T00:27:36.939526+0000 2024-09-16T00:52:30.991 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/2126830902 2024-09-17T00:18:36.660338+0000 2024-09-16T00:52:30.991 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6801/2480414866 2024-09-17T00:26:21.037829+0000 2024-09-16T00:52:30.991 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.169:0/2593080060 2024-09-17T00:27:36.939526+0000 2024-09-16T00:52:30.992 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6801/663349889 2024-09-17T00:17:05.974495+0000 2024-09-16T00:52:30.992 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/3430050847 2024-09-17T00:18:36.660338+0000 2024-09-16T00:52:30.992 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6827/777980670 2024-09-17T00:23:37.740225+0000 2024-09-16T00:52:30.992 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/3031370105 2024-09-17T00:26:21.037829+0000 2024-09-16T00:52:30.992 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/2350812481 2024-09-17T00:29:16.474829+0000 2024-09-16T00:52:30.992 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/2081985644 2024-09-17T00:17:05.974495+0000 2024-09-16T00:52:30.992 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.169:0/3537145635 2024-09-17T00:27:36.939526+0000 2024-09-16T00:52:30.992 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/2455516344 2024-09-17T00:26:21.037829+0000 2024-09-16T00:52:30.992 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/3758162203 2024-09-17T00:26:21.037829+0000 2024-09-16T00:52:30.992 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/3817770007 2024-09-17T00:18:36.660338+0000 2024-09-16T00:52:30.992 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/648744088 2024-09-17T00:17:35.477448+0000 2024-09-16T00:52:30.992 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.169:6828/1220507355 2024-09-17T00:27:36.939526+0000 2024-09-16T00:52:30.992 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.169:0/3788010385 2024-09-17T00:27:36.939526+0000 2024-09-16T00:52:30.992 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6800/2480414866 2024-09-17T00:26:21.037829+0000 2024-09-16T00:52:30.993 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.169:0/3513433251 2024-09-17T00:27:36.939526+0000 2024-09-16T00:52:30.993 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/2155316648 2024-09-17T00:17:05.974495+0000 2024-09-16T00:52:30.993 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6800/122493397 2024-09-17T00:17:35.477448+0000 2024-09-16T00:52:30.993 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6829/486654885 2024-09-17T00:33:29.012500+0000 2024-09-16T00:52:30.993 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.169:0/1621361774 2024-09-17T00:27:36.939526+0000 2024-09-16T00:52:30.993 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6801/2021251488 2024-09-17T00:18:36.660338+0000 2024-09-16T00:52:30.993 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6801/122493397 2024-09-17T00:17:35.477448+0000 2024-09-16T00:52:30.993 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6800/1965658616 2024-09-17T00:29:16.474829+0000 2024-09-16T00:52:30.993 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/214318070 2024-09-17T00:17:05.974495+0000 2024-09-16T00:52:30.994 INFO:teuthology.orchestra.run.smithi112.stderr:listed 41 entries 2024-09-16T00:52:30.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:30.990+0000 7f9004be0640 1 -- 172.21.15.112:0/3987559349 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f8fec06e530 msgr2=0x7f8fec0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:52:30.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:30.990+0000 7f9004be0640 1 --2- 172.21.15.112:0/3987559349 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f8fec06e530 0x7f8fec0709e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f90000f7e00 tx=0x7f8fe8009040 comp rx=0 tx=0).stop 2024-09-16T00:52:30.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:30.990+0000 7f9004be0640 1 -- 172.21.15.112:0/3987559349 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90000fabe0 msgr2=0x7f90001120c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:52:30.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:30.990+0000 7f9004be0640 1 --2- 172.21.15.112:0/3987559349 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90000fabe0 0x7f90001120c0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f8ff0004a30 tx=0x7f8ff0004d10 comp rx=0 tx=0).stop 2024-09-16T00:52:30.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:30.990+0000 7f9004be0640 1 -- 172.21.15.112:0/3987559349 shutdown_connections 2024-09-16T00:52:30.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:30.991+0000 7f9004be0640 1 --2- 172.21.15.112:0/3987559349 >> [v2:172.21.15.112:6800/1498212142,v1:172.21.15.112:6801/1498212142] conn(0x7f8fec06e530 0x7f8fec0709e0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:30.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:30.991+0000 7f9004be0640 1 --2- 172.21.15.112:0/3987559349 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90000fe460 0x7f9000112600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:30.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:30.991+0000 7f9004be0640 1 --2- 172.21.15.112:0/3987559349 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90000fabe0 0x7f90001120c0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:52:30.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:30.991+0000 7f9004be0640 1 -- 172.21.15.112:0/3987559349 >> 172.21.15.112:0/3987559349 conn(0x7f9000071b00 msgr2=0x7f90000f9a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:52:30.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:30.991+0000 7f9004be0640 1 -- 172.21.15.112:0/3987559349 shutdown_connections 2024-09-16T00:52:30.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-16T00:52:30.991+0000 7f9004be0640 1 -- 172.21.15.112:0/3987559349 wait complete. 2024-09-16T00:52:31.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:52:30 smithi169 ceph-mon[77626]: pgmap v744: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:52:31.628 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi112.front.sepia.ceph.com... 2024-09-16T00:52:31.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:52:31.628 DEBUG:teuthology.orchestra.run.smithi112:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-16T00:52:31.656 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T00:52:32.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:52:31 smithi169 ceph-mon[77626]: from='client.? 172.21.15.112:0/3987559349' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T00:52:32.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:52:31 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:52:32.316 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:52:31 smithi112 ceph-mon[97750]: from='client.? 172.21.15.112:0/3987559349' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T00:52:32.316 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:52:31 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:52:33.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:52:32 smithi169 ceph-mon[77626]: pgmap v745: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:52:33.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:52:32 smithi112 ceph-mon[97750]: pgmap v745: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:52:35.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:52:34 smithi169 ceph-mon[77626]: pgmap v746: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:52:35.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:52:34 smithi112 ceph-mon[97750]: pgmap v746: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:52:37.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:52:36 smithi169 ceph-mon[77626]: pgmap v747: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:52:37.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:52:36 smithi112 ceph-mon[97750]: pgmap v747: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:52:39.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:52:38 smithi169 ceph-mon[77626]: pgmap v748: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:52:39.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:52:38 smithi112 ceph-mon[97750]: pgmap v748: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:52:40.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:52:39 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:52:40.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:52:39 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:52:40.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:52:39 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:52:40.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:52:39 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:52:40.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:52:39 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:52:40.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:52:39 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:52:40.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:52:39 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:52:40.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:52:39 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:52:41.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:52:40 smithi169 ceph-mon[77626]: pgmap v749: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:52:41.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:52:40 smithi112 ceph-mon[97750]: pgmap v749: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:52:43.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:52:42 smithi169 ceph-mon[77626]: pgmap v750: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:52:43.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:52:42 smithi112 ceph-mon[97750]: pgmap v750: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:52:45.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:52:44 smithi169 ceph-mon[77626]: pgmap v751: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:52:45.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:52:44 smithi112 ceph-mon[97750]: pgmap v751: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:52:46.777 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:52:46 smithi169 ceph-mon[77626]: pgmap v752: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:52:46.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:52:46 smithi112 ceph-mon[97750]: pgmap v752: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:52:47.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:52:47 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:52:47.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:52:47 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:52:48.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:52:48 smithi169 ceph-mon[77626]: pgmap v753: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:52:48.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:52:48 smithi112 ceph-mon[97750]: pgmap v753: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:52:51.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:52:50 smithi112 ceph-mon[97750]: pgmap v754: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:52:51.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:52:50 smithi169 ceph-mon[77626]: pgmap v754: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:52:53.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:52:52 smithi112 ceph-mon[97750]: pgmap v755: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:52:53.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:52:52 smithi169 ceph-mon[77626]: pgmap v755: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:52:55.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:52:54 smithi112 ceph-mon[97750]: pgmap v756: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:52:55.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:52:54 smithi169 ceph-mon[77626]: pgmap v756: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:52:57.078 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:52:56 smithi169 ceph-mon[77626]: pgmap v757: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:52:57.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:52:56 smithi112 ceph-mon[97750]: pgmap v757: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:52:59.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:52:58 smithi112 ceph-mon[97750]: pgmap v758: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:52:59.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:52:58 smithi169 ceph-mon[77626]: pgmap v758: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:53:01.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:53:00 smithi112 ceph-mon[97750]: pgmap v759: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:53:01.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:53:00 smithi169 ceph-mon[77626]: pgmap v759: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:53:02.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:53:01 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:53:02.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:53:01 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:53:03.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:53:02 smithi112 ceph-mon[97750]: pgmap v760: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:53:03.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:53:02 smithi169 ceph-mon[77626]: pgmap v760: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:53:05.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:53:04 smithi112 ceph-mon[97750]: pgmap v761: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:53:05.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:53:04 smithi169 ceph-mon[77626]: pgmap v761: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:53:07.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:53:06 smithi112 ceph-mon[97750]: pgmap v762: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:53:07.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:53:06 smithi169 ceph-mon[77626]: pgmap v762: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:53:09.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:53:08 smithi112 ceph-mon[97750]: pgmap v763: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:53:09.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:53:08 smithi169 ceph-mon[77626]: pgmap v763: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:53:11.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:53:10 smithi112 ceph-mon[97750]: pgmap v764: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:53:11.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:53:10 smithi169 ceph-mon[77626]: pgmap v764: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:53:13.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:53:12 smithi112 ceph-mon[97750]: pgmap v765: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:53:13.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:53:12 smithi169 ceph-mon[77626]: pgmap v765: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:53:15.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:53:14 smithi112 ceph-mon[97750]: pgmap v766: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:53:15.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:53:14 smithi169 ceph-mon[77626]: pgmap v766: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:53:17.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:53:16 smithi112 ceph-mon[97750]: pgmap v767: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:53:17.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:53:16 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:53:17.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:53:16 smithi169 ceph-mon[77626]: pgmap v767: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:53:17.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:53:16 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:53:19.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:53:18 smithi112 ceph-mon[97750]: pgmap v768: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:53:19.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:53:18 smithi169 ceph-mon[77626]: pgmap v768: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:53:21.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:53:20 smithi112 ceph-mon[97750]: pgmap v769: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:53:21.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:53:20 smithi169 ceph-mon[77626]: pgmap v769: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:53:23.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:53:22 smithi112 ceph-mon[97750]: pgmap v770: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:53:23.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:53:22 smithi169 ceph-mon[77626]: pgmap v770: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:53:25.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:53:24 smithi112 ceph-mon[97750]: pgmap v771: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:53:25.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:53:24 smithi169 ceph-mon[77626]: pgmap v771: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:53:27.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:53:26 smithi112 ceph-mon[97750]: pgmap v772: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:53:27.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:53:26 smithi169 ceph-mon[77626]: pgmap v772: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:53:29.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:53:28 smithi169 ceph-mon[77626]: pgmap v773: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:53:29.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:53:28 smithi112 ceph-mon[97750]: pgmap v773: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:53:31.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:53:30 smithi169 ceph-mon[77626]: pgmap v774: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:53:31.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:53:30 smithi112 ceph-mon[97750]: pgmap v774: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:53:32.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:53:31 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:53:32.314 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:53:31 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:53:33.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:53:32 smithi169 ceph-mon[77626]: pgmap v775: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:53:33.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:53:32 smithi112 ceph-mon[97750]: pgmap v775: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:53:35.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:53:34 smithi169 ceph-mon[77626]: pgmap v776: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:53:35.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:53:34 smithi112 ceph-mon[97750]: pgmap v776: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:53:37.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:53:36 smithi169 ceph-mon[77626]: pgmap v777: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:53:37.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:53:36 smithi112 ceph-mon[97750]: pgmap v777: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:53:39.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:53:38 smithi169 ceph-mon[77626]: pgmap v778: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:53:39.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:53:38 smithi112 ceph-mon[97750]: pgmap v778: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:53:40.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:53:39 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:53:40.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:53:39 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:53:41.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:53:40 smithi169 ceph-mon[77626]: pgmap v779: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:53:41.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:53:40 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:53:41.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:53:40 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:53:41.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:53:40 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:53:41.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:53:40 smithi112 ceph-mon[97750]: pgmap v779: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:53:41.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:53:40 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:53:41.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:53:40 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:53:41.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:53:40 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:53:43.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:53:42 smithi169 ceph-mon[77626]: pgmap v780: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:53:43.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:53:42 smithi112 ceph-mon[97750]: pgmap v780: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:53:45.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:53:44 smithi169 ceph-mon[77626]: pgmap v781: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:53:45.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:53:44 smithi112 ceph-mon[97750]: pgmap v781: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:53:47.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:53:46 smithi169 ceph-mon[77626]: pgmap v782: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:53:47.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:53:46 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:53:47.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:53:46 smithi112 ceph-mon[97750]: pgmap v782: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:53:47.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:53:46 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:53:49.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:53:48 smithi169 ceph-mon[77626]: pgmap v783: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:53:49.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:53:48 smithi112 ceph-mon[97750]: pgmap v783: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:53:51.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:53:50 smithi169 ceph-mon[77626]: pgmap v784: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:53:51.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:53:50 smithi112 ceph-mon[97750]: pgmap v784: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:53:53.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:53:52 smithi169 ceph-mon[77626]: pgmap v785: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:53:53.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:53:52 smithi112 ceph-mon[97750]: pgmap v785: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:53:55.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:53:54 smithi169 ceph-mon[77626]: pgmap v786: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:53:55.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:53:54 smithi112 ceph-mon[97750]: pgmap v786: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:53:57.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:53:56 smithi169 ceph-mon[77626]: pgmap v787: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:53:57.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:53:56 smithi112 ceph-mon[97750]: pgmap v787: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:53:59.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:53:58 smithi169 ceph-mon[77626]: pgmap v788: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:53:59.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:53:58 smithi112 ceph-mon[97750]: pgmap v788: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:54:01.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:54:00 smithi169 ceph-mon[77626]: pgmap v789: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:54:01.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:54:00 smithi112 ceph-mon[97750]: pgmap v789: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:54:02.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:54:01 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:54:02.314 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:54:01 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:54:03.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:54:02 smithi169 ceph-mon[77626]: pgmap v790: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:54:03.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:54:02 smithi112 ceph-mon[97750]: pgmap v790: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:54:05.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:54:04 smithi169 ceph-mon[77626]: pgmap v791: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:54:05.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:54:04 smithi112 ceph-mon[97750]: pgmap v791: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:54:07.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:54:06 smithi169 ceph-mon[77626]: pgmap v792: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:54:07.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:54:06 smithi112 ceph-mon[97750]: pgmap v792: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:54:09.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:54:08 smithi169 ceph-mon[77626]: pgmap v793: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:54:09.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:54:08 smithi112 ceph-mon[97750]: pgmap v793: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:54:11.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:54:10 smithi169 ceph-mon[77626]: pgmap v794: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:54:11.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:54:10 smithi112 ceph-mon[97750]: pgmap v794: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:54:13.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:54:12 smithi169 ceph-mon[77626]: pgmap v795: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:54:13.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:54:12 smithi112 ceph-mon[97750]: pgmap v795: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:54:15.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:54:14 smithi169 ceph-mon[77626]: pgmap v796: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:54:15.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:54:14 smithi112 ceph-mon[97750]: pgmap v796: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:54:17.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:54:16 smithi169 ceph-mon[77626]: pgmap v797: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:54:17.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:54:16 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:54:17.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:54:16 smithi112 ceph-mon[97750]: pgmap v797: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:54:17.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:54:16 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:54:19.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:54:18 smithi169 ceph-mon[77626]: pgmap v798: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:54:19.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:54:18 smithi112 ceph-mon[97750]: pgmap v798: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:54:21.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:54:20 smithi169 ceph-mon[77626]: pgmap v799: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:54:21.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:54:20 smithi112 ceph-mon[97750]: pgmap v799: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:54:23.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:54:22 smithi169 ceph-mon[77626]: pgmap v800: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:54:23.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:54:22 smithi112 ceph-mon[97750]: pgmap v800: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:54:25.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:54:24 smithi169 ceph-mon[77626]: pgmap v801: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:54:25.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:54:24 smithi112 ceph-mon[97750]: pgmap v801: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:54:27.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:54:26 smithi169 ceph-mon[77626]: pgmap v802: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:54:27.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:54:26 smithi112 ceph-mon[97750]: pgmap v802: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:54:29.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:54:28 smithi169 ceph-mon[77626]: pgmap v803: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:54:29.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:54:28 smithi112 ceph-mon[97750]: pgmap v803: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:54:31.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:54:30 smithi169 ceph-mon[77626]: pgmap v804: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:54:31.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:54:30 smithi112 ceph-mon[97750]: pgmap v804: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:54:32.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:54:31 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:54:32.315 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:54:31 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:54:33.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:54:32 smithi169 ceph-mon[77626]: pgmap v805: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:54:33.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:54:32 smithi112 ceph-mon[97750]: pgmap v805: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:54:35.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:54:34 smithi169 ceph-mon[77626]: pgmap v806: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:54:35.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:54:34 smithi112 ceph-mon[97750]: pgmap v806: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:54:37.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:54:36 smithi169 ceph-mon[77626]: pgmap v807: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:54:37.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:54:36 smithi112 ceph-mon[97750]: pgmap v807: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:54:39.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:54:38 smithi169 ceph-mon[77626]: pgmap v808: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:54:39.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:54:38 smithi112 ceph-mon[97750]: pgmap v808: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:54:41.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:54:40 smithi169 ceph-mon[77626]: pgmap v809: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:54:41.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:54:40 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:54:41.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:54:40 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:54:41.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:54:40 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:54:41.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:54:40 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:54:41.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:54:40 smithi112 ceph-mon[97750]: pgmap v809: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:54:41.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:54:40 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:54:41.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:54:40 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:54:41.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:54:40 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:54:41.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:54:40 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:54:43.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:54:42 smithi169 ceph-mon[77626]: pgmap v810: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:54:43.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:54:42 smithi112 ceph-mon[97750]: pgmap v810: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:54:45.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:54:44 smithi169 ceph-mon[77626]: pgmap v811: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:54:45.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:54:44 smithi112 ceph-mon[97750]: pgmap v811: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:54:47.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:54:46 smithi169 ceph-mon[77626]: pgmap v812: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:54:47.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:54:46 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:54:47.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:54:46 smithi112 ceph-mon[97750]: pgmap v812: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:54:47.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:54:46 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:54:48.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:54:48 smithi112 ceph-mon[97750]: pgmap v813: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:54:48.526 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:54:48 smithi169 ceph-mon[77626]: pgmap v813: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:54:51.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:54:50 smithi112 ceph-mon[97750]: pgmap v814: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:54:51.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:54:50 smithi169 ceph-mon[77626]: pgmap v814: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:54:53.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:54:52 smithi112 ceph-mon[97750]: pgmap v815: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:54:53.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:54:52 smithi169 ceph-mon[77626]: pgmap v815: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:54:55.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:54:54 smithi112 ceph-mon[97750]: pgmap v816: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:54:55.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:54:54 smithi169 ceph-mon[77626]: pgmap v816: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:54:57.079 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:54:56 smithi169 ceph-mon[77626]: pgmap v817: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:54:57.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:54:56 smithi112 ceph-mon[97750]: pgmap v817: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:54:59.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:54:58 smithi112 ceph-mon[97750]: pgmap v818: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:54:59.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:54:58 smithi169 ceph-mon[77626]: pgmap v818: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:55:01.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:55:00 smithi112 ceph-mon[97750]: pgmap v819: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:55:01.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:55:00 smithi169 ceph-mon[77626]: pgmap v819: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:55:02.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:55:01 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:55:02.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:55:01 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:55:03.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:55:02 smithi112 ceph-mon[97750]: pgmap v820: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:55:03.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:55:02 smithi169 ceph-mon[77626]: pgmap v820: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:55:05.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:55:04 smithi112 ceph-mon[97750]: pgmap v821: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:55:05.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:55:04 smithi169 ceph-mon[77626]: pgmap v821: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:55:07.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:55:06 smithi112 ceph-mon[97750]: pgmap v822: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:55:07.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:55:06 smithi169 ceph-mon[77626]: pgmap v822: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:55:09.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:55:08 smithi169 ceph-mon[77626]: pgmap v823: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:55:09.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:55:08 smithi112 ceph-mon[97750]: pgmap v823: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:55:11.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:55:10 smithi169 ceph-mon[77626]: pgmap v824: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:55:11.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:55:10 smithi112 ceph-mon[97750]: pgmap v824: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:55:13.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:55:12 smithi169 ceph-mon[77626]: pgmap v825: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:55:13.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:55:12 smithi112 ceph-mon[97750]: pgmap v825: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:55:15.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:55:14 smithi169 ceph-mon[77626]: pgmap v826: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:55:15.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:55:14 smithi112 ceph-mon[97750]: pgmap v826: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:55:17.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:55:16 smithi169 ceph-mon[77626]: pgmap v827: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:55:17.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:55:16 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:55:17.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:55:16 smithi112 ceph-mon[97750]: pgmap v827: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:55:17.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:55:16 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:55:19.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:55:18 smithi169 ceph-mon[77626]: pgmap v828: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:55:19.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:55:18 smithi112 ceph-mon[97750]: pgmap v828: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:55:21.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:55:20 smithi169 ceph-mon[77626]: pgmap v829: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:55:21.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:55:20 smithi112 ceph-mon[97750]: pgmap v829: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:55:23.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:55:22 smithi169 ceph-mon[77626]: pgmap v830: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:55:23.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:55:22 smithi112 ceph-mon[97750]: pgmap v830: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:55:25.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:55:24 smithi169 ceph-mon[77626]: pgmap v831: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:55:25.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:55:24 smithi112 ceph-mon[97750]: pgmap v831: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:55:27.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:55:26 smithi169 ceph-mon[77626]: pgmap v832: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:55:27.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:55:26 smithi112 ceph-mon[97750]: pgmap v832: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:55:29.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:55:28 smithi169 ceph-mon[77626]: pgmap v833: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:55:29.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:55:28 smithi112 ceph-mon[97750]: pgmap v833: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:55:31.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:55:30 smithi169 ceph-mon[77626]: pgmap v834: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:55:31.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:55:30 smithi112 ceph-mon[97750]: pgmap v834: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:55:32.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:55:31 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:55:32.314 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:55:31 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:55:33.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:55:32 smithi169 ceph-mon[77626]: pgmap v835: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:55:33.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:55:32 smithi112 ceph-mon[97750]: pgmap v835: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:55:35.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:55:34 smithi169 ceph-mon[77626]: pgmap v836: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:55:35.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:55:34 smithi112 ceph-mon[97750]: pgmap v836: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:55:37.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:55:36 smithi169 ceph-mon[77626]: pgmap v837: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:55:37.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:55:36 smithi112 ceph-mon[97750]: pgmap v837: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:55:39.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:55:38 smithi169 ceph-mon[77626]: pgmap v838: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:55:39.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:55:38 smithi112 ceph-mon[97750]: pgmap v838: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:55:41.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:55:40 smithi169 ceph-mon[77626]: pgmap v839: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:55:41.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:55:40 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:55:41.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:55:40 smithi112 ceph-mon[97750]: pgmap v839: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:55:41.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:55:40 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:55:42.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:55:41 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:55:42.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:55:41 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:55:42.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:55:41 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:55:42.315 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:55:41 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:55:42.315 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:55:41 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:55:42.315 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:55:41 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:55:43.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:55:42 smithi169 ceph-mon[77626]: pgmap v840: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:55:43.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:55:42 smithi112 ceph-mon[97750]: pgmap v840: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:55:45.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:55:44 smithi169 ceph-mon[77626]: pgmap v841: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:55:45.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:55:44 smithi112 ceph-mon[97750]: pgmap v841: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:55:47.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:55:46 smithi169 ceph-mon[77626]: pgmap v842: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:55:47.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:55:46 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:55:47.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:55:46 smithi112 ceph-mon[97750]: pgmap v842: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:55:47.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:55:46 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:55:49.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:55:48 smithi169 ceph-mon[77626]: pgmap v843: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:55:49.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:55:48 smithi112 ceph-mon[97750]: pgmap v843: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:55:51.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:55:50 smithi169 ceph-mon[77626]: pgmap v844: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:55:51.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:55:50 smithi112 ceph-mon[97750]: pgmap v844: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:55:53.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:55:52 smithi169 ceph-mon[77626]: pgmap v845: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:55:53.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:55:52 smithi112 ceph-mon[97750]: pgmap v845: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:55:55.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:55:54 smithi169 ceph-mon[77626]: pgmap v846: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:55:55.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:55:54 smithi112 ceph-mon[97750]: pgmap v846: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:55:57.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:55:56 smithi169 ceph-mon[77626]: pgmap v847: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:55:57.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:55:56 smithi112 ceph-mon[97750]: pgmap v847: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:55:59.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:55:58 smithi169 ceph-mon[77626]: pgmap v848: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:55:59.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:55:58 smithi112 ceph-mon[97750]: pgmap v848: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:56:01.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:56:00 smithi169 ceph-mon[77626]: pgmap v849: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:56:01.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:56:00 smithi112 ceph-mon[97750]: pgmap v849: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:56:02.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:56:01 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:56:02.314 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:56:01 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:56:03.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:56:02 smithi169 ceph-mon[77626]: pgmap v850: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:56:03.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:56:02 smithi112 ceph-mon[97750]: pgmap v850: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:56:05.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:56:04 smithi169 ceph-mon[77626]: pgmap v851: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:56:05.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:56:04 smithi112 ceph-mon[97750]: pgmap v851: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:56:07.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:56:06 smithi169 ceph-mon[77626]: pgmap v852: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:56:07.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:56:06 smithi112 ceph-mon[97750]: pgmap v852: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:56:09.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:56:08 smithi169 ceph-mon[77626]: pgmap v853: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:56:09.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:56:08 smithi112 ceph-mon[97750]: pgmap v853: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:56:11.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:56:10 smithi169 ceph-mon[77626]: pgmap v854: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:56:11.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:56:10 smithi112 ceph-mon[97750]: pgmap v854: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:56:13.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:56:12 smithi169 ceph-mon[77626]: pgmap v855: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:56:13.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:56:12 smithi112 ceph-mon[97750]: pgmap v855: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:56:15.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:56:14 smithi169 ceph-mon[77626]: pgmap v856: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:56:15.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:56:14 smithi112 ceph-mon[97750]: pgmap v856: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:56:17.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:56:16 smithi169 ceph-mon[77626]: pgmap v857: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:56:17.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:56:16 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:56:17.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:56:16 smithi112 ceph-mon[97750]: pgmap v857: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:56:17.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:56:16 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:56:19.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:56:18 smithi169 ceph-mon[77626]: pgmap v858: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:56:19.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:56:18 smithi112 ceph-mon[97750]: pgmap v858: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:56:21.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:56:20 smithi169 ceph-mon[77626]: pgmap v859: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:56:21.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:56:20 smithi112 ceph-mon[97750]: pgmap v859: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:56:23.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:56:22 smithi169 ceph-mon[77626]: pgmap v860: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:56:23.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:56:22 smithi112 ceph-mon[97750]: pgmap v860: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:56:25.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:56:24 smithi169 ceph-mon[77626]: pgmap v861: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:56:25.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:56:24 smithi112 ceph-mon[97750]: pgmap v861: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:56:27.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:56:26 smithi169 ceph-mon[77626]: pgmap v862: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:56:27.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:56:26 smithi112 ceph-mon[97750]: pgmap v862: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:56:29.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:56:28 smithi169 ceph-mon[77626]: pgmap v863: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:56:29.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:56:28 smithi112 ceph-mon[97750]: pgmap v863: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:56:31.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:56:30 smithi169 ceph-mon[77626]: pgmap v864: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:56:31.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:56:30 smithi112 ceph-mon[97750]: pgmap v864: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:56:32.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:56:31 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:56:32.315 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:56:31 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:56:33.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:56:32 smithi169 ceph-mon[77626]: pgmap v865: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:56:33.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:56:32 smithi112 ceph-mon[97750]: pgmap v865: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:56:35.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:56:34 smithi169 ceph-mon[77626]: pgmap v866: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:56:35.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:56:34 smithi112 ceph-mon[97750]: pgmap v866: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:56:36.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:56:36 smithi112 ceph-mon[97750]: pgmap v867: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:56:37.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:56:36 smithi169 ceph-mon[77626]: pgmap v867: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:56:39.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:56:38 smithi112 ceph-mon[97750]: pgmap v868: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:56:39.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:56:38 smithi169 ceph-mon[77626]: pgmap v868: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:56:41.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:56:40 smithi169 ceph-mon[77626]: pgmap v869: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:56:41.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:56:40 smithi112 ceph-mon[97750]: pgmap v869: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:56:42.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:56:41 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:56:42.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:56:41 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:56:42.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:56:41 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:56:42.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:56:41 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:56:42.315 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:56:41 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:56:42.315 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:56:41 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:56:42.315 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:56:41 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:56:42.315 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:56:41 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' 2024-09-16T00:56:43.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:56:42 smithi169 ceph-mon[77626]: pgmap v870: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:56:43.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:56:42 smithi112 ceph-mon[97750]: pgmap v870: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:56:45.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:56:44 smithi169 ceph-mon[77626]: pgmap v871: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:56:45.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:56:44 smithi112 ceph-mon[97750]: pgmap v871: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:56:46.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:56:46 smithi112 ceph-mon[97750]: pgmap v872: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:56:47.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:56:46 smithi169 ceph-mon[77626]: pgmap v872: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:56:47.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:56:47 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:56:48.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:56:47 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:56:48.839 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:56:48 smithi112 ceph-mon[97750]: pgmap v873: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:56:49.026 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:56:48 smithi169 ceph-mon[77626]: pgmap v873: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:56:51.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:56:50 smithi169 ceph-mon[77626]: pgmap v874: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:56:51.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:56:50 smithi112 ceph-mon[97750]: pgmap v874: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:56:53.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:56:52 smithi169 ceph-mon[77626]: pgmap v875: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:56:53.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:56:52 smithi112 ceph-mon[97750]: pgmap v875: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:56:55.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:56:54 smithi169 ceph-mon[77626]: pgmap v876: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:56:55.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:56:54 smithi112 ceph-mon[97750]: pgmap v876: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:56:57.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:56:56 smithi169 ceph-mon[77626]: pgmap v877: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:56:57.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:56:56 smithi112 ceph-mon[97750]: pgmap v877: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:56:59.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:56:58 smithi169 ceph-mon[77626]: pgmap v878: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:56:59.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:56:58 smithi112 ceph-mon[97750]: pgmap v878: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:57:01.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:57:00 smithi169 ceph-mon[77626]: pgmap v879: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:57:01.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:57:00 smithi112 ceph-mon[97750]: pgmap v879: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:57:02.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:57:01 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:57:02.315 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:57:01 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:57:03.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:57:02 smithi169 ceph-mon[77626]: pgmap v880: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:57:03.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:57:02 smithi112 ceph-mon[97750]: pgmap v880: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:57:05.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:57:04 smithi169 ceph-mon[77626]: pgmap v881: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:57:05.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:57:04 smithi112 ceph-mon[97750]: pgmap v881: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:57:07.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:57:06 smithi169 ceph-mon[77626]: pgmap v882: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:57:07.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:57:06 smithi112 ceph-mon[97750]: pgmap v882: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:57:09.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:57:08 smithi169 ceph-mon[77626]: pgmap v883: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:57:09.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:57:08 smithi112 ceph-mon[97750]: pgmap v883: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:57:11.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:57:10 smithi169 ceph-mon[77626]: pgmap v884: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:57:11.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:57:10 smithi112 ceph-mon[97750]: pgmap v884: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:57:13.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:57:12 smithi169 ceph-mon[77626]: pgmap v885: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:57:13.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:57:12 smithi112 ceph-mon[97750]: pgmap v885: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:57:15.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:57:14 smithi169 ceph-mon[77626]: pgmap v886: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:57:15.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:57:14 smithi112 ceph-mon[97750]: pgmap v886: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:57:17.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:57:16 smithi169 ceph-mon[77626]: pgmap v887: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:57:17.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:57:16 smithi169 ceph-mon[77626]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:57:17.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:57:16 smithi112 ceph-mon[97750]: pgmap v887: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:57:17.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:57:16 smithi112 ceph-mon[97750]: from='mgr.34104 172.21.15.112:0/3019967566' entity='mgr.smithi112.xrkkbc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T00:57:19.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:57:18 smithi169 ceph-mon[77626]: pgmap v888: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:57:19.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:57:18 smithi112 ceph-mon[97750]: pgmap v888: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:57:21.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:57:20 smithi169 ceph-mon[77626]: pgmap v889: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:57:21.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:57:20 smithi112 ceph-mon[97750]: pgmap v889: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:57:23.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:57:22 smithi169 ceph-mon[77626]: pgmap v890: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:57:23.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:57:22 smithi112 ceph-mon[97750]: pgmap v890: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:57:25.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:57:24 smithi169 ceph-mon[77626]: pgmap v891: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:57:25.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:57:24 smithi112 ceph-mon[97750]: pgmap v891: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:57:27.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:57:26 smithi169 ceph-mon[77626]: pgmap v892: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:57:27.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:57:26 smithi112 ceph-mon[97750]: pgmap v892: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-16T00:57:29.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:57:28 smithi169 ceph-mon[77626]: pgmap v893: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:57:29.339 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:57:28 smithi112 ceph-mon[97750]: pgmap v893: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-16T00:57:30.729 ERROR:tasks.cephfs.fuse_mount:process failed to terminate after unmount. This probably indicates a bug within ceph-fuse. 2024-09-16T00:57:30.729 ERROR:teuthology.run_tasks:Manager failed: ceph-fuse Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa/tasks/ceph_fuse.py", line 248, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (50) after waiting for 300 seconds 2024-09-16T00:57:30.731 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-16T00:57:30.742 INFO:tasks.cephadm:Teardown begin 2024-09-16T00:57:30.742 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa/tasks/cephadm.py", line 2236, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa/tasks/ceph_fuse.py", line 248, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (50) after waiting for 300 seconds 2024-09-16T00:57:30.744 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T00:57:30.778 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T00:57:30.811 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-16T00:57:30.812 DEBUG:teuthology.orchestra.run.smithi112:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-16T00:57:30.835 DEBUG:teuthology.orchestra.run.smithi169:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-16T00:57:30.869 INFO:tasks.cephadm:Stopping all daemons... 2024-09-16T00:57:30.869 INFO:tasks.cephadm.mon.smithi112:Stopping mon.smithi112... 2024-09-16T00:57:30.869 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-d346dd56-73c0-11ef-bceb-c7b262605968@mon.smithi112 2024-09-16T00:57:31.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:57:30 smithi112 ceph-mon[97750]: pgmap v894: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:57:31.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:57:30 smithi112 systemd[1]: Stopping Ceph mon.smithi112 for d346dd56-73c0-11ef-bceb-c7b262605968... 2024-09-16T00:57:31.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:57:30 smithi169 ceph-mon[77626]: pgmap v894: 65 pgs: 65 active+clean; 325 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:57:31.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:57:31 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-mon-smithi112[97746]: 2024-09-16T00:57:31.176+0000 7fcd01112640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi112 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:57:31.589 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:57:31 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-mon-smithi112[97746]: 2024-09-16T00:57:31.176+0000 7fcd01112640 -1 mon.smithi112@0(leader) e3 *** Got Signal Terminated *** 2024-09-16T00:57:31.590 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:57:31 smithi112 podman[146306]: 2024-09-16 00:57:31.391029845 +0000 UTC m=+0.311169643 container died 5aaec033c042c58ff4bf136853bbb4200f4a68bd6285561dcbfd6effed675cef (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-mon-smithi112, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:57:32.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:57:31 smithi112 podman[146306]: 2024-09-16 00:57:31.852007659 +0000 UTC m=+0.772147493 container cleanup 5aaec033c042c58ff4bf136853bbb4200f4a68bd6285561dcbfd6effed675cef (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-mon-smithi112, RELEASE=squid-8921c8e, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux ) 2024-09-16T00:57:32.089 INFO:journalctl@ceph.mon.smithi112.smithi112.stdout:Sep 16 00:57:31 smithi112 bash[146306]: ceph-d346dd56-73c0-11ef-bceb-c7b262605968-mon-smithi112 2024-09-16T00:57:32.323 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d346dd56-73c0-11ef-bceb-c7b262605968@mon.smithi112.service' 2024-09-16T00:57:32.370 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:57:32.370 INFO:tasks.cephadm.mon.smithi112:Stopped mon.smithi112 2024-09-16T00:57:32.370 INFO:tasks.cephadm.mon.smithi169:Stopping mon.smithi169... 2024-09-16T00:57:32.370 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-d346dd56-73c0-11ef-bceb-c7b262605968@mon.smithi169 2024-09-16T00:57:32.658 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:57:32 smithi169 systemd[1]: Stopping Ceph mon.smithi169 for d346dd56-73c0-11ef-bceb-c7b262605968... 2024-09-16T00:57:32.950 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:57:32 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-mon-smithi169[77622]: 2024-09-16T00:57:32.657+0000 7fd9608e4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi169 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:57:32.950 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:57:32 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-mon-smithi169[77622]: 2024-09-16T00:57:32.657+0000 7fd9608e4640 -1 mon.smithi169@1(peon) e3 *** Got Signal Terminated *** 2024-09-16T00:57:32.950 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:57:32 smithi169 podman[110010]: 2024-09-16 00:57:32.667986994 +0000 UTC m=+0.106150605 container died 7bc609155f3f4d9e90ff4a79ab736e132c55ea2fa375c91ac74905745e94971c (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-mon-smithi169, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, ceph=True, io.buildah.version=1.37.2) 2024-09-16T00:57:33.276 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:57:33 smithi169 podman[110010]: 2024-09-16 00:57:33.141588026 +0000 UTC m=+0.579751636 container cleanup 7bc609155f3f4d9e90ff4a79ab736e132c55ea2fa375c91ac74905745e94971c (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-mon-smithi169, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-16T00:57:33.277 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Sep 16 00:57:33 smithi169 bash[110010]: ceph-d346dd56-73c0-11ef-bceb-c7b262605968-mon-smithi169 2024-09-16T00:57:33.566 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d346dd56-73c0-11ef-bceb-c7b262605968@mon.smithi169.service' 2024-09-16T00:57:33.616 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:57:33.616 INFO:tasks.cephadm.mon.smithi169:Stopped mon.smithi169 2024-09-16T00:57:33.617 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-16T00:57:33.617 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-d346dd56-73c0-11ef-bceb-c7b262605968@osd.0 2024-09-16T00:57:34.092 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:57:33 smithi112 systemd[1]: Stopping Ceph osd.0 for d346dd56-73c0-11ef-bceb-c7b262605968... 2024-09-16T00:57:34.589 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:57:34 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-0[108893]: 2024-09-16T00:57:34.148+0000 7fa8b0771640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:57:34.589 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:57:34 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-0[108893]: 2024-09-16T00:57:34.148+0000 7fa8b0771640 -1 osd.0 90 *** Got signal Terminated *** 2024-09-16T00:57:34.590 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:57:34 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-0[108893]: 2024-09-16T00:57:34.148+0000 7fa8b0771640 -1 osd.0 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:57:39.884 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:57:39 smithi112 podman[146438]: 2024-09-16 00:57:39.59256386 +0000 UTC m=+5.777026271 container died 53efc9397017496cfaf270727f80e9d180674ea6c17b00a0dbb179233d014524 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-0, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:57:40.233 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:57:39 smithi112 podman[146438]: 2024-09-16 00:57:39.981660058 +0000 UTC m=+6.166122454 container cleanup 53efc9397017496cfaf270727f80e9d180674ea6c17b00a0dbb179233d014524 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-0, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-16T00:57:40.233 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:57:39 smithi112 bash[146438]: ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-0 2024-09-16T00:57:40.589 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:57:40 smithi112 podman[146452]: 2024-09-16 00:57:40.233041103 +0000 UTC m=+0.638988260 container remove 53efc9397017496cfaf270727f80e9d180674ea6c17b00a0dbb179233d014524 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:57:40.912 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:57:40 smithi112 podman[146544]: 2024-09-16 00:57:40.500499733 +0000 UTC m=+0.016257928 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:57:40.912 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:57:40 smithi112 podman[146544]: 2024-09-16 00:57:40.600832677 +0000 UTC m=+0.116590861 container create 06090a53077cb079afae685299c82448bfb2f10e899997526a9f477f0a23b5e2 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-16T00:57:40.913 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:57:40 smithi112 podman[146544]: 2024-09-16 00:57:40.812150375 +0000 UTC m=+0.327908566 container init 06090a53077cb079afae685299c82448bfb2f10e899997526a9f477f0a23b5e2 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-16T00:57:40.913 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:57:40 smithi112 podman[146544]: 2024-09-16 00:57:40.815561087 +0000 UTC m=+0.331319271 container start 06090a53077cb079afae685299c82448bfb2f10e899997526a9f477f0a23b5e2 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-16T00:57:41.274 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:57:40 smithi112 podman[146544]: 2024-09-16 00:57:40.912078969 +0000 UTC m=+0.427837188 container attach 06090a53077cb079afae685299c82448bfb2f10e899997526a9f477f0a23b5e2 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-0-deactivate, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-16T00:57:41.274 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 16 00:57:41 smithi112 podman[146544]: 2024-09-16 00:57:41.001562214 +0000 UTC m=+0.517320413 container died 06090a53077cb079afae685299c82448bfb2f10e899997526a9f477f0a23b5e2 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0) 2024-09-16T00:57:41.650 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d346dd56-73c0-11ef-bceb-c7b262605968@osd.0.service' 2024-09-16T00:57:41.690 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:57:41.691 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-16T00:57:41.691 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-16T00:57:41.691 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-d346dd56-73c0-11ef-bceb-c7b262605968@osd.1 2024-09-16T00:57:42.028 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:57:41 smithi112 systemd[1]: Stopping Ceph osd.1 for d346dd56-73c0-11ef-bceb-c7b262605968... 2024-09-16T00:57:42.315 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:57:42 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-1[114685]: 2024-09-16T00:57:42.026+0000 7fa0f71c9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:57:42.315 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:57:42 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-1[114685]: 2024-09-16T00:57:42.026+0000 7fa0f71c9640 -1 osd.1 90 *** Got signal Terminated *** 2024-09-16T00:57:42.315 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:57:42 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-1[114685]: 2024-09-16T00:57:42.026+0000 7fa0f71c9640 -1 osd.1 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:57:47.645 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:57:47 smithi112 podman[146656]: 2024-09-16 00:57:47.356720484 +0000 UTC m=+5.434186220 container died fe6f1a4a2cf45b04eb78e499b9269bcdf555fbe6d14b1e9c8f5fbddb8d70e707 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-1, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-16T00:57:48.012 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:57:47 smithi112 podman[146656]: 2024-09-16 00:57:47.767363411 +0000 UTC m=+5.844829147 container cleanup fe6f1a4a2cf45b04eb78e499b9269bcdf555fbe6d14b1e9c8f5fbddb8d70e707 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-1, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-16T00:57:48.012 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:57:47 smithi112 bash[146656]: ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-1 2024-09-16T00:57:48.339 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:57:48 smithi112 podman[146672]: 2024-09-16 00:57:48.012104358 +0000 UTC m=+0.650969398 container remove fe6f1a4a2cf45b04eb78e499b9269bcdf555fbe6d14b1e9c8f5fbddb8d70e707 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-1, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e) 2024-09-16T00:57:48.724 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:57:48 smithi112 podman[146760]: 2024-09-16 00:57:48.342675438 +0000 UTC m=+0.016041353 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:57:48.724 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:57:48 smithi112 podman[146760]: 2024-09-16 00:57:48.454719316 +0000 UTC m=+0.128085220 container create 8453f10ec6460addbce5f5c73a3fb2d8d1925a5bf575506d6d3df409cb993725 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-1-deactivate, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:57:48.724 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:57:48 smithi112 podman[146760]: 2024-09-16 00:57:48.648985811 +0000 UTC m=+0.322351716 container init 8453f10ec6460addbce5f5c73a3fb2d8d1925a5bf575506d6d3df409cb993725 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, CEPH_POINT_RELEASE=) 2024-09-16T00:57:48.724 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:57:48 smithi112 podman[146760]: 2024-09-16 00:57:48.65235463 +0000 UTC m=+0.325720535 container start 8453f10ec6460addbce5f5c73a3fb2d8d1925a5bf575506d6d3df409cb993725 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-8921c8e) 2024-09-16T00:57:48.724 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:57:48 smithi112 podman[146760]: 2024-09-16 00:57:48.723800542 +0000 UTC m=+0.397166466 container attach 8453f10ec6460addbce5f5c73a3fb2d8d1925a5bf575506d6d3df409cb993725 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-16T00:57:49.089 INFO:journalctl@ceph.osd.1.smithi112.stdout:Sep 16 00:57:48 smithi112 podman[146760]: 2024-09-16 00:57:48.854189653 +0000 UTC m=+0.527555575 container died 8453f10ec6460addbce5f5c73a3fb2d8d1925a5bf575506d6d3df409cb993725 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-16T00:57:49.418 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d346dd56-73c0-11ef-bceb-c7b262605968@osd.1.service' 2024-09-16T00:57:49.460 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:57:49.460 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-16T00:57:49.461 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-16T00:57:49.461 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-d346dd56-73c0-11ef-bceb-c7b262605968@osd.2 2024-09-16T00:57:49.781 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:57:49 smithi112 systemd[1]: Stopping Ceph osd.2 for d346dd56-73c0-11ef-bceb-c7b262605968... 2024-09-16T00:57:50.089 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:57:49 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-2[120025]: 2024-09-16T00:57:49.780+0000 7f749806b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:57:50.089 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:57:49 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-2[120025]: 2024-09-16T00:57:49.780+0000 7f749806b640 -1 osd.2 90 *** Got signal Terminated *** 2024-09-16T00:57:50.089 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:57:49 smithi112 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-2[120025]: 2024-09-16T00:57:49.780+0000 7f749806b640 -1 osd.2 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:57:55.398 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:57:55 smithi112 podman[146863]: 2024-09-16 00:57:55.118675603 +0000 UTC m=+5.431971265 container died 5f7863712057784c00db544571a201c6629a07fdd60e2133f923e0af6086cd4c (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-16T00:57:55.764 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:57:55 smithi112 podman[146863]: 2024-09-16 00:57:55.497089005 +0000 UTC m=+5.810384656 container cleanup 5f7863712057784c00db544571a201c6629a07fdd60e2133f923e0af6086cd4c (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-2, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-16T00:57:55.765 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:57:55 smithi112 bash[146863]: ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-2 2024-09-16T00:57:56.089 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:57:55 smithi112 podman[146890]: 2024-09-16 00:57:55.76500084 +0000 UTC m=+0.648948568 container remove 5f7863712057784c00db544571a201c6629a07fdd60e2133f923e0af6086cd4c (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-16T00:57:56.427 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:57:56 smithi112 podman[146984]: 2024-09-16 00:57:56.016845927 +0000 UTC m=+0.016039708 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:57:56.427 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:57:56 smithi112 podman[146984]: 2024-09-16 00:57:56.132782823 +0000 UTC m=+0.131976637 container create 192fc9b2b4d700f7e5ee0a484f49707f348664a833c15e456a416058d4369972 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:57:56.427 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:57:56 smithi112 podman[146984]: 2024-09-16 00:57:56.335216435 +0000 UTC m=+0.334410207 container init 192fc9b2b4d700f7e5ee0a484f49707f348664a833c15e456a416058d4369972 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-16T00:57:56.427 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:57:56 smithi112 podman[146984]: 2024-09-16 00:57:56.3385481 +0000 UTC m=+0.337741874 container start 192fc9b2b4d700f7e5ee0a484f49707f348664a833c15e456a416058d4369972 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-16T00:57:56.427 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:57:56 smithi112 podman[146984]: 2024-09-16 00:57:56.426941057 +0000 UTC m=+0.426134868 container attach 192fc9b2b4d700f7e5ee0a484f49707f348664a833c15e456a416058d4369972 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD) 2024-09-16T00:57:56.721 INFO:journalctl@ceph.osd.2.smithi112.stdout:Sep 16 00:57:56 smithi112 podman[146984]: 2024-09-16 00:57:56.523145537 +0000 UTC m=+0.522339316 container died 192fc9b2b4d700f7e5ee0a484f49707f348664a833c15e456a416058d4369972 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-2-deactivate, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:57:57.097 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d346dd56-73c0-11ef-bceb-c7b262605968@osd.2.service' 2024-09-16T00:57:57.137 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:57:57.137 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-16T00:57:57.137 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-16T00:57:57.137 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-d346dd56-73c0-11ef-bceb-c7b262605968@osd.3 2024-09-16T00:57:57.453 INFO:journalctl@ceph.osd.3.smithi169.stdout:Sep 16 00:57:57 smithi169 systemd[1]: Stopping Ceph osd.3 for d346dd56-73c0-11ef-bceb-c7b262605968... 2024-09-16T00:57:57.776 INFO:journalctl@ceph.osd.3.smithi169.stdout:Sep 16 00:57:57 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-3[86440]: 2024-09-16T00:57:57.452+0000 7fbd2c4d9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:57:57.776 INFO:journalctl@ceph.osd.3.smithi169.stdout:Sep 16 00:57:57 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-3[86440]: 2024-09-16T00:57:57.452+0000 7fbd2c4d9640 -1 osd.3 90 *** Got signal Terminated *** 2024-09-16T00:57:57.777 INFO:journalctl@ceph.osd.3.smithi169.stdout:Sep 16 00:57:57 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-3[86440]: 2024-09-16T00:57:57.452+0000 7fbd2c4d9640 -1 osd.3 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:58:01.277 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:58:00 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5[96391]: 2024-09-16T00:58:00.993+0000 7f4ed211c640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-16T00:57:37.062061+0000 front 2024-09-16T00:57:37.062079+0000 (oldest deadline 2024-09-16T00:58:00.561901+0000) 2024-09-16T00:58:01.777 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:58:01 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-4[91272]: 2024-09-16T00:58:01.350+0000 7f5b6907f640 -1 osd.4 90 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-16T00:57:38.236200+0000 front 2024-09-16T00:57:38.236300+0000 (oldest deadline 2024-09-16T00:58:00.536063+0000) 2024-09-16T00:58:02.311 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:58:02 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5[96391]: 2024-09-16T00:58:02.038+0000 7f4ed211c640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-16T00:57:37.062061+0000 front 2024-09-16T00:57:37.062079+0000 (oldest deadline 2024-09-16T00:58:00.561901+0000) 2024-09-16T00:58:02.777 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:58:02 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-4[91272]: 2024-09-16T00:58:02.310+0000 7f5b6907f640 -1 osd.4 90 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-16T00:57:38.236200+0000 front 2024-09-16T00:57:38.236300+0000 (oldest deadline 2024-09-16T00:58:00.536063+0000) 2024-09-16T00:58:03.146 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:58:03 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5[96391]: 2024-09-16T00:58:03.003+0000 7f4ed211c640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-16T00:57:37.062061+0000 front 2024-09-16T00:57:37.062079+0000 (oldest deadline 2024-09-16T00:58:00.561901+0000) 2024-09-16T00:58:03.146 INFO:journalctl@ceph.osd.3.smithi169.stdout:Sep 16 00:58:02 smithi169 podman[110132]: 2024-09-16 00:58:02.865101498 +0000 UTC m=+5.526979320 container died ee07b557781030b3c0039b44dd7fa8ff42e95f779334231057459f8a4fd594f6 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-3, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:58:03.512 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:58:03 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-4[91272]: 2024-09-16T00:58:03.337+0000 7f5b6907f640 -1 osd.4 90 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-16T00:57:38.236200+0000 front 2024-09-16T00:57:38.236300+0000 (oldest deadline 2024-09-16T00:58:00.536063+0000) 2024-09-16T00:58:03.513 INFO:journalctl@ceph.osd.3.smithi169.stdout:Sep 16 00:58:03 smithi169 podman[110132]: 2024-09-16 00:58:03.244029893 +0000 UTC m=+5.905907716 container cleanup ee07b557781030b3c0039b44dd7fa8ff42e95f779334231057459f8a4fd594f6 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-3, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD) 2024-09-16T00:58:03.513 INFO:journalctl@ceph.osd.3.smithi169.stdout:Sep 16 00:58:03 smithi169 bash[110132]: ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-3 2024-09-16T00:58:03.776 INFO:journalctl@ceph.osd.3.smithi169.stdout:Sep 16 00:58:03 smithi169 podman[110160]: 2024-09-16 00:58:03.512553024 +0000 UTC m=+0.643268624 container remove ee07b557781030b3c0039b44dd7fa8ff42e95f779334231057459f8a4fd594f6 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-3, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-16T00:58:04.165 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:58:03 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5[96391]: 2024-09-16T00:58:03.983+0000 7f4ed211c640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-16T00:57:37.062061+0000 front 2024-09-16T00:57:37.062079+0000 (oldest deadline 2024-09-16T00:58:00.561901+0000) 2024-09-16T00:58:04.166 INFO:journalctl@ceph.osd.3.smithi169.stdout:Sep 16 00:58:03 smithi169 podman[110250]: 2024-09-16 00:58:03.77139951 +0000 UTC m=+0.016167720 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:58:04.166 INFO:journalctl@ceph.osd.3.smithi169.stdout:Sep 16 00:58:03 smithi169 podman[110250]: 2024-09-16 00:58:03.880199688 +0000 UTC m=+0.124967935 container create c3f0cdbc501c7f41186189f9abc71652dd8505a44a055c8f4d9d02b6a2fd98b2 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True) 2024-09-16T00:58:04.166 INFO:journalctl@ceph.osd.3.smithi169.stdout:Sep 16 00:58:04 smithi169 podman[110250]: 2024-09-16 00:58:04.090922873 +0000 UTC m=+0.335691095 container init c3f0cdbc501c7f41186189f9abc71652dd8505a44a055c8f4d9d02b6a2fd98b2 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:58:04.166 INFO:journalctl@ceph.osd.3.smithi169.stdout:Sep 16 00:58:04 smithi169 podman[110250]: 2024-09-16 00:58:04.09426959 +0000 UTC m=+0.339037797 container start c3f0cdbc501c7f41186189f9abc71652dd8505a44a055c8f4d9d02b6a2fd98b2 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-16T00:58:04.166 INFO:journalctl@ceph.osd.3.smithi169.stdout:Sep 16 00:58:04 smithi169 podman[110250]: 2024-09-16 00:58:04.165728645 +0000 UTC m=+0.410496850 container attach c3f0cdbc501c7f41186189f9abc71652dd8505a44a055c8f4d9d02b6a2fd98b2 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e) 2024-09-16T00:58:04.527 INFO:journalctl@ceph.osd.3.smithi169.stdout:Sep 16 00:58:04 smithi169 podman[110250]: 2024-09-16 00:58:04.279999795 +0000 UTC m=+0.524767999 container died c3f0cdbc501c7f41186189f9abc71652dd8505a44a055c8f4d9d02b6a2fd98b2 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, ceph=True) 2024-09-16T00:58:04.527 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:58:04 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-4[91272]: 2024-09-16T00:58:04.306+0000 7f5b6907f640 -1 osd.4 90 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-16T00:57:38.236200+0000 front 2024-09-16T00:57:38.236300+0000 (oldest deadline 2024-09-16T00:58:00.536063+0000) 2024-09-16T00:58:04.947 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d346dd56-73c0-11ef-bceb-c7b262605968@osd.3.service' 2024-09-16T00:58:04.987 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:58:04.987 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-16T00:58:04.987 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-16T00:58:04.987 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-d346dd56-73c0-11ef-bceb-c7b262605968@osd.4 2024-09-16T00:58:05.276 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:58:05 smithi169 systemd[1]: Stopping Ceph osd.4 for d346dd56-73c0-11ef-bceb-c7b262605968... 2024-09-16T00:58:05.277 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:58:04 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5[96391]: 2024-09-16T00:58:04.947+0000 7f4ed211c640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-16T00:57:37.062061+0000 front 2024-09-16T00:57:37.062079+0000 (oldest deadline 2024-09-16T00:58:00.561901+0000) 2024-09-16T00:58:05.776 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:58:05 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-4[91272]: 2024-09-16T00:58:05.286+0000 7f5b6907f640 -1 osd.4 90 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-16T00:57:38.236200+0000 front 2024-09-16T00:57:38.236300+0000 (oldest deadline 2024-09-16T00:58:00.536063+0000) 2024-09-16T00:58:05.777 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:58:05 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-4[91272]: 2024-09-16T00:58:05.324+0000 7f5b70a82640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:58:05.777 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:58:05 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-4[91272]: 2024-09-16T00:58:05.324+0000 7f5b70a82640 -1 osd.4 90 *** Got signal Terminated *** 2024-09-16T00:58:05.777 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:58:05 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-4[91272]: 2024-09-16T00:58:05.324+0000 7f5b70a82640 -1 osd.4 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:58:06.261 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:58:05 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5[96391]: 2024-09-16T00:58:05.946+0000 7f4ed211c640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-16T00:57:37.062061+0000 front 2024-09-16T00:57:37.062079+0000 (oldest deadline 2024-09-16T00:58:00.561901+0000) 2024-09-16T00:58:06.526 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:58:06 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-4[91272]: 2024-09-16T00:58:06.260+0000 7f5b6907f640 -1 osd.4 90 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-16T00:57:38.236200+0000 front 2024-09-16T00:57:38.236300+0000 (oldest deadline 2024-09-16T00:58:00.536063+0000) 2024-09-16T00:58:07.262 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:58:06 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5[96391]: 2024-09-16T00:58:06.973+0000 7f4ed211c640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-16T00:57:37.062061+0000 front 2024-09-16T00:57:37.062079+0000 (oldest deadline 2024-09-16T00:58:00.561901+0000) 2024-09-16T00:58:07.526 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:58:07 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-4[91272]: 2024-09-16T00:58:07.260+0000 7f5b6907f640 -1 osd.4 90 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-16T00:57:38.236200+0000 front 2024-09-16T00:57:38.236300+0000 (oldest deadline 2024-09-16T00:58:00.536063+0000) 2024-09-16T00:58:08.250 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:58:07 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5[96391]: 2024-09-16T00:58:07.929+0000 7f4ed211c640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-16T00:57:37.062061+0000 front 2024-09-16T00:57:37.062079+0000 (oldest deadline 2024-09-16T00:58:00.561901+0000) 2024-09-16T00:58:08.250 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:58:07 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5[96391]: 2024-09-16T00:58:07.929+0000 7f4ed211c640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.112:6814 osd.1 since back 2024-09-16T00:57:46.263957+0000 front 2024-09-16T00:57:46.264020+0000 (oldest deadline 2024-09-16T00:58:07.363909+0000) 2024-09-16T00:58:08.526 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:58:08 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-4[91272]: 2024-09-16T00:58:08.248+0000 7f5b6907f640 -1 osd.4 90 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-16T00:57:38.236200+0000 front 2024-09-16T00:57:38.236300+0000 (oldest deadline 2024-09-16T00:58:00.536063+0000) 2024-09-16T00:58:09.269 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:58:08 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5[96391]: 2024-09-16T00:58:08.969+0000 7f4ed211c640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-16T00:57:37.062061+0000 front 2024-09-16T00:57:37.062079+0000 (oldest deadline 2024-09-16T00:58:00.561901+0000) 2024-09-16T00:58:09.270 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:58:08 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5[96391]: 2024-09-16T00:58:08.969+0000 7f4ed211c640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.112:6814 osd.1 since back 2024-09-16T00:57:46.263957+0000 front 2024-09-16T00:57:46.264020+0000 (oldest deadline 2024-09-16T00:58:07.363909+0000) 2024-09-16T00:58:09.526 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:58:09 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-4[91272]: 2024-09-16T00:58:09.268+0000 7f5b6907f640 -1 osd.4 90 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-16T00:57:38.236200+0000 front 2024-09-16T00:57:38.236300+0000 (oldest deadline 2024-09-16T00:58:00.536063+0000) 2024-09-16T00:58:10.277 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:58:10 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5[96391]: 2024-09-16T00:58:10.005+0000 7f4ed211c640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-16T00:57:37.062061+0000 front 2024-09-16T00:57:37.062079+0000 (oldest deadline 2024-09-16T00:58:00.561901+0000) 2024-09-16T00:58:10.277 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:58:10 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5[96391]: 2024-09-16T00:58:10.005+0000 7f4ed211c640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.112:6814 osd.1 since back 2024-09-16T00:57:46.263957+0000 front 2024-09-16T00:57:46.264020+0000 (oldest deadline 2024-09-16T00:58:07.363909+0000) 2024-09-16T00:58:10.707 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:58:10 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-4[91272]: 2024-09-16T00:58:10.314+0000 7f5b6907f640 -1 osd.4 90 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-16T00:57:38.236200+0000 front 2024-09-16T00:57:38.236300+0000 (oldest deadline 2024-09-16T00:58:00.536063+0000) 2024-09-16T00:58:10.707 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:58:10 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-4[91272]: 2024-09-16T00:58:10.314+0000 7f5b6907f640 -1 osd.4 90 heartbeat_check: no reply from 172.21.15.112:6814 osd.1 since back 2024-09-16T00:57:44.636970+0000 front 2024-09-16T00:57:44.637084+0000 (oldest deadline 2024-09-16T00:58:09.337044+0000) 2024-09-16T00:58:10.968 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:58:10 smithi169 podman[110352]: 2024-09-16 00:58:10.706873688 +0000 UTC m=+5.503074552 container died 769f026a54eb1a19f1cbead116d6087157579022785250a7841626a5d0e8f1d6 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-4, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-16T00:58:11.276 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:58:11 smithi169 podman[110352]: 2024-09-16 00:58:11.100265507 +0000 UTC m=+5.896466371 container cleanup 769f026a54eb1a19f1cbead116d6087157579022785250a7841626a5d0e8f1d6 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-4, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=) 2024-09-16T00:58:11.277 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:58:11 smithi169 bash[110352]: ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-4 2024-09-16T00:58:11.277 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:58:10 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5[96391]: 2024-09-16T00:58:10.967+0000 7f4ed211c640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-16T00:57:37.062061+0000 front 2024-09-16T00:57:37.062079+0000 (oldest deadline 2024-09-16T00:58:00.561901+0000) 2024-09-16T00:58:11.277 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:58:10 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5[96391]: 2024-09-16T00:58:10.967+0000 7f4ed211c640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.112:6814 osd.1 since back 2024-09-16T00:57:46.263957+0000 front 2024-09-16T00:57:46.264020+0000 (oldest deadline 2024-09-16T00:58:07.363909+0000) 2024-09-16T00:58:11.675 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:58:11 smithi169 podman[110369]: 2024-09-16 00:58:11.351948989 +0000 UTC m=+0.643715686 container remove 769f026a54eb1a19f1cbead116d6087157579022785250a7841626a5d0e8f1d6 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-4, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-16T00:58:11.930 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:58:11 smithi169 podman[110460]: 2024-09-16 00:58:11.578180145 +0000 UTC m=+0.015953127 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:58:11.931 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:58:11 smithi169 podman[110460]: 2024-09-16 00:58:11.694351061 +0000 UTC m=+0.132124029 container create 304147a48ba113805cb37bfc01b693eb926c85c38cf4920322c3a24643da5623 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-16T00:58:11.931 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:58:11 smithi169 podman[110460]: 2024-09-16 00:58:11.930722169 +0000 UTC m=+0.368495145 container init 304147a48ba113805cb37bfc01b693eb926c85c38cf4920322c3a24643da5623 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True) 2024-09-16T00:58:12.277 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:58:11 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5[96391]: 2024-09-16T00:58:11.947+0000 7f4ed211c640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-16T00:57:37.062061+0000 front 2024-09-16T00:57:37.062079+0000 (oldest deadline 2024-09-16T00:58:00.561901+0000) 2024-09-16T00:58:12.277 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:58:11 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5[96391]: 2024-09-16T00:58:11.947+0000 7f4ed211c640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.112:6814 osd.1 since back 2024-09-16T00:57:46.263957+0000 front 2024-09-16T00:57:46.264020+0000 (oldest deadline 2024-09-16T00:58:07.363909+0000) 2024-09-16T00:58:12.277 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:58:11 smithi169 podman[110460]: 2024-09-16 00:58:11.934293683 +0000 UTC m=+0.372066656 container start 304147a48ba113805cb37bfc01b693eb926c85c38cf4920322c3a24643da5623 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-16T00:58:12.277 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:58:12 smithi169 podman[110460]: 2024-09-16 00:58:12.022311746 +0000 UTC m=+0.460084720 container attach 304147a48ba113805cb37bfc01b693eb926c85c38cf4920322c3a24643da5623 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-16T00:58:12.277 INFO:journalctl@ceph.osd.4.smithi169.stdout:Sep 16 00:58:12 smithi169 podman[110460]: 2024-09-16 00:58:12.11537837 +0000 UTC m=+0.553151352 container died 304147a48ba113805cb37bfc01b693eb926c85c38cf4920322c3a24643da5623 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-16T00:58:12.681 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d346dd56-73c0-11ef-bceb-c7b262605968@osd.4.service' 2024-09-16T00:58:12.721 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:58:12.721 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-16T00:58:12.721 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-16T00:58:12.721 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-d346dd56-73c0-11ef-bceb-c7b262605968@osd.5 2024-09-16T00:58:12.952 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:58:12 smithi169 systemd[1]: Stopping Ceph osd.5 for d346dd56-73c0-11ef-bceb-c7b262605968... 2024-09-16T00:58:13.277 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:58:12 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5[96391]: 2024-09-16T00:58:12.951+0000 7f4ed211c640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-16T00:57:37.062061+0000 front 2024-09-16T00:57:37.062079+0000 (oldest deadline 2024-09-16T00:58:00.561901+0000) 2024-09-16T00:58:13.277 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:58:12 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5[96391]: 2024-09-16T00:58:12.951+0000 7f4ed211c640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.112:6814 osd.1 since back 2024-09-16T00:57:46.263957+0000 front 2024-09-16T00:57:46.264020+0000 (oldest deadline 2024-09-16T00:58:07.363909+0000) 2024-09-16T00:58:13.277 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:58:13 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5[96391]: 2024-09-16T00:58:13.053+0000 7f4ed5b17640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:58:13.277 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:58:13 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5[96391]: 2024-09-16T00:58:13.053+0000 7f4ed5b17640 -1 osd.5 90 *** Got signal Terminated *** 2024-09-16T00:58:13.277 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:58:13 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5[96391]: 2024-09-16T00:58:13.053+0000 7f4ed5b17640 -1 osd.5 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:58:14.276 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:58:13 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5[96391]: 2024-09-16T00:58:13.935+0000 7f4ed211c640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-16T00:57:37.062061+0000 front 2024-09-16T00:57:37.062079+0000 (oldest deadline 2024-09-16T00:58:00.561901+0000) 2024-09-16T00:58:14.277 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:58:13 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5[96391]: 2024-09-16T00:58:13.935+0000 7f4ed211c640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.112:6814 osd.1 since back 2024-09-16T00:57:46.263957+0000 front 2024-09-16T00:57:46.264020+0000 (oldest deadline 2024-09-16T00:58:07.363909+0000) 2024-09-16T00:58:15.277 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:58:14 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5[96391]: 2024-09-16T00:58:14.935+0000 7f4ed211c640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-16T00:57:37.062061+0000 front 2024-09-16T00:57:37.062079+0000 (oldest deadline 2024-09-16T00:58:00.561901+0000) 2024-09-16T00:58:15.277 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:58:14 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5[96391]: 2024-09-16T00:58:14.935+0000 7f4ed211c640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.112:6814 osd.1 since back 2024-09-16T00:57:46.263957+0000 front 2024-09-16T00:57:46.264020+0000 (oldest deadline 2024-09-16T00:58:07.363909+0000) 2024-09-16T00:58:16.277 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:58:15 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5[96391]: 2024-09-16T00:58:15.972+0000 7f4ed211c640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-16T00:57:37.062061+0000 front 2024-09-16T00:57:37.062079+0000 (oldest deadline 2024-09-16T00:58:00.561901+0000) 2024-09-16T00:58:16.277 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:58:15 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5[96391]: 2024-09-16T00:58:15.972+0000 7f4ed211c640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.112:6814 osd.1 since back 2024-09-16T00:57:46.263957+0000 front 2024-09-16T00:57:46.264020+0000 (oldest deadline 2024-09-16T00:58:07.363909+0000) 2024-09-16T00:58:16.277 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:58:15 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5[96391]: 2024-09-16T00:58:15.972+0000 7f4ed211c640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.112:6822 osd.2 since back 2024-09-16T00:57:53.264739+0000 front 2024-09-16T00:57:53.264803+0000 (oldest deadline 2024-09-16T00:58:15.564675+0000) 2024-09-16T00:58:17.276 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:58:16 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5[96391]: 2024-09-16T00:58:16.989+0000 7f4ed211c640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-16T00:57:37.062061+0000 front 2024-09-16T00:57:37.062079+0000 (oldest deadline 2024-09-16T00:58:00.561901+0000) 2024-09-16T00:58:17.277 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:58:16 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5[96391]: 2024-09-16T00:58:16.989+0000 7f4ed211c640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.112:6814 osd.1 since back 2024-09-16T00:57:46.263957+0000 front 2024-09-16T00:57:46.264020+0000 (oldest deadline 2024-09-16T00:58:07.363909+0000) 2024-09-16T00:58:17.277 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:58:16 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5[96391]: 2024-09-16T00:58:16.989+0000 7f4ed211c640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.112:6822 osd.2 since back 2024-09-16T00:57:53.264739+0000 front 2024-09-16T00:57:53.264803+0000 (oldest deadline 2024-09-16T00:58:15.564675+0000) 2024-09-16T00:58:18.277 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:58:17 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5[96391]: 2024-09-16T00:58:17.985+0000 7f4ed211c640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-16T00:57:37.062061+0000 front 2024-09-16T00:57:37.062079+0000 (oldest deadline 2024-09-16T00:58:00.561901+0000) 2024-09-16T00:58:18.277 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:58:17 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5[96391]: 2024-09-16T00:58:17.985+0000 7f4ed211c640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.112:6814 osd.1 since back 2024-09-16T00:57:46.263957+0000 front 2024-09-16T00:57:46.264020+0000 (oldest deadline 2024-09-16T00:58:07.363909+0000) 2024-09-16T00:58:18.277 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:58:17 smithi169 ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5[96391]: 2024-09-16T00:58:17.985+0000 7f4ed211c640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.112:6822 osd.2 since back 2024-09-16T00:57:53.264739+0000 front 2024-09-16T00:57:53.264803+0000 (oldest deadline 2024-09-16T00:58:15.564675+0000) 2024-09-16T00:58:18.864 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:58:18 smithi169 podman[110565]: 2024-09-16 00:58:18.595980982 +0000 UTC m=+5.650578306 container died a6007c13cf78e7f0ea64d10360a3f08e6538f290598f551c004741057c5e116c (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=) 2024-09-16T00:58:19.222 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:58:18 smithi169 podman[110565]: 2024-09-16 00:58:18.962394764 +0000 UTC m=+6.016992088 container cleanup a6007c13cf78e7f0ea64d10360a3f08e6538f290598f551c004741057c5e116c (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-16T00:58:19.222 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:58:18 smithi169 bash[110565]: ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5 2024-09-16T00:58:19.526 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:58:19 smithi169 podman[110581]: 2024-09-16 00:58:19.222050575 +0000 UTC m=+0.616700069 container remove a6007c13cf78e7f0ea64d10360a3f08e6538f290598f551c004741057c5e116c (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-16T00:58:19.909 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:58:19 smithi169 podman[110669]: 2024-09-16 00:58:19.48713488 +0000 UTC m=+0.016445700 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:58:19.909 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:58:19 smithi169 podman[110669]: 2024-09-16 00:58:19.598050829 +0000 UTC m=+0.127361642 container create f5f2247ca15fe8f2df68246f49ac4f29e5bf9120b184c2b8bb879855f1f40a71 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-16T00:58:19.909 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:58:19 smithi169 podman[110669]: 2024-09-16 00:58:19.825824169 +0000 UTC m=+0.355134980 container init f5f2247ca15fe8f2df68246f49ac4f29e5bf9120b184c2b8bb879855f1f40a71 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5-deactivate, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:58:19.909 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:58:19 smithi169 podman[110669]: 2024-09-16 00:58:19.829149043 +0000 UTC m=+0.358459858 container start f5f2247ca15fe8f2df68246f49ac4f29e5bf9120b184c2b8bb879855f1f40a71 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5-deactivate, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.build-date=20240731) 2024-09-16T00:58:19.909 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:58:19 smithi169 podman[110669]: 2024-09-16 00:58:19.908897767 +0000 UTC m=+0.438208578 container attach f5f2247ca15fe8f2df68246f49ac4f29e5bf9120b184c2b8bb879855f1f40a71 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e) 2024-09-16T00:58:20.276 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:58:20 smithi169 podman[110669]: 2024-09-16 00:58:20.018138365 +0000 UTC m=+0.547449187 container died f5f2247ca15fe8f2df68246f49ac4f29e5bf9120b184c2b8bb879855f1f40a71 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e) 2024-09-16T00:58:20.571 INFO:journalctl@ceph.osd.5.smithi169.stdout:Sep 16 00:58:20 smithi169 podman[110692]: 2024-09-16 00:58:20.554455439 +0000 UTC m=+0.530426480 container remove f5f2247ca15fe8f2df68246f49ac4f29e5bf9120b184c2b8bb879855f1f40a71 (image=quay.ceph.io/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-d346dd56-73c0-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-16T00:58:20.576 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d346dd56-73c0-11ef-bceb-c7b262605968@osd.5.service' 2024-09-16T00:58:20.615 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:58:20.615 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-16T00:58:20.615 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d346dd56-73c0-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-16T00:58:24.176 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi112.stderr:ceph-fuse[74690]: fuse finished with error 0 and tester_r 0 2024-09-16T00:58:45.104 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d346dd56-73c0-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-16T00:58:59.490 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T00:58:59.524 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T00:58:59.551 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-16T00:58:59.553 DEBUG:teuthology.misc:Transferring archived files from smithi112:/var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906384/remote/smithi112/crash 2024-09-16T00:58:59.553 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar c -f - -C /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/crash -- . 2024-09-16T00:58:59.591 INFO:teuthology.orchestra.run.smithi112.stderr:tar: /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-16T00:58:59.591 INFO:teuthology.orchestra.run.smithi112.stderr:tar: Error is not recoverable: exiting now 2024-09-16T00:58:59.593 DEBUG:teuthology.misc:Transferring archived files from smithi169:/var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906384/remote/smithi169/crash 2024-09-16T00:58:59.594 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar c -f - -C /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/crash -- . 2024-09-16T00:58:59.617 INFO:teuthology.orchestra.run.smithi169.stderr:tar: /var/lib/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-16T00:58:59.618 INFO:teuthology.orchestra.run.smithi169.stderr:tar: Error is not recoverable: exiting now 2024-09-16T00:58:59.619 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-16T00:58:59.619 DEBUG:teuthology.orchestra.run.smithi112:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-09-16T00:58:59.673 INFO:teuthology.orchestra.run.smithi112.stdout:2024-09-16T00:40:00.000239+0000 mon.smithi112 (mon.0) 758 : cluster [ERR] fs cephfs is degraded 2024-09-16T00:58:59.673 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-09-16T00:58:59.674 INFO:tasks.cephadm:Compressing logs... 2024-09-16T00:58:59.674 DEBUG:teuthology.orchestra.run.smithi112:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-16T00:58:59.716 DEBUG:teuthology.orchestra.run.smithi169:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-16T00:58:59.739 INFO:teuthology.orchestra.run.smithi112.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose --: No such file or directory /var/log/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/ceph-osd.0.log 2024-09-16T00:58:59.740 INFO:teuthology.orchestra.run.smithi112.stderr: 2024-09-16T00:58:59.740 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/ceph-osd.1.log 2024-09-16T00:58:59.740 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-16T00:58:59.740 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-16T00:58:59.742 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-16T00:58:59.742 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/ceph-osd.2.log 2024-09-16T00:58:59.742 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/ceph.log 2024-09-16T00:58:59.745 INFO:teuthology.orchestra.run.smithi169.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/ceph-mgr.smithi169.mtrobk.log 2024-09-16T00:58:59.745 INFO:teuthology.orchestra.run.smithi169.stderr:: No such file or directory 2024-09-16T00:58:59.745 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-16T00:58:59.745 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose --/var/log/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/ceph-mgr.smithi169.mtrobk.log: /var/log/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/ceph-osd.4.log 2024-09-16T00:58:59.745 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-16T00:58:59.745 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-16T00:58:59.745 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/ceph-osd.5.log 2024-09-16T00:58:59.745 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/ceph.audit.log: /var/log/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/ceph.log 2024-09-16T00:58:59.746 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/ceph-mon.smithi169.log 2024-09-16T00:58:59.746 INFO:teuthology.orchestra.run.smithi169.stderr: 85.2%/var/log/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/ceph.log: -- replaced with /var/log/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-16T00:58:59.746 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/ceph-mds.cephfs.smithi169.lgeqrj.log 2024-09-16T00:58:59.746 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/ceph-mon.smithi169.log: gzip -5 --verbose -- /var/log/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/ceph-osd.3.log 2024-09-16T00:58:59.749 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/ceph-mgr.smithi112.xrkkbc.log 2024-09-16T00:58:59.749 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/ceph.log: 86.4% -- replaced with /var/log/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-16T00:58:59.749 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/ceph-mgr.smithi112.xrkkbc.log: 91.2% -- replaced with /var/log/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-16T00:58:59.749 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/ceph-mds.cephfs.smithi112.hxfwkh.log 2024-09-16T00:58:59.749 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/ceph-mds.cephfs.smithi112.ithwzj.log 2024-09-16T00:58:59.749 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/ceph-mds.cephfs.smithi112.hxfwkh.log: 89.1% -- replaced with /var/log/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-16T00:58:59.749 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/ceph-mon.smithi112.log 2024-09-16T00:58:59.750 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/ceph-mds.cephfs.smithi169.lgeqrj.log: 91.3% -- replaced with /var/log/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-16T00:58:59.750 INFO:teuthology.orchestra.run.smithi169.stderr: 89.2% -- replaced with /var/log/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-16T00:58:59.750 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/ceph-mds.cephfs.smithi169.gawwgw.log 2024-09-16T00:58:59.750 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.log 2024-09-16T00:58:59.756 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/ceph-mds.cephfs.smithi112.ithwzj.log: 91.0% -- replaced with /var/log/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-16T00:58:59.761 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/ceph-mds.cephfs.smithi169.gawwgw.log: 91.0% -- replaced with /var/log/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-16T00:58:59.761 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-16T00:58:59.765 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-09-16T00:58:59.765 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/ceph-mon.smithi112.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-16T00:58:59.777 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.1.log: /var/log/ceph/cephadm.log: 92.0% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-16T00:58:59.807 INFO:teuthology.orchestra.run.smithi169.stderr: 90.2% -- replaced with /var/log/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/ceph-mgr.smithi169.mtrobk.log.gz 2024-09-16T00:58:59.842 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.0.log: /var/log/ceph/cephadm.log: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-16T00:59:00.225 INFO:teuthology.orchestra.run.smithi112.stderr: 89.5% -- replaced with /var/log/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/ceph-mgr.smithi112.xrkkbc.log.gz 2024-09-16T00:59:00.366 INFO:teuthology.orchestra.run.smithi169.stderr: 92.5% -- replaced with /var/log/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/ceph-mon.smithi169.log.gz 2024-09-16T00:59:01.508 INFO:teuthology.orchestra.run.smithi112.stderr: 90.2% -- replaced with /var/log/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/ceph-mon.smithi112.log.gz 2024-09-16T00:59:03.705 INFO:teuthology.orchestra.run.smithi169.stderr: 93.5% -- replaced with /var/log/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/ceph-osd.4.log.gz 2024-09-16T00:59:04.931 INFO:teuthology.orchestra.run.smithi112.stderr: 93.4% -- replaced with /var/log/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/ceph-osd.2.log.gz 2024-09-16T00:59:05.904 INFO:teuthology.orchestra.run.smithi169.stderr: 93.6% -- replaced with /var/log/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/ceph-osd.5.log.gz 2024-09-16T00:59:05.978 INFO:teuthology.orchestra.run.smithi112.stderr: 93.5% -- replaced with /var/log/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/ceph-osd.0.log.gz 2024-09-16T00:59:06.344 INFO:teuthology.orchestra.run.smithi112.stderr: 93.5% -- replaced with /var/log/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/ceph-osd.1.log.gz 2024-09-16T00:59:06.524 INFO:teuthology.orchestra.run.smithi169.stderr: 95.1% -- replaced with /var/log/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/ceph-mds.cephfs.smithi169.lgeqrj.log.gz 2024-09-16T00:59:07.323 INFO:teuthology.orchestra.run.smithi169.stderr: 93.6% -- replaced with /var/log/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/ceph-osd.3.log.gz 2024-09-16T00:59:08.198 INFO:teuthology.orchestra.run.smithi112.stderr: 95.1% -- replaced with /var/log/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/ceph-mds.cephfs.smithi112.ithwzj.log.gz 2024-09-16T00:59:14.825 INFO:teuthology.orchestra.run.smithi112.stderr: 93.2% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-09-16T00:59:15.007 INFO:teuthology.orchestra.run.smithi169.stderr:gzip: /var/log/ceph/ceph-client.1.log: file size changed while zipping 2024-09-16T00:59:15.007 INFO:teuthology.orchestra.run.smithi169.stderr: 93.2% -- replaced with /var/log/ceph/ceph-client.1.log.gz 2024-09-16T00:59:42.377 INFO:teuthology.orchestra.run.smithi169.stderr: 93.6% -- replaced with /var/log/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/ceph-mds.cephfs.smithi169.gawwgw.log.gz 2024-09-16T00:59:42.378 INFO:teuthology.orchestra.run.smithi169.stderr: 2024-09-16T00:59:42.379 INFO:teuthology.orchestra.run.smithi169.stderr:real 0m42.648s 2024-09-16T00:59:42.379 INFO:teuthology.orchestra.run.smithi169.stderr:user 1m20.469s 2024-09-16T00:59:42.379 INFO:teuthology.orchestra.run.smithi169.stderr:sys 0m2.442s 2024-09-16T00:59:52.471 INFO:teuthology.orchestra.run.smithi112.stderr: 93.4% -- replaced with /var/log/ceph/d346dd56-73c0-11ef-bceb-c7b262605968/ceph-mds.cephfs.smithi112.hxfwkh.log.gz 2024-09-16T00:59:52.473 INFO:teuthology.orchestra.run.smithi112.stderr: 2024-09-16T00:59:52.473 INFO:teuthology.orchestra.run.smithi112.stderr:real 0m52.744s 2024-09-16T00:59:52.473 INFO:teuthology.orchestra.run.smithi112.stderr:user 1m33.218s 2024-09-16T00:59:52.473 INFO:teuthology.orchestra.run.smithi112.stderr:sys 0m3.010s 2024-09-16T00:59:52.473 INFO:tasks.cephadm:Archiving logs... 2024-09-16T00:59:52.474 DEBUG:teuthology.misc:Transferring archived files from smithi112:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906384/remote/smithi112/log 2024-09-16T00:59:52.474 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T01:00:04.937 DEBUG:teuthology.misc:Transferring archived files from smithi169:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906384/remote/smithi169/log 2024-09-16T01:00:04.938 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T01:00:15.851 INFO:tasks.cephadm:Removing cluster... 2024-09-16T01:00:15.851 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d346dd56-73c0-11ef-bceb-c7b262605968 --force 2024-09-16T01:00:16.409 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d346dd56-73c0-11ef-bceb-c7b262605968 --force 2024-09-16T01:00:16.966 INFO:tasks.cephadm:Removing cephadm ... 2024-09-16T01:00:16.966 DEBUG:teuthology.orchestra.run.smithi112:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-16T01:00:16.982 DEBUG:teuthology.orchestra.run.smithi169:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-16T01:00:16.998 INFO:tasks.cephadm:Teardown complete 2024-09-16T01:00:16.999 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-16T01:00:17.016 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/task/install/__init__.py", line 641, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa/tasks/ceph_fuse.py", line 248, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (50) after waiting for 300 seconds 2024-09-16T01:00:17.017 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-16T01:00:17.017 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-16T01:00:17.025 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-16T01:00:17.108 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-16T01:00:17.119 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-16T01:00:17.119 DEBUG:teuthology.orchestra.run.smithi112:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T01:00:17.122 DEBUG:teuthology.orchestra.run.smithi169:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T01:00:17.134 INFO:teuthology.orchestra.run.smithi112.stderr:bash: line 1: ntpq: command not found 2024-09-16T01:00:17.135 INFO:teuthology.orchestra.run.smithi169.stderr:bash: line 1: ntpq: command not found 2024-09-16T01:00:17.138 INFO:teuthology.orchestra.run.smithi112.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T01:00:17.138 INFO:teuthology.orchestra.run.smithi112.stdout:=============================================================================== 2024-09-16T01:00:17.138 INFO:teuthology.orchestra.run.smithi112.stdout:^* hv01.front.sepia.ceph.com 3 6 377 13 -173us[ -193us] +/- 45ms 2024-09-16T01:00:17.139 INFO:teuthology.orchestra.run.smithi112.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 11 -647us[ -647us] +/- 53ms 2024-09-16T01:00:17.139 INFO:teuthology.orchestra.run.smithi112.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 11 +1430us[+1430us] +/- 83ms 2024-09-16T01:00:17.139 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T01:00:17.140 INFO:teuthology.orchestra.run.smithi169.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T01:00:17.140 INFO:teuthology.orchestra.run.smithi169.stdout:=============================================================================== 2024-09-16T01:00:17.140 INFO:teuthology.orchestra.run.smithi169.stdout:^* hv01.front.sepia.ceph.com 3 6 377 12 -176us[ -158us] +/- 45ms 2024-09-16T01:00:17.140 INFO:teuthology.orchestra.run.smithi169.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 15 -643us[ -624us] +/- 53ms 2024-09-16T01:00:17.140 INFO:teuthology.orchestra.run.smithi169.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 14 +1435us[+1454us] +/- 82ms 2024-09-16T01:00:17.141 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T01:00:17.141 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-16T01:00:17.151 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-16T01:00:17.151 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-16T01:00:17.161 DEBUG:teuthology.orchestra.run.smithi112:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-16T01:00:17.261 DEBUG:teuthology.orchestra.run.smithi169:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-16T01:00:17.335 DEBUG:teuthology.orchestra.run.smithi112:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-16T01:00:17.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T01:00:17.364 DEBUG:teuthology.orchestra.run.smithi169:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-16T01:00:17.395 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T01:00:17.396 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-16T01:00:17.409 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-16T01:00:17.418 INFO:teuthology.task.internal:Duration was 3026.699487 seconds 2024-09-16T01:00:17.419 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-16T01:00:17.437 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-16T01:00:17.437 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T01:00:17.440 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T01:00:17.476 INFO:teuthology.orchestra.run.smithi112.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T01:00:17.478 INFO:teuthology.orchestra.run.smithi169.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T01:00:17.879 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-16T01:00:17.880 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi112.front.sepia.ceph.com 2024-09-16T01:00:17.880 DEBUG:teuthology.orchestra.run.smithi112:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T01:00:17.939 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi169.front.sepia.ceph.com 2024-09-16T01:00:17.940 DEBUG:teuthology.orchestra.run.smithi169:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T01:00:17.976 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-16T01:00:17.976 DEBUG:teuthology.orchestra.run.smithi112:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T01:00:17.982 DEBUG:teuthology.orchestra.run.smithi169:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T01:00:18.042 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-16T01:00:18.043 DEBUG:teuthology.orchestra.run.smithi112:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T01:00:18.049 DEBUG:teuthology.orchestra.run.smithi169:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T01:00:18.963 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-16T01:00:18.980 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-16T01:00:18.981 DEBUG:teuthology.orchestra.run.smithi112:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T01:00:19.006 DEBUG:teuthology.orchestra.run.smithi169:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T01:00:19.034 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-16T01:00:19.045 DEBUG:teuthology.orchestra.run.smithi112:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T01:00:19.049 DEBUG:teuthology.orchestra.run.smithi169:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T01:00:19.069 INFO:teuthology.orchestra.run.smithi112.stdout:kernel.core_pattern = core 2024-09-16T01:00:19.099 INFO:teuthology.orchestra.run.smithi169.stdout:kernel.core_pattern = core 2024-09-16T01:00:19.127 DEBUG:teuthology.orchestra.run.smithi112:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T01:00:19.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T01:00:19.150 DEBUG:teuthology.orchestra.run.smithi169:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T01:00:19.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T01:00:19.183 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-16T01:00:19.194 INFO:teuthology.task.internal:Transferring archived files... 2024-09-16T01:00:19.195 DEBUG:teuthology.misc:Transferring archived files from smithi112:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906384/remote/smithi112 2024-09-16T01:00:19.195 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T01:00:19.249 DEBUG:teuthology.misc:Transferring archived files from smithi169:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906384/remote/smithi169 2024-09-16T01:00:19.249 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T01:00:19.287 INFO:teuthology.task.internal:Removing archive directory... 2024-09-16T01:00:19.287 DEBUG:teuthology.orchestra.run.smithi112:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T01:00:19.289 DEBUG:teuthology.orchestra.run.smithi169:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T01:00:19.337 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-16T01:00:19.351 INFO:teuthology.task.internal:Not uploading archives. 2024-09-16T01:00:19.351 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-16T01:00:19.361 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-16T01:00:19.361 DEBUG:teuthology.orchestra.run.smithi112:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T01:00:19.363 DEBUG:teuthology.orchestra.run.smithi169:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T01:00:19.376 INFO:teuthology.orchestra.run.smithi112.stdout: 265324 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Sep 16 01:00 /home/ubuntu/cephtest 2024-09-16T01:00:19.376 INFO:teuthology.orchestra.run.smithi112.stdout: 655688 4 d--------- 2 ubuntu ubuntu 4096 Sep 16 00:23 /home/ubuntu/cephtest/mnt.0 2024-09-16T01:00:19.377 INFO:teuthology.orchestra.run.smithi112.stderr:find: ‘/home/ubuntu/cephtest/mnt.0’: Permission denied 2024-09-16T01:00:19.377 INFO:teuthology.orchestra.run.smithi112.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2024-09-16T01:00:19.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T01:00:19.380 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa/tasks/ceph_fuse.py", line 248, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (50) after waiting for 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 153, in __exit__ self.gen.throw(typ, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi112 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2024-09-16T01:00:19.380 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-16T01:00:19.392 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-16T01:00:19.403 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: MaxWhileTries: reached maximum tries (50) after waiting for 300 seconds 2024-09-16T01:00:19.405 INFO:teuthology.run:Summary data: description: fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} duration: 3026.699486732483 failure_reason: reached maximum tries (50) after waiting for 300 seconds flavor: default owner: scheduled_teuthology@teuthology status: fail success: false 2024-09-16T01:00:19.406 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-16T01:00:19.496 INFO:teuthology.run:FAIL